Chronos: Tokenization and Language Model Pretraining for Forecasting
What You'll Learn
You'll understand how Chronos converts continuous time series into discrete tokens and applies language model pretraining to enable zero-shot forecasting.
The Core Idea
Chronos takes a radical approach: it treats time series like text. Just as language models learn from billions of words, Chronos learns from millions of time series by first tokenizing them.
How Tokenization Works
Traditional neural forecasters work directly with continuous values (like 23.4°C or $105.32). Chronos instead:
- Scales each time series to a standard range (typically [-1, 1] or [0, 1])
- Quantizes continuous values into discrete bins (e.g., 4096 bins)
- Maps each bin to a unique token ID, just like words in a vocabulary
Think of it like converting a smooth temperature curve into a bar chart with fixed height levels—each bar height becomes a "word."
Language Model Pretraining
Once tokenized, Chronos uses the same next-token prediction objective as GPT:
- Given tokens
[t₁, t₂, t₃, ...], predictt₄ - The model learns temporal patterns across diverse datasets
- Uses Transformer architectures (like T5 or GPT) adapted for these numeric tokens
This pretraining on massive collections of tokenized time series enables the model to capture universal temporal patterns—from financial trends to weather cycles—without seeing your specific dataset.
Why This Works
By discretizing values, Chronos can:
- Apply proven language model architectures directly
- Share statistical strength across different scales and domains
- Perform zero-shot forecasting on new series after pretraining
Key Takeaway: Chronos bridges time series and language modeling by tokenizing continuous values into discrete bins and pretraining Transformers with next-token prediction, enabling foundation models to forecast on unseen series without fine-tuning.