Course contentsShow
← Machine Learning and Deep Learning
Lesson 2428 of 3,53852. Time Series and ForecastingFree lesson

Chronos: Tokenization and Language Model Pretraining for Forecasting

How Chronos tokenizes time series values and uses language model pretraining objectives for forecasting.

Chronos: Tokenization and Language Model Pretraining for Forecasting

What You'll Learn

You'll understand how Chronos converts continuous time series into discrete tokens and applies language model pretraining to enable zero-shot forecasting.

The Core Idea

Chronos takes a radical approach: it treats time series like text. Just as language models learn from billions of words, Chronos learns from millions of time series by first tokenizing them.

How Tokenization Works

Traditional neural forecasters work directly with continuous values (like 23.4°C or $105.32). Chronos instead:

  1. Scales each time series to a standard range (typically [-1, 1] or [0, 1])
  2. Quantizes continuous values into discrete bins (e.g., 4096 bins)
  3. Maps each bin to a unique token ID, just like words in a vocabulary

Think of it like converting a smooth temperature curve into a bar chart with fixed height levels—each bar height becomes a "word."

Language Model Pretraining

Once tokenized, Chronos uses the same next-token prediction objective as GPT:

  • Given tokens [t₁, t₂, t₃, ...], predict t₄
  • The model learns temporal patterns across diverse datasets
  • Uses Transformer architectures (like T5 or GPT) adapted for these numeric tokens

This pretraining on massive collections of tokenized time series enables the model to capture universal temporal patterns—from financial trends to weather cycles—without seeing your specific dataset.

Why This Works

By discretizing values, Chronos can:

  • Apply proven language model architectures directly
  • Share statistical strength across different scales and domains
  • Perform zero-shot forecasting on new series after pretraining

Key Takeaway: Chronos bridges time series and language modeling by tokenizing continuous values into discrete bins and pretraining Transformers with next-token prediction, enabling foundation models to forecast on unseen series without fine-tuning.