Step-by-Step Teaching

Step 1: The Core Problem — Why Univariate Forecasting Falls Short

What is Univariate Forecasting?

Univariate forecasting means predicting a future value using only that variable's own past history.

Past Sales → [Model] → Future Sales Prediction

The Fundamental Limitation

Real-world outcomes are rarely driven by a single variable. Consider:

Key Insight: A univariate model is like predicting tomorrow's weather using only yesterday's temperature — ignoring wind, humidity, and pressure entirely.

---

Step 2: Understanding Multivariate Forecasting

What Changes?

Multivariate forecasting incorporates multiple related signals simultaneously.

Past Sales ──────────────────────┐
Past Promotions ─────────────────┤→ [Model] → Future Sales Prediction
Past Foot Traffic ───────────────┤
Future Planned Promotions ───────┘

Three Types of Variables (Critical Distinction)

┌─────────────────────────────────────────────────────────────┐
│                    VARIABLE TYPES                           │
├──────────────────┬──────────────────┬───────────────────────┤
│  TARGET SERIES   │  PAST COVARIATES │  PAST-FUTURE          │
│                  │                  │  COVARIATES           │
├──────────────────┼──────────────────┼───────────────────────┤
│ What you want    │ Variables known  │ Variables known BOTH  │
│ to predict       │ only in the past │ historically AND in   │
│                  │                  │ the future            │
├──────────────────┼──────────────────┼───────────────────────┤
│ Ice cream sales  │ Foot traffic     │ Planned promotions    │
│ Energy demand    │ Past weather     │ Holidays              │
│ Product demand   │ Related sales    │ Scheduled events      │
└──────────────────┴──────────────────┴───────────────────────┘

Why does this distinction matter? The model must treat these differently — you cannot look at future foot traffic (unknown), but you can look at future holidays (scheduled in advance).

---

Step 3: How TimesFM-3 Processes Data — The Architecture

3.1 Patching: Grouping Time Steps Efficiently

Instead of processing one time step at a time, TimesFM-3 groups data into patches of 32 time steps.

Raw Time Series:
[d1, d2, d3, ... d32 | d33, d34, ... d64 | d65, ...]
         ↓                    ↓                 ↓
      Patch 1             Patch 2           Patch 3

Why patches?

• Reduces sequence length dramatically
• Captures local temporal patterns within each patch
• Improves computational efficiency

3.2 The "Lookahead" Strategy for Future Covariates

For past-future covariates (like planned promotions), each token is built differently:

Standard Token (Target/Past Covariate):
[Current Patch Only]

Lookahead Token (Past-Future Covariate):
[Current Patch + Future Patches]
         ↑
   "Peeking ahead" at known future signals

This is architecturally clever — the model legitimately uses future information only when that information is actually available in the real world.

3.3 The 2D Transformer Grid — Two Attention Mechanisms

TimesFM-3 uses a 2D attention grid that alternates between two types of attention:

         Series A    Series B    Series C
            │           │           │
Time 1  ────●───────────●───────────●──── ← Cross-Series Attention
            │           │           │         (Who influences whom?)
Time 2  ────●───────────●───────────●────
            │           │           │
Time 3  ────●───────────●───────────●────
            ↕           ↕           ↕
       Temporal Attention (What happened before?)

These two mechanisms alternate across multiple layers, progressively building a richer understanding of both time and relationships.

---

Step 4: The Key Innovation — Contiguous Patch Masking

The Old Problem (Previous Models)

Previous TimesFM versions generated forecasts one patch at a time:

Step 1: Predict Patch 1 → use it as input
Step 2: Predict Patch 2 → use Patch 1 as input
Step 3: Predict Patch 3 → use Patches 1 & 2 as input

Problems this caused:

• ❌ Slow (iterative loop)
• ❌ Error accumulation (mistakes compound)
• ❌ High computational cost

The New Solution — Single Forward Pass

TimesFM-3 appends masked placeholder tokens for the entire future horizon upfront:

CONTEXT WINDOW                    │  FORECAST HORIZON
─────────────────────────────────────────────────────
Target:    [P1][P2][P3][P4][P5]  │  [?][?][?][?][?]  ← Masked
Promotion: [P1][P2][P3][P4][P5]  │  [K][K][K][K][K]  ← Known (visible)
─────────────────────────────────────────────────────
                                  ↓
                    All masked patches filled
                    SIMULTANEOUSLY in one pass

Analogy: Instead of writing an essay one sentence at a time (where each sentence depends on the previous), the model sees the entire outline and fills in all sections at once.

Probabilistic Output

Rather than a single prediction, TimesFM-3 outputs 9 quantiles (10th through 90th percentile):

90th percentile ─────────────────────── (optimistic scenario)
75th percentile ─────────────────
50th percentile ──────────────────────── (median forecast)
25th percentile ─────────────────
10th percentile ─────────────────────── (pessimistic scenario)

This gives decision-makers a full picture of uncertainty, not just a single number.

---

Step 5: Concrete Example — Ice Cream Sales Forecasting

Let's walk through the ice cream promotion scenario end-to-end:

Setup

Goal: Forecast next month's daily ice cream sales
Target: Daily sales figures
Past Covariate: Historical foot traffic
Past-Future Covariate: Promotion schedule (known in advance)

What Each Model Does

Univariate Model (Red Line):

Sees: [Past Sales History]
Predicts: Continuation of weekly pattern
Result: Flat weekly cycle — no promotion awareness

TimesFM-3 Multivariate (Blue Line):

Sees: [Past Sales] + [Past Promotions] + [Future Promotions]
Learns: "When promotions occurred historically, sales rose ~20%"
Applies: That learned relationship to future promotion days
Result: Forecast spikes on planned promotion days

Visual Outcome

Sales
  │                    ↑ Promo    ↑ Promo    ↑ Promo
  │              ┌─────┐    ┌─────┐    ┌─────┐
  │  Blue ───────┘     └────┘     └────┘     └────
  │  Red  ─────────────────────────────────────────
  │
  └──────────────────────────────────────────────→ Time

Business Impact: The multivariate forecast captures ~20% sales lift on promotion days, leading to more accurate revenue projections and better inventory planning.

---

Step 6: Performance and Zero-Shot Capability

What "Zero-Shot" Means

TimesFM-3 requires no task-specific fine-tuning. It was pre-trained on:

• 330 million parameters
• 1 trillion+ time points from real-world and synthetic data

It can be applied directly to new domains without retraining.

Benchmark Results Summary

Performance Ranking (Lower = Better)

                    Univariate Mode    Multivariate Mode
TimesFM-3               ★ #1               ★★ #1 (better)
Chronos-2               #2                 #2
Toto 2.0                #3                 #3
TimesFM-2.5             #4                 N/A (no multivariate)

Key finding: Even in univariate mode (ignoring all covariates), TimesFM-3 already outperforms competitors. Multivariate mode provides an additional performance leap.

---

Summary: Concept Map

                    TIMESFM-3
                        │
        ┌───────────────┼───────────────┐
        ↓               ↓               ↓
   WHAT IT IS      HOW IT WORKS    WHY IT'S BETTER
        │               │               │
  Zero-shot        Patching (32)   Single forward
  foundation       + 2D Attention  pass (no error
  model            (Temporal +     accumulation)
  330M params      Cross-Series)       │
        │               │          Probabilistic
  Pre-trained      Lookahead for   output (9
  on 1T+ points    future covars   quantiles)
        │               │               │
        └───────────────┴───────────────┘
                        │
              RESULT: State-of-the-art
              multivariate forecasting
              across all major benchmarks

---

Quick Self-Check Questions

1. What is the difference between a past covariate and a past-future covariate? (Hint: Think about what you know about the future)

2. Why does Contiguous Patch Masking improve over the previous iterative approach?

3. In the ice cream example, why can the model use the promotion schedule as a future input but NOT future foot traffic?

4. What does it mean for a model to be "zero-shot"?

5. Why does TimesFM-3 predict 9 quantiles instead of a single value?