Shopify builds a generative recommender to enhance product recommendations at scale.

Concept 1: The Buyer Journey as a Sequence

What is it?

A buyer's journey is not a single action. It is a series of events in order:

Search → View Product → Add to Cart → Favorite → Purchase

Why does order matter?

Think of it like reading a sentence:

The same events in different orders carry different meaning.

What extra information lives in the sequence?

• Order → What came before what
• Timing → How long between events
• Gaps → What the buyer didn't do (skipped products)

Key Takeaway

A good recommendation system must read the full sequence, not just a summary like "this user likes shoes."

---

Concept 2: Framing Recommendations as Next-Step Prediction

The Core Idea

Instead of asking:

"What does this user generally like?"

Ask:

"Given everything that happened so far, what comes next?"

Why is this powerful?

This is called next-token prediction — the same idea behind how ChatGPT works, but applied to products instead of words.

[Search: boots] → [View: brown boots] → [View: black boots] → PREDICT: ???

The model learns to predict the next product in the sequence.

What is an Autoregressive Model?

An autoregressive model predicts the next item using only past items (not future ones). It uses a causal mask to enforce this:

Event 1 ✓ can see: nothing before it
Event 2 ✓ can see: Event 1
Event 3 ✓ can see: Events 1, 2
Event 4 ✓ can see: Events 1, 2, 3

This mirrors how real shopping actually unfolds — one step at a time.

Key Takeaway

Recommendation becomes a prediction problem over sequences, which is exactly where generative AI models excel.

---

Concept 3: The HSTU Architecture

What is HSTU?

HSTU stands for Hierarchical Sequential Transduction Units — a generative recommender architecture built specifically for sequential recommendation.

Why use it?

It has a critical property: favorable scaling behavior

More model capacity → Lower validation loss → Better recommendations

This is like saying: the bigger the model, the smarter it gets — and that relationship holds consistently.

The Three Dimensions Shopify Scaled Along

Key Takeaway

HSTU gave Shopify a foundation that improves predictably as you invest more resources into it.

---

Concept 4: Time as a First-Class Signal

The Problem

Imagine a buyer who searched for t-shirts in June and bought one.

The same history should produce different recommendations depending on when you're asking.

How Shopify Solved This

They encoded time directly into the model's attention mechanism using two techniques:

Technique 1: RoPE-Inspired Rotary Encoding

RoPE = Rotary Position Encoding

Think of it like giving each event a clock stamp that the model can read:

[View boots: Oct 1] [Add to cart: Oct 1] [Purchase: Oct 2] → Current date: Dec 15

The model understands absolute time — it knows December is different from October.

Technique 2: Relative Attention Bias

This lets the model understand time gaps between events:

Event A ──── 2 hours ──── Event B ──── 3 weeks ──── Event C

A 2-hour gap means something different than a 3-week gap.

The Combined Effect

Absolute time (RoPE) + Relative gaps (attention bias) = Full temporal understanding

The model can also take the current session's timestamp at inference time, so it adapts to seasonality automatically — no manual rules needed.

Key Takeaway

Time is not just metadata. It is meaning. Encoding it properly transforms a generic sequence predictor into a commerce-aware model.

---

Concept 5: Negative Sampling

Why Do We Need Negatives?

Training a model to predict the next product from millions of options is computationally impossible if you consider every product at every step.

Sampled Softmax is the solution: instead of comparing against all products, compare against a sample of products the buyer did not choose (called negatives).

True next product: Black boots ✓
Negatives (random sample): Red dress, Garden hose, Laptop stand...

The model learns: "Black boots fits here, these others don't."

The Quality Problem

Easy negatives = like asking someone to distinguish a cat from a car Hard negatives = like asking someone to distinguish two very similar cat breeds

Hard negatives make the model much smarter.

Strategy 1: Shared Negatives

Instead of picking separate negatives for each training example, share negatives across the entire batch:

Batch of 100 examples → Share one pool of negatives
Result: Effectively larger negative pool, same memory cost

Strategy 2: Positive-Aware Hard Negatives

A dangerous mistake in negative sampling:

User bought: Black boots
Sampled "negative": Very similar black boots the user never saw

This is a false negative — the model is penalized for recommending something the user would probably like, just because they never saw it.

Positive-aware sampling avoids this by being careful about what counts as a true negative.

Key Takeaway

Negative sampling is not a minor training detail. It is a core scaling lever — better negatives directly improve model quality, but require careful systems engineering to manage memory.

---

Concept 6: Incremental Recall in an Ensemble System

How Production Recommenders Actually Work

A real recommendation system is not one model. It is a pipeline:

Retrieval Models (multiple) → Ranker → Aggregation → Final Recommendations
                ↑
    Your new model lives here

The Problem: Redundancy

A new model might score very well in isolation but add no real value to the system if it just finds the same products other models already find.

Model A finds: Products 1, 2, 3, 4, 5
Model B finds: Products 1, 2, 3, 4, 5  ← No new value!
Model C finds: Products 1, 2, 6, 7, 8  ← Adds value!

Incremental recall = finding true positives that other models miss.

The Boosting-Inspired Approach

Shopify used a technique inspired by boosting (a classic ML ensemble method):

1. Identify where the existing ensemble is weak (misses good products)
2. Increase training pressure on those areas
3. Treat other models' predictions as hard negatives

Other models predict: Product X
Training signal: "Don't just repeat Product X — find what they missed"

Key Takeaway

The goal is not to build the best standalone model. The goal is to build a model that makes the entire system stronger.

---

Concept 7: Training Efficiency and Speed

Why Speed Matters Beyond Just Saving Time

Faster training = more experiments = better final model

Slow training (1 week per run):
  → Test 4 ideas per month
  → Find 1 good idea

Fast training (1 day per run):
  → Test 28 ideas per month
  → Find 7 good ideas

Iteration velocity is a competitive advantage.

What Shopify Did

They wrote optimized CUDA kernels — low-level GPU code that makes specific operations much faster.

Result: 7.3x speedup over baseline implementation.

What This Unlocked

With 7.3x faster training, they could explore:

Key Takeaway

Systems efficiency is part of the modeling problem, not separate from it. A slow model that can't be iterated on is a dead end.

---

Concept 8: Measuring What Actually Matters — Online vs. Offline Metrics

The Gap Between Lab and Reality

A model can look great in offline testing but fail in production. Shopify tracked both:

Offline Metric: Recall@K

"Of the products the user actually engaged with, how many did the model retrieve in its top K recommendations?"

This is measured in the lab, before deployment.

Online Metrics (Real Users, A/B Test)

Why "High-Quality CTR" Matters

Not all clicks are equal:

Low-quality click: User clicks, immediately leaves
High-quality click: User clicks → favorites / adds to cart / purchases

Shopify specifically tracked clicks that led to actions closer to buying.

Key Takeaway

Offline metrics are a proxy. Online metrics are the truth. The goal is to close the gap between them.

---

Concept 9: Semantic IDs — The Future Direction

The Current Limitation

Right now, each product has a unique product ID (like a number). The model learns an embedding (a vector of numbers) for each product ID.

Product #4,521,847 → [0.2, 0.8, 0.1, 0.5, ...]  (embedding vector)

With millions of products, this creates a massive embedding table — expensive to store and serve.

The Semantic ID Idea

Instead of one ID per product, represent each product as a short sequence of tokens from a small shared vocabulary:

Traditional:  Product #4,521,847 → one unique ID
Semantic ID:  Product #4,521,847 → [token_42, token_7, token_156]

Think of it like how words are made from letters:

Vocabulary of 26 letters → Can express unlimited words
Small token vocabulary → Can express millions of products

Why This Is Powerful

The Vision

Text query: "I need a gift for my mom who loves gardening"
     ↓
Model works in unified token space
     ↓
Retrieves semantically relevant products

Key Takeaway

Semantic IDs are a path toward unifying product understanding and language understanding in a single model.

---

Summary: All Concepts Together

┌─────────────────────────────────────────────────────────┐
│           THE GENERATIVE RECOMMENDER SYSTEM             │
├─────────────────────────────────────────────────────────┤
│                                                         │
│  INPUT: Buyer Journey Sequence                          │
│  [Search] → [View] → [Cart] → [Purchase] → ???         │
│                                                         │
│  ARCHITECTURE: HSTU (scales with capacity)             │
│                                                         │
│  KEY INNOVATIONS:                                       │
│  ┌─────────────────────────────────────────────────┐   │
│  │ 1. Time Encoding    → Seasonality awareness     │   │
│  │ 2. Negative Sampling → Sharp product boundaries │   │
│  │ 3. Incremental Recall → Ensemble value          │   │
│  │ 4. CUDA Optimization → Fast iteration           │   │
│  └─────────────────────────────────────────────────┘   │
│                                                         │
│  OUTPUT: Next product predictions                       │
│                                                         │
│  RESULTS: +0.94% orders, +5% quality CTR               │
│                                                         │
│  FUTURE: Semantic IDs → unified text + product space   │
└─────────────────────────────────────────────────────────┘

---

One-Line Principle Behind Everything

Commerce is context — and every design decision in this system is about extracting maximum meaning from that context.