Step-by-Step Teaching
Step 1: Understanding the Problem (The "Why")
Why does Netflix need a Foundation Model?
BEFORE Foundation Model:
┌─────────────┐ ┌─────────────┐ ┌─────────────┐
│ Model A │ │ Model B │ │ Model C │
│ (Trending) │ │ (Similar) │ │ (Continue) │
│ │ │ │ │ │
│ Learns │ │ Learns │ │ Learns │
│ preferences │ │ preferences │ │ preferences │
│ separately │ │ separately │ │ separately │
└─────────────┘ └─────────────┘ └─────────────┘
❌ Redundant work, high maintenance cost
AFTER Foundation Model:
┌──────────────────────┐
│ Foundation Model │
│ (Central Learning) │
│ User histories + │
│ Content data │
└──────────┬───────────┘
│ Distributes learnings
┌──────────┼───────────┐
▼ ▼ ▼
Model A Model B Model C
✅ Efficient, consistent, powerful
Key Concept:
The Foundation Model acts like a shared brain — trained once on massive data, then its knowledge is distributed to specialized models
Step 2: The Three Integration Approaches Overview
Foundation Model Integration Options
├── 1. Embeddings → Pre-computed representations stored & reused
├── 2. Subgraph → Foundation Model runs inside application model
└── 3. Fine-tuning → Customize Foundation Model for specific tasks
Think of it like using a powerful encyclopedia:
- Embeddings = Reading the summary pages (fast, but possibly outdated)
- Subgraph = Consulting the full encyclopedia in real-time (thorough, but slower)
- Fine-tuning = Creating a specialized edition of the encyclopedia (tailored, but more to maintain)
Step 3: Deep Dive — Approach 1: Embeddings
What is an Embedding?
An embedding is a numerical representation (vector) that captures the meaning/preferences of a user or item
User Interaction History → Foundation Model → User Profile Embedding
→ Item Embeddings
Example:
User watches: [Action Movie, Thriller, Documentary]
↓
Foundation Model processes sequence
↓
User Embedding: [0.82, 0.15, 0.67, ...] (dense vector)
How Netflix Refreshes Embeddings
Monthly Cycle:
┌─────────────────────────────────────────────┐
│ Pre-train Foundation Model from scratch │
│ (Full retraining on all data) │
└─────────────────────┬───────────────────────┘
│
Daily Cycle: ▼
┌─────────────────────────────────────────────┐
│ Fine-tune on latest data │
│ + Add newly launching titles │
└─────────────────────┬───────────────────────┘
│
▼
┌─────────────────────────────────────────────┐
│ Run Batch Inference │
│ Generate fresh embeddings │
└─────────────────────┬───────────────────────┘
│
▼
┌─────────────────────────────────────────────┐
│ Publish to Embedding Store │
│ (Versioned + Timestamped) │
└─────────────────────────────────────────────┘
Critical Problem: Embedding Drift
Month 1 Training: Month 2 Training:
"Action" → [0.9, 0.1] "Action" → [0.2, 0.8]
↑
Random initialization causes
completely different spaces!
Solution: Embedding Stabilization Technique
→ Maps all embeddings into the SAME space
→ Downstream models can reliably consume them
Embedding Store — The Infrastructure Backbone
Embedding Store Functions:
├── Versioning (tracks which model generated embeddings)
├── Timestamping (knows when embeddings were created)
├── Offline access (for model training)
└── Online access (for real-time serving)
Pros and Cons Summary
| ✅ Pros | ❌ Cons |
|---|---|
| Low barrier to entry | Staleness (daily refresh gap) |
| Low compute cost | May miss real-time signals |
| Easy to integrate | Not full Foundation Model power |
| Good starting point | Less adaptive to session behavior |
When to Use:
Use embeddings as your default starting point when freshness requirements are flexible and you want low cost, high leverage
Step 4: Deep Dive — Approach 2: Subgraph
What is the Subgraph Approach?
Traditional Application Model:
┌────────────────────────────────┐
│ Application Model │
│ [Features] → [Layers] → Output│
└────────────────────────────────┘
Subgraph Integration:
┌────────────────────────────────────────────┐
│ Application Model │
│ │
│ ┌──────────────────────┐ │
│ │ Foundation Model │ ← Subgraph │
│ │ (Decoder Stack) │ │
│ │ Processes raw user │ │
│ │ interaction sequence│ │
│ └──────────┬───────────┘ │
│ │ Representations │
│ ▼ │
│ [Application-specific layers] → Output │
└────────────────────────────────────────────┘
Key Optimization: Run Once, Share Everywhere
Without optimization:
Request for User X with 10 items:
Foundation Model runs 10 times ❌ (expensive!)
With optimization:
Foundation Model runs ONCE per user profile
↓
Result shared across all 10 items ✅
Pros and Cons Summary
| ✅ Pros | ❌ Cons |
|---|---|
| No staleness (real-time) | Larger model size |
| Access to all FM layers | Slower inference |
| Can fine-tune FM jointly | Complex feature generation |
| Full Foundation Model power | Higher compute cost |
When to Use:
Reserve for high-impact use cases where metric improvements justify the added complexity and cost
Step 5: Deep Dive — Approach 3: Fine-Tuning
What is Fine-Tuning?
Foundation Model (General Knowledge)
Pre-trained on: next-token prediction
Tokens = different user interactions
↓ Fine-tuning
Application-Specific Model
Trained on: specific surface data
Example: "Trending Now" row
→ Weights recent interactions on trending titles MORE
Fine-Tuning Options Available
Fine-Tuning Choices:
├── Full Parameter Fine-tuning
│ └── Update ALL weights (maximum customization)
│
├── Frozen Layers Fine-tuning
│ └── Keep some layers fixed (faster, cheaper)
│
└── Custom Output Head
└── Add new objective layer (task-specific)
Practical Example
Foundation Model trained on:
[Watch, Search, Rate, Browse, ...] → Next interaction prediction
Fine-tuned for "Trending Now":
[Recent trending watches] → Trending score prediction
↑
Different objective, different data emphasis
Pros and Cons Summary
| ✅ Pros | ❌ Cons |
|---|---|
| Optimized for specific use case | More models to maintain |
| Built-in baseline for new apps | Latency must be carefully managed |
| No months of feature engineering | SLA optimization required |
| Adapts FM to specific objectives | Organization-wide pipeline complexity |
When to Use:
Use when you need task-specific optimization and want to avoid building a new model stack from scratch
Step 6: Comparing All Three Approaches
Decision Framework:
START
│
▼
Do you need real-time adaptability?
│
├── NO → Start with EMBEDDINGS
│ (Low cost, easy integration)
│
└── YES
│
▼
Do you need full FM power with
joint training?
│
├── YES, high impact use case → SUBGRAPH
│ (Accept complexity for performance)
│
└── Need task-specific optimization → FINE-TUNING
(Custom objective, new surface)
Side-by-Side Comparison Table
| Dimension | Embeddings | Subgraph | Fine-Tuning |
|---|---|---|---|
| Freshness | Daily (stale) | Real-time | Depends on retraining |
| Compute Cost | Low | High | Medium-High |
| Integration Complexity | Low | High | Medium |
| FM Power Utilized | Partial | Full | Full (specialized) |
| Maintenance Burden | Low | Medium | High |
| Best For | Starting point | High-impact features | New surfaces/tasks |
Step 7: Future Directions
Netflix is actively improving all three approaches:
Current Limitations → Solutions in Progress
Embeddings staleness → Near-real-time embedding inference
(Updates during user session!)
Subgraph latency → Distilled (smaller) Foundation Model
(Same knowledge, faster inference)
Fine-tuning effort → Standardized fine-tuning framework
(Easier for application teams)
Summary: The Big Picture
Netflix Foundation Model Integration — Mental Model
Foundation Model
(Central Knowledge)
│
┌───────────────┼───────────────┐
▼ ▼ ▼
EMBEDDINGS SUBGRAPH FINE-TUNING
"Pre-read "Live "Specialized
summary" consultation" edition"
Fast & cheap Powerful & Task-optimized
Slightly stale Real-time Custom objective
→ Start here → Scale up → New surfaces
Core Takeaway:
There is no single best approach — the right integration depends on your latency requirements, compute budget, freshness needs, and impact potential. Netflix uses all three in production simultaneously for different use cases.