After studying this material, you should be able to:

1. Explain the limitations of batch-based ML pipelines in real-time personalization systems
2. Describe how event-driven architectures solve data freshness problems
3. Understand the two-stage inference pattern (offline encoding + online retrieval)
4. Explain Push Mode and NRT Model Transform as platform-level solutions
5. Evaluate the business and technical impact of moving from batch to near-real-time ML systems

---

Step-by-Step Study Material

Step 1: Understanding the Problem — Why Batch Processing Falls Short

What is a Batch Job?

A batch job is a scheduled process that runs at fixed intervals (e.g., nightly) to process accumulated data.

Timeline Example:
Monday 2:00 PM  → Guest browses 10 listings
Monday 8:00 PM  → Guest searches again
                  ❌ System still uses YESTERDAY'S data
Tuesday 2:00 AM → Nightly batch job runs
Tuesday 2:00 PM → Updated recommendations finally available
                  ⏱️ Total lag: ~24-48 hours

Why Does Staleness Matter?

Key Insight: A guest actively planning a trip changes their preferences rapidly. Yesterday's browsing data may no longer reflect today's intent.

---

Step 2: The Original Architecture — Two-Stage Inference

Understanding the Split Design

Airbnb's original system separated heavy computation from real-time serving:

STAGE 1: Offline (Nightly Batch)
┌─────────────────────────────────────┐
│  Guest History Data                 │
│  (bookings + reviews + browsing)    │
│           ↓                         │
│  Transformer Sequence Encoder       │
│  (computationally expensive)        │
│           ↓                         │
│  Guest Embedding stored in          │
│  Low-Latency Store                  │
└─────────────────────────────────────┘

STAGE 2: Online (Real-Time, milliseconds)
┌─────────────────────────────────────┐
│  Guest runs a search                │
│           ↓                         │
│  Fetch embedding from store         │
│  + Combine with live search query   │
│           ↓                         │
│  Score and rank listings            │
└─────────────────────────────────────┘

Why This Design Existed

• Transformers are expensive — running them on every search request would be too slow
• Embeddings are reusable — a guest's history doesn't change every second
• Low-latency serving — reading from a store is fast; running a Transformer is not

Key Concept — Embedding: A numerical vector representation of a guest's preferences and history, compressed into a format the ranking model can use quickly.

---

Step 3: Identifying the Three Challenges of Going Real-Time

Before seeing the solution, understand what makes real-time updates difficult:

Challenge 1: TRIGGERING
"How do we know WHEN to update an embedding?"
→ Batch: easy — run every night
→ Real-time: must detect each individual activity event

Challenge 2: COMPUTATION ON THE CRITICAL PATH
"Transformers are slow — we can't run them during a search request"
→ Must run inference BEFORE the search, not during it

Challenge 3: INTEGRATION
"How do we update without breaking the existing serving system?"
→ The ranking model already reads from the low-latency store
→ Solution must write to the same place

---

Step 4: The Solution — Two New Platform Capabilities

Capability 1: Push Mode

Old approach (Pull/Poll model):

Downstream system: "Any new data?" → checks every N minutes
                   "Any new data?" → checks again
                   "Any new data?" → checks again
                   "Yes, here it is" → finally acts

New approach (Push/Event-driven model):

Streaming job: detects new guest activity
             → immediately publishes notification
             → downstream consumer acts RIGHT NOW

Analogy: The difference between checking your email every hour (pull) versus receiving a push notification the moment a message arrives (push).

Push Mode Flow:
Guest views listing
      ↓
Streaming job captures event
      ↓
Feature value updated
      ↓
Push notification sent immediately ← NEW CAPABILITY
      ↓
Downstream process triggered

Capability 2: Near-Real-Time (NRT) Model Transform

Old approach:

Option A: Embed model logic in application code (inflexible, hard to update)
Option B: Wait for batch job to run inference (slow, stale)

New approach:

NRT Model Transform:
Streaming pipeline → calls deployed Transformer model → writes result as feature value
                     (same model as batch pipeline)      (immediately available)

Key Insight: The same Transformer model used in the nightly batch job is now called from within the streaming pipeline — no need to maintain two separate models.

---

Step 5: The Combined Pipeline — How It All Works Together

COMPLETE NEAR-REAL-TIME PIPELINE:

1. Guest Event Occurs
   (page view, new search, listing click)
          ↓
2. Streaming Job Captures Event
   + merges with existing long-term & short-term sequence data
          ↓
3. PUSH MODE signals: "New sequence data is ready"
          ↓
4. NRT MODEL TRANSFORM runs Transformer sequence encoder
   → produces fresh embedding
          ↓
5. Embedding written to LOW-LATENCY STORE
   (same store the batch job always used)
          ↓
6. Next search request reads fresh embedding
   → ranking model uses up-to-date guest representation

Total time: 10-30 seconds (vs. up to 2 days before)

What Stayed the Same vs. What Changed

---

Step 6: Measuring Impact — Freshness and Quality

Freshness Impact

Before: ~48 hours lag
After:  10-30 seconds lag

Improvement: ~5,760x faster updates

Quality Impact

NDCG (Normalized Discounted Cumulative Gain)

• Measures how well a ranking system orders results
• Higher = better results ranked higher
• Even 0.1% improvement is considered significant in mature systems

Result: +1.67% NDCG improvement
→ This is a very large gain for a decade-old, highly optimized system

Business Impact

~0.33% increase in uncancelled bookings

Key insight confirmed: Freshness itself improves ranking quality
→ The SAME guest representation becomes MORE valuable simply by being current

---

Step 7: The Broader Principle — Generalizing the Pattern

The Reusable Pattern

1. REACT    → detect an activity event (Push Mode)
2. MERGE    → combine with existing state
3. INFER    → run model immediately (NRT Model Transform)
4. SERVE    → write result to serving store

This pattern is not specific to Airbnb's use case — it applies to any system where:

• User behavior changes rapidly
• Model inference is too expensive for the critical serving path
• Freshness directly impacts recommendation quality

Platform vs. Pipeline Thinking

Key Engineering Principle: Solve problems at the platform level when the solution generalizes — don't build one-off integrations.

---

Summary: Concept Map

PROBLEM
└── Batch embeddings are stale (up to 2 days)
    └── Recent activity is most relevant to current intent

SOLUTION
├── Push Mode
│   └── Event-driven triggering instead of scheduled polling
│
└── NRT Model Transform
    └── Run Transformer inference inside streaming pipeline

RESULT
├── Technical: 10-30 second freshness (vs. 2 days)
├── Quality: +1.67% NDCG improvement
├── Business: ~0.33% more uncancelled bookings
└── Strategic: Reusable platform foundation for future models

---

Self-Check Questions

1. Why couldn't Airbnb simply run the Transformer model on every search request?
2. What is the difference between Push Mode and a polling approach?
3. Why is it significant that the new pipeline writes to the same low-latency store?
4. Why does a +1.67% NDCG improvement matter more in a mature system than in a new one?
5. What does it mean to solve a problem at the platform level rather than the pipeline level?