After studying this material, you should be able to:

1. Explain why native LLM search grounding creates problems in production pipelines
2. Describe the architecture and request lifecycle of Decoupled Search Grounding (DSG)
3. Analyze the trade-offs between native search and DSG across cost, accuracy, latency, and control
4. Evaluate when to use DSG versus native search based on workload requirements
5. Understand how caching and retrieval depth tuning contribute to cost and performance optimization

---

Step-by-Step Study Material

---

Step 1: Understand the Problem Context

What is Query Intent Understanding (QIU)?

Before diving into DSG, you need to understand why it was built.

Scenario: A user searches "450 north" on DoorDash.

Key Insight: Correct intent classification directly affects what inventory gets shown to users. Wrong classification = wrong results = poor user experience.

Why Grounding Matters

Grounding means giving the LLM real-world evidence (from web search or catalogs) to support its reasoning, rather than relying solely on its training data.

Native Search = The model provider (e.g., Google, Anthropic) handles search internally within their API.

---

Step 2: Identify the Three Core Problems with Native Search

Think of native search like a black box. One request goes in, one answer comes out, and you cannot see or control what happens inside.

Problem 1: Control

Native Search:
[Your Agent] → [Model + Search bundled together] → [Answer]
                        ↑
              You cannot separate these

• Cannot swap models without also changing search
• Cannot react to vendor price changes independently
• Cannot choose your own search provider

Problem 2: Reuse

Agent A asks "What is X?" → pays for search
Agent B asks "What is X?" → pays for search again
                              ↑
                    No shared cache exists

• Every agent pays for the same search independently
• No mechanism to intercept and reuse results

Problem 3: Observability and Contract Safety

• You cannot measure per-query cost or latency
• Search-induced verbosity: enabling search causes models to break output formatting rules

The Verbosity Problem Explained

This is subtle but critical for production systems.

Instruction given to model: "Provide ONLY the final answer entity with no explanation"

Why does this happen? Native search injects results inside the generation stream, which changes how the model behaves. DSG returns evidence as a separate structured tool response, keeping generation behavior consistent.

Measured impact: On HotpotQA, Claude Sonnet 4 violated "answer only" instructions 78.1% of the time when native search was enabled.

---

Step 3: Understand the DSG Architecture

The Core Idea

Native Search:
[Agent] ←→ [Model + Search Provider]  ← tightly coupled

DSG:
[Agent] ←→ [Reasoning Model]
               ↕
         [DSG Gateway]  ← shared, controllable layer
               ↕
    [Serper | BrightData | Firecrawl | Exa | Tavily]

DSG separates reasoning from retrieval so either can change independently.

Key Architectural Properties

Why Normalization Matters

Different providers return data in different formats. DSG converts all of them into one consistent schema:

Raw Provider Output (varies):        DSG Normalized Output (always):
{result_title: "...",          →     {title: "...",
 link: "...",                         url: "...",
 description: "..."}                  snippet: "..."}

This gives the model explicit provenance (where the information came from) and gives engineers an inspectable artifact for debugging.

---

Step 4: Walk Through the Request Lifecycle

Every search query follows this exact sequence. Think of it as a decision tree that prioritizes cheap options first.

Query arrives
     ↓
1. NORMALIZE the query
   (so "best pizza NYC" and "best pizza in NYC" hit same cache)
     ↓
2. CHECK EXACT CACHE
   → HIT? Return immediately. No provider call. Done.
   → MISS? Continue...
     ↓
3. CHECK SEMANTIC CACHE
   (embedding similarity - finds near-duplicate queries)
   → HIT? Return cached result. Done.
   → MISS? Continue...
     ↓
4. WALK PROVIDER FALLBACK CHAIN
   Try Provider 1 → timeout/error? → Try Provider 2 → etc.
     ↓
5. NORMALIZE the response
   (convert to title/URL/snippet schema)
     ↓
6. WRITE BOTH CACHE LAYERS
   (with TTL tuned to how fast this answer goes stale)
     ↓
7. RETURN structured tool response
   (preserves output contract)
     ↓
8. EMIT TELEMETRY
   (provider, cache outcome, latency, cost)

Design Principle: The system is built to avoid calling external providers whenever possible. External calls are the expensive last resort.

Cache TTL Strategy

Not all information ages at the same rate:

---

Step 5: Analyze the Results

Benchmark Performance

From Figure 3 and the production data:

Public Benchmarks (HotpotQA, SimpleQA, FreshQA):

• DSG accuracy: 83–86%
• Native search accuracy: 87.7%
• DSG cost: 10–30x lower

Production QIU Workload:

Critical observation: DSG slightly exceeded native search accuracy in production while costing 98%+ less. This is the headline result.

The Caching Impact

Warm cache = 68% latency reduction + ~99% cost reduction on search

Compounding effect: Because the cache is shared across all agents and workloads, it warms faster than per-agent caches would. A query answered for one workload benefits all other workloads.

Retrieval Depth as a Tunable Parameter

This is a key advantage of decoupling. You can now measure the effect of retrieval depth:

Depth 2  → 78.7% accuracy
Depth 6  → 84.9% accuracy  (+6.2 points)  ← good default
Depth 10 → 85.5% accuracy  (+0.6 points)  ← diminishing returns

Practical decision: Use depth 6 as default. Only increase to 10 when that extra 0.6% accuracy justifies the added cost and latency.

---

Step 6: Know When to Use Each Approach

The Trade-off Table

Decision Rule

Does freshness matter more than cost/control?
    YES → Consider native search (or register it as a DSG provider)
    NO  → Use DSG with appropriate provider and depth settings

Important nuance: DSG underperformed on FreshQA (recency-sensitive questions). Native search providers have indexing advantages for very recent information. The solution is to register native search as one provider option within DSG, not to abandon DSG entirely.

---

Step 7: Understand the Limitations

Knowing what a system cannot do is as important as knowing what it can.

Limitation 1: Third-Party Vendor Dependency

• DSG relies on external search APIs (Serper, BrightData, etc.)
• Their pricing, uptime, and data quality can change
• Mitigation: The interface is durable even if specific price points change

Limitation 2: Multi-Hop Reasoning

• DSG works best for single-hop lookups (one search answers the question)
• Complex reasoning chains that require iterative searching benefit less
• Example: "Who was the president when the tallest building in the city where X was born was constructed?" requires multiple dependent searches

Limitation 3: Verbosity Mitigation, Not Elimination

• The structured tool boundary reduces format violations
• It does not guarantee zero violations
• Next step: Contract-aware validators at the gateway level

---

Summary: The Core Engineering Lesson

┌─────────────────────────────────────────────────────┐
│  REASONING  and  RETRIEVAL  evolve at different     │
│  speeds. Separating them lets you:                  │
│                                                     │
│  • Improve either independently                     │
│  • Choose quality/freshness/latency/cost balance    │
│    per workload                                     │
│  • Measure what you cannot currently see            │
│  • Reuse what you currently pay for twice           │
└─────────────────────────────────────────────────────┘

The 98% cost reduction was not achieved by sacrificing quality. It was achieved by:

1. Making retrieval an explicit, measurable interface
2. Caching aggressively across a shared layer
3. Tuning retrieval depth based on evidence
4. Routing to cost-effective providers where freshness is not critical