After studying this material, students should be able to:

1. Explain the core context problem in agentic LLM systems
2. Describe the four categories of selective persistent memory
3. Distinguish between naive full-history persistence vs. selective memory vs. no memory
4. Understand how shared memory enables collaborative reuse
5. Interpret performance metrics and why selective memory outperforms alternatives

---

Step-by-Step Teaching

Step 1: The Fundamental Problem

What is an Agentic LLM System?

Think of an agentic LLM as an AI assistant that:

• Takes multiple turns to complete complex tasks
• Uses tools (databases, APIs, code execution)
• Generates artifacts (dashboards, reports, code)

User → Agent → Tool Use → Output
         ↑
    [Uses context]

The "Starting From Zero" Problem

Analogy: Imagine hiring an expert consultant who, every Monday morning, forgets everything from previous weeks:

• Your company's data structure
• Your preferred reporting format
• Which tools worked and which failed
• Your domain-specific rules

Every session, users must re-explain everything from scratch

This is computationally wasteful AND degrades output quality.

---

Step 2: The Naive "Fix" and Why It Fails

Obvious Solution: Save Everything

One might think: "Just save the entire conversation history!"

Why this backfires:

Empirical Evidence from the Paper

Task Completion Rate:
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
No Memory:          ████████████████░░░░  79%
Full History:       ██████████████░░░░░░  71%  ← WORSE than no memory!
Selective Memory:   ███████████████████░  96%
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━

Key Insight: Full history persistence actively harms performance because stale reasoning traces mislead the agent

---

Step 3: The Solution — Shared Selective Persistent Memory

Core Concept

Instead of saving everything or nothing, selectively retain only what generalizes across sessions.

Session 1 ──┐
Session 2 ──┼──► [Filter] ──► SELECTIVE MEMORY ──► Session N
Session 3 ──┘
              ↑
         Keep reusable
         Discard specific

The Four Categories of Reusable Context

Category 1: Task Specifications

• What it is: The goals, requirements, and constraints of recurring tasks
• Example: "Generate monthly sales reports comparing regional performance"
• Why save it: Task goals rarely change between sessions

Category 2: Data Schemas

• What it is: Structure of databases, CSV formats, API response shapes
• Example: {customer_id: int, revenue: float, region: string}
• Why save it: Data structure is stable; re-discovering it wastes tokens

Category 3: Tool Configurations

• What it is: How tools are connected and configured (SQL connections, API endpoints, MCP servers)
• Example: "Sales database is at endpoint X, requires auth token Y"
• Why save it: Configuration doesn't change session-to-session

Category 4: Output Constraints

• What it is: Format requirements, style guides, business rules
• Example: "Charts must use company color palette; revenue in thousands"
• Why save it: Standards are persistent organizational knowledge

What Gets DISCARDED

✅ KEEP (Reusable)          ❌ DISCARD (Session-Specific)
─────────────────────       ──────────────────────────────
Task specifications    →    "Let me think about this..."
Data schemas           →    Intermediate reasoning steps
Tool configurations    →    Error recovery attempts
Output constraints     →    One-time clarifications

---

Step 4: The "Shared" Dimension — Collaborative Memory

Why Sharing Matters

Without shared memory:

User A learns: "SQL schema looks like X, tool Y works best"
User B starts: [Learns everything from scratch again]
User C starts: [Learns everything from scratch again]

With shared memory:

User A builds memory workspace
         ↓
    [Workspace shared with role-based access]
         ↓
User B inherits accumulated knowledge instantly
User C inherits accumulated knowledge instantly

Role-Based Access Control (RBAC)

Analogy: Like a shared Google Drive folder for institutional knowledge, but for AI context

---

Step 5: Supporting Mechanisms

Mechanism 1: Git-Backed Versioning

The system stores generated artifacts (dashboards, reports) with git versioning:

main branch:    [Dashboard v1] → [Dashboard v2] → [Dashboard v3]
                                        ↓
draft branch:              [Experimental edit] ← Safe to explore

Benefits:

• Explore modifications risk-free
• Restore any prior state without re-invoking the model
• Draft isolation prevents breaking production artifacts

Mechanism 2: Zero-Token Data Refresh

The Problem it Solves:

Traditional approach:
New data available → Re-invoke LLM → Re-generate artifact → [Costs tokens + time]

Zero-token refresh:
New data available → Re-run existing program with new data → [Zero LLM cost]

How it works:

• Generated programs are decoupled from runtime data
• The program logic is saved; only data inputs change
• No LLM re-invocation needed for data updates

Performance Impact:

Task time reduction: 14× faster
Token cost reduction: 97× cheaper (summary-driven generation)
Zero-token refresh success: 12/12 trials on public datasets

---

Step 6: System Architecture — Putting It Together

┌─────────────────────────────────────────────────────┐
│                  USER INTERFACE                      │
└──────────────────────┬──────────────────────────────┘
                       │
┌──────────────────────▼──────────────────────────────┐
│              AGENTIC LLM LAYER                       │
│  • Multi-turn tool use                               │
│  • Code generation                                   │
└──────┬───────────────┬───────────────────────────────┘
       │               │
┌──────▼──────┐  ┌─────▼──────────────────────────────┐
│   TOOLS     │  │    SELECTIVE MEMORY LAYER            │
│ • CSV       │  │  ┌──────────────────────────────┐   │
│ • SQL       │  │  │ Task Specs | Schemas          │   │
│ • REST API  │  │  │ Tool Config | Output Rules    │   │
│ • MCP Server│  │  └──────────────────────────────┘   │
└─────────────┘  │  [Shared across users via RBAC]      │
                 └─────────────────────────────────────┘
                       │
┌──────────────────────▼──────────────────────────────┐
│              GIT-VERSIONED ARTIFACTS                 │
│  • Dashboards  • Reports  • Data Documents           │
│  [Draft isolation + Zero-token data refresh]         │
└─────────────────────────────────────────────────────┘

---

Step 7: Validation and Results Summary

Enterprise Deployment Results

Generalizability

• Tested on 4 public datasets
• Zero-token refresh: 12/12 trials successful
• Confirms the approach isn't domain-specific

---

Concept Summary

CORE INSIGHT:
─────────────────────────────────────────────────────
Memory Quality > Memory Quantity

The right 4 categories of context (selectively retained)
outperform both no memory AND complete history retention.

Sharing this selective memory across users multiplies
its value without multiplying specification effort.
─────────────────────────────────────────────────────

Key Takeaways

1. Selective > Complete: Filtering memory is better than saving everything
2. Shared memory = organizational knowledge asset: Context built once, reused by many
3. Decouple programs from data: Enables zero-cost artifact refresh
4. Stale reasoning is toxic: Old thought processes actively mislead agents
5. Four categories are sufficient: Task specs, schemas, tool configs, output constraints capture what generalizes