How Agents Turn Experience into Reusable Skills

Learning Outcomes

After studying this material, you should be able to:

1. Explain the core problem of learning from experience in LLM agents
2. Describe what a "harness" is and how it differs from other knowledge representations
3. Understand the concept of context-to-harness skill compilation
4. Distinguish between online vs. offline learning approaches for agents
5. Evaluate why one-shot learning scenarios are challenging and how Evo-Harness addresses them

---

Step 1: The Foundation Problem — Why Do Agents Need to Learn from Experience?

The Core Challenge

Imagine you hire a new employee. They complete a task, make mistakes, and learn lessons. The question is: how do they carry those lessons forward?

LLM agents face the same challenge. A capable agent should:

Task 1 → Execute → Learn Something → 
Task 2 → Execute → Learn Something → 
Task N → Execute → (Agent is now smarter than at Task 1)

Current Approaches and Their Limitations

Prior methods extract knowledge through:

Key Insight: All these methods struggle when the agent only gets one shot at a task before moving on.

---

Step 2: Understanding the "One-Shot" Problem

What is a One-Shot Opportunity?

In realistic environments, agents often encounter a task once. There is no:

• Second attempt
• Iterative refinement
• Repeated exposure to the same problem

Real World Timeline:
[Novel Task A] → [Novel Task B] → [Novel Task C] → ...
      ↑                ↑                ↑
  Never seen       Never seen       Never seen
    again            again            again

Why This Creates Noisy Contexts

When an agent executes a one-shot task, the resulting execution context contains:

Execution Context = 
    [Useful General Lessons]     ← We WANT this
  + [Task-Specific Artifacts]    ← We DON'T want this
  + [Noise and Irrelevant Steps] ← We DON'T want this

Example:

An agent debugging a Python web scraper might learn:

• ✅ "Always check for rate limiting headers" (broadly useful)
• ❌ "The target URL was example.com/data" (task-specific artifact)
• ❌ "Retry step 3 because of a typo" (noise)

The challenge is separating signal from noise in a single execution.

---

Step 3: What is a "Harness"?

Definition

A harness is a structured wrapper or scaffolding around an agent that shapes how it behaves, reasons, and acts.

Think of it like this:

Without Harness:          With Harness:
┌─────────────┐           ┌─────────────────────────┐
│  Raw Agent  │           │  Harness                │
│  (frozen)   │           │  ┌─────────────────┐    │
│             │           │  │  Skill: Debug   │    │
│  Receives   │           │  │  Skill: Search  │    │
│  task →     │           │  │  Skill: Verify  │    │
│  responds   │           │  └─────────────────┘    │
└─────────────┘           │       ↓                 │
                          │  Raw Agent (frozen)     │
                          │       ↓                 │
                          │  Better Response        │
                          └─────────────────────────┘

Key Properties of a Harness

1. Structured — organized, not just free-form text
2. Reusable — applicable across multiple future tasks
3. Updatable — can be improved as the agent encounters more tasks
4. External to the agent — the base LLM stays frozen (unchanged)

Critical Point: The agent itself never changes. Only the harness evolves. This is important because it avoids expensive retraining.

---

Step 4: The Evo-Harness Framework

Core Architecture

┌──────────────────────────────────────────────────────┐
│                    EVO-HARNESS                       │
│                                                      │
│  Task arrives                                        │
│      ↓                                               │
│  [Current Harness] + [Frozen Agent] → Execute Task   │
│                              ↓                       │
│              Noisy Execution Context                 │
│                              ↓                       │
│         CONTEXT-TO-HARNESS SKILL COMPILATION         │
│                              ↓                       │
│              Updated/New Skill Harness               │
│                              ↓                       │
│         Ready for Next Task (improved agent)         │
└──────────────────────────────────────────────────────┘

The Two Key Dimensions of Adaptation

Evo-Harness supports skills at two levels:

---

Step 5: Context-to-Harness Skill Compilation (The Core Innovation)

What is Compilation Here?

In programming, a compiler transforms raw source code into efficient, executable machine code.

Evo-Harness does something analogous:

Compiler Analogy:
Source Code (messy, human-readable) → Compiler → Machine Code (efficient, reusable)

Evo-Harness:
Noisy Execution Context → Skill Compiler → Structured Skill Harness

The Compilation Process (Step by Step)

Step 5.1 — Context Capture

Agent executes task → Full trajectory recorded
(every action, observation, error, success)

Step 5.2 — Signal Extraction

Compiler analyzes trajectory:
- What worked? Why?
- What failed? Why?
- What patterns appeared?

Step 5.3 — Generalization Filter

Remove task-specific artifacts:
- Specific URLs, file names, IDs → REMOVED
- General strategies, patterns → KEPT

Step 5.4 — Skill Structuring

Package into harness format:
{
  skill_name: "handle_rate_limiting",
  trigger: "when making repeated API calls",
  action: "check response headers for Retry-After",
  domain: "web_interaction",
  confidence: 0.85
}

Step 5.5 — Harness Integration

New skill merged into existing harness
Conflicts resolved, redundancies removed
Harness ready for next task

---

Step 6: Online vs. Offline Learning — Why "Online" Matters

Definitions

Evo-Harness is Online Learning

Time →
T1: [Task] → Execute → Compile → Harness v1.0
T2: [Task] → Execute → Compile → Harness v1.1  ← slightly better
T3: [Task] → Execute → Compile → Harness v1.2  ← better still
...
Tn: [Task] → Execute → Compile → Harness v1.n  ← much better

Why this matters: The agent improves while working, without needing to stop, collect data, and retrain.

---

Step 7: Evaluation — How Do We Know It Works?

The Five Benchmarks

Evo-Harness was tested on five realistic benchmarks:

What Makes This Evaluation Rigorous?

The paper emphasizes two things prior work often missed:

1. Validation on complex real-world tasks (not just toy problems)
2. Isolating the drivers of improvement (understanding why it works, not just that it works)

Prior Work:          Evo-Harness:
"It works! ✓"   →   "It works, AND here's exactly why ✓"

---

Step 8: Putting It All Together — The Big Picture

Conceptual Summary

PROBLEM:
  Agents encounter novel tasks once → noisy contexts → hard to learn

SOLUTION (Evo-Harness):
  1. Keep agent frozen (no retraining)
  2. Build a structured harness around it
  3. After each task, compile noisy context → clean skills
  4. Update harness with new skills
  5. Agent performs better on next task

KEY INNOVATION:
  Context-to-Harness Skill Compilation
  = turning messy one-shot experience into reusable structured knowledge

Why This Approach is Significant

---

Quick Self-Check Questions

Test your understanding:

1. What problem does Evo-Harness solve? (Hint: noisy, one-shot executions)

2. Why is the base agent kept frozen? (Hint: think about cost and practicality)

3. What is the difference between cross-domain and topic-level skills?

4. What does "compilation" mean in the context of skill compilation?

5. Why is online learning important for realistic agent deployment?

---

Key Vocabulary Summary