SkillOpt turns AI agent skills into trainable assets

Concept 1: The Problem — Why AI Agent Skills Break Down

What is an "agent skill"?

When an AI agent performs multi-step tasks (searching the web, using tools, filling spreadsheets), it follows a set of written instructions that guide its behavior. These instructions are called skills.

How are skills currently created?

There are three common approaches:

Why do all three fail?

They share the same fundamental flaw — they don't behave like a learning system. Specifically, they lack:

• Step-size control → edits can be too large or too small
• Held-out validation → no way to check if a change actually helps
• Memory of failed revisions → the same bad edits get repeated

🔑 Key Insight: Skills tend to grow longer and drift with each rewrite. A revision that seems reasonable can quietly degrade real performance.

---

Concept 2: The Core Reframe — Treat the Skill as a Trainable Parameter

The shift in thinking

SkillOpt asks a fundamentally different question:

❌ Old question: "How do we write a better prompt?" ✅ New question: "How do we train the skill?"

What does "trainable parameter" mean here?

In deep learning, a parameter is something the optimizer adjusts to improve performance (like neural network weights). SkillOpt borrows this idea but applies it to text instead of numbers.

Traditional deep learning:
  Model weights → frozen or updated during training

SkillOpt:
  Model weights → FROZEN (never touched)
  Skill file    → treated as the trainable parameter

Why keep the model frozen?

• No expensive retraining
• No risk of catastrophic forgetting
• The skill file remains readable, auditable, and reversible

🔑 Key Insight: You don't need to change the model to improve the agent. You optimize what sits outside the model — the skill layer.

---

Concept 3: The Forward–Backward–Update Cycle

The training loop in text space

SkillOpt mirrors the classic deep learning training loop, but everything happens in natural language.

---

Step 1: Forward Pass

Current Skill → Frozen Target Model → Executes batch of tasks → Trajectories (successes + failures)

• The agent runs real tasks using the current skill file
• A rollout batch controls how many tasks are run before updating
• The output is a set of trajectories — records of what the agent did and whether it worked

---

Step 2: Backward Pass (Reflection)

Trajectories → Optimizer Model → Identifies patterns to KEEP and patterns to FIX

• A separate optimizer model (not the frozen target model) reads the trajectories
• It processes them in reflection minibatches
• It asks: "What worked? What failed? Why?"

📌 Note: Two separate models are used — one to do the task, one to learn from it. This separation is intentional and important.

---

Step 3: Update Step

Optimizer → Proposes small edits (add / delete / replace) → Merges, deduplicates, ranks, clips

The optimizer proposes bounded text edits — small, specific changes:

A textual learning rate (edit budget) limits how much can change per step — analogous to a learning rate in gradient descent.

🔑 Key Insight: Small, controlled edits prevent the skill from drifting wildly — the same way a small learning rate prevents unstable training in neural networks.

---

Concept 4: Validation Gating — Only Accept Improvements

The core quality control mechanism

After edits are proposed, they must pass a strict validation gate:

Candidate skill → Tested on held-out validation split
                       ↓
         Scores HIGHER than current skill?
              ↙ YES              ↘ NO
         Adopted ✅          Rejected ❌

This is equivalent to early stopping or model selection in traditional machine learning — you only keep a version if it's provably better.

What happens to rejected edits?

They are not thrown away. They enter a rejected-edit buffer — a memory of what didn't work. This buffer is fed back to the optimizer as negative feedback in later calls during the same epoch.

Rejected edit → Rejected-edit buffer → "Don't try this again" signal to optimizer

🔑 Key Insight: Learning from failures is just as important as learning from successes. The rejected-edit buffer is SkillOpt's equivalent of negative gradient signal.

---

Concept 5: Slow/Meta Updates — Learning Across Epochs

The two-speed learning system

SkillOpt operates on two timescales, similar to fast and slow learning in cognitive science:

Why are slow updates necessary?

Some lessons only become visible after many batches. A single batch might not reveal a systematic weakness. The epoch-wise meta update looks at the big picture — patterns that span the entire training epoch.

Batch 1 → fast update
Batch 2 → fast update
Batch 3 → fast update
...
End of Epoch → slow/meta update (consolidates what batches couldn't see)

🔑 Key Insight: This two-speed system mirrors how humans learn — quick adjustments in the moment, deeper reflection over time.

---

Concept 6: The Resulting Skill File — Compact and Auditable

What does the final skill look like?

The output is a file called best_skill.md — a plain Markdown file.

Key properties:

A striking example

OfficeQA benchmark: +39.0 point improvement from a single accepted edit

This shows that the validation gate is doing real work — most proposals are rejected, and only genuinely useful changes survive.

---

Concept 7: Transfer — Train Once, Reuse Everywhere

Why does transfer matter?

If a skill only works for the exact model and environment it was trained on, it's not very useful. SkillOpt skills are designed to capture general workflow logic, not benchmark-specific tricks.

Three types of transfer tested:

1. Cross-model scale:
   Skill trained on GPT-5.4 → applied to GPT-5.4-mini → still improves performance

2. Cross-harness:
   Skill trained in Codex → dropped into Claude Code → +59.7 point improvement
   (vs. +58.3 from training directly in Claude Code)

3. Cross-task:
   Skill trained on one math benchmark → transferred to a related math benchmark

The small model + skill file insight

SkillOpt narrows the gap between small and large models:

GPT-5.4-mini + optimized skill (64.3) > GPT-5.4 with no skill (59.7)
Qwen3.5-4B   + optimized skill       > GPT-5.2 with no skill

🔑 Key Insight: An optimized skill file can substitute for a larger model — without any additional inference cost.

---

Putting It All Together

Here is the complete SkillOpt picture:

┌─────────────────────────────────────────────────────┐
│                    SKILLOPT LOOP                    │
│                                                     │
│  Current Skill                                      │
│       ↓                                             │
│  [FORWARD] Frozen model runs tasks → trajectories   │
│       ↓                                             │
│  [BACKWARD] Optimizer reflects on successes/fails   │
│       ↓                                             │
│  [UPDATE] Propose bounded edits (add/delete/replace)│
│       ↓                                             │
│  [GATE] Validate on held-out split                  │
│       ↙ pass              ↘ fail                    │
│  Accept edit ✅      Rejected-edit buffer ❌         │
│       ↓                                             │
│  [SLOW UPDATE] Epoch-wise meta consolidation        │
│       ↓                                             │
│  best_skill.md → compact, readable, transferable   │
└─────────────────────────────────────────────────────┘

The five analogies to traditional deep learning

---

Summary of Core Concepts

1. The Problem: Agent skills drift and degrade because they lack learning-system controls
2. The Reframe: Treat the skill file as a trainable parameter outside a frozen model
3. The Loop: Forward (execute) → Backward (reflect) → Update (edit) — in text space
4. Validation Gating: Only accept edits that provably improve performance
5. Rejected-Edit Buffer: Learn from failures, not just successes
6. Slow/Meta Updates: Capture long-horizon lessons across epochs
7. Transfer: Optimized skills encode general workflow knowledge, not benchmark tricks

🎯 The Big Takeaway: Training doesn't have to mean updating model weights. Procedural knowledge outside the model — written as natural language — can also be systematically optimized, validated, and transferred.