Step-by-Step Study Material
Step 1: What Is a Long-Horizon AI Agent?
Most people imagine an AI agent as something that answers a single question. A long-horizon agent is fundamentally different.
Definition
A long-horizon agent completes work that spans:
- Multiple hours of execution
- Hundreds of sequential decisions
- Steps where later decisions depend on earlier ones
Why This Is Hard
Think of it like a chain of dominoes:
Decision 1 → Decision 2 → Decision 3 → ... → Final Output
↑
If this is wrong, everything after it may also be wrong
Three specific problems in accounting make this worse:
| Problem | Why It Matters |
|---|---|
| No cheap objective test | You cannot instantly verify if a tax return is correct |
| Ground-truth examples are expensive | Real accounting work is hard to use as training data |
| Results take hours or days to review | Feedback is slow |
Key Insight
A correct final answer can still hide a bad process
Example: An agent produces the right tax number but never checked a primary legal authority. The answer looks right today but the process will fail on a different case tomorrow.
Step 2: Why Context Is a Production Input
What Is "Context" in an AI Agent?
Context is everything the agent reads while doing its work:
- Instructions
- Domain knowledge
- Examples
- Tool descriptions
- Memory from earlier steps
The Critical Difference From Traditional Software
| Traditional Code | AI Agent Context |
|---|---|
| Same valid code = same behavior always | Same meaning, different wording = different behavior |
| Organization of files does not change logic | Organization and wording directly change what the model does |
| Bugs are usually explicit errors | Problems can be subtle: a vague sentence, a buried exception |
Practical Consequence
If you generate a context file and ship it without reading it, you are taking a production risk.
Rule: Context must be read, understood, and revised by engineers — not just generated and deployed.
Step 3: What Is a Behavior Spec?
Definition
A behavior spec is a Markdown document that defines one specific, recurring behavior expected from an agent in a specific situation.
Critical Distinctions
Behavior Spec ≠ Prompt
- A prompt is shown to the agent and tells it what to do
- A behavior spec is shown to a judge and defines what correct behavior looks like
What a Good Behavior Spec Contains
A useful spec answers six questions:
- When does this behavior apply? (the triggering situation)
- What evidence should the agent inspect?
- What decision should the agent make?
- What action should follow that decision?
- What should happen when evidence is incomplete?
- What does failure look like?
How Judgment Works
Judge receives:
├── The behavior spec (the standard)
├── The recorded trajectory (what the agent actually did)
└── The evidence (tool calls, artifacts, sources, decisions)
Judge returns:
├── TRUE → behavior appeared correctly
├── FALSE → behavior was missing or wrong
└── NA → behavior was not applicable in this case
Why This Matters
You can evaluate specific parts of the process without needing a complete ground-truth answer for the entire task. This solves the expensive evaluation problem from Step 1.
Step 4: The Development Loop
This is the iterative cycle Basis uses to continuously improve agent behavior.
The Loop Visualized
1. Agree on a behavior worth measuring
↓
2. Write or refine the behavior spec (in Cursor)
↓
3. Agent runs in production → recorded trajectory produced
↓
4. Judge evaluates trajectory against the spec
↓
5. FALSE verdict? → Gap identified between standard and reality
↓
6. Update runtime context, prompts, tools (revised in Cursor)
↓
7. Run agent again → measure if behavior improved
↑_____________________________________________|
The Fundamental Principle
Spec = The Standard (stays stable)
Runtime = The Implementation (changes until it meets the standard)
These two things are deliberately kept separate. You do not change the spec to match what the agent does. You change the agent until it matches the spec.
Step 5: Connecting the Pieces — The Full Mental Model
Here is how everything fits together:
PROBLEM
Long-horizon agents make hundreds of decisions.
Errors compound. Final output hides bad process.
↓
INSIGHT
Context shapes every decision the agent makes.
Context must be treated like production code.
↓
SOLUTION: Behavior Specs
Define explicit standards for specific behaviors.
Written for judges, not for the agent.
↓
PROCESS: Development Loop
Write spec → Run agent → Judge trajectory →
Find gaps → Revise context → Repeat
↓
RESULT
Agents that are reliable, not just occasionally correct.
Failures are diagnosable. Improvements are measurable.
Step 6: Key Vocabulary Summary
| Term | Definition |
|---|---|
| Long-horizon agent | An AI agent that completes multi-step work over hours with hundreds of interdependent decisions |
| Context | All natural language input the agent reads during its work (instructions, examples, tool descriptions, etc.) |
| Behavior spec | A Markdown document defining expected agent behavior for a judge to evaluate — not a prompt |
| Trajectory | The recorded sequence of decisions, tool calls, and actions an agent took during a task |
| Judge | A system or model that evaluates a trajectory against a behavior spec, returning true/false/NA |
| Runtime | The live implementation — prompts, tools, instructions — that the agent actually uses |
Self-Check Questions
Test your understanding before moving on:
- Why can a correct final output still represent a failure in a long-horizon agent system?
- What is the difference between a behavior spec and a prompt?
- Why does the wording of context matter in ways that traditional code organization does not?
- In the development loop, what changes when a judge returns FALSE — the spec or the runtime? Why?
- What three verdicts can a judge return, and what does each mean?
Core Takeaway
Building reliable long-horizon agents requires treating context as a production input, defining explicit behavioral standards through specs, and running a disciplined iterative loop that separates the standard from the implementation. Correct outputs are necessary but not sufficient — the process must be reliable and inspectable.