How Orchard Makes Agentic AI Scalable and Open

Concept 1: The Core Problem — Why Agentic AI Research Is Bottlenecked

What is an "agent" in AI?

Unlike a chatbot that answers a single question, an agentic AI can:

• Plan across multiple steps
• Use tools (search, code execution, file editing)
• Recover from mistakes
• Complete complex real-world tasks autonomously

The bottleneck

Building these systems traditionally requires:

• Custom sandboxes (isolated environments to safely run agent actions)
• Closed training pipelines (proprietary step-by-step workflows)
• Proprietary datasets (training data most researchers can't access)

Think of it this way: Imagine every car manufacturer had to build their own roads before testing a car. That's the situation agentic AI researchers faced — rebuilding infrastructure from scratch every time.

---

Concept 2: Orchard Env — The Reusable Environment Layer

The central idea

Instead of embedding the runtime environment inside each specific training system, Orchard makes the environment a standalone, reusable service.

How it works technically

• Built on Kubernetes (a system for managing thousands of isolated containers in parallel)
• Creates, manages, and removes isolated components at scale
• Works across different task types (coding, web browsing, productivity tools)
• Works across different training stages (data collection → supervised fine-tuning → reinforcement learning → evaluation)

Why this matters

Analogy: Orchard Env is like a universal power outlet. Instead of every device needing its own power source, everything plugs into the same standardized infrastructure.

---

Concept 3: Training Inside Real Deployment Harnesses

What is a "harness"?

Modern AI agents don't run as bare models. They operate through harnesses — sophisticated wrappers that manage:

• Multi-turn reasoning (remembering context across steps)
• Tool use (calling search engines, running code)
• Connections to external systems

Examples: Claude Code, Codex, OpenClaw, ZeroClaw

The mismatch problem

Most open training tools cannot handle these stateful, multi-process harnesses.

So researchers were forced to:

1. Train on a simplified stand-in environment
2. Deploy in the real, complex harness
3. Accept a performance gap between training and deployment

How Orchard solves this

• A lightweight proxy records the harness's own model calls as training data
• Each rollout runs in its own container
• The agent trains end-to-end directly inside the real harness it will be deployed with

Key insight: Training and deployment now happen in the same environment, eliminating the mismatch.

---

Concept 4: Orchard-SWE — Training Software Engineering Agents

The challenge

Software engineering agents must:

• Navigate real codebases
• Diagnose bugs
• Write and verify fixes
• Recover from failed attempts

The training pipeline (step by step)

Step 1: Data Distillation

• Collected 107,000 agent interactions from two large open-weight models
• Covered a broad range of real GitHub issues

Step 2: Credit-Assignment Supervised Fine-Tuning

• Instead of discarding failed attempts entirely, the system learns from the productive portions of partial failures
• This expands the useful training data significantly

Analogy: Like a student learning from a partially correct exam answer, not just perfect ones.

Step 3: Reinforcement Learning with Sparse Feedback

• The agent learns whether its final patch passed or failed hidden tests
• Problem: feedback is rare (sparse)
• Solution: Balanced Adaptive Rollout — designed to extract maximum learning from infrequent success signals

Step 4: Dense Reward Techniques (Richer Guidance)

Two methods added on top of sparse RL:

Step 5: Value Model Reranking

• RL generates many practice trajectories that are normally discarded
• Instead, these trajectories train a compact 4-billion-parameter value model
• At problem-solving time, the value model scores multiple candidate solutions and picks the best

Results

Using only ~3 billion active parameters, approaching frontier systems 10x larger

---

Concept 5: Orchard-GUI — Training Web Navigation Agents

The challenge

Web agents must:

• Interpret visual layouts
• Interact with dynamic interfaces
• Complete open-ended tasks described only in natural language

The approach

• Model size: 4 billion parameters (vision-language model)
• Training data: surprisingly small — 400 distilled demonstrations + 2,200 open-ended tasks

Key insight demonstrated

Data efficiency matters more than data volume when the training approach and environment are well-designed.

Results

This makes Orchard-GUI the strongest open-source web agent while remaining competitive with larger proprietary models.

---

Concept 6: Orchard-Claw — Training Personal Assistant Agents

The challenge

Personal assistant tasks include:

• Reading and drafting emails
• Managing calendars
• Searching for information
• Coordinating across tools

The approach

• Trained on just 200 synthetic tasks (extremely small dataset)
• Trained across multiple real harnesses simultaneously: ReACT, ZeroClaw, OpenClaw, Codex

Why multi-harness training matters

Training inside real harnesses substantially improves reliability:

Results on Claw-Eval benchmark

---

Concept 7: Cumulative Agent Learning — The Future Direction

The current waste problem

In standard RL training:

• Agents generate thousands of practice trajectories
• Once training ends, these trajectories are discarded
• The next generation of agents starts from scratch

Orchard's proposed solution

Treat trajectories as persistent assets:

Generation 1 agent → produces trajectories
         ↓
    Distilled into value models / datasets
         ↓
Generation 2 agent → inherits knowledge + adds new experience
         ↓
    Continues accumulating...

Analogy: Like how human knowledge accumulates across generations through books and education, rather than each person rediscovering everything from scratch.

---

Summary: The Big Picture

PROBLEM: Agentic AI research requires expensive, proprietary infrastructure
         ↓
SOLUTION: Orchard — open, reusable, scalable framework
         ↓
CORE: Orchard Env (Kubernetes-based, works across tasks and training stages)
         ↓
KEY INNOVATION: Train inside real deployment harnesses (no train/deploy mismatch)
         ↓
DEMONSTRATED ACROSS THREE DOMAINS:
  • Orchard-SWE → software engineering (69.7% → 73% with reranking)
  • Orchard-GUI → web navigation (68.4% average, data-efficient)
  • Orchard-Claw → personal assistant (multi-harness training)
         ↓
FUTURE: Cumulative learning — experience accumulates across agent generations

The overarching lesson

Small, well-trained open models + the right infrastructure can approach or match frontier systems many times their size. The environment layer is not just scaffolding — it is a core determinant of agent capability.