Auto-review enhances agent autonomy by assessing risks contextually before actions.

Concept 1: The Autonomy Problem

What is it?

Agents (AI systems that perform tasks) need autonomy — the ability to act independently without constantly asking for permission.

The Tension

The Key Insight

Asking for permission too often creates its own safety problem — users stop reading carefully, and approvals become meaningless (like clicking "I Agree" without reading).

---

Concept 2: The Dial vs. The Switch

What is it?

The core design philosophy behind Auto-review.

The Switch (Old Way)

Agent Action → Ask User? YES or NO → Proceed

• Binary: either always ask, or never ask
• No middle ground

The Dial (New Way)

Low Stakes ←————————————→ High Stakes
Agent moves freely          Agent slows down

• Autonomy is a spectrum, not an on/off setting
• The agent moves freely when stakes are low
• The agent slows down when crossing a meaningful boundary

---

Concept 3: Context-Dependent Risk

What is it?

Risk is not absolute — the same action can be safe or dangerous depending on the situation.

Example

python script.py

• ✅ Safe: if the script just formats text
• ❌ Dangerous: if the script deletes production data

What Actually Determines Risk?

Three factors together:

1. The action itself — what is being done
2. The user's request — what the user actually asked for
3. The consequence of being wrong — how bad is a mistake here?

Risk = Action + Context + Consequence

---

Concept 4: The Classifier Agent

What is it?

A specialized small AI model that reviews actions before they execute, acting as a contextual gatekeeper.

How It Works

User Request
     ↓
Parent Agent plans an action
     ↓
Classifier Agent reviews the action ← (NEW STEP)
     ↓
Allow or Block
     ↓
Action executes (or doesn't)

Design Goals

The Central Rule Given to the Classifier

Be more lenient when security stakes are lower. Be more cautious when security stakes are higher.

---

Concept 5: Making the Classifier Agentic

What is it?

The classifier isn't just a passive rule-checker — it can actively investigate before making a decision.

Why This Matters

Some actions can't be judged from the command alone:

python script.py  ← Can't tell if safe just from this

Tools the Classifier Can Use

• ReadFile — read the contents of a file
• Grep — search for patterns inside files
• Glob — find files matching a pattern
• ListDir — see what's in a directory

The Flow

Classifier sees: python script.py
       ↓
Classifier uses ReadFile to inspect script.py
       ↓
Classifier sees: script deletes /prod database
       ↓
Classifier BLOCKS the action

---

Concept 6: The Feedback Loop (Not Just Blocking)

What is it?

When the classifier blocks an action, it explains why to the parent agent — not just to the user.

Why This Is Clever

Classifier blocks action
        ↓
Sends explanation to Parent Agent
        ↓
Parent Agent finds a SAFER alternative
        ↓
User is never interrupted

The Key Question the Classifier Asks

Is this action justified by what the user asked for?

• Not: "Does this action look risky in isolation?"
• But: "Does this action make sense given the user's intent?"

This allows normal development work to continue while flagging actions that go beyond what was requested.

---

Concept 7: Architecture — Avoiding Extra Latency

The Problem

If the classifier were a separate service, every action would require:

Agent → [Network call] → Classifier Service → [Network call back] → Agent

This adds delay before every single tool call.

The Solution

The classifier runs in the same stream as the parent agent — similar to how subagents work.

Same RPC Stream
┌─────────────────────────────┐
│  Parent Agent               │
│       ↓                     │
│  Classifier (inline)        │
│       ↓                     │
│  Tool executes              │
└─────────────────────────────┘

• No extra network round trip
• Faster decisions
• Same infrastructure

---

Concept 8: Testing the Classifier (Evals)

What is it?

A rigorous testing process to ensure the classifier catches risky actions without blocking normal work.

Two Types of Test Data

1. Real Usage Data

• ~12 hours of internal developer sessions
• Deduplicated into 6,122 labeled rows
• Shows the "normal shape" of agent work

2. Synthetic Data

• Generated worst-case scenarios
• Examples: reading secrets, touching production data, following untrusted instructions
• Covers failures that rarely appear in normal usage

The Labeling Challenge

When the policy changed, the entire dataset had to be re-labeled — otherwise you'd be testing a new classifier against an outdated standard.

---

Concept 9: Detecting "Flapping"

What is it?

A quality signal that reveals unstable classifier behavior.

What Flapping Looks Like

Same action tested 10 times:
✅ Allow ✅ Allow ❌ Block ✅ Allow ❌ Block ❌ Block ✅ Allow ❌ Block ✅ Allow ✅ Allow
= 6 allows, 4 blocks → FLAPPING

What It Means

• The policy or prompt is underspecified
• The classifier is uncertain about this case
• Fix: tighten the policy until behavior is consistent

---

Concept 10: Real-World Results

The Numbers

What This Means in Practice

100 agent actions
       ↓
~4 get blocked by classifier
       ↓
Most blocks → Parent Agent finds safer path (no user interruption)
       ↓
Only ~7% of total chats ever interrupt the user

The Improvement

From 40% of actions blocked (old way) → 7% of chats interrupted (new way)

---

Summary: How All Concepts Connect

┌─────────────────────────────────────────────────────┐
│              THE BIG PICTURE                        │
│                                                     │
│  User Request                                       │
│       ↓                                             │
│  Parent Agent plans action                          │
│       ↓                                             │
│  Classifier Agent (small, fast, agentic)            │
│    • Reads context (files, tools)                   │
│    • Asks: "Is this justified by user's intent?"    │
│    • Considers: action + context + consequence      │
│       ↓                    ↓                        │
│    ALLOW               BLOCK + Explanation          │
│       ↓                    ↓                        │
│  Action runs      Parent Agent finds safer path     │
│                            ↓                        │
│                   (rarely) Interrupt user           │
└─────────────────────────────────────────────────────┘

The Core Philosophy

Autonomy is a dial, not a switch. Agents move freely when stakes are low. Agents slow down when crossing meaningful boundaries. The classifier makes that judgment — in context, in real time, without slowing everything down.