Step-by-Step Teaching

Step 1: The Core Problem — Why Biology Is Uniquely Risky

AI models powerful enough to help doctors and researchers are equally powerful enough to help bad actors.

This creates a fundamental tension:

Same capability → Beneficial use (drug development)
              → Harmful use (biological weapons)

Real example from the article:

Developing captopril (blood pressure drug) required isolating toxic snake venom components

The knowledge needed for good and harm is often identical. This is called dual-use risk.

---

Step 2: Understanding the Safeguard Mechanism — Safety Classifiers

A safety classifier is a smaller, automated AI system that monitors what users ask the main model.

Think of it like a security checkpoint:

User Query
    ↓
[Safety Classifier] ← Scans the request
    ↓              ↓
ALLOWED          BLOCKED
    ↓              ↓
Fable 5        Opus 5
(powerful)     (less capable, safer)

The switch to a less capable model is called a "fallback"

---

Step 3: The Classification Challenge — Two Types of Errors

Classifiers must avoid two opposite mistakes:

Key insight: These errors pull in opposite directions

• Reducing false positives → risk of more false negatives
• Reducing false negatives → risk of more false positives

This is why classifier design requires careful balance

---

Step 4: The Classifier Boundary — A Visual Mental Model

Imagine a spectrum of biology requests:

CLEARLY SAFE ←————————————————→ CLEARLY HARMFUL

[Lab results] [Symptoms] [Virology] [Weapon synthesis]
     ↑              ↑         ↑              ↑
  Dark green    Dark green  Orange          Red
  (allowed)     (allowed)  (blocked)      (blocked)

There is also a safety margin (light green zone):

• Requests that are very likely safe
• But blocked anyway out of caution
• This is where most false positives occur

The classifier boundary is the line separating allowed from blocked content

---

Step 5: The Launch Tradeoff — Why Start Broad?

At Fable 5's launch, Anthropic faced a strategic decision:

Option A: Wait until classifiers are perfectly refined

• ✅ Fewer false positives from day one
• ❌ Delays model access by weeks or months
• ❌ Delays all beneficial uses

Option B: Launch with broad (strict) classifiers, refine over time

• ✅ Users get access immediately
• ❌ High false positive rate initially
• ✅ Can improve iteratively with real-world feedback

They chose Option B — accepting short-term frustration for faster overall access

---

Step 6: How Classifiers Are Improved — The Refinement Process

The improvement process followed these steps:

Step 1: Rewrite the "classifier constitution"
        (rules defining safe vs. harmful content)
            ↓
Step 2: Gather expert feedback
        (internal + external reviewers)
            ↓
Step 3: Generate new training data
        based on updated rules
            ↓
Step 4: Retrain the classifier
            ↓
Step 5: Verify results
        (still blocks harmful content,
         now allows more benign content)

Result: ~85% reduction in biology-related fallbacks

---

Step 7: What Remains Blocked — The Dual-Use Boundary

Even after improvements, some content remains blocked:

Why keep these blocked?

The cost of misuse in dual-use domains could be catastrophic — state actors with active bioweapons programs could use frontier AI to accelerate development

---

Step 8: The Bigger Picture — Trusted Access Pathways

The long-term solution isn't just better classifiers — it's verified access:

General Public → Standard classifiers (broad protection)
      ↓
Verified Researchers → Trusted Access Program
      ↓
Frontier biology capabilities unlocked

This mirrors how society handles other dual-use knowledge:

• Pharmacists have medication access others don't
• Security researchers have vulnerability access others don't

---

Summary: Key Concepts Consolidated

DUAL-USE RISK
    → Same knowledge serves both beneficial and harmful purposes
    → Makes classification inherently difficult

SAFETY CLASSIFIERS
    → Automated systems monitoring AI queries
    → Must balance false positives vs. false negatives
    → Defined by a "classifier boundary"

ITERATIVE REFINEMENT
    → Launch broad → gather data → refine → expand access
    → Classifier constitution → training data → retraining → verification

REMAINING GAPS
    → Dual-use professional biology still blocked
    → Trusted access pathways = long-term solution

---

Self-Check Questions

1. What makes biology a dual-use domain? Give an example not from the article.
2. What is the difference between a false positive and a false negative in classifier terms?
3. Why did Anthropic choose to launch with broad classifiers rather than waiting for perfect ones?
4. What is a fallback, and why does it exist?
5. Why can't classifiers simply block all biology queries to be safe?