Step-by-Step Teaching

Step 1: The Problem — Why Monolithic VLA Models Fall Short

What is a VLA Model?

A Vision-Language-Action (VLA) model takes:

• 👁️ Visual input (camera feed)
• 💬 Language instruction ("pick up the cup")
• ➡️ Outputs raw motor commands (move arm 2cm left, close gripper, etc.)

The Monolithic Problem

Instruction → [Big Black Box Model] → raw motor commands
                                       (no structure, no reuse)

Think of it like this:

Imagine writing a computer program where every task requires rewriting everything from scratch, with no functions, no libraries, no reusable code.

Two core problems:

---

Step 2: Existing Approaches and Their Gaps

Before REFACTOR-VLA, two main approaches existed:

Approach A: Clustering Embeddings

(e.g., AtomicVLA, AtomSkill)

Action sequences → Contrastive Embeddings → Cluster similar ones

❌ Problem: Clustering embeddings doesn't directly answer: "Do these two action sequences produce the same behavioral outcome?"

Approach B: LLM as Judge

(e.g., BLADE, LRLL)

Action sequences → Ask LLM: "Are these equivalent?" → Group them

❌ Problem: The LLM has no knowledge of the robot's actual physics/dynamics. It's like asking someone who has never driven a car to judge if two driving maneuvers are equivalent.

The Core Gap

None of these methods properly define behavioral equivalence in terms of what actually happens when the robot executes the actions.

---

Step 3: REFACTOR-VLA's Solution — The Big Picture

REFACTOR-VLA introduces a wake/sleep architecture — borrowed conceptually from how the human brain consolidates learning:

┌─────────────────────────────────────────────┐
│              REFACTOR-VLA                   │
│                                             │
│  😴 SLEEP PHASE          ☀️ WAKE PHASE      │
│  (Discover Skills)       (Use Skills)       │
│                                             │
│  Cluster motor           Generate typed     │
│  programs using BEK  →   lambda programs    │
│                          from skill library │
└─────────────────────────────────────────────┘

---

Step 4: The Sleep Phase — Discovering Skills with BEK

What is the Behavioral-Equivalence Kernel (BEK)?

The BEK answers: "Are two action sequences behaviorally equivalent?"

It does this by simulating outcomes in a learned latent world model (Mφ):

Action Sequence A ──┐
                    ├──► Mφ (World Model) ──► Predicted Outcome A ──┐
Action Sequence B ──┘                         Predicted Outcome B ──┴──► Compare → Similarity Score

Analogy:

Instead of asking "do these two recipes look similar on paper?", you actually cook both dishes (in simulation) and compare the results.

Why a Latent World Model?

• Running real robot experiments for every comparison is too slow and expensive
• The world model Mφ simulates outcomes cheaply in a compressed (latent) space
• It is calibrated to the robot's actual dynamics — unlike an LLM judge

Clustering Result

Similar action segments → grouped into reusable skills

---

Step 5: The Wake Phase — Using Skills with Typed Lambda Terms

What are Typed Lambda Terms?

These are structured, composable mini-programs inspired by the Hindley-Milner type system (a formal system from programming language theory).

Simple analogy:

# Instead of raw commands:
[move_x: 2, move_y: 0, gripper: close, move_x: -1, ...]

# You get structured programs:
λ pick(object: Cup) → λ place(location: Table) → done

Why types matter:

The Action Decoder

The typed lambda terms are fed into a library-conditioned rectified-flow action decoder that converts the structured program back into actual motor commands.

Typed Lambda Term → [Rectified Flow Decoder] → Motor Commands
     (structured)      (conditioned on skill library)    (executable)

---

Step 6: Quality Gates — Not All Skills Are Accepted

REFACTOR-VLA uses two filters before adding a skill to the library:

Gate 1: Minimum Description Length (MDL)

"Does adding this skill actually compress our description of behavior?"

Analogy: Only add a word to a dictionary if it saves space overall — not if it's so specific it's only used once.

MDL Check: Length(library + skill) < Length(without skill)?
           YES → Keep skill | NO → Discard

Gate 2: Return-Preservation Gate

"Does using this skill maintain task performance?"

Performance with skill ≥ threshold? 
YES → Accept | NO → Reject

Together: These gates ensure the library stays compact, useful, and performance-preserving.

---

Step 7: Three-Phase Training Schedule

Phase A          Phase B              Phase C
────────         ────────             ────────
World Model  →   Policy           →   Skill
Warmup           Optimization         Discovery

Train Mφ         Train policy         Cluster action
(+ InfoNCE       using skill          fragments into
 loss)           library              reusable skills

Key Detail: InfoNCE Loss in Phase A

• InfoNCE is a supervised contrastive loss — it trains the world model to pull similar outcomes together and push different outcomes apart
• This dramatically improves the quality of skill clustering in Phase C

---

Step 8: Experimental Findings — What the Results Tell Us

Finding 1: Bigger ≠ Better (World Model Size)

Lesson: Simply scaling up the world model hurts performance. Architecture and training objectives matter more than raw size.

Finding 2: InfoNCE Loss Dramatically Improves Skill Quality

Measured by Normalized Mutual Information (NMI) — scores closer to 1.0 = better clustering:

Lesson: The training objective (adding InfoNCE) is a high-leverage design choice — more impactful than model size.

---

Summary: The Complete Picture

REFACTOR-VLA System
═══════════════════════════════════════════════════════

INPUT: Raw demonstrations (video + language + actions)

PHASE A: Train World Model Mφ with InfoNCE loss
         → Learns to simulate robot outcomes accurately

PHASE C: Sleep — Use BEK + Mφ to cluster action segments
         → Discover behaviorally equivalent skill groups
         → Filter with MDL + Return-Preservation gates
         → Build reusable SKILL LIBRARY

PHASE B: Wake — Policy generates typed lambda terms
         → Selects and composes skills from library
         → Rectified-flow decoder produces motor commands

OUTPUT: Structured, interpretable, reusable robot behaviors
        that work on long-horizon tasks

---

Key Concepts Checklist