A VLA is a robot control model built by taking a vision-language model (already trained on massive internet data) and fine-tuning it on robot demonstrations.
Internet-scale image/text data
↓
Pretrain VLM
↓
Fine-tune on robot demos
↓
Robot Policy (VLA)
Researchers discovered that making the model think step-by-step before acting (called a Chain-of-Thought / CoT) improves performance — similar to how humans plan before moving.
Current approach (fixed template):
At EVERY timestep, the robot thinks:
1. List ALL visible objects with bounding boxes
2. Create a high-level plan
3. Describe affordances
4. Identify gripper position
5. ... then act
| Issue | Why It Hurts |
|---|---|
| Expensive to design | Teams spend months writing annotation guidelines |
| May be wrong | Listing every object in a cluttered scene drowns out what actually matters |
| Redundant | Re-planning every timestep is often unnecessary |
| Slow | Generating verbose text creates real lag (5+ seconds per step) |
Key Insight: A fixed reasoning template assumes we already know what a robot should think about — but we don't.
This is the core challenge the paper addresses.
To validate good reasoning → need a policy that understands physical action
↑ ↓
To train that policy ← need validated reasoning data
In other words:
| Domain | Data Available |
|---|---|
| Language | Billions of documents with reasoning |
| Vision | Massive image-caption datasets |
| Robotics | Scarce demos, NO reasoning-to-action links |
Instead of treating reasoning as a fixed annotation, treat it as an unobserved latent variable Z.
Observed: C (scene image + task) → A (expert action)
Hidden: Z (reasoning) — the "explanation" connecting C to A
Good reasoning = whatever best explains why the expert action was correct.
This reframing allows the use of variational inference — a well-established mathematical framework for learning hidden variables.
R&B-EnCoRe solves the circular dependency with a 3-stage loop that requires no human annotation, no rewards, and no external verifiers.
┌─────────────────────────────────────────────┐
│ │
│ Generate → Train → Refine & Bootstrap │
│ ↑ │ │
│ └────────────────────────┘ │
│ (repeat) │
└─────────────────────────────────────────────┘
Goal: Create a diverse pool of candidate reasoning strategies.
The model hypothesizes many reasoning primitives, such as:
Key innovation — Reasoning Dropout:
Primitive 1 (Plan): KEEP ✓
Primitive 2 (Objects): DROP ✗
Primitive 3 (Subtask): KEEP ✓
Primitive 4 (Gripper): DROP ✗
Primitive 5 (Affordance):KEEP ✓
Each primitive is independently kept or dropped with some probability, generating traces ranging from sparse to verbose. This seeds the cycle with many possible reasoning strategies instead of one rigid template.
This mirrors the structure of a Variational Autoencoder (VAE).
| Role | Model | Inputs | Purpose |
|---|---|---|---|
Prior p(Z, A | C) | Online policy | Context only | Generates reasoning + action at test time |
Posterior q(Z | C, A) | Hindsight explainer | Context + true action | Proposes reasoning that explains why the action was right |
Prior: [Scene Image + Task] → reasoning → action
Posterior: [Scene Image + Task + Action] → reasoning (hindsight)
Intuition: The posterior "cheats" by seeing the answer, learning to explain it. The prior must learn to reason without seeing the answer. Training both together creates a self-critique loop.
Goal: Keep only the reasoning that actually helps predict the correct action.
Step 1: Draw K candidate reasoning traces from the posterior for each demonstration.
Step 2: Score each trace with an importance weight:
w(Z) = p(Z, A | C) ← prior probability (how likely without seeing action)
─────────────
q(Z | C, A) ← posterior probability (how likely given action)
Step 3: Resample one trace per demonstration proportional to these weights.
What does this weight measure?
The paper proves this ratio captures the information benefit of a reasoning strategy:
High w(Z) → reasoning makes the correct action MORE predictable
→ AMPLIFY this reasoning
Low w(Z) → reasoning is distracting or redundant
→ SUPPRESS this reasoning
Step 4: Retrain the VLA on this self-curated, action-predictive dataset.
Key Insight: The cycle automatically discovers which reasoning primitives matter for each specific robot embodiment — without anyone telling it.
The most interpretable output is what reasoning the model converges to for different robot types.
KEPT: Meta-action descriptions ("Move"), gripper position, subtasks
PRUNED: Exhaustive object enumeration
RESULT: ~50% fewer tokens, more focused reasoning
KEPT: Affordances (what the terrain allows), proposed movements
PRUNED: Counterfactual reasoning ("what if I went left instead?")
KEPT: Goal-related cues, object-related cues
PRUNED: Hallucinated "experience" (fabricated prior knowledge)
Pattern: Each embodiment converges to different reasoning — because different robots need to think about different things. A fixed template would fail all of them.
| Domain | Model Size | Improvement |
|---|---|---|
| Manipulation (LIBERO-90) | 1B | +28% success rate |
| Manipulation (real robot) | 7B | Holds up on out-of-distribution scenes where baseline drops 31% |
| Legged Navigation | 30B | +101% navigation score |
| Autonomous Driving | 4B | -21% collision rate |
Before R&B-EnCoRe: >5 seconds per control step
After R&B-EnCoRe: ~3 seconds per control step
Performance improves with more posterior samples K — exactly what the theory predicts, validating the mathematical framework.
PROBLEM:
Fixed reasoning templates are expensive, possibly wrong, and slow.
We can't validate reasoning without a good policy.
We can't build a good policy without validated reasoning.
SOLUTION (R&B-EnCoRe):
1. Treat reasoning as a latent variable Z
2. Generate diverse candidate reasoning via Reasoning Dropout
3. Train prior (policy) + posterior (hindsight explainer) jointly
4. Score reasoning by information benefit (importance weighting)
5. Retrain on self-curated, action-predictive traces
6. Repeat → self-improving loop
KEY PRINCIPLE:
Selective reasoning > Exhaustive reasoning
The cycle discovers WHAT to think about, not just HOW to think.
The elegance of R&B-EnCoRe is that it turns the circular dependency into a feature: the policy and its reasoning data co-evolve, each making the other better, with no external supervision required.