How Rubrics Improve Evidence-Grounded Answers

Learning Outcomes

After studying this material, you should be able to:

1. Explain why designing reward signals for open-domain question answering (QA) is challenging
2. Describe what a rubric-based reward framework is and how it works
3. Identify the key quality dimensions used to evaluate answers
4. Understand how grounding rubrics in retrieved evidence improves factual accuracy
5. Compare rubric-based approaches against baseline methods
6. Recognize extensions of open-domain QA into visual and multilingual contexts

---

Step 1: Understanding the Problem Space

What is Open-Domain Question Answering (QA)?

Open-domain QA means building systems that can answer any question across any topic, not just a narrow domain.

User asks: "What causes northern lights?"
         ↓
System searches knowledge sources
         ↓
System generates a comprehensive answer

Why is it Hard to Train These Systems?

The core challenge is defining what "good" looks like.

Key Insight: Telling a model "this answer is better" is like telling a student "your essay is bad" without explaining whether the problem is grammar, logic, or evidence.

---

Step 2: The Traditional Approach and Its Limitations

Preference-Based Training (The Old Way)

Traditional alignment uses human preferences:

Answer A  vs  Answer B
    ↓              ↓
Human picks the "better" one
    ↓
Model learns from this signal

Why This Falls Short

• Produces a flat, holistic signal — no explanation of what made one answer better
• Cannot distinguish between:
  • Answer that is factually correct but poorly organized
  • Answer that is well-written but unsupported by evidence
• Hard to scale — requires massive human annotation

---

Step 3: Introducing Rubric-Based Reward Framework

What is a Rubric?

A rubric is a structured scoring guide that breaks evaluation into specific, measurable criteria.

Example Rubric for a QA Answer:

Question: "What are the health effects of sleep deprivation?"

Rubric Dimensions:
├── Factual Accuracy (Is it supported by evidence?)
├── Coherence (Does it flow logically?)
├── Organization (Is it well-structured?)
├── Instruction Following (Did it answer what was asked?)
└── Grounding (Is it tied to retrieved sources?)

How the Framework Works — Step by Step

Step 1: User submits a query
           ↓
Step 2: System retrieves relevant evidence documents
           ↓
Step 3: A rubric is GENERATED specific to this query + evidence
           ↓
Step 4: Model generates an answer
           ↓
Step 5: Answer is scored against each rubric dimension
           ↓
Step 6: Fine-grained scores guide model training

Critical Distinction: The rubric is query-specific — it's not a generic template but tailored to what this particular question requires.

---

Step 4: The Three Evaluation Axes

The framework evaluates answers across three main axes:

Axis 1: Composition

• Is the answer well-written?
• Is it coherent and logically organized?
• Does it communicate clearly?

Axis 2: Grounding

• Is the answer supported by retrieved evidence?
• Are claims traceable to sources?
• Does it avoid hallucination (making things up)?

Axis 3: Instruction-Following

• Did the model answer what was actually asked?
• Did it respect format requirements?
• Did it address all parts of the question?

Final Score = Average(Composition + Grounding + Instruction-Following)

---

Step 5: Understanding "Grounded" Rubrics

What Does "Grounding" Mean?

Grounding means anchoring the rubric to retrieved evidence, not just general quality principles.

Without Grounding:

"The answer should be factually accurate."

With Grounding:

"The answer should reference the retrieved study from Johns Hopkins (2023) showing that 7-9 hours of sleep is optimal, and explain the cortisol mechanism described in Source 2."

Why This Matters

---

Step 6: Results and What They Mean

Performance Improvements

Instruction-tuned baseline:     [████████░░] 
Flat rubric variants:           [████████░░░] +0% vs rubric-based
Rubric-based framework:         [██████████░] +6.5% over baseline
                                              +4.0% over flat rubrics

Breaking Down the Gains

Key Takeaway: Each design choice had a specific, measurable benefit — this is the power of multi-dimensional evaluation.

---

Step 7: Extensions — Visual and Multilingual QA

Visual QA (OKVQA)

The challenge extends beyond text:

Image of Eiffel Tower
    +
Question: "When was this landmark built?"
    ↓
System must:
1. Understand the image (visual grounding)
2. Retrieve external knowledge about the Eiffel Tower
3. Generate an accurate answer

Key Insight: Many visual questions can be rewritten as text-only questions by resolving what the image refers to — enabling reuse of text-based QA systems.

Multilingual QA (MKQA)

MKQA Dataset:
- 10,000 question-answer pairs
- Aligned across 26 languages
- 260,000 total QA pairs
- Tests cross-lingual generalization

This tests whether QA systems work across languages, not just English — a critical real-world requirement.

---

Summary: The Big Picture

PROBLEM:  Single reward scores are too vague for complex QA training
    ↓
SOLUTION: Generate query-specific, evidence-grounded, multi-dimensional rubrics
    ↓
RESULT:   +6.5% improvement across composition, grounding, instruction-following
    ↓
EXTENSION: Same challenges apply to visual QA and multilingual QA

Core Principles to Remember

---

Quick Self-Check Questions

1. Why is a single scalar reward insufficient for training QA systems?
2. What are the three evaluation axes in the rubric framework?
3. How does grounding a rubric in retrieved evidence improve answer quality?
4. What is the difference between a flat rubric and a decomposed rubric?
5. How does OKVQA extend the open-domain QA challenge?