Scribe v3: More Accurate Transcription in 102 Languages

Image for Introducing Scribe v3 and v3 Live, our most accurate speech to text models

After studying this material, students should be able to:

  1. Explain what Scribe v3 and Scribe v3 Live are and what problems they solve
  2. Distinguish between batch and realtime speech-to-text transcription models
  3. Interpret accuracy metrics like Word Error Rate (WER) and Diarization Error Rate (DER)
  4. Evaluate when to use each model based on real-world use cases
  5. Understand the key features and improvements over previous versions

Step-by-Step Study Material

Step 1: Understanding the Problem First

Before learning about the solution, understand why accurate speech-to-text matters.

The Core Challenge

Real-world audio is messy because of:

ProblemExample
Multiple speakersPeople talking over each other in meetings
Background noiseCustomer calling from a busy street
Mid-sentence pausesSomeone reading out a number slowly
Language mixingSwitching between Hindi and English

Why Errors Are Costly

Think of transcription errors like a broken telephone game:

Audio Input → Transcript → Summary → Decision

⚠️ Every error made early travels downstream and becomes more expensive to fix later

Real-World Impact Example

A one-hour meeting at 150 words per minute = ~9,000 words

  • At 4% error rate = 360 wrong words
  • At 2% error rate = 180 wrong words
  • Difference = 180 wrong words = 3 errors per minute

On an earnings call, "fifteen" misheard as "fifty" could end up in a financial analyst's model — a potentially serious consequence.


Step 2: Meet the Two New Models

ElevenLabs released two distinct models to handle two different scenarios.

Model 1: Scribe v3 (Batch/Recorded Audio)

Use case: Pre-recorded audio files
Examples: Meeting recordings, podcast archives, clinical notes

Key improvements over Scribe v2:

  • ✅ 27% fewer word errors across 102 languages
  • ✅ 12% fewer diarization errors overall
  • ✅ 17% fewer diarization errors with 3+ speakers
  • ✅ Best-in-class speaker labeling

Model 2: Scribe v3 Live (Realtime/Streaming Audio)

Use case: Live, streaming audio
Examples: Voice agents, live customer calls, real-time captioning

Key improvements over Scribe v2 Realtime:

  • ✅ 43% fewer semantic word errors
  • ✅ 31% fewer standard word errors
  • ✅ Speaker diarization for the first time in realtime
  • ✅ 3.1% diarization error rate vs competitors' 5.9–7.3%
  • ✅ Smarter turn detection

Step 3: Understanding Key Technical Concepts

Concept 1: Word Error Rate (WER)

Definition: The percentage of words in a transcript that are incorrect

WER = (Wrong Words / Total Words) × 100

Example:

  • Actual speech: "The invoice is due on Friday"
  • Transcript: "The invoice is blue on Friday"
  • 1 wrong word out of 6 = ~16.7% WER

Concept 2: Semantic WER

Definition: Only counts an error when the meaning of the transcript changes

Why it matters for realtime models: Streaming models write text before hearing the full sentence, so they may format things differently mid-stream:

  • "one hundred and twenty three dollars" vs "$123"
  • Standard WER = error
  • Semantic WER = not an error (same meaning)

💡 Semantic WER is the fairer metric for realtime/streaming models


Concept 3: Speaker Diarization

Definition: The process of identifying who said what in a multi-speaker recording

Why it's hard:

  • People interrupt each other
  • Voices can sound similar
  • Background noise confuses the model

Measured by: Diarization Error Rate (DER)

  • Measured in time, not words
  • In a 1-hour meeting, every 1% DER = 36 seconds attributed to the wrong speaker

Concept 4: Turn Detection

Definition: Predicting when a speaker has finished talking so the system can respond

Old approach (silence-based):

Short silence threshold → Interrupts people mid-sentence ❌
Long silence threshold → Agent feels slow and unresponsive ❌

New approach (context-aware):

Silence + Conversation Context → Smarter prediction ✅

Example: A caller pausing while reading out a phone number is NOT done speaking — Scribe v3 Live understands this.


Step 4: Key Features Breakdown

Feature 1: Multilingual Support

  • Covers 102 languages
  • Improvements vary by language:
LanguageWER Improvement (Batch)
German42%
Spanish35%
English23%
Swahili34%
Tamil29%
Thai28%
  • Scribe v3 Live adds multilingual mode for mid-sentence language switching

Feature 2: Keyterm Prompting vs Keyterm Library

FeatureKeyterm PromptingKeyterm Library
SizeUp to 1,000 termsThousands of terms
Best forTerms you know will appearTerms that might appear
How it worksAlways appliedConsulted only on likely match
Example useBrand names, acronymsFull drug name catalog

Feature 3: Output Formatting Controls

Users can now control:

  • Numbers: "123" vs "one hundred and twenty-three"
  • Dates: Custom formats like YYYY-MM-DD
  • Custom rules: Written in plain English

Step 5: Real-World Application — Eneco Case Study

Company: Eneco (energy supplier, 2M+ customers in Netherlands) Use case: Voice agent for customer calls Model used: Scribe v3 Live

Results:

  • Dutch postcodes recognized correctly on first try: +40% improvement
  • House numbers recognized correctly on first try: +15% improvement

🔑 Key insight: A single misheard digit (like a postcode) can fail the entire customer interaction — accuracy directly impacts business outcomes


Step 6: How to Choose the Right Model

Use this decision framework:

Is your audio pre-recorded or live?
        |                    |
   Pre-recorded            Live/Streaming
        |                    |
   Scribe v3           Scribe v3 Live
        |                    |
Best for:              Best for:
- Meeting notes        - Voice agents
- Podcasts             - Live captions
- Clinical docs        - Real-time calls
- Archives             - Customer support

Step 7: Enterprise and Compliance Considerations

For teams building production systems, note:

FeatureDetail
Default regionUnited States
Isolated environmentsEU, India, Singapore
Data privacyZero Retention Mode available
CertificationsSOC 2, ISO 27001, PCI DSS Level 1
HealthcareHIPAA-eligible with BAA
Medical versionScribe v3 Live optimized for medical conversations

Summary: The Big Picture

PROBLEM: Real-world audio is messy → errors travel downstream → costly to fix

SOLUTION:
┌─────────────────────────────────────────────────────┐
│  Scribe v3          │  Scribe v3 Live               │
│  (Batch)            │  (Realtime)                   │
│  27% fewer errors   │  43% fewer semantic errors    │
│  Best diarization   │  First realtime diarization   │
│  102 languages      │  Smart turn detection         │
└─────────────────────────────────────────────────────┘

KEY METRICS TO REMEMBER:
• WER = word-level accuracy
• Semantic WER = meaning-level accuracy (better for realtime)
• DER = speaker attribution accuracy

CORE FEATURES:
• Diarization → Who said what
• Turn detection → When to respond
• Keyterm support → Domain-specific vocabulary
• Multilingual → 102 languages, mid-sentence switching

Quick Knowledge Check

Test yourself with these questions:

  1. What is the difference between WER and Semantic WER, and why does it matter for realtime models?
  2. Why is a silence-only turn detection system problematic for voice agents?
  3. In a 2-hour meeting, what does a 1% improvement in DER actually mean in practical terms?
  4. When would you use Keyterm Library instead of Keyterm Prompting?
  5. Why did Eneco's postcode recognition improve with Scribe v3 Live?

More to study