
After studying this material, students should be able to:
Before learning about the solution, understand why accurate speech-to-text matters.
Real-world audio is messy because of:
| Problem | Example |
|---|---|
| Multiple speakers | People talking over each other in meetings |
| Background noise | Customer calling from a busy street |
| Mid-sentence pauses | Someone reading out a number slowly |
| Language mixing | Switching between Hindi and English |
Think of transcription errors like a broken telephone game:
Audio Input → Transcript → Summary → Decision
⚠️ Every error made early travels downstream and becomes more expensive to fix later
A one-hour meeting at 150 words per minute = ~9,000 words
On an earnings call, "fifteen" misheard as "fifty" could end up in a financial analyst's model — a potentially serious consequence.
ElevenLabs released two distinct models to handle two different scenarios.
Use case: Pre-recorded audio files
Examples: Meeting recordings, podcast archives, clinical notes
Key improvements over Scribe v2:
Use case: Live, streaming audio
Examples: Voice agents, live customer calls, real-time captioning
Key improvements over Scribe v2 Realtime:
Definition: The percentage of words in a transcript that are incorrect
WER = (Wrong Words / Total Words) × 100
Example:
Definition: Only counts an error when the meaning of the transcript changes
Why it matters for realtime models: Streaming models write text before hearing the full sentence, so they may format things differently mid-stream:
💡 Semantic WER is the fairer metric for realtime/streaming models
Definition: The process of identifying who said what in a multi-speaker recording
Why it's hard:
Measured by: Diarization Error Rate (DER)
Definition: Predicting when a speaker has finished talking so the system can respond
Old approach (silence-based):
Short silence threshold → Interrupts people mid-sentence ❌
Long silence threshold → Agent feels slow and unresponsive ❌
New approach (context-aware):
Silence + Conversation Context → Smarter prediction ✅
Example: A caller pausing while reading out a phone number is NOT done speaking — Scribe v3 Live understands this.
| Language | WER Improvement (Batch) |
|---|---|
| German | 42% |
| Spanish | 35% |
| English | 23% |
| Swahili | 34% |
| Tamil | 29% |
| Thai | 28% |
| Feature | Keyterm Prompting | Keyterm Library |
|---|---|---|
| Size | Up to 1,000 terms | Thousands of terms |
| Best for | Terms you know will appear | Terms that might appear |
| How it works | Always applied | Consulted only on likely match |
| Example use | Brand names, acronyms | Full drug name catalog |
Users can now control:
Company: Eneco (energy supplier, 2M+ customers in Netherlands) Use case: Voice agent for customer calls Model used: Scribe v3 Live
Results:
🔑 Key insight: A single misheard digit (like a postcode) can fail the entire customer interaction — accuracy directly impacts business outcomes
Use this decision framework:
Is your audio pre-recorded or live?
| |
Pre-recorded Live/Streaming
| |
Scribe v3 Scribe v3 Live
| |
Best for: Best for:
- Meeting notes - Voice agents
- Podcasts - Live captions
- Clinical docs - Real-time calls
- Archives - Customer support
For teams building production systems, note:
| Feature | Detail |
|---|---|
| Default region | United States |
| Isolated environments | EU, India, Singapore |
| Data privacy | Zero Retention Mode available |
| Certifications | SOC 2, ISO 27001, PCI DSS Level 1 |
| Healthcare | HIPAA-eligible with BAA |
| Medical version | Scribe v3 Live optimized for medical conversations |
PROBLEM: Real-world audio is messy → errors travel downstream → costly to fix
SOLUTION:
┌─────────────────────────────────────────────────────┐
│ Scribe v3 │ Scribe v3 Live │
│ (Batch) │ (Realtime) │
│ 27% fewer errors │ 43% fewer semantic errors │
│ Best diarization │ First realtime diarization │
│ 102 languages │ Smart turn detection │
└─────────────────────────────────────────────────────┘
KEY METRICS TO REMEMBER:
• WER = word-level accuracy
• Semantic WER = meaning-level accuracy (better for realtime)
• DER = speaker attribution accuracy
CORE FEATURES:
• Diarization → Who said what
• Turn detection → When to respond
• Keyterm support → Domain-specific vocabulary
• Multilingual → 102 languages, mid-sentence switching
Test yourself with these questions: