Introducing Scribe v2 Realtime

Concept 1: What is Real-Time Speech-to-Text (STT)?

Speech-to-Text (STT) is technology that converts spoken audio into written text.

• Traditional STT works on recorded audio — you upload a file, it processes it, returns text
• Real-Time STT works on live audio — it transcribes speech as it happens

🎯 Why it matters: Live use cases like voice assistants, meeting tools, and live captions cannot wait for a recording to finish. They need text immediately.

---

Concept 2: Latency — The Core Challenge of Real-Time STT

Latency = the delay between when someone speaks and when the text appears.

Scribe v2 Realtime targets under 150ms latency, which is roughly the threshold of human perception for "real-time."

---

Concept 3: Negative Latency — Predicting Before You Finish Speaking

This is the most advanced concept in the article.

Normal latency: System waits for you to finish a word → then transcribes it

Negative latency: System predicts the next word and punctuation before you finish saying it

You say:    "I'd like to schedule a meet—"
System:      Already predicting → "meeting"

🧠 Think of it like autocomplete, but for live speech. The model uses context to anticipate what comes next, effectively getting "ahead" of the speaker.

---

Concept 4: Voice Activity Detection (VAD)

VAD answers one question: "Is someone speaking right now, or is this silence/noise?"

Without VAD:

• System transcribes background noise as gibberish
• Wastes processing power on empty audio

With VAD:

• System only activates when real speech is detected
• Cleaner transcripts, lower compute cost

🎙️ VAD is especially critical in noisy environments like call centers or meetings.

---

Concept 5: Text Conditioning — Maintaining Context Across Connections

Real-time audio streams can drop and reconnect (network issues, timeouts, etc.).

The problem without text conditioning:

Connection 1: "The meeting is scheduled for—" [drops]
Connection 2: [restarts with no memory] → confused transcription

With text conditioning:

• The system uses the previous transcript batch as context
• When reconnecting, it picks up where it left off seamlessly

🔗 Think of it like giving the model a "memory" of what was already said.

---

Concept 6: Manual Commit — Controlling Transcript Finalization

In real-time STT, transcripts go through two states:

Manual commit gives you (the developer) control over when a segment is finalized.

Use cases:

• Wait until a speaker finishes a full sentence before committing
• Sync transcript timing with other events in your app
• Avoid premature finalization mid-sentence

---

Concept 7: Automatic Language Detection & Mid-Conversation Switching

Traditional STT requires you to declare the language upfront.

Scribe v2 Realtime:

• Detects the language automatically from the audio
• Supports switching languages mid-conversation

Speaker: "Hello, how are you? ... Bonjour, comment ça va?"
System:   English ✓ ————————→ French ✓  (seamless switch)

🌍 This is critical for multilingual environments like international support lines or global meetings. Supports 90 languages.

---

Concept 8: Agentic Use Cases — Why All This Matters Together

An AI Agent (like a voice assistant) needs to:

1. Hear what you say → STT
2. Understand it instantly → low latency + accuracy
3. Respond naturally → fast enough to feel conversational

Scribe v2 Realtime is specifically optimized for this pipeline:

User speaks → [< 150ms] → Text → AI processes → Voice response

🤖 Without fast, accurate STT, the entire agent feels slow and robotic — even if the AI reasoning and voice synthesis are fast.

---

Concept 9: Accuracy in Noisy/Complex Conditions

Speed means nothing without accuracy. Key benchmarks:

• 93.5% accuracy across 30 European and Asian languages
• Tested on 500 hard samples with:
  • Background noise
  • Complex vocabulary/information

📊 This is described as "human-level understanding" — meaning it performs comparably to a human transcriber under difficult conditions.

---

Summary: How All Concepts Connect

Audio Input
    ↓
[VAD] — filters silence/noise
    ↓
[Language Detection] — identifies language automatically
    ↓
[Real-Time Transcription] — < 150ms latency
    ↓
[Negative Latency] — predicts next words
    ↓
[Text Conditioning] — maintains context if connection drops
    ↓
[Manual Commit] — you control when transcript is finalized
    ↓
Final Transcript → AI Agent / Captions / Meeting Tool

Each feature solves a specific real-world problem that makes live transcription difficult. Together, they enable natural, human-like real-time conversation with AI systems.