Skip to content
Comparison

Transcription Accuracy Test 2026: ElevenLabs Scribe v2 vs Whisper (real WER data)

Our word-error-rate test on LibriSpeech test-clean: TranscribTxt's live pipeline (November 2026), ElevenLabs Scribe v2 and four Whisper sizes (June 2026).

By Serhii MaysonPublished Updated 8 min read

A card with five vertical bars of rising height, the shortest one highlighted: a transcription accuracy test
Across 100 utterances (2,111 reference words), Scribe v2 produced the fewest errors.

We ran our own accuracy benchmark instead of quoting a marketing number. On 100 clean-speech utterances from the standard LibriSpeech test-clean set, ElevenLabs Scribe v2 scored a 2.08% word error rate (97.9% accuracy), the lowest of every engine we tested, ahead of all four sizes of OpenAI Whisper. We ran it in June 2026. In November 2026 we ran this test through TranscribTxt itself, as one file on the live site: 2.84% WER (97.16% accuracy). Here is exactly how we measured both, so you can reproduce them.

Word Error Rate by transcription engine on LibriSpeech test-clean (June 2026) — ElevenLabs Scribe v2 2.08%, Whisper medium 2.89%, large-v3 3.41%, small 3.60%, base 5.31%

The results

EngineTestedWord Error RateWord accuracy
TranscribTxt, live pipeline (one 15-minute file)November 22, 20262.84%97.16%
ElevenLabs Scribe v2June 20262.08%97.92%
OpenAI Whisper mediumJune 20262.89%97.11%
OpenAI Whisper large-v3June 20263.41%96.59%
OpenAI Whisper smallJune 20263.60%96.40%
OpenAI Whisper baseJune 20265.31%94.69%

Lower WER is better. Across 100 utterances (2,111 reference words), Scribe v2 produced the fewest errors. The practical story is twofold: Scribe v2 is at least on par with the best open model, and it is clearly ahead of the small Whisper models (base, small) that quietly power a lot of "free" transcription tools.

The November 2026 rerun on TranscribTxt

On November 22, 2026 we ran the test through TranscribTxt itself, with 100 utterances picked by the same selection rule (every 26th utterance in name order, 2,111 words). We joined them, with a second of silence between each, into one 15-minute FLAC file and uploaded it on transcribtxt.com like any user's file, with no settings changed. Because it was one file, we scored the whole transcript against the 100 reference lines in order, with the same normalization as in June. The result: 60 errors in 2,111 words, a 2.84% word error rate.

Our first pass at this test found a real problem. The speech engine skipped one 25-second passage, a Shakespeare sonnet read aloud, so the transcript had a gap where it should have been. TranscribTxt now looks for gaps of more than 8 seconds between words in the transcript. When the recording has at least 3 seconds of sound in such a gap, TranscribTxt transcribes that part again and adds the words it finds, for up to three gaps in every 10 minutes of audio. The result above is the same file after that fix. Most of the 60 errors left are spelling and formatting differences rather than wrong words: "Mr" for "Mister," "2000" for "two thousand," "make-believe" with a hyphen, or "reinforcements" for the reference's old spelling "reenforcements."

The June rows and this one aren't a like-for-like ranking: in June each engine got 100 short clips one at a time, and in November TranscribTxt got one long file, the way you would upload a recording. See how we test for the full method.

Why we ran our own test

Every transcription tool claims to be "99% accurate." Almost none show the math. After Google's 2026 shift toward answering questions directly, the pages that earn trust — from readers and from AI search — are the ones built on first-party evidence, not recycled claims. So rather than repeat a number, we measured one.

This is also the honest version of advice we already give in our guide to the most accurate transcription software: on clean audio the leading models are close, and your recording quality usually matters more than the brand on the box.

Method (so you can reproduce it)

  • Dataset: LibriSpeech test-clean — a public benchmark of real human readers with ground-truth transcripts. We selected 100 utterances deterministically (evenly spread across speakers), totaling 2,111 reference words.
  • Engines: ElevenLabs Scribe v2, called through the ElevenLabs API; and OpenAI Whisper in four sizes (base, small, medium, large-v3) via faster-whisper, int8, beam size 5.
  • Metric: Word Error Rate computed with the jiwer library, after lowercasing, stripping punctuation, and collapsing whitespace — the standard normalization for ASR evaluation.
  • No cherry-picking: the utterance selection is fixed and the same audio went to every engine.
  • November 2026 rerun: 100 utterances picked by the same selection rule (every 26th of the set in name order), joined into one 15-minute file with a second of silence between them, uploaded to TranscribTxt's live site and scored as one text with the same normalization. The table shows the run after the gap fix described above.

Reading the numbers honestly

A few things worth stating plainly:

  • All of these are good. On clean read speech, every engine here lands between 94.7% and 97.9% accuracy. The difference between the top engines is a handful of words across thousands.
  • Medium beat large-v3. That looks backwards, but it is real: on short, clean clips Whisper large-v3 sometimes hallucinates or repeats words, nudging its error rate above the steadier medium model. Bigger is not always more accurate.
  • Clean audio is close to a best case. Background noise, multiple speakers, strong accents, and weak microphones make every engine worse. This benchmark measures the ceiling, not your podcast recorded in a café.
  • It is a 100-utterance sample. Big enough to be directional, not a peer-reviewed study. The honest claim is "Scribe v2 led our test and sits with the best models," not "Scribe v2 wins every recording."

What this means for you

On clean audio every engine here is good, and the gap between the leaders is a handful of words per thousand. If you have the technical skills and time, self-hosted Whisper large-v3 is free and excellent — just expect setup work and the occasional hallucination on clean clips.

Either way, test your own audio. Take a representative 2–3 minute clip, run it through our audio to text converter and one alternative, and read both transcripts against what was actually said. The most accurate tool for clean studio audio is not always the best for noisy field recordings — your sample is the only benchmark that matters for your work.

Try TranscribTxt free

Free for files up to 30 min: your first one needs no account, a free account gets 3 a week. 99 languages.

First 30 min free · the whole file (up to 3 h) US$2

Try it free

Frequently Asked Questions

How accurate is TranscribTxt?

In our November 22, 2026 run, 100 LibriSpeech test-clean utterances, picked by a fixed rule (every 26th utterance), went through TranscribTxt's live pipeline as one 15-minute file. The transcript had a 2.84% word error rate (97.16% word accuracy): 60 errors in 2,111 words, most of them differences in spelling or format, such as "Mr" for "Mister". Clean read speech is close to a best case, so test a few minutes of your own audio before you rely on any figure.

How accurate is AI transcription on clean speech?

In this June 2026 test on LibriSpeech test-clean, every engine landed between 94.7% and 97.9% word accuracy: ElevenLabs Scribe v2 led at 2.08% WER, and the four Whisper sizes ranged from 2.89% (medium) to 5.31% (base). On clean, clearly recorded speech the leading engines are close; noise, heavy accents and crosstalk lower accuracy for every tool. The reliable test is a 2–3 minute clip of your own audio.

What is Word Error Rate (WER)?

Word Error Rate is the standard accuracy metric for speech-to-text. It counts the words a transcript gets wrong — substitutions, insertions, and deletions — divided by the number of words actually spoken. A 2% WER means 2 errors per 100 words, or 98% accuracy. Lower is better. It is measured against a known-correct reference transcript after normalizing case and punctuation.

Was Scribe v2 more accurate than Whisper?

In this June 2026 test, yes — Scribe v2 at 2.08% WER beat every Whisper size, including large-v3 (3.41%). But all leading models cluster in the 95–98% range on clean audio, so the gap is small in absolute terms.

Why did Whisper medium beat Whisper large-v3 in your test?

On clean, short utterances, Whisper large-v3 is known to occasionally hallucinate extra words or repeat phrases, which raises its word error rate above the smaller, steadier medium model. This is a documented behavior, not a setup error — we used the same standard configuration (beam size 5, int8) for every Whisper size. It is a good reminder that bigger is not automatically more accurate.

Can I reproduce this benchmark?

Yes. We used the public LibriSpeech test-clean dataset, selected 100 utterances deterministically, transcribed each with ElevenLabs Scribe v2 and with faster-whisper (base, small, medium, large-v3, int8, beam 5), normalized case and punctuation, and computed WER with the jiwer library against the ground-truth transcripts. For the November 2026 rerun we joined 100 utterances, picked by the same rule, into one file and uploaded it to TranscribTxt like any user's file. Anyone can run the same dataset through the same tools and check the numbers.

Does this mean I'll get 98% accuracy on my audio?

Not necessarily. This test uses clean, single-speaker read speech, which is close to a best case. Real-world recordings with background noise, multiple speakers, accents, or poor microphones will be harder for every engine. The reliable way to know is to test your own audio: run a representative 2–3 minute clip and read the transcript against what was actually said.

Back to all guides