Apple's SpeechAnalyzer, Measured Against Whisper on One Mac

Two point one two percent is the word error rate Apple's new on-device speech engine posted on the cleanest split of a benchmark that fed every engine the same 5,559 spoken sentences.

The measurement that arrived this week

On 2026-07-13, the transcription app maker Inscribe published a head-to-head comparison of Apple's new SpeechAnalyzer API, three sizes of OpenAI's Whisper, and Apple's aging SFSpeechRecognizer framework. Every engine ran fully on-device. Cloud fallback was explicitly refused, so the numbers reflect what runs on a user's own Mac and never leaves it.

The corpus was LibriSpeech, the standard open academic dataset of English audiobook read speech. Its test-clean split contains 2,620 utterances of clean read speech; its test-other split contains 2,939 harder, noisier utterances. Across every engine and both splits, Inscribe ran 27,795 total transcriptions on the same production code paths users hit inside its app. The test hardware was a single Apple M2 Pro Mac with 32 gigabytes of RAM running macOS 26.5.1.

Before scoring, text was normalized identically across engines: casing, punctuation, digits-to-words expansion, and contractions were all rewritten to a common form. Corpus word error rate was computed as total errors divided by total reference words, not averaged per-utterance, which prevents a handful of short mistakes from dominating an engine's score. Only one transcription failed across the entire sweep of 27,795 runs. It was the legacy SFSpeechRecognizer on the harder test-other split, and Inscribe counted the failure at 100 percent word error rather than hiding it.

Inscribe cross-checked its Whisper numbers against OpenAI's own published figures. Every measurement landed within 0.5 percentage points, which is the strongest indication available that the harness itself was faithful. Inscribe also released the raw output alongside the writeup: a 3-kilobyte summary.json plus two 620-kilobyte raw-transcript archives that let third parties rescore the same runs.

What SpeechAnalyzer replaced, and what Whisper still is

Apple has shipped SFSpeechRecognizer since 2016. It was accurate enough for Siri-length commands but visibly weak on long, noisy audio, and until this year it was the only Apple-provided way to transcribe speech on a Mac or iPhone without sending the audio to a server. Apple introduced SpeechAnalyzer in iOS 26 and macOS 26 as the successor framework, and Inscribe's benchmark is one of the first apples-to-apples measurements of what changed underneath.

The independent alternative for the last four years has been OpenAI's Whisper, released September 21, 2022, trained on 680,000 hours of labeled audio and covering 97 or more languages. Whisper is distributed under the MIT license, which is why WhisperKit and the wider ecosystem of CoreML ports exist at all. On Apple hardware, a developer who wanted meeting-grade accuracy without a cloud round-trip has generally reached for Whisper rather than Apple's own API.

On-device matters because the alternative carries real costs. Streaming a microphone to a hosted API incurs latency, dollars per minute, and, for meeting or medical notes, the disclosure of private speech to a third party. The question the benchmark answers is narrow but load-bearing: on the same machine, on the same audio, with the same normalization, which engine actually wins.

The numbers

Apple SpeechAnalyzer posted a word error rate of 2.12 percent on test-clean and 4.56 percent on test-other. Whisper Small, running through WhisperKit's CoreML port, posted 3.74 percent on test-clean and 7.95 percent on test-other. Whisper Base landed at 5.42 percent on test-clean and 12.51 percent on test-other. Whisper Tiny landed at 7.88 percent on test-clean and 17.04 percent on test-other. Apple's legacy SFSpeechRecognizer, on the same audio, posted 9.02 percent on test-clean and 16.25 percent on test-other.

The Whisper models are not the same size. Whisper Small ships as roughly a 460-megabyte model file. Whisper Base ships as roughly a 140-megabyte model file. Whisper Tiny ships as roughly a 40-megabyte model file. SpeechAnalyzer, by contrast, ships as part of the operating system. On the M2 Pro test machine, SpeechAnalyzer ran roughly three times faster than Whisper Small per second of audio processed, and Inscribe reports that every engine in the sweep ran between roughly 12 and 40 times real-time. An hour-long meeting, on that hardware, transcribes in about one and a half to five minutes without ever leaving the machine.

Inscribe's own reading of its numbers is direct. "Apple's new SpeechAnalyzer is the most accurate on-device speech engine we tested," the research writeup states. It also notes that "the new API cuts word error rate by 3.5 to 4 times on the same audio" compared with Apple's legacy engine. During the sweep, Inscribe found and fixed a shipping bug in its own SpeechAnalyzer file-import path: the code was missing the required finalizeAndFinishThroughEndOfInput() call, and the fix was rolled into the results.

What the benchmark does not settle

The corpus is English audiobook read speech. It is not meeting audio, not phone-call audio, and not conversational speech in any other language. The hardware is a single M2 Pro Mac running macOS 26.5.1. It is not an iPhone, an iPad, or an M-series chip other than the one tested. Apple's SpeechTranscriber module inside SpeechAnalyzer supports around 30 locales at launch. Whisper covers 97 or more languages and, being MIT licensed, runs on any hardware a developer can put it on.

Inscribe made its own product decision on the basis of these facts rather than presenting a single winner. After the benchmark, the company changed its Auto engine to prefer SpeechAnalyzer for the roughly 30 locales it supports and Whisper for every other language. The rationale is in the writeup itself: "Whisper keeps two real advantages: it covers far more languages, and it runs anywhere."

The narrower claim, and the one the numbers actually support, is the one Inscribe puts on the page: "For English, on Apple hardware, the built-in engine is now the strongest on-device option." Outside those two constraints, the benchmark does not speak. What it does speak to, cleanly, is that a decade-old default has been replaced by something measurably better on the audio it was tested on, and that the multilingual open-source alternative is still the right tool for most of the languages an application actually needs to serve.

Sources

• Inscribe on-device speech benchmark, 2026-07-13
• Apple SpeechAnalyzer framework documentation
• OpenAI Whisper research release, September 2022