IBM has released Granite Speech 5.0, a pair of compact 470-million-parameter automatic speech recognition (ASR) models capable of transcribing over 3.5 hours of audio in one second on modern datacenter silicon. In benchmark evaluations, IBM demonstrated aggregate throughput exceeding 12,600x real-time (12,600 RTFx) on a single NVIDIA H200 GPU.
The release includes two variants: Granite Speech 5.0 TurboCTC under the permissive Apache 2.0 license, and an extended research checkpoint licensed under CC-BY-NC-SA-4.0 for noncommercial use.
Conformer CTC Architecture and Frame Subsampling
Unlike previous Granite Speech iterations (versions 3.3 and 4.x), which integrated a Conformer acoustic encoder with an autoregressive language model via LoRA adapters, Granite Speech 5.0 abandons autoregressive language model decoding entirely. The 5.0 architecture consists of a 16-layer Conformer encoder trained with Connectionist Temporal Classification (CTC) for non-autoregressive, single-pass greedy decoding.
Key architectural mechanisms delivering the 20x throughput increase over earlier Granite Speech models include:
- Temporal Subsampling: Three consecutive 2x temporal subsampling stages reduce the 100-frames-per-second log-Mel spectrogram front end down to 12.5 tokens per second.
- Expanded Token Vocabulary: Moving from character-level outputs to a 16,384-subword vocabulary (Byte-Pair Encoding for Apache 2.0, SentencePiece for noncommercial) reduces the sequence length required to decode spoken phrases.
- Single-Pass Inference: Eliminating recurrent autoregressive token generation reduces memory bandwidth pressure during batched serving.
By removing the language model decoder, the architecture trades multi-task capabilities such as real-time speech translation and keyword biasing for maximal throughput and lightweight deployment footprints on local workstations and edge hardware.
Accuracy and Leaderboard Benchmarks
On public short-form English test suites evaluated via the OpenASR Leaderboard harness, IBM reports competitive error rates:
| Variant | License | Training Corpus | Aggregate Word Error Rate (WER) | | :--- | :--- | :--- | :--- | | Granite Speech 5.0 TurboCTC | Apache 2.0 | ~60,000 Hours | 5.00% | | Granite Speech 5.0 TurboCTC (NC) | CC-BY-NC-SA-4.0 | ~75,000 Hours | 4.85% |
The noncommercial variant includes additional training data from GigaSpeech and SPGI Speech, improving precision on specialized domain terminology while maintaining similar inference latency.
On the Far-Field ASR (FFASR) Leaderboard—which benchmarks acoustic robustness against noisy, reverberant environments at varying signal-to-noise ratios—the noncommercial checkpoint ranked 5th in accuracy and the Apache 2.0 model 9th, with both securing the fastest processing speeds on NVIDIA L4 hardware.
Training Hardware and Synthetic Currencies
The models were trained over 10 days on 8 NVIDIA H100 GPUs within IBM's Blue Vela compute cluster. The training mixture incorporated 2,740 hours of synthetic data, including 2,500 hours of concatenated multi-speaker audio and 240 hours of synthetic utterances generated via OpenAI open-weight models (gpt-oss) and synthesized using StyleTTS2 to improve numeric, currency, and address transcription accuracy.
Both model weights are available on Hugging Face with native support in Hugging Face Transformers and WebGPU browser-based streaming runtimes.
