whisper small
whisper-small is an Apache-2.0 licensed automatic speech recognition (ASR) model from openai/whisper-small, stored in safetensors format. It is a Transformer-based encoder-decoder (sequence-to-sequence) model trained on 680k hours of labeled speech data using large-scale weak supervision. According to the official model card, it supports multilingual speech recognition and speech translation; for ASR it predicts transcriptions in the same language as the audio, and for translation it predicts transcriptions in a different language. The model card lists many supported languages, including English, Chinese, German, Spanish, Russian, Korean, French, Japanese, Portuguese, Turkish, Polish, Catalan, Dutch, Arabic, and others. Reported model-index results include Test WER 3.432213777886737 on LibriSpeech (clean), Test WER 7.628304527060248 on LibriSpeech (other), Test WER 87.3 on Common Voice 11.0 (hi), and Test WER 125.69809089960707 on Common Voice 13.0 (dv).
At a glance
- Parameters
- 0.24B
- Architecture
- whisper
- License
- apache-2.0
- Disk space
- 0.9 GB
- Software
- Transformers
Will it run on your machine?
Save your machine to see a personalized rating and its reasoning.
Add my hardwareBest for
- Automatic speech recognition for audio in supported languages.
- Speech translation from audio in one language to text in a different language.
- Transcribing English audio, as evaluated on LibriSpeech clean and other test sets.
- Generalizing to many datasets and domains without fine-tuning, as stated in the model card.
- Multilingual speech processing across the model card's listed language set.
Tradeoffs
- The model card does not state performance, compatibility, or numerical hardware requirements beyond the provided facts.
- Reported WER is high for some evaluated languages, such as 87.3 on Common Voice 11.0 (hi) and 125.69809089960707 on Common Voice 13.0 (dv).
- The model was trained using large-scale weak supervision, which may affect transcription reliability in unreported conditions.
- No fine-tuning, testing, or performance claims beyond the model card should be inferred.
- The supplied source facts are partial and may omit other important limitations or requirements.
Ways to run it
Transformers · See official guide
Repository-specific example documented in the publisher model card; confirm dependencies and platform in the source.
>>> from transformers import WhisperProcessor, WhisperForConditionalGeneration
>>> from datasets import load_dataset
>>> # load model and processor
>>> processor = WhisperProcessor.from_pretrained("openai/whisper-small")
>>> model = WhisperForConditionalGeneration.from_pretrained("openai/whisper-small")
>>> model.config.forced_decoder_ids = None
>>> # load dummy dataset and read audio files
>>> ds = load_dataset("hf-internal-testing/librispeech_asr_dummy", "clean", split="validation")
>>> sample = ds[0]["audio"]
>>> input_features = processor(sample["array"], sampling_rate=sample["sampling_rate"], return_tensors="pt").input_features
>>> # generate token ids
>>> predicted_ids = model.generate(input_features)
>>> # decode token ids to text
>>> transcription = processor.batch_decode(predicted_ids, skip_special_tokens=False)
['<|startoftranscript|><|en|><|transcribe|><|notimestamps|> Mr. Quilter is the apostle of the middle classes and we are glad to welcome his gospel.<|endoftext|>']
>>> transcription = processor.batch_decode(predicted_ids, skip_special_tokens=True)
[' Mr. Quilter is the apostle of the middle classes and we are glad to welcome his gospel.']Explore its uses
Keep exploring
Voxtral Mini · 4B Realtime
Voxtral Mini 4B Realtime 2602 is a multilingual, realtime automatic speech recognition model from Mistral AI, released under the Apache-2.0 license. It features a natively streaming architecture with a causal audio encoder and sliding window attention, enabling configurable transcription delays from 240ms to 2.4s (and reportedly 80ms to 2.4s). The model has approximately 4B parameters (≈3.4B language model and ≈970M audio encoder) and is described as optimized for on-device deployment. It supports 13 languages and is presented as achieving accuracy comparable to offline systems with less than 500ms delay, specifically matching leading offline open-source models and realtime APIs at a 480ms delay. The model card recommends a temperature of 0.0, using websockets for audio streaming, and a delay of 480ms as a balance between performance and latency.
YuE2 · 3B
Create a full song from style and lyrics with a publisher Python package on a Linux NVIDIA GPU; outputs include a playable FLAC.
granite speech 5.0 470m turboctc
Granite Speech 5.0 470M TurboCTC is a compact 470 million parameter English automatic speech recognition model released by IBM under the Apache 2.0 license. It uses a conformer acoustic encoder with block self-attention, self-conditioning, and temporal downsampling, with an output layer of 16,384 BPE units. The model was trained on approximately 60,000 hours of English audio from public corpora using Connectionist Temporal Classification (CTC), and inference is non-autoregressive with greedy decoding. It is designed for low-latency and high-throughput English speech-to-text transcription, and is described as suitable for deployment on laptops, smartphones, and other edge devices. Native support is available in transformers>=5.16.0, with additional usage options for Apple Silicon via mlx-audio and local GGUF inference via transcribe.cpp.