Use the publisher’s Transformers path for Voxtral Mini Realtime to transcribe a local recording, check the words against the audio, and keep the file off hosted APIs.
Hardware recommendations for mistralai-voxtral-mini-realtime
| Machine | Voxtral Mini Realtime |
|---|---|
| NVIDIA GPU 16 GB VRAM | 16 GB VRAM min per publisher vLLM streaming; 24 GB recommended for BF16 8.9 GB weights + runtime. |
| AMD GPU 16 GB VRAM | Guide covers CUDA/vLLM; AMD not documented for Voxtral. 16 GB VRAM estimate from 8.9 GB weights; verify backend. |
| Intel Arc GPU 16 GB VRAM | No Intel Arc route documented. 16 GB VRAM estimate from 8.9 GB BF16 weights; verify backend. |
| Apple Silicon 16 GB unified memory | 16 GB unified memory: MLX 4-bit alternative ~3.1 GB weights; official fp16 ~8.9 GB weights needs headroom. |
| CPU 16 GB system RAM | 16 GB system RAM estimate: 8.9 GB BF16 weights + runtime; CPU path slow, 32 GB headroom. |
Use the package's complete weight size, context, and runtime overhead when estimating capacity. Memory fit does not prove backend support.
A local-first transcription path for Mistral AI’s 4B realtime speech-to-text model, using the publisher’s Transformers example.
This setup uses the package documented for this task. Review its source and supported platforms before starting.
Save your machine in My Hardware to get an automatic starting choice. You can always choose any package yourself.
Confirm the complete runtime memory requirement before downloading.
Choose where you want to run it. The instructions below update to match your selection.
Use MLX or CPU-only Transformers.
Open Terminal. Ensure Homebrew is available, then install Python 3.11 or 3.12 plus ffmpeg.
Python Downloads/bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/HEAD/install.sh)"
brew install python@3.12 ffmpeg
python3.12 --version
ffmpeg -versionCreate a dedicated folder and virtual environment so the model and client files stay separate from system Python.
mkdir -p ~/YouRunAI-Voxtral-Realtime
cd ~/YouRunAI-Voxtral-Realtime
python3.12 -m venv .venv
source .venv/bin/activate
python -m pip install --upgrade pip setuptools wheelUse the publisher's documented packages: upgrade Transformers and install mistral-common with audio extras, plus soxr, librosa and soundfile.
Transformers installation Transformers repositorypip install --upgrade transformers
pip install --upgrade "mistral-common[audio]"
pip install soxr librosa soundfileCheck that the installed Transformers version contains VoxtralRealtimeForConditionalGeneration. If the import fails, upgrade Transformers again before continuing.
Voxtral config referencepython -c "import transformers; print(transformers.__version__)"
python -c "from transformers import VoxtralRealtimeForConditionalGeneration; print(VoxtralRealtimeForConditionalGeneration)"The publisher README uses the patrickvonplaten/audio_samples dataset file bcn_weather.mp3. Convert it to 16 kHz mono WAV because the model expects that sampling rate.
curl -L -o bcn_weather.mp3 "https://huggingface.co/datasets/patrickvonplaten/audio_samples/resolve/main/bcn_weather.mp3"
ffmpeg -i bcn_weather.mp3 -ar 16000 -ac 1 bcn_weather.wav
ls -lh bcn_weather.wavWrite a local Python script that loads the exact repository mistralai/Voxtral-Mini-4B-Realtime-2602, resamples audio to the processor sampling rate, runs greedy generation and writes the transcript to transcript.txt in the working folder. The script prints the transcript and saves it as transcript.txt. To transcribe your own file, pass its path after transcribe.py; the sample is only the first-run check.
Publisher README usagecat > transcribe.py <<'PY'
from transformers import VoxtralRealtimeForConditionalGeneration, AutoProcessor
from mistral_common.tokens.tokenizers.audio import Audio
import torch
import sys
repo_id = "mistralai/Voxtral-Mini-4B-Realtime-2602"
processor = AutoProcessor.from_pretrained(repo_id)
model = VoxtralRealtimeForConditionalGeneration.from_pretrained(repo_id, device_map="auto", torch_dtype=torch.bfloat16)
model.eval()
audio = Audio.from_file(sys.argv[1] if len(sys.argv) > 1 else "bcn_weather.wav", strict=False)
audio.resample(processor.feature_extractor.sampling_rate)
inputs = processor(audio.audio_array, return_tensors="pt")
inputs = inputs.to(model.device, dtype=model.dtype)
with torch.no_grad():
outputs = model.generate(**inputs)
decoded = processor.batch_decode(outputs, skip_special_tokens=True)
text = decoded[0]
print(text)
with open("transcript.txt", "w", encoding="utf-8") as f:
f.write(text)
PY
python transcribe.pyRun the script, listen to bcn_weather.mp3 or bcn_weather.wav while reading transcript.txt. The transcript should be a readable weather-related text in the spoken language, and transcript.txt should exist in the working folder. If memory is tight, shorten the input WAV or use the smaller MLX 4-bit path in the next step.
python transcribe.py
cat transcript.txt
open transcript.txtFor a lighter Apple Silicon path, use the MLX community converted Voxtral Mini Realtime 4-bit weights and mlx-audio. This transcribes the same base model but uses MLX instead of standard Transformers. The script writes transcript_mlx.txt.
MLX community fp16 model MLX community 4-bit modelpip install mlx-audio[stt]
cat > transcribe_mlx.py <<'PY'
from mlx_audio.stt.utils import load
model = load("mlx-community/Voxtral-Mini-4B-Realtime-2602-4bit")
result = model.generate("bcn_weather.wav", transcription_delay_ms=480)
print(result.text)
with open("transcript_mlx.txt", "w", encoding="utf-8") as f:
f.write(result.text)
PY
python transcribe_mlx.py
cat transcript_mlx.txt
open transcript_mlx.txtThe local script prints a readable transcript for a short, known recording, and you can verify names, numbers, and key phrases by listening to the source audio.
Voxtral Mini Realtime is a 4B speech-transcription model with a natively streaming architecture, configurable delays from 240 ms to 2.4 s, and support for 13 languages. The publisher’s example uses a CUDA GPU; MLX and GGUF alternatives exist for Apple silicon and other portable routes.
Confirm sample.mp3 exists beside the script and install the audio extras from the publisher instructions. Try a short standard MP3 or WAV that plays locally.
Use a larger CUDA GPU or rented GPU. The documented 16 GB minimum applies to vLLM; this Transformers example has no separately published memory minimum.
Check the source recording manually, use clearer input where possible, and review names and numbers before relying on the transcript.
Original instructions, model files, and compatibility notes behind this setup.
Source-backed package and platform-route plan for private local chat with Qwen3.5 · 9B. Preserves the official Ollama 9B package, a smaller 4B family fallback, and a clearly labeled community uncensored GGUF option.
Run Qwen3.8 27B with Ollama, connect it to Cline in VS Code, and make one reviewable code change in a tiny local project.
Generate images from text prompts in ComfyUI with the fast distilled or more adjustable base 4B model. Save your results and reuse the visual workflow.