Whisper User Guide

Whisper is an open-source automatic speech recognition (ASR) model from OpenAI that transcribes speech to text and can also translate non-English speech directly into English. It ships as a Python package with a one-shot CLI (whisper) and a small Python API (whisper.load_model, model.transcribe(...)). All inference runs locally — your audio never leaves your machine.

This guide is the practical "how do I use it" reference for end users: researchers transcribing a corpus, developers wiring ASR into an app, and hobbyists subtitling their podcasts. For architecture, deployment, and internals, see the Whisper Administrator & Developer Guide.

1. Quick Start

The easiest install is a single pip command, plus FFmpeg on your system PATH. Once those are in place, whisper works on any file FFmpeg can decode.

pip install -U openai-whisper
whisper audio.mp3

That command downloads the default model (turbo) on first run, transcribes audio.mp3, and writes audio.txt, audio.srt, audio.vtt, audio.tsv, and audio.json to the current directory.

Tip

The turbo model is the new default — it is roughly as accurate as large while being ~8x faster. Drop it in for a one-liner: whisper audio.mp3.

Prerequisites

• Python 3.8+ with pip.
• FFmpeg on PATH. Whisper shells out to ffmpeg to decode every input.
• PyTorch. Installed automatically as a dependency, but for GPU use you may want a CUDA-matched build (see Installation).
• Disk space for model checkpoints (39 MB to ~3 GB depending on model).

A 30-second tour

# 1. Transcribe with defaults (model=turbo, all output formats)
whisper meeting.wav

# 2. Pin a small model and force English
whisper meeting.wav --model small.en --language en

# 3. Translate French audio into English text
whisper paris_interview.mp3 --language French --task translate

# 4. Word-level timestamps with karaoke-style highlighted SRT
whisper song.mp3 --word_timestamps True --highlight_words True --output_format srt

Note

First run for any new model name pulls the checkpoint from openaipublic.azureedge.net into ~/.cache/whisper/. Plan for ~140 MB (base) up to ~2.9 GB (large-v3).

What to try next

• Section 2 — install on a fresh machine (Linux, macOS, Windows, Docker).
• Section 4 — the full CLI, the model table, and the Python API.
• Section 5 — recipes for batching, GPU, prompt-conditioning, and more.

2. Installation

2.1 Install FFmpeg first

Whisper's load_audio() calls the ffmpeg CLI to decode inputs. Without it, every transcription fails with Failed to load audio: ....

# Ubuntu / Debian
sudo apt update && sudo apt install -y ffmpeg

# macOS (Homebrew)
brew install ffmpeg

# Windows (Chocolatey)
choco install ffmpeg

# Windows (Scoop)
scoop install ffmpeg

# Arch Linux
sudo pacman -S ffmpeg

# Verify
ffmpeg -version

Important

If you see Failed to load audio: ... 'ffmpeg' is not recognized (or command not found), Whisper cannot find FFmpeg on PATH. Reopen your shell after installing, or set PATH manually.

2.2 Install the package

The release on PyPI is the recommended path:

pip install -U openai-whisper

For the absolute latest commit on main (bug fixes that have not been released yet):

pip install -U --no-deps --force-reinstall \
  git+https://github.com/openai/whisper.git

For a development checkout (you want to read or modify the code):

git clone https://github.com/openai/whisper.git
cd whisper
pip install -e .

2.3 PyTorch / CUDA notes

pip install openai-whisper pulls the default PyTorch wheel. For dedicated NVIDIA GPU acceleration, install a CUDA-matched PyTorch build first, then install Whisper:

# Example: CUDA 12.1 build of PyTorch
pip install torch --index-url https://download.pytorch.org/whl/cu121
pip install -U openai-whisper

# Verify CUDA is visible
python -c "import torch; print(torch.cuda.is_available(), torch.cuda.device_count())"

Note

Whisper auto-selects cuda if torch.cuda.is_available() is true, otherwise cpu. Override with --device cpu, --device cuda, or a specific device like --device cuda:1.

2.4 Optional: Triton for word-timestamp acceleration

Word-level timestamps run a dynamic time warping (DTW) pass on cross-attention weights. On Linux/x86_64 with a GPU, the package can JIT a Triton CUDA kernel that is faster than the CPU fallback. Triton is listed in requirements.txt as triton>=2.0.0 (Linux/x86_64 only) — pip install openai-whisper will install it automatically on supported platforms. On Windows or Apple Silicon, the CPU numba DTW path is used instead.

Warning

If you see triton import failed; try 'pip install --pre triton', Triton was requested but cannot be loaded. Whisper continues to work — it falls back to a slower median-filter and DTW implementation in pure NumPy/Numba.

2.5 Verify the install

python -c "import whisper; print(whisper.__version__); print(whisper.available_models())"

20250625
['tiny.en', 'tiny', 'base.en', 'base', 'small.en', 'small', 'medium.en', 'medium', 'large-v1', 'large-v2', 'large-v3', 'large', 'large-v3-turbo', 'turbo']

The whisper CLI also installs as a console script:

whisper --help

3. Configuration

Whisper has no config file. All behaviour is set via CLI flags or via the keyword arguments to whisper.transcribe(...). The only environment variable the package reads is XDG_CACHE_HOME, which is honoured when computing the default model download directory (~/.cache/whisper/).

3.1 Most-used flags

3.2 Model storage

Models are cached at ~/.cache/whisper/<model>.pt by default. Override per invocation with --model_dir /path/to/dir (CLI) or whisper.load_model(name, download_root="/path/to/dir") (API).

# Keep models on a different disk
export XDG_CACHE_HOME=/data/caches
whisper audio.mp3 --model medium
# checkpoint lands in /data/caches/whisper/medium.pt

Tip

Pre-download a model once (e.g. on a build machine) and copy the *.pt files into the cache directory of every other machine. Whisper verifies a SHA-256 over the file before loading, so corrupted copies are rejected.

3.3 Output formats

Whisper writes to disk through five named writers. The CLI's --output_format maps directly to these:

The full configuration reference (every keyword arg to transcribe(), every CLI flag, every default) lives in the Administrator & Developer Guide, Section 3.

4. Core Usage

4.1 The CLI

whisper [-h] [--model MODEL] [--model_dir MODEL_DIR] [--device DEVICE]
        [--output_dir OUTPUT_DIR] [--output_format {txt,vtt,srt,tsv,json,all}]
        [--verbose VERBOSE] [--task {transcribe,translate}]
        [--language LANGUAGE] [--temperature TEMPERATURE]
        [--best_of BEST_OF] [--beam_size BEAM_SIZE] [--patience PATIENCE]
        [--length_penalty LENGTH_PENALTY] [--suppress_tokens SUPPRESS_TOKENS]
        [--initial_prompt INITIAL_PROMPT] [--carry_initial_prompt CARRY_INITIAL_PROMPT]
        [--condition_on_previous_text CONDITION_ON_PREVIOUS_TEXT] [--fp16 FP16]
        [--temperature_increment_on_fallback TEMPERATURE_INCREMENT_ON_FALLBACK]
        [--compression_ratio_threshold COMPRESSION_RATIO_THRESHOLD]
        [--logprob_threshold LOGPROB_THRESHOLD]
        [--no_speech_threshold NO_SPEECH_THRESHOLD]
        [--word_timestamps WORD_TIMESTAMPS]
        [--prepend_punctuations PREPEND_PUNCTUATIONS]
        [--append_punctuations APPEND_PUNCTUATIONS]
        [--highlight_words HIGHLIGHT_WORDS]
        [--max_line_width MAX_LINE_WIDTH] [--max_line_count MAX_LINE_COUNT]
        [--max_words_per_line MAX_WORDS_PER_LINE] [--threads THREADS]
        [--clip_timestamps CLIP_TIMESTAMPS]
        [--hallucination_silence_threshold HALLUCINATION_SILENCE_THRESHOLD]
        audio [audio ...]

The first positional argument is one or more audio files. Everything else is optional. Run whisper --help for the in-tool listing including default values.

4.2 Models — pick the right size

Speeds are measured against large on an A100 with FP16. Real numbers vary with batch size, sequence length, and disk speed. Use .en variants for English-only audio — they tend to be slightly more accurate at the smaller sizes because they were trained without the multilingual capacity tax.

Note

When you ask for a .en model with --language set to anything other than en / English, Whisper logs a warning and forces English. The English-only models simply don't have the other language tokens.

4.3 Language detection vs. forced language

By default, multilingual models examine the first 30 seconds of audio and pick the most probable language. To skip detection, pass --language:

whisper interview.mp3 --language ja            # Japanese
whisper interview.mp3 --language Japanese      # also accepted
whisper interview.mp3 --language fr            # French

Whisper supports 99 languages in large-v3 / turbo (and 100 in large-v2). Common codes: en, es, fr, de, it, pt, ru, zh, ja, ko, ar, hi. Language names like "English", "Mandarin", "Castilian" are also accepted.

4.4 Transcribe vs. translate

Whisper has two tasks built into a single model:

• --task transcribe (default) — output text in the same language as the audio (X→X).
• --task translate — output English text regardless of input language (X→English). Translation is always to English; there is no "translate to Spanish" mode.

# Native French transcript
whisper paris.mp3 --language fr

# English translation of the same French audio
whisper paris.mp3 --language fr --task translate

4.5 The Python API

The CLI is a thin wrapper around three calls. Use them directly when you want results in-process or want to reuse a loaded model across many files.

import whisper

model = whisper.load_model("turbo")            # download/cache + load
result = model.transcribe("audio.mp3")
print(result["text"])

The result is a dict with three keys:

Each segment dict looks like:

{
    "id": 0,
    "seek": 0,
    "start": 0.0,
    "end": 5.32,
    "text": " Good evening, ladies and gentlemen.",
    "tokens": [50364, 2205, 6432, 11, 9555, 293, 16924, 13, 50630],
    "temperature": 0.0,
    "avg_logprob": -0.18,
    "compression_ratio": 1.51,
    "no_speech_prob": 0.0014,
    "words": [],   # populated only when word_timestamps=True
}

Three top-level functions you'll use:

whisper.load_model(name, device=None, download_root=None, in_memory=False)
whisper.transcribe(model, audio, **opts)         # also model.transcribe(audio, **opts)
whisper.decode(model, mel, options=DecodingOptions(), **kwargs)

load_model accepts any name from whisper.available_models() or a path to a .pt checkpoint. transcribe accepts a file path, a NumPy array, or a torch.Tensor of 16 kHz mono float32 samples. decode is the lower-level single-30-second-window primitive (see Admin Guide, Section 8 for when to use it).

4.6 Decoding parameters that actually matter

Threshold-based fallback (re-decode at higher temperature when output looks bad) is controlled by:

Important

beam_size and best_of are mutually exclusive in a single call. Whisper raises ValueError("beam_size and best_of can't be given together"). Use beam_size when temperature == 0, best_of when temperature > 0.

5. Common Tasks

5.1 Transcribe one file with sensible defaults

whisper podcast.mp3 --model turbo --output_dir transcripts

Outputs transcripts/podcast.{txt,srt,vtt,tsv,json}.

5.2 Transcribe a long file with timestamps and progress

--verbose True (the default) prints each segment as it is decoded, so you get an obvious progress signal on long jobs. Use --verbose False if you prefer a single tqdm progress bar.

whisper lecture_2h.wav --model medium --verbose False

5.3 Translate non-English audio into English

whisper interview_es.mp3 --language es --task translate \
  --output_format txt --output_dir english_translations

Note

Word-timestamps on translations are unreliable — Whisper warns Word-level timestamps on translations may not be reliable. because the alignment heads are calibrated against transcribed (same-language) tokens.

5.4 Batch a whole directory

The CLI accepts multiple positional arguments:

whisper recordings/*.mp3 --model small.en --output_dir transcripts

If a single file fails, Whisper prints the traceback and continues with the rest:

Skipping recordings/corrupt.mp3 due to RuntimeError: Failed to load audio: ...

5.5 Force a language to skip auto-detection

Auto-detection looks at the first 30 seconds and is wrong on short files that start with music or silence. Pin it:

whisper short_clip.mp3 --language en

The same clip without --language may be detected as welsh or latin if the opening 30 seconds are non-speech.

5.6 Word-level timestamps + karaoke-style SRT

whisper song.mp3 \
  --word_timestamps True \
  --highlight_words True \
  --max_line_width 42 \
  --max_line_count 2 \
  --output_format srt

--highlight_words underlines the currently-spoken word inside each subtitle line (<u>...</u> in SRT/VTT), so video players that respect inline styling render karaoke-style highlighting.

Tip

--max_words_per_line is an alternative to --max_line_width — it segments by token count instead of character count. They cancel out if both are set; pick one.

5.7 GPU vs. CPU and FP16/FP32

# Force GPU 1 in a multi-GPU box
whisper audio.wav --device cuda:1

# Force CPU (e.g. on a headless box without CUDA)
whisper audio.wav --device cpu --fp16 False

On CPU, FP16 is silently disabled (with a warning) — the matrix kernels in PyTorch don't support FP16 on most CPU backends. On GPU, FP16 is the default and 1.5–2x faster than FP32 with negligible accuracy difference.

5.8 Prompt-condition the decoder for custom vocabulary

whisper meeting.wav --initial_prompt "Discussion about Kubernetes, kubelet, etcd, and CoreDNS."

The prompt does not prepend to the output — it is fed to the decoder as context so spellings and proper nouns are biased toward your domain. Use it to nail names and acronyms (Acme Corp, Llama 3, kubectl).

Warning

Prompts longer than half the model's text-context window are truncated from the left. Keep them under ~100 words. Use --carry_initial_prompt True if you want the prompt re-prepended to every 30-second window instead of just the first one.

5.9 Transcribe only specific clips of a long file

# Just 0:30–1:15 and 5:00–6:00
whisper podcast.mp3 --clip_timestamps "30,75,300,360"

Pairs of start,end seconds. Whisper skips the audio between pairs.

5.10 Stream-style processing with the Python API

For a long-running app that processes many files without paying the load-model cost each time, hold the model resident:

import whisper

model = whisper.load_model("turbo", device="cuda")

for path in get_inbox():
    result = model.transcribe(
        path,
        language="en",
        word_timestamps=True,
        initial_prompt="Acme Corp engineering retro.",
    )
    save_to_db(path, result["text"], result["segments"])

To reach below transcribe() for tighter loops on 30-second snippets, see the Admin Guide's section on whisper.decode and the DecodingOptions dataclass.

6. Troubleshooting

Error: Failed to load audio: ... 'ffmpeg' (or ffmpeg: command not found)

Cause: The ffmpeg binary is not on PATH. Whisper shells out to it for every input.

Solution:

1. Install FFmpeg using the Section 2.1 instructions for your OS.
2. Open a fresh shell so the new PATH is picked up.
3. Confirm with ffmpeg -version.

Prevention: Bake ffmpeg into your container/AMI alongside Python.

Error: Model {name} not found; available models = [...]

Cause: The --model argument is neither in whisper.available_models() nor a path to an existing .pt checkpoint.

Solution:

1. Check the spelling. Valid names: tiny, tiny.en, base, base.en, small, small.en, medium, medium.en, large-v1, large-v2, large-v3, large, large-v3-turbo, turbo.
2. Pass an absolute path if you have a custom checkpoint.

Prevention: Run whisper --help — --model shows the choices for the installed version.

Error: RuntimeError: Model has been downloaded but the SHA256 checksum does not not match.

Cause: A model download was truncated or corrupted, or the file at ~/.cache/whisper/<model>.pt was overwritten by something else. Whisper verifies SHA-256 against the hash baked into the URL.

Solution:

1. Delete the bad file: rm ~/.cache/whisper/<model>.pt.
2. Re-run; Whisper re-downloads it.

Prevention: Don't manually edit checkpoints in the cache directory.

Error: beam_size and best_of can't be given together

Cause: You set both beam_size (for greedy temperature=0 beam search) and best_of (for sampling at temperature>0). Only one is valid for any single decode call.

Solution: Set beam_size for temperature == 0, best_of for temperature > 0. Drop the other.

Prevention: The default-temperature fallback strategy (the API tuple (0.0, 0.2, 0.4, 0.6, 0.8, 1.0)) handles this for you — it drops beam_size/patience for t > 0 and best_of for t == 0 automatically.

Error: triton import failed; try 'pip install --pre triton'

Cause: The triton_ops module loaded but the triton package itself failed to import. Triton is only packaged for Linux/x86_64.

Solution:

1. On Linux/x86_64: pip install --pre triton. If that also fails, your GPU/driver may be unsupported.
2. On Windows or macOS: this is expected — Whisper falls back to a CPU implementation. Word timestamps are slower but still work.

Prevention: None needed; the fallback is automatic.

Warning: Performing inference on CPU when CUDA is available

Cause: You loaded the model with device="cpu" (or --device cpu) on a machine that has a CUDA GPU. Whisper warns because GPU is roughly 30–100x faster.

Solution: Drop --device cpu (Whisper will pick cuda automatically), or set --device cuda explicitly.

Prevention: Don't set --device unless you have a reason.

Output is gibberish / repetitive / stuck in a loop

Cause: Whisper's decoder occasionally falls into a degenerate loop on silence, music, or out-of-distribution audio. Symptoms include the same phrase repeating, ratios like "... ... ... ...", or text from a hallucinated language.

Solution:

1. Try a larger model: tiny/base hallucinate more than medium/large.
2. Pin language with --language (auto-detect can mis-fire on quiet starts).
3. Lower the failure thresholds: --logprob_threshold -1.0 (default) is the right starting point — try -0.5 to be stricter.
4. Disable carry-over with --condition_on_previous_text False if loops are spreading across windows.
5. Enable hallucination skipping: --word_timestamps True --hallucination_silence_threshold 2.0.

Prevention: Pre-process with VAD (e.g. silero-vad) to strip leading/ trailing silence and music.

Long files get cut off mid-sentence at 30-second boundaries

Cause: Whisper processes audio in 30-second windows. With condition_on_previous_text=True (default) the model usually carries context across windows, but timestamp drift can occasionally cause boundary issues.

Solution: Make sure you are using a recent model (large-v3, turbo) and --condition_on_previous_text True. If the issue persists, try --word_timestamps True so segmentation is driven by word-level alignment rather than the model's predicted timestamps.

Prevention: None — this is inherent to chunked processing.

CPU run is unbearably slow

Cause: Whisper on CPU is bottlenecked by single-threaded matrix multiplies in PyTorch.

Solution:

1. Use --threads N to enable multithreaded MKL (e.g. --threads 8).
2. Drop to a smaller model: tiny.en is ~32x faster than large and good enough for clean audio.
3. Move to GPU — even a low-end consumer GPU is dramatically faster.

Prevention: Plan compute. CPU is fine for tiny/base/small; not for medium/large.

UnicodeEncodeError or garbled console output on Windows

Cause: The Windows default code page (often cp1252) cannot represent the characters Whisper is printing.

Solution: Whisper applies a make_safe() round-trip when the system encoding is not UTF-8, replacing un-encodable characters with ?. If you still see crashes, set the console to UTF-8: chcp 65001 (cmd) or $env:PYTHONIOENCODING="utf-8" (PowerShell). Setting PYTHONIOENCODING=utf-8 in your environment is the most reliable fix.

Prevention: Use Windows Terminal with UTF-8 enabled.

Disk fills up with model checkpoints

Cause: Each model size you experiment with is permanently cached at ~/.cache/whisper/.

Solution: Delete unused ones — ls -lh ~/.cache/whisper/. Each large model is ~3 GB.

Prevention: Pick one model and stick with it for batch jobs.

7. FAQ

Q: Which model should I use?

For most users: turbo (alias for large-v3-turbo). It is roughly as accurate as large-v3 but ~8x faster and uses ~6 GB of VRAM. If you are running on CPU, drop to small.en (English) or small (multilingual) — larger models are too slow without a GPU. For maximum accuracy at any cost, use large-v3.

Q: What audio formats does Whisper support?

Anything FFmpeg can decode — MP3, WAV, FLAC, M4A, OGG, OPUS, WMA, AAC, WebM, MP4 (audio extracted), and more. Whisper resamples internally to 16 kHz mono float32, so you don't need to pre-convert.

Q: Does Whisper need an internet connection?

Only on the first run for each model — the checkpoint is downloaded once into ~/.cache/whisper/. After that it works fully offline. Pre-populate the cache for air-gapped deployments by copying the *.pt files.

Q: Is my audio sent to OpenAI?

No. The Whisper open-source package runs the model entirely locally. There are no telemetry or network calls beyond the one-time checkpoint download.

Q: What is the difference between large, large-v3, and turbo?

large-v1, large-v2, and large-v3 are successive generations of the 1.55B-parameter multilingual model, each retrained on more data. large without a suffix is an alias for large-v3. turbo (alias for large-v3-turbo) is a distilled 809M-parameter model — faster and uses less memory for nearly the same accuracy.

Q: Why does Whisper sometimes invent text during silence?

The decoder is a generative language model and can hallucinate when given non-speech input. Mitigations: pre-trim with VAD, set --no_speech_threshold (default 0.6), and enable --word_timestamps True --hallucination_silence_threshold 2.0 to drop silent regions adjacent to suspected hallucinations.

Q: Can I fine-tune Whisper?

The official OpenAI repo does not include fine-tuning code, only inference. Community fine-tuning recipes exist (typically on top of Hugging Face's transformers port of the same checkpoint), but they are out of scope for this package.

Q: Why are my word timestamps unreliable on translations?

The alignment heads are calibrated against same-language transcription tokens. For --task translate, the aligned tokens are English while the audio is non-English, so the dynamic-time-warping pass produces less trustworthy timings. Whisper logs a warning when both are enabled.

Q: How do I run Whisper without a GPU?

pip install -U openai-whisper works on CPU-only machines. Use a smaller model (tiny/base/small) and pass --device cpu --fp16 False (the package auto-disables FP16 on CPU and warns once). Expect ~5–10x real-time on small.en with 4 threads, slower for larger models.

Q: How do I get verbatim text without timestamps?

whisper audio.mp3 --output_format txt writes only the text lines. Programmatically, just use result["text"] from model.transcribe(...).

Q: What languages are supported?

99 languages in large-v3 and turbo, 100 in large-v2. Common ones: English, Mandarin Chinese, Spanish, French, German, Italian, Portuguese, Russian, Japanese, Korean, Arabic, Hindi. The full list is in whisper.tokenizer.LANGUAGES. Pass a code (fr) or a name (French, Castilian, Mandarin) to --language.

Q: Why does the first run take so long?

It downloads the model weights (39 MB to ~3 GB depending on which model) and unpacks them into PyTorch. Subsequent runs load from the local cache in seconds.

Q: Can I run Whisper from inside a Jupyter notebook?

Yes. import whisper; whisper.load_model("turbo").transcribe("audio.mp3") works the same way. The repository includes example notebooks at notebooks/LibriSpeech.ipynb and notebooks/Multilingual_ASR.ipynb.