whisper.cpp Admin & Developer Guide

This guide is for engineers, architects, and operators who integrate, build, deploy, or extend whisper.cpp. It covers the system architecture, the complete build-option matrix across every acceleration backend, the C API, the HTTP server's operational characteristics, model conversion and quantization, security considerations, and the project's internal structure.

For installation basics and day-to-day CLI usage, see the companion User Guide.

Table of Contents

1. Executive Summary
2. Architecture
3. Repository Structure
4. System Requirements
5. Build System & Configuration Reference
6. Acceleration Backends
7. The C API (whisper.h)
8. Models: Format, Conversion, Quantization
9. Operating whisper-server
10. Language Bindings
11. Security Analysis
12. Code Quality & Conventions
13. Testing & CI
14. Potential Improvements
15. Troubleshooting (Operational)
16. Quick Reference

1. Executive Summary

whisper.cpp is a plain C/C++ port of OpenAI's Whisper ASR model, built on the ggml tensor library. Its design goals are portability, zero runtime dependencies, and broad hardware acceleration.

Key architectural properties:

• The entire high-level model is implemented in just two files — include/whisper.h (the C API) and src/whisper.cpp (the implementation). Everything else is either the ggml backend or example programs.
• All heavy numerical work is delegated to ggml, which provides a pluggable backend abstraction. The same model file runs unchanged on CPU, Metal, CUDA, Vulkan, ROCm/HIP, SYCL, OpenVINO, Core ML, BLAS, CANN, MUSA, and more — the backend is selected at compile time.
• Memory is pre-allocated; the design advertises "zero memory allocations at runtime" during inference.
• The C API is thread-safe per whisper_context as long as a single context is not used by multiple threads concurrently.

2. Architecture

2.1 Inference pipeline

                       whisper.cpp inference pipeline
 ┌──────────┐   ┌───────────────┐   ┌───────────┐   ┌───────────┐   ┌────────┐
 │  Audio   │──▶│ Log-Mel       │──▶│ Encoder   │──▶│ Decoder   │──▶│  Text  │
 │ 16kHz PCM│   │ spectrogram   │   │ (Transf.) │   │ (Transf.) │   │segments│
 └──────────┘   └───────────────┘   └───────────┘   └───────────┘   └────────┘
       │              │                   │                │
   miniaudio /    whisper_pcm_to_mel  whisper_encode   whisper_decode
   ffmpeg         (n_mel = 80/128)    (audio ctx)      (autoregressive,
   decode                                               beam / greedy)

1. Audio load — example programs decode files with miniaudio (or FFmpeg if built in) into 16 kHz mono (or stereo) 32-bit float PCM.
2. Mel spectrogram — whisper_pcm_to_mel() converts PCM to a log-mel spectrogram using the model's embedded mel filters (WHISPER_N_FFT=400, WHISPER_HOP_LENGTH=160, 30-second WHISPER_CHUNK_SIZE).
3. Encoder — a Transformer encoder produces audio features. Optionally offloaded to Core ML (Apple Neural Engine) or OpenVINO.
4. Decoder — a Transformer decoder autoregressively emits text tokens, using greedy or beam-search sampling, conditioned on prior context, an optional initial prompt, and an optional GBNF grammar.
5. Post-processing — token streams are grouped into timestamped segments, optionally with token-level timestamps (DTW), confidence values, and speaker-turn markers.

2.2 Layered design

 ┌─────────────────────────────────────────────────────────────┐
 │  Example programs / apps                                      │
 │  whisper-cli · whisper-server · whisper-stream · command ·    │
 │  bench · quantize · talk-llama · mobile (objc/swiftui/android)│
 ├─────────────────────────────────────────────────────────────┤
 │  Public C API  —  include/whisper.h                           │
 │  whisper_init_* · whisper_full · whisper_vad_* · accessors    │
 ├─────────────────────────────────────────────────────────────┤
 │  Core implementation  —  src/whisper.cpp (+ coreml/openvino)  │
 ├─────────────────────────────────────────────────────────────┤
 │  ggml tensor library  —  ggml/ (compute graph + backends)     │
 │  CPU · Metal · CUDA · Vulkan · HIP · SYCL · BLAS · CANN ·      │
 │  MUSA · OpenVINO · WebGPU · OpenCL · RPC · zDNN · Hexagon      │
 └─────────────────────────────────────────────────────────────┘

2.3 The encoder offload paths

The encoder is the most compute-intensive part and has two optional alternative implementations:

• Core ML (src/coreml/) — on Apple Silicon, the encoder can run on the Neural Engine, reportedly >3× faster than CPU. Enabled with WHISPER_COREML. The first run compiles the model to a device-specific format and is slow; subsequent runs are fast.
• OpenVINO (src/openvino/) — on Intel CPUs/GPUs, the encoder can run via OpenVINO. Enabled with WHISPER_OPENVINO. Initialized at runtime via whisper_ctx_init_openvino_encoder(); the IR model path defaults next to the ggml model.

Both are no-ops if not compiled in, so code that calls them stays portable.

3. Repository Structure

whisper.cpp/
├── include/whisper.h          # Public C API (single header)
├── src/
│   ├── whisper.cpp            # Core model implementation
│   ├── coreml/                # Core ML encoder (Apple ANE)
│   └── openvino/              # OpenVINO encoder (Intel)
├── ggml/
│   ├── include/               # ggml public headers (ggml.h, ggml-backend.h, ...)
│   └── src/                   # ggml + every backend (ggml-cuda, ggml-metal,
│                              #   ggml-vulkan, ggml-sycl, ggml-cann, ggml-musa,
│                              #   ggml-hip, ggml-blas, ggml-opencl, ggml-rpc, ...)
├── examples/
│   ├── cli/                   # whisper-cli (main tool)
│   ├── server/                # whisper-server (HTTP API + vendored httplib)
│   ├── stream/                # whisper-stream (SDL2 live mic)
│   ├── command/               # whisper-command (SDL2 voice assistant)
│   ├── bench/                 # whisper-bench
│   ├── quantize/              # quantize tool
│   ├── talk-llama/            # voice chat with LLaMA
│   ├── vad-speech-segments/   # VAD segment extractor
│   ├── whisper.objc/ .swiftui/ .android/ .wasm/   # platform demos
│   └── ...                    # lsp, wchess, addon.node, python, nvim, etc.
├── bindings/
│   ├── go/  java/  ruby/  javascript/
├── models/                    # download/convert scripts, model docs
├── scripts/                   # bench.py, apple build scripts, helpers
├── tests/                     # earnings21 / librispeech WER harnesses
├── cmake/                     # toolchain files
├── CMakeLists.txt             # top-level build
└── build-xcframework.sh       # Apple XCFramework build

Note

The high-level model logic is intentionally tiny (whisper.h + whisper.cpp). The bulk of the source tree is the ggml backend matrix and the example/binding ecosystem. When integrating, you only link against the core library and ggml.

4. System Requirements

Build toolchain

Optional toolchains by backend

Runtime memory/disk

See the model table in the User Guide: from tiny (75 MiB disk / ~273 MB RAM) to large (2.9 GiB / ~3.9 GB).

5. Build System & Configuration Reference

whisper.cpp uses CMake. The canonical flow:

cmake -B build [options]
cmake --build build -j --config Release

5.1 Top-level whisper.cpp (WHISPER_*) options

From the root CMakeLists.txt:

Important

Deprecated WHISPER_* acceleration flags. Older backend flags now live under the GGML_* namespace and are remapped at configure time (some with a fatal error). Use the new names:

Deprecated	Replacement
WHISPER_CUBLAS	GGML_CUDA (now fatal — must update)
WHISPER_CUDA	GGML_CUDA
WHISPER_METAL	GGML_METAL
WHISPER_METAL_EMBED_LIBRARY	GGML_METAL_EMBED_LIBRARY
WHISPER_KOMPUTE	GGML_KOMPUTE
WHISPER_NATIVE	GGML_NATIVE
WHISPER_OPENMP	GGML_OPENMP
WHISPER_RPC	GGML_RPC
WHISPER_SYCL / WHISPER_SYCL_F16	GGML_SYCL / GGML_SYCL_F16
WHISPER_CCACHE	GGML_CCACHE
WHISPER_FFMPEG	WHISPER_COMMON_FFMPEG

5.2 CMake presets

The repo ships CMakePresets.json with ready-made configurations. Highlights:

• Per-platform toolchains: x64-linux-gcc-*, arm64-apple-clang-*, x64-windows-msvc-*, x64-windows-llvm-*, arm64-windows-llvm-*.
• Backend presets: x64-windows-sycl-* (with optional -f16), x64-windows-vulkan-*.
• Build types are composed from hidden presets: debug (Debug), release (Release), reldbg (RelWithDebInfo), static (GGML_STATIC=ON).

Use one with:

cmake --preset x64-linux-gcc-release
cmake --build build-x64-linux-gcc-release -j

6. Acceleration Backends

All backends compile against the same ggml model file. Choose at configure time. At runtime, only -ng/--no-gpu and -dev/--device (and WHISPER_ARG_DEVICE) are adjustable.

Representative build commands:

# NVIDIA (RTX 50-series)
cmake -B build -DGGML_CUDA=1 -DCMAKE_CUDA_ARCHITECTURES="86"

# AMD RX 9070 XT
cmake -B build -DGGML_HIP=1 -DAMDGPU_TARGETS="gfx1201"

# Vulkan
cmake -B build -DGGML_VULKAN=1

# Apple Core ML
cmake -B build -DWHISPER_COREML=1

# CPU BLAS (also the path for POWER VSX)
cmake -B build -DGGML_BLAS=1

Tip

Confirm the active backend at runtime: the system_info: line printed at startup shows feature flags like AVX2, NEON, BLAS, COREML, OPENVINO, etc.

The ggml/src/ tree contains additional backends not all wired into whisper.cpp's documented build recipes (WebGPU, OpenCL, RPC, zDNN, ZenDNN, Hexagon, virtgpu). Treat those as experimental/advanced.

7. The C API (whisper.h)

whisper.h is a stable C interface (usable from C and C++). It is thread-safe per context, provided one whisper_context is not shared across threads concurrently.

7.1 Minimal usage

#include "whisper.h"

struct whisper_context_params cparams = whisper_context_default_params();
struct whisper_context * ctx =
    whisper_init_from_file_with_params("/path/to/ggml-base.en.bin", cparams);

struct whisper_full_params wparams =
    whisper_full_default_params(WHISPER_SAMPLING_GREEDY);

if (whisper_full(ctx, wparams, pcmf32.data(), pcmf32.size()) != 0) {
    fprintf(stderr, "failed to process audio\n");
    return 7;
}

const int n = whisper_full_n_segments(ctx);
for (int i = 0; i < n; ++i) {
    printf("%s", whisper_full_get_segment_text(ctx, i));
}

whisper_free(ctx);

pcmf32 is raw 32-bit float PCM at 16 kHz.

7.2 Lifecycle functions

Caution

The legacy initializers without a params argument (whisper_init_from_file, whisper_init, etc.) are deprecated via WHISPER_DEPRECATED. Use the _with_params forms.

7.3 The core call: whisper_full

int whisper_full(struct whisper_context * ctx,
                 struct whisper_full_params params,
                 const float * samples, int n_samples);

Runs the full pipeline (PCM → mel → encode → decode → text). Variants:

• whisper_full_with_state — run against an explicit state.
• whisper_full_parallel — split the audio across n_processors for speed (with some accuracy loss at chunk boundaries). This is what whisper-server uses.

Lower-level building blocks are also exposed: whisper_pcm_to_mel, whisper_set_mel, whisper_encode, whisper_decode, whisper_tokenize.

7.4 Key parameter structs

whisper_context_params — set once at load time: use_gpu, flash_attn, gpu_device, and the experimental DTW token-timestamp fields (dtw_token_timestamps, dtw_aheads_preset, dtw_n_top, dtw_aheads).

whisper_full_params — per-inference. Notable fields: strategy (greedy / beam search), n_threads, n_max_text_ctx, offset_ms, duration_ms, translate, no_context, no_timestamps, single_segment, token_timestamps, max_len, split_on_word, max_tokens, audio_ctx, tdrz_enable, suppress_regex, initial_prompt / carry_initial_prompt / prompt_tokens, language / detect_language, suppress_blank / suppress_nst, the decoding thresholds (temperature, temperature_inc, entropy_thold, logprob_thold, no_speech_thold, max_initial_ts, length_penalty), greedy.best_of, beam_search.beam_size, the GBNF grammar fields, and the VAD block (vad, vad_model_path, vad_params).

Obtain defaults with whisper_full_default_params(strategy) and whisper_context_default_params().

7.5 Callbacks

whisper_full_params accepts function pointers for: new-segment notification (whisper_new_segment_callback), progress (whisper_progress_callback), encoder-begin (return false to abort), an abort callback (ggml_abort_callback — the server uses it to stop on client disconnect), and a logits filter (whisper_logits_filter_callback). Set logging globally with whisper_log_set().

7.6 Result accessors

After whisper_full, read results: whisper_full_n_segments, whisper_full_get_segment_text, whisper_full_get_segment_t0/t1, whisper_full_lang_id, whisper_full_n_tokens, whisper_full_get_token_text/_id/_data/_p, whisper_full_get_segment_speaker_turn_next (tinydiarize), and whisper_full_get_segment_no_speech_prob.

7.7 Voice Activity Detection API

A self-contained VAD API mirrors the main flow: whisper_vad_init_from_file_with_params, whisper_vad_detect_speech (and _no_reset + whisper_vad_reset_state for streaming), whisper_vad_segments_from_probs / _from_samples, and the whisper_vad_params struct (threshold, min_speech_duration_ms, min_silence_duration_ms, max_speech_duration_s, speech_pad_ms, samples_overlap).

7.8 Language & model introspection

whisper_lang_id, whisper_lang_str, whisper_lang_str_full, whisper_lang_max_id, whisper_lang_auto_detect, plus a family of whisper_model_n_* accessors (vocab, audio/text ctx/state/head/layer, mels, ftype) and whisper_print_system_info().

Note

Constants: WHISPER_SAMPLE_RATE = 16000, WHISPER_N_FFT = 400, WHISPER_HOP_LENGTH = 160, WHISPER_CHUNK_SIZE = 30.

8. Models: Format, Conversion, Quantization

8.1 The ggml model format

A model is a single binary file packing everything the runtime needs: the ggml magic (0x67676d6c), hyperparameters (vocab size, audio/text context/state/head/layer counts, n_mels, ftype), the mel filter bank, the tokenizer vocabulary, and the weights (F16 by default, with small tensors kept F32). Download with models/download-ggml-model.sh or fetch from huggingface.co/ggerganov/whisper.cpp.

8.2 Converting from PyTorch / Hugging Face

The models/ directory contains conversion scripts (Python; require PyTorch and OpenAI whisper):

Example:

python3 models/convert-pt-to-ggml.py ~/.cache/whisper/medium.pt ~/whisper/ ./models/whisper-medium

Valid model names enforced by the converters: tiny, tiny.en, base, base.en, small, small.en, medium, medium.en, large-v1, large-v2, large-v3, large-v3-turbo (an invalid name raises ValueError("Invalid model name")).

8.3 Core ML / OpenVINO encoder generation

# Core ML (Apple): pip install ane_transformers openai-whisper coremltools
./models/generate-coreml-model.sh base.en      # → models/ggml-base.en-encoder.mlmodelc

# OpenVINO (Intel): pip install -r models/requirements-openvino.txt
python models/convert-whisper-to-openvino.py --model base.en   # → ggml-base.en-encoder-openvino.xml/.bin

Build the matching backend (-DWHISPER_COREML=1 or -DWHISPER_OPENVINO=1) and place the generated files alongside the ggml model so they are auto-discovered at runtime.

8.4 Quantization

./build/bin/quantize models/ggml-base.en.bin models/ggml-base.en-q5_0.bin q5_0

quantize <in-f32.bin> <out.bin> <type> validates the magic number, copies hyperparameters with an updated quantization version, and re-encodes weights with ggml_common_quantize_0. Supported types include q4_0, q5_0, q5_1, q8_0 (the tool prints all ggml ftypes when run without arguments). Many quantized models are also published for direct download.

9. Operating whisper-server

whisper-server (built when WHISPER_BUILD_SERVER is on) is a single-binary HTTP service built on a vendored cpp-httplib and nlohmann::json.

9.1 Endpoints

Request/response details (form fields, response formats) are in the User Guide §12.

9.2 Concurrency & lifecycle

• A single whisper_context is guarded by a std::mutex (whisper_mutex). Requests are serialized — the server handles one transcription at a time. Scale out with multiple instances behind a reverse proxy/load balancer.
• An abort_callback checks req.is_connection_closed(); if the client disconnects mid-inference, processing aborts and the server returns HTTP 499 (nginx convention).
• A server state flag (SERVER_STATE_LOADING_MODEL / SERVER_STATE_READY) drives /health. During POST /load, state flips to loading; if the new model fails to initialize, the process exits (exit(1)) rather than continuing without a model.
• Read/write timeouts default to 600 seconds.
• SIGINT/SIGTERM trigger a graceful shutdown; a second interrupt forces immediate exit.

9.3 Deployment recommendations

• Bind address: defaults to 127.0.0.1. Use --host 0.0.0.0 only behind a trusted proxy; do not expose the raw port to the internet.
• Format conversion: enable --convert (and install ffmpeg) to accept arbitrary audio formats; otherwise only miniaudio-decodable formats work. Set --tmp-dir to a writable scratch path.
• Path prefixing: --request-path lets you mount the server under a sub-path behind a proxy; --inference-path renames the transcription endpoint.
• Static assets: --public serves a directory at /; drop a custom index.html there to replace the built-in form.
• Docker: use ghcr.io/ggml-org/whisper.cpp:main (or -cuda/-vulkan/-musa), mounting a models volume, e.g. whisper-server --host 0.0.0.0 -m /models/ggml-base.bin.

Warning

--convert invokes ffmpeg via std::system() on a server-generated temp filename. Keep ffmpeg patched and --tmp-dir on a non-shared, least-privileged path. See also the related issues #3774 / #3819 about non-WAV handling when conversion is disabled.

10. Language Bindings

Official bindings live under bindings/:

The Java binding's Gradle build copies the native libraries (libwhisper.{so,dylib}, whisper.dll, ggml*.dll) into resources via the copyLibs task — your packaging must ship the matching native artifact for the target platform.

Many community bindings exist (Rust whisper-rs, Python pywhispercpp/whispercpp, .NET whisper.net, Swift SwiftWhisper, Unity, R). The Objective-C/Swift path is typically consumed via the prebuilt XCFramework (build-xcframework.sh).

11. Security Analysis

Caution

whisper.cpp parses untrusted binary inputs (audio files and model files). Treat both as attack surface, especially in server deployments. Community-reported issues at this commit:

• #3807 — null-pointer dereference + assertion abort via zero-dimension model parameters in whisper_model_load().
• #3802 — heap buffer overflow in the vendored stb_vorbis decoder via setup_malloc integer truncation.

Input handling. Audio is decoded by miniaudio (or ggml's/example helpers; stb_vorbis for OGG). A malicious file can trigger the decoder bugs above. Only process trusted inputs, or sandbox the process.

The server.

• No authentication or authorization is built in — anyone who can reach the port can transcribe, and via POST /load can load any model file path readable by the server process. Put it behind an authenticating proxy and restrict the filesystem.
• --convert shells out to ffmpeg with std::system(). The command is built from a server-generated temp filename (not directly from the uploaded filename), which limits injection, but ffmpeg itself parses untrusted media — keep it updated and run least-privileged.
• Default bind is 127.0.0.1; binding to 0.0.0.0 exposes the service.
• Requests are serialized behind a mutex, so a single slow/large upload blocks others — a denial-of-service consideration for public endpoints (mitigate with proxy timeouts, body-size limits, and the existing 600 s read/write timeouts).

Secrets. HF_TOKEN is read by the download script and sent as an Authorization: Bearer header to Hugging Face. The Java publishing path reads MAVEN_USERNAME/MAVEN_PASSWORD/PGP_SECRET/PGP_PASSPHRASE from the environment — relevant only to maintainers publishing artifacts.

Hardening recommendations.

• [ ] Run whisper-server behind a reverse proxy that enforces auth, TLS, rate limits, and request-size caps.
• [ ] Restrict the process's filesystem so POST /load cannot reach sensitive paths.
• [ ] Keep ffmpeg and the vendored decoders current; rebuild after upstream security fixes.
• [ ] Validate/limit upload size and duration at the proxy.
• [ ] Consider sanitizer builds (WHISPER_SANITIZE_ADDRESS/_UNDEFINED) when fuzzing or vetting inputs.

12. Code Quality & Conventions

• C-style public API. The interface is plain C in whisper.h, with WHISPER_API export macros (handling __declspec(dllexport/dllimport) on Windows and visibility on others) and WHISPER_DEPRECATED for graceful migration. This keeps FFI/binding surfaces simple.
• Single-responsibility core. Model logic is concentrated in whisper.cpp; numerical kernels are delegated to ggml. This separation lets backends evolve independently of the ASR logic.
• Hand-rolled CLI parsing. Each example program defines its own whisper_params struct (with inline defaults) and a manual for (i < argc) argument loop paired with a whisper_print_usage() block. There is no shared options library across examples, so the same flag (e.g. -fa) is re-declared per tool. This is a deliberate ggml-family idiom favoring zero dependencies.
• Defaults as source of truth. Default values live in the param structs and whisper_full_default_params(); the header explicitly warns that reordering whisper_full_params requires updating the defaults function.
• Experimental features are flagged. Token-level timestamps (DTW), tinydiarize ([TDRZ]), and speed-up techniques are marked [EXPERIMENTAL] in the header with caveats about quality.
• Windows/Unicode care. whisper-cli sets the console output code page to UTF-8 and merges multi-byte UTF-8 tokens (CJK) when emitting JSON (refs issue #1798).
• Response-file support. whisper-cli accepts @file to read args from a file — convenient for long invocations.

13. Testing & CI

• WER harnesses. tests/earnings21/ and tests/librispeech/ compute Word Error Rate against standard datasets. Both declare a minimal Python dependency set (jiwer, regex, more-itertools) in requirements.txt.
• Binding tests. The Java binding runs JUnit 5 tests with JNA on the built native library path; the Node addon uses Jest.
• CI. GitHub Actions runs the build matrix (badge in the README). A scheduled close-issue workflow marks issues stale after 30 days and closes them after 14 more, exempting labels like bug, roadmap, help wanted.
• Sanitizers. Thread/Address/Undefined sanitizers are available as WHISPER_SANITIZE_* build options for local verification.

Note

The confidence/triage signal for this repo flagged limited test-coverage depth and many shallow (LOW-priority) folders — most of the tree is example apps and per-architecture ggml kernels rather than ASR-logic tests. Functional correctness is largely validated through the WER harnesses and downstream usage.

14. Potential Improvements

Derived from the code, comments, and community issues:

• Decoder concurrency. whisper_decode carries a TODO: add support for multiple decoders; beam search currently has limited parallelism. Server throughput is bounded by the single-context mutex — a context pool or worker model would help.
• Server format coverage. verbose_json notes compression_ratio and no_speech_prob are partially unimplemented (compression_ratio is TODO). The server also has a TODO: add more output formats.
• Speaker diarization. Real diarization is a top community request (#64); current options (--diarize stereo energy, --tinydiarize) are heuristic/experimental.
• Beam-search patience. whisper_full_params.beam_search.patience is defined but TODO: not implemented.
• DTW memory. whisper_context_params.dtw_mem_size is marked TODO: remove.
• Hardware edge cases. Open issues report a CUDA-image SIGILL on AMD Zen 3 (#3814) and a Vulkan/flash-attention crash on RDNA4 (#3806) — backend stability across exotic hardware is an ongoing effort.
• Packaging. Requests for pkg-config files (#781) and prebuilt Linux binaries (#3852) remain open.

15. Troubleshooting (Operational)

Build can't find a backend toolkit

Cause: Backend enabled but its SDK/driver is missing or not on the path. Solution: Install the toolkit (CUDA/ROCm/OpenVINO/etc.), source its environment (e.g. OpenVINO setupvars.sh), and reconfigure from a clean build/.

Configure fails with a deprecated-flag error

Cause: Using an old WHISPER_* acceleration flag (e.g. WHISPER_CUBLAS, which is now fatal). Solution: Switch to the GGML_* equivalent (see §5.1).

CUDA image crashes with SIGILL after model load

Cause: CPU-feature mismatch / LD_LIBRARY_PATH shadowing host libcuda (issue #3814). Solution: Avoid overriding LD_LIBRARY_PATH to host CUDA libs inside the container; use the image's bundled libraries.

Server crashes with flash attention on some GPUs

Cause: Flash-attention path unstable on certain Vulkan drivers/GPUs (issue #3806). Solution: Start the server/CLI with -nfa/--no-flash-attn.

/load makes the server exit

Cause: The requested model failed to initialize; the server calls exit(1) rather than run model-less. Solution: Validate the model path/file before calling /load; supervise the process (systemd/Docker restart) to recover.

High latency under concurrent load

Cause: Single-context mutex serializes requests. Solution: Run multiple server instances behind a load balancer; consider VAD and smaller/quantized models to cut per-request time.

Core ML / OpenVINO first run is very slow

Cause: The device compiles the model to a device-specific blob on first use. Solution: Expected; subsequent runs are cached and fast. Pre-warm during deployment.

16. Quick Reference

Build cheat-sheet

cmake -B build                              # CPU
cmake -B build -DGGML_CUDA=1                # NVIDIA
cmake -B build -DGGML_VULKAN=1              # Vulkan
cmake -B build -DGGML_HIP=1 -DAMDGPU_TARGETS="gfx1201"   # AMD
cmake -B build -DWHISPER_COREML=1           # Apple ANE
cmake -B build -DWHISPER_OPENVINO=1         # Intel
cmake -B build -DWHISPER_SDL2=ON            # + stream/command
cmake -B build -DWHISPER_COMMON_FFMPEG=yes  # + broad audio formats
cmake --build build -j --config Release

Server endpoints

Core C API calls

whisper_context_default_params()            → context params
whisper_init_from_file_with_params(path, cp)→ ctx
whisper_full_default_params(strategy)       → inference params
whisper_full(ctx, wp, samples, n)           → 0 on success
whisper_full_n_segments(ctx)                → segment count
whisper_full_get_segment_text(ctx, i)       → segment text
whisper_free(ctx)                           → cleanup

Key constants

Environment variables