🐸 Coqui TTS Admin & Developer Guide

This guide is for the engineers, ops teams, and architects who deploy, operate, extend, or train with Coqui TTS. It covers the system architecture, the model and config systems, deployment topologies, the training pipeline, security posture, and the internals needed to maintain or fork the project. End-user synthesis instructions live in the companion User Guide.

Important

This documentation describes Coqui TTS v0.22.0 at commit dbf1a08 (2024-02-10). The company that built Coqui (Coqui GmbH) has shut down, so this codebase is effectively frozen at this release and is now community-maintained. There will be no upstream patches for new PyTorch/transformers incompatibilities or security findings unless contributed by the community. Treat any deployment as running on unmaintained, archived software and plan dependency pinning and security mitigations accordingly.

1. Executive Summary

Coqui TTS is a PyTorch-based deep-learning toolkit for text-to-speech. It packages:

• A model zoo — ~70 pretrained TTS models, ~17 vocoders, and a voice-conversion model, registered in a single TTS/.models.json manifest, plus on-demand access to ~1100 Meta MMS/Fairseq voices.
• Three runtime surfaces — a tts CLI (TTS/bin/synthesize.py), a Python API (TTS/api.py), and a Flask demo server (TTS/server/server.py, console script tts-server).
• A training stack — per-architecture model classes under TTS/tts, TTS/vocoder, TTS/vc, TTS/encoder, driven by the external trainer package and coqpit dataclass configs, with ready-made recipes under recipes/.

Note

The code is MPL-2.0, but individual models carry their own licenses spanning Apache-2.0, MPL, MIT, CC-BY-SA, CC-BY-NC-ND, and the bespoke CPML (XTTS — non-commercial). Licensing is per-model, not project-wide. See §5.4.

2. Architecture

2.1 High-level component map

                         ┌─────────────────────────────────────────────┐
   text ──▶  Entry surface │  tts (CLI)   tts-server (Flask)   TTS.api   │
                         └───────────────────────┬─────────────────────┘
                                                 │
                                       ┌─────────▼──────────┐
                                       │   Synthesizer       │  TTS/utils/synthesizer.py
                                       │  (orchestrator)     │
                                       └───┬───────┬─────┬───┘
                          ┌────────────────┘       │     └───────────────┐
                  ┌───────▼────────┐      ┌────────▼───────┐    ┌─────────▼────────┐
                  │  tts_model      │      │  vocoder model │    │ speaker encoder  │
                  │ TTS/tts/models  │      │ TTS/vocoder    │    │ TTS/encoder      │
                  └───────┬─────────┘      └────────────────┘    └──────────────────┘
                          │
                 ┌────────▼─────────┐   ┌───────────────────┐
                 │  ModelManager     │   │ load_config        │
                 │ TTS/utils/manage  │   │ TTS/config         │  (coqpit)
                 │ download + cache  │   └───────────────────┘
                 └────────┬──────────┘
                          ▼
                  TTS/.models.json  (model registry) ──▶ remote model archives (HF / GitHub releases)

The Synthesizer (TTS/utils/synthesizer.py) is the orchestration hub. It loads a TTS model, an optional vocoder, and an optional speaker encoder, then exposes .tts() and .voice_conversion(). Both the CLI and the server construct a Synthesizer; the TTS.api.TTS class wraps it with a friendlier interface.

2.2 Package layout

2.3 The model abstraction

Every model inherits BaseTrainerModel (TTS/model.py), which extends TrainerModel from the external trainer package and requires three methods:

class BaseTrainerModel(TrainerModel):
    @staticmethod
    @abstractmethod
    def init_from_config(config: Coqpit): ...   # build the model from its config

    @abstractmethod
    def inference(self, input, aux_input={}) -> Dict:  # returns {"model_outputs": ...}

    @abstractmethod
    def load_checkpoint(self, config, checkpoint_path, eval=False, strict=True, cache=False): ...

This uniform contract is why one Synthesizer can drive Tacotron2, VITS, XTTS, or a vocoder interchangeably — they all expose init_from_config / inference / load_checkpoint.

2.4 Model families

Coqui implements a broad set of architectures. Knowing which family a model belongs to tells you whether it needs a separate vocoder and what its inference cost looks like.

Note

Two-stage models output a mel-spectrogram and are paired with a vocoder (each model declares a default_vocoder in .models.json). If no vocoder is supplied, the Synthesizer falls back to Griffin-Lim, which is lower quality. End-to-end models (VITS/XTTS) skip the vocoder entirely.

3. The Model Registry & Manager

3.1 .models.json

TTS/.models.json is the canonical registry. It is a nested tree keyed by model_type → language → dataset → model_name. Each leaf describes one model:

"xtts_v2": {
  "description": "XTTS-v2.0.3 by Coqui with 17 languages.",
  "hf_url": ["https://.../model.pth", "https://.../config.json", "..."],
  "model_hash": "10f92b55c512af7a8d39d650547a15a7",
  "default_vocoder": null,
  "license": "CPML",
  "tos_required": true
}

Key fields:

3.2 ModelManager (TTS/utils/manage.py)

ModelManager reads .models.json, resolves a model name to a download, fetches and verifies archives, and caches them. Responsibilities:

• list_models() — enumerate the registry (backs tts --list_models).
• download_model(name) — resolve, download, extract, and return local paths.
• model_info_by_idx / model_info_by_full_name — registry lookups.
• Terms-of-service gate — raises Exception: [!] You must agree to the terms of service to use this model. for tos_required models unless COQUI_TOS_AGREED=1.
• Hash verification — TTS/utils/download.py raises RuntimeError: The hash of {file} does not match. on a checksum mismatch.

3.3 Cache resolution

The model cache directory is resolved by get_user_data_dir() in TTS/utils/generic_utils.py:

Tip

In containers and CI, set TTS_HOME to a mounted volume so model downloads (which can be hundreds of MB to several GB for XTTS/Bark/Tortoise) persist across runs and don't bloat the image layer or re-download on every start.

4. Configuration

Coqui has almost no global configuration — there is no central YAML or .env for the library itself. Behavior is driven two ways:

1. Per-model config — each model ships a config.json parsed into a coqpit dataclass via TTS/config/load_config(). This holds audio settings (sample rate, mel parameters), model hyperparameters, dataset definitions, and inference options. When you run a released model, its config travels with the download; when you run your own, you pass --config_path.
2. Environment variables — the only three runtime env vars are below.

4.1 Working with a model config

from TTS.config import load_config

config = load_config("path/to/config.json")
print(config.audio.sample_rate)
print(config.model)            # architecture name

load_config() raises:

• ModuleNotFoundError: [!] Config for {model_name} cannot be found. — unknown model config.
• TypeError: [!] Unknown config file type {ext} — config file is neither .json nor a supported type.

Note

Because config is a coqpit dataclass, the schema is the Python class (e.g. Tacotron2Config, VitsConfig, XttsConfig under TTS/tts/configs/), not a documented key list. To see every tunable for a model, read its *Config dataclass. This is the authoritative config reference for training and custom inference.

5. Security

Caution

Coqui TTS has several material security considerations. Because the project is unmaintained, you own these mitigations — there will be no upstream fix. Review this section before any production or multi-user deployment.

5.1 Unsafe model deserialization (pickle)

Model checkpoints are PyTorch .pth files loaded via torch.load() (through load_fsspec), which uses Python's pickle and can execute arbitrary code during deserialization. Combined with on-demand downloads from remote URLs, a tampered checkpoint is a remote-code-execution vector.

• Mitigation: Only load checkpoints from sources you trust. Verify the .models.json checksums (Coqui does this for registry models). For custom or third-party checkpoints, scan them and load in an isolated environment. Where possible on newer PyTorch, prefer weights_only=True loading semantics.
• Reference: community issue #4409 (Unsafe Deserialization of Model Checkpoints).

5.2 Archive extraction path traversal

Model archives are unpacked with extractall() in TTS/utils/download.py without sanitizing member paths, so a malicious archive could write files outside the cache directory (Zip-Slip).

• Mitigation: Only download models from the official registry/URLs. Run the process as a non-root user with a constrained TTS_HOME so any traversal is contained.
• Reference: community issue #4409 (Path Traversal in Archive Extraction). download.py only accepts .tar.gz, .tgz, and .zip and raises NotImplementedError for other formats — but does not guard member paths.

5.3 Demo server exposure

The Flask server (TTS/server/server.py) is for local experimentation:

• It binds to host="::" (all interfaces) in app.run().
• It has no authentication, rate limiting, or CSRF protection.
• --debug true enables Flask's debugger (Werkzeug), which must never be exposed publicly.
• /api/tts synthesizes arbitrary attacker-supplied text; a single global Lock serializes requests, so it is also trivially DoS-able by long inputs.

Warning

Never expose tts-server directly to untrusted networks. Front it with an authenticated reverse proxy, bind it to localhost, disable --debug, and cap request sizes. Treat it as a developer convenience, not a production service.

5.4 Model licensing

Licensing is per-model and some models are non-commercial:

Important

Always read the license field from .models.json (or tts --model_info_by_name) before shipping a product that uses a Coqui voice. XTTS — the flagship model — is non-commercial CPML and is gated behind a terms-of-service acceptance (tos_required: true).

5.5 Defunct domain

The original coqui.ai domain is no longer controlled by the project after the company's shutdown (community issues #4418, #4392 report it as hijacked). Do not fetch models, configs, or scripts from coqui.ai; rely on the GitHub repository and the registry URLs (which point at coqui.gateway.scarf.sh and Hugging Face mirrors).

6. Deployment

6.1 Topologies

6.2 Docker

docker run --rm -it -p 5002:5002 --entrypoint /bin/bash ghcr.io/coqui-ai/tts-cpu
python3 TTS/server/server.py --model_name tts_models/en/vctk/vits

GPU variants are published alongside the CPU image. Mount a volume at TTS_HOME to persist the model cache.

6.3 Deployment checklist

• [ ] Pin the Python interpreter to 3.9–3.11 (the package refuses 3.12+).
• [ ] Pin transformers to the 4.x line (5.x breaks XTTS — see §8).
• [ ] Pin setuptools to a version that still ships pkg_resources.
• [ ] Pre-download models into a persistent TTS_HOME volume.
• [ ] Set COQUI_TOS_AGREED=1 if (and only if) the model license permits your use.
• [ ] Confirm each model's license fits your distribution model.
• [ ] If serving HTTP: bind to localhost, add auth, disable --debug, cap input length.
• [ ] Run as a non-root user with a constrained cache directory.
• [ ] Provision a GPU for XTTS/Bark/Tortoise workloads.

Tip

Capture a known-good environment with python -m TTS.bin.collect_env_info and store it next to your deployment manifests. Given the unmaintained dependency surface, an exact, reproducible set of versions is your most important operational artifact.

7. Training & Operations

7.1 Training entry points (TTS/bin/)

7.2 Recipes

recipes/<dataset>/<model>/ holds ready-to-run training scripts (e.g. recipes/ljspeech/vits_tts, recipes/vctk/yourtts, recipes/thorsten_DE/...). A recipe wires a dataset config to a model config and calls the trainer. Start from the recipe closest to your data and adjust the config dataclass.

# Example shape of a recipe run
python recipes/ljspeech/vits_tts/train_vits.py

7.3 The training engine

Training is delegated to the external trainer package (trainer>=0.0.36). Models implement the BaseTrainerModel contract; trainer handles the loop, logging (terminal + TensorBoard), checkpointing, and distributed runs. Configs are coqpit dataclasses.

Caution

XTTS cannot be trained through the generic path. TTS/tts/models/xtts.py raises NotImplementedError: XTTS has a dedicated trainer ... from its training hooks. Use the dedicated XTTS fine-tuning recipe/demo (TTS/demos/xtts_ft_demo/) instead. Likewise, Tortoise training is not implemented (NotImplementedError: 'Tortoise Training is not implemented').

7.4 Day-to-day operations

8. Known Issues & Compatibility

The most common production problems stem from dependency drift, because the codebase is frozen but its dependencies move.

Note

Feature requests that remain open against the frozen codebase include FunASR annotation support (#4422) and a standalone compiled binary (#4419). These will not land without community forks.

9. Code Quality & Conventions

• Uniform model contract. Every model implements init_from_config / inference / load_checkpoint, which is what makes the Synthesizer polymorphic across architectures.
• Config-as-dataclass. coqpit dataclasses give typed, serializable configs; the config is the schema. This keeps training reproducible but means there is no separate documented config reference.
• Registry-driven distribution. .models.json + ModelManager cleanly separate "what models exist" from "how to fetch them," and include checksums and license/ToS metadata.
• Late imports for CLI speed. synthesize.py imports TTS.api/Synthesizer/ModelManager inside main() so tts --list_models and help screens start fast.
• Cython hot path. TTS/tts/utils/monotonic_align/core.pyx is compiled for alignment performance — note the build dependency (cython) when packaging.
• Broad text-frontend support. Per-language G2P/phonemizers (gruut, jieba/pypinyin, jamo/g2pkk, bangla/bnnumerizer) live under TTS/tts/utils/text/.

Improvement opportunities (for forks)

1. Harden checkpoint loading — adopt weights_only=True and reject untrusted pickles (§5.1).
2. Sanitize archive extraction — validate member paths before extractall() (§5.2).
3. Modernize dependency pins — support transformers 5.x and Python 3.12+ (#4394, #4416).
4. Secure the server — optional auth, request-size limits, and a localhost default bind (§5.3).
5. Document config dataclasses — auto-generate a config reference from the *Config classes.

10. Quick Reference

10.1 Console scripts

10.2 HTTP API (demo server)

10.3 Environment variables

10.4 Key dependencies

10.5 Critical operational facts

• Python 3.9–3.11 only; transformers 4.x only.
• Model cache at $TTS_HOME/tts → $XDG_DATA_HOME/tts → ~/.local/share/tts.
• XTTS is CPML (non-commercial) and gated (COQUI_TOS_AGREED=1).
• Demo server has no auth and binds all interfaces — never expose it directly.
• Checkpoints are pickle; archives are extracted without path guards — load only trusted sources.
• The project is unmaintained — you own all patches and mitigations.