Black Admin & Architecture Guide

This guide is for the engineers, architects, and integrators who need to understand Black from the inside: how its formatting pipeline is built, how it discovers and parses files, how to deploy it at scale (including the blackd server), its security posture, and how the codebase is organized for contributors. Everything here is derived from Black's source at commit ce1897a.

Note

Black is a command-line tool and a library, not a long-running service (with the single exception of the optional blackd HTTP server). "Deployment" for Black mostly means embedding it in developer workflows, CI pipelines, and editor tooling rather than running a daemon.

1. Executive Summary

Black is the uncompromising Python code formatter — an opinionated, deterministic tool that rewrites Python source into a single canonical style. It is maintained under the Python Software Foundation (psf/black), authored by Łukasz Langa and contributors, and licensed MIT.

Architectural pillars:

1. Lossless concrete-syntax parsing. Black parses with a forked lib2to3 (blib2to3) that preserves comments and whitespace, so it can reformat without losing information the standard ast module discards.
2. A two-pass, idempotent formatting pipeline. Source → CST → line generation → line splitting → output, run twice to converge on a stable result.
3. Correctness over speed by default. Every format is verified to produce an equivalent AST before it is written; the check is opt-out (--fast), not opt-in.
4. Sensible-defaults configuration. A tiny set of knobs, all settable in pyproject.toml, with target Python version auto-inferred from project metadata.
5. Embarrassingly parallel. Multi-file runs fan out across a process pool.

2. System Architecture

2.1 High-level flow

                        ┌──────────────────────────────────────────────┐
                        │  black CLI (click)  src/black/__init__.py     │
                        │  parse options → build Mode → find root       │
                        └───────────────┬──────────────────────────────┘
                                        │
                 ┌──────────────────────┼───────────────────────────┐
                 │                      │                            │
          -c/--code              single SRC                  multiple SRC
                 │                      │                            │
                 ▼                      ▼                            ▼
         reformat_code          reformat_one              reformat_many
        (string, no procs)    (cache-aware,           (concurrency.py:
                 │             single file)            ProcessPoolExecutor)
                 └──────────────────────┼───────────────────────────┘
                                        ▼
                          format_file_contents()
                                        │
                          ┌─────────────┴─────────────┐
                          │  is_ipynb?                 │
                          ▼                            ▼
                   format_ipynb_string          format_str()
                   (cell-by-cell)                     │
                                                      ▼
                                         _format_str_once()  ×2 (forced 2nd pass)
                                                      │
        ┌─────────────────────────────────────────────┼─────────────────────────────┐
        ▼                          ▼                   ▼                ▼              ▼
  lib2to3_parse           normalize_fmt_off     LineGenerator    transform_line   EmptyLineTracker
 (blib2to3 → CST)         (# fmt: off/skip)     (visit CST →     (split long      (blank-line
                                                 logical Lines)   lines, strings)   policy)
                                                      │
                                                      ▼
                                              assembled output string
                                                      │
                                   ┌──────────────────┴──────────────────┐
                                   │  not --fast?                        │
                                   ▼                                     ▼
                       assert_equivalent (AST match)        assert_stable (idempotence)

2.2 The formatting pipeline in detail

The heart of Black is format_str() (src/black/__init__.py), which calls _format_str_once() and then — crucially — calls it a second time if the first pass changed anything:

def format_str(src_contents, *, mode, lines=()):
    dst_contents = _format_str_once(src_contents, mode=mode, lines=lines)
    # Forced second pass to work around optional trailing commas interacting
    # with optional parentheses.
    if src_contents != dst_contents:
        return _format_str_once(dst_contents, mode=mode, lines=lines)
    return dst_contents

Note

The forced second pass exists because optional trailing commas can become forced trailing commas on the next pass, which interacts with optional parentheses. Running twice guarantees the output is stable. The assert_stable safety check later verifies this convergence held.

Inside _format_str_once() the stages are:

1. Decode & normalize newlines (decode_bytes) — detects the file's encoding (honoring magic encoding comments) and newline style (\n, \r\n, or \r), normalizing internally to \n.
2. Parse (lib2to3_parse) — produces a concrete syntax tree using blib2to3.
3. Determine target versions — uses mode.target_versions if set, otherwise detects __future__ imports and calls detect_target_versions() to infer from syntax features used.
4. Process # fmt: directives (normalize_fmt_off) — converts # fmt: off/on regions into standalone, untouchable comment nodes.
5. Apply line ranges (convert_unchanged_lines) — when --line-ranges is set, marks out-of-range lines as unchanged.
6. Generate logical lines — LineGenerator visits the CST and emits Line objects (one per logical line of output).
7. Track blank lines — EmptyLineTracker decides how many blank lines go before/after each line per Black's rules.
8. Split lines — transform_line() applies splitting transforms (bracket splitting, string handling) to fit the line-length budget.
9. Reassemble — blocks are joined and the internal \n is converted back to the original newline style.

2.3 File discovery

When given paths rather than a string, get_sources() (__init__.py) computes the set of files to format:

• For directories, it recurses via gen_python_files() (files.py), applying, in order: .gitignore rules → --exclude → --extend-exclude → --force-exclude, then keeping only paths matching --include.
• For explicit files, it skips the include/exclude logic except --force-exclude (which is why --force-exclude exists: to exclude files even when passed explicitly, e.g. by a pre-commit hook).
• Symlinks pointing outside the project root are skipped (resolves_outside_root_or_cannot_stat).
• .gitignore files are honored hierarchically: each subdirectory's .gitignore is layered on top of its parents'.

Important

Default exclusions are baked in: .git, .hg, .venv, .tox, .nox, .mypy_cache, .pytest_cache, .ruff_cache, build, dist, __pypackages__, and several others (see DEFAULT_EXCLUDES in const.py). Setting --exclude replaces this list; use --extend-exclude to add to it.

2.4 The blib2to3 parser

Black does not use Python's built-in ast module for parsing (only for the safety check). Instead it vendors blib2to3 — a fork of the standard library's lib2to3 — under src/blib2to3/.

Key modules: pytree.py (the Node/Leaf/Base CST classes and pattern-matching machinery), pygram.py (symbol tables), and pgen2/ (the parser generator and tokenizer).

2.5 Module inventory (src/black/)

3. Requirements

3.1 Runtime

Caution

Python 3.12.5 is explicitly rejected at startup: it has a memory-safety issue that can cause Black's AST safety check to fail. Use 3.12.6+ or 3.12.4.

3.2 Dependencies

Core (always installed):

Optional extras:

4. Configuration Mastery

4.1 Resolution order

Black builds each option's effective value from (lowest to highest precedence):

1. Built-in default (e.g. line length 88).
2. Config file — [tool.black] in the discovered pyproject.toml (or --config / user-level file), injected as Click's default_map.
3. Command-line flags.

The config injection happens in read_pyproject_toml() (an eager Click callback on --config), which calls parse_pyproject_toml() in files.py. Keys are normalized: --key/key-name → key_name.

4.2 Project-root and config discovery

find_project_root() walks up from the common parent of the source paths and stops at the first directory containing .git, .hg, or a pyproject.toml with a [tool.black] table. That directory is the root; its pyproject.toml is the config. The discovery method is reported under -v.

User-level fallback config:

4.3 The Mode object

All formatting behavior is captured in the Mode dataclass (mode.py). This is the single source of truth for "how to format", and is what blackd and library callers construct directly.

Note

Mode is also exported as FileMode for backwards compatibility with integrations. Mode.get_cache_key() derives the cache filename, so any change to these fields invalidates the cache (see Section 6).

4.4 Target version inference

When target-version is unset, infer_target_version() (files.py) reads requires-python from [project] and maps it to a set of TargetVersion values (handling both exact versions like "3.10" and specifiers like ">=3.9,<3.12"). If that yields nothing, Black falls back to per-file detection: get_features_used() scans the CST for version-gated syntax (f-strings, match, except*, type params, t-strings, lazy imports, etc.) and detect_target_versions() returns every version whose feature set is a superset.

The full TargetVersion enum spans PY33 through PY315. Feature gating is defined by the VERSION_TO_FEATURES map and consulted via supports_feature() — for example, trailing commas after *args (TRAILING_COMMA_IN_CALL/_IN_DEF) are only emitted when all targets support them.

4.5 Preview and unstable styles

Experimental formatting lives behind three gates, resolved by Mode.__contains__:

• --unstable → every preview feature is on.
• --preview → every preview feature except those in UNSTABLE_FEATURES is on.
• --enable-unstable-feature NAME → that one Preview feature is on (requires --preview).

The current Preview features (names usable with --enable-unstable-feature) include string_processing, hug_parens_with_braces_and_square_brackets, wrap_comprehension_in, simplify_power_operator_hugging, wrap_long_dict_values_in_parens, fix_if_guard_explosion_in_case_statement, pyi_overload_group_blank_lines, pyi_blank_line_before_decorated_class, and hug_comparator. string_processing and hug_parens_with_braces_and_square_brackets are classified unstable (known bugs).

Warning

The set of preview/unstable features changes between releases with no compatibility guarantees. Don't pin workflows to a specific unstable feature name.

5. Deployment & Integration Strategies

Black is consumed in four main ways. Pick based on where formatting needs to happen.

5.1 Local CLI / library

The simplest integration. As a library, the stable public surface is:

import black

# Format a string (raises black.NothingChanged if already formatted)
formatted = black.format_str(source, mode=black.Mode(line_length=100))

# Format a whole file's contents (adds the safety check unless fast=True)
result = black.format_file_contents(source, fast=False, mode=black.Mode())

Important

format_str and format_file_contents raise black.NothingChanged when the input is already formatted — catch it. format_file_contents also performs the equivalence/stability checks (skippable with fast=True).

5.2 pre-commit

The canonical developer-machine integration. Black ships two hooks (.pre-commit-hooks.yaml): black (.py/.pyi) and black-jupyter (adds .ipynb via the jupyter extra).

Tip

Use the psf/black-pre-commit-mirror repo rather than psf/black in .pre-commit-config.yaml — it's ~2× faster because it installs a built wheel instead of compiling from source. Both hooks set require_serial: true and a minimum pre-commit version of 2.9.2.

5.3 GitHub Action

The repository is a composite GitHub Action (action.yml → action/main.py). The action script:

1. Creates a fresh virtualenv (.black-env).
2. Resolves the version to install from the version input, or from pyproject.toml when use_pyproject: true (reading required-version, then dependency groups/project.dependencies), or from the action's own git-archival tag.
3. pip installs Black (plus colorama, and jupyter when jupyter: true).
4. Runs Black with options + src, tears the venv down, optionally writes output to a file and the job summary, and exits with Black's exit code.

5.4 blackd HTTP server

A persistent formatter for editors/tooling. See Section 6.4 for operations. Architecturally it is a thin aiohttp app: a single POST / route handled by handle(), which parses X-* headers into a Mode (parse_mode), runs black.format_file_contents inside a ProcessPoolExecutor, and maps Black exceptions to HTTP status codes. A CORS middleware (middlewares.py) gates cross-origin browser access.

6. Operations

6.1 Exit codes and reporting

The Report object (report.py) accumulates counters and computes the process exit code via return_code:

str(report) renders the human summary ("N files reformatted, M files left unchanged."). Under --check/--diff the verbs switch to the conditional mood ("would be reformatted").

6.2 The cache

Black caches results to skip unchanged files on subsequent runs (cache.py):

• Location: platformdirs.user_cache_dir("black")/<black-version>/, overridable with BLACK_CACHE_DIR.
• Keying: one cache file per Mode, named cache.<mode.get_cache_key()>.pickle. Changing line length, target version, preview flags, etc. produces a different cache file — so toggling options never serves stale results.
• Invalidation: a file is considered changed if its size differs, or its mtime differs and its SHA-256 differs. (mtime-only changes with identical content don't trigger a reformat.)
• Writes: atomic — written to a temp file in the same directory, then os.replaced in. Cache write failures are swallowed (OSError → no-op), so a read-only cache dir never breaks formatting.

Tip

In CI, persisting the Black cache between runs across an unchanged dependency set speeds up --check. Key any CI cache on the Black version and your pyproject.toml, mirroring Black's own keying.

6.3 Parallelism

reformat_many() (concurrency.py) handles multi-file runs:

• Worker count = --workers, else BLACK_NUM_WORKERS, else os.cpu_count().
• On Windows the count is capped at 60 (CPython limitation); in frozen/PyInstaller builds it drops to 1.
• Uses a ProcessPoolExecutor; if the platform can't fork/spawn (AWS Lambda, Termux), it gracefully falls back to a single-worker ThreadPoolExecutor.
• maybe_use_uvloop() swaps in uvloop/winloop if installed, else stock asyncio.
• For --diff output across processes, a multiprocessing.Manager().Lock() serializes writes so diffs don't interleave.

6.4 Running blackd

pip install "black[d]"
blackd --bind-host=127.0.0.1 --bind-port=45484 --max-body-size=5242880

Status-code contract: 200 formatted · 204 unchanged · 400 syntax/header error · 500 unexpected error · 501 bad protocol version.

7. Security Analysis

Caution

Black executes as a developer/CI tool with the privileges of whoever runs it. It reads and overwrites source files in place. Treat the paths you hand it, and any config it reads, with the same trust you'd give arbitrary code in the repo.

7.1 Trust boundaries

Warning

The cache is a pickle file (cache.py). Pickle deserialization can execute arbitrary code if the file is attacker-controlled. The cache lives in a per-user, per-version cache directory, so the risk is low in normal use — but do not point BLACK_CACHE_DIR at a shared or untrusted location, and don't restore an untrusted cache into CI. Unpickling errors are caught and treated as a cache miss, but that does not protect against a deliberately malicious payload.

7.2 Correctness as a safety property

Black's defining safety feature is that it will not silently change your program's meaning:

• assert_equivalent() re-parses source and output with Python's ast and compares a normalized stringification. A mismatch raises ASTSafetyError and the file is not written.
• assert_stable() re-formats the output and asserts it doesn't change (idempotence).
• Both run by default; --fast (or X-Fast-Or-Safe: fast in blackd) opts out for speed.

This means a Black bug surfaces as a loud internal error with a bug-report link and a diff dump — never as corrupted code on disk.

7.3 blackd network exposure

blackd has no authentication and no TLS. Its security model is "bind to localhost". The only access control is the CORS middleware, which rejects any Origin not listed in --cors-allow-origin (returning 403) and is otherwise permissive only to same-host, no-Origin clients.

Hardening checklist for blackd:

• [ ] Bind to 127.0.0.1 (the default), never 0.0.0.0, unless fronted by an authenticating reverse proxy.
• [ ] Set --max-body-size to bound memory from oversized requests.
• [ ] Restrict --cors-allow-origin to the exact browser origins you trust; omit it entirely if no browser client exists.
• [ ] Run it as an unprivileged user with no write access to anything important — it formats request bodies, not files, so it needs no source-tree access.
• [ ] Treat it as a single-tenant developer convenience, not a multi-tenant service.

7.4 Supply chain

• Black is a pure-data-config tool with a small, well-known dependency set, all version-pinned with minimums. The mypyc-compiled wheels are built in CI (cibuildwheel).
• Pin Black's version across pyproject.toml, pre-commit rev, and the GitHub Action (version/use_pyproject) so formatting is reproducible and a compromised release can't silently change your diffs. --required-version enforces this at runtime.

8. Code Quality & Conventions

Note

The CST classes (Node, Leaf, BasePattern and its subclasses) and the StringTransformer hierarchy in trans.py are the most intricate parts of the codebase. The string transformers (StringMerger, StringSplitter, StringParenWrapper, StringParenStripper) implement the preview "string processing" style and are where most known preview bugs live.

9. Internals: How a Line Gets Split

Understanding line splitting is the key to understanding Black. The pipeline from CST to output text:

1. LineGenerator.visit(node) walks the CST and yields Line objects. A Line is a sequence of Leafs plus depth/bracket metadata. Visit methods (visit_stmt, visit_decorators, visit_match_case, …) handle each construct.
2. EmptyLineTracker.maybe_empty_lines(line) returns a LinesBlock describing how many blank lines precede this line, enforcing Black's blank-line rules (2 before top-level defs, 1 inside classes, special-casing stubs, etc.).
3. transform_line(line, mode, features) decides whether the line fits in line_length. If not, it applies transforms in order — delimiter split, bracket split (left_hand_split/right_hand_split), optional-parenthesis insertion, and (in preview) string transforms — guided by BracketTracker delimiter priorities (brackets.py).
4. Transforms communicate failure with CannotSplit/CannotTransform exceptions, letting Black try the next strategy.
5. The magic trailing comma: if a bracket pair contains a trailing comma, the line is forced to explode regardless of length (unless --skip-magic-trailing-comma).

This is why Black's output is deterministic: every construct has exactly one preferred shape, and the splitter tries strategies in a fixed order until one produces a fitting, stable result — which the second pass and assert_stable then confirm.

10. Potential Improvements & Known Issues

The following are drawn from the project's public issue tracker as of this snapshot — useful context for integrators and contributors evaluating edge cases.

Open formatting issues

Open feature requests

Architectural observations

• String-processing preview style is the main source of instability. The StringMerger/StringSplitter transforms (trans.py) are explicitly flagged UNSTABLE_FEATURES. Recently-closed bugs like #5058 (different code on the second pass with long type annotations) and #5001 (two distinct stable states for consecutive @overloads) live in this area. The forced second pass and assert_stable are the guardrails that keep these from shipping bad output.
• # fmt: skip with multiline constructs has historically been fragile (#4513); the directive handling in comments.py is intricate and a good area for hardening.
• The pickle cache (Section 7.1) is the one notable security sharp edge; a content-addressed or signed cache format would remove it.

11. Quick Reference

Key files

Constants

Environment variables

Operational commands

# Dry-run check for CI (exit 1 if anything would change)
black --check --diff .

# Format with an explicit config and version guard
black --config=pyproject.toml --required-version=25.1.0 src/

# Maximum-throughput run
black --workers=$(nproc) .            # uvloop extra recommended

# Run the format server
blackd --bind-host=127.0.0.1 --bind-port=45484