# Black Admin & Architecture Guide

This guide is for the engineers, architects, and integrators who need to understand Black from the inside: how its formatting pipeline is built, how it discovers and parses files, how to deploy it at scale (including the `blackd` server), its security posture, and how the codebase is organized for contributors. Everything here is derived from Black's source at commit `ce1897a`.

> [!NOTE]
> Black is a **command-line tool and a library**, not a long-running service (with the single exception of the optional `blackd` HTTP server). "Deployment" for Black mostly means embedding it in developer workflows, CI pipelines, and editor tooling rather than running a daemon.

---

## 1. Executive Summary

**Black** is the uncompromising Python code formatter — an opinionated, deterministic tool that rewrites Python source into a single canonical style. It is maintained under the Python Software Foundation (`psf/black`), authored by Łukasz Langa and contributors, and licensed MIT.

| Attribute | Value |
|---|---|
| Language | Python (≥ 3.10 to run) |
| Distribution | PyPI package `black`; mypyc-compiled wheels for speed |
| Entry points | `black` (CLI), `blackd` (HTTP server) |
| Core dependencies | `click`, `mypy-extensions`, `packaging`, `pathspec`, `platformdirs`, `pytokens` |
| Parser | `blib2to3` — a vendored, modified fork of lib2to3 |
| Safety model | AST-equivalence + idempotence checks after every format |
| Configuration | `pyproject.toml` `[tool.black]` |
| Lines targeted | Python 3.3 → 3.15 output |

**Architectural pillars:**

1. **Lossless concrete-syntax parsing.** Black parses with a forked lib2to3 (`blib2to3`) that preserves comments and whitespace, so it can reformat without losing information the standard `ast` module discards.
2. **A two-pass, idempotent formatting pipeline.** Source → CST → line generation → line splitting → output, run twice to converge on a stable result.
3. **Correctness over speed by default.** Every format is verified to produce an equivalent AST before it is written; the check is opt-out (`--fast`), not opt-in.
4. **Sensible-defaults configuration.** A tiny set of knobs, all settable in `pyproject.toml`, with target Python version auto-inferred from project metadata.
5. **Embarrassingly parallel.** Multi-file runs fan out across a process pool.

---

## 2. System Architecture

### 2.1 High-level flow

```text
                        ┌──────────────────────────────────────────────┐
                        │  black CLI (click)  src/black/__init__.py     │
                        │  parse options → build Mode → find root       │
                        └───────────────┬──────────────────────────────┘
                                        │
                 ┌──────────────────────┼───────────────────────────┐
                 │                      │                            │
          -c/--code              single SRC                  multiple SRC
                 │                      │                            │
                 ▼                      ▼                            ▼
         reformat_code          reformat_one              reformat_many
        (string, no procs)    (cache-aware,           (concurrency.py:
                 │             single file)            ProcessPoolExecutor)
                 └──────────────────────┼───────────────────────────┘
                                        ▼
                          format_file_contents()
                                        │
                          ┌─────────────┴─────────────┐
                          │  is_ipynb?                 │
                          ▼                            ▼
                   format_ipynb_string          format_str()
                   (cell-by-cell)                     │
                                                      ▼
                                         _format_str_once()  ×2 (forced 2nd pass)
                                                      │
        ┌─────────────────────────────────────────────┼─────────────────────────────┐
        ▼                          ▼                   ▼                ▼              ▼
  lib2to3_parse           normalize_fmt_off     LineGenerator    transform_line   EmptyLineTracker
 (blib2to3 → CST)         (# fmt: off/skip)     (visit CST →     (split long      (blank-line
                                                 logical Lines)   lines, strings)   policy)
                                                      │
                                                      ▼
                                              assembled output string
                                                      │
                                   ┌──────────────────┴──────────────────┐
                                   │  not --fast?                        │
                                   ▼                                     ▼
                       assert_equivalent (AST match)        assert_stable (idempotence)
```

### 2.2 The formatting pipeline in detail

The heart of Black is `format_str()` (`src/black/__init__.py`), which calls `_format_str_once()` and then — crucially — **calls it a second time** if the first pass changed anything:

```python
def format_str(src_contents, *, mode, lines=()):
    dst_contents = _format_str_once(src_contents, mode=mode, lines=lines)
    # Forced second pass to work around optional trailing commas interacting
    # with optional parentheses.
    if src_contents != dst_contents:
        return _format_str_once(dst_contents, mode=mode, lines=lines)
    return dst_contents
```

> [!NOTE]
> The forced second pass exists because optional trailing commas can become *forced* trailing commas on the next pass, which interacts with optional parentheses. Running twice guarantees the output is stable. The `assert_stable` safety check later verifies this convergence held.

Inside `_format_str_once()` the stages are:

1. **Decode & normalize newlines** (`decode_bytes`) — detects the file's encoding (honoring magic encoding comments) and newline style (`\n`, `\r\n`, or `\r`), normalizing internally to `\n`.
2. **Parse** (`lib2to3_parse`) — produces a concrete syntax tree using `blib2to3`.
3. **Determine target versions** — uses `mode.target_versions` if set, otherwise detects `__future__` imports and calls `detect_target_versions()` to infer from syntax features used.
4. **Process `# fmt:` directives** (`normalize_fmt_off`) — converts `# fmt: off/on` regions into standalone, untouchable comment nodes.
5. **Apply line ranges** (`convert_unchanged_lines`) — when `--line-ranges` is set, marks out-of-range lines as unchanged.
6. **Generate logical lines** — `LineGenerator` visits the CST and emits `Line` objects (one per logical line of output).
7. **Track blank lines** — `EmptyLineTracker` decides how many blank lines go before/after each line per Black's rules.
8. **Split lines** — `transform_line()` applies splitting transforms (bracket splitting, string handling) to fit the line-length budget.
9. **Reassemble** — blocks are joined and the internal `\n` is converted back to the original newline style.

### 2.3 File discovery

When given paths rather than a string, `get_sources()` (`__init__.py`) computes the set of files to format:

- For **directories**, it recurses via `gen_python_files()` (`files.py`), applying, **in order**: `.gitignore` rules → `--exclude` → `--extend-exclude` → `--force-exclude`, then keeping only paths matching `--include`.
- For **explicit files**, it skips the include/exclude logic *except* `--force-exclude` (which is why `--force-exclude` exists: to exclude files even when passed explicitly, e.g. by a pre-commit hook).
- **Symlinks** pointing outside the project root are skipped (`resolves_outside_root_or_cannot_stat`).
- `.gitignore` files are honored hierarchically: each subdirectory's `.gitignore` is layered on top of its parents'.

> [!IMPORTANT]
> Default exclusions are baked in: `.git`, `.hg`, `.venv`, `.tox`, `.nox`, `.mypy_cache`, `.pytest_cache`, `.ruff_cache`, `build`, `dist`, `__pypackages__`, and several others (see `DEFAULT_EXCLUDES` in `const.py`). Setting `--exclude` **replaces** this list; use `--extend-exclude` to add to it.

### 2.4 The `blib2to3` parser

Black does not use Python's built-in `ast` module for parsing (only for the safety check). Instead it vendors **`blib2to3`** — a fork of the standard library's `lib2to3` — under `src/blib2to3/`.

| Concern | Why a fork is needed |
|---|---|
| Comment & whitespace preservation | `lib2to3` produces a **concrete** syntax tree (CST) that keeps prefixes (whitespace/comments), which a formatter needs and `ast` throws away. |
| New syntax support | The grammar (`Grammar.txt`, `PatternGrammar.txt`) and tokenizer are extended to parse syntax from Python versions newer than the running interpreter. |
| Decoupling from the runtime | Black on Python 3.10 can still parse `match` statements, `except*`, type parameters, t-strings, etc. |

Key modules: `pytree.py` (the `Node`/`Leaf`/`Base` CST classes and pattern-matching machinery), `pygram.py` (symbol tables), and `pgen2/` (the parser generator and tokenizer).

### 2.5 Module inventory (`src/black/`)

| Module | Responsibility |
|---|---|
| `__init__.py` | CLI definition, top-level format orchestration, public API. |
| `mode.py` | `Mode` dataclass, `TargetVersion`/`Feature`/`Preview` enums, version→feature map. |
| `linegen.py` | `LineGenerator` — turns the CST into `Line` objects; line-splitting entry (`transform_line`). |
| `lines.py` | `Line`, `EmptyLineTracker`, `LinesBlock` — line model and blank-line policy. |
| `trans.py` | String transformers (`StringMerger`, `StringSplitter`, `StringParenWrapper`, …) for the preview string-processing style. |
| `brackets.py` | `BracketTracker`, delimiter priority — the bracket-matching that drives splitting. |
| `comments.py` | Comment normalization and `# fmt: off/on/skip` handling. |
| `nodes.py` | CST helpers, the `Visitor` base, symbol constants. |
| `parsing.py` | `lib2to3_parse`, `parse_ast`, `stringify_ast`; defines `InvalidInput`, `ASTSafetyError`, `SourceASTParseError`. |
| `files.py` | Project-root discovery, `pyproject.toml` parsing, gitignore/exclude logic, target-version inference. |
| `mode.py`/`const.py` | Configuration model and constants (`DEFAULT_LINE_LENGTH=88`, exclude/include regexes). |
| `cache.py` | `Cache` — feature-keyed, mtime/size/hash-based incremental cache. |
| `concurrency.py` | `reformat_many`, `schedule_formatting`, `maybe_use_uvloop` — the parallel path. |
| `report.py` | `Report` (counters, `return_code`), `Changed`, `NothingChanged`. |
| `handle_ipynb_magics.py` | Masking/unmasking IPython magics so notebook cells can be formatted. |
| `ranges.py` | `--line-ranges` parsing and line-mapping across passes. |
| `numerics.py`, `strings.py` | Numeric literal and string normalization helpers. |
| `output.py` | Colored terminal output, diffs (`diff`, `color_diff`, `ipynb_diff`). |
| `rusty.py` | `Ok`/`Err`/`Result` — a small Rust-style result type used internally. |
| `schema.py` | Loads the JSON schema for `[tool.black]` (used by `validate-pyproject`). |
| `debug.py` | `DebugVisitor` for pretty-printing the CST while debugging. |

---

## 3. Requirements

### 3.1 Runtime

| Requirement | Detail |
|---|---|
| Python | **3.10+** (asserted at startup). |
| OS | OS-independent (Linux, macOS, Windows). |
| Disk | Negligible, plus a per-version cache directory. |
| Memory | Proportional to the largest single file; the parallel path holds several files at once. |

> [!CAUTION]
> Python **3.12.5** is explicitly rejected at startup: it has a memory-safety issue that can cause Black's AST safety check to fail. Use 3.12.6+ or 3.12.4.

### 3.2 Dependencies

**Core (always installed):**

| Package | Constraint | Role |
|---|---|---|
| `click` | `>=8.0.0` | CLI framework. |
| `mypy-extensions` | `>=0.4.3` | `@mypyc_attr` decorators for the compiled build. |
| `packaging` | `>=22.0` | Parse `requires-python` for target-version inference. |
| `pathspec` | `>=1.0.0` | `.gitignore` matching (`GitIgnoreSpec`). |
| `platformdirs` | `>=2` | Locate the user cache directory. |
| `pytokens` | `~=0.4.0` | Tokenizer support. |
| `tomli` | `>=1.1.0` (Python < 3.11) | TOML parsing (stdlib `tomllib` on 3.11+). |
| `typing-extensions` | `>=4.0.1` (Python < 3.11) | Back-ported typing features. |

**Optional extras:**

| Extra | Pulls in | Enables |
|---|---|---|
| `colorama` | `colorama>=0.4.3` | Colored diffs on Windows. |
| `uvloop` | `uvloop` / `winloop` | Faster event loop for parallel runs. |
| `d` | `aiohttp>=3.10` | The `blackd` server. |
| `jupyter` | `ipython>=7.8.0`, `tokenize-rt>=3.2.0` | Notebook formatting. |

---

## 4. Configuration Mastery

### 4.1 Resolution order

Black builds each option's effective value from (lowest to highest precedence):

1. **Built-in default** (e.g. line length 88).
2. **Config file** — `[tool.black]` in the discovered `pyproject.toml` (or `--config` / user-level file), injected as Click's `default_map`.
3. **Command-line flags.**

The config injection happens in `read_pyproject_toml()` (an eager Click callback on `--config`), which calls `parse_pyproject_toml()` in `files.py`. Keys are normalized: `--key`/`key-name` → `key_name`.

### 4.2 Project-root and config discovery

`find_project_root()` walks up from the common parent of the source paths and stops at the first directory containing `.git`, `.hg`, or a `pyproject.toml` **with a `[tool.black]` table**. That directory is the root; its `pyproject.toml` is the config. The discovery *method* is reported under `-v`.

User-level fallback config:

| Platform | Path |
|---|---|
| Windows | `~\.black` |
| Unix | `$XDG_CONFIG_HOME/black`, defaulting to `~/.config/black` |

### 4.3 The `Mode` object

All formatting behavior is captured in the `Mode` dataclass (`mode.py`). This is the single source of truth for "how to format", and is what `blackd` and library callers construct directly.

| `Mode` field | Default | CLI origin |
|---|---|---|
| `target_versions` | `set()` (auto-detect) | `--target-version` |
| `line_length` | `88` | `--line-length` |
| `string_normalization` | `True` | `--skip-string-normalization` (inverts) |
| `is_pyi` | `False` | `--pyi` |
| `is_ipynb` | `False` | `--ipynb` |
| `skip_source_first_line` | `False` | `--skip-source-first-line` |
| `magic_trailing_comma` | `True` | `--skip-magic-trailing-comma` (inverts) |
| `python_cell_magics` | `set()` | `--python-cell-magics` |
| `preview` | `False` | `--preview` |
| `unstable` | `False` | `--unstable` |
| `enabled_features` | `set()` | `--enable-unstable-feature` |

> [!NOTE]
> `Mode` is also exported as `FileMode` for backwards compatibility with integrations. `Mode.get_cache_key()` derives the cache filename, so any change to these fields invalidates the cache (see [Section 6](#6-operations)).

### 4.4 Target version inference

When `target-version` is unset, `infer_target_version()` (`files.py`) reads `requires-python` from `[project]` and maps it to a set of `TargetVersion` values (handling both exact versions like `"3.10"` and specifiers like `">=3.9,<3.12"`). If that yields nothing, Black falls back to **per-file** detection: `get_features_used()` scans the CST for version-gated syntax (f-strings, `match`, `except*`, type params, t-strings, lazy imports, etc.) and `detect_target_versions()` returns every version whose feature set is a superset.

The full `TargetVersion` enum spans `PY33` through `PY315`. Feature gating is defined by the `VERSION_TO_FEATURES` map and consulted via `supports_feature()` — for example, trailing commas after `*args` (`TRAILING_COMMA_IN_CALL`/`_IN_DEF`) are only emitted when all targets support them.

### 4.5 Preview and unstable styles

Experimental formatting lives behind three gates, resolved by `Mode.__contains__`:

- `--unstable` → every preview feature is on.
- `--preview` → every preview feature **except** those in `UNSTABLE_FEATURES` is on.
- `--enable-unstable-feature NAME` → that one `Preview` feature is on (requires `--preview`).

The current `Preview` features (names usable with `--enable-unstable-feature`) include `string_processing`, `hug_parens_with_braces_and_square_brackets`, `wrap_comprehension_in`, `simplify_power_operator_hugging`, `wrap_long_dict_values_in_parens`, `fix_if_guard_explosion_in_case_statement`, `pyi_overload_group_blank_lines`, `pyi_blank_line_before_decorated_class`, and `hug_comparator`. `string_processing` and `hug_parens_with_braces_and_square_brackets` are classified **unstable** (known bugs).

> [!WARNING]
> The set of preview/unstable features changes between releases with **no compatibility guarantees**. Don't pin workflows to a specific unstable feature name.

---

## 5. Deployment & Integration Strategies

Black is consumed in four main ways. Pick based on where formatting needs to happen.

### 5.1 Local CLI / library

The simplest integration. As a library, the stable public surface is:

```python
import black

# Format a string (raises black.NothingChanged if already formatted)
formatted = black.format_str(source, mode=black.Mode(line_length=100))

# Format a whole file's contents (adds the safety check unless fast=True)
result = black.format_file_contents(source, fast=False, mode=black.Mode())
```

> [!IMPORTANT]
> `format_str` and `format_file_contents` raise `black.NothingChanged` when the input is already formatted — catch it. `format_file_contents` also performs the equivalence/stability checks (skippable with `fast=True`).

### 5.2 pre-commit

The canonical developer-machine integration. Black ships two hooks (`.pre-commit-hooks.yaml`): `black` (`.py`/`.pyi`) and `black-jupyter` (adds `.ipynb` via the `jupyter` extra).

> [!TIP]
> Use the **`psf/black-pre-commit-mirror`** repo rather than `psf/black` in `.pre-commit-config.yaml` — it's ~2× faster because it installs a built wheel instead of compiling from source. Both hooks set `require_serial: true` and a minimum pre-commit version of 2.9.2.

### 5.3 GitHub Action

The repository **is** a composite GitHub Action (`action.yml` → `action/main.py`). The action script:

1. Creates a fresh virtualenv (`.black-env`).
2. Resolves the version to install from the `version` input, or from `pyproject.toml` when `use_pyproject: true` (reading `required-version`, then dependency groups/`project.dependencies`), or from the action's own git-archival tag.
3. `pip install`s Black (plus `colorama`, and `jupyter` when `jupyter: true`).
4. Runs Black with `options` + `src`, tears the venv down, optionally writes output to a file and the job summary, and exits with Black's exit code.

| Input | Default | Notes |
|---|---|---|
| `options` | `--check --diff` | Flags passed to Black. |
| `src` | `.` | Paths. |
| `jupyter` | `false` | Installs the jupyter extra and colorama. |
| `version` | `""` | PEP 440 specifier; mutually exclusive with `use_pyproject`. |
| `use_pyproject` | `false` | Read version from `pyproject.toml` (requires Python 3.11+ in the runner for `tomllib`). |
| `summary` | `true` | Append output to the job summary. |
| `output-file` | `""` | Write output to a file too. |
| `black_args` | `""` | **Deprecated** — use `options` + `src`. |

### 5.4 `blackd` HTTP server

A persistent formatter for editors/tooling. See [Section 6.4](#64-running-blackd) for operations. Architecturally it is a thin `aiohttp` app: a single `POST /` route handled by `handle()`, which parses `X-*` headers into a `Mode` (`parse_mode`), runs `black.format_file_contents` inside a `ProcessPoolExecutor`, and maps Black exceptions to HTTP status codes. A CORS middleware (`middlewares.py`) gates cross-origin browser access.

---

## 6. Operations

### 6.1 Exit codes and reporting

The `Report` object (`report.py`) accumulates counters and computes the process exit code via `return_code`:

| Condition | Exit code |
|---|---|
| Any file failed to format | `123` |
| Files changed **and** `--check` is set | `1` |
| Otherwise | `0` |

`str(report)` renders the human summary ("N files reformatted, M files left unchanged."). Under `--check`/`--diff` the verbs switch to the conditional mood ("would be reformatted").

### 6.2 The cache

Black caches results to skip unchanged files on subsequent runs (`cache.py`):

- **Location:** `platformdirs.user_cache_dir("black")/<black-version>/`, overridable with `BLACK_CACHE_DIR`.
- **Keying:** one cache file per `Mode`, named `cache.<mode.get_cache_key()>.pickle`. Changing line length, target version, preview flags, etc. produces a *different* cache file — so toggling options never serves stale results.
- **Invalidation:** a file is considered changed if its size differs, or its mtime differs **and** its SHA-256 differs. (mtime-only changes with identical content don't trigger a reformat.)
- **Writes:** atomic — written to a temp file in the same directory, then `os.replace`d in. Cache write failures are swallowed (`OSError` → no-op), so a read-only cache dir never breaks formatting.

> [!TIP]
> In CI, persisting the Black cache between runs across an unchanged dependency set speeds up `--check`. Key any CI cache on the Black version and your `pyproject.toml`, mirroring Black's own keying.

### 6.3 Parallelism

`reformat_many()` (`concurrency.py`) handles multi-file runs:

- Worker count = `--workers`, else `BLACK_NUM_WORKERS`, else `os.cpu_count()`.
- On Windows the count is capped at 60 (CPython limitation); in frozen/PyInstaller builds it drops to 1.
- Uses a `ProcessPoolExecutor`; if the platform can't fork/spawn (AWS Lambda, Termux), it **gracefully falls back** to a single-worker `ThreadPoolExecutor`.
- `maybe_use_uvloop()` swaps in `uvloop`/`winloop` if installed, else stock asyncio.
- For `--diff` output across processes, a `multiprocessing.Manager().Lock()` serializes writes so diffs don't interleave.

### 6.4 Running `blackd`

```bash
pip install "black[d]"
blackd --bind-host=127.0.0.1 --bind-port=45484 --max-body-size=5242880
```

| Operational concern | Detail |
|---|---|
| Default bind | `localhost:45484`. |
| Concurrency | A `ProcessPoolExecutor` sized to `os.cpu_count()`, fronted by a bounded semaphore. |
| Body size limit | `--max-body-size`, default 5 MiB; oversized requests are rejected by aiohttp. |
| Protocol | Only `X-Protocol-Version: 1` is accepted (else `501`). |
| Shutdown | Handles signals; closes the event loop on exit. |
| Client | `blackd.client.BlackDClient` is a reference async client mapping options to `X-*` headers. |

Status-code contract: `200` formatted · `204` unchanged · `400` syntax/header error · `500` unexpected error · `501` bad protocol version.

---

## 7. Security Analysis

> [!CAUTION]
> Black executes as a developer/CI tool with the privileges of whoever runs it. It reads and **overwrites** source files in place. Treat the paths you hand it, and any config it reads, with the same trust you'd give arbitrary code in the repo.

### 7.1 Trust boundaries

| Surface | Exposure | Notes |
|---|---|---|
| Source files | Read + **overwritten in place** under `--write` (default). | Use `--check`/`--diff` for a dry run before trusting a batch operation. |
| `pyproject.toml` | Read and trusted. | A malicious config can set regexes/flags but cannot execute code — TOML is data-only, parsed by `tomllib`/`tomli`. |
| `.gitignore` files | Read. | Parsed by `pathspec`; a malformed gitignore aborts the run with an error rather than misbehaving silently. |
| Symlinks | Resolved; links pointing **outside** the project root are skipped. | Prevents Black from following a symlink out of the tree (`resolves_outside_root_or_cannot_stat`). |
| Cache pickle | Read with `pickle.load`. | See the warning below. |

> [!WARNING]
> The cache is a **pickle** file (`cache.py`). Pickle deserialization can execute arbitrary code if the file is attacker-controlled. The cache lives in a per-user, per-version cache directory, so the risk is low in normal use — but **do not point `BLACK_CACHE_DIR` at a shared or untrusted location**, and don't restore an untrusted cache into CI. Unpickling errors are caught and treated as a cache miss, but that does not protect against a deliberately malicious payload.

### 7.2 Correctness as a safety property

Black's defining safety feature is that it will not silently change your program's meaning:

- `assert_equivalent()` re-parses source and output with Python's `ast` and compares a normalized stringification. A mismatch raises `ASTSafetyError` and the file is **not** written.
- `assert_stable()` re-formats the output and asserts it doesn't change (idempotence).
- Both run by default; `--fast` (or `X-Fast-Or-Safe: fast` in blackd) opts out for speed.

This means a Black bug surfaces as a loud internal error with a bug-report link and a diff dump — never as corrupted code on disk.

### 7.3 `blackd` network exposure

`blackd` has **no authentication and no TLS**. Its security model is "bind to localhost". The only access control is the CORS middleware, which rejects any `Origin` not listed in `--cors-allow-origin` (returning `403`) and is otherwise permissive only to same-host, no-`Origin` clients.

**Hardening checklist for `blackd`:**

- [ ] Bind to `127.0.0.1` (the default), never `0.0.0.0`, unless fronted by an authenticating reverse proxy.
- [ ] Set `--max-body-size` to bound memory from oversized requests.
- [ ] Restrict `--cors-allow-origin` to the exact browser origins you trust; omit it entirely if no browser client exists.
- [ ] Run it as an unprivileged user with no write access to anything important — it formats request bodies, not files, so it needs no source-tree access.
- [ ] Treat it as a single-tenant developer convenience, not a multi-tenant service.

### 7.4 Supply chain

- Black is a pure-data-config tool with a small, well-known dependency set, all version-pinned with minimums. The mypyc-compiled wheels are built in CI (`cibuildwheel`).
- Pin Black's version across `pyproject.toml`, pre-commit `rev`, and the GitHub Action (`version`/`use_pyproject`) so formatting is reproducible and a compromised release can't silently change your diffs. `--required-version` enforces this at runtime.

---

## 8. Code Quality & Conventions

| Aspect | Practice |
|---|---|
| Type checking | `mypy` in `strict` mode (`strict_bytes`, `warn_unreachable`, `local_partial_types`); `python_version = "3.10"`. |
| Lint | `flake8` (with `bugbear`, `comprehensions`, `simplify`), `isort` (profile `black`), `prettier` for md/yaml/json — all wired through pre-commit. |
| Self-formatting | Black formats itself, using `unstable = true` in its own `[tool.black]` (the project dogfoods its bleeding edge). |
| Tests | `pytest` with `--strict-config --strict-markers`, `xfail_strict`, and `filterwarnings = ["error"]` (warnings are errors). Parallelized with `pytest-xdist`. Optional-dependency tests are gated by custom markers (`no_blackd`, `no_jupyter`). |
| Fuzzing | `hypothesis` + `hypothesmith` generate random valid Python to stress the formatter. |
| Regression corpus | `diff-shades` formats large real-world projects to catch unintended style changes. |
| Compilation | Optional mypyc build for speed; several modules are explicitly excluded from compilation (`blackd`, `__main__`, `debug.py`, `output.py`, `files.py`, etc.). |
| Error model | A small Rust-inspired `Ok`/`Err`/`Result` type (`rusty.py`) plus typed exceptions (`InvalidInput`, `CannotSplit`, `CannotTransform`, `ASTSafetyError`). |

> [!NOTE]
> The CST classes (`Node`, `Leaf`, `BasePattern` and its subclasses) and the `StringTransformer` hierarchy in `trans.py` are the most intricate parts of the codebase. The string transformers (`StringMerger`, `StringSplitter`, `StringParenWrapper`, `StringParenStripper`) implement the preview "string processing" style and are where most known preview bugs live.

---

## 9. Internals: How a Line Gets Split

Understanding line splitting is the key to understanding Black. The pipeline from CST to output text:

1. **`LineGenerator.visit(node)`** walks the CST and yields `Line` objects. A `Line` is a sequence of `Leaf`s plus depth/bracket metadata. Visit methods (`visit_stmt`, `visit_decorators`, `visit_match_case`, …) handle each construct.
2. **`EmptyLineTracker.maybe_empty_lines(line)`** returns a `LinesBlock` describing how many blank lines precede this line, enforcing Black's blank-line rules (2 before top-level defs, 1 inside classes, special-casing stubs, etc.).
3. **`transform_line(line, mode, features)`** decides whether the line fits in `line_length`. If not, it applies transforms in order — delimiter split, bracket split (`left_hand_split`/`right_hand_split`), optional-parenthesis insertion, and (in preview) string transforms — guided by `BracketTracker` delimiter priorities (`brackets.py`).
4. Transforms communicate failure with `CannotSplit`/`CannotTransform` exceptions, letting Black try the next strategy.
5. The **magic trailing comma**: if a bracket pair contains a trailing comma, the line is forced to explode regardless of length (unless `--skip-magic-trailing-comma`).

This is why Black's output is deterministic: every construct has exactly one preferred shape, and the splitter tries strategies in a fixed order until one produces a fitting, stable result — which the second pass and `assert_stable` then confirm.

---

## 10. Potential Improvements & Known Issues

The following are drawn from the project's public issue tracker as of this snapshot — useful context for integrators and contributors evaluating edge cases.

### Open formatting issues

| Issue | Summary |
|---|---|
| [#4349](https://github.com/psf/black/issues/4349) | Unnecessary parentheses added to an expression in an indexed assignment. |
| [#3946](https://github.com/psf/black/issues/3946) | `# pyright: ignore` comments placed on the wrong line. |
| [#2156](https://github.com/psf/black/issues/2156) | Request to use optional parentheses more often. |
| [#4651](https://github.com/psf/black/issues/4651) | Preserve alignment of inline comments. |
| [#809](https://github.com/psf/black/issues/809) | Long lambda with many parameters reported as "line too long". |
| [#5164](https://github.com/psf/black/issues/5164) | Unexpected version warning. |

### Open feature requests

| Issue | Summary |
|---|---|
| [#359](https://github.com/psf/black/issues/359) | Cython grammar support. |
| [#3668](https://github.com/psf/black/issues/3668) | Exceptions to comment spacing. |
| [#4025](https://github.com/psf/black/issues/4025) | Support for percent-format notebooks. |
| [#5120](https://github.com/psf/black/issues/5120) | Make releases immutable. |

### Architectural observations

- **String-processing preview style is the main source of instability.** The `StringMerger`/`StringSplitter` transforms (`trans.py`) are explicitly flagged `UNSTABLE_FEATURES`. Recently-closed bugs like [#5058](https://github.com/psf/black/issues/5058) (different code on the second pass with long type annotations) and [#5001](https://github.com/psf/black/issues/5001) (two distinct stable states for consecutive `@overload`s) live in this area. The forced second pass and `assert_stable` are the guardrails that keep these from shipping bad output.
- **`# fmt: skip` with multiline constructs** has historically been fragile ([#4513](https://github.com/psf/black/issues/4513)); the directive handling in `comments.py` is intricate and a good area for hardening.
- **The pickle cache** (Section 7.1) is the one notable security sharp edge; a content-addressed or signed cache format would remove it.

---

## 11. Quick Reference

### Key files

| Path | What lives there |
|---|---|
| `src/black/__init__.py` | CLI + format orchestration + public API. |
| `src/black/mode.py` | `Mode`, `TargetVersion`/`Feature`/`Preview`. |
| `src/black/files.py` | Config & file discovery. |
| `src/black/linegen.py`, `lines.py`, `trans.py`, `brackets.py` | The formatting engine. |
| `src/black/concurrency.py` | Parallel formatting. |
| `src/black/cache.py` | Incremental cache. |
| `src/blackd/` | The HTTP format server. |
| `src/blib2to3/` | The vendored CST parser. |
| `pyproject.toml` | Deps, `[tool.black]`, entry points, build config. |
| `action.yml`, `action/main.py` | The GitHub Action. |

### Constants

| Constant | Value |
|---|---|
| `DEFAULT_LINE_LENGTH` | `88` |
| `DEFAULT_INCLUDES` | `(\.pyi?\|\.ipynb)$` |
| `blackd` default port | `45484` |
| `blackd` default max body | `5 MiB` |
| Supported target versions | `py33` … `py315` |

### Environment variables

| Variable | Effect |
|---|---|
| `BLACK_CACHE_DIR` | Cache directory override. |
| `BLACK_NUM_WORKERS` | Default worker-process count. |
| `XDG_CONFIG_HOME` | User-config base on Unix. |

### Operational commands

```bash
# Dry-run check for CI (exit 1 if anything would change)
black --check --diff .

# Format with an explicit config and version guard
black --config=pyproject.toml --required-version=25.1.0 src/

# Maximum-throughput run
black --workers=$(nproc) .            # uvloop extra recommended

# Run the format server
blackd --bind-host=127.0.0.1 --bind-port=45484
```

---

_Generated June 2026 from psf/black at ce1897a. AI-generated documentation. The library is checked by sampling (three repositories so far); this guide was not individually reviewed. Verify against the project before relying on it. Unofficial; not affiliated with or reviewed by the project._