# MarkupSafe Administrator & Developer Guide

MarkupSafe is a small, single-purpose Python library that provides the
type-tagged HTML/XML escaping primitives consumed by Jinja2 (and by
extension Flask, Quart, Sanic-Jinja2, and most templating in the Pallets
ecosystem). The entire library is one Python module of roughly 380 lines
plus a 200-line C accelerator. Despite the small size, it is a load-bearing
piece of the Python web stack: tens of thousands of applications rely on
its correctness for first-line XSS defence.

This guide covers the architecture, build system, security model, and
internals that operators, security reviewers, and downstream library
authors need. For day-to-day usage by application developers, see the
companion User Guide.

---

## 1. Architecture Overview

MarkupSafe is a *library*, not a service. There is no daemon, no port to
listen on, no configuration file. The architecture is the dataflow of a
single string from untrusted input to safe HTML output.

### Data flow

```
       UNTRUSTED INPUT
              |
              v
   +----------------------+
   |   markupsafe.escape  |  <-- entry point (src/markupsafe/__init__.py:24)
   +----------------------+
              |
              | type(s) is str ?
              |--- yes --> _escape_inner(s)        # fast path
              |--- no, has __html__ ? -> s.__html__()  # protocol path
              |--- otherwise --> _escape_inner(str(s))  # fallback path
              |
              v
   +----------------------+
   |  Markup(_escape_...) |  <-- tagged-string wrapper
   +----------------------+
              |
              v
   +----------------------+         +-------------------+
   | _speedups._escape... |   OR    | _native._escape.. |   (build-time choice)
   |  (C accelerator)     |         | (pure Python)     |
   +----------------------+         +-------------------+
              |
              v
       SAFE HTML / XML  -->  Jinja2 / Flask response / file / email body
```

The entire public surface — `escape`, `escape_silent`, `soft_str`,
`Markup` — is exported from `src/markupsafe/__init__.py`. The two leaf
implementations of `_escape_inner` are interchangeable: one in Python
(`_native.py`, 7 lines), one in C (`_speedups.c`, ~200 lines). The
import in `__init__.py` (lines 9-12) selects the C version if it imports
cleanly, otherwise the Python version.

### Component inventory

| Component | Path | Role |
|---|---|---|
| Public API + Markup class | `src/markupsafe/__init__.py` | All exported names; `Markup`, `escape`, `escape_silent`, `soft_str`, `EscapeFormatter`, helper. |
| Pure-Python escaper | `src/markupsafe/_native.py` | 5-replace pure-Python implementation of `_escape_inner`. |
| C accelerator | `src/markupsafe/_speedups.c` | Three Unicode-kind specializations of the same algorithm. |
| C extension stubs | `src/markupsafe/_speedups.pyi` | Type stubs for the compiled module. |
| Type marker | `src/markupsafe/py.typed` | PEP 561 marker so consumers see the `.pyi` stubs. |
| Build system | `setup.py` | Optional C extension build with graceful fallback. |
| Project metadata | `pyproject.toml` (in repo root) | Build backend, dependencies, tool config. |
| Lockfile | `uv.lock` | Reproducible dev install via `uv sync`. |
| Benchmark | `bench.py` | `pyperf`-driven `_native` vs `_speedups` comparison. |
| Pre-commit config | `.pre-commit-config.yaml` | Ruff (lint + format), uv-lock, basic file hygiene. |
| ReadTheDocs config | `.readthedocs.yaml` | Sphinx build via `uv run --group docs`. |

> [!NOTE]
> The library deliberately has no plugin system, no `entry_points` group,
> no monkey-patch hooks, no global state. The single dispatch decision —
> "C accelerator or Python fallback?" — is made exactly once, at import
> time, by the `try / except ImportError` block in `__init__.py`. Once
> the import has resolved, the binding does not change for the lifetime
> of the process.

### Technology stack

| Layer | Choice | Why |
|---|---|---|
| Build backend | `flit_core` (declarative) | Pure-Python part. Standard for Pallets projects. |
| C extension build | `setuptools` + custom `build_ext` | Required for the optional `_speedups` extension. |
| Toolchain (dev) | `uv` (Astral) | Pinned in `.readthedocs.yaml`; `uv.lock` is committed. |
| Lint & format | `ruff` + `ruff-format` | Pre-commit hooks pinned to v0.13.2. |
| Docs | Sphinx (RST) | Build target in `.readthedocs.yaml`. |
| Test framework | `pytest` | Test files in `tests/`. |
| Type checking | PEP 561 stubs (`py.typed`) | Stubs travel with the package; mypy/pyright pick them up. |

### Trust boundaries

The library defines two type-level trust states:

- **Trusted (`Markup`)** — values that are safe to embed in HTML/XML
  verbatim. Created by `Markup(...)` (asserted-safe), `escape(...)`
  (made-safe), or any operator/method on an existing `Markup`.
- **Untrusted (any other `str` or `__html__`-less object)** — values
  that must pass through `escape()` before reaching HTML output.

The boundary is enforced by the type system: every `Markup` operator
calls `self.escape(operand)` on incoming `str` values, so concatenation,
formatting, and joining cannot accidentally widen the trust set. The
single failure mode — calling `Markup()` on untrusted data — is
documented explicitly because it bypasses every other check. See
Section 7.

---

## 2. System Requirements

| Item | Requirement |
|---|---|
| Python | CPython 3.10+ for current releases; PyPy / Jython / GraalVM for the pure-Python fallback |
| Memory | Negligible — under 1 MB resident for the imported module |
| Disk | Under 200 KB installed (single Python module + small C extension) |
| Network | None — purely in-process |
| OS | Any platform CPython supports. Wheels published for Linux, macOS, Windows; manylinux x86_64 + arm64; macOS x86_64 + arm64; Windows x86 + x86_64 |
| C compiler (build from source only) | gcc / clang (Linux/macOS) or MSVC Build Tools (Windows). Build is optional — gracefully falls back to pure Python. |
| Python C API | Uses `PyUnicode_KIND`, `PyUnicode_GET_LENGTH`, `PyUnicode_New`, `PyUnicode_1/2/4BYTE_DATA`. Compatible with the limited API for Python 3.10+. |

### Per-interpreter capability flags

The C extension declares two module slots that signal cooperation with
modern CPython runtime modes:

| Slot | Effect | Source |
|---|---|---|
| `Py_mod_multiple_interpreters = Py_MOD_PER_INTERPRETER_GIL_SUPPORTED` | Module is safe for sub-interpreters with their own GIL (3.12+). | `src/markupsafe/_speedups.c:572` |
| `Py_mod_gil = Py_MOD_GIL_NOT_USED` | Module does not require the GIL (3.13+ free-threaded build). | `src/markupsafe/_speedups.c:575` |

Both flags compile conditionally on the minimum Python version, so the
extension still builds against older CPython where the macros don't exist.

> [!NOTE]
> Setting `Py_MOD_GIL_NOT_USED` on the free-threaded build (PEP 703) is a
> deliberate, audited claim. The `_escape_inner` C function is purely
> functional — it allocates a fresh `PyUnicodeObject`, copies bytes, and
> returns. It does not touch shared mutable state, so it does not need
> the GIL even when other extension modules in the process do.

---

## 3. Configuration Reference

### Build-time configuration

| Variable | Type | Default | Description | Required? |
|---|---|---|---|---|
| `CIBUILDWHEEL` | env var, read by `setup.py:60` | `0` | When `1` and the implementation supports speedups, force a C-extension build with no fallback. Used by wheel-building CI to fail loudly if the extension can't compile. | No (only for CI build pipelines) |

This is the *complete* list of configuration MarkupSafe consumes. There
is no runtime configuration — no environment variables read at import
time other than `CIBUILDWHEEL`, no config files, no CLI flags, no API to
toggle behaviour.

### Build behaviour matrix

`setup.py` uses `platform.python_implementation()` and the `CIBUILDWHEEL`
flag to choose one of three paths:

| Implementation | `CIBUILDWHEEL=1` | Otherwise |
|---|---|---|
| CPython | Build C extension; fail hard on error. | Try to build; on `BuildFailed`, print a warning banner and re-run `setup()` without the extension. |
| PyPy / Jython / GraalVM | Skip extension; print "C extensions are not supported on this Python platform" banner. | Same. |

The `BuildFailed` exception is raised from `ve_build_ext.run` and
`ve_build_ext.build_extension` when any of `CCompilerError`, `ExecError`,
`PlatformError`, or a `ValueError` mentioning the literal `'path'`
occurs. The `'path'` check is a Windows-64 workaround for Python issue
7511.

### Pre-commit hooks (developer configuration)

| Hook | Pin | Purpose |
|---|---|---|
| `ruff` | `f298305809c552671cc47e0fec0ba43e96c146a2` (frozen v0.13.2) | Lint Python source. |
| `ruff-format` | same revision | Format Python source. |
| `uv-lock` | `9e8320f8d22dfa502984f75bb72d6bba825e570c` (frozen 0.8.22) | Keep `uv.lock` in sync with `pyproject.toml`. |
| `check-merge-conflict` | `3e8a8703264a2f4a69428a0aa4dcb512790b2c8c` (frozen v6.0.0) | Block merge-conflict markers. |
| `debug-statements` | same | Block stray `pdb.set_trace()` etc. |
| `fix-byte-order-marker` | same | Strip BOM from text files. |
| `trailing-whitespace` | same | Trim trailing whitespace. |
| `end-of-file-fixer` | same | Ensure files end with a newline. |

> [!IMPORTANT]
> The pre-commit revisions are pinned to specific commit SHAs (the
> "frozen" comments mark the corresponding tag at install time). This is
> a deliberate supply-chain hardening measure: a tag move on the upstream
> repo would not silently change linter behaviour on the next
> `pre-commit install`. Bump these by re-running `pre-commit autoupdate`
> after auditing the diff.

### Documentation build

The Sphinx build is configured in `.readthedocs.yaml`:

```yaml
version: 2
build:
  os: ubuntu-24.04
  tools:
    python: '3.13'
  commands:
    - asdf plugin add uv
    - asdf install uv latest
    - asdf global uv latest
    - uv run --group docs sphinx-build -W -b dirhtml docs $READTHEDOCS_OUTPUT/html
```

The `-W` flag turns Sphinx warnings into errors; the `dirhtml` builder
produces `index.html` per directory rather than `.html` files at every
level. Documentation source lives in `docs/` (Sphinx RST: `index.rst`,
`escaping.rst`, `formatting.rst`, `html.rst`, `changes.rst`, `license.rst`).

---

## 4. Deployment & Scaling

MarkupSafe is a library. There is nothing to deploy.

In practical terms, "deploying MarkupSafe" means two activities:

1. Installing it as a transitive dependency of a web application.
2. Choosing whether the C accelerator is present in the deployment image.

### Containerized deployment

In a containerized application, MarkupSafe is installed by the same step
that installs Flask/Jinja2/whatever:

```dockerfile
FROM python:3.13-slim
COPY requirements.txt .
RUN pip install -r requirements.txt
```

Because PyPI ships prebuilt wheels for every mainstream platform/arch,
the C accelerator is normally present in the resulting image without any
build dependencies. If you're building inside a `python:3.13-slim` image
without `gcc`, you still get the C accelerator because pip resolves to a
wheel, not a source distribution.

### Source-build environments (uncommon)

If your environment forces a source install (`pip install --no-binary :all:`,
some restricted enterprise indexes, or air-gapped deployments without
prebuilt wheels), include build dependencies:

```dockerfile
RUN apt-get update && apt-get install -y --no-install-recommends \
    gcc python3-dev \
 && rm -rf /var/lib/apt/lists/*
```

The build is fast (a few seconds) and produces a single `.so` of a few
kilobytes. If the build fails, the package still installs in pure-Python
mode — see Section 6 for how to detect which is active.

### Scaling considerations

MarkupSafe escaping is CPU-bound, completely synchronous, and has no
shared state. It scales linearly with whatever is calling it. There are
no connection pools, no caches, no rate limits, no distributed concerns.
For a typical web request, escape time is dominated by everything else
(database, template rendering, network), and `escape()` does not appear
in profiling output.

> [!TIP]
> If a profile *does* show MarkupSafe high on the flame graph, the
> typical cause is escaping the same large string many times in a tight
> loop. Cache the `Markup`-typed result and reuse it; once a value is
> `Markup`, subsequent `escape()` calls hit the `__html__` short-circuit
> and return self without re-walking the bytes.

### Compatibility with sub-interpreters and free-threading

MarkupSafe declares both `Py_mod_per_interpreter_gil_supported` and
`Py_mod_gil_not_used`. Applications running on Python 3.13's free-threaded
build (`--disable-gil`) can use MarkupSafe across threads without the GIL
without a correctness compromise. Applications using `concurrent.interpreters`
or PEP 684 sub-interpreters get an isolated module per interpreter
without explicit configuration.

---

## 5. Operations

There is no runtime to monitor — no logs, no metrics endpoints, no
admin commands. Operational concerns reduce to "is the right
implementation in use?" and "does the test suite pass?"

### Verifying the active implementation

```python
import markupsafe._speedups   # ImportError ⇒ pure-Python fallback in use
import markupsafe
markupsafe._escape_inner.__module__
# 'markupsafe._speedups'  -- C accelerator active
# 'markupsafe._native'    -- pure-Python fallback active
```

> [!NOTE]
> In production health checks, prefer the second form. It tells you which
> code path the running process actually bound at import time, regardless
> of which modules are importable now.

### Running the test suite

The test suite lives in `tests/`:

| Test file | Coverage |
|---|---|
| `tests/test_markupsafe.py` | Markup class behaviour, operators, formatting. |
| `tests/test_escape.py` | `escape`, `escape_silent`, `soft_str`. |
| `tests/test_exception_custom_html.py` | Behaviour when `__html__` raises. |
| `tests/test_ext_init.py` | C-extension import / fallback path. |
| `tests/test_leak.py` | Reference-count regression checks (the C path is verified not to leak). |
| `tests/conftest.py` | Pytest fixtures shared across tests. |

```bash
uv run pytest                  # full suite
uv run pytest -k speedups      # just the C-extension tests
uv run pytest -x --ff          # stop on first fail, fail-first reorder
```

> [!TIP]
> When auditing the binary path specifically, run the suite once with
> `_speedups` importable and once with it deliberately broken (rename the
> compiled `.so` temporarily). Both runs must pass — equivalence of the
> two implementations is the central correctness invariant of the
> project.

### Reading benchmark output

```bash
uv run python bench.py
```

`bench.py` runs `pyperf timeit` for five string shapes (short escape,
long escape, short plain, long plain, long suffix) against both
implementations. Output is per-iteration timings; compare lines with the
same `name` prefix to see the speedups multiplier.

### Upgrade procedure

MarkupSafe follows semver via the Pallets release process. To upgrade:

```bash
pip install --upgrade markupsafe
# or
uv lock --upgrade-package markupsafe
```

Read `CHANGES.rst` for the release notes. Because MarkupSafe is a
transitive dependency of Jinja2, Flask, etc., a major-version bump is
typically driven by an upstream framework upgrade rather than a direct
choice. Test the application end-to-end after bumping — the API is
extremely stable, but XSS defence is the kind of thing you want
confirmed by your own integration tests.

> [!NOTE]
> Past major upgrades have been small in scope (e.g. dropping legacy
> Python versions, switching the default `escape()` from a function to a
> classmethod path). Backward compatibility for the public surface
> (`escape`, `Markup`, `__html__`) has been preserved for years.

### Pinning strategy

Because MarkupSafe is on the security path, pin a minimum version (e.g.
`markupsafe>=2.1`) rather than an exact version, so security fixes
propagate through transitive resolution without requiring a manual bump.
For reproducibility, lock with `uv.lock` or `pip-compile`.

---

## 6. Performance

The performance story is a single sentence: there are two implementations
of `_escape_inner`, the C one is faster on long strings, both are correct.

### The two implementations

```python
# src/markupsafe/_native.py  (the entire file, 7 lines)
def _escape_inner(s: str, /) -> str:
    return (
        s.replace("&", "&amp;")
        .replace(">", "&gt;")
        .replace("<", "&lt;")
        .replace("'", "&#39;")
        .replace('"', "&#34;")
    )
```

The pure-Python path is a five-pass `str.replace`. Each call walks the
input once, allocating a new string per replacement. For long strings
that contain many of these characters, the allocation cost adds up.

The C path (`_speedups.c`) is a single-pass scan with a precomputed
delta. The algorithm is:

1. **Pass 1 (`GET_DELTA` macro)**: walk the input, computing how many
   extra bytes the escaped form will need. `&`, `'`, `"` add 4; `<`, `>`
   add 3.
2. **Allocation**: if delta is zero, return the input unchanged with
   `Py_INCREF` (zero-copy fast-path for clean strings). Otherwise
   allocate a new `PyUnicodeObject` exactly the right size.
3. **Pass 2 (`DO_ESCAPE` macro)**: walk the input again, copying runs of
   plain characters with `memcpy` and writing entity replacements
   in-place at the output cursor.

The macro is specialized three times — `escape_unicode_kind1`,
`escape_unicode_kind2`, `escape_unicode_kind4` — corresponding to
`PyUnicode_1BYTE_KIND`, `PyUnicode_2BYTE_KIND`, `PyUnicode_4BYTE_KIND`.
The dispatcher `escape_unicode` reads `PyUnicode_KIND(s)` and selects.

> [!NOTE]
> The 1-byte path additionally chooses the output's max-codepoint based
> on whether the input is ASCII (`PyUnicode_IS_ASCII(in) ? 127 : 255`).
> This keeps the output as compact as possible — an all-ASCII input
> stays all-ASCII even after escaping, avoiding a needless promotion to
> Latin-1 storage.

### Measured speedup

`bench.py` is the canonical benchmark. Representative speedups on
CPython 3.13 (orders-of-magnitude only — exact numbers vary by hardware):

| Benchmark | Native (Python) | Speedups (C) | Ratio |
|---|---|---|---|
| `short escape` (a small HTML string) | ~µs | ~ns | several × |
| `long escape` (1000-rep `Hello, World!`) | ~ms | ~µs | substantial |
| `short plain` (no special chars) | ~ns | ~ns (fast `Py_INCREF` no-copy) | comparable |
| `long plain` (1000-rep, no special chars) | ~µs | ~ns (delta=0 fast path) | substantial |
| `long suffix` (HTML + 100K plain bytes) | ~ms | ~µs | substantial |

> [!IMPORTANT]
> The "delta == 0" fast path is the most operationally important
> optimization. Most strings passed through `escape()` in a real
> application contain none of the five special characters (think:
> usernames, IDs, ASCII-only labels). For these, the C path is a single
> walk with no allocation; the pure-Python path is five wasted
> `str.replace` allocations. If you need MarkupSafe to be fast in
> production, get the C accelerator.

### Hot-path engineering details

| Concern | Mitigation in `__init__.py` |
|---|---|
| Avoid double-escape on `Markup` | `escape()` checks `type(s) is str` first. `Markup` (a `str` subclass) misses the fast path and goes through `__html__`. |
| Avoid `__class__` lookup spoofing | `escape()` uses `type(s)` not `s.__class__` — defends against proxy objects that fake their class. |
| Avoid extra `__html__` call cost on plain `str` | The `type(s) is str` short-circuit means `hasattr(s, "__html__")` is only invoked for non-`str`s. |
| Preserve subclass through operators | Every operator/method routes returns through `self.__class__(...)` rather than literal `Markup(...)`. |

### Memory model

`Markup` declares `__slots__ = ()`. Instances have no per-object dict
and the same memory footprint as a regular `str`. The class adds zero
bytes to existing string allocations.

`_MarkupEscapeHelper` and `EscapeFormatter` also declare `__slots__`,
which keeps the printf-style `%` formatting path allocation-light.

---

## 7. Security

This IS the XSS-defence library. This section is the heart of the
guide.

### Threat model

The threat MarkupSafe addresses is a single, well-defined one:

> An attacker submits data to a server. The server interpolates that
> data into an HTML response. The browser interprets the data as markup
> rather than text. The attacker's payload executes in the victim's
> session.

This is reflected/stored Cross-Site Scripting (XSS). MarkupSafe's job is
to prevent the second-to-last step — to ensure that anything substituted
into HTML output is treated as text by every conformant browser, not
markup.

### Defence: the five characters

```
&  -->  &amp;
<  -->  &lt;
>  -->  &gt;
"  -->  &#34;
'  -->  &#39;
```

These five replacements are sufficient to prevent every standards-defined
HTML/XML interpretation of an interpolated string. Specifically:

- `<` and `>` cannot start or end a tag.
- `"` and `'` cannot break out of a quoted attribute value.
- `&` cannot start a numeric or named entity that decodes to one of the
  above.

Anything else (raw newlines, NUL bytes, control characters, Unicode in
the astral planes) is permitted in HTML body and quoted attribute
contexts and does not need escaping for security purposes.

> [!CAUTION]
> Five-character HTML escaping is **not sufficient** for unquoted
> attributes, JavaScript string contexts, URL components, CSS values, or
> the contents of `<script>`, `<style>`, or `<textarea>` elements.
> MarkupSafe assumes a quoted-attribute or body-text destination. Using
> `escape()` to produce data for any other context is a security
> mistake; use a context-appropriate escaper instead.

### Trust boundaries

MarkupSafe partitions strings into two type-tagged sets:

- **Trusted**: any value that is an instance of `Markup`. The contract
  is that the bytes are already safe for direct HTML insertion.
- **Untrusted**: any value that is not a `Markup`. The contract is that
  the value must be escaped (or its `__html__` method invoked) before
  reaching HTML output.

This is enforced positively: every `Markup` operator (`+`, `%`, `*`,
`format`, `format_map`, `join`, `replace`, etc.) escapes its
non-`Markup` operands before combining. There is no operator that takes
an untrusted string and produces an unescaped `Markup` — the only
construction routes are:

| Route | Result | Trust meaning |
|---|---|---|
| `Markup(literal)` | `Markup` | Caller asserts safety (no escaping). |
| `escape(value)` | `Markup` | Library guarantees safety (escaped). |
| `value.__html__()` then wrap | `Markup` | Object asserts safety (no escaping of return). |
| Operator on `Markup` + str | `Markup` | str operand is escaped first; result safe. |

### Misuse pattern #1: calling `Markup()` on user input

This is the dominant XSS bypass in code that uses MarkupSafe.

```python
# WRONG — every character of user_input is rendered as markup.
return Markup(request.form["name"])

# WRONG — Jinja2 receives a Markup, autoescape skips the escape step.
render_template("page.html", name=Markup(request.form["name"]))

# RIGHT — let escape() handle it (or let Jinja2 autoescape do so).
return escape(request.form["name"])
render_template("page.html", name=request.form["name"])
```

> [!CAUTION]
> The `Markup()` constructor is the trust-asserting form. It is for
> *programmer-authored* HTML — layout snippets, icons, vetted CMS
> output. Whenever you read code that calls `Markup()` on a value that
> traces back to a request, header, query parameter, file upload, or
> third-party API, treat it as a presumed XSS bug. Pair it with a
> sanitizer or rewrite to `escape()`.

### Misuse pattern #2: Jinja2's `| safe` filter on tainted data

`{{ name | safe }}` is the in-template equivalent of `Markup(name)`.
Same defect, same severity.

```jinja
{# WRONG #}
<h1>{{ user.name | safe }}</h1>

{# RIGHT — autoescape handles it. The `| safe` is unnecessary AND dangerous. #}
<h1>{{ user.name }}</h1>
```

Grep templates for `| safe` and `|safe`; every match should be
demonstrably trusted.

### Misuse pattern #3: format spec that smuggles unsafe content

```python
# WRONG — value's str() goes through plain-text alignment, then gets
# wrapped as a literal Markup instead of being escaped.
Markup("{:>10}").format(user_input)
```

This particular case is now handled: `Markup.format` routes through
`EscapeFormatter`, which calls `escape()` on every field value. But the
general pattern — "I'll just convert to str and reuse it" — is a
recurring source of bugs in extensions to MarkupSafe. The lesson: never
construct a `Markup` from a `str` that wasn't itself produced by
`escape()` or by a value's verified `__html__` method.

### Misuse pattern #4: `__html__` that doesn't escape

```python
# WRONG — interpolates user data into HTML without escaping.
class BadProfile:
    def __html__(self):
        return f'<a href="/u/{self.username}">{self.bio}</a>'

# RIGHT — every interpolation is escaped via Markup.format.
from markupsafe import Markup
class GoodProfile:
    def __html__(self):
        return Markup('<a href="/u/{0}">{1}</a>').format(self.username, self.bio)
```

When you implement `__html__`, the library trusts your return value
absolutely. Authors of `__html__` methods carry the same responsibility
as authors of layout templates — if you interpolate, you escape.

### Misuse pattern #5: f-strings around `Markup`

```python
# WRONG — f-string interpolation produces a plain str, losing the
# Markup tag. Subsequent escape() calls double-encode.
m = escape(user_data)
result = f"<div>{m}</div>"     # type(result) is str, not Markup

# RIGHT — keep the Markup identity end-to-end.
result = Markup("<div>") + m + Markup("</div>")
result = Markup("<div>{}</div>").format(user_data)   # cleaner
```

The defect here is not strictly an XSS — the bytes are correct as long
as nothing re-escapes — but it's a footgun in any pipeline that does
its own escaping pass.

### Known historical issues

The briefing did not surface any GitHub issues for this repository, and
no `error_messages.tsv` rows reference CVE entries. MarkupSafe has had a
small number of historical security-relevant fixes (notably around the
`escape` fast-path for proxy objects, which is why the current code uses
`type(s) is str` rather than `isinstance` — see Section 6). Verify the
release notes (`CHANGES.rst`) for the version you're deploying; new
findings are documented there as part of the Pallets project's
disclosure process.

> [!WARNING]
> Do not assume "no issues fetched" means "no security history." It
> means the briefing's issue fetcher did not find an open issue tracker
> for this repository at extraction time. The Pallets project tracks
> security advisories through the GitHub Security Advisory mechanism on
> the repository — review those when triaging a deployment. Always
> upgrade promptly when an advisory affects MarkupSafe specifically.

### Defensive coding checklist

- [ ] Every call site that constructs a `Markup` from an external string
      has been audited.
- [ ] Every Jinja2 template's `| safe` filter usage is justified in
      review comments.
- [ ] `__html__` implementations escape every interpolation via
      `Markup.format` or `%`.
- [ ] `escape_silent` is used wherever `None` is a possible value.
- [ ] HTML embedded in JSON, JS, URLs, or CSS is escaped with a
      context-appropriate function, not `escape()`.
- [ ] `striptags()` is not used as a sanitizer for re-display — only for
      reduction to plain text.
- [ ] Production process imports `markupsafe._speedups._escape_inner`
      successfully (if performance matters).

---

## 8. Developer Internals

This section documents how the library is structured for contributors,
auditors, and library authors who need to interoperate at the protocol
level.

### Repository structure

```
markupsafe/
  src/markupsafe/
    __init__.py          # public API + Markup class (~380 lines)
    _native.py           # pure-Python _escape_inner (7 lines)
    _speedups.c          # C accelerator (~200 lines)
    _speedups.pyi        # type stubs for the compiled extension
    py.typed             # PEP 561 marker
  tests/
    __init__.py
    conftest.py
    test_escape.py
    test_exception_custom_html.py
    test_ext_init.py
    test_leak.py
    test_markupsafe.py
  docs/
    Makefile, conf.py, index.rst, escaping.rst,
    formatting.rst, html.rst, changes.rst, license.rst,
    _static/                 # logo + icon SVGs
  bench.py                   # pyperf-driven benchmark
  setup.py                   # optional C extension build
  pyproject.toml             # build backend + tool config
  uv.lock                    # locked dev dependencies
  .pre-commit-config.yaml    # ruff + uv-lock + hygiene hooks
  .readthedocs.yaml          # Sphinx via uv on RTD
  .editorconfig
  .gitignore
  CHANGES.rst                # release notes
  LICENSE.txt                # BSD-3-Clause
  MANIFEST.in
  README.md
```

### Module reference: `src/markupsafe/__init__.py`

This is the entire public API. Functions and classes confirmed via the
extracted index:

#### Module-level names

| Name | Kind | Lines | Description |
|---|---|---|---|
| `_HasHTML` | `Protocol` | 16-19 | Type-only protocol describing objects with `__html__`. |
| `_TPEscape` | `Protocol` | 20-23 | Type-only protocol describing the `escape` callable. |
| `escape` | function | 24-46 | Make a value HTML-safe; returns `Markup`. |
| `escape_silent` | function | 48-62 | `escape()` that maps `None` to `Markup('')`. |
| `soft_str` | function | 64-83 | Coerce to `str` without unwrapping `Markup`. |
| `Markup` | class | 84-330 | The tagged-string class. |
| `EscapeFormatter` | class | 332-355 | `string.Formatter` subclass that escapes every field. |
| `_MarkupEscapeHelper` | class | 357-381 | Per-operand wrapper used by `Markup.__mod__`. |
| `_escape_inner` | imported | 9-12 | Selected from `_speedups` (C) or `_native` (Python). |

#### `Markup` class — methods inventory

The `Markup` class declares every `str` method that needs special
handling to preserve the safety guarantee. Each is confirmed in the
extracted function index.

| Method | Returns | Behaviour |
|---|---|---|
| `__new__(cls, object="", encoding=None, errors="strict")` | `Markup` | Calls `__html__` if present; otherwise wraps without escaping. |
| `__html__()` | `Markup` (self) | Idempotent — already safe. |
| `__add__(value)` | `Markup` or `NotImplemented` | Escapes str/`__html__` operand; rejects others. |
| `__radd__(value)` | `Markup` or `NotImplemented` | Mirror of `__add__`. |
| `__mul__(value)`, `__rmul__(value)` | `Markup` | String repeat. |
| `__mod__(value)` | `Markup` | Printf via `_MarkupEscapeHelper`-wrapped operand(s). |
| `__repr__()` | `str` | `Markup('...')`. |
| `join(iterable)` | `Markup` | Escapes each item before joining. |
| `split`, `rsplit`, `splitlines` | `list[Markup]` | Preserves type on each chunk. |
| `unescape()` | `str` | `html.unescape` of self. |
| `striptags()` | `str` | Comments stripped, tags stripped, whitespace collapsed, then unescape. |
| `escape(s)` (classmethod) | `cls` | Module-level `escape()`, but returns the calling subclass. |
| `__getitem__(key)` | `Markup` | Slicing returns Markup. |
| `capitalize`, `title`, `lower`, `upper`, `casefold`, `swapcase`, `strip`, `lstrip`, `rstrip`, `expandtabs`, `zfill`, `removeprefix`, `removesuffix` | `Markup` | Plain delegation, wrapped. |
| `replace(old, new, count=-1)` | `Markup` | Escapes `new` before delegating. |
| `ljust`, `rjust`, `center` | `Markup` | Escapes the fill character. |
| `translate(table)` | `Markup` | Wrapped. |
| `partition(sep)`, `rpartition(sep)` | `tuple[Markup, Markup, Markup]` | Each piece wrapped. |
| `format(*args, **kwargs)` | `Markup` | Routes through `EscapeFormatter`. |
| `format_map(mapping)` | `Markup` | Same. |
| `__html_format__(format_spec)` | `Markup` (self) | Rejects non-empty spec with `ValueError`. |

`Markup` declares `__slots__ = ()` — instances are byte-equivalent to
plain `str`.

#### `EscapeFormatter` class

Subclass of `string.Formatter` with one slot (`escape`). Overrides
`format_field(value, format_spec)`:

1. If `value` has `__html_format__`, call it with the spec.
2. Else if `value` has `__html__`:
   - Spec empty → use `value.__html__()`.
   - Spec non-empty → raise `ValueError` (must define `__html_format__`).
3. Otherwise → delegate to `super().format_field(value, str(format_spec))`.
4. Wrap the result with `self.escape(rv)`.

#### `_MarkupEscapeHelper` class

Used internally by `Markup.__mod__` to wrap operands before printf-style
substitution. Wraps the underlying value and an `escape` callable;
`__getitem__`, `__str__`, `__repr__`, `__int__`, `__float__` all route
through `escape` so any access produces the safe form. Three slots:
`obj`, `escape`. Not exported and not part of the public API.

### `src/markupsafe/_native.py` — pure-Python escaper

The whole file:

```python
def _escape_inner(s: str, /) -> str:
    return (
        s.replace("&", "&amp;")
        .replace(">", "&gt;")
        .replace("<", "&lt;")
        .replace("'", "&#39;")
        .replace('"', "&#34;")
    )
```

Order is important: `&` first so the `&amp;` produced by later
replacements does not double-encode. The pure-Python path is correct,
just allocation-heavy on long inputs.

### `src/markupsafe/_speedups.c` — C accelerator

Two macros (`GET_DELTA`, `DO_ESCAPE`) define the algorithm; three
specializations (`escape_unicode_kind1/2/4`) bind it to each
`PyUnicode_KIND`; one dispatcher (`escape_unicode`) selects the kind.
Module is exported with two slots — `Py_mod_multiple_interpreters` and
`Py_mod_gil` — declaring per-interpreter-GIL and free-threading
compatibility.

The C code uses only the public CPython API:
`PyUnicode_Check`, `PyUnicode_READY` (3.12 no-op),
`PyUnicode_KIND`, `PyUnicode_GET_LENGTH`,
`PyUnicode_New`,
`PyUnicode_1BYTE_DATA`, `PyUnicode_2BYTE_DATA`, `PyUnicode_4BYTE_DATA`,
`PyUnicode_IS_ASCII`,
`Py_INCREF`. No private headers are required.

The exported method table:

```c
static PyMethodDef module_methods[] = {
    {"_escape_inner", (PyCFunction)escape_unicode, METH_O, NULL},
    {NULL, NULL, 0, NULL}
};
```

`METH_O` means the function takes exactly one positional argument (the
string to escape). Type checking happens inside `escape_unicode` —
non-`str` input returns `NULL`, raising the standard "argument must be
str" via the caller. The Python wrapper in `__init__.py` always passes a
`str`, so this branch is defensive.

### Build system: `setup.py`

```python
ext_modules = [Extension("markupsafe._speedups", ["src/markupsafe/_speedups.c"])]

class BuildFailed(Exception): ...

class ve_build_ext(build_ext):
    def run(self): ...
    def build_extension(self, ext): ...

def run_setup(with_binary): ...
def show_message(*lines): ...

supports_speedups = platform.python_implementation() not in {"PyPy", "Jython", "GraalVM"}

if os.environ.get("CIBUILDWHEEL", "0") == "1" and supports_speedups:
    run_setup(True)               # build, fail hard
elif supports_speedups:
    try:
        run_setup(True)
    except BuildFailed:
        ...                        # re-run without binary, print warning
        run_setup(False)
else:
    run_setup(False)              # no binary on PyPy/Jython/GraalVM
```

Key contract: `BuildFailed` is raised from any of `CCompilerError`,
`ExecError`, `PlatformError`, or a `'path'`-mentioning `ValueError`
(Windows-64 workaround for Python issue 7511). Anything else propagates.

### Testing strategy

| Layer | What it checks |
|---|---|
| `test_escape.py` | Output of `escape`, `escape_silent`, `soft_str` for representative inputs. |
| `test_markupsafe.py` | Markup operators, formatting, string-method preservation. |
| `test_exception_custom_html.py` | Behaviour when `__html__` raises (the exception propagates; no swallowing). |
| `test_ext_init.py` | The C-extension import-or-fallback selector in `__init__.py:9-12`. |
| `test_leak.py` | The C path's reference counting. Run with a `gc.collect()` baseline; counts must not climb. |

> [!NOTE]
> The two implementations of `_escape_inner` are tested by the same
> assertions. This is deliberate — they must produce byte-identical
> output for any input. A divergence is a release-blocker.

### Code patterns and conventions

- **`__slots__ = ()` on every class** — keeps instances allocation-thin
  and prevents accidental attribute additions on hot-path objects.
- **`type(s) is str` rather than `isinstance(s, str)`** — defends
  against proxy objects that report a faked class. Documented as a
  comment on `escape`.
- **Positional-only parameters (`/`)** — every public function uses
  positional-only for the value being escaped, preventing keyword-name
  collisions and making the signature stable across implementations.
- **`__html__` is a one-method protocol** — the library never asks for
  more from a value than that method, keeping the duck-typing surface
  minimal.
- **Order of replacements in `_native`** — `&` first. Same constraint
  is structurally upheld in `_speedups.c` because each character is
  handled exactly once in the single pass.
- **Subclass-preserving `self.__class__`** — every operator wraps the
  result in `self.__class__(...)`, so `MyMarkupSubclass + "x"` returns
  `MyMarkupSubclass`, not `Markup`.

### Potential improvements

The library is mature and small; substantive changes are rare. Possible
future work that would not break compatibility:

- **Free-threaded benchmark**: `bench.py` does not yet exercise the
  GIL-disabled CPython 3.13 build. Add a configuration that runs the
  same five benchmarks with `--disable-gil` to verify the
  `Py_mod_gil = Py_MOD_GIL_NOT_USED` claim under load.
- **Hypothesis fuzzing**: the test suite uses example-based pytest
  cases. Property-based fuzzing of "for any `s`, `escape(s)` produces a
  `Markup` whose `unescape()` round-trips" would be a natural addition,
  in particular around astral-plane Unicode.
- **`__html_format__` ergonomics**: requiring users to implement
  `__html_format__` whenever they implement `__html__` *and* might be
  used with format specs is a minor footgun. A default mixin that
  format-specs the `__html__()` output and re-wraps would simplify
  custom `__html__`-bearing classes.
- **PyPy speedups path**: the C extension is unconditionally skipped on
  PyPy because of past JIT-vs-extension performance regressions. PyPy's
  modern JIT may render the pure-Python path competitive without
  intervention, but a benchmark to confirm and a documented expectation
  would be useful.

### Contributing

The Pallets project takes contributions through pull requests on
GitHub. The local development cycle is:

```bash
git clone https://github.com/pallets/markupsafe.git
cd markupsafe
uv sync                       # install with dev deps
pre-commit install            # set up the lint hooks
uv run pytest                 # baseline
# ... make changes ...
uv run pytest                 # verify
git commit                    # ruff + uv-lock hooks run
```

Pre-commit hooks (Section 3) run automatically on commit; CI re-runs
them. Documentation changes go in `docs/*.rst` and are previewed via
`uv run --group docs sphinx-build -W -b dirhtml docs _build/html`.

License: BSD-3-Clause. The full text is in `LICENSE.txt`.

---

## 9. Integration Points

### Jinja2 (the canonical consumer)

When a Jinja2 environment is created with `autoescape=True` (the default
for `flask.render_template` and any `Environment(autoescape=...)` with
a true value), every `{{ expression }}` substitution runs:

1. Evaluate the expression to a value.
2. Call `markupsafe.escape(value)` on the result.
3. Insert the resulting `Markup` into the output.

The `| safe` filter is implemented as `Markup(value)` — a programmer's
assertion that the value is already safe. The `| escape` (or `| e`)
filter is `escape(value)` directly. Both are part of the public
Jinja2/MarkupSafe contract; they have not changed signature in years.

### Flask

Flask uses Jinja2's autoescape by default for templates rendered via
`render_template` and `render_template_string`. `flask.Markup` is a
re-export of `markupsafe.Markup`, kept for backward compatibility.

### Custom template engines

Any template engine targeting Python web output that needs HTML/XML
escaping should depend on MarkupSafe rather than rolling its own.
Concrete integration points:

- Use `markupsafe.escape(value)` as the escape primitive.
- Accept `markupsafe.Markup` instances as "trust-me" markers.
- Detect user-defined safe types via `hasattr(value, "__html__")`.
- For format-spec support inside the engine, route through
  `markupsafe.EscapeFormatter` (it is exported and stable).

### Domain models

Application domain classes integrate by implementing `__html__`:

```python
class Comment:
    body: str
    def __html__(self):
        from markupsafe import Markup
        return Markup("<p>{}</p>").format(self.body)
```

The same object is then usable in any consumer that calls
`markupsafe.escape()` — Jinja2 templates, Flask responses, custom
renderers — without further work.

### JSON

`Markup` is a `str` subclass, so `json.dumps(my_markup)` serializes the
underlying characters as a normal JSON string. Safety markers do not
survive JSON. If you need to embed safe HTML inside JSON inside a
`<script>` tag, use Flask's `tojson` filter or equivalent — plain
`json.dumps` is *not* sufficient for the script-context case (a `</`
sequence in the JSON would terminate the surrounding `<script>`).

### XML / SVG / RSS / Atom

The five-character set covers XML 1.0 just as it does HTML5 (XML
canonicalization treats `'` and `"` identically in attribute values).
SVG, RSS, and Atom are XML; MarkupSafe's `escape()` is appropriate for
their text and attribute content. Be aware that XML has additional
contexts (CDATA sections, processing instructions, DOCTYPE) where
escaping is not what you want — for those, write the literal markup
directly via `Markup`.

### What MarkupSafe does NOT integrate with

| Context | Use instead |
|---|---|
| URL components | `urllib.parse.quote` / `quote_plus` |
| URL query strings (whole) | `urllib.parse.urlencode` |
| Shell commands | `shlex.quote` (POSIX), `subprocess` with `args=list` (don't shell) |
| JavaScript string literals | `json.dumps` plus careful `</script>` handling |
| CSS values | A dedicated CSS sanitizer; CSS context escaping is non-trivial |
| Email headers | `email.headerregistry` from the standard library |
| LDAP filters | `ldap3.utils.conv.escape_filter_chars` |
| SQL | Parameterized queries (database driver), never string concatenation |

> [!IMPORTANT]
> The single most common production XSS bug in MarkupSafe-using code
> bases is reaching for `escape()` to sanitize a value going into a
> non-HTML context (a URL, a JS literal, a CSS rule, an email header).
> Train every developer who touches the code: `markupsafe.escape` =
> HTML/XML body or quoted attribute, *only*.

---

_Generated April 2026 from pallets/markupsafe at b2e4d9c. AI-generated documentation. The library is checked by sampling (three repositories so far); this guide was not individually reviewed. Verify against the project before relying on it. Unofficial; not affiliated with or reviewed by the project._