CPython — Administrator & Developer Guide

CPython is the reference implementation of the Python programming language — the canonical interpreter that defines what "Python" means. This guide covers everything an administrator, packager, platform integrator, or core developer needs: building from source, configuring the interpreter and build system, deploying across six supported platforms, operating at scale, performance tuning, security hardening, and deep internals of the compiler pipeline, object model, memory allocator, and JIT. See User Guide for everyday usage.

1. Architecture Overview

1.1 High-Level Architecture

CPython is a bytecode-compiling interpreter with an optional JIT backend. Source code flows through a well-defined pipeline:

Source Text
    │
    ▼
Tokenizer (Parser/tokenizer/, Parser/lexer/)
    │
    ▼
PEG Parser (Parser/) → Concrete Syntax Tree
    │
    ▼
AST Compiler (Python/ast.c, Python/ast_opt.c)
    │
    ▼
Bytecode Compiler (Python/compile.c, Python/codegen.c, Python/flowgraph.c)
    │
    ▼
Code Objects (Objects/codeobject.c)
    │
    ▼
Evaluation Loop (Python/ceval.c, Python/generated_cases.c.h)
    │                          │
    ▼                          ▼
Standard Interpreter      JIT Compiler (Tools/jit/ → native code)
    │
    ▼
Object Model (Objects/*.c) ←→ Memory Allocator (Objects/obmalloc.c, Objects/mimalloc/)
    │
    ▼
C Extension Modules (Modules/*.c) ←→ Standard Library (Lib/*.py)

The interpreter core is written in C (approximately 14M characters of C source across Python/, Objects/, Modules/, Parser/, Include/, and Programs/). The standard library adds another 10M+ characters of Python code in Lib/.

Note

CPython's PEG parser (introduced in Python 3.9, sole parser since 3.10) replaced the legacy LL(1) parser. The grammar definition lives in Grammar/python.gram and is processed by Tools/peg_generator/ to produce Parser/parser.c. The tokenizer was refactored into separate Parser/tokenizer/ and Parser/lexer/ subdirectories.

1.2 Core Subsystems

1.3 Platform Support Matrix

CPython officially supports six platform families:

1.4 Build System Architecture

CPython uses two distinct build systems:

POSIX (Linux, macOS, Android, iOS, WASM):

configure.ac → autoconf → configure → Makefile.pre.in → Makefile → make
                                         │
                                    pyconfig.h (generated)

Windows:

PCbuild/*.vcxproj → MSBuild → python.exe, pythonXY.dll
    │
    ├── get_external.py (downloads dependencies: OpenSSL, libffi, sqlite, etc.)
    └── PCbuild/build.bat (wrapper)

1.5 Memory Management Architecture

CPython uses a layered memory architecture:

Layer 3:  Object-specific allocators (e.g., PyUnicode, PyLong small-int cache)
Layer 2:  Python object allocator (obmalloc.c — arena-based, or mimalloc)
Layer 1:  Python raw allocator (wraps malloc/free with hooks)
Layer 0:  System allocator (libc malloc, or platform equivalent)

The mimalloc allocator (Objects/mimalloc/) is an integrated copy of Microsoft's mimalloc, providing significant performance improvements for multi-threaded workloads, especially under the free-threading (no-GIL) build.

1.6 Free-Threading Architecture

CPython now includes experimental free-threading support (building without the GIL):

Traditional CPython:          Free-Threaded CPython:
┌──────────────┐              ┌──────────────┐
│  Thread 1    │              │  Thread 1    │
│  (holds GIL) │              │  (no GIL)    │
├──────────────┤              ├──────────────┤
│  Thread 2    │              │  Thread 2    │
│  (waiting)   │              │  (running)   │
├──────────────┤              ├──────────────┤
│  Thread 3    │              │  Thread 3    │
│  (waiting)   │              │  (running)   │
└──────────────┘              └──────────────┘
         │                             │
    Sequential                   True parallel
    execution                    execution

Key components:

• PYTHON_GIL=0 environment variable to disable GIL at runtime
• --disable-gil configure flag for free-threaded builds
• QSBR (Quiescent State-Based Reclamation) for safe memory reclamation (Include/internal/pycore_qsbr.h)
• Biased reference counting for reduced contention
• Per-object locks for mutable containers
• mimalloc as the default allocator in free-threaded builds

Note

Free-threading is experimental. The PYTHON_GIL environment variable controls GIL behavior at runtime: PYTHON_GIL=1 forces the GIL on (default for standard builds), PYTHON_GIL=0 disables it (only effective in free-threaded builds). See issue #113956 regarding _Py_SetImmortal on allocating threads and issue #148953 regarding potential deadlocks in _Py_qsbr_reserve.

1.7 JIT Compiler Architecture

The experimental JIT compiler uses a copy-and-patch approach:

Tools/jit/build.py
    │
    ▼
LLVM/Clang compiles C stubs → object files
    │
    ▼
Extracts machine code templates (one per micro-op)
    │
    ▼
Generates jit_stencils.h (embedded in CPython)
    │
    ▼
At runtime: Python/jit.c patches templates together
    │
    ▼
Executable native code for hot loops

The JIT operates at the micro-op (uop) level, below the bytecode tier. The Tools/cases_generator/ produces the tier-1 (bytecode) and tier-2 (uop) interpreter implementations from a single definition in Python/bytecodes.c.

2. System Requirements

2.1 Build Requirements by Platform

2.1.1 Linux / POSIX

Required system libraries for full feature builds:

# Debian/Ubuntu
sudo apt-get install build-essential gdb lcov pkg-config \
    libbz2-dev libffi-dev libgdbm-dev libgdbm-compat-dev liblzma-dev \
    libncurses5-dev libreadline6-dev libsqlite3-dev libssl-dev \
    lzma lzma-dev tk-dev uuid-dev zlib1g-dev libzstd-dev

# Fedora/RHEL
sudo dnf install gcc make pkgconfig \
    bzip2-devel libffi-devel gdbm-devel ncurses-devel openssl-devel \
    readline-devel sqlite-devel tk-devel uuid-devel xz-devel zlib-devel \
    libzstd-devel

Optional libraries that enable specific modules:

2.1.2 macOS

# Install dependencies via Homebrew
brew install openssl@3 xz gdbm tcl-tk libffi zstd

2.1.3 Windows

External dependencies are downloaded automatically by PCbuild/get_external.py:

# From PCbuild/ directory
python get_external.py --all

2.1.4 Android

2.1.5 iOS / Apple

2.1.6 Emscripten (WebAssembly)

2.1.7 WASI (WebAssembly)

2.2 Documentation Build Requirements

# In Doc/ directory
make venv    # Creates virtual environment with dependencies
make html    # Builds HTML documentation

Dependencies (from Doc/requirements.txt):

• sphinx < 9.0.0
• blurb
• sphinxext-opengraph
• sphinx-notfound-page
• python-docs-theme

2.3 Hardware Recommendations

Important

Profile-Guided Optimization (PGO) and Link-Time Optimization (LTO) builds require significantly more time and memory than standard builds. A PGO+LTO build on a 4-core machine can take 30+ minutes. The --enable-optimizations flag enables both PGO and LTO.

3. Configuration Reference

3.1 Build Configuration (configure Flags)

The configure script (configure.ac → autoconf → configure) accepts hundreds of options. The most significant:

3.1.1 Core Build Options

3.1.2 Optimization Options

3.1.3 Debug Options

3.1.4 Feature Options

3.1.5 Dependency Override Options

Important

When cross-compiling, you must supply --host=TRIPLE and --build=TRIPLE along with a host Python (--with-build-python=PATH) that matches the target version exactly. Cross-compilation is required for Android, iOS, Emscripten, and WASI targets.

3.1.6 Installation Layout Options

3.2 Compile-Time Definitions (pyconfig.h)

The configure script generates pyconfig.h from pyconfig.h.in. Key defines:

3.3 Runtime Environment Variables

3.3.1 Interpreter Behavior

3.3.2 SSL/Security

Warning

Setting SSLKEYLOGFILE in production enables extraction of TLS session keys, allowing decryption of all TLS traffic. This should only be used in development/debugging environments. Ensure the log file has restrictive permissions if used at all.

3.3.3 Build and Test Environment

3.3.4 REPL and IDE

3.3.5 Platform-Specific

3.4 Configuration Files

3.5 Makefile.pre.in Key Targets

Tip

For development, use make -j$(nproc) for parallel builds. The regen-all target is needed after modifying grammar files, ASDL definitions, opcode definitions, or Argument Clinic inputs. Running make clinic alone regenerates only Argument Clinic output.

4. Deployment & Scaling

4.1 Building from Source

4.1.1 Standard POSIX Build

# Clone
git clone https://github.com/python/cpython.git
cd cpython

# Configure (debug build for development)
./configure --with-pydebug

# Build
make -j$(nproc)

# Verify
./python -c "import sys; print(sys.version)"

Expected output:

Python 3.15.0a0 (heads/main:49918f5, May 7 2026, 12:00:00) [GCC 13.2.0] on linux

4.1.2 Optimized Release Build

./configure --enable-optimizations --with-lto --prefix=/opt/python3.15

# PGO training + LTO compilation (slow, 15-40 minutes)
make -j$(nproc)

# Install
sudo make install

Tip

The --enable-optimizations flag runs the test suite as PGO training data, then rebuilds with profile feedback plus LTO. For faster iteration during development, skip this and use --with-pydebug instead.

4.1.3 Free-Threaded Build

./configure --disable-gil --with-pydebug
make -j$(nproc)

# Verify GIL is disabled
./python -c "import sys; print(sys._is_gil_enabled())"

Expected output:

False

4.1.4 JIT-Enabled Build

# Requires LLVM 18+
./configure --enable-experimental-jit --with-pydebug
make -j$(nproc)

# Verify JIT is available
./python -c "import sys; print(sys._jit)"

4.1.5 Windows Build

# From repository root in Developer Command Prompt
PCbuild\build.bat -e -p x64

# Or for debug build:
PCbuild\build.bat -e -d -p x64

The -e flag runs get_external.py to download dependencies (OpenSSL, Tcl/Tk, SQLite, etc.). Build output goes to PCbuild/amd64/ (release) or PCbuild/amd64_d/ (debug).

4.1.6 macOS Framework Build

./configure --enable-framework=/Library/Frameworks \
            --with-openssl=$(brew --prefix openssl@3)
make -j$(nproc)
sudo make frameworkinstall

4.1.7 Android Cross-Compilation

# Using the platform build tool
python Platforms/Android/__main__.py \
    --cross-build-dir=build/android \
    arm64-v8a  # or armeabi-v7a, x86, x86_64

Key flags:

4.1.8 iOS / Apple Cross-Compilation

python Platforms/Apple/__main__.py \
    --cross-build-dir=build/ios \
    iOS arm64

Key flags:

4.1.9 Emscripten (WebAssembly) Build

# Ensure EMSDK is activated
python Platforms/emscripten/__main__.py \
    --cross-build-dir=build/emscripten \
    wasm32-emscripten

Key flags:

4.1.10 WASI Build

python Platforms/WASI/__main__.py \
    --wasi-sdk=/opt/wasi-sdk \
    --host-runner=wasmtime

Key flags:

4.2 Installation

4.2.1 Standard Installation

# Install to --prefix (default /usr/local)
sudo make install

# Creates:
#   /usr/local/bin/python3.15
#   /usr/local/bin/python3
#   /usr/local/lib/python3.15/
#   /usr/local/lib/libpython3.15.a (or .so if --enable-shared)
#   /usr/local/include/python3.15/

4.2.2 Alternative Installation (Side-by-Side)

# Does NOT create 'python3' symlink — safe alongside system Python
sudo make altinstall

# Creates only versioned binaries:
#   /usr/local/bin/python3.15
#   /usr/local/bin/pip3.15

Warning

Never run make install on a system where python3 is the system Python (e.g., modern Ubuntu, Fedora). Use make altinstall or install to a separate --prefix. Overwriting the system Python breaks package managers and system tools.

4.2.3 Windows Installation Layout

The PC/layout/main.py tool handles Windows packaging with extensive options:

python PC/layout/main.py --help

Key layout flags:

4.3 Embedding CPython

CPython can be embedded in other applications via the C API:

#include <Python.h>

int main(int argc, char *argv[]) {
    PyStatus status;
    PyConfig config;

    PyConfig_InitPythonConfig(&config);
    config.isolated = 1;
    config.site_import = 0;

    status = Py_InitializeFromConfig(&config);
    if (PyStatus_Exception(status)) {
        PyConfig_Clear(&config);
        Py_ExitStatusException(status);
    }

    PyRun_SimpleString("print('Hello from embedded Python!')");

    Py_Finalize();
    return 0;
}

Compile with:

gcc -o embed embed.c $(python3.15-config --cflags --ldflags --embed)

The Programs/_testembed.c file is the comprehensive test and example for all embedding scenarios.

4.4 Shared Library Deployment

When built with --enable-shared:

# Set library path
export LD_LIBRARY_PATH=/opt/python3.15/lib:$LD_LIBRARY_PATH

# Or configure ldconfig
echo "/opt/python3.15/lib" | sudo tee /etc/ld.so.conf.d/python3.15.conf
sudo ldconfig

4.5 Scaling Considerations

CPython's GIL (Global Interpreter Lock) historically limits CPU-bound parallelism to one thread. Strategies for scaling:

Note

The concurrent.interpreters module (subinterpreters) provides a middle ground: each interpreter has its own GIL, enabling true parallelism without the complexity of multiprocessing. Combined with Lib/concurrent/ futures, this provides a practical scaling path.

5. Operations

5.1 Building the Documentation

cd Doc/

# First time: create venv with sphinx + dependencies
make venv

# Build HTML docs
make html

# Output in Doc/build/html/

Other documentation targets:

5.2 Running the Test Suite

5.2.1 Basic Test Run

# Run the full test suite
make test

# Or directly:
./python -m test

# Run specific test module
./python -m test test_list

# Run with verbose output
./python -m test -v test_dict

5.2.2 Test Runner Flags (libregrtest)

The test runner (Lib/test/libregrtest/cmdline.py) has 50+ flags. Key ones:

Tip

For development, ./python -m test -j0 -v test_name runs a single test module with all available cores in verbose mode. Use -R 3:5 to hunt for reference leaks when modifying C code: it runs the test 3 times to warm up, then 5 times to detect leaks.

5.2.3 Resource-Intensive Tests

Some tests are skipped by default. Enable them with -u:

# Run everything
./python -m test -u all -j$(nproc)

5.2.4 Cross-Platform Testing

For cross-compiled builds, use the _PYTHON_HOSTRUNNER environment variable:

# Emscripten: run via Node.js
export _PYTHON_HOSTRUNNER="node"

# WASI: run via wasmtime
export _PYTHON_HOSTRUNNER="wasmtime run --dir=. --"

# Android: run via adb
export _PYTHON_HOSTRUNNER="adb shell"

5.3 Argument Clinic

Argument Clinic processes C extension source files to generate argument parsing boilerplate:

# Run Clinic on all files
make clinic

# Or directly:
python Tools/clinic/libclinic/cli.py --make --srcdir .

CLI flags:

Note

Argument Clinic generates the Modules/clinic/ directory (79 files, 2.2M chars). These files should be committed to the repository — they are regenerated by make clinic but checked in for bootstrapping (building Python without Python).

5.4 Code Generation Tools

5.4.1 Cases Generator

The Tools/cases_generator/ builds the bytecode interpreter from Python/bytecodes.c:

All generators accept -o/--output and input file arguments. Tier-1 and tier-2 generators also accept -l/--emit-line-directives.

# Regenerate all cases
make regen-cases

# Or individually:
python Tools/cases_generator/tier1_generator.py \
    -o Python/generated_cases.c.h \
    Python/bytecodes.c

5.4.2 PEG Parser Generator

# Regenerate the parser from Grammar/python.gram
make regen-pegen

# Or directly:
python -m Tools.peg_generator Grammar/python.gram \
    -o Parser/parser.c

PEG generator flags:

5.4.3 JIT Build Tool

# Build JIT stencils
python Tools/jit/build.py x86_64-unknown-linux-gnu \
    -o Python/ \
    -v

JIT build flags:

5.5 External Dependency Management (Windows)

# Download all external dependencies
python PCbuild/get_external.py --all

# Download specific dependency
python PCbuild/get_external.py openssl

The EXTERNALS_DIR environment variable overrides the default download location. CACHE_DIR controls where downloaded archives are cached.

5.6 Regeneration Targets Summary

5.7 Continuous Integration

CPython uses Buildbot for CI (Tools/buildbot/). The .github/ directory contains GitHub Actions workflows. Key CI dimensions:

• Multiple platforms (Linux, macOS, Windows, WASI, Emscripten)
• Multiple architectures (x86_64, arm64, s390x, ppc64le)
• Debug and release builds
• Free-threaded builds
• Address/memory/undefined-behavior/thread sanitizer builds
• Reference leak hunts
• PGO builds

5.8 Pre-commit Hooks

The .pre-commit-config.yaml configures automated checks:

# Install pre-commit hooks
pip install pre-commit
pre-commit install

# Run manually
pre-commit run --all-files

5.9 Linting and Code Style

CPython uses Ruff (.ruff.toml) for Python code linting:

# Check
ruff check .

# Fix
ruff check --fix .

C code follows the CPython style guide (PEP 7): 4-space indentation, specific brace placement, 79-char line limit.

6. Performance

6.1 Build-Time Optimizations

6.1.1 Profile-Guided Optimization (PGO)

PGO uses runtime profiling data to optimize hot code paths:

# Combined PGO + LTO
./configure --enable-optimizations
make -j$(nproc)

The process:

1. Build instrumented binary
2. Run training workload (test suite subset via --pgo flag)
3. Rebuild with profile data
4. Apply LTO

Typical speedup: 5-20% on real workloads.

6.1.2 Link-Time Optimization (LTO)

# Thin LTO (faster compile, slightly less optimization)
./configure --with-lto=thin

# Full LTO (slower compile, maximum optimization)
./configure --with-lto=full

6.1.3 BOLT Optimization

BOLT is a post-link optimizer that rearranges code layout based on profile data:

./configure --enable-bolt
make -j$(nproc)
make bolt-opt

Requires LLVM's llvm-bolt tool. Provides 2-5% additional speedup on top of PGO+LTO.

6.1.4 Computed Gotos

Computed gotos use GCC/Clang's &&label extension for faster bytecode dispatch:

./configure --with-computed-gotos  # Usually auto-detected

Provides 15-20% speedup in the bytecode interpreter loop compared to switch-based dispatch.

6.2 Runtime Performance Tuning

6.2.1 Memory Allocators

# Use mimalloc (if built with mimalloc support)
PYTHONMALLOC=mimalloc python script.py

# Debug allocator (catches buffer overflows, use-after-free)
PYTHONMALLOC=debug python script.py

6.2.2 JIT Compiler

The experimental JIT compiler targets the micro-op (uop) tier:

# Build with JIT
./configure --enable-experimental-jit --enable-optimizations
make -j$(nproc)

The JIT uses LLVM to compile C stubs into machine code templates at build time, then patches them together at runtime for hot code paths. The copy-and-patch approach avoids the overhead of a full JIT compilation pipeline at runtime.

6.2.3 Free-Threading Performance

Free-threaded builds trade single-threaded performance (~5-10% regression due to atomic operations) for true multi-threaded parallelism:

# Build free-threaded
./configure --disable-gil --enable-optimizations
make -j$(nproc)

# Control at runtime
PYTHON_GIL=0 python threaded_workload.py

Important

Free-threaded CPython uses biased reference counting: the owning thread uses non-atomic increments/decrements, while other threads use atomic operations. This minimizes contention for objects that stay on one thread. Extension modules must be updated to be thread-safe — check Py_mod_gil slot.

6.3 Profiling and Benchmarking

6.3.1 New Profiling Module

CPython now includes Lib/profiling/ with both sampling and tracing profilers:

# Sampling profiler
python -m profiling.sampling script.py

# Tracing profiler
python -m profiling.tracing script.py

The sampling profiler (Lib/profiling/sampling/, 19 files, 244K chars) periodically samples the call stack, providing low-overhead profiling suitable for production. The tracing profiler (Lib/profiling/tracing/, 3 files) hooks into the interpreter's tracing mechanism for precise call/return timing.

6.3.2 Import Profiling

# Profile import times
PYTHONPROFILEIMPORTTIME=1 python -c "import ssl"

Expected output:

import time: self [us] | cumulative | imported package
import time:       342 |        342 |   _io
import time:       123 |        123 |   marshal
...

6.3.3 Tracemalloc

# Track memory allocations (10-frame traceback)
PYTHONTRACEMALLOC=10 python script.py

6.3.4 Built-in Benchmarking Tools

6.3.5 DTrace / SystemTap Probes

When built with --with-dtrace:

# List probes
sudo dtrace -l -n 'python*:::'

# Trace function calls
sudo dtrace -n 'python$target:::function-entry { printf("%s %s", copyinstr(arg0), copyinstr(arg1)); }' -c "./python script.py"

Available probes:

• function-entry / function-return — Python function calls
• line — Line execution
• gc-start / gc-done — Garbage collection
• import-find-load-start / import-find-load-done — Import events

6.4 Statistics Collection

When built with --enable-pystats:

./python -X pystats script.py
# Creates /tmp/py_stats/ with detailed internal statistics

Statistics cover: opcode frequencies, specialization success/failure rates, call sites, object allocations, and more.

6.5 Valgrind Integration

./configure --with-valgrind
make -j$(nproc)

# Run with suppression file
valgrind --suppressions=Misc/valgrind-python.supp ./python script.py

The Misc/valgrind-python.supp file suppresses known false positives from Python's memory allocator. For most accurate results, also use PYTHONMALLOC=malloc to bypass pymalloc.

7. Security

7.1 Security Architecture

CPython's security model operates at multiple levels:

Application Level:  hashlib, ssl, secrets, hmac
    │
Interpreter Level:  hash randomization, safe eval, import restrictions
    │
Memory Level:       buffer overflow protections, object type checking
    │
Build Level:        ASAN/MSAN/UBSAN, stack protections, ASLR
    │
Network Level:      ssl module (OpenSSL), http.server TLS, certificate verification

7.2 Hash Randomization

Since Python 3.3, string hashing is randomized per-process to prevent hash-collision denial-of-service attacks:

# Default: random seed each process
python -c "print(hash('test'))"  # Different each run

# Reproducible hashing (testing only)
PYTHONHASHSEED=42 python -c "print(hash('test'))"

# Disable randomization (INSECURE)
PYTHONHASHSEED=0 python -c "print(hash('test'))"

Warning

Setting PYTHONHASHSEED=0 disables hash randomization, making the process vulnerable to hash-collision DoS attacks. Never do this in production. Use a fixed seed (PYTHONHASHSEED=N where N > 0) only for reproducible testing.

7.3 SSL/TLS Configuration

CPython's _ssl module wraps OpenSSL. Key security considerations:

import ssl

# Create secure default context
ctx = ssl.create_default_context()

# Properties that are ON by default:
#   - Certificate verification (CERT_REQUIRED)
#   - Hostname checking
#   - Modern protocol versions only (TLSv1.2+)
#   - Secure cipher suites

Build-time SSL configuration:

# Specify OpenSSL location
./configure --with-openssl=/opt/openssl-3.0 \
            --with-openssl-rpath=auto

# Override default cipher suites
./configure --with-ssl-default-suites="TLS_AES_256_GCM_SHA384:TLS_CHACHA20_POLY25519_SHA256"

# Select hash algorithms to compile in
./configure --with-builtin-hashlib-hashes=md5,sha1,sha256,sha512,sha3,blake2

The Tools/ssl/multissltests.py tool tests against multiple OpenSSL versions simultaneously.

7.4 Integer-String Conversion Limit

To prevent denial-of-service via extremely large integer-to-string conversions:

# Default limit: 4300 digits
python -c "int('1' * 5000)"
# ValueError: Exceeds the limit (4300 digits) for integer string conversion

# Override via environment variable
PYTHONINTMAXSTRDIGITS=10000 python -c "print(len(str(10**9999)))"

# Disable limit (INSECURE for untrusted input)
PYTHONINTMAXSTRDIGITS=0 python script.py

7.5 Sandboxing and Isolation

7.5.1 Isolated Mode

# Isolated mode: no user site-packages, no PYTHON* env vars
python -I script.py

Equivalent to -s -E -P:

• -s: Don't add user site-packages
• -E: Ignore all PYTHON* environment variables
• -P: Don't prepend script directory to sys.path

7.5.2 Restricted Imports

import importlib

# Disable import of specific modules
# (custom import hook approach)

7.5.3 Subinterpreters for Isolation

import concurrent.interpreters

# Each interpreter has its own GIL and module state
interp = concurrent.interpreters.create()
interp.exec("import os; print(os.getpid())")

7.6 Secure Build Flags

# Full security-hardened build
CFLAGS="-fstack-protector-strong -D_FORTIFY_SOURCE=2 -fPIE" \
LDFLAGS="-Wl,-z,relro,-z,now -pie" \
./configure --enable-optimizations --with-openssl=/usr/local/ssl

# Sanitizer builds for security testing
./configure --with-address-sanitizer    # Buffer overflows
./configure --with-memory-sanitizer     # Uninitialized reads
./configure --with-undefined-behavior-sanitizer  # UB
./configure --with-thread-sanitizer     # Data races

7.7 Known Security Issues

From the issue tracker:

Caution

The xmlrpc.client module does not sanitize method names (issue #148954). If your application constructs XML-RPC method names from user input, validate and sanitize the input before passing it to xmlrpc.client.ServerProxy. Consider using JSON-RPC or REST APIs instead for new applications.

7.8 HTTP Server TLS

CPython's http.server now includes HTTPS support:

from http.server import HTTPSServer, SimpleHTTPRequestHandler
import ssl

context = ssl.create_default_context(ssl.Purpose.CLIENT_AUTH)
context.load_cert_chain('cert.pem', 'key.pem')

server = HTTPSServer(('', 443), SimpleHTTPRequestHandler, context=context)
server.serve_forever()

The ThreadingHTTPSServer variant handles each request in a new thread.

Warning

The http.server module (including HTTPSServer) is not designed for production use. It lacks hardening against slowloris attacks, request smuggling, and other HTTP-level attacks. Use a production WSGI/ASGI server (Gunicorn, uvicorn, etc.) for public-facing deployments.

7.9 Pickle Security

The pickle module is inherently insecure for untrusted data:

import pickle

# NEVER unpickle untrusted data
# pickle.loads(untrusted_bytes)  # Arbitrary code execution!

# The Tools/picklebench/memory_dos_impact.py benchmarks
# pickle memory and DoS characteristics

7.10 Source Date Epoch

For reproducible builds, set SOURCE_DATE_EPOCH:

# Reproducible .pyc timestamps
SOURCE_DATE_EPOCH=$(date +%s) make install

This ensures .pyc files and other build artifacts have deterministic timestamps.

8. Developer Internals

8.1 Object Model

Every Python object starts with a PyObject header:

// Include/object.h
typedef struct _object {
    Py_ssize_t ob_refcnt;       // Reference count (or biased refcnt in free-threaded)
    PyTypeObject *ob_type;       // Type pointer
} PyObject;

typedef struct {
    PyObject ob_base;
    Py_ssize_t ob_size;          // Number of items (for variable-size objects)
} PyVarObject;

Key type implementations in Objects/:

8.2 Bytecode Compiler Pipeline

The compilation pipeline transforms source to executable code objects:

Source string
    │
    ▼
tokenize (Parser/tokenizer/, Parser/lexer/)
    │  Produces token stream
    ▼
parse (Parser/parser.c, generated from Grammar/python.gram)
    │  Produces CST → AST via Parser/action_helpers.c
    ▼
ast_opt (Python/ast_opt.c)
    │  Constant folding, optimization
    ▼
symtable (Python/symtable.c)
    │  Variable scope analysis (local/global/free/cell)
    ▼
compile (Python/compile.c)
    │  AST → intermediate representation
    ▼
codegen (Python/codegen.c)
    │  IR → bytecode instructions
    ▼
flowgraph (Python/flowgraph.c)
    │  Basic block optimization, jump threading
    ▼
assemble
    │  Emit final bytecode, create code object
    ▼
PyCodeObject (Objects/codeobject.c)

8.3 Evaluation Loop

The main interpreter loop lives in Python/ceval.c, with the actual opcode dispatch in Python/generated_cases.c.h (generated from Python/bytecodes.c).

Tier 1 (Bytecode):
  Python/ceval.c          — Main eval loop, frame management
  Python/generated_cases.c.h — Opcode implementations (generated)
  Python/ceval_macros.h   — Helper macros

Tier 2 (Micro-ops):
  Python/executor_cases.c.h — Uop implementations (generated)
  Python/optimizer.c       — Trace optimization
  Python/optimizer_analysis.c — Abstract interpretation
  Python/optimizer_bytecodes.c — Optimizer uop definitions
  Python/optimizer_symbols.c — Symbolic analysis

JIT:
  Python/jit.c             — JIT runtime (copy-and-patch)
  Tools/jit/build.py       — JIT stencil compiler

The evaluation loop uses a two-tier strategy:

1. Tier 1: Standard bytecode dispatch (switch or computed gotos)
2. Tier 2: Hot loops are "specialized" into micro-op traces, then optionally JIT-compiled

8.4 Specialization (Adaptive Interpreter)

CPython uses inline caching and instruction specialization:

LOAD_ATTR (generic)
    │  After N executions of same pattern
    ▼
LOAD_ATTR_INSTANCE_VALUE (specialized for instance attribute)
    │  If type changes
    ▼
LOAD_ATTR (deoptimized back to generic)

Specialized opcodes exist for:

• LOAD_ATTR → LOAD_ATTR_INSTANCE_VALUE, LOAD_ATTR_MODULE, LOAD_ATTR_CLASS, etc.
• LOAD_GLOBAL → LOAD_GLOBAL_BUILTIN, LOAD_GLOBAL_MODULE
• BINARY_OP → BINARY_OP_ADD_INT, BINARY_OP_ADD_FLOAT, BINARY_OP_ADD_UNICODE
• CALL → CALL_PY_EXACT_ARGS, CALL_BUILTIN_CLASS, CALL_BUILTIN_FAST, etc.
• COMPARE_OP → COMPARE_OP_INT, COMPARE_OP_FLOAT, COMPARE_OP_STR

8.5 Garbage Collection

CPython uses reference counting as the primary memory management strategy, supplemented by a cyclic garbage collector:

Primary: Reference Counting
  - Immediate cleanup when refcount reaches 0
  - Deterministic finalization (predictable __del__ timing)
  - Free-threaded builds use biased reference counting

Secondary: Cyclic GC (Python/gc.c)
  - Detects reference cycles (unreachable but non-zero refcount)
  - Three generations (0, 1, 2)
  - Generation 0: checked frequently (every ~700 allocations)
  - Generation 1: checked less often
  - Generation 2: checked rarely (long-lived objects)

GC control:

import gc

gc.disable()                    # Disable cyclic GC
gc.collect()                    # Force collection
gc.set_threshold(700, 10, 10)   # Set generation thresholds
gc.get_stats()                  # Collection statistics
gc.freeze()                     # Freeze all tracked objects

8.6 Import System

The import system (Python/import.c, Lib/importlib/) follows this resolution order:

import foo
    │
    ▼
1. sys.modules cache?  → Return cached module
    │ (miss)
    ▼
2. sys.meta_path finders (in order):
    ├── BuiltinImporter (built-in C modules)
    ├── FrozenImporter (frozen modules)
    └── PathFinder (sys.path-based)
            │
            ▼
        3. sys.path entries → sys.path_hooks → Finders:
            ├── zipimport.zipimporter (.zip files)
            └── FileFinder (.py, .pyc, .so/.pyd)
                    │
                    ▼
                4. Loader.exec_module()
                    │
                    ▼
                5. Module in sys.modules

CPython now supports lazy imports:

# Deferred import (evaluated on first attribute access)
lazy import _colorize

8.7 Memory Allocator Internals

8.7.1 pymalloc (Objects/obmalloc.c)

Arena (256 KB, mmap'd)
    │
    ├── Pool (4 KB page, one size class)
    │   ├── Block (8 bytes)
    │   ├── Block (8 bytes)
    │   └── ...
    │
    ├── Pool (4 KB, different size class)
    │   ├── Block (16 bytes)
    │   └── ...
    │
    └── ...

Size classes: 8, 16, 24, 32, ..., 512 bytes (64 classes)
Requests > 512 bytes → system malloc

8.7.2 mimalloc (Objects/mimalloc/)

Microsoft's mimalloc is integrated for improved multi-threaded performance:

Heap (per-thread)
    │
    ├── Segment (configurable, typically 4 MB)
    │   ├── Page (size-class specific)
    │   │   ├── Block
    │   │   └── ...
    │   └── ...
    │
    └── Large allocations → OS directly

mimalloc is the default allocator in free-threaded builds. Key advantages:

• Per-thread heaps reduce contention
• Bounded worst-case fragmentation
• Compatible with fork() (unlike some alternatives)

8.8 C API Layers

CPython exposes three API tiers:

Key API families:

Include/
├── Python.h          — Master include (pulls in everything public)
├── object.h          — PyObject, Py_INCREF/DECREF, type system
├── abstract.h        — Abstract object protocol (PySequence_*, PyMapping_*)
├── pystate.h         — Thread/interpreter state
├── pylifecycle.h     — Py_Initialize, Py_Finalize
├── import.h          — Import API
├── compile.h         — Compilation API
├── eval.h            — Evaluation API
├── pyerrors.h        — Exception handling
├── pymem.h           — Memory allocation (PyMem_*, PyObject_*)
├── pycapsule.h       — Capsule objects (opaque C pointer passing)
├── moduleobject.h    — Module definition API
├── methodobject.h    — C function/method objects
└── ...

Include/internal/     (144 files, 1.6M chars)
├── pycore_ceval.h       — Eval loop internals
├── pycore_gc.h          — GC internals
├── pycore_interp.h      — Interpreter state
├── pycore_object.h      — Object internals
├── pycore_pystate.h     — Thread state internals
├── pycore_qsbr.h        — QSBR (free-threading)
├── pycore_dict.h        — Dict internals
├── pycore_long.h        — Int internals
├── pycore_uop_ids.h     — Micro-op IDs
├── pycore_opcode_metadata.h — Opcode metadata
└── ...

8.9 C Extension Module Structure

Modern C extension modules use multi-phase initialization:

// Module definition
static struct PyModuleDef mymodule = {
    PyModuleDef_HEAD_INIT,
    .m_name = "mymodule",
    .m_doc = "My extension module",
    .m_size = -1,  // or sizeof(module_state) for per-interpreter state
    .m_methods = mymodule_methods,
    .m_slots = mymodule_slots,  // Multi-phase init
};

// Slots for multi-phase init
static PyModuleDef_Slot mymodule_slots[] = {
    {Py_mod_exec, mymodule_exec},
    {Py_mod_multiple_interpreters, Py_MOD_PER_INTERPRETER_GIL_SUPPORTED},
    {Py_mod_gil, Py_MOD_GIL_NOT_USED},  // Free-threading support
    {0, NULL}
};

PyMODINIT_FUNC
PyInit_mymodule(void)
{
    return PyModuleDef_Init(&mymodule);
}

The Py_mod_gil slot is critical for free-threading compatibility:

• Py_MOD_GIL_USED: Module requires GIL (default)
• Py_MOD_GIL_NOT_USED: Module is thread-safe

8.10 Standard Library Module Architecture

8.10.1 asyncio (Lib/asyncio/, 35 files, 530K chars)

asyncio/
├── base_events.py      — BaseEventLoop (core event loop)
├── events.py           — Event loop policy, abstract base
├── futures.py          — Future objects
├── tasks.py            — Task objects (coroutine wrappers)
├── streams.py          — High-level stream API
├── protocols.py        — Protocol abstract base classes
├── transports.py       — Transport abstract base classes
├── selector_events.py  — select/poll/epoll-based event loop
├── unix_events.py      — Unix-specific (signals, child processes)
├── windows_events.py   — IOCP-based event loop
├── proactor_events.py  — Proactor-based event loop (Windows)
├── sslproto.py         — SSL/TLS protocol implementation
├── subprocess.py       — Subprocess management
├── queues.py           — Async queues
├── locks.py            — Async synchronization primitives
├── runners.py          — asyncio.run() and Runner
├── taskgroups.py       — TaskGroup (structured concurrency)
├── timeouts.py         — Timeout context manager
├── staggered.py        — Staggered connection racing
├── trsock.py           — Transport socket wrapper
├── mixins.py           — Event loop mixins
├── exceptions.py       — asyncio-specific exceptions
├── coroutines.py       — Coroutine utilities
├── constants.py        — Module constants
└── __main__.py         — python -m asyncio (async REPL)

8.10.2 multiprocessing (Lib/multiprocessing/, 21 files, 306K chars)

multiprocessing/
├── context.py          — Process start method contexts (fork, spawn, forkserver)
├── process.py          — Process class
├── pool.py             — Worker pool
├── queues.py           — Process-safe queues
├── managers.py         — Shared state managers
├── shared_memory.py    — SharedMemory (zero-copy IPC)
├── connection.py       — Pipe/socket connections
├── synchronize.py      — Locks, semaphores, events, conditions
├── sharedctypes.py     — Shared ctypes objects
├── spawn.py            — Spawn helpers
├── forkserver.py       — Fork server implementation
├── heap.py             — Shared memory heap
├── popen_*.py          — Platform-specific process creation
├── reduction.py        — Object serialization for IPC
├── resource_tracker.py — Shared resource cleanup
└── util.py             — Utility functions

8.10.3 importlib (Lib/importlib/, 9 files, 141K chars)

importlib/
├── __init__.py         — Public API (import_module, reload, etc.)
├── abc.py              — Abstract base classes (Finder, Loader)
├── machinery.py        — Import machinery access
├── metadata/           — Package metadata (PEP 566)
├── readers.py          — Resource readers
├── resources/          — Package resources (PEP 302)
├── _abc.py             — ABC implementation details
├── _bootstrap.py       — Core import bootstrapping (frozen)
└── _bootstrap_external.py — File-based importing (frozen)

8.10.4 http (Lib/http/, 5 files, 209K chars)

http/
├── __init__.py     — HTTP status codes
├── server.py       — HTTP/HTTPS servers (HTTPServer, HTTPSServer,
│                     ThreadingHTTPServer, ThreadingHTTPSServer,
│                     BaseHTTPRequestHandler, SimpleHTTPRequestHandler,
│                     CGIHTTPRequestHandler)
├── client.py       — HTTP client (HTTPConnection, HTTPSConnection)
├── cookiejar.py    — Cookie handling
└── cookies.py      — Cookie parsing (SimpleCookie)

8.10.5 email (Lib/email/, 20 files, 376K chars)

email/
├── message.py          — Message/EmailMessage classes
├── policy.py           — Email policies (compat32, email5)
├── contentmanager.py   — Content type management
├── headerregistry.py   — Structured header parsing
├── generator.py        — Message serialization
├── parser.py           — Message parsing (FeedParser, BytesFeedParser)
├── mime/               — MIME type classes
├── charset.py          — Character set handling
├── encoders.py         — Content transfer encoding
├── errors.py           — Email-specific exceptions
├── utils.py            — Utility functions (parseaddr, formatdate, etc.)
├── header.py           — Legacy header handling
├── base64mime.py       — Base64 MIME encoding
├── quoprimime.py       — Quoted-printable MIME encoding
└── iterators.py        — Message iteration

8.10.6 Key C Extension Modules (Modules/)

8.11 Parser Architecture

The PEG parser replaced the LL(1) parser in Python 3.9:

Grammar/python.gram (PEG grammar definition)
    │
    ▼
Tools/peg_generator/ (pegen)
    │  Generates C parser from grammar
    ▼
Parser/parser.c (generated, ~24K lines)
    │
    ├── Parser/tokenizer/tokenize.c — Byte-level tokenization
    ├── Parser/lexer/lexer.c — Token classification
    ├── Parser/string_parser.c — f-string/string literal parsing
    ├── Parser/action_helpers.c — Semantic actions (AST construction)
    └── Parser/pegen.c — Parser runtime support

8.12 Compilation to Bytecode

The compiler (Python/compile.c, Python/codegen.c, Python/flowgraph.c) performs:

1. Symbol table analysis (Python/symtable.c): Determines variable scoping (local, global, free, cell)
2. AST optimization (Python/ast_opt.c): Constant folding, dead code elimination
3. Code generation (Python/codegen.c): AST → basic blocks with instructions
4. Flow graph optimization (Python/flowgraph.c): Jump threading, unreachable code removal, peephole optimizations
5. Assembly (in Python/compile.c): Final bytecode emission, line number table, exception table

8.13 String Implementation

Strings use a flexible internal representation (PEP 393):

PyUnicodeObject
├── ASCII (all chars < 128): 1 byte per char, compact layout
├── Latin-1 (all chars < 256): 1 byte per char, compact layout
├── BMP (all chars < 65536): 2 bytes per char (UCS-2)
└── Full Unicode: 4 bytes per char (UCS-4)

The Objects/stringlib/ directory (20 files, 158K chars) contains shared algorithms used by both str and bytes:

• Boyer-Moore-Horspool search
• Adaptive merge sort
• Split/join/replace algorithms
• Numeric formatting

8.14 Dict Implementation

Python dicts use a compact, ordered layout:

Compact dict structure:
┌─────────────┐
│  Hash table │  Sparse array of indices (1/2/4/8 bytes each)
│  (indices)  │  Maps hash(key) % size → index into entries
├─────────────┤
│  Entries    │  Dense array of (hash, key, value) tuples
│  (compact)  │  Insertion-ordered
└─────────────┘

Split-table optimization: When multiple instances of the same class share the same attribute names, they share a single keys table, saving significant memory.

8.15 GDB Debugging Helpers

Tools/gdb/ provides GDB Python extensions for debugging CPython:

# Load in GDB
source Tools/gdb/libpython.py

# Commands available:
py-bt          # Python traceback
py-list        # Python source listing
py-print       # Print Python variable
py-up/py-down  # Navigate Python frames
py-locals      # Print local variables

8.16 IDLE Architecture (Lib/idlelib/)

IDLE is CPython's integrated development environment (60 files, 771K chars):

idlelib/
├── pyshell.py       — Main shell window
├── editor.py        — Editor window
├── run.py           — Execution engine (subprocess)
├── rpc.py           — RPC between IDLE GUI and execution subprocess
├── debugger.py      — Integrated debugger
├── colorizer.py     — Syntax highlighting
├── autocomplete.py  — Tab completion
├── calltip.py       — Function signature tooltips
├── config.py        — Configuration system
├── configdialog.py  — Settings dialog
├── codecontext.py   — Code context (scope display)
├── outwin.py        — Output window
├── tree.py          — Tree widget
├── browser.py       — Module browser
├── pathbrowser.py   — Path browser
├── help.py          — Help system
├── query.py         — Dialog boxes
├── replace.py       — Find/replace
├── grep.py          — Search in files
├── undo.py          — Undo/redo
└── ...

8.17 Interactive REPL (Lib/_pyrepl/)

The new _pyrepl (29 files, 293K chars) replaces the basic readline-based REPL:

_pyrepl/
├── console.py       — Console abstraction
├── reader.py        — Input reader with multi-line editing
├── readline.py      — readline compatibility layer
├── commands.py      — Editor commands
├── completing_reader.py — Tab completion
├── keymap.py        — Key binding maps
├── unix_console.py  — Unix terminal implementation
├── windows_console.py — Windows terminal implementation
├── trace.py         — Debug tracing (PYREPL_TRACE)
└── ...

Set PYTHON_BASIC_REPL=1 to fall back to the classic readline REPL.

8.18 Encoding System

Lib/encodings/ contains 123 encoding modules covering:

• ASCII, UTF-8, UTF-16, UTF-32 (with BOM variants)
• Latin/ISO-8859 family (1-16)
• Windows code pages (cp1250-cp1258, cp437, cp850, etc.)
• CJK encodings (GB2312, GBK, GB18030, Big5, EUC-JP, Shift_JIS, EUC-KR, ISO-2022-*)
• EBCDIC variants
• Special encodings (rot_13, base64_codec, hex_codec, unicode_escape)
• Platform-specific (palmos, hp_roman8, mac_roman, etc.)

The Modules/cjkcodecs/ provides C-accelerated CJK codec implementations.

8.19 Compression Architecture

CPython supports multiple compression formats:

The new Lib/compression/ module (4 files) provides a unified interface across compression formats.

8.20 Remote Debugging (Modules/_remote_debugging/)

A new module for external process inspection:

_remote_debugging/
├── _remote_debugging.c     — C extension for attaching to processes
└── ...

The Tools/inspection/benchmark_external_inspection.py benchmarks the overhead of external debugging attachment.

9. Integration Points

9.1 Extending CPython with C Extensions

The extension module ABI layers:

Building an extension module:

# Using setuptools/distutils
python setup.py build_ext --inplace

# Using pkg-config
gcc -shared -o mymodule.so mymodule.c \
    $(python3.15-config --cflags --ldflags) \
    -fPIC

9.2 Argument Clinic Integration

When writing C extensions for CPython's standard library, Argument Clinic automates argument parsing:

/*[clinic input]
module _mymodule

_mymodule.my_function

    name: str
        The name parameter.
    count: int = 1
        Number of repetitions.
    /

My function's docstring.
[clinic start generated code]*/

Running make clinic or python Tools/clinic/libclinic/cli.py -f Modules/_mymodule.c generates the argument parsing code in Modules/clinic/_mymodule.c.h.

9.3 sysconfig Integration

Lib/sysconfig/ (2 files, 37K chars) provides build configuration to downstream tools:

import sysconfig

# Get install paths
sysconfig.get_paths()

# Get build variables
sysconfig.get_config_vars()

# Get specific variable
sysconfig.get_config_var('CFLAGS')

# Command-line access
python -m sysconfig

9.4 ensurepip

Lib/ensurepip/ (3 files, 9K chars) bootstraps pip into a Python installation:

# Bootstrap pip
python -m ensurepip

# Upgrade pip
python -m ensurepip --upgrade

# Install to specific location
python -m ensurepip --root /path/to/root

Configure at build time:

./configure --with-ensurepip=upgrade  # Default
./configure --with-ensurepip=install  # Don't upgrade existing
./configure --with-ensurepip=no       # Skip pip entirely

9.5 Freeze Tool

Tools/freeze/ (12 files) creates standalone executables by embedding Python modules into C code:

# Freeze a script into a standalone executable
python Tools/freeze/freeze.py script.py
make -f Makefile

9.6 Internationalization

Tools/i18n/ provides tools for extracting and compiling gettext translations:

# Extract translatable strings
python Tools/i18n/pygettext.py -o messages.pot script.py

# Compile .po files to .mo
python Tools/i18n/msgfmt.py messages.po

9.7 Unicode Data Generation

Tools/unicode/ generates Unicode data tables used by the interpreter:

# Regenerate Unicode database from UCD
python Tools/unicode/makeunicodedata.py

9.8 C Analyzer

Tools/c-analyzer/ analyzes C global variables in CPython's source, used to track and minimize mutable global state (important for subinterpreters and free-threading):

# Analyze C globals
python -m Tools.c-analyzer.c_analyzer

9.9 MSI/Installer Tools

• Tools/msi/: Windows MSI installer generation
• Mac/BuildScript/: macOS installer (.pkg) generation
• PC/layout/: Windows distribution layout tool

9.10 WebAssembly Integration

For embedding CPython in web applications or serverless environments:

# Emscripten build (browser/Node.js)
python Platforms/emscripten/__main__.py \
    --cross-build-dir=build/wasm \
    wasm32-emscripten

# WASI build (standalone WASM runtime)
python Platforms/WASI/__main__.py \
    --wasi-sdk=/opt/wasi-sdk

Browser test dependencies:

• @playwright/test — Browser automation
• http-server — Local HTTP server for serving WASM

9.11 wsgiref (Lib/wsgiref/, 7 files, 57K chars)

Reference implementation of PEP 3333 (WSGI):

from wsgiref.simple_server import make_server

def app(environ, start_response):
    start_response('200 OK', [('Content-Type', 'text/plain')])
    return [b'Hello, World!']

httpd = make_server('', 8000, app)
httpd.serve_forever()

9.12 tkinter (Lib/tkinter/, 13 files, 308K chars)

Python binding for Tcl/Tk GUI toolkit. Requires TCL_LIBRARY environment variable if Tcl is installed in a non-standard location. Build with --with-tcltk-includes and --with-tcltk-libs if auto-detection fails.

9.13 SQLite Integration

Lib/sqlite3/ (5 files, 19K chars) wraps the C _sqlite3 module:

# SQLite CLI
python -m sqlite3 database.db

9.14 JSON CLI

# JSON tool (pretty-print, validate)
python -m json.tool input.json

# JSON Lines mode
echo '{"a":1}' | python -m json.tool --json-lines

Note

Issue #132631 documents a known bug where the JSON CLI's --json-lines mode can raise "I/O operation on closed file" errors under certain piping conditions.

9.15 zoneinfo (Lib/zoneinfo/, 4 files, 37K chars)

Timezone database integration:

from zoneinfo import ZoneInfo
from datetime import datetime

dt = datetime(2026, 5, 7, tzinfo=ZoneInfo("America/New_York"))

The PYTHONTZPATH environment variable controls the search path for timezone data files.

9.16 tomllib (Lib/tomllib/, 4 files, 29K chars)

TOML parser (read-only, added in Python 3.11):

import tomllib

with open("pyproject.toml", "rb") as f:
    config = tomllib.load(f)

9.17 dbm (Lib/dbm/, 5 files, 23K chars)

Simple database interface with multiple backends:

import dbm

# Backend selection order controlled by --with-dbmliborder
with dbm.open('mydb', 'c') as db:
    db['key'] = 'value'

Configure backend order at build time:

./configure --with-dbmliborder=gdbm:ndbm:bdb

Appendix A: Troubleshooting

A.1 Build Failures

Missing libffi

*** WARNING: building of the _ctypes extension module failed

Fix: Install libffi-dev (Debian) or libffi-devel (Fedora).

Missing OpenSSL

Could not build the ssl module!
Python requires a OpenSSL 1.1.1 or newer

Fix: Install libssl-dev or build OpenSSL from source and use --with-openssl=DIR.

Missing zlib

*** WARNING: building of the zlib extension module failed

Fix: Install zlib1g-dev (Debian) or zlib-devel (Fedora).

Missing Tcl/Tk

*** WARNING: building of the _tkinter extension module failed

Fix: Install tk-dev (Debian) or tk-devel (Fedora). Use --with-tcltk-includes and --with-tcltk-libs if installed in non-standard locations.

Missing libzstd

*** WARNING: building of the _zstd extension module failed

Fix: Install libzstd-dev (Debian) or libzstd-devel (Fedora). Zstandard is a newer dependency.

Tip

Run make and check the summary at the end. It lists all modules that failed to build and why. The interpreter itself will still build — only the affected extension modules are skipped.

configure failures after modifying configure.ac

configure: error: cannot find install-sh, install.sh, or shtool

Fix: Regenerate configure:

autoreconf -ivf

Requires autoconf 2.71+.

PGO build failure (insufficient test coverage)

error: no profile data available for file 'xxx'

Fix: Ensure the PGO training run completed successfully. Check for test failures during the profiling phase.

Cross-compilation --with-build-python version mismatch

Fatal Python error: init_importlib: Failed to import the site module

Fix: The build Python must be the exact same version as the target. Build a native Python first, then use it for cross-compilation.

A.2 Runtime Issues

Segfault in C extension

# Build with debug and address sanitizer
./configure --with-pydebug --with-address-sanitizer
make -j$(nproc)

# Run with GDB helpers
gdb -ex "source Tools/gdb/libpython.py" -ex run --args ./python script.py

# In GDB:
(gdb) py-bt     # Python-level traceback
(gdb) py-locals  # Python local variables

Reference leak detection

# Hunt for reference leaks
./python -m test -R 3:5 test_mymodule

Output:

test_mymodule leaked [1, 1, 1] references, sum=3
test_mymodule leaked [56, 56, 56] memory blocks, sum=168

Hash randomization issues

# Reproduce with fixed seed
PYTHONHASHSEED=0 python script.py

# Check current seed
python -c "import sys; print(sys.hash_info)"

Import errors after installation

ModuleNotFoundError: No module named '_ssl'

Fix: The C extension failed to build (missing dependency). Check make output for warnings. Rebuild after installing the missing library.

Free-threading crashes

Fatal Python error: _Py_qsbr_reserve: ...

Fix: Known issue (#148953). Free-threading is experimental — test thoroughly before deploying. Consider running with PYTHON_GIL=1 to isolate whether the crash is GIL-related.

dataclass(slots=True) crash

TypeError: type.__new__() takes exactly 3 arguments (0 given)

Fix: Known issue (#148947) where dataclass(slots=True) crashes with an empty __class__ cell. Workaround: avoid slots=True if the class has complex closure interactions.

dataclass with existing __init__

TypeError: Cannot overwrite attribute __init__

Fix: Known issue (#148941). Use @dataclass(init=False) if you provide your own __init__.

pkgutil.walk_packages(None) importing test modules

Unexpected imports triggered by pkgutil.walk_packages(None)

Fix: Known issue (#131684). Pass explicit paths instead of None:

import pkgutil, sys
pkgutil.walk_packages(sys.path)  # Still problematic — filter test paths

Caution

pkgutil.walk_packages(None) will import test modules, which may have side effects including executing test code. Always specify explicit paths.

Integer string conversion limit

ValueError: Exceeds the limit (4300 digits) for integer string conversion

Fix: Increase the limit if needed:

PYTHONINTMAXSTRDIGITS=0 python script.py  # Disable limit

Or in code:

import sys
sys.set_int_max_str_digits(10000)

A.3 Windows-Specific Issues

External dependency download failures

Error downloading openssl-...

Fix: Set EXTERNALS_DIR to a writable location, or manually download dependencies:

python PCbuild/get_external.py --all

Visual Studio version mismatch

error MSB8020: The build tools for v143 cannot be found

Fix: Ensure the correct Visual Studio workload ("Desktop development with C++") is installed with the matching platform toolset.

Debug/Release DLL mismatch

ImportError: DLL load failed while importing _ssl

Fix: Ensure debug Python (python_d.exe) loads debug extensions (_ssl_d.pyd) and release Python loads release extensions. Don't mix.

A.4 macOS-Specific Issues

Framework build + Homebrew OpenSSL

Could not build the ssl module! ... openssl not found

Fix:

./configure --enable-framework \
    --with-openssl=$(brew --prefix openssl@3)

Universal binary build failures

lipo: can't figure out the architecture type of ...

Fix: Ensure all dependencies (OpenSSL, Tcl/Tk, etc.) are built as universal binaries matching your --with-universal-archs setting.

A.5 Test Suite Issues

Flaky tests

# Re-run failed tests
./python -m test -w test_asyncio

# Run with randomized order to find ordering dependencies
./python -m test --randomize

# Run forever until failure
./python -m test -F test_suspect

Network tests skipped

test_urllib skipped -- Use of the 'network' resource not enabled

Fix:

./python -m test -u network test_urllib

GUI tests skipped

test_tkinter skipped -- No module named '_tkinter'

Fix: Install Tcl/Tk development headers and rebuild. On headless systems, use xvfb-run:

xvfb-run ./python -m test -u gui test_tkinter

Note

Many test modules require specific system resources. Use ./python -m test -u all to enable all resource-intensive tests, but expect some platform-specific failures on constrained environments.

Test timeout

Timeout (900.0 seconds) for test_xxx

Fix: Increase timeout:

./python -m test --timeout 1800 test_xxx

Reference leak false positives

test_xxx leaked [2, 0, 0] references

Fix: Some reference count changes are expected (e.g., warming up caches). Use more warmup iterations:

./python -m test -R 10:5 test_xxx

A.6 Documentation Build Issues

Sphinx version mismatch

sphinx.errors.VersionRequirementError: ...

Fix: Recreate the venv:

cd Doc
make clean
rm -rf venv
make venv

Missing blurb or theme

ModuleNotFoundError: No module named 'blurb'

Fix: The venv should install all dependencies. If not:

Doc/venv/bin/pip install -r Doc/requirements.txt

A.7 Emscripten / WASI Issues

Emscripten SDK not found

emcc: command not found

Fix: Activate the Emscripten SDK:

source /path/to/emsdk/emsdk_env.sh

Or set EMSDK_CACHE:

export EMSDK_CACHE=/path/to/emsdk/upstream/emscripten/cache

WASI SDK path

wasi-sdk not found

Fix: Set the path:

export WASI_SDK_PATH=/opt/wasi-sdk

A.8 Performance Issues

Slow startup

# Profile import times
PYTHONPROFILEIMPORTTIME=1 python -c "pass"

Fix: Use python -S to skip site-packages import, or python -I for fully isolated startup.

Memory growth

# Track allocations
PYTHONTRACEMALLOC=25 python script.py

import tracemalloc
tracemalloc.start(25)
# ... run workload ...
snapshot = tracemalloc.take_snapshot()
for stat in snapshot.statistics('lineno')[:10]:
    print(stat)

Tip

For production memory profiling, the new profiling.sampling module provides lower overhead than tracemalloc. Use tracemalloc for precise allocation tracking during development, and profiling.sampling for production sampling.

Appendix B: Quick Reference Tables

B.1 Make Targets

B.2 python3.15-config Flags

python3.15-config --cflags      # Compiler flags
python3.15-config --ldflags     # Linker flags
python3.15-config --includes    # Include paths
python3.15-config --libs        # Libraries
python3.15-config --prefix      # Installation prefix
python3.15-config --embed       # Embedding flags
python3.15-config --configdir   # Config directory

B.3 Special Builds

B.4 Key Source Files by Function

B.5 File Counts by Top-Level Directory