Jesse A. Green / notebook

Live demo: the DocCompiler guide library as a static, read-only snapshot (October 2026). No backend, no accounts, nothing to submit.

The Rust Compiler (rustc) — Administrator & Developer Guide

Generated May 2026 from rust-lang/rust at b2f1ccf

AI-generated documentation. The library is checked by sampling (three repositories so far); this guide was not individually reviewed. Verify against the project before relying on it.

1. Architecture Overview

The rust-lang/rust monorepo (57,526 files, ~66 million characters of source at commit b2f1ccf5) produces the Rust compiler (rustc), the standard library (core, alloc, std, proc_macro, test), and the complete development toolchain. It is a self-hosted compiler: the Rust compiler is written in Rust and compiles itself through a multi-stage bootstrap process.

The compiler transforms Rust source code through a pipeline of intermediate representations — from the token stream through AST, HIR, THIR, and MIR, then lowers to machine code via LLVM (primary), Cranelift (experimental, faster debug builds), or GCC (experimental, broader platform reach). The standard library is compiled as part of the build and distributed alongside the compiler as part of the "sysroot."

Architecture Diagram

┌─────────────────────────────────────────────────────────────────────┐
│                         x.py / bootstrap                            │
│  (Python + Rust build orchestrator, multi-stage management)         │
├─────────────────────────────────────────────────────────────────────┤
│                                                                     │
│  ┌───────────────────── compiler/ ────────────────────────────────┐ │
│  │                                                                │ │
│  │  Source → rustc_lexer → rustc_parse → rustc_ast                │ │
│  │            (tokenize)    (parse)      (AST)                    │ │
│  │                                                                │ │
│  │  → rustc_ast_lowering → rustc_hir → rustc_hir_analysis        │ │
│  │    (AST→HIR)            (HIR)       (type checking)           │ │
│  │                                                                │ │
│  │  → rustc_hir_typeck → rustc_borrowck → rustc_mir_build        │ │
│  │    (type inference)   (borrow check)   (HIR→MIR)              │ │
│  │                                                                │ │
│  │  → rustc_mir_transform → rustc_codegen_ssa                    │ │
│  │    (MIR optimizations)   (codegen interface)                  │ │
│  │                                                                │ │
│  │  → rustc_codegen_llvm ──→ LLVM ──→ Object files ──→ Linker   │ │
│  │    rustc_codegen_cranelift (alt)                               │ │
│  │    rustc_codegen_gcc      (alt)                               │ │
│  └────────────────────────────────────────────────────────────────┘ │
│                                                                     │
│  ┌───────────── library/ ──────────┐  ┌──── src/tools/ ──────────┐ │
│  │ core (no_std foundation)        │  │ clippy (linter)          │ │
│  │ alloc (heap allocation)         │  │ rustfmt (formatter)      │ │
│  │ std (full OS integration)       │  │ miri (MIR interpreter)   │ │
│  │ proc_macro (procedural macros)  │  │ rust-analyzer (IDE)      │ │
│  │ test (test framework)           │  │ rustdoc (via librustdoc) │ │
│  │ compiler-builtins (intrinsics)  │  │ compiletest (test runner)│ │
│  │ stdarch (SIMD intrinsics)       │  │ tidy (style checker)     │ │
│  │ backtrace (stack traces)        │  │ opt-dist (PGO builds)    │ │
│  │ portable-simd (SIMD types)      │  │ rust-installer (pkging)  │ │
│  └─────────────────────────────────┘  └──────────────────────────┘ │
│                                                                     │
│  ┌──────────── tests/ ─────────────────────────────────────────┐   │
│  │ ui/ (diagnostics) │ mir-opt/ │ codegen-llvm/ │ run-make/    │   │
│  │ rustdoc-js/       │ rustdoc-gui/ │ ui-fulldeps/ │ assembly/ │   │
│  └─────────────────────────────────────────────────────────────┘   │
└─────────────────────────────────────────────────────────────────────┘

Component Inventory

Component Path Role
rustc binary compiler/rustc/ Thin entry point (calls rustc_driver)
Compiler driver compiler/rustc_driver_impl/ CLI parsing, session setup, orchestration (7 files, 100K chars)
Lexer compiler/rustc_lexer/ Tokenization (2 files, 56K chars)
Parser compiler/rustc_parse/ Token stream → AST
AST compiler/rustc_ast/ Abstract Syntax Tree data structures
AST lowering compiler/rustc_ast_lowering/ AST → HIR transformation
HIR compiler/rustc_hir/ High-level IR (17 files, 374K chars)
HIR analysis compiler/rustc_hir_analysis/ Trait resolution, coherence
HIR type checking compiler/rustc_hir_typeck/ Type inference and checking (27 files, 1.1M chars)
MIR compiler/rustc_middle/ Mid-level IR, query system, TyCtxt (11 files, 230K chars)
MIR transforms compiler/rustc_mir_transform/ Optimization passes (78 files, 1.2M chars)
Borrow checker compiler/rustc_borrowck/ Ownership and lifetime analysis (17 files, 321K chars)
Name resolution compiler/rustc_resolve/ Import/name resolution (12 files, 940K chars)
Codegen (LLVM) compiler/rustc_codegen_llvm/ LLVM IR emission (22 files, 601K chars)
Codegen (Cranelift) compiler/rustc_codegen_cranelift/ Alternative Cranelift backend
Codegen (GCC) compiler/rustc_codegen_gcc/ Alternative GCC/libgccjit backend
Codegen interface compiler/rustc_codegen_ssa/ Shared codegen traits and logic
Session compiler/rustc_session/ Compiler options, diagnostics context (5 files in config/, 38K chars)
Type system IR compiler/rustc_type_ir/ Type representation shared with chalk (30 files, 398K chars)
Lints compiler/rustc_lint/ Built-in lint passes (53 files, 847K chars)
Lint definitions compiler/rustc_lint_defs/ Lint metadata and severity (2 files, 233K chars)
Features compiler/rustc_feature/ Feature gate definitions (6 files, 128K chars)
Pattern analysis compiler/rustc_pattern_analysis/ Exhaustiveness checking (9 files, 208K chars)
Macro expansion compiler/rustc_builtin_macros/ Built-in derive, format!, etc. (33 files, 346K chars)
Attribute parsing compiler/rustc_attr_parsing/ Attribute interpretation (10 files, 160K chars)
Data structures compiler/rustc_data_structures/ Shared concurrent data structures (32 files, 216K chars)
Monomorphization compiler/rustc_monomorphize/ Generic instantiation and collection (5 files, 144K chars)
Query system compiler/rustc_query_impl/ Incremental compilation queries
Traits/chalk compiler/rustc_traits/ Trait solving (9 files, 28K chars)
Thread pool compiler/rustc_thread_pool/ Rayon fork with deadlock detection (9 files, 108K chars)
Error messages compiler/rustc_error_messages/ Fluent-based i18n diagnostics (2 files, 19K chars)
AST passes compiler/rustc_ast_passes/ Post-expansion AST validation
Passes compiler/rustc_passes/ Miscellaneous analysis passes (19 files, 346K chars)
Type utilities compiler/rustc_ty_utils/ Type-level utilities (16 files, 209K chars)
Incremental compiler/rustc_incremental/ Incremental compilation support
Target specs compiler/rustc_target/ Platform target definitions
LLVM bindings compiler/rustc_llvm/ FFI to LLVM C++ API (1 file, 8K chars)
Log compiler/rustc_log/ Logging infrastructure (1 file, 10K chars)
FS utilities compiler/rustc_fs_util/ File system helpers (1 file, 5K chars)
Hashing compiler/rustc_hashes/ Hash algorithm implementations (1 file, 4K chars)
Windows RC compiler/rustc_windows_rc/ Windows resource compilation (1 file, 5K chars)

Technology Stack

Layer Technology
Language Rust (self-hosted), Python (build system), C++ (LLVM bindings)
Code generation LLVM (primary), Cranelift (experimental), GCC/libgccjit (experimental)
Build system x.py / bootstrap (Python + Rust hybrid)
Parallelism Custom Rayon fork (rustc_thread_pool) with deadlock detection
Logging tracing crate with RUSTC_LOG env var
Diagnostics Fluent-based error messages (rustc_error_messages)
Testing compiletest (custom test harness), tidy (style enforcement)
CI GitHub Actions with custom citool orchestration
Serialization Custom rustc_serialize for incremental compilation
Dependencies (JS) browser-ui-test ^0.23.3, eslint ^8.57.1, typescript ^5.8.3 (for rustdoc tests)
License MIT OR Apache-2.0

Compilation Pipeline Data Flow

Source (.rs)
  │
  ▼
Lexer (rustc_lexer) ──→ Token stream
  │
  ▼
Parser (rustc_parse) ──→ AST (rustc_ast)
  │
  ▼
Macro expansion (rustc_expand + rustc_builtin_macros)
  │
  ▼
AST validation (rustc_ast_passes)
  │
  ▼
AST lowering (rustc_ast_lowering) ──→ HIR (rustc_hir)
  │
  ▼
Name resolution (rustc_resolve)
  │
  ├──→ Type checking (rustc_hir_typeck + rustc_hir_analysis)
  │      │
  │      ▼
  │    Trait solving (rustc_traits)
  │
  ▼
MIR construction (rustc_mir_build) ──→ MIR (in rustc_middle)
  │
  ▼
Borrow checking (rustc_borrowck)
  │
  ▼
MIR optimization (rustc_mir_transform)
  │
  ▼
Monomorphization (rustc_monomorphize)
  │
  ▼
Code generation (rustc_codegen_ssa → rustc_codegen_llvm)
  │
  ▼
LLVM optimization passes → Object file (.o)
  │
  ▼
Linker → Executable / Library

Note

The query system (rustc_query_impl / rustc_middle) underpins the entire pipeline. Rather than a linear pass sequence, the compiler is demand-driven: each compilation stage is a query that lazily computes and caches its result. This enables incremental compilation — on subsequent builds, only invalidated queries are recomputed.

2. System Requirements

Build Host Requirements

Requirement Detail
OS Linux (x86_64, aarch64), macOS (x86_64, aarch64), Windows (x86_64)
Rust (bootstrap) Automatically downloaded; specified in src/stage0
Python 3.x required for x.py
C/C++ compiler GCC 7+, Clang 8+, or MSVC 2019+ (for LLVM)
CMake 3.13.4+ (for LLVM build)
Ninja Recommended (faster LLVM builds)
Disk space 30+ GB for full build with tests
RAM 8+ GB (16+ GB recommended for LLVM build with full parallelism)
Network Required for Stage 0 download and CI LLVM download

Runtime Dependencies

The built compiler has minimal runtime dependencies:

Dependency Platform Purpose
System linker All Linking object files into executables
libc / CRT All C runtime (linked by default)
libstdc++ / libc++ Linux/macOS For LLVM dynamic linking
kernel32.dll, etc. Windows OS API access

3. Configuration Reference

bootstrap.toml Sections

The bootstrap.toml file (generated by ./x setup or copied from bootstrap.example.toml) is the central build configuration. Below are the key sections and options derived from source.

[build] — General Build Settings

Key Type Description
host string[] Host triples to build for
target string[] Target triples to build standard library for
build string Build triple (auto-detected)
submodules bool Automatically update git submodules
docs bool Build documentation
compiler-docs bool Build compiler internal documentation
profiler bool Build the profiler runtime
sanitizers bool Build sanitizer runtimes
cargo-native-static bool Build cargo with static native libs
extended bool Build extended tool set (Clippy, rustfmt, etc.)
tools string[] Specific tools to build

[llvm] — LLVM Configuration

Key Type Description
download-ci-llvm bool Download pre-built LLVM instead of building from source
assertions bool Enable LLVM assertions (debug builds)
targets string LLVM targets to build (e.g., "X86;ARM;AArch64")
ninja bool Use Ninja for LLVM build (faster, higher peak memory)
release-debuginfo bool Build LLVM with debug info in release mode
use-linker string Linker for LLVM build (e.g., "lld")
link-shared bool Link LLVM as a shared library

[rust] — Compiler Configuration

Key Type Description
debug-assertions bool Enable debug assertions in the compiler
overflow-checks bool Enable integer overflow checks
incremental bool Enable incremental compilation
optimize int/bool Optimization level (0-3 or true/false)
codegen-units int Number of codegen units (lower = slower build, faster code)
lto string Link-time optimization ("off", "thin", "fat")
codegen-backends string[] Codegen backends to build (["llvm"], ["cranelift"], ["gcc"])
debug bool Build with debug info
channel string Release channel ("dev", "nightly", "beta", "stable")
description string Extra version description string
deny-warnings bool Deny compiler warnings
debug-assertions-std bool Enable debug assertions in the standard library

[target.<triple>] — Per-Target Configuration

Key Type Description
cc string C compiler path
cxx string C++ compiler path
ar string Archiver path
linker string Linker path
llvm-config string Path to llvm-config for external LLVM
android-ndk string Path to Android NDK

Environment Variables — Complete Reference

Compiler Runtime Variables

Variable Source Purpose
RUSTC_BOOTSTRAP compiler/rustc_feature/src/lib.rs:56 Enable unstable features on stable
RUSTC_BREAK_ON_ICE compiler/rustc_driver_impl/src/lib.rs:1606 Debugger breakpoint on ICE
RUSTC_OVERRIDE_VERSION_STRING compiler/rustc_driver_impl/src/lib.rs:894 Override version display
RUST_BACKTRACE compiler/rustc_codegen_gcc/build_system/src/main.rs:67 Enable panic backtraces
RUST_DEP_GRAPH compiler/rustc_incremental/src/assert_dep_graph.rs:207 Dump dependency graph
RUST_DEP_GRAPH_FILTER compiler/rustc_incremental/src/assert_dep_graph.rs:209 Filter dependency graph output

Thread Pool Variables

Variable Source Purpose
RAYON_NUM_THREADS compiler/rustc_thread_pool/src/lib.rs:491 Thread pool size
RAYON_RS_NUM_CPUS compiler/rustc_thread_pool/src/lib.rs:498 Override detected CPU count

Test Framework Variables

Variable Source Purpose
RUST_TEST_THREADS src/tools/compiletest/src/executor.rs:317 Parallel test threads
RUST_TEST_SHUFFLE library/test/src/cli.rs:349 Randomize test order
RUST_TEST_SHUFFLE_SEED library/test/src/cli.rs:373 Seed for test shuffling
RUST_TEST_NOCAPTURE library/test/src/cli.rs:455 Don't capture test output
RUST_TARGET_PATH src/tools/compiletest/src/common.rs:906 Custom target spec search path
NO_COLOR src/tools/compiletest/src/cli.rs:16 Disable colored output

Clippy Variables

Variable Source Purpose
CLIPPY_CONF_DIR src/tools/clippy/src/driver.rs:149 Configuration directory path
CLIPPY_ARGS src/tools/clippy/src/driver.rs:279 Additional Clippy arguments
CLIPPY_TERMINAL_WIDTH src/tools/clippy/clippy_config/src/conf.rs:1178 Terminal width
CLIPPY_DISABLE_DOCS_LINKS src/tools/clippy/clippy_utils/src/diagnostics.rs:20 Disable doc links
SYSROOT src/tools/clippy/src/driver.rs:230 Override sysroot path
CARGO_PRIMARY_PACKAGE src/tools/clippy/src/driver.rs:300 Primary workspace package

Miri Variables

Variable Source Purpose
MIRIFLAGS src/tools/miri/cargo-miri/src/phases.rs:599 Additional Miri flags
MIRI_VERBOSE src/tools/miri/cargo-miri/src/phases.rs:339 Verbose output
MIRI_LOCAL_CRATES src/tools/miri/src/machine.rs:866 Local crate list
MIRI_ORIG_RUSTDOC src/tools/miri/cargo-miri/src/phases.rs:628 Original rustdoc path
CARGO_EXTRA_FLAGS src/tools/miri/miri-script/src/util.rs:115 Extra Cargo flags
HYPERFINE src/tools/miri/miri-script/src/commands.rs:267 Hyperfine path for benchmarks

rust-analyzer Variables

Variable Source Purpose
RA_TARGET src/tools/rust-analyzer/xtask/src/util.rs:36 Target triple override
RA_LRU_CAP src/tools/rust-analyzer/crates/load-cargo/src/lib.rs:87 LRU cache capacity
RUST_SRC_PATH src/tools/rust-analyzer/crates/project-model/src/sysroot.rs:451 Rust source path
RA_METRICS src/tools/rust-analyzer/crates/rust-analyzer/src/cli.rs:54 Metrics output path
RUN_SLOW_BENCHES src/tools/rust-analyzer/crates/rust-analyzer/src/integrated_benchmarks.rs:38 Enable slow benchmarks
SKIP_SLOW_TESTS src/tools/rust-analyzer/crates/test-utils/src/lib.rs:395 Skip slow tests
RA_DONT_COPY_PROC_MACRO_DLL src/tools/rust-analyzer/crates/proc-macro-srv/src/dylib.rs:176 Skip DLL copy
CHALK_DEBUG src/tools/rust-analyzer/crates/hir-ty/src/lib.rs:693 Chalk solver debug

Cranelift Backend Variables

Variable Source Purpose
CG_CLIF_JIT_ARGS compiler/rustc_codegen_cranelift/src/config.rs:20 JIT arguments
CG_CLIF_ENABLE_VERIFIER compiler/rustc_codegen_cranelift/src/lib.rs:243 Enable IR verifier
CG_CLIF_EXPENSIVE_CHECKS compiler/rustc_codegen_cranelift/build_system/build_backend.rs:30 Expensive validation
BENCH_RUNS compiler/rustc_codegen_cranelift/build_system/bench.rs:29 Benchmark iteration count

GCC Backend Variables

Variable Source Purpose
CG_GCCJIT_DUMP_GIMPLE compiler/rustc_codegen_gcc/src/base.rs:185 Dump GIMPLE IR
CG_GCCJIT_DUMP_RTL compiler/rustc_codegen_gcc/src/base.rs:170 Dump RTL
CG_GCCJIT_DUMP_RTL_ALL compiler/rustc_codegen_gcc/src/base.rs:173 Dump all RTL
CG_GCCJIT_DUMP_TREE_ALL compiler/rustc_codegen_gcc/src/base.rs:176 Dump all tree IR
CG_GCCJIT_DUMP_IPA_ALL compiler/rustc_codegen_gcc/src/base.rs:179 Dump all IPA
CG_GCCJIT_DUMP_CODE compiler/rustc_codegen_gcc/src/base.rs:182 Dump generated assembly
CG_GCCJIT_DUMP_EVERYTHING compiler/rustc_codegen_gcc/src/base.rs:188 Enable all dumps
CG_GCCJIT_KEEP_INTERMEDIATES compiler/rustc_codegen_gcc/src/base.rs:191 Retain intermediate files
CG_GCCJIT_VERBOSE compiler/rustc_codegen_gcc/src/base.rs:194 Verbose mode
CG_RUSTFLAGS compiler/rustc_codegen_gcc/build_system/src/build.rs:168 Extra rustc flags

Build System / CI Variables

Variable Source Purpose
RUST_IGNORE_OLD_PYTHON x.py:33 Suppress old Python warnings
GITHUB_ACTIONS compiler/rustc_codegen_cranelift/build_system/utils.rs:234 CI detection
GITHUB_STEP_SUMMARY compiler/rustc_codegen_cranelift/build_system/bench.rs:31 CI summary output
GITHUB_TOKEN src/tools/rust-analyzer/xtask/src/publish.rs:74 GitHub API auth
CI src/tools/rust-analyzer/xtask/src/codegen.rs:188 CI detection
BOOTSTRAP_SKIP_YARN_LOCK_CHECK src/build_helper/src/npm.rs:41 Skip yarn.lock check

Standard Library Build Variables

Variable Source Purpose
CARGO_CFG_TARGET_OS library/profiler_builtins/build.rs:22 Target OS for build scripts
CARGO_CFG_TARGET_ENV library/profiler_builtins/build.rs:23 Target environment
CARGO_CFG_TARGET_ARCH compiler/rustc_windows_rc/src/lib.rs:39 Target architecture
CARGO_CFG_TARGET_VENDOR library/std/build.rs:8 Target vendor
RUSTC_STD_NETBSD10 library/std/build.rs:12 NetBSD 10 detection
RUSTC_BOOTSTRAP_SYNTHETIC_TARGET library/std/build.rs:62 Synthetic target for bootstrap
CFG_VERSION compiler/rustc_windows_rc/src/lib.rs:76 Version string
CFG_RELEASE_CHANNEL compiler/rustc_windows_rc/src/lib.rs:79 Release channel
CFG_RELEASE compiler/rustc_windows_rc/src/lib.rs:83 Release version

Important

Variables prefixed with CG_GCCJIT_* and CG_CLIF_* only affect their respective codegen backends and are ignored when using the default LLVM backend.

Standard Library stdarch Test Variables

Variable Source Purpose
STDARCH_ASSERT_INSTR_LIMIT library/stdarch/crates/stdarch-test/src/lib.rs:130 Instruction count limit
STDARCH_TEST_EVERYTHING library/stdarch/crates/stdarch-test/src/lib.rs:225 Test all intrinsics
STDARCH_DISABLE_ASSERT_INSTR library/stdarch/crates/assert-instr-macro/src/lib.rs:52 Disable instruction checks
OBJDUMP library/stdarch/crates/stdarch-test/src/disassembly.rs:78 Objdump binary path

4. Deployment & Scaling

Multi-Stage Bootstrap

The compiler builds itself through three stages:

Stage Input Output Purpose
0 Pre-built beta compiler (downloaded) Stage 0 artifacts Compile the in-tree compiler source
1 Stage 0 output Stage 1 compiler + std Primary development/testing compiler
2 Stage 1 output Stage 2 compiler + std Self-hosted validation, release builds

The bootstrap process is managed by src/bootstrap/ (excluded from the Cargo workspace — it builds independently) and orchestrated by x.py.

Distribution Builds

./x setup dist
./x dist

This produces platform-specific tarballs in build/dist/ containing:

  • The compiler (rustc)
  • Standard library
  • Cargo (external, fetched)
  • Clippy, rustfmt, and other extended tools
  • Offline documentation
  • Source distribution

The src/tools/build-manifest/ (4 files, 37K chars) generates the manifest files that rustup uses to discover available components. The src/tools/rust-installer/ (8 files, 35K chars) handles tarball creation and install scripts.

Cross-Compilation

To build the standard library for additional targets:

# bootstrap.toml
[build]
target = ["x86_64-unknown-linux-gnu", "aarch64-unknown-linux-gnu", "wasm32-unknown-unknown"]

[target.aarch64-unknown-linux-gnu]
cc = "aarch64-linux-gnu-gcc"
cxx = "aarch64-linux-gnu-g++"
linker = "aarch64-linux-gnu-gcc"

Note

Cross-compilation for embedded targets (e.g., thumbv6m-none-eabi, wasm32-unknown-unknown) only builds core and alloc (no std). The standard library conditionally compiles OS-specific code based on cfg(target_os), cfg(target_arch), and related configuration values controlled by compiler/rustc_session/src/config/cfg.rs.

Profile-Guided Optimization (PGO)

The src/tools/opt-dist/ tool automates PGO builds for the compiler:

# Set up PGO build
DIST_TRY_BUILD=1 ./x dist
Variable Source Purpose
DIST_TRY_BUILD src/tools/opt-dist/src/main.rs:130 Enable PGO build
PGO_HOST src/tools/opt-dist/src/main.rs:175 PGO host triple

5. Operations

Build System Commands

Operation Command
Full build ./x build
Type-check only ./x check
Run tests ./x test
Build docs ./x doc
Format code ./x fmt
Run lints ./x clippy
Build distribution ./x dist
Install locally ./x install
Clean build artifacts ./x clean
Run benchmarks ./x bench
Vendor dependencies ./x vendor

Tip

For CI workflows, the src/ci/citool/ (9 files, 67K chars) provides job orchestration, metric collection, and test scheduling. It reads workflow definitions and routes jobs to appropriate runners.

Monitoring Compiler Internals

Technique Command/Flag
Time each compiler pass rustc -Z time-passes
Dump MIR rustc -Z dump-mir=all
Dump HIR rustc -Z unpretty=hir-tree
Print type sizes rustc -Z print-type-sizes
Enable logging RUSTC_LOG=rustc_borrowck=debug
Dependency graph RUST_DEP_GRAPH=1 rustc ...
Query profiling rustc -Z self-profile

Tip

The -Z self-profile flag produces a trace file that can be analyzed with summarize and crox tools to visualize which queries take the most time. This is the primary tool for compiler performance work.

Incremental Compilation

Incremental compilation is managed by compiler/rustc_incremental/ and the query system. Key files:

  • compiler/rustc_incremental/src/assert_dep_graph.rs — dependency graph assertions
  • compiler/rustc_incremental/src/persist/ — cache serialization

Enable with:

[rust]
incremental = true

On subsequent builds, the compiler loads the dependency graph, invalidates changed queries, and only recomputes affected results.

6. Performance

Build Performance Optimization

Technique Impact Configuration
Download CI LLVM Saves 20–40 min [llvm] download-ci-llvm = true
Incremental builds Saves 80–95% on small changes [rust] incremental = true
--keep-stage 1 Skip Stage 0 rebuild Command flag
./x check Skip codegen entirely Build type
Reduce codegen-units Faster compilation, less optimization [rust] codegen-units = 16
Ninja for LLVM 10–20% faster LLVM build [llvm] ninja = true
LLD linker Faster linking [rust] use-lld = true

Compiler Runtime Performance

The compiler uses several techniques for runtime performance:

  • Parallel frontend: The rustc_thread_pool (a fork of Rayon with deadlock detection) parallelizes type checking and other frontend passes. Thread count is controlled by RAYON_NUM_THREADS.
  • Query caching: Every intermediate result is cached as a query. The rustc_middle crate manages TyCtxt, the central context that holds all cached state.
  • Arena allocation: compiler/rustc_arena/ provides typed arena allocators to reduce allocation overhead.
  • Jemalloc: The compiler binary uses tikv-jemalloc-sys for memory allocation when built for distribution, providing better allocation performance than the system allocator.

7. Security

Compiler Integrity

The Rust compiler is built from a trusted bootstrap chain:

  1. Stage 0 is downloaded from Rust infrastructure with verified checksums
  2. Each subsequent stage is built by the previous stage
  3. CI enforces reproducible builds and tests on every commit

Security-Relevant Configuration

Setting Description Risk
RUSTC_BOOTSTRAP=1 Enables unstable features Bypasses stability guarantees
debug-assertions = false Disables internal checks Miscompilation may go undetected
-Z shell-argfiles Enable shell-style argument file expansion Argument injection if files are untrusted
Feature gates compiler/rustc_feature/src/ Unstable features behind feature flags

Warning

The RUSTC_BOOTSTRAP variable should only be set by the build system itself. External use bypasses the stability guarantee that is the primary safety contract of the Rust release channels.

Built-in Cfg Protections

The compiler (compiler/rustc_session/src/config/cfg.rs) enforces that certain --cfg values cannot be set by the user via the command line because they would create incoherent behavior:

Protected Cfg Controlled By
overflow_checks -C overflow-checks
debug_assertions -C debug-assertions
ub_checks -Z ub-checks
contract_checks -Z contract-checks
sanitize -Z sanitizer
proc_macro --crate-type proc-macro
panic -C panic
target_feature -C target-feature
target_arch, target_os, etc. --target
fmt_debug -Z fmt-debug

Attempting to set these via --cfg emits the EXPLICIT_BUILTIN_CFGS_IN_FLAGS lint.

Caution

Sanitizer flags (-Z sanitizer=address,cfi,memory,thread,hwaddress,kcfi) fundamentally change code generation. KASAN maps to ADDRESS and KHWASAN maps to HWADDRESS internally. Mixing sanitizer-instrumented and uninstrumented code in the same binary causes undefined behavior.

Memory Safety in the Compiler Itself

The compiler is written in Rust and benefits from its own safety guarantees. The few unsafe blocks are concentrated in:

  • compiler/rustc_data_structures/ (concurrent data structures, vec_cache, steal)
  • compiler/rustc_arena/ (arena allocation with smallvec may_dangle)
  • compiler/rustc_codegen_llvm/ (FFI to LLVM C++ API)

8. Developer Internals

Repository Structure

rust-lang/rust/
├── compiler/                  # ~50 crates comprising the compiler
│   ├── rustc/                 # Entry point binary (main.rs → rustc_driver)
│   ├── rustc_driver_impl/     # CLI, session setup, compilation orchestration
│   ├── rustc_lexer/           # Tokenizer (no dependencies on rest of compiler)
│   ├── rustc_parse/           # Parser
│   ├── rustc_ast/             # AST data structures
│   ├── rustc_ast_lowering/    # AST → HIR
│   ├── rustc_hir/             # HIR data structures
│   ├── rustc_hir_analysis/    # Trait resolution, coherence
│   ├── rustc_hir_typeck/      # Type inference (27 files, 1.1M chars)
│   ├── rustc_borrowck/        # Borrow checker (17 files, 321K chars)
│   ├── rustc_resolve/         # Name resolution (12 files, 940K chars)
│   ├── rustc_middle/          # Central types: TyCtxt, MIR, query system
│   ├── rustc_mir_transform/   # MIR optimization passes (78 files, 1.2M chars)
│   ├── rustc_codegen_llvm/    # LLVM code generation (22 files, 601K chars)
│   ├── rustc_codegen_cranelift/  # Cranelift backend
│   ├── rustc_codegen_gcc/     # GCC/libgccjit backend
│   ├── rustc_codegen_ssa/     # Shared codegen traits
│   ├── rustc_lint/            # Built-in lints (53 files, 847K chars)
│   ├── rustc_session/         # Compiler options and session state
│   ├── rustc_type_ir/         # Type IR shared with chalk (30 files, 398K chars)
│   ├── rustc_feature/         # Feature gate definitions
│   ├── rustc_thread_pool/     # Rayon fork with deadlock detection
│   ├── rustc_data_structures/ # Shared data structures (32 files, 216K chars)
│   └── ...                    # ~30 more crates
├── library/                   # Standard library
│   ├── core/                  # #![no_std] foundation
│   ├── alloc/                 # Heap allocation (15 files, 707K chars)
│   ├── std/                   # Full standard library with OS integration
│   ├── proc_macro/            # Procedural macro support
│   ├── test/                  # Test framework (#[test])
│   ├── compiler-builtins/     # Compiler intrinsics (12 files, 62K chars)
│   ├── stdarch/               # Platform SIMD intrinsics
│   ├── portable-simd/         # Portable SIMD abstractions
│   └── backtrace/             # Stack trace generation
├── src/
│   ├── bootstrap/             # Build system (excluded from workspace)
│   ├── tools/                 # Development tools
│   │   ├── clippy/            # Linter (800+ lints)
│   │   ├── rustfmt/           # Code formatter
│   │   ├── miri/              # MIR interpreter
│   │   ├── rust-analyzer/     # IDE/LSP server
│   │   ├── compiletest/       # Test runner
│   │   ├── tidy/              # Style/hygiene enforcement
│   │   ├── opt-dist/          # PGO build orchestration
│   │   ├── rust-installer/    # Packaging/distribution
│   │   ├── build-manifest/    # Rustup manifest generation
│   │   └── ...                # ~30 more tools
│   ├── librustdoc/            # Rustdoc implementation
│   │   ├── html/              # HTML generation (13 files, 244K chars)
│   │   └── json/              # JSON output (4 files, 64K chars)
│   ├── ci/                    # CI tooling
│   │   └── citool/            # Job orchestration (9 files, 67K chars)
│   ├── build_helper/          # Build script utilities (8 files, 34K chars)
│   └── doc/                   # Documentation sources
│       ├── rustc-dev-guide/   # Compiler contributor guide
│       ├── rust-by-example/   # Examples
│       ├── embedded-book/     # Embedded Rust book
│       └── style-guide/       # Rust style guide
├── tests/                     # 50,000+ test files
│   ├── ui/                    # Compiler diagnostics
│   ├── mir-opt/               # MIR optimization (135 files, 237K chars)
│   ├── codegen-llvm/          # LLVM codegen (351 files, 553K chars)
│   ├── run-make/              # Integration tests
│   ├── run-make-cargo/        # Cargo-based integration
│   ├── rustdoc-js/            # Rustdoc search tests (152 files, 125K chars)
│   ├── rustdoc-gui/           # Rustdoc rendering tests
│   └── ui-fulldeps/           # Tests requiring full deps (20 files, 46K chars)
├── x.py                       # Build system entry point
├── x / x.ps1                  # Shell wrappers
├── Cargo.toml                 # Workspace manifest
├── bootstrap.example.toml     # Annotated configuration template
└── REUSE.toml                 # License annotations

Architecture Deep Dive

The Query System

The query system is the backbone of the compiler. Rather than executing compilation as a linear sequence of passes, each computational step is modeled as a query that takes inputs and produces cached outputs. Queries are:

  • Demand-driven: computed lazily when their result is first needed
  • Memoized: results are cached for the duration of the compilation
  • Incremental: on re-compilation, only queries whose inputs changed are recomputed

The central query context is TyCtxt<'tcx> in compiler/rustc_middle/, which holds all compilation state and provides access to every query.

Thread Pool and Parallelism

The compiler uses a custom fork of Rayon (compiler/rustc_thread_pool/, 9 files, 108K chars) with additional deadlock detection. The sleep module implements a sophisticated protocol using atomic counters and sequentially consistent fences to coordinate worker threads:

  • Active threads: executing a job
  • Idle threads: searching for work (stealing from other queues)
  • Sleeping threads: blocked on a condition variable

The Jobs Event Counter (JEC) uses an even/odd toggle protocol to detect when new work is available without expensive synchronization. Deadlock detection monitors active_threads and blocked_threads counts — a deadlock is flagged when active_threads == 0 && blocked_threads > 0.

Diagnostic System

Error messages use Fluent-based internationalization (compiler/rustc_error_messages/, 2 files, 19K chars). The compiler's error index contains 997 unique error patterns spanning:

  • Pattern analysis errors (malformed ranges, unexpected constructors)
  • MIR transformation errors (invalid phase transitions)
  • Type checking errors (inference failures, trait resolution)
  • Borrow checking errors (moved values, lifetime violations)
  • Lint infrastructure errors (push/pop mismatches, renamed lints)

The driver (compiler/rustc_driver_impl/src/highlighter.rs) includes a syntax highlighter for rustc --explain output, coloring keywords, types, literals, and function calls.

Codegen Backends

Backend Path Status Use Case
LLVM compiler/rustc_codegen_llvm/ Production Default, best optimization
Cranelift compiler/rustc_codegen_cranelift/ Experimental Faster debug builds
GCC compiler/rustc_codegen_gcc/ Experimental Broader platform support

All backends implement the traits defined in compiler/rustc_codegen_ssa/. The LLVM backend interfaces with LLVM through compiler/rustc_llvm/ (C++ FFI wrapper).

The entry point binary (compiler/rustc/src/main.rs) is minimal:

fn main() -> ExitCode {
    rustc_driver::main()
}

It uses jemalloc on supported platforms through the tikv-jemalloc-sys crate with override_allocator_on_supported_platforms, which replaces the system allocator's malloc/free at the libc level.

Module Reference

compiler/rustc_driver_impl/ — Compiler Driver

The driver (7 files, 100K chars) orchestrates the compilation pipeline:

  • lib.rs — Entry point: CLI parsing, session creation, callback registration, --explain handler, ICE hook installation. Key types: TimePassesCallbacks, Compilation (enum: Continue/Stop), HandledOptions
  • args.rs — Argument file expansion (@file syntax). Supports -Z shell-argfiles for shell-style quoting
  • highlighter.rs — Syntax highlighting for --explain output. Recognizes Rust keywords, primitive types, string/char/number literals, function calls, and derive attributes
  • pretty.rs — Pretty-printing passes (-Z unpretty=...)

compiler/rustc_session/ — Session Configuration

The session crate manages compiler options and the diagnostic context. The config/ subdirectory (5 files, 38K chars) includes:

  • cfg.rs — --cfg and --check-cfg processing, built-in cfg protections, default configuration generation
  • Key types: Cfg (set of name-value pairs), CheckCfg (exhaustive cfg validation), ExpectedValues
  • Built-in cfgs: debug_assertions, overflow_checks, ub_checks, contract_checks, panic, proc_macro, target_*, sanitize, fmt_debug

compiler/rustc_hir_typeck/ — Type Inference

The largest compiler crate by source volume (27 files, 1.1M chars). Handles:

  • Type inference with unification variables
  • Method resolution
  • Coercion and cast checking
  • Closure typing
  • Operator overloading resolution

compiler/rustc_borrowck/ — Borrow Checker

17 files, 321K chars. Implements Non-Lexical Lifetimes (NLL):

  • Region inference
  • Move/borrow analysis
  • Lifetime error diagnostics

compiler/rustc_mir_transform/ — MIR Optimizations

The largest by file count (78 files, 1.2M chars). Individual optimization passes include:

  • Inlining (inline.rs) — with ForceInline for #[inline(always)]
  • Constant propagation
  • Dead code elimination
  • Copy propagation
  • Promotion of constants to statics

compiler/rustc_resolve/ — Name Resolution

12 files, 940K chars. Resolves all names in the program:

  • Import resolution (glob, named, re-exports)
  • Macro name resolution
  • Lifetime name resolution
  • Privacy checking

compiler/rustc_lint/ — Lint Infrastructure

53 files, 847K chars. The compiler ships hundreds of built-in lints organized into:

  • levels.rs — Lint level computation (allow, warn, deny, forbid)
  • expect.rs — #[expect(lint)] tracking
  • Individual lint passes (non_fmt_panic, lifetime_syntax, etc.)

The lint definitions themselves live in compiler/rustc_lint_defs/ (2 files, 233K chars).

compiler/rustc_data_structures/ — Shared Data Structures

32 files, 216K chars. Thread-safe data structures used across the compiler:

  • steal.rs — Single-writer steal cells for inter-query data transfer
  • sync.rs — Dynamic thread-safety mode selection
  • vec_cache.rs — Bucket-indexed cache with atomic operations
  • small_c_str.rs — Small C string optimization for FFI

compiler/rustc_thread_pool/ — Rayon Fork

9 files, 108K chars. A fork of the Rayon work-stealing thread pool with:

  • Deadlock detection for user-code blocking
  • Configurable via RAYON_NUM_THREADS and RAYON_RS_NUM_CPUS
  • Atomic sleep/wake protocol using seq-cst fences
  • Jobs Event Counter (JEC) for efficient new-work detection

compiler/rustc_pattern_analysis/ — Exhaustiveness Checking

9 files, 208K chars. Implements pattern exhaustiveness and usefulness analysis:

  • constructor.rs — Pattern constructors (int ranges, slices, variants)
  • usefulness.rs — Core exhaustiveness algorithm
  • rustc.rs — Rust-specific pattern lowering

library/alloc/ — Allocation Library

15 files, 707K chars. Provides heap-allocated types for #![no_std] environments:

  • Vec<T>, String, Box<T>, Rc<T>, Arc<T>
  • B-tree maps and sets
  • Binary heap
  • Linked list

src/librustdoc/ — Rustdoc

Documentation generator with two output modes:

  • html/ (13 files, 244K chars) — HTML generation with syntax highlighting, search index, and responsive CSS
  • json/ (4 files, 64K chars) — Machine-readable JSON documentation format

Code Patterns and Conventions

Naming Conventions

Pattern Usage
rustc_* crate prefix All compiler crates
TyCtxt<'tcx> The central query/type context
'tcx lifetime Tied to the type context arena
tcx variable name Instance of TyCtxt
dcx() Diagnostic context accessor
DefId / LocalDefId Definition identifiers
HirId / OwnerId HIR node identifiers
ItemLocalId Local item identifier within an owner
sym::* Interned symbol constants
kw::* Keyword symbols

Common Patterns

  • Query definition: Queries are defined via macros in rustc_middle and implemented in the crate that owns the computation
  • Arena allocation: Long-lived data is allocated in arenas tied to 'tcx, avoiding individual heap allocations
  • Visitor pattern: Both AST and HIR use visitor traits (intravisit::Visitor) for tree traversal
  • Diagnostic builder: Errors are constructed via struct_span_err! / span_bug! macros, allowing machine-readable structured diagnostics
  • Feature gates: Unstable features are gated in compiler/rustc_feature/src/ and checked at the AST pass level
  • #[instrument]: The tracing crate's #[instrument] attribute is used throughout for structured logging (e.g., #[instrument(level = "debug", skip(self))])
  • tidy-alphabetical-start/-end: Comments marking sections that must be kept alphabetically sorted

The Workspace Manifest

The root Cargo.toml defines a Rust workspace with:

  • ~45 workspace members (compiler, tools, support crates)
  • Excludes: build/, compiler/rustc_codegen_cranelift, compiler/rustc_codegen_gcc, src/bootstrap, tests/rustdoc-gui
  • Custom profiles: overflow-checks = false for rustc_thread_pool (workaround for CI stack overflows, see issue #90227), strip = true for lld-wrapper and wasm-component-ld

Testing Strategy

The test infrastructure is managed by compiletest (src/tools/compiletest/) with multiple test suites:

Suite Directory Tests Purpose
UI tests/ui/ Thousands Compiler diagnostics — expected errors, warnings, suggestions
MIR opt tests/mir-opt/ 135 files MIR optimization correctness
Codegen LLVM tests/codegen-llvm/ 351 files Generated LLVM IR verification
Run-make tests/run-make/ Hundreds Integration: compile + link + run
Run-make Cargo tests/run-make-cargo/ Dozens Cargo-based integration
Rustdoc JS tests/rustdoc-js/ 152 files Documentation search functionality
Rustdoc GUI tests/rustdoc-gui/ — Browser-based rendering tests
UI full-deps tests/ui-fulldeps/ 20 files Tests requiring full compiler dependencies
Incremental tests/incremental/ — Incremental compilation correctness
Assembly tests/assembly/ — Generated assembly verification

Test annotations (in .rs files):

  • //~ ERROR — expected error on this line
  • //~ WARN — expected warning
  • //~ HELP — expected help suggestion
  • //@ compile-flags: — extra compiler flags
  • //@ run-pass — test should compile and run successfully
  • //@ ignore-* / //@ only-* — platform filtering
  • //@ revisions: — parameterized test variants

Style enforcement: The tidy tool (src/tools/tidy/) checks:

  • File formatting and line length
  • License headers
  • Feature gate documentation
  • Alphabetical ordering of declared lists
  • External dependency allowlists

Note

The --bless flag updates expected output to match actual compiler output. This is the standard workflow for updating tests after intentional diagnostic changes.

Potential Improvements

Based on source analysis:

  1. Incremental compilation robustness: The RUST_DEP_GRAPH / RUST_DEP_GRAPH_FILTER debugging tools suggest ongoing work on dependency graph correctness. The assert_dep_graph module contains assertion infrastructure for catching incremental compilation bugs.

  2. Thread pool deadlock detection: The rustc_thread_pool fork includes custom deadlock detection (SleepData tracking) that is more conservative than upstream Rayon — the overflow-checks = false profile override for this crate is a workaround for CI stability (issue #90227).

  3. GCC backend maturity: compiler/rustc_codegen_gcc/ provides an alternative code generation path through libgccjit. The extensive CG_GCCJIT_* debugging variables indicate active development. This backend could expand Rust's platform support to architectures not covered by LLVM.

  4. Diagnostic infrastructure: The 997 unique error patterns extracted from source span every major compilation phase. The Fluent-based i18n system (rustc_error_messages) is relatively new and positions the compiler for multilingual diagnostics.

  5. Self-profile tooling: The -Z self-profile flag and associated tooling provide detailed query-level performance data. This is the primary mechanism for identifying compilation bottlenecks and has been used to drive significant performance improvements.

Contributing Guide

  1. Find an issue: Browse issues labeled E-easy or E-mentor at https://github.com/rust-lang/rust/issues
  2. Set up: ./x setup compiler (or library/tools depending on focus)
  3. Make changes: Edit code, run ./x check for fast feedback
  4. Test: ./x test tests/ui (or the appropriate suite), use ./x suggest for guidance
  5. Format: ./x fmt
  6. Lint: ./x test tidy
  7. Submit: Push to a fork, open a PR. CI runs the full test matrix.

The comprehensive contributor guide is at https://rustc-dev-guide.rust-lang.org/ (source: src/doc/rustc-dev-guide/).

Key development conventions:

  • All compiler crates use the rustc_ prefix
  • Dependencies within compiler/ use path dependencies
  • External dependencies require allowlisting in src/tools/tidy/
  • Code changes must pass ./x test tidy and ./x fmt --check
  • New features require a feature gate in compiler/rustc_feature/
  • Diagnostic changes should update affected UI test expectations with --bless

9. Integration Points

Rustup Integration

The src/tools/build-manifest/ generates manifests consumed by rustup. Distribution artifacts are produced by ./x dist and uploaded to the Rust release infrastructure.

IDE Integration (rust-analyzer)

The src/tools/rust-analyzer/ is a full Language Server Protocol implementation. It uses several compiler crates directly (proc-macro-srv for proc macro expansion). Key integration points:

  • editors/code/ — VS Code extension (TypeScript, reads VSCODE_EXEC_PATH, PATH, CARGO_HOME)
  • crates/rust-analyzer/src/handlers/ — LSP request handlers (3 files, 143K chars)
  • crates/ide/src/ — Core IDE features: goto-definition, rename, completion, highlighting (43 files, 988K chars)
  • crates/ide-assists/src/handlers/ — Code actions and refactorings (134 files, 2.3M chars)

CI Integration

The src/ci/citool/ orchestrates GitHub Actions CI:

  • jobs.rs — Workflow type detection and job list computation
  • metrics.rs — Build metrics deserialization from CI URLs
  • utils.rs — Git command helpers
Error Source Meaning
Cannot determine the type of workflow citool/src/jobs.rs:376 Unknown CI workflow type
Computed job list is empty citool/src/jobs.rs:382 No CI jobs matched filters
Only Linux jobs can be executed locally citool/src/jobs.rs:413 Local execution limitation

Unstable Feature Documentation

The src/tools/unstable-book-gen/ automatically generates documentation for unstable features from their feature gate definitions in compiler/rustc_feature/. The src/tools/features-status-dump/ produces machine-readable feature status reports.

License Tooling

The src/tools/collect-license-metadata/ (4 files, 20K chars) and src/tools/generate-copyright/ (2 files, 14K chars) automate license compliance. The root REUSE.toml provides structured license annotations following the REUSE specification (version 1, using reuse-tool v4.0.3). The primary license is MIT OR Apache-2.0, with specific exceptions for LLVM (Apache-2.0 WITH LLVM-exception), GCC components (GPL-3.0+), and font files (OFL-1.1).