d2l-zh Administrator & Developer Guide

Administrator, maintainer, and contributor documentation for the Chinese edition of Dive into Deep Learning (动手学深度学习) — covering build pipeline, multi-framework code extraction, LaTeX post-processing, CI infrastructure, and fork strategy.

1. Architecture Overview

1.1 What this repo is, operationally

d2l-ai/d2l-zh is not a conventional software project. It is an open, reproducible textbook: source Markdown chapters that contain executable code cells, which are compiled by a custom book-build tool (d2lbook) into:

• A browsable HTML website (https://zh-v2.d2l.ai/)
• Four framework-specific PDFs (MXNet, PyTorch, TensorFlow, PaddlePaddle)
• A downloadable ZIP of Jupyter notebooks
• A pip-installable helper Python package (d2l) generated from annotated code cells

Every chapter renders four parallel implementations of the same deep-learning concepts — one per supported framework — from a single Markdown source. The d2lbook tool handles the fan-out. This design is what makes the repository operationally unusual and what most of this guide addresses.

1.2 The two surfaces

d2l-zh ships two distinct artifacts from one source tree:

                         source tree
                             │
        ┌────────────────────┼────────────────────┐
        │                                         │
  chapter_*/                                  d2l/
  (Markdown + code cells)                     (Python package)
        │                                         │
        ▼                                         ▼
 ┌──────────────────┐                     ┌──────────────────┐
 │   d2lbook        │                     │   setup.py       │
 │   - evaluates    │                     │   - packages     │
 │   - renders HTML │                     │   - publishes    │
 │   - renders PDF  │                     │   - versioned    │
 │   - zips nb's    │                     │   - pinned deps  │
 └────────┬─────────┘                     └────────┬─────────┘
          │                                        │
          ▼                                        ▼
  zh-v2.d2l.ai/*                           pip install d2l
  zh-v2.d2l.ai/d2l-zh.pdf                  (4 frameworks, one wheel)
  zh-v2.d2l.ai/d2l-zh.zip

The two are linked: the d2l Python package is partially generated from the chapter code cells via the #@save annotation. A reusable function defined inline in a chapter (e.g., load_data_fashion_mnist) is tagged with #@save and automatically collected by d2lbook build lib into d2l/<framework>.py. Readers then import d2l.torch as d2l and call d2l.load_data_fashion_mnist(...) from their own notebooks.

This loop (book code ↔ library code) is the core architectural idea:

1. The book teaches the function.
2. The book uses the function in later chapters.
3. The d2l package exports the function to the reader's own code.

Note

You cannot refactor a #@save-annotated function purely as a library maintainer. The function's name, signature, and semantics are published to readers both textually (in the book) and programmatically (via pip). Breaking changes require a version bump and coordinated edits to many chapters.

1.3 Component inventory

1.4 Technology stack

1.5 At-a-glance stats

1.6 Build pipeline (end to end)

   source .md (with :begin_tab:<fw> code cells)
        │
        │  d2lbook build eval --tab <fw>  ← runs per framework on GPU
        ▼
   _build/eval_<fw>/*.md (with evaluated outputs embedded)
        │
        │  d2lbook build html  (Sphinx HTML theme)
        │  d2lbook build pdf --tab <fw>  (Sphinx LaTeX → post_latex/main.py → xelatex)
        │  d2lbook build pkg  (collects #@save cells into d2l/<fw>.py)
        ▼
   _build/html/  ,  _build/pdf/d2l-zh.pdf  ,  d2l-*.whl
        │
        │  d2lbook deploy html pdf pkg colab sagemaker slides --s3 s3://zh-v2.d2l.ai/
        ▼
   CloudFront → https://zh-v2.d2l.ai/

1.7 CI architecture

GitHub push/PR
     │
     ▼
.github/workflows/ci.yml  (GitHub Actions, ubuntu-latest runner)
     │
     │  for each framework (torch, tf, mxnet, paddle):
     │    use ./.github/actions/submit-job → ci/submit-job.py → boto3 AWS Batch
     │
     ▼
AWS Batch  (D2L-CI-GPU queue,  job definitions d2l-ci-zh-gpu-<fw>:N)
     │
     │  each GPU job:
     │    pull ECR image (e.g. d2l-zh-torch)
     │    d2l_job.sh clones repo, restores S3 cache, runs build_<fw>.sh
     │    build_<fw>.sh: d2lbook build eval --tab <fw>; slides; upload cache
     │
     ▼
After all four GPU jobs succeed → build_and_deploy (CPU queue, D2L-Builder image)
     │
     │  d2lbook build pdf  (mxnet default)
     │  d2lbook build pdf --tab pytorch
     │  d2lbook build html
     │  if JOB_NAME == d2l-zh/release:
     │      d2lbook build pkg
     │      d2lbook deploy ... --s3 s3://zh-v2.d2l.ai/
     │  else:
     │      d2lbook deploy html pdf --s3 s3://preview.d2l.ai/<JOB_NAME>/
     │
     ▼
Live site / PR preview comment with preview URL

Three key trade-offs baked in:

• Evaluation is cached in S3 (s3://preview.d2l.ai/<cache_dir>/<repo>-<branch>/_build/). A chapter only re-evaluates if its modification time differs from the cached copy. git timesync in d2l_job.sh restores modification times from git metadata because cloning resets them.
• CI is split CPU/GPU by function. Per-framework eval runs on GPU (torch/tf/mxnet/paddle); final HTML/PDF assembly runs on CPU. Separation keeps GPU minutes minimal.
• The SAFE_TO_USE_SCRIPT parameter gates untrusted PRs (see §7 Security). For PRs from forks without write permission, the CI overwrites the submitted workflow_scripts/ with the master copy to prevent code execution via PR.

1.8 Red flags a new maintainer should know immediately

• MXNet is end-of-life upstream. Apache archived MXNet in late 2023. The d2l/mxnet.py backend and MXNet PDF builds will continue working for historical/reference purposes, but MXNet is not a target for new feature development.
• d2l==0.17.6 is hardcoded in CI build scripts, not d2l==2.0.0. This is deliberate — the CI installs an old published d2l package for legacy compatibility while the source tree in the repo becomes the next published d2l package.
• Code cells tagged #@save are load-bearing. Removing one changes the public API of the d2l package. Never delete a #@save cell without checking downstream usage.
• Framework tabs must stay in sync. Every :begin_tab:<framework> block must have equivalent implementations across all four frameworks, or d2lbook build tabcheck fails in CI.
• The CI pipeline requires AWS Batch with GPU compute environments. A fork that wants the full build pipeline will need to replicate this infrastructure or replace it.

2. System Requirements

2.1 For building locally (contributor)

2.2 For running CI (platform operator)

2.3 Framework version matrix (as of commit e6b18cc)

Determined from ECR Dockerfiles and print_versions.py:

2.4 Reader-facing runtime

The d2l package pins:

jupyter==1.0.0
numpy==1.21.5
matplotlib==3.5.1
requests==2.25.1
pandas==1.2.4

These are the pins for the helper package, not for readers' framework choice. The tight pins exist because the book's evaluated notebook outputs were generated against these versions; newer versions may produce different tensor shapes, warning messages, or plot styling that diverges from the printed/rendered book.

3. Configuration Reference

3.1 config.ini — the master switchboard

d2lbook reads config.ini for every build action. The file is organized into sections, each controlling a pipeline stage.

Important

config.ini is the single most operationally significant file in the repository. Changes here affect all four framework builds, HTML output, PDF rendering, deployment targets, and the d2l package itself. Always test a full build cycle after modifying it.

[project]

[translation]

[build]

exclusions is where you add a chapter-in-progress that should not yet compile.

[html]

header_links defines the navbar (PDF download, Jupyter ZIP, course link, GitHub, English edition). favicon and html_logo point to static/ assets.

[pdf]

[library] and [library-<framework>]

• [library] version_file = d2l/__init__.py — d2lbook reads __version__ from this file.
• [library-<fw>] lib_file = d2l/<fw>.py — the file where #@save cells are collected.
• [library-<fw>] lib_name = <name> — the import alias (np for mxnet, torch for torch, tf for tensorflow, paddle for paddle).
• simple_alias, fluent_alias, alias, reverse_alias — four classes of rewrite rules that transform d2l.foo(...) calls in Markdown code cells into framework-native calls.

[deploy]

• other_file_s3urls — legacy ZIPs to include in release artifacts (the 1.0, 1.1, 2.0.0 notebook bundles).
• google_analytics_tracking_id = UA-96378503-2 — embedded in generated HTML.

[colab] and [sagemaker]

Define derivative notebook repos for each framework:

These are push-only targets. Never edit them directly; they are regenerated every release.

[slides]

3.2 Environment variables affecting CI

3.3 Local configuration overrides

d2lbook does not natively support a config.local.ini override file. To do per-environment config:

• Option A: Checkout a separate branch with a modified config.ini (ephemeral, never push).
• Option B: Use shell-level env vars consumed by build_*.sh (e.g., DISABLE_CACHE=true).

There is no per-user personal-config file; the canonical config.ini is always the one in the repo.

4. Deployment & Scaling

4.1 Branch model

4.2 Deployment flow

build_and_deploy.sh branches on JOB_NAME:

if [[ "$JOB_NAME" == "$REPO_NAME/release" ]]; then
  d2lbook build pkg
  d2lbook deploy html pdf pkg colab sagemaker slides --s3 "s3://${LANG}-v2.d2l.ai/"
else
  d2lbook deploy html pdf --s3 "s3://preview.d2l.ai/${JOB_NAME}/"
fi

Release path (pushed to release branch):

• Builds the d2l wheel (d2lbook build pkg).
• Deploys HTML, PDFs, wheel, Colab notebooks, SageMaker notebooks, and slides to s3://zh-v2.d2l.ai/.
• CloudFront fronts the S3 bucket; updates are live within minutes (subject to CF cache invalidation).

Preview path (any other branch or PR):

• Deploys only HTML and PDFs to s3://preview.d2l.ai/<JOB_NAME>/.
• GitHub Actions posts a comment on the PR with the preview URL.

Tip

Use [skip frameworks] in commit messages for prose-only PRs where re-evaluating notebooks is wasteful. This skips all four GPU evaluation jobs and goes straight to the build-and-deploy assembly using cached outputs.

4.3 Cutting a release

1. Bump d2l/__init__.py __version__ on master (semantic version — major if API breaks, minor for new chapters/#@save functions, patch for prose/figures).
2. Merge all pending content PRs into master.
3. Verify master CI is green across all four frameworks and the builder job.
4. git checkout release && git merge --ff-only master && git push (fast-forward only; never force-push release).
5. Release-branch CI triggers production deploy.
6. (External, manual) Publish the wheel to PyPI with twine upload.
7. Tag the repo: git tag v2.0.0 && git push --tags.

4.4 Derivative notebook repos

Four Colab repos and four SageMaker repos are pushed on every release. These are regenerated from the build output. Do not cherry-pick into them. If an external contributor PRs into one of these repos, redirect them to d2l-zh itself.

4.5 Legacy ZIPs

config.ini [deploy] other_file_s3urls lists three legacy bundles:

s3://d2l-webdata/releases/d2l-zh/d2l-zh-1.0.zip
s3://d2l-webdata/releases/d2l-zh/d2l-zh-1.1.zip
s3://d2l-webdata/releases/d2l-zh/d2l-zh-2.0.0.zip

Do not delete on rotation — they are referenced by external courses, papers, and books.

Note

Deployed URLs: https://zh-v2.d2l.ai/ (live HTML), https://zh-v2.d2l.ai/d2l-zh.pdf (MXNet PDF), https://zh-v2.d2l.ai/d2l-zh-pytorch.pdf (PyTorch PDF), https://zh-v2.d2l.ai/d2l-zh.zip (notebook ZIP), https://preview.d2l.ai/d2l-zh/PR-<N>-<SHA>/ (PR previews).

5. Operations

5.1 Local build

From INFO.md:

# HTML
make html

# PDF (requires XeLaTeX + Chinese fonts)
make pdf

make targets map to d2lbook build commands. A typical local workflow for prose-only edits:

d2lbook build html          # no eval, for fast iteration
d2lbook build eval --tab pytorch   # eval only PyTorch tab
d2lbook build pdf --tab pytorch    # PDF for PyTorch tab

To build everything the way CI does:

# per framework
d2lbook build eval --tab mxnet
d2lbook build eval --tab pytorch
d2lbook build eval --tab tensorflow
d2lbook build eval --tab paddle

# final assembly
d2lbook build html
d2lbook build pdf          # default tab (mxnet)
d2lbook build pdf --tab pytorch
d2lbook build pkg

Output lands in _build/.

Tip

For a single-chapter rebuild: d2lbook build eval --tab pytorch --md chapter_linear-networks/*. d2lbook incrementally evaluates based on modification time — editing one .md and rebuilding only runs that file.

5.2 Sanity checks

Run during CI before per-framework evaluation:

d2lbook build outputcheck tabcheck

• outputcheck: ensures no Jupyter cell output has been committed accidentally (the book's outputs are always regenerated by CI).
• tabcheck: ensures every :begin_tab:<fw> section has a matching implementation for all frameworks declared in [build] tabs.

A tabcheck failure means you added or edited a multi-framework code section without updating all four tabs.

5.3 CI entry points

Five jobs, four parallel + one gated:

1. build_torch → AWS Batch → build_pytorch.sh (evaluate PyTorch tab)
2. build_tf → AWS Batch → build_tf.sh
3. build_mxnet → AWS Batch → build_mxnet.sh
4. build_paddle → AWS Batch → build_paddle.sh
5. build_and_deploy (needs all four above) → AWS Batch → build_and_deploy.sh (HTML + PDFs + conditional deploy)

Note

Commit message skip tags: [skip torch], [skip tf], [skip mxnet], [skip paddle], [skip frameworks] (all four evals), [skip builder] (final assembly).

5.4 ci/submit-job.py — Batch job submission

CLI flags (argparse):

Job type → queue mapping:

5.5 Inside the Batch container — d2l_job.sh

ci/docker/d2l_job.sh is the container entrypoint:

git clone https://github.com/"$ORIGINAL_ORG"/"$ORIGINAL_REPO".git
cp -R "$ORIGINAL_REPO"/.github/workflow_scripts .

cd "$ORIGINAL_REPO"
git remote set-url origin $REMOTE
git fetch origin $SOURCE_REF:working
git checkout working

# Restore mtimes from git so S3 cache keys are stable
git timesync *.md **/*.md

# Untrusted PR guard
if [[ ${SAFE_TO_USE_SCRIPT,,} != true ]]; then
    rm -rf .github/workflow_scripts
    mv ../workflow_scripts .github/
fi

cd $WORK_DIR
/bin/bash -o pipefail -c "eval $COMMAND"

Key points:

• git timesync resets file modification times after a fresh clone so the S3 evaluation cache is valid.
• Workflow script replacement prevents untrusted PRs from injecting code into build scripts.

5.6 Cache operations

Two caches:

1. S3 evaluation cache (s3://preview.d2l.ai/ci_cache_{pr,push}/<repo>-<branch>/_build/): per-framework evaluated .md files. Keyed by file mtime.
2. Docker layer cache (ECR): built by .github/workflows/build-docker.yml on demand.

To invalidate: .github/workflows/clear-cache.yml is a manual workflow dispatch.

6. Performance

6.1 Build time optimization

The primary performance lever is the S3 evaluation cache. Chapters only re-evaluate when their modification time changes. git timesync ensures clones restore original mtimes.

6.2 Reader-side performance

The d2l helper library uses get_dataloader_workers() to set data-loading parallelism:

def get_dataloader_workers():
    return 0 if sys.platform.startswith('win') else 4

On Windows, multi-process data loading is disabled (returns 0 workers) because Windows lacks fast fork. On Linux/macOS, 4 worker processes are used.

7. Security

7.1 Untrusted PR handling — SAFE_TO_USE_SCRIPT

The primary threat model: a contributor submits a PR from a fork that modifies .github/workflow_scripts/build_torch.sh to exfiltrate AWS credentials or deploy malicious content.

Mitigation in d2l_job.sh:

if [[ ${SAFE_TO_USE_SCRIPT,,} != true ]]; then
    rm -rf .github/workflow_scripts
    mv ../workflow_scripts .github/   # master's copy
fi

SAFE_TO_USE_SCRIPT should be set to true only when the PR author has write access to d2l-ai/d2l-zh.

Warning

Never edit d2l_job.sh to bypass the SAFE_TO_USE_SCRIPT guard for convenience. If a PR fails for lack of access, rebase it from a trusted branch inside the org. Removing this guard exposes AWS credentials to arbitrary code execution via PR.

7.2 pull_request_target — know what it means

ci.yml uses pull_request_target (not pull_request). This event fires with the secrets and permissions of the target repository, and checks out the base branch by default. The d2l-zh workflow checks out the PR's SHA into the Batch job ($SOURCE_REF) but executes the build scripts from master (via SAFE_TO_USE_SCRIPT=false). The GitHub Actions runner itself only does orchestration — it never runs PR code.

7.3 Organization pinning

ORIGINAL_ORG=${9:-'d2l-ai'}
# TODO @anirudhdagar: hardcode ORIGINAL_ORG

The d2l-infra repo defines the Batch job definition arguments and does not expose position 9 as a configurable input — so in production ORIGINAL_ORG is effectively pinned to d2l-ai. The TODO flags the desire to remove the default and make it fail loudly if not set.

Caution

The default fallback in ORIGINAL_ORG=${9:-'d2l-ai'} means an attacker who controls the Batch job parameter 9 could redirect the clone. Production is safe because the Batch job definition doesn't expose this parameter, but the code-level guard should be hardened by removing the default.

7.4 Secret handling

The workflow uses OIDC (permissions: id-token: write) to assume an AWS role without long-lived secrets stored in GitHub. No AWS access keys in the repo. The IAM role attached to the Batch job has scoped access to:

• The CI cache S3 bucket
• The ECR repositories for the framework images
• The preview.d2l.ai S3 bucket

Release deploys to zh-v2.d2l.ai run under a potentially broader IAM role. Audit quarterly.

7.5 No telemetry from reader code

The shipped d2l package contains no telemetry. setup.py pins five widely-used packages and declares nothing network-facing beyond requests. Readers running the notebooks locally never phone home.

Google Analytics UA-96378503-2 is injected into the HTML build only (config.ini [deploy]). It tracks web visits to zh-v2.d2l.ai, not reader code execution.

7.6 Content licensing

• Code (d2l/, chapter code cells, build scripts): MIT-0 per setup.py.
• Text and diagrams: CC-BY-SA-4.0 per config.ini [project] copyright.

Forks that modify content must retain attribution and share-alike under CC-BY-SA-4.0.

8. Developer Internals

8.1 The d2l Python package

Package layout

d2l/
├── __init__.py        # Holds __version__ = "2.0.0"
├── mxnet.py           # MXNet backend (import d2l.mxnet as d2l)
├── torch.py           # PyTorch backend
├── tensorflow.py      # TensorFlow backend
└── paddle.py          # PaddlePaddle backend

Each backend file is generated (or manually synchronized) so that all four expose the same Python API surface.

#@save — how book code becomes library code

In any chapter, a code cell annotated with #@save is automatically collected into the library:

#@save
def load_data_fashion_mnist(batch_size, resize=None):
    """Download the Fashion-MNIST dataset and load it into memory."""
    # ...

When d2lbook build pkg runs, it scans all chapters, extracts every #@save-annotated definition, and writes them to d2l/<framework>.py (per tab).

Important

Do not add #@save casually. Each annotation is an implicit public API commitment. Any rename or signature change is a breaking change for reader notebooks, later chapters, and downstream forks. The version in d2l/__init__.py must be bumped for API changes.

Alias system — how d2l.foo(...) becomes framework-native

Four classes of alias rules live in config.ini [library-<fw>]:

simple_alias — direct attribute mapping:

• d2l.ones(...) → np.ones(...) (MXNet) / torch.ones(...) (PyTorch)
• d2l.tensor(...) → np.array(...) (MXNet) / torch.tensor(...) (PyTorch)

fluent_alias — method calls on the first argument:

• d2l.reshape(x, shape) → x.reshape(shape) (PyTorch)
• d2l.numpy(x) → x.detach().numpy() (PyTorch) / x.asnumpy() (MXNet)

alias — arbitrary Python expressions:

• d2l.nn_Module → nn.Block (MXNet) / nn.Module (PyTorch) / tf.keras.Model (TF) / nn.Layer (Paddle)

reverse_alias — regex substitutions on evaluated output.

Version pinning

config.ini [library] version_file = d2l/__init__.py tells d2lbook where the version lives. d2lbook build pkg reads this, tags the wheel, and publishes with that version. Do not edit setup.py's version directly; it reads from d2l.__version__.

8.2 LaTeX post-processing (static/post_latex/main.py)

Chinese typography requires fixes that Sphinx's LaTeX writer does not handle natively. The script mutates the .tex file in place with three transformations:

_unnumber_chaps_and_secs(lines)

NUM_UNNUMBERED_CHAPS = 3   # Preface, Installation, Notation
TOC2_START_CHAP_NO   = 5   # Preliminaries onward

• First three \chapter{...} → \chapter*{...} (unnumbered but in TOC).
• All sections within those chapters are likewise starred.
• From chapter 5 onward, TOC depth drops to 2 (suppresses subsections in PDF TOC).
• \section{小结..., \section{练习... are unnumbered everywhere (summaries and exercises).

_sec_to_chap(lines)

Regex rewrites {Section \ref{\detokenize{...chapter_.../index:...}}} to Chapter \ref. Uses the regex module for recursive balanced braces.

Commented-out functions

_edit_titlepage (blank date on title page) and _delete_discussions_title (remove forum QR codes) are both disabled. Dead code — either uncomment or delete.

8.3 Content conventions

These conventions affect build correctness, not just style:

Framework tabs: every framework-specific code block is wrapped in :begin_tab:<fw> / :end_tab: directives. d2lbook build tabcheck fails if any tab is missing.

#@save rules:

• Always place #@save at the top of the cell, first line after the opening fence.
• Don't rename #@save-marked functions in later chapters — collisions silently overwrite.

Cross-references:

• Section labels: sec_<name>. Referenced with :numref:.
• Chapter labels: chap_<name> on index.md. Referenced with :numref:.
• Figure labels: fig_<name>. Must use :numref: for "图 3.2" style numbering.
• Equation labels: eq_<name>. Must use :eqref:.

No Jupyter outputs in source: d2lbook build outputcheck fails if any .md contains embedded notebook output.

8.4 Translation workflow

When upstream d2l-ai/d2l-en updates a section:

1. Translator pulls the new English version.
2. Diffs against the local _origin.md to see what changed.
3. Applies the diff to the Chinese .md.
4. Overwrites _origin.md with the new English source.
5. Opens a PR.

8.5 Diagrams — OmniGraffle

Diagrams live in graffle/ (source) and img/ (rendered PNG/SVG). OmniGraffle is macOS-only. Fork operators without macOS can replace with draw.io or Mermaid — commit both source and rendered image.

8.6 Class hierarchy (126 classes)

The d2l package across all four backends defines these major class families:

Each class exists independently in each backend file with framework-specific base classes (e.g., nn.Module for PyTorch, nn.Block for MXNet, tf.keras.Model for TF, nn.Layer for Paddle).

9. Integration Points

9.1 External dependencies

9.2 Forking — Running your own edition

What you can reuse without infrastructure:

• The entire chapter_*/ tree — edit prose.
• config.ini — change project/deploy settings.
• static/post_latex/main.py — edit the UNNUMBERED set for your language.
• setup.py — change package name if forking the library.

What requires infrastructure replacement:

• AWS Batch + ECR + S3 (your own accounts and Docker images).
• CI job definitions (create equivalents in your AWS account).
• Domain + CloudFront.

Alternative infrastructures:

• GitHub Actions with GPU runners (replace submit-job with direct workflow).
• Self-hosted Jupyter cluster for evaluation.
• CPU-only build (skip eval_notebook; prose only).

What NOT to change without coordination:

• The d2l package name on PyPI (owned by upstream). Pick a different name for forks.
• Colab/SageMaker repo targets — repoint or blank out.
• Google Analytics ID — replace with yours or remove.

10. Troubleshooting

Common CI failures

Warning

d2lbook build pkg produces its API from evaluated output, not raw source. Running d2lbook build pkg without first running d2lbook build eval --tab <fw> will produce an incomplete or outdated library. Always eval before pkg.

d2lbook build pkg produces wrong API

• Check that the #@save annotation is on the first line of the cell. A blank line before it defeats extraction.
• Check that config.ini [library-<fw>] lib_file points to the right file.
• Re-run d2lbook build pkg after d2lbook build eval --tab <fw>.

When the English source has diverged significantly

Translation-workflow drift: after a major upstream refactor, every chapter may have diverging _origin.md files.

1. git log --oneline upstream/master -- chapter_<x>/<y>.md in the d2l-en checkout.
2. Translate incrementally per commit, not per file, if the drift is multi-year.
3. If a chapter was removed upstream, flag for removal and open a PR to drop both files.

Note

Graffle source without macOS: OmniGraffle is macOS-only. Use a cloud Mac (MacStadium), or replace the diagram with draw.io/Mermaid/TikZ. Commit the new source alongside the rendered image; update STYLE_GUIDE.md if this becomes policy. Do not commit half-converted .graffle files.

CI troubleshooting flowchart

CI failed?
├── One framework job failed → check build_<fw>.sh logs in AWS Batch
│   ├── "CUDA out of memory" → reduce batch_size in offending chapter
│   ├── "eval timed out" → split chapter or raise timeout in ci/submit-job.py
│   ├── "tabcheck failed"  → missing tab block; add it
│   └── "outputcheck failed" → strip committed outputs, re-commit
├── All four framework jobs green, build_and_deploy failed →
│   ├── "xelatex error" → new Chinese character without font coverage
│   │                      or mis-balanced LaTeX env from Markdown typo
│   ├── "Sphinx reference not found" → broken :numref: or :ref: label
│   └── "deploy to S3 failed" → IAM role perms; check OIDC assume-role
└── All jobs green, preview shows wrong content →
    ├── CloudFront cache (wait TTL or manual invalidation)
    └── Browser cache (hard reload)

11. Known Issues & Tech Debt

11.1 MXNet is end-of-life

Apache archived MXNet in late 2023. nvcr.io/mxnet images are no longer updated. The d2l/mxnet.py backend works with pinned versions but will degrade as CUDA drivers advance.

Mitigation options: freeze MXNet builds, remove from [build] tabs, or maintain a forked image indefinitely.

11.2 d2l==0.17.6 hardcoded in build scripts

CI installs an old published d2l for bootstrap. The source tree builds the next d2l via d2lbook build pkg. When the source tree's exports diverge from 0.17.6, compatibility issues arise.

Fix: use d2l==$(python -c "from d2l import __version__; print(__version__)") or lock to whatever was last published.

11.3 ORIGINAL_ORG default in d2l_job.sh

The default fallback should be removed and hardcoded to prevent potential org-swap attacks.

11.4 Dead code in main.py

_delete_discussions_title and _edit_titlepage are commented out. Either uncomment or delete.

11.5 numpy==1.21.5 pin is stale

NumPy 1.21.x is past its maintenance window. Readers get conflicts with newer ML libraries. Update pin to a supported version, retest all notebooks.

11.6 No test suite

d2l/ has no tests. Correctness is verified only by CI producing a successful build. Low-hanging fruit: add tests/test_d2l_api.py that imports each backend and asserts every expected function exists.

11.7 contrib/to-rm-mx-contrib-text marked for removal

Directory name literally says "to-rm". Removing it is a mechanical PR.

11.8 Brittle Dockerfile with pinned 2022 base image

Dockerfile.d2l-zh-torch uses nvcr.io/nvidia/pytorch:22.06-py3 from May 2022. Accumulates CVEs. Set a monthly calendar reminder to rebuild or automate via cron workflow.

12. Quick Reference

Command cheat sheet

# Local: fast iteration on prose (no eval)
d2lbook build html

# Local: single-framework PDF
d2lbook build eval --tab pytorch && d2lbook build pdf --tab pytorch

# Local: package build (all frameworks)
for fw in mxnet pytorch tensorflow paddle; do d2lbook build eval --tab $fw; done
d2lbook build pkg

# Sanity
d2lbook build outputcheck tabcheck

# CI: submit a Batch job manually (AWS CLI configured)
python ci/submit-job.py \
  --job-type ci-gpu-torch \
  --source-ref refs/pull/500/head \
  --safe-to-use-script \
  --command "chmod +x .github/workflow_scripts/build_pytorch.sh && .github/workflow_scripts/build_pytorch.sh d2l-zh master ci_cache_pr" \
  --wait

# CI: force preview rebuild bypassing cache
DISABLE_CACHE=true ./.github/workflow_scripts/build_pytorch.sh d2l-zh master ci_cache_pr

File map

Entry points reference

Useful external projects