tldr-pages — Administrator & Developer Guide

1. Architecture Overview

tldr-pages/tldr is the canonical repository of community-maintained, simplified help pages for command-line tools. It is a content-first project: the vast majority of its 35,000+ files are Markdown help pages organized by platform and language. A thin layer of Node.js and Python tooling handles linting, indexing, testing, and PDF generation.

The project operates as a data source consumed by dozens of independently developed tldr client applications. The repository itself is not an executable application — it is a structured content collection with CI/CD automation for quality enforcement and artifact generation.

Component Architecture

┌─────────────────────────────────────────────────┐
│                  tldr-pages/tldr                 │
├─────────────────────────────────────────────────┤
│  Content Layer (35,000+ Markdown files)         │
│  ├── pages/            English pages (11 platforms)│
│  ├── pages.{lang}/     40+ translation dirs     │
│  └── contributing-guides/  Templates & standards│
├─────────────────────────────────────────────────┤
│  Tooling Layer                                  │
│  ├── Node.js: markdownlint-cli, tldr-lint, husky│
│  ├── Python: render.py (PDF), utility scripts   │
│  └── Bash: build.sh, deploy.sh, test.sh        │
├─────────────────────────────────────────────────┤
│  CI/CD Layer (GitHub Actions)                   │
│  ├── Lint + test on every PR                    │
│  ├── Build ZIP archives on merge                │
│  └── Build PDFs per language on merge           │
└─────────────────────────────────────────────────┘
         │                          │
         ▼                          ▼
   ZIP archives              PDF books
   (client consumption)      (offline reading)

Component Inventory

Technology Stack

Note

The project has intentionally minimal code. The value is in the content (pages), not the tooling. The tooling exists solely to enforce quality and generate distribution artifacts.

Data Flow

Page contribution flow:

Contributor writes page.md
  → git commit (husky pre-commit hook triggers)
  → markdownlint + tldr-lint validate format
  → git push → GitHub PR
  → CI: test.sh + check-pr.sh
  → Maintainer review → merge
  → GitHub Actions: build.sh (ZIP) + build-pdf.sh (PDF)
  → deploy.sh pushes artifacts to static site
  → tldr clients fetch updated index + pages

PDF generation flow:

pages/{platform}/*.md
  → render.py reads all .md files
  → markdown lib converts to HTML
  → CSS stylesheets applied (basic / solarized-light / solarized-dark)
  → WeasyPrint renders to PDF
  → Output: tldr-book.pdf (or per-language/platform variants)

2. System Requirements

Important

Shell scripts (build.sh, build-pdf.sh, deploy.sh, test.sh, check-pr.sh) are Linux/macOS only. On Windows, use WSL. Python scripts (render.py, set-alias-page.py, set-more-info-link.py, set-see-also.py, set-page-title.py, wrong-filename.py, update-command.py) work cross-platform.

Platform Compatibility Matrix

3. Configuration Reference

Markdownlint Configuration

File: .markdownlint.json

{
  "default": true,
  "MD003": { "style": "atx" },
  "MD007": { "indent": 4 },
  "MD013": { "line_length": 250 },
  "MD026": false,
  "MD029": false,
  "MD033": false,
  "MD034": false,
  "no-hard-tabs": false,
  "whitespace": false
}

Important

The high line_length of 250 and the disabled whitespace/HTML rules are intentional — tldr pages contain command examples with long lines and use specific formatting patterns that standard markdownlint rules would reject.

Package.json Scripts

File: package.json

Note

lint-tldr-pages only lints the English pages/ directory. Translated pages (pages.*/) are covered by lint-markdown (markdownlint) but not by tldr-lint.

Python Dependencies

File: requirements.txt (root — for development/CI)

File: scripts/pdf/requirements.txt (for PDF generation)

4. Deployment & Scaling

Build Pipeline

The deployment pipeline is driven by GitHub Actions. On merge to main:

1. test.sh — validates all pages pass linting
2. build.sh — generates ZIP archives of the pages/ directories
3. build-pdf.sh — generates per-language PDF books
4. deploy.sh — pushes ZIP and PDF artifacts to the static website repository

Build Scripts Detail

scripts/build.sh — Builds ZIP archives:

• Packages each pages* directory into a distributable archive
• Used by CI for artifact publishing

scripts/build-index.js — Generates index.json:

• Scans all pages/ directories
• Outputs a JSON index of all available pages, platforms, and languages
• Consumed by tldr clients for page discovery

scripts/deploy.sh — Deploys artifacts:

• Copies ZIP and PDF archives to the static website repository
• Triggered after successful build

scripts/build-pdf.sh — Orchestrates PDF generation:

# Process all languages
bash scripts/pdf/build-pdf.sh all

# Process only changed languages (CI mode — auto-detects from git diff)
bash scripts/pdf/build-pdf.sh

# Process a specific language
bash scripts/pdf/build-pdf.sh es

Warning

CJK languages (Bengali, Japanese, Korean, Malayalam, Tamil, Thai, Chinese, Chinese Traditional) are skipped by build-pdf.sh due to font rendering limitations in the PDF pipeline.

PDF Rendering Architecture

The PDF pipeline (scripts/pdf/render.py) uses a multi-step process:

1. Read all .md files from the target directory
2. Sort by platform subdirectory, then alphabetically
3. Convert each page from Markdown to HTML using the markdown Python library
4. Transform # titles to <h2> (grouped under <h1> platform headers)
5. Convert > description lines to <h4> elements
6. Apply CSS stylesheets (basic.css, optionally solarized-light.css or solarized-dark.css)
7. Render the combined HTML to PDF via WeasyPrint

CSS themes available in scripts/pdf/:

Tip

When building PDFs for a specific platform subset, use the -p flag: python3 render.py ../../pages -p common linux. This produces a smaller, focused PDF.

5. Operations

Running Lint Checks

Full lint suite (all languages, markdownlint):

npm run lint-markdown

English pages only (tldr-lint + markdownlint):

npm run lint-tldr-pages

Run the complete test suite:

npm test

This executes scripts/test.sh, which runs basic syntax and format tests on all pages.

Note

The CI pipeline runs both test.sh and check-pr.sh on every pull request. check-pr.sh performs additional PR-specific checks (e.g., verifying changed files match expected patterns).

Page Maintenance Scripts

Tip

When making bulk changes across all translations (e.g., updating a "More information" link), use the dedicated scripts rather than manual edits. They handle the cross-language consistency automatically.

Monitoring Repository Health

Key metrics to track:

• Lint pass rate: All pages must pass both markdownlint and tldr-lint
• Translation coverage: Compare page counts in pages.{lang}/ against pages/
• Title/filename consistency: Run wrong-filename.py periodically
• Stale "More information" links: Links rot over time; consider periodic URL validation

6. Performance

Repository Scale

Note

Due to the massive file count, operations like npm run lint-markdown (which globs pages*/**/*.md) can take several minutes. The glob 13.0.6 dependency is used for efficient file matching.

PDF Build Times

PDF generation time scales linearly with the number of pages. English (pages/) is the largest and slowest. Individual language PDFs are significantly faster.

Git Operations

With 35,000+ files, initial git clone is large. Shallow clones reduce time significantly:

git clone --depth 1 https://github.com/tldr-pages/tldr.git

Tip

Contributors should use full clones (not shallow) to ensure check-pr.sh can accurately diff against the base branch.

7. Security

Supply Chain

The repository's npm dependencies are minimal and well-maintained:

Warning

The package-lock.json pins exact versions. Always review dependency updates carefully, especially for markdownlint-cli which processes all page content. Run npm audit periodically.

Content Security

Since the repository is community-contributed Markdown, the primary security concern is malicious content injection:

• No executable code in pages — pages contain only descriptive text and example commands
• CI linting catches format deviations that could indicate injection attempts
• Maintainer review is required for all PRs
• HTML is allowed (MD033 disabled) — maintainers should review any inline HTML for XSS vectors if pages are rendered in web contexts

Bot Integration

send-to-bot.py uses the requests library to send build output to an external bot service. The target URL and authentication are configured within the script (not exposed as environment variables in the repository).

8. Developer Internals

Repository Structure

tldr/
├── .github/                 GitHub Actions workflows
├── .markdownlint.json       Markdownlint configuration
├── package.json             npm dependencies and scripts
├── package-lock.json        Locked dependency versions
├── requirements.txt         Python dev dependencies
├── contributing-guides/     Contribution standards
│   └── translation-templates/  Per-language phrasing guides
├── images/                  Repository images/assets
├── pages/                   English pages
│   ├── common/              Cross-platform commands
│   ├── linux/               Linux commands
│   ├── osx/                 macOS commands
│   ├── windows/             Windows commands
│   ├── android/             Android (Termux) commands
│   ├── sunos/               SunOS/Solaris commands
│   ├── freebsd/             FreeBSD commands
│   ├── openbsd/             OpenBSD commands
│   ├── netbsd/              NetBSD commands
│   ├── dos/                 DOS commands
│   └── cisco-ios/           Cisco IOS commands
├── pages.{lang}/            Translated pages (40+ languages)
│   └── {platform}/          Same platform structure as pages/
└── scripts/
    ├── build.sh             Build ZIP archives
    ├── build-index.js       Generate index.json
    ├── check-pr.sh          PR validation checks
    ├── deploy.sh            Deploy artifacts to website
    ├── test.sh              Test suite runner
    ├── check-errors.sh      Grep-based error finder
    ├── send-to-bot.py       Bot notification script
    ├── set-alias-page.py    Alias page generator
    ├── set-more-info-link.py  URL updater
    ├── set-see-also.py      Cross-reference updater
    ├── set-page-title.py    Title updater
    ├── wrong-filename.py    Title/filename mismatch detector
    ├── update-command.py    Cross-language example updater
    └── pdf/
        ├── render.py        PDF generator (Python)
        ├── build-pdf.sh     PDF build orchestrator
        ├── requirements.txt WeasyPrint + markdown deps
        ├── basic.css        Default PDF stylesheet
        ├── solarized-light.css  Light theme
        └── solarized-dark.css   Dark theme

Module Reference

scripts/pdf/render.py

The primary code module in the repository. Converts Markdown pages to a single PDF document.

Function: main(loc, colorscheme, output_filename, platform)

Behavior:

1. Validates loc is a valid directory (exits with code 1 if not)
2. Builds a CSS list: always includes basic.css, appends theme CSS if not basic
3. Iterates platform subdirectories alphabetically
4. For each platform, iterates .md files alphabetically
5. Transforms the Markdown: converts # title to <h2>, > lines to ####
6. Converts each line to HTML via markdown.markdown()
7. Inserts page breaks between entries
8. Renders the combined HTML string to PDF via WeasyPrint.HTML(string=).write_pdf()
9. Appends platform names to filename when -p filter is used

CLI entry point: argparse-based, invoked as python3 render.py <args>.

scripts/pdf/build-pdf.sh

Orchestrates PDF builds for CI. Determines which language directories changed (via git diff-tree) and calls render.py for each.

Behavior:

• No argument: auto-detect changed pages* directories from the latest commit
• Argument all: build PDFs for all pages* directories
• Argument en: build for pages/ (English, uses solarized-light theme)
• Argument {lang}: build for pages.{lang}/ (uses basic theme)
• CJK languages (bn, ja, ko, ml, ta, th, zh, zh_TW) are skipped with a message

Code Patterns and Conventions

Testing Strategy

The repository uses a shell-based testing approach rather than a formal test framework:

• scripts/test.sh — Primary test runner invoked by npm test. Runs syntax and format validation across all pages.
• scripts/check-pr.sh — PR-specific checks: validates changed files conform to naming conventions, proper directory placement, and format rules.
• Pre-commit hooks — husky runs linting before every commit (configured via npm run prepare).
• Python formatting — black (26.3.1) and flake8 (7.3.0) for Python script quality.

Note

There is no unit test framework (no Jest, pytest, etc.) for the scripts themselves. Testing is focused on content validation rather than code correctness.

Potential Improvements

• CJK PDF support — build-pdf.sh currently skips 8 languages. Adding CJK font support to the PDF pipeline would cover a significant portion of translations.
• tldr-lint coverage for translations — Currently, npm run lint-tldr-pages only validates English pages. Extending it to pages.*/ would improve quality across all languages.
• URL validation — The "More information" links in pages are not automatically validated. A periodic link-checker script would catch dead links.
• Script test coverage — The Python utility scripts lack automated tests. Adding pytest for render.py, set-alias-page.py, etc. would improve reliability.
• Incremental PDF builds — Currently, changing one page triggers a full PDF rebuild for that language. Incremental builds could speed up CI significantly.

Contributing Guide

Setting up the development environment:

1. Clone the repository:

   git clone https://github.com/tldr-pages/tldr.git
   cd tldr
   

2. Install Node.js dependencies (linting tools + husky):

   npm install
   

3. Install Python dependencies (for scripts):

   pip install -r requirements.txt
   pip install -r scripts/pdf/requirements.txt
   

4. Verify the setup:

   npm test
   npm run lint-tldr-pages
   

Development workflow:

1. Create a feature branch
2. Make changes (new pages, translations, script improvements)
3. Run linting: npm run lint-markdown and npm run lint-tldr-pages
4. Commit (husky hook auto-validates)
5. Push and open a PR
6. CI runs test.sh + check-pr.sh
7. Maintainer review and merge

Caution

Do not commit changes to package-lock.json unless you are intentionally updating dependencies. Unintended lockfile changes cause merge conflicts and can introduce supply chain risks.

9. Integration Points

Client Ecosystem

The repository serves as the upstream data source for all tldr client implementations. Clients consume pages via:

• GitHub raw file access — Direct Markdown file fetches
• ZIP archives — Built by scripts/build.sh, hosted on the static site
• index.json — Generated by scripts/build-index.js, used for page discovery and search

GitHub Actions CI/CD

The repository uses GitHub Actions for automation:

• On PR: test.sh + check-pr.sh run to validate contributions
• On merge: build.sh generates ZIPs, build-pdf.sh generates PDFs, deploy.sh pushes to the website

Bot Integration

scripts/send-to-bot.py sends build and test output to a tldr-bot service for community notifications (e.g., Gitter, Matrix, or other chat platforms).