# tldr-pages — Administrator & Developer Guide

## 1. Architecture Overview

**tldr-pages/tldr** is the canonical repository of community-maintained, simplified help pages for command-line tools. It is a content-first project: the vast majority of its 35,000+ files are Markdown help pages organized by platform and language. A thin layer of Node.js and Python tooling handles linting, indexing, testing, and PDF generation.

The project operates as a data source consumed by dozens of independently developed tldr client applications. The repository itself is not an executable application — it is a structured content collection with CI/CD automation for quality enforcement and artifact generation.

### Component Architecture

```text
┌─────────────────────────────────────────────────┐
│                  tldr-pages/tldr                 │
├─────────────────────────────────────────────────┤
│  Content Layer (35,000+ Markdown files)         │
│  ├── pages/            English pages (11 platforms)│
│  ├── pages.{lang}/     40+ translation dirs     │
│  └── contributing-guides/  Templates & standards│
├─────────────────────────────────────────────────┤
│  Tooling Layer                                  │
│  ├── Node.js: markdownlint-cli, tldr-lint, husky│
│  ├── Python: render.py (PDF), utility scripts   │
│  └── Bash: build.sh, deploy.sh, test.sh        │
├─────────────────────────────────────────────────┤
│  CI/CD Layer (GitHub Actions)                   │
│  ├── Lint + test on every PR                    │
│  ├── Build ZIP archives on merge                │
│  └── Build PDFs per language on merge           │
└─────────────────────────────────────────────────┘
         │                          │
         ▼                          ▼
   ZIP archives              PDF books
   (client consumption)      (offline reading)
```

### Component Inventory

| Component | Path | Role |
|---|---|---|
| English pages | `pages/{platform}/` | Canonical command help pages (11 platform dirs) |
| Translated pages | `pages.{lang}/{platform}/` | 40+ language translations |
| Contributing guides | `contributing-guides/` | Page format standards and translation templates |
| Markdown linter | `node_modules/markdownlint-cli/` | Enforces GFM formatting rules |
| tldr linter | `node_modules/tldr-lint/` | Enforces tldr-specific page format rules |
| Pre-commit hooks | `node_modules/husky/` | Runs linting before commits |
| Build scripts | `scripts/` | Index, archive, PDF, and utility scripts |
| PDF renderer | `scripts/pdf/render.py` | Generates PDF books from page collections |
| Lint config | `.markdownlint.json` | Markdownlint rule overrides |
| Package config | `package.json` | npm scripts, dependencies |

### Technology Stack

| Technology | Version / Detail | Purpose |
|---|---|---|
| Node.js | 20+ | Linting tools, build-index, husky hooks |
| Python | 3.x | PDF generation, page utility scripts |
| markdownlint-cli | ^0.48.0 | Markdown formatting enforcement |
| tldr-lint | ^0.0.22 | tldr-specific page validation |
| glob | 13.0.6 | File pattern matching for build scripts |
| husky | ^9.1.7 | Git hook management (dev dependency) |
| WeasyPrint | (via pip) | HTML-to-PDF rendering |
| markdown (Python) | (via pip) | Markdown-to-HTML conversion for PDF pipeline |
| black | 26.3.1 | Python code formatting |
| flake8 | 7.3.0 | Python linting |
| requests | 2.33.1 | HTTP requests (for bot integration) |

> [!NOTE]
> The project has intentionally minimal code. The value is in the content (pages), not the tooling. The tooling exists solely to enforce quality and generate distribution artifacts.

### Data Flow

**Page contribution flow:**

```text
Contributor writes page.md
  → git commit (husky pre-commit hook triggers)
  → markdownlint + tldr-lint validate format
  → git push → GitHub PR
  → CI: test.sh + check-pr.sh
  → Maintainer review → merge
  → GitHub Actions: build.sh (ZIP) + build-pdf.sh (PDF)
  → deploy.sh pushes artifacts to static site
  → tldr clients fetch updated index + pages
```

**PDF generation flow:**

```text
pages/{platform}/*.md
  → render.py reads all .md files
  → markdown lib converts to HTML
  → CSS stylesheets applied (basic / solarized-light / solarized-dark)
  → WeasyPrint renders to PDF
  → Output: tldr-book.pdf (or per-language/platform variants)
```

## 2. System Requirements

| Requirement | Minimum | Purpose |
|---|---|---|
| Node.js | 20+ | Linting, indexing, husky hooks |
| npm | (bundled with Node.js) | Dependency management |
| Python | 3.x | PDF generation, utility scripts |
| Git | Any recent | Repository operations |
| Bash | 4+ | Shell scripts (`build.sh`, `test.sh`, `deploy.sh`) |
| WeasyPrint system deps | GTK3, Pango, Cairo | PDF rendering (only for PDF generation) |

> [!IMPORTANT]
> Shell scripts (`build.sh`, `build-pdf.sh`, `deploy.sh`, `test.sh`, `check-pr.sh`) are Linux/macOS only. On Windows, use WSL. Python scripts (`render.py`, `set-alias-page.py`, `set-more-info-link.py`, `set-see-also.py`, `set-page-title.py`, `wrong-filename.py`, `update-command.py`) work cross-platform.

### Platform Compatibility Matrix

| Script | Linux | macOS | Windows |
|---|---|---|---|
| `render.py` | Yes | Yes | Yes |
| `build-pdf.sh` | Yes | Yes | WSL only |
| `build.sh` | Yes | Yes | WSL only |
| `set-alias-page.py` | Yes | Yes | Yes |
| `set-more-info-link.py` | Yes | Yes | Yes |
| `set-see-also.py` | Yes | Yes | Yes |
| `set-page-title.py` | Yes | Yes | Yes |
| `wrong-filename.py` | Yes | Yes | Yes |
| `update-command.py` | Yes | Yes | Yes |
| `check-errors.sh` | Yes | Untested | Untested |

## 3. Configuration Reference

### Markdownlint Configuration

**File:** `.markdownlint.json`

```json
{
  "default": true,
  "MD003": { "style": "atx" },
  "MD007": { "indent": 4 },
  "MD013": { "line_length": 250 },
  "MD026": false,
  "MD029": false,
  "MD033": false,
  "MD034": false,
  "no-hard-tabs": false,
  "whitespace": false
}
```

| Rule | Setting | Effect |
|---|---|---|
| `default` | `true` | All markdownlint rules enabled by default |
| `MD003` | `style: "atx"` | Heading style must be ATX (`#` prefix), not setext |
| `MD007` | `indent: 4` | Unordered list items must use 4-space indentation |
| `MD013` | `line_length: 250` | Maximum line length set to 250 characters |
| `MD026` | `false` | Disabled — trailing punctuation in headings allowed |
| `MD029` | `false` | Disabled — ordered list item prefix style not enforced |
| `MD033` | `false` | Disabled — inline HTML allowed |
| `MD034` | `false` | Disabled — bare URLs allowed |
| `no-hard-tabs` | `false` | Disabled — hard tabs permitted |
| `whitespace` | `false` | Disabled — whitespace rules relaxed |

> [!IMPORTANT]
> The high `line_length` of 250 and the disabled whitespace/HTML rules are intentional — tldr pages contain command examples with long lines and use specific formatting patterns that standard markdownlint rules would reject.

### Package.json Scripts

**File:** `package.json`

| Script | Command | Purpose |
|---|---|---|
| `lint-markdown` | `markdownlint pages*/**/*.md` | Lint all pages (all languages) with markdownlint |
| `lint-tldr-pages` | `tldr-lint ./pages` | Lint English pages with tldr-specific rules |
| `test` | `bash scripts/test.sh` | Run the full test suite |
| `build-index` | `node ./scripts/build-index.js > index.json` | Generate the page index as JSON |
| `prepare` | `husky` | Set up husky pre-commit hooks on `npm install` |

> [!NOTE]
> `lint-tldr-pages` only lints the English `pages/` directory. Translated pages (`pages.*/`) are covered by `lint-markdown` (markdownlint) but not by `tldr-lint`.

### Python Dependencies

**File:** `requirements.txt` (root — for development/CI)

| Package | Version | Purpose |
|---|---|---|
| `black` | 26.3.1 | Python code formatter |
| `flake8` | 7.3.0 | Python linter |
| `requests` | 2.33.1 | HTTP requests (for `send-to-bot.py`) |

**File:** `scripts/pdf/requirements.txt` (for PDF generation)

| Package | Version | Purpose |
|---|---|---|
| `markdown` | Latest | Markdown-to-HTML conversion |
| `weasyprint` | Latest | HTML-to-PDF rendering |

## 4. Deployment & Scaling

### Build Pipeline

The deployment pipeline is driven by GitHub Actions. On merge to `main`:

1. **`test.sh`** — validates all pages pass linting
2. **`build.sh`** — generates ZIP archives of the `pages/` directories
3. **`build-pdf.sh`** — generates per-language PDF books
4. **`deploy.sh`** — pushes ZIP and PDF artifacts to the static website repository

### Build Scripts Detail

**`scripts/build.sh`** — Builds ZIP archives:
- Packages each `pages*` directory into a distributable archive
- Used by CI for artifact publishing

**`scripts/build-index.js`** — Generates `index.json`:
- Scans all `pages/` directories
- Outputs a JSON index of all available pages, platforms, and languages
- Consumed by tldr clients for page discovery

**`scripts/deploy.sh`** — Deploys artifacts:
- Copies ZIP and PDF archives to the static website repository
- Triggered after successful build

**`scripts/build-pdf.sh`** — Orchestrates PDF generation:

```bash
# Process all languages
bash scripts/pdf/build-pdf.sh all

# Process only changed languages (CI mode — auto-detects from git diff)
bash scripts/pdf/build-pdf.sh

# Process a specific language
bash scripts/pdf/build-pdf.sh es
```

> [!WARNING]
> CJK languages (Bengali, Japanese, Korean, Malayalam, Tamil, Thai, Chinese, Chinese Traditional) are skipped by `build-pdf.sh` due to font rendering limitations in the PDF pipeline.

### PDF Rendering Architecture

The PDF pipeline (`scripts/pdf/render.py`) uses a multi-step process:

1. Read all `.md` files from the target directory
2. Sort by platform subdirectory, then alphabetically
3. Convert each page from Markdown to HTML using the `markdown` Python library
4. Transform `#` titles to `<h2>` (grouped under `<h1>` platform headers)
5. Convert `>` description lines to `<h4>` elements
6. Apply CSS stylesheets (`basic.css`, optionally `solarized-light.css` or `solarized-dark.css`)
7. Render the combined HTML to PDF via WeasyPrint

**CSS themes available in `scripts/pdf/`:**

| Theme | File | Description |
|---|---|---|
| Basic | `basic.css` | Default monochrome style (always loaded) |
| Solarized Light | `solarized-light.css` | Light variant of Solarized color scheme |
| Solarized Dark | `solarized-dark.css` | Dark variant of Solarized color scheme |

> [!TIP]
> When building PDFs for a specific platform subset, use the `-p` flag: `python3 render.py ../../pages -p common linux`. This produces a smaller, focused PDF.

## 5. Operations

### Running Lint Checks

**Full lint suite (all languages, markdownlint):**

```bash
npm run lint-markdown
```

**English pages only (tldr-lint + markdownlint):**

```bash
npm run lint-tldr-pages
```

**Run the complete test suite:**

```bash
npm test
```

This executes `scripts/test.sh`, which runs basic syntax and format tests on all pages.

> [!NOTE]
> The CI pipeline runs both `test.sh` and `check-pr.sh` on every pull request. `check-pr.sh` performs additional PR-specific checks (e.g., verifying changed files match expected patterns).

### Page Maintenance Scripts

| Script | Purpose | Usage |
|---|---|---|
| `set-alias-page.py` | Generate/update alias redirect pages | `python3 scripts/set-alias-page.py` |
| `set-more-info-link.py` | Update "More information" URLs | `python3 scripts/set-more-info-link.py` |
| `set-see-also.py` | Update "See also" cross-references | `python3 scripts/set-see-also.py` |
| `set-page-title.py` | Update page titles across languages | `python3 scripts/set-page-title.py` |
| `wrong-filename.py` | Detect title/filename mismatches | `python3 scripts/wrong-filename.py` |
| `update-command.py` | Update command examples across all languages | `python3 scripts/update-command.py` |
| `check-errors.sh` | Grep for common formatting errors | `cd pages.{lang} && ../scripts/check-errors.sh` |
| `send-to-bot.py` | Send build/test output to tldr-bot | `python3 scripts/send-to-bot.py` |

> [!TIP]
> When making bulk changes across all translations (e.g., updating a "More information" link), use the dedicated scripts rather than manual edits. They handle the cross-language consistency automatically.

### Monitoring Repository Health

Key metrics to track:

- **Lint pass rate:** All pages must pass both markdownlint and tldr-lint
- **Translation coverage:** Compare page counts in `pages.{lang}/` against `pages/`
- **Title/filename consistency:** Run `wrong-filename.py` periodically
- **Stale "More information" links:** Links rot over time; consider periodic URL validation

## 6. Performance

### Repository Scale

| Metric | Value |
|---|---|
| Total files | 35,171 |
| Language translations | 40+ |
| Platform categories | 11 |
| English pages (estimated) | ~3,000+ (across all platforms) |

> [!NOTE]
> Due to the massive file count, operations like `npm run lint-markdown` (which globs `pages*/**/*.md`) can take several minutes. The `glob` 13.0.6 dependency is used for efficient file matching.

### PDF Build Times

PDF generation time scales linearly with the number of pages. English (`pages/`) is the largest and slowest. Individual language PDFs are significantly faster.

### Git Operations

With 35,000+ files, initial `git clone` is large. Shallow clones reduce time significantly:

```bash
git clone --depth 1 https://github.com/tldr-pages/tldr.git
```

> [!TIP]
> Contributors should use full clones (not shallow) to ensure `check-pr.sh` can accurately diff against the base branch.

## 7. Security

### Supply Chain

The repository's npm dependencies are minimal and well-maintained:

| Dependency | Risk Level | Notes |
|---|---|---|
| `markdownlint-cli` | Low | Widely used Markdown linter |
| `tldr-lint` | Low | tldr-specific linter, maintained by the tldr-pages org |
| `glob` | Low | Core Node.js ecosystem library |
| `husky` | Low (dev only) | Git hook manager, not shipped |

> [!WARNING]
> The `package-lock.json` pins exact versions. Always review dependency updates carefully, especially for `markdownlint-cli` which processes all page content. Run `npm audit` periodically.

### Content Security

Since the repository is community-contributed Markdown, the primary security concern is malicious content injection:

- **No executable code** in pages — pages contain only descriptive text and example commands
- **CI linting** catches format deviations that could indicate injection attempts
- **Maintainer review** is required for all PRs
- **HTML is allowed** (MD033 disabled) — maintainers should review any inline HTML for XSS vectors if pages are rendered in web contexts

### Bot Integration

`send-to-bot.py` uses the `requests` library to send build output to an external bot service. The target URL and authentication are configured within the script (not exposed as environment variables in the repository).

## 8. Developer Internals

### Repository Structure

```text
tldr/
├── .github/                 GitHub Actions workflows
├── .markdownlint.json       Markdownlint configuration
├── package.json             npm dependencies and scripts
├── package-lock.json        Locked dependency versions
├── requirements.txt         Python dev dependencies
├── contributing-guides/     Contribution standards
│   └── translation-templates/  Per-language phrasing guides
├── images/                  Repository images/assets
├── pages/                   English pages
│   ├── common/              Cross-platform commands
│   ├── linux/               Linux commands
│   ├── osx/                 macOS commands
│   ├── windows/             Windows commands
│   ├── android/             Android (Termux) commands
│   ├── sunos/               SunOS/Solaris commands
│   ├── freebsd/             FreeBSD commands
│   ├── openbsd/             OpenBSD commands
│   ├── netbsd/              NetBSD commands
│   ├── dos/                 DOS commands
│   └── cisco-ios/           Cisco IOS commands
├── pages.{lang}/            Translated pages (40+ languages)
│   └── {platform}/          Same platform structure as pages/
└── scripts/
    ├── build.sh             Build ZIP archives
    ├── build-index.js       Generate index.json
    ├── check-pr.sh          PR validation checks
    ├── deploy.sh            Deploy artifacts to website
    ├── test.sh              Test suite runner
    ├── check-errors.sh      Grep-based error finder
    ├── send-to-bot.py       Bot notification script
    ├── set-alias-page.py    Alias page generator
    ├── set-more-info-link.py  URL updater
    ├── set-see-also.py      Cross-reference updater
    ├── set-page-title.py    Title updater
    ├── wrong-filename.py    Title/filename mismatch detector
    ├── update-command.py    Cross-language example updater
    └── pdf/
        ├── render.py        PDF generator (Python)
        ├── build-pdf.sh     PDF build orchestrator
        ├── requirements.txt WeasyPrint + markdown deps
        ├── basic.css        Default PDF stylesheet
        ├── solarized-light.css  Light theme
        └── solarized-dark.css   Dark theme
```

### Module Reference

#### `scripts/pdf/render.py`

The primary code module in the repository. Converts Markdown pages to a single PDF document.

**Function:** `main(loc, colorscheme, output_filename, platform)`

| Parameter | Type | Description |
|---|---|---|
| `loc` | `str` | Path to the pages directory |
| `colorscheme` | `str` | CSS theme name: `basic`, `solarized-light`, or `solarized-dark` |
| `output_filename` | `str` | Output PDF filename |
| `platform` | `list[str]` or `None` | Platform filter (e.g., `["common", "linux"]`) |

**Behavior:**

1. Validates `loc` is a valid directory (exits with code 1 if not)
2. Builds a CSS list: always includes `basic.css`, appends theme CSS if not `basic`
3. Iterates platform subdirectories alphabetically
4. For each platform, iterates `.md` files alphabetically
5. Transforms the Markdown: converts `#` title to `<h2>`, `>` lines to `####`
6. Converts each line to HTML via `markdown.markdown()`
7. Inserts page breaks between entries
8. Renders the combined HTML string to PDF via `WeasyPrint.HTML(string=).write_pdf()`
9. Appends platform names to filename when `-p` filter is used

**CLI entry point:** `argparse`-based, invoked as `python3 render.py <args>`.

#### `scripts/pdf/build-pdf.sh`

Orchestrates PDF builds for CI. Determines which language directories changed (via `git diff-tree`) and calls `render.py` for each.

**Behavior:**

- No argument: auto-detect changed `pages*` directories from the latest commit
- Argument `all`: build PDFs for all `pages*` directories
- Argument `en`: build for `pages/` (English, uses `solarized-light` theme)
- Argument `{lang}`: build for `pages.{lang}/` (uses `basic` theme)
- CJK languages (`bn`, `ja`, `ko`, `ml`, `ta`, `th`, `zh`, `zh_TW`) are skipped with a message

### Code Patterns and Conventions

| Pattern | Convention |
|---|---|
| Page filenames | Lowercase, hyphen-separated: `command-name.md` |
| Platform directories | `common`, `linux`, `osx`, `windows`, `android`, `sunos`, `freebsd`, `openbsd`, `netbsd`, `dos`, `cisco-ios` |
| Language codes | ISO 639-1 (e.g., `es`, `fr`, `zh`) or locale (e.g., `pt_BR`, `zh_TW`) |
| Translation directories | `pages.{lang_code}/` — dot-separated from `pages` |
| CSS theming | Additive: `basic.css` always loaded, theme CSS layered on top |
| Page structure | `# title` → `> description` → `> More information` → `- example` → `` `command` `` |
| Script languages | Bash for CI/build; Python for content manipulation; Node.js for linting |
| Linting approach | Two-tier: `markdownlint` for GFM rules, `tldr-lint` for domain-specific rules |

### Testing Strategy

The repository uses a shell-based testing approach rather than a formal test framework:

- **`scripts/test.sh`** — Primary test runner invoked by `npm test`. Runs syntax and format validation across all pages.
- **`scripts/check-pr.sh`** — PR-specific checks: validates changed files conform to naming conventions, proper directory placement, and format rules.
- **Pre-commit hooks** — `husky` runs linting before every commit (configured via `npm run prepare`).
- **Python formatting** — `black` (26.3.1) and `flake8` (7.3.0) for Python script quality.

> [!NOTE]
> There is no unit test framework (no Jest, pytest, etc.) for the scripts themselves. Testing is focused on content validation rather than code correctness.

### Potential Improvements

- **CJK PDF support** — `build-pdf.sh` currently skips 8 languages. Adding CJK font support to the PDF pipeline would cover a significant portion of translations.
- **`tldr-lint` coverage for translations** — Currently, `npm run lint-tldr-pages` only validates English pages. Extending it to `pages.*/` would improve quality across all languages.
- **URL validation** — The "More information" links in pages are not automatically validated. A periodic link-checker script would catch dead links.
- **Script test coverage** — The Python utility scripts lack automated tests. Adding pytest for `render.py`, `set-alias-page.py`, etc. would improve reliability.
- **Incremental PDF builds** — Currently, changing one page triggers a full PDF rebuild for that language. Incremental builds could speed up CI significantly.

### Contributing Guide

**Setting up the development environment:**

1. Clone the repository:
   ```bash
   git clone https://github.com/tldr-pages/tldr.git
   cd tldr
   ```

2. Install Node.js dependencies (linting tools + husky):
   ```bash
   npm install
   ```

3. Install Python dependencies (for scripts):
   ```bash
   pip install -r requirements.txt
   pip install -r scripts/pdf/requirements.txt
   ```

4. Verify the setup:
   ```bash
   npm test
   npm run lint-tldr-pages
   ```

**Development workflow:**

1. Create a feature branch
2. Make changes (new pages, translations, script improvements)
3. Run linting: `npm run lint-markdown` and `npm run lint-tldr-pages`
4. Commit (husky hook auto-validates)
5. Push and open a PR
6. CI runs `test.sh` + `check-pr.sh`
7. Maintainer review and merge

> [!CAUTION]
> Do not commit changes to `package-lock.json` unless you are intentionally updating dependencies. Unintended lockfile changes cause merge conflicts and can introduce supply chain risks.

## 9. Integration Points

### Client Ecosystem

The repository serves as the upstream data source for all tldr client implementations. Clients consume pages via:

- **GitHub raw file access** — Direct Markdown file fetches
- **ZIP archives** — Built by `scripts/build.sh`, hosted on the static site
- **`index.json`** — Generated by `scripts/build-index.js`, used for page discovery and search

### GitHub Actions CI/CD

The repository uses GitHub Actions for automation:

- **On PR:** `test.sh` + `check-pr.sh` run to validate contributions
- **On merge:** `build.sh` generates ZIPs, `build-pdf.sh` generates PDFs, `deploy.sh` pushes to the website

### Bot Integration

`scripts/send-to-bot.py` sends build and test output to a tldr-bot service for community notifications (e.g., Gitter, Matrix, or other chat platforms).

---

_Generated May 2026 from tldr-pages/tldr at 6145185. AI-generated documentation. The library is checked by sampling (three repositories so far); this guide was not individually reviewed. Verify against the project before relying on it. Unofficial; not affiliated with or reviewed by the project._