# Gatsby Administrator & Developer Guide

## 1. Architecture Overview

Gatsby is a React-based static site generator organized as a monorepo of ~90 packages. At build time, it sources data from arbitrary inputs (filesystems, CMSs, APIs), unifies them into a GraphQL schema, runs user-defined queries, and renders static HTML with embedded React for client-side hydration. The result is a `public/` directory of static assets ready for CDN deployment.

### High-Level Architecture

```text
┌──────────────────────────────────────────────────────────────┐
│                       gatsby-cli                             │
│  CLI entry point: develop / build / serve / clean / info     │
└──────────────┬───────────────────────────────────────────────┘
               │
┌──────────────▼───────────────────────────────────────────────┐
│                    gatsby (core)                              │
│                                                              │
│  ┌─────────────┐  ┌──────────────┐  ┌─────────────────────┐ │
│  │ Bootstrap    │  │ Data Layer   │  │ Build Pipeline      │ │
│  │ (initialize, │  │ (sourceNodes,│  │ (createPages,       │ │
│  │  load        │  │  schema,     │  │  extract queries,   │ │
│  │  plugins)    │  │  GraphQL)    │  │  run queries,       │ │
│  └──────┬──────┘  └──────┬───────┘  │  build HTML,        │ │
│         │                │          │  webpack bundle)     │ │
│         │                │          └──────────┬──────────┘ │
│  ┌──────▼────────────────▼─────────────────────▼──────────┐ │
│  │              State Machine (xstate)                     │ │
│  │  Services: buildSchema, createPages, extractQueries,    │ │
│  │  runQueries, buildHTML, graphQLTypegen                   │ │
│  └─────────────────────────────────────────────────────────┘ │
│                                                              │
│  ┌───────────────────┐  ┌──────────────┐  ┌──────────────┐  │
│  │ Redux Store       │  │ LMDB Cache   │  │ Worker Pool  │  │
│  │ (pages, nodes,    │  │ (persistent  │  │ (parallel    │  │
│  │  schema, config)  │  │  datastore)  │  │  processing) │  │
│  └───────────────────┘  └──────────────┘  └──────────────┘  │
└──────────────────────────────────────────────────────────────┘
               │
┌──────────────▼───────────────────────────────────────────────┐
│                     Plugin Ecosystem                          │
│  Source: filesystem, WordPress, Contentful, Drupal, Shopify  │
│  Transform: remark, sharp, MDX, JSON, YAML, CSV, XML        │
│  General: image, offline, manifest, sitemap, TypeScript      │
└──────────────────────────────────────────────────────────────┘
               │
┌──────────────▼───────────────────────────────────────────────┐
│                     Output: public/                           │
│  Static HTML, JS bundles, CSS, optimized images, page-data   │
└──────────────────────────────────────────────────────────────┘
```

### Component Inventory

| Component | Path | Role |
|---|---|---|
| `gatsby-cli` | `packages/gatsby-cli/` | CLI entry point, Ink-based reporter, command routing |
| `gatsby` (core) | `packages/gatsby/` | Bootstrap, data layer, build pipeline, Redux store, webpack config |
| `create-gatsby` | `packages/create-gatsby/` | Interactive project scaffolding wizard |
| `gatsby-plugin-sharp` | `packages/gatsby-plugin-sharp/` | Image processing via libvips/sharp |
| `gatsby-plugin-image` | `packages/gatsby-plugin-image/` | `<StaticImage>` and `<GatsbyImage>` components |
| `gatsby-transformer-sharp` | `packages/gatsby-transformer-sharp/` | Creates `ImageSharp` nodes from image files |
| `gatsby-source-filesystem` | `packages/gatsby-source-filesystem/` | Source files from local filesystem |
| `gatsby-source-wordpress` | `packages/gatsby-source-wordpress/` | Source WordPress content via WPGraphQL |
| `gatsby-transformer-remark` | `packages/gatsby-transformer-remark/` | Markdown to HTML/AST transformation |
| `gatsby-plugin-mdx` | `packages/gatsby-plugin-mdx/` | MDX support (Markdown + JSX) |
| `gatsby-link` | `packages/gatsby-link/` | `<Link>` component with prefetching |
| `gatsby-core-utils` | `packages/gatsby-core-utils/` | Shared utilities (hashing, CPU count, CI detection, file fetching) |
| `gatsby-plugin-utils` | `packages/gatsby-plugin-utils/` | Plugin author utilities, remote file polyfill |
| `gatsby-page-utils` | `packages/gatsby-page-utils/` | File-system route API, page path utilities |
| `gatsby-worker` | `packages/gatsby-worker/` | Worker pool for parallel build tasks |
| `gatsby-graphiql-explorer` | `packages/gatsby-graphiql-explorer/` | Customized GraphiQL IDE |
| `gatsby-plugin-page-creator` | `packages/gatsby-plugin-page-creator/` | Automatic page creation from `src/pages/` |
| `gatsby-plugin-offline` | `packages/gatsby-plugin-offline/` | Service worker for offline support |
| `babel-preset-gatsby` | `packages/babel-preset-gatsby/` | Default Babel preset for Gatsby projects |
| `babel-preset-gatsby-package` | `packages/babel-preset-gatsby-package/` | Babel preset for Gatsby packages (monorepo) |
| `babel-plugin-remove-graphql-queries` | `packages/babel-plugin-remove-graphql-queries/` | Strips GraphQL queries from production bundles |
| `gatsby-react-router-scroll` | `packages/gatsby-react-router-scroll/` | Scroll position management |
| `gatsby-adapter-netlify` | `packages/gatsby-adapter-netlify/` | Netlify deployment adapter |
| `gatsby-dev-cli` | `packages/gatsby-dev-cli/` | Local development CLI for Gatsby contributors |

### Technology Stack

| Layer | Technology |
|---|---|
| **Language** | JavaScript / TypeScript |
| **Runtime** | Node.js >=18, <26 |
| **UI Framework** | React 18 |
| **Build Tool** | Webpack 5 |
| **Data Layer** | GraphQL (custom schema builder, not Apollo) |
| **State Management** | Redux (internal build state) |
| **State Machine** | XState (build orchestration) |
| **Persistent Cache** | LMDB (Lightning Memory-Mapped Database) |
| **Image Processing** | Sharp (libvips) |
| **CLI** | Yargs + Ink (React-based terminal UI) |
| **Package Manager** | Yarn 1.22.19 (workspaces) |
| **Monorepo Management** | Lerna 3.x |
| **Testing** | Jest 29 |
| **Linting** | ESLint + Prettier |
| **E2E Testing** | Cypress |
| **Transpilation** | Babel 7 |

### Build Lifecycle (Data Flow)

The Gatsby build process follows a deterministic state machine:

1. **Initialize** — Load `gatsby-config.js`, resolve plugins, set up Redux store, initialize LMDB cache
2. **onPreInit** — First plugin hook; runs immediately after plugin load
3. **onPreBootstrap** — Run before bootstrap sequence
4. **Source Nodes** (`sourceNodes`) — Plugins create data nodes in the Redux store
5. **onCreateNode** — Transformer plugins react to new nodes (e.g., Markdown → HTML)
6. **Create Schema** (`createSchemaCustomization`) — Plugins define custom GraphQL types
7. **Build Schema** — Gatsby infers types from nodes and builds the GraphQL schema
8. **Create Resolvers** (`createResolvers`) — Plugins add custom field resolvers
9. **Create Pages** (`createPages`) — Plugins create pages from data
10. **Extract Queries** — Babel plugin scans components for `graphql` tagged template literals
11. **Run Queries** — Execute all page queries and static queries against the schema
12. **Build HTML** — Render each page to static HTML via server-side rendering
13. **Bundle JavaScript** — Webpack bundles client-side JavaScript
14. **onPostBuild** — Final plugin hook after build completes

> [!NOTE]
> In `develop` mode, steps 10-13 run incrementally. Gatsby uses query-on-demand to only execute queries for pages currently viewed in the browser, significantly reducing dev server startup time.

## 2. System Requirements

| Requirement | Minimum | Recommended |
|---|---|---|
| **Node.js** | 18.0.0 | 20.x LTS |
| **npm** | 8.0.0 | Latest |
| **Yarn** | 1.17.3 | 1.22.x |
| **RAM** | 2 GB | 4+ GB (8+ GB for large sites) |
| **Disk** | 1 GB free | SSD recommended |
| **OS** | macOS, Linux, Windows | Any |

### Node.js Compatibility

The `engines` field in the root `package.json` enforces:

```json
{
  "engines": {
    "yarn": "^1.17.3",
    "node": ">=18.0.0 <26",
    "npm": ">=8.0.0"
  }
}
```

> [!WARNING]
> Node.js 23 and 24 have a known buffer bounds issue on macOS (#39223). Use Node.js 18 or 20 for production builds.

## 3. Configuration Reference

### Configuration Files

| File | Format | Location | Purpose |
|---|---|---|---|
| `gatsby-config.js` | JS/TS | Project root | Site metadata, plugin list, path prefix, trailing slash |
| `gatsby-node.js` | JS/TS | Project root | Node API hooks (createPages, sourceNodes, onCreateWebpackConfig) |
| `gatsby-browser.js` | JS/TS | Project root | Browser API hooks (wrapRootElement, onRouteUpdate) |
| `gatsby-ssr.js` | JS/TS | Project root | SSR API hooks (wrapRootElement, onRenderBody) |
| `.env.development` | dotenv | Project root | Environment variables for `gatsby develop` |
| `.env.production` | dotenv | Project root | Environment variables for `gatsby build` |
| `.babelrc.js` | JS | Project root | Custom Babel configuration (extends `babel-preset-gatsby`) |

### gatsby-config.js Options

| Option | Type | Default | Description |
|---|---|---|---|
| `siteMetadata` | `object` | `{}` | Arbitrary site-wide data queryable via `site.siteMetadata` in GraphQL |
| `plugins` | `array` | `[]` | Plugin declarations: strings (name only) or `{resolve, options}` objects |
| `pathPrefix` | `string` | `""` | URL path prefix; used when site is served from a subdirectory |
| `trailingSlash` | `string` | `"always"` | `"always"`, `"never"`, or `"ignore"` |
| `graphqlTypegen` | `boolean\|object` | `false` | Generate TypeScript types from GraphQL queries |
| `jsxRuntime` | `string` | `"automatic"` | `"automatic"` (React 17+ new transform) or `"classic"` |
| `jsxImportSource` | `string` | `"react"` | JSX import source for automatic runtime (e.g., `"@emotion/react"`) |
| `polyfill` | `boolean` | `true` | Include `core-js` polyfills in the client bundle |
| `flags` | `object` | `{}` | Feature flags for experimental features |
| `adapter` | `function` | — | Deployment adapter (e.g., `gatsby-adapter-netlify`) |

### Environment Variable Reference

The following environment variables are recognized by Gatsby core and official plugins. Variables prefixed with `GATSBY_` are embedded in the client-side bundle.

#### Core Build Variables

| Variable | Default | Description |
|---|---|---|
| `NODE_ENV` | `development` | Build mode |
| `GATSBY_ACTIVE_ENV` | — | Override env file selection (loads `.env.{value}`) |
| `GATSBY_CPU_COUNT` | auto | CPU core count override. Accepts number or `logical_cores`/`physical_cores` |
| `GATSBY_CONCURRENT_DOWNLOAD` | `200` | Max concurrent remote file downloads |
| `GATSBY_WEBPACK_PUBLICPATH` | — | Override webpack `output.publicPath` |
| `GATSBY_WEBPACK_LOGGING` | — | Enable webpack logging |
| `PREFIX_PATHS` | — | Enable path prefix from CLI |
| `REACT_PROFILE` | — | Enable React profiling in production build |
| `VERBOSE` | — | Enable verbose build output |
| `NO_COLOR` | — | Disable colored CLI output |
| `GATSBY_LOGGER` | — | Logger selection |

#### Experimental Feature Flags

| Variable | Description |
|---|---|
| `GATSBY_EXPERIMENTAL_PARALLEL_SOURCING` | Run source plugins in parallel |
| `GATSBY_EXPERIMENTAL_DEV_SSR` | Enable SSR in development mode |
| `GATSBY_EXPERIMENTAL_FAST_DEV` | Enable fast dev optimizations |
| `GATSBY_EXPERIMENTAL_DISABLE_SCHEMA_REBUILD` | Skip schema rebuild on refresh |
| `GATSBY_EXPERIMENTAL_ENABLE_ADMIN` | Enable Gatsby Admin UI |
| `GATSBY_EXPERIMENTAL_PRESERVE_FILE_DOWNLOAD_CACHE` | Keep file download cache across builds |
| `GATSBY_EXPERIMENTAL_CLOUD_CLI` | Enable Gatsby Cloud CLI commands |

#### Cache and Data Variables

| Variable | Description |
|---|---|
| `GATSBY_DISABLE_CACHE_PERSISTENCE` | Disable persistent LMDB cache |
| `PRESERVE_FILE_DOWNLOAD_CACHE` | Keep file download cache |
| `GATSBY_DETECT_NODE_MUTATIONS` | Enable mutation detection for debugging |
| `GATSBY_SKIP_WRITING_SCHEMA_TO_FILE` | Skip writing GraphQL schema to `.cache` |

#### Development Server Variables

| Variable | Default | Description |
|---|---|---|
| `GATSBY_HOST` | `localhost` | Dev server host |
| `ENABLE_GATSBY_REFRESH_ENDPOINT` | — | Enable `/__refresh` webhook endpoint |
| `GATSBY_REFRESH_TOKEN` | — | Auth token for the refresh endpoint |
| `GATSBY_QUERY_ON_DEMAND` | `true` (dev) | Run queries only when pages are visited |
| `GATSBY_QUERY_ON_DEMAND_LOADING_INDICATOR` | — | Show loading indicator for on-demand queries |
| `GATSBY_SOCKET_IO_DEFAULT_TRANSPORT` | — | Override Socket.IO transport |

> [!IMPORTANT]
> `ENABLE_GATSBY_REFRESH_ENDPOINT` is required for CMS preview integrations (e.g., Drupal, Contentful). When set to `true`, a `POST` to `/__refresh` triggers a full data refresh in development.

#### Image Processing Variables

| Variable | Description |
|---|---|
| `GATSBY_JPEG_ENCODER` | JPEG encoder override for `gatsby-plugin-sharp` |
| `GATSBY_ENABLE_LAZY_IMAGES_IN_CI` | Enable lazy image processing in CI environments |
| `ENABLE_GATSBY_EXTERNAL_JOBS` | Enable external job processing for images |
| `IMAGE_CDN_ENCRYPTION_SECRET_KEY` | Encryption key for Image CDN URLs |
| `IMAGE_CDN_ENCRYPTION_IV` | Initialization vector for Image CDN encryption |
| `IMAGE_CDN_HOSTNAME` | Hostname for Image CDN URLs |
| `IMAGE_CDN_ROUTE_PREFIX` | Route prefix for Image CDN |
| `GATSBY_SCREENSHOT_PLACEHOLDER` | Placeholder for screenshot transformer |

#### Plugin-Specific Variables

| Variable | Plugin | Description |
|---|---|---|
| `HTTPBASICAUTH_USERNAME` | `gatsby-source-wordpress` | HTTP Basic Auth username |
| `HTTPBASICAUTH_PASSWORD` | `gatsby-source-wordpress` | HTTP Basic Auth password |
| `GATSBY_CONTENTFUL_OFFLINE` | `gatsby-source-contentful` | Enable offline mode for Contentful |
| `NETLIFY_IMAGE_CDN` | `gatsby-adapter-netlify` | Enable Netlify Image CDN |
| `GATSBY_EXCLUDE_DATASTORE_FROM_BUNDLE` | `gatsby-adapter-netlify` | Exclude LMDB datastore from function bundle |

#### Benchmark and Internal Variables

| Variable | Description |
|---|---|
| `BENCHMARK_REPORTING_URL` | URL for benchmark data reporting |
| `BENCHMARK_REPORTING_SECRET` | Auth secret for benchmark reporting |
| `CI_NAME` | CI provider name for benchmark identification |
| `BLUEBIRD_DEBUG` | Enable Bluebird promise debugging |
| `BLUEBIRD_LONG_STACK_TRACES` | Enable long stack traces for promises |
| `GATSBY_OPEN_TRACING_CONFIG_FILE` | OpenTracing configuration file path |
| `GATSBY_NODE_GLOBALS` | Enable experimental Node.js globals |

### CLI Flags Reference

#### `gatsby develop`

| Flag | Short | Type | Default | Description |
|---|---|---|---|---|
| `--host` | `-H` | `string` | `localhost` | Dev server host |
| `--port` | `-p` | `number` | `8000` | Dev server port |
| `--open` | `-o` | `boolean` | `false` | Open browser on start |
| `--https` | | `boolean` | `false` | Use HTTPS with self-signed cert |
| `--cert-file` | | `string` | — | Custom SSL certificate file |
| `--key-file` | | `string` | — | Custom SSL key file |
| `--ca-file` | | `string` | — | Custom CA certificate file |
| `--graphql-tracing` | | `boolean` | `false` | Enable GraphQL tracing |
| `--open-tracing-config-file` | | `string` | — | OpenTracing config file |
| `--inspect` | | `number` | `9229` | Enable Node.js inspector |
| `--inspect-brk` | | `number` | `9229` | Enable inspector with break |

#### `gatsby build`

| Flag | Short | Type | Default | Description |
|---|---|---|---|---|
| `--prefix-paths` | | `boolean` | `false` | Build with `pathPrefix` from config |
| `--no-uglify` | | `boolean` | `false` | Skip JS minification |
| `--profile` | | `boolean` | `false` | Enable React profiling |
| `--graphql-tracing` | | `boolean` | `false` | Enable GraphQL tracing |
| `--log-pages` | | `boolean` | `false` | Log generated pages |
| `--write-to-file` | | `boolean` | `false` | Write page data to files |
| `--functions-platform` | | `string` | — | Target platform for serverless functions |
| `--functions-arch` | | `string` | — | Target architecture for serverless functions |

#### `gatsby serve`

| Flag | Short | Type | Default | Description |
|---|---|---|---|---|
| `--host` | `-H` | `string` | `localhost` | Server host |
| `--port` | `-p` | `number` | `9000` | Server port |
| `--open` | `-o` | `boolean` | `false` | Open browser on start |
| `--prefix-paths` | | `boolean` | `false` | Serve with path prefix |

## 4. Deployment & Scaling

### Build Output Structure

After `gatsby build`, the `public/` directory contains:

```text
public/
├── index.html                    # Homepage
├── 404.html                      # 404 page
├── page-data/
│   ├── index/page-data.json     # Page data for homepage
│   ├── 404.html/page-data.json  # Page data for 404
│   └── sq/d/                    # Static query results
├── static/                       # Static assets (copied from src/static/)
├── chunk-map.json               # Webpack chunk mapping
├── webpack.stats.json           # Webpack build statistics
└── [page-paths]/                # Static HTML for each page
```

### Deployment Targets

Gatsby produces static files deployable to any static hosting platform:

| Platform | Adapter | Notes |
|---|---|---|
| **Netlify** | `gatsby-adapter-netlify` | Native adapter support, Image CDN |
| **Vercel** | — | Auto-detected from framework |
| **GitHub Pages** | — | Use `--prefix-paths` with `pathPrefix` config |
| **AWS S3 + CloudFront** | — | Upload `public/` to S3, configure CloudFront |
| **Docker (nginx)** | — | Serve `public/` with nginx |

### Adapter System

Gatsby 5 introduced adapters for platform-specific optimizations:

```javascript
// gatsby-config.js
const adapter = require("gatsby-adapter-netlify").default

module.exports = {
  adapter: adapter({
    excludeDatastoreFromEngineFunction: false,
  }),
}
```

The adapter hooks into the build pipeline to:
- Configure server-side rendering routing
- Optimize function bundling for the target platform
- Set up CDN cache headers
- Configure redirects and rewrites

> [!NOTE]
> The `GATSBY_CONTINUE_BUILD_ON_ADAPTER_MISMATCH` env var allows builds to proceed when the detected deployment platform doesn't match the configured adapter.

### Scaling Large Builds

For sites with thousands of pages:

1. **Increase Node.js heap:**
   ```bash
   NODE_OPTIONS="--max-old-space-size=8192" gatsby build
   ```

2. **Maximize CPU utilization:**
   ```bash
   GATSBY_CPU_COUNT=logical_cores gatsby build
   ```

3. **Enable parallel sourcing:**
   ```bash
   GATSBY_EXPERIMENTAL_PARALLEL_SOURCING=true gatsby build
   ```

4. **Preserve download cache:**
   ```bash
   GATSBY_EXPERIMENTAL_PRESERVE_FILE_DOWNLOAD_CACHE=true gatsby build
   ```

> [!TIP]
> The `gatsby-worker` package provides a custom worker pool (not Node.js `worker_threads` directly) that serializes tasks via IPC. The `WorkerPool` class in `packages/gatsby-worker/src/index.ts` manages lifecycle, task queuing, and error propagation. Configuring `GATSBY_CPU_COUNT` directly controls pool size.

## 5. Operations

### Cache Management

Gatsby uses multiple cache layers:

| Cache | Location | Purpose | Invalidation |
|---|---|---|---|
| **LMDB datastore** | `.cache/data/` | Persistent node storage | `gatsby clean` |
| **Webpack cache** | `.cache/webpack/` | Module compilation cache | `gatsby clean` |
| **Query results** | `.cache/` | Compiled query results | Automatic on data change |
| **Image cache** | `.cache/caches/gatsby-plugin-sharp/` | Processed image variants | `gatsby clean` |
| **File download cache** | `.cache/caches/gatsby-source-filesystem/` | Downloaded remote files | Configurable via env |

```bash
# Full cache clear
gatsby clean

# Selective cache management
rm -rf .cache/webpack  # Clear only webpack cache
```

> [!TIP]
> In CI environments, persist `.cache/` and `public/` between builds for incremental builds. This can reduce build times by 50-80% for sites that primarily add/update content.

### Monitoring Build Performance

Enable verbose output to diagnose slow builds:

```bash
VERBOSE=true gatsby build
```

This reports per-lifecycle timing:
```text
success open and validate gatsby-configs - 0.058s
success load plugins - 0.284s
success source and transform nodes - 12.345s  ← bottleneck
success building schema - 0.265s
success createPages - 1.234s
success run page queries - 45.678s  ← bottleneck
success Building production JavaScript and CSS bundles - 23.456s
success Building static HTML for pages - 8.901s
```

For distributed tracing, use OpenTracing:

```bash
GATSBY_OPEN_TRACING_CONFIG_FILE=./jaeger-config.js gatsby build
```

> [!NOTE]
> The `gatsby-plugin-benchmark-reporting` package in the monorepo provides automated build performance reporting. It captures `onPostBuild` data and sends it to a configurable endpoint via `BENCHMARK_REPORTING_URL`.

### Health and Diagnostics

The `gatsby info` command outputs environment diagnostics:

```bash
gatsby info --clipboard
```

```text
System:
  OS: macOS 14.0
  CPU: (8) arm64 Apple M1
  Shell: /bin/zsh
Binaries:
  Node: 20.10.0
  npm: 10.2.3
  Yarn: 1.22.19
npmPackages:
  gatsby: ^5.13.0 => 5.13.3
  gatsby-plugin-image: ^3.13.0 => 3.13.1
  gatsby-plugin-sharp: ^5.13.0 => 5.13.1
```

## 6. Performance

### Webpack Configuration

Gatsby's webpack configuration (`packages/gatsby/src/utils/webpack.config.js`) supports four stages:

| Stage | Purpose |
|---|---|
| `develop` | Dev server with HMR |
| `develop-html` | SSR during development |
| `build-javascript` | Production JS bundles |
| `build-html` | Static HTML rendering |

Key webpack behaviors:

- **Code splitting** — automatic chunk splitting per route
- **Asset hashing** — content-based filenames for long-term caching
- **Polyfills** — `core-js` included by default (disable with `polyfill: false`)
- **Public path** — configurable via `GATSBY_WEBPACK_PUBLICPATH` or `pathPrefix`

```javascript
// Custom webpack config via gatsby-node.js
exports.onCreateWebpackConfig = ({ stage, getConfig, rules, loaders, plugins, actions }) => {
  // Access current config
  const config = getConfig()

  // Modify via setWebpackConfig (merged with existing)
  actions.setWebpackConfig({
    resolve: {
      alias: {
        '@components': path.resolve(__dirname, 'src/components'),
      },
    },
  })

  // Or replace entire config (use with caution)
  actions.replaceWebpackConfig(config)
}
```

### Image Processing Pipeline

`gatsby-plugin-sharp` wraps the Sharp library (libvips) for image transformations:

- **Formats:** JPEG, PNG, WebP, AVIF, TIFF
- **Operations:** resize, crop, rotate, duotone, blur, grayscale
- **Lazy processing:** In development, images are processed on-demand when requested by the browser
- **Worker pool:** Image operations run in parallel worker processes
- **Error handling:** `SharpError` class wraps failures with file context

The image CDN subsystem (`packages/gatsby-plugin-utils/src/polyfill-remote-file/`) supports encrypted URL generation for CDN-optimized image delivery:

| Variable | Purpose |
|---|---|
| `IMAGE_CDN_ENCRYPTION_SECRET_KEY` | AES encryption key for CDN URLs |
| `IMAGE_CDN_ENCRYPTION_IV` | Initialization vector |
| `IMAGE_CDN_HOSTNAME` | CDN hostname |
| `IMAGE_CDN_ROUTE_PREFIX` | URL route prefix |

### Query-on-Demand

In development, Gatsby defers query execution until a page is viewed:

- Only 404 pages, currently active browser tabs, and pending websocket requests run immediately
- Static queries (`sq--` prefix) and slice queries (`slice--` prefix) always run
- Controlled via `GATSBY_QUERY_ON_DEMAND` (default: `true` in develop)
- `GATSBY_ENABLE_QUERY_ON_DEMAND_IN_CI` overrides for CI environments

## 7. Security

### Client-Side Data Exposure

> [!CAUTION]
> All `GATSBY_*` environment variables are embedded in the webpack bundle and visible in client-side JavaScript. Never use this prefix for secrets, API keys, or credentials.

- `.env.development` and `.env.production` files should be in `.gitignore`
- Server-only variables (without `GATSBY_` prefix) are safe for secrets during build
- GraphQL data is serialized to `page-data.json` files in `public/` — any data queried is publicly accessible

### Authentication Patterns

Gatsby supports client-side authentication via client-only routes:

```javascript
// gatsby-node.js — mark routes as client-only
exports.onCreatePage = async ({ page, actions }) => {
  if (page.path.match(/^\/app/)) {
    page.matchPath = "/app/*"
    actions.createPage(page)
  }
}
```

The SSR build will render a shell for these routes; client-side React handles authentication checks.

### Serverless Function Security

Functions in `src/api/` receive standard Node.js `req`/`res` objects. Platform-specific middleware (e.g., Netlify Identity, Vercel auth) provides authentication for serverless endpoints.

### Refresh Endpoint

When `ENABLE_GATSBY_REFRESH_ENDPOINT=true`:
- The `/__refresh` endpoint accepts POST requests to trigger data refresh
- `GATSBY_REFRESH_TOKEN` provides optional bearer token authentication
- The initial webhook body can be passed via `GATSBY_INITIAL_WEBHOOK_BODY`

> [!WARNING]
> The refresh endpoint is intended for development and CMS preview integrations only. Do not expose it in production without authentication.

### Supply Chain

The monorepo uses:
- Yarn 1.22.19 with `workspaces` for deterministic resolution
- `lint-staged` + `husky` for pre-commit checks
- ESLint + Prettier for code quality enforcement
- TypeScript strict mode for type safety in core packages

## 8. Developer Internals

### Repository Structure

```text
gatsbyjs/gatsby/
├── packages/                    # ~90 npm packages
│   ├── gatsby/                  # Core framework (5.8M chars)
│   │   ├── src/
│   │   │   ├── commands/        # CLI command implementations
│   │   │   ├── services/        # Build lifecycle services (21 files)
│   │   │   ├── utils/           # Core utilities (89 files, 350K chars)
│   │   │   ├── schema/          # GraphQL schema builder
│   │   │   ├── redux/           # Redux store, actions, reducers
│   │   │   ├── query/           # Query extraction, planning, execution
│   │   │   ├── datastore/       # LMDB-backed data storage
│   │   │   ├── state-machines/  # XState state machines
│   │   │   └── internal-plugins/ # Built-in plugins (dev-404, partytown)
│   │   └── cache-dir/           # Client-side runtime (loader, navigation)
│   ├── gatsby-cli/              # CLI entry point
│   ├── create-gatsby/           # Project scaffolding
│   ├── gatsby-plugin-sharp/     # Image processing (75K chars)
│   ├── gatsby-source-filesystem/ # File sourcing (23K chars)
│   ├── gatsby-source-wordpress/ # WordPress sourcing (50K chars)
│   ├── gatsby-transformer-remark/ # Markdown transformation (31K chars)
│   ├── gatsby-core-utils/       # Shared utilities (35K chars)
│   ├── gatsby-worker/           # Worker pool (22K chars)
│   └── [60+ more packages]
├── starters/                    # Starter templates
│   ├── blog/
│   ├── default/
│   ├── gatsby-starter-minimal-ts/
│   ├── gatsby-starter-plugin/
│   └── gatsby-starter-wordpress-blog/
├── examples/                    # Example projects (~50)
├── e2e-tests/                   # Cypress E2E test suites
├── integration-tests/           # Integration test suites
├── benchmarks/                  # Performance benchmarks
├── deprecated-packages/         # Deprecated packages (recipes, telemetry, admin)
├── scripts/                     # Monorepo tooling (changelog, publish)
└── peril/                       # GitHub bot (Danger.js)
```

### State Machine Architecture

Gatsby uses XState to orchestrate the build pipeline. The state machines in `packages/gatsby/src/state-machines/` define:

- **Data Layer Machine** — Sources nodes, builds schema, creates pages
- **Query Running Machine** — Extracts queries, calculates dirty queries, executes them
- **Waiting Machine** — Handles the develop-mode idle loop (listens for file changes, webhooks)

Services (in `packages/gatsby/src/services/`) are invoked by the state machines:

| Service | File | Purpose |
|---|---|---|
| `buildSchema` | `build-schema.ts` | Build GraphQL schema from nodes |
| `calculateDirtyQueries` | `calculate-dirty-queries.ts` | Identify queries needing re-execution |
| `createPages` | `create-pages.ts` | Execute `createPages` API, track page changes |
| `customizeSchema` | `customize-schema.ts` | Execute `createSchemaCustomization` API |
| `extractQueries` | `extract-queries.ts` | Scan JS/TS files for GraphQL queries |
| `graphQLTypegen` | `graphql-typegen.ts` | Generate TypeScript types from schema |
| `initialize` | `initialize.ts` | Bootstrap: load plugins, initialize cache, setup config |
| `rebuildSchemaWithSitePage` | `rebuild-schema-with-site-page.ts` | Add `SitePage` type after page creation |
| `recompile` | `recompile.ts` | Webpack recompilation on file changes |
| `runQueries` | `run-queries.ts` | Execute page and static queries |
| `startWebpackServer` | `start-webpack-server.ts` | Launch webpack dev server |
| `writeOutRedirects` | `write-out-redirects.ts` | Write redirect configuration |

### Redux Store Shape

The internal Redux store (in `packages/gatsby/src/redux/`) tracks:

- `pages` — Map of all pages (path → page object)
- `nodes` — All data nodes (internal, backed by LMDB)
- `schema` — GraphQL schema object
- `config` — Resolved `gatsby-config.js`
- `components` — Map of page component paths
- `staticQueryComponents` — Static query definitions
- `status` — Build status flags
- `queries` — Query tracking and dirty state
- `slices` — Gatsby Slice components

### Node API Reference

These hooks are exported from `gatsby-node.js`:

| Hook | Phase | Purpose |
|---|---|---|
| `onPreInit` | Bootstrap | First hook, runs immediately after plugin load |
| `onPluginInit` | Bootstrap | Per-process initialization (v3.9+) |
| `onPreBootstrap` | Bootstrap | Before bootstrap sequence |
| `onPostBootstrap` | Bootstrap | After bootstrap complete |
| `sourceNodes` | Data | Create data nodes from sources |
| `onCreateNode` | Data | React to node creation (transform/extend) |
| `shouldOnCreateNode` | Data | Filter which nodes trigger `onCreateNode` (v5+) |
| `createSchemaCustomization` | Schema | Define custom GraphQL types |
| `setFieldsOnGraphQLNodeType` | Schema | Add fields to inferred types |
| `createResolvers` | Schema | Add custom field resolvers |
| `createPages` | Pages | Create pages from data |
| `createPagesStatefully` | Pages | Stateful page creation (for plugins managing own state) |
| `onCreatePage` | Pages | Modify page after creation |
| `onCreateBabelConfig` | Build | Extend Babel configuration |
| `onCreateWebpackConfig` | Build | Extend webpack configuration |
| `preprocessSource` | Build | Process compile-to-JS source |
| `resolvableExtensions` | Build | Add resolvable file extensions |
| `onPreExtractQueries` | Queries | Before query extraction |
| `onPreBuild` | Build | Before production build |
| `onPostBuild` | Build | After production build |
| `onCreateDevServer` | Dev | After dev server starts (Express app) |
| `pluginOptionsSchema` | Config | Define plugin options schema (Joi) |

### Browser API Reference

These hooks are exported from `gatsby-browser.js`:

| Hook | Purpose |
|---|---|
| `onClientEntry` | Called when browser runtime starts |
| `onInitialClientRender` | After initial React render |
| `onPreRouteUpdate` | Before route change begins |
| `onRouteUpdate` | After route change (including initial load) |
| `onRouteUpdateDelayed` | When route change takes >1 second |
| `shouldUpdateScroll` | Control scroll behavior on navigation |
| `wrapPageElement` | Wrap each page (persists across navigations) |
| `wrapRootElement` | Wrap root React element (context providers) |
| `onPrefetchPathname` | When prefetching is triggered |
| `onPostPrefetchPathname` | After prefetch completes |
| `disableCorePrefetching` | Disable built-in prefetching |
| `replaceHydrateFunction` | Custom `ReactDOM.createRoot`/`render` |
| `registerServiceWorker` | Enable service worker registration |
| `onServiceWorkerInstalled` | Service worker installed |
| `onServiceWorkerUpdateFound` | Service worker update available |
| `onServiceWorkerUpdateReady` | Service worker update ready to apply |
| `onServiceWorkerActive` | Service worker activated |
| `onServiceWorkerRedundant` | Service worker became redundant |
| `onCaughtError` | React 19 caught error handler |
| `onUncaughtError` | React 19 uncaught error handler |

### Node API Helpers

Every Node API hook receives a helpers object with:

| Helper | Type | Description |
|---|---|---|
| `actions` | `Actions` | Functions to modify Gatsby state (createNode, createPage, etc.) |
| `cache` | `GatsbyCache` | Key-value cache (`.get()`, `.set()`, `.del()`) backed by LMDB |
| `createNodeId` | `function` | Generate stable, globally unique node IDs |
| `createContentDigest` | `function` | Create content hash for change detection |
| `reporter` | `GatsbyReporter` | Logging (`.info()`, `.warn()`, `.error()`, `.panic()`, `.verbose()`, `.activityTimer()`) |
| `getNodes` | `function` | Get all nodes |
| `getNode` | `function` | Get node by ID |
| `getNodesByType` | `function` | Get nodes of a specific type |
| `loadNodeContent` | `function` | Get content from node's source plugin |
| `store` | `Redux.Store` | Internal Redux store (private API) |
| `emitter` | `Emitter` | Internal event emitter (private API) |
| `tracing` | `GatsbyTracing` | OpenTracing integration (`.tracer`, `.parentSpan`, `.startSpan()`) |
| `graphql` | `function` | Run GraphQL queries (available in `createPages`) |
| `getCache` | `function` | Get cache for a specific plugin |
| `pathPrefix` | `string` | Resolved path prefix |
| `basePath` | `string` | Base path (without asset prefix) |

### Client-Side Runtime

The `packages/gatsby/cache-dir/` directory contains the client-side runtime:

| Module | Purpose |
|---|---|
| `loader.js` | Page data loader (`BaseLoader`, `ProdLoader`, `DevLoader`). Manages prefetching, caching, and loading of `page-data.json` and component chunks |
| `navigation.js` | Client-side router integration, `RouteAnnouncer` for a11y, history management |
| `production-app.js` | Production app entry point, hydration setup |
| `root.js` | Development app entry point, React Fast Refresh integration |
| `static-entry.js` | SSR entry point for `gatsby build` (HTML generation) |
| `ssr-develop-static-entry.js` | SSR entry point for `gatsby develop` |
| `ensure-resources.js` | `<EnsureResources>` component — blocks rendering until page data is loaded |
| `query-result-store.js` | `PageQueryStore`, `StaticQueryStore`, `SliceDataStore` — manage query result state |
| `slice.js` | Gatsby Slice component for shared layout sections |
| `static-query.js` | `useStaticQuery` hook and `<StaticQuery>` component |

### Key Utility Modules

The `packages/gatsby/src/utils/` directory (89 files, 350K chars) contains the core utility layer:

| Module | Purpose |
|---|---|
| `api-runner-node.js` | Executes Node API hooks across all plugins with proper ordering |
| `api-browser-docs.ts` | Browser API documentation and type exports |
| `api-node-docs.ts` | Node API documentation and type exports |
| `webpack.config.js` | Webpack configuration factory for all four build stages |
| `babel-loader.ts` | Custom Babel loader with caching (`BabelConfigItemsCacheInvalidatorPlugin`) |
| `gatsby-webpack-stats-extractor.ts` | Webpack plugin to extract build stats (`GatsbyWebpackStatsExtractor`) |
| `gatsby-webpack-virtual-modules.ts` | Virtual module injection (`GatsbyWebpackVirtualModules`) |
| `websocket-manager.ts` | WebSocket server for dev mode hot reload (`WebsocketManager`) |
| `start-server.ts` | Express dev server setup (serves pages, handles refresh endpoint) |
| `page-data.ts` | Page data JSON file management |
| `page-mode.ts` | Determine page rendering mode (SSG/DSG/SSR) |
| `node-manifest.ts` | Node manifest for CMS preview (limited by `NODE_MANIFEST_FILE_LIMIT`) |
| `detect-port-in-use-and-prompt.ts` | Port conflict resolution |
| `get-ssl-cert.ts` | Self-signed SSL cert generation for HTTPS dev |
| `loading-indicator.ts` | Loading indicator for query-on-demand |
| `feedback.ts` | User feedback prompt (disable with `GATSBY_FEEDBACK_DISABLED`) |
| `cache-lmdb.ts` | LMDB cache implementation |
| `detect-node-mutations.ts` | Debug tool for detecting illegal node mutations |
| `adapter/` | Adapter system (7 files) for deployment platform integration |

### Worker Pool Architecture

`packages/gatsby-worker/` provides a custom worker pool:

- **`WorkerPool`** class manages child processes via `child_process.fork()`
- **`TaskQueue`** implements FIFO task scheduling with priority support
- Workers are identified by `GATSBY_WORKER_ID` env var
- In-flight task state can be dumped to disk via `GATSBY_WORKER_IN_FLIGHT_DUMP_LOCATION`
- Error: `"There is no worker with ..."` indicates a worker process crashed

### Plugin Architecture

Each Gatsby plugin is an npm package that can implement:

1. **`gatsby-node.js`** — Node.js build-time hooks
2. **`gatsby-browser.js`** — Client-side browser hooks
3. **`gatsby-ssr.js`** — Server-side rendering hooks
4. **`package.json`** — with `"keywords": ["gatsby-plugin"]`

Plugin options are validated via `pluginOptionsSchema` using Joi:

```javascript
exports.pluginOptionsSchema = ({ Joi }) => {
  return Joi.object({
    path: Joi.string().required().description(`Path to source directory`),
    name: Joi.string().default(`default`).description(`Source instance name`),
    ignore: Joi.array().items(Joi.string()).description(`File patterns to ignore`),
  })
}
```

### Theme System

Gatsby themes are plugins that can:
- Define their own `gatsby-config.js` (merged with site config)
- Provide pages, components, and data
- Support **component shadowing** — the site can override any theme component by placing a file at `src/{theme-package-name}/{original-path}`

Theme resolution follows the webpack theme component shadowing system in `packages/gatsby/src/internal-plugins/webpack-theme-component-shadowing/`.

## 9. Integration Points

### Source Plugin Contract

Source plugins implement `sourceNodes` to create nodes:

```javascript
exports.sourceNodes = async ({ actions, createNodeId, createContentDigest }) => {
  const { createNode } = actions

  // Fetch data from external source
  const data = await fetchDataFromAPI()

  data.forEach(item => {
    createNode({
      ...item,
      id: createNodeId(`my-source-${item.id}`),
      parent: null,
      children: [],
      internal: {
        type: `MySourceType`,
        contentDigest: createContentDigest(item),
      },
    })
  })
}
```

### Transformer Plugin Contract

Transformer plugins implement `onCreateNode` to transform source nodes:

```javascript
exports.onCreateNode = async ({ node, actions, loadNodeContent, createNodeId, createContentDigest }) => {
  if (node.internal.mediaType !== `text/markdown`) return

  const { createNode, createParentChildLink } = actions
  const content = await loadNodeContent(node)

  const markdownNode = {
    id: createNodeId(`${node.id} >>> MarkdownRemark`),
    children: [],
    parent: node.id,
    internal: {
      type: `MarkdownRemark`,
      content,
      contentDigest: createContentDigest(content),
    },
  }

  createNode(markdownNode)
  createParentChildLink({ parent: node, child: markdownNode })
}
```

### gatsby-source-filesystem

Creates `File` nodes from local filesystem paths:

| Function | Purpose |
|---|---|
| `createFileNode` | Create a `File` node from a local file path |
| `createFileNodeFromBuffer` | Create a `File` node from an in-memory buffer |
| `createRemoteFileNode` | Download and create a `File` node from a remote URL |

### gatsby-source-wordpress

Sources WordPress content via WPGraphQL:

- **Hooks:** `onPluginInit`, `createSchemaCustomization`, `sourceNodes`, `onCreatePage`, `onCreateDevServer`
- **State management:** Rematch (Redux-like) with async local storage
- **Browser hook:** `onRouteUpdate` hydrates inline WordPress images
- **Auth:** HTTP Basic Auth via `HTTPBASICAUTH_USERNAME`/`HTTPBASICAUTH_PASSWORD`
- **Preview:** Integrates with `GATSBY_IS_PREVIEW` and `ENABLE_GATSBY_REFRESH_ENDPOINT`
- **Error handling:** Validates GraphQL responses, detects empty strings, redirects, and connection failures

### gatsby-transformer-remark

Transforms Markdown files to `MarkdownRemark` nodes:

- **Processor:** Unified/remark with GFM and footnotes support
- **Output types:** `MarkdownRemark` (HTML, excerpt, headings, table of contents, word count, time to read)
- **Caching:** AST, HTML, headings, and TOC are cached by content digest
- **Sub-plugins:** Supports remark sub-plugins via `createSchemaCustomization`
- **Heading extraction:** Generates IDs and extracts `{id, value, depth}` tuples

### Deprecated Packages

The following packages are in `deprecated-packages/`:

| Package | Status | Replacement |
|---|---|---|
| `gatsby-recipes` | Deprecated | Manual setup / `create-gatsby` |
| `gatsby-admin` | Deprecated | None (experimental) |
| `gatsby-telemetry` | Deprecated | Removed from core |
| `gatsby-plugin-gatsby-cloud` | Deprecated | `gatsby-adapter-netlify` or platform-specific adapters |
| `gatsby-plugin-remove-trailing-slashes` | Deprecated | `trailingSlash: "never"` in gatsby-config |
| `gatsby-plugin-guess-js` | Deprecated | None |
| `gatsby-image` | Deprecated | `gatsby-plugin-image` |
| `gatsby-plugin-create-client-paths` | Deprecated | `matchPath` in `onCreatePage` |
| `gatsby-transformer-javascript-static-exports` | Deprecated | None |

### Testing Strategy

| Type | Framework | Location |
|---|---|---|
| **Unit tests** | Jest 29 | `packages/*/src/__tests__/` |
| **Integration tests** | Jest | `integration-tests/` |
| **E2E tests** | Cypress 12 | `e2e-tests/` |
| **Benchmarks** | Custom | `benchmarks/` |

Running tests:

```bash
# All tests
yarn jest

# Specific package
yarn jest packages/gatsby-plugin-sharp

# Integration tests
yarn test:integration

# Type checking
yarn typecheck
```

### Contributing

The monorepo uses:

- **Yarn workspaces** (`packages/*`) for dependency management
- **Lerna** for versioning and publishing
- **ESLint** with Google style base + custom rules (backtick quotes, camelCase, file naming)
- **Prettier** for formatting
- **Husky** for pre-commit hooks (`lint-staged`)
- **TypeScript** for core packages (`.ts`/`.tsx`)

Development workflow:

```bash
# Bootstrap all packages
yarn bootstrap

# Watch for changes
yarn watch

# Run tests
yarn jest

# Lint
yarn lint

# Format
yarn format
```

The changelog is automatically generated by `scripts/gatsby-changelog-generator/` during Lerna version bumps.

### Error Classes

| Error Class | Module | Description |
|---|---|---|
| `SharpError` | `gatsby-plugin-sharp/src/sharp-error.js` | Image processing failure |
| `BuildHTMLError` | `gatsby/src/commands/build-html.ts` | HTML rendering failure |
| `StringInterpolationNotAllowedError` | `babel-plugin-remove-graphql-queries` | Template literal in GraphQL tag |
| `EmptyGraphQLTagError` | `babel-plugin-remove-graphql-queries` | Empty `graphql` tag |
| `GraphQLSyntaxError` | `babel-plugin-remove-graphql-queries` | Invalid GraphQL syntax |
| `ExportIsNotAsyncError` | `babel-plugin-remove-graphql-queries` | Non-async config export |
| `SlicePropsError` | `gatsby/cache-dir/slice.js` | Invalid Slice component props |
| `OperationError` | `gatsby-source-shopify` | Shopify API operation failure |
| `HttpError` | `gatsby-source-shopify` | Shopify HTTP request failure |

---

_Generated May 2026 from gatsbyjs/gatsby at 47bd906. AI-generated documentation. The library is checked by sampling (three repositories so far); this guide was not individually reviewed. Verify against the project before relying on it. Unofficial; not affiliated with or reviewed by the project._