# Hugging Face Transformers — Administrator & Developer Guide

## 1. Architecture Overview

Hugging Face Transformers v5.6.0.dev0 is the model-definition framework for state-of-the-art machine learning across text, vision, audio, and multimodal tasks, supporting both inference and training. The library is built around a small number of foundational architectural patterns that govern how the 1,000+ supported model families are structured, imported, and executed.

> [!NOTE]
> Transformers follows a **lazy import architecture**. No backend-specific code (PyTorch, TensorFlow, JAX, vision, audio) is loaded until the symbol is actually accessed. This means `import transformers` is cheap regardless of which optional dependencies are installed.

### 1.1 Lazy Import System

The central design pattern is deferred loading via three cooperating components:

1. **`_import_structure` dict** — Defined in `src/transformers/__init__.py`, this maps every public symbol to the module path that contains it. The dict is populated at module load time but no modules are actually imported.

2. **`_LazyModule`** — A custom module subclass that intercepts `__getattr__` calls. When code accesses `transformers.BertModel` for the first time, `_LazyModule` resolves the path from `_import_structure` and imports only that specific module.

3. **`define_import_structure()`** — Helper that populates the `_import_structure` dict from structured declarations, keeping the init file maintainable at scale.

```python
# Conceptual illustration of how lazy loading works
import transformers
# At this point, torch has NOT been imported

model_class = transformers.BertModel
# Now, and only now, src/transformers/models/bert/modeling_bert.py is imported
```

### 1.2 Optional Dependency Pattern

Backend-specific code is guarded by two mechanisms:

- **`OptionalDependencyNotAvailable`** — A sentinel exception raised when a required optional dependency (torch, vision libraries, audio libraries) is not installed.
- **`requires_backends()`** — Decorator/guard that attaches a meaningful error message to any class or function that requires a specific backend, rather than surfacing a raw `ImportError`.

This means a user without PyTorch installed can still import `transformers`, use tokenizers, and access configuration objects — only model classes raise errors at instantiation time.

### 1.3 Auto Classes and Model Registry

The Auto class system (`AutoModel`, `AutoTokenizer`, `AutoConfig`, `AutoModelForCausalLM`, and ~40 more task-specific variants) provides a unified interface for loading any supported model by name.

The registry is maintained in `src/transformers/models/auto/auto_mappings.py` via two families of mapping dicts:

| Mapping | Purpose |
|---|---|
| `CONFIG_MAPPING_NAMES` | Maps string model type identifiers to `*Config` class names |
| `MODEL_FOR_*_MAPPING_NAMES` | Maps model types to task-specific model class names |

When `AutoModel.from_pretrained("bert-base-uncased")` is called, the library:
1. Downloads or loads the `config.json` from the model hub
2. Reads the `"model_type"` field
3. Looks up the matching class in `CONFIG_MAPPING_NAMES` / `MODEL_FOR_*_MAPPING_NAMES`
4. Instantiates and returns the concrete class

### 1.4 Model Directory Convention

Each model family lives at `src/transformers/models/{model_name}/` and contains a standardized set of files:

| File pattern | Purpose |
|---|---|
| `configuration_{model}.py` | Config class, inherits `PreTrainedConfig` |
| `modeling_{model}.py` | PyTorch model class(es), inherits `PreTrainedModel` |
| `tokenization_{model}.py` | Slow tokenizer, inherits `PreTrainedTokenizer` |
| `tokenization_{model}_fast.py` | Fast tokenizer backed by `tokenizers` library |
| `processing_{model}.py` | Processor for multimodal models |
| `modular_{model}.py` | Modular definition (see §1.5) |

### 1.5 Modular Model System

New models may be defined using `modular_{model}.py` files that subclass existing models and override only what changes. These files are **not shipped as-is** — the repo maintenance tool `utils/modular_model_converter.py` converts them to full standalone modeling files. This workflow:

- Reduces copy-paste across similar architectures
- Keeps diffs small during model development
- Produces self-contained output files that the lazy import system can load normally

### 1.6 Pipeline System

The `pipeline()` factory function in `src/transformers/pipelines/__init__.py` is the primary high-level inference interface. It:

1. Accepts a task string (e.g., `"text-generation"`, `"image-classification"`) or a model identifier
2. Auto-detects the task from the loaded model config when not specified
3. Loads the appropriate pipeline class from the 20+ implementations in `src/transformers/pipelines/`
4. Returns a callable that handles tokenization, inference, and post-processing

### 1.7 Generation Subsystem

The `src/transformers/generation/` directory implements text generation for all autoregressive models:

- **`GenerationMixin`** — Mixed into `PreTrainedModel`, provides `generate()`, `greedy_search()`, `beam_search()`, `sample()`, and related methods
- **`GenerationConfig`** — Dataclass capturing all generation hyperparameters (temperature, top-k, top-p, repetition penalty, etc.)
- **Logits processors** — Composable transforms applied to the logit distribution at each decoding step (see §11.1 for the complete list)
- **Stopping criteria** — Pluggable conditions that terminate generation
- **Streaming** — `TextStreamer`, `TextIteratorStreamer`, `AsyncTextIteratorStreamer` for token-by-token output
- **Continuous batching** — `ContinuousBatchingManager` and `ContinuousMixin` in the `continuous_batching` subdirectory

### 1.8 Training Stack

The `Trainer` class in `src/transformers/trainer.py` wraps a full PyTorch training loop with:

- Callback system for extensibility
- Logging integrations (TensorBoard, W&B, etc.) via `src/transformers/integrations/`
- Distributed training support (DDP, FSDP, DeepSpeed) via Accelerate
- `TrainingArguments` dataclass in `src/transformers/training_args.py` for all hyperparameters and infrastructure options

### 1.9 Quantization Architecture

22+ quantization backends are supported via a plugin-style system in `src/transformers/quantizers/`. Each backend implements a common interface and is activated by passing the corresponding config object to `from_pretrained()`.

### 1.10 Serving Architecture

`src/transformers/cli/serving/server.py` exposes a `build_server()` function that returns a FastAPI application with OpenAI-compatible endpoints. A `ModelManager` handles model lifecycle (load, reset). Request ID tracking is provided via X-Request-ID middleware. CORS middleware is optionally enabled.

```
┌─────────────────────────────────────────────────────────┐
│  transformers serve (CLI)                               │
│     └─ build_server() → FastAPI app                    │
│           ├── POST /v1/chat/completions                 │
│           ├── POST /v1/responses                        │
│           ├── POST /v1/audio/transcriptions             │
│           ├── POST /load_model                          │
│           ├── POST /reset                               │
│           ├── GET  /v1/models                           │
│           └── GET  /health                              │
└─────────────────────────────────────────────────────────┘
```

---

## 2. System Requirements

### 2.1 Python

| Constraint | Value |
|---|---|
| Minimum | Python 3.10 |
| Maximum tested | Python 3.14 |
| Recommended | Python 3.11 or 3.12 for production |

### 2.2 Required Dependencies

These packages are installed automatically with `pip install transformers`:

| Package | Version Constraint |
|---|---|
| `huggingface-hub` | `>=1.5.0,<2.0` |
| `numpy` | `>=1.17` |
| `packaging` | `>=20.0` |
| `pyyaml` | `>=5.1` |
| `regex` | `>=2025.10.22` |
| `tokenizers` | `>=0.22.0,<=0.23.0` |
| `typer` | (any current) |
| `safetensors` | `>=0.4.3` |
| `tqdm` | `>=4.27` |

> [!IMPORTANT]
> The `tokenizers` version is **upper-bounded at 0.23.0**. Installing a newer `tokenizers` release without first checking compatibility may cause silent failures in fast tokenizer behavior. Pin this dependency explicitly in production environments.

### 2.3 Optional Backend Dependencies

None of these are required for the library to import, but are needed for the capabilities shown:

| Dependency | Capability unlocked |
|---|---|
| `torch` | PyTorch model inference and training |
| `tensorflow` | TensorFlow model support |
| `jax` / `flax` | JAX/Flax model support |
| `torchvision` | Vision model inputs and transforms |
| `torchaudio` | Audio model inputs |
| `accelerate` | Multi-GPU, FSDP, DeepSpeed training via `Trainer` |
| `deepspeed` | DeepSpeed ZeRO training stages |
| `peft` | Parameter-efficient fine-tuning (LoRA, etc.) |
| `bitsandbytes` | BitsAndBytes quantization backend |
| `auto-gptq` / `optimum` | GPTQ quantization |
| `quanto` | Quanto quantization backend |
| `scipy` | Some tokenizer and model utilities |
| `sentencepiece` | SentencePiece tokenizer support |
| `protobuf` | Protobuf-based tokenizer loading |
| `fastapi` / `uvicorn` | `transformers serve` command |
| `rich` | `transformers chat` TUI |

### 2.4 Hardware

| Scenario | Minimum |
|---|---|
| CPU inference (small models) | 4 GB RAM |
| Single-GPU inference | CUDA-compatible GPU, driver ≥ CUDA 11.8 |
| Multi-GPU training | NCCL-compatible GPUs, NVLink or PCIe |
| XLA / TPU | TPU v2+, `torch_xla` installed |
| Apple Silicon | MPS backend via PyTorch ≥ 2.0 |

---

## 3. Configuration Reference

### 3.1 `PreTrainedConfig` Base Class

All model configs inherit from `PreTrainedConfig` (`src/transformers/configuration_utils.py`). Common attributes shared across all models:

| Attribute | Type | Description |
|---|---|---|
| `model_type` | `str` | String identifier used by Auto classes for registry lookup |
| `is_encoder_decoder` | `bool` | Whether model has separate encoder and decoder |
| `tie_word_embeddings` | `bool` | Whether input/output embeddings share weights |
| `torch_dtype` | `str` or `torch.dtype` | Default dtype for model weights |
| `attn_implementation` | `str` | Attention kernel selection (`"eager"`, `"sdpa"`, `"flash_attention_2"`) |
| `hidden_size` | `int` | Primary hidden dimension (model-specific) |
| `num_hidden_layers` | `int` | Depth (model-specific) |
| `num_attention_heads` | `int` | Attention heads (model-specific) |

### 3.2 `TrainingArguments` Reference

`TrainingArguments` (`src/transformers/training_args.py`) is a `@dataclass` that controls all aspects of the `Trainer`. Key fields:

| Argument | Default | Description |
|---|---|---|
| `output_dir` | required | Directory for checkpoints and final model |
| `num_train_epochs` | `3.0` | Number of training epochs |
| `per_device_train_batch_size` | `8` | Batch size per GPU/CPU for training |
| `per_device_eval_batch_size` | `8` | Batch size per GPU/CPU for evaluation |
| `learning_rate` | `5e-5` | Initial learning rate |
| `weight_decay` | `0.0` | Weight decay applied to all non-bias/LayerNorm params |
| `lr_scheduler_type` | `"linear"` | LR schedule (see §11.2 for all types) |
| `warmup_steps` | `0` | Linear warmup steps at start of training |
| `warmup_ratio` | `0.0` | Warmup as fraction of total steps (alternative to `warmup_steps`) |
| `logging_steps` | `500` | Log every N steps |
| `save_steps` | `500` | Checkpoint every N steps |
| `evaluation_strategy` | `"no"` | When to evaluate: `"no"`, `"steps"`, `"epoch"` |
| `fp16` | `False` | Enable AMP float16 training |
| `bf16` | `False` | Enable bfloat16 training |
| `gradient_accumulation_steps` | `1` | Accumulate gradients over N steps before update |
| `gradient_checkpointing` | `False` | Trade compute for memory during backward pass |
| `dataloader_num_workers` | `0` | Subprocess workers for DataLoader |
| `report_to` | `"all"` | Logging integrations: `"tensorboard"`, `"wandb"`, `"none"`, etc. |
| `deepspeed` | `None` | Path to DeepSpeed config JSON |
| `fsdp` | `""` | FSDP strategy string |
| `push_to_hub` | `False` | Push checkpoints to Hub after each save |
| `hub_model_id` | `None` | Hub repository to push to |

### 3.3 `GenerationConfig` Reference

`GenerationConfig` controls all autoregressive decoding behavior. Key fields:

| Parameter | Default | Description |
|---|---|---|
| `max_new_tokens` | `None` | Hard cap on tokens generated |
| `max_length` | `20` | Max total sequence length (prompt + generated) |
| `min_new_tokens` | `None` | Force at least N new tokens |
| `do_sample` | `False` | If `False`, greedy decoding; if `True`, sampling |
| `temperature` | `1.0` | Sampling temperature (higher = more random) |
| `top_k` | `50` | Top-K filtering |
| `top_p` | `1.0` | Nucleus sampling probability mass |
| `min_p` | `None` | Minimum probability threshold (MinP sampling) |
| `repetition_penalty` | `1.0` | Penalty for repeated tokens (>1.0 = penalize) |
| `no_repeat_ngram_size` | `0` | Block repeated n-grams of this size |
| `num_beams` | `1` | Beam width (1 = no beam search) |
| `num_return_sequences` | `1` | Number of independently sampled sequences |
| `pad_token_id` | `None` | Token ID used for padding |
| `bos_token_id` | `None` | Beginning-of-sequence token ID |
| `eos_token_id` | `None` | End-of-sequence token ID(s) |
| `forced_bos_token_id` | `None` | Token forced as first generated token |
| `forced_eos_token_id` | `None` | Token forced as last generated token |
| `cache_implementation` | `None` | KV-cache type: `"dynamic"`, `"static"`, `"quantized"` |

### 3.4 Environment Variables

All 69 recognized environment variables are listed below, organized by domain.

**Library behavior:**

| Variable | Default | Effect |
|---|---|---|
| `TRANSFORMERS_VERBOSITY` | (info) | Logging verbosity level |
| `TRANSFORMERS_NO_ADVISORY_WARNINGS` | unset | When set, suppresses non-critical advisory warnings |
| `TRANSFORMERS_DISABLE_TORCH_CHECK` | unset | Disables PyTorch version compatibility check |
| `TRANSFORMERS_IS_CI` | unset | Activates CI-specific behavior (stricter checks, etc.) |
| `HF_MODULES_CACHE` | `~/.cache/huggingface/modules` | Override directory for cached custom modules |

**XLA / TPU:**

| Variable | Default | Effect |
|---|---|---|
| `USE_TORCH_XLA` | `1` | Enable/disable PyTorch XLA backend |
| `XLA_USE_BF16` | unset | Force bfloat16 on XLA devices |
| `XLA_DOWNCAST_BF16` | unset | Downcast float32 to bfloat16 on XLA |

**Attention:**

| Variable | Default | Effect |
|---|---|---|
| `FLASH_ATTENTION_DETERMINISTIC` | `0` | Set to `1` for deterministic flash attention (slower) |

**Distributed training:**

| Variable | Default | Effect |
|---|---|---|
| `LOCAL_RANK` | `-1` | Local rank for distributed training processes |
| `ACCELERATE_MIXED_PRECISION` | `no` | AMP setting passed to Accelerate (`no`, `fp16`, `bf16`) |
| `ACCELERATE_USE_DEEPSPEED` | `false` | Enable DeepSpeed via Accelerate |
| `ACCELERATE_USE_CPU` | `False` | Force CPU execution in Accelerate |
| `ACCELERATE_TORCH_DEVICE` | `cpu` | Target device when CPU mode is active |
| `TORCHELASTIC_RUN_ID` | unset | Elastic training run identifier |

**Debugging:**

| Variable | Default | Effect |
|---|---|---|
| `NETWORK_DEBUG_REPORT` | unset | Enable network debug reporting |
| `NETWORK_DEBUG_REPORT_PATH` | unset | File path for network debug output |

**SageMaker:**

| Variable | Description |
|---|---|
| `TRAINING_JOB_ARN` | ARN of the SageMaker training job |
| `SM_FRAMEWORK_PARAMS` | Framework parameters JSON from SageMaker |
| `SM_FRAMEWORK_MODULE` | Framework module identifier |
| `AWS_REGION` | AWS region for SageMaker job |
| `SM_NUM_GPUS` | Number of GPUs visible to the container |
| `SM_NUM_CPUS` | Number of CPUs visible to the container |
| `SM_HP_MP_PARAMETERS` | SageMaker model parallelism hyperparameters |
| `SMDATAPARALLEL_LOCAL_RANK` | SageMaker data parallel local rank |

**Testing:**

| Variable | Default | Effect |
|---|---|---|
| `RUN_SLOW` | `0` | Set to `1` to include slow integration tests |
| `PYTEST_TIMEOUT` | `600` | Per-test timeout in seconds |
| `SKIP_CUDA_DOCTEST` | unset | Skip GPU-dependent doctests |

> [!IMPORTANT]
> `LOCAL_RANK` defaults to `-1`, which signals single-process mode. Multi-GPU training launched via `torchrun` or `accelerate launch` will set this automatically. Do not manually set `LOCAL_RANK` in production unless you are managing process launch yourself.

---

## 4. Deployment & Scaling

### 4.1 Installation

**Standard installation (PyTorch, CPU):**

```bash
pip install transformers
```

**With PyTorch GPU support:**

```bash
pip install transformers torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu121
```

**With all extras for training:**

```bash
pip install transformers[torch,accelerate,deepspeed,peft]
```

**From source (development):**

```bash
git clone https://github.com/huggingface/transformers
cd transformers
pip install -e ".[dev]"
```

### 4.2 Serving via CLI

The `transformers serve` command starts an OpenAI-compatible HTTP server backed by `build_server()` in `src/transformers/cli/serving/server.py`.

```bash
# Basic serve
transformers serve --model-name-or-path meta-llama/Llama-3.1-8B-Instruct

# With custom host/port
transformers serve \
  --model-name-or-path Qwen/Qwen2.5-7B-Instruct \
  --host 0.0.0.0 \
  --port 8080
```

**Available endpoints on the running server:**

| Method | Path | Description |
|---|---|---|
| `GET` | `/health` | Server health check |
| `GET` | `/v1/models` | List loaded models |
| `POST` | `/load_model` | Load a model into the manager |
| `POST` | `/reset` | Unload current model, free memory |
| `POST` | `/v1/chat/completions` | OpenAI-compatible chat completions |
| `POST` | `/v1/responses` | Response generation endpoint |
| `POST` | `/v1/audio/transcriptions` | Audio transcription (Whisper-compatible) |

All requests include an `X-Request-ID` header for tracing.

### 4.3 Distributed Training

**Using `torchrun` (PyTorch native DDP):**

```bash
torchrun --nproc_per_node=4 examples/pytorch/text-generation/run_clm.py \
  --model_name_or_path gpt2 \
  --dataset_name wikitext \
  --dataset_config_name wikitext-2-raw-v1 \
  --output_dir /tmp/clm-output \
  --per_device_train_batch_size 4 \
  --num_train_epochs 3
```

**Using Accelerate:**

```bash
accelerate config  # One-time setup
accelerate launch examples/pytorch/token-classification/run_ner.py \
  --model_name_or_path bert-base-cased \
  --output_dir /tmp/ner-output
```

**Using DeepSpeed:**

```bash
# Set ACCELERATE_USE_DEEPSPEED=true or pass --deepspeed config.json
deepspeed --num_gpus=8 examples/pytorch/text-generation/run_clm.py \
  --deepspeed ds_config_zero3.json \
  --model_name_or_path meta-llama/Llama-3.1-8B \
  --output_dir /tmp/llama-finetuned
```

### 4.4 Quantization for Deployment

Pass a quantization config to `from_pretrained()` to load a model in reduced precision:

```python
from transformers import AutoModelForCausalLM, BitsAndBytesConfig
import torch

bnb_config = BitsAndBytesConfig(
    load_in_4bit=True,
    bnb_4bit_compute_dtype=torch.bfloat16,
    bnb_4bit_use_double_quant=True,
    bnb_4bit_quant_type="nf4",
)

model = AutoModelForCausalLM.from_pretrained(
    "meta-llama/Llama-3.1-8B",
    quantization_config=bnb_config,
    device_map="auto",
)
```

All 22+ quantization backends follow the same pattern — swap the config class. See §11.3 for the complete backend list.

### 4.5 KV-Cache Strategies

| Cache type | Class | Use case |
|---|---|---|
| Dynamic (default) | `DynamicCache` | Variable-length inference, most common |
| Static | `StaticCache` | Fixed-length sequences, torch.compile-friendly |
| Encoder-decoder | `EncoderDecoderCache` | Seq2seq models (T5, BART) |
| Quantized | `QuantizedCache` | Memory reduction via weight compression in cache |
| Quanto quantized | `QuantoQuantizedLayer` | Quanto backend for cache quantization |
| HQQ quantized | `HQQQuantizedLayer` | HQQ backend for cache quantization |

Set via `GenerationConfig.cache_implementation` or pass directly to `generate()`.

### 4.6 Compile for Production Throughput

```python
import torch
model = AutoModelForCausalLM.from_pretrained("Qwen/Qwen2.5-7B")
model = torch.compile(model, mode="reduce-overhead")

# Use StaticCache to avoid recompilation on shape changes
from transformers import StaticCache
model.generation_config.cache_implementation = "static"
```

Use `CompileConfig` for fine-grained control over compilation behavior via `GenerationConfig`.

---

## 5. Operations

### 5.1 CLI Reference

The `transformers` command is the single entry point, backed by a Typer app at `src/transformers/cli/transformers.py`.

```bash
transformers --help
```

| Subcommand | Description |
|---|---|
| `serve` | Start OpenAI-compatible inference server |
| `chat` | Launch interactive chat TUI (Rich interface) |
| `download` | Download model weights and tokenizer |
| `env` | Print environment diagnostic info |
| `add-new-model-like` | Scaffold a new model from an existing one |
| `lfs-enable-largefiles` | Configure Git LFS for large model files |
| `lfs-multipart-upload` | Multipart upload via Git LFS |
| `pt-to-tf` | Convert PyTorch weights to TensorFlow format |

### 5.2 Interactive Chat

```bash
transformers chat --model-name-or-path Qwen/Qwen2.5-7B-Instruct
```

The chat command uses `RichInterface` from `src/transformers/cli/chat.py` to render a terminal TUI. Requires `rich` to be installed.

### 5.3 Environment Diagnostics

```bash
transformers env
```

Prints installed versions of Python, PyTorch, TensorFlow, JAX, Transformers, Tokenizers, and relevant system info. Always run this first when filing a bug report.

### 5.4 Hub Operations

```bash
# Download a model
transformers download meta-llama/Llama-3.1-8B --cache-dir /data/hf-cache

# Enable Git LFS tracking for .bin and .safetensors
transformers lfs-enable-largefiles .
```

### 5.5 Model Conversion

```bash
# Convert PyTorch checkpoint to TensorFlow
transformers pt-to-tf \
  --model-name-or-path bert-base-uncased \
  --local-dir ./tf-bert
```

### 5.6 Continuous Batching

For high-throughput serving, `ContinuousBatchingManager` (in `src/transformers/generation/continuous_batching/`) implements iteration-level batching — new requests are inserted into an in-flight batch at each decoding step, avoiding the padding waste of static batching.

```python
from transformers.generation import ContinuousBatchingConfig

gen_config = GenerationConfig(
    continuous_batching=ContinuousBatchingConfig(max_batch_size=32)
)
```

> [!NOTE]
> Continuous batching is most effective when combined with a static KV-cache (`cache_implementation="static"`) and `torch.compile`. It is not compatible with beam search (`num_beams > 1`).

### 5.7 Monitoring and Logging

Set the verbosity level via environment variable or programmatically:

```bash
export TRANSFORMERS_VERBOSITY=debug
```

```python
import transformers
transformers.logging.set_verbosity_debug()
transformers.logging.enable_progress_bar()
```

Available levels (lowest to highest verbosity): `critical`, `error`, `warning`, `info`, `debug`.

To suppress the common advisory warnings without changing log level:

```bash
export TRANSFORMERS_NO_ADVISORY_WARNINGS=1
```

> [!WARNING]
> Setting `TRANSFORMERS_VERBOSITY=debug` in production will produce extremely high log volume including per-tensor shape information during model loading. Reserve debug level for isolated troubleshooting sessions only.

> [!CAUTION]
> Enabling `NETWORK_DEBUG_REPORT` logs all outbound HTTP requests made by the library, which may include authentication tokens sent to the Hub. Ensure the `NETWORK_DEBUG_REPORT_PATH` output file is protected with appropriate filesystem permissions and is excluded from log aggregation pipelines.

---

## 6. Performance

### 6.1 Inference Optimization Checklist

| Technique | How to enable | Benefit |
|---|---|---|
| `device_map="auto"` | Pass to `from_pretrained()` | Automatically spreads layers across available GPUs |
| `torch_dtype=torch.bfloat16` | Pass to `from_pretrained()` | Halves memory, near-identical quality on modern hardware |
| `attn_implementation="flash_attention_2"` | Pass to `from_pretrained()` | 2–4× faster attention on long sequences |
| `attn_implementation="sdpa"` | Pass to `from_pretrained()` | PyTorch scaled-dot-product attention (no extra install) |
| `torch.compile()` | Wrap model after loading | Reduces Python overhead on repeated calls |
| Static KV-cache | `cache_implementation="static"` | Enables graph capture with `torch.compile` |
| 4-bit quantization | `BitsAndBytesConfig(load_in_4bit=True)` | ~75% memory reduction vs. float32 |
| 8-bit quantization | `BitsAndBytesConfig(load_in_8bit=True)` | ~50% memory reduction vs. float32 |

### 6.2 Training Optimization

**Gradient checkpointing** (trades compute for memory):

```python
training_args = TrainingArguments(
    output_dir="./output",
    gradient_checkpointing=True,
    gradient_accumulation_steps=8,
    per_device_train_batch_size=2,
    bf16=True,
)
```

**Flash attention during training:**

```python
model = AutoModelForCausalLM.from_pretrained(
    "meta-llama/Llama-3.1-8B",
    attn_implementation="flash_attention_2",
    torch_dtype=torch.bfloat16,
)
```

**FSDP (Fully Sharded Data Parallel):**

```python
training_args = TrainingArguments(
    output_dir="./output",
    fsdp="full_shard auto_wrap",
    fsdp_config={"transformer_layer_cls_to_wrap": ["LlamaDecoderLayer"]},
)
```

### 6.3 Determinism

For reproducible results:

```bash
export FLASH_ATTENTION_DETERMINISTIC=1
```

Note that deterministic flash attention disables the non-deterministic tiling optimizations and will reduce throughput.

### 6.4 LR Scheduler Selection

| Scheduler | Function | Best for |
|---|---|---|
| Linear with warmup | `get_linear_schedule_with_warmup` | General fine-tuning |
| Cosine with warmup | `get_cosine_schedule_with_warmup` | Pre-training, long runs |
| Cosine with hard restarts | `get_cosine_with_hard_restarts_schedule_with_warmup` | Cyclical fine-tuning |
| Cosine with min LR | `get_cosine_with_min_lr_schedule_with_warmup` | When a floor LR is desired |
| Polynomial decay | `get_polynomial_decay_schedule_with_warmup` | Flexible decay curve |
| Constant | `get_constant_schedule` | Debugging, short runs |
| Constant with warmup | `get_constant_schedule_with_warmup` | Stable mid-training |
| Inverse square root | `get_inverse_sqrt_schedule` | Transformer pre-training (original schedule) |
| Reduce on plateau | `get_reduce_on_plateau_schedule` | Metric-adaptive reduction |
| WSD | `get_wsd_schedule` | Warmup-stable-decay cycles |
| Greedy | `get_greedy_schedule` | Greedy LR selection |

Adafactor optimizer is also exported from `transformers.optimization` as a memory-efficient Adam alternative for large-scale training.

### 6.5 Benchmarking

The `benchmark_v2/` directory contains a dedicated benchmarking framework for measuring inference and training throughput. Use this to establish baselines before and after optimization changes.

---

## 7. Security

> [!WARNING]
> Model weights loaded from untrusted sources can execute arbitrary code during deserialization when using the legacy `.bin` (pickle) format. **Always prefer `.safetensors` format** for models from untrusted or third-party sources. The `safetensors` library (required dependency `>=0.4.3`) provides a safe zero-copy loading path with no deserialization code execution.

> [!CAUTION]
> The `trust_remote_code=True` parameter on `from_pretrained()` and `pipeline()` executes Python files downloaded from the model hub inside your process. Never set `trust_remote_code=True` for models from unverified publishers. Only use it for well-known model families from verified organizations when no native Transformers implementation exists.

### 7.1 Model Weight Security

| Format | Safety level | Notes |
|---|---|---|
| `.safetensors` | Safe | No code execution, memory-mapped, default for new uploads |
| `.bin` (pickle) | Unsafe from untrusted sources | Legacy format; can embed arbitrary Python via pickle |
| `.h5` (TF) | Conditionally safe | TF's SavedModel format carries similar risks |

Force safetensors-only loading:

```python
model = AutoModelForCausalLM.from_pretrained(
    "org/model",
    use_safetensors=True,  # Raises error if safetensors not available
)
```

### 7.2 Hub Authentication

The library uses `huggingface-hub` for all Hub operations. Credentials are stored in `~/.cache/huggingface/token` (or `HF_HOME`). Never commit this file or expose the token in environment variables accessible to untrusted code.

```bash
# Login
huggingface-cli login

# Or set token explicitly (avoid in shared environments)
export HF_TOKEN=hf_...
```

### 7.3 Serving Security

When running `transformers serve` on a network-accessible interface:

- The server has no built-in authentication. Place it behind a reverse proxy (nginx, Traefik) with auth middleware for any non-localhost deployment.
- Enable CORS middleware only for known origins.
- The `/load_model` and `/reset` endpoints can load arbitrary model names from the Hub — restrict access to these endpoints in multi-tenant environments.
- All requests include `X-Request-ID` for audit logging at the proxy layer.

### 7.4 Watermarking

For detecting AI-generated text, the generation subsystem includes two watermarking implementations:

| Component | Type |
|---|---|
| `WatermarkLogitsProcessor` | Generation-time watermark injection |
| `WatermarkDetector` | Post-hoc detection |
| `SynthIDTextWatermarkLogitsProcessor` | Google SynthID watermark injection |
| `SynthIDTextWatermarkDetector` | SynthID detection |
| `BayesianDetectorModel` + `BayesianDetectorConfig` | Bayesian statistical detection |
| `SynthIDTextWatermarkingConfig` | SynthID configuration |

Configure via `WatermarkingConfig` in `GenerationConfig`.

---

## 8. Developer Internals

### 8.1 Adding a New Model

The recommended workflow uses the modular model system:

```bash
transformers add-new-model-like <existing_model_name>
```

This invokes `src/transformers/cli/add_new_model_like.py` (`ModelInfos` class), which:
1. Prompts for the new model name and its base architecture
2. Creates `src/transformers/models/{new_model}/modular_{new_model}.py`
3. Scaffolds the configuration, modeling, tokenizer, and processing files by copying the base model structure

After implementing your changes in the modular file, convert to standalone:

```bash
python utils/modular_model_converter.py --model_type {new_model}
```

Register the model in Auto classes by adding entries to `CONFIG_MAPPING_NAMES` and the relevant `MODEL_FOR_*_MAPPING_NAMES` in `src/transformers/models/auto/auto_mappings.py`.

### 8.2 Repository Maintenance Tools (`utils/`)

The `utils/` directory (64 files) contains tooling for maintaining the repository's internal consistency. These are run in CI and can be run locally:

| Script | Purpose |
|---|---|
| `modular_model_converter.py` | Convert `modular_*.py` → standalone model files |
| `create_dummy_models.py` | Create tiny randomly-initialized models for CI testing |
| `check_copies.py` | Verify that code marked with `# Copied from` comments matches the source |
| `check_auto.py` | Verify Auto class registries are consistent |
| `check_repo.py` | General repository consistency checks |
| `tests_fetcher.py` | Determine which tests to run from a git diff (CI optimization) |
| `checkers.py` | Unified checker runner with result caching |
| `mlinter/` | Custom modeling linter enforcing TRF rules |

**Running consistency checks:**

```bash
python utils/check_repo.py
python utils/check_copies.py
python utils/check_auto.py
```

### 8.3 `PreTrainedModel` Base Class

All PyTorch models inherit from `PreTrainedModel` (`src/transformers/modeling_utils.py`). Key methods:

| Method | Description |
|---|---|
| `from_pretrained()` | Class method; download weights, apply quantization, map devices |
| `save_pretrained()` | Save weights and config to directory |
| `push_to_hub()` | Upload to Hugging Face Hub |
| `resize_token_embeddings()` | Extend/shrink embedding matrix after tokenizer changes |
| `tie_weights()` | Tie input/output embeddings (called automatically) |
| `gradient_checkpointing_enable()` | Enable activation checkpointing |
| `get_memory_footprint()` | Return model size in bytes |
| `generate()` | Inherited from `GenerationMixin`; full text generation |

### 8.4 Implementing a Custom Logits Processor

Logits processors allow arbitrary transforms to the output distribution at each generation step. Implement the `LogitsProcessor` interface:

```python
from transformers.generation import LogitsProcessor
import torch

class BlockTokenProcessor(LogitsProcessor):
    def __init__(self, blocked_token_ids: list[int]):
        self.blocked = blocked_token_ids

    def __call__(
        self, input_ids: torch.LongTensor, scores: torch.FloatTensor
    ) -> torch.FloatTensor:
        scores[:, self.blocked] = float("-inf")
        return scores

# Use in generation
from transformers import LogitsProcessorList
processor_list = LogitsProcessorList([BlockTokenProcessor([1234, 5678])])
model.generate(inputs, logits_processor=processor_list)
```

### 8.5 Implementing a Custom Stopping Criterion

```python
from transformers.generation import StoppingCriteria

class KeywordCriteria(StoppingCriteria):
    def __init__(self, keyword_ids: list[int], tokenizer):
        self.keyword_ids = keyword_ids

    def __call__(self, input_ids: torch.LongTensor, scores: torch.FloatTensor) -> bool:
        for kw_id in self.keyword_ids:
            if kw_id in input_ids[0][-5:]:
                return True
        return False
```

### 8.6 Custom Trainer Callbacks

The `Trainer` callback system allows hooking into training events:

```python
from transformers import TrainerCallback

class EarlyStoppingCallback(TrainerCallback):
    def __init__(self, patience: int = 3):
        self.patience = patience
        self._no_improve = 0

    def on_evaluate(self, args, state, control, metrics, **kwargs):
        metric = metrics.get("eval_loss")
        # Custom logic here
        if self._no_improve >= self.patience:
            control.should_training_stop = True
        return control
```

### 8.7 The `mlinter` Modeling Linter

`utils/mlinter/` implements a custom linter enforcing TRF (Transformers Rules Framework) rules specific to model implementation patterns. Run it before submitting model PRs:

```bash
python utils/mlinter/run_mlinter.py src/transformers/models/your_model/
```

### 8.8 Testing Conventions

The test suite in `tests/` is pytest-based with extensive custom infrastructure in `testing_utils.py`:

**Custom decorators:**

| Decorator | Skips when |
|---|---|
| `@require_torch` | PyTorch not installed |
| `@require_vision` | Vision libraries not installed |
| `@require_tf` | TensorFlow not installed |
| `@require_flax` | JAX/Flax not installed |
| `@slow` | `RUN_SLOW != "1"` |

**Running tests:**

```bash
# All fast tests
pytest tests/ -x

# Include slow tests
RUN_SLOW=1 pytest tests/

# Specific model
pytest tests/models/bert/ -v

# Pipeline tests
pytest tests/pipelines/ -v

# With timeout override
PYTEST_TIMEOUT=1200 pytest tests/models/llama/
```

**Generating dummy models for testing:**

```bash
python utils/create_dummy_models.py --model_type bert --output_dir /tmp/dummy-bert
```

**CI-optimized test selection (run only tests affected by a diff):**

```bash
python utils/tests_fetcher.py --diff_with_last_commit
```

---

## 9. Integration Points

### 9.1 Logging Integrations (via Trainer)

Set `report_to` in `TrainingArguments` to enable:

| Integration | `report_to` value | Notes |
|---|---|---|
| TensorBoard | `"tensorboard"` | Built into PyTorch |
| Weights & Biases | `"wandb"` | `pip install wandb` |
| MLflow | `"mlflow"` | `pip install mlflow` |
| Comet ML | `"comet_ml"` | `pip install comet_ml` |
| DagsHub | `"dagshub"` | |
| ClearML | `"clearml"` | |
| All enabled | `"all"` | Default |
| None | `"none"` | Disable all logging |

### 9.2 PEFT Integration

Parameter-efficient fine-tuning via the `peft` library integrates directly with `Trainer`:

```python
from peft import get_peft_model, LoraConfig, TaskType

peft_config = LoraConfig(
    task_type=TaskType.CAUSAL_LM,
    inference_mode=False,
    r=8,
    lora_alpha=32,
    lora_dropout=0.1,
)
model = get_peft_model(model, peft_config)

trainer = Trainer(
    model=model,
    args=training_args,
    train_dataset=train_dataset,
)
```

### 9.3 ExecuTorch Integration

`src/transformers/integrations/` includes ExecuTorch integration for on-device inference export. This enables exporting Transformers models to the ExecuTorch runtime for mobile and edge deployment.

### 9.4 SageMaker

The library detects SageMaker environments via the `SM_*` environment variables (see §3.4). The `Trainer` automatically adapts to SageMaker distributed training when `TRAINING_JOB_ARN` is set.

```bash
# SageMaker sets these automatically in training containers
TRAINING_JOB_ARN=arn:aws:sagemaker:us-east-1:123456789:training-job/my-job
SM_NUM_GPUS=8
SM_NUM_CPUS=32
```

### 9.5 Hub Webhooks and Push

```python
# Push model after training
training_args = TrainingArguments(
    output_dir="./output",
    push_to_hub=True,
    hub_model_id="your-org/your-model-name",
    hub_strategy="every_save",  # or "end", "checkpoint", "all_checkpoints"
)
```

### 9.6 OpenAI-Compatible API Clients

When running `transformers serve`, any client that speaks the OpenAI API format can connect:

```python
from openai import OpenAI

client = OpenAI(
    base_url="http://localhost:8000/v1",
    api_key="not-required",  # Server has no auth by default
)

response = client.chat.completions.create(
    model="loaded-model",
    messages=[{"role": "user", "content": "Hello!"}],
)
```

---

## 10. Troubleshooting

### 10.1 CUDA Out of Memory

**Error:**

```
torch.OutOfMemoryError: CUDA out of memory. Tried to allocate X GiB...
```

**Cause:**

Model size exceeds available GPU VRAM, often compounded by KV-cache growth during generation or large batch sizes.

**Solution:**

```python
# 1. Use quantization to reduce model footprint
model = AutoModelForCausalLM.from_pretrained(
    model_id,
    quantization_config=BitsAndBytesConfig(load_in_4bit=True),
    device_map="auto",
)

# 2. Enable gradient checkpointing for training
training_args = TrainingArguments(gradient_checkpointing=True)

# 3. Reduce batch size and accumulate gradients
training_args = TrainingArguments(
    per_device_train_batch_size=1,
    gradient_accumulation_steps=16,
)

# 4. Free cache between inferences
import torch
torch.cuda.empty_cache()
```

**Prevention:**

Estimate memory requirements before loading: a float16 model uses ~2 bytes/parameter. A 7B model needs ~14 GB; use 4-bit quantization to bring this to ~4 GB.

---

### 10.2 Multi-GPU Cross-Device Tensor Errors

**Error:**

```
RuntimeError: Expected all tensors to be on the same device...
```

**Cause:**

With `device_map="auto"` and CPU offload, tensors may be on different devices. This is the root cause of issues like GitHub #45482 (Gemma4 with CPU offload).

**Solution:**

```python
# Explicitly control layer placement
from accelerate import dispatch_model, infer_auto_device_map

device_map = infer_auto_device_map(
    model,
    max_memory={0: "20GiB", 1: "20GiB", "cpu": "30GiB"},
    no_split_module_classes=["LlamaDecoderLayer"],
)
```

**Prevention:**

When using CPU offload (`device_map="auto"` with limited GPU memory), test the specific model architecture with your hardware configuration before production deployment.

---

### 10.3 `from_pretrained` Fails with `KeyError` on Model Type

**Error:**

```
KeyError: 'qwen3_moe' is not a supported model type.
```

**Cause:**

The installed version of Transformers does not include the model architecture. This is the root cause of GitHub #45478 (Qwen3.5 Moe on `>=5.4.0` in older installs).

**Solution:**

```bash
# Update to latest version
pip install --upgrade transformers

# Or install from source for bleeding-edge models
pip install git+https://github.com/huggingface/transformers
```

**Prevention:**

Pin `transformers` version in deployment manifests and test model loading as part of your CI pipeline.

---

### 10.4 `tokenizers` Version Conflict

**Error:**

```
ImportError: cannot import name 'XTokenizer' from 'tokenizers'
```

or silent behavioral differences in fast tokenizers.

**Cause:**

`tokenizers` is bounded at `<=0.23.0`. A newer version may introduce breaking API changes or behavioral differences.

**Solution:**

```bash
pip install "tokenizers>=0.22.0,<=0.23.0" --force-reinstall
```

**Prevention:**

Lock `tokenizers` version explicitly in `requirements.txt` or `pyproject.toml`.

---

### 10.5 `No module named transformers.onnx`

**Error:**

```
ModuleNotFoundError: No module named 'transformers.onnx'
```

**Cause:**

The `transformers.onnx` submodule was removed from the library (GitHub #22355, closed). ONNX export is now handled by the `optimum` library.

**Solution:**

```bash
pip install optimum[onnxruntime]
optimum-cli export onnx --model bert-base-uncased ./bert-onnx
```

**Prevention:**

Migrate all `transformers.onnx` usages to `optimum` before updating Transformers.

---

### 10.6 Zero Loss with `num_labels=1` for Single-Label Classification

**Error:**

Training loss stays at `0.0` for sequence classification with `num_labels=1`.

**Cause:**

GitHub #45479: the `single_label_classification` problem type with `num_labels=1` produces zero loss due to how the loss function is selected.

**Solution:**

Use `num_labels=2` for binary classification and encode labels as `0`/`1`, or set `problem_type="regression"` if your task is truly single-output regression.

**Prevention:**

When working with custom classification heads, verify loss values are non-zero in the first training step.

---

### 10.7 Stuck/Hanging Distributed Training

**Error:**

Training hangs indefinitely, no progress after a few steps.

**Cause:**

NCCL collective operation timeout, typically caused by a GPU process falling behind (OOM on one rank, uneven data distribution, or a dead process in the group).

**Solution:**

```bash
# Enable NCCL debug output
export NCCL_DEBUG=INFO
export NCCL_DEBUG_SUBSYS=ALL

# Check which rank is hanging
export NETWORK_DEBUG_REPORT=1
export NETWORK_DEBUG_REPORT_PATH=/tmp/network_debug.log
```

**Prevention:**

Set `NCCL_TIMEOUT` (via Accelerate config) to a value that allows catching hangs without waiting indefinitely. Use `--max_steps` with periodic checkpointing so partial progress is not lost.

---

## 11. Reference

### 11.1 Complete Logits Processors

| Class | Effect |
|---|---|
| `TemperatureLogitsWarper` | Divide logits by temperature before softmax |
| `TopKLogitsWarper` | Zero probability for all tokens outside top-K |
| `TopPLogitsWarper` | Nucleus sampling — keep top tokens summing to P |
| `MinPLogitsWarper` | Filter tokens below P × max probability |
| `TopHLogitsWarper` | Top-H (entropy-based) filtering |
| `TypicalLogitsWarper` | Typical decoding filter |
| `EpsilonLogitsWarper` | Filter tokens below absolute epsilon probability |
| `EtaLogitsWarper` | Adaptive epsilon based on entropy |
| `RepetitionPenaltyLogitsProcessor` | Penalize tokens already in context |
| `NoRepeatNGramLogitsProcessor` | Hard-block repeated n-grams |
| `NoBadWordsLogitsProcessor` | Hard-block specific token sequences |
| `MinLengthLogitsProcessor` | Prevent EOS before minimum length |
| `MinNewTokensLengthLogitsProcessor` | Prevent EOS before minimum new tokens |
| `SequenceBiasLogitsProcessor` | Add/subtract bias for specific sequences |
| `PrefixConstrainedLogitsProcessor` | Constrain output to a prefix tree |
| `ForcedBOSTokenLogitsProcessor` | Force a specific first token |
| `ForcedEOSTokenLogitsProcessor` | Force a specific last token |
| `InfNanRemoveLogitsProcessor` | Replace inf/nan with safe values |
| `LogitNormalization` | Normalize logits to sum to 1 |
| `SuppressTokensLogitsProcessor` | Suppress a set of token IDs |
| `SuppressTokensAtBeginLogitsProcessor` | Suppress tokens only at sequence start |
| `WhisperTimeStampLogitsProcessor` | Timestamp token handling for Whisper |
| `ClassifierFreeGuidanceLogitsProcessor` | CFG blending for guided generation |
| `UnbatchedClassifierFreeGuidanceLogitsProcessor` | CFG for unbatched inputs |
| `AlternatingCodebooksLogitsProcessor` | Alternating codebook constraint (audio codecs) |

### 11.2 Stopping Criteria

| Class | Stops when |
|---|---|
| `MaxLengthCriteria` | Sequence reaches `max_length` total tokens |
| `MaxTimeCriteria` | Wall-clock time exceeds budget |
| `EosTokenCriteria` | EOS token generated |
| `StopStringCriteria` | Decoded text contains a stop string |

### 11.3 Quantization Backends

| Config class | Backend | Precision |
|---|---|---|
| `BitsAndBytesConfig` | bitsandbytes | 4-bit, 8-bit |
| `GPTQConfig` | AutoGPTQ / Optimum | 2/3/4/8-bit GPTQ |
| `AwqConfig` | AutoAWQ | 4-bit AWQ |
| `HqqConfig` | Half-Quadratic Quantization | 1–8 bit |
| `QuantoConfig` | Quanto | 2/4/8-bit |
| `AqlmConfig` | AQLM | 1–2 bit additive quantization |
| `AutoRoundConfig` | AutoRound | GPTQ variant |
| `CompressedTensorsConfig` | Neural Magic | Mixed precision |
| `EetqConfig` | EETQ | 8-bit |
| `FbgemmFp8Config` | FBGEMM FP8 | FP8 |
| `FineGrainedFP8Config` | Fine-grained FP8 | FP8 |
| `FourOverSixConfig` | 4-over-6 | 4/6-bit |
| `FPQuantConfig` | FP quantization | FP variants |
| `HiggsConfig` | HIGGS | — |
| `MetalConfig` | Metal (Apple) | — |
| `Mxfp4Config` | MXFP4 microscaling | FP4 |
| `QuarkConfig` | Quark | — |
| `SinqConfig` | SINQ | — |
| `SpQRConfig` | SpQR | Mixed precision |
| `TorchAoConfig` | torchao | Various |
| `VptqConfig` | VPTQ | — |
| `BitNetQuantConfig` | BitNet | 1-bit |

### 11.4 Cache Classes

| Class | Best for |
|---|---|
| `DynamicCache` | Default; variable-length sequences |
| `StaticCache` | `torch.compile`; fixed max length |
| `EncoderDecoderCache` | T5, BART, Whisper |
| `QuantizedCache` | Memory-constrained long-context inference |
| `QuantoQuantizedLayer` | Quanto-backed cache quantization |
| `HQQQuantizedLayer` | HQQ-backed cache quantization |

### 11.5 Key Source File Index

| File | Description |
|---|---|
| `src/transformers/__init__.py` | Lazy import structure, version `5.6.0.dev0` |
| `src/transformers/modeling_utils.py` | `PreTrainedModel` base class |
| `src/transformers/configuration_utils.py` | `PreTrainedConfig` base class |
| `src/transformers/trainer.py` | `Trainer` class |
| `src/transformers/training_args.py` | `TrainingArguments` dataclass |
| `src/transformers/generation/` | `GenerationMixin`, `GenerationConfig`, processors |
| `src/transformers/pipelines/__init__.py` | `pipeline()` factory |
| `src/transformers/models/auto/auto_mappings.py` | Auto class registries |
| `src/transformers/cli/transformers.py` | CLI entry point (Typer app) |
| `src/transformers/cli/serving/server.py` | `build_server()` FastAPI factory |
| `src/transformers/cli/chat.py` | `RichInterface` TUI |
| `src/transformers/cli/add_new_model_like.py` | `ModelInfos`, model scaffolding |
| `src/transformers/quantizers/` | 22+ quantization backend implementations |
| `src/transformers/utils/` | 39 utility modules (hub, logging, import utils) |
| `utils/modular_model_converter.py` | Convert modular → standalone model files |
| `utils/check_copies.py` | Copied-from comment consistency |
| `utils/check_auto.py` | Auto registry consistency |
| `utils/tests_fetcher.py` | CI test selection from git diff |
| `utils/mlinter/` | TRF modeling linter |
| `setup.py` | All dependencies, extras, entry points |

### 11.6 Entry Points

| Command | Module path | Description |
|---|---|---|
| `transformers` | `transformers.cli.transformers:main` | Primary CLI |

### 11.7 Package Metadata

| Field | Value |
|---|---|
| Version | `5.6.0.dev0` |
| License | Apache 2.0 |
| Source commit | `71560f4` (2026-04-17) |
| Repository | `huggingface/transformers` |
| Python support | 3.10 – 3.14 |
| Codebase size | 5,695 files, ~14M source characters |

---

_Generated May 2026 from huggingface/transformers at 71560f4. AI-generated documentation. The library is checked by sampling (three repositories so far); this guide was not individually reviewed. Verify against the project before relying on it. Unofficial; not affiliated with or reviewed by the project._