# TextGen (text-generation-webui) User Guide

TextGen is a self-hosted web interface for running large language models on your own
computer. You point it at a model file, pick how you want to talk to it — a chat window, a
freeform notebook, or an OpenAI-compatible API — and it handles loading the model, managing
context, sampling, and streaming the reply. It runs locally with no account, no cloud
dependency, and no telemetry.

A single install can load models in five different runtimes (llama.cpp, Transformers,
ExLlamaV3, and TensorRT-LLM), generate images, transcribe audio, search the web mid-chat,
call tools over MCP, and serve the whole thing to other applications through a drop-in
replacement for the OpenAI and Anthropic APIs.

> [!NOTE]
> This guide covers what TextGen does and how to use it day to day. For architecture,
> the full configuration reference, deployment, and security hardening, see the companion
> **Admin Guide**.

---

## 1. Quick Start

The fastest path is a **portable build** — a self-contained zip that needs no installation
and runs models through llama.cpp. It opens as a desktop app.

=== "Windows"

    1. Download the latest `textgen-portable-*-windows-*.zip` from the project's releases.
    2. Extract it anywhere **without spaces in the path**.
    3. Double-click `start_windows.bat` (or the bundled desktop launcher).

=== "Linux"

    ```bash
    unzip textgen-portable-*-linux-*.zip
    cd textgen-portable-*
    ./start_linux.sh
    ```

=== "macOS"

    ```bash
    unzip textgen-portable-*-macos-*.zip
    cd textgen-portable-*
    ./start_macos.sh
    ```

On first launch TextGen tells you that no model is downloaded yet:

```text
You haven't downloaded any model yet.
Once the web UI launches, head over to the "Model" tab and download one.
```

Open the **Model** tab, paste a Hugging Face model ID (for example
`bartowski/Llama-3.2-3B-Instruct-GGUF`) into the download box, download it, select it from
the dropdown, and click **Load**. Switch to the **Chat** tab and start typing.

> [!TIP]
> Portable builds run **GGUF** models through llama.cpp only. If you want Transformers,
> ExLlamaV3, image generation, or LoRA training, use the one-click installer instead
> (see [Installation](#3-installation)).

---

## 2. Requirements

| Requirement | Detail |
|---|---|
| **OS** | Windows, Linux, or macOS |
| **Python** | 3.13 (provided automatically by portable builds and the one-click installer) |
| **GPU (recommended)** | NVIDIA (CUDA 12.8), AMD (ROCm 7.2, Linux only), Apple M-series, or Intel Arc (beta) |
| **CPU-only** | Supported — pass `--cpu`; expect slow generation |
| **Disk** | Enough for your models (a 7B GGUF Q4 model is ~4 GB; full-precision models are far larger) |
| **RAM / VRAM** | Must fit the model plus its KV cache; see the model's own size |

> [!IMPORTANT]
> A GPU is strongly recommended but not required. TextGen can offload some or all model
> layers to the GPU and keep the rest in CPU RAM, so you can run models larger than your
> VRAM — just more slowly.

---

## 3. Installation

TextGen offers three installation paths. Pick the one that matches how much you need.

### 3.1 Portable build (no install)

Already covered in [Quick Start](#1-quick-start). Best for: trying TextGen quickly, running
GGUF models, machines where you don't want a multi-gigabyte Python environment.

**Capability trade-off:** portable builds run with `--portable`, which hides features that
aren't available in that mode:

| Feature | Portable build | Full install |
|---|---|---|
| llama.cpp (GGUF) | ✅ | ✅ |
| Transformers / ExLlamaV3 / TensorRT-LLM loaders | ❌ | ✅ |
| Image generation tab | ❌ | ✅ |
| LoRA training tab | ❌ | ✅ |
| OpenAI/Anthropic API server | ✅ | ✅ |
| Extensions | ✅ | ✅ |
| Desktop (Electron) window | ✅ | optional |

### 3.2 One-click installer (full features)

The one-click installer downloads a private Python 3.13 environment via Miniforge/conda,
installs PyTorch for your hardware, and sets everything up. It supports every loader and
feature.

1. Download or `git clone` the repository.
2. Run the start script for your OS:

=== "Windows"

    Double-click `start_windows.bat`.

=== "Linux"

    ```bash
    ./start_linux.sh
    ```

=== "macOS"

    ```bash
    ./start_macos.sh
    ```

=== "WSL"

    Double-click `start_wsl.bat`.

3. On first run you'll be asked which hardware you have:

```text
What is your GPU?

A) NVIDIA
B) AMD - Linux only, ROCm 7.2
C) Apple M Series
D) Intel Arc (beta)
N) CPU mode
```

Pick the letter that matches your machine. The installer remembers your choice in
`.installer_state.json` and installs the matching requirements
(`requirements/full/requirements_*.txt`). When it finishes, the web UI launches automatically.

> [!TIP]
> To pre-select your GPU non-interactively (useful for scripted installs), set the
> `GPU_CHOICE` environment variable to `A`, `B`, `C`, `D`, or `N` before running the start
> script.

> [!CAUTION]
> The conda-based installer relies on Miniforge, which **cannot be installed under a path
> containing spaces**. Install TextGen into a path like `C:\textgen`, not
> `C:\My Programs\textgen`.

### 3.3 Updating

Run the update wizard for your OS (`update_wizard_windows.bat`, `update_wizard_linux.sh`,
`update_wizard_macos.sh`, or `update_wizard_wsl.bat`). It opens a menu:

```text
What would you like to do?

A) Update the web UI
B) Install/update extensions requirements
C) Revert local changes to repository files with "git reset --hard"
N) Nothing (exit)
```

Choose **A** to pull the latest release and refresh dependencies.

> [!WARNING]
> Major Python or CUDA version bumps require a **clean reinstall**. If you see a message
> instructing you to delete the `installer_files` folder, do so and re-run the start script —
> the installer cannot upgrade across those boundaries in place.

---

## 4. Downloading Models

You have two ways to get models: the **Model** tab in the UI, or the `download-model.py`
command-line tool. Both save into `user_data/models/` (LoRAs go to `user_data/loras/`).

### 4.1 From the Model tab

Open the **Model** tab, paste a Hugging Face model ID into the download field, and download.
TextGen automatically picks sensible files:

- If both PyTorch and `safetensors` weights exist, it downloads **safetensors only**.
- For **GGUF** repositories with many quantizations, it downloads the **Q4_K_M** file by
  default (a good size/quality balance) unless you specify a particular file.

### 4.2 From the command line

```bash
python download-model.py facebook/opt-1.3b
```

```text
Downloading the model to user_data/models/facebook_opt-1.3b
Download of 5 files to user_data/models/facebook_opt-1.3b completed.
```

**Download a specific GGUF quantization** (recommended for large GGUF repos):

```bash
python download-model.py bartowski/Llama-3.2-3B-Instruct-GGUF --specific-file Llama-3.2-3B-Instruct-Q4_K_M.gguf
```

**Common `download-model.py` options:**

| Option | Default | Purpose |
|---|---|---|
| `MODEL` | — | Hugging Face model ID (`owner/name`) or a direct `…/resolve/…` file URL |
| `--branch` | `main` | Git branch to download from |
| `--threads` | `4` | Number of files to download in parallel |
| `--specific-file` | — | Download just one file (e.g. a single GGUF) |
| `--exclude-pattern` | — | Regex of filenames to skip |
| `--text-only` | off | Download only text files (`txt`/`json`), not weights |
| `--output` | — | Save into this exact folder |
| `--model-dir` | — | Save into a subfolder of this directory instead of `user_data/models` |
| `--clean` | off | Restart the download instead of resuming |
| `--check` | off | Validate SHA-256 checksums of already-downloaded files |
| `--max-retries` | `7` | Retry attempts per file on network errors |

> [!TIP]
> For **gated or private** Hugging Face models, set `HF_TOKEN` (or `HF_USER` / `HF_PASS`)
> in your environment before downloading. You can also point `HF_ENDPOINT` at a mirror.

> [!NOTE]
> Downloads resume automatically if interrupted — re-run the same command. Use `--check`
> afterward to confirm every file's checksum matches Hugging Face, and `--clean` to force a
> fresh download if a checksum fails.

---

## 5. Loading a Model

Open the **Model** tab, choose your model from the dropdown, and click **Load**. TextGen
auto-detects the right loader from the files present, but you can override it.

### 5.1 The five loaders

| Loader | Use it for | Notes |
|---|---|---|
| **llama.cpp** | GGUF files | Default; runs a `llama-server` subprocess; supports CPU+GPU split, vision models, speculative decoding. The only loader in portable builds. |
| **Transformers** | Hugging Face `safetensors`/PyTorch models | Most compatible; supports 8-bit/4-bit (bitsandbytes) quantization, CPU/disk offload |
| **ExLlamav3_HF** | EXL3 quantized models | Fast GPU inference with full Transformers sampler support |
| **ExLlamav3** | EXL3 quantized models | ExLlamaV3's own sampler path; supports speculative decoding |
| **TensorRT-LLM** | TensorRT-LLM engines | Highest throughput on NVIDIA; requires a separately built engine |

> [!TIP]
> If you're unsure, just download a **GGUF** model and let the default **llama.cpp** loader
> handle it. It works on CPU, NVIDIA, AMD, Apple, and Intel hardware.

### 5.2 Fitting a model to your hardware

The most important load-time settings (all on the Model tab, and all available as CLI flags):

| Setting | Flag | What it does |
|---|---|---|
| **GPU layers** | `--gpu-layers N` | How many model layers to put on the GPU. `-1` = auto-fit. Lower it if you run out of VRAM. |
| **Context size** | `--ctx-size N` | Maximum tokens of context. `0` = auto for llama.cpp. Larger = more memory. |
| **Cache type** | `--cache-type fp16` | KV-cache precision. `q8_0`/`q4_0` shrink memory at a small quality cost. |
| **CPU offload (MoE)** | `--cpu-moe` | Keep mixture-of-experts weights on the CPU to save VRAM. |
| **Tensor split** | `--tensor-split 60,40` | Distribute a model across multiple GPUs by proportion. |

> [!WARNING]
> If loading fails with an out-of-memory error, reduce `--gpu-layers`, lower `--ctx-size`,
> or switch the cache type to `q8_0`. See [Troubleshooting](#12-troubleshooting).

### 5.3 Idle unloading

Pass `--idle-timeout 30` to automatically unload the model after 30 minutes of inactivity
(freeing VRAM); it reloads automatically on your next request.

---

## 6. Chatting With a Model

The **Chat** tab is the main way to talk to a model. Type in the input box at the bottom and
press **Enter** to send.

### 6.1 Chat modes

TextGen has three conversation modes (set in the chat controls or via the `mode` setting):

| Mode | Behavior |
|---|---|
| **instruct** | Plain assistant chat using the model's instruction template. The default. |
| **chat** | Roleplay-style chat between you (`name1`) and a character (`name2`). |
| **chat-instruct** | Roleplay framing wrapped inside the instruction template — best of both. |

### 6.2 Everyday chat actions

| Action | Button / shortcut | Effect |
|---|---|---|
| Send | `Enter` | Send your message |
| Stop | `Esc` or **Stop** | Halt generation immediately |
| Regenerate | `Ctrl+Enter` | Re-roll the last reply |
| Continue | `Alt+Enter` | Make the model keep writing from where it stopped |
| Remove last | `Ctrl+Shift+Backspace` | Delete the last exchange |
| Impersonate | `Ctrl+Shift+M` | Have the model write *your* next message |
| Version navigation | `←` / `→` arrows | Step through alternate regenerated replies |
| Show/hide controls | `Ctrl+S` | Toggle the chat parameter sidebar |

> [!TIP]
> Pressing `→` (right arrow) when you're already on the newest version of a reply
> regenerates a fresh one — a quick way to keep re-rolling.

### 6.3 Managing conversations

- **New chat** — starts a fresh conversation. The dropdown arrow next to it offers an
  **Incognito chat** that isn't saved to disk.
- **Past chats** — the left sidebar lists saved conversations. Hover a chat to reveal a
  trash icon for quick deletion.
- Chats are saved automatically (except in incognito or `--multi-user` mode).

### 6.4 Attachments and pasting long text

Use the upload button in the chat input to attach **text files, PDFs, DOCX documents, and
images** (vision models can see images). If you paste a very long block of text, TextGen can
automatically turn it into an attachment instead of cluttering the input — enable
**Paste to attachment** in settings.

### 6.5 Web search

Click the **web search** toggle (a globe icon, tooltip "Search the internet with
DuckDuckGo") to let the model search the web for your query. TextGen fetches the top results,
extracts their main text, and adds them as attachments the model can read.

```text
Using search query: latest mars rover mission
Added 3 successful web search results as attachments.
```

Control how many pages are fetched with the `web_search_pages` setting (default 3).

> [!NOTE]
> Web search has built-in protection against being tricked into fetching internal/private
> network addresses, so it's safe to leave enabled.

---

## 7. The Notebook and Default Tabs

Not every task is a conversation. The **Notebook** tab gives you a single freeform text area
where the model continues whatever you've written — ideal for story writing, code completion,
or prompt experiments. The **Default** tab splits input and output into two boxes for
classic prompt-in/completion-out workflows.

- Prompts are saved as `.txt` files under `user_data/logs/notebook/` and can be reloaded
  from the prompt dropdown.
- Click **Token count** to see how many tokens your prompt uses against the model's context
  window.

> [!TIP]
> Use the Notebook tab to test how a model behaves *without* a chat template — it's the
> closest thing to talking to the raw model.

---

## 8. Controlling the Output: Presets and Samplers

How creative, focused, or repetitive a model is comes down to **sampler** settings, grouped
into reusable **presets** on the **Parameters** tab.

### 8.1 Using presets

Select a preset from the dropdown on the Parameters tab. TextGen ships several; **Top-P** is
the default starting point. Changing a preset instantly applies its sampler values. You can
also:

- **Save** the current sliders as a new preset (written to `user_data/presets/<name>.yaml`).
- **Neutralize samplers** to reset everything to neutral/off values.
- **Reset** back to the preset's saved values.

### 8.2 The most useful samplers

| Sampler | Default | Effect |
|---|---|---|
| `temperature` | `1` | Higher = more random/creative, lower = more focused |
| `top_p` | `0.95` | Nucleus sampling — keep tokens until cumulative probability ≥ this |
| `top_k` | `0` (off) | Keep only the K most likely tokens |
| `min_p` | `0` | Drop tokens below this fraction of the top token's probability |
| `repetition_penalty` | `1` (off) | Penalize tokens that already appeared |
| `dry_multiplier` | `0` (off) | DRY anti-repetition — strongly discourages repeated phrases |
| `max_new_tokens` | `512` | Maximum length of each reply |

> [!TIP]
> For factual/coding tasks, lower the temperature (e.g. `0.2–0.5`). For creative writing,
> raise it (`0.9–1.2`) and enable a repetition control like `dry_multiplier`.

> [!NOTE]
> Different loaders support different samplers. The Parameters tab automatically hides
> samplers your current loader doesn't implement, so you only see what will actually take
> effect. The Admin Guide has the full per-loader sampler matrix.

### 8.3 Grammars (constrained output)

In the Parameters tab you can supply a **GBNF grammar** to force the model's output into a
specific shape — valid JSON, a restricted vocabulary, a list format, and so on. Several
ready-made grammars ship in `user_data/grammars/` (e.g. `json.gbnf`, `list.gbnf`).

---

## 9. Characters and Instruction Templates

The **Character** tab (under Parameters) lets you define *who* the model is.

- **Character** — a name, context/personality, and greeting. Switch personas from the
  dropdown; cards live in `user_data/characters/`.
- **Your profile** — your display name (`name1`) and an optional bio the model is told about
  you.
- **Instruction template** — the Jinja2 template that formats messages for the model. TextGen
  auto-selects the right one for most known models; templates live in
  `user_data/instruction-templates/`.

> [!IMPORTANT]
> If a model gives garbled or "off" replies, the wrong instruction template is the usual
> cause. Most models include their own template metadata that TextGen applies automatically,
> but you can override it on the Model or Character tab.

### 9.1 Reasoning / "thinking" models

For models that emit a hidden reasoning block (DeepSeek-R1, Qwen3, GPT-OSS, Gemma, and
similar), TextGen automatically detects the thinking section and shows it collapsed, with the
final answer below. Three settings control this:

| Setting | Default | Effect |
|---|---|---|
| `enable_thinking` | on | Allow the model to produce a reasoning block |
| `reasoning_effort` | `medium` | Hint at how much reasoning to do (`low`/`medium`/`high`) |
| `preserve_thinking` | off | Keep prior turns' reasoning in the context |

---

## 10. Generating Images

> [!NOTE]
> Image generation is available in **full installs only** (not portable builds).

The **Image generation** tab runs diffusion image models locally. Download an image model
into `user_data/image_models/`, select it, enter a prompt, and generate.

Key controls: prompt and negative prompt, width/height (or aspect ratio), steps, CFG scale,
seed, and batch size/count. You can also start the server with an image model preselected:

```bash
python server.py --image-model FLUX.1-schnell --image-quant bnb-4bit
```

> [!TIP]
> Enable **LLM prompt variations** to have your loaded language model rewrite/expand your
> image prompt before generating — handy for richer results from a short prompt.

You can also generate images from chat or over the API (`/v1/images/generations`).

---

## 11. Using the API

TextGen can serve an **OpenAI-compatible** (and **Anthropic-compatible**) API, so any tool
that talks to ChatGPT or Claude can talk to your local model. Start it with `--api`:

```bash
python server.py --api --api-key my-secret-key
```

```text
Starting TextGen
OpenAI-compatible API URL:
http://127.0.0.1:5000
```

Run **headless** (API only, no web UI) with `--nowebui`.

### 11.1 Chat completions (OpenAI format)

```bash
curl http://127.0.0.1:5000/v1/chat/completions \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer my-secret-key" \
  -d '{
    "messages": [{"role": "user", "content": "Write a haiku about local LLMs."}],
    "temperature": 0.7,
    "max_tokens": 200
  }'
```

```json
{
  "id": "chatcmpl-...",
  "object": "chat.completion",
  "choices": [{"message": {"role": "assistant", "content": "Weights on my own drive..."}}]
}
```

Set `"stream": true` to receive tokens as server-sent events as they're generated.

### 11.2 Pointing existing tools at TextGen

Most OpenAI client libraries just need the base URL and key changed:

```python
from openai import OpenAI

client = OpenAI(base_url="http://127.0.0.1:5000/v1", api_key="my-secret-key")
resp = client.chat.completions.create(
    model="local",  # the loaded model is used regardless of this value
    messages=[{"role": "user", "content": "Hello!"}],
)
print(resp.choices[0].message.content)
```

### 11.3 Anthropic-compatible endpoint

TextGen also serves `/v1/messages` in Anthropic's format. Authenticate with the `x-api-key`
header (instead of `Authorization: Bearer`):

```bash
curl http://127.0.0.1:5000/v1/messages \
  -H "Content-Type: application/json" \
  -H "x-api-key: my-secret-key" \
  -d '{"messages": [{"role": "user", "content": "Hi"}], "max_tokens": 100}'
```

### 11.4 What the API exposes

| Category | Endpoints |
|---|---|
| **Text** | `POST /v1/completions`, `POST /v1/chat/completions`, `POST /v1/messages` |
| **Models** | `GET /v1/models`, `GET /v1/models/{model}` |
| **Audio** | `POST /v1/audio/transcriptions` (speech-to-text) |
| **Images** | `POST /v1/images/generations` |
| **Embeddings** | `POST /v1/embeddings` |
| **Moderation** | `POST /v1/moderations` |
| **Tokens** | `POST /v1/internal/encode`, `/decode`, `/token-count`, `/logits` |
| **Model mgmt** | `POST /v1/internal/model/load`, `/unload`, `GET /v1/internal/model/list`, `/info` |

> [!IMPORTANT]
> By default the API only accepts connections from your own machine (`localhost`). To let
> other devices reach it you must pass `--listen` (or `--public-api`). When you do, **always**
> set `--api-key`. See the Admin Guide's Security section before exposing the API.

> [!TIP]
> The model load/unload endpoints (`/v1/internal/model/*`) require the **admin** key. Set a
> separate `--admin-key` so untrusted API clients can generate text but can't swap your model.

---

## 12. Troubleshooting

Issue numbers below reference the project's GitHub issue tracker for additional context.

### Error: `libllama-common.so.0: cannot open shared object file`

**Cause:** The bundled llama.cpp binary can't find its shared library, usually after copying
a portable build between machines or a partial extraction (#7592).
**Solution:** Re-extract the portable zip fresh; don't move individual files out of it. On
Linux, ensure the `installer_files`/portable folder wasn't relocated.
**Prevention:** Keep the portable build's folder structure intact.

### A GGUF model fails to load

**Cause:** Corrupted or partial download, an outdated GGUF format, or insufficient
VRAM/RAM (#7530).
**Solution:** Re-download with `python download-model.py <repo> --specific-file <file>.gguf`
and verify with `--check`. If you see *"You are using an outdated GGUF, please download a new
one"*, get a freshly re-quantized version. Lower `--gpu-layers` if it's a memory issue.
**Prevention:** Prefer well-maintained GGUF repositories and validate checksums.

### AMD GPU model won't load

**Cause:** AMD support requires **Linux + ROCm 7.2**; AMD on Windows is not supported by the
installer (#7546).
**Solution:** Use Linux for AMD GPUs, or fall back to `--cpu` mode on Windows.
**Prevention:** Choose option **B (AMD)** in the installer only on Linux.

### `Failed to load the extension "coqui_tts"`

**Cause:** The extension's Python requirements aren't installed (#7580).
**Solution:** Run the update wizard → **B) Install/update extensions requirements**, or
`pip install -r extensions/coqui_tts/requirements.txt` inside the environment.
**Prevention:** Install extension requirements before enabling an extension.

### Out-of-memory when loading or generating

**Cause:** Model + context + KV cache exceeds your VRAM/RAM.
**Solution:** Lower `--gpu-layers`, reduce `--ctx-size`, set `--cache-type q8_0`, or enable
`--cpu-moe` for MoE models.
**Prevention:** Start with a smaller quantization (Q4_K_M) and a modest context size.

### API requests return `401 Unauthorized`

**Cause:** Missing or wrong key. The server expects `Authorization: Bearer <key>` (or
`x-api-key` for `/v1/messages`).
**Solution:** Pass the exact value you set with `--api-key`. Remember model load/unload needs
the `--admin-key`.
**Prevention:** Keep your key handy; set `--api-key` and `--admin-key` explicitly.

### API returns `Invalid host header` or refuses remote connections

**Cause:** The API is localhost-locked unless you opt in (#7545 for proxy setups).
**Solution:** Start with `--listen` (LAN) or `--public-api` (Cloudflare tunnel) plus
`--api-key`. Behind a reverse proxy, also set `--subpath`.
**Prevention:** Decide up front whether the API should be local-only or networked.

### The model "talks to itself" or produces garbled output

**Cause:** Wrong instruction template or chat mode for the model.
**Solution:** On the Model/Character tab, select the correct instruction template; try
**instruct** mode. Most models carry their own template — reloading often fixes it.
**Prevention:** Use models that include template metadata, and don't override the template
unless you know it's needed.

### Generation won't stop / VRAM stays used after closing (desktop app)

**Cause:** In some cases the `llama-server` subprocess lingers after exit (#7574).
**Solution:** Fully quit the app; if VRAM stays occupied, end any leftover `llama-server`
process. Press **Esc** to stop a runaway generation.
**Prevention:** Use the app's normal quit; avoid force-killing the window mid-generation.

---

## 13. Frequently Asked Questions

**Q: Does TextGen send my chats anywhere?**
No. Everything runs locally. Gradio analytics are disabled and there's no built-in telemetry.
Web search (opt-in) and the `--share`/`--public-api` tunnels are the only features that reach
the internet.

**Q: What model formats are supported?**
GGUF (llama.cpp), Hugging Face safetensors/PyTorch (Transformers), EXL3 (ExLlamaV3), and
TensorRT-LLM engines. GGUF is the easiest to start with.

**Q: Can I run a model bigger than my GPU's VRAM?**
Yes. Lower `--gpu-layers` so only some layers go to the GPU and the rest stay in CPU RAM, or
use `--cpu` for CPU-only. It'll be slower but it works.

**Q: How do I make my command-line flags stick across restarts?**
Put them in `user_data/CMD_FLAGS.txt`, one flag per line. They're applied on every launch
(command-line flags still take precedence). Interface settings can likewise be saved in
`user_data/settings.yaml`.

**Q: Where are my models, characters, and chat logs stored?**
Everything lives under `user_data/` — `models/`, `loras/`, `characters/`, `presets/`,
`grammars/`, and `logs/`. Back up that folder to preserve your setup.

**Q: How do I use TextGen from another program?**
Start it with `--api` and point any OpenAI-compatible client at
`http://127.0.0.1:5000/v1`. See [Using the API](#11-using-the-api).

**Q: Can multiple people use one instance?**
Yes, with `--multi-user`, but it's meant for **small trusted teams** only: chat history isn't
saved, the Stop button stops everyone's generation, and grayed-out actions can be bypassed
client-side. Don't expose it publicly.

**Q: What's the difference between the portable build and the installer?**
Portable is a no-install zip limited to llama.cpp/GGUF (plus the desktop window and API). The
one-click installer adds the Transformers/ExLlamaV3/TensorRT-LLM loaders, image generation,
and LoRA training.

**Q: How do I get speech, translation, or other extras?**
Enable the bundled extensions (Whisper STT, Coqui/Silero TTS, Google Translate, the
SuperboogaV2 RAG memory, image generation via Stable Diffusion API, etc.) from the **Session**
tab, after installing their requirements via the update wizard.

**Q: Can the model use tools or call external services?**
Yes. TextGen supports tool calling and **MCP** (Model Context Protocol) servers configured in
`user_data/mcp.json`. Select tools in the chat controls; enable confirmation prompts with the
`confirm_tool_calls` setting.

**Q: How do I update without losing my models and settings?**
Run the update wizard and choose **A**. Your `user_data/` folder is never touched by updates.

**Q: The web UI is slow over a remote/proxy connection — what can I do?**
Heavy proxying (e.g. Cloudflare in front of a remote host) can slow the UI (#7545). Prefer a
direct `--listen` connection on a trusted network, or the desktop app locally.

---

_Generated June 2026 from oobabooga/textgen at ed888c7. AI-generated documentation. The library is checked by sampling (three repositories so far); this guide was not individually reviewed. Verify against the project before relying on it. Unofficial; not affiliated with or reviewed by the project._