# PrivateGPT User Guide

## 1. Quick Start

PrivateGPT is a production-ready AI project that lets you interact with your documents using large language models (LLMs), completely privately. No data leaves your execution environment. It exposes an OpenAI-compatible API and a built-in Gradio web UI with RAG (Retrieval-Augmented Generation), search, chat, and summarization modes.

**Key features:**

- Fully private document Q&A — your data never leaves your machine
- OpenAI-compatible REST API for programmatic access
- Built-in Gradio web UI with four interaction modes
- Support for multiple LLM backends (Ollama, llama.cpp, OpenAI, Azure OpenAI, Gemini, SageMaker)
- Multiple vector store options (Qdrant, Chroma, PostgreSQL, ClickHouse, Milvus)
- Document ingestion with file upload, text input, and bulk/watch modes
- Streaming responses with source attribution
- Optional HTTP basic authentication

**Prerequisites:** Python 3.11+, Poetry, and an LLM backend (Ollama is the easiest to start with).

**Fastest start with Ollama:**

```bash
git clone https://github.com/zylon-ai/private-gpt
cd private-gpt
poetry install --extras "ui llms-ollama embeddings-ollama vector-stores-qdrant"
PGPT_PROFILES=ollama python -m private_gpt
```

```text
12:00:00.000 [INFO   ]      private_gpt.launcher - Mounting the gradio UI, at path=/
INFO:     Uvicorn running on http://0.0.0.0:8001 (Press CTRL+C to quit)
```

Open `http://localhost:8001` in your browser to access the web UI. Upload a document, select **RAG** mode, and ask a question about it.

> [!TIP]
> If you already have Ollama running with `llama3.1` and `nomic-embed-text` models pulled, PrivateGPT will work immediately with the `ollama` profile — no additional model downloads needed.

**What to try next:**

- Upload documents via the web UI sidebar
- Switch between RAG, Search, Basic Chat, and Summarize modes
- Use the API directly at `http://localhost:8001/v1/chat/completions`
- See the Configuration section for customizing your setup

## 2. Installation

### System Requirements

| Requirement | Minimum | Recommended |
|---|---|---|
| Python | 3.11 | 3.11+ |
| RAM | 8 GB | 16 GB+ (for local LLMs) |
| Disk | 10 GB (models) | 20 GB+ |
| Package manager | Poetry | Poetry |

### Installation Methods

=== "Ollama (Recommended)"
    ```bash
    git clone https://github.com/zylon-ai/private-gpt
    cd private-gpt
    poetry install --extras "ui llms-ollama embeddings-ollama vector-stores-qdrant"
    ```

=== "Local llama.cpp"
    ```bash
    git clone https://github.com/zylon-ai/private-gpt
    cd private-gpt
    poetry install --extras "ui llms-llama-cpp vector-stores-qdrant embeddings-huggingface"
    ```

=== "OpenAI"
    ```bash
    git clone https://github.com/zylon-ai/private-gpt
    cd private-gpt
    poetry install --extras "ui llms-openai embeddings-openai vector-stores-qdrant"
    ```

=== "Docker (llama.cpp CPU)"
    ```bash
    git clone https://github.com/zylon-ai/private-gpt
    cd private-gpt
    docker compose up -d
    ```

=== "Docker (Ollama)"
    ```bash
    git clone https://github.com/zylon-ai/private-gpt
    cd private-gpt
    docker build -t private-gpt -f Dockerfile.ollama .
    docker run -p 8001:8080 private-gpt
    ```

> [!IMPORTANT]
> You must install the correct Poetry extras for your chosen LLM and embedding backend. Mismatched extras cause `ImportError` at startup (e.g., `UI dependencies not found, install with poetry install --extras ui`).

### Post-Install Verification

```bash
PGPT_PROFILES=ollama python -m private_gpt
```

```text
12:00:00.000 [INFO   ]      private_gpt.launcher - Mounting the gradio UI, at path=/
INFO:     Uvicorn running on http://0.0.0.0:8001 (Press CTRL+C to quit)
```

Verify the health endpoint:

```bash
curl http://localhost:8001/health
```

```json
{"status": "ok"}
```

### Poetry Extras Reference

| Extra | Purpose |
|---|---|
| `ui` | Gradio web interface |
| `llms-llama-cpp` | llama.cpp LLM backend |
| `llms-ollama` | Ollama LLM backend |
| `llms-openai` | OpenAI / OpenAI-like LLM backend |
| `embeddings-huggingface` | HuggingFace embedding models |
| `embeddings-ollama` | Ollama embedding models |
| `embeddings-openai` | OpenAI embedding models |
| `vector-stores-qdrant` | Qdrant vector store |
| `vector-stores-postgres` | PostgreSQL vector store |
| `storage-nodestore-postgres` | PostgreSQL node store |

### Install Troubleshooting

### Error: `Preparing metadata (pyproject.toml) did not run successfully`

**Cause:** Missing system dependencies or incompatible Python version. This is the most commonly reported installation issue (#942, #1314).

**Solution:**
1. Verify Python version is 3.11+:
   ```bash
   python --version
   ```
2. Ensure Poetry is up to date:
   ```bash
   pip install --upgrade poetry
   ```
3. On Ubuntu/Debian, install build dependencies:
   ```bash
   sudo apt-get install build-essential python3-dev
   ```

**Prevention:** Always use Python 3.11+ and the latest Poetry version.

### Error: `Python was not found` on Windows

**Cause:** Python is not installed or not in PATH (#539).

**Solution:**
1. Install Python 3.11+ from python.org
2. Ensure "Add Python to PATH" is checked during installation
3. Restart your terminal

### Error: `AttributeError: type object 'hnswlib.Index' has no attribute 'file_handle_count'`

**Cause:** Version incompatibility with the hnswlib package (#1012).

**Solution:**
```bash
pip install --force-reinstall hnswlib
```

### Error: Dependency conflict on Apple Silicon (Mac M1/M2)

**Cause:** Some native dependencies lack pre-built wheels for ARM macOS (#2076).

**Solution:**
```bash
brew install cmake
poetry install --extras "ui llms-ollama embeddings-ollama vector-stores-qdrant"
```

### Error: Building Docker image fails on Ubuntu

**Cause:** Missing Docker build dependencies or outdated Docker version (#1244).

**Solution:**
Ensure Docker and Docker Compose are installed and up to date:
```bash
docker compose version
```

## 3. Configuration

PrivateGPT uses a YAML-based profile system. The base configuration is in `settings.yaml`, and profiles override specific values.

### Profile System

Activate profiles by setting the `PGPT_PROFILES` environment variable:

```bash
PGPT_PROFILES=ollama python -m private_gpt
```

Multiple profiles can be comma-separated and are merged in order:

```bash
PGPT_PROFILES=ollama,local python -m private_gpt
```

> [!NOTE]
> The `default` profile (`settings.yaml`) is always loaded first. Additional profiles are merged on top, with later profiles taking priority.

### Available Profiles

| Profile | File | LLM Mode | Embedding Mode | Use Case |
|---|---|---|---|---|
| default | `settings.yaml` | `llamacpp` | `huggingface` | Base configuration |
| local | `settings-local.yaml` | `llamacpp` | `huggingface` | Local development with Qdrant |
| ollama | `settings-ollama.yaml` | `ollama` | `ollama` | Ollama backend with Qdrant |
| ollama-pg | `settings-ollama-pg.yaml` | `ollama` | `ollama` | Ollama with PostgreSQL storage |
| openai | `settings-openai.yaml` | `openai` | `openai` | OpenAI API |
| azopenai | `settings-azopenai.yaml` | `azopenai` | `azopenai` | Azure OpenAI |
| gemini | `settings-gemini.yaml` | `gemini` | `gemini` | Google Gemini |
| sagemaker | `settings-sagemaker.yaml` | `sagemaker` | `sagemaker` | AWS SageMaker |
| vllm | `settings-vllm.yaml` | `openailike` | `huggingface` | vLLM serving |
| docker | `settings-docker.yaml` | configurable | configurable | Docker deployment |
| mock | `settings-mock.yaml` | `mock` | `huggingface` | Testing without GPU |
| test | `settings-test.yaml` | `mock` | `mock` | Automated tests |

### Environment Variable Substitution

YAML values support `${VAR}` and `${VAR:default}` syntax for environment variables:

```yaml
openai:
  api_key: ${OPENAI_API_KEY:}
  model: gpt-3.5-turbo
```

> [!WARNING]
> If an environment variable is referenced with `${VAR}` (no default) and is not set, PrivateGPT will fail to start with: `ValueError: Environment variable VAR is not set and not default was provided`.

### Common Configuration Options

| Setting | Default | Description |
|---|---|---|
| `server.port` | `8001` | API server port |
| `server.env_name` | `prod` | Environment name |
| `llm.mode` | `llamacpp` | LLM backend selection |
| `llm.max_new_tokens` | `256` | Max tokens per response |
| `llm.context_window` | `3900` | Max context tokens |
| `llm.temperature` | `0.1` | Response creativity (0.0–1.0) |
| `embedding.mode` | `huggingface` | Embedding backend selection |
| `embedding.ingest_mode` | `simple` | Ingestion parallelism |
| `ui.enabled` | `true` | Enable/disable web UI |
| `ui.default_mode` | `RAG` | Default UI mode |
| `vectorstore.database` | `qdrant` | Vector store backend |

### Settings Folder Override

To load settings from a custom directory:

```bash
PGPT_SETTINGS_FOLDER=/path/to/settings PGPT_PROFILES=ollama python -m private_gpt
```

For the full configuration reference with all settings, see the Admin Guide.

## 4. Core Usage

### Web UI

The Gradio-based web UI is available at the server root (default `http://localhost:8001/`). It provides four interaction modes:

#### RAG Mode (Default)

Get answers grounded in your uploaded documents. The LLM uses ingested document context to generate responses with source citations.

1. Upload one or more documents using the **Upload File(s)** button
2. Ensure **RAG** mode is selected
3. Type your question in the chat input
4. The response includes source file names and page numbers

> [!TIP]
> Select a specific file from the ingested files list to restrict RAG queries to that document only. Click **De-select selected file** to query across all documents again.

#### Search Mode

Find relevant text chunks across your documents without LLM generation. Returns up to 4 matching chunks with file name, page number, and the matching text. Useful for quickly locating specific information.

#### Basic Chat Mode

Chat directly with the LLM using its training data. Uploaded documents are ignored in this mode.

#### Summarize Mode

Generate summaries of ingested documents. Optionally select a specific file and provide custom summarization instructions in the chat input.

### System Prompts

Each mode has a configurable system prompt that influences how the LLM responds. The defaults are set in `settings.yaml` under `ui.default_chat_system_prompt`, `ui.default_query_system_prompt`, and `ui.default_summarization_system_prompt`. You can override them in the UI's **System Prompt** field during a session.

> [!NOTE]
> The chat history is capped at 20 messages to avoid exceeding the LLM's context window. Older messages are silently dropped.

### File Management

- **Upload:** Click **Upload File(s)** — supports multiple files. Re-uploading a file with the same name replaces the previous version automatically.
- **Select:** Click a file in the ingested files list to scope queries and deletion to that file.
- **Delete selected:** Select a file in the list, then click **Delete selected file**.
- **Delete all:** Click **Delete ALL files** to clear the entire ingestion store.

> [!WARNING]
> File deletion is permanent. There is no undo. Deleted documents are removed from the vector store immediately.

### API Usage

PrivateGPT exposes an OpenAI-compatible REST API. All versioned endpoints are under `/v1/`.

#### Chat Completions

```bash
curl -X POST http://localhost:8001/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{
    "messages": [
      {"role": "system", "content": "You are a helpful assistant."},
      {"role": "user", "content": "What is PrivateGPT?"}
    ],
    "use_context": false,
    "stream": false
  }'
```

```json
{
  "id": "c202d5e6-7b69-4869-81cc-dd574ee8ee11",
  "object": "completion",
  "created": 1623340000,
  "model": "private-gpt",
  "choices": [
    {
      "finish_reason": "stop",
      "message": {
        "role": "assistant",
        "content": "PrivateGPT is a privacy-focused AI assistant..."
      },
      "sources": null,
      "index": 0
    }
  ]
}
```

#### RAG Chat (with Document Context)

```bash
curl -X POST http://localhost:8001/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{
    "messages": [
      {"role": "user", "content": "Summarize the key findings"}
    ],
    "use_context": true,
    "include_sources": true,
    "stream": false
  }'
```

When `use_context` is `true`, the response includes `sources` in each choice — an array of chunks with document IDs, file names, page labels, relevance scores, and the matching text.

#### Streaming Responses

Set `"stream": true` to receive Server-Sent Events (SSE). Each event follows the OpenAI streaming format:

```text
data: {"id":"12345","object":"completion.chunk","created":1694268190,"model":"private-gpt","choices":[{"index":0,"delta":{"content":"Hello"},"finish_reason":null}]}

data: {"id":"12345","object":"completion.chunk","created":1694268190,"model":"private-gpt","choices":[{"index":0,"delta":{"content":""},"finish_reason":"stop"}]}

data: [DONE]
```

#### Text Completions

```bash
curl -X POST http://localhost:8001/v1/completions \
  -H "Content-Type: application/json" \
  -d '{
    "prompt": "How do you fry an egg?",
    "system_prompt": "You are a helpful chef.",
    "use_context": false,
    "stream": false
  }'
```

> [!TIP]
> The `/v1/chat/completions` endpoint is recommended over `/v1/completions` for most use cases.

#### Ingest a File

```bash
curl -X POST http://localhost:8001/v1/ingest/file \
  -F "file=@/path/to/document.pdf"
```

```json
{
  "object": "list",
  "model": "private-gpt",
  "data": [
    {
      "object": "ingest.document",
      "doc_id": "c202d5e6-7b69-4869-81cc-dd574ee8ee11",
      "doc_metadata": {"file_name": "document.pdf", "page_label": "1"}
    }
  ]
}
```

#### Ingest Text

```bash
curl -X POST http://localhost:8001/v1/ingest/text \
  -H "Content-Type: application/json" \
  -d '{
    "file_name": "Company Policy",
    "text": "All employees must complete security training annually."
  }'
```

#### List Ingested Documents

```bash
curl http://localhost:8001/v1/ingest/list
```

#### Delete a Document

```bash
curl -X DELETE http://localhost:8001/v1/ingest/c202d5e6-7b69-4869-81cc-dd574ee8ee11
```

#### Context Filtering

Restrict queries to specific documents by passing `context_filter` with document IDs (obtained from `/v1/ingest/list`):

```json
{
  "messages": [{"role": "user", "content": "What does this say?"}],
  "use_context": true,
  "context_filter": {
    "docs_ids": ["c202d5e6-7b69-4869-81cc-dd574ee8ee11"]
  }
}
```

#### Retrieve Chunks

```bash
curl -X POST http://localhost:8001/v1/chunks \
  -H "Content-Type: application/json" \
  -d '{
    "text": "quarterly revenue",
    "limit": 5,
    "prev_next_chunks": 2
  }'
```

#### Summarize

```bash
curl -X POST http://localhost:8001/v1/summarize \
  -H "Content-Type: application/json" \
  -d '{
    "use_context": true,
    "instructions": "Provide a concise executive summary",
    "stream": false
  }'
```

#### Generate Embeddings

```bash
curl -X POST http://localhost:8001/v1/embeddings \
  -H "Content-Type: application/json" \
  -d '{"input": "Hello world"}'
```

## 5. Common Tasks

### Task: Set Up PrivateGPT with Ollama

**Goal:** Run PrivateGPT locally with Ollama as the LLM backend.

**Prerequisites:** Ollama installed and running.

**Steps:**
1. Pull the required models:
   ```bash
   ollama pull llama3.1
   ollama pull nomic-embed-text
   ```
2. Install PrivateGPT with Ollama extras:
   ```bash
   poetry install --extras "ui llms-ollama embeddings-ollama vector-stores-qdrant"
   ```
3. Start with the Ollama profile:
   ```bash
   PGPT_PROFILES=ollama python -m private_gpt
   ```
4. Open `http://localhost:8001` in your browser.

**Verify:** The health endpoint returns `{"status": "ok"}`.

**If it fails:** Check that Ollama is running (`ollama list`) and the models are pulled. See Troubleshooting for connection issues.

### Task: Use OpenAI as the Backend

**Goal:** Use OpenAI's API instead of a local model.

**Steps:**
1. Install with OpenAI extras:
   ```bash
   poetry install --extras "ui llms-openai embeddings-openai vector-stores-qdrant"
   ```
2. Set your API key:
   ```bash
   export OPENAI_API_KEY=sk-your-key-here
   ```
3. Start with the OpenAI profile:
   ```bash
   PGPT_PROFILES=openai python -m private_gpt
   ```

### Task: Ingest a Folder of Documents

**Goal:** Bulk-ingest an entire folder of documents.

**Prerequisites:** Enable local ingestion in settings.

**Steps:**
1. In your settings YAML, enable local ingestion:
   ```yaml
   data:
     local_ingestion:
       enabled: true
       allow_ingest_from: ["*"]
   ```
2. Use the provided script:
   ```bash
   python scripts/ingest_folder.py /path/to/documents
   ```

> [!WARNING]
> Local ingestion is not secure by default and should only be used in controlled environments. Set `allow_ingest_from` to restrict which folders can be ingested.

### Task: Use PostgreSQL as Vector and Node Store

**Goal:** Use PostgreSQL for persistent storage instead of Qdrant.

**Steps:**
1. Install with PostgreSQL extras:
   ```bash
   poetry install --extras "ui llms-ollama embeddings-ollama vector-stores-postgres storage-nodestore-postgres"
   ```
2. Start with the Ollama + PostgreSQL profile:
   ```bash
   PGPT_PROFILES=ollama-pg python -m private_gpt
   ```
3. Default PostgreSQL connection: `localhost:5432`, database `postgres`, user `postgres`, schema `private_gpt`.

### Task: Enable Authentication

**Goal:** Protect the API with HTTP basic authentication.

**Steps:**
1. Generate your auth secret:
   ```bash
   python -c 'import base64; print("Basic " + base64.b64encode("myuser:mypass".encode()).decode())'
   ```
   ```text
   Basic bXl1c2VyOm15cGFzcw==
   ```
2. Add to your settings YAML:
   ```yaml
   server:
     auth:
       enabled: true
       secret: "Basic bXl1c2VyOm15cGFzcw=="
   ```
3. Include the `Authorization` header with every API request:
   ```bash
   curl -H "Authorization: Basic bXl1c2VyOm15cGFzcw==" http://localhost:8001/v1/ingest/list
   ```

> [!CAUTION]
> The default configuration ships with a hardcoded auth secret (`Basic c2VjcmV0OmtleQ==`, which decodes to `secret:key`). Always change this before exposing PrivateGPT to any network (#2204).

### Task: Deploy with Docker Compose

**Goal:** Run PrivateGPT in a container.

**Steps:**
1. Configure environment variables in your shell or a `.env` file:
   ```bash
   export PGPT_MODE=ollama
   export PGPT_EMBED_MODE=ollama
   export PGPT_OLLAMA_API_BASE=http://ollama:11434
   ```
2. Start the stack:
   ```bash
   docker compose up -d
   ```
3. Access the UI at `http://localhost:8080` (Docker profile uses port 8080).

### Task: Use Azure OpenAI

**Goal:** Connect to Azure OpenAI instead of direct OpenAI.

**Steps:**
1. Set environment variables:
   ```bash
   export AZ_OPENAI_API_KEY=your-key
   export AZ_OPENAI_ENDPOINT=https://your-resource.openai.azure.com/
   export AZ_OPENAI_EMBEDDING_DEPLOYMENT_NAME=your-embedding-deployment
   export AZ_OPENAI_LLM_DEPLOYMENT_NAME=your-llm-deployment
   ```
2. Start with the Azure profile:
   ```bash
   PGPT_PROFILES=azopenai python -m private_gpt
   ```

### Task: Speed Up Document Ingestion

**Goal:** Use parallel processing for faster ingestion.

Update your settings YAML:

```yaml
embedding:
  mode: huggingface
  ingest_mode: parallel
  count_workers: 4
```

| Mode | Behavior |
|---|---|
| `simple` | Sequential, one file at a time (default) |
| `batch` | Parse files in parallel, embed in batch |
| `parallel` | Parse and embed in parallel (fastest for local) |
| `pipeline` | Keep the embedding engine maximally busy |

> [!TIP]
> For local setups, `parallel` mode is fastest. Do not set `count_workers` higher than your CPU thread count — it may cause memory issues.

### Task: Use Google Gemini

**Goal:** Use Google's Gemini models.

**Steps:**
1. Set your API key:
   ```bash
   export GOOGLE_API_KEY=your-gemini-api-key
   ```
2. Start with the Gemini profile:
   ```bash
   PGPT_PROFILES=gemini python -m private_gpt
   ```

### Task: Enable Reranking for Better RAG Results

**Goal:** Improve retrieval quality with a cross-encoder reranker.

Add to your settings YAML:

```yaml
rag:
  similarity_top_k: 5
  rerank:
    enabled: true
    model: cross-encoder/ms-marco-MiniLM-L-2-v2
    top_n: 2
```

This retrieves the top 5 candidates, then reranks to return the top 2 most relevant.

## 6. Troubleshooting

### Diagnostic Decision Tree

```text
PrivateGPT won't start
├── ImportError?
│   └── Install missing extras: poetry install --extras "..."
├── ValueError about environment variable?
│   └── Set the required env var or add a default in YAML
├── Connection refused on port 8001?
│   └── Check if another process uses the port
└── Model download hangs?
    └── Check internet connection / Ollama service status
```

### Error: `ImportError: UI dependencies not found, install with poetry install --extras ui`

**Cause:** The `ui` extra was not installed, but `ui.enabled` is `true` in settings.

**Solution:**
```bash
poetry install --extras "ui"
```

**Prevention:** Always include `ui` in your extras if you want the web interface.

### Error: `ImportError: Ollama dependencies not found, install with poetry install --extras llms-ollama or embeddings-ollama`

**Cause:** Ollama Python package not installed but an Ollama LLM or embedding mode is configured.

**Solution:**
```bash
poetry install --extras "llms-ollama embeddings-ollama"
```

### Error: `Failed to connect to Ollama`

**Cause:** Ollama service is not running or unreachable at the configured `api_base` (#2071).

**Solution:**
1. Verify Ollama is running:
   ```bash
   ollama list
   ```
2. Check the `ollama.api_base` setting matches your Ollama instance (default: `http://localhost:11434`).
3. If running in Docker, use the Docker service name (e.g., `http://ollama:11434`).

**Prevention:** Set `ollama.autopull_models: true` to automatically pull missing models on startup.

### Error: `HNSW index loading error when ingesting multiple documents`

**Cause:** Concurrent writes to the HNSW index in local-only mode (#2180).

**Solution:**
1. Reduce `embedding.count_workers` to `1`
2. Use `simple` ingest mode
3. Consider switching to a client-server vector store (Qdrant server, PostgreSQL)

**Prevention:** For multi-document ingestion, use a vector store that handles concurrent writes natively.

### Error: `TypeError: Config file has no top-level mapping`

**Cause:** A settings YAML file is empty or malformed.

**Solution:**
Verify the profile YAML file has valid YAML with a top-level mapping:
```bash
python -c "import yaml; print(yaml.safe_load(open('settings-ollama.yaml')))"
```

### Error: `ValueError: Environment variable ... is not set and not default was provided`

**Cause:** A `${VAR}` reference in YAML has no matching environment variable and no default.

**Solution:**
Either set the environment variable or provide a default in the YAML:
```yaml
# Before (fails if OPENAI_API_KEY not set):
api_key: ${OPENAI_API_KEY}

# After (empty string default):
api_key: ${OPENAI_API_KEY:}
```

### Error: HTTP 401 `Not authenticated`

**Cause:** Authentication is enabled and the request lacks a valid `Authorization` header.

**Solution:**
Include the configured auth secret in the header:
```bash
curl -H "Authorization: Basic c2VjcmV0OmtleQ==" http://localhost:8001/v1/ingest/list
```

> [!WARNING]
> The secret must match the exact value in `server.auth.secret`, including the `Basic ` prefix. The comparison uses constant-time comparison to prevent timing attacks.

### Error: `ValueError: Settings are not configured`

**Cause:** Settings object is `None`, typically from a corrupted or missing settings file.

**Solution:**
1. Ensure `settings.yaml` exists in the project root
2. Verify `PGPT_SETTINGS_FOLDER` points to the correct directory
3. Check that the YAML is valid

### Error: `Preparing metadata (pyproject.toml) did not run successfully`

**Cause:** Missing system dependencies or incompatible Python version (#942, #1314).

**Solution:**
1. Ensure Python 3.11+
2. Install build tools: `sudo apt-get install build-essential python3-dev`
3. Update Poetry: `pip install --upgrade poetry`

### UI shows no model label

**Cause:** The `llm.mode` value doesn't match any known mode.

**Solution:** Ensure `llm.mode` is one of: `llamacpp`, `openai`, `openailike`, `azopenai`, `sagemaker`, `mock`, `ollama`, `gemini`.

## 7. FAQ

**Q: Which LLM backend should I choose?**

For the easiest local setup, use **Ollama** — it manages model downloads and inference automatically. For maximum privacy with no external dependencies, use **llama.cpp**. For best quality without privacy constraints, use **OpenAI** or **Azure OpenAI**.

**Q: What document formats are supported?**

PrivateGPT uses LlamaIndex for document parsing, which supports PDF, DOCX, TXT, CSV, HTML, Markdown, and many other formats. You may need to install an extra dependency for specific file types.

**Q: Can I use PrivateGPT without the web UI?**

Yes. Set `ui.enabled: false` in your settings and interact entirely through the REST API. This also removes the need for the `ui` Poetry extra.

**Q: How do I change the default LLM model?**

Update the model setting in your profile YAML. For Ollama: `ollama.llm_model`. For llama.cpp: `llamacpp.llm_hf_repo_id` and `llamacpp.llm_hf_model_file`. For OpenAI: `openai.model`.

**Q: What vector stores are supported?**

Qdrant (local file or server), Chroma, PostgreSQL (with pgvector), ClickHouse, and Milvus. Set via `vectorstore.database`.

**Q: How do I reset all ingested documents?**

Delete all documents via the API — list with `GET /v1/ingest/list` then delete each with `DELETE /v1/ingest/{doc_id}`. Or use the **Delete ALL files** button in the web UI.

**Q: Can multiple users share the same instance?**

All ingested documents are shared across all sessions. There is no per-user document isolation (#1796). Enable authentication to control API access.

**Q: What is the `prompt_style` setting?**

It controls how messages are formatted for the LLM. Options: `default`, `llama2`, `llama3`, `tag`, `mistral`, `chatml`. Use `llama3` for Llama 3.x models, `mistral` for Mistral models.

**Q: How does the tiktoken cache work?**

PrivateGPT bundles a `tiktoken_cache/` directory so tokenization works in fully offline environments. The `TIKTOKEN_CACHE_DIR` environment variable is set automatically at import time.

**Q: Is Gradio analytics enabled?**

No. PrivateGPT explicitly disables Gradio analytics by setting `GRADIO_ANALYTICS_ENABLED=False` at import time.

---

_Generated May 2026 from zylon-ai/private-gpt at 7effd57. AI-generated documentation. The library is checked by sampling (three repositories so far); this guide was not individually reviewed. Verify against the project before relying on it. Unofficial; not affiliated with or reviewed by the project._