Pathway LLM App Templates User Guide

pathwaycom/llm-app is a catalogue of ten production-shape reference applications that sit on top of the Pathway streaming-dataflow engine. It is not a library, not a single app, and not a framework — it is a curated set of working RAG (Retrieval-Augmented Generation), document-indexing, and alerting services, each one already wired with a Dockerfile, an app.yaml configuration, sample data, and (for most templates) a Streamlit UI. Pick the template that matches your use case, point it at your documents, set your API keys, and run.

This guide walks you through choosing, configuring, and running the templates. For architecture, deployment hardening, and the shared App-class internals, see the Pathway LLM App Templates Administrator & Developer Guide.

1. Quick Start

What this repository ships

• 8 deployable templates under templates/ — each a Pathway pipeline with a REST or MCP server.
• 2 Jupyter cookbooks under cookbooks/self-rag-agents/ — interactive tutorials for self-RAG agentic workflows with LangGraph; these are learning notebooks, not deployable apps.
• Per-template Dockerfile, docker-compose.yml, app.yaml, and seed data so each template runs end-to-end in one command.
• Streamlit UIs for four of the templates (question_answering_rag/ui, slides_ai_search/ui, drive_alert/ui, unstructured_to_sql_on_the_fly/ui) so you can chat with your data in a browser.

Key features

• Real-time, always-fresh document indexes — Pathway watches your data sources and updates the vector index as files change.
• Pluggable LLM, embedder, parser, splitter, and retriever — swap any component by editing one YAML block.
• Built-in connectors for the local filesystem, Google Drive, and Microsoft SharePoint (configured via YAML, no code changes).
• Cloud or fully-private deployment — drop OpenAIChat for LiteLLMChat pointed at Ollama and run with zero outbound calls.
• REST API and Model Context Protocol (MCP) servers exposed by the templates' Pathway xpacks.
• Optional persistent caching so a restart re-uses already-computed embeddings.

Prerequisites

Docker 24+ with the Compose plugin (or Python 3.10+ with pip if you prefer to run templates natively). An OpenAI API key for templates that default to GPT — or local Ollama for the private template.

One-command Quick Start (the canonical template)

The reference template — the one whose shape every other App-class template mirrors — is question_answering_rag. Run it like this:

git clone https://github.com/pathwaycom/llm-app.git
cd llm-app/templates/question_answering_rag

# Provide your OpenAI key
echo "OPENAI_API_KEY=sk-..." > .env

# Build and run with the bundled compose file
docker compose up --build

[+] Building 78.4s (12/12) FINISHED
[+] Running 2/2
 ✔ Container question_answering_rag-app-1  Started
 ✔ Container question_answering_rag-ui-1   Started
2026-01-07 12:00:00 pathway INFO Indexed 1 documents
2026-01-07 12:00:01 streamlit INFO Running on http://0.0.0.0:8501

Open http://localhost:8501 in a browser. You will see the Streamlit RAG UI with the bundled sample PDF (IdeanomicsInc_..._Content License Agreement.pdf) already indexed. Ask a question; the answer comes back with the source documents that were used as context.

Tip

You can hit the Pathway REST API directly without the Streamlit UI: curl -X POST http://localhost:8000/v2/answer -H "Content-Type: application/json" -d '{"prompt": "What is this document about?"}'. Every App-class template exposes the same endpoint shape on port 8000 by default.

What to try next

• Pick a different template — see the selector table in section 4 to choose by use case.
• Index your own documents — drop PDFs into templates/question_answering_rag/data/ and the index updates automatically (no restart).
• Switch to local LLMs — try templates/private_rag/ to run with Ollama instead of OpenAI.
• Read the per-template README — every templates/<name>/README.md documents that specific template in depth.

2. Installation

System requirements

Important

The templates default to OpenAI models (gpt-4.1-mini, gpt-4o, text-embedding-3-small) and will fail to start without OPENAI_API_KEY unless you switch the LLM block in app.yaml to a local provider. Keep the key in .env next to app.yaml; never commit it.

Install methods

Docker Compose (recommended)

git clone https://github.com/pathwaycom/llm-app.git
cd llm-app/templates/<template_name>
cp .env.example .env  # (when present) — fill in your secrets
docker compose up --build

Native Python

git clone https://github.com/pathwaycom/llm-app.git
cd llm-app/templates/<template_name>
python -m venv .venv && source .venv/bin/activate
pip install pathway[all] python-dotenv
pip install -r requirements.txt  # adds template-specific extras
cp .env.example .env  # when present
python app.py

Single Dockerfile

cd llm-app/templates/<template_name>
docker build -t pathway-llm-app .
docker run --rm -p 8000:8000 \
  -v "$PWD/data:/app/data" \
  --env-file .env \
  pathway-llm-app

Pathway license key

Each template's app.py calls pw.set_license_key("demo-license-key-with-telemetry") so the demo runs out of the box with anonymous telemetry. To unlock advanced features (multi-machine clustering via --addresses, autoscaling, certain enterprise persistence backends), get a free Scale key from https://pathway.com/features and either edit the line in app.py or set PATHWAY_LICENSE_KEY in .env and replace the call with pw.set_license_key(os.environ["PATHWAY_LICENSE_KEY"]). Comment the call out entirely to use Pathway Community (no telemetry, no advanced features).

Note

The demo license key is real and works — it just enables anonymous usage telemetry. There is no quota or expiration. Switch to a personal Scale key if your environment cannot send telemetry, or remove the call to opt out entirely.

Post-install verification

After docker compose up --build completes for question_answering_rag:

# REST API: ask a question
curl -X POST http://localhost:8000/v2/answer \
  -H "Content-Type: application/json" \
  -d '{"prompt": "Summarise the document"}'

# REST API: list indexed documents
curl http://localhost:8000/v2/list_documents

# REST API: retrieve top-k chunks for a query (no LLM call)
curl -X POST http://localhost:8000/v1/retrieve \
  -H "Content-Type: application/json" \
  -d '{"query": "license agreement", "k": 4}'

{"response": "This document is a content license agreement between..."}

If you see a JSON response, the pipeline is live, the documents are indexed, and your API key works.

Install troubleshooting

3. Configuration

YAML-first, environment-second

Every modern template is configured by exactly one file: app.yaml next to app.py. The YAML uses Pathway's !pw.<class> tag syntax to instantiate Python objects directly. $variables (anchors) let you define an embedder once and reference it from both the parser and the retriever. The two legacy templates (drive_alert and unstructured_to_sql_on_the_fly) instead use procedural Python kwargs and os.environ.get defaults — they predate the App-class refactor.

Annotated minimal config (Question-Answering RAG)

The smallest meaningful app.yaml:

# Where to read documents from
$sources:
  - !pw.io.fs.read
    path: data
    format: binary
    with_metadata: true

# The chat model
$llm: !pw.xpacks.llm.llms.OpenAIChat
  model: "gpt-4.1-mini"
  cache_strategy: !pw.udfs.DefaultCache {}
  temperature: 0
  capacity: 8
  async_mode: "fully_async"

# The embedding model
$embedder: !pw.xpacks.llm.embedders.OpenAIEmbedder
  model: "text-embedding-3-small"
  cache_strategy: !pw.udfs.DefaultCache {}

# Chunking + parsing
$splitter: !pw.xpacks.llm.splitters.TokenCountSplitter
  max_tokens: 400
$parser: !pw.xpacks.llm.parsers.DoclingParser
  table_parsing_strategy: "llm"
  async_mode: "fully_async"
  chunk: false
  cache_strategy: !pw.udfs.DefaultCache {}

# Vector index
$retriever_factory: !pw.indexing.UsearchKnnFactory
  reserved_space: 1000
  embedder: $embedder
  metric: !pw.indexing.USearchMetricKind.COS

# The actual document store + question-answerer wired together
$document_store: !pw.xpacks.llm.document_store.DocumentStore
  docs: $sources
  parser: $parser
  splitter: $splitter
  retriever_factory: $retriever_factory

question_answerer: !pw.xpacks.llm.question_answering.BaseRAGQuestionAnswerer
  llm: $llm
  indexer: $document_store

Every YAML field maps directly to a Pydantic field on App in app.py — extra="forbid" is set, so a typo in a key fails fast at startup with a Pydantic validation error.

Tip

The $ prefix is a YAML anchor: $llm: defines, llm: $llm references. This lets one component (e.g. $embedder) be reused by both the parser and retriever without duplication. Only fields that are not prefixed with $ (top-level: question_answerer, host, port, persistence_mode, terminate_on_error) are read by the App-class itself.

Common option combinations

Environment variables you typically set

These are referenced inside app.yaml (with $VARIABLE) or by os.environ.get in legacy templates:

Note

Legacy templates (drive_alert, unstructured_to_sql_on_the_fly) get every parameter from os.environ.get calls inside app.py. Modern App-class templates pull values from app.yaml, so the equivalent of "setting an env var" there is editing the YAML.

For deeper config — including persistence-backend tuning, retry strategies, and connector-specific knobs — see the Pathway LLM App Templates Administrator & Developer Guide, section 3.

4. Core Usage

4.1 Pick the template that fits your problem

This is the most important table in the guide. Each template is a complete, self-contained service with its own folder, README, Dockerfile, and YAML. Pick by use case, not by name.

Tip

When in doubt, start with question_answering_rag. It is the canonical template; every other modern template is a variation on its App(BaseModel) shape. Once you understand it, switching to adaptive_rag or multimodal_rag is a YAML diff away.

4.2 The shared shape (every modern template)

Every App-class template follows the same lifecycle:

app.yaml ──► pw.load_yaml(f) ──► App(**config) ──► app.run()
                                       │
                                       ├── instantiate xpack server (QASummaryRestServer / DocumentStoreServer / PathwayMcp)
                                       ├── resolve persistence_mode + persistence_backend
                                       └── pw.run(persistence_config=…, terminate_on_error=…, monitoring_level=NONE)

The Pydantic shell carries seven public fields (question_answerer or document_store or mcp_http, host, port, with_cache (deprecated), persistence_backend, persistence_mode, terminate_on_error) and one method (run). When you start the container, app.py reads app.yaml, builds the App, and calls pw.run to boot the streaming dataflow.

4.3 Run the canonical RAG (question_answering_rag)

cd templates/question_answering_rag
echo "OPENAI_API_KEY=sk-..." > .env
docker compose up --build

The compose file starts two containers: app (port 8000, the Pathway REST API + index) and ui (port 8501, the Streamlit UI). The UI uses RAGClient(url=f"http://{PATHWAY_HOST}:{PATHWAY_PORT}") — by default app:8000 inside the compose network — to talk to the API.

REST endpoints (canonical RAG and adaptive/multimodal/private RAG)

# Ask a question
curl -X POST http://localhost:8000/v2/answer \
  -H "Content-Type: application/json" \
  -d '{"prompt": "What are the contract terms?"}'

# Retrieve only — useful for debugging the index
curl -X POST http://localhost:8000/v1/retrieve \
  -H "Content-Type: application/json" \
  -d '{"query": "license", "k": 4}'

# Health-check the indexer
curl -X POST http://localhost:8000/v1/statistics

Note

The endpoints above belong to Pathway's xpacks (QASummaryRestServer, DocumentStoreServer). They are not implemented in the templates — the templates configure them. For exhaustive REST-shape details (every JSON field, every status code) consult Pathway's API docs at https://pathway.com/developers/api-docs/pathway-xpacks-llm/.

4.4 Adaptive RAG — same shape, smarter retrieval

adaptive_rag substitutes BaseRAGQuestionAnswerer for AdaptiveRAGQuestionAnswerer, which retrieves documents iteratively: it starts with n_starting_documents: 2, and if the model is not confident enough, it requests factor: 2× more, up to max_iterations: 4. This usually halves token spend on questions where the answer sits in 1-2 chunks while still escalating for hard questions.

question_answerer: !pw.xpacks.llm.question_answering.AdaptiveRAGQuestionAnswerer
  llm: $llm
  indexer: $document_store
  n_starting_documents: 2
  factor: 2
  max_iterations: 4

Run identically to the canonical template — same endpoints, same UI compatibility.

4.5 Multimodal RAG — image and table understanding

multimodal_rag upgrades the parser to read tables and images by way of a second OpenAI client (gpt-4o) used only inside DoclingParser:

$parsing_llm: !pw.xpacks.llm.llms.OpenAIChat
  model: "gpt-4o"
  ...

$parser: !pw.xpacks.llm.parsers.DoclingParser
  multimodal_llm: $parsing_llm
  image_parsing_strategy: "llm"
  table_parsing_strategy: "llm"
  ...

The answering LLM stays as gpt-4.1-mini — only parsing is multimodal. Use this for finance reports, scientific papers, slide-derived PDFs, anywhere a chart or a table holds the actual answer.

4.6 Private RAG — fully local

private_rag keeps the same shape but swaps every external service for a local one:

$llm_model: "ollama/mistral"

$llm: !pw.xpacks.llm.llms.LiteLLMChat
  model: $llm_model
  api_base: "http://localhost:11434"
  # api_base: "http://host.docker.internal:11434"  # for Docker-on-Mac/Windows
  async_mode: "fully_async"

$embedding_model: "avsolatorio/GIST-small-Embedding-v0"
$embedder: !pw.xpacks.llm.embedders.SentenceTransformerEmbedder
  model: $embedding_model

Prerequisites:

1. Install Ollama on the host.
2. ollama pull mistral (or any model your $llm_model references).
3. ollama serve (or run the official Ollama Docker image on the host network).
4. From inside the Docker container, localhost won't work — use host.docker.internal on Mac/Windows, or --network host on Linux.

ollama pull mistral
ollama serve &
cd templates/private_rag
docker compose up --build

Important

Private RAG uses AdaptiveRAGQuestionAnswerer with strict_prompt: true. The strict prompt is more careful about saying "I don't know" when the context isn't sufficient — usually desirable for sensitive data, occasionally surprising if you expect the model to extrapolate.

4.7 Document indexing only — no LLM

document_indexing skips the answerer entirely and exposes a pure document store on REST. Use it when you want Pathway to maintain the index but call your own LLM downstream:

The default embedder is mixedbread-ai/mxbai-embed-large-v1 running locally via SentenceTransformer — no OpenAI key needed. Drop files into templates/document_indexing/files-for-indexing/ and they get indexed live.

4.8 MCP server — feed Pathway to MCP clients

document_store_mcp_server extends the document_indexing shape with an MCP (Model Context Protocol) server that streams over HTTP:

mcp_http: !pw.xpacks.llm.mcp_server.PathwayMcp
  name: "Streamable MCP Server"
  transport: "streamable-http"
  host: "0.0.0.0"
  port: 8068
  serve:
    - $document_store

Point any MCP-aware client (Claude Desktop, an MCP-aware coding agent, etc.) at http://localhost:8068 to access the live, always-current index as a tool.

Tip

The MCP server lives on port 8068, not 8000. The App container itself still binds 8000, but in the MCP template that port carries no useful endpoints — clients should target 8068.

4.9 Slides search — semantic search over presentations

slides_ai_search is the most elaborate stack. It indexes .pdf and .pptx files, parses every slide as an image, classifies each slide along a 15-category taxonomy (Problem Statement, Solution Overview, Pricing and Packaging, …), tags it (e.g. ['price', 'shopping']), detects language, and exposes search filtered by category and language.

The template ships three containers in its docker-compose.yml: the Pathway app (8000), a Streamlit UI (8501), and an nginx file server (8080) that serves the slide images and original PDFs back to the UI for rendering.

cd templates/slides_ai_search
cp .env.example .env  # add OPENAI_API_KEY
docker compose up --build

UI at http://localhost:8501 — it shows a query box, two multi-selects (categories and languages), and renders the top-k slides as images you can click through.

4.10 Drive Alert (legacy) — RAG with Slack notifications on document change

drive_alert is one of the two legacy templates — it predates the App-class refactor and uses procedural run(*, ...) kwargs with os.environ.get defaults. It indexes Google Drive, exposes a POST / REST endpoint on port 8080 (not 8000), and on top of that sends a Slack alert whenever a document change makes the answer to a previously-asked alerting query materially different.

Set up:

1. Create a GCP service account, download secrets.json, share the target Drive folder with the service-account email.
2. Set FILE_OR_DIRECTORY_ID to the Drive folder/file ID, GOOGLE_CREDS to secrets.json's path.
3. Create a Slack bot, install it to your workspace, copy the bot token to SLACK_ALERT_TOKEN, set SLACK_ALERT_CHANNEL_ID.
4. OPENAI_API_KEY for the LLM.

Ask the assistant Tell me and alert about the start date of the campaign for Magic Cola. The keyword "alert" registers the query for monitoring; the next time a Drive doc change shifts the answer, you receive a Slack notification.

Warning

The legacy templates default to gpt-3.5-turbo and text-embedding-ada-002, which are older than the App-class defaults. They also bind to port 8080 — if you run them next to a modern template that uses 8000, the ports do not collide, but if you reverse-proxy be aware of the mismatch.

4.11 Unstructured-to-SQL (legacy) — turn PDFs into a Postgres table you can query

unstructured_to_sql_on_the_fly is the second legacy template. Two pipelines run side by side inside one process:

1. Pipeline 1 (extraction): Watch data/quarterly_earnings/, send each PDF to OpenAI with a prompt that extracts the schema below into JSON, write the rows to Postgres and to ./data/quarterly_earnings.csv.
2. Pipeline 2 (query): Expose a REST endpoint on port 8080 that accepts natural-language questions, asks GPT to translate them to a constrained SELECT against the quarterly_earnings table, executes the SQL, returns the result.

The hard-coded extraction schema (you cannot change this without editing app.py):

Run it with the bundled compose file (which also starts a Postgres container with postgres/init-db.sql):

cd templates/unstructured_to_sql_on_the_fly
docker compose up --build

Browse http://localhost:8501 for the Streamlit UI, or curl -X POST http://localhost:8080/ -d '{"user":"u","query":"What is the average net income in Q2 2023?"}' -H "Content-Type: application/json".

Caution

Pipeline 2 builds and executes SQL produced by an LLM. The prompt constrains the grammar (SELECT only, restricted aggregators, no DDL) but you are still trusting an LLM with database access. Run against a read-only role and a database that only contains the extracted financial data — never against your production warehouse.

4.12 Cookbooks — self-RAG agents (Jupyter, learning-only)

cookbooks/self-rag-agents/ ships two notebooks:

• pathway_langgraph_agentic_rag.ipynb — building an agentic RAG with LangGraph and Pathway.
• pathway_deploy_langgraph_agents.ipynb — deploying the same graph as a service.

These are interactive tutorials. They are not deployable (no Dockerfile, no compose, no app.yaml) and they import from LangGraph, which is not in any template's requirements.txt. Treat them as walkthroughs, not products. Open them in Jupyter Lab with pip install pathway[all] langgraph jupyter && jupyter lab.

5. Common Tasks

5.1 Add your own PDFs to a RAG template

cp /path/to/your.pdf templates/question_answering_rag/data/
# Pathway picks it up live; no restart needed
curl -X POST http://localhost:8000/v2/list_documents

5.2 Swap to a cheaper or stronger OpenAI model

# in app.yaml
$llm: !pw.xpacks.llm.llms.OpenAIChat
  model: "gpt-4o-mini"   # was "gpt-4.1-mini"

Restart the container; the cache will need to rebuild for any prompts that hit the LLM differently.

5.3 Index a Google Drive folder instead of local files

In app.yaml, comment out the filesystem source, uncomment the Google Drive source, fill in $DRIVE_ID, and place gdrive_indexer.json (your service-account JSON) next to app.yaml:

$sources:
  - !pw.io.gdrive.read
    object_id: $DRIVE_ID
    service_user_credentials_file: gdrive_indexer.json
    file_name_pattern:
      - "*.pdf"
      - "*.pptx"
    object_size_limit: null
    with_metadata: true
    refresh_interval: 30

Set DRIVE_ID in .env.

5.4 Index SharePoint instead of local files

$sources:
  - !pw.xpacks.connectors.sharepoint.read
    url: $SHAREPOINT_URL
    tenant: $SHAREPOINT_TENANT
    client_id: $SHAREPOINT_CLIENT_ID
    cert_path: sharepointcert.pem
    thumbprint: $SHAREPOINT_THUMBPRINT
    root_path: $SHAREPOINT_ROOT
    with_metadata: true
    refresh_interval: 30

Place sharepointcert.pem next to app.yaml. Set the five env vars in .env.

5.5 Mix multiple sources in one index

$sources: is a list — repeat as many !pw.io... entries as you like; the document store unions all of them.

$sources:
  - !pw.io.fs.read
    path: data
    format: binary
    with_metadata: true
  - !pw.io.gdrive.read
    object_id: $DRIVE_ID
    service_user_credentials_file: gdrive_indexer.json

5.6 Persist the cache between restarts

# At the top level of app.yaml, alongside `host:` and `port:`
persistence_mode: !pw.PersistenceMode.PERSISTING
persistence_backend: !pw.persistence.Backend.filesystem
  path: ".Cache"

Mount the path as a Docker volume so it survives container rebuilds.

5.7 Customize the RAG prompt

question_answerer: !pw.xpacks.llm.question_answering.BaseRAGQuestionAnswerer
  llm: $llm
  indexer: $document_store
  prompt_template: |
    You are a contracts assistant. Use ONLY the provided context.
    Context: {context}
    Question: {query}
    Answer (cite the document path):

5.8 Run with a different port

host: "0.0.0.0"
port: 9000

Update the compose file's port mapping to match.

5.9 Wire the Streamlit UI to a remote Pathway app

The bundled UIs default PATHWAY_HOST=app (the compose service name). To point a UI at a Pathway service running elsewhere:

PATHWAY_HOST=rag.example.com PATHWAY_PORT=443 streamlit run ui.py

5.10 Halt on first error

For CI or strict production deployments where partial answers are worse than failure:

terminate_on_error: true

5.11 Use a local embedding model (cost / privacy)

Replace OpenAIEmbedder with SentenceTransformerEmbedder:

$embedder: !pw.xpacks.llm.embedders.SentenceTransformerEmbedder
  model: "mixedbread-ai/mxbai-embed-large-v1"
  call_kwargs:
    show_progress_bar: False

The first run downloads the model (~700 MB). After that it is fully local.

5.12 Build a private, end-to-end air-gapped stack

Use private_rag's app.yaml as a starting point. Confirm:

• $llm uses LiteLLMChat with an Ollama api_base.
• $embedder uses SentenceTransformerEmbedder.
• $parser is DoclingParser with table_parsing_strategy: "llm" only if you point multimodal_llm at a local model — otherwise drop it to "text".
• $sources: lists only the local filesystem.
• The Docker container runs with --network host (Linux) so it can reach localhost:11434.

5.13 Deploy to a cluster (multi-machine)

Multi-machine deployment uses Pathway's --addresses flag and requires a PATHWAY_LICENSE_KEY (Scale tier). Each machine runs the same app.py binary; the framework coordinates work across them. See the Admin Guide for the full deployment story.

Warning

Multi-machine clustering is gated by the Pathway license. Without a Scale key, --addresses raises a license error at startup.

6. Troubleshooting

Diagnostic decision tree

1. Container exits immediately. Check docker compose logs app. 90 % of the time it's a missing OPENAI_API_KEY or a YAML typo (extra="forbid" rejects unknown fields).
2. API responds but answers are empty. curl http://localhost:8000/v2/list_documents — if the list is empty, Pathway hasn't found your data; check $sources: paths and Docker volume mounts.
3. Answers are wrong but documents are indexed. curl -X POST http://localhost:8000/v1/retrieve — if retrieval is wrong, the embedder is the problem (try a stronger model or smaller chunks). If retrieval is right but the answer is wrong, the LLM is the problem (try gpt-4o).
4. Answers are slow. First-time queries pay full LLM + embedding cost. Cached results are sub-second; cold runs are seconds. If consistently slow, check the OpenAI capacity limit — capacity: 8 in $llm is a per-model rate limit.
5. Streamlit shows "Connection error". The UI cannot reach the Pathway app. Confirm PATHWAY_HOST matches your topology (compose: app; native: localhost).

Error: pydantic.ValidationError: 1 validation error for App ... extra inputs are not permitted

Cause: Your app.yaml includes a key that the App Pydantic model does not declare. extra="forbid" is set on every modern template.

Solution:

1. Compare your app.yaml against the bundled one for that template.
2. Remove any unknown top-level keys.
3. Confirm the spelling: question_answerer (not question_answer), persistence_mode (not persistance_mode).

Prevention: Start from the shipped app.yaml and edit only what you need.

Error: RuntimeError: OpenAI API key not provided

Cause: OPENAI_API_KEY is missing from the environment when OpenAIChat or OpenAIEmbedder instantiates.

Solution:

1. Confirm a .env file sits next to app.py (or app.yaml) and contains OPENAI_API_KEY=sk-....
2. For Docker, the compose file or --env-file .env flag must explicitly load it.
3. Restart the container — dotenv.load_dotenv() runs at process start.

Prevention: Use cp .env.example .env when an .env.example is present and immediately fill in the keys.

Error: KeyError: 'SHAREPOINT_URL'

Cause: You uncommented the SharePoint connector block in app.yaml but did not set the SharePoint env vars.

Solution:

1. Set SHAREPOINT_URL, SHAREPOINT_TENANT, SHAREPOINT_CLIENT_ID, SHAREPOINT_THUMBPRINT, SHAREPOINT_ROOT in .env.
2. Place the cert file at the path given by cert_path: (default sharepointcert.pem next to app.yaml).
3. Confirm the SharePoint app registration has application (not delegated) permissions for Sites.Read.All.

Prevention: The relevant .env.example files (when shipped) list every required variable.

Error: psycopg.OperationalError: connection to server on socket failed

Cause: unstructured_to_sql_on_the_fly cannot reach Postgres. Most often you ran docker run instead of the bundled compose file.

Solution: Run docker compose up --build from the template's directory. The compose file starts a Postgres container with postgres/init-db.sql already loaded.

Prevention: This template is the only one that requires a database; do not skip the compose file.

Error: ConnectionRefusedError: [Errno 111] ... ollama:11434

Cause: private_rag's container cannot reach the Ollama service.

Solution:

1. On the host: ollama serve (or run the official Docker image with -p 11434:11434).
2. On Mac/Windows Docker, edit app.yaml's api_base to http://host.docker.internal:11434.
3. On Linux Docker, run the container with --network host and keep api_base: "http://localhost:11434".

Prevention: Pull the Ollama model and start the daemon before docker compose up.

Error: Streamlit page loads but says "Connection error"

Cause: The UI's RAGClient(url=f"http://{PATHWAY_HOST}:{PATHWAY_PORT}") cannot reach the app.

Solution:

1. With compose: ensure PATHWAY_HOST=app and the app service is named app (or update both).
2. Native run: set PATHWAY_HOST=localhost.
3. Confirm the app container actually finished its first index build — the UI hits list_documents on load and that endpoint isn't ready until indexing completes.

Prevention: Always wait for the app log line "INFO Indexed N documents" before opening the UI.

Error: Slack returns 401 invalid_auth

Cause: drive_alert cannot post to Slack.

Solution:

1. Re-issue the bot token from the Slack app settings page.
2. Confirm the bot user has been installed to the workspace and invited to the channel.
3. Token must start with xoxb-, not xapp- or xoxp-.

Prevention: Slack rotates tokens on app reinstall — keep .env in sync.

Error: Index is stale, doesn't pick up new files

Cause: Pathway's filesystem connector watches the path you configured in $sources: — if Docker's bind mount points elsewhere or the path is wrong, changes are invisible.

Solution:

1. Confirm $sources: path: matches the in-container path.
2. Confirm the compose file mounts the host directory to that container path.
3. For Google Drive, refresh_interval: 30 controls the poll interval; lower it for faster updates.

Prevention: Always test by adding a small file and watching the app logs for the "Indexed N documents" increment.

Issue: GitHub #124 — Getting Started instructions are fragmented across templates

Cause: Each template has its own README, and there is no single entry-point doc.

Solution: Use this guide's selector table (section 4.1) to pick a template, then the per-template README for deep details. Treat templates/<name>/README.md as the canonical source for that template.

Prevention: Until upstream consolidates, this guide is the index — link or bookmark section 4.1.

Issue: GitHub #46 — OpenAI streaming response support requested

Cause: The shipped templates use OpenAIChat in non-streaming mode; the response is delivered as a single JSON body.

Solution: Streaming is a Pathway xpacks feature, not a template feature. If your downstream client requires SSE or chunked transfer, write a thin proxy in front of the Pathway REST endpoint that re-streams the response. Track the upstream issue on the Pathway core repo.

Issue: GitHub #82 — How to use a local embedding model

Cause: Default templates use OpenAI embeddings; users want fully local.

Solution: Either start from private_rag (local Ollama LLM + SentenceTransformer embedder) or copy its $embedder: block into any other template's app.yaml.

Prevention: Cross-reference the template defaults table (section 4.1) before starting a new project.

7. FAQ

General

Q: Is pathwaycom/llm-app a library?

No. It is a catalogue of ten reference applications (eight templates plus two cookbooks). The library doing the heavy lifting is Pathway, specifically its pathway.xpacks.llm module. The templates here are recipes that wire Pathway's xpacks pieces into deployable services.

Q: Where do I find Pathway's full API?

https://pathway.com/developers/api-docs/pathway-xpacks-llm/. The Admin Guide covers the slice of Pathway exercised by the templates in section 8.

Q: Can I use a model other than OpenAI?

Yes. The $llm: block can be OpenAIChat, LiteLLMChat (Ollama, Anthropic, Bedrock, Vertex, etc.), or any class under pathway.xpacks.llm.llms. private_rag shows the LiteLLM pattern.

Q: Are the templates production-ready?

They are production-shape — real Dockerfiles, real config, real connectors — but they default to demo settings (small capacity:, single-machine, demo license key). For production: rotate to a Scale license, raise capacity, add persistence, and put the API behind a reverse proxy with auth. See the Admin Guide.

Q: What is a cookbook vs a template?

A template is a deployable app (Dockerfile + compose + app.yaml). A cookbook is a Jupyter notebook tutorial. The cookbooks live under cookbooks/self-rag-agents/ and demonstrate LangGraph integration; they are for learning, not deployment.

Configuration

Q: How do I switch from gpt-4.1-mini to gpt-4o?

Edit app.yaml, change model: "gpt-4.1-mini" to model: "gpt-4o" under $llm:. Restart the container. Cached cache entries for the previous model still apply (the cache key includes the prompt, not the model — verify before caring about token cost).

Q: How do I disable caching?

In app.yaml set persistence_mode: null at the top level. Per-component caches (cache_strategy: !pw.udfs.DefaultCache {}) cache LLM and embedder calls in-process for the lifetime of the run regardless.

Q: Where does the cache live?

By default, the container's working directory under ./Cache. Change with persistence_backend: !pw.persistence.Backend.filesystem and path: "/data/cache". Mount that path as a Docker volume to make it persistent.

Q: How do I customize the RAG prompt?

Add prompt_template: under question_answerer:. Use {query} and {context} placeholders. See section 5.7.

Q: What does extra="forbid" mean?

The Pydantic App model rejects YAML keys it doesn't recognise — a typo fails fast at startup with a clear error rather than silently ignoring your override.

Connectors and data sources

Q: How many connectors do the templates use?

Five: filesystem (pw.io.fs.read), Google Drive (pw.io.gdrive.read), SharePoint (pw.xpacks.connectors.sharepoint.read), Postgres (pw.io.postgres.write, output side, unstructured_to_sql_on_the_fly), and Slack (pw.io.slack.send_alerts, output, drive_alert). The parent Pathway library has dozens more — these templates use a subset.

Q: Can I use S3 as a source?

Pathway supports pw.io.s3.read (mentioned in drive_alert's code comments). It is not used in any shipped app.yaml, but you can add it under $sources: the same way the GDrive and SharePoint blocks are added.

Q: Does it support Postgres as a source?

Pathway supports pw.io.postgres.read; none of the templates use it as input. unstructured_to_sql_on_the_fly writes to Postgres, not reads from it.

Q: How fresh is the index?

The filesystem connector polls effectively continuously (Pathway uses a streaming dataflow). Google Drive defaults to refresh_interval: 30 seconds. SharePoint same.

Deployment

Q: Do I need Docker?

No, but it is the recommended path. Native Python works — pip install pathway[all] python-dotenv && python app.py — but you are responsible for matching system libraries (Pathway includes a Rust core that links against system OpenSSL and similar).

Q: Is GPU required?

No. Templates default to CPU-friendly settings. private_rag benefits from a GPU because of the local embedder, but does not require one.

Q: What ports do the templates use?

App-class templates: 8000 for the Pathway REST API. The MCP template adds 8068. Streamlit UIs: 8501. Legacy templates: 8080. The slides template adds an nginx file server on 8080 internally.

Q: How do I deploy on Kubernetes?

The templates do not ship K8s manifests. Build the bundled Dockerfile, push to a registry, write a Deployment + Service per template. For multi-machine clustering, see the Admin Guide on --addresses and the Pathway license.

Cost and limits

Q: How expensive is a default RAG run?

Indexing a small PDF set with text-embedding-3-small is fractions of a cent. Each query at gpt-4.1-mini is roughly the cost of one prompt + ~500-token response. The first answer is most expensive (cold cache); repeated queries hit the cache and cost nothing.

Q: What is the capacity: field on $llm:?

A simple in-process rate limit on concurrent OpenAI calls. The default is 8. Lower it if you hit OpenAI rate limits; raise it for faster indexing on a high-tier account.

Q: Why does the app retry?

retry_strategy: !pw.udfs.ExponentialBackoffRetryStrategy max_retries: 6 is set on every LLM and embedder. Transient OpenAI errors are retried with exponential backoff before the pipeline fails.

Security and privacy

Q: Are documents sent to OpenAI?

By default, yes — both for embedding (so the index can be built) and for question answering. private_rag and document_indexing (with mxbai-embed-large-v1) keep documents local.

Q: Can I run this in a regulated environment?

Yes — start from private_rag and confirm every external service is replaced with a local one. The Admin Guide section 7 enumerates the exact things to audit.

Q: How do I rotate the OpenAI key?

Update .env, then docker compose restart app. The key is read once at process start.

Cookbooks and integrations

Q: Why does cookbooks/self-rag-agents/ import LangGraph?

The cookbooks demonstrate combining Pathway (for the live document store) with LangGraph (for the agent state machine). LangGraph is a dependency only of the notebooks, not of any template's requirements.txt.

Q: Is there an integration with LangChain?

LangChain is not bundled. You can use pw.xpacks.llm.question_answering.RAGClient (used by the Streamlit UIs) from any Python client, including LangChain.