Quivr User Guide

Quivr (quivr-core) is an opinionated Retrieval-Augmented Generation (RAG) library for Python. You give it your files, it builds a searchable "brain" from them, and you ask questions in natural language — Quivr retrieves the relevant passages and has a large language model (LLM) answer using only those passages, with citations.

The project's tagline is "We take care of the RAG so you can focus on your product." You install one package, point it at your documents, and you have a working question-answering system in five lines of code. When you outgrow the defaults, every stage — the LLM, the embedding model, the vector store, the document parsers, and the entire retrieval workflow — is swappable through a single YAML configuration file.

Note

This guide documents quivr-core, the open-source Python package published to PyPI. It is the engine behind the hosted product at quivr.com, but everything here runs locally in your own Python process — no Quivr account or server is required.

1. Quick Start

1.1 Install

pip install quivr-core

Important

Quivr requires Python 3.11 or newer (the package metadata sets requires-python = ">= 3.11"). The project README mentions 3.10, but the packaged quivr-core is built against 3.11+. Use 3.11 or later to avoid install-time resolution errors.

1.2 Set your API key

By default Quivr uses OpenAI for both the chat model (gpt-4o) and embeddings (text-embedding). Export your key before running any code:

macOS / Linux

export OPENAI_API_KEY="sk-..."

Windows (PowerShell)

$env:OPENAI_API_KEY = "sk-..."

In Python

import os
os.environ["OPENAI_API_KEY"] = "sk-..."

1.3 Ask your first question

import tempfile
from quivr_core import Brain

with tempfile.NamedTemporaryFile(mode="w", suffix=".txt") as temp_file:
    temp_file.write("Gold is a liquid of blue-like colour.")
    temp_file.flush()

    brain = Brain.from_files(
        name="test_brain",
        file_paths=[temp_file.name],
    )

    answer = brain.ask("what is gold? answer in french")
    print("answer:", answer.answer)

Expected output (the model answers strictly from your file, so it reports the made-up fact):

answer: L'or est un liquide de couleur bleutée.

Tip

Quivr is grounded: its default prompt instructs the model to use only the provided context and never its own prior knowledge. That is why the answer above repeats the (false) statement from your file instead of correcting it. This is the intended behaviour for trustworthy document Q&A.

That is the whole loop: create a brain from files → ask questions. The rest of this guide explains how to feed it real documents, stream answers, customize the LLM and retrieval, and save your brain for reuse.

2. Core Concepts

You only need four ideas to use Quivr effectively.

Brain : The central object. A Brain bundles your documents (in storage), their searchable vector index (in a vector store), an LLM for answering, and an embedder for indexing. You create one with a factory method and then call .ask() on it.

Ingestion : The one-time process of turning your raw files into searchable knowledge: read each file → parse it to text with the right processor → split the text into overlapping chunks → embed each chunk into a vector → store the vectors. Brain.from_files() does all of this for you.

Retrieval : What happens on every question: your question is embedded, the most similar chunks are fetched from the vector store, optionally re-ranked, and packed into the LLM's prompt as context.

Workflow : The ordered sequence of steps that runs for each question (filter history → rewrite question → retrieve → generate answer). The default workflow is good enough for most uses, but you can redefine it entirely in a YAML file to add web search, tools, or dynamic retrieval.

        Ingestion (once)                         Retrieval (per question)
  files ─► parse ─► chunk ─► embed ─► vector  ─►  question ─► retrieve chunks ─► LLM ─► answer
                                      store

3. Installation in Depth

3.1 The base install

pip install quivr-core

This pulls in the default stack: LangChain + LangGraph (orchestration), langchain-openai (default LLM and embeddings), faiss-cpu (default vector store), tiktoken and transformers (tokenizers), and clients for Anthropic, Mistral, Gemini, Groq, and Cohere so you can switch providers without extra installs.

3.2 Optional parser dependencies

Quivr can read many file types, but some parsers need extra libraries. If you try to ingest a file type whose parser isn't installed, Quivr logs a warning and falls back to the next available parser; if none works it raises an error telling you exactly what to install. Install the heavy document parsers with the unstructured library:

pip install "unstructured[all-docs]"

Note

Plain .txt files always work with the base install (via the built-in SimpleTxtProcessor). PDFs work out of the box only if you run a Tika server (see §6.3) or install MegaParse / unstructured.

3.3 Verify the install

from quivr_core import Brain
print(Brain)

<class 'quivr_core.brain.brain.Brain'>

4. Everyday Usage

4.1 Build a brain from your files

Brain.from_files() is the synchronous, do-everything constructor. Point it at a list of paths:

from quivr_core import Brain

brain = Brain.from_files(
    name="my smart brain",
    file_paths=["./my_first_doc.pdf", "./my_second_doc.txt"],
)

Warning

Creating a brain with an empty file_paths list and no vector_db raises ValueError: can't initialize brain without documents. Either provide at least one file or pass your own vector_db.

4.2 Ask a question

brain.ask() returns a response object whose .answer is the text and whose .metadata carries citations, follow-up questions, and sources.

response = brain.ask("What are the key findings of the report?")
print(response.answer)
print("Sources:", response.metadata.sources)
print("Suggested follow-ups:", response.metadata.followup_questions)

The report's key findings are ...
Sources: [Document(...), Document(...)]
Suggested follow-ups: ['How was the data collected?', 'What is the sample size?']

4.3 See what's inside a brain

brain.print_info()

╭───────────────── Brain Info ─────────────────╮
│ 📊 Brain Information                          │
│ ├── 🆔 ID: 7f3c...                            │
│ ├── 🧠 Brain Name: my smart brain            │
│ ├── 📁 Files                                  │
│ │   ├── Storage Type: transparent_storage    │
│ │   └── Number of Files: 2                    │
│ ├── 💬 Chats                                  │
│ │   ├── Number of Chats: 1                    │
│ │   └── Current Chat History Length: 0        │
│ └── 🤖 LLM                                    │
│     ├── Model: gpt-4o                         │
│     ├── Temperature: 0.3                      │
│     ├── Max Tokens: 4096                      │
│     └── Supports Function Calling: True       │
╰───────────────────────────────────────────────╯

4.4 Search without asking

If you just want the relevant passages (no LLM answer), use asearch. It returns scored chunks, nearest first:

results = await brain.asearch("Why does everybody love Quivr?", n_results=5)
for r in results:
    print(r.distance, r.chunk.page_content[:80])

Each result is a SearchResult with .chunk (a LangChain Document) and .distance (lower = more similar).

5. Streaming and Chat

5.1 Stream the answer token by token

For responsive UIs, stream chunks as the model produces them. ask_streaming is an async generator:

from uuid import uuid4

async for chunk in brain.ask_streaming("Summarize chapter 3.", run_id=uuid4()):
    print(chunk.answer, end="", flush=True)

Each yielded ParsedRAGChunkResponse has an incremental .answer string and a .metadata object. The final chunk has .last_chunk == True and carries the complete metadata (citations, sources, the Langfuse trace URL if observability is enabled).

Tip

run_id is a uuid4() you generate per question. It ties together logs and observability traces for that one answer — handy when debugging which retrieval led to which response.

5.2 Conversations remember context

A brain keeps a default chat history. Each call to ask / aask / ask_streaming appends the question and answer, so follow-up questions understand earlier turns automatically:

brain.ask("Who is the author?")          # answered from the docs
brain.ask("What else did they write?")   # "they" is resolved from history

Quivr filters history to fit the model's context window and, in the default workflow, rewrites each follow-up into a standalone question before retrieval.

5.3 Async vs. sync

Quivr is async-first. Every operation has an a-prefixed coroutine and a synchronous wrapper:

Caution

The synchronous wrappers call asyncio.get_event_loop().run_until_complete(...). Inside an environment that already has a running event loop — a Jupyter notebook, an async def, or an ASGI web handler — this raises RuntimeError: There is no current event loop or This event loop is already running (community issue #3650). In those contexts always use the async methods (afrom_files, aask, ask_streaming).

6. Working With Documents

6.1 Supported file types

Quivr maps file extensions to processors. Out of the box it knows how to handle:

Note

Most non-.txt formats are read through the unstructured library or MegaParse, so install those extras (§3.2) to enable them. If a file's extension isn't recognized, Quivr warns "extension isn't recognized — make sure you have registered a parser" and skips or errors depending on skip_file_error.

6.2 Control how documents are chunked

Chunking decides how your text is sliced before embedding. Smaller chunks give more precise retrieval; larger chunks give more context per hit. Configure it with a YAML ingestion file:

# basic_ingestion_workflow.yaml
parser_config:
  megaparse_config:
    strategy: "auto"        # auto | fast | hi_res
    pdf_parser: "unstructured"
  splitter_config:
    chunk_size: 400         # in tokens
    chunk_overlap: 100      # in tokens

from quivr_core import Brain
from quivr_core.config import IngestionConfig

ingestion_config = IngestionConfig.from_yaml("./basic_ingestion_workflow.yaml")
processor_kwargs = {
    "megaparse_config": ingestion_config.parser_config.megaparse_config,
    "splitter_config": ingestion_config.parser_config.splitter_config,
}

brain = Brain.from_files(
    name="my smart brain",
    file_paths=["./my_first_doc.pdf", "./my_second_doc.txt"],
    processor_kwargs=processor_kwargs,
)

The default split is 400 tokens per chunk with 100 tokens of overlap.

6.3 PDFs via a Tika server

The default PDF processor talks to an Apache Tika server. The quickest way to get one:

docker run -d -p 9998:9998 apache/tika

Unable to find image 'apache/tika:latest' locally
latest: Pulling from apache/tika
...
Status: Downloaded newer image for apache/tika:latest

Quivr looks for the server at http://localhost:9998/tika by default; override it with the TIKA_SERVER_URL environment variable.

6.4 Build from LangChain documents

If you already have langchain_core.documents.Document objects (e.g. from your own loader), skip file parsing entirely:

from langchain_core.documents import Document
from quivr_core import Brain

documents = [Document(page_content="Hello, world!")]
brain = await Brain.afrom_langchain_documents(
    name="My Brain",
    langchain_documents=documents,
)

7. Customizing the LLM

7.1 Switch providers

Quivr supports OpenAI, Azure OpenAI, Anthropic, Mistral, Google Gemini, and Groq, plus any OpenAI-compatible local server (such as Ollama). Build an LLMEndpoint from an LLMEndpointConfig and hand it to the brain:

from quivr_core import Brain
from quivr_core.llm import LLMEndpoint
from quivr_core.rag.entities.config import LLMEndpointConfig

llm = LLMEndpoint.from_config(
    LLMEndpointConfig(model="claude-3-5-sonnet", supplier="anthropic")
)

brain = Brain.from_files(name="brain", file_paths=["doc.pdf"], llm=llm)

Set the matching API key in the environment. Quivr derives the variable name from the supplier:

Tip

You usually don't need to set supplier — Quivr infers it from the model name (e.g. any claude-* model → Anthropic, any gpt-*/o1/o3 → OpenAI). Setting it explicitly is only required for ambiguous or custom model names.

7.2 Use a local model (Ollama)

Point Quivr at any OpenAI-compatible endpoint with llm_base_url:

from quivr_core.rag.entities.config import LLMEndpointConfig

config = LLMEndpointConfig(
    model="llama2",
    llm_base_url="http://localhost:8441",
    llm_api_key="not-needed",
)

Note

Some models — llama2, ollama3, and the test model — are treated as not supporting function calling. Quivr automatically falls back to a plain (non-tool) answering path for them, so citations may be unavailable with those models.

7.3 Key LLM options

Important

Quivr automatically caps max_context_tokens and max_output_tokens to the real limits of the chosen model. If you ask for more than the model allows, Quivr logs a warning and lowers the value. If you set them below 4096, it raises ValueError: max_context_tokens is too low — the minimum for both is 4096.

8. Customizing Retrieval With Workflows

The real power of Quivr is the workflow YAML: a description of the steps each question goes through, expressed as a graph of named nodes and edges. Load it once and pass it to ask.

8.1 The standard RAG workflow

# basic_rag_workflow.yaml
workflow_config:
  name: "standard RAG"
  nodes:
    - name: "START"
      edges: ["filter_history"]
    - name: "filter_history"
      edges: ["rewrite"]
    - name: "rewrite"
      edges: ["retrieve"]
    - name: "retrieve"
      edges: ["generate_rag"]
    - name: "generate_rag"     # last node — its output is streamed to the user
      edges: ["END"]
      tools:
        - name: "cited_answer"

max_history: 10                # conversation turns kept in context

llm_config:
  max_output_tokens: 4000
  temperature: 0.7

from quivr_core import Brain
from quivr_core.rag.entities.config import RetrievalConfig

brain = Brain.from_files(name="my brain", file_paths=["./doc.pdf"])
retrieval_config = RetrievalConfig.from_yaml("./basic_rag_workflow.yaml")

answer = brain.ask("What is the conclusion?", retrieval_config=retrieval_config)
print(answer.answer)

Caution

Use max_output_tokens (and/or max_context_tokens) in the llm_config block. Some examples in the project's own README show max_input_tokens — that key does not exist and Quivr will reject it with a Pydantic validation error because the config models forbid unknown fields. When in doubt, copy the YAML from the core/example_workflows/ folder in the repository, which uses the correct keys.

8.2 What the standard nodes do

8.3 Tune retrieval breadth and history

These top-level keys live in the same YAML file:

9. Advanced Features

9.1 Re-rank retrieved chunks for accuracy

A re-ranker re-scores retrieved chunks so the most relevant ones reach the LLM. Quivr supports Cohere and Jina:

reranker_config:
  supplier: "cohere"                  # or "jina"
  model: "rerank-multilingual-v3.0"
  top_n: 5                            # chunks kept after re-ranking
  relevance_score_threshold: 0.01     # drop chunks below this score (0–1)

Set COHERE_API_KEY (or JINA_API_KEY) in the environment. If the supplier is set but the key is missing, Quivr raises a clear ValueError naming the variable to set.

Tip

A relevance_score_threshold of 0.01 works well with both Cohere and Jina. Combined with the dynamic_retrieve node, this lets Quivr automatically decide how many chunks are actually relevant instead of always sending a fixed number.

9.2 Add web search

Give the agent a web-search tool (powered by Tavily) so it can answer questions your documents don't cover:

workflow_config:
  name: "RAG with web search"
  available_tools:
    - "web search"
  nodes:
    - name: "START"
      conditional_edge:
        routing_function: "routing_split"
        conditions: ["edit_system_prompt", "filter_history"]
    - name: "edit_system_prompt"
      edges: ["filter_history"]
    - name: "filter_history"
      edges: ["dynamic_retrieve"]
    - name: "dynamic_retrieve"
      conditional_edge:
        routing_function: "tool_routing"
        conditions: ["run_tool", "generate_rag"]
    - name: "run_tool"
      edges: ["generate_rag"]
    - name: "generate_rag"
      edges: ["END"]
      tools:
        - name: "cited_answer"

Set TAVILY_API_KEY in your environment. With this workflow the agent detects when the documents are insufficient and searches the web automatically.

9.3 Save and reload a brain

Building a brain re-parses and re-embeds every file, which costs time and API calls. Save the finished brain (vectors + config) and reload it instantly:

# Save (async)
await brain.save("~/.local/quivr")

# Reload later (sync) — no re-ingestion, no embedding calls
from quivr_core import Brain
brain = Brain.load("~/.local/quivr/brain_<id>")
brain.print_info()

save writes a config.json plus the FAISS index. load reconstructs the brain from them.

Caution

Loading a saved brain deserializes the FAISS index with Python pickle (allow_dangerous_deserialization=True). Only load brains you created yourself or fully trust — a malicious saved brain could execute arbitrary code. Never load a brain downloaded from an untrusted source.

Note

Saving currently supports the FAISS vector store and OpenAI embedder only. Other backends raise "can't serialize ... for now". Storage backends supported for save/load are TransparentStorage and LocalStorage.

9.4 Keep files on disk with LocalStorage

The default TransparentStorage holds file references in memory. To copy files into a managed directory, use LocalStorage:

from quivr_core import Brain
from quivr_core.storage.local_storage import LocalStorage

brain = Brain.from_files(
    name="brain",
    file_paths=["doc.pdf"],
    storage=LocalStorage(),
)

By default LocalStorage writes to ~/.cache/quivr/files (override with the QUIVR_LOCAL_STORAGE environment variable) and copies files; pass copy_flag=False to symlink instead.

10. Example Applications

The repository ships runnable examples under examples/. They are the fastest way to see Quivr in a real app.

Run the Chainlit chatbot:

cd examples/chatbot
rye sync
rye run chainlit run main.py

Your app is available at http://localhost:8000

Tip

Each example folder has its own basic_rag_workflow.yaml. Edit it to experiment with different retrieval strategies without touching any Python code.

11. Troubleshooting

Error: There is no current event loop in thread 'MainThread'

Cause: You called a synchronous method (Brain.from_files, brain.ask) from inside an environment that already runs an event loop — a Jupyter notebook, an async def, or a web framework handler. (Community issue #3650.)

Solution: Use the async variants and await them:

brain = await Brain.afrom_files(name="b", file_paths=["doc.txt"])
answer = await brain.aask("question", run_id=uuid4())

Prevention: In any async or notebook context, default to afrom_files / aask / ask_streaming.

Error: can't initialize brain without documents

Cause: You called from_files with an empty file_paths list and did not supply your own vector_db.

Solution: Provide at least one readable file, or pass a pre-built vector_db.

Prevention: Validate that your file list is non-empty before constructing the brain.

Error: max_context_tokens is too low / max_output_tokens is too low

Cause: You set a token limit below the minimum of 4096 in llm_config.

Solution: Set both max_context_tokens and max_output_tokens to 4096 or higher.

Prevention: Leave them at their defaults (20000 / 4096) unless you have a specific reason to change them.

Error: Pydantic validation error on an unknown YAML key (e.g. max_input_tokens)

Cause: The config models reject unknown fields. max_input_tokens is not a valid key.

Solution: Use max_output_tokens and/or max_context_tokens in the llm_config block.

Prevention: Base your YAML on the files in core/example_workflows/, which use the correct keys.

Error: The API key for supplier '<x>' is not set

Cause: The environment variable for your LLM or re-ranker provider is missing.

Solution: Export the matching key — OPENAI_API_KEY, ANTHROPIC_API_KEY, COHERE_API_KEY, TAVILY_API_KEY, etc.

Prevention: Keep a .env file and load it (e.g. with python-dotenv) at startup.

Error: can't send parse request to tika server (PDFs)

Cause: No Apache Tika server is reachable for PDF parsing.

Solution: Start one with docker run -d -p 9998:9998 apache/tika, or set TIKA_SERVER_URL to your server. Alternatively install MegaParse / unstructured to use a different PDF path.

Prevention: Ensure your PDF parsing backend is running before ingesting PDFs.

Error: file ... already uploaded (FileExistsError)

Cause: With LocalStorage, you tried to ingest a file whose content (SHA-1) was already added.

Solution: Remove the duplicate, or pass exists_ok=True when uploading.

Prevention: De-duplicate your file list before ingestion.

Answers are empty or "I don't have the answer"

Cause: The retrieved context didn't contain the answer — Quivr is grounded and refuses to answer from outside the documents.

Solution: Ingest more relevant documents, increase k / top_n, lower relevance_score_threshold, or add the web-search tool (§9.2).

Prevention: Verify your documents actually cover the questions you expect, and tune retrieval breadth.

12. FAQ

Q: Do I need a Quivr account or the quivr.com service? No. quivr-core runs entirely in your Python process. The hosted service is separate.

Q: Which LLM providers are supported? OpenAI, Azure OpenAI, Anthropic, Mistral, Google Gemini, and Groq out of the box, plus any OpenAI-compatible endpoint (e.g. Ollama) via llm_base_url.

Q: What is the default model and vector store? gpt-4o for answering, OpenAIEmbeddings for indexing, and an in-memory FAISS index for storage.

Q: Can I use my own vector store (Postgres/pgvector, etc.)? Yes — pass any LangChain VectorStore as vector_db. Note that save()/load() currently only persist FAISS.

Q: How do I make answers cite their sources? Keep the cited_answer tool in your workflow's final node and use a model that supports function calling (most OpenAI/Anthropic models do). Citations appear in response.metadata.citations and .sources.

Q: How do I get follow-up question suggestions? They're produced automatically by the cited_answer tool and returned in response.metadata.followup_questions (up to three).

Q: Can Quivr search the internet? Yes, via the Tavily web-search tool in a custom workflow (§9.2). Set TAVILY_API_KEY.

Q: How do I avoid re-embedding my files every run? Call await brain.save(path) once, then Brain.load(path) afterwards. Reloading skips ingestion and embedding entirely.

Q: Does Quivr keep conversation context? Yes. Each brain has a chat history; follow-up questions are interpreted in light of earlier turns and rewritten into standalone queries before retrieval.

Q: What languages are supported? Document language is auto-detected (≈170 languages) and the default prompt answers in the user's language unless told otherwise.

Q: Is the project still maintained? Activity has slowed (see community discussion #3681). quivr-core remains usable and is the supported, documented package. Pin a version in production.