# LlamaIndex User Guide

LlamaIndex is an open-source **data framework for building LLM and agentic applications in Python**. It connects your private data — PDFs, documents, APIs, SQL databases, and more than 300 other sources — to large language models so you can ask questions over it, build retrieval-augmented generation (RAG) pipelines, and run autonomous agents grounded in your own content.

This guide shows you how to install LlamaIndex, load your data, build an index, query it, and customize every part of the pipeline using its plugin ecosystem. It is written for application developers who want to *use* LlamaIndex. If you operate, deploy, or extend the framework, see the companion **Admin Guide**.

> [!NOTE]
> LlamaIndex is a **Python library**, not a command-line application. You build with it by importing classes into your own Python code. The only shipped command-line tool, `llama-dev`, is a contributor utility for the source repository (covered in the Admin Guide), not something end users run.

---

## 1. Quick Start

If you already have Python 3.10 or newer and an OpenAI API key, you can index a folder of documents and query it in five lines of code.

```bash
pip install llama-index
```

```python
import os

os.environ["OPENAI_API_KEY"] = "YOUR_OPENAI_API_KEY"

from llama_index.core import VectorStoreIndex, SimpleDirectoryReader

documents = SimpleDirectoryReader("YOUR_DATA_DIRECTORY").load_data()
index = VectorStoreIndex.from_documents(documents)

query_engine = index.as_query_engine()
response = query_engine.query("What is this collection of documents about?")
print(response)
```

```text
This collection of documents describes ... (an answer synthesized from your files,
grounded in the most relevant passages retrieved from YOUR_DATA_DIRECTORY)
```

What happened in those lines:

1. `SimpleDirectoryReader` loaded every supported file in the directory into a list of `Document` objects.
2. `VectorStoreIndex.from_documents` split the documents into chunks (**nodes**), embedded each chunk, and stored the vectors in an in-memory index.
3. `as_query_engine()` created a query engine that, on each `query()`, retrieves the most relevant chunks and asks the LLM to answer using them as context.

> [!TIP]
> The `llama-index` starter package bundles core plus the OpenAI LLM and embedding integrations, so the snippet above works with no extra installs. To use a different provider (Ollama, Anthropic, Hugging Face, etc.), install that provider's integration package — see [Section 7](#7-the-integration-ecosystem).

---

## 2. Installation

LlamaIndex offers two installation paths. Choose based on how much control you want over which providers are pulled in.

### 2.1 Starter install (recommended for getting started)

The umbrella `llama-index` package includes core plus a curated selection of integrations so you can be productive immediately.

```bash
pip install llama-index
```

This installs:

| Package | Role |
|---------|------|
| `llama-index-core` | The framework: indexes, retrievers, query engines, schema |
| `llama-index-llms-openai` | OpenAI LLM integration (default LLM) |
| `llama-index-embeddings-openai` | OpenAI embedding integration (default embeddings) |
| `nltk` | Tokenization support used by core |

> [!IMPORTANT]
> The starter package defaults to OpenAI for both the LLM and embeddings. You must set the `OPENAI_API_KEY` environment variable before running, or supply a different provider via `Settings` (see [Section 6](#6-customizing-the-llm-and-embeddings)).

### 2.2 Customized install (recommended for production)

Install core alone, then add only the integration packages your application needs. This keeps your dependency tree small and avoids pulling in OpenAI if you don't use it.

```bash
pip install llama-index-core
pip install llama-index-llms-ollama
pip install llama-index-embeddings-huggingface
pip install llama-index-vector-stores-qdrant
```

There are **over 300 integration packages** on [LlamaHub](https://llamahub.ai/) that work seamlessly with core, letting you choose your preferred LLM, embedding model, and vector store providers.

### 2.3 Requirements

| Requirement | Value |
|-------------|-------|
| Python | `>=3.10, <4.0` |
| Package manager | `pip` (or `uv`, `poetry`) |
| License | MIT |
| Latest version (this build) | `llama-index` 0.14.22 / `llama-index-core` 0.14.x |

> [!NOTE]
> LlamaIndex bundles a small `_static` folder inside `llama-index-core` containing cached `nltk` and `tiktoken` data, so it can run in environments with restrictive disk access at runtime. These assets are verifiable with GitHub build-provenance attestation; see the Admin Guide's security section.

### 2.4 Verify your installation

```python
import llama_index.core
print(llama_index.core.__version__)
```

```text
0.14.22
```

---

## 3. Understanding the Import Convention

LlamaIndex is namespaced so that you can always tell core code from integration code by reading the import.

```python
# typical pattern
from llama_index.core.xxx import ClassABC          # core submodule xxx
from llama_index.xxx.yyy import SubclassABC          # integration yyy for submodule xxx

# concrete example
from llama_index.core.llms import LLM                # the abstract LLM interface (core)
from llama_index.llms.openai import OpenAI           # a concrete LLM (integration)
```

The rule:

- An import path **with** `core` (e.g. `llama_index.core.llms`) comes from the framework itself.
- An import path **without** `core` (e.g. `llama_index.llms.openai`, `llama_index.vector_stores.qdrant`) comes from a separately installed integration package.

> [!TIP]
> If you get a `ModuleNotFoundError` for an import like `llama_index.llms.anthropic`, it almost always means you haven't installed that integration. Run `pip install llama-index-llms-anthropic`.

---

## 4. Loading Your Data

Everything in LlamaIndex starts with a `Document` — a container for a piece of source content plus metadata.

### 4.1 Load files from a directory

`SimpleDirectoryReader` is the fastest way to ingest local files. It auto-detects file types and reads every supported file in a folder.

```python
from llama_index.core import SimpleDirectoryReader

documents = SimpleDirectoryReader("./data").load_data()
print(f"Loaded {len(documents)} documents")
```

```text
Loaded 12 documents
```

Useful options:

```python
documents = SimpleDirectoryReader(
    input_dir="./data",
    recursive=True,                       # descend into subdirectories
    required_exts=[".pdf", ".md", ".txt"],  # only these extensions
).load_data()
```

### 4.2 Create documents manually

When your text comes from somewhere other than files (a database row, an API response, a string), build `Document` objects directly.

```python
from llama_index.core import Document

doc = Document(
    text="LlamaIndex connects your data to large language models.",
    metadata={"source": "notes", "author": "team"},
)
```

Metadata is powerful: it is injected into the text shown to the LLM and used for filtering during retrieval. You can control exactly what metadata the embedding model and the LLM each see:

```python
doc.excluded_embed_metadata_keys = ["author"]   # hide from the embedding model
doc.excluded_llm_metadata_keys = ["source"]     # hide from the LLM prompt
```

### 4.3 Load from other sources (readers / connectors)

Local files are only the beginning. LlamaIndex ships **data connectors** (also called *readers*) for hundreds of sources — Notion, Slack, Google Drive, S3, SQL databases, web pages, GitHub, Confluence, MongoDB, and many more. Each lives in its own integration package.

```bash
pip install llama-index-readers-web
```

```python
from llama_index.readers.web import SimpleWebPageReader

documents = SimpleWebPageReader().load_data(
    ["https://example.com/article"]
)
```

The reader naming convention is always `llama-index-readers-<source>`. Browse the full catalog on [LlamaHub](https://llamahub.ai/).

---

## 5. Building and Querying an Index

An **index** is a data structure built over your documents that makes them efficiently searchable by an LLM. The most common is `VectorStoreIndex`, which uses semantic (embedding) search.

### 5.1 Build a vector index

```python
from llama_index.core import VectorStoreIndex, SimpleDirectoryReader

documents = SimpleDirectoryReader("./data").load_data()
index = VectorStoreIndex.from_documents(documents)
```

When you build the index, LlamaIndex automatically:

- Splits each document into **nodes** (chunks) — by default about **1024 tokens** each with **20 tokens of overlap**.
- Generates an embedding vector for each node.
- Stores nodes and vectors for retrieval.

### 5.2 Query the index

```python
query_engine = index.as_query_engine()
response = query_engine.query("How do I configure authentication?")
print(response)
```

By default the query engine retrieves the **2 most similar** nodes and passes them to the LLM as context. You can change this:

```python
query_engine = index.as_query_engine(similarity_top_k=5)
```

### 5.3 Inspect the sources behind an answer

A response object carries the source nodes it was built from, so you can show citations or verify grounding.

```python
response = query_engine.query("What is the refund policy?")

print(response.response)            # the synthesized answer text
for node in response.source_nodes:  # the retrieved chunks
    print(node.score, node.node.metadata.get("file_name"))
    print(node.node.get_content()[:200])
```

### 5.4 Chat instead of one-shot Q&A

For multi-turn conversations that remember context, use a chat engine instead of a query engine.

```python
chat_engine = index.as_chat_engine()
print(chat_engine.chat("Summarize the onboarding document."))
print(chat_engine.chat("Now list the action items from it."))
```

### 5.5 Retrieve without synthesizing an answer

If you only want the relevant chunks (for example, to feed them into your own prompt), use a retriever directly.

```python
retriever = index.as_retriever(similarity_top_k=3)
nodes = retriever.retrieve("payment failures")
for n in nodes:
    print(n.score, n.get_content()[:100])
```

### 5.6 Run queries asynchronously

Every major entry point has an async counterpart prefixed with `a` (`aquery`, `aretrieve`, `achat`). Use these inside async applications.

```python
response = await query_engine.aquery("What changed in the latest release?")
```

> [!WARNING]
> Calling a synchronous method like `query()` from inside an already-running event loop (for example, a Jupyter notebook or an async web handler) can raise a *"Detected nested async"* error. Use the async entry methods (`aquery`, `aretrieve`, `achat`) in those contexts, or apply `nest_asyncio.apply()`. See [Troubleshooting](#10-troubleshooting).

---

## 6. Customizing the LLM and Embeddings

By default LlamaIndex uses OpenAI. To use any other provider, install its integration and assign it. There are two scopes: **global** (applies everywhere) and **local** (applies to one call).

### 6.1 Global configuration with `Settings`

`Settings` is the global configuration object. Set it once at startup and every index, retriever, and query engine picks it up.

```python
from llama_index.core import Settings, VectorStoreIndex, SimpleDirectoryReader
from llama_index.embeddings.huggingface import HuggingFaceEmbedding
from llama_index.llms.ollama import Ollama
from transformers import AutoTokenizer

# set the LLM (served locally through Ollama)
Settings.llm = Ollama(model="llama-3.1:latest", request_timeout=360.0)

# set the tokenizer to match the LLM
Settings.tokenizer = AutoTokenizer.from_pretrained(
    "meta-llama/Llama-3.1-8B-Instruct"
)

# set the embedding model (runs locally via Hugging Face)
Settings.embed_model = HuggingFaceEmbedding(model_name="BAAI/bge-small-en-v1.5")

documents = SimpleDirectoryReader("./data").load_data()
index = VectorStoreIndex.from_documents(documents)
```

Commonly set fields on `Settings`:

| Field | Purpose |
|-------|---------|
| `llm` | The default language model for generation |
| `embed_model` | The default model used to embed nodes and queries |
| `tokenizer` | Tokenizer used for counting/splitting tokens |
| `node_parser` / `text_splitter` | How documents are chunked into nodes |
| `chunk_size` / `chunk_overlap` | Convenience shortcuts for the default chunk sizing |
| `callback_manager` | Hooks for tracing and observability |
| `context_window` / `num_output` | Token budgeting hints for the LLM |

> [!TIP]
> Setting the embedding model is the most important customization for cost and privacy. A local Hugging Face embedding model means your documents are never sent to a third-party embedding API.

### 6.2 Local (per-call) configuration

Pass a model directly to a single call to override the global default just for that operation.

```python
from llama_index.llms.anthropic import Anthropic

query_engine = index.as_query_engine(llm=Anthropic(model="claude-sonnet-4-5"))
```

> [!CAUTION]
> Older tutorials use `ServiceContext` to bundle these settings. **`ServiceContext` is deprecated.** Use `Settings` for global configuration, or pass modules directly to functions. Code that constructs a `ServiceContext` will raise an error pointing to the migration guide.

---

## 7. The Integration Ecosystem

LlamaIndex's power comes from its plugin model: a small, stable **core** plus 300+ swappable integration packages. Every integration implements an abstract base class from core, so you can change providers without rewriting your application logic.

### 7.1 Categories of integrations

| Category | Package prefix | Examples | What it provides |
|----------|----------------|----------|------------------|
| LLMs | `llama-index-llms-*` | openai, anthropic, ollama, mistralai, bedrock, gemini (google-genai), vllm, huggingface | Text generation backends |
| Embeddings | `llama-index-embeddings-*` | openai, huggingface, cohere, voyageai, bedrock, ollama | Vectorize text for semantic search |
| Vector stores | `llama-index-vector-stores-*` | qdrant, pinecone, chroma, milvus, weaviate, postgres, redis, elasticsearch | Persistent, scalable vector storage |
| Readers / connectors | `llama-index-readers-*` | web, file, github, notion, slack, s3, database, confluence | Ingest data from external sources |
| Tools | `llama-index-tools-*` | google, wikipedia, tavily-research, code-interpreter, mcp, database | Capabilities agents can call |
| Postprocessors / rerankers | `llama-index-postprocessor-*` | cohere-rerank, colbert-rerank, sbert-rerank, flag-embedding-reranker | Re-score and filter retrieved nodes |
| Graph stores | `llama-index-graph-stores-*` | neo4j, nebula, falkordb, memgraph | Backing stores for knowledge/property graphs |
| Storage | `llama-index-storage-*` | docstore/index-store/kvstore/chat-store for redis, postgres, mongodb, dynamodb | Persist documents, index metadata, chat history |
| Node parsers | `llama-index-node-parser-*` | docling, topic, slide | Specialized document chunking strategies |
| Multi-modal & more | `llama-index-multi-modal-llms-*`, `voice_agents`, `observability` | — | Images/audio/video, voice agents, tracing |

### 7.2 The swap-a-provider pattern

Because every provider implements the same interface, switching is a one-line change. To move from OpenAI to Anthropic:

```python
# before
from llama_index.llms.openai import OpenAI
Settings.llm = OpenAI(model="gpt-4o")

# after — install llama-index-llms-anthropic, then:
from llama_index.llms.anthropic import Anthropic
Settings.llm = Anthropic(model="claude-sonnet-4-5")
```

The same applies to embeddings, vector stores, and readers. Your indexing and querying code never changes.

### 7.3 Use a persistent vector store

For anything beyond prototyping, store vectors in a real database instead of memory. The pattern is: create the store, wrap it in a `StorageContext`, and pass that when building the index.

```bash
pip install llama-index-vector-stores-qdrant
```

```python
import qdrant_client
from llama_index.core import VectorStoreIndex, StorageContext, SimpleDirectoryReader
from llama_index.vector_stores.qdrant import QdrantVectorStore

client = qdrant_client.QdrantClient(path="./qdrant_data")
vector_store = QdrantVectorStore(client=client, collection_name="my_docs")
storage_context = StorageContext.from_defaults(vector_store=vector_store)

documents = SimpleDirectoryReader("./data").load_data()
index = VectorStoreIndex.from_documents(documents, storage_context=storage_context)
```

---

## 8. Persisting and Reloading an Index

By default, an in-memory `VectorStoreIndex` is lost when your program exits. You can persist it to disk and reload it later without re-embedding.

### 8.1 Persist to disk

```python
index.storage_context.persist()              # writes to ./storage by default
index.storage_context.persist(persist_dir="./my_storage")
```

### 8.2 Reload from disk

```python
from llama_index.core import StorageContext, load_index_from_storage

storage_context = StorageContext.from_defaults(persist_dir="./storage")
index = load_index_from_storage(storage_context)
```

> [!IMPORTANT]
> When reloading, make sure your `Settings.embed_model` matches the model used to build the index. Reloading with a different embedding model produces meaningless similarity scores. The LLM can differ; the embedding model should not.

> [!NOTE]
> Disk persistence is for local file-based storage. When you use an external vector store (Qdrant, Pinecone, etc.), the vectors already live in that database — you reconnect to it instead of loading from `./storage`.

---

## 9. Choosing the Right Index

`VectorStoreIndex` covers most use cases, but LlamaIndex offers several index types for different retrieval needs. All are importable from `llama_index.core`.

| Index | Best for | How it retrieves |
|-------|----------|------------------|
| `VectorStoreIndex` | General semantic search / RAG (the default choice) | Embedding similarity |
| `SummaryIndex` | "Summarize everything" over a small/medium corpus | Iterates over all nodes |
| `DocumentSummaryIndex` | Routing queries to the right document by summary | Summary-based selection |
| `KeywordTableIndex` | Keyword-driven lookup | Keyword extraction & matching |
| `TreeIndex` | Hierarchical summarization | Traverses a summary tree |
| `KnowledgeGraphIndex` | Entity/relationship questions | Graph traversal |
| `PropertyGraphIndex` | Rich, typed knowledge graphs (modern graph RAG) | Property-graph queries |

```python
from llama_index.core import SummaryIndex

summary_index = SummaryIndex.from_documents(documents)
response = summary_index.as_query_engine().query(
    "Give me a one-paragraph summary of all the documents."
)
```

> [!TIP]
> Start with `VectorStoreIndex`. Reach for `SummaryIndex` only when you genuinely need to consider *every* node (full-corpus summaries), and for graph-style questions about entities and relationships, use `PropertyGraphIndex`.

---

## 10. Troubleshooting

These errors come directly from LlamaIndex's source. Each entry gives the message, cause, and fix.

### Error: `Detected nested async. Please use nest_asyncio.apply() to allow nested event loops.`

- **Cause:** You called a synchronous method (e.g. `query()`) from inside an already-running event loop — common in Jupyter notebooks and async web frameworks.
- **Solution:** Use the async methods instead (`aquery`, `aretrieve`, `achat`), or call `import nest_asyncio; nest_asyncio.apply()` once at startup.
- **Prevention:** In async codebases, default to the `a`-prefixed methods everywhere.

### Error: `ServiceContext is deprecated. Use llama_index.settings.Settings instead...`

- **Cause:** Your code (or an old tutorial you copied) constructs a `ServiceContext`.
- **Solution:** Replace it with `Settings` for global config (`Settings.llm = ...`, `Settings.embed_model = ...`), or pass `llm=`/`embed_model=` directly to the call.
- **Prevention:** Follow current docs; treat any `ServiceContext` reference as outdated.

### Error: `ModuleNotFoundError: No module named 'llama_index.llms.<provider>'`

- **Cause:** The integration package isn't installed. Core does not bundle every provider.
- **Solution:** `pip install llama-index-llms-<provider>` (or `-embeddings-`, `-vector-stores-`, `-readers-` for other categories).
- **Prevention:** Remember the import rule — any import without `core` requires a separately installed package.

### Error: `embedding not set.`

- **Cause:** You tried to read `.get_embedding()` on a node that was never embedded.
- **Solution:** Build the node through an index (which embeds it) or set `Settings.embed_model` before indexing.
- **Prevention:** Don't access embeddings on raw `Document`/`TextNode` objects you created by hand.

### Error: `Configured node parser does not have chunk size.` / `...chunk overlap.`

- **Cause:** You set a custom `node_parser` on `Settings` and then read `Settings.chunk_size`/`chunk_overlap`, but that parser doesn't expose those attributes.
- **Solution:** Configure chunk size/overlap on the parser itself, or use a sentence-splitter-style parser that supports them.
- **Prevention:** Set `chunk_size`/`chunk_overlap` *or* a full custom `node_parser`, not both inconsistently.

### Error: `Source object must be a single RelatedNodeInfo object` (also Previous/Next/Parent/Child variants)

- **Cause:** A node's `relationships` were set with the wrong shape — e.g. a list where a single relation is expected, or vice versa for child nodes.
- **Solution:** Set `SOURCE`/`PREVIOUS`/`NEXT`/`PARENT` to a single `RelatedNodeInfo`, and `CHILD` to a *list* of them.
- **Prevention:** Use `node.as_related_node_info()` to build relationship values correctly.

### Authentication / rate-limit failures from your provider

- **Cause:** Missing API key (e.g. `OPENAI_API_KEY` not set) or hitting your provider's request/token limits.
- **Solution:** Set the required environment variable for your provider. For rate limits, attach a rate limiter (see the Admin Guide) or lower concurrency.
- **Prevention:** Verify the key is set before your first call; budget for provider limits in batch jobs.

> [!TIP]
> For community-reported issues and their resolutions, search the project's GitHub issue tracker. Recent examples include obj-serialization mismatches and `refresh_ref_docs()` dropping keyword arguments after the first document in a batch — both useful to know if you build incremental ingestion pipelines.

---

## 11. Frequently Asked Questions

**Q: What is the difference between `llama-index` and `llama-index-core`?**
`llama-index` is a starter bundle (core + OpenAI LLM/embeddings + nltk). `llama-index-core` is just the framework; you add the integration packages you want on top of it.

**Q: Do I have to use OpenAI?**
No. OpenAI is only the default in the starter package. Install any LLM/embedding integration and assign it via `Settings` to use Ollama, Anthropic, Hugging Face, Bedrock, Mistral, and many others.

**Q: Can I run everything locally with no external API calls?**
Yes. Use a local LLM (e.g. Ollama, vLLM) and a local embedding model (e.g. Hugging Face) and a local vector store. Your data then never leaves your machine.

**Q: How large can my data be?**
LlamaIndex scales by swapping the in-memory index for an external vector store (Qdrant, Pinecone, Milvus, Postgres/pgvector, etc.). The framework itself doesn't impose a hard limit; your vector store and embedding throughput do.

**Q: How do I cite or verify the sources of an answer?**
Read `response.source_nodes` — each carries the retrieved text, its metadata (such as `file_name`), and a relevance `score`.

**Q: My answers ignore some documents. Why?**
By default only the top 2 most similar chunks are retrieved. Increase `similarity_top_k`, or add a reranker postprocessor to improve which chunks reach the LLM.

**Q: How do I do multi-turn chat?**
Use `index.as_chat_engine()` instead of `as_query_engine()`. The chat engine maintains conversation history across turns.

**Q: What's the difference between a retriever, a query engine, and a chat engine?**
A **retriever** returns relevant chunks. A **query engine** retrieves *and* asks the LLM to synthesize an answer (one-shot). A **chat engine** does the same but remembers conversation history.

**Q: Is LlamaIndex safe to expose directly to end users on the internet?**
LlamaIndex is designed to run inside a trusted environment (backend code, jobs, scripts). If you put it behind a public endpoint, input validation, authentication, and rate limiting are *your* responsibility — see the Admin Guide's security section.

**Q: What is LlamaParse / LlamaCloud?**
A separate commercial platform from the same company for agentic document parsing, extraction, and indexing. It works with this open-source framework but is not part of it.

**Q: How do I build an agent that can call tools?**
Install `llama-index-tools-*` packages for the capabilities you want and compose them with LlamaIndex's agent and Workflows abstractions. See the framework documentation for agent tutorials.

---

## 12. Quick Reference

### Install

```bash
pip install llama-index                      # starter (core + OpenAI + nltk)
pip install llama-index-core                 # core only
pip install llama-index-llms-<provider>      # add an LLM
pip install llama-index-embeddings-<provider> # add embeddings
pip install llama-index-vector-stores-<db>   # add a vector store
pip install llama-index-readers-<source>     # add a data connector
```

### Minimal RAG pipeline

```python
from llama_index.core import VectorStoreIndex, SimpleDirectoryReader

documents = SimpleDirectoryReader("./data").load_data()
index = VectorStoreIndex.from_documents(documents)
print(index.as_query_engine().query("..."))
```

### Common defaults (from core)

| Setting | Default |
|---------|---------|
| Chunk size | 1024 tokens |
| Chunk overlap | 20 tokens |
| Similarity top-k | 2 |
| Temperature | 0.1 |
| Context window | 3900 tokens |
| Num output tokens | 256 |
| Embedding batch size | 10 |

### Key imports from `llama_index.core`

```python
from llama_index.core import (
    Document, Settings, StorageContext,
    VectorStoreIndex, SummaryIndex, TreeIndex,
    KeywordTableIndex, KnowledgeGraphIndex, PropertyGraphIndex,
    DocumentSummaryIndex, SimpleDirectoryReader,
    load_index_from_storage,
)
```

### Async entry points

| Sync | Async |
|------|-------|
| `query_engine.query()` | `query_engine.aquery()` |
| `retriever.retrieve()` | `retriever.aretrieve()` |
| `chat_engine.chat()` | `chat_engine.achat()` |

---

_Generated June 2026 from run-llama/llama_index at 9aa5ee5. AI-generated documentation. The library is checked by sampling (three repositories so far); this guide was not individually reviewed. Verify against the project before relying on it. Unofficial; not affiliated with or reviewed by the project._