exo User Guide

exo connects all of your devices into a single AI cluster so you can run frontier-scale models — large language models and image models — across your own hardware. Instead of being limited to what fits on one machine, exo pools the memory and compute of every Mac (and, on CPU, Linux box) you own, automatically discovers them on the network, figures out the best way to split a model across them, and serves the result through APIs you already know: OpenAI, Claude, Ollama, and the OpenAI Responses API.

This guide shows you how to install exo, start a cluster, load models, and run inference — from a single laptop to a multi-node Thunderbolt cluster.

Note

exo is built and maintained by exo labs. On macOS it runs on the GPU via Apple's MLX framework. On Linux it currently runs on CPU (GPU support is in development). Models run faster as you add more devices, especially with RDMA over Thunderbolt 5.

Table of Contents

1. Quick Start
2. Requirements
3. Installation
4. Running exo
5. Using the Dashboard
6. Loading and Running a Model (the API Workflow)
7. Everyday Usage: Talking to Your Cluster
8. Working with Models
9. Image Generation and Editing
10. Building a Multi-Device Cluster
11. Enabling RDMA over Thunderbolt (macOS)
12. Command-Line Reference
13. Environment Variables
14. Benchmarking Your Cluster
15. How It Works (Concepts)
16. Troubleshooting
17. FAQ
18. Reference

1. Quick Start

If you have Nix on a Mac, you can launch exo in one command:

nix run .#exo

Otherwise, the source workflow is three steps — clone, build the dashboard, run:

# Clone exo
git clone https://github.com/exo-explore/exo
cd exo

# Build the dashboard
cd dashboard && npm install && npm run build && cd ..

# Run exo
uv run exo

When exo starts, the dashboard and API are available at:

http://localhost:52415/

Open that URL in a browser. Every device running exo serves its own copy of the dashboard and API on this port, and devices on the same network discover each other automatically — no manual configuration needed.

Tip

Start with a small model such as mlx-community/Llama-3.2-1B-Instruct-4bit to confirm everything works before loading anything large.

2. Requirements

Supported platforms

Tier 1 hardware that the project actively tests includes the Mac Studio (M3 Ultra), Mac Mini (M4 Pro), and MacBook Pro (M4 Max / M5).

Important

The macOS background app requires macOS Tahoe 26.2 or later. RDMA over Thunderbolt is a macOS 26.2 capability and works on Macs with Thunderbolt 5 (M4 Pro Mac Mini, M4 Max Mac Studio, M4 Max MacBook Pro, M3 Ultra Mac Studio).

Software prerequisites (run-from-source)

• Python 3.13 (exo requires ==3.13.*)
• uv — Python dependency management
• node — builds the dashboard (Linux: v18+)
• rust — builds the native bindings (nightly toolchain)
• macOS only: Xcode (Metal toolchain for MLX), Homebrew, and macmon (hardware monitoring on Apple Silicon)

Memory

exo's whole purpose is to run models that don't fit on one device, so "requirements" scale with the model. The cluster needs enough combined RAM across participating nodes to hold the model. exo computes this for you and refuses placements that don't fit (see Troubleshooting).

3. Installation

exo offers three installation paths. Choose the one that matches your setup.

macOS — Run from Source

Install the prerequisites:

# Homebrew (if you don't have it)
/bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/HEAD/install.sh)"

# uv and node
brew install uv node

# Rust (nightly)
curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh
rustup toolchain install nightly

# macmon — install the pinned fork revision (Homebrew's macmon 0.6.1 crashes on M5)
cargo install --git https://github.com/vladkens/macmon \
  --rev a1cd06b6cc0d5e61db24fd8832e74cd992097a7d \
  macmon --force

Then clone, build the dashboard, and run:

git clone https://github.com/exo-explore/exo
cd exo/dashboard && npm install && npm run build && cd ..
uv run exo

macOS — Nix

With Nix installed:

nix run .#exo

To accept the Cachix binary cache (and avoid building the Xcode Metal toolchain), add to /etc/nix/nix.conf:

trusted-users = root        # or your username
experimental-features = nix-command flakes

Then restart the Nix daemon:

sudo launchctl kickstart -k system/org.nixos.nix-daemon

macOS — App (.dmg / Homebrew Cask)

Download the prebuilt menu-bar app:

• Direct download: EXO-latest.dmg

• Homebrew:

  brew install --cask exo
  

The app runs in the background and will ask for permission to modify system settings and install a network profile (needed for device discovery and RDMA). Requires macOS Tahoe 26.2+.

Linux — Run from Source

# Node.js + npm (Ubuntu/Debian)
sudo apt update && sudo apt install nodejs npm

# uv
curl -LsSf https://astral.sh/uv/install.sh | sh

# Rust (nightly)
curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh
rustup toolchain install nightly

# Clone, build dashboard, run
git clone https://github.com/exo-explore/exo
cd exo/dashboard && npm install && npm run build && cd ..
uv run exo

macmon is not required on Linux.

Warning

On Linux, exo currently runs inference on the CPU. It is fully functional but slower than GPU. If you want GPU support for your Linux hardware, search the issue tracker and add a 👍 to the relevant request.

Note

The dashboard must be built (npm run build) before running from source. If you skip it, exo can't find the dashboard assets and will tell you so on startup (see Troubleshooting).

4. Running exo

The simplest invocation is:

uv run exo

http://localhost:52415/

That single process becomes a full cluster node: it discovers peers, can act as the cluster coordinator (master), downloads models, runs inference (worker), and serves the API and dashboard.

Common run modes

Default (everything)

uv run exo

Discovery + worker + API + downloads, all on.

Coordinator-only node

uv run exo --no-worker

Runs networking and orchestration but does not execute inference. Useful for a machine with good network connectivity but little GPU/RAM.

Offline / air-gapped

uv run exo --offline

Skips internet checks and uses only models you've already staged locally.

Background daemon (legacy)

uv run exo --legacy-daemon

Double-fork SysV-style daemonization. Intended for legacy init scripts only — under systemd or launchd, run exo in the foreground (no flag).

When the API node starts, exo prints a startup banner showing the address. Leave the process running; press Ctrl-C to shut it down cleanly.

Tip

Add -v for more logging detail (repeat for more, e.g. -vv), or -q to quiet it down. See the Command-Line Reference.

5. Using the Dashboard

exo ships a built-in web dashboard for managing your cluster and chatting with models. Once exo is running, open:

http://localhost:52415/

From the dashboard you can:

• See your cluster — every discovered device, its memory, and the network links between devices (Thunderbolt, Ethernet, Wi-Fi).
• Browse and download models — search HuggingFace, start downloads, and watch progress.
• Load (place) a model — pick a model and let exo choose a placement across your devices, or preview the options.
• Chat — send prompts to a loaded model and stream the response, with support for reasoning/"thinking" models.
• Generate images — when image models are enabled.
• Inspect performance traces — when tracing is enabled.

The dashboard is the easiest way to get started. The rest of this guide focuses on the API, which gives you the same capabilities programmatically and lets you point existing tools at your cluster.

6. Loading and Running a Model (the API Workflow)

Loading a model in exo is a deliberate, two-phase process: first you place an instance (decide how the model is split across devices and start loading it), then you send inference requests to it. This section walks through the full lifecycle with curl.

The base URL is your exo node, e.g. http://localhost:52415.

Step 1 — Preview placements

Ask exo for all valid ways to place a model across your current cluster:

curl "http://localhost:52415/instance/previews?model_id=llama-3.2-1b"

{
  "previews": [
    {
      "model_id": "mlx-community/Llama-3.2-1B-Instruct-4bit",
      "sharding": "Pipeline",
      "instance_meta": "MlxRing",
      "instance": { "...": "..." },
      "memory_delta_by_node": { "local": 729808896 },
      "error": null
    }
  ]
}

Each preview describes a sharding strategy (Pipeline or Tensor), a backend (MlxRing or MlxJaccl), and the per-node memory cost. Placements that won't work include a non-null error explaining why.

To grab the first valid placement programmatically, pipe through jq:

curl "http://localhost:52415/instance/previews?model_id=llama-3.2-1b" \
  | jq -c '.previews[] | select(.error == null) | .instance' | head -n1

Step 2 — Create the instance

POST the placement you chose to /instance:

curl -X POST http://localhost:52415/instance \
  -H 'Content-Type: application/json' \
  -d '{
    "instance": { "...": "...paste the instance object from step 1..." }
  }'

{
  "message": "Command received.",
  "command_id": "e9d1a8ab-...."
}

This is asynchronous — the model now begins downloading (if needed) and loading across the assigned devices.

Step 3 — Wait until the model is ready

Don't send inference yet. Wait for the instance to come online using the streaming await endpoint:

curl -N "http://localhost:52415/instance/await?model_id=mlx-community/Llama-3.2-1B-Instruct-4bit"

This returns a Server-Sent Events (SSE) stream. A successful wait emits a message with "type": "ready" and the matching instance; a timeout emits "type": "timeout". By default it waits indefinitely; add &timeout_seconds=120 (max 300) to bound the wait.

data: {"type": "ready", "instance": {...}}

Step 4 — Run inference

Now send a chat completion (OpenAI format):

curl -N -X POST http://localhost:52415/v1/chat/completions \
  -H 'Content-Type: application/json' \
  -d '{
    "model": "mlx-community/Llama-3.2-1B-Instruct-4bit",
    "messages": [
      {"role": "user", "content": "What is Llama 3.2 1B?"}
    ],
    "stream": true
  }'

Step 5 — Delete the instance when done

Free the memory by deleting the instance (find its ID via /state or /instance):

curl -X DELETE http://localhost:52415/instance/YOUR_INSTANCE_ID

Note

If you send a chat request for a model that has no running instance, exo responds with 404 No instance found for model <id>. Place the instance first (steps 1–3). The dashboard performs this whole flow for you when you click "load".

7. Everyday Usage: Talking to Your Cluster

Because exo speaks four popular API dialects, you can usually point an existing tool at it by changing only the base URL to your exo node. All inference endpoints below assume a model instance is already loaded (see Section 6).

OpenAI Chat Completions

curl -N -X POST http://localhost:52415/v1/chat/completions \
  -H 'Content-Type: application/json' \
  -d '{
    "model": "mlx-community/Llama-3.2-1B-Instruct-4bit",
    "messages": [
      {"role": "system", "content": "You are a helpful assistant."},
      {"role": "user", "content": "Hello"}
    ],
    "stream": false
  }'

Non-streaming responses include usage statistics:

{
  "choices": [{"index": 0, "message": {"role": "assistant", "content": "Hello! How can I help you?"}, "finish_reason": "stop"}],
  "usage": {"prompt_tokens": 15, "completion_tokens": 8, "total_tokens": 23}
}

Supported request parameters include model, messages, stream, max_tokens, temperature, top_p, top_k, stop, seed, tools (function calling), logprobs, top_logprobs, and enable_thinking (turns on reasoning mode for capable models such as DeepSeek V3.1, Qwen3, and GLM-4.7).

Claude Messages API

Point an Anthropic-style client at /v1/messages:

curl -N -X POST http://localhost:52415/v1/messages \
  -H 'Content-Type: application/json' \
  -d '{
    "model": "mlx-community/Llama-3.2-1B-Instruct-4bit",
    "messages": [{"role": "user", "content": "Hello"}],
    "max_tokens": 1024,
    "stream": true
  }'

Streaming uses Claude's event types: message_start, content_block_start, content_block_delta, content_block_stop, message_delta, message_stop.

OpenAI Responses API

curl -N -X POST http://localhost:52415/v1/responses \
  -H 'Content-Type: application/json' \
  -d '{
    "model": "mlx-community/Llama-3.2-1B-Instruct-4bit",
    "messages": [{"role": "user", "content": "Hello"}],
    "stream": true
  }'

Streaming uses Responses event types: response.created, response.in_progress, response.output_item.added, response.output_item.done, response.done.

Ollama API (works with OpenWebUI and friends)

# Chat
curl -X POST http://localhost:52415/ollama/api/chat \
  -H 'Content-Type: application/json' \
  -d '{
    "model": "mlx-community/Llama-3.2-1B-Instruct-4bit",
    "messages": [{"role": "user", "content": "Hello"}],
    "stream": false
  }'

# List models, Ollama-style
curl http://localhost:52415/ollama/api/tags

exo also exposes /ollama/api/generate (prompt-completion), /ollama/api/show, /ollama/api/ps (running models), and /ollama/api/version.

Tip

Configuring a tool that expects "Ollama"? Use http://<your-exo-node>:52415/ollama as the Ollama base URL. For OpenAI-compatible tools, use http://<your-exo-node>:52415/v1.

Cancelling a request

You can stop an in-flight generation two ways:

1. Close the HTTP connection (for streaming requests) — the server detects the disconnect and stops.

2. Call the cancel endpoint with the command's ID:

   curl -X POST http://localhost:52415/v1/cancel/COMMAND_ID
   

   Returns 404 if the command is unknown or already finished.

8. Working with Models

Listing what's available

# All known models
curl http://localhost:52415/models

# Only models already downloaded to the cluster
curl "http://localhost:52415/models?status=downloaded"

Model descriptors include the model ID, family, quantization, context length, supported tasks, capabilities, whether the model supports tensor parallelism, and an is_custom flag.

exo ships with a large built-in catalog of model cards — DeepSeek (V3.1/V3.2/V4), GLM (4.5/4.7/5/5.1), Llama, Qwen, Kimi K2, and many more — predominantly in mlx-community quantized form.

Searching HuggingFace

curl "http://localhost:52415/models/search?query=llama&limit=10"

exo searches mlx-community first (the quantized models it runs best), then falls back to all of HuggingFace.

Adding a custom model

Pull any compatible model from the HuggingFace hub and register it as a custom model card that syncs across your cluster:

curl -X POST http://localhost:52415/models/add \
  -H 'Content-Type: application/json' \
  -d '{ "model_id": "mlx-community/my-custom-model" }'

Remove a custom model card later:

curl -X DELETE http://localhost:52415/models/custom/mlx-community/my-custom-model

Caution

Models whose configuration requires trust_remote_code execute code from the model repository. exo disables this by default. Only enable it for models you fully trust — remote code runs on your machine.

Where models are stored

Downloaded models live in exo's data directory by default (~/.exo/models on macOS, ~/.local/share/exo/models on Linux). You can redirect downloads to another disk, or point exo at a shared read-only model store, using environment variables — see Section 13.

9. Image Generation and Editing

exo can run image models (e.g. FLUX) in addition to LLMs. Image support is off by default and must be enabled when you start the node:

EXO_ENABLE_IMAGE_MODELS=true uv run exo

Load an image model the same way you load an LLM (place an instance, wait for ready), then call the OpenAI-compatible image endpoints.

Generate an image

curl -X POST http://localhost:52415/v1/images/generations \
  -H 'Content-Type: application/json' \
  -d '{
    "prompt": "a robot playing chess",
    "model": "exolabs/FLUX.1-dev",
    "n": 1,
    "size": "1024x1024",
    "response_format": "b64_json"
  }'

Key parameters:

Fine-grained control lives in advanced_params:

Edit an image (img2img)

Send the source image as multipart form data to /v1/images/edits:

curl -X POST http://localhost:52415/v1/images/edits \
  -F image=@input.png \
  -F prompt="make it snowy" \
  -F model="exolabs/FLUX.1-Kontext-dev" \
  -F input_fidelity=high \
  -F output_format=png

input_fidelity (low/high) controls how closely the output follows the original image.

Retrieving images

When response_format is url, exo stores the image and returns a link. Stored images expire after about an hour.

curl http://localhost:52415/images               # list stored images
curl http://localhost:52415/images/IMAGE_ID -o out.png   # fetch one

Note

Streaming image responses emit SSE events of type: "partial" (in-progress previews) and type: "final", terminated by data: [DONE].

10. Building a Multi-Device Cluster

The payoff of exo is pooling devices. The process is intentionally hands-off:

1. Install exo on each device (Section 3).
2. Connect the devices to the same network. For maximum speed on Apple Silicon, connect them with Thunderbolt cables (and enable RDMA — Section 11). Ethernet and Wi-Fi also work.
3. Run exo on each device. They discover each other automatically — no IP addresses or config files to edit.
4. Open the dashboard on any node and you'll see all devices and the links between them.
5. Place a model. exo's topology-aware placement automatically splits the model across devices based on each device's memory and the speed of the links between them. A model too large for any single device will run across several.

Keeping clusters separate

If multiple exo clusters share a network (e.g. dev vs. production), isolate them with a namespace. Nodes with different namespaces will not connect:

EXO_ZENOH_NAMESPACE=my-dev-cluster uv run exo
# or equivalently
uv run exo --namespace my-dev-cluster

In the macOS app, set this under Advanced settings.

Tip

A device with lots of bandwidth but little RAM/GPU makes a good --no-worker coordinator: it helps route and orchestrate without being asked to hold model layers.

How exo decides the split

You don't have to choose, but it helps to know the options exo weighs:

• Pipeline parallelism — different layers of the model live on different devices; tokens flow around the ring. Works with the most models.
• Tensor parallelism — each layer is split across devices, for up to ~1.8× speedup on 2 devices and ~3.2× on 4. Requires a model that supports it.
• Ring backend (MlxRing) — neighbour-to-neighbour communication over sockets/Thunderbolt; the general-purpose default.
• JACCL backend (MlxJaccl) — all-to-all RDMA over Thunderbolt 5 for the lowest latency. Requires RDMA to be enabled and every device connected to every other.

exo picks among the valid combinations using available memory and link types, preferring faster links. See How It Works.

11. Enabling RDMA over Thunderbolt (macOS)

RDMA (Remote Direct Memory Access) over Thunderbolt 5 dramatically reduces inter-device latency, making multi-device inference much faster. It's a macOS 26.2 capability available on Macs with Thunderbolt 5.

Enable RDMA on each Mac:

1. Shut down the Mac.

2. Hold the power button for ~10 seconds until the boot menu appears.

3. Choose Options to enter Recovery mode.

4. In Recovery, open Terminal from the Utilities menu.

5. Run:

   rdma_ctl enable
   

6. Reboot.

After that, exo detects and uses RDMA automatically.

Important

RDMA caveats:

• Every device in an RDMA cluster must be physically connected to every other device.
• Cables must support Thunderbolt 5.
• On a Mac Studio, do not use the Thunderbolt 5 port next to the Ethernet port.
• If running from source, run tmp/set_rdma_network_config.sh to disable Thunderbolt Bridge and set DHCP on each RDMA port.
• All devices must run the exact same macOS version (including beta build numbers), or RDMA ports may fail to discover each other.

Pre-flight checklist:

• [ ] All Macs have Thunderbolt 5 ports
• [ ] All Macs run identical macOS versions (26.2+)
• [ ] Every device cabled to every other device with TB5 cables
• [ ] rdma_ctl enable run in Recovery on each Mac
• [ ] (From source) tmp/set_rdma_network_config.sh executed

12. Command-Line Reference

Run uv run exo [OPTIONS]. All options are optional; defaults make a sensible single-node setup.

Note

--fast-synch and --no-fast-synch are mutually exclusive; omit both to let exo decide automatically.

Warning

--bootstrap-peers (and the EXO_BOOTSTRAP_PEERS variable) are currently not functional — supplying them raises Bootstrap peers has been temporarily removed. Likewise, the old EXO_LIBP2P_NAMESPACE variable was renamed; from source use EXO_ZENOH_NAMESPACE instead (the macOS app still exposes the namespace as a setting).

13. Environment Variables

Set these before launching exo (e.g. VAR=value uv run exo).

Examples:

# Use pre-downloaded models from an NFS mount (read-only)
EXO_MODELS_READ_ONLY_DIRS=/mnt/nfs/models:/opt/ai-models uv run exo

# Download models to an external SSD (falls back to default dir if full)
EXO_MODELS_DIRS=/Volumes/ExternalSSD/exo-models uv run exo

# Offline mode
EXO_OFFLINE=true uv run exo

# Enable image models
EXO_ENABLE_IMAGE_MODELS=true uv run exo

Tip

On Linux, exo follows the XDG Base Directory spec: config in ~/.config/exo/, data in ~/.local/share/exo/, cache and logs in ~/.cache/exo/. Override with the standard XDG_* variables.

For advanced/internal variables (EXO_MAX_CONCURRENT_REQUESTS, ENABLE_DISAGGREGATION, EXO_RESOURCES_DIR, EXO_DASHBOARD_DIR, and more), see the Admin Guide.

14. Benchmarking Your Cluster

exo includes exo-bench, a tool to measure prefill and generation speed across placement configurations — useful for finding the fastest way to run a model on your hardware.

Prerequisite: nodes must already be running (uv run exo); the tool drives the /bench/chat/completions endpoint.

uv run bench/exo_bench.py \
  --model Llama-3.2-1B-Instruct-4bit \
  --pp 128,256,512 \
  --tg 128,256

Frequently used options:

The output reports prompt tokens/sec, generation tokens/sec, and peak memory per configuration.

Note

exo also bundles exo_eval.py (run standard benchmarks/evals against a model), prefill_decode_bench.py (measure prefill/decode disaggregation), and eval_tool_calls.py (test tool-calling across the API dialects). These are advanced tools covered in the Admin Guide.

15. How It Works (Concepts)

You don't need this section to use exo, but it explains the words you'll see in the dashboard and API.

Node. One device running exo. Every node is equal at startup and can take on any role.

Master (coordinator). Exactly one node coordinates the cluster at a time. Nodes elect a master automatically; if it disappears, a new one is elected. --force-master biases the election toward a specific node.

Worker. The part of a node that actually runs model code in a separate runner process. Disable with --no-worker.

Topology. exo's live map of which devices are connected and how (Thunderbolt, Ethernet, Wi-Fi), including link quality. Placement decisions use this map.

Instance. A loaded, ready-to-serve copy of a model, spread across one or more devices according to a placement. You create instances, run inference against them, and delete them to reclaim memory.

Placement / sharding. How a model is divided across devices. exo finds device "cycles" (rings) with enough combined memory, then allocates layers proportionally to each device's available RAM. It supports pipeline parallelism (layers split across devices) and tensor parallelism (each layer split across devices), over either the ring (MlxRing) or RDMA (MlxJaccl) backend.

Continuous batching. By default, exo batches concurrent requests for throughput. --no-batch falls back to sequential generation.

The flow of a single request: your client hits the API → the master turns it into a task and assigns it to the least-busy instance → workers on each device run their shard of the model → tokens (or image chunks) stream back to the API → the API relays them to you as SSE.

16. Troubleshooting

Error: No instance found for model <model_id>

Cause: You sent an inference request for a model that has no running instance (or the model isn't downloaded). Solution: Place an instance first — preview placements, POST to /instance, then wait on /instance/await (Section 6). Confirm the model appears in curl http://localhost:52415/models?status=downloaded. Prevention: Use the dashboard's "load" flow, which handles placement and readiness for you.

Error: Insufficient memory to create instance. Required: X GB, Available: Y GB

Cause: The cluster's combined available RAM can't hold the model. Solution: Add more devices, free memory on existing devices, or choose a smaller/more-quantized model (e.g. a 4-bit instead of 8-bit variant). Prevention: Check /instance/previews first — memory_delta_by_node shows the per-node cost before you commit.

Error: No cycles found with sufficient memory

Cause: No connected group of devices has enough total memory for this model. Solution: Connect more devices into the cluster, or pick a smaller model.

Error: Requested Tensor sharding but this model does not support tensor parallelism

Cause: You asked for tensor parallelism on a model whose card doesn't support it. Solution: Use pipeline sharding for this model, or pick a model with supports_tensor: true (visible in /models).

Error: Requested RDMA (MlxJaccl) but no RDMA-connected cycles available

Cause: The JACCL backend needs an all-to-all RDMA-connected set of devices, and none is available. Solution: Enable RDMA on every device (Section 11), cable every device to every other with TB5, and ensure identical macOS versions. Otherwise use the ring backend.

Error: Unable to locate dashboard assets - you probably forgot to run cd dashboard && npm install && npm run build && cd ..

Cause: Running from source without building the dashboard. Solution: Build it: cd dashboard && npm install && npm run build && cd .., then restart exo.

Error: Unable to locate resources. Did you clone the repo properly?

Cause: exo can't find its bundled resource files (model cards, etc.). Solution: Ensure you cloned the full repository (not a partial/shallow copy missing resources/), and run from the repo root.

Error: EXO_LIBP2P_NAMESPACE has been removed - use EXO_ZENOH_NAMESPACE instead

Cause: You set the old namespace variable when running from source. Solution: Use EXO_ZENOH_NAMESPACE (or the --namespace flag) instead.

Error: Bootstrap peers has been temporarily removed

Cause: You passed --bootstrap-peers / EXO_BOOTSTRAP_PEERS. Solution: Remove it. Discovery is automatic; manual peer bootstrapping is not currently supported.

exo won't start: PID file error

Cause: Another exo process is already running (it holds a PID lock), or a stale lock remains after a crash. Solution: Stop the other instance. If none is running, the stale PID file lives in exo's cache dir (~/.exo/ or ~/.cache/exo/); remove it and retry.

Devices don't see each other

Cause: Different namespaces, blocked discovery ports, or no network path between devices. Solution: Ensure all nodes use the same namespace (or none), that UDP discovery port 52413 and TCP zenoh port 52414 aren't firewalled, and that the devices share a network. On macOS, allow the network profile the app installs.

Generation seems stuck or very slow

Cause: Model still loading, slow links (Wi-Fi instead of Thunderbolt), or a large model on minimal hardware. On Linux, inference runs on CPU. Solution: Wait for /instance/await to report ready; prefer Thunderbolt/RDMA links; benchmark placements with exo-bench to find the fastest configuration.

macOS RDMA ports stop working after idle, or run at half speed

Cause: Known issues on some hardware/macOS combinations (e.g. RDMA ports going inactive, or certain MacBook Pro pairings running slower over RDMA). Solution: Reconnect/recheck cabling, confirm identical macOS versions, and consult the issue tracker for your specific hardware.

17. FAQ

Q: Do I have to configure IP addresses or a config file to make a cluster? No. Devices on the same network discover each other automatically. You only configure things like ports or namespaces if you have a specific need.

Q: What's the difference between "place an instance" and just sending a chat request? Placing an instance loads the model across your devices and makes it ready to serve. Chat requests then run against that loaded instance. A chat request for a model with no instance returns 404.

Q: Which API should I use — OpenAI, Claude, Ollama, or Responses? Whichever your existing tool already speaks. exo supports all four; just point the tool's base URL at your exo node. Capabilities are equivalent for text generation.

Q: Can I run models bigger than any single device's memory? Yes — that's the point. exo pools memory across devices and splits the model so the combined memory holds it.

Q: Does exo run on Windows? Not yet. macOS (GPU) and Linux (CPU) are supported today; Windows is on the roadmap.

Q: Is a GPU required? On macOS, exo uses the Apple Silicon GPU automatically. On Linux it runs on CPU today.

Q: How do I make a multi-Mac cluster as fast as possible? Connect the Macs with Thunderbolt 5 and enable RDMA (Section 11). Use tensor parallelism on models that support it. Benchmark with exo-bench.

Q: How do I add a model that isn't in the built-in list? POST its HuggingFace ID to /models/add. It becomes a custom model card synced across the cluster. Only enable trust_remote_code for models you trust.

Q: Where are my downloaded models? In ~/.exo/models (macOS) or ~/.local/share/exo/models (Linux) by default. Redirect with EXO_DEFAULT_MODELS_DIR / EXO_MODELS_DIRS, or share a read-only store with EXO_MODELS_READ_ONLY_DIRS.

Q: Can I use exo without internet? Yes — run with --offline (or EXO_OFFLINE=true) once your models are staged locally.

Q: How do I generate images? Start exo with EXO_ENABLE_IMAGE_MODELS=true, load an image model, and call /v1/images/generations or /v1/images/edits.

Q: How do I stop a runaway generation? Close the connection, or POST /v1/cancel/{command_id}.

Q: Can I dedicate a low-power machine to coordination only? Yes — run it with --no-worker. It helps with networking and orchestration without holding model layers.

Q: Why is there a --no-batch flag? exo batches concurrent requests for throughput by default. --no-batch switches to sequential generation, which can be useful for debugging or strict reproducibility.

18. Reference

Default ports

Core inference endpoints (instance must be loaded)

Cluster / model management endpoints

Useful links

• Source: https://github.com/exo-explore/exo
• API technical reference: docs/api.md in the repository
• Community / support: Discord · @exolabs
• License: Apache-2.0