This guide is for the engineers and operators who run MindsDB: it covers architecture, configuration, deployment, security, day-to-day operations, and internals. It assumes familiarity with the user-facing SQL surface documented in the companion User Guide.
MindsDB is a self-hosted federated query-and-AI engine written in Python. It presents a SQL interface (over the MySQL wire protocol and over HTTP), connects to external data sources through pluggable handlers, and orchestrates AI/ML providers as engines. It stores only its own metadata; your data stays in the systems it connects to.
Table of Contents
- Executive Summary
- Architecture
- System Requirements
- Configuration Mastery
- Deployment
- Security
- Operations
- REST API Reference
- Repository & Code Structure
- Internals Deep Dive
- Code Quality & Conventions
- Known Issues & Improvement Areas
- Troubleshooting (Operational)
- Quick Reference
Executive Summary
| Aspect | Detail |
|---|---|
| What it is | Federated SQL query engine + AI/ML orchestration layer |
| Language / runtime | Python (3.10–3.13) |
| Primary interfaces | MySQL wire protocol (47335), HTTP REST + GUI (47334), MCP (/mcp), A2A (/a2a), LiteLLM (8000) |
| Process model | Multi-process supervisor ("trunk") spawning per-API worker processes |
| Metadata store | SQLite by default; any SQLAlchemy-supported DB (Postgres) via MINDSDB_DB_CON |
| Extensibility | ~100+ handlers under mindsdb/integrations/handlers/, installed as pip "extras" |
| Auth | Disabled by default; session and/or bearer-token HTTP auth, MySQL user/password |
| Config sources | Defaults → config.json → config.auto.json → environment → CLI args (increasing priority) |
| License | Server Side Public License (SSPL v1) |
Important
MindsDB is insecure by default for convenience in local development: no authentication, GUI auto-download enabled, permissive CORS for MCP. None of this is appropriate for a network-exposed deployment. The Security section lists exactly what to change.
Architecture
Process topology
The entry point mindsdb/__main__.py is a supervisor that launches independent worker processes using the spawn start method (it prefers torch.multiprocessing when available). Each worker runs one subsystem:
python -m mindsdb (supervisor / "trunk")
│
┌──────────────┬───────────┼───────────────┬───────────────┬───────────────┐
▼ ▼ ▼ ▼ ▼ ▼
HTTP API MySQL API JOBS TASKS ML_TASK_QUEUE LITELLM
:47334 :47335 (scheduler) (triggers) (consumer) :8000
uvicorn/ MySQL wire start_ start_ start_ml_ start_
Starlette protocol scheduler tasks task_queue litellm
+ Flask
├─ / (WSGI Flask app via a2wsgi WSGIMiddleware) → REST + GUI
├─ /a2a (Starlette sub-app, optional)
├─ /mcp (Starlette sub-app, optional)
└─ /api/util/ping (async health check, bypasses WSGI worker pool)
- Which workers run: By default the supervisor starts
HTTPandMYSQL. Override with--api(CLI) orMINDSDB_APIS(env), e.g.--api=http,mysql,mongodb.JOBSandTASKSstart unless disabled in config;ML_TASK_QUEUEstarts only with--ml_task_queue_consumer;LITELLMstarts when requested. - Restart on failure: HTTP, MySQL, and LiteLLM workers can restart automatically. On Linux/macOS a restart is triggered specifically on OOM kill (
SIGKILL); on other platforms any failure triggers a restart, bounded bymax_restart_countwithinmax_restart_interval_seconds. In cloud mode, auto-restart is disabled. - Graceful shutdown:
atexithandlers terminate child processes, delete the PID file, and clean the temp directory.
The HTTP layer
The HTTP API is a Starlette/uvicorn ASGI application (mindsdb/api/http/start.py) that mounts:
- An async health check at
/api/util/pingfirst (so health checks answer even when the WSGI pool is saturated). - Optional
A2A(/a2a) andMCP(/mcp) Starlette sub-apps — each guarded byPATAuthMiddleware(personal-access-token auth) unless MCP OAuth is enabled. These are mounted only if their optional dependencies are installed; otherwise a warning is logged and they're skipped. - The RFC 9728
/.well-known/oauth-protected-resourceroutes (for MCP OAuth) at the server root. - The main Flask app last, wrapped by
a2wsgi.WSGIMiddleware(configurable worker pool), so it doesn't shadow the sub-apps.
The Flask app (mindsdb/api/http/initialize.py) uses flask-restx and registers one namespace per object type (sql, databases, models, projects, agents, chatbots, jobs, knowledge_bases, integrations, handlers, views, files, webhooks, auth, config, tree, tab, analysis, util). Most namespaces are protected (subject to auth when enabled). Errors are returned as RFC 7807 application/problem+json.
The query path
A statement entering through any front door is parsed by the external mindsdb-sql-parser package into an AST, planned by the query planner (mindsdb/api/executor/planner/), and executed by the SQL executor (mindsdb/api/executor/). The planner decomposes a statement into steps — fetch from integrations, invoke predictors, join intermediate results — and the executor runs them against the data hub (datahub/datanodes). Time-series predictors get special planning (windowing and group partitioning).
Persistence
MindsDB's own state — integrations, projects, models, jobs, KB metadata — lives in an internal database accessed through SQLAlchemy (mindsdb/interfaces/storage/db.py). The default is SQLite (<storage_root>/mindsdb.sqlite3.db); set MINDSDB_DB_CON to use Postgres. Schema migrations are managed with Alembic (mindsdb/migrations/) and applied automatically at startup in non-cloud mode. Model artifacts and uploaded files live under the storage root's storage/, content/, and cache/ directories.
System Requirements
| Resource | Recommendation |
|---|---|
| Python | 3.10, 3.11, 3.12, or 3.13 (>=3.10,<3.14) |
| RAM | 2 GB minimum for the core; MindsDB warns below 1 GB free. Local models need far more — Hugging Face/Ollama: 8 GB (7B), 16 GB (13B), 64 GB (70B) |
| Disk | Core install is small; model artifacts, FAISS indexes, and caches grow under the storage root. FAISS index creation transiently needs 2–3× the index size |
| Network | Outbound HTTPS for GUI download (first start), AI provider APIs, and external data sources |
| OS | Linux (production), macOS, Windows. Some handlers have platform caveats (e.g. FAISS ivf_file index unsupported on Windows) |
Note
Core runtime dependencies include Flask 3.x, Werkzeug, SQLAlchemy 2.x, pandas 2.x, uvicorn, Starlette,
a2wsgi,mindsdb-sql-parser, pydantic 2.x,redis/walrus(for the optional Redis ML queue and cache),cryptography,pyjwt,boto3,prometheus-client, andsentry-sdk. Each handler adds its own dependencies, installed as extras.
Configuration Mastery
Configuration sources and precedence
mindsdb/utilities/config.py builds the effective config by merging several sources. Later sources override earlier ones:
- Built-in defaults (lowest priority)
- User config file —
config.json, provided via--config <path>orMINDSDB_CONFIG_PATH config.auto.json— written by the app itself (e.g. when you change settings in the GUI), stored in the storage root- Environment variables
- Command-line arguments (highest priority)
The Config object is a singleton; it re-reads config.auto.json when that file's mtime changes.
The default configuration
The full default config tree (abridged to the operationally important keys):
{
"storage_db": "sqlite:///<root>/mindsdb.sqlite3.db?check_same_thread=False&timeout=30",
"paths": {
"root": "<storage_root>", "content": "…/content", "storage": "…/storage",
"static": "…/static", "tmp": "…/tmp", "log": "…/log",
"cache": "…/cache", "locks": "…/locks"
},
"auth": {
"http_auth_type": "session_or_token",
"http_auth_enabled": false,
"http_permanent_session_lifetime": "31 days",
"username": "mindsdb", "password": "", "token": null
},
"logging": {
"handlers": {
"console": {"enabled": true, "level": "INFO"},
"file": {"enabled": false, "level": "INFO", "filename": "app.log",
"maxBytes": 524288, "backupCount": 3}
},
"resources_log": {"enabled": false, "level": "INFO", "interval": 60}
},
"gui": {"open_on_start": true, "autoupdate": true},
"api": {
"http": {"host": "127.0.0.1", "port": "47334", "restart_on_failure": true,
"max_restart_count": 1, "max_restart_interval_seconds": 60,
"a2wsgi": {"workers": 10, "send_queue_size": 10}},
"mysql": {"host": "127.0.0.1", "port": "47335", "database": "mindsdb",
"ssl": true, "restart_on_failure": true, "max_restart_count": 1},
"litellm": {"host": "0.0.0.0", "port": "8000"},
"mcp": {
"cors": {"enabled": true, "allow_origins": [],
"allow_origin_regex": "https?://(localhost|127\\.0\\.0\\.1)(:\\d+)?",
"allow_headers": ["*"]},
"rate_limit": {"enabled": false, "requests_per_minute": 60},
"oauth": {"enabled": false, "issuer_url": "", "client_id": "",
"client_secret": "", "scope": "mcp:tools", "public_url": ""},
"dns_rebinding_protection": false
}
},
"cache": {"type": "local"},
"ml_task_queue": {"type": "local"},
"url_file_upload": {"enabled": true, "allowed_origins": [], "disallowed_origins": []},
"web_crawling_allowed_sites": [],
"cloud": false,
"jobs": {"disable": false},
"tasks": {"disable": false},
"default_project": "mindsdb",
"data_catalog": {"enabled": false},
"byom": {"enabled": false},
"knowledge_bases": {"disable_autobatch": false,
"disable_pgvector_autobatch": true, "storage": null}
}
Note
When MindsDB runs in Docker (
MINDSDB_DOCKER_ENVset), the default API host becomes0.0.0.0instead of127.0.0.1so the APIs are reachable from outside the container.
Environment variable reference
These are read directly by mindsdb/utilities/config.py (and a few other modules). Setting any of them overrides the corresponding config value.
| Variable | Purpose | Default |
|---|---|---|
MINDSDB_CONFIG_PATH |
Path to user config.json |
— |
MINDSDB_STORAGE_DIR |
Storage root for all data/artifacts | platform user-data dir |
MINDSDB_DOCKER_ENV |
Signals Docker (binds APIs to 0.0.0.0) |
unset |
MINDSDB_APIS |
APIs to start (same as --api) |
http,mysql |
MINDSDB_NO_STUDIO |
Disable GUI auto-open/update | unset |
MINDSDB_GUI_AUTOUPDATE |
Toggle GUI auto-update | true |
MINDSDB_DB_CON |
SQLAlchemy URL for the internal DB | SQLite |
MINDSDB_DEFAULT_PROJECT |
Default project name | mindsdb |
MINDSDB_USERNAME / MINDSDB_PASSWORD |
Set together to enable HTTP auth | unset (auth off) |
MINDSDB_HTTP_AUTH_TYPE |
session, token, or session_or_token |
session_or_token |
MINDSDB_AUTH_TOKEN |
Static bearer token | — |
FLASK_SECRET_KEY |
Flask session signing key | random per start |
AUTH_SECRET_KEY |
Key for PAT/auth middleware | — |
MINDSDB_LOG_LEVEL / MINDSDB_CONSOLE_LOG_LEVEL / MINDSDB_FILE_LOG_LEVEL |
Logging levels | INFO |
MINDSDB_STORAGE_BACKUP_DISABLED |
Disable permanent storage backup | unset |
MINDSDB_ML_QUEUE_TYPE |
redis to use a Redis-backed ML queue |
local |
MINDSDB_ML_QUEUE_HOST / _PORT / _DB / _USERNAME / _PASSWORD |
Redis ML-queue connection | localhost/6379/0 |
MINDSDB_BYOM_ENABLED |
Enable Bring-Your-Own-Model | false |
MINDSDB_DATA_CATALOG_ENABLED |
Enable data catalog | false |
MINDSDB_DEFAULT_LLM_API_KEY |
Fallback LLM key | — |
MINDSDB_DEFAULT_EMBEDDING_MODEL_API_KEY |
Fallback embedding key | — |
MINDSDB_DEFAULT_RERANKING_MODEL_API_KEY |
Fallback reranker key | — |
MINDSDB_RERANKER_N / _LOGPROBS / _TOP_LOGPROBS / _MAX_TOKENS / _VALID_CLASS_TOKENS |
Reranker tuning | — |
KB_PGVECTOR_URL |
Use pgvector as default KB store | — |
KB_MAX_INSERT_BATCH_SIZE |
KB insert batch size | 50000 |
MINDSDB_MAX_PARTITIONING_THREADS |
Max partitioning threads | 10 |
AZURE_OPENAI_API_KEY / _ENDPOINT / _API_VERSION |
Azure OpenAI for KBs | — |
MINDSDB_MCP_CORS_ENABLED / _ALLOW_ORIGINS / _ALLOW_HEADERS / _ALLOW_ORIGIN_REGEXP |
MCP CORS | see defaults |
MINDSDB_MCP_RATE_LIMIT_ENABLED / _RATE_LIMIT_RPM |
MCP rate limiting | false/60 |
MINDSDB_MCP_OAUTH_ENABLED / _ISSUER_URL / _CLIENT_ID / _CLIENT_SECRET / _SCOPE / _PUBLIC_URL |
MCP OAuth | disabled |
MINDSDB_MCP_DNS_REBINDING_PROTECTION |
MCP DNS-rebinding guard | false |
PROMETHEUS_MULTIPROC_DIR |
Enables the Prometheus metrics endpoint | unset (metrics off) |
SENTRY_IO_DSN (+ SENTRY_IO_*) |
Error reporting to Sentry | unset |
LANGFUSE_* |
LLM observability via Langfuse | unset |
Warning
MINDSDB_USERNAMEandMINDSDB_PASSWORDmust be set together; setting only one raises a startup error. Setting both also flipshttp_auth_enabledtotrue.
Editing configuration
- For static settings, write a
config.jsonand pass--config /path/config.json(or setMINDSDB_CONFIG_PATH). Example:
{
"api": { "http": { "host": "0.0.0.0", "port": "47334" } },
"auth": { "http_auth_enabled": true, "username": "admin", "password": "<secret>" }
}
config.auto.jsonin the storage root is managed by MindsDB itself (GUI-driven changes). You generally shouldn't hand-edit it; invalid JSON there raises a startup error.
Deployment
Docker (recommended)
docker run -d --name mindsdb \
-p 47334:47334 -p 47335:47335 \
-v mindsdb_data:/root/mdb_storage \
-e MINDSDB_STORAGE_DIR=/root/mdb_storage \
mindsdb/mindsdb
- Mount a volume at the storage root so connections, models, and the internal DB survive container replacement.
- In Docker the APIs bind to
0.0.0.0automatically; publish only the ports you need and put authentication and a reverse proxy in front for any non-local exposure. - Pre-install handler dependencies by extending the image or installing extras at build time.
pip / bare metal
python -m venv venv && source venv/bin/activate
pip install "mindsdb[postgres,openai]"
MINDSDB_STORAGE_DIR=/var/lib/mindsdb \
python -m mindsdb --api=http,mysql --config /etc/mindsdb/config.json
Run under a process supervisor (systemd, supervisord) so the trunk process is restarted if it exits. A PID file is created in the storage root.
External metadata database
For multi-instance or durable deployments, back the internal store with Postgres:
export MINDSDB_DB_CON="postgresql://user:pass@db-host:5432/mindsdb"
Migrations run automatically at startup (non-cloud). The connection string format is validated at boot.
Scaling AI workloads
Offload model training/inference to a Redis-backed queue and run dedicated consumer processes:
export MINDSDB_ML_QUEUE_TYPE=redis
export MINDSDB_ML_QUEUE_HOST=redis-host
# On worker hosts:
python -m mindsdb --ml_task_queue_consumer
Reverse proxy & TLS
MindsDB does not terminate TLS for the HTTP API itself. Front it with a reverse proxy (nginx, Traefik, Caddy) that handles TLS, and forward to port 47334. The MySQL API has its own ssl setting (enabled by default).
Tip
Health checks: use
GET /api/util/ping(async, returns{"status": "ok"}even when the WSGI pool is busy) for liveness probes rather than a heavier endpoint.
Security
MindsDB's defaults favor local convenience. Harden the following before any shared or network-exposed deployment.
Authentication
- HTTP auth is OFF by default (
http_auth_enabled: false). Enable it by settingMINDSDB_USERNAME+MINDSDB_PASSWORD, orauth.http_auth_enabledin config. - Choose an auth mode with
MINDSDB_HTTP_AUTH_TYPE:session(cookie),token(bearer PAT), orsession_or_token(default). Protected namespaces enforce it in abefore_requesthook. - Set a stable
FLASK_SECRET_KEY(andAUTH_SECRET_KEY) — otherwise the session signing key is randomized on every start, invalidating existing sessions on restart. - The MySQL API uses the
auth.username/auth.password(default usermindsdb, empty password). Set a password before exposing port 47335.
Network exposure
- Bind APIs to
127.0.0.1unless you intend remote access; in Docker they default to0.0.0.0, so control exposure at the published-ports / firewall layer. - The MCP server defaults to permissive CORS (regex allowing localhost), no rate limiting, and no OAuth. For remote MCP, enable OAuth (
MINDSDB_MCP_OAUTH_ENABLED+ issuer/client settings), turn on rate limiting, restrictallow_origins, and considerMINDSDB_MCP_DNS_REBINDING_PROTECTION. - The A2A server is mounted with wide-open CORS (
allow_origins=["*"]) — a known TODO in the code. Restrict it at the proxy layer.
Secrets handling
- Connection parameters (DB passwords, API keys, service-account JSON) are persisted in the internal store so MindsDB can reconnect. Protect the storage root and the metadata DB accordingly; prefer an encrypted volume.
- AI provider keys can be set once at the engine level and reused; avoid embedding them in per-query SQL that ends up in logs.
Input-handling safeguards (already in the codebase)
- Archive extraction guards (
mindsdb/utilities/fs.py): zip and tar extraction reject path traversal (Attempted Path Traversal in Zip File), symlink/hardlink entries, and unsupported member types — mitigating malicious model/handler bundles. - URL/SSRF validation (
mindsdb/utilities/security.py): URLs must include protocol and host; web crawling is restricted toweb_crawling_allowed_sites, and file-upload origins are gated byurl_file_upload.allowed_origins. - Agent SQL allowlisting (
mindsdb/interfaces/agents/utils/sql_toolkit.py): queries an agent may run are validated; disallowed queries raiseQuery is not allowed. - Static file serving (
root_index): requested paths are resolved and checked to be within the static root before serving, returning403 Forbiddenotherwise.
Caution
Two security items are tracked in the issue tracker and worth monitoring: an SSRF bypass via improper URL validation in the MindsDB editor (open) and a general request to encrypt integration credentials. Review the project's security advisories before exposing an instance, and restrict the editor/file-upload surface on untrusted networks.
Warning
Bring-Your-Own-Model (
MINDSDB_BYOM_ENABLED) and handlers that execute native queries or load serialized artifacts execute code/SQL against connected systems with the privileges of the configured credentials. Use least-privilege service accounts for every data source, and keep BYOM disabled unless you trust every model author.
Operations
Starting and stopping
# Start (foreground)
python -m mindsdb --api=http,mysql
# Start a subset of APIs / headless
python -m mindsdb --api=http --no_studio
# Update only the GUI bundle and exit
python -m mindsdb --update-gui
# Stop: send SIGTERM/SIGINT (Ctrl-C); the supervisor terminates children gracefully
Logging
- Console logging is on at
INFOby default; enable file logging viaMINDSDB_FILE_LOG_LEVEL(rotates at 0.5 MB, 3 backups, fileapp.logunder the log path). - Optional periodic resource logging: set
logging.resources_log.enabledto log CPU/memory at an interval. - Adjust verbosity at runtime with the
MINDSDB_*_LOG_LEVELvariables or--verbose.
Metrics & observability
- Prometheus: set
PROMETHEUS_MULTIPROC_DIRto a writable directory to enable the/metricsendpoint (multiprocess collector). Without it, the metrics server stays off. - Sentry: set
SENTRY_IO_DSN(+SENTRY_IO_*) for error reporting. - Langfuse: set
LANGFUSE_*for LLM tracing.
Backups
- Everything stateful lives under the storage root: the internal DB, model artifacts (
storage/), uploaded content (content/), and caches. Back up the storage root (or snapshot the volume) and, if usingMINDSDB_DB_CON, back up that external database. - A "permanent storage" backup mechanism exists and can be disabled with
MINDSDB_STORAGE_BACKUP_DISABLED.
Upgrades
- Pull a newer image /
pip install -U mindsdb. On startup (non-cloud), Alembic migrations bring the internal DB schema to head automatically. If the DB schema version is higher than the application, startup fails withDatabase version higher than application— do not point a newer-schema store at an older binary. - Models left in a non-terminal state after a crash are marked
erroron the next start (their training PIDs no longer exist).
Jobs & tasks
- The
JOBSscheduler andTASKSrunner are separate processes, enabled by default. Disable viajobs.disable/tasks.disablein config if you run scheduling externally.
REST API Reference
The HTTP API mirrors the SQL surface — most endpoints correspond to a SQL statement. Protected namespaces require auth when it is enabled.
Namespace (prefix /api) |
Purpose |
|---|---|
/sql/query |
Execute any SQL statement (the universal endpoint) |
/databases |
Manage data-source connections (GET/POST/PUT/DELETE) |
/models, /models/{name}, /models/{name}/describe |
Train, list, predict with, and inspect models |
/projects |
Manage projects |
/agents, /chatbots |
Manage agents and chatbots |
/knowledge_bases |
Manage knowledge bases |
/jobs |
Manage scheduled jobs |
/integrations, /handlers |
Inspect available/configured integrations and handlers |
/views, /files, /tab, /tree, /analysis, /config |
Views, file uploads, GUI tabs/tree, analysis, config |
/auth |
Authentication endpoints |
/webhooks |
Inbound webhooks (e.g. chat platforms) |
/util/ping |
Async liveness health check |
- Interactive docs (Swagger UI):
GET /api/doc/. - Sub-apps:
/a2a(Agent-to-Agent JSON-RPC, streaming via SSE) and/mcp(Model Context Protocol). A2A also serves/.well-known/agent.jsonand/status. - The metrics endpoint
/metricsis served by a separate Flask app within the HTTP worker when Prometheus is enabled.
Note
The bundled
openapi.ymldocuments core resources (/databases,/models, …) and uses anX-API-KEYheader for auth in that spec; the live, authoritative API is the flask-restx Swagger at/api/doc/, which reflects the configured auth scheme (session cookie and/or bearer token).
Repository & Code Structure
mindsdb/
├── __main__.py # Supervisor: spawns per-API worker processes
├── __about__.py # Version & package metadata
├── api/
│ ├── http/ # Flask (flask-restx) REST + GUI; Starlette/uvicorn host (start.py)
│ │ ├── initialize.py # App factory, namespaces, auth before_request, error handling
│ │ ├── start.py # ASGI host; mounts A2A, MCP, health, WSGI Flask
│ │ └── namespaces/ # One module per object type (sql, databases, models, …)
│ ├── mysql/mysql_proxy/ # Hand-written MySQL wire-protocol server (packets, datums)
│ ├── mcp/ # MCP server: app, tools, resources, prompts, completions
│ ├── a2a/ # Agent-to-Agent JSON-RPC server (Starlette + SSE)
│ ├── litellm/ # LiteLLM-compatible API
│ ├── common/ # Shared API middleware (rate limit, PAT auth)
│ └── executor/ # SQL executor, planner, datahub, controllers
├── integrations/
│ ├── handlers/<name>_handler/ # ~100+ data & ML handlers, each with README + requirements
│ └── libs/ # Handler base classes, ML execution, LLM helpers
├── interfaces/
│ ├── knowledge_base/ # KB controller, executor, evaluation, storage resolver
│ ├── agents/ # Agent runtime, pydantic-ai model factory, SQL toolkit
│ ├── database/ # Database/integration controllers
│ ├── model/, jobs/, triggers/, chatbot/, file/, storage/ # Object controllers + DB models
│ └── functions/ # SQL function controller (incl. to_markdown)
├── migrations/ # Alembic migration scripts (versions/)
├── metrics/ # Prometheus metrics + server
└── utilities/ # config.py, log, fs, security, context, auth, otel, sentry, …
| Area | Where to look |
|---|---|
| Config & ports | mindsdb/utilities/config.py |
| Startup / process model | mindsdb/__main__.py |
| HTTP routing & auth | mindsdb/api/http/initialize.py, start.py |
| MySQL protocol | mindsdb/api/mysql/mysql_proxy/ |
| Query planning/execution | mindsdb/api/executor/ |
| Knowledge bases | mindsdb/interfaces/knowledge_base/controller.py |
| Agents | mindsdb/interfaces/agents/ |
| A handler's contract | mindsdb/integrations/libs/ + any *_handler/ |
Internals Deep Dive
Handlers
Every integration is a handler subclassing a base in mindsdb/integrations/libs/. Data handlers translate MindsDB SQL into the target system's native calls (and pass native queries through verbatim when wrapped in parentheses). ML handlers implement create/predict/finetune. Handlers are discovered dynamically and their Python dependencies are declared in a per-handler requirements*.txt, exposed to setup.py as extras (the extra name is the handler folder minus _handler). A "permanent" files integration is created on first boot.
Knowledge base subsystem
mindsdb/interfaces/knowledge_base/controller.py orchestrates embedding + vector storage. It validates embedding/reranker configs, resolves a default vector store (pgvector when KB_PGVECTOR_URL is set, otherwise the embedded DuckDB+FAISS handler), enforces relevance thresholds (0–1) and hybrid-search alpha (0–1), and supports metadata-filtered, keyword, and hybrid retrieval. The evaluate.py module can score retrieval quality against a labeled question/answer (or doc_id) test table. The duckdb_faiss handler manages FLAT vs IVF indexes, RAM/disk forecasting before inserts, and a cache lock to keep large indexes single-owner.
Agent subsystem
mindsdb/interfaces/agents/ builds LLM agents via a pydantic_ai_model_factory that maps provider + model_name to a concrete model instance (OpenAI, Anthropic, Google, Ollama, LiteLLM, vLLM, …). A sql_toolkit gives the agent a constrained, allowlisted SQL capability over connected data and knowledge bases. Unsupported providers raise explicit errors (e.g. Bedrock/Writer/MindsDB providers are not yet available as direct model instances).
MySQL wire protocol
mindsdb/api/mysql/mysql_proxy/ is a hand-written MySQL 4.1+ server (packets, datums, handshake, OK/ERR). It implements enough of the protocol — including COM_STMT_PREPARE (prepared statements use the planner's prepare_steps) — for standard MySQL clients and BI tools to connect. Borrowed third-party code is isolated under external_libs/.
Configuration singleton
Config is a process-singleton that merges default/user/auto/env/CLI configs, creates and validates storage directories (writability checked), and hot-reloads config.auto.json on mtime change. Deprecated keys (storage_dir, log, file_upload_domains) emit warnings and are mapped to their replacements.
Code Quality & Conventions
- Language: Python 3.10+ with modern syntax (structural pattern matching,
X | Noneunions, dataclasses). - Extraction signal: Static analysis of the codebase surfaces ~968 functions, ~216 classes, and ~124 distinct raised error types — a large, mature surface. CLI/HTTP-endpoint extraction is intentionally sparse because the user surface is the SQL grammar (provided by the external
mindsdb-sql-parserpackage), not argparse flags or decorated routes. - Error handling: Rich, message-bearing exceptions throughout, surfaced to clients as RFC 7807 problem+json (HTTP) or MySQL error packets. Many sentinel/validation errors carry actionable text (see Troubleshooting).
- Process isolation: Each API runs in its own spawned process with bounded auto-restart — favoring resilience over shared in-process state.
- Dependency hygiene: Several transitive pins are annotated as Snyk-driven security pins; handler deps are isolated as extras to keep the core install lean.
- Testing: Unit tests under
tests/unit/(handlers, executor, planner, interfaces, API), integration tests undertests/integration/driven byMINDSDB_*and per-source env vars (e.g.PG_SOURCE_*,SNOWFLAKE_*) pointing at sample databases.
Known Issues & Improvement Areas
Derived from the codebase and the project's issue tracker at the documented commit:
- SSRF bypass in the editor (open): Improper URL validation in the MindsDB editor allows SSRF. Restrict the editor/file-upload surface and
web_crawling_allowed_sites/url_file_upload.allowed_originsuntil patched. - Credential encryption requested: Integration credentials are stored to enable reconnection; an enhancement to encrypt them at rest is tracked. Until then, rely on volume/DB encryption and least-privilege accounts.
- A2A CORS is wide open:
allow_origins=["*"]is a code-level TODO; constrain it at the proxy. - Community-handler breakage: Some community handlers are reported as not working; verify a handler's status before depending on it in production, and pin known-good versions.
- Agent visibility lag in GUI: Agents created via SQL may not appear in the chat UI until reload (UI reads the list at load time).
- Provider gaps in the agent factory: Bedrock/Writer/"MindsDB" providers are not yet usable as direct agent model instances; choose a supported provider.
- GUI download dependency: First start downloads the GUI from S3; air-gapped deployments must pre-seed the static directory or disable the GUI (
--no_studio).
Tip
For air-gapped or locked-down environments: disable the GUI (
--no_studio), pin handler extras explicitly, back the internal store with Postgres (MINDSDB_DB_CON), enable HTTP auth and a stableFLASK_SECRET_KEY, and put a TLS-terminating reverse proxy in front of port 47334.
Troubleshooting (Operational)
Database version higher than application
- Cause: The internal DB was migrated by a newer MindsDB than the one now starting.
- Solution: Run the newer version, or restore a backup taken with the older version. Never downgrade the binary against an upgraded schema.
Migrations fail to apply at startup
- Cause: The internal DB is unreachable or partially migrated.
- Solution: Check
MINDSDB_DB_CONconnectivity; review the startup log (the supervisor logs a migration failure but continues — the instance may then misbehave). Fix connectivity and restart.
API fails to start on its port
- Cause: Port already in use, or a worker crashed past its restart budget.
- Solution: Check for
ERROR: <api> API cant start on <port>; free the port; raisemax_restart_count/max_restart_interval_secondsif OOM restarts are exhausting the budget; investigate the OOM (large local models).
Found PID file with existing process
- Cause: A stale or conflicting PID file in the storage root.
- Solution: Ensure no other MindsDB is running against the same storage root; remove the stale PID file if the process is truly gone.
Workers repeatedly restart
- Cause: OOM kills (Linux restarts specifically on
SIGKILL). - Solution: Increase memory or move heavy ML to a Redis queue with dedicated consumers; reduce concurrent large-model usage.
Metrics endpoint returns nothing / is absent
- Cause:
PROMETHEUS_MULTIPROC_DIRis unset or not a writable directory. - Solution: Set it to an empty writable directory before start; the multiprocess collector requires it.
MCP/A2A endpoints 404 or "support is disabled"
- Cause: The optional MCP/A2A dependencies aren't installed.
- Solution: Install the corresponding extra (
pip install "mindsdb[mcp]"/[a2a]); the log prints the exact extra to install.
Quick Reference
Ports
| API | Port | Notes |
|---|---|---|
HTTP (REST + GUI + /mcp + /a2a) |
47334 |
Swagger at /api/doc/; health at /api/util/ping |
| MySQL wire protocol | 47335 |
Default user mindsdb, ssl: true |
| LiteLLM | 8000 |
Started on request |
| Metrics | (HTTP worker) | /metrics when PROMETHEUS_MULTIPROC_DIR set |
Storage layout (under the storage root)
| Path | Contents |
|---|---|
mindsdb.sqlite3.db |
Internal metadata DB (default) |
storage/ |
Model artifacts |
content/ |
Uploaded files |
static/ |
Downloaded GUI bundle (version.txt) |
tmp/, cache/, locks/, log/ |
Temp, cache, locks, logs |
config.auto.json |
App-managed config (GUI changes) |
Operational commands
python -m mindsdb --version # version
python -m mindsdb --api=http,mysql # choose APIs
python -m mindsdb --no_studio # headless (no GUI)
python -m mindsdb --update-gui # refresh GUI bundle and exit
python -m mindsdb --mcp-stdio # MCP over stdio
curl localhost:47334/api/util/ping # liveness check
Hardening checklist
- [ ] Set
MINDSDB_USERNAME+MINDSDB_PASSWORD(enables HTTP auth) - [ ] Set a stable
FLASK_SECRET_KEY(andAUTH_SECRET_KEY) - [ ] Set a MySQL password; restrict port 47335
- [ ] Put TLS-terminating reverse proxy in front of 47334
- [ ] Restrict MCP (OAuth + rate limit +
allow_origins) and A2A CORS - [ ] Constrain
web_crawling_allowed_sitesandurl_file_upload.allowed_origins - [ ] Use least-privilege accounts for every data source; keep BYOM disabled unless trusted
- [ ] Back the internal store with Postgres for durability; back up the storage root
- [ ] Encrypt the storage volume / metadata DB (credentials are stored there)