Dify -- Admin Guide

This guide is for the people who deploy, operate, secure, extend, and contribute to Dify. It assumes you have read the User Guide and have a Dify instance running. If you are new to Dify, start there.

Note

Dify is classified as a "massive" complexity project: 11,069 files, 31.7 million source characters, 258 significant modules. This guide covers the full operational surface.

Table of Contents

• Executive Summary
• Architecture
• System Requirements
• Installation and Deployment
• Configuration Reference
• Security
• Operations
• Workflow Engine Internals
• RAG Pipeline Internals
• Plugin System Internals
• Vector Database Backend System
• Frontend Architecture
• Backend Architecture
• Database Schema and Migrations
• Celery Task System
• Observability and Telemetry
• Code Patterns and Conventions
• SDKs and Client Libraries
• E2E Testing
• Contributing
• Known Issues and Improvements
• CLI Reference
• Quick Reference

Executive Summary

Dify is an open-source LLM application development platform that provides a visual workflow builder, RAG pipeline, agent framework, model abstraction layer, and plugin ecosystem. It serves as the full-stack infrastructure for building and deploying AI applications.

Technology Stack

Key Metrics (Source: commit 13a9359)

Architecture

System Architecture

                    Internet
                       |
              +--------v--------+
              |     nginx       |  Port 80/443, SSL termination
              |  (reverse proxy)|  Static files, WebSocket upgrade
              +---+--------+---+
                  |        |
        +---------v--+  +--v---------+
        |   web      |  |   api      |  Flask + gevent + WebSocket
        | (Next.js)  |  | :5001      |  4 API surfaces
        | :3000      |  +--+-+--+----+
        +------------+     | |  |
                           | |  +----> plugin_daemon (Go) :5002
                           | |          |-> Plugin processes (stdio JSON-RPC)
                           | |
                    +------+ +------+
                    |               |
              +-----v----+  +------v------+
              |  worker   |  |   sandbox   |  Code execution :8194
              | (Celery)  |  +-------------+
              +--+--+-----+
                 |  |
          +------+  +------+
          |                |
    +-----v------+  +------v-------+
    | PostgreSQL |  |    Redis     |  Cache, broker, sessions,
    |  :5432     |  |   :6379      |  rate limiting, pubsub
    +------------+  +--------------+
          |
    +-----v---------+
    | Vector Store   |  Weaviate :8080 (default)
    | (pluggable)    |  or any of 30+ backends
    +----------------+

Four API Surfaces

Dify exposes four distinct API surfaces, each serving a different audience:

Important

The Inner API is exclusively for plugin daemon communication. It is not intended for external use and uses a shared secret key (INNER_API_KEY_FOR_PLUGIN) for authentication.

Process Model

The API server runs Flask under Gunicorn with the gevent worker class, providing cooperative multitasking for handling concurrent requests. The entry point is api/app.py:

# api/app.py
app = create_app()
celery = app.extensions["celery"]

if __name__ == "__main__":
    app.run(host="0.0.0.0", port=5001)

In production:

gunicorn --bind 0.0.0.0:5001 \
  --worker-class gevent \
  --workers 1 \
  --timeout 200 \
  app:app

Warning

The gevent worker requires monkey-patching at import time. The app_factory.py handles this. Do not import standard library threading or socket modules before gevent patches are applied.

System Requirements

Hardware Requirements

Software Dependencies

Installation and Deployment

Docker Compose (Production)

The docker/ directory contains the complete production deployment configuration.

Container Orchestration

Deployment Steps

cd dify/docker
cp .env.example .env
# Edit .env with your production values
docker compose up -d

# Verify all containers are healthy
docker compose ps
docker compose logs --tail 50 api

nginx Configuration

The bundled nginx config (docker/nginx/) handles:

• SSL termination (with Certbot-managed Let's Encrypt certificates)
• WebSocket upgrade for real-time streaming
• Static file serving for the frontend
• Proxy pass to api:5001 for API requests
• Proxy pass to web:3000 for frontend requests

Key nginx directives for Dify:

# Required for SSE streaming
proxy_buffering off;

# Required for WebSocket
proxy_http_version 1.1;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection "upgrade";

Source Deployment

Backend Setup

cd api
pip install uv
uv sync --all-extras

# Configure
cp .env.example .env
# Edit .env

# Database initialization
uv run flask upgrade-db

# Start API server
uv run gunicorn --bind 0.0.0.0:5001 \
  --worker-class gevent \
  --workers 1 \
  --timeout 200 \
  app:app

# Start Celery worker (separate terminal)
uv run celery -A app.celery worker \
  -Q dataset,generation,mail,ops_trace \
  --loglevel INFO

Frontend Setup

cd web
pnpm install
pnpm build
pnpm start

Development mode with hot reload:

pnpm dev
# With debugging proxy:
pnpm dev:proxy

Monorepo Structure

The repository is a pnpm monorepo:

dify/
  package.json          # Root workspace config (pnpm 10.33.0, Node 22.22.1)
  web/                  # Main frontend (Next.js 15, version 1.13.3)
  e2e/                  # End-to-end tests (Playwright + Cucumber)
  packages/
    dify-ui/            # Shared UI components (@langgenius/dify-ui)
    iconify-collections/ # Custom icon sets
    tsconfig/           # Shared TypeScript configs
    migrate-no-unchecked-indexed-access/  # TS migration tool
  sdks/
    nodejs-client/      # Node.js SDK (dify-client v3.1.0)
    php-client/         # PHP SDK (Guzzle-based)

SSRF Proxy

The ssrf_proxy container runs Squid configured to block requests to internal/private network ranges. All outbound HTTP requests from workflow HTTP Request nodes and tool calls are routed through this proxy to prevent Server-Side Request Forgery attacks.

Configuration:

SSRF_PROXY_HTTP_URL=http://ssrf_proxy:3128
SSRF_PROXY_HTTPS_URL=http://ssrf_proxy:3128
SSRF_DEFAULT_MAX_RETRIES=3
SSRF_DEFAULT_TIME_OUT=5
SSRF_DEFAULT_CONNECT_TIME_OUT=5
SSRF_DEFAULT_READ_TIME_OUT=5
SSRF_DEFAULT_WRITE_TIME_OUT=5

Sandbox (Code Execution)

The sandbox container provides isolated execution for Code nodes in workflows. It runs user-submitted Python and JavaScript in a restricted environment with resource limits.

CODE_EXECUTION_ENDPOINT=http://127.0.0.1:8194
CODE_EXECUTION_API_KEY=dify-sandbox
CODE_MAX_NUMBER=9223372036854775807
CODE_MIN_NUMBER=-9223372036854775808
CODE_MAX_STRING_LENGTH=80000
CODE_MAX_STRING_ARRAY_LENGTH=30
CODE_MAX_OBJECT_ARRAY_LENGTH=30
CODE_MAX_NUMBER_ARRAY_LENGTH=1000
TEMPLATE_TRANSFORM_MAX_LENGTH=80000

The sandbox config lives at docker/volumes/sandbox/conf/.

Configuration Reference

All configuration is via environment variables, defined in .env. The authoritative list is in docker/.env.example. Below is the complete reference organized by category.

Core Application

URLs

Database (PostgreSQL)

Caution

Change DB_PASSWORD and REDIS_PASSWORD from their defaults before exposing Dify to any network. The defaults (difyai123456) are for local development only.

Redis

Storage

Vector Store

Authentication and Sessions

CORS

Mail

Notion Integration

Upload Limits

Workflow Execution

Application Limits

Code Execution (Sandbox)

HTTP Request Node

API Tool

Webhook

Plugin System

Marketplace

Endpoint Extensions

SSRF Protection

Model Configuration

Indexing

Proxy

Position Configuration

Billing (Enterprise)

Sentry

Telemetry

Misc

Frontend Environment Variables

All NEXT_PUBLIC_* variables are embedded at build time and available to the client:

Security

Authentication Architecture

Dify uses JWT-based authentication with access/refresh token pairs:

1. Login -- User authenticates with email/password or OAuth (Google)
2. Access Token -- Short-lived JWT (ACCESS_TOKEN_EXPIRE_MINUTES, default 60 min)
3. Refresh Token -- Long-lived token (REFRESH_TOKEN_EXPIRE_DAYS, default 30 days)
4. API Keys -- Per-app keys for Service API access (Bearer token)

Brute-Force Protection

Failed login attempts trigger account lockout after repeated failures. The lockout duration is controlled by LOGIN_LOCKOUT_DURATION (default: 86400 seconds / 24 hours).

OAuth Integration

Google OAuth is supported natively. Configure via GOOGLE_CLIENT_ID and GOOGLE_CLIENT_SECRET in .env.

Encryption

LLM Credential Storage

LLM API keys and model credentials are encrypted at rest using workspace-specific asymmetric key pairs. Each workspace generates a unique key pair on creation.

Caution

Running flask reset-encrypt-key-pair regenerates the workspace encryption keys. All stored LLM credentials become invalid and must be re-entered. Only use this command when the key pair is compromised.

Secret Key

The SECRET_KEY is used for:

• Flask session signing
• JWT token signing
• CSRF protection

Generate a strong key: openssl rand -base64 42

API Key Security

• Service API keys are scoped to individual apps
• Keys are hashed before storage; the plaintext is shown only once on creation
• Rotate keys regularly; old keys can be revoked from the app settings

SSRF Protection

The ssrf_proxy container blocks outbound requests to private network ranges (10.0.0.0/8, 172.16.0.0/12, 192.168.0.0/16, etc.) to prevent Server-Side Request Forgery attacks from user-controlled workflow nodes.

Content Security

Code Sandbox Isolation

User-submitted code in workflow Code nodes runs in the sandbox container with:

• Process isolation from the main application
• Resource limits (CPU, memory, execution time)
• No network access (unless explicitly configured)
• No filesystem access outside the sandbox

Content Moderation

Three layers of content moderation are available:

1. OpenAI Moderation API -- Automated content classification
2. Custom Keywords -- Regex/exact-match blocklist
3. External API -- Custom moderation endpoint

File Access Control

Files are served with signed URLs that expire after FILES_ACCESS_TIMEOUT seconds (default: 300). The signature prevents unauthorized access to uploaded files.

CORS Configuration

Restrict allowed origins in production:

WEB_API_CORS_ALLOW_ORIGINS=https://your-domain.com
CONSOLE_CORS_ALLOW_ORIGINS=https://your-domain.com

Warning

Never use * for CORS origins in production. This allows any website to make API requests on behalf of authenticated users.

X-Forwarded Headers

Set RESPECT_XFORWARD_HEADERS_ENABLED=true only when Dify is behind a trusted reverse proxy. This tells Flask to trust X-Forwarded-For, X-Forwarded-Proto, and X-Forwarded-Host headers for correct URL generation and client IP detection.

Operations

Health Checks

Backup and Restore

PostgreSQL

# Backup
docker compose exec db pg_dump -U postgres dify > dify_backup_$(date +%Y%m%d).sql

# Restore
docker compose exec -T db psql -U postgres dify < dify_backup_20260513.sql

Redis

Redis data is ephemeral (cache, sessions) and does not require routine backup. For persistence, configure Redis RDB/AOF snapshots in redis.conf.

File Storage

Back up the storage directory based on your STORAGE_TYPE:

• opendal (fs): Back up the directory at OPENDAL_FS_ROOT
• S3/cloud: Rely on provider's backup mechanisms

Complete Backup Checklist

1. PostgreSQL dump
2. .env file
3. Storage directory (uploads, keys)
4. nginx configuration (if customized)
5. Plugin data (if using custom plugins)

Scaling

Horizontal Scaling

• API servers: Run multiple api containers behind a load balancer. Ensure all share the same SECRET_KEY, database, and Redis.
• Celery workers: Add worker containers with --concurrency flag. All workers share the same Redis broker.
• Frontend: The web container is stateless; scale freely.

Vertical Scaling

• Database: Increase PostgreSQL shared_buffers, work_mem, max_connections
• Redis: Increase maxmemory; configure eviction policy (allkeys-lru recommended)
• Vector store: Scale according to your backend's documentation

Queue Scaling

Celery queues and their purposes:

• dataset -- Document processing, embedding, indexing
• generation -- LLM generation tasks
• mail -- Email delivery
• ops_trace -- Operational tracing/logging

Scale selectively:

# Dedicated worker for dataset processing
celery -A app.celery worker -Q dataset --concurrency 4

# Dedicated worker for generation
celery -A app.celery worker -Q generation --concurrency 2

Log Management

API Logs

# View recent API logs
docker compose logs --tail 200 api

# Follow logs in real time
docker compose logs -f api

# Filter by level
docker compose logs api 2>&1 | grep ERROR

Log Configuration

LOG_LEVEL=INFO
LOG_FILE=/var/log/dify/api.log
LOG_FILE_MAX_SIZE=20
LOG_FILE_BACKUP_COUNT=5
LOG_TZ=UTC

Aliyun SLS Integration

For cloud-based log management:

ALIYUN_SLS_ENDPOINT=<endpoint>
ALIYUN_SLS_REGION=<region>
ALIYUN_SLS_PROJECT_NAME=<project>
ALIYUN_SLS_CHECK_CONNECTIVITY_TIMEOUT=30

Database Migrations

Migrations use Flask-Migrate (Alembic). They run automatically on API container startup.

# Run migrations manually
docker compose exec api flask upgrade-db

# Check current migration version
docker compose exec api flask db current

Storage Migration

Migrate files between storage backends:

docker compose exec api flask migrate-oss \
  --source opendal \
  --path upload_files \
  --path image_files \
  --dry-run

# Execute the migration
docker compose exec api flask migrate-oss \
  --source opendal \
  --update-db

Supported paths: privkeys, upload_files, image_files, tools, website_files, keyword_files, ops_trace.

Data Retention

Workflow Runs

# Preview cleanup (dry run)
docker compose exec api flask clean-workflow-runs \
  --before-days 30 --dry-run

# Execute cleanup
docker compose exec api flask clean-workflow-runs \
  --before-days 30 --batch-size 200

# Archive to S3 before deleting
docker compose exec api flask archive-workflow-runs \
  --before-days 90 --delete-after-archive --workers 4

# Restore from archive
docker compose exec api flask restore-workflow-runs \
  --tenant-ids "tenant-uuid" --run-id "run-uuid"

Messages

docker compose exec api flask clean-expired-messages \
  --before-days 90 --graceful-period 21 --dry-run

The --graceful-period adds extra days after subscription expiration before cleanup.

Export Data

docker compose exec api flask export-app-messages \
  --app-id "app-uuid" \
  --end-before "2026-01-01" \
  --filename "export/messages" \
  --batch-size 1000

# Upload directly to cloud storage
docker compose exec api flask export-app-messages \
  --app-id "app-uuid" \
  --end-before "2026-01-01" \
  --use-cloud-storage

Orphaned Resources

# Find orphaned file records (DB records without storage files)
docker compose exec api flask clear-orphaned-file-records --force

# Find orphaned storage files (storage files without DB records)
docker compose exec api flask remove-orphaned-files-on-storage --force

# Query file usage
docker compose exec api flask file-usage --file-id "uuid" --json

Draft Variables

docker compose exec api flask cleanup_orphaned_draft_variables \
  --dry-run --batch-size 1000 --max-apps 100

Workflow Engine Internals

Graph Execution Model

The workflow engine uses a DAG-based graph executor built on the graphon package. Key concepts:

1. Graph Definition -- Nodes and edges defined in the workflow DSL (JSON/YAML)
2. Topological Execution -- Nodes execute in dependency order
3. Parallel Branches -- Independent branches run concurrently via a ThreadPoolExecutor (max MAX_SUBMIT_COUNT = 100 threads)
4. Variable Aggregation -- Variable Aggregator nodes act as join points, waiting for all upstream branches

Execution Lifecycle

Start Node
  -> Resolve dependencies
  -> Execute node
    -> If parallel paths: submit to thread pool
    -> If conditional: evaluate IF/ELSE
    -> If iteration: loop over items
    -> If sub-workflow: recursive execution (depth <= WORKFLOW_CALL_MAX_DEPTH)
  -> Collect outputs
  -> Route to next node(s)
  -> ... repeat ...
End Node
  -> Assemble final outputs
  -> Persist workflow_run record

Node Execution Details

Each node type has a specific executor:

• LLM Node -- Assembles prompt from template + variables, calls model, streams response
• Knowledge Retrieval Node -- Queries vector store, optionally re-ranks, returns chunks
• Code Node -- Sends code to sandbox via HTTP, receives output
• HTTP Request Node -- Makes HTTP call through SSRF proxy, parses response
• Sub-Workflow Node -- Creates a child workflow run, shares trace_id for observability

Human-in-the-Loop

Workflows support human input nodes that pause execution and wait for user approval/input. The resume mechanism:

1. Workflow reaches human input node
2. Execution state is persisted to PostgreSQL
3. API returns a task_id for the paused run
4. User submits input via API
5. Workflow resumes from the paused node

Draft/Preview Mode

Single-node execution for debugging:

• Creates its own trace_id derived from node_execution_id
• Does not create a workflow run record
• Useful for testing LLM prompts, code, and HTTP requests in isolation

RAG Pipeline Internals

Document Processing Pipeline

Upload -> Extractor -> Cleaner -> Chunker -> Embedder -> Vector Store

Extractors

Chunking Strategies

1. Automatic -- Heuristic-based splitting by paragraphs and sections
2. Custom -- User-defined separator (e.g., \n\n) + max chunk size + overlap
3. Parent-Child -- Hierarchical chunking for context preservation

Key parameter: INDEXING_MAX_SEGMENTATION_TOKENS_LENGTH (default: 4000) sets the maximum tokens per chunk.

Embedding Pipeline

1. Text chunks are batched and sent to the configured embedding model
2. Vectors are stored in the configured vector backend
3. Full-text indexes are maintained in parallel (for keyword/hybrid search)
4. The dataset Celery queue handles all processing asynchronously

Retrieval Pipeline

Query -> Query Embedding -> Vector Search (Top-K)
              |
              +-> Full-Text Search (BM25)
              |
              +-> Hybrid (merge + deduplicate)
                    |
                    +-> Reranker (optional)
                          |
                          +-> Score Threshold Filter
                                |
                                +-> Context Assembly -> LLM

Vector Store Registration System

Vector backends are registered via Python entry points under the dify.vector_backends group. The discovery mechanism:

1. Backend packages live in api/providers/vdb/<backend>/
2. Each has a pyproject.toml with entry point registration
3. vector_backend_registry.py lazily loads and caches factory classes
4. At runtime, get_vector_factory_class(vector_type) resolves the backend

See Vector Database Backend System for details.

Plugin System Internals

Architecture

Flask API
  |
  v
Inner API (/inner/api/*)
  |  (INNER_API_KEY_FOR_PLUGIN auth)
  v
plugin_daemon (Go binary, :5002)
  |
  +-> Plugin Process 1 (stdio JSON-RPC)
  +-> Plugin Process 2 (stdio JSON-RPC)
  +-> ...

Plugin Types

Plugin Package Format

Plugins are distributed as .difypkg files containing:

• Manifest (plugin metadata, version, dependencies)
• Implementation code
• Configuration schema

Plugin Communication

The plugin_daemon manages plugin lifecycle:

1. Plugin installation (download, extract, validate)
2. Plugin startup (spawn process, establish stdio JSON-RPC channel)
3. Request routing (API -> Inner API -> daemon -> plugin process)
4. Plugin shutdown and cleanup

Plugin CLI Commands

# Migrate existing data for plugin system
flask migrate-data-for-plugin

# Extract plugin definitions from existing configuration
flask extract-plugins --output_file plugins.jsonl --workers 10

# Bulk install plugins
flask install-plugins --input_file plugins.jsonl --output_file installed_plugins.jsonl --workers 100

# Install RAG pipeline plugins
flask install-rag-pipeline-plugins --workers 100

# Configure OAuth for tool plugins
flask setup-system-tool-oauth-client --provider github --client-params '{"client_id":"...","client_secret":"..."}'

# Configure OAuth for trigger plugins
flask setup-system-trigger-oauth-client --provider github --client-params '...'

# Configure OAuth for datasource plugins
flask setup-datasource-oauth-client --provider notion --client-params '...'

# Transform datasource credentials
flask transform-datasource-credentials --environment online

Vector Database Backend System

Architecture

The vector store system uses a plugin architecture based on Python entry points:

api/core/rag/datasource/vdb/
  vector_factory.py         # Main entry: Vector class
  vector_backend_registry.py # Entry point discovery + caching
  abstract_vector_factory.py # AbstractVectorFactory interface
  base_vector.py            # BaseVector interface

api/providers/vdb/
  <backend>/
    pyproject.toml          # Package + entry point registration
    src/dify_vdb_<backend>/
      <backend>_vector.py   # Implementation

Interfaces

Complete Backend List

Adding a New Backend

1. Create api/providers/vdb/<your_backend>/
2. Add pyproject.toml:

   [project]
   name = "dify-vdb-your-backend"
   version = "0.0.1"
   dependencies = ["your-client-lib>=1.0"]
   
   [project.entry-points."dify.vector_backends"]
   your_backend = "dify_vdb_your_backend.your_backend_vector:YourBackendFactory"
   
   [tool.setuptools.packages.find]
   where = ["src"]
   

3. Implement AbstractVectorFactory and BaseVector
4. Add to api/pyproject.toml:
   • [tool.uv.sources]: dify-vdb-your-backend = { workspace = true }
   • [project.optional-dependencies]: add to vdb-all group
5. Run uv sync in api/

Vector Migration

flask vdb-migrate --scope all

Migrates all datasets from the current vector backend to the one configured in VECTOR_STORE.

Frontend Architecture

Technology Stack

Project Structure

web/
  app/                    # Next.js App Router pages
    (commonLayout)/       # Authenticated layout group
      datasets/           # Knowledge base UI
      apps/               # Application management
    signin/               # Authentication pages
    components/           # Shared components
      workflow/           # Workflow editor (ReactFlow)
      datasets/           # Dataset management
      plugins/            # Plugin management
      billing/            # Billing (enterprise)
  config/                 # App configuration
  hooks/                  # Custom React hooks (26 files)
  service/                # API service layer (55 files)
  models/                 # TypeScript type definitions (10 files)
  utils/                  # Utility functions (52 files)
  i18n/                   # Translations (20+ locales)
  i18n-config/            # i18n configuration
  contract/               # API contracts / types
  constants/              # App constants
  public/                 # Static assets
    vs/                   # Monaco editor (bundled)
  docker/                 # Frontend Dockerfile

Build and Development

# Development
pnpm dev                  # Next.js dev server
pnpm dev:proxy            # Dev with Hono proxy
pnpm dev:vinext           # Vite-based dev (experimental)

# Build
pnpm build                # Next.js production build
pnpm build:vinext         # Vite-based build (experimental)

# Quality
pnpm type-check           # TypeScript checking
pnpm lint                 # ESLint
pnpm test                 # Vitest tests
pnpm test:coverage        # Test coverage
pnpm storybook            # Component development

# Utilities
pnpm gen-icons            # Regenerate icon components
pnpm i18n:check           # Validate translations
pnpm knip                 # Find unused code
pnpm analyze              # Bundle analysis

Internationalization

Translations live in web/i18n/<locale>/ (30 files per locale). Supported locales: en-US, zh-Hans, zh-Hant, ja-JP, ko-KR, de-DE, fr-FR, it-IT, es-ES, pt-BR, ru-RU, tr-TR, vi-VN, th-TH, ar-TN, hi-IN, pl-PL, ro-RO, sl-SI, uk-UA, fa-IR.

Backend Architecture

Module Organization

api/
  app.py                  # Entry point (Flask + gevent WebSocket)
  app_factory.py          # Flask app factory (gevent monkey-patch)
  commands/               # Flask CLI commands (7 files, 129 commands)
    account.py            # reset-password, reset-email, create-tenant
    plugin.py             # Plugin migration and OAuth setup
    retention.py          # Data cleanup and archival
    storage.py            # File management and migration
    system.py             # DB upgrades, encryption, app conversion
    vector.py             # Vector DB migration
  configs/                # Pydantic v2 configuration
    middleware/            # Redis, PostgreSQL, storage, etc.
  constants/              # Application constants (6 files, 350KB)
  context/                # Request context management (4 files)
  controllers/            # API route handlers
    console/              # Console API (16 files + app/ 25 files + auth/ 9 files)
    service_api/          # Service API (3 files + app/ 11 files)
    web/                  # Web API (19 files)
    inner_api/            # Inner API (app/ 2 files + plugin/ 2 files)
    common/               # Shared controller utilities (6 files)
  core/                   # Core business logic
    app/apps/             # Application runtime (pipeline, common)
    moderation/           # Content moderation
    external_data_tool/   # External data integration
    tools/                # Tool system (12 util files)
    trigger/              # Trigger system
    plugin/               # Plugin integration (3 util files)
    rag/datasource/vdb/   # Vector store interfaces
    prompt/               # Prompt utilities (4 files)
    schemas/              # Core schemas (4 files)
    errors/               # Error definitions
    datasource/           # Data source utilities
  enums/                  # Enum definitions (3 files)
  extensions/             # Flask extensions
    otel/                 # OpenTelemetry integration
    ext_logstore.py       # Aliyun SLS log integration
  fields/                 # API response field definitions (26 files)
  libs/                   # Utility libraries (35 files)
  migrations/             # Alembic database migrations
  models/                 # SQLAlchemy ORM models (22 files, 395KB)
  repositories/           # Data access layer (11 files)
  schedule/               # Celery beat scheduled tasks (13 files)
  services/               # Business logic services (57 files, 1MB)
    auth/                 # Authentication services (5 files)
    plugin/               # Plugin services (9 files)
  providers/vdb/          # Vector database backends (30+ packages)

Key Modules by Size

Configuration System

Dify uses Pydantic v2 settings classes for configuration (api/configs/). The middleware config (api/configs/middleware/) defines typed settings for:

• PostgreSQL connection
• Redis connection
• Storage backends
• Vector store selection
• Celery configuration

All settings are loaded from environment variables at startup.

Error Handling

The codebase defines specific error classes for different domains:

• Auth errors: ApiKeyAuthFailedError, InvalidEmailError, PasswordMismatchError, AccountLoginError, UnauthorizedError
• Dataset errors: NoFileUploadedError, UnsupportedFileTypeError, FileTooLargeError, TooManyFilesError
• Workspace errors: WorkspaceMemberNotFoundError, WorkspaceQuotaExhaustedError

Errors are caught by Flask error handlers and returned as structured JSON responses.

Database Schema and Migrations

ORM Layer

Dify uses SQLAlchemy ORM with 22 model files defining the complete schema. Key models include:

• Account -- User accounts, credentials, OAuth tokens
• Tenant -- Workspaces, member relationships, roles
• App -- Application definitions, configurations, DSL
• Conversation -- Chat conversations, message history
• Message -- Individual messages with metadata
• Dataset -- Knowledge base definitions
• Document -- Documents within datasets
• DocumentSegment -- Chunks/segments with embeddings
• WorkflowRun -- Workflow execution records
• WorkflowNodeExecution -- Per-node execution records
• InstalledPlugin -- Plugin installation records
• ApiToken -- API key records (hashed)
• UploadFile -- File upload metadata

Migration Management

Migrations use Flask-Migrate (Alembic wrapper):

# Run pending migrations
flask upgrade-db

# Check current version
flask db current

# Generate new migration (development)
flask db migrate -m "description"

Migrations run automatically on API container startup. The database uses WAL mode with foreign keys enabled.

Celery Task System

Queue Architecture

Scheduled Tasks

Celery Beat runs scheduled tasks defined in api/schedule/ (13 files). The beat polling interval is CELERY_BEAT_SCHEDULER_TIME (default: 1 second).

Worker Configuration

# Single worker, all queues
celery -A app.celery worker -Q dataset,generation,mail,ops_trace --loglevel INFO

# Dedicated workers per queue
celery -A app.celery worker -Q dataset --concurrency 4 --loglevel INFO
celery -A app.celery worker -Q generation --concurrency 2 --loglevel INFO

Broker Configuration

Redis is the default and only supported broker:

CELERY_BROKER_URL=redis://:password@redis:6379/1

The broker uses Redis DB 1 (separate from the main cache on DB 0).

Observability and Telemetry

Enterprise Telemetry (OTEL)

Dify Enterprise supports OpenTelemetry integration with a "slim span + rich companion log" architecture:

Configuration

Trace ID Correlation

• Workflow runs: trace_id = UUID(workflow_run_id)
• Node executions: span_id = hash(node_execution_id), child of workflow span
• Sub-workflows: Shares parent's trace_id, linked via parent_span_id
• Draft executions: Own trace_id = UUID(node_execution_id) (not part of any workflow trace)

Content Gating

When ENTERPRISE_INCLUDE_CONTENT=false, sensitive content is replaced with reference strings:

ref:workflow_run_id=550e8400-e29b-41d4-a716-446655440000
ref:node_execution_id=660e8400-e29b-41d4-a716-446655440001

Sentry Integration

Set SENTRY_DSN for backend error tracking and NEXT_PUBLIC_SENTRY_DSN for frontend.

Amplitude Analytics

Set NEXT_PUBLIC_AMPLITUDE_API_KEY for frontend user analytics.

Code Patterns and Conventions

Backend Patterns

Frontend Patterns

API Response Format

API responses follow a consistent structure:

{
  "result": "success",
  "data": { ... }
}

Error responses:

{
  "code": "error_code",
  "message": "Human-readable description",
  "status": 400
}

Testing

# Backend unit tests
cd api
uv run pytest tests/unit_tests/

# Backend integration tests
uv run pytest tests/integration_tests/

# Specific test
uv run pytest tests/unit_tests/core/workflow/graph_engine/layers/test_llm_quota.py

# Frontend tests
cd web
pnpm test
pnpm test:coverage

SDKs and Client Libraries

Node.js SDK (dify-client v3.1.0)

npm install dify-client

Requires Node.js >= 18.0.0. ESM-only ("type": "module").

PHP SDK

Based on Guzzle HTTP client v7.9. Requires PHP >= 7.2.

composer require langgenius/dify-client

Python SDK

Maintained in a separate repository: github.com/langgenius/dify-sdk-python.

E2E Testing

Framework

End-to-end tests use Playwright + Cucumber (BDD):

cd e2e
pnpm e2e:install        # Install Playwright + Chromium
pnpm e2e:middleware:up   # Start test infrastructure
pnpm e2e                 # Run tests
pnpm e2e:full            # Full test suite
pnpm e2e:full:headed     # Full suite with browser UI
pnpm e2e:reset           # Reset test state
pnpm e2e:middleware:down # Stop test infrastructure

Tests are in e2e/features/ with step definitions in e2e/features/step-definitions/.

Contributing

Development Setup

1. Fork and clone the repository
2. Set up the backend and frontend as described in Source Deployment
3. Create a feature branch

Code Quality Tools

# Linting (root)
pnpm lint
pnpm lint:fix

# Type checking (frontend)
cd web && pnpm type-check

# Backend tests
cd api && uv run pytest

# Frontend tests
cd web && pnpm test

Commit and PR Guidelines

• PRs should target the main branch
• Include tests for new features
• Follow existing code patterns and naming conventions
• Update translations for UI-facing changes

Known Issues and Improvements

Open Issues (from GitHub)

Key bugs and feature requests from the issue tracker:

• Performance: Workflow execution with many parallel branches can hit thread pool limits (MAX_SUBMIT_COUNT)
• Vector Migration: vdb-migrate requires downtime; no online migration path
• Plugin Isolation: Plugin crashes can affect the daemon's ability to route to other plugins
• Large File Processing: Documents near the UPLOAD_FILE_SIZE_LIMIT can cause worker memory spikes during extraction

Architecture Improvement Opportunities

1. API server decomposition -- api/controllers/console/app/ (25 files, 329 KB) handles too many concerns; could benefit from further route grouping
2. Service layer complexity -- api/services/ (57 files, 1 MB) is the largest module; some services could be broken into domain-specific sub-modules
3. Workflow engine hardening -- The graph engine delegates to graphon but the Dify layer adds complexity with human-input resume and sub-workflow recursion
4. Frontend bundle size -- The bundled Monaco editor (web/public/vs/, 377 KB compressed) loads on all pages; could be lazy-loaded
5. Test coverage -- Integration tests exist but are concentrated in api/tests/test_containers_integration_tests/services (65 files, 1.7 MB); unit test coverage for controllers is lighter

CLI Reference

All CLI commands are invoked via Flask. In Docker: docker compose exec api flask <command>.

Account Commands (api/commands/account.py)

reset-password

Reset an account's password.

flask reset-password \
  --email [email removed] \
  --new-password "SecurePassword123" \
  --password-confirm "SecurePassword123"

reset-email

Change an account's email address.

flask reset-email \
  --email [email removed] \
  --new-email [email removed] \
  --email-confirm [email removed]

create-tenant

Create a new workspace and account.

flask create-tenant \
  --email [email removed] \
  --name "My Workspace" \
  --language en-US

Plugin Commands (api/commands/plugin.py)

setup-system-tool-oauth-client

Configure OAuth for a tool plugin provider.

flask setup-system-tool-oauth-client \
  --provider github \
  --client-params '{"client_id":"...","client_secret":"..."}'

setup-system-trigger-oauth-client

Configure OAuth for a trigger plugin provider.

flask setup-system-trigger-oauth-client \
  --provider github \
  --client-params '{"client_id":"...","client_secret":"..."}'

setup-datasource-oauth-client

Configure OAuth for a datasource plugin provider.

flask setup-datasource-oauth-client \
  --provider notion \
  --client-params '{"client_id":"...","client_secret":"..."}'

transform-datasource-credentials

Transform datasource credentials for a specific environment.

flask transform-datasource-credentials --environment online

migrate-data-for-plugin

Migrate existing data for the plugin system.

flask migrate-data-for-plugin

extract-plugins

Extract plugin definitions from existing configuration.

flask extract-plugins --output_file plugins.jsonl --workers 10

extract-unique-identifiers

Extract unique plugin identifiers.

flask extract-unique-identifiers \
  --input_file plugins.jsonl \
  --output_file unique_identifiers.json

install-plugins

Batch install plugins from extracted definitions.

flask install-plugins \
  --input_file plugins.jsonl \
  --output_file installed_plugins.jsonl \
  --workers 100

install-rag-pipeline-plugins

Install RAG pipeline plugins.

flask install-rag-pipeline-plugins \
  --input_file plugins.jsonl \
  --output_file installed_plugins.jsonl \
  --workers 100

Retention Commands (api/commands/retention.py)

clear-free-plan-tenant-expired-logs

Clear expired logs for free-plan tenants.

flask clear-free-plan-tenant-expired-logs \
  --days 30 \
  --batch 100 \
  --tenant_ids "uuid1,uuid2"

clean-workflow-runs

Delete expired workflow runs and related data.

flask clean-workflow-runs \
  --before-days 30 \
  --batch-size 200 \
  --dry-run

# Date range mode
flask clean-workflow-runs \
  --start-from "2026-01-01" \
  --end-before "2026-04-01"

# Relative range mode
flask clean-workflow-runs \
  --from-days-ago 90 \
  --to-days-ago 30

archive-workflow-runs

Archive workflow runs to S3-compatible storage.

flask archive-workflow-runs \
  --before-days 90 \
  --workers 4 \
  --limit 1000 \
  --delete-after-archive \
  --dry-run

# Grayscale rollout
flask archive-workflow-runs \
  --tenant-ids "uuid1,uuid2" \
  --before-days 90

restore-workflow-runs

Restore archived workflow runs.

flask restore-workflow-runs \
  --tenant-ids "uuid1" \
  --run-id "run-uuid" \
  --workers 4 \
  --limit 100

delete-archived-workflow-runs

Delete archived workflow runs from the database.

flask delete-archived-workflow-runs \
  --tenant-ids "uuid1" \
  --run-id "run-uuid" \
  --dry-run

cleanup_orphaned_draft_variables

Remove orphaned draft variables.

flask cleanup_orphaned_draft_variables \
  --dry-run \
  --batch-size 1000 \
  --max-apps 100 \
  --force

clean-expired-messages

Clean expired messages.

flask clean-expired-messages \
  --before-days 90 \
  --from-days-ago 180 \
  --batch-size 1000 \
  --graceful-period 21 \
  --dry-run

export-app-messages

Export messages for an app to JSONL.GZ.

flask export-app-messages \
  --app-id "app-uuid" \
  --end-before "2026-06-01" \
  --filename "export/messages" \
  --batch-size 1000 \
  --use-cloud-storage \
  --dry-run

Storage Commands (api/commands/storage.py)

clear-orphaned-file-records

Remove database records for files that no longer exist in storage.

flask clear-orphaned-file-records --force

remove-orphaned-files-on-storage

Delete storage files that have no corresponding database records.

flask remove-orphaned-files-on-storage --force

file-usage

Query where files are referenced in the system.

flask file-usage --file-id "uuid"
flask file-usage --key "upload_files/path"
flask file-usage --src "documents.%" --limit 50 --json

migrate-oss

Migrate files from local/OpenDAL to cloud storage.

flask migrate-oss \
  --source opendal \
  --path upload_files \
  --path image_files \
  --overwrite \
  --update-db \
  --dry-run

Default paths: privkeys, upload_files, image_files, tools, website_files, keyword_files, ops_trace.

System Commands (api/commands/system.py)

reset-encrypt-key-pair

Reset workspace asymmetric key pair for credential encryption.

flask reset-encrypt-key-pair

Caution

This invalidates ALL stored LLM credentials across ALL workspaces. Every model provider must be re-configured after running this command. Only use when the key pair is compromised. Only works in SELF_HOSTED mode.

convert-to-agent-apps

Convert legacy Agent Assistant apps to the new Agent App type.

flask convert-to-agent-apps

upgrade-db

Run pending database migrations.

flask upgrade-db

fix-app-site-missing

Repair apps that are missing their associated site records.

flask fix-app-site-missing

Vector Commands (api/commands/vector.py)

vdb-migrate

Migrate vectors between backends.

flask vdb-migrate --scope all

The --scope option accepts all (default) to migrate all datasets.

add-qdrant-index

Add an index to Qdrant collections.

flask add-qdrant-index --field metadata.doc_id

old-metadata-migration

Migrate legacy vector metadata format.

flask old-metadata-migration

Quick Reference

Docker Commands

# Start all services
docker compose up -d

# Stop all services
docker compose down

# View logs
docker compose logs -f api
docker compose logs --tail 100 worker

# Restart a service
docker compose restart api

# Shell into container
docker compose exec api bash
docker compose exec db psql -U postgres dify

# Check health
curl http://localhost/health
curl http://localhost/ping

Common Operations Cheat Sheet

API Surface Summary

System Endpoints

Port Reference