Daytona Administrator & Developer Guide

Daytona is a cloud development sandbox platform that provisions isolated, reproducible development environments at scale. This guide covers the complete operational surface: architecture, configuration, deployment, security, and internals for administrators managing Daytona infrastructure and developers extending the platform.

Daytona is a polyglot monorepo (Go + TypeScript) managed by Nx 21.4.1 and Yarn 4.13.0, comprising ten distinct services that coordinate to deliver on-demand sandboxes with full lifecycle management, snapshot persistence, multi-region support, and organization-level resource governance.

1. Architecture Overview

1.1 System Topology

Daytona follows a hub-and-spoke architecture. The API server is the central control plane; runners are distributed worker nodes that execute sandbox operations. All other services (proxy, SSH gateway, dashboard, daemon) connect through the API or directly to runners.

                                 Internet
                                    |
                       +------------+------------+
                       |                         |
                  Dashboard (React)         CLI (Go/Cobra)
                       |                         |
                       v                         v
                 +---------------------------+
                 |     API Server (NestJS)    |
                 |  - Job orchestration       |
                 |  - Auth (OIDC)             |
                 |  - Organization mgmt       |
                 |  - Runner scoring          |
                 |  - Webhook dispatch        |
                 +---------------------------+
                    |        |           |
           +-------+    +---+---+    +--+--------+
           |             |           |            |
      PostgreSQL      Redis       S3/MinIO    ClickHouse
                                                  |
           +----------------------------------+   |
           |          Runner (Go/Gin)         |   |
           |  - Sandbox CRUD                  |   |
           |  - Image build (BuildKit)        |   |
           |  - Snapshot management           |   |
           +----------------------------------+   |
              |              |            |       |
         +---+---+    +-----+----+  +----+----+  |
         | Proxy |    |SSH Gateway|  | Daemon  |  |
         | (Go)  |    |   (Go)   |  |  (Go)   |  |
         +-------+    +----------+  +---------+  |
                                                  |
                                    OTel Collector

1.2 Component Responsibilities

1.3 SDK Ecosystem

Daytona provides first-party SDKs for programmatic sandbox management:

Each SDK has a corresponding auto-generated API client (both sync and async variants for Python). The clients are generated from the OpenAPI specification the API server exposes.

1.4 Communication Patterns

API to Runner: The API server dispatches jobs to runners via a polling mechanism. Runners poll the API at a configurable interval (POLL_TIMEOUT, default 30s) for pending jobs up to POLL_LIMIT (default 10) at a time.

Runner to API: Runners report job status, health metrics (CPU, memory, disk), and sandbox state transitions back to the API.

Proxy Routing: The proxy resolves sandbox-specific subdomains to the correct runner and sandbox, forwarding traffic transparently. The proxy domain and protocol are configured via PROXY_DOMAIN and PROXY_PROTOCOL.

Webhook Delivery: Webhook events are dispatched via Svix. Each organization manages its own webhook endpoints. Events cover sandbox lifecycle, snapshot operations, and organization changes.

Telemetry Pipeline: Sandbox-level metrics flow through the custom OTel collector, which exports to ClickHouse for analytics and optionally to PostHog for product telemetry.

Note

The runner uses a pull-based job model, not push. The API never initiates connections to runners. This simplifies firewall rules: runners only need outbound HTTPS to the API server.

2. System Requirements

2.1 API Server Node

2.2 Runner Node

Default runner resource allocation per sandbox: 4 CPU cores, 8 GB memory, 50 GB disk (configurable via DEFAULT_RUNNER_CPU, DEFAULT_RUNNER_MEMORY, DEFAULT_RUNNER_DISK).

2.3 Infrastructure Dependencies

Important

PostgreSQL must run in WAL (Write-Ahead Logging) mode. TypeORM manages the connection pool, but WAL mode must be enabled at the PostgreSQL server level for the concurrent access patterns Daytona's job system generates.

2.4 Network Requirements

2.5 Build Toolchain (Development)

3. Configuration Reference

3.1 Core Application

These variables control the API server's fundamental behavior.

Warning

Setting RUN_MIGRATIONS=true on every instance in a multi-replica deployment will cause migration race conditions. Designate exactly one instance to run migrations, or use a separate migration job before rolling out new instances.

3.2 Database (PostgreSQL)

Connection pool tuning guidelines:

• DB_POOL_MAX should not exceed PostgreSQL's max_connections divided by the number of API replicas.
• For a single-replica deployment handling moderate load, DB_POOL_MAX=20 and DB_POOL_MIN=5 is a reasonable starting point.
• Set DB_POOL_IDLE_TIMEOUT_MS to 30000 (30s) to reclaim idle connections without excessive reconnection overhead.
• DB_POOL_CONNECTION_TIMEOUT_MS of 5000 (5s) surfaces connection exhaustion quickly rather than queueing indefinitely.

3.3 Redis

Redis is used for three distinct purposes: response caching, rate limiting counters, and activity throttling. A single Redis instance handles all three. For high-availability deployments, Redis Sentinel or Redis Cluster can be placed behind the same host/port configuration.

3.4 S3 / Object Storage

Storage backend compatibility:

3.5 OIDC / Authentication

Tip

Both OIDC_* and OID_* prefixes are accepted for client ID, issuer URL, and audience. This dual-prefix support exists for backward compatibility. Prefer the OIDC_* variants in new deployments.

3.6 SMTP

SMTP is used for organization invitations and email notifications. If SMTP is not configured, invitation flows and email-dependent features are silently disabled.

3.7 Docker Registries

Daytona uses four registry types: internal (platform images), transient (ephemeral build artifacts), organization (user-managed), and backup (snapshot storage).

3.8 Proxy

3.9 SSH Gateway

3.10 Webhooks (Svix)

Webhook endpoints are managed per-organization. When both variables are configured, the API server registers webhook event types with Svix and dispatches events for sandbox lifecycle transitions, snapshot operations, and organization changes.

3.11 Rate Limiting

Daytona implements five rate-limiting tiers, each with a TTL (time window in seconds) and a request limit within that window:

Rate limit state is stored in Redis. The Failed Auth tier is particularly important: it defends against credential-stuffing attacks by throttling repeated authentication failures per source IP.

3.12 Organization Quotas

Default quotas applied to new organizations. These can be overridden per-organization through the admin API.

3.13 Admin Overrides

Admin quotas override organization-level limits for administrator accounts.

3.14 Regions

3.15 Runner Defaults (API-side)

These configure the default runner that the API server expects on initial setup.

3.16 Audit and Logging

3.17 Kafka

3.18 OpenSearch

3.19 Telemetry

3.20 ClickHouse

3.21 Encryption and Security

3.22 Caching

3.23 Sandbox Behavior

3.24 Dashboard and UI

3.25 External Integrations

3.26 Runner Service Configuration

These variables are set on the runner nodes themselves (not the API server).

4. Deployment & Scaling

4.1 Deployment Architecture

A production Daytona deployment consists of three tiers:

1. Control Plane: API server instance(s), PostgreSQL, Redis, S3
2. Worker Plane: Runner nodes (one or more, horizontally scalable)
3. Edge Plane: Proxy, SSH gateway, dashboard (stateless, horizontally scalable)

                    Load Balancer (L7)
                    /        |         \
               API (x3)   Proxy (x2)  SSH GW (x2)
                 |           |            |
            PostgreSQL    Redis        (stateless)
                 |
                S3
                 |
        +--------+--------+--------+
        |        |        |        |
    Runner-1  Runner-2  Runner-3  Runner-N

4.2 Database Migrations: Expand and Contract

Daytona uses the Expand and Contract migration pattern for zero-downtime deployments. This is a critical operational detail.

Phase 1: Pre-deploy (Expand)

Pre-deploy migrations are strictly additive. They only add new columns, new tables, or new indexes. They never remove, rename, or alter existing columns. This means the old application code continues to function against the new schema.

# Run pre-deploy migrations before deploying new code
RUN_MIGRATIONS=true node dist/main.js --migrate-only

Phase 2: Deploy

Roll out the new application code. Both old and new code versions work against the expanded schema simultaneously during the rolling update.

Phase 3: Post-deploy (Contract)

After all instances are running the new code, run post-deploy migrations to clean up: remove deprecated columns, drop old tables, tighten constraints.

# Run post-deploy migrations after all instances are updated
RUN_MIGRATIONS=true APP_MODE=post-deploy node dist/main.js --migrate-only

Caution

Never skip the expand phase or run contract migrations while old code is still serving traffic. Removing columns that old code depends on will cause immediate 500 errors across all affected endpoints.

4.3 Multi-Instance API Deployment

The API server supports horizontal scaling with these considerations:

Stateless request handling: All session and cache state lives in Redis. Any API instance can handle any request.

Migration coordination: Set RUN_MIGRATIONS=true on exactly one instance (or run migrations as a separate init container/job). All other instances should have RUN_MIGRATIONS unset or false.

Cron job deduplication: Set DISABLE_CRON_JOBS=true on all but one instance to prevent duplicate scheduled task execution. Alternatively, use APP_ROLE to designate which instance runs cron jobs.

Maintenance mode: Setting MAINTENANCE_MODE=true on any instance causes it to reject non-health-check traffic, useful for draining during rolling updates.

4.4 Runner Scaling

Runners scale horizontally. Each runner:

1. Registers with the API server using its DAYTONA_RUNNER_TOKEN
2. Polls the API for pending jobs
3. Reports health metrics (CPU, memory, disk utilization)
4. Executes sandbox operations locally

Adding a runner:

# On the new runner node
export DAYTONA_API_URL=https://api.daytona.example.com
export DAYTONA_RUNNER_TOKEN=<token-from-api>
export API_PORT=8080
export BUILD_ENGINE=buildkit

./daytona-runner

The API server's runner scoring system automatically distributes workload across available runners based on their reported resource utilization.

Removing a runner: Stop the runner process. The API server detects the runner as unhealthy after RUNNER_HEALTH_TIMEOUT_SECONDS and stops routing new jobs to it. Existing sandboxes on the runner continue to function but cannot be managed until the runner returns or sandboxes are recovered.

4.5 Warm Pool

The warm pool pre-provisions sandboxes to reduce cold-start latency. Configure WARM_POOL_CANDIDATE_LIMIT to set the number of pre-warmed sandboxes maintained. The API server's cron system replenishes the pool as sandboxes are claimed.

4.6 Auto-Archive

Idle sandboxes are automatically archived after MAX_AUTO_ARCHIVE_INTERVAL. Archived sandboxes retain their state in S3 but release runner resources. They can be restored on demand, though restoration incurs a cold-start penalty.

4.7 Backup and Recovery

Each runner supports concurrent backup operations, limited by MAX_CONCURRENT_BACKUPS_PER_RUNNER. Backups are stored in the backup registry.

The RECOVER_SANDBOX job type handles sandbox recovery. The runner exposes an is-recoverable endpoint to check whether a sandbox's state can be recovered before attempting the operation.

5. Operations

5.1 Sandbox Lifecycle State Machine

Every sandbox transitions through a well-defined state machine:

                                 +---> error
                                 |
  pending_build --> building_snapshot --> pulling_snapshot --> creating
                                                                |
                                                                v
                                          +---------- started <---+
                                          |              |        |
                                          v              v        |
                                       stopped      (running)    |
                                          |                       |
                                          v                       |
                                       archived                   |
                                          |                       |
                                          +----> restoring -------+
                                          |
                                          v
                                       destroying --> destroyed

Primary states:

Transitional states:

5.2 Job System

The job system is the mechanism by which the API server dispatches work to runners.

Job types:

Job flow:

1. API server creates a job record in PostgreSQL
2. Runner polls GET /jobs on the API
3. Runner claims a job (atomic update)
4. Runner executes the operation
5. Runner reports completion/failure back to the API
6. API updates sandbox state accordingly

5.3 Runner Internal API

The runner exposes 19 internal endpoints for sandbox and image management:

Sandbox operations:

Image operations:

Other:

5.4 Health Monitoring

The API server exposes a health endpoint (GET /api/v1/health) that reports:

• Database connectivity
• Redis connectivity
• S3 accessibility
• Runner health status (based on RUNNER_HEALTH_TIMEOUT_SECONDS)

Protect the health endpoint with HEALTH_CHECK_API_KEY in production to prevent information leakage.

5.5 Cron Jobs

The API server runs background cron jobs for:

• Warm pool replenishment: Maintains pre-provisioned sandboxes
• Auto-archive: Archives idle sandboxes past the threshold
• Stale build cleanup: Removes build info older than BUILDINFO_SNAPSHOT_RUNNER_STALENESS_DAYS
• Failed snapshot cleanup: Removes failed snapshot data after FAILED_SNAPSHOT_RUNNER_RETENTION_HOURS

Control cron execution with:

• DISABLE_CRON_JOBS=true to disable all cron jobs on a given instance
• CRON_TIMEZONE to set the scheduling timezone

5.6 Logging

Configure logging behavior with:

LOG_LEVEL=info              # debug, info, warn, error
LOG_CONSOLE_DISABLED=false  # Set true to suppress console output
LOG_REQUESTS_ENABLED=true   # Log HTTP request/response pairs

For structured log aggregation, keep console logging enabled and pipe stdout to your log collector (Fluentd, Filebeat, Vector).

5.7 Audit Trail

Daytona provides a comprehensive audit system with two storage backends:

Kafka mode (AUDIT_PUBLISH_STORAGE_ADAPTER=kafka): Publishes audit events to Kafka topics for downstream consumption.

OpenSearch mode (AUDIT_PUBLISH_STORAGE_ADAPTER=opensearch): Writes audit events directly to an OpenSearch index.

Both modes support batching (AUDIT_PUBLISH_BATCH_SIZE) and can run alongside console audit logging (AUDIT_CONSOLE_LOG_ENABLED).

Audit log retention is controlled by AUDIT_LOG_RETENTION_DAYS. Set this according to your compliance requirements.

Note

When AUDIT_TOOLBOX_REQUESTS_ENABLED=true, all Toolbox API calls from sandboxes are logged. This generates significant volume in active deployments. Enable selectively or ensure your audit storage is sized accordingly.

5.8 Troubleshooting

Sandbox Creation Fails

Runner Connectivity Issues

Database Issues

Known Issues

• go-git fails on small sandboxes (GitHub #4544): Git operations via go-git can fail in resource-constrained sandboxes. Ensure minimum memory allocation is sufficient.
• Docker-in-docker base image fails (GitHub #4461): Certain DinD base images are incompatible. Check the issue for verified compatible images.
• Runner image missing mount-s3 (GitHub #3273): The mount-s3 utility may be missing from runner images, affecting S3 filesystem mounts.
• Toolbox API proxy returns bare 502 (GitHub #4545): Proxy returns 502 without meaningful error body when the Toolbox API is unreachable.
• listFiles trailing slash issue (GitHub #4542): File listing API behaves differently with and without trailing slashes in the path.

6. Performance

6.1 Runner Scoring System

The API server uses a weighted scoring algorithm to distribute sandbox operations across runners. This is the core load-balancing mechanism.

Score Components

Each runner receives a composite score based on three categories of metrics:

Resource utilization (current usage):

Resource allocation (committed but not necessarily used):

Threshold levels (three tiers per metric):

Scoring Thresholds

Three operation-specific thresholds determine whether a runner is eligible:

How Scoring Works

1. The API server queries each runner's reported metrics
2. For each metric, the score starts at 1.0 (optimal) and degrades as usage approaches critical thresholds
3. When usage exceeds a penalty threshold, a penalty function applies: penalty = (usage / threshold) ^ exponent
4. The final score is a weighted sum of all component scores
5. Runners below the relevant threshold for the requested operation are excluded
6. Among eligible runners, the highest-scoring runner receives the job

Tip

Start with conservative thresholds (e.g., optimal CPU at 40%, critical at 85%) and adjust based on observed performance. Aggressive thresholds lead to uneven distribution; too-conservative thresholds leave capacity unused. Monitor the runner scoring to understand how scores distribute across your fleet.

6.2 Connection Pool Tuning

PostgreSQL

The default TypeORM pool settings are conservative. For high-throughput deployments:

DB_POOL_MAX=50
DB_POOL_MIN=10
DB_POOL_IDLE_TIMEOUT_MS=30000
DB_POOL_CONNECTION_TIMEOUT_MS=5000

Ensure PostgreSQL's max_connections accommodates all API replicas: max_connections >= (DB_POOL_MAX * replica_count) + 20 (reserve 20 for admin/monitoring).

Redis

The ioredis client manages its own connection pool. Ensure Redis maxclients is set high enough for all connected API instances plus any other Redis consumers.

6.3 Caching Strategy

Daytona caches two hot paths in Redis:

• API key validation: Cached for API_KEY_VALIDATION_CACHE_TTL_SECONDS. Reduces database round-trips for authenticated requests.
• API key to user mapping: Cached for API_KEY_USER_CACHE_TTL_SECONDS. Avoids repeated user lookups.

For deployments with high request volume, set both TTLs to 60-300 seconds. Shorter TTLs mean faster propagation of key revocations; longer TTLs mean fewer database queries.

6.4 Activity Throttling

Sandbox activity events (heartbeats, file changes) are throttled to prevent overwhelming the database:

• SANDBOX_ACTIVITY_THROTTLE_TTL_SECONDS: Minimum interval between activity events for the same sandbox
• SANDBOX_ACTIVITY_FLUSH_BATCH_SIZE: Number of activity events batched before flushing to the database

6.5 Build Performance

Image builds are resource-intensive. Tune per runner:

• BUILD_CPU_CORES: More cores speed up multi-stage builds
• BUILD_MEMORY_GB: Increase for builds that compile large projects
• BUILD_ENGINE=buildkit: BuildKit is the only supported engine; it provides layer caching and parallel stage execution

The runner shares build resources with running sandboxes. Size runner nodes to accommodate both build workloads and steady-state sandbox operation.

7. Security

7.1 Authentication Architecture

Daytona delegates authentication to an external OIDC provider (Auth0, Dex, Keycloak). The API server validates JWT tokens against the configured issuer.

Authentication flow:

1. Client redirects to OIDC provider (OIDC_ISSUER_BASE_URL)
2. User authenticates with the provider
3. Provider issues a JWT with the configured audience (OIDC_AUDIENCE)
4. Client presents the JWT to the Daytona API
5. API validates the token signature, issuer, audience, and expiration
6. API maps the token subject to a Daytona user account

API key authentication is available as an alternative for programmatic access. API keys are validated against the database (with Redis caching). The admin API key (ADMIN_API_KEY) bypasses normal authentication for administrative operations.

7.2 Authorization Model

Daytona uses an organization-based authorization model with role-based access control.

Organization roles:

Custom role permissions:

Personal organizations are created automatically for each user. They cannot be deleted and do not support member invitations.

7.3 Rate Limiting

The five-tier rate-limiting system protects against abuse:

Rate limit counters are stored in Redis with per-tier TTLs. When a limit is exceeded, the API returns 429 Too Many Requests with a Retry-After header.

Warning

The Failed Auth rate limit is your primary defense against credential-stuffing attacks. Set it conservatively (e.g., 5 attempts per 300 seconds). A lenient failed-auth limit combined with no IP-based blocking makes brute-force attacks feasible.

7.4 Data Encryption

• At rest: Sensitive fields (API keys, credentials, tokens) are encrypted using ENCRYPTION_KEY with ENCRYPTION_SALT for key derivation
• In transit: TLS for all external-facing endpoints (CERT_PATH, CERT_KEY_PATH); optional TLS for database (DB_TLS_ENABLED), Redis (REDIS_TLS), Kafka (KAFKA_TLS_ENABLED), and OpenSearch (OPENSEARCH_TLS_REJECT_UNAUTHORIZED)
• S3: Use HTTPS endpoints and ensure bucket policies enforce encryption

7.5 Network Security

Egress control: ORGANIZATION_SANDBOX_DEFAULT_LIMITED_NETWORK_EGRESS controls whether sandboxes have restricted outbound network access by default. This is critical for multi-tenant deployments where sandbox workloads are untrusted.

Inter-service authentication:

Caution

Every inter-service API key (PROXY_API_KEY, SSH_GATEWAY_API_KEY, DEFAULT_RUNNER_API_KEY, ADMIN_API_KEY) should be unique, high-entropy, and rotated periodically. A compromised runner token grants full sandbox management capability on that runner.

7.6 Organization Suspension

Organizations can be suspended with a recorded reason. Suspended organizations cannot create sandboxes, manage resources, or perform any mutating operations. Only admin-level API access can suspend or unsuspend organizations.

7.7 Invitation Security

Organization invitations follow a strict state machine:

created --> pending --> accepted | declined | cancelled

Invitations are tied to email addresses and validated against OIDC identity. SKIP_USER_EMAIL_VERIFICATION bypasses email verification, which should only be enabled in development environments.

7.8 Secret Management Checklist

8. Developer Internals

8.1 Monorepo Structure

apps/
  api/                 NestJS API server (TypeScript)
  cli/                 Go CLI (Cobra)
  runner/              Go runner service (Gin)
  daemon/              Go daemon (runs inside sandboxes)
  proxy/               Go traffic proxy
  ssh-gateway/         Go SSH gateway
  dashboard/           React dashboard (Vite)
  docs/                Astro documentation site
  snapshot-manager/    Go snapshot manager
  otel-collector/      Custom OpenTelemetry collector

libs/
  api-client/          Auto-generated API clients (Python, TS, Go, Java, Ruby)
  api-client-async/    Async API clients (Python)
  sdk-go/              Go SDK
  sdk-java/            Java SDK
  sdk-ruby/            Ruby SDK
  common-go/           Shared Go utilities
  computer-use/        Computer-use automation library
  runner-proto/        Protobuf definitions for runner communication
  opencode-plugin/     OpenCode plugin integration

The monorepo is managed by Nx 21.4.1 for task orchestration and dependency graph management, with Yarn 4.13.0 for JavaScript/TypeScript dependency management.

8.2 API Server Internals

The API server (apps/api/) is a NestJS 11 application running on Express 5.

Configuration loading (apps/api/src/config/configuration.ts): All environment variables are loaded and validated at startup. The configuration module provides typed access throughout the application via NestJS dependency injection.

Module organization: NestJS modules encapsulate domains (sandboxes, organizations, runners, snapshots, audit, etc.). Each module provides its own controllers, services, and entities.

Database layer: TypeORM entities define the schema. Migrations are generated with TypeORM CLI and follow the Expand and Contract pattern described in Section 4.2.

Job dispatch: When a sandbox operation is requested, the API creates a job record and assigns it to the best-scoring runner. The runner picks up the job on its next poll cycle.

8.3 Runner Internals

The runner (apps/runner/) is a Go service built with the Gin HTTP framework.

Job polling: The runner's main loop polls the API server every POLL_TIMEOUT (default 30s) for up to POLL_LIMIT (default 10) pending jobs. Claimed jobs are executed concurrently.

Build engine: BuildKit is the sole supported build engine (BUILD_ENGINE=buildkit). The runner manages a BuildKit daemon for container image builds, allocating BUILD_CPU_CORES and BUILD_MEMORY_GB to the build process.

Sandbox management: The runner creates, starts, stops, and destroys sandboxes using the container runtime. Each sandbox runs the Daemon process internally for Toolbox API access.

Image operations: The runner handles image pull, build, tag, remove, and inspect operations. Build logs are streamed through the /logs endpoint.

8.4 Daemon (Toolbox API)

The daemon (apps/daemon/) runs inside each sandbox and exposes the Toolbox API. This API provides:

• File system operations (read, write, list, search)
• Process management (execute, signal, list)
• Git operations (clone, pull, push, status)
• Port forwarding management
• Environment variable management

The Toolbox API is accessed via the proxy, which routes requests based on sandbox ID. When AUDIT_TOOLBOX_REQUESTS_ENABLED=true, all Toolbox API calls are logged.

8.5 Proxy Internals

The proxy (apps/proxy/) routes HTTP and WebSocket traffic to sandboxes. It resolves sandbox-specific subdomains using the PROXY_TEMPLATE_URL pattern and forwards traffic to the appropriate runner and sandbox.

The proxy authenticates with the API server using PROXY_API_KEY to resolve sandbox routing information.

8.6 SSH Gateway Internals

The SSH gateway (apps/ssh-gateway/) provides SSH access to running sandboxes. It authenticates users via the API server (using SSH_GATEWAY_API_KEY), maps SSH connections to sandbox instances, and proxies the SSH session to the sandbox's daemon.

8.7 Protobuf Communication

The libs/runner-proto/ package defines Protocol Buffer message types for structured communication between the API server and runners. This provides type-safe serialization for job payloads and status updates.

8.8 Auto-generated API Clients

API clients for all five languages are auto-generated from the OpenAPI specification. The generation process:

1. API server exports its OpenAPI spec
2. Code generators produce client libraries in libs/api-client/ (sync) and libs/api-client-async/ (Python async)
3. Generated clients are versioned alongside the API

Do not manually edit generated client code. Changes must go through the OpenAPI spec and regeneration pipeline.

8.9 Testing

The monorepo uses Nx to orchestrate test execution across projects:

# Run all tests
yarn nx run-many --target=test

# Run tests for a specific project
yarn nx test api

# Run Go tests
yarn nx test runner
yarn nx test cli

8.10 Building

# Build all projects
yarn nx run-many --target=build

# Build specific project
yarn nx build api
yarn nx build dashboard

# Build Go binaries
yarn nx build runner
yarn nx build cli

9. Integration Points

9.1 Webhook Events (Svix)

Daytona dispatches webhook events via Svix for real-time integration with external systems. Webhooks are managed per-organization.

Prerequisites:

• SVIX_AUTH_TOKEN configured
• SVIX_SERVER_URL pointing to a Svix instance (self-hosted or cloud)

Event categories:

Webhook endpoints are registered through the API. Svix handles delivery, retries, and endpoint health monitoring.

9.2 OpenTelemetry

The custom OTel collector (apps/otel-collector/) aggregates telemetry from all components:

Traces: Distributed traces across API server, runners, and proxy. Each span is tagged with ENVIRONMENT for filtering.

Metrics: Resource utilization, job latency, sandbox lifecycle timing, error rates.

Export targets: ClickHouse (primary analytics store), plus any OTLP-compatible backend.

Enable with OTEL_ENABLED=true and point OTEL_COLLECTOR_ENDPOINT_URL to the collector's gRPC endpoint.

9.3 PostHog Analytics

Optional product analytics integration:

POSTHOG_HOST=https://app.posthog.com
POSTHOG_API_KEY=phc_...
POSTHOG_ENVIRONMENT=production

PostHog receives anonymized usage events for product development insights. This is optional and can be disabled by leaving the variables unset.

9.4 Billing Integration

External billing is supported via BILLING_API_URL. The API server communicates with the billing service for:

• Usage metering (CPU-hours, storage, bandwidth)
• Quota enforcement
• Subscription management

9.5 Analytics Integration

External analytics via ANALYTICS_API_URL supplements the built-in ClickHouse analytics for business intelligence dashboards and reporting.

9.6 Pylon Integration

PYLON_APP_ID enables Pylon integration for customer support workflows, connecting sandbox issues to support tickets.

9.7 SDK Integration Patterns

Python SDK:

from daytona import Daytona

daytona = Daytona(api_key="your-api-key", api_url="https://api.daytona.example.com")

# Create a sandbox
sandbox = daytona.create()

# Execute a command
response = sandbox.process.exec("echo 'Hello from sandbox'")
print(response.output)

# Clean up
sandbox.delete()

TypeScript SDK:

import { Daytona } from '@daytona/sdk';

const daytona = new Daytona({
  apiKey: 'your-api-key',
  apiUrl: 'https://api.daytona.example.com',
});

const sandbox = await daytona.create();
const result = await sandbox.process.exec('echo "Hello from sandbox"');
console.log(result.output);
await sandbox.delete();

Go SDK:

package main

import (
    "fmt"
    daytona "github.com/daytonaio/daytona/libs/sdk-go"
)

func main() {
    client := daytona.NewClient("your-api-key", "https://api.daytona.example.com")
    sandbox, _ := client.Create()
    result, _ := sandbox.Process.Exec("echo 'Hello from sandbox'")
    fmt.Println(result.Output)
    sandbox.Delete()
}

Tip

The Python SDK provides both synchronous and asynchronous clients. For high-throughput automation, use the async client from libs/api-client-async/ to avoid blocking on sandbox operations.

9.8 Docker Registry Types

Daytona manages four types of container registries:

9.9 Multi-Region Architecture

Daytona supports multi-region deployments where runners in different regions serve sandboxes with region affinity:

• DEFAULT_REGION_ID and DEFAULT_REGION_NAME define the primary region
• DEFAULT_REGION_ENFORCE_QUOTAS controls whether resource quotas are enforced per-region
• Additional regions are created via the API and assigned runners

Region-aware scheduling routes sandbox creation to runners in the requested region, falling back to other regions based on runner scoring.

Important

In multi-region deployments, ensure S3 storage is either replicated across regions or uses a globally accessible endpoint. Sandbox snapshots stored in one region must be accessible from runners in other regions for cross-region restore operations.