Deployment

Docker Compose configurations to run the Arcane stack locally with different trace backends and message brokers.

Prerequisites

Docker and Docker Compose
Images are pre-built and published to a Docker registry; Compose pulls them automatically

Choose a Compose

Variant	Trace Backend	Message Broker	Compose File
Base	None	RabbitMQ	`base/docker-compose.base.yml`
Tempo	Tempo	RabbitMQ	`tempo/docker-compose.tempo-rabbitmq.yml`
Tempo	Tempo	Kafka	`tempo/docker-compose.tempo-kafka.yml`
Jaeger	Jaeger	RabbitMQ	`jaeger/docker-compose.jaeger-rabbitmq.yml`
Jaeger	Jaeger	Kafka	`jaeger/docker-compose.jaeger-kafka.yml`
ClickHouse	ClickHouse	RabbitMQ	`clickhouse/docker-compose.clickhouse-rabbitmq.yml`
ClickHouse	ClickHouse	Kafka	`clickhouse/docker-compose.clickhouse-kafka.yml`

Base: Minimal run (postgres, rabbitmq, frontend, backend, worker). No trace backend.
Trace variants: Add Grafana Tempo, Jaeger, or ClickHouse for trace storage and querying.

Quick Start

Step 1: Clone the repository

git clone git@github.com:ArcaneTracing/arcane-deployment.git arcane-deployment
cd arcane-deployment

Verify: You should see directories base/, tempo/, jaeger/, clickhouse/, and a file .env.example.

Step 2: Copy and configure environment

cp .env.example .env
# Edit .env as needed for production, or use defaults for local dev

Verify the copy worked:

ls -la .env
# Should show .env exists
cat .env | head -5
# Should show your variables (or defaults)

Step 2b: Generate and set auth & security secrets (required for production)

These four variables are critical — never use defaults in production. Generate secure values and add them to .env:

1. Generate values:

# BETTER_AUTH_SECRET, INTERNAL_API_KEY, API_KEY_SALT (base64, 32 bytes)
openssl rand -base64 32

# ENCRYPTION_KEY (64 hex chars) — run this separately
openssl rand -hex 32

2. Edit .env and replace the defaults with your generated values:

# Auth & Security (required in production)
BETTER_AUTH_SECRET='<paste output of: openssl rand -base64 32>'
INTERNAL_API_KEY='<paste output of: openssl rand -base64 32>'
API_KEY_SALT='<paste output of: openssl rand -base64 32>'

# Generate with: openssl rand -hex 32 (produces 64 chars)
ENCRYPTION_KEY=<paste output of: openssl rand -hex 32>

Example (do not use these — generate your own):

BETTER_AUTH_SECRET='mTRokjXWBfabfjF5P5+welH+JXp1QhDCGmmZlyJKzlk='
INTERNAL_API_KEY='AEQcfdFdn8yXGVzis09qwBOkTtVu7Kp83Rcps085fss='
API_KEY_SALT='ndKTQciT/TaZmOupPJ0oTO0EaiYozFxPLI909nHZr+g='

# Generate with: openssl rand -hex 32
ENCRYPTION_KEY=61d24cf9bcbe65d0d114b1341614f164f41f17f12c3154e604616ad21201b5f0

Verify: grep -E "BETTER_AUTH_SECRET|INTERNAL_API_KEY|API_KEY_SALT|ENCRYPTION_KEY" .env — all four should show your values, not the dev- defaults.

Step 3: Load environment variables

Compose uses variable substitution (e.g. ${DATABASE_URL:-default}). Load .env into your shell before running compose:

set -a && source .env && set +a

Verify: Run echo $DATABASE_URL — you should see the connection string (or empty if using default).

Step 4: Run the compose

From the project root, run the compose file for your chosen variant:

# Base (minimal, no trace backend)
docker compose -f base/docker-compose.base.yml up -d

# Example: Tempo + RabbitMQ
docker compose -f tempo/docker-compose.tempo-rabbitmq.yml up -d

# Example: ClickHouse + Kafka
docker compose -f clickhouse/docker-compose.clickhouse-kafka.yml up -d

Verify: Containers should start. Check with docker compose -f <your-compose-file> ps — all services should show "running".

Step 5: Add a datasource in Arcane

Once the stack is running, open the Arcane frontend and add a datasource. The backend runs in a container, so use these URLs when configuring the datasource:

Trace Backend	Datasource URL (use in Arcane)
Tempo	`http://tempo:3200` or `http://localhost:3200`
Jaeger	`http://jaeger:16686`
ClickHouse	`http://clickhouse:8123`

Tempo compose files add extra_hosts so the backend can reach host-exposed ports via localhost. For Jaeger/ClickHouse, use the service name (e.g. http://jaeger:16686).

Access URLs

Service	URL
Frontend	http://localhost:3000
Backend API	http://localhost:8085
Jaeger UI	http://localhost:16686 (Jaeger variants only)
RabbitMQ Management	http://localhost:15672 (RabbitMQ variants)
Tempo	http://localhost:3200 (Tempo variants)
ClickHouse	http://localhost:8123 (ClickHouse variants)
Kafka	localhost:9092 (Kafka variants)

Environment Variables

All compose files use environment variables. Set variables in .env and load them into your shell before running compose (e.g. set -a && source .env && set +a) so variable substitution works.

Database

Variable	Description	Default
`DATABASE_URL`	PostgreSQL connection string	`postgresql://user:password@postgres:5432/arcanedb`
`DATABASE_SSL`	Enable SSL for DB	`false`
`POSTGRES_USER`	Postgres container user	`user`
`POSTGRES_PASSWORD`	Postgres container password	`password`
`POSTGRES_DB`	Postgres database name	`arcanedb`

Message Broker

RabbitMQ variants:

Variable	Description	Default
`RABBITMQ_URL`	RabbitMQ connection URL	`amqp://guest:guest@rabbitmq:5672/`

Kafka variants:

Variable	Description	Default
`KAFKA_BROKERS`	Comma-separated broker list	`kafka:9092`

Worker–Backend Topics (evaluation & experiment queues)

These topics/queues handle communication between the worker and backend for evaluations and experiments. Set them if you need custom topic names; otherwise defaults from the compose apply.

Variable	Description
`EVALUATION_JOBS_TOPIC`	Topic for evaluation job requests (backend → worker)
`EVALUATION_RESULTS_TOPIC`	Topic for evaluation results (worker → backend)
`EXPERIMENT_JOBS_TOPIC`	Topic for experiment job requests (backend → worker)
`EXPERIMENT_RESULTS_TOPIC`	Topic for experiment results (worker → backend)

Auth & Security (required in production)

Variable	Description	Default
`BETTER_AUTH_SECRET`	Auth token encryption	`dev-secret-change-in-production`
`INTERNAL_API_KEY`	Internal API signing key	`dev-jwt-secret`
`ENCRYPTION_KEY`	64+ char key for model config encryption	`dev-encryption-key-64-chars-minimum-required-for-production-safe`
`API_KEY_SALT`	Salt for API key hashing	`dev-api-key-salt`

Generate secure values: openssl rand -base64 32

For ENCRYPTION_KEY (64 chars): openssl rand -hex 32

URLs

Variable	Description	Default
`BETTER_AUTH_URL`	Backend URL (auth callbacks)	`http://localhost:8085` or `http://backend:8085`
`FRONTEND_URL`	Frontend URL (CORS, redirects)	`http://localhost:3000` or `http://frontend:3000`

SMTP (optional, for invite emails)

Variable	Description	Default
`SMTP_HOST`	SMTP server host	(empty)
`SMTP_PORT`	SMTP port	`587`
`SMTP_SECURE`	Use TLS	`false`
`SMTP_USER`	SMTP username	(empty)
`SMTP_PASS`	SMTP password	(empty)
`MAIL_FROM`	From address	`no-reply@example.com`

Optional

Variable	Description
`ARCANE_ENTERPRISE_LICENSE`	Enterprise license key (base64)

Directory Layout

arcane-deployment/
├── config/          # OTEL and Tempo configs
├── base/            # Base compose (minimal, no trace backend)
├── tempo/           # Tempo + Kafka/RabbitMQ
├── jaeger/          # Jaeger + Kafka/RabbitMQ
├── clickhouse/      # ClickHouse + Kafka/RabbitMQ
├── .env.example     # Example env file
└── README.md

SSO Setup with Okta — Configure Okta SAML SSO (enterprise)

Prerequisites​

Choose a Compose​

Quick Start​

Step 1: Clone the repository​

Step 2: Copy and configure environment​

Step 2b: Generate and set auth & security secrets (required for production)​

Step 3: Load environment variables​

Step 4: Run the compose​

Step 5: Add a datasource in Arcane​

Access URLs​

Environment Variables​

Database​

Message Broker​

Worker–Backend Topics (evaluation & experiment queues)​

Auth & Security (required in production)​

URLs​

SMTP (optional, for invite emails)​

Optional​

Directory Layout​

Related​