wrenn-releases/CLAUDE.md

CLAUDE.md

This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.

Project Overview

Wrenn Sandbox is a microVM-based code execution platform. Users create isolated sandboxes (Firecracker microVMs), run code inside them, and get output back via SDKs. Think E2B but with persistent sandboxes, pool-based pricing, and a single-binary deployment story.

Build & Development Commands

All commands go through the Makefile. Never use raw go build or go run.

make build              # Build all binaries → builds/
make build-cp           # Control plane only (builds frontend first)
make build-agent        # Host agent only
make build-envd         # envd static binary (verified statically linked)
make build-frontend     # SvelteKit dashboard → internal/dashboard/static/

make dev                # Full local dev: infra + migrate + control plane
make dev-infra          # Start PostgreSQL + Prometheus + Grafana (Docker)
make dev-down           # Stop dev infra
make dev-cp             # Control plane with hot reload (if air installed)
make dev-frontend       # Vite dev server with HMR (port 5173)
make dev-agent          # Host agent (sudo required)
make dev-envd           # envd in TCP debug mode

make check              # fmt + vet + lint + test (CI order)
make test               # Unit tests: go test -race -v ./internal/...
make test-integration   # Integration tests (require host agent + Firecracker)
make fmt                # gofmt both modules
make vet                # go vet both modules
make lint               # golangci-lint

make migrate-up         # Apply pending migrations
make migrate-down       # Rollback last migration
make migrate-create name=xxx  # Scaffold new goose migration (never create manually)
make migrate-reset      # Drop + re-apply all

make generate           # Proto (buf) + sqlc codegen
make proto              # buf generate for all proto dirs
make tidy               # go mod tidy both modules

Run a single test: go test -race -v -run TestName ./internal/path/...

Architecture

User SDK → HTTPS/WS → Control Plane → Connect RPC → Host Agent → HTTP/Connect RPC over TAP → envd (inside VM)

Three binaries, two Go modules:

Binary	Module	Entry point	Runs as
wrenn-cp	git.omukk.dev/wrenn/wrenn	cmd/control-plane/main.go	Unprivileged
wrenn-agent	git.omukk.dev/wrenn/wrenn	cmd/host-agent/main.go	Root (NET_ADMIN + /dev/kvm)
envd	git.omukk.dev/wrenn/wrenn/envd (standalone envd/go.mod)	envd/main.go	PID 1 inside guest VM

envd is a completely independent Go module. It is never imported by the main module. The only connection is the protobuf contract. It compiles to a static binary baked into rootfs images.

Key architectural invariant: The host agent is stateful (in-memory boxes map is the source of truth for running VMs). The control plane is stateless (all persistent state in PostgreSQL). The reconciler (internal/api/reconciler.go) bridges the gap — it periodically compares DB records against the host agent's live state and marks orphaned sandboxes as "stopped".

Control Plane

Packages: internal/api/, internal/dashboard/, internal/auth/, internal/scheduler/, internal/lifecycle/, internal/config/, internal/db/

Startup (cmd/control-plane/main.go) wires: config (env vars) → pgxpool → db.Queries (sqlc-generated) → Connect RPC client to host agent → api.Server. Everything flows through constructor injection.

• API Server (internal/api/server.go): chi router with middleware. Creates handler structs (sandboxHandler, execHandler, filesHandler, etc.) injected with db.Queries and the host agent Connect RPC client. Routes under /v1/capsules/*.
• Reconciler (internal/api/reconciler.go): background goroutine (every 30s) that compares DB records against agent.ListSandboxes() RPC. Marks orphaned DB entries as "stopped".
• Dashboard (SvelteKit + Tailwind + Bits UI, statically built and embedded via go:embed, served as catch-all at root)
• Database: PostgreSQL via pgx/v5. Queries generated by sqlc from db/queries/sandboxes.sql. Migrations in db/migrations/ (goose, plain SQL).
• Config (internal/config/config.go): purely environment variables (DATABASE_URL, CP_LISTEN_ADDR, CP_HOST_AGENT_ADDR), no YAML/file config.

Host Agent

Packages: internal/hostagent/, internal/sandbox/, internal/vm/, internal/network/, internal/devicemapper/, internal/envdclient/, internal/snapshot/

Startup (cmd/host-agent/main.go) wires: root check → enable IP forwarding → clean up stale dm devices → sandbox.Manager (containing vm.Manager + network.SlotAllocator + devicemapper.LoopRegistry) → hostagent.Server (Connect RPC handler) → HTTP server.

• RPC Server (internal/hostagent/server.go): implements hostagentv1connect.HostAgentServiceHandler. Thin wrapper — every method delegates to sandbox.Manager. Maps Connect error codes on return.
• Sandbox Manager (internal/sandbox/manager.go): the core orchestration layer. Maintains in-memory state in boxes map[string]*sandboxState (protected by sync.RWMutex). Each sandboxState holds a models.Sandbox, a *network.Slot, and an *envdclient.Client. Runs a TTL reaper (every 10s) that auto-destroys timed-out sandboxes.
• VM Manager (internal/vm/manager.go, fc.go, config.go): manages Firecracker processes. Uses raw HTTP API over Unix socket (/tmp/fc-{sandboxID}.sock), not the firecracker-go-sdk Machine type. Launches Firecracker via unshare -m + ip netns exec. Configures VM via PUT to /boot-source, /drives/rootfs, /network-interfaces/eth0, /machine-config, then starts with PUT /actions.
• Network (internal/network/setup.go, allocator.go): per-sandbox network namespace with veth pair + TAP device. See Networking section below.
• Device Mapper (internal/devicemapper/devicemapper.go): CoW rootfs via device-mapper snapshots. Shared read-only loop devices per base template (refcounted LoopRegistry), per-sandbox sparse CoW files, dm-snapshot create/restore/remove/flatten operations.
• envd Client (internal/envdclient/client.go, health.go): dual interface to the guest agent. Connect RPC for streaming process exec (process.Start() bidirectional stream). Plain HTTP for file operations (POST/GET /files?path=...&username=root). Health check polls GET /health every 100ms until ready (30s timeout).

envd (Guest Agent)

Module: envd/ with its own go.mod (git.omukk.dev/wrenn/wrenn/envd)

Runs as PID 1 inside the microVM via wrenn-init.sh (mounts procfs/sysfs/dev, sets hostname, writes resolv.conf, then execs envd). Extracted from E2B (Apache 2.0), with shared packages internalized into envd/internal/shared/. Listens on TCP 0.0.0.0:49983.

• ProcessService: start processes, stream stdout/stderr, signal handling, PTY support
• FilesystemService: stat/list/mkdir/move/remove/watch files
• Health: GET /health

Dashboard (Frontend)

Directory: frontend/ — standalone SvelteKit app (Svelte 5, runes mode)

• Stack: SvelteKit + adapter-static + Tailwind CSS v4 + Bits UI (headless accessible components)
• Package manager: pnpm
• Routing: SvelteKit file-based routing under frontend/src/routes/
• Routing layout: /login and /signup at root, authenticated pages under /dashboard/* (e.g. /dashboard/capsules, /dashboard/keys)
• Build output: frontend/build/ → copied to internal/dashboard/static/ → embedded via go:embed into the control plane binary
• Serving: internal/dashboard/dashboard.go registers a NotFound catch-all SPA handler with fallback to index.html. API routes (/v1/*, /openapi.yaml, /docs) are registered first and take priority
• Dev workflow: make dev-frontend runs Vite dev server on port 5173 with HMR. API calls proxy to http://localhost:8000
• Fonts: Manrope (UI), Instrument Serif (headings), JetBrains Mono (code), Alice (brand wordmark) — all self-hosted via @fontsource
• Dark mode: class-based (.dark on <html>) with system preference detection + localStorage persistence

To add a new page: create frontend/src/routes/your-page/+page.svelte.

Networking (per sandbox)

Each sandbox gets its own Linux network namespace (ns-{idx}). Slot index (1-based, up to 65534) determines all addressing:

Host Namespace                      Namespace "ns-{idx}"                   Guest VM
──────────────────────────────────────────────────────────────────────────────────────
veth-{idx}  ←──── veth pair ────→  eth0
10.12.0.{idx*2}/31                 10.12.0.{idx*2+1}/31
                                     │
                                   tap0 (169.254.0.22/30) ←── TAP ──→ eth0 (169.254.0.21)
                                                                          ↑ kernel ip= boot arg

• Host-reachable IP: 10.11.0.{idx}/32 — routed through veth to namespace, DNAT'd to guest
• Outbound NAT: guest (169.254.0.21) → SNAT to vpeerIP inside namespace → MASQUERADE on host to default interface
• Inbound NAT: host traffic to 10.11.0.{idx} → DNAT to 169.254.0.21 inside namespace
• IP forwarding enabled inside each namespace
• All details in internal/network/setup.go

Sandbox State Machine

PENDING → STARTING → RUNNING → PAUSED → HIBERNATED
                       │          │
                       ↓          ↓
                    STOPPED    STOPPED → (destroyed)

Any state → ERROR (on crash/failure)
PAUSED → RUNNING (warm snapshot resume)
HIBERNATED → RUNNING (cold snapshot resume, slower)

Key Request Flows

Sandbox creation (POST /v1/capsules):

1. API handler generates sandbox ID, inserts into DB as "pending"
2. RPC CreateSandbox → host agent → sandbox.Manager.Create()
3. Manager: resolve base rootfs → acquire shared loop device → create dm-snapshot (sparse CoW file) → allocate network slot → CreateNetwork() (netns + veth + tap + NAT) → vm.Create() (start Firecracker with /dev/mapper/wrenn-{id}, configure via HTTP API, boot) → envdclient.WaitUntilReady() (poll /health) → store in-memory state
4. API handler updates DB to "running" with host_ip

Command execution (POST /v1/capsules/{id}/exec):

1. API handler verifies sandbox is "running" in DB
2. RPC Exec → host agent → sandbox.Manager.Exec() → envdclient.Exec()
3. envd client opens bidirectional Connect RPC stream (process.Start), collects stdout/stderr/exit_code
4. API handler checks UTF-8 validity (base64-encodes if binary), updates last_active_at, returns result

Streaming exec (WS /v1/capsules/{id}/exec/stream):

1. WebSocket upgrade, read first message for cmd/args
2. RPC ExecStream → host agent → sandbox.Manager.ExecStream() → envdclient.ExecStream()
3. envd client returns a channel of events; host agent forwards events through the RPC stream
4. API handler forwards stream events to WebSocket as JSON messages ({type: "stdout"|"stderr"|"exit", ...})

File transfer: Write uses multipart POST to envd /files; read uses GET. Streaming variants chunk in 64KB pieces through the RPC stream.

REST API

Routes defined in internal/api/server.go, handlers in internal/api/handlers_*.go. OpenAPI spec embedded via //go:embed and served at /openapi.yaml (Swagger UI at /docs). JSON request/response. API key auth via X-API-Key header. Error responses: {"error": {"code": "...", "message": "..."}}.

Code Generation

Proto (Connect RPC)

Proto source of truth is proto/envd/*.proto and proto/hostagent/*.proto. Run make proto to regenerate. Three buf.gen.yaml files control output:

buf.gen.yaml location	Generates to	Used by
proto/envd/buf.gen.yaml	proto/envd/gen/	Main module (host agent's envd client)
proto/hostagent/buf.gen.yaml	proto/hostagent/gen/	Main module (control plane ↔ host agent)
envd/spec/buf.gen.yaml	envd/internal/services/spec/	envd module (guest agent server)

The envd buf.gen.yaml reads from ../../proto/envd/ (same source protos) but generates into envd's own module. This means the same .proto files produce two independent sets of Go stubs — one for each Go module.

To add a new RPC method: edit the .proto file → make proto → implement the handler on both sides.

sqlc

Config: sqlc.yaml (project root). Reads queries from db/queries/*.sql, reads schema from db/migrations/, outputs to internal/db/.

To add a new query: add it to the appropriate .sql file in db/queries/ → make generate → use the new method on *db.Queries.

Key Technical Decisions

• Connect RPC (not gRPC) for all RPC communication between components
• Buf + protoc-gen-connect-go for code generation (not protoc-gen-go-grpc)
• Raw Firecracker HTTP API via Unix socket (not firecracker-go-sdk Machine type)
• TAP networking (not vsock) for host-to-envd communication
• Device-mapper snapshots for rootfs CoW — shared read-only loop device per base template, per-sandbox sparse CoW file, Firecracker gets /dev/mapper/wrenn-{id}
• PostgreSQL via pgx/v5 + sqlc (type-safe query generation). Goose for migrations (plain SQL, up/down)
• Dashboard: SvelteKit (Svelte 5, adapter-static) + Tailwind CSS v4 + Bits UI. Built to static files, embedded into the Go binary via go:embed, served as catch-all at root
• Lago for billing (external service, not in this codebase)

Coding Conventions

• Go style: gofmt, go vet, context.Context everywhere, errors wrapped with fmt.Errorf("action: %w", err), slog for logging, no global state
• Naming: Sandbox IDs sb- + 8 hex, API keys wrn_ + 32 chars, Host IDs host- + 8 hex
• Dependencies: Use go get to add deps, never hand-edit go.mod. For envd deps: cd envd && go get ... (separate module)
• Generated code: Always commit generated code (proto stubs, sqlc). Never add generated code to .gitignore
• Migrations: Always use make migrate-create name=xxx, never create migration files manually
• Testing: Table-driven tests for handlers and state machine transitions

Two-module gotcha

The main module (go.mod) and envd (envd/go.mod) are fully independent. make tidy, make fmt, make vet already operate on both. But when adding dependencies manually, remember to target the correct module (cd envd && go get ... for envd deps). make proto also generates stubs for both modules from the same proto sources.

Rootfs & Guest Init

• wrenn-init (images/wrenn-init.sh): the PID 1 init script baked into every rootfs. Mounts virtual filesystems, sets hostname, writes /etc/resolv.conf, then execs envd.
• Updating the rootfs after changing envd or wrenn-init: bash scripts/update-debug-rootfs.sh [rootfs_path]. This builds envd via make build-envd, mounts the rootfs image, copies in the new binaries, and unmounts. Defaults to /var/lib/wrenn/images/minimal.ext4.
• Rootfs images are minimal debootstrap — no systemd, no coreutils beyond busybox. Use /bin/sh -c for shell builtins inside the guest.

Fixed Paths (on host machine)

• Kernel: /var/lib/wrenn/kernels/vmlinux
• Base rootfs images: /var/lib/wrenn/images/{template}.ext4
• Sandbox clones: /var/lib/wrenn/sandboxes/
• Firecracker: /usr/local/bin/firecracker (e2b's fork of firecracker)

Design Context

Users

Developers across the full spectrum — solo engineers building side projects, startup teams integrating sandboxed execution into products, and platform/infra engineers at larger organizations. The interface must feel at home for all three: approachable enough not to intimidate a hacker, precise enough to earn the trust of a production ops team. Never condescend, never oversimplify. Trust the user to understand what they're looking at.

Brand Personality

Precise. Warm. Uncompromising.

Wrenn is an engineer's favorite tool — built with visible care, not assembled from defaults. It runs real infrastructure (Firecracker microVMs), so the UI should reflect that seriousness without becoming cold or corporate. The warmth comes from the typography and color palette; the precision comes from hierarchy, density, and data fidelity.

Emotional goal: in control. Users leave a session with full confidence in what's running, what happened, and what comes next. Nothing is hidden, nothing is ambiguous.

Aesthetic Direction

Dark-first, industrial-warm, data-forward.

The near-black-green background palette (#0a0c0b through #2a302d) reads as "black with intention" — not pitch black (cold) and not charcoal (dated). The sage green accent (#5e8c58) is muted and organic, a meaningful departure from the startup-green neon that saturates the developer tool space.

Anti-references:

• Supabase: avoid the friendly, approachable startup-green energy — too generic, too eager to please
• AWS / GCP consoles: avoid utility-first density without craft — functional but joyless, visually dated

References that capture the right spirit:

• The precision of a well-calibrated instrument
• Editorial typography from technical publications
• The quiet confidence of tools that don't need to explain themselves

Type System

Four fonts with strict roles — this is the design system's strongest personality trait and must be respected:

Font	Role	When to use
Manrope (variable, sans)	UI workhorse	All body copy, nav, labels, buttons, form text
Instrument Serif	Display / editorial	Page titles (h1), dialog headings, metric values, hero moments
JetBrains Mono (variable)	Data / code	IDs, timestamps, key prefixes, file paths, terminal output, metrics
Alice	Brand wordmark	"Wrenn" in sidebar and login only — nowhere else

Instrument Serif at scale creates the signature editorial moments. Mono provides the precision signal for technical data. Never swap these roles.

Color System

Backgrounds: bg-0 (#0a0c0b) through bg-5 (#2a302d) — 6 steps
Text:        bright > primary > secondary > tertiary > muted — 5 levels
Accent:      accent (#5e8c58) / accent-mid / accent-bright / glow / glow-mid
Status:      amber (#d4a73c) / red (#cf8172) / blue (#5a9fd4)

Use accent sparingly. It should feel earned — reserved for live/active state indicators, primary CTAs, focus rings, and active nav. When accent appears, it should register.

Upcoming Surfaces (design must accommodate)

• Terminal / shell output: streaming exec output, TTY sessions. Needs strong mono treatment, high contrast for long sessions.
• File browser: filesystem tree inside capsule. Density matters — breadcrumbs, file icons, permission bits.
• SDK / docs embedding: code samples, quickstart flows inline in dashboard. Code blocks must feel premium, not afterthought.
• Billing / usage charts: pool consumption, cost curves, usage over time. Instrument Serif at large scale for metrics; chart containers should feel like instruments, not dashboards.

Design Principles

1. Precision over friendliness. Every element earns its place. Wrenn doesn't need to tell you it's developer-friendly — that should be self-evident from the quality of the information architecture.

2. Density with breathing room. Data-forward doesn't mean cramped. Strategic whitespace creates calm hierarchy within dense contexts. Sections breathe; rows don't waste space.

3. Industrial warmth. The serif + mono + warm-black combination prevents sterility. This is a forge, not a gallery. The warmth is in the details, not the primary colors.

4. Legible at speed. Users scan dashboards in seconds. Strong typographic contrast (serif h1, mono IDs, sans body), consistent patterns, and predictable placement let users orientate instantly without reading everything.

5. Craft signals trust. For infrastructure that runs production code, the quality of the UI is a proxy for the quality of the product. Pixel-level decisions matter. Polish is not decoration — it's a trust signal.