wrenn-releases/CLAUDE.md

CLAUDE.md

This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.

Project Overview

Wrenn Sandbox is a microVM-based code execution platform. Users create isolated sandboxes (Firecracker microVMs), run code inside them, and get output back via SDKs. Think E2B but with persistent sandboxes, pool-based pricing, and a single-binary deployment story.

Build & Development Commands

All commands go through the Makefile. Never use raw go build or go run.

make build              # Build all binaries → builds/
make build-cp           # Control plane only (builds frontend first)
make build-agent        # Host agent only
make build-envd         # envd static binary (verified statically linked)
make build-frontend     # SvelteKit dashboard → internal/dashboard/static/

make dev                # Full local dev: infra + migrate + control plane
make dev-infra          # Start PostgreSQL + Prometheus + Grafana (Docker)
make dev-down           # Stop dev infra
make dev-cp             # Control plane with hot reload (if air installed)
make dev-frontend       # Vite dev server with HMR (port 5173)
make dev-agent          # Host agent (sudo required)
make dev-envd           # envd in TCP debug mode

make check              # fmt + vet + lint + test (CI order)
make test               # Unit tests: go test -race -v ./internal/...
make test-integration   # Integration tests (require host agent + Firecracker)
make fmt                # gofmt both modules
make vet                # go vet both modules
make lint               # golangci-lint

make migrate-up         # Apply pending migrations
make migrate-down       # Rollback last migration
make migrate-create name=xxx  # Scaffold new goose migration (never create manually)
make migrate-reset      # Drop + re-apply all

make generate           # Proto (buf) + sqlc codegen
make proto              # buf generate for all proto dirs
make tidy               # go mod tidy both modules

Run a single test: go test -race -v -run TestName ./internal/path/...

Architecture

User SDK → HTTPS/WS → Control Plane → Connect RPC → Host Agent → HTTP/Connect RPC over TAP → envd (inside VM)

Three binaries, two Go modules:

Binary	Module	Entry point	Runs as
wrenn-cp	git.omukk.dev/wrenn/sandbox	cmd/control-plane/main.go	Unprivileged
wrenn-agent	git.omukk.dev/wrenn/sandbox	cmd/host-agent/main.go	Root (NET_ADMIN + /dev/kvm)
envd	git.omukk.dev/wrenn/sandbox/envd (standalone envd/go.mod)	envd/main.go	PID 1 inside guest VM

envd is a completely independent Go module. It is never imported by the main module. The only connection is the protobuf contract. It compiles to a static binary baked into rootfs images.

Key architectural invariant: The host agent is stateful (in-memory boxes map is the source of truth for running VMs). The control plane is stateless (all persistent state in PostgreSQL). The reconciler (internal/api/reconciler.go) bridges the gap — it periodically compares DB records against the host agent's live state and marks orphaned sandboxes as "stopped".

Control Plane

Packages: internal/api/, internal/dashboard/, internal/auth/, internal/scheduler/, internal/lifecycle/, internal/config/, internal/db/

Startup (cmd/control-plane/main.go) wires: config (env vars) → pgxpool → db.Queries (sqlc-generated) → Connect RPC client to host agent → api.Server. Everything flows through constructor injection.

• API Server (internal/api/server.go): chi router with middleware. Creates handler structs (sandboxHandler, execHandler, filesHandler, etc.) injected with db.Queries and the host agent Connect RPC client. Routes under /v1/sandboxes/*.
• Reconciler (internal/api/reconciler.go): background goroutine (every 30s) that compares DB records against agent.ListSandboxes() RPC. Marks orphaned DB entries as "stopped".
• Dashboard (SvelteKit + Tailwind + Bits UI, statically built and embedded via go:embed, served as catch-all at root)
• Database: PostgreSQL via pgx/v5. Queries generated by sqlc from db/queries/sandboxes.sql. Migrations in db/migrations/ (goose, plain SQL).
• Config (internal/config/config.go): purely environment variables (DATABASE_URL, CP_LISTEN_ADDR, CP_HOST_AGENT_ADDR), no YAML/file config.

Host Agent

Packages: internal/hostagent/, internal/sandbox/, internal/vm/, internal/network/, internal/devicemapper/, internal/envdclient/, internal/snapshot/

Startup (cmd/host-agent/main.go) wires: root check → enable IP forwarding → clean up stale dm devices → sandbox.Manager (containing vm.Manager + network.SlotAllocator + devicemapper.LoopRegistry) → hostagent.Server (Connect RPC handler) → HTTP server.

• RPC Server (internal/hostagent/server.go): implements hostagentv1connect.HostAgentServiceHandler. Thin wrapper — every method delegates to sandbox.Manager. Maps Connect error codes on return.
• Sandbox Manager (internal/sandbox/manager.go): the core orchestration layer. Maintains in-memory state in boxes map[string]*sandboxState (protected by sync.RWMutex). Each sandboxState holds a models.Sandbox, a *network.Slot, and an *envdclient.Client. Runs a TTL reaper (every 10s) that auto-destroys timed-out sandboxes.
• VM Manager (internal/vm/manager.go, fc.go, config.go): manages Firecracker processes. Uses raw HTTP API over Unix socket (/tmp/fc-{sandboxID}.sock), not the firecracker-go-sdk Machine type. Launches Firecracker via unshare -m + ip netns exec. Configures VM via PUT to /boot-source, /drives/rootfs, /network-interfaces/eth0, /machine-config, then starts with PUT /actions.
• Network (internal/network/setup.go, allocator.go): per-sandbox network namespace with veth pair + TAP device. See Networking section below.
• Device Mapper (internal/devicemapper/devicemapper.go): CoW rootfs via device-mapper snapshots. Shared read-only loop devices per base template (refcounted LoopRegistry), per-sandbox sparse CoW files, dm-snapshot create/restore/remove/flatten operations.
• envd Client (internal/envdclient/client.go, health.go): dual interface to the guest agent. Connect RPC for streaming process exec (process.Start() bidirectional stream). Plain HTTP for file operations (POST/GET /files?path=...&username=root). Health check polls GET /health every 100ms until ready (30s timeout).

envd (Guest Agent)

Module: envd/ with its own go.mod (git.omukk.dev/wrenn/sandbox/envd)

Runs as PID 1 inside the microVM via wrenn-init.sh (mounts procfs/sysfs/dev, sets hostname, writes resolv.conf, then execs envd). Extracted from E2B (Apache 2.0), with shared packages internalized into envd/internal/shared/. Listens on TCP 0.0.0.0:49983.

• ProcessService: start processes, stream stdout/stderr, signal handling, PTY support
• FilesystemService: stat/list/mkdir/move/remove/watch files
• Health: GET /health

Dashboard (Frontend)

Directory: frontend/ — standalone SvelteKit app (Svelte 5, runes mode)

• Stack: SvelteKit + adapter-static + Tailwind CSS v4 + Bits UI (headless accessible components)
• Package manager: pnpm
• Routing: SvelteKit file-based routing under frontend/src/routes/
• Routing layout: /login and /signup at root, authenticated pages under /dashboard/* (e.g. /dashboard/capsules, /dashboard/keys)
• Build output: frontend/build/ → copied to internal/dashboard/static/ → embedded via go:embed into the control plane binary
• Serving: internal/dashboard/dashboard.go registers a NotFound catch-all SPA handler with fallback to index.html. API routes (/v1/*, /openapi.yaml, /docs) are registered first and take priority
• Dev workflow: make dev-frontend runs Vite dev server on port 5173 with HMR. API calls proxy to http://localhost:8000
• Fonts: Manrope (UI), Instrument Serif (headings), JetBrains Mono (code), Alice (brand wordmark) — all self-hosted via @fontsource
• Dark mode: class-based (.dark on <html>) with system preference detection + localStorage persistence

To add a new page: create frontend/src/routes/your-page/+page.svelte.

Networking (per sandbox)

Each sandbox gets its own Linux network namespace (ns-{idx}). Slot index (1-based, up to 65534) determines all addressing:

Host Namespace                      Namespace "ns-{idx}"                   Guest VM
──────────────────────────────────────────────────────────────────────────────────────
veth-{idx}  ←──── veth pair ────→  eth0
10.12.0.{idx*2}/31                 10.12.0.{idx*2+1}/31
                                     │
                                   tap0 (169.254.0.22/30) ←── TAP ──→ eth0 (169.254.0.21)
                                                                          ↑ kernel ip= boot arg

• Host-reachable IP: 10.11.0.{idx}/32 — routed through veth to namespace, DNAT'd to guest
• Outbound NAT: guest (169.254.0.21) → SNAT to vpeerIP inside namespace → MASQUERADE on host to default interface
• Inbound NAT: host traffic to 10.11.0.{idx} → DNAT to 169.254.0.21 inside namespace
• IP forwarding enabled inside each namespace
• All details in internal/network/setup.go

Sandbox State Machine

PENDING → STARTING → RUNNING → PAUSED → HIBERNATED
                       │          │
                       ↓          ↓
                    STOPPED    STOPPED → (destroyed)

Any state → ERROR (on crash/failure)
PAUSED → RUNNING (warm snapshot resume)
HIBERNATED → RUNNING (cold snapshot resume, slower)

Key Request Flows

Sandbox creation (POST /v1/sandboxes):

1. API handler generates sandbox ID, inserts into DB as "pending"
2. RPC CreateSandbox → host agent → sandbox.Manager.Create()
3. Manager: resolve base rootfs → acquire shared loop device → create dm-snapshot (sparse CoW file) → allocate network slot → CreateNetwork() (netns + veth + tap + NAT) → vm.Create() (start Firecracker with /dev/mapper/wrenn-{id}, configure via HTTP API, boot) → envdclient.WaitUntilReady() (poll /health) → store in-memory state
4. API handler updates DB to "running" with host_ip

Command execution (POST /v1/sandboxes/{id}/exec):

1. API handler verifies sandbox is "running" in DB
2. RPC Exec → host agent → sandbox.Manager.Exec() → envdclient.Exec()
3. envd client opens bidirectional Connect RPC stream (process.Start), collects stdout/stderr/exit_code
4. API handler checks UTF-8 validity (base64-encodes if binary), updates last_active_at, returns result

Streaming exec (WS /v1/sandboxes/{id}/exec/stream):

1. WebSocket upgrade, read first message for cmd/args
2. RPC ExecStream → host agent → sandbox.Manager.ExecStream() → envdclient.ExecStream()
3. envd client returns a channel of events; host agent forwards events through the RPC stream
4. API handler forwards stream events to WebSocket as JSON messages ({type: "stdout"|"stderr"|"exit", ...})

File transfer: Write uses multipart POST to envd /files; read uses GET. Streaming variants chunk in 64KB pieces through the RPC stream.

REST API

Routes defined in internal/api/server.go, handlers in internal/api/handlers_*.go. OpenAPI spec embedded via //go:embed and served at /openapi.yaml (Swagger UI at /docs). JSON request/response. API key auth via X-API-Key header. Error responses: {"error": {"code": "...", "message": "..."}}.

Code Generation

Proto (Connect RPC)

Proto source of truth is proto/envd/*.proto and proto/hostagent/*.proto. Run make proto to regenerate. Three buf.gen.yaml files control output:

buf.gen.yaml location	Generates to	Used by
proto/envd/buf.gen.yaml	proto/envd/gen/	Main module (host agent's envd client)
proto/hostagent/buf.gen.yaml	proto/hostagent/gen/	Main module (control plane ↔ host agent)
envd/spec/buf.gen.yaml	envd/internal/services/spec/	envd module (guest agent server)

The envd buf.gen.yaml reads from ../../proto/envd/ (same source protos) but generates into envd's own module. This means the same .proto files produce two independent sets of Go stubs — one for each Go module.

To add a new RPC method: edit the .proto file → make proto → implement the handler on both sides.

sqlc

Config: sqlc.yaml (project root). Reads queries from db/queries/*.sql, reads schema from db/migrations/, outputs to internal/db/.

To add a new query: add it to the appropriate .sql file in db/queries/ → make generate → use the new method on *db.Queries.

Key Technical Decisions

• Connect RPC (not gRPC) for all RPC communication between components
• Buf + protoc-gen-connect-go for code generation (not protoc-gen-go-grpc)
• Raw Firecracker HTTP API via Unix socket (not firecracker-go-sdk Machine type)
• TAP networking (not vsock) for host-to-envd communication
• Device-mapper snapshots for rootfs CoW — shared read-only loop device per base template, per-sandbox sparse CoW file, Firecracker gets /dev/mapper/wrenn-{id}
• PostgreSQL via pgx/v5 + sqlc (type-safe query generation). Goose for migrations (plain SQL, up/down)
• Dashboard: SvelteKit (Svelte 5, adapter-static) + Tailwind CSS v4 + Bits UI. Built to static files, embedded into the Go binary via go:embed, served as catch-all at root
• Lago for billing (external service, not in this codebase)

Coding Conventions

• Go style: gofmt, go vet, context.Context everywhere, errors wrapped with fmt.Errorf("action: %w", err), slog for logging, no global state
• Naming: Sandbox IDs sb- + 8 hex, API keys wrn_ + 32 chars, Host IDs host- + 8 hex
• Dependencies: Use go get to add deps, never hand-edit go.mod. For envd deps: cd envd && go get ... (separate module)
• Generated code: Always commit generated code (proto stubs, sqlc). Never add generated code to .gitignore
• Migrations: Always use make migrate-create name=xxx, never create migration files manually
• Testing: Table-driven tests for handlers and state machine transitions

Two-module gotcha

The main module (go.mod) and envd (envd/go.mod) are fully independent. make tidy, make fmt, make vet already operate on both. But when adding dependencies manually, remember to target the correct module (cd envd && go get ... for envd deps). make proto also generates stubs for both modules from the same proto sources.

Rootfs & Guest Init

• wrenn-init (images/wrenn-init.sh): the PID 1 init script baked into every rootfs. Mounts virtual filesystems, sets hostname, writes /etc/resolv.conf, then execs envd.
• Updating the rootfs after changing envd or wrenn-init: bash scripts/update-debug-rootfs.sh [rootfs_path]. This builds envd via make build-envd, mounts the rootfs image, copies in the new binaries, and unmounts. Defaults to /var/lib/wrenn/images/minimal.ext4.
• Rootfs images are minimal debootstrap — no systemd, no coreutils beyond busybox. Use /bin/sh -c for shell builtins inside the guest.

Fixed Paths (on host machine)

• Kernel: /var/lib/wrenn/kernels/vmlinux
• Base rootfs images: /var/lib/wrenn/images/{template}.ext4
• Sandbox clones: /var/lib/wrenn/sandboxes/
• Firecracker: /usr/local/bin/firecracker (e2b's fork of firecracker)

Web UI Styling

Identity

Warm, confident developer tool with industrial precision and crafted organic character. The feel is sharp and data-forward — not cold or sterile, but not soft either. Think: an engineer's favorite tool, built with care.

---

Color Palette (Dark Mode)

Background scale (6 steps, near-black-green): #0a0c0b (bg-0, page base) → #0f1211 (bg-1, sidebar/topbar) → #141817 (bg-2, cards/surfaces) → #1a1e1c (bg-3, hover states/elevated) → #212624 (bg-4, inputs/avatars) → #2a302d (bg-5, active controls)

Text hierarchy (5 levels):

• Bright #eae7e2 — page titles, metric values, active states
• Primary #d0cdc6 — body text, nav labels, readable content
• Secondary #9b9790 — supporting text, inactive nav, descriptions
• Tertiary #6b6862 — labels, section headers, timestamps
• Muted #454340 — ghost text, disabled states, grid labels

Sage green brand accent (3 tiers + 2 glows):

• Solid #5e8c58 — primary accent, buttons, borders, active indicators
• Mid #89a785 — badges, chart lines, secondary accent
• Bright #a4c89f — active nav text, live counts, chart dots
• Glow rgba(94,140,88,0.07) — active nav backgrounds, subtle highlights
• Glow Mid rgba(94,140,88,0.14) — live badges, status badge backgrounds

Borders (2 levels):

• Default #1f2321 — card edges, dividers, sidebar borders
• Mid #2a2f2c — hover states, interactive borders, stronger separation

Semantic status colors:

• Amber #d4a73c — warning, building, countdown timers
• Red #cf8172 — error, failed, destructive actions
• Blue #5a9fd4 — info, stopped (use sparingly)

Light mode: (TBD — follow same warm-tinted approach. Background scale from #f8f6f1 → #dedbd5. Text hierarchy inverts. Accent stays #5e8c58 for solid.)

---

Typography

Four fonts, each with a clear role:

Font	Role	Weights	Where
Manrope (variable)	Body, UI	400–700	All body text, nav labels, buttons, descriptions, section headers
Instrument Serif	Display, metrics	400	Page titles (h1), large metric values, empty-state headings only
JetBrains Mono	Code, data	400–600	Status bar, time range buttons, search inputs, IDs, commit SHAs, countdown timers, log viewer, URL paths, code blocks
Alice	Brand wordmark	400	Sidebar wordmark only — never used elsewhere

Sizing:

• Base body: 14px
• Page title (h1): 24px serif, letter-spacing: -0.02em
• Card metric values: 36px serif, letter-spacing: -0.04em
• Chart inline metric: 30px serif, letter-spacing: -0.04em
• Nav items: 13px body, weight 500
• Section/group labels: 11px body, uppercase, letter-spacing: 0.06em, weight 600
• Chart section labels: 12px body, uppercase, letter-spacing: 0.05em, weight 600
• Stat cell labels: 11px body, uppercase, letter-spacing: 0.05em, weight 600
• Badge text: 10px, uppercase, letter-spacing: 0.04em, weight 600
• Status bar / footer links: 11–12px mono
• Table headers: 11px body, uppercase, letter-spacing: 0.05em, weight 600, color muted
• Table body cells: 13px

Key rule: Instrument Serif is reserved exclusively for page-level titles and large numeric values. It provides warmth and character without softness. Everything else uses Manrope (UI) or JetBrains Mono (data/code).

---

Spacing

4px base unit (Tailwind scale). Moderate density — functional and confident, never cramped.

• Page content padding: 24–28px
• Card/surface internal padding: 18–20px
• Sidebar width: 230px
• Sidebar nav item padding: 8px 10px
• Sidebar brand area: 18px 16px 16px
• Tab bar items: 10px 16px
• Topbar: 16px 28px
• Metric strip cell: 18px 20px
• Chart header: 18px 20px
• Chart canvas: 14px 20px 12px
• Table header cells: 11px 16px
• Table body cells: 12px 16px
• Status bar: 6px 28px
• Between sections (cards): 20–24px margin-bottom

---

Borders & Depth

Flat aesthetic — no drop shadows. Depth comes from background color stepping (bg-0 → bg-1 → bg-2 → bg-3), not shadows. --shadow-sm: 0 0 #0000.

• All borders: 1px solid in warm muted tones
• Corner radii: cards/surfaces 8px, inputs/buttons 5px, logo mark 6px, avatars 5px, dots 50%
• Connected metric cells use shared border container with border-left: 1px solid between cells (no gap/grid trick) — creates the industrial panel look
• Tables wrapped in border-radius: 8px container with overflow hidden

---

Components

Sidebar navigation:

• Active items use 3px left-border in sage solid (#5e8c58) with accent glow background (rgba(94,140,88,0.07))
• Active text color: accent-bright (#a4c89f)
• Icons at 16px, opacity 0.5 default, 1.0 on active
• Group labels: 11px uppercase with 0.06em tracking, muted color

Status chip (live indicator):

• Rounded 8px border, bg-2 background, border-mid border
• Pulsing dot: 7px, accent-solid fill, box-shadow: 0 0 8px rgba(94,140,88,0.5) with glow animation
• Count in mono at 14px accent-bright, label in secondary text

Live badges (inline):

• 10px text, uppercase, 3px border-radius
• Background: accent-glow-mid (rgba(94,140,88,0.14)), text: accent mid
• Includes 5px pulsing dot with box-shadow

Metric strip:

• 3-column grid, connected cells (single outer border, inner dividers)
• Hover: background steps from bg-2 to bg-3
• Value: 36px serif, bright text
• Label: 11px uppercase, tertiary
• Sub-metadata row with 1px divider between items

Chart cards:

• 8px border-radius, bg-2 background, default border
• Header: section label (12px uppercase) + large serif metric + live badge
• Range group: segmented buttons with 1px borders, mono text, active state uses bg-5
• Chart area: SVG with 0.5px grid lines in border color, 10px mono axis labels in muted
• Data line: 1.5px accent-solid stroke, stroke-linejoin: round
• Area fill: gradient from rgba(94,140,88,0.28) → transparent
• Data dot: accent-bright fill, 2.5px bg-2 stroke, 4px radius

Buttons hierarchy:

1. Ghost (icon-btn): transparent bg, default border, tertiary color → border-mid + secondary on hover
2. Outline: no bg, border-mid border → accent-solid border + primary text on hover
3. Tool: bg-2 background, default border → border-mid + primary on hover
4. Filled/CTA: accent-solid background, white text → lighter green on hover, subtle translateY(-1px) lift

Tables:

• Container: 8px border-radius, border, overflow hidden
• Header: bg-3 background, 11px uppercase muted text
• Body: default bg, 1px border-bottom between rows
• Row hover: bg-3

Empty states:

• Centered, 72px vertical padding
• Icon container: 56px square, bg-3, border-mid border, 8px radius
• Heading: 20px serif, bright text
• Description: 13px body, tertiary text
• CTA button below

Inputs:

• bg-2 background, default border, 5px radius
• Mono font for search/filter inputs
• Focus: border-color: accent-solid (clean single ring, no double-ring)
• Placeholder: muted color

Focus rings: Single accent-solid border-color change on focus. Clean and minimal — no double-ring outlines.

---

Animation

• All interactive transitions: 150ms ease
• Page load / section entrance: fadeUp — opacity: 0, translateY(6px) → visible, 0.35s ease, staggered with 60–80ms delays between elements
• Chart data animation: SVG <animate> on path d, polyline points, and circle cy — 0.5–0.6s duration, 0.2–0.35s begin delay, fill: freeze
• Live status dot: glow keyframe — 2.5s ease infinite box-shadow bloom from 0 0 6px rgba(94,140,88,0.5) → 0 0 14px rgba(94,140,88,0.2)
• CTA buttons: subtle translateY(-1px) on hover for lift feel

---

Dark Mode

Primary and default mode. Very dark near-black-green backgrounds (#0a0c0b base) with warm off-white text and desaturated sage accent. Completely flat — no card shadows anywhere. System preference detection + localStorage persistence.

---

Overall Feel

Sharp, warm, industrial-confident. Avoids cold grays entirely — palette leans slightly warm/brown-tinted throughout. The serif display type provides organic character and warmth on titles and metrics, while Manrope handles readable UI text and JetBrains Mono anchors the data-forward, developer-tool identity. Connected metric panels, tight chart cards, and uppercase section labels create engineering density without sacrificing readability. The result is a tool that feels crafted and precise — designed by someone who uses developer tools daily.