tksadik92/wrenn-releases
1
0
Fork 0
forked from wrenn/wrenn
Code Pull Requests Activity

Minor patch

Browse Source
This commit is contained in:
Rafeed M. Bhuiyan
2026-04-16 18:14:50 +06:00
parent 9c4fea93bc
commit cc63ed2197
3 changed files with 93 additions and 68 deletions
Download Patch File Download Diff File Expand all files Collapse all files

14
CLAUDE.md
View File

@ -12,10 +12,10 @@ All commands go through the Makefile. Never use raw `go build` or `go run`.
```bash ```bash
make build # Build all binaries → builds/ make build # Build all binaries → builds/
make build-cp # Control plane only (builds frontend first) make build-cp # Control plane only
make build-agent # Host agent only make build-agent # Host agent only
make build-envd # envd static binary (verified statically linked) make build-envd # envd static binary (verified statically linked)
make build-frontend # SvelteKit dashboard → internal/dashboard/static/ make build-frontend # SvelteKit dashboard → frontend/build/ (served by Caddy)
make dev # Full local dev: infra + migrate + control plane make dev # Full local dev: infra + migrate + control plane
make dev-infra # Start PostgreSQL + Prometheus + Grafana (Docker) make dev-infra # Start PostgreSQL + Prometheus + Grafana (Docker)
@ -64,7 +64,7 @@ envd is a **completely independent Go module**. It is never imported by the main
### Control Plane ### Control Plane
**Internal packages:** `internal/api/`, `internal/dashboard/`, `internal/email/` **Internal packages:** `internal/api/`, `internal/email/`
**Public packages (importable by cloud repo):** `pkg/config/`, `pkg/db/`, `pkg/auth/`, `pkg/auth/oauth/`, `pkg/scheduler/`, `pkg/lifecycle/`, `pkg/channels/`, `pkg/audit/`, `pkg/service/`, `pkg/events/`, `pkg/id/`, `pkg/validate/` **Public packages (importable by cloud repo):** `pkg/config/`, `pkg/db/`, `pkg/auth/`, `pkg/auth/oauth/`, `pkg/scheduler/`, `pkg/lifecycle/`, `pkg/channels/`, `pkg/audit/`, `pkg/service/`, `pkg/events/`, `pkg/id/`, `pkg/validate/`
@ -78,7 +78,7 @@ Startup (`cmd/control-plane/main.go`) is a thin wrapper: `cpserver.Run(cpserver.
- **API Server** (`internal/api/server.go`): chi router with middleware. Creates handler structs (`sandboxHandler`, `execHandler`, `filesHandler`, etc.) injected with `db.Queries` and the host agent Connect RPC client. Routes under `/v1/capsules/*`. Accepts `[]cpextension.Extension` — each extension's `RegisterRoutes()` is called after all core routes are registered. - **API Server** (`internal/api/server.go`): chi router with middleware. Creates handler structs (`sandboxHandler`, `execHandler`, `filesHandler`, etc.) injected with `db.Queries` and the host agent Connect RPC client. Routes under `/v1/capsules/*`. Accepts `[]cpextension.Extension` — each extension's `RegisterRoutes()` is called after all core routes are registered.
- **Reconciler** (`internal/api/reconciler.go`): background goroutine (every 30s) that compares DB records against `agent.ListSandboxes()` RPC. Marks orphaned DB entries as "stopped". - **Reconciler** (`internal/api/reconciler.go`): background goroutine (every 30s) that compares DB records against `agent.ListSandboxes()` RPC. Marks orphaned DB entries as "stopped".
- **Dashboard** (SvelteKit + Tailwind + Bits UI, statically built and embedded via `go:embed`, served as catch-all at root) - **Dashboard** (SvelteKit + Tailwind + Bits UI, built to static files in `frontend/build/`, served by Caddy as a reverse proxy)
- **Database**: PostgreSQL via pgx/v5. Queries generated by sqlc from `db/queries/*.sql` → `pkg/db/`. Migrations in `db/migrations/` (goose, plain SQL). `db/migrations/embed.go` exposes `migrations.FS` so the cloud repo can run OSS migrations via `go:embed`. - **Database**: PostgreSQL via pgx/v5. Queries generated by sqlc from `db/queries/*.sql` → `pkg/db/`. Migrations in `db/migrations/` (goose, plain SQL). `db/migrations/embed.go` exposes `migrations.FS` so the cloud repo can run OSS migrations via `go:embed`.
- **Config** (`pkg/config/config.go`): purely environment variables (`DATABASE_URL`, `CP_LISTEN_ADDR`, `CP_HOST_AGENT_ADDR`), no YAML/file config. - **Config** (`pkg/config/config.go`): purely environment variables (`DATABASE_URL`, `CP_LISTEN_ADDR`, `CP_HOST_AGENT_ADDR`), no YAML/file config.
@ -115,8 +115,8 @@ Runs as PID 1 inside the microVM via `wrenn-init.sh` (mounts procfs/sysfs/dev, s
- **Package manager**: pnpm - **Package manager**: pnpm
- **Routing**: SvelteKit file-based routing under `frontend/src/routes/` - **Routing**: SvelteKit file-based routing under `frontend/src/routes/`
- **Routing layout**: `/login` and `/signup` at root, authenticated pages under `/dashboard/*` (e.g. `/dashboard/capsules`, `/dashboard/keys`) - **Routing layout**: `/login` and `/signup` at root, authenticated pages under `/dashboard/*` (e.g. `/dashboard/capsules`, `/dashboard/keys`)
- **Build output**: `frontend/build/` → copied to `internal/dashboard/static/` → embedded via `go:embed` into the control plane binary - **Build output**: `frontend/build/` — static files served by Caddy
- **Serving**: `internal/dashboard/dashboard.go` registers a `NotFound` catch-all SPA handler with fallback to `index.html`. API routes (`/v1/*`, `/openapi.yaml`, `/docs`) are registered first and take priority - **Serving**: Caddy reverse-proxies API requests to the control plane and serves the SvelteKit SPA directly. The control plane does not serve frontend assets.
- **Dev workflow**: `make dev-frontend` runs Vite dev server on port 5173 with HMR. API calls proxy to `http://localhost:8000` - **Dev workflow**: `make dev-frontend` runs Vite dev server on port 5173 with HMR. API calls proxy to `http://localhost:8000`
- **Fonts**: Manrope (UI), Instrument Serif (headings), JetBrains Mono (code), Alice (brand wordmark) — all self-hosted via `@fontsource` - **Fonts**: Manrope (UI), Instrument Serif (headings), JetBrains Mono (code), Alice (brand wordmark) — all self-hosted via `@fontsource`
- **Dark mode**: class-based (`.dark` on `<html>`) with system preference detection + localStorage persistence - **Dark mode**: class-based (`.dark` on `<html>`) with system preference detection + localStorage persistence
@ -211,7 +211,7 @@ To add a new query: add it to the appropriate `.sql` file in `db/queries/` → `
- **TAP networking** (not vsock) for host-to-envd communication - **TAP networking** (not vsock) for host-to-envd communication
- **Device-mapper snapshots** for rootfs CoW — shared read-only loop device per base template, per-sandbox sparse CoW file, Firecracker gets `/dev/mapper/wrenn-{id}` - **Device-mapper snapshots** for rootfs CoW — shared read-only loop device per base template, per-sandbox sparse CoW file, Firecracker gets `/dev/mapper/wrenn-{id}`
- **PostgreSQL** via pgx/v5 + sqlc (type-safe query generation). Goose for migrations (plain SQL, up/down) - **PostgreSQL** via pgx/v5 + sqlc (type-safe query generation). Goose for migrations (plain SQL, up/down)
- **Dashboard**: SvelteKit (Svelte 5, adapter-static) + Tailwind CSS v4 + Bits UI. Built to static files, embedded into the Go binary via `go:embed`, served as catch-all at root - **Dashboard**: SvelteKit (Svelte 5, adapter-static) + Tailwind CSS v4 + Bits UI. Built to static files in `frontend/build/`, served by Caddy (not embedded in the Go binary)
- **Lago** for billing (external service, not in this codebase) - **Lago** for billing (external service, not in this codebase)
## Coding Conventions ## Coding Conventions

133
README.md
View File

@ -2,16 +2,16 @@
Secure infrastructure for AI Secure infrastructure for AI
## Deployment ## Prerequisites
### Prerequisites
- Linux host with `/dev/kvm` access (bare metal or nested virt) - Linux host with `/dev/kvm` access (bare metal or nested virt)
- Firecracker binary at `/usr/local/bin/firecracker` - Firecracker binary at `/usr/local/bin/firecracker`
- PostgreSQL - PostgreSQL
- Go 1.25+ - Go 1.25+
- pnpm (for frontend)
- Docker (for dev infra and rootfs builds)
### Build ## Build
```bash ```bash
make build # outputs to builds/ make build # outputs to builds/
@ -19,30 +19,77 @@ make build # outputs to builds/
Produces three binaries: `wrenn-cp` (control plane), `wrenn-agent` (host agent), `envd` (guest agent). Produces three binaries: `wrenn-cp` (control plane), `wrenn-agent` (host agent), `envd` (guest agent).
### Host setup ## Host setup
The host agent machine needs: The host agent needs a kernel, a minimal rootfs image, and working directories on the host machine.
```bash ### Directory structure
# Kernel for guest VMs
mkdir -p /var/lib/wrenn/kernels
# Place a vmlinux kernel at /var/lib/wrenn/kernels/vmlinux
# Rootfs images ```
mkdir -p /var/lib/wrenn/images /var/lib/wrenn/
# Build or place .ext4 rootfs images (e.g., minimal.ext4) ├── kernels/
│ └── vmlinux # uncompressed Linux kernel (not bzImage)
# Sandbox working directory ├── images/
mkdir -p /var/lib/wrenn/sandboxes │ └── minimal/
│ └── rootfs.ext4 # base rootfs (all other templates snapshot from this)
# Snapshots directory ├── sandboxes/ # per-sandbox CoW files (created at runtime)
mkdir -p /var/lib/wrenn/snapshots └── snapshots/ # pause/hibernate snapshot files (created at runtime)
# Enable IP forwarding
sysctl -w net.ipv4.ip_forward=1
``` ```
### Configure Create the directories:
```bash
sudo mkdir -p /var/lib/wrenn/{kernels,images/minimal,sandboxes,snapshots}
```
### Kernel
Place an uncompressed `vmlinux` kernel at `/var/lib/wrenn/kernels/vmlinux`. Versioned kernels (`vmlinux-{semver}`) are also supported — the agent picks the latest by semver.
### Minimal rootfs
The minimal rootfs is the base image that all other templates (Python, Node, etc.) are built on top of via device-mapper snapshots. It must contain:
| Package | Why |
|---------|-----|
| `socat` | Bidirectional relay for port forwarding |
| `chrony` | Time sync from KVM PTP clock (`/dev/ptp0`) |
| `tini` | PID 1 zombie reaper (injected by build script, not apt) |
| `sudo` | User privilege management inside the guest |
| `wget` | HTTP fetching |
| `curl` | HTTP client |
| `ca-certificates` | TLS certificate verification |
**To build a rootfs from a Docker container:**
1. Create and configure a container with the required packages:
```bash
docker run -it --name wrenn-minimal debian:bookworm bash
# Inside the container:
apt update && apt install -y socat chrony sudo wget curl ca-certificates
exit
```
2. Export to a rootfs image (builds envd, injects wrenn-init + tini, shrinks to minimum size):
```bash
sudo bash scripts/rootfs-from-container.sh wrenn-minimal minimal
```
**To update an existing rootfs** after changing envd or `wrenn-init.sh`:
```bash
bash scripts/update-minimal-rootfs.sh
```
This rebuilds envd via `make build-envd` and copies the fresh binaries into the mounted rootfs image.
### IP forwarding
```bash
sudo sysctl -w net.ipv4.ip_forward=1
```
## Configure
Copy `.env.example` to `.env` and edit: Copy `.env.example` to `.env` and edit:
@ -59,25 +106,21 @@ WRENN_HOST_LISTEN_ADDR=:50051
WRENN_DIR=/var/lib/wrenn WRENN_DIR=/var/lib/wrenn
``` ```
### Run ## Development
```bash ```bash
# Apply database migrations make dev # Start PostgreSQL (Docker), run migrations, start control plane
make migrate-up make dev-agent # Start host agent (separate terminal, sudo)
make dev-frontend # Vite dev server with HMR (port 5173)
# Start control plane make check # fmt + vet + lint + test
./builds/wrenn-cp
``` ```
Control plane listens on `WRENN_CP_LISTEN_ADDR` (default `:8000`).
### Host registration ### Host registration
Hosts must be registered with the control plane before they can serve sandboxes. Hosts must be registered with the control plane before they can serve sandboxes.
1. **Create a host record** (via API or dashboard): 1. **Create a host record** (via API or dashboard):
```bash ```bash
# As an admin (JWT auth)
curl -X POST http://localhost:8000/v1/hosts \ curl -X POST http://localhost:8000/v1/hosts \
-H "Authorization: Bearer $JWT_TOKEN" \ -H "Authorization: Bearer $JWT_TOKEN" \
-H "Content-Type: application/json" \ -H "Content-Type: application/json" \
@ -87,17 +130,16 @@ Hosts must be registered with the control plane before they can serve sandboxes.
2. **Start the host agent** with the registration token and its externally-reachable address: 2. **Start the host agent** with the registration token and its externally-reachable address:
```bash ```bash
sudo WRENN_CP_URL=http://cp-host:8000 \ sudo WRENN_CP_URL=http://localhost:8000 \
./builds/wrenn-agent \ ./builds/wrenn-agent \
--register <token-from-step-1> \ --register <token-from-step-1> \
--address 10.0.1.5:50051 --address <host-ip>:50051
``` ```
On first startup the agent sends its specs (arch, CPU, memory, disk) to the control plane, receives a long-lived host JWT, and saves it to `$WRENN_DIR/host-token`. On first startup the agent sends its specs (arch, CPU, memory, disk) to the control plane, receives a long-lived host JWT, and saves it to `$WRENN_DIR/host-token`.
3. **Subsequent startups** don't need `--register` — the agent loads the saved JWT automatically: 3. **Subsequent startups** don't need `--register` — the agent loads the saved JWT automatically:
```bash ```bash
sudo WRENN_CP_URL=http://cp-host:8000 \ sudo ./builds/wrenn-agent --address <host-ip>:50051
./builds/wrenn-agent --address 10.0.1.5:50051
``` ```
4. **If registration fails** (e.g., network error after token was consumed), regenerate a token: 4. **If registration fails** (e.g., network error after token was consumed), regenerate a token:
@ -107,23 +149,6 @@ Hosts must be registered with the control plane before they can serve sandboxes.
``` ```
Then restart the agent with the new token. Then restart the agent with the new token.
The agent sends heartbeats to the control plane every 30 seconds. Host agent listens on `WRENN_HOST_LISTEN_ADDR` (default `:50051`). The agent sends heartbeats to the control plane every 30 seconds.
### Rootfs images
envd must be baked into every rootfs image. After building:
```bash
make build-envd
bash scripts/update-debug-rootfs.sh /var/lib/wrenn/images/minimal.ext4
```
## Development
```bash
make dev # Start PostgreSQL (Docker), run migrations, start control plane
make dev-agent # Start host agent (separate terminal, sudo)
make check # fmt + vet + lint + test
```
See `CLAUDE.md` for full architecture documentation. See `CLAUDE.md` for full architecture documentation.

14
cmd/host-agent/main.go
View File

@ -271,13 +271,13 @@ func checkPrivileges() error {
bit uint bit uint
name string name string
}{ }{
{1, "CAP_DAC_OVERRIDE"}, // /dev/loop*, /dev/mapper/*, /dev/net/tun {1, "CAP_DAC_OVERRIDE"}, // /dev/loop*, /dev/mapper/*, /dev/net/tun
{5, "CAP_KILL"}, // SIGTERM/SIGKILL to Firecracker processes {5, "CAP_KILL"}, // SIGTERM/SIGKILL to Firecracker processes
{12, "CAP_NET_ADMIN"}, // netlink, iptables, routing, TAP/veth {12, "CAP_NET_ADMIN"}, // netlink, iptables, routing, TAP/veth
{13, "CAP_NET_RAW"}, // raw sockets (iptables) {13, "CAP_NET_RAW"}, // raw sockets (iptables)
{19, "CAP_SYS_PTRACE"}, // reading /proc/self/ns/net (netns.Get) {19, "CAP_SYS_PTRACE"}, // reading /proc/self/ns/net (netns.Get)
{21, "CAP_SYS_ADMIN"}, // netns, mount ns, losetup, dmsetup {21, "CAP_SYS_ADMIN"}, // netns, mount ns, losetup, dmsetup
{27, "CAP_MKNOD"}, // device-mapper node creation {27, "CAP_MKNOD"}, // device-mapper node creation
} }
var missing []string var missing []string