pptx704
2c66959b92
Implements the full host ↔ control plane connection flow:
- Host CRUD endpoints (POST/GET/DELETE /v1/hosts) with role-based access:
regular hosts admin-only, BYOC hosts for admins and team owners
- One-time registration token flow: admin creates host → gets token (1hr TTL
in Redis + Postgres audit trail) → host agent registers with specs → gets
long-lived JWT (1yr)
- Host agent registration client with automatic spec detection (arch, CPU,
memory, disk) and token persistence to disk
- Periodic heartbeat (30s) via POST /v1/hosts/{id}/heartbeat with X-Host-Token
auth and host ID cross-check
- Token regeneration endpoint (POST /v1/hosts/{id}/token) for retry after
failed registration
- Tag management (add/remove/list) with team-scoped access control
- Host JWT with typ:"host" claim, cross-use prevention in both VerifyJWT and
VerifyHostJWT
- requireHostToken middleware for host agent authentication
- DB-level race protection: RegisterHost uses AND status='pending' with
rows-affected check; Redis GetDel for atomic token consume
- Migration for future mTLS support (cert_fingerprint, mtls_enabled columns)
- Host agent flags: --register (one-time token), --address (required ip:port)
- serviceErrToHTTP extended with "forbidden" → 403 mapping
- OpenAPI spec, .env.example, and README updated
129 lines
3.3 KiB
Markdown
129 lines
3.3 KiB
Markdown
# Wrenn Sandbox
|
|
|
|
MicroVM-based code execution platform. Firecracker VMs, not containers. Pool-based pricing, persistent sandboxes, Python/TS/Go SDKs.
|
|
|
|
## Deployment
|
|
|
|
### Prerequisites
|
|
|
|
- Linux host with `/dev/kvm` access (bare metal or nested virt)
|
|
- Firecracker binary at `/usr/local/bin/firecracker`
|
|
- PostgreSQL
|
|
- Go 1.25+
|
|
|
|
### Build
|
|
|
|
```bash
|
|
make build # outputs to builds/
|
|
```
|
|
|
|
Produces three binaries: `wrenn-cp` (control plane), `wrenn-agent` (host agent), `envd` (guest agent).
|
|
|
|
### Host setup
|
|
|
|
The host agent machine needs:
|
|
|
|
```bash
|
|
# Kernel for guest VMs
|
|
mkdir -p /var/lib/wrenn/kernels
|
|
# Place a vmlinux kernel at /var/lib/wrenn/kernels/vmlinux
|
|
|
|
# Rootfs images
|
|
mkdir -p /var/lib/wrenn/images
|
|
# Build or place .ext4 rootfs images (e.g., minimal.ext4)
|
|
|
|
# Sandbox working directory
|
|
mkdir -p /var/lib/wrenn/sandboxes
|
|
|
|
# Enable IP forwarding
|
|
sysctl -w net.ipv4.ip_forward=1
|
|
```
|
|
|
|
### Configure
|
|
|
|
Copy `.env.example` to `.env` and edit:
|
|
|
|
```bash
|
|
# Required
|
|
DATABASE_URL=postgres://wrenn:wrenn@localhost:5432/wrenn?sslmode=disable
|
|
|
|
# Control plane
|
|
CP_LISTEN_ADDR=:8000
|
|
CP_HOST_AGENT_ADDR=http://localhost:50051
|
|
|
|
# Host agent
|
|
AGENT_LISTEN_ADDR=:50051
|
|
AGENT_KERNEL_PATH=/var/lib/wrenn/kernels/vmlinux
|
|
AGENT_IMAGES_PATH=/var/lib/wrenn/images
|
|
AGENT_SANDBOXES_PATH=/var/lib/wrenn/sandboxes
|
|
```
|
|
|
|
### Run
|
|
|
|
```bash
|
|
# Apply database migrations
|
|
make migrate-up
|
|
|
|
# Start control plane
|
|
./builds/wrenn-cp
|
|
```
|
|
|
|
Control plane listens on `CP_LISTEN_ADDR` (default `:8000`).
|
|
|
|
### Host registration
|
|
|
|
Hosts must be registered with the control plane before they can serve sandboxes.
|
|
|
|
1. **Create a host record** (via API or admin UI):
|
|
```bash
|
|
# As an admin (JWT auth)
|
|
curl -X POST http://localhost:8000/v1/hosts \
|
|
-H "Authorization: Bearer $JWT_TOKEN" \
|
|
-H "Content-Type: application/json" \
|
|
-d '{"type": "regular"}'
|
|
```
|
|
This returns a `registration_token` (valid for 1 hour).
|
|
|
|
2. **Start the host agent** with the registration token and its externally-reachable address:
|
|
```bash
|
|
sudo AGENT_CP_URL=http://cp-host:8000 \
|
|
./builds/wrenn-agent \
|
|
--register <token-from-step-1> \
|
|
--address 10.0.1.5:50051
|
|
```
|
|
On first startup the agent sends its specs (arch, CPU, memory, disk) to the control plane, receives a long-lived host JWT, and saves it to `AGENT_TOKEN_FILE` (default `/var/lib/wrenn/host-token`).
|
|
|
|
3. **Subsequent startups** don't need `--register` — the agent loads the saved JWT automatically:
|
|
```bash
|
|
sudo AGENT_CP_URL=http://cp-host:8000 \
|
|
./builds/wrenn-agent --address 10.0.1.5:50051
|
|
```
|
|
|
|
4. **If registration fails** (e.g., network error after token was consumed), regenerate a token:
|
|
```bash
|
|
curl -X POST http://localhost:8000/v1/hosts/$HOST_ID/token \
|
|
-H "Authorization: Bearer $JWT_TOKEN"
|
|
```
|
|
Then restart the agent with the new token.
|
|
|
|
The agent sends heartbeats to the control plane every 30 seconds. Host agent listens on `AGENT_LISTEN_ADDR` (default `:50051`).
|
|
|
|
### Rootfs images
|
|
|
|
envd must be baked into every rootfs image. After building:
|
|
|
|
```bash
|
|
make build-envd
|
|
bash scripts/update-debug-rootfs.sh /var/lib/wrenn/images/minimal.ext4
|
|
```
|
|
|
|
## Development
|
|
|
|
```bash
|
|
make dev # Start PostgreSQL (Docker), run migrations, start control plane
|
|
make dev-agent # Start host agent (separate terminal, sudo)
|
|
make check # fmt + vet + lint + test
|
|
```
|
|
|
|
See `CLAUDE.md` for full architecture documentation.
|