forked from wrenn/wrenn
pptx704
bd98610153
Running port-binding applications (Jupyter, http.server, NextJS) inside
sandboxes caused severe PTY sluggishness and proxy navigation errors.
Root cause: the CP sandbox proxy and Connect RPC pool shared a single
HTTP transport. Heavy proxy traffic (Jupyter WebSocket, REST polling)
interfered with PTY RPC streams via HTTP/2 flow control contention.
Transport isolation (main fix):
- Add dedicated proxy transport on CP (NewProxyTransport) with HTTP/2
disabled, separate from the RPC pool transport
- Add dedicated proxy transport on host agent, replacing
http.DefaultTransport
- Add dedicated envdclient transport with tuned connection pooling
- Replace http.DefaultClient in file streaming RPCs with per-sandbox
envd client
Proxy path rewriting (navigation fix):
- Add ModifyResponse to rewrite Location headers with /proxy/{id}/{port}
prefix, handling both root-relative and absolute-URL redirects
- Strip prefix back out in CP subdomain proxy for correct browser
behavior
- Replace path.Join with string concat in CP Director to preserve
trailing slashes (prevents redirect loops on directory listings)
Proxy resilience:
- Add dial retry with linear backoff (3 attempts) to handle socat
startup delay when ports are first detected
- Cache ReverseProxy instances per sandbox+port+host in sync.Map
- Add EvictProxy callback wired into sandbox Manager.Destroy
Buffer and server hardening:
- Increase PTY and exec stream channel buffers from 16 to 256
- Add ReadHeaderTimeout (10s) and IdleTimeout (620s) to host agent
HTTP server
Network tuning:
- Set TAP device TxQueueLen to 5000 (up from default 1000)
- Add Firecracker tx_rate_limiter (200 MB/s sustained, 100 MB burst)
to prevent guest traffic from saturating the TAP
34 lines
906 B
Go
34 lines
906 B
Go
package envdclient
|
|
|
|
import (
|
|
"fmt"
|
|
"net"
|
|
"net/http"
|
|
"time"
|
|
)
|
|
|
|
// envdPort is the default port envd listens on inside the guest.
|
|
const envdPort = 49983
|
|
|
|
// baseURL returns the HTTP base URL for reaching envd at the given host IP.
|
|
func baseURL(hostIP string) string {
|
|
return fmt.Sprintf("http://%s:%d", hostIP, envdPort)
|
|
}
|
|
|
|
// newHTTPClient returns an http.Client with a dedicated transport for talking
|
|
// to envd. The transport is intentionally separate from http.DefaultTransport
|
|
// so that proxy traffic to user services inside the sandbox cannot interfere
|
|
// with envd RPC connections (PTY streams, exec, file ops).
|
|
func newHTTPClient() *http.Client {
|
|
return &http.Client{
|
|
Transport: &http.Transport{
|
|
MaxIdleConnsPerHost: 10,
|
|
IdleConnTimeout: 90 * time.Second,
|
|
DialContext: (&net.Dialer{
|
|
Timeout: 10 * time.Second,
|
|
KeepAlive: 30 * time.Second,
|
|
}).DialContext,
|
|
},
|
|
}
|
|
}
|