mocache

mocache is a single static Go binary that replaces the Python/nginx implementation in nul/mocache-python as a reverse cache for public, immutable Mastodon S3 media. One binary, four roles, configured by one YAML file:

Role	What it does
cache	Reverse cache only (no agent).
single	Cache and read-only metrics agent in one process (all-in-one).
router	Stateless consistent-hash proxy across N ≥ 2 cache nodes, with failover.
panel	Read-only status page + JSON API aggregating agent endpoints.

Security scope (read this first)

mocache is deliberately built for one threat model only: public, immutable media objects served from a single fixed S3-compatible origin.

• No private or signed S3. Presigned URLs, Authorization and Cookie headers are never forwarded to the origin, and mocache does not implement request signing. Pointing it at a bucket that requires auth will not work and is not a goal.
• No open proxy. Exactly one fixed cache.upstream per cache process. There is no per-request upstream selection.
• No purge / delete / invalidate API. Cached objects live until evicted (idle > 24 h or max size exceeded) or the cache directory is wiped. If an object changes at the origin within its freshness window, mocache keeps serving the old copy. Only deploy this for content that is truly immutable (Mastodon media attachments are content-addressed in practice).
• Miss-flood risk. On a cold cache every distinct URI triggers exactly one full origin fetch (singleflight collapses concurrent requests for the same URI, but not across distinct URIs). Restarting all cache nodes at once against a slow origin can recreate the equivalent of a cache-miss storm. Scale/restart nodes one at a time and watch origin request counts in the panel.
• Origin responses are sanitized: no S3 XML, no x-amz-*, no Set-Cookie ever reaches the client. Errors are generic 404/502 and are never cached. Success responses carry exactly one Cache-Control: public, max-age=31536000, immutable plus X-Mocache: HIT|MISS|STALE and X-Mocache-Node. The latter is the visible, stable cache node name (hostname) kept for nginx-parity checks and panel/debug routing — deliberately not a secret; it grants no access and is not part of any secret rotation.

Agent and panel are read-only (GET only, no mutating routes), hold all state in memory, and mask agent errors in the aggregated view.

The agent answers on two JSON routes: /metrics (flat native shape) and /api/stats, a legacy-compatible alias mirroring the original Python agent (node, version, generated_at plus a nested metrics object with requests, hit, miss, stale, other, bytes_sent, cache_files, cache_bytes, upstream_samples, upstream_seconds, filesystem_free, filesystem_total, hit_ratio, upstream_avg_seconds). It exists so the unchanged Python panel aggregator (mocache_panel.py, mixed-fleet canary deployments) accepts Go nodes; upstream_avg_seconds is null without samples.

Quickstart (native, no Docker)

CGO_ENABLED=0 go build -ldflags="-s -w" -o mocache ./cmd/mocache
sudo useradd -r -s /usr/sbin/nologin mocache 2>/dev/null || true
sudo mkdir -p /var/cache/mocache /etc/mocache
sudo cp deploy/config.example.yaml /etc/mocache/config.yaml
# edit /etc/mocache/config.yaml: set cache.upstream to your S3 base URL
sudo chown -R mocache:mocache /var/cache/mocache
sudo ./mocache -config /etc/mocache/config.yaml
curl -sS -D - http://127.0.0.1:8080/healthz

The binary is fully static (CGO_ENABLED=0, no cgo, pure stdlib + gopkg.in/yaml.v3), so it runs on any Linux without glibc worries and inside FROM scratch containers. A systemd unit is provided in deploy/systemd/mocache.service:

sudo cp deploy/systemd/mocache.service /etc/systemd/system/
sudo systemctl daemon-reload && sudo systemctl enable --now mocache

The service runs as user mocache with ProtectSystem=strict, ReadWritePaths=/var/cache/mocache, NoNewPrivileges=true and restarts on failure.

Configuration reference

See deploy/config.example.yaml for a fully commented file with three loadable profiles (single, router, panel). YAML keys map onto the mocache-python environment variables as follows:

YAML key	mocache-python variable	Default
cache.upstream	UPSTREAM_ORIGIN	(required)
cache.dir	cache directory	/var/cache/mocache
cache.max_size_bytes	CACHE_MAX_SIZE	21474836480 (20 GiB)
cache.inactive	CACHE_INACTIVE	24h
cache.fresh_for	freshness window (stale-serve)	48h
cache.lock_timeout	singleflight wait timeout	30s
cache.connect_timeout	origin connect timeout	3s
cache.read_timeout	origin read timeout	60s
cache.eviction_interval	eviction sweep interval	1m
cache.browser_max_age	browser Cache-Control max-age on successes (optional; unset = 1 year)	unset (31536000)
router.upstreams	nginx upstream list	N ≥ 2 entries
router.tries	proxy_next_upstream retries	2
panel.agent_urls	agent base URLs (aggregator appends /metrics)	[]
panel.agent_timeout	agent scrape timeout	2s
agent.listen	agent port (single role)	:8081
log.access_log	request logging	true

Strict validation at load time: unknown YAML keys are rejected, cache.upstream must be http(s):// without userinfo/query/fragment, router.upstreams entries must be bare host:port, durations and sizes must be positive, cache.dir must be absolute. When set, cache.browser_max_age must be a positive duration; omitting it keeps the 1-year default.

Behavioural reference

• Cache key = URI path without query. Root and .. paths → 404. Only GET/HEAD; everything else → generic 404.
• Range requests are served from cache files as single-range 206; invalid or multi-range requests fall back to a full 200.
• A HEAD on a fresh object performs the single origin fetch and fills the cache; the following GET is a HIT.
• Concurrent misses on the same URI collapse to one origin fetch (singleflight). A client disconnecting mid-miss does not abort the fill.
• On origin 5xx/timeout with a stale (older than fresh_for) copy in cache, the stale copy is served with X-Mocache: STALE.
• Eviction runs in the background: first everything idle for more than inactive, then least-recently-accessed objects until max_size_bytes is met.
• The router pins every URI to a node by consistent hash (Ketama-style, sha256, 160 vnodes per node) and fails over clockwise to the next node on transport errors or 502/503/504. Adding or removing nodes only remaps the removed node's share of URIs.

Multi-node operation (four cache nodes)

The standard topology is a StatefulSet of four cache pods plus two stateless router replicas (see deploy/kubernetes/):

• Each cache pod runs one mocache process (role single) serving cache on :8080 and agent on :8081.
• A headless service (clusterIP: None) gives every pod a stable DNS name: mocache-cache-0.mocache-cache … mocache-cache-3.mocache-cache. These names — not raw pod IPs — go into router.upstreams (port 8080) and panel.agent_urls (port 8081). Because the StatefulSet keeps pod names stable across reschedules, the consistent-hash ring stays intact and no rehashing/discovery service is needed.
• The router deployment (2 replicas) holds the four pod DNS names in a ConfigMap and is stateless; panel holds the four agent URLs.
• There is deliberately no dynamic discovery: pod IPs are ephemeral and would silently shuffle the ring.

Tests

go test ./...                     # unit tests
go test -race ./...               # unit tests with race detector
go test -tags integration ./...   # integration tests (Fake-S3)
make test-all                     # vet + fmt + all of the above + release build check
make release-check                # fmt, vet, static build, ldd/file check

Migration from nul/mocache-python

mocache implements the mocache-python/nginx behaviour as its test specification: same cache keys, same header policy, same retention defaults (24 h idle, 48 h freshness, 20 GiB), same generic error responses. Differences are documented in the plan (docs/plans/2026-11-01-go-rewrite.md):

• Metrics are collected in-process instead of parsing access logs.
• The router is a Go consistent-hash implementation instead of nginx hash $uri consistent, with the same two-try failover semantics.
• No conditional revalidation (documented deviation): client If-None-Match/If-Modified-Since headers are ignored — answers are always full 200 bodies, never 304. Freshness is purely time-based (cache.fresh_for, stale-on-error); the origin ETag is stored per object but not used for revalidation. This matches the immutable public-media use case; adding 304/revalidation would be a new feature, not a parity fix.
• Agent uptime window is "since process start".

The production migration to the ieji cluster follows the gated canary runbook in docs/ieji-canary.md; nginx rollback stays available until the final gate.