Upgrades

An update brings the checkout and the API image to the new version and recreates the containers that changed; the API applies any new database migrations when it starts. Every command runs from the repository root.

Before you updateCopy Anchor Link

• Take a backup. Migrations only run forward: there is no command that undoes one. Run a backup from Admin → Backups (Run backup now) and wait for it to succeed. See Backups and restore.
• Redeploy the Worker too when frontend/ changed, so the web app matches the API (Cloudflare, step 2).

Updating from sourceCopy Anchor Link

The default, and today the only way: the API image is built on the server (API_IMAGE=mine-api:local).

git pull
backend/scripts/setup.sh https://analytics.example.com you@example.com   # with .env: the URL and email you first gave it
docker compose pull --ignore-buildable
CLOUDFLARE_TUNNEL_TOKEN=$(cat ~/.pilotfox/tunnel-token) docker compose up -d --build --wait --remove-orphans

With Doppler, run the two compose commands through backend/scripts/compose-doppler.sh instead.

• setup.sh only appends the keys your .env lacks, such as a secret a newer version of the script generates, and changes no existing line. Run it before any docker compose command: every command fails with required variable ... is missing a value while a secret the compose file requires is missing.
• pull --ignore-buildable pulls the images of the services compose does not build (Postgres and pgbackrest, ClickHouse, Valkey, cloudflared) and skips api. The Postgres image's pg17 tag is rebuilt every week with Postgres's patch releases and Debian's security fixes, so an update often recreates postgres and pgbackrest: a fast shutdown, then a normal start on the same volume.
• up -d --build --wait builds the API image from the new code, recreates the containers that changed, and waits until every container is healthy.
• Base images. To also fetch newer versions of the images the API's Dockerfile starts from (node:24-trixie-slim, oven/bun:1.3.13), run docker compose build --pull before up.

Running a release imageCopy Anchor Link

The repository's CI workflow is written to build tagged releases (v1.2.3) and push the API image to ghcr.io/<owner>/mine-api as :1.2.3 and :1.2 (a prerelease tag such as v1.2.3-rc.1 only as :1.2.3-rc.1; never :latest; linux/amd64 only). The workflow is kept at backend/ci/github/workflows/ci.yml and runs only once it is moved to .github/workflows/ at the repository root, which has not happened yet: no release image exists today, so update from source as above.

Once one exists, check out the release (so docker-compose.yml, backend/docker/ and frontend/ are the same version as the image), set API_IMAGE in .env, then:

git fetch --tags && git checkout v1.2.3
# in .env: API_IMAGE=ghcr.io/<owner>/mine-api:1.2.3
docker compose pull api postgres pgbackrest clickhouse valkey cloudflared
CLOUDFLARE_TUNNEL_TOKEN=$(cat ~/.pilotfox/tunnel-token) docker compose up -d --no-build --wait --remove-orphans

• Name the services to pull. pull --ignore-buildable skips api even when API_IMAGE names a release image, because the service has a build: section.
• Never up --build with a release image. It would build the API from source too and tag that build with the release image's name, replacing the image you pulled.
• GHCR packages start private. Log in on the server (docker login ghcr.io) before pulling, or make the package public in its GitHub settings; a package made public cannot be made private again.

What happens during the updateCopy Anchor Link

• Migrations run when the API starts, before it accepts requests: Postgres (Drizzle migrations), ClickHouse, and the job queue's own schema. The data-retention settings are applied at the same time. Until migrations finish, the container is not healthy and --wait keeps waiting. Its health check allows a 60-second start period, then 6 failed checks 10 seconds apart: a migration that takes longer than about 2 minutes makes Docker mark the container unhealthy, and --wait exits with container ... is unhealthy. The migration still runs to its end, and the container turns healthy once the API listens: follow it with docker compose logs -f api.
• The old API shuts down cleanly. On stop it finishes in-flight requests and flushes buffered events within 20 seconds; compose gives it 30. Never docker kill it. Compose stops cloudflared first, then the API, then the stores (pgbackrest before Postgres), and gives ClickHouse 60 seconds and Postgres 30 for their own shutdown.
• The API is briefly unavailable while its container is replaced and migrates: requests that arrive meanwhile fail. Update at a quiet hour.
• Restores stay possible across versions in one direction: a restore brings a backup from an older version up to the current one, and refuses a backup taken by a newer version.

Checks after the updateCopy Anchor Link

1. up -d --wait returned without an error: every container is running and healthy. List them, with their images, with docker compose ps.

2. The API answers through the tunnel:

   Copy Text

   curl https://analytics.example.com/api/v1/health

   Copy Text

   {"data":{"ok":true}}

3. The stores and GeoIP are ready (the route is not public; this prints the GeoIP state, ok once both databases are loaded):

   Copy Text

   docker compose exec -T api \
     node -e "fetch('http://127.0.0.1:3001/api/v1/ready').then((r) => r.json()).then((j) => console.log(j.data.geoip))"

4. Admin → Jobs shows no new failures, and Admin → Backups shows the pg_dump version with no warning in red, and the next scheduled backup.

5. Sign in, open a site, and check that new pageviews arrive in the realtime view.

Moving from an external PostgresCopy Anchor Link

From 2026-11-01 to 2026-11-02, docker-compose.yml had no Postgres: the API connected to your own database through DATABASE_URL. Now Postgres is part of the file. Move the data into it once, with the API stopped; the instance is down from the second step to the last (minutes for a small database). The steps use the API image's pg_dump and the Postgres container's own pg_restore, both PostgreSQL 17. With Doppler, run every compose command through backend/scripts/compose-doppler.sh, and in step 3 generate POSTGRES_PASSWORD and PGBACKREST_REPO1_CIPHER_PASS into Doppler instead (64 hex characters each, as the other secrets).

docker compose run --rm --no-deps -T api sh -c 'case "$DATABASE_SSL_MODE" in verify) printf "%b\n" "$DATABASE_CA_CERT" >/tmp/ca.pem; export PGSSLMODE=verify-full PGSSLROOTCERT=/tmp/ca.pem ;; *) export PGSSLMODE="$DATABASE_SSL_MODE" ;; esac; exec pg_dump --format=custom --no-owner --no-acl "$DATABASE_URL"' > pilotfox-postgres.dump

git pull
backend/scripts/setup.sh https://analytics.example.com you@example.com   # appends POSTGRES_PASSWORD and PGBACKREST_REPO1_CIPHER_PASS

docker compose up -d --wait postgres
docker compose exec -T postgres pg_restore -U app -d app --no-owner --no-acl --exit-on-error < pilotfox-postgres.dump
CLOUDFLARE_TUNNEL_TOKEN=$(cat ~/.pilotfox/tunnel-token) docker compose up -d --build --wait --remove-orphans

Everyone keeps their account, organizations, sites, keys and share links; ClickHouse, Valkey and R2 are untouched. The DATABASE_URL, DATABASE_SSL_MODE and DATABASE_CA_CERT lines are not read any more: delete them from your secrets, and your old database once you no longer need it. With R2 set, the pgbackrest service takes a full backup of the moved data at once (Backups and restore).

Moving from the earlier compose filesCopy Anchor Link

Until 2026-11-01 the stack was backend/docker/compose.prod.yml (with Caddy, a web image and a bundled Postgres), compose.external-postgres.yml, and override files for proxies, Cloudflare and tunnels. docker-compose.yml replaces them all. It keeps their project name, mine-analytics, and the names and settings of every volume and network it uses, so a server keeps its data, and no network is recreated.

• From compose.external-postgres.yml with compose.workers.yml (the API behind a tunnel, the web app on a Worker): with a plain .env, move it from backend/.env to the root (mv backend/.env .env); with Doppler, run doppler setup once in the repository root. Then move your database in as above.
• From the bundled Postgres (compose.prod.yml): Postgres keeps its volume mine-analytics_postgres-data, its user and database app and the same Postgres 17, now from an image that adds pgBackRest. Keep POSTGRES_PASSWORD in .env: it is that database's password. If you deleted it, setup.sh writes a new one: start Postgres alone (docker compose up -d --wait postgres) and set it there before starting the rest (docker compose exec postgres psql -U app -d app -c '\password app', with the new value).
• From Caddy in front (any mode that served the web app from the web image): set up the Cloudflare side (Cloudflare, all steps; a tunnel that pointed at http://caddy:80 now points at http://api:3001), then start docker-compose.yml, and close ports 80 and 443 in the firewall.
• Left behind, safe to remove once the new stack runs: --remove-orphans stops and removes the old caddy and geoipupdate containers; the volumes mine-analytics_caddy-data and mine-analytics_caddy-config, the networks mine-analytics_edge and mine-analytics_geoip-egress, the mine-web:local image and any scripts/backup.sh cron jobs stay until you remove them (docker volume rm …, docker network rm …, docker image rm …). Turn on the schedule in Admin → Backups.

After network changesCopy Anchor Link

Since 2026-10-31 the API and the stores share the network stores, which has no route out, instead of backend, and ClickHouse reaches R2 through a network of its own, clickhouse-egress. The first up -d after updating recreates the API and the stores on it; nothing else is needed. The old network stays behind, empty; remove it with docker network rm mine-analytics_backend.

After ClickHouse configuration changesCopy Anchor Link

Removing a ClickHouse system log in backend/docker/clickhouse/config.d/small.xml stops it being written but does not drop a table an existing volume already has; drop it by hand if you want the space back.