PilotFoxDocs

Requirements

The server, the Cloudflare account and the optional services a self-hosted PilotFox needs, and the Postgres it runs itself.

The server

  • Linux with Docker Engine and the Docker Compose plugin. Tested with Docker Engine 29.9.2 and Compose 5.6.1. Check with docker version and docker compose version.
  • A checkout of this repository. docker-compose.yml builds the API image from backend/ and mounts the ClickHouse configuration from backend/docker/clickhouse/. No release images are published yet (Upgrades). The other images are pulled, Postgres's from ghcr.io.
  • bash and openssl for backend/scripts/setup.sh, which writes the env file with generated secrets, or the Doppler CLI to keep the secrets in Doppler instead; git to update.
  • No inbound port. The server only connects out (below).
  • One API container. Its event spool lives in a volume that two processes must never share, so never scale it with --scale api=2.

Some sandboxed or rootless Docker setups cap the open files of containers below 262 144, and ClickHouse then cannot start ("error setting rlimit type 7"). Lower ulimits.nofile for clickhouse in a docker-compose.override.yml next to docker-compose.yml (Compose reads it together with the main file), never in docker-compose.yml itself.

Memory

The default memory limits suit a server with 8 GB of RAM or more: 3 GB for the API container and 2 GB for ClickHouse, while Postgres, its backup service, Valkey and cloudflared have no limit (a few hundred MB together at small scale). A container that reaches its limit is killed and restarted by Docker; for the API that loses the events still in memory (normally the last second) and every request in flight. Docker allows limits that add up to more than the RAM, and the host's own out-of-memory killer then picks a victim.

The API's largest consumer is session replay: decoding recordings uses up to 4 worker threads of about 560 MiB each. Replay only runs with R2 configured and replay turned on for a site. Profiles for smaller servers, set in .env (the repository marks them provisional until a target server is known):

ServerSettings in .envNotes
8 GB or morenone (API_MEM_LIMIT=3g, CLICKHOUSE_MEM_LIMIT=2g)The defaults.
4 GB, replay onAPI_MEM_LIMIT=2560m, CLICKHOUSE_MEM_LIMIT=1g, CLICKHOUSE_QUERY_MAX_MEMORY=536870912Do not go below 2560m for the API with replay on. Replay's worst case and a heavy ClickHouse query at the same moment can still exhaust the host.
2 to 4 GB, replay offAPI_MEM_LIMIT=1g, CLICKHOUSE_MEM_LIMIT=1g, CLICKHOUSE_QUERY_MAX_MEMORY=536870912R2 unset, or replay off on every site: the replay decoders never start.

ClickHouse caps itself at 0.8 times its container limit, and each dashboard query or export at CLICKHOUSE_QUERY_MAX_MEMORY (1 GiB by default); keep the second well under the first, hence 512 MiB with a 1 GB limit. Never put --max-old-space-size in API_NODE_OPTIONS: size the API with API_MEM_LIMIT. Valkey has no memory cap on purpose (every key expires); watch it with docker compose exec valkey valkey-cli info memory.

Disk

Plan disk for the data, which grows with your traffic: events in ClickHouse, accounts and settings in Postgres, both on the server. Restores and restore drills need free disk about the size of the data, because the restored copy sits next to the live one until it is swapped in or dropped, and a restore also downloads the Postgres dump to the API's volume. With R2 set, Postgres also keeps its write-ahead log until it is in R2: about 1 GB normally, and at most 4 GiB more while R2 cannot be reached (PGBACKREST_ARCHIVE_PUSH_QUEUE_MAX; lower it on a small disk). Container logs rotate at 10 MB × 5 files per service.

Postgres

Nothing to set up: docker-compose.yml runs PostgreSQL 17 itself, the postgres service, from the image ghcr.io/saiakashneela/postgres-ssl-pgbackrest:pg17 (Postgres 17 with TLS and pgBackRest). On its first start it creates the database app with the user app in the volume postgres-data, with the password POSTGRES_PASSWORD from your secrets, and the API migrates it.

WhatWhy
PostgreSQL 17, kept on the pg17 tagThe API image's pg_dump is version 17, and pg_dump refuses to dump a server of a newer major version: on 18 every backup, drill and restore from Admin → Backups would fail. The tag is rebuilt every week with Postgres's patch releases.
TLS between the API and Postgres (require)The image always serves TLS, so the password and the data never cross Docker's network in clear. The certificate names only localhost, so the API does not check it; only your own services share that network.
No published portPostgres is on the internal stores network for the API, and on postgres-egress to send its backups to R2. For administration: docker compose exec postgres psql -U app -d app.
Runs as postgresFrom the start, with no Linux capability and a read-only root filesystem; it writes only to its volume and to memory.

With R2 set, Postgres also backs itself up continuously to your bucket (Backups and restore).

Cloudflare

  • A Cloudflare account with your domain on Cloudflare DNS. The web app and the public site are a Worker on the route <your domain>/*, three routes without a Worker send /api/*, /.well-known/* and /maps/* to the tunnel, and the tunnel's published application is the API. See Cloudflare.
  • A plan for the Worker's requests. Static files are free and unlimited; the Worker script runs only for the first load of an app URL, unknown paths and redirects. The Workers Free plan allows 100 000 of those a day; Workers Paid has no daily cap (and is needed for Cloudflare Email Sending to arbitrary recipients).
  • Request bodies above the plan's limit (100 MB on Free and Pro) never reach the server, so an import file larger than that cannot be uploaded.

Network

Inbound: nothing. Outbound, the server needs to reach:

DestinationNeeded for
region1.v2.argotunnel.com and region2.v2.argotunnel.com, port 7844 over TCP and UDP, and DNSThe cloudflared container: the tunnel.
Docker image registries, the npm registry and Debian's package mirrorsPulling the base images and building the API image (docker compose up --build).
download.maxmind.com and the host it redirects downloads toGeoIP.
https://<R2_ACCOUNT_ID>.r2.cloudflarestorage.com (or R2_ENDPOINT)Session replay, imports and backups, from the API, ClickHouse and Postgres.
https://api.cloudflare.comMail through Cloudflare Email Sending.
Your SMTP serverMail over SMTP instead.
https://api.stripe.com, https://live.dodopayments.com, https://test.dodopayments.com, https://api.polar.sh, https://sandbox-api.polar.shRevenue from the payment providers a site connects (test keys use the second host of each pair).
hooks.slack.com, discord.comNew-sale messages to Slack or Discord.
oauth2.googleapis.com, searchconsole.googleapis.comSearch Console (both), and Google sign-in (the first).
github.com, api.github.comGitHub sign-in.

Accounts and services

Cloudflare is required. Each other service turns on features; without it the instance still runs.

ServiceWhat it givesSettingsWithout it
Mail: Cloudflare Email Sending (a Workers Paid plan to send to anyone), or an SMTP serverOpen sign-up, password resets, email sign-in codes, email changes, invitations by mail, emailed reports and new-sale emails.CLOUDFLARE_ACCOUNT_ID and CLOUDFLARE_EMAIL_TOKEN, or SMTP_URL; MAIL_FROM, MAIL_REPLY_TOSign-up is invite-only, and an inviter gets the invite link to pass on.
MaxMind account (free GeoLite2)Visitors' country, region and city, their network (ASN) and the datacenter signal for bot detection.MAXMIND_ACCOUNT_ID, MAXMIND_LICENSE_KEYLocation comes from Cloudflare's headers only; network columns stay empty.
Cloudflare R2 bucket and API tokenSession replay, imports and backups, Postgres's continuous ones included.R2_ACCOUNT_ID, R2_ACCESS_KEY_ID, R2_SECRET_ACCESS_KEY, R2_BUCKETNo replay, no imports, no backups.
Google Cloud OAuth clientSearch Console data per site; optionally Google sign-in.SEARCH_CONSOLE_CLIENT_ID and SEARCH_CONSOLE_CLIENT_SECRET, or GOOGLE_CLIENT_ID and GOOGLE_CLIENT_SECRETThe Search Console pages show the setup steps.
GitHub OAuth appGitHub sign-in.GITHUB_CLIENT_ID, GITHUB_CLIENT_SECRETNo GitHub button on the sign-in page.
Mapbox public tokenMapbox as a map provider, next to the default MapLibre styles.MAPBOX_PUBLIC_TOKENMapLibre maps from OpenFreeMap styles.
DopplerThe secrets kept out of files on the server.backend/scripts/compose-doppler.shA .env file next to docker-compose.yml.

The R2 token needs Object Read & Write on the bucket, and Admin Read & Write as well if backups should maintain their retention rule. Every setting is described in the configuration reference.

On this page