Skip to main content

Upgrades

Every TOW image in the stack is pinned to one immutable release tag: TOW_VERSION in .env. Upgrading is changing that one value and letting Compose reconcile. Release tags are never reused; do not use latest or an unattended image updater.

Before you upgrade​

Confirm that a recent managed backup exists, that its encryption identity is available from your secrets manager, and that you have exercised the restore procedure at least once. This is an explicit operator checkpoint: the stack does not create a backup as part of migration.

The update sequence​

cd tow-deployment
git pull                    # updates the kit; each release pins its TOW_VERSION
docker compose pull
docker compose stop backend frontend search-worker email-worker inbound-email-worker migration-worker
docker compose up -d --wait
curl -fsS http://127.0.0.1:8080/api/health   # match your PROXY_HTTP_PORT

(If you prefer to manage the tag yourself, skip git pull and edit TOW_VERSION in .env instead; .env is yours and git pull never touches it. Keep any Compose profiles you use on the pull and up commands; the stop always names the same application services.)

Pulling happens before the stop, so a registry failure leaves the current release online. The explicit stop is the maintenance boundary: it prevents the old backend and workers from writing while the target schema is applied.

What happens during the maintenance window​

With the writers stopped, the one-shot schema-migrate service (running the target backend image) waits for PostgreSQL, takes an advisory lock so no other update can migrate concurrently, validates the database and Alembic lineage, applies migrations, and verifies the resulting schema. The backend starts only after it succeeds, and every backend and worker also re-checks the schema before accepting work.

The bundled proxy stays up throughout and answers for unavailable upstreams with TOW's branded 503 Service Unavailable page and a Retry-After header.

If docker compose up -d --wait fails, do not start services manually:

docker compose ps --all
docker compose logs schema-migrate

Correct the problem with a forward release and rerun pull + up, or restore the verified pre-update backup.

No image-only downgrade

After an incompatible migration, rolling back to an older image is not supported. The way back is the verified backup.

Large deployments: search permission refresh​

Membership, group, or role changes mark the organisation's search-permission cache stale; search workers rebuild it in bounded batches. Users keep their access throughout, and search never over-returns; it may just be slower or temporarily return fewer permitted documents. Before a customer-sized migration or bulk membership change, provide enough search-worker capacity for the acl_profile_repair_queue backlog metric to return to zero. See Search after access changes.

Upgrading a pre-kit deployment​

Deployments created before the deploy kit existed keep working; they own their copies of compose.yaml, .env, and config/tow.yaml, and the update sequence above applies unchanged inside their deployment directory.

To adopt the kit (recommended, so future releases arrive with git pull):

  1. Take and verify a backup.
  2. git clone https://github.com/theonlyworkspace/tow-deployment.git next to the old directory and copy your operator-owned files in: .env, config/tow.yaml, deploy/certs/*, and any compose.override.yaml.
  3. Run ./install.sh once; it keeps every existing value and only fills in keys the current release added.
  4. Stop the old stack from the old directory (docker compose stop, never down --volumes), then docker compose up -d --wait from the kit.

Because the Compose project is named tow in both layouts, the kit reuses the existing tow_* Docker volumes; data is untouched. Verify with docker volume ls before starting if in doubt.

Much older layouts. Deployments that still use the retired layered Compose files (docker-compose.yml + .production.yml + .proxy.yml, with .env.launch / config/tow.launch.yaml) or that need the legacy tow_postgres-style volume names should first complete the one-time adoption steps from the release they are on (see that release's deployment guide; the legacy volume mapping is archived as compose.legacy-volumes.example.yaml in the monorepo). Then adopt the kit as above.

Updating the operations scripts​

The backup, restore, prune, and scheduling scripts ship inside the kit and are version-matched to the release; git pull updates them together with TOW_VERSION. (They are also baked into the backend image under /usr/local/share/tow-ops/ for air-gapped hosts.) Re-running sudo scripts/install-backup-schedule.sh --output-dir <backup root> after an upgrade refreshes the systemd units in place.