Production deployment
Compose runs Caddy, the Search API, and a private OpenSearch node. Only Caddy
publishes ports 80 and 443; backend and OpenSearch have no host ports. The
OpenSearch security plugin is disabled to match the current API client, so keep
it on the private network.
Host access and configuration
The production environment file is /etc/akyldash/production.env; the review
secret is /etc/akyldash/review_secret. Both must stay outside the checkout,
root-owned, and mode 0600. DATA_ROOT must be an absolute path. The codex
account is intentionally outside the sudo and docker groups because access
to the Docker socket grants root-equivalent control. Run host operations with
the narrowly required sudo command; do not use docker as codex directly.
After reviewing the host audit and confirming these files do not already exist,
the owner can create them from the approved checkout with
sudo bash infra/deploy/prepare-production-config.sh. The script refuses to
overwrite existing configuration, installs the tracked env example with mode
0600, and generates a random review secret with mode 0600. It does not
start Compose or create volumes. Review and set DATA_ROOT in the protected
env file before deployment.
Set DATA_ROOT to the approved data directory and ensure the normalized corpus
is at $DATA_ROOT/minjust-normalized/. The first host setup is documented in
docs/operations/DEPLOYMENT.md; the
root bootstrap installs packages and creates directories, but does not deploy
the application or alter SSH, firewall, user groups, or sudoers. The restricted
akyldash-compose helper allows only root-owned Compose config and status;
service start/stop remains an owner-run root operation.
The repository Gitea Actions runner is an isolated CI worker described in
docs/operations/CI_RUNNER.md. It has no
production Docker socket, production env files, or application checkout. CI
does not deploy or change production; all deployment remains owner-run.
First deployment and upgrades
Run commands from the root-owned checkout at /opt/akyldash/current. Render
and review Compose before changing services:
cd /opt/akyldash/current
sudo docker compose --env-file /etc/akyldash/production.env -f deploy/production/compose.yaml config --quiet
On a clean first deployment, infra/backup/archive.py create reports that no
backup is required when all persistent mounts are empty and no Compose
containers exist. Continue without a backup in that case. If any persistent
data exists, create and verify a backup before changing schema, images, or
starting a new application version. This also applies when containers are
absent, including an initial installation where the normalized corpus was
preloaded under DATA_ROOT; create the archive before the first up. When
named volumes do not exist, pass --initial-deployment; this is accepted only
without Compose containers and when DATA_ROOT contains only
minjust-normalized/. During upgrades, missing volumes are an error because
they can indicate lost production data. The archive tool also refuses
ambiguous storage state, running containers, and incomplete Compose service
containers. See
docs/operations/BACKUP.md for the exact
commands. Do not remove an older usable backup to make room; retain the
pre-change archive until the new deployment and a newer verified backup are
confirmed.
For an initial clean installation, start the stack only after config review:
sudo docker compose --env-file /etc/akyldash/production.env -f deploy/production/compose.yaml up -d --build
sudo docker compose --env-file /etc/akyldash/production.env -f deploy/production/compose.yaml ps
curl -fsS https://akyldash.kg/review >/dev/null
Before this step, confirm DNS and have the owner review firewall rules for TCP
80 and 443. Caddy manages the origin certificate; confirm HTTPS before
setting Cloudflare to Full (strict). On the initial start, create the admin
account interactively:
sudo docker compose --env-file /etc/akyldash/production.env -f deploy/production/compose.yaml exec backend python -m search.api --create-admin
The account database is $DATA_ROOT/search-users.sqlite3 and is included in
the data backup. Admin password reset is also interactive:
sudo docker compose --env-file /etc/akyldash/production.env -f deploy/production/compose.yaml exec backend python -m search.api --reset-admin-password
After the initial OpenSearch index is loaded and validated, the admin account
is created, and /review works over HTTPS, create and verify the required
post-setup baseline backup before treating deployment as complete. It captures
the initialized index, account database, and Caddy certificate data. Follow
the exact stop/create/verify/start sequence in
docs/operations/BACKUP.md.
Backup, restore, and rollback
Backups are cold and require downtime. Stop services, create the archive, then
verify both its checksum and contents before proceeding. Do not run an upgrade
or schema change until verification succeeds. Keep archives root-only under
/var/backups/akyldash and copy them to separately managed encrypted storage.
Restore is a staged operation on a marked replacement host; it does not
overwrite production. For application-only rollback, use the guarded helper
with a verified pre-release backup and explicit data-compatibility review.
Follow the procedures in BACKUP.md,
ROLLBACK.md, and
RECOVERY.md.
Automated production deployment is not enabled. Deployments remain owner-run; record the release and verify service health before considering it complete.