diff --git a/.gitea/workflows/ci.yaml b/.gitea/workflows/ci.yaml index a3cb1aa..5739fae 100644 --- a/.gitea/workflows/ci.yaml +++ b/.gitea/workflows/ci.yaml @@ -21,6 +21,10 @@ jobs: run: python3 -m unittest discover -s infra/backup -p 'test_*.py' -v - name: Check recovery host guard run: bash infra/bootstrap/test-host-guard.sh + - name: Check infrastructure shell syntax + run: | + bash -n infra/bootstrap/bootstrap-ubuntu-24.04.sh infra/bootstrap/host-guard.sh \ + infra/bootstrap/verify.sh infra/bootstrap/verify-helper.sh infra/deploy/install-helper.sh - name: Run rollback helper tests run: python3 -m unittest discover -s infra/deploy -p 'test_*.py' -v - name: Validate production Compose and Caddyfile diff --git a/docs/operations/ACCESS.md b/docs/operations/ACCESS.md index 26f3470..c33fd9b 100644 --- a/docs/operations/ACCESS.md +++ b/docs/operations/ACCESS.md @@ -26,9 +26,9 @@ checks. Never print a private key, token, or complete environment file. - Host package installation, service setup, firewall changes, secret creation, backup creation, recovery apply, and deployment require owner approval and a reviewed root command. After the owner installs the root-owned repository - checkout, `infra/deploy/install-helper.sh` can grant only the validated - rollback command through sudoers; that helper is not installed on the audited - VPS. + checkout, `infra/deploy/install-helper.sh` can grant a fixed read-only + verifier and guarded rollback command through sudoers; neither helper is + installed on the audited VPS. - Never connect to other VPS hosts or Synology as part of this deployment. This execution environment is WSL2. The repository is on the Windows-mounted @@ -59,6 +59,7 @@ limitation and do not retrieve it elsewhere. | Role | Operations | |---|---| | `codex` over SSH | Read-only host diagnostics; inspect service health and public listeners | +| Restricted verifier | Runs the root-owned read-only drift check; not installed on the audited VPS | | Restricted rollback helper | Can restore prior local backend/OpenSearch images from a verified backup after an explicit data-compatibility confirmation; not installed on the audited VPS | | VPS owner/root | Bootstrap, package installation, SSH/firewall, secret setup, recovery, and approval of production changes | | Gitea repository writer | Feature branch push and PR creation; protected-main merge needs an approval | diff --git a/docs/operations/CURRENT_STATE.md b/docs/operations/CURRENT_STATE.md index 8f3da6d..18b0ade 100644 --- a/docs/operations/CURRENT_STATE.md +++ b/docs/operations/CURRENT_STATE.md @@ -52,8 +52,9 @@ from planned configuration. - Recovery bootstrap has an explicit `--recovery-target` mode that refuses the inventoried production hostname/IP; its host-identity guard has shell tests. The mode has not been run on a VPS. -- The repository defines a root-owned rollback helper and owner-run installer - with a single-command sudoers allowlist. Neither is installed on the VPS. +- The repository defines a root-owned read-only verifier, rollback helper, and + owner-run installer with exact sudoers command rules. None is installed on + the VPS. - An isolated local Compose smoke test built the backend/OpenSearch images, reached healthy status for OpenSearch, backend, and Caddy, and returned HTTP 200 for `/review` through Caddy. Its containers, networks, and volumes were diff --git a/docs/operations/DECISIONS.md b/docs/operations/DECISIONS.md index f57e17b..32d6056 100644 --- a/docs/operations/DECISIONS.md +++ b/docs/operations/DECISIONS.md @@ -3,7 +3,7 @@ | Date | Decision | Reason and alternatives | Status / limits | |---|---|---|---| | 2026-10-11 | Use Docker Compose on the dedicated Ubuntu VPS | Matches the existing production Compose baseline and workload; Kubernetes adds no demonstrated need | Planned; Docker is not installed | -| 2026-10-11 | Keep `codex` out of sudo and the Docker group | Docker socket access is effectively root; a separate installer can grant only the root-owned rollback helper after checkout ownership and sudoers validation | Helper is defined in Git; not installed on VPS | +| 2026-10-11 | Keep `codex` out of sudo and the Docker group | Docker socket access is effectively root; a separate installer can grant only a fixed read-only verifier and root-owned rollback helper after checkout ownership and sudoers validation | Helpers are defined in Git; not installed on VPS | | 2026-10-11 | Put Caddy and Search API on a proxy network and OpenSearch on a separate internal network | Keeps OpenSearch unreachable from the proxy and host; the API has no published host port | Repository Compose target only; not deployed | | 2026-10-11 | Use Caddy in Compose for HTTPS at `akyldash.kg` | Automatic certificate issuance/renewal keeps proxy config small; Nginx with separately managed certificates was considered | Caddy image pinned by digest; requires public ports 80/443 and owner confirmation of Cloudflare Full (strict) | | 2026-10-11 | Mount the review signing key as a Compose secret file | Avoids putting its value in command-line arguments; an environment variable or CLI value is easier to inspect in process metadata | Implemented in repository Compose; not deployed | diff --git a/docs/operations/DEPLOYMENT.md b/docs/operations/DEPLOYMENT.md index 070569d..bb55f5f 100644 --- a/docs/operations/DEPLOYMENT.md +++ b/docs/operations/DEPLOYMENT.md @@ -5,9 +5,9 @@ Production deployment is not enabled or tested on the VPS. An isolated local Compose smoke test passed; the VPS has no Docker, Git, application data, proxy, registry pull configuration, backup process, or -restricted deploy-helper. Do not deploy until the owner has completed the root -bootstrap, provided a protected secret file and a reviewed proxy/firewall -configuration, and a backup and restore have been tested. +restricted verification/rollback helpers. Do not deploy until the owner has +completed the root bootstrap, provided a protected secret file and a reviewed +proxy/firewall configuration, and a backup and restore have been tested. ## First host preparation @@ -23,9 +23,9 @@ configuration, and a backup and restore have been tested. `/etc/akyldash/production.env` plus `/etc/akyldash/review_secret` outside Git. The secret file must be root-readable only. 3. After reviewing the root-owned checkout, run - `sudo bash infra/deploy/install-helper.sh`. It installs the backup reader and - rollback helper and adds a sudoers rule for that helper only; it does not - grant general sudo or Docker access. + `sudo bash infra/deploy/install-helper.sh`. It installs the backup reader, + read-only verifier, and rollback helper with exact sudoers command rules; it + does not grant general sudo or Docker access. 4. Confirm DNS points through Cloudflare to this origin, then have the owner review firewall rules and open only TCP `80` and `443` in addition to the existing SSH access. The Compose Caddy service obtains and renews its HTTPS diff --git a/docs/operations/RECOVERY.md b/docs/operations/RECOVERY.md index 96b86ca..03ed5e0 100644 --- a/docs/operations/RECOVERY.md +++ b/docs/operations/RECOVERY.md @@ -21,9 +21,9 @@ run recovery commands against the existing production by accident. recovery mode refuses the inventoried production hostname/IP and writes a root-only marker at `/etc/akyldash/recovery-target`. It installs Docker Engine, Compose, Git, Docker log limits, and target directories. -4. Install the limited rollback helper with +4. Install the limited read-only verifier and rollback helper with `sudo bash infra/deploy/install-helper.sh`. It validates the root-owned - checkout and gives `codex` access only to that helper. + checkout and gives `codex` access only to those fixed commands. 5. Configure DNS to point through Cloudflare to the recovery host and allow the owner to review/open TCP `80` and `443`. The production Compose Caddy service obtains its origin certificate automatically. Verify Cloudflare Full diff --git a/infra/bootstrap/verify-helper.sh b/infra/bootstrap/verify-helper.sh new file mode 100644 index 0000000..98247ce --- /dev/null +++ b/infra/bootstrap/verify-helper.sh @@ -0,0 +1,14 @@ +#!/usr/bin/env bash +set -euo pipefail + +checkout=/opt/akyldash/current +if [[ $# -ne 0 || $EUID -ne 0 || ! -d $checkout || -L $checkout || $(stat -c %U "$checkout") != root ]]; then + echo 'Refusing verification: the approved root-owned checkout is unavailable.' >&2 + exit 1 +fi +unsafe=$(find "$checkout" -xdev \( ! -user root -o -perm /022 \) -print -quit) +if [[ -n $unsafe ]]; then + echo 'Refusing verification: the checkout contains a writable or non-root-owned path.' >&2 + exit 1 +fi +exec /bin/bash "$checkout/infra/bootstrap/verify.sh" --remote diff --git a/infra/bootstrap/verify.sh b/infra/bootstrap/verify.sh index e472eba..115b367 100755 --- a/infra/bootstrap/verify.sh +++ b/infra/bootstrap/verify.sh @@ -3,7 +3,7 @@ set -euo pipefail if [[ ${1:-} != --remote ]]; then ssh_alias=${1:-akyldash_vps} - exec ssh -F "${HOME}/.ssh/config" -o BatchMode=yes "$ssh_alias" 'sudo -n bash -s -- --remote' < "$0" + exec ssh -F "${HOME}/.ssh/config" -o BatchMode=yes "$ssh_alias" 'sudo -n /usr/local/sbin/akyldash-verify' fi if [[ ${EUID} -ne 0 ]]; then @@ -129,14 +129,15 @@ for endpoint in $udp_listeners; do failed=1 done -sudoers_rule='codex ALL=(root) NOPASSWD: /usr/local/sbin/akyldash-rollback *' -if [[ -x /usr/local/sbin/akyldash-rollback && $(stat -c %U:%a /usr/local/sbin/akyldash-rollback) == root:750 && \ +expected_sudoers=$'codex ALL=(root) NOPASSWD: /usr/local/sbin/akyldash-verify\ncodex ALL=(root) NOPASSWD: /usr/local/sbin/akyldash-rollback *' +if [[ -x /usr/local/sbin/akyldash-verify && $(stat -c %U:%a /usr/local/sbin/akyldash-verify) == root:750 && \ + -x /usr/local/sbin/akyldash-rollback && $(stat -c %U:%a /usr/local/sbin/akyldash-rollback) == root:750 && \ -f /usr/local/lib/akyldash/archive.py && $(stat -c %U:%a /usr/local/lib/akyldash/archive.py) == root:644 && \ -f /etc/sudoers.d/akyldash-codex && $(stat -c %U:%a /etc/sudoers.d/akyldash-codex) == root:440 && \ - $(cat /etc/sudoers.d/akyldash-codex) == "$sudoers_rule" ]]; then - echo 'PASS limited rollback helper and sudoers rule present' + $(cat /etc/sudoers.d/akyldash-codex) == "$expected_sudoers" ]]; then + echo 'PASS read-only verifier, rollback helper, and exact sudoers rules present' else - echo 'FAIL limited rollback helper or sudoers rule missing/different' + echo 'FAIL restricted helper or sudoers installation missing/different' failed=1 fi if [[ -f /opt/akyldash/deployed-image.txt ]]; then diff --git a/infra/deploy/install-helper.sh b/infra/deploy/install-helper.sh index ec14933..258af93 100644 --- a/infra/deploy/install-helper.sh +++ b/infra/deploy/install-helper.sh @@ -9,7 +9,8 @@ fi checkout=/opt/akyldash/current rollback_source=$checkout/infra/deploy/rollback.py backup_source=$checkout/infra/backup/archive.py -if [[ ! -d $checkout/.git || ! -f $rollback_source || ! -f $backup_source ]]; then +verify_source=$checkout/infra/bootstrap/verify-helper.sh +if [[ ! -d $checkout/.git || ! -f $rollback_source || ! -f $backup_source || ! -f $verify_source ]]; then echo "Install the approved repository checkout at $checkout first." >&2 exit 1 fi @@ -32,7 +33,8 @@ if grep -R -nE '(^|[[:space:],])codex([[:space:],]|$)' /etc/sudoers /etc/sudoers echo "Existing codex sudoers rules need owner review before installing the helper." >&2 exit 1 fi -if [[ $(stat -c %U "$checkout") != root ]] || find "$checkout" -xdev \( ! -user root -o -perm /022 \) -print -quit | grep -q .; then +unsafe=$(find "$checkout" -xdev \( ! -user root -o -perm /022 \) -print -quit) +if [[ $(stat -c %U "$checkout") != root || -n $unsafe ]]; then echo "The checkout must be root-owned and not group/world-writable." >&2 exit 1 fi @@ -41,7 +43,7 @@ if ! command -v visudo >/dev/null 2>&1; then exit 1 fi -for path in /usr/local/sbin/akyldash-rollback /usr/local/lib/akyldash/archive.py /etc/sudoers.d/akyldash-codex; do +for path in /usr/local/sbin/akyldash-verify /usr/local/sbin/akyldash-rollback /usr/local/lib/akyldash/archive.py /etc/sudoers.d/akyldash-codex; do if [[ -L $path ]]; then echo "Refusing symlink at $path." >&2 exit 1 @@ -50,10 +52,11 @@ done install -d -o root -g root -m 0755 /usr/local/lib/akyldash install -o root -g root -m 0644 "$backup_source" /usr/local/lib/akyldash/archive.py +install -o root -g root -m 0750 "$verify_source" /usr/local/sbin/akyldash-verify install -o root -g root -m 0750 "$rollback_source" /usr/local/sbin/akyldash-rollback sudoers_file=/etc/sudoers.d/akyldash-codex -expected='codex ALL=(root) NOPASSWD: /usr/local/sbin/akyldash-rollback *' +expected=$'codex ALL=(root) NOPASSWD: /usr/local/sbin/akyldash-verify\ncodex ALL=(root) NOPASSWD: /usr/local/sbin/akyldash-rollback *' created=false if [[ -e $sudoers_file ]]; then if [[ $(stat -c %U:%a "$sudoers_file") != root:440 || $(cat "$sudoers_file") != "$expected" ]]; then @@ -82,4 +85,4 @@ if ! visudo -c -f /etc/sudoers; then echo "Sudoers validation failed; the newly added rule was removed." >&2 exit 1 fi -echo "Installed root-owned rollback helper; codex receives no general sudo or Docker access." +echo "Installed read-only verification and guarded rollback helpers; codex receives no general sudo or Docker access." diff --git a/infra/inventory/production.yml b/infra/inventory/production.yml index 2beb0cf..0c3917c 100644 --- a/infra/inventory/production.yml +++ b/infra/inventory/production.yml @@ -70,6 +70,9 @@ target: restricted_helper: path: /usr/local/sbin/akyldash-rollback status: repository code and owner-run installer exist; not installed on VPS + read_only_verifier: + path: /usr/local/sbin/akyldash-verify + status: repository code and owner-run installer exist; not installed on VPS checks: - infra/bootstrap/verify.sh akyldash_vps - docker compose -f deploy/production/compose.yaml ps