Files
akyldash/docs/operations/CI_RUNNER.md
T
agent 00922e6c09
CI / tests-and-images (pull_request) Has been cancelled
Runner smoke test / runner-smoke (pull_request) Has been cancelled
fix: make rootless Docker home writable
2026-10-12 22:18:51 +03:00

9.5 KiB

AkylDash Gitea Actions runner

Scope and isolation

The deployment follows the official Gitea 1.25 act_runner guide for repository-scoped registration. Gitea warns that access to a Docker daemon is root-equivalent; the dedicated rootless daemon is used instead of the production daemon. The pinned binary and checksum are published in the official Gitea download directory.

The runner is reserved to repository admin/akyldash, using the token under https://git.vakanaut.ru/admin/akyldash/settings/actions/runners. Do not use organization or instance runner settings. Pin the official act_runner 0.2.13 x86_64 binary from dl.gitea.com; the installer verifies its SHA-256 before placing it on the host.

CI jobs run in docker.gitea.com/runner-images:ubuntu-24.04 containers with capacity 1, per-job limits of 2 CPU, 3 GiB RAM and 512 processes, and a 15 GiB dedicated ext4 loop-backed Docker data filesystem. The runner account's home directory is root-owned mode 0711, so CI cannot rename the image or mountpoint entries. Docker config remains under .config; registration state and the runner workspace live in .config/act-runner, which is runner-owned. The rootless Docker setup and user service use .config/act-runner as writable HOME, .config as XDG_CONFIG_HOME, and .config/act-runner/.local/share as XDG_DATA_HOME. This keeps Docker config and its systemd user unit in the existing .config tree without granting write access to the protected home. rootless Docker daemon and runner user are separate from production Docker; codex is not added to sudo or Docker groups. Jobs do not receive the rootless Docker socket, the production Docker socket, production environment/secrets, or arbitrary host bind mounts. The runner account has no access to /etc/akyldash or the root-owned application checkout. These are host permission boundaries, not a claim that untrusted project code is safe; review workflow changes and actions before merging them.

The runner service restarts on failure and boot. Systemd caps the runner account's shared slice to 2.5 CPU, 4 GiB RAM and 2048 tasks; the rootless daemon is separately capped to 2 CPU, 4 GiB and 1024 tasks, while each job has the limits above. Runner logs go to a rotated file (10 MiB, 7 copies). A systemd timer checks the runner service and the rootless Docker API each minute; on failure it restarts the fixed runner service. No UFW changes or public metrics ports are needed. Runner cache is disabled.

The label akyldash-ci is used by runner-smoke.yaml on pull requests, pushes to main, and manual dispatch. Existing CI image-build work remains on its current ubuntu-latest target until it has been tested separately against the isolated runner. Production deployment is not automated.

Installation and registration

Review the root script and host audit output before running it. It installs uidmap, dbus-user-session, curl, logrotate, and Docker's rootless extras; creates the dedicated system user and selects the first free, shared 65536-ID range from /etc/subuid and /etc/subgid; reserves 15 GiB for runner images/workspaces; and configures systemd services. It starts only the separate rootless Docker daemon; the runner and health timer stay disabled until repository registration. It does not change SSH/UFW, production Docker configuration, application data, or production secrets. It can be rerun after partial setup: it validates and reuses an existing runner account, subordinate ID mappings, image, and mount unit. An existing image must be exactly 15 GiB and contain ext4; a mismatch stops the installer without resizing or formatting it. The runner account must retain a non-root system UID below UID_MIN, its expected home, primary group, and sole supplementary group. After validating the account, the installer locks its home directory to root ownership and mode 0711; it refuses to rerun while the runner service is active, so it cannot interrupt a job. The mount is started only when it is not already active, and an active mount must be the expected ext4 loop device backed by the runner image with both nosuid and nodev active. Runner-home directories and user configuration files are created in runner-owned subdirectories. Root creates only the first .config entry beneath the already locked home; subsequent directory and config writes run as akyldash-runner. The installer rejects static symlinks and validates ownership and mode after setup. This also normalizes the 0755 intermediate directories left by the first installer version. Conflicting files or state stop the installer for manual inspection. A new image requires at least 25 GiB free on /. Image creation opens each parent component without following symlinks, creates the image exclusively relative to the opened directory, and passes the open file descriptor to mkfs.ext4; validation checks the same inode and rejects path replacement. Systemd mounts the image under the root-owned home entry, which the runner cannot replace after validation.

If the reviewed checkout is not present at the inventoried path, first verify that the target is absent and clone the reviewed main there. Do not overwrite an existing checkout.

sudo test ! -e /opt/akyldash/current
sudo git clone --branch main \
  --single-branch https://git.vakanaut.ru/admin/akyldash.git \
  /opt/akyldash/current

From that root-owned reviewed checkout on the VPS:

sudo bash /opt/akyldash/current/infra/runner/install-runner.sh

Then, as a repository owner, open repository Settings → Actions → Runners, create a repository-level runner, and copy its registration token locally. Do not send the token in chat or save it in the checkout. Register interactively in a terminal so it is not placed in shell history or process arguments:

sudo -u akyldash-runner env HOME=/var/lib/akyldash-runner/.config/act-runner \
  /usr/local/bin/act_runner \
  --config /etc/akyldash-runner/config.yaml register

Enter https://git.vakanaut.ru/, the token, runner name akyldash-ci-vps-01, and label akyldash-ci:docker://docker.gitea.com/runner-images:ubuntu-24.04 when prompted. The config sets the label and capacity; verify them in the repository runner settings. The token is saved only in the mode-0700 runner runtime directory, outside Git. Start/enable the runner after registration:

sudo systemctl enable --now akyldash-act-runner.service
sudo systemctl enable --now akyldash-runner-healthcheck.timer
sudo systemctl status akyldash-act-runner.service
sudo systemctl status akyldash-runner-healthcheck.timer

The interactive command's terminal behavior should be confirmed before pasting the token. If it echoes input, stop and use the repository settings to revoke that token; do not put it in command-line arguments or environment variables.

Validation

  1. Confirm the runner is Online in the repository runner settings, with the expected name and akyldash-ci label.
  2. Confirm Runner smoke test runs for the open PR (or dispatch it manually). It checks that production sockets, files, environment secrets, and writable application paths are not present in the job container.
  3. Confirm the run reports the expected runner, succeeds, and the runner service and health timer remain active.
  4. Only after the repository owner reviews this boundary should CI jobs be moved to akyldash-ci. Do not add a production deploy workflow or secrets to this runner until a separate restricted deploy mechanism is reviewed.

Upgrade

Update runner_version and its expected official SHA-256 together in infra/runner/install-runner.sh, review upstream release compatibility with the installed Gitea, and open a PR. Do not replace the binary while a job is active. Stop the runner, install/verify the new binary, then start it and confirm Online plus a smoke run. Preserve /var/lib/akyldash-runner/.config/act-runner/.runner.

Recovery and rollback

The runner is disposable: production data and secrets are not on its filesystem. For a runner process failure, inspect systemctl status and journalctl -u akyldash-act-runner.service; restart the fixed runner service. If installation stops after creating or enabling the mount unit, inspect the image and mount with stat, blkid, findmnt, and losetup, then rerun the reviewed installer only when the image is 15 GiB ext4 and the active mount is backed by it. The installer recognizes that state and does not format or restart the mounted filesystem. If size, filesystem, unit contents, or backing device differ, stop and resolve the state manually; never format the existing image as a retry. For lost/corrupt registration state, stop the service and obtain a fresh repository-level token from Settings → Actions → Runners. Preserve the 15 GiB data image and state for diagnosis; re-register only after the repository owner confirms/revokes the old runner in Gitea. The images/cache can be rebuilt from the workflow. Never format or remove an existing runner store as an automatic recovery step.

Status

The root installer and registration are incomplete. The owner reports that the 15 GiB ext4 loop-backed mount is active on /dev/loop0 after manually running systemctl daemon-reload, enable, and start when the first installer run stopped at mount activation. This PR has not changed the VPS and does not claim that remaining runner components are installed, a repository token is available, the runner is Online, or a workflow has run.