478 lines
31 KiB
Markdown
478 lines
31 KiB
Markdown
# Infrastructure current state
|
|
|
|
## Windows ACL SID resolution — 2026-11-03
|
|
|
|
- Read-only PowerShell 5.1 inspection found that display-name conversion of
|
|
ACL entries can throw `IdentityNotMappedException` even though the ACL itself
|
|
is readable. The installer and external-receiver guards now enumerate access
|
|
rules directly as `SecurityIdentifier` values and read owners as SIDs; unknown
|
|
principals remain visible to the existing fail-closed policy. Regression
|
|
coverage exercises synthetic unmapped SIDs through the actual installer guard
|
|
functions. These repository changes are pending review; no Windows ACL,
|
|
installer, task, VPS, or production state was changed. The native PowerShell
|
|
harness still requires execution on Windows before this behavior is confirmed
|
|
in that runtime.
|
|
|
|
## External deployment runtime binding — 2026-11-03
|
|
|
|
- PR #59 is merged to `main`; it contains the root-only read-only status
|
|
endpoint and exact-literal SSH dispatcher for `status` and `deploy`. The
|
|
status path bounds the Gitea run list to ten entries, verifies the selected
|
|
run/workflow/current `main`,
|
|
validates artifact metadata and ZIP bytes, and returns only a release tuple.
|
|
This unmerged change adds a two-SHA compatibility gate: the fixed root helper
|
|
compares protected Compose/Caddy bytes and installed release policy with
|
|
Gitea content pinned to the release SHA, verifying each Git blob hash and
|
|
size. This permits application-only release B after bootstrap A without
|
|
advancing the root checkout or executing new root code. A runtime config or
|
|
policy change fails closed and requires a reviewed bootstrap update. The
|
|
Windows response state is `prepared`, never `candidate`; the `--once` task
|
|
stores a protected local tuple and cannot call deploy. A separate disabled
|
|
auto-apply task and config gate require the tuple to match a fresh status
|
|
response from a previous poll. Status and apply retain current-main,
|
|
successful CI/release provenance, descriptor digest, external backup,
|
|
cutover, canary, and rollback checks. The receiver repeats the config check
|
|
before journaling/stopping services and again after backup immediately before
|
|
switching refs.
|
|
- Windows client предназначен для подготовки и auto-apply; обе задачи по умолчанию
|
|
выключены, целевой путь фиксирован: `D:\Dev\akyldash`, SSH alias —
|
|
`akyldash_release`. Примеры config и Task Scheduler XML также выключены; задачи
|
|
запускаются от интерактивного пользователя с минимальными правами, не допускают
|
|
параллельных запусков и не будят компьютер. Windows task, SSH key, VPS command,
|
|
helper и trigger не устанавливались и не запускались. Путь canonical
|
|
`C:\ProgramData\Akyldash\release-trigger.json` известен, но содержимое и
|
|
enabled-состояние не читались. Метаданные Windows подтверждают
|
|
`C:\Python313\python.exe`, действительную Authenticode-подпись Python Software
|
|
Foundation и защищённые ACL интерпретатора.
|
|
- Linux fixtures прошли для Python import и command-path poisoning, включая очистку
|
|
унаследованных `GIT_*` и фиксацию путей `ssh.exe`/`git.exe`/`powershell.exe`.
|
|
Installer получает ограниченный bundle из точной вершины `main`: упаковщик
|
|
читает Git objects commit, tree и allowlist-файлов; elevated installer проверяет
|
|
SHA-256 ZIP и цепочку `commit → tree → файлы` из одного read-locked потока до
|
|
записи и не запускает Git. ACL проверяются явными масками мутаций и подмены;
|
|
текущая учётная запись не может владеть кодовыми путями, а право создать
|
|
соседнюю папку не считается подменой предка. Windows ACL/install проверки пока
|
|
только прошли код-ревью; installer и launcher на Windows не запускались.
|
|
Production CD остаётся выключенным.
|
|
- Remaining owner prerequisites are installation of the reviewed root helper,
|
|
exact forced-command/SSH host-key setup, a protected Windows client and its
|
|
private state/config ACLs, private D: backup ACLs, and all previously listed
|
|
backup/restore and production readiness gates. Neither the status endpoint nor client advances
|
|
the host checkout or updates root helper code.
|
|
|
|
- Read-only Gitea check for fresh `main` SHA
|
|
`f19faf628e67015febac22d0ec93ce4782caf4bf`: CI run #706 succeeded; release
|
|
run #708 failed at the explicit registry credential-presence check because
|
|
`REGISTRY_USERNAME` and `REGISTRY_PASSWORD` are absent; handoff run #709 was
|
|
skipped. No secret values were read. Image publication remains owner-blocked.
|
|
|
|
Owner decision recorded 2026-10-26: a complete backup must not be stored on
|
|
the VPS, including temporarily. The local archive/full cold-backup commands in
|
|
`BACKUP.md` are superseded and unauthorized. PR #59 now contains a proposed
|
|
Windows-controlled external stream receiver and root-side deploy/rollback
|
|
code. The Windows rerun passed 12 synthetic client checks and 8 protocol
|
|
checks, plus a disposable real-NTFS fixture for the private owner ACL. The
|
|
production D: path, SSH connection, VPS installation, full backup, and
|
|
separate-target restore remain unverified. No verified external backup or
|
|
separate-target restore exists, so production deployment remains blocked by
|
|
the backup gate.
|
|
|
|
## Code-only deployment work — 2026-10-31
|
|
|
|
- Work is being prepared in the existing PR #59 branch from the then-current
|
|
`main`; no merge, VPS operation, live release, or workflow dispatch was
|
|
performed for this work.
|
|
- The release descriptor now binds both the release run and its successful CI
|
|
source run. Host-side source verification uses the installed `gitea-api`
|
|
helper contract; the PR carries the inspected helper fixture plus a bounded
|
|
binary-download patch. The installer verifies the fixture hash before
|
|
applying the patch.
|
|
- Backup bytes are streamed through a bounded SSH protocol to the fixed
|
|
Windows destination. Root release state stores only a secret-free backup
|
|
identity. The rollback handshake requires the exact previously pinned
|
|
archive and creates a fresh external backup before image references change.
|
|
Local VPS archive paths fail closed.
|
|
- Review fixes in this PR require Gitea `status=completed` with
|
|
`conclusion=success`, resolve `main` from `/opt/akyldash/current`, write a
|
|
durable pending journal before stopping services, and reject an already
|
|
applied release before downtime. The Windows client checks D: capacity,
|
|
NTFS/path/ACL owner and allowed entries, and the local ignore rule before
|
|
SSH; it requires 100 GiB plus 1 MiB free. The Windows rerun passed 12
|
|
synthetic client checks, 8 protocol checks, and a disposable real-NTFS
|
|
private owner-ACL fixture; actual D:, SSH, and VPS remain unverified.
|
|
- Receiver cleanup after a post-publication ACL failure is identity-bound to
|
|
the owned staging directory and permits only the expected archive files. A
|
|
replaced, reparse, or unexpected target is left for manual review without an
|
|
ACK; this path is covered by local fixtures and the Windows rerun.
|
|
- Synthetic tests cover archive/protocol integrity, helper download rules,
|
|
artifact/run provenance, receiver publication, release schema, and rollback
|
|
refusal. The 12 synthetic Windows client checks, 8 protocol checks, and
|
|
disposable real-NTFS private owner-ACL fixture passed. The actual D: path,
|
|
SSH alias, Gitea helper against
|
|
the live instance, root helper installation, artifact redirect, production
|
|
backup/restore, and actual release/rollback remain unverified.
|
|
- Owner bootstrap blockers remain: verify a root-controlled SSH route for the
|
|
fixed commands using existing host-key checking; provision only the
|
|
necessary read-only Gitea token, exact redirect origin if needed, and
|
|
approved RU/KY canaries; confirm registry pull and retention policy; test
|
|
the Windows client and a separate-target restore. No new key, sudo rule,
|
|
Docker access, credential, or VPS setting is added by the code change.
|
|
|
|
Read-only deployment recheck: 2026-10-25. Source: SSH as `codex`, direct-origin
|
|
and public HTTPS probes, local `main` checkout, and Gitea API. No VPS files,
|
|
services, firewall, containers, volumes, production data, or credentials were
|
|
changed or inspected. Root-only Docker state, UFW rule details, protected file
|
|
metadata, current backup/restore validity, and the live corpus/index remain
|
|
unverified. The 2026-10-18 inventory below is a historical snapshot where it
|
|
conflicts with this dated recheck.
|
|
|
|
## Latest read-only recheck — 2026-10-25
|
|
|
|
- `https://akyldash.kg/review` returned HTTP 200 with valid TLS through the
|
|
public Cloudflare edge and through direct HTTPS to origin `206.223.249.118`.
|
|
This verifies that route only; it does not prove search or index readiness.
|
|
- TCP listeners now include 22, 80, and 443. The older no-listener/Cloudflare
|
|
521 observations below are historical, not current state.
|
|
- VPS reports 4 CPUs, 7.7 GiB RAM (about 3.7 GiB available), and 31 GiB free
|
|
on `/`. Current runner-mount free space was not rechecked.
|
|
- `/opt/akyldash/current` exists, is a root-owned directory mode `0755`.
|
|
Reading its Git SHA failed with Git's `dubious ownership` safety guard; no
|
|
`safe.directory` setting was added, so the installed revision is unknown.
|
|
- `akyldash-act-runner.service` is active under `akyldash-runner`; installed
|
|
`act_runner` is `0.2.13`. The separate `akyldash-runner-restart.service`
|
|
remains failed since 2026-10-14; its cause could not be read from the
|
|
unprivileged journal view. No restart was attempted.
|
|
- `akyldash-verify`, `akyldash-compose`, and `akyldash-rollback` are absent
|
|
from PATH in the unprivileged check; no root helper or Docker operation was
|
|
run.
|
|
- Gitea is `1.25.5`; `main` is `b5adc09203cbbbd298054248206b9f47ae4a44ca`.
|
|
CI run 175 and runner smoke 176 succeeded on that commit. Package write
|
|
access and runner inventory remain outside this API principal's permissions.
|
|
- PR #59 head `852d639c455ff1c60202405e112d47b8128d66c6` received formal
|
|
review #308 with five requested fixes. CI run 177 failed in
|
|
`test_digest_rollback_fails_before_stopping_or_changing_references` because
|
|
the test's global `Path.stat` mock made `Path.exists()` report a false
|
|
recovery-journal hit under CI Python 3.12; runner smoke 178 succeeded. A
|
|
fix batch in this PR isolates state paths, handles ambiguous alias/cutover
|
|
outcomes, verifies rollback image IDs, and adds a synthetic artifact upload
|
|
check. On 2026-10-25 CI run 179 and runner smoke 180 succeeded at PR head
|
|
`9d8ce6b6`; CI took 2m43s and the synthetic upload step reported success.
|
|
The run API identifies run ID 567 and reports `run_attempt: 0`, Gitea's
|
|
legacy sentinel. The release workflow therefore uses the unique run ID and
|
|
does not depend on the unavailable attempt value. The run's artifact link was
|
|
downloaded from Gitea UI as `artifact-smoke-567-`: the ZIP was 191 bytes,
|
|
SHA-256 `ae0a2262147f222aacb4fdd1d12459f7160c67b1d0ee5d7ae2db1de8958f120f`,
|
|
and contained the expected 51-byte `marker.txt`. The API artifact listing
|
|
available to this principal returned no entries, but the downloaded file
|
|
verifies artifact availability. No live release or VPS change was made.
|
|
- The release descriptor SHA-256 sidecar now records `release-manifest.json`
|
|
relative to the `release` directory, matching the flattened layout observed
|
|
for downloaded Gitea artifacts. A local synthetic regression verifies the
|
|
checksum from the artifact extraction root; the credentialed release
|
|
workflow remains unrun.
|
|
- Owner-only outstanding evidence: sanitized container/volume and protected
|
|
file metadata; current backup and separate-target restore verification;
|
|
live corpus/index counts and RU/KY canaries; Cloudflare TLS mode; registry
|
|
writer identity and digest retention. Never include secret values in that
|
|
evidence packet. The checklist is in
|
|
[`DEPLOYMENT_OWNER_PACKET.md`](DEPLOYMENT_OWNER_PACKET.md).
|
|
|
|
The deployment PR adds a manual main-only image-publish workflow and an
|
|
owner-run `--no-build` cutover path, but neither has been executed against the
|
|
VPS. Its release descriptor is an unsigned trace record; cryptographic image
|
|
attestation remains a separate owner decision.
|
|
|
|
## Historical host snapshot — 2026-10-18
|
|
|
|
Source: SSH as `codex`, local repository inspection, and Gitea API. Root-only
|
|
Docker state, UFW rule details, and protected file metadata were not verified.
|
|
|
|
## Historical VPS facts — 2026-10-18
|
|
|
|
- Host: `avprogs.hlab.kz`, `206.223.249.118`, SSH port `22`, user `codex`.
|
|
- OS: Ubuntu Server `24.04.5 LTS`; kernel `6.8.0-142-generic` from the prior
|
|
inventory audit.
|
|
- Resources: 4 vCPU, 7.7 GiB RAM, 3.8 GiB swap, 99 GiB root filesystem with
|
|
84 GiB free on 2026-10-18. Runner storage is a separate 15 GiB ext4 loop
|
|
mount with 12 GiB free at that check.
|
|
- Network: interface `ens3`, IPv4 `206.223.249.118/32`.
|
|
- `codex` is UID/GID 1000, groups `codex` and `users`; `sudo -n true` requires
|
|
a password. Docker CLI cannot access `/var/run/docker.sock` as `codex`.
|
|
- SSH service is running. The inspected sshd configuration has public-key and
|
|
password authentication enabled and `PermitRootLogin yes`; no SSH changes
|
|
were made.
|
|
- Bootstrap completed. Docker CLI `29.9.0` and Compose `5.6.0` were rechecked
|
|
over SSH on 2026-10-18; containerd `2.4.1` and runc `1.5.2` are from the
|
|
prior package audit. The Docker daemon API remains root-only.
|
|
- `/etc/docker/daemon.json` has the expected `local` log driver and `10m` / `3`
|
|
rotation limits. Bootstrap paths exist with root ownership: `/opt/akyldash`
|
|
mode `0755`, `/etc/akyldash` and `/var/backups/akyldash` mode `0700`, and
|
|
`/var/lib/akyldash/data` mode `0750`.
|
|
- `/opt/akyldash/current`, verifier/rollback binaries, and the sudoers file are
|
|
absent. `codex` has not received sudo or Docker-group access. Metadata for
|
|
`/etc/akyldash/production.env` and `/etc/akyldash/review_secret` could not be
|
|
checked because access is denied. The target
|
|
`/var/lib/akyldash/data/minjust-normalized` and its manifest metadata were
|
|
also denied; their presence, ownership, and contents are unknown. No sudo was
|
|
used.
|
|
- TCP listeners observed: SSH `22` and loopback DNS only; no listener on `1500`,
|
|
`80`, or `443`. No ISPmanager package or service unit was found through the
|
|
unprivileged checks; root can confirm with the audit script below.
|
|
- Owner reports UFW is active and allows TCP `22`, `80`, `443`, and `1500`.
|
|
Rule details were not independently read; no firewall changes were made.
|
|
- Additional read-only runner checks on 2026-10-12 confirmed `/` is ext4 with
|
|
about 92 GiB free at the prior audit, `/usr/bin/dockerd-rootless.sh` exists,
|
|
and `git.vakanaut.ru` resolves to `195.209.48.28`. From the VPS,
|
|
`https://git.vakanaut.ru/api/v1/version` returned Gitea `1.25.5`; runner
|
|
registration and Gitea Actions protocol connectivity are not confirmed.
|
|
- `akyldash.kg` resolved to Cloudflare IPv4/IPv6 edge addresses. HTTPS returned
|
|
Cloudflare `521` on this historical 2026-10-18 check; current recheck is
|
|
recorded above. Cloudflare Full (strict) mode remains unverified.
|
|
|
|
## Historical application and Gitea facts — 2026-10-18
|
|
|
|
- Repository: `admin/akyldash`, default branch `main`; Gitea version `1.25.5`.
|
|
- The repository API reports push permission, Actions and packages enabled, and
|
|
PRs enabled. `main` is protected and requires one approval; the current API
|
|
principal cannot push or merge directly to `main`.
|
|
- The repository runner-list API returns `403` because this API principal is
|
|
not the repository owner. The latest verified CI run 75 and smoke run 76 both
|
|
succeeded on `main` head `0e276ae6`. The Actions run-list endpoint returned
|
|
`403` for this API principal during this recheck, so no newer run state was
|
|
independently retrieved. The owner reports the runner is registered and
|
|
Idle; repository runner inventory and package publishing permissions remain
|
|
unverified.
|
|
- No public application listener is observed on TCP `80`/`443`. Docker
|
|
container/volume state still requires the root audit. Repository Compose describes
|
|
Caddy `2.11.6` pinned by digest, Python Search API `0.12.2` in this branch,
|
|
SQLite account/review data, and OpenSearch `3.7.0` with `analysis-icu`.
|
|
- The target publishes only Caddy on TCP `80`/`443`. Backend and Caddy share a
|
|
proxy network; OpenSearch and backend share a separate internal network, and
|
|
backend has no host port. Caddy certificate storage, OpenSearch, and
|
|
application data use persistent volumes/bind mounts.
|
|
- The root-owned repository checkout, production env/secret files, Compose
|
|
volumes/containers, backup, verifier/rollback helpers, and application are
|
|
not confirmed installed. TLS and registry digest deployment for
|
|
backend/OpenSearch are not configured.
|
|
- The repository has a cold-backup archive/verification tool and a one-shot
|
|
restore apply tool that requires a marked replacement host. Backup/restore
|
|
tests use synthetic data; neither tool has run against production. The
|
|
archive preflight permits a clean first deployment to skip backup, requires
|
|
backup for existing persistent data even without containers, allows absent
|
|
named volumes only in explicit corpus-only initial-deployment mode, and
|
|
rejects inaccessible/ambiguous mounts and incomplete/running service
|
|
containers. Initial deployment docs require a second verified backup after
|
|
index and admin initialization.
|
|
- Recovery bootstrap has an explicit `--recovery-target` mode that refuses the
|
|
inventoried production hostname/IP; its host-identity guard has shell tests.
|
|
The mode has not been run on a VPS.
|
|
- The repository defines a root-owned read-only verifier, Compose `config` and
|
|
`status` helper, guarded rollback helper, and installer with exact sudoers
|
|
command rules. None was installed on the VPS at this inventory. The current
|
|
PR adds fixed root-side release/rollback receivers without adding a `codex`
|
|
sudoers rule or Docker access. Neither receiver has been installed or run.
|
|
- The former local-VPS rollback path has been superseded and now fails closed.
|
|
The PR rollback path requires a root-pinned external archive identity,
|
|
rechecks its exact bytes through the owner client, creates a fresh external
|
|
backup before changing refs, and preserves a secret-free recovery journal on
|
|
failure. Synthetic tests only; no production rollback was performed.
|
|
- The image release workflow now starts after successful CI for a push to
|
|
`main`. It checks the source workflow/event/branch/conclusion and exact source
|
|
SHA, checks out that SHA, and uses both release and source run IDs in the
|
|
schema-v4 descriptor. Registry secrets `REGISTRY_USERNAME` and
|
|
`REGISTRY_PASSWORD` remain unconfigured/unverified; no values were created or
|
|
inspected. PR image builds remain no-push. The descriptor SHA-256 is included
|
|
in the artifact name, and the artifact also retains its checksum sidecar.
|
|
The descriptor remains unsigned trace metadata, not cryptographic
|
|
attestation.
|
|
- A separate `deploy-handoff.yaml` runs after a successful release and checks
|
|
the source run and stale-main guards. Its job is explicitly
|
|
`NOT DEPLOYED - handoff status only`; a green result is not a deployment. It
|
|
reads artifact metadata through `gitea-api` only if that helper is available
|
|
in the job image; otherwise it reports metadata and host handoff as
|
|
unverified. It does not download artifact ZIPs, connect to SSH, or apply a
|
|
release. No repository workflow binding to a Windows runtime starts the
|
|
receiver. PR #59 now proposes a separate disabled local `--once` trigger
|
|
that queries a root-verified tuple; it is not installed or active. Image
|
|
release is automated, but production deployment is not yet automatic CD. The original
|
|
audited `gitea-api` helper lacks binary output and safe 302 handling; the
|
|
current PR carries a tested patch and host-side artifact verifier, but
|
|
neither is installed. No deploy key, fixed SSH entry, extra sudo rule, or
|
|
Gitea CD variable has been configured. Production apply remains blocked on
|
|
owner bootstrap, external backup and separate-target restore, and the other
|
|
gates above.
|
|
- A repository-scoped official `act_runner` `0.2.13` installation is
|
|
configured in `infra/runner/`: rootless Docker under `akyldash-runner`, 15 GiB
|
|
ext4 loop-backed image/workspace storage, capacity 1, per-job limits (2 CPU,
|
|
3 GiB RAM, 512 processes), account slice limits (2.5 CPU, 4 GiB RAM, 2048
|
|
tasks), rotated file logs, and a systemd health timer. Jobs target the
|
|
`akyldash-ci` label and do not receive either Docker socket. Latest verified
|
|
main CI run 175 and smoke run 176 succeeded on `b5adc092`. On 2026-10-18,
|
|
SSH showed `akyldash-act-runner.service` active under `akyldash-runner`, with
|
|
zero restarts and about 115 MiB current memory. Its rootless Docker store is
|
|
mounted from `/dev/loop0` as ext4 with `rw,nosuid,nodev`; the 15 GiB mount had
|
|
12 GiB free. The separate `akyldash-runner-restart.service` remains failed
|
|
with exit status 1 since 2026-10-14; its journal had no entries, so the cause
|
|
is unknown. The latest verified main CI run 75 and smoke run 76 succeeded on
|
|
head `0e276ae6`. The Actions API returned 403 during this recheck, so no newer
|
|
run state was independently retrieved. A read-only SSH recheck on 2026-10-21
|
|
confirmed the active `akyldash-act-runner.service` runs as
|
|
`akyldash-runner` with `ProtectHome=read-only`; `ReadWritePaths` includes the
|
|
runner runtime/config/log paths and `/run/user/999`. Its `ExecStart` uses
|
|
`/etc/akyldash-runner/config.yaml`. The live unit retains a writable carveout
|
|
for `/run/user/999`; the installer keeps that existing carveout and changes
|
|
`ProtectHome` from `yes` to `read-only` so the runtime path remains visible.
|
|
No VPS unit was changed. Main commit
|
|
`d2fb4b81` passed CI run 115 and smoke run 116. On
|
|
head `3a89dc21`, CI run 63 failed at the host audit root guard and smoke run 64
|
|
succeeded. On head `436806a8`, CI run 65 failed in the Caddy container action
|
|
with `exec: "validate": executable file not found`; Kaniko was skipped and
|
|
smoke run 66 succeeded. On head `d96fc4d3`, CI run 67 failed because the
|
|
Caddy action's fixed `/github/workspace/...` path did not exist; Kaniko was
|
|
skipped and smoke run 68 succeeded. On head `5d72bb09`, CI run 69 and smoke
|
|
run 70 succeeded. CI validated the Caddyfile and built the backend and
|
|
OpenSearch images using `${{ github.workspace }}`. Actual per-action
|
|
resource-limit enforcement and peak usage were not independently measured or
|
|
verified. Image publishing, host-wide resource metrics, and production
|
|
deployment remain unverified. No VPS changes are made.
|
|
- An isolated local Compose smoke test built backend/OpenSearch images,
|
|
reached healthy status for OpenSearch, backend, and Caddy, and returned HTTP
|
|
200 for `/review` through Caddy. Its containers, networks, and volumes were
|
|
removed afterward. This does not verify VPS startup, DNS, or public TLS.
|
|
- The app has no migration framework or SMTP setup. Repository versions on this
|
|
branch are:
|
|
backend `0.13.0`, frontend `0.2.0`, Telegram bot `0.2.2` (the bot is not
|
|
deployed on this VPS).
|
|
|
|
## Corpus readiness (2026-10-18)
|
|
|
|
- The local raw archive and normalized corpus are ignored under `data/`; no
|
|
corpus files were changed. Their read-only SQLite manifests both pass
|
|
`quick_check` and each report 209,958 documents. The raw manifest is
|
|
28,295,168 bytes with SHA-256
|
|
`f8e3dd2fdec607583ca09cfdb5b76171bb7bfa1e41a0e07ee5aca8481e18e17e`; it has
|
|
one fetch error, code `6`, absent from `documents`. The normalized manifest
|
|
is 31,571,968 bytes with SHA-256
|
|
`1679c20e6d0f3e5a9b50737c006c4417a28ebcaf7467d194d3bf8243538f83bf`; all
|
|
entries are `success`, schema `1`, normalizer `1.0.0`.
|
|
- The read-only full-file inventory in `/tmp/akyldash-corpus-audit.Mjr6aa`
|
|
stopped before validation in run `b4a4556e30cc464d90bc8d75ef1673e9` with
|
|
`Python int too large to convert to SQLite INTEGER`. At stop it had processed
|
|
2,120,420 raw entries (1,081,428 files, 1,038,992 directories) and 397,856
|
|
normalized entries (246,321 files, 151,535 directories). This is a verifier
|
|
metadata storage failure, not a corpus finding. A full rehash with the
|
|
corrected, versioned verifier is required before readiness can be stated.
|
|
- A fresh full rehash is running in `/tmp/akyldash-corpus-audit-final.gtsD87vc`,
|
|
run `f4dd5e53020f442bbbecee2d0e4986a6`, with verifier SHA-256
|
|
`83dac230099fc91972cd14e86e9f8ec72d13bca08f53d402f13a169e096d98f5`.
|
|
`result.json` currently reports `scan_status: running`; its live counters
|
|
must be recorded after completion. Sources remain read-only.
|
|
- Through the repository client, bounded MinJust OpenData requests for
|
|
`GetDocumentListByQuery`, `GetDocument(6)`, control `GetDocument(1096)`, and
|
|
`CheckAvailable` each returned HTTP `404`. The list API returned no code list;
|
|
the direct requests do not establish whether code `6` exists. The public
|
|
listing page is reachable, but it does not map the numeric API code. A
|
|
metadata-only `CheckAvailable` request from the VPS failed in HTTP/2 with
|
|
`curl` exit 92 (`INTERNAL_ERROR`), HTTP status `000`, and TLS verification 0;
|
|
the HTTP/1.1 retry was not completed. Treat the code `6` disposition and
|
|
full corpus readiness as unresolved.
|
|
- On the VPS, read-only `stat` of `/var/lib/akyldash/data/minjust-normalized`
|
|
and its manifest was denied. Production corpus presence and metadata remain
|
|
unknown. No corpus was transferred and no production service, Docker socket,
|
|
runner policy, or VPS configuration was changed.
|
|
|
|
## Current MinJust API staging (2026-10-20)
|
|
|
|
- The public site's current API uses `POST /api/v1/GetDocuments` with page size
|
|
12, `GET /api/v1/GetDocument?DocumentCode=...`, and
|
|
`GET /api/v1/GetEdition?editionId=...&lang=ru|kg`. A bounded page-1 probe
|
|
reported 210,639 filtered records; no full listing was enumerated for this
|
|
recovery. Official alias `1-4` resolves to canonical document code `205064`.
|
|
- Document code `6` has 201 official editions. The API's `lang=kg` response
|
|
places Kyrgyz HTML in `contentRu` while returning `contentKg=null`; staging
|
|
maps by requested language (`kg` to archive `ky`) and preserves the edition
|
|
order from the document response. Normalization and search export now retain
|
|
and use that order instead of sorting IDs.
|
|
- Without `exact=true`, one probe for edition `920292` returned response field
|
|
`id=920492` despite a different text length/hash. Two bounded `exact=true`
|
|
probes returned matching IDs and names/dates from `GetDocument`: edition
|
|
`920292` / `04.08.1998` (485,487 RU characters, SHA-256
|
|
`2b51c2ff4e9b91cbe2a7ed052a5a9ed9150bda993fc15c120ac948580926b718`) and
|
|
edition `920492` / `13.04.2017` (1,139,908 RU characters, SHA-256
|
|
`e5f0fdbb47766e5a5dbb971183d765e4c1dcaf9c969a218a84206244ce9132e8`).
|
|
Every edition request now uses `exact=true`, requires returned ID equality,
|
|
and checks RU/KY names against `GetDocument` metadata.
|
|
- `backend/ingestion/minjust_current.py` stages one document into a separate
|
|
generation, at no more than one request per second, with a 16 MiB response
|
|
limit, a 4 GiB text budget per document, contract-bound checkpoints, and
|
|
atomic publication. Staging requires a private 0700 directory owned by the
|
|
current user and rejects symlinks under checkpoints/document paths; legacy
|
|
corpus exclusions are anchored to the repository root. The corpus verifier
|
|
continues to validate unchanged normalized manifests from versions 1.0.0
|
|
and 1.1.0. The earlier non-exact pass is quarantined at
|
|
`/tmp/akyldash-current-code6.ZAMC37u4ex/qualification.json`: its 240 language
|
|
responses (304,493,520 bytes) remain unverified and are not reusable. The
|
|
retained work-tree SHA-256 is
|
|
`1e874c1d7fab1e00e7e45ff1b16422ea474e22ee13c901ac9a0612f47aa5d24a`.
|
|
- Exact-only code-6 staging is complete at
|
|
`/tmp/akyldash-current-code6-exact.L4k3AvnOlP`: 201 editions and 402
|
|
complete language checkpoints, 604,377,885 raw bytes. The raw manifest
|
|
passes `quick_check`; its document checksum matches the staged directory at
|
|
`81d8e1a53b2ebef61f08b56d04b3f0a80077d587a4fadae0152bb4b7f13055ea`.
|
|
Separate normalization at
|
|
`/tmp/akyldash-current-code6-exact.L4k3AvnOlP-normalized` contains 201
|
|
editions in official response order, passes `quick_check`, and records
|
|
normalizer `1.1.0` (2,048,797,024 bytes). The prior `data/minjust-cbd` and
|
|
`data/minjust-normalized` trees and production were not used as input or
|
|
modified. The raw staging manifest still records attachments and embedded
|
|
media as unverified; this single document does not establish corpus
|
|
completeness or production readiness.
|
|
|
|
## Checks not completed
|
|
|
|
- Docker Engine version from the server API, container and volume inventory,
|
|
protected env/secret metadata, sudoers, and detailed UFW rules require root.
|
|
`infra/bootstrap/audit-host.sh` is provided for an owner-run read-only audit.
|
|
- The backup archive/restore tool was tested with synthetic local data; a backup
|
|
and restore of real production data remain unverified because production is
|
|
not deployed.
|
|
- Cloudflare edge response was checked; origin DNS target, origin certificate,
|
|
Full (strict), registry publishing, host-wide runner metrics, and production
|
|
deployment have not been verified. The
|
|
owner-reported Idle state has not been independently checked through runner
|
|
inventory because the API returned 403.
|
|
- This audit made no host changes. No package or service was installed by this
|
|
audit; the owner performed the initial bootstrap beforehand.
|
|
|
|
## Next owner actions
|
|
|
|
1. Install the approved root-owned checkout after its PR is reviewed, then run
|
|
`sudo bash /opt/akyldash/current/infra/bootstrap/audit-host.sh` and share
|
|
the output. The script prints no secret values and does not mutate the host.
|
|
2. After reviewing the audit output and confirming the target files are absent,
|
|
run
|
|
`sudo bash /opt/akyldash/current/infra/deploy/prepare-production-config.sh`;
|
|
it refuses to overwrite existing files. Set the approved `DATA_ROOT` and
|
|
place the approved corpus/index and Cloudflare origin configuration. Do not
|
|
deploy until data/service state and the required backup path are reviewed.
|
|
3. Run `sudo bash /opt/akyldash/current/infra/deploy/install-helper.sh` after
|
|
reviewing the checkout; it grants only fixed verification, Compose
|
|
`config`/`status`, and guarded rollback commands.
|
|
4. After merging the reviewed PR, confirm a subsequent CI and smoke run on
|
|
`akyldash-ci`. Image publishing and production deployment remain manual and
|
|
are not part of CI.
|
|
5. Test backup/restore and rollback only with synthetic data in an isolated
|
|
environment before any production deployment.
|
|
|
|
## Change in this PR
|
|
|
|
Adds a read-only, resumable full-file corpus inventory and normalized-data
|
|
cross-check, a source/staging comparator requiring nonzero documents and
|
|
fragments plus held shared artifact locks, synthetic regression tests, and an
|
|
operator handoff runbook. CI's
|
|
existing `infra/deploy` unittest discovery includes the new tests. No workflow,
|
|
runner, VPS, production data, or Docker settings are changed. The first real
|
|
corpus scan stopped on an integer-storage bug; full validation with the
|
|
corrected verifier is still required. The source API's code `6` outcome remains
|
|
unconfirmed.
|