ObliviOS Developer portal Project site

Home / Decisions / ADR-041

Accept the alpha-channel OTA signing infrastructure design.

Accepted, recorded 19 August 2026.

Decision
ADR-041
Date
2026-08-19
Status
AcceptedAccepted
Mentioned by
ADR-043, ADR-046, ADR-079

1Decision

Accept the alpha-channel OTA signing infrastructure design (docs/superpowers/specs/2026-08-18-ota-signing-infrastructure-design.md) and its implementation plan (docs/superpowers/plans/2026-08-18-ota-signing-infrastructure.md) as complete for the ceremony's control-flow and fail-closed behavior, with the real sign/publish success path still unexecuted.

2Why and evidence

Two-VM split-signing design (the build host produces candidates, the signing host holds the key and signs) was hardened this session (access, key-storage and network hardening of the signing host) and then implemented against real infrastructure, not simulated.

A real 9-key + avb.pem/avb_pkmd.bin bundle (script/generate-keys) was generated inside the vault; tools/signing/sign-ota.sh was built to require an interactively-supplied, independently-sourced hash before signing (never a co-located hash file), run GrapheneOS's own script/generate-release.sh, independently re-verify the signature (check_ota_package_signature.py) before publishing, and always leave the machine powered off and the vault closed.

Real execution — not code review alone — found and fixed several genuine defects a design pass would not have caught: a Unix symlink-crossing quirk in generate-keys (cd through a symlink sets the process's real working directory to the symlink's target, breaking ../../ relative references used afterward); a /dev/shm capacity bug (a full-tree tooling copy into a 984M tmpfs on a 1.9GB-RAM VM, guaranteed to fail against any real multi-GB candidate); a shutdown interaction with the access hardening of the signing host that broke the "machine always powers off" guarantee for non-interactive invocation (fixed narrowly); and, found only by a final whole-branch review after all five plan tasks individually passed, a Critical regression where the shutdown trap itself could abort under errexit before reaching sudo poweroff if an internal cleanup command failed — reproduced empirically before and after the fix — plus a real design gap where otatools.zip executed (via GrapheneOS's own generate-release.sh prepending its unzipped bin/ to PATH) with the decrypted key bundle already staged, while being entirely outside the ceremony's hash-verification gate.

All Critical/Important findings were fixed, independently re-verified by a separate reviewer pass, and re-tested for real against the corrected script on the physical the signing host afterward, rather than trusted on the strength of a plausible-looking diff.

One independent-reviewer dispatch was blocked twice by a weekly usage limit; a controller self-review was substituted and disclosed as such rather than silently treated as equivalent, and was itself later checked by a successful independent pass.

3Consequences

The vault-resident key bundle, tools/signing/sign-ota.sh (tracked in this repo, deployed to the signing host's home directory, never into the virtiofs-shared paths), and a persistent the signing host network egress block now exist. Both fail-closed paths (missing/incomplete candidate; hash mismatch on either target_files.zip or otatools.zip; fake-content-after-hash-match) are confirmed on real hardware, including after the final-review fixes.

The real success path — an actual generate-release.sh completion, a real check_ota_package_signature.py pass, and a real atomic publish to <build-root>/signed/ — has never executed, because no real lynx-target_files.zip/lynx-otatools.zip candidate has been built (a full build is a separate 8+ hour undertaking, deliberately out of this plan's scope). Real disk capacity on the signing host for a full-size candidate is consequently unmeasured.

generate-release.sh also produces factory/img bundles alongside the OTA update zip; the ceremony currently deletes those along with everything else in cleanup, preserving only the signed OTA — a real, undecided product-scope question, not something inferred from cleanup code.

4Review when

Revisit before any alpha-channel OTA is actually distributed: run one real ceremony against a genuine built candidate, start to finish including a real publish, checking the signing host's real free disk space first. Revisit sooner if that run's disk usage approaches or exceeds the signing host's current headroom, or if the owner decides factory/img bundles should be preserved per-ceremony rather than discarded.

Read from docs/governance/decision-log.md, entry ADR-041.