1Decision
Accept tools/signing/sign-baseline-images.sh (the otacerts/test-key incident's fix tooling, docs/devices/lynx-ota-otacerts-testkey-incident.md) as fixed and working after five real bugs; reject the "swap only system.img, keep the device's other 11 partitions unchanged" baseline-flash approach the whole fix plan was built on, after a real flash produced a cross-partition package-signature crash; device reverted to its last known-good state, safe but not fixed.
2Why and evidence
Code review plus real execution against candidate 2026082507 found and fixed five bugs across four ceremony runs, each exposed only once the previous one let the script run further: (1) /dev/shm disk exhaustion during the script's own verification step, reproduced and fixed by relocating that step's working directory to the larger shared pool; (2) the same class of write-error-on-full-disk could still recur unprotected at the new location and had no non-interactive-safe handling — fixed with a real df-based precheck, </dev/null on the extraction, and explicit failure logging, all reproduced locally with ulimit -f before trusting the fix; (3) avbtool verify_image requires every one of this device's 12 partition images present in the same directory as vbmeta.img, not just system.img (confirmed by reading AOSP's real external/avb/avbtool.py — desc.verify() is called unconditionally for every descriptor, no CLI flag skips it) — the original "isolate just two files" design had misread docs/research/build-and-signing-engineering-notes.md's existing same-directory-footgun finding as calling for a minimal file set, when the established fix there was actually an isolated directory of all correctly-named partition images; (4) the otacerts.zip path the script checked, META/otacerts.zip, was never real — confirmed by reading AOSP's real sign_target_files_apks.py (ReplaceOtaKeys/WriteOtacerts rewrite whatever path already exists in the input zip ending /otacerts.zip, no hardcoded path) and by directly listing the real candidate's own zip contents (SYSTEM/etc/security/otacerts.zip, the path RecoverySystem.verifyPackage() actually reads); (5) the test-key detection's string match (*"CN=Android"*) would have missed a real leftover test cert on the signing host's actual OpenSSL, which formats -subject output with spaces around = unlike the OpenSSL version an earlier local reproduction had used — found by inspection after a real run already succeeded, fixed by stripping whitespace before matching.
All five verified against real upstream source or real local reproduction, not assumed. The fourth, fully-corrected ceremony run then completed cleanly end to end: AVB verification passed, otacerts.zip confirmed carrying the real release cert (CN = GrapheneOS, not CN = Android), image bundle published.
That system.img was re-signed (fresh add_hashtree_footer + make_vbmeta_image, rollback index 1783209600 freshly re-confirmed against the real currently-flashed vbmeta-v3.img rather than reused from memory, 11 unchanged partitions sourced from the same duress-dummy-task3-signing evidence ADR-046/047 already proved correct, sha256-verified byte-identical before use) and independently verified 13/13 on two separate machines before flashing — same bar as every prior ceremony in this project.
The flash itself and AVB verification were both clean. The device still never reached the lock screen: adb logcat (device stayed reachable, this was a crash loop, not a hang) showed system_server repeatedly crashing with IllegalStateException: Signature mismatch on system package com.google.android.iwlan for shared user ... android.uid.system/1000.
Root cause, one level up from AVB: Android's package manager requires every APK sharing a sharedUserId to carry the same signing certificate regardless of which partition it lives on. sign_target_files_apks had swapped every system-partition APK's signature to the vault's real release key; vendor (left unchanged, carrying com.google.android.iwlan, part of the same shared UID) still carried whatever key Task 3's 2026-08-16 build used.
Neither AVB verify_image nor the otacerts subject check could have caught this — both passed cleanly — because cross-partition package-signature consistency is a third, independent trust mechanism neither one checks. Recovered by re-flashing the last known-good system.img/vbmeta-v3.img pair (kept locally, already proven to boot clean before this session); confirmed clean: ro.build.version.incremental back to 2026081401, 0 Dropbox entries.
3Consequences
tools/signing/sign-baseline-images.sh is now a correct, working, real-hardware-validated tool for producing a release-signed image bundle from a candidate — that part of this incident's fix is durably complete (final hash 0c7c93294e7f6d10881abf92eb7b37de; the whitespace fix lands before any future run, not retroactively).
The "splice a newly-signed system.img onto the device's otherwise-unchanged partitions" design this whole fix plan assumed, however, cannot work as designed and is rejected — a real, reproduced, general finding recorded in docs/research/build-and-signing-engineering-notes.md and docs/devices/lynx-ota-otacerts-testkey-incident.md: cross-partition shared-UID package-signature consistency is a genuine third constraint, independent of AVB and of whole-package release signing, invisible to every verification this project has used until an actual boot is attempted.
The physical device is back on BUILD_NUMBER=2026082417's known-good system.img/vbmeta-ota-server-signed-v3.img (ADR-047's state), confirmed booting clean — safe, stable, but the otacerts/test-key incident remains genuinely open; the already-published OTA (build 20260825082317) needs no changes, only the device's trust store does.
4Review when
Before any further baseline re-flash attempt: decide and design (not improvise ad hoc) whether the fix release-signs every partition carrying a package in a shared-UID group crossing partition boundaries (at minimum vendor, not yet fully enumerated which others) or instead builds and signs one fully internally-consistent 12-partition candidate end to end rather than splicing a new partition onto old ones. Revisit sign-baseline-images.sh itself only if a future real run surfaces a sixth defect this review didn't find.
Read from docs/governance/decision-log.md, entry ADR-048.