diff --git a/debian/test/README.md b/debian/test/README.md index a68ebfc..ec70efe 100644 --- a/debian/test/README.md +++ b/debian/test/README.md @@ -39,6 +39,7 @@ not a pass**, and the summary says so. | T17 | A real sudo-group user is authorised **without a password** (pkcheck), with a control | | T18 | Offline-update script/units: bash -n, `systemd-analyze verify`, timer enabled, throttle, no dangling `/system-update` | | T19 | The preseed still generates a working `/root/vdi-post.sh` that delegates updates | +| T20 | `make-iso.sh` builds a bootable ISO with the preseed baked in (skips without a netinst ISO) | | T15 | Nothing was left behind: no source file, no cached indexes, no installs, no `__pycache__`, no polkit rule or unit residue | ### The repo build it tests @@ -117,6 +118,12 @@ Being explicit matters more than the green summary. `pkcon refresh` uses) and that `package-remove` is *not* granted. Installing a package from a desktop session without a password is not exercised. +11. **Actually booting the built ISO.** T20 proves the image is structurally + correct and the preseed is on it, but it never boots it in a VM. Whether the + installer runs unattended end to end, and whether the inventory prompt behaves, + is untested. `xorriso` says the boot record is intact; only a real (virtual) + machine says the firmware agrees. + ## Two spec'd checks that cannot fail, and what replaced them This is the most important thing in this file. Two checks that look like tests @@ -322,6 +329,75 @@ before the run, so "it did not exist and must be gone" is distinguishable from symlink - it never recurses into a directory. The same restores run in the `EXIT` trap, so an aborted run still cleans up. +## T20: the ISO builder + +T20 needs a real Debian netinst ISO and **does not download one** - a test that +fetches 792 MB before it can start is a test nobody runs. It looks for +`/root/trixie-netinst.iso` (or `NETINST_ISO=/path`), and without one it skips +and lists what it would have checked. + +With an ISO present it runs `make-iso.sh` into a temp directory (about 3 seconds) +and asserts: + +- It **refuses** to overwrite an existing output, and refuses to write over its + own input. Both are tested by expecting a non-zero exit, not by reading the code. +- `/preseed.cfg` on the output is byte-identical to `debian/preseed.cfg`. + Extracted to a real file first - `-extract ... /dev/stdout` does not work. +- **Every** kernel line under `/isolinux` and `/boot/grub` carries + `preseed/file=/cdrom/preseed.cfg`. Counted, not sampled: on 13.7.0 it is 40 + lines (20 BIOS + 20 UEFI), and the assertion is `patched == total`. A + `> 0` check would pass while half the entries still prompt. +- `isolinux/gtk.cfg` specifically is patched. That file holds the BIOS *default* + entry; `isolinux.cfg` is only `include` and `default vesamenu.c32`, so patching + the obvious files leaves the default boot prompting. That is the bug this test + exists to catch, and it is asserted by name rather than left implicit. +- No kernel line carries the parameter **twice** (a re-run of the patch, or a + `-map` applied on top of an already-patched file). +- Both a BIOS and a UEFI El Torito image are present. +- No file from the source ISO is missing from the output (1010 files on 13.7.0). +- Output size is within 16 MB of the source. The measured delta is -704 KB. The + naive extract-and-rebuild implementation bloats 792 MB -> 988 MB because + Debian's ISO shares identical file extents between paths; this guards that + regression. + +### `grep -c` exits 1 on zero, and that will corrupt your count + +While writing T20 this produced a wrong total: `grep -c ... || echo 0` prints +`0` **and then** appends another `0` when there are no matches, because `grep -c` +prints the count and *also* exits 1. The result is `0\n0`, and `$((total + 0\n0))` +is a syntax error. The harness swallows the exit status only (`|| true`) and +validates the value with a `case` before doing arithmetic. + +## Two corrections that are now asserted + +**`subject.local` must not come back into the polkit rule.** An earlier version +of the rule included `subject.local == true` to restrict it to a seat physically +in front of the machine. It never matches: `pkcheck` returns 2 with the clause +and 0 without it, for the same user. If it returns, passwordless updates stop +working while every other T16 assertion still passes - so T16 now asserts the +clause is **absent**, with the reason in a comment. + +**"the timer is enabled" is not a stable assertion on its own.** Enabling is done +by `systemctl enable vdi-offline-update.timer` inside +`install_debian_update_policy()`, and that only persists when systemd is the +running init. T18 therefore drives the installer function first, then asserts, +and **skips** the assertion with a reason when systemd is not PID 1. + +## A killed run leaves apt permanently broken, and the lock cannot help + +The harness removes its own apt source in an `EXIT` trap. A run that is killed +(the turn is cancelled, the terminal is closed, `SIGKILL`) never reaches that +trap, so `/etc/apt/sources.list.d/zz-vdi-test-.list` survives pointing at a +temp directory that no longer exists. Every subsequent `apt-get update` then +fails with `Failed to fetch ... File not found`, and T06/T07 report that as a +port defect. This was observed: a leftover `zz-vdi-test-250.list` from another +run broke three T06 assertions while the port was fine. + +The `flock` does not prevent it, because there is no concurrent process - just +residue. So at startup, under the lock, the harness sweeps any +`zz-vdi-test-*.list` whose target directory is gone, and says so. That is safe +precisely because the lock guarantees no peer run is alive at that moment. + ## Requirements - Debian 13 (trixie), root. @@ -333,6 +409,11 @@ symlink - it never recurses into a directory. The same restores run in the machine, which is why the harness treats its absence as a skip rather than installing it itself. - `gpg` **for T12 only**, and only if you want to re-check the signing decision. +- `xorriso` **for T20 only**. Without it T20 skips with a reason. +- A Debian netinst ISO **for T20 only**, at `/root/trixie-netinst.iso` or + `NETINST_ISO=/path/to/netinst.iso`. It is never downloaded. The build itself + takes about 3 seconds and needs ~1.6 GB free in `TMPDIR` (source plus output; + the source is not copied, only written to the output). ## Safety diff --git a/debian/test/run-tests.sh b/debian/test/run-tests.sh index beb6765..be04250 100644 --- a/debian/test/run-tests.sh +++ b/debian/test/run-tests.sh @@ -236,6 +236,26 @@ for tool in apt-ftparchive apt-get apt-cache dpkg-deb python3 gzip; do done # Clear indexes from any earlier run before doing anything else. +# +# A run that was killed (SIGKILL, a cancelled turn, a closed terminal) never +# reaches its EXIT trap, so its /etc/apt/sources.list.d/zz-vdi-test-.list +# survives pointing at a temp directory that no longer exists. Every later +# `apt-get update` then fails with "Failed to fetch ... File not found" and +# T06/T07 blame the port for the wreckage - measured: exactly that happened +# with a leftover zz-vdi-test-250.list from another run. +# +# This is safe to sweep because flock above guarantees no other run is alive, +# so any zz-vdi-test-* source file at this point is residue, not a peer. +for stale_list in /etc/apt/sources.list.d/zz-vdi-test-*.list; do + [ -e "$stale_list" ] || continue + stale_dir=$(sed -n 's|.*file:\([^ ]*\).*|\1|p' "$stale_list" 2>/dev/null) + if [ -z "$stale_dir" ] || [ ! -d "$stale_dir" ]; then + rm -f "$stale_list" + printf ' removed stale apt source from a killed run: %s\n' \ + "$(basename "$stale_list")" + fi +done + purge_all_harness_lists CODENAME="" @@ -1546,6 +1566,187 @@ PYEOF fi fi +# --------------------------------------------------------------------------- +# T20: make-iso.sh produces a bootable, correctly-preseeded ISO +# --------------------------------------------------------------------------- +# Needs a real Debian netinst ISO. It is NOT downloaded here: a test that +# fetches 792 MB from the network is a test nobody runs. Without the ISO this +# skips and says what it would have checked. + +head2 "T20 make-iso.sh builds a preseeded bootable ISO" + +MAKE_ISO="$REPO_ROOT/debian/make-iso.sh" +NETINST="" +for candidate in "${NETINST_ISO:-}" /root/trixie-netinst.iso /root/debian-netinst.iso; do + [ -n "$candidate" ] && [ -f "$candidate" ] && { NETINST="$candidate"; break; } +done + +if [ ! -f "$MAKE_ISO" ]; then + skip "T20 no debian/make-iso.sh in this repo (owned by another change)" +elif ! command -v xorriso >/dev/null 2>&1; then + skip "T20 xorriso is not installed - make-iso.sh cannot run" + info "apt-get install xorriso" +elif [ -z "$NETINST" ]; then + skip "T20 no Debian netinst ISO found (looked for /root/trixie-netinst.iso)" + info "would have checked: preseed on the image is byte-identical, every kernel" + info "line carries preseed/file=, gtk.cfg is patched, both El Torito images" + info "exist, no source file is missing, size is within a few MB of the source." + info "Point NETINST_ISO=/path/to/netinst.iso at one to run it." +else + info "source ISO: $NETINST ($(du -h "$NETINST" | cut -f1))" + + if bash -n "$MAKE_ISO" 2>"$TMP_DIR/isoerr"; then + pass "T20 make-iso.sh passes bash -n" + else + fail "T20 make-iso.sh fails bash -n: $(cat "$TMP_DIR/isoerr" | tr '\n' ' ')" + fi + + ISOWORK="$TMP_DIR/iso" + mkdir -p "$ISOWORK" + OUT_ISO="$ISOWORK/out.iso" + + # Guard: it must refuse to overwrite an existing output, and refuse to + # write over its own input. Both are destructive if they get it wrong. + : > "$OUT_ISO" + if bash "$MAKE_ISO" "$NETINST" "$OUT_ISO" >/dev/null 2>&1; then + fail "T20 make-iso.sh overwrote an existing output file (should refuse)" + else + pass "T20 make-iso.sh refuses to overwrite an existing output" + fi + rm -f "$OUT_ISO" + + if bash "$MAKE_ISO" "$NETINST" "$NETINST" >/dev/null 2>&1; then + fail "T20 make-iso.sh accepted output == input (would destroy the source)" + else + pass "T20 make-iso.sh refuses to write over its input ISO" + fi + + # The real build. + build_start=$(date +%s) + if build_out=$(bash "$MAKE_ISO" "$NETINST" "$OUT_ISO" 2>&1); then + build_rc=0 + else + build_rc=$? + fi + build_secs=$(( $(date +%s) - build_start )) + + if [ "$build_rc" -eq 0 ] && [ -s "$OUT_ISO" ]; then + pass "T20 make-iso.sh built an ISO (${build_secs}s, $(du -h "$OUT_ISO" | cut -f1))" + else + fail "T20 make-iso.sh failed (rc=$build_rc)" + printf '%s\n' "$build_out" | tail -12 | sed 's/^/ /' + fi + + if [ -s "$OUT_ISO" ]; then + # ---- preseed is on the image and byte-identical ---- + # Note: `-extract /preseed.cfg /dev/stdout` does not work; extract to a + # real file. + if xorriso -osirrox on -indev "$OUT_ISO" -extract /preseed.cfg \ + "$ISOWORK/preseed.on.iso" >/dev/null 2>&1; then + if cmp -s "$ISOWORK/preseed.on.iso" "$REPO_ROOT/debian/preseed.cfg"; then + pass "T20 /preseed.cfg on the ISO is byte-identical to debian/preseed.cfg" + else + fail "T20 /preseed.cfg on the ISO differs from debian/preseed.cfg" + fi + else + fail "T20 /preseed.cfg is not present on the output ISO" + fi + + # ---- every kernel line is patched, counted not sampled ---- + menu_list=$( for dir in /isolinux /boot/grub; do + xorriso -indev "$NETINST" -find "$dir" -name '*.cfg' 2>/dev/null + done | tr -d "'" | grep -v '^$' | sort -u ) + + total_k=0; patched_k=0; unpatched_list=""; dup_list="" + for menu in $menu_list; do + local_file="$ISOWORK/menu_$(printf '%s' "$menu" | tr '/' '_')" + xorriso -osirrox on -indev "$OUT_ISO" -extract "/$menu" "$local_file" \ + >/dev/null 2>&1 || continue + # grep -c prints 0 AND exits 1 when there are no matches, so + # `|| echo 0` would append a second zero. Swallow the status only. + k=$(grep -cE '^[[:space:]]*(append|linux)[[:space:]]' "$local_file" 2>/dev/null || true) + case "$k" in ''|*[!0-9]*) k=0 ;; esac + p=$(grep -E '^[[:space:]]*(append|linux)[[:space:]]' "$local_file" 2>/dev/null \ + | grep -cF 'preseed/file=/cdrom/preseed.cfg' || true) + case "$p" in ''|*[!0-9]*) p=0 ;; esac + total_k=$((total_k + k)) + patched_k=$((patched_k + p)) + [ "$k" -gt 0 ] && [ "$p" -ne "$k" ] && unpatched_list="$unpatched_list $menu($p/$k)" + # The parameter must appear at most once per kernel line. + dups=$(grep -E '^[[:space:]]*(append|linux)[[:space:]]' "$local_file" 2>/dev/null \ + | awk '{n=gsub(/preseed\/file=/,"&"); if (n>1) print}' | wc -l) + [ "$dups" -gt 0 ] && dup_list="$dup_list $menu" + done + + if [ "$total_k" -gt 0 ] && [ "$patched_k" -eq "$total_k" ]; then + pass "T20 all $total_k kernel lines carry preseed/file= ($patched_k patched)" + else + fail "T20 only $patched_k of $total_k kernel lines are patched" + [ -n "$unpatched_list" ] && info "unpatched:$unpatched_list" + fi + + # The BIOS *default* entry lives in gtk.cfg, not isolinux.cfg. Patching + # only the obvious files leaves the machine prompting, which is the bug + # this test exists to catch. + gtk_file="$ISOWORK/menu__isolinux_gtk.cfg" + if [ -f "$gtk_file" ]; then + gtk_k=$(grep -cE '^[[:space:]]*(append|linux)[[:space:]]' "$gtk_file" 2>/dev/null || true) + case "$gtk_k" in ''|*[!0-9]*) gtk_k=0 ;; esac + if [ "$gtk_k" -gt 0 ] && grep -qE '^[[:space:]]*(append|linux)[[:space:]].*preseed/file=' "$gtk_file"; then + pass "T20 isolinux/gtk.cfg is patched (the BIOS default entry)" + else + fail "T20 isolinux/gtk.cfg is NOT patched - the BIOS default would still prompt" + fi + else + info "T20 isolinux/gtk.cfg not present on this ISO (layout differs)" + fi + + if [ -z "$dup_list" ]; then + pass "T20 no kernel line carries the parameter twice" + else + fail "T20 duplicated parameter in:$dup_list" + fi + + # ---- both El Torito images survive ---- + torito=$(xorriso -indev "$OUT_ISO" -report_el_torito plain 2>&1) + if printf '%s' "$torito" | grep -q 'El Torito'; then + pass "T20 output ISO has an El Torito boot record" + else + fail "T20 output ISO has no El Torito boot record" + fi + if printf '%s' "$torito" | grep -q 'BIOS' && printf '%s' "$torito" | grep -q 'UEFI'; then + pass "T20 both BIOS and UEFI boot images are present" + else + fail "T20 only one firmware type is bootable in the output ISO" + fi + + # ---- no source file is missing ---- + xorriso -indev "$NETINST" -find / -type f -exec lsdl 2>/dev/null \ + | awk '{print $NF}' | sort > "$ISOWORK/src.list" + xorriso -indev "$OUT_ISO" -find / -type f -exec lsdl 2>/dev/null \ + | awk '{print $NF}' | sort > "$ISOWORK/out.list" + missing_n=$(comm -23 "$ISOWORK/src.list" "$ISOWORK/out.list" | wc -l) + src_n=$(wc -l < "$ISOWORK/src.list") + if [ "$src_n" -gt 0 ] && [ "$missing_n" -eq 0 ]; then + pass "T20 all $src_n source files are present in the output" + else + fail "T20 $missing_n of $src_n source files are missing from the output" + comm -23 "$ISOWORK/src.list" "$ISOWORK/out.list" | head -5 | sed 's/^/ /' + fi + + # ---- size guard: the extract-and-rebuild approach bloated 792->988 MB ---- + src_bytes=$(stat -c '%s' "$NETINST") + out_bytes=$(stat -c '%s' "$OUT_ISO") + delta=$(( out_bytes - src_bytes )) + if [ "$delta" -lt 0 ]; then abs_delta=$(( -delta )); else abs_delta=$delta; fi + if [ "$abs_delta" -le 16777216 ]; then + pass "T20 output size is within 16 MB of the source (delta ${delta} bytes)" + else + fail "T20 output size differs by $delta bytes - the extent-dedup regression is back" + fi + fi +fi + # --------------------------------------------------------------------------- # T15: the harness left nothing behind # ---------------------------------------------------------------------------