tests: cover polkit rule, offline-update timer, preseed generation and ISO build

Adds T16-T20 to the Debian harness (103 passed, 0 failed, 2 skipped):

T16  the polkit rule is installed, mode 644, covers the two update actions,
     does not over-grant package-remove/reinstall/untrusted, and is idempotent.
T17  the real decision, in both directions: a sudo-group user is authorised
     (pkcheck rc=0) with the rule, and still prompted (rc=2) without it. The
     control is the point - without it the test cannot fail.
T18  vdi-offline-update: bash -n, unit verification, the throttle stamp, that
     a second run is skipped, that a stale stamp lets it run, that pkcon exit 5
     is reported as 'no updates available' rather than a download failure, and
     that /system-update is never armed without a prepared payload.
T19  the preseed still generates a script that parses, delegates to
     vdi-offline-update instead of an inline pkcon sequence, and enables cron
     and the timer.
T20  make-iso.sh: refuses to overwrite its input or an existing output, then
     checks the built image - preseed byte-identical, all 40 kernel lines
     patched, gtk.cfg patched, no doubled parameters, El Torito present with
     both BIOS and UEFI images, no source file missing, and size within 16MB of
     the original (the bloat regression).

Skips stay honest: T12 needs gpg, T13's ks.cfg assertion is vacuous because
the file was never in the repo.
This commit is contained in:
Mistercoolertyper committed 2026-11-04 10:30:30 +01:00
1 parent 15d65183dc
commit 402fea66fd
2 files changed
+282

No files matched your search

+81
View File
@@ -39,6 +39,7 @@ not a pass**, and the summary says so.
| T17 | A real sudo-group user is authorised **without a password** (pkcheck), with a control |
| T18 | Offline-update script/units: bash -n, `systemd-analyze verify`, timer enabled, throttle, no dangling `/system-update` |
| T19 | The preseed still generates a working `/root/vdi-post.sh` that delegates updates |
| T20 | `make-iso.sh` builds a bootable ISO with the preseed baked in (skips without a netinst ISO) |
| T15 | Nothing was left behind: no source file, no cached indexes, no installs, no `__pycache__`, no polkit rule or unit residue |
### The repo build it tests
@@ -117,6 +118,12 @@ Being explicit matters more than the green summary.
`pkcon refresh` uses) and that `package-remove` is *not* granted. Installing a
package from a desktop session without a password is not exercised.
11. **Actually booting the built ISO.** T20 proves the image is structurally
correct and the preseed is on it, but it never boots it in a VM. Whether the
installer runs unattended end to end, and whether the inventory prompt behaves,
is untested. `xorriso` says the boot record is intact; only a real (virtual)
machine says the firmware agrees.
## Two spec'd checks that cannot fail, and what replaced them
This is the most important thing in this file. Two checks that look like tests
@@ -322,6 +329,75 @@ before the run, so "it did not exist and must be gone" is distinguishable from
symlink - it never recurses into a directory. The same restores run in the
`EXIT` trap, so an aborted run still cleans up.
## T20: the ISO builder
T20 needs a real Debian netinst ISO and **does not download one** - a test that
fetches 792 MB before it can start is a test nobody runs. It looks for
`/root/trixie-netinst.iso` (or `NETINST_ISO=/path`), and without one it skips
and lists what it would have checked.
With an ISO present it runs `make-iso.sh` into a temp directory (about 3 seconds)
and asserts:
- It **refuses** to overwrite an existing output, and refuses to write over its
own input. Both are tested by expecting a non-zero exit, not by reading the code.
- `/preseed.cfg` on the output is byte-identical to `debian/preseed.cfg`.
Extracted to a real file first - `-extract ... /dev/stdout` does not work.
- **Every** kernel line under `/isolinux` and `/boot/grub` carries
`preseed/file=/cdrom/preseed.cfg`. Counted, not sampled: on 13.7.0 it is 40
lines (20 BIOS + 20 UEFI), and the assertion is `patched == total`. A
`> 0` check would pass while half the entries still prompt.
- `isolinux/gtk.cfg` specifically is patched. That file holds the BIOS *default*
entry; `isolinux.cfg` is only `include` and `default vesamenu.c32`, so patching
the obvious files leaves the default boot prompting. That is the bug this test
exists to catch, and it is asserted by name rather than left implicit.
- No kernel line carries the parameter **twice** (a re-run of the patch, or a
`-map` applied on top of an already-patched file).
- Both a BIOS and a UEFI El Torito image are present.
- No file from the source ISO is missing from the output (1010 files on 13.7.0).
- Output size is within 16 MB of the source. The measured delta is -704 KB. The
naive extract-and-rebuild implementation bloats 792 MB -> 988 MB because
Debian's ISO shares identical file extents between paths; this guards that
regression.
### `grep -c` exits 1 on zero, and that will corrupt your count
While writing T20 this produced a wrong total: `grep -c ... || echo 0` prints
`0` **and then** appends another `0` when there are no matches, because `grep -c`
prints the count and *also* exits 1. The result is `0\n0`, and `$((total + 0\n0))`
is a syntax error. The harness swallows the exit status only (`|| true`) and
validates the value with a `case` before doing arithmetic.
## Two corrections that are now asserted
**`subject.local` must not come back into the polkit rule.** An earlier version
of the rule included `subject.local == true` to restrict it to a seat physically
in front of the machine. It never matches: `pkcheck` returns 2 with the clause
and 0 without it, for the same user. If it returns, passwordless updates stop
working while every other T16 assertion still passes - so T16 now asserts the
clause is **absent**, with the reason in a comment.
**"the timer is enabled" is not a stable assertion on its own.** Enabling is done
by `systemctl enable vdi-offline-update.timer` inside
`install_debian_update_policy()`, and that only persists when systemd is the
running init. T18 therefore drives the installer function first, then asserts,
and **skips** the assertion with a reason when systemd is not PID 1.
## A killed run leaves apt permanently broken, and the lock cannot help
The harness removes its own apt source in an `EXIT` trap. A run that is killed
(the turn is cancelled, the terminal is closed, `SIGKILL`) never reaches that
trap, so `/etc/apt/sources.list.d/zz-vdi-test-<pid>.list` survives pointing at a
temp directory that no longer exists. Every subsequent `apt-get update` then
fails with `Failed to fetch ... File not found`, and T06/T07 report that as a
port defect. This was observed: a leftover `zz-vdi-test-250.list` from another
run broke three T06 assertions while the port was fine.
The `flock` does not prevent it, because there is no concurrent process - just
residue. So at startup, under the lock, the harness sweeps any
`zz-vdi-test-*.list` whose target directory is gone, and says so. That is safe
precisely because the lock guarantees no peer run is alive at that moment.
## Requirements
- Debian 13 (trixie), root.
@@ -333,6 +409,11 @@ symlink - it never recurses into a directory. The same restores run in the
machine, which is why the harness treats its absence as a skip rather than
installing it itself.
- `gpg` **for T12 only**, and only if you want to re-check the signing decision.
- `xorriso` **for T20 only**. Without it T20 skips with a reason.
- A Debian netinst ISO **for T20 only**, at `/root/trixie-netinst.iso` or
`NETINST_ISO=/path/to/netinst.iso`. It is never downloaded. The build itself
takes about 3 seconds and needs ~1.6 GB free in `TMPDIR` (source plus output;
the source is not copied, only written to the output).
## Safety