tests: cover polkit rule, offline-update timer, preseed generation and ISO build
Adds T16-T20 to the Debian harness (103 passed, 0 failed, 2 skipped):
T16 the polkit rule is installed, mode 644, covers the two update actions,
does not over-grant package-remove/reinstall/untrusted, and is idempotent.
T17 the real decision, in both directions: a sudo-group user is authorised
(pkcheck rc=0) with the rule, and still prompted (rc=2) without it. The
control is the point - without it the test cannot fail.
T18 vdi-offline-update: bash -n, unit verification, the throttle stamp, that
a second run is skipped, that a stale stamp lets it run, that pkcon exit 5
is reported as 'no updates available' rather than a download failure, and
that /system-update is never armed without a prepared payload.
T19 the preseed still generates a script that parses, delegates to
vdi-offline-update instead of an inline pkcon sequence, and enables cron
and the timer.
T20 make-iso.sh: refuses to overwrite its input or an existing output, then
checks the built image - preseed byte-identical, all 40 kernel lines
patched, gtk.cfg patched, no doubled parameters, El Torito present with
both BIOS and UEFI images, no source file missing, and size within 16MB of
the original (the bloat regression).
Skips stay honest: T12 needs gpg, T13's ks.cfg assertion is vacuous because
the file was never in the repo.
This commit is contained in:
1 parent
15d65183dc
commit
402fea66fd
2 files changed
+282
No files matched your search
Vendored
+81
@@ -39,6 +39,7 @@ not a pass**, and the summary says so.
|
||||
| T17 | A real sudo-group user is authorised **without a password** (pkcheck), with a control |
|
||||
| T18 | Offline-update script/units: bash -n, `systemd-analyze verify`, timer enabled, throttle, no dangling `/system-update` |
|
||||
| T19 | The preseed still generates a working `/root/vdi-post.sh` that delegates updates |
|
||||
| T20 | `make-iso.sh` builds a bootable ISO with the preseed baked in (skips without a netinst ISO) |
|
||||
| T15 | Nothing was left behind: no source file, no cached indexes, no installs, no `__pycache__`, no polkit rule or unit residue |
|
||||
|
||||
### The repo build it tests
|
||||
@@ -117,6 +118,12 @@ Being explicit matters more than the green summary.
|
||||
`pkcon refresh` uses) and that `package-remove` is *not* granted. Installing a
|
||||
package from a desktop session without a password is not exercised.
|
||||
|
||||
11. **Actually booting the built ISO.** T20 proves the image is structurally
|
||||
correct and the preseed is on it, but it never boots it in a VM. Whether the
|
||||
installer runs unattended end to end, and whether the inventory prompt behaves,
|
||||
is untested. `xorriso` says the boot record is intact; only a real (virtual)
|
||||
machine says the firmware agrees.
|
||||
|
||||
## Two spec'd checks that cannot fail, and what replaced them
|
||||
|
||||
This is the most important thing in this file. Two checks that look like tests
|
||||
@@ -322,6 +329,75 @@ before the run, so "it did not exist and must be gone" is distinguishable from
|
||||
symlink - it never recurses into a directory. The same restores run in the
|
||||
`EXIT` trap, so an aborted run still cleans up.
|
||||
|
||||
## T20: the ISO builder
|
||||
|
||||
T20 needs a real Debian netinst ISO and **does not download one** - a test that
|
||||
fetches 792 MB before it can start is a test nobody runs. It looks for
|
||||
`/root/trixie-netinst.iso` (or `NETINST_ISO=/path`), and without one it skips
|
||||
and lists what it would have checked.
|
||||
|
||||
With an ISO present it runs `make-iso.sh` into a temp directory (about 3 seconds)
|
||||
and asserts:
|
||||
|
||||
- It **refuses** to overwrite an existing output, and refuses to write over its
|
||||
own input. Both are tested by expecting a non-zero exit, not by reading the code.
|
||||
- `/preseed.cfg` on the output is byte-identical to `debian/preseed.cfg`.
|
||||
Extracted to a real file first - `-extract ... /dev/stdout` does not work.
|
||||
- **Every** kernel line under `/isolinux` and `/boot/grub` carries
|
||||
`preseed/file=/cdrom/preseed.cfg`. Counted, not sampled: on 13.7.0 it is 40
|
||||
lines (20 BIOS + 20 UEFI), and the assertion is `patched == total`. A
|
||||
`> 0` check would pass while half the entries still prompt.
|
||||
- `isolinux/gtk.cfg` specifically is patched. That file holds the BIOS *default*
|
||||
entry; `isolinux.cfg` is only `include` and `default vesamenu.c32`, so patching
|
||||
the obvious files leaves the default boot prompting. That is the bug this test
|
||||
exists to catch, and it is asserted by name rather than left implicit.
|
||||
- No kernel line carries the parameter **twice** (a re-run of the patch, or a
|
||||
`-map` applied on top of an already-patched file).
|
||||
- Both a BIOS and a UEFI El Torito image are present.
|
||||
- No file from the source ISO is missing from the output (1010 files on 13.7.0).
|
||||
- Output size is within 16 MB of the source. The measured delta is -704 KB. The
|
||||
naive extract-and-rebuild implementation bloats 792 MB -> 988 MB because
|
||||
Debian's ISO shares identical file extents between paths; this guards that
|
||||
regression.
|
||||
|
||||
### `grep -c` exits 1 on zero, and that will corrupt your count
|
||||
|
||||
While writing T20 this produced a wrong total: `grep -c ... || echo 0` prints
|
||||
`0` **and then** appends another `0` when there are no matches, because `grep -c`
|
||||
prints the count and *also* exits 1. The result is `0\n0`, and `$((total + 0\n0))`
|
||||
is a syntax error. The harness swallows the exit status only (`|| true`) and
|
||||
validates the value with a `case` before doing arithmetic.
|
||||
|
||||
## Two corrections that are now asserted
|
||||
|
||||
**`subject.local` must not come back into the polkit rule.** An earlier version
|
||||
of the rule included `subject.local == true` to restrict it to a seat physically
|
||||
in front of the machine. It never matches: `pkcheck` returns 2 with the clause
|
||||
and 0 without it, for the same user. If it returns, passwordless updates stop
|
||||
working while every other T16 assertion still passes - so T16 now asserts the
|
||||
clause is **absent**, with the reason in a comment.
|
||||
|
||||
**"the timer is enabled" is not a stable assertion on its own.** Enabling is done
|
||||
by `systemctl enable vdi-offline-update.timer` inside
|
||||
`install_debian_update_policy()`, and that only persists when systemd is the
|
||||
running init. T18 therefore drives the installer function first, then asserts,
|
||||
and **skips** the assertion with a reason when systemd is not PID 1.
|
||||
|
||||
## A killed run leaves apt permanently broken, and the lock cannot help
|
||||
|
||||
The harness removes its own apt source in an `EXIT` trap. A run that is killed
|
||||
(the turn is cancelled, the terminal is closed, `SIGKILL`) never reaches that
|
||||
trap, so `/etc/apt/sources.list.d/zz-vdi-test-<pid>.list` survives pointing at a
|
||||
temp directory that no longer exists. Every subsequent `apt-get update` then
|
||||
fails with `Failed to fetch ... File not found`, and T06/T07 report that as a
|
||||
port defect. This was observed: a leftover `zz-vdi-test-250.list` from another
|
||||
run broke three T06 assertions while the port was fine.
|
||||
|
||||
The `flock` does not prevent it, because there is no concurrent process - just
|
||||
residue. So at startup, under the lock, the harness sweeps any
|
||||
`zz-vdi-test-*.list` whose target directory is gone, and says so. That is safe
|
||||
precisely because the lock guarantees no peer run is alive at that moment.
|
||||
|
||||
## Requirements
|
||||
|
||||
- Debian 13 (trixie), root.
|
||||
@@ -333,6 +409,11 @@ symlink - it never recurses into a directory. The same restores run in the
|
||||
machine, which is why the harness treats its absence as a skip rather than
|
||||
installing it itself.
|
||||
- `gpg` **for T12 only**, and only if you want to re-check the signing decision.
|
||||
- `xorriso` **for T20 only**. Without it T20 skips with a reason.
|
||||
- A Debian netinst ISO **for T20 only**, at `/root/trixie-netinst.iso` or
|
||||
`NETINST_ISO=/path/to/netinst.iso`. It is never downloaded. The build itself
|
||||
takes about 3 seconds and needs ~1.6 GB free in `TMPDIR` (source plus output;
|
||||
the source is not copied, only written to the output).
|
||||
|
||||
## Safety
|
||||
|
||||
|
||||
Vendored
+201
@@ -236,6 +236,26 @@ for tool in apt-ftparchive apt-get apt-cache dpkg-deb python3 gzip; do
|
||||
done
|
||||
|
||||
# Clear indexes from any earlier run before doing anything else.
|
||||
#
|
||||
# A run that was killed (SIGKILL, a cancelled turn, a closed terminal) never
|
||||
# reaches its EXIT trap, so its /etc/apt/sources.list.d/zz-vdi-test-<pid>.list
|
||||
# survives pointing at a temp directory that no longer exists. Every later
|
||||
# `apt-get update` then fails with "Failed to fetch ... File not found" and
|
||||
# T06/T07 blame the port for the wreckage - measured: exactly that happened
|
||||
# with a leftover zz-vdi-test-250.list from another run.
|
||||
#
|
||||
# This is safe to sweep because flock above guarantees no other run is alive,
|
||||
# so any zz-vdi-test-* source file at this point is residue, not a peer.
|
||||
for stale_list in /etc/apt/sources.list.d/zz-vdi-test-*.list; do
|
||||
[ -e "$stale_list" ] || continue
|
||||
stale_dir=$(sed -n 's|.*file:\([^ ]*\).*|\1|p' "$stale_list" 2>/dev/null)
|
||||
if [ -z "$stale_dir" ] || [ ! -d "$stale_dir" ]; then
|
||||
rm -f "$stale_list"
|
||||
printf ' removed stale apt source from a killed run: %s\n' \
|
||||
"$(basename "$stale_list")"
|
||||
fi
|
||||
done
|
||||
|
||||
purge_all_harness_lists
|
||||
|
||||
CODENAME=""
|
||||
@@ -1546,6 +1566,187 @@ PYEOF
|
||||
fi
|
||||
fi
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# T20: make-iso.sh produces a bootable, correctly-preseeded ISO
|
||||
# ---------------------------------------------------------------------------
|
||||
# Needs a real Debian netinst ISO. It is NOT downloaded here: a test that
|
||||
# fetches 792 MB from the network is a test nobody runs. Without the ISO this
|
||||
# skips and says what it would have checked.
|
||||
|
||||
head2 "T20 make-iso.sh builds a preseeded bootable ISO"
|
||||
|
||||
MAKE_ISO="$REPO_ROOT/debian/make-iso.sh"
|
||||
NETINST=""
|
||||
for candidate in "${NETINST_ISO:-}" /root/trixie-netinst.iso /root/debian-netinst.iso; do
|
||||
[ -n "$candidate" ] && [ -f "$candidate" ] && { NETINST="$candidate"; break; }
|
||||
done
|
||||
|
||||
if [ ! -f "$MAKE_ISO" ]; then
|
||||
skip "T20 no debian/make-iso.sh in this repo (owned by another change)"
|
||||
elif ! command -v xorriso >/dev/null 2>&1; then
|
||||
skip "T20 xorriso is not installed - make-iso.sh cannot run"
|
||||
info "apt-get install xorriso"
|
||||
elif [ -z "$NETINST" ]; then
|
||||
skip "T20 no Debian netinst ISO found (looked for /root/trixie-netinst.iso)"
|
||||
info "would have checked: preseed on the image is byte-identical, every kernel"
|
||||
info "line carries preseed/file=, gtk.cfg is patched, both El Torito images"
|
||||
info "exist, no source file is missing, size is within a few MB of the source."
|
||||
info "Point NETINST_ISO=/path/to/netinst.iso at one to run it."
|
||||
else
|
||||
info "source ISO: $NETINST ($(du -h "$NETINST" | cut -f1))"
|
||||
|
||||
if bash -n "$MAKE_ISO" 2>"$TMP_DIR/isoerr"; then
|
||||
pass "T20 make-iso.sh passes bash -n"
|
||||
else
|
||||
fail "T20 make-iso.sh fails bash -n: $(cat "$TMP_DIR/isoerr" | tr '\n' ' ')"
|
||||
fi
|
||||
|
||||
ISOWORK="$TMP_DIR/iso"
|
||||
mkdir -p "$ISOWORK"
|
||||
OUT_ISO="$ISOWORK/out.iso"
|
||||
|
||||
# Guard: it must refuse to overwrite an existing output, and refuse to
|
||||
# write over its own input. Both are destructive if they get it wrong.
|
||||
: > "$OUT_ISO"
|
||||
if bash "$MAKE_ISO" "$NETINST" "$OUT_ISO" >/dev/null 2>&1; then
|
||||
fail "T20 make-iso.sh overwrote an existing output file (should refuse)"
|
||||
else
|
||||
pass "T20 make-iso.sh refuses to overwrite an existing output"
|
||||
fi
|
||||
rm -f "$OUT_ISO"
|
||||
|
||||
if bash "$MAKE_ISO" "$NETINST" "$NETINST" >/dev/null 2>&1; then
|
||||
fail "T20 make-iso.sh accepted output == input (would destroy the source)"
|
||||
else
|
||||
pass "T20 make-iso.sh refuses to write over its input ISO"
|
||||
fi
|
||||
|
||||
# The real build.
|
||||
build_start=$(date +%s)
|
||||
if build_out=$(bash "$MAKE_ISO" "$NETINST" "$OUT_ISO" 2>&1); then
|
||||
build_rc=0
|
||||
else
|
||||
build_rc=$?
|
||||
fi
|
||||
build_secs=$(( $(date +%s) - build_start ))
|
||||
|
||||
if [ "$build_rc" -eq 0 ] && [ -s "$OUT_ISO" ]; then
|
||||
pass "T20 make-iso.sh built an ISO (${build_secs}s, $(du -h "$OUT_ISO" | cut -f1))"
|
||||
else
|
||||
fail "T20 make-iso.sh failed (rc=$build_rc)"
|
||||
printf '%s\n' "$build_out" | tail -12 | sed 's/^/ /'
|
||||
fi
|
||||
|
||||
if [ -s "$OUT_ISO" ]; then
|
||||
# ---- preseed is on the image and byte-identical ----
|
||||
# Note: `-extract /preseed.cfg /dev/stdout` does not work; extract to a
|
||||
# real file.
|
||||
if xorriso -osirrox on -indev "$OUT_ISO" -extract /preseed.cfg \
|
||||
"$ISOWORK/preseed.on.iso" >/dev/null 2>&1; then
|
||||
if cmp -s "$ISOWORK/preseed.on.iso" "$REPO_ROOT/debian/preseed.cfg"; then
|
||||
pass "T20 /preseed.cfg on the ISO is byte-identical to debian/preseed.cfg"
|
||||
else
|
||||
fail "T20 /preseed.cfg on the ISO differs from debian/preseed.cfg"
|
||||
fi
|
||||
else
|
||||
fail "T20 /preseed.cfg is not present on the output ISO"
|
||||
fi
|
||||
|
||||
# ---- every kernel line is patched, counted not sampled ----
|
||||
menu_list=$( for dir in /isolinux /boot/grub; do
|
||||
xorriso -indev "$NETINST" -find "$dir" -name '*.cfg' 2>/dev/null
|
||||
done | tr -d "'" | grep -v '^$' | sort -u )
|
||||
|
||||
total_k=0; patched_k=0; unpatched_list=""; dup_list=""
|
||||
for menu in $menu_list; do
|
||||
local_file="$ISOWORK/menu_$(printf '%s' "$menu" | tr '/' '_')"
|
||||
xorriso -osirrox on -indev "$OUT_ISO" -extract "/$menu" "$local_file" \
|
||||
>/dev/null 2>&1 || continue
|
||||
# grep -c prints 0 AND exits 1 when there are no matches, so
|
||||
# `|| echo 0` would append a second zero. Swallow the status only.
|
||||
k=$(grep -cE '^[[:space:]]*(append|linux)[[:space:]]' "$local_file" 2>/dev/null || true)
|
||||
case "$k" in ''|*[!0-9]*) k=0 ;; esac
|
||||
p=$(grep -E '^[[:space:]]*(append|linux)[[:space:]]' "$local_file" 2>/dev/null \
|
||||
| grep -cF 'preseed/file=/cdrom/preseed.cfg' || true)
|
||||
case "$p" in ''|*[!0-9]*) p=0 ;; esac
|
||||
total_k=$((total_k + k))
|
||||
patched_k=$((patched_k + p))
|
||||
[ "$k" -gt 0 ] && [ "$p" -ne "$k" ] && unpatched_list="$unpatched_list $menu($p/$k)"
|
||||
# The parameter must appear at most once per kernel line.
|
||||
dups=$(grep -E '^[[:space:]]*(append|linux)[[:space:]]' "$local_file" 2>/dev/null \
|
||||
| awk '{n=gsub(/preseed\/file=/,"&"); if (n>1) print}' | wc -l)
|
||||
[ "$dups" -gt 0 ] && dup_list="$dup_list $menu"
|
||||
done
|
||||
|
||||
if [ "$total_k" -gt 0 ] && [ "$patched_k" -eq "$total_k" ]; then
|
||||
pass "T20 all $total_k kernel lines carry preseed/file= ($patched_k patched)"
|
||||
else
|
||||
fail "T20 only $patched_k of $total_k kernel lines are patched"
|
||||
[ -n "$unpatched_list" ] && info "unpatched:$unpatched_list"
|
||||
fi
|
||||
|
||||
# The BIOS *default* entry lives in gtk.cfg, not isolinux.cfg. Patching
|
||||
# only the obvious files leaves the machine prompting, which is the bug
|
||||
# this test exists to catch.
|
||||
gtk_file="$ISOWORK/menu__isolinux_gtk.cfg"
|
||||
if [ -f "$gtk_file" ]; then
|
||||
gtk_k=$(grep -cE '^[[:space:]]*(append|linux)[[:space:]]' "$gtk_file" 2>/dev/null || true)
|
||||
case "$gtk_k" in ''|*[!0-9]*) gtk_k=0 ;; esac
|
||||
if [ "$gtk_k" -gt 0 ] && grep -qE '^[[:space:]]*(append|linux)[[:space:]].*preseed/file=' "$gtk_file"; then
|
||||
pass "T20 isolinux/gtk.cfg is patched (the BIOS default entry)"
|
||||
else
|
||||
fail "T20 isolinux/gtk.cfg is NOT patched - the BIOS default would still prompt"
|
||||
fi
|
||||
else
|
||||
info "T20 isolinux/gtk.cfg not present on this ISO (layout differs)"
|
||||
fi
|
||||
|
||||
if [ -z "$dup_list" ]; then
|
||||
pass "T20 no kernel line carries the parameter twice"
|
||||
else
|
||||
fail "T20 duplicated parameter in:$dup_list"
|
||||
fi
|
||||
|
||||
# ---- both El Torito images survive ----
|
||||
torito=$(xorriso -indev "$OUT_ISO" -report_el_torito plain 2>&1)
|
||||
if printf '%s' "$torito" | grep -q 'El Torito'; then
|
||||
pass "T20 output ISO has an El Torito boot record"
|
||||
else
|
||||
fail "T20 output ISO has no El Torito boot record"
|
||||
fi
|
||||
if printf '%s' "$torito" | grep -q 'BIOS' && printf '%s' "$torito" | grep -q 'UEFI'; then
|
||||
pass "T20 both BIOS and UEFI boot images are present"
|
||||
else
|
||||
fail "T20 only one firmware type is bootable in the output ISO"
|
||||
fi
|
||||
|
||||
# ---- no source file is missing ----
|
||||
xorriso -indev "$NETINST" -find / -type f -exec lsdl 2>/dev/null \
|
||||
| awk '{print $NF}' | sort > "$ISOWORK/src.list"
|
||||
xorriso -indev "$OUT_ISO" -find / -type f -exec lsdl 2>/dev/null \
|
||||
| awk '{print $NF}' | sort > "$ISOWORK/out.list"
|
||||
missing_n=$(comm -23 "$ISOWORK/src.list" "$ISOWORK/out.list" | wc -l)
|
||||
src_n=$(wc -l < "$ISOWORK/src.list")
|
||||
if [ "$src_n" -gt 0 ] && [ "$missing_n" -eq 0 ]; then
|
||||
pass "T20 all $src_n source files are present in the output"
|
||||
else
|
||||
fail "T20 $missing_n of $src_n source files are missing from the output"
|
||||
comm -23 "$ISOWORK/src.list" "$ISOWORK/out.list" | head -5 | sed 's/^/ /'
|
||||
fi
|
||||
|
||||
# ---- size guard: the extract-and-rebuild approach bloated 792->988 MB ----
|
||||
src_bytes=$(stat -c '%s' "$NETINST")
|
||||
out_bytes=$(stat -c '%s' "$OUT_ISO")
|
||||
delta=$(( out_bytes - src_bytes ))
|
||||
if [ "$delta" -lt 0 ]; then abs_delta=$(( -delta )); else abs_delta=$delta; fi
|
||||
if [ "$abs_delta" -le 16777216 ]; then
|
||||
pass "T20 output size is within 16 MB of the source (delta ${delta} bytes)"
|
||||
else
|
||||
fail "T20 output size differs by $delta bytes - the extent-dedup regression is back"
|
||||
fi
|
||||
fi
|
||||
fi
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# T15: the harness left nothing behind
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
Reference in new issue
Block a user