tests: strengthen T17's over-grant check, add T21 modes, cross-file contract in T19

T17's package-remove assertion could not fire: it used a sleeper that had already
exited, so pkcheck returned 127 and the check printed 'info' rather than a
verdict. A negative assertion that cannot fail is worse than none. It now owns a
sleeper (setsid + exec -a, pid confirmed via /proc before pkcheck is trusted),
and 127 or any unexpected code is a FAIL rather than a pass. Verified capable of
failing by injecting package-remove into the rule: pkcheck went rc=2 -> rc=0 and
the assertion went red; the rule was then restored (sha256 unchanged).

T21 asserts both scripts are recorded as 100755 in the index. core.fileMode=false
means a Windows checkout silently drops the executable bit, so a fresh Linux
clone got non-executable files.

T19 now reads update_local_repo.OFFLINE_UPDATE_SCRIPT and asserts the preseed
names the same path. Both files hard-code it, and nothing tied them together, so
a rename would leave auto_update.sh printing 'not installed yet' with everything
else green.

Final: 107 passed, 0 failed, 2 skipped. The skips stay honest (T12 needs gpg;
T13's ks.cfg assertion is vacuous because the file is not in the repo).
This commit is contained in:
Mistercoolertyper committed 2026-11-04 10:38:17 +01:00
1 parent e9a447da4d
commit be6379107f
2 files changed
+167 -10

No files matched your search

+50 -1
View File
@@ -40,6 +40,7 @@ not a pass**, and the summary says so.
| T18 | Offline-update script/units: bash -n, `systemd-analyze verify`, timer enabled, throttle, no dangling `/system-update` | | T18 | Offline-update script/units: bash -n, `systemd-analyze verify`, timer enabled, throttle, no dangling `/system-update` |
| T19 | The preseed still generates a working `/root/vdi-post.sh` that delegates updates | | T19 | The preseed still generates a working `/root/vdi-post.sh` that delegates updates |
| T20 | `make-iso.sh` builds a bootable ISO with the preseed baked in (skips without a netinst ISO) | | T20 | `make-iso.sh` builds a bootable ISO with the preseed baked in (skips without a netinst ISO) |
| T21 | The executable bit is recorded as `100755` **in git**, not just on disk |
| T15 | Nothing was left behind: no source file, no cached indexes, no installs, no `__pycache__`, no polkit rule or unit residue | | T15 | Nothing was left behind: no source file, no cached indexes, no installs, no `__pycache__`, no polkit rule or unit residue |
### The repo build it tests ### The repo build it tests
@@ -117,7 +118,6 @@ Being explicit matters more than the green summary.
`package-install`, but T17 only checks `system-sources-refresh` (the action `package-install`, but T17 only checks `system-sources-refresh` (the action
`pkcon refresh` uses) and that `package-remove` is *not* granted. Installing a `pkcon refresh` uses) and that `package-remove` is *not* granted. Installing a
package from a desktop session without a password is not exercised. package from a desktop session without a password is not exercised.
11. **Actually booting the built ISO.** T20 proves the image is structurally 11. **Actually booting the built ISO.** T20 proves the image is structurally
correct and the preseed is on it, but it never boots it in a VM. Whether the correct and the preseed is on it, but it never boots it in a VM. Whether the
installer runs unattended end to end, and whether the inventory prompt behaves, installer runs unattended end to end, and whether the inventory prompt behaves,
@@ -258,6 +258,21 @@ It also asserts the rule does **not** grant `package-remove`, `package-reinstall
or `package-install-untrusted` - a user who can remove packages without a or `package-install-untrusted` - a user who can remove packages without a
password can break the machine. password can break the machine.
That refusal is checked three ways, because a negative assertion is easy to
write in a way that cannot fail:
1. The rule **text** must not mention the action (T16).
2. polkit itself must refuse it: T17 starts its **own** detached sleeper owned by
the sudo-group user, proves `/proc/<pid>` is owned by that user, then asks
`pkcheck --action-id ...package-remove --process <pid>` and requires rc=2.
A missing process or any unexpected rc is a **FAIL**, never a pass - a 127
must not be able to masquerade as "denied".
3. **The check was proven able to fail.** With `package-remove` added to the
rule by hand and polkit restarted, `pkcheck` returned **rc=0** (granted), so
assertion 2 goes red. Restored afterwards and verified by sha256. Without
this step the assertion would only be known to pass, not known to be capable
of failing.
`pkcheck` is only conclusive because it consults the live polkitd. If polkit is `pkcheck` is only conclusive because it consults the live polkitd. If polkit is
not installed, T16 and T17 **skip** with a reason rather than passing quietly. not installed, T16 and T17 **skip** with a reason rather than passing quietly.
@@ -398,6 +413,40 @@ residue. So at startup, under the lock, the harness sweeps any
`zz-vdi-test-*.list` whose target directory is gone, and says so. That is safe `zz-vdi-test-*.list` whose target directory is gone, and says so. That is safe
precisely because the lock guarantees no peer run is alive at that moment. precisely because the lock guarantees no peer run is alive at that moment.
## T21: executable on disk is not executable in a clone
This repo is developed on a Windows checkout with `core.fileMode=false`, so git
records mode `100644` regardless of the mode on disk. Both scripts were
executable locally and non-executable in the index - a fresh Linux clone gets
`Permission denied` running `./debian/test/run-tests.sh`.
T21 asserts `git ls-files -s` reports `100755` for `debian/test/run-tests.sh`
and `debian/make-iso.sh`. It skips honestly when git is unavailable or the
directory is not a checkout. Fix a failure with:
```sh
git update-index --chmod=+x <path>
```
## T19 cross-checks the preseed against the installer
T19 asserts the generated `auto_update.sh` delegates to
`/usr/local/sbin/vdi-offline-update`. That string is hard-coded in the preseed,
while the *installer* of that path lives in `update_local_repo.py` - two
independent files with nothing tying them together. If the installer's path ever
changed, the preseed would keep pointing at the old one and `auto_update.sh`
would degrade to printing `vdi-offline-update not installed yet`:
gracefully, silently, with every other assertion still green.
So T19 reads `update_local_repo.OFFLINE_UPDATE_SCRIPT` and asserts the generated
script names that same path. It skips if that constant does not exist.
What T19 still does **not** prove: that the helper exists and runs on a real
target. It is installed by `update_local_repo.py` at first cron run, so on a
fresh install `auto_update.sh` may legitimately find it absent. That ordering is
by design, not a defect, but it means T19 is evidence about the *contract*
between the two files, not about the end state of an installed machine.
## Requirements ## Requirements
- Debian 13 (trixie), root. - Debian 13 (trixie), root.
+117 -9
View File
@@ -197,7 +197,6 @@ cleanup() {
done done
command -v systemctl >/dev/null 2>&1 && systemctl daemon-reload >/dev/null 2>&1 command -v systemctl >/dev/null 2>&1 && systemctl daemon-reload >/dev/null 2>&1
pkill -f 'vdi-test-sleeper' >/dev/null 2>&1 pkill -f 'vdi-test-sleeper' >/dev/null 2>&1
rm -f "$LIST_FILE" 2>/dev/null rm -f "$LIST_FILE" 2>/dev/null
# Remove any keyring this run installed, by its unique name. # Remove any keyring this run installed, by its unique name.
rm -f "$KEYRING_DIR"/zz-vdi-test-* 2>/dev/null rm -f "$KEYRING_DIR"/zz-vdi-test-* 2>/dev/null
@@ -1208,6 +1207,28 @@ pkcheck_for_user() {
echo "$rc" echo "$rc"
} }
# Start a detached process owned by $1 and ECHO ITS PID (empty on failure).
# Detached with setsid and all three streams redirected so nothing can block on
# a pipe and the process outlives the read from it. A negative pkcheck assertion
# is worthless without a real pid to read, so this is a separate, explicit step
# rather than a reuse of a pid from earlier in the test.
start_sleeper() {
local user="$1" pid
pkill -u "$user" -f 'vdi-test-sleeper' >/dev/null 2>&1
setsid runuser -u "$user" -- bash -c 'exec -a vdi-test-sleeper sleep 180' \
</dev/null >/dev/null 2>&1 &
sleep 1
pid=$(pgrep -u "$user" -f 'vdi-test-sleeper' | head -1)
printf '%s' "${pid:-}"
}
stop_sleeper() {
[ -n "${1:-}" ] || return 0
kill "$1" >/dev/null 2>&1
[ -n "${2:-}" ] && pkill -u "$2" -f 'vdi-test-sleeper' >/dev/null 2>&1
return 0
}
TEST_USER="" TEST_USER=""
if [ "$POLKIT_OK" != "1" ]; then if [ "$POLKIT_OK" != "1" ]; then
skip "T17 skipped with T16: no polkit on this machine" skip "T17 skipped with T16: no polkit on this machine"
@@ -1280,15 +1301,42 @@ u.install_debian_update_policy()" ) >/dev/null 2>&1
;; ;;
esac esac
# The action the rule must NOT grant. # The action the rule must NOT grant. This needs its OWN live process:
denied=$(pkcheck --action-id org.freedesktop.packagekit.package-remove \ # pkcheck --process wants a pid it can read, and reusing a pid from
--process "$(pgrep -u "$TEST_USER" -f 'vdi-test-sleeper' | head -1 || echo 0)" >/dev/null 2>&1; echo $?) # earlier in the test is how a negative assertion silently becomes a
if [ "$denied" = "2" ] || [ "$denied" = "3" ]; then # 127-and-a-shrug. So start one here, prove it exists and is owned by
pass "T17 package-remove still requires authentication (not over-granted)" # the right user, and only then ask polkit.
elif [ "$denied" = "0" ]; then #
fail "T17 package-remove is authorised without a password - the rule over-grants" # The assertion means "our rule does not accidentally grant passwordless
# package removal", so a 127 must NEVER become a pass.
deny_pid=$(start_sleeper "$TEST_USER")
if [ -z "$deny_pid" ]; then
fail "T17 could not start a sleeper for the package-remove check"
elif [ "$(stat -c '%U' "/proc/$deny_pid" 2>/dev/null)" != "$TEST_USER" ]; then
fail "T17 sleeper pid $deny_pid is not owned by $TEST_USER - cannot trust pkcheck"
stop_sleeper "$deny_pid" "$TEST_USER"
else else
info "T17 package-remove pkcheck rc=$denied (no process to check against)" denied=0
pkcheck --action-id org.freedesktop.packagekit.package-remove \
--process "$deny_pid" >/dev/null 2>&1 || denied=$?
stop_sleeper "$deny_pid" "$TEST_USER"
case "$denied" in
0)
fail "T17 package-remove is authorised without a password - the rule over-grants"
;;
2)
pass "T17 package-remove still requires authentication (rc=2, not over-granted)"
;;
3)
pass "T17 package-remove is not authorised for this subject (rc=3)"
;;
*)
# 127, 1, anything else: the check did not actually run.
# Saying PASS here is the vacuous-test failure mode.
fail "T17 package-remove check did not run (pkcheck rc=$denied); not evidence either way"
;;
esac
fi fi
fi fi
fi fi
@@ -1550,6 +1598,25 @@ PYEOF
fail "T19 generated script does not reference the offline-update script" fail "T19 generated script does not reference the offline-update script"
fi fi
# Cross-check the path the preseed hard-codes against the path
# update_local_repo.py actually installs. These are two independent
# files, and nothing else ties them together: if the installer's path
# ever changes, auto_update.sh keeps pointing at the old one and
# degenerates to printing "not installed yet" - gracefully, silently,
# and with every other assertion here still green.
installed_path=$(cd "$REPO_ROOT" && python3 -c "
import sys; sys.path.insert(0, '$REPO_ROOT'); import update_local_repo as u
print(u.OFFLINE_UPDATE_SCRIPT)" 2>/dev/null)
if [ -n "$installed_path" ] && grep -qF "$installed_path" "$GEN"; then
pass "T19 preseed delegates to the same path update_local_repo.py installs ($installed_path)"
elif [ -z "$installed_path" ]; then
skip "T19 update_local_repo has no OFFLINE_UPDATE_SCRIPT constant to cross-check"
else
fail "T19 preseed delegates to a different path than update_local_repo.py installs"
info "preseed references /usr/local/sbin/vdi-offline-update;"
info "update_local_repo.OFFLINE_UPDATE_SCRIPT is $installed_path"
fi
if grep -qE 'systemctl enable cron' "$GEN"; then if grep -qE 'systemctl enable cron' "$GEN"; then
pass "T19 generated script enables cron" pass "T19 generated script enables cron"
else else
@@ -1747,6 +1814,47 @@ else
fi fi
fi fi
# ---------------------------------------------------------------------------
# T21: the files that must be executable are executable IN GIT
# ---------------------------------------------------------------------------
# Being executable on the current filesystem is not the property that matters:
# a fresh clone is what a user gets. This repo is developed on a Windows
# checkout where core.fileMode=false, so git silently records 100644 no matter
# what the file mode is on disk - a fresh Linux clone then gets a
# non-executable harness and `./run-tests.sh` fails with "Permission denied".
# That happened once already; this test is why it cannot happen quietly again.
head2 "T21 executable bit is recorded in git"
if ! command -v git >/dev/null 2>&1; then
skip "T21 git is not installed - cannot check the recorded file modes"
info "would have checked: git ls-files -s reports mode 100755 for the"
info "scripts the port requires to be executable."
elif ! git -C "$REPO_ROOT" rev-parse --git-dir >/dev/null 2>&1; then
skip "T21 $REPO_ROOT is not a git checkout - nothing to compare"
else
for must_exec in debian/test/run-tests.sh debian/make-iso.sh; do
# Only assert for files that exist in the repo at all; make-iso.sh is
# owned by another change and may not be present.
entry=$(git -C "$REPO_ROOT" ls-files -s -- "$must_exec" 2>/dev/null)
if [ -z "$entry" ]; then
if [ -f "$REPO_ROOT/$must_exec" ]; then
skip "T21 $must_exec exists but is not tracked by git"
else
info "T21 $must_exec not present in this checkout"
fi
continue
fi
mode=$(printf '%s' "$entry" | awk '{print $1}')
if [ "$mode" = "100755" ]; then
pass "T21 $must_exec is recorded as 100755 in git"
else
fail "T21 $must_exec is recorded as $mode in git, expected 100755"
info "fix: git update-index --chmod=+x $must_exec"
fi
done
fi
# --------------------------------------------------------------------------- # ---------------------------------------------------------------------------
# T15: the harness left nothing behind # T15: the harness left nothing behind
# --------------------------------------------------------------------------- # ---------------------------------------------------------------------------