# Debian port test harness `run-tests.sh` proves that the Debian side of the VDI-Linux port actually works, and that the Fedora side has not regressed. It is a real harness: it builds a real apt repository from the real Citrix `.deb` packages, adds a real apt source, and asks real apt whether it can install the result. Nothing is simulated except the parts that would change the machine. ```sh sudo bash debian/test/run-tests.sh ``` Run as root. It needs the two Citrix `.deb` files (see [Fixtures](#fixtures)). Exit status is 0 when no test FAILED, 1 otherwise. A SKIP does not fail the run, but every skip prints why it skipped and what it would have checked. **A skip is not a pass**, and the summary says so. ## What it tests | Test | What it proves | |------|----------------| | T00 | Environment: root, required tools, trixie, fixtures present | | T01 | The `.deb` fixtures really are version `26.08.0.225` | | T02 | `python3 -m py_compile` passes for `update_local_repo.py` and `configure_kde.py` | | T03 | Both modules **import** cleanly on Debian (no `tqdm`, no `pip`) | | T04 | The `tqdm` fallback is usable, not merely importable | | T05 | The flat repo builds; `Packages` has exactly 2 stanzas at the right version | | T06 | `apt-get update` exits 0 with no `E:`, no `Err:` and no warnings for the source | | T07 | apt can *see* the package: correct candidate, correct origin, install resolves | | T08 | Rebuilding twice is byte-identical and produces no duplicate stanzas | | T09 | `sync_debian` prunes stale `.deb` files (version bump cannot leave two versions) | | T10 | A corrupted index is rejected even with `trusted=yes` (control for T06/T07) | | T11 | The production `local-citrix-repo.list` has the shape the harness verified | | T12 | GPG `signed-by` path (design check; skips without a signing tool) | | T13 | Fedora path unregressed: tracked files identical, `sync_fedora` command sequence identical | | T14 | `bash -n` on every shell script the port adds | | T16 | polkit rule installed, mode 644, correct actions, no over-grant, balanced, idempotent | | T17 | A real sudo-group user is authorised **without a password** (pkcheck), with a control | | T18 | Offline-update script/units: bash -n, `systemd-analyze verify`, timer enabled, throttle, no dangling `/system-update` | | T19 | The preseed still generates a working `/root/vdi-post.sh` that delegates updates | | T20 | `make-iso.sh` builds a bootable ISO with the preseed baked in (skips without a netinst ISO) | | T15 | Nothing was left behind: no source file, no cached indexes, no installs, no `__pycache__`, no polkit rule or unit residue | ### The repo build it tests The exact production sequence, on a copy of the real `.deb` files in a temp directory: ```sh cd "$repo" apt-ftparchive packages . > Packages gzip -c Packages > Packages.gz apt-ftparchive release . > Release ``` and the exact production source line: ``` deb [trusted=yes] file:/var/local/citrix-repo ./ ``` with the path substituted for the temp directory. A flat ("trivial") archive: no `dists/` tree, suite `./`. ## What it does NOT test, and cannot Being explicit matters more than the green summary. 1. **A real d-i install.** The harness never boots an installer. `debian/preseed.cfg` is owned and validated by another change; this harness does not test it at all. Note that `debconf-set-selections` validates line shape only - it silently accepts an invented question name - so it is not evidence that directives exist. 2. **An actual GUI session.** Nothing here starts Plasma, and no test asserts that `configure_kde.py` produces a working desktop. T03 proves it *imports*; whether the touchpad and power settings take effect on a real session is untested. `kwriteconfig6` is not installed here, so `configure_kde.KWRITE` is `None` and every write would no-op. That path needs a real Plasma machine. 3. **The Citrix preinst/postinst maintainer scripts.** `apt-get install --simulate` resolves dependencies; it does not run the package's scripts. Whether `icaclient`'s postinst succeeds on a real machine is untested. This is the single biggest gap: a dependency-resolvable package can still fail to configure. 4. **A real d-i/target filesystem for the repo path.** The harness builds the repo in a temp directory, never at `/var/local/citrix-repo`. It asserts the *shape* is identical (T11), not that the production path is writable in a target environment. 5. **The network path.** The Gitea release API and the download code are not exercised. T09 redirects the source files to a local directory, so `download_file` is only exercised indirectly. A changed release asset name would not be caught here. 6. **`apt-get update` inside a d-i chroot.** The preseed's `late_command` runs in an `in-target` context where apt behaves differently (no `_apt` user, different permissions). Untested. 7. **Fedora at runtime.** T13 compares source and command sequences; it does not run `createrepo_c`/`dnf`/`rpm` on Fedora. The command sequence matching is strong evidence but it is not an execution test. A Fedora 44 WSL distro exists on the development machine and was not used. 8. **A real desktop session.** T17 proves polkit authorises a sudo-group user without a password, which is the actual requirement, but it does not open a GUI and click an update. `pkcon refresh` from a live Plasma session, with an authentication agent running, is not exercised. 9. **An actual offline update reboot.** T18 checks that `/system-update` is armed with an existing target and that `systemd-analyze verify` is clean. It does not reboot and let `packagekit-offline-update.service` apply anything. Whether the update actually installs at the next boot is untested - only that it was correctly prepared. 10. **`package-install` end to end.** The polkit rule also grants `package-install`, but T17 only checks `system-sources-refresh` (the action `pkcon refresh` uses) and that `package-remove` is *not* granted. Installing a package from a desktop session without a password is not exercised. 11. **Actually booting the built ISO.** T20 proves the image is structurally correct and the preseed is on it, but it never boots it in a VM. Whether the installer runs unattended end to end, and whether the inventory prompt behaves, is untested. `xorriso` says the boot record is intact; only a real (virtual) machine says the firmware agrees. ## Two spec'd checks that cannot fail, and what replaced them This is the most important thing in this file. Two checks that look like tests are not tests. ### `apt-get update` exiting 0 with no `E:` lines Measured on Debian 13: a repo whose `Packages` is **unreadable** produces ``` Err:4 file:/tmp/.../repo ./ Packages Could not open file .../Packages - open (13: Permission denied) ``` and `apt-get update` still **exits 0 with no `E:` lines**. The same is true of a repo with **no `Release` file at all**, and of a repo with **duplicate stanzas**. So "exit 0 and no `E:`" passes on a broken repo. The harness runs the spec'd check as T06 *and* the stronger check separately: no `Err:` lines, no `W:` lines for that source, and then T07 requires apt to name the exact version from that exact directory. T10 is the control that proves the strong checks can fail: it corrupts `Packages` after `Release` was built and asserts apt **rejects** it with a hash mismatch. Without T10, T06/T07 could be passing vacuously. ### `git diff main -- ks.cfg` `ks.cfg` is **not in the repository**. It is not in `main`, not in `debian-port`, and `git log --all -- ks.cfg` is empty. It lives one level above the repo, in the parent directory, untracked. So `git diff main -- ks.cfg` exits 0 and prints nothing - because the path does not exist on either side, not because the file is unchanged. The harness detects this with `git cat-file -e main:ks.cfg` and reports it as **SKIP, explicitly labelled VACUOUS**, rather than as a pass. It then prints the sha256 of the real out-of-tree file so a change to it is at least visible. ## Line endings will fool you This repo is checked out on Windows. Every tracked file in the working tree has CRLF line endings; every blob in git has LF. A plain `git diff main` therefore reports **all five files as completely rewritten**, including `RPM-GPG-KEY-citrix-local` with 104 changed lines - the key itself is byte-identical. The harness compares modulo CRLF (`tr -d '\r' | cmp`). If you see a huge diff, check line endings before believing it. ## The apt index cache will fool you `apt-cache policy` and `apt-get update` read cached indexes in `/var/lib/apt/lists`. During development, a **wrong GPG key** appeared to be accepted - `exit 0`, no `E:`, package still visible - purely because a good index was cached from an earlier run. Every case that must fail purges that repo's cached index files first (`purge_apt_lists_for`). Any test that checks "can apt see the package" without purging is testing the cache, not apt. Related: `/tmp` is wiped between separate `wsl` invocations on this machine (`systemd-tmpfiles`). Anything expected to survive across invocations must live in `/root`. ## GPG signing: why `trusted=yes` is the right call Production does not sign the Debian repo, and T12 exists to check that decision rather than assume it. A stock trixie ships **no `gpg`** - only `/usr/bin/sqv`, the Sequoia verifier that apt calls internally. So signing would add a `gpg` dependency that Debian 13 does not have by default, in an installer environment where new dependencies are the fragile thing. The signing path was measured out of band (a throwaway key generated with Fedora WSL's `gpg`, because Debian has none), with the apt cache purged between cases: | Keyring | Result | |---------|--------| | armored `.asc`, correct key | exit 0, no `E:`, package visible | | armored `.asc`, **wrong** key | exit 100, `E:`, `Missing key`, not visible | | binary `.gpg`, correct key | exit 0, no `E:`, package visible | | binary `.gpg`, **wrong** key | exit 100, `E:`, `Missing key`, not visible | | `trusted=yes`, no signature | exit 0, no `E:`, package visible | Two conclusions: - `signed-by` **works correctly** on a flat `file:` repo on trixie, for both keyring formats. It is not broken; it is simply not worth it here. - `trusted=yes` does **not** mean "no integrity checking". T10 shows a tampered `Packages` is rejected with `E: ... Hash Sum mismatch` regardless. Signing would only add proof of *authorship*, which is meaningless for a repository the machine builds from packages it just downloaded and verified itself. T12 skips loudly when no signing tool is present, which is the normal case on a stock trixie. Install `gpg` to make it run. ## Fixtures The two real packages are expected next to the repo, not inside it: ``` LinuxISO/ icaclient_26.08.0.225_amd64.deb (256 MB) ctxusb_26.08.0.225_amd64.deb (178 KB) VDI-Linux/ <- the repo debian/test/run-tests.sh <- this harness ``` Override with `DEB_DIR=/path/to/debs sudo -E bash debian/test/run-tests.sh`. The 256 MB file is copied into a temp directory, which is fast on native filesystem and slow off `/mnt/c`. If `/tmp` is small, set `TMPDIR` to somewhere with ~600 MB free. ## T17 is the test that matters, and why it needs a control The user's requirement is that a desktop user can update **without an admin password**. Asserting that the rule file contains the right action names proves nothing about whether polkit acts on it - a rule can be syntactically perfect, in the right directory, and still never match (which is exactly what happened during development with a `subject.local == true` clause). So T17 asks polkit itself: 1. Start a process owned by a real member of group `sudo` (`setsid`, so nothing blocks), then `pkcheck --action-id ...system-sources-refresh --process ` as root. Assert **rc=0**. 2. **The control:** move the rule away, restart polkit, repeat. Assert **rc=2** (authentication required). Then restore. Without step 2 the test cannot fail: a polkit configured to answer YES to everything would also give rc=0. The control is what makes step 1 evidence. It also asserts the rule does **not** grant `package-remove`, `package-reinstall` or `package-install-untrusted` - a user who can remove packages without a password can break the machine. `pkcheck` is only conclusive because it consults the live polkitd. If polkit is not installed, T16 and T17 **skip** with a reason rather than passing quietly. ## T18: the dangerous failure mode A dangling `/system-update` symlink **fails the next boot**. The offline-update script arms that symlink, so the harness asserts, after every run, that if `/system-update` exists then its `readlink -f` target exists too. It is checked both immediately after the run and again after the forced (stamp=0) run, which is the one most likely to arm it. Also asserted, because it is a real behaviour and not a detail: - Running the script twice in a row is **throttled** (prints "skipping"). - Setting the stamp to `0` defeats the throttle - so it is time-based, not a permanent latch. - `pkcon` exit 5 means "no packages to update" and must not be reported as `download failed`. On an up-to-date box the output must not contain "download failed". ## T19: why it never executes the generated script The preseed's `late_command` writes `/root/vdi-post.sh` and then runs it with `in-target`. That script writes `/etc/hostname`, `/etc/hosts`, `/etc/crontab` and systemd units **by absolute path** - running it here would reconfigure the machine the tests are running on. T19 therefore only *regenerates* the script by running the preseed's own `printf` into a temp file, then checks it: `bash -n`, that it delegates to `/usr/local/sbin/vdi-offline-update` instead of duplicating the `pkcon` sequence (comments stripped first, so the explanatory comment does not produce a false positive), and that it still enables `cron` and `vdi-offline-update.timer`. Extracting it is fiddly, and the naive approaches are all wrong in ways that look like preseed bugs: - A regex that crosses `;` captures the **firefox** `printf` too, which then tries to write `/target/etc/firefox` and fails. - Splitting on `;` breaks, because the payload contains `;` and `>` inside quoted arguments. - Leaving the leading `\` of the first continuation line makes `sh -c` skip the `printf` entirely, producing a "No such file" that looks exactly like a preseed defect. The harness anchors on two unambiguous markers instead: the payload's own first argument `'#!/bin/bash'` and the `> /target/root/vdi-post.sh` redirect. It also asserts the captured text does **not** contain `ManagedBookmarks`, so a future change to the preseed fails loudly rather than silently testing the wrong block. ## Tests that mutate the machine, and how that is undone T16-T19 install a polkit rule and systemd units and can arm `/system-update`. T15 asserts no collateral damage, so every mutated path is **backed up before the test and restored at teardown**: | Path | How it is handled | |------|-------------------| | `/etc/polkit-1/rules.d/60-vdi-no-password-updates.rules` | backed up, restored | | `/usr/local/sbin/vdi-offline-update` | backed up, restored | | `/etc/systemd/system/vdi-offline-update.{service,timer}` | backed up, restored | | `/var/lib/vdi-offline-update.last` | backed up, restored | | `/system-update` | backed up as a symlink target, restored; **never `rm -rf`** | `restore_path` records whether a path was `present`, `absent` or a `symlink` before the run, so "it did not exist and must be gone" is distinguishable from "the backup failed". For an absent path it removes only a regular file or symlink - it never recurses into a directory. The same restores run in the `EXIT` trap, so an aborted run still cleans up. ## T20: the ISO builder T20 needs a real Debian netinst ISO and **does not download one** - a test that fetches 792 MB before it can start is a test nobody runs. It looks for `/root/trixie-netinst.iso` (or `NETINST_ISO=/path`), and without one it skips and lists what it would have checked. With an ISO present it runs `make-iso.sh` into a temp directory (about 3 seconds) and asserts: - It **refuses** to overwrite an existing output, and refuses to write over its own input. Both are tested by expecting a non-zero exit, not by reading the code. - `/preseed.cfg` on the output is byte-identical to `debian/preseed.cfg`. Extracted to a real file first - `-extract ... /dev/stdout` does not work. - **Every** kernel line under `/isolinux` and `/boot/grub` carries `preseed/file=/cdrom/preseed.cfg`. Counted, not sampled: on 13.7.0 it is 40 lines (20 BIOS + 20 UEFI), and the assertion is `patched == total`. A `> 0` check would pass while half the entries still prompt. - `isolinux/gtk.cfg` specifically is patched. That file holds the BIOS *default* entry; `isolinux.cfg` is only `include` and `default vesamenu.c32`, so patching the obvious files leaves the default boot prompting. That is the bug this test exists to catch, and it is asserted by name rather than left implicit. - No kernel line carries the parameter **twice** (a re-run of the patch, or a `-map` applied on top of an already-patched file). - Both a BIOS and a UEFI El Torito image are present. - No file from the source ISO is missing from the output (1010 files on 13.7.0). - Output size is within 16 MB of the source. The measured delta is -704 KB. The naive extract-and-rebuild implementation bloats 792 MB -> 988 MB because Debian's ISO shares identical file extents between paths; this guards that regression. ### `grep -c` exits 1 on zero, and that will corrupt your count While writing T20 this produced a wrong total: `grep -c ... || echo 0` prints `0` **and then** appends another `0` when there are no matches, because `grep -c` prints the count and *also* exits 1. The result is `0\n0`, and `$((total + 0\n0))` is a syntax error. The harness swallows the exit status only (`|| true`) and validates the value with a `case` before doing arithmetic. ## Two corrections that are now asserted **`subject.local` must not come back into the polkit rule.** An earlier version of the rule included `subject.local == true` to restrict it to a seat physically in front of the machine. It never matches: `pkcheck` returns 2 with the clause and 0 without it, for the same user. If it returns, passwordless updates stop working while every other T16 assertion still passes - so T16 now asserts the clause is **absent**, with the reason in a comment. **"the timer is enabled" is not a stable assertion on its own.** Enabling is done by `systemctl enable vdi-offline-update.timer` inside `install_debian_update_policy()`, and that only persists when systemd is the running init. T18 therefore drives the installer function first, then asserts, and **skips** the assertion with a reason when systemd is not PID 1. ## A killed run leaves apt permanently broken, and the lock cannot help The harness removes its own apt source in an `EXIT` trap. A run that is killed (the turn is cancelled, the terminal is closed, `SIGKILL`) never reaches that trap, so `/etc/apt/sources.list.d/zz-vdi-test-.list` survives pointing at a temp directory that no longer exists. Every subsequent `apt-get update` then fails with `Failed to fetch ... File not found`, and T06/T07 report that as a port defect. This was observed: a leftover `zz-vdi-test-250.list` from another run broke three T06 assertions while the port was fine. The `flock` does not prevent it, because there is no concurrent process - just residue. So at startup, under the lock, the harness sweeps any `zz-vdi-test-*.list` whose target directory is gone, and says so. That is safe precisely because the lock guarantees no peer run is alive at that moment. ## Requirements - Debian 13 (trixie), root. - `apt-ftparchive`, `apt-get`, `apt-cache`, `dpkg-deb`, `python3`, `gzip` - all present in a stock install. - `git` **for T13 only**. It is not part of a stock trixie image; without it T13 skips and says so. Install it if you want the Fedora regression checks to run: `apt-get install -y --no-install-recommends git`. Note this changes the test machine, which is why the harness treats its absence as a skip rather than installing it itself. - `gpg` **for T12 only**, and only if you want to re-check the signing decision. - `xorriso` **for T20 only**. Without it T20 skips with a reason. - A Debian netinst ISO **for T20 only**, at `/root/trixie-netinst.iso` or `NETINST_ISO=/path/to/netinst.iso`. It is never downloaded. The build itself takes about 3 seconds and needs ~1.6 GB free in `TMPDIR` (source plus output; the source is not copied, only written to the output). ## Safety The harness is designed to be safe to run on a machine you care about. - It never writes to `/var/local/citrix-repo`. It records that directory's mtime before the run and asserts it is unchanged in T15. - It never installs anything. The only `apt-get install` is `--simulate`. T15 compares the installed package count before and after. - It adds exactly one apt source file, `/etc/apt/sources.list.d/zz-vdi-test-$$.list`, and removes it. T15 verifies the removal happened. - It removes only the apt index files it created, matched by the encoded repo path, never `/var/lib/apt/lists` wholesale. - It removes any keyring it installed, matched by the `zz-vdi-test-*` prefix, and only `rmdir`s `/etc/apt/keyrings` (which fails harmlessly if it was already there with other keys in it). - It never writes `__pycache__` into the repo: `PYTHONDONTWRITEBYTECODE=1` and `PYTHONPYCACHEPREFIX` are set, and T15 asserts the repo is clean. - All scratch work happens in one `mktemp -d` directory, removed at the end. - A failing test does not abort the run, so you get the whole picture in one pass. ## Interpreting a failure - **T05/T06/T07 failing together** - the repo build is broken. Look at T05 first. - **T07 fails but T05 passes** - the repo is fine but apt cannot consume it; check directory permissions (apt drops to `_apt`, so the repo path must be traversable). - **T09 fails** - `sync_debian` is leaving old versions in the published directory. On Debian, filenames carry the version, so copying without deleting leaves every previous release indexed and makes rollback impossible. - **T13 fails on the command sequence** - compare the printed `MAIN-CMDS` and `FEDORA-CMDS` lists. Both sides are normalised to ``; if they differ, the Fedora path genuinely changed. - **T15 fails on residue** - the teardown did not finish. Check the messages immediately above it.