Files
VDI-Linux/debian/test/README.md
T

369 lines
18 KiB
Markdown
Raw Normal View History

# Debian port test harness
`run-tests.sh` proves that the Debian side of the VDI-Linux port actually works,
and that the Fedora side has not regressed. It is a real harness: it builds a
real apt repository from the real Citrix `.deb` packages, adds a real apt
source, and asks real apt whether it can install the result. Nothing is
simulated except the parts that would change the machine.
```sh
sudo bash debian/test/run-tests.sh
```
Run as root. It needs the two Citrix `.deb` files (see [Fixtures](#fixtures)).
Exit status is 0 when no test FAILED, 1 otherwise. A SKIP does not fail the run,
but every skip prints why it skipped and what it would have checked. **A skip is
not a pass**, and the summary says so.
## What it tests
| Test | What it proves |
|------|----------------|
| T00 | Environment: root, required tools, trixie, fixtures present |
| T01 | The `.deb` fixtures really are version `26.08.0.225` |
| T02 | `python3 -m py_compile` passes for `update_local_repo.py` and `configure_kde.py` |
| T03 | Both modules **import** cleanly on Debian (no `tqdm`, no `pip`) |
| T04 | The `tqdm` fallback is usable, not merely importable |
| T05 | The flat repo builds; `Packages` has exactly 2 stanzas at the right version |
| T06 | `apt-get update` exits 0 with no `E:`, no `Err:` and no warnings for the source |
| T07 | apt can *see* the package: correct candidate, correct origin, install resolves |
| T08 | Rebuilding twice is byte-identical and produces no duplicate stanzas |
| T09 | `sync_debian` prunes stale `.deb` files (version bump cannot leave two versions) |
| T10 | A corrupted index is rejected even with `trusted=yes` (control for T06/T07) |
| T11 | The production `local-citrix-repo.list` has the shape the harness verified |
| T12 | GPG `signed-by` path (design check; skips without a signing tool) |
| T13 | Fedora path unregressed: tracked files identical, `sync_fedora` command sequence identical |
| T14 | `bash -n` on every shell script the port adds |
| T16 | polkit rule installed, mode 644, correct actions, no over-grant, balanced, idempotent |
| T17 | A real sudo-group user is authorised **without a password** (pkcheck), with a control |
| T18 | Offline-update script/units: bash -n, `systemd-analyze verify`, timer enabled, throttle, no dangling `/system-update` |
| T19 | The preseed still generates a working `/root/vdi-post.sh` that delegates updates |
| T15 | Nothing was left behind: no source file, no cached indexes, no installs, no `__pycache__`, no polkit rule or unit residue |
### The repo build it tests
The exact production sequence, on a copy of the real `.deb` files in a temp
directory:
```sh
cd "$repo"
apt-ftparchive packages . > Packages
gzip -c Packages > Packages.gz
apt-ftparchive release . > Release
```
and the exact production source line:
```
deb [trusted=yes] file:/var/local/citrix-repo ./
```
with the path substituted for the temp directory. A flat ("trivial") archive:
no `dists/` tree, suite `./`.
## What it does NOT test, and cannot
Being explicit matters more than the green summary.
1. **A real d-i install.** The harness never boots an installer. `debian/preseed.cfg`
is owned and validated by another change; this harness does not test it at all.
Note that `debconf-set-selections` validates line shape only - it silently accepts
an invented question name - so it is not evidence that directives exist.
2. **An actual GUI session.** Nothing here starts Plasma, and no test asserts that
`configure_kde.py` produces a working desktop. T03 proves it *imports*; whether
the touchpad and power settings take effect on a real session is untested.
`kwriteconfig6` is not installed here, so `configure_kde.KWRITE` is `None` and
every write would no-op. That path needs a real Plasma machine.
3. **The Citrix preinst/postinst maintainer scripts.** `apt-get install --simulate`
resolves dependencies; it does not run the package's scripts. Whether
`icaclient`'s postinst succeeds on a real machine is untested. This is the
single biggest gap: a dependency-resolvable package can still fail to configure.
4. **A real d-i/target filesystem for the repo path.** The harness builds the repo
in a temp directory, never at `/var/local/citrix-repo`. It asserts the
*shape* is identical (T11), not that the production path is writable in a
target environment.
5. **The network path.** The Gitea release API and the download code are not
exercised. T09 redirects the source files to a local directory, so
`download_file` is only exercised indirectly. A changed release asset name
would not be caught here.
6. **`apt-get update` inside a d-i chroot.** The preseed's `late_command` runs in
an `in-target` context where apt behaves differently (no `_apt` user, different
permissions). Untested.
7. **Fedora at runtime.** T13 compares source and command sequences; it does not
run `createrepo_c`/`dnf`/`rpm` on Fedora. The command sequence matching is
strong evidence but it is not an execution test. A Fedora 44 WSL distro exists
on the development machine and was not used.
8. **A real desktop session.** T17 proves polkit authorises a sudo-group user
without a password, which is the actual requirement, but it does not open a
GUI and click an update. `pkcon refresh` from a live Plasma session, with an
authentication agent running, is not exercised.
9. **An actual offline update reboot.** T18 checks that `/system-update` is armed
with an existing target and that `systemd-analyze verify` is clean. It does not
reboot and let `packagekit-offline-update.service` apply anything. Whether the
update actually installs at the next boot is untested - only that it was
correctly prepared.
10. **`package-install` end to end.** The polkit rule also grants
`package-install`, but T17 only checks `system-sources-refresh` (the action
`pkcon refresh` uses) and that `package-remove` is *not* granted. Installing a
package from a desktop session without a password is not exercised.
## Two spec'd checks that cannot fail, and what replaced them
This is the most important thing in this file. Two checks that look like tests
are not tests.
### `apt-get update` exiting 0 with no `E:` lines
Measured on Debian 13: a repo whose `Packages` is **unreadable** produces
```
Err:4 file:/tmp/.../repo ./ Packages
Could not open file .../Packages - open (13: Permission denied)
```
and `apt-get update` still **exits 0 with no `E:` lines**. The same is true of a
repo with **no `Release` file at all**, and of a repo with **duplicate stanzas**.
So "exit 0 and no `E:`" passes on a broken repo.
The harness runs the spec'd check as T06 *and* the stronger check separately: no
`Err:` lines, no `W:` lines for that source, and then T07 requires apt to name the
exact version from that exact directory. T10 is the control that proves the
strong checks can fail: it corrupts `Packages` after `Release` was built and
asserts apt **rejects** it with a hash mismatch. Without T10, T06/T07 could be
passing vacuously.
### `git diff main -- ks.cfg`
`ks.cfg` is **not in the repository**. It is not in `main`, not in `debian-port`,
and `git log --all -- ks.cfg` is empty. It lives one level above the repo, in the
parent directory, untracked.
So `git diff main -- ks.cfg` exits 0 and prints nothing - because the path does
not exist on either side, not because the file is unchanged. The harness detects
this with `git cat-file -e main:ks.cfg` and reports it as **SKIP, explicitly
labelled VACUOUS**, rather than as a pass. It then prints the sha256 of the real
out-of-tree file so a change to it is at least visible.
## Line endings will fool you
This repo is checked out on Windows. Every tracked file in the working tree has
CRLF line endings; every blob in git has LF. A plain `git diff main` therefore
reports **all five files as completely rewritten**, including
`RPM-GPG-KEY-citrix-local` with 104 changed lines - the key itself is byte-identical.
The harness compares modulo CRLF (`tr -d '\r' | cmp`). If you see a huge diff,
check line endings before believing it.
## The apt index cache will fool you
`apt-cache policy` and `apt-get update` read cached indexes in
`/var/lib/apt/lists`. During development, a **wrong GPG key** appeared to be
accepted - `exit 0`, no `E:`, package still visible - purely because a good index
was cached from an earlier run.
Every case that must fail purges that repo's cached index files first
(`purge_apt_lists_for`). Any test that checks "can apt see the package" without
purging is testing the cache, not apt.
Related: `/tmp` is wiped between separate `wsl` invocations on this machine
(`systemd-tmpfiles`). Anything expected to survive across invocations must live
in `/root`.
## GPG signing: why `trusted=yes` is the right call
Production does not sign the Debian repo, and T12 exists to check that decision
rather than assume it. A stock trixie ships **no `gpg`** - only `/usr/bin/sqv`,
the Sequoia verifier that apt calls internally. So signing would add a `gpg`
dependency that Debian 13 does not have by default, in an installer environment
where new dependencies are the fragile thing.
The signing path was measured out of band (a throwaway key generated with Fedora
WSL's `gpg`, because Debian has none), with the apt cache purged between cases:
| Keyring | Result |
|---------|--------|
| armored `.asc`, correct key | exit 0, no `E:`, package visible |
| armored `.asc`, **wrong** key | exit 100, `E:`, `Missing key`, not visible |
| binary `.gpg`, correct key | exit 0, no `E:`, package visible |
| binary `.gpg`, **wrong** key | exit 100, `E:`, `Missing key`, not visible |
| `trusted=yes`, no signature | exit 0, no `E:`, package visible |
Two conclusions:
- `signed-by` **works correctly** on a flat `file:` repo on trixie, for both
keyring formats. It is not broken; it is simply not worth it here.
- `trusted=yes` does **not** mean "no integrity checking". T10 shows a tampered
`Packages` is rejected with `E: ... Hash Sum mismatch` regardless. Signing
would only add proof of *authorship*, which is meaningless for a repository the
machine builds from packages it just downloaded and verified itself.
T12 skips loudly when no signing tool is present, which is the normal case on a
stock trixie. Install `gpg` to make it run.
## Fixtures
The two real packages are expected next to the repo, not inside it:
```
LinuxISO/
icaclient_26.08.0.225_amd64.deb (256 MB)
ctxusb_26.08.0.225_amd64.deb (178 KB)
VDI-Linux/ <- the repo
debian/test/run-tests.sh <- this harness
```
Override with `DEB_DIR=/path/to/debs sudo -E bash debian/test/run-tests.sh`.
The 256 MB file is copied into a temp directory, which is fast on native
filesystem and slow off `/mnt/c`. If `/tmp` is small, set `TMPDIR` to somewhere
with ~600 MB free.
## T17 is the test that matters, and why it needs a control
The user's requirement is that a desktop user can update **without an admin
password**. Asserting that the rule file contains the right action names proves
nothing about whether polkit acts on it - a rule can be syntactically perfect,
in the right directory, and still never match (which is exactly what happened
during development with a `subject.local == true` clause).
So T17 asks polkit itself:
1. Start a process owned by a real member of group `sudo` (`setsid`, so nothing
blocks), then `pkcheck --action-id ...system-sources-refresh --process <pid>`
as root. Assert **rc=0**.
2. **The control:** move the rule away, restart polkit, repeat. Assert **rc=2**
(authentication required). Then restore.
Without step 2 the test cannot fail: a polkit configured to answer YES to
everything would also give rc=0. The control is what makes step 1 evidence.
It also asserts the rule does **not** grant `package-remove`, `package-reinstall`
or `package-install-untrusted` - a user who can remove packages without a
password can break the machine.
`pkcheck` is only conclusive because it consults the live polkitd. If polkit is
not installed, T16 and T17 **skip** with a reason rather than passing quietly.
## T18: the dangerous failure mode
A dangling `/system-update` symlink **fails the next boot**. The offline-update
script arms that symlink, so the harness asserts, after every run, that if
`/system-update` exists then its `readlink -f` target exists too. It is checked
both immediately after the run and again after the forced (stamp=0) run, which
is the one most likely to arm it.
Also asserted, because it is a real behaviour and not a detail:
- Running the script twice in a row is **throttled** (prints "skipping").
- Setting the stamp to `0` defeats the throttle - so it is time-based, not a
permanent latch.
- `pkcon` exit 5 means "no packages to update" and must not be reported as
`download failed`. On an up-to-date box the output must not contain
"download failed".
## T19: why it never executes the generated script
The preseed's `late_command` writes `/root/vdi-post.sh` and then runs it with
`in-target`. That script writes `/etc/hostname`, `/etc/hosts`, `/etc/crontab`
and systemd units **by absolute path** - running it here would reconfigure the
machine the tests are running on.
T19 therefore only *regenerates* the script by running the preseed's own
`printf` into a temp file, then checks it: `bash -n`, that it delegates to
`/usr/local/sbin/vdi-offline-update` instead of duplicating the `pkcon`
sequence (comments stripped first, so the explanatory comment does not produce
a false positive), and that it still enables `cron` and
`vdi-offline-update.timer`.
Extracting it is fiddly, and the naive approaches are all wrong in ways that
look like preseed bugs:
- A regex that crosses `;` captures the **firefox** `printf` too, which then
tries to write `/target/etc/firefox` and fails.
- Splitting on `;` breaks, because the payload contains `;` and `>` inside
quoted arguments.
- Leaving the leading `\` of the first continuation line makes `sh -c` skip the
`printf` entirely, producing a "No such file" that looks exactly like a
preseed defect.
The harness anchors on two unambiguous markers instead: the payload's own first
argument `'#!/bin/bash'` and the `> /target/root/vdi-post.sh` redirect. It also
asserts the captured text does **not** contain `ManagedBookmarks`, so a future
change to the preseed fails loudly rather than silently testing the wrong
block.
## Tests that mutate the machine, and how that is undone
T16-T19 install a polkit rule and systemd units and can arm `/system-update`.
T15 asserts no collateral damage, so every mutated path is **backed up before
the test and restored at teardown**:
| Path | How it is handled |
|------|-------------------|
| `/etc/polkit-1/rules.d/60-vdi-no-password-updates.rules` | backed up, restored |
| `/usr/local/sbin/vdi-offline-update` | backed up, restored |
| `/etc/systemd/system/vdi-offline-update.{service,timer}` | backed up, restored |
| `/var/lib/vdi-offline-update.last` | backed up, restored |
| `/system-update` | backed up as a symlink target, restored; **never `rm -rf`** |
`restore_path` records whether a path was `present`, `absent` or a `symlink`
before the run, so "it did not exist and must be gone" is distinguishable from
"the backup failed". For an absent path it removes only a regular file or
symlink - it never recurses into a directory. The same restores run in the
`EXIT` trap, so an aborted run still cleans up.
## Requirements
- Debian 13 (trixie), root.
- `apt-ftparchive`, `apt-get`, `apt-cache`, `dpkg-deb`, `python3`, `gzip` - all
present in a stock install.
- `git` **for T13 only**. It is not part of a stock trixie image; without it T13
skips and says so. Install it if you want the Fedora regression checks to run:
`apt-get install -y --no-install-recommends git`. Note this changes the test
machine, which is why the harness treats its absence as a skip rather than
installing it itself.
- `gpg` **for T12 only**, and only if you want to re-check the signing decision.
## Safety
The harness is designed to be safe to run on a machine you care about.
- It never writes to `/var/local/citrix-repo`. It records that directory's mtime
before the run and asserts it is unchanged in T15.
- It never installs anything. The only `apt-get install` is `--simulate`. T15
compares the installed package count before and after.
- It adds exactly one apt source file, `/etc/apt/sources.list.d/zz-vdi-test-$$.list`,
and removes it. T15 verifies the removal happened.
- It removes only the apt index files it created, matched by the encoded repo path,
never `/var/lib/apt/lists` wholesale.
- It removes any keyring it installed, matched by the `zz-vdi-test-*` prefix, and
only `rmdir`s `/etc/apt/keyrings` (which fails harmlessly if it was already there
with other keys in it).
- It never writes `__pycache__` into the repo: `PYTHONDONTWRITEBYTECODE=1` and
`PYTHONPYCACHEPREFIX` are set, and T15 asserts the repo is clean.
- All scratch work happens in one `mktemp -d` directory, removed at the end.
- A failing test does not abort the run, so you get the whole picture in one pass.
## Interpreting a failure
- **T05/T06/T07 failing together** - the repo build is broken. Look at T05 first.
- **T07 fails but T05 passes** - the repo is fine but apt cannot consume it; check
directory permissions (apt drops to `_apt`, so the repo path must be traversable).
- **T09 fails** - `sync_debian` is leaving old versions in the published
directory. On Debian, filenames carry the version, so copying without deleting
leaves every previous release indexed and makes rollback impossible.
- **T13 fails on the command sequence** - compare the printed `MAIN-CMDS` and
`FEDORA-CMDS` lists. Both sides are normalised to `<REPO>`; if they differ, the
Fedora path genuinely changed.
- **T15 fails on residue** - the teardown did not finish. Check the messages
immediately above it.