feat(tests): implement live VM acceptance test strategy (TODO-006 to TODO-010)

2026-08-03 12:57:53 +05:00 · 2026-07-04 13:38:52 +02:00
parent 7b5522ac94
commit 1d1bac1524
25 changed files with 926 additions and 2 deletions
@@ -0,0 +1,75 @@
+# Live System Testing
+
+Live tests run KIAUH workflows against a real Debian 12 QEMU/KVM VM. They are
+isolated from the local developer machine by design.
+
+## Safety Rules
+
+- Live tests NEVER run on the local machine.
+- They require `KIAUH_LIVE_ALLOW=1`.
+- The target host must be explicitly set via `KIAUH_LIVE_TARGET_HOST` and match
+the VM in the inventory.
+- Local hostnames, loopback addresses, and the current hostname are blocked.
+
+## Prepare a VM
+
+1. Create a Debian 12 QEMU/KVM VM.
+2. Create a user with passwordless sudo.
+3. Install an SSH key for that user.
+4. Install KIAUH on the VM (e.g. clone this repository).
+5. Create a clean snapshot named `clean`:
+   ```bash
+   virsh snapshot-create-as debian12-kiauh clean
+   ```
+
+## Inventory
+
+Edit `kiauh/live/inventory.yaml` or point to a custom file:
+
+```yaml
+vms:
+  - name: debian12-kiauh
+    host: 192.168.122.10
+    user: kiauh
+    key_file: ~/.ssh/kiauh_vm
+    os: debian-12
+    domain: debian12-kiauh
+    snapshot: clean
+```
+
+## Run Live Tests
+
+```bash
+export KIAUH_LIVE_ALLOW=1
+export KIAUH_LIVE_TARGET_HOST=192.168.122.10
+pytest -m live
+```
+
+Each scenario reverts the VM to the clean snapshot first, so scenarios are
+independent.
+
+## Add a Scenario
+
+Create a YAML file in `kiauh/live/scenarios/`:
+
+```yaml
+name: Install Klipper on Debian 12
+vm: debian12-kiauh
+os: debian-12
+steps:
+  - command: ["kiauh", "install", "klipper", "--count", "1"]
+    timeout: 600
+expected:
+  - type: service
+    name: klipper.service
+    state: running
+```
+
+Supported assertion types: `service`, `file`, `package`, `port`, `command`.
+
+## Troubleshooting
+
+- `UnsafeTargetError`: check `KIAUH_LIVE_ALLOW` and `KIAUH_LIVE_TARGET_HOST`.
+- `InventoryError`: check the inventory YAML path and format.
+- `LiveRunnerError` during snapshot revert: ensure `virsh` works and the domain
+  and snapshot names match the inventory.