mitchell/hamlab
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases 14 Wiki Activity
Files
16e85e1e428777ee14e149cecb37a14a41bd4b01
hamlab/VIRTUALIZATION.md
T

13 KiB
Raw Blame History

Virtualization tab — plan (2026-10-09, not started)

A tab for creating, watching, consoling into and deleting LXC containers and KVM virtual machines, the way Proxmox does with pct and qm — per machine, like every other page. This is the scoping PLAN.md asked for. Nothing here is built. Live state of the hosts stays in hamlet/docs/hosts.md.

Decision: Incus 6.0 LTS, on grid first

Option Verdict
Incus (LXD fork, linuxcontainers.org) Chosen. One daemon, one CLI and one JSON API for both containers and VMs. In Debian 13 as incus 6.0.4 LTS — apt install incus pulls qemu-system-x86, swtpm, virtiofsd, incus-agent, so VMs work out of the box. Fedora 42 has 6.23 for the Mac mini if that is ever wanted. Native ZFS storage driver (instant snapshots and clones), built-in image server (images:debian/13, images:ubuntu/24.04, Alpine, Fedora, Arch… as container or VM), snapshot schedules with expiry, incus export backups, per-instance CPU/memory limits, cloud-init, text and VGA console over websocket. Everything is reachable as incus … --format json or incus query, which is exactly how Ham Lab already drives Docker: exec.Command, no new Go dependency.
libvirt/KVM + virsh, plus LXC separately Two daemons, two models, XML. libvirt's official Go binding needs cgo, which the static binary rules out (the pure-Go go-libvirt exists but then containers are a second project). Rejected.
Proxmox itself Means reinstalling grid's OS. No.

Where it can run

Probed 2026-10-09 (/dev/kvm, cpu flags, kvm_intel/kvm modules, RAM, disk).

Host KVM RAM Verdict
grid yes — i7-11700T, VT-x, kvm_intel loaded, nested on 62 GiB; ZFS ARC is at 23 GiB and allowed 40 The VM host. 790 GiB free on the NVMe root, 4.5 TiB free on tank. Cap the ARC at 16 GiB and ~30 GiB is free for guests with headroom for Immich and the encode jobs.
core yes — N100, kvm_intel loaded 7.5 GiB, ~4 GiB available Small LXC containers only, never a VM. Second host, later.
Mac mini yes — Asahi KVM, /dev/kvm present, 16K pages 15 GiB, 8 GiB available Technically workable (Fedora ships Incus 6.23 and qemu-system-aarch64; arm64 guests only, no x86 VMs; 4K-page guests on the 16K host are what muvm already does). Excluded by the rule: single-core-performance workloads only.
hamtv yes 3.7 GiB In service for HamTV. Excluded.
gate no /dev/kvm (OpenWrt) 8 GiB Excluded. The tab says "no virtualisation on this machine".

Nothing is installed anywhere today: no qemu, libvirt, lxc or incus on any host.

What the tab does (v1)

Mirrors the Docker tab's shape: a list, a detail page, actions that are followed tasks, destructive ones confirmed by typing the name.

  • Instances — containers and VMs in one list: state, IPv4, CPU and memory live, disk, snapshots count, uptime. Badge for type. Click for detail.
  • Create — pick container or VM; image from the images: remote (searchable, with the handful most people want pinned: Debian 13, Ubuntu 24.04, Alpine, Fedora); name, CPUs, memory, root disk size; network (NAT bridge, or LAN once br0 exists — below); optional cloud-init user-data (SSH key, packages, a first command); profile. Create, or create and start. Collisions (name taken) named before Create.
  • Lifecycle — start, stop, restart, force stop, rename, delete (type the name; the instance's snapshots go with it, and the dialog says so).
  • Console — two buttons, both open as tabs in the Terminal page through the existing PTY manager (term.CreateCmd, the docker exec path): Console (incus console, the serial console — works before the network is up, shows boot output) and Shell (incus exec <n> -- login shell). Works for containers and, via incus-agent, VMs.
  • Snapshots — take, restore (confirmed), delete; a schedule per instance with an expiry, which is Incus's own snapshots.schedule/snapshots.expiry, so no cron file of ours this time.
  • Resources — edit CPU and memory limits (live for containers; VMs take CPU hotplug and memory balloon live with the agent, else on restart), grow the root disk, add a bind mount (a tank path into a container; virtiofs into a VM).
  • Images — what is cached locally, delete, aliases; publish a stopped instance as an image (Proxmox's "convert to template").
  • Storage and networks — the pools with their fill, the bridges with their subnets. Read-only in v1 beyond what Create needs.
  • Not installed here — a host with KVM but no Incus gets an Install button that runs the whole phase-0 recipe below as a task (the smartmontools-install precedent). A host with no KVM gets one sentence.
  • Sensors — instance.<name>.cpu, .memory, .running from Incus's state, like the container.* sensors, so Alerts gets an "instance down" default and widgets can show one.

Not in v1, with reasons

Thing Why not yet
VGA console in the browser Incus hands the VM's SPICE socket over a websocket; drawing it needs spice-html5 in the UI. Doable (the Incus UI does it), but text console + shell cover every Linux guest. Phase 3.
Windows VMs Incus runs them (ISO repacked with virtio drivers by distrobuilder), but without a VGA console in the browser they are unusable from the tab. After the above.
GPU passthrough to a VM Takes the Arc A380 from the host, and from Jellyfin. No. Sharing the GPU into a container (gpu device) is fine and is how r720-jellyfin used to run. Phase 3.
Clustering / live migration One real host; the others are different sizes and one is arm64; no shared storage. Not planned.
A fleet-wide instance list One machine per view. The switcher is the machine.
Mac mini, hamtv, gate Above.

Decisions baked in (say if any is wrong)

  1. Storage: instances on grid's NVMe, data on tank. A ZFS pool for Incus made as a 300 GiB loop file on the NVMe root (incus storage create default zfs size=300GiB) — fast enough for VM root disks where raidz2 spinners are not, and still ZFS so snapshots and clones are instant. Big data is bind-mounted in from tank, the same split Docker already has (apps on core, data on grid). A second pool on tank/incus can be added later for a guest that genuinely wants terabytes. Consequence: grid's root NVMe is not backed up anywhere (backups land on grid), so instances need the nightly export below.
  2. Network: NAT bridge first, pinned to 192.168.200.1/24. Incus's default picks a random 10.x.y.0/24, and the LAN here is 10.0.0.0/8 — every host has a /8 on its NIC, so any 10.x bridge collides. 192.168.200.0/24 is clear of Docker's pools on both hosts (172.28.0.0/14 on grid; 172.19–172.29 and 192.168.80–143 on core). Instances get name.incus DNS from Incus's dnsmasq. LAN addresses in phase 2: a br0 on grid's enp10s0 (the 10 GbE uplink) so an instance can take a lease from gate and a Pi-hole name. That rewrites /etc/network/interfaces and drops grid off the network for seconds — its own step, with the NanoKVM standing by. macvlan was considered and rejected: no host network change, but host and guests cannot talk, so Immich/Jellyfin/backups on grid could never reach an instance.
  3. Docker and Incus on one host: Docker sets the FORWARD policy to DROP, which silently kills the Incus bridge's traffic. Fix is two DOCKER-USER accept rules for incusbr0, persisted — part of the install recipe, on grid and later core.
  4. ARC cap on grid: zfs_arc_max from 40 GiB to 16 GiB so guest RAM is deterministic.
  5. Privilege: core joins incus-admin (socket access, the docker-group shape) and hamlab.service gets SupplementaryGroups=incus-admin; no sudo in the hot path.
  6. Backups: nightly incus export --instance-only of every instance to /tank/backups/incus/ with a retention, as a cron line like the ZFS schedules (hamlab incus-export), from phase 2. Plus the instance's own snapshot schedule.

The old Proxmox estate (restorable, phase 3)

/tank/backups/proxmox/ holds the retired r720 node: /etc/pve (12 LXC confs, 2 VM confs), 118 GiB of vzdump archives, and three raw VM disks. Latest of each:

ID Name Latest archive Size
100 r720-docker 2026-01-11 20.7 GB
101 r720-cockpit 2026-01-11 439 MB
102 r720-wordpress-alexblaskovich 2026-01-11 1.2 GB
103 r720-spoonterm 2025-06-14 494 MB
104 r720-ollama 2026-01-11 11.3 GB
105 r720-transmission 2026-01-11 322 MB
106 r720-apache 2026-01-11 347 MB
107 r720-nut 2025-03-29 205 MB
108 r720-filebrowser 2026-01-11 262 MB
109 r720-bookstack 2025-04-27 516 MB
110 r720-jellyfin 2026-01-11 2.2 GB
111 r720-wireguard 2025-04-13 510 MB
112 r720-eggwiki 2026-01-11 634 MB
VMs steam (102), powerpanel (105/107/112) .vma.zst 2024–2025; raw disks vm-105, vm-112, base-104 19 GB, 1.9 GB ×6, 97 GiB
  • LXC: feasible and scriptable. Unpack the rootfs (tar --zstd -xpf … --numeric-owner), read etc/vzdump/pct.conf for arch/ostype, drop it, write metadata.yaml, pack, incus image import, launch. Unprivileged uid shifting is Incus's job after that. A wizard in the tab is plausible once the script works by hand.
  • VMs: the raw disks import directly (incus-migrate, or an empty VM with the disk imported into its root volume). The .vma.zst archives need Proxmox's vma extractor (or the Python port) to become raw disks first. Most of these are the same powerpanel box backed up six times, and tank-space-audit.md already has them down for deletion — decide which, if any, are wanted before anything is restored.

Build plan

Phase What Effort
0 — host prep on grid (by hand, documented) apt install incus; incus admin init --preseed with the pool and bridge above; DOCKER-USER rules; ARC cap; incus-admin group + unit edit; launch one Debian 13 container and one Debian 13 VM; verify console, shell, network, snapshot, export. Write it up in hamlet/docs/hosts.md and the preseed into hamlet/scripts/. 1 session
1 — the tab internal/incus (CLI wrapper, JSON types, incus monitor for live state), internal/api/virt.go (routes below), Virtualization.tsx (list, detail, create dialog, snapshot panel), rail entry virt, terminal integration, Install button, the no-KVM state. 2 sessions
2 — operations Snapshot schedules UI, nightly exports with retention, limits editor, bind mounts, br0 on grid and the LAN network option, instance sensors + alert default, cloud-init presets. 1–2 sessions
3 — the rest Proxmox import wizard, publish-as-image, GPU device for containers, VGA console (spice-html5), core as a second (containers-only) host. as wanted

API sketch

GET    /api/virt                               {installed, kvm, pools[], networks[], instances[]}
POST   /api/virt/install                       task: the phase-0 recipe
GET    /api/virt/images?remote=images&q=       local cache + remote search
DELETE /api/virt/images/{fingerprint}
POST   /api/virt/instances                     {name, type, image, cpu, memory, disk, network, cloudinit, profile, start}  task
GET    /api/virt/instances/{name}              config, state, devices, snapshots
PATCH  /api/virt/instances/{name}              limits, description, devices
POST   /api/virt/instances/{name}/{start|stop|restart|kill|delete|rename|publish|export}
POST   /api/virt/instances/{name}/console      → terminal {id}   (incus console)
POST   /api/virt/instances/{name}/exec         → terminal {id}   (incus exec -- sh -lc …)
POST   /api/virt/instances/{name}/snapshots    {name}; POST …/{snap}/restore; DELETE …/{snap}
PUT    /api/virt/instances/{name}/schedule     {cron, expiry}
POST   /api/virt/import                        {path}  task (export tarball, or phase-3 Proxmox archive)

Everything is per machine and reaches a node through the existing /api/nodes/{name}/… proxy with no hub-side code; consoles attach over the existing /api/terminals/{id}/attach. Tasks stream through /api/tasks/{id}/stream as Docker's do.

Sources

Reference in New Issue View Git Blame Copy Permalink