From af53b45b67917b46c2b280ff1182933f1a613954 Mon Sep 17 00:00:00 2001 From: Mitchell-Johnson Date: Fri, 9 Oct 2026 13:48:31 -0700 Subject: [PATCH] Virtualization plan: every artefact under Ham Lab's data directory, one virt/ tree per machine Co-Authored-By: Claude Fable 5.1 --- PLAN.md | 1 + VIRTUALIZATION.md | 107 ++++++++++++++++++++++++++++++++-------------- 2 files changed, 75 insertions(+), 33 deletions(-) diff --git a/PLAN.md b/PLAN.md index ad619ce..9259e60 100644 --- a/PLAN.md +++ b/PLAN.md @@ -21,6 +21,7 @@ stub tab or a roadmap item. | Guest logins | A **guest** is an account like any other, with `guest: true` and an expiry: `login`, sessions and prefs need no special case, and **it may do everything an account may do**. The expiry is the whole feature — checked on every request and swept once a minute — with revoking as the manual version of it. | Built 2026-09-27. A read-only version was built first and **cut the same day**: Mitchell hands these to people he trusts and revokes them otherwise, so a console with half its tabs missing was solving a problem he doesn't have. Consequences, stated once and accepted: a guest has the Terminal tab, which is a root-capable shell on every host, and can make an account or token of its own that outlives the guest login. The one thing a guest is not is *an account*: it is left out of the Users list, out of "the last account can't be removed", and out of the single account an empty user name resolves to. | | Machines | **A view shows one machine: the one in the switcher.** No hub-side page lists or aggregates what its nodes are doing. To see or change anything on another machine, switch to it. | Mitchell, 2026-09-25, cutting a fleet-wide updates list out of the Docker tab. The switcher is the only place a machine is chosen, so there is exactly one answer to "which machine am I looking at" — a page that quietly reaches across breaks that, and it is never obvious which machine an action would land on. The hub still *proxies* for a node, and hub-only admin (Machines, Users, Settings, Alerts) is unaffected. | | Capability, not policy | **A tab offers what the selected machine can do, detected live — never what a rule says should run there.** Where the software is missing but installable, Ham Lab installs it; where it cannot be installed or the hardware lacks it, the tab says so in a sentence and offers what does work. Every action lands on the machine in the switcher. | Mitchell, 2026-10-09, after the Virtualization plan's first draft excluded the Mac mini and `hamtv` on the strength of `hamlet/CLAUDE.md`'s deployment rules: *"I want the option to do it anywhere I want, unless the hardware / software literally doesn't support it. Ham Lab needs to be built in a general-purpose way that can run on any Linux system."* Those rules bind the agent's unasked deployments, not the product. | +| Ham Lab's data lives in Ham Lab's folder | **Everything a tab manages sits under the machine's data directory (`config.DataDir()`), never at a convenient path elsewhere.** Existing material is imported by copying. The engine's runtime state (`/var/lib/docker`, `/var/lib/incus`) is the engine's; Ham Lab keeps what would rebuild it. | Mitchell, 2026-10-09, when the Virtualization plan pointed its ISO library at `/tank/software/isos/`: *"ALL data needs to be under a HamLab folder, because it manages its own data."* The same reason `core`'s compose files moved under `apps/hamlab/stacks/` on 2026-10-05. `grid`'s and the Mac's stacks at `~/stacks//` are the standing exception. | ## Layout diff --git a/VIRTUALIZATION.md b/VIRTUALIZATION.md index 2ace1fc..eb2a9a9 100644 --- a/VIRTUALIZATION.md +++ b/VIRTUALIZATION.md @@ -26,11 +26,20 @@ So: single-core workloads, `hamtv` not spare hardware) bind the *agent's* unasked deployments. They are not feature gates for the product, and the first draft was wrong to treat them as such. -- **Ham Lab owns the install and the data layout.** It detects what is missing, offers to - install it where it knows how, chooses and records where instance storage lives and which - subnet the bridge takes, and runs the backups. Where it cannot install or the hardware - cannot do the job, it says so in one sentence and offers what *does* work (containers - without KVM, for instance). +- **Ham Lab owns its data, and its data lives in its own folder.** Everything this tab + manages — instance storage, ISOs, image tarballs, exports — sits under Ham Lab's data + directory on *that* machine (`config.DataDir()`, `~/apps/hamlab` on `core`, `grid` and + the Mac), in one `virt/` tree. Never a path elsewhere on the machine, however convenient + the existing folder. Stated 2026-10-09 after the first Library draft pointed the ISO shelf + at `/tank/software/isos/`: *"ALL data needs to be under a HamLab folder, because it + manages its own data."* This is the rule that moved `core`'s compose files under + `apps/hamlab/stacks/` on 2026-10-05; `grid`'s and the Mac's stacks, still at + `~/stacks//`, are the outstanding exception and are not touched by this plan. + The engine's own runtime state (`/var/lib/incus`, like `/var/lib/docker`) is the engine's, + not Ham Lab's data: Ham Lab keeps what it would need to rebuild it. +- **Ham Lab installs or reports.** It detects what is missing, offers to install it where it + knows how, and where it cannot or the hardware cannot do the job, says so in one sentence + and offers what *does* work (containers without KVM, for instance). ## Decision: Incus 6.0 LTS as the engine @@ -78,8 +87,9 @@ destructive ones confirmed by typing the name. add Ham Lab's user to `incus-admin` (socket access, the `docker`-group shape) and add it to the unit's `SupplementaryGroups`, apply the Docker fix where Docker is present (below), then restart Ham Lab. Idempotent, so it doubles as "repair". The dialog shows and lets him - edit: pool driver (detected), pool size (proposed from free space), pool path, bridge - subnet (computed), whether to apply the Docker rules. + edit: pool driver (detected), pool size (proposed from free space), bridge subnet + (computed), whether to apply the Docker rules. Where it all goes is not a choice: Ham + Lab's folder. - **Instances** — containers and VMs in one list: type, state, IPv4, CPU and memory live, disk, snapshot count, uptime. Click for detail. - **Create** — container or VM (VM only where KVM is); image from the `images:` remote, @@ -107,6 +117,37 @@ destructive ones confirmed by typing the name. - **Sensors** — `instance..cpu`, `.memory`, `.running` from Incus's state, like the `container.*` sensors, so Alerts gets an "instance down" default and widgets can show one. +## Where the data lives + +One tree per machine, under Ham Lab's data directory, created by Install: + +``` +~/apps/hamlab/virt/ + pool.img the instance pool's backing file (zfs or btrfs loop; a plain directory + pool/ when the driver is dir) — every instance's disks live in here + isos/ the ISO library: the files themselves + images/ the image library: template tarballs and VM disk images as files, the + way Proxmox keeps CT templates — Incus's image store is loaded from here + exports/ nightly `incus export` tarballs, with retention + virt.json what Ham Lab chose and when: driver, pool size, bridge subnet, ISO and + image inventory, export schedule +``` + +Consequences, stated once: + +- The tree is on whatever disk holds Ham Lab's data directory — the boot NVMe on all three. + Install proposes the pool size from that disk's free space. On `grid` that disk has + 790 GiB free; on `core`, 175 GiB. +- Bulk data a guest needs is **bind-mounted in from where it already is** (`/tank/…` into an + instance on `grid`), the same split Docker has: the instance is Ham Lab's, the media is + not. That is a device on the instance, recorded in its config, not a second home. +- On `core` the whole tree rides the existing nightly rsync to `grid` for free. On `grid` + and the Mac it is as backed up as the rest of their Ham Lab folders — which is to say the + exports are the backup, and the backup job will need to be told about them in phase 2. +- Existing material elsewhere — the 105 GiB of ISOs at `/tank/software/isos/`, the Proxmox + archives — is **imported into the tree by copying**, file by file, on request. The + library never indexes a folder it does not own. + ## Library: ISOs and images (asked for 2026-10-09) Proxmox has a storage view with **ISO Images** and **CT Templates**. Incus has the same two @@ -115,19 +156,19 @@ like everything else. | Proxmox calls it | Incus calls it | What it is here | |---|---|---| -| ISO Images | an ISO file attached to a VM as a `disk` device (added as a ROM device; VMs only), or a custom storage volume of type `iso` | **A directory of `.iso` files on the machine that Ham Lab indexes** — no copy into the pool | -| CT Templates | **images** — the `images:` remote (linuxcontainers.org), the local image store, anything imported or published | **The local image store**, with the remote as the catalogue to pull from | +| ISO Images | an ISO file attached to a VM as a `disk` device (added as a ROM device; VMs only), or a custom storage volume of type `iso` | **`virt/isos/` in Ham Lab's folder**, attached by path — no copy into the pool | +| CT Templates | **images** — the `images:` remote (linuxcontainers.org), the local image store, anything imported or published | **`virt/images/` in Ham Lab's folder** — tarballs and disk images as files, loaded into Incus from there; the remote is the catalogue to pull from | ### ISOs -- **One ISO directory per machine**, chosen in the Install dialog and changeable later. - Default is under Ham Lab's data directory; on `grid` it should be the existing - **`/tank/software/isos/`** — 105 GiB already there (`linux/`, `windows/`, `truenas/`, - `virtio_drivers/`, `pi/`, `hackberry/`, `sipeed/`), scanned recursively. +- **The ISO library is `virt/isos/` in Ham Lab's folder**, on every machine, full stop. The + 105 GiB at `/tank/software/isos/` on `grid` (`linux/`, `windows/`, `truenas/`, + `virtio_drivers/`…) is a *source* to import from — pick the ones wanted, Ham Lab copies them + in — not the library itself. - **Getting an ISO in:** upload from the browser (the Files tab's `PUT /api/files/upload` with its progress bar, reused), **download from a URL** as a followed task with an optional checksum (the usual way — an ISO starts as a link), or **import from a path** already on the - machine. Rename, delete (refused while a VM has it attached, naming the VM). + machine — which **copies** the file into `virt/isos/` and says so. Rename, delete (refused while a VM has it attached, naming the VM). - **Using one:** attach to / detach from a VM as a ROM device by path, with `boot.priority` set so it boots first; **New VM from ISO** makes an empty VM (`incus init --empty --vm`) with the ISO attached, starts it and opens the console. After install, Detach is one click @@ -141,8 +182,8 @@ like everything else. and the VGA console ship together, in phase 2**, moved up from phase 3. - ISO *volumes* (`incus storage volume import --type=iso`) are the alternative: read-only, attachable to several VMs at once, but they copy the file into the - pool. Not the default here — the directory honours the library that already exists and - keeps 105 GiB out of a 300 GiB pool. Offered as an option only if a need shows up. + pool. Not used here — a file in `virt/isos/` is already Ham Lab's, and a volume would copy it + a second time into the pool. Offered as an option only if a need shows up. ### Images (container templates and VM images) @@ -161,8 +202,10 @@ like everything else. image — by upload, URL, or a path on the machine. Same three ways in as ISOs. - **Publish:** a stopped instance becomes an image with an alias — Proxmox's "convert to template", the one thing the Images section of v1 always had. -- **Where they live:** Incus's image store, in the pool, so the pool's fill on the Storage - view includes them and says so. +- **Where they live:** as files in `virt/images/` — the downloaded or imported tarball, the + published image exported to one (`incus image export`) — so the library is complete without + Incus and can be reloaded into a fresh install. Incus's own image store under + `/var/lib/incus/images/` is a cache derived from it, and the page says so. ## Not in v1, with reasons @@ -177,13 +220,12 @@ like everything else. ## Layout decisions Ham Lab makes per machine (overridable in the Install dialog) -1. **Storage.** A pool on the machine's own disk, driver by the detection above, as a - loop-backed image where the driver needs one (`incus storage create default zfs - size=…`) — sized from free space, shown and editable. Bulk data is **bind-mounted in**, - not stored in the pool: on `grid` that means `/tank/…` paths into instances, the same - split Docker already has. A second pool on an existing dataset (`tank/incus`) is one - extra command if a guest wants terabytes. The pool's path and driver are recorded in - Ham Lab's state so the page can say where the data is. +1. **Storage.** The instance pool's backing lives in Ham Lab's folder: Install creates a + sparse `virt/pool.img` and hands it to Incus as the pool's `source` (zfs or btrfs by the + detection above), or `virt/pool/` as a `dir` pool where neither is available. Sized from + the disk's free space, shown and editable. Bulk data is **bind-mounted in**, not stored in + the pool. The driver, path and size are recorded in `virt.json` so the page can say where + the data is and Install can rebuild the pool from the file after a reinstall. 2. **Bridge subnet, computed.** Incus's `auto` picks a random `10.x.y.0/24`, and **this LAN is `10.0.0.0/8`** — every host has a /8 on its NIC, so any 10.x bridge collides. Ham Lab reads the machine's routes and picks the first free /24 from `192.168.0.0/16`, @@ -205,11 +247,10 @@ like everything else. the ARC (on `grid`, 40 GiB → 16 GiB) so guest RAM is deterministic; his choice. 6. **Privilege.** Ham Lab's user joins `incus-admin`; no sudo in the hot path. Install itself goes through `priv` like the other installers. -7. **Backups.** Nightly `incus export --instance-only` of every instance to a per-machine - export directory with a retention, as a cron line like the ZFS schedules - (`hamlab incus-export`), from phase 2. Default is under Ham Lab's data directory (so on - `core` the nightly rsync already carries it); on `grid` he will point it at - `/tank/backups/incus/`, since `grid`'s own root disk is backed up nowhere. +7. **Backups.** Nightly `incus export --instance-only` of every instance to `virt/exports/` + with a retention, as a cron line like the ZFS schedules (`hamlab incus-export`), from + phase 2. On `core` that folder is inside the nightly rsync already; on `grid` and the Mac + the backup job gets a line for it in phase 2. ## The old Proxmox estate (importable, phase 3) @@ -271,15 +312,15 @@ POST /api/virt/instances/{name}/snapshots {name}; POST …/{snap}/restore; PUT /api/virt/instances/{name}/schedule {cron, expiry} POST /api/virt/import {path} task (export tarball, or phase-3 Proxmox archive) -GET /api/virt/isos the ISO directory, scanned: name, size, attached-to -PUT /api/virt/isos/dir {path} change this machine's ISO directory +GET /api/virt/isos virt/isos/, scanned: name, size, attached-to +POST /api/virt/isos/import {path} task: copy a file on this machine into virt/isos/ POST /api/virt/isos/fetch {url, sha256?} task; uploads reuse PUT /api/files/upload DELETE /api/virt/isos/{name} refused while attached POST /api/virt/instances/{name}/iso {iso, boot_first} attach; DELETE …/iso detach POST /api/virt/instances/from-iso {name, iso, cpu, memory, disk, network} task → console GET /api/virt/images/remote?q=&type= the `images:` catalogue for this arch POST /api/virt/images/download {alias} task (incus image copy images:… local:) -POST /api/virt/images/import {path | url, alias, type} task (tarball, or qcow2/raw + metadata) +POST /api/virt/images/import {path | url, alias, type} task: copy into virt/images/, then load into Incus GET /api/virt/instances/{name}/vga websocket: Incus VGA console → spice-html5 (phase 2) ```