hamlab/PLAN.md

# Ham Lab — MVP plan

Proxmox-style web console for a bare-metal Debian 13 host. First target: `core`
(10.0.0.234). MVP is **service + shell + persistent terminals**; everything else is a
stub tab or a roadmap item.

## Decisions

| Area | Decision | Why |
|---|---|---|
| Backend | **Go 1.24**, single static binary, frontend embedded via `embed` | `core` has ~370 MiB free RAM; a Go daemon idles at ~15 MiB vs ~80+ for Node. One file to deploy, no nvm path in the unit. Same binary later installs on `grid` unchanged. |
| PTY | `creack/pty` — server-owned PTYs running `core`'s login shell | Sessions live in the daemon, not the browser, so refresh/tab-switch reattaches. No tmux dependency (not installed on `core`). |
| Transport | One WebSocket per attached terminal; REST for everything else | Standard; NPM passes WebSockets with one toggle. |
| Frontend | **React + TypeScript + Vite**, `xterm.js` (+ fit, webgl, web-links addons) | Matches `calorie-counter` / `wedding-website`, so no new toolchain to learn. |
| Auth | Single-user **password login**, argon2id hash, HttpOnly/Secure/SameSite=Strict session cookie, login rate-limit, Origin check on WS upgrade | This is a root-capable shell on the public internet. Auth is MVP, not roadmap. |
| Service | `hamlab.service`, `User=core`, `Restart=always`, binary at `/usr/local/bin/hamlab` | Same pattern as `calorie-counter.service`. Runs as `core` so `sudo` works in the terminal without a password, as on the SSH path. |
| Listen | `10.0.0.234:8100` (free) | NPM is a container, so loopback-only won't do. Firewalling to NPM is a later hardening step. |
| Data | `/home/core/apps/hamlab/` — `config.yaml`, `state.db` (SQLite) | `apps/` is the nightly rsync target to `grid`, so it's backed up for free. |
| Ingress | NPM proxy host `hamlab.bigham.dev` → `:8100`, LE cert, WebSockets on, caching off, force-SSL + HSTS + HTTP/2 set in a **second** API call (NPM drops them on create-with-new-cert) | `*.bigham.dev` already resolves; the proxy host alone makes it live. |
| Repo | Gitea `mitchell/hamlab`, private, `main` | Created 2026-09-17. |

## Layout

```
┌──────────────────────────────────────────────────────────────┐
│ header: Ham Lab · core · load 0.42 · ● connected · [logout]    │
├──────────────┬───────────────────────────────────────────────┤
│ Dashboard    │ shell 1 ● │ backups │ +                      │
│ Processes    ├───────────────────────────────────────────────┤
│ Terminal     │                                               │
│ Services     │        main content (xterm)                   │
│ Cron         │                                               │
│ …            │                                               │
├──────────────┴───────────────────────────────────────────────┤
│ footer: v0.1.0 · up 12d · hamlab.bigham.dev                   │
└──────────────────────────────────────────────────────────────┘
```

- Left rail: flat list of pages, active one highlighted. No nested leaves.
- Terminal page: tab strip across the top of the pane, one tab per session, `● ` when
  a process is running, `+` pinned at the right edge. Active tab carries rename and
  close; double-click renames, middle-click closes. (Until 2026-09-18 the sessions
  were leaves under a collapsible Terminal group in the rail.)
- Non-MVP groups render as disabled leaves with "coming soon" — the rail shape is
  the product; stubs keep it honest about scope.
- **Mobile**: rail becomes a slide-in drawer; xterm fits the viewport; a sticky key
  bar above the keyboard (`Esc Tab Ctrl Alt ↑ ↓ ← → |`) because phone keyboards
  have none of those. This is what makes it usable as a PWA rather than a novelty.

## Terminal semantics

- A session is a PTY + shell owned by the daemon, with an ID, name, cwd hint, and a
  **scrollback ring buffer** (256 KiB). Attaching replays the buffer, then streams.
- Any number of browser tabs can attach to the same session (all get output; all can
  type). Resize follows the most recent attacher.
- Switching tabs in the UI just detaches/attaches; the shell keeps running.
- Closing a session sends `SIGHUP` to the process group and drops it. Shell exit
  closes the session automatically (leaf shows "exited" until dismissed).
- Session names persist in `state.db` so the sidebar order survives a service
  restart. **The shells themselves do not survive a daemon restart** in the MVP —
  see roadmap (tmux backing).
- Idle sessions cost one shell process each; no automatic reaping in MVP.

## API (MVP)

```
POST /api/login            {password}            → 204 + cookie
POST /api/logout                                 → 204
GET  /api/me                                     → {host, user, version, uptime, load}
GET  /api/terminals                              → [{id, name, running, created}]
POST /api/terminals        {name?}               → {id, ...}
PATCH /api/terminals/:id   {name}                → 204
DELETE /api/terminals/:id                        → 204
WS   /api/terminals/:id/attach   binary frames = raw bytes; JSON text frames = {resize}
GET  /healthz                                    → 200 (unauthenticated, for NPM)
```

Everything under `/api` except `login` and `/healthz` requires the session cookie.

## Project layout

```
hamlab/
  cmd/hamlab/main.go          serve | passwd | version
  internal/
    auth/                     argon2id, sessions, rate limit
    term/                     PTY manager, ring buffer, attach/detach
    api/                      chi router, handlers, WS
    state/                    SQLite (modernc, no cgo)
    web/                      go:embed of web/dist
  web/                        Vite + React + TS + xterm.js, PWA manifest + SW
  deploy/
    hamlab.service
    install.sh                build → /usr/local/bin/hamlab → systemctl restart
  Makefile                    build / dev / deploy / lint / test
```

## Milestones

**M0 — skeleton** (service boots, login works)
- Go module, chi, config loading, `hamlab passwd`, argon2id login, session cookie.
- `hamlab.service` on `core`, `:8100`, serves a placeholder page.
- `make deploy` from `core` itself.

**M1 — terminals** (the MVP feature)
- PTY manager with ring buffer, create/list/rename/close, WS attach with resize.
- React shell: header, rail, footer, Terminal group, xterm main pane.
- Persistence of session list across refresh and rail switching.

**M2 — polish + PWA + ingress**
- Mobile drawer, key bar, fit-on-rotate.
- `manifest.webmanifest`, icons (Ham logo on black, same recipe as HamTV), service
  worker caching the app shell only (never `/api`).
- NPM proxy host for `hamlab.bigham.dev`; verify WS through NPM; force-SSL/HSTS/HTTP2.
- Stub groups for Services / Cron / ZFS / Dashboard.

**Done when:** from the phone, open the PWA, log in, open two terminals, run
something long in one, switch away and back, refresh — it's all still there.

## Roadmap (not MVP — notes only)

- **Updates tab** — **shipped 2026-09-17** (`internal/updates`, `internal/tasks`,
  `Updates.tsx`): upgradable packages with current → candidate, suite, security badge
  and description; index age, last upgrade (from `/var/log/apt/history.log`),
  unattended-upgrades on/off, reboot-required banner with the packages that caused it;
  Refresh index, Upgrade all (dist-upgrade), Upgrade selected; changelog per package on
  demand. Runs go through the new **tasks** package: background command, output kept,
  followed live over `/api/tasks/{id}/stream`, cancellable, listed at `/api/tasks` (the
  seed of a Task History tab). apt is fully supported; dnf/pacman list only.
- **Power** — header ⏻ button (amber when a reboot is pending), dialog that requires the
  hostname typed back, then `systemctl reboot` / `poweroff`. `/api/me` carries
  `reboot_needed` so the header knows.

- **tmux-backed sessions** so shells survive a daemon restart / redeploy. Would spawn
  `tmux new -A -s hamlab-<id>`; needs `mouse on` and status-bar handling in xterm.
- **Services tab** — `systemctl list-units`, per-unit page: status, last 200 journal
  lines (live-tail via WS), start/stop/restart/enable buttons, unit file viewer.
- **Cron tab** — **shipped 2026-09-17** (`internal/cron`, `Cron.tsx`): one list built
  from `/etc/crontab`, `/etc/cron.d/*`, every user crontab (via sudo), the scripts in
  the `run-parts` directories those drive, plus systemd timers. Next run computed by an
  in-house cron-expression evaluator (tested: ranges, steps, names, dom-OR-dow rule,
  `@` aliases); last run from the journal's `(user) CMD (command)` lines (cron logs via
  syslog to journald; reading needs sudo/adm). Commented-out jobs show as disabled;
  job-shaped lines inside comment blocks (Debian's header examples) are ignored. Per
  job: Run now (background, as the job's user, `sh -c` with crontab `%` rules, output
  captured), Enable/Disable (comments the line in place, refuses if the file changed
  underneath), Edit source (textarea; validated line-by-line; user crontabs installed
  with `crontab -`, system files written via `sudo tee`), Cron log, and Output file
  when the command redirects to one. Timers: Run now (starts the activated service),
  start/stop/enable/disable, live logs of the service via the Services tab's tail.
- **Disks tab** — **shipped 2026-09-17** (`internal/storage`, `Disks.tsx`), replacing
  the planned ZFS tab: block devices from `lsblk -J` (kind, model, size, temperature
  from the device's hwmon node, partitions with filesystem usage, live read/write/busy
  from the metrics sampler); SMART via `smartctl -j` when smartmontools is present (one
  button installs it as a task otherwise): health, temperature, power-on time, self-test
  status, ATA attribute table with the failure-predicting IDs flagged, NVMe endurance and
  error counters, short/long self-tests. Spinning disks are read with `-n standby` so
  the tab never wakes a sleeping drive; "Wake and read" is explicit. ZFS section when
  the tools and module are there: pools with health, capacity, fragmentation, last scan
  and vdev tree; scrub start/pause/stop; datasets tree with used/refer/in-snapshots/
  snapshot count/compressratio/mount; per-dataset snapshot list and "Snapshot now".
  Verified against `grid` (raidz2 `tank`, 10 datasets, 515 snapshots, 8 WD40EZAZ + 970
  EVO Plus SMART). Snapshot destroy and dataset property editing are deliberately not
  there yet.
- **Dashboard** — **first cut shipped 2026-09-17** (`internal/metrics`, `Dashboard.tsx`):
  CPU (total + per core), memory/swap, load, uptime, hwmon temps and fans read straight
  from `/sys` (no lm-sensors), per-interface network rates, per-disk read/write/busy
  from `/proc/diskstats`, filesystem usage, UPS via `sudo -n pwrstat -status`. Sampled
  every 2 s into an in-memory ring of 10 min; the UI (the **Dashboard** tab
  since 2026-09-19) reads it all through the sensor API — see Sensors below. **Customizable**: a
  Customize mode reorders and hides modules (CPU, Memory, Load, Uptime, Temperatures,
  Fans, Network, Disks, Filesystems, UPS) and hides individual rows (an interface, a
  mount, a disk, a sensor); the layout is stored server-side in `state.json` via
  `GET/PUT /api/prefs/dashboard` so every device sees the same dashboard. The update
  interval (100 ms – 2 s) is part of the layout and drives both the browser's polling and
  the daemon's sampling clock; the chart window is always 300 samples, so 30 s at 100 ms
  and 10 min at 2 s. Still to do: persistent
  history (SQLite ring, e.g. 10 s × 7 days), SMART (`smartctl -j`), longer time ranges.
- **Sensors** — **shipped 2026-09-19** (`internal/sensors`, `internal/metrics/sensors.go`,
  `internal/api/status_sensors.go`, `web/src/sensors.ts`, `widgets.tsx`, `Embed.tsx`;
  reference in `SENSORS.md`): everything measured is a sensor with a stable dotted ID;
  catalog, single value, history, a **shaped query** (the client sends the object it wants
  and gets it back filled in — GraphQL's idea without GraphQL, since sensors differ per
  machine and a static schema would collapse to `sensor(id:)`), the same query as a live
  SSE stream, Prometheus `/metrics`, read-only `sensors:read` tokens (never full tokens in
  a URL), status sensors (services, updates, ZFS, SMART), a widget library, custom widgets
  on the Dashboard tab (briefly renamed "Monitors" 2026-09-19, back to "Dashboard"
  2026-09-20 at Mitchell's request; the view id and URL were always `dashboard`), and a
  frameable `/embed/widget` for Glance and friends. The stock Dashboard modules and the Disks
  tab read the sensor API too (`sampleAdapter.ts`); `/api/metrics` is retired.
  Also per-service and per-container sensors (memory and CPU from cgroups), tokens limited
  to ID patterns, and `/api/fleet/sensors/query` for the whole fleet in one request.
- **Alerts** — **shipped 2026-09-19** (`internal/api/alerts.go`, `Alerts.tsx`): rules over
  sensors (`container.*.running == false for 120 s`), evaluated by the hub every 15 s for
  itself and every managed machine in one shaped query each; `hamlab.reachable` is supplied
  by the hub so "a machine stopped answering" is a rule like any other. Ten defaults
  (machine offline, container down/unhealthy, failed service, disk full, pool unhealthy,
  SMART failing, UPS on battery, CPU hot, memory). Firing and resolved are events: kept
  (last 200, in `state.json`), shown on the hub-only Alerts tab with a count on the rail,
  and POSTed to channels — **ntfy** (title/priority headers) or a **webhook** (JSON; Home
  Assistant, Slack). The rule editor shows what a selector matches across the fleet as
  you type. Not done: quiet hours, per-rule channels, Web Push straight to the PWA.
- **Docker tab** — **shipped 2026-09-19** (`internal/api/docker.go`, `Docker.tsx`):
  containers grouped by compose stack with live memory and CPU from the container
  sensors, start / stop / restart (asked first), logs that follow (polled with `--since`,
  no held connection), inspect (env *names* only), and images / volumes / networks
  tables. Works on every machine through the hub; `#/grid/docker/immich_server` links to
  one. **Stacks** (`internal/api/stacks.go`): every compose project, found from its
  containers' labels, with its file to edit and Deploy (`up -d`), Update (pull, then
  deploy), Restart and Stop as followed tasks. The file is looked for where the label says
  *and through every container's mounts*, because an orchestrator that runs compose inside
  its own container (Dockhand here; Dockge, Portainer) records a path in its own
  filesystem; compose is then run **inside that container**, so `.env`, relative paths
  and the compose labels stay exactly as deployed (verified: labels identical after a
  redeploy). Save validates with `compose config` first, keeps the previous file as
  `<file>.hamlab-bak`, and never deploys by itself. A stack deployed from another host
  (Dockhand on `core` driving `grid` over tcp) has no file on that machine, so the node
  alone lists it read-only — but **the hub fills that in** (`hubStacks`): asked about a
  node, it resolves the files on itself and runs compose here with `DOCKER_HOST` pointed
  at the node's `tcp://…:2375`, the way it was deployed; the task runs on the hub and the
  node's page follows it. Also per stack: every file it was deployed from (chips), its
  **`.env`** (values masked until "Reveal and edit"; validated in place, put back if
  compose objects), **what Deploy would change** (a line diff against the copy kept at the
  last deploy from here, `<file>.hamlab-deployed`, else the copy the last save replaced;
  `.env` values never shown), and **image update checks** — asked for, never polled, since
  each is a registry request Docker Hub counts; remembered six hours. A save refuses a
  file changed on disk meanwhile.
  **Making and unmaking (2026-09-20):** "+ New stack" — a name, a folder, compose text and
  an optional `.env` (written root-only), checked by compose before it is kept, then
  "Create and deploy" or "Create only". "+ New container" is the same dialog opened on a
  form (image, ports, volumes, environment, restart, command, host networking) that writes
  the compose for you: a container made here is a one-service stack, so it stays editable
  and redeployable, where a bare `docker run` could only be thrown away and typed again.
  New stacks go in a **home** — a folder where two or more stacks are already deployed from
  (Dockhand's `stacks/core` here; `~/stacks` on a machine with none) — and are deployed the
  way their neighbours were (inside `dockhand`, or over tcp to a node from the hub). Homes
  are also scanned, so a stack with no containers ("not deployed": never deployed, or taken
  down) is still listed. **Down** removes a stack's containers; **Delete…** (type its name)
  takes it down and moves its own files to `<home>/.hamlab-trash/<name>-<time>/` — moved,
  not erased; volumes and bind-mounted data are never touched. A stopped container can be
  **removed** (a running one must be stopped first, on purpose). Images: **pull** (a followed
  task), **remove** (unforced: Docker refuses one still in use) and **prune** (dangling only).
  **Volumes (2026-09-20):** each says which containers use it; one that nothing uses can be
  deleted — a named one by typing its name (it may be the data of a stack that is only
  down), an anonymous one by confirming its full name — and "Prune anonymous" clears the
  nameless leftovers of recreated containers, never named volumes. Images are marked "in
  use", and "Remove unused" takes every image no container uses. Nothing is ever forced:
  Docker refuses what is still in use, and that refusal is the answer. Still absent:
  removing networks.
- **Skins** — 13 since 2026-09-19 (`themes.ts`: a palette is one `mk(…)` line): Dark,
  Light, Cyberpunk, Retro, **HUD** (hairlines of cyan on near-black, corner ticks, thin
  type), Nord, Gruvbox, Catppuccin + Latte, Dracula, Tokyo Night, Solarized Dark + Light;
  each repaints the terminal too. **Each machine wears a colour** (`accents.ts`): a stripe
  across the header and its name in the switcher — picked from ten swatches or any colour
  on the Machines page, else derived from its name — saved with the theme on the hub.
- **Multi-host** — **Phase 1 shipped 2026-09-18** (`internal/nodes`, `internal/tlsauto`,
  `MachineSwitcher.tsx`, `Machines.tsx`). One binary, hub and spoke: a host whose
  `config.json` lists `nodes` is a hub; any other is a standalone node with its full UI.
  Node side: `"tls": "auto"` makes a self-signed cert in the data dir and serves HTTPS;
  `POST /api/tokens {password,name}` trades the login password for a long-lived bearer
  token (SHA-256 stored in `tokens`), accepted by the auth middleware alongside the
  cookie; `DELETE /api/tokens/{name}` revokes. Hub side: `POST /api/nodes {name,url,
  password}` peeks the node's certificate (trust on first use, fingerprint pinned),
  obtains a token named `hub:<hostname>`, stores `{name,url,fingerprint,token}`; a
  15 s prober keeps each node's status; `/api/nodes/{name}/…` is a reverse proxy to the
  node's `/api/…` (bearer swapped in, cookie and Origin stripped, streaming flush) that
  carries WebSockets, so terminals, log tails and task output work remotely.
  Frontend: `setMachine()` rewrites every `/api/…` path; the rail has a machine
  switcher at the top and a Machines page to add/remove/open. Pages remount on switch;
  terminals live on the node. Nodes: `grid`, `wyse`. **Phase 3 (update from hub)
  shipped 2026-09-18**: node `POST /api/self-update` takes a binary, checks it answers
  `version`, keeps `hamlab.prev` in the data dir, installs over the executable via
  `sudo install`, replies, then restarts the service; the hub pushes its own executable
  with `POST /api/machines/{name}/update` or `/api/machines-update-all` (architecture
  checked first via `/api/me.arch`), the Machines page shows an **outdated** badge and
  Update buttons, and `hamlab push-nodes` does the same from the CLI — `make deploy` runs
  it last, so one deploy on the hub updates every machine. First install on a new host
  is still by hand (scp binary + `install.sh`, `hamlab passwd`, `tls auto`). Still to
  do: Phase 2 Datacenter overview; optional firewall rule limiting a node's port to
  the hub. **Cross-architecture** (2026-09-18): `make deploy` also builds
  `bin/hamlab-linux-arm64`, `install.sh` installs it beside the binary as
  `hamlab-arm64`, and the hub picks the sibling matching a node's `/api/me.arch` when
  pushing — the Mac mini (Fedora Asahi, aarch64, node `mac`) updates like the rest.
- **Portability + install** — **shipped 2026-09-18** (`internal/setup`, `internal/priv`,
  `install.sh`, `make release`): the binary installs itself (`hamlab install` /
  `uninstall`; systemd or OpenRC; root, sudo or doas), all builds static on amd64, arm64,
  arm, 386 and riscv64, `install.sh` as the curl-able bootstrap with checksums. Every
  privileged call goes through `priv` instead of a literal `sudo -n`. Updates tab acts on
  dnf, zypper, pacman and apk as well as apt. Services list falls back to the plain
  table on systemd < 246; Disks falls back to `/sys/block` without `lsblk`. Listener
  uses `IP_FREEBIND`. Self-update: pre-flight `check` against the live config, atomic
  replace, exec in place. Config default moved to `~/.config/hamlab` (existing
  `~/apps/hamlab` still found by the installer and the CLI). Tested in
  containers: Debian 12, Ubuntu 22.04, Fedora 40, Rocky 8, openSUSE Leap 15.5, Alpine
  3.18/3.20 (OpenRC + doas, non-root), plus the four lab hosts. Not done: an OpenRC
  backend for the Services tab; runit/s6/sysvinit service files; public hosting for
  `install.sh` (the Gitea repo is private, so the one-liner needs `HAMLAB_TOKEN`).
- **Users, sessions and settings that follow the account** — **shipped 2026-09-22**
  (`internal/api/users.go`, `config.User`, `store.Session`/`UserState`, `Login.tsx`,
  `Users.tsx` — a hub-only **Users** tab in the rail, added the same day at Mitchell's
  request, holds everything about accounts: your own password, your sessions, the
  accounts; Settings keeps appearance, rates, tokens and About). **Rail order is per account** too (2026-09-22,
  `railorder.ts`, `Rail.tsx`): "⇅ Arrange" at the foot of the rail puts a handle on every
  page; drag one up or down (pointer events, so a finger works), let go and the order is
  saved in the UI prefs with the theme; "Reset" restores the built-in order; pages added
  in a later build go after the ones the saved order names. The login is a user name and a password. `config.json` holds `users`
  (name, argon2id hash, created, last seen); the old single `password_hash` becomes the
  account named after the OS user the console runs as on first boot, its signed-in
  sessions become that account's, and the machine-wide theme/accents/dashboard become
  theirs — nothing to redo after the upgrade. A session carries its user, an ID, when it
  was made and last seen, the IP and the browser; **Users → Sessions** lists yours
  ("Safari on iPhone · 10.0.0.5 · seen 2 min ago"), revokes one, or signs out everywhere
  else. **Users → Users** lists accounts with their session counts, adds one, removes
  one (their sessions and settings go too; never yourself, never the last), and resets
  another's password (they are signed out); removing and resetting take your own
  password first. Everyone signed in can do everything — a household console, not a
  tenancy — and each machine keeps its own accounts: a hub reaches a node with its token,
  so pairing (Machines → Add) now takes a user and password on the node, the user
  optional while it has one account. Per account: `ui` (theme, accents) and `dashboard`
  in `state.json` under `users.<name>`. Still the machine's: refresh rate (it is how
  often every machine samples — it rides in the UI prefs object for the client but is
  stored once, machine-wide), graph history, alert rules, terminals (the shells run as
  one OS user anyway). A login attempt for a missing user takes as long as a wrong
  password, so the form can't list accounts. Every write's log line now names the
  account or token (`core@10.0.0.5 via …`), and Access tokens say who issued them. CLI:
  `hamlab passwd [user]` (makes the account if new; reads stdin when piped), `hamlab
  users`, `hamlab user-rm <user>`; `hamlab install` names the first account after the
  service user. Not done: a read-only viewer role.
- **Space tab** — **shipped 2026-09-22** (`internal/space`, `internal/api/space.go`,
  `Space.tsx`), asked for as "exactly WizTree": what is using a disk. The unit is a whole
  volume, picked from a dropdown at the top: a **disk on its own** (every filesystem its
  partitions back) or a **ZFS pool as a whole** (every mounted dataset) — a pool's member
  disks are listed under it with the layout (raidz2, 8 disks: sda …) and never offered
  apart, since one disk's share of a pool means nothing (Mitchell's call, same day, after a
  first cut that listed the disks as chips). The walk runs as a helper process, `hamlab space-walk <roots>`,
  as root through `priv` when the host allows (so `/var/lib/docker` and `/root` count), 32
  workers reading directories depth-first, never crossing a filesystem boundary by itself;
  each root is grafted into one tree at its path. Files under 128 KiB are not kept one by one
  but summed per folder as "N smaller files" — what keeps a scan of millions of files to tens
  of MB in the daemon; the tree crosses back as a varint stream (`codec.go`), and the helper's
  memory goes with it. core's root: 646k files in ~1.2 s. Progress (files, bytes, current
  folder) every 250 ms; Cancel sends SIGTERM. The page: picker → Scan/Rescan → **Folders**
  (a tree largest-first: size, share of parent as a bar, files, folders, modified; expand in
  place) over a **treemap** of the selected folder — squarified, nested to the file, tiles
  coloured by kind (video, images, audio, archives & disk images, documents, code & config,
  databases & logs, other; the validated categorical set, dark and light) with folders as
  labelled boxes; hover for path, size and share; click a folder to drill, breadcrumbs and ↑
  to come back. The map asks the server for the folder pruned to what a tile could show
  (`min` = 1/6000 of the folder, 24 deep; deeper folders come back as one lump), the list
  one level at a time. **Largest files**: the 1,000 biggest with a path filter. Works on
  every machine through the hub.
  **Smarter and faster (2026-09-23, at Mitchell's request):** a scan is *kept* — in memory
  and in `<data>/space/<target>.scan` (the same varint stream) + `.json`, read back after a
  restart — and the next scan *starts from it*: the helper is handed the last tree on stdin
  and reads a directory again only when something says it changed. On a ZFS pool with
  snapshots, the scan records the newest snapshot of each dataset at start (`bases`), and the
  next one runs `zfs diff -H -F <base>` per dataset: only the directories it names are
  read, their ancestors kept but looked into, everything else kept wholesale without a
  stat. The diffs run side by side and one is given four minutes (its process group is
  killed, sudo and zfs both), after which that dataset is walked by mtime instead — on
  `tank`, `immich` changes constantly and its diff ran past twenty minutes the first time,
  longer than walking it. Elsewhere (ext4, or a dataset with no snapshot) a directory whose mtime — to the
  nanosecond — is unchanged keeps its list and only its kept files are stat'ed again, since
  a file can grow without its directory noticing. A **cancelled** scan keeps what it walked,
  marks the folders it never got to (`partial`, "not yet read" in the tree) and **Resume**
  reads just those. core: full 1.2 s → refresh 0.3 s (2 folders read, 84,755 kept). The
  status line says which (`method`, `by_root`, read/kept counts); **Forget** drops the saved
  result so the next scan is full. The old result stays browsable while a refresh runs.
  **Treemap rethought** the same day: every file a tile grouped by folder — folders are only
  a thin line and a name strip on the first level — instead of nested boxes with headers,
  which on Linux's single-child folder chains was a staircase of headers around nothing.
  Not done: delete from here, a size-over-time history, per-user column choices.
- **Auth hardening** — WebAuthn passkeys (phone Face ID), TOTP fallback, optional NPM
  access list restricting to WireGuard/LAN.
- **Files tab** — **shipped 2026-09-18** (`internal/files`, `Files.tsx`): browse from
  `/` with breadcrumbs, an editable path box, sort, hidden-file toggle and a filter;
  rows show type icon, size, modified, perms and owner, link targets and broken links.
  Text files (≤ 2 MiB, sniffed) open in an editor with Ctrl+S / Save; images preview;
  everything else downloads. Upload via drag-and-drop or picker with per-file progress
  (streamed, no size limit). New folder / file, rename, move, chmod, copy path, delete
  (folders need the name typed back), and "open in terminal", which `cd`s the active
  shell (or a fresh one) to the folder. **Sudo toggle**: root access re-executes the
  same binary as `hamlab fs <op>` under `sudo -n`, streaming bodies over stdin/stdout,
  so both modes share one implementation and one set of rules (absolute paths only;
  `/`, `/etc`, `/usr`, … are never deleted or moved whole). Every write is logged with
  the caller's IP.
- **Docker read-only view** — `docker ps`/logs; dockhand stays the source of truth.
- **Notifications** — push on service failure / SMART warning / UPS event.
- **Terminal extras** — search, copy-mode, font/theme, split panes, upload via
  drag-drop into cwd.

- **Settings + themes** — **shipped 2026-09-18** (`Settings.tsx`, `themes.ts`): Settings
  page with Appearance (Dark, Light, Cyberpunk, Retro — CSS variables on `<html>` plus a
  matching xterm palette, applied before first paint from localStorage and synced hub-side
  via `GET/PUT /api/prefs/ui` so the choice follows the login), Account (change password,
  `POST /api/password`, current password required), Access tokens (list/revoke the tokens
  hubs hold), About (version, host, arch, uptime, certificate).

## To do

- **A public download page, and an updater that uses it** — asked for 2026-09-22, not
  started. Today a release is *published* but not *offered*: `make publish` writes five
  architectures, `SHA256SUMS`, `VERSION` and `install.sh` into
  `/home/core/apps/hamlab-demo/site/dl/` (public at `https://hamlab-demo.bigham.dev`), and
  the only way in is `curl -fsSL https://hamlab-demo.bigham.dev/install.sh | sh`. Updating
  an existing install is a *hub* pushing its own binary to machines it manages
  (`POST /api/self-update`, `hamlab push-nodes`, the Update buttons on the Machines page) —
  which does nothing for an install that no hub of mine owns. Two halves:
  - **The page.** A human-facing page at the download site: current version and when it was
    built, a row per architecture with its size and checksum, the one-line install, and how
    to update. Today `/dl/` is a bare directory listing. It should be generated by
    `make publish` from the same VERSION and SHA256SUMS the release already writes, so it
    cannot drift from the binaries beside it, and it should say plainly that the binaries
    are unsigned and served over TLS from my own box.
  - **The updater.** A Ham Lab anywhere asks the download site what the current version is,
    says so in the UI when it is behind (the About card, and probably a rail badge like
    Alerts has), and updates itself on request: fetch the binary for its own architecture,
    check it against `SHA256SUMS`, then the same path `POST /api/self-update` already
    takes — verify it runs, install atomically, re-exec in place. Decisions to make when it
    is built: where the check lives (a `update_url` in `config.json`, defaulting to the
    public site, so someone else's install can point elsewhere or switch it off); how often
    it asks (daily is plenty, and it must be possible to say never); whether it ever updates
    unattended (default no — an update restarts the daemon and drops open terminals); and
    whether a hub offers to update its machines from the site when it is itself behind.
    Downgrades are already refused by the installer's `vnum()` comparison; the updater
    should use the same rule.

## Open questions

- Idle-session reaping policy (never / after N days) — leaning never; it's a
  single-user box.
- Whether `sudo -i`-style root sessions get their own leaf colour. Cosmetic.