hamlab/PLAN.md

Ham Lab — MVP plan

Proxmox-style web console for a bare-metal Debian 13 host. First target: core (10.0.0.234). MVP is service + shell + persistent terminals; everything else is a stub tab or a roadmap item.

Decisions

Area	Decision	Why
Backend	Go 1.24, single static binary, frontend embedded via embed	core has ~370 MiB free RAM; a Go daemon idles at ~15 MiB vs ~80+ for Node. One file to deploy, no nvm path in the unit. Same binary later installs on grid unchanged.
PTY	creack/pty — server-owned PTYs running core's login shell	Sessions live in the daemon, not the browser, so refresh/tab-switch reattaches. No tmux dependency (not installed on core).
Transport	One WebSocket per attached terminal; REST for everything else	Standard; NPM passes WebSockets with one toggle.
Frontend	React + TypeScript + Vite, xterm.js (+ fit, webgl, web-links addons)	Matches calorie-counter / wedding-website, so no new toolchain to learn.
Auth	Single-user password login, argon2id hash, HttpOnly/Secure/SameSite=Strict session cookie, login rate-limit, Origin check on WS upgrade	This is a root-capable shell on the public internet. Auth is MVP, not roadmap.
Service	hamlab.service, User=core, Restart=always, binary at /usr/local/bin/hamlab	Same pattern as calorie-counter.service. Runs as core so sudo works in the terminal without a password, as on the SSH path.
Listen	10.0.0.234:8100 (free)	NPM is a container, so loopback-only won't do. Firewalling to NPM is a later hardening step.
Data	/home/core/apps/hamlab/ — config.yaml, state.db (SQLite)	apps/ is the nightly rsync target to grid, so it's backed up for free.
Ingress	NPM proxy host hamlab.bigham.dev → :8100, LE cert, WebSockets on, caching off, force-SSL + HSTS + HTTP/2 set in a second API call (NPM drops them on create-with-new-cert)	*.bigham.dev already resolves; the proxy host alone makes it live.
Repo	Gitea mitchell/hamlab, private, main	Created 2026-09-17.

Layout

┌──────────────────────────────────────────────────────────────┐
│ header: Ham Lab · core · load 0.42 · ● connected · [logout]    │
├──────────────┬───────────────────────────────────────────────┤
│ Dashboard    │ shell 1 ● │ backups │ +                      │
│ Processes    ├───────────────────────────────────────────────┤
│ Terminal     │                                               │
│ Services     │        main content (xterm)                   │
│ Cron         │                                               │
│ …            │                                               │
├──────────────┴───────────────────────────────────────────────┤
│ footer: v0.1.0 · up 12d · hamlab.bigham.dev                   │
└──────────────────────────────────────────────────────────────┘

• Left rail: flat list of pages, active one highlighted. No nested leaves.
• Terminal page: tab strip across the top of the pane, one tab per session, ● when a process is running, + pinned at the right edge. Active tab carries rename and close; double-click renames, middle-click closes. (Until 2026-09-18 the sessions were leaves under a collapsible Terminal group in the rail.)
• Non-MVP groups render as disabled leaves with "coming soon" — the rail shape is the product; stubs keep it honest about scope.
• Mobile: rail becomes a slide-in drawer; xterm fits the viewport; a sticky key bar above the keyboard (Esc Tab Ctrl Alt ↑ ↓ ← → |) because phone keyboards have none of those. This is what makes it usable as a PWA rather than a novelty.

Terminal semantics

• A session is a PTY + shell owned by the daemon, with an ID, name, cwd hint, and a scrollback ring buffer (256 KiB). Attaching replays the buffer, then streams.
• Any number of browser tabs can attach to the same session (all get output; all can type). Resize follows the most recent attacher.
• Switching tabs in the UI just detaches/attaches; the shell keeps running.
• Closing a session sends SIGHUP to the process group and drops it. Shell exit closes the session automatically (leaf shows "exited" until dismissed).
• Session names persist in state.db so the sidebar order survives a service restart. The shells themselves do not survive a daemon restart in the MVP — see roadmap (tmux backing).
• Idle sessions cost one shell process each; no automatic reaping in MVP.

API (MVP)

POST /api/login            {password}            → 204 + cookie
POST /api/logout                                 → 204
GET  /api/me                                     → {host, user, version, uptime, load}
GET  /api/terminals                              → [{id, name, running, created}]
POST /api/terminals        {name?}               → {id, ...}
PATCH /api/terminals/:id   {name}                → 204
DELETE /api/terminals/:id                        → 204
WS   /api/terminals/:id/attach   binary frames = raw bytes; JSON text frames = {resize}
GET  /healthz                                    → 200 (unauthenticated, for NPM)

Everything under /api except login and /healthz requires the session cookie.

Project layout

hamlab/
  cmd/hamlab/main.go          serve | passwd | version
  internal/
    auth/                     argon2id, sessions, rate limit
    term/                     PTY manager, ring buffer, attach/detach
    api/                      chi router, handlers, WS
    state/                    SQLite (modernc, no cgo)
    web/                      go:embed of web/dist
  web/                        Vite + React + TS + xterm.js, PWA manifest + SW
  deploy/
    hamlab.service
    install.sh                build → /usr/local/bin/hamlab → systemctl restart
  Makefile                    build / dev / deploy / lint / test

Milestones

M0 — skeleton (service boots, login works)

• Go module, chi, config loading, hamlab passwd, argon2id login, session cookie.
• hamlab.service on core, :8100, serves a placeholder page.
• make deploy from core itself.

M1 — terminals (the MVP feature)

• PTY manager with ring buffer, create/list/rename/close, WS attach with resize.
• React shell: header, rail, footer, Terminal group, xterm main pane.
• Persistence of session list across refresh and rail switching.

M2 — polish + PWA + ingress

• Mobile drawer, key bar, fit-on-rotate.
• manifest.webmanifest, icons (Ham logo on black, same recipe as HamTV), service worker caching the app shell only (never /api).
• NPM proxy host for hamlab.bigham.dev; verify WS through NPM; force-SSL/HSTS/HTTP2.
• Stub groups for Services / Cron / ZFS / Dashboard.

Done when: from the phone, open the PWA, log in, open two terminals, run something long in one, switch away and back, refresh — it's all still there.

Roadmap (not MVP — notes only)

• Updates tab — shipped 2026-09-17 (internal/updates, internal/tasks, Updates.tsx): upgradable packages with current → candidate, suite, security badge and description; index age, last upgrade (from /var/log/apt/history.log), unattended-upgrades on/off, reboot-required banner with the packages that caused it; Refresh index, Upgrade all (dist-upgrade), Upgrade selected; changelog per package on demand. Runs go through the new tasks package: background command, output kept, followed live over /api/tasks/{id}/stream, cancellable, listed at /api/tasks (the seed of a Task History tab). apt is fully supported; dnf/pacman list only.

• Power — header ⏻ button (amber when a reboot is pending), dialog that requires the hostname typed back, then systemctl reboot / poweroff. /api/me carries reboot_needed so the header knows.

• tmux-backed sessions so shells survive a daemon restart / redeploy. Would spawn tmux new -A -s hamlab-<id>; needs mouse on and status-bar handling in xterm.

• Services tab — systemctl list-units, per-unit page: status, last 200 journal lines (live-tail via WS), start/stop/restart/enable buttons, unit file viewer.

• Cron tab — shipped 2026-09-17 (internal/cron, Cron.tsx): one list built from /etc/crontab, /etc/cron.d/*, every user crontab (via sudo), the scripts in the run-parts directories those drive, plus systemd timers. Next run computed by an in-house cron-expression evaluator (tested: ranges, steps, names, dom-OR-dow rule, @ aliases); last run from the journal's (user) CMD (command) lines (cron logs via syslog to journald; reading needs sudo/adm). Commented-out jobs show as disabled; job-shaped lines inside comment blocks (Debian's header examples) are ignored. Per job: Run now (background, as the job's user, sh -c with crontab % rules, output captured), Enable/Disable (comments the line in place, refuses if the file changed underneath), Edit source (textarea; validated line-by-line; user crontabs installed with crontab -, system files written via sudo tee), Cron log, and Output file when the command redirects to one. Timers: Run now (starts the activated service), start/stop/enable/disable, live logs of the service via the Services tab's tail.

• Disks tab — shipped 2026-09-17 (internal/storage, Disks.tsx), replacing the planned ZFS tab: block devices from lsblk -J (kind, model, size, temperature from the device's hwmon node, partitions with filesystem usage, live read/write/busy from the metrics sampler); SMART via smartctl -j when smartmontools is present (one button installs it as a task otherwise): health, temperature, power-on time, self-test status, ATA attribute table with the failure-predicting IDs flagged, NVMe endurance and error counters, short/long self-tests. Spinning disks are read with -n standby so the tab never wakes a sleeping drive; "Wake and read" is explicit. ZFS section when the tools and module are there: pools with health, capacity, fragmentation, last scan and vdev tree; scrub start/pause/stop; datasets tree with used/refer/in-snapshots/ snapshot count/compressratio/mount; per-dataset snapshot list and "Snapshot now". Verified against grid (raidz2 tank, 10 datasets, 515 snapshots, 8 WD40EZAZ + 970 EVO Plus SMART). Snapshot destroy and dataset property editing are deliberately not there yet.

• Dashboard — first cut shipped 2026-09-17 (internal/metrics, Dashboard.tsx): CPU (total + per core), memory/swap, load, uptime, hwmon temps and fans read straight from /sys (no lm-sensors), per-interface network rates, per-disk read/write/busy from /proc/diskstats, filesystem usage, UPS via sudo -n pwrstat -status. Sampled every 2 s into an in-memory ring of 10 min; the UI (the Dashboard tab since 2026-09-19) reads it all through the sensor API — see Sensors below. Customizable: a Customize mode reorders and hides modules (CPU, Memory, Load, Uptime, Temperatures, Fans, Network, Disks, Filesystems, UPS) and hides individual rows (an interface, a mount, a disk, a sensor); the layout is stored server-side in state.json via GET/PUT /api/prefs/dashboard so every device sees the same dashboard. The update interval (100 ms – 2 s) is part of the layout and drives both the browser's polling and the daemon's sampling clock; the chart window is always 300 samples, so 30 s at 100 ms and 10 min at 2 s. Still to do: persistent history (SQLite ring, e.g. 10 s × 7 days), SMART (smartctl -j), longer time ranges.

• Sensors — shipped 2026-09-19 (internal/sensors, internal/metrics/sensors.go, internal/api/status_sensors.go, web/src/sensors.ts, widgets.tsx, Embed.tsx; reference in SENSORS.md): everything measured is a sensor with a stable dotted ID; catalog, single value, history, a shaped query (the client sends the object it wants and gets it back filled in — GraphQL's idea without GraphQL, since sensors differ per machine and a static schema would collapse to sensor(id:)), the same query as a live SSE stream, Prometheus /metrics, read-only sensors:read tokens (never full tokens in a URL), status sensors (services, updates, ZFS, SMART), a widget library, custom widgets on the Dashboard tab (briefly renamed "Monitors" 2026-09-19, back to "Dashboard" 2026-09-20 at Mitchell's request; the view id and URL were always dashboard), and a frameable /embed/widget for Glance and friends. The stock Dashboard modules and the Disks tab read the sensor API too (sampleAdapter.ts); /api/metrics is retired. Also per-service and per-container sensors (memory and CPU from cgroups), tokens limited to ID patterns, and /api/fleet/sensors/query for the whole fleet in one request.

• Alerts — shipped 2026-09-19 (internal/api/alerts.go, Alerts.tsx): rules over sensors (container.*.running == false for 120 s), evaluated by the hub every 15 s for itself and every managed machine in one shaped query each; hamlab.reachable is supplied by the hub so "a machine stopped answering" is a rule like any other. Ten defaults (machine offline, container down/unhealthy, failed service, disk full, pool unhealthy, SMART failing, UPS on battery, CPU hot, memory). Firing and resolved are events: kept (last 200, in state.json), shown on the hub-only Alerts tab with a count on the rail, and POSTed to channels — ntfy (title/priority headers) or a webhook (JSON; Home Assistant, Slack). The rule editor shows what a selector matches across the fleet as you type. Not done: quiet hours, per-rule channels, Web Push straight to the PWA.

• Docker tab — shipped 2026-09-19 (internal/api/docker.go, Docker.tsx): containers grouped by compose stack with live memory and CPU from the container sensors, start / stop / restart (asked first), logs that follow (polled with --since, no held connection), inspect (env names only), and images / volumes / networks tables. Works on every machine through the hub; #/grid/docker/immich_server links to one. Stacks (internal/api/stacks.go): every compose project, found from its containers' labels, with its file to edit and Deploy (up -d), Update (pull, then deploy), Restart and Stop as followed tasks. The file is looked for where the label says and through every container's mounts, because an orchestrator that runs compose inside its own container (Dockhand here; Dockge, Portainer) records a path in its own filesystem; compose is then run inside that container, so .env, relative paths and the compose labels stay exactly as deployed (verified: labels identical after a redeploy). Save validates with compose config first, keeps the previous file as <file>.hamlab-bak, and never deploys by itself. A stack deployed from another host (Dockhand on core driving grid over tcp) has no file on that machine, so the node alone lists it read-only — but the hub fills that in (hubStacks): asked about a node, it resolves the files on itself and runs compose here with DOCKER_HOST pointed at the node's tcp://…:2375, the way it was deployed; the task runs on the hub and the node's page follows it. Also per stack: every file it was deployed from (chips), its .env (values masked until "Reveal and edit"; validated in place, put back if compose objects), what Deploy would change (a line diff against the copy kept at the last deploy from here, <file>.hamlab-deployed, else the copy the last save replaced; .env values never shown), and image update checks — asked for, never polled, since each is a registry request Docker Hub counts; remembered six hours. A save refuses a file changed on disk meanwhile. Making and unmaking (2026-09-20): "+ New stack" — a name, a folder, compose text and an optional .env (written root-only), checked by compose before it is kept, then "Create and deploy" or "Create only". "+ New container" is the same dialog opened on a form (image, ports, volumes, environment, restart, command, host networking) that writes the compose for you: a container made here is a one-service stack, so it stays editable and redeployable, where a bare docker run could only be thrown away and typed again. New stacks go in a home — a folder where two or more stacks are already deployed from (Dockhand's stacks/core here; ~/stacks on a machine with none) — and are deployed the way their neighbours were (inside dockhand, or over tcp to a node from the hub). Homes are also scanned, so a stack with no containers ("not deployed": never deployed, or taken down) is still listed. Down removes a stack's containers; Delete… (type its name) takes it down and moves its own files to <home>/.hamlab-trash/<name>-<time>/ — moved, not erased; volumes and bind-mounted data are never touched. A stopped container can be removed (a running one must be stopped first, on purpose). Images: pull (a followed task), remove (unforced: Docker refuses one still in use) and prune (dangling only). Volumes (2026-09-20): each says which containers use it; one that nothing uses can be deleted — a named one by typing its name (it may be the data of a stack that is only down), an anonymous one by confirming its full name — and "Prune anonymous" clears the nameless leftovers of recreated containers, never named volumes. Images are marked "in use", and "Remove unused" takes every image no container uses. Nothing is ever forced: Docker refuses what is still in use, and that refusal is the answer. Still absent: removing networks.

• Skins — 13 since 2026-09-19 (themes.ts: a palette is one mk(…) line): Dark, Light, Cyberpunk, Retro, HUD (hairlines of cyan on near-black, corner ticks, thin type), Nord, Gruvbox, Catppuccin + Latte, Dracula, Tokyo Night, Solarized Dark + Light; each repaints the terminal too. Each machine wears a colour (accents.ts): a stripe across the header and its name in the switcher — picked from ten swatches or any colour on the Machines page, else derived from its name — saved with the theme on the hub.

• Multi-host — Phase 1 shipped 2026-09-18 (internal/nodes, internal/tlsauto, MachineSwitcher.tsx, Machines.tsx). One binary, hub and spoke: a host whose config.json lists nodes is a hub; any other is a standalone node with its full UI. Node side: "tls": "auto" makes a self-signed cert in the data dir and serves HTTPS; POST /api/tokens {password,name} trades the login password for a long-lived bearer token (SHA-256 stored in tokens), accepted by the auth middleware alongside the cookie; DELETE /api/tokens/{name} revokes. Hub side: POST /api/nodes {name,url, password} peeks the node's certificate (trust on first use, fingerprint pinned), obtains a token named hub:<hostname>, stores {name,url,fingerprint,token}; a 15 s prober keeps each node's status; /api/nodes/{name}/… is a reverse proxy to the node's /api/… (bearer swapped in, cookie and Origin stripped, streaming flush) that carries WebSockets, so terminals, log tails and task output work remotely. Frontend: setMachine() rewrites every /api/… path; the rail has a machine switcher at the top and a Machines page to add/remove/open. Pages remount on switch; terminals live on the node. Nodes: grid, wyse. Phase 3 (update from hub) shipped 2026-09-18: node POST /api/self-update takes a binary, checks it answers version, keeps hamlab.prev in the data dir, installs over the executable via sudo install, replies, then restarts the service; the hub pushes its own executable with POST /api/machines/{name}/update or /api/machines-update-all (architecture checked first via /api/me.arch), the Machines page shows an outdated badge and Update buttons, and hamlab push-nodes does the same from the CLI — make deploy runs it last, so one deploy on the hub updates every machine. First install on a new host is still by hand (scp binary + install.sh, hamlab passwd, tls auto). Still to do: Phase 2 Datacenter overview; optional firewall rule limiting a node's port to the hub. Cross-architecture (2026-09-18): make deploy also builds bin/hamlab-linux-arm64, install.sh installs it beside the binary as hamlab-arm64, and the hub picks the sibling matching a node's /api/me.arch when pushing — the Mac mini (Fedora Asahi, aarch64, node mac) updates like the rest.

• Portability + install — shipped 2026-09-18 (internal/setup, internal/priv, install.sh, make release): the binary installs itself (hamlab install / uninstall; systemd or OpenRC; root, sudo or doas), all builds static on amd64, arm64, arm, 386 and riscv64, install.sh as the curl-able bootstrap with checksums. Every privileged call goes through priv instead of a literal sudo -n. Updates tab acts on dnf, zypper, pacman and apk as well as apt. Services list falls back to the plain table on systemd < 246; Disks falls back to /sys/block without lsblk. Listener uses IP_FREEBIND. Self-update: pre-flight check against the live config, atomic replace, exec in place. Config default moved to ~/.config/hamlab (existing ~/apps/hamlab still found by the installer and the CLI). Tested in containers: Debian 12, Ubuntu 22.04, Fedora 40, Rocky 8, openSUSE Leap 15.5, Alpine 3.18/3.20 (OpenRC + doas, non-root), plus the four lab hosts. Not done: an OpenRC backend for the Services tab; runit/s6/sysvinit service files; public hosting for install.sh (the Gitea repo is private, so the one-liner needs HAMLAB_TOKEN).

• Users, sessions and settings that follow the account — shipped 2026-09-22 (internal/api/users.go, config.User, store.Session/UserState, Login.tsx, Users.tsx — a hub-only Users tab in the rail, added the same day at Mitchell's request, holds everything about accounts: your own password, your sessions, the accounts; Settings keeps appearance, rates, tokens and About). Rail order is per account too (2026-09-22, railorder.ts, Rail.tsx): "⇅ Arrange" at the foot of the rail puts a handle on every page; drag one up or down (pointer events, so a finger works), let go and the order is saved in the UI prefs with the theme; "Reset" restores the built-in order; pages added in a later build go after the ones the saved order names. The login is a user name and a password. config.json holds users (name, argon2id hash, created, last seen); the old single password_hash becomes the account named after the OS user the console runs as on first boot, its signed-in sessions become that account's, and the machine-wide theme/accents/dashboard become theirs — nothing to redo after the upgrade. A session carries its user, an ID, when it was made and last seen, the IP and the browser; Users → Sessions lists yours ("Safari on iPhone · 10.0.0.5 · seen 2 min ago"), revokes one, or signs out everywhere else. Users → Users lists accounts with their session counts, adds one, removes one (their sessions and settings go too; never yourself, never the last), and resets another's password (they are signed out); removing and resetting take your own password first. Everyone signed in can do everything — a household console, not a tenancy — and each machine keeps its own accounts: a hub reaches a node with its token, so pairing (Machines → Add) now takes a user and password on the node, the user optional while it has one account. Per account: ui (theme, accents) and dashboard in state.json under users.<name>. Still the machine's: refresh rate (it is how often every machine samples — it rides in the UI prefs object for the client but is stored once, machine-wide), graph history, alert rules, terminals (the shells run as one OS user anyway). A login attempt for a missing user takes as long as a wrong password, so the form can't list accounts. Every write's log line now names the account or token (core@10.0.0.5 via …), and Access tokens say who issued them. CLI: hamlab passwd [user] (makes the account if new; reads stdin when piped), hamlab users, hamlab user-rm <user>; hamlab install names the first account after the service user. Not done: a read-only viewer role.

• Space tab — shipped 2026-09-22 (internal/space, internal/api/space.go, Space.tsx), asked for as "exactly WizTree": what is using a disk. The unit is a whole volume, picked from a dropdown at the top: a disk on its own (every filesystem its partitions back) or a ZFS pool as a whole (every mounted dataset) — a pool's member disks are listed under it with the layout (raidz2, 8 disks: sda …) and never offered apart, since one disk's share of a pool means nothing (Mitchell's call, same day, after a first cut that listed the disks as chips). The walk runs as a helper process, hamlab space-walk <roots>, as root through priv when the host allows (so /var/lib/docker and /root count), 32 workers reading directories depth-first, never crossing a filesystem boundary by itself; each root is grafted into one tree at its path. Files under 128 KiB are not kept one by one but summed per folder as "N smaller files" — what keeps a scan of millions of files to tens of MB in the daemon; the tree crosses back as a varint stream (codec.go), and the helper's memory goes with it. core's root: 646k files in ~1.2 s. Progress (files, bytes, current folder) every 250 ms; Cancel sends SIGTERM. The page: picker → Scan/Rescan → Folders (a tree largest-first: size, share of parent as a bar, files, folders, modified; expand in place) over a treemap of the selected folder — squarified, nested to the file, tiles coloured by kind (video, images, audio, archives & disk images, documents, code & config, databases & logs, other; the validated categorical set, dark and light) with folders as labelled boxes; hover for path, size and share; click a folder to drill, breadcrumbs and ↑ to come back. The map asks the server for the folder pruned to what a tile could show (min = 1/6000 of the folder, 24 deep; deeper folders come back as one lump), the list one level at a time. Largest files: the 1,000 biggest with a path filter. Works on every machine through the hub. Smarter and faster (2026-09-23, at Mitchell's request): a scan is kept — in memory and in <data>/space/<target>.scan (the same varint stream) + .json, read back after a restart — and the next scan starts from it: the helper is handed the last tree on stdin and reads a directory again only when something says it changed. On a ZFS pool with snapshots, the scan records the newest snapshot of each dataset at start (bases), and the next one runs zfs diff -H -F <base> per dataset: only the directories it names are read, their ancestors kept but looked into, everything else kept wholesale without a stat. The diffs run side by side and one is given three minutes — timeout -s KILL inside sudo, since Debian's sudo gives the command its own session and a process-group kill from outside never reached zfs — after which that dataset is walked by mtime instead. On tank a diff is metadata-bound like the walk: 8 s to 3 min per dataset for a handful of changes, and immich, which changes constantly, ran past twenty minutes the first time. A refresh of tank took 4 min 19 s against 7 min for the full scan. Elsewhere (ext4, or a dataset with no snapshot) a directory whose mtime — to the nanosecond — is unchanged keeps its list and only its kept files are stat'ed again, since a file can grow without its directory noticing. A cancelled scan keeps what it walked, marks the folders it never got to (partial, "not yet read" in the tree) and Resume reads just those. core: full 1.2 s → refresh 0.3 s (2 folders read, 84,755 kept). The status line says which (method, by_root, read/kept counts); Forget drops the saved result so the next scan is full. The old result stays browsable while a refresh runs. Treemap rethought the same day: every file a tile grouped by folder — folders are only a thin line and a name strip on the first level — instead of nested boxes with headers, which on Linux's single-child folder chains was a staircase of headers around nothing. Not done: delete from here, a size-over-time history, per-user column choices.

• Updater — shipped 2026-09-23 (internal/selfupdate, internal/api/update.go, the Updates card in Settings.tsx, scripts/gitea-release.sh). A Ham Lab asks one URL what the newest build is and installs it on request. update.url in config.json is either a Gitea/Forgejo repo API base (…/api/v1/repos/<owner>/<repo>, whose newest release names the version and carries the binaries as assets) or any static directory holding VERSION, SHA256SUMS and hamlab-linux-<arch> — which is exactly what make release writes and what install.sh already downloads, so the demo site, a folder on a NAS and a Gitea release are all the same thing to it. "off" never looks. update.token is sent when the source is behind a login, so a private Gitea serves its own fleet without the instance being opened to the world. The binary is verified against SHA256SUMS before it goes anywhere near the running one, then takes the same path POST /api/self-update always did (runs version, runs check against this machine's config, atomic replace keeping hamlab.prev, re-exec in place) — that path is now one shared method, installBinary. Version ordering is install.sh's vnum() rule ported to Go and tested against it, so the daemon and the installer never disagree about which of two builds is newer; the commit count since the tag counts, the hash does not. Checked ~90 s after boot and daily (update.check: "never" switches that off), never installed unasked — an update restarts the daemon and drops open terminals, and the dialog says so. Settings → Updates: installed, published, when it last looked, the source, the release notes, Check for updates, Install; the rail badges Settings when a newer build is out (update_available rides on /api/me). Publishing is make gitea-release → scripts/gitea-release.sh, which creates the release and uploads the five architectures, SHA256SUMS, install.sh and VERSION; it is deliberately not part of make deploy, because deploy pushes this build to the machines this hub owns while publishing offers it to every install anywhere, and one tag per deploy would be noise. Re-publishing a version needs --force.

• Auth hardening — WebAuthn passkeys (phone Face ID), TOTP fallback, optional NPM access list restricting to WireGuard/LAN.

• Files tab — shipped 2026-09-18 (internal/files, Files.tsx): browse from / with breadcrumbs, an editable path box, sort, hidden-file toggle and a filter; rows show type icon, size, modified, perms and owner, link targets and broken links. Text files (≤ 2 MiB, sniffed) open in an editor with Ctrl+S / Save; images preview; everything else downloads. Upload via drag-and-drop or picker with per-file progress (streamed, no size limit). New folder / file, rename, move, chmod, copy path, delete (folders need the name typed back), and "open in terminal", which cds the active shell (or a fresh one) to the folder. Sudo toggle: root access re-executes the same binary as hamlab fs <op> under sudo -n, streaming bodies over stdin/stdout, so both modes share one implementation and one set of rules (absolute paths only; /, /etc, /usr, … are never deleted or moved whole). Every write is logged with the caller's IP.

• Docker read-only view — docker ps/logs; dockhand stays the source of truth.

• Notifications — push on service failure / SMART warning / UPS event.

• Terminal extras — search, copy-mode, font/theme, split panes, upload via drag-drop into cwd.

• Settings + themes — shipped 2026-09-18 (Settings.tsx, themes.ts): Settings page with Appearance (Dark, Light, Cyberpunk, Retro — CSS variables on <html> plus a matching xterm palette, applied before first paint from localStorage and synced hub-side via GET/PUT /api/prefs/ui so the choice follows the login), Account (change password, POST /api/password, current password required), Access tokens (list/revoke the tokens hubs hold), About (version, host, arch, uptime, certificate).

To do

• Somewhere public to download from — asked for 2026-09-22, half done. The updater shipped 2026-09-23 (above) and reads releases from Gitea, but git.bigham.dev runs with REQUIRE_SIGNIN_VIEW=true, so those releases are reachable only with a token. That is fine for this fleet — each machine carries a read-only read:repository token — and no use at all to a stranger running the install.sh one-liner, who still gets the rolling copy on hamlab-demo.bigham.dev. What is left is deciding where a public download actually lives:

  • Lower the wall on Gitea. REQUIRE_SIGNIN_VIEW=false and per-repo visibility does the rest: every repo there is private and stays hidden, and only a repo marked public is visible. That is how Gitea is meant to be configured, and it is the one change that makes "releases on my Gitea" literally true. It is an instance-wide posture change on a public hostname, so it is Mitchell's call — the same trade was declined once before (docs/docker.md, the demo site's install.sh is hosted outside Gitea for exactly this reason).
  • Or keep the wall and keep the demo site as the public face: make publish already writes the rolling dl/ directory there, and a generated index page (version, built when, a row per architecture with size and checksum, the one-line install, and that the binaries are unsigned and served from his own box) would finish the "page" half without touching Gitea at all.
  • Either way the release notes and version history live on Gitea, and the updater does not care which of the two it is pointed at.

• Deprecate copyparty — asked for 2026-09-23, not started. copyparty on grid (copyparty.service, :3923, /etc/copyparty.conf, serving all of /tank as user core with one account behind NPM) overlaps the Files tab almost entirely. The aim is to stop the service and keep only Ham Lab. What Files must gain first, in the order that matters:

  • Multi-select and bulk actions. copyparty selects many rows and moves, deletes or downloads them together; Files acts on one row at a time.
  • Download a folder, and a multi-file download, as a streamed zip or tar — the thing copyparty is reached for most.
  • Resumable uploads. copyparty's up2k chunks a file, survives a dropped connection and skips what is already there; Files streams one PUT, so a phone upload that drops at 90% starts over.
  • A gallery. Thumbnails for a folder of images and video, and play audio/video in place — /tank/media and /tank/immich are most of what is browsed.
  • Search below the current folder by name, since find in a terminal is the fallback today.
  • Decide when built: whether share links (copyparty's public per-path links) are wanted at all — they are the one feature that hands out access without a login, and Ham Lab has no notion of that; if not, say so in the docs rather than leaving a gap unexplained.
  • After: systemctl disable --now copyparty, remove its NPM host, keep /etc/copyparty.conf until a rollback is clearly not needed, then update hamlet's docs/hosts.md bare-metal list, docs/gotchas.md (the "docker ps is not the inventory" note names it) and the grid row in CLAUDE.md.

• Deprecate Dockhand — asked for 2026-09-23, not started; the homelab half of this (migrating every stack off Dockhand's data dir, then removing the container) is hamlet's docs/pending.md → "Retire Dockhand; Ham Lab becomes the orchestrator", agreed 2026-09-20. This entry is the app half only: what Ham Lab must do before that migration can start. Its Stacks page already creates, edits, deploys, updates, restarts, stops, downs and deletes stacks on every machine, so the gaps are Dockhand's extras:

  • Git-sourced stacks. Dockhand's stack_sources table points a stack at a repo and pulls it. Ham Lab has no equivalent. Decide whether to build it or to drop it — most stacks here have no row in that table anyway, so dropping it may be honest.
  • Scheduled image updates. Ham Lab checks for new images only when asked (a registry request Docker Hub counts) and never updates by itself. If anything relies on Dockhand updating on a schedule, that must move here or be dropped on purpose.
  • Webhooks / per-host environments, if either is in use — inventory before deciding.
  • Deploying from a host path without an orchestrator container. Ham Lab already runs compose inside dockhand because that is how the stacks were deployed; once they move to a neutral home the "via" path is unused for them, but a stack deployed to grid or the Mac still goes over tcp://<node>:2375 from the hub. Decide whether Ham Lab should instead ask the node's own Ham Lab to deploy locally, which would end the *:2375 exposure (its own item in docs/pending.md).
  • Not app work, but the trigger: nothing here blocks starting the migration except the inventory. Do that first.

Open questions

• Idle-session reaping policy (never / after N days) — leaning never; it's a single-user box.
• Whether sudo -i-style root sessions get their own leaf colour. Cosmetic.