Mitchell-JohnsonandClaude Opus 5
363ff02bda
The hamlab-demo site, its Widgets-tab button, the demo_url config key and the make publish target that fed it are all gone, along with the public read-only token it needed. The example stays in examples/external-demo as something to read. The installer one-liner already comes from Gitea releases. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
495 lines
38 KiB
Markdown
495 lines
38 KiB
Markdown
# Ham Lab — MVP plan
|
||
|
||
Proxmox-style web console for a bare-metal Debian 13 host. First target: `core`
|
||
(10.0.0.234). MVP is **service + shell + persistent terminals**; everything else is a
|
||
stub tab or a roadmap item.
|
||
|
||
## Decisions
|
||
|
||
| Area | Decision | Why |
|
||
|---|---|---|
|
||
| Backend | **Go 1.24**, single static binary, frontend embedded via `embed` | `core` has ~370 MiB free RAM; a Go daemon idles at ~15 MiB vs ~80+ for Node. One file to deploy, no nvm path in the unit. Same binary later installs on `grid` unchanged. |
|
||
| PTY | `creack/pty` — server-owned PTYs running `core`'s login shell | Sessions live in the daemon, not the browser, so refresh/tab-switch reattaches. No tmux dependency (not installed on `core`). |
|
||
| Transport | One WebSocket per attached terminal; REST for everything else | Standard; NPM passes WebSockets with one toggle. |
|
||
| Frontend | **React + TypeScript + Vite**, `xterm.js` (+ fit, webgl, web-links addons) | Matches `calorie-counter` / `wedding-website`, so no new toolchain to learn. |
|
||
| Auth | Single-user **password login**, argon2id hash, HttpOnly/Secure/SameSite=Strict session cookie, login rate-limit, Origin check on WS upgrade | This is a root-capable shell on the public internet. Auth is MVP, not roadmap. |
|
||
| Service | `hamlab.service`, `User=core`, `Restart=always`, binary at `/usr/local/bin/hamlab` | Same pattern as `calorie-counter.service`. Runs as `core` so `sudo` works in the terminal without a password, as on the SSH path. |
|
||
| Listen | `10.0.0.234:8100` (free) | NPM is a container, so loopback-only won't do. Firewalling to NPM is a later hardening step. |
|
||
| Data | `/home/core/apps/hamlab/` — `config.yaml`, `state.db` (SQLite) | `apps/` is the nightly rsync target to `grid`, so it's backed up for free. |
|
||
| Ingress | NPM proxy host `hamlab.bigham.dev` → `:8100`, LE cert, WebSockets on, caching off, force-SSL + HSTS + HTTP/2 set in a **second** API call (NPM drops them on create-with-new-cert) | `*.bigham.dev` already resolves; the proxy host alone makes it live. |
|
||
| Repo | Gitea `mitchell/hamlab`, private, `main` | Created 2026-09-17. |
|
||
|
||
## Layout
|
||
|
||
```
|
||
┌──────────────────────────────────────────────────────────────┐
|
||
│ header: Ham Lab · core · load 0.42 · ● connected · [logout] │
|
||
├──────────────┬───────────────────────────────────────────────┤
|
||
│ Dashboard │ shell 1 ● │ backups │ + │
|
||
│ Processes ├───────────────────────────────────────────────┤
|
||
│ Terminal │ │
|
||
│ Services │ main content (xterm) │
|
||
│ Cron │ │
|
||
│ … │ │
|
||
├──────────────┴───────────────────────────────────────────────┤
|
||
│ footer: v0.1.0 · up 12d · hamlab.bigham.dev │
|
||
└──────────────────────────────────────────────────────────────┘
|
||
```
|
||
|
||
- Left rail: flat list of pages, active one highlighted. No nested leaves.
|
||
- Terminal page: tab strip across the top of the pane, one tab per session, `● ` when
|
||
a process is running, `+` pinned at the right edge. Active tab carries rename and
|
||
close; double-click renames, middle-click closes. (Until 2026-09-18 the sessions
|
||
were leaves under a collapsible Terminal group in the rail.)
|
||
- Non-MVP groups render as disabled leaves with "coming soon" — the rail shape is
|
||
the product; stubs keep it honest about scope.
|
||
- **Mobile**: rail becomes a slide-in drawer; xterm fits the viewport; a sticky key
|
||
bar above the keyboard (`Esc Tab Ctrl Alt ↑ ↓ ← → |`) because phone keyboards
|
||
have none of those. This is what makes it usable as a PWA rather than a novelty.
|
||
|
||
## Terminal semantics
|
||
|
||
- A session is a PTY + shell owned by the daemon, with an ID, name, cwd hint, and a
|
||
**scrollback ring buffer** (256 KiB). Attaching replays the buffer, then streams.
|
||
- Any number of browser tabs can attach to the same session (all get output; all can
|
||
type). Resize follows the most recent attacher.
|
||
- Switching tabs in the UI just detaches/attaches; the shell keeps running.
|
||
- Closing a session sends `SIGHUP` to the process group and drops it. Shell exit
|
||
closes the session automatically (leaf shows "exited" until dismissed).
|
||
- Session names persist in `state.db` so the sidebar order survives a service
|
||
restart. **The shells themselves do not survive a daemon restart** in the MVP —
|
||
see roadmap (tmux backing).
|
||
- Idle sessions cost one shell process each; no automatic reaping in MVP.
|
||
|
||
## API (MVP)
|
||
|
||
```
|
||
POST /api/login {password} → 204 + cookie
|
||
POST /api/logout → 204
|
||
GET /api/me → {host, user, version, uptime, load}
|
||
GET /api/terminals → [{id, name, running, created}]
|
||
POST /api/terminals {name?} → {id, ...}
|
||
PATCH /api/terminals/:id {name} → 204
|
||
DELETE /api/terminals/:id → 204
|
||
WS /api/terminals/:id/attach binary frames = raw bytes; JSON text frames = {resize}
|
||
GET /healthz → 200 (unauthenticated, for NPM)
|
||
```
|
||
|
||
Everything under `/api` except `login` and `/healthz` requires the session cookie.
|
||
|
||
## Project layout
|
||
|
||
```
|
||
hamlab/
|
||
cmd/hamlab/main.go serve | passwd | version
|
||
internal/
|
||
auth/ argon2id, sessions, rate limit
|
||
term/ PTY manager, ring buffer, attach/detach
|
||
api/ chi router, handlers, WS
|
||
state/ SQLite (modernc, no cgo)
|
||
web/ go:embed of web/dist
|
||
web/ Vite + React + TS + xterm.js, PWA manifest + SW
|
||
deploy/
|
||
hamlab.service
|
||
install.sh build → /usr/local/bin/hamlab → systemctl restart
|
||
Makefile build / dev / deploy / lint / test
|
||
```
|
||
|
||
## Milestones
|
||
|
||
**M0 — skeleton** (service boots, login works)
|
||
- Go module, chi, config loading, `hamlab passwd`, argon2id login, session cookie.
|
||
- `hamlab.service` on `core`, `:8100`, serves a placeholder page.
|
||
- `make deploy` from `core` itself.
|
||
|
||
**M1 — terminals** (the MVP feature)
|
||
- PTY manager with ring buffer, create/list/rename/close, WS attach with resize.
|
||
- React shell: header, rail, footer, Terminal group, xterm main pane.
|
||
- Persistence of session list across refresh and rail switching.
|
||
|
||
**M2 — polish + PWA + ingress**
|
||
- Mobile drawer, key bar, fit-on-rotate.
|
||
- `manifest.webmanifest`, icons (Ham logo on black, same recipe as HamTV), service
|
||
worker caching the app shell only (never `/api`).
|
||
- NPM proxy host for `hamlab.bigham.dev`; verify WS through NPM; force-SSL/HSTS/HTTP2.
|
||
- Stub groups for Services / Cron / ZFS / Dashboard.
|
||
|
||
**Done when:** from the phone, open the PWA, log in, open two terminals, run
|
||
something long in one, switch away and back, refresh — it's all still there.
|
||
|
||
## Roadmap (not MVP — notes only)
|
||
|
||
- **Updates tab** — **shipped 2026-09-17** (`internal/updates`, `internal/tasks`,
|
||
`Updates.tsx`): upgradable packages with current → candidate, suite, security badge
|
||
and description; index age, last upgrade (from `/var/log/apt/history.log`),
|
||
unattended-upgrades on/off, reboot-required banner with the packages that caused it;
|
||
Refresh index, Upgrade all (dist-upgrade), Upgrade selected; changelog per package on
|
||
demand. Runs go through the new **tasks** package: background command, output kept,
|
||
followed live over `/api/tasks/{id}/stream`, cancellable, listed at `/api/tasks` (the
|
||
seed of a Task History tab). apt is fully supported; dnf/pacman list only.
|
||
- **Power** — header ⏻ button (amber when a reboot is pending), dialog that requires the
|
||
hostname typed back, then `systemctl reboot` / `poweroff`. `/api/me` carries
|
||
`reboot_needed` so the header knows.
|
||
|
||
- **tmux-backed sessions** so shells survive a daemon restart / redeploy. Would spawn
|
||
`tmux new -A -s hamlab-<id>`; needs `mouse on` and status-bar handling in xterm.
|
||
- **Services tab** — `systemctl list-units`, per-unit page: status, last 200 journal
|
||
lines (live-tail via WS), start/stop/restart/enable buttons, unit file viewer.
|
||
- **Cron tab** — **shipped 2026-09-17** (`internal/cron`, `Cron.tsx`): one list built
|
||
from `/etc/crontab`, `/etc/cron.d/*`, every user crontab (via sudo), the scripts in
|
||
the `run-parts` directories those drive, plus systemd timers. Next run computed by an
|
||
in-house cron-expression evaluator (tested: ranges, steps, names, dom-OR-dow rule,
|
||
`@` aliases); last run from the journal's `(user) CMD (command)` lines (cron logs via
|
||
syslog to journald; reading needs sudo/adm). Commented-out jobs show as disabled;
|
||
job-shaped lines inside comment blocks (Debian's header examples) are ignored. Per
|
||
job: Run now (background, as the job's user, `sh -c` with crontab `%` rules, output
|
||
captured), Enable/Disable (comments the line in place, refuses if the file changed
|
||
underneath), Edit source (textarea; validated line-by-line; user crontabs installed
|
||
with `crontab -`, system files written via `sudo tee`), Cron log, and Output file
|
||
when the command redirects to one. Timers: Run now (starts the activated service),
|
||
start/stop/enable/disable, live logs of the service via the Services tab's tail.
|
||
- **Disks tab** — **shipped 2026-09-17** (`internal/storage`, `Disks.tsx`), replacing
|
||
the planned ZFS tab: block devices from `lsblk -J` (kind, model, size, temperature
|
||
from the device's hwmon node, partitions with filesystem usage, live read/write/busy
|
||
from the metrics sampler); SMART via `smartctl -j` when smartmontools is present (one
|
||
button installs it as a task otherwise): health, temperature, power-on time, self-test
|
||
status, ATA attribute table with the failure-predicting IDs flagged, NVMe endurance and
|
||
error counters, short/long self-tests. Spinning disks are read with `-n standby` so
|
||
the tab never wakes a sleeping drive; "Wake and read" is explicit. ZFS section when
|
||
the tools and module are there: pools with health, capacity, fragmentation, last scan
|
||
and vdev tree; scrub start/pause/stop; datasets tree with used/refer/in-snapshots/
|
||
snapshot count/compressratio/mount; per-dataset snapshot list and "Snapshot now".
|
||
Verified against `grid` (raidz2 `tank`, 10 datasets, 515 snapshots, 8 WD40EZAZ + 970
|
||
EVO Plus SMART). Snapshot destroy and dataset property editing are deliberately not
|
||
there yet.
|
||
- **Dashboard** — **first cut shipped 2026-09-17** (`internal/metrics`, `Dashboard.tsx`):
|
||
CPU (total + per core), memory/swap, load, uptime, hwmon temps and fans read straight
|
||
from `/sys` (no lm-sensors), per-interface network rates, per-disk read/write/busy
|
||
from `/proc/diskstats`, filesystem usage, UPS via `sudo -n pwrstat -status`. Sampled
|
||
every 2 s into an in-memory ring of 10 min; the UI (the **Dashboard** tab
|
||
since 2026-09-19) reads it all through the sensor API — see Sensors below. **Customizable**: a
|
||
Customize mode reorders and hides modules (CPU, Memory, Load, Uptime, Temperatures,
|
||
Fans, Network, Disks, Filesystems, UPS) and hides individual rows (an interface, a
|
||
mount, a disk, a sensor); the layout is stored server-side in `state.json` via
|
||
`GET/PUT /api/prefs/dashboard` so every device sees the same dashboard. The update
|
||
interval (100 ms – 2 s) is part of the layout and drives both the browser's polling and
|
||
the daemon's sampling clock; the chart window is always 300 samples, so 30 s at 100 ms
|
||
and 10 min at 2 s. Still to do: persistent
|
||
history (SQLite ring, e.g. 10 s × 7 days), SMART (`smartctl -j`), longer time ranges.
|
||
- **Sensors** — **shipped 2026-09-19** (`internal/sensors`, `internal/metrics/sensors.go`,
|
||
`internal/api/status_sensors.go`, `web/src/sensors.ts`, `widgets.tsx`, `Embed.tsx`;
|
||
reference in `SENSORS.md`): everything measured is a sensor with a stable dotted ID;
|
||
catalog, single value, history, a **shaped query** (the client sends the object it wants
|
||
and gets it back filled in — GraphQL's idea without GraphQL, since sensors differ per
|
||
machine and a static schema would collapse to `sensor(id:)`), the same query as a live
|
||
SSE stream, Prometheus `/metrics`, read-only `sensors:read` tokens (never full tokens in
|
||
a URL), status sensors (services, updates, ZFS, SMART), a widget library, custom widgets
|
||
on the Dashboard tab (briefly renamed "Monitors" 2026-09-19, back to "Dashboard"
|
||
2026-09-20 at Mitchell's request; the view id and URL were always `dashboard`), and a
|
||
frameable `/embed/widget` for Glance and friends. (An external demo page was deployed at
|
||
`hamlab-demo.bigham.dev` and linked from the Widgets tab 2026-09-19; **both removed
|
||
2026-09-23** at Mitchell's request — the Widgets tab shows the same thing against the
|
||
machine you are signed in to, so the second copy earned nothing and cost a public
|
||
read-only token, which was revoked with it. `examples/external-demo/` stays as an example
|
||
to read.) The stock Dashboard modules and the Disks
|
||
tab read the sensor API too (`sampleAdapter.ts`); `/api/metrics` is retired.
|
||
Also per-service and per-container sensors (memory and CPU from cgroups), tokens limited
|
||
to ID patterns, and `/api/fleet/sensors/query` for the whole fleet in one request.
|
||
- **Alerts** — **shipped 2026-09-19** (`internal/api/alerts.go`, `Alerts.tsx`): rules over
|
||
sensors (`container.*.running == false for 120 s`), evaluated by the hub every 15 s for
|
||
itself and every managed machine in one shaped query each; `hamlab.reachable` is supplied
|
||
by the hub so "a machine stopped answering" is a rule like any other. Ten defaults
|
||
(machine offline, container down/unhealthy, failed service, disk full, pool unhealthy,
|
||
SMART failing, UPS on battery, CPU hot, memory). Firing and resolved are events: kept
|
||
(last 200, in `state.json`), shown on the hub-only Alerts tab with a count on the rail,
|
||
and POSTed to channels — **ntfy** (title/priority headers) or a **webhook** (JSON; Home
|
||
Assistant, Slack). The rule editor shows what a selector matches across the fleet as
|
||
you type. Not done: quiet hours, per-rule channels, Web Push straight to the PWA.
|
||
- **Docker tab** — **shipped 2026-09-19** (`internal/api/docker.go`, `Docker.tsx`):
|
||
containers grouped by compose stack with live memory and CPU from the container
|
||
sensors, start / stop / restart (asked first), logs that follow (polled with `--since`,
|
||
no held connection), inspect (env *names* only), and images / volumes / networks
|
||
tables. Works on every machine through the hub; `#/grid/docker/immich_server` links to
|
||
one. **Stacks** (`internal/api/stacks.go`): every compose project, found from its
|
||
containers' labels, with its file to edit and Deploy (`up -d`), Update (pull, then
|
||
deploy), Restart and Stop as followed tasks. The file is looked for where the label says
|
||
*and through every container's mounts*, because an orchestrator that runs compose inside
|
||
its own container (Dockhand here; Dockge, Portainer) records a path in its own
|
||
filesystem; compose is then run **inside that container**, so `.env`, relative paths
|
||
and the compose labels stay exactly as deployed (verified: labels identical after a
|
||
redeploy). Save validates with `compose config` first, keeps the previous file as
|
||
`<file>.hamlab-bak`, and never deploys by itself. A stack deployed from another host
|
||
(Dockhand on `core` driving `grid` over tcp) has no file on that machine, so the node
|
||
alone lists it read-only — but **the hub fills that in** (`hubStacks`): asked about a
|
||
node, it resolves the files on itself and runs compose here with `DOCKER_HOST` pointed
|
||
at the node's `tcp://…:2375`, the way it was deployed; the task runs on the hub and the
|
||
node's page follows it. Also per stack: every file it was deployed from (chips), its
|
||
**`.env`** (values masked until "Reveal and edit"; validated in place, put back if
|
||
compose objects), **what Deploy would change** (a line diff against the copy kept at the
|
||
last deploy from here, `<file>.hamlab-deployed`, else the copy the last save replaced;
|
||
`.env` values never shown), and **image update checks** — asked for, never polled, since
|
||
each is a registry request Docker Hub counts; remembered six hours. A save refuses a
|
||
file changed on disk meanwhile.
|
||
**Making and unmaking (2026-09-20):** "+ New stack" — a name, a folder, compose text and
|
||
an optional `.env` (written root-only), checked by compose before it is kept, then
|
||
"Create and deploy" or "Create only". "+ New container" is the same dialog opened on a
|
||
form (image, ports, volumes, environment, restart, command, host networking) that writes
|
||
the compose for you: a container made here is a one-service stack, so it stays editable
|
||
and redeployable, where a bare `docker run` could only be thrown away and typed again.
|
||
New stacks go in a **home** — a folder where two or more stacks are already deployed from
|
||
(Dockhand's `stacks/core` here; `~/stacks` on a machine with none) — and are deployed the
|
||
way their neighbours were (inside `dockhand`, or over tcp to a node from the hub). Homes
|
||
are also scanned, so a stack with no containers ("not deployed": never deployed, or taken
|
||
down) is still listed. **Down** removes a stack's containers; **Delete…** (type its name)
|
||
takes it down and moves its own files to `<home>/.hamlab-trash/<name>-<time>/` — moved,
|
||
not erased; volumes and bind-mounted data are never touched. A stopped container can be
|
||
**removed** (a running one must be stopped first, on purpose). Images: **pull** (a followed
|
||
task), **remove** (unforced: Docker refuses one still in use) and **prune** (dangling only).
|
||
**Volumes (2026-09-20):** each says which containers use it; one that nothing uses can be
|
||
deleted — a named one by typing its name (it may be the data of a stack that is only
|
||
down), an anonymous one by confirming its full name — and "Prune anonymous" clears the
|
||
nameless leftovers of recreated containers, never named volumes. Images are marked "in
|
||
use", and "Remove unused" takes every image no container uses. Nothing is ever forced:
|
||
Docker refuses what is still in use, and that refusal is the answer. Still absent:
|
||
removing networks.
|
||
- **Skins** — 13 since 2026-09-19 (`themes.ts`: a palette is one `mk(…)` line): Dark,
|
||
Light, Cyberpunk, Retro, **HUD** (hairlines of cyan on near-black, corner ticks, thin
|
||
type), Nord, Gruvbox, Catppuccin + Latte, Dracula, Tokyo Night, Solarized Dark + Light;
|
||
each repaints the terminal too. **Each machine wears a colour** (`accents.ts`): a stripe
|
||
across the header and its name in the switcher — picked from ten swatches or any colour
|
||
on the Machines page, else derived from its name — saved with the theme on the hub.
|
||
- **Multi-host** — **Phase 1 shipped 2026-09-18** (`internal/nodes`, `internal/tlsauto`,
|
||
`MachineSwitcher.tsx`, `Machines.tsx`). One binary, hub and spoke: a host whose
|
||
`config.json` lists `nodes` is a hub; any other is a standalone node with its full UI.
|
||
Node side: `"tls": "auto"` makes a self-signed cert in the data dir and serves HTTPS;
|
||
`POST /api/tokens {password,name}` trades the login password for a long-lived bearer
|
||
token (SHA-256 stored in `tokens`), accepted by the auth middleware alongside the
|
||
cookie; `DELETE /api/tokens/{name}` revokes. Hub side: `POST /api/nodes {name,url,
|
||
password}` peeks the node's certificate (trust on first use, fingerprint pinned),
|
||
obtains a token named `hub:<hostname>`, stores `{name,url,fingerprint,token}`; a
|
||
15 s prober keeps each node's status; `/api/nodes/{name}/…` is a reverse proxy to the
|
||
node's `/api/…` (bearer swapped in, cookie and Origin stripped, streaming flush) that
|
||
carries WebSockets, so terminals, log tails and task output work remotely.
|
||
Frontend: `setMachine()` rewrites every `/api/…` path; the rail has a machine
|
||
switcher at the top and a Machines page to add/remove/open. Pages remount on switch;
|
||
terminals live on the node. Nodes: `grid`, `wyse`. **Phase 3 (update from hub)
|
||
shipped 2026-09-18**: node `POST /api/self-update` takes a binary, checks it answers
|
||
`version`, keeps `hamlab.prev` in the data dir, installs over the executable via
|
||
`sudo install`, replies, then restarts the service; the hub pushes its own executable
|
||
with `POST /api/machines/{name}/update` or `/api/machines-update-all` (architecture
|
||
checked first via `/api/me.arch`), the Machines page shows an **outdated** badge and
|
||
Update buttons, and `hamlab push-nodes` does the same from the CLI — `make deploy` runs
|
||
it last, so one deploy on the hub updates every machine. First install on a new host
|
||
is still by hand (scp binary + `install.sh`, `hamlab passwd`, `tls auto`). Still to
|
||
do: Phase 2 Datacenter overview; optional firewall rule limiting a node's port to
|
||
the hub. **Cross-architecture** (2026-09-18): `make deploy` also builds
|
||
`bin/hamlab-linux-arm64`, `install.sh` installs it beside the binary as
|
||
`hamlab-arm64`, and the hub picks the sibling matching a node's `/api/me.arch` when
|
||
pushing — the Mac mini (Fedora Asahi, aarch64, node `mac`) updates like the rest.
|
||
- **Portability + install** — **shipped 2026-09-18** (`internal/setup`, `internal/priv`,
|
||
`install.sh`, `make release`): the binary installs itself (`hamlab install` /
|
||
`uninstall`; systemd or OpenRC; root, sudo or doas), all builds static on amd64, arm64,
|
||
arm, 386 and riscv64, `install.sh` as the curl-able bootstrap with checksums. Every
|
||
privileged call goes through `priv` instead of a literal `sudo -n`. Updates tab acts on
|
||
dnf, zypper, pacman and apk as well as apt. Services list falls back to the plain
|
||
table on systemd < 246; Disks falls back to `/sys/block` without `lsblk`. Listener
|
||
uses `IP_FREEBIND`. Self-update: pre-flight `check` against the live config, atomic
|
||
replace, exec in place. Config default moved to `~/.config/hamlab` (existing
|
||
`~/apps/hamlab` still found by the installer and the CLI). Tested in
|
||
containers: Debian 12, Ubuntu 22.04, Fedora 40, Rocky 8, openSUSE Leap 15.5, Alpine
|
||
3.18/3.20 (OpenRC + doas, non-root), plus the four lab hosts. Not done: an OpenRC
|
||
backend for the Services tab; runit/s6/sysvinit service files; public hosting for
|
||
`install.sh` (the Gitea repo is private, so the one-liner needs `HAMLAB_TOKEN`).
|
||
- **Users, sessions and settings that follow the account** — **shipped 2026-09-22**
|
||
(`internal/api/users.go`, `config.User`, `store.Session`/`UserState`, `Login.tsx`,
|
||
`Users.tsx` — a hub-only **Users** tab in the rail, added the same day at Mitchell's
|
||
request, holds everything about accounts: your own password, your sessions, the
|
||
accounts; Settings keeps appearance, rates, tokens and About). **Rail order is per account** too (2026-09-22,
|
||
`railorder.ts`, `Rail.tsx`): "⇅ Arrange" at the foot of the rail puts a handle on every
|
||
page; drag one up or down (pointer events, so a finger works), let go and the order is
|
||
saved in the UI prefs with the theme; "Reset" restores the built-in order; pages added
|
||
in a later build go after the ones the saved order names. The login is a user name and a password. `config.json` holds `users`
|
||
(name, argon2id hash, created, last seen); the old single `password_hash` becomes the
|
||
account named after the OS user the console runs as on first boot, its signed-in
|
||
sessions become that account's, and the machine-wide theme/accents/dashboard become
|
||
theirs — nothing to redo after the upgrade. A session carries its user, an ID, when it
|
||
was made and last seen, the IP and the browser; **Users → Sessions** lists yours
|
||
("Safari on iPhone · 10.0.0.5 · seen 2 min ago"), revokes one, or signs out everywhere
|
||
else. **Users → Users** lists accounts with their session counts, adds one, removes
|
||
one (their sessions and settings go too; never yourself, never the last), and resets
|
||
another's password (they are signed out); removing and resetting take your own
|
||
password first. Everyone signed in can do everything — a household console, not a
|
||
tenancy — and each machine keeps its own accounts: a hub reaches a node with its token,
|
||
so pairing (Machines → Add) now takes a user and password on the node, the user
|
||
optional while it has one account. Per account: `ui` (theme, accents) and `dashboard`
|
||
in `state.json` under `users.<name>`. Still the machine's: refresh rate (it is how
|
||
often every machine samples — it rides in the UI prefs object for the client but is
|
||
stored once, machine-wide), graph history, alert rules, terminals (the shells run as
|
||
one OS user anyway). A login attempt for a missing user takes as long as a wrong
|
||
password, so the form can't list accounts. Every write's log line now names the
|
||
account or token (`core@10.0.0.5 via …`), and Access tokens say who issued them. CLI:
|
||
`hamlab passwd [user]` (makes the account if new; reads stdin when piped), `hamlab
|
||
users`, `hamlab user-rm <user>`; `hamlab install` names the first account after the
|
||
service user. Not done: a read-only viewer role.
|
||
- **Space tab** — **shipped 2026-09-22** (`internal/space`, `internal/api/space.go`,
|
||
`Space.tsx`), asked for as "exactly WizTree": what is using a disk. The unit is a whole
|
||
volume, picked from a dropdown at the top: a **disk on its own** (every filesystem its
|
||
partitions back) or a **ZFS pool as a whole** (every mounted dataset) — a pool's member
|
||
disks are listed under it with the layout (raidz2, 8 disks: sda …) and never offered
|
||
apart, since one disk's share of a pool means nothing (Mitchell's call, same day, after a
|
||
first cut that listed the disks as chips). The walk runs as a helper process, `hamlab space-walk <roots>`,
|
||
as root through `priv` when the host allows (so `/var/lib/docker` and `/root` count), 32
|
||
workers reading directories depth-first, never crossing a filesystem boundary by itself;
|
||
each root is grafted into one tree at its path. Files under 128 KiB are not kept one by one
|
||
but summed per folder as "N smaller files" — what keeps a scan of millions of files to tens
|
||
of MB in the daemon; the tree crosses back as a varint stream (`codec.go`), and the helper's
|
||
memory goes with it. core's root: 646k files in ~1.2 s. Progress (files, bytes, current
|
||
folder) every 250 ms; Cancel sends SIGTERM. The page: picker → Scan/Rescan → **Folders**
|
||
(a tree largest-first: size, share of parent as a bar, files, folders, modified; expand in
|
||
place) over a **treemap** of the selected folder — squarified, nested to the file, tiles
|
||
coloured by kind (video, images, audio, archives & disk images, documents, code & config,
|
||
databases & logs, other; the validated categorical set, dark and light) with folders as
|
||
labelled boxes; hover for path, size and share; click a folder to drill, breadcrumbs and ↑
|
||
to come back. The map asks the server for the folder pruned to what a tile could show
|
||
(`min` = 1/6000 of the folder, 24 deep; deeper folders come back as one lump), the list
|
||
one level at a time. **Largest files**: the 1,000 biggest with a path filter. Works on
|
||
every machine through the hub.
|
||
**Smarter and faster (2026-09-23, at Mitchell's request):** a scan is *kept* — in memory
|
||
and in `<data>/space/<target>.scan` (the same varint stream) + `.json`, read back after a
|
||
restart — and the next scan *starts from it*: the helper is handed the last tree on stdin
|
||
and reads a directory again only when something says it changed. On a ZFS pool with
|
||
snapshots, the scan records the newest snapshot of each dataset at start (`bases`), and the
|
||
next one runs `zfs diff -H -F <base>` per dataset: only the directories it names are
|
||
read, their ancestors kept but looked into, everything else kept wholesale without a
|
||
stat. The diffs run side by side and one is given three minutes — `timeout -s KILL`
|
||
inside sudo, since Debian's sudo gives the command its own session and a process-group
|
||
kill from outside never reached zfs — after which that dataset is walked by mtime
|
||
instead. On `tank` a diff is metadata-bound like the walk: 8 s to 3 min per dataset for
|
||
a handful of changes, and `immich`, which changes constantly, ran past twenty minutes
|
||
the first time. A refresh of `tank` took 4 min 19 s against 7 min for the full scan. Elsewhere (ext4, or a dataset with no snapshot) a directory whose mtime — to the
|
||
nanosecond — is unchanged keeps its list and only its kept files are stat'ed again, since
|
||
a file can grow without its directory noticing. A **cancelled** scan keeps what it walked,
|
||
marks the folders it never got to (`partial`, "not yet read" in the tree) and **Resume**
|
||
reads just those. core: full 1.2 s → refresh 0.3 s (2 folders read, 84,755 kept). The
|
||
status line says which (`method`, `by_root`, read/kept counts); **Forget** drops the saved
|
||
result so the next scan is full. The old result stays browsable while a refresh runs.
|
||
**Treemap rethought** the same day: every file a tile grouped by folder — folders are only
|
||
a thin line and a name strip on the first level — instead of nested boxes with headers,
|
||
which on Linux's single-child folder chains was a staircase of headers around nothing.
|
||
Not done: delete from here, a size-over-time history, per-user column choices.
|
||
- **Updater** — **shipped 2026-09-23** (`internal/selfupdate`, `internal/api/update.go`,
|
||
the Updates card in `Settings.tsx`, `scripts/gitea-release.sh`). A Ham Lab asks one URL
|
||
what the newest build is and installs it on request. `update.url` in `config.json` is
|
||
either a Gitea/Forgejo repo API base (`…/api/v1/repos/<owner>/<repo>`, whose newest
|
||
release names the version and carries the binaries as assets) or any static directory
|
||
holding `VERSION`, `SHA256SUMS` and `hamlab-linux-<arch>` — which is exactly what
|
||
`make release` writes and what `install.sh` already downloads, so the demo site, a
|
||
folder on a NAS and a Gitea release are all the same thing to it. `"off"` never looks.
|
||
`update.token` is sent when the source is behind a login, so a **private** Gitea serves
|
||
its own fleet without the instance being opened to the world. The binary is verified
|
||
against `SHA256SUMS` before it goes anywhere near the running one, then takes the same
|
||
path `POST /api/self-update` always did (runs `version`, runs `check` against this
|
||
machine's config, atomic replace keeping `hamlab.prev`, re-exec in place) — that path is
|
||
now one shared method, `installBinary`. Version ordering is `install.sh`'s `vnum()` rule
|
||
ported to Go and tested against it, so the daemon and the installer never disagree about
|
||
which of two builds is newer; the commit count since the tag counts, the hash does not.
|
||
Checked ~90 s after boot and daily (`update.check: "never"` switches that off), never
|
||
installed unasked — an update restarts the daemon and drops open terminals, and the
|
||
dialog says so. Settings → **Updates**: installed, published, when it last looked, the
|
||
source, the release notes, Check for updates, Install; the rail badges Settings when a
|
||
newer build is out (`update_available` rides on `/api/me`). Publishing is
|
||
`make gitea-release` → `scripts/gitea-release.sh`, which creates the release and uploads
|
||
the five architectures, `SHA256SUMS`, `install.sh` and `VERSION`; it is deliberately
|
||
**not** part of `make deploy`, because deploy pushes this build to the machines this hub
|
||
owns while publishing offers it to every install anywhere, and one tag per deploy would
|
||
be noise. Re-publishing a version needs `--force`.
|
||
**Public since 2026-09-23**, at Mitchell's instruction and after the trade was put to him
|
||
twice: Gitea's instance-wide `REQUIRE_SIGNIN_VIEW` is off and `mitchell/hamlab` is marked
|
||
public, so anyone can browse it, clone it and download a release with no account. The
|
||
other five repos there are each marked private and are invisible — an anonymous request
|
||
for one is a 404, not a login prompt — registration is still disabled, and new repos
|
||
still default to private. No token is needed by anything any more (the read-only one was
|
||
revoked; only `hamlab-publish`, which creates releases, remains). Gitea resolves
|
||
`releases/download/latest/<asset>`, so `install.sh` and `HAMLAB_URL` now point straight
|
||
at the newest release and never need changing. What is public is the source, the history,
|
||
`BUGS.md` and this file — which describes the lab and names the console's hostname.
|
||
- **Auth hardening** — WebAuthn passkeys (phone Face ID), TOTP fallback, optional NPM
|
||
access list restricting to WireGuard/LAN.
|
||
- **Files tab** — **shipped 2026-09-18** (`internal/files`, `Files.tsx`): browse from
|
||
`/` with breadcrumbs, an editable path box, sort, hidden-file toggle and a filter;
|
||
rows show type icon, size, modified, perms and owner, link targets and broken links.
|
||
Text files (≤ 2 MiB, sniffed) open in an editor with Ctrl+S / Save; images preview;
|
||
everything else downloads. Upload via drag-and-drop or picker with per-file progress
|
||
(streamed, no size limit). New folder / file, rename, move, chmod, copy path, delete
|
||
(folders need the name typed back), and "open in terminal", which `cd`s the active
|
||
shell (or a fresh one) to the folder. **Sudo toggle**: root access re-executes the
|
||
same binary as `hamlab fs <op>` under `sudo -n`, streaming bodies over stdin/stdout,
|
||
so both modes share one implementation and one set of rules (absolute paths only;
|
||
`/`, `/etc`, `/usr`, … are never deleted or moved whole). Every write is logged with
|
||
the caller's IP.
|
||
- **Docker read-only view** — `docker ps`/logs; dockhand stays the source of truth.
|
||
- **Notifications** — push on service failure / SMART warning / UPS event.
|
||
- **Terminal extras** — search, copy-mode, font/theme, split panes, upload via
|
||
drag-drop into cwd.
|
||
|
||
- **Settings + themes** — **shipped 2026-09-18** (`Settings.tsx`, `themes.ts`): Settings
|
||
page with Appearance (Dark, Light, Cyberpunk, Retro — CSS variables on `<html>` plus a
|
||
matching xterm palette, applied before first paint from localStorage and synced hub-side
|
||
via `GET/PUT /api/prefs/ui` so the choice follows the login), Account (change password,
|
||
`POST /api/password`, current password required), Access tokens (list/revoke the tokens
|
||
hubs hold), About (version, host, arch, uptime, certificate).
|
||
|
||
## To do
|
||
|
||
- **Deprecate copyparty** — asked for 2026-09-23, not started. `copyparty` on `grid`
|
||
(`copyparty.service`, `:3923`, `/etc/copyparty.conf`, serving all of `/tank` as user
|
||
`core` with one account behind NPM) overlaps the **Files** tab almost entirely. The aim
|
||
is to stop the service and keep only Ham Lab. What Files must gain first, in the order
|
||
that matters:
|
||
- **Multi-select and bulk actions.** copyparty selects many rows and moves, deletes or
|
||
downloads them together; Files acts on one row at a time.
|
||
- **Download a folder**, and a multi-file download, as a streamed zip or tar — the thing
|
||
copyparty is reached for most.
|
||
- **Resumable uploads.** copyparty's up2k chunks a file, survives a dropped connection
|
||
and skips what is already there; Files streams one PUT, so a phone upload that drops at
|
||
90% starts over.
|
||
- **A gallery.** Thumbnails for a folder of images and video, and play audio/video in
|
||
place — `/tank/media` and `/tank/immich` are most of what is browsed.
|
||
- **Search** below the current folder by name, since `find` in a terminal is the fallback
|
||
today.
|
||
- *Decide when built:* whether share links (copyparty's public per-path links) are wanted
|
||
at all — they are the one feature that hands out access without a login, and Ham Lab has
|
||
no notion of that; if not, say so in the docs rather than leaving a gap unexplained.
|
||
- *After:* `systemctl disable --now copyparty`, remove its NPM host, keep
|
||
`/etc/copyparty.conf` until a rollback is clearly not needed, then update
|
||
`hamlet`'s `docs/hosts.md` bare-metal list, `docs/gotchas.md` (the "`docker ps` is not
|
||
the inventory" note names it) and the `grid` row in `CLAUDE.md`.
|
||
|
||
- **Deprecate Dockhand** — asked for 2026-09-23, not started; the *homelab* half of this
|
||
(migrating every stack off Dockhand's data dir, then removing the container) is
|
||
`hamlet`'s `docs/pending.md` → "Retire Dockhand; Ham Lab becomes the orchestrator",
|
||
agreed 2026-09-20. This entry is the *app* half only: what Ham Lab must do before that
|
||
migration can start. Its Stacks page already creates, edits, deploys, updates, restarts,
|
||
stops, downs and deletes stacks on every machine, so the gaps are Dockhand's extras:
|
||
- **Git-sourced stacks.** Dockhand's `stack_sources` table points a stack at a repo and
|
||
pulls it. Ham Lab has no equivalent. Decide whether to build it or to drop it — most
|
||
stacks here have no row in that table anyway, so dropping it may be honest.
|
||
- **Scheduled image updates.** Ham Lab checks for new images only when asked (a registry
|
||
request Docker Hub counts) and never updates by itself. If anything relies on Dockhand
|
||
updating on a schedule, that must move here or be dropped on purpose.
|
||
- **Webhooks / per-host environments**, if either is in use — inventory before deciding.
|
||
- **Deploying from a host path without an orchestrator container.** Ham Lab already runs
|
||
compose inside `dockhand` because that is how the stacks were deployed; once they move
|
||
to a neutral home the "via" path is unused for them, but a stack deployed to `grid` or
|
||
the Mac still goes over `tcp://<node>:2375` from the hub. Decide whether Ham Lab should
|
||
instead ask the node's own Ham Lab to deploy locally, which would end the `*:2375`
|
||
exposure (its own item in `docs/pending.md`).
|
||
- *Not app work, but the trigger:* nothing here blocks starting the migration except the
|
||
inventory. Do that first.
|
||
|
||
## Open questions
|
||
|
||
- Idle-session reaping policy (never / after N days) — leaning never; it's a
|
||
single-user box.
|
||
- Whether `sudo -i`-style root sessions get their own leaf colour. Cosmetic.
|