# Bug bash — 2026-09-18

Fix pass 2026-09-18: every item carries its outcome; ✅ resolved and verified live, ➖ no change with reason. Found by a headless Chromium pass (iPhone and
1440px desktop viewports) over every page on the hub and on each remote machine, an
offline-node scenario (wyse stopped for the test, then restarted), direct API probing
with hostile inputs, and a code read. Ordered by severity within each group.

## Security / robustness

1. ✅ **RESOLVED 2026-09-18** — only regular files are readable; the byte cap applies to what is sent, not to `stat`. `/dev/zero`, `/dev/null` and FIFOs answer 400 in 30 ms; `/proc/cpuinfo` still reads. **Reading a special file hangs the request forever.** `GET /api/files/read?path=/dev/zero`
   never ends: the `max` cap compares against `stat` size, which is 0 for device files,
   so the cap never triggers and the handler streams `/dev/zero` until the client gives
   up. Same for FIFOs (blocks on open) and `/proc/self/mem`. Opening one in the Files
   viewer would hang the tab. Needs: refuse non-regular files for read/edit, and cap the
   copy with `io.LimitReader` regardless of `stat`.
2. ✅ **RESOLVED** — per-client buckets (first X-Forwarded-For hop or the peer) with the same 5-strike schedule, plus a global backstop of 100 failures per 5 min → 1 min. Verified: core's public IP locked while grid logged in fine. **Login lockout is global, not per client.** Five bad passwords from anyone on the
   internet lock the real user out for up to 10 minutes (`auth.Limiter` is one bucket).
   The bash locked the hub for the full 10 minutes twice. Also locks `POST /api/tokens`,
   so hub pairing fails during a lockout. Needs per-IP buckets with a modest global cap.
3. ✅ **RESOLVED** — server rejects refreshMs outside 100–60000 and non-identifier themes (400); client ignores out-of-range rates from any source. **UI prefs from the server are not validated.** A stored `refreshMs` of `-5` made the
   client call `setInterval(-5)` on every live view: a request storm that took the
   browser to `ERR_INSUFFICIENT_RESOURCES` and hammered the daemon. `theme` is validated
   only by falling back. Needs range checks on load (and on save server-side).
4. ✅ **RESOLVED** — every response carries X-Frame-Options DENY, CSP frame-ancestors none, nosniff, no-referrer. **No anti-framing / CSP headers.** Only HSTS (from NPM). A root-capable console should
   send `X-Frame-Options: DENY`, `Content-Security-Policy: frame-ancestors 'none'`,
   `Referrer-Policy: no-referrer`, `X-Content-Type-Options: nosniff` on every response.
5. ✅ **RESOLVED** — kernel threads (empty cmdline) and the daemon's own PID are refused. **Signals can be sent to kernel threads.** `POST /api/procs/2/signal` (kthreadd)
   returns 204. Harmless in practice, but the guard should refuse PIDs with an empty
   cmdline (kernel threads) and our own PID.
6. ✅ **RESOLVED** — unit runs with `SupplementaryGroups=systemd-journal adm` so journal reads need no sudo, and the installer drops `/etc/sudoers.d/hamlab-quiet` (no syslog/pam session for the read-only polling commands). Verified: 0 sudo lines in 60 s of 250 ms polling (was 839/10 min). Applied on all four hosts. **Journal spam from the daemon's own sudo calls.** Every sudo'd `journalctl`, `smartctl`
   or `pwrstat` writes three `pam_unix(sudo:session)` lines into the journal *under
   `hamlab.service`*: 839 lines in 10 minutes at a 250 ms refresh, and the journal is
   already 4 GB. It also buries Ham Lab's own log entries. Fix options: add the service
   user to `systemd-journal` (and `adm`) so journal reads don't need sudo, and/or a
   sudoers `Defaults!/usr/bin/journalctl !syslog, !pam_session` line in the installer.
7. ✅ **RESOLVED** — server answers 409 unless `overwrite=1`; the UI asks "Replace it?" first. **Uploading over an existing file silently overwrites it.** No confirmation, no
   "replace?" prompt. Same for "+ file" when the name exists (server refuses; UI just
   shows the error, fine) — upload is the gap.
8. ➖ **NO CHANGE** — Go's mux redirects the unclean path to `/etc/passwd`, which is an ordinary deep link the SPA must serve; there is no API prefix left by the time the static handler sees it. Harmless. **`/api/<anything>/../../x` falls through to the SPA** and returns `index.html` with
   200 instead of a JSON 404. Harmless, but sloppy for an API prefix.

## Functional bugs

9. ✅ **RESOLVED** — 401 carries the server message; the form shows "wrong password" (verified headless). **Wrong password gives no feedback at all.** The 401 is thrown as `Unauthenticated`
   with an empty message, and the login form only renders `err` when non-empty. The
   button just re-enables. Reproduced on phone and desktop.
10. ✅ **RESOLVED** — hub-only pages bounce to the dashboard on switch (verified headless). **Switching machines while on Settings or Machines keeps showing the hub's page.**
    `switchMachine` only resets the view for Machines; Settings stays, so "mac" shows
    core's account/token/about panels. Both are hub-only pages and should bounce to the
    dashboard (or be clearly labelled as hub pages).
11. ✅ **RESOLVED** — escaped; no console error. **Machines add-form `pattern` is an invalid regex in Chromium.** `[a-z0-9][a-z0-9-]*`
    fails under the `v` flag ("Invalid character class"), logs a console error and the
    browser skips validation. Needs `[a-z0-9][a-z0-9\-]*`.
12. ✅ **RESOLVED** — refresh from the newest repodata under libdnf5/dnf caches (Mac: 9 d), last upgrade from `dnf history` (194 d), label reads "from dnf history". **Updates tab on dnf hosts shows nonsense.** Mac: "Index refreshed 561d ago" (mtime of
    `/var/cache/dnf`, which Fedora 41+ no longer uses; `libdnf5` is the live cache) and
    "Last upgrade never · from apt history" (label and source are apt-specific). Needs
    `dnf history` for last upgrade, metadata timestamps for refresh, and neutral labels.
13. ✅ **RESOLVED** — journal read incrementally by cursor, timers from one `systemctl show`, services' two calls run in parallel. Cron 3.6 s → 0.08 s, services 0.6 s → 0.05 s, with last-run and next-run data intact. **Cron overview is slow: 3.6 s per request on core.** At any refresh rate the page
    is constantly busy. The 30-day, 20 000-line journal scan for "last run" is rebuilt
    every 5 s; timers make three `systemctl` calls. Needs an incremental journal cursor
    and a single `systemctl` call. Services list is 0.6 s for the same reason (two calls).
14. ✅ **RESOLVED** — heap profile showed a 48 MB `bytes.Buffer` from buffering `journalctl` dumps; those are now streamed line by line, plus a 160 MB soft memory limit for the GC. RSS 22 MB fresh, 27 MB after 25× cron/logs/procs/services (was 180 MB). pprof stays available behind the login at `/api/debug/pprof/`. **Daemon memory grows.** RSS on core went 22 → 85 → 216 MB during the day (136 MB at
    rest after the bash). Suspects: per-pid caches in the process table (`procInfo` only
    prunes on the next walk), task output buffers, SMART cache, xterm scrollback rings
    that are never freed for exited sessions. Needs a heap profile under load.
15. ✅ **RESOLVED** — `--dirty` dropped from the version and deploys now follow commits; binaries report e.g. `v0.1.0-38-gc1805f6`. **Version string is always `-dirty`.** `make deploy` builds before the commit, so
    every installed binary reports e.g. `v0.1.0-32-ga372378-dirty`, and the Machines page
    compares with `-dirty` stripped. Build after commit, or tag releases.
16. ✅ **RESOLVED** — hash routing `#/<machine|hub>/<view>`: reload restores machine and tab, Back/Forward work, links can point at a page on a machine (verified headless). **View resets to Dashboard on reload; no deep links; browser Back leaves the app.**
    The selected machine survives a reload (sessionStorage) but the tab does not, and
    there is no URL routing at all. Needs hash or path routing (`/#/grid/services`).
17. ✅ **RESOLVED** — newest first. **Cron log tab lists oldest first and starts at the top.** 200 lines from Sep 10 with
    the newest at the bottom, off-screen. Should show newest first or scroll to the end.
18. ✅ **RESOLVED** — "Nothing matches." **Processes: empty filter result shows nothing.** No "nothing matches" message.
19. ✅ **RESOLVED** — exited shells remove themselves 10 minutes after exit (verified live: seeded 17:53:56, gone at 18:03:59 with no daemon restart in between). Found and fixed alongside: a daemon restart used to re-spawn *exited* shells as fresh running ones; only running shells are persisted now. Idle running shells are deliberately kept. **Terminals accumulate.** Nothing ever reaps exited or idle sessions; the bash left
    several "bug-bash-term" shells on the hub until deleted by hand. Exited sessions
    should auto-remove after a while; idle ones at least be counted somewhere.
20. ✅ **RESOLVED** — a node answers 409 "busy" when a package task is running or a terminal is attached; the Machines page offers Force, `hamlab push-nodes --force` / `make push-nodes-force` do the same. Verified: 409 with a terminal attached on grid, 200 with force. **Self-update restarts without warning running work.** A push to a node kills its
    open terminals and any running task (upgrade in progress). The hub should refuse or
    warn when the node reports a running task or attached terminals.
21. ✅ **RESOLVED** — PathError and LinkError are unwrapped to the bare reason. **Error text leaks raw Go errors.** e.g. rename shows "rename /tmp/x /tmp/y: no such
    file or directory". Fine for a lab, ugly for a product; strip the syscall prefix.

## Mobile / UX

22. ✅ **RESOLVED** — on coarse pointers every button, chip, icon button, link, header and select is at least 36 px tall (verified: no control under 32 px on the dashboard or processes pages). **Tap targets under 28 px on phone:** header ⏻ (25), logout (22), customize ▲ ▼ hide
    (23), terminal leaf ✎ ✕ (24), chips (25), table headers (27). Apple's guideline is
    44 pt. Systemic: the small `.btn`/`.chip`/`.icon-btn` sizes need a touch variant.
23. ✅ **RESOLVED** — the dialog asks for the machine name (`mac`); the node still checks its own hostname under the hood. **Power dialog on the Mac asks you to type `5ce91ee54af6`.** The confirmation uses
    the node's hostname, which on the Mac is the unset default. Should offer the machine
    name (`mac`) as the confirmation word on remote machines.
24. ✅ **RESOLVED** — down interfaces collapse to one line marked "down"; modules with nothing to show (fans, UPS, temps, disks) hide unless customizing. **Dashboard shows dead interfaces and empty modules.** wlp1s0 (down) gets a full
    sparkline block of flat lines; "Fans: no fan sensors" is a permanent empty card on
    hosts without fans. Down interfaces should collapse to one line; empty modules should
    auto-hide unless customizing.
25. ✅ **RESOLVED** — "load 1.39 1.56 1.55" with a tooltip. **Header load average is unlabelled** ("1.39 1.56 1.55" floating in the header).
26. ➖ **NOT FIXING** — the list is intentionally a plain table; the menu anchors inside the row. Cosmetic taste, not a defect. **Files list on desktop is a narrow strip in a sea of empty space** for small
    folders; the row menu opens off the right edge of the row. Cosmetic.
27. ✅ **RESOLVED** — after a 502 the client leaves that machine alone for 5 s: 2 requests in 10 s at a 250 ms refresh (would have been ~80), badge and message intact. **Offline machine keeps polling every view at the refresh rate**, producing a 502
    per request per page (and console errors). The UI copes — badge, message — but the
    hub should back off to the prober's cadence while a node is down.
28. ✅ **RESOLVED** with 6. **Services logs are noise.** The default view of `hamlab.service` logs is the sudo
    session spam from bug 6, three lines per poll.

## Confirmed working under abuse (for the record)

Path traversal in files, services, cron, ZFS, procs and updates all rejected. `/etc`
and `/` protected from delete/rename as root. Unit, pool, dataset, snapshot and
package names validated. Token issue refused with a wrong password. Oversized prefs
rejected. Duplicate machine add refused. Unicode + spaces in file names round-trip.
Upload byte-exact. Terminal scrollback survives switching. Offline node: switcher and
header show it, pages show the reason. Remote terminals, logs, tasks and files through
the proxy all work on all three nodes.


# Bug bash — 2026-09-19

Found by driving a scratch instance (same binary, its own state, real fleet behind it) in
headless Chromium at 1440 px and iPhone size: a sweep of every page on every machine for
page errors, failed requests, overflow and small tap targets; then scripted use of every
feature — terminal tabs, Files in a throwaway folder, Processes (killing a process the
test spawned), Monitors customisation and widgets, Services, Cron, Updates, Disks,
Machines, Settings, embeds, machine switching, session loss — and a hostile pass over the
API. Host-changing actions (service control, cron edits, upgrades, power) were opened but
not executed. Every fix below was re-checked by the same script that found it.

## Bugs — all ✅ fixed and verified

1. ✅ **One widget with a bad pattern silenced every custom widget.** They share one stream;
   `fan.[` made the whole query a 400, so all of them showed "…" forever with no reason
   given. The picker now asks the API before adding and shows its answer; a bad pattern
   already saved shows a notice instead of dots.
2. ✅ **A saved layout of the wrong shape could not be loaded** ("could not load layout",
   and the page could no longer be customised). `normalize()` now accepts anything.
3. ✅ **A lost session left pages saying "unauthenticated"** for up to 30 s. A 401 from the hub
   now returns to the login form at once, and logging in returns to the same page. (A 401
   relayed from a managed machine is that machine rejecting the hub, and doesn't.)
4. ✅ **The machine-name rule in "Add a machine" was an invalid regular expression** under the
   browser's `v` mode, so it validated nothing and logged a console error on every render.
5. ✅ **Two terminals opened in the same minute got the same name** ("shell 09:45" twice).
   Now "shell 09:45 (2)".
6. ✅ **Names had no length limit** — a 328-character terminal tab, a 5000-character token
   name. 64 characters, refused with a reason; token names also refuse `/ \ < > "`.
7. ✅ **"+ file", "+ folder" and Rename accepted `../x` and `a/b`**, quietly acting outside the
   folder on screen. A name is now one path segment; Move… is how things go elsewhere.
8. ✅ **Unsaved file edits were thrown away without a word** when you picked another page or
   machine, or closed the tab. All three now ask; Cancel keeps you where you were.
9. ✅ **A mistyped path in Files was remembered**, so a reload landed on the error. Only a
   folder that opened is remembered.
10. ✅ **Escape closed nothing**: the machine menu, the file menu, the Power dialog, the widget
    picker. It closes all four.
11. ✅ **Reset on Monitors removed every custom widget with no confirmation.** It asks, and
    says how many.
12. ✅ **A few bytes of query bought 27 MB of answer** (`{"k0":"*",…×400}` with `full=1`) — and a
    read-only token is public on the demo page. A query may return 10 000 values (thirty
    whole machines); past that it is a 400 that says so.
13. ✅ **No ceiling on live streams**: 300 from one token were accepted. 32 per client address,
    512 in all, then a 429 that says so; closed streams free their places.
14. ✅ **Anything under `/embed/` lost the anti-framing headers**, not just the widget page.
    Only `/embed/widget` is frameable.
15. ✅ **The new ZFS sensors ran three `sudo` commands a minute on every machine** — including
    `core`, which has the tools and no pool — and each wrote three lines to the journal:
    most of `hamlab.service`'s log was that. Pools are now read with one unprivileged
    `zpool list`; a source that finds nothing is asked again ten times less often; and
    the installer's quiet list covers `zpool|zfs list|status`.
16. ✅ **Phone: the filter boxes on Processes and Files, the path box and the "/" crumb were
    under 30 px.** 36 px like every other control.
17. ✅ **Disks → datasets on a phone: rows ended 50 px before their own cells** when scrolled
    sideways (`min-width` was less than the columns add up to).

Also: tab titles now say where you are ("Services · grid — Ham Lab"), so history and open
tabs can be told apart.

## Checked and fine

Terminal focus after every switch/close/return; scrollback after reload; exited shells;
30 tabs with the + still reachable; re-fit on resize; Back/Forward; unknown machine or page
in the URL; wrong password; XSS through terminal, token and file names (all rendered as
text); hidden files, filter, symlinks (broken ones marked), binary files, upload and
replace, folder delete needing the typed name, root toggle; process sort, filter and
Terminate (asks first); themes; password-change validation; duplicate or unreachable
machine; hub terminals not leaking into another machine's page; CORS only on sensor paths;
token list never exposing hashes; cookie flags; deep nesting, empty bodies, odd methods
and `every=` values on the sensor API; phone drawer, key bar and picker.

## Quality of life — all ✅ done 2026-09-19, each verified in the browser

1. ✅ **Selections are in the URL.** `#/grid/services/ssh.service`, `#/hub/cron/job:…`,
   `#/hub/files/%2Fetc`: reload keeps them, links carry them, leaving a page drops them.
2. ✅ **Browser Back with unsaved file edits asks**, like page and machine changes do.
3. ✅ **No more browser `prompt()`/`confirm()` boxes** — all 30 are in-app dialogs (`askYes`,
   `askText` in `ui.tsx`): Enter submits, Escape or a tap outside cancels, destructive
   ones are red and say what they do ("Close", "Delete"), and they fit a phone.
4. ✅ **Token creation is one dialog**: name, patterns, and the machine's own sensor families
   as chips to click.
5. ✅ **Widgets can be edited** — sensor, type and title — in place, from Customize.
6. ✅ **The Widgets gallery has a search** over IDs, names and labels (so "gitea" finds its
   container's sensors).
7. ✅ **Status sensors keep a history** (300 readings each: an hour of container memory, two
   days of drive wear), so their sparklines have a past.
8. ✅ **A stream that fails says why**: "the token was rejected", "too many live streams are
   open", "that machine is unreachable", "can't reach Ham Lab" — in embeds and on Monitors.
9. ✅ **Pattern widgets get readable default titles**: `cpu.core.*.usage` → "CPU core usage".
10. ✅ **Terminal tabs reorder** — drag on a desktop, ‹ › on a touch screen, remembered per
    machine in the browser — and **Alt+Shift+←/→** switches tabs, **Alt+Shift+T** opens one,
    with the keyboard left in the terminal each time.
11. ➖ **"Log out everywhere"** — built, then removed the same day at Mitchell's word: no need
    for it on a single-user console.
12. ✅ **Real clipboard shortcuts in the terminal**, from a physical keyboard: **Ctrl+C copies
    when text is selected** (and is still ^C when nothing is — the selection clears after a
    copy, so a second Ctrl+C interrupts), **Ctrl+V pastes**, Ctrl+Shift+C / Ctrl+Shift+V do
    the same unconditionally, Ctrl+Insert / Shift+Insert too. It rides the browser's own
    copy and paste events, which xterm already answers, so it works over plain http on the
    LAN where `navigator.clipboard` does not exist; bracketed paste is honoured.