nvidia-smi on a busy card can take past the 3 s deadline, which latched the card
as stalled until restart. A late read now carries the last readings and is used
when it lands; no second read starts while one is out; only one out for 15 s
(past both nvidia-smi timeouts) marks the card stalled, and its return clears it.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Its sensors were gpu.<name>.usage_2 and so on; the page looks under gpu.<name>_2. Each
card now gets its own stem, as the page already assumed.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The sensor feed dropped which socket a coretemp reading came from, so every core matched
the first CPU's chip. coretemp sensors now carry a package label, and history folding
keeps the two sockets' same-named cores apart.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Per-package RAPL watts in the sample, the process table and as cpu.package.N.power
sensors. The socket rows collapse with an arrow; the fold is remembered per browser.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Hardware counts CPUs, cores and threads per CPU. Top groups threads under a row
per socket, reads each core's temperature from its own socket's coretemp chip,
and the CPU temperature is the hotter package.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A txg records its bytes only when it syncs and the mover reports every 2 s, so a per-sample delta
read 0 most of the time. Rates are now taken over the gap between syncs (or mover updates) and held
until the next one.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Live from arcstats (dirty), dmu_tx (throttled writes), each pool's txgs (bytes synced to disk)
and objset kstats (bytes from programs). A landing tier — a mergerfs pool whose first branch is
fast storage and whose last is a ZFS dataset — is detected from mergerfs's branch xattr; a mover
status file in /run/landing-mover/ adds its queue, the file in flight and totals. Sensors
zfs.write.*, zfs.<pool>.write_bps|logical_bps and zfs.<pool>.landing.* carry the history.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Graphs stepped once a history point (1 s at a 5 min window) while the numbers changed every
tick; they now slide a fraction of a column each tick. The process table (and the GPU boxes,
MHz and watts it carries) was polled on its own timer; it is now read per sample and committed
with it. GPU clients are walked every sample, and the 1–2 s floors on the gallery, Network,
Docker and ZFS ARC live views are gone.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The proprietary driver's DRM card says DRIVER=nvidia, which the driver list did not
know, so the card was never enumerated; and it exposes no sysfs busy counter or
fdinfo engine totals. Read it with nvidia-smi instead (--query-gpu per sample,
pmon -c 1 -s um for clients), by PCI address. Parsers unit-tested, including an
older pmon column layout and [N/A] fields.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
The viewer tries the file as it is first. When the browser gives up — Safari and an MKV,
HEVC without a hardware decoder — or on a tap, a picker: codec (H.264, HEVC, AV1, or the
original codec repackaged) and size (original, 1080p, 720p, 480p), each codec labelled
with the engine this machine would use. POST /api/files/play runs ffmpeg into an HLS
event playlist of fragmented-MP4 segments under <data dir>/transcode/<id>/; Safari plays
HLS itself and everything else goes through hls.js, loaded only then. The recipes are
tried best first: a card on the bus before the integrated GPU, decoding on the same card
before decoding on the CPU and uploading, the software encoders last; a recipe that
produces no first segment is dropped for the next. The seekable range grows as ffmpeg
gets ahead; status says how far. Streams are reused for the same request, closed when
the player goes, swept after ten idle minutes, and the folder is emptied at start. Root
mode runs ffmpeg as root through a helper that ends when the daemon lets go of it.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
A previews button on the Files tab turns the icon column into a 256 px thumbnail for
every image and video row, loaded as rows scroll near. internal/thumbs probes what the
machine can do rather than assuming it: ffmpeg and ffprobe, every /dev/dri/renderD*
opened and exercised, vainfo's decode profiles per node where it exists. A video goes to
the first GPU that lists its codec, with ffmpeg's own software fallback behind it and a
CPU retry behind that; images decode on the CPU, JPEGs at a reduced DCT size. Previews
are kept under <data dir>/thumbs/ keyed by path, size and mtime, with undecodable files
remembered too. Root mode makes them through the same root helper the file reads use.
Where ffmpeg is missing the tab offers to install it; where a render node exists but is
closed to the daemon's user, it offers a drop-in adding the group and a restart.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
New sensor cpu.power: watts from the RAPL energy counters (Intel, and AMD
Zen through the same driver), summed over every package and averaged into
history. Parts of a package, psys and the MMIO duplicate are left out.
The counters are root-only, so a daemon that isn't root reads them through
`hamlab cpu-energy`, a root helper started once with sudo or doas and asked
over a pipe each sample, the same shape as the GPU client walk's helper.
It reads only the counters it finds itself. No counters or no root means
no reading, and the CPU box shows nothing extra.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The router has no lspci, so its GeminiLake chip came up as 'Intel GPU
3185' beside the identical chip's 'Intel UHD Graphics 600' on hamtv.
A trimmed pci.ids — the graphics devices of Intel, AMD and NVIDIA, and the
virtual GPUs, 115 KB — rides in the binary and is consulted first; lspci
is the fallback for a card newer than the table. scripts/pci-ids-gpu.sh
regenerates it from the hub's copy.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
On grid the sampler sat 42 minutes in runtime_pollWait on the Arc A380's
hwmon fan1_input: kernfs accepts epoll, so os.ReadFile made the fd
non-blocking, i915 answered EAGAIN once, and the poller waited for a
readiness kernfs only signals on sysfs_notify. Every graph on the machine
froze while the API carried on.
readStr now reads with plain blocking syscalls. Each GPU's sysfs reads run
under a three-second deadline; a device that does not answer is marked
stalled, logged, and left alone until restart, with the box saying so.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
A Cache section between the pools and the schedules. One card for the ARC:
size against its ceiling with the target and floor, sparklines of size and
hit rate from the sampler's history, what it holds (MRU/MFU/dirty, data vs
metadata, compression, the ghost lists), the prefetcher and dbuf cache, the
memory-pressure counters, and the since-boot breakdown by demand/prefetch
and data/metadata. Then a card per pool for L2ARC, SLOG, special vdev,
dedup table and spares — the devices and their fill where they exist, and
where they don't, plainly none and what that means (the ZIL writing to the
pool's own vdevs, with how much it has written).
Live numbers come from the sampler, which reads arcstats every tick and
publishes zfs.arc.* sensors; the fuller picture is read on request, with
zpool list -Hpv and zfs get run without sudo and cached for thirty seconds.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Every DRM device with a real GPU driver gets a box under the cpu box, laid out
like it: the busy curve, a meter for the device and one per engine, temperature,
clock, power and fan where the card has them, and the processes holding it open
with their share. The process table gains a Gpu% column and sort.
Device busy is Intel's RC6 idle residency inverted (what intel_gpu_top shows,
and readable without root) or amdgpu's gpu_busy_percent. Per-process shares are
DRM fdinfo deltas, as nvtop reads them; other users' fdinfo needs root, so a
daemon running as a user keeps one root child (hamlab drm-clients) over a pipe
rather than paying a sudo a second. Everything is also a gpu.* sensor.
What cannot be measured is left out, not shown as zero: an Arc A380 on i915 6.12
has no temperature sensor, an integrated GPU wears the package temperature
marked *, and Asahi has neither a busy counter nor fdinfo yet.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Disks: one row per physical drive with its model, size, temperature and IO, and a usage
bar built from the filesystems the sampler can attribute to it. Attribution is new — a
mount's source device is resolved through sysfs to the drive behind it, following one
layer of device-mapper or md and folding NVMe namespaces the way the sampler does. A pool
member gets no usage bar, because one drive's share of a raidz pool is not a real number.
Net: every interface that is up is listed and tappable, defaulting to the one that has
actually carried traffic rather than whichever the kernel listed first.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
core mounts grid's tank over CIFS, and it was listed beside root and efi as though it
were a disk in the box. Filesystems whose storage is on another machine are now marked
remote — a label on the fs sensors, so the sensor stays for anyone who wants it — and the
Processes tab's disks box leaves them out.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The process table carries each logical CPU's core id from sysfs topology; a row with
no reading still has the temperature cells, so the % column is one column.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
An Apple SSD presents three namespaces and the kernel lists each as a whole disk, so the
dashboard was showing one drive as three. The Disks page has always folded them; now the
dashboard, the Top tab and the disk.* sensors do too.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The process table's columns fit the box they are in, the rate control saves, curves are
timed on the viewer's clock and keep to the stride, the power icon is drawn, and a young
history draws in from the right instead of being stretched.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Each machine keeps 300 points per curve whatever the window, so memory doesn't follow it;
a wider window means wider points, each the average of its bucket rather than one instant
out of it. The hub passes the setting to every machine it manages.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
- service.<unit>.active|state for services that are enabled, up or failed
- containers.* and container.<name>.running|state|health|mem|cpu (cgroup reads, no docker stats)
- sensors:read tokens can be limited to ID patterns, enforced on every endpoint and,
through the hub's filter header, on managed machines; the header can only narrow
- /api/fleet/sensors/query: one shaped query, every machine, one request
- the sampler waits what's left of the interval, so 250 ms means 4 a second
- Monitors asks for the families it draws instead of *
- unpowered hwmon sentinels (-93 °C) are not temperatures
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
- stock Monitors modules fed by one sensor stream, the catalog and one batch of
histories (GET /api/sensors/history?ids=…), through sampleAdapter.ts
- the stream closes while the page is hidden and refills its curves on return
- Disks tab live rates from the stream
- new sensors: hamlab.sample_interval, ups.last_event
- stream: asking for about the sampling rate means every sample
- NUT runtime rounded at the source
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
- sensors:read tokens open the sensor endpoints (here and on managed machines
through the hub) and nothing else; full tokens are never accepted from a URL
- /api/sensors/stream: a shaped query that keeps answering, over SSE
- /metrics: Prometheus text format
- services, updates, ZFS and SMART as background status sensors
- widget library over the sensor API; custom widgets on the dashboard
- /embed/widget: one live widget in its own bundle, the only frameable path
- Dashboard tab is now Monitors
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Every number the collector samples becomes a sensor with a stable dotted ID
(cpu.core.3.usage, fan.nct6775.fan2, fs.root.used_pct). Clients discover them
from the catalog and ask for exactly what they want, including as a shaped
query: send the object you want, get it back with the values filled in.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
mmcblkNboot0/boot1/rpmb are hardware areas of the chip; hide them on Disks and
Dashboard. Badge eMMC/SD instead of SSD. smartctl cannot talk to MMC, so read
life_time and pre_eol_info from /sys instead of showing its error.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
- hamlab install / uninstall (internal/setup): systemd or OpenRC, elevates through
sudo or doas, runs as the invoking user, sets or generates the password, names only
journal groups that exist, waits for /healthz, reports an active firewalld/ufw.
--purge and the data chown never touch a directory that isn't Ham Lab's alone.
- install.sh: curl-able bootstrap (arch detection, SHA256SUMS, offline beside the
binaries, noexec /tmp fallback). make release builds dist/ for amd64, arm64, arm
(ARMv6), 386, riscv64. Every build is CGO_ENABLED=0; the native one used to link glibc.
- internal/priv replaces every literal `sudo -n`: uid 0, sudo or doas.
- Updates acts on dnf, zypper, pacman and apk as well as apt; dnf shows installed versions.
- Services falls back to the plain systemctl table before systemd 246 and says so on
hosts without systemd. Disks falls back to /sys/block without lsblk.
- Listener binds with IP_FREEBIND, so a LAN address that isn't up at boot is no failure.
- Self-update: the new build must pass `check` against the live config, is put in
place by atomic rename, and the daemon re-execs itself (same under any supervisor).
- State and config writes are fsynced; a corrupt state.json is set aside, not fatal.
- Terminal shutdown no longer empties the saved list, so names come back after a
restart by design rather than by winning a race.
- Config default is ~/.config/hamlab; an existing ~/apps/hamlab is still found.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
