Mitchell-JohnsonandClaude Fable 5.1
b1c1404181
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
205 lines
9.3 KiB
Markdown
205 lines
9.3 KiB
Markdown
# Sensors — the standard read interface
|
||
|
||
Everything Ham Lab measures is a **sensor**: a stable dotted ID, a little metadata, a
|
||
current value. Any consumer — Ham Lab's own dashboard, Glance, Home Assistant, a shell
|
||
script — discovers them from the catalog and asks for exactly the ones it wants.
|
||
|
||
Code: `internal/sensors` (types, registry, selectors, shaped query) and
|
||
`internal/metrics/sensors.go` (the collector as a provider). The registry measures
|
||
nothing; providers do. A new source of sensors implements `sensors.Provider` and is
|
||
added to the registry in `api.New`.
|
||
|
||
## IDs
|
||
|
||
```
|
||
cpu.usage cpu.core.<n>.usage cpu.temp load.1|5|15 uptime
|
||
mem.total|used|avail|free|cached|buffers|used_pct swap.total|used|avail|used_pct
|
||
temp.<chip>.<label> fan.<chip>.<label>
|
||
net.<iface>.rx|tx|rx_total|tx_total|up
|
||
disk.<dev>.read|write|util|size|temp
|
||
fs.<mount>.size|used|used_pct
|
||
ups.battery|runtime|load|load_pct|in_v|out_v|on_battery|state|source
|
||
```
|
||
|
||
- Segments are lower case `[a-z0-9_-]`: `Package id 0` → `package_id_0`, `/mnt/tank` →
|
||
`mnt_tank`, `/` → `root`. The real names are in the sensor's `labels`.
|
||
- Built from names the hardware reports, never enumeration order (`hwmon3` moves between
|
||
boots). Two identical chips collide; the second gets `_2`.
|
||
- Which sensors exist differs per machine. That is the point of the catalog.
|
||
|
||
## A sensor
|
||
|
||
```json
|
||
{ "id": "temp.coretemp.package_id_0", "name": "coretemp Package id 0",
|
||
"kind": "temperature", "unit": "°C", "value": 51, "t": 1789826600878,
|
||
"warn": 105, "crit": 105, "labels": { "chip": "coretemp", "label": "Package id 0" } }
|
||
```
|
||
|
||
`value` is a number, bool or string. `t` is when it was really read, in unix ms — slow
|
||
sections (filesystems 30 s, UPS 10 s) are older than the tick. `min`/`max` are the scale
|
||
(0–100 for percentages, 0–total for used bytes); `warn`/`crit` are the hardware's own
|
||
thresholds where it states them.
|
||
|
||
## Endpoints
|
||
|
||
All behind the same auth as the rest of `/api` (session cookie or `Authorization: Bearer`).
|
||
On a hub, every one is also at `/api/nodes/<machine>/sensors…` for each managed machine.
|
||
|
||
| | |
|
||
|---|---|
|
||
| `GET /api/sensors` | Catalog, with current values. `?ids=fan.*,cpu.usage` narrows it. |
|
||
| `GET /api/sensors/{id}` | One sensor. `?format=text` → the bare value as text/plain. |
|
||
| `GET /api/sensors/{id}/history` | `{id, unit, t[], v[]}` — what the ring holds (300 samples at the sampling interval). Numbers only. |
|
||
| `GET /api/sensors/history?ids=cpu.usage,net.*.rx` | Many histories in one request: `{series: [{id, unit, t[], v[]}]}`. A sensor younger than the ring has a shorter series. |
|
||
| `POST /api/sensors/query` | Shaped query, below. |
|
||
| `GET /api/sensors/query?k=selector&…` | The same, flat, for consumers that can only fetch a URL. |
|
||
|
||
A selector is an exact ID or a glob. `*` crosses dots: `fan.*` is every fan,
|
||
`cpu.core.*.usage` every core.
|
||
|
||
## Shaped query
|
||
|
||
Send the object you want, with selectors where the values go. Get it back filled in.
|
||
|
||
```
|
||
POST /api/sensors/query
|
||
{ "cpu": "cpu.usage", "fans": "fan.*", "box": { "temp": "cpu.temp", "up": "uptime" } }
|
||
|
||
→ { "cpu": 12.4,
|
||
"fans": { "fan.nct6775.fan1": 820, "fan.nct6775.fan2": 1140 },
|
||
"box": { "temp": 41, "up": 80520 } }
|
||
```
|
||
|
||
- Exact ID → its value; `null` if this machine has no such sensor (so one query works
|
||
across machines that differ).
|
||
- Glob → an object keyed by ID; `{}` if nothing matches.
|
||
- Objects and arrays are walked; anything else in a query is a 400.
|
||
- `?full=1` → every value is the whole sensor object instead of the bare value. (In the
|
||
GET form `full` is therefore not usable as a key.)
|
||
|
||
## Live stream
|
||
|
||
`GET /api/sensors/stream` is a shaped query that keeps answering: the same object, pushed
|
||
as a server-sent event after every sample. The query is `?q=<URL-encoded JSON shape>` or
|
||
the flat form; `?full=1` as above; `?every=<ms>` spaces events out for a consumer that
|
||
doesn't want them at the sampling rate. A keepalive comment goes out every 25 s, and
|
||
`X-Accel-Buffering: no` keeps nginx from holding events back.
|
||
|
||
```js
|
||
new EventSource('/api/sensors/stream?token=…&q=' + encodeURIComponent(JSON.stringify({ cpu: 'cpu.usage', fans: 'fan.*' })))
|
||
.onmessage = (e) => render(JSON.parse(e.data))
|
||
```
|
||
|
||
## Auth: read-only tokens
|
||
|
||
Settings → Access tokens → **+ Read-only sensor token** (or `POST /api/tokens/read
|
||
{"name"}` with a session). Scope `sensors:read`: it opens every endpoint in this file,
|
||
here and on each managed machine through the hub (`/api/nodes/<machine>/sensors…`; the
|
||
hub talks to the node as itself, the caller's token stops at the hub), and nothing else.
|
||
|
||
- Send it as `Authorization: Bearer <token>`, or as `?token=<token>` where headers are
|
||
impossible (an iframe, a browser `EventSource`).
|
||
- A read-only token can be **limited to ID patterns** (`{"name": "glance", "sensors":
|
||
["cpu.*", "zfs.tank.*"]}`, or the second prompt in Settings). Everything outside them
|
||
does not exist for that token — catalog, query, history, stream, Prometheus, and on
|
||
managed machines through the hub, which passes the restriction along.
|
||
- A **full-control token is never accepted from a URL**, and a read-only token is never
|
||
accepted anywhere outside this file's endpoints.
|
||
|
||
## Prometheus
|
||
|
||
`GET /metrics` (or `/api/sensors/prometheus`, which also works through the hub as
|
||
`/api/nodes/<machine>/sensors/prometheus`). One gauge family per kind of measurement,
|
||
base units, the sensor ID and its real-world names as labels:
|
||
|
||
```
|
||
hamlab_temperature_celsius{id="temp.coretemp.core_0",chip="coretemp",label="Core 0"} 50
|
||
hamlab_usage_percent{id="fs.root.used_pct",fstype="ext4",mount="/"} 10.1
|
||
hamlab_state{id="ups.state",model="CPS CP1500PFCRM2U",value="Normal"} 1
|
||
hamlab_updates_pending{id="updates.pending",manager="apt"} 3
|
||
```
|
||
|
||
```yaml
|
||
scrape_configs:
|
||
- job_name: hamlab
|
||
authorization: { credentials: <read-only token> }
|
||
static_configs: [{ targets: ['10.0.0.234:8100'] }]
|
||
```
|
||
|
||
## Status sensors
|
||
|
||
Not everything is a /proc number. These are read in the background on their own clocks
|
||
(each sensor's `t` is its real age) by `internal/api/status_sensors.go`:
|
||
|
||
```
|
||
services.failed|failed_names|running every 30 s
|
||
service.<unit>.active|state one pair per service that is enabled, up or failed
|
||
containers.total|running|unhealthy every 15 s (Docker or Podman, if present)
|
||
container.<name>.running|state|health|mem|cpu mem and cpu read from the container's cgroup
|
||
updates.pending|security|reboot_required every 5 min
|
||
zfs.<pool>.health|healthy|size|used|used_pct|frag every 1 min
|
||
smart.<dev>.passed|warnings|power_on|wear every 10 min, never waking a sleeping disk
|
||
```
|
||
|
||
## Widgets
|
||
|
||
A widget is a spec — `{type, sensor, title?}` — over this API and nothing else
|
||
(`web/src/widgets.tsx`, fed by `useSensorStream` in `web/src/sensors.ts`). Types: `stat`,
|
||
`gauge`, `bar`, `spark`, and `list` for a pattern. Scale and colour come from the
|
||
sensor's own `min`/`max`/`warn`/`crit`.
|
||
|
||
- **In Ham Lab:** Monitors → Customize → **+ Add widget**. Saved with the layout,
|
||
per machine; all of a page's widgets share one stream.
|
||
- **Widgets tab:** every widget type, then every sensor on the machine drawn once, live,
|
||
grouped by family (`web/src/Gallery.tsx`). Tap a tile to copy its sensor ID — the
|
||
quickest way to find what to put in a widget, a query or a Glance tile.
|
||
- **Anywhere else:** `/embed/widget` is one widget alone on a page, live, in its own small
|
||
bundle (the console never loads), and the only path Ham Lab allows to be framed.
|
||
|
||
```
|
||
/embed/widget?type=gauge&sensor=cpu.usage&token=<read-only token>
|
||
&machine=grid &title=… &theme=dark|light|cyberpunk|retro &every=<ms> &bare=1
|
||
```
|
||
|
||
### Glance
|
||
|
||
Glance's own `custom-api` widget refetches on page load / cache expiry — fine for slow
|
||
facts, never live. An `iframe` of the embed is live.
|
||
|
||
```yaml
|
||
- type: iframe
|
||
title: core CPU
|
||
source: https://hamlab.bigham.dev/embed/widget?type=gauge&sensor=cpu.usage&bare=1&token=<read-only token>
|
||
height: 130
|
||
|
||
- type: custom-api
|
||
title: grid
|
||
cache: 1m
|
||
url: https://hamlab.bigham.dev/api/nodes/grid/sensors/query?pool=zfs.tank.used_pct&failed=services.failed&updates=updates.pending
|
||
headers: { Authorization: Bearer <read-only token> }
|
||
template: |
|
||
<div>tank {{ .JSON.Float "pool" }}% · {{ .JSON.Int "failed" }} failed · {{ .JSON.Int "updates" }} updates</div>
|
||
```
|
||
|
||
## The whole fleet in one request
|
||
|
||
On a hub, `/api/fleet/sensors/query` asks every machine (or `machines`) the same shaped
|
||
query and answers `{machine: result}`; this machine answers under its own hostname, and
|
||
one that can't be reached answers `{"error": …}` while the rest still do.
|
||
|
||
```
|
||
POST /api/fleet/sensors/query {"machines": "*", "query": {"cpu": "cpu.usage", "up": "uptime"}}
|
||
GET /api/fleet/sensors/query?machines=grid,wyse&cpu=cpu.usage&failed=services.failed
|
||
|
||
→ {"core": {"cpu": 4.8, "up": 50211}, "grid": {"cpu": 0.4, "up": 437177}, "mac": {"error": "timeout"}}
|
||
```
|
||
|
||
## What Ham Lab itself runs on
|
||
|
||
Since 2026-09-19 the Monitors tab (stock modules and custom widgets) and the Disks tab's
|
||
live rates read only this API: the catalog once, one batch of histories, one stream —
|
||
closed while the page is hidden, refilled when it returns. `web/src/sampleAdapter.ts` is
|
||
the seam that hands the stock modules the object they were written against. The old
|
||
`/api/metrics` and `/api/metrics/history` are gone. `hamlab.sample_interval` is the
|
||
sampler's own rate, as a sensor.
|