Sensors: one standard read interface — catalog, single value, history, shaped query
Every number the collector samples becomes a sensor with a stable dotted ID (cpu.core.3.usage, fan.nct6775.fan2, fs.root.used_pct). Clients discover them from the catalog and ask for exactly what they want, including as a shaped query: send the object you want, get it back with the values filled in. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
This commit is contained in:
co-authored by
Claude Fable 5.1
parent
01aeb226ea
commit
43bf787696
@@ -176,6 +176,18 @@ something long in one, switch away and back, refresh — it's all still there.
|
|||||||
the daemon's sampling clock; the chart window is always 300 samples, so 30 s at 100 ms
|
the daemon's sampling clock; the chart window is always 300 samples, so 30 s at 100 ms
|
||||||
and 10 min at 2 s. Still to do: persistent
|
and 10 min at 2 s. Still to do: persistent
|
||||||
history (SQLite ring, e.g. 10 s × 7 days), SMART (`smartctl -j`), longer time ranges.
|
history (SQLite ring, e.g. 10 s × 7 days), SMART (`smartctl -j`), longer time ranges.
|
||||||
|
- **Sensors** — **step 1 shipped 2026-09-19** (`internal/sensors`,
|
||||||
|
`internal/metrics/sensors.go`, reference in `SENSORS.md`): everything measured is a
|
||||||
|
sensor with a stable dotted ID; catalog, single value (`?format=text`), per-sensor
|
||||||
|
history, and a **shaped query** where the client sends the object it wants and gets it
|
||||||
|
back filled in (GraphQL's idea without GraphQL: sensors differ per machine, so a static
|
||||||
|
schema would collapse to `sensor(id:)` anyway). `/api/metrics` and the Dashboard are
|
||||||
|
untouched so far. **Remaining, in order:** (2) read-only scoped tokens + Prometheus
|
||||||
|
`/metrics`; (3) push stream + widget library, Dashboard rebuilt on its own sensors;
|
||||||
|
(4) embeddable widget page + Glance config (Glance's `custom-api` only refetches on page
|
||||||
|
load, an `iframe` of our embed is what's live); (5) non-metric sensors — services,
|
||||||
|
SMART, ZFS, updates, containers. One sampler rate, set in one place, is part of (3):
|
||||||
|
today the open dashboard still drives it.
|
||||||
- **Multi-host** — **Phase 1 shipped 2026-09-18** (`internal/nodes`, `internal/tlsauto`,
|
- **Multi-host** — **Phase 1 shipped 2026-09-18** (`internal/nodes`, `internal/tlsauto`,
|
||||||
`MachineSwitcher.tsx`, `Machines.tsx`). One binary, hub and spoke: a host whose
|
`MachineSwitcher.tsx`, `Machines.tsx`). One binary, hub and spoke: a host whose
|
||||||
`config.json` lists `nodes` is a hub; any other is a standalone node with its full UI.
|
`config.json` lists `nodes` is a hub; any other is a standalone node with its full UI.
|
||||||
|
|||||||
+84
@@ -0,0 +1,84 @@
|
|||||||
|
# Sensors — the standard read interface
|
||||||
|
|
||||||
|
Everything Ham Lab measures is a **sensor**: a stable dotted ID, a little metadata, a
|
||||||
|
current value. Any consumer — Ham Lab's own dashboard, Glance, Home Assistant, a shell
|
||||||
|
script — discovers them from the catalog and asks for exactly the ones it wants.
|
||||||
|
|
||||||
|
Code: `internal/sensors` (types, registry, selectors, shaped query) and
|
||||||
|
`internal/metrics/sensors.go` (the collector as a provider). The registry measures
|
||||||
|
nothing; providers do. A new source of sensors implements `sensors.Provider` and is
|
||||||
|
added to the registry in `api.New`.
|
||||||
|
|
||||||
|
## IDs
|
||||||
|
|
||||||
|
```
|
||||||
|
cpu.usage cpu.core.<n>.usage cpu.temp load.1|5|15 uptime
|
||||||
|
mem.total|used|avail|free|cached|buffers|used_pct swap.total|used|avail|used_pct
|
||||||
|
temp.<chip>.<label> fan.<chip>.<label>
|
||||||
|
net.<iface>.rx|tx|rx_total|tx_total|up
|
||||||
|
disk.<dev>.read|write|util|size|temp
|
||||||
|
fs.<mount>.size|used|used_pct
|
||||||
|
ups.battery|runtime|load|load_pct|in_v|out_v|on_battery|state|source
|
||||||
|
```
|
||||||
|
|
||||||
|
- Segments are lower case `[a-z0-9_-]`: `Package id 0` → `package_id_0`, `/mnt/tank` →
|
||||||
|
`mnt_tank`, `/` → `root`. The real names are in the sensor's `labels`.
|
||||||
|
- Built from names the hardware reports, never enumeration order (`hwmon3` moves between
|
||||||
|
boots). Two identical chips collide; the second gets `_2`.
|
||||||
|
- Which sensors exist differs per machine. That is the point of the catalog.
|
||||||
|
|
||||||
|
## A sensor
|
||||||
|
|
||||||
|
```json
|
||||||
|
{ "id": "temp.coretemp.package_id_0", "name": "coretemp Package id 0",
|
||||||
|
"kind": "temperature", "unit": "°C", "value": 51, "t": 1789826600878,
|
||||||
|
"warn": 105, "crit": 105, "labels": { "chip": "coretemp", "label": "Package id 0" } }
|
||||||
|
```
|
||||||
|
|
||||||
|
`value` is a number, bool or string. `t` is when it was really read, in unix ms — slow
|
||||||
|
sections (filesystems 30 s, UPS 10 s) are older than the tick. `min`/`max` are the scale
|
||||||
|
(0–100 for percentages, 0–total for used bytes); `warn`/`crit` are the hardware's own
|
||||||
|
thresholds where it states them.
|
||||||
|
|
||||||
|
## Endpoints
|
||||||
|
|
||||||
|
All behind the same auth as the rest of `/api` (session cookie or `Authorization: Bearer`).
|
||||||
|
On a hub, every one is also at `/api/nodes/<machine>/sensors…` for each managed machine.
|
||||||
|
|
||||||
|
| | |
|
||||||
|
|---|---|
|
||||||
|
| `GET /api/sensors` | Catalog, with current values. `?ids=fan.*,cpu.usage` narrows it. |
|
||||||
|
| `GET /api/sensors/{id}` | One sensor. `?format=text` → the bare value as text/plain. |
|
||||||
|
| `GET /api/sensors/{id}/history` | `{id, unit, t[], v[]}` — what the ring holds (300 samples at the sampling interval). Numbers only. |
|
||||||
|
| `POST /api/sensors/query` | Shaped query, below. |
|
||||||
|
| `GET /api/sensors/query?k=selector&…` | The same, flat, for consumers that can only fetch a URL. |
|
||||||
|
|
||||||
|
A selector is an exact ID or a glob. `*` crosses dots: `fan.*` is every fan,
|
||||||
|
`cpu.core.*.usage` every core.
|
||||||
|
|
||||||
|
## Shaped query
|
||||||
|
|
||||||
|
Send the object you want, with selectors where the values go. Get it back filled in.
|
||||||
|
|
||||||
|
```
|
||||||
|
POST /api/sensors/query
|
||||||
|
{ "cpu": "cpu.usage", "fans": "fan.*", "box": { "temp": "cpu.temp", "up": "uptime" } }
|
||||||
|
|
||||||
|
→ { "cpu": 12.4,
|
||||||
|
"fans": { "fan.nct6775.fan1": 820, "fan.nct6775.fan2": 1140 },
|
||||||
|
"box": { "temp": 41, "up": 80520 } }
|
||||||
|
```
|
||||||
|
|
||||||
|
- Exact ID → its value; `null` if this machine has no such sensor (so one query works
|
||||||
|
across machines that differ).
|
||||||
|
- Glob → an object keyed by ID; `{}` if nothing matches.
|
||||||
|
- Objects and arrays are walked; anything else in a query is a 400.
|
||||||
|
- `?full=1` → every value is the whole sensor object instead of the bare value. (In the
|
||||||
|
GET form `full` is therefore not usable as a key.)
|
||||||
|
|
||||||
|
## Not yet
|
||||||
|
|
||||||
|
Read-only scoped tokens, Prometheus `/metrics`, a push stream, widgets built on this,
|
||||||
|
an embeddable widget page, and non-metric sensors (services, SMART, ZFS, updates). Until
|
||||||
|
scoped tokens exist, **do not hand a Ham Lab token to a dashboard** — today's tokens are
|
||||||
|
full control of the machine.
|
||||||
+84
-1
@@ -34,6 +34,7 @@ import (
|
|||||||
"hamlab/internal/metrics"
|
"hamlab/internal/metrics"
|
||||||
"hamlab/internal/nodes"
|
"hamlab/internal/nodes"
|
||||||
"hamlab/internal/priv"
|
"hamlab/internal/priv"
|
||||||
|
"hamlab/internal/sensors"
|
||||||
"hamlab/internal/services"
|
"hamlab/internal/services"
|
||||||
"hamlab/internal/setup"
|
"hamlab/internal/setup"
|
||||||
"hamlab/internal/storage"
|
"hamlab/internal/storage"
|
||||||
@@ -54,6 +55,7 @@ type Server struct {
|
|||||||
limiter auth.Limiter
|
limiter auth.Limiter
|
||||||
mgr *term.Manager
|
mgr *term.Manager
|
||||||
metrics *metrics.Collector
|
metrics *metrics.Collector
|
||||||
|
sensors *sensors.Registry
|
||||||
store *store.Store
|
store *store.Store
|
||||||
svc *services.Manager
|
svc *services.Manager
|
||||||
cron *cron.Manager
|
cron *cron.Manager
|
||||||
@@ -74,7 +76,7 @@ type Server struct {
|
|||||||
|
|
||||||
func New(cfg *config.Config, sessions *auth.Sessions, mgr *term.Manager, col *metrics.Collector, st *store.Store, svc *services.Manager, cr *cron.Manager, tm *tasks.Manager, up *updates.Manager, sto *storage.Manager, fsLocal, fsSudo files.FS, nm *nodes.Manager, fingerprint string, ui fs.FS, version string) *Server {
|
func New(cfg *config.Config, sessions *auth.Sessions, mgr *term.Manager, col *metrics.Collector, st *store.Store, svc *services.Manager, cr *cron.Manager, tm *tasks.Manager, up *updates.Manager, sto *storage.Manager, fsLocal, fsSudo files.FS, nm *nodes.Manager, fingerprint string, ui fs.FS, version string) *Server {
|
||||||
host, _ := os.Hostname()
|
host, _ := os.Hostname()
|
||||||
s := &Server{cfg: cfg, sessions: sessions, mgr: mgr, metrics: col, store: st, svc: svc, cron: cr, tasks: tm, updates: up, storage: sto, fsLocal: fsLocal, fsSudo: fsSudo, nodes: nm, fingerprint: fingerprint, ui: ui, version: version,
|
s := &Server{cfg: cfg, sessions: sessions, mgr: mgr, metrics: col, sensors: sensors.NewRegistry(col), store: st, svc: svc, cron: cr, tasks: tm, updates: up, storage: sto, fsLocal: fsLocal, fsSudo: fsSudo, nodes: nm, fingerprint: fingerprint, ui: ui, version: version,
|
||||||
started: time.Now(), host: host, user: os.Getenv("USER")}
|
started: time.Now(), host: host, user: os.Getenv("USER")}
|
||||||
if u, err := user.Current(); err == nil && u.Username != "" {
|
if u, err := user.Current(); err == nil && u.Username != "" {
|
||||||
s.user = u.Username
|
s.user = u.Username
|
||||||
@@ -109,6 +111,11 @@ func New(cfg *config.Config, sessions *auth.Sessions, mgr *term.Manager, col *me
|
|||||||
m.HandleFunc("GET /api/terminals/{id}/attach", s.auth(s.attach))
|
m.HandleFunc("GET /api/terminals/{id}/attach", s.auth(s.attach))
|
||||||
m.HandleFunc("GET /api/metrics", s.auth(s.metricsLatest))
|
m.HandleFunc("GET /api/metrics", s.auth(s.metricsLatest))
|
||||||
m.HandleFunc("GET /api/metrics/history", s.auth(s.metricsHistory))
|
m.HandleFunc("GET /api/metrics/history", s.auth(s.metricsHistory))
|
||||||
|
m.HandleFunc("GET /api/sensors", s.auth(s.sensorCatalog))
|
||||||
|
m.HandleFunc("GET /api/sensors/query", s.auth(s.sensorQuery))
|
||||||
|
m.HandleFunc("POST /api/sensors/query", s.auth(s.sensorQuery))
|
||||||
|
m.HandleFunc("GET /api/sensors/{id}", s.auth(s.sensorOne))
|
||||||
|
m.HandleFunc("GET /api/sensors/{id}/history", s.auth(s.sensorHistory))
|
||||||
m.HandleFunc("GET /api/services", s.auth(s.listServices))
|
m.HandleFunc("GET /api/services", s.auth(s.listServices))
|
||||||
m.HandleFunc("GET /api/services/{unit}", s.auth(s.showService))
|
m.HandleFunc("GET /api/services/{unit}", s.auth(s.showService))
|
||||||
m.HandleFunc("GET /api/services/{unit}/file", s.auth(s.serviceFile))
|
m.HandleFunc("GET /api/services/{unit}/file", s.auth(s.serviceFile))
|
||||||
@@ -568,6 +575,82 @@ func (s *Server) metricsHistory(w http.ResponseWriter, _ *http.Request) {
|
|||||||
jsonOK(w, 200, s.metrics.History())
|
jsonOK(w, 200, s.metrics.History())
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// ---- sensors: the standard read interface (SENSORS.md) ----
|
||||||
|
|
||||||
|
// sensorCatalog lists every sensor with its current value; ?ids= narrows it with
|
||||||
|
// comma-separated IDs or globs.
|
||||||
|
func (s *Server) sensorCatalog(w http.ResponseWriter, r *http.Request) {
|
||||||
|
all := s.sensors.All()
|
||||||
|
if ids := r.URL.Query().Get("ids"); ids != "" {
|
||||||
|
picked := []sensors.Sensor{}
|
||||||
|
for _, sel := range strings.Split(ids, ",") {
|
||||||
|
got, err := sensors.Select(all, strings.TrimSpace(sel))
|
||||||
|
if err != nil {
|
||||||
|
jsonErr(w, 400, err.Error())
|
||||||
|
return
|
||||||
|
}
|
||||||
|
picked = append(picked, got...)
|
||||||
|
}
|
||||||
|
all = picked
|
||||||
|
}
|
||||||
|
jsonOK(w, 200, map[string]any{"sensors": all})
|
||||||
|
}
|
||||||
|
|
||||||
|
// sensorOne is a single sensor; ?format=text is the bare value, for shell scripts and
|
||||||
|
// anything else that can only fetch a URL and read a number.
|
||||||
|
func (s *Server) sensorOne(w http.ResponseWriter, r *http.Request) {
|
||||||
|
got, _ := sensors.Select(s.sensors.All(), r.PathValue("id"))
|
||||||
|
if len(got) != 1 {
|
||||||
|
jsonErr(w, 404, "no such sensor")
|
||||||
|
return
|
||||||
|
}
|
||||||
|
if r.URL.Query().Get("format") == "text" {
|
||||||
|
w.Header().Set("Content-Type", "text/plain; charset=utf-8")
|
||||||
|
fmt.Fprintln(w, got[0].Value)
|
||||||
|
return
|
||||||
|
}
|
||||||
|
jsonOK(w, 200, got[0])
|
||||||
|
}
|
||||||
|
|
||||||
|
func (s *Server) sensorHistory(w http.ResponseWriter, r *http.Request) {
|
||||||
|
h, ok := s.sensors.History(r.PathValue("id"))
|
||||||
|
if !ok {
|
||||||
|
jsonErr(w, 404, "no history for that sensor")
|
||||||
|
return
|
||||||
|
}
|
||||||
|
jsonOK(w, 200, h)
|
||||||
|
}
|
||||||
|
|
||||||
|
// sensorQuery answers a shaped query. POST sends the shape as the body; GET builds a
|
||||||
|
// flat one from the query string (?cpu=cpu.usage&fans=fan.*). ?full=1 returns whole
|
||||||
|
// sensors instead of bare values.
|
||||||
|
func (s *Server) sensorQuery(w http.ResponseWriter, r *http.Request) {
|
||||||
|
full := r.URL.Query().Get("full") == "1"
|
||||||
|
var q json.RawMessage
|
||||||
|
if r.Method == http.MethodPost {
|
||||||
|
b, err := io.ReadAll(http.MaxBytesReader(w, r.Body, 64<<10))
|
||||||
|
if err != nil {
|
||||||
|
jsonErr(w, 413, "query larger than 64 KiB")
|
||||||
|
return
|
||||||
|
}
|
||||||
|
q = b
|
||||||
|
} else {
|
||||||
|
flat := map[string]string{}
|
||||||
|
for k, v := range r.URL.Query() {
|
||||||
|
if k != "full" && len(v) > 0 {
|
||||||
|
flat[k] = v[0]
|
||||||
|
}
|
||||||
|
}
|
||||||
|
q, _ = json.Marshal(flat)
|
||||||
|
}
|
||||||
|
out, err := sensors.Shape(s.sensors.All(), q, full)
|
||||||
|
if err != nil {
|
||||||
|
jsonErr(w, 400, err.Error())
|
||||||
|
return
|
||||||
|
}
|
||||||
|
jsonOK(w, 200, out)
|
||||||
|
}
|
||||||
|
|
||||||
// ---- services (systemd) ----
|
// ---- services (systemd) ----
|
||||||
|
|
||||||
func (s *Server) listServices(w http.ResponseWriter, r *http.Request) {
|
func (s *Server) listServices(w http.ResponseWriter, r *http.Request) {
|
||||||
|
|||||||
@@ -132,6 +132,8 @@ type Sample struct {
|
|||||||
FS []FS `json:"fs"`
|
FS []FS `json:"fs"`
|
||||||
UPS *UPS `json:"ups"`
|
UPS *UPS `json:"ups"`
|
||||||
Uptime float64 `json:"uptime"`
|
Uptime float64 `json:"uptime"`
|
||||||
|
|
||||||
|
fsT, upsT int64 // when the slow sections were really read; sensors report these, not T
|
||||||
}
|
}
|
||||||
|
|
||||||
// Series is the compact history the UI seeds its sparklines from.
|
// Series is the compact history the UI seeds its sparklines from.
|
||||||
@@ -292,11 +294,11 @@ func (c *Collector) sample() {
|
|||||||
if c.lastFS == nil || now.Sub(c.fsAt) >= fsEvery {
|
if c.lastFS == nil || now.Sub(c.fsAt) >= fsEvery {
|
||||||
c.lastFS, c.fsAt = filesystems(), now
|
c.lastFS, c.fsAt = filesystems(), now
|
||||||
}
|
}
|
||||||
s.FS = c.lastFS
|
s.FS, s.fsT = c.lastFS, c.fsAt.UnixMilli()
|
||||||
if c.upsProbe != nil && (c.upsAt.IsZero() || now.Sub(c.upsAt) >= upsEvery) {
|
if c.upsProbe != nil && (c.upsAt.IsZero() || now.Sub(c.upsAt) >= upsEvery) {
|
||||||
c.lastUPS, c.upsAt = c.upsProbe(), now
|
c.lastUPS, c.upsAt = c.upsProbe(), now
|
||||||
}
|
}
|
||||||
s.UPS = c.lastUPS
|
s.UPS, s.upsT = c.lastUPS, c.upsAt.UnixMilli()
|
||||||
|
|
||||||
if first {
|
if first {
|
||||||
return // deltas are meaningless on the priming pass
|
return // deltas are meaningless on the priming pass
|
||||||
|
|||||||
@@ -0,0 +1,169 @@
|
|||||||
|
package metrics
|
||||||
|
|
||||||
|
import (
|
||||||
|
"fmt"
|
||||||
|
"strconv"
|
||||||
|
|
||||||
|
"hamlab/internal/sensors"
|
||||||
|
)
|
||||||
|
|
||||||
|
// The collector as a sensors.Provider: every number in a Sample, one sensor each. The
|
||||||
|
// IDs are built from names the hardware reports (chip, label, interface, device, mount),
|
||||||
|
// never from enumeration order like hwmon3, which moves between boots.
|
||||||
|
|
||||||
|
// flattener turns one Sample into sensors. With only set it builds just that one, so
|
||||||
|
// walking the whole ring for a history costs string compares, not allocations.
|
||||||
|
type flattener struct {
|
||||||
|
only string
|
||||||
|
seen map[string]int
|
||||||
|
out []sensors.Sensor
|
||||||
|
}
|
||||||
|
|
||||||
|
type bounds struct{ min, max, warn, crit *float64 }
|
||||||
|
|
||||||
|
var pct = bounds{min: sensors.F(0), max: sensors.F(100)}
|
||||||
|
|
||||||
|
func upTo(max float64) bounds { return bounds{min: sensors.F(0), max: sensors.F(max)} }
|
||||||
|
|
||||||
|
func (f *flattener) add(id, name, kind, unit string, v any, t int64, b bounds, labels ...string) {
|
||||||
|
f.seen[id]++
|
||||||
|
if n := f.seen[id]; n > 1 { // two identical chips: the second is coretemp_2
|
||||||
|
id = fmt.Sprintf("%s_%d", id, n)
|
||||||
|
}
|
||||||
|
if f.only != "" && id != f.only {
|
||||||
|
return
|
||||||
|
}
|
||||||
|
s := sensors.Sensor{ID: id, Name: name, Kind: kind, Unit: unit, Value: v, T: t, Min: b.min, Max: b.max, Warn: b.warn, Crit: b.crit}
|
||||||
|
if len(labels) > 0 {
|
||||||
|
s.Labels = make(map[string]string, len(labels)/2)
|
||||||
|
for i := 0; i+1 < len(labels); i += 2 {
|
||||||
|
s.Labels[labels[i]] = labels[i+1]
|
||||||
|
}
|
||||||
|
}
|
||||||
|
f.out = append(f.out, s)
|
||||||
|
}
|
||||||
|
|
||||||
|
func ratio(part, whole uint64) float64 {
|
||||||
|
if whole == 0 {
|
||||||
|
return 0
|
||||||
|
}
|
||||||
|
return round1(float64(part) / float64(whole) * 100)
|
||||||
|
}
|
||||||
|
|
||||||
|
func flatten(s Sample, only string) []sensors.Sensor {
|
||||||
|
f := &flattener{only: only, seen: map[string]int{}}
|
||||||
|
t, none := s.T, bounds{}
|
||||||
|
|
||||||
|
f.add("cpu.usage", "CPU", "usage", "%", s.CPU, t, pct)
|
||||||
|
for i, c := range s.Cores {
|
||||||
|
n := strconv.Itoa(i)
|
||||||
|
f.add("cpu.core."+n+".usage", "Core "+n, "usage", "%", c, t, pct, "core", n)
|
||||||
|
}
|
||||||
|
if c := cpuTemp(s.Temps); c > 0 {
|
||||||
|
f.add("cpu.temp", "CPU temperature", "temperature", "°C", c, t, none)
|
||||||
|
}
|
||||||
|
for i, w := range []string{"1", "5", "15"} {
|
||||||
|
f.add("load."+w, "Load "+w+" min", "load", "", s.Load[i], t, none)
|
||||||
|
}
|
||||||
|
f.add("uptime", "Uptime", "duration", "s", s.Uptime, t, none)
|
||||||
|
|
||||||
|
for _, m := range []struct {
|
||||||
|
id, name string
|
||||||
|
v Mem
|
||||||
|
}{{"mem", "Memory", s.Mem}, {"swap", "Swap", s.Swap}} {
|
||||||
|
f.add(m.id+".total", m.name+" total", "bytes", "B", float64(m.v.Total), t, none)
|
||||||
|
f.add(m.id+".used", m.name+" used", "bytes", "B", float64(m.v.Used), t, upTo(float64(m.v.Total)))
|
||||||
|
f.add(m.id+".avail", m.name+" available", "bytes", "B", float64(m.v.Avail), t, upTo(float64(m.v.Total)))
|
||||||
|
f.add(m.id+".used_pct", m.name+" used", "usage", "%", ratio(m.v.Used, m.v.Total), t, pct)
|
||||||
|
}
|
||||||
|
f.add("mem.free", "Memory free", "bytes", "B", float64(s.Mem.Free), t, upTo(float64(s.Mem.Total)))
|
||||||
|
f.add("mem.cached", "Memory cached", "bytes", "B", float64(s.Mem.Cached), t, upTo(float64(s.Mem.Total)))
|
||||||
|
f.add("mem.buffers", "Memory buffers", "bytes", "B", float64(s.Mem.Buffers), t, upTo(float64(s.Mem.Total)))
|
||||||
|
|
||||||
|
for _, x := range s.Temps {
|
||||||
|
b := bounds{}
|
||||||
|
if x.Max > 0 {
|
||||||
|
b.warn = sensors.F(x.Max)
|
||||||
|
}
|
||||||
|
if x.Crit > 0 {
|
||||||
|
b.crit = sensors.F(x.Crit)
|
||||||
|
}
|
||||||
|
f.add(sensors.ID("temp", sensors.Seg(x.Chip), sensors.Seg(x.Label)), x.Chip+" "+x.Label, "temperature", "°C", x.C, t, b, "chip", x.Chip, "label", x.Label)
|
||||||
|
}
|
||||||
|
for _, x := range s.Fans {
|
||||||
|
f.add(sensors.ID("fan", sensors.Seg(x.Chip), sensors.Seg(x.Label)), x.Chip+" "+x.Label, "fan", "rpm", float64(x.RPM), t, none, "chip", x.Chip, "label", x.Label)
|
||||||
|
}
|
||||||
|
|
||||||
|
for _, x := range s.Net {
|
||||||
|
p := "net." + sensors.Seg(x.Iface)
|
||||||
|
f.add(p+".rx", x.Iface+" receive", "rate", "B/s", x.RxBps, t, none, "iface", x.Iface)
|
||||||
|
f.add(p+".tx", x.Iface+" transmit", "rate", "B/s", x.TxBps, t, none, "iface", x.Iface)
|
||||||
|
f.add(p+".rx_total", x.Iface+" received", "bytes", "B", float64(x.RxTotal), t, none, "iface", x.Iface)
|
||||||
|
f.add(p+".tx_total", x.Iface+" transmitted", "bytes", "B", float64(x.TxTotal), t, none, "iface", x.Iface)
|
||||||
|
f.add(p+".up", x.Iface+" link", "state", "", x.Up, t, none, "iface", x.Iface)
|
||||||
|
}
|
||||||
|
|
||||||
|
for _, x := range s.Disks {
|
||||||
|
p := "disk." + sensors.Seg(x.Name)
|
||||||
|
f.add(p+".read", x.Name+" read", "rate", "B/s", x.ReadBps, t, none, "device", x.Name, "model", x.Model)
|
||||||
|
f.add(p+".write", x.Name+" write", "rate", "B/s", x.WriteBps, t, none, "device", x.Name, "model", x.Model)
|
||||||
|
f.add(p+".util", x.Name+" busy", "usage", "%", x.UtilPct, t, pct, "device", x.Name, "model", x.Model)
|
||||||
|
f.add(p+".size", x.Name+" size", "bytes", "B", float64(x.Size), t, none, "device", x.Name, "model", x.Model)
|
||||||
|
if x.Temp > 0 {
|
||||||
|
f.add(p+".temp", x.Name+" temperature", "temperature", "°C", x.Temp, t, none, "device", x.Name, "model", x.Model)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
for _, x := range s.FS {
|
||||||
|
p := "fs." + sensors.Seg(x.Mount)
|
||||||
|
f.add(p+".size", x.Mount+" size", "bytes", "B", float64(x.Size), s.fsT, none, "mount", x.Mount, "fstype", x.Type)
|
||||||
|
f.add(p+".used", x.Mount+" used", "bytes", "B", float64(x.Used), s.fsT, upTo(float64(x.Size)), "mount", x.Mount, "fstype", x.Type)
|
||||||
|
f.add(p+".used_pct", x.Mount+" used", "usage", "%", ratio(x.Used, x.Size), s.fsT, pct, "mount", x.Mount, "fstype", x.Type)
|
||||||
|
}
|
||||||
|
|
||||||
|
if u := s.UPS; u != nil {
|
||||||
|
ut := s.upsT
|
||||||
|
f.add("ups.battery", "UPS battery", "usage", "%", u.BatteryPct, ut, pct, "model", u.Model)
|
||||||
|
f.add("ups.runtime", "UPS runtime", "duration", "min", u.RuntimeMin, ut, none, "model", u.Model)
|
||||||
|
f.add("ups.load", "UPS load", "power", "W", u.LoadW, ut, none, "model", u.Model)
|
||||||
|
f.add("ups.load_pct", "UPS load", "usage", "%", u.LoadPct, ut, pct, "model", u.Model)
|
||||||
|
f.add("ups.in_v", "UPS input", "voltage", "V", u.InV, ut, none, "model", u.Model)
|
||||||
|
f.add("ups.out_v", "UPS output", "voltage", "V", u.OutV, ut, none, "model", u.Model)
|
||||||
|
f.add("ups.on_battery", "UPS on battery", "state", "", u.OnBattery, ut, none, "model", u.Model)
|
||||||
|
f.add("ups.state", "UPS state", "state", "", u.State, ut, none, "model", u.Model)
|
||||||
|
f.add("ups.source", "UPS power source", "state", "", u.Source, ut, none, "model", u.Model)
|
||||||
|
}
|
||||||
|
return f.out
|
||||||
|
}
|
||||||
|
|
||||||
|
// Sensors implements sensors.Provider.
|
||||||
|
func (c *Collector) Sensors() []sensors.Sensor {
|
||||||
|
s, ok := c.Latest()
|
||||||
|
if !ok {
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
return flatten(s, "")
|
||||||
|
}
|
||||||
|
|
||||||
|
// SensorHistory implements sensors.Historian: the ring, seen through one sensor. Samples
|
||||||
|
// where the sensor didn't exist (a disk plugged in since) are left out, not zero-filled.
|
||||||
|
func (c *Collector) SensorHistory(id string) (sensors.Series, bool) {
|
||||||
|
c.mu.RLock()
|
||||||
|
defer c.mu.RUnlock()
|
||||||
|
out := sensors.Series{ID: id, T: []int64{}, V: []float64{}}
|
||||||
|
found := false
|
||||||
|
for _, smp := range c.ring {
|
||||||
|
got := flatten(smp, id)
|
||||||
|
if len(got) == 0 {
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
found = true
|
||||||
|
v, ok := got[0].Value.(float64)
|
||||||
|
if !ok {
|
||||||
|
return sensors.Series{}, false // states have no curve
|
||||||
|
}
|
||||||
|
out.Unit = got[0].Unit
|
||||||
|
out.T, out.V = append(out.T, smp.T), append(out.V, v)
|
||||||
|
}
|
||||||
|
return out, found
|
||||||
|
}
|
||||||
@@ -0,0 +1,50 @@
|
|||||||
|
package metrics
|
||||||
|
|
||||||
|
import "testing"
|
||||||
|
|
||||||
|
func TestFlatten(t *testing.T) {
|
||||||
|
s := Sample{T: 1000, CPU: 10, Cores: []float64{5, 15}, Mem: Mem{Total: 200, Used: 50},
|
||||||
|
Temps: []Temp{{Chip: "nvme", Label: "Composite", C: 40, Crit: 85}, {Chip: "nvme", Label: "Composite", C: 45}},
|
||||||
|
Fans: []Fan{{Chip: "nct6775", Label: "fan2", RPM: 1100}},
|
||||||
|
FS: []FS{{Mount: "/", Size: 100, Used: 25}}, fsT: 900}
|
||||||
|
got := map[string]any{}
|
||||||
|
for _, x := range flatten(s, "") {
|
||||||
|
if _, dup := got[x.ID]; dup {
|
||||||
|
t.Errorf("duplicate id %s", x.ID)
|
||||||
|
}
|
||||||
|
got[x.ID] = x.Value
|
||||||
|
}
|
||||||
|
for id, want := range map[string]any{"cpu.usage": 10.0, "cpu.core.1.usage": 15.0, "mem.used_pct": 25.0, "temp.nvme.composite": 40.0,
|
||||||
|
"temp.nvme.composite_2": 45.0, "fan.nct6775.fan2": 1100.0, "fs.root.used_pct": 25.0} {
|
||||||
|
if got[id] != want {
|
||||||
|
t.Errorf("%s = %v, want %v", id, got[id], want)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
one := flatten(s, "temp.nvme.composite_2")
|
||||||
|
if len(one) != 1 || one[0].Value != 45.0 {
|
||||||
|
t.Errorf("only: got %+v", one)
|
||||||
|
}
|
||||||
|
for _, x := range flatten(s, "fs.root.used") {
|
||||||
|
if x.T != 900 {
|
||||||
|
t.Errorf("fs sensor should carry the fs read time, got %d", x.T)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestSensorHistory(t *testing.T) {
|
||||||
|
c := NewCollector()
|
||||||
|
c.ring = []Sample{{T: 1, CPU: 10}, {T: 2, CPU: 20, Net: []Net{{Iface: "eth0", Up: true, RxBps: 5}}}}
|
||||||
|
h, ok := c.SensorHistory("cpu.usage")
|
||||||
|
if !ok || len(h.V) != 2 || h.V[1] != 20 || h.Unit != "%" {
|
||||||
|
t.Fatalf("cpu: %+v %v", h, ok)
|
||||||
|
}
|
||||||
|
if h, ok := c.SensorHistory("net.eth0.rx"); !ok || len(h.V) != 1 || h.T[0] != 2 {
|
||||||
|
t.Fatalf("net: a sensor that appeared later has a shorter history, got %+v %v", h, ok)
|
||||||
|
}
|
||||||
|
if _, ok := c.SensorHistory("net.eth0.up"); ok {
|
||||||
|
t.Error("a state has no history")
|
||||||
|
}
|
||||||
|
if _, ok := c.SensorHistory("nope"); ok {
|
||||||
|
t.Error("unknown id")
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -0,0 +1,217 @@
|
|||||||
|
// Package sensors is the one standard way anything reads what Ham Lab measures.
|
||||||
|
//
|
||||||
|
// Every measurable thing is a Sensor: a stable dotted ID ("cpu.core.3.usage",
|
||||||
|
// "fan.nct6775.fan2", "fs.root.used_pct"), a little metadata, and a current value. A
|
||||||
|
// consumer discovers them from the catalog, then asks for exactly the ones it wants —
|
||||||
|
// one by ID, many by glob, or as a shaped query where the client sends the object it
|
||||||
|
// wants and gets it back with the values filled in. See SENSORS.md.
|
||||||
|
//
|
||||||
|
// Sources of sensors are Providers. The registry only joins them; it measures nothing.
|
||||||
|
package sensors
|
||||||
|
|
||||||
|
import (
|
||||||
|
"encoding/json"
|
||||||
|
"errors"
|
||||||
|
"fmt"
|
||||||
|
"path"
|
||||||
|
"sort"
|
||||||
|
"strings"
|
||||||
|
)
|
||||||
|
|
||||||
|
// Sensor is one measurable thing and its reading. Value is a float64, bool or string.
|
||||||
|
type Sensor struct {
|
||||||
|
ID string `json:"id"`
|
||||||
|
Name string `json:"name"` // human label
|
||||||
|
Kind string `json:"kind"` // usage | load | temperature | fan | rate | bytes | duration | voltage | power | state
|
||||||
|
Unit string `json:"unit,omitempty"` // % · °C · rpm · B/s · B · s · min · V · W · MHz
|
||||||
|
Value any `json:"value"`
|
||||||
|
T int64 `json:"t"` // unix ms the value was read — slow sensors are older than the tick
|
||||||
|
Min *float64 `json:"min,omitempty"`
|
||||||
|
Max *float64 `json:"max,omitempty"`
|
||||||
|
Warn *float64 `json:"warn,omitempty"` // the hardware's own thresholds, where it states them
|
||||||
|
Crit *float64 `json:"crit,omitempty"`
|
||||||
|
Labels map[string]string `json:"labels,omitempty"` // the real names behind the ID: chip, iface, mount…
|
||||||
|
}
|
||||||
|
|
||||||
|
// Series is a numeric sensor's recent past.
|
||||||
|
type Series struct {
|
||||||
|
ID string `json:"id"`
|
||||||
|
Unit string `json:"unit,omitempty"`
|
||||||
|
T []int64 `json:"t"`
|
||||||
|
V []float64 `json:"v"`
|
||||||
|
}
|
||||||
|
|
||||||
|
// Provider is a source of sensors. Sensors returns every sensor it has, with current values.
|
||||||
|
type Provider interface {
|
||||||
|
Sensors() []Sensor
|
||||||
|
}
|
||||||
|
|
||||||
|
// Historian is a Provider that also remembers. ok is false for an ID it doesn't hold.
|
||||||
|
type Historian interface {
|
||||||
|
SensorHistory(id string) (s Series, ok bool)
|
||||||
|
}
|
||||||
|
|
||||||
|
// F is a pointer to a float, for the optional bounds.
|
||||||
|
func F(v float64) *float64 { return &v }
|
||||||
|
|
||||||
|
// Seg makes one ID segment from a real-world name: lower case, [a-z0-9_-] only, so IDs
|
||||||
|
// survive URLs, query strings and templating languages untouched. "Package id 0" →
|
||||||
|
// "package_id_0", "/mnt/tank" → "mnt_tank", "/" → "root".
|
||||||
|
func Seg(name string) string {
|
||||||
|
if name == "/" {
|
||||||
|
return "root"
|
||||||
|
}
|
||||||
|
var b strings.Builder
|
||||||
|
gap := false
|
||||||
|
for _, r := range strings.ToLower(strings.TrimSpace(name)) {
|
||||||
|
switch {
|
||||||
|
case r >= 'a' && r <= 'z', r >= '0' && r <= '9', r == '-':
|
||||||
|
if gap && b.Len() > 0 {
|
||||||
|
b.WriteByte('_')
|
||||||
|
}
|
||||||
|
gap = false
|
||||||
|
b.WriteRune(r)
|
||||||
|
default:
|
||||||
|
gap = true
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if b.Len() == 0 {
|
||||||
|
return "unnamed"
|
||||||
|
}
|
||||||
|
return b.String()
|
||||||
|
}
|
||||||
|
|
||||||
|
// ID joins already-safe segments.
|
||||||
|
func ID(segs ...string) string { return strings.Join(segs, ".") }
|
||||||
|
|
||||||
|
// Registry joins providers into one namespace.
|
||||||
|
type Registry struct{ providers []Provider }
|
||||||
|
|
||||||
|
func NewRegistry(p ...Provider) *Registry { return &Registry{providers: p} }
|
||||||
|
|
||||||
|
func (r *Registry) Add(p Provider) { r.providers = append(r.providers, p) }
|
||||||
|
|
||||||
|
// All returns every sensor, sorted by ID. Two sensors that would share an ID (two
|
||||||
|
// identical chips) get "_2", "_3" on the colliding one, in the provider's order.
|
||||||
|
func (r *Registry) All() []Sensor {
|
||||||
|
var out []Sensor
|
||||||
|
seen := map[string]int{}
|
||||||
|
for _, p := range r.providers {
|
||||||
|
for _, s := range p.Sensors() {
|
||||||
|
seen[s.ID]++
|
||||||
|
if n := seen[s.ID]; n > 1 {
|
||||||
|
s.ID = fmt.Sprintf("%s_%d", s.ID, n)
|
||||||
|
}
|
||||||
|
out = append(out, s)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
sort.SliceStable(out, func(i, j int) bool { return out[i].ID < out[j].ID })
|
||||||
|
return out
|
||||||
|
}
|
||||||
|
|
||||||
|
// History finds the provider that remembers id.
|
||||||
|
func (r *Registry) History(id string) (Series, bool) {
|
||||||
|
for _, p := range r.providers {
|
||||||
|
if h, ok := p.(Historian); ok {
|
||||||
|
if s, ok := h.SensorHistory(id); ok {
|
||||||
|
return s, true
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return Series{}, false
|
||||||
|
}
|
||||||
|
|
||||||
|
// IsGlob reports whether a selector names many sensors rather than one.
|
||||||
|
func IsGlob(sel string) bool { return strings.ContainsAny(sel, "*?[") }
|
||||||
|
|
||||||
|
// Select returns the sensors a selector names: an exact ID, or a glob where * crosses
|
||||||
|
// dots ("fan.*" is every fan, "cpu.core.*.usage" every core).
|
||||||
|
func Select(all []Sensor, sel string) ([]Sensor, error) {
|
||||||
|
if !IsGlob(sel) {
|
||||||
|
for _, s := range all {
|
||||||
|
if s.ID == sel {
|
||||||
|
return []Sensor{s}, nil
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return nil, nil
|
||||||
|
}
|
||||||
|
if _, err := path.Match(sel, ""); err != nil {
|
||||||
|
return nil, fmt.Errorf("bad pattern %q", sel)
|
||||||
|
}
|
||||||
|
var out []Sensor
|
||||||
|
for _, s := range all {
|
||||||
|
if ok, _ := path.Match(sel, s.ID); ok {
|
||||||
|
out = append(out, s)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return out, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
// Shape answers a shaped query: the client sends the object it wants, with selectors
|
||||||
|
// where the values go, and gets the same object back with them filled in.
|
||||||
|
//
|
||||||
|
// {"cpu": "cpu.usage", "fans": "fan.*", "box": {"up": "uptime"}}
|
||||||
|
// → {"cpu": 12.4, "fans": {"fan.nct6775.fan1": 820}, "box": {"up": 80520}}
|
||||||
|
//
|
||||||
|
// An exact ID becomes its value (null if there is no such sensor here — machines differ,
|
||||||
|
// and a dashboard shouldn't fail because one has no fans). A glob becomes an object keyed
|
||||||
|
// by ID. Objects and arrays are walked. With full, every value is the whole Sensor.
|
||||||
|
func Shape(all []Sensor, query json.RawMessage, full bool) (any, error) {
|
||||||
|
var q any
|
||||||
|
if err := json.Unmarshal(query, &q); err != nil {
|
||||||
|
return nil, errors.New("query is not valid JSON")
|
||||||
|
}
|
||||||
|
return shape(all, q, full, 0)
|
||||||
|
}
|
||||||
|
|
||||||
|
func shape(all []Sensor, q any, full bool, depth int) (any, error) {
|
||||||
|
if depth > 16 {
|
||||||
|
return nil, errors.New("query nested too deep")
|
||||||
|
}
|
||||||
|
leaf := func(s Sensor) any {
|
||||||
|
if full {
|
||||||
|
return s
|
||||||
|
}
|
||||||
|
return s.Value
|
||||||
|
}
|
||||||
|
switch v := q.(type) {
|
||||||
|
case string:
|
||||||
|
got, err := Select(all, v)
|
||||||
|
if err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
if !IsGlob(v) {
|
||||||
|
if len(got) == 0 {
|
||||||
|
return nil, nil
|
||||||
|
}
|
||||||
|
return leaf(got[0]), nil
|
||||||
|
}
|
||||||
|
m := make(map[string]any, len(got))
|
||||||
|
for _, s := range got {
|
||||||
|
m[s.ID] = leaf(s)
|
||||||
|
}
|
||||||
|
return m, nil
|
||||||
|
case map[string]any:
|
||||||
|
m := make(map[string]any, len(v))
|
||||||
|
for k, sub := range v {
|
||||||
|
r, err := shape(all, sub, full, depth+1)
|
||||||
|
if err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
m[k] = r
|
||||||
|
}
|
||||||
|
return m, nil
|
||||||
|
case []any:
|
||||||
|
a := make([]any, len(v))
|
||||||
|
for i, sub := range v {
|
||||||
|
r, err := shape(all, sub, full, depth+1)
|
||||||
|
if err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
a[i] = r
|
||||||
|
}
|
||||||
|
return a, nil
|
||||||
|
default:
|
||||||
|
return nil, fmt.Errorf("a query holds sensor IDs, objects and arrays; got %v", v)
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -0,0 +1,53 @@
|
|||||||
|
package sensors
|
||||||
|
|
||||||
|
import (
|
||||||
|
"encoding/json"
|
||||||
|
"reflect"
|
||||||
|
"testing"
|
||||||
|
)
|
||||||
|
|
||||||
|
type fake []Sensor
|
||||||
|
|
||||||
|
func (f fake) Sensors() []Sensor { return f }
|
||||||
|
|
||||||
|
func TestSeg(t *testing.T) {
|
||||||
|
for in, want := range map[string]string{"/": "root", "/mnt/tank": "mnt_tank", "Package id 0": "package_id_0", "enp3s0": "enp3s0", " ": "unnamed", "Tctl+": "tctl", "nvme0n1": "nvme0n1"} {
|
||||||
|
if got := Seg(in); got != want {
|
||||||
|
t.Errorf("Seg(%q) = %q, want %q", in, got, want)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestRegistryDedupes(t *testing.T) {
|
||||||
|
r := NewRegistry(fake{{ID: "temp.nvme.composite", Value: 40.0}}, fake{{ID: "temp.nvme.composite", Value: 50.0}})
|
||||||
|
all := r.All()
|
||||||
|
if len(all) != 2 || all[0].ID != "temp.nvme.composite" || all[1].ID != "temp.nvme.composite_2" {
|
||||||
|
t.Fatalf("got %+v", all)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestShape(t *testing.T) {
|
||||||
|
all := []Sensor{{ID: "cpu.usage", Value: 12.5, Unit: "%"}, {ID: "fan.a.fan1", Value: 800.0}, {ID: "fan.a.fan2", Value: 900.0}, {ID: "net.eth0.up", Value: true}}
|
||||||
|
got, err := Shape(all, json.RawMessage(`{"cpu":"cpu.usage","fans":"fan.*","box":{"link":"net.eth0.up","gone":"ups.battery"},"list":["cpu.usage"]}`), false)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
want := map[string]any{
|
||||||
|
"cpu": 12.5,
|
||||||
|
"fans": map[string]any{"fan.a.fan1": 800.0, "fan.a.fan2": 900.0},
|
||||||
|
"box": map[string]any{"link": true, "gone": nil},
|
||||||
|
"list": []any{12.5},
|
||||||
|
}
|
||||||
|
if !reflect.DeepEqual(got, want) {
|
||||||
|
t.Fatalf("got %#v\nwant %#v", got, want)
|
||||||
|
}
|
||||||
|
full, _ := Shape(all, json.RawMessage(`"cpu.usage"`), true)
|
||||||
|
if s, ok := full.(Sensor); !ok || s.Unit != "%" {
|
||||||
|
t.Fatalf("full: got %#v", full)
|
||||||
|
}
|
||||||
|
for _, bad := range []string{`{"x": 5}`, `{"x": "fan.["}`, `not json`} {
|
||||||
|
if _, err := Shape(all, json.RawMessage(bad), false); err == nil {
|
||||||
|
t.Errorf("Shape(%s) should fail", bad)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
Reference in New Issue
Block a user