GPUs: fan speed in % per card (nvidia-smi fan.speed, hwmon pwm1); 0% shows

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

PLAN.md

@@ -16,7 +16,7 @@ stub tab or a roadmap item.
 | Service | `hamlab.service`, `User=core`, `Restart=always`, binary at `/usr/local/bin/hamlab` | Same pattern as `calorie-counter.service`. Runs as `core` so `sudo` works in the terminal without a password, as on the SSH path. |
 | Listen | `10.0.0.234:8100` (free) | NPM is a container, so loopback-only won't do. Firewalling to NPM is a later hardening step. |
 | Data | `/home/core/apps/hamlab/` — `config.yaml`, `state.db` (SQLite) | `apps/` is the nightly rsync target to `grid`, so it's backed up for free. |
-| Ingress | NPM proxy host `hamlab.bigham.dev` → `:8100`, LE cert, WebSockets on, caching off, force-SSL + HSTS + HTTP/2 set in a **second** API call (NPM drops them on create-with-new-cert) | `*.bigham.dev` already resolves; the proxy host alone makes it live. |
+| Ingress | NPM proxy host `lab.bigham.dev` → `:8100`, LE cert, WebSockets on, caching off, force-SSL + HSTS + HTTP/2 set in a **second** API call (NPM drops them on create-with-new-cert) | `*.bigham.dev` already resolves; the proxy host alone makes it live. |
 | Repo | Gitea `mitchell/hamlab`, private, `main` | Created 2026-09-17. |
 | Guest logins | A **guest** is an account like any other, with `guest: true` and an expiry: `login`, sessions and prefs need no special case, and **it may do everything an account may do**. The expiry is the whole feature — checked on every request and swept once a minute — with revoking as the manual version of it. | Built 2026-09-27. A read-only version was built first and **cut the same day**: Mitchell hands these to people he trusts and revokes them otherwise, so a console with half its tabs missing was solving a problem he doesn't have. Consequences, stated once and accepted: a guest has the Terminal tab, which is a root-capable shell on every host, and can make an account or token of its own that outlives the guest login. The one thing a guest is not is *an account*: it is left out of the Users list, out of "the last account can't be removed", and out of the single account an empty user name resolves to. |
 | Machines | **A view shows one machine: the one in the switcher.** No hub-side page lists or aggregates what its nodes are doing. To see or change anything on another machine, switch to it. | Mitchell, 2026-09-25, cutting a fleet-wide updates list out of the Docker tab. The switcher is the only place a machine is chosen, so there is exactly one answer to "which machine am I looking at" — a page that quietly reaches across breaks that, and it is never obvious which machine an action would land on. The hub still *proxies* for a node, and hub-only admin (Machines, Users, Settings, Alerts) is unaffected. |
@@ -37,7 +37,7 @@ stub tab or a roadmap item.
 │ Cron         │                                               │
 │ …            │                                               │
 ├──────────────┴───────────────────────────────────────────────┤
-│ footer: v0.1.0 · up 12d · hamlab.bigham.dev                   │
+│ footer: v0.1.0 · up 12d · lab.bigham.dev                   │
 └──────────────────────────────────────────────────────────────┘
 ```
 
@@ -116,7 +116,7 @@ hamlab/
 - Mobile drawer, key bar, fit-on-rotate.
 - `manifest.webmanifest`, icons (Ham logo on black, same recipe as HamTV), service
   worker caching the app shell only (never `/api`).
-- NPM proxy host for `hamlab.bigham.dev`; verify WS through NPM; force-SSL/HSTS/HTTP2.
+- NPM proxy host for `lab.bigham.dev`; verify WS through NPM; force-SSL/HSTS/HTTP2.
 - Stub groups for Services / Cron / ZFS / Dashboard.
 
 **Done when:** from the phone, open the PWA, log in, open two terminals, run

PORTABILITY.md

@@ -20,10 +20,10 @@ default or phones home to this lab · **C** cosmetic (placeholder text, comments
 | 1.1 | **B** | `internal/selfupdate/selfupdate.go:36` `DefaultURL = "https://git.bigham.dev/api/v1/repos/mitchell/hamlab"` | Every install with no `update.url` checks **this** Gitea 90 s after boot and daily (`internal/api/update.go:1951, 2016`). A stranger's install phones home here by default. | Default should be **off** (or a build-time `-X` variable set at release). Installer asks or `install.sh` passes `HAMLAB_URL` through into `update.url`. |
 | 1.2 | **B** | `install.sh:4, 27` `HAMLAB_URL` default; `README.md:23-25, 31` | The curl one-liner and the install source are this Gitea. | Default comes from the release that served the script (`@URL@` filled by `make release`), or is required. |
 | 1.3 | **B** | `scripts/gitea-release.sh:14` `REPO` default | Publishing goes to `mitchell/hamlab` on this Gitea. | Require `HAMLAB_REPO`; no default. |
-| 1.4 | **C** | `README.md:100` "On `core` it is public at `https://hamlab.bigham.dev` behind Nginx Proxy Manager" | Lab deployment fact in the product README. | Move to `hamlet/docs/hosts.md`. |
+| 1.4 | **C** | `README.md:100` "On `core` it is public at `https://lab.bigham.dev` behind Nginx Proxy Manager" | Lab deployment fact in the product README. | Move to `hamlet/docs/hosts.md`. |
 | 1.5 | **C** | `examples/external-demo/index.html:8` `base: 'http://10.0.0.234:8100'` | Demo config names the hub's LAN IP. | `http://<hamlab-host>:8100`. |
-| 1.6 | **C** | `SENSORS.md:149` Prometheus target `10.0.0.234:8100`; `:187` `&machine=grid`; `:196-208` Glance examples at `hamlab.bigham.dev`, `zfs.tank.used_pct`, `/api/nodes/grid/...`; `:218-220` `grid,wyse`, `{"core":…,"grid":…,"mac":…}` | Docs use real host names, pool and domain. | Generic placeholders (`<host>`, `<machine>`, `<pool>`). |
-| 1.7 | **C** | `PLAN.md:3-4` "bare-metal Debian 13 host. First target: `core` (10.0.0.234)"; decisions table rows (Listen `10.0.0.234:8100`, Data `/home/core/apps/hamlab/`, Ingress NPM/`hamlab.bigham.dev`, Repo) | Product plan is written as this lab's plan. | Keep as history, or split lab facts out to `hamlet`. The repo is public. |
+| 1.6 | **C** | `SENSORS.md:149` Prometheus target `10.0.0.234:8100`; `:187` `&machine=grid`; `:196-208` Glance examples at `lab.bigham.dev`, `zfs.tank.used_pct`, `/api/nodes/grid/...`; `:218-220` `grid,wyse`, `{"core":…,"grid":…,"mac":…}` | Docs use real host names, pool and domain. | Generic placeholders (`<host>`, `<machine>`, `<pool>`). |
+| 1.7 | **C** | `PLAN.md:3-4` "bare-metal Debian 13 host. First target: `core` (10.0.0.234)"; decisions table rows (Listen `10.0.0.234:8100`, Data `/home/core/apps/hamlab/`, Ingress NPM/`lab.bigham.dev`, Repo) | Product plan is written as this lab's plan. | Keep as history, or split lab facts out to `hamlet`. The repo is public. |
 | 1.9 | **C** | `BUGS.md` throughout | Bug bashes narrate `core`, `grid`, `wyse`, `hamtv`, `gate`, the Mac. | History; fine to leave, but it is public. |
 | 1.10 | **C** | `cmd/hamlab/main.go:1` "Proxmox-style web console for a bare-metal Debian host"; `internal/hardware/hardware.go:5` "what Debian and its kin already have"; `internal/hardware/hardware.go:29` `// "GMKtec NucBox G3"`; `internal/metrics/metrics.go:1358` "did, on grid, 2026-10-02"; `internal/api/server.go:1382-1383` `["grid", …]`, `?machines=grid,wyse`; `internal/api/stacks.go:10` `/home/core/apps/dockhand/stacks/…`; `internal/api/dockersched.go:5` "unbound, nginx and pihole"; `internal/api/zfs.go:5` "grid already runs zfSnap"; `internal/storage/snapshots.go:6` "grid has run zfSnap v1.11.1"; `internal/api/guests.go:8` "Mitchell, 2026-09-27"; `internal/network/network.go:306` "the public address here is a DHCP lease"; `internal/metrics/gpu.go:17` "Asahi, today"; `internal/metrics/pciids.go:4-6` "the router's GPU"; `web/src/Top.tsx:522, 576`; `web/src/hashsel.ts:1-9`; `web/src/Embed.tsx:3` `&machine=grid` | Comments explain decisions by naming lab hosts. | Harmless, but rewrite the ones in package doc comments so a reader of the source doesn't need the lab to understand the code. |
 

README.md

@@ -97,5 +97,5 @@ page flags machines running an older version and offers the same push per machin
 Mixed architectures are fine: `make deploy` also cross-compiles every other architecture,
 and the hub sends each machine the build for its own.
 
-- On `core` it is public at `https://hamlab.bigham.dev` behind Nginx Proxy Manager.
+- On `core` it is public at `https://lab.bigham.dev` behind Nginx Proxy Manager.
 - Installable as a PWA.

SENSORS.md

@@ -203,13 +203,13 @@ facts, never live. An `iframe` of the embed is live.
 ```yaml
 - type: iframe
   title: core CPU
-  source: https://hamlab.bigham.dev/embed/widget?type=gauge&sensor=cpu.usage&bare=1&token=<read-only token>
+  source: https://lab.bigham.dev/embed/widget?type=gauge&sensor=cpu.usage&bare=1&token=<read-only token>
   height: 130
 
 - type: custom-api
   title: grid
   cache: 1m
-  url: https://hamlab.bigham.dev/api/nodes/grid/sensors/query?pool=zfs.tank.used_pct&failed=services.failed&updates=updates.pending
+  url: https://lab.bigham.dev/api/nodes/grid/sensors/query?pool=zfs.tank.used_pct&failed=services.failed&updates=updates.pending
   headers: { Authorization: Bearer <read-only token> }
   template: |
     <div>tank {{ .JSON.Float "pool" }}% · {{ .JSON.Int "failed" }} failed · {{ .JSON.Int "updates" }} updates</div>

internal/metrics/cpupower.go

@@ -95,30 +95,41 @@ func readRAPL(root string, dirs []string) ([]raplCount, bool) {
 // is carried across its range. One that went backwards with no known range, or a set of
 // counters that changed between readings, gives no answer rather than a wrong one.
 func raplWatts(prev, cur []raplCount, max map[string]uint64, dt float64) (float64, bool) {
-	if dt <= 0 || len(prev) == 0 || len(prev) != len(cur) {
-		return 0, false
+	each, ok := raplWattsEach(prev, cur, max, dt)
+	var w float64
+	for _, x := range each {
+		w += x
 	}
-	var uj float64
+	return w, ok
+}
+
+// raplWattsEach is raplWatts per counter: one package each.
+func raplWattsEach(prev, cur []raplCount, max map[string]uint64, dt float64) ([]float64, bool) {
+	if dt <= 0 || len(prev) == 0 || len(prev) != len(cur) {
+		return nil, false
+	}
+	out := make([]float64, len(cur))
 	for i := range cur {
 		if cur[i].Dir != prev[i].Dir {
-			return 0, false
+			return nil, false
 		}
 		if cur[i].UJ >= prev[i].UJ {
-			uj += float64(cur[i].UJ - prev[i].UJ)
+			out[i] = float64(cur[i].UJ-prev[i].UJ) / 1e6 / dt
 			continue
 		}
 		m := max[cur[i].Dir]
 		if m == 0 || prev[i].UJ > m {
-			return 0, false
+			return nil, false
 		}
-		uj += float64(m - prev[i].UJ + cur[i].UJ)
+		out[i] = float64(m-prev[i].UJ+cur[i].UJ) / 1e6 / dt
 	}
-	return uj / 1e6 / dt, true
+	return out, true
 }
 
 // cpuPower is the CPU packages' power since the previous sample, in watts; 0 when this
-// machine has no counters or no way to read them.
-func (c *Collector) cpuPower(now time.Time) float64 {
+// machine has no counters or no way to read them. On a machine with more than one package
+// it also gives each one's, indexed by package id (the socket).
+func (c *Collector) cpuPower(now time.Time) (float64, []float64) {
 	p := &c.power
 	if !p.probed {
 		p.probed = true
@@ -131,7 +142,7 @@ func (c *Collector) cpuPower(now time.Time) float64 {
 		}
 	}
 	if len(p.dirs) == 0 {
-		return 0
+		return 0, nil
 	}
 	var cur []raplCount
 	ok := false
@@ -148,14 +159,40 @@ func (c *Collector) cpuPower(now time.Time) float64 {
 	}
 	if !ok {
 		p.prev = nil
-		return 0
+		return 0, nil
 	}
-	w, good := raplWatts(p.prev, cur, p.max, now.Sub(p.prevAt).Seconds())
+	each, good := raplWattsEach(p.prev, cur, p.max, now.Sub(p.prevAt).Seconds())
 	p.prev, p.prevAt = cur, now
 	if !good {
-		return 0
+		return 0, nil
 	}
-	return round1(w)
+	var w float64
+	for _, x := range each {
+		w += x
+	}
+	return round1(w), perPackage(cur, each)
+}
+
+// perPackage puts each counter's watts at its package's id — intel-rapl:1 is "package-1" —
+// and is nil for a single package, whose figure is the total.
+func perPackage(cur []raplCount, each []float64) []float64 {
+	if len(cur) < 2 {
+		return nil
+	}
+	var out []float64
+	for i, c := range cur {
+		id := i
+		if n, ok := strings.CutPrefix(strings.TrimSpace(readStr(filepath.Join(raplRoot, c.Dir, "name"))), "package-"); ok {
+			if v, err := strconv.Atoi(n); err == nil && v >= 0 && v < 64 {
+				id = v
+			}
+		}
+		for len(out) <= id {
+			out = append(out, 0)
+		}
+		out[id] = round1(each[i])
+	}
+	return out
 }
 
 // energyHelper is the root child that answers each newline with the package counters, one

internal/metrics/cputemp_test.go

@@ -0,0 +1,16 @@
+package metrics
+
+import "testing"
+
+// a two-socket board: two coretemp chips, and the CPU's temperature is the hotter package
+func TestCPUTempHottestPackage(t *testing.T) {
+	temps := []Temp{
+		{Chip: "coretemp", Label: "Core 0", C: 31},
+		{Chip: "coretemp", Label: "Core 0", C: 44, Pkg: 1},
+		{Chip: "coretemp", Label: "Package id 0", C: 35},
+		{Chip: "coretemp", Label: "Package id 1", C: 41, Pkg: 1},
+	}
+	if got := cpuTemp(temps); got != 41 {
+		t.Fatalf("cpuTemp = %v, want 41", got)
+	}
+}

internal/metrics/fold.go

@@ -1,5 +1,7 @@
 package metrics
 
+import "strconv"
+
 // One kept history point is the average of the samples behind it. At the default ten-minute
 // window that bucket is a single sample and nothing is averaged; at a day it is five minutes
 // of them, and the difference matters — a graph of the last day drawn from whichever instant
@@ -43,6 +45,21 @@ func fold(b []Sample) Sample {
 	if pn > 0 {
 		out.CPUPower = round1(pw / pn)
 	}
+	// and each package's, the same way
+	out.CPUPowerPkgs = nil
+	for _, s := range b {
+		for i, w := range s.CPUPowerPkgs {
+			for len(out.CPUPowerPkgs) <= i {
+				out.CPUPowerPkgs = append(out.CPUPowerPkgs, 0)
+			}
+			out.CPUPowerPkgs[i] += w
+		}
+	}
+	if pn > 0 {
+		for i := range out.CPUPowerPkgs {
+			out.CPUPowerPkgs[i] = round1(out.CPUPowerPkgs[i] / pn)
+		}
+	}
 	out.Load[0] = sum(func(s Sample) float64 { return s.Load[0] })
 	out.Load[1] = sum(func(s Sample) float64 { return s.Load[1] })
 	out.Load[2] = sum(func(s Sample) float64 { return s.Load[2] })
@@ -70,13 +87,13 @@ func fold(b []Sample) Sample {
 	temps := avgBy(b, func(s Sample) map[string][]float64 {
 		m := make(map[string][]float64, len(s.Temps))
 		for _, x := range s.Temps {
-			m[x.Chip+"/"+x.Label] = []float64{x.C}
+			m[x.Chip+"/"+x.Label+"/"+strconv.Itoa(x.Pkg)] = []float64{x.C}
 		}
 		return m
 	})
 	out.Temps = append([]Temp(nil), out.Temps...)
 	for i, x := range out.Temps {
-		if v, ok := temps[x.Chip+"/"+x.Label]; ok {
+		if v, ok := temps[x.Chip+"/"+x.Label+"/"+strconv.Itoa(x.Pkg)]; ok {
 			out.Temps[i].C = v[0]
 		}
 	}

internal/metrics/gpu.go

@@ -49,6 +49,7 @@ type GPU struct {
 	PowerW     float64            `json:"power_w,omitempty"`
 	PowerCapW  float64            `json:"power_cap_w,omitempty"`
 	FanRPM     int                `json:"fan_rpm,omitempty"`
+	FanPct     *float64           `json:"fan_pct,omitempty"` // fan speed as a % of its range: nvidia-smi's fan.speed, or the hwmon's pwm1; nil when the card has no such reading (0 is a stopped fan)
 	MHz        float64            `json:"mhz,omitempty"`     // actual clock (0 when power-gated)
 	ReqMHz     float64            `json:"req_mhz,omitempty"` // what the driver asked for
 	MaxMHz     float64            `json:"max_mhz,omitempty"`
@@ -56,7 +57,7 @@ type GPU struct {
 	MemTotal   uint64             `json:"mem_total,omitempty"`
 	Clients    []GPUClient        `json:"clients"`
 	ClientsOK  bool               `json:"clients_ok"`        // false when fdinfo of other users' processes could not be read
-	Stalled    bool               `json:"stalled,omitempty"` // a sysfs read of this card never returned; its readings are paused until restart
+	Stalled    bool               `json:"stalled,omitempty"` // a read of this card has been out for gpuStallAfter; its readings are paused until it returns
 }
 
 // GPUClient is one process with the device open.
@@ -78,7 +79,17 @@ type gpuDev struct {
 	integrated, stalled                      bool
 	prevIdle, prevEnergy                     float64
 	prevAt                                   time.Time
-	nv                                       []GPUClient // nvidia: the process table from the last read (pmon), fdinfo says nothing there
+	nv                                       []GPUClient  // nvidia: the process table from the last read (pmon), fdinfo says nothing there
+	nvRead                                   []GPUClient  // the same, as the read in flight left it: only that read touches it
+	pending                                  chan gpuRead // the read in flight, nil when none is
+	pendingAt                                time.Time
+	last                                     *GPU                  // the last read that came back
+	readFn                                   func(*GPU, time.Time) // nil is read (tests stand a slow one in)
+}
+
+type gpuRead struct {
+	x  GPU
+	nv []GPUClient
 }
 
 type gpuState struct {
@@ -348,28 +359,60 @@ func lspciName(desc string) string {
 	return strings.TrimSpace(vendor + " " + desc)
 }
 
-// readWithin is read with a deadline. A sysfs read that does not come back must not take
-// the whole sampler with it — every graph on every page of this machine stops — so the
-// device is marked stalled, its goroutine abandoned, and the rest of the sample goes on.
-const gpuReadWithin = 3 * time.Second
+// readWithin is read with a deadline. A read that does not come back must not take the
+// whole sampler with it — every graph on every page of this machine stops — so one still out
+// after gpuReadWithin is left running, the sample goes on with the card's last readings, and
+// no second read starts until it returns. Late is normal for nvidia-smi on a busy card; only a
+// read out for gpuStallAfter marks the card stalled, and that clears the moment it returns. A
+// sysfs read that never returns keeps it stalled, at the cost of one goroutine.
+var (
+	gpuReadWithin = 3 * time.Second
+	gpuStallAfter = 15 * time.Second // past both of readNvidia's nvidia-smi calls timing out (runOut, 5 s each)
+)
 
 func (d *gpuDev) readWithin(x *GPU, now time.Time) {
-	if d.stalled {
-		x.Stalled = true
+	var r gpuRead
+	back := false
+	if d.pending == nil {
+		done := make(chan gpuRead, 1)
+		y := *x
+		read := d.read
+		if d.readFn != nil {
+			read = d.readFn
+		}
+		go func() {
+			read(&y, now)
+			done <- gpuRead{y, d.nvRead}
+		}()
+		d.pending, d.pendingAt = done, now
+		select {
+		case r, back = <-done:
+		case <-time.After(gpuReadWithin):
+		}
+	} else { // started on an earlier sample: taken if it is back, never waited on again
+		select {
+		case r, back = <-d.pending:
+		default:
+		}
+	}
+	ok := x.ClientsOK
+	defer func() { x.ClientsOK = ok }()
+	if back {
+		if d.stalled {
+			log.Printf("gpu: %s (%s): its read came back after %s; readings resume", d.name, d.card, now.Sub(d.pendingAt).Round(time.Second))
+		}
+		d.pending, d.nv, d.stalled, d.last = nil, r.nv, false, &r.x
+		*x = r.x
 		return
 	}
-	done := make(chan GPU, 1)
-	go func() {
-		y := *x
-		d.read(&y, now)
-		done <- y
-	}()
-	select {
-	case y := <-done:
-		*x = y
-	case <-time.After(gpuReadWithin):
+	if d.last != nil {
+		*x = *d.last
+	}
+	if now.Sub(d.pendingAt) >= gpuStallAfter {
+		if !d.stalled {
+			log.Printf("gpu: %s (%s): a read has not returned in %s; its readings are paused until it does", d.name, d.card, gpuStallAfter)
+		}
 		d.stalled, x.Stalled = true, true
-		log.Printf("gpu: %s (%s): a sysfs read did not return within %s; its readings are paused until restart", d.name, d.card, gpuReadWithin)
 	}
 }
 
@@ -437,6 +480,14 @@ func (d *gpuDev) read(x *GPU, now time.Time) {
 			x.PowerCapW = p / 1e6
 		}
 		x.FanRPM = int(readF(d.hwmon + "/fan1_input"))
+		if pwm, err := strconv.ParseFloat(strings.TrimSpace(readStr(d.hwmon+"/pwm1")), 64); err == nil {
+			top := readF(d.hwmon + "/pwm1_max")
+			if top <= 0 {
+				top = 255
+			}
+			v := round1(pwm / top * 100)
+			x.FanPct = &v
+		}
 	}
 	d.prevAt = now
 }
@@ -720,11 +771,11 @@ func ueventValue(p, key string) string {
 // sample (~20 ms and ~75 ms on a GTX 965M), no root needed. The PCI address selects the
 // card, so two NVIDIA GPUs stay apart.
 func (d *gpuDev) readNvidia(x *GPU) {
-	if out, ok := runOut("nvidia-smi", "--id="+d.bus, "--query-gpu=utilization.gpu,utilization.encoder,utilization.decoder,memory.used,memory.total,clocks.sm,clocks.max.sm,temperature.gpu,power.draw,power.limit", "--format=csv,noheader,nounits"); ok {
+	if out, ok := runOut("nvidia-smi", "--id="+d.bus, "--query-gpu=utilization.gpu,utilization.encoder,utilization.decoder,memory.used,memory.total,clocks.sm,clocks.max.sm,temperature.gpu,power.draw,power.limit,fan.speed", "--format=csv,noheader,nounits"); ok {
 		parseNvidiaQuery(x, out)
 	}
 	if out, ok := runOut("nvidia-smi", "pmon", "-c", "1", "-s", "um", "--id="+d.bus); ok {
-		d.nv = parseNvidiaPmon(out)
+		d.nvRead = parseNvidiaPmon(out)
 	}
 }
 
@@ -739,7 +790,8 @@ func nvNum(s string) (float64, bool) {
 }
 
 // parseNvidiaQuery fills a GPU from one --query-gpu line in the field order readNvidia asks
-// for: gpu%, encoder%, decoder%, mem used MiB, mem total MiB, SM MHz, max SM MHz, °C, W, W cap.
+// for: gpu%, encoder%, decoder%, mem used MiB, mem total MiB, SM MHz, max SM MHz, °C, W, W cap,
+// fan % ("[N/A]" on a card with no fan of its own: a laptop's, a passive datacentre card).
 func parseNvidiaQuery(x *GPU, line string) {
 	f := strings.Split(strings.TrimSpace(line), ",")
 	if len(f) < 10 {
@@ -775,6 +827,11 @@ func parseNvidiaQuery(x *GPU, line string) {
 	if v, ok := nvNum(f[9]); ok {
 		x.PowerCapW = v
 	}
+	if len(f) > 10 {
+		if v, ok := nvNum(f[10]); ok {
+			x.FanPct = &v
+		}
+	}
 }
 
 // parseNvidiaPmon reads `nvidia-smi pmon -c 1 -s um`: a header naming the columns, then one

internal/metrics/gpu_stall_test.go

@@ -0,0 +1,52 @@
+package metrics
+
+import (
+	"testing"
+	"time"
+)
+
+// A read that comes back late carries the last readings meanwhile and is used when it lands;
+// only one out past gpuStallAfter marks the card stalled, and its return clears that.
+func TestGPUReadLateThenStalledThenBack(t *testing.T) {
+	defer func(w, s time.Duration) { gpuReadWithin, gpuStallAfter = w, s }(gpuReadWithin, gpuStallAfter)
+	gpuReadWithin, gpuStallAfter = 20*time.Millisecond, 10*time.Second
+	release := make(chan struct{})
+	n := 0.0 // only the read goroutine writes it, one read at a time
+	d := &gpuDev{name: "test", card: "card0"}
+	d.readFn = func(x *GPU, _ time.Time) {
+		n++
+		if n == 2 {
+			<-release
+		}
+		x.Usage = n * 10
+	}
+	t0 := time.Now()
+	sample := func(at time.Duration) GPU {
+		x := GPU{Name: "test"}
+		d.readWithin(&x, t0.Add(at))
+		return x
+	}
+	if x := sample(0); x.Usage != 10 || x.Stalled {
+		t.Fatalf("first: %+v", x)
+	}
+	if x := sample(time.Second); x.Usage != 10 || x.Stalled { // late: the last readings
+		t.Fatalf("late: %+v", x)
+	}
+	if x := sample(5 * time.Second); x.Usage != 10 || x.Stalled {
+		t.Fatalf("still late: %+v", x)
+	}
+	if x := sample(12 * time.Second); !x.Stalled {
+		t.Fatalf("out past gpuStallAfter, not stalled: %+v", x)
+	}
+	if x := sample(12*time.Second + 1); !x.Stalled {
+		t.Fatal("stall did not hold")
+	}
+	close(release)
+	time.Sleep(10 * time.Millisecond)
+	if x := sample(13 * time.Second); x.Usage != 20 || x.Stalled {
+		t.Fatalf("back: %+v", x)
+	}
+	if x := sample(14 * time.Second); x.Usage != 30 || x.Stalled { // 30: no extra read ran while one was out
+		t.Fatalf("next: %+v", x)
+	}
+}

internal/metrics/gpu_test.go

@@ -182,6 +182,20 @@ func TestParseNvidiaQuery(t *testing.T) {
 	if g.Temp != 57 || g.TempSrc != "gpu" || g.PowerW != 23.4 || g.PowerCapW != 0 {
 		t.Errorf("temp/power (cap must stay 0 for [N/A]): %+v", g)
 	}
+	if g.FanPct != nil {
+		t.Errorf("no fan field must leave the fan unset: %+v", g)
+	}
+	var fan, nofan GPU
+	parseNvidiaQuery(&fan, "99, 0, 0, 20000, 24576, 1695, 2100, 83, 403.99, 420.00, 34\n")
+	parseNvidiaQuery(&nofan, "5, 0, 0, 500, 2048, 300, 1097, 45, 8.0, [N/A], [N/A]\n")
+	if fan.FanPct == nil || *fan.FanPct != 34 || nofan.FanPct != nil {
+		t.Errorf("fan: %v, [N/A]: %v", fan.FanPct, nofan.FanPct)
+	}
+	var stopped GPU
+	parseNvidiaQuery(&stopped, "0, 0, 0, 500, 24576, 210, 2100, 35, 20.0, 420.00, 0\n")
+	if stopped.FanPct == nil || *stopped.FanPct != 0 {
+		t.Errorf("a stopped fan must read 0, not nothing: %v", stopped.FanPct)
+	}
 	var short GPU
 	parseNvidiaQuery(&short, "garbage")
 	if short.UsageSrc != "" {

internal/metrics/metrics.go

@@ -60,6 +60,7 @@ type Temp struct {
 	C     float64 `json:"c"`
 	Max   float64 `json:"max,omitempty"`
 	Crit  float64 `json:"crit,omitempty"`
+	Pkg   int     `json:"pkg,omitempty"` // coretemp: the CPU package the chip belongs to — a two-socket board has two chips, both with a "Core 0"
 }
 
 type Fan struct {
@@ -101,6 +102,9 @@ type FS struct {
 	// said honestly: a ZFS pool or an md array spreads a filesystem over several drives, and
 	// a remote mount is on none of them.
 	Disk string `json:"disk,omitempty"`
+	// Source is the device the mount is of ("/dev/nvme0n1p1"). A bind mount has the same
+	// source as the filesystem it shows, so a sum over a drive counts each source once.
+	Source string `json:"source,omitempty"`
 }
 
 type UPS struct {
@@ -135,24 +139,25 @@ type Proc struct {
 }
 
 type Sample struct {
-	T        int64      `json:"t"`           // unix ms
-	Interval int64      `json:"interval_ms"` // sampling interval in effect
-	CPU      float64    `json:"cpu"`
-	Cores    []float64  `json:"cores"`
-	CPUPower float64    `json:"cpu_power_w,omitempty"` // the CPU packages' own energy counters; 0 = no reading
-	Load     [3]float64 `json:"load"`
-	Mem      Mem        `json:"mem"`
-	Swap     Mem        `json:"swap"`
-	Temps    []Temp     `json:"temps"`
-	Fans     []Fan      `json:"fans"`
-	Net      []Net      `json:"net"`
-	Disks    []Disk     `json:"disks"`
-	FS       []FS       `json:"fs"`
-	UPS      *UPS       `json:"ups"`
-	GPUs     []GPU      `json:"gpus"`
-	ARC      *ARC       `json:"arc"`        // the ZFS read cache; nil without the module
-	ZFSWrite *ZFSWrites `json:"zfs_writes"` // the ZFS write path; nil without the module
-	Uptime   float64    `json:"uptime"`
+	T            int64      `json:"t"`           // unix ms
+	Interval     int64      `json:"interval_ms"` // sampling interval in effect
+	CPU          float64    `json:"cpu"`
+	Cores        []float64  `json:"cores"`
+	CPUPower     float64    `json:"cpu_power_w,omitempty"`      // the CPU packages' own energy counters; 0 = no reading
+	CPUPowerPkgs []float64  `json:"cpu_power_pkgs_w,omitempty"` // each package's share, by socket; only with more than one
+	Load         [3]float64 `json:"load"`
+	Mem          Mem        `json:"mem"`
+	Swap         Mem        `json:"swap"`
+	Temps        []Temp     `json:"temps"`
+	Fans         []Fan      `json:"fans"`
+	Net          []Net      `json:"net"`
+	Disks        []Disk     `json:"disks"`
+	FS           []FS       `json:"fs"`
+	UPS          *UPS       `json:"ups"`
+	GPUs         []GPU      `json:"gpus"`
+	ARC          *ARC       `json:"arc"`        // the ZFS read cache; nil without the module
+	ZFSWrite     *ZFSWrites `json:"zfs_writes"` // the ZFS write path; nil without the module
+	Uptime       float64    `json:"uptime"`
 
 	fsT, upsT int64 // when the slow sections were really read; sensors report these, not T
 }
@@ -349,7 +354,7 @@ func (c *Collector) sample() {
 
 	s := Sample{T: now.UnixMilli(), Interval: c.Interval().Milliseconds(), Temps: []Temp{}, Fans: []Fan{}, Net: []Net{}, Disks: []Disk{}}
 	s.CPU, s.Cores = c.cpu()
-	s.CPUPower = c.cpuPower(now)
+	s.CPUPower, s.CPUPowerPkgs = c.cpuPower(now)
 	s.Load = loadavg()
 	s.Mem, s.Swap = meminfo()
 	s.Uptime = uptime()
@@ -544,6 +549,19 @@ func hwmonDir(d string) (chip string, temps []Temp, fans []Fan) {
 			fans = append(fans, Fan{Chip: chip, Label: label, RPM: int(raw)})
 		}
 	}
+	// coretemp is one chip per CPU package, and only its "Package id N" says which
+	if chip == "coretemp" {
+		for _, t := range temps {
+			if n, ok := strings.CutPrefix(t.Label, "Package id "); ok {
+				if pkg, err := strconv.Atoi(n); err == nil {
+					for i := range temps {
+						temps[i].Pkg = pkg
+					}
+				}
+				break
+			}
+		}
+	}
 	return
 }
 
@@ -559,7 +577,10 @@ func hwmon() ([]Temp, []Fan) {
 		if temps[i].Chip != temps[j].Chip {
 			return temps[i].Chip < temps[j].Chip
 		}
-		return temps[i].Label < temps[j].Label
+		if temps[i].Label != temps[j].Label {
+			return temps[i].Label < temps[j].Label
+		}
+		return temps[i].Pkg < temps[j].Pkg
 	})
 	return temps, fans
 }
@@ -845,7 +866,7 @@ func filesystems() []FS {
 				return
 			}
 			bs := uint64(st.Bsize)
-			done <- FS{Mount: mount, Type: fs[2], Size: st.Blocks * bs, Used: (st.Blocks - st.Bfree) * bs, Remote: remoteFS[fs[2]], Disk: diskOf(fs[0])}
+			done <- FS{Mount: mount, Type: fs[2], Size: st.Blocks * bs, Used: (st.Blocks - st.Bfree) * bs, Remote: remoteFS[fs[2]], Disk: diskOf(fs[0]), Source: fs[0]}
 		}()
 		select {
 		case r := <-done:
@@ -1112,27 +1133,61 @@ func coreIDs(n int) []int {
 	return coreIDsVal[:n]
 }
 
+// corePkgs maps logical CPUs to the CPU package (socket) each is on, and is nil on a
+// one-socket machine: there is nothing to tell apart.
+var (
+	corePkgsOnce sync.Once
+	corePkgsVal  []int
+)
+
+func corePkgs(n int) []int {
+	corePkgsOnce.Do(func() {
+		seen := map[int]bool{}
+		var out []int
+		for i := 0; ; i++ {
+			b, err := os.ReadFile("/sys/devices/system/cpu/cpu" + strconv.Itoa(i) + "/topology/physical_package_id")
+			if err != nil {
+				break
+			}
+			id, err := strconv.Atoi(strings.TrimSpace(string(b)))
+			if err != nil {
+				break
+			}
+			out, seen[id] = append(out, id), true
+		}
+		if len(seen) > 1 {
+			corePkgsVal = out
+		}
+	})
+	if len(corePkgsVal) < n {
+		return nil
+	}
+	return corePkgsVal[:n]
+}
+
 // ProcTable is the Processes tab payload.
 type ProcTable struct {
-	T         int64      `json:"t"`
-	CPU       float64    `json:"cpu"`
-	Cores     []float64  `json:"cores"`
-	CPUHist   []float64  `json:"cpu_hist"` // recent total-CPU samples, oldest first
-	CPUModel  string     `json:"cpu_model,omitempty"`
-	CPUMHz    float64    `json:"cpu_mhz,omitempty"`
-	CPUMaxMHz float64    `json:"cpu_max_mhz,omitempty"`
-	CPUPowerW float64    `json:"cpu_power_w,omitempty"`
-	CPUTemp   float64    `json:"cpu_temp,omitempty"`
-	CoreIDs   []int      `json:"core_ids,omitempty"` // for each logical CPU, the physical core it is a thread of (what the temperature sensors are named by)
-	Load      [3]float64 `json:"load"`
-	Mem       Mem        `json:"mem"`
-	Swap      Mem        `json:"swap"`
-	Net       []Net      `json:"net"`
-	GPUs      []GPU      `json:"gpus"`
-	Uptime    float64    `json:"uptime"`
-	Procs     []Proc     `json:"procs"`
-	Total     int        `json:"total"`
-	Running   int        `json:"running"`
+	T             int64      `json:"t"`
+	CPU           float64    `json:"cpu"`
+	Cores         []float64  `json:"cores"`
+	CPUHist       []float64  `json:"cpu_hist"` // recent total-CPU samples, oldest first
+	CPUModel      string     `json:"cpu_model,omitempty"`
+	CPUMHz        float64    `json:"cpu_mhz,omitempty"`
+	CPUMaxMHz     float64    `json:"cpu_max_mhz,omitempty"`
+	CPUPowerW     float64    `json:"cpu_power_w,omitempty"`
+	CPUPowerPkgsW []float64  `json:"cpu_power_pkgs_w,omitempty"`
+	CPUTemp       float64    `json:"cpu_temp,omitempty"`
+	CoreIDs       []int      `json:"core_ids,omitempty"`  // for each logical CPU, the physical core it is a thread of (what the temperature sensors are named by)
+	CorePkgs      []int      `json:"core_pkgs,omitempty"` // for each logical CPU, the socket it is on; only on a machine with more than one
+	Load          [3]float64 `json:"load"`
+	Mem           Mem        `json:"mem"`
+	Swap          Mem        `json:"swap"`
+	Net           []Net      `json:"net"`
+	GPUs          []GPU      `json:"gpus"`
+	Uptime        float64    `json:"uptime"`
+	Procs         []Proc     `json:"procs"`
+	Total         int        `json:"total"`
+	Running       int        `json:"running"`
 }
 
 var cpuModel = func() string {
@@ -1182,18 +1237,24 @@ func cpuMHz() float64 {
 
 // cpuTemp picks the sensor that best represents the CPU package.
 func cpuTemp(temps []Temp) float64 {
-	best, max := 0.0, 0.0
+	// the hottest package: a two-socket board has two, and the hotter one is the one to watch
+	pkg, max := 0.0, 0.0
 	for _, t := range temps {
 		l := strings.ToLower(t.Label)
 		if t.Chip == "coretemp" && strings.HasPrefix(l, "package") || t.Chip == "k10temp" && (l == "tctl" || l == "tdie") ||
 			t.Chip == "thermal" && strings.Contains(l, "cpu") || t.Chip == "cpu_thermal" || t.Chip == "soc_thermal" {
-			return t.C
+			if t.C > pkg {
+				pkg = t.C
+			}
 		}
 		if (t.Chip == "coretemp" || t.Chip == "k10temp") && t.C > max {
-			max, best = t.C, t.C
+			max = t.C
 		}
 	}
-	return best
+	if pkg > 0 {
+		return pkg
+	}
+	return max
 }
 
 var bootTime = func() int64 {
@@ -1321,8 +1382,9 @@ func (c *Collector) Processes() ProcTable {
 	t.Mem, t.Swap = meminfo()
 	if n := len(c.ring); n > 0 {
 		last := c.ring[n-1]
-		t.CPU, t.Cores, t.Net, t.CPUTemp, t.CPUPowerW = last.CPU, last.Cores, last.Net, cpuTemp(last.Temps), last.CPUPower
+		t.CPU, t.Cores, t.Net, t.CPUTemp, t.CPUPowerW, t.CPUPowerPkgsW = last.CPU, last.Cores, last.Net, cpuTemp(last.Temps), last.CPUPower, last.CPUPowerPkgs
 		t.CoreIDs = coreIDs(len(last.Cores))
+		t.CorePkgs = corePkgs(len(last.Cores))
 		from := n - 90
 		if from < 0 {
 			from = 0

internal/metrics/sensors.go

@@ -65,6 +65,10 @@ func flatten(s Sample, want func(id string) bool) []sensors.Sensor {
 	if s.CPUPower > 0 {
 		f.add("cpu.power", "CPU power", "power", "W", s.CPUPower, t, none)
 	}
+	for i, w := range s.CPUPowerPkgs {
+		n := strconv.Itoa(i)
+		f.add("cpu.package."+n+".power", "CPU "+strconv.Itoa(i+1)+" power", "power", "W", w, t, none, "package", n)
+	}
 	for i, w := range []string{"1", "5", "15"} {
 		f.add("load."+w, "Load "+w+" min", "load", "", s.Load[i], t, none)
 	}
@@ -92,7 +96,11 @@ func flatten(s Sample, want func(id string) bool) []sensors.Sensor {
 		if x.Crit > 0 {
 			b.crit = sensors.F(x.Crit)
 		}
-		f.add(sensors.ID("temp", sensors.Seg(x.Chip), sensors.Seg(x.Label)), x.Chip+" "+x.Label, "temperature", "°C", x.C, t, b, "chip", x.Chip, "label", x.Label)
+		labels := []string{"chip", x.Chip, "label", x.Label}
+		if x.Chip == "coretemp" { // a two-socket board has a coretemp chip per CPU, both with a "Core 0"
+			labels = append(labels, "package", strconv.Itoa(x.Pkg))
+		}
+		f.add(sensors.ID("temp", sensors.Seg(x.Chip), sensors.Seg(x.Label)), x.Chip+" "+x.Label, "temperature", "°C", x.C, t, b, labels...)
 	}
 	for _, x := range s.Fans {
 		f.add(sensors.ID("fan", sensors.Seg(x.Chip), sensors.Seg(x.Label)), x.Chip+" "+x.Label, "fan", "rpm", float64(x.RPM), t, none, "chip", x.Chip, "label", x.Label)
@@ -129,13 +137,22 @@ func flatten(s Sample, want func(id string) bool) []sensors.Sensor {
 		if x.Disk != "" {
 			lbl = append(lbl, "disk", x.Disk)
 		}
+		if x.Source != "" {
+			lbl = append(lbl, "source", x.Source)
+		}
 		f.add(p+".size", x.Mount+" size", "bytes", "B", float64(x.Size), s.fsT, none, lbl...)
 		f.add(p+".used", x.Mount+" used", "bytes", "B", float64(x.Used), s.fsT, upTo(float64(x.Size)), lbl...)
 		f.add(p+".used_pct", x.Mount+" used", "usage", "%", ratio(x.Used, x.Size), s.fsT, pct, lbl...)
 	}
 
+	gpuSeen := map[string]int{}
 	for _, g := range s.GPUs {
+		// two identical cards are gpu.<name> and gpu.<name>_2, every sensor of the second under
+		// its own stem (not add's suffix on the end of each id, which the page can't find)
 		p := "gpu." + sensors.Seg(g.Name)
+		if gpuSeen[p]++; gpuSeen[p] > 1 {
+			p = fmt.Sprintf("%s_%d", p, gpuSeen[p])
+		}
 		lbl := []string{"name", g.Name, "driver", g.Driver}
 		if g.Bus != "" {
 			lbl = append(lbl, "bus", g.Bus)
@@ -164,6 +181,9 @@ func flatten(s Sample, want func(id string) bool) []sensors.Sensor {
 		if g.FanRPM > 0 {
 			f.add(p+".fan", g.Name+" fan", "fan", "rpm", float64(g.FanRPM), t, none, lbl...)
 		}
+		if g.FanPct != nil {
+			f.add(p+".fan_pct", g.Name+" fan", "fan", "%", *g.FanPct, t, pct, lbl...)
+		}
 		if g.MaxMHz > 0 {
 			f.add(p+".freq", g.Name+" clock", "frequency", "MHz", g.MHz, t, upTo(g.MaxMHz), lbl...)
 		}

web/package.json

@@ -7,7 +7,7 @@
     "dev": "vite",
     "build": "tsc --noEmit && vite build",
     "icons": "node scripts/icons.mjs",
-    "test": "node scripts/test-graphcols.mjs && node scripts/test-css-imports.mjs && node scripts/test-resolve-path.mjs"
+    "test": "node scripts/test-graphcols.mjs && node scripts/test-sockets.mjs && node scripts/test-css-imports.mjs && node scripts/test-resolve-path.mjs"
   },
   "dependencies": {
     "@xterm/addon-fit": "^0.10.0",

web/scripts/test-sockets.mjs

@@ -0,0 +1,34 @@
+// node scripts/test-sockets.mjs — src/sockets.ts: each CPU's threads, counted under it.
+import { buildSync } from 'esbuild'
+import { pathToFileURL } from 'node:url'
+import { mkdtempSync } from 'node:fs'
+import { tmpdir } from 'node:os'
+import { join } from 'node:path'
+
+const out = join(mkdtempSync(join(tmpdir(), 'sockets-')), 'sockets.mjs')
+buildSync({ entryPoints: [new URL('../src/sockets.ts', import.meta.url).pathname], outfile: out, format: 'esm', logLevel: 'error' })
+const { socketThreads } = await import(pathToFileURL(out).href)
+
+const fails = []
+const eq = (got, want, what) => { if (JSON.stringify(got) !== JSON.stringify(want)) fails.push(`${what} — got ${JSON.stringify(got)}, want ${JSON.stringify(want)}`) }
+const range = (a, b) => Array.from({ length: b - a }, (_, i) => a + i)
+
+// nexus: 2 × E5-2680 v2, as /sys/devices/system/cpu/cpu*/topology/physical_package_id reads
+const nexus = [...Array(10).fill(0), ...Array(10).fill(1), ...Array(10).fill(0), ...Array(10).fill(1)]
+const got = socketThreads(nexus)
+eq(got.map((s) => s.pkg), [0, 1], 'two sockets, in order')
+eq(got[0].cpus, [...range(0, 10), ...range(20, 30)], "CPU 1's threads")
+eq(got[1].cpus, [...range(10, 20), ...range(30, 40)], "CPU 2's threads")
+eq(got.map((s) => s.cpus.length), [20, 20], 'twenty threads each')
+// every thread is under exactly one CPU
+eq(got.flatMap((s) => s.cpus).sort((a, b) => a - b), range(0, 40), 'every thread once')
+// a thread's number under its CPU runs 0…19 on both
+for (const s of got) eq(s.cpus.map((_, n) => n), range(0, 20), `CPU ${s.pkg + 1} counts from 0`)
+
+// a board that numbers each socket's threads together, and one whose socket ids are not 0 and 1
+eq(socketThreads([0, 0, 0, 0, 1, 1, 1, 1]), [{ pkg: 0, cpus: [0, 1, 2, 3] }, { pkg: 1, cpus: [4, 5, 6, 7] }], 'contiguous numbering')
+eq(socketThreads([3, 1, 3, 1]), [{ pkg: 1, cpus: [1, 3] }, { pkg: 3, cpus: [0, 2] }], 'socket ids sorted')
+eq(socketThreads([]), [], 'nothing')
+
+if (fails.length) { console.error('sockets: FAIL\n  ' + fails.join('\n  ')); process.exit(1) }
+console.log('sockets: ok')

web/src/Hardware.tsx

@@ -158,6 +158,10 @@ export default function Hardware() {
   const capacity = drives.reduce((n, d) => n + d.size, 0)
   const threads = cpu.threads ?? 0
   const cores = cpu.cores ?? 0
+  // a two-socket board is two processors, not one with twice the cores: count and name each
+  const sockets = Math.max(1, cpu.sockets ?? 1)
+  const multi = sockets > 1
+  const per = (n: number) => Math.round(n / sockets)
 
   return (
     <div className="dash">
@@ -171,8 +175,11 @@ export default function Hardware() {
       <div className="kpis">
         <div className="tile">
           <div className="label">Processor</div>
-          <div className="value" title={cpu.model} style={(cpu.model?.length ?? 0) > 20 ? { fontSize: 16 } : undefined}>{cpu.model}</div>
-          <div className="sub">{[cores ? `${cores} core${cores === 1 ? '' : 's'}` : '', threads && threads !== cores ? `${threads} threads` : '', cpu.max_mhz ? `up to ${ghz(cpu.max_mhz)}` : ''].filter(Boolean).join(' · ')}</div>
+          <div className="value" title={cpu.model} style={(cpu.model?.length ?? 0) > (multi ? 16 : 20) ? { fontSize: 16 } : undefined}>{multi ? `${sockets} × ${cpu.model}` : cpu.model}</div>
+          <div className="sub">{(multi
+            ? [`${sockets} CPUs`, cores ? `${per(cores)} cores${threads && threads !== cores ? ` / ${per(threads)} threads` : ''} each` : '', cpu.max_mhz ? `up to ${ghz(cpu.max_mhz)}` : '']
+            : [cores ? `${cores} core${cores === 1 ? '' : 's'}` : '', threads && threads !== cores ? `${threads} threads` : '', cpu.max_mhz ? `up to ${ghz(cpu.max_mhz)}` : '']
+          ).filter(Boolean).join(' · ')}</div>
         </div>
         <div className="tile">
           <div className="label">Memory</div>
@@ -199,9 +206,11 @@ export default function Hardware() {
           <div className="props">
             <P k="Model" v={cpu.model} />
             <P k="Vendor" v={cpu.vendor} />
-            <P k="Cores" v={cores ? `${cores}${cpu.threads_per_core && cpu.threads_per_core > 1 ? ` · ${cpu.threads_per_core} threads each` : ''}` : ''} />
+            {multi && <P k="CPUs" v={`${sockets}`} />}
+            <P k="Cores" v={cores ? `${multi ? `${per(cores)} per CPU · ${cores} in all` : cores}${cpu.threads_per_core && cpu.threads_per_core > 1 ? ` · ${cpu.threads_per_core} threads each` : ''}` : ''} />
+            {multi && threads > 0 && <P k="Threads" v={`${per(threads)} per CPU · ${threads} in all`} />}
             <P k="Clock" v={cpu.min_mhz && cpu.max_mhz ? `${ghz(cpu.min_mhz)} – ${ghz(cpu.max_mhz)}` : ghz(cpu.max_mhz)} />
-            <P k="Socket" v={(cpu.socket_names ?? []).join(', ')} />
+            <P k={multi ? 'Sockets' : 'Socket'} v={(cpu.socket_names ?? []).join(', ')} />
             <P k="Arch" v={cpu.arch} />
             <P k="Family" v={[cpu.family && `family ${cpu.family}`, cpu.model_id && `model ${cpu.model_id}`, cpu.stepping && `stepping ${cpu.stepping}`].filter(Boolean).join(' · ')} />
             <P k="Microcode" v={cpu.microcode} mono />

web/src/Top.tsx

@@ -3,7 +3,7 @@
 // as they climb. The numbers are Ham Lab's own — the sensor stream for everything measured and
 // /api/procs for the table — so this works on any machine the console can see, whether or not
 // btop is installed on it.
-import { createContext, memo, useCallback, useContext, useEffect, useLayoutEffect, useMemo, useRef, useState } from 'react'
+import { Fragment, createContext, memo, useCallback, useContext, useEffect, useLayoutEffect, useMemo, useRef, useState } from 'react'
 import { api, GPU, ProcTable, Proc, Sample } from './api'
 import { Catalog, MONITOR_SHAPE, Values, mergeValues, sampleOf, stale } from './sampleAdapter'
 import { fetchCatalog, fetchHistories, useSensorStream } from './sensors'
@@ -11,6 +11,7 @@ import { fmtMs, nextPoint, REFRESH_OPTIONS, setRefreshMs, strideMs, useHistorySe
 import { Loading, askYes, toast } from './ui'
 import { Theme, currentThemeId, themeById } from './themes'
 import { columnsAt } from './graphcols'
+import { socketThreads } from './sockets'
 import { savePrefs } from './accents'
 
 const MAX = 300 // points kept per curve, as the machines keep them
@@ -294,6 +295,13 @@ export default function Top() {
   // the process table, and the CPU model, come from the same place the Processes page reads,
   // on the sample's beat (above)
   const [pickedIf, setPickedIf] = useState<string | null>(null) // which NIC the net box draws
+  // which sockets' threads are folded away on a two-socket machine; remembered in this browser
+  const [folded, setFolded] = useState<number[]>(() => { try { return JSON.parse(localStorage.getItem('top.folded-cpus') ?? '[]') } catch { return [] } })
+  const fold = (pk: number) => setFolded((f) => {
+    const n = f.includes(pk) ? f.filter((x) => x !== pk) : [...f, pk]
+    try { localStorage.setItem('top.folded-cpus', JSON.stringify(n)) } catch { /* private window */ }
+    return n
+  })
   // Which pool each drive belongs to, asked once: a drive in a ZFS pool has no filesystem of
   // its own, and "nothing is on it" would be a lie about eight drives holding a NAS.
   const [pools, setPools] = useState<Record<string, string>>({})
@@ -360,7 +368,16 @@ export default function Top() {
   const model = (pt?.cpu_model ?? '').replace(/\(R\)|\(TM\)|CPU|Processor/g, '').replace(/@.*$/, '').replace(/^\d+th Gen\s*/i, '').replace(/\s+/g, ' ').trim()
   // the hottest thing that looks like a CPU sensor, and one per core where the chip names them;
   // a second thread of a core (CPU 8 on an 8-core chip with hyper-threading) wears its core's
-  const coreTemp = (i: number) => { const id = pt?.core_ids?.[i] ?? i; return s?.temps.find((t) => new RegExp(`core ${id}\\b`, 'i').test(t.label))?.c }
+  // On a two-socket board each CPU has its own sensor chip, both counting from "Core 0", so a
+  // thread's core is looked up on its own socket's chip.
+  const pkgs = pt?.core_pkgs?.length === cores.length ? pt.core_pkgs : undefined
+  // each CPU's own threads, counted from 0 under it (sockets.ts): the kernel's one numbering
+  // of the whole machine puts cpu 0–9 and 20–29 on the first and 10–19 and 30–39 on the second
+  const socks = pkgs ? socketThreads(pkgs) : []
+  const sockets = socks.map((x) => x.pkg)
+  const coreTemp = (i: number) => { const id = pt?.core_ids?.[i] ?? i, pk = pkgs?.[i]; return s?.temps.find((t) => new RegExp(`core ${id}\\b`, 'i').test(t.label) && (pk === undefined || (t.pkg ?? 0) === pk))?.c }
+  const sockTemp = (pk: number) => s?.temps.find((t) => t.chip === 'coretemp' && new RegExp(`^package id ${pk}$`, 'i').test(t.label))?.c
+  const sockUse = (pk: number) => { const v = cores.filter((_, i) => pkgs?.[i] === pk); return v.length ? v.reduce((a, b) => a + b, 0) / v.length : 0 }
   const cpuTemp = pt?.cpu_temp ?? s?.temps.find((t) => /package|tctl|tdie|cpu/i.test(t.label))?.c ?? coreTemp(0)
   // the temperature columns are there for every row once any row has one, so the % column lines up
   const anyTemp = cpuTemp !== undefined || cores.some((_, i) => coreTemp(i) !== undefined)
@@ -403,26 +420,43 @@ export default function Top() {
             <Graph values={curves.get('cpu.usage') ?? []} tick={tick} advance={advance} now={now} cap={cap} max={100} grad="cpu" />
             <div className="t-up">up {upfor(uptime)}</div>
           </div>
-          <div className="t-cores">
-            <div className="t-cores-head"><b>{model || 'cpu'}</b><span className="spacer" />{pt?.cpu_mhz ? `${Math.round(pt.cpu_mhz)} MHz` : ''}
-              {pt?.cpu_power_w ? <span title="the CPU's own power reading: cores, built-in graphics and memory controller, not the whole machine">{pt.cpu_mhz ? ' · ' : ''}{pt.cpu_power_w < 10 ? pt.cpu_power_w.toFixed(1) : Math.round(pt.cpu_power_w)} W</span> : null}</div>
-            <div className="t-core">
+          <div className={'t-cores' + (sockets.length > 1 ? ' multi' : '')}>
+            <div className="t-cores-head"><b>{sockets.length > 1 ? `${sockets.length} × ` : ''}{model || 'cpu'}</b><span className="spacer" />{pt?.cpu_mhz ? `${Math.round(pt.cpu_mhz)} MHz` : ''}
+              {pt?.cpu_power_w && sockets.length < 2 ? <span title="the CPU's own power reading: cores, built-in graphics and memory controller, not the whole machine">{pt.cpu_mhz ? ' · ' : ''}{pt.cpu_power_w < 10 ? pt.cpu_power_w.toFixed(1) : Math.round(pt.cpu_power_w)} W</span> : null}</div>
+            {sockets.length < 2 && <div className="t-core">
               <span className="t-core-n">CPU</span>
               <Meter pct={cpu} grad="cpu" />
               <span className="t-core-p">{Math.round(cpu)}%</span>
               {anyTemp && <><span className="t-core-t">{cpuTemp !== undefined && <Meter pct={Math.min(100, (cpuTemp / 100) * 100)} grad="temp" />}</span><span className="t-core-c">{cpuTemp !== undefined ? `${Math.round(cpuTemp)}°C` : ''}</span></>}
-            </div>
-            {cores.map((v, i) => {
-              const t = coreTemp(i)
-              return (
-                <div className="t-core" key={i}>
-                  <span className="t-core-n">C{i}</span>
-                  <span className="t-core-g" title={pt?.core_ids && pt.core_ids[i] !== i ? `cpu ${i} · a thread of core ${pt.core_ids[i]}` : `core ${i}`}>{braille(recent.get(`cpu.core.${i}.usage`) ?? [], 9)}</span>
-                  <span className="t-core-p">{Math.round(v)}%</span>
-                  {anyTemp && <><span className="t-core-t">{t !== undefined && <Meter pct={Math.min(100, t)} grad="temp" />}</span><span className="t-core-c">{t !== undefined ? `${Math.round(t)}°C` : ''}</span></>}
-                </div>
-              )
-            })}
+            </div>}
+            {(sockets.length > 1 ? socks : [{ pkg: undefined, cpus: cores.map((_, i) => i) }]).map(({ pkg: pk, cpus }) => (
+              <Fragment key={pk ?? 'all'}>
+                {pk !== undefined && (() => {
+                  const u = sockUse(pk), t = sockTemp(pk), w = pt?.cpu_power_pkgs_w?.[pk], open = !folded.includes(pk)
+                  return (
+                    <div className="t-core t-sock" role="button" aria-expanded={open} title={open ? 'hide its threads' : 'show its threads'} onClick={() => fold(pk)}>
+                      <span className="t-core-n"><span className="t-fold">{open ? '▾' : '▸'}</span>CPU{pk + 1}</span>
+                      <span className="t-core-w" title="this CPU's own power reading: its cores, cache and memory controller">{w ? `${w < 10 ? w.toFixed(1) : Math.round(w)} W` : ''}</span>
+                      <Meter pct={u} grad="cpu" />
+                      <span className="t-core-p">{Math.round(u)}%</span>
+                      {anyTemp && <><span className="t-core-t">{t !== undefined && <Meter pct={Math.min(100, t)} grad="temp" />}</span><span className="t-core-c">{t !== undefined ? `${Math.round(t)}°C` : ''}</span></>}
+                    </div>
+                  )
+                })()}
+                {(pk !== undefined && folded.includes(pk) ? [] : cpus).map((i, n) => {
+                  // n is the thread's number under its own CPU, i the kernel's for the whole machine
+                  const t = coreTemp(i), v = cores[i] ?? 0
+                  return (
+                    <div className="t-core" key={i}>
+                      <span className="t-core-n">C{n}</span>
+                      <span className="t-core-g" title={pk !== undefined ? `CPU ${pk + 1} · thread ${n} · cpu ${i} to the kernel${pt?.core_ids ? ` · a thread of core ${pt.core_ids[i]}` : ''}` : pt?.core_ids && pt.core_ids[i] !== i ? `cpu ${i} · a thread of core ${pt.core_ids[i]}` : `core ${i}`}>{braille(recent.get(`cpu.core.${i}.usage`) ?? [], 9)}</span>
+                      <span className="t-core-p">{Math.round(v)}%</span>
+                      {anyTemp && <><span className="t-core-t">{t !== undefined && <Meter pct={Math.min(100, t)} grad="temp" />}</span><span className="t-core-c">{t !== undefined ? `${Math.round(t)}°C` : ''}</span></>}
+                    </div>
+                  )
+                })}
+              </Fragment>
+            ))}
             <div className="t-load">Load AVG:<span className="spacer" />{load.map((x) => x.toFixed(2)).join('   ')}</div>
           </div>
         </div>
@@ -456,7 +490,9 @@ export default function Top() {
             // pool or an md array spans several drives, so nothing is attributed and the row
             // shows what is true of the hardware — how busy it is, and how hot.
             const on = fs.filter((f) => f.disk === d.name)
-            const used = on.reduce((a, f) => a + f.used, 0)
+            // a bind mount (nexus's /var/log is /data/var-log) is the same filesystem twice:
+            // count each source device once, or the drive reads double
+            const used = [...new Map(on.map((f) => [f.source || f.mount, f.used])).values()].reduce((a, b) => a + b, 0)
             return <Disk key={d.name} name={d.name} note={d.model} size={d.size} used={on.length ? used : undefined} io={d.util_pct} temp={d.temp} mounts={on.map((f) => f.mount)} pool={pools[d.name]} />
           })}
           {(s?.disks ?? []).length === 0 && <div className="t-dim">no drives reported</div>}
@@ -560,13 +596,11 @@ const ENGINES: [string, string][] = [['render', 'render'], ['compute', 'compute'
 function GPUBox({ g, values, tick, advance, now, cap }: { g: GPU; values: number[]; tick: number; advance: number; now: number; cap: number }) {
   const engines = ENGINES.filter(([k]) => g.engines && g.engines[k] !== undefined)
   const temp = g.temp !== undefined && g.temp > 0 ? g.temp : undefined
-  const how = g.stalled ? 'a sysfs read of this card stopped answering; its readings are paused until the console restarts'
-    : g.usage_src === 'idle' ? 'busy, from idle residency' : g.usage_src === 'busy' ? 'busy, as the driver reports it' : g.usage_src === 'clients' ? 'the busiest engine, summed over its clients' : 'this driver exposes no activity counter'
   return (
     <div className="t-gpu">
       <div className="t-gpu-graph">
         <Graph values={values} tick={tick} advance={advance} now={now} cap={cap} max={100} grad="cpu" />
-        <div className="t-up">{g.usage_src ? `${Math.round(g.usage)}% ` : ''}{how}</div>
+        {g.stalled && <div className="t-up">a read of this card stopped answering; its readings are paused until it does</div>}
       </div>
       <div className="t-gpu-stats">
         <div className="t-cores-head"><b>{g.name}</b><span className="spacer" />{g.driver}{g.integrated ? ' · integrated' : ''}</div>
@@ -590,7 +624,7 @@ function GPUBox({ g, values, tick, advance, now, cap }: { g: GPU; values: number
             <span className="t-fact" title={g.req_mhz ? `${Math.round(g.req_mhz)} MHz requested${g.max_mhz ? `, ${Math.round(g.max_mhz)} MHz max` : ''}` : undefined}>{Math.round(g.mhz ?? 0)}{g.max_mhz ? ` / ${Math.round(g.max_mhz)}` : ''} MHz</span></div> : null}
           {(g.power_w !== undefined && g.power_w > 0) || g.power_cap_w ? <div className="t-row t-sub"><span>Power</span><span className="spacer" />
             <span className="t-fact">{(g.power_w ?? 0).toFixed(1)}{g.power_cap_w ? ` / ${Math.round(g.power_cap_w)}` : ''} W</span></div> : null}
-          {g.fan_rpm ? <div className="t-row t-sub"><span>Fan</span><span className="spacer" /><span className="t-fact">{g.fan_rpm} rpm</span></div> : null}
+          {g.fan_rpm || g.fan_pct !== undefined ? <div className="t-row t-sub"><span>Fan</span><span className="spacer" /><span className="t-fact">{[g.fan_pct !== undefined ? `${Math.round(g.fan_pct)}%` : '', g.fan_rpm ? `${g.fan_rpm} rpm` : ''].filter(Boolean).join(' · ')}</span></div> : null}
           {g.mem_total ? <div className="t-row t-sub"><span>Mem</span><span className="t-pct">{Math.round(((g.mem_used ?? 0) / g.mem_total) * 100)}%</span><Meter pct={((g.mem_used ?? 0) / g.mem_total) * 100} grad="used" /><span className="t-amt">{bytes(g.mem_used ?? 0, 1)}</span></div>
             : g.mem_used ? <div className="t-row t-sub"><span>Mem</span><span className="spacer" /><span className="t-fact" title="resident memory of the processes that have it open">{bytes(g.mem_used, 1)}</span></div> : null}
         </div>
@@ -611,13 +645,13 @@ function GPUBox({ g, values, tick, advance, now, cap }: { g: GPU; values: number
 // Nothing on this GPU can be read — no busy counter, engine, temperature, clock, power, memory,
 // nor a process with it open: the Mac's Asahi, today. Its box would be a graph that can never
 // draw beside a column of dashes, so it is one line instead. A client opening it brings the box back.
-const unmeasured = (g: GPU) => !g.usage_src && !g.engines && !(g.temp && g.temp > 0) && !g.mhz && !g.max_mhz && !g.power_w && !g.power_cap_w && !g.fan_rpm && !g.mem_total && !g.mem_used && g.clients.length === 0
+const unmeasured = (g: GPU) => !g.usage_src && !g.engines && !(g.temp && g.temp > 0) && !g.mhz && !g.max_mhz && !g.power_w && !g.power_cap_w && !g.fan_rpm && g.fan_pct === undefined && !g.mem_total && !g.mem_used && g.clients.length === 0
 function GPULine({ g }: { g: GPU }) {
   return (
     <div className="t-row t-gpu-line">
       <span>{g.driver}{g.integrated ? ' · integrated' : ''}</span>
       {g.stalled
-        ? <span className="t-dim" title="a sysfs read of this card stopped answering">readings paused until the console restarts</span>
+        ? <span className="t-dim" title="a read of this card stopped answering">readings paused until it answers</span>
         : <span className="t-dim" title="the driver exposes no busy counter, clock, temperature, power or memory reading">nothing measurable</span>}
       <span className="t-dim t-gpu-who" title={g.clients_ok ? undefined : 'the console has no root here'}>{g.clients_ok ? 'no process has it open' : 'other users’ processes are hidden'}</span>
     </div>

web/src/api.ts

@@ -71,11 +71,11 @@ export const titleOf = (name: string | null, machines: MachineStatus[], hubHost:
 export interface MachineStatus { name: string; url: string; fingerprint?: string; added: string; online: boolean; host?: string; version?: string; load?: string; uptime?: number; reboot_needed?: boolean; last_seen?: string; error?: string }
 export interface Term { id: string; name: string; running: boolean; exit_code: number; created: string; clients: number }
 
-export interface Temp { chip: string; label: string; c: number; max?: number; crit?: number }
+export interface Temp { chip: string; label: string; c: number; max?: number; crit?: number; pkg?: number }
 export interface Fan { chip: string; label: string; rpm: number }
 export interface NetIf { iface: string; up: boolean; rx_bps: number; tx_bps: number; rx_total: number; tx_total: number }
 export interface Disk { name: string; model?: string; size: number; read_bps: number; write_bps: number; util_pct: number; temp?: number }
-export interface FS { mount: string; type: string; size: number; used: number; remote?: boolean; disk?: string }
+export interface FS { mount: string; type: string; size: number; used: number; remote?: boolean; disk?: string; source?: string }
 export interface UPS { driver: string; model: string; state: string; source: string; on_battery: boolean; in_v: number; out_v: number; battery_pct: number; runtime_min: number; load_w: number; load_pct: number; last_event?: string }
 export interface Proc { pid: number; ppid: number; name: string; cmd?: string; user: string; cpu: number; gpu?: number; cpu_time: number; rss: number; threads: number; state: string; nice: number; started: number }
 export interface MemInfo { total: number; used: number; avail: number; free?: number; cached?: number; buffers?: number }
@@ -86,11 +86,11 @@ export interface GPUClient { pid: number; name: string; usage: number; engines?:
 export interface GPU {
   name: string; driver: string; bus?: string; integrated?: boolean
   usage: number; usage_src?: 'idle' | 'busy' | 'clients'; engines?: Record<string, number>
-  temp?: number; temp_src?: 'gpu' | 'package'; power_w?: number; power_cap_w?: number; fan_rpm?: number
+  temp?: number; temp_src?: 'gpu' | 'package'; power_w?: number; power_cap_w?: number; fan_rpm?: number; fan_pct?: number
   mhz?: number; req_mhz?: number; max_mhz?: number; mem_used?: number; mem_total?: number
   clients: GPUClient[]; clients_ok: boolean; stalled?: boolean
 }
-export interface ProcTable { t: number; cpu: number; cores: number[]; cpu_hist: number[]; cpu_model?: string; cpu_mhz?: number; cpu_max_mhz?: number; cpu_power_w?: number; cpu_temp?: number; core_ids?: number[]; load: [number, number, number]; mem: MemInfo; swap: MemInfo; net: NetIf[]; gpus?: GPU[]; uptime: number; procs: Proc[]; total: number; running: number }
+export interface ProcTable { t: number; cpu: number; cores: number[]; cpu_hist: number[]; cpu_model?: string; cpu_mhz?: number; cpu_max_mhz?: number; cpu_power_w?: number; cpu_power_pkgs_w?: number[]; cpu_temp?: number; core_ids?: number[]; core_pkgs?: number[]; load: [number, number, number]; mem: MemInfo; swap: MemInfo; net: NetIf[]; gpus?: GPU[]; uptime: number; procs: Proc[]; total: number; running: number }
 export interface Sample {
   t: number; interval_ms: number; cpu: number; cores: number[]; load: [number, number, number]
   mem: { total: number; used: number; avail: number }; swap: { total: number; used: number; avail: number }

web/src/sampleAdapter.ts

@@ -47,11 +47,11 @@ export function sampleOf(v: Values, cat: Catalog): Sample {
     t: Date.now(), interval_ms: num(v, 'hamlab.sample_interval'),
     cpu: num(v, 'cpu.usage'), cores, load: [num(v, 'load.1'), num(v, 'load.5'), num(v, 'load.15')],
     mem: mem('mem'), swap: mem('swap'), uptime: num(v, 'uptime'),
-    temps: hw('temp.').map(({ id, s }) => ({ chip: s.labels!.chip, label: s.labels!.label, c: num(v, id), max: s.warn, crit: s.crit })),
+    temps: hw('temp.').map(({ id, s }) => ({ chip: s.labels!.chip, label: s.labels!.label, c: num(v, id), max: s.warn, crit: s.crit, pkg: s.labels!.package !== undefined ? Number(s.labels!.package) : undefined })),
     fans: hw('fan.').map(({ id, s }) => ({ chip: s.labels!.chip, label: s.labels!.label, rpm: num(v, id) })),
     net: [...groups(v, cat, 'net.', 'iface')].map(([iface, p]) => ({ iface, up: v[p + '.up'] === true, rx_bps: num(v, p + '.rx'), tx_bps: num(v, p + '.tx'), rx_total: num(v, p + '.rx_total'), tx_total: num(v, p + '.tx_total') })),
     disks: [...groups(v, cat, 'disk.', 'device')].map(([name, p]) => ({ name, model: cat.get(p + '.size')?.labels?.model || undefined, size: num(v, p + '.size'), read_bps: num(v, p + '.read'), write_bps: num(v, p + '.write'), util_pct: num(v, p + '.util'), temp: num(v, p + '.temp') || undefined })),
-    fs: [...groups(v, cat, 'fs.', 'mount')].map(([mount, p]) => ({ mount, type: cat.get(p + '.size')?.labels?.fstype ?? '', size: num(v, p + '.size'), used: num(v, p + '.used'), remote: cat.get(p + '.size')?.labels?.remote === '1', disk: cat.get(p + '.size')?.labels?.disk })),
+    fs: [...groups(v, cat, 'fs.', 'mount')].map(([mount, p]) => ({ mount, type: cat.get(p + '.size')?.labels?.fstype ?? '', size: num(v, p + '.size'), used: num(v, p + '.used'), remote: cat.get(p + '.size')?.labels?.remote === '1', disk: cat.get(p + '.size')?.labels?.disk, source: cat.get(p + '.size')?.labels?.source })),
     ups: v['ups.battery'] === undefined ? null : {
       driver: cat.get('ups.last_event')?.labels?.driver ?? '', model: cat.get('ups.battery')?.labels?.model ?? '', state: str(v, 'ups.state'), source: str(v, 'ups.source'),
       on_battery: v['ups.on_battery'] === true, in_v: num(v, 'ups.in_v'), out_v: num(v, 'ups.out_v'), battery_pct: num(v, 'ups.battery'), runtime_min: num(v, 'ups.runtime'),

web/src/sockets.ts

@@ -0,0 +1,17 @@
+// Which threads belong to which CPU, on a machine with more than one.
+//
+// The kernel numbers every thread of the machine in one run, and on a two-socket board with
+// hyper-threading it deals them out a socket at a time and then goes round again: cpu 0–9 are
+// CPU 1's cores, 10–19 CPU 2's, 20–29 the second threads of CPU 1's, 30–39 of CPU 2's. Listed
+// under its CPU by that number, a 20-thread processor reads "C0…C9, C20…C29" and the one
+// beside it "C10…C19, C30…C39" — neither looks like twenty of anything. So each CPU's threads
+// are counted from 0 under it, in the kernel's order, the way a one-socket machine's are: the
+// first half are its cores, the second half their second threads.
+//
+// `pkgs[i]` is the socket logical CPU `i` is on. The answer is, for each socket (ascending),
+// the logical CPUs on it (ascending); a thread's number under its CPU is its place in that list.
+export function socketThreads(pkgs: number[]): { pkg: number; cpus: number[] }[] {
+  const by = new Map<number, number[]>()
+  pkgs.forEach((pk, i) => { const l = by.get(pk); if (l) l.push(i); else by.set(pk, [i]) })
+  return [...by.entries()].sort((a, b) => a[0] - b[0]).map(([pkg, cpus]) => ({ pkg, cpus }))
+}

web/src/styles.css

@@ -794,6 +794,11 @@ pre.diff .add { color: var(--ok); background: color-mix(in srgb, var(--ok) 12%,
 .t-cores-head b { color: var(--fg); font-weight: 400; overflow: hidden; text-overflow: ellipsis; }
 .t-core { display: flex; align-items: center; gap: 6px; }
 .t-core-n { width: 30px; } .t-core-p { width: 34px; text-align: right; }
+/* a two-socket board: a row per socket — arrow, name, watts — with its threads indented under it */
+.t-cores.multi .t-core-n { width: 50px; } .t-cores.multi .t-core:not(.t-sock) .t-core-n { padding-left: 14px; box-sizing: border-box; }
+.t-sock { color: var(--fg); margin-top: 2px; cursor: pointer; user-select: none; }
+.t-fold { display: inline-block; width: 12px; color: var(--muted); }
+.t-core-w { width: 40px; flex-shrink: 0; text-align: right; color: var(--muted); }
 .t-core-g { flex: 1; color: var(--ok); letter-spacing: 0; overflow: hidden; white-space: nowrap; }
 .t-core .tm { flex: 1; } .t-core-t { flex: 0 0 62px; display: flex; } .t-core-c { width: 42px; text-align: right; color: var(--t-blue); }
 .t-load { display: flex; margin-top: 2px; }