Compare commits
28
Commits
@@ -259,6 +259,100 @@ something long in one, switch away and back, refresh — it's all still there.
|
||||
id and its route are gone. A rail order saved before this keeps its shape — `railorder.ts`
|
||||
maps the retired `top` to `procs` and drops the duplicate, so the merged page sits where
|
||||
Top sat rather than falling to the bottom.
|
||||
- **Dashboard retired** — **2026-09-23**, at Mitchell's instruction, the same reasoning that
|
||||
merged Top into Processes: the app repeated itself. The customisable module list is gone
|
||||
(`Dashboard.tsx` deleted), and with it `/api/prefs/dashboard`, `applyPrefs`, and the
|
||||
per-user `dashboard` blob in `state.json`. The Processes tab already shows those numbers
|
||||
on one screen. `fmtBytes`/`fmtRate` moved to `fmt.ts` (Disks, Files and Space use them);
|
||||
`MONITOR_HISTORY` and `seriesOf` went with the page. The default view after login, after
|
||||
switching machines, and when a hub-only page is left behind, is now **Processes**. A saved
|
||||
rail order naming `dashboard` degrades quietly — `orderBy` drops ids with no page.
|
||||
**Widgets stays and says what it is for**: a notice at the top explains these are for
|
||||
dashboards *outside* Ham Lab — Glance, Home Assistant, a wall display — served from
|
||||
`/embed/widget` with a read-only sensor token, not a second copy of the console's screens.
|
||||
**It also closed a real bug.** `applyPrefs` set the machine's sampling rate from
|
||||
`intervalMs` inside the dashboard layout, and its `else` branch reset the sampler to the
|
||||
default whenever that field was absent. Since layouts became per-user, one account saving
|
||||
a dashboard could silently reset the sampling rate for the whole machine, overriding the
|
||||
fleet refresh rate set in Settings. The sampler now has one source: `refreshMs` in the UI
|
||||
prefs, which the hub fans out, remembered per machine as `sampler_ms`.
|
||||
- **ZFS tab, with scheduled snapshots** — **shipped 2026-09-24** (`internal/storage/snapshots.go`,
|
||||
`internal/api/zfs.go`, `ZFS.tsx`). Pools, datasets and snapshots moved out of Disks —
|
||||
Disks is the hardware, ZFS is the filesystem on top of it — and the tab gained what it was
|
||||
split out for: **snapshot schedules**. Hourly, daily, weekly or monthly, per dataset,
|
||||
optionally recursive, with a **retention period** and a floor that keeps the newest few
|
||||
whatever their age. A missed window (the machine was off) runs once on return rather than
|
||||
being skipped.
|
||||
**The naming is zfSnap's, deliberately.** `grid` has run zfSnap v1.11.1 from root cron at
|
||||
02:30 with a 6-week TTL since before this repo existed, and it works. So rather than
|
||||
invent a second convention for two tools to disagree over, this writes and reads exactly
|
||||
what zfSnap does — `tank@2026-09-24_02.30.01--6w`, **retention encoded in the name**. Three
|
||||
consequences: Ham Lab shows and expires the snapshots zfSnap already took; the two can run
|
||||
side by side without fighting, since both only remove what has passed the expiry written
|
||||
into its own name; and changing a schedule never re-dates old snapshots, because each
|
||||
carries the rule it was taken under. The page **detects the existing cron job**
|
||||
(zfSnap, zfs-auto-snapshot, sanoid, or a bare `zfs snapshot`) and says so, warning if a Ham
|
||||
Lab schedule is running as well.
|
||||
**Nothing is destroyed unseen.** Only names that parse as a stamp and a TTL are ever
|
||||
candidates for expiry, so a snapshot taken by hand is invisible to a schedule; Run now
|
||||
says that expired snapshots go with it; and a snapshot deleted by hand is confirmed first
|
||||
and needs its full name echoed back by the caller. (A "what would it delete?" preview
|
||||
shipped and was removed the same day as redundant — Mitchell's call.)
|
||||
The datasets table is a **snapshot explorer**: each dataset shows how many snapshots it has
|
||||
and what they cost, and opens into the snapshots themselves — when each was taken, **when
|
||||
it expires** (read from the TTL in its own name, so no server call), its size, and Delete.
|
||||
Deleting takes the same-named snapshot on descendant datasets too, which is how a recursive
|
||||
schedule made it.
|
||||
**A schedule is a real cron job** (2026-09-24, Mitchell's call after the first cut ran it
|
||||
from a ticker inside the daemon). Ham Lab writes `/etc/cron.d/hamlab-zfs`, owns it, and
|
||||
rewrites it whole on every change; each line calls `hamlab zfs-snapshot <id>` as the
|
||||
account Ham Lab runs as, reaching zfs through sudo as the daemon does. Snapshots are
|
||||
backup-shaped and must not stop because a console is down, upgrading, or uninstalled — and
|
||||
a cron line is greppable by anyone looking at the machine the normal way. **A cron
|
||||
expression is the single description of when a schedule fires**: the four common shapes
|
||||
render to one, `custom` states one outright, and the next run is read back with the same
|
||||
evaluator the Jobs tab uses on everyone else's lines, so the page, the file and the
|
||||
prediction cannot disagree. A run's outcome is written to `zfs-runs/<id>.json` beside the
|
||||
config, never to `state.json`, because the cron job is a separate process and the daemon
|
||||
would clobber it. An existing external job can be **adopted** — its schedule, retention
|
||||
and dataset read out of the cron line into a new Ham Lab schedule.
|
||||
**zfSnap is retired on `grid`** as of that day: Ham Lab now takes the nightly `tank`
|
||||
snapshot, the package is purged and the old cron line is gone, with the 515 snapshots it
|
||||
had taken left untouched and fully readable here — which is what adopting its naming
|
||||
bought. See `hamlet/docs/hosts.md`.
|
||||
- **KVM tab** — **shipped 2026-09-24** (`internal/api/kvm.go`, `KVM.tsx`, `config.KVM`).
|
||||
Paste a device's address; its own web page appears in the tab. **Nothing in it knows
|
||||
anything about any particular KVM** — the first device here is a Sipeed NanoKVM on `grid`
|
||||
at `10.0.0.217`, but none of that is in the code.
|
||||
**Why it proxies rather than framing the device directly.** Ham Lab is served over TLS and
|
||||
a KVM usually is not, so a bare `http://…` iframe is blocked as mixed content before it
|
||||
loads. Serving the device from Ham Lab's own origin (`/kvm/<id>/…`) fixes that, puts it
|
||||
behind **Ham Lab's** login rather than a second one, and makes the iframe same-origin.
|
||||
**The general problem, and the answer.** A device UI asks for its assets by absolute path
|
||||
— `/assets/app.js`, `/api/whatever` — which no path prefix can rewrite away, and rewriting
|
||||
bodies per device would not be general. So the prefix is only the entry point: any request
|
||||
Ham Lab does not serve itself is routed to whichever KVM the asking page belongs to, read
|
||||
from the **Referer**, falling back to a **cookie** when there is none (a WebSocket upgrade).
|
||||
Ham Lab's own routes always match first, so a device can never shadow the console, and the
|
||||
fallback is gated on a live session. The device never receives Ham Lab's session cookie,
|
||||
and its `X-Frame-Options`/CSP/HSTS are dropped on the way out since Ham Lab is the origin
|
||||
now. Redirects to the device's root are rewritten back through the prefix.
|
||||
**One catch found in testing:** Ham Lab's own `frame-ancestors 'none'` forbids framing even
|
||||
from the same origin, so `/kvm/` paths send `'self'` instead. Without that the iframe was
|
||||
refused before it loaded.
|
||||
**The keyboard** needs less than it sounds: a page cannot forward keys into an iframe and
|
||||
does not need to, since one with focus receives them directly. Fullscreen plus
|
||||
`navigator.keyboard.lock()` is what takes Escape, Tab and Ctrl-W back from the browser;
|
||||
that is Chromium-only, and the tab says which version you are getting rather than
|
||||
pretending.
|
||||
**One KVM per machine, and the tab follows the machine switcher** (corrected the same day:
|
||||
the first cut was a hub-side list you pointed at machines, which had it backwards). Switch
|
||||
to `grid` and you get grid's; adding a second for a machine replaces the first. The
|
||||
addresses are still kept **on the hub** rather than on each machine, because a KVM earns
|
||||
its keep exactly when the machine is down and its own Ham Lab cannot answer — the hub
|
||||
proxies the device either way — and a KVM need not belong to a machine Ham Lab manages at
|
||||
all. Not done: holding the device's own login on the hub, so its page does not ask for a
|
||||
second password.
|
||||
- **Skins** — 13 since 2026-09-19 (`themes.ts`: a palette is one `mk(…)` line): Dark,
|
||||
Light, Cyberpunk, Retro, **HUD** (hairlines of cyan on near-black, corner ticks, thin
|
||||
type), Nord, Gruvbox, Catppuccin + Latte, Dracula, Tokyo Night, Solarized Dark + Light;
|
||||
@@ -448,11 +542,12 @@ something long in one, switch away and back, refresh — it's all still there.
|
||||
|
||||
## To do
|
||||
|
||||
- **Deprecate copyparty** — asked for 2026-09-23, not started. `copyparty` on `grid`
|
||||
(`copyparty.service`, `:3923`, `/etc/copyparty.conf`, serving all of `/tank` as user
|
||||
`core` with one account behind NPM) overlaps the **Files** tab almost entirely. The aim
|
||||
is to stop the service and keep only Ham Lab. What Files must gain first, in the order
|
||||
that matters:
|
||||
- **Grow the Files tab until copyparty can go** — asked for 2026-09-23 as two items (Files
|
||||
polish, and deprecating copyparty), merged 2026-09-24: they are the same tab and the same
|
||||
work. `copyparty` on `grid` (`copyparty.service`, `:3923`, `/etc/copyparty.conf`, serving
|
||||
all of `/tank` as user `core` with one account behind NPM) overlaps **Files** almost
|
||||
entirely. The aim is to stop the service and keep only Ham Lab, with general ease-of-use
|
||||
polish along the way. What Files must gain first, in the order that matters:
|
||||
- **Multi-select and bulk actions.** copyparty selects many rows and moves, deletes or
|
||||
downloads them together; Files acts on one row at a time.
|
||||
- **Download a folder**, and a multi-file download, as a streamed zip or tar — the thing
|
||||
@@ -472,53 +567,44 @@ something long in one, switch away and back, refresh — it's all still there.
|
||||
`hamlet`'s `docs/hosts.md` bare-metal list, `docs/gotchas.md` (the "`docker ps` is not
|
||||
the inventory" note names it) and the `grid` row in `CLAUDE.md`.
|
||||
|
||||
- **Deprecate Dockhand** — asked for 2026-09-23, not started; the *homelab* half of this
|
||||
(migrating every stack off Dockhand's data dir, then removing the container) is
|
||||
`hamlet`'s `docs/pending.md` → "Retire Dockhand; Ham Lab becomes the orchestrator",
|
||||
agreed 2026-09-20. This entry is the *app* half only: what Ham Lab must do before that
|
||||
migration can start. Its Stacks page already creates, edits, deploys, updates, restarts,
|
||||
stops, downs and deletes stacks on every machine, so the gaps are Dockhand's extras:
|
||||
- **Git-sourced stacks.** Dockhand's `stack_sources` table points a stack at a repo and
|
||||
pulls it. Ham Lab has no equivalent. Decide whether to build it or to drop it — most
|
||||
stacks here have no row in that table anyway, so dropping it may be honest.
|
||||
- **Scheduled image updates.** Ham Lab checks for new images only when asked (a registry
|
||||
request Docker Hub counts) and never updates by itself. If anything relies on Dockhand
|
||||
updating on a schedule, that must move here or be dropped on purpose.
|
||||
- **Webhooks / per-host environments**, if either is in use — inventory before deciding.
|
||||
- **Deploying from a host path without an orchestrator container.** Ham Lab already runs
|
||||
compose inside `dockhand` because that is how the stacks were deployed; once they move
|
||||
to a neutral home the "via" path is unused for them, but a stack deployed to `grid` or
|
||||
the Mac still goes over `tcp://<node>:2375` from the hub. Decide whether Ham Lab should
|
||||
instead ask the node's own Ham Lab to deploy locally, which would end the `*:2375`
|
||||
exposure (its own item in `docs/pending.md`).
|
||||
- *Not app work, but the trigger:* nothing here blocks starting the migration except the
|
||||
inventory. Do that first.
|
||||
- **Grow the Docker tab until Dockhand can go** — asked for 2026-09-23 and widened
|
||||
2026-09-24 to take in Docker tab polish and feature work, on the same reasoning as the
|
||||
Files item: it is one tab, and the gaps that block retiring the old tool and the things
|
||||
that would simply make it better are the same body of work. The *homelab* half —
|
||||
the cutover itself, then moving every stack off Dockhand's data dir — is `hamlet`'s
|
||||
`docs/pending.md` → "Retire Dockhand; Ham Lab becomes the orchestrator", agreed
|
||||
2026-09-20. This entry is the *app* half.
|
||||
|
||||
- **Rename the Space tab** — asked for 2026-09-23, not started. "Space" is vague; the tab
|
||||
is WizTree for a disk or pool. Candidates: **Usage**, **Disk Usage**, **Storage**. Note
|
||||
`Disks` already exists and `Storage` would muddle with it, so Usage is probably the one.
|
||||
The rename touches the rail label, the `View` id, the hash route (`#/<machine>/space`,
|
||||
which people may have bookmarked — decide whether to keep `space` as the id and change
|
||||
only the label), the footer's page name and `Space.tsx`'s own strings.
|
||||
**Done 2026-09-24 — the eight Mitchell picked from the inventory**, all in the Docker tab:
|
||||
per-container **Stats**, **Health** (with the last few probe outputs), **Events** (Ham Lab
|
||||
keeps its own 4000-entry ring, since `docker events` forgets after about a minute),
|
||||
**Shell** (`docker exec`, handed to the Terminal tab) and **Limits** (`docker update` for
|
||||
memory, CPU and restart policy); an **updates** tab giving the whole machine at once with
|
||||
a per-container indicator in the list; the same across every machine, asked of the hub;
|
||||
and **schedules**, which write `/etc/cron.d/hamlab-docker` and call
|
||||
`hamlab docker-update <id>` — the shape the ZFS schedules already use.
|
||||
Also done: Ham Lab keeps working when `dockhand` is removed. It remembers where a
|
||||
container's bind mounts pointed (`mounts` in `state.json`) and resolves a stack through
|
||||
that record when no running container explains its path, running compose on the host
|
||||
instead; `repoint` does the same for a schedule at run time. Without this, removing the
|
||||
container would have made 17 stacks on `core` and 5 on `grid` unfindable at once.
|
||||
|
||||
- **ZFS gets its own tab, under Disks** — asked for 2026-09-23, not started. Today ZFS is a
|
||||
section inside `Disks.tsx`: pools with health, capacity, fragmentation, scrub controls, a
|
||||
dataset tree and per-dataset snapshot lists with "Snapshot now". Moving it out makes room
|
||||
for what is actually wanted: **scheduled snapshots with a retention policy**, configured
|
||||
from the console rather than by hand. What exists to build on — `grid` runs `zfSnap`
|
||||
nightly at 02:30 from root cron with 6-week expiry (`zfsnap-cron-6w-daily.sh`, predates
|
||||
this repo), and Ham Lab's Cron tab can already read, write and run root crontabs, so the
|
||||
job can be managed rather than reinvented. Decide: does Ham Lab drive `zfSnap`/`zfs-auto-
|
||||
snapshot` if present, or write and own its own timer? Per-dataset policies or one for the
|
||||
pool? And it must show what a retention rule *will* delete before it runs.
|
||||
|
||||
- **Rename Cron to something generic** — asked for 2026-09-23, not started. Mitchell's
|
||||
reasoning: the Services tab is systemd but "Services" is the right generic name for it,
|
||||
so the scheduling tab should be named for the job it does, not the tool. **Scheduled** or
|
||||
**Jobs**. The tab already covers more than cron — `/etc/crontab`, `/etc/cron.d`, user
|
||||
crontabs, `run-parts` directories *and* systemd timers — so the generic name is the
|
||||
honest one. Same rename surface as the Space tab: rail label, `View` id, hash route,
|
||||
footer name.
|
||||
- **Left from the Dockhand inventory, both to decide rather than build.**
|
||||
- *Git-sourced stacks.* Dockhand's `stack_sources` table points a stack at a repo and
|
||||
pulls it. Its `git_repositories` and `git_stacks` tables are **empty**, so nothing
|
||||
here uses it — dropping it is the honest answer unless Mitchell says otherwise.
|
||||
- *Webhooks, OIDC, LDAP, vulnerability scans.* All four tables empty. Nothing to move.
|
||||
- **Known gaps in the tab itself**, worth closing whether or not Dockhand goes:
|
||||
- *Removing networks* — volumes and images can be pruned and removed, networks cannot.
|
||||
- *Logs are polled with `--since` rather than followed over a held connection.* That was
|
||||
deliberate and works, but a long tail is choppier than the Services tab's live tail,
|
||||
which does hold a connection. Worth reconciling the two.
|
||||
- **The `*:2375` question, which this is the moment to settle.** A stack deployed to
|
||||
`grid` or the Mac still goes over `tcp://<node>:2375` from the hub. Decide whether the
|
||||
hub should instead ask the node's own Ham Lab to deploy locally, which would end the
|
||||
unauthenticated Docker socket exposure — its own item in `docs/pending.md`, currently
|
||||
filed under "decided, no action", and this is the change that would make closing it
|
||||
free.
|
||||
|
||||
- **A Virtualization tab** — asked for 2026-09-23, not started, and **large**. Run and
|
||||
manage **LXC containers and virtual machines** from the console, the way Proxmox does —
|
||||
@@ -531,22 +617,6 @@ something long in one, switch away and back, refresh — it's all still there.
|
||||
`internal/term`); snapshots and clones; and what it does on a host with no virtualisation
|
||||
at all, which is every host here today. Worth scoping as its own plan before starting.
|
||||
|
||||
- **Deprecate the Dashboard tab** — asked for 2026-09-23, not started. Mitchell: it is nice
|
||||
but the app repeats itself — the same reasoning that merged Top into Processes the same
|
||||
day. Dashboard is the customisable module list (CPU, memory, load, uptime, temps, fans,
|
||||
network, disks, filesystems, UPS) plus custom sensor widgets; the Processes tab shows
|
||||
most of the same numbers on one screen already. **Widgets stays**, reframed: its page should say plainly that the
|
||||
widgets there are for **external dashboards** (Glance, Home Assistant, a wall display)
|
||||
via `/embed/widget` and read-only sensor tokens, not a second copy of this console's own
|
||||
screens. Care needed on the way out: the dashboard layout is per-user in `state.json`
|
||||
(`users.<name>.dashboard`) and carries `intervalMs`, which the server reads in
|
||||
`applyPrefs` to set the sampler — that wiring has to move or the sampling rate loses its
|
||||
source. `dashboard` is also the default view after login and after switching machines.
|
||||
|
||||
- **Files explorer: cleanup and ease of use** — asked for 2026-09-23, not started.
|
||||
General polish and ease of use. The **Deprecate copyparty**
|
||||
item names the heavier pieces (multi-select and bulk actions, folder download, resumable
|
||||
uploads, a gallery, search) — worth doing the two together, since they are the same tab.
|
||||
|
||||
## Open questions
|
||||
|
||||
|
||||
@@ -4,6 +4,7 @@ package main
|
||||
import (
|
||||
"bufio"
|
||||
"context"
|
||||
"encoding/json"
|
||||
"errors"
|
||||
"flag"
|
||||
"fmt"
|
||||
@@ -31,6 +32,7 @@ import (
|
||||
"hamlab/internal/files"
|
||||
"hamlab/internal/metrics"
|
||||
"hamlab/internal/nodes"
|
||||
"hamlab/internal/priv"
|
||||
"hamlab/internal/services"
|
||||
"hamlab/internal/setup"
|
||||
"hamlab/internal/space"
|
||||
@@ -59,6 +61,8 @@ func main() {
|
||||
serve run the console in the foreground (what the service runs)
|
||||
passwd [user] set a user's password (makes the account if it is new; reads stdin when piped)
|
||||
users list the accounts
|
||||
zfs-snapshot <id> run one ZFS snapshot schedule (what its cron line calls)
|
||||
docker-update <id> run one image-update schedule (what its cron line calls)
|
||||
user-rm <user> remove an account and sign it out everywhere
|
||||
check verify the config and state load, then exit
|
||||
version
|
||||
@@ -129,6 +133,14 @@ func main() {
|
||||
if err := passwd(cfg, flag.Arg(1)); err != nil {
|
||||
log.Fatal(err)
|
||||
}
|
||||
case "docker-update":
|
||||
if err := dockerUpdateJob(cfg, flag.Arg(1)); err != nil {
|
||||
log.Fatal(err)
|
||||
}
|
||||
case "zfs-snapshot":
|
||||
if err := zfsSnapshot(cfg, flag.Arg(1)); err != nil {
|
||||
log.Fatal(err)
|
||||
}
|
||||
case "users":
|
||||
migrateUsers(cfg)
|
||||
if len(cfg.Users) == 0 {
|
||||
@@ -178,6 +190,100 @@ func main() {
|
||||
|
||||
// migrateUsers gives a pre-users config its first account, named after this OS user, so
|
||||
// the CLI sees the same accounts the daemon will.
|
||||
// dockerUpdateJob pulls a stack's images, and deploys them when the schedule says to. This
|
||||
// is what the cron line Ham Lab writes actually calls, so image updates keep happening
|
||||
// whether or not the console is up.
|
||||
func dockerUpdateJob(cfg *config.Config, id string) error {
|
||||
if strings.TrimSpace(id) == "" {
|
||||
return errors.New("usage: hamlab docker-update <schedule-id>")
|
||||
}
|
||||
st, err := store.Open(filepath.Join(cfg.DataDir(), "state.json"))
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
var raw json.RawMessage
|
||||
st.View(func(s *store.State) { raw = s.DockerJobs })
|
||||
var d struct {
|
||||
Jobs []api.UpdateJob `json:"jobs"`
|
||||
}
|
||||
if len(raw) > 0 {
|
||||
if err := json.Unmarshal(raw, &d); err != nil {
|
||||
return fmt.Errorf("reading the schedules: %w", err)
|
||||
}
|
||||
}
|
||||
for _, j := range d.Jobs {
|
||||
if j.ID != id {
|
||||
continue
|
||||
}
|
||||
// A pull that never finishes must not leave a cron job resident until the next reboot.
|
||||
ctx, cancel := context.WithTimeout(context.Background(), 2*time.Hour)
|
||||
defer cancel()
|
||||
out, runErr := api.RunUpdate(ctx, st, j, api.DockerTool())
|
||||
rec := storage.RunRecord{At: time.Now(), Taken: j.Stack}
|
||||
if runErr != nil {
|
||||
rec.Error = runErr.Error()
|
||||
}
|
||||
if err := storage.WriteRun(cfg.DataDir(), "docker-"+id, rec); err != nil {
|
||||
fmt.Fprintln(os.Stderr, "recording the run:", err)
|
||||
}
|
||||
fmt.Print(out)
|
||||
return runErr
|
||||
}
|
||||
return fmt.Errorf("no image-update schedule %q — it may have been removed; this cron line is stale", id)
|
||||
}
|
||||
|
||||
// zfsSnapshot runs one snapshot schedule: take it, then expire what has aged out. This is
|
||||
// what the cron line Ham Lab writes actually calls, so a schedule keeps running whether or
|
||||
// not the console is up — and keeps running if Ham Lab is never opened again.
|
||||
//
|
||||
// It deliberately does not write state.json. The daemon owns that file and would clobber
|
||||
// anything written from out here; the outcome goes beside it instead, in zfs-runs/.
|
||||
func zfsSnapshot(cfg *config.Config, id string) error {
|
||||
if strings.TrimSpace(id) == "" {
|
||||
return errors.New("usage: hamlab zfs-snapshot <schedule-id>")
|
||||
}
|
||||
st, err := store.Open(filepath.Join(cfg.DataDir(), "state.json"))
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
var raw json.RawMessage
|
||||
st.View(func(s *store.State) { raw = s.ZFS })
|
||||
var z struct {
|
||||
Policies []storage.Policy `json:"policies"`
|
||||
}
|
||||
if len(raw) > 0 {
|
||||
if err := json.Unmarshal(raw, &z); err != nil {
|
||||
return fmt.Errorf("reading the schedules: %w", err)
|
||||
}
|
||||
}
|
||||
var p *storage.Policy
|
||||
for i := range z.Policies {
|
||||
if z.Policies[i].ID == id {
|
||||
p = &z.Policies[i]
|
||||
}
|
||||
}
|
||||
if p == nil {
|
||||
return fmt.Errorf("no snapshot schedule %q — it may have been removed; this cron line is stale", id)
|
||||
}
|
||||
res, runErr := storage.New(priv.Available()).Run(context.Background(), p)
|
||||
rec := storage.RunRecord{At: time.Now(), Taken: res.Taken, Pruned: res.Pruned}
|
||||
if runErr != nil {
|
||||
rec.Error = runErr.Error()
|
||||
}
|
||||
if err := storage.WriteRun(cfg.DataDir(), id, rec); err != nil {
|
||||
fmt.Fprintln(os.Stderr, "recording the run:", err)
|
||||
}
|
||||
if runErr != nil {
|
||||
return runErr
|
||||
}
|
||||
fmt.Printf("%s: took %s", p.Name, res.Taken)
|
||||
if len(res.Pruned) > 0 {
|
||||
fmt.Printf(", expired %d", len(res.Pruned))
|
||||
}
|
||||
fmt.Println()
|
||||
return nil
|
||||
}
|
||||
|
||||
func migrateUsers(cfg *config.Config) {
|
||||
name := os.Getenv("USER")
|
||||
if u, err := user.Current(); err == nil && u.Username != "" {
|
||||
|
||||
@@ -49,6 +49,10 @@ func dockerTool() string {
|
||||
}
|
||||
|
||||
// dockerRun runs the tool as this user, and again as root if the socket said no.
|
||||
// DockerTool is whichever of docker or podman this machine has, for callers outside the
|
||||
// package — the scheduled updates the CLI runs.
|
||||
func DockerTool() string { return dockerTool() }
|
||||
|
||||
func dockerRun(ctx context.Context, args ...string) ([]byte, error) {
|
||||
tool := dockerTool()
|
||||
if tool == "" {
|
||||
@@ -394,3 +398,161 @@ func (s *Server) volumePrune(w http.ResponseWriter, r *http.Request) {
|
||||
log.Printf("docker: volume prune (from %s)", clientIP(r))
|
||||
jsonOK(w, 200, map[string]string{"result": lines[len(lines)-1]})
|
||||
}
|
||||
|
||||
// ---- a container's health, its recent life, its console, and its limits ----
|
||||
|
||||
// dockerHealth is the health check's verdict and the last few probe results. Docker keeps
|
||||
// the last five itself, which is all anyone wants: is it failing, and what did it say.
|
||||
func (s *Server) dockerHealth(w http.ResponseWriter, r *http.Request) {
|
||||
id := r.PathValue("id")
|
||||
if !dockerRef.MatchString(id) {
|
||||
jsonErr(w, 400, "bad container")
|
||||
return
|
||||
}
|
||||
ctx, cancel := context.WithTimeout(r.Context(), 15*time.Second)
|
||||
defer cancel()
|
||||
out, err := dockerRun(ctx, "inspect", "--format", "{{json .State}}", id)
|
||||
if err != nil {
|
||||
jsonErr(w, 500, err.Error())
|
||||
return
|
||||
}
|
||||
var st struct {
|
||||
Status string `json:"Status"`
|
||||
Health *struct {
|
||||
Status string `json:"Status"`
|
||||
FailingStreak int `json:"FailingStreak"`
|
||||
Log []struct {
|
||||
Start time.Time `json:"Start"`
|
||||
End time.Time `json:"End"`
|
||||
ExitCode int `json:"ExitCode"`
|
||||
Output string `json:"Output"`
|
||||
} `json:"Log"`
|
||||
} `json:"Health"`
|
||||
}
|
||||
if json.Unmarshal(out, &st) != nil {
|
||||
jsonErr(w, 500, "could not read the container's state")
|
||||
return
|
||||
}
|
||||
if st.Health == nil {
|
||||
jsonOK(w, 200, map[string]any{"configured": false, "state": st.Status})
|
||||
return
|
||||
}
|
||||
jsonOK(w, 200, map[string]any{
|
||||
"configured": true, "state": st.Status, "status": st.Health.Status,
|
||||
"failing_streak": st.Health.FailingStreak, "log": st.Health.Log,
|
||||
})
|
||||
}
|
||||
|
||||
// dockerExec opens a shell inside a container, as a terminal session like any other — so
|
||||
// it reattaches, resizes and scrolls back the same way, and works on a managed machine
|
||||
// through the same proxy.
|
||||
func (s *Server) dockerExec(w http.ResponseWriter, r *http.Request) {
|
||||
id := r.PathValue("id")
|
||||
if !dockerRef.MatchString(id) {
|
||||
jsonErr(w, 400, "bad container")
|
||||
return
|
||||
}
|
||||
tool := dockerTool()
|
||||
if tool == "" {
|
||||
jsonErr(w, 500, "neither docker nor podman is installed here")
|
||||
return
|
||||
}
|
||||
var body struct {
|
||||
Shell string `json:"shell"`
|
||||
}
|
||||
_ = json.NewDecoder(io.LimitReader(r.Body, 4096)).Decode(&body)
|
||||
shell := strings.TrimSpace(body.Shell)
|
||||
if shell == "" {
|
||||
// most images have sh; bash is nicer where it exists, so try it first and fall back
|
||||
shell = "if command -v bash >/dev/null 2>&1; then exec bash; else exec sh; fi"
|
||||
} else if strings.ContainsAny(shell, "\n\r") {
|
||||
jsonErr(w, 400, "bad shell")
|
||||
return
|
||||
}
|
||||
argv := []string{tool, "exec", "-it", id, "sh", "-c", shell}
|
||||
if priv.Available() && !dockerReachable() {
|
||||
argv = priv.Wrap(argv...)
|
||||
}
|
||||
sess, err := s.mgr.CreateCmd("", "sh "+shortName(id), argv)
|
||||
if err != nil {
|
||||
jsonErr(w, 500, err.Error())
|
||||
return
|
||||
}
|
||||
log.Printf("docker exec into %s as terminal %s by %s", id, sess.ID, clientIP(r))
|
||||
jsonOK(w, 200, map[string]string{"id": sess.ID, "name": "sh " + shortName(id)})
|
||||
}
|
||||
|
||||
func shortName(id string) string {
|
||||
if len(id) > 20 {
|
||||
return id[:20]
|
||||
}
|
||||
return id
|
||||
}
|
||||
|
||||
// dockerReachable reports whether docker answers without help.
|
||||
func dockerReachable() bool {
|
||||
tool := dockerTool()
|
||||
if tool == "" {
|
||||
return false
|
||||
}
|
||||
ctx, cancel := context.WithTimeout(context.Background(), 5*time.Second)
|
||||
defer cancel()
|
||||
return exec.CommandContext(ctx, tool, "version", "--format", "{{.Server.Version}}").Run() == nil
|
||||
}
|
||||
|
||||
// dockerUpdate changes a running container's limits and restart policy. `docker update`
|
||||
// does this live, without recreating it and without touching the compose file — which is
|
||||
// also the catch, and the caller is told: a deploy from compose puts back whatever compose
|
||||
// says.
|
||||
func (s *Server) dockerUpdate(w http.ResponseWriter, r *http.Request) {
|
||||
id := r.PathValue("id")
|
||||
if !dockerRef.MatchString(id) {
|
||||
jsonErr(w, 400, "bad container")
|
||||
return
|
||||
}
|
||||
var body struct {
|
||||
Memory string `json:"memory"` // "512m", "2g", or "" to leave alone; "0" removes the limit
|
||||
CPUs string `json:"cpus"` // "1.5", or "0" for no limit
|
||||
Restart string `json:"restart"` // no | on-failure | unless-stopped | always
|
||||
}
|
||||
if err := json.NewDecoder(io.LimitReader(r.Body, 4096)).Decode(&body); err != nil {
|
||||
jsonErr(w, 400, "bad request")
|
||||
return
|
||||
}
|
||||
args := []string{"update"}
|
||||
if m := strings.TrimSpace(body.Memory); m != "" {
|
||||
if !regexp.MustCompile(`^[0-9]+([bkmgBKMG])?$`).MatchString(m) {
|
||||
jsonErr(w, 400, "memory: a number, optionally with b, k, m or g — 0 removes the limit")
|
||||
return
|
||||
}
|
||||
args = append(args, "--memory", m, "--memory-swap", m)
|
||||
}
|
||||
if c := strings.TrimSpace(body.CPUs); c != "" {
|
||||
if !regexp.MustCompile(`^[0-9]+(\.[0-9]+)?$`).MatchString(c) {
|
||||
jsonErr(w, 400, "cpus: a number like 1.5 — 0 removes the limit")
|
||||
return
|
||||
}
|
||||
args = append(args, "--cpus", c)
|
||||
}
|
||||
if p := strings.TrimSpace(body.Restart); p != "" {
|
||||
switch p {
|
||||
case "no", "on-failure", "unless-stopped", "always":
|
||||
default:
|
||||
jsonErr(w, 400, "restart: no, on-failure, unless-stopped or always")
|
||||
return
|
||||
}
|
||||
args = append(args, "--restart", p)
|
||||
}
|
||||
if len(args) == 1 {
|
||||
jsonErr(w, 400, "nothing to change")
|
||||
return
|
||||
}
|
||||
ctx, cancel := context.WithTimeout(r.Context(), 30*time.Second)
|
||||
defer cancel()
|
||||
if _, err := dockerRun(ctx, append(args, id)...); err != nil {
|
||||
jsonErr(w, 500, err.Error())
|
||||
return
|
||||
}
|
||||
log.Printf("docker update %s %v by %s", id, args[1:], clientIP(r))
|
||||
w.WriteHeader(204)
|
||||
}
|
||||
|
||||
@@ -0,0 +1,229 @@
|
||||
package api
|
||||
|
||||
// A container's history.
|
||||
//
|
||||
// `docker events` is a live stream with only a short in-memory backlog — ask it for the
|
||||
// last thirty days and you get the last few minutes, because the daemon keeps events in a
|
||||
// small ring and loses them all when it restarts. That is why Dockhand kept its own copy,
|
||||
// and why this does too: a watcher follows the stream and appends the lifecycle events to a
|
||||
// ring kept beside the config, so "what happened to this container last Tuesday" has an
|
||||
// answer.
|
||||
//
|
||||
// Only lifecycle events are kept. A health check produces three exec events every probe,
|
||||
// which is thousands a day and tells you nothing that the health panel does not.
|
||||
|
||||
import (
|
||||
"bufio"
|
||||
"context"
|
||||
"encoding/json"
|
||||
"log"
|
||||
"net/http"
|
||||
"os"
|
||||
"os/exec"
|
||||
"path/filepath"
|
||||
"sort"
|
||||
"strings"
|
||||
"sync"
|
||||
"time"
|
||||
|
||||
"hamlab/internal/priv"
|
||||
)
|
||||
|
||||
const (
|
||||
eventsKept = 4000
|
||||
eventsSaveEvery = 30 * time.Second
|
||||
)
|
||||
|
||||
// ContainerEvent is one thing that happened to one container.
|
||||
type ContainerEvent struct {
|
||||
T int64 `json:"t"` // unix ms
|
||||
Name string `json:"name"`
|
||||
Action string `json:"action"`
|
||||
Image string `json:"image,omitempty"`
|
||||
ExitCode string `json:"exit_code,omitempty"`
|
||||
Signal string `json:"signal,omitempty"`
|
||||
Health string `json:"health,omitempty"`
|
||||
}
|
||||
|
||||
type eventLog struct {
|
||||
mu sync.Mutex
|
||||
ring []ContainerEvent
|
||||
dirty bool
|
||||
path string
|
||||
}
|
||||
|
||||
func (e *eventLog) load(path string) {
|
||||
e.mu.Lock()
|
||||
defer e.mu.Unlock()
|
||||
e.path = path
|
||||
b, err := os.ReadFile(path)
|
||||
if err != nil {
|
||||
return
|
||||
}
|
||||
var list []ContainerEvent
|
||||
if json.Unmarshal(b, &list) == nil {
|
||||
e.ring = list
|
||||
}
|
||||
}
|
||||
|
||||
func (e *eventLog) add(ev ContainerEvent) {
|
||||
e.mu.Lock()
|
||||
defer e.mu.Unlock()
|
||||
e.ring = append(e.ring, ev)
|
||||
if len(e.ring) > eventsKept {
|
||||
e.ring = append([]ContainerEvent(nil), e.ring[len(e.ring)-eventsKept:]...)
|
||||
}
|
||||
e.dirty = true
|
||||
}
|
||||
|
||||
// forName is one container's events, newest first.
|
||||
func (e *eventLog) forName(name string, since time.Time, limit int) []ContainerEvent {
|
||||
e.mu.Lock()
|
||||
defer e.mu.Unlock()
|
||||
out := []ContainerEvent{}
|
||||
for _, ev := range e.ring {
|
||||
if ev.Name == name && ev.T >= since.UnixMilli() {
|
||||
out = append(out, ev)
|
||||
}
|
||||
}
|
||||
sort.Slice(out, func(i, j int) bool { return out[i].T > out[j].T })
|
||||
if len(out) > limit {
|
||||
out = out[:limit]
|
||||
}
|
||||
return out
|
||||
}
|
||||
|
||||
func (e *eventLog) save() {
|
||||
e.mu.Lock()
|
||||
if !e.dirty || e.path == "" {
|
||||
e.mu.Unlock()
|
||||
return
|
||||
}
|
||||
b, err := json.Marshal(e.ring)
|
||||
e.dirty = false
|
||||
path := e.path
|
||||
e.mu.Unlock()
|
||||
if err != nil {
|
||||
return
|
||||
}
|
||||
_ = os.MkdirAll(filepath.Dir(path), 0o700)
|
||||
tmp := path + ".tmp"
|
||||
if os.WriteFile(tmp, b, 0o600) == nil {
|
||||
_ = os.Rename(tmp, path)
|
||||
}
|
||||
}
|
||||
|
||||
// watchDockerEvents follows the stream, restarting if Docker goes away. It is the only
|
||||
// thing that writes the log, so nothing contends with it.
|
||||
func (s *Server) watchDockerEvents(ctx context.Context) {
|
||||
s.events.load(filepath.Join(s.cfg.DataDir(), "docker-events.json"))
|
||||
go func() { // a periodic flush, so a crash loses at most half a minute of history
|
||||
t := time.NewTicker(eventsSaveEvery)
|
||||
defer t.Stop()
|
||||
for {
|
||||
select {
|
||||
case <-ctx.Done():
|
||||
s.events.save()
|
||||
return
|
||||
case <-t.C:
|
||||
s.events.save()
|
||||
}
|
||||
}
|
||||
}()
|
||||
for {
|
||||
if ctx.Err() != nil {
|
||||
return
|
||||
}
|
||||
if dockerTool() == "" {
|
||||
time.Sleep(time.Minute) // no Docker here; look again later in case it is installed
|
||||
continue
|
||||
}
|
||||
s.followEvents(ctx)
|
||||
select {
|
||||
case <-ctx.Done():
|
||||
return
|
||||
case <-time.After(10 * time.Second): // the daemon restarted, or the pipe broke
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func (s *Server) followEvents(ctx context.Context) {
|
||||
args := []string{"events", "--format", "{{json .}}", "--filter", "type=container"}
|
||||
for _, e := range []string{"create", "start", "restart", "stop", "kill", "die", "destroy", "oom", "pause", "unpause", "rename", "update", "health_status"} {
|
||||
args = append(args, "--filter", "event="+e)
|
||||
}
|
||||
tool := dockerTool()
|
||||
full := append([]string{tool}, args...)
|
||||
if !dockerReachable() && priv.Available() {
|
||||
full = priv.Wrap(full...)
|
||||
}
|
||||
cmd := exec.CommandContext(ctx, full[0], full[1:]...)
|
||||
out, err := cmd.StdoutPipe()
|
||||
if err != nil {
|
||||
return
|
||||
}
|
||||
if err := cmd.Start(); err != nil {
|
||||
return
|
||||
}
|
||||
sc := bufio.NewScanner(out)
|
||||
sc.Buffer(make([]byte, 64<<10), 1<<20)
|
||||
n := 0
|
||||
for sc.Scan() {
|
||||
var e struct {
|
||||
Action string `json:"Action"`
|
||||
Time int64 `json:"time"`
|
||||
TimeNano int64 `json:"timeNano"`
|
||||
Actor struct {
|
||||
Attributes map[string]string `json:"Attributes"`
|
||||
} `json:"Actor"`
|
||||
}
|
||||
if json.Unmarshal(sc.Bytes(), &e) != nil || e.Action == "" {
|
||||
continue
|
||||
}
|
||||
at := e.Actor.Attributes
|
||||
ev := ContainerEvent{
|
||||
T: e.Time * 1000, Name: at["name"], Image: at["image"],
|
||||
Action: e.Action, ExitCode: at["exitCode"], Signal: at["signal"],
|
||||
}
|
||||
if e.TimeNano > 0 {
|
||||
ev.T = e.TimeNano / 1e6
|
||||
}
|
||||
// health_status comes through as "health_status: healthy"
|
||||
if a, h, ok := strings.Cut(e.Action, ": "); ok {
|
||||
ev.Action, ev.Health = a, h
|
||||
}
|
||||
if ev.Name == "" {
|
||||
continue
|
||||
}
|
||||
s.events.add(ev)
|
||||
n++
|
||||
}
|
||||
_ = cmd.Wait()
|
||||
if n > 0 {
|
||||
log.Printf("docker events: stream ended after %d events; reconnecting", n)
|
||||
}
|
||||
}
|
||||
|
||||
// dockerEvents answers from Ham Lab's own history, not Docker's short memory.
|
||||
func (s *Server) dockerEvents(w http.ResponseWriter, r *http.Request) {
|
||||
id := r.PathValue("id")
|
||||
if !dockerRef.MatchString(id) {
|
||||
jsonErr(w, 400, "bad container")
|
||||
return
|
||||
}
|
||||
since := r.URL.Query().Get("since")
|
||||
d := 7 * 24 * time.Hour
|
||||
switch since {
|
||||
case "24h":
|
||||
d = 24 * time.Hour
|
||||
case "30d":
|
||||
d = 30 * 24 * time.Hour
|
||||
default:
|
||||
since = "7d"
|
||||
}
|
||||
ev := s.events.forName(id, time.Now().Add(-d), 500)
|
||||
s.events.mu.Lock()
|
||||
total := len(s.events.ring)
|
||||
s.events.mu.Unlock()
|
||||
jsonOK(w, 200, map[string]any{"events": ev, "since": since, "kept": total, "capacity": eventsKept})
|
||||
}
|
||||
@@ -0,0 +1,351 @@
|
||||
package api
|
||||
|
||||
// Scheduled image updates.
|
||||
//
|
||||
// Dockhand kept three containers up to date nightly — unbound, nginx and pihole, which is
|
||||
// DNS and ingress — and that is the one thing it does that Ham Lab did not. This replaces
|
||||
// it, and deliberately in the same shape as the ZFS snapshot schedules: a real line in a
|
||||
// cron file Ham Lab owns, calling the binary, rather than a ticker inside the daemon.
|
||||
// Updating an image is not something that should stop happening because a web console is
|
||||
// down, and a cron line is visible to anyone looking at the machine the normal way.
|
||||
//
|
||||
// A schedule stores everything the run needs — project, compose files, how compose has to
|
||||
// be invoked for that stack — so the cron job is self-contained and does not have to
|
||||
// rediscover the world before it can act.
|
||||
|
||||
import (
|
||||
"context"
|
||||
"crypto/rand"
|
||||
"encoding/hex"
|
||||
"encoding/json"
|
||||
"errors"
|
||||
"io"
|
||||
"log"
|
||||
"net/http"
|
||||
"os"
|
||||
"path/filepath"
|
||||
"strings"
|
||||
"time"
|
||||
|
||||
"hamlab/internal/compose"
|
||||
"hamlab/internal/cron"
|
||||
"hamlab/internal/storage"
|
||||
"hamlab/internal/store"
|
||||
)
|
||||
|
||||
// UpdateJob is a standing instruction to pull a stack's images, and optionally deploy them.
|
||||
type UpdateJob struct {
|
||||
ID string `json:"id"`
|
||||
Name string `json:"name"`
|
||||
Stack string `json:"stack"`
|
||||
Machine string `json:"machine,omitempty"` // for display; the job runs where it was made
|
||||
// Services narrows it to some of the stack's services; empty is all of them.
|
||||
Services []string `json:"services,omitempty"`
|
||||
Cron string `json:"cron"`
|
||||
// Apply says whether a newer image is deployed or only fetched. Check-only still pulls,
|
||||
// so the Docker tab shows what is behind without a registry round trip.
|
||||
Apply bool `json:"apply"`
|
||||
Enabled bool `json:"enabled"`
|
||||
|
||||
// How to run compose for this stack, captured when the schedule was made.
|
||||
Spec composeSpec `json:"spec"`
|
||||
}
|
||||
|
||||
type composeSpec struct {
|
||||
Project string `json:"project"`
|
||||
Files []string `json:"files"`
|
||||
Via string `json:"via,omitempty"`
|
||||
Remote string `json:"remote,omitempty"`
|
||||
}
|
||||
|
||||
func (c composeSpec) toSpec(tool string) compose.Spec {
|
||||
return compose.Spec{Project: c.Project, Files: c.Files, Via: c.Via, Remote: c.Remote, Tool: tool}
|
||||
}
|
||||
|
||||
const dockerCronFile = "/etc/cron.d/hamlab-docker"
|
||||
|
||||
type dockerSchedState struct {
|
||||
Jobs []UpdateJob `json:"jobs"`
|
||||
}
|
||||
|
||||
func (s *Server) dsLoad() dockerSchedState {
|
||||
var raw json.RawMessage
|
||||
s.store.View(func(st *store.State) { raw = st.DockerJobs })
|
||||
var d dockerSchedState
|
||||
if len(raw) > 0 {
|
||||
_ = json.Unmarshal(raw, &d)
|
||||
}
|
||||
if d.Jobs == nil {
|
||||
d.Jobs = []UpdateJob{}
|
||||
}
|
||||
return d
|
||||
}
|
||||
|
||||
func (s *Server) dsSave(d dockerSchedState) error {
|
||||
b, err := json.Marshal(d)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
return s.store.Update(func(st *store.State) { st.DockerJobs = b })
|
||||
}
|
||||
|
||||
// dsCronText renders the schedules as cron lines. Same shape as the ZFS file: Ham Lab owns
|
||||
// it whole and rewrites it on every change.
|
||||
func dsCronText(jobs []UpdateJob, user, exe, config string) (string, error) {
|
||||
var b strings.Builder
|
||||
b.WriteString("# Ham Lab image-update schedules — written by the Docker tab, rewritten whole on every change.\n")
|
||||
b.WriteString("# Edit them there, not here: anything added by hand is lost the next time a schedule changes.\n")
|
||||
b.WriteString("SHELL=/bin/sh\nPATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin\n\n")
|
||||
for i := range jobs {
|
||||
j := &jobs[i]
|
||||
if !j.Enabled {
|
||||
b.WriteString("# " + j.Name + " — paused\n")
|
||||
continue
|
||||
}
|
||||
expr := strings.Join(strings.Fields(j.Cron), " ")
|
||||
if err := cron.ValidSchedule(expr); err != nil {
|
||||
return "", err
|
||||
}
|
||||
what := "pull only"
|
||||
if j.Apply {
|
||||
what = "pull and deploy"
|
||||
}
|
||||
b.WriteString("# " + j.Name + " — " + j.Stack + ", " + what + "\n")
|
||||
b.WriteString(expr + " " + user + " " + exe + " -config " + config + " docker-update " + j.ID + "\n\n")
|
||||
}
|
||||
return b.String(), nil
|
||||
}
|
||||
|
||||
func (s *Server) dsSyncCron(ctx context.Context, d dockerSchedState) error {
|
||||
live := false
|
||||
for i := range d.Jobs {
|
||||
if d.Jobs[i].Enabled {
|
||||
live = true
|
||||
}
|
||||
}
|
||||
if !live {
|
||||
return s.storage.RemoveRootFile(ctx, dockerCronFile)
|
||||
}
|
||||
exe, err := os.Executable()
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
exe, _ = filepath.EvalSymlinks(exe)
|
||||
text, err := dsCronText(d.Jobs, s.user, exe, s.cfg.Path())
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
return s.storage.WriteRootFile(ctx, dockerCronFile, text)
|
||||
}
|
||||
|
||||
// repoint keeps a schedule working when what deployed its stack has been removed.
|
||||
//
|
||||
// A schedule captures how to invoke compose at the moment it is made, so the cron job never
|
||||
// has to rediscover the world at 3am. But "run compose inside this container" stops being
|
||||
// true the day that container is deleted — which is exactly what happens when an
|
||||
// orchestrator is retired, and would otherwise break every schedule made before it went.
|
||||
// So: if the container named is not running, the files are looked up again, through live
|
||||
// mounts and remembered ones both, and compose runs on the host instead.
|
||||
func repoint(ctx context.Context, db *store.Store, spec compose.Spec) compose.Spec {
|
||||
if spec.Via == "" || db == nil {
|
||||
return spec
|
||||
}
|
||||
if out, err := dockerRun(ctx, "inspect", "-f", "{{.State.Running}}", spec.Via); err == nil && strings.TrimSpace(string(out)) == "true" {
|
||||
return spec
|
||||
}
|
||||
st := stack{Name: spec.Project, File: spec.Files[0], Files: spec.Files}
|
||||
st.resolve(mountsSeen(ctx, db)) // read-only: this also runs from cron, where the daemon owns state.json
|
||||
if !st.Editable || st.Via != "" {
|
||||
return spec // still only reachable through a container, or not reachable at all
|
||||
}
|
||||
log.Printf("docker schedule %s: %s is gone, running compose here against %s", spec.Project, spec.Via, st.HostFiles[0])
|
||||
spec.Files, spec.Via = st.HostFiles, ""
|
||||
return spec
|
||||
}
|
||||
|
||||
// RunUpdate pulls, and deploys when the schedule says to. Shared with the CLI so the button
|
||||
// and the cron line cannot drift apart.
|
||||
func RunUpdate(ctx context.Context, db *store.Store, j UpdateJob, tool string) (string, error) {
|
||||
if len(j.Spec.Files) == 0 {
|
||||
return "", errors.New("this schedule has no compose file recorded — open it in the Docker tab and save it again")
|
||||
}
|
||||
spec := repoint(ctx, db, j.Spec.toSpec(tool))
|
||||
var log strings.Builder
|
||||
run := func(args ...string) error {
|
||||
cmd := compose.Cmd(ctx, spec, args...)
|
||||
out, err := cmd.CombinedOutput()
|
||||
log.Write(out)
|
||||
return err
|
||||
}
|
||||
args := append([]string{"pull"}, j.Services...)
|
||||
if err := run(args...); err != nil {
|
||||
return log.String(), err
|
||||
}
|
||||
if !j.Apply {
|
||||
return log.String(), nil
|
||||
}
|
||||
up := append([]string{"up", "-d"}, j.Services...)
|
||||
if err := run(up...); err != nil {
|
||||
return log.String(), err
|
||||
}
|
||||
return log.String(), nil
|
||||
}
|
||||
|
||||
// ---- endpoints ----
|
||||
|
||||
func (s *Server) dsList(w http.ResponseWriter, _ *http.Request) {
|
||||
d := s.dsLoad()
|
||||
now := time.Now()
|
||||
out := make([]map[string]any, 0, len(d.Jobs))
|
||||
for _, j := range d.Jobs {
|
||||
m := map[string]any{"job": j, "last": storage.ReadRun(s.cfg.DataDir(), "docker-"+j.ID)}
|
||||
if j.Enabled {
|
||||
if n, ok := cron.NextRun(strings.Join(strings.Fields(j.Cron), " "), now); ok {
|
||||
m["next"] = n
|
||||
}
|
||||
}
|
||||
out = append(out, m)
|
||||
}
|
||||
jsonOK(w, 200, map[string]any{"jobs": out, "file": dockerCronFile})
|
||||
}
|
||||
|
||||
func (s *Server) dsPut(w http.ResponseWriter, r *http.Request) {
|
||||
var j UpdateJob
|
||||
if err := json.NewDecoder(io.LimitReader(r.Body, 32<<10)).Decode(&j); err != nil {
|
||||
jsonErr(w, 400, "bad request")
|
||||
return
|
||||
}
|
||||
j.Cron = strings.Join(strings.Fields(j.Cron), " ")
|
||||
if err := cron.ValidSchedule(j.Cron); err != nil {
|
||||
jsonErr(w, 400, "schedule: "+err.Error())
|
||||
return
|
||||
}
|
||||
if strings.TrimSpace(j.Stack) == "" {
|
||||
jsonErr(w, 400, "which stack?")
|
||||
return
|
||||
}
|
||||
// resolve the stack now, so the cron job never has to
|
||||
ctx, cancel := context.WithTimeout(r.Context(), 60*time.Second)
|
||||
defer cancel()
|
||||
stacks, err := s.findStacks(ctx)
|
||||
if err != nil {
|
||||
jsonErr(w, 500, err.Error())
|
||||
return
|
||||
}
|
||||
found := false
|
||||
for _, st := range stacks {
|
||||
if st.Name == j.Stack {
|
||||
// seenFiles is the list as compose itself sees it: host paths normally, the
|
||||
// orchestrator container's own paths when compose runs inside one. Getting this
|
||||
// backwards gives a cron line that works from the console and fails at 3am.
|
||||
j.Spec = composeSpec{Project: st.Name, Files: seenFiles(st, -1, ""), Via: st.Via, Remote: st.Remote}
|
||||
if len(j.Spec.Files) == 0 {
|
||||
jsonErr(w, 409, "that stack's compose file isn't anywhere this console can reach")
|
||||
return
|
||||
}
|
||||
found = true
|
||||
}
|
||||
}
|
||||
if !found {
|
||||
jsonErr(w, 404, "no stack called "+j.Stack+" on this machine")
|
||||
return
|
||||
}
|
||||
if strings.TrimSpace(j.Name) == "" {
|
||||
j.Name = j.Stack
|
||||
}
|
||||
d := s.dsLoad()
|
||||
replaced := false
|
||||
for i := range d.Jobs {
|
||||
if d.Jobs[i].ID == j.ID && j.ID != "" {
|
||||
d.Jobs[i] = j
|
||||
replaced = true
|
||||
}
|
||||
}
|
||||
if !replaced {
|
||||
b := make([]byte, 6)
|
||||
if _, err := rand.Read(b); err != nil {
|
||||
jsonErr(w, 500, err.Error())
|
||||
return
|
||||
}
|
||||
j.ID = hex.EncodeToString(b)
|
||||
d.Jobs = append(d.Jobs, j)
|
||||
}
|
||||
if err := s.dsSave(d); err != nil {
|
||||
jsonErr(w, 500, err.Error())
|
||||
return
|
||||
}
|
||||
if err := s.dsSyncCron(r.Context(), d); err != nil {
|
||||
jsonErr(w, 500, "saved, but writing "+dockerCronFile+" failed: "+err.Error())
|
||||
return
|
||||
}
|
||||
log.Printf("docker schedule %q saved by %s (%s, %q, %s)", j.Name, clientIP(r), j.Stack, j.Cron, map[bool]string{true: "pull and deploy", false: "pull only"}[j.Apply])
|
||||
jsonOK(w, 200, j)
|
||||
}
|
||||
|
||||
func (s *Server) dsDelete(w http.ResponseWriter, r *http.Request) {
|
||||
id := r.PathValue("id")
|
||||
d := s.dsLoad()
|
||||
kept := d.Jobs[:0]
|
||||
found := false
|
||||
for _, j := range d.Jobs {
|
||||
if j.ID == id {
|
||||
found = true
|
||||
continue
|
||||
}
|
||||
kept = append(kept, j)
|
||||
}
|
||||
if !found {
|
||||
jsonErr(w, 404, "no such schedule")
|
||||
return
|
||||
}
|
||||
d.Jobs = kept
|
||||
if err := s.dsSave(d); err != nil {
|
||||
jsonErr(w, 500, err.Error())
|
||||
return
|
||||
}
|
||||
if err := s.dsSyncCron(r.Context(), d); err != nil {
|
||||
jsonErr(w, 500, "removed, but writing "+dockerCronFile+" failed: "+err.Error())
|
||||
return
|
||||
}
|
||||
storage.ForgetRun(s.cfg.DataDir(), "docker-"+id)
|
||||
log.Printf("docker schedule %s removed by %s", id, clientIP(r))
|
||||
w.WriteHeader(204)
|
||||
}
|
||||
|
||||
func (s *Server) dsRun(w http.ResponseWriter, r *http.Request) {
|
||||
id := r.PathValue("id")
|
||||
var job UpdateJob
|
||||
found := false
|
||||
for _, j := range s.dsLoad().Jobs {
|
||||
if j.ID == id {
|
||||
job, found = j, true
|
||||
}
|
||||
}
|
||||
if !found {
|
||||
jsonErr(w, 404, "no such schedule")
|
||||
return
|
||||
}
|
||||
// Outliving the request: a pull the browser gave up on still finishes, but not forever.
|
||||
ctx, cancel := context.WithTimeout(context.WithoutCancel(r.Context()), 30*time.Minute)
|
||||
defer cancel()
|
||||
out, err := RunUpdate(ctx, s.store, job, dockerTool())
|
||||
rec := storage.RunRecord{At: time.Now(), Taken: job.Stack}
|
||||
if err != nil {
|
||||
rec.Error = err.Error()
|
||||
}
|
||||
_ = storage.WriteRun(s.cfg.DataDir(), "docker-"+id, rec)
|
||||
stackUpdatesForget("/" + job.Stack)
|
||||
if err != nil {
|
||||
jsonErr(w, 500, err.Error()+"\n"+tail(out, 2000))
|
||||
return
|
||||
}
|
||||
log.Printf("docker schedule %q run by %s", job.Name, clientIP(r))
|
||||
jsonOK(w, 200, map[string]string{"output": tail(out, 4000)})
|
||||
}
|
||||
|
||||
func tail(s string, n int) string {
|
||||
if len(s) <= n {
|
||||
return s
|
||||
}
|
||||
return "…" + s[len(s)-n:]
|
||||
}
|
||||
@@ -0,0 +1,310 @@
|
||||
package api
|
||||
|
||||
// IP-KVMs: a device's own web page, shown inside Ham Lab.
|
||||
//
|
||||
// The whole difficulty is that Ham Lab is served over TLS and a KVM usually is not, so a
|
||||
// browser refuses to put the device's page in an iframe — mixed content, blocked before
|
||||
// anything loads. The fix is to serve the device from Ham Lab's own origin, which also
|
||||
// means it inherits Ham Lab's login instead of needing a second one, and makes the iframe
|
||||
// same-origin.
|
||||
//
|
||||
// Proxying an arbitrary web app under a path prefix is the hard part, and nothing here
|
||||
// knows anything about any particular device. A device UI asks for its assets by absolute
|
||||
// path — "/assets/app.js", "/api/whatever" — which a prefix cannot rewrite away. So the
|
||||
// prefix is only the entry point: any request Ham Lab does not serve itself is routed to
|
||||
// whichever KVM the asking page belongs to, worked out from the Referer, and from a cookie
|
||||
// when there is no Referer (a WebSocket upgrade, say). Ham Lab's own routes always win, so
|
||||
// a device can never shadow the console.
|
||||
|
||||
import (
|
||||
"crypto/rand"
|
||||
"encoding/hex"
|
||||
"encoding/json"
|
||||
"errors"
|
||||
"io"
|
||||
"log"
|
||||
"net/http"
|
||||
"net/http/httputil"
|
||||
"net/url"
|
||||
"strings"
|
||||
"sync"
|
||||
"time"
|
||||
|
||||
"hamlab/internal/config"
|
||||
)
|
||||
|
||||
const kvmPrefix = "/kvm/"
|
||||
const kvmCookie = "hamlab_kvm"
|
||||
|
||||
type kvmProxies struct {
|
||||
mu sync.Mutex
|
||||
m map[string]*httputil.ReverseProxy
|
||||
}
|
||||
|
||||
func (s *Server) kvm(id string) (config.KVM, bool) {
|
||||
var found config.KVM
|
||||
ok := false
|
||||
s.cfg.View(func(c *config.Config) {
|
||||
for _, k := range c.KVMs {
|
||||
if k.ID == id {
|
||||
found, ok = k, true
|
||||
}
|
||||
}
|
||||
})
|
||||
return found, ok
|
||||
}
|
||||
|
||||
// kvmProxy is the reverse proxy for one device, made once and kept.
|
||||
func (s *Server) kvmProxy(k config.KVM) (*httputil.ReverseProxy, error) {
|
||||
s.kvms.mu.Lock()
|
||||
defer s.kvms.mu.Unlock()
|
||||
if s.kvms.m == nil {
|
||||
s.kvms.m = map[string]*httputil.ReverseProxy{}
|
||||
}
|
||||
if p, ok := s.kvms.m[k.ID]; ok {
|
||||
return p, nil
|
||||
}
|
||||
target, err := url.Parse(k.URL)
|
||||
if err != nil || target.Host == "" {
|
||||
return nil, errors.New("this KVM's address is not a URL")
|
||||
}
|
||||
prefix := kvmPrefix + k.ID
|
||||
p := &httputil.ReverseProxy{
|
||||
Director: func(r *http.Request) {
|
||||
r.URL.Scheme, r.URL.Host = target.Scheme, target.Host
|
||||
// Entry through the prefix keeps the path after it; anything else arrived by its
|
||||
// own absolute path and is passed through untouched.
|
||||
if rest, ok := strings.CutPrefix(r.URL.Path, prefix); ok {
|
||||
if rest == "" {
|
||||
rest = "/"
|
||||
}
|
||||
r.URL.Path = rest
|
||||
}
|
||||
r.URL.RawPath = ""
|
||||
r.Host = target.Host
|
||||
// The device gets its own host and none of Ham Lab's session.
|
||||
r.Header.Del("Origin")
|
||||
stripCookie(r, cookieName, kvmCookie)
|
||||
},
|
||||
FlushInterval: -1, // a video stream must not be buffered
|
||||
ModifyResponse: func(resp *http.Response) error {
|
||||
// The device's own framing and transport policy would stop it being embedded, or
|
||||
// send a browser off to plain HTTP. Ham Lab is the origin now, and says what it
|
||||
// serves; its own headers are added on the way out.
|
||||
for _, h := range []string{"X-Frame-Options", "Content-Security-Policy", "Content-Security-Policy-Report-Only", "Strict-Transport-Security", "Referrer-Policy"} {
|
||||
resp.Header.Del(h)
|
||||
}
|
||||
// A redirect to the device's own root has to come back through the prefix.
|
||||
if loc := resp.Header.Get("Location"); loc != "" {
|
||||
if u, err := url.Parse(loc); err == nil && (u.Host == "" || u.Host == target.Host) && strings.HasPrefix(u.Path, "/") && !strings.HasPrefix(u.Path, prefix) {
|
||||
u.Scheme, u.Host = "", ""
|
||||
u.Path = prefix + u.Path
|
||||
resp.Header.Set("Location", u.String())
|
||||
}
|
||||
}
|
||||
return nil
|
||||
},
|
||||
ErrorHandler: func(w http.ResponseWriter, r *http.Request, err error) {
|
||||
w.Header().Set("Content-Type", "application/json")
|
||||
w.WriteHeader(http.StatusBadGateway)
|
||||
_ = json.NewEncoder(w).Encode(map[string]string{"error": k.Name + " unreachable: " + err.Error()})
|
||||
},
|
||||
}
|
||||
s.kvms.m[k.ID] = p
|
||||
return p, nil
|
||||
}
|
||||
|
||||
// stripCookie removes named cookies from a request, so Ham Lab's session is never handed to
|
||||
// a device and the device never sees which KVM the browser is looking at.
|
||||
func stripCookie(r *http.Request, names ...string) {
|
||||
cs := r.Cookies()
|
||||
kept := cs[:0]
|
||||
for _, c := range cs {
|
||||
drop := false
|
||||
for _, n := range names {
|
||||
if c.Name == n {
|
||||
drop = true
|
||||
}
|
||||
}
|
||||
if !drop {
|
||||
kept = append(kept, c)
|
||||
}
|
||||
}
|
||||
r.Header.Del("Cookie")
|
||||
for _, c := range kept {
|
||||
r.AddCookie(c)
|
||||
}
|
||||
}
|
||||
|
||||
func (s *Server) forgetKVMProxy(id string) {
|
||||
s.kvms.mu.Lock()
|
||||
delete(s.kvms.m, id)
|
||||
s.kvms.mu.Unlock()
|
||||
}
|
||||
|
||||
// serveKVM is the entry point: /kvm/<id>/… . It also drops a cookie naming the device, so
|
||||
// that requests the page makes later by absolute path can be traced back to it.
|
||||
func (s *Server) serveKVM(w http.ResponseWriter, r *http.Request) {
|
||||
id := r.PathValue("id")
|
||||
k, ok := s.kvm(id)
|
||||
if !ok {
|
||||
jsonErr(w, 404, "no such KVM")
|
||||
return
|
||||
}
|
||||
p, err := s.kvmProxy(k)
|
||||
if err != nil {
|
||||
jsonErr(w, 400, err.Error())
|
||||
return
|
||||
}
|
||||
http.SetCookie(w, &http.Cookie{Name: kvmCookie, Value: id, Path: "/", HttpOnly: true, Secure: secure(r), SameSite: http.SameSiteLaxMode})
|
||||
p.ServeHTTP(w, r)
|
||||
}
|
||||
|
||||
// kvmFor works out which device a request belongs to when it arrived by absolute path
|
||||
// rather than through the prefix: the Referer names the page that asked, and a cookie
|
||||
// covers what has no Referer at all. Nothing device-specific: it never has to know which
|
||||
// paths a given KVM uses.
|
||||
func (s *Server) kvmFor(r *http.Request) (config.KVM, bool) {
|
||||
if ref := r.Header.Get("Referer"); ref != "" {
|
||||
if u, err := url.Parse(ref); err == nil {
|
||||
if rest, ok := strings.CutPrefix(u.Path, kvmPrefix); ok {
|
||||
if id, _, _ := strings.Cut(rest, "/"); id != "" {
|
||||
if k, ok := s.kvm(id); ok {
|
||||
return k, true
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
// asked for by a page that is not a KVM: not ours to answer
|
||||
return config.KVM{}, false
|
||||
}
|
||||
if c, err := r.Cookie(kvmCookie); err == nil && c.Value != "" {
|
||||
return s.kvm(c.Value)
|
||||
}
|
||||
return config.KVM{}, false
|
||||
}
|
||||
|
||||
// kvmFallback answers a request Ham Lab does not serve itself, when it came from inside a
|
||||
// KVM's page. It returns false when the request has nothing to do with a KVM, and the
|
||||
// caller carries on with its own 404.
|
||||
func (s *Server) kvmFallback(w http.ResponseWriter, r *http.Request) bool {
|
||||
k, ok := s.kvmFor(r)
|
||||
if !ok {
|
||||
return false
|
||||
}
|
||||
p, err := s.kvmProxy(k)
|
||||
if err != nil {
|
||||
return false
|
||||
}
|
||||
p.ServeHTTP(w, r)
|
||||
return true
|
||||
}
|
||||
|
||||
func machineLabel(m string) string {
|
||||
if m == "" {
|
||||
return "this host"
|
||||
}
|
||||
return m
|
||||
}
|
||||
|
||||
// ---- the list ----
|
||||
|
||||
func (s *Server) listKVMs(w http.ResponseWriter, _ *http.Request) {
|
||||
out := []config.KVM{}
|
||||
s.cfg.View(func(c *config.Config) { out = append(out, c.KVMs...) })
|
||||
jsonOK(w, 200, out)
|
||||
}
|
||||
|
||||
// kvmForMachine is the KVM attached to a machine, "" being the hub itself.
|
||||
func (s *Server) kvmForMachine(machine string) (config.KVM, bool) {
|
||||
var found config.KVM
|
||||
ok := false
|
||||
s.cfg.View(func(c *config.Config) {
|
||||
for _, k := range c.KVMs {
|
||||
if k.Machine == machine {
|
||||
found, ok = k, true
|
||||
}
|
||||
}
|
||||
})
|
||||
return found, ok
|
||||
}
|
||||
|
||||
func (s *Server) addKVM(w http.ResponseWriter, r *http.Request) {
|
||||
var body struct{ Name, URL, Machine string }
|
||||
if err := json.NewDecoder(io.LimitReader(r.Body, 4096)).Decode(&body); err != nil {
|
||||
jsonErr(w, 400, "bad request")
|
||||
return
|
||||
}
|
||||
raw := strings.TrimSpace(body.URL)
|
||||
if raw != "" && !strings.Contains(raw, "://") {
|
||||
raw = "http://" + raw // what you would type: an address, not a scheme
|
||||
}
|
||||
u, err := url.Parse(raw)
|
||||
if err != nil || (u.Scheme != "http" && u.Scheme != "https") || u.Host == "" {
|
||||
jsonErr(w, 400, "address: http(s)://host, or just the host")
|
||||
return
|
||||
}
|
||||
name := strings.TrimSpace(body.Name)
|
||||
if name == "" {
|
||||
name = u.Host
|
||||
}
|
||||
if len(name) > maxNameLen {
|
||||
jsonErr(w, 400, "that name is too long")
|
||||
return
|
||||
}
|
||||
b := make([]byte, 6)
|
||||
if _, err := rand.Read(b); err != nil {
|
||||
jsonErr(w, 500, err.Error())
|
||||
return
|
||||
}
|
||||
machine := strings.TrimSpace(body.Machine)
|
||||
k := config.KVM{ID: hex.EncodeToString(b), Name: name, URL: u.Scheme + "://" + u.Host + strings.TrimSuffix(u.Path, "/"), Machine: machine, Added: time.Now()}
|
||||
// One per machine: a second one for the same host replaces the first rather than sitting
|
||||
// beside it, since the tab shows the machine you are looking at.
|
||||
var replaced string
|
||||
if err := s.cfg.Update(func(c *config.Config) {
|
||||
kept := c.KVMs[:0]
|
||||
for _, old := range c.KVMs {
|
||||
if old.Machine == machine {
|
||||
replaced = old.ID
|
||||
continue
|
||||
}
|
||||
kept = append(kept, old)
|
||||
}
|
||||
c.KVMs = append(kept, k)
|
||||
}); err != nil {
|
||||
jsonErr(w, 500, err.Error())
|
||||
return
|
||||
}
|
||||
if replaced != "" {
|
||||
s.forgetKVMProxy(replaced)
|
||||
}
|
||||
log.Printf("kvm %q added for %s (%s) by %s", k.Name, machineLabel(machine), k.URL, clientIP(r))
|
||||
jsonOK(w, 201, k)
|
||||
}
|
||||
|
||||
func (s *Server) removeKVM(w http.ResponseWriter, r *http.Request) {
|
||||
id := r.PathValue("id")
|
||||
found := false
|
||||
if err := s.cfg.Update(func(c *config.Config) {
|
||||
kept := c.KVMs[:0]
|
||||
for _, k := range c.KVMs {
|
||||
if k.ID == id {
|
||||
found = true
|
||||
continue
|
||||
}
|
||||
kept = append(kept, k)
|
||||
}
|
||||
c.KVMs = kept
|
||||
}); err != nil {
|
||||
jsonErr(w, 500, err.Error())
|
||||
return
|
||||
}
|
||||
if !found {
|
||||
jsonErr(w, 404, "no such KVM")
|
||||
return
|
||||
}
|
||||
s.forgetKVMProxy(id)
|
||||
log.Printf("kvm %s removed by %s", id, clientIP(r))
|
||||
w.WriteHeader(204)
|
||||
}
|
||||
+63
-62
@@ -69,6 +69,8 @@ type Server struct {
|
||||
storage *storage.Manager
|
||||
space *space.Manager
|
||||
update updateState
|
||||
events eventLog
|
||||
kvms kvmProxies
|
||||
fsLocal files.FS
|
||||
fsSudo files.FS // nil without passwordless sudo
|
||||
nodes *nodes.Manager
|
||||
@@ -102,18 +104,8 @@ func New(cfg *config.Config, sessions *auth.Sessions, mgr *term.Manager, col *me
|
||||
s.sensors.Add(newStatusSensors(s))
|
||||
s.alerts = newAlertEngine(s)
|
||||
// The saved dashboard layout carries the sampling interval; apply it on boot.
|
||||
var raw json.RawMessage
|
||||
var histSec int
|
||||
st.View(func(x *store.State) {
|
||||
raw, histSec = x.Dashboard, x.HistorySec
|
||||
if len(raw) == 0 {
|
||||
for _, u := range x.Users {
|
||||
raw = u.Dashboard
|
||||
break
|
||||
}
|
||||
}
|
||||
})
|
||||
s.applyPrefs(raw)
|
||||
st.View(func(x *store.State) { histSec = x.HistorySec })
|
||||
if histSec > 0 {
|
||||
col.SetWindow(time.Duration(histSec) * time.Second)
|
||||
}
|
||||
@@ -147,6 +139,7 @@ func New(cfg *config.Config, sessions *auth.Sessions, mgr *term.Manager, col *me
|
||||
go s.keepFleetInStep()
|
||||
}
|
||||
go s.watchForUpdates(context.Background())
|
||||
go s.watchDockerEvents(context.Background())
|
||||
|
||||
m := http.NewServeMux()
|
||||
m.HandleFunc("GET /healthz", func(w http.ResponseWriter, _ *http.Request) { w.WriteHeader(204) })
|
||||
@@ -199,6 +192,12 @@ func New(cfg *config.Config, sessions *auth.Sessions, mgr *term.Manager, col *me
|
||||
m.HandleFunc("POST /api/docker/stacks/{name}/{action}", s.auth(s.stackAction))
|
||||
m.HandleFunc("GET /api/docker/stacks/{name}/diff", s.auth(s.stackDiff))
|
||||
m.HandleFunc("GET /api/docker/stacks/{name}/updates", s.auth(s.stackUpdates))
|
||||
m.HandleFunc("GET /api/docker/updates", s.auth(s.machineUpdates))
|
||||
m.HandleFunc("GET /api/docker/schedules", s.auth(s.dsList))
|
||||
m.HandleFunc("PUT /api/docker/schedules", s.auth(s.dsPut))
|
||||
m.HandleFunc("DELETE /api/docker/schedules/{id}", s.auth(s.dsDelete))
|
||||
m.HandleFunc("POST /api/docker/schedules/{id}/run", s.auth(s.dsRun))
|
||||
m.HandleFunc("GET /api/nodes/{node}/docker/updates", s.auth(s.machineUpdates))
|
||||
// The same, asked of a node through this hub: the hub answers for the stacks whose files
|
||||
// it holds (deployed to the node over tcp) and passes everything else to the node.
|
||||
m.HandleFunc("GET /api/nodes/{node}/docker/stacks", s.auth(s.stacksList))
|
||||
@@ -215,6 +214,10 @@ func New(cfg *config.Config, sessions *auth.Sessions, mgr *term.Manager, col *me
|
||||
m.HandleFunc("POST /api/docker/containers/{id}/{action}", s.auth(s.dockerAction))
|
||||
m.HandleFunc("GET /api/docker/containers/{id}/logs", s.auth(s.dockerLogs))
|
||||
m.HandleFunc("GET /api/docker/containers/{id}/inspect", s.auth(s.dockerInspect))
|
||||
m.HandleFunc("GET /api/docker/containers/{id}/health", s.auth(s.dockerHealth))
|
||||
m.HandleFunc("GET /api/docker/containers/{id}/events", s.auth(s.dockerEvents))
|
||||
m.HandleFunc("POST /api/docker/containers/{id}/exec", s.auth(s.dockerExec))
|
||||
m.HandleFunc("POST /api/docker/containers/{id}/limits", s.auth(s.dockerUpdate))
|
||||
m.HandleFunc("GET /api/alerts", s.auth(s.alertsGet))
|
||||
m.HandleFunc("PUT /api/alerts", s.auth(s.alertsPut))
|
||||
m.HandleFunc("POST /api/alerts/test", s.auth(s.alertsTest))
|
||||
@@ -261,6 +264,11 @@ func New(cfg *config.Config, sessions *auth.Sessions, mgr *term.Manager, col *me
|
||||
m.HandleFunc("GET /api/storage/zfs/snapshots", s.auth(s.zfsSnapshots))
|
||||
m.HandleFunc("POST /api/storage/zfs/scrub/{pool}/{action}", s.auth(s.zfsScrub))
|
||||
m.HandleFunc("POST /api/storage/zfs/snapshot", s.auth(s.zfsSnapshot))
|
||||
m.HandleFunc("GET /api/zfs", s.auth(s.zfsOverview))
|
||||
m.HandleFunc("PUT /api/zfs/schedules", s.auth(s.zfsPolicyPut))
|
||||
m.HandleFunc("DELETE /api/zfs/schedules/{id}", s.auth(s.zfsPolicyDelete))
|
||||
m.HandleFunc("POST /api/zfs/schedules/{id}/run", s.auth(s.zfsRunNow))
|
||||
m.HandleFunc("POST /api/zfs/snapshots/delete", s.auth(s.zfsSnapshotDelete))
|
||||
m.HandleFunc("GET /api/space", s.auth(s.spaceOverview))
|
||||
m.HandleFunc("GET /api/space/{disk}", s.auth(s.spaceStatus))
|
||||
m.HandleFunc("POST /api/space/{disk}/scan", s.auth(s.spaceScan))
|
||||
@@ -289,9 +297,20 @@ func New(cfg *config.Config, sessions *auth.Sessions, mgr *term.Manager, col *me
|
||||
m.HandleFunc("PUT /api/prefs/sampler", s.auth(s.putSampler))
|
||||
m.HandleFunc("GET /api/prefs/history", s.auth(s.getHistoryWindow))
|
||||
m.HandleFunc("PUT /api/prefs/history", s.auth(s.putHistoryWindow))
|
||||
m.HandleFunc("GET /api/prefs/dashboard", s.auth(s.getDashPrefs))
|
||||
m.HandleFunc("PUT /api/prefs/dashboard", s.auth(s.putDashPrefs))
|
||||
m.HandleFunc("/api/", func(w http.ResponseWriter, _ *http.Request) { jsonErr(w, 404, "not found") })
|
||||
m.HandleFunc("GET /api/kvms", s.auth(s.listKVMs))
|
||||
m.HandleFunc("POST /api/kvms", s.auth(s.addKVM))
|
||||
m.HandleFunc("DELETE /api/kvms/{id}", s.auth(s.removeKVM))
|
||||
m.HandleFunc("/kvm/{id}/{rest...}", s.auth(s.serveKVM))
|
||||
m.HandleFunc("/kvm/{id}", s.auth(s.serveKVM))
|
||||
// A path Ham Lab does not serve may still belong to a KVM's page, which asks for its
|
||||
// own assets by absolute path. Ham Lab's own routes matched first, so this can only
|
||||
// catch what would otherwise have been a 404.
|
||||
m.HandleFunc("/api/", s.auth(func(w http.ResponseWriter, r *http.Request) {
|
||||
if s.kvmFallback(w, r) {
|
||||
return
|
||||
}
|
||||
jsonErr(w, 404, "not found")
|
||||
}))
|
||||
m.HandleFunc("/", s.static)
|
||||
s.mux = m
|
||||
return s
|
||||
@@ -314,9 +333,16 @@ func sensorPath(p string) bool {
|
||||
func (s *Server) Handler() http.Handler {
|
||||
return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
|
||||
h := w.Header()
|
||||
// The console must never be framed. An embedded widget exists to be: it is a page
|
||||
// that can only read sensors, and only with a read-only token in its URL.
|
||||
if r.URL.Path != "/embed/widget" {
|
||||
// The console must never be framed. Two exceptions: an embedded widget exists to be
|
||||
// (a page that can only read sensors, and only with a read-only token in its URL),
|
||||
// and a proxied KVM is framed by the console itself — 'none' forbids that even from
|
||||
// the same origin, so those paths say 'self' instead of opening up.
|
||||
switch {
|
||||
case r.URL.Path == "/embed/widget":
|
||||
case strings.HasPrefix(r.URL.Path, kvmPrefix):
|
||||
h.Set("X-Frame-Options", "SAMEORIGIN")
|
||||
h.Set("Content-Security-Policy", "frame-ancestors 'self'")
|
||||
default:
|
||||
h.Set("X-Frame-Options", "DENY")
|
||||
h.Set("Content-Security-Policy", "frame-ancestors 'none'")
|
||||
}
|
||||
@@ -362,6 +388,17 @@ func (s *Server) setCookie(w http.ResponseWriter, r *http.Request, tok string, m
|
||||
})
|
||||
}
|
||||
|
||||
// signedIn reports whether a request carries a live browser session, without the
|
||||
// side effects of the auth middleware.
|
||||
func (s *Server) signedIn(r *http.Request) bool {
|
||||
c, err := r.Cookie(cookieName)
|
||||
if err != nil {
|
||||
return false
|
||||
}
|
||||
_, ok := s.sessions.Lookup(c.Value, clientKey(r))
|
||||
return ok
|
||||
}
|
||||
|
||||
// auth accepts the browser session cookie or a bearer API token issued to a hub.
|
||||
func (s *Server) auth(next http.HandlerFunc) http.HandlerFunc {
|
||||
return func(w http.ResponseWriter, r *http.Request) {
|
||||
@@ -930,15 +967,11 @@ func (s *Server) migrateUsers() {
|
||||
st.Sessions[tok] = ses
|
||||
}
|
||||
}
|
||||
if len(st.UI) > 0 || len(st.Dashboard) > 0 {
|
||||
u := st.UserState(first)
|
||||
if len(u.UI) == 0 {
|
||||
if len(st.UI) > 0 {
|
||||
if u := st.UserState(first); len(u.UI) == 0 {
|
||||
u.UI = st.UI
|
||||
}
|
||||
if len(u.Dashboard) == 0 {
|
||||
u.Dashboard = st.Dashboard
|
||||
}
|
||||
st.UI, st.Dashboard = nil, nil
|
||||
st.UI = nil
|
||||
}
|
||||
})
|
||||
}
|
||||
@@ -2282,44 +2315,6 @@ func (s *Server) pushHistoryWindow(seconds int) {
|
||||
}
|
||||
}
|
||||
|
||||
// ---- dashboard layout prefs (opaque JSON object, owned by the UI) ----
|
||||
|
||||
func (s *Server) getDashPrefs(w http.ResponseWriter, r *http.Request) {
|
||||
w.Header().Set("Cache-Control", "no-store")
|
||||
jsonOK(w, 200, s.userPrefs(r, func(u *store.UserState) json.RawMessage { return u.Dashboard }))
|
||||
}
|
||||
|
||||
func (s *Server) putDashPrefs(w http.ResponseWriter, r *http.Request) {
|
||||
b, err := io.ReadAll(io.LimitReader(r.Body, 64<<10))
|
||||
if err != nil || !json.Valid(b) || len(b) == 0 || b[0] != '{' {
|
||||
jsonErr(w, 400, "expected a JSON object")
|
||||
return
|
||||
}
|
||||
if user := actorOf(r).User; user != "" {
|
||||
if err := s.store.Update(func(st *store.State) { st.UserState(user).Dashboard = json.RawMessage(b) }); err != nil {
|
||||
jsonErr(w, 500, err.Error())
|
||||
return
|
||||
}
|
||||
}
|
||||
s.applyPrefs(b)
|
||||
w.WriteHeader(204)
|
||||
}
|
||||
|
||||
// applyPrefs picks the few layout fields the server acts on (just the interval).
|
||||
func (s *Server) applyPrefs(raw []byte) {
|
||||
var p struct {
|
||||
IntervalMs int `json:"intervalMs"`
|
||||
}
|
||||
if len(raw) == 0 || json.Unmarshal(raw, &p) != nil {
|
||||
return
|
||||
}
|
||||
if p.IntervalMs > 0 {
|
||||
s.metrics.SetInterval(time.Duration(p.IntervalMs) * time.Millisecond)
|
||||
} else {
|
||||
s.metrics.SetInterval(metrics.DefaultInterval)
|
||||
}
|
||||
}
|
||||
|
||||
// ---- terminals ----
|
||||
|
||||
func (s *Server) listTerminals(w http.ResponseWriter, _ *http.Request) {
|
||||
@@ -2494,6 +2489,12 @@ func (s *Server) static(w http.ResponseWriter, r *http.Request) {
|
||||
return
|
||||
}
|
||||
}
|
||||
// A path with no file of ours behind it may belong to a KVM's page, which asks for its
|
||||
// own assets by absolute path. Checked here rather than in the router because this
|
||||
// handler is the catch-all — and gated on a session, since this handler is public.
|
||||
if s.signedIn(r) && s.kvmFallback(w, r) {
|
||||
return
|
||||
}
|
||||
// SPA fallback
|
||||
w.Header().Set("Cache-Control", "no-cache")
|
||||
http.ServeFileFS(w, r, s.ui, "index.html")
|
||||
|
||||
+209
-57
@@ -12,6 +12,12 @@ package api
|
||||
// relative paths, the .env beside them and the labels all stay exactly as they were
|
||||
// deployed — else on the host.
|
||||
//
|
||||
// Those mounts are also written down. An orchestrator can be removed, and when it is, the
|
||||
// containers it left behind still carry labels naming paths inside a filesystem that no
|
||||
// longer exists; without a record of where that filesystem was mounted, every stack on the
|
||||
// machine would become unfindable at once. A remembered mount resolves with no container to
|
||||
// run in, which is right: compose runs on the host against the host's copy of the file.
|
||||
//
|
||||
// A stack can also be deployed *from another machine*: Dockhand on the hub drives a node's
|
||||
// Docker over tcp, so the node runs the containers and the hub holds the files. The node
|
||||
// alone can only list such a stack read-only. The hub, asked about one of its nodes, fills
|
||||
@@ -39,7 +45,9 @@ import (
|
||||
"sync"
|
||||
"time"
|
||||
|
||||
"hamlab/internal/compose"
|
||||
"hamlab/internal/priv"
|
||||
"hamlab/internal/store"
|
||||
)
|
||||
|
||||
type stack struct {
|
||||
@@ -89,6 +97,100 @@ func stackMounts(ctx context.Context) []mountMap {
|
||||
return maps
|
||||
}
|
||||
|
||||
// mountsFor is stackMounts plus memory. An orchestrator that deployed everything here and
|
||||
// has since been removed takes the only record of where its files really are with it: the
|
||||
// containers it left behind are labelled with paths inside *its* filesystem. So every map
|
||||
// seen is written down, and a path that no live container explains is looked for in what
|
||||
// was seen before. A remembered map resolves with no `via`, which is correct — the
|
||||
// container is gone, so compose runs on the host, against the host's copy of the file.
|
||||
func mountsFor(ctx context.Context, db *store.Store) []mountMap {
|
||||
live := stackMounts(ctx)
|
||||
rememberMounts(db, live)
|
||||
return append(live, recallMounts(db, live)...)
|
||||
}
|
||||
|
||||
// mountsSeen is the same without writing anything down. The daemon owns state.json; a cron
|
||||
// job that wrote to it would be clobbered by the next thing the daemon saved, so out there
|
||||
// the record is read and never added to.
|
||||
func mountsSeen(ctx context.Context, db *store.Store) []mountMap {
|
||||
live := stackMounts(ctx)
|
||||
return append(live, recallMounts(db, live)...)
|
||||
}
|
||||
|
||||
// rememberMounts folds what is mounted now into what was mounted before, and forgets a map
|
||||
// whose source has gone from disk — that one is not coming back, and keeping it would make
|
||||
// a stack look editable when its file no longer exists.
|
||||
//
|
||||
// This is on the path of every stack listing, which the Docker tab polls, so it reads first
|
||||
// and only writes when the set actually changed.
|
||||
func rememberMounts(db *store.Store, live []mountMap) {
|
||||
if len(live) == 0 {
|
||||
return // docker is down or answered nothing; that is not evidence of anything
|
||||
}
|
||||
key := func(c, d string) string { return c + "\x00" + d }
|
||||
var have []store.RememberedMount
|
||||
db.View(func(st *store.State) { have = append([]store.RememberedMount(nil), st.Mounts...) })
|
||||
at := map[string]int{}
|
||||
for i, m := range have {
|
||||
at[key(m.Container, m.Dest)] = i
|
||||
}
|
||||
now, add, moved := time.Now(), 0, false
|
||||
for _, m := range live {
|
||||
i, ok := at[key(m.container, m.dest)]
|
||||
if !ok {
|
||||
add++
|
||||
continue
|
||||
}
|
||||
if have[i].Src != m.src {
|
||||
moved = true
|
||||
}
|
||||
}
|
||||
// Pruning is the expensive half — a stat, or a sudo test(1), per remembered map — so it
|
||||
// only happens when something else has already made a write necessary.
|
||||
if add == 0 && !moved {
|
||||
return
|
||||
}
|
||||
_ = db.Update(func(st *store.State) {
|
||||
at := map[string]int{}
|
||||
for i, m := range st.Mounts {
|
||||
at[key(m.Container, m.Dest)] = i
|
||||
}
|
||||
for _, m := range live {
|
||||
if i, ok := at[key(m.container, m.dest)]; ok {
|
||||
st.Mounts[i].Src, st.Mounts[i].Seen = m.src, now
|
||||
continue
|
||||
}
|
||||
st.Mounts = append(st.Mounts, store.RememberedMount{Container: m.container, Dest: m.dest, Src: m.src, Seen: now})
|
||||
}
|
||||
kept := st.Mounts[:0]
|
||||
for _, m := range st.Mounts {
|
||||
if readable(m.Src) {
|
||||
kept = append(kept, m)
|
||||
}
|
||||
}
|
||||
st.Mounts = kept
|
||||
})
|
||||
}
|
||||
|
||||
// recallMounts is what was seen before and is not running now.
|
||||
func recallMounts(db *store.Store, live []mountMap) []mountMap {
|
||||
var remembered []store.RememberedMount
|
||||
db.View(func(st *store.State) { remembered = append([]store.RememberedMount(nil), st.Mounts...) })
|
||||
now := map[string]bool{}
|
||||
for _, m := range live {
|
||||
now[m.container+"\x00"+m.dest] = true
|
||||
}
|
||||
var out []mountMap
|
||||
for _, m := range remembered {
|
||||
if now[m.Container+"\x00"+m.Dest] {
|
||||
continue
|
||||
}
|
||||
out = append(out, mountMap{"", filepath.Clean(m.Dest), filepath.Clean(m.Src)}) // no container: run compose here
|
||||
}
|
||||
sort.Slice(out, func(i, j int) bool { return len(out[i].dest) > len(out[j].dest) })
|
||||
return out
|
||||
}
|
||||
|
||||
func readable(p string) bool {
|
||||
if _, err := os.Stat(p); err == nil {
|
||||
return true
|
||||
@@ -172,7 +274,7 @@ func (s *Server) findStacks(ctx context.Context) ([]stack, error) {
|
||||
}
|
||||
}
|
||||
}
|
||||
maps := stackMounts(ctx)
|
||||
maps := mountsFor(ctx, s.store)
|
||||
stacks := []stack{}
|
||||
for _, st := range by {
|
||||
sort.Strings(st.Containers)
|
||||
@@ -353,7 +455,7 @@ func (s *Server) hubStacks(ctx context.Context, node string) ([]stack, error) {
|
||||
continue
|
||||
}
|
||||
if remote == "" {
|
||||
remote, maps = s.nodeDocker(ctx, node), stackMounts(ctx)
|
||||
remote, maps = s.nodeDocker(ctx, node), mountsFor(ctx, s.store)
|
||||
}
|
||||
if remote == "-" {
|
||||
continue
|
||||
@@ -511,24 +613,13 @@ func (s *Server) stackFile(w http.ResponseWriter, r *http.Request) {
|
||||
// composeCmd is `docker compose …` for this stack, run where its files make sense and
|
||||
// pointed at the Docker that runs its containers. files are given as compose will see them.
|
||||
func composeCmd(ctx context.Context, st stack, files []string, args ...string) *exec.Cmd {
|
||||
tool := dockerTool()
|
||||
base := []string{"compose", "-p", st.Name}
|
||||
for _, f := range files {
|
||||
base = append(base, "-f", f)
|
||||
}
|
||||
if st.Via != "" {
|
||||
pre := []string{"exec"}
|
||||
if st.Remote != "" {
|
||||
pre = append(pre, "-e", "DOCKER_HOST="+st.Remote)
|
||||
}
|
||||
return exec.CommandContext(ctx, tool, append(append(append(pre, st.Via, "docker"), base...), args...)...)
|
||||
}
|
||||
cmd := exec.CommandContext(ctx, tool, append(base, args...)...)
|
||||
cmd.Dir = filepath.Dir(files[0])
|
||||
if st.Remote != "" {
|
||||
cmd.Env = append(os.Environ(), "DOCKER_HOST="+st.Remote)
|
||||
}
|
||||
return cmd
|
||||
return compose.Cmd(ctx, specOf(st, files), args...)
|
||||
}
|
||||
|
||||
// specOf is a stack as the compose runner needs it — and as a scheduled update stores it,
|
||||
// so the cron job and the console invoke compose identically.
|
||||
func specOf(st stack, files []string) compose.Spec {
|
||||
return compose.Spec{Project: st.Name, Files: files, Via: st.Via, Remote: st.Remote, Tool: dockerTool()}
|
||||
}
|
||||
|
||||
// seenFiles is the stack's file list as compose runs it, with file i replaced by a candidate.
|
||||
@@ -788,6 +879,103 @@ var (
|
||||
|
||||
func stackUpdatesForget(key string) { updMu.Lock(); delete(updCache, key); updMu.Unlock() }
|
||||
|
||||
// checkImages asks the registry whether each image is still the one we hold. remote is a
|
||||
// DOCKER_HOST for a machine whose images live elsewhere, "" for this one. Each image is one
|
||||
// registry request, which Docker Hub counts — so this is only ever called when asked for.
|
||||
func checkImages(ctx context.Context, images []string, remote string) []imageCheck {
|
||||
local := func(args ...string) ([]byte, error) { // the Docker that holds the images
|
||||
if remote != "" {
|
||||
args = append([]string{"-H", remote}, args...)
|
||||
}
|
||||
return dockerRun(ctx, args...)
|
||||
}
|
||||
res := []imageCheck{}
|
||||
for _, img := range images {
|
||||
ic := imageCheck{Image: img, State: "unknown"}
|
||||
out, err := local("image", "inspect", "-f", "{{json .RepoDigests}}", img)
|
||||
var have []string
|
||||
if err != nil || json.Unmarshal(bytes.TrimSpace(out), &have) != nil || len(have) == 0 {
|
||||
ic.Detail = "built or loaded here, not pulled from a registry"
|
||||
res = append(res, ic)
|
||||
continue
|
||||
}
|
||||
rem, err := dockerRun(ctx, "buildx", "imagetools", "inspect", img, "--format", "{{.Manifest.Digest}}")
|
||||
digest := strings.TrimSpace(string(rem))
|
||||
if i := strings.LastIndex(digest, "\n"); i >= 0 {
|
||||
digest = digest[i+1:]
|
||||
}
|
||||
if err != nil || !strings.HasPrefix(digest, "sha256:") {
|
||||
ic.Detail = "the registry didn't answer"
|
||||
if err != nil {
|
||||
ic.Detail += ": " + err.Error()
|
||||
}
|
||||
res = append(res, ic)
|
||||
continue
|
||||
}
|
||||
ic.State = "update"
|
||||
for _, h := range have {
|
||||
if strings.HasSuffix(h, "@"+digest) {
|
||||
ic.State = "current"
|
||||
}
|
||||
}
|
||||
res = append(res, ic)
|
||||
}
|
||||
return res
|
||||
}
|
||||
|
||||
// machineUpdates checks every image on this machine in one pass, so the container list can
|
||||
// be coloured and the fleet can be asked "what is behind?" without walking stack by stack.
|
||||
func (s *Server) machineUpdates(w http.ResponseWriter, r *http.Request) {
|
||||
node := r.PathValue("node")
|
||||
ctx, cancel := context.WithTimeout(r.Context(), 10*time.Minute)
|
||||
defer cancel()
|
||||
stacks, err := s.findStacks(ctx)
|
||||
if node != "" {
|
||||
stacks, err = s.hubStacks(ctx, node)
|
||||
}
|
||||
if err != nil {
|
||||
jsonErr(w, 500, err.Error())
|
||||
return
|
||||
}
|
||||
// one entry per image, however many stacks use it
|
||||
type want struct{ remote string }
|
||||
imgs := map[string]want{}
|
||||
byImage := map[string][]string{} // image -> stacks
|
||||
for _, st := range stacks {
|
||||
for _, img := range st.Images {
|
||||
imgs[img] = want{remote: st.Remote}
|
||||
byImage[img] = append(byImage[img], st.Name)
|
||||
}
|
||||
}
|
||||
refresh := r.URL.Query().Get("refresh") == "1"
|
||||
key := "machine:" + node
|
||||
updMu.Lock()
|
||||
c, ok := updCache[key]
|
||||
updMu.Unlock()
|
||||
if ok && time.Since(c.at) < 6*time.Hour && !refresh {
|
||||
jsonOK(w, 200, map[string]any{"images": c.res, "checked": c.at.UnixMilli(), "stacks": byImage})
|
||||
return
|
||||
}
|
||||
if !refresh {
|
||||
jsonOK(w, 200, map[string]any{"images": []imageCheck{}, "checked": 0, "stacks": byImage})
|
||||
return
|
||||
}
|
||||
var list []string
|
||||
remote := ""
|
||||
for img, wnt := range imgs {
|
||||
list = append(list, img)
|
||||
if wnt.remote != "" {
|
||||
remote = wnt.remote
|
||||
}
|
||||
}
|
||||
sort.Strings(list)
|
||||
res := checkImages(ctx, list, remote)
|
||||
updMu.Lock()
|
||||
updCache[key] = updAnswer{time.Now(), res}
|
||||
updMu.Unlock()
|
||||
jsonOK(w, 200, map[string]any{"images": res, "checked": time.Now().UnixMilli(), "stacks": byImage})
|
||||
}
|
||||
|
||||
// stackUpdates asks each image's registry for the digest its tag points at now and compares
|
||||
// it with the digest that was pulled. It is asked for, never polled: every answer is a
|
||||
// registry request, and Docker Hub counts those. Answers are kept for six hours.
|
||||
@@ -816,43 +1004,7 @@ func (s *Server) stackUpdates(w http.ResponseWriter, r *http.Request) {
|
||||
jsonOK(w, 200, map[string]any{"images": []imageCheck{}, "checked": 0})
|
||||
return
|
||||
}
|
||||
local := func(args ...string) ([]byte, error) { // the Docker that holds the images
|
||||
if st.Remote != "" {
|
||||
args = append([]string{"-H", st.Remote}, args...)
|
||||
}
|
||||
return dockerRun(ctx, args...)
|
||||
}
|
||||
res := []imageCheck{}
|
||||
for _, img := range st.Images {
|
||||
ic := imageCheck{Image: img, State: "unknown"}
|
||||
out, err := local("image", "inspect", "-f", "{{json .RepoDigests}}", img)
|
||||
var have []string
|
||||
if err != nil || json.Unmarshal(bytes.TrimSpace(out), &have) != nil || len(have) == 0 {
|
||||
ic.Detail = "built or loaded here, not pulled from a registry"
|
||||
res = append(res, ic)
|
||||
continue
|
||||
}
|
||||
remote, err := dockerRun(ctx, "buildx", "imagetools", "inspect", img, "--format", "{{.Manifest.Digest}}")
|
||||
digest := strings.TrimSpace(string(remote))
|
||||
if i := strings.LastIndex(digest, "\n"); i >= 0 {
|
||||
digest = digest[i+1:]
|
||||
}
|
||||
if err != nil || !strings.HasPrefix(digest, "sha256:") {
|
||||
ic.Detail = "the registry didn't answer"
|
||||
if err != nil {
|
||||
ic.Detail += ": " + err.Error()
|
||||
}
|
||||
res = append(res, ic)
|
||||
continue
|
||||
}
|
||||
ic.State = "update"
|
||||
for _, h := range have {
|
||||
if strings.HasSuffix(h, "@"+digest) {
|
||||
ic.State = "current"
|
||||
}
|
||||
}
|
||||
res = append(res, ic)
|
||||
}
|
||||
res := checkImages(ctx, st.Images, st.Remote)
|
||||
updMu.Lock()
|
||||
updCache[key] = updAnswer{time.Now(), res}
|
||||
updMu.Unlock()
|
||||
|
||||
@@ -0,0 +1,121 @@
|
||||
package api
|
||||
|
||||
import (
|
||||
"context"
|
||||
"os"
|
||||
"path/filepath"
|
||||
"testing"
|
||||
|
||||
"hamlab/internal/compose"
|
||||
"hamlab/internal/store"
|
||||
)
|
||||
|
||||
// A stack deployed by an orchestrator running in a container records the path its compose
|
||||
// file had inside that container. Resolving it has to work both while the orchestrator is
|
||||
// running — compose is then run inside it — and after it has been removed, where the same
|
||||
// mount is only remembered and compose has to run on the host instead.
|
||||
func TestResolveThroughMounts(t *testing.T) {
|
||||
root := t.TempDir()
|
||||
dir := filepath.Join(root, "orch", "stacks", "app")
|
||||
if err := os.MkdirAll(dir, 0o755); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
file := filepath.Join(dir, "compose.yaml")
|
||||
if err := os.WriteFile(file, []byte("services: {}\n"), 0o644); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
inner := "/app/data/stacks/app/compose.yaml"
|
||||
|
||||
for _, tc := range []struct {
|
||||
name, container, wantVia string
|
||||
}{
|
||||
{"orchestrator running", "orch", "orch"},
|
||||
{"orchestrator removed", "", ""},
|
||||
} {
|
||||
t.Run(tc.name, func(t *testing.T) {
|
||||
st := stack{Name: "app", File: inner, Files: []string{inner}}
|
||||
st.resolve([]mountMap{{tc.container, "/app/data", filepath.Join(root, "orch")}})
|
||||
if !st.Editable {
|
||||
t.Fatalf("not editable: %s", st.Note)
|
||||
}
|
||||
if st.HostFile != file {
|
||||
t.Errorf("host file = %q, want %q", st.HostFile, file)
|
||||
}
|
||||
if st.Via != tc.wantVia {
|
||||
t.Errorf("via = %q, want %q", st.Via, tc.wantVia)
|
||||
}
|
||||
})
|
||||
}
|
||||
|
||||
// Nothing explains the path: the stack is listed, but read-only and says why.
|
||||
st := stack{Name: "app", File: inner, Files: []string{inner}}
|
||||
st.resolve(nil)
|
||||
if st.Editable || st.Note == "" {
|
||||
t.Errorf("unresolvable stack should be read-only with a note, got editable=%v note=%q", st.Editable, st.Note)
|
||||
}
|
||||
}
|
||||
|
||||
// Every file of a multi-file stack must resolve through the same place. Redeploying from
|
||||
// the first file alone would silently drop whatever the override set.
|
||||
func TestResolveRefusesSplitStack(t *testing.T) {
|
||||
root := t.TempDir()
|
||||
dir := filepath.Join(root, "orch", "stacks", "app")
|
||||
if err := os.MkdirAll(dir, 0o755); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if err := os.WriteFile(filepath.Join(dir, "compose.yaml"), []byte("services: {}\n"), 0o644); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
st := stack{Name: "app", File: "/app/data/stacks/app/compose.yaml", Files: []string{
|
||||
"/app/data/stacks/app/compose.yaml",
|
||||
"/app/data/stacks/app/override.yaml", // never written: not on disk
|
||||
}}
|
||||
st.resolve([]mountMap{{"orch", "/app/data", filepath.Join(root, "orch")}})
|
||||
if st.Editable {
|
||||
t.Error("a stack with a file that cannot be found should not be editable")
|
||||
}
|
||||
}
|
||||
|
||||
// A schedule made while an orchestrator was running says "run compose inside that
|
||||
// container". When the container is gone the schedule must still work, against the same
|
||||
// files on the host — otherwise retiring the orchestrator silently breaks every scheduled
|
||||
// update made before it went.
|
||||
func TestRepointSurvivesAMissingContainer(t *testing.T) {
|
||||
root := t.TempDir()
|
||||
dir := filepath.Join(root, "orch", "stacks", "app")
|
||||
if err := os.MkdirAll(dir, 0o755); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
host := filepath.Join(dir, "compose.yaml")
|
||||
if err := os.WriteFile(host, []byte("services: {}\n"), 0o644); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
db, err := store.Open(filepath.Join(root, "state.json"))
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if err := db.Update(func(st *store.State) {
|
||||
st.Mounts = []store.RememberedMount{{Container: "gone-orchestrator", Dest: "/app/data", Src: filepath.Join(root, "orch")}}
|
||||
}); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
|
||||
in := compose.Spec{Project: "app", Files: []string{"/app/data/stacks/app/compose.yaml"}, Via: "gone-orchestrator", Tool: "docker"}
|
||||
got := repoint(context.Background(), db, in)
|
||||
if got.Via != "" {
|
||||
t.Errorf("via = %q, want it dropped", got.Via)
|
||||
}
|
||||
if len(got.Files) != 1 || got.Files[0] != host {
|
||||
t.Errorf("files = %v, want [%s]", got.Files, host)
|
||||
}
|
||||
|
||||
// Nothing remembered: the schedule is left exactly as it was rather than pointed at a
|
||||
// guess. It will fail loudly, which is better than deploying the wrong file.
|
||||
empty, err := store.Open(filepath.Join(root, "empty.json"))
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if got := repoint(context.Background(), empty, in); got.Via != in.Via || len(got.Files) != 1 || got.Files[0] != in.Files[0] {
|
||||
t.Errorf("unresolvable spec was changed: %+v", got)
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,262 @@
|
||||
package api
|
||||
|
||||
// The ZFS tab: pools, datasets, snapshots, and a schedule for taking and expiring them.
|
||||
//
|
||||
// grid already runs zfSnap from root cron and has done for longer than this repo has
|
||||
// existed, so this does not replace it. It reads the same snapshot names, expires by the
|
||||
// same rule, and surfaces the cron job that drives it — so what is on screen is the truth
|
||||
// about the machine, whether Ham Lab or zfSnap is doing the work. A Ham Lab schedule is for
|
||||
// a machine that has no such tool, and the page says plainly when both are running.
|
||||
|
||||
import (
|
||||
"context"
|
||||
"crypto/rand"
|
||||
"encoding/hex"
|
||||
"encoding/json"
|
||||
"io"
|
||||
"log"
|
||||
"net/http"
|
||||
"os"
|
||||
"path/filepath"
|
||||
"strings"
|
||||
"time"
|
||||
|
||||
"hamlab/internal/storage"
|
||||
"hamlab/internal/store"
|
||||
)
|
||||
|
||||
type zfsState struct {
|
||||
Policies []storage.Policy `json:"policies"`
|
||||
}
|
||||
|
||||
func (s *Server) zfsLoad() zfsState {
|
||||
var raw json.RawMessage
|
||||
s.store.View(func(st *store.State) { raw = st.ZFS })
|
||||
var z zfsState
|
||||
if len(raw) > 0 {
|
||||
_ = json.Unmarshal(raw, &z)
|
||||
}
|
||||
if z.Policies == nil {
|
||||
z.Policies = []storage.Policy{}
|
||||
}
|
||||
return z
|
||||
}
|
||||
|
||||
func (s *Server) zfsSave(z zfsState) error {
|
||||
b, err := json.Marshal(z)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
return s.store.Update(func(st *store.State) { st.ZFS = b })
|
||||
}
|
||||
|
||||
// externalSnapshotJob finds a cron job that is already taking snapshots on this machine, so
|
||||
// the page can say so rather than let two schedules quietly double up.
|
||||
func (s *Server) externalSnapshotJob(ctx context.Context) map[string]any {
|
||||
if s.cron == nil {
|
||||
return nil
|
||||
}
|
||||
ov := s.cron.Overview(ctx)
|
||||
for _, j := range ov.Jobs {
|
||||
if !j.Enabled {
|
||||
continue
|
||||
}
|
||||
c := strings.ToLower(j.Command)
|
||||
if !strings.Contains(c, "zfsnap") && !strings.Contains(c, "zfs-auto-snapshot") && !strings.Contains(c, "sanoid") &&
|
||||
!(strings.Contains(c, "zfs ") && strings.Contains(c, "snapshot")) {
|
||||
continue
|
||||
}
|
||||
return map[string]any{
|
||||
"tool": toolName(c), "schedule": j.Schedule, "command": j.Command,
|
||||
"user": j.User, "source": j.Source, "id": j.ID, "next": j.Next, "last": j.Last,
|
||||
}
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
func toolName(cmd string) string {
|
||||
switch {
|
||||
case strings.Contains(cmd, "zfsnap"):
|
||||
return "zfSnap"
|
||||
case strings.Contains(cmd, "zfs-auto-snapshot"):
|
||||
return "zfs-auto-snapshot"
|
||||
case strings.Contains(cmd, "sanoid"):
|
||||
return "sanoid"
|
||||
}
|
||||
return "a cron job"
|
||||
}
|
||||
|
||||
// randomID is a short opaque id for a schedule.
|
||||
func randomID() (string, error) {
|
||||
b := make([]byte, 6)
|
||||
if _, err := rand.Read(b); err != nil {
|
||||
return "", err
|
||||
}
|
||||
return hex.EncodeToString(b), nil
|
||||
}
|
||||
|
||||
// ---- the cron file ----
|
||||
//
|
||||
// A schedule is a cron line, written to /etc/cron.d/hamlab-zfs, which Ham Lab owns and
|
||||
// rewrites whole whenever the schedules change. Nothing in the daemon runs them: snapshots
|
||||
// are backup-shaped and should not stop because a web console is down, being upgraded, or
|
||||
// uninstalled. The line calls `hamlab zfs-snapshot <id>` as the account Ham Lab runs as,
|
||||
// which reaches zfs through sudo exactly as the daemon does.
|
||||
|
||||
// syncCron puts the current schedules on the machine.
|
||||
func (s *Server) syncCron(ctx context.Context, z zfsState) error {
|
||||
exe, err := os.Executable()
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
exe, _ = filepath.EvalSymlinks(exe)
|
||||
return s.storage.WriteCronFile(ctx, z.Policies, s.user, exe, s.cfg.Path())
|
||||
}
|
||||
|
||||
// ---- endpoints ----
|
||||
|
||||
func (s *Server) zfsOverview(w http.ResponseWriter, r *http.Request) {
|
||||
z := s.zfsLoad()
|
||||
now := time.Now()
|
||||
dir := s.cfg.DataDir()
|
||||
out := make([]map[string]any, 0, len(z.Policies))
|
||||
for i := range z.Policies {
|
||||
p := z.Policies[i]
|
||||
expr, _ := p.CronExpr()
|
||||
m := map[string]any{"policy": p, "cron": expr, "last": storage.ReadRun(dir, p.ID)}
|
||||
if p.Enabled {
|
||||
if n := p.NextRun(now); !n.IsZero() {
|
||||
m["next"] = n
|
||||
}
|
||||
}
|
||||
out = append(out, m)
|
||||
}
|
||||
jsonOK(w, 200, map[string]any{
|
||||
"zfs": s.storage.ZFS(r.Context()), "policies": out, "external": s.externalSnapshotJob(r.Context()),
|
||||
})
|
||||
}
|
||||
|
||||
func (s *Server) zfsPolicyPut(w http.ResponseWriter, r *http.Request) {
|
||||
var p storage.Policy
|
||||
if err := json.NewDecoder(io.LimitReader(r.Body, 16<<10)).Decode(&p); err != nil {
|
||||
jsonErr(w, 400, "bad request")
|
||||
return
|
||||
}
|
||||
if err := p.Validate(); err != nil {
|
||||
jsonErr(w, 400, err.Error())
|
||||
return
|
||||
}
|
||||
z := s.zfsLoad()
|
||||
found := false
|
||||
for i := range z.Policies {
|
||||
if z.Policies[i].ID == p.ID && p.ID != "" {
|
||||
z.Policies[i] = p
|
||||
found = true
|
||||
}
|
||||
}
|
||||
if !found {
|
||||
if p.ID == "" {
|
||||
id, err := randomID()
|
||||
if err != nil {
|
||||
jsonErr(w, 500, err.Error())
|
||||
return
|
||||
}
|
||||
p.ID = id
|
||||
}
|
||||
z.Policies = append(z.Policies, p)
|
||||
}
|
||||
if err := s.zfsSave(z); err != nil {
|
||||
jsonErr(w, 500, err.Error())
|
||||
return
|
||||
}
|
||||
if err := s.syncCron(r.Context(), z); err != nil {
|
||||
jsonErr(w, 500, "saved, but writing "+storage.CronFile+" failed: "+err.Error())
|
||||
return
|
||||
}
|
||||
expr, _ := p.CronExpr()
|
||||
log.Printf("zfs schedule %q saved by %s (%s, %q, keep %s)", p.Name, clientIP(r), p.Dataset, expr, p.KeepFor)
|
||||
jsonOK(w, 200, p)
|
||||
}
|
||||
|
||||
func (s *Server) zfsPolicyDelete(w http.ResponseWriter, r *http.Request) {
|
||||
id := r.PathValue("id")
|
||||
z := s.zfsLoad()
|
||||
kept := z.Policies[:0]
|
||||
found := false
|
||||
for _, p := range z.Policies {
|
||||
if p.ID == id {
|
||||
found = true
|
||||
continue
|
||||
}
|
||||
kept = append(kept, p)
|
||||
}
|
||||
if !found {
|
||||
jsonErr(w, 404, "no such schedule")
|
||||
return
|
||||
}
|
||||
z.Policies = kept
|
||||
if err := s.zfsSave(z); err != nil {
|
||||
jsonErr(w, 500, err.Error())
|
||||
return
|
||||
}
|
||||
if err := s.syncCron(r.Context(), z); err != nil {
|
||||
jsonErr(w, 500, "removed, but writing "+storage.CronFile+" failed: "+err.Error())
|
||||
return
|
||||
}
|
||||
storage.ForgetRun(s.cfg.DataDir(), id)
|
||||
log.Printf("zfs schedule %s removed by %s (its snapshots are left alone)", id, clientIP(r))
|
||||
w.WriteHeader(204)
|
||||
}
|
||||
|
||||
func (s *Server) zfsPolicy(id string) (storage.Policy, bool) {
|
||||
for _, p := range s.zfsLoad().Policies {
|
||||
if p.ID == id {
|
||||
return p, true
|
||||
}
|
||||
}
|
||||
return storage.Policy{}, false
|
||||
}
|
||||
|
||||
func (s *Server) zfsRunNow(w http.ResponseWriter, r *http.Request) {
|
||||
id := r.PathValue("id")
|
||||
p, ok := s.zfsPolicy(id)
|
||||
if !ok {
|
||||
jsonErr(w, 404, "no such schedule")
|
||||
return
|
||||
}
|
||||
res, err := s.storage.Run(r.Context(), &p)
|
||||
rec := storage.RunRecord{At: time.Now(), Taken: res.Taken, Pruned: res.Pruned}
|
||||
if err != nil {
|
||||
rec.Error = err.Error()
|
||||
}
|
||||
_ = storage.WriteRun(s.cfg.DataDir(), id, rec)
|
||||
if err != nil {
|
||||
jsonErr(w, 500, err.Error())
|
||||
return
|
||||
}
|
||||
log.Printf("zfs schedule %q run by %s: took %s, expired %d", p.Name, clientIP(r), res.Taken, len(res.Pruned))
|
||||
jsonOK(w, 200, res)
|
||||
}
|
||||
|
||||
// zfsSnapshotDelete removes one snapshot, named in full and typed back by the caller.
|
||||
func (s *Server) zfsSnapshotDelete(w http.ResponseWriter, r *http.Request) {
|
||||
var body struct {
|
||||
Name string `json:"name"`
|
||||
Confirm string `json:"confirm"`
|
||||
Recursive bool `json:"recursive"`
|
||||
}
|
||||
if err := json.NewDecoder(io.LimitReader(r.Body, 4096)).Decode(&body); err != nil {
|
||||
jsonErr(w, 400, "bad request")
|
||||
return
|
||||
}
|
||||
if body.Confirm != body.Name {
|
||||
jsonErr(w, 400, "type the snapshot's full name to confirm")
|
||||
return
|
||||
}
|
||||
if err := s.storage.DestroySnapshot(r.Context(), body.Name, body.Recursive); err != nil {
|
||||
jsonErr(w, 500, err.Error())
|
||||
return
|
||||
}
|
||||
log.Printf("zfs snapshot %s destroyed by %s%s", body.Name, clientIP(r), map[bool]string{true: " (recursive)", false: ""}[body.Recursive])
|
||||
w.WriteHeader(204)
|
||||
}
|
||||
@@ -0,0 +1,50 @@
|
||||
// Package compose runs docker compose the way a stack was deployed — which is not always
|
||||
// simply "on this host". A stack deployed by an orchestrator running inside its own
|
||||
// container records paths in that container's filesystem, so compose has to run in there;
|
||||
// a stack deployed to another machine over tcp needs DOCKER_HOST pointed at it. Both the
|
||||
// console and the scheduled updates it writes must invoke it identically, or a redeploy
|
||||
// from one would not match the other's labels.
|
||||
package compose
|
||||
|
||||
import (
|
||||
"context"
|
||||
"os"
|
||||
"os/exec"
|
||||
"path/filepath"
|
||||
)
|
||||
|
||||
// Spec is everything needed to run compose for one stack.
|
||||
type Spec struct {
|
||||
Project string // -p
|
||||
Files []string // -f, in order
|
||||
Via string // a container to run compose inside; "" is this host
|
||||
Remote string // a DOCKER_HOST; "" is this machine's Docker
|
||||
Tool string // "docker" or "podman"
|
||||
}
|
||||
|
||||
// Cmd builds the command. args are compose's own, e.g. "pull" or "up", "-d".
|
||||
func Cmd(ctx context.Context, s Spec, args ...string) *exec.Cmd {
|
||||
tool := s.Tool
|
||||
if tool == "" {
|
||||
tool = "docker"
|
||||
}
|
||||
base := []string{"compose", "-p", s.Project}
|
||||
for _, f := range s.Files {
|
||||
base = append(base, "-f", f)
|
||||
}
|
||||
if s.Via != "" {
|
||||
pre := []string{"exec"}
|
||||
if s.Remote != "" {
|
||||
pre = append(pre, "-e", "DOCKER_HOST="+s.Remote)
|
||||
}
|
||||
return exec.CommandContext(ctx, tool, append(append(append(pre, s.Via, "docker"), base...), args...)...)
|
||||
}
|
||||
cmd := exec.CommandContext(ctx, tool, append(base, args...)...)
|
||||
if len(s.Files) > 0 {
|
||||
cmd.Dir = filepath.Dir(s.Files[0])
|
||||
}
|
||||
if s.Remote != "" {
|
||||
cmd.Env = append(os.Environ(), "DOCKER_HOST="+s.Remote)
|
||||
}
|
||||
return cmd
|
||||
}
|
||||
@@ -61,6 +61,23 @@ type UpdateSource struct {
|
||||
Check string `json:"check,omitempty"`
|
||||
}
|
||||
|
||||
// KVM is an IP-KVM: a device with a web page that gives screen, keyboard and mouse access
|
||||
// to a machine, including when that machine has no OS running. Ham Lab holds the list on
|
||||
// the hub and proxies each one, because a KVM matters exactly when the machine it is
|
||||
// attached to cannot answer for itself.
|
||||
type KVM struct {
|
||||
ID string `json:"id"`
|
||||
Name string `json:"name"`
|
||||
// URL is the device's own address, as you would type it into a browser on the LAN.
|
||||
URL string `json:"url"`
|
||||
// Machine is the host this KVM is attached to, and is the key: one KVM per machine, shown
|
||||
// on that machine's KVM tab. Empty means the hub itself. The list lives on the hub even
|
||||
// for a node's KVM, because a KVM earns its keep when the machine is down and cannot
|
||||
// answer for itself.
|
||||
Machine string `json:"machine"`
|
||||
Added time.Time `json:"added"`
|
||||
}
|
||||
|
||||
type Config struct {
|
||||
Listen string `json:"listen"`
|
||||
// PasswordHash is the single login of installs before users existed. On load it becomes
|
||||
@@ -70,6 +87,7 @@ type Config struct {
|
||||
TLS string `json:"tls,omitempty"` // "auto" → self-signed cert in the data dir; "" → plain HTTP
|
||||
Tokens []Token `json:"tokens,omitempty"`
|
||||
Nodes []Node `json:"nodes,omitempty"`
|
||||
KVMs []KVM `json:"kvms,omitempty"`
|
||||
// Updates: where a newer build comes from. See UpdateSource.
|
||||
Updates UpdateSource `json:"update,omitempty"`
|
||||
|
||||
|
||||
@@ -1,3 +1,6 @@
|
||||
// Package cron reads and writes what the UI calls the **Jobs** tab (named "Cron" until
|
||||
// 2026-09-24; the package and its /api/cron endpoints kept the old name).
|
||||
//
|
||||
// Package cron reads every scheduled job on the host into one list: /etc/crontab,
|
||||
// /etc/cron.d/*, per-user crontabs, the run-parts directories they drive, and systemd
|
||||
// timers. It computes next runs from the cron expressions, pulls last runs from the
|
||||
|
||||
@@ -132,6 +132,17 @@ func (s *schedule) dayMatches(t time.Time) bool {
|
||||
}
|
||||
}
|
||||
|
||||
// NextRun is the first time strictly after `from` that a cron expression fires. Exported so
|
||||
// anything that writes a cron line — the ZFS snapshot schedules — can say when it runs next
|
||||
// using the same evaluator that reads everyone else's.
|
||||
func NextRun(expr string, from time.Time) (time.Time, bool) { return nextRun(expr, from) }
|
||||
|
||||
// ValidSchedule reports whether a cron expression parses.
|
||||
func ValidSchedule(expr string) error {
|
||||
_, err := parseSchedule(expr)
|
||||
return err
|
||||
}
|
||||
|
||||
// nextRun finds the first time strictly after `from` that the schedule fires.
|
||||
func nextRun(expr string, from time.Time) (time.Time, bool) {
|
||||
s, err := parseSchedule(expr)
|
||||
|
||||
@@ -90,6 +90,15 @@ type FS struct {
|
||||
Type string `json:"type"`
|
||||
Size uint64 `json:"size"`
|
||||
Used uint64 `json:"used"`
|
||||
// Remote is set when the filesystem lives on another machine — a CIFS or NFS share,
|
||||
// sshfs, and so on. Its bytes are somebody else's disk, so a panel about this machine's
|
||||
// hardware leaves it out, while the sensor stays for anyone who does want it.
|
||||
Remote bool `json:"remote,omitempty"`
|
||||
// Disk is the physical drive these bytes are on ("nvme0", "sda"), named as the sampler
|
||||
// names it; empty when that cannot be
|
||||
// said honestly: a ZFS pool or an md array spreads a filesystem over several drives, and
|
||||
// a remote mount is on none of them.
|
||||
Disk string `json:"disk,omitempty"`
|
||||
}
|
||||
|
||||
type UPS struct {
|
||||
@@ -766,6 +775,15 @@ func foldNVMe(devs []Disk) []Disk {
|
||||
|
||||
// Kernel/virtual filesystems that aren't storage. Anything else that's mounted is
|
||||
// shown, so new real filesystems (bcachefs, exotic FUSE) work without a code change.
|
||||
// remoteFS is every filesystem type whose storage is not in this machine.
|
||||
var remoteFS = map[string]bool{
|
||||
"cifs": true, "smb3": true, "smbfs": true, "nfs": true, "nfs4": true, "afs": true,
|
||||
"ceph": true, "glusterfs": true, "9p": true, "afpfs": true, "davfs": true,
|
||||
"fuse.sshfs": true, "sshfs": true, "fuse.rclone": true, "fuse.s3fs": true,
|
||||
"fuse.davfs2": true, "fuse.curlftpfs": true, "fuse.gcsfuse": true, "beegfs": true,
|
||||
"lustre": true, "orangefs": true, "pvfs2": true, "coda": true, "ncpfs": true,
|
||||
}
|
||||
|
||||
var pseudoFS = map[string]bool{
|
||||
"proc": true, "sysfs": true, "devtmpfs": true, "devpts": true, "tmpfs": true, "ramfs": true,
|
||||
"cgroup": true, "cgroup2": true, "pstore": true, "bpf": true, "securityfs": true, "debugfs": true,
|
||||
@@ -812,7 +830,7 @@ func filesystems() []FS {
|
||||
return
|
||||
}
|
||||
bs := uint64(st.Bsize)
|
||||
done <- FS{Mount: mount, Type: fs[2], Size: st.Blocks * bs, Used: (st.Blocks - st.Bfree) * bs}
|
||||
done <- FS{Mount: mount, Type: fs[2], Size: st.Blocks * bs, Used: (st.Blocks - st.Bfree) * bs, Remote: remoteFS[fs[2]], Disk: diskOf(fs[0])}
|
||||
}()
|
||||
select {
|
||||
case r := <-done:
|
||||
@@ -820,12 +838,52 @@ func filesystems() []FS {
|
||||
out = append(out, r)
|
||||
}
|
||||
case <-time.After(3 * time.Second):
|
||||
out = append(out, FS{Mount: mount, Type: fs[2]}) // unreachable: size 0 signals it
|
||||
out = append(out, FS{Mount: mount, Type: fs[2], Remote: remoteFS[fs[2]]}) // unreachable: size 0 signals it
|
||||
}
|
||||
}
|
||||
return out
|
||||
}
|
||||
|
||||
// diskOf names the physical drive a mount's source device sits on — "/dev/nvme0n1p2" is on
|
||||
// "nvme0n1". It follows one layer of device-mapper or md (LUKS on a partition, say) but
|
||||
// gives up when a thing spans several drives, because then no single drive is the answer.
|
||||
func diskOf(source string) string {
|
||||
if !strings.HasPrefix(source, "/dev/") {
|
||||
return "" // a ZFS pool, a network share, a tmpfs: not a block device at all
|
||||
}
|
||||
d := parentDisk(strings.TrimPrefix(source, "/dev/"), 0)
|
||||
// the sampler reports one drive per NVMe controller, so name it the same way
|
||||
if m := nsRe.FindStringSubmatch(d); m != nil {
|
||||
return m[1]
|
||||
}
|
||||
return d
|
||||
}
|
||||
|
||||
func parentDisk(name string, depth int) string {
|
||||
if name == "" || depth > 4 || strings.Contains(name, "/") {
|
||||
return ""
|
||||
}
|
||||
base := "/sys/class/block/" + name
|
||||
if _, err := os.Stat(base); err != nil {
|
||||
return ""
|
||||
}
|
||||
// a partition: its parent directory in sysfs is the whole drive
|
||||
if _, err := os.Stat(base + "/partition"); err == nil {
|
||||
if target, err := os.Readlink(base); err == nil {
|
||||
return filepath.Base(filepath.Dir(target))
|
||||
}
|
||||
return ""
|
||||
}
|
||||
// device-mapper or md: one drive underneath is an answer, several is not
|
||||
if ents, err := os.ReadDir(base + "/slaves"); err == nil && len(ents) > 0 {
|
||||
if len(ents) > 1 {
|
||||
return ""
|
||||
}
|
||||
return parentDisk(ents[0].Name(), depth+1)
|
||||
}
|
||||
return name // a whole drive, used without partitioning
|
||||
}
|
||||
|
||||
// ---- UPS ----
|
||||
|
||||
func runOut(args ...string) (string, bool) {
|
||||
|
||||
@@ -117,9 +117,18 @@ func flatten(s Sample, want func(id string) bool) []sensors.Sensor {
|
||||
|
||||
for _, x := range s.FS {
|
||||
p := "fs." + sensors.Seg(x.Mount)
|
||||
f.add(p+".size", x.Mount+" size", "bytes", "B", float64(x.Size), s.fsT, none, "mount", x.Mount, "fstype", x.Type)
|
||||
f.add(p+".used", x.Mount+" used", "bytes", "B", float64(x.Used), s.fsT, upTo(float64(x.Size)), "mount", x.Mount, "fstype", x.Type)
|
||||
f.add(p+".used_pct", x.Mount+" used", "usage", "%", ratio(x.Used, x.Size), s.fsT, pct, "mount", x.Mount, "fstype", x.Type)
|
||||
// remote says the bytes are on another machine (a CIFS or NFS share), so a view of
|
||||
// this machine's hardware can leave it out without losing the sensor
|
||||
lbl := []string{"mount", x.Mount, "fstype", x.Type}
|
||||
if x.Remote {
|
||||
lbl = append(lbl, "remote", "1")
|
||||
}
|
||||
if x.Disk != "" {
|
||||
lbl = append(lbl, "disk", x.Disk)
|
||||
}
|
||||
f.add(p+".size", x.Mount+" size", "bytes", "B", float64(x.Size), s.fsT, none, lbl...)
|
||||
f.add(p+".used", x.Mount+" used", "bytes", "B", float64(x.Used), s.fsT, upTo(float64(x.Size)), lbl...)
|
||||
f.add(p+".used_pct", x.Mount+" used", "usage", "%", ratio(x.Used, x.Size), s.fsT, pct, lbl...)
|
||||
}
|
||||
|
||||
if u := s.UPS; u != nil {
|
||||
|
||||
@@ -1,4 +1,6 @@
|
||||
// Package space measures what is using a disk: a parallel walk of the filesystems a disk
|
||||
// Package space measures what is using a disk — it is what the UI calls the **Usage** tab
|
||||
// (renamed from "Space" on 2026-09-24; the package and its /api/space endpoints kept the
|
||||
// old name rather than churn a working API): a parallel walk of the filesystems a disk
|
||||
// backs, kept as a tree of directories and the files worth seeing, the way WizTree does it.
|
||||
//
|
||||
// The walk never crosses a filesystem boundary by itself — /proc, /sys, another disk's
|
||||
|
||||
@@ -0,0 +1,493 @@
|
||||
package storage
|
||||
|
||||
// Scheduled snapshots with a retention period: take a snapshot of a dataset on a schedule,
|
||||
// and drop the ones that have aged out.
|
||||
//
|
||||
// **The naming is zfSnap's, deliberately.** grid has run zfSnap v1.11.1 from root cron since
|
||||
// before this repo existed and it works well, so rather than invent a second convention and
|
||||
// have two tools disagree about what a snapshot is, this writes and reads exactly what
|
||||
// zfSnap does: `<dataset>@2026-09-24_02.30.01--6w`, with the retention encoded in the name.
|
||||
// That has three consequences worth knowing. Ham Lab can show and manage the snapshots
|
||||
// zfSnap already took. The two can run side by side without fighting, because both only
|
||||
// ever destroy snapshots that have passed the expiry written into their own name. And
|
||||
// changing a policy's retention does not retroactively re-date old snapshots — each one
|
||||
// carries the rule it was taken under, which is the behaviour you want from a backup.
|
||||
//
|
||||
// Two rules make this safe to run unattended. Only a snapshot whose name parses as a stamp
|
||||
// and a TTL is ever a candidate — anything taken by hand or named anything else is invisible
|
||||
// to it. And every policy can be asked what it would delete, before it deletes anything.
|
||||
|
||||
import (
|
||||
"bytes"
|
||||
"context"
|
||||
"encoding/json"
|
||||
"errors"
|
||||
"fmt"
|
||||
"os"
|
||||
"os/exec"
|
||||
"path/filepath"
|
||||
"sort"
|
||||
"strconv"
|
||||
"strings"
|
||||
"time"
|
||||
|
||||
"hamlab/internal/cron"
|
||||
"hamlab/internal/priv"
|
||||
)
|
||||
|
||||
// stampLayout is zfSnap's timestamp: 2026-09-24_02.30.01.
|
||||
const stampLayout = "2006-01-02_15.04.05"
|
||||
|
||||
// ttlSep separates the stamp from the retention in a snapshot name, as zfSnap writes it.
|
||||
const ttlSep = "--"
|
||||
|
||||
// Policy is a standing instruction: snapshot this dataset on this schedule, keep the
|
||||
// snapshots this long.
|
||||
type Policy struct {
|
||||
ID string `json:"id"`
|
||||
Name string `json:"name"`
|
||||
Dataset string `json:"dataset"`
|
||||
// Recursive takes one atomic snapshot of the dataset and everything under it (zfs -r).
|
||||
Recursive bool `json:"recursive"`
|
||||
// Every is hourly | daily | weekly | monthly | custom. At is "HH:MM" (the minute alone
|
||||
// is used for hourly). Weekday is 0=Sunday, Day is 1–28. With "custom", Cron is a plain
|
||||
// cron expression and the rest is ignored — the escape hatch for anything the four
|
||||
// common shapes cannot say.
|
||||
Every string `json:"every"`
|
||||
At string `json:"at"`
|
||||
Weekday int `json:"weekday"`
|
||||
Day int `json:"day"`
|
||||
Cron string `json:"cron,omitempty"`
|
||||
// KeepFor is how long a snapshot lives: "6w", "30d", "12h". It is written into the
|
||||
// snapshot's own name, zfSnap-style, so each one carries the rule it was taken under.
|
||||
// KeepMin is a floor — that many newest snapshots survive however old they are, so a
|
||||
// dataset is never left with none.
|
||||
KeepFor string `json:"keep_for"`
|
||||
KeepMin int `json:"keep_min"`
|
||||
Prefix string `json:"prefix"`
|
||||
Enabled bool `json:"enabled"`
|
||||
|
||||
}
|
||||
|
||||
func (p *Policy) prefix() string { return strings.TrimSpace(p.Prefix) }
|
||||
|
||||
// Validate reports what is wrong with a policy, or nil.
|
||||
func (p *Policy) Validate() error {
|
||||
if !datasetRe.MatchString(p.Dataset) {
|
||||
return errors.New("dataset: a pool or pool/dataset name")
|
||||
}
|
||||
if pre := p.prefix(); pre != "" && (!snapRe.MatchString(pre) || strings.Contains(pre, ttlSep)) {
|
||||
return errors.New("prefix: letters, digits, . _ : and - only, and no --")
|
||||
}
|
||||
switch p.Every {
|
||||
case "hourly", "daily", "weekly", "monthly":
|
||||
case "custom":
|
||||
if strings.TrimSpace(p.Cron) == "" {
|
||||
return errors.New("a custom schedule needs a cron expression")
|
||||
}
|
||||
default:
|
||||
return errors.New("every: hourly, daily, weekly, monthly or custom")
|
||||
}
|
||||
if _, err := p.CronExpr(); err != nil {
|
||||
return err
|
||||
}
|
||||
if _, _, err := p.clock(); err != nil {
|
||||
return err
|
||||
}
|
||||
if p.Every == "weekly" && (p.Weekday < 0 || p.Weekday > 6) {
|
||||
return errors.New("weekday: 0 (Sunday) to 6")
|
||||
}
|
||||
if p.Every == "monthly" && (p.Day < 1 || p.Day > 28) {
|
||||
return errors.New("day: 1 to 28, so every month has one")
|
||||
}
|
||||
if p.KeepFor != "" {
|
||||
if _, err := ParseAge(p.KeepFor); err != nil {
|
||||
return err
|
||||
}
|
||||
}
|
||||
if p.KeepMin < 0 {
|
||||
return errors.New("keep at least: 0 or more")
|
||||
}
|
||||
if p.KeepFor == "" {
|
||||
return errors.New("a retention period is required — it is written into each snapshot's name, and is what lets it be cleaned up later")
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
func (p *Policy) clock() (hour, min int, err error) {
|
||||
at := strings.TrimSpace(p.At)
|
||||
if at == "" {
|
||||
return 0, 0, nil
|
||||
}
|
||||
h, m, ok := strings.Cut(at, ":")
|
||||
if !ok {
|
||||
return 0, 0, errors.New("at: HH:MM")
|
||||
}
|
||||
hour, err = strconv.Atoi(h)
|
||||
if err == nil {
|
||||
min, err = strconv.Atoi(m)
|
||||
}
|
||||
if err != nil || hour < 0 || hour > 23 || min < 0 || min > 59 {
|
||||
return 0, 0, errors.New("at: HH:MM, 00:00 to 23:59")
|
||||
}
|
||||
return hour, min, nil
|
||||
}
|
||||
|
||||
// ParseAge reads "6w", "30d", "12h", "90m" as a duration. Weeks and days are what a
|
||||
// retention period is actually written in, and time.ParseDuration knows neither.
|
||||
func ParseAge(s string) (time.Duration, error) {
|
||||
s = strings.TrimSpace(strings.ToLower(s))
|
||||
if s == "" {
|
||||
return 0, errors.New("empty period")
|
||||
}
|
||||
unit := s[len(s)-1]
|
||||
n, err := strconv.Atoi(s[:len(s)-1])
|
||||
if err != nil || n <= 0 {
|
||||
return 0, fmt.Errorf("%q: a number and a unit, like 6w, 30d or 12h", s)
|
||||
}
|
||||
switch unit {
|
||||
case 'm':
|
||||
return time.Duration(n) * time.Minute, nil
|
||||
case 'h':
|
||||
return time.Duration(n) * time.Hour, nil
|
||||
case 'd':
|
||||
return time.Duration(n) * 24 * time.Hour, nil
|
||||
case 'w':
|
||||
return time.Duration(n) * 7 * 24 * time.Hour, nil
|
||||
}
|
||||
return 0, fmt.Errorf("%q: the unit is m, h, d or w", s)
|
||||
}
|
||||
|
||||
// CronExpr renders the schedule as a cron expression — which is what actually runs it. The
|
||||
// four common shapes are sugar over this; a custom policy states it outright. One expression
|
||||
// is the only description of when a schedule fires, so the page, the cron file and the next
|
||||
// run can never disagree.
|
||||
func (p *Policy) CronExpr() (string, error) {
|
||||
if p.Every == "custom" {
|
||||
expr := strings.Join(strings.Fields(p.Cron), " ")
|
||||
if err := cron.ValidSchedule(expr); err != nil {
|
||||
return "", fmt.Errorf("cron expression: %w", err)
|
||||
}
|
||||
if strings.HasPrefix(expr, "@reboot") {
|
||||
return "", errors.New("@reboot is not a schedule for snapshots")
|
||||
}
|
||||
return expr, nil
|
||||
}
|
||||
hour, min, err := p.clock()
|
||||
if err != nil {
|
||||
return "", err
|
||||
}
|
||||
switch p.Every {
|
||||
case "hourly":
|
||||
return fmt.Sprintf("%d * * * *", min), nil
|
||||
case "daily":
|
||||
return fmt.Sprintf("%d %d * * *", min, hour), nil
|
||||
case "weekly":
|
||||
return fmt.Sprintf("%d %d * * %d", min, hour, p.Weekday), nil
|
||||
case "monthly":
|
||||
return fmt.Sprintf("%d %d %d * *", min, hour, p.Day), nil
|
||||
}
|
||||
return "", errors.New("unknown schedule")
|
||||
}
|
||||
|
||||
// NextRun is when cron will next fire this schedule, read with the same evaluator the Jobs
|
||||
// tab uses on everyone else's lines.
|
||||
func (p *Policy) NextRun(now time.Time) time.Time {
|
||||
expr, err := p.CronExpr()
|
||||
if err != nil {
|
||||
return time.Time{}
|
||||
}
|
||||
t, ok := cron.NextRun(expr, now)
|
||||
if !ok {
|
||||
return time.Time{}
|
||||
}
|
||||
return t
|
||||
}
|
||||
|
||||
// SnapTime reads a snapshot's name in zfSnap's convention and returns when it was taken,
|
||||
// when it expires, and whether the name parsed at all. A name without a TTL never expires.
|
||||
// An optional prefix in front of the stamp is allowed, so a policy can label its own.
|
||||
func SnapTime(name string) (taken time.Time, expires time.Time, ok bool) {
|
||||
_, snap, cut := strings.Cut(name, "@")
|
||||
if !cut {
|
||||
return
|
||||
}
|
||||
body, ttl, hasTTL := strings.Cut(snap, ttlSep)
|
||||
// an optional "<prefix>-" before the stamp
|
||||
stamp := body
|
||||
if len(body) > len(stampLayout) {
|
||||
stamp = body[len(body)-len(stampLayout):]
|
||||
}
|
||||
t, err := time.ParseInLocation(stampLayout, stamp, time.Local)
|
||||
if err != nil {
|
||||
return time.Time{}, time.Time{}, false
|
||||
}
|
||||
taken, ok = t, true
|
||||
if hasTTL {
|
||||
if d, err := ParseAge(ttl); err == nil {
|
||||
expires = t.Add(d)
|
||||
}
|
||||
}
|
||||
return
|
||||
}
|
||||
|
||||
// Doomed is which of this dataset's snapshots have passed the expiry written into their own
|
||||
// names, oldest first — exactly what zfSnap -d would remove. This is what the UI shows
|
||||
// before anything is deleted, and what Run then acts on. KeepMin holds back that many of the
|
||||
// newest whatever their names say, so a dataset is never left with none.
|
||||
func (m *Manager) Doomed(ctx context.Context, p *Policy) ([]Snapshot, error) {
|
||||
if err := p.Validate(); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
all, err := m.Snapshots(ctx, p.Dataset)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
var dated []Snapshot
|
||||
expiry := map[string]time.Time{}
|
||||
for _, s := range all {
|
||||
taken, exp, ok := SnapTime(s.Name)
|
||||
if !ok || exp.IsZero() {
|
||||
continue // hand-made, or kept forever: not ours to remove
|
||||
}
|
||||
_ = taken
|
||||
dated = append(dated, s)
|
||||
expiry[s.Name] = exp
|
||||
}
|
||||
sort.Slice(dated, func(i, j int) bool { return dated[i].Created > dated[j].Created }) // newest first
|
||||
now := time.Now()
|
||||
var doomed []Snapshot
|
||||
for i, s := range dated {
|
||||
if i < p.KeepMin {
|
||||
continue // the floor: these survive however old they are
|
||||
}
|
||||
if expiry[s.Name].Before(now) {
|
||||
doomed = append(doomed, s)
|
||||
}
|
||||
}
|
||||
sort.Slice(doomed, func(i, j int) bool { return doomed[i].Created < doomed[j].Created }) // oldest first
|
||||
return doomed, nil
|
||||
}
|
||||
|
||||
// DestroySnapshot removes one snapshot. recursive also removes the same-named snapshot on
|
||||
// every dataset beneath it, which is what a recursive policy created.
|
||||
func (m *Manager) DestroySnapshot(ctx context.Context, full string, recursive bool) error {
|
||||
dataset, snap, ok := strings.Cut(full, "@")
|
||||
if !ok || !datasetRe.MatchString(dataset) || !snapRe.MatchString(snap) {
|
||||
return errors.New("bad snapshot name")
|
||||
}
|
||||
args := []string{"zfs", "destroy"}
|
||||
if recursive {
|
||||
args = append(args, "-r")
|
||||
}
|
||||
out, code, err := m.run(ctx, append(args, full)...)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
if code != 0 {
|
||||
return errors.New(strings.TrimSpace(string(out)))
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
// SnapshotRecursive takes dataset@name, optionally of everything beneath it too.
|
||||
func (m *Manager) SnapshotRecursive(ctx context.Context, dataset, name string, recursive bool) error {
|
||||
if !datasetRe.MatchString(dataset) || !snapRe.MatchString(name) {
|
||||
return errors.New("bad snapshot name")
|
||||
}
|
||||
args := []string{"zfs", "snapshot"}
|
||||
if recursive {
|
||||
args = append(args, "-r")
|
||||
}
|
||||
out, code, err := m.run(ctx, append(args, dataset+"@"+name)...)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
if code != 0 {
|
||||
return errors.New(strings.TrimSpace(string(out)))
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
// RunResult is what one pass of a policy did.
|
||||
type RunResult struct {
|
||||
Taken string `json:"taken"`
|
||||
Pruned []string `json:"pruned"`
|
||||
Skipped int `json:"skipped"` // snapshots that are not this policy's, left alone
|
||||
}
|
||||
|
||||
// Run takes the snapshot and prunes what has aged out. It is the same code the scheduler
|
||||
// calls, so "Run now" and the timer cannot drift apart.
|
||||
func (m *Manager) Run(ctx context.Context, p *Policy) (RunResult, error) {
|
||||
var res RunResult
|
||||
if err := p.Validate(); err != nil {
|
||||
return res, err
|
||||
}
|
||||
name := time.Now().Format(stampLayout)
|
||||
if pre := strings.TrimSpace(p.Prefix); pre != "" {
|
||||
name = pre + "-" + name
|
||||
}
|
||||
if p.KeepFor != "" {
|
||||
name += ttlSep + strings.TrimSpace(strings.ToLower(p.KeepFor)) // zfSnap's convention: the retention lives in the name
|
||||
}
|
||||
if err := m.SnapshotRecursive(ctx, p.Dataset, name, p.Recursive); err != nil {
|
||||
return res, err
|
||||
}
|
||||
res.Taken = p.Dataset + "@" + name
|
||||
doomed, err := m.Doomed(ctx, p)
|
||||
if err != nil {
|
||||
return res, err // the snapshot was taken; the prune is what failed
|
||||
}
|
||||
for _, s := range doomed {
|
||||
if err := m.DestroySnapshot(ctx, s.Name, p.Recursive); err != nil {
|
||||
return res, fmt.Errorf("destroying %s: %w", s.Name, err)
|
||||
}
|
||||
res.Pruned = append(res.Pruned, s.Name)
|
||||
}
|
||||
return res, nil
|
||||
}
|
||||
|
||||
// ---- what happened when it ran ----
|
||||
//
|
||||
// The policies are a definition the daemon owns; what a run did is written by whatever ran
|
||||
// it, which is a cron job in its own process. Keeping the two apart means the CLI never has
|
||||
// to write state.json behind the daemon's back — the last writer would win and one of them
|
||||
// would lose its work.
|
||||
|
||||
// RunRecord is the outcome of one run of one schedule.
|
||||
type RunRecord struct {
|
||||
At time.Time `json:"at"`
|
||||
Taken string `json:"taken,omitempty"`
|
||||
Pruned []string `json:"pruned,omitempty"`
|
||||
Error string `json:"error,omitempty"`
|
||||
}
|
||||
|
||||
func runPath(dir, id string) string { return filepath.Join(dir, "zfs-runs", id+".json") }
|
||||
|
||||
// WriteRun records what a run did, beside the config.
|
||||
func WriteRun(dir, id string, r RunRecord) error {
|
||||
if err := os.MkdirAll(filepath.Join(dir, "zfs-runs"), 0o700); err != nil {
|
||||
return err
|
||||
}
|
||||
b, err := json.Marshal(r)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
p := runPath(dir, id)
|
||||
tmp := p + ".tmp"
|
||||
if err := os.WriteFile(tmp, b, 0o600); err != nil {
|
||||
return err
|
||||
}
|
||||
return os.Rename(tmp, p)
|
||||
}
|
||||
|
||||
// ReadRun is the last thing a schedule did, or the zero value if it has never run.
|
||||
func ReadRun(dir, id string) RunRecord {
|
||||
var r RunRecord
|
||||
b, err := os.ReadFile(runPath(dir, id))
|
||||
if err == nil {
|
||||
_ = json.Unmarshal(b, &r)
|
||||
}
|
||||
return r
|
||||
}
|
||||
|
||||
// ForgetRun drops a schedule's record, for when the schedule itself goes.
|
||||
func ForgetRun(dir, id string) { _ = os.Remove(runPath(dir, id)) }
|
||||
|
||||
// ---- the cron file ----
|
||||
|
||||
// CronFile is where Ham Lab writes the snapshot schedules. One file it owns entirely and
|
||||
// rewrites whole, so there is never a half-updated set of lines, and so a schedule removed
|
||||
// in the UI is removed from the machine.
|
||||
const CronFile = "/etc/cron.d/hamlab-zfs"
|
||||
|
||||
// CronFileText renders every schedule as cron lines. user is who they run as — the account
|
||||
// Ham Lab itself runs as, which reaches zfs through sudo exactly as the daemon does, so the
|
||||
// files it writes stay owned by that account rather than by root.
|
||||
func CronFileText(policies []Policy, user, exe, config string) (string, error) {
|
||||
var b strings.Builder
|
||||
b.WriteString("# Ham Lab snapshot schedules — written by the ZFS tab, rewritten whole on every change.\n")
|
||||
b.WriteString("# Edit them there, not here: anything added by hand is lost the next time a schedule changes.\n")
|
||||
b.WriteString("SHELL=/bin/sh\n")
|
||||
b.WriteString("PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin\n\n")
|
||||
for i := range policies {
|
||||
p := &policies[i]
|
||||
if !p.Enabled {
|
||||
b.WriteString("# " + p.Name + " — paused\n")
|
||||
continue
|
||||
}
|
||||
expr, err := p.CronExpr()
|
||||
if err != nil {
|
||||
return "", fmt.Errorf("%s: %w", p.Name, err)
|
||||
}
|
||||
name := strings.ReplaceAll(p.Name, "\n", " ")
|
||||
fmt.Fprintf(&b, "# %s — %s, keep %s\n", name, p.Dataset, p.KeepFor)
|
||||
fmt.Fprintf(&b, "%s %s %s -config %s zfs-snapshot %s\n\n", expr, user, exe, config, p.ID)
|
||||
}
|
||||
return b.String(), nil
|
||||
}
|
||||
|
||||
// WriteCronFile puts the schedules on the machine, or removes the file when there are none.
|
||||
func (m *Manager) WriteCronFile(ctx context.Context, policies []Policy, user, exe, config string) error {
|
||||
live := false
|
||||
for i := range policies {
|
||||
if policies[i].Enabled {
|
||||
live = true
|
||||
}
|
||||
}
|
||||
if !live {
|
||||
out, code, err := m.run(ctx, "rm", "-f", CronFile)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
if code != 0 {
|
||||
return errors.New(strings.TrimSpace(string(out)))
|
||||
}
|
||||
return nil
|
||||
}
|
||||
text, err := CronFileText(policies, user, exe, config)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
return m.writeRoot(ctx, CronFile, text)
|
||||
}
|
||||
|
||||
// WriteRootFile puts text at a root-owned path, through the same privilege escalation every
|
||||
// other write here uses. Shared by the snapshot schedules and the image-update schedules,
|
||||
// which both own a file in /etc/cron.d.
|
||||
func (m *Manager) WriteRootFile(ctx context.Context, path, text string) error {
|
||||
return m.writeRoot(ctx, path, text)
|
||||
}
|
||||
|
||||
// RemoveRootFile deletes one, for when the last schedule of a kind goes away.
|
||||
func (m *Manager) RemoveRootFile(ctx context.Context, path string) error {
|
||||
out, code, err := m.run(ctx, "rm", "-f", path)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
if code != 0 {
|
||||
return errors.New(strings.TrimSpace(string(out)))
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
func (m *Manager) writeRoot(ctx context.Context, path, text string) error {
|
||||
args := []string{"tee", path}
|
||||
if m.sudo {
|
||||
args = priv.Wrap(args...)
|
||||
}
|
||||
ctx, cancel := context.WithTimeout(ctx, 30*time.Second)
|
||||
defer cancel()
|
||||
cmd := exec.CommandContext(ctx, args[0], args[1:]...)
|
||||
cmd.Stdin = strings.NewReader(text)
|
||||
var errb bytes.Buffer
|
||||
cmd.Stdout, cmd.Stderr = nil, &errb
|
||||
if err := cmd.Run(); err != nil {
|
||||
msg := strings.TrimSpace(errb.String())
|
||||
if msg == "" {
|
||||
msg = err.Error()
|
||||
}
|
||||
return errors.New(msg)
|
||||
}
|
||||
return nil
|
||||
}
|
||||
@@ -0,0 +1,135 @@
|
||||
package storage
|
||||
|
||||
import (
|
||||
"strings"
|
||||
"testing"
|
||||
"time"
|
||||
|
||||
"hamlab/internal/cron"
|
||||
)
|
||||
|
||||
func TestSnapTimeReadsZfSnapNames(t *testing.T) {
|
||||
// the real thing off grid
|
||||
taken, exp, ok := SnapTime("tank@2026-09-24_02.30.01--6w")
|
||||
if !ok {
|
||||
t.Fatal("zfSnap's own name must parse")
|
||||
}
|
||||
if taken.Format("2006-01-02 15:04:05") != "2026-09-24 02:30:01" {
|
||||
t.Fatalf("taken = %v", taken)
|
||||
}
|
||||
if got := exp.Sub(taken); got != 6*7*24*time.Hour {
|
||||
t.Fatalf("ttl = %v, want 6w", got)
|
||||
}
|
||||
// a labelled one from a policy with a prefix
|
||||
if _, e, ok := SnapTime("tank/apps@nightly-2026-09-24_02.30.01--30d"); !ok || e.IsZero() {
|
||||
t.Fatal("a prefixed name must parse and expire")
|
||||
}
|
||||
// no TTL: never expires
|
||||
if _, e, ok := SnapTime("tank@2026-09-24_02.30.01"); !ok || !e.IsZero() {
|
||||
t.Fatal("a name without a TTL must parse and never expire")
|
||||
}
|
||||
// things that are not ours
|
||||
for _, n := range []string{"tank@before-upgrade", "tank@manual", "tank", "tank@2026-13-45_99.99.99--6w"} {
|
||||
if _, _, ok := SnapTime(n); ok {
|
||||
t.Errorf("%q must not parse as a scheduled snapshot", n)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func TestCronExprAndNextRun(t *testing.T) {
|
||||
now := time.Date(2026, 9, 24, 9, 15, 0, 0, time.Local)
|
||||
for _, c := range []struct {
|
||||
p Policy
|
||||
want string
|
||||
}{
|
||||
{Policy{Every: "daily", At: "02:30"}, "30 2 * * *"},
|
||||
{Policy{Every: "hourly", At: "00:30"}, "30 * * * *"},
|
||||
{Policy{Every: "weekly", At: "04:00", Weekday: 0}, "0 4 * * 0"},
|
||||
{Policy{Every: "monthly", At: "05:15", Day: 12}, "15 5 12 * *"},
|
||||
{Policy{Every: "custom", Cron: "*/15 9-17 * * 1-5"}, "*/15 9-17 * * 1-5"},
|
||||
} {
|
||||
got, err := c.p.CronExpr()
|
||||
if err != nil || got != c.want {
|
||||
t.Errorf("CronExpr(%+v) = %q, %v; want %q", c.p.Every, got, err, c.want)
|
||||
}
|
||||
}
|
||||
// the next run is read with the same evaluator the Jobs tab uses
|
||||
daily := Policy{Every: "daily", At: "02:30"}
|
||||
if n := daily.NextRun(now); n.Hour() != 2 || n.Minute() != 30 || n.Day() != 25 {
|
||||
t.Fatalf("NextRun = %v, want 25th 02:30", n)
|
||||
}
|
||||
// a custom expression that does not parse is refused, not silently ignored
|
||||
if _, err := (&Policy{Every: "custom", Cron: "not a schedule"}).CronExpr(); err == nil {
|
||||
t.Fatal("a bad cron expression must be refused")
|
||||
}
|
||||
if _, err := (&Policy{Every: "custom", Cron: "@reboot"}).CronExpr(); err == nil {
|
||||
t.Fatal("@reboot is not a snapshot schedule")
|
||||
}
|
||||
}
|
||||
|
||||
func TestCronFileText(t *testing.T) {
|
||||
ps := []Policy{
|
||||
{ID: "abc", Name: "tank nightly", Dataset: "tank", Every: "daily", At: "02:30", KeepFor: "6w", Enabled: true},
|
||||
{ID: "def", Name: "paused one", Dataset: "tank/apps", Every: "hourly", At: "00:05", KeepFor: "2d"},
|
||||
}
|
||||
text, err := CronFileText(ps, "core", "/usr/local/bin/hamlab", "/home/core/apps/hamlab/config.json")
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if !strings.Contains(text, "30 2 * * * core /usr/local/bin/hamlab -config /home/core/apps/hamlab/config.json zfs-snapshot abc") {
|
||||
t.Fatalf("the enabled schedule is missing or malformed:\n%s", text)
|
||||
}
|
||||
if strings.Contains(text, "zfs-snapshot def") {
|
||||
t.Fatalf("a paused schedule must not be scheduled:\n%s", text)
|
||||
}
|
||||
// every line cron will read must parse as a schedule
|
||||
for _, line := range strings.Split(text, "\n") {
|
||||
line = strings.TrimSpace(line)
|
||||
if line == "" || strings.HasPrefix(line, "#") || strings.Contains(line, "=") {
|
||||
continue
|
||||
}
|
||||
f := strings.Fields(line)
|
||||
if err := cron.ValidSchedule(strings.Join(f[:5], " ")); err != nil {
|
||||
t.Errorf("line %q: %v", line, err)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func TestValidateRefusesNonsense(t *testing.T) {
|
||||
ok := Policy{Dataset: "tank/apps", Every: "daily", At: "02:30", KeepFor: "6w"}
|
||||
if err := ok.Validate(); err != nil {
|
||||
t.Fatalf("a sane policy: %v", err)
|
||||
}
|
||||
bad := []Policy{
|
||||
{Dataset: "", Every: "daily", At: "02:30", KeepFor: "6w"},
|
||||
{Dataset: "tank", Every: "yearly", At: "02:30", KeepFor: "6w"},
|
||||
{Dataset: "tank", Every: "custom", KeepFor: "6w"},
|
||||
{Dataset: "tank", Every: "custom", Cron: "not a schedule", KeepFor: "6w"},
|
||||
{Dataset: "tank", Every: "daily", At: "25:00", KeepFor: "6w"},
|
||||
{Dataset: "tank", Every: "daily", At: "02:30"}, // no retention
|
||||
{Dataset: "tank", Every: "daily", At: "02:30", KeepFor: "6 weeks"}, // unparseable
|
||||
{Dataset: "tank", Every: "monthly", At: "02:30", Day: 31, KeepFor: "6w"}, // not every month has one
|
||||
{Dataset: "tank", Every: "daily", At: "02:30", KeepFor: "6w", Prefix: "a--b"},
|
||||
}
|
||||
for i, p := range bad {
|
||||
if err := p.Validate(); err == nil {
|
||||
t.Errorf("bad policy %d was accepted: %+v", i, p)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func TestParseAge(t *testing.T) {
|
||||
for s, want := range map[string]time.Duration{
|
||||
"6w": 6 * 7 * 24 * time.Hour, "30d": 30 * 24 * time.Hour,
|
||||
"12h": 12 * time.Hour, "90m": 90 * time.Minute,
|
||||
} {
|
||||
if got, err := ParseAge(s); err != nil || got != want {
|
||||
t.Errorf("ParseAge(%q) = %v, %v; want %v", s, got, err, want)
|
||||
}
|
||||
}
|
||||
for _, s := range []string{"", "6", "w", "-1d", "0w", "6y", "six weeks"} {
|
||||
if _, err := ParseAge(s); err == nil {
|
||||
t.Errorf("ParseAge(%q) should fail", s)
|
||||
}
|
||||
}
|
||||
}
|
||||
+19
-5
@@ -41,19 +41,33 @@ func (s *Session) UnmarshalJSON(b []byte) error {
|
||||
// UserState is what follows an account rather than the machine: how the console looks
|
||||
// to that person and how their dashboard is laid out.
|
||||
type UserState struct {
|
||||
Dashboard json.RawMessage `json:"dashboard,omitempty"` // layout prefs, opaque to the server
|
||||
UI json.RawMessage `json:"ui,omitempty"` // theme, accents; opaque to the server
|
||||
UI json.RawMessage `json:"ui,omitempty"` // theme, accents; opaque to the server
|
||||
}
|
||||
|
||||
// RememberedMount is one bind mount of a container that was running when Ham Lab last
|
||||
// looked. A compose stack records the path its file had *inside* whatever deployed it, so
|
||||
// when that container is gone the path is the only way back to the file on disk.
|
||||
type RememberedMount struct {
|
||||
Container string `json:"container"`
|
||||
Dest string `json:"dest"` // the path inside the container
|
||||
Src string `json:"src"` // the path on the machine
|
||||
Seen time.Time `json:"seen"`
|
||||
}
|
||||
|
||||
type State struct {
|
||||
Terminals []Terminal `json:"terminals"`
|
||||
Sessions map[string]Session `json:"sessions"` // token -> session
|
||||
Users map[string]*UserState `json:"users,omitempty"`
|
||||
// Dashboard and UI are the pre-users machine-wide prefs; on first boot with users they
|
||||
// move to the first account. Kept so an old state file still reads.
|
||||
Dashboard json.RawMessage `json:"dashboard,omitempty"`
|
||||
// UI is the pre-users machine-wide look; on first boot with users it moves to the first
|
||||
// account. Kept so an old state file still reads.
|
||||
UI json.RawMessage `json:"ui,omitempty"`
|
||||
Alerts json.RawMessage `json:"alerts,omitempty"` // rules, channels and recent events (internal/api/alerts.go)
|
||||
ZFS json.RawMessage `json:"zfs,omitempty"` // snapshot schedules (internal/api/zfs.go)
|
||||
DockerJobs json.RawMessage `json:"docker_jobs,omitempty"` // image-update schedules (internal/api/dockersched.go)
|
||||
// Mounts remembers where a container's bind mounts pointed, so a stack deployed by an
|
||||
// orchestrator that has since been removed can still be found on disk. See
|
||||
// internal/api/stacks.go.
|
||||
Mounts []RememberedMount `json:"mounts,omitempty"`
|
||||
SamplerMs int `json:"sampler_ms,omitempty"` // how often this machine samples; the hub keeps every machine on the console's refresh rate
|
||||
HistorySec int `json:"history_sec,omitempty"` // how far back the graphs reach; the hub keeps every machine on the same one
|
||||
}
|
||||
|
||||
@@ -84,10 +84,21 @@ func newID() string {
|
||||
|
||||
// Create starts a new login shell. id may be empty.
|
||||
func (m *Manager) Create(id, name string) (*Session, error) {
|
||||
return m.CreateCmd(id, name, nil)
|
||||
}
|
||||
|
||||
// CreateCmd is Create with something other than a login shell: argv nil is the shell, and
|
||||
// anything else runs that instead — `docker exec -it …` for a container's console. Every
|
||||
// session is the same thing afterwards, so attach, resize, the scrollback ring and the tab
|
||||
// strip work without knowing which it is.
|
||||
func (m *Manager) CreateCmd(id, name string, argv []string) (*Session, error) {
|
||||
if id == "" {
|
||||
id = newID()
|
||||
}
|
||||
cmd := exec.Command(m.shell, "-l")
|
||||
if len(argv) > 0 {
|
||||
cmd = exec.Command(argv[0], argv[1:]...)
|
||||
}
|
||||
cmd.Dir = m.home
|
||||
cmd.Env = append(os.Environ(), "TERM=xterm-256color", "COLORTERM=truecolor", "HAMLAB=1")
|
||||
ptmx, err := pty.StartWithSize(cmd, &pty.Winsize{Cols: defaultCol, Rows: defaultRow})
|
||||
|
||||
@@ -1,474 +0,0 @@
|
||||
import { ReactNode, useCallback, useEffect, useRef, useState } from 'react'
|
||||
import { api, DashPrefs, Sample, Series, Temp } from './api'
|
||||
import Sparkline from './Sparkline'
|
||||
import { Loading, Spinner, askYes, toast } from './ui'
|
||||
import { fmtMs, fmtWindowSec, nextPoint, strideMs, useHistorySec, useRefreshMs } from './prefs'
|
||||
import { fetchCatalog, fetchHistories, useSensorStream } from './sensors'
|
||||
import { Catalog, MONITOR_HISTORY, MONITOR_SHAPE, Values, mergeValues, sampleOf, seriesOf, stale } from './sampleAdapter'
|
||||
import { Widget, WidgetData, WidgetSpec, shapeOf } from './widgets'
|
||||
import WidgetPicker from './WidgetPicker'
|
||||
|
||||
const MAX = 300
|
||||
const S1 = 'var(--s1)', S2 = 'var(--s2)'
|
||||
|
||||
// ---- module registry: the customizable blocks, in default order ----
|
||||
type Kind = 'tile' | 'card'
|
||||
interface Module { id: string; label: string; kind: Kind }
|
||||
const MODULES: Module[] = [
|
||||
{ id: 'cpu', label: 'CPU', kind: 'tile' },
|
||||
{ id: 'mem', label: 'Memory', kind: 'tile' },
|
||||
{ id: 'load', label: 'Load', kind: 'tile' },
|
||||
{ id: 'uptime', label: 'Uptime', kind: 'tile' },
|
||||
{ id: 'temps', label: 'Temperatures', kind: 'card' },
|
||||
{ id: 'fans', label: 'Fans', kind: 'card' },
|
||||
{ id: 'net', label: 'Network', kind: 'card' },
|
||||
{ id: 'disks', label: 'Disks', kind: 'card' },
|
||||
{ id: 'fs', label: 'Filesystems', kind: 'card' },
|
||||
{ id: 'ups', label: 'UPS', kind: 'card' },
|
||||
]
|
||||
const byId = (id: string) => MODULES.find((m) => m.id === id)
|
||||
|
||||
const DEFAULT_INTERVAL = 2000
|
||||
const clampSampler = (ms: number) => Math.min(10000, Math.max(100, ms))
|
||||
|
||||
interface Layout { order: string[]; hidden: Set<string>; hiddenItems: Set<string>; intervalMs: number; widgets: WidgetSpec[] }
|
||||
|
||||
// Unknown ids are dropped, missing ones appended, so an old saved layout survives new modules.
|
||||
function normalize(raw: DashPrefs): Layout {
|
||||
// Saved prefs are whatever some client last wrote; a wrong shape must not take the page down.
|
||||
const list = <T,>(v: unknown): T[] => (Array.isArray(v) ? (v as T[]) : [])
|
||||
const p = { order: list<string>(raw?.order), hidden: list<string>(raw?.hidden), hiddenItems: list<string>(raw?.hiddenItems), widgets: list<WidgetSpec>(raw?.widgets), intervalMs: typeof raw?.intervalMs === 'number' ? raw.intervalMs : undefined }
|
||||
const known = new Set(MODULES.map((m) => m.id))
|
||||
const order = (p.order ?? []).filter((id) => known.has(id))
|
||||
for (const m of MODULES) if (!order.includes(m.id)) order.push(m.id)
|
||||
const iv = p.intervalMs && p.intervalMs >= 100 && p.intervalMs <= 2000 ? p.intervalMs : DEFAULT_INTERVAL
|
||||
return { order, hidden: new Set((p.hidden ?? []).filter((id) => known.has(id))), hiddenItems: new Set(p.hiddenItems ?? []), intervalMs: iv,
|
||||
widgets: (p.widgets ?? []).filter((w) => w && typeof w.id === 'string' && typeof w.sensor === 'string') }
|
||||
}
|
||||
const toPrefs = (l: Layout): DashPrefs => ({ order: l.order, hidden: [...l.hidden], hiddenItems: [...l.hiddenItems], intervalMs: l.intervalMs, widgets: l.widgets })
|
||||
|
||||
// ---- formatting ----
|
||||
// A live value either starts a new point or is averaged into the one being built — the same
|
||||
// rule the machine itself uses to keep its history, so the tail of a six-hour graph isn't a
|
||||
// minute of 2-second samples crushed against the right edge. n is how many values this point
|
||||
// has had, including this one.
|
||||
function pushPoint(arr: number[], v: number, fresh: boolean, n: number) {
|
||||
if (!fresh && arr.length) { arr[arr.length - 1] += (v - arr[arr.length - 1]) / n; return }
|
||||
arr.push(v)
|
||||
if (arr.length > MAX) arr.shift()
|
||||
}
|
||||
function pad(arr: number[], n: number) { while (arr.length < n) arr.unshift(0) }
|
||||
|
||||
export function fmtBytes(b: number): string {
|
||||
const u = ['B', 'KiB', 'MiB', 'GiB', 'TiB']
|
||||
let i = 0
|
||||
while (b >= 1024 && i < u.length - 1) { b /= 1024; i++ }
|
||||
return `${b < 10 && i > 0 ? b.toFixed(1) : Math.round(b)} ${u[i]}`
|
||||
}
|
||||
export function fmtRate(bps: number): string {
|
||||
const u = ['B/s', 'kB/s', 'MB/s', 'GB/s']
|
||||
let i = 0
|
||||
while (bps >= 1000 && i < u.length - 1) { bps /= 1000; i++ }
|
||||
return `${bps < 10 && i > 0 ? bps.toFixed(1) : Math.round(bps)} ${u[i]}`
|
||||
}
|
||||
function fmtUptime(s: number): string {
|
||||
const d = Math.floor(s / 86400), h = Math.floor((s % 86400) / 3600), m = Math.floor((s % 3600) / 60)
|
||||
return d > 0 ? `${d}d ${h}h` : h > 0 ? `${h}h ${m}m` : `${m}m`
|
||||
}
|
||||
const pct = (v: number) => `${Math.round(v)}%`
|
||||
function level(frac: number, warn = 0.85, bad = 0.95): '' | 'warn' | 'bad' { return frac >= bad ? 'bad' : frac >= warn ? 'warn' : '' }
|
||||
|
||||
// Chips with many sensors (per-core temps on a big CPU) collapse to one row showing the hottest.
|
||||
function collapseTemps(temps: Temp[]): (Temp & { key: string })[] {
|
||||
const byChip = new Map<string, Temp[]>()
|
||||
for (const t of temps) (byChip.get(t.chip) ?? byChip.set(t.chip, []).get(t.chip)!).push(t)
|
||||
const out: (Temp & { key: string })[] = []
|
||||
for (const [chip, list] of byChip) {
|
||||
if (list.length <= 8) { out.push(...list.map((t) => ({ ...t, key: `${t.chip}/${t.label}` }))); continue }
|
||||
const hot = list.reduce((a, b) => (b.c > a.c ? b : a))
|
||||
out.push({ ...hot, label: `${list.length} sensors · hottest ${hot.label}`, chip, key: `${chip}/*` })
|
||||
}
|
||||
return out
|
||||
}
|
||||
|
||||
// ---- small building blocks ----
|
||||
function Meter({ frac, cls }: { frac: number; cls?: string }) {
|
||||
return <div className="meter"><div className={'fill ' + (cls ?? level(frac))} style={{ width: `${Math.min(100, Math.max(0, frac * 100))}%` }} /></div>
|
||||
}
|
||||
function Legend({ items }: { items: { name: string; color: string }[] }) {
|
||||
return <div className="legend">{items.map((i) => <span key={i.name}><span className="sw" style={{ background: i.color }} />{i.name}</span>)}</div>
|
||||
}
|
||||
|
||||
interface Edit {
|
||||
on: boolean
|
||||
hideItem: (key: string) => void
|
||||
unhideItem: (key: string) => void
|
||||
hiddenItems: Set<string>
|
||||
}
|
||||
// In customize mode every row gets a ✕; hidden rows come back as chips under the module.
|
||||
function HideBtn({ k, edit }: { k: string; edit: Edit }) {
|
||||
return edit.on ? <button className="hide-item" title="hide" onClick={() => edit.hideItem(k)}>✕</button> : null
|
||||
}
|
||||
function HiddenChips({ prefix, edit, label }: { prefix: string; edit: Edit; label: (k: string) => string }) {
|
||||
if (!edit.on) return null
|
||||
const keys = [...edit.hiddenItems].filter((k) => k.startsWith(prefix + ':'))
|
||||
if (keys.length === 0) return null
|
||||
return <div className="chips inline">{keys.map((k) => <button key={k} className="chip" title="show again" onClick={() => edit.unhideItem(k)}>+ {label(k.slice(prefix.length + 1))}</button>)}</div>
|
||||
}
|
||||
|
||||
// ---- module bodies ----
|
||||
interface Ctx { s: Sample; h: Series | null; edit: Edit }
|
||||
|
||||
function Tile({ label, value, sub, children }: { label: string; value: string; sub?: string; children?: ReactNode }) {
|
||||
return <><div className="label">{label}</div><div className="value">{value}</div>{sub && <div className="sub">{sub}</div>}{children}</>
|
||||
}
|
||||
|
||||
function body(id: string, { s, h, edit }: Ctx): ReactNode {
|
||||
const shown = (key: string) => !edit.hiddenItems.has(key)
|
||||
const memFrac = s.mem.total ? s.mem.used / s.mem.total : 0
|
||||
switch (id) {
|
||||
case 'cpu':
|
||||
return (
|
||||
<Tile label="CPU" value={pct(s.cpu)} sub={s.cores.length === 0 ? undefined : s.cores.length <= 16 ? `cores ${s.cores.map((c) => Math.round(c)).join(' · ')}` : `${s.cores.length} cores · busiest ${pct(Math.max(...s.cores))}`}>
|
||||
{h && <Sparkline series={[{ name: 'cpu', values: h.cpu, color: S1 }]} max={100} format={pct} height={36} />}
|
||||
</Tile>
|
||||
)
|
||||
case 'mem':
|
||||
return (
|
||||
<Tile label="Memory" value={fmtBytes(s.mem.used)} sub={`of ${fmtBytes(s.mem.total)} · ${pct(memFrac * 100)}${s.swap.total ? ` · swap ${fmtBytes(s.swap.used)}` : ''}`}>
|
||||
{h && <Sparkline series={[{ name: 'used', values: h.mem, color: S1 }]} max={s.mem.total} format={fmtBytes} height={36} />}
|
||||
</Tile>
|
||||
)
|
||||
case 'load':
|
||||
return <Tile label="Load" value={s.load.map((l) => l.toFixed(2)).join(' ')} sub={`1 · 5 · 15 min, ${s.cores.length || '?'} cores`} />
|
||||
case 'uptime':
|
||||
return <Tile label="Uptime" value={fmtUptime(s.uptime)} sub={new Date(s.t).toLocaleTimeString()} />
|
||||
|
||||
case 'temps': {
|
||||
const rows = collapseTemps(s.temps).filter((t) => shown(`temp:${t.key}`))
|
||||
return (
|
||||
<>
|
||||
<h2>Temperatures</h2>
|
||||
{s.temps.length === 0 && <div className="empty">No temperature sensors reported by this host.</div>}
|
||||
{rows.map((t) => {
|
||||
const limit = t.crit || t.max || 100
|
||||
const cls = t.c >= limit - 5 ? 'bad' : t.max && t.c >= t.max - 15 ? 'warn' : ''
|
||||
return (
|
||||
<div className="row" key={t.key}>
|
||||
<span className="k">{t.label}<small>{t.chip}</small></span>
|
||||
<Meter frac={t.c / limit} cls={cls} />
|
||||
<span className="num">{t.c.toFixed(0)}°C{t.max ? ` / ${t.max.toFixed(0)}` : ''}<HideBtn k={`temp:${t.key}`} edit={edit} /></span>
|
||||
</div>
|
||||
)
|
||||
})}
|
||||
<HiddenChips prefix="temp" edit={edit} label={(k) => k.replace('/*', '')} />
|
||||
</>
|
||||
)
|
||||
}
|
||||
case 'fans': {
|
||||
const rows = s.fans.filter((f) => shown(`fan:${f.chip}/${f.label}`))
|
||||
return (
|
||||
<>
|
||||
<h2>Fans</h2>
|
||||
{s.fans.length === 0 && <div className="empty">No fan sensors reported by this host.</div>}
|
||||
{rows.map((f) => (
|
||||
<div className="row" key={`${f.chip}/${f.label}`}>
|
||||
<span className="k">{f.label}<small>{f.chip}</small></span>
|
||||
<span />
|
||||
<span className="num">{f.rpm} rpm<HideBtn k={`fan:${f.chip}/${f.label}`} edit={edit} /></span>
|
||||
</div>
|
||||
))}
|
||||
<HiddenChips prefix="fan" edit={edit} label={(k) => k} />
|
||||
</>
|
||||
)
|
||||
}
|
||||
case 'net':
|
||||
return (
|
||||
<>
|
||||
<h2>Network</h2>
|
||||
{s.net.filter((x) => shown(`net:${x.iface}`)).map((x) => (
|
||||
<div className="sub-card" key={x.iface}>
|
||||
<div className="sub-head">
|
||||
<span className={'dot ' + (x.up ? 'ok' : '')} />
|
||||
<span>{x.iface}</span>
|
||||
{!x.up && <span className="muted small">down</span>}
|
||||
<span className="spacer" />
|
||||
{x.up && <span className="num">↓ {fmtRate(x.rx_bps)} ↑ {fmtRate(x.tx_bps)}</span>}
|
||||
<HideBtn k={`net:${x.iface}`} edit={edit} />
|
||||
</div>
|
||||
{x.up && h?.net[x.iface] && (
|
||||
<Sparkline series={[{ name: 'rx', values: h.net[x.iface].rx, color: S1 }, { name: 'tx', values: h.net[x.iface].tx, color: S2 }]} format={fmtRate} />
|
||||
)}
|
||||
{x.up && <div className="legend">
|
||||
<span><span className="sw" style={{ background: S1 }} />rx · {fmtBytes(x.rx_total)} total</span>
|
||||
<span><span className="sw" style={{ background: S2 }} />tx · {fmtBytes(x.tx_total)} total</span>
|
||||
</div>}
|
||||
</div>
|
||||
))}
|
||||
<HiddenChips prefix="net" edit={edit} label={(k) => k} />
|
||||
</>
|
||||
)
|
||||
case 'disks':
|
||||
return (
|
||||
<>
|
||||
<h2>Disks</h2>
|
||||
{s.disks.length === 0 && <div className="empty">No block devices found.</div>}
|
||||
{s.disks.filter((d) => shown(`disk:${d.name}`)).map((d) => (
|
||||
<div className="sub-card" key={d.name}>
|
||||
<div className="sub-head">
|
||||
<span>{d.name}</span>
|
||||
<span className="model">{d.model}</span>
|
||||
<span className="spacer" />
|
||||
<span className="num">{fmtBytes(d.size)} · {d.temp ? `${d.temp.toFixed(0)}°C · ` : ''}{pct(d.util_pct)} busy</span>
|
||||
<HideBtn k={`disk:${d.name}`} edit={edit} />
|
||||
</div>
|
||||
<div className="sub-head num">read {fmtRate(d.read_bps)}<span className="spacer" />write {fmtRate(d.write_bps)}</div>
|
||||
{h?.disk[d.name] && (
|
||||
<Sparkline series={[{ name: 'read', values: h.disk[d.name].r, color: S1 }, { name: 'write', values: h.disk[d.name].w, color: S2 }]} format={fmtRate} />
|
||||
)}
|
||||
<Legend items={[{ name: 'read', color: S1 }, { name: 'write', color: S2 }]} />
|
||||
</div>
|
||||
))}
|
||||
<HiddenChips prefix="disk" edit={edit} label={(k) => k} />
|
||||
</>
|
||||
)
|
||||
case 'fs':
|
||||
return (
|
||||
<>
|
||||
<h2>Filesystems</h2>
|
||||
{s.fs.filter((f) => shown(`fs:${f.mount}`)).map((f) => (
|
||||
<div className="row" key={f.mount}>
|
||||
<span className="k">{f.mount}<small>{f.type}</small></span>
|
||||
{f.size ? <Meter frac={f.used / f.size} /> : <span className="muted">unreachable</span>}
|
||||
<span className="num">{f.size ? `${fmtBytes(f.used)} / ${fmtBytes(f.size)}` : ''}<HideBtn k={`fs:${f.mount}`} edit={edit} /></span>
|
||||
</div>
|
||||
))}
|
||||
<HiddenChips prefix="fs" edit={edit} label={(k) => k} />
|
||||
</>
|
||||
)
|
||||
case 'ups':
|
||||
return (
|
||||
<>
|
||||
<h2>UPS</h2>
|
||||
{!s.ups && <div className="empty">No UPS daemon found (looks for pwrstat, NUT upsc, or apcaccess).</div>}
|
||||
{s.ups && (
|
||||
<>
|
||||
<div className="sub-head">
|
||||
<span className={'dot ' + (s.ups.on_battery ? 'bad' : 'ok')} />
|
||||
<span>{s.ups.state} · {s.ups.source}</span>
|
||||
<span className="spacer" />
|
||||
<span className="model">{s.ups.model} · {s.ups.driver}</span>
|
||||
</div>
|
||||
<div className="row"><span className="k">Battery</span><Meter frac={s.ups.battery_pct / 100} cls={s.ups.battery_pct < 20 ? 'bad' : s.ups.battery_pct < 50 ? 'warn' : ''} /><span className="num">{pct(s.ups.battery_pct)} · {s.ups.runtime_min} min</span></div>
|
||||
<div className="row"><span className="k">Load</span><Meter frac={s.ups.load_pct / 100} /><span className="num">{s.ups.load_w} W · {pct(s.ups.load_pct)}</span></div>
|
||||
<div className="row"><span className="k">Voltage</span><span /><span className="num">in {s.ups.in_v} V · out {s.ups.out_v} V</span></div>
|
||||
{s.ups.last_event && <div className="row"><span className="k">Last event</span><span className="num" style={{ gridColumn: '2 / 4', whiteSpace: 'normal' }}>{s.ups.last_event}</span></div>}
|
||||
</>
|
||||
)}
|
||||
</>
|
||||
)
|
||||
}
|
||||
return null
|
||||
}
|
||||
|
||||
// ---- the dashboard ----
|
||||
export default function Dashboard() {
|
||||
const [latest, setLatest] = useState<Sample | null>(null)
|
||||
const [err, setErr] = useState('')
|
||||
const [layout, setLayout] = useState<Layout>(() => normalize({}))
|
||||
const [editing, setEditing] = useState(false)
|
||||
const [loaded, setLoaded] = useState(false)
|
||||
const [saving, setSaving] = useState<'' | 'saving' | 'saved'>('')
|
||||
const savedTimer = useRef<number | undefined>(undefined)
|
||||
const hist = useRef<Series | null>(null)
|
||||
const saveTimer = useRef<number | undefined>(undefined)
|
||||
const refreshMs = useRefreshMs()
|
||||
const historySec = useHistorySec()
|
||||
|
||||
// Persist layout changes server-side (debounced) so every device sees the same dashboard.
|
||||
const update = useCallback((fn: (l: Layout) => Layout) => {
|
||||
setLayout((prev) => {
|
||||
const next = fn(prev)
|
||||
clearTimeout(saveTimer.current)
|
||||
setSaving('saving')
|
||||
saveTimer.current = window.setTimeout(async () => {
|
||||
try {
|
||||
await api.saveDashPrefs(toPrefs(next))
|
||||
setSaving('saved')
|
||||
clearTimeout(savedTimer.current)
|
||||
savedTimer.current = window.setTimeout(() => setSaving(''), 1500)
|
||||
} catch (e) { setSaving(''); toast(`layout not saved — ${e instanceof Error ? e.message : 'failed'}`, 'err') }
|
||||
}, 300)
|
||||
return next
|
||||
})
|
||||
}, [])
|
||||
|
||||
useEffect(() => {
|
||||
let stop = false
|
||||
api.dashPrefs().then((p) => { if (!stop) { setLayout(normalize(p)); setLoaded(true) } }).catch(() => setErr('could not load layout'))
|
||||
return () => { stop = true }
|
||||
}, [])
|
||||
|
||||
// Everything on this page comes from the sensor API: the catalog for names, labels and
|
||||
// thresholds; one batch of histories to seed the curves; one stream for the values.
|
||||
const catalog = useRef<Catalog>(new Map())
|
||||
const refetching = useRef(false)
|
||||
const binN = useRef(1) // values averaged into the point currently being built
|
||||
const lastPush = useRef(0) // when that point was started, by this browser's clock
|
||||
const stock = useSensorStream<unknown[]>(MONITOR_SHAPE, { full: false, everyMs: refreshMs })
|
||||
const seed = useCallback(async () => {
|
||||
try {
|
||||
catalog.current = new Map((await fetchCatalog()).map((x) => [x.id, x]))
|
||||
hist.current = seriesOf(await fetchHistories(MONITOR_HISTORY), catalog.current)
|
||||
lastPush.current = Date.now(); binN.current = 1
|
||||
setErr('')
|
||||
} catch (e) { setErr(e instanceof Error ? e.message : 'failed') }
|
||||
}, [])
|
||||
useEffect(() => { seed() }, [seed, stock.resumed]) // on load, and again after the page was hidden: the curves have a gap to refill
|
||||
useEffect(() => {
|
||||
if (!stock.data || catalog.current.size === 0) return
|
||||
const v: Values = mergeValues(stock.data)
|
||||
if (stale(v, catalog.current) && !refetching.current) { // something appeared (a disk, an interface): learn its labels
|
||||
refetching.current = true
|
||||
fetchCatalog().then((c) => { catalog.current = new Map(c.map((x) => [x.id, x])) }).catch(() => {}).finally(() => window.setTimeout(() => { refetching.current = false }, 10000))
|
||||
}
|
||||
const s = sampleOf(v, catalog.current)
|
||||
const h = hist.current
|
||||
if (h) {
|
||||
// The stride is timed on this clock alone (lastPush), never against the history's own
|
||||
// timestamps: those are the machine's, and a viewer whose clock runs minutes slow would
|
||||
// otherwise wait those minutes before its curves gained a point.
|
||||
const np = nextPoint(s.t, h.t.length === 0 ? 0 : lastPush.current, strideMs(historySec, refreshMs), refreshMs)
|
||||
const fresh = np.fresh
|
||||
lastPush.current = np.start
|
||||
binN.current = fresh ? 1 : binN.current + 1
|
||||
if (fresh) pushPoint(h.t, s.t, true, 1)
|
||||
const n = h.t.length
|
||||
// a curve that has no point in this one yet starts one, whatever the others are doing:
|
||||
// an interface or disk that appeared mid-point must still line up with the time axis
|
||||
const push = (arr: number[], x: number) => pushPoint(arr, x, fresh || arr.length < n, binN.current)
|
||||
push(h.cpu, s.cpu); push(h.mem, s.mem.used)
|
||||
for (const x of s.net) {
|
||||
const c = (h.net[x.iface] ??= { rx: [], tx: [] })
|
||||
pad(c.rx, n - 1); pad(c.tx, n - 1); push(c.rx, x.rx_bps); push(c.tx, x.tx_bps)
|
||||
}
|
||||
for (const d of s.disks) {
|
||||
const c = (h.disk[d.name] ??= { r: [], w: [], u: [] })
|
||||
pad(c.r, n - 1); pad(c.w, n - 1); pad(c.u, n - 1); push(c.r, d.read_bps); push(c.w, d.write_bps); push(c.u, d.util_pct)
|
||||
}
|
||||
for (const t of s.temps) {
|
||||
const c = (h.temp[`${t.chip}/${t.label}`] ??= [])
|
||||
pad(c, n - 1); push(c, t.c)
|
||||
}
|
||||
}
|
||||
setLatest(s)
|
||||
}, [stock.data, historySec, refreshMs])
|
||||
useEffect(() => { if (stock.error && !latest) setErr(stock.error) }, [stock.error, latest])
|
||||
|
||||
// The sampler on this machine follows the global rate (clamped to what it supports).
|
||||
useEffect(() => {
|
||||
if (!loaded) return
|
||||
const want = clampSampler(refreshMs)
|
||||
if (layout.intervalMs !== want) update((l) => ({ ...l, intervalMs: want }))
|
||||
}, [refreshMs, loaded]) // eslint-disable-line react-hooks/exhaustive-deps
|
||||
|
||||
// Custom widgets are fed by one shaped query over the sensor stream — the same API any outside dashboard uses.
|
||||
const [picking, setPicking] = useState(false)
|
||||
const [editingWidget, setEditingWidget] = useState<WidgetSpec | null>(null)
|
||||
const live = useSensorStream<Record<string, WidgetData>>(shapeOf(layout.widgets))
|
||||
|
||||
if (!latest) return <div className="dash">{err ? <div className="error">{err}</div> : <Loading label="Collecting metrics…" />}</div>
|
||||
|
||||
const edit: Edit = {
|
||||
on: editing,
|
||||
hiddenItems: layout.hiddenItems,
|
||||
hideItem: (k) => update((l) => ({ ...l, hiddenItems: new Set([...l.hiddenItems, k]) })),
|
||||
unhideItem: (k) => update((l) => { const s = new Set(l.hiddenItems); s.delete(k); return { ...l, hiddenItems: s } }),
|
||||
}
|
||||
const ctx: Ctx = { s: latest, h: hist.current, edit }
|
||||
// Modules with nothing to show on this host (no fans, no UPS…) stay out of the way unless customizing.
|
||||
const empty = (id: string) => (id === 'fans' && latest.fans.length === 0) || (id === 'ups' && !latest.ups) || (id === 'temps' && latest.temps.length === 0) || (id === 'disks' && latest.disks.length === 0)
|
||||
const visible = (kind: Kind) => layout.order.filter((id) => byId(id)?.kind === kind && !layout.hidden.has(id) && (editing || !empty(id)))
|
||||
const hiddenMods = layout.order.filter((id) => layout.hidden.has(id))
|
||||
|
||||
// Move a module past its nearest same-kind neighbour (tiles stay with tiles, cards with cards).
|
||||
const move = (id: string, dir: -1 | 1) => update((l) => {
|
||||
const kind = byId(id)!.kind
|
||||
const same = l.order.filter((x) => byId(x)?.kind === kind && !l.hidden.has(x))
|
||||
const i = same.indexOf(id), j = i + dir
|
||||
if (j < 0 || j >= same.length) return l
|
||||
const order = [...l.order]
|
||||
const a = order.indexOf(id), b = order.indexOf(same[j])
|
||||
;[order[a], order[b]] = [order[b], order[a]]
|
||||
return { ...l, order }
|
||||
})
|
||||
const hideMod = (id: string) => update((l) => ({ ...l, hidden: new Set([...l.hidden, id]) }))
|
||||
const showMod = (id: string) => update((l) => { const s = new Set(l.hidden); s.delete(id); return { ...l, hidden: s } })
|
||||
const reset = async () => { if (await askYes(layout.widgets.length ? `Reset the layout and remove ${layout.widgets.length} custom widget${layout.widgets.length > 1 ? 's' : ''}?` : 'Reset the layout to the default?')) update(() => normalize({})) }
|
||||
|
||||
const tools = (id: string, kind: Kind) => {
|
||||
if (!editing) return null
|
||||
const same = visible(kind), i = same.indexOf(id)
|
||||
return (
|
||||
<div className="mod-tools">
|
||||
<button title="move up" disabled={i === 0} onClick={() => move(id, -1)}>▲</button>
|
||||
<button title="move down" disabled={i === same.length - 1} onClick={() => move(id, 1)}>▼</button>
|
||||
<button title="hide module" onClick={() => hideMod(id)}>hide</button>
|
||||
</div>
|
||||
)
|
||||
}
|
||||
|
||||
return (
|
||||
<div className={'dash' + (editing ? ' editing' : '')}>
|
||||
<div className="dash-bar">
|
||||
{err && <span className="error">{err}</span>}
|
||||
{editing ? <span className="hint">last {fmtWindowSec(historySec)} · ▲▼ reorder · ✕ hide · refresh rate is in Settings</span>
|
||||
: <span className="hint">last {fmtWindowSec(historySec)} · {fmtMs(latest.interval_ms || layout.intervalMs)}</span>}
|
||||
<span className="spacer" />
|
||||
{saving === 'saving' && <span className="saved"><Spinner /> saving</span>}
|
||||
{saving === 'saved' && <span className="saved">saved ✓</span>}
|
||||
{editing && <button className="btn" onClick={reset}>Reset</button>}
|
||||
<button className={'btn' + (editing ? ' on' : '')} disabled={!loaded} title={loaded ? '' : 'layout not loaded'} onClick={() => setEditing(!editing)}>{editing ? 'Done' : 'Customize'}</button>
|
||||
</div>
|
||||
|
||||
{visible('tile').length > 0 && (
|
||||
<div className="kpis">
|
||||
{visible('tile').map((id) => <div className="tile" key={id}>{tools(id, 'tile')}{body(id, ctx)}</div>)}
|
||||
</div>
|
||||
)}
|
||||
{(layout.widgets.length > 0 || editing) && (
|
||||
<div className="kpis widgets">
|
||||
{layout.widgets.map((w, i) => (
|
||||
<div className={'tile w-' + w.type} key={w.id}>
|
||||
{editing && (
|
||||
<div className="mod-tools">
|
||||
<button title="move left" disabled={i === 0} onClick={() => update((l) => { const a = [...l.widgets]; [a[i - 1], a[i]] = [a[i], a[i - 1]]; return { ...l, widgets: a } })}>◀</button>
|
||||
<button title="move right" disabled={i === layout.widgets.length - 1} onClick={() => update((l) => { const a = [...l.widgets]; [a[i + 1], a[i]] = [a[i], a[i + 1]]; return { ...l, widgets: a } })}>▶</button>
|
||||
<button title="change its sensor, type or title" onClick={() => setEditingWidget(w)}>edit</button>
|
||||
<button title="remove widget" onClick={() => update((l) => ({ ...l, widgets: l.widgets.filter((x) => x.id !== w.id) }))}>remove</button>
|
||||
</div>
|
||||
)}
|
||||
<Widget spec={w} data={live.data ? live.data[w.id] : undefined} />
|
||||
</div>
|
||||
))}
|
||||
{editing && <button className="tile add-widget" onClick={() => setPicking(true)}>+ Add widget</button>}
|
||||
{live.error && !live.data && layout.widgets.length > 0 && <div className="notice attn" style={{ gridColumn: '1 / -1' }}>The widgets' stream isn't answering: {live.error}.</div>}
|
||||
</div>
|
||||
)}
|
||||
{editingWidget && <WidgetPicker initial={editingWidget} onClose={() => setEditingWidget(null)} onAdd={(w) => { update((l) => ({ ...l, widgets: l.widgets.map((x) => (x.id === w.id ? w : x)) })); setEditingWidget(null) }} />}
|
||||
{picking && <WidgetPicker onClose={() => setPicking(false)} onAdd={(w) => { update((l) => ({ ...l, widgets: [...l.widgets, w] })); setPicking(false) }} />}
|
||||
{visible('card').length > 0 && (
|
||||
<div className="cards">
|
||||
{visible('card').map((id) => <section className="card" key={id}>{tools(id, 'card')}{body(id, ctx)}</section>)}
|
||||
</div>
|
||||
)}
|
||||
|
||||
{editing && hiddenMods.length > 0 && (
|
||||
<div className="card">
|
||||
<h2>Hidden modules</h2>
|
||||
<div className="chips">{hiddenMods.map((id) => <button key={id} className="chip" onClick={() => showMod(id)}>+ {byId(id)!.label}</button>)}</div>
|
||||
</div>
|
||||
)}
|
||||
</div>
|
||||
)
|
||||
}
|
||||
+4
-93
@@ -1,8 +1,8 @@
|
||||
import { useCallback, useEffect, useMemo, useState } from 'react'
|
||||
import { api, SmartInfo, StorageDevice, ZFSInfo, ZFSSnapshot } from './api'
|
||||
import { fmtBytes, fmtRate } from './Dashboard'
|
||||
import { api, SmartInfo, StorageDevice } from './api'
|
||||
import { fmtBytes, fmtRate } from './fmt'
|
||||
import TaskOutput from './TaskOutput'
|
||||
import { Btn, Loading, Spinner, askText, toast } from './ui'
|
||||
import { askText, Btn, Loading, Meter, Spinner, toast } from './ui'
|
||||
import { useRefreshMs } from './prefs'
|
||||
import { Sensor, useSensorStream } from './sensors'
|
||||
|
||||
@@ -11,10 +11,6 @@ const KIND: Record<string, string> = { nvme: 'NVMe', ssd: 'SSD', hdd: 'HDD', usb
|
||||
const hours = (h: number) => (h >= 24 * 365 ? `${(h / 8760).toFixed(1)} y` : h >= 48 ? `${Math.round(h / 24)} d` : `${h} h`)
|
||||
const ago = (ms: number) => { const s = (Date.now() - ms) / 1000; return s < 3600 ? `${Math.round(s / 60)}m ago` : s < 86400 ? `${Math.round(s / 3600)}h ago` : `${Math.round(s / 86400)}d ago` }
|
||||
|
||||
function Meter({ frac, cls }: { frac: number; cls?: string }) {
|
||||
const c = cls ?? (frac >= 0.95 ? 'bad' : frac >= 0.85 ? 'warn' : '')
|
||||
return <div className="meter"><div className={'fill ' + c} style={{ width: `${Math.min(100, Math.max(0, frac * 100))}%` }} /></div>
|
||||
}
|
||||
|
||||
// ---- SMART panel ----
|
||||
function SmartPanel({ dev, rotational }: { dev: string; rotational: boolean }) {
|
||||
@@ -109,82 +105,9 @@ function DeviceCard({ d, live, forceOpen }: { d: StorageDevice; live?: { read_bp
|
||||
)
|
||||
}
|
||||
|
||||
// ---- ZFS ----
|
||||
function PoolCard({ p, onChanged }: { p: ZFSInfo['pools'][number]; onChanged: () => void }) {
|
||||
const scrubbing = /scrub in progress/i.test(p.scan ?? '')
|
||||
const paused = /scrub paused/i.test(p.scan ?? '')
|
||||
const [cfg, setCfg] = useState(false)
|
||||
const act = (a: 'start' | 'pause' | 'stop') => async () => {
|
||||
try { await api.zfsScrub(p.name, a); toast(`scrub ${a}: ok`); onChanged() } catch (e) { toast(e instanceof Error ? e.message : 'failed', 'err') }
|
||||
}
|
||||
return (
|
||||
<section className="card">
|
||||
<div className="sub-head">
|
||||
<span className={'dot ' + (p.health === 'ONLINE' ? 'ok' : p.health === 'DEGRADED' ? 'warn' : 'bad')} />
|
||||
<span className="dname">{p.name}</span>
|
||||
<span className="badge">{p.health}</span>
|
||||
<span className="spacer" />
|
||||
<span className="num">{fmtBytes(p.alloc)} / {fmtBytes(p.size)} · {p.cap}%</span>
|
||||
</div>
|
||||
<Meter frac={p.cap / 100} cls={p.cap >= 90 ? 'bad' : p.cap >= 80 ? 'warn' : ''} />
|
||||
<div className="props" style={{ marginTop: 8 }}>
|
||||
<div><span className="k">Free</span><span>{fmtBytes(p.free)} · fragmentation {p.frag}% · dedup {p.dedup}</span></div>
|
||||
{p.scan && <div><span className="k">Scan</span><span style={{ whiteSpace: 'normal' }}>{p.scan}</span></div>}
|
||||
{p.errors && <div><span className="k">Errors</span><span className={p.errors.startsWith('No known') ? '' : 'warn-text'}>{p.errors}</span></div>}
|
||||
</div>
|
||||
<div className="svc-actions">
|
||||
{!scrubbing && <Btn className="btn" confirm={`Start a scrub of ${p.name}? It reads the whole pool and can take many hours.`} onClick={act('start')}>{paused ? 'Resume scrub' : 'Scrub'}</Btn>}
|
||||
{scrubbing && <Btn className="btn" onClick={act('pause')}>Pause scrub</Btn>}
|
||||
{(scrubbing || paused) && <Btn className="btn" confirm="Stop the scrub?" onClick={act('stop')}>Stop scrub</Btn>}
|
||||
<button className="btn" onClick={() => setCfg(!cfg)}>{cfg ? 'Hide vdevs' : 'Vdevs'}</button>
|
||||
</div>
|
||||
{cfg && <pre className="unit-file">{p.config}</pre>}
|
||||
</section>
|
||||
)
|
||||
}
|
||||
|
||||
function DatasetRow({ d, onChanged }: { d: ZFSInfo['datasets'][number]; onChanged: () => void }) {
|
||||
const [snaps, setSnaps] = useState<ZFSSnapshot[] | null>(null)
|
||||
const [open, setOpen] = useState(false)
|
||||
async function toggle() {
|
||||
setOpen(!open)
|
||||
if (!open && !snaps) { try { setSnaps(await api.zfsSnapshots(d.name)) } catch (e) { toast(e instanceof Error ? e.message : 'failed', 'err'); setSnaps([]) } }
|
||||
}
|
||||
async function snapshot() {
|
||||
const def = 'hamlab-' + new Date().toISOString().slice(0, 16).replace(/[-:T]/g, '').replace(/(\d{8})(\d{4})/, '$1-$2')
|
||||
const name = await askText(`Snapshot name for ${d.name}@`, def)
|
||||
if (!name) return
|
||||
try { await api.zfsSnapshot(d.name, name.trim()); toast(`${d.name}@${name.trim()} created`); setSnaps(null); onChanged() }
|
||||
catch (e) { toast(e instanceof Error ? e.message : 'failed', 'err') }
|
||||
}
|
||||
const depth = d.name.split('/').length - 1
|
||||
return (
|
||||
<>
|
||||
<div className="ds-row" style={{ paddingLeft: 12 + depth * 16 }}>
|
||||
<span className="ds-name" onClick={toggle}><span className="chev">{open ? '▾' : '▸'}</span>{d.name.split('/').pop()}{d.type === 'volume' && <span className="badge">zvol</span>}</span>
|
||||
<span className="num">{fmtBytes(d.used)}</span>
|
||||
<span className="num muted">{fmtBytes(d.refer)}</span>
|
||||
<span className="num muted">{d.used_by_snapshots ? fmtBytes(d.used_by_snapshots) : '—'}</span>
|
||||
<span className="num muted">{d.snapshots}</span>
|
||||
<span className="num muted">{d.compressratio}</span>
|
||||
<span className="ds-mount muted">{d.mount}</span>
|
||||
</div>
|
||||
{open && (
|
||||
<div className="ds-snaps">
|
||||
<div className="svc-actions"><Btn className="btn" onClick={snapshot}>Snapshot now</Btn><span className="muted">avail {fmtBytes(d.avail)}</span></div>
|
||||
{snaps === null ? <div className="muted"><Spinner /> listing…</div> : snaps.length === 0 ? <div className="muted">No snapshots.</div> : (
|
||||
<div className="snap-list">{snaps.slice().reverse().map((s) => <div key={s.name} className="snap"><span className="mono">{s.name.split('@')[1]}</span><span className="num muted">{new Date(s.created).toLocaleString()} · {fmtBytes(s.used)}</span></div>)}</div>
|
||||
)}
|
||||
</div>
|
||||
)}
|
||||
</>
|
||||
)
|
||||
}
|
||||
|
||||
// ---- page ----
|
||||
export default function Disks() {
|
||||
const [devs, setDevs] = useState<StorageDevice[] | null>(null)
|
||||
const [zfs, setZfs] = useState<ZFSInfo | null>(null)
|
||||
const [hasSmart, setHasSmart] = useState(true)
|
||||
const [err, setErr] = useState('')
|
||||
const [task, setTask] = useState<string | null>(null)
|
||||
@@ -194,7 +117,7 @@ export default function Disks() {
|
||||
const load = useCallback(async () => {
|
||||
try {
|
||||
const r = await api.storage()
|
||||
setDevs(r.devices); setZfs(r.zfs); setHasSmart(r.smartctl); setErr('')
|
||||
setDevs(r.devices); setHasSmart(r.smartctl); setErr('')
|
||||
} catch (e) { setErr(e instanceof Error ? e.message : 'failed') }
|
||||
}, [])
|
||||
useEffect(() => { load() }, [load])
|
||||
@@ -237,18 +160,6 @@ export default function Disks() {
|
||||
{devs.map((d) => <DeviceCard key={d.name} d={d} live={liveFor(d)} forceOpen={allSmart ? true : null} />)}
|
||||
</div>
|
||||
|
||||
<div className="dash-bar"><h2 className="section">ZFS</h2></div>
|
||||
{zfs && !zfs.available && <div className="notice">{zfs.message || 'No ZFS pools on this host.'}</div>}
|
||||
{zfs?.available && zfs.pools.length === 0 && <div className="notice">ZFS is present but no pools are imported.</div>}
|
||||
{zfs?.available && zfs.pools.length > 0 && (
|
||||
<>
|
||||
<div className="cards">{zfs.pools.map((p) => <PoolCard key={p.name} p={p} onChanged={load} />)}</div>
|
||||
<section className="card datasets">
|
||||
<div className="ds-row head"><span>Dataset</span><span className="num">Used</span><span className="num">Refer</span><span className="num">In snaps</span><span className="num">Snaps</span><span className="num">Ratio</span><span>Mount</span></div>
|
||||
{zfs.datasets.map((d) => <DatasetRow key={d.name} d={d} onChanged={load} />)}
|
||||
</section>
|
||||
</>
|
||||
)}
|
||||
</div>
|
||||
)
|
||||
}
|
||||
|
||||
+385
-18
@@ -1,16 +1,18 @@
|
||||
// Docker: containers grouped by the stack that deployed them, with start / stop / restart,
|
||||
// logs that follow, and inspect — then images, volumes and networks. Live CPU and memory
|
||||
// come from the same container sensors everything else uses.
|
||||
import { useCallback, useEffect, useMemo, useRef, useState } from 'react'
|
||||
import { api, DockerContainer, DockerInfo, ImageCheck, Stack, StackDiff } from './api'
|
||||
import { FormEvent, useCallback, useEffect, useMemo, useRef, useState } from 'react'
|
||||
import { api, apiPath, DockerContainer, DockerEvent, DockerHealth, DockerInfo, ImageCheck, MachineStatus, MachineUpdates, Stack, StackDiff, UpdateJob, UpdateJobRow } from './api'
|
||||
import TaskOutput from './TaskOutput'
|
||||
import StackNew, { DupFrom } from './StackNew'
|
||||
import { Sensor, fmtValue, useSensorStream } from './sensors'
|
||||
import { Sensor, fetchHistories, fmtValue, useSensorStream } from './sensors'
|
||||
import { useHashSel } from './hashsel'
|
||||
import { useRefreshMs } from './prefs'
|
||||
import { askText, askYes, Btn, CloseBtn, Loading, setUnsaved, toast } from './ui'
|
||||
import { fmtWindowSec, useHistorySec, useRefreshMs } from './prefs'
|
||||
import Sparkline from './Sparkline'
|
||||
import { fmtBytes } from './fmt'
|
||||
import { askText, askYes, Btn, CloseBtn, Loading, setUnsaved, toast, useEscape } from './ui'
|
||||
|
||||
type Tab = 'containers' | 'stacks' | 'images' | 'volumes' | 'networks'
|
||||
type Tab = 'containers' | 'stacks' | 'updates' | 'images' | 'volumes' | 'networks'
|
||||
const dot = (c: DockerContainer) => (c.state !== 'running' ? (c.state === 'exited' || c.state === 'dead' ? 'bad' : 'warn') : c.health === 'unhealthy' ? 'bad' : c.health === 'starting' ? 'warn' : 'ok')
|
||||
const ports = (p: string) => [...new Set(p.split(', ').map((x) => x.replace(/^\[?::\]?:/, '').replace(/^0\.0\.0\.0:/, '').replace(/^[\d.]+:/, '')).filter((x) => x.includes('->')))].join(' ')
|
||||
|
||||
@@ -50,18 +52,26 @@ function LogView({ id }: { id: string }) {
|
||||
)
|
||||
}
|
||||
|
||||
// What a container is doing, and what can be done to it: its numbers over time, its health
|
||||
// checks, what has happened to it lately, a shell inside it, and its limits.
|
||||
function Detail({ c, live, onBack, onChanged }: { c: DockerContainer; live: Record<string, Sensor>; onBack: () => void; onChanged: () => void }) {
|
||||
const [tab, setTab] = useState<'logs' | 'inspect'>('logs')
|
||||
const [tab, setTab] = useState<'logs' | 'stats' | 'health' | 'events' | 'inspect'>('logs')
|
||||
const [info, setInfo] = useState<Record<string, unknown> | null>(null)
|
||||
useEffect(() => { setInfo(null); if (tab === 'inspect') api.dockerInspect(c.name).then(setInfo).catch((e) => toast(e instanceof Error ? e.message : 'failed', 'err')) }, [tab, c.name])
|
||||
const [limits, setLimits] = useState(false)
|
||||
useEffect(() => { setInfo(null); if (tab === 'inspect' || limits) api.dockerInspect(c.name).then(setInfo).catch((e) => toast(e instanceof Error ? e.message : 'failed', 'err')) }, [tab, limits, c.name])
|
||||
const act = (a: 'start' | 'stop' | 'restart' | 'remove') => async () => { try { await api.dockerAction(c.name, a); toast(`${c.name}: ${a} ok`); onChanged() } catch (e) { toast(`${c.name}: ${e instanceof Error ? e.message : 'failed'}`, 'err') } }
|
||||
const running = c.state === 'running'
|
||||
const s = (k: string) => live[`container.${c.name.toLowerCase().replace(/[^a-z0-9-]+/g, '_').replace(/^_|_$/g, '')}.${k}`]
|
||||
const seg = c.name.toLowerCase().replace(/[^a-z0-9-]+/g, '_').replace(/^_|_$/g, '')
|
||||
const s = (k: string) => live[`container.${seg}.${k}`]
|
||||
const mounts = (info?.mounts as { Source: string; Destination: string; RW: boolean }[] | undefined) ?? []
|
||||
const shell = async () => {
|
||||
try { const t = await api.dockerExec(c.name); toast(`shell open as “${t.name}” on the Terminal tab`); window.dispatchEvent(new CustomEvent('hamlab:terminal', { detail: t.id })) }
|
||||
catch (e) { toast(e instanceof Error ? e.message : 'failed', 'err') }
|
||||
}
|
||||
return (
|
||||
<div className="svc-detail">
|
||||
<CloseBtn onClick={onBack} />
|
||||
<div className="svc-head"><button className="btn back" onClick={onBack}>‹ Back</button><span className={'sdot ' + dot(c)} /><span className="dname">{c.name}</span></div>
|
||||
<div className="svc-head"><button className="btn back" onClick={onBack}>‹ Back</button><span className={'sdot ' + dot(c)} /><span className="dname">{c.name}</span>{c.health && c.health !== 'none' && <span className={'badge' + (c.health === 'healthy' ? '' : ' bad')}>{c.health}</span>}</div>
|
||||
<div className="props">
|
||||
<div><span className="k">Image</span><span className="mono small">{c.image}</span></div>
|
||||
<div><span className="k">Status</span><span>{c.status}</span></div>
|
||||
@@ -72,11 +82,22 @@ function Detail({ c, live, onBack, onChanged }: { c: DockerContainer; live: Reco
|
||||
<div className="svc-actions">
|
||||
{!running && <Btn className="btn on" onClick={act('start')}>Start</Btn>}
|
||||
{!running && <Btn className="btn danger" confirm={`Remove the container ${c.name}? Its image, volumes and bind-mounted data stay; anything written inside the container itself is lost.${c.stack ? ` Deploying the ${c.stack} stack makes it again.` : ''}`} onClick={async () => { await act('remove')(); onBack() }}>Remove</Btn>}
|
||||
{running && <Btn className="btn" onClick={shell}>Shell</Btn>}
|
||||
{running && <Btn className="btn" confirm={`Restart ${c.name}?`} onClick={act('restart')}>Restart</Btn>}
|
||||
<Btn className="btn" onClick={() => setLimits(true)}>Limits</Btn>
|
||||
{running && <Btn className="btn danger" confirm={`Stop ${c.name}? It stays stopped until something starts it.`} onClick={act('stop')}>Stop</Btn>}
|
||||
</div>
|
||||
<div className="tabs"><button className={tab === 'logs' ? 'on' : ''} onClick={() => setTab('logs')}>Logs</button><button className={tab === 'inspect' ? 'on' : ''} onClick={() => setTab('inspect')}>Inspect</button></div>
|
||||
<div className="tabs">
|
||||
<button className={tab === 'logs' ? 'on' : ''} onClick={() => setTab('logs')}>Logs</button>
|
||||
<button className={tab === 'stats' ? 'on' : ''} onClick={() => setTab('stats')}>Stats</button>
|
||||
<button className={tab === 'health' ? 'on' : ''} onClick={() => setTab('health')}>Health</button>
|
||||
<button className={tab === 'events' ? 'on' : ''} onClick={() => setTab('events')}>Events</button>
|
||||
<button className={tab === 'inspect' ? 'on' : ''} onClick={() => setTab('inspect')}>Inspect</button>
|
||||
</div>
|
||||
{tab === 'logs' && <LogView id={c.name} />}
|
||||
{tab === 'stats' && <Stats seg={seg} live={live} />}
|
||||
{tab === 'health' && <Health id={c.name} />}
|
||||
{tab === 'events' && <Events id={c.name} />}
|
||||
{tab === 'inspect' && (!info ? <Loading label="Inspecting…" /> : (
|
||||
<div className="props">
|
||||
<div><span className="k">Restarts</span><span>{String(info.restart || 'no')} policy · restarted {String(info.restarts ?? 0)}×</span></div>
|
||||
@@ -88,11 +109,154 @@ function Detail({ c, live, onBack, onChanged }: { c: DockerContainer; live: Reco
|
||||
<div><span className="k">Env</span><span className="mono small muted">{((info.env_names as string[]) ?? []).join(' ')} <span className="muted">(names only)</span></span></div>
|
||||
</div>
|
||||
))}
|
||||
{limits && <LimitsDialog c={c} info={info} onClose={() => setLimits(false)} onSaved={() => { setLimits(false); onChanged() }} />}
|
||||
</div>
|
||||
)
|
||||
}
|
||||
|
||||
// CPU and memory over time, from the container sensors the sampler already keeps.
|
||||
function Stats({ seg, live }: { seg: string; live: Record<string, Sensor> }) {
|
||||
const [hist, setHist] = useState<{ id: string; t: number[]; v: number[] }[] | null>(null)
|
||||
const historySec = useHistorySec()
|
||||
useEffect(() => {
|
||||
let alive = true
|
||||
fetchHistories(`container.${seg}.cpu,container.${seg}.mem`)
|
||||
.then((h) => { if (alive) setHist(h) })
|
||||
.catch(() => { if (alive) setHist([]) })
|
||||
return () => { alive = false }
|
||||
}, [seg])
|
||||
const css = (n: string) => getComputedStyle(document.documentElement).getPropertyValue(n).trim() || '#3987e5'
|
||||
const of = (k: string) => hist?.find((h) => h.id === `container.${seg}.${k}`)?.v ?? []
|
||||
const cpu = of('cpu'), mem = of('mem')
|
||||
if (hist === null) return <Loading label="Reading the history…" />
|
||||
if (cpu.length < 2 && mem.length < 2) return <div className="notice">No history yet for this container — it is sampled while it runs, so give it a moment.</div>
|
||||
return (
|
||||
<div className="cards">
|
||||
<section className="card">
|
||||
<div className="sub-head"><span className="dname">CPU</span><span className="spacer" /><span className="num">{fmtValue(live[`container.${seg}.cpu`]?.value, '%')} of a core</span></div>
|
||||
<Sparkline series={[{ name: 'cpu', values: cpu, color: css('--s1') }]} format={(v) => `${v.toFixed(1)}%`} height={64} />
|
||||
<div className="muted small">the last {fmtWindowSec(historySec)}</div>
|
||||
</section>
|
||||
<section className="card">
|
||||
<div className="sub-head"><span className="dname">Memory</span><span className="spacer" /><span className="num">{fmtValue(live[`container.${seg}.mem`]?.value, 'B')}</span></div>
|
||||
<Sparkline series={[{ name: 'mem', values: mem, color: css('--s2') }]} format={(v) => fmtBytes(v)} height={64} />
|
||||
<div className="muted small">the last {fmtWindowSec(historySec)}</div>
|
||||
</section>
|
||||
</div>
|
||||
)
|
||||
}
|
||||
|
||||
// The health check's verdict, and what its last few probes actually said.
|
||||
function Health({ id }: { id: string }) {
|
||||
const [h, setH] = useState<DockerHealth | null>(null)
|
||||
const [err, setErr] = useState('')
|
||||
useEffect(() => { setH(null); api.dockerHealth(id).then(setH).catch((e) => setErr(e instanceof Error ? e.message : 'failed')) }, [id])
|
||||
if (err) return <div className="error">{err}</div>
|
||||
if (!h) return <Loading label="Asking…" />
|
||||
if (!h.configured) return <div className="notice">This image defines no health check, so Docker only knows whether the process is running — which it is ({h.state}). A health check is a <span className="mono">healthcheck:</span> block in the compose file.</div>
|
||||
return (
|
||||
<>
|
||||
<div className="props">
|
||||
<div><span className="k">Health</span><span className={h.status === 'healthy' ? '' : 'warn-text'}>{h.status}</span></div>
|
||||
<div><span className="k">State</span><span>{h.state}</span></div>
|
||||
{!!h.failing_streak && <div><span className="k">Failing</span><span className="warn-text">{h.failing_streak} probe{h.failing_streak === 1 ? '' : 's'} in a row</span></div>}
|
||||
</div>
|
||||
<div className="muted small" style={{ marginTop: 8 }}>The last {h.log?.length ?? 0} probes — Docker keeps five.</div>
|
||||
{(h.log ?? []).slice().reverse().map((p, i) => (
|
||||
<section className="card" key={i} style={{ marginTop: 6 }}>
|
||||
<div className="sub-head">
|
||||
<span className={'sdot ' + (p.ExitCode === 0 ? 'ok' : 'bad')} />
|
||||
<span>{new Date(p.Start).toLocaleString()}</span>
|
||||
<span className="spacer" />
|
||||
<span className="muted small">exit {p.ExitCode} · {Math.max(0, new Date(p.End).getTime() - new Date(p.Start).getTime())} ms</span>
|
||||
</div>
|
||||
{p.Output.trim() && <pre className="run-out" style={{ maxHeight: 140 }}>{p.Output.trim()}</pre>}
|
||||
</section>
|
||||
))}
|
||||
</>
|
||||
)
|
||||
}
|
||||
|
||||
// What has happened to this container lately, asked of Docker rather than kept here.
|
||||
function Events({ id }: { id: string }) {
|
||||
const [ev, setEv] = useState<DockerEvent[] | null>(null)
|
||||
const [kept, setKept] = useState(0)
|
||||
const [since, setSince] = useState('7d')
|
||||
const [err, setErr] = useState('')
|
||||
useEffect(() => { setEv(null); setErr(''); api.dockerEvents(id, since).then((r) => { setEv(r.events); setKept(r.kept) }).catch((e) => setErr(e instanceof Error ? e.message : 'failed')) }, [id, since])
|
||||
const bad = (a: string, h?: string) => /die|kill|oom|destroy/.test(a) || (h !== undefined && h !== 'healthy')
|
||||
return (
|
||||
<>
|
||||
<div className="chips" style={{ marginBottom: 8 }}>
|
||||
{['24h', '7d', '30d'].map((x) => <button key={x} className={'chip' + (since === x ? ' on' : '')} onClick={() => setSince(x)}>{x}</button>)}
|
||||
</div>
|
||||
{err && <div className="error">{err}</div>}
|
||||
{!ev && !err && <Loading label="Asking Docker…" />}
|
||||
{ev && ev.length === 0 && <div className="notice">Nothing for this container in the last {since}. Ham Lab keeps its own record — Docker's own only reaches back a few minutes — so history starts from when this version was installed. {kept} events kept across every container.</div>}
|
||||
{ev && ev.length > 0 && (
|
||||
<div className="snap-list">
|
||||
{ev.map((e, i) => (
|
||||
<div className="snap" key={i}>
|
||||
<span className={bad(e.action, e.health) ? 'warn-text' : ''}>{e.action}{e.health ? `: ${e.health}` : ''}</span>
|
||||
<span className="muted">{new Date(e.t).toLocaleString()}</span>
|
||||
<span className="muted small">{e.exit_code ? `exit ${e.exit_code}` : ''}{e.signal ? ` signal ${e.signal}` : ''}</span>
|
||||
<span />
|
||||
<span />
|
||||
</div>
|
||||
))}
|
||||
</div>
|
||||
)}
|
||||
</>
|
||||
)
|
||||
}
|
||||
|
||||
// Memory, CPU and restart policy, changed on the running container. Docker does this live,
|
||||
// which is the point — and also the catch, said plainly below.
|
||||
function LimitsDialog({ c, info, onClose, onSaved }: { c: DockerContainer; info: Record<string, unknown> | null; onClose: () => void; onSaved: () => void }) {
|
||||
const curMem = Number(info?.memory_limit ?? 0)
|
||||
const [mem, setMem] = useState(curMem > 0 ? String(Math.round(curMem / (1 << 20))) + 'm' : '')
|
||||
const [cpus, setCpus] = useState('')
|
||||
const [restart, setRestart] = useState(String(info?.restart ?? ''))
|
||||
const [err, setErr] = useState('')
|
||||
const [busy, setBusy] = useState(false)
|
||||
useEscape(onClose)
|
||||
const save = async (e: FormEvent) => {
|
||||
e.preventDefault(); setErr(''); setBusy(true)
|
||||
try { await api.dockerLimits(c.name, { memory: mem, cpus, restart }); toast(`${c.name} updated`); onSaved() }
|
||||
catch (e) { setErr(e instanceof Error ? e.message : 'failed'); setBusy(false) }
|
||||
}
|
||||
return (
|
||||
<div className="modal-back" onClick={onClose}>
|
||||
<form className="modal" onClick={(e) => e.stopPropagation()} onSubmit={save}>
|
||||
<h3>Limits for {c.name}</h3>
|
||||
<label className="muted">Memory — 512m, 2g, or 0 for no limit{curMem > 0 ? ` (now ${fmtBytes(curMem)})` : ' (no limit now)'}
|
||||
<input className="mono" value={mem} onChange={(e) => setMem(e.target.value)} placeholder="leave blank to keep" spellCheck={false} />
|
||||
</label>
|
||||
<label className="muted">CPUs — 1.5 means one and a half cores, 0 for no limit
|
||||
<input className="mono" value={cpus} onChange={(e) => setCpus(e.target.value)} placeholder="leave blank to keep" spellCheck={false} />
|
||||
</label>
|
||||
<label className="muted">Restart policy
|
||||
<select value={restart} onChange={(e) => setRestart(e.target.value)}>
|
||||
<option value="">leave as it is</option>
|
||||
{['no', 'on-failure', 'unless-stopped', 'always'].map((p) => <option key={p} value={p}>{p}</option>)}
|
||||
</select>
|
||||
</label>
|
||||
<div className="notice attn">
|
||||
This changes the running container now, without recreating it. It does <b>not</b> edit the
|
||||
compose file{c.stack ? <> — so deploying <b>{c.stack}</b> puts back whatever the file says</> : ''}.
|
||||
Edit the stack to make it stick.
|
||||
</div>
|
||||
{err && <div className="error">{err}</div>}
|
||||
<div className="svc-actions">
|
||||
<button type="submit" className="btn on" disabled={busy || (!mem && !cpus && !restart)}>Apply</button>
|
||||
<span className="spacer" />
|
||||
<button type="button" className="btn" onClick={onClose}>Cancel</button>
|
||||
</div>
|
||||
</form>
|
||||
</div>
|
||||
)
|
||||
}
|
||||
|
||||
// A plain line diff (longest common subsequence): compose files are short, so the simple way is fine.
|
||||
function lineDiff(a: string, b: string): { s: ' ' | '+' | '-'; t: string }[] {
|
||||
const x = a.split('\n'), y = b.split('\n'); const n = x.length, m = y.length
|
||||
const L = Array.from({ length: n + 1 }, () => new Uint16Array(m + 1))
|
||||
@@ -120,6 +284,7 @@ const maskEnv = (text: string) => text.split('\n').map((l) => { const i = l.inde
|
||||
|
||||
// One stack: its compose file to edit, and the four things to do with it. Saving validates
|
||||
// with compose itself and keeps the previous file beside it; deploying is a separate, asked-for act.
|
||||
|
||||
function StackDetail({ name, startTask, onBack, onChanged, onDuplicate }: { name: string; startTask?: string; onBack: () => void; onChanged: () => void; onDuplicate: (from: DupFrom) => void }) {
|
||||
const [st, setSt] = useState<Stack | null>(null)
|
||||
const [text, setText] = useState(''); const [orig, setOrig] = useState('')
|
||||
@@ -203,6 +368,12 @@ export default function Docker() {
|
||||
// what a new stack could join, and what it could collide with
|
||||
const nets = useMemo(() => (d?.networks ?? []).map((n) => String(n.Name)).filter((n) => !['bridge', 'host', 'none'].includes(n)).sort(), [d])
|
||||
const taken = useMemo(() => { const ports: Record<string, string> = {}; for (const c of d?.containers ?? []) for (const m of (c.ports ?? '').matchAll(/:(\d+)->/g)) ports[m[1]] = c.name; return { names: (d?.containers ?? []).map((c) => c.name), ports } }, [d])
|
||||
// What the registry last said about each image, so a container can be marked without
|
||||
// asking again. Only ever read from the cache here — checking is a deliberate act.
|
||||
const [upd, setUpd] = useState<MachineUpdates | null>(null)
|
||||
useEffect(() => { api.machineUpdates().then(setUpd).catch(() => setUpd(null)) }, [])
|
||||
const imgState = useMemo(() => Object.fromEntries((upd?.images ?? []).map((i) => [i.image, i.state])), [upd])
|
||||
const behind = (upd?.images ?? []).filter((i) => i.state === 'update').length
|
||||
if (!d) return <div className="dash">{err ? <div className="notice">{err === 'not found' ? "This machine's Ham Lab is older than the Docker tab. Update it from Machines." : err}</div> : <Loading label="Asking Docker…" />}</div>
|
||||
if (!d.available) return <div className="dash"><div className="notice">{d.message}</div></div>
|
||||
const selStack = sel.startsWith('stack:') ? sel.slice(6) : ''
|
||||
@@ -210,12 +381,12 @@ export default function Docker() {
|
||||
const shownTab: Tab = selStack ? 'stacks' : tab
|
||||
const up = d.containers.filter((c) => c.state === 'running').length
|
||||
const sid = (c: DockerContainer, k: string) => sensors[`container.${c.name.toLowerCase().replace(/[^a-z0-9-]+/g, '_').replace(/^_|_$/g, '')}.${k}`]
|
||||
const cols: Record<Exclude<Tab, 'containers' | 'stacks'>, [string, string][]> = { images: [['Repository', 'Repository'], ['Tag', 'Tag'], ['Size', 'Size'], ['CreatedSince', 'Created']], volumes: [['Name', 'Name'], ['Driver', 'Driver'], ['Mountpoint', 'Mountpoint']], networks: [['Name', 'Name'], ['Driver', 'Driver'], ['Scope', 'Scope']] }
|
||||
const cols: Record<Exclude<Tab, 'containers' | 'stacks' | 'updates'>, [string, string][]> = { images: [['Repository', 'Repository'], ['Tag', 'Tag'], ['Size', 'Size'], ['CreatedSince', 'Created']], volumes: [['Name', 'Name'], ['Driver', 'Driver'], ['Mountpoint', 'Mountpoint']], networks: [['Name', 'Name'], ['Driver', 'Driver'], ['Scope', 'Scope']] }
|
||||
return (
|
||||
<div className={'svc docker' + (((cur || sel) && shownTab === 'containers') || selStack ? ' has-sel' : '')}>
|
||||
<div className="svc-list">
|
||||
<div className="svc-tools">
|
||||
<div className="chips">{(['containers', 'stacks', 'images', 'volumes', 'networks'] as Tab[]).map((t) => <button key={t} className={'chip' + (shownTab === t ? ' on' : '')} onClick={() => { setTab(t); if (sel) setSel('') }}>{t} · {t === 'containers' ? `${up}/${d.containers.length}` : t === 'stacks' ? (stacks?.length ?? '…') : d[t].length}</button>)}</div>
|
||||
<div className="chips">{(['containers', 'stacks', 'updates', 'images', 'volumes', 'networks'] as Tab[]).map((t) => <button key={t} className={'chip' + (shownTab === t ? ' on' : '')} onClick={() => { setTab(t); if (sel) setSel('') }}>{t}{t === 'containers' ? ` · ${up}/${d.containers.length}` : t === 'stacks' ? ` · ${stacks?.length ?? '…'}` : t === 'updates' ? (behind > 0 ? ` · ${behind}` : '') : ` · ${d[t].length}`}</button>)}</div>
|
||||
{(shownTab === 'containers' || shownTab === 'stacks') && <div className="svc-actions"><button className="btn" onClick={() => setMaking(shownTab === 'stacks' ? 'stack' : 'container')}>{shownTab === 'stacks' ? '+ New stack' : '+ New container'}</button></div>}
|
||||
{shownTab === 'images' && <form className="chan-add" onSubmit={async (e) => { e.preventDefault(); try { setPullTask((await api.pullImage(pull.trim())).id) } catch (er) { toast(er instanceof Error ? er.message : 'failed', 'err') } }}>
|
||||
<input className="mono" value={pull} onChange={(e) => setPull(e.target.value)} placeholder="pull an image — nginx:alpine, ghcr.io/owner/name:tag" spellCheck={false} autoCapitalize="off" autoCorrect="off" />
|
||||
@@ -242,7 +413,7 @@ export default function Docker() {
|
||||
{list.map((c) => (
|
||||
<div key={c.id} className={'svc-row' + (c.name === sel ? ' active' : '')} onClick={() => setSel(c.name)}>
|
||||
<span className={'sdot ' + dot(c)} />
|
||||
<div className="svc-name"><div className="n">{c.name}</div><div className="d">{c.image}</div></div>
|
||||
<div className="svc-name"><div className="n">{c.name}<UpdateDot state={imgState[c.image]} /></div><div className="d">{c.image}</div></div>
|
||||
<div className="svc-state">{c.state === 'running' ? <>{fmtValue(sid(c, 'mem')?.value ?? c.mem, 'B')}<br />{fmtValue(sid(c, 'cpu')?.value, '%')}</> : c.state}</div>
|
||||
</div>
|
||||
))}
|
||||
@@ -260,16 +431,212 @@ export default function Docker() {
|
||||
return <tr key={name}><td className="mono small" title={String(row.Mountpoint ?? '')}>{anon ? <>{name.slice(0, 12)}… <span className="muted">anonymous</span></> : name}</td><td className="small">{users.length ? users.join(', ') : <span className="muted">nothing</span>}</td><td className="row-end">{users.length ? <span className="badge">in use</span> : <button className="icon-btn" title="delete this volume and its data" onClick={remove}>✕</button>}</td></tr> })}
|
||||
{d.volumes.length === 0 && <tr><td colSpan={3} className="muted small">No volumes.</td></tr>}</tbody></table>
|
||||
)}
|
||||
{shownTab !== 'containers' && shownTab !== 'stacks' && shownTab !== 'volumes' && (
|
||||
<table className="dtable"><thead><tr>{cols[shownTab as Exclude<Tab, 'containers' | 'stacks'>].map(([, h]) => <th key={h}>{h}</th>)}{shownTab === 'images' && <th />}</tr></thead>
|
||||
<tbody>{d[shownTab as Exclude<Tab, 'containers' | 'stacks'>].map((row, i) => <tr key={i}>{cols[shownTab as Exclude<Tab, 'containers' | 'stacks'>].map(([k]) => <td key={k} className="mono small">{String(row[k] ?? '')}</td>)}{shownTab === 'images' && <td className="row-end">{row.in_use ? <span className="badge" title="a container uses this image">in use</span> : <Btn className="icon-btn" title="remove this image" confirm={`Remove the image ${row.Repository}:${row.Tag}? Docker refuses if a container still uses it.`} onClick={async () => { const ref = row.Tag && row.Tag !== '<none>' ? `${row.Repository}:${row.Tag}` : String(row.ID); try { await api.removeImage(ref); toast(`${ref} removed`); load() } catch (er) { toast(er instanceof Error ? er.message : 'failed', 'err') } }}>✕</Btn>}</td>}</tr>)}</tbody></table>
|
||||
{shownTab !== 'containers' && shownTab !== 'stacks' && shownTab !== 'volumes' && shownTab !== 'updates' && (
|
||||
<table className="dtable"><thead><tr>{cols[shownTab as Exclude<Tab, 'containers' | 'stacks' | 'volumes' | 'updates'>].map(([, h]) => <th key={h}>{h}</th>)}{shownTab === 'images' && <th />}</tr></thead>
|
||||
<tbody>{d[shownTab as Exclude<Tab, 'containers' | 'stacks' | 'volumes' | 'updates'>].map((row, i) => <tr key={i}>{cols[shownTab as Exclude<Tab, 'containers' | 'stacks' | 'volumes' | 'updates'>].map(([k]) => <td key={k} className="mono small">{String(row[k] ?? '')}</td>)}{shownTab === 'images' && <td className="row-end">{row.in_use ? <span className="badge" title="a container uses this image">in use</span> : <Btn className="icon-btn" title="remove this image" confirm={`Remove the image ${row.Repository}:${row.Tag}? Docker refuses if a container still uses it.`} onClick={async () => { const ref = row.Tag && row.Tag !== '<none>' ? `${row.Repository}:${row.Tag}` : String(row.ID); try { await api.removeImage(ref); toast(`${ref} removed`); load() } catch (er) { toast(er instanceof Error ? er.message : 'failed', 'err') } }}>✕</Btn>}</td>}</tr>)}</tbody></table>
|
||||
)}
|
||||
</div>
|
||||
</div>
|
||||
{shownTab === 'containers' && !cur && sel && !selStack && <div className="svc-detail"><CloseBtn onClick={() => setSel('')} /><button className="btn back" onClick={() => setSel('')}>‹ Back</button><div className="notice">There is no container called <b className="mono">{sel}</b> on this machine. It may have been removed, renamed or recreated under another name. <button className="link" onClick={() => setSel('')}>Show them all</button></div></div>}
|
||||
{shownTab === 'containers' && (cur ? <Detail c={cur} live={sensors} onBack={() => setSel('')} onChanged={load} /> : (sel && !selStack ? null : <div className="svc-empty muted">Select a container.</div>))}
|
||||
{making && <StackNew key={'new:' + (dup?.name ?? making)} mode={making} from={dup ?? undefined} networks={nets} taken={taken} onClose={() => { setMaking(''); setDup(null) }} onCreated={async (name, deploy) => { setMaking(''); setDup(null); setTab('stacks'); if (deploy) { try { setStartTask({ name, id: (await api.stackAction(name, 'deploy')).id }) } catch (e) { toast(e instanceof Error ? e.message : 'failed', 'err') } } else setStartTask(null); await load(); setSel('stack:' + name) }} />}
|
||||
{shownTab === 'updates' && <Updates upd={upd} onChecked={setUpd} />}
|
||||
{shownTab === 'stacks' && (selStack ? <StackDetail key={selStack + (startTask?.name === selStack ? startTask.id : '')} name={selStack} startTask={startTask?.name === selStack ? startTask.id : undefined} onBack={() => setSel('')} onChanged={load} onDuplicate={(f) => { setDup(f); setMaking('stack') }} /> : <div className="svc-empty muted">Select a stack to edit its compose file.</div>)}
|
||||
</div>
|
||||
)
|
||||
}
|
||||
|
||||
// A container's image against the registry, as a single mark: nothing when nobody has
|
||||
// asked, so an uncoloured list means "not checked" rather than "all current".
|
||||
function UpdateDot({ state }: { state?: string }) {
|
||||
if (!state || state === 'current') return null
|
||||
const known = state === 'update'
|
||||
return <span className={'updot' + (known ? ' behind' : ' unknown')} title={known ? 'a newer image has been published' : 'the registry could not say'}>{known ? '↑' : '?'}</span>
|
||||
}
|
||||
|
||||
// What is out of date here, and the schedules that keep it from happening again.
|
||||
function Updates({ upd, onChecked }: { upd: MachineUpdates | null; onChecked: (u: MachineUpdates) => void }) {
|
||||
const onHub = apiPath('/api/x') === '/api/x'
|
||||
const [busy, setBusy] = useState(false)
|
||||
const [jobs, setJobs] = useState<UpdateJobRow[] | null>(null)
|
||||
const [file, setFile] = useState('')
|
||||
const [editing, setEditing] = useState<Partial<UpdateJob> | null>(null)
|
||||
const [stacks, setStacks] = useState<string[]>([])
|
||||
const loadJobs = useCallback(async () => {
|
||||
try { const r = await api.updateJobs(); setJobs(r.jobs); setFile(r.file) } catch (e) { toast(e instanceof Error ? e.message : 'failed', 'err'); setJobs([]) }
|
||||
}, [])
|
||||
useEffect(() => { loadJobs(); api.stacks().then((r) => setStacks(r.stacks.map((s) => s.name))).catch(() => {}) }, [loadJobs])
|
||||
const check = async () => {
|
||||
setBusy(true)
|
||||
try { onChecked(await api.machineUpdates(true)); toast('checked') } catch (e) { toast(e instanceof Error ? e.message : 'failed', 'err') } finally { setBusy(false) }
|
||||
}
|
||||
const run = async (j: UpdateJob) => {
|
||||
if (!await askYes(`Run “${j.name}” now? It pulls ${j.stack}'s images${j.apply ? ' and deploys them, which restarts its containers' : ', without deploying them'}.`, 'Run now')) return
|
||||
try { await api.runUpdateJob(j.id); toast(`${j.name}: done`); loadJobs() } catch (e) { toast(e instanceof Error ? e.message : 'failed', 'err') }
|
||||
}
|
||||
const remove = async (j: UpdateJob) => {
|
||||
if (!await askYes(`Remove the schedule “${j.name}”? Nothing is updated or rolled back — the schedule simply stops.`, 'Remove')) return
|
||||
try { await api.deleteUpdateJob(j.id); toast('removed'); loadJobs() } catch (e) { toast(e instanceof Error ? e.message : 'failed', 'err') }
|
||||
}
|
||||
const imgs = upd?.images ?? []
|
||||
const behind = imgs.filter((i) => i.state === 'update')
|
||||
return (
|
||||
<div className="svc-detail">
|
||||
<div className="dash-bar"><h2 className="section">What is out of date</h2><span className="spacer" />
|
||||
<span className="muted small">{upd?.checked ? `checked ${new Date(upd.checked).toLocaleString()}` : 'not checked yet'}</span>
|
||||
<Btn className="btn" busy={busy} onClick={check}>Check now</Btn>
|
||||
</div>
|
||||
<div className="muted small">Each check is one request per image, which registries count — so it happens when asked, and the answer is kept for six hours.</div>
|
||||
{imgs.length === 0 && <div className="notice" style={{ marginTop: 8 }}>Nothing checked yet on this machine.</div>}
|
||||
{imgs.length > 0 && (
|
||||
<section className="card" style={{ marginTop: 8 }}>
|
||||
{behind.length === 0 && <div className="muted">Every image that could be checked is current.</div>}
|
||||
{imgs.filter((i) => i.state !== 'current').map((i) => (
|
||||
<div className="row" key={i.image}>
|
||||
<span className="k mono small">{i.image}</span>
|
||||
<span className={i.state === 'update' ? 'warn-text' : 'muted small'}>{i.state === 'update' ? 'a newer image is published' : i.detail || 'unknown'}</span>
|
||||
<span className="muted small">{(upd?.stacks?.[i.image] ?? []).join(' ')}</span>
|
||||
</div>
|
||||
))}
|
||||
{behind.length === 0 && imgs.some((i) => i.state === 'current') && <div className="muted small" style={{ marginTop: 6 }}>{imgs.filter((i) => i.state === 'current').length} current.</div>}
|
||||
</section>
|
||||
)}
|
||||
|
||||
{onHub && <Fleet />}
|
||||
|
||||
<div className="dash-bar" style={{ marginTop: 14 }}><h2 className="section">Schedules</h2><span className="spacer" />
|
||||
<Btn className="btn on" onClick={() => setEditing({ cron: '0 3 * * *', apply: true, enabled: true })}>+ New schedule</Btn>
|
||||
</div>
|
||||
<div className="muted small">Written to <span className="mono">{file || '/etc/cron.d/hamlab-docker'}</span>, which Ham Lab owns — so they keep running whether or not this console is up.</div>
|
||||
{jobs === null && <Loading label="Reading…" />}
|
||||
{jobs?.length === 0 && <div className="notice" style={{ marginTop: 8 }}>No schedule yet. Nothing on this machine updates itself.</div>}
|
||||
<div className="cards" style={{ marginTop: 8 }}>
|
||||
{(jobs ?? []).map(({ job, next, last }) => (
|
||||
<section className="card" key={job.id}>
|
||||
<div className="sub-head">
|
||||
<span className={'dot ' + (last?.error ? 'bad' : job.enabled ? 'ok' : '')} />
|
||||
<span className="dname">{job.name}</span>
|
||||
{!job.enabled && <span className="badge">paused</span>}
|
||||
<span className="spacer" />
|
||||
<span className="muted small">{job.enabled && next ? new Date(next).toLocaleString() : 'not scheduled'}</span>
|
||||
</div>
|
||||
<div className="props" style={{ marginTop: 6 }}>
|
||||
<div><span className="k">Stack</span><span>{job.stack}{job.services?.length ? ` · ${job.services.join(' ')}` : ''}</span></div>
|
||||
<div><span className="k">Cron</span><span className="mono small">{job.cron}</span></div>
|
||||
<div><span className="k">Does</span><span>{job.apply ? 'pulls and deploys' : 'pulls only — you deploy'}</span></div>
|
||||
<div><span className="k">Last run</span><span>{last?.at ? new Date(last.at).toLocaleString() : 'never'}</span></div>
|
||||
</div>
|
||||
{last?.error && <div className="error" style={{ marginTop: 6 }}>{last.error}</div>}
|
||||
<div className="svc-actions">
|
||||
<Btn className="btn on" onClick={() => run(job)}>Run now</Btn>
|
||||
<button className="btn" onClick={() => setEditing(job)}>Edit</button>
|
||||
<Btn className="btn" onClick={async () => { await api.saveUpdateJob({ ...job, enabled: !job.enabled }); loadJobs() }}>{job.enabled ? 'Pause' : 'Resume'}</Btn>
|
||||
<Btn className="btn danger" onClick={() => remove(job)}>Remove</Btn>
|
||||
</div>
|
||||
</section>
|
||||
))}
|
||||
</div>
|
||||
{editing && <JobDialog j={editing} stacks={stacks} onClose={() => setEditing(null)} onSaved={() => { setEditing(null); loadJobs() }} />}
|
||||
</div>
|
||||
)
|
||||
}
|
||||
|
||||
// Every other machine at once. Only shown on the hub, because only the hub can reach them,
|
||||
// and each answer is that machine's own six-hour cache — nothing is checked here without
|
||||
// being asked for.
|
||||
function Fleet() {
|
||||
const [rows, setRows] = useState<{ name: string; online: boolean; upd?: MachineUpdates; err?: string }[] | null>(null)
|
||||
const [busy, setBusy] = useState('')
|
||||
const load = useCallback(async (refresh?: string) => {
|
||||
let nodes: MachineStatus[] = []
|
||||
try { nodes = (await api.machines()).nodes } catch { setRows([]); return }
|
||||
const out = await Promise.all(nodes.map(async (n) => {
|
||||
if (!n.online) return { name: n.name, online: false }
|
||||
try { return { name: n.name, online: true, upd: await api.nodeUpdates(n.name, refresh === n.name) } }
|
||||
catch (e) { return { name: n.name, online: true, err: e instanceof Error ? e.message : 'failed' } }
|
||||
}))
|
||||
setRows(out)
|
||||
}, [])
|
||||
useEffect(() => { load() }, [load])
|
||||
if (rows === null) return <div style={{ marginTop: 14 }}><Loading label="Asking the other machines…" /></div>
|
||||
if (rows.length === 0) return null
|
||||
return (
|
||||
<>
|
||||
<h2 className="section" style={{ marginTop: 14 }}>The other machines</h2>
|
||||
<section className="card">
|
||||
{rows.map((r) => {
|
||||
const behind = (r.upd?.images ?? []).filter((i) => i.state === 'update')
|
||||
return (
|
||||
<div className="row" key={r.name}>
|
||||
<span className="k">{r.name}</span>
|
||||
<span>
|
||||
{!r.online ? <span className="muted">offline</span>
|
||||
: r.err ? <span className="muted small">{r.err}</span>
|
||||
: !r.upd?.checked ? <span className="muted">not checked</span>
|
||||
: behind.length === 0 ? <span className="muted">all current</span>
|
||||
: <span className="warn-text" title={behind.map((i) => i.image).join('\n')}>{behind.length} behind</span>}
|
||||
</span>
|
||||
<span className="row-end">{r.online && !r.err && <Btn className="link" busy={busy === r.name} onClick={async () => { setBusy(r.name); try { await load(r.name) } finally { setBusy('') } }}>check</Btn>}</span>
|
||||
</div>
|
||||
)
|
||||
})}
|
||||
</section>
|
||||
<div className="muted small">Open a machine from the switcher to schedule anything on it — a schedule lives on the machine that runs it.</div>
|
||||
</>
|
||||
)
|
||||
}
|
||||
|
||||
function JobDialog({ j, stacks, onClose, onSaved }: { j: Partial<UpdateJob>; stacks: string[]; onClose: () => void; onSaved: () => void }) {
|
||||
const [f, setF] = useState<Partial<UpdateJob>>(j)
|
||||
const [svc, setSvc] = useState((j.services ?? []).join(' '))
|
||||
const [err, setErr] = useState('')
|
||||
const [busy, setBusy] = useState(false)
|
||||
useEscape(onClose)
|
||||
const set = (k: keyof UpdateJob, v: unknown) => setF((x) => ({ ...x, [k]: v }))
|
||||
const save = async (e: FormEvent) => {
|
||||
e.preventDefault(); setErr(''); setBusy(true)
|
||||
try { await api.saveUpdateJob({ ...f, services: svc.split(/\s+/).filter(Boolean) }); onSaved() }
|
||||
catch (e) { setErr(e instanceof Error ? e.message : 'failed'); setBusy(false) }
|
||||
}
|
||||
return (
|
||||
<div className="modal-back" onClick={onClose}>
|
||||
<form className="modal" onClick={(e) => e.stopPropagation()} onSubmit={save}>
|
||||
<h3>{f.id ? 'Edit schedule' : 'Keep a stack up to date'}</h3>
|
||||
<label className="muted">Stack
|
||||
<select value={f.stack ?? ''} onChange={(e) => set('stack', e.target.value)} required>
|
||||
<option value="">choose…</option>
|
||||
{stacks.map((s) => <option key={s} value={s}>{s}</option>)}
|
||||
</select>
|
||||
</label>
|
||||
<label className="muted">Name — optional
|
||||
<input value={f.name ?? ''} onChange={(e) => set('name', e.target.value)} placeholder={f.stack ?? 'nightly'} maxLength={64} />
|
||||
</label>
|
||||
<label className="muted">Only these services — optional, space separated; blank means the whole stack
|
||||
<input className="mono" value={svc} onChange={(e) => setSvc(e.target.value)} placeholder="all of them" spellCheck={false} />
|
||||
</label>
|
||||
<label className="muted">When — a cron expression
|
||||
<input className="mono" value={f.cron ?? ''} onChange={(e) => set('cron', e.target.value)} placeholder="0 3 * * *" spellCheck={false} required />
|
||||
</label>
|
||||
<label className="muted" style={{ flexDirection: 'row', alignItems: 'center', gap: 8 }}>
|
||||
<input type="checkbox" checked={!!f.apply} onChange={(e) => set('apply', e.target.checked)} style={{ width: 'auto' }} />
|
||||
Deploy the new image, not just fetch it
|
||||
</label>
|
||||
<label className="muted" style={{ flexDirection: 'row', alignItems: 'center', gap: 8 }}>
|
||||
<input type="checkbox" checked={!!f.enabled} onChange={(e) => set('enabled', e.target.checked)} style={{ width: 'auto' }} />
|
||||
Enabled
|
||||
</label>
|
||||
<div className="notice">
|
||||
{f.apply
|
||||
? 'Deploying restarts the stack’s containers. On something that carries DNS or ingress, pick an hour you are asleep.'
|
||||
: 'Pulling only means the new image is on the machine and the list shows it as available — nothing restarts until you deploy.'}
|
||||
</div>
|
||||
{err && <div className="error">{err}</div>}
|
||||
<div className="svc-actions">
|
||||
<button type="submit" className="btn on" disabled={busy || !f.stack || !f.cron}>Save</button>
|
||||
<span className="spacer" />
|
||||
<button type="button" className="btn" onClick={onClose}>Cancel</button>
|
||||
</div>
|
||||
</form>
|
||||
</div>
|
||||
)
|
||||
}
|
||||
|
||||
+1
-1
@@ -1,6 +1,6 @@
|
||||
import { DragEvent, useCallback, useEffect, useRef, useState } from 'react'
|
||||
import { api, FileEntry, FileInfo } from './api'
|
||||
import { fmtBytes } from './Dashboard'
|
||||
import { fmtBytes } from './fmt'
|
||||
import { Btn, Loading, Spinner, askText, askYes, setUnsaved, toast, useEscape } from './ui'
|
||||
import { useHashSel } from './hashsel'
|
||||
|
||||
|
||||
@@ -57,6 +57,13 @@ export default function Gallery() {
|
||||
|
||||
return (
|
||||
<div className="dash gallery">
|
||||
<div className="notice" style={{ marginBottom: 10 }}>
|
||||
These widgets are for <b>dashboards outside Ham Lab</b> — Glance, Home Assistant, a wall
|
||||
display. Each one is a page at <span className="mono">/embed/widget</span> you put in an
|
||||
iframe, reading this machine through a <b>read-only sensor token</b> (Settings → Access
|
||||
tokens). Ham Lab shows you these numbers on its own pages; this tab is for handing them
|
||||
to something else.
|
||||
</div>
|
||||
<div className="dash-bar">
|
||||
<span className="hint">{needle ? `${shown.length} of ${sensors.length}` : sensors.length} sensors on this machine · live · tap a tile to copy its ID</span>
|
||||
<input className="gallery-find" value={find} onChange={(e) => setFind(e.target.value)} placeholder="find — gitea, temp, enp3s0…" spellCheck={false} autoCapitalize="off" />
|
||||
|
||||
@@ -175,11 +175,11 @@ function TimerDetail({ t, onChanged, onBack }: { t: CronTimer; onChanged: () =>
|
||||
}
|
||||
|
||||
// ---- page ----
|
||||
export default function Cron() {
|
||||
export default function Jobs() {
|
||||
const [ov, setOv] = useState<CronOverview | null>(null)
|
||||
const [err, setErr] = useState('')
|
||||
const [q, setQ] = useState('')
|
||||
const [hashSel, setHashSel] = useHashSel('cron') // "job:<id>" or "timer:<unit>", in the URL
|
||||
const [hashSel, setHashSel] = useHashSel('jobs') // "job:<id>" or "timer:<unit>", in the URL
|
||||
const sel = /^(job|timer):./.test(hashSel) ? { kind: hashSel.split(':')[0] as 'job' | 'timer', id: hashSel.slice(hashSel.indexOf(':') + 1) } : null
|
||||
const setSel = (v: { kind: 'job' | 'timer'; id: string } | null) => setHashSel(v ? `${v.kind}:${v.id}` : '')
|
||||
const [editing, setEditing] = useState<string | null>(null)
|
||||
@@ -206,7 +206,7 @@ export default function Cron() {
|
||||
else pane = <div className="svc-empty muted">Select a job or timer.</div>
|
||||
|
||||
return (
|
||||
<div className={'cron' + (sel || editing ? ' has-sel' : '')}>
|
||||
<div className={'cron jobs' + (sel || editing ? ' has-sel' : '')}>
|
||||
<div className="cron-list">
|
||||
<div className="svc-tools">
|
||||
<div className="tools-row"><input placeholder="filter jobs and timers" value={q} onChange={(e) => setQ(e.target.value)} /><Btn className="btn" title="reload" onClick={load}>↻</Btn></div>
|
||||
@@ -218,7 +218,7 @@ export default function Cron() {
|
||||
{ov && !ov.logging && <div className="notice" style={{ margin: 8 }}>cron isn’t logging to the journal on this host, so “last run” is unavailable for cron jobs.</div>}
|
||||
<div className="cron-rows">
|
||||
{!ov && !err && <Loading label="Reading crontabs…" />}
|
||||
{ov && <h3>Cron jobs · {jobs.length}</h3>}
|
||||
{ov && <h3>Scheduled jobs · {jobs.length}</h3>}
|
||||
{jobs.map((j) => (
|
||||
<div key={j.id} className={'cron-row' + (sel?.kind === 'job' && sel.id === j.id ? ' active' : '') + (j.enabled ? '' : ' off')} onClick={() => { setEditing(null); setSel({ kind: 'job', id: j.id }) }}>
|
||||
<span className={'sdot ' + (j.enabled ? 'ok' : '')} />
|
||||
+187
@@ -0,0 +1,187 @@
|
||||
// The KVM of the machine you are looking at: the device's own web page, shown in the tab.
|
||||
//
|
||||
// One per machine, and it follows the machine switcher — switch to `grid` and you get
|
||||
// grid's KVM. The address is held on the **hub**, not on the machine, because a KVM earns
|
||||
// its keep exactly when that machine is down and cannot answer for itself; the hub proxies
|
||||
// the device either way.
|
||||
//
|
||||
// Nothing here knows anything about any particular KVM. It is an iframe pointed at Ham
|
||||
// Lab's own proxy, so whatever web UI the device serves is what you get.
|
||||
//
|
||||
// On the keyboard: a page cannot capture keys and forward them into an iframe, and does not
|
||||
// need to. An iframe with focus receives them directly. What the browser itself swallows —
|
||||
// Escape, Tab, Ctrl-W, the function keys — is what `navigator.keyboard.lock()` releases,
|
||||
// and that needs fullscreen and only exists in Chromium. Everywhere else you get the plain
|
||||
// version, which is most keys, and the device's own on-screen buttons for the rest.
|
||||
import { FormEvent, useCallback, useEffect, useRef, useState } from 'react'
|
||||
import { KVM as Device, api } from './api'
|
||||
import { Btn, Loading, askYes, toast, useEscape } from './ui'
|
||||
|
||||
// lock() releases the keys the browser would otherwise keep for itself. Chromium only; a
|
||||
// browser without it simply does not get the extra keys.
|
||||
type KeyboardLock = { lock?: (keys?: string[]) => Promise<void>; unlock?: () => void }
|
||||
const keyboardAPI = (): KeyboardLock | undefined => (navigator as unknown as { keyboard?: KeyboardLock }).keyboard
|
||||
|
||||
// Fullscreen, across the three ways browsers spell it. Safari on a Mac wants the webkit
|
||||
// prefix; Safari on iOS has no fullscreen for an ordinary element at all — only for a
|
||||
// <video> — so there the page expands inside the viewport instead, which is the same thing
|
||||
// minus the browser's own chrome.
|
||||
type FSElement = HTMLElement & { webkitRequestFullscreen?: () => Promise<void> | void }
|
||||
type FSDocument = Document & { webkitFullscreenElement?: Element | null; webkitExitFullscreen?: () => Promise<void> | void }
|
||||
const nativeFullscreen = () => {
|
||||
if (typeof document === 'undefined') return false
|
||||
const el = document.documentElement as FSElement
|
||||
return typeof el.requestFullscreen === 'function' || typeof el.webkitRequestFullscreen === 'function'
|
||||
}
|
||||
const fullscreenElement = () => (document as FSDocument).fullscreenElement ?? (document as FSDocument).webkitFullscreenElement ?? null
|
||||
async function enterFullscreen(el: FSElement) {
|
||||
if (typeof el.requestFullscreen === 'function') return el.requestFullscreen()
|
||||
if (typeof el.webkitRequestFullscreen === 'function') return el.webkitRequestFullscreen()
|
||||
throw new Error('this browser has no fullscreen for a page element')
|
||||
}
|
||||
function exitFullscreen() {
|
||||
const d = document as FSDocument
|
||||
if (typeof d.exitFullscreen === 'function') return d.exitFullscreen()
|
||||
if (typeof d.webkitExitFullscreen === 'function') return d.webkitExitFullscreen()
|
||||
}
|
||||
|
||||
export default function KVM({ machine, title }: { machine: string | null; title: string }) {
|
||||
const [kvm, setKvm] = useState<Device | null | undefined>(undefined)
|
||||
const [adding, setAdding] = useState(false)
|
||||
const [err, setErr] = useState('')
|
||||
const [full, setFull] = useState(false) // native fullscreen is on
|
||||
const [expanded, setExpanded] = useState(false) // the fallback: filling the viewport in-page
|
||||
const box = useRef<HTMLDivElement>(null)
|
||||
const frame = useRef<HTMLIFrameElement>(null)
|
||||
const key = machine ?? '' // the hub is a machine too, and its key is the empty one
|
||||
|
||||
const load = useCallback(async () => {
|
||||
try { setKvm((await api.kvms()).find((k) => k.machine === key) ?? null) }
|
||||
catch (e) { setErr(e instanceof Error ? e.message : 'failed') }
|
||||
}, [key])
|
||||
useEffect(() => { load() }, [load])
|
||||
|
||||
// Fullscreen plus a keyboard lock is what makes Escape and Tab reach the machine rather
|
||||
// than the browser. Both are undone together when fullscreen ends, however it ends.
|
||||
useEffect(() => {
|
||||
const onChange = () => {
|
||||
const on = !!fullscreenElement()
|
||||
setFull(on)
|
||||
if (!on) keyboardAPI()?.unlock?.()
|
||||
}
|
||||
document.addEventListener('fullscreenchange', onChange)
|
||||
document.addEventListener('webkitfullscreenchange', onChange)
|
||||
return () => {
|
||||
document.removeEventListener('fullscreenchange', onChange)
|
||||
document.removeEventListener('webkitfullscreenchange', onChange)
|
||||
keyboardAPI()?.unlock?.()
|
||||
}
|
||||
}, [])
|
||||
// Expanding is in-page, so Escape is ours to handle.
|
||||
useEscape(() => setExpanded(false), expanded)
|
||||
|
||||
const goBig = async () => {
|
||||
if (!nativeFullscreen()) { setExpanded(true); frame.current?.focus(); return }
|
||||
try {
|
||||
if (box.current) await enterFullscreen(box.current)
|
||||
await keyboardAPI()?.lock?.()
|
||||
frame.current?.focus()
|
||||
} catch {
|
||||
setExpanded(true) // refused (an iframe policy, a permissions policy): fill the page instead
|
||||
frame.current?.focus()
|
||||
}
|
||||
}
|
||||
const goSmall = () => { setExpanded(false); if (fullscreenElement()) exitFullscreen() }
|
||||
|
||||
const remove = async () => {
|
||||
if (!kvm || !await askYes(`Forget ${title}'s KVM? The device itself is not touched — this only removes its address.`, 'Remove')) return
|
||||
try { await api.removeKVM(kvm.id); toast('removed'); await load() } catch (e) { toast(e instanceof Error ? e.message : 'failed', 'err') }
|
||||
}
|
||||
|
||||
if (err && kvm === undefined) return <div className="dash"><div className="error">{err}</div></div>
|
||||
if (kvm === undefined) return <div className="dash"><Loading label="Looking…" /></div>
|
||||
|
||||
const locks = !!keyboardAPI()?.lock
|
||||
const big = full || expanded
|
||||
|
||||
return (
|
||||
<div className="kvm">
|
||||
<div className="kvm-bar">
|
||||
<span className="dname">{title}</span>
|
||||
{kvm
|
||||
? <span className="muted small mono kvm-url" title={kvm.url}>{kvm.name !== kvm.url ? `${kvm.name} · ` : ''}{kvm.url}</span>
|
||||
: <span className="muted small">no KVM</span>}
|
||||
<span className="spacer" />
|
||||
{kvm && <>
|
||||
<Btn className="btn" onClick={goBig} title={!nativeFullscreen() ? 'fill the screen — this browser has no true fullscreen for a page element' : locks ? 'fullscreen, and the browser stops keeping Escape, Tab and Ctrl-W for itself' : 'fullscreen — this browser will still keep some keys for itself'}>{nativeFullscreen() ? 'Fullscreen' : 'Expand'}</Btn>
|
||||
<a className="btn" href={kvm.url} target="_blank" rel="noreferrer" title="the device directly, outside Ham Lab — only works on the LAN">Open ↗</a>
|
||||
<Btn className="btn" onClick={() => setAdding(true)}>Change</Btn>
|
||||
<Btn className="btn danger" onClick={remove}>Remove</Btn>
|
||||
</>}
|
||||
{!kvm && <Btn className="btn on" onClick={() => setAdding(true)}>+ Add a KVM for {title}</Btn>}
|
||||
</div>
|
||||
|
||||
{kvm ? (
|
||||
<div className={'kvm-frame' + (full ? ' full' : '') + (expanded ? ' expanded' : '')} ref={box}>
|
||||
<iframe ref={frame} key={kvm.id} src={`/kvm/${kvm.id}/`} title={kvm.name}
|
||||
allow="fullscreen; clipboard-read; clipboard-write" />
|
||||
{expanded && <button className="btn kvm-shrink" onClick={goSmall} title="back to the console">Done</button>}
|
||||
</div>
|
||||
) : (
|
||||
<div className="svc-empty muted kvm-none">
|
||||
<div>
|
||||
<p>No KVM for <b>{title}</b>.</p>
|
||||
<p className="small">Add one with its address and its page appears here. The address is kept on the hub,
|
||||
so it still works when {title} does not — which is when you will want it.</p>
|
||||
</div>
|
||||
</div>
|
||||
)}
|
||||
{kvm && !big && (
|
||||
<div className="kvm-hint muted small">
|
||||
Click the picture, then type — the keys go to {title}.
|
||||
{locks ? ' Fullscreen also takes Escape, Tab and Ctrl-W back from the browser.' : ' This browser keeps Escape, Tab and Ctrl-W for itself; use the device’s own on-screen keys for those, or a Chromium browser.'}
|
||||
</div>
|
||||
)}
|
||||
{adding && <AddDialog machine={key} title={title} existing={kvm} onClose={() => setAdding(false)} onAdded={() => { setAdding(false); load() }} />}
|
||||
</div>
|
||||
)
|
||||
}
|
||||
|
||||
function AddDialog({ machine, title, existing, onClose, onAdded }: { machine: string; title: string; existing: Device | null; onClose: () => void; onAdded: () => void }) {
|
||||
const [url, setUrl] = useState(existing?.url ?? '')
|
||||
const [name, setName] = useState(existing?.name ?? '')
|
||||
const [err, setErr] = useState('')
|
||||
const [busy, setBusy] = useState(false)
|
||||
useEscape(onClose)
|
||||
const save = async (e: FormEvent) => {
|
||||
e.preventDefault(); setErr(''); setBusy(true)
|
||||
try { await api.addKVM(name.trim(), url.trim(), machine); onAdded() }
|
||||
catch (e) { setErr(e instanceof Error ? e.message : 'failed'); setBusy(false) }
|
||||
}
|
||||
return (
|
||||
<div className="modal-back" onClick={onClose}>
|
||||
<form className="modal" onClick={(e) => e.stopPropagation()} onSubmit={save}>
|
||||
<h3>{existing ? `Change ${title}'s KVM` : `A KVM for ${title}`}</h3>
|
||||
<label className="muted">Address — whatever you would type into a browser
|
||||
<input autoFocus className="mono" value={url} onChange={(e) => setUrl(e.target.value)} placeholder="10.0.0.217" spellCheck={false} autoCapitalize="off" required />
|
||||
</label>
|
||||
<label className="muted">Name — optional; the address is used otherwise
|
||||
<input value={name} onChange={(e) => setName(e.target.value)} placeholder={`${title} KVM`} maxLength={64} />
|
||||
</label>
|
||||
<div className="muted small">
|
||||
Ham Lab serves the device from its own address, so a KVM that speaks only plain HTTP
|
||||
still works here, and it is behind this login rather than needing a second one. The
|
||||
address is kept on the hub, so it still reaches the device when {title} is down. The
|
||||
device itself is not changed and is never reachable from outside.
|
||||
{existing && <> Saving replaces the one there now — a machine has one KVM.</>}
|
||||
</div>
|
||||
{err && <div className="error">{err}</div>}
|
||||
<div className="svc-actions">
|
||||
<button type="submit" className="btn on" disabled={busy || !url.trim()}>{existing ? 'Save' : 'Add'}</button>
|
||||
<span className="spacer" />
|
||||
<button type="button" className="btn" onClick={onClose}>Cancel</button>
|
||||
</div>
|
||||
</form>
|
||||
</div>
|
||||
)
|
||||
}
|
||||
+42
-20
@@ -4,19 +4,20 @@ import Rail, { Group } from './Rail'
|
||||
import TerminalPane, { TermHandle } from './TerminalPane'
|
||||
import TermTabs from './TermTabs'
|
||||
import KeyBar from './KeyBar'
|
||||
import Dashboard from './Dashboard'
|
||||
import Gallery from './Gallery'
|
||||
import Docker from './Docker'
|
||||
import Alerts from './Alerts'
|
||||
import Services from './Services'
|
||||
import Cron from './Cron'
|
||||
import Jobs from './Jobs'
|
||||
import Updates from './Updates'
|
||||
import Disks from './Disks'
|
||||
import ZFS from './ZFS'
|
||||
import KVM from './KVM'
|
||||
import Files from './Files'
|
||||
import Top from './Top'
|
||||
import Machines from './Machines'
|
||||
import Settings from './Settings'
|
||||
import Space from './Space'
|
||||
import Usage from './Usage'
|
||||
import Users from './Users'
|
||||
import MachineSwitcher from './MachineSwitcher'
|
||||
import { MachineStatus, setMachine } from './api'
|
||||
@@ -51,33 +52,39 @@ const BellIcon = (
|
||||
)
|
||||
|
||||
const GROUPS: Group[] = [
|
||||
{ id: 'dashboard', label: 'Dashboard', icon: '◔' },
|
||||
{ id: 'procs', label: 'Processes', icon: '▚' },
|
||||
{ id: 'terminal', label: 'Terminal', icon: '>_' },
|
||||
{ id: 'services', label: 'Services', icon: '◈' },
|
||||
{ id: 'docker', label: 'Docker', icon: DockerIcon },
|
||||
{ id: 'cron', label: 'Cron', icon: ClockIcon },
|
||||
{ id: 'updates', label: 'Updates', icon: '⇪' },
|
||||
{ id: 'jobs', label: 'Jobs', icon: ClockIcon },
|
||||
{ id: 'disks', label: 'Disks', icon: '▤' },
|
||||
{ id: 'space', label: 'Space', icon: '▧' },
|
||||
{ id: 'files', label: 'Files', icon: FolderIcon },
|
||||
{ id: 'usage', label: 'Usage', icon: '▧' },
|
||||
{ id: 'zfs', label: 'ZFS', icon: '⛁' },
|
||||
{ id: 'docker', label: 'Docker', icon: DockerIcon },
|
||||
{ id: 'kvm', label: 'KVM', icon: '⌨' },
|
||||
{ id: 'updates', label: 'Updates', icon: '⇪' },
|
||||
{ id: 'alerts', label: 'Alerts', icon: BellIcon },
|
||||
{ id: 'machines', label: 'Machines', icon: '⌂' },
|
||||
{ id: 'users', label: 'Users', icon: '⚇' },
|
||||
{ id: 'widgets', label: 'Widgets', icon: '▦' },
|
||||
{ id: 'users', label: 'Users', icon: '⚇' },
|
||||
{ id: 'settings', label: 'Settings', icon: '⚙' },
|
||||
]
|
||||
|
||||
type View = 'terminal' | 'dashboard' | 'widgets' | 'docker' | 'alerts' | 'services' | 'cron' | 'updates' | 'disks' | 'space' | 'files' | 'procs' | 'machines' | 'users' | 'settings'
|
||||
const VIEWS: View[] = ['terminal', 'dashboard', 'widgets', 'docker', 'alerts', 'services', 'cron', 'updates', 'disks', 'space', 'files', 'procs', 'machines', 'users', 'settings']
|
||||
type View = 'terminal' | 'widgets' | 'docker' | 'alerts' | 'services' | 'jobs' | 'updates' | 'disks' | 'zfs' | 'usage' | 'files' | 'procs' | 'machines' | 'kvm' | 'users' | 'settings'
|
||||
const VIEWS: View[] = ['terminal', 'widgets', 'docker', 'alerts', 'services', 'jobs', 'updates', 'disks', 'zfs', 'usage', 'files', 'procs', 'machines', 'kvm', 'users', 'settings']
|
||||
const HUB_ONLY: View[] = ['machines', 'users', 'settings', 'alerts']
|
||||
|
||||
// URL hash is the source of truth for where you are: #/<machine|hub>/<view>. Back and
|
||||
// forward work, reload restores the tab, and links can point at a page on a machine.
|
||||
// Pages that were renamed, so a link or a bookmark made before it still lands where it meant.
|
||||
const RENAMED_VIEWS: Record<string, View> = { space: 'usage', cron: 'jobs', top: 'procs', dashboard: 'procs' }
|
||||
|
||||
function parseHash(): { machine: string | null; view: View } | null {
|
||||
const m = location.hash.match(/^#\/([a-z0-9-]+)\/([a-z]+)(?:\/.*)?$/) // anything after the page is that page's own selection
|
||||
if (!m || !VIEWS.includes(m[2] as View)) return null
|
||||
return { machine: m[1] === 'hub' ? null : m[1], view: m[2] as View }
|
||||
if (!m) return null
|
||||
const view = VIEWS.includes(m[2] as View) ? (m[2] as View) : RENAMED_VIEWS[m[2]]
|
||||
if (!view) return null
|
||||
return { machine: m[1] === 'hub' ? null : m[1], view }
|
||||
}
|
||||
|
||||
function fmtUptime(s: number): string {
|
||||
@@ -102,7 +109,7 @@ export default function Shell({ me: hubMe, onLoggedOut }: { me: Me; onLoggedOut:
|
||||
const listRate = useRefreshMs(LIST_FLOOR)
|
||||
const [terms, setTerms] = useState<Term[]>([])
|
||||
const [active, setActive] = useState<string | null>(null)
|
||||
const [view, setView] = useState<View>(() => parseHash()?.view ?? 'dashboard')
|
||||
const [view, setView] = useState<View>(() => parseHash()?.view ?? 'procs')
|
||||
const [drawer, setDrawer] = useState(false)
|
||||
const [online, setOnline] = useState(true)
|
||||
const [creating, setCreating] = useState(false)
|
||||
@@ -118,6 +125,20 @@ export default function Shell({ me: hubMe, onLoggedOut }: { me: Me; onLoggedOut:
|
||||
|
||||
useEffect(() => { reload() }, [reload])
|
||||
|
||||
// A shell opened inside a container from the Docker tab is a terminal like any other:
|
||||
// pick it up, show the Terminal tab, and make it the active one.
|
||||
useEffect(() => {
|
||||
const h = async (e: Event) => {
|
||||
const id = (e as CustomEvent<string>).detail
|
||||
if (!id) return
|
||||
await reload()
|
||||
setActive(id)
|
||||
setView('terminal')
|
||||
}
|
||||
window.addEventListener('hamlab:terminal', h)
|
||||
return () => window.removeEventListener('hamlab:terminal', h)
|
||||
}, [reload])
|
||||
|
||||
// Machine list for the switcher (hub-local), refreshed while the console is open.
|
||||
const loadMachines = useCallback(async () => { try { setMachines((await api.machines()).nodes) } catch { /* not a hub or offline */ } }, [])
|
||||
useEffect(() => { loadMachines(); const t = setInterval(loadMachines, listRate); return () => clearInterval(t) }, [loadMachines, listRate])
|
||||
@@ -142,7 +163,7 @@ export default function Shell({ me: hubMe, onLoggedOut }: { me: Me; onLoggedOut:
|
||||
// the machine is changing: the URL sets the machine and the page in one go, and deciding from
|
||||
// inside the machine change read the page we were leaving — so a link from Settings to a
|
||||
// machine's Disks arrived at its Dashboard.
|
||||
useEffect(() => { if (machine && HUB_ONLY.includes(view)) setView('dashboard') }, [machine, view])
|
||||
useEffect(() => { if (machine && HUB_ONLY.includes(view)) setView('procs') }, [machine, view])
|
||||
|
||||
useEffect(() => { const paint = () => applyAccent(machine ?? hubMe.host); paint(); window.addEventListener('hamlab:accents', paint); return () => window.removeEventListener('hamlab:accents', paint) }, [machine, hubMe.host])
|
||||
// The tab's title says where you are, so history entries and open tabs can be told apart.
|
||||
@@ -264,17 +285,18 @@ export default function Shell({ me: hubMe, onLoggedOut }: { me: Me; onLoggedOut:
|
||||
<main className="main" key={machine ?? 'hub'}>
|
||||
{view === 'settings' && <Settings me={hubMe} />}
|
||||
{view === 'users' && <Users me={hubMe} />}
|
||||
{view === 'space' && <Space key={machine ?? ''} />}
|
||||
{view === 'machines' && <Machines hub={hubMe} onChanged={loadMachines} onOpen={(n) => { switchMachine(n); setView('dashboard') }} />}
|
||||
{view === 'dashboard' && <Dashboard />}
|
||||
{view === 'kvm' && <KVM key={machine ?? 'hub'} machine={machine} title={title} />}
|
||||
{view === 'usage' && <Usage key={machine ?? ''} />}
|
||||
{view === 'machines' && <Machines hub={hubMe} onChanged={loadMachines} onOpen={(n) => { switchMachine(n); setView('procs') }} />}
|
||||
{view === 'widgets' && <Gallery />}
|
||||
{view === 'docker' && <Docker />}
|
||||
{view === 'alerts' && <Alerts machines={machines.map((m) => m.name)} titleOf={(n) => titleOf(n === hubMe.host ? null : n, machines, hubMe.host)} />}
|
||||
{view === 'procs' && <Top />}
|
||||
{view === 'services' && <Services />}
|
||||
{view === 'cron' && <Cron />}
|
||||
{view === 'jobs' && <Jobs />}
|
||||
{view === 'updates' && <Updates />}
|
||||
{view === 'disks' && <Disks />}
|
||||
{view === 'zfs' && <ZFS />}
|
||||
{view === 'files' && <Files onOpenInTerminal={runInTerminal} />}
|
||||
{termView && <TermTabs terms={ordered} active={active} creating={creating}
|
||||
onSelect={setActive} onNew={create} onRename={rename} onClose={close} onMove={moveTab}
|
||||
@@ -301,7 +323,7 @@ export default function Shell({ me: hubMe, onLoggedOut }: { me: Me; onLoggedOut:
|
||||
<span>{machine ? `${title} · ` : ''}{me.version.startsWith('v') ? me.version : 'v' + me.version}</span>
|
||||
<span>up {fmtUptime(me.uptime)}</span>
|
||||
<span className="spacer" />
|
||||
<span className="muted">{view === 'dashboard' ? 'Dashboard' : view === 'widgets' ? 'Widgets' : view === 'docker' ? 'Docker' : view === 'alerts' ? 'Alerts' : view === 'services' ? 'Services' : view === 'cron' ? 'Cron' : view === 'updates' ? 'Updates' : view === 'disks' ? 'Disks' : view === 'space' ? 'Space' : view === 'files' ? 'Files' : view === 'procs' ? 'Processes' : view === 'machines' ? 'Machines' : view === 'users' ? 'Users' : view === 'settings' ? 'Settings' : activeTerm ? activeTerm.name : ''}</span>
|
||||
<span className="muted">{view === 'widgets' ? 'Widgets' : view === 'docker' ? 'Docker' : view === 'alerts' ? 'Alerts' : view === 'services' ? 'Services' : view === 'jobs' ? 'Jobs' : view === 'updates' ? 'Updates' : view === 'disks' ? 'Disks' : view === 'zfs' ? 'ZFS' : view === 'usage' ? 'Usage' : view === 'files' ? 'Files' : view === 'procs' ? 'Processes' : view === 'machines' ? 'Machines' : view === 'kvm' ? 'KVM' : view === 'users' ? 'Users' : view === 'settings' ? 'Settings' : activeTerm ? activeTerm.name : ''}</span>
|
||||
</footer>
|
||||
</div>
|
||||
)
|
||||
|
||||
+56
-15
@@ -250,6 +250,20 @@ export default function Top() {
|
||||
|
||||
// the process table, and the CPU model, come from the same place the Processes page reads
|
||||
const [pt, setPt] = useState<ProcTable | null>(null)
|
||||
const [pickedIf, setPickedIf] = useState<string | null>(null) // which NIC the net box draws
|
||||
// Which pool each drive belongs to, asked once: a drive in a ZFS pool has no filesystem of
|
||||
// its own, and "nothing is on it" would be a lie about eight drives holding a NAS.
|
||||
const [pools, setPools] = useState<Record<string, string>>({})
|
||||
useEffect(() => {
|
||||
let live = true
|
||||
api.storage().then((r) => {
|
||||
if (!live) return
|
||||
const m: Record<string, string> = {}
|
||||
for (const d of r.devices) for (const part of d.parts) if (part.fstype === 'zfs_member' && part.label) m[d.name] = part.label
|
||||
setPools(m)
|
||||
}).catch(() => { /* the rows simply say less */ })
|
||||
return () => { live = false }
|
||||
}, [])
|
||||
const [err, setErr] = useState('')
|
||||
useEffect(() => {
|
||||
let stop = false
|
||||
@@ -319,8 +333,12 @@ export default function Top() {
|
||||
const free = (mem as { free?: number }).free ?? Math.max(0, mem.total - mem.used - cached)
|
||||
const swap = s?.swap ?? pt?.swap
|
||||
const net = (s?.net ?? pt?.net ?? []).filter((n) => n.up && n.iface !== 'lo')
|
||||
const iface = net[0]
|
||||
const fs = s?.fs ?? []
|
||||
// the one that has actually carried traffic, not whichever the kernel listed first — wg0
|
||||
// and a dormant wlan should not push the real NIC out of the way. Tapping a name switches.
|
||||
const busiest = net.length ? net.reduce((a, b) => (b.rx_total + b.tx_total > a.rx_total + a.tx_total ? b : a)) : undefined
|
||||
const iface = net.find((n) => n.iface === pickedIf) ?? busiest
|
||||
// the disks box is this machine's own storage: a CIFS or NFS share is somebody else's disk
|
||||
const fs = (s?.fs ?? []).filter((f) => !f.remote)
|
||||
const pct = (a: number, b: number) => (b > 0 ? (a / b) * 100 : 0)
|
||||
// the top of a graph's scale: the peak of what it is drawing — the window, not everything kept
|
||||
const peak = (id: string) => Math.max(1, ...(curves.get(id) ?? []).slice(-cap))
|
||||
@@ -378,18 +396,26 @@ export default function Top() {
|
||||
|
||||
<Box n={3} name="disks" className="top-disks" right="io">
|
||||
<div className="t-disks">
|
||||
{swap && swap.total > 0 && <Disk name="swap" size={swap.total} used={swap.used} />}
|
||||
{fs.map((f) => {
|
||||
// which drive is behind a mount isn't in the sample, so only the root filesystem
|
||||
// gets an IO line, against the machine's largest drive — which is the one it is on
|
||||
const d = f.mount === '/' ? [...(s?.disks ?? [])].sort((a, b) => b.size - a.size)[0] : undefined
|
||||
return <Disk key={f.mount} name={f.mount === '/' ? 'root' : f.mount.split('/').filter(Boolean).pop() ?? f.mount} size={f.size} used={f.used} io={d ? d.util_pct : undefined} />
|
||||
{(s?.disks ?? []).map((d) => {
|
||||
// what is on this drive: the filesystems the sampler could attribute to it. A ZFS
|
||||
// pool or an md array spans several drives, so nothing is attributed and the row
|
||||
// shows what is true of the hardware — how busy it is, and how hot.
|
||||
const on = fs.filter((f) => f.disk === d.name)
|
||||
const used = on.reduce((a, f) => a + f.used, 0)
|
||||
return <Disk key={d.name} name={d.name} note={d.model} size={d.size} used={on.length ? used : undefined} io={d.util_pct} temp={d.temp} mounts={on.map((f) => f.mount)} pool={pools[d.name]} />
|
||||
})}
|
||||
{fs.length === 0 && <div className="t-dim">no filesystems reported</div>}
|
||||
{(s?.disks ?? []).length === 0 && <div className="t-dim">no drives reported</div>}
|
||||
</div>
|
||||
</Box>
|
||||
|
||||
<Box n={4} name="net" className="top-net" right={iface ? <span className="t-if">{iface.iface}</span> : undefined}>
|
||||
<Box n={4} name="net" className="top-net" right={net.length ? (
|
||||
<span className="t-ifs">
|
||||
{net.map((n) => (
|
||||
<button key={n.iface} className={'t-if' + (iface && n.iface === iface.iface ? ' on' : '')}
|
||||
title={`${n.iface} — ${bytes(n.rx_total)} in, ${bytes(n.tx_total)} out`} onClick={() => setPickedIf(n.iface)}>{n.iface}</button>
|
||||
))}
|
||||
</span>
|
||||
) : undefined}>
|
||||
{iface ? (
|
||||
<div className="t-net">
|
||||
<div className="t-net-graphs">
|
||||
@@ -470,14 +496,29 @@ const Rows = memo(function Rows({ procs, sel, onSel, curves, empty }: { procs: P
|
||||
)
|
||||
})
|
||||
|
||||
function Disk({ name, size, used, io }: { name: string; size: number; used: number; io?: number }) {
|
||||
const upct = size > 0 ? (used / size) * 100 : 0
|
||||
// One physical drive. `used` is the bytes of the filesystems on it, left out when none can
|
||||
// be attributed to it — a pool member's share of a pool is not a number worth inventing.
|
||||
function Disk({ name, note, size, used, io, temp, mounts, pool }: { name: string; note?: string; size: number; used?: number; io?: number; temp?: number; mounts?: string[]; pool?: string }) {
|
||||
const upct = used !== undefined && size > 0 ? (used / size) * 100 : 0
|
||||
return (
|
||||
<div className="t-disk">
|
||||
<div className="t-row"><b>{name}</b><span className="spacer" />{bytes(size)}</div>
|
||||
<div className="t-row">
|
||||
<b>{name}</b>
|
||||
{note && <span className="t-dim t-model" title={note}>{note}</span>}
|
||||
<span className="spacer" />
|
||||
{temp !== undefined && temp > 0 && <span className="t-temp">{Math.round(temp)}°C</span>}
|
||||
{bytes(size)}
|
||||
</div>
|
||||
{io !== undefined && <div className="t-row t-sub"><span>IO%</span><Meter pct={io} grad="io" /></div>}
|
||||
<div className="t-row t-sub"><span>Used:</span><span className="t-pct">{Math.round(upct)}%</span><Meter pct={upct} grad="used" /><span className="t-amt">{bytes(used, 1)}</span></div>
|
||||
<div className="t-row t-sub"><span>Free:</span><span className="t-pct">{Math.round(100 - upct)}%</span><Meter pct={100 - upct} grad="free" /><span className="t-amt">{bytes(Math.max(0, size - used), 1)}</span></div>
|
||||
{used !== undefined ? (
|
||||
<>
|
||||
<div className="t-row t-sub"><span>Used:</span><span className="t-pct">{Math.round(upct)}%</span><Meter pct={upct} grad="used" /><span className="t-amt">{bytes(used, 1)}</span></div>
|
||||
<div className="t-row t-sub"><span>Free:</span><span className="t-pct">{Math.round(100 - upct)}%</span><Meter pct={100 - upct} grad="free" /><span className="t-amt">{bytes(Math.max(0, size - used), 1)}</span></div>
|
||||
{mounts && mounts.length > 0 && <div className="t-row t-sub t-dim t-mounts" title={mounts.join(' ')}>{mounts.join(' ')}</div>}
|
||||
</>
|
||||
) : (
|
||||
<div className="t-row t-sub t-dim">{pool ? `member of ZFS pool ${pool}` : 'nothing mounted from it'}</div>
|
||||
)}
|
||||
</div>
|
||||
)
|
||||
}
|
||||
|
||||
@@ -1,10 +1,11 @@
|
||||
// Space: what is using a disk, WizTree-style. Pick a disk, scan it (a parallel walk of every
|
||||
// Usage: what is using a disk, WizTree-style. (The tab was called Space until 2026-09-24;
|
||||
// the Go package and the /api/space endpoints behind it still carry the old name.) Pick a disk, scan it (a parallel walk of every
|
||||
// filesystem the disk backs, as root when the host allows), then read the result two ways:
|
||||
// a tree of folders largest-first, and a treemap of whatever folder is selected, where every
|
||||
// file is a tile sized by what it takes on disk and coloured by what kind of thing it is.
|
||||
import { useCallback, useEffect, useMemo, useRef, useState } from 'react'
|
||||
import { api, SpaceFile, SpaceTarget, SpaceTree } from './api'
|
||||
import { fmtBytes } from './Dashboard'
|
||||
import { fmtBytes } from './fmt'
|
||||
import { useHashSel } from './hashsel'
|
||||
import { useRefreshMs } from './prefs'
|
||||
import { Btn, Loading, Spinner, askYes, toast, useEscape } from './ui'
|
||||
@@ -45,10 +46,10 @@ const ago = (iso?: string) => {
|
||||
const fmtDur = (a: string, b?: string) => { const ms = (b ? new Date(b).getTime() : Date.now()) - new Date(a).getTime(); return ms < 1000 ? `${ms} ms` : ms < 60000 ? `${(ms / 1000).toFixed(1)} s` : `${Math.floor(ms / 60000)} min ${Math.round((ms % 60000) / 1000)} s` }
|
||||
const parentOf = (p: string) => (p === '/' ? '/' : p.slice(0, p.lastIndexOf('/')) || '/')
|
||||
|
||||
export default function Space() {
|
||||
export default function Usage() {
|
||||
const [targets, setTargets] = useState<SpaceTarget[] | null>(null)
|
||||
const [err, setErr] = useState('')
|
||||
const [sel, setSel] = useHashSel('space')
|
||||
const [sel, setSel] = useHashSel('usage')
|
||||
const [tab, setTab] = useState<'tree' | 'files'>('tree')
|
||||
const rate = useRefreshMs()
|
||||
|
||||
+359
@@ -0,0 +1,359 @@
|
||||
// ZFS: pools, datasets, snapshots, and a schedule for taking and expiring them.
|
||||
//
|
||||
// Split out of the Disks tab on 2026-09-24. Disks is the hardware; this is the filesystem
|
||||
// on top of it, which has enough of its own to say.
|
||||
//
|
||||
// On the schedule: `grid` has run zfSnap from root cron since before Ham Lab existed, and it
|
||||
// works. So this reads and writes the same snapshot names — `2026-09-24_02.30.01--6w`, with
|
||||
// the retention in the name — rather than inventing a second convention for the two tools to
|
||||
// disagree over. An existing cron job is detected and shown, so the page tells the truth
|
||||
// about the machine whether Ham Lab or something else is doing the work.
|
||||
import { FormEvent, useCallback, useEffect, useState } from 'react'
|
||||
import { api, ZFSExternal, ZFSInfo, ZFSOverview, ZFSPolicy, ZFSRun, ZFSSnapshot } from './api'
|
||||
import { fmtBytes } from './fmt'
|
||||
import { useRefreshMs } from './prefs'
|
||||
import { askText, askYes, Btn, Loading, Meter, Spinner, toast, useEscape } from './ui'
|
||||
|
||||
const ago = (t?: string | number) => {
|
||||
if (!t) return 'never'
|
||||
const s = Math.max(0, (Date.now() - new Date(t).getTime()) / 1000)
|
||||
return s < 60 ? 'just now' : s < 3600 ? `${Math.floor(s / 60)} min ago` : s < 86400 ? `${Math.floor(s / 3600)} h ago` : `${Math.floor(s / 86400)} d ago`
|
||||
}
|
||||
const soon = (t?: string) => {
|
||||
if (!t) return '—'
|
||||
const s = (new Date(t).getTime() - Date.now()) / 1000
|
||||
return s < 60 ? 'any moment' : s < 3600 ? `in ${Math.round(s / 60)} min` : s < 86400 ? `in ${Math.round(s / 3600)} h` : `in ${Math.round(s / 86400)} d`
|
||||
}
|
||||
const WEEKDAYS = ['Sunday', 'Monday', 'Tuesday', 'Wednesday', 'Thursday', 'Friday', 'Saturday']
|
||||
const describe = (p: ZFSPolicy) => {
|
||||
const when = p.every === 'custom' ? `on ${p.cron}`
|
||||
: p.every === 'hourly' ? `every hour at :${(p.at.split(':')[1] ?? '00').padStart(2, '0')}`
|
||||
: p.every === 'daily' ? `every day at ${p.at}`
|
||||
: p.every === 'weekly' ? `every ${WEEKDAYS[p.weekday] ?? 'Sunday'} at ${p.at}`
|
||||
: `on day ${p.day} of the month at ${p.at}`
|
||||
return `${when} · keep ${p.keep_for}${p.recursive ? ' · recursive' : ''}`
|
||||
}
|
||||
|
||||
// ---- pools and datasets (moved from the Disks tab) ----
|
||||
function PoolCard({ p, onChanged }: { p: ZFSInfo['pools'][number]; onChanged: () => void }) {
|
||||
const scrubbing = /scrub in progress/i.test(p.scan ?? '')
|
||||
const paused = /scrub paused/i.test(p.scan ?? '')
|
||||
const [cfg, setCfg] = useState(false)
|
||||
const act = (a: 'start' | 'pause' | 'stop') => async () => {
|
||||
try { await api.zfsScrub(p.name, a); toast(`scrub ${a}: ok`); onChanged() } catch (e) { toast(e instanceof Error ? e.message : 'failed', 'err') }
|
||||
}
|
||||
return (
|
||||
<section className="card">
|
||||
<div className="sub-head">
|
||||
<span className={'dot ' + (p.health === 'ONLINE' ? 'ok' : p.health === 'DEGRADED' ? 'warn' : 'bad')} />
|
||||
<span className="dname">{p.name}</span>
|
||||
<span className="badge">{p.health}</span>
|
||||
<span className="spacer" />
|
||||
<span className="num">{fmtBytes(p.alloc)} / {fmtBytes(p.size)} · {p.cap}%</span>
|
||||
</div>
|
||||
<Meter frac={p.cap / 100} cls={p.cap >= 90 ? 'bad' : p.cap >= 80 ? 'warn' : ''} />
|
||||
<div className="props" style={{ marginTop: 8 }}>
|
||||
<div><span className="k">Free</span><span>{fmtBytes(p.free)} · fragmentation {p.frag}% · dedup {p.dedup}</span></div>
|
||||
{p.scan && <div><span className="k">Scan</span><span style={{ whiteSpace: 'normal' }}>{p.scan}</span></div>}
|
||||
{p.errors && <div><span className="k">Errors</span><span className={p.errors.startsWith('No known') ? '' : 'warn-text'}>{p.errors}</span></div>}
|
||||
</div>
|
||||
<div className="svc-actions">
|
||||
{!scrubbing && <Btn className="btn" confirm={`Start a scrub of ${p.name}? It reads the whole pool and can take many hours.`} onClick={act('start')}>{paused ? 'Resume scrub' : 'Scrub'}</Btn>}
|
||||
{scrubbing && <Btn className="btn" onClick={act('pause')}>Pause scrub</Btn>}
|
||||
{(scrubbing || paused) && <Btn className="btn" confirm="Stop the scrub?" onClick={act('stop')}>Stop scrub</Btn>}
|
||||
<button className="btn" onClick={() => setCfg(!cfg)}>{cfg ? 'Hide vdevs' : 'Vdevs'}</button>
|
||||
</div>
|
||||
{cfg && <pre className="unit-file">{p.config}</pre>}
|
||||
</section>
|
||||
)
|
||||
}
|
||||
|
||||
// A snapshot's name carries when it was taken and how long it lives — 2026-09-24_02.30.01--6w
|
||||
// — so the explorer can say when each one goes without asking the server anything.
|
||||
function snapMeta(full: string): { taken?: Date; expires?: Date } {
|
||||
const snap = full.split('@')[1] ?? ''
|
||||
const [body, ttl] = snap.split('--')
|
||||
const m = /(\d{4})-(\d{2})-(\d{2})_(\d{2})\.(\d{2})\.(\d{2})$/.exec(body)
|
||||
if (!m) return {}
|
||||
const taken = new Date(+m[1], +m[2] - 1, +m[3], +m[4], +m[5], +m[6])
|
||||
const a = /^(\d+)([mhdw])$/.exec(ttl ?? '')
|
||||
if (!a) return { taken }
|
||||
const ms = { m: 60e3, h: 3600e3, d: 86400e3, w: 7 * 86400e3 }[a[2]]!
|
||||
return { taken, expires: new Date(taken.getTime() + +a[1] * ms) }
|
||||
}
|
||||
|
||||
// One dataset in the snapshot explorer: how many it has and what they cost, opening into
|
||||
// the snapshots themselves, each of which can be taken now or deleted.
|
||||
function DatasetRow({ d, onChanged }: { d: ZFSInfo['datasets'][number]; onChanged: () => void }) {
|
||||
const [snaps, setSnaps] = useState<ZFSSnapshot[] | null>(null)
|
||||
const [open, setOpen] = useState(false)
|
||||
const reload = useCallback(async () => {
|
||||
try { setSnaps(await api.zfsSnapshots(d.name)) } catch (e) { toast(e instanceof Error ? e.message : 'failed', 'err'); setSnaps([]) }
|
||||
}, [d.name])
|
||||
async function toggle() {
|
||||
setOpen(!open)
|
||||
if (!open && !snaps) await reload()
|
||||
}
|
||||
async function snapshot() {
|
||||
const def = new Date().toISOString().slice(0, 19).replace('T', '_').replace(/:/g, '.')
|
||||
const name = await askText(`Snapshot name for ${d.name}@`, def)
|
||||
if (!name) return
|
||||
try { await api.zfsSnapshot(d.name, name.trim()); toast(`${d.name}@${name.trim()} created`); await reload(); onChanged() }
|
||||
catch (e) { toast(e instanceof Error ? e.message : 'failed', 'err') }
|
||||
}
|
||||
async function destroy(s: ZFSSnapshot) {
|
||||
const short = s.name.split('@')[1]
|
||||
if (!await askYes(`Delete ${s.name}? It is removed from this dataset and from every dataset beneath it that has a snapshot of the same name. This cannot be undone.`, 'Delete')) return
|
||||
try { await api.deleteZfsSnapshot(s.name, true); toast(`${short} deleted`); await reload(); onChanged() }
|
||||
catch (e) { toast(e instanceof Error ? e.message : 'failed', 'err') }
|
||||
}
|
||||
const depth = d.name.split('/').length - 1
|
||||
return (
|
||||
<>
|
||||
<div className="ds-row" style={{ paddingLeft: 12 + depth * 16 }}>
|
||||
<span className="ds-name" onClick={toggle}><span className="chev">{open ? '▾' : '▸'}</span>{d.name.split('/').pop()}{d.type === 'volume' && <span className="badge">zvol</span>}</span>
|
||||
<span className="num">{d.snapshots}</span>
|
||||
<span className="num muted">{d.used_by_snapshots ? fmtBytes(d.used_by_snapshots) : '—'}</span>
|
||||
<span className="num muted">{fmtBytes(d.used)}</span>
|
||||
<span className="num muted">{fmtBytes(d.refer)}</span>
|
||||
<span className="ds-mount muted">{d.mount}</span>
|
||||
</div>
|
||||
{open && (
|
||||
<div className="ds-snaps">
|
||||
<div className="svc-actions"><Btn className="btn" onClick={snapshot}>Snapshot now</Btn><span className="muted small">{d.snapshots} snapshot{d.snapshots === 1 ? '' : 's'} · {fmtBytes(d.avail)} free on this dataset</span></div>
|
||||
{snaps === null ? <div className="muted"><Spinner /> listing…</div> : snaps.length === 0 ? <div className="muted">No snapshots.</div> : (
|
||||
<div className="snap-list">{snaps.slice().reverse().map((s) => {
|
||||
const { taken, expires } = snapMeta(s.name)
|
||||
const gone = expires && expires.getTime() < Date.now()
|
||||
return (
|
||||
<div key={s.name} className="snap">
|
||||
<span className="mono snap-name" title={s.name}>{s.name.split('@')[1]}</span>
|
||||
<span className="muted snap-when">{taken ? taken.toLocaleString() : new Date(s.created).toLocaleString()}</span>
|
||||
<span className={'muted snap-exp' + (gone ? ' warn' : '')}>{expires ? (gone ? 'expired' : `expires ${expires.toLocaleDateString()}`) : 'kept'}</span>
|
||||
<span className="num muted">{fmtBytes(s.used)}</span>
|
||||
<Btn className="btn danger snap-del" onClick={() => destroy(s)}>Delete</Btn>
|
||||
</div>
|
||||
)
|
||||
})}</div>
|
||||
)}
|
||||
</div>
|
||||
)}
|
||||
</>
|
||||
)
|
||||
}
|
||||
|
||||
// ---- schedules ----
|
||||
|
||||
function ExternalNotice({ x, mine, onAdopt }: { x: ExternalJob; mine: number; onAdopt: () => void }) {
|
||||
return (
|
||||
<div className={'notice' + (mine > 0 ? ' attn' : '')} style={{ marginBottom: 8 }}>
|
||||
<b>{x.tool}</b> is already taking snapshots on this machine, from cron as <b>{x.user}</b>:
|
||||
<div className="mono small" style={{ margin: '4px 0', wordBreak: 'break-all' }}>{x.schedule} {x.command}</div>
|
||||
<span className="muted">in {x.source}{x.last ? ` · last ran ${ago(x.last)}` : ''}</span>
|
||||
{mine > 0
|
||||
? <div className="warn" style={{ marginTop: 6 }}>A Ham Lab schedule is running as well. Both use the same names and both only remove snapshots that have passed the expiry written into them, so they will not fight — but they will take twice as many. Keep one.</div>
|
||||
: <div className="svc-actions" style={{ marginTop: 6 }}>
|
||||
<Btn className="btn" onClick={onAdopt}>Adopt as a schedule</Btn>
|
||||
<span className="muted small">Copies it into a Ham Lab schedule you can edit here. Turn the old cron job off on the Jobs tab once you are happy with it.</span>
|
||||
</div>}
|
||||
</div>
|
||||
)
|
||||
}
|
||||
type ExternalJob = ZFSExternal
|
||||
|
||||
function ScheduleCard({ p, cron, next, last, onChanged, onEdit }: { p: ZFSPolicy; cron: string; next?: string; last: ZFSRun; onChanged: () => void; onEdit: () => void }) {
|
||||
const [busy, setBusy] = useState('')
|
||||
const run = async () => {
|
||||
if (!await askYes(`Take a snapshot of ${p.dataset} now? Snapshots past their retention are removed at the same time, as they would be on the schedule.`, 'Run now')) return
|
||||
setBusy('run')
|
||||
try { const r = await api.runZfsSchedule(p.id); toast(`${r.taken}${r.pruned.length ? ` · ${r.pruned.length} expired removed` : ''}`); onChanged() }
|
||||
catch (e) { toast(e instanceof Error ? e.message : 'failed', 'err') } finally { setBusy('') }
|
||||
}
|
||||
const remove = async () => {
|
||||
if (!await askYes(`Remove the schedule "${p.name}"? The snapshots it already took are left exactly as they are.`, 'Remove')) return
|
||||
try { await api.deleteZfsSchedule(p.id); toast('schedule removed'); onChanged() } catch (e) { toast(e instanceof Error ? e.message : 'failed', 'err') }
|
||||
}
|
||||
const toggle = async () => {
|
||||
try { await api.saveZfsSchedule({ ...p, enabled: !p.enabled }); onChanged() } catch (e) { toast(e instanceof Error ? e.message : 'failed', 'err') }
|
||||
}
|
||||
return (
|
||||
<section className="card">
|
||||
<div className="sub-head">
|
||||
<span className={'dot ' + (last.error ? 'bad' : p.enabled ? 'ok' : '')} />
|
||||
<span className="dname">{p.name || p.dataset}</span>
|
||||
{!p.enabled && <span className="badge">paused</span>}
|
||||
<span className="spacer" />
|
||||
<span className="muted small">{p.enabled ? soon(next) : 'not scheduled'}</span>
|
||||
</div>
|
||||
<div className="props" style={{ marginTop: 6 }}>
|
||||
<div><span className="k">Dataset</span><span className="mono">{p.dataset}</span></div>
|
||||
<div><span className="k">When</span><span>{describe(p)}</span></div>
|
||||
<div><span className="k">Cron</span><span className="mono small" title={`in ${'/etc/cron.d/hamlab-zfs'}`}>{cron}</span></div>
|
||||
<div><span className="k">Last run</span><span>{ago(last.at)}{last.pruned?.length ? ` · ${last.pruned.length} expired removed` : ''}</span></div>
|
||||
{last.taken && <div><span className="k">Last snapshot</span><span className="mono small">{last.taken.split('@')[1]}</span></div>}
|
||||
{p.keep_min > 0 && <div><span className="k">Floor</span><span>always keep the newest {p.keep_min}</span></div>}
|
||||
</div>
|
||||
{last.error && <div className="error" style={{ marginTop: 8 }}>{last.error}</div>}
|
||||
<div className="svc-actions">
|
||||
<Btn className="btn on" busy={busy === 'run'} onClick={run}>Run now</Btn>
|
||||
<button className="btn" onClick={onEdit}>Edit</button>
|
||||
<Btn className="btn" onClick={toggle}>{p.enabled ? 'Pause' : 'Resume'}</Btn>
|
||||
<Btn className="btn danger" onClick={remove}>Remove</Btn>
|
||||
</div>
|
||||
</section>
|
||||
)
|
||||
}
|
||||
|
||||
const BLANK: Partial<ZFSPolicy> = { name: '', dataset: '', recursive: true, every: 'daily', at: '02:30', weekday: 0, day: 1, keep_for: '6w', keep_min: 1, enabled: true }
|
||||
|
||||
function ScheduleDialog({ p, datasets, onClose, onSaved }: { p: Partial<ZFSPolicy>; datasets: string[]; onClose: () => void; onSaved: () => void }) {
|
||||
const [f, setF] = useState<Partial<ZFSPolicy>>({ ...BLANK, ...p })
|
||||
const [err, setErr] = useState('')
|
||||
useEscape(onClose)
|
||||
const set = (k: keyof ZFSPolicy, v: unknown) => setF((x) => ({ ...x, [k]: v }))
|
||||
const save = async (e: FormEvent) => {
|
||||
e.preventDefault(); setErr('')
|
||||
try { await api.saveZfsSchedule({ ...f, name: (f.name || '').trim() || f.dataset }); onSaved() }
|
||||
catch (e) { setErr(e instanceof Error ? e.message : 'failed') }
|
||||
}
|
||||
return (
|
||||
<div className="modal-back" onClick={onClose}>
|
||||
<form className="modal" onClick={(e) => e.stopPropagation()} onSubmit={save}>
|
||||
<h3>{f.id ? 'Edit schedule' : 'New snapshot schedule'}</h3>
|
||||
<label className="muted">Dataset
|
||||
<select value={f.dataset} onChange={(e) => set('dataset', e.target.value)} required>
|
||||
<option value="">choose…</option>
|
||||
{datasets.map((d) => <option key={d} value={d}>{d}</option>)}
|
||||
</select>
|
||||
</label>
|
||||
<label className="muted">Name — what this schedule is for
|
||||
<input value={f.name} onChange={(e) => set('name', e.target.value)} placeholder={f.dataset || 'nightly'} maxLength={64} />
|
||||
</label>
|
||||
<label className="muted">How often
|
||||
<span className="chips">
|
||||
{(['hourly', 'daily', 'weekly', 'monthly', 'custom'] as const).map((k) => (
|
||||
<button type="button" key={k} className={'chip' + (f.every === k ? ' on' : '')} onClick={() => set('every', k)}>{k}</button>
|
||||
))}
|
||||
</span>
|
||||
</label>
|
||||
{f.every === 'custom' ? (
|
||||
<label className="muted">Cron expression — minute hour day month weekday
|
||||
<input className="mono" value={f.cron ?? ''} onChange={(e) => set('cron', e.target.value)} placeholder="0 */6 * * *" spellCheck={false} required />
|
||||
</label>
|
||||
) : (
|
||||
<label className="muted">At
|
||||
<input type="time" value={f.at} onChange={(e) => set('at', e.target.value)} />
|
||||
</label>
|
||||
)}
|
||||
{f.every === 'weekly' && (
|
||||
<label className="muted">On
|
||||
<select value={f.weekday} onChange={(e) => set('weekday', Number(e.target.value))}>
|
||||
{WEEKDAYS.map((d, i) => <option key={d} value={i}>{d}</option>)}
|
||||
</select>
|
||||
</label>
|
||||
)}
|
||||
{f.every === 'monthly' && (
|
||||
<label className="muted">Day of the month — 1 to 28, so every month has one
|
||||
<input type="number" min={1} max={28} value={f.day} onChange={(e) => set('day', Number(e.target.value))} />
|
||||
</label>
|
||||
)}
|
||||
<label className="muted">Snapshot name prefix — optional; the timestamp and retention follow it
|
||||
<input value={f.prefix ?? ''} onChange={(e) => set('prefix', e.target.value)} placeholder="none" spellCheck={false} />
|
||||
</label>
|
||||
<label className="muted">Keep for — written into each snapshot's name, as zfSnap does it: 6w, 30d, 12h
|
||||
<input value={f.keep_for} onChange={(e) => set('keep_for', e.target.value)} placeholder="6w" spellCheck={false} required />
|
||||
</label>
|
||||
<label className="muted">Always keep the newest — a floor, whatever their age
|
||||
<input type="number" min={0} max={99} value={f.keep_min} onChange={(e) => set('keep_min', Number(e.target.value))} />
|
||||
</label>
|
||||
<label className="muted" style={{ flexDirection: 'row', alignItems: 'center', gap: 8 }}>
|
||||
<input type="checkbox" checked={!!f.recursive} onChange={(e) => set('recursive', e.target.checked)} style={{ width: 'auto' }} />
|
||||
Recursive — one atomic snapshot of this dataset and everything under it
|
||||
</label>
|
||||
<label className="muted" style={{ flexDirection: 'row', alignItems: 'center', gap: 8 }}>
|
||||
<input type="checkbox" checked={!!f.enabled} onChange={(e) => set('enabled', e.target.checked)} style={{ width: 'auto' }} />
|
||||
Enabled
|
||||
</label>
|
||||
{err && <div className="error">{err}</div>}
|
||||
<div className="svc-actions">
|
||||
<button type="submit" className="btn on" disabled={!f.dataset || !f.keep_for}>Save</button>
|
||||
<span className="spacer" />
|
||||
<button type="button" className="btn" onClick={onClose}>Cancel</button>
|
||||
</div>
|
||||
</form>
|
||||
</div>
|
||||
)
|
||||
}
|
||||
|
||||
// adopt reads an existing cron job into the fields of a new schedule, so replacing it is a
|
||||
// matter of checking the numbers rather than retyping them. zfSnap's own flags are read
|
||||
// where they can be: -a <age> is the retention, -r is recursive.
|
||||
function adopt(x: ExternalJob, fallbackDataset: string): Partial<ZFSPolicy> {
|
||||
const cmd = x.command
|
||||
const keep = /-a\s+(\d+[mhdwMy])/.exec(cmd)?.[1] ?? /--\s*(\d+[mhdw])\b/.exec(cmd)?.[1] ?? '6w'
|
||||
const f = x.schedule.trim().split(/\s+/)
|
||||
const min = Number(f[0]), hour = Number(f[1])
|
||||
const simple = f.length === 5 && Number.isInteger(min) && Number.isInteger(hour) && f[2] === '*' && f[3] === '*' && f[4] === '*'
|
||||
const ds = /\b(tank[\w/-]*)\b/.exec(cmd)?.[1] ?? fallbackDataset
|
||||
return {
|
||||
...BLANK, name: `${x.tool} (adopted)`, dataset: ds,
|
||||
recursive: /\s-\w*r/.test(cmd) || true,
|
||||
every: simple ? 'daily' : 'custom',
|
||||
at: simple ? `${String(hour).padStart(2, '0')}:${String(min).padStart(2, '0')}` : '02:30',
|
||||
cron: simple ? '' : x.schedule.trim(),
|
||||
keep_for: keep, keep_min: 1, enabled: true,
|
||||
}
|
||||
}
|
||||
|
||||
// ---- page ----
|
||||
|
||||
export default function ZFS() {
|
||||
const [ov, setOv] = useState<ZFSOverview | null>(null)
|
||||
const [err, setErr] = useState('')
|
||||
const [editing, setEditing] = useState<Partial<ZFSPolicy> | null>(null)
|
||||
const rate = useRefreshMs()
|
||||
|
||||
const load = useCallback(async () => {
|
||||
try { setOv(await api.zfsOverview()); setErr('') } catch (e) { setErr(e instanceof Error ? e.message : 'failed') }
|
||||
}, [])
|
||||
useEffect(() => { load() }, [load])
|
||||
useEffect(() => { const t = setInterval(load, Math.max(5000, rate * 10)); return () => clearInterval(t) }, [load, rate])
|
||||
|
||||
if (err && !ov) return <div className="dash"><div className="error">{err}</div></div>
|
||||
if (!ov) return <div className="dash"><Loading label="Reading ZFS…" /></div>
|
||||
const zfs: ZFSInfo = ov.zfs
|
||||
const datasets = zfs.datasets.filter((d) => d.type !== 'volume').map((d) => d.name)
|
||||
const mine = ov.policies.filter((x) => x.policy.enabled).length
|
||||
|
||||
if (!zfs.available) return <div className="dash"><div className="notice">{zfs.message || 'No ZFS on this host.'}</div></div>
|
||||
|
||||
return (
|
||||
<div className="dash">
|
||||
<div className="dash-bar"><h2 className="section">Pools</h2></div>
|
||||
{zfs.pools.length === 0 && <div className="notice">ZFS is present but no pools are imported.</div>}
|
||||
<div className="cards">{zfs.pools.map((p) => <PoolCard key={p.name} p={p} onChanged={load} />)}</div>
|
||||
|
||||
<div className="dash-bar" style={{ marginTop: 12 }}><h2 className="section">Snapshot schedules</h2><span className="spacer" />
|
||||
<Btn className="btn on" onClick={() => setEditing({ ...BLANK, dataset: zfs.pools[0]?.name ?? '' })}>+ New schedule</Btn>
|
||||
</div>
|
||||
{ov.external && <ExternalNotice x={ov.external} mine={mine} onAdopt={() => setEditing(adopt(ov.external!, zfs.pools[0]?.name ?? ''))} />}
|
||||
{ov.policies.length === 0 && !ov.external && <div className="notice">Nothing is taking snapshots on a schedule. Snapshots are written the way zfSnap writes them, with the retention in the name, so anything already here is understood.</div>}
|
||||
<div className="cards">{ov.policies.map(({ policy, cron, next, last }) => (
|
||||
<ScheduleCard key={policy.id} p={policy} cron={cron} next={next} last={last ?? {}} onChanged={load} onEdit={() => setEditing(policy)} />
|
||||
))}</div>
|
||||
|
||||
<div className="dash-bar" style={{ marginTop: 12 }}><h2 className="section">Snapshots</h2><span className="spacer" />
|
||||
<span className="muted small">{zfs.datasets.reduce((a, d) => a + d.snapshots, 0)} across {zfs.datasets.length} datasets · open one to browse, take or delete</span>
|
||||
</div>
|
||||
<section className="card ds-table">
|
||||
<div className="ds-row head">
|
||||
<span>Dataset</span><span className="num">Snapshots</span><span className="num">They use</span>
|
||||
<span className="num">Dataset uses</span><span className="num">Referenced</span><span>Mount</span>
|
||||
</div>
|
||||
{zfs.datasets.map((d) => <DatasetRow key={d.name} d={d} onChanged={load} />)}
|
||||
</section>
|
||||
|
||||
{editing && <ScheduleDialog p={editing} datasets={datasets.length ? datasets : zfs.pools.map((p) => p.name)} onClose={() => setEditing(null)} onSaved={() => { setEditing(null); load() }} />}
|
||||
</div>
|
||||
)
|
||||
}
|
||||
+36
-4
@@ -70,7 +70,7 @@ export interface Temp { chip: string; label: string; c: number; max?: number; cr
|
||||
export interface Fan { chip: string; label: string; rpm: number }
|
||||
export interface NetIf { iface: string; up: boolean; rx_bps: number; tx_bps: number; rx_total: number; tx_total: number }
|
||||
export interface Disk { name: string; model?: string; size: number; read_bps: number; write_bps: number; util_pct: number; temp?: number }
|
||||
export interface FS { mount: string; type: string; size: number; used: number }
|
||||
export interface FS { mount: string; type: string; size: number; used: number; remote?: boolean; disk?: string }
|
||||
export interface UPS { driver: string; model: string; state: string; source: string; on_battery: boolean; in_v: number; out_v: number; battery_pct: number; runtime_min: number; load_w: number; load_pct: number; last_event?: string }
|
||||
export interface Proc { pid: number; ppid: number; name: string; cmd?: string; user: string; cpu: number; cpu_time: number; rss: number; threads: number; state: string; nice: number; started: number }
|
||||
export interface MemInfo { total: number; used: number; avail: number; free?: number; cached?: number; buffers?: number }
|
||||
@@ -90,7 +90,6 @@ export interface Series {
|
||||
// Dashboard layout. `order` lists module ids top-to-bottom; `hidden` are modules switched
|
||||
// off; `hiddenItems` are individual rows ("net:wlp1s0", "fs:/boot/efi", "disk:sda", "temp:chip/label").
|
||||
export interface HistoryWindow { seconds: number; points: number; stride_sec: number; min_seconds: number; max_seconds: number }
|
||||
export interface DashPrefs { order?: string[]; hidden?: string[]; hiddenItems?: string[]; intervalMs?: number; widgets?: { id: string; type: 'stat' | 'gauge' | 'bar' | 'spark' | 'list'; sensor: string; title?: string }[] }
|
||||
|
||||
export interface ServiceUnit { name: string; description: string; load: string; active: string; sub: string; enabled: string; failed: boolean }
|
||||
export interface ServiceDetail extends ServiceUnit { props: Record<string, string> }
|
||||
@@ -110,6 +109,14 @@ export interface StorageDevice { name: string; kind: string; model?: string; ven
|
||||
export interface SmartAttr { id: number; name: string; value: number; worst: number; thresh: number; raw: string; failed?: string; warn: boolean }
|
||||
export interface SmartInfo { device: string; available: boolean; supported: boolean; standby: boolean; passed: boolean | null; temp: number; power_on_hours: number; power_cycles: number; model?: string; serial?: string; firmware?: string; attrs: SmartAttr[]; nvme?: { critical_warning: number; percentage_used: number; available_spare: number; spare_threshold: number; media_errors: number; error_log_entries: number; data_read_bytes: number; data_written_bytes: number; unsafe_shutdowns: number }; mmc?: { life_a: number; life_b: number; pre_eol: number }; self_test?: string; self_test_remaining?: number; self_test_supported: boolean; self_test_note?: string; warnings: string[]; error?: string; fetched_at: number }
|
||||
export interface DockerContainer { id: string; name: string; image: string; state: string; status: string; health: string; ports: string; stack: string; service: string; created: string; mem: number }
|
||||
export interface UpdateJob { id: string; name: string; stack: string; machine?: string; services?: string[]; cron: string; apply: boolean; enabled: boolean }
|
||||
export interface UpdateJobRow { job: UpdateJob; next?: string; last?: { at?: string; taken?: string; error?: string } }
|
||||
export interface MachineUpdates { images: ImageCheck[]; checked: number; stacks: Record<string, string[]> }
|
||||
|
||||
export interface DockerHealthProbe { Start: string; End: string; ExitCode: number; Output: string }
|
||||
export interface DockerHealth { configured: boolean; state: string; status?: string; failing_streak?: number; log?: DockerHealthProbe[] }
|
||||
export interface DockerEvent { t: number; name: string; action: string; image?: string; exit_code?: string; signal?: string; health?: string }
|
||||
|
||||
export interface DockerInfo { available: boolean; message?: string; tool?: string; containers: DockerContainer[]; images: Record<string, unknown>[]; volumes: Record<string, unknown>[]; networks: Record<string, unknown>[] }
|
||||
export interface Stack { name: string; containers: string[]; running: number; file: string; host_file: string; files: string[]; host_files: string[] | null; images: string[] | null; via: string; remote: string; held_by: string; editable: boolean; note?: string; has_env: boolean }
|
||||
export interface StackHome { dir: string; inner: string; via: string; remote: string; held_by: string; stacks: number; default: boolean }
|
||||
@@ -134,6 +141,13 @@ export interface SpaceNode { name: string; dir: boolean; size: number; apparent:
|
||||
export interface SpaceTree extends SpaceNode { path: string; children?: SpaceTree[]; more_n?: number; more_size?: number; total: number }
|
||||
export interface SpaceFile { path: string; size: number; apparent: number; mtime: number }
|
||||
|
||||
export interface KVM { id: string; name: string; url: string; machine: string; added: string }
|
||||
|
||||
export interface ZFSPolicy { id: string; name: string; dataset: string; recursive: boolean; every: 'hourly' | 'daily' | 'weekly' | 'monthly' | 'custom'; at: string; weekday: number; day: number; cron?: string; keep_for: string; keep_min: number; prefix?: string; enabled: boolean }
|
||||
export interface ZFSRun { at?: string; taken?: string; pruned?: string[]; error?: string }
|
||||
export interface ZFSExternal { tool: string; schedule: string; command: string; user: string; source: string; id: string; next?: number; last?: number }
|
||||
export interface ZFSOverview { zfs: ZFSInfo; policies: { policy: ZFSPolicy; cron: string; next?: string; last: ZFSRun }[]; external?: ZFSExternal | null }
|
||||
|
||||
export interface FileEntry { name: string; type: 'dir' | 'file' | 'link' | 'other'; size: number; mode: string; perm: string; mtime: number; user: string; group: string; target?: string; broken?: boolean }
|
||||
export interface FileInfo extends FileEntry { path: string; text: boolean }
|
||||
|
||||
@@ -180,6 +194,15 @@ export const api = {
|
||||
storage: () => req<{ devices: StorageDevice[]; zfs: ZFSInfo; smartctl: boolean }>('GET', '/api/storage'),
|
||||
smart: (dev: string, force = false) => req<SmartInfo>('GET', `/api/storage/smart/${dev}${force ? '?force=1' : ''}`),
|
||||
smartTest: (dev: string, kind: 'short' | 'long') => req<void>('POST', `/api/storage/smart/${dev}/test/${kind}`),
|
||||
// Always the hub's: it holds every machine's KVM, so the tab works when the machine does not.
|
||||
kvms: () => req<KVM[]>('GET', '/api/kvms', undefined, true),
|
||||
addKVM: (name: string, url: string, machine: string) => req<KVM>('POST', '/api/kvms', { name, url, machine }, true),
|
||||
removeKVM: (id: string) => req<void>('DELETE', `/api/kvms/${encodeURIComponent(id)}`, undefined, true),
|
||||
zfsOverview: () => req<ZFSOverview>('GET', '/api/zfs'),
|
||||
saveZfsSchedule: (p: Partial<ZFSPolicy>) => req<ZFSPolicy>('PUT', '/api/zfs/schedules', p),
|
||||
deleteZfsSchedule: (id: string) => req<void>('DELETE', `/api/zfs/schedules/${encodeURIComponent(id)}`),
|
||||
runZfsSchedule: (id: string) => req<{ taken: string; pruned: string[] }>('POST', `/api/zfs/schedules/${encodeURIComponent(id)}/run`),
|
||||
deleteZfsSnapshot: (name: string, recursive: boolean) => req<void>('POST', '/api/zfs/snapshots/delete', { name, confirm: name, recursive }),
|
||||
zfsSnapshots: (dataset: string) => req<ZFSSnapshot[]>('GET', `/api/storage/zfs/snapshots?dataset=${encodeURIComponent(dataset)}`),
|
||||
zfsScrub: (pool: string, action: 'start' | 'pause' | 'stop') => req<void>('POST', `/api/storage/zfs/scrub/${encodeURIComponent(pool)}/${action}`),
|
||||
zfsSnapshot: (dataset: string, name: string) => req<void>('POST', '/api/storage/zfs/snapshot', { dataset, name }),
|
||||
@@ -208,8 +231,6 @@ export const api = {
|
||||
serviceAction: (unit: string, action: string) => req<void>('POST', `/api/services/${encodeURIComponent(unit)}/${action}`),
|
||||
historyWindow: () => req<HistoryWindow>('GET', '/api/prefs/history', undefined, true),
|
||||
setHistoryWindow: (seconds: number) => req<HistoryWindow>('PUT', '/api/prefs/history', { seconds }, true),
|
||||
dashPrefs: () => req<DashPrefs>('GET', '/api/prefs/dashboard'),
|
||||
saveDashPrefs: (p: DashPrefs) => req<void>('PUT', '/api/prefs/dashboard', p),
|
||||
login: (username: string, password: string) => req<{ user: string }>('POST', '/api/login', { username, password }, true),
|
||||
users: () => req<User[]>('GET', '/api/users', undefined, true),
|
||||
addUser: (name: string, password: string) => req<{ name: string }>('POST', '/api/users', { name, password }, true),
|
||||
@@ -235,6 +256,17 @@ export const api = {
|
||||
dockerAction: (id: string, action: 'start' | 'stop' | 'restart' | 'remove') => req<void>('POST', `/api/docker/containers/${encodeURIComponent(id)}/${action}`),
|
||||
dockerLogs: (id: string, since = '') => req<{ lines: { t: string; m: string }[] }>('GET', `/api/docker/containers/${encodeURIComponent(id)}/logs${since ? `?since=${encodeURIComponent(since)}` : '?n=300'}`),
|
||||
dockerInspect: (id: string) => req<Record<string, unknown>>('GET', `/api/docker/containers/${encodeURIComponent(id)}/inspect`),
|
||||
machineUpdates: (refresh = false) => req<MachineUpdates>('GET', `/api/docker/updates${refresh ? '?refresh=1' : ''}`),
|
||||
// One named machine, asked of the hub — the fleet view, which is on the hub by definition.
|
||||
nodeUpdates: (node: string, refresh = false) => req<MachineUpdates>('GET', `/api/nodes/${encodeURIComponent(node)}/docker/updates${refresh ? '?refresh=1' : ''}`, undefined, true),
|
||||
updateJobs: () => req<{ jobs: UpdateJobRow[]; file: string }>('GET', '/api/docker/schedules'),
|
||||
saveUpdateJob: (j: Partial<UpdateJob>) => req<UpdateJob>('PUT', '/api/docker/schedules', j),
|
||||
deleteUpdateJob: (id: string) => req<void>('DELETE', `/api/docker/schedules/${encodeURIComponent(id)}`),
|
||||
runUpdateJob: (id: string) => req<{ output: string }>('POST', `/api/docker/schedules/${encodeURIComponent(id)}/run`),
|
||||
dockerHealth: (id: string) => req<DockerHealth>('GET', `/api/docker/containers/${encodeURIComponent(id)}/health`),
|
||||
dockerEvents: (id: string, since = '7d') => req<{ events: DockerEvent[]; since: string; kept: number; capacity: number }>('GET', `/api/docker/containers/${encodeURIComponent(id)}/events?since=${since}`),
|
||||
dockerExec: (id: string) => req<{ id: string; name: string }>('POST', `/api/docker/containers/${encodeURIComponent(id)}/exec`, {}),
|
||||
dockerLimits: (id: string, p: { memory?: string; cpus?: string; restart?: string }) => req<void>('POST', `/api/docker/containers/${encodeURIComponent(id)}/limits`, p),
|
||||
stacks: () => req<{ stacks: Stack[] }>('GET', '/api/docker/stacks'),
|
||||
stackHomes: () => req<{ homes: StackHome[] }>('GET', '/api/docker/stacks/homes'),
|
||||
createStack: (p: { name: string; text: string; env: string; home: string; env_from?: string }) => req<{ stack: Stack }>('POST', '/api/docker/stacks', p),
|
||||
|
||||
@@ -0,0 +1,19 @@
|
||||
// Sizes and rates, shown the way the rest of the console shows them. These lived in
|
||||
// Dashboard.tsx until that page was retired (2026-09-23); Disks, Files and Space all use
|
||||
// them, so they belong on their own.
|
||||
|
||||
// fmtBytes is binary — what a disk or a file actually occupies.
|
||||
export function fmtBytes(b: number): string {
|
||||
const u = ['B', 'KiB', 'MiB', 'GiB', 'TiB']
|
||||
let i = 0
|
||||
while (b >= 1024 && i < u.length - 1) { b /= 1024; i++ }
|
||||
return `${b < 10 && i > 0 ? b.toFixed(1) : Math.round(b)} ${u[i]}`
|
||||
}
|
||||
|
||||
// fmtRate is decimal — how throughput is quoted everywhere else.
|
||||
export function fmtRate(bps: number): string {
|
||||
const u = ['B/s', 'kB/s', 'MB/s', 'GB/s']
|
||||
let i = 0
|
||||
while (bps >= 1000 && i < u.length - 1) { bps /= 1000; i++ }
|
||||
return `${bps < 10 && i > 0 ? bps.toFixed(1) : Math.round(bps)} ${u[i]}`
|
||||
}
|
||||
+18
-1
@@ -5,8 +5,25 @@ import { useEffect, useState } from 'react'
|
||||
|
||||
const KEY = 'hamlab.rail'
|
||||
|
||||
// Pages that were renamed, so an order saved before it still puts the page where it was put:
|
||||
// Top became Processes, Space became Usage, Cron became Jobs. A page that was removed
|
||||
// outright (Dashboard) is not here — orderBy drops ids that match no page.
|
||||
const RENAMED: Record<string, string> = { top: 'procs', space: 'usage', cron: 'jobs' }
|
||||
|
||||
export function getRailOrder(): string[] {
|
||||
try { const v = JSON.parse(localStorage.getItem(KEY) ?? '[]'); return Array.isArray(v) ? v.filter((x) => typeof x === 'string') : [] } catch { return [] }
|
||||
let raw: unknown
|
||||
try { raw = JSON.parse(localStorage.getItem(KEY) ?? '[]') } catch { return [] }
|
||||
if (!Array.isArray(raw)) return []
|
||||
const seen = new Set<string>()
|
||||
const out: string[] = []
|
||||
for (const x of raw) {
|
||||
if (typeof x !== 'string') continue
|
||||
const id = RENAMED[x] ?? x
|
||||
if (seen.has(id)) continue // a merged page keeps the earlier of the two slots
|
||||
seen.add(id)
|
||||
out.push(id)
|
||||
}
|
||||
return out
|
||||
}
|
||||
|
||||
export function setRailOrder(ids: string[], save = true) {
|
||||
|
||||
@@ -1,7 +1,7 @@
|
||||
// The stock Dashboard modules were written against one Sample object. They now read the
|
||||
// sensor API like everything else; this file is the seam — it rebuilds that object from
|
||||
// sensor values (the live stream) and sensor metadata (the catalog: labels, thresholds).
|
||||
import { Sample, Series } from './api'
|
||||
import { Sample } from './api'
|
||||
import { Sensor } from './sensors'
|
||||
|
||||
export type Values = Record<string, number | boolean | string>
|
||||
@@ -19,7 +19,6 @@ export function mergeValues(parts: unknown[]): Values {
|
||||
return out
|
||||
}
|
||||
// The curves they keep: CPU, memory, per-NIC and per-disk rates, every temperature.
|
||||
export const MONITOR_HISTORY = 'cpu.usage,mem.used,net.*.rx,net.*.tx,disk.*.read,disk.*.write,disk.*.util,temp.*'
|
||||
|
||||
const num = (v: Values, id: string) => (typeof v[id] === 'number' ? (v[id] as number) : 0)
|
||||
const str = (v: Values, id: string) => (typeof v[id] === 'string' ? (v[id] as string) : '')
|
||||
@@ -52,7 +51,7 @@ export function sampleOf(v: Values, cat: Catalog): Sample {
|
||||
fans: hw('fan.').map(({ id, s }) => ({ chip: s.labels!.chip, label: s.labels!.label, rpm: num(v, id) })),
|
||||
net: [...groups(v, cat, 'net.', 'iface')].map(([iface, p]) => ({ iface, up: v[p + '.up'] === true, rx_bps: num(v, p + '.rx'), tx_bps: num(v, p + '.tx'), rx_total: num(v, p + '.rx_total'), tx_total: num(v, p + '.tx_total') })),
|
||||
disks: [...groups(v, cat, 'disk.', 'device')].map(([name, p]) => ({ name, model: cat.get(p + '.size')?.labels?.model || undefined, size: num(v, p + '.size'), read_bps: num(v, p + '.read'), write_bps: num(v, p + '.write'), util_pct: num(v, p + '.util'), temp: num(v, p + '.temp') || undefined })),
|
||||
fs: [...groups(v, cat, 'fs.', 'mount')].map(([mount, p]) => ({ mount, type: cat.get(p + '.size')?.labels?.fstype ?? '', size: num(v, p + '.size'), used: num(v, p + '.used') })),
|
||||
fs: [...groups(v, cat, 'fs.', 'mount')].map(([mount, p]) => ({ mount, type: cat.get(p + '.size')?.labels?.fstype ?? '', size: num(v, p + '.size'), used: num(v, p + '.used'), remote: cat.get(p + '.size')?.labels?.remote === '1', disk: cat.get(p + '.size')?.labels?.disk })),
|
||||
ups: v['ups.battery'] === undefined ? null : {
|
||||
driver: cat.get('ups.last_event')?.labels?.driver ?? '', model: cat.get('ups.battery')?.labels?.model ?? '', state: str(v, 'ups.state'), source: str(v, 'ups.source'),
|
||||
on_battery: v['ups.on_battery'] === true, in_v: num(v, 'ups.in_v'), out_v: num(v, 'ups.out_v'), battery_pct: num(v, 'ups.battery'), runtime_min: num(v, 'ups.runtime'),
|
||||
@@ -63,18 +62,3 @@ export function sampleOf(v: Values, cat: Catalog): Sample {
|
||||
|
||||
// The modules' history: one shared time axis (the CPU's), every other curve right-aligned
|
||||
// to it and zero-padded on the left where the sensor is younger than the ring.
|
||||
export function seriesOf(hist: { id: string; t: number[]; v: number[] }[], cat: Catalog): Series {
|
||||
const by = new Map(hist.map((h) => [h.id, h]))
|
||||
const t = by.get('cpu.usage')?.t ?? []
|
||||
const n = t.length
|
||||
const fit = (id: string) => { const v = (by.get(id)?.v ?? []).slice(-n); return [...new Array(Math.max(0, n - v.length)).fill(0), ...v] as number[] }
|
||||
const out: Series = { t: [...t], cpu: fit('cpu.usage'), mem: fit('mem.used'), net: {}, disk: {}, temp: {} }
|
||||
for (const h of hist) {
|
||||
const l = cat.get(h.id)?.labels
|
||||
const stem = h.id.slice(0, h.id.lastIndexOf('.'))
|
||||
if (h.id.startsWith('net.') && l?.iface) out.net[l.iface] ??= { rx: fit(stem + '.rx'), tx: fit(stem + '.tx') }
|
||||
else if (h.id.startsWith('disk.') && l?.device) out.disk[l.device] ??= { r: fit(stem + '.read'), w: fit(stem + '.write'), u: fit(stem + '.util') }
|
||||
else if (h.id.startsWith('temp.') && l?.chip !== undefined) out.temp[`${l.chip}/${l.label}`] = fit(h.id)
|
||||
}
|
||||
return out
|
||||
}
|
||||
|
||||
+41
-2
@@ -285,14 +285,21 @@ h2.section { margin: 0; font-size: 13px; font-weight: 600; color: var(--muted);
|
||||
the fallback above stays a fallback. */
|
||||
.disk-cards { grid-template-columns: repeat(auto-fill, minmax(min(420px, 100%), 1fr)); }
|
||||
.datasets { padding: 0; overflow-x: auto; }
|
||||
.ds-row { display: grid; grid-template-columns: minmax(160px, 1fr) 80px 80px 80px 50px 50px minmax(120px, 1fr); gap: 8px; align-items: center; padding: 6px 12px; border-bottom: 1px solid var(--border); font-size: 13px; min-width: 700px; } /* the columns and gaps add up to 692: at 640 the row ended before its own cells did */
|
||||
.ds-row { display: grid; grid-template-columns: minmax(160px, 1fr) 80px 90px 90px 90px minmax(120px, 1fr); gap: 8px; align-items: center; padding: 6px 12px; border-bottom: 1px solid var(--border); font-size: 13px; min-width: 700px; }
|
||||
.ds-row.head { color: var(--muted); font-size: 11px; text-transform: uppercase; letter-spacing: .5px; }
|
||||
.ds-row .num { text-align: right; }
|
||||
.ds-name { cursor: pointer; display: flex; align-items: center; gap: 6px; white-space: nowrap; overflow: hidden; text-overflow: ellipsis; }
|
||||
.ds-mount { overflow: hidden; text-overflow: ellipsis; white-space: nowrap; font-family: ui-monospace, monospace; font-size: 12px; }
|
||||
.ds-snaps { padding: 8px 12px 10px 40px; border-bottom: 1px solid var(--border); background: var(--bg); display: flex; flex-direction: column; gap: 6px; }
|
||||
.snap-list { display: flex; flex-direction: column; gap: 2px; max-height: 40vh; overflow-y: auto; }
|
||||
.snap { display: flex; justify-content: space-between; gap: 12px; font-size: 12px; }
|
||||
.snap { display: grid; grid-template-columns: minmax(150px, 1fr) 150px 120px 70px auto; gap: 10px; align-items: center; font-size: 12px; padding: 2px 0; border-bottom: 1px solid var(--border); }
|
||||
.snap .snap-name { overflow: hidden; text-overflow: ellipsis; white-space: nowrap; }
|
||||
.snap .snap-when, .snap .snap-exp { white-space: nowrap; }
|
||||
.snap .snap-exp.warn { color: var(--s2); }
|
||||
.snap .num { text-align: right; }
|
||||
.snap .snap-del { padding: 0 6px; font-size: 11px; opacity: .55; }
|
||||
.snap:hover .snap-del { opacity: 1; }
|
||||
@media (max-width: 800px) { .snap { grid-template-columns: minmax(120px, 1fr) auto auto; } .snap .snap-when, .snap .snap-exp { display: none; } }
|
||||
|
||||
/* files */
|
||||
.files { position: absolute; inset: 0; display: flex; flex-direction: column; background: var(--bg); }
|
||||
@@ -720,3 +727,35 @@ pre.diff .add { color: var(--ok); background: color-mix(in srgb, var(--ok) 12%,
|
||||
.space-row.file { grid-template-columns: minmax(120px, 1fr) 72px 80px; }
|
||||
.space-split { padding: 8px; } .space-tree { flex: 1 1 45%; } .space-map { flex: 1 1 55%; }
|
||||
}
|
||||
|
||||
/* the Processes tab's drive rows and NIC switcher */
|
||||
.t-disk .t-model { overflow: hidden; text-overflow: ellipsis; white-space: nowrap; max-width: 14ch; margin-left: 6px; }
|
||||
.t-disk .t-temp { margin-right: 8px; color: var(--t-blue, var(--accent2)); }
|
||||
.t-disk .t-mounts { overflow: hidden; text-overflow: ellipsis; white-space: nowrap; }
|
||||
.t-ifs { display: inline-flex; gap: 4px; flex-wrap: wrap; justify-content: flex-end; }
|
||||
.t-ifs .t-if { padding: 0 4px; border-radius: 3px; color: var(--muted); }
|
||||
.t-ifs .t-if:hover { color: var(--fg); background: var(--bg3); }
|
||||
.t-ifs .t-if.on { color: var(--fg); box-shadow: inset 0 -2px 0 var(--accent); }
|
||||
|
||||
/* ZFS: the schedule dialog's controls match the rest of the modal forms */
|
||||
.modal select, .modal input[type=time], .modal input[type=number] { padding: 9px 10px; background: var(--bg); border: 1px solid var(--border); border-radius: 6px; color: var(--fg); font-size: 16px; }
|
||||
.modal .chips { margin-top: 2px; }
|
||||
|
||||
/* ---- KVM: a device's own web page, framed ---- */
|
||||
.kvm { position: absolute; inset: 0; display: flex; flex-direction: column; background: var(--bg); min-height: 0; }
|
||||
.kvm-bar { display: flex; align-items: center; gap: 8px; padding: 8px 12px; border-bottom: 1px solid var(--border); background: var(--bg2); flex-wrap: wrap; }
|
||||
.kvm-bar .kvm-url { overflow: hidden; text-overflow: ellipsis; white-space: nowrap; max-width: 28ch; }
|
||||
.kvm-frame { flex: 1; min-height: 0; background: #000; position: relative; }
|
||||
.kvm-frame iframe { width: 100%; height: 100%; border: 0; display: block; background: #000; }
|
||||
.kvm-frame.full { background: #000; }
|
||||
.kvm-hint { padding: 6px 12px; border-top: 1px solid var(--border); }
|
||||
@media (max-width: 800px) { .kvm-bar .kvm-url { display: none; } }
|
||||
.kvm-none { text-align: center; } .kvm-none p { margin: 4px 0; }
|
||||
/* No true fullscreen for a page element (Safari on iOS): fill the viewport in-page instead. */
|
||||
.kvm-frame.expanded { position: fixed; inset: 0; z-index: 60; }
|
||||
.kvm-frame.expanded iframe { height: 100%; }
|
||||
.kvm-shrink { position: absolute; right: max(10px, env(safe-area-inset-right)); bottom: max(10px, env(safe-area-inset-bottom)); z-index: 61; background: var(--bg2); border: 1px solid var(--border); opacity: .75; }
|
||||
.kvm-shrink:hover, .kvm-shrink:active { opacity: 1; }
|
||||
.updot { margin-left: 6px; font-size: 11px; font-weight: 700; }
|
||||
.updot.behind { color: var(--s2); } .updot.unknown { color: var(--muted); }
|
||||
.card > .row > .row-end { text-align: right; white-space: nowrap; }
|
||||
|
||||
@@ -14,6 +14,12 @@ export function CloseBtn({ onClick, title = 'Close' }: { onClick: () => void; ti
|
||||
)
|
||||
}
|
||||
|
||||
// A filled bar. Red past 95%, amber past 85%, unless the caller says otherwise.
|
||||
export function Meter({ frac, cls }: { frac: number; cls?: string }) {
|
||||
const c = cls ?? (frac >= 0.95 ? 'bad' : frac >= 0.85 ? 'warn' : '')
|
||||
return <div className="meter"><div className={'fill ' + c} style={{ width: `${Math.min(100, Math.max(0, frac * 100))}%` }} /></div>
|
||||
}
|
||||
|
||||
export function Loading({ label = 'Loading…' }: { label?: string }) {
|
||||
return <div className="center muted loading"><Spinner size={22} /><div>{label}</div></div>
|
||||
}
|
||||
|
||||
Reference in New Issue
Block a user