Stacks: one machine, one set of files; default home is <data dir>/stacks
The hub no longer probes a node's Docker on tcp :2375, holds compose files
for a node, or runs compose against another machine's Docker. hubStacks,
nodeDocker, taskOnHub and the hub-side /api/nodes/{node}/docker/... routes
are gone; the generic node proxy hands those requests to the node's own
Ham Lab. Stack.remote / held_by and compose.Spec.Remote are removed.
New stacks default to <data dir>/stacks on every machine. Folders with two
or more deployed stacks are still offered, after it, never as default.
The ~/stacks fallback is gone. A missing home is skipped without a sudo find.
A schedule saved earlier with a tcp target is refused with a message
instead of being run against this machine's Docker.
PORTABILITY.md 3.1, 3.2, 3.3 and 9.2 done; 3.5 checked, no change needed.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
co-authored by
Claude Opus 5.5
parent
a0ce3e5eb9
commit
c27d9f0ada
@@ -21,7 +21,7 @@ stub tab or a roadmap item.
|
||||
| Guest logins | A **guest** is an account like any other, with `guest: true` and an expiry: `login`, sessions and prefs need no special case, and **it may do everything an account may do**. The expiry is the whole feature — checked on every request and swept once a minute — with revoking as the manual version of it. | Built 2026-09-27. A read-only version was built first and **cut the same day**: Mitchell hands these to people he trusts and revokes them otherwise, so a console with half its tabs missing was solving a problem he doesn't have. Consequences, stated once and accepted: a guest has the Terminal tab, which is a root-capable shell on every host, and can make an account or token of its own that outlives the guest login. The one thing a guest is not is *an account*: it is left out of the Users list, out of "the last account can't be removed", and out of the single account an empty user name resolves to. |
|
||||
| Machines | **A view shows one machine: the one in the switcher.** No hub-side page lists or aggregates what its nodes are doing. To see or change anything on another machine, switch to it. | Mitchell, 2026-09-25, cutting a fleet-wide updates list out of the Docker tab. The switcher is the only place a machine is chosen, so there is exactly one answer to "which machine am I looking at" — a page that quietly reaches across breaks that, and it is never obvious which machine an action would land on. The hub still *proxies* for a node, and hub-only admin (Machines, Users, Settings, Alerts) is unaffected. |
|
||||
| Capability, not policy | **A tab offers what the selected machine can do, detected live — never what a rule says should run there.** Where the software is missing but installable, Ham Lab installs it; where it cannot be installed or the hardware lacks it, the tab says so in a sentence and offers what does work. Every action lands on the machine in the switcher. | Mitchell, 2026-10-09, after the Virtualization plan's first draft excluded the Mac mini and `hamtv` on the strength of `hamlet/CLAUDE.md`'s deployment rules: *"I want the option to do it anywhere I want, unless the hardware / software literally doesn't support it. Ham Lab needs to be built in a general-purpose way that can run on any Linux system."* Those rules bind the agent's unasked deployments, not the product. |
|
||||
| Ham Lab's data lives in Ham Lab's folder | **Everything a tab manages sits under the machine's data directory (`config.DataDir()`), never at a convenient path elsewhere.** Existing material is imported by copying. The engine's runtime state (`/var/lib/docker`, `/var/lib/incus`) is the engine's; Ham Lab keeps what would rebuild it. | Mitchell, 2026-10-09, when the Virtualization plan pointed its ISO library at `/tank/software/isos/`: *"ALL data needs to be under a HamLab folder, because it manages its own data."* The same reason `core`'s compose files moved under `apps/hamlab/stacks/` on 2026-10-05. `grid`'s and the Mac's stacks at `~/stacks/<machine>/` are the standing exception. |
|
||||
| Ham Lab's data lives in Ham Lab's folder | **Everything a tab manages sits under the machine's data directory (`config.DataDir()`), never at a convenient path elsewhere.** Existing material is imported by copying. The engine's runtime state (`/var/lib/docker`, `/var/lib/incus`) is the engine's; Ham Lab keeps what would rebuild it. | Mitchell, 2026-10-09, when the Virtualization plan pointed its ISO library at `/tank/software/isos/`: *"ALL data needs to be under a HamLab folder, because it manages its own data."* The same reason `core`'s compose files moved under `apps/hamlab/stacks/` on 2026-10-05. `grid`'s and the Mac's stacks at `~/stacks/<machine>/` were the standing exception; since 2026-10-10 the stacks code offers `<data dir>/stacks` as the default home everywhere and such folders only as discovered extras. |
|
||||
|
||||
## Layout
|
||||
|
||||
@@ -260,12 +260,13 @@ something long in one, switch away and back, refresh — it's all still there.
|
||||
filesystem; compose is then run **inside that container**, so `.env`, relative paths
|
||||
and the compose labels stay exactly as deployed (verified: labels identical after a
|
||||
redeploy). Save validates with `compose config` first, keeps the previous file as
|
||||
`<file>.hamlab-bak`, and never deploys by itself. A stack deployed from another host
|
||||
(Dockhand on `core` driving `grid` over tcp) has no file on that machine, so the node
|
||||
alone lists it read-only — but **the hub fills that in** (`hubStacks`): asked about a
|
||||
node, it resolves the files on itself and runs compose here with `DOCKER_HOST` pointed
|
||||
at the node's `tcp://…:2375`, the way it was deployed; the task runs on the hub and the
|
||||
node's page follows it. Also per stack: every file it was deployed from (chips), its
|
||||
`<file>.hamlab-bak`, and never deploys by itself. A stack whose file is not on the
|
||||
machine that runs it is listed read-only, and that is the end of it: **since 2026-10-10
|
||||
the hub holds no files for a node and never points compose at another machine's
|
||||
Docker.** The `hubStacks` path — which resolved a node's files on the hub and deployed
|
||||
over `tcp://…:2375`, the way Dockhand had — is gone, and every
|
||||
`/api/nodes/<node>/docker/…` request is the node's own Ham Lab's to answer through the
|
||||
generic proxy. Also per stack: every file it was deployed from (chips), its
|
||||
**`.env`** (values masked until "Reveal and edit"; validated in place, put back if
|
||||
compose objects), **what Deploy would change** (a line diff against the copy kept at the
|
||||
last deploy from here, `<file>.hamlab-deployed`, else the copy the last save replaced;
|
||||
@@ -278,9 +279,10 @@ something long in one, switch away and back, refresh — it's all still there.
|
||||
form (image, ports, volumes, environment, restart, command, host networking) that writes
|
||||
the compose for you: a container made here is a one-service stack, so it stays editable
|
||||
and redeployable, where a bare `docker run` could only be thrown away and typed again.
|
||||
New stacks go in a **home** — a folder where two or more stacks are already deployed from
|
||||
(Dockhand's `stacks/core` here; `~/stacks` on a machine with none) — and are deployed the
|
||||
way their neighbours were (inside `dockhand`, or over tcp to a node from the hub). Homes
|
||||
New stacks go in a **home**. Since 2026-10-10 the default is Ham Lab's own
|
||||
`<data dir>/stacks`, offered first on every machine; a folder where two or more stacks
|
||||
are already deployed from is offered as well, and a stack made there is deployed the way
|
||||
its neighbours were (inside an orchestrator's container, when that is how they run). Homes
|
||||
are also scanned, so a stack with no containers ("not deployed": never deployed, or taken
|
||||
down) is still listed. **Down** removes a stack's containers; **Delete…** (type its name)
|
||||
takes it down and moves its own files to `<home>/.hamlab-trash/<name>-<time>/` — moved,
|
||||
@@ -780,12 +782,12 @@ what it has moved.
|
||||
stacks that are merely down (`pihole_default`, `transmission_default`, `gluetun_default`
|
||||
among them), and pruning would take them and hand them new subnets on the next deploy.
|
||||
|
||||
- **The `*:2375` question, which this is the moment to settle.** A stack deployed to
|
||||
`grid` or the Mac still goes over `tcp://<node>:2375` from the hub. Decide whether the
|
||||
hub should instead ask the node's own Ham Lab to deploy locally, which would end the
|
||||
unauthenticated Docker socket exposure — its own item in `docs/pending.md`, currently
|
||||
filed under "decided, no action", and this is the change that would make closing it
|
||||
free.
|
||||
- **The `*:2375` question — settled, and the code removed 2026-10-10.** The hub asks the
|
||||
node's own Ham Lab; nothing probes or uses a node's Docker port. `hubStacks`,
|
||||
`nodeDocker`, `taskOnHub`, the hub-side `/api/nodes/{node}/docker/…` routes and the
|
||||
`remote` / `held_by` fields are gone. A schedule saved before this that still carries a
|
||||
`remote` is refused with a message, never run against the wrong Docker. `PORTABILITY.md`
|
||||
§3 has the audit it came from.
|
||||
|
||||
- **A Virtualization tab** — asked for 2026-09-23; **scoped, built and deployed 2026-10-09 — see
|
||||
`VIRTUALIZATION.md`** for what shipped (phases 1 and 2) and what is left. Incus 6.0 LTS as the engine; the tab detects per machine whether Incus is
|
||||
|
||||
+7
-7
@@ -44,11 +44,11 @@ default or phones home to this lab · **C** cosmetic (placeholder text, comments
|
||||
|
||||
| # | Sev | Where | Assumes | General form |
|
||||
|---|---|---|---|---|
|
||||
| 3.1 | **A** | `internal/api/stacks.go:483-509` `nodeDocker()` | For every node whose stacks aren't editable locally, the hub **probes `tcp://<node-host>:2375`** with `docker -H … version` and, if it answers, deploys to the node over an unauthenticated Docker socket. This is the Dockhand-era layout; on any other network an open 2375 is either absent or a security problem the console should not quietly use. | Remove the probe. The node's own Ham Lab deploys its stacks (the files live on the node, under its data dir). `PLAN.md` "To do" already names this decision. |
|
||||
| 3.2 | **A** | `internal/api/stacks.go:439-479` `hubStacks()`, `:1036-1074` `homesFor()`, `web/src/Docker.tsx` "kept on `<hub>`, deployed to this machine over `tcp://…`" | A hub can hold compose files for a node and run compose remotely; new stacks for a node may be created **on the hub**. Only true here because Dockhand on `core` deployed `grid` and the Mac. | Same as 3.1: one machine, one set of files, one Docker. Hub-held stacks become an import path (copy to the node) rather than a steady state. |
|
||||
| 3.3 | **B** | `internal/api/stacks.go:290-382` `homesOf()`; `:373-376` fallback `~/stacks` | Where new stacks go is **discovered** from where ≥2 stacks already are (Dockhand's `stacks/<machine>` habit), else `~/stacks`. Neither is Ham Lab's data directory, which the project's own rule says everything managed must live under. | Default home = `<DataDir>/stacks`. Discovered homes stay as *additional* places, never the default. |
|
||||
| 3.1 | **done** 2026-10-10 | `internal/api/stacks.go` `nodeDocker()` (removed) | For every node whose stacks aren't editable locally, the hub **probes `tcp://<node-host>:2375`** with `docker -H … version` and, if it answers, deploys to the node over an unauthenticated Docker socket. This is the Dockhand-era layout; on any other network an open 2375 is either absent or a security problem the console should not quietly use. | Remove the probe. The node's own Ham Lab deploys its stacks (the files live on the node, under its data dir). `PLAN.md` "To do" already names this decision. |
|
||||
| 3.2 | **done** 2026-10-10 | `hubStacks()`, `taskOnHub()`, the hub-side `/api/nodes/{node}/docker/…` routes, `Stack.remote`/`held_by` (all removed; the generic node proxy carries those requests) | A hub can hold compose files for a node and run compose remotely; new stacks for a node may be created **on the hub**. Only true here because Dockhand on `core` deployed `grid` and the Mac. | Same as 3.1: one machine, one set of files, one Docker. Hub-held stacks become an import path (copy to the node) rather than a steady state. |
|
||||
| 3.3 | **done** 2026-10-10 | `internal/api/stacks.go` `homesOf()` — `<DataDir>/stacks` is always offered first and default; the `~/stacks` fallback is gone; `TestHomesDefaultToDataDir` | Where new stacks go is **discovered** from where ≥2 stacks already are (Dockhand's `stacks/<machine>` habit), else `~/stacks`. Neither is Ham Lab's data directory, which the project's own rule says everything managed must live under. | Default home = `<DataDir>/stacks`. Discovered homes stay as *additional* places, never the default. |
|
||||
| 3.4 | **B** | `internal/api/stacks.go:100-203` remembered mounts, `internal/api/dockersched.go:772-795` `repoint()` | Machinery exists to find compose files **inside a removed orchestrator container's filesystem**. Generic in form, but it exists because of Dockhand and is exercised only here. | Keep (it is general), but it is not a reason to keep 3.1/3.2. |
|
||||
| 3.5 | **B** | `internal/api/stacks.go:801` | Inside a `Via` container the command is literally `docker compose`, even when `dockerTool()` is `podman`. | Use the detected tool. |
|
||||
| 3.5 | — | `internal/api/stacks.go` `stackAction`, `internal/compose/compose.go` | Inside a `Via` container the command is literally `docker compose`. | **Checked 2026-10-10, no change:** the CLI inside an orchestrator's container is that container's own `docker`, independent of what the host runs; the host's tool only chooses the `exec`. Both places now say so in a comment. |
|
||||
| 3.6 | **B** | `internal/api/stacks.go:908` `docker buildx imagetools inspect` | Update checks need the **buildx** plugin. Docker CE on Debian has it; Podman and minimal Docker installs do not → every image reports "the registry didn't answer". | Fall back to `skopeo inspect` or a direct registry `HEAD` for the digest. |
|
||||
| 3.7 | **B** | `Makefile:445-448` `deploy` | The machine you build on **is** the hub: installs locally, then `push-nodes`. | Fine for a developer; document that `deploy` means "I am the hub". A plain `make install` target for a single box. |
|
||||
| 3.8 | **B** | `internal/nodes/nodes.go:316-323` | A hub pushes other architectures only if `<prefix>/hamlab-<arch>` siblings exist, which only `make deploy`/`install.sh`-from-dist put there. | A hub without siblings should fetch the right build from `update.url` instead of erroring. |
|
||||
@@ -125,7 +125,7 @@ The installer supports systemd, procd and OpenRC (`internal/setup/setup.go:217-2
|
||||
| # | Sev | Where | Assumes | General form |
|
||||
|---|---|---|---|---|
|
||||
| 9.1 | **B** | `internal/config/config.go:115-117` `Candidates()`; `internal/setup/setup.go:413-422`; `README.md:38` | `~/apps/hamlab/config.json` is searched as a legacy location — the lab's pre-v0.2 path, kept so `core`, `grid` and the Mac keep working. | Keep for one more release with a one-time migration log line, then drop. A stranger never had it. |
|
||||
| 9.2 | **B** | 3.3 above; `PLAN.md:24` "`grid`'s and the Mac's stacks at `~/stacks/<machine>/` are the standing exception" | The "everything under DataDir" rule has a lab-shaped exception baked into the stacks code. | Default to DataDir; treat `~/stacks` as a discovered home like any other. |
|
||||
| 9.2 | **done** 2026-10-10 | 3.3 above; `PLAN.md:24` "`grid`'s and the Mac's stacks at `~/stacks/<machine>/` are the standing exception" | The "everything under DataDir" rule has a lab-shaped exception baked into the stacks code. | Default to DataDir; treat `~/stacks` as a discovered home like any other. |
|
||||
| 9.3 | **C** | `internal/setup/setup.go:351` `ownFiles` | Correctly scoped; note only. | None. |
|
||||
|
||||
## 10. Build and release
|
||||
@@ -157,8 +157,8 @@ Listed so the sweep is known to be complete, not because they need work.
|
||||
|
||||
## Suggested order of work
|
||||
|
||||
1. **Posture and safety first (A):** 7.2 trusted proxies; 7.1 installer-written scoped sudoers; 3.1 drop the `:2375` probe (with 3.2 as its consequence).
|
||||
1. **Posture and safety first (A):** 7.2 trusted proxies; 7.1 installer-written scoped sudoers; ~~3.1 drop the `:2375` probe (with 3.2 as its consequence)~~ done 2026-10-10.
|
||||
2. **"Any Linux" promises the README already makes (A):** 4.1 power, 4.4/4.5 cron on busybox and timer-only hosts, 4.2 Incus install on non-systemd, 4.3 a Services backend for OpenRC/procd, 2.10 the `tank` regex.
|
||||
3. **Lab defaults that phone home or mislead (B):** 1.1–1.3 update/install URLs, 2.9 schedule defaults, 6.9/6.10 speed-test sources and an off switch, 8.1 memory limit, 3.3/9.2 stack home = DataDir.
|
||||
3. **Lab defaults that phone home or mislead (B):** 1.1–1.3 update/install URLs, 2.9 schedule defaults, 6.9/6.10 speed-test sources and an off switch, 8.1 memory limit, ~~3.3/9.2 stack home = DataDir~~ done 2026-10-10.
|
||||
4. **Breadth (B):** 5.1 zypper for Incus, 5.5 cgroup v1, 6.1/6.2 GPUs beyond Intel/AMD, 6.4 UPS order, 6.11/6.12 wg-quick/NetworkManager, 6.15 IPv6.
|
||||
5. **Cosmetics (C):** every placeholder and comment in §1.10 and §2, then split lab facts out of `PLAN.md`, `VIRTUALIZATION.md`, `SENSORS.md`, `README.md`.
|
||||
|
||||
@@ -55,11 +55,14 @@ type composeSpec struct {
|
||||
Project string `json:"project"`
|
||||
Files []string `json:"files"`
|
||||
Via string `json:"via,omitempty"`
|
||||
// Remote is read but never written: schedules made before v0.2 could point compose at
|
||||
// another machine's Docker over tcp. Such a job is refused rather than run against this
|
||||
// machine's Docker, which is where it would land without the variable.
|
||||
Remote string `json:"remote,omitempty"`
|
||||
}
|
||||
|
||||
func (c composeSpec) toSpec(tool string) compose.Spec {
|
||||
return compose.Spec{Project: c.Project, Files: c.Files, Via: c.Via, Remote: c.Remote, Tool: tool}
|
||||
return compose.Spec{Project: c.Project, Files: c.Files, Via: c.Via, Tool: tool}
|
||||
}
|
||||
|
||||
const dockerCronFile = "/etc/cron.d/hamlab-docker"
|
||||
@@ -169,6 +172,9 @@ func RunUpdate(ctx context.Context, db *store.Store, j UpdateJob, tool string) (
|
||||
if len(j.Spec.Files) == 0 {
|
||||
return "", errors.New("this schedule has no compose file recorded — open it in the Docker tab and save it again")
|
||||
}
|
||||
if j.Spec.Remote != "" {
|
||||
return "", errors.New("this schedule was made to deploy to another machine over tcp, which this console no longer does — make it again on that machine")
|
||||
}
|
||||
spec := repoint(ctx, db, j.Spec.toSpec(tool))
|
||||
var log strings.Builder
|
||||
run := func(args ...string) error {
|
||||
@@ -238,7 +244,7 @@ func (s *Server) dsPut(w http.ResponseWriter, r *http.Request) {
|
||||
// seenFiles is the list as compose itself sees it: host paths normally, the
|
||||
// orchestrator container's own paths when compose runs inside one. Getting this
|
||||
// backwards gives a cron line that works from the console and fails at 3am.
|
||||
j.Spec = composeSpec{Project: st.Name, Files: seenFiles(st, -1, ""), Via: st.Via, Remote: st.Remote}
|
||||
j.Spec = composeSpec{Project: st.Name, Files: seenFiles(st, -1, ""), Via: st.Via}
|
||||
if len(j.Spec.Files) == 0 {
|
||||
jsonErr(w, 409, "that stack's compose file isn't anywhere this console can reach")
|
||||
return
|
||||
|
||||
+3
-19
@@ -220,20 +220,8 @@ func (s *Server) routes() *http.ServeMux {
|
||||
m.HandleFunc("PUT /api/docker/schedules", s.auth(s.dsPut))
|
||||
m.HandleFunc("DELETE /api/docker/schedules/{id}", s.auth(s.dsDelete))
|
||||
m.HandleFunc("POST /api/docker/schedules/{id}/run", s.auth(s.dsRun))
|
||||
m.HandleFunc("GET /api/nodes/{node}/docker/updates", s.auth(s.machineUpdates))
|
||||
// The same, asked of a node through this hub: the hub answers for the stacks whose files
|
||||
// it holds (deployed to the node over tcp) and passes everything else to the node.
|
||||
m.HandleFunc("GET /api/nodes/{node}/docker/stacks", s.auth(s.stacksList))
|
||||
m.HandleFunc("POST /api/nodes/{node}/docker/stacks", s.auth(s.stackCreate))
|
||||
m.HandleFunc("GET /api/nodes/{node}/docker/stacks/homes", s.auth(s.stackHomes))
|
||||
m.HandleFunc("POST /api/nodes/{node}/docker/stacks/{name}/delete", s.auth(s.stackDelete))
|
||||
m.HandleFunc("GET /api/nodes/{node}/docker/stacks/{name}", s.auth(s.stackFile))
|
||||
m.HandleFunc("PUT /api/nodes/{node}/docker/stacks/{name}", s.auth(s.stackSave))
|
||||
m.HandleFunc("GET /api/nodes/{node}/docker/stacks/{name}/diff", s.auth(s.stackDiff))
|
||||
m.HandleFunc("GET /api/nodes/{node}/docker/stacks/{name}/updates", s.auth(s.stackUpdates))
|
||||
m.HandleFunc("POST /api/nodes/{node}/docker/stacks/{name}/{action}", s.auth(s.stackAction))
|
||||
m.HandleFunc("GET /api/nodes/{node}/tasks/{id}", s.auth(s.taskOnHub(s.getTask)))
|
||||
m.HandleFunc("GET /api/nodes/{node}/tasks/{id}/stream", s.auth(s.taskOnHub(s.streamTask)))
|
||||
// Asked of a node through this hub, every one of these is the node's to answer: the
|
||||
// generic /api/nodes/{name}/{rest...} proxy above carries them there unchanged.
|
||||
m.HandleFunc("POST /api/docker/containers/{id}/{action}", s.auth(s.dockerAction))
|
||||
m.HandleFunc("GET /api/docker/containers/{id}/logs", s.auth(s.dockerLogs))
|
||||
m.HandleFunc("GET /api/docker/containers/{id}/logs/tail", s.auth(s.dockerTail))
|
||||
@@ -960,11 +948,7 @@ func (s *Server) updateAllMachines(w http.ResponseWriter, r *http.Request) {
|
||||
}
|
||||
|
||||
func (s *Server) proxyNode(w http.ResponseWriter, r *http.Request) {
|
||||
node := r.PathValue("node") // routes that also carry a {name} of their own call the machine {node}
|
||||
if node == "" {
|
||||
node = r.PathValue("name")
|
||||
}
|
||||
p, ok := s.nodes.Proxy(node)
|
||||
p, ok := s.nodes.Proxy(r.PathValue("name"))
|
||||
if !ok {
|
||||
jsonErr(w, 404, "no such machine")
|
||||
return
|
||||
|
||||
+73
-272
@@ -18,11 +18,10 @@ package api
|
||||
// machine would become unfindable at once. A remembered mount resolves with no container to
|
||||
// run in, which is right: compose runs on the host against the host's copy of the file.
|
||||
//
|
||||
// A stack can also be deployed *from another machine*: Dockhand on the hub drives a node's
|
||||
// Docker over tcp, so the node runs the containers and the hub holds the files. The node
|
||||
// alone can only list such a stack read-only. The hub, asked about one of its nodes, fills
|
||||
// in what the node couldn't (hubStacks): it resolves the files on itself and runs compose
|
||||
// here with DOCKER_HOST pointed at the node.
|
||||
// A stack is one machine's affair: its files and the Docker that runs it are on the same
|
||||
// box, and that box's Ham Lab is the only thing that touches either. Asked about a node's
|
||||
// stacks, a hub passes the request to the node's own Ham Lab and relays the answer; it holds
|
||||
// no files for anyone else and never points compose at another machine's Docker.
|
||||
|
||||
import (
|
||||
"bytes"
|
||||
@@ -35,7 +34,6 @@ import (
|
||||
"io"
|
||||
"log"
|
||||
"net/http"
|
||||
"net/url"
|
||||
"os"
|
||||
"os/exec"
|
||||
"path/filepath"
|
||||
@@ -60,8 +58,6 @@ type stack struct {
|
||||
Files []string `json:"files"` // every file it was deployed from, in order: -f a -f b
|
||||
HostFiles []string `json:"host_files"` // the same, on the machine that holds them
|
||||
Via string `json:"via"` // container compose runs inside; "" = on the host
|
||||
Remote string `json:"remote"` // DOCKER_HOST compose is pointed at; "" = this machine's Docker
|
||||
HeldBy string `json:"held_by"` // the machine holding the files, when it isn't the one running them
|
||||
Editable bool `json:"editable"`
|
||||
Note string `json:"note,omitempty"`
|
||||
HasEnv bool `json:"has_env"`
|
||||
@@ -282,7 +278,7 @@ func (s *Server) findStacks(ctx context.Context) ([]stack, error) {
|
||||
st.resolve(maps)
|
||||
stacks = append(stacks, *st)
|
||||
}
|
||||
stacks = append(stacks, scanHomes(ctx, s.homesOf(stacks, true), stacks)...)
|
||||
stacks = append(stacks, scanHomes(ctx, s.homesOf(stacks), stacks)...)
|
||||
sort.Slice(stacks, func(i, j int) bool { return stacks[i].Name < stacks[j].Name })
|
||||
return stacks, nil
|
||||
}
|
||||
@@ -290,19 +286,23 @@ func (s *Server) findStacks(ctx context.Context) ([]stack, error) {
|
||||
// ---- where stacks live ----
|
||||
|
||||
// A home is a directory of stacks, one folder each: where new ones are made, and where ones
|
||||
// that aren't deployed (never were, or were taken down) are still found. Homes aren't
|
||||
// configured; they are wherever this machine's stacks already are. With none at all, ~/stacks.
|
||||
// that aren't deployed (never were, or were taken down) are still found. The default is
|
||||
// Ham Lab's own: <data dir>/stacks, which is where everything this console manages lives.
|
||||
// Any folder this machine already keeps stacks in — two or more deployed from it — is
|
||||
// offered as well, so what was deployed before Ham Lab arrived stays editable and gains
|
||||
// neighbours where it is, but it is never the default.
|
||||
type stackHome struct {
|
||||
Dir string `json:"dir"` // on the machine that holds the files
|
||||
Dir string `json:"dir"` // on this machine
|
||||
Inner string `json:"inner"` // the same directory as compose sees it (inside Via); Dir otherwise
|
||||
Via string `json:"via"` // container compose runs inside
|
||||
Remote string `json:"remote"` // DOCKER_HOST, when the containers run on another machine
|
||||
HeldBy string `json:"held_by"` // the machine holding the files, when it isn't the one running them
|
||||
Stacks int `json:"stacks"`
|
||||
Default bool `json:"default"`
|
||||
}
|
||||
|
||||
func (s *Server) homesOf(stacks []stack, withFallback bool) []stackHome {
|
||||
// stacksDir is the home Ham Lab owns.
|
||||
func (s *Server) stacksDir() string { return filepath.Join(s.cfg.DataDir(), "stacks") }
|
||||
|
||||
func (s *Server) homesOf(stacks []stack) []stackHome {
|
||||
type tally struct {
|
||||
home stackHome
|
||||
deployed int
|
||||
@@ -318,11 +318,10 @@ func (s *Server) homesOf(stacks []stack, withFallback bool) []stackHome {
|
||||
if strings.Count(dir, "/") < 2 { // "/" or "/tmp": not somewhere to make folders in
|
||||
continue
|
||||
}
|
||||
key := dir + "|" + st.Remote
|
||||
t := by[key]
|
||||
t := by[dir]
|
||||
if t == nil {
|
||||
t = &tally{home: stackHome{Dir: dir, Remote: st.Remote, HeldBy: st.HeldBy}, byVia: map[string]int{}, inner: map[string]string{}}
|
||||
by[key] = t
|
||||
t = &tally{home: stackHome{Dir: dir}, byVia: map[string]int{}, inner: map[string]string{}}
|
||||
by[dir] = t
|
||||
}
|
||||
t.home.Stacks++
|
||||
if len(st.Containers) > 0 {
|
||||
@@ -335,11 +334,13 @@ func (s *Server) homesOf(stacks []stack, withFallback bool) []stackHome {
|
||||
}
|
||||
}
|
||||
// A folder is a home when stacks are kept there as a habit: two or more deployed from it.
|
||||
// One stack on its own (Dockhand's own compose file in ~/apps/dockhand) says nothing about
|
||||
// where the next should go, and would have every compose file near it listed as a stack.
|
||||
// New stacks are deployed the way most of their neighbours were.
|
||||
var homes, lone []stackHome
|
||||
for _, t := range by {
|
||||
// One stack on its own (an orchestrator's own compose file in its app folder) says nothing
|
||||
// about where the next should go, and would have every compose file near it listed as a
|
||||
// stack — unless it is already in Ham Lab's folder. New stacks are deployed the way most
|
||||
// of their neighbours were.
|
||||
own := s.stacksDir()
|
||||
var homes []stackHome
|
||||
for dir, t := range by {
|
||||
best, n := "", -1
|
||||
for via, c := range t.byVia {
|
||||
if c > n || (c == n && via > best) {
|
||||
@@ -355,29 +356,23 @@ func (s *Server) homesOf(stacks []stack, withFallback bool) []stackHome {
|
||||
if t.home.Inner == "" {
|
||||
t.home.Inner = t.home.Dir
|
||||
}
|
||||
if t.deployed >= 2 {
|
||||
if t.deployed >= 2 || dir == own {
|
||||
homes = append(homes, t.home)
|
||||
} else {
|
||||
lone = append(lone, t.home)
|
||||
}
|
||||
}
|
||||
if len(homes) == 0 {
|
||||
homes = lone
|
||||
}
|
||||
sort.Slice(homes, func(i, j int) bool {
|
||||
if (homes[i].Dir == own) != (homes[j].Dir == own) {
|
||||
return homes[i].Dir == own
|
||||
}
|
||||
if homes[i].Stacks != homes[j].Stacks {
|
||||
return homes[i].Stacks > homes[j].Stacks
|
||||
}
|
||||
return homes[i].Dir < homes[j].Dir
|
||||
})
|
||||
if len(homes) == 0 && withFallback {
|
||||
if home, err := os.UserHomeDir(); err == nil {
|
||||
homes = append(homes, stackHome{Dir: filepath.Join(home, "stacks"), Inner: filepath.Join(home, "stacks")})
|
||||
if len(homes) == 0 || homes[0].Dir != own {
|
||||
homes = append([]stackHome{{Dir: own, Inner: own}}, homes...)
|
||||
}
|
||||
}
|
||||
if len(homes) > 0 {
|
||||
homes[0].Default = true
|
||||
}
|
||||
return homes
|
||||
}
|
||||
|
||||
@@ -394,6 +389,9 @@ func scanHomes(ctx context.Context, homes []stackHome, known []stack) []stack {
|
||||
}
|
||||
var out []stack
|
||||
for _, h := range homes {
|
||||
if _, err := os.Stat(h.Dir); errors.Is(err, os.ErrNotExist) {
|
||||
continue // Ham Lab's own home before its first stack: nothing to find, and no sudo to spend finding it
|
||||
}
|
||||
args := []string{h.Dir, "-mindepth", "2", "-maxdepth", "2", "-type", "f", "("}
|
||||
for i, n := range composeNames {
|
||||
if i > 0 {
|
||||
@@ -417,7 +415,7 @@ func scanHomes(ctx context.Context, homes []stackHome, known []stack) []stack {
|
||||
continue
|
||||
}
|
||||
have[name], have[dir] = true, true
|
||||
st := stack{Name: name, Containers: []string{}, Images: []string{}, HostFiles: []string{f}, HostFile: f, Via: h.Via, Remote: h.Remote, HeldBy: h.HeldBy, Editable: true, Note: "not deployed"}
|
||||
st := stack{Name: name, Containers: []string{}, Images: []string{}, HostFiles: []string{f}, HostFile: f, Via: h.Via, Editable: true, Note: "not deployed"}
|
||||
st.Files = []string{filepath.Join(h.Inner, filepath.Base(dir), filepath.Base(f))}
|
||||
st.File = st.Files[0]
|
||||
st.HasEnv = readable(st.envPath())
|
||||
@@ -436,114 +434,23 @@ func contains(list []string, s string) bool {
|
||||
return false
|
||||
}
|
||||
|
||||
// hubStacks is a node's stack list with the hub's help: what the node could not find is
|
||||
// looked for here, and if it is here — and the node's Docker answers over tcp, which is how
|
||||
// it was deployed in the first place — the stack becomes editable from this hub.
|
||||
func (s *Server) hubStacks(ctx context.Context, node string) ([]stack, error) {
|
||||
code, b, err := s.nodes.Do(ctx, node, "GET", "/api/docker/stacks", nil, nil)
|
||||
// stackFor finds the stack a request means.
|
||||
func (s *Server) stackFor(ctx context.Context, r *http.Request) (stack, error) {
|
||||
all, err := s.findStacks(ctx)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
var res struct{ Stacks []stack }
|
||||
if code != 200 || json.Unmarshal(b, &res) != nil {
|
||||
return nil, fmt.Errorf("%s answered %d", node, code)
|
||||
}
|
||||
remote, maps := "", []mountMap(nil)
|
||||
for i := range res.Stacks {
|
||||
st := &res.Stacks[i]
|
||||
if st.Editable || len(st.Files) == 0 {
|
||||
continue
|
||||
}
|
||||
if remote == "" {
|
||||
remote, maps = s.nodeDocker(ctx, node), mountsFor(ctx, s.store)
|
||||
}
|
||||
if remote == "-" {
|
||||
continue
|
||||
}
|
||||
note := st.Note
|
||||
if st.resolve(maps); !st.Editable {
|
||||
st.Note = note
|
||||
continue
|
||||
}
|
||||
st.Remote, st.HeldBy = remote, s.hostName()
|
||||
}
|
||||
var held []stack
|
||||
for _, st := range res.Stacks {
|
||||
if st.HeldBy != "" {
|
||||
held = append(held, st)
|
||||
}
|
||||
}
|
||||
res.Stacks = append(res.Stacks, scanHomes(ctx, s.homesOf(held, false), res.Stacks)...)
|
||||
sort.Slice(res.Stacks, func(i, j int) bool { return res.Stacks[i].Name < res.Stacks[j].Name })
|
||||
return res.Stacks, nil
|
||||
}
|
||||
|
||||
var (
|
||||
nodeDockerMu sync.Mutex
|
||||
nodeDockerOK = map[string]string{} // node → "tcp://host:2375", or "-" when it doesn't answer
|
||||
)
|
||||
|
||||
// nodeDocker is the node's Docker endpoint as Dockhand-style deployments use it, if it answers.
|
||||
func (s *Server) nodeDocker(ctx context.Context, node string) string {
|
||||
nodeDockerMu.Lock()
|
||||
defer nodeDockerMu.Unlock()
|
||||
if v, ok := nodeDockerOK[node]; ok && v != "-" {
|
||||
return v
|
||||
}
|
||||
ep := "-"
|
||||
for _, n := range s.nodes.List() {
|
||||
if n.Name != node {
|
||||
continue
|
||||
}
|
||||
if u, err := url.Parse(n.URL); err == nil && u.Hostname() != "" {
|
||||
cand := "tcp://" + u.Hostname() + ":2375"
|
||||
c, cancel := context.WithTimeout(ctx, 5*time.Second)
|
||||
if exec.CommandContext(c, dockerTool(), "-H", cand, "version", "--format", "{{.Server.Version}}").Run() == nil {
|
||||
ep = cand
|
||||
}
|
||||
cancel()
|
||||
}
|
||||
}
|
||||
nodeDockerOK[node] = ep
|
||||
return ep
|
||||
}
|
||||
|
||||
// stackFor finds the stack a request means: on this machine, or (on a hub route) one of a
|
||||
// node's stacks that this hub holds the files for. handled is false when the request should
|
||||
// simply be passed on to the node.
|
||||
func (s *Server) stackFor(ctx context.Context, r *http.Request) (st stack, handled bool, err error) {
|
||||
var all []stack
|
||||
if node := r.PathValue("node"); node != "" {
|
||||
all, err = s.hubStacks(ctx, node)
|
||||
} else {
|
||||
all, err = s.findStacks(ctx)
|
||||
}
|
||||
if err != nil {
|
||||
return stack{}, true, err
|
||||
return stack{}, err
|
||||
}
|
||||
for _, x := range all {
|
||||
if x.Name == r.PathValue("name") {
|
||||
if r.PathValue("node") != "" && x.Remote == "" {
|
||||
return x, false, nil // the node can serve this one itself
|
||||
}
|
||||
return x, true, nil
|
||||
return x, nil
|
||||
}
|
||||
}
|
||||
return stack{}, true, errors.New("no such stack")
|
||||
return stack{}, errors.New("no such stack")
|
||||
}
|
||||
|
||||
func (s *Server) stacksList(w http.ResponseWriter, r *http.Request) {
|
||||
ctx, cancel := context.WithTimeout(r.Context(), 40*time.Second)
|
||||
defer cancel()
|
||||
if node := r.PathValue("node"); node != "" {
|
||||
stacks, err := s.hubStacks(ctx, node)
|
||||
if err != nil {
|
||||
s.proxyNode(w, r) // let the node's own answer (or failure) through
|
||||
return
|
||||
}
|
||||
jsonOK(w, 200, map[string]any{"stacks": stacks})
|
||||
return
|
||||
}
|
||||
if dockerTool() == "" { // no Docker is an empty answer, not a failure
|
||||
jsonOK(w, 200, map[string]any{"stacks": []stack{}})
|
||||
return
|
||||
@@ -587,11 +494,7 @@ func target(r *http.Request, st stack) (host string, index int, ok bool) {
|
||||
func (s *Server) stackFile(w http.ResponseWriter, r *http.Request) {
|
||||
ctx, cancel := context.WithTimeout(r.Context(), 40*time.Second)
|
||||
defer cancel()
|
||||
st, handled, err := s.stackFor(ctx, r)
|
||||
if !handled {
|
||||
s.proxyNode(w, r)
|
||||
return
|
||||
}
|
||||
st, err := s.stackFor(ctx, r)
|
||||
if err != nil || !st.Editable {
|
||||
jsonErr(w, 404, "that stack's compose file isn't anywhere this console can reach")
|
||||
return
|
||||
@@ -610,8 +513,8 @@ func (s *Server) stackFile(w http.ResponseWriter, r *http.Request) {
|
||||
jsonOK(w, 200, map[string]any{"stack": st, "text": string(b), "sum": sumOf(b), "index": i})
|
||||
}
|
||||
|
||||
// composeCmd is `docker compose …` for this stack, run where its files make sense and
|
||||
// pointed at the Docker that runs its containers. files are given as compose will see them.
|
||||
// composeCmd is `docker compose …` for this stack, run where its files make sense. files are
|
||||
// given as compose will see them.
|
||||
func composeCmd(ctx context.Context, st stack, files []string, args ...string) *exec.Cmd {
|
||||
return compose.Cmd(ctx, specOf(st, files), args...)
|
||||
}
|
||||
@@ -619,7 +522,7 @@ func composeCmd(ctx context.Context, st stack, files []string, args ...string) *
|
||||
// specOf is a stack as the compose runner needs it — and as a scheduled update stores it,
|
||||
// so the cron job and the console invoke compose identically.
|
||||
func specOf(st stack, files []string) compose.Spec {
|
||||
return compose.Spec{Project: st.Name, Files: files, Via: st.Via, Remote: st.Remote, Tool: dockerTool()}
|
||||
return compose.Spec{Project: st.Name, Files: files, Via: st.Via, Tool: dockerTool()}
|
||||
}
|
||||
|
||||
// seenFiles is the stack's file list as compose runs it, with file i replaced by a candidate.
|
||||
@@ -647,11 +550,7 @@ func (s *Server) writeAsOwner(ctx context.Context, path, text string) error {
|
||||
func (s *Server) stackSave(w http.ResponseWriter, r *http.Request) {
|
||||
ctx, cancel := context.WithTimeout(r.Context(), 60*time.Second)
|
||||
defer cancel()
|
||||
st, handled, err := s.stackFor(ctx, r)
|
||||
if !handled {
|
||||
s.proxyNode(w, r)
|
||||
return
|
||||
}
|
||||
st, err := s.stackFor(ctx, r)
|
||||
if err != nil || !st.Editable {
|
||||
jsonErr(w, 404, "that stack's compose file isn't anywhere this console can reach")
|
||||
return
|
||||
@@ -723,11 +622,7 @@ func (s *Server) stackSave(w http.ResponseWriter, r *http.Request) {
|
||||
func (s *Server) stackDiff(w http.ResponseWriter, r *http.Request) {
|
||||
ctx, cancel := context.WithTimeout(r.Context(), 40*time.Second)
|
||||
defer cancel()
|
||||
st, handled, err := s.stackFor(ctx, r)
|
||||
if !handled {
|
||||
s.proxyNode(w, r)
|
||||
return
|
||||
}
|
||||
st, err := s.stackFor(ctx, r)
|
||||
if err != nil || !st.Editable {
|
||||
jsonErr(w, 404, "that stack's compose file isn't anywhere this console can reach")
|
||||
return
|
||||
@@ -777,11 +672,7 @@ func maskEnv(text string) string {
|
||||
func (s *Server) stackAction(w http.ResponseWriter, r *http.Request) {
|
||||
ctx, cancel := context.WithTimeout(r.Context(), 40*time.Second)
|
||||
defer cancel()
|
||||
st, handled, err := s.stackFor(ctx, r)
|
||||
if !handled {
|
||||
s.proxyNode(w, r)
|
||||
return
|
||||
}
|
||||
st, err := s.stackFor(ctx, r)
|
||||
if err != nil || !st.Editable {
|
||||
jsonErr(w, 404, "that stack's compose file isn't anywhere this console can reach")
|
||||
return
|
||||
@@ -794,17 +685,10 @@ func (s *Server) stackAction(w http.ResponseWriter, r *http.Request) {
|
||||
flags := priv.Quote(append([]string{"-p", st.Name}, fl...))
|
||||
base := func(args string) string {
|
||||
if st.Via != "" {
|
||||
env := ""
|
||||
if st.Remote != "" {
|
||||
env = "-e " + priv.Quote([]string{"DOCKER_HOST=" + st.Remote}) + " "
|
||||
// The CLI inside the orchestrator's container is its own `docker`, whatever this host runs.
|
||||
return fmt.Sprintf("%s exec %s docker compose %s %s", dockerTool(), priv.Quote([]string{st.Via}), flags, args)
|
||||
}
|
||||
return fmt.Sprintf("%s exec %s%s docker compose %s %s", dockerTool(), env, priv.Quote([]string{st.Via}), flags, args)
|
||||
}
|
||||
env := ""
|
||||
if st.Remote != "" { // through env(1): a quoted NAME=value in front of a command is a command name to sh, not an assignment
|
||||
env = "env " + priv.Quote([]string{"DOCKER_HOST=" + st.Remote}) + " "
|
||||
}
|
||||
return fmt.Sprintf("cd %s && %s%s compose %s %s", priv.Quote([]string{filepath.Dir(files[0])}), env, dockerTool(), flags, args)
|
||||
return fmt.Sprintf("cd %s && %s compose %s %s", priv.Quote([]string{filepath.Dir(files[0])}), dockerTool(), flags, args)
|
||||
}
|
||||
// After a deploy that worked, keep a copy of exactly what was deployed: the next diff is against it.
|
||||
remember := ""
|
||||
@@ -860,7 +744,7 @@ func (s *Server) stackAction(w http.ResponseWriter, r *http.Request) {
|
||||
_ = exec.CommandContext(ctx, dockerTool(), "exec", via, "pkill", "-TERM", "-f", pattern).Run()
|
||||
})
|
||||
}
|
||||
stackUpdatesForget(r.PathValue("node") + "/" + st.Name)
|
||||
stackUpdatesForget(st.Name)
|
||||
log.Printf("stack %s: %s started as task %s (from %s)", st.Name, r.PathValue("action"), t.ID, clientIP(r))
|
||||
jsonOK(w, 202, t.Snapshot(false))
|
||||
}
|
||||
@@ -885,20 +769,13 @@ var (
|
||||
|
||||
func stackUpdatesForget(key string) { updMu.Lock(); delete(updCache, key); updMu.Unlock() }
|
||||
|
||||
// checkImages asks the registry whether each image is still the one we hold. remote is a
|
||||
// DOCKER_HOST for a machine whose images live elsewhere, "" for this one. Each image is one
|
||||
// registry request, which Docker Hub counts — so this is only ever called when asked for.
|
||||
func checkImages(ctx context.Context, images []string, remote string) []imageCheck {
|
||||
local := func(args ...string) ([]byte, error) { // the Docker that holds the images
|
||||
if remote != "" {
|
||||
args = append([]string{"-H", remote}, args...)
|
||||
}
|
||||
return dockerRun(ctx, args...)
|
||||
}
|
||||
// checkImages asks the registry whether each image is still the one we hold. Each image is
|
||||
// one registry request, which Docker Hub counts — so this is only ever called when asked for.
|
||||
func checkImages(ctx context.Context, images []string) []imageCheck {
|
||||
res := []imageCheck{}
|
||||
for _, img := range images {
|
||||
ic := imageCheck{Image: img, State: "unknown"}
|
||||
out, err := local("image", "inspect", "-f", "{{json .RepoDigests}}", img)
|
||||
out, err := dockerRun(ctx, "image", "inspect", "-f", "{{json .RepoDigests}}", img)
|
||||
var have []string
|
||||
if err != nil || json.Unmarshal(bytes.TrimSpace(out), &have) != nil || len(have) == 0 {
|
||||
ic.Detail = "built or loaded here, not pulled from a registry"
|
||||
@@ -930,31 +807,24 @@ func checkImages(ctx context.Context, images []string, remote string) []imageChe
|
||||
}
|
||||
|
||||
// machineUpdates checks every image on this machine in one pass, so the container list can
|
||||
// be coloured and the fleet can be asked "what is behind?" without walking stack by stack.
|
||||
// be coloured and "what is behind?" answered without walking stack by stack.
|
||||
func (s *Server) machineUpdates(w http.ResponseWriter, r *http.Request) {
|
||||
node := r.PathValue("node")
|
||||
ctx, cancel := context.WithTimeout(r.Context(), 10*time.Minute)
|
||||
defer cancel()
|
||||
stacks, err := s.findStacks(ctx)
|
||||
if node != "" {
|
||||
stacks, err = s.hubStacks(ctx, node)
|
||||
}
|
||||
if err != nil {
|
||||
jsonErr(w, 500, err.Error())
|
||||
return
|
||||
}
|
||||
// one entry per image, however many stacks use it
|
||||
type want struct{ remote string }
|
||||
imgs := map[string]want{}
|
||||
byImage := map[string][]string{} // image -> stacks
|
||||
for _, st := range stacks {
|
||||
for _, img := range st.Images {
|
||||
imgs[img] = want{remote: st.Remote}
|
||||
byImage[img] = append(byImage[img], st.Name)
|
||||
}
|
||||
}
|
||||
refresh := r.URL.Query().Get("refresh") == "1"
|
||||
key := "machine:" + node
|
||||
key := "machine"
|
||||
updMu.Lock()
|
||||
c, ok := updCache[key]
|
||||
updMu.Unlock()
|
||||
@@ -967,15 +837,11 @@ func (s *Server) machineUpdates(w http.ResponseWriter, r *http.Request) {
|
||||
return
|
||||
}
|
||||
var list []string
|
||||
remote := ""
|
||||
for img, wnt := range imgs {
|
||||
for img := range byImage {
|
||||
list = append(list, img)
|
||||
if wnt.remote != "" {
|
||||
remote = wnt.remote
|
||||
}
|
||||
}
|
||||
sort.Strings(list)
|
||||
res := checkImages(ctx, list, remote)
|
||||
res := checkImages(ctx, list)
|
||||
updMu.Lock()
|
||||
updCache[key] = updAnswer{time.Now(), res}
|
||||
updMu.Unlock()
|
||||
@@ -988,16 +854,12 @@ func (s *Server) machineUpdates(w http.ResponseWriter, r *http.Request) {
|
||||
func (s *Server) stackUpdates(w http.ResponseWriter, r *http.Request) {
|
||||
ctx, cancel := context.WithTimeout(r.Context(), 90*time.Second)
|
||||
defer cancel()
|
||||
st, handled, err := s.stackFor(ctx, r)
|
||||
if !handled {
|
||||
s.proxyNode(w, r)
|
||||
return
|
||||
}
|
||||
st, err := s.stackFor(ctx, r)
|
||||
if err != nil {
|
||||
jsonErr(w, 404, err.Error())
|
||||
return
|
||||
}
|
||||
key := r.PathValue("node") + "/" + st.Name
|
||||
key := st.Name
|
||||
updMu.Lock()
|
||||
c, ok := updCache[key]
|
||||
updMu.Unlock()
|
||||
@@ -1010,84 +872,38 @@ func (s *Server) stackUpdates(w http.ResponseWriter, r *http.Request) {
|
||||
jsonOK(w, 200, map[string]any{"images": []imageCheck{}, "checked": 0})
|
||||
return
|
||||
}
|
||||
res := checkImages(ctx, st.Images, st.Remote)
|
||||
res := checkImages(ctx, st.Images)
|
||||
updMu.Lock()
|
||||
updCache[key] = updAnswer{time.Now(), res}
|
||||
updMu.Unlock()
|
||||
jsonOK(w, 200, map[string]any{"images": res, "checked": time.Now().UnixMilli()})
|
||||
}
|
||||
|
||||
// taskOnHub serves a task this hub is running for one of its nodes' stacks; anything else
|
||||
// under /api/nodes/<node>/tasks belongs to the node.
|
||||
func (s *Server) taskOnHub(local http.HandlerFunc) http.HandlerFunc {
|
||||
return func(w http.ResponseWriter, r *http.Request) {
|
||||
if _, ok := s.tasks.Get(r.PathValue("id")); ok {
|
||||
local(w, r)
|
||||
return
|
||||
}
|
||||
s.proxyNode(w, r)
|
||||
}
|
||||
}
|
||||
|
||||
// ---- making and unmaking stacks ----
|
||||
|
||||
var stackNameRe = regexp.MustCompile(`^[a-z0-9][a-z0-9_-]{0,62}$`)
|
||||
|
||||
// homesFor is where a new stack could go, seen from this request: this machine's homes, or —
|
||||
// on a hub route — the node's own homes plus the ones this hub holds for it.
|
||||
func (s *Server) homesFor(ctx context.Context, r *http.Request) ([]stackHome, error) {
|
||||
node := r.PathValue("node")
|
||||
if node == "" {
|
||||
// homesFor is where a new stack could go on this machine.
|
||||
func (s *Server) homesFor(ctx context.Context) ([]stackHome, error) {
|
||||
stacks, err := s.findStacks(ctx)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
return s.homesOf(stacks, true), nil
|
||||
}
|
||||
stacks, err := s.hubStacks(ctx, node)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
var held []stack
|
||||
for _, st := range stacks {
|
||||
if st.HeldBy != "" {
|
||||
held = append(held, st)
|
||||
}
|
||||
}
|
||||
homes := s.homesOf(held, false)
|
||||
if code, b, err := s.nodes.Do(ctx, node, "GET", "/api/docker/stacks/homes", nil, nil); err == nil && code == 200 {
|
||||
var res struct{ Homes []stackHome }
|
||||
if json.Unmarshal(b, &res) == nil {
|
||||
for _, h := range res.Homes {
|
||||
if len(homes) > 0 && h.Stacks == 0 {
|
||||
continue // the node's empty ~/stacks is only worth offering when there is nothing else
|
||||
}
|
||||
h.Default = false
|
||||
homes = append(homes, h)
|
||||
}
|
||||
}
|
||||
}
|
||||
if len(homes) > 0 {
|
||||
homes[0].Default = true
|
||||
}
|
||||
return homes, nil
|
||||
return s.homesOf(stacks), nil
|
||||
}
|
||||
|
||||
func (s *Server) stackHomes(w http.ResponseWriter, r *http.Request) {
|
||||
ctx, cancel := context.WithTimeout(r.Context(), 40*time.Second)
|
||||
defer cancel()
|
||||
if dockerTool() == "" && r.PathValue("node") == "" {
|
||||
if dockerTool() == "" {
|
||||
jsonOK(w, 200, map[string]any{"homes": []stackHome{}})
|
||||
return
|
||||
}
|
||||
homes, err := s.homesFor(ctx, r)
|
||||
homes, err := s.homesFor(ctx)
|
||||
if err != nil {
|
||||
jsonErr(w, 500, err.Error())
|
||||
return
|
||||
}
|
||||
if homes == nil { // a machine with no Docker and nothing held for it: an empty list, never null
|
||||
homes = []stackHome{}
|
||||
}
|
||||
jsonOK(w, 200, map[string]any{"homes": homes})
|
||||
}
|
||||
|
||||
@@ -1114,7 +930,7 @@ func (s *Server) stackCreate(w http.ResponseWriter, r *http.Request) {
|
||||
jsonErr(w, 400, "the compose file can't be empty")
|
||||
return
|
||||
}
|
||||
homes, err := s.homesFor(ctx, r)
|
||||
homes, err := s.homesFor(ctx)
|
||||
if err != nil {
|
||||
jsonErr(w, 500, err.Error())
|
||||
return
|
||||
@@ -1129,17 +945,7 @@ func (s *Server) stackCreate(w http.ResponseWriter, r *http.Request) {
|
||||
jsonErr(w, 400, "that isn't one of the places this machine keeps stacks")
|
||||
return
|
||||
}
|
||||
if r.PathValue("node") != "" && home.HeldBy == "" { // the node's own folder: the node makes it
|
||||
r.Body, r.ContentLength = io.NopCloser(bytes.NewReader(raw)), int64(len(raw))
|
||||
s.proxyNode(w, r)
|
||||
return
|
||||
}
|
||||
var all []stack
|
||||
if node := r.PathValue("node"); node != "" {
|
||||
all, _ = s.hubStacks(ctx, node)
|
||||
} else {
|
||||
all, _ = s.findStacks(ctx)
|
||||
}
|
||||
all, _ := s.findStacks(ctx)
|
||||
for _, st := range all {
|
||||
if st.Name == name {
|
||||
jsonErr(w, 409, "there is already a stack called "+name)
|
||||
@@ -1201,7 +1007,7 @@ func (s *Server) stackCreate(w http.ResponseWriter, r *http.Request) {
|
||||
_ = sudo("chmod", "600", filepath.Join(dir, ".env")) // it is where the secrets go
|
||||
}
|
||||
st := stack{Name: name, Containers: []string{}, Images: []string{}, HostFiles: []string{host}, HostFile: host, Files: []string{filepath.Join(home.Inner, name, "compose.yaml")},
|
||||
Via: home.Via, Remote: home.Remote, HeldBy: home.HeldBy, Editable: true, Note: "not deployed", HasEnv: strings.TrimSpace(body.Env) != ""}
|
||||
Via: home.Via, Editable: true, Note: "not deployed", HasEnv: strings.TrimSpace(body.Env) != ""}
|
||||
st.File = st.Files[0]
|
||||
if msg, err := composeCmd(ctx, st, seenFiles(st, -1, ""), "config", "-q").CombinedOutput(); err != nil {
|
||||
undo()
|
||||
@@ -1220,12 +1026,7 @@ func (s *Server) stackDelete(w http.ResponseWriter, r *http.Request) {
|
||||
ctx, cancel := context.WithTimeout(r.Context(), 4*time.Minute)
|
||||
defer cancel()
|
||||
raw, _ := io.ReadAll(io.LimitReader(r.Body, 4096))
|
||||
st, handled, err := s.stackFor(ctx, r)
|
||||
if !handled {
|
||||
r.Body, r.ContentLength = io.NopCloser(bytes.NewReader(raw)), int64(len(raw))
|
||||
s.proxyNode(w, r)
|
||||
return
|
||||
}
|
||||
st, err := s.stackFor(ctx, r)
|
||||
if err != nil || !st.Editable {
|
||||
jsonErr(w, 404, "that stack's compose file isn't anywhere this console can reach")
|
||||
return
|
||||
@@ -1263,7 +1064,7 @@ func (s *Server) stackDelete(w http.ResponseWriter, r *http.Request) {
|
||||
}
|
||||
}
|
||||
_ = run("rmdir", dir) // only if nothing else was in it
|
||||
stackUpdatesForget(r.PathValue("node") + "/" + st.Name)
|
||||
stackUpdatesForget(st.Name)
|
||||
log.Printf("stack %s: taken down and %d file(s) moved to %s (from %s)", st.Name, moved, trash, clientIP(r))
|
||||
jsonOK(w, 200, map[string]any{"trash": trash, "moved": moved})
|
||||
}
|
||||
|
||||
@@ -7,6 +7,7 @@ import (
|
||||
"testing"
|
||||
|
||||
"hamlab/internal/compose"
|
||||
"hamlab/internal/config"
|
||||
"hamlab/internal/store"
|
||||
)
|
||||
|
||||
@@ -119,3 +120,37 @@ func TestRepointSurvivesAMissingContainer(t *testing.T) {
|
||||
t.Errorf("unresolvable spec was changed: %+v", got)
|
||||
}
|
||||
}
|
||||
|
||||
// Where new stacks go: Ham Lab's own <data dir>/stacks is always offered, first and as the
|
||||
// default, whether or not anything is in it yet. A folder this machine already keeps stacks
|
||||
// in — two or more deployed from it — is offered too, but never ahead of Ham Lab's own.
|
||||
func TestHomesDefaultToDataDir(t *testing.T) {
|
||||
root := t.TempDir()
|
||||
cfg := config.Default(filepath.Join(root, "data", "config.json"))
|
||||
s := &Server{cfg: cfg}
|
||||
own := filepath.Join(root, "data", "stacks")
|
||||
legacy := filepath.Join(root, "home", "stacks", "box")
|
||||
deployed := func(dir, name string) stack {
|
||||
f := filepath.Join(dir, name, "compose.yaml")
|
||||
return stack{Name: name, Containers: []string{name + "-1"}, Files: []string{f}, File: f, HostFiles: []string{f}, HostFile: f, Editable: true}
|
||||
}
|
||||
|
||||
// Nothing deployed anywhere: one home, Ham Lab's own.
|
||||
homes := s.homesOf(nil)
|
||||
if len(homes) != 1 || homes[0].Dir != own || !homes[0].Default {
|
||||
t.Fatalf("empty machine: homes = %+v, want just %s as default", homes, own)
|
||||
}
|
||||
|
||||
// Two stacks deployed from a pre-existing folder: it is offered, after Ham Lab's own.
|
||||
homes = s.homesOf([]stack{deployed(legacy, "a"), deployed(legacy, "b")})
|
||||
if len(homes) != 2 || homes[0].Dir != own || !homes[0].Default || homes[1].Dir != legacy || homes[1].Default || homes[1].Stacks != 2 {
|
||||
t.Fatalf("legacy folder: homes = %+v, want [%s (default), %s]", homes, own, legacy)
|
||||
}
|
||||
|
||||
// One lone stack elsewhere says nothing about where the next should go; one in Ham Lab's
|
||||
// own folder is already home.
|
||||
homes = s.homesOf([]stack{deployed(filepath.Join(root, "home", "apps"), "orchestrator"), deployed(own, "c")})
|
||||
if len(homes) != 1 || homes[0].Dir != own || homes[0].Stacks != 1 || !homes[0].Default {
|
||||
t.Fatalf("lone stacks: homes = %+v, want just %s", homes, own)
|
||||
}
|
||||
}
|
||||
|
||||
@@ -1,14 +1,13 @@
|
||||
// Package compose runs docker compose the way a stack was deployed — which is not always
|
||||
// simply "on this host". A stack deployed by an orchestrator running inside its own
|
||||
// container records paths in that container's filesystem, so compose has to run in there;
|
||||
// a stack deployed to another machine over tcp needs DOCKER_HOST pointed at it. Both the
|
||||
// console and the scheduled updates it writes must invoke it identically, or a redeploy
|
||||
// from one would not match the other's labels.
|
||||
// container records paths in that container's filesystem, so compose has to run in there.
|
||||
// Both the console and the scheduled updates it writes must invoke it identically, or a
|
||||
// redeploy from one would not match the other's labels. Compose always drives this
|
||||
// machine's own Docker: a stack's files and its containers are on the same box.
|
||||
package compose
|
||||
|
||||
import (
|
||||
"context"
|
||||
"os"
|
||||
"os/exec"
|
||||
"path/filepath"
|
||||
)
|
||||
@@ -18,7 +17,6 @@ type Spec struct {
|
||||
Project string // -p
|
||||
Files []string // -f, in order
|
||||
Via string // a container to run compose inside; "" is this host
|
||||
Remote string // a DOCKER_HOST; "" is this machine's Docker
|
||||
Tool string // "docker" or "podman"
|
||||
}
|
||||
|
||||
@@ -33,18 +31,12 @@ func Cmd(ctx context.Context, s Spec, args ...string) *exec.Cmd {
|
||||
base = append(base, "-f", f)
|
||||
}
|
||||
if s.Via != "" {
|
||||
pre := []string{"exec"}
|
||||
if s.Remote != "" {
|
||||
pre = append(pre, "-e", "DOCKER_HOST="+s.Remote)
|
||||
}
|
||||
return exec.CommandContext(ctx, tool, append(append(append(pre, s.Via, "docker"), base...), args...)...)
|
||||
// The CLI inside the orchestrator's container is its own `docker`, whatever this host runs.
|
||||
return exec.CommandContext(ctx, tool, append(append([]string{"exec", s.Via, "docker"}, base...), args...)...)
|
||||
}
|
||||
cmd := exec.CommandContext(ctx, tool, append(base, args...)...)
|
||||
if len(s.Files) > 0 {
|
||||
cmd.Dir = filepath.Dir(s.Files[0])
|
||||
}
|
||||
if s.Remote != "" {
|
||||
cmd.Env = append(os.Environ(), "DOCKER_HOST="+s.Remote)
|
||||
}
|
||||
return cmd
|
||||
}
|
||||
|
||||
+2
-2
@@ -355,7 +355,7 @@ function StackDetail({ name, startTask, onBack, onChanged, onDuplicate }: { name
|
||||
<div className="svc-detail">
|
||||
<CloseBtn onClick={onBack} />
|
||||
<div className="svc-head"><button className="btn back" onClick={onBack}>‹ Back</button><span className={'sdot ' + (st.containers.length === 0 ? '' : st.running === st.containers.length ? 'ok' : st.running ? 'warn' : 'bad')} /><span className="dname">{st.name}</span><span className="muted small">{st.running}/{st.containers.length} running</span></div>
|
||||
<div className="muted small mono" style={{ overflowWrap: 'anywhere' }}>{file === 'env' ? (st.host_file.replace(/[^/]+$/, '') + '.env') : (st.host_files?.[file] ?? st.host_file)}{st.held_by && <span className="muted"> · kept on <b>{st.held_by}</b>, deployed to this machine over {st.remote}</span>}{st.via && <span className="muted"> · compose runs inside <b>{st.via}</b>, as it was deployed</span>}{st.has_env && ' · has a .env beside it'}</div>
|
||||
<div className="muted small mono" style={{ overflowWrap: 'anywhere' }}>{file === 'env' ? (st.host_file.replace(/[^/]+$/, '') + '.env') : (st.host_files?.[file] ?? st.host_file)}{st.via && <span className="muted"> · compose runs inside <b>{st.via}</b>, as it was deployed</span>}{st.has_env && ' · has a .env beside it'}</div>
|
||||
{((st.host_files?.length ?? 0) > 1 || st.has_env) && <div className="chips" title="this stack is deployed from several files, in this order; Deploy uses all of them">{st.host_files!.map((f, i) => <button key={f} className={'chip' + (i === file ? ' on' : '')} onClick={async () => { if (dirty && !(await askYes('Discard unsaved changes to this file?', 'Discard'))) return; setFile(i) }}>{f.split('/').pop()}</button>)}{st.has_env && <button className={'chip' + (file === 'env' ? ' on' : '')} title="the .env beside the compose file: variables compose substitutes" onClick={async () => { if (dirty && !(await askYes('Discard unsaved changes to this file?', 'Discard'))) return; setFile('env') }}>.env</button>}</div>}
|
||||
{file === 'env' && !reveal ? <div className="env-masked"><pre className="editor stack-editor">{maskEnv(text)}</pre><button className="btn" onClick={() => setReveal(true)}>Reveal and edit</button></div> :
|
||||
<CodeEditor className="stack-editor" language={file === 'env' ? 'env' : 'yaml'} value={text} onChange={setText} />}
|
||||
@@ -447,7 +447,7 @@ export default function Docker() {
|
||||
{shownTab === 'stacks' && (stacks ?? []).map((st) => (
|
||||
<div key={st.name} className={'svc-row' + (st.name === selStack ? ' active' : '')} onClick={() => st.editable ? setSel('stack:' + st.name) : toast(st.note ?? 'not editable here', 'info')}>
|
||||
<span className={'sdot ' + (st.containers.length === 0 ? '' : st.running === st.containers.length ? 'ok' : st.running ? 'warn' : 'bad')} />
|
||||
<div className="svc-name"><div className="n">{st.name}</div><div className="d">{st.editable ? (st.held_by ? `on ${st.held_by}: ` : '') + st.host_file : st.note}</div></div>
|
||||
<div className="svc-name"><div className="n">{st.name}</div><div className="d">{st.editable ? st.host_file : st.note}</div></div>
|
||||
<div className="svc-state">{st.containers.length === 0 ? 'not deployed' : `${st.running}/${st.containers.length}`}{!st.editable && <><br />read-only</>}</div>
|
||||
</div>
|
||||
))}
|
||||
|
||||
@@ -101,7 +101,7 @@ export default function StackNew({ mode, from, networks = [], taken, onClose, on
|
||||
{homes.length > 1 && <label className="muted">Where
|
||||
<select value={home} onChange={(e) => setHome(e.target.value)}>{homes.map((h) => <option key={h.dir + h.via} value={h.dir}>{sameTail(h) ? h.dir : tail(h.dir)} · {h.stacks} stack{h.stacks === 1 ? '' : 's'}</option>)}</select>
|
||||
</label>}
|
||||
{picked && <div className="muted small mono" style={{ overflowWrap: 'anywhere' }}>{picked.dir}/{name || '…'}/compose.yaml{picked.held_by && ` — kept on ${picked.held_by}, deployed here over ${picked.remote}`}{picked.via && ` — compose runs inside ${picked.via}, like its neighbours`}</div>}
|
||||
{picked && <div className="muted small mono" style={{ overflowWrap: 'anywhere' }}>{picked.dir}/{name || '…'}/compose.yaml{picked.via && ` — compose runs inside ${picked.via}, like its neighbours`}</div>}
|
||||
{!from && <div className="tabs"><button className={how === 'form' ? 'on' : ''} onClick={() => setHow('form')}>Form</button><button className={how === 'compose' ? 'on' : ''} onClick={toCompose}>Compose</button></div>}
|
||||
{how === 'form' ? <>
|
||||
<label className="muted">Image<input className="mono" value={form.image} onChange={set('image')} placeholder="traefik/whoami:latest" spellCheck={false} autoCapitalize="off" autoCorrect="off" /></label>
|
||||
|
||||
+2
-2
@@ -154,8 +154,8 @@ export interface Hardware { at: string; system: HWSystem; board: HWBoard; cpu: H
|
||||
|
||||
export interface DockerLogLine { t: string; m: string; err?: boolean }
|
||||
export interface DockerInfo { available: boolean; message?: string; tool?: string; containers: DockerContainer[]; images: Record<string, unknown>[]; volumes: Record<string, unknown>[]; networks: Record<string, unknown>[] }
|
||||
export interface Stack { name: string; containers: string[]; running: number; file: string; host_file: string; files: string[]; host_files: string[] | null; images: string[] | null; via: string; remote: string; held_by: string; editable: boolean; note?: string; has_env: boolean }
|
||||
export interface StackHome { dir: string; inner: string; via: string; remote: string; held_by: string; stacks: number; default: boolean }
|
||||
export interface Stack { name: string; containers: string[]; running: number; file: string; host_file: string; files: string[]; host_files: string[] | null; images: string[] | null; via: string; editable: boolean; note?: string; has_env: boolean }
|
||||
export interface StackHome { dir: string; inner: string; via: string; stacks: number; default: boolean }
|
||||
export interface ImageCheck { image: string; state: 'current' | 'update' | 'unknown'; detail?: string }
|
||||
export interface StackDiff { name: string; base: '' | 'deployed' | 'saved'; old: string; new: string }
|
||||
export interface AlertRule { id: string; name: string; sensor: string; op: string; value: number | boolean | string; for_sec: number; severity: 'info' | 'warn' | 'crit'; enabled: boolean; machines: string }
|
||||
|
||||
Reference in New Issue
Block a user