docker: the eight things Dockhand did that Ham Lab did not

Dockhand is being retired, so the Docker tab has to cover what it actually
earned its keep doing. Eight pieces, all per container or per machine:

- Stats: CPU, memory and IO over time, from sensors already collected.
- Health: the container's own healthcheck, with the last few probe outputs —
  which is where a failing check says why.
- Events: `docker events` keeps a minute of backlog and nothing more, so Ham
  Lab follows the stream and keeps a ring of its own. A restart at 4am is
  still there at 9.
- Shell: `docker exec` into a container, in the terminal already here.
- Limits: memory, CPU and restart policy changed live, without hand-editing
  the compose file for a number you want to try.
- Updates: what the registry has that this machine does not, for every image
  at once rather than one stack at a time.
- The same across every machine, asked of the hub — which is the only thing
  that can reach them all.
- Schedules: a real line in /etc/cron.d/hamlab-docker calling the binary,
  in the same shape as the ZFS snapshot schedules. Dockhand updated DNS and
  ingress nightly from a ticker inside a web app; a cron line keeps running
  when the console does not, and is visible to anyone reading the machine the
  normal way.

internal/compose is new and shared, so a deploy from the console and a deploy
from cron invoke compose identically — including the case where compose has to
run inside the orchestrator's own container, where the paths are that
container's and not the host's.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

cmd/hamlab/main.go

@@ -32,8 +32,8 @@ import (
 	"hamlab/internal/files"
 	"hamlab/internal/metrics"
 	"hamlab/internal/nodes"
-	"hamlab/internal/services"
 	"hamlab/internal/priv"
+	"hamlab/internal/services"
 	"hamlab/internal/setup"
 	"hamlab/internal/space"
 	"hamlab/internal/storage"
@@ -62,6 +62,7 @@ func main() {
   passwd [user]        set a user's password (makes the account if it is new; reads stdin when piped)
   users                list the accounts
   zfs-snapshot <id>    run one ZFS snapshot schedule (what its cron line calls)
+  docker-update <id>   run one image-update schedule (what its cron line calls)
   user-rm <user>       remove an account and sign it out everywhere
   check                verify the config and state load, then exit
   version
@@ -132,6 +133,10 @@ func main() {
 		if err := passwd(cfg, flag.Arg(1)); err != nil {
 			log.Fatal(err)
 		}
+	case "docker-update":
+		if err := dockerUpdateJob(cfg, flag.Arg(1)); err != nil {
+			log.Fatal(err)
+		}
 	case "zfs-snapshot":
 		if err := zfsSnapshot(cfg, flag.Arg(1)); err != nil {
 			log.Fatal(err)
@@ -185,6 +190,48 @@ func main() {
 
 // migrateUsers gives a pre-users config its first account, named after this OS user, so
 // the CLI sees the same accounts the daemon will.
+// dockerUpdateJob pulls a stack's images, and deploys them when the schedule says to. This
+// is what the cron line Ham Lab writes actually calls, so image updates keep happening
+// whether or not the console is up.
+func dockerUpdateJob(cfg *config.Config, id string) error {
+	if strings.TrimSpace(id) == "" {
+		return errors.New("usage: hamlab docker-update <schedule-id>")
+	}
+	st, err := store.Open(filepath.Join(cfg.DataDir(), "state.json"))
+	if err != nil {
+		return err
+	}
+	var raw json.RawMessage
+	st.View(func(s *store.State) { raw = s.DockerJobs })
+	var d struct {
+		Jobs []api.UpdateJob `json:"jobs"`
+	}
+	if len(raw) > 0 {
+		if err := json.Unmarshal(raw, &d); err != nil {
+			return fmt.Errorf("reading the schedules: %w", err)
+		}
+	}
+	for _, j := range d.Jobs {
+		if j.ID != id {
+			continue
+		}
+		// A pull that never finishes must not leave a cron job resident until the next reboot.
+		ctx, cancel := context.WithTimeout(context.Background(), 2*time.Hour)
+		defer cancel()
+		out, runErr := api.RunUpdate(ctx, j, api.DockerTool())
+		rec := storage.RunRecord{At: time.Now(), Taken: j.Stack}
+		if runErr != nil {
+			rec.Error = runErr.Error()
+		}
+		if err := storage.WriteRun(cfg.DataDir(), "docker-"+id, rec); err != nil {
+			fmt.Fprintln(os.Stderr, "recording the run:", err)
+		}
+		fmt.Print(out)
+		return runErr
+	}
+	return fmt.Errorf("no image-update schedule %q — it may have been removed; this cron line is stale", id)
+}
+
 // zfsSnapshot runs one snapshot schedule: take it, then expire what has aged out. This is
 // what the cron line Ham Lab writes actually calls, so a schedule keeps running whether or
 // not the console is up — and keeps running if Ham Lab is never opened again.