Compare commits
23
Commits
@@ -625,6 +625,197 @@ something long in one, switch away and back, refresh — it's all still there.
|
|||||||
so both modes share one implementation and one set of rules (absolute paths only;
|
so both modes share one implementation and one set of rules (absolute paths only;
|
||||||
`/`, `/etc`, `/usr`, … are never deleted or moved whole). Every write is logged with
|
`/`, `/etc`, `/usr`, … are never deleted or moved whole). Every write is logged with
|
||||||
the caller's IP.
|
the caller's IP.
|
||||||
|
- **Previews on the Files tab** — **shipped 2026-10-11** (v0.1.0-200, `internal/thumbs`,
|
||||||
|
`internal/api/thumbs.go`, `Thumb`/`ThumbNote` in `Files.tsx`). Asked for as a button that
|
||||||
|
toggles a file preview, "obviously not for everything", cached, "use whatever GPU is on the
|
||||||
|
machine we're looking at, and fall back to CPU", "a single small frame preview so it's
|
||||||
|
fast". A **previews** button beside `.hidden` turns the icon column into a 256 px
|
||||||
|
thumbnail for every image and video row (the listing says which: `preview: image|video`
|
||||||
|
per entry, decided by extension in one place, `thumbs.Kind`), loaded as rows scroll within
|
||||||
|
400 px of view — the list scrolls in a box of its own, so it has its own
|
||||||
|
IntersectionObserver rather than the browser's window-based lazy loading. A video thumb
|
||||||
|
carries a ▶ badge. The setting is in localStorage and follows the browser.
|
||||||
|
- **ffmpeg is the one engine**, for images and video alike; `GET /api/files/thumb` is one
|
||||||
|
ffprobe and one ffmpeg: `-ss` a tenth of the way in (30 s at most, `-noaccurate_seek`
|
||||||
|
so one keyframe is decoded), the first real video stream (cover art is skipped), one
|
||||||
|
frame, `scale` to fit 256, MJPEG q4 to a pipe. JPEGs decode at a reduced DCT size
|
||||||
|
(`-lowres`, enough halvings to stay at twice the preview) — a 24 MP photo is read as
|
||||||
|
1.5 MP. Thumbnails are the only thing Ham Lab asks a GPU to do, and it asks the way the
|
||||||
|
Processes tab reads one: **probed, never assumed.** The probe looks ffmpeg and ffprobe
|
||||||
|
up, opens every `/dev/dri/renderD*`, pushes a synthetic frame through each with VAAPI,
|
||||||
|
and reads `vainfo`'s decode profiles per node where it exists; CUDA is tried the same
|
||||||
|
way when `/dev/nvidiactl` is there (untested: nothing here has one). A video goes to the
|
||||||
|
first GPU listing its codec; ffmpeg's own software fallback sits behind that for a
|
||||||
|
profile the list did not distinguish, and a CPU retry behind a hardware run that fails
|
||||||
|
outright. Images are CPU only — one frame through libjpeg at a quarter size beats a
|
||||||
|
round trip through a GPU. Measured on `grid`: 1080p h264 0.4 s on the UHD 750 against
|
||||||
|
0.7 s on the CPU; the 8K Osmo HEVC 1.0 s against 2.9 s. `grid`'s two GPUs are listed in
|
||||||
|
node order and both decode the same set (h264, hevc, vp9, mjpeg, mpeg2 — **no AV1 on
|
||||||
|
either**, per vainfo, so AV1 is dav1d on the CPU).
|
||||||
|
- **Cached in one database file under the machine's data dir**, `<data dir>/thumbs.db`
|
||||||
|
(bbolt — pure Go, so the binary stays static; v0.1.0-202, the first build kept them as
|
||||||
|
a tree of hash-named JPEGs under `thumbs/`, replaced the same day when Mitchell asked
|
||||||
|
where they were kept: "stored in hamlab, maybe under a DB, up to you"). An entry is
|
||||||
|
keyed by path, size and mtime — a changed file gets a new preview, an unchanged one is
|
||||||
|
never decoded twice — and carries the engine, codec, source size and time beside the
|
||||||
|
picture; a count rides in the file so the previews button's tooltip can say "N kept
|
||||||
|
(M MB)" without walking anything. The URL carries `v=<mtime>-<size>` so the browser's
|
||||||
|
own cache is right too (7 days, ETag, 304). A file ffmpeg could not decode is kept with
|
||||||
|
ffmpeg's reason and no picture, answered as 422, retried only when the file changes; a
|
||||||
|
timeout is not kept. A miss never creates the file, so a machine nobody browses with
|
||||||
|
previews on never has one; a cache that cannot be opened is logged and previews are
|
||||||
|
still made and served, just not kept. Nothing expires: a preview is a few KB, and only
|
||||||
|
files somebody scrolled past are ever made. At most 2–4 ffmpegs at once per machine
|
||||||
|
(half the cores, clamped), one per file however many rows ask, and a preview's work
|
||||||
|
runs on a clock of its own so the row that asked first leaving does not cancel it for
|
||||||
|
the rest. Each ffmpeg runs under `nice -n 10`. `hamlet`'s backup job skips the file.
|
||||||
|
- **Root mode makes them as root** (`hamlab thumb`, the sibling of `hamlab fs`, caps on
|
||||||
|
stdin, JSON out), so a file only root can read gets a preview, and the probe runs as
|
||||||
|
root too — root opens every render node whatever groups the daemon has. Which matters:
|
||||||
|
a render node is `root:render 0660`, and on `core` and `hamtv` the `core` user is not in
|
||||||
|
`render`, so there the daemon's own previews are CPU and the tab says so, naming the GPU
|
||||||
|
and the owner, with an **Allow GPU access** button (confirms, then a task writes
|
||||||
|
`hamlab.service.d/gpu.conf` adding `render`/`video` as supplementary groups and
|
||||||
|
restarts Ham Lab three seconds later, the shape the Incus install uses for
|
||||||
|
`incus-admin`). Not pressed anywhere yet. Where ffmpeg is missing (the Mac mini, `gate`)
|
||||||
|
the note offers **Install ffmpeg with dnf/apk** through the Updates machinery
|
||||||
|
(`ffmpeg-free` on Fedora, `ffmpeg` elsewhere); the status re-probes when the task ends.
|
||||||
|
- *Not done, on purpose:* no cap on the cache and no clear button (not asked; it only
|
||||||
|
holds what was looked at); no tonemapping for HDR sources (a 10-bit HEVC frame comes out
|
||||||
|
flat); no preview in the viewer for a video (it still says "binary, download it").
|
||||||
|
- **Move… is a folder picker** — **shipped 2026-10-11** (v0.1.0-204, `MovePicker` in
|
||||||
|
`Files.tsx`, `internal/files/move.go`, `POST /api/files/move`). Asked for as: instead of
|
||||||
|
a path prompt, "an interactive file explorer, where I can go back/forward, etc and then
|
||||||
|
click some button, and it will move the file/folder to where I was". The picker opens in
|
||||||
|
the item's own folder and walks the machine the way the tab does — back and forward
|
||||||
|
through where it has been, up, breadcrumbs, a path box, folders only, `.hidden`
|
||||||
|
following the tab's toggle, **+ folder** to make the destination on the spot — and
|
||||||
|
**Move here** puts the item in the folder being looked at. It says why it can't when it
|
||||||
|
can't, with the button greyed: already here, a folder into itself, a name clash in that
|
||||||
|
folder (read from the listing before anything is tried). The row being moved shows in
|
||||||
|
its own folder, greyed. Rename stays a name prompt: it is a different question.
|
||||||
|
- **Across filesystems it copies, then removes.** `os.Rename` is EXDEV between
|
||||||
|
filesystems, and on `grid` every dataset is one — `/tank/media` to `/tank/files` was a
|
||||||
|
"invalid cross-device link" error before this, and the picker would have made that
|
||||||
|
move easy to attempt. `/api/files/move` renames when it can (204) and otherwise runs
|
||||||
|
`hamlab fs move` as a task (202 with the task; the tab shows its output and reloads
|
||||||
|
when it ends). The copy is built under a hidden name beside its destination
|
||||||
|
(`.<name>.hamlab-moving`), renamed into place once complete, and only then is the
|
||||||
|
original removed — a failure part-way removes the partial copy and leaves the original
|
||||||
|
as it was, and a cancel (SIGTERM to the helper) does the same. Modes, mtimes and
|
||||||
|
symlinks are kept; owners too when the backend is root. Devices, pipes and sockets are
|
||||||
|
refused outright rather than half-copied. Progress every 500 files or 512 MiB.
|
||||||
|
Verified on `grid` ext4 → ZFS with a tree holding a 0600 file, a symlink and an old
|
||||||
|
mtime, all intact on the other side.
|
||||||
|
- **Video and audio play in the viewer** — **shipped 2026-10-11** (v0.1.0-208). Asked for
|
||||||
|
as "keep it simple for now, and just send whatever codec the file is in to the client,
|
||||||
|
and let the client try to decode and watch it" — and that is the whole design. Opening a
|
||||||
|
video or audio file shows a `<video>`/`<audio>` with the browser's own controls, fed the
|
||||||
|
file as it is by `/api/files/read`; nothing is transcoded or remuxed, so what plays is
|
||||||
|
what the browser decodes (Safari: h264/hevc/aac in mp4/mov; Chrome: those plus vp9/av1 in
|
||||||
|
mkv/webm, hevc only where the machine has a hardware decoder; nothing plays MPEG-TS or a
|
||||||
|
GoPro `.360`'s raw EAC strip). When the element errors the viewer says so and leaves the
|
||||||
|
download. What the server gained is the part a player cannot do without: reads of a
|
||||||
|
real file with a known size go through `http.ServeContent`, so **Range requests** (every
|
||||||
|
seek, and Safari's first request) and HEAD work, and a download can resume. The root
|
||||||
|
backend's helper takes `--offset` for that, and `files.Seeker` opens the backend once, at
|
||||||
|
the range asked for, rather than streaming from the top and skipping; seeking costs
|
||||||
|
nothing until a read wants somewhere else. `/proc` files, which stat at size 0, keep the
|
||||||
|
plain streaming path, as does the editor's capped read. Audio became a preview kind at
|
||||||
|
the same time (`thumbs.Kind` → `audio`): its cover art, where it has any, is the row's
|
||||||
|
thumbnail, with a ♪ badge.
|
||||||
|
- *Not done, on purpose:* no playlist or next/previous, no subtitles. (Transcoding came
|
||||||
|
the same evening after all — the next entry — because Safari turned out not to play
|
||||||
|
the gaming captures.)
|
||||||
|
- **Re-encoding for the browser, on the machine's GPU** — **shipped 2026-10-11**
|
||||||
|
(v0.1.0-212, `internal/media/{encode,hls,ffrun,inspect}.go`, `internal/api/play.go`,
|
||||||
|
`Player.tsx`, `hls.js`). Asked for when Safari would not play the gaming captures:
|
||||||
|
"add GPU transcoding to a supported codec when the client browser can't play it. Each
|
||||||
|
machine should use its own GPU (if available) and transcode to probably H.264. Maybe we
|
||||||
|
need a quality selector, and a codec selector, up to you." What Safari actually refuses
|
||||||
|
is the **MKV container** — the captures checked are H.264 High 4:2:0, 18–36 Mbps — so
|
||||||
|
the picker has a "repackaged" choice beside the codecs, and a 36 Mbps capture over
|
||||||
|
WireGuard is the other reason to re-encode at all.
|
||||||
|
- **The viewer tries the file as it is first**, as before. When the browser gives up, or
|
||||||
|
on **Re-encode…** in the bar, a picker: **codec** — H.264, HEVC, AV1, or the original
|
||||||
|
codec repackaged — each labelled with the engine this machine would use (or greyed as
|
||||||
|
"not on this machine"), and **size** — original, 1080p, 720p, 480p; the choice is
|
||||||
|
remembered. Play starts the re-encode and the player follows it: "h264 High 2292×960
|
||||||
|
60 fps 36 Mbps in matroska → H.264 2292×960 5.0 Mbps on Intel Arc A380 (vaapi) · 7.3× ·
|
||||||
|
34% through — seek within what's done", then "all of it is ready". **Change…** drops
|
||||||
|
the stream and goes back to the picker.
|
||||||
|
- **HLS, made on the machine in the switcher.** `POST /api/files/play` inspects the file
|
||||||
|
(ffprobe, as root in root mode) and runs ffmpeg into an *event* playlist of 4 s
|
||||||
|
fragmented-MP4 segments under `<data dir>/transcode/<id>/`, answering once the first
|
||||||
|
segment exists (0.5–1 s on `grid`); the player reads the playlist and segments from
|
||||||
|
`/api/files/play/<id>/…` (only those three names are served), Safari natively and
|
||||||
|
everything else through hls.js, loaded only then. ffmpeg runs ahead at the engine's
|
||||||
|
pace, so the seekable range grows and the whole file is through long before it is
|
||||||
|
watched. Keyframes are forced every 4 s so segments cut cleanly; sound is AAC stereo
|
||||||
|
unless it already is AAC; HDR is tone-mapped (`tonemap_vaapi`, or zscale/hable on the
|
||||||
|
CPU). Nothing is kept: the same request within ten minutes reuses the stream, the
|
||||||
|
player closes it when it goes (and the "Change…" path does), the sweeper closes what
|
||||||
|
nobody touched for ten minutes, and the folder is emptied when the daemon starts. At
|
||||||
|
most three at once per machine (429 with the reason). Root mode runs ffmpeg as root
|
||||||
|
through `hamlab run-ffmpeg`, which ends when the daemon lets go of its stdin — so a
|
||||||
|
stream cannot outlive the console.
|
||||||
|
- **Engines are probed, never assumed**, the same probe the previews use, now also
|
||||||
|
reading each node's encode profiles from vainfo and ffmpeg's own encoder list. The
|
||||||
|
recipes for a codec are tried best first until one produces a first segment: a card on
|
||||||
|
the bus before the integrated GPU (the A380 before the UHD 750), decoding on the same
|
||||||
|
card so frames never leave it, then decoding on the CPU and uploading (a source the
|
||||||
|
card cannot decode — AV1 on `grid`), then libx264 / libx265 / SVT-AV1. The size is
|
||||||
|
never larger than the source, and the bit rate follows the output size (2/4/8/12/20
|
||||||
|
Mbps for 480p/720p/1080p/1440p/2160p, +25 % at 50 fps and up, ×0.6 for HEVC and AV1).
|
||||||
|
Measured on `grid` with a 2292×960 60 fps capture: H.264 7.3× realtime, HEVC 720p
|
||||||
|
11.7×, AV1 720p 13×, all on the Arc. `core` and `hamtv` re-encode on the CPU until
|
||||||
|
*Allow GPU access* is pressed (libx264 at 9.4× for a 720p→480p clip on `core`); the Mac
|
||||||
|
and `gate` need ffmpeg first, which the previews note offers to install.
|
||||||
|
- *Not done, on purpose:* no restart-from-a-seek beyond what is encoded (the encode is
|
||||||
|
faster than watching; a seek far ahead waits), no bit-rate ladder or adaptive
|
||||||
|
switching, no subtitles, no keeping a finished re-encode for next time.
|
||||||
|
- **Files edge sweep** — **2026-10-11** (v0.1.0-214), asked for as "a full bug sweep of
|
||||||
|
the Files section, every example and edge case". Every handler and both UI files read
|
||||||
|
through with odd inputs in mind, then a battery of ~60 requests against the live hub
|
||||||
|
(`sweep.py`, not kept: listings of files, missing and forbidden paths, NUL bytes, names
|
||||||
|
with spaces/#/%/?/umlauts, links to files, folders and nothing, a pipe, over-size saves,
|
||||||
|
uploads with bad names and clashes, renames and moves onto things, protected paths,
|
||||||
|
thumbnails of empty and mislabelled images, playback of an image and an audio-only mp4,
|
||||||
|
root-mode variants). One real data bug: **a save over the 8 MiB cap truncated the file
|
||||||
|
and kept the first 8 MiB, answering 204** — the body is now read whole first and refused
|
||||||
|
with 413, the file untouched. The rest: a link to a file showed the link's own time; a
|
||||||
|
link to nothing read as an empty 200; a bad codec or size was answered by inspecting
|
||||||
|
the file; a failed stream was handed back to the next identical request; a player
|
||||||
|
leaving mid-start made the recipe loop try the next engine; a root-mode ffmpeg's end
|
||||||
|
could wait forever; a page closing kept its stream for ten minutes (a beacon now
|
||||||
|
releases it at once); the path box took only absolute paths (it resolves like a shell
|
||||||
|
now, in the tab and the Move picker, with a test); a dropped folder became a nameless
|
||||||
|
empty file; the upload toast counted intentions; the viewer called a pipe a binary
|
||||||
|
file. Not bugs, left alone: `rm -rf` of a mount point empties it (the console is
|
||||||
|
root-capable by design); a folder of ten thousand rows is slow (no virtual list).
|
||||||
|
- **Whole-console sweep** — **2026-10-13** (v0.1.0-216), asked for as "a full sweep of
|
||||||
|
Ham Lab again, look for any bugs and fix them". Four nets, then reading: the Go race
|
||||||
|
detector over the whole suite (clean), staticcheck's development build (the release does
|
||||||
|
not know Go 1.26; it found only dead code and one capitalised error string), a
|
||||||
|
garbage-in battery over every route safe to poke — 202 routes, bad identifiers, empty
|
||||||
|
and malformed bodies, with the destructive ones held back — and a headless Chromium pass
|
||||||
|
through all twenty tabs on the hub and on `grid`, collecting exceptions, console errors
|
||||||
|
and failed requests (none). By hand: the auth middleware and sessions, token issue, the
|
||||||
|
hub→node and KVM proxies, every validation regex before a shell, the cron and cron.d
|
||||||
|
writers, config and state atomicity, every WebSocket and SSE loop's exit, the terminal
|
||||||
|
manager, the task manager, and every destructive confirmation in the UI. Fixed, all
|
||||||
|
small: docker inspect/health of a missing container was a 500 (404 now, a daemon that
|
||||||
|
is down 503), `/api/virt/instances/{name}` without Incus was a 502 (409 like its
|
||||||
|
siblings), a ZFS snapshot delete with a bad name was a 500 (400; 404 for one that does
|
||||||
|
not exist), the task list read a task's Running flag outside its lock (a real data race,
|
||||||
|
never seen), the Virtualization export config was written in place rather than
|
||||||
|
atomically, and the dead code. Nothing in auth, the proxies or the shell boundaries.
|
||||||
|
One self-inflicted finding: the battery's skip list left out `POST
|
||||||
|
/api/files/thumbs/gpu-access`, so the sweep pressed *Allow GPU access* on `core` three
|
||||||
|
times over — the drop-in landed and the hub restarted (that is what the headless pass
|
||||||
|
saw as a refused connection). Kept, since it is the fix the tab offers; `core` makes
|
||||||
|
previews and re-encodes on its UHD iGPU now. The handler refuses a second run while one
|
||||||
|
is in flight (409), since the second `systemd-run` only collided with the first's timer.
|
||||||
- **Docker read-only view** — `docker ps`/logs; dockhand stays the source of truth.
|
- **Docker read-only view** — `docker ps`/logs; dockhand stays the source of truth.
|
||||||
- **Notifications** — push on service failure / SMART warning / UPS event.
|
- **Notifications** — push on service failure / SMART warning / UPS event.
|
||||||
- **Terminal extras** — search, copy-mode, font/theme, split panes, upload via
|
- **Terminal extras** — search, copy-mode, font/theme, split panes, upload via
|
||||||
@@ -719,12 +910,12 @@ what it has moved.
|
|||||||
|
|
||||||
## To do
|
## To do
|
||||||
|
|
||||||
- **Grow the Files tab until copyparty can go** — asked for 2026-09-23 as two items (Files
|
- **Grow the Files tab** — asked for 2026-09-23 as two items (Files polish, and deprecating
|
||||||
polish, and deprecating copyparty), merged 2026-09-24: they are the same tab and the same
|
copyparty), merged 2026-09-24. **copyparty was retired 2026-10-11** at Mitchell's call,
|
||||||
work. `copyparty` on `grid` (`copyparty.service`, `:3923`, `/etc/copyparty.conf`, serving
|
ahead of these gaps rather than after them: service, files and NPM host all gone, with a
|
||||||
all of `/tank` as user `core` with one account behind NPM) overlaps **Files** almost
|
rollback archive kept on `grid` (`hamlet`'s `docs/hosts.md` has the detail). Files is now
|
||||||
entirely. The aim is to stop the service and keep only Ham Lab, with general ease-of-use
|
the only file browser on `grid`, so the list below is no longer parity — it is the polish
|
||||||
polish along the way. What Files must gain first, in the order that matters:
|
still owed, in the order that matters:
|
||||||
- **Multi-select and bulk actions.** copyparty selects many rows and moves, deletes or
|
- **Multi-select and bulk actions.** copyparty selects many rows and moves, deletes or
|
||||||
downloads them together; Files acts on one row at a time.
|
downloads them together; Files acts on one row at a time.
|
||||||
- **Download a folder**, and a multi-file download, as a streamed zip or tar — the thing
|
- **Download a folder**, and a multi-file download, as a streamed zip or tar — the thing
|
||||||
@@ -732,17 +923,14 @@ what it has moved.
|
|||||||
- **Resumable uploads.** copyparty's up2k chunks a file, survives a dropped connection
|
- **Resumable uploads.** copyparty's up2k chunks a file, survives a dropped connection
|
||||||
and skips what is already there; Files streams one PUT, so a phone upload that drops at
|
and skips what is already there; Files streams one PUT, so a phone upload that drops at
|
||||||
90% starts over.
|
90% starts over.
|
||||||
- **A gallery.** Thumbnails for a folder of images and video, and play audio/video in
|
- **A gallery.** ~~Thumbnails for a folder of images and video~~ (the **previews** toggle,
|
||||||
place — `/tank/media` and `/tank/immich` are most of what is browsed.
|
2026-10-11 — see the shipped entry above; a grid view is not built), and play
|
||||||
|
audio/video in place — `/tank/media` and `/tank/immich` are most of what is browsed.
|
||||||
- **Search** below the current folder by name, since `find` in a terminal is the fallback
|
- **Search** below the current folder by name, since `find` in a terminal is the fallback
|
||||||
today.
|
today.
|
||||||
- *Decide when built:* whether share links (copyparty's public per-path links) are wanted
|
- *Decide when built:* whether share links (copyparty's public per-path links) are wanted
|
||||||
at all — they are the one feature that hands out access without a login, and Ham Lab has
|
at all — they are the one feature that hands out access without a login, and Ham Lab has
|
||||||
no notion of that; if not, say so in the docs rather than leaving a gap unexplained.
|
no notion of that; if not, say so in the docs rather than leaving a gap unexplained.
|
||||||
- *After:* `systemctl disable --now copyparty`, remove its NPM host, keep
|
|
||||||
`/etc/copyparty.conf` until a rollback is clearly not needed, then update
|
|
||||||
`hamlet`'s `docs/hosts.md` bare-metal list, `docs/gotchas.md` (the "`docker ps` is not
|
|
||||||
the inventory" note names it) and the `grid` row in `CLAUDE.md`.
|
|
||||||
|
|
||||||
- **Grow the Docker tab until Dockhand can go** — asked for 2026-09-23 and widened
|
- **Grow the Docker tab until Dockhand can go** — asked for 2026-09-23 and widened
|
||||||
2026-09-24 to take in Docker tab polish and feature work, on the same reasoning as the
|
2026-09-24 to take in Docker tab polish and feature work, on the same reasoning as the
|
||||||
|
|||||||
+24
-21
@@ -1,20 +1,17 @@
|
|||||||
# Virtualization tab — plan (2026-10-09; phase 1 built the same day)
|
# Virtualization tab — plan (2026-10-09; phase 1 built the same day)
|
||||||
|
|
||||||
**Status:** phases 1 and 2 are **deployed to all five machines as v0.1.0-188** (2026-10-09).
|
**Status:** phases 1 and 2 are **deployed to all five machines as v0.1.0-196**. **No machine
|
||||||
Incus runs on `grid` (installed from the tab); the other machines show Install or, on `gate`,
|
has Incus installed as of 2026-10-11**: `grid`'s install of 2026-10-09 was removed at
|
||||||
unavailable. Phase 2 verified live on `grid` through the hub: ISO fetched by URL with its
|
Mitchell's request so he can install it himself from the tab and ask questions along the way
|
||||||
sha256 checked, **New VM from ISO** booted Alpine with Secure Boot off, **the VGA console
|
(`hamlet/docs/hosts.md` has what was removed and what the two days established). The tab
|
||||||
drew it in a real browser** (headless Chromium, 1280×800 canvas, keyboard and mouse live),
|
shows Install on `grid`, `core`, the Mac mini and `hamtv`, unavailable on `gate`. Both create
|
||||||
a virtiofs mount appeared in a running VM, an export tarball landed in `virt/exports/` and the
|
forms carry a **Disk location** picker with add/remove of locations; instance storage as
|
||||||
nightly cron line is written, `instance.*` sensors flow and the "VM or container down" alert
|
plain files on the machine's own disk is what he wants on `grid`. Everything in phase 2 was
|
||||||
default is in. Two things found and fixed on the way: spice-html5 wants the `binary`
|
verified live on `grid` before the removal: ISO fetch with checksum, New VM from ISO, the VGA
|
||||||
websocket subprotocol agreed; and it opens one websocket per SPICE channel, so the bridge
|
console in a real browser (headless Chromium; `binary` subprotocol, one console operation per
|
||||||
holds **one console operation per VM** and gives each channel its own data websocket on it
|
VM with a data websocket per SPICE channel), a virtiofs mount into a running VM, exports and
|
||||||
(Incus refuses a second operation as "already connected"). **Left from phase 2:** the LAN
|
their cron line, `instance.*` sensors and the alert default. **Left:** the LAN bridge step
|
||||||
bridge step (`br0` on the uplink — needs a live test on a machine that can afford to drop
|
(his call), cloud-init presets beyond "Use my SSH keys", a backup line for `virt/exports/`.
|
||||||
off the network for a moment, which is a decision, not a build), cloud-init presets beyond
|
|
||||||
"Use my SSH keys", and a backup line for `grid`'s `virt/exports/` (see `hamlet/docs/backups.md`).
|
|
||||||
`t-alpine` on `grid` is the live example; delete it from the tab when done.
|
|
||||||
|
|
||||||
A tab for creating, watching, consoling into and deleting **LXC containers and KVM virtual
|
A tab for creating, watching, consoling into and deleting **LXC containers and KVM virtual
|
||||||
machines**, the way Proxmox does with `pct` and `qm` — **per machine, like every other
|
machines**, the way Proxmox does with `pct` and `qm` — **per machine, like every other
|
||||||
@@ -236,12 +233,18 @@ like everything else.
|
|||||||
|
|
||||||
## Layout decisions Ham Lab makes per machine (overridable in the Install dialog)
|
## Layout decisions Ham Lab makes per machine (overridable in the Install dialog)
|
||||||
|
|
||||||
1. **Storage.** The instance pool's backing lives in Ham Lab's folder: Install creates a
|
1. **Storage — revised 2026-10-11.** The create forms carry a **Disk location** picker: a
|
||||||
sparse `virt/pool.img` and hands it to Incus as the pool's `source` (zfs or btrfs by the
|
list of the machine's Incus storage pools, each described by where its bytes land, with
|
||||||
detection above), or `virt/pool/` as a `dir` pool where neither is available. Sized from
|
"+ add a location" (any folder on the machine becomes a plain-files pool). On `grid` the
|
||||||
the disk's free space, shown and editable. Bulk data is **bind-mounted in**, not stored in
|
one location is `local`: plain files under `virt/pool/` on the root NVMe, each VM's disk
|
||||||
the pool. The driver, path and size are recorded in `virt.json` so the page can say where
|
its own sparse file sized on its own, each container a directory, nothing pre-sized.
|
||||||
the data is and Install can rebuild the pool from the file after a reinstall.
|
*The first design* (a ZFS pool inside one sparse file, handed to Incus as the pool's
|
||||||
|
source, for instant snapshots and clones) was built, ran for two days, and was taken out
|
||||||
|
at Mitchell's instruction: one pre-sized file holding every disk is an opaque middle
|
||||||
|
layer with a number on it that reads as a reservation, and "zfs · hamlab-virt" on the page
|
||||||
|
read as the NAS pool. The Install dialog still offers zfs/btrfs-in-a-file for a machine
|
||||||
|
that wants instant snapshots; the labels now say exactly what each is. Bulk data is still
|
||||||
|
**bind-mounted in**, not stored in the pool.
|
||||||
2. **Bridge subnet, computed.** Incus's `auto` picks a random `10.x.y.0/24`, and **this
|
2. **Bridge subnet, computed.** Incus's `auto` picks a random `10.x.y.0/24`, and **this
|
||||||
LAN is `10.0.0.0/8`** — every host has a /8 on its NIC, so any 10.x bridge collides.
|
LAN is `10.0.0.0/8`** — every host has a /8 on its NIC, so any 10.x bridge collides.
|
||||||
Ham Lab reads the machine's routes and picks the first free /24 from `192.168.0.0/16`,
|
Ham Lab reads the machine's routes and picks the first free /24 from `192.168.0.0/16`,
|
||||||
|
|||||||
@@ -31,6 +31,7 @@ import (
|
|||||||
"hamlab/internal/cron"
|
"hamlab/internal/cron"
|
||||||
"hamlab/internal/files"
|
"hamlab/internal/files"
|
||||||
"hamlab/internal/incus"
|
"hamlab/internal/incus"
|
||||||
|
"hamlab/internal/media"
|
||||||
"hamlab/internal/metrics"
|
"hamlab/internal/metrics"
|
||||||
"hamlab/internal/nodes"
|
"hamlab/internal/nodes"
|
||||||
"hamlab/internal/priv"
|
"hamlab/internal/priv"
|
||||||
@@ -82,6 +83,10 @@ func main() {
|
|||||||
switch cmd { // commands that don't read the config
|
switch cmd { // commands that don't read the config
|
||||||
case "fs": // privileged helper for the Files tab (run as root by the daemon)
|
case "fs": // privileged helper for the Files tab (run as root by the daemon)
|
||||||
os.Exit(files.Helper(flag.Args()[1:], os.Stdin, os.Stdout, os.Stderr))
|
os.Exit(files.Helper(flag.Args()[1:], os.Stdin, os.Stdout, os.Stderr))
|
||||||
|
case "thumb": // privileged helper for the Files tab's previews in root mode (run as root by the daemon)
|
||||||
|
os.Exit(media.Helper(flag.Args()[1:], os.Stdin, os.Stdout, os.Stderr))
|
||||||
|
case "run-ffmpeg": // privileged helper for playback in root mode: ffmpeg as root, ending when stdin closes
|
||||||
|
os.Exit(media.RunFFmpegHelper(flag.Args()[1:], os.Stdin, os.Stderr))
|
||||||
case "space-walk": // helper for the Space tab: measure mountpoints, stream the tree back
|
case "space-walk": // helper for the Space tab: measure mountpoints, stream the tree back
|
||||||
os.Exit(space.Helper(flag.Args()[1:], os.Stdin, os.Stdout, os.Stderr))
|
os.Exit(space.Helper(flag.Args()[1:], os.Stdin, os.Stdout, os.Stderr))
|
||||||
case "drm-clients": // privileged helper for the GPU readings: who has the GPU open (run as root by the daemon)
|
case "drm-clients": // privileged helper for the GPU readings: who has the GPU open (run as root by the daemon)
|
||||||
|
|||||||
@@ -5,6 +5,7 @@ go 1.26.0
|
|||||||
require (
|
require (
|
||||||
github.com/coder/websocket v1.8.15
|
github.com/coder/websocket v1.8.15
|
||||||
github.com/creack/pty v1.1.24
|
github.com/creack/pty v1.1.24
|
||||||
|
go.etcd.io/bbolt v1.5.0
|
||||||
golang.org/x/crypto v0.57.0
|
golang.org/x/crypto v0.57.0
|
||||||
golang.org/x/term v0.46.0
|
golang.org/x/term v0.46.0
|
||||||
)
|
)
|
||||||
|
|||||||
@@ -2,9 +2,21 @@ github.com/coder/websocket v1.8.15 h1:6B2JPeOGlpff2Uz6vOEH1Vzpi0iUz20A+lPVhPHtNU
|
|||||||
github.com/coder/websocket v1.8.15/go.mod h1:NX3SzP+inril6yawo5CQXx8+fk145lPDC6pumgx0mVg=
|
github.com/coder/websocket v1.8.15/go.mod h1:NX3SzP+inril6yawo5CQXx8+fk145lPDC6pumgx0mVg=
|
||||||
github.com/creack/pty v1.1.24 h1:bJrF4RRfyJnbTJqzRLHzcGaZK1NeM5kTC9jGgovnR1s=
|
github.com/creack/pty v1.1.24 h1:bJrF4RRfyJnbTJqzRLHzcGaZK1NeM5kTC9jGgovnR1s=
|
||||||
github.com/creack/pty v1.1.24/go.mod h1:08sCNb52WyoAwi2QDyzUCTgcvVFhUzewun7wtTfvcwE=
|
github.com/creack/pty v1.1.24/go.mod h1:08sCNb52WyoAwi2QDyzUCTgcvVFhUzewun7wtTfvcwE=
|
||||||
|
github.com/davecgh/go-spew v1.1.1 h1:vj9j/u1bqnvCEfJOwUhtlOARqs3+rkHYY13jYWTU97c=
|
||||||
|
github.com/davecgh/go-spew v1.1.1/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
|
||||||
|
github.com/pmezard/go-difflib v1.0.0 h1:4DBwDE0NGyQoBHbLQYPwSUPoCMWR5BEzIk/f1lZbAQM=
|
||||||
|
github.com/pmezard/go-difflib v1.0.0/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4=
|
||||||
|
github.com/stretchr/testify v1.11.1 h1:7s2iGBzp5EwR7/aIZr8ao5+dra3wiQyKjjFuvgVKu7U=
|
||||||
|
github.com/stretchr/testify v1.11.1/go.mod h1:wZwfW3scLgRK+23gO65QZefKpKQRnfz6sD981Nm4B6U=
|
||||||
|
go.etcd.io/bbolt v1.5.0 h1:S7GAl7Fxv12yohbwFfIbQCGDWbQbtDGPET4P/bD4lxU=
|
||||||
|
go.etcd.io/bbolt v1.5.0/go.mod h1:mkltfYE5aUHQxUct9N9V+Kp7aSjFqjgrhcXIS70Lrdk=
|
||||||
golang.org/x/crypto v0.57.0 h1:3ZVCjf8Ggz7zneR/EHRVx68Ctf+2pmIMP2UFhh9cC6M=
|
golang.org/x/crypto v0.57.0 h1:3ZVCjf8Ggz7zneR/EHRVx68Ctf+2pmIMP2UFhh9cC6M=
|
||||||
golang.org/x/crypto v0.57.0/go.mod h1:Fdz0i5U6CoizGwLda9DttjSk6qlZo25zYNtR+ycvuZA=
|
golang.org/x/crypto v0.57.0/go.mod h1:Fdz0i5U6CoizGwLda9DttjSk6qlZo25zYNtR+ycvuZA=
|
||||||
|
golang.org/x/sync v0.20.0 h1:e0PTpb7pjO8GAtTs2dQ6jYa5BWYlMuX047Dco/pItO4=
|
||||||
|
golang.org/x/sync v0.20.0/go.mod h1:9xrNwdLfx4jkKbNva9FpL6vEN7evnE43NNNJQ2LF3+0=
|
||||||
golang.org/x/sys v0.48.0 h1:bbX/i/6MgT9BVLM9RT1thmxL04yeTAhbEz4SyadbXoo=
|
golang.org/x/sys v0.48.0 h1:bbX/i/6MgT9BVLM9RT1thmxL04yeTAhbEz4SyadbXoo=
|
||||||
golang.org/x/sys v0.48.0/go.mod h1:hNLxWAXmnKAxqDtdwIYC4bM9oQPEecfsnNMuSxOs3og=
|
golang.org/x/sys v0.48.0/go.mod h1:hNLxWAXmnKAxqDtdwIYC4bM9oQPEecfsnNMuSxOs3og=
|
||||||
golang.org/x/term v0.46.0 h1:3+OXuTbaKDgwk8jTi3aSLHRlmWqHEUDUtxnbFigO4YE=
|
golang.org/x/term v0.46.0 h1:3+OXuTbaKDgwk8jTi3aSLHRlmWqHEUDUtxnbFigO4YE=
|
||||||
golang.org/x/term v0.46.0/go.mod h1:+K02xbkittuwc0Am4abfA3Fc+XRGXkvBXNO88NCXPoc=
|
golang.org/x/term v0.46.0/go.mod h1:+K02xbkittuwc0Am4abfA3Fc+XRGXkvBXNO88NCXPoc=
|
||||||
|
gopkg.in/yaml.v3 v3.0.1 h1:fxVm/GzAzEWqLHuvctI91KS9hhNmmWOoWu0XTYJS7CA=
|
||||||
|
gopkg.in/yaml.v3 v3.0.1/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM=
|
||||||
|
|||||||
+22
-3
@@ -343,6 +343,21 @@ func (s *Server) dockerLogs(w http.ResponseWriter, r *http.Request) {
|
|||||||
}
|
}
|
||||||
|
|
||||||
// dockerInspect is the parts of `docker inspect` worth reading on a phone.
|
// dockerInspect is the parts of `docker inspect` worth reading on a phone.
|
||||||
|
// dockerErr answers for a docker command that failed: a thing that isn't there is a 404,
|
||||||
|
// the daemon being away a 503, anything else a 500.
|
||||||
|
func dockerErr(w http.ResponseWriter, err error) {
|
||||||
|
msg := strings.TrimSpace(strings.TrimPrefix(strings.TrimSpace(err.Error()), "error: "))
|
||||||
|
low := strings.ToLower(msg)
|
||||||
|
switch {
|
||||||
|
case strings.Contains(low, "no such object") || strings.Contains(low, "no such container") || strings.Contains(low, "not found"):
|
||||||
|
jsonErr(w, 404, msg)
|
||||||
|
case strings.Contains(low, "cannot connect to the docker daemon") || strings.Contains(low, "is the docker daemon running"):
|
||||||
|
jsonErr(w, 503, msg)
|
||||||
|
default:
|
||||||
|
jsonErr(w, 500, msg)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
func (s *Server) dockerInspect(w http.ResponseWriter, r *http.Request) {
|
func (s *Server) dockerInspect(w http.ResponseWriter, r *http.Request) {
|
||||||
id := r.PathValue("id")
|
id := r.PathValue("id")
|
||||||
if !dockerRef.MatchString(id) {
|
if !dockerRef.MatchString(id) {
|
||||||
@@ -352,9 +367,13 @@ func (s *Server) dockerInspect(w http.ResponseWriter, r *http.Request) {
|
|||||||
ctx, cancel := context.WithTimeout(r.Context(), 15*time.Second)
|
ctx, cancel := context.WithTimeout(r.Context(), 15*time.Second)
|
||||||
defer cancel()
|
defer cancel()
|
||||||
out, err := dockerRun(ctx, "inspect", id)
|
out, err := dockerRun(ctx, "inspect", id)
|
||||||
|
if err != nil {
|
||||||
|
dockerErr(w, err)
|
||||||
|
return
|
||||||
|
}
|
||||||
var all []map[string]any
|
var all []map[string]any
|
||||||
if err != nil || json.Unmarshal(out, &all) != nil || len(all) == 0 {
|
if json.Unmarshal(out, &all) != nil || len(all) == 0 {
|
||||||
jsonErr(w, 500, fmt.Sprint("inspect failed: ", err))
|
jsonErr(w, 500, "inspect: unreadable answer")
|
||||||
return
|
return
|
||||||
}
|
}
|
||||||
c := all[0]
|
c := all[0]
|
||||||
@@ -486,7 +505,7 @@ func (s *Server) dockerHealth(w http.ResponseWriter, r *http.Request) {
|
|||||||
defer cancel()
|
defer cancel()
|
||||||
out, err := dockerRun(ctx, "inspect", "--format", "{{json .State}}", id)
|
out, err := dockerRun(ctx, "inspect", "--format", "{{json .State}}", id)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
jsonErr(w, 500, err.Error())
|
dockerErr(w, err)
|
||||||
return
|
return
|
||||||
}
|
}
|
||||||
var st struct {
|
var st struct {
|
||||||
|
|||||||
@@ -0,0 +1,123 @@
|
|||||||
|
package api
|
||||||
|
|
||||||
|
import (
|
||||||
|
"bytes"
|
||||||
|
"errors"
|
||||||
|
"net/http/httptest"
|
||||||
|
"net/url"
|
||||||
|
"os"
|
||||||
|
"path/filepath"
|
||||||
|
"testing"
|
||||||
|
|
||||||
|
"hamlab/internal/files"
|
||||||
|
)
|
||||||
|
|
||||||
|
// A file read answers Range requests — what a <video> needs to seek — and HEAD, through
|
||||||
|
// the same handler that streams everything else.
|
||||||
|
func TestFilesReadRanges(t *testing.T) {
|
||||||
|
dir := t.TempDir()
|
||||||
|
p := filepath.Join(dir, "clip.mp4")
|
||||||
|
data := make([]byte, 10000)
|
||||||
|
for i := range data {
|
||||||
|
data[i] = byte(i % 251)
|
||||||
|
}
|
||||||
|
if err := os.WriteFile(p, data, 0o644); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
s := &Server{fsLocal: files.Local{}}
|
||||||
|
get := func(method, extra string, hdr map[string]string) *httptest.ResponseRecorder {
|
||||||
|
r := httptest.NewRequest(method, "/api/files/read?path="+url.QueryEscape(p)+extra, nil)
|
||||||
|
for k, v := range hdr {
|
||||||
|
r.Header.Set(k, v)
|
||||||
|
}
|
||||||
|
w := httptest.NewRecorder()
|
||||||
|
s.filesRead(w, r)
|
||||||
|
return w
|
||||||
|
}
|
||||||
|
w := get("GET", "", nil)
|
||||||
|
if w.Code != 200 || w.Body.Len() != 10000 || w.Header().Get("Content-Type") != "video/mp4" || w.Header().Get("Accept-Ranges") != "bytes" {
|
||||||
|
t.Fatalf("whole: %d %d %q %q", w.Code, w.Body.Len(), w.Header().Get("Content-Type"), w.Header().Get("Accept-Ranges"))
|
||||||
|
}
|
||||||
|
w = get("GET", "", map[string]string{"Range": "bytes=100-199"})
|
||||||
|
if w.Code != 206 || w.Header().Get("Content-Range") != "bytes 100-199/10000" || w.Body.Len() != 100 || w.Body.Bytes()[0] != data[100] {
|
||||||
|
t.Fatalf("range: %d %q %d", w.Code, w.Header().Get("Content-Range"), w.Body.Len())
|
||||||
|
}
|
||||||
|
w = get("GET", "", map[string]string{"Range": "bytes=9990-"})
|
||||||
|
if w.Code != 206 || w.Body.Len() != 10 || w.Body.Bytes()[9] != data[9999] {
|
||||||
|
t.Fatalf("tail range: %d %d", w.Code, w.Body.Len())
|
||||||
|
}
|
||||||
|
w = get("HEAD", "", nil)
|
||||||
|
if w.Code != 200 || w.Body.Len() != 0 || w.Header().Get("Content-Length") != "10000" {
|
||||||
|
t.Fatalf("head: %d %d %q", w.Code, w.Body.Len(), w.Header().Get("Content-Length"))
|
||||||
|
}
|
||||||
|
if w := get("GET", "&max=50", nil); w.Code != 413 {
|
||||||
|
t.Fatalf("the editor's cap: %d", w.Code)
|
||||||
|
}
|
||||||
|
if w := get("GET", "", map[string]string{"Range": "bytes=20000-"}); w.Code != 416 {
|
||||||
|
t.Fatalf("a range past the end: %d", w.Code)
|
||||||
|
}
|
||||||
|
if err := os.Chmod(p, 0); err == nil && os.Geteuid() != 0 {
|
||||||
|
if w := get("GET", "", nil); w.Code != 403 {
|
||||||
|
t.Fatalf("unreadable: %d", w.Code)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// A write over the cap is refused whole: the file is not truncated first.
|
||||||
|
func TestFilesWriteCap(t *testing.T) {
|
||||||
|
p := filepath.Join(t.TempDir(), "f.txt")
|
||||||
|
if err := os.WriteFile(p, []byte("keep me"), 0o644); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
s := &Server{fsLocal: files.Local{}}
|
||||||
|
body := make([]byte, 8<<20+1)
|
||||||
|
r := httptest.NewRequest("PUT", "/api/files/write?path="+url.QueryEscape(p), bytes.NewReader(body))
|
||||||
|
w := httptest.NewRecorder()
|
||||||
|
s.filesWrite(w, r)
|
||||||
|
if w.Code != 413 {
|
||||||
|
t.Fatalf("got %d", w.Code)
|
||||||
|
}
|
||||||
|
if b, _ := os.ReadFile(p); string(b) != "keep me" {
|
||||||
|
t.Fatalf("the file was touched: %q", b)
|
||||||
|
}
|
||||||
|
r = httptest.NewRequest("PUT", "/api/files/write?path="+url.QueryEscape(p), bytes.NewReader([]byte("new")))
|
||||||
|
w = httptest.NewRecorder()
|
||||||
|
s.filesWrite(w, r)
|
||||||
|
if b, _ := os.ReadFile(p); w.Code != 204 || string(b) != "new" {
|
||||||
|
t.Fatalf("a write within the cap: %d %q", w.Code, b)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// A link to nothing is a 404, not an empty 200.
|
||||||
|
func TestFilesReadDanglingLink(t *testing.T) {
|
||||||
|
dir := t.TempDir()
|
||||||
|
l := filepath.Join(dir, "broken")
|
||||||
|
if err := os.Symlink("nowhere", l); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
s := &Server{fsLocal: files.Local{}}
|
||||||
|
r := httptest.NewRequest("GET", "/api/files/read?path="+url.QueryEscape(l), nil)
|
||||||
|
w := httptest.NewRecorder()
|
||||||
|
s.filesRead(w, r)
|
||||||
|
if w.Code != 404 {
|
||||||
|
t.Fatalf("got %d: %s", w.Code, w.Body.String())
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// dockerErr tells a thing that isn't there from a daemon that isn't up from anything else.
|
||||||
|
func TestDockerErr(t *testing.T) {
|
||||||
|
for _, c := range []struct {
|
||||||
|
msg string
|
||||||
|
want int
|
||||||
|
}{
|
||||||
|
{"error: No such object: zzz", 404}, {"Error response from daemon: No such container: x", 404},
|
||||||
|
{"Cannot connect to the Docker daemon at unix:///var/run/docker.sock. Is the docker daemon running?", 503},
|
||||||
|
{"something else entirely", 500},
|
||||||
|
} {
|
||||||
|
w := httptest.NewRecorder()
|
||||||
|
dockerErr(w, errors.New(c.msg))
|
||||||
|
if w.Code != c.want {
|
||||||
|
t.Errorf("%q → %d, want %d", c.msg, w.Code, c.want)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -338,20 +338,6 @@ func (s *Server) listKVMs(w http.ResponseWriter, _ *http.Request) {
|
|||||||
jsonOK(w, 200, out)
|
jsonOK(w, 200, out)
|
||||||
}
|
}
|
||||||
|
|
||||||
// kvmForMachine is the KVM attached to a machine, "" being the hub itself.
|
|
||||||
func (s *Server) kvmForMachine(machine string) (config.KVM, bool) {
|
|
||||||
var found config.KVM
|
|
||||||
ok := false
|
|
||||||
s.cfg.View(func(c *config.Config) {
|
|
||||||
for _, k := range c.KVMs {
|
|
||||||
if k.Machine == machine {
|
|
||||||
found, ok = k, true
|
|
||||||
}
|
|
||||||
}
|
|
||||||
})
|
|
||||||
return found, ok
|
|
||||||
}
|
|
||||||
|
|
||||||
func (s *Server) addKVM(w http.ResponseWriter, r *http.Request) {
|
func (s *Server) addKVM(w http.ResponseWriter, r *http.Request) {
|
||||||
var body struct{ Name, URL, Machine string }
|
var body struct{ Name, URL, Machine string }
|
||||||
if err := json.NewDecoder(io.LimitReader(r.Body, 4096)).Decode(&body); err != nil {
|
if err := json.NewDecoder(io.LimitReader(r.Body, 4096)).Decode(&body); err != nil {
|
||||||
|
|||||||
@@ -0,0 +1,169 @@
|
|||||||
|
package api
|
||||||
|
|
||||||
|
// Playing a video the browser cannot play as it is: POST /api/files/play starts an HLS
|
||||||
|
// re-encode on this machine (internal/media), GET …/play/{id}/index.m3u8 and the segments
|
||||||
|
// feed the player, GET …/play/{id} is its progress, DELETE ends it. /api/files/play-caps
|
||||||
|
// says which codecs this machine can make and on what. Root mode (?sudo=1) reads the file
|
||||||
|
// and runs ffmpeg as root, through the same sudo the file reads use.
|
||||||
|
|
||||||
|
import (
|
||||||
|
"context"
|
||||||
|
"encoding/json"
|
||||||
|
"errors"
|
||||||
|
"io"
|
||||||
|
"log"
|
||||||
|
"net/http"
|
||||||
|
"path/filepath"
|
||||||
|
"strings"
|
||||||
|
"time"
|
||||||
|
|
||||||
|
"hamlab/internal/files"
|
||||||
|
"hamlab/internal/media"
|
||||||
|
)
|
||||||
|
|
||||||
|
const playLimit = 3 // re-encodes at once on one machine; a GPU does two or three, a CPU one
|
||||||
|
|
||||||
|
type playCaps struct {
|
||||||
|
Codecs []playCodec `json:"codecs"`
|
||||||
|
Qualities []media.Quality `json:"qualities"`
|
||||||
|
Error string `json:"error,omitempty"`
|
||||||
|
}
|
||||||
|
|
||||||
|
type playCodec struct {
|
||||||
|
media.Codec
|
||||||
|
Engine string `json:"engine,omitempty"` // "" when nothing here encodes it
|
||||||
|
}
|
||||||
|
|
||||||
|
// playCapsHandler: GET /api/files/play-caps[?sudo=1]
|
||||||
|
func (s *Server) playCapsHandler(w http.ResponseWriter, r *http.Request) {
|
||||||
|
root := r.URL.Query().Get("sudo") == "1"
|
||||||
|
if root && s.fsSudo == nil {
|
||||||
|
jsonErr(w, 403, "root access needs passwordless sudo on the host")
|
||||||
|
return
|
||||||
|
}
|
||||||
|
caps, err := s.thumbs.capsFor(r.Context(), root, false)
|
||||||
|
pc := playCaps{Codecs: []playCodec{}, Qualities: media.Qualities}
|
||||||
|
if err != nil {
|
||||||
|
pc.Error = err.Error()
|
||||||
|
} else if caps.Error != "" {
|
||||||
|
pc.Error = caps.Error
|
||||||
|
}
|
||||||
|
for _, c := range media.Codecs {
|
||||||
|
x := playCodec{Codec: c}
|
||||||
|
if pc.Error == "" {
|
||||||
|
x.Engine = media.EngineFor(caps, c.ID)
|
||||||
|
}
|
||||||
|
pc.Codecs = append(pc.Codecs, x)
|
||||||
|
}
|
||||||
|
jsonOK(w, 200, pc)
|
||||||
|
}
|
||||||
|
|
||||||
|
// playStart: POST /api/files/play[?sudo=1] {path, codec, quality}
|
||||||
|
func (s *Server) playStart(w http.ResponseWriter, r *http.Request) {
|
||||||
|
fsys, ok := s.fsFor(w, r)
|
||||||
|
if !ok {
|
||||||
|
return
|
||||||
|
}
|
||||||
|
root := r.URL.Query().Get("sudo") == "1"
|
||||||
|
var body struct{ Path, Codec, Quality string }
|
||||||
|
if err := json.NewDecoder(io.LimitReader(r.Body, 64<<10)).Decode(&body); err != nil {
|
||||||
|
jsonErr(w, 400, "bad request")
|
||||||
|
return
|
||||||
|
}
|
||||||
|
p, err := files.Clean(body.Path)
|
||||||
|
if err != nil {
|
||||||
|
jsonErr(w, 400, err.Error())
|
||||||
|
return
|
||||||
|
}
|
||||||
|
if media.Kind(p) != "video" {
|
||||||
|
jsonErr(w, 415, "not a video")
|
||||||
|
return
|
||||||
|
}
|
||||||
|
if err := media.ValidChoice(body.Codec, body.Quality); err != nil {
|
||||||
|
jsonErr(w, 400, err.Error())
|
||||||
|
return
|
||||||
|
}
|
||||||
|
if _, err := fsys.Stat(r.Context(), p); err != nil {
|
||||||
|
fsErr(w, err)
|
||||||
|
return
|
||||||
|
}
|
||||||
|
caps, err := s.thumbs.capsFor(r.Context(), root, false)
|
||||||
|
if err != nil {
|
||||||
|
jsonErr(w, 503, err.Error())
|
||||||
|
return
|
||||||
|
}
|
||||||
|
if caps.Error != "" {
|
||||||
|
jsonErr(w, 503, caps.Error)
|
||||||
|
return
|
||||||
|
}
|
||||||
|
ictx, cancel := context.WithTimeout(r.Context(), 30*time.Second)
|
||||||
|
src, err := media.Inspect(ictx, caps, p, root)
|
||||||
|
cancel()
|
||||||
|
if err != nil {
|
||||||
|
playErr(w, err)
|
||||||
|
return
|
||||||
|
}
|
||||||
|
st, err := s.plays.Open(r.Context(), caps, root, p, body.Codec, body.Quality, src)
|
||||||
|
if err != nil {
|
||||||
|
playErr(w, err)
|
||||||
|
return
|
||||||
|
}
|
||||||
|
log.Printf("files: play %s as %s %s on %s (from %s)", p, st.Codec, st.Quality, st.Engine, clientIP(r))
|
||||||
|
jsonOK(w, 200, map[string]any{"stream": st, "url": "/api/files/play/" + st.ID + "/index.m3u8"})
|
||||||
|
}
|
||||||
|
|
||||||
|
func playErr(w http.ResponseWriter, err error) {
|
||||||
|
var de *media.DecodeError
|
||||||
|
switch {
|
||||||
|
case errors.As(err, &de):
|
||||||
|
jsonErr(w, 422, de.Msg)
|
||||||
|
case strings.Contains(err.Error(), "already being re-encoded"):
|
||||||
|
jsonErr(w, 429, err.Error())
|
||||||
|
case strings.HasPrefix(err.Error(), "unknown ") || strings.HasPrefix(err.Error(), "nothing on this machine encodes"):
|
||||||
|
jsonErr(w, 400, err.Error())
|
||||||
|
case errors.Is(err, context.DeadlineExceeded):
|
||||||
|
jsonErr(w, 504, "the file took too long to read")
|
||||||
|
default:
|
||||||
|
jsonErr(w, 500, err.Error())
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// playStatus: GET /api/files/play/{id}
|
||||||
|
func (s *Server) playStatus(w http.ResponseWriter, r *http.Request) {
|
||||||
|
st, ok := s.plays.Find(r.PathValue("id"))
|
||||||
|
if !ok {
|
||||||
|
jsonErr(w, 404, "no such stream")
|
||||||
|
return
|
||||||
|
}
|
||||||
|
jsonOK(w, 200, st.Status())
|
||||||
|
}
|
||||||
|
|
||||||
|
// playStop: DELETE /api/files/play/{id}, or POST …/close — the form a page can send on its
|
||||||
|
// way out (navigator.sendBeacon), so a phone closing the tab releases the machine at once.
|
||||||
|
func (s *Server) playStop(w http.ResponseWriter, r *http.Request) {
|
||||||
|
s.plays.Close(r.PathValue("id"))
|
||||||
|
w.WriteHeader(204)
|
||||||
|
}
|
||||||
|
|
||||||
|
// playFile: GET /api/files/play/{id}/{file} — the playlist, the init segment, a segment.
|
||||||
|
func (s *Server) playFile(w http.ResponseWriter, r *http.Request) {
|
||||||
|
st, ok := s.plays.Find(r.PathValue("id"))
|
||||||
|
if !ok {
|
||||||
|
jsonErr(w, 404, "no such stream")
|
||||||
|
return
|
||||||
|
}
|
||||||
|
p, ok := st.File(r.PathValue("file"))
|
||||||
|
if !ok {
|
||||||
|
jsonErr(w, 404, "not a stream file")
|
||||||
|
return
|
||||||
|
}
|
||||||
|
switch filepath.Ext(p) {
|
||||||
|
case ".m3u8":
|
||||||
|
w.Header().Set("Content-Type", "application/vnd.apple.mpegurl")
|
||||||
|
default:
|
||||||
|
w.Header().Set("Content-Type", "video/mp4")
|
||||||
|
}
|
||||||
|
w.Header().Set("Cache-Control", "no-store")
|
||||||
|
w.Header().Set("X-Content-Type-Options", "nosniff")
|
||||||
|
http.ServeFile(w, r, p)
|
||||||
|
}
|
||||||
+111
-12
@@ -35,6 +35,7 @@ import (
|
|||||||
"hamlab/internal/files"
|
"hamlab/internal/files"
|
||||||
"hamlab/internal/hardware"
|
"hamlab/internal/hardware"
|
||||||
"hamlab/internal/incus"
|
"hamlab/internal/incus"
|
||||||
|
"hamlab/internal/media"
|
||||||
"hamlab/internal/metrics"
|
"hamlab/internal/metrics"
|
||||||
"hamlab/internal/network"
|
"hamlab/internal/network"
|
||||||
"hamlab/internal/nodes"
|
"hamlab/internal/nodes"
|
||||||
@@ -79,6 +80,9 @@ type Server struct {
|
|||||||
virt *incus.Manager
|
virt *incus.Manager
|
||||||
fsLocal files.FS
|
fsLocal files.FS
|
||||||
fsSudo files.FS // nil without passwordless sudo
|
fsSudo files.FS // nil without passwordless sudo
|
||||||
|
thumbs *thumbSvc
|
||||||
|
plays *media.Streamer
|
||||||
|
self string // this binary, for helpers run as tasks
|
||||||
nodes *nodes.Manager
|
nodes *nodes.Manager
|
||||||
fingerprint string
|
fingerprint string
|
||||||
version string
|
version string
|
||||||
@@ -98,7 +102,10 @@ func New(cfg *config.Config, sessions *auth.Sessions, mgr *term.Manager, col *me
|
|||||||
if err != nil {
|
if err != nil {
|
||||||
self = "hamlab"
|
self = "hamlab"
|
||||||
}
|
}
|
||||||
|
s.self = self
|
||||||
s.space = space.New(self, filepath.Join(cfg.DataDir(), "space"))
|
s.space = space.New(self, filepath.Join(cfg.DataDir(), "space"))
|
||||||
|
s.thumbs = newThumbSvc(cfg.DataDir(), self)
|
||||||
|
s.plays = media.NewStreamer(cfg.DataDir(), self, playLimit)
|
||||||
s.hostName() // read it once now so the first request doesn't have to
|
s.hostName() // read it once now so the first request doesn't have to
|
||||||
if u, err := user.Current(); err == nil && u.Username != "" {
|
if u, err := user.Current(); err == nil && u.Username != "" {
|
||||||
s.user = u.Username
|
s.user = u.Username
|
||||||
@@ -259,6 +266,8 @@ func (s *Server) routes() *http.ServeMux {
|
|||||||
m.HandleFunc("POST /api/virt/exports/run", s.auth(s.virtExportsRun))
|
m.HandleFunc("POST /api/virt/exports/run", s.auth(s.virtExportsRun))
|
||||||
m.HandleFunc("POST /api/virt/exports/delete", s.auth(s.virtExportDelete))
|
m.HandleFunc("POST /api/virt/exports/delete", s.auth(s.virtExportDelete))
|
||||||
m.HandleFunc("GET /api/virt/sshkeys", s.auth(s.virtSSHKeys))
|
m.HandleFunc("GET /api/virt/sshkeys", s.auth(s.virtSSHKeys))
|
||||||
|
m.HandleFunc("POST /api/virt/pools", s.auth(s.virtPoolCreate))
|
||||||
|
m.HandleFunc("POST /api/virt/pools/delete", s.auth(s.virtPoolDelete))
|
||||||
m.HandleFunc("GET /api/alerts", s.auth(s.alertsGet))
|
m.HandleFunc("GET /api/alerts", s.auth(s.alertsGet))
|
||||||
m.HandleFunc("PUT /api/alerts", s.auth(s.alertsPut))
|
m.HandleFunc("PUT /api/alerts", s.auth(s.alertsPut))
|
||||||
m.HandleFunc("POST /api/alerts/test", s.auth(s.alertsTest))
|
m.HandleFunc("POST /api/alerts/test", s.auth(s.alertsTest))
|
||||||
@@ -297,8 +306,19 @@ func (s *Server) routes() *http.ServeMux {
|
|||||||
m.HandleFunc("POST /api/files/mkdir", s.auth(s.filesMkdir))
|
m.HandleFunc("POST /api/files/mkdir", s.auth(s.filesMkdir))
|
||||||
m.HandleFunc("POST /api/files/touch", s.auth(s.filesTouch))
|
m.HandleFunc("POST /api/files/touch", s.auth(s.filesTouch))
|
||||||
m.HandleFunc("POST /api/files/rename", s.auth(s.filesRename))
|
m.HandleFunc("POST /api/files/rename", s.auth(s.filesRename))
|
||||||
|
m.HandleFunc("POST /api/files/move", s.auth(s.filesMove))
|
||||||
m.HandleFunc("POST /api/files/delete", s.auth(s.filesDelete))
|
m.HandleFunc("POST /api/files/delete", s.auth(s.filesDelete))
|
||||||
m.HandleFunc("POST /api/files/chmod", s.auth(s.filesChmod))
|
m.HandleFunc("POST /api/files/chmod", s.auth(s.filesChmod))
|
||||||
|
m.HandleFunc("GET /api/files/thumb", s.auth(s.filesThumb))
|
||||||
|
m.HandleFunc("GET /api/files/thumbs", s.auth(s.thumbsStatus))
|
||||||
|
m.HandleFunc("POST /api/files/thumbs/install", s.auth(s.thumbsInstall))
|
||||||
|
m.HandleFunc("POST /api/files/thumbs/gpu-access", s.auth(s.thumbsGPUAccess))
|
||||||
|
m.HandleFunc("GET /api/files/play-caps", s.auth(s.playCapsHandler))
|
||||||
|
m.HandleFunc("POST /api/files/play", s.auth(s.playStart))
|
||||||
|
m.HandleFunc("GET /api/files/play/{id}", s.auth(s.playStatus))
|
||||||
|
m.HandleFunc("DELETE /api/files/play/{id}", s.auth(s.playStop))
|
||||||
|
m.HandleFunc("POST /api/files/play/{id}/close", s.auth(s.playStop))
|
||||||
|
m.HandleFunc("GET /api/files/play/{id}/{file}", s.auth(s.playFile))
|
||||||
m.HandleFunc("GET /api/hardware", s.auth(s.hardwareReport))
|
m.HandleFunc("GET /api/hardware", s.auth(s.hardwareReport))
|
||||||
m.HandleFunc("GET /api/network", s.auth(s.networkOverview))
|
m.HandleFunc("GET /api/network", s.auth(s.networkOverview))
|
||||||
m.HandleFunc("GET /api/vpn", s.auth(s.vpnReport))
|
m.HandleFunc("GET /api/vpn", s.auth(s.vpnReport))
|
||||||
@@ -1912,21 +1932,15 @@ func (s *Server) filesRead(w http.ResponseWriter, r *http.Request) {
|
|||||||
jsonErr(w, 400, "not a regular file (device, pipe or socket)")
|
jsonErr(w, 400, "not a regular file (device, pipe or socket)")
|
||||||
return
|
return
|
||||||
}
|
}
|
||||||
|
if in.Broken {
|
||||||
|
jsonErr(w, 404, "a link to nothing: "+in.Target)
|
||||||
|
return
|
||||||
|
}
|
||||||
mx, _ := strconv.ParseInt(r.URL.Query().Get("max"), 10, 64)
|
mx, _ := strconv.ParseInt(r.URL.Query().Get("max"), 10, 64)
|
||||||
if mx > 0 && in.Size > mx {
|
if mx > 0 && in.Size > mx {
|
||||||
jsonErr(w, 413, "file too large")
|
jsonErr(w, 413, "file too large")
|
||||||
return
|
return
|
||||||
}
|
}
|
||||||
f, err := fsys.Open(r.Context(), p)
|
|
||||||
if err != nil {
|
|
||||||
fsErr(w, err)
|
|
||||||
return
|
|
||||||
}
|
|
||||||
defer f.Close()
|
|
||||||
var body io.Reader = f
|
|
||||||
if mx > 0 { // stat can lie (size 0 on /proc); the cap must hold on the bytes actually sent
|
|
||||||
body = io.LimitReader(f, mx+1)
|
|
||||||
}
|
|
||||||
name := filepath.Base(p)
|
name := filepath.Base(p)
|
||||||
ct := mime.TypeByExtension(filepath.Ext(name))
|
ct := mime.TypeByExtension(filepath.Ext(name))
|
||||||
if ct == "" {
|
if ct == "" {
|
||||||
@@ -1945,7 +1959,28 @@ func (s *Server) filesRead(w http.ResponseWriter, r *http.Request) {
|
|||||||
w.Header().Set("Content-Disposition", "inline; filename*=UTF-8''"+url.PathEscape(name))
|
w.Header().Set("Content-Disposition", "inline; filename*=UTF-8''"+url.PathEscape(name))
|
||||||
}
|
}
|
||||||
if in.Size > 0 && mx == 0 {
|
if in.Size > 0 && mx == 0 {
|
||||||
w.Header().Set("Content-Length", strconv.FormatInt(in.Size, 10))
|
// A real file of known size goes through net/http, which speaks Range: a video player
|
||||||
|
// can seek and a download can resume. /proc files say size 0 and take the plain path.
|
||||||
|
rs := files.NewSeeker(r.Context(), fsys, p, in.Size)
|
||||||
|
defer rs.Close()
|
||||||
|
if _, local := fsys.(files.Local); local { // cheap here, and turns "can't open" into a status rather than a cut-off body
|
||||||
|
if err := rs.Check(); err != nil {
|
||||||
|
fsErr(w, err)
|
||||||
|
return
|
||||||
|
}
|
||||||
|
}
|
||||||
|
http.ServeContent(w, r, name, time.UnixMilli(in.Mtime), rs)
|
||||||
|
return
|
||||||
|
}
|
||||||
|
f, err := fsys.Open(r.Context(), p)
|
||||||
|
if err != nil {
|
||||||
|
fsErr(w, err)
|
||||||
|
return
|
||||||
|
}
|
||||||
|
defer f.Close()
|
||||||
|
var body io.Reader = f
|
||||||
|
if mx > 0 { // stat can lie (size 0 on /proc); the cap must hold on the bytes actually sent
|
||||||
|
body = io.LimitReader(f, mx+1)
|
||||||
}
|
}
|
||||||
_, _ = io.Copy(w, body)
|
_, _ = io.Copy(w, body)
|
||||||
}
|
}
|
||||||
@@ -1959,7 +1994,14 @@ func (s *Server) filesWrite(w http.ResponseWriter, r *http.Request) {
|
|||||||
if !ok {
|
if !ok {
|
||||||
return
|
return
|
||||||
}
|
}
|
||||||
if err := fsys.Write(r.Context(), p, io.LimitReader(r.Body, 8<<20)); err != nil {
|
// Whole in memory first: the write truncates the file, so a body over the cap must be
|
||||||
|
// refused before that, not cut short after it.
|
||||||
|
body, err := io.ReadAll(http.MaxBytesReader(w, r.Body, 8<<20))
|
||||||
|
if err != nil {
|
||||||
|
jsonErr(w, 413, "too large to save here (8 MiB at most)")
|
||||||
|
return
|
||||||
|
}
|
||||||
|
if err := fsys.Write(r.Context(), p, bytes.NewReader(body)); err != nil {
|
||||||
fsErr(w, err)
|
fsErr(w, err)
|
||||||
return
|
return
|
||||||
}
|
}
|
||||||
@@ -2046,6 +2088,63 @@ func (s *Server) filesRename(w http.ResponseWriter, r *http.Request) {
|
|||||||
})
|
})
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// filesMove: POST {path, to} — to is the full destination path. A rename when the two
|
||||||
|
// ends share a filesystem (204); otherwise the copy-and-remove runs as a task (202 with the
|
||||||
|
// task), since a folder between datasets can take as long as it takes.
|
||||||
|
func (s *Server) filesMove(w http.ResponseWriter, r *http.Request) {
|
||||||
|
fsys, ok := s.fsFor(w, r)
|
||||||
|
if !ok {
|
||||||
|
return
|
||||||
|
}
|
||||||
|
var body struct{ Path, To string }
|
||||||
|
if err := json.NewDecoder(io.LimitReader(r.Body, 64<<10)).Decode(&body); err != nil {
|
||||||
|
jsonErr(w, 400, "bad request")
|
||||||
|
return
|
||||||
|
}
|
||||||
|
from, err := files.Clean(body.Path)
|
||||||
|
if err != nil {
|
||||||
|
jsonErr(w, 400, err.Error())
|
||||||
|
return
|
||||||
|
}
|
||||||
|
to, err := files.Clean(body.To)
|
||||||
|
if err != nil {
|
||||||
|
jsonErr(w, 400, err.Error())
|
||||||
|
return
|
||||||
|
}
|
||||||
|
if from == to {
|
||||||
|
jsonErr(w, 400, "it is already there")
|
||||||
|
return
|
||||||
|
}
|
||||||
|
if files.Inside(from, to) {
|
||||||
|
jsonErr(w, 400, "a folder can't be moved into itself")
|
||||||
|
return
|
||||||
|
}
|
||||||
|
// The kernel answers EXDEV for two paths on different filesystems before it looks for
|
||||||
|
// the source, so a missing file would otherwise become a task that fails on arrival.
|
||||||
|
if _, err := fsys.Stat(r.Context(), from); err != nil {
|
||||||
|
fsErr(w, err)
|
||||||
|
return
|
||||||
|
}
|
||||||
|
err = fsys.Rename(r.Context(), from, to)
|
||||||
|
if err == nil {
|
||||||
|
log.Printf("files: move %s -> %s (from %s)", from, to, clientIP(r))
|
||||||
|
w.WriteHeader(204)
|
||||||
|
return
|
||||||
|
}
|
||||||
|
if !files.IsCrossDevice(err) {
|
||||||
|
fsErr(w, err)
|
||||||
|
return
|
||||||
|
}
|
||||||
|
cmd := files.MoveCmd(context.Background(), fsys, s.self, from, to)
|
||||||
|
t, err := s.tasks.Start("files:move", "move "+filepath.Base(from)+" to "+filepath.Dir(to), cmd)
|
||||||
|
if err != nil {
|
||||||
|
jsonErr(w, 500, err.Error())
|
||||||
|
return
|
||||||
|
}
|
||||||
|
log.Printf("files: move %s -> %s across filesystems as task %s (from %s)", from, to, t.ID, clientIP(r))
|
||||||
|
jsonOK(w, 202, t.Snapshot(false))
|
||||||
|
}
|
||||||
|
|
||||||
func (s *Server) filesDelete(w http.ResponseWriter, r *http.Request) {
|
func (s *Server) filesDelete(w http.ResponseWriter, r *http.Request) {
|
||||||
s.filesOp(w, r, func(fsys files.FS, body map[string]any, p string) error {
|
s.filesOp(w, r, func(fsys files.FS, body map[string]any, p string) error {
|
||||||
rec, _ := body["recursive"].(bool)
|
rec, _ := body["recursive"].(bool)
|
||||||
|
|||||||
@@ -0,0 +1,418 @@
|
|||||||
|
package api
|
||||||
|
|
||||||
|
// Previews for the Files tab: GET /api/files/thumb hands back one small JPEG of an image or
|
||||||
|
// a video, made by internal/thumbs and kept in <data dir>/thumbs.db. The work is bounded
|
||||||
|
// two ways — a handful of ffmpegs at once, and one per file however many rows ask for it —
|
||||||
|
// because a folder of a thousand photos asks for every row the moment previews go on.
|
||||||
|
//
|
||||||
|
// Root mode (?sudo=1) makes the preview through the same root helper the file reads use,
|
||||||
|
// so a file only root can read gets a preview too, and the probe runs as root as well: root
|
||||||
|
// opens every render node, whatever groups the daemon's user has.
|
||||||
|
|
||||||
|
import (
|
||||||
|
"context"
|
||||||
|
"errors"
|
||||||
|
"log"
|
||||||
|
"net/http"
|
||||||
|
"os"
|
||||||
|
"os/exec"
|
||||||
|
"path/filepath"
|
||||||
|
"runtime"
|
||||||
|
"strconv"
|
||||||
|
"strings"
|
||||||
|
"sync"
|
||||||
|
"time"
|
||||||
|
|
||||||
|
"hamlab/internal/media"
|
||||||
|
"hamlab/internal/priv"
|
||||||
|
)
|
||||||
|
|
||||||
|
const thumbSize = 256
|
||||||
|
const thumbTimeout = 30 * time.Second
|
||||||
|
const capsTTL = 10 * time.Minute // how long a probe is believed; a refresh after an install or a group change does not wait
|
||||||
|
|
||||||
|
type thumbSvc struct {
|
||||||
|
cache *media.Cache
|
||||||
|
self string
|
||||||
|
sem chan struct{}
|
||||||
|
mu sync.Mutex
|
||||||
|
caps map[bool]*capsEntry // keyed by root
|
||||||
|
calls map[string]*thumbCall
|
||||||
|
}
|
||||||
|
|
||||||
|
type capsEntry struct {
|
||||||
|
mu sync.Mutex
|
||||||
|
caps media.Caps
|
||||||
|
err error
|
||||||
|
at time.Time
|
||||||
|
}
|
||||||
|
|
||||||
|
type thumbCall struct {
|
||||||
|
done chan struct{}
|
||||||
|
res media.Result
|
||||||
|
err error
|
||||||
|
}
|
||||||
|
|
||||||
|
func newThumbSvc(dataDir, self string) *thumbSvc {
|
||||||
|
n := runtime.NumCPU() / 2 // each ffmpeg is itself threaded; previews must not take the whole machine
|
||||||
|
if n < 2 {
|
||||||
|
n = 2
|
||||||
|
}
|
||||||
|
if n > 4 {
|
||||||
|
n = 4
|
||||||
|
}
|
||||||
|
return &thumbSvc{cache: media.NewCache(filepath.Join(dataDir, "thumbs.db")), self: self, sem: make(chan struct{}, n), caps: map[bool]*capsEntry{}, calls: map[string]*thumbCall{}}
|
||||||
|
}
|
||||||
|
|
||||||
|
// capsFor is what previews can be made with, probed once and kept for a while.
|
||||||
|
func (t *thumbSvc) capsFor(ctx context.Context, root, refresh bool) (media.Caps, error) {
|
||||||
|
t.mu.Lock()
|
||||||
|
e := t.caps[root]
|
||||||
|
if e == nil {
|
||||||
|
e = &capsEntry{}
|
||||||
|
t.caps[root] = e
|
||||||
|
}
|
||||||
|
t.mu.Unlock()
|
||||||
|
e.mu.Lock()
|
||||||
|
defer e.mu.Unlock()
|
||||||
|
if !refresh && !e.at.IsZero() && time.Since(e.at) < capsTTL {
|
||||||
|
return e.caps, e.err
|
||||||
|
}
|
||||||
|
if root {
|
||||||
|
e.caps, e.err = media.ProbeRoot(ctx, t.self)
|
||||||
|
} else {
|
||||||
|
e.caps, e.err = media.Probe(ctx), nil
|
||||||
|
}
|
||||||
|
e.at = time.Now()
|
||||||
|
mode := "as " + map[bool]string{false: "the daemon", true: "root"}[root]
|
||||||
|
switch {
|
||||||
|
case e.err != nil:
|
||||||
|
log.Printf("thumbs: probe %s failed: %v", mode, e.err)
|
||||||
|
case e.caps.Error != "":
|
||||||
|
log.Printf("thumbs: %s: %s", mode, e.caps.Error)
|
||||||
|
default:
|
||||||
|
log.Printf("thumbs: %s: ffmpeg %s, %s", mode, e.caps.Version, describeCaps(e.caps))
|
||||||
|
}
|
||||||
|
return e.caps, e.err
|
||||||
|
}
|
||||||
|
|
||||||
|
// describeCaps is the one line that says how previews get made here.
|
||||||
|
func describeCaps(c media.Caps) string {
|
||||||
|
if len(c.GPUs) == 0 {
|
||||||
|
return "CPU only"
|
||||||
|
}
|
||||||
|
parts := make([]string, 0, len(c.GPUs))
|
||||||
|
for _, g := range c.GPUs {
|
||||||
|
name := g.Name
|
||||||
|
if name == "" {
|
||||||
|
name = g.Device
|
||||||
|
}
|
||||||
|
p := name + " (" + g.API
|
||||||
|
if len(g.Codecs) > 0 {
|
||||||
|
p += ": " + strings.Join(g.Codecs, ", ")
|
||||||
|
}
|
||||||
|
parts = append(parts, p+")")
|
||||||
|
}
|
||||||
|
return "GPU " + strings.Join(parts, ", ")
|
||||||
|
}
|
||||||
|
|
||||||
|
// make produces a preview, sharing the work when the same file is asked for twice at once.
|
||||||
|
func (t *thumbSvc) make(ctx context.Context, root bool, key, path string) (media.Result, error) {
|
||||||
|
t.mu.Lock()
|
||||||
|
if c, ok := t.calls[key]; ok {
|
||||||
|
t.mu.Unlock()
|
||||||
|
select {
|
||||||
|
case <-c.done:
|
||||||
|
return c.res, c.err
|
||||||
|
case <-ctx.Done():
|
||||||
|
return media.Result{}, ctx.Err()
|
||||||
|
}
|
||||||
|
}
|
||||||
|
c := &thumbCall{done: make(chan struct{})}
|
||||||
|
t.calls[key] = c
|
||||||
|
t.mu.Unlock()
|
||||||
|
defer func() {
|
||||||
|
t.mu.Lock()
|
||||||
|
delete(t.calls, key)
|
||||||
|
t.mu.Unlock()
|
||||||
|
close(c.done)
|
||||||
|
}()
|
||||||
|
// The work itself runs on a context of its own: the first requester going away must not
|
||||||
|
// cancel a preview three other rows are waiting for.
|
||||||
|
wctx, cancel := context.WithTimeout(context.Background(), thumbTimeout+5*time.Second)
|
||||||
|
defer cancel()
|
||||||
|
select {
|
||||||
|
case t.sem <- struct{}{}:
|
||||||
|
defer func() { <-t.sem }()
|
||||||
|
case <-ctx.Done():
|
||||||
|
c.err = ctx.Err()
|
||||||
|
return c.res, c.err
|
||||||
|
}
|
||||||
|
caps, err := t.capsFor(wctx, root, false)
|
||||||
|
if err != nil {
|
||||||
|
c.err = err
|
||||||
|
return c.res, c.err
|
||||||
|
}
|
||||||
|
o := media.Opts{Size: thumbSize, Timeout: thumbTimeout}
|
||||||
|
if root {
|
||||||
|
c.res, c.err = media.MakeRoot(wctx, t.self, caps, path, o)
|
||||||
|
} else {
|
||||||
|
c.res, c.err = media.Make(wctx, caps, path, o)
|
||||||
|
}
|
||||||
|
return c.res, c.err
|
||||||
|
}
|
||||||
|
|
||||||
|
// ---- handlers ----
|
||||||
|
|
||||||
|
// filesThumb: GET /api/files/thumb?path=…&v=<mtime>-<size>[&sudo=1]. v is the client's idea of
|
||||||
|
// the file, from its listing; it makes a cache hit free of a stat, and a changed file a new URL.
|
||||||
|
func (s *Server) filesThumb(w http.ResponseWriter, r *http.Request) {
|
||||||
|
fsys, ok := s.fsFor(w, r)
|
||||||
|
if !ok {
|
||||||
|
return
|
||||||
|
}
|
||||||
|
p, ok := cleanQuery(w, r, "path")
|
||||||
|
if !ok {
|
||||||
|
return
|
||||||
|
}
|
||||||
|
if media.Kind(p) == "" {
|
||||||
|
jsonErr(w, 415, "no preview for this kind of file")
|
||||||
|
return
|
||||||
|
}
|
||||||
|
root := r.URL.Query().Get("sudo") == "1"
|
||||||
|
if mt, sz, ok := parseV(r.URL.Query().Get("v")); ok {
|
||||||
|
key := media.Key(p, sz, mt)
|
||||||
|
if r.Header.Get("If-None-Match") == `"`+key+`"` {
|
||||||
|
w.WriteHeader(304)
|
||||||
|
return
|
||||||
|
}
|
||||||
|
if s.serveCachedThumb(w, key) {
|
||||||
|
return
|
||||||
|
}
|
||||||
|
}
|
||||||
|
in, err := fsys.Stat(r.Context(), p)
|
||||||
|
if err != nil {
|
||||||
|
fsErr(w, err)
|
||||||
|
return
|
||||||
|
}
|
||||||
|
if in.Type == "dir" || in.Type == "other" || in.Broken {
|
||||||
|
jsonErr(w, 400, "not a regular file")
|
||||||
|
return
|
||||||
|
}
|
||||||
|
key := media.Key(p, in.Size, in.Mtime)
|
||||||
|
if s.serveCachedThumb(w, key) {
|
||||||
|
return
|
||||||
|
}
|
||||||
|
res, err := s.thumbs.make(r.Context(), root, key, p)
|
||||||
|
if err != nil {
|
||||||
|
var de *media.DecodeError
|
||||||
|
switch {
|
||||||
|
case errors.As(err, &de):
|
||||||
|
if err := s.thumbs.cache.Put(key, media.Entry{Path: p, Failure: de.Msg}); err != nil {
|
||||||
|
log.Printf("thumbs: cache: %v", err)
|
||||||
|
}
|
||||||
|
log.Printf("thumbs: %s: %s", p, de.Msg)
|
||||||
|
jsonErr(w, 422, de.Msg)
|
||||||
|
case errors.Is(err, context.DeadlineExceeded):
|
||||||
|
log.Printf("thumbs: %s: timed out", p)
|
||||||
|
jsonErr(w, 504, "preview timed out")
|
||||||
|
case errors.Is(err, context.Canceled):
|
||||||
|
jsonErr(w, 499, "cancelled")
|
||||||
|
default:
|
||||||
|
jsonErr(w, 503, err.Error())
|
||||||
|
}
|
||||||
|
return
|
||||||
|
}
|
||||||
|
if err := s.thumbs.cache.Put(key, media.Entry{Path: p, Engine: res.Engine, Codec: res.Codec, Width: res.Width, Height: res.Height, JPEG: res.JPEG}); err != nil {
|
||||||
|
log.Printf("thumbs: cache: %v", err)
|
||||||
|
}
|
||||||
|
writeThumb(w, key, res.JPEG, res.Engine)
|
||||||
|
}
|
||||||
|
|
||||||
|
// serveCachedThumb answers from the cache when it can. A cache that cannot be read is a
|
||||||
|
// miss: the preview is made again rather than refused.
|
||||||
|
func (s *Server) serveCachedThumb(w http.ResponseWriter, key string) bool {
|
||||||
|
e, found, err := s.thumbs.cache.Get(key)
|
||||||
|
if err != nil {
|
||||||
|
log.Printf("thumbs: cache: %v", err)
|
||||||
|
}
|
||||||
|
switch {
|
||||||
|
case !found:
|
||||||
|
return false
|
||||||
|
case e.Failure != "":
|
||||||
|
jsonErr(w, 422, e.Failure)
|
||||||
|
default:
|
||||||
|
writeThumb(w, key, e.JPEG, "cache ("+e.Engine+")")
|
||||||
|
}
|
||||||
|
return true
|
||||||
|
}
|
||||||
|
|
||||||
|
func writeThumb(w http.ResponseWriter, key string, jpeg []byte, engine string) {
|
||||||
|
w.Header().Set("Content-Type", "image/jpeg")
|
||||||
|
w.Header().Set("Content-Length", strconv.Itoa(len(jpeg)))
|
||||||
|
w.Header().Set("Cache-Control", "private, max-age=604800")
|
||||||
|
w.Header().Set("ETag", `"`+key+`"`)
|
||||||
|
w.Header().Set("X-Hamlab-Thumb", engine)
|
||||||
|
w.Header().Set("X-Content-Type-Options", "nosniff")
|
||||||
|
_, _ = w.Write(jpeg)
|
||||||
|
}
|
||||||
|
|
||||||
|
func parseV(v string) (mtime, size int64, ok bool) {
|
||||||
|
a, b, found := strings.Cut(v, "-")
|
||||||
|
if !found {
|
||||||
|
return 0, 0, false
|
||||||
|
}
|
||||||
|
mtime, err1 := strconv.ParseInt(a, 10, 64)
|
||||||
|
size, err2 := strconv.ParseInt(b, 10, 64)
|
||||||
|
return mtime, size, err1 == nil && err2 == nil
|
||||||
|
}
|
||||||
|
|
||||||
|
type thumbStatus struct {
|
||||||
|
OK bool `json:"ok"`
|
||||||
|
Engine string `json:"engine"` // gpu | cpu | ""
|
||||||
|
Summary string `json:"summary"`
|
||||||
|
FFmpeg string `json:"ffmpeg,omitempty"`
|
||||||
|
GPUs []media.GPU `json:"gpus"`
|
||||||
|
Unusable []media.GPU `json:"unusable,omitempty"`
|
||||||
|
Error string `json:"error,omitempty"`
|
||||||
|
Install *thumbInstall `json:"install,omitempty"` // ffmpeg is missing and this can put it on
|
||||||
|
GPUFix bool `json:"gpu_fix,omitempty"` // a render node is there but closed to the daemon's user, and this can open it
|
||||||
|
User string `json:"user"`
|
||||||
|
Cache thumbCache `json:"cache"`
|
||||||
|
}
|
||||||
|
|
||||||
|
type thumbCache struct {
|
||||||
|
Path string `json:"path"`
|
||||||
|
Entries int64 `json:"entries"`
|
||||||
|
Bytes int64 `json:"bytes"`
|
||||||
|
}
|
||||||
|
|
||||||
|
type thumbInstall struct {
|
||||||
|
PM string `json:"pm"`
|
||||||
|
Package string `json:"package"`
|
||||||
|
}
|
||||||
|
|
||||||
|
// thumbsStatus: GET /api/files/thumbs[?sudo=1][&refresh=1] — how previews get made on this machine.
|
||||||
|
func (s *Server) thumbsStatus(w http.ResponseWriter, r *http.Request) {
|
||||||
|
root := r.URL.Query().Get("sudo") == "1"
|
||||||
|
if root && s.fsSudo == nil {
|
||||||
|
jsonErr(w, 403, "root access needs passwordless sudo on the host")
|
||||||
|
return
|
||||||
|
}
|
||||||
|
caps, err := s.thumbs.capsFor(r.Context(), root, r.URL.Query().Get("refresh") == "1")
|
||||||
|
st := thumbStatus{GPUs: []media.GPU{}, User: s.user, Cache: thumbCache{Path: s.thumbs.cache.Path}}
|
||||||
|
st.Cache.Entries, st.Cache.Bytes = s.thumbs.cache.Stats()
|
||||||
|
if root {
|
||||||
|
st.User = "root"
|
||||||
|
}
|
||||||
|
if err != nil {
|
||||||
|
st.Error = err.Error()
|
||||||
|
jsonOK(w, 200, st)
|
||||||
|
return
|
||||||
|
}
|
||||||
|
st.FFmpeg, st.GPUs, st.Unusable, st.Error = caps.Version, caps.GPUs, caps.Unusable, caps.Error
|
||||||
|
if st.GPUs == nil {
|
||||||
|
st.GPUs = []media.GPU{}
|
||||||
|
}
|
||||||
|
st.OK = caps.Error == ""
|
||||||
|
switch {
|
||||||
|
case !st.OK:
|
||||||
|
if pkg := ffmpegPackage(s.updates.PM()); pkg != "" && priv.Available() {
|
||||||
|
st.Install = &thumbInstall{PM: s.updates.PM(), Package: pkg}
|
||||||
|
}
|
||||||
|
st.Summary = st.Error
|
||||||
|
case len(caps.GPUs) > 0:
|
||||||
|
st.Engine = "gpu"
|
||||||
|
st.Summary = describeCaps(caps) + " · images on the CPU"
|
||||||
|
default:
|
||||||
|
st.Engine = "cpu"
|
||||||
|
st.Summary = "CPU only"
|
||||||
|
}
|
||||||
|
if !root {
|
||||||
|
for _, g := range caps.Unusable {
|
||||||
|
if g.Permission && gpuFixPossible() {
|
||||||
|
st.GPUFix = true
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
jsonOK(w, 200, st)
|
||||||
|
}
|
||||||
|
|
||||||
|
// ffmpegPackage is what to install for ffmpeg on a package manager. Fedora ships it as
|
||||||
|
// ffmpeg-free; the others under its own name.
|
||||||
|
func ffmpegPackage(pm string) string {
|
||||||
|
switch pm {
|
||||||
|
case "":
|
||||||
|
return ""
|
||||||
|
case "dnf":
|
||||||
|
if b, _ := os.ReadFile("/etc/os-release"); strings.Contains(string(b), "ID=fedora") {
|
||||||
|
return "ffmpeg-free"
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return "ffmpeg"
|
||||||
|
}
|
||||||
|
|
||||||
|
const hamlabUnit = "/etc/systemd/system/hamlab.service"
|
||||||
|
|
||||||
|
// gpuFixPossible: the daemon is an ordinary user under systemd, with root to hand.
|
||||||
|
func gpuFixPossible() bool {
|
||||||
|
if os.Geteuid() == 0 || !priv.Available() {
|
||||||
|
return false
|
||||||
|
}
|
||||||
|
_, err := os.Stat(hamlabUnit)
|
||||||
|
return err == nil
|
||||||
|
}
|
||||||
|
|
||||||
|
// thumbsInstall: POST /api/files/thumbs/install — put ffmpeg on with the machine's package manager.
|
||||||
|
func (s *Server) thumbsInstall(w http.ResponseWriter, r *http.Request) {
|
||||||
|
pkg := ffmpegPackage(s.updates.PM())
|
||||||
|
if pkg == "" {
|
||||||
|
jsonErr(w, 400, "no supported package manager on this machine")
|
||||||
|
return
|
||||||
|
}
|
||||||
|
t, err := s.updates.Install([]string{pkg})
|
||||||
|
if err != nil {
|
||||||
|
jsonErr(w, 400, err.Error())
|
||||||
|
return
|
||||||
|
}
|
||||||
|
log.Printf("thumbs: installing %s as task %s (from %s)", pkg, t.ID, clientIP(r))
|
||||||
|
jsonOK(w, 202, t.Snapshot(false))
|
||||||
|
}
|
||||||
|
|
||||||
|
// gpuAccessScript gives hamlab.service the groups that own the render nodes, by a drop-in,
|
||||||
|
// then restarts it: a running process cannot gain a group.
|
||||||
|
const gpuAccessScript = `set -e
|
||||||
|
unit=` + hamlabUnit + `
|
||||||
|
[ -f "$unit" ] || { echo "no $unit: this machine does not run Ham Lab under systemd"; exit 1; }
|
||||||
|
groups=""
|
||||||
|
for g in render video; do
|
||||||
|
if getent group "$g" >/dev/null 2>&1; then groups="$groups $g"; fi
|
||||||
|
done
|
||||||
|
[ -n "$groups" ] || { echo "neither a render nor a video group exists here"; exit 1; }
|
||||||
|
mkdir -p "$unit.d"
|
||||||
|
printf '[Service]\nSupplementaryGroups=%s\n' "${groups# }" > "$unit.d/gpu.conf"
|
||||||
|
echo "hamlab.service: supplementary groups${groups} ($unit.d/gpu.conf)"
|
||||||
|
systemctl daemon-reload
|
||||||
|
echo 'Ham Lab restarts in 3 seconds to pick up its new groups; the page reconnects by itself.'
|
||||||
|
systemd-run --quiet --on-active=3 --unit=hamlab-gpu-restart systemctl restart hamlab.service
|
||||||
|
`
|
||||||
|
|
||||||
|
// thumbsGPUAccess: POST /api/files/thumbs/gpu-access — open the render nodes to the daemon.
|
||||||
|
func (s *Server) thumbsGPUAccess(w http.ResponseWriter, r *http.Request) {
|
||||||
|
if !gpuFixPossible() {
|
||||||
|
jsonErr(w, 400, "nothing to do: Ham Lab is root, has no root, or is not under systemd here")
|
||||||
|
return
|
||||||
|
}
|
||||||
|
if t, busy := s.tasks.RunningOf("thumbs:gpu"); busy { // a second run would only collide with the first's restart timer
|
||||||
|
jsonErr(w, 409, "already being done (task "+t.ID+")")
|
||||||
|
return
|
||||||
|
}
|
||||||
|
full := priv.Wrap("sh", "-c", gpuAccessScript)
|
||||||
|
t, err := s.tasks.Start("thumbs:gpu", "give Ham Lab access to the GPU", exec.Command(full[0], full[1:]...))
|
||||||
|
if err != nil {
|
||||||
|
jsonErr(w, 400, err.Error())
|
||||||
|
return
|
||||||
|
}
|
||||||
|
log.Printf("thumbs: GPU access drop-in as task %s, restart follows (from %s)", t.ID, clientIP(r))
|
||||||
|
jsonOK(w, 202, t.Snapshot(false))
|
||||||
|
}
|
||||||
@@ -48,6 +48,7 @@ func (s *Server) virtOverview(w http.ResponseWriter, r *http.Request) {
|
|||||||
imgs, _ := s.virt.Images(ctx)
|
imgs, _ := s.virt.Images(ctx)
|
||||||
out["images"] = imgs
|
out["images"] = imgs
|
||||||
out["pools"] = s.virt.Pools(ctx)
|
out["pools"] = s.virt.Pools(ctx)
|
||||||
|
out["default_pool"] = s.virt.DefaultPool(ctx)
|
||||||
out["networks"] = s.virt.Networks(ctx)
|
out["networks"] = s.virt.Networks(ctx)
|
||||||
}
|
}
|
||||||
isos := s.virt.Library(s.virt.ISOsDir())
|
isos := s.virt.Library(s.virt.ISOsDir())
|
||||||
@@ -285,6 +286,9 @@ func (s *Server) virtInstance(w http.ResponseWriter, r *http.Request) {
|
|||||||
jsonErr(w, 400, "bad name")
|
jsonErr(w, 400, "bad name")
|
||||||
return
|
return
|
||||||
}
|
}
|
||||||
|
if !s.virtReady(w, r) {
|
||||||
|
return
|
||||||
|
}
|
||||||
list, err := s.virt.Instances(r.Context())
|
list, err := s.virt.Instances(r.Context())
|
||||||
if err != nil {
|
if err != nil {
|
||||||
jsonErr(w, 502, err.Error())
|
jsonErr(w, 502, err.Error())
|
||||||
@@ -1094,3 +1098,70 @@ func (s *Server) virtSSHKeys(w http.ResponseWriter, _ *http.Request) {
|
|||||||
}
|
}
|
||||||
jsonOK(w, 200, map[string]any{"keys": keys, "user": s.user})
|
jsonOK(w, 200, map[string]any{"keys": keys, "user": s.user})
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// ---- locations: storage pools, added as plain-files pools at a path of his choosing ----
|
||||||
|
|
||||||
|
func (s *Server) virtPoolCreate(w http.ResponseWriter, r *http.Request) {
|
||||||
|
var body struct {
|
||||||
|
Name string `json:"name"`
|
||||||
|
Path string `json:"path"`
|
||||||
|
}
|
||||||
|
_ = readBody(r, &body, 8192)
|
||||||
|
p := filepath.Clean(body.Path)
|
||||||
|
name := strings.TrimSpace(body.Name)
|
||||||
|
if name == "" {
|
||||||
|
name = sensors.Seg(filepath.Base(p))
|
||||||
|
}
|
||||||
|
if !incus.ValidName(name) {
|
||||||
|
jsonErr(w, 400, "name: letters, digits and dashes, starting with a letter")
|
||||||
|
return
|
||||||
|
}
|
||||||
|
if !filepath.IsAbs(p) || p == "/" {
|
||||||
|
jsonErr(w, 400, "path: an absolute path to a folder on this machine")
|
||||||
|
return
|
||||||
|
}
|
||||||
|
if !s.virtReady(w, r) {
|
||||||
|
return
|
||||||
|
}
|
||||||
|
ctx, cancel := context.WithTimeout(r.Context(), 2*time.Minute)
|
||||||
|
defer cancel()
|
||||||
|
if st, err := os.Stat(p); err == nil && !st.IsDir() {
|
||||||
|
jsonErr(w, 400, p+" is a file, not a folder")
|
||||||
|
return
|
||||||
|
} else if err != nil {
|
||||||
|
mk := priv.Wrap("mkdir", "-p", p) // Incus wants the folder there; it fills it as root
|
||||||
|
if out, err := exec.CommandContext(ctx, mk[0], mk[1:]...).CombinedOutput(); err != nil {
|
||||||
|
jsonErr(w, 500, "could not make "+p+": "+strings.TrimSpace(string(out)))
|
||||||
|
return
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if out, err := s.virt.Cmd(ctx, "storage", "create", name, "dir", "source="+p).CombinedOutput(); err != nil {
|
||||||
|
jsonErr(w, 409, strings.TrimPrefix(strings.TrimSpace(string(out)), "Error: "))
|
||||||
|
return
|
||||||
|
}
|
||||||
|
log.Printf("virt: location %s added at %s (from %s)", name, p, clientIP(r))
|
||||||
|
jsonOK(w, 200, map[string]string{"name": name, "path": p})
|
||||||
|
}
|
||||||
|
|
||||||
|
func (s *Server) virtPoolDelete(w http.ResponseWriter, r *http.Request) {
|
||||||
|
var body struct {
|
||||||
|
Name string `json:"name"`
|
||||||
|
}
|
||||||
|
_ = readBody(r, &body, 4096)
|
||||||
|
if !incus.ValidName(body.Name) {
|
||||||
|
jsonErr(w, 400, "bad name")
|
||||||
|
return
|
||||||
|
}
|
||||||
|
if !s.virtReady(w, r) {
|
||||||
|
return
|
||||||
|
}
|
||||||
|
ctx, cancel := context.WithTimeout(r.Context(), 2*time.Minute)
|
||||||
|
defer cancel()
|
||||||
|
// Incus refuses a pool anything still uses; that refusal is the answer
|
||||||
|
if out, err := s.virt.Cmd(ctx, "storage", "delete", body.Name).CombinedOutput(); err != nil {
|
||||||
|
jsonErr(w, 409, strings.TrimPrefix(strings.TrimSpace(string(out)), "Error: "))
|
||||||
|
return
|
||||||
|
}
|
||||||
|
log.Printf("virt: location %s removed (from %s)", body.Name, clientIP(r))
|
||||||
|
w.WriteHeader(204)
|
||||||
|
}
|
||||||
|
|||||||
+9
-1
@@ -265,7 +265,15 @@ func (s *Server) zfsSnapshotDelete(w http.ResponseWriter, r *http.Request) {
|
|||||||
return
|
return
|
||||||
}
|
}
|
||||||
if err := s.storage.DestroySnapshot(r.Context(), body.Name, body.Recursive); err != nil {
|
if err := s.storage.DestroySnapshot(r.Context(), body.Name, body.Recursive); err != nil {
|
||||||
jsonErr(w, 500, err.Error())
|
msg := err.Error()
|
||||||
|
switch {
|
||||||
|
case msg == "bad snapshot name":
|
||||||
|
jsonErr(w, 400, msg)
|
||||||
|
case strings.Contains(msg, "does not exist"):
|
||||||
|
jsonErr(w, 404, msg)
|
||||||
|
default:
|
||||||
|
jsonErr(w, 500, msg)
|
||||||
|
}
|
||||||
return
|
return
|
||||||
}
|
}
|
||||||
log.Printf("zfs snapshot %s destroyed by %s%s", body.Name, clientIP(r), map[bool]string{true: " (recursive)", false: ""}[body.Recursive])
|
log.Printf("zfs snapshot %s destroyed by %s%s", body.Name, clientIP(r), map[bool]string{true: " (recursive)", false: ""}[body.Recursive])
|
||||||
|
|||||||
+73
-5
@@ -15,6 +15,7 @@ import (
|
|||||||
"io"
|
"io"
|
||||||
"os"
|
"os"
|
||||||
"os/exec"
|
"os/exec"
|
||||||
|
"os/signal"
|
||||||
"os/user"
|
"os/user"
|
||||||
"path/filepath"
|
"path/filepath"
|
||||||
"sort"
|
"sort"
|
||||||
@@ -23,6 +24,7 @@ import (
|
|||||||
"syscall"
|
"syscall"
|
||||||
"time"
|
"time"
|
||||||
|
|
||||||
|
"hamlab/internal/media"
|
||||||
"hamlab/internal/priv"
|
"hamlab/internal/priv"
|
||||||
)
|
)
|
||||||
|
|
||||||
@@ -37,6 +39,7 @@ type Entry struct {
|
|||||||
Group string `json:"group"`
|
Group string `json:"group"`
|
||||||
Target string `json:"target,omitempty"` // link target
|
Target string `json:"target,omitempty"` // link target
|
||||||
Broken bool `json:"broken,omitempty"` // dangling link
|
Broken bool `json:"broken,omitempty"` // dangling link
|
||||||
|
Preview string `json:"preview,omitempty"` // "image" | "video": the Files tab can show a thumbnail of it
|
||||||
}
|
}
|
||||||
|
|
||||||
type Listing struct {
|
type Listing struct {
|
||||||
@@ -56,10 +59,12 @@ type FS interface {
|
|||||||
List(ctx context.Context, path string) (Listing, error)
|
List(ctx context.Context, path string) (Listing, error)
|
||||||
Stat(ctx context.Context, path string) (Info, error)
|
Stat(ctx context.Context, path string) (Info, error)
|
||||||
Open(ctx context.Context, path string) (io.ReadCloser, error)
|
Open(ctx context.Context, path string) (io.ReadCloser, error)
|
||||||
|
OpenAt(ctx context.Context, path string, offset int64) (io.ReadCloser, error) // Open from a byte offset: a range request, a player seeking
|
||||||
Write(ctx context.Context, path string, r io.Reader) error
|
Write(ctx context.Context, path string, r io.Reader) error
|
||||||
Mkdir(ctx context.Context, path string) error
|
Mkdir(ctx context.Context, path string) error
|
||||||
Touch(ctx context.Context, path string) error
|
Touch(ctx context.Context, path string) error
|
||||||
Rename(ctx context.Context, from, to string) error
|
Rename(ctx context.Context, from, to string) error
|
||||||
|
Move(ctx context.Context, from, to string, progress io.Writer) error // Rename, or across filesystems a copy and a removal; long, so callers run it as a task
|
||||||
Delete(ctx context.Context, path string, recursive bool) error
|
Delete(ctx context.Context, path string, recursive bool) error
|
||||||
Chmod(ctx context.Context, path string, perm os.FileMode) error
|
Chmod(ctx context.Context, path string, perm os.FileMode) error
|
||||||
}
|
}
|
||||||
@@ -137,7 +142,7 @@ func entryOf(dir, name string, lst os.FileInfo) Entry {
|
|||||||
} else if st.IsDir() {
|
} else if st.IsDir() {
|
||||||
e.Type = "dir" // keep dir semantics for navigation; Target says it's a link
|
e.Type = "dir" // keep dir semantics for navigation; Target says it's a link
|
||||||
} else {
|
} else {
|
||||||
e.Size = st.Size()
|
e.Size, e.Mtime = st.Size(), st.ModTime().UnixMilli() // the file's, not the link's
|
||||||
}
|
}
|
||||||
case lst.IsDir():
|
case lst.IsDir():
|
||||||
e.Type = "dir"
|
e.Type = "dir"
|
||||||
@@ -146,6 +151,9 @@ func entryOf(dir, name string, lst os.FileInfo) Entry {
|
|||||||
default:
|
default:
|
||||||
e.Type = "other"
|
e.Type = "other"
|
||||||
}
|
}
|
||||||
|
if e.Type == "file" || (e.Type == "link" && !e.Broken) {
|
||||||
|
e.Preview = media.Kind(name)
|
||||||
|
}
|
||||||
return e
|
return e
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -224,6 +232,20 @@ func (Local) Open(_ context.Context, path string) (io.ReadCloser, error) {
|
|||||||
return os.Open(path)
|
return os.Open(path)
|
||||||
}
|
}
|
||||||
|
|
||||||
|
func (l Local) OpenAt(ctx context.Context, path string, offset int64) (io.ReadCloser, error) {
|
||||||
|
f, err := l.Open(ctx, path)
|
||||||
|
if err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
if offset > 0 {
|
||||||
|
if _, err := f.(*os.File).Seek(offset, io.SeekStart); err != nil {
|
||||||
|
f.Close()
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return f, nil
|
||||||
|
}
|
||||||
|
|
||||||
// Write replaces the file's contents, keeping its mode; new files get 0644 (umask).
|
// Write replaces the file's contents, keeping its mode; new files get 0644 (umask).
|
||||||
func (Local) Write(_ context.Context, path string, r io.Reader) error {
|
func (Local) Write(_ context.Context, path string, r io.Reader) error {
|
||||||
f, err := os.OpenFile(path, os.O_WRONLY|os.O_CREATE|os.O_TRUNC, 0o644)
|
f, err := os.OpenFile(path, os.O_WRONLY|os.O_CREATE|os.O_TRUNC, 0o644)
|
||||||
@@ -329,7 +351,15 @@ func (r *cmdReader) Close() error {
|
|||||||
}
|
}
|
||||||
|
|
||||||
func (s Sudo) Open(ctx context.Context, path string) (io.ReadCloser, error) {
|
func (s Sudo) Open(ctx context.Context, path string) (io.ReadCloser, error) {
|
||||||
c := s.cmd(ctx, "read", path)
|
return s.open(ctx, "read", path)
|
||||||
|
}
|
||||||
|
|
||||||
|
func (s Sudo) OpenAt(ctx context.Context, path string, offset int64) (io.ReadCloser, error) {
|
||||||
|
return s.open(ctx, "read", "--offset", strconv.FormatInt(offset, 10), path)
|
||||||
|
}
|
||||||
|
|
||||||
|
func (s Sudo) open(ctx context.Context, args ...string) (io.ReadCloser, error) {
|
||||||
|
c := s.cmd(ctx, args...)
|
||||||
errb := &strings.Builder{}
|
errb := &strings.Builder{}
|
||||||
c.Stderr = errb
|
c.Stderr = errb
|
||||||
out, err := c.StdoutPipe()
|
out, err := c.StdoutPipe()
|
||||||
@@ -380,6 +410,30 @@ func (s Sudo) Touch(ctx context.Context, path string) error {
|
|||||||
func (s Sudo) Rename(ctx context.Context, from, to string) error {
|
func (s Sudo) Rename(ctx context.Context, from, to string) error {
|
||||||
return s.runJSON(ctx, nil, "rename", from, to)
|
return s.runJSON(ctx, nil, "rename", from, to)
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// Move as root: the helper does the copying and talks on stdout.
|
||||||
|
func (s Sudo) Move(ctx context.Context, from, to string, progress io.Writer) error {
|
||||||
|
c := s.cmd(ctx, "move", from, to)
|
||||||
|
c.Stdout = progress
|
||||||
|
var errb strings.Builder
|
||||||
|
c.Stderr = &errb
|
||||||
|
if err := c.Run(); err != nil {
|
||||||
|
if msg := strings.TrimSpace(errb.String()); msg != "" {
|
||||||
|
return errors.New(msg)
|
||||||
|
}
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
|
||||||
|
// MoveCmd is the command a task runs for a long move: this binary's helper, as root when
|
||||||
|
// the backend is.
|
||||||
|
func MoveCmd(ctx context.Context, fsys FS, self, from, to string) *exec.Cmd {
|
||||||
|
if s, ok := fsys.(Sudo); ok {
|
||||||
|
return s.cmd(ctx, "move", from, to)
|
||||||
|
}
|
||||||
|
return exec.CommandContext(ctx, self, "fs", "move", from, to)
|
||||||
|
}
|
||||||
func (s Sudo) Delete(ctx context.Context, path string, recursive bool) error {
|
func (s Sudo) Delete(ctx context.Context, path string, recursive bool) error {
|
||||||
if recursive {
|
if recursive {
|
||||||
return s.runJSON(ctx, nil, "delete", "-r", path)
|
return s.runJSON(ctx, nil, "delete", "-r", path)
|
||||||
@@ -394,11 +448,16 @@ func (s Sudo) Chmod(ctx context.Context, path string, perm os.FileMode) error {
|
|||||||
func Helper(args []string, stdin io.Reader, stdout io.Writer, stderr io.Writer) int {
|
func Helper(args []string, stdin io.Reader, stdout io.Writer, stderr io.Writer) int {
|
||||||
fail := func(err error) int { fmt.Fprintln(stderr, err.Error()); return 1 }
|
fail := func(err error) int { fmt.Fprintln(stderr, err.Error()); return 1 }
|
||||||
if len(args) < 2 {
|
if len(args) < 2 {
|
||||||
return fail(errors.New("usage: hamlab fs <list|stat|read|write|mkdir|touch|rename|delete|chmod> …"))
|
return fail(errors.New("usage: hamlab fs <list|stat|read|write|mkdir|touch|rename|move|delete|chmod> …"))
|
||||||
}
|
}
|
||||||
ctx := context.Background()
|
ctx := context.Background()
|
||||||
var l Local
|
var l Local
|
||||||
op := args[0]
|
op := args[0]
|
||||||
|
var offset int64
|
||||||
|
if op == "read" && len(args) >= 4 && args[1] == "--offset" { // read --offset N <path>
|
||||||
|
offset, _ = strconv.ParseInt(args[2], 10, 64)
|
||||||
|
args = append([]string{"read"}, args[3:]...)
|
||||||
|
}
|
||||||
clean := func(i int) (string, error) {
|
clean := func(i int) (string, error) {
|
||||||
if i >= len(args) {
|
if i >= len(args) {
|
||||||
return "", errors.New("path required")
|
return "", errors.New("path required")
|
||||||
@@ -425,7 +484,7 @@ func Helper(args []string, stdin io.Reader, stdout io.Writer, stderr io.Writer)
|
|||||||
}
|
}
|
||||||
return emit(stdout, out, stderr)
|
return emit(stdout, out, stderr)
|
||||||
case "read":
|
case "read":
|
||||||
f, err := l.Open(ctx, p)
|
f, err := l.OpenAt(ctx, p, offset)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
return fail(err)
|
return fail(err)
|
||||||
}
|
}
|
||||||
@@ -447,7 +506,7 @@ func Helper(args []string, stdin io.Reader, stdout io.Writer, stderr io.Writer)
|
|||||||
return fail(err)
|
return fail(err)
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
case "rename":
|
case "rename", "move":
|
||||||
from, err := clean(1)
|
from, err := clean(1)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
return fail(err)
|
return fail(err)
|
||||||
@@ -456,6 +515,15 @@ func Helper(args []string, stdin io.Reader, stdout io.Writer, stderr io.Writer)
|
|||||||
if err != nil {
|
if err != nil {
|
||||||
return fail(err)
|
return fail(err)
|
||||||
}
|
}
|
||||||
|
if op == "move" {
|
||||||
|
// A cancelled task is a SIGTERM: stop copying and take the partial copy away.
|
||||||
|
mctx, stop := signal.NotifyContext(ctx, syscall.SIGTERM, syscall.SIGINT)
|
||||||
|
defer stop()
|
||||||
|
if err := l.Move(mctx, from, to, stdout); err != nil {
|
||||||
|
return fail(err)
|
||||||
|
}
|
||||||
|
return 0
|
||||||
|
}
|
||||||
if err := l.Rename(ctx, from, to); err != nil {
|
if err := l.Rename(ctx, from, to); err != nil {
|
||||||
return fail(err)
|
return fail(err)
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -0,0 +1,197 @@
|
|||||||
|
package files
|
||||||
|
|
||||||
|
import (
|
||||||
|
"context"
|
||||||
|
"errors"
|
||||||
|
"fmt"
|
||||||
|
"io"
|
||||||
|
"os"
|
||||||
|
"path/filepath"
|
||||||
|
"strings"
|
||||||
|
"syscall"
|
||||||
|
)
|
||||||
|
|
||||||
|
// IsCrossDevice reports a rename refused because the two ends are on different
|
||||||
|
// filesystems — on a NAS that is every move between datasets. The Sudo backend only has
|
||||||
|
// the helper's words for it.
|
||||||
|
func IsCrossDevice(err error) bool {
|
||||||
|
return errors.Is(err, syscall.EXDEV) || (err != nil && strings.Contains(err.Error(), "cross-device"))
|
||||||
|
}
|
||||||
|
|
||||||
|
// Inside reports whether to lies within from — a folder cannot be moved into itself.
|
||||||
|
func Inside(from, to string) bool {
|
||||||
|
return to == from || strings.HasPrefix(to, strings.TrimSuffix(from, "/")+"/")
|
||||||
|
}
|
||||||
|
|
||||||
|
// Move is Rename, and across filesystems a copy and a removal. The copy is built under a
|
||||||
|
// hidden temporary name beside where it will land and renamed into place only once it is
|
||||||
|
// complete, and the source is removed only after that: a failure part-way, or a cancel,
|
||||||
|
// leaves the source exactly as it was. Progress goes to w, a line at a time, for the task
|
||||||
|
// output that a long move is watched through.
|
||||||
|
func (Local) Move(ctx context.Context, from, to string, w io.Writer) error {
|
||||||
|
if protected[from] {
|
||||||
|
return errors.New("refusing to move " + from)
|
||||||
|
}
|
||||||
|
if Inside(from, to) {
|
||||||
|
return errors.New("a folder can't be moved into itself")
|
||||||
|
}
|
||||||
|
if _, err := os.Lstat(to); err == nil {
|
||||||
|
return errors.New("destination exists")
|
||||||
|
}
|
||||||
|
err := os.Rename(from, to)
|
||||||
|
if err == nil || !IsCrossDevice(err) {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
return moveByCopy(ctx, from, to, w)
|
||||||
|
}
|
||||||
|
|
||||||
|
// moveByCopy is the slow path: copy, rename into place, remove the original.
|
||||||
|
func moveByCopy(ctx context.Context, from, to string, w io.Writer) error {
|
||||||
|
st, err := os.Lstat(from)
|
||||||
|
if err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
tmp := filepath.Join(filepath.Dir(to), "."+filepath.Base(to)+".hamlab-moving")
|
||||||
|
_ = os.RemoveAll(tmp) // a previous attempt that died
|
||||||
|
fmt.Fprintf(w, "%s is on another filesystem: copying, then removing the original\n", filepath.Dir(to))
|
||||||
|
p := &progress{w: w}
|
||||||
|
if err := copyPath(ctx, from, tmp, st, p); err != nil {
|
||||||
|
fmt.Fprintln(w, "copy failed; removing the partial copy, the original is untouched")
|
||||||
|
_ = os.RemoveAll(tmp)
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
if err := os.Rename(tmp, to); err != nil {
|
||||||
|
_ = os.RemoveAll(tmp)
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
fmt.Fprintf(w, "copied %s; removing the original\n", p.String())
|
||||||
|
if err := os.RemoveAll(from); err != nil {
|
||||||
|
return fmt.Errorf("copied to %s, but the original could not be removed: %w", to, err)
|
||||||
|
}
|
||||||
|
fmt.Fprintf(w, "moved %s to %s\n", from, to)
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
|
||||||
|
type progress struct {
|
||||||
|
w io.Writer
|
||||||
|
files int64
|
||||||
|
bytes int64
|
||||||
|
shownFiles int64
|
||||||
|
shownBytes int64
|
||||||
|
}
|
||||||
|
|
||||||
|
func (p *progress) add(files, bytes int64) {
|
||||||
|
p.files += files
|
||||||
|
p.bytes += bytes
|
||||||
|
if p.files-p.shownFiles >= 500 || p.bytes-p.shownBytes >= 512<<20 {
|
||||||
|
fmt.Fprintf(p.w, "copying… %s\n", p.String())
|
||||||
|
p.shownFiles, p.shownBytes = p.files, p.bytes
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func (p *progress) String() string {
|
||||||
|
return fmt.Sprintf("%d files, %s", p.files, fmtSize(p.bytes))
|
||||||
|
}
|
||||||
|
|
||||||
|
func fmtSize(b int64) string {
|
||||||
|
const unit = 1024
|
||||||
|
if b < unit {
|
||||||
|
return fmt.Sprintf("%d B", b)
|
||||||
|
}
|
||||||
|
div, exp := int64(unit), 0
|
||||||
|
for n := b / unit; n >= unit && exp < 4; n /= unit {
|
||||||
|
div *= unit
|
||||||
|
exp++
|
||||||
|
}
|
||||||
|
return fmt.Sprintf("%.1f %ciB", float64(b)/float64(div), "KMGTP"[exp])
|
||||||
|
}
|
||||||
|
|
||||||
|
// copyPath copies one entry of any kind, recursively for a directory, keeping mode, times
|
||||||
|
// and — when running as root — ownership. Devices, pipes and sockets are refused: a copy
|
||||||
|
// of one is not the thing itself.
|
||||||
|
func copyPath(ctx context.Context, src, dst string, st os.FileInfo, p *progress) error {
|
||||||
|
if err := ctx.Err(); err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
mode := st.Mode()
|
||||||
|
switch {
|
||||||
|
case mode&os.ModeSymlink != 0:
|
||||||
|
target, err := os.Readlink(src)
|
||||||
|
if err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
if err := os.Symlink(target, dst); err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
chown(dst, st)
|
||||||
|
p.add(1, 0)
|
||||||
|
return nil
|
||||||
|
case mode.IsDir():
|
||||||
|
if err := os.Mkdir(dst, mode.Perm()|0o700); err != nil { // writable while it fills, exact mode after
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
ents, err := os.ReadDir(src)
|
||||||
|
if err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
for _, e := range ents {
|
||||||
|
lst, err := e.Info()
|
||||||
|
if err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
if err := copyPath(ctx, filepath.Join(src, e.Name()), filepath.Join(dst, e.Name()), lst, p); err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
}
|
||||||
|
chown(dst, st)
|
||||||
|
if err := os.Chmod(dst, mode.Perm()); err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
return os.Chtimes(dst, st.ModTime(), st.ModTime())
|
||||||
|
case mode.IsRegular():
|
||||||
|
n, err := copyFile(src, dst, st)
|
||||||
|
if err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
p.add(1, n)
|
||||||
|
return nil
|
||||||
|
default:
|
||||||
|
return fmt.Errorf("%s: can't copy a device, pipe or socket", src)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func copyFile(src, dst string, st os.FileInfo) (int64, error) {
|
||||||
|
in, err := os.Open(src)
|
||||||
|
if err != nil {
|
||||||
|
return 0, err
|
||||||
|
}
|
||||||
|
defer in.Close()
|
||||||
|
out, err := os.OpenFile(dst, os.O_WRONLY|os.O_CREATE|os.O_EXCL, st.Mode().Perm())
|
||||||
|
if err != nil {
|
||||||
|
return 0, err
|
||||||
|
}
|
||||||
|
n, err := io.Copy(out, in)
|
||||||
|
if err != nil {
|
||||||
|
out.Close()
|
||||||
|
return n, err
|
||||||
|
}
|
||||||
|
if err := out.Close(); err != nil {
|
||||||
|
return n, err
|
||||||
|
}
|
||||||
|
chown(dst, st)
|
||||||
|
if err := os.Chmod(dst, st.Mode().Perm()); err != nil { // umask has no say
|
||||||
|
return n, err
|
||||||
|
}
|
||||||
|
return n, os.Chtimes(dst, st.ModTime(), st.ModTime())
|
||||||
|
}
|
||||||
|
|
||||||
|
// chown keeps the owner when this process may set one; an ordinary user's copy is the
|
||||||
|
// user's, which is what a rename would have left anyway.
|
||||||
|
func chown(dst string, st os.FileInfo) {
|
||||||
|
if os.Geteuid() != 0 {
|
||||||
|
return
|
||||||
|
}
|
||||||
|
if sys, ok := st.Sys().(*syscall.Stat_t); ok {
|
||||||
|
_ = os.Lchown(dst, int(sys.Uid), int(sys.Gid))
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -0,0 +1,195 @@
|
|||||||
|
package files
|
||||||
|
|
||||||
|
import (
|
||||||
|
"context"
|
||||||
|
"errors"
|
||||||
|
"os"
|
||||||
|
"path/filepath"
|
||||||
|
"strings"
|
||||||
|
"syscall"
|
||||||
|
"testing"
|
||||||
|
"time"
|
||||||
|
)
|
||||||
|
|
||||||
|
func TestInside(t *testing.T) {
|
||||||
|
for _, c := range []struct {
|
||||||
|
from, to string
|
||||||
|
want bool
|
||||||
|
}{
|
||||||
|
{"/a/b", "/a/b", true}, {"/a/b", "/a/b/c", true}, {"/a/b", "/a/bc", false}, {"/a/b", "/a", false}, {"/", "/x", true},
|
||||||
|
} {
|
||||||
|
if got := Inside(c.from, c.to); got != c.want {
|
||||||
|
t.Errorf("Inside(%q, %q) = %v, want %v", c.from, c.to, got, c.want)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestIsCrossDevice(t *testing.T) {
|
||||||
|
if !IsCrossDevice(&os.LinkError{Op: "rename", Old: "/a", New: "/b", Err: syscall.EXDEV}) {
|
||||||
|
t.Fatal("the kernel's EXDEV")
|
||||||
|
}
|
||||||
|
if !IsCrossDevice(errors.New("rename /a /b: invalid cross-device link")) {
|
||||||
|
t.Fatal("the helper's words for it")
|
||||||
|
}
|
||||||
|
if IsCrossDevice(errors.New("permission denied")) || IsCrossDevice(nil) {
|
||||||
|
t.Fatal("anything else is not")
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// The copy path, exercised directly: a tree with nested folders, a symlink, an odd mode
|
||||||
|
// and old mtimes lands whole under the new name, and the original is gone only then.
|
||||||
|
func TestMoveByCopy(t *testing.T) {
|
||||||
|
root := t.TempDir()
|
||||||
|
src := filepath.Join(root, "src", "album")
|
||||||
|
if err := os.MkdirAll(filepath.Join(src, "sub", "deep"), 0o755); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
old := time.Date(2020, 1, 2, 3, 4, 5, 0, time.UTC)
|
||||||
|
write := func(p, s string, mode os.FileMode) {
|
||||||
|
if err := os.WriteFile(p, []byte(s), mode); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
_ = os.Chmod(p, mode)
|
||||||
|
_ = os.Chtimes(p, old, old)
|
||||||
|
}
|
||||||
|
write(filepath.Join(src, "a.txt"), "hello", 0o644)
|
||||||
|
write(filepath.Join(src, "sub", "deep", "b.bin"), strings.Repeat("x", 100000), 0o600)
|
||||||
|
write(filepath.Join(src, "run.sh"), "#!/bin/sh\n", 0o755)
|
||||||
|
if err := os.Symlink("a.txt", filepath.Join(src, "link")); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
_ = os.Chtimes(filepath.Join(src, "sub"), old, old)
|
||||||
|
|
||||||
|
dst := filepath.Join(root, "dst", "moved")
|
||||||
|
if err := os.MkdirAll(filepath.Dir(dst), 0o755); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
var out strings.Builder
|
||||||
|
if err := moveByCopy(context.Background(), src, dst, &out); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if _, err := os.Lstat(src); !os.IsNotExist(err) {
|
||||||
|
t.Fatal("the original is still there")
|
||||||
|
}
|
||||||
|
if b, _ := os.ReadFile(filepath.Join(dst, "a.txt")); string(b) != "hello" {
|
||||||
|
t.Fatal("a.txt")
|
||||||
|
}
|
||||||
|
if st, err := os.Stat(filepath.Join(dst, "sub", "deep", "b.bin")); err != nil || st.Size() != 100000 || st.Mode().Perm() != 0o600 || !st.ModTime().Equal(old) {
|
||||||
|
t.Fatalf("b.bin: %v %v", st, err)
|
||||||
|
}
|
||||||
|
if st, _ := os.Stat(filepath.Join(dst, "run.sh")); st.Mode().Perm() != 0o755 {
|
||||||
|
t.Fatal("mode lost")
|
||||||
|
}
|
||||||
|
if target, err := os.Readlink(filepath.Join(dst, "link")); err != nil || target != "a.txt" {
|
||||||
|
t.Fatal("symlink not copied as a symlink")
|
||||||
|
}
|
||||||
|
if st, _ := os.Stat(filepath.Join(dst, "sub")); !st.ModTime().Equal(old) {
|
||||||
|
t.Fatal("folder mtime lost")
|
||||||
|
}
|
||||||
|
if left, _ := filepath.Glob(filepath.Join(root, "dst", ".*hamlab-moving*")); len(left) != 0 {
|
||||||
|
t.Fatal("temporary copy left behind")
|
||||||
|
}
|
||||||
|
if !strings.Contains(out.String(), "moved ") || !strings.Contains(out.String(), "4 files") {
|
||||||
|
t.Fatalf("progress said: %q", out.String())
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// A failure part-way leaves the original as it was and no partial copy.
|
||||||
|
func TestMoveByCopyFailsClean(t *testing.T) {
|
||||||
|
root := t.TempDir()
|
||||||
|
src := filepath.Join(root, "src")
|
||||||
|
if err := os.MkdirAll(src, 0o755); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if err := os.WriteFile(filepath.Join(src, "a"), []byte("a"), 0o644); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if err := syscall.Mkfifo(filepath.Join(src, "pipe"), 0o644); err != nil {
|
||||||
|
t.Skip("no fifos here")
|
||||||
|
}
|
||||||
|
dst := filepath.Join(root, "dst")
|
||||||
|
err := moveByCopy(context.Background(), src, dst, &strings.Builder{})
|
||||||
|
if err == nil || !strings.Contains(err.Error(), "pipe") {
|
||||||
|
t.Fatalf("expected the pipe to be refused, got %v", err)
|
||||||
|
}
|
||||||
|
if _, err := os.Stat(filepath.Join(src, "a")); err != nil {
|
||||||
|
t.Fatal("the original was touched")
|
||||||
|
}
|
||||||
|
if _, err := os.Lstat(dst); !os.IsNotExist(err) {
|
||||||
|
t.Fatal("a partial copy was left")
|
||||||
|
}
|
||||||
|
if left, _ := filepath.Glob(filepath.Join(root, ".*hamlab-moving*")); len(left) != 0 {
|
||||||
|
t.Fatal("temporary copy left behind")
|
||||||
|
}
|
||||||
|
// a cancel is the same story
|
||||||
|
if err := os.Remove(filepath.Join(src, "pipe")); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
ctx, cancel := context.WithCancel(context.Background())
|
||||||
|
cancel()
|
||||||
|
if err := moveByCopy(ctx, src, dst, &strings.Builder{}); err == nil {
|
||||||
|
t.Fatal("a cancelled move must fail")
|
||||||
|
}
|
||||||
|
if _, err := os.Stat(filepath.Join(src, "a")); err != nil {
|
||||||
|
t.Fatal("the original was touched by a cancel")
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// Move on one filesystem is a rename: nothing is copied, and the guards hold.
|
||||||
|
func TestMoveSameFilesystem(t *testing.T) {
|
||||||
|
root := t.TempDir()
|
||||||
|
src := filepath.Join(root, "f")
|
||||||
|
if err := os.WriteFile(src, []byte("x"), 0o644); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
var l Local
|
||||||
|
var out strings.Builder
|
||||||
|
if err := l.Move(context.Background(), src, filepath.Join(root, "g"), &out); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if out.Len() != 0 {
|
||||||
|
t.Fatal("a rename has nothing to say")
|
||||||
|
}
|
||||||
|
if _, err := os.Stat(filepath.Join(root, "g")); err != nil {
|
||||||
|
t.Fatal("not renamed")
|
||||||
|
}
|
||||||
|
if err := l.Move(context.Background(), root, filepath.Join(root, "inside"), &out); err == nil {
|
||||||
|
t.Fatal("into itself")
|
||||||
|
}
|
||||||
|
if err := l.Move(context.Background(), filepath.Join(root, "g"), filepath.Join(root, "g"), &out); err == nil {
|
||||||
|
t.Fatal("onto itself")
|
||||||
|
}
|
||||||
|
if err := os.WriteFile(filepath.Join(root, "h"), nil, 0o644); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if err := l.Move(context.Background(), filepath.Join(root, "g"), filepath.Join(root, "h"), &out); err == nil || !strings.Contains(err.Error(), "exists") {
|
||||||
|
t.Fatalf("onto an existing file: %v", err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// A link to a file carries the file's size and time, so the row and the preview key follow
|
||||||
|
// the file; a link to a folder is a folder; a link to nothing says so.
|
||||||
|
func TestEntryOfLinks(t *testing.T) {
|
||||||
|
dir := t.TempDir()
|
||||||
|
if err := os.WriteFile(filepath.Join(dir, "f"), []byte("12345"), 0o644); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
old := time.Date(2021, 1, 1, 0, 0, 0, 0, time.UTC)
|
||||||
|
_ = os.Chtimes(filepath.Join(dir, "f"), old, old)
|
||||||
|
_ = os.Mkdir(filepath.Join(dir, "d"), 0o755)
|
||||||
|
for _, l := range [][2]string{{"lf", "f"}, {"ld", "d"}, {"lx", "nowhere"}} {
|
||||||
|
if err := os.Symlink(l[1], filepath.Join(dir, l[0])); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
ent := func(n string) Entry { st, _ := os.Lstat(filepath.Join(dir, n)); return entryOf(dir, n, st) }
|
||||||
|
if e := ent("lf"); e.Type != "link" || e.Size != 5 || e.Mtime != old.UnixMilli() || e.Target != "f" || e.Broken {
|
||||||
|
t.Fatalf("link to file: %+v", e)
|
||||||
|
}
|
||||||
|
if e := ent("ld"); e.Type != "dir" || e.Target != "d" {
|
||||||
|
t.Fatalf("link to dir: %+v", e)
|
||||||
|
}
|
||||||
|
if e := ent("lx"); e.Type != "link" || !e.Broken || e.Preview != "" {
|
||||||
|
t.Fatalf("dangling: %+v", e)
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -0,0 +1,89 @@
|
|||||||
|
package files
|
||||||
|
|
||||||
|
import (
|
||||||
|
"context"
|
||||||
|
"errors"
|
||||||
|
"io"
|
||||||
|
)
|
||||||
|
|
||||||
|
// Seeker is an FS file as http.ServeContent wants one — something to seek in — for a file
|
||||||
|
// whose size is already known from a stat. ServeContent seeks to the end for the size, to
|
||||||
|
// the start of the range asked for, then reads: so the backend is opened once, at that
|
||||||
|
// position, and only ever read forward. The Sudo backend's helper is started at the offset
|
||||||
|
// rather than streamed from the top and skipped.
|
||||||
|
type Seeker struct {
|
||||||
|
ctx context.Context
|
||||||
|
fsys FS
|
||||||
|
path string
|
||||||
|
size int64
|
||||||
|
pos int64 // where the next Read reads from
|
||||||
|
at int64 // where cur is positioned, when open
|
||||||
|
cur io.ReadCloser
|
||||||
|
}
|
||||||
|
|
||||||
|
func NewSeeker(ctx context.Context, fsys FS, path string, size int64) *Seeker {
|
||||||
|
return &Seeker{ctx: ctx, fsys: fsys, path: path, size: size}
|
||||||
|
}
|
||||||
|
|
||||||
|
// Check opens the file now, from the start, so a file that cannot be read is known before
|
||||||
|
// any headers go out. Seeking costs nothing; a Read from somewhere else reopens there.
|
||||||
|
func (s *Seeker) Check() error {
|
||||||
|
if s.cur != nil {
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
f, err := s.fsys.OpenAt(s.ctx, s.path, 0)
|
||||||
|
if err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
s.cur, s.at = f, 0
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
|
||||||
|
func (s *Seeker) Seek(offset int64, whence int) (int64, error) {
|
||||||
|
var pos int64
|
||||||
|
switch whence {
|
||||||
|
case io.SeekStart:
|
||||||
|
pos = offset
|
||||||
|
case io.SeekCurrent:
|
||||||
|
pos = s.pos + offset
|
||||||
|
case io.SeekEnd:
|
||||||
|
pos = s.size + offset
|
||||||
|
default:
|
||||||
|
return 0, errors.New("bad whence")
|
||||||
|
}
|
||||||
|
if pos < 0 {
|
||||||
|
return 0, errors.New("seek before start")
|
||||||
|
}
|
||||||
|
s.pos = pos // the stream, if open, stays where it is until a Read wants somewhere else
|
||||||
|
return pos, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
func (s *Seeker) Read(p []byte) (int, error) {
|
||||||
|
if s.pos >= s.size {
|
||||||
|
return 0, io.EOF
|
||||||
|
}
|
||||||
|
if s.cur != nil && s.pos != s.at {
|
||||||
|
s.cur.Close()
|
||||||
|
s.cur = nil
|
||||||
|
}
|
||||||
|
if s.cur == nil {
|
||||||
|
f, err := s.fsys.OpenAt(s.ctx, s.path, s.pos)
|
||||||
|
if err != nil {
|
||||||
|
return 0, err
|
||||||
|
}
|
||||||
|
s.cur, s.at = f, s.pos
|
||||||
|
}
|
||||||
|
n, err := s.cur.Read(p)
|
||||||
|
s.pos += int64(n)
|
||||||
|
s.at += int64(n)
|
||||||
|
return n, err
|
||||||
|
}
|
||||||
|
|
||||||
|
func (s *Seeker) Close() error {
|
||||||
|
if s.cur == nil {
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
err := s.cur.Close()
|
||||||
|
s.cur = nil
|
||||||
|
return err
|
||||||
|
}
|
||||||
@@ -0,0 +1,112 @@
|
|||||||
|
package files
|
||||||
|
|
||||||
|
import (
|
||||||
|
"context"
|
||||||
|
"io"
|
||||||
|
"os"
|
||||||
|
"path/filepath"
|
||||||
|
"testing"
|
||||||
|
)
|
||||||
|
|
||||||
|
import "strings"
|
||||||
|
|
||||||
|
type stringsBuilder = strings.Builder
|
||||||
|
|
||||||
|
type countingFS struct {
|
||||||
|
Local
|
||||||
|
opens []int64
|
||||||
|
}
|
||||||
|
|
||||||
|
func (c *countingFS) OpenAt(ctx context.Context, path string, off int64) (io.ReadCloser, error) {
|
||||||
|
c.opens = append(c.opens, off)
|
||||||
|
return c.Local.OpenAt(ctx, path, off)
|
||||||
|
}
|
||||||
|
|
||||||
|
// ServeContent's moves — size from the end, seek to the range, read forward — open the
|
||||||
|
// backend once, at the range; a seek to where the stream already is costs nothing.
|
||||||
|
func TestSeeker(t *testing.T) {
|
||||||
|
p := filepath.Join(t.TempDir(), "f")
|
||||||
|
data := make([]byte, 1000)
|
||||||
|
for i := range data {
|
||||||
|
data[i] = byte(i)
|
||||||
|
}
|
||||||
|
if err := os.WriteFile(p, data, 0o644); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
fs := &countingFS{}
|
||||||
|
s := NewSeeker(context.Background(), fs, p, int64(len(data)))
|
||||||
|
if n, _ := s.Seek(0, io.SeekEnd); n != 1000 {
|
||||||
|
t.Fatalf("end = %d", n)
|
||||||
|
}
|
||||||
|
if _, err := s.Seek(100, io.SeekStart); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
buf := make([]byte, 50)
|
||||||
|
if n, err := io.ReadFull(s, buf); err != nil || n != 50 || buf[0] != 100 || buf[49] != 149 {
|
||||||
|
t.Fatalf("read at 100: %d %v %v", n, err, buf[:2])
|
||||||
|
}
|
||||||
|
if _, err := s.Seek(150, io.SeekStart); err != nil { // where it already is
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if n, _ := io.ReadFull(s, buf); n != 50 || buf[0] != 150 {
|
||||||
|
t.Fatal("read after a no-op seek")
|
||||||
|
}
|
||||||
|
if len(fs.opens) != 1 || fs.opens[0] != 100 {
|
||||||
|
t.Fatalf("opens = %v, want [100]", fs.opens)
|
||||||
|
}
|
||||||
|
if _, err := s.Seek(0, io.SeekStart); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
rest, err := io.ReadAll(s)
|
||||||
|
if err != nil || len(rest) != 1000 || rest[999] != data[999] {
|
||||||
|
t.Fatalf("read all from the start: %d %v", len(rest), err)
|
||||||
|
}
|
||||||
|
if len(fs.opens) != 2 || fs.opens[1] != 0 {
|
||||||
|
t.Fatalf("opens = %v, want [100 0]", fs.opens)
|
||||||
|
}
|
||||||
|
if n, err := s.Read(buf); n != 0 || err != io.EOF {
|
||||||
|
t.Fatal("past the end is EOF")
|
||||||
|
}
|
||||||
|
if _, err := s.Seek(-1, io.SeekStart); err == nil {
|
||||||
|
t.Fatal("before the start")
|
||||||
|
}
|
||||||
|
if err := s.Close(); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
// Check opens at the start and keeps that stream for a read from the start
|
||||||
|
s2 := NewSeeker(context.Background(), fs, p, 1000)
|
||||||
|
if err := s2.Check(); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
s2.Seek(0, io.SeekEnd)
|
||||||
|
s2.Seek(0, io.SeekStart)
|
||||||
|
if n, _ := io.ReadFull(s2, buf); n != 50 || buf[0] != 0 {
|
||||||
|
t.Fatal("read from the start after Check")
|
||||||
|
}
|
||||||
|
if len(fs.opens) != 3 {
|
||||||
|
t.Fatalf("Check's open was not reused: %v", fs.opens)
|
||||||
|
}
|
||||||
|
s2.Close()
|
||||||
|
if err := NewSeeker(context.Background(), fs, filepath.Join(t.TempDir(), "nope"), 5).Check(); err == nil {
|
||||||
|
t.Fatal("Check must report a file that can't be opened")
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// The helper's read takes an offset, for the Sudo backend.
|
||||||
|
func TestHelperReadOffset(t *testing.T) {
|
||||||
|
p := filepath.Join(t.TempDir(), "f")
|
||||||
|
if err := os.WriteFile(p, []byte("0123456789"), 0o644); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
var out, errb stringsBuilder
|
||||||
|
if rc := Helper([]string{"read", "--offset", "7", p}, nil, &out, &errb); rc != 0 {
|
||||||
|
t.Fatalf("rc %d: %s", rc, errb.String())
|
||||||
|
}
|
||||||
|
if out.String() != "789" {
|
||||||
|
t.Fatalf("got %q", out.String())
|
||||||
|
}
|
||||||
|
out.Reset()
|
||||||
|
if rc := Helper([]string{"read", p}, nil, &out, &errb); rc != 0 || out.String() != "0123456789" {
|
||||||
|
t.Fatalf("plain read: rc %d %q", rc, out.String())
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -10,6 +10,7 @@ import (
|
|||||||
"encoding/json"
|
"encoding/json"
|
||||||
"errors"
|
"errors"
|
||||||
"fmt"
|
"fmt"
|
||||||
|
"hamlab/internal/store"
|
||||||
"os"
|
"os"
|
||||||
"os/exec"
|
"os/exec"
|
||||||
"path/filepath"
|
"path/filepath"
|
||||||
@@ -91,7 +92,7 @@ func (m *Manager) SaveExportConfig(c ExportConfig) error {
|
|||||||
}
|
}
|
||||||
c.Updated = time.Now()
|
c.Updated = time.Now()
|
||||||
b, _ := json.MarshalIndent(c, "", " ")
|
b, _ := json.MarshalIndent(c, "", " ")
|
||||||
return os.WriteFile(m.exportConfigPath(), b, 0o644)
|
return store.WriteFileAtomic(m.exportConfigPath(), b, 0o644)
|
||||||
}
|
}
|
||||||
|
|
||||||
type ExportRun struct {
|
type ExportRun struct {
|
||||||
@@ -226,6 +227,6 @@ func (m *Manager) RunExports(ctx context.Context, names []string, w func(string)
|
|||||||
}
|
}
|
||||||
run.Seconds = time.Since(start).Seconds()
|
run.Seconds = time.Since(start).Seconds()
|
||||||
b, _ := json.MarshalIndent(run, "", " ")
|
b, _ := json.MarshalIndent(run, "", " ")
|
||||||
_ = os.WriteFile(m.exportRunPath(), b, 0o644)
|
_ = store.WriteFileAtomic(m.exportRunPath(), b, 0o644)
|
||||||
return run
|
return run
|
||||||
}
|
}
|
||||||
|
|||||||
+66
-6
@@ -28,7 +28,7 @@ import (
|
|||||||
)
|
)
|
||||||
|
|
||||||
const (
|
const (
|
||||||
PoolName = "default"
|
PoolName = "local" // plain files on the machine's own disk, in Ham Lab's folder
|
||||||
BridgeName = "incusbr0"
|
BridgeName = "incusbr0"
|
||||||
ZPoolName = "hamlab-virt" // the ZFS pool made on the backing file; Incus is handed it by name
|
ZPoolName = "hamlab-virt" // the ZFS pool made on the backing file; Incus is handed it by name
|
||||||
AdminGroup = "incus-admin"
|
AdminGroup = "incus-admin"
|
||||||
@@ -615,11 +615,38 @@ type Pool struct {
|
|||||||
Name string `json:"name"`
|
Name string `json:"name"`
|
||||||
Driver string `json:"driver"`
|
Driver string `json:"driver"`
|
||||||
Source string `json:"source"`
|
Source string `json:"source"`
|
||||||
|
Where string `json:"where"` // in words: what disk this is, so a pool is picked by place, not by driver
|
||||||
Used uint64 `json:"used"`
|
Used uint64 `json:"used"`
|
||||||
Total uint64 `json:"total"`
|
Total uint64 `json:"total"`
|
||||||
UsedBy int `json:"used_by"`
|
UsedBy int `json:"used_by"`
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// poolWhere describes a pool by where its bytes land.
|
||||||
|
func (m *Manager) poolWhere(driver, source string) string {
|
||||||
|
inHam := strings.HasPrefix(source, m.dir+"/") || source == m.dir
|
||||||
|
switch driver {
|
||||||
|
case "dir":
|
||||||
|
if inHam {
|
||||||
|
return "this machine's own disk — plain files, each disk sized on its own"
|
||||||
|
}
|
||||||
|
return "plain files at " + source
|
||||||
|
case "zfs":
|
||||||
|
if strings.HasSuffix(source, ".img") || source == ZPoolName {
|
||||||
|
return "a ZFS pool inside one file on this machine's disk"
|
||||||
|
}
|
||||||
|
if strings.Contains(source, "/") {
|
||||||
|
return "ZFS dataset " + source
|
||||||
|
}
|
||||||
|
return "ZFS pool " + source
|
||||||
|
case "btrfs":
|
||||||
|
if inHam {
|
||||||
|
return "btrfs inside one file on this machine's disk"
|
||||||
|
}
|
||||||
|
return "btrfs at " + source
|
||||||
|
}
|
||||||
|
return driver + " at " + source
|
||||||
|
}
|
||||||
|
|
||||||
type Network struct {
|
type Network struct {
|
||||||
Name string `json:"name"`
|
Name string `json:"name"`
|
||||||
Type string `json:"type"`
|
Type string `json:"type"`
|
||||||
@@ -647,7 +674,7 @@ func (m *Manager) Pools(ctx context.Context) []Pool {
|
|||||||
}
|
}
|
||||||
var out2 []Pool
|
var out2 []Pool
|
||||||
for _, p := range raw {
|
for _, p := range raw {
|
||||||
pool := Pool{Name: p.Name, Driver: p.Driver, Source: p.Config["source"], UsedBy: len(p.UsedBy)}
|
pool := Pool{Name: p.Name, Driver: p.Driver, Source: p.Config["source"], Where: m.poolWhere(p.Driver, p.Config["source"]), UsedBy: len(p.UsedBy)}
|
||||||
if b, err := m.run(c, "query", "/1.0/storage-pools/"+p.Name+"/resources"); err == nil {
|
if b, err := m.run(c, "query", "/1.0/storage-pools/"+p.Name+"/resources"); err == nil {
|
||||||
var res struct {
|
var res struct {
|
||||||
Space struct {
|
Space struct {
|
||||||
@@ -664,6 +691,24 @@ func (m *Manager) Pools(ctx context.Context) []Pool {
|
|||||||
return out2
|
return out2
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// DefaultPool is where a new instance's disk goes unless the form says otherwise: the pool
|
||||||
|
// on the default profile's root device.
|
||||||
|
func (m *Manager) DefaultPool(ctx context.Context) string {
|
||||||
|
c, cancel := context.WithTimeout(ctx, 10*time.Second)
|
||||||
|
defer cancel()
|
||||||
|
out, err := m.run(c, "query", "/1.0/profiles/default")
|
||||||
|
if err != nil {
|
||||||
|
return ""
|
||||||
|
}
|
||||||
|
var p struct {
|
||||||
|
Devices map[string]map[string]string `json:"devices"`
|
||||||
|
}
|
||||||
|
if json.Unmarshal(out, &p) != nil {
|
||||||
|
return ""
|
||||||
|
}
|
||||||
|
return p.Devices["root"]["pool"]
|
||||||
|
}
|
||||||
|
|
||||||
func (m *Manager) Networks(ctx context.Context) []Network {
|
func (m *Manager) Networks(ctx context.Context) []Network {
|
||||||
c, cancel := context.WithTimeout(ctx, 20*time.Second)
|
c, cancel := context.WithTimeout(ctx, 20*time.Second)
|
||||||
defer cancel()
|
defer cancel()
|
||||||
@@ -869,6 +914,7 @@ type CreateSpec struct {
|
|||||||
CPU string `json:"cpu"`
|
CPU string `json:"cpu"`
|
||||||
Memory string `json:"memory"`
|
Memory string `json:"memory"`
|
||||||
Disk string `json:"disk"`
|
Disk string `json:"disk"`
|
||||||
|
Pool string `json:"pool"` // where the root disk goes; blank is the default profile's
|
||||||
Network string `json:"network"`
|
Network string `json:"network"`
|
||||||
CloudInit string `json:"cloud_init"`
|
CloudInit string `json:"cloud_init"`
|
||||||
Profiles []string `json:"profiles"`
|
Profiles []string `json:"profiles"`
|
||||||
@@ -891,6 +937,8 @@ func CreateArgs(sp CreateSpec, kvm bool) ([]string, error) {
|
|||||||
return nil, errors.New("memory: a size, like 2GiB")
|
return nil, errors.New("memory: a size, like 2GiB")
|
||||||
case !ValidSize(sp.Disk):
|
case !ValidSize(sp.Disk):
|
||||||
return nil, errors.New("disk: a size, like 20GiB")
|
return nil, errors.New("disk: a size, like 20GiB")
|
||||||
|
case sp.Pool != "" && !ValidName(sp.Pool):
|
||||||
|
return nil, errors.New("pool: a storage pool's name")
|
||||||
case !ValidNetwork(sp.Network):
|
case !ValidNetwork(sp.Network):
|
||||||
return nil, errors.New("network: a bridge name")
|
return nil, errors.New("network: a bridge name")
|
||||||
case len(sp.CloudInit) > 256<<10:
|
case len(sp.CloudInit) > 256<<10:
|
||||||
@@ -912,8 +960,16 @@ func CreateArgs(sp CreateSpec, kvm bool) ([]string, error) {
|
|||||||
if sp.Memory != "" {
|
if sp.Memory != "" {
|
||||||
args = append(args, "-c", "limits.memory="+sp.Memory)
|
args = append(args, "-c", "limits.memory="+sp.Memory)
|
||||||
}
|
}
|
||||||
|
// the root disk: the profile's, with the pool and size overridden where given
|
||||||
|
var root []string
|
||||||
|
if sp.Pool != "" {
|
||||||
|
root = append(root, "pool="+sp.Pool)
|
||||||
|
}
|
||||||
if sp.Disk != "" {
|
if sp.Disk != "" {
|
||||||
args = append(args, "-d", "root,size="+sp.Disk)
|
root = append(root, "size="+sp.Disk)
|
||||||
|
}
|
||||||
|
if len(root) > 0 {
|
||||||
|
args = append(args, "-d", "root,"+strings.Join(root, ","))
|
||||||
}
|
}
|
||||||
if sp.Network != "" {
|
if sp.Network != "" {
|
||||||
args = append(args, "-n", sp.Network)
|
args = append(args, "-n", sp.Network)
|
||||||
@@ -1104,6 +1160,7 @@ type FromISOSpec struct {
|
|||||||
CPU string `json:"cpu"`
|
CPU string `json:"cpu"`
|
||||||
Memory string `json:"memory"`
|
Memory string `json:"memory"`
|
||||||
Disk string `json:"disk"`
|
Disk string `json:"disk"`
|
||||||
|
Pool string `json:"pool"`
|
||||||
Network string `json:"network"`
|
Network string `json:"network"`
|
||||||
SecureBoot bool `json:"secureboot"`
|
SecureBoot bool `json:"secureboot"`
|
||||||
Start bool `json:"start"`
|
Start bool `json:"start"`
|
||||||
@@ -1121,6 +1178,8 @@ func FromISOScript(sp FromISOSpec, kvm bool) (script string, params []string, er
|
|||||||
return "", nil, errors.New("iso: pick one from the library")
|
return "", nil, errors.New("iso: pick one from the library")
|
||||||
case !ValidCPU(sp.CPU) || !ValidSize(sp.Memory) || !ValidSize(sp.Disk) || !ValidNetwork(sp.Network):
|
case !ValidCPU(sp.CPU) || !ValidSize(sp.Memory) || !ValidSize(sp.Disk) || !ValidNetwork(sp.Network):
|
||||||
return "", nil, errors.New("cpu, memory, disk: a count and two sizes, like 2, 2GiB, 20GiB")
|
return "", nil, errors.New("cpu, memory, disk: a count and two sizes, like 2, 2GiB, 20GiB")
|
||||||
|
case sp.Pool != "" && !ValidName(sp.Pool):
|
||||||
|
return "", nil, errors.New("pool: a storage pool's name")
|
||||||
}
|
}
|
||||||
cpu, mem, disk := sp.CPU, sp.Memory, sp.Disk
|
cpu, mem, disk := sp.CPU, sp.Memory, sp.Disk
|
||||||
if cpu == "" {
|
if cpu == "" {
|
||||||
@@ -1140,13 +1199,14 @@ func FromISOScript(sp FromISOSpec, kvm bool) (script string, params []string, er
|
|||||||
if sp.Start {
|
if sp.Start {
|
||||||
start = "1"
|
start = "1"
|
||||||
}
|
}
|
||||||
// $1 name $2 iso path $3 cpu $4 memory $5 disk $6 network $7 secureboot $8 start
|
// $1 name $2 iso path $3 cpu $4 memory $5 disk $6 network $7 secureboot $8 start $9 pool
|
||||||
script = `set -e
|
script = `set -e
|
||||||
NET=""; [ -n "$6" ] && NET="-n $6"
|
NET=""; [ -n "$6" ] && NET="-n $6"
|
||||||
incus init --empty --vm "$1" -c limits.cpu="$3" -c limits.memory="$4" -c security.secureboot="$7" -d root,size="$5" $NET
|
ROOT="root,size=$5"; [ -n "$9" ] && ROOT="root,pool=$9,size=$5"
|
||||||
|
incus init --empty --vm "$1" -c limits.cpu="$3" -c limits.memory="$4" -c security.secureboot="$7" -d "$ROOT" $NET
|
||||||
incus config device add "$1" iso0 disk source="$2" boot.priority=10
|
incus config device add "$1" iso0 disk source="$2" boot.priority=10
|
||||||
echo "made $1 with $2 as its first boot device"
|
echo "made $1 with $2 as its first boot device"
|
||||||
[ "$8" = 1 ] && incus start "$1" && echo "started; open the VGA console to run the installer"
|
[ "$8" = 1 ] && incus start "$1" && echo "started; open the VGA console to run the installer"
|
||||||
exit 0`
|
exit 0`
|
||||||
return script, []string{sp.Name, sp.ISO, cpu, mem, disk, sp.Network, sb, start}, nil
|
return script, []string{sp.Name, sp.ISO, cpu, mem, disk, sp.Network, sb, start, sp.Pool}, nil
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -21,13 +21,13 @@ func TestInstallScriptShapes(t *testing.T) {
|
|||||||
}{
|
}{
|
||||||
{"grid: zfs, apt, docker", Status{State: "installable", PM: "apt", Arch: "x86_64", Drivers: []string{"zfs", "btrfs", "dir"}, Docker: true, ZFSHost: true, DiskFree: 700 << 30, MemTotal: 62 << 30},
|
{"grid: zfs, apt, docker", Status{State: "installable", PM: "apt", Arch: "x86_64", Drivers: []string{"zfs", "btrfs", "dir"}, Docker: true, ZFSHost: true, DiskFree: 700 << 30, MemTotal: 62 << 30},
|
||||||
InstallOpts{Driver: "zfs", PoolSizeGiB: 300, Subnet: "172.16.100.1/24", DockerRules: true, ARCMaxGiB: 16},
|
InstallOpts{Driver: "zfs", PoolSizeGiB: 300, Subnet: "172.16.100.1/24", DockerRules: true, ARCMaxGiB: 16},
|
||||||
[]string{"apt-get", "zpool create", "incus storage create default zfs source=hamlab-virt", "ipv4.address='172.16.100.1/24'", "DOCKER-USER", "zfs_arc_max", "usermod -aG incus-admin 'core'", "SupplementaryGroups=incus-admin", "systemd-run"}},
|
[]string{"apt-get", "zpool create", "incus storage create local zfs source=hamlab-virt", "ipv4.address='172.16.100.1/24'", "DOCKER-USER", "zfs_arc_max", "usermod -aG incus-admin 'core'", "SupplementaryGroups=incus-admin", "systemd-run"}},
|
||||||
{"core: btrfs, apt, docker, no tools yet", Status{State: "installable", PM: "apt", Arch: "x86_64", Drivers: []string{"btrfs", "dir"}, Docker: true, DiskFree: 170 << 30, MemTotal: 8 << 30},
|
{"core: btrfs, apt, docker, no tools yet", Status{State: "installable", PM: "apt", Arch: "x86_64", Drivers: []string{"btrfs", "dir"}, Docker: true, DiskFree: 170 << 30, MemTotal: 8 << 30},
|
||||||
InstallOpts{Driver: "btrfs", PoolSizeGiB: 40, Subnet: "172.16.100.1/24", DockerRules: true},
|
InstallOpts{Driver: "btrfs", PoolSizeGiB: 40, Subnet: "172.16.100.1/24", DockerRules: true},
|
||||||
[]string{"btrfs-progs", "mkfs.btrfs", "/etc/fstab", "incus storage create default btrfs source="}},
|
[]string{"btrfs-progs", "mkfs.btrfs", "/etc/fstab", "incus storage create local btrfs source="}},
|
||||||
{"mac: dir, dnf, arm64", Status{State: "installable", PM: "dnf", Arch: "aarch64", Drivers: []string{"dir"}, DiskFree: 200 << 30, MemTotal: 15 << 30},
|
{"mac: dir, dnf, arm64", Status{State: "installable", PM: "dnf", Arch: "aarch64", Drivers: []string{"dir"}, DiskFree: 200 << 30, MemTotal: 15 << 30},
|
||||||
InstallOpts{Driver: "dir", Subnet: "172.16.101.1/24"},
|
InstallOpts{Driver: "dir", Subnet: "172.16.101.1/24"},
|
||||||
[]string{"dnf -y install incus qemu-system-aarch64-core edk2-aarch64", "incus storage create default dir source="}},
|
[]string{"dnf -y install incus qemu-system-aarch64-core edk2-aarch64", "incus storage create local dir source="}},
|
||||||
{"repair: already installed", Status{State: "repair", PM: "apt", Arch: "x86_64", Drivers: []string{"dir"}, DiskFree: 10 << 30, MemTotal: 4 << 30},
|
{"repair: already installed", Status{State: "repair", PM: "apt", Arch: "x86_64", Drivers: []string{"dir"}, DiskFree: 10 << 30, MemTotal: 4 << 30},
|
||||||
InstallOpts{Driver: "dir", Subnet: "172.16.100.1/24"},
|
InstallOpts{Driver: "dir", Subnet: "172.16.100.1/24"},
|
||||||
[]string{"already installed"}},
|
[]string{"already installed"}},
|
||||||
@@ -75,6 +75,9 @@ func TestCreateArgs(t *testing.T) {
|
|||||||
t.Fatal(err)
|
t.Fatal(err)
|
||||||
}
|
}
|
||||||
want := "launch images:debian/13 web-1 --vm -c limits.cpu=2 -c limits.memory=2GiB -d root,size=20GiB -n incusbr0"
|
want := "launch images:debian/13 web-1 --vm -c limits.cpu=2 -c limits.memory=2GiB -d root,size=20GiB -n incusbr0"
|
||||||
|
if a, err := CreateArgs(CreateSpec{Name: "x", Image: "images:alpine/3.22", Pool: "local"}, true); err != nil || strings.Join(a, " ") != "init images:alpine/3.22 x -d root,pool=local" {
|
||||||
|
t.Fatalf("pool override: %v %v", a, err)
|
||||||
|
}
|
||||||
if got := strings.Join(args, " "); got != want {
|
if got := strings.Join(args, " "); got != want {
|
||||||
t.Fatalf("got %q", got)
|
t.Fatalf("got %q", got)
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -14,7 +14,6 @@ import (
|
|||||||
"net/netip"
|
"net/netip"
|
||||||
"os"
|
"os"
|
||||||
"os/exec"
|
"os/exec"
|
||||||
"strings"
|
|
||||||
"time"
|
"time"
|
||||||
)
|
)
|
||||||
|
|
||||||
@@ -125,6 +124,3 @@ func dockerPools() []string {
|
|||||||
}
|
}
|
||||||
return []string{"172.17.0.0/12", "192.168.0.0/16"}
|
return []string{"172.17.0.0/12", "192.168.0.0/16"}
|
||||||
}
|
}
|
||||||
|
|
||||||
// String form used in the layout file and on the page.
|
|
||||||
func prefixString(p netip.Prefix) string { return strings.TrimSpace(p.Masked().String()) }
|
|
||||||
|
|||||||
@@ -130,7 +130,7 @@ func (m *Manager) openVGA(ctx context.Context, client *http.Client, name string,
|
|||||||
resp, err := client.Do(req)
|
resp, err := client.Do(req)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
if isPermission(err) {
|
if isPermission(err) {
|
||||||
return nil, errors.New("Ham Lab's account can't reach Incus's socket; Finish setup on the Virtualization tab puts it in incus-admin")
|
return nil, errors.New("this account can't reach Incus's socket; Finish setup on the Virtualization tab puts Ham Lab in incus-admin")
|
||||||
}
|
}
|
||||||
return nil, err
|
return nil, err
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -0,0 +1,218 @@
|
|||||||
|
package media
|
||||||
|
|
||||||
|
import (
|
||||||
|
"crypto/sha256"
|
||||||
|
"encoding/binary"
|
||||||
|
"encoding/hex"
|
||||||
|
"encoding/json"
|
||||||
|
"errors"
|
||||||
|
"os"
|
||||||
|
"strconv"
|
||||||
|
"sync"
|
||||||
|
"time"
|
||||||
|
|
||||||
|
bolt "go.etcd.io/bbolt"
|
||||||
|
)
|
||||||
|
|
||||||
|
// Cache keeps finished previews in one database file under Ham Lab's own data directory —
|
||||||
|
// thumbs.db, a bbolt key/value store: pure Go, so the binary stays static, and one file
|
||||||
|
// per machine rather than a tree of hash-named JPEGs. An entry is one file's preview keyed
|
||||||
|
// by its path, size and mtime, so a file that changes gets a new preview and an unchanged
|
||||||
|
// one is never decoded twice. A file ffmpeg could not decode is kept the same way, with
|
||||||
|
// ffmpeg's reason and no picture, so a folder of odd files does not re-run ffmpeg on every
|
||||||
|
// visit. Nothing expires: a preview is a few kilobytes, and only files somebody actually
|
||||||
|
// scrolled past are ever made. The file is opened the first time something is stored, so a
|
||||||
|
// machine nobody browses with previews on never has one.
|
||||||
|
type Cache struct {
|
||||||
|
Path string
|
||||||
|
mu sync.Mutex
|
||||||
|
db *bolt.DB
|
||||||
|
err error
|
||||||
|
}
|
||||||
|
|
||||||
|
// Entry is one cached preview, or the remembered reason there is none.
|
||||||
|
type Entry struct {
|
||||||
|
Path string `json:"path"`
|
||||||
|
Engine string `json:"engine,omitempty"`
|
||||||
|
Codec string `json:"codec,omitempty"`
|
||||||
|
Width int `json:"width,omitempty"`
|
||||||
|
Height int `json:"height,omitempty"`
|
||||||
|
Made time.Time `json:"made"`
|
||||||
|
Failure string `json:"failure,omitempty"` // set instead of JPEG when ffmpeg refused the file
|
||||||
|
JPEG []byte `json:"-"`
|
||||||
|
}
|
||||||
|
|
||||||
|
var (
|
||||||
|
bucketThumbs = []byte("thumbs")
|
||||||
|
bucketMeta = []byte("meta")
|
||||||
|
keyCount = []byte("count")
|
||||||
|
)
|
||||||
|
|
||||||
|
// Key identifies one file's preview. "v1" is the preview format: bump it to retire every
|
||||||
|
// cached preview at once.
|
||||||
|
func Key(path string, size, mtime int64) string {
|
||||||
|
h := sha256.Sum256([]byte("v1\x00" + path + "\x00" + strconv.FormatInt(size, 10) + "\x00" + strconv.FormatInt(mtime, 10)))
|
||||||
|
return hex.EncodeToString(h[:])
|
||||||
|
}
|
||||||
|
|
||||||
|
func NewCache(path string) *Cache { return &Cache{Path: path} }
|
||||||
|
|
||||||
|
// open the database once; an open that failed is not retried, and previews still get made
|
||||||
|
// and served — they are simply not kept, and the log says why.
|
||||||
|
func (c *Cache) open() (*bolt.DB, error) {
|
||||||
|
c.mu.Lock()
|
||||||
|
defer c.mu.Unlock()
|
||||||
|
if c.db != nil || c.err != nil {
|
||||||
|
return c.db, c.err
|
||||||
|
}
|
||||||
|
db, err := bolt.Open(c.Path, 0o600, &bolt.Options{Timeout: 2 * time.Second})
|
||||||
|
if err != nil {
|
||||||
|
c.err = err
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
if err := db.Update(func(tx *bolt.Tx) error {
|
||||||
|
if _, err := tx.CreateBucketIfNotExists(bucketThumbs); err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
_, err := tx.CreateBucketIfNotExists(bucketMeta)
|
||||||
|
return err
|
||||||
|
}); err != nil {
|
||||||
|
db.Close()
|
||||||
|
c.err = err
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
c.db = db
|
||||||
|
return db, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
// Close releases the file. Previews after it are made but not kept.
|
||||||
|
func (c *Cache) Close() error {
|
||||||
|
c.mu.Lock()
|
||||||
|
defer c.mu.Unlock()
|
||||||
|
if c.db == nil {
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
err := c.db.Close()
|
||||||
|
c.db, c.err = nil, errors.New("cache closed")
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
|
||||||
|
// Get returns the entry for a key. A database that does not exist yet is a miss, not a reason
|
||||||
|
// to make one.
|
||||||
|
func (c *Cache) Get(key string) (Entry, bool, error) {
|
||||||
|
c.mu.Lock()
|
||||||
|
db := c.db
|
||||||
|
c.mu.Unlock()
|
||||||
|
if db == nil {
|
||||||
|
if _, err := os.Stat(c.Path); err != nil {
|
||||||
|
return Entry{}, false, nil
|
||||||
|
}
|
||||||
|
var err error
|
||||||
|
if db, err = c.open(); err != nil {
|
||||||
|
return Entry{}, false, err
|
||||||
|
}
|
||||||
|
}
|
||||||
|
var e Entry
|
||||||
|
found := false
|
||||||
|
err := db.View(func(tx *bolt.Tx) error {
|
||||||
|
v := tx.Bucket(bucketThumbs).Get([]byte(key))
|
||||||
|
if v == nil {
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
var err error
|
||||||
|
e, err = decode(v) // copies: a value is only valid inside the transaction
|
||||||
|
found = err == nil
|
||||||
|
return err
|
||||||
|
})
|
||||||
|
return e, found, err
|
||||||
|
}
|
||||||
|
|
||||||
|
// Put stores an entry, replacing any under the same key.
|
||||||
|
func (c *Cache) Put(key string, e Entry) error {
|
||||||
|
if len(e.JPEG) == 0 && e.Failure == "" {
|
||||||
|
return errors.New("empty preview")
|
||||||
|
}
|
||||||
|
if e.Made.IsZero() {
|
||||||
|
e.Made = time.Now()
|
||||||
|
}
|
||||||
|
db, err := c.open()
|
||||||
|
if err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
v, err := encode(e)
|
||||||
|
if err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
return db.Update(func(tx *bolt.Tx) error {
|
||||||
|
b := tx.Bucket(bucketThumbs)
|
||||||
|
k := []byte(key)
|
||||||
|
fresh := b.Get(k) == nil
|
||||||
|
if err := b.Put(k, v); err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
if fresh {
|
||||||
|
m := tx.Bucket(bucketMeta)
|
||||||
|
return m.Put(keyCount, be64(count(m)+1))
|
||||||
|
}
|
||||||
|
return nil
|
||||||
|
})
|
||||||
|
}
|
||||||
|
|
||||||
|
// Stats is how many previews are kept and how big the file is.
|
||||||
|
func (c *Cache) Stats() (entries int64, bytes int64) {
|
||||||
|
st, err := os.Stat(c.Path)
|
||||||
|
if err != nil {
|
||||||
|
return 0, 0
|
||||||
|
}
|
||||||
|
bytes = st.Size()
|
||||||
|
db, err := c.open()
|
||||||
|
if err != nil {
|
||||||
|
return 0, bytes
|
||||||
|
}
|
||||||
|
_ = db.View(func(tx *bolt.Tx) error { entries = count(tx.Bucket(bucketMeta)); return nil })
|
||||||
|
return entries, bytes
|
||||||
|
}
|
||||||
|
|
||||||
|
func count(m *bolt.Bucket) int64 {
|
||||||
|
if v := m.Get(keyCount); len(v) == 8 {
|
||||||
|
return int64(binary.BigEndian.Uint64(v))
|
||||||
|
}
|
||||||
|
return 0
|
||||||
|
}
|
||||||
|
|
||||||
|
func be64(n int64) []byte {
|
||||||
|
b := make([]byte, 8)
|
||||||
|
binary.BigEndian.PutUint64(b, uint64(n))
|
||||||
|
return b
|
||||||
|
}
|
||||||
|
|
||||||
|
// A value is a 4-byte header length, the header as JSON, then the JPEG bytes.
|
||||||
|
func encode(e Entry) ([]byte, error) {
|
||||||
|
hdr, err := json.Marshal(e)
|
||||||
|
if err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
v := make([]byte, 4+len(hdr)+len(e.JPEG))
|
||||||
|
binary.BigEndian.PutUint32(v, uint32(len(hdr)))
|
||||||
|
copy(v[4:], hdr)
|
||||||
|
copy(v[4+len(hdr):], e.JPEG)
|
||||||
|
return v, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
func decode(v []byte) (Entry, error) {
|
||||||
|
if len(v) < 4 {
|
||||||
|
return Entry{}, errors.New("short value")
|
||||||
|
}
|
||||||
|
n := int(binary.BigEndian.Uint32(v))
|
||||||
|
if 4+n > len(v) {
|
||||||
|
return Entry{}, errors.New("bad header length")
|
||||||
|
}
|
||||||
|
var e Entry
|
||||||
|
if err := json.Unmarshal(v[4:4+n], &e); err != nil {
|
||||||
|
return Entry{}, err
|
||||||
|
}
|
||||||
|
if rest := v[4+n:]; len(rest) > 0 {
|
||||||
|
e.JPEG = append([]byte(nil), rest...)
|
||||||
|
}
|
||||||
|
return e, nil
|
||||||
|
}
|
||||||
@@ -0,0 +1,222 @@
|
|||||||
|
package media
|
||||||
|
|
||||||
|
// What to run when a file is to be re-encoded for a browser that cannot play it as it is:
|
||||||
|
// the codecs and sizes on offer, the output size and bit rate a choice comes to for a
|
||||||
|
// given source, the ways this machine could do it in the order worth trying, and the
|
||||||
|
// ffmpeg argument list for one of them. All decisions, no processes, so it can be tested.
|
||||||
|
|
||||||
|
import (
|
||||||
|
"fmt"
|
||||||
|
"math"
|
||||||
|
"sort"
|
||||||
|
"strconv"
|
||||||
|
)
|
||||||
|
|
||||||
|
// Codec is one target a stream may be re-encoded to. "copy" keeps the codec and only
|
||||||
|
// repackages: for a browser that lacks the container, not the decoder.
|
||||||
|
type Codec struct {
|
||||||
|
ID string `json:"id"`
|
||||||
|
Label string `json:"label"`
|
||||||
|
}
|
||||||
|
|
||||||
|
var Codecs = []Codec{{"h264", "H.264"}, {"hevc", "HEVC"}, {"av1", "AV1"}, {"copy", "Original codec, repackaged"}}
|
||||||
|
|
||||||
|
// Quality is a picture height; the bit rate follows from the size the output ends up at.
|
||||||
|
type Quality struct {
|
||||||
|
ID string `json:"id"`
|
||||||
|
Label string `json:"label"`
|
||||||
|
Height int `json:"height"` // 0: the source's own
|
||||||
|
}
|
||||||
|
|
||||||
|
var Qualities = []Quality{{"original", "Original size", 0}, {"1080p", "1080p", 1080}, {"720p", "720p", 720}, {"480p", "480p", 480}}
|
||||||
|
|
||||||
|
func qualityByID(id string) (Quality, bool) {
|
||||||
|
for _, q := range Qualities {
|
||||||
|
if q.ID == id {
|
||||||
|
return q, true
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return Quality{}, false
|
||||||
|
}
|
||||||
|
|
||||||
|
// ValidChoice says whether a codec and quality are ones on offer — checked before any work.
|
||||||
|
func ValidChoice(codec, quality string) error {
|
||||||
|
if !codecKnown(codec) {
|
||||||
|
return fmt.Errorf("unknown codec %q", codec)
|
||||||
|
}
|
||||||
|
if _, ok := qualityByID(quality); !ok {
|
||||||
|
return fmt.Errorf("unknown quality %q", quality)
|
||||||
|
}
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
|
||||||
|
func codecKnown(id string) bool {
|
||||||
|
for _, c := range Codecs {
|
||||||
|
if c.ID == id {
|
||||||
|
return true
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return false
|
||||||
|
}
|
||||||
|
|
||||||
|
// fit is the output: never larger than the source, even dimensions, and a bit rate by the
|
||||||
|
// size it comes to — a quarter more at 50 fps and up, less for the codecs that need less.
|
||||||
|
func fit(q Quality, src Info, codec string) (w, h, kbps int) {
|
||||||
|
w, h = src.Video.Width, src.Video.Height
|
||||||
|
if q.Height > 0 && h > q.Height {
|
||||||
|
w = int(math.Round(float64(w) * float64(q.Height) / float64(h)))
|
||||||
|
h = q.Height
|
||||||
|
}
|
||||||
|
w, h = w&^1, h&^1
|
||||||
|
switch {
|
||||||
|
case h >= 2160:
|
||||||
|
kbps = 20000
|
||||||
|
case h >= 1440:
|
||||||
|
kbps = 12000
|
||||||
|
case h >= 1080:
|
||||||
|
kbps = 8000
|
||||||
|
case h >= 720:
|
||||||
|
kbps = 4000
|
||||||
|
default:
|
||||||
|
kbps = 2000
|
||||||
|
}
|
||||||
|
if src.Video.FPS >= 50 {
|
||||||
|
kbps = kbps * 5 / 4
|
||||||
|
}
|
||||||
|
if codec == "hevc" || codec == "av1" {
|
||||||
|
kbps = kbps * 6 / 10
|
||||||
|
}
|
||||||
|
return w, h, kbps
|
||||||
|
}
|
||||||
|
|
||||||
|
// Recipe is one way to make the stream: the engine's name for the record, and the ffmpeg
|
||||||
|
// arguments that go before the input, the video filter, and the video codec.
|
||||||
|
type Recipe struct {
|
||||||
|
Engine string
|
||||||
|
Pre []string
|
||||||
|
VF string
|
||||||
|
Video []string
|
||||||
|
}
|
||||||
|
|
||||||
|
// recipes are the ways to try, best first: a card on the bus before the integrated GPU;
|
||||||
|
// on a GPU, decoding on the same card (frames never leave it) before decoding in software
|
||||||
|
// and uploading; the software encoders last. An empty list means nothing here can.
|
||||||
|
func recipes(c Caps, codec string, src Info, w, h, kbps int) []Recipe {
|
||||||
|
if codec == "copy" {
|
||||||
|
v := []string{"-c:v", "copy"}
|
||||||
|
if src.Video.Codec == "hevc" {
|
||||||
|
v = append(v, "-tag:v", "hvc1") // Safari reads hvc1, not hev1
|
||||||
|
}
|
||||||
|
return []Recipe{{Engine: "Original codec", Video: v}}
|
||||||
|
}
|
||||||
|
var out []Recipe
|
||||||
|
gpus := append([]GPU(nil), c.GPUs...)
|
||||||
|
sort.SliceStable(gpus, func(i, j int) bool { return !gpus[i].Integrated && gpus[j].Integrated })
|
||||||
|
rate := []string{"-b:v", strconv.Itoa(kbps) + "k", "-maxrate", strconv.Itoa(kbps*5/4) + "k", "-bufsize", strconv.Itoa(kbps*5/2) + "k"}
|
||||||
|
for i := range gpus {
|
||||||
|
g := &gpus[i]
|
||||||
|
if g.API != "vaapi" || !has(g.Encodes, codec) {
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
video := append([]string{"-c:v", codec + "_vaapi"}, rate...)
|
||||||
|
switch codec {
|
||||||
|
case "h264":
|
||||||
|
video = append(video, "-profile:v", "high")
|
||||||
|
case "hevc":
|
||||||
|
video = append(video, "-tag:v", "hvc1")
|
||||||
|
}
|
||||||
|
name := g.Name
|
||||||
|
if name == "" {
|
||||||
|
name = g.Device
|
||||||
|
}
|
||||||
|
scale := fmt.Sprintf("scale_vaapi=w=%d:h=%d:format=nv12", w, h)
|
||||||
|
if has(g.Codecs, src.Video.Codec) {
|
||||||
|
vf := scale
|
||||||
|
if src.HDR() {
|
||||||
|
vf = "tonemap_vaapi=format=nv12:t=bt709:p=bt709:m=bt709," + vf
|
||||||
|
}
|
||||||
|
out = append(out, Recipe{Engine: name + " (vaapi)", Pre: []string{"-hwaccel", "vaapi", "-hwaccel_device", g.Device, "-hwaccel_output_format", "vaapi"}, VF: vf, Video: video})
|
||||||
|
}
|
||||||
|
out = append(out, Recipe{Engine: name + " (vaapi, decoded on the CPU)", Pre: []string{"-init_hw_device", "vaapi=va:" + g.Device, "-filter_hw_device", "va"},
|
||||||
|
VF: fmt.Sprintf("format=nv12,scale=%d:%d,hwupload", w, h), Video: video})
|
||||||
|
}
|
||||||
|
swVF := fmt.Sprintf("scale=%d:%d,format=yuv420p", w, h)
|
||||||
|
if src.HDR() {
|
||||||
|
swVF = fmt.Sprintf("zscale=t=linear:npl=100,format=gbrpf32le,zscale=p=bt709,tonemap=hable,zscale=t=bt709:m=bt709,scale=%d:%d,format=yuv420p", w, h)
|
||||||
|
}
|
||||||
|
cap := []string{"-maxrate", strconv.Itoa(kbps*5/4) + "k", "-bufsize", strconv.Itoa(kbps*5/2) + "k"}
|
||||||
|
switch {
|
||||||
|
case codec == "h264" && has(c.Encoders, "libx264"):
|
||||||
|
out = append(out, Recipe{Engine: "CPU (libx264)", VF: swVF, Video: append([]string{"-c:v", "libx264", "-preset", "veryfast", "-crf", "23", "-profile:v", "high"}, cap...)})
|
||||||
|
case codec == "hevc" && has(c.Encoders, "libx265"):
|
||||||
|
out = append(out, Recipe{Engine: "CPU (libx265)", VF: swVF, Video: append([]string{"-c:v", "libx265", "-preset", "fast", "-crf", "26", "-tag:v", "hvc1"}, cap...)})
|
||||||
|
case codec == "av1" && has(c.Encoders, "libsvtav1"):
|
||||||
|
out = append(out, Recipe{Engine: "CPU (SVT-AV1)", VF: swVF, Video: append([]string{"-c:v", "libsvtav1", "-preset", "8", "-crf", "35"}, cap...)})
|
||||||
|
}
|
||||||
|
return out
|
||||||
|
}
|
||||||
|
|
||||||
|
// EngineFor is the engine the picker will say a codec runs on, "" when none can.
|
||||||
|
func EngineFor(c Caps, codec string) string {
|
||||||
|
if c.Error != "" {
|
||||||
|
return ""
|
||||||
|
}
|
||||||
|
if codec == "copy" {
|
||||||
|
return "Original codec"
|
||||||
|
}
|
||||||
|
r := recipes(c, codec, Info{Video: &Track{Codec: "h264", Width: 1920, Height: 1080}}, 1920, 1080, 8000)
|
||||||
|
if len(r) == 0 {
|
||||||
|
return ""
|
||||||
|
}
|
||||||
|
return r[0].Engine
|
||||||
|
}
|
||||||
|
|
||||||
|
// hlsArgs is the whole ffmpeg argument list for one recipe: the input, the first video and
|
||||||
|
// audio streams, the picture through the recipe, sound as AAC unless it already is, a
|
||||||
|
// keyframe every segment, and an event playlist of fragmented-MP4 segments in dir.
|
||||||
|
func hlsArgs(dir, in string, src Info, r Recipe, copyCodec bool) []string {
|
||||||
|
a := []string{"-nostdin", "-hide_banner", "-loglevel", "error", "-y"}
|
||||||
|
a = append(a, r.Pre...)
|
||||||
|
a = append(a, "-i", in, "-map", "0:v:0", "-map", "0:a:0?", "-sn", "-dn")
|
||||||
|
if r.VF != "" {
|
||||||
|
a = append(a, "-vf", r.VF)
|
||||||
|
}
|
||||||
|
a = append(a, r.Video...)
|
||||||
|
switch {
|
||||||
|
case src.Audio == nil:
|
||||||
|
a = append(a, "-an")
|
||||||
|
case src.Audio.Codec == "aac":
|
||||||
|
a = append(a, "-c:a", "copy")
|
||||||
|
default:
|
||||||
|
a = append(a, "-c:a", "aac", "-b:a", "160k", "-ac", "2")
|
||||||
|
}
|
||||||
|
if !copyCodec {
|
||||||
|
fps := src.Video.FPS
|
||||||
|
if fps <= 0 {
|
||||||
|
fps = 30
|
||||||
|
}
|
||||||
|
gop := int(math.Round(fps * hlsSeconds))
|
||||||
|
if gop < 24 {
|
||||||
|
gop = 24
|
||||||
|
}
|
||||||
|
if gop > 300 {
|
||||||
|
gop = 300
|
||||||
|
}
|
||||||
|
a = append(a, "-g", strconv.Itoa(gop), "-force_key_frames", fmt.Sprintf("expr:gte(t,n_forced*%d)", hlsSeconds))
|
||||||
|
}
|
||||||
|
a = append(a, "-f", "hls", "-hls_time", strconv.Itoa(hlsSeconds), "-hls_playlist_type", "event", "-hls_segment_type", "fmp4",
|
||||||
|
"-hls_fmp4_init_filename", "init.mp4", "-hls_segment_filename", dir+"/seg%05d.m4s", "-hls_flags", "independent_segments+temp_file",
|
||||||
|
"-progress", dir+"/progress", "-stats_period", "1", dir+"/index.m3u8")
|
||||||
|
return a
|
||||||
|
}
|
||||||
|
|
||||||
|
const hlsSeconds = 4
|
||||||
|
|
||||||
|
func has(list []string, x string) bool {
|
||||||
|
for _, v := range list {
|
||||||
|
if v == x {
|
||||||
|
return true
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return false
|
||||||
|
}
|
||||||
@@ -0,0 +1,128 @@
|
|||||||
|
package media
|
||||||
|
|
||||||
|
import (
|
||||||
|
"strings"
|
||||||
|
"testing"
|
||||||
|
)
|
||||||
|
|
||||||
|
func src(codec string, w, h int, fps float64) Info {
|
||||||
|
return Info{Duration: 100, Video: &Track{Codec: codec, Width: w, Height: h, FPS: fps}, Audio: &Track{Codec: "opus", Channels: 2}}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestFit(t *testing.T) {
|
||||||
|
q := func(id string) Quality { x, _ := qualityByID(id); return x }
|
||||||
|
for _, c := range []struct {
|
||||||
|
q string
|
||||||
|
in Info
|
||||||
|
codec string
|
||||||
|
w, h, kbs int
|
||||||
|
}{
|
||||||
|
{"1080p", src("hevc", 3840, 2160, 60), "h264", 1920, 1080, 10000},
|
||||||
|
{"original", src("hevc", 3840, 2160, 30), "h264", 3840, 2160, 20000},
|
||||||
|
{"720p", src("h264", 2292, 960, 60), "h264", 1718, 720, 5000}, // odd width made even
|
||||||
|
{"1080p", src("h264", 1280, 720, 30), "h264", 1280, 720, 4000}, // never upscaled
|
||||||
|
{"480p", src("h264", 1920, 1080, 30), "hevc", 852, 480, 1200}, // 853.3 rounds down, then even
|
||||||
|
{"original", src("av1", 640, 360, 24), "av1", 640, 360, 1200},
|
||||||
|
} {
|
||||||
|
w, h, k := fit(q(c.q), c.in, c.codec)
|
||||||
|
if w != c.w || h != c.h || k != c.kbs {
|
||||||
|
t.Errorf("%s %s %dx%d@%.0f → %dx%d %dk, want %dx%d %dk", c.q, c.codec, c.in.Video.Width, c.in.Video.Height, c.in.Video.FPS, w, h, k, c.w, c.h, c.kbs)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func gridCaps() Caps {
|
||||||
|
return Caps{FFmpeg: "ffmpeg", FFprobe: "ffprobe", Encoders: []string{"aac", "libsvtav1", "libx264", "libx265"},
|
||||||
|
GPUs: []GPU{
|
||||||
|
{API: "vaapi", Device: "/dev/dri/renderD128", Name: "Intel UHD Graphics 750", Integrated: true, Codecs: []string{"h264", "hevc", "vp9"}, Encodes: []string{"h264", "hevc"}},
|
||||||
|
{API: "vaapi", Device: "/dev/dri/renderD129", Name: "Intel Arc A380", Codecs: []string{"h264", "hevc", "vp9"}, Encodes: []string{"av1", "h264", "hevc"}},
|
||||||
|
}}
|
||||||
|
}
|
||||||
|
|
||||||
|
func engineNames(r []Recipe) []string {
|
||||||
|
out := make([]string, len(r))
|
||||||
|
for i := range r {
|
||||||
|
out[i] = r[i].Engine
|
||||||
|
}
|
||||||
|
return out
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestRecipesOrder(t *testing.T) {
|
||||||
|
c := gridCaps()
|
||||||
|
// h264 from hevc: the card first, decoding on it; then the iGPU the same way; then the CPU
|
||||||
|
got := engineNames(recipes(c, "h264", src("hevc", 3840, 2160, 60), 1920, 1080, 10000))
|
||||||
|
want := []string{"Intel Arc A380 (vaapi)", "Intel Arc A380 (vaapi, decoded on the CPU)", "Intel UHD Graphics 750 (vaapi)", "Intel UHD Graphics 750 (vaapi, decoded on the CPU)", "CPU (libx264)"}
|
||||||
|
if strings.Join(got, "|") != strings.Join(want, "|") {
|
||||||
|
t.Fatalf("got %v", got)
|
||||||
|
}
|
||||||
|
// av1 source: neither card decodes it, so no same-card recipe, and only the Arc encodes av1
|
||||||
|
got = engineNames(recipes(c, "av1", src("av1", 1920, 1080, 30), 1920, 1080, 4800))
|
||||||
|
want = []string{"Intel Arc A380 (vaapi, decoded on the CPU)", "CPU (SVT-AV1)"}
|
||||||
|
if strings.Join(got, "|") != strings.Join(want, "|") {
|
||||||
|
t.Fatalf("av1: got %v", got)
|
||||||
|
}
|
||||||
|
// copy is one recipe, no filter, hvc1 for hevc
|
||||||
|
r := recipes(c, "copy", src("hevc", 1920, 1080, 30), 1920, 1080, 0)
|
||||||
|
if len(r) != 1 || r[0].VF != "" || !has(r[0].Video, "hvc1") || !has(r[0].Video, "copy") {
|
||||||
|
t.Fatalf("copy: %+v", r)
|
||||||
|
}
|
||||||
|
// a machine with no GPU and no x264: nothing for h264, something for av1
|
||||||
|
bare := Caps{FFmpeg: "ffmpeg", FFprobe: "ffprobe", Encoders: []string{"aac", "libsvtav1"}}
|
||||||
|
if len(recipes(bare, "h264", src("h264", 1280, 720, 30), 1280, 720, 4000)) != 0 {
|
||||||
|
t.Fatal("no x264 means no h264")
|
||||||
|
}
|
||||||
|
if EngineFor(bare, "av1") != "CPU (SVT-AV1)" || EngineFor(bare, "h264") != "" || EngineFor(bare, "copy") != "Original codec" {
|
||||||
|
t.Fatal("engineFor")
|
||||||
|
}
|
||||||
|
if EngineFor(c, "hevc") != "Intel Arc A380 (vaapi)" {
|
||||||
|
t.Fatalf("engineFor on grid: %q", EngineFor(c, "hevc"))
|
||||||
|
}
|
||||||
|
// HDR gets tone-mapped on both paths
|
||||||
|
hdr := src("hevc", 3840, 2160, 60)
|
||||||
|
hdr.Video.ColorTransfer = "smpte2084"
|
||||||
|
r = recipes(c, "h264", hdr, 1920, 1080, 10000)
|
||||||
|
if !strings.HasPrefix(r[0].VF, "tonemap_vaapi") || !strings.Contains(r[len(r)-1].VF, "tonemap=hable") {
|
||||||
|
t.Fatalf("hdr: %q … %q", r[0].VF, r[len(r)-1].VF)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestHLSArgs(t *testing.T) {
|
||||||
|
in := src("h264", 1920, 1080, 60)
|
||||||
|
r := recipes(gridCaps(), "h264", in, 1920, 1080, 10000)[0]
|
||||||
|
a := strings.Join(hlsArgs("/d", "/v.mkv", in, r, false), " ")
|
||||||
|
for _, want := range []string{"-hwaccel vaapi -hwaccel_device /dev/dri/renderD129", "-i /v.mkv", "-map 0:v:0 -map 0:a:0?", "-vf scale_vaapi=w=1920:h=1080:format=nv12", "-c:v h264_vaapi -b:v 10000k -maxrate 12500k", "-c:a aac -b:a 160k -ac 2", "-g 240 -force_key_frames expr:gte(t,n_forced*4)", "-hls_segment_type fmp4", "-hls_segment_filename /d/seg%05d.m4s", "/d/index.m3u8"} {
|
||||||
|
if !strings.Contains(a, want) {
|
||||||
|
t.Errorf("missing %q in\n%s", want, a)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
// aac stays as it is; no audio means none; copy sets no keyframes
|
||||||
|
in.Audio.Codec = "aac"
|
||||||
|
if a := strings.Join(hlsArgs("/d", "/v", in, r, false), " "); !strings.Contains(a, "-c:a copy") {
|
||||||
|
t.Error("aac should be copied")
|
||||||
|
}
|
||||||
|
in.Audio = nil
|
||||||
|
if a := strings.Join(hlsArgs("/d", "/v", in, r, false), " "); !strings.Contains(a, " -an ") {
|
||||||
|
t.Error("no audio stream → -an")
|
||||||
|
}
|
||||||
|
cp := recipes(gridCaps(), "copy", in, 1920, 1080, 0)[0]
|
||||||
|
if a := strings.Join(hlsArgs("/d", "/v", in, cp, true), " "); strings.Contains(a, "force_key_frames") || strings.Contains(a, "-vf") {
|
||||||
|
t.Error("copy must not ask for keyframes or filter")
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestParseProgress(t *testing.T) {
|
||||||
|
b := []byte("frame=10\nout_time_us=1000000\nspeed=2.5x\nprogress=continue\nframe=20\nout_time_us=2500000\nspeed=3.1x\nprogress=continue\n")
|
||||||
|
s, sp := parseProgress(b)
|
||||||
|
if s != 2.5 || sp != 3.1 {
|
||||||
|
t.Fatalf("got %v %v", s, sp)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestValidChoice(t *testing.T) {
|
||||||
|
if err := ValidChoice("h264", "720p"); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if ValidChoice("vp8", "720p") == nil || ValidChoice("h264", "9p") == nil || ValidChoice("", "") == nil {
|
||||||
|
t.Fatal("bad choices must be refused")
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -0,0 +1,188 @@
|
|||||||
|
package media
|
||||||
|
|
||||||
|
// Running one long ffmpeg: as the daemon's user directly, or as root through this binary's
|
||||||
|
// run-ffmpeg helper when the file needs root to read. The helper is told to finish by its
|
||||||
|
// standard input closing, which also happens by itself if the daemon goes away — so a
|
||||||
|
// transcode never outlives the console that asked for it.
|
||||||
|
|
||||||
|
import (
|
||||||
|
"bytes"
|
||||||
|
"context"
|
||||||
|
"errors"
|
||||||
|
"io"
|
||||||
|
"os"
|
||||||
|
"os/exec"
|
||||||
|
"strings"
|
||||||
|
"sync"
|
||||||
|
"syscall"
|
||||||
|
"time"
|
||||||
|
|
||||||
|
"hamlab/internal/priv"
|
||||||
|
)
|
||||||
|
|
||||||
|
type ffproc struct {
|
||||||
|
cmd *exec.Cmd
|
||||||
|
in io.WriteCloser // the helper's stdin, root mode only
|
||||||
|
done chan struct{}
|
||||||
|
err error
|
||||||
|
tail *tailBuf
|
||||||
|
}
|
||||||
|
|
||||||
|
// startFF launches ffmpeg with args, under nice where there is one.
|
||||||
|
func startFF(c Caps, self string, root bool, args []string) (*ffproc, error) {
|
||||||
|
var argv []string
|
||||||
|
if root {
|
||||||
|
argv = priv.Wrap(append([]string{self, "run-ffmpeg"}, args...)...)
|
||||||
|
} else {
|
||||||
|
argv = append([]string{c.FFmpeg}, args...)
|
||||||
|
if c.Nice != "" {
|
||||||
|
argv = append([]string{c.Nice, "-n", "10"}, argv...)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
cmd := exec.Command(argv[0], argv[1:]...)
|
||||||
|
cmd.SysProcAttr = &syscall.SysProcAttr{Setpgid: true}
|
||||||
|
p := &ffproc{cmd: cmd, done: make(chan struct{}), tail: &tailBuf{}}
|
||||||
|
cmd.Stderr = p.tail
|
||||||
|
cmd.Stdout = io.Discard
|
||||||
|
if root {
|
||||||
|
in, err := cmd.StdinPipe()
|
||||||
|
if err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
p.in = in
|
||||||
|
}
|
||||||
|
if err := cmd.Start(); err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
go func() {
|
||||||
|
p.err = cmd.Wait()
|
||||||
|
close(p.done)
|
||||||
|
}()
|
||||||
|
return p, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
// exited reports whether ffmpeg has finished, and how.
|
||||||
|
func (p *ffproc) exited() (bool, error) {
|
||||||
|
select {
|
||||||
|
case <-p.done:
|
||||||
|
return true, p.err
|
||||||
|
default:
|
||||||
|
return false, nil
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// end asks ffmpeg to finish and waits a little for it.
|
||||||
|
func (p *ffproc) end() {
|
||||||
|
if done, _ := p.exited(); done {
|
||||||
|
return
|
||||||
|
}
|
||||||
|
if p.in != nil {
|
||||||
|
p.in.Close()
|
||||||
|
} else {
|
||||||
|
_ = syscall.Kill(-p.cmd.Process.Pid, syscall.SIGTERM)
|
||||||
|
}
|
||||||
|
select {
|
||||||
|
case <-p.done:
|
||||||
|
case <-time.After(5 * time.Second):
|
||||||
|
_ = syscall.Kill(-p.cmd.Process.Pid, syscall.SIGKILL) // a root helper ignores this; it ends on its own within its own grace
|
||||||
|
select {
|
||||||
|
case <-p.done:
|
||||||
|
case <-time.After(10 * time.Second):
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// lastWords is what ffmpeg said last, for the record.
|
||||||
|
func (p *ffproc) lastWords() string { return p.tail.last() }
|
||||||
|
|
||||||
|
// tailBuf keeps the last few KB written to it.
|
||||||
|
type tailBuf struct {
|
||||||
|
mu sync.Mutex
|
||||||
|
b []byte
|
||||||
|
}
|
||||||
|
|
||||||
|
const tailKeep = 4096
|
||||||
|
|
||||||
|
func (t *tailBuf) Write(b []byte) (int, error) {
|
||||||
|
t.mu.Lock()
|
||||||
|
defer t.mu.Unlock()
|
||||||
|
t.b = append(t.b, b...)
|
||||||
|
if len(t.b) > tailKeep {
|
||||||
|
t.b = t.b[len(t.b)-tailKeep:]
|
||||||
|
}
|
||||||
|
return len(b), nil
|
||||||
|
}
|
||||||
|
|
||||||
|
func (t *tailBuf) last() string {
|
||||||
|
t.mu.Lock()
|
||||||
|
defer t.mu.Unlock()
|
||||||
|
return lastLine(string(bytes.TrimSpace(t.b)))
|
||||||
|
}
|
||||||
|
|
||||||
|
// RunFFmpegHelper implements `hamlab run-ffmpeg <args…>`: ffmpeg as this user (root, when
|
||||||
|
// the daemon asked through sudo), finishing when stdin closes. Returns ffmpeg's exit code.
|
||||||
|
func RunFFmpegHelper(args []string, stdin io.Reader, stderr io.Writer) int {
|
||||||
|
ffmpeg, err := exec.LookPath("ffmpeg")
|
||||||
|
if err != nil {
|
||||||
|
io.WriteString(stderr, "ffmpeg is not installed\n")
|
||||||
|
return 1
|
||||||
|
}
|
||||||
|
argv := []string{ffmpeg}
|
||||||
|
if nice, err := exec.LookPath("nice"); err == nil {
|
||||||
|
argv = []string{nice, "-n", "10", ffmpeg}
|
||||||
|
}
|
||||||
|
cmd := exec.Command(argv[0], append(argv[1:], args...)...)
|
||||||
|
cmd.SysProcAttr = &syscall.SysProcAttr{Setpgid: true}
|
||||||
|
cmd.Stderr = stderr
|
||||||
|
cmd.Stdout = io.Discard
|
||||||
|
if err := cmd.Start(); err != nil {
|
||||||
|
io.WriteString(stderr, err.Error()+"\n")
|
||||||
|
return 1
|
||||||
|
}
|
||||||
|
ctx, cancel := context.WithCancel(context.Background())
|
||||||
|
go func() {
|
||||||
|
_, _ = io.Copy(io.Discard, stdin) // returns when the daemon lets go
|
||||||
|
cancel()
|
||||||
|
}()
|
||||||
|
done := make(chan error, 1)
|
||||||
|
go func() { done <- cmd.Wait() }()
|
||||||
|
var werr error
|
||||||
|
select {
|
||||||
|
case werr = <-done:
|
||||||
|
case <-ctx.Done():
|
||||||
|
_ = syscall.Kill(-cmd.Process.Pid, syscall.SIGTERM)
|
||||||
|
select {
|
||||||
|
case werr = <-done:
|
||||||
|
case <-time.After(5 * time.Second):
|
||||||
|
_ = syscall.Kill(-cmd.Process.Pid, syscall.SIGKILL)
|
||||||
|
werr = <-done
|
||||||
|
}
|
||||||
|
}
|
||||||
|
var ee *exec.ExitError
|
||||||
|
if errors.As(werr, &ee) {
|
||||||
|
return ee.ExitCode()
|
||||||
|
}
|
||||||
|
if werr != nil {
|
||||||
|
return 1
|
||||||
|
}
|
||||||
|
return 0
|
||||||
|
}
|
||||||
|
|
||||||
|
// clearDir empties a folder without removing it.
|
||||||
|
func clearDir(dir string) error {
|
||||||
|
ents, err := os.ReadDir(dir)
|
||||||
|
if err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
for _, e := range ents {
|
||||||
|
if err := os.RemoveAll(dir + "/" + e.Name()); err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
|
||||||
|
func hasSegments(playlist string) bool {
|
||||||
|
b, err := os.ReadFile(playlist)
|
||||||
|
return err == nil && strings.Contains(string(b), "#EXTINF")
|
||||||
|
}
|
||||||
@@ -0,0 +1,344 @@
|
|||||||
|
package media
|
||||||
|
|
||||||
|
// Playing what the browser cannot play as it is: an HLS stream, re-encoded on demand by
|
||||||
|
// ffmpeg on the machine's own GPU where it has one it can use, on its CPU otherwise. A
|
||||||
|
// stream is one file at one codec and size, in a folder of its own under
|
||||||
|
// <data dir>/transcode/: ffmpeg writes an event playlist and fragmented-MP4 segments there
|
||||||
|
// as fast as it can, and the player reads them as they land, so the range it can seek in
|
||||||
|
// grows until the whole file is through — on a GPU, several times faster than watching.
|
||||||
|
// Nothing is kept: a stream nobody has touched for a while is ended and its folder
|
||||||
|
// removed, and the folder is emptied when the daemon starts.
|
||||||
|
|
||||||
|
import (
|
||||||
|
"bufio"
|
||||||
|
"context"
|
||||||
|
"crypto/rand"
|
||||||
|
"encoding/hex"
|
||||||
|
"errors"
|
||||||
|
"fmt"
|
||||||
|
"log"
|
||||||
|
"os"
|
||||||
|
"path/filepath"
|
||||||
|
"regexp"
|
||||||
|
"strconv"
|
||||||
|
"strings"
|
||||||
|
"sync"
|
||||||
|
"time"
|
||||||
|
)
|
||||||
|
|
||||||
|
// Stream is one re-encode, in progress or finished.
|
||||||
|
type Stream struct {
|
||||||
|
ID string `json:"id"`
|
||||||
|
Path string `json:"path"`
|
||||||
|
Codec string `json:"codec"`
|
||||||
|
Quality string `json:"quality"`
|
||||||
|
Engine string `json:"engine"`
|
||||||
|
Width int `json:"width"`
|
||||||
|
Height int `json:"height"`
|
||||||
|
Kbps int `json:"kbps"`
|
||||||
|
Source Info `json:"source"`
|
||||||
|
Started time.Time `json:"started"`
|
||||||
|
Root bool `json:"root"`
|
||||||
|
|
||||||
|
key string
|
||||||
|
dir string
|
||||||
|
mu sync.Mutex
|
||||||
|
used time.Time
|
||||||
|
proc *ffproc
|
||||||
|
finished bool
|
||||||
|
failure string
|
||||||
|
gone bool // forgotten by the Streamer
|
||||||
|
}
|
||||||
|
|
||||||
|
// StreamStatus is what the player asks for while the file is still going through.
|
||||||
|
type StreamStatus struct {
|
||||||
|
Running bool `json:"running"`
|
||||||
|
Done bool `json:"done"`
|
||||||
|
Error string `json:"error,omitempty"`
|
||||||
|
Progress float64 `json:"progress"` // seconds of output written
|
||||||
|
Speed float64 `json:"speed"` // × realtime
|
||||||
|
Segments int `json:"segments"`
|
||||||
|
}
|
||||||
|
|
||||||
|
// Streamer keeps one machine's streams.
|
||||||
|
type Streamer struct {
|
||||||
|
dir string
|
||||||
|
self string
|
||||||
|
limit int
|
||||||
|
mu sync.Mutex
|
||||||
|
streams map[string]*Stream
|
||||||
|
}
|
||||||
|
|
||||||
|
const streamIdle = 10 * time.Minute
|
||||||
|
const firstSegmentWait = 40 * time.Second
|
||||||
|
|
||||||
|
var streamFile = regexp.MustCompile(`^(index\.m3u8|init\.mp4|seg\d{5}\.m4s)$`)
|
||||||
|
|
||||||
|
// NewStreamer empties the folder — a daemon restart orphans every stream — and starts the
|
||||||
|
// sweeper.
|
||||||
|
func NewStreamer(dataDir, self string, limit int) *Streamer {
|
||||||
|
m := &Streamer{dir: filepath.Join(dataDir, "transcode"), self: self, limit: limit, streams: map[string]*Stream{}}
|
||||||
|
_ = os.MkdirAll(m.dir, 0o755)
|
||||||
|
_ = clearDir(m.dir)
|
||||||
|
go func() {
|
||||||
|
for range time.Tick(30 * time.Second) {
|
||||||
|
m.Sweep(streamIdle)
|
||||||
|
}
|
||||||
|
}()
|
||||||
|
return m
|
||||||
|
}
|
||||||
|
|
||||||
|
// Open starts a stream, or hands back the one already serving the same request. It
|
||||||
|
// returns once the first segment is on disk, so the player's first fetch is answered.
|
||||||
|
func (m *Streamer) Open(ctx context.Context, c Caps, root bool, path, codec, quality string, src Info) (*Stream, error) {
|
||||||
|
if c.Error != "" {
|
||||||
|
return nil, errors.New(c.Error)
|
||||||
|
}
|
||||||
|
if err := ValidChoice(codec, quality); err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
q, _ := qualityByID(quality)
|
||||||
|
if src.Video == nil {
|
||||||
|
return nil, &DecodeError{Msg: "no video stream"}
|
||||||
|
}
|
||||||
|
if src.Video.Width <= 0 || src.Video.Height <= 0 {
|
||||||
|
return nil, &DecodeError{Msg: "the picture's size is unknown"}
|
||||||
|
}
|
||||||
|
key := fmt.Sprintf("%s|%d|%.0f|%s|%s|%v", path, src.Size, src.Duration, codec, quality, root)
|
||||||
|
m.mu.Lock()
|
||||||
|
for _, s := range m.streams {
|
||||||
|
if s.key == key {
|
||||||
|
if s.failed() { // not the stream to hand back: let it go and make a fresh one
|
||||||
|
delete(m.streams, s.ID)
|
||||||
|
_ = os.RemoveAll(s.dir)
|
||||||
|
break
|
||||||
|
}
|
||||||
|
m.mu.Unlock()
|
||||||
|
s.touch()
|
||||||
|
return s, nil
|
||||||
|
}
|
||||||
|
}
|
||||||
|
running := 0
|
||||||
|
for _, s := range m.streams {
|
||||||
|
if s.running() {
|
||||||
|
running++
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if running >= m.limit {
|
||||||
|
m.mu.Unlock()
|
||||||
|
return nil, fmt.Errorf("%d videos are already being re-encoded on this machine; one must finish or be closed first", running)
|
||||||
|
}
|
||||||
|
s := &Stream{ID: newID(), Path: path, Codec: codec, Quality: quality, Source: src, Started: time.Now(), Root: root, key: key, used: time.Now()}
|
||||||
|
s.dir = filepath.Join(m.dir, s.ID)
|
||||||
|
m.streams[s.ID] = s
|
||||||
|
m.mu.Unlock()
|
||||||
|
|
||||||
|
s.Width, s.Height, s.Kbps = fit(q, src, codec)
|
||||||
|
if codec == "copy" {
|
||||||
|
s.Width, s.Height, s.Kbps = src.Video.Width, src.Video.Height, int(src.BitRate/1000)
|
||||||
|
}
|
||||||
|
rs := recipes(c, codec, src, s.Width, s.Height, s.Kbps)
|
||||||
|
if len(rs) == 0 {
|
||||||
|
m.forget(s)
|
||||||
|
return nil, errors.New("nothing on this machine encodes " + codec)
|
||||||
|
}
|
||||||
|
var last string
|
||||||
|
for _, r := range rs {
|
||||||
|
if err := os.MkdirAll(s.dir, 0o755); err != nil { // the root helper's ffmpeg writes here too
|
||||||
|
m.forget(s)
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
_ = clearDir(s.dir)
|
||||||
|
args := hlsArgs(s.dir, path, src, r, codec == "copy")
|
||||||
|
p, err := startFF(c, m.self, root, args)
|
||||||
|
if err != nil {
|
||||||
|
last = r.Engine + ": " + err.Error()
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
s.mu.Lock()
|
||||||
|
s.proc, s.Engine = p, r.Engine
|
||||||
|
s.mu.Unlock()
|
||||||
|
msg, ok := s.waitFirstSegment(ctx)
|
||||||
|
if ok {
|
||||||
|
go s.watch()
|
||||||
|
log.Printf("media: stream %s: %s → %s %s %dx%d %dk on %s", s.ID, filepath.Base(path), codec, quality, s.Width, s.Height, s.Kbps, r.Engine)
|
||||||
|
return s, nil
|
||||||
|
}
|
||||||
|
p.end()
|
||||||
|
if ctx.Err() != nil || s.closed() { // the caller left, or somebody closed it meanwhile: not the recipe's fault
|
||||||
|
m.forget(s)
|
||||||
|
return nil, errors.New("cancelled")
|
||||||
|
}
|
||||||
|
last = r.Engine + ": " + msg
|
||||||
|
log.Printf("media: stream %s on %s: %s", filepath.Base(path), r.Engine, msg)
|
||||||
|
}
|
||||||
|
m.forget(s)
|
||||||
|
return nil, &DecodeError{Msg: last}
|
||||||
|
}
|
||||||
|
|
||||||
|
// waitFirstSegment watches for the playlist's first segment, or ffmpeg giving up.
|
||||||
|
func (s *Stream) waitFirstSegment(ctx context.Context) (string, bool) {
|
||||||
|
deadline := time.Now().Add(firstSegmentWait)
|
||||||
|
playlist := filepath.Join(s.dir, "index.m3u8")
|
||||||
|
for {
|
||||||
|
if hasSegments(playlist) {
|
||||||
|
return "", true
|
||||||
|
}
|
||||||
|
if done, err := s.proc.exited(); done {
|
||||||
|
msg := s.proc.lastWords()
|
||||||
|
if msg == "" && err != nil {
|
||||||
|
msg = err.Error()
|
||||||
|
}
|
||||||
|
if msg == "" {
|
||||||
|
msg = "ffmpeg ended before writing a segment"
|
||||||
|
}
|
||||||
|
return msg, false
|
||||||
|
}
|
||||||
|
if time.Now().After(deadline) {
|
||||||
|
return "no segment after " + firstSegmentWait.String(), false
|
||||||
|
}
|
||||||
|
select {
|
||||||
|
case <-ctx.Done():
|
||||||
|
return ctx.Err().Error(), false
|
||||||
|
case <-time.After(250 * time.Millisecond):
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// watch records how the encode ended.
|
||||||
|
func (s *Stream) watch() {
|
||||||
|
<-s.proc.done
|
||||||
|
s.mu.Lock()
|
||||||
|
s.finished = true
|
||||||
|
if s.proc.err != nil {
|
||||||
|
s.failure = s.proc.lastWords()
|
||||||
|
if s.failure == "" {
|
||||||
|
s.failure = s.proc.err.Error()
|
||||||
|
}
|
||||||
|
}
|
||||||
|
s.mu.Unlock()
|
||||||
|
if s.failure != "" {
|
||||||
|
log.Printf("media: stream %s ended early: %s", s.ID, s.failure)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func (m *Streamer) Find(id string) (*Stream, bool) {
|
||||||
|
m.mu.Lock()
|
||||||
|
defer m.mu.Unlock()
|
||||||
|
s, ok := m.streams[id]
|
||||||
|
return s, ok
|
||||||
|
}
|
||||||
|
|
||||||
|
// Close ends a stream and removes its folder.
|
||||||
|
func (m *Streamer) Close(id string) {
|
||||||
|
m.mu.Lock()
|
||||||
|
s, ok := m.streams[id]
|
||||||
|
m.mu.Unlock()
|
||||||
|
if !ok {
|
||||||
|
return
|
||||||
|
}
|
||||||
|
s.mu.Lock()
|
||||||
|
p := s.proc
|
||||||
|
s.mu.Unlock()
|
||||||
|
if p != nil {
|
||||||
|
p.end()
|
||||||
|
}
|
||||||
|
m.forget(s)
|
||||||
|
}
|
||||||
|
|
||||||
|
func (m *Streamer) forget(s *Stream) {
|
||||||
|
m.mu.Lock()
|
||||||
|
delete(m.streams, s.ID)
|
||||||
|
m.mu.Unlock()
|
||||||
|
s.mu.Lock()
|
||||||
|
s.gone = true
|
||||||
|
s.mu.Unlock()
|
||||||
|
_ = os.RemoveAll(s.dir)
|
||||||
|
}
|
||||||
|
|
||||||
|
func (s *Stream) closed() bool { s.mu.Lock(); defer s.mu.Unlock(); return s.gone }
|
||||||
|
|
||||||
|
func (s *Stream) failed() bool {
|
||||||
|
s.mu.Lock()
|
||||||
|
defer s.mu.Unlock()
|
||||||
|
return s.finished && s.failure != ""
|
||||||
|
}
|
||||||
|
|
||||||
|
// Sweep closes what nobody has touched for a while.
|
||||||
|
func (m *Streamer) Sweep(idle time.Duration) {
|
||||||
|
m.mu.Lock()
|
||||||
|
var old []string
|
||||||
|
for id, s := range m.streams {
|
||||||
|
s.mu.Lock()
|
||||||
|
if time.Since(s.used) > idle {
|
||||||
|
old = append(old, id)
|
||||||
|
}
|
||||||
|
s.mu.Unlock()
|
||||||
|
}
|
||||||
|
m.mu.Unlock()
|
||||||
|
for _, id := range old {
|
||||||
|
log.Printf("media: stream %s idle, closed", id)
|
||||||
|
m.Close(id)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// File is the on-disk path of one of the stream's files — the playlist, the init segment
|
||||||
|
// or a media segment — and nothing else.
|
||||||
|
func (s *Stream) File(name string) (string, bool) {
|
||||||
|
if !streamFile.MatchString(name) {
|
||||||
|
return "", false
|
||||||
|
}
|
||||||
|
s.touch()
|
||||||
|
return filepath.Join(s.dir, name), true
|
||||||
|
}
|
||||||
|
|
||||||
|
func (s *Stream) touch() { s.mu.Lock(); s.used = time.Now(); s.mu.Unlock() }
|
||||||
|
|
||||||
|
func (s *Stream) running() bool {
|
||||||
|
s.mu.Lock()
|
||||||
|
defer s.mu.Unlock()
|
||||||
|
return s.proc != nil && !s.finished
|
||||||
|
}
|
||||||
|
|
||||||
|
// Status is read from ffmpeg's own progress file and the playlist.
|
||||||
|
func (s *Stream) Status() StreamStatus {
|
||||||
|
s.mu.Lock()
|
||||||
|
st := StreamStatus{Running: s.proc != nil && !s.finished, Done: s.finished && s.failure == "", Error: s.failure}
|
||||||
|
s.mu.Unlock()
|
||||||
|
if b, err := os.ReadFile(filepath.Join(s.dir, "progress")); err == nil {
|
||||||
|
st.Progress, st.Speed = parseProgress(b)
|
||||||
|
}
|
||||||
|
if b, err := os.ReadFile(filepath.Join(s.dir, "index.m3u8")); err == nil {
|
||||||
|
st.Segments = strings.Count(string(b), "#EXTINF")
|
||||||
|
}
|
||||||
|
return st
|
||||||
|
}
|
||||||
|
|
||||||
|
// parseProgress takes the last out_time_us and speed from ffmpeg's -progress output.
|
||||||
|
func parseProgress(b []byte) (seconds, speed float64) {
|
||||||
|
sc := bufio.NewScanner(strings.NewReader(string(b)))
|
||||||
|
for sc.Scan() {
|
||||||
|
k, v, ok := strings.Cut(sc.Text(), "=")
|
||||||
|
if !ok {
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
v = strings.TrimSpace(v)
|
||||||
|
switch k {
|
||||||
|
case "out_time_us":
|
||||||
|
if us, err := strconv.ParseInt(v, 10, 64); err == nil && us >= 0 {
|
||||||
|
seconds = float64(us) / 1e6
|
||||||
|
}
|
||||||
|
case "speed":
|
||||||
|
if x, err := strconv.ParseFloat(strings.TrimSuffix(v, "x"), 64); err == nil {
|
||||||
|
speed = x
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return seconds, speed
|
||||||
|
}
|
||||||
|
|
||||||
|
func newID() string {
|
||||||
|
b := make([]byte, 8)
|
||||||
|
_, _ = rand.Read(b)
|
||||||
|
return hex.EncodeToString(b)
|
||||||
|
}
|
||||||
@@ -0,0 +1,85 @@
|
|||||||
|
package media
|
||||||
|
|
||||||
|
import (
|
||||||
|
"context"
|
||||||
|
"os"
|
||||||
|
"os/exec"
|
||||||
|
"path/filepath"
|
||||||
|
"testing"
|
||||||
|
"time"
|
||||||
|
)
|
||||||
|
|
||||||
|
// The real thing, when ffmpeg is here: a generated clip goes through as H.264 and comes out
|
||||||
|
// as a playlist of segments, the status follows it to the end, and closing removes it all.
|
||||||
|
func TestStreamWithFFmpeg(t *testing.T) {
|
||||||
|
if _, err := exec.LookPath("ffmpeg"); err != nil {
|
||||||
|
t.Skip("no ffmpeg")
|
||||||
|
}
|
||||||
|
c := Probe(context.Background())
|
||||||
|
if c.Error != "" {
|
||||||
|
t.Skip(c.Error)
|
||||||
|
}
|
||||||
|
if EngineFor(c, "h264") == "" {
|
||||||
|
t.Skip("nothing here encodes h264")
|
||||||
|
}
|
||||||
|
dir := t.TempDir()
|
||||||
|
clip := filepath.Join(dir, "clip.mp4")
|
||||||
|
out, err := exec.Command("ffmpeg", "-nostdin", "-v", "error", "-f", "lavfi", "-i", "testsrc=size=640x360:rate=25:duration=9", "-f", "lavfi", "-i", "sine=frequency=440:duration=9",
|
||||||
|
"-c:v", "mpeg4", "-q:v", "5", "-c:a", "aac", "-shortest", clip).CombinedOutput()
|
||||||
|
if err != nil {
|
||||||
|
t.Skipf("could not make a clip: %s", out)
|
||||||
|
}
|
||||||
|
src, err := Inspect(context.Background(), c, clip, false)
|
||||||
|
if err != nil || src.Video == nil || src.Audio == nil || src.Video.Codec != "mpeg4" || src.Duration < 8 {
|
||||||
|
t.Fatalf("inspect: %+v %v", src, err)
|
||||||
|
}
|
||||||
|
m := NewStreamer(dir, "hamlab", 2)
|
||||||
|
s, err := m.Open(context.Background(), c, false, clip, "h264", "480p", src)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if s.Width != 640 || s.Height != 360 || s.Engine == "" {
|
||||||
|
t.Fatalf("stream: %+v", s)
|
||||||
|
}
|
||||||
|
if p, ok := s.File("index.m3u8"); !ok {
|
||||||
|
t.Fatal("no playlist")
|
||||||
|
} else if !hasSegments(p) {
|
||||||
|
t.Fatal("Open returned before the first segment")
|
||||||
|
}
|
||||||
|
for _, bad := range []string{"../x", "progress", "seg1.m4s", "index.m3u8/", "init.mp4.tmp"} {
|
||||||
|
if _, ok := s.File(bad); ok {
|
||||||
|
t.Fatalf("%q must not be served", bad)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
// the same request is the same stream
|
||||||
|
if again, err := m.Open(context.Background(), c, false, clip, "h264", "480p", src); err != nil || again.ID != s.ID {
|
||||||
|
t.Fatal("not reused")
|
||||||
|
}
|
||||||
|
var st StreamStatus
|
||||||
|
for i := 0; i < 120; i++ {
|
||||||
|
st = s.Status()
|
||||||
|
if st.Done || st.Error != "" {
|
||||||
|
break
|
||||||
|
}
|
||||||
|
time.Sleep(250 * time.Millisecond)
|
||||||
|
}
|
||||||
|
if !st.Done || st.Error != "" || st.Segments < 2 || st.Progress < 8 {
|
||||||
|
t.Fatalf("status at the end: %+v", st)
|
||||||
|
}
|
||||||
|
if init, _ := s.File("init.mp4"); !exists(init) {
|
||||||
|
t.Fatal("no init segment")
|
||||||
|
}
|
||||||
|
m.Close(s.ID)
|
||||||
|
if _, ok := m.Find(s.ID); ok || exists(s.dir) {
|
||||||
|
t.Fatal("close must forget and remove")
|
||||||
|
}
|
||||||
|
// an unplayable source is a DecodeError, and leaves nothing behind
|
||||||
|
bad := filepath.Join(dir, "bad.mkv")
|
||||||
|
_ = os.WriteFile(bad, []byte("not a video"), 0o644)
|
||||||
|
if _, err := Inspect(context.Background(), c, bad, false); err == nil {
|
||||||
|
t.Fatal("inspect of garbage")
|
||||||
|
}
|
||||||
|
if left, _ := os.ReadDir(m.dir); len(left) != 0 {
|
||||||
|
t.Fatalf("left behind: %v", left)
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -0,0 +1,142 @@
|
|||||||
|
package media
|
||||||
|
|
||||||
|
import (
|
||||||
|
"bytes"
|
||||||
|
"context"
|
||||||
|
"encoding/json"
|
||||||
|
"errors"
|
||||||
|
"os/exec"
|
||||||
|
"strconv"
|
||||||
|
"strings"
|
||||||
|
|
||||||
|
"hamlab/internal/priv"
|
||||||
|
)
|
||||||
|
|
||||||
|
// Track is one stream of a media file, as ffprobe reports it.
|
||||||
|
type Track struct {
|
||||||
|
Index int `json:"index"`
|
||||||
|
Type string `json:"type"` // video | audio
|
||||||
|
Codec string `json:"codec"`
|
||||||
|
Profile string `json:"profile,omitempty"`
|
||||||
|
PixFmt string `json:"pix_fmt,omitempty"`
|
||||||
|
Width int `json:"width,omitempty"`
|
||||||
|
Height int `json:"height,omitempty"`
|
||||||
|
FPS float64 `json:"fps,omitempty"`
|
||||||
|
ColorTransfer string `json:"color_transfer,omitempty"` // smpte2084 / arib-std-b67 mean HDR
|
||||||
|
Channels int `json:"channels,omitempty"`
|
||||||
|
AttachedPic bool `json:"attached_pic,omitempty"`
|
||||||
|
}
|
||||||
|
|
||||||
|
// Info is what a file holds: the picture to take, the sound to go with it, and the whole.
|
||||||
|
type Info struct {
|
||||||
|
Container string `json:"container"` // "matroska,webm", "mov,mp4,m4a,3gp,3g2,mj2"
|
||||||
|
Duration float64 `json:"duration"`
|
||||||
|
BitRate int64 `json:"bit_rate,omitempty"` // bits per second, whole file
|
||||||
|
Size int64 `json:"size,omitempty"`
|
||||||
|
Video *Track `json:"video,omitempty"` // the first real picture stream; cover art is not it
|
||||||
|
Audio *Track `json:"audio,omitempty"` // the first sound stream
|
||||||
|
}
|
||||||
|
|
||||||
|
// HDR reports a picture that needs tone mapping to look right on an SDR screen.
|
||||||
|
func (i Info) HDR() bool {
|
||||||
|
return i.Video != nil && (i.Video.ColorTransfer == "smpte2084" || i.Video.ColorTransfer == "arib-std-b67")
|
||||||
|
}
|
||||||
|
|
||||||
|
type probeJSON struct {
|
||||||
|
Streams []struct {
|
||||||
|
Index int `json:"index"`
|
||||||
|
CodecType string `json:"codec_type"`
|
||||||
|
CodecName string `json:"codec_name"`
|
||||||
|
Profile string `json:"profile"`
|
||||||
|
PixFmt string `json:"pix_fmt"`
|
||||||
|
Width int `json:"width"`
|
||||||
|
Height int `json:"height"`
|
||||||
|
RFrameRate string `json:"r_frame_rate"`
|
||||||
|
AvgFrameRate string `json:"avg_frame_rate"`
|
||||||
|
ColorTransfer string `json:"color_transfer"`
|
||||||
|
Channels int `json:"channels"`
|
||||||
|
Disposition struct {
|
||||||
|
AttachedPic int `json:"attached_pic"`
|
||||||
|
} `json:"disposition"`
|
||||||
|
} `json:"streams"`
|
||||||
|
Format struct {
|
||||||
|
FormatName string `json:"format_name"`
|
||||||
|
Duration string `json:"duration"`
|
||||||
|
BitRate string `json:"bit_rate"`
|
||||||
|
Size string `json:"size"`
|
||||||
|
} `json:"format"`
|
||||||
|
}
|
||||||
|
|
||||||
|
// Inspect asks ffprobe what a file is. As root through sudo when asked, for a file only
|
||||||
|
// root may read. An error about the file itself is a DecodeError.
|
||||||
|
func Inspect(ctx context.Context, c Caps, path string, root bool) (Info, error) {
|
||||||
|
if c.FFprobe == "" {
|
||||||
|
return Info{}, errors.New("ffprobe is not installed")
|
||||||
|
}
|
||||||
|
argv := []string{c.FFprobe, "-v", "error", "-show_entries",
|
||||||
|
"stream=index,codec_type,codec_name,profile,pix_fmt,width,height,r_frame_rate,avg_frame_rate,color_transfer,channels,disposition:format=format_name,duration,bit_rate,size",
|
||||||
|
"-of", "json", path}
|
||||||
|
if root {
|
||||||
|
argv = priv.Wrap(argv...)
|
||||||
|
}
|
||||||
|
cmd := exec.CommandContext(ctx, argv[0], argv[1:]...)
|
||||||
|
var errb bytes.Buffer
|
||||||
|
cmd.Stderr = &errb
|
||||||
|
out, err := cmd.Output()
|
||||||
|
if err != nil {
|
||||||
|
if ctx.Err() != nil {
|
||||||
|
return Info{}, ctx.Err()
|
||||||
|
}
|
||||||
|
msg := lastLine(errb.String())
|
||||||
|
if msg == "" {
|
||||||
|
msg = "ffprobe: " + err.Error()
|
||||||
|
}
|
||||||
|
return Info{}, &DecodeError{Msg: msg}
|
||||||
|
}
|
||||||
|
var pj probeJSON
|
||||||
|
if err := json.Unmarshal(out, &pj); err != nil {
|
||||||
|
return Info{}, &DecodeError{Msg: "ffprobe: unreadable answer"}
|
||||||
|
}
|
||||||
|
in := Info{Container: pj.Format.FormatName}
|
||||||
|
in.Duration, _ = strconv.ParseFloat(pj.Format.Duration, 64)
|
||||||
|
in.BitRate, _ = strconv.ParseInt(pj.Format.BitRate, 10, 64)
|
||||||
|
in.Size, _ = strconv.ParseInt(pj.Format.Size, 10, 64)
|
||||||
|
var art *Track
|
||||||
|
for _, s := range pj.Streams {
|
||||||
|
st := &Track{Index: s.Index, Type: s.CodecType, Codec: s.CodecName, Profile: s.Profile, PixFmt: s.PixFmt, Width: s.Width, Height: s.Height,
|
||||||
|
ColorTransfer: s.ColorTransfer, Channels: s.Channels, AttachedPic: s.Disposition.AttachedPic != 0}
|
||||||
|
st.FPS = fps(s.AvgFrameRate)
|
||||||
|
if st.FPS == 0 {
|
||||||
|
st.FPS = fps(s.RFrameRate)
|
||||||
|
}
|
||||||
|
switch {
|
||||||
|
case s.CodecType == "video" && st.AttachedPic:
|
||||||
|
if art == nil {
|
||||||
|
art = st
|
||||||
|
}
|
||||||
|
case s.CodecType == "video" && in.Video == nil:
|
||||||
|
in.Video = st
|
||||||
|
case s.CodecType == "audio" && in.Audio == nil:
|
||||||
|
in.Audio = st
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if in.Video == nil {
|
||||||
|
in.Video = art // cover art is the only picture there is
|
||||||
|
}
|
||||||
|
return in, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
// fps parses ffprobe's "60000/1001".
|
||||||
|
func fps(r string) float64 {
|
||||||
|
n, d, ok := strings.Cut(r, "/")
|
||||||
|
if !ok {
|
||||||
|
v, _ := strconv.ParseFloat(r, 64)
|
||||||
|
return v
|
||||||
|
}
|
||||||
|
nf, _ := strconv.ParseFloat(n, 64)
|
||||||
|
df, _ := strconv.ParseFloat(d, 64)
|
||||||
|
if df == 0 {
|
||||||
|
return 0
|
||||||
|
}
|
||||||
|
return nf / df
|
||||||
|
}
|
||||||
@@ -0,0 +1,26 @@
|
|||||||
|
package media
|
||||||
|
|
||||||
|
import (
|
||||||
|
"fmt"
|
||||||
|
"os"
|
||||||
|
"os/user"
|
||||||
|
"strconv"
|
||||||
|
"syscall"
|
||||||
|
)
|
||||||
|
|
||||||
|
// ownerOf is "root:render 0660" for a device node, the thing to say when it cannot be opened.
|
||||||
|
func ownerOf(st os.FileInfo) string {
|
||||||
|
sys, ok := st.Sys().(*syscall.Stat_t)
|
||||||
|
if !ok {
|
||||||
|
return ""
|
||||||
|
}
|
||||||
|
u := strconv.Itoa(int(sys.Uid))
|
||||||
|
if x, err := user.LookupId(u); err == nil {
|
||||||
|
u = x.Username
|
||||||
|
}
|
||||||
|
g := strconv.Itoa(int(sys.Gid))
|
||||||
|
if x, err := user.LookupGroupId(g); err == nil {
|
||||||
|
g = x.Name
|
||||||
|
}
|
||||||
|
return fmt.Sprintf("%s:%s %04o", u, g, st.Mode().Perm())
|
||||||
|
}
|
||||||
@@ -0,0 +1,656 @@
|
|||||||
|
// Package thumbs makes the Files tab's previews: one small JPEG for an image or a video,
|
||||||
|
// by ffmpeg, decoded on a GPU where the machine has one it can use and on the CPU
|
||||||
|
// otherwise.
|
||||||
|
//
|
||||||
|
// Nothing about the machine is assumed. A probe looks ffmpeg and ffprobe up, opens every
|
||||||
|
// /dev/dri/renderD* and exercises it (a VAAPI upload and download of a synthetic frame),
|
||||||
|
// so a node the daemon's user cannot open, or a driver that does not speak VAAPI, is
|
||||||
|
// reported once rather than tried per file. vainfo, where installed, says which codecs
|
||||||
|
// each node decodes, and a video goes to the first node that lists its codec; without
|
||||||
|
// vainfo the first working node is tried and ffmpeg falls back to software on its own when
|
||||||
|
// the stream turns out unsupported. A hardware run that fails outright is retried on the
|
||||||
|
// CPU before the file is called undecodable. Images are always decoded on the CPU: one
|
||||||
|
// frame of a JPEG at a reduced DCT size costs less than the trip through a GPU.
|
||||||
|
package media
|
||||||
|
|
||||||
|
import (
|
||||||
|
"bytes"
|
||||||
|
"context"
|
||||||
|
"encoding/json"
|
||||||
|
"errors"
|
||||||
|
"fmt"
|
||||||
|
"io"
|
||||||
|
"os"
|
||||||
|
"os/exec"
|
||||||
|
"path/filepath"
|
||||||
|
"regexp"
|
||||||
|
"sort"
|
||||||
|
"strconv"
|
||||||
|
"strings"
|
||||||
|
"time"
|
||||||
|
|
||||||
|
"hamlab/internal/metrics"
|
||||||
|
"hamlab/internal/priv"
|
||||||
|
)
|
||||||
|
|
||||||
|
// GPU is one hardware decoder as the probe found it.
|
||||||
|
type GPU struct {
|
||||||
|
API string `json:"api"` // vaapi | cuda
|
||||||
|
Device string `json:"device"` // /dev/dri/renderD128; "0" for cuda (its device index)
|
||||||
|
Name string `json:"name,omitempty"` // "Intel Arc A380"
|
||||||
|
Codecs []string `json:"codecs,omitempty"` // ffmpeg codec names it decodes; nil when nothing could say
|
||||||
|
Encodes []string `json:"encodes,omitempty"` // ffmpeg codec names it encodes, from vainfo
|
||||||
|
Integrated bool `json:"integrated,omitempty"` // on the CPU die; a card on the bus is preferred for encoding
|
||||||
|
Reason string `json:"reason,omitempty"` // Unusable only: why
|
||||||
|
Permission bool `json:"permission,omitempty"` // Unusable only: the node exists but this user may not open it
|
||||||
|
}
|
||||||
|
|
||||||
|
// Caps is what the machine can make previews with, as probed.
|
||||||
|
type Caps struct {
|
||||||
|
FFmpeg string `json:"ffmpeg,omitempty"`
|
||||||
|
FFprobe string `json:"ffprobe,omitempty"`
|
||||||
|
Nice string `json:"nice,omitempty"`
|
||||||
|
Version string `json:"version,omitempty"`
|
||||||
|
GPUs []GPU `json:"gpus"` // usable, in node order
|
||||||
|
Encoders []string `json:"encoders,omitempty"` // the encoders ffmpeg itself was built with that matter here
|
||||||
|
Unusable []GPU `json:"unusable,omitempty"` // render nodes that exist but cannot be used, with the reason
|
||||||
|
Error string `json:"error,omitempty"` // why nothing can be made at all
|
||||||
|
ProbedAt time.Time `json:"probed_at"`
|
||||||
|
}
|
||||||
|
|
||||||
|
// Opts shapes one preview.
|
||||||
|
type Opts struct {
|
||||||
|
Size int // the long edge in pixels; 256 when zero
|
||||||
|
Timeout time.Duration // for the whole job, ffprobe and every ffmpeg attempt; 30 s when zero
|
||||||
|
}
|
||||||
|
|
||||||
|
func (o Opts) defaults() Opts {
|
||||||
|
if o.Size <= 0 {
|
||||||
|
o.Size = 256
|
||||||
|
}
|
||||||
|
if o.Timeout <= 0 {
|
||||||
|
o.Timeout = 30 * time.Second
|
||||||
|
}
|
||||||
|
return o
|
||||||
|
}
|
||||||
|
|
||||||
|
// Result is a finished preview.
|
||||||
|
type Result struct {
|
||||||
|
JPEG []byte `json:"-"`
|
||||||
|
Engine string `json:"engine"` // "cpu", or "vaapi:/dev/dri/renderD129"
|
||||||
|
Codec string `json:"codec"`
|
||||||
|
Width int `json:"width"`
|
||||||
|
Height int `json:"height"`
|
||||||
|
}
|
||||||
|
|
||||||
|
// ErrUnsupported: not an image or a video by its name.
|
||||||
|
var ErrUnsupported = errors.New("not an image or video")
|
||||||
|
|
||||||
|
// A DecodeError is ffmpeg's verdict on the file itself. It will not change until the
|
||||||
|
// file does, so the caller may remember it.
|
||||||
|
type DecodeError struct{ Msg string }
|
||||||
|
|
||||||
|
func (e *DecodeError) Error() string { return e.Msg }
|
||||||
|
|
||||||
|
// ---- what gets a preview ----
|
||||||
|
|
||||||
|
var imageExt = set("png jpg jpeg jpe jfif gif webp bmp tif tiff avif heic heif hif jxl svg ico psd tga thm")
|
||||||
|
var videoExt = set("mp4 m4v mkv webm mov avi wmv flv mpg mpeg mpe m2v ts m2ts mts 3gp 3g2 ogv ogm mxf vob asf rm rmvb divx f4v mjpeg mjpg 360 osv insv lrf")
|
||||||
|
var audioExt = set("mp3 flac m4a m4b aac ogg oga opus wav wma aiff aif alac ape wv mka")
|
||||||
|
|
||||||
|
func set(s string) map[string]bool {
|
||||||
|
m := map[string]bool{}
|
||||||
|
for _, w := range strings.Fields(s) {
|
||||||
|
m[w] = true
|
||||||
|
}
|
||||||
|
return m
|
||||||
|
}
|
||||||
|
|
||||||
|
// Kind is "image", "video", "audio" or "" for a file name. An audio file's preview is its
|
||||||
|
// cover art, when it carries any.
|
||||||
|
func Kind(name string) string {
|
||||||
|
ext := strings.ToLower(strings.TrimPrefix(filepath.Ext(name), "."))
|
||||||
|
switch {
|
||||||
|
case imageExt[ext]:
|
||||||
|
return "image"
|
||||||
|
case videoExt[ext]:
|
||||||
|
return "video"
|
||||||
|
case audioExt[ext]:
|
||||||
|
return "audio"
|
||||||
|
}
|
||||||
|
return ""
|
||||||
|
}
|
||||||
|
|
||||||
|
// ---- the probe ----
|
||||||
|
|
||||||
|
// Probe finds out what this process can make previews with. It runs ffmpeg against every
|
||||||
|
// render node, so it takes a few hundred milliseconds; callers keep the answer.
|
||||||
|
func Probe(ctx context.Context) Caps {
|
||||||
|
c := Caps{ProbedAt: time.Now(), GPUs: []GPU{}}
|
||||||
|
c.FFmpeg, _ = exec.LookPath("ffmpeg")
|
||||||
|
c.FFprobe, _ = exec.LookPath("ffprobe")
|
||||||
|
c.Nice, _ = exec.LookPath("nice")
|
||||||
|
switch {
|
||||||
|
case c.FFmpeg == "" && c.FFprobe == "":
|
||||||
|
c.Error = "ffmpeg is not installed"
|
||||||
|
case c.FFmpeg == "":
|
||||||
|
c.Error = "ffmpeg is not installed (ffprobe is)"
|
||||||
|
case c.FFprobe == "":
|
||||||
|
c.Error = "ffprobe is not installed (ffmpeg is)"
|
||||||
|
}
|
||||||
|
if c.Error != "" {
|
||||||
|
return c
|
||||||
|
}
|
||||||
|
c.Version = ffmpegVersion(ctx, c.FFmpeg)
|
||||||
|
c.Encoders = encoders(ctx, c.FFmpeg)
|
||||||
|
accels := hwaccels(ctx, c.FFmpeg)
|
||||||
|
if accels["vaapi"] {
|
||||||
|
nodes, _ := filepath.Glob("/dev/dri/renderD*")
|
||||||
|
sort.Strings(nodes)
|
||||||
|
vainfo, _ := exec.LookPath("vainfo")
|
||||||
|
for _, dev := range nodes {
|
||||||
|
g := GPU{API: "vaapi", Device: dev, Name: metrics.RenderNodeName(dev), Integrated: metrics.RenderNodeIntegrated(dev)}
|
||||||
|
if err := openable(dev); err != nil {
|
||||||
|
g.Reason, g.Permission = err.Error(), errors.Is(err, os.ErrPermission)
|
||||||
|
c.Unusable = append(c.Unusable, g)
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
if msg := hwWorks(ctx, c.FFmpeg, "vaapi=va:"+dev); msg != "" {
|
||||||
|
g.Reason = msg
|
||||||
|
c.Unusable = append(c.Unusable, g)
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
if vainfo != "" {
|
||||||
|
g.Codecs, g.Encodes = vainfoCodecs(ctx, vainfo, dev)
|
||||||
|
}
|
||||||
|
c.GPUs = append(c.GPUs, g)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if accels["cuda"] && exists("/dev/nvidiactl") {
|
||||||
|
g := GPU{API: "cuda", Device: "0", Name: "NVIDIA GPU"}
|
||||||
|
if msg := hwWorks(ctx, c.FFmpeg, "cuda=cu:0"); msg != "" {
|
||||||
|
g.Reason = msg
|
||||||
|
c.Unusable = append(c.Unusable, g)
|
||||||
|
} else {
|
||||||
|
c.GPUs = append(c.GPUs, g)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return c
|
||||||
|
}
|
||||||
|
|
||||||
|
// openable: the node can be opened read-write, which is what a VAAPI driver needs. The
|
||||||
|
// usual reason it can't be is the owning group: a render node is root:render 0660 on
|
||||||
|
// most distributions, and a daemon running as an ordinary user is not in render.
|
||||||
|
func openable(dev string) error {
|
||||||
|
f, err := os.OpenFile(dev, os.O_RDWR, 0)
|
||||||
|
if err != nil {
|
||||||
|
if errors.Is(err, os.ErrPermission) {
|
||||||
|
owner := ""
|
||||||
|
if st, serr := os.Stat(dev); serr == nil {
|
||||||
|
owner = ownerOf(st)
|
||||||
|
}
|
||||||
|
if owner != "" {
|
||||||
|
return fmt.Errorf("%w: %s is %s", os.ErrPermission, dev, owner)
|
||||||
|
}
|
||||||
|
return fmt.Errorf("%w: cannot open %s", os.ErrPermission, dev)
|
||||||
|
}
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
f.Close()
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
|
||||||
|
// hwWorks initialises a hardware device in ffmpeg and moves one synthetic frame through
|
||||||
|
// it. "" when it works, else the last thing ffmpeg said.
|
||||||
|
func hwWorks(ctx context.Context, ffmpeg, device string) string {
|
||||||
|
ctx, cancel := context.WithTimeout(ctx, 15*time.Second)
|
||||||
|
defer cancel()
|
||||||
|
name, rest, _ := strings.Cut(device, "=") // "vaapi", "va:/dev/…"
|
||||||
|
label, _, _ := strings.Cut(rest, ":") // "va"
|
||||||
|
cmd := exec.CommandContext(ctx, ffmpeg, "-nostdin", "-hide_banner", "-v", "error", "-init_hw_device", device, "-filter_hw_device", label,
|
||||||
|
"-f", "lavfi", "-i", "color=c=black:s=64x64:r=1:d=1", "-vf", "format=nv12,hwupload,hwdownload,format=nv12", "-frames:v", "1", "-f", "null", "-")
|
||||||
|
var errb bytes.Buffer
|
||||||
|
cmd.Stderr = &errb
|
||||||
|
if err := cmd.Run(); err != nil {
|
||||||
|
msg := lastLine(errb.String())
|
||||||
|
if msg == "" {
|
||||||
|
msg = err.Error()
|
||||||
|
}
|
||||||
|
return name + ": " + msg
|
||||||
|
}
|
||||||
|
return ""
|
||||||
|
}
|
||||||
|
|
||||||
|
func ffmpegVersion(ctx context.Context, ffmpeg string) string {
|
||||||
|
ctx, cancel := context.WithTimeout(ctx, 5*time.Second)
|
||||||
|
defer cancel()
|
||||||
|
out, _ := exec.CommandContext(ctx, ffmpeg, "-version").Output()
|
||||||
|
f := strings.Fields(string(out)) // "ffmpeg version 7.1.5-0+deb13u1 Copyright …"
|
||||||
|
if len(f) >= 3 && f[1] == "version" {
|
||||||
|
v := f[2]
|
||||||
|
if i := strings.IndexAny(v, "-+~"); i > 0 {
|
||||||
|
v = v[:i]
|
||||||
|
}
|
||||||
|
return v
|
||||||
|
}
|
||||||
|
return ""
|
||||||
|
}
|
||||||
|
|
||||||
|
func hwaccels(ctx context.Context, ffmpeg string) map[string]bool {
|
||||||
|
ctx, cancel := context.WithTimeout(ctx, 5*time.Second)
|
||||||
|
defer cancel()
|
||||||
|
out, _ := exec.CommandContext(ctx, ffmpeg, "-hide_banner", "-hwaccels").Output()
|
||||||
|
m := map[string]bool{}
|
||||||
|
for _, l := range strings.Split(string(out), "\n") {
|
||||||
|
l = strings.TrimSpace(l)
|
||||||
|
if l != "" && !strings.HasPrefix(l, "Hardware acceleration methods") {
|
||||||
|
m[l] = true
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return m
|
||||||
|
}
|
||||||
|
|
||||||
|
var vaProfileRe = regexp.MustCompile(`VAProfile([A-Za-z0-9]+?)(?:Simple|Main|High|Profile|Baseline|Constrained|Scc|Advanced|Version|Stereo|Multiview)\w*\s*:\s*VAEntrypoint(VLD|EncSlice|EncSliceLP)\b`)
|
||||||
|
|
||||||
|
// wantedEncoders are the encoders this package may ask ffmpeg for.
|
||||||
|
var wantedEncoders = set("libx264 libx265 libsvtav1 libaom-av1 libopenh264 h264_vaapi hevc_vaapi av1_vaapi aac")
|
||||||
|
|
||||||
|
// encoders lists the wanted encoders this ffmpeg was built with.
|
||||||
|
func encoders(ctx context.Context, ffmpeg string) []string {
|
||||||
|
ctx, cancel := context.WithTimeout(ctx, 5*time.Second)
|
||||||
|
defer cancel()
|
||||||
|
out, _ := exec.CommandContext(ctx, ffmpeg, "-hide_banner", "-encoders").Output()
|
||||||
|
var found []string
|
||||||
|
for _, l := range strings.Split(string(out), "\n") {
|
||||||
|
f := strings.Fields(l)
|
||||||
|
if len(f) >= 2 && wantedEncoders[f[1]] {
|
||||||
|
found = append(found, f[1])
|
||||||
|
}
|
||||||
|
}
|
||||||
|
sort.Strings(found)
|
||||||
|
return found
|
||||||
|
}
|
||||||
|
|
||||||
|
// vaCodec maps a VAProfile family to the ffmpeg codec it decodes.
|
||||||
|
var vaCodec = map[string]string{"H264": "h264", "HEVC": "hevc", "VP9": "vp9", "AV1": "av1", "MPEG2": "mpeg2video", "VC1": "vc1", "VP8": "vp8", "JPEG": "mjpeg", "MPEG4": "mpeg4", "H263": "h263"}
|
||||||
|
|
||||||
|
func vainfoCodecs(ctx context.Context, vainfo, dev string) (decodes, encodes []string) {
|
||||||
|
ctx, cancel := context.WithTimeout(ctx, 10*time.Second)
|
||||||
|
defer cancel()
|
||||||
|
out, _ := exec.CommandContext(ctx, vainfo, "--display", "drm", "--device", dev).Output()
|
||||||
|
return parseVainfo(string(out))
|
||||||
|
}
|
||||||
|
|
||||||
|
// parseVainfo names the codecs the decode (VLD) and encode (EncSlice) profiles cover,
|
||||||
|
// the way ffmpeg names them.
|
||||||
|
func parseVainfo(text string) (decodes, encodes []string) {
|
||||||
|
dec, enc := map[string]bool{}, map[string]bool{}
|
||||||
|
for _, m := range vaProfileRe.FindAllStringSubmatch(text, -1) {
|
||||||
|
c, ok := vaCodec[m[1]]
|
||||||
|
if !ok {
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
if m[2] == "VLD" {
|
||||||
|
dec[c] = true
|
||||||
|
} else {
|
||||||
|
enc[c] = true
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return keys(dec), keys(enc)
|
||||||
|
}
|
||||||
|
|
||||||
|
func keys(m map[string]bool) []string {
|
||||||
|
if len(m) == 0 {
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
out := make([]string, 0, len(m))
|
||||||
|
for c := range m {
|
||||||
|
out = append(out, c)
|
||||||
|
}
|
||||||
|
sort.Strings(out)
|
||||||
|
return out
|
||||||
|
}
|
||||||
|
|
||||||
|
// pick is the GPU for a codec: the first that lists it, else the first that could not
|
||||||
|
// say (ffmpeg will fall back to software by itself if that guess is wrong), else none.
|
||||||
|
func (c Caps) pick(codec string) *GPU {
|
||||||
|
var unknown *GPU
|
||||||
|
for i := range c.GPUs {
|
||||||
|
g := &c.GPUs[i]
|
||||||
|
if g.Codecs == nil {
|
||||||
|
if unknown == nil {
|
||||||
|
unknown = g
|
||||||
|
}
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
for _, x := range g.Codecs {
|
||||||
|
if x == codec {
|
||||||
|
return g
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return unknown
|
||||||
|
}
|
||||||
|
|
||||||
|
// ---- making one ----
|
||||||
|
|
||||||
|
type fileInfo struct {
|
||||||
|
index string // the stream to take
|
||||||
|
codec string
|
||||||
|
w, h int
|
||||||
|
duration float64
|
||||||
|
}
|
||||||
|
|
||||||
|
type attempt struct {
|
||||||
|
gpu *GPU
|
||||||
|
lowres int
|
||||||
|
seek float64
|
||||||
|
}
|
||||||
|
|
||||||
|
func (a attempt) engine() string {
|
||||||
|
if a.gpu == nil {
|
||||||
|
return "cpu"
|
||||||
|
}
|
||||||
|
return a.gpu.API + ":" + a.gpu.Device
|
||||||
|
}
|
||||||
|
|
||||||
|
// Make produces the preview of one file.
|
||||||
|
func Make(ctx context.Context, c Caps, path string, o Opts) (Result, error) {
|
||||||
|
kind := Kind(path)
|
||||||
|
if kind == "" {
|
||||||
|
return Result{}, ErrUnsupported
|
||||||
|
}
|
||||||
|
if c.Error != "" {
|
||||||
|
return Result{}, errors.New(c.Error)
|
||||||
|
}
|
||||||
|
o = o.defaults()
|
||||||
|
ctx, cancel := context.WithTimeout(ctx, o.Timeout)
|
||||||
|
defer cancel()
|
||||||
|
fi, err := probeFile(ctx, c, path)
|
||||||
|
if err != nil {
|
||||||
|
return Result{}, err
|
||||||
|
}
|
||||||
|
var last string
|
||||||
|
for _, eng := range engines(c, kind, fi) {
|
||||||
|
for _, a := range attempts(eng, kind, fi, o.Size) {
|
||||||
|
data, msg, err := run(ctx, c, a, path, fi, o.Size)
|
||||||
|
if err == nil && len(data) > 0 {
|
||||||
|
return Result{JPEG: data, Engine: a.engine(), Codec: fi.codec, Width: fi.w, Height: fi.h}, nil
|
||||||
|
}
|
||||||
|
if ctx.Err() != nil {
|
||||||
|
return Result{}, ctx.Err() // the clock ran out: not the file's fault
|
||||||
|
}
|
||||||
|
if err != nil { // the decoder refused: try the next engine, not another seek
|
||||||
|
last = msg
|
||||||
|
break
|
||||||
|
}
|
||||||
|
last = "no frame at " + strconv.FormatFloat(a.seek, 'f', 1, 64) + " s"
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if last == "" {
|
||||||
|
last = "could not decode"
|
||||||
|
}
|
||||||
|
return Result{}, &DecodeError{Msg: last}
|
||||||
|
}
|
||||||
|
|
||||||
|
// engines is the order to try: the fitting GPU first for a video, then the CPU.
|
||||||
|
func engines(c Caps, kind string, fi fileInfo) []*GPU {
|
||||||
|
if kind == "video" {
|
||||||
|
if g := c.pick(fi.codec); g != nil {
|
||||||
|
return []*GPU{g, nil}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return []*GPU{nil}
|
||||||
|
}
|
||||||
|
|
||||||
|
// attempts for one engine: a seek into a video, then from its start if that found no
|
||||||
|
// frame (a wrong duration in the header); a reduced-size decode of a JPEG, then full size.
|
||||||
|
func attempts(gpu *GPU, kind string, fi fileInfo, size int) []attempt {
|
||||||
|
if kind == "video" {
|
||||||
|
s := seekFor(fi.duration)
|
||||||
|
if s > 0 {
|
||||||
|
return []attempt{{gpu: gpu, seek: s}, {gpu: gpu}}
|
||||||
|
}
|
||||||
|
return []attempt{{gpu: gpu}}
|
||||||
|
}
|
||||||
|
if fi.codec == "mjpeg" && gpu == nil {
|
||||||
|
if l := lowresFor(fi.w, fi.h, size); l > 0 {
|
||||||
|
return []attempt{{lowres: l}, {}}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return []attempt{{gpu: gpu}}
|
||||||
|
}
|
||||||
|
|
||||||
|
// seekFor: a tenth of the way in, never more than 30 s, so a title card or a black lead-in
|
||||||
|
// is skipped and a long file still seeks to a keyframe near its start.
|
||||||
|
func seekFor(duration float64) float64 {
|
||||||
|
if duration <= 0 {
|
||||||
|
return 0
|
||||||
|
}
|
||||||
|
s := duration / 10
|
||||||
|
if s > 30 {
|
||||||
|
s = 30
|
||||||
|
}
|
||||||
|
if s < 0.5 {
|
||||||
|
return 0
|
||||||
|
}
|
||||||
|
return s
|
||||||
|
}
|
||||||
|
|
||||||
|
// lowresFor: how many halvings a JPEG decoder may take while the picture stays at least
|
||||||
|
// twice the preview size. libjpeg does this in the DCT, so a 24 MP photo decodes at 1.5 MP.
|
||||||
|
func lowresFor(w, h, size int) int {
|
||||||
|
m := w
|
||||||
|
if h < m {
|
||||||
|
m = h
|
||||||
|
}
|
||||||
|
l := 0
|
||||||
|
for l < 3 && m>>(l+1) >= size*2 {
|
||||||
|
l++
|
||||||
|
}
|
||||||
|
return l
|
||||||
|
}
|
||||||
|
|
||||||
|
type probeOut struct {
|
||||||
|
Streams []struct {
|
||||||
|
Index int `json:"index"`
|
||||||
|
Codec string `json:"codec_name"`
|
||||||
|
Width int `json:"width"`
|
||||||
|
Height int `json:"height"`
|
||||||
|
Disposition struct {
|
||||||
|
AttachedPic int `json:"attached_pic"`
|
||||||
|
} `json:"disposition"`
|
||||||
|
} `json:"streams"`
|
||||||
|
Format struct {
|
||||||
|
Duration string `json:"duration"`
|
||||||
|
} `json:"format"`
|
||||||
|
}
|
||||||
|
|
||||||
|
// probeFile asks ffprobe which stream to take and what is in it.
|
||||||
|
func probeFile(ctx context.Context, c Caps, path string) (fileInfo, error) {
|
||||||
|
cmd := exec.CommandContext(ctx, c.FFprobe, "-v", "error", "-select_streams", "v", "-show_entries", "stream=index,codec_name,width,height,disposition:format=duration", "-of", "json", path)
|
||||||
|
var errb bytes.Buffer
|
||||||
|
cmd.Stderr = &errb
|
||||||
|
out, err := cmd.Output()
|
||||||
|
if err != nil {
|
||||||
|
if ctx.Err() != nil {
|
||||||
|
return fileInfo{}, ctx.Err()
|
||||||
|
}
|
||||||
|
msg := lastLine(errb.String())
|
||||||
|
if msg == "" {
|
||||||
|
msg = "ffprobe: " + err.Error()
|
||||||
|
}
|
||||||
|
return fileInfo{}, &DecodeError{Msg: msg}
|
||||||
|
}
|
||||||
|
var po probeOut
|
||||||
|
if err := json.Unmarshal(out, &po); err != nil {
|
||||||
|
return fileInfo{}, &DecodeError{Msg: "ffprobe: unreadable answer"}
|
||||||
|
}
|
||||||
|
if len(po.Streams) == 0 {
|
||||||
|
return fileInfo{}, &DecodeError{Msg: "no video stream"}
|
||||||
|
}
|
||||||
|
s := po.Streams[0]
|
||||||
|
for _, x := range po.Streams { // cover art rides along as a video stream; the picture is not the film
|
||||||
|
if x.Disposition.AttachedPic == 0 {
|
||||||
|
s = x
|
||||||
|
break
|
||||||
|
}
|
||||||
|
}
|
||||||
|
fi := fileInfo{index: strconv.Itoa(s.Index), codec: s.Codec, w: s.Width, h: s.Height}
|
||||||
|
fi.duration, _ = strconv.ParseFloat(po.Format.Duration, 64)
|
||||||
|
return fi, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
// run is one ffmpeg: the JPEG on success; on failure what ffmpeg said and the error.
|
||||||
|
// Empty output with a clean exit means the seek found nothing.
|
||||||
|
func run(ctx context.Context, c Caps, a attempt, path string, fi fileInfo, size int) ([]byte, string, error) {
|
||||||
|
args := []string{"-nostdin", "-hide_banner", "-loglevel", "error", "-y"}
|
||||||
|
if a.gpu != nil {
|
||||||
|
args = append(args, "-hwaccel", a.gpu.API, "-hwaccel_device", a.gpu.Device)
|
||||||
|
}
|
||||||
|
if a.lowres > 0 {
|
||||||
|
args = append(args, "-lowres", strconv.Itoa(a.lowres))
|
||||||
|
}
|
||||||
|
if a.seek > 0 {
|
||||||
|
args = append(args, "-noaccurate_seek", "-ss", strconv.FormatFloat(a.seek, 'f', 3, 64))
|
||||||
|
}
|
||||||
|
args = append(args, "-i", path, "-map", "0:"+fi.index, "-frames:v", "1",
|
||||||
|
"-vf", fmt.Sprintf("scale=w=%d:h=%d:force_original_aspect_ratio=decrease:flags=bicubic", size, size),
|
||||||
|
"-f", "image2pipe", "-c:v", "mjpeg", "-q:v", "4", "pipe:1")
|
||||||
|
name, argv := c.FFmpeg, args
|
||||||
|
if c.Nice != "" { // a burst of previews must not crowd out what the machine is for
|
||||||
|
name, argv = c.Nice, append([]string{"-n", "10", c.FFmpeg}, args...)
|
||||||
|
}
|
||||||
|
cmd := exec.CommandContext(ctx, name, argv...)
|
||||||
|
var out, errb bytes.Buffer
|
||||||
|
cmd.Stdout, cmd.Stderr = &out, &errb
|
||||||
|
if err := cmd.Run(); err != nil {
|
||||||
|
msg := lastLine(errb.String())
|
||||||
|
if msg == "" {
|
||||||
|
msg = err.Error()
|
||||||
|
}
|
||||||
|
return nil, msg, err
|
||||||
|
}
|
||||||
|
return out.Bytes(), "", nil
|
||||||
|
}
|
||||||
|
|
||||||
|
// ---- the same, as root ----
|
||||||
|
|
||||||
|
// ProbeRoot is Probe run as root through this binary, for the Files tab's root mode: root
|
||||||
|
// opens every render node, whatever group the daemon's user is in.
|
||||||
|
func ProbeRoot(ctx context.Context, self string) (Caps, error) {
|
||||||
|
full := priv.Wrap(self, "thumb", "--probe")
|
||||||
|
cmd := exec.CommandContext(ctx, full[0], full[1:]...)
|
||||||
|
var errb bytes.Buffer
|
||||||
|
cmd.Stderr = &errb
|
||||||
|
out, err := cmd.Output()
|
||||||
|
if err != nil {
|
||||||
|
if msg := lastLine(errb.String()); msg != "" {
|
||||||
|
return Caps{}, errors.New(msg)
|
||||||
|
}
|
||||||
|
return Caps{}, err
|
||||||
|
}
|
||||||
|
var c Caps
|
||||||
|
if err := json.Unmarshal(out, &c); err != nil {
|
||||||
|
return Caps{}, err
|
||||||
|
}
|
||||||
|
return c, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
type helperOut struct {
|
||||||
|
Engine string `json:"engine,omitempty"`
|
||||||
|
Codec string `json:"codec,omitempty"`
|
||||||
|
Width int `json:"width,omitempty"`
|
||||||
|
Height int `json:"height,omitempty"`
|
||||||
|
JPEG []byte `json:"jpeg,omitempty"`
|
||||||
|
Error string `json:"error,omitempty"`
|
||||||
|
Permanent bool `json:"permanent,omitempty"` // a DecodeError: the file, not the moment
|
||||||
|
}
|
||||||
|
|
||||||
|
// MakeRoot is Make run as root through this binary, with the caps ProbeRoot found.
|
||||||
|
func MakeRoot(ctx context.Context, self string, c Caps, path string, o Opts) (Result, error) {
|
||||||
|
o = o.defaults()
|
||||||
|
caps, _ := json.Marshal(c)
|
||||||
|
full := priv.Wrap(self, "thumb", "--size", strconv.Itoa(o.Size), "--timeout", o.Timeout.String(), path)
|
||||||
|
cmd := exec.CommandContext(ctx, full[0], full[1:]...)
|
||||||
|
cmd.Stdin = bytes.NewReader(caps)
|
||||||
|
var errb bytes.Buffer
|
||||||
|
cmd.Stderr = &errb
|
||||||
|
out, err := cmd.Output()
|
||||||
|
if err != nil {
|
||||||
|
if ctx.Err() != nil {
|
||||||
|
return Result{}, ctx.Err()
|
||||||
|
}
|
||||||
|
if msg := lastLine(errb.String()); msg != "" {
|
||||||
|
return Result{}, errors.New(msg)
|
||||||
|
}
|
||||||
|
return Result{}, err
|
||||||
|
}
|
||||||
|
var h helperOut
|
||||||
|
if err := json.Unmarshal(out, &h); err != nil {
|
||||||
|
return Result{}, err
|
||||||
|
}
|
||||||
|
if h.Error != "" {
|
||||||
|
if h.Permanent {
|
||||||
|
return Result{}, &DecodeError{Msg: h.Error}
|
||||||
|
}
|
||||||
|
return Result{}, errors.New(h.Error)
|
||||||
|
}
|
||||||
|
return Result{JPEG: h.JPEG, Engine: h.Engine, Codec: h.Codec, Width: h.Width, Height: h.Height}, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
// Helper implements `hamlab thumb …`, the root side of the above. `--probe` prints Caps;
|
||||||
|
// `[--size N] [--timeout D] <path>` reads Caps from stdin and prints the preview as JSON.
|
||||||
|
// Returns the exit code.
|
||||||
|
func Helper(args []string, stdin io.Reader, stdout, stderr io.Writer) int {
|
||||||
|
fail := func(err error) int { fmt.Fprintln(stderr, err.Error()); return 1 }
|
||||||
|
ctx := context.Background()
|
||||||
|
o := Opts{}
|
||||||
|
path := ""
|
||||||
|
for i := 0; i < len(args); i++ {
|
||||||
|
switch a := args[i]; {
|
||||||
|
case a == "--probe":
|
||||||
|
return emit(stdout, Probe(ctx), stderr)
|
||||||
|
case a == "--size" && i+1 < len(args):
|
||||||
|
o.Size, _ = strconv.Atoi(args[i+1])
|
||||||
|
i++
|
||||||
|
case a == "--timeout" && i+1 < len(args):
|
||||||
|
o.Timeout, _ = time.ParseDuration(args[i+1])
|
||||||
|
i++
|
||||||
|
case strings.HasPrefix(a, "-"):
|
||||||
|
return fail(errors.New("usage: hamlab thumb --probe | [--size N] [--timeout D] <path>"))
|
||||||
|
default:
|
||||||
|
path = a
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if path == "" || !filepath.IsAbs(path) {
|
||||||
|
return fail(errors.New("usage: hamlab thumb --probe | [--size N] [--timeout D] <absolute path>"))
|
||||||
|
}
|
||||||
|
var c Caps
|
||||||
|
if err := json.NewDecoder(io.LimitReader(stdin, 1<<20)).Decode(&c); err != nil {
|
||||||
|
return fail(fmt.Errorf("caps on stdin: %w", err))
|
||||||
|
}
|
||||||
|
res, err := Make(ctx, c, path, o)
|
||||||
|
if err != nil {
|
||||||
|
var de *DecodeError
|
||||||
|
return emit(stdout, helperOut{Error: err.Error(), Permanent: errors.As(err, &de)}, stderr)
|
||||||
|
}
|
||||||
|
return emit(stdout, helperOut{Engine: res.Engine, Codec: res.Codec, Width: res.Width, Height: res.Height, JPEG: res.JPEG}, stderr)
|
||||||
|
}
|
||||||
|
|
||||||
|
func emit(w io.Writer, v any, stderr io.Writer) int {
|
||||||
|
if err := json.NewEncoder(w).Encode(v); err != nil {
|
||||||
|
fmt.Fprintln(stderr, err.Error())
|
||||||
|
return 1
|
||||||
|
}
|
||||||
|
return 0
|
||||||
|
}
|
||||||
|
|
||||||
|
// ---- small helpers ----
|
||||||
|
|
||||||
|
func lastLine(s string) string {
|
||||||
|
lines := strings.Split(strings.TrimSpace(s), "\n")
|
||||||
|
return strings.TrimSpace(lines[len(lines)-1])
|
||||||
|
}
|
||||||
|
|
||||||
|
func exists(p string) bool { _, err := os.Stat(p); return err == nil }
|
||||||
@@ -0,0 +1,253 @@
|
|||||||
|
package media
|
||||||
|
|
||||||
|
import (
|
||||||
|
"bytes"
|
||||||
|
"context"
|
||||||
|
"image"
|
||||||
|
"image/color"
|
||||||
|
"image/jpeg"
|
||||||
|
"image/png"
|
||||||
|
"os"
|
||||||
|
"os/exec"
|
||||||
|
"path/filepath"
|
||||||
|
"testing"
|
||||||
|
"time"
|
||||||
|
)
|
||||||
|
|
||||||
|
func TestKind(t *testing.T) {
|
||||||
|
for name, want := range map[string]string{"a.JPG": "image", "b.heic": "image", "c.mkv": "video", "GH010098.MP4": "video", "x.OSV": "video", "d.txt": "", "e": "", ".jpg": "image"} {
|
||||||
|
if got := Kind(name); got != want {
|
||||||
|
t.Errorf("Kind(%q) = %q, want %q", name, got, want)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestKeyFollowsTheFile(t *testing.T) {
|
||||||
|
a := Key("/x/a.jpg", 10, 20)
|
||||||
|
if a != Key("/x/a.jpg", 10, 20) || len(a) != 64 {
|
||||||
|
t.Fatal("key is not stable")
|
||||||
|
}
|
||||||
|
for _, b := range []string{Key("/x/b.jpg", 10, 20), Key("/x/a.jpg", 11, 20), Key("/x/a.jpg", 10, 21)} {
|
||||||
|
if b == a {
|
||||||
|
t.Fatal("key ignores part of the file's identity")
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestParseVainfo(t *testing.T) {
|
||||||
|
text := `vainfo: Driver version: Intel iHD driver for Intel(R) Gen Graphics - 25.2.3 ()
|
||||||
|
VAProfileMPEG2Simple : VAEntrypointVLD
|
||||||
|
VAProfileH264Main : VAEntrypointVLD
|
||||||
|
VAProfileH264High : VAEntrypointEncSliceLP
|
||||||
|
VAProfileJPEGBaseline : VAEntrypointVLD
|
||||||
|
VAProfileHEVCMain10 : VAEntrypointVLD
|
||||||
|
VAProfileHEVCMain10 : VAEntrypointEncSlice
|
||||||
|
VAProfileHEVCMain444_12 : VAEntrypointVLD
|
||||||
|
VAProfileVP9Profile2 : VAEntrypointVLD
|
||||||
|
VAProfileAV1Profile0 : VAEntrypointEncSliceLP
|
||||||
|
`
|
||||||
|
dec, enc := parseVainfo(text)
|
||||||
|
same := func(got, want []string) bool {
|
||||||
|
if len(got) != len(want) {
|
||||||
|
return false
|
||||||
|
}
|
||||||
|
for i := range want {
|
||||||
|
if got[i] != want[i] {
|
||||||
|
return false
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return true
|
||||||
|
}
|
||||||
|
if !same(dec, []string{"h264", "hevc", "mjpeg", "mpeg2video", "vp9"}) {
|
||||||
|
t.Fatalf("decodes = %v", dec)
|
||||||
|
}
|
||||||
|
if !same(enc, []string{"av1", "h264", "hevc"}) {
|
||||||
|
t.Fatalf("encodes = %v", enc)
|
||||||
|
}
|
||||||
|
if d, e := parseVainfo(""); d != nil || e != nil {
|
||||||
|
t.Fatal("nothing from nothing")
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestPickAndEngines(t *testing.T) {
|
||||||
|
igpu := GPU{API: "vaapi", Device: "/dev/dri/renderD128", Codecs: []string{"h264", "hevc"}}
|
||||||
|
arc := GPU{API: "vaapi", Device: "/dev/dri/renderD129", Codecs: []string{"av1", "h264", "hevc"}}
|
||||||
|
c := Caps{FFmpeg: "ffmpeg", FFprobe: "ffprobe", GPUs: []GPU{igpu, arc}}
|
||||||
|
if g := c.pick("av1"); g == nil || g.Device != arc.Device {
|
||||||
|
t.Fatal("av1 should go to the card that lists it")
|
||||||
|
}
|
||||||
|
if g := c.pick("h264"); g == nil || g.Device != igpu.Device {
|
||||||
|
t.Fatal("h264 should go to the first card that lists it")
|
||||||
|
}
|
||||||
|
if g := c.pick("vc1"); g != nil {
|
||||||
|
t.Fatal("a codec nobody lists is software")
|
||||||
|
}
|
||||||
|
// a card nothing could describe is tried, after the ones that answered
|
||||||
|
c.GPUs = append(c.GPUs, GPU{API: "cuda", Device: "0"})
|
||||||
|
if g := c.pick("vc1"); g == nil || g.API != "cuda" {
|
||||||
|
t.Fatal("an undescribed card is the guess of last resort")
|
||||||
|
}
|
||||||
|
// videos: GPU then CPU; images: CPU only
|
||||||
|
if e := engines(c, "video", fileInfo{codec: "hevc"}); len(e) != 2 || e[0] == nil || e[1] != nil {
|
||||||
|
t.Fatal("a video tries the GPU, then the CPU")
|
||||||
|
}
|
||||||
|
if e := engines(c, "image", fileInfo{codec: "mjpeg"}); len(e) != 1 || e[0] != nil {
|
||||||
|
t.Fatal("an image is CPU only")
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestAttempts(t *testing.T) {
|
||||||
|
a := attempts(nil, "video", fileInfo{duration: 120}, 256)
|
||||||
|
if len(a) != 2 || a[0].seek != 12 || a[1].seek != 0 {
|
||||||
|
t.Fatalf("video attempts = %+v", a)
|
||||||
|
}
|
||||||
|
if a := attempts(nil, "video", fileInfo{duration: 7200}, 256); a[0].seek != 30 {
|
||||||
|
t.Fatalf("seek capped at 30 s, got %v", a[0].seek)
|
||||||
|
}
|
||||||
|
if a := attempts(nil, "video", fileInfo{duration: 2}, 256); len(a) != 1 || a[0].seek != 0 {
|
||||||
|
t.Fatalf("a short clip is taken from its start, got %+v", a)
|
||||||
|
}
|
||||||
|
if a := attempts(nil, "image", fileInfo{codec: "mjpeg", w: 6000, h: 4000}, 256); len(a) != 2 || a[0].lowres != 2 || a[1].lowres != 0 {
|
||||||
|
t.Fatalf("jpeg attempts = %+v", a)
|
||||||
|
}
|
||||||
|
if a := attempts(nil, "image", fileInfo{codec: "mjpeg", w: 800, h: 600}, 256); len(a) != 1 || a[0].lowres != 0 {
|
||||||
|
t.Fatalf("a small jpeg decodes whole, got %+v", a)
|
||||||
|
}
|
||||||
|
if a := attempts(nil, "image", fileInfo{codec: "png", w: 6000, h: 4000}, 256); len(a) != 1 || a[0].lowres != 0 {
|
||||||
|
t.Fatalf("only jpeg has lowres, got %+v", a)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestCache(t *testing.T) {
|
||||||
|
c := NewCache(filepath.Join(t.TempDir(), "thumbs.db"))
|
||||||
|
k := Key("/a", 1, 2)
|
||||||
|
if _, found, err := c.Get(k); found || err != nil {
|
||||||
|
t.Fatal("an empty cache found something, or complained")
|
||||||
|
}
|
||||||
|
if _, err := os.Stat(c.Path); err == nil {
|
||||||
|
t.Fatal("a miss must not create the database")
|
||||||
|
}
|
||||||
|
if n, b := c.Stats(); n != 0 || b != 0 {
|
||||||
|
t.Fatal("stats of nothing")
|
||||||
|
}
|
||||||
|
if err := c.Put(k, Entry{Path: "/a", Engine: "cpu", Codec: "png", Width: 640, Height: 360, JPEG: []byte("jpeg")}); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
e, found, err := c.Get(k)
|
||||||
|
if err != nil || !found || string(e.JPEG) != "jpeg" || e.Engine != "cpu" || e.Width != 640 || e.Path != "/a" || e.Made.IsZero() {
|
||||||
|
t.Fatalf("put then get: %+v %v %v", e, found, err)
|
||||||
|
}
|
||||||
|
k2 := Key("/b", 1, 2)
|
||||||
|
if err := c.Put(k2, Entry{Path: "/b", Failure: "no video stream"}); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if e, found, _ := c.Get(k2); !found || e.JPEG != nil || e.Failure != "no video stream" {
|
||||||
|
t.Fatal("a failure is remembered with its reason")
|
||||||
|
}
|
||||||
|
if err := c.Put(k, Entry{}); err == nil {
|
||||||
|
t.Fatal("an empty preview must not be kept")
|
||||||
|
}
|
||||||
|
// replacing an entry does not count it twice
|
||||||
|
if err := c.Put(k, Entry{Path: "/a", JPEG: []byte("jpeg2")}); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
n, b := c.Stats()
|
||||||
|
if n != 2 || b == 0 {
|
||||||
|
t.Fatalf("stats = %d entries, %d bytes", n, b)
|
||||||
|
}
|
||||||
|
// survives a reopen
|
||||||
|
if err := c.Close(); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
c2 := NewCache(c.Path)
|
||||||
|
if e, found, _ := c2.Get(k); !found || string(e.JPEG) != "jpeg2" {
|
||||||
|
t.Fatal("not there after reopening")
|
||||||
|
}
|
||||||
|
if n, _ := c2.Stats(); n != 2 {
|
||||||
|
t.Fatal("count lost on reopen")
|
||||||
|
}
|
||||||
|
c2.Close()
|
||||||
|
}
|
||||||
|
|
||||||
|
// The real thing, when ffmpeg is here: a PNG and a generated video each come back as a
|
||||||
|
// JPEG no larger than the asked size, with the aspect kept.
|
||||||
|
func TestMakeWithFFmpeg(t *testing.T) {
|
||||||
|
if _, err := exec.LookPath("ffmpeg"); err != nil {
|
||||||
|
t.Skip("no ffmpeg")
|
||||||
|
}
|
||||||
|
c := Probe(context.Background())
|
||||||
|
if c.Error != "" {
|
||||||
|
t.Skip(c.Error)
|
||||||
|
}
|
||||||
|
dir := t.TempDir()
|
||||||
|
img := image.NewRGBA(image.Rect(0, 0, 640, 360))
|
||||||
|
for y := 0; y < 360; y++ {
|
||||||
|
for x := 0; x < 640; x++ {
|
||||||
|
img.Set(x, y, color.RGBA{uint8(x / 3), uint8(y), 90, 255})
|
||||||
|
}
|
||||||
|
}
|
||||||
|
pngPath := filepath.Join(dir, "a.png")
|
||||||
|
f, _ := os.Create(pngPath)
|
||||||
|
_ = png.Encode(f, img)
|
||||||
|
f.Close()
|
||||||
|
res, err := Make(context.Background(), c, pngPath, Opts{Size: 200})
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
check(t, res, 200, 112, "cpu")
|
||||||
|
|
||||||
|
// a two-second test pattern, with the always-built-in mpeg4 encoder
|
||||||
|
vid := filepath.Join(dir, "v.mp4")
|
||||||
|
out, err := exec.Command("ffmpeg", "-nostdin", "-v", "error", "-f", "lavfi", "-i", "testsrc=size=320x180:rate=10:duration=2", "-c:v", "mpeg4", "-q:v", "5", vid).CombinedOutput()
|
||||||
|
if err != nil {
|
||||||
|
t.Skipf("could not make a test video: %s", out)
|
||||||
|
}
|
||||||
|
res, err = Make(context.Background(), c, vid, Opts{Size: 100})
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
check(t, res, 100, 56, "")
|
||||||
|
if res.Codec != "mpeg4" || res.Width != 320 {
|
||||||
|
t.Fatalf("probe said %s %dx%d", res.Codec, res.Width, res.Height)
|
||||||
|
}
|
||||||
|
|
||||||
|
// not media, and media that isn't
|
||||||
|
if _, err := Make(context.Background(), c, filepath.Join(dir, "x.txt"), Opts{}); err != ErrUnsupported {
|
||||||
|
t.Fatalf("a .txt is unsupported, got %v", err)
|
||||||
|
}
|
||||||
|
bad := filepath.Join(dir, "bad.mp4")
|
||||||
|
_ = os.WriteFile(bad, []byte("not a video at all"), 0o644)
|
||||||
|
_, err = Make(context.Background(), c, bad, Opts{})
|
||||||
|
var de *DecodeError
|
||||||
|
if err == nil || !isDecodeErr(err, &de) {
|
||||||
|
t.Fatalf("garbage should be a DecodeError, got %v", err)
|
||||||
|
}
|
||||||
|
// the clock: a cancelled context is not the file's fault
|
||||||
|
ctx, cancel := context.WithTimeout(context.Background(), time.Nanosecond)
|
||||||
|
defer cancel()
|
||||||
|
if _, err := Make(ctx, c, pngPath, Opts{}); err == nil || isDecodeErr(err, &de) {
|
||||||
|
t.Fatalf("a timeout must not read as a decode failure, got %v", err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func check(t *testing.T, res Result, w, h int, engine string) {
|
||||||
|
t.Helper()
|
||||||
|
im, err := jpeg.Decode(bytes.NewReader(res.JPEG))
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("not a jpeg: %v", err)
|
||||||
|
}
|
||||||
|
if b := im.Bounds(); b.Dx() != w || b.Dy() < h-1 || b.Dy() > h+1 { // the short side rounds either way
|
||||||
|
t.Fatalf("got %dx%d, want %dx%d", b.Dx(), b.Dy(), w, h)
|
||||||
|
}
|
||||||
|
if engine != "" && res.Engine != engine {
|
||||||
|
t.Fatalf("engine %q, want %q", res.Engine, engine)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func isDecodeErr(err error, de **DecodeError) bool {
|
||||||
|
x, ok := err.(*DecodeError)
|
||||||
|
if ok {
|
||||||
|
*de = x
|
||||||
|
}
|
||||||
|
return ok
|
||||||
|
}
|
||||||
@@ -250,6 +250,31 @@ func listGPUs(known []*gpuDev) []*gpuDev {
|
|||||||
return out
|
return out
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// RenderNodeName is the GPU behind a render node ("/dev/dri/renderD128" → "Intel Arc A380"),
|
||||||
|
// named the same way the Processes tab names it; "" when sysfs has nothing for it.
|
||||||
|
func RenderNodeName(node string) string {
|
||||||
|
dev, err := filepath.EvalSymlinks("/sys/class/drm/" + filepath.Base(node) + "/device")
|
||||||
|
if err != nil {
|
||||||
|
return ""
|
||||||
|
}
|
||||||
|
driver := ueventValue(dev+"/uevent", "DRIVER")
|
||||||
|
if driver == "" {
|
||||||
|
driver = filepath.Base(readLink(dev + "/driver"))
|
||||||
|
}
|
||||||
|
return gpuName(dev, ueventValue(dev+"/uevent", "PCI_SLOT_NAME"), driver)
|
||||||
|
}
|
||||||
|
|
||||||
|
// RenderNodeIntegrated reports a render node whose GPU sits on the CPU die (PCI root bus 0,
|
||||||
|
// or no PCI bus at all), as opposed to a card on the bus.
|
||||||
|
func RenderNodeIntegrated(node string) bool {
|
||||||
|
dev, err := filepath.EvalSymlinks("/sys/class/drm/" + filepath.Base(node) + "/device")
|
||||||
|
if err != nil {
|
||||||
|
return true
|
||||||
|
}
|
||||||
|
pci := ueventValue(dev+"/uevent", "PCI_SLOT_NAME")
|
||||||
|
return pci == "" || strings.HasPrefix(pci, "0000:00:")
|
||||||
|
}
|
||||||
|
|
||||||
var gpuVendors = map[string]string{"0x8086": "Intel", "0x1002": "AMD", "0x10de": "NVIDIA", "0x1af4": "Virtio", "0x106b": "Apple"}
|
var gpuVendors = map[string]string{"0x8086": "Intel", "0x1002": "AMD", "0x10de": "NVIDIA", "0x1af4": "Virtio", "0x106b": "Apple"}
|
||||||
|
|
||||||
var (
|
var (
|
||||||
|
|||||||
@@ -66,7 +66,6 @@ type Policy struct {
|
|||||||
KeepMin int `json:"keep_min"`
|
KeepMin int `json:"keep_min"`
|
||||||
Prefix string `json:"prefix"`
|
Prefix string `json:"prefix"`
|
||||||
Enabled bool `json:"enabled"`
|
Enabled bool `json:"enabled"`
|
||||||
|
|
||||||
}
|
}
|
||||||
|
|
||||||
func (p *Policy) prefix() string { return strings.TrimSpace(p.Prefix) }
|
func (p *Policy) prefix() string { return strings.TrimSpace(p.Prefix) }
|
||||||
|
|||||||
@@ -146,7 +146,7 @@ func (m *Manager) prune() {
|
|||||||
}
|
}
|
||||||
var finished []*Task
|
var finished []*Task
|
||||||
for _, t := range m.tasks {
|
for _, t := range m.tasks {
|
||||||
if !t.Running {
|
if !t.running() {
|
||||||
finished = append(finished, t)
|
finished = append(finished, t)
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
@@ -184,13 +184,20 @@ func (m *Manager) RunningOf(kind string) (*Task, bool) {
|
|||||||
m.mu.Lock()
|
m.mu.Lock()
|
||||||
defer m.mu.Unlock()
|
defer m.mu.Unlock()
|
||||||
for _, t := range m.tasks {
|
for _, t := range m.tasks {
|
||||||
if t.Kind == kind && t.Running {
|
if t.Kind == kind && t.running() {
|
||||||
return t, true
|
return t, true
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
return nil, false
|
return nil, false
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// running reads the flag the waiter goroutine clears, under the task's own lock.
|
||||||
|
func (t *Task) running() bool {
|
||||||
|
t.mu.Lock()
|
||||||
|
defer t.mu.Unlock()
|
||||||
|
return t.Running
|
||||||
|
}
|
||||||
|
|
||||||
func (t *Task) Snapshot(withOutput bool) Snapshot {
|
func (t *Task) Snapshot(withOutput bool) Snapshot {
|
||||||
t.mu.Lock()
|
t.mu.Lock()
|
||||||
defer t.mu.Unlock()
|
defer t.mu.Unlock()
|
||||||
|
|||||||
@@ -64,6 +64,9 @@ func New(sudo bool, tm *tasks.Manager) *Manager {
|
|||||||
return m
|
return m
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// PM is the package manager found: apt | dnf | zypper | pacman | apk, or "".
|
||||||
|
func (m *Manager) PM() string { return m.pm }
|
||||||
|
|
||||||
func (m *Manager) sudoArgs(args ...string) []string {
|
func (m *Manager) sudoArgs(args ...string) []string {
|
||||||
if m.sudo {
|
if m.sudo {
|
||||||
return priv.Wrap(args...)
|
return priv.Wrap(args...)
|
||||||
|
|||||||
Generated
+7
@@ -11,6 +11,7 @@
|
|||||||
"@xterm/addon-fit": "^0.10.0",
|
"@xterm/addon-fit": "^0.10.0",
|
||||||
"@xterm/addon-web-links": "^0.11.0",
|
"@xterm/addon-web-links": "^0.11.0",
|
||||||
"@xterm/xterm": "^5.5.0",
|
"@xterm/xterm": "^5.5.0",
|
||||||
|
"hls.js": "^1.7.3",
|
||||||
"qrcode-generator": "^2.0.4",
|
"qrcode-generator": "^2.0.4",
|
||||||
"react": "^18.3.1",
|
"react": "^18.3.1",
|
||||||
"react-dom": "^18.3.1"
|
"react-dom": "^18.3.1"
|
||||||
@@ -1426,6 +1427,12 @@
|
|||||||
"node": ">=6.9.0"
|
"node": ">=6.9.0"
|
||||||
}
|
}
|
||||||
},
|
},
|
||||||
|
"node_modules/hls.js": {
|
||||||
|
"version": "1.7.3",
|
||||||
|
"resolved": "https://registry.npmjs.org/hls.js/-/hls.js-1.7.3.tgz",
|
||||||
|
"integrity": "sha512-MsPlx6yVW4Qv4C7mEVou4/gk/5cN2dVxOTnrNjPSmyH2f0Ln+/UkG10Ax1PE6OvrmTVPKgQYozTtBaPOuCzq7A==",
|
||||||
|
"license": "Apache-2.0"
|
||||||
|
},
|
||||||
"node_modules/js-tokens": {
|
"node_modules/js-tokens": {
|
||||||
"version": "4.0.0",
|
"version": "4.0.0",
|
||||||
"resolved": "https://registry.npmjs.org/js-tokens/-/js-tokens-4.0.0.tgz",
|
"resolved": "https://registry.npmjs.org/js-tokens/-/js-tokens-4.0.0.tgz",
|
||||||
|
|||||||
+2
-1
@@ -7,12 +7,13 @@
|
|||||||
"dev": "vite",
|
"dev": "vite",
|
||||||
"build": "tsc --noEmit && vite build",
|
"build": "tsc --noEmit && vite build",
|
||||||
"icons": "node scripts/icons.mjs",
|
"icons": "node scripts/icons.mjs",
|
||||||
"test": "node scripts/test-graphcols.mjs && node scripts/test-css-imports.mjs"
|
"test": "node scripts/test-graphcols.mjs && node scripts/test-css-imports.mjs && node scripts/test-resolve-path.mjs"
|
||||||
},
|
},
|
||||||
"dependencies": {
|
"dependencies": {
|
||||||
"@xterm/addon-fit": "^0.10.0",
|
"@xterm/addon-fit": "^0.10.0",
|
||||||
"@xterm/addon-web-links": "^0.11.0",
|
"@xterm/addon-web-links": "^0.11.0",
|
||||||
"@xterm/xterm": "^5.5.0",
|
"@xterm/xterm": "^5.5.0",
|
||||||
|
"hls.js": "^1.7.3",
|
||||||
"qrcode-generator": "^2.0.4",
|
"qrcode-generator": "^2.0.4",
|
||||||
"react": "^18.3.1",
|
"react": "^18.3.1",
|
||||||
"react-dom": "^18.3.1"
|
"react-dom": "^18.3.1"
|
||||||
|
|||||||
@@ -0,0 +1,16 @@
|
|||||||
|
// node scripts/test-resolve-path.mjs — the path box resolves like a shell: relative to here,
|
||||||
|
// . and .. applied, trailing slashes dropped, never anything but an absolute path.
|
||||||
|
import { readFileSync } from 'node:fs'
|
||||||
|
const src = readFileSync(new URL('../src/Files.tsx', import.meta.url), 'utf8')
|
||||||
|
const m = src.match(/export function resolvePath[\s\S]*?\n}\n/)
|
||||||
|
if (!m) { console.error('resolvePath not found'); process.exit(1) }
|
||||||
|
const resolvePath = new Function(m[0].replace('export function', 'return function').replace(/: string\[\]/g, '').replace(/: string/g, '') + ' ')()
|
||||||
|
const cases = [
|
||||||
|
['/tank/media', '/x', '/tank/media'], ['/tank/media/', '/x', '/tank/media'], ['', '/here', '/here'], [' ', '/here', '/here'],
|
||||||
|
['sub', '/here', '/here/sub'], ['./sub/', '/here', '/here/sub'], ['..', '/a/b', '/a'], ['../..', '/a/b', '/'], ['../../..', '/a', '/'],
|
||||||
|
['/', '/a', '/'], ['a//b', '/', '/a/b'], ['/tank/./x/../y', '/', '/tank/y'], ['sp ace #1', '/d', '/d/sp ace #1'],
|
||||||
|
]
|
||||||
|
let bad = 0
|
||||||
|
for (const [input, here, want] of cases) { const got = resolvePath(input, here); if (got !== want) { bad++; console.error(`resolvePath(${JSON.stringify(input)}, ${here}) = ${got}, want ${want}`) } }
|
||||||
|
if (bad) process.exit(1)
|
||||||
|
console.log(`resolve path: ok (${cases.length} cases)`)
|
||||||
+229
-15
@@ -1,10 +1,25 @@
|
|||||||
import { DragEvent, useCallback, useEffect, useRef, useState } from 'react'
|
import { DragEvent, useCallback, useEffect, useMemo, useRef, useState } from 'react'
|
||||||
import { api, FileEntry, FileInfo } from './api'
|
import { api, FileEntry, FileInfo, TaskInfo, ThumbStatus } from './api'
|
||||||
import { fmtBytes } from './fmt'
|
import { fmtBytes } from './fmt'
|
||||||
import { Btn, Loading, Spinner, askText, askYes, setUnsaved, toast, useEscape } from './ui'
|
import { Btn, Loading, Spinner, askText, askYes, setUnsaved, toast, useEscape } from './ui'
|
||||||
import { useHashSel } from './hashsel'
|
import { useHashSel } from './hashsel'
|
||||||
|
import TaskOutput from './TaskOutput'
|
||||||
|
import Player from './Player'
|
||||||
|
|
||||||
const join = (dir: string, name: string) => (dir === '/' ? '/' + name : dir + '/' + name)
|
const join = (dir: string, name: string) => (dir === '/' ? '/' + name : dir + '/' + name)
|
||||||
|
// A typed path, made absolute: relative to the folder being looked at, "." and ".." applied,
|
||||||
|
// trailing slashes dropped — what a shell would make of it.
|
||||||
|
export function resolvePath(input: string, here: string): string {
|
||||||
|
const raw = input.trim()
|
||||||
|
if (!raw) return here
|
||||||
|
const parts = (raw.startsWith('/') ? [] : here.split('/')).concat(raw.split('/'))
|
||||||
|
const out: string[] = []
|
||||||
|
for (const p of parts) {
|
||||||
|
if (p === '' || p === '.') continue
|
||||||
|
if (p === '..') out.pop(); else out.push(p)
|
||||||
|
}
|
||||||
|
return '/' + out.join('/')
|
||||||
|
}
|
||||||
const fmtTime = (ms: number) => { const d = new Date(ms); const now = new Date(); return d.toDateString() === now.toDateString() ? d.toLocaleTimeString([], { hour: '2-digit', minute: '2-digit' }) : d.toLocaleDateString(undefined, { year: d.getFullYear() === now.getFullYear() ? undefined : 'numeric', month: 'short', day: '2-digit' }) }
|
const fmtTime = (ms: number) => { const d = new Date(ms); const now = new Date(); return d.toDateString() === now.toDateString() ? d.toLocaleTimeString([], { hour: '2-digit', minute: '2-digit' }) : d.toLocaleDateString(undefined, { year: d.getFullYear() === now.getFullYear() ? undefined : 'numeric', month: 'short', day: '2-digit' }) }
|
||||||
const isImage = (n: string) => /\.(png|jpe?g|gif|webp|svg|bmp|ico|avif)$/i.test(n)
|
const isImage = (n: string) => /\.(png|jpe?g|gif|webp|svg|bmp|ico|avif)$/i.test(n)
|
||||||
const shellQuote = (s: string) => "'" + s.replace(/'/g, "'\\''") + "'"
|
const shellQuote = (s: string) => "'" + s.replace(/'/g, "'\\''") + "'"
|
||||||
@@ -14,7 +29,9 @@ function icon(e: FileEntry): string {
|
|||||||
if (e.type === 'dir') return e.target ? '🔗' : '📁'
|
if (e.type === 'dir') return e.target ? '🔗' : '📁'
|
||||||
if (e.type === 'link') return e.broken ? '⛓️💥' : '🔗'
|
if (e.type === 'link') return e.broken ? '⛓️💥' : '🔗'
|
||||||
if (e.type === 'other') return '⚙️'
|
if (e.type === 'other') return '⚙️'
|
||||||
if (isImage(e.name)) return '🖼️'
|
if (e.preview === 'image' || isImage(e.name)) return '🖼️'
|
||||||
|
if (e.preview === 'video') return '🎞️'
|
||||||
|
if (e.preview === 'audio') return '🎵'
|
||||||
if (/\.(zip|gz|bz2|xz|zst|tar|tgz|7z|rar)$/i.test(e.name)) return '🗜️'
|
if (/\.(zip|gz|bz2|xz|zst|tar|tgz|7z|rar)$/i.test(e.name)) return '🗜️'
|
||||||
if (/\.(sh|bash|zsh|py|go|js|ts|tsx|rb|pl|php|rs|c|h|cpp|java|lua)$/i.test(e.name) || e.mode.includes('x')) return '📜'
|
if (/\.(sh|bash|zsh|py|go|js|ts|tsx|rb|pl|php|rs|c|h|cpp|java|lua)$/i.test(e.name) || e.mode.includes('x')) return '📜'
|
||||||
if (/\.(log|txt|md|json|ya?ml|toml|ini|conf|cfg|env|service|timer)$/i.test(e.name)) return '📄'
|
if (/\.(log|txt|md|json|ya?ml|toml|ini|conf|cfg|env|service|timer)$/i.test(e.name)) return '📄'
|
||||||
@@ -22,6 +39,163 @@ function icon(e: FileEntry): string {
|
|||||||
}
|
}
|
||||||
|
|
||||||
type Sort = 'name' | 'size' | 'mtime'
|
type Sort = 'name' | 'size' | 'mtime'
|
||||||
|
const PREVIEWS_KEY = 'hamlab.files.previews'
|
||||||
|
const byName = (a: FileEntry, b: FileEntry) => a.name.localeCompare(b.name, undefined, { numeric: true, sensitivity: 'base' })
|
||||||
|
// A name is one path segment. ("../x" and "a/b" would quietly act somewhere else.)
|
||||||
|
const okName = (n?: string | null): n is string => { const t = n?.trim(); if (!t) return false; if (t.includes('/') || t === '.' || t === '..') { toast('a name can\'t contain / or be . or .. — use Move… to put it elsewhere', 'err'); return false } return true }
|
||||||
|
|
||||||
|
// ---- move: walk to the destination, then "Move here" ----
|
||||||
|
function MovePicker({ item, from, sudo, hidden, onClose, onMoved }: { item: FileEntry; from: string; sudo: boolean; hidden: boolean; onClose: () => void; onMoved: (task?: TaskInfo) => void }) {
|
||||||
|
const src = join(from, item.name)
|
||||||
|
const isFolder = item.type === 'dir' && !item.target
|
||||||
|
const [hist, setHist] = useState<string[]>([from]) // where the picker has been, for back and forward
|
||||||
|
const [at, setAt] = useState(0)
|
||||||
|
const dir = hist[at]
|
||||||
|
const [entries, setEntries] = useState<FileEntry[] | null>(null)
|
||||||
|
const [err, setErr] = useState('')
|
||||||
|
const [pathInput, setPathInput] = useState(from)
|
||||||
|
const [busy, setBusy] = useState(false)
|
||||||
|
const load = useCallback(async (p: string) => {
|
||||||
|
setEntries(null); setErr('')
|
||||||
|
try { const l = await api.files(p, sudo); setEntries(l.entries) } catch (e) { setErr(e instanceof Error ? e.message : 'failed'); setEntries([]) }
|
||||||
|
}, [sudo])
|
||||||
|
useEffect(() => { setPathInput(dir); load(dir) }, [dir, load])
|
||||||
|
useEscape(onClose)
|
||||||
|
const go = (input: string) => {
|
||||||
|
const p = resolvePath(input, dir)
|
||||||
|
if (p === dir) return
|
||||||
|
setHist((h) => [...h.slice(0, at + 1), p]); setAt(at + 1)
|
||||||
|
}
|
||||||
|
const up = () => { if (dir !== '/') go(dir.slice(0, dir.lastIndexOf('/')) || '/') }
|
||||||
|
const folders = (entries ?? []).filter((e) => e.type === 'dir' && (hidden || !e.name.startsWith('.'))).sort(byName)
|
||||||
|
const crumbs = dir === '/' ? [] : dir.slice(1).split('/')
|
||||||
|
const dest = join(dir, item.name)
|
||||||
|
const why = dir === from ? 'it is already here'
|
||||||
|
: isFolder && (dir === src || dir.startsWith(src + '/')) ? 'a folder can\'t move into itself'
|
||||||
|
: (entries ?? []).some((e) => e.name === item.name) ? `something called ${item.name} is already there` : ''
|
||||||
|
const move = async () => {
|
||||||
|
setBusy(true)
|
||||||
|
try {
|
||||||
|
const t = await api.fileMove(src, dest, sudo)
|
||||||
|
if (t) toast(`${dir} is on another filesystem — copying ${item.name} there, then removing it here`, 'info')
|
||||||
|
else toast(`moved ${item.name} to ${dir}`)
|
||||||
|
onMoved(t)
|
||||||
|
} catch (e) { toast(`move — ${e instanceof Error ? e.message : 'failed'}`, 'err'); setBusy(false) }
|
||||||
|
}
|
||||||
|
const mkdir = async () => {
|
||||||
|
const n = await askText('New folder name')
|
||||||
|
if (!okName(n)) return
|
||||||
|
try { await api.fileMkdir(join(dir, n.trim()), sudo); go(join(dir, n.trim())) } catch (e) { toast(`create folder — ${e instanceof Error ? e.message : 'failed'}`, 'err') }
|
||||||
|
}
|
||||||
|
return (
|
||||||
|
<div className="modal-back" onClick={onClose}>
|
||||||
|
<div className="modal mover" onClick={(e) => e.stopPropagation()}>
|
||||||
|
<h3>Move {isFolder ? 'folder ' : ''}<span className="mono">{item.name}</span></h3>
|
||||||
|
<div className="mover-nav">
|
||||||
|
<button className="icon-btn" title="back" disabled={at === 0} onClick={() => setAt(at - 1)}>‹</button>
|
||||||
|
<button className="icon-btn" title="forward" disabled={at >= hist.length - 1} onClick={() => setAt(at + 1)}>›</button>
|
||||||
|
<button className="icon-btn" title="up one level" disabled={dir === '/'} onClick={up}>↑</button>
|
||||||
|
<form className="crumbs" onSubmit={(e) => { e.preventDefault(); go(pathInput) }}>
|
||||||
|
<button type="button" className="crumb" onClick={() => go('/')}>/</button>
|
||||||
|
{crumbs.map((c, i) => <button type="button" key={i} className="crumb" onClick={() => go('/' + crumbs.slice(0, i + 1).join('/'))}>{c}</button>)}
|
||||||
|
<input className="path-input" value={pathInput} onChange={(e) => setPathInput(e.target.value)} onBlur={() => setPathInput(dir)} spellCheck={false} autoCapitalize="off" autoCorrect="off" />
|
||||||
|
</form>
|
||||||
|
</div>
|
||||||
|
<div className="mover-list">
|
||||||
|
{entries === null && <Loading label="Listing…" />}
|
||||||
|
{err && <div className="error" style={{ margin: 10 }}>{err}</div>}
|
||||||
|
{folders.map((e) => {
|
||||||
|
const self = dir === from && e.name === item.name
|
||||||
|
return (
|
||||||
|
<div key={e.name} className={'mover-row' + (self ? ' self' : '')} onClick={() => { if (!self) go(join(dir, e.name)) }}>
|
||||||
|
<span className="ficon">{e.target ? '🔗' : '📁'}</span>
|
||||||
|
<span className="fname">{e.name}</span>
|
||||||
|
{self && <span className="muted small">this is what's moving</span>}
|
||||||
|
</div>
|
||||||
|
)
|
||||||
|
})}
|
||||||
|
{entries !== null && folders.length === 0 && !err && <div className="muted small" style={{ padding: 12 }}>No folders here — “Move here” puts it in this one.</div>}
|
||||||
|
</div>
|
||||||
|
<div className="modal-foot">
|
||||||
|
<div className="svc-actions">
|
||||||
|
<button className="btn" onClick={mkdir}>+ folder</button>
|
||||||
|
<span className="spacer" />
|
||||||
|
<button className="btn" onClick={onClose}>Cancel</button>
|
||||||
|
<Btn className="btn on" disabled={!!why || entries === null} busy={busy} onClick={move}>Move here</Btn>
|
||||||
|
</div>
|
||||||
|
<div className="muted small mono mover-dest">{why || `→ ${dest}`}</div>
|
||||||
|
</div>
|
||||||
|
</div>
|
||||||
|
</div>
|
||||||
|
)
|
||||||
|
}
|
||||||
|
|
||||||
|
// ---- previews ----
|
||||||
|
|
||||||
|
// Rows ask for their thumbnail only as they scroll near: a browser's own lazy loading
|
||||||
|
// watches the window, and this list scrolls inside a box of its own. One observer for the
|
||||||
|
// list, a callback per row, each fired once.
|
||||||
|
class Near {
|
||||||
|
private obs: IntersectionObserver
|
||||||
|
private cbs = new Map<Element, () => void>()
|
||||||
|
constructor(root: Element) {
|
||||||
|
this.obs = new IntersectionObserver((ents) => {
|
||||||
|
for (const e of ents) if (e.isIntersecting) { const cb = this.cbs.get(e.target); if (cb) { this.forget(e.target); cb() } }
|
||||||
|
}, { root, rootMargin: '400px 0px' })
|
||||||
|
}
|
||||||
|
watch(el: Element, cb: () => void) { this.cbs.set(el, cb); this.obs.observe(el); return () => this.forget(el) }
|
||||||
|
private forget(el: Element) { this.cbs.delete(el); this.obs.unobserve(el) }
|
||||||
|
}
|
||||||
|
|
||||||
|
function Thumb({ url, kind, near, fallback }: { url: string; kind: 'image' | 'video' | 'audio'; near: Near | null; fallback: string }) {
|
||||||
|
const ref = useRef<HTMLSpanElement>(null)
|
||||||
|
const [show, setShow] = useState(false)
|
||||||
|
const [failed, setFailed] = useState('')
|
||||||
|
useEffect(() => { const el = ref.current; if (!el || !near || show) return; return near.watch(el, () => setShow(true)) }, [near, show])
|
||||||
|
useEffect(() => { setFailed('') }, [url]) // a changed file is a new URL, and worth another try
|
||||||
|
const broken = failed === url
|
||||||
|
return (
|
||||||
|
<span ref={ref} className={'fthumb ' + kind} title={broken ? 'no preview for this file' : undefined}>
|
||||||
|
{show && !broken && <img src={url} alt="" decoding="async" onError={() => setFailed(url)} />}
|
||||||
|
{(!show || broken) && <span className="ficon">{fallback}</span>}
|
||||||
|
</span>
|
||||||
|
)
|
||||||
|
}
|
||||||
|
|
||||||
|
// What the previews button says about how they get made here, and the one-tap fixes.
|
||||||
|
function ThumbNote({ st, sudo, onChanged }: { st: ThumbStatus | null; sudo: boolean; onChanged: () => void }) {
|
||||||
|
const [task, setTask] = useState<{ id: string; restart: boolean } | null>(null)
|
||||||
|
const [waiting, setWaiting] = useState(false)
|
||||||
|
// the GPU fix ends by restarting Ham Lab: poll until it answers again, then re-read the status
|
||||||
|
useEffect(() => {
|
||||||
|
if (!waiting) return
|
||||||
|
const t = setInterval(async () => { try { await api.thumbStatus(sudo, true); setWaiting(false); onChanged() } catch { /* restarting */ } }, 3000)
|
||||||
|
return () => clearInterval(t)
|
||||||
|
}, [waiting, sudo, onChanged])
|
||||||
|
if (!st) return null
|
||||||
|
const blocked = (st.unusable ?? []).filter((g) => g.permission)
|
||||||
|
const start = async (label: string, fn: () => Promise<{ id: string }>, restart: boolean) => {
|
||||||
|
try { const t = await fn(); setTask({ id: t.id, restart }) } catch (e) { toast(`${label} — ${e instanceof Error ? e.message : 'failed'}`, 'err') }
|
||||||
|
}
|
||||||
|
return (
|
||||||
|
<>
|
||||||
|
{!st.ok && (
|
||||||
|
<div className="thumb-note notice">
|
||||||
|
<span>Previews need ffmpeg: {st.error || 'it is not installed on this machine'}.</span>
|
||||||
|
{st.install && !task && <Btn className="btn on" onClick={() => start('install', api.thumbInstall, false)}>Install ffmpeg with {st.install.pm}</Btn>}
|
||||||
|
</div>
|
||||||
|
)}
|
||||||
|
{st.ok && blocked.length > 0 && (
|
||||||
|
<div className="thumb-note notice">
|
||||||
|
<span>{blocked.map((g) => g.name || g.device).join(', ')} is here, but Ham Lab runs as {st.user} and can't open {blocked.length > 1 ? 'them' : 'it'} ({blocked[0].reason}). Previews of videos are on the CPU until it can.</span>
|
||||||
|
{st.gpu_fix && !task && !waiting && <Btn className="btn on" confirm="Ham Lab adds itself to the GPU's group and restarts. Open terminals are interrupted. Continue?" onClick={() => start('GPU access', api.thumbGPUAccess, true)}>Allow GPU access</Btn>}
|
||||||
|
{waiting && <span className="muted small"><Spinner /> Ham Lab is restarting; the page reconnects by itself.</span>}
|
||||||
|
</div>
|
||||||
|
)}
|
||||||
|
{task && <TaskOutput id={task.id} onDone={(t) => { const ok = t.exit_code === 0 && !t.error; if (ok && task.restart) setWaiting(true); else if (ok) onChanged(); else toast('failed — the output says where', 'err'); setTask(null) }} />}
|
||||||
|
</>
|
||||||
|
)
|
||||||
|
}
|
||||||
|
|
||||||
// ---- editor / viewer ----
|
// ---- editor / viewer ----
|
||||||
function Viewer({ path, sudo, onClose, onChanged }: { path: string; sudo: boolean; onClose: () => void; onChanged: () => void }) {
|
function Viewer({ path, sudo, onClose, onChanged }: { path: string; sudo: boolean; onClose: () => void; onChanged: () => void }) {
|
||||||
@@ -30,11 +204,12 @@ function Viewer({ path, sudo, onClose, onChanged }: { path: string; sudo: boolea
|
|||||||
const [orig, setOrig] = useState('')
|
const [orig, setOrig] = useState('')
|
||||||
const [err, setErr] = useState('')
|
const [err, setErr] = useState('')
|
||||||
const [saving, setSaving] = useState(false)
|
const [saving, setSaving] = useState(false)
|
||||||
|
const [cantPlay, setCantPlay] = useState(false) // the browser tried the file as it is and gave up
|
||||||
const name = path.split('/').pop() ?? path
|
const name = path.split('/').pop() ?? path
|
||||||
const dirty = text !== null && text !== orig
|
const dirty = text !== null && text !== orig
|
||||||
|
|
||||||
useEffect(() => {
|
useEffect(() => {
|
||||||
setInfo(null); setText(null); setErr('')
|
setInfo(null); setText(null); setErr(''); setCantPlay(false)
|
||||||
let stop = false
|
let stop = false
|
||||||
;(async () => {
|
;(async () => {
|
||||||
try {
|
try {
|
||||||
@@ -92,8 +267,16 @@ function Viewer({ path, sudo, onClose, onChanged }: { path: string; sudo: boolea
|
|||||||
<textarea className="editor file-editor" value={text} spellCheck={false} onChange={(e) => setText(e.target.value)} />
|
<textarea className="editor file-editor" value={text} spellCheck={false} onChange={(e) => setText(e.target.value)} />
|
||||||
)}
|
)}
|
||||||
{info && text === null && isImage(name) && <div className="preview"><img src={api.fileReadURL(path, sudo)} alt={name} /></div>}
|
{info && text === null && isImage(name) && <div className="preview"><img src={api.fileReadURL(path, sudo)} alt={name} /></div>}
|
||||||
{info && text === null && !isImage(name) && !err && (
|
{/* Video: the browser's own decoder first, and a re-encode on the machine when that fails. Audio as it is. */}
|
||||||
<div className="notice">{info.type === 'dir' ? 'This is a directory.' : info.size > MAX_EDIT ? `Too large to edit here (${fmtBytes(info.size)}). Download it instead.` : 'Binary file. Download it instead.'}</div>
|
{info && text === null && !isImage(name) && info.preview === 'video' && <Player path={path} sudo={sudo} name={name} />}
|
||||||
|
{info && text === null && !isImage(name) && info.preview === 'audio' && !cantPlay && (
|
||||||
|
<div className="preview audio"><div className="audio-box"><div className="ficon">🎵</div><div className="mono small">{name}</div><audio controls preload="metadata" src={api.fileReadURL(path, sudo)} onError={() => setCantPlay(true)} /></div></div>
|
||||||
|
)}
|
||||||
|
{info && text === null && !isImage(name) && info.preview === 'audio' && cantPlay && (
|
||||||
|
<div className="notice">This browser can't play {name} as it is — the codec isn't one it decodes. Download it instead.</div>
|
||||||
|
)}
|
||||||
|
{info && text === null && !isImage(name) && !info.preview && !err && (
|
||||||
|
<div className="notice">{info.type === 'dir' ? 'This is a directory.' : info.type === 'other' ? 'This is a device, pipe or socket — nothing to show.' : info.broken ? `This link points at nothing (${info.target}).` : info.size > MAX_EDIT ? `Too large to edit here (${fmtBytes(info.size)}). Download it instead.` : 'Binary file. Download it instead.'}</div>
|
||||||
)}
|
)}
|
||||||
</div>
|
</div>
|
||||||
)
|
)
|
||||||
@@ -114,10 +297,19 @@ export default function Files({ onOpenInTerminal }: { onOpenInTerminal?: (cmd: s
|
|||||||
const [pathInput, setPathInput] = useState(path)
|
const [pathInput, setPathInput] = useState(path)
|
||||||
const [open, setOpen] = useState<string | null>(null)
|
const [open, setOpen] = useState<string | null>(null)
|
||||||
const [menu, setMenu] = useState<string | null>(null)
|
const [menu, setMenu] = useState<string | null>(null)
|
||||||
|
const [moving, setMoving] = useState<FileEntry | null>(null)
|
||||||
|
const [moveTask, setMoveTask] = useState<TaskInfo | null>(null)
|
||||||
const [upload, setUpload] = useState<{ name: string; pct: number; n: number; of: number } | null>(null)
|
const [upload, setUpload] = useState<{ name: string; pct: number; n: number; of: number } | null>(null)
|
||||||
const [drag, setDrag] = useState(false)
|
const [drag, setDrag] = useState(false)
|
||||||
const fileInput = useRef<HTMLInputElement>(null)
|
const fileInput = useRef<HTMLInputElement>(null)
|
||||||
const canSudo = useRef(true)
|
const canSudo = useRef(true)
|
||||||
|
const [previews, setPreviews] = useState<boolean>(() => { try { return localStorage.getItem(PREVIEWS_KEY) === '1' } catch { return false } })
|
||||||
|
const [thumbs, setThumbs] = useState<ThumbStatus | null>(null)
|
||||||
|
const [listEl, setListEl] = useState<HTMLDivElement | null>(null)
|
||||||
|
const near = useMemo(() => (listEl && previews ? new Near(listEl) : null), [listEl, previews])
|
||||||
|
const readThumbs = useCallback((refresh = false) => { api.thumbStatus(sudo, refresh).then(setThumbs).catch((e) => setThumbs({ ok: false, engine: '', summary: '', gpus: [], error: e instanceof Error ? e.message : 'failed', user: '', cache: { path: '', entries: 0, bytes: 0 } })) }, [sudo])
|
||||||
|
useEffect(() => { try { localStorage.setItem(PREVIEWS_KEY, previews ? '1' : '0') } catch { /* private mode */ } }, [previews])
|
||||||
|
useEffect(() => { if (previews) readThumbs(); else setThumbs(null) }, [previews, readThumbs])
|
||||||
|
|
||||||
const load = useCallback(async (p = path, s = sudo) => {
|
const load = useCallback(async (p = path, s = sudo) => {
|
||||||
try {
|
try {
|
||||||
@@ -127,15 +319,22 @@ export default function Files({ onOpenInTerminal }: { onOpenInTerminal?: (cmd: s
|
|||||||
sessionStorage.setItem('hamlab.files.path', p) // only a folder that opened is worth coming back to
|
sessionStorage.setItem('hamlab.files.path', p) // only a folder that opened is worth coming back to
|
||||||
} catch (e) {
|
} catch (e) {
|
||||||
const msg = e instanceof Error ? e.message : 'failed'
|
const msg = e instanceof Error ? e.message : 'failed'
|
||||||
|
if (/not a directory/i.test(msg) && p !== '/') { // a file was typed: open it, from its folder
|
||||||
|
setPath(p.slice(0, p.lastIndexOf('/')) || '/'); setOpen(p); return
|
||||||
|
}
|
||||||
|
if (/no such file/i.test(msg) && p !== '/' && !sessionStorage.getItem('hamlab.files.lost')) { // gone since last time (another machine, a deleted folder)
|
||||||
|
sessionStorage.setItem('hamlab.files.lost', '1'); toast(`${p} isn't here — showing /`, 'info'); setPath('/'); return
|
||||||
|
}
|
||||||
setErr(msg); setEntries([])
|
setErr(msg); setEntries([])
|
||||||
}
|
}
|
||||||
|
sessionStorage.removeItem('hamlab.files.lost')
|
||||||
}, [path, sudo])
|
}, [path, sudo])
|
||||||
|
|
||||||
useEffect(() => { setEntries(null); setPathInput(path); setMenu(null); load(path, sudo); sessionStorage.setItem('hamlab.files.sudo', sudo ? '1' : '0') }, [path, sudo, load])
|
useEffect(() => { setEntries(null); setPathInput(path); setMenu(null); load(path, sudo); sessionStorage.setItem('hamlab.files.sudo', sudo ? '1' : '0') }, [path, sudo, load])
|
||||||
useEffect(() => { const h = () => setMenu(null); window.addEventListener('click', h); return () => window.removeEventListener('click', h) }, [])
|
useEffect(() => { const h = () => setMenu(null); window.addEventListener('click', h); return () => window.removeEventListener('click', h) }, [])
|
||||||
useEscape(() => setMenu(null), menu !== null)
|
useEscape(() => setMenu(null), menu !== null)
|
||||||
|
|
||||||
const go = (p: string) => { setOpen(null); setPath(p || '/') }
|
const go = (p: string) => { setOpen(null); setPath(resolvePath(p, path)) }
|
||||||
const crumbs = path === '/' ? [] : path.slice(1).split('/')
|
const crumbs = path === '/' ? [] : path.slice(1).split('/')
|
||||||
|
|
||||||
const shown = (entries ?? []).filter((e) => (hidden || !e.name.startsWith('.')) && (!q || e.name.toLowerCase().includes(q.toLowerCase())))
|
const shown = (entries ?? []).filter((e) => (hidden || !e.name.startsWith('.')) && (!q || e.name.toLowerCase().includes(q.toLowerCase())))
|
||||||
@@ -150,12 +349,10 @@ export default function Files({ onOpenInTerminal }: { onOpenInTerminal?: (cmd: s
|
|||||||
const act = async (label: string, fn: () => Promise<unknown>): Promise<boolean> => {
|
const act = async (label: string, fn: () => Promise<unknown>): Promise<boolean> => {
|
||||||
try { await fn(); toast(label); await load(); return true } catch (e) { toast(`${label} — ${e instanceof Error ? e.message : 'failed'}`, 'err'); return false }
|
try { await fn(); toast(label); await load(); return true } catch (e) { toast(`${label} — ${e instanceof Error ? e.message : 'failed'}`, 'err'); return false }
|
||||||
}
|
}
|
||||||
// A name is one path segment. ("../x" and "a/b" would quietly act somewhere else.)
|
|
||||||
const okName = (n?: string | null): n is string => { const t = n?.trim(); if (!t) return false; if (t.includes('/') || t === '.' || t === '..') { toast('a name can\'t contain / or be . or .. — use Move… to put it elsewhere', 'err'); return false } return true }
|
|
||||||
const mkdir = async () => { const n = await askText('New folder name'); if (okName(n)) act(`created ${n.trim()}`, () => api.fileMkdir(join(path, n.trim()), sudo)) }
|
const mkdir = async () => { const n = await askText('New folder name'); if (okName(n)) act(`created ${n.trim()}`, () => api.fileMkdir(join(path, n.trim()), sudo)) }
|
||||||
const touch = async () => { const n = await askText('New file name'); if (okName(n) && await act(`created ${n.trim()}`, () => api.fileTouch(join(path, n.trim()), sudo))) setOpen(join(path, n.trim())) } // opened only once it exists
|
const touch = async () => { const n = await askText('New file name'); if (okName(n) && await act(`created ${n.trim()}`, () => api.fileTouch(join(path, n.trim()), sudo))) setOpen(join(path, n.trim())) } // opened only once it exists
|
||||||
const rename = async (e: FileEntry) => { const n = await askText('Rename to', e.name); if (okName(n) && n.trim() !== e.name) act(`renamed to ${n.trim()}`, () => api.fileRename(join(path, e.name), join(path, n.trim()), sudo)) }
|
const rename = async (e: FileEntry) => { const n = await askText('Rename to', e.name); if (okName(n) && n.trim() !== e.name) act(`renamed to ${n.trim()}`, () => api.fileRename(join(path, e.name), join(path, n.trim()), sudo)) }
|
||||||
const move = async (e: FileEntry) => { const n = await askText('Move to (absolute path)', join(path, e.name)); if (n?.trim() && n.trim() !== join(path, e.name)) act(`moved to ${n.trim()}`, () => api.fileRename(join(path, e.name), n.trim(), sudo)) }
|
const move = (e: FileEntry) => setMoving(e)
|
||||||
const chmod = async (e: FileEntry) => { const n = await askText(`Permissions for ${e.name} (octal)`, e.perm); if (n?.trim() && n.trim() !== e.perm) act(`chmod ${n.trim()} ${e.name}`, () => api.fileChmod(join(path, e.name), n.trim(), sudo)) }
|
const chmod = async (e: FileEntry) => { const n = await askText(`Permissions for ${e.name} (octal)`, e.perm); if (n?.trim() && n.trim() !== e.perm) act(`chmod ${n.trim()} ${e.name}`, () => api.fileChmod(join(path, e.name), n.trim(), sudo)) }
|
||||||
const del = async (e: FileEntry) => {
|
const del = async (e: FileEntry) => {
|
||||||
const full = join(path, e.name)
|
const full = join(path, e.name)
|
||||||
@@ -171,6 +368,7 @@ export default function Files({ onOpenInTerminal }: { onOpenInTerminal?: (cmd: s
|
|||||||
// ---- upload ----
|
// ---- upload ----
|
||||||
const uploadFiles = async (files: FileList | File[]) => {
|
const uploadFiles = async (files: FileList | File[]) => {
|
||||||
const list = Array.from(files)
|
const list = Array.from(files)
|
||||||
|
let done = 0
|
||||||
for (let i = 0; i < list.length; i++) {
|
for (let i = 0; i < list.length; i++) {
|
||||||
const f = list[i]
|
const f = list[i]
|
||||||
let overwrite = false
|
let overwrite = false
|
||||||
@@ -181,13 +379,22 @@ export default function Files({ onOpenInTerminal }: { onOpenInTerminal?: (cmd: s
|
|||||||
setUpload({ name: f.name, pct: 0, n: i + 1, of: list.length })
|
setUpload({ name: f.name, pct: 0, n: i + 1, of: list.length })
|
||||||
try {
|
try {
|
||||||
await api.fileUpload(path, f, sudo, (pct) => setUpload({ name: f.name, pct, n: i + 1, of: list.length }), overwrite)
|
await api.fileUpload(path, f, sudo, (pct) => setUpload({ name: f.name, pct, n: i + 1, of: list.length }), overwrite)
|
||||||
|
done++
|
||||||
} catch (e) { toast(`upload ${f.name} — ${e instanceof Error ? e.message : 'failed'}`, 'err'); break }
|
} catch (e) { toast(`upload ${f.name} — ${e instanceof Error ? e.message : 'failed'}`, 'err'); break }
|
||||||
}
|
}
|
||||||
setUpload(null)
|
setUpload(null)
|
||||||
toast(list.length === 1 ? `uploaded ${list[0].name}` : `uploaded ${list.length} files`)
|
if (done === 1) toast(`uploaded ${list.length === 1 ? list[0].name : '1 file'}`)
|
||||||
load()
|
else if (done > 1) toast(`uploaded ${done} files`)
|
||||||
|
if (done > 0) load()
|
||||||
|
}
|
||||||
|
// A dropped folder arrives as a nameless 0-byte "file"; the browser gives no way to read it here.
|
||||||
|
const onDrop = (e: DragEvent) => {
|
||||||
|
e.preventDefault(); setDrag(false)
|
||||||
|
const items = Array.from(e.dataTransfer.items ?? [])
|
||||||
|
const files = Array.from(e.dataTransfer.files).filter((_, i) => !(items[i]?.webkitGetAsEntry?.()?.isDirectory))
|
||||||
|
if (files.length < e.dataTransfer.files.length) toast('folders can\'t be dropped — drop the files inside, or upload a zip', 'err')
|
||||||
|
if (files.length) uploadFiles(files)
|
||||||
}
|
}
|
||||||
const onDrop = (e: DragEvent) => { e.preventDefault(); setDrag(false); if (e.dataTransfer.files.length) uploadFiles(e.dataTransfer.files) }
|
|
||||||
|
|
||||||
if (open) return <div className="files"><Viewer path={open} sudo={sudo} onClose={() => setOpen(null)} onChanged={() => load()} /></div>
|
if (open) return <div className="files"><Viewer path={open} sudo={sudo} onClose={() => setOpen(null)} onChanged={() => load()} /></div>
|
||||||
|
|
||||||
@@ -202,6 +409,7 @@ export default function Files({ onOpenInTerminal }: { onOpenInTerminal?: (cmd: s
|
|||||||
<div className="files-tools">
|
<div className="files-tools">
|
||||||
<button className={'btn' + (sudo ? ' on' : '')} title="browse as root" onClick={() => setSudo(!sudo)}>{sudo ? 'root' : 'sudo'}</button>
|
<button className={'btn' + (sudo ? ' on' : '')} title="browse as root" onClick={() => setSudo(!sudo)}>{sudo ? 'root' : 'sudo'}</button>
|
||||||
<button className={'btn' + (hidden ? ' on' : '')} title="show hidden files" onClick={() => setHidden(!hidden)}>.hidden</button>
|
<button className={'btn' + (hidden ? ' on' : '')} title="show hidden files" onClick={() => setHidden(!hidden)}>.hidden</button>
|
||||||
|
<button className={'btn' + (previews ? ' on' : '')} title={'thumbnails for images and videos' + (thumbs ? ` — ${thumbs.summary}` + (thumbs.cache.entries ? ` · ${thumbs.cache.entries} kept (${fmtBytes(thumbs.cache.bytes)})` : '') : '')} onClick={() => setPreviews(!previews)}>previews</button>
|
||||||
<select className="btn" value={sort} onChange={(e) => setSort(e.target.value as Sort)}><option value="name">name</option><option value="size">size</option><option value="mtime">modified</option></select>
|
<select className="btn" value={sort} onChange={(e) => setSort(e.target.value as Sort)}><option value="name">name</option><option value="size">size</option><option value="mtime">modified</option></select>
|
||||||
<input className="filter" placeholder="filter" value={q} onChange={(e) => setQ(e.target.value)} />
|
<input className="filter" placeholder="filter" value={q} onChange={(e) => setQ(e.target.value)} />
|
||||||
<button className="btn" onClick={mkdir}>+ folder</button>
|
<button className="btn" onClick={mkdir}>+ folder</button>
|
||||||
@@ -213,10 +421,13 @@ export default function Files({ onOpenInTerminal }: { onOpenInTerminal?: (cmd: s
|
|||||||
</div>
|
</div>
|
||||||
</div>
|
</div>
|
||||||
|
|
||||||
|
{previews && <ThumbNote st={thumbs} sudo={sudo} onChanged={() => readThumbs(true)} />}
|
||||||
|
{moveTask && <TaskOutput id={moveTask.id} onDone={(t) => { load(); if (t.exit_code === 0 && !t.error) toast(`${moveTask.title}: done`); else toast(`${moveTask.title}: failed`, 'err') }} />}
|
||||||
|
{moveTask && <div className="thumb-note"><button className="btn" onClick={() => setMoveTask(null)}>Hide output</button></div>}
|
||||||
{upload && <div className="upload-bar"><Spinner /> {upload.of > 1 ? `${upload.n}/${upload.of} · ` : ''}{upload.name} · {upload.pct}%<div className="meter"><div className="fill" style={{ width: `${upload.pct}%` }} /></div></div>}
|
{upload && <div className="upload-bar"><Spinner /> {upload.of > 1 ? `${upload.n}/${upload.of} · ` : ''}{upload.name} · {upload.pct}%<div className="meter"><div className="fill" style={{ width: `${upload.pct}%` }} /></div></div>}
|
||||||
{err && <div className="error" style={{ padding: '6px 12px' }}>{err}{/permission denied/i.test(err) && !sudo && <> · <button className="link" onClick={() => setSudo(true)}>retry as root</button></>}</div>}
|
{err && <div className="error" style={{ padding: '6px 12px' }}>{err}{/permission denied/i.test(err) && !sudo && <> · <button className="link" onClick={() => setSudo(true)}>retry as root</button></>}</div>}
|
||||||
|
|
||||||
<div className="file-list">
|
<div className={'file-list' + (previews ? ' thumbs' : '')} ref={setListEl}>
|
||||||
{entries === null && <Loading label="Listing…" />}
|
{entries === null && <Loading label="Listing…" />}
|
||||||
{path !== '/' && entries !== null && (
|
{path !== '/' && entries !== null && (
|
||||||
<div className="file-row up" onClick={() => go(path.slice(0, path.lastIndexOf('/')) || '/')}><span className="ficon">↩</span><span className="fname">..</span></div>
|
<div className="file-row up" onClick={() => go(path.slice(0, path.lastIndexOf('/')) || '/')}><span className="ficon">↩</span><span className="fname">..</span></div>
|
||||||
@@ -226,7 +437,9 @@ export default function Files({ onOpenInTerminal }: { onOpenInTerminal?: (cmd: s
|
|||||||
const isDir = e.type === 'dir'
|
const isDir = e.type === 'dir'
|
||||||
return (
|
return (
|
||||||
<div key={e.name} className={'file-row' + (e.broken ? ' broken' : '')} onClick={() => (isDir ? go(full) : setOpen(full))}>
|
<div key={e.name} className={'file-row' + (e.broken ? ' broken' : '')} onClick={() => (isDir ? go(full) : setOpen(full))}>
|
||||||
<span className="ficon">{icon(e)}</span>
|
{previews && e.preview && thumbs?.ok
|
||||||
|
? <Thumb url={api.fileThumbURL(full, sudo, e.mtime, e.size)} kind={e.preview} near={near} fallback={icon(e)} />
|
||||||
|
: <span className="ficon">{icon(e)}</span>}
|
||||||
<span className="fname" title={e.target ? `→ ${e.target}` : e.name}>{e.name}{e.target && <span className="muted"> → {e.target}</span>}</span>
|
<span className="fname" title={e.target ? `→ ${e.target}` : e.name}>{e.name}{e.target && <span className="muted"> → {e.target}</span>}</span>
|
||||||
<span className="fmeta num">{isDir ? '' : fmtBytes(e.size)}</span>
|
<span className="fmeta num">{isDir ? '' : fmtBytes(e.size)}</span>
|
||||||
<span className="fmeta num">{fmtTime(e.mtime)}</span>
|
<span className="fmeta num">{fmtTime(e.mtime)}</span>
|
||||||
@@ -249,6 +462,7 @@ export default function Files({ onOpenInTerminal }: { onOpenInTerminal?: (cmd: s
|
|||||||
{entries !== null && shown.length === 0 && !err && <div className="muted" style={{ padding: 12 }}>{q ? 'Nothing matches.' : 'Empty folder.'}{!hidden && entries.some((e) => e.name.startsWith('.')) ? ' (hidden files not shown)' : ''}</div>}
|
{entries !== null && shown.length === 0 && !err && <div className="muted" style={{ padding: 12 }}>{q ? 'Nothing matches.' : 'Empty folder.'}{!hidden && entries.some((e) => e.name.startsWith('.')) ? ' (hidden files not shown)' : ''}</div>}
|
||||||
</div>
|
</div>
|
||||||
{drag && <div className="drop-hint">Drop to upload into {path}</div>}
|
{drag && <div className="drop-hint">Drop to upload into {path}</div>}
|
||||||
|
{moving && <MovePicker item={moving} from={path} sudo={sudo} hidden={hidden} onClose={() => setMoving(null)} onMoved={(t) => { setMoving(null); if (t) setMoveTask(t); load() }} />}
|
||||||
</div>
|
</div>
|
||||||
)
|
)
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -0,0 +1,147 @@
|
|||||||
|
// A video in the Files viewer. First the file as it is, handed to the browser's own
|
||||||
|
// decoder; when that fails — Safari and an MKV, HEVC without a hardware decoder — a
|
||||||
|
// picker: codec and size, and the machine re-encodes it to HLS on its own GPU (or CPU),
|
||||||
|
// which plays natively on Safari and through hls.js everywhere else. The same picker is
|
||||||
|
// one tap away even when the file plays, for a 36 Mbps capture over a slow link.
|
||||||
|
import { useCallback, useEffect, useRef, useState } from 'react'
|
||||||
|
import { api, apiPath, PlayCaps, PlayStatus, PlayStream } from './api'
|
||||||
|
import { Btn, Spinner } from './ui'
|
||||||
|
|
||||||
|
type Mode = 'native' | 'pick' | 'hls'
|
||||||
|
const CODEC_KEY = 'hamlab.play.codec'
|
||||||
|
const QUALITY_KEY = 'hamlab.play.quality'
|
||||||
|
const remember = (k: string, v: string) => { try { localStorage.setItem(k, v) } catch { /* private mode */ } }
|
||||||
|
const recall = (k: string, d: string) => { try { return localStorage.getItem(k) || d } catch { return d } }
|
||||||
|
|
||||||
|
const fmtMbps = (bps?: number) => (bps ? `${(bps / 1e6).toFixed(bps >= 10e6 ? 0 : 1)} Mbps` : '')
|
||||||
|
const describe = (s: PlayStream) => {
|
||||||
|
const v = s.source.video
|
||||||
|
const src = v ? `${v.codec}${v.profile ? ' ' + v.profile : ''} ${v.width}×${v.height}${v.fps ? ` ${Math.round(v.fps)} fps` : ''}${s.source.bit_rate ? ` ${fmtMbps(s.source.bit_rate)}` : ''} in ${s.source.container.split(',')[0]}` : s.source.container
|
||||||
|
const out = s.codec === 'copy' ? 'repackaged' : `${s.codec === 'h264' ? 'H.264' : s.codec.toUpperCase()} ${s.width}×${s.height} ${(s.kbps / 1000).toFixed(s.kbps >= 10000 ? 0 : 1)} Mbps`
|
||||||
|
return `${src} → ${out} on ${s.engine}`
|
||||||
|
}
|
||||||
|
|
||||||
|
export default function Player({ path, sudo, name }: { path: string; sudo: boolean; name: string }) {
|
||||||
|
const [mode, setMode] = useState<Mode>('native')
|
||||||
|
const [why, setWhy] = useState('') // what brought the picker up
|
||||||
|
const [caps, setCaps] = useState<PlayCaps | null>(null)
|
||||||
|
const [codec, setCodec] = useState(() => recall(CODEC_KEY, 'h264'))
|
||||||
|
const [quality, setQuality] = useState(() => recall(QUALITY_KEY, '1080p'))
|
||||||
|
const [stream, setStream] = useState<{ s: PlayStream; url: string } | null>(null)
|
||||||
|
const [status, setStatus] = useState<PlayStatus | null>(null)
|
||||||
|
const [err, setErr] = useState('')
|
||||||
|
const video = useRef<HTMLVideoElement>(null)
|
||||||
|
const streamRef = useRef(stream)
|
||||||
|
streamRef.current = stream
|
||||||
|
|
||||||
|
useEffect(() => { setMode('native'); setWhy(''); setStream(null); setStatus(null); setErr('') }, [path, sudo])
|
||||||
|
useEffect(() => { remember(CODEC_KEY, codec) }, [codec])
|
||||||
|
useEffect(() => { remember(QUALITY_KEY, quality) }, [quality])
|
||||||
|
useEffect(() => { if (mode === 'pick' && !caps) api.playCaps(sudo).then(setCaps).catch((e) => setCaps({ codecs: [], qualities: [], error: e instanceof Error ? e.message : 'failed' })) }, [mode, caps, sudo])
|
||||||
|
|
||||||
|
// a stream is the machine's work: let go of it when this player does — and when the page
|
||||||
|
// itself goes, by a beacon, the one request a browser still sends on its way out
|
||||||
|
const stop = useCallback(() => { const s = streamRef.current; if (s) { api.playStop(s.s.id).catch(() => { /* gone already */ }); setStream(null); setStatus(null) } }, [])
|
||||||
|
useEffect(() => () => stop(), [stop])
|
||||||
|
useEffect(() => {
|
||||||
|
const h = () => { const s = streamRef.current; if (s) navigator.sendBeacon(apiPath(`/api/files/play/${s.s.id}/close`)) }
|
||||||
|
window.addEventListener('pagehide', h)
|
||||||
|
return () => window.removeEventListener('pagehide', h)
|
||||||
|
}, [])
|
||||||
|
|
||||||
|
const start = async () => {
|
||||||
|
setErr('')
|
||||||
|
try {
|
||||||
|
stop()
|
||||||
|
const r = await api.playStart(path, codec, quality, sudo)
|
||||||
|
setStream({ s: r.stream, url: r.url }); setMode('hls')
|
||||||
|
} catch (e) { setErr(e instanceof Error ? e.message : 'failed') }
|
||||||
|
}
|
||||||
|
|
||||||
|
// feed the stream to the element: Safari plays HLS itself, the rest through hls.js
|
||||||
|
useEffect(() => {
|
||||||
|
const v = video.current
|
||||||
|
if (mode !== 'hls' || !stream || !v) return
|
||||||
|
const url = apiPath(stream.url)
|
||||||
|
let hls: { destroy: () => void } | null = null
|
||||||
|
let gone = false
|
||||||
|
;(async () => {
|
||||||
|
if (v.canPlayType('application/vnd.apple.mpegurl')) {
|
||||||
|
v.src = url
|
||||||
|
} else {
|
||||||
|
const { default: Hls } = await import('hls.js')
|
||||||
|
if (gone) return
|
||||||
|
if (!Hls.isSupported()) { setErr('this browser has neither HLS nor MediaSource — nothing can play here'); return }
|
||||||
|
const h = new Hls({ maxBufferLength: 60, backBufferLength: 30 })
|
||||||
|
hls = h
|
||||||
|
h.on(Hls.Events.ERROR, (_e, d) => { if (d.fatal) setErr(`playback failed: ${d.details}`) })
|
||||||
|
h.loadSource(url)
|
||||||
|
h.attachMedia(v)
|
||||||
|
}
|
||||||
|
v.play().catch(() => { /* needs a tap on some phones */ })
|
||||||
|
})()
|
||||||
|
return () => { gone = true; hls?.destroy() }
|
||||||
|
}, [mode, stream])
|
||||||
|
|
||||||
|
// follow the re-encode while it runs
|
||||||
|
useEffect(() => {
|
||||||
|
if (mode !== 'hls' || !stream) return
|
||||||
|
let stopPolling = false
|
||||||
|
const tick = async () => { try { const st = await api.playStatus(stream.s.id); if (!stopPolling) { setStatus(st); if (st.running) setTimeout(tick, 2000) } } catch { /* ended */ } }
|
||||||
|
tick()
|
||||||
|
return () => { stopPolling = true }
|
||||||
|
}, [mode, stream])
|
||||||
|
|
||||||
|
const pick = (reason: string) => { setWhy(reason); setMode('pick') }
|
||||||
|
const codecs = caps?.codecs ?? []
|
||||||
|
|
||||||
|
return (
|
||||||
|
<div className="player">
|
||||||
|
{mode === 'native' && (
|
||||||
|
<>
|
||||||
|
<div className="play-bar"><span>as it is</span><span className="spacer" /><button className="btn" onClick={() => pick('')}>Re-encode…</button></div>
|
||||||
|
<div className="preview"><video controls playsInline preload="metadata" src={api.fileReadURL(path, sudo)} onError={() => pick(`This browser can't play ${name} as it is — the container or codec isn't one it decodes.`)} /></div>
|
||||||
|
</>
|
||||||
|
)}
|
||||||
|
{mode === 'pick' && (
|
||||||
|
<div className="play-pick notice">
|
||||||
|
<div>{why || `Re-encode ${name} on this machine and play that instead.`}</div>
|
||||||
|
{caps?.error && <div className="error" style={{ marginTop: 8 }}>{caps.error}</div>}
|
||||||
|
{!caps && !err && <div className="muted small" style={{ marginTop: 8 }}><Spinner /> asking the machine what it can encode…</div>}
|
||||||
|
{caps && !caps.error && (
|
||||||
|
<div className="row">
|
||||||
|
<label>codec
|
||||||
|
<select value={codec} onChange={(e) => setCodec(e.target.value)}>
|
||||||
|
{codecs.map((c) => <option key={c.id} value={c.id} disabled={!c.engine}>{c.label}{c.engine ? ` — ${c.engine}` : ' — not on this machine'}</option>)}
|
||||||
|
</select>
|
||||||
|
</label>
|
||||||
|
<label>size
|
||||||
|
<select value={quality} onChange={(e) => setQuality(e.target.value)} disabled={codec === 'copy'}>
|
||||||
|
{caps.qualities.map((q) => <option key={q.id} value={q.id}>{q.label}</option>)}
|
||||||
|
</select>
|
||||||
|
</label>
|
||||||
|
<Btn className="btn on" disabled={!codecs.find((c) => c.id === codec)?.engine} onClick={start}>Play</Btn>
|
||||||
|
<a className="btn" href={api.fileDownloadURL(path, sudo)} download={name}>Download instead</a>
|
||||||
|
</div>
|
||||||
|
)}
|
||||||
|
{err && <div className="error" style={{ marginTop: 8 }}>{err}</div>}
|
||||||
|
</div>
|
||||||
|
)}
|
||||||
|
{mode === 'hls' && stream && (
|
||||||
|
<>
|
||||||
|
<div className="play-bar">
|
||||||
|
<span title={describe(stream.s)}>{describe(stream.s)}</span>
|
||||||
|
{status && status.running && <span>· {status.speed ? `${status.speed.toFixed(1)}×` : ''} {stream.s.source.duration ? `${Math.min(100, Math.round(status.progress / stream.s.source.duration * 100))}% through` : `${Math.round(status.progress)} s done`} — seek within what's done</span>}
|
||||||
|
{status?.done && <span>· all of it is ready</span>}
|
||||||
|
{status?.error && <span className="error">· stopped early: {status.error}</span>}
|
||||||
|
<span className="spacer" />
|
||||||
|
<button className="btn" onClick={() => { stop(); setMode('pick'); setWhy('') }}>Change…</button>
|
||||||
|
</div>
|
||||||
|
<div className="preview"><video ref={video} controls autoPlay playsInline /></div>
|
||||||
|
{err && <div className="error" style={{ margin: 12 }}>{err}</div>}
|
||||||
|
</>
|
||||||
|
)}
|
||||||
|
</div>
|
||||||
|
)
|
||||||
|
}
|
||||||
|
|
||||||
+41
-11
@@ -81,7 +81,7 @@ export default function Virtualization() {
|
|||||||
{tab === 'images' && <Images st={st} images={images} instances={instances} onTask={startTask} onNew={(image, vm) => { setTab('instances'); setMaking({ image, vm }) }} onChanged={load} />}
|
{tab === 'images' && <Images st={st} images={images} instances={instances} onTask={startTask} onNew={(image, vm) => { setTab('instances'); setMaking({ image, vm }) }} onChanged={load} />}
|
||||||
{tab === 'library' && <Library st={st} ov={ov} onTask={startTask} onChanged={load} onNewVM={(iso) => setFromIso(iso)} />}
|
{tab === 'library' && <Library st={st} ov={ov} onTask={startTask} onChanged={load} onNewVM={(iso) => setFromIso(iso)} />}
|
||||||
</div>
|
</div>
|
||||||
{tab === 'instances' && (cur ? <Detail i={cur} st={st} onBack={() => setSel('')} onChanged={load} onTask={startTask} /> : <Summary ov={ov} />)}
|
{tab === 'instances' && (cur ? <Detail i={cur} st={st} onBack={() => setSel('')} onChanged={load} onTask={startTask} /> : <Summary ov={ov} onChanged={load} />)}
|
||||||
{making && <NewInstance st={st} ov={ov} preset={making} onClose={() => setMaking(null)} onStarted={(t) => { setMaking(null); startTask(t) }} />}
|
{making && <NewInstance st={st} ov={ov} preset={making} onClose={() => setMaking(null)} onStarted={(t) => { setMaking(null); startTask(t) }} />}
|
||||||
{fromIso !== null && <NewVMFromISO st={st} ov={ov} iso={fromIso} onClose={() => setFromIso(null)} onStarted={(t) => { setFromIso(null); startTask(t); setTab('instances') }} />}
|
{fromIso !== null && <NewVMFromISO st={st} ov={ov} iso={fromIso} onClose={() => setFromIso(null)} onStarted={(t) => { setFromIso(null); startTask(t); setTab('instances') }} />}
|
||||||
</div>
|
</div>
|
||||||
@@ -151,11 +151,11 @@ function InstallDialog({ st, onClose, onStarted }: { st: VirtStatus; onClose: ()
|
|||||||
<div className="notice">Everything goes under <span className="mono">{st.dir}</span>: the pool's backing file, ISOs, images, exports. Where is not a choice; how big is.</div>
|
<div className="notice">Everything goes under <span className="mono">{st.dir}</span>: the pool's backing file, ISOs, images, exports. Where is not a choice; how big is.</div>
|
||||||
<label className="muted">Pool driver
|
<label className="muted">Pool driver
|
||||||
<select value={driver} onChange={(e) => setDriver(e.target.value)}>
|
<select value={driver} onChange={(e) => setDriver(e.target.value)}>
|
||||||
{st.drivers.map((d) => <option key={d} value={d}>{d}{d === 'zfs' ? ' — snapshots and clones are instant' : d === 'btrfs' ? ' — snapshots, on a loop file' : ' — plain files, no snapshots to speak of'}</option>)}
|
{st.drivers.map((d) => <option key={d} value={d}>{d === 'dir' ? 'plain files on this disk — each VM\u2019s disk is its own file, sized on its own; snapshots are copies' : d === 'zfs' ? 'zfs — one pre-sized file holding every disk; snapshots and clones are instant' : 'btrfs — one pre-sized file holding every disk; snapshots are instant'}</option>)}
|
||||||
</select>
|
</select>
|
||||||
</label>
|
</label>
|
||||||
{driver !== 'dir' && (
|
{driver !== 'dir' && (
|
||||||
<label className="muted">Pool size, GiB — a sparse file, grows as used · {fmtBytes(st.disk_free)} free
|
<label className="muted">Ceiling of that file, GiB — sparse, it grows only as disks fill it · {fmtBytes(st.disk_free)} free
|
||||||
<input type="number" min={5} max={Math.floor(st.disk_free / GiB)} value={size} onChange={(e) => setSize(e.target.value)} required />
|
<input type="number" min={5} max={Math.floor(st.disk_free / GiB)} value={size} onChange={(e) => setSize(e.target.value)} required />
|
||||||
</label>
|
</label>
|
||||||
)}
|
)}
|
||||||
@@ -187,7 +187,17 @@ function InstallDialog({ st, onClose, onStarted }: { st: VirtStatus; onClose: ()
|
|||||||
|
|
||||||
// ---- the right-hand pane when nothing is selected: what this machine's Incus is ----
|
// ---- the right-hand pane when nothing is selected: what this machine's Incus is ----
|
||||||
|
|
||||||
function Summary({ ov }: { ov: VirtOverview }) {
|
// A location is a plain-files storage pool at a folder of his choosing.
|
||||||
|
async function addLocation(onDone: (name: string) => void) {
|
||||||
|
const path = await askText('Folder on this machine for instance disks', '/', 'Next')
|
||||||
|
if (!path) return
|
||||||
|
const base = path.split('/').filter(Boolean).pop() ?? 'disks'
|
||||||
|
const name = await askText(`A name for ${path}`, base.toLowerCase().replace(/[^a-z0-9-]+/g, '-').replace(/^-|-$/g, ''), 'Add')
|
||||||
|
if (!name) return
|
||||||
|
try { const r = await api.virtPoolCreate(name, path); toast(`${r.name} added at ${r.path}`); onDone(r.name) } catch (e) { toast(errMsg(e), 'err') }
|
||||||
|
}
|
||||||
|
|
||||||
|
function Summary({ ov, onChanged }: { ov: VirtOverview; onChanged: () => void }) {
|
||||||
const st = ov.status
|
const st = ov.status
|
||||||
return (
|
return (
|
||||||
<div className="svc-detail">
|
<div className="svc-detail">
|
||||||
@@ -198,9 +208,11 @@ function Summary({ ov }: { ov: VirtOverview }) {
|
|||||||
{st.layout && <div><span className="k">Installed</span><span>{when(st.layout.attempted)} by Ham Lab {st.layout.hamlab_version ?? ''}</span></div>}
|
{st.layout && <div><span className="k">Installed</span><span>{when(st.layout.attempted)} by Ham Lab {st.layout.hamlab_version ?? ''}</span></div>}
|
||||||
{!st.layout && <div><span className="k">Installed</span><span>outside Ham Lab — pools and networks below are as found</span></div>}
|
{!st.layout && <div><span className="k">Installed</span><span>outside Ham Lab — pools and networks below are as found</span></div>}
|
||||||
</div>
|
</div>
|
||||||
<h3 className="stack-head">Storage pools</h3>
|
<h3 className="stack-head">Disk locations <span className="muted">· where an instance\u2019s disk can go</span></h3>
|
||||||
<table className="virt-table"><thead><tr><th>Pool</th><th>Driver</th><th>Source</th><th className="num">Used</th><th className="num">Of</th><th className="num">Users</th></tr></thead>
|
<table className="virt-table"><thead><tr><th>Location</th><th>Where</th><th className="num">Used</th><th className="num">Free</th><th className="num">Disks</th><th /></tr></thead>
|
||||||
<tbody>{(ov.pools ?? []).map((p) => <tr key={p.name}><td>{p.name}</td><td>{p.driver}</td><td className="mono small">{p.source}</td><td className="num">{p.total ? fmtBytes(p.used) : '—'}</td><td className="num">{p.total ? fmtBytes(p.total) : '—'}</td><td className="num">{p.used_by}</td></tr>)}</tbody></table>
|
<tbody>{(ov.pools ?? []).map((p) => <tr key={p.name}><td>{p.name}{p.name === ov.default_pool && <span className="badge">default</span>}</td><td>{p.where}<div className="muted small mono">{p.driver} · {p.source}</div></td><td className="num">{p.total ? fmtBytes(p.used) : '—'}</td><td className="num">{p.total ? fmtBytes(p.total - p.used) : '—'}</td><td className="num">{p.used_by}</td>
|
||||||
|
<td className="num">{p.used_by === 0 && p.name !== ov.default_pool && <Btn className="btn danger" confirm={`Remove the location ${p.name}? Nothing uses it; the folder itself is left alone.`} onClick={async () => { try { await api.virtPoolDelete(p.name); toast('removed'); onChanged() } catch (e) { toast(errMsg(e), 'err') } }}>Remove</Btn>}</td></tr>)}</tbody></table>
|
||||||
|
<div className="svc-actions"><Btn className="btn" onClick={() => addLocation(onChanged)}>+ Add a location…</Btn><span className="muted small">a folder on this machine; VM disks become files in it, containers folders</span></div>
|
||||||
<h3 className="stack-head">Networks</h3>
|
<h3 className="stack-head">Networks</h3>
|
||||||
<table className="virt-table"><thead><tr><th>Network</th><th>Type</th><th>Address</th><th>NAT</th><th className="num">Users</th></tr></thead>
|
<table className="virt-table"><thead><tr><th>Network</th><th>Type</th><th>Address</th><th>NAT</th><th className="num">Users</th></tr></thead>
|
||||||
<tbody>{(ov.networks ?? []).filter((n) => n.managed).map((n) => <tr key={n.name}><td>{n.name}</td><td>{n.type}</td><td className="mono small">{n.address ?? '—'}</td><td>{n.nat ? 'yes' : 'no'}</td><td className="num">{n.used_by}</td></tr>)}</tbody></table>
|
<tbody>{(ov.networks ?? []).filter((n) => n.managed).map((n) => <tr key={n.name}><td>{n.name}</td><td>{n.type}</td><td className="mono small">{n.address ?? '—'}</td><td>{n.nat ? 'yes' : 'no'}</td><td className="num">{n.used_by}</td></tr>)}</tbody></table>
|
||||||
@@ -248,7 +260,7 @@ function Detail({ i, st, onBack, onChanged, onTask }: { i: VirtInstance; st: Vir
|
|||||||
{i.description && <div><span className="k">About</span><span>{i.description}</span></div>}
|
{i.description && <div><span className="k">About</span><span>{i.description}</span></div>}
|
||||||
{running && <div><span className="k">Using</span><span>{i.cpu_pct.toFixed(0)}% of a core · {fmtBytes(i.mem_usage)}{i.mem_peak ? ` (peak ${fmtBytes(i.mem_peak)})` : ''}{i.processes > 0 ? ` · ${i.processes} processes` : ''}</span></div>}
|
{running && <div><span className="k">Using</span><span>{i.cpu_pct.toFixed(0)}% of a core · {fmtBytes(i.mem_usage)}{i.mem_peak ? ` (peak ${fmtBytes(i.mem_peak)})` : ''}{i.processes > 0 ? ` · ${i.processes} processes` : ''}</span></div>}
|
||||||
<div><span className="k">Limits</span><span>{i.cpu ? `${i.cpu} CPU` : 'all CPUs'} · {i.memory || 'unlimited memory'}</span></div>
|
<div><span className="k">Limits</span><span>{i.cpu ? `${i.cpu} CPU` : 'all CPUs'} · {i.memory || 'unlimited memory'}</span></div>
|
||||||
{i.disk_total > 0 && <div><span className="k">Disk</span><span>{fmtBytes(i.disk_usage)} of {fmtBytes(i.disk_total)}</span></div>}
|
<div><span className="k">Disk</span><span>{i.disk_total > 0 ? `${fmtBytes(i.disk_usage)} of ${fmtBytes(i.disk_total)}` : i.disk_usage > 0 ? fmtBytes(i.disk_usage) : 'no limit'}{i.devices.root?.pool ? <span className="muted"> · on {i.devices.root.pool}</span> : ''}</span></div>
|
||||||
<div><span className="k">Network</span><span className="mono small">{i.ipv4.length ? i.ipv4.join(' ') : running ? 'no address yet' : '—'}</span></div>
|
<div><span className="k">Network</span><span className="mono small">{i.ipv4.length ? i.ipv4.join(' ') : running ? 'no address yet' : '—'}</span></div>
|
||||||
<div><span className="k">Created</span><span>{when(i.created)}</span></div>
|
<div><span className="k">Created</span><span>{when(i.created)}</span></div>
|
||||||
</div>
|
</div>
|
||||||
@@ -348,6 +360,20 @@ function Snapshots({ i, onChanged }: { i: VirtInstance; onChanged: () => void })
|
|||||||
|
|
||||||
// ---- making one ----
|
// ---- making one ----
|
||||||
|
|
||||||
|
// Where an instance's disk goes. One entry is still a choice shown; a new one can be added here.
|
||||||
|
function PoolPick({ ov, value, onChange }: { ov: VirtOverview; value: string; onChange: (v: string) => void }) {
|
||||||
|
const [extra, setExtra] = useState<{ name: string; where: string }[]>([])
|
||||||
|
const pools = [...(ov.pools ?? []).map((p) => ({ name: p.name, where: p.where + (p.total ? ` · ${fmtBytes(p.total - p.used)} free` : '') })), ...extra.filter((e) => !(ov.pools ?? []).some((p) => p.name === e.name))]
|
||||||
|
return (
|
||||||
|
<label className="muted">Disk location
|
||||||
|
<select value={value} onChange={(e) => { if (e.target.value === '+') addLocation((n) => { setExtra((x) => [...x, { name: n, where: 'just added' }]); onChange(n) }); else onChange(e.target.value) }}>
|
||||||
|
{pools.map((p) => <option key={p.name} value={p.name}>{p.name} — {p.where}</option>)}
|
||||||
|
<option value="+">+ add a location…</option>
|
||||||
|
</select>
|
||||||
|
</label>
|
||||||
|
)
|
||||||
|
}
|
||||||
|
|
||||||
function NewInstance({ st, ov, preset, onClose, onStarted }: { st: VirtStatus; ov: VirtOverview; preset: { image?: string; vm?: boolean }; onClose: () => void; onStarted: (t: TaskInfo) => void }) {
|
function NewInstance({ st, ov, preset, onClose, onStarted }: { st: VirtStatus; ov: VirtOverview; preset: { image?: string; vm?: boolean }; onClose: () => void; onStarted: (t: TaskInfo) => void }) {
|
||||||
const [vm, setVm] = useState(!!preset.vm && st.kvm)
|
const [vm, setVm] = useState(!!preset.vm && st.kvm)
|
||||||
const [image, setImage] = useState(preset.image ?? '')
|
const [image, setImage] = useState(preset.image ?? '')
|
||||||
@@ -355,6 +381,7 @@ function NewInstance({ st, ov, preset, onClose, onStarted }: { st: VirtStatus; o
|
|||||||
const [cpu, setCpu] = useState('')
|
const [cpu, setCpu] = useState('')
|
||||||
const [mem, setMem] = useState('')
|
const [mem, setMem] = useState('')
|
||||||
const [disk, setDisk] = useState('')
|
const [disk, setDisk] = useState('')
|
||||||
|
const [pool, setPool] = useState(ov.default_pool || ov.pools?.[0]?.name || '')
|
||||||
const nets = (ov.networks ?? []).filter((n) => n.managed || n.type === 'bridge')
|
const nets = (ov.networks ?? []).filter((n) => n.managed || n.type === 'bridge')
|
||||||
const [net, setNet] = useState(nets.find((n) => n.name === 'incusbr0')?.name ?? nets[0]?.name ?? '')
|
const [net, setNet] = useState(nets.find((n) => n.name === 'incusbr0')?.name ?? nets[0]?.name ?? '')
|
||||||
const [cloud, setCloud] = useState('')
|
const [cloud, setCloud] = useState('')
|
||||||
@@ -387,7 +414,7 @@ function NewInstance({ st, ov, preset, onClose, onStarted }: { st: VirtStatus; o
|
|||||||
}, [q, local, remote])
|
}, [q, local, remote])
|
||||||
const submit = async (e: FormEvent) => {
|
const submit = async (e: FormEvent) => {
|
||||||
e.preventDefault(); setErr(''); setBusy(true)
|
e.preventDefault(); setErr(''); setBusy(true)
|
||||||
try { onStarted(await api.virtCreate({ name: name.trim(), image: image.trim(), vm, cpu: cpu.trim(), memory: mem.trim(), disk: disk.trim(), network: net, cloud_init: showCloud ? cloud : '', profiles: [], description: desc.trim(), start })) }
|
try { onStarted(await api.virtCreate({ name: name.trim(), image: image.trim(), vm, cpu: cpu.trim(), memory: mem.trim(), disk: disk.trim(), pool, network: net, cloud_init: showCloud ? cloud : '', profiles: [], description: desc.trim(), start })) }
|
||||||
catch (e) { setErr(errMsg(e)); setBusy(false) }
|
catch (e) { setErr(errMsg(e)); setBusy(false) }
|
||||||
}
|
}
|
||||||
return (
|
return (
|
||||||
@@ -418,8 +445,9 @@ function NewInstance({ st, ov, preset, onClose, onStarted }: { st: VirtStatus; o
|
|||||||
<div className="virt-form-row">
|
<div className="virt-form-row">
|
||||||
<label className="muted">CPUs<input className="mono" value={cpu} onChange={(e) => setCpu(e.target.value)} placeholder={vm ? '2' : 'all'} /></label>
|
<label className="muted">CPUs<input className="mono" value={cpu} onChange={(e) => setCpu(e.target.value)} placeholder={vm ? '2' : 'all'} /></label>
|
||||||
<label className="muted">Memory<input className="mono" value={mem} onChange={(e) => setMem(e.target.value)} placeholder={vm ? '2GiB' : 'unlimited'} /></label>
|
<label className="muted">Memory<input className="mono" value={mem} onChange={(e) => setMem(e.target.value)} placeholder={vm ? '2GiB' : 'unlimited'} /></label>
|
||||||
<label className="muted">Root disk<input className="mono" value={disk} onChange={(e) => setDisk(e.target.value)} placeholder={vm ? '20GiB' : 'pool default'} /></label>
|
<label className="muted">Root disk<input className="mono" value={disk} onChange={(e) => setDisk(e.target.value)} placeholder={vm ? '20GiB' : 'no limit'} /></label>
|
||||||
</div>
|
</div>
|
||||||
|
<PoolPick ov={ov} value={pool} onChange={setPool} />
|
||||||
<div className="muted small">{fmtBytes(st.mem_avail)} free on this machine.{vm ? ' A VM’s memory is a reservation.' : ''}</div>
|
<div className="muted small">{fmtBytes(st.mem_avail)} free on this machine.{vm ? ' A VM’s memory is a reservation.' : ''}</div>
|
||||||
<label className="muted">Description — optional<input value={desc} onChange={(e) => setDesc(e.target.value)} maxLength={200} /></label>
|
<label className="muted">Description — optional<input value={desc} onChange={(e) => setDesc(e.target.value)} maxLength={200} /></label>
|
||||||
<label className="muted" style={{ flexDirection: 'row', alignItems: 'center', gap: 8 }}>
|
<label className="muted" style={{ flexDirection: 'row', alignItems: 'center', gap: 8 }}>
|
||||||
@@ -556,6 +584,7 @@ function NewVMFromISO({ st, ov, iso, onClose, onStarted }: { st: VirtStatus; ov:
|
|||||||
const [cpu, setCpu] = useState('2')
|
const [cpu, setCpu] = useState('2')
|
||||||
const [mem, setMem] = useState('2GiB')
|
const [mem, setMem] = useState('2GiB')
|
||||||
const [disk, setDisk] = useState('20GiB')
|
const [disk, setDisk] = useState('20GiB')
|
||||||
|
const [pool, setPool] = useState(ov.default_pool || ov.pools?.[0]?.name || '')
|
||||||
const nets = (ov.networks ?? []).filter((n) => n.managed || n.type === 'bridge')
|
const nets = (ov.networks ?? []).filter((n) => n.managed || n.type === 'bridge')
|
||||||
const [net, setNet] = useState(nets.find((n) => n.name === 'incusbr0')?.name ?? nets[0]?.name ?? '')
|
const [net, setNet] = useState(nets.find((n) => n.name === 'incusbr0')?.name ?? nets[0]?.name ?? '')
|
||||||
const [sb, setSb] = useState(true)
|
const [sb, setSb] = useState(true)
|
||||||
@@ -564,7 +593,7 @@ function NewVMFromISO({ st, ov, iso, onClose, onStarted }: { st: VirtStatus; ov:
|
|||||||
useEscape(onClose)
|
useEscape(onClose)
|
||||||
const submit = async (e: FormEvent) => {
|
const submit = async (e: FormEvent) => {
|
||||||
e.preventDefault(); setErr(''); setBusy(true)
|
e.preventDefault(); setErr(''); setBusy(true)
|
||||||
try { onStarted(await api.virtFromISO({ name: name.trim(), iso: pick, cpu: cpu.trim(), memory: mem.trim(), disk: disk.trim(), network: net, secureboot: sb, start: true })) }
|
try { onStarted(await api.virtFromISO({ name: name.trim(), iso: pick, cpu: cpu.trim(), memory: mem.trim(), disk: disk.trim(), pool, network: net, secureboot: sb, start: true })) }
|
||||||
catch (e) { setErr(errMsg(e)); setBusy(false) }
|
catch (e) { setErr(errMsg(e)); setBusy(false) }
|
||||||
}
|
}
|
||||||
return (
|
return (
|
||||||
@@ -581,6 +610,7 @@ function NewVMFromISO({ st, ov, iso, onClose, onStarted }: { st: VirtStatus; ov:
|
|||||||
<label className="muted">Memory<input className="mono" value={mem} onChange={(e) => setMem(e.target.value)} /></label>
|
<label className="muted">Memory<input className="mono" value={mem} onChange={(e) => setMem(e.target.value)} /></label>
|
||||||
<label className="muted">Disk<input className="mono" value={disk} onChange={(e) => setDisk(e.target.value)} /></label>
|
<label className="muted">Disk<input className="mono" value={disk} onChange={(e) => setDisk(e.target.value)} /></label>
|
||||||
</div>
|
</div>
|
||||||
|
<PoolPick ov={ov} value={pool} onChange={setPool} />
|
||||||
<label className="muted" style={{ flexDirection: 'row', alignItems: 'center', gap: 8 }}>
|
<label className="muted" style={{ flexDirection: 'row', alignItems: 'center', gap: 8 }}>
|
||||||
<input type="checkbox" checked={sb} onChange={(e) => setSb(e.target.checked)} style={{ width: 'auto' }} />Secure Boot — Windows, Debian, Ubuntu and Fedora boot with it; Alpine, Arch, TrueNAS and most others need it off
|
<input type="checkbox" checked={sb} onChange={(e) => setSb(e.target.checked)} style={{ width: 'auto' }} />Secure Boot — Windows, Debian, Ubuntu and Fedora boot with it; Alpine, Arch, TrueNAS and most others need it off
|
||||||
</label>
|
</label>
|
||||||
|
|||||||
+26
-5
@@ -249,7 +249,14 @@ export interface ARCDetail {
|
|||||||
}
|
}
|
||||||
export interface ARCView { detail: ARCDetail; live: ARCLive | null }
|
export interface ARCView { detail: ARCDetail; live: ARCLive | null }
|
||||||
|
|
||||||
export interface FileEntry { name: string; type: 'dir' | 'file' | 'link' | 'other'; size: number; mode: string; perm: string; mtime: number; user: string; group: string; target?: string; broken?: boolean }
|
export interface FileEntry { name: string; type: 'dir' | 'file' | 'link' | 'other'; size: number; mode: string; perm: string; mtime: number; user: string; group: string; target?: string; broken?: boolean; preview?: 'image' | 'video' | 'audio' }
|
||||||
|
export interface PlayCaps { codecs: { id: string; label: string; engine?: string }[]; qualities: { id: string; label: string; height: number }[]; error?: string }
|
||||||
|
export interface PlayTrack { codec: string; profile?: string; pix_fmt?: string; width?: number; height?: number; fps?: number; channels?: number }
|
||||||
|
export interface PlayStream { id: string; codec: string; quality: string; engine: string; width: number; height: number; kbps: number; source: { container: string; duration: number; bit_rate?: number; video?: PlayTrack; audio?: PlayTrack } }
|
||||||
|
export interface PlayStatus { running: boolean; done: boolean; error?: string; progress: number; speed: number; segments: number }
|
||||||
|
export interface ThumbGPU { api: string; device: string; name?: string; codecs?: string[]; reason?: string; permission?: boolean }
|
||||||
|
// How previews get made on the machine: ok with an engine, or why not and what would fix it.
|
||||||
|
export interface ThumbStatus { ok: boolean; engine: 'gpu' | 'cpu' | ''; summary: string; ffmpeg?: string; gpus: ThumbGPU[]; unusable?: ThumbGPU[]; error?: string; install?: { pm: string; package: string }; gpu_fix?: boolean; user: string; cache: { path: string; entries: number; bytes: number } }
|
||||||
export interface FileInfo extends FileEntry { path: string; text: boolean }
|
export interface FileInfo extends FileEntry { path: string; text: boolean }
|
||||||
|
|
||||||
const q = (path: string, sudo: boolean, extra = '') => `path=${encodeURIComponent(path)}${sudo ? '&sudo=1' : ''}${extra}`
|
const q = (path: string, sudo: boolean, extra = '') => `path=${encodeURIComponent(path)}${sudo ? '&sudo=1' : ''}${extra}`
|
||||||
@@ -260,17 +267,17 @@ export interface VirtStatus { state: 'ready' | 'repair' | 'installable' | 'unava
|
|||||||
export interface VirtSnapshot { name: string; created: string; expires?: string; size?: number }
|
export interface VirtSnapshot { name: string; created: string; expires?: string; size?: number }
|
||||||
export interface VirtInstance { name: string; type: 'container' | 'virtual-machine'; status: string; arch: string; description?: string; image: string; os?: string; release?: string; cpu?: string; memory?: string; created: string; last_used: string; ipv4: string[]; snapshots: VirtSnapshot[]; profiles: string[]; ephemeral: boolean; cpu_pct: number; mem_usage: number; mem_peak: number; disk_usage: number; disk_total: number; pid: number; processes: number; config: Record<string, string>; devices: Record<string, Record<string, string>>; schedule?: string; expiry?: string }
|
export interface VirtInstance { name: string; type: 'container' | 'virtual-machine'; status: string; arch: string; description?: string; image: string; os?: string; release?: string; cpu?: string; memory?: string; created: string; last_used: string; ipv4: string[]; snapshots: VirtSnapshot[]; profiles: string[]; ephemeral: boolean; cpu_pct: number; mem_usage: number; mem_peak: number; disk_usage: number; disk_total: number; pid: number; processes: number; config: Record<string, string>; devices: Record<string, Record<string, string>>; schedule?: string; expiry?: string }
|
||||||
export interface VirtImage { fingerprint: string; aliases: string[]; arch: string; type: string; size: number; created: string; last_used: string; description: string; os?: string; release?: string; variant?: string; cached: boolean; source?: string; public: boolean }
|
export interface VirtImage { fingerprint: string; aliases: string[]; arch: string; type: string; size: number; created: string; last_used: string; description: string; os?: string; release?: string; variant?: string; cached: boolean; source?: string; public: boolean }
|
||||||
export interface VirtPool { name: string; driver: string; source: string; used: number; total: number; used_by: number }
|
export interface VirtPool { name: string; driver: string; source: string; where: string; used: number; total: number; used_by: number }
|
||||||
export interface VirtNetwork { name: string; type: string; managed: boolean; address?: string; nat: boolean; used_by: number }
|
export interface VirtNetwork { name: string; type: string; managed: boolean; address?: string; nat: boolean; used_by: number }
|
||||||
export interface LibraryFile { name: string; size: number; modified: string; attached_to?: string[] }
|
export interface LibraryFile { name: string; size: number; modified: string; attached_to?: string[] }
|
||||||
export interface ExportConfig { enabled: boolean; hour: number; minute: number; keep: number; instance_only: boolean; instances?: string[]; updated?: string }
|
export interface ExportConfig { enabled: boolean; hour: number; minute: number; keep: number; instance_only: boolean; instances?: string[]; updated?: string }
|
||||||
export interface ExportRun { at: string; exported: string[]; pruned: string[]; errors?: string[]; seconds: number }
|
export interface ExportRun { at: string; exported: string[]; pruned: string[]; errors?: string[]; seconds: number }
|
||||||
export interface ExportFile { instance: string; name: string; size: number; modified: string }
|
export interface ExportFile { instance: string; name: string; size: number; modified: string }
|
||||||
export interface ExportsView { config: ExportConfig; cron: string; next?: string; last: ExportRun | null; files: ExportFile[]; dir: string; cron_file: string }
|
export interface ExportsView { config: ExportConfig; cron: string; next?: string; last: ExportRun | null; files: ExportFile[]; dir: string; cron_file: string }
|
||||||
export interface FromISO { name: string; iso: string; cpu: string; memory: string; disk: string; network: string; secureboot: boolean; start: boolean }
|
export interface FromISO { name: string; iso: string; cpu: string; memory: string; disk: string; pool: string; network: string; secureboot: boolean; start: boolean }
|
||||||
export interface VirtOverview { status: VirtStatus; error?: string; instances?: VirtInstance[]; images?: VirtImage[]; pools?: VirtPool[]; networks?: VirtNetwork[]; library: { images: LibraryFile[]; isos: LibraryFile[] } }
|
export interface VirtOverview { status: VirtStatus; error?: string; instances?: VirtInstance[]; images?: VirtImage[]; pools?: VirtPool[]; default_pool?: string; networks?: VirtNetwork[]; library: { images: LibraryFile[]; isos: LibraryFile[] } }
|
||||||
export interface VirtInstallOpts { driver: string; pool_size_gib: number; subnet: string; docker_rules: boolean; arc_max_gib: number }
|
export interface VirtInstallOpts { driver: string; pool_size_gib: number; subnet: string; docker_rules: boolean; arc_max_gib: number }
|
||||||
export interface VirtCreate { name: string; image: string; vm: boolean; cpu: string; memory: string; disk: string; network: string; cloud_init: string; profiles: string[]; description: string; start: boolean }
|
export interface VirtCreate { name: string; image: string; vm: boolean; cpu: string; memory: string; disk: string; pool: string; network: string; cloud_init: string; profiles: string[]; description: string; start: boolean }
|
||||||
|
|
||||||
export const api = {
|
export const api = {
|
||||||
procs: () => req<ProcTable>('GET', '/api/procs'),
|
procs: () => req<ProcTable>('GET', '/api/procs'),
|
||||||
@@ -292,6 +299,16 @@ export const api = {
|
|||||||
},
|
},
|
||||||
fileReadURL: (path: string, sudo: boolean) => apiPath(`/api/files/read?${q(path, sudo)}`),
|
fileReadURL: (path: string, sudo: boolean) => apiPath(`/api/files/read?${q(path, sudo)}`),
|
||||||
fileDownloadURL: (path: string, sudo: boolean) => apiPath(`/api/files/download?${q(path, sudo)}`),
|
fileDownloadURL: (path: string, sudo: boolean) => apiPath(`/api/files/download?${q(path, sudo)}`),
|
||||||
|
// v carries the file's mtime and size, so a changed file is a new URL and an unchanged one a cached one
|
||||||
|
fileThumbURL: (path: string, sudo: boolean, mtime: number, size: number) => apiPath(`/api/files/thumb?${q(path, sudo, `&v=${mtime}-${size}`)}`),
|
||||||
|
thumbStatus: (sudo: boolean, refresh = false) => req<ThumbStatus>('GET', `/api/files/thumbs?${sudo ? 'sudo=1&' : ''}${refresh ? 'refresh=1' : ''}`),
|
||||||
|
thumbInstall: () => req<TaskInfo>('POST', '/api/files/thumbs/install'),
|
||||||
|
thumbGPUAccess: () => req<TaskInfo>('POST', '/api/files/thumbs/gpu-access'),
|
||||||
|
// playback of what the browser can't play as it is: an HLS re-encode on the machine
|
||||||
|
playCaps: (sudo: boolean) => req<PlayCaps>('GET', `/api/files/play-caps${sudo ? '?sudo=1' : ''}`),
|
||||||
|
playStart: (path: string, codec: string, quality: string, sudo: boolean) => req<{ stream: PlayStream; url: string }>('POST', `/api/files/play${sudo ? '?sudo=1' : ''}`, { path, codec, quality }),
|
||||||
|
playStatus: (id: string) => req<PlayStatus>('GET', `/api/files/play/${id}`),
|
||||||
|
playStop: (id: string) => req<void>('DELETE', `/api/files/play/${id}`),
|
||||||
fileWrite: async (path: string, text: string, sudo: boolean) => {
|
fileWrite: async (path: string, text: string, sudo: boolean) => {
|
||||||
const r = await fetch(apiPath(`/api/files/write?${q(path, sudo)}`), { method: 'PUT', credentials: 'same-origin', body: text, headers: { 'Content-Type': 'application/octet-stream' } })
|
const r = await fetch(apiPath(`/api/files/write?${q(path, sudo)}`), { method: 'PUT', credentials: 'same-origin', body: text, headers: { 'Content-Type': 'application/octet-stream' } })
|
||||||
if (r.status === 401) throw new Unauthenticated()
|
if (r.status === 401) throw new Unauthenticated()
|
||||||
@@ -308,6 +325,8 @@ export const api = {
|
|||||||
fileMkdir: (path: string, sudo: boolean) => req<void>('POST', `/api/files/mkdir${sudo ? '?sudo=1' : ''}`, { path }),
|
fileMkdir: (path: string, sudo: boolean) => req<void>('POST', `/api/files/mkdir${sudo ? '?sudo=1' : ''}`, { path }),
|
||||||
fileTouch: (path: string, sudo: boolean) => req<void>('POST', `/api/files/touch${sudo ? '?sudo=1' : ''}`, { path }),
|
fileTouch: (path: string, sudo: boolean) => req<void>('POST', `/api/files/touch${sudo ? '?sudo=1' : ''}`, { path }),
|
||||||
fileRename: (path: string, to: string, sudo: boolean) => req<void>('POST', `/api/files/rename${sudo ? '?sudo=1' : ''}`, { path, to }),
|
fileRename: (path: string, to: string, sudo: boolean) => req<void>('POST', `/api/files/rename${sudo ? '?sudo=1' : ''}`, { path, to }),
|
||||||
|
// a rename when the ends share a filesystem (nothing back), otherwise the copy-and-remove runs as a task (the task back)
|
||||||
|
fileMove: (path: string, to: string, sudo: boolean) => req<TaskInfo | undefined>('POST', `/api/files/move${sudo ? '?sudo=1' : ''}`, { path, to }),
|
||||||
fileDelete: (path: string, recursive: boolean, sudo: boolean) => req<void>('POST', `/api/files/delete${sudo ? '?sudo=1' : ''}`, { path, recursive }),
|
fileDelete: (path: string, recursive: boolean, sudo: boolean) => req<void>('POST', `/api/files/delete${sudo ? '?sudo=1' : ''}`, { path, recursive }),
|
||||||
fileChmod: (path: string, mode: string, sudo: boolean) => req<void>('POST', `/api/files/chmod${sudo ? '?sudo=1' : ''}`, { path, mode }),
|
fileChmod: (path: string, mode: string, sudo: boolean) => req<void>('POST', `/api/files/chmod${sudo ? '?sudo=1' : ''}`, { path, mode }),
|
||||||
storage: () => req<{ devices: StorageDevice[]; zfs: ZFSInfo; smartctl: boolean }>('GET', '/api/storage'),
|
storage: () => req<{ devices: StorageDevice[]; zfs: ZFSInfo; smartctl: boolean }>('GET', '/api/storage'),
|
||||||
@@ -456,6 +475,8 @@ export const api = {
|
|||||||
virtExportsRun: (instance = '') => req<TaskInfo>('POST', '/api/virt/exports/run', { instance }),
|
virtExportsRun: (instance = '') => req<TaskInfo>('POST', '/api/virt/exports/run', { instance }),
|
||||||
virtExportDelete: (instance: string, name: string) => req<void>('POST', '/api/virt/exports/delete', { instance, name }),
|
virtExportDelete: (instance: string, name: string) => req<void>('POST', '/api/virt/exports/delete', { instance, name }),
|
||||||
virtSSHKeys: () => req<{ keys: string[]; user: string }>('GET', '/api/virt/sshkeys'),
|
virtSSHKeys: () => req<{ keys: string[]; user: string }>('GET', '/api/virt/sshkeys'),
|
||||||
|
virtPoolCreate: (name: string, path: string) => req<{ name: string; path: string }>('POST', '/api/virt/pools', { name, path }),
|
||||||
|
virtPoolDelete: (name: string) => req<void>('POST', '/api/virt/pools/delete', { name }),
|
||||||
}
|
}
|
||||||
|
|
||||||
export function wsURLPath(path: string): string {
|
export function wsURLPath(path: string): string {
|
||||||
|
|||||||
@@ -364,8 +364,44 @@ h2.section { margin: 0; font-size: 13px; font-weight: 600; color: var(--muted);
|
|||||||
.file-editor { flex: 1; min-height: 0; border: 0; border-radius: 0; resize: none; font-size: 13px; }
|
.file-editor { flex: 1; min-height: 0; border: 0; border-radius: 0; resize: none; font-size: 13px; }
|
||||||
.preview { flex: 1; display: flex; align-items: center; justify-content: center; overflow: auto; padding: 12px; }
|
.preview { flex: 1; display: flex; align-items: center; justify-content: center; overflow: auto; padding: 12px; }
|
||||||
.preview img { max-width: 100%; max-height: 100%; object-fit: contain; }
|
.preview img { max-width: 100%; max-height: 100%; object-fit: contain; }
|
||||||
|
.preview video { max-width: 100%; max-height: 100%; background: #000; border-radius: 4px; outline: none; }
|
||||||
|
.player { flex: 1; min-height: 0; display: flex; flex-direction: column; }
|
||||||
|
.player .preview { flex: 1; min-height: 0; }
|
||||||
|
.play-bar { display: flex; flex-wrap: wrap; align-items: center; gap: 8px; padding: 6px 12px; font-size: 12px; color: var(--muted); border-bottom: 1px solid var(--border); }
|
||||||
|
.play-bar select { padding: 4px 8px; background: var(--bg); border: 1px solid var(--border); border-radius: 5px; color: var(--fg); font-size: 12px; }
|
||||||
|
.play-pick { margin: 12px; }
|
||||||
|
.play-pick .row { display: flex; flex-wrap: wrap; align-items: center; gap: 8px; margin-top: 10px; }
|
||||||
|
.play-pick label { display: flex; align-items: center; gap: 6px; font-size: 12px; color: var(--muted); }
|
||||||
|
.preview.audio { align-items: flex-start; }
|
||||||
|
.audio-box { display: flex; flex-direction: column; align-items: center; gap: 10px; width: min(560px, 100%); margin-top: 8vh; }
|
||||||
|
.audio-box .ficon { font-size: 48px; }
|
||||||
|
.audio-box audio { width: 100%; }
|
||||||
|
/* previews: the icon column becomes a thumbnail box for images and videos */
|
||||||
|
.file-list.thumbs .file-row { grid-template-columns: 72px 1fr 80px 80px 130px 32px; padding-top: 4px; padding-bottom: 4px; min-height: 52px; }
|
||||||
|
.fthumb { position: relative; width: 72px; height: 44px; display: flex; align-items: center; justify-content: center; background: var(--bg3); border-radius: 4px; overflow: hidden; }
|
||||||
|
.fthumb img { max-width: 100%; max-height: 100%; object-fit: contain; display: block; }
|
||||||
|
.fthumb.video::after, .fthumb.audio::after { content: '▶'; position: absolute; right: 2px; bottom: 2px; font-size: 9px; line-height: 1; padding: 2px 3px; border-radius: 3px; background: rgba(0,0,0,.6); color: #fff; }
|
||||||
|
.fthumb.audio::after { content: '♪'; }
|
||||||
|
.fthumb .ficon { font-size: 18px; }
|
||||||
|
.thumb-note { display: flex; flex-wrap: wrap; align-items: center; gap: 8px; margin: 6px 10px 0; }
|
||||||
|
/* move: the folder picker */
|
||||||
|
.modal.mover { width: min(560px, 100%); height: min(76vh, 100%); overflow: hidden; }
|
||||||
|
.mover-nav { display: flex; align-items: center; gap: 2px; }
|
||||||
|
.mover-nav .icon-btn { font-size: 18px; min-width: 30px; line-height: 1; }
|
||||||
|
.mover-nav .icon-btn:disabled { opacity: .3; }
|
||||||
|
.mover-nav .crumbs { flex: 1; min-width: 0; margin-left: 4px; }
|
||||||
|
.modal.mover > .mover-list { flex: 1 1 auto; min-height: 80px; overflow-y: auto; margin: 0 -16px; border-top: 1px solid var(--border); }
|
||||||
|
.mover-row { display: flex; align-items: center; gap: 8px; padding: 9px 16px; cursor: pointer; border-bottom: 1px solid var(--border); min-height: 38px; }
|
||||||
|
.mover-row:hover { background: var(--bg3); }
|
||||||
|
.mover-row.self { opacity: .5; cursor: default; }
|
||||||
|
.mover-row .fname { flex: 1; }
|
||||||
|
.mover-dest { overflow-wrap: anywhere; }
|
||||||
|
.thumb-note .btn { padding-top: 3px; padding-bottom: 3px; }
|
||||||
|
.files-bar .task-out { margin: 6px 10px 0; max-height: 160px; }
|
||||||
@media (max-width: 800px) {
|
@media (max-width: 800px) {
|
||||||
.file-row { grid-template-columns: 24px 1fr 70px 32px; } .file-row .fmeta:nth-child(4), .file-row .perm { display: none; }
|
.file-row { grid-template-columns: 24px 1fr 70px 32px; } .file-row .fmeta:nth-child(4), .file-row .perm { display: none; }
|
||||||
|
.file-list.thumbs .file-row { grid-template-columns: 60px 1fr 70px 32px; min-height: 46px; }
|
||||||
|
.fthumb { width: 60px; height: 38px; }
|
||||||
.path-input { width: 100%; margin: 4px 0 0; }
|
.path-input { width: 100%; margin: 4px 0 0; }
|
||||||
.files-tools .filter { width: 80px; }
|
.files-tools .filter { width: 80px; }
|
||||||
}
|
}
|
||||||
|
|||||||
Reference in New Issue
Block a user