From d200e5dc2e5fbcb2c7fb37064a85e84538758bbc Mon Sep 17 00:00:00 2001 From: Mitchell-Johnson Date: Sun, 11 Oct 2026 17:46:35 -0700 Subject: [PATCH] PLAN.md: re-encoding for the browser, on the machine's GPU Co-Authored-By: Claude Fable 5.1 --- PLAN.md | 53 ++++++++++++++++++++++++++++++++++++++++++++++++++--- 1 file changed, 50 insertions(+), 3 deletions(-) diff --git a/PLAN.md b/PLAN.md index ffd1960..3ba644f 100644 --- a/PLAN.md +++ b/PLAN.md @@ -724,9 +724,56 @@ something long in one, switch away and back, refresh — it's all still there. plain streaming path, as does the editor's capped read. Audio became a preview kind at the same time (`thumbs.Kind` → `audio`): its cover art, where it has any, is the row's thumbnail, with a ♪ badge. - - *Not done, on purpose:* no transcoding or HLS for what the browser lacks (that is the - opposite of "keep it simple"; Jellyfin exists for it), no playlist or next/previous, no - subtitles. + - *Not done, on purpose:* no playlist or next/previous, no subtitles. (Transcoding came + the same evening after all — the next entry — because Safari turned out not to play + the gaming captures.) +- **Re-encoding for the browser, on the machine's GPU** — **shipped 2026-10-11** + (v0.1.0-212, `internal/media/{encode,hls,ffrun,inspect}.go`, `internal/api/play.go`, + `Player.tsx`, `hls.js`). Asked for when Safari would not play the gaming captures: + "add GPU transcoding to a supported codec when the client browser can't play it. Each + machine should use its own GPU (if available) and transcode to probably H.264. Maybe we + need a quality selector, and a codec selector, up to you." What Safari actually refuses + is the **MKV container** — the captures checked are H.264 High 4:2:0, 18–36 Mbps — so + the picker has a "repackaged" choice beside the codecs, and a 36 Mbps capture over + WireGuard is the other reason to re-encode at all. + - **The viewer tries the file as it is first**, as before. When the browser gives up, or + on **Re-encode…** in the bar, a picker: **codec** — H.264, HEVC, AV1, or the original + codec repackaged — each labelled with the engine this machine would use (or greyed as + "not on this machine"), and **size** — original, 1080p, 720p, 480p; the choice is + remembered. Play starts the re-encode and the player follows it: "h264 High 2292×960 + 60 fps 36 Mbps in matroska → H.264 2292×960 5.0 Mbps on Intel Arc A380 (vaapi) · 7.3× · + 34% through — seek within what's done", then "all of it is ready". **Change…** drops + the stream and goes back to the picker. + - **HLS, made on the machine in the switcher.** `POST /api/files/play` inspects the file + (ffprobe, as root in root mode) and runs ffmpeg into an *event* playlist of 4 s + fragmented-MP4 segments under `/transcode//`, answering once the first + segment exists (0.5–1 s on `grid`); the player reads the playlist and segments from + `/api/files/play//…` (only those three names are served), Safari natively and + everything else through hls.js, loaded only then. ffmpeg runs ahead at the engine's + pace, so the seekable range grows and the whole file is through long before it is + watched. Keyframes are forced every 4 s so segments cut cleanly; sound is AAC stereo + unless it already is AAC; HDR is tone-mapped (`tonemap_vaapi`, or zscale/hable on the + CPU). Nothing is kept: the same request within ten minutes reuses the stream, the + player closes it when it goes (and the "Change…" path does), the sweeper closes what + nobody touched for ten minutes, and the folder is emptied when the daemon starts. At + most three at once per machine (429 with the reason). Root mode runs ffmpeg as root + through `hamlab run-ffmpeg`, which ends when the daemon lets go of its stdin — so a + stream cannot outlive the console. + - **Engines are probed, never assumed**, the same probe the previews use, now also + reading each node's encode profiles from vainfo and ffmpeg's own encoder list. The + recipes for a codec are tried best first until one produces a first segment: a card on + the bus before the integrated GPU (the A380 before the UHD 750), decoding on the same + card so frames never leave it, then decoding on the CPU and uploading (a source the + card cannot decode — AV1 on `grid`), then libx264 / libx265 / SVT-AV1. The size is + never larger than the source, and the bit rate follows the output size (2/4/8/12/20 + Mbps for 480p/720p/1080p/1440p/2160p, +25 % at 50 fps and up, ×0.6 for HEVC and AV1). + Measured on `grid` with a 2292×960 60 fps capture: H.264 7.3× realtime, HEVC 720p + 11.7×, AV1 720p 13×, all on the Arc. `core` and `hamtv` re-encode on the CPU until + *Allow GPU access* is pressed (libx264 at 9.4× for a 720p→480p clip on `core`); the Mac + and `gate` need ffmpeg first, which the previews note offers to install. + - *Not done, on purpose:* no restart-from-a-seek beyond what is encoded (the encode is + faster than watching; a seek far ahead waits), no bit-rate ladder or adaptive + switching, no subtitles, no keeping a finished re-encode for next time. - **Docker read-only view** — `docker ps`/logs; dockhand stays the source of truth. - **Notifications** — push on service failure / SMART warning / UPS event. - **Terminal extras** — search, copy-mode, font/theme, split panes, upload via