Compare commits
2
Commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
51c718e8e3 | ||
|
|
f8c9479e93 |
@@ -625,6 +625,57 @@ something long in one, switch away and back, refresh — it's all still there.
|
||||
so both modes share one implementation and one set of rules (absolute paths only;
|
||||
`/`, `/etc`, `/usr`, … are never deleted or moved whole). Every write is logged with
|
||||
the caller's IP.
|
||||
- **Previews on the Files tab** — **shipped 2026-10-11** (v0.1.0-200, `internal/thumbs`,
|
||||
`internal/api/thumbs.go`, `Thumb`/`ThumbNote` in `Files.tsx`). Asked for as a button that
|
||||
toggles a file preview, "obviously not for everything", cached, "use whatever GPU is on the
|
||||
machine we're looking at, and fall back to CPU", "a single small frame preview so it's
|
||||
fast". A **previews** button beside `.hidden` turns the icon column into a 256 px
|
||||
thumbnail for every image and video row (the listing says which: `preview: image|video`
|
||||
per entry, decided by extension in one place, `thumbs.Kind`), loaded as rows scroll within
|
||||
400 px of view — the list scrolls in a box of its own, so it has its own
|
||||
IntersectionObserver rather than the browser's window-based lazy loading. A video thumb
|
||||
carries a ▶ badge. The setting is in localStorage and follows the browser.
|
||||
- **ffmpeg is the one engine**, for images and video alike; `GET /api/files/thumb` is one
|
||||
ffprobe and one ffmpeg: `-ss` a tenth of the way in (30 s at most, `-noaccurate_seek`
|
||||
so one keyframe is decoded), the first real video stream (cover art is skipped), one
|
||||
frame, `scale` to fit 256, MJPEG q4 to a pipe. JPEGs decode at a reduced DCT size
|
||||
(`-lowres`, enough halvings to stay at twice the preview) — a 24 MP photo is read as
|
||||
1.5 MP. Thumbnails are the only thing Ham Lab asks a GPU to do, and it asks the way the
|
||||
Processes tab reads one: **probed, never assumed.** The probe looks ffmpeg and ffprobe
|
||||
up, opens every `/dev/dri/renderD*`, pushes a synthetic frame through each with VAAPI,
|
||||
and reads `vainfo`'s decode profiles per node where it exists; CUDA is tried the same
|
||||
way when `/dev/nvidiactl` is there (untested: nothing here has one). A video goes to the
|
||||
first GPU listing its codec; ffmpeg's own software fallback sits behind that for a
|
||||
profile the list did not distinguish, and a CPU retry behind a hardware run that fails
|
||||
outright. Images are CPU only — one frame through libjpeg at a quarter size beats a
|
||||
round trip through a GPU. Measured on `grid`: 1080p h264 0.4 s on the UHD 750 against
|
||||
0.7 s on the CPU; the 8K Osmo HEVC 1.0 s against 2.9 s. `grid`'s two GPUs are listed in
|
||||
node order and both decode the same set (h264, hevc, vp9, mjpeg, mpeg2 — **no AV1 on
|
||||
either**, per vainfo, so AV1 is dav1d on the CPU).
|
||||
- **Cached under the machine's data dir**, `<data dir>/thumbs/<xx>/<sha256>.jpg`, keyed by
|
||||
path, size and mtime — a changed file gets a new preview, an unchanged one is never
|
||||
decoded twice, and the URL carries `v=<mtime>-<size>` so the browser's own cache is
|
||||
right too (7 days, ETag, 304). A file ffmpeg could not decode is remembered beside the
|
||||
JPEGs as `.err` with ffmpeg's reason, answered as 422, and retried only when the file
|
||||
changes; a timeout is not remembered. Nothing expires: a preview is a few KB, and only
|
||||
files somebody scrolled past are ever made. At most 2–4 ffmpegs at once per machine
|
||||
(half the cores, clamped), one per file however many rows ask, and a preview's work
|
||||
runs on a clock of its own so the row that asked first leaving does not cancel it for
|
||||
the rest. Each ffmpeg runs under `nice -n 10`. `hamlet`'s backup job skips the folder.
|
||||
- **Root mode makes them as root** (`hamlab thumb`, the sibling of `hamlab fs`, caps on
|
||||
stdin, JSON out), so a file only root can read gets a preview, and the probe runs as
|
||||
root too — root opens every render node whatever groups the daemon has. Which matters:
|
||||
a render node is `root:render 0660`, and on `core` and `hamtv` the `core` user is not in
|
||||
`render`, so there the daemon's own previews are CPU and the tab says so, naming the GPU
|
||||
and the owner, with an **Allow GPU access** button (confirms, then a task writes
|
||||
`hamlab.service.d/gpu.conf` adding `render`/`video` as supplementary groups and
|
||||
restarts Ham Lab three seconds later, the shape the Incus install uses for
|
||||
`incus-admin`). Not pressed anywhere yet. Where ffmpeg is missing (the Mac mini, `gate`)
|
||||
the note offers **Install ffmpeg with dnf/apk** through the Updates machinery
|
||||
(`ffmpeg-free` on Fedora, `ffmpeg` elsewhere); the status re-probes when the task ends.
|
||||
- *Not done, on purpose:* no cap on the cache and no clear button (not asked; it only
|
||||
holds what was looked at); no tonemapping for HDR sources (a 10-bit HEVC frame comes out
|
||||
flat); no preview in the viewer for a video (it still says "binary, download it").
|
||||
- **Docker read-only view** — `docker ps`/logs; dockhand stays the source of truth.
|
||||
- **Notifications** — push on service failure / SMART warning / UPS event.
|
||||
- **Terminal extras** — search, copy-mode, font/theme, split panes, upload via
|
||||
@@ -732,8 +783,9 @@ what it has moved.
|
||||
- **Resumable uploads.** copyparty's up2k chunks a file, survives a dropped connection
|
||||
and skips what is already there; Files streams one PUT, so a phone upload that drops at
|
||||
90% starts over.
|
||||
- **A gallery.** Thumbnails for a folder of images and video, and play audio/video in
|
||||
place — `/tank/media` and `/tank/immich` are most of what is browsed.
|
||||
- **A gallery.** ~~Thumbnails for a folder of images and video~~ (the **previews** toggle,
|
||||
2026-10-11 — see the shipped entry above; a grid view is not built), and play
|
||||
audio/video in place — `/tank/media` and `/tank/immich` are most of what is browsed.
|
||||
- **Search** below the current folder by name, since `find` in a terminal is the fallback
|
||||
today.
|
||||
- *Decide when built:* whether share links (copyparty's public per-path links) are wanted
|
||||
|
||||
@@ -41,6 +41,7 @@ import (
|
||||
"hamlab/internal/store"
|
||||
"hamlab/internal/tasks"
|
||||
hterm "hamlab/internal/term"
|
||||
"hamlab/internal/thumbs"
|
||||
"hamlab/internal/tlsauto"
|
||||
"hamlab/internal/updates"
|
||||
"hamlab/internal/web"
|
||||
@@ -82,6 +83,8 @@ func main() {
|
||||
switch cmd { // commands that don't read the config
|
||||
case "fs": // privileged helper for the Files tab (run as root by the daemon)
|
||||
os.Exit(files.Helper(flag.Args()[1:], os.Stdin, os.Stdout, os.Stderr))
|
||||
case "thumb": // privileged helper for the Files tab's previews in root mode (run as root by the daemon)
|
||||
os.Exit(thumbs.Helper(flag.Args()[1:], os.Stdin, os.Stdout, os.Stderr))
|
||||
case "space-walk": // helper for the Space tab: measure mountpoints, stream the tree back
|
||||
os.Exit(space.Helper(flag.Args()[1:], os.Stdin, os.Stdout, os.Stderr))
|
||||
case "drm-clients": // privileged helper for the GPU readings: who has the GPU open (run as root by the daemon)
|
||||
|
||||
@@ -79,6 +79,7 @@ type Server struct {
|
||||
virt *incus.Manager
|
||||
fsLocal files.FS
|
||||
fsSudo files.FS // nil without passwordless sudo
|
||||
thumbs *thumbSvc
|
||||
nodes *nodes.Manager
|
||||
fingerprint string
|
||||
version string
|
||||
@@ -99,6 +100,7 @@ func New(cfg *config.Config, sessions *auth.Sessions, mgr *term.Manager, col *me
|
||||
self = "hamlab"
|
||||
}
|
||||
s.space = space.New(self, filepath.Join(cfg.DataDir(), "space"))
|
||||
s.thumbs = newThumbSvc(cfg.DataDir(), self)
|
||||
s.hostName() // read it once now so the first request doesn't have to
|
||||
if u, err := user.Current(); err == nil && u.Username != "" {
|
||||
s.user = u.Username
|
||||
@@ -301,6 +303,10 @@ func (s *Server) routes() *http.ServeMux {
|
||||
m.HandleFunc("POST /api/files/rename", s.auth(s.filesRename))
|
||||
m.HandleFunc("POST /api/files/delete", s.auth(s.filesDelete))
|
||||
m.HandleFunc("POST /api/files/chmod", s.auth(s.filesChmod))
|
||||
m.HandleFunc("GET /api/files/thumb", s.auth(s.filesThumb))
|
||||
m.HandleFunc("GET /api/files/thumbs", s.auth(s.thumbsStatus))
|
||||
m.HandleFunc("POST /api/files/thumbs/install", s.auth(s.thumbsInstall))
|
||||
m.HandleFunc("POST /api/files/thumbs/gpu-access", s.auth(s.thumbsGPUAccess))
|
||||
m.HandleFunc("GET /api/hardware", s.auth(s.hardwareReport))
|
||||
m.HandleFunc("GET /api/network", s.auth(s.networkOverview))
|
||||
m.HandleFunc("GET /api/vpn", s.auth(s.vpnReport))
|
||||
|
||||
@@ -0,0 +1,400 @@
|
||||
package api
|
||||
|
||||
// Previews for the Files tab: GET /api/files/thumb hands back one small JPEG of an image or
|
||||
// a video, made by internal/thumbs and kept under <data dir>/thumbs/. The work is bounded
|
||||
// two ways — a handful of ffmpegs at once, and one per file however many rows ask for it —
|
||||
// because a folder of a thousand photos asks for every row the moment previews go on.
|
||||
//
|
||||
// Root mode (?sudo=1) makes the preview through the same root helper the file reads use,
|
||||
// so a file only root can read gets a preview too, and the probe runs as root as well: root
|
||||
// opens every render node, whatever groups the daemon's user has.
|
||||
|
||||
import (
|
||||
"context"
|
||||
"errors"
|
||||
"log"
|
||||
"net/http"
|
||||
"os"
|
||||
"os/exec"
|
||||
"path/filepath"
|
||||
"runtime"
|
||||
"strconv"
|
||||
"strings"
|
||||
"sync"
|
||||
"time"
|
||||
|
||||
"hamlab/internal/priv"
|
||||
"hamlab/internal/thumbs"
|
||||
)
|
||||
|
||||
const thumbSize = 256
|
||||
const thumbTimeout = 30 * time.Second
|
||||
const capsTTL = 10 * time.Minute // how long a probe is believed; a refresh after an install or a group change does not wait
|
||||
|
||||
type thumbSvc struct {
|
||||
cache thumbs.Cache
|
||||
self string
|
||||
sem chan struct{}
|
||||
mu sync.Mutex
|
||||
caps map[bool]*capsEntry // keyed by root
|
||||
calls map[string]*thumbCall
|
||||
}
|
||||
|
||||
type capsEntry struct {
|
||||
mu sync.Mutex
|
||||
caps thumbs.Caps
|
||||
err error
|
||||
at time.Time
|
||||
}
|
||||
|
||||
type thumbCall struct {
|
||||
done chan struct{}
|
||||
res thumbs.Result
|
||||
err error
|
||||
}
|
||||
|
||||
func newThumbSvc(dataDir, self string) *thumbSvc {
|
||||
n := runtime.NumCPU() / 2 // each ffmpeg is itself threaded; previews must not take the whole machine
|
||||
if n < 2 {
|
||||
n = 2
|
||||
}
|
||||
if n > 4 {
|
||||
n = 4
|
||||
}
|
||||
return &thumbSvc{cache: thumbs.Cache{Dir: filepath.Join(dataDir, "thumbs")}, self: self, sem: make(chan struct{}, n), caps: map[bool]*capsEntry{}, calls: map[string]*thumbCall{}}
|
||||
}
|
||||
|
||||
// capsFor is what previews can be made with, probed once and kept for a while.
|
||||
func (t *thumbSvc) capsFor(ctx context.Context, root, refresh bool) (thumbs.Caps, error) {
|
||||
t.mu.Lock()
|
||||
e := t.caps[root]
|
||||
if e == nil {
|
||||
e = &capsEntry{}
|
||||
t.caps[root] = e
|
||||
}
|
||||
t.mu.Unlock()
|
||||
e.mu.Lock()
|
||||
defer e.mu.Unlock()
|
||||
if !refresh && !e.at.IsZero() && time.Since(e.at) < capsTTL {
|
||||
return e.caps, e.err
|
||||
}
|
||||
if root {
|
||||
e.caps, e.err = thumbs.ProbeRoot(ctx, t.self)
|
||||
} else {
|
||||
e.caps, e.err = thumbs.Probe(ctx), nil
|
||||
}
|
||||
e.at = time.Now()
|
||||
mode := "as " + map[bool]string{false: "the daemon", true: "root"}[root]
|
||||
switch {
|
||||
case e.err != nil:
|
||||
log.Printf("thumbs: probe %s failed: %v", mode, e.err)
|
||||
case e.caps.Error != "":
|
||||
log.Printf("thumbs: %s: %s", mode, e.caps.Error)
|
||||
default:
|
||||
log.Printf("thumbs: %s: ffmpeg %s, %s", mode, e.caps.Version, describeCaps(e.caps))
|
||||
}
|
||||
return e.caps, e.err
|
||||
}
|
||||
|
||||
// describeCaps is the one line that says how previews get made here.
|
||||
func describeCaps(c thumbs.Caps) string {
|
||||
if len(c.GPUs) == 0 {
|
||||
return "CPU only"
|
||||
}
|
||||
parts := make([]string, 0, len(c.GPUs))
|
||||
for _, g := range c.GPUs {
|
||||
name := g.Name
|
||||
if name == "" {
|
||||
name = g.Device
|
||||
}
|
||||
p := name + " (" + g.API
|
||||
if len(g.Codecs) > 0 {
|
||||
p += ": " + strings.Join(g.Codecs, ", ")
|
||||
}
|
||||
parts = append(parts, p+")")
|
||||
}
|
||||
return "GPU " + strings.Join(parts, ", ")
|
||||
}
|
||||
|
||||
// make produces a preview, sharing the work when the same file is asked for twice at once.
|
||||
func (t *thumbSvc) make(ctx context.Context, root bool, key, path string) (thumbs.Result, error) {
|
||||
t.mu.Lock()
|
||||
if c, ok := t.calls[key]; ok {
|
||||
t.mu.Unlock()
|
||||
select {
|
||||
case <-c.done:
|
||||
return c.res, c.err
|
||||
case <-ctx.Done():
|
||||
return thumbs.Result{}, ctx.Err()
|
||||
}
|
||||
}
|
||||
c := &thumbCall{done: make(chan struct{})}
|
||||
t.calls[key] = c
|
||||
t.mu.Unlock()
|
||||
defer func() {
|
||||
t.mu.Lock()
|
||||
delete(t.calls, key)
|
||||
t.mu.Unlock()
|
||||
close(c.done)
|
||||
}()
|
||||
// The work itself runs on a context of its own: the first requester going away must not
|
||||
// cancel a preview three other rows are waiting for.
|
||||
wctx, cancel := context.WithTimeout(context.Background(), thumbTimeout+5*time.Second)
|
||||
defer cancel()
|
||||
select {
|
||||
case t.sem <- struct{}{}:
|
||||
defer func() { <-t.sem }()
|
||||
case <-ctx.Done():
|
||||
c.err = ctx.Err()
|
||||
return c.res, c.err
|
||||
}
|
||||
caps, err := t.capsFor(wctx, root, false)
|
||||
if err != nil {
|
||||
c.err = err
|
||||
return c.res, c.err
|
||||
}
|
||||
o := thumbs.Opts{Size: thumbSize, Timeout: thumbTimeout}
|
||||
if root {
|
||||
c.res, c.err = thumbs.MakeRoot(wctx, t.self, caps, path, o)
|
||||
} else {
|
||||
c.res, c.err = thumbs.Make(wctx, caps, path, o)
|
||||
}
|
||||
return c.res, c.err
|
||||
}
|
||||
|
||||
// ---- handlers ----
|
||||
|
||||
// filesThumb: GET /api/files/thumb?path=…&v=<mtime>-<size>[&sudo=1]. v is the client's idea of
|
||||
// the file, from its listing; it makes a cache hit free of a stat, and a changed file a new URL.
|
||||
func (s *Server) filesThumb(w http.ResponseWriter, r *http.Request) {
|
||||
fsys, ok := s.fsFor(w, r)
|
||||
if !ok {
|
||||
return
|
||||
}
|
||||
p, ok := cleanQuery(w, r, "path")
|
||||
if !ok {
|
||||
return
|
||||
}
|
||||
if thumbs.Kind(p) == "" {
|
||||
jsonErr(w, 415, "no preview for this kind of file")
|
||||
return
|
||||
}
|
||||
root := r.URL.Query().Get("sudo") == "1"
|
||||
if mt, sz, ok := parseV(r.URL.Query().Get("v")); ok {
|
||||
key := thumbs.Key(p, sz, mt)
|
||||
if r.Header.Get("If-None-Match") == `"`+key+`"` {
|
||||
w.WriteHeader(304)
|
||||
return
|
||||
}
|
||||
if s.serveCachedThumb(w, key) {
|
||||
return
|
||||
}
|
||||
}
|
||||
in, err := fsys.Stat(r.Context(), p)
|
||||
if err != nil {
|
||||
fsErr(w, err)
|
||||
return
|
||||
}
|
||||
if in.Type == "dir" || in.Type == "other" || in.Broken {
|
||||
jsonErr(w, 400, "not a regular file")
|
||||
return
|
||||
}
|
||||
key := thumbs.Key(p, in.Size, in.Mtime)
|
||||
if s.serveCachedThumb(w, key) {
|
||||
return
|
||||
}
|
||||
res, err := s.thumbs.make(r.Context(), root, key, p)
|
||||
if err != nil {
|
||||
var de *thumbs.DecodeError
|
||||
switch {
|
||||
case errors.As(err, &de):
|
||||
_ = s.thumbs.cache.PutFailure(key, de.Msg)
|
||||
log.Printf("thumbs: %s: %s", p, de.Msg)
|
||||
jsonErr(w, 422, de.Msg)
|
||||
case errors.Is(err, context.DeadlineExceeded):
|
||||
log.Printf("thumbs: %s: timed out", p)
|
||||
jsonErr(w, 504, "preview timed out")
|
||||
case errors.Is(err, context.Canceled):
|
||||
jsonErr(w, 499, "cancelled")
|
||||
default:
|
||||
jsonErr(w, 503, err.Error())
|
||||
}
|
||||
return
|
||||
}
|
||||
if err := s.thumbs.cache.Put(key, res.JPEG); err != nil {
|
||||
log.Printf("thumbs: cache %s: %v", key[:8], err)
|
||||
}
|
||||
writeThumb(w, key, res.JPEG, res.Engine)
|
||||
}
|
||||
|
||||
func (s *Server) serveCachedThumb(w http.ResponseWriter, key string) bool {
|
||||
data, why, found := s.thumbs.cache.Get(key)
|
||||
switch {
|
||||
case !found:
|
||||
return false
|
||||
case data == nil:
|
||||
jsonErr(w, 422, why)
|
||||
default:
|
||||
writeThumb(w, key, data, "cache")
|
||||
}
|
||||
return true
|
||||
}
|
||||
|
||||
func writeThumb(w http.ResponseWriter, key string, jpeg []byte, engine string) {
|
||||
w.Header().Set("Content-Type", "image/jpeg")
|
||||
w.Header().Set("Content-Length", strconv.Itoa(len(jpeg)))
|
||||
w.Header().Set("Cache-Control", "private, max-age=604800")
|
||||
w.Header().Set("ETag", `"`+key+`"`)
|
||||
w.Header().Set("X-Hamlab-Thumb", engine)
|
||||
w.Header().Set("X-Content-Type-Options", "nosniff")
|
||||
_, _ = w.Write(jpeg)
|
||||
}
|
||||
|
||||
func parseV(v string) (mtime, size int64, ok bool) {
|
||||
a, b, found := strings.Cut(v, "-")
|
||||
if !found {
|
||||
return 0, 0, false
|
||||
}
|
||||
mtime, err1 := strconv.ParseInt(a, 10, 64)
|
||||
size, err2 := strconv.ParseInt(b, 10, 64)
|
||||
return mtime, size, err1 == nil && err2 == nil
|
||||
}
|
||||
|
||||
type thumbStatus struct {
|
||||
OK bool `json:"ok"`
|
||||
Engine string `json:"engine"` // gpu | cpu | ""
|
||||
Summary string `json:"summary"`
|
||||
FFmpeg string `json:"ffmpeg,omitempty"`
|
||||
GPUs []thumbs.GPU `json:"gpus"`
|
||||
Unusable []thumbs.GPU `json:"unusable,omitempty"`
|
||||
Error string `json:"error,omitempty"`
|
||||
Install *thumbInstall `json:"install,omitempty"` // ffmpeg is missing and this can put it on
|
||||
GPUFix bool `json:"gpu_fix,omitempty"` // a render node is there but closed to the daemon's user, and this can open it
|
||||
User string `json:"user"`
|
||||
CacheDir string `json:"cache_dir"`
|
||||
}
|
||||
|
||||
type thumbInstall struct {
|
||||
PM string `json:"pm"`
|
||||
Package string `json:"package"`
|
||||
}
|
||||
|
||||
// thumbsStatus: GET /api/files/thumbs[?sudo=1][&refresh=1] — how previews get made on this machine.
|
||||
func (s *Server) thumbsStatus(w http.ResponseWriter, r *http.Request) {
|
||||
root := r.URL.Query().Get("sudo") == "1"
|
||||
if root && s.fsSudo == nil {
|
||||
jsonErr(w, 403, "root access needs passwordless sudo on the host")
|
||||
return
|
||||
}
|
||||
caps, err := s.thumbs.capsFor(r.Context(), root, r.URL.Query().Get("refresh") == "1")
|
||||
st := thumbStatus{GPUs: []thumbs.GPU{}, User: s.user, CacheDir: s.thumbs.cache.Dir}
|
||||
if root {
|
||||
st.User = "root"
|
||||
}
|
||||
if err != nil {
|
||||
st.Error = err.Error()
|
||||
jsonOK(w, 200, st)
|
||||
return
|
||||
}
|
||||
st.FFmpeg, st.GPUs, st.Unusable, st.Error = caps.Version, caps.GPUs, caps.Unusable, caps.Error
|
||||
if st.GPUs == nil {
|
||||
st.GPUs = []thumbs.GPU{}
|
||||
}
|
||||
st.OK = caps.Error == ""
|
||||
switch {
|
||||
case !st.OK:
|
||||
if pkg := ffmpegPackage(s.updates.PM()); pkg != "" && priv.Available() {
|
||||
st.Install = &thumbInstall{PM: s.updates.PM(), Package: pkg}
|
||||
}
|
||||
st.Summary = st.Error
|
||||
case len(caps.GPUs) > 0:
|
||||
st.Engine = "gpu"
|
||||
st.Summary = describeCaps(caps) + " · images on the CPU"
|
||||
default:
|
||||
st.Engine = "cpu"
|
||||
st.Summary = "CPU only"
|
||||
}
|
||||
if !root {
|
||||
for _, g := range caps.Unusable {
|
||||
if g.Permission && gpuFixPossible() {
|
||||
st.GPUFix = true
|
||||
}
|
||||
}
|
||||
}
|
||||
jsonOK(w, 200, st)
|
||||
}
|
||||
|
||||
// ffmpegPackage is what to install for ffmpeg on a package manager. Fedora ships it as
|
||||
// ffmpeg-free; the others under its own name.
|
||||
func ffmpegPackage(pm string) string {
|
||||
switch pm {
|
||||
case "":
|
||||
return ""
|
||||
case "dnf":
|
||||
if b, _ := os.ReadFile("/etc/os-release"); strings.Contains(string(b), "ID=fedora") {
|
||||
return "ffmpeg-free"
|
||||
}
|
||||
}
|
||||
return "ffmpeg"
|
||||
}
|
||||
|
||||
const hamlabUnit = "/etc/systemd/system/hamlab.service"
|
||||
|
||||
// gpuFixPossible: the daemon is an ordinary user under systemd, with root to hand.
|
||||
func gpuFixPossible() bool {
|
||||
if os.Geteuid() == 0 || !priv.Available() {
|
||||
return false
|
||||
}
|
||||
_, err := os.Stat(hamlabUnit)
|
||||
return err == nil
|
||||
}
|
||||
|
||||
// thumbsInstall: POST /api/files/thumbs/install — put ffmpeg on with the machine's package manager.
|
||||
func (s *Server) thumbsInstall(w http.ResponseWriter, r *http.Request) {
|
||||
pkg := ffmpegPackage(s.updates.PM())
|
||||
if pkg == "" {
|
||||
jsonErr(w, 400, "no supported package manager on this machine")
|
||||
return
|
||||
}
|
||||
t, err := s.updates.Install([]string{pkg})
|
||||
if err != nil {
|
||||
jsonErr(w, 400, err.Error())
|
||||
return
|
||||
}
|
||||
log.Printf("thumbs: installing %s as task %s (from %s)", pkg, t.ID, clientIP(r))
|
||||
jsonOK(w, 202, t.Snapshot(false))
|
||||
}
|
||||
|
||||
// gpuAccessScript gives hamlab.service the groups that own the render nodes, by a drop-in,
|
||||
// then restarts it: a running process cannot gain a group.
|
||||
const gpuAccessScript = `set -e
|
||||
unit=` + hamlabUnit + `
|
||||
[ -f "$unit" ] || { echo "no $unit: this machine does not run Ham Lab under systemd"; exit 1; }
|
||||
groups=""
|
||||
for g in render video; do
|
||||
if getent group "$g" >/dev/null 2>&1; then groups="$groups $g"; fi
|
||||
done
|
||||
[ -n "$groups" ] || { echo "neither a render nor a video group exists here"; exit 1; }
|
||||
mkdir -p "$unit.d"
|
||||
printf '[Service]\nSupplementaryGroups=%s\n' "${groups# }" > "$unit.d/gpu.conf"
|
||||
echo "hamlab.service: supplementary groups${groups} ($unit.d/gpu.conf)"
|
||||
systemctl daemon-reload
|
||||
echo 'Ham Lab restarts in 3 seconds to pick up its new groups; the page reconnects by itself.'
|
||||
systemd-run --quiet --on-active=3 --unit=hamlab-gpu-restart systemctl restart hamlab.service
|
||||
`
|
||||
|
||||
// thumbsGPUAccess: POST /api/files/thumbs/gpu-access — open the render nodes to the daemon.
|
||||
func (s *Server) thumbsGPUAccess(w http.ResponseWriter, r *http.Request) {
|
||||
if !gpuFixPossible() {
|
||||
jsonErr(w, 400, "nothing to do: Ham Lab is root, has no root, or is not under systemd here")
|
||||
return
|
||||
}
|
||||
full := priv.Wrap("sh", "-c", gpuAccessScript)
|
||||
t, err := s.tasks.Start("thumbs:gpu", "give Ham Lab access to the GPU", exec.Command(full[0], full[1:]...))
|
||||
if err != nil {
|
||||
jsonErr(w, 400, err.Error())
|
||||
return
|
||||
}
|
||||
log.Printf("thumbs: GPU access drop-in as task %s, restart follows (from %s)", t.ID, clientIP(r))
|
||||
jsonOK(w, 202, t.Snapshot(false))
|
||||
}
|
||||
+15
-10
@@ -24,19 +24,21 @@ import (
|
||||
"time"
|
||||
|
||||
"hamlab/internal/priv"
|
||||
"hamlab/internal/thumbs"
|
||||
)
|
||||
|
||||
type Entry struct {
|
||||
Name string `json:"name"`
|
||||
Type string `json:"type"` // dir | file | link | other
|
||||
Size int64 `json:"size"`
|
||||
Mode string `json:"mode"` // "-rw-r--r--"
|
||||
Perm string `json:"perm"` // "0644"
|
||||
Mtime int64 `json:"mtime"`
|
||||
User string `json:"user"`
|
||||
Group string `json:"group"`
|
||||
Target string `json:"target,omitempty"` // link target
|
||||
Broken bool `json:"broken,omitempty"` // dangling link
|
||||
Name string `json:"name"`
|
||||
Type string `json:"type"` // dir | file | link | other
|
||||
Size int64 `json:"size"`
|
||||
Mode string `json:"mode"` // "-rw-r--r--"
|
||||
Perm string `json:"perm"` // "0644"
|
||||
Mtime int64 `json:"mtime"`
|
||||
User string `json:"user"`
|
||||
Group string `json:"group"`
|
||||
Target string `json:"target,omitempty"` // link target
|
||||
Broken bool `json:"broken,omitempty"` // dangling link
|
||||
Preview string `json:"preview,omitempty"` // "image" | "video": the Files tab can show a thumbnail of it
|
||||
}
|
||||
|
||||
type Listing struct {
|
||||
@@ -146,6 +148,9 @@ func entryOf(dir, name string, lst os.FileInfo) Entry {
|
||||
default:
|
||||
e.Type = "other"
|
||||
}
|
||||
if e.Type == "file" || (e.Type == "link" && !e.Broken) {
|
||||
e.Preview = thumbs.Kind(name)
|
||||
}
|
||||
return e
|
||||
}
|
||||
|
||||
|
||||
@@ -250,6 +250,20 @@ func listGPUs(known []*gpuDev) []*gpuDev {
|
||||
return out
|
||||
}
|
||||
|
||||
// RenderNodeName is the GPU behind a render node ("/dev/dri/renderD128" → "Intel Arc A380"),
|
||||
// named the same way the Processes tab names it; "" when sysfs has nothing for it.
|
||||
func RenderNodeName(node string) string {
|
||||
dev, err := filepath.EvalSymlinks("/sys/class/drm/" + filepath.Base(node) + "/device")
|
||||
if err != nil {
|
||||
return ""
|
||||
}
|
||||
driver := ueventValue(dev+"/uevent", "DRIVER")
|
||||
if driver == "" {
|
||||
driver = filepath.Base(readLink(dev + "/driver"))
|
||||
}
|
||||
return gpuName(dev, ueventValue(dev+"/uevent", "PCI_SLOT_NAME"), driver)
|
||||
}
|
||||
|
||||
var gpuVendors = map[string]string{"0x8086": "Intel", "0x1002": "AMD", "0x10de": "NVIDIA", "0x1af4": "Virtio", "0x106b": "Apple"}
|
||||
|
||||
var (
|
||||
|
||||
@@ -0,0 +1,78 @@
|
||||
package thumbs
|
||||
|
||||
import (
|
||||
"crypto/sha256"
|
||||
"encoding/hex"
|
||||
"errors"
|
||||
"os"
|
||||
"path/filepath"
|
||||
"strconv"
|
||||
)
|
||||
|
||||
// Cache keeps finished previews on disk, under Ham Lab's own data directory: one JPEG per
|
||||
// (path, size, mtime), so a file that changes gets a new preview and an unchanged one is
|
||||
// never decoded twice. A file ffmpeg could not decode is remembered the same way, with
|
||||
// ffmpeg's reason beside it, so a folder of odd files does not re-run ffmpeg on every
|
||||
// visit. Nothing expires: a preview is a few kilobytes, and only files somebody actually
|
||||
// scrolled past are ever made.
|
||||
type Cache struct{ Dir string }
|
||||
|
||||
// Key identifies one file's preview. "v1" is the preview format: bump it to retire every
|
||||
// cached preview at once.
|
||||
func Key(path string, size, mtime int64) string {
|
||||
h := sha256.Sum256([]byte("v1\x00" + path + "\x00" + strconv.FormatInt(size, 10) + "\x00" + strconv.FormatInt(mtime, 10)))
|
||||
return hex.EncodeToString(h[:])
|
||||
}
|
||||
|
||||
func (c Cache) file(key, ext string) string { return filepath.Join(c.Dir, key[:2], key+ext) }
|
||||
|
||||
// Get returns the preview, or the remembered reason it could not be made.
|
||||
func (c Cache) Get(key string) (jpeg []byte, failure string, found bool) {
|
||||
if len(key) < 2 {
|
||||
return nil, "", false
|
||||
}
|
||||
if b, err := os.ReadFile(c.file(key, ".jpg")); err == nil && len(b) > 0 {
|
||||
return b, "", true
|
||||
}
|
||||
if b, err := os.ReadFile(c.file(key, ".err")); err == nil {
|
||||
return nil, string(b), true
|
||||
}
|
||||
return nil, "", false
|
||||
}
|
||||
|
||||
// Put keeps a preview; PutFailure keeps the reason there is none.
|
||||
func (c Cache) Put(key string, jpeg []byte) error {
|
||||
if len(jpeg) == 0 {
|
||||
return errors.New("empty preview")
|
||||
}
|
||||
return c.write(c.file(key, ".jpg"), jpeg)
|
||||
}
|
||||
|
||||
func (c Cache) PutFailure(key, reason string) error {
|
||||
return c.write(c.file(key, ".err"), []byte(reason))
|
||||
}
|
||||
|
||||
// write lands the file whole or not at all: a reader never sees a half-written JPEG.
|
||||
func (c Cache) write(dst string, b []byte) error {
|
||||
if err := os.MkdirAll(filepath.Dir(dst), 0o700); err != nil {
|
||||
return err
|
||||
}
|
||||
tmp, err := os.CreateTemp(filepath.Dir(dst), ".tmp-*")
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
if _, err := tmp.Write(b); err != nil {
|
||||
tmp.Close()
|
||||
os.Remove(tmp.Name())
|
||||
return err
|
||||
}
|
||||
if err := tmp.Close(); err != nil {
|
||||
os.Remove(tmp.Name())
|
||||
return err
|
||||
}
|
||||
if err := os.Rename(tmp.Name(), dst); err != nil {
|
||||
os.Remove(tmp.Name())
|
||||
return err
|
||||
}
|
||||
return nil
|
||||
}
|
||||
@@ -0,0 +1,26 @@
|
||||
package thumbs
|
||||
|
||||
import (
|
||||
"fmt"
|
||||
"os"
|
||||
"os/user"
|
||||
"strconv"
|
||||
"syscall"
|
||||
)
|
||||
|
||||
// ownerOf is "root:render 0660" for a device node, the thing to say when it cannot be opened.
|
||||
func ownerOf(st os.FileInfo) string {
|
||||
sys, ok := st.Sys().(*syscall.Stat_t)
|
||||
if !ok {
|
||||
return ""
|
||||
}
|
||||
u := strconv.Itoa(int(sys.Uid))
|
||||
if x, err := user.LookupId(u); err == nil {
|
||||
u = x.Username
|
||||
}
|
||||
g := strconv.Itoa(int(sys.Gid))
|
||||
if x, err := user.LookupGroupId(g); err == nil {
|
||||
g = x.Name
|
||||
}
|
||||
return fmt.Sprintf("%s:%s %04o", u, g, st.Mode().Perm())
|
||||
}
|
||||
@@ -0,0 +1,615 @@
|
||||
// Package thumbs makes the Files tab's previews: one small JPEG for an image or a video,
|
||||
// by ffmpeg, decoded on a GPU where the machine has one it can use and on the CPU
|
||||
// otherwise.
|
||||
//
|
||||
// Nothing about the machine is assumed. A probe looks ffmpeg and ffprobe up, opens every
|
||||
// /dev/dri/renderD* and exercises it (a VAAPI upload and download of a synthetic frame),
|
||||
// so a node the daemon's user cannot open, or a driver that does not speak VAAPI, is
|
||||
// reported once rather than tried per file. vainfo, where installed, says which codecs
|
||||
// each node decodes, and a video goes to the first node that lists its codec; without
|
||||
// vainfo the first working node is tried and ffmpeg falls back to software on its own when
|
||||
// the stream turns out unsupported. A hardware run that fails outright is retried on the
|
||||
// CPU before the file is called undecodable. Images are always decoded on the CPU: one
|
||||
// frame of a JPEG at a reduced DCT size costs less than the trip through a GPU.
|
||||
package thumbs
|
||||
|
||||
import (
|
||||
"bytes"
|
||||
"context"
|
||||
"encoding/json"
|
||||
"errors"
|
||||
"fmt"
|
||||
"io"
|
||||
"os"
|
||||
"os/exec"
|
||||
"path/filepath"
|
||||
"regexp"
|
||||
"sort"
|
||||
"strconv"
|
||||
"strings"
|
||||
"time"
|
||||
|
||||
"hamlab/internal/metrics"
|
||||
"hamlab/internal/priv"
|
||||
)
|
||||
|
||||
// GPU is one hardware decoder as the probe found it.
|
||||
type GPU struct {
|
||||
API string `json:"api"` // vaapi | cuda
|
||||
Device string `json:"device"` // /dev/dri/renderD128; "0" for cuda (its device index)
|
||||
Name string `json:"name,omitempty"` // "Intel Arc A380"
|
||||
Codecs []string `json:"codecs,omitempty"` // ffmpeg codec names it decodes; nil when nothing could say
|
||||
Reason string `json:"reason,omitempty"` // Unusable only: why
|
||||
Permission bool `json:"permission,omitempty"` // Unusable only: the node exists but this user may not open it
|
||||
}
|
||||
|
||||
// Caps is what the machine can make previews with, as probed.
|
||||
type Caps struct {
|
||||
FFmpeg string `json:"ffmpeg,omitempty"`
|
||||
FFprobe string `json:"ffprobe,omitempty"`
|
||||
Nice string `json:"nice,omitempty"`
|
||||
Version string `json:"version,omitempty"`
|
||||
GPUs []GPU `json:"gpus"` // usable, in preference order
|
||||
Unusable []GPU `json:"unusable,omitempty"` // render nodes that exist but cannot be used, with the reason
|
||||
Error string `json:"error,omitempty"` // why nothing can be made at all
|
||||
ProbedAt time.Time `json:"probed_at"`
|
||||
}
|
||||
|
||||
// Opts shapes one preview.
|
||||
type Opts struct {
|
||||
Size int // the long edge in pixels; 256 when zero
|
||||
Timeout time.Duration // for the whole job, ffprobe and every ffmpeg attempt; 30 s when zero
|
||||
}
|
||||
|
||||
func (o Opts) defaults() Opts {
|
||||
if o.Size <= 0 {
|
||||
o.Size = 256
|
||||
}
|
||||
if o.Timeout <= 0 {
|
||||
o.Timeout = 30 * time.Second
|
||||
}
|
||||
return o
|
||||
}
|
||||
|
||||
// Result is a finished preview.
|
||||
type Result struct {
|
||||
JPEG []byte `json:"-"`
|
||||
Engine string `json:"engine"` // "cpu", or "vaapi:/dev/dri/renderD129"
|
||||
Codec string `json:"codec"`
|
||||
Width int `json:"width"`
|
||||
Height int `json:"height"`
|
||||
}
|
||||
|
||||
// ErrUnsupported: not an image or a video by its name.
|
||||
var ErrUnsupported = errors.New("not an image or video")
|
||||
|
||||
// A DecodeError is ffmpeg's verdict on the file itself. It will not change until the
|
||||
// file does, so the caller may remember it.
|
||||
type DecodeError struct{ Msg string }
|
||||
|
||||
func (e *DecodeError) Error() string { return e.Msg }
|
||||
|
||||
// ---- what gets a preview ----
|
||||
|
||||
var imageExt = set("png jpg jpeg jpe jfif gif webp bmp tif tiff avif heic heif hif jxl svg ico psd tga thm")
|
||||
var videoExt = set("mp4 m4v mkv webm mov avi wmv flv mpg mpeg mpe m2v ts m2ts mts 3gp 3g2 ogv ogm mxf vob asf rm rmvb divx f4v mjpeg mjpg 360 osv insv lrf")
|
||||
|
||||
func set(s string) map[string]bool {
|
||||
m := map[string]bool{}
|
||||
for _, w := range strings.Fields(s) {
|
||||
m[w] = true
|
||||
}
|
||||
return m
|
||||
}
|
||||
|
||||
// Kind is "image", "video" or "" for a file name.
|
||||
func Kind(name string) string {
|
||||
ext := strings.ToLower(strings.TrimPrefix(filepath.Ext(name), "."))
|
||||
switch {
|
||||
case imageExt[ext]:
|
||||
return "image"
|
||||
case videoExt[ext]:
|
||||
return "video"
|
||||
}
|
||||
return ""
|
||||
}
|
||||
|
||||
// ---- the probe ----
|
||||
|
||||
// Probe finds out what this process can make previews with. It runs ffmpeg against every
|
||||
// render node, so it takes a few hundred milliseconds; callers keep the answer.
|
||||
func Probe(ctx context.Context) Caps {
|
||||
c := Caps{ProbedAt: time.Now(), GPUs: []GPU{}}
|
||||
c.FFmpeg, _ = exec.LookPath("ffmpeg")
|
||||
c.FFprobe, _ = exec.LookPath("ffprobe")
|
||||
c.Nice, _ = exec.LookPath("nice")
|
||||
switch {
|
||||
case c.FFmpeg == "" && c.FFprobe == "":
|
||||
c.Error = "ffmpeg is not installed"
|
||||
case c.FFmpeg == "":
|
||||
c.Error = "ffmpeg is not installed (ffprobe is)"
|
||||
case c.FFprobe == "":
|
||||
c.Error = "ffprobe is not installed (ffmpeg is)"
|
||||
}
|
||||
if c.Error != "" {
|
||||
return c
|
||||
}
|
||||
c.Version = ffmpegVersion(ctx, c.FFmpeg)
|
||||
accels := hwaccels(ctx, c.FFmpeg)
|
||||
if accels["vaapi"] {
|
||||
nodes, _ := filepath.Glob("/dev/dri/renderD*")
|
||||
sort.Strings(nodes)
|
||||
vainfo, _ := exec.LookPath("vainfo")
|
||||
for _, dev := range nodes {
|
||||
g := GPU{API: "vaapi", Device: dev, Name: metrics.RenderNodeName(dev)}
|
||||
if err := openable(dev); err != nil {
|
||||
g.Reason, g.Permission = err.Error(), errors.Is(err, os.ErrPermission)
|
||||
c.Unusable = append(c.Unusable, g)
|
||||
continue
|
||||
}
|
||||
if msg := hwWorks(ctx, c.FFmpeg, "vaapi=va:"+dev); msg != "" {
|
||||
g.Reason = msg
|
||||
c.Unusable = append(c.Unusable, g)
|
||||
continue
|
||||
}
|
||||
if vainfo != "" {
|
||||
g.Codecs = vainfoCodecs(ctx, vainfo, dev)
|
||||
}
|
||||
c.GPUs = append(c.GPUs, g)
|
||||
}
|
||||
}
|
||||
if accels["cuda"] && exists("/dev/nvidiactl") {
|
||||
g := GPU{API: "cuda", Device: "0", Name: "NVIDIA GPU"}
|
||||
if msg := hwWorks(ctx, c.FFmpeg, "cuda=cu:0"); msg != "" {
|
||||
g.Reason = msg
|
||||
c.Unusable = append(c.Unusable, g)
|
||||
} else {
|
||||
c.GPUs = append(c.GPUs, g)
|
||||
}
|
||||
}
|
||||
return c
|
||||
}
|
||||
|
||||
// openable: the node can be opened read-write, which is what a VAAPI driver needs. The
|
||||
// usual reason it can't be is the owning group: a render node is root:render 0660 on
|
||||
// most distributions, and a daemon running as an ordinary user is not in render.
|
||||
func openable(dev string) error {
|
||||
f, err := os.OpenFile(dev, os.O_RDWR, 0)
|
||||
if err != nil {
|
||||
if errors.Is(err, os.ErrPermission) {
|
||||
owner := ""
|
||||
if st, serr := os.Stat(dev); serr == nil {
|
||||
owner = ownerOf(st)
|
||||
}
|
||||
if owner != "" {
|
||||
return fmt.Errorf("%w: %s is %s", os.ErrPermission, dev, owner)
|
||||
}
|
||||
return fmt.Errorf("%w: cannot open %s", os.ErrPermission, dev)
|
||||
}
|
||||
return err
|
||||
}
|
||||
f.Close()
|
||||
return nil
|
||||
}
|
||||
|
||||
// hwWorks initialises a hardware device in ffmpeg and moves one synthetic frame through
|
||||
// it. "" when it works, else the last thing ffmpeg said.
|
||||
func hwWorks(ctx context.Context, ffmpeg, device string) string {
|
||||
ctx, cancel := context.WithTimeout(ctx, 15*time.Second)
|
||||
defer cancel()
|
||||
name, rest, _ := strings.Cut(device, "=") // "vaapi", "va:/dev/…"
|
||||
label, _, _ := strings.Cut(rest, ":") // "va"
|
||||
cmd := exec.CommandContext(ctx, ffmpeg, "-nostdin", "-hide_banner", "-v", "error", "-init_hw_device", device, "-filter_hw_device", label,
|
||||
"-f", "lavfi", "-i", "color=c=black:s=64x64:r=1:d=1", "-vf", "format=nv12,hwupload,hwdownload,format=nv12", "-frames:v", "1", "-f", "null", "-")
|
||||
var errb bytes.Buffer
|
||||
cmd.Stderr = &errb
|
||||
if err := cmd.Run(); err != nil {
|
||||
msg := lastLine(errb.String())
|
||||
if msg == "" {
|
||||
msg = err.Error()
|
||||
}
|
||||
return name + ": " + msg
|
||||
}
|
||||
return ""
|
||||
}
|
||||
|
||||
func ffmpegVersion(ctx context.Context, ffmpeg string) string {
|
||||
ctx, cancel := context.WithTimeout(ctx, 5*time.Second)
|
||||
defer cancel()
|
||||
out, _ := exec.CommandContext(ctx, ffmpeg, "-version").Output()
|
||||
f := strings.Fields(string(out)) // "ffmpeg version 7.1.5-0+deb13u1 Copyright …"
|
||||
if len(f) >= 3 && f[1] == "version" {
|
||||
v := f[2]
|
||||
if i := strings.IndexAny(v, "-+~"); i > 0 {
|
||||
v = v[:i]
|
||||
}
|
||||
return v
|
||||
}
|
||||
return ""
|
||||
}
|
||||
|
||||
func hwaccels(ctx context.Context, ffmpeg string) map[string]bool {
|
||||
ctx, cancel := context.WithTimeout(ctx, 5*time.Second)
|
||||
defer cancel()
|
||||
out, _ := exec.CommandContext(ctx, ffmpeg, "-hide_banner", "-hwaccels").Output()
|
||||
m := map[string]bool{}
|
||||
for _, l := range strings.Split(string(out), "\n") {
|
||||
l = strings.TrimSpace(l)
|
||||
if l != "" && !strings.HasPrefix(l, "Hardware acceleration methods") {
|
||||
m[l] = true
|
||||
}
|
||||
}
|
||||
return m
|
||||
}
|
||||
|
||||
var vaProfileRe = regexp.MustCompile(`VAProfile([A-Za-z0-9]+?)(?:Simple|Main|High|Profile|Baseline|Constrained|Scc|Advanced|Version|Stereo|Multiview)\w*\s*:\s*VAEntrypointVLD`)
|
||||
|
||||
// vaCodec maps a VAProfile family to the ffmpeg codec it decodes.
|
||||
var vaCodec = map[string]string{"H264": "h264", "HEVC": "hevc", "VP9": "vp9", "AV1": "av1", "MPEG2": "mpeg2video", "VC1": "vc1", "VP8": "vp8", "JPEG": "mjpeg", "MPEG4": "mpeg4", "H263": "h263"}
|
||||
|
||||
func vainfoCodecs(ctx context.Context, vainfo, dev string) []string {
|
||||
ctx, cancel := context.WithTimeout(ctx, 10*time.Second)
|
||||
defer cancel()
|
||||
out, _ := exec.CommandContext(ctx, vainfo, "--display", "drm", "--device", dev).Output()
|
||||
return parseVainfo(string(out))
|
||||
}
|
||||
|
||||
// parseVainfo keeps the decode (VLD) profiles and names their codecs the way ffmpeg does.
|
||||
func parseVainfo(text string) []string {
|
||||
seen := map[string]bool{}
|
||||
for _, m := range vaProfileRe.FindAllStringSubmatch(text, -1) {
|
||||
if c, ok := vaCodec[m[1]]; ok {
|
||||
seen[c] = true
|
||||
}
|
||||
}
|
||||
out := make([]string, 0, len(seen))
|
||||
for c := range seen {
|
||||
out = append(out, c)
|
||||
}
|
||||
sort.Strings(out)
|
||||
return out
|
||||
}
|
||||
|
||||
// pick is the GPU for a codec: the first that lists it, else the first that could not
|
||||
// say (ffmpeg will fall back to software by itself if that guess is wrong), else none.
|
||||
func (c Caps) pick(codec string) *GPU {
|
||||
var unknown *GPU
|
||||
for i := range c.GPUs {
|
||||
g := &c.GPUs[i]
|
||||
if g.Codecs == nil {
|
||||
if unknown == nil {
|
||||
unknown = g
|
||||
}
|
||||
continue
|
||||
}
|
||||
for _, x := range g.Codecs {
|
||||
if x == codec {
|
||||
return g
|
||||
}
|
||||
}
|
||||
}
|
||||
return unknown
|
||||
}
|
||||
|
||||
// ---- making one ----
|
||||
|
||||
type fileInfo struct {
|
||||
index string // the stream to take
|
||||
codec string
|
||||
w, h int
|
||||
duration float64
|
||||
}
|
||||
|
||||
type attempt struct {
|
||||
gpu *GPU
|
||||
lowres int
|
||||
seek float64
|
||||
}
|
||||
|
||||
func (a attempt) engine() string {
|
||||
if a.gpu == nil {
|
||||
return "cpu"
|
||||
}
|
||||
return a.gpu.API + ":" + a.gpu.Device
|
||||
}
|
||||
|
||||
// Make produces the preview of one file.
|
||||
func Make(ctx context.Context, c Caps, path string, o Opts) (Result, error) {
|
||||
kind := Kind(path)
|
||||
if kind == "" {
|
||||
return Result{}, ErrUnsupported
|
||||
}
|
||||
if c.Error != "" {
|
||||
return Result{}, errors.New(c.Error)
|
||||
}
|
||||
o = o.defaults()
|
||||
ctx, cancel := context.WithTimeout(ctx, o.Timeout)
|
||||
defer cancel()
|
||||
fi, err := probeFile(ctx, c, path)
|
||||
if err != nil {
|
||||
return Result{}, err
|
||||
}
|
||||
var last string
|
||||
for _, eng := range engines(c, kind, fi) {
|
||||
for _, a := range attempts(eng, kind, fi, o.Size) {
|
||||
data, msg, err := run(ctx, c, a, path, fi, o.Size)
|
||||
if err == nil && len(data) > 0 {
|
||||
return Result{JPEG: data, Engine: a.engine(), Codec: fi.codec, Width: fi.w, Height: fi.h}, nil
|
||||
}
|
||||
if ctx.Err() != nil {
|
||||
return Result{}, ctx.Err() // the clock ran out: not the file's fault
|
||||
}
|
||||
if err != nil { // the decoder refused: try the next engine, not another seek
|
||||
last = msg
|
||||
break
|
||||
}
|
||||
last = "no frame at " + strconv.FormatFloat(a.seek, 'f', 1, 64) + " s"
|
||||
}
|
||||
}
|
||||
if last == "" {
|
||||
last = "could not decode"
|
||||
}
|
||||
return Result{}, &DecodeError{Msg: last}
|
||||
}
|
||||
|
||||
// engines is the order to try: the fitting GPU first for a video, then the CPU.
|
||||
func engines(c Caps, kind string, fi fileInfo) []*GPU {
|
||||
if kind == "video" {
|
||||
if g := c.pick(fi.codec); g != nil {
|
||||
return []*GPU{g, nil}
|
||||
}
|
||||
}
|
||||
return []*GPU{nil}
|
||||
}
|
||||
|
||||
// attempts for one engine: a seek into a video, then from its start if that found no
|
||||
// frame (a wrong duration in the header); a reduced-size decode of a JPEG, then full size.
|
||||
func attempts(gpu *GPU, kind string, fi fileInfo, size int) []attempt {
|
||||
if kind == "video" {
|
||||
s := seekFor(fi.duration)
|
||||
if s > 0 {
|
||||
return []attempt{{gpu: gpu, seek: s}, {gpu: gpu}}
|
||||
}
|
||||
return []attempt{{gpu: gpu}}
|
||||
}
|
||||
if fi.codec == "mjpeg" && gpu == nil {
|
||||
if l := lowresFor(fi.w, fi.h, size); l > 0 {
|
||||
return []attempt{{lowres: l}, {}}
|
||||
}
|
||||
}
|
||||
return []attempt{{gpu: gpu}}
|
||||
}
|
||||
|
||||
// seekFor: a tenth of the way in, never more than 30 s, so a title card or a black lead-in
|
||||
// is skipped and a long file still seeks to a keyframe near its start.
|
||||
func seekFor(duration float64) float64 {
|
||||
if duration <= 0 {
|
||||
return 0
|
||||
}
|
||||
s := duration / 10
|
||||
if s > 30 {
|
||||
s = 30
|
||||
}
|
||||
if s < 0.5 {
|
||||
return 0
|
||||
}
|
||||
return s
|
||||
}
|
||||
|
||||
// lowresFor: how many halvings a JPEG decoder may take while the picture stays at least
|
||||
// twice the preview size. libjpeg does this in the DCT, so a 24 MP photo decodes at 1.5 MP.
|
||||
func lowresFor(w, h, size int) int {
|
||||
m := w
|
||||
if h < m {
|
||||
m = h
|
||||
}
|
||||
l := 0
|
||||
for l < 3 && m>>(l+1) >= size*2 {
|
||||
l++
|
||||
}
|
||||
return l
|
||||
}
|
||||
|
||||
type probeOut struct {
|
||||
Streams []struct {
|
||||
Index int `json:"index"`
|
||||
Codec string `json:"codec_name"`
|
||||
Width int `json:"width"`
|
||||
Height int `json:"height"`
|
||||
Disposition struct {
|
||||
AttachedPic int `json:"attached_pic"`
|
||||
} `json:"disposition"`
|
||||
} `json:"streams"`
|
||||
Format struct {
|
||||
Duration string `json:"duration"`
|
||||
} `json:"format"`
|
||||
}
|
||||
|
||||
// probeFile asks ffprobe which stream to take and what is in it.
|
||||
func probeFile(ctx context.Context, c Caps, path string) (fileInfo, error) {
|
||||
cmd := exec.CommandContext(ctx, c.FFprobe, "-v", "error", "-select_streams", "v", "-show_entries", "stream=index,codec_name,width,height,disposition:format=duration", "-of", "json", path)
|
||||
var errb bytes.Buffer
|
||||
cmd.Stderr = &errb
|
||||
out, err := cmd.Output()
|
||||
if err != nil {
|
||||
if ctx.Err() != nil {
|
||||
return fileInfo{}, ctx.Err()
|
||||
}
|
||||
msg := lastLine(errb.String())
|
||||
if msg == "" {
|
||||
msg = "ffprobe: " + err.Error()
|
||||
}
|
||||
return fileInfo{}, &DecodeError{Msg: msg}
|
||||
}
|
||||
var po probeOut
|
||||
if err := json.Unmarshal(out, &po); err != nil {
|
||||
return fileInfo{}, &DecodeError{Msg: "ffprobe: unreadable answer"}
|
||||
}
|
||||
if len(po.Streams) == 0 {
|
||||
return fileInfo{}, &DecodeError{Msg: "no video stream"}
|
||||
}
|
||||
s := po.Streams[0]
|
||||
for _, x := range po.Streams { // cover art rides along as a video stream; the picture is not the film
|
||||
if x.Disposition.AttachedPic == 0 {
|
||||
s = x
|
||||
break
|
||||
}
|
||||
}
|
||||
fi := fileInfo{index: strconv.Itoa(s.Index), codec: s.Codec, w: s.Width, h: s.Height}
|
||||
fi.duration, _ = strconv.ParseFloat(po.Format.Duration, 64)
|
||||
return fi, nil
|
||||
}
|
||||
|
||||
// run is one ffmpeg: the JPEG on success; on failure what ffmpeg said and the error.
|
||||
// Empty output with a clean exit means the seek found nothing.
|
||||
func run(ctx context.Context, c Caps, a attempt, path string, fi fileInfo, size int) ([]byte, string, error) {
|
||||
args := []string{"-nostdin", "-hide_banner", "-loglevel", "error", "-y"}
|
||||
if a.gpu != nil {
|
||||
args = append(args, "-hwaccel", a.gpu.API, "-hwaccel_device", a.gpu.Device)
|
||||
}
|
||||
if a.lowres > 0 {
|
||||
args = append(args, "-lowres", strconv.Itoa(a.lowres))
|
||||
}
|
||||
if a.seek > 0 {
|
||||
args = append(args, "-noaccurate_seek", "-ss", strconv.FormatFloat(a.seek, 'f', 3, 64))
|
||||
}
|
||||
args = append(args, "-i", path, "-map", "0:"+fi.index, "-frames:v", "1",
|
||||
"-vf", fmt.Sprintf("scale=w=%d:h=%d:force_original_aspect_ratio=decrease:flags=bicubic", size, size),
|
||||
"-f", "image2pipe", "-c:v", "mjpeg", "-q:v", "4", "pipe:1")
|
||||
name, argv := c.FFmpeg, args
|
||||
if c.Nice != "" { // a burst of previews must not crowd out what the machine is for
|
||||
name, argv = c.Nice, append([]string{"-n", "10", c.FFmpeg}, args...)
|
||||
}
|
||||
cmd := exec.CommandContext(ctx, name, argv...)
|
||||
var out, errb bytes.Buffer
|
||||
cmd.Stdout, cmd.Stderr = &out, &errb
|
||||
if err := cmd.Run(); err != nil {
|
||||
msg := lastLine(errb.String())
|
||||
if msg == "" {
|
||||
msg = err.Error()
|
||||
}
|
||||
return nil, msg, err
|
||||
}
|
||||
return out.Bytes(), "", nil
|
||||
}
|
||||
|
||||
// ---- the same, as root ----
|
||||
|
||||
// ProbeRoot is Probe run as root through this binary, for the Files tab's root mode: root
|
||||
// opens every render node, whatever group the daemon's user is in.
|
||||
func ProbeRoot(ctx context.Context, self string) (Caps, error) {
|
||||
full := priv.Wrap(self, "thumb", "--probe")
|
||||
cmd := exec.CommandContext(ctx, full[0], full[1:]...)
|
||||
var errb bytes.Buffer
|
||||
cmd.Stderr = &errb
|
||||
out, err := cmd.Output()
|
||||
if err != nil {
|
||||
if msg := lastLine(errb.String()); msg != "" {
|
||||
return Caps{}, errors.New(msg)
|
||||
}
|
||||
return Caps{}, err
|
||||
}
|
||||
var c Caps
|
||||
if err := json.Unmarshal(out, &c); err != nil {
|
||||
return Caps{}, err
|
||||
}
|
||||
return c, nil
|
||||
}
|
||||
|
||||
type helperOut struct {
|
||||
Engine string `json:"engine,omitempty"`
|
||||
Codec string `json:"codec,omitempty"`
|
||||
Width int `json:"width,omitempty"`
|
||||
Height int `json:"height,omitempty"`
|
||||
JPEG []byte `json:"jpeg,omitempty"`
|
||||
Error string `json:"error,omitempty"`
|
||||
Permanent bool `json:"permanent,omitempty"` // a DecodeError: the file, not the moment
|
||||
}
|
||||
|
||||
// MakeRoot is Make run as root through this binary, with the caps ProbeRoot found.
|
||||
func MakeRoot(ctx context.Context, self string, c Caps, path string, o Opts) (Result, error) {
|
||||
o = o.defaults()
|
||||
caps, _ := json.Marshal(c)
|
||||
full := priv.Wrap(self, "thumb", "--size", strconv.Itoa(o.Size), "--timeout", o.Timeout.String(), path)
|
||||
cmd := exec.CommandContext(ctx, full[0], full[1:]...)
|
||||
cmd.Stdin = bytes.NewReader(caps)
|
||||
var errb bytes.Buffer
|
||||
cmd.Stderr = &errb
|
||||
out, err := cmd.Output()
|
||||
if err != nil {
|
||||
if ctx.Err() != nil {
|
||||
return Result{}, ctx.Err()
|
||||
}
|
||||
if msg := lastLine(errb.String()); msg != "" {
|
||||
return Result{}, errors.New(msg)
|
||||
}
|
||||
return Result{}, err
|
||||
}
|
||||
var h helperOut
|
||||
if err := json.Unmarshal(out, &h); err != nil {
|
||||
return Result{}, err
|
||||
}
|
||||
if h.Error != "" {
|
||||
if h.Permanent {
|
||||
return Result{}, &DecodeError{Msg: h.Error}
|
||||
}
|
||||
return Result{}, errors.New(h.Error)
|
||||
}
|
||||
return Result{JPEG: h.JPEG, Engine: h.Engine, Codec: h.Codec, Width: h.Width, Height: h.Height}, nil
|
||||
}
|
||||
|
||||
// Helper implements `hamlab thumb …`, the root side of the above. `--probe` prints Caps;
|
||||
// `[--size N] [--timeout D] <path>` reads Caps from stdin and prints the preview as JSON.
|
||||
// Returns the exit code.
|
||||
func Helper(args []string, stdin io.Reader, stdout, stderr io.Writer) int {
|
||||
fail := func(err error) int { fmt.Fprintln(stderr, err.Error()); return 1 }
|
||||
ctx := context.Background()
|
||||
o := Opts{}
|
||||
path := ""
|
||||
for i := 0; i < len(args); i++ {
|
||||
switch a := args[i]; {
|
||||
case a == "--probe":
|
||||
return emit(stdout, Probe(ctx), stderr)
|
||||
case a == "--size" && i+1 < len(args):
|
||||
o.Size, _ = strconv.Atoi(args[i+1])
|
||||
i++
|
||||
case a == "--timeout" && i+1 < len(args):
|
||||
o.Timeout, _ = time.ParseDuration(args[i+1])
|
||||
i++
|
||||
case strings.HasPrefix(a, "-"):
|
||||
return fail(errors.New("usage: hamlab thumb --probe | [--size N] [--timeout D] <path>"))
|
||||
default:
|
||||
path = a
|
||||
}
|
||||
}
|
||||
if path == "" || !filepath.IsAbs(path) {
|
||||
return fail(errors.New("usage: hamlab thumb --probe | [--size N] [--timeout D] <absolute path>"))
|
||||
}
|
||||
var c Caps
|
||||
if err := json.NewDecoder(io.LimitReader(stdin, 1<<20)).Decode(&c); err != nil {
|
||||
return fail(fmt.Errorf("caps on stdin: %w", err))
|
||||
}
|
||||
res, err := Make(ctx, c, path, o)
|
||||
if err != nil {
|
||||
var de *DecodeError
|
||||
return emit(stdout, helperOut{Error: err.Error(), Permanent: errors.As(err, &de)}, stderr)
|
||||
}
|
||||
return emit(stdout, helperOut{Engine: res.Engine, Codec: res.Codec, Width: res.Width, Height: res.Height, JPEG: res.JPEG}, stderr)
|
||||
}
|
||||
|
||||
func emit(w io.Writer, v any, stderr io.Writer) int {
|
||||
if err := json.NewEncoder(w).Encode(v); err != nil {
|
||||
fmt.Fprintln(stderr, err.Error())
|
||||
return 1
|
||||
}
|
||||
return 0
|
||||
}
|
||||
|
||||
// ---- small helpers ----
|
||||
|
||||
func lastLine(s string) string {
|
||||
lines := strings.Split(strings.TrimSpace(s), "\n")
|
||||
return strings.TrimSpace(lines[len(lines)-1])
|
||||
}
|
||||
|
||||
func exists(p string) bool { _, err := os.Stat(p); return err == nil }
|
||||
@@ -0,0 +1,217 @@
|
||||
package thumbs
|
||||
|
||||
import (
|
||||
"bytes"
|
||||
"context"
|
||||
"image"
|
||||
"image/color"
|
||||
"image/jpeg"
|
||||
"image/png"
|
||||
"os"
|
||||
"os/exec"
|
||||
"path/filepath"
|
||||
"testing"
|
||||
"time"
|
||||
)
|
||||
|
||||
func TestKind(t *testing.T) {
|
||||
for name, want := range map[string]string{"a.JPG": "image", "b.heic": "image", "c.mkv": "video", "GH010098.MP4": "video", "x.OSV": "video", "d.txt": "", "e": "", ".jpg": "image"} {
|
||||
if got := Kind(name); got != want {
|
||||
t.Errorf("Kind(%q) = %q, want %q", name, got, want)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func TestKeyFollowsTheFile(t *testing.T) {
|
||||
a := Key("/x/a.jpg", 10, 20)
|
||||
if a != Key("/x/a.jpg", 10, 20) || len(a) != 64 {
|
||||
t.Fatal("key is not stable")
|
||||
}
|
||||
for _, b := range []string{Key("/x/b.jpg", 10, 20), Key("/x/a.jpg", 11, 20), Key("/x/a.jpg", 10, 21)} {
|
||||
if b == a {
|
||||
t.Fatal("key ignores part of the file's identity")
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func TestParseVainfo(t *testing.T) {
|
||||
text := `vainfo: Driver version: Intel iHD driver for Intel(R) Gen Graphics - 25.2.3 ()
|
||||
VAProfileMPEG2Simple : VAEntrypointVLD
|
||||
VAProfileH264Main : VAEntrypointVLD
|
||||
VAProfileH264High : VAEntrypointEncSliceLP
|
||||
VAProfileJPEGBaseline : VAEntrypointVLD
|
||||
VAProfileHEVCMain10 : VAEntrypointVLD
|
||||
VAProfileHEVCMain444_12 : VAEntrypointVLD
|
||||
VAProfileVP9Profile2 : VAEntrypointVLD
|
||||
VAProfileAV1Profile0 : VAEntrypointEncSliceLP
|
||||
`
|
||||
got := parseVainfo(text)
|
||||
want := []string{"h264", "hevc", "mjpeg", "mpeg2video", "vp9"}
|
||||
if len(got) != len(want) {
|
||||
t.Fatalf("got %v, want %v", got, want)
|
||||
}
|
||||
for i := range want {
|
||||
if got[i] != want[i] {
|
||||
t.Fatalf("got %v, want %v", got, want)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func TestPickAndEngines(t *testing.T) {
|
||||
igpu := GPU{API: "vaapi", Device: "/dev/dri/renderD128", Codecs: []string{"h264", "hevc"}}
|
||||
arc := GPU{API: "vaapi", Device: "/dev/dri/renderD129", Codecs: []string{"av1", "h264", "hevc"}}
|
||||
c := Caps{FFmpeg: "ffmpeg", FFprobe: "ffprobe", GPUs: []GPU{igpu, arc}}
|
||||
if g := c.pick("av1"); g == nil || g.Device != arc.Device {
|
||||
t.Fatal("av1 should go to the card that lists it")
|
||||
}
|
||||
if g := c.pick("h264"); g == nil || g.Device != igpu.Device {
|
||||
t.Fatal("h264 should go to the first card that lists it")
|
||||
}
|
||||
if g := c.pick("vc1"); g != nil {
|
||||
t.Fatal("a codec nobody lists is software")
|
||||
}
|
||||
// a card nothing could describe is tried, after the ones that answered
|
||||
c.GPUs = append(c.GPUs, GPU{API: "cuda", Device: "0"})
|
||||
if g := c.pick("vc1"); g == nil || g.API != "cuda" {
|
||||
t.Fatal("an undescribed card is the guess of last resort")
|
||||
}
|
||||
// videos: GPU then CPU; images: CPU only
|
||||
if e := engines(c, "video", fileInfo{codec: "hevc"}); len(e) != 2 || e[0] == nil || e[1] != nil {
|
||||
t.Fatal("a video tries the GPU, then the CPU")
|
||||
}
|
||||
if e := engines(c, "image", fileInfo{codec: "mjpeg"}); len(e) != 1 || e[0] != nil {
|
||||
t.Fatal("an image is CPU only")
|
||||
}
|
||||
}
|
||||
|
||||
func TestAttempts(t *testing.T) {
|
||||
a := attempts(nil, "video", fileInfo{duration: 120}, 256)
|
||||
if len(a) != 2 || a[0].seek != 12 || a[1].seek != 0 {
|
||||
t.Fatalf("video attempts = %+v", a)
|
||||
}
|
||||
if a := attempts(nil, "video", fileInfo{duration: 7200}, 256); a[0].seek != 30 {
|
||||
t.Fatalf("seek capped at 30 s, got %v", a[0].seek)
|
||||
}
|
||||
if a := attempts(nil, "video", fileInfo{duration: 2}, 256); len(a) != 1 || a[0].seek != 0 {
|
||||
t.Fatalf("a short clip is taken from its start, got %+v", a)
|
||||
}
|
||||
if a := attempts(nil, "image", fileInfo{codec: "mjpeg", w: 6000, h: 4000}, 256); len(a) != 2 || a[0].lowres != 2 || a[1].lowres != 0 {
|
||||
t.Fatalf("jpeg attempts = %+v", a)
|
||||
}
|
||||
if a := attempts(nil, "image", fileInfo{codec: "mjpeg", w: 800, h: 600}, 256); len(a) != 1 || a[0].lowres != 0 {
|
||||
t.Fatalf("a small jpeg decodes whole, got %+v", a)
|
||||
}
|
||||
if a := attempts(nil, "image", fileInfo{codec: "png", w: 6000, h: 4000}, 256); len(a) != 1 || a[0].lowres != 0 {
|
||||
t.Fatalf("only jpeg has lowres, got %+v", a)
|
||||
}
|
||||
}
|
||||
|
||||
func TestCache(t *testing.T) {
|
||||
c := Cache{Dir: t.TempDir()}
|
||||
k := Key("/a", 1, 2)
|
||||
if _, _, found := c.Get(k); found {
|
||||
t.Fatal("empty cache found something")
|
||||
}
|
||||
if err := c.Put(k, []byte("jpeg")); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if b, _, found := c.Get(k); !found || string(b) != "jpeg" {
|
||||
t.Fatal("put then get")
|
||||
}
|
||||
k2 := Key("/b", 1, 2)
|
||||
if err := c.PutFailure(k2, "no video stream"); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if b, why, found := c.Get(k2); !found || b != nil || why != "no video stream" {
|
||||
t.Fatal("a failure is remembered with its reason")
|
||||
}
|
||||
if err := c.Put(k, nil); err == nil {
|
||||
t.Fatal("an empty preview must not be kept")
|
||||
}
|
||||
if left, _ := filepath.Glob(filepath.Join(c.Dir, "*", ".tmp-*")); len(left) != 0 {
|
||||
t.Fatal("temp files left behind")
|
||||
}
|
||||
}
|
||||
|
||||
// The real thing, when ffmpeg is here: a PNG and a generated video each come back as a
|
||||
// JPEG no larger than the asked size, with the aspect kept.
|
||||
func TestMakeWithFFmpeg(t *testing.T) {
|
||||
if _, err := exec.LookPath("ffmpeg"); err != nil {
|
||||
t.Skip("no ffmpeg")
|
||||
}
|
||||
c := Probe(context.Background())
|
||||
if c.Error != "" {
|
||||
t.Skip(c.Error)
|
||||
}
|
||||
dir := t.TempDir()
|
||||
img := image.NewRGBA(image.Rect(0, 0, 640, 360))
|
||||
for y := 0; y < 360; y++ {
|
||||
for x := 0; x < 640; x++ {
|
||||
img.Set(x, y, color.RGBA{uint8(x / 3), uint8(y), 90, 255})
|
||||
}
|
||||
}
|
||||
pngPath := filepath.Join(dir, "a.png")
|
||||
f, _ := os.Create(pngPath)
|
||||
_ = png.Encode(f, img)
|
||||
f.Close()
|
||||
res, err := Make(context.Background(), c, pngPath, Opts{Size: 200})
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
check(t, res, 200, 112, "cpu")
|
||||
|
||||
// a two-second test pattern, with the always-built-in mpeg4 encoder
|
||||
vid := filepath.Join(dir, "v.mp4")
|
||||
out, err := exec.Command("ffmpeg", "-nostdin", "-v", "error", "-f", "lavfi", "-i", "testsrc=size=320x180:rate=10:duration=2", "-c:v", "mpeg4", "-q:v", "5", vid).CombinedOutput()
|
||||
if err != nil {
|
||||
t.Skipf("could not make a test video: %s", out)
|
||||
}
|
||||
res, err = Make(context.Background(), c, vid, Opts{Size: 100})
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
check(t, res, 100, 56, "")
|
||||
if res.Codec != "mpeg4" || res.Width != 320 {
|
||||
t.Fatalf("probe said %s %dx%d", res.Codec, res.Width, res.Height)
|
||||
}
|
||||
|
||||
// not media, and media that isn't
|
||||
if _, err := Make(context.Background(), c, filepath.Join(dir, "x.txt"), Opts{}); err != ErrUnsupported {
|
||||
t.Fatalf("a .txt is unsupported, got %v", err)
|
||||
}
|
||||
bad := filepath.Join(dir, "bad.mp4")
|
||||
_ = os.WriteFile(bad, []byte("not a video at all"), 0o644)
|
||||
_, err = Make(context.Background(), c, bad, Opts{})
|
||||
var de *DecodeError
|
||||
if err == nil || !isDecodeErr(err, &de) {
|
||||
t.Fatalf("garbage should be a DecodeError, got %v", err)
|
||||
}
|
||||
// the clock: a cancelled context is not the file's fault
|
||||
ctx, cancel := context.WithTimeout(context.Background(), time.Nanosecond)
|
||||
defer cancel()
|
||||
if _, err := Make(ctx, c, pngPath, Opts{}); err == nil || isDecodeErr(err, &de) {
|
||||
t.Fatalf("a timeout must not read as a decode failure, got %v", err)
|
||||
}
|
||||
}
|
||||
|
||||
func check(t *testing.T, res Result, w, h int, engine string) {
|
||||
t.Helper()
|
||||
im, err := jpeg.Decode(bytes.NewReader(res.JPEG))
|
||||
if err != nil {
|
||||
t.Fatalf("not a jpeg: %v", err)
|
||||
}
|
||||
if b := im.Bounds(); b.Dx() != w || b.Dy() < h-1 || b.Dy() > h+1 { // the short side rounds either way
|
||||
t.Fatalf("got %dx%d, want %dx%d", b.Dx(), b.Dy(), w, h)
|
||||
}
|
||||
if engine != "" && res.Engine != engine {
|
||||
t.Fatalf("engine %q, want %q", res.Engine, engine)
|
||||
}
|
||||
}
|
||||
|
||||
func isDecodeErr(err error, de **DecodeError) bool {
|
||||
x, ok := err.(*DecodeError)
|
||||
if ok {
|
||||
*de = x
|
||||
}
|
||||
return ok
|
||||
}
|
||||
@@ -64,6 +64,9 @@ func New(sudo bool, tm *tasks.Manager) *Manager {
|
||||
return m
|
||||
}
|
||||
|
||||
// PM is the package manager found: apt | dnf | zypper | pacman | apk, or "".
|
||||
func (m *Manager) PM() string { return m.pm }
|
||||
|
||||
func (m *Manager) sudoArgs(args ...string) []string {
|
||||
if m.sudo {
|
||||
return priv.Wrap(args...)
|
||||
|
||||
+86
-5
@@ -1,8 +1,9 @@
|
||||
import { DragEvent, useCallback, useEffect, useRef, useState } from 'react'
|
||||
import { api, FileEntry, FileInfo } from './api'
|
||||
import { DragEvent, useCallback, useEffect, useMemo, useRef, useState } from 'react'
|
||||
import { api, FileEntry, FileInfo, ThumbStatus } from './api'
|
||||
import { fmtBytes } from './fmt'
|
||||
import { Btn, Loading, Spinner, askText, askYes, setUnsaved, toast, useEscape } from './ui'
|
||||
import { useHashSel } from './hashsel'
|
||||
import TaskOutput from './TaskOutput'
|
||||
|
||||
const join = (dir: string, name: string) => (dir === '/' ? '/' + name : dir + '/' + name)
|
||||
const fmtTime = (ms: number) => { const d = new Date(ms); const now = new Date(); return d.toDateString() === now.toDateString() ? d.toLocaleTimeString([], { hour: '2-digit', minute: '2-digit' }) : d.toLocaleDateString(undefined, { year: d.getFullYear() === now.getFullYear() ? undefined : 'numeric', month: 'short', day: '2-digit' }) }
|
||||
@@ -14,7 +15,8 @@ function icon(e: FileEntry): string {
|
||||
if (e.type === 'dir') return e.target ? '🔗' : '📁'
|
||||
if (e.type === 'link') return e.broken ? '⛓️💥' : '🔗'
|
||||
if (e.type === 'other') return '⚙️'
|
||||
if (isImage(e.name)) return '🖼️'
|
||||
if (e.preview === 'image' || isImage(e.name)) return '🖼️'
|
||||
if (e.preview === 'video') return '🎞️'
|
||||
if (/\.(zip|gz|bz2|xz|zst|tar|tgz|7z|rar)$/i.test(e.name)) return '🗜️'
|
||||
if (/\.(sh|bash|zsh|py|go|js|ts|tsx|rb|pl|php|rs|c|h|cpp|java|lua)$/i.test(e.name) || e.mode.includes('x')) return '📜'
|
||||
if (/\.(log|txt|md|json|ya?ml|toml|ini|conf|cfg|env|service|timer)$/i.test(e.name)) return '📄'
|
||||
@@ -22,6 +24,74 @@ function icon(e: FileEntry): string {
|
||||
}
|
||||
|
||||
type Sort = 'name' | 'size' | 'mtime'
|
||||
const PREVIEWS_KEY = 'hamlab.files.previews'
|
||||
|
||||
// ---- previews ----
|
||||
|
||||
// Rows ask for their thumbnail only as they scroll near: a browser's own lazy loading
|
||||
// watches the window, and this list scrolls inside a box of its own. One observer for the
|
||||
// list, a callback per row, each fired once.
|
||||
class Near {
|
||||
private obs: IntersectionObserver
|
||||
private cbs = new Map<Element, () => void>()
|
||||
constructor(root: Element) {
|
||||
this.obs = new IntersectionObserver((ents) => {
|
||||
for (const e of ents) if (e.isIntersecting) { const cb = this.cbs.get(e.target); if (cb) { this.forget(e.target); cb() } }
|
||||
}, { root, rootMargin: '400px 0px' })
|
||||
}
|
||||
watch(el: Element, cb: () => void) { this.cbs.set(el, cb); this.obs.observe(el); return () => this.forget(el) }
|
||||
private forget(el: Element) { this.cbs.delete(el); this.obs.unobserve(el) }
|
||||
}
|
||||
|
||||
function Thumb({ url, kind, near, fallback }: { url: string; kind: 'image' | 'video'; near: Near | null; fallback: string }) {
|
||||
const ref = useRef<HTMLSpanElement>(null)
|
||||
const [show, setShow] = useState(false)
|
||||
const [failed, setFailed] = useState('')
|
||||
useEffect(() => { const el = ref.current; if (!el || !near || show) return; return near.watch(el, () => setShow(true)) }, [near, show])
|
||||
useEffect(() => { setFailed('') }, [url]) // a changed file is a new URL, and worth another try
|
||||
const broken = failed === url
|
||||
return (
|
||||
<span ref={ref} className={'fthumb ' + kind} title={broken ? 'no preview for this file' : undefined}>
|
||||
{show && !broken && <img src={url} alt="" decoding="async" onError={() => setFailed(url)} />}
|
||||
{(!show || broken) && <span className="ficon">{fallback}</span>}
|
||||
</span>
|
||||
)
|
||||
}
|
||||
|
||||
// What the previews button says about how they get made here, and the one-tap fixes.
|
||||
function ThumbNote({ st, sudo, onChanged }: { st: ThumbStatus | null; sudo: boolean; onChanged: () => void }) {
|
||||
const [task, setTask] = useState<{ id: string; restart: boolean } | null>(null)
|
||||
const [waiting, setWaiting] = useState(false)
|
||||
// the GPU fix ends by restarting Ham Lab: poll until it answers again, then re-read the status
|
||||
useEffect(() => {
|
||||
if (!waiting) return
|
||||
const t = setInterval(async () => { try { await api.thumbStatus(sudo, true); setWaiting(false); onChanged() } catch { /* restarting */ } }, 3000)
|
||||
return () => clearInterval(t)
|
||||
}, [waiting, sudo, onChanged])
|
||||
if (!st) return null
|
||||
const blocked = (st.unusable ?? []).filter((g) => g.permission)
|
||||
const start = async (label: string, fn: () => Promise<{ id: string }>, restart: boolean) => {
|
||||
try { const t = await fn(); setTask({ id: t.id, restart }) } catch (e) { toast(`${label} — ${e instanceof Error ? e.message : 'failed'}`, 'err') }
|
||||
}
|
||||
return (
|
||||
<>
|
||||
{!st.ok && (
|
||||
<div className="thumb-note notice">
|
||||
<span>Previews need ffmpeg: {st.error || 'it is not installed on this machine'}.</span>
|
||||
{st.install && !task && <Btn className="btn on" onClick={() => start('install', api.thumbInstall, false)}>Install ffmpeg with {st.install.pm}</Btn>}
|
||||
</div>
|
||||
)}
|
||||
{st.ok && blocked.length > 0 && (
|
||||
<div className="thumb-note notice">
|
||||
<span>{blocked.map((g) => g.name || g.device).join(', ')} is here, but Ham Lab runs as {st.user} and can't open {blocked.length > 1 ? 'them' : 'it'} ({blocked[0].reason}). Previews of videos are on the CPU until it can.</span>
|
||||
{st.gpu_fix && !task && !waiting && <Btn className="btn on" confirm="Ham Lab adds itself to the GPU's group and restarts. Open terminals are interrupted. Continue?" onClick={() => start('GPU access', api.thumbGPUAccess, true)}>Allow GPU access</Btn>}
|
||||
{waiting && <span className="muted small"><Spinner /> Ham Lab is restarting; the page reconnects by itself.</span>}
|
||||
</div>
|
||||
)}
|
||||
{task && <TaskOutput id={task.id} onDone={(t) => { const ok = t.exit_code === 0 && !t.error; if (ok && task.restart) setWaiting(true); else if (ok) onChanged(); else toast('failed — the output says where', 'err'); setTask(null) }} />}
|
||||
</>
|
||||
)
|
||||
}
|
||||
|
||||
// ---- editor / viewer ----
|
||||
function Viewer({ path, sudo, onClose, onChanged }: { path: string; sudo: boolean; onClose: () => void; onChanged: () => void }) {
|
||||
@@ -118,6 +188,13 @@ export default function Files({ onOpenInTerminal }: { onOpenInTerminal?: (cmd: s
|
||||
const [drag, setDrag] = useState(false)
|
||||
const fileInput = useRef<HTMLInputElement>(null)
|
||||
const canSudo = useRef(true)
|
||||
const [previews, setPreviews] = useState<boolean>(() => { try { return localStorage.getItem(PREVIEWS_KEY) === '1' } catch { return false } })
|
||||
const [thumbs, setThumbs] = useState<ThumbStatus | null>(null)
|
||||
const [listEl, setListEl] = useState<HTMLDivElement | null>(null)
|
||||
const near = useMemo(() => (listEl && previews ? new Near(listEl) : null), [listEl, previews])
|
||||
const readThumbs = useCallback((refresh = false) => { api.thumbStatus(sudo, refresh).then(setThumbs).catch((e) => setThumbs({ ok: false, engine: '', summary: '', gpus: [], error: e instanceof Error ? e.message : 'failed', user: '', cache_dir: '' })) }, [sudo])
|
||||
useEffect(() => { try { localStorage.setItem(PREVIEWS_KEY, previews ? '1' : '0') } catch { /* private mode */ } }, [previews])
|
||||
useEffect(() => { if (previews) readThumbs(); else setThumbs(null) }, [previews, readThumbs])
|
||||
|
||||
const load = useCallback(async (p = path, s = sudo) => {
|
||||
try {
|
||||
@@ -202,6 +279,7 @@ export default function Files({ onOpenInTerminal }: { onOpenInTerminal?: (cmd: s
|
||||
<div className="files-tools">
|
||||
<button className={'btn' + (sudo ? ' on' : '')} title="browse as root" onClick={() => setSudo(!sudo)}>{sudo ? 'root' : 'sudo'}</button>
|
||||
<button className={'btn' + (hidden ? ' on' : '')} title="show hidden files" onClick={() => setHidden(!hidden)}>.hidden</button>
|
||||
<button className={'btn' + (previews ? ' on' : '')} title={'thumbnails for images and videos' + (thumbs ? ` — ${thumbs.summary}` : '')} onClick={() => setPreviews(!previews)}>previews</button>
|
||||
<select className="btn" value={sort} onChange={(e) => setSort(e.target.value as Sort)}><option value="name">name</option><option value="size">size</option><option value="mtime">modified</option></select>
|
||||
<input className="filter" placeholder="filter" value={q} onChange={(e) => setQ(e.target.value)} />
|
||||
<button className="btn" onClick={mkdir}>+ folder</button>
|
||||
@@ -213,10 +291,11 @@ export default function Files({ onOpenInTerminal }: { onOpenInTerminal?: (cmd: s
|
||||
</div>
|
||||
</div>
|
||||
|
||||
{previews && <ThumbNote st={thumbs} sudo={sudo} onChanged={() => readThumbs(true)} />}
|
||||
{upload && <div className="upload-bar"><Spinner /> {upload.of > 1 ? `${upload.n}/${upload.of} · ` : ''}{upload.name} · {upload.pct}%<div className="meter"><div className="fill" style={{ width: `${upload.pct}%` }} /></div></div>}
|
||||
{err && <div className="error" style={{ padding: '6px 12px' }}>{err}{/permission denied/i.test(err) && !sudo && <> · <button className="link" onClick={() => setSudo(true)}>retry as root</button></>}</div>}
|
||||
|
||||
<div className="file-list">
|
||||
<div className={'file-list' + (previews ? ' thumbs' : '')} ref={setListEl}>
|
||||
{entries === null && <Loading label="Listing…" />}
|
||||
{path !== '/' && entries !== null && (
|
||||
<div className="file-row up" onClick={() => go(path.slice(0, path.lastIndexOf('/')) || '/')}><span className="ficon">↩</span><span className="fname">..</span></div>
|
||||
@@ -226,7 +305,9 @@ export default function Files({ onOpenInTerminal }: { onOpenInTerminal?: (cmd: s
|
||||
const isDir = e.type === 'dir'
|
||||
return (
|
||||
<div key={e.name} className={'file-row' + (e.broken ? ' broken' : '')} onClick={() => (isDir ? go(full) : setOpen(full))}>
|
||||
<span className="ficon">{icon(e)}</span>
|
||||
{previews && e.preview && thumbs?.ok
|
||||
? <Thumb url={api.fileThumbURL(full, sudo, e.mtime, e.size)} kind={e.preview} near={near} fallback={icon(e)} />
|
||||
: <span className="ficon">{icon(e)}</span>}
|
||||
<span className="fname" title={e.target ? `→ ${e.target}` : e.name}>{e.name}{e.target && <span className="muted"> → {e.target}</span>}</span>
|
||||
<span className="fmeta num">{isDir ? '' : fmtBytes(e.size)}</span>
|
||||
<span className="fmeta num">{fmtTime(e.mtime)}</span>
|
||||
|
||||
+9
-1
@@ -249,7 +249,10 @@ export interface ARCDetail {
|
||||
}
|
||||
export interface ARCView { detail: ARCDetail; live: ARCLive | null }
|
||||
|
||||
export interface FileEntry { name: string; type: 'dir' | 'file' | 'link' | 'other'; size: number; mode: string; perm: string; mtime: number; user: string; group: string; target?: string; broken?: boolean }
|
||||
export interface FileEntry { name: string; type: 'dir' | 'file' | 'link' | 'other'; size: number; mode: string; perm: string; mtime: number; user: string; group: string; target?: string; broken?: boolean; preview?: 'image' | 'video' }
|
||||
export interface ThumbGPU { api: string; device: string; name?: string; codecs?: string[]; reason?: string; permission?: boolean }
|
||||
// How previews get made on the machine: ok with an engine, or why not and what would fix it.
|
||||
export interface ThumbStatus { ok: boolean; engine: 'gpu' | 'cpu' | ''; summary: string; ffmpeg?: string; gpus: ThumbGPU[]; unusable?: ThumbGPU[]; error?: string; install?: { pm: string; package: string }; gpu_fix?: boolean; user: string; cache_dir: string }
|
||||
export interface FileInfo extends FileEntry { path: string; text: boolean }
|
||||
|
||||
const q = (path: string, sudo: boolean, extra = '') => `path=${encodeURIComponent(path)}${sudo ? '&sudo=1' : ''}${extra}`
|
||||
@@ -292,6 +295,11 @@ export const api = {
|
||||
},
|
||||
fileReadURL: (path: string, sudo: boolean) => apiPath(`/api/files/read?${q(path, sudo)}`),
|
||||
fileDownloadURL: (path: string, sudo: boolean) => apiPath(`/api/files/download?${q(path, sudo)}`),
|
||||
// v carries the file's mtime and size, so a changed file is a new URL and an unchanged one a cached one
|
||||
fileThumbURL: (path: string, sudo: boolean, mtime: number, size: number) => apiPath(`/api/files/thumb?${q(path, sudo, `&v=${mtime}-${size}`)}`),
|
||||
thumbStatus: (sudo: boolean, refresh = false) => req<ThumbStatus>('GET', `/api/files/thumbs?${sudo ? 'sudo=1&' : ''}${refresh ? 'refresh=1' : ''}`),
|
||||
thumbInstall: () => req<TaskInfo>('POST', '/api/files/thumbs/install'),
|
||||
thumbGPUAccess: () => req<TaskInfo>('POST', '/api/files/thumbs/gpu-access'),
|
||||
fileWrite: async (path: string, text: string, sudo: boolean) => {
|
||||
const r = await fetch(apiPath(`/api/files/write?${q(path, sudo)}`), { method: 'PUT', credentials: 'same-origin', body: text, headers: { 'Content-Type': 'application/octet-stream' } })
|
||||
if (r.status === 401) throw new Unauthenticated()
|
||||
|
||||
@@ -364,8 +364,19 @@ h2.section { margin: 0; font-size: 13px; font-weight: 600; color: var(--muted);
|
||||
.file-editor { flex: 1; min-height: 0; border: 0; border-radius: 0; resize: none; font-size: 13px; }
|
||||
.preview { flex: 1; display: flex; align-items: center; justify-content: center; overflow: auto; padding: 12px; }
|
||||
.preview img { max-width: 100%; max-height: 100%; object-fit: contain; }
|
||||
/* previews: the icon column becomes a thumbnail box for images and videos */
|
||||
.file-list.thumbs .file-row { grid-template-columns: 72px 1fr 80px 80px 130px 32px; padding-top: 4px; padding-bottom: 4px; min-height: 52px; }
|
||||
.fthumb { position: relative; width: 72px; height: 44px; display: flex; align-items: center; justify-content: center; background: var(--bg3); border-radius: 4px; overflow: hidden; }
|
||||
.fthumb img { max-width: 100%; max-height: 100%; object-fit: contain; display: block; }
|
||||
.fthumb.video::after { content: '▶'; position: absolute; right: 2px; bottom: 2px; font-size: 9px; line-height: 1; padding: 2px 3px; border-radius: 3px; background: rgba(0,0,0,.6); color: #fff; }
|
||||
.fthumb .ficon { font-size: 18px; }
|
||||
.thumb-note { display: flex; flex-wrap: wrap; align-items: center; gap: 8px; margin: 6px 10px 0; }
|
||||
.thumb-note .btn { padding-top: 3px; padding-bottom: 3px; }
|
||||
.files-bar .task-out { margin: 6px 10px 0; max-height: 160px; }
|
||||
@media (max-width: 800px) {
|
||||
.file-row { grid-template-columns: 24px 1fr 70px 32px; } .file-row .fmeta:nth-child(4), .file-row .perm { display: none; }
|
||||
.file-list.thumbs .file-row { grid-template-columns: 60px 1fr 70px 32px; min-height: 46px; }
|
||||
.fthumb { width: 60px; height: 38px; }
|
||||
.path-input { width: 100%; margin: 4px 0 0; }
|
||||
.files-tools .filter { width: 80px; }
|
||||
}
|
||||
|
||||
Reference in New Issue
Block a user