The scene changed. An 8x8 grid of brightness caught it.
Framepick pulls one keyframe per scene from any clip, each stamped with its timecode, decoding the whole video in your browser tab with WebCodecs. No upload, no ffmpeg flags.
A cut in a video is a moment where almost every pixel changes at once. The frame before and the frame after share nothing: different room, different light, different faces. That single property is enough to find every scene boundary in a clip without understanding a thing about what's on screen, and it's cheap enough to run on 64 numbers per frame.
That is roughly what Framepick does. You drop a video on the page and it hands back one keyframe per scene, a contact sheet of the shots that actually represent the clip, each stamped with its exact timecode. The file never leaves your machine.
Why this was annoying before
If you want the representative frames from a clip today, you have two options and both are
friction. Upload it to a web tool, which ingests your footage to a server, runs ffmpeg, and
hands back thumbnails, meaning your private or unreleased video now sits on someone else's
disk. Or run ffmpeg yourself with -vf "select='gt(scene,0.4)'", which works if you have
ffmpeg installed, remember the incantation, and enjoy tuning the threshold by hand. Neither is
something you send to a non-technical collaborator.
Framepick does the whole thing in the browser tab, on-device. It demuxes the container with
mp4box.js, feeds the encoded chunks to a hardware-accelerated WebCodecs VideoDecoder, and
walks the actual decoded frames. The decode runs in a Web Worker, off the UI thread, so the
page stays responsive while a 200 MB clip gets analyzed with zero bytes crossing the network.
The "server" is your own GPU.
Signatures, not pixels
Here's the part worth knowing even if you never open the app. Comparing full frames to find cuts would be slow and, worse, jumpy: film grain and compression noise wiggle individual pixels constantly, so a pixel-diff spikes on things that aren't cuts at all.
So Framepick doesn't compare pixels. Each sampled frame (about four per second) gets reduced
to an 8x8 grid of average brightness, sixty-four numbers, a fingerprint of the frame's
large-scale light and dark structure. Averaging over each cell throws away exactly the
high-frequency noise that fools a pixel-diff, while keeping the layout that actually changes
at a cut. Then it's just a distance between two 64-value vectors. Where that distance spikes,
a scene boundary gets marked; Framepick picks one establishing frame per scene and drops the
near-duplicate shots so the contact sheet is shots, not noise. The math lives in
framework-free, unit-tested modules (src/lib/signature.js, src/lib/scenes.js), so the
interesting logic is verifiable in Node without a browser anywhere in the loop.
One knob, and it doesn't re-decode
There's a single sensitivity slider: more keyframes or fewer. Users think in that, not in distance thresholds. The nice property is what happens when you move it. The expensive work is the decode, pulling frames off the GPU, and that already produced the cached signatures. Re-segmenting is just re-running the cut detection over sixty-four-number vectors that are already in memory. So the slider responds live, re-cutting the whole video the instant you drag it, with no second decode. Cheap comparisons are what make that feel instant.
The honest limitation is the browser. WebCodecs is required, so you need Chrome or Edge 94+ or a recent Safari; Framepick detects an unsupported browser on load and says so plainly rather than failing silently. Containers and codecs are bounded by what the browser's decoder plus mp4box.js can handle, which is MP4 and MOV with H.264, HEVC, or AV1 as the platform allows. A DRM-protected or otherwise undecodable file surfaces a designed error state instead of a spinner that never stops. And Framepick extracts, it doesn't edit: no trimming, no audio analysis, no transcription. It pulls the frames out and gets out of the way.
Try it
Open Framepick, drop in any clip with a few scene changes, and watch the contact sheet fill in as it decodes. Drag the sensitivity slider and notice the sheet re-cut instantly, no reload, no re-decode. Click a keyframe to save it as a PNG, or grab the whole set as a zip for a shot list. Do it with your network panel open if you want proof: nothing uploads. The code is on GitHub if you'd rather read the signature math than take my word for it.
This post is part of the build log: every app my automated factory ships gets written up here, honestly. Browse everything at apps.charliekrug.com. Comments are open below.
Loading comments…