# Video-driven canvas scrub — POC notes

Branch: `poc/video-scrub`. This is a **proof of concept**, not a production port.
The goal is to test whether replacing the 1,331-frame WebP sequence with a
single scrubbed video fixes the cold-start stutter both static and React
versions of the film currently exhibit.

## What changed vs `main`

| | `main` (frame-based) | `poc/video-scrub` |
|---|---|---|
| Root document | `index.html` — the full film experience, 1,511 lines | New minimal scrub demo, ~200 lines. Original preserved as `index-frames.html`. |
| Source assets | 1,331 WebP files under `frames/v{1..12}/` (~100 MB total) | Same frames, PLUS `film.webm` and `film.mp4` at repo root, encoded from those frames |
| Scrub mechanism | `new Image()` per frame, sliding-window preload, `drawImage(img)` on scroll | `<video>` element, seek via `video.currentTime`, `drawImage(video)` on `seeked` / `requestVideoFrameCallback` |
| HTTP requests | ~1,331 (browser fetches on demand, capped at ~6-8 parallel per origin) | 1 (progressive stream via HTTP range requests) |
| Cold-start wait | Long — buffer needs to build across many small files | Short — first ~2s of video streams and playback-ready |

## Why the switch could help

The stutter you experienced comes from two things happening at once:

1. **HTTP concurrency ceiling.** Browsers cap parallel requests per origin at 6-8.
   With 1,331 files needed, the browser is queuing constantly during scroll —
   any frame not yet fetched shows the previous one until it arrives.
2. **Total bytes over the wire.** ~100 MB of frames means even on a fast
   connection, buffer takes 8-20 seconds to fill enough for smooth playback.

A single video file addresses both:

- **One TCP/QUIC connection**, one file → no HTTP concurrency ceiling.
- **Massive byte reduction.** Video codecs (VP9, H.264) exploit temporal
  redundancy — consecutive frames share most pixels, so only the deltas ship.
  Estimated: 100 MB → **5–20 MB** (10–30× reduction). Confirmed after encoding.
- **Progressive stream.** Video starts playing after ~1–2 seconds of buffer;
  seek-ahead-of-play works with byte-range requests.

## How to run the POC

You need ffmpeg with `libvpx-vp9` and `libx264` support. Lightest install (works
on macOS/Linux/Windows, no admin, ~60 MB, deletable with one `rm -rf`):

```bash
npm install --no-save ffmpeg-static
```

The build script auto-detects it under `node_modules/ffmpeg-static/`. Or install
system-wide:
- macOS: `brew install ffmpeg`
- Linux: `apt install ffmpeg` (or your distro equivalent)
- Windows: `winget install ffmpeg` (built-in on Win10/11) or `choco install ffmpeg`
- Windows portable: pre-built binary from https://www.gyan.dev/ffmpeg/builds/ → point `FFMPEG` env var at it

```bash
git checkout poc/video-scrub
npm install --no-save ffmpeg-static   # or skip if ffmpeg already on PATH
./scripts/build-film-video.sh    # ~1-5 min depending on machine
# This creates film.webm + film.mp4 at the repo root.

# Optional: commit them to preview on Cloudflare Pages branch deploy.
git add film.webm film.mp4
git commit -m "poc: build encoded film video from frames"
git push origin poc/video-scrub

# Or preview locally:
npx serve .          # or any static file server
# Open http://localhost:3000/
```

The video files are gitignored by default (see `.gitignore`) to keep the
branch light. If you want the CF Pages preview URL to work without a build
step, commit them.

## Browser compatibility

The scrub technique — `video.currentTime = t` + `drawImage(video)` on `seeked`
— works everywhere modern (Chrome, Edge, Firefox, Safari 15+). Real-world
gotchas:

| Browser | Seek precision @ 24fps | Notes |
|---|---|---|
| Chrome / Edge | Frame-accurate | `requestVideoFrameCallback` helps |
| Firefox | Frame-accurate | `requestVideoFrameCallback` supported |
| Safari 16+ (desktop + iOS) | Frame-accurate | `requestVideoFrameCallback` in 16.4+ |
| Safari 15 (older iOS) | Slight jitter possible | `seeked` fallback path handles it |
| Mobile browsers on slow networks | Depends on buffer | First few seconds of scrub can be janky until video buffered ahead |

Notes for a full production port:
- Set `preload="auto"` so the browser aggressively buffers the video.
- Muted + `playsinline` are required for iOS to allow autoload without user gesture.
- The `<source>` order matters — WebM first so modern browsers pick VP9 (smaller);
  Safari falls through to MP4.
- `moovflags +faststart` on the MP4 is critical — without it, Safari waits for
  the entire file to download before decoding.

## Open questions before productionizing

1. **Do we want frame-accurate seek or is time-accurate enough?**
   The current frame-based version snaps to specific frame indices. Time-based
   seek at 24fps is usually within ±1 frame. For the "pool settling on the final
   clip" (`POOL_FRAME` in index-frames.html), we may want to pad an extra second
   in the encoded video so we always land on the pool shot even under seek jitter.

2. **How to preserve the intro clip.**
   The clone loads a separate `v111` clip during the keyhole-reveal phase (see
   `introCache` on line 887 of index-frames.html). Options: (a) include v111 in
   the encoded video and start scrubbing there, (b) keep v111 as a separate short
   video, (c) drop the intro and start straight in v1. Design decision.

3. **Captions and quicknav are timing-driven from `currentFrame`.**
   In the frame-based version, `CAPS` array is built with `s`/`e` frame ranges
   (line 1030 of index-frames.html). Those ranges map cleanly to video time
   (`s / FPS`, `e / FPS`). Full port needs to update the caption engine's frame
   → time conversion — one function change.

4. **Preload strategy.**
   Video `preload="auto"` doesn't tell us HOW aggressively the browser buffers.
   For instant-scrub-ready UX, we may want to explicitly buffer with
   `fetch(url, { headers: { Range: 'bytes=0-2000000' } })` to force the first
   ~2 MB of the video into HTTP cache before setting `<source>`. Only if the
   default `preload="auto"` isn't aggressive enough.

## Rough performance estimate — filled in after encoding

*Fill after running `scripts/build-film-video.sh`:*

| Metric | Frame-based | Video-scrub |
|---|---|---|
| Total bytes | ~100 MB | *?* MB |
| HTTP requests | 1,331 | 2 (webm + mp4 sources, one used) |
| Time to first paint | ? s | ? s |
| Time to smooth scrub across full film | ? s | ? s |

Fill this table on a real network trace, add screenshots.

## If we like this — the full port

Not part of this POC branch. If Carasso decides to go video-scrub for
production, the follow-up is straightforward:

1. Encode `film.webm` + `film.mp4` (script here).
2. In `Crasso-Nadlan/summer/src/lib/film.ts` (or equivalent static version),
   swap the frame-preload loop for a `<video>` element + seek scheduling.
3. Keep captions, quicknav, lead bar, modal — all still scroll-driven.
4. Test on iOS Safari + slow-3G mobile.
5. Ship to `exclusivedeal.carasso-nadlan.co.il`.

Estimated effort for the full port: **~1 day** (including QA on real devices).
Estimated bytes-served reduction: **10–30×** on cold visits.