# Test gaps: what the test runs did not capture

Running list. Add to it after every run. Strike an item only when it has evidence (a file or a verdict).
Last updated: 2026-11-12, after T8 and the Nov 12 decisions.

## Tests that never ran

| Test | What it checks | Why it was missed | What unblocks it |
|---|---|---|---|
| T2 Storyboard keyframes | Do layout images help H3 frame a shot | Layout images were never made; T2.1 (which 2 shots) never answered. Suggested G1A + G1X1 | Answer T2.1, make the 2 layout images |
| T4 Flat 2D style ("More to the Map") | Caelan's look | Caelan's inputs never landed in `inputs/T4-more-to-the-map` (3 shots, narration, June ref, 2-3 Grenada look images, June on screen yes/no) | Caelan fills T4.1-T4.7 and drops the files |
| T7 Branded shirt | Can a real shirt go on a character | Shirt photo never landed in `inputs/T7-shirt` | Aydan drops the shirt photo |

## Tests that ran but are not finished

| Item | What's missing |
|---|---|
| T1 G1A (patio, two people, voice-driven) | PICKED Nov 11: method B seed 3 (T1B_G1A_s3). Still needs the full-size lock (768x1344). Two speakers in one shot works (Aydan, Nov 12: takes a couple of runs), so no split rule |
| T1 lip sync | Judged by eye only; only word timing was checked with a transcriber |
| T5 Turbo drafts | Never checked whether a turbo pick locks the same at full size (20 steps, same seed). This decides if turbo becomes the default |
| T6 Third still | G2Cfix and G2D never checked frame by frame. No verdict from Aydan |
| T3 Motion control | Ball read as the cup (floats on top). No fix tried. "Different body" use case (Aydan's idea) untested |

## Same Day episode, still open

- No music bed
- Sync only spot-checked: Mike's head drop at 12.5 s lands; top panel chin drop not verified
- B3 range shot: feet and ball not visible in the panel crop
- Mike's middle bottom shots play at 93% speed (watch for it looking slow)

## Things we did not record during the runs

- Final spend never confirmed against the Lium bill (estimate ~$10.85: A ~7.8 h, B and C ~3.2 h each, $0.76/hr)
- No written Test Run 2 results doc on the drive; `Runs/2026-11-10-aydan-video-test-2` holds only intake files and empty input folders
- Reject reasons for unpicked takes were not written down (only the picks)
- No per-shot timing log (draft and final minutes) saved as data; numbers live in chat only
- G1A final from Nov 7 still not picked

## Process gap that caused most of this

Boxes were rented before every test's inputs were in, and nothing flagged the missing inputs. Fix for next run: no rent until each test shows "ready", and each blocked test names who it's waiting on.

## T8 chain test (started Nov 12): sheets -> storyboard -> stills -> H3
- Shots 1B + 2D, start and end frame each.
- Storyboard DONE Nov 12: arm A text only vs arm B with locked sheets as edit refs (S3 ink, gpt-image-2 low, factory worker). Sheet: `work/test_run_2/t8_chain/storyboard/t8_storyboard_A_vs_B.jpg`. Sheets uploaded to `b2://aif-media/remix/t8-chain/sheets/`. Pick A or B is Aydan's.
- Stills: BLOCKED on tall size in the image worker (square + low only). Brief for Caelan: `work/pipeline_review/IMAGE-WORKER-TALL-SIZE-BRIEF.md`.
- Video arms V1 sheets only / V2 + storyboard / V3 + start still / V4 + start+end, 480x864, seeds 1-2. Needs a box + go.
- Also recorded Nov 12: small reference-video feed (480x864) = 1375 s vs big 2327 s for cut 8b; both kept as a choice (H3-RULES s4). Storyboard style locked S3 (H3-RULES s1). T2 all 24 drafts home Nov 12 02:03, not yet reviewed.
- Nov 12 02:15: stills made (sheets + storyboard B, square cropped to 9:16 for this test only, per Aydan). Aydan: detailed storyboards get copied exactly, mistakes included (palms, cart seat, 2 beers, Mike's place); storyboards should be LOOSE (generic figures, block props). Made loose storyboards (pass 2 with explicit layout) + stills from them. Found: palms + 2 beers on one table are IN the locked patio sheet (`sheets/world/loc_patio.png`), so every patio shot inherits them. Sheet fix is Aydan's call.
- Queue on box 2 (auto-terminate pushed to 04:17 local): seed 1 V1-V4 detailed, V1 s2, V2L/V3L/V4L s1 (loose). Watcher: scratchpad pull_t8.sh -> `work/test_run_2/t8_chain/out/`.
- Nov 12 03:47: all 16 T8 drafts home. Labeled: `work/test_run_2/t8_chain/labeled/T8_1B_compare.mp4`, `T8_2D_compare.mp4`.
- **T8 VERDICT (Aydan):** loose storyboard ref (V2L) or loose start still (V3L) are the best of all arms. End still (V4) makes H3 do weird cliche moves to reach the end frame: start only. Recorded in H3-RULES s1.
- Still open from T8: V2L vs V3L as the standard (one image step fewer vs a picture in the real look); only seed 1 per arm, so no seed spread yet.

## Box 2 archive + run data (pulled Nov 12 03:55 ET, before shutdown)
- Everything off the box except model weights: `work/test_run_2/box2_archive_20261112/` (all 45 outputs in `output/tuning/`, runs.jsonl with 36 fields per take, worker + ComfyUI logs, every job list, prompts, workflows, run scripts, `_diag/` = nvidia-smi, pip freeze, ComfyUI commit 1332fabf, model list, CPU/RAM/disk).
- Includes the 5 raw T3F full-size motion-copy clips, which were not on the PC before (only the labeled compares were).
- 45 of 45 takes succeeded, 0 errors, 0 out-of-memory. 9.07 GPU-hours of generation on 2x L40S over about 5 h rented (about 90% busy). Spend about $3.85 at $0.76/h (03:52Z to about 08:55Z), under the $8 cap.
- Timing per take (one L40S, 20 steps), now saved as data:
  - 480x864, 5 s: 3 refs 295 s, 4 refs 311 s.
  - 480x864, 10 s: 4 refs 630 s, 5 refs 652-655 s, 6 refs 673-675 s. Each extra reference costs about 20 s on a 10 s draft.
  - 768x1344, 5 s with a reference video: big feed 2280-2330 s (38-39 min), small feed 1375 s (23 min).
- Peak card memory 45.2 of 46 GB on T2_G1A_start_s1 (10 s, 5 refs, 480x864): a 10 s draft with 6 refs is at the edge of an L40S.
- T8 ran with NO pinned voice on any arm (1B and 2D both silent). The verdict is about layout and motion only; voice + loose still together is untested.

## Nov 12 decisions that open new tests
- Last frame of a picked take as the next shot's start (continuing shots skip the keyframe). Never run.
- V2L vs V3L with more than one seed and with a pinned voice.