DriftHaven: Building an AI Video Pipeline on One 16 GB GPU (Every Wall I Hit)

DriftHaven is an AI video production pipeline I built to publish cinematic 4K ambience videos — cozy interiors, fantasy realms, wide natural landscapes — as a fully automated overnight system on a single consumer GPU. One machine, one RTX 5070 Ti with 16 GB of VRAM, one world rendered per night while I sleep.

This post is the iteration log, including everything that failed first. Especially everything that failed first.

The constraint that designed the product

Before any code: YouTube's "inauthentic content" policy explicitly targets mass-produced, templated content, and enforcement happens at the channel level — one batch of templated slop can demonetize everything. Dozens of big AI-slop channels have been removed under it.

That policy is DriftHaven's real design document. The goal was never "generate maximum videos"; it was use AI to produce handcrafted-feeling work faster — original visuals, layered original audio, real variation between videos. Every technical decision below traces back to that line.

The pipeline

Per world, the chain runs: a Flux still sets the scene → FramePack animates it into a short video loop → a script makes the loop seamless → a 4K upscale pass → multiple generated audio layers get mixed into an original soundscape bed → a one-pass branded render (watermark, subscribe animations, endcard) straight to the delivery codec → the finished long-form video and a teaser get enqueued for upload. A nightly orchestrator runs one world end-to-end at 1 AM; a separate worker drains the upload queue serially so uploads never fight for bandwidth. Every stage skips if its output already exists, so a crashed night resumes instead of restarting.

The architectural idea that makes 3-hour videos affordable on consumer hardware: the final video is a stream-copy concatenation of a short unit. Anything expensive — interpolation, film grain, detail synthesis — is done once to the unit and amortized across three hours. Spend freely on the unit; never add per-frame work to the full runtime.

The walls, in order

The 60 GB problem. The first render path produced a ~60 GB H.264 master per 3-hour video, taking ~12 hours to upload. The fix was switching delivery to AV1 and rendering once, directly to the delivery codec, with no intermediate master: ~17 GB and ~3.5 hours. Later, dark nature scenes (night skies, mist) showed banding at the initial bitrate, so it went up a notch — still roughly a third of the H.264 size.

The emoji that took down production. The 1 AM run kept dying before the generation server even bound its port. Root cause: launched headless on Windows, Python falls back to the cp1252 codec — and a custom node's import banner contained a 🚀. The orchestrator's own log line with a → arrow crashed it the same way. A UnicodeEncodeError, from decorative characters, in strings nobody reads. Fix: force UTF-8 stdio everywhere, at every entry point. If you automate anything headless on Windows, do this preemptively.

The VRAM ceiling hunt. The wide landscape worlds were configured to generate at a base resolution of 960 — and rendered at 550–788 seconds per iteration, a six-hour ETA for one loop. Nothing errored; it was just impossibly slow. The cause: Windows itself holds ~2.6 GB of VRAM for the desktop, so the model spilled into system-memory fallback and quietly crawled instead of crashing. Tested down the ladder: 960 failed, 832 still fell back (174 s/it), 768 ran clean at ~12–32 s/it. The hard ceiling on a "16 GB" card is not 16 GB. Silent performance cliffs are worse than crashes — instrument seconds-per-iteration, not just success.

GPU rendering: abandoned on evidence. The obvious optimization — move the ffmpeg compositing to the GPU — turned out slower than CPU, because that ffmpeg build couldn't alpha-blend on GPU and the hybrid path added transfer overhead. The render wasn't the bottleneck anyway. It stays CPU. Optimizations have to earn their complexity with measurements.

The upscaler realization. The generator is VRAM-capped around 480–768p, so the 4K output quality is decided almost entirely by the upscaler, not the generator. The fast interpolative upscaler (RealESRGAN) smooths rather than invents texture — grass and rock read soft. The upgrade path is a detail-synthesizing tiled diffusion upscale at very low denoise: it adds real texture while staying faithful enough to avoid temporal flicker in a seamless loop, and tiling keeps it inside 16 GB.

Killing the slideshow tell. Raw generated loops run at 16 fps, which reads as "AI slideshow." Frame interpolation to 32 fps on the unit — again, a one-time cost amortized over three hours — plus film-grain encoding got the output from "generated" to "produced." The current quality phase chains continuation segments into multi-minute unique motion and composites static true-4K regions with moving ones, so a three-hour video is no longer thirty seconds repeated 360 times.

Where it stands

The pipeline is built, validated end-to-end, and decoupled into render → queue → publish. It runs one world per night unattended, and the quality upgrades (interpolation, grain, chaining, composites) shipped and are being validated against real renders. The channel side is deliberately unhurried — the thesis is brand and consistency over volume, and the pipeline finally produces at the quality bar the thesis demands.

The meta-lesson, since this is ostensibly a portfolio: AI pipelines are 10% model selection and 90% systems engineering — encoders, queues, encodings, VRAM accounting, and Unicode. The models were never the hard part.