From Article to Film: How We Built an AI-Assisted Filmmaking Pipeline

Disclosure: Written with AI assistance from preserved production records with editorial approval from Alejandro Perez. Voice references remained local. Model-license limitations are discussed in the article.

I used AI to help make a film about what AI might do to the value of my work.

The source was my essay “If AI Can Do the Work, What Is Our Work Worth?”. After years in computer graphics, I was asking what our experience is worth when software can perform more of the work we spent so long learning. Turning that question into an AI-assisted production made the contradiction concrete.

Could we preserve the argument, reproduce my voice locally, establish a consistent illustrated world, and sustain it through an entire essay? Could we then extract a process useful for a different article?

The result is a 17-minute documentary, a reusable local toolkit, and a short technical pilot. Getting there involved good narration, weak animation, corrected storyboards, selective generative motion, a substantial editorial pass, and a soundtrack I ultimately rejected. The useful story is how those decisions changed the work—and which parts another creator can adapt.

## Start with a production question

My CGI and Houdini background shaped the approach: separate a difficult result into controllable systems, preserve what works, and make a correction cheap enough to attempt. Houdini itself was not the renderer for this production. The recorded toolchain used Python, local speech synthesis, ComfyUI, image generation, Wan image-to-video, scripted graphics and compositing, and FFmpeg assembly.

There were three distinct roles. Artie, my ChatGPT collaborator, helped with strategy, critical review and production specifications. Codex implemented and tested the local workflows. I supplied direction, evaluated results, and approved or rejected changes. I carried the decisions and specifications between conversations; there was no direct Artie-to-Codex communication.

Both the original feature and the later Pilot 01 were produced with Codex using Astra Extra High. A small Sol comparison happened later on two text tasks. It was not the system that made the pilot.

The constraints were deliberately concrete. Preserve the authored message. Use my authorized voice recording without sending it to an external voice service. Keep the illustrated identity recognizable. Use the workstation and available local models. Minimize repetitive work while retaining editorial authority over the result.

“Local” describes the media processing and voice handling here. Codex and ChatGPT still supplied AI-assisted planning and implementation; this was not an entirely offline intelligence system.

A convincing voice, then an unconvincing film

Experiment 01 covered only the introduction. The text became roughly 95 seconds of narration, assembled from manageable sentence segments using Qwen3-TTS-12Hz-0.6B-Base. A 25.86-second crop of my reference recording supplied the vocal identity. The delivery was reflective and conversational, appropriate to a personal essay rather than a trailer.

Qwen was the practical local choice after inspecting the environment. Chatterbox was considered, but it was not installed and benchmarked against Qwen. This was a workable selection, not a controlled finding that one voice system was superior.

I considered the resulting narration very good. That was my assessment of the performance, not a conclusion established by a metric. The automated checks answered narrower questions: whether expected words appeared in transcription, whether the waveform had suitable levels, and whether the assembled segments had the expected duration. None of those checks alone establishes that a voice sounds like me or that a pause feels natural.

The visual side exposed the first major gap. Sixteen shots used the same illustration, with reframing, zooms and transitions. The image was attractive, but the sequence did not demonstrate characters doing anything. Camera movement created activity around a still picture; it did not create performance.

Experiment 02 isolated two actions: pressing an imaginary button and moving a magical slider. Python and compositing moved the hand and button, activated a screen character, and developed an effect. The actions were real animation, but the flatter procedural graphics did not yet carry the original illustration’s richness. Wan-compatible nodes were present, but a complete ready-to-run model set was not found at that point. We used the available method rather than describing an unavailable generation as a quality failure.

Experiment 03 returned to the full introduction with more imagery and compositing. It improved the coverage, but still did not solve visual consistency and internal motion together. The next useful investment was a storyboard that could survive animation.

Establish the picture before asking it to move

Experiment 04 established sixteen shot-specific illustrations. The visual language combined dimensional painted faces with dark ink contours, selective crosshatching, ivory space, charcoal shadows and orange accents. The references described both the medium and the recurring character; “cinematic illustration” by itself was too vague.

We used the approved original illustration and a facial reference crop, with explicit reference ordering in each workflow. A separate layout guide could describe composition when needed. This avoided relying on an accumulating chain of generated edits, where each new image could inherit another small identity or style error.

Reading the sheet: sixteen storyboard panels establish one recurring character, palette and staging vocabulary. Read left to right, row by row. Use the full-resolution view to inspect individual panels.

I approved that foundation in principle, then asked for three targeted fixes. The production-team scene contained two figures who looked too much like versions of me. The returning button scene repeated an earlier visual beat. A close-up had distracting glasses perspective.

Experiment 05 removed the duplicate-looking protagonist problem, changed the button return into a more useful workstation composition, and corrected the glasses. The other thirteen stills stayed byte-identical. That constraint mattered: improving three weak panels did not justify reopening thirteen successful ones.

Three rows, before → after: Shot 07 removes the duplicate-looking protagonist; Shot 12 changes the returning-button composition; Shot 13 repairs the glasses perspective. Thirteen other panels stay unchanged.

Motion was still primarily compositing: screen changes, prop activation, light, atmosphere and restrained camera movement. When a foreground matte produced pale edges, the intact illustration was preferable. A technically ambitious treatment had to earn its place by improving the picture.

The first genuinely generated performance

Experiment 06 was a controlled test of one existing shot: an artist working alone at night. The brief requested small shoulder and arm movements, changing monitor light, and a largely stable room. No new composition, no unnecessary camera move, no additional characters.

The native ComfyUI workflow used Wan 2.2 I2V 14B FP8, its high- and low-noise models, and the matching LightX2V four-step LoRAs. It produced 89 frames at 16 fps and 768 × 432: about 5.56 seconds. The saved seed was 6000601.

The ComfyUI execution record was 48.72 seconds. Sampled whole-device GPU memory reached 22,208 MiB on the RTX 4090 workstation. That number includes other GPU use and is not a precise measurement of tensors allocated by the model. The machine had 128 GB of system RAM; the experiment did not establish a minimum RAM requirement.

The artist moved inside the image. That was the important result. It also softened fine hatching and hand detail. The comparison was useful precisely because it showed both the gain in presence and the loss of some illustration fidelity.

E05 vs E06 · silent comparison

Open MP4

Experiment 07 extended generated motion across most of the introduction, but local repairs remained necessary. A button cap could drift. A background could invent a passerby. An explosion became a radial, flower-like form. The revised effect was more usable as cooling smoke, especially when confined to its screen, but it was not a physically exact explosion.

These failures changed the shot-planning question. Instead of asking whether a model could animate an entire frame, we asked which part of the frame benefited from generation and which parts needed to remain fixed.

Scaling the method to seventeen minutes

Experiment 08 expanded the introduction into the complete essay. Its cut ran 17:01.125, with 113 shots. The production added 146 narration sentences to the existing introduction, making 162 in total. The new work recorded 75 image attempts and 67 video attempts, selecting 64 new illustration plates and 65 new native animation clips.

Those are different counts for different things. A successful computation is not an accepted take, and an accepted plate can support more than one editorial use. The report’s 109 distinct compositions also counts repeated callbacks differently from the shot count.

The architecture was straightforward:

Article → narration → word timing → shot plan → image plates → motion → compositing → timeline → render.

The narration established a duration we could actually edit against. Shot records then linked passages to narrative purposes, visual methods, selected assets and frame ranges. Generated images supplied settings and characters. Native motion clips supplied selected performances. Python graphics handled exact diagrams and mechanisms. FFmpeg encoded and assembled the result.

The distinction between methods was practical. A small human gesture could benefit from Wan. A diagram needed exact relationships and readable type. A mechanical arm needed stable segment lengths and a gripper that did not multiply. When the generated arm duplicated joint or gripper elements, a procedural rigid-body replacement was the more credible choice.

The feature’s later generated footage was typically 1024 × 576 at 16 fps, assembled into a 1920 × 1080, 24 fps delivery. Upscaling and frame sampling did not make it native 1080p animation. We did not use optical-flow interpolation to imply additional captured motion.

Saved prompts, references, seeds, workflow graphs and model names made individual jobs inspectable. Asset hashes identified exactly which result an edit used. They did not promise that rerunning a neural model on different software would reproduce every pixel. For reliable editorial rebuilding, retaining the accepted outputs was essential.

The recorded Experiment 08 image, video and TTS jobs sum to about 4.54 hours. That is logged generation work, including job execution overhead, not the elapsed making of the film and not my labor. It excludes portions of setup, review, orchestration and rendering. We did not measure a defensible “hours saved” figure.

The edit mattered more than another generation pass

A complete film is not necessarily a well-shaped film. Experiment 09 focused on the opening, pacing, distracting performances and a small amount of narration repair.

The original essay began with a career anecdote. For the film, we moved an existing closing sentence to the beginning:

There’s something ironic about spending much of your career developing tools and automating production workflows, only to find yourself wondering whether automation might eventually make your own work unnecessary.

That sentence gave the viewer the personal contradiction immediately. It became a 305-frame, 12.708-second cold open, supported by images of the creator and the value of work, the button, and an empty desk. The sentence was moved, not duplicated, so the change did not pad the running time.

Cold-open timing at 24 fps: hook_01 uses ch3_21 for frames 0–96 (4.000 s); hook_02 uses intro12 for 96–192 (4.000 s); hook_03 uses ch1_08 for 192–305 (4.708 s). End frames are exclusive; total 305 frames / 12.708 s.

Start here: compare the pictures in silence—original at left, revised at right. The archived header says “audio: after”; this excerpt has no audio. Open the narrated versions below to hear each opening.

Original vs revised picture · 30 seconds · silent

Open MP4

Hear the two openings with their own narration

Original opening · its own narration

Open MP4 · Read transcript · WebVTT captions

Revised opening · its own narration

Open MP4 · Read transcript · WebVTT captions

The opening review considered what a viewer understood after five, fifteen, thirty and sixty seconds. These were editorial checkpoints, not a measured audience-retention study. We cannot attribute a retention increase to this change without appropriate viewing data.

Elsewhere, a long team composition became a wide shot followed by a detail. A few similar splits created emphasis using existing coverage. The final edit had 120 shots, but the objective was not simply more cuts. A cut should reveal something, clarify a relationship, or move attention when the narration needs it.

Generated mouth movements were particularly troublesome. The film had voiceover, not characters delivering dialogue. A figure apparently talking could create an unintended second performance. In selected shots, quieter coverage or a still was more coherent than preserving motion just because it had been generated.

The archived “audio: after” header is retained as evidence; this comparison excerpt is silent.

Performance correction · silent comparison

Open MP4

Audio corrections were similarly limited. Four local gain reductions and one continuous retake of the passage about adaptation addressed specific issues. The edit report records 17.11 seconds of changed waveform, with the remaining approved material preserved after the opening rearrangement. It would be inaccurate to say Experiment 09 made no narration changes; it would be equally misleading to imply the entire voice track was regenerated.

A repaired hand and a rejected soundtrack

Experiment 10 produced a particularly clear example of useful restraint. One pointing hand was distracting in an otherwise successful image. We edited a crop, resized the candidate back into place, and blended it through a localized mask.

The final correction changed 38,075 pixels out of 2,359,296, approximately 1.61% of the original plate. Every pixel outside the nonzero mask remained unchanged. That is a verifiable claim about the operation, rather than a vague assurance that the rest of the scene “looked the same.”

The shot remained a corrected illustration with its restrained camera push. We did not solve the hand by inventing a new animated gesture or regenerating the whole group.

The same experiment developed synthesized music and sound effects. I rejected that treatment. A mix can render correctly, meet technical requirements and still be the wrong choice for the piece. The reflective narration was the element I wanted to carry into the release.

Experiment 11 therefore combined Experiment 10’s corrected picture with Experiment 09’s narration-only audio. The release script copied the selected streams rather than re-encoding them. Packet-level checks verified the picture against its picture source and the audio against its audio source. The accepted master did not contain the rejected music or effects.

That separation let us keep a successful visual repair without accepting everything else produced in the same experiment. A production should retain enough structure to make that possible.

Extracting a reusable workflow

After the feature, we extracted Article-to-Film Pipeline v1.0. The recurring production decisions became stage boundaries, shared records, caches, checks, recovery rules and delivery packaging. That is the bridge from this particular film to a workflow another creator can adapt. The practical walkthrough shows how to begin; the stages below explain what must be ready before the next step can succeed.

The six stages are:

1. Preflight and rights: establish the source, intended use, permissions and available resources.
2. Script and story plan: preserve the text, identify beats, and describe what each shot contributes.
3. Narration and timed storyboard: generate or reuse approved speech, check wording and cadence, then establish timing and continuity.
4. Generation and assembly: run cleared jobs, review takes, register selected assets and build the film.
5. Targeted polish: correct specific editorial problems while preserving successful material.
6. QA and delivery: validate media, captions and provenance; package a reproducible project and present the remaining decisions.

Roles: Alejandro directs and accepts; Artie helps with strategy/specifications; Alejandro carries the handoffs; Codex implements and invokes the toolkit. Stages: 1 preflight/rights → 2 script/plan → 3 narration/storyboard → 4 generation/assembly → 5 targeted polish → 6 QA/delivery. Shared manifests, selected assets and caches connect them.

Stable sentence and shot IDs connect those stages. The toolkit stores article passages, narration, shots, assets, timeline, jobs, QA and delivery as machine-readable records. Its timeline uses integer frame boundaries with an exclusive end frame, avoiding accumulating ambiguity about where one shot stops and another starts.

Hashes let the system distinguish unchanged inputs from revised ones. Caches allow an approved narration take or neighboring shot to survive a correction. Review status is separate from job completion, so a rejected image can remain in the record without becoming eligible for the edit.

Recovery deserves the same attention as generation. If a ComfyUI submission loses its response, the system must first determine whether the job was accepted. It records submission state and uses the job’s identifiers and history for recovery. Blindly resubmitting could spend GPU time twice or confuse which result belongs to the edit.

The master prompt makes the limit explicit: “Use the CLI scaffold only as a starting point; fill every narrative purpose/description.” The command-line program does not call a language model to invent and approve a film. The Codex session supplies orchestration and creative implementation; the scripts execute defined operations and enforce checks.

The first validation used 39 implementation tests and 579 historical regression frames: the 305-frame opening and the 274-frame hand shot. That verified important engineering behavior, including bounded cached reconstruction. It did not demonstrate a fresh article becoming a film. That required another test.

Pilot 01: a different article, a different test

“The Gray Between Black and White” began with a new, approved 129-word article. The resulting pilot runs 63.083 seconds, with eight shots and ten narration sentences. It combines local speech, precise technical graphics, a generated character plate and a brief Wan performance.

The central demonstration compares averaging equal black and white values in encoded sRGB with averaging them in linear light. With normalized black at zero, white at one, equal weights and no tone mapping, the encoded average is 0.5, or approximately 128 in eight-bit values. A linear-light average of 0.5 becomes approximately 0.73536 after the sRGB transfer function, or 188.

That result describes those assumptions, not every color pipeline. The calculation uses the piecewise sRGB function, not a blanket gamma-2.2 shortcut. Blending color in an appropriate linear working space and applying the intended display transform is different from treating roughness or normal maps as display-encoded color. The W3C color-conversion reference documents the transfer-function form used in the check.

Those diagrams needed deterministic values and typography. Generation was reserved for the illustrated character. The pilot made three visual-generation attempts: two images and one video. The first image’s smooth finish and visible hands were rejected. The second had stronger ink treatment but still required a crop excluding hands. Its 41-frame, 16 fps video added a short blink and head/eye movement; the edit held its settled ending where needed.

One editorial correction pass revised two shots: speech-aligned highlighting made the workflow diagram easier to follow, and a reframing improved the character shot. The other six shot clips remained byte-identical. No new neural generation or narration was needed for that pass.

Complete Pilot 01 · 63 seconds · narration

Open MP4 · Read transcript · WebVTT captions

Native Wan take · 2.56 seconds · silent

Open MP4

The technical results were strong within their scope: complete media decode, expected frame count, exact coverage of the 129 words, and a packaged replay that rebuilt the 1,514-frame film byte-identically using retained assets. The pilot’s reviewed engine copy also added a narrowly scoped adapter for the different image workflow, bringing its test count to 43. That extension was not silently promoted into the preserved v1.0 toolkit.

I had approved the article and narration. Final visual and editorial acceptance remained pending in the delivery record. A technical pass does not erase that distinction.

The Sol/Astra comparison was smaller still: one paired invocation per model, each covering a shot-plan task and a repair-plan task. Both passed the recorded hard constraints without a corrective turn. It was not a blind quality study, a cost comparison or competing full-film productions. It does not establish equivalence or a generally preferable production model.

A workflow another creator can adapt

1. Prepare the source and the working environment

Begin with a short article you can actually evaluate. Save the exact approved text as UTF-8, retaining paragraph boundaries. Keep proposed rewrites separate. Record the title, audience, factual references and rights assumptions. If the argument is unclear on paper, animation gives you more material to repair rather than resolving it automatically.

You need a configured copy of our toolkit, its Python runtime and FFmpeg, plus the local speech and ComfyUI environments for any generation you intend to use. Our historical workstation had an RTX 4090 with 24 GiB VRAM and 128 GB system RAM; ComfyUI 0.43.1 and FFmpeg 7.1 were recorded versions, not universal requirements. Check model, encoder, VAE and node compatibility, available memory, and permissions for the intended use. Test one representative job before planning a sequence around it.

Prepare an approved style image, an identity crop, palette/material notes and any layout guide, with explicit reference-slot ordering. Keep an authorized clean voice recording and checked transcript local. Listen to a short narration test using the article’s vocabulary before generating the full text.

2. Start a project, then complete its plan

pipeline.ps1 belongs to our implemented local Article-to-Film toolkit. It is not a universally available command or a public installation service. Run these examples from a configured toolkit root; substitute your own paths:

.\pipeline.ps1 init projects/new-film --article 'C:\local\article.txt' --title 'Film title'
.\pipeline.ps1 run projects/new-film --stage 01
.\pipeline.ps1 run projects/new-film --stage 02
.\pipeline.ps1 status projects/new-film

These commands create and check records; they do not invent and approve a film. Preflight may correctly report a rights or resource blocker. The Codex session helps develop the plan and implement graphics; the scripts execute defined operations. The author still decides what the film should communicate and whether a result works.

Fill each shot’s narrative purpose, method and asset choices. At five seconds, what question is visible? At fifteen, what promise has the piece made? At thirty and sixty, is it delivering useful development? Choose procedural graphics for exact text, values and mechanisms; stills for compositions that already carry an idea; generated motion for useful small performances; and compositing when only one region needs to move.

3. Establish accepted narration and timing

Use the implemented TTS and transcription steps, listen to the result, and record genuine review evidence before accepting it. Stage 03 then connects the narration and timed storyboard. A transcription match helps find omissions, but it cannot judge likeness or cadence. The technical appendix contains the full command sequence and its prerequisites.

4. Keep the records connected

shots.json identifies what each shot needs; assets.json records selected outputs and their provenance; timeline.json places them on the frame grid. jobs.json retains generation settings, while state.json records task fingerprints and cached results. Stable IDs join these records. Hashes help preserve unchanged work, but a cache hit is not creative approval. Register reviewed assets before Stage 04 assembly.

Set an attempt budget and keep the prompt, seed, references, graph and result together. Retain rejected takes with specific reasons: “gripper duplicates,” “mouth appears to speak,” or “style became smooth plastic.” A completed job is evidence of execution, not permission to put its output in the edit.

5. Recover an interruption without duplicating work

If a submission loses its response, inspect the saved attempt and the owned ComfyUI server’s history first. Recover the known job rather than assuming it never ran:

.\pipeline.ps1 comfy projects/new-film JOB_ID --recover

JOB_ID must identify a real project job. If the attempt has no prompt_id, locate it in server history by client_id and record the verified prompt ID first; recovery stops until that ambiguity is resolved. Recovery reconciles its recorded submission; it does not improve a rejected take. For rejection, record the reason and prepare a distinct, bounded attempt if the budget allows. Keep the rejected output out of the selected asset list. Do not erase history, weaken resource guards or blindly resubmit.

6. Correct only what needs correction

Assemble against accepted narration timing. Inspect shot entrances and settled endings, not just an attractive middle frame. Check diagrams at phone size. For a correction, identify the shot and timecode, state the problem, define the smallest acceptable change, and preserve the successful neighbors.

A sentence-specific TTS selector and a shot-specific render selector exist in the toolkit; their examples are in the appendix. Reuse accepted audio and asset bytes where possible. Pilot 01 demonstrates the principle: two shot revisions, six neighboring clips unchanged, and no additional neural generation.

7. Validate, screen and package

After the targeted polish, run Stage 06, validate the selected assets and create the delivery package:

.\pipeline.ps1 run projects/new-film --stage 06
.\pipeline.ps1 validate projects/new-film --assets
.\pipeline.ps1 package projects/new-film

Technical QA checks decoding, duration, word coverage, caption timing, levels, hashes and reconstruction. Editorial review checks meaning, performance, rhythm, visual clarity and whether the work sounds and feels like its author. Screen the film, record that review’s actual scope, and label pending acceptance. Preserve the selected assets needed for rebuilding while keeping raw voice references and model weights outside the deliverable. A reproducible package and an accepted film are separate achievements.

What I would change next time

Licensing belongs before production, not near delivery. The historical Qwen Image 2.1 model carries a research license with a separate commercial-permission requirement. I chose to proceed with the existing artwork while acknowledging an unresolved interpretation; that decision was not independent legal clearance. The later toolkit consequently blocks new use of that model under its commercial preset. Qwen Image 2.1 license.

Pilot 01 used the separately reviewed Qwen Image Edit 2511 workflow. Its upstream model card identifies Apache 2.0, and the pilot recorded checkpoint hashes and matched converted weights to their upstream provenance. That does not retroactively change the historical model’s license or settle rights in every reference image. The different model, conversion, dependencies and intended use still need their own record. Qwen Image Edit 2511 model card.

I would also lock references earlier, classify animation methods before generating, and give the first minute more attention before filling the rest of the timeline. Technical graphics should be designed for a small screen from the beginning. Approved narration should be treated as an asset to preserve, with retakes confined to actual problems.

The manifests and caches already demonstrated a useful capability: two pilot shots could change while six stayed identical, and a retained-asset package could rebuild its film. Broader claims about productivity remain unmeasured. Logged machine time, total elapsed time, my active attention and monetary cost are different quantities. The pilot’s recorded window included orchestration, review waits and other tasks; it cannot support a simple speedup comparison with the feature.

Future tests should measure those quantities separately across more than one new article. They should also record editorial acceptance, not merely successful renders. A pipeline that produces an editable result and exposes its weaknesses is useful; one that hides uncertainty behind a green completion badge is difficult to trust.

What the experiment actually demonstrated

We made and delivered a coherent film from a personal essay, then used the extracted workflow on a different technical article. Local models supplied usable speech, illustrations and selected motion. Scripts handled precise graphics, assembly, preservation and checks. Editorial decisions determined what survived.

The process still depended on experience: recognizing when a performance distracted, when a diagram needed mathematical control, when a soundtrack did not belong, and when a successful image should simply be left alone.

That brings me back to the original question. As tools take over more operations, our work includes deciding what is worth making, understanding what a result communicates, and taking responsibility for what we release. This experiment gave me a practical way to explore that responsibility. It also gave CGI Nerd a process we can test further, with its successes and limitations still visible.

Keep exploring

• If AI Can Do the Work, What Is Our Work Worth? · Journal