Engineering story
AI Video Generation Pipeline
An agentic AI pipeline that turns creative briefs into production-ready videos while keeping humans in control.
In one sentence
I built an agentic AI pipeline that plans, generates, reviews, and assembles videos with human approval built into every stage.
Role
Founder
Owned pipeline architecture, agent orchestration, evaluation criteria, and human-in-the-loop design.
Role
Founder
Timeline
2025–2026
Status
Prototype
Platform
Python · GCP
Team
Solo founder
Users
Internal testing
- Idea
- Stage interfaces
- Agent orchestration
- Human checkpoints
- Prototype runs
Why this problem mattered
Content teams need leverage, not fully autonomous black boxes. Manual production is slow and expensive. Naive automation fails on quality, brand consistency, and reviewability.
I wanted a system that accelerates the repetitive work while preserving human judgment where it still matters most—structure, taste, and final quality gates.
Overview
This pipeline transforms creative briefs into structured scripts, generated assets, and rendered video. Each stage emits artifacts for review—so teams get leverage from automation without surrendering editorial control.
What I owned
I led the project end-to-end as a solo founder, from defining the pipeline stages to building orchestration, evaluation, and review interfaces.
- Pipeline architecture
- Agent orchestration
- Human-in-the-loop checkpoints
- Evaluation criteria
- Artifact interfaces between stages
- Composition and QA hooks
Goals
- Automate repetitive production steps
- Preserve human judgment at key checkpoints
- Keep stages swappable as models improve
- Make every run inspectable and reproducible
Engineering
Highlights
How it works
Every brief follows the same path—plan, script, review, generate assets, compose, then QA—so quality stays reviewable even as models change.
Agent orchestration runs over modular stages. Each stage emits artifacts for human review before the next step begins. Composition uses FFmpeg, with QA hooks before publish.
- Brief ingestion
- Planning agent
- Script generation
- Human review
- Asset generation
- Composition / FFmpeg
- QA hooks
Example workflow
A single request walks through the full pipeline—from brief to final render—with review built in.
- Input"Create a 5-minute video about quantum computing."
- PlannerCreates the outline and section structure
- ResearchCollects facts and supporting references
- ScriptWrites narration for each section
- AssetsGenerates images and supporting media
- Human approvalReview structure, tone, and brand fit
- RenderingAssembles the timeline with FFmpeg
- Final MP4QA hooks before publish
Why not just ChatGPT?
A single LLM prompt wasn't enough because video production is a workflow rather than a single task. Splitting the work into specialized agents made outputs easier to review, debug, and improve.
Monolithic generation hides failure. Modular stages surface it—at the script, the assets, or the final compose—before anything ships.
Product & engineering decisions
Human-in-the-loop by default
Why
Quality and brand consistency still need judgment. Full autonomy looks impressive until the first off-brand render ships.
Pros
- Editorial control at every stage
- Safer brand outcomes
- Clearer accountability
Tradeoff
Control over full autonomy.
Artifact-first stage interfaces
Why
Monolithic prompts are hard to evaluate, debug, or swap. Explicit artifacts make each stage inspectable and replaceable.
Pros
- Easier evaluation
- Swappable models and tools
- Reproducible runs
Tradeoff
Pipeline complexity for modularity and reviewability.
Evaluation as a first-class concern
Why
Generative systems fail quietly. Without pass/fail signals, teams can't trust automation enough to use it.
Pros
- Clearer quality gates
- Faster iteration on prompts and models
- Less blind trust in outputs
Tradeoff
More upfront design—but evaluation is the product for generative systems.
Challenges I had to solve
Challenge
Stabilizing multi-model workflows.
Solution
Modular stages with explicit artifacts, retries, and inspectable outputs.
Result
Runs become reproducible and easier to debug.
Challenge
Defining pass/fail quality signals.
Solution
Treat evaluation criteria as first-class pipeline concerns—not afterthoughts.
Result
Clearer gates before render and publish.
Challenge
Keeping humans useful without becoming a bottleneck.
Solution
Checkpoints only where judgment matters—structure, brand, and final quality.
Result
Automation handles volume; people protect taste.
Current snapshot
Outcomes
Automation
Scripts, assets, and editing stages
Human review
Every stage
Repeatability
High — inspectable artifacts per run
Primary users
Creators and content teams
Content quality
Protected by checkpoints, not left to chance
Current stage
Prototype architecture
Lessons learned
Building this pipeline reinforced that evaluation is the product for generative systems—and modular stages beat clever monolithic prompts every time.
- Evaluation is the product for generative systems.
- Modular stages beat monolithic prompts.
- Human checkpoints should protect taste—not rubber-stamp every step.
- If a run isn't inspectable, it isn't production-ready.
What I'd do differently
- I'd define brand-kit constraints as inputs from day one.
- I'd build automatic QA scorers earlier alongside human review.
- I'd invest sooner in golden-set evaluations for script and composition quality.
Where I'd take it next
If I continued investing here, I'd focus on:
- Stronger automatic QA scorers
- Brand kit constraints as first-class inputs
- Faster iteration loops for script and asset stages
- Better cost and latency observability per stage
- Reusable templates for common content formats
Biggest takeaway
AI isn't difficult because it can generate content. It's difficult because production systems must generate content that people can trust, review, and improve.
← All projects