Latest / AI Fire Daily / #89 Robin: The AI Video Pipeline - Storyboard-First Prompting, Agentic Workflowsi

Transcript

#89 Robin: The AI Video Pipeline - Storyboard-First Prompting, Agentic Workflowsi · AI Fire Daily

• 0:00
  You create a perfect AI video scene. The lighting
• 0:03
  is beautifully moody. The framing is exact. But
• 0:08
  then you generate the next shot. The character's
• 0:12
  face just changes completely. The camera unexpectedly
• 0:16
  spins out of control. The entire illusion just
• 0:19
  breaks. It's like the AI woke up with amnesia.
• 0:23
  It's incredibly frustrating, isn't it? The whole
• 0:25
  cohesive narrative just, you know, falls apart
• 0:28
  instantly. Welcome to our deep dive. I'm really
• 0:31
  glad you're here with us today. Absolutely. We
• 0:33
  are exploring a very specific structural solution
• 0:36
  today. It's called a storyboard -first workflow.
• 0:39
  Right. This framework basically solves that notoriously
• 0:42
  messy AI video problem. We're going to cover
• 0:44
  the manual chatbot method first. Yeah, that helps
• 0:47
  you really learn the underlying mechanics. Exactly.
• 0:49
  Then we'll shift gears. We'll look at the agentic
• 0:51
  method. That's how you actually scale up your
• 0:53
  production. Which is where it gets really fun.
• 0:55
  Whether you want to make flashy product ads or,
• 0:57
  say, Pixar -style cartoons, this matters. This
• 1:01
  is your ultimate shortcut. It helps you avoid
• 1:03
  massively wasted time, and it saves your expensive
• 1:07
  generation credits. Yeah, because the default
• 1:10
  way of making AI video is just structurally broken.
• 1:14
  It fails completely when you need more than a
• 1:16
  single isolated shot. Let's unpack that a bit.
• 1:19
  What's the underlying mechanism here? Why does
• 1:21
  it get so messy so fast? Well, think about prompting
• 1:24
  shot -by -shot from scratch. You ask a neural
• 1:27
  network for a scene. It hallucinates a beautiful
• 1:30
  image based on your text. Right. Then you ask
• 1:32
  for the next chronological scene, but text is
• 1:35
  notoriously vague. The style inevitably slips.
• 1:38
  The characters, they warp into completely different
• 1:41
  people. Right. And the AI is basically just guessing.
• 1:43
  Exactly. It is completely random because it lacks
• 1:46
  a shared visual memory between those separate
• 1:49
  requests. It reminds me of a terrible, chaotic
• 1:52
  movie production. Ho -ho. Imagine a director
• 1:54
  shooting a two -second clip of a scene. Then,
• 1:57
  immediately after, they fire the entire film
• 2:00
  crew. Wow, okay. They fire the director of photography,
• 2:02
  they fire the actors, the lighting technicians,
• 2:05
  they hire a completely new, totally different
• 2:07
  crew. That sounds like a nightmare. They do this
• 2:09
  absurd reset for every single shot. There's simply
• 2:12
  no institutional memory. That's a perfect way
• 2:15
  to visualize it. There's zero continuity. You
• 2:17
  end up spinning all your generation credits just
• 2:21
  trying to fix the inconsistencies. Yeah. And
• 2:24
  even then, the video still feels totally unfinished
• 2:26
  and disjointed. It's an endless cycle of trial
• 2:30
  and error. That brings us to the actual solution,
• 2:32
  the storyboard -first workflow. This changes
• 2:35
  the entire paradigm. A storyboard gives your
• 2:38
  AI video generator one clear, undeniable visual
• 2:42
  guide. You create this visual anchor before any
• 2:46
  animation happens. It locks the important details
• 2:48
  in. The main character, the sequential scene
• 2:51
  order, the specific camera angles, the overall
• 2:54
  visual composition. It is all explicitly mapped
• 2:56
  out right there in one single image. Right. That
• 2:59
  single image becomes the foundational anchor.
• 3:01
  The final video feels so much more consistent.
• 3:03
  Yeah. You move from shot to shot seamlessly,
• 3:07
  mostly because the AI isn't guessing anymore.
• 3:09
  I have a question about this underlying architecture.
• 3:12
  Why do these models struggle so much with consistency?
• 3:15
  I mean, why do they fail without a visual anchor?
• 3:19
  Text instructions are just too vague for neural
• 3:21
  networks. Words leave way too much room for visual
• 3:24
  interpretation. Ah, I see. An image, however,
• 3:27
  provides strict mathematical boundaries for the
• 3:30
  pixels. It forces the model to stay strictly
• 3:32
  on track. So the image acts as a hard boundary
• 3:36
  for the AI's imagination. Exactly. It completely
• 3:38
  reigns in the chaos. Now we know exactly why
• 3:41
  we need a storyboard. But we still need to build
• 3:43
  one. We should definitely do it by hand first.
• 3:46
  We really have to understand the mechanics. Then
• 3:48
  we can confidently automate it. The manual method
• 3:51
  is always the best teacher. You only need three
• 3:54
  basic structural parts here. A text chatbot,
• 3:57
  an image generator, and a dedicated video AI
• 4:00
  generator. Let's start with the first phase.
• 4:02
  Setting up the brain of the operation, the chatbot.
• 4:04
  You have a few distinct options here based on
• 4:07
  your needs. Yeah, JAT GPT is generally the best
• 4:09
  overall choice. It works great if you already
• 4:11
  use it daily. Claude is another really strong
• 4:14
  option. It is exceptionally good for detailed,
• 4:17
  logical reasoning. It handles long, complex instructions
• 4:21
  beautifully. And there is a highly capable free
• 4:24
  option too. Yeah, DeepSeek. It's absolutely the
• 4:26
  best free pick right now. There's no credit card
• 4:29
  required at all. Oh, that's great. There's no
• 4:31
  frustrating paywall blocking you. So the chat
• 4:33
  bot essentially handles the hard logistical thinking.
• 4:37
  You just creatively guide the main idea. Right.
• 4:40
  Normally, you have to meticulously write everything
• 4:43
  yourself. The specific camera direction, the
• 4:46
  exact scene timing, the ambient sound design.
• 4:49
  It gets overwhelming really fast, especially
• 4:52
  for beginners just starting out. You feel pressured
• 4:55
  to write the perfect prompt. Let's quickly define
• 4:58
  that term for clarity. A prompt is simple. Instructions
• 5:01
  you type to tell the AI exactly what to make.
• 5:04
  Right. And writing those instructions consistently
• 5:06
  is a difficult art form. I have to admit something
• 5:08
  here. I still wrestle with prompt drift myself
• 5:10
  when I try to write these by hand. Oh, totally.
• 5:12
  My ideas start focused, but the AI just wanders
• 5:15
  off. Beat. We all do. Human language is inherently
• 5:18
  messy. That's exactly why we use the Chatbot
• 5:21
  as a middleman. Yeah. It basically becomes your
• 5:23
  dedicated prompt writer. It turns your rough
• 5:26
  creative idea into a rigidly structured format.
• 5:30
  And better structured prompts naturally mean
• 5:32
  fewer bad visual output. Exactly. You waste significantly
• 5:35
  less time. You do far fewer frustrating retries.
• 5:38
  So how do we actually start this process? You
• 5:41
  need the master instruction file. It's called
• 5:43
  the Chatbot PDF in the source materials. OK.
• 5:45
  you open the designated storyboard folder in
• 5:48
  Google Drive, you download that specific PDF
• 5:51
  file. Let's talk about the logic here. What exactly
• 5:54
  is happening when we use that file? So you drag
• 5:56
  it in your chat window, you just type run this
• 5:58
  and send it. What you're actually doing is loading
• 6:00
  a complete logical framework into the AI. It's
• 6:03
  setting the rigid rules of the game. It really
• 6:06
  is that simple. The PDF contains the full workflow
• 6:09
  logic. You don't have to explain the production
• 6:11
  process from zero. That saves so much time. Yeah,
• 6:13
  the chatbot reads the document thoroughly. It
• 6:16
  instantly understands exactly how to build a
• 6:18
  visual narrative structure. So it's not just
• 6:21
  a normal conversational assistant anymore, it
• 6:23
  essentially becomes a miniature production system.
• 6:27
  Then the chatbot actively asks you a question.
• 6:30
  It wants to know your desired storyboard type.
• 6:32
  You choose either the ad or non -ad pathway.
• 6:35
  Ads are obviously for specific product videos,
• 6:38
  slick brand clips, punchy short promos. And the
• 6:42
  non -ad pathway is for everything else. Cartoons.
• 6:45
  Flowing anime. short narrative stories, engaging
• 6:48
  educational videos. You just pick what fits your
• 6:51
  specific goal. Yeah. Then the chatbot generates
• 6:53
  10 distinct ideas. This is incredibly amazing
• 6:57
  for beginners. Staring at a completely blank
• 6:59
  page is notoriously hard. It really is paralyzing
• 7:02
  sometimes. It is. The chatbot might suggest an
• 7:05
  animated cat learning to cook a meal. or perhaps
• 7:07
  a Pixar -style educational video explaining the
• 7:09
  entire water cycle. You can even upload a specific
• 7:12
  reference image, right? Like maybe you have a
• 7:14
  particular character design already in mind.
• 7:16
  The chat bot uses that visual to carefully build
• 7:18
  the prompt. It saves an absolutely massive amount
• 7:22
  of time. You just review the lift and pick the
• 7:25
  core idea you like best. Once you have your idea,
• 7:27
  we move to the next phase, generating the actual
• 7:30
  storyboard image. The chat bot gives you a highly
• 7:33
  detailed visual prompt. You take that specific
• 7:35
  prompt block, you paste it directly into an image
• 7:39
  generator. The source materials show ChatGPT
• 7:42
  actually gave the strongest storyboards. The
• 7:45
  visual panels were remarkably clean. The characters
• 7:48
  stayed highly consistent across the grid. That
• 7:50
  makes perfect logical sense. The exact same underlying
• 7:53
  model wrote the prompt. It deeply understands
• 7:56
  its own internal structure and formatting quirks.
• 7:59
  What about testing the other available tools?
• 8:02
  Gemini and Google Flow can definitely work, but
• 8:05
  they require much more careful checking. Really?
• 8:07
  Yeah, sometimes the visual outputs look oddly
• 8:09
  distorted. The generated panels might not strictly
• 8:12
  follow the requested grid structure. So the strategy
• 8:15
  is to save a few different variants. Generating
• 8:17
  two to four versions is a good practice. One
• 8:20
  specific layout might animate far better than
• 8:22
  the others. Right. You're not just looking for
• 8:24
  a single pretty image. You want clear, readable
• 8:28
  camera flow and coherent action across the panels.
• 8:31
  The final phase of this manual process is the
• 8:34
  magic. Yeah. Generating the actual animation.
• 8:37
  You go back to your active chatbot window. You
• 8:40
  simply type next. The chatbot then gives you
• 8:42
  the final animation prompt. This explicitly tells
• 8:45
  the video generator how to move the pixels. Nice.
• 8:48
  It includes precise camera movement, the action
• 8:51
  timing. the overall mood. You take your chosen
• 8:54
  storyboard image, you take that new animation
• 8:56
  prompt, you put both of them into your chosen
• 8:58
  video tool. The comparative testing here is really
• 9:01
  fascinating. Gemini using VO3 completely failed
• 9:04
  the test. Wait, really? Yeah, it just fundamentally
• 9:07
  did not generate the requested video. Wow. Google
• 9:10
  Flow using Omniflash was considered passable.
• 9:12
  It worked okay for those standard ad style videos,
• 9:15
  but the generated sound felt highly repetitive.
• 9:18
  It really struggled with complex non -ad character
• 9:20
  animation. The clear undisputed winner was Hicksfield,
• 9:24
  specifically using their Seedens 2 .0 model.
• 9:27
  It consistently gave the absolute cleanest result.
• 9:30
  The characters remained remarkably consistent.
• 9:32
  It had very low visual distortion during movement.
• 9:35
  So, you open the Higgs field platform, you upload
• 9:38
  your clean storyboard image, you carefully paste
• 9:41
  the animation prompt, you select the Seidens
• 9:44
  2 .0 model, then you generate the final video.
• 9:48
  If you truly want the cleanest possible result,
• 9:51
  use Seidens. That is undoubtedly the definitive
• 9:54
  first choice. I have to ask something here about
• 9:56
  the philosophy of this. Does relying so heavily
• 9:59
  on this master PDF limit your own creative input?
• 10:03
  Not at all. The PDF simply structures the underlying
• 10:06
  logistical format. You are still the one dictating
• 10:09
  the core narrative idea. The AI just handles
• 10:12
  the formatting. The PDF is the scaffolding, but
• 10:14
  you still build the actual house. Perfectly said.
• 10:16
  You repeat this manual flow for every single
• 10:19
  new video. It deeply teaches you how the system
• 10:21
  actually works under the hood. You watch the
• 10:24
  core idea become a structured prompt. The prompt
• 10:26
  becomes a visual storyboard. The storyboard dynamically
• 10:30
  becomes a moving video sponsor. We are back!
• 10:35
  We just thoroughly covered the manual method.
• 10:38
  It's an undeniably great teacher, but constantly
• 10:40
  copying and pasting gets incredibly painful.
• 10:43
  Especially if you want to produce 50 videos a
• 10:45
  week. Yeah. The friction is just too high. Right.
• 10:47
  We need to evolve from being a manual laborer
• 10:50
  to being a true director. We need to implement
• 10:53
  the agentic method. This is where things get
• 10:55
  really fascinating. We bring in AI agents. Okay.
• 10:59
  These aren't just chat bots that wait for text.
• 11:01
  They intelligently automate the entire production
• 11:04
  pipeline. They proactively suggest ideas. Wow.
• 11:07
  They autonomously write the prompts. They generate
• 11:09
  the storyboards themselves. They even route the
• 11:11
  files directly to the video generator. You essentially
• 11:13
  just give the agent a broad goal. Then you simply
• 11:17
  check the final preview. There are a few really
• 11:19
  good agent software options available. Codex
• 11:22
  is highly recommended as very beginner friendly.
• 11:24
  The interface feels a whole lot like standard
• 11:26
  chat GPT. Claude code is another powerful one.
• 11:29
  It gives you incredibly deep project control.
• 11:33
  But it can definitely cost a bit more to run.
• 11:35
  And then there's anti -gravity. That is considered
• 11:38
  the best mostly free setup available. If your
• 11:41
  production budget matters, absolutely start right
• 11:44
  there. The initial software setup is pretty normal.
• 11:47
  You download it, you install it on your machine,
• 11:49
  you securely sign in. But you really need to
• 11:52
  strategically organize things first. Yes, you
• 11:55
  have to create a deeply organized workspace.
• 11:58
  The Autonomous Agent needs a structured place
• 12:00
  to confidently save and retrieve files. It acts
• 12:03
  as the agent's digital brain. You make a specific
• 12:05
  folder strictly for the storyboards. Another
• 12:08
  isolated folder solely for the final videos.
• 12:10
  Right. You need one designated for the text prompts
• 12:12
  and finally one folder for the ultimate exports.
• 12:15
  After you build that structure, you choose your
• 12:17
  operational mode. You select either manual mode
• 12:19
  or auto mode. I want to push back on this a bit.
• 12:22
  If we use this auto mode and the agent does everything,
• 12:25
  do we lose the creative soul of the project by
• 12:28
  letting the AI run so freely? I completely see
• 12:32
  why you might feel that way. It's a very valid
• 12:34
  philosophical concern. Yeah. But think of manual
• 12:37
  mode as your essential training wheels. It teaches
• 12:40
  you the basic mechanics. Auto mode is strictly
• 12:42
  for scaling your broader vision. So you're not
• 12:45
  out of the loop. Exactly. You are still the core
• 12:47
  director. You still set the overall aesthetic
• 12:49
  vision and boundaries. So we download the designated
• 12:52
  agent PDF. We upload it directly to the agent's
• 12:55
  interface. We type run this to initiate the sequence.
• 12:59
  The agent thoroughly reads the file. It autonomously
• 13:02
  sets up the entire project environment. It might
• 13:05
  even install necessary software dependencies.
• 13:08
  It prepares all those folders we just talked
• 13:10
  about. It then asks you clarifying questions.
• 13:13
  Maybe it asks which specific engine backbone
• 13:16
  to use for generation. You carefully choose your
• 13:19
  desired technical path. You no longer handle
• 13:21
  every tedious setup step manually. The software
• 13:25
  agent literally becomes your dedicated technical
• 13:27
  operator. You choose the add or non -add pathway
• 13:30
  again. The agent proactively suggests creative
• 13:32
  ideas. Or... You can explicitly ask for something
• 13:36
  highly specific. Like a vibrant orange iPhone
• 13:39
  product ad. Or a cute clumsy monster desperately
• 13:42
  trying to get some grapes. The monster makes
• 13:44
  a huge mess. I absolutely love that visual example.
• 13:47
  It's great. The agent seamlessly handles the
• 13:49
  next steps completely automatically. It expertly
• 13:52
  writes the prompt. It securely generates the
• 13:55
  image. It instantly shows you a visual preview.
• 13:58
  Whoa. Imagine scaling to a billion queries. Or,
• 14:02
  in this case, generating a whole animated series
• 14:05
  while you sleep to sex silence. That level of
• 14:08
  autonomous scale is genuinely hard to grasp.
• 14:11
  If you like the generated storyboard, you just
• 14:14
  intuitively type next. And then? The agent then
• 14:16
  autonomously creates the final video. It conveniently
• 14:19
  gives you a direct preview link. If it looks
• 14:21
  fundamentally wrong, you simply type regenerate.
• 14:24
  You can give highly specific feedback, right?
• 14:26
  Oh yeah. Like, make the monster look cuter, use
• 14:28
  much warmer lighting colors, make a significantly
• 14:31
  cleaner four -panel storyboard grid. The agent
• 14:34
  actively fixes it based on your notes. You are
• 14:36
  never permanently stuck with the very first result
• 14:38
  it spits out. You can even ask for multiple variants
• 14:41
  at once. This specific capability is incredibly
• 14:44
  powerful for testing. You basically tell the
• 14:47
  agent to seamlessly create five different storyboard
• 14:50
  variants. You carefully pick your absolute favorite
• 14:53
  layout. Right? Then you ask for three distinct
• 14:57
  animation variants based on that single image.
• 14:59
  Because AI video is inherently unpredictable,
• 15:02
  the third generated version might be way better
• 15:05
  than the first attempt. The agent happily handles
• 15:07
  all that tedious repeated work. Yeah. Testing
• 15:09
  endless visual variants is suddenly very easy
• 15:12
  and frictionless. You essentially run a completely
• 15:15
  repeatable content system. You just sit back
• 15:17
  and say, give me 10 brand new ideas. You critically
• 15:20
  pick the strongest ones. The agent generates
• 15:22
  the storyboards. It expertly animates them. You
• 15:24
  just repeat the cycle. This workflow genuinely
• 15:27
  works for product ads, quirky cartoon shorts,
• 15:30
  dynamic anime clips, engaging educational videos,
• 15:34
  high volume social content for TikTok or Instagram.
• 15:37
  The agentic method absolutely shines when you
• 15:39
  need serious volume. It is a fully realized batch
• 15:42
  production studio system. What is the absolute
• 15:45
  biggest bottleneck when producing at this massive
• 15:48
  scale? You still have to review the outputs carefully.
• 15:51
  You must ensure the storyboard panels are perfectly
• 15:53
  clean before animating. If the input is flawed,
• 15:56
  the video is flawed. Volume is infinite, but
• 15:59
  human taste and curation are still the ultimate
• 16:02
  filters. Exactly. The tedious heavy lifting is
• 16:05
  fully automated. But you still fundamentally
• 16:07
  decide what is actually good art. Let's bring
• 16:10
  this all together. The big underlying idea here
• 16:12
  is incredibly clear. A storyboard -first workflow
• 16:16
  unequivocally equals tight visual consistency.
• 16:19
  It mathematically stops the AI from wildly guessing.
• 16:22
  It firmly keeps your characters and camera flow
• 16:25
  completely locked in. The winning software combo
• 16:27
  is really straightforward to remember. Use GPT
• 16:30
  specifically for generating the initial storyboard
• 16:32
  images. And then crucially use C -dance and Higgs
• 16:35
  field for the final AI animation. That specific
• 16:37
  combination gave the absolute cleanest motion
• 16:40
  in testing. You can remarkably run this entire
• 16:42
  system for free. Tools like DeepSeq and AntiGravity
• 16:45
  make it totally possible for anyone. It completely
• 16:48
  changes how you approach the entire video creation
• 16:50
  process. If tools like AntiGravity and BeepSeq
• 16:53
  allow anyone to orchestrate a massive, free,
• 16:56
  automated video studio from their laptop, the
• 16:59
  barrier to entry for storytelling is now zero.
• 17:02
  What does a world look like when ideas can instantly
• 17:04
  become cinematic reality? to sex silence. It's
• 17:08
  a pretty wild thought. Try the manual method
• 17:10
  just once this week. Start with one single focused
• 17:12
  storyboard. Animate it carefully. See exactly
• 17:15
  how it feels to bring your idea to life. Thank
• 17:18
  you for diving deep with us today. Keep learning.
• 17:20
  At TO Music.