Latest / AI Fire Daily / #684 Neil: AI Video Editing With Claude Code From Raw Clips to MP4

Transcript

#684 Neil: AI Video Editing With Claude Code From Raw Clips to MP4 · AI Fire Daily

• 0:00
  Imagine editing a 10 -minute YouTube video without
• 0:03
  ever opening a timeline, just by texting your
• 0:06
  computer. Yeah, and what if that text conversation
• 0:09
  actually produced a fully -polished, color -graded,
• 0:13
  subtitled cut? It completely shatters the traditional
• 0:16
  editing workflow. Welcome to this deep dive.
• 0:20
  Hey, everyone. Today we are unpacking a really
• 0:22
  fascinating guide. It explores using Claude Code
• 0:25
  for automated AI video editing. Our mission here
• 0:29
  is to understand how this fundamental shift in
• 0:32
  human computer interaction actually works in
• 0:34
  practice. It is a massive shift. Yeah. We will
• 0:37
  trace the entire journey, starting with the core
• 0:39
  concept and the local setup environment. Right.
• 0:43
  From there, we will look at how the AI analyzes
• 0:45
  and trims raw footage to create a rough cut.
• 0:48
  And finally, we'll explore how it polishes the
• 0:50
  video with visual effects before exporting the
• 0:53
  final file. You're completely bypassing traditional
• 0:56
  nonlinear editors, like no Premiere Pro, no CapCut.
• 1:00
  You don't even open a graphical interface. Right.
• 1:03
  But we can't really edit a video with AI until
• 1:05
  we understand what tools are actually talking
• 1:07
  to each other. Exactly. So let's establish the
• 1:10
  foundation. How is this even happening without
• 1:13
  an editing program? Well, that is the core premise
• 1:16
  of the source material. The user basically integrates
• 1:19
  an open source AI skill called video use, which
• 1:22
  is derived from the browser use project. And
• 1:25
  they connect it directly into Cloud Code. So
• 1:28
  Cloud Code just acts as your terminal -based
• 1:30
  assistant. It's like stacking Lego blocks of
• 1:33
  data but using your voice instead of your hands.
• 1:35
  That is a great way to look at it. You are just
• 1:37
  telling the machine how to build the final product,
• 1:40
  and it figures out how to snap the pieces together.
• 1:42
  Yeah, and the setup process they describe is
• 1:45
  surprisingly elegant. The user basically just
• 1:47
  points Claude code to a specific GitHub repository.
• 1:50
  Just a link? Yeah, literally just github .com
• 1:53
  slash browser dash use slash video use. So they
• 1:56
  just handed the link and say figure it out. Almost
• 1:59
  exactly that. They prompt Claude code to read
• 2:02
  two specific files from that repo, the install
• 2:05
  .md and the skill .md. So Claude is essentially
• 2:09
  reading its own instruction manual, figures out
• 2:12
  what dependencies it needs, installs the required
• 2:15
  Python tools, and registers video use as a brand
• 2:17
  new capability. It's literally learning a new
• 2:20
  skill in real time. Yeah, it is wild. But the
• 2:23
  source mentions a very specific choice here.
• 2:25
  The user actively chooses to configure a local
• 2:28
  transcription model instead of relying on default
• 2:31
  cloud options like 11Labs Scribe. Right. They
• 2:34
  instruct Claude to switch over to Faster Whisper
• 2:36
  instead. OK. Let me quickly clarify that for
• 2:39
  everyone listening. Faster Whisper is a fast,
• 2:44
  locally run tool that turns speech into text.
• 2:46
  Exactly. And Claude code handles that transition
• 2:49
  seamlessly. It verifies the installation. Then
• 2:52
  it tests Faster Whisper with a tiny audio clip
• 2:55
  just to make sure it works. Makes sense. And
• 2:57
  it successfully generates a transcript with exact
• 3:00
  timestamps. At that point, the basic editing
• 3:03
  engine is ready. I have to ask though, why would
• 3:06
  someone specifically request local transcription
• 3:09
  instead of the default? Mostly for privacy. Avoiding
• 3:12
  API costs and keeping all the processing right
• 3:14
  on your machine. Because of the file sizes? Yeah.
• 3:17
  When you're dealing with gigabytes of raw video,
• 3:19
  cloud servers create a huge bottleneck. plus
• 3:22
  the data security risks if you are editing sensitive
• 3:25
  material. So it saves money and keeps your data
• 3:28
  completely private. That makes a lot of sense.
• 3:31
  Exactly. It gives you total control. OK. So now
• 3:34
  the AI can essentially hear the audio locally.
• 3:38
  How does it actually decide what to cut? This
• 3:41
  is where I get a little skeptical. Yeah. Editing
• 3:43
  is so subjective. Right. Well, this is where
• 3:46
  the real experiment begins. The user creates
• 3:49
  a folder called YouTube -tutorial and drops in
• 3:52
  a 48 -second raw video file. This is the raw,
• 3:55
  messy, unedited footage. Exactly. RawAITutorial
• 3:58
  .mp4. They open Claude code in that folder and
• 4:02
  prompt it to analyze the video. Using that new
• 4:05
  video use skill. Right. And they remind Claude
• 4:08
  to strictly use Faster Whisper. They give it
• 4:11
  a very crucial boundary, though. They say, don't
• 4:13
  edit anything until I approve your plan. Yes.
• 4:16
  They ask for a comprehensive editing strategy,
• 4:19
  explicitly telling it to identify awkward pauses,
• 4:22
  speaking mistakes, and unnecessary sections.
• 4:25
  I have to admit, I still wrestle with prompt
• 4:27
  drift myself, so letting it dictate the cuts
• 4:30
  is daunting. Handing over creative pacing feels
• 4:33
  risky. It totally feels risky. But Claude analyzes
• 4:36
  that 48 -second clip, and it comes back with
• 4:38
  a surprisingly accurate hit list. Wow. It finds
• 4:42
  a false start right at the beginning. It identifies
• 4:45
  awkward pauses where the speaker was clearly
• 4:47
  thinking. It even catches a specific speaking
• 4:51
  mistake. And it suggests cutting the 48 seconds
• 4:54
  down to about 30. Yeah. And the user approves
• 4:57
  it. But they establish very specific constraints
• 5:00
  for the rough cut. Right. Technical and creative
• 5:02
  guardrails. They tell it to keep the 16 by 9
• 5:04
  ratio at 720p. But the really interesting part
• 5:07
  is the pacing. They tell Claude to trim the silent
• 5:10
  pauses down to exactly 0 .3 seconds. That is
• 5:14
  incredibly specific. It is, and it's basically
• 5:17
  the secret sauce of modern video editing. A 0
• 5:20
  .3 second pause feels perfectly human. Yeah,
• 5:23
  it gives you just enough time to process the
• 5:24
  sentence. Exactly, without losing momentum. Because
• 5:27
  if the pause is longer, the video drags. But
• 5:29
  if you strip silence entirely, the speaker sounds
• 5:32
  like a breathless robot. Right. Editors usually
• 5:34
  spend hours manually slicing dead air down to
• 5:37
  specific milliseconds. This user just programmed
• 5:40
  that workflow into one sentence. trim pauses
• 5:42
  to 0 .3 seconds. That is profound. They are mathematically
• 5:47
  programming the cadence of human speech. And
• 5:49
  he goes further. They tell it to remove a specific
• 5:52
  filler word. They explicitly want the word, uh,
• 5:55
  removed from the final cut. But the guide notes
• 5:57
  an interesting glitch here. The local transcriber
• 5:59
  actually missed that specific, uh, in the text
• 6:02
  output. Completely missed it. The text looked
• 6:05
  clean, but the user knew they stumbled. So they
• 6:08
  told Claude to remove it anyway, overwriting
• 6:10
  the transcript. And Cloud executes all this using
• 6:13
  FFmpeg, right? FFmpeg is basically the Swiss
• 6:16
  army knife of media processing, a command line
• 6:18
  tool that can splice almost any video format.
• 6:22
  Exactly. Cloud calculates the exact cut points
• 6:25
  and writes an FFmpeg script to handle the brute
• 6:27
  force splicing. Creating a preview .mp4 file.
• 6:30
  Yes. And the numbers on this are striking. It
• 6:33
  drops from 48 .2 seconds down to a tight 29 .9
• 6:37
  seconds. Almost... 20 seconds of dead air just
• 6:40
  vaporized. And the user checks it. The cuts sound
• 6:44
  incredibly natural. No important words get clipped.
• 6:47
  Let's unpack that transcription error for a second.
• 6:50
  If the transcriber missed the uh, how did the
• 6:53
  AI know to cut it? Claude cross -references the
• 6:55
  text with the audio pacing, allowing the user's
• 6:58
  prompt to override minor transcription errors.
• 7:00
  It looks at the temporal gaps. Great. If there
• 7:03
  is an audio gap where the user says there's an
• 7:05
  uh, Claude infers the location and splices it
• 7:08
  out. The prompt acts as a safety net for the
• 7:11
  AI's mistakes. You are actively managing its
• 7:14
  blind spots. Exactly. The human director is still
• 7:16
  entirely essential. OK, so a tightly cut 30 -second
• 7:21
  video is a great start. But it is really just
• 7:24
  the skeleton. Next, we need to add the skin and
• 7:27
  polish. Right. And without opening a traditional
• 7:30
  editing interface, no timelines. This brings
• 7:33
  us to polishing with prompts. The user submits
• 7:36
  a massive prompt to finalize the video. But their
• 7:39
  first instruction is a strict defensive maneuver.
• 7:42
  They demand that the original cuts remain completely
• 7:45
  untouched. Keep all approved cuts and the original
• 7:48
  narration. Priority one. Language models can
• 7:51
  get overzealous trying to re -optimize things
• 7:53
  you already approved. So from there, it's three
• 7:56
  distinct tasks. First being subtitles. Right.
• 7:59
  They want Accurate subtitles matching the new
• 8:02
  29 .9 second duration. And they give very strict,
• 8:05
  granular style guidelines. Very strict. Exactly
• 8:08
  three to five words per segment. A clean, bold
• 8:11
  sans serif font. White text with a subtle black
• 8:14
  outline. Placed near the bottom center. Yeah,
• 8:16
  and they explicitly command Claude to keep them
• 8:18
  within safe margins, like make sure subtitles
• 8:21
  don't cover important slide content. Task two
• 8:24
  focuses on video colors. And I find it hard to
• 8:27
  believe an AI got a color grade right on the
• 8:30
  first try from text. I mean, how do you describe
• 8:32
  a color grade to code? It is incredibly difficult,
• 8:36
  which is why the prompt is brilliant in its restraint.
• 8:39
  They ask for a subtle, natural color grade. Improve
• 8:42
  contrast only where needed. They tell it to avoid
• 8:44
  oversaturation and cinematic filters. Exactly.
• 8:47
  Keep backgrounds readable and maintain consistent
• 8:49
  colors. It is a very restrained, professional
• 8:53
  approach. Task 3 gets genuinely ambitious. Visual
• 8:57
  effects. The most dynamic part. They ask Claude
• 8:59
  to find two or three moments where simple animations
• 9:02
  would clarify the tutorial. Look, notice they
• 9:04
  don't dictate the exact time stamps. No, they
• 9:07
  give the AI creative freedom within established
• 9:09
  constraints, highlighting keywords, subtle motion,
• 9:11
  or a simple call out. But they require the animations
• 9:14
  to match the narration perfectly. Keep them short,
• 9:18
  easy to read, and do not cover the subtitles.
• 9:21
  So Claude processes this massive prompt and renders
• 9:25
  an updated preview .mp4. And it succeeds on all
• 9:28
  fronts. It places 18 subtitle cues perfectly,
• 9:31
  adjusts the colors subtly, and successfully adds
• 9:34
  three animated highlights. It actually found
• 9:36
  the right moments itself based purely on context.
• 9:39
  Yes. Highlighting phrases like three simple ways
• 9:42
  and explain the goal exactly when spoken. It
• 9:45
  understood the semantic importance. Whoa. Imagine
• 9:49
  scaling this to a billion queries or automating
• 9:51
  100 tutorials overnight. Oh yeah, the efficiency
• 9:54
  gains are staggering. If you don't have to manually
• 9:57
  hunt for key phrases, the bottleneck just disappears.
• 10:00
  Exactly. The user reviews it, subtitles are clear,
• 10:04
  animations hit perfectly, audio is natural. And
• 10:07
  the video structure was completely maintained.
• 10:10
  Yes. So the user gives final approval to export
• 10:14
  final .mp4. But I have a question about those
• 10:16
  visual effects. Doesn't adding visual effects
• 10:19
  purely via text prompt risk things overlapping
• 10:22
  or looking messy? Yes, which is why the prompt
• 10:25
  explicitly commanded Claude to keep subtitles
• 10:28
  within safe margins and not cover important slide
• 10:31
  content. You have to code common sense into the
• 10:35
  prompt itself. Precisely. Because the machine
• 10:37
  doesn't just know what visually looks good. Precisely.
• 10:42
  You define the boundaries of the canvas before
• 10:43
  you tell it to paint. We will explore the final
• 10:46
  export details right after this. Insert sponsor
• 10:49
  read here. We are looking at the final stage
• 10:52
  of this AI video editing workflow. The user is
• 10:54
  ready to export. They give Claude the final export
• 10:57
  settings. 1280 by 720 resolution, 16 by 9 ratio.
• 11:02
  The frame rate matches the source video. And
• 11:04
  they specify the exact codex. H .264 for video,
• 11:08
  AAC for audio. The universal industry standards
• 11:10
  for web video. Upload that to YouTube or Twitter
• 11:13
  and it plays flawlessly. Yes, Claude exports
• 11:16
  final .mp4 and actually checks its own work afterward,
• 11:19
  verifying there are no playback errors. And through
• 11:22
  all this, the original raw footage remains completely
• 11:25
  unchanged. Always preserved. The final video
• 11:28
  is exactly 29 .86 seconds, 18 perfectly timed
• 11:32
  subtitles, three context -aware highlights, a
• 11:36
  natural color grade. All done without a timeline
• 11:38
  or a razor tool. Let's pull back. What is the
• 11:42
  major takeaway we should be pulling from this
• 11:44
  source? The major takeaway is that we are witnessing
• 11:47
  a fundamental shift in workflow. Going from 48
• 11:50
  seconds of messy raw footage to a highly polished
• 11:53
  deliverable. A color -graded, subtitled, 29 .86
• 11:57
  second MP4. Right. And achieved entirely by typing
• 12:00
  conversational instructions into a terminal.
• 12:02
  Language has officially become the new user interface
• 12:05
  for video editing. Exactly. The author heavily
• 12:08
  recommends this specific workflow for YouTube
• 12:10
  tutorials. It is incredibly efficient for repetitive
• 12:13
  formats. But it's important to remember that
• 12:15
  you still need to review the output. You are
• 12:18
  the director. You have to catch things like that
• 12:20
  missed uh. Absolutely. But the heavy, tedious
• 12:23
  lifting is automated. The author suggests starting
• 12:26
  with a short video just to get comfortable. Which
• 12:28
  brings us to our call to action for you, the
• 12:30
  listener. Try this out. Yes. Record a quick 30
• 12:34
  second talking head video on your phone. Just
• 12:37
  talk about your breakfast. Set up a local LLM
• 12:39
  environment and run your messy video through
• 12:41
  this workflow. Watch it analyze your speech and
• 12:44
  slice out awkward pauses based purely on your
• 12:46
  commands. It is a profound experience to see
• 12:49
  a computer edit a video without you touching
• 12:52
  a timeline. It permanently changes how you think
• 12:54
  about media creation. It really does. we are
• 12:57
  rapidly moving toward having nuanced, collaborative
• 13:00
  conversations with our tools. The tools are finally
• 13:02
  learning to speak our language. Which leaves
• 13:04
  us with this final thought. If an AI can accurately
• 13:08
  edit a video based on your text prompt today,
• 13:12
  how long until the AI edits the video dynamically,
• 13:15
  in real time, matching the specific attention
• 13:18
  span and learning style of whoever's watching
• 13:20
  it? Think about that.