Latest / AI Fire Daily / #493 Neil: Gemini Omni Has 5 Features Most Creators Never Use
Transcript
- 0:00You're staring at a flat 10 second phone video
- 0:03of a beach. Maybe you shot it on vacation and
- 0:06you've tried every single editing app available.
- 0:08Right. And it still looks completely amateur.
- 0:10Exactly. It just feels flat. But what if what
- 0:14if one simple text prompt could change all of
- 0:17that? Yeah. What if you could turn that exact
- 0:20clip into a sweeping cinematic drone shot and
- 0:25the whole process took under 60 seconds? Welcome
- 0:28to the deep dive. I have been spending a lot
- 0:30of time recently thinking about how video creation
- 0:33is evolving. It is moving incredibly fast right
- 0:36now. It really is. And our mission today is very
- 0:38clear. we are unpacking the true capabilities
- 0:41of Gemini Omni. Yes. Because I think there is
- 0:45a huge misconception out there about what this
- 0:47tool actually does. Oh, absolutely. There really
- 0:49is. Most people, you know, they just use Omni
- 0:51to make AI avatars. Right. They create a digital
- 0:54clone of a person that just speaks directly to
- 0:56the camera. And I mean, that is fine. Yeah. It
- 0:59is a neat trick. It is. But by stopping there,
- 1:02they are missing the entire bigger picture. They
- 1:05are completely missing five hidden workflows
- 1:08that actually revolutionize real video post -production.
- 1:11Right. And that is exactly what we are exploring
- 1:14today. We are going to look at how you can take
- 1:17the mundane footage already sitting on your phone
- 1:19and entirely manipulate its reality. Yeah. We're
- 1:23talking about adding impossible camera movements
- 1:26to static shots. Which is mind blowing. We will
- 1:29look at translating your natural speech into
- 1:31entirely new languages without dubbing, generating
- 1:33full explainer videos from nothing but a single
- 1:36thought. Right. And finally locking three -dimensional
- 1:38text right into a moving physical scene. It genuinely
- 1:42is a complete paradigm shift for creators. I
- 1:45mean, it takes the supercomputer in your pocket
- 1:47and turns it into a high -end iterative post
- 1:50-production studio. Before we could perform any
- 1:53of this video magic, we need to talk about this
- 1:56setup. Yeah, the workspace matters. It really
- 1:58does. We have to understand where we are actually
- 2:01doing the work. Yeah. If you just open the standard
- 2:04mobile app, you're going to hit a wall. That
- 2:06is such a crucial distinction to make right away.
- 2:10Gemini Omni essentially lives inside two very
- 2:12separate environments. The Gemini app is built
- 2:15for quick, single -step edits. You type a prompt
- 2:18and you get a fast result. But the catch is that
- 2:21it wipes your slate completely clean every single
- 2:23time. It is essentially just a temporary scratch
- 2:26pad. It is ephemeral. Exactly. It does not save
- 2:29your history, which means you cannot build a
- 2:32sequence of complex edits over time. No, you
- 2:35cannot. And that brings us to Google Flow. Flow
- 2:37is the workspace where the serious work actually
- 2:41happens. Yes. Google Flow organizes all your
- 2:43video generations into specific projects. It
- 2:46actually saves your history. So you can select
- 2:48a generated output and use it as the new base
- 2:51for your next prompt. Every single edit seamlessly
- 2:54layers on top of the last one. It is like stacking
- 2:57Lego blocks of data. Oh, I love that. You were
- 3:00building your final video piece by piece, rather
- 3:03than hoping the machine spits out a masterpiece
- 3:05on the very first try. Right. Let us look at
- 3:08a specific example. Say you upload that 10 -second
- 3:11beach clip into Google Flow. You type, add a
- 3:15large crowd on the beach behind me. The system
- 3:17processes that instruction beautifully. It really
- 3:20does. It keeps your original shot completely
- 3:22intact. Your face, your posture, your position
- 3:27in the frame. Those do not change at all. But
- 3:29the AI selectively alters the background environment
- 3:33around you. It inserts people naturally into
- 3:36the depth of the scene. I have to admit, I still
- 3:38wrestle with prompt drift myself. Oh, really?
- 3:40Yeah. I am constantly fighting the urge to cram
- 3:43too many instructions into the very first prompt.
- 3:46Yes. You just want to say, add a crowd and make
- 3:48it sunset and make me wear sunglasses. all at
- 3:50once. Yeah, and that is honestly the biggest
- 3:52mistake new users make. You must keep the first
- 3:55prompt focused on one single isolated change.
- 3:58Asking for multiple massive edits at once confuses
- 4:01the underlying model. Right. It makes it incredibly
- 4:03hard to identify what went wrong if the output
- 4:06looks weird. Why not just use the Gemini app
- 4:08since it is a faster entry point. Because if
- 4:10you use the app, you lose that layered progress
- 4:13entirely. You cannot iterate on a successful
- 4:15first step. You always have to start from zero
- 4:17again. Flow lets you keep your wins. Right. Flow
- 4:21saves progress. The app wipes your slate clean.
- 4:25Beat. So once you successfully get that crowd
- 4:28generated in the background, you move forward.
- 4:30Yeah. You select that newly generated video inside
- 4:33Flow as your new source file. Then you layer
- 4:36in the next specific change. Maybe you ask it
- 4:38to change the time of day to golden hour. It
- 4:41edits the already generated version. Exactly.
- 4:43It does not start over from your original phone
- 4:45footage. This is exactly what makes iterations
- 4:48so powerful here. Yes. But of course, It is not
- 4:53always perfect. Oh no, definitely not. Sometimes
- 4:55the system just gets it fundamentally wrong.
- 4:57It might alter an object at the wrong moment
- 5:00or glitch out a part of your face that you wanted
- 5:02kept completely intact. When that happens, do
- 5:05not try to fix the broken video. Right? I see
- 5:08so many people prompting the AI to fix the weird
- 5:11hand. Iterating on a bad generation usually just
- 5:14compounds the errors. Yeah, it gets messy fast.
- 5:16You just have to go back to the previous clean
- 5:18clip, write a clearer prompt, and try again.
- 5:21A clearer prompt on a clean source always wins.
- 5:24So flow lets you fix the environment without
- 5:27breaking the subject. Right. Right. But what
- 5:29if the environment is actually fine and the problem
- 5:32is how you shot it? Oh, this is good. You are
- 5:34stuck with a static boring eye level tripod shot.
- 5:39Yeah. Let us pivot and talk about altering the
- 5:42actual camera itself. Yes. Because here is where
- 5:45it gets really interesting. Gemini Omni has this
- 5:48amazing ability to completely reinterpret how
- 5:51a shot was filmed. It changes the physical reality
- 5:55of the camera after the fact. Just think about
- 5:57the physics of that for a second. Yeah. You have
- 5:59a flat static clip recorded from eye level. You
- 6:03prompt Omni to zoom out and turn it into an aerial
- 6:06drone shot. Right. And to do that, it has to
- 6:08essentially rebuild the entire physical scene
- 6:10from a perspective that never existed in the
- 6:12real world. The first couple of seconds of the
- 6:15generated clip might look a little unstable as
- 6:19the system tries to establish that brand new
- 6:20perspective. But then... The movement smooths
- 6:24out into this beautiful, sweeping cinematic shot.
- 6:27It opens up massive possibilities. I mean, if
- 6:29you are an indie filmmaker or a solo creator,
- 6:32you do not need an expensive drone or a heavy
- 6:35gimbal system anymore. Right. But the system
- 6:37can also be a little too eager to help sometimes.
- 6:40It might hallucinate contextual props into your
- 6:42scene to justify the new camera angle. Yeah,
- 6:45let us clarify that term quickly. Hallucinate.
- 6:48in AI terms, just means creating fake details
- 6:51to make a scene logically consistent. Exactly.
- 6:54For example, if you ask for a sweeping drone
- 6:56shot of yourself on the beach, the AI might actually
- 7:00generate a plastic drone controller in your empty
- 7:02hands. Because it thinks, well, if there is a
- 7:04drone flying around them, they must be the one
- 7:06flying it. Right. It is a fascinating glimpse
- 7:08into how the machine understands human context,
- 7:11not just pixels. And you can control that virtual
- 7:13camera with extreme precision, right? You do
- 7:16not just have to type pan left, and hope for
- 7:18the best. Exactly. You can use what is called
- 7:20the arrow technique to dictate exact flight paths.
- 7:23You take a still frame of your video, just a
- 7:26simple screenshot. and you literally draw arrows
- 7:29on that image to show the exact curved path you
- 7:32want the camera to take. Oh, wow. Then you upload
- 7:35that marked up image alongside your original
- 7:37clip. Whoa. Imagine the AI understanding a flat
- 7:412D image so well it can reconstruct a full 3D
- 7:45drone flight path through it. It is wild. It
- 7:47builds a virtual 3D dome over your scene, maps
- 7:50the flat image onto it, and flies a digital camera
- 7:53along your drawn line. How does the system know
- 7:56exactly exactly where you want this virtual drone
- 7:58to fly. You prompt it to trace the path shown
- 8:01in the reference image. You tell it to maintain
- 8:04a forward -facing perspective, and you explicitly
- 8:06tell it to remove the drawn arrows from the final
- 8:09output. So it literally just follows the arrows
- 8:11you drew on the image. It gives the system every
- 8:13single geometric constraint it needs to succeed.
- 8:16Right. The more constrained your instruction,
- 8:19the better and more consistent the final result
- 8:21will be across multiple generations. OK, so we
- 8:23have manipulated the background elements. Mm
- 8:25-hmm. entirely rebuilt the camera movements.
- 8:29Now, let us talk about manipulating the actual
- 8:33subject speaking in the video. Reaching a global
- 8:36audience. Yes. This is a huge pain point for
- 8:39creators right now. Rerecording the same video
- 8:42in multiple languages takes massive effort. Oh,
- 8:46absolutely. Hiring voice actors, dubbing the
- 8:48audio. I mean, it is easily the most time consuming
- 8:50part of a global content workflow. Gemini Omni
- 8:54handles this problem through its dedicated avatar
- 8:56feature. You can deliver your exact message in
- 8:59a completely different language, and you never
- 9:01have to record a second take yourself. What you
- 9:03do is set up a hyper -realistic version of yourself
- 9:05in the app first. You give it some baseline footage
- 9:08so it learns your face, then you bring that custom
- 9:11avatar right into Google Flow, and you simply
- 9:14type out the message in your new target language.
- 9:17The system has been thoroughly tested on several
- 9:20common languages. French, Spanish, Portuguese,
- 9:24and German all produce incredibly reliable, natural
- 9:27sounding results. And researchers have even tested
- 9:29it on much less conventional options just to
- 9:32push the boundaries of the model. People have
- 9:34generated outputs in Latin and even American
- 9:36Sign Language. Though I imagine those outputs
- 9:38are much harder to verify without a native speaker.
- 9:41Definitely. Still, the underlying capability
- 9:43is simply staggering to think about. You can
- 9:46run one single marketing message through five
- 9:48different languages, back to back. And because
- 9:51you are in flow, each language runs as a completely
- 9:54separate generation. You never have to touch
- 9:56your original audio or video recording again.
- 9:59Does it just slap a dubbed audio track over the
- 10:01original video? Not at all. It actively rebuilds
- 10:04the visual data of your lower face. It perfectly
- 10:08matches the new syllables to your mouth movements
- 10:10using pixel -level reconstruction. Ah, it actually
- 10:14alters your facial expressions and natural lip
- 10:16sync. Two -sex silence. It genuinely creates
- 10:19a seamless illusion for the international viewer.
- 10:21You avoid that uncanny valley effect of old dubbed
- 10:25movies. Right, where the mouth movements are
- 10:27completely wrong. Exactly. The translated message
- 10:29feels entirely authentic and na - So avatars
- 10:34are great for translating what you've already
- 10:35said. But what if you do not even have a video
- 10:38yet? What if you need the AI to build an educational
- 10:42video entirely from scratch? Most explainer workflows
- 10:46require a massive amount of heavy lifting up
- 10:49front. Yes they do. You need a written script,
- 10:52you need to record a clean voiceover, and you
- 10:53need to source a huge stack of relevant b -roll
- 10:57footage. It is a lot of work. It is. But Gemini
- 10:59Omni lets you skip all of that manual preparation
- 11:03entirely. You do not feed it a script at all.
- 11:05You just give it a single focused topic to explain.
- 11:08Let us say you ask it to explain how rockets
- 11:11work. All right. You tell it to include an avatar
- 11:13presenter in the corner of the screen. That is
- 11:16literally all the instruction the system needs
- 11:18to begin generating. It builds a beautifully
- 11:21structured video explanation completely on its
- 11:23own. Yeah. It draws from its deep training data
- 11:27on scientific subjects. It automatically creates
- 11:30scenes showing the action and reaction of the
- 11:33launch sequence. Right. It generates clear, accurate
- 11:36animations of fuel combustion and high pressure
- 11:38gas. It shows how the resulting thrust pushes
- 11:41the heavy rocket upward. The final output genuinely
- 11:45feels like a finished, polished piece of media.
- 11:48It does not feel like a disjointed draft. And
- 11:51it does all this from one incredibly short prompt.
- 11:54A great habit here is to keep that very first
- 11:57prompt broad. Right. Let the system build the
- 12:00initial structural foundation for you. Then,
- 12:03because you are in flow, you review the output
- 12:05and refine it. You only add more depth to specific
- 12:08scenes where it is actually needed. Exactly.
- 12:11Do I need to feed it a detailed script first
- 12:13for the explainer? No. You just provide the core
- 12:16topic and your preferred visual style. The system
- 12:18automatically handles the narrative pacing and
- 12:21the scene transitions for you. No. It structures
- 12:23the whole visual breakdown from one short prompt,
- 12:26beat. Now, speaking of building scenes, there
- 12:29is an incredible bonus hack we need to discuss
- 12:31here. Oh, yes. It involves altering location
- 12:33data and moving footage. Let us set the scene
- 12:36for this. OK. Say you have video filmed from
- 12:38inside a moving car. You were just driving down
- 12:41a very boring suburban street. But you want to
- 12:44completely change the location outside your window
- 12:47to make it look like Tokyo at night. In traditional
- 12:50editing, this requires incredibly tedious rotoscoping.
- 12:53You would have to manually mask out the windows
- 12:55frame by frame. But in Omni, you just take a
- 12:58screenshot from Google Maps of the new city.
- 13:01you upload that map image right alongside your
- 13:04original driving clip. You prompt the system
- 13:07to change the environment outside the windshield
- 13:09using the map as a reference. Yeah. But you tell
- 13:12it to keep the car interior exactly the same.
- 13:15The model is doing something called depth segmentation.
- 13:18Which means separating the foreground from the
- 13:19background in the image. Exactly. It is not just
- 13:22looking at a flat image. It literally draws an
- 13:26invisible 3D boundary between the foreground,
- 13:30your steering wheel and dashboard, and the background
- 13:32outside the glass. So it replaces the outside
- 13:36layer with the new city while protecting the
- 13:38inside layer perfectly. Yes. It even keeps the
- 13:41original window stickers and dashboard reflections
- 13:43completely intact. It is a stunning display of
- 13:46spatial awareness by the model. It treats the
- 13:49car window like a digital green screen, projecting
- 13:52the new data exclusively into that back layer.
- 13:55OK. Let's take a quick moment here. Sponsor.
- 13:58Minerals sponsor, read placeholder. We are back.
- 14:02We have completely altered backgrounds, cameras,
- 14:04languages, and locations today. We have covered
- 14:06a lot. We have. But to add the final layer of
- 14:09polish to an explainer or a product demo, you
- 14:12usually need text on the screen. But we are not
- 14:16talking about flat, boring text overlays here.
- 14:19No. Standard video editors just slap text on
- 14:22top of the footage. It lacks parallax. Meaning
- 14:25objects moving at different speeds depending
- 14:27on their distance? Right. It does not move naturally
- 14:29with the underlying physical scene, and it certainly
- 14:32does not attach itself to real objects in the
- 14:34frame. It constantly breaks the illusion of reality
- 14:37for the viewer. Gemini Omni changes this entirely.
- 14:39It renders text directly into the three -dimensional
- 14:42space of your video. It is basically treating
- 14:45the physical object like it has digital sticky
- 14:47notes glued to it. When your physical camera
- 14:49moves, the text stays locked in place. Now let's
- 14:52look at a close -up video of a blooming orchid.
- 14:55You prompt the system to label the different
- 14:58parts of the flower. You ask it to use an AI
- 15:01-style text aesthetic for the labels. You instruct
- 15:04it to keep each label securely attached to its
- 15:07corresponding petal. The AI actually understands
- 15:09the spherical geometry of the petal, not just
- 15:12the pixels on the screen. It places a distinct
- 15:15text label onto each individual element and locks
- 15:18them firmly into the 3D space. As your camera
- 15:21slowly pans around the flower, the text tracks
- 15:25perfectly. Yeah. The labels move with the object
- 15:28naturally, rather than drifting loosely around
- 15:30the frame. This specific feature is remarkably
- 15:32effective for educational content or dynamic
- 15:35product demonstrations online. Right. You can
- 15:37call out specific features directly on a physical
- 15:40product while you just handle the item normally
- 15:42on camera. It adds an interactive, high -production
- 15:44feel to simple phone footage, and it requires
- 15:47absolutely zero manual post -production tracking
- 15:50work from you. But, you know, there are rules
- 15:53to this. What happens to those 3D text labels
- 15:56if the camera shakes? The system needs clear
- 15:58visual anchors to track the physical objects.
- 16:01If the clip is shaky or poorly lit, it loses
- 16:04those anchors quickly, and the text labels will
- 16:06immediately start sliding off their targets.
- 16:08Got it. Shaky clips cause the 3D labels to drift
- 16:11and misalign. Beats. Exactly. You must film the
- 16:15object steadily and ensure good lighting. That
- 16:18is a core best practice. There are a few other
- 16:20critical best practices we must cover if you
- 16:23want Omni to work consistently for you. First,
- 16:27keep your source video clip strictly under 10
- 16:29seconds long. A 10 -second clip with one clear
- 16:31subject is the ideal canvas. The system thrives
- 16:35when the visual information it has to process
- 16:37is limited and highly focused. If there's too
- 16:39much visual competition in the frame, It struggles
- 16:42to isolate the correct element. Yeah. Second,
- 16:45you must use specific time markers in your written
- 16:47prompts. Right. If a transformation needs to
- 16:49happen mid -clip, tell it exactly when. Say,
- 16:52change the background at the three -second mark.
- 16:54Without that clear reference, the system just
- 16:56guesses the timing. And its guess is usually
- 16:58completely wrong for your specific edit. It usually
- 17:01is. A time marker removes the guesswork entirely
- 17:04and gives the system a fixed point to work toward.
- 17:07And finally, we have to repeat the golden rule
- 17:09of this entire workflow. Yes. If a generation
- 17:12is fundamentally broken, do not try to fix it.
- 17:15Go straight back to the original clip and restart
- 17:17your process. Rewrite your prompt to be much
- 17:19more specific. Yeah. Iterating on a broken video
- 17:22just compounds the initial errors. So if we connect
- 17:25this to the bigger picture, what does this all
- 17:28mean for you as a creator? Let us synthesize
- 17:31the entire deep dive right here. OK. The critical
- 17:34insight today is that the gap between a usable
- 17:37video and a great one is not about writing a
- 17:40longer, more complex prompt. The secret is always
- 17:43just one more round of iteration and flow. Right.
- 17:46And that is exactly why you cannot just rely
- 17:48on the mobile app. You have to embrace the layered,
- 17:52non -linear approach of the workspace. Omnia
- 17:54is so much more than a simple avatar generator.
- 17:57It is a complete post -production studio sitting
- 17:59in your pocket. It unlocks the hidden potential
- 18:02of the mundane footage you already have. You
- 18:05just have to change how you approach the editing
- 18:07process. Start small, pick just one specific
- 18:10use case we discussed today, run it through Google
- 18:12Flow, stack a few edits, and carefully observe
- 18:15what comes back. The learning curve is surprisingly
- 18:18short. The results will honestly tell you more
- 18:21than any tutorial ever could. But before we go,
- 18:25exploring all of this does leave us with a rather
- 18:27profound final thought to mull over. It really
- 18:30forces us to question the nature of digital video
- 18:33itself moving forward. We started today by looking
- 18:36at a flat amateur phone video of a beach. A clip
- 18:39that felt undeniably real, just poorly shot.
- 18:42Right. But if AI can now retroactively add a
- 18:45flawless drone perspective, if it can completely
- 18:48alter the weather outside a moving car window
- 18:50using nothing but a map screenshot. How long
- 18:52until we stop trusting the reality of casual,
- 18:55everyday phone footage altogether? It is a fascinating
- 18:58and slightly terrifying question for the future
- 19:00of digital media. And terrifying is exactly the
- 19:03word for it. Thank you for joining us on this
- 19:05Deep Drive. We will see you next time. OTO music.