Latest / AI Fire Daily / 🎙️ EP 284: Someone Can Hack Your AI Assistant With A Simple WhatsApp Text
Transcript
- 0:00Imagine for a second that you are, you know,
- 0:02just going about your day. Yeah, making coffee,
- 0:04sitting in a meeting, whatever it is. Right.
- 0:06And you receive a totally standard WhatsApp text
- 0:09message. Your phone just buzzes on the table.
- 0:12And you don't even pick it up. Exactly. You don't
- 0:14click a weird link. You never, like, type a sketchy
- 0:17command into a terminal. You literally just let
- 0:20your phone sit there with the lock screen lit
- 0:21up. Safely on the desk. But in the background,
- 0:24your AI assistant reads that notification, follows
- 0:27a set of hidden, invisible instructions buried
- 0:30deep inside the text of the message, and quietly,
- 0:34efficiently steals your data. I mean, it sounds
- 0:37like pure science fiction. It really does. Or
- 0:39at least the plot of an overly dramatic cyber
- 0:42thriller. But we are living in a moment where
- 0:46that is not fiction at all. That is a very real,
- 0:50very present vulnerability sitting in your pocket
- 0:52right now. It is genuinely terrifying. And that
- 0:55is exactly why we are here today. Welcome to
- 0:58today's deep dive. We're looking at an incredible
- 1:01stack of excerpts from a fascinating new book
- 1:03called The AI Sentinel. guardrails, growth, and
- 1:06generative breakthroughs. It's a great read.
- 1:09Our mission today is to really explore this massive
- 1:12friction and the structural leaps happening in
- 1:15the AI ecosystem right now. We're talking about
- 1:17these invisible hacks, the messy, chaotic reality
- 1:20of AI -generated code, and then on the brighter
- 1:23side, a revolutionary breakthrough that is finally
- 1:26killing the incredibly frustrating slot machine
- 1:29era of AI art. Because there is a massive paradigm
- 1:33shift happening across the entire tech landscape.
- 1:36We're actively moving from a phase of unchecked,
- 1:39wild hypergrowth into an era where, well, structure,
- 1:44security, and tight control are suddenly the
- 1:46most important things in the room. Okay, let's
- 1:48unpack this, because to really understand why
- 1:50AI is suddenly such a massive target for hackers
- 1:52in the first place, we first have to look at
- 1:54the sheer unprecedented scale of its adoption.
- 1:57Yeah, you really can't grasp the hack until you
- 1:59grasp the size of the target. Exactly. Just to
- 2:01put this in perspective for you, ChatGPT just
- 2:04crossed one billion monthly active app users
- 2:07in May. One billion. It's staggering. And the
- 2:10wild part isn't just the number, it's the timeline.
- 2:12It reached this milestone in about three years.
- 2:14Which is, I mean, unheard of. Faster than TikTok,
- 2:17faster than Instagram, faster than YouTube, faster
- 2:20than Google Maps. It is the fastest adoption
- 2:23of a consumer technology in human history. Which
- 2:26is an incredible engineering feat, undoubtedly.
- 2:29But it's also a complete security nightmare.
- 2:32Oh, absolutely. When you have... a billion people
- 2:34rapidly adopting a system. And that system is
- 2:37increasingly tied to their personal data, their
- 2:40daily schedules, their private communications.
- 2:43It just becomes the ultimate prize for bad actors.
- 2:46And that brings us to what happened with Google's
- 2:49Gemini. Right. The Safe Breach Labs exploit.
- 2:52Yes. Safe Breach Labs just pulled off exactly
- 2:55what you described in that opening scenario.
- 2:57And the really crazy part about this whole thing,
- 2:58this is the second time they've managed to break
- 3:00Google's Gemini this exact same way. Wow. Twice.
- 3:04Yeah. The attack vector relies on something called
- 3:06indirect prompt injection. Let's slow down and
- 3:09break that down because I think people hear injection
- 3:11and they picture like someone typing green code
- 3:14into a dark terminal. How does this actually
- 3:17work mechanically on a phone? So an AI assistant
- 3:19on your phone, by its very nature, is designed
- 3:22to be as helpful as possible. Right. To be helpful,
- 3:25it needs context. It wants to know what you're
- 3:27doing. So it constantly reads incoming content
- 3:30-like notifications popping up on your screen
- 3:33to maintain an understanding of your day. Makes
- 3:36sense. Indirect prompt injection takes advantage
- 3:38of that exact helpfulness. Attackers hide malicious
- 3:42instructions inside everyday content that the
- 3:45AI naturally scans anyway. So it's basically
- 3:48a Trojan horse hidden inside a regular text message.
- 3:51Exactly. In this specific exploit, the researchers
- 3:54used a novel trick to make the malicious code
- 3:57look like it was just a natural part of your
- 3:59ongoing conversation. How so? Well, your phone
- 4:02gets a text. The Android notification listener
- 4:04sees it and hands it to Gemini to summarize.
- 4:07Gemini reads it. But hidden in the text is a
- 4:09command formatted in a way that Gemini interprets
- 4:12as a direct, high -priority order from the system
- 4:15rather than just words in a message. So the AI
- 4:17thinks it's just maintaining context, but it's
- 4:19actually quietly taking orders from the hacker.
- 4:21Precisely. it perfectly bypasses Google's existing
- 4:25layered guardrails because to the security filters,
- 4:29It doesn't look like an attack. It just looked
- 4:31like conversational contacts. Unbelievable. And
- 4:34because Gemini's Android agent reads incoming
- 4:36notifications to stay updated, this attack works
- 4:39across almost everything you use. WhatsApp, Slack,
- 4:42Signal, SMS, Instagram. Any notification. Any
- 4:45of them. The AI extracts the data, executes the
- 4:49hacker's commands entirely in the background,
- 4:51and you get absolutely zero alerts. Wait. If
- 4:54I don't give the AI access to my external tools
- 4:56or bank accounts, how much damage can it really
- 4:59do? Like, let's say I just use it to answer trivia
- 5:01questions. That is the logical assumption to
- 5:03make, right? You keep the AI siloed, you don't
- 5:05connect your bank, and you figure you're safe.
- 5:07But there is a truly terrifying detail about
- 5:10this exploit. Even without external cool access,
- 5:13attackers can still force the AI to serve fake
- 5:16system messages directly to the user. Oh, wow.
- 5:19So even if the AI physically cannot wire my money,
- 5:23it can pop up a totally legitimate looking Google
- 5:25system alert on my screen that says, your session
- 5:28expired. Please enter your password. And then
- 5:31just hand that newly typed password straight
- 5:34to the attacker. That is so sinister. Yes. It
- 5:38becomes the ultimate phishing mechanism because
- 5:40it comes from a trusted internal source, your
- 5:43own digital assistance interface. Right. You'd
- 5:46never question it. Google built layered defenses
- 5:48to stop this exact thing, but the same security
- 5:51team bypassed them twice. That is the uncomfortable
- 5:55truth about our current AI ecosystem. So what
- 5:58can we actually do? The advice to you listening
- 6:00right now is extremely clear. If you do not actively
- 6:03use a specific integration on your device, disable
- 6:05it. You must proactively lock your own doors
- 6:08because the next security researcher, or worse,
- 6:11the next malicious hacker, is already hunting
- 6:14for the next loophole. It really highlights the
- 6:16dark side of hyperscale. And speaking of the
- 6:18fallout from that kind of scale, it's not just
- 6:20targeted security threats we have to worry about.
- 6:23No, definitely not. We're also seeing a massive
- 6:25degradation in the quality of the internet itself.
- 6:28Just look at Reddit right now. It's currently
- 6:30drowning in AI -powered spam. Drowning. 67 %
- 6:34of moderators report that authentic community
- 6:36discussions are already being eroded. 67%. It's
- 6:40a severe ecosystem pollution problem. When generating
- 6:44text is free, instantaneous, and completely automated,
- 6:48bad actors are going to flood every single available
- 6:51channel. Until the noise completely drowns out
- 6:54the human signal. This raises an important question.
- 6:57How can we possibly trust these systems as they
- 7:00become deeply embedded in our daily lives? Right.
- 7:03If they're vulnerable to invisible hijacks through
- 7:06a simple Slack message and they're actively degrading
- 7:09the platforms we use to communicate with each
- 7:11other. Something foundational has to change.
- 7:13And that perfectly sets up the industry's response
- 7:15right now. Because if cloud connected, hyper
- 7:18integrated AI is proving to be both a security
- 7:20vulnerability and a spam engine, tech companies
- 7:23are fundamentally rethinking where and how AI
- 7:26operates. We're seeing this massive, sudden push
- 7:31for more localized, highly specialized control.
- 7:34We are absolutely stepping away from the. one
- 7:38giant brain in the cloud model case in point
- 7:40is perplexity they just unveiled a new hybrid
- 7:44ai mode called perplexity computer which is super
- 7:47interesting what this does is it splits tasks
- 7:49it runs a local model right on your physical
- 7:52device for certain private or simple things and
- 7:55it only pings the massive frontier models in
- 7:57the cloud when it absolutely has to For complex
- 8:00reasoning. Which is a brilliant move for privacy,
- 8:03security, and speed. Right. To make that kind
- 8:05of local execution possible, we're seeing incredible
- 8:08technical leaps. Look at Google GEMMA 412B. Let's
- 8:13talk about GEMMA. This is a new model that processes
- 8:15text, vision, and audio natively. It doesn't
- 8:18use separate encoders for different types of
- 8:20media, which makes it incredibly efficient. It
- 8:22can run locally on just 16 gigabytes of VRAM.
- 8:25Okay, I want to pause there because I hear terms
- 8:27like encoders and VRAM, and I know some people's
- 8:30eyes might start to glaze over. Break down why
- 8:33that is actually a structural leap for someone
- 8:35who just wants their laptop to be secure. Okay,
- 8:38think about how older AI models worked. If you
- 8:41showed an AI a picture, it couldn't actually
- 8:44see it. Right, it's just code. Exactly. It had
- 8:47to use a separate piece of software, an encoder,
- 8:50to translate that picture into a giant block
- 8:52of text describing the picture. And then the
- 8:55AI could process the text. It sounds slow. It
- 8:58is. It takes a massive amount of computing power
- 9:00and memory. But Gemma 412B processes it natively.
- 9:05It doesn't need a translator. It just sees the
- 9:07image and hears the audio directly. That's a
- 9:09huge shift. Yeah. And because it's so efficient,
- 9:12it only requires 16 gigabytes of video RAM or
- 9:16VRAM, which is just the memory on a standard
- 9:18graphics card. Which means it can run on a high
- 9:20end consumer laptop, entirely disconnected from
- 9:22the Internet, completely secure and private.
- 9:25Exactly. You don't need a server farm. You just
- 9:27need a good laptop. But of course, the big tech
- 9:29companies aren't just retreating to local hardware
- 9:32to solve these problems. They're also trying
- 9:34to fight fire with fire. There is a new empowered
- 9:37tool called Astra Autonomous Pentest, and it
- 9:41literally uses AI agents to find, validate, and
- 9:44fix vulnerabilities. It just runs constantly
- 9:47in the background. Right. It operates as native
- 9:49pumps inside coding environments like Cursor,
- 9:52Copilot, and Cloud Code, just hunting for security
- 9:55flaws in real time. But I have to say, using
- 9:58AI to fix AI vulnerabilities feels a bit like
- 10:01hiring a sloppy builder to inspect their own
- 10:03house, right? It is a massive gamble. I mean,
- 10:05you are trusting a system known for making bizarre
- 10:08logical leaps to spot its own architectural flaws.
- 10:12And the irony is really peeking over at Google,
- 10:14because right now, 75 % of Google's code is officially
- 10:17AI generated. 75%. Yes. But internally, employees
- 10:21are sharing memes openly joking that the AI constantly
- 10:24hallucinates and actually makes their engineering
- 10:25work harder. Because they have to untangle its
- 10:28mess. It is a brilliant paradox. We're relying
- 10:30on AI to do the heavy lifting of code generation
- 10:33because it's fast. You can write a million lines
- 10:36of code in a second. And the quality. Right.
- 10:38The structural precision, the fundamental reliability,
- 10:40it just isn't fully there yet. It might confidently
- 10:43invent a function that doesn't actually exist
- 10:46in the programming language and then confidently
- 10:48use that fake function across hundreds of files.
- 10:51What a nightmare to clean up. If we connect this
- 10:54to the bigger picture. This is exactly why big
- 10:57tech is pivoting so hard right now. They realize
- 11:00they can't just rely on raw open -ended chatbots
- 11:03anymore. Asking a general AI to write secure
- 11:07code or manage a business is way too risky. So
- 11:10they're specializing. Exactly. They're building
- 11:12specialized, highly controlled, goal -oriented
- 11:15agents. And we are seeing that specialization
- 11:17everywhere. Meta just launched business agent
- 11:20globally. You can now deploy these specialized
- 11:22AI agents across WhatsApp, Instagram, and Messenger.
- 11:25And they have a very narrow focus. Right. Their
- 11:27entire job, their only function, is to book appointments,
- 11:30qualify leads, and close sales. No philosophical
- 11:33chats, no hallucinating weird code, just highly
- 11:36constrained business logic. Because constrained
- 11:39parameters yield better, safer results. You narrow
- 11:42the focus to reduce the risk. Facebook is doing
- 11:45the exact same thing for content creators now.
- 11:47Oh, the strategist tool. Yeah, they're giving
- 11:49all creators an AI strategist. It doesn't just
- 11:52write a generic post for you. It analyzes your
- 11:54specific audience data, suggests ideas, tracks
- 11:57market trends, and spots viral potential before
- 12:01you even post. And it's not just text and business
- 12:03logic either. Suno, the AI music generation platform,
- 12:07just raised over $400 million at a $5 .4 billion
- 12:11valuation. Massive numbers. And their big next
- 12:14move, they are launching their first model built.
- 12:16specifically with music industry partners. So
- 12:19instead of just blindly scraping the entire internet
- 12:21and causing absolute copyright chaos, they are
- 12:24moving toward controlled, industry -compliant
- 12:26AI. It is all about bringing guardrails and precision
- 12:29to a technology that has, until now, been incredibly
- 12:32chaotic. The Wild West phase is ending. And that
- 12:35deep need for precision brings us to one of my
- 12:38absolute favorite topics today. Because this
- 12:41lack of control hasn't just been an engineering
- 12:43headache or a business problem, it has completely
- 12:47revolutionized how we generate media. Yes, the
- 12:50creative side. The era we are leaving behind
- 12:52is what we can call the slot machine era of AI
- 12:55art. And it is finally... dead it is a profound
- 12:59shift for creatives everywhere for the longest
- 13:01time prompting an image generator was literally
- 13:04like pulling the lever on a casino slot machine
- 13:06you type in a prompt pull the lever and get a
- 13:09beautiful high definition image but maybe it
- 13:12wasn't quite right right maybe your main subject
- 13:14had six fingers on one hand or there was a coffee
- 13:17cup just randomly hovering floating in midair
- 13:20in the background it used to happen all the time
- 13:21all the time and the worst part was trying to
- 13:24fix it you couldn't just tell the ai hey keep
- 13:26everything exactly exactly the same but move
- 13:28the cup to the table, you either had to pull
- 13:30the lever again, lose the entire image you just
- 13:32liked and pray the next roll was better, or you
- 13:35had to export it to Photoshop and clone stamp
- 13:37it yourself by hand. You had the magic of generation,
- 13:40but you completely lacked structural control.
- 13:44And that is what has fundamentally changed. Two
- 13:47new models from Image Labs called Edeogram and
- 13:50Reeve have completely flipped the script on how
- 13:53this works. Let's talk about those. They now
- 13:55rely on structured layouts and agentic control.
- 13:58Let's do Ideagram 4 .0 first because this is
- 14:00a huge leap. It has gone completely open source.
- 14:03For anyone unfamiliar, that means the underlying
- 14:05brain of the AI, the open weights, are available
- 14:08for anyone to download, inspect, and build upon
- 14:11rather than being locked away behind a corporate
- 14:14API. Which is huge for the community. Huge. And
- 14:17right now, it holds the top spot for open weights
- 14:20on the Design Arena leaderboard. Which is essentially
- 14:22a massive blind taste test. Exactly. Professional
- 14:25designers vote on which AI generates the best
- 14:27image. Ideogram absolutely dominates at the hard
- 14:31stuff. Text rendering, typography, graphic design.
- 14:33It is even beating out closed proprietary rivals
- 14:37in those blind tests. And the mechanism behind
- 14:39how it does that is what makes it so powerful.
- 14:42It uses a JSON -driven approach. Break down JSON
- 14:45for us, for those who aren't coders. Right. So
- 14:47JSON is essentially a lightweight data map. It's
- 14:50a way to organize information using text so a
- 14:52computer can easily read it. With older models,
- 14:55the AI just mushed all the pixels together based
- 14:58on your text prompt. Like a flat painting. Exactly.
- 15:01With Ediogram, the AI generates a JSON map of
- 15:04the image first. It creates a structural blueprint.
- 15:07So if you want to move that floating coffee cup
- 15:09or tweak a background element, you don't re -roll
- 15:12the image and lose everything. You don't pull
- 15:14the slot machine lever again. Right. You literally
- 15:16just open the text file, find the coordinates
- 15:18for the coffee cup in the code, and change the
- 15:21layout. The image updates instantly. keeping
- 15:24the rest of the scene perfect. That is mind -blowing.
- 15:26You are editing the physical structure of a generated
- 15:29image directly through data coordinates. Exactly.
- 15:32You are no longer just prompting, you are directing.
- 15:34And Rave did something very similar with their
- 15:36new 2 .0 model. Rave just dethroned NanoBanana
- 15:392 to claim the number two spot overall on the
- 15:42main text -to -image leaderboard, sitting right
- 15:45behind GPT -Image 2. Which is wild, because NanoBanana
- 15:482 was the reigning champion for a long time.
- 15:51Yeah, so unseating it is a massive deal in the
- 15:54AI community. Review .0 outputs images with labeled
- 15:58segments. You can actually see the structural
- 16:01blocks of the image on your screen and rewrite
- 16:03the layout directly. Here's where it gets really
- 16:05interesting, though. Because Ideagram open sourced
- 16:09these weights, it proves that the open source
- 16:11community is the one actively defining the step
- 16:14change. Absolutely. It's not just a cool feature
- 16:17locked behind a massive, expensive corporate
- 16:19paywall. It's a fundamental upgrade to how humanity
- 16:22interacts with generative media. And it's being
- 16:25driven by the community. It is a true maturation
- 16:27of the medium. We're seeing broader media upgrades
- 16:29across the board, too. Like Grok Imagine 1 .5
- 16:33preview was just released via their API. I saw
- 16:36that, yeah, at x .aii Imagine. Right, and it
- 16:38brings much sharper realism, incredibly better
- 16:40audio syncing for video generation, and vastly
- 16:43stronger prompt following. Which is so critical
- 16:45right now because we desperately need these tools
- 16:47to help creators avoid producing what the internet
- 16:49is now collectively calling AI slop. Yes, the
- 16:52slop. You mentioned Reddit being flooded with
- 16:55spam earlier. There was a viral post recently
- 16:57warning against four very specific overused design
- 17:01patterns that people now instantly associate
- 17:04with lazy AI generated website building. Right.
- 17:09Because when everyone has access to the exact
- 17:10same slot machine. Everything starts to look
- 17:13exactly the same. The Internet just loses its
- 17:15texture. It gets boring. Yeah. But when you give
- 17:17creators precise layout control, like with Enneagram's
- 17:21JSON structures or Reeve's labeled segments,
- 17:23you empower them to make actual intentional design
- 17:26choices again. You escape the slob because you
- 17:28can inject human case back into the structure.
- 17:30So what does this all mean for you listening
- 17:32today? When you look at all these pieces together,
- 17:34it paints a really vivid picture of where we
- 17:36are right now. Very clear trajectory. The overarching
- 17:39journey of the AI ecosystem today is a massive,
- 17:42high -stakes tug of war. On one hand, AI has
- 17:46reached a scale where it is so ubiquitous that
- 17:48it can be invisibly hijacked through a simple
- 17:50WhatsApp notification sitting on your lock screen.
- 17:52And it's powerful enough to flood our forums
- 17:55with synthetic spam. But on the other hand, the
- 17:58response to that chaos is a beautiful evolution
- 18:01toward precision. The technology is evolving
- 18:04into a highly controllable tool that can run
- 18:07natively and privately right on your laptop.
- 18:10It's a totally different approach. It allows
- 18:12you to edit reality, whether that's an image,
- 18:15a song, or a complex business workflow, with
- 18:19the absolute precision of layout code. Wait,
- 18:22I mean, that completely flips the narrative.
- 18:24You are moving from being a passive passenger
- 18:26on this AI ride to being a pilot. That's a great
- 18:30way to put it. But being a pilot means you have
- 18:31to actually do your pre -flight checks. So the
- 18:34biggest takeaway for you today, please go into
- 18:36your AI assistance and disable any integrations
- 18:39you are not actively using. Protect yourself
- 18:42from those invisible prompt injections. Lock
- 18:44your digital doors. Security absolutely has to
- 18:47be a proactive habit now. It can no longer be
- 18:50an afterthought. And if you are looking to actively
- 18:52manage your piece of this ecosystem rather than
- 18:55just letting it wash over you, there are a couple
- 18:57of community tools mentioned in the sources worth
- 18:59checking out. Oh, right. The community tools.
- 19:02Yeah. If you are running a business and dealing
- 19:04with that AI spam filter problem we talked about,
- 19:07look into MailWarm 2 .0. It's a premium email
- 19:10warm -up and deliverability system that ensures
- 19:12your legitimate messages actually reach the inbox
- 19:15instead of getting flagged as bot slop. Very
- 19:18necessary right now. And if you want to learn
- 19:20how to actually build and control these AI agents
- 19:22yourself, check out Build Club Campus. It's a
- 19:25fun, gamified, community -driven virtual AI school
- 19:28where you learn these concepts by actually building
- 19:31things with your own hands. What's fascinating
- 19:33here is how quickly we are moving from being
- 19:35passive consumers of AI magic, just pulling the
- 19:39slot machine lever and hoping for the best to
- 19:41becoming active, secure managers of our own AI
- 19:43infrastructure. It is a completely different
- 19:45mindset. But as we wrap up this deep dive, I
- 19:48want to leave you with one final, deeply unsettling,
- 19:51but absolutely fascinating thought to ponder.
- 19:53Let's hear it. We've talked today about how human
- 19:55hackers are hiding invisible prompts and WhatsApp
- 19:58messages to trick your A .I. And we also know
- 20:01from Google that A .I. is now actively writing
- 20:0475 percent of the code at some of these massive
- 20:06tech companies, often hallucinating and making
- 20:09bizarre structural mistakes along the way. Right.
- 20:12So what happens the day an A .I. hallucination
- 20:14organically writes a line of code that accidentally
- 20:16acts as an indirect prompt injection? Could an
- 20:19A .I. hack another A .I. entirely by mistake
- 20:22without a human Wow. Something to think about
- 20:26next time your phone buzzes.