Latest / AI Fire Daily / #585 Neil: AI Image Model Battle Puts GPT, Nano And Grok To The Test

Transcript

#585 Neil: AI Image Model Battle Puts GPT, Nano And Grok To The Test · AI Fire Daily

• 0:00
  Imagine asking an AI for a simple photo of a
• 0:03
  rainy Tokyo street. In seconds, it hands you
• 0:07
  this stunning cinematic masterpiece. Right. I
• 0:10
  mean, it looks completely flawless at first glance.
• 0:13
  But then you'll look closer. The customer on
• 0:16
  the corner is holding two umbrellas. One of those
• 0:19
  umbrellas is somehow melting into a bowl of ramen.
• 0:22
  Yeah. And the neon street signs are just, you
• 0:25
  know, alien gibberish. It really is a beautiful
• 0:28
  illusion. Welcome to the deep dive. I'm so glad
• 0:30
  you're joining us today. We are tearing down
• 0:32
  that illusion today. We have a pretty serious
• 0:34
  mission here. We absolutely do, because honestly,
• 0:37
  I still wrestle with prompt drift myself. Like,
• 0:40
  fixing just one word often breaks the entire
• 0:42
  image. It drives everyone crazy, and that is
• 0:45
  exactly why we're here. We are putting three
• 0:47
  top -tier models through an absolute gauntlet.
• 0:49
  We have GBT Image 2, NanoBanana 2, and Grok Image
• 0:52
  2. Right, we are running them through five brutal
• 0:55
  real -world tests to see which one is actually
• 0:57
  worth paying for. Let us jump into the first
• 0:59
  scenario. We are testing complex prompt following.
• 1:02
  The scene we requested is a rainy Tokyo ramen
• 1:05
  stall at night. That classic cyberpunk aesthetic.
• 1:08
  Models usually eat that up. They do, but we loaded
• 1:11
  the prompt with heavy conflicting constraints.
• 1:14
  We asked for exactly five people. And the vendor
• 1:17
  had to wear a red apron, hold a white towel,
• 1:20
  and carry a ladle. Plus, we wanted a customer
• 1:22
  in a yellow raincoat. They need a clear umbrella
• 1:25
  with a blue handle. And crucially, they must
• 1:28
  have absolutely no phone in their hand. Oh, that
• 1:31
  negative prompt is a trap. Telling a machine
• 1:33
  what not to draw is notoriously difficult. Wait,
• 1:36
  I want to unpack that. If I explicitly type the
• 1:38
  words no phone, isn't the model just deleting
• 1:41
  phones from its process? Well, no, because of
• 1:44
  tokenization. When you type the word phone, you
• 1:46
  are introducing that mathematical token into
• 1:48
  the model's latent space. It is like me telling
• 1:51
  you, do not think about a pink elephant. Right,
• 1:53
  and suddenly I am picturing one. Exactly. The
• 1:56
  concept is now active in the math. The model
• 1:58
  has to work incredibly hard to suppress the visual
• 2:01
  representation of that token. That explains so
• 2:04
  much. To finish the prompt, we also asked for
• 2:06
  a specific bowl order on the counter. Red, white,
• 2:09
  and black. With steam only allowed from the white
• 2:12
  one. Right. And a red bicycle on the far left
• 2:15
  with its wheel turned. That is just a massive
• 2:17
  cognitive load, so how did they handle the pressure?
• 2:20
  GPT Image 2 took the win for this test. It scored
• 2:23
  a 9 .5 out of 10. Wow. It nailed almost everything,
• 2:27
  exactly five people, and the bowl order was perfect.
• 2:30
  It kept that rainy cinematic mood intact too,
• 2:33
  right, without hallucinating random objects.
• 2:35
  Yeah, it did really well. Nano Banana 2 struggled
• 2:38
  more, scoring a 6 .5. The realism looked great,
• 2:42
  but it failed the strict counts entirely. It
• 2:45
  added extra people into the street, and it gave
• 2:47
  the Yellow Raincoat customer a second umbrella.
• 2:50
  Grok Image 2 was the fastest to generate, but
• 2:52
  it scored a 5 .5. It completely failed the negative
• 2:56
  prompt. Let me guess the customer was holding
• 2:58
  a phone. Right in their hand. It also broke the
• 3:01
  spatial rules. The background people were supposed
• 3:03
  to be under a striped awning. But Grok just scattered
• 3:06
  them into the rain. Trying to render this prompt
• 3:08
  is like balancing a checkbook, where the math
• 3:11
  keeps changing on its own. You fix one variable
• 3:13
  and three others shift. The attention mechanism
• 3:16
  in the model just gets spread too thin. It loses
• 3:18
  track of the rules. Is speed worth it if the
• 3:20
  model completely ignores the negative instructions?
• 3:23
  No, ignoring constraints just creates more editing
• 3:25
  work later. Beat. So, if GPT Image 2 can perfectly
• 3:30
  parse physical objects like raincoats and bowls,
• 3:33
  how does it handle human language? Language is
• 3:35
  abstract. This is a fascinating leap. Text is
• 3:39
  notoriously difficult for AI. They do not understand
• 3:42
  phonetic letters. They just see complex geometric
• 3:44
  shapes. Exactly. We tested them with a graphic
• 3:47
  design poster, a fictional Tokyo event called
• 3:50
  Midnight Market. The constraints here were incredibly
• 3:53
  strict layout rules. We demanded a dark black
• 3:55
  background, red cream and silver accents, strict
• 3:59
  typography rules like Midnight Market in the
• 4:01
  top left. And the date needed to sit inside a
• 4:04
  red box in the top right. We also explicitly
• 4:06
  forbade fake Japanese characters. No random symbols
• 4:09
  allowed. Right in the center we asked for a chrome
• 4:12
  sphere reflecting light and exactly three thin
• 4:15
  red geometric lines behind it. GPT image 2 took
• 4:18
  the win again with a 9 .5. The text was beautifully
• 4:22
  clean and the layout was spot on. What was its
• 4:24
  only flaw? The red lines, they looked a bit more
• 4:27
  like thick geometric shapes rather than thin
• 4:30
  lines, but a very minor deviation. Nano Banana
• 4:33
  2 scored an 8 .5. The text readability was actually
• 4:36
  great. Yeah, but it failed on constraint discipline.
• 4:39
  It added extra red lines that we never asked
• 4:41
  for. And the sparkle. Right. It threw in this
• 4:44
  completely unprompted aesthetic sparkle near
• 4:46
  the text. Grok Image 2 scored an 8 .7. Surprisingly
• 4:49
  clean text, but it added a massive red bar on
• 4:52
  the left side. Totally unprompted. Why do these
• 4:55
  models still invent extra graphics when told
• 4:57
  not to? They overcompensate, trying to fill empty
• 5:00
  space with aesthetic noise. Beat. We're going
• 5:03
  to take a quick pause here. Sponsor. Welcome
• 5:05
  back. So text on a poster is essentially flat.
• 5:08
  It is a two -dimensional layout. But the real
• 5:10
  world has depth. Do these models actually understand
• 5:13
  space, or are they just faking it? That brings
• 5:15
  us to our third test, spatial reasoning. Which
• 5:18
  is such a huge hurdle for these systems. Let
• 5:21
  us define that briefly. Spatial reasoning means
• 5:24
  understanding how objects physically relate to
• 5:27
  each other in 3D space. It tests logic. Left,
• 5:30
  right, behind, under, above. We gave them a modern
• 5:32
  reading room to build, a desk in the center,
• 5:35
  a red chair on the left, a blue chair on the
• 5:38
  right. The trickiest element was the backpack.
• 5:41
  Oh, absolutely. We asked for a green backpack
• 5:44
  on the floor. It had to sit exactly between the
• 5:46
  red chair and the desk. Completely outside the
• 5:49
  footprint of the desk itself. We also added a
• 5:51
  white cat. It needed to sit under the desk, but
• 5:54
  specifically facing the blue chair. Plus, framed
• 5:57
  pictures aligned above specific items. GPT Image
• 6:00
  2 won this round with a 9 .4. It is remarkably
• 6:04
  reliable for these spatial puzzles. The chairs
• 6:06
  were correct. The backpack was in the right zone.
• 6:09
  His cat was positioned beautifully. Did it miss
• 6:11
  any relational logic at all? Just one small alignment
• 6:14
  issue. A square picture on the wall wasn't perfectly
• 6:16
  centered over the blue chair. But the overall
• 6:18
  comprehension is staggering. Nano Banana 2 scored
• 6:22
  an 8 .6. It handled the backpack well, but the
• 6:25
  cat was facing the entirely wrong direction.
• 6:27
  Grok Image 2 really fell apart here. It scored
• 6:30
  an 8 .1. Yeah, Grok shoved the green backpack
• 6:34
  completely under the desk. Yeah. And the cat
• 6:36
  was facing the front left. ignoring the blue
• 6:38
  chair entirely. So what is actually happening
• 6:40
  under the hood here? Why is left and right so
• 6:43
  hard for a supercomputer? Because the AI does
• 6:47
  not have a 3D engine. When a video game renders
• 6:49
  a room, it creates a mathematical z -axis. It
• 6:52
  plots coordinates in three dimensions, but the
• 6:55
  AI doesn't do that. No. The AI predicts flat
• 6:57
  pixels. It knows that pixels representing a cat
• 7:00
  and pixels representing a desk often overlap
• 7:03
  in its training data. So it just groups those
• 7:05
  concepts together based on statistical probability?
• 7:08
  Yes. It does not truly understand the semantic
• 7:10
  meaning of under. It just mimics the visual patterns
• 7:13
  associated with the word. Does struggling with
• 7:15
  left and right mean the model lacks true comprehension?
• 7:18
  Yeah, they predict pixels. They don't actually
• 7:20
  build a 3D mental map. Two sec silence. Let us
• 7:24
  shift away from pure logic. Understanding where
• 7:26
  a chair goes is one thing. But making the wood
• 7:29
  grain and the leather look absolutely real is
• 7:32
  an entirely different beast. Test 4 focuses purely
• 7:35
  on aesthetics. We are testing photorealism. We
• 7:39
  asked the models for a portrait of a 35 -year
• 7:42
  -old female ceramic artist at Golden Hour. And
• 7:45
  we demanded raw realism. We explicitly asked
• 7:48
  for visible pores, freckles, and absolutely no
• 7:51
  beauty filters. Hands are always the ultimate
• 7:53
  AI test. We needed five perfect fingers, unpainted
• 7:57
  nails, and traces of wet clay on the skin. We
• 7:59
  also demanded distinct material textures. Linen
• 8:02
  on the apron, cotton for the shirt, wet clay,
• 8:05
  and dry wood. plus three distinct bowls on a
• 8:07
  shelf behind her. Grok Image 2 finally took a
• 8:10
  decisive win here. It scored a 9 .5. The textures
• 8:13
  were unbelievable. Captured the most natural
• 8:15
  camera feel. The way the clay dust hung in the
• 8:18
  air caught the golden hourlight perfectly. Whoa.
• 8:21
  Imagine conjuring an editorial -grade hypertextured
• 8:24
  portrait from your living room in seconds. It
• 8:26
  is truly mind -bending. Grok did miss one logical
• 8:29
  rule, though. The bowl was resting on the bench
• 8:31
  instead of fully in her hands. But the raw realism
• 8:33
  was so breathtaking, it won the category. GPT
• 8:36
  Image 2 scored a 9 .2. It followed the rules
• 8:39
  much better, but it had a totally different vibe.
• 8:41
  It looked way too perfect. It looked like a heavily
• 8:44
  produced million -dollar advertising campaign.
• 8:47
  Nano Banana 2 got an 8 .7. It rendered very believable
• 8:51
  skin. But it moved the balls from the shelf to
• 8:53
  the window, and it added another unprompted watermark
• 8:57
  and sparkle. I want to talk about GPT's hyper
• 9:00
  -polished look. Why does it default to that advertising
• 9:04
  sheen? It comes down to human reinforcement.
• 9:06
  These models are trained on internet data, which
• 9:09
  is heavily photoshopped. But then they go through
• 9:11
  reinforcement learning from human feedback. Exactly.
• 9:14
  Human testers tell the AI what looks good. And
• 9:17
  human testers consistently rate bright, smooth,
• 9:20
  idealized images higher than gritty, imperfect
• 9:23
  ones. So the AI learns to default to perfection.
• 9:26
  Is GPT's tendency to over -polish actually a
• 9:29
  flaw for professional photography? Absolutely.
• 9:31
  Perfection destroys the gritty authenticity needed
• 9:34
  for true photorealism. Beat. We have tested logic.
• 9:37
  text, spatial awareness, and realism individually.
• 9:41
  What happens when we force a model to juggle
• 9:43
  all of those elements at the exact same time?
• 9:45
  That is the final gauntlet. Test 5 is multi -constraint
• 9:49
  design. We essentially asked the models to act
• 9:51
  as a full -service ad agency. We created a brief
• 9:54
  for a fictional sparkling water called NoVA.
• 9:58
  This brief was an absolute nightmare. We asked
• 10:00
  for a deep midnight blue background and three
• 10:03
  slim cans. We gave exact color codes. Matte silver,
• 10:07
  glossy blue, and matte white. The word NOVA had
• 10:11
  to be written vertically down each can. We needed
• 10:14
  exactly four ice cubes and none of them could
• 10:17
  sit in front of the center can. We asked for
• 10:18
  a thin curved splash of water. Exactly five surface
• 10:22
  water droplets. And specific typography. A headline
• 10:25
  reading Cased the Night. and a white call to
• 10:28
  action box saying, try Novia. Trying to render
• 10:30
  this prompt is like stacking Lego blocks of data
• 10:33
  while someone bumps the table. That is exactly
• 10:35
  how the Attention Matrix feels. Nano Banana 2
• 10:38
  actually won this round. It scored a 9 .3 out
• 10:40
  of 10. Nano Banana was the dark horse. It really
• 10:43
  was. It held the complex brief together beautifully.
• 10:46
  It kept the water splash perfectly situated behind
• 10:48
  the cans. And it managed all the typography without
• 10:50
  cluttering the layout. GPT Image 2 scored an
• 10:53
  8 .8. It really lost control of the water mechanics.
• 10:56
  It rendered a massive, chaotic splash that drowned
• 10:59
  the cans, and way too many water droplets. The
• 11:02
  subtle elements just collapsed. Grok Image 2
• 11:05
  scored a 7 .9. Grok completely dropped the Tree
• 11:09
  Rye Novia call to action. The buttons just vanished.
• 11:13
  It also missed the surface droplets entirely.
• 11:15
  Why does a powerful model just forget a huge
• 11:18
  element like a CTA button? It is context dilution.
• 11:21
  Every constraint you add forces the model to
• 11:23
  divide its attention weights. When you demand
• 11:25
  five droplets, three colors, and four ice cubes,
• 11:28
  the mathematical weight assigned to the text
• 11:30
  gets crushed. It literally runs out of focus.
• 11:33
  When a model forgets the core CTA button, is
• 11:35
  the image essentially useless for marketers?
• 11:38
  Yes, a gorgeous ad that forgets to sell is just
• 11:40
  expensive wallpaper meat. So we have run the
• 11:43
  gauntlet. Let us tally the final standings and
• 11:44
  figure out what this actually means for you.
• 11:46
  GPT Image 2 takes the overall crown. It finished
• 11:49
  with a 9 .28 average. It won 3 out of the 5 tests
• 11:53
  outright. It is definitely the safest choice
• 11:55
  for strict detail. If you need precise text,
• 11:56
  perfect counts, and spatial obedience, GPT is
• 11:59
  your primary tool. It obeys logic better than
• 12:02
  the rest. But NanoBanana 2 surprised me. It finished
• 12:06
  with an 8 .32 average. It is the absolute dark
• 12:10
  horse for heavy design briefs. When you need
• 12:12
  a multi -constraint layout, it balances all its
• 12:15
  competing elements wonderfully. And Grok Image
• 12:17
  2 finishes with a 7 .94 average. Grok struggles
• 12:21
  deeply with strict rules and negative prompts.
• 12:24
  But if you want the undisputed king of raw, unpolished
• 12:27
  editorial photorealism... Grok is untouchable.
• 12:30
  You know, seeing these scores really drives home
• 12:32
  a crucial point. Do not just look at generic
• 12:34
  top 10 lists online. You have to pick your tool
• 12:37
  based on the specific battle you are fighting
• 12:39
  today. Exactly. Are you fighting for perfect
• 12:41
  typography on a poster? Or are you fighting for
• 12:44
  the gritty realism of human skin texture? The
• 12:47
  right model changes entirely based on that answer.
• 12:49
  I highly encourage you to test this phenomenon
• 12:51
  yourself. Yeah. Take one single highly detailed
• 12:54
  prompt from your own work. Run it through two
• 12:56
  different models today. You will clearly see
• 12:58
  these hidden biases and architectural flaws emerge
• 13:01
  in real time. It really is the best way to understand
• 13:03
  how these diffusion systems actually process
• 13:06
  information. Absolutely. Which brings up a lingering,
• 13:09
  somewhat philosophical question to leave you
• 13:11
  with. We have seen that these models still struggle
• 13:14
  with basic object permanence, like failing to
• 13:17
  understand that a cat is facing left instead
• 13:19
  of right. So how long until they finally realize
• 13:22
  they are not just painting static pixels on a
• 13:24
  flat canvas, but mathematically simulating our
• 13:26
  entire 3D reality?