Latest / AI Fire Daily / #676 Neil: Kimi vs Claude Built 3 Projects With Surprising Results

Transcript

#676 Neil: Kimi vs Claude Built 3 Projects With Surprising Results · AI Fire Daily

• 0:00
  Imagine getting top -tier engineering results
• 0:02
  for half the price. Yeah. So a brand new AI model
• 0:05
  costs exactly half as much as the reigning champion
• 0:08
  Claude Opus 5 .5. But in a fully autonomous coding
• 0:12
  duel, this cheaper underdog didn't just keep
• 0:14
  up. It actually crushed the champion in a speed
• 0:17
  and accuracy test. Welcome to this deep dive.
• 0:21
  We're unpacking an intense autonomous... coding
• 0:23
  face -off today. It went for three full rounds.
• 0:26
  We're looking at Kimi K3 .1 Pro and Claude Opus
• 0:29
  5 .5. Right. Both models were unleashed inside
• 0:32
  Claude code. They had to build software entirely
• 0:35
  from scratch. Absolutely no human intervention
• 0:37
  was allowed here. We'll explore their performance
• 0:39
  across three distinct challenges. First up is
• 0:42
  a visual website replication test. Yeah. And
• 0:45
  second is a fully functional interactive web
• 0:47
  app. And third is a complex 3D browser game.
• 0:50
  So we're trying to answer one core question for
• 0:52
  you today. Can you really get top tier engineering
• 0:55
  results for half the API token price? That's
• 0:58
  the multi -million dollar question right now.
• 1:00
  We're putting these models through an absolute
• 1:02
  gauntlet. They aren't just writing a few lines
• 1:04
  of script. They have to plan, write code, and
• 1:07
  actually test their own work. Wow. They even
• 1:10
  have to find and fix their own mistakes. Their
• 1:13
  operating is completely independent software
• 1:15
  engineers. Let's get right into this first challenge.
• 1:18
  We're starting with surface -level front -end
• 1:20
  engineering. The models are translating static
• 1:23
  pixels into functional code. The challenge was
• 1:26
  to recreate a website called Orbit. And they
• 1:28
  only get a single screenshot as a reference.
• 1:31
  They have to match the layout exactly from that
• 1:34
  one image. That means reproducing typography,
• 1:37
  colors, and the specific spatial spacing. They're
• 1:41
  using a very modern web development stack here.
• 1:43
  They're building with React and TypeScript for
• 1:46
  the core engine. It's kind of like stacking Lego
• 1:48
  blocks of data. That's a great way to put it.
• 1:51
  They're using Vite for the build tooling. They
• 1:53
  also use Tailwind CSS for the styling. But the
• 1:57
  real secret weapon here is Playwright. Right.
• 1:59
  Playwright acts like a robot QA tester. It can
• 2:02
  actually look at the web page. Exactly. Playwright
• 2:05
  is crucial for the automated visual testing.
• 2:07
  The models can't just guess at the visual layout.
• 2:10
  They have to complete five distinct visual review
• 2:13
  cycles. They essentially take a photo of their
• 2:16
  own coded website. Then they compare it directly
• 2:19
  to the original reference screenshot. This mechanism
• 2:21
  is honestly brilliant to think about. It's like
• 2:24
  playing a high speed game of spot the difference.
• 2:26
  The AI overlays that photo onto the reference
• 2:29
  image. It highlights the mismatched pixels to
• 2:31
  figure out where it messed up. Right. And it
• 2:34
  checks the viewport at three exact widths. It
• 2:36
  looks at the mobile view at 375 pixels. It checks
• 2:40
  the tablet view at 768 pixels. And the full desktop
• 2:43
  view at 1440 pixels. Wow. So it has to fix responsive
• 2:47
  layout issues autonomously based on those screenshots.
• 2:51
  The constraints on this prompt are incredibly
• 2:53
  intense. It forces a strict six -phase development
• 2:56
  cycle. It really does. I mean, I still wrestle
• 3:00
  with prompt drift. myself. Keeping an AI focused
• 3:03
  through six distinct phases is incredibly tough.
• 3:07
  Right. Prompt drift happens when a model forgets
• 3:09
  its original goal. It just wanders off topic
• 3:11
  over a long conversation. Exactly. But this rigid,
• 3:15
  forced structure prevents them from just cutting
• 3:17
  corners. I'm looking at the actual results here.
• 3:20
  The cheaper model, Kemi K3 .1 Pro, finished in
• 3:23
  just 12 minutes. It achieved perfect visual accuracy
• 3:26
  across the board. That speed is honestly staggering
• 3:29
  to me. 12 minutes for a fully responsive, pixel
• 3:33
  -perfect React clone. It managed to perfectly
• 3:36
  replicate the entire visual structure flawlessly.
• 3:39
  Meanwhile, the heavy hitter took 20 full minutes.
• 3:42
  Claude Opus 5 .5 actually left a noticeable text
• 3:46
  alignment error. The error was inside the See
• 3:49
  How It Works button. Yeah, Claude just couldn't
• 3:51
  get that button aligned properly. Despite having
• 3:54
  the same five visual review cycles, it failed.
• 3:58
  There's a very visible obvious front -end mistake.
• 4:01
  You can see the text clipping out of the button
• 4:03
  boundaries. I'm honestly confused by this result.
• 4:06
  You'd think the heavier, smarter model would
• 4:08
  crush this. Is Claude getting bogged down overthinking
• 4:12
  the underlying architecture? Why might a cheaper
• 4:14
  model actually be 40 % faster and more accurate
• 4:17
  here? It likely comes down to how these different
• 4:19
  models process spatial constraints. Claude Opus
• 4:22
  5 .5 is a deeply analytical, highly complex reasoning
• 4:27
  engine. It might overcomplicate the relationship
• 4:30
  between the webpage elements. Ah, I see. It tries
• 4:33
  to build a perfectly robust logical hierarchy.
• 4:37
  Kimi, on the other hand, might just map visual
• 4:40
  coordinates directly to CSS rules. It doesn't
• 4:43
  overthink the underlying logic of the page. It
• 4:45
  just translates space into code very efficiently.
• 4:48
  For pure visual replication, brute force simplicity
• 4:51
  is sometimes actually better. So, Kimi prioritizes
• 4:55
  raw layout efficiency over deep logical processing.
• 4:58
  Exactly. When you just need to copy a static
• 5:00
  picture, overthinking hurts you. That brings
• 5:03
  us to a critical pivot point. Perfect visuals
• 5:06
  are great, but a website isn't a static painting.
• 5:08
  It has to actually do something for the user.
• 5:11
  What happens when we move from static pixel matching
• 5:14
  to interactive logic, when the AI has to invent
• 5:17
  the user experience from scratch? Does that raw
• 5:20
  speed still hold up? This is round two of our
• 5:23
  autonomous face -off. The models had to build
• 5:26
  an app called Taskflow entirely from scratch.
• 5:29
  It's a Trello -style Kanban task management application.
• 5:34
  And crucially, there are no reference images
• 5:35
  provided for this round. They have to design
• 5:37
  the entire user interface themselves from zero.
• 5:40
  The requirements were pretty heavy for a zero
• 5:42
  -human run. They had to build working drag -and
• 5:45
  -drop functionality for the task cards. You have
• 5:48
  to be able to move tasks between to do, in progress,
• 5:51
  and done. They also had to build search and filter
• 5:54
  functions. They had to handle application errors
• 5:56
  gracefully without crashing. The tech stack was
• 5:59
  React, Tailwind, and a library called DNDKit.
• 6:02
  Yeah, and there was also a very strict data persistence
• 6:04
  requirement. They had to use local storage to
• 6:07
  save the task data. Saving data right in your
• 6:10
  browser so it survives a refresh. Right. If you
• 6:12
  close the tab, your tasks absolutely need to
• 6:14
  stay there. The models had to plan the architecture
• 6:17
  and write the data tests. They used Playwright
• 6:19
  again to verify the drag -and -drop physics actually
• 6:22
  worked. The results here are really fascinating
• 6:24
  to compare side by side. Both models took exactly
• 6:27
  the same amount of time to finish. But the final
• 6:29
  interactive products looked completely different.
• 6:32
  Kimi built a very clean, somewhat basic dashboard
• 6:35
  interface. It worked well functionally, but it
• 6:38
  hid all the task details. You had to click deep
• 6:41
  into a card to see anything useful. Exactly.
• 6:43
  Claude, on the other hand, built a highly dense
• 6:46
  SaaS -style dashboard. It looked like something
• 6:48
  you'd pay a monthly subscription for. You could
• 6:52
  see descriptions, deadlines, and priorities at
• 6:54
  a single glance. Wow. So everything was visible
• 6:57
  right on the main Kanban board without clicking.
• 7:01
  Claude Opus 5 .5 definitely wins this round for
• 7:04
  actual product design. It understands what a
• 7:07
  human user actually wants from a task manager.
• 7:09
  I'm going to push back on that conclusion a bit.
• 7:12
  Is Claude's dense data layout actually objectively
• 7:16
  superior? Or is Kimi's hidden detail approach
• 7:20
  just a minimalist design preference? That's a
• 7:23
  fair point. I mean, some users absolutely hate
• 7:27
  cluttered enterprise interfaces. Sometimes they
• 7:31
  just want a clean, simple screen for their tasks.
• 7:34
  That is a very fair counterpoint to make. Minimalism
• 7:37
  is a highly valid design choice in modern software.
• 7:41
  In a project management Kanban board, context
• 7:44
  is usually king. Claude's layout dramatically
• 7:48
  reduces the cognitive load for the user. Right.
• 7:51
  You don't have to click 10 times just to check
• 7:53
  your upcoming deadlines. It feels like a mature,
• 7:57
  battle -tested product right out of the box.
• 8:00
  Kimi's version felt more like a quick wireframe
• 8:02
  mock -up a junior designer would make. Do these
• 8:05
  AI models have inherent design biases based on
• 8:08
  their training data? They absolutely do. This
• 8:11
  goes back to how these models are aligned during
• 8:14
  training. Claude was likely trained on millions
• 8:17
  of examples of successful enterprise software.
• 8:20
  It recognizes the winning patterns of complex
• 8:23
  SaaS products. Interesting. Yeah. It inherently
• 8:26
  knows that priority tags and deadlines are critical
• 8:29
  surface information. It bakes those specific
• 8:32
  UX best practices directly into its code generation.
• 8:36
  Kimmy might just be trained on lighter data sets
• 8:38
  where hiding data is normal. So Claude simply
• 8:41
  understands the practical context of what users
• 8:43
  need. Precisely. It builds for the human context,
• 8:47
  not just the technical spec sheet. We've covered
• 8:49
  UI replication and state management logic. Both
• 8:52
  models now have one distinct win under their
• 8:54
  belts. Kimi took the visual clone and Claude
• 8:57
  took the interactive product. Now we steer into
• 9:00
  the absolute hardest test of all. We're looking
• 9:03
  at complex spatial physics and 3D environment
• 9:06
  generation. The challenge was to build a fully
• 9:09
  playable 3D racing game. It had to run smoothly
• 9:12
  directly in the web browser. The developers called
• 9:15
  this test challenge Velocity Rush. The specific
• 9:18
  requirements for this game were honestly staggering
• 9:20
  for an AI. They needed to generate a coastal
• 9:23
  highway circuit with actual scenery. They needed
• 9:26
  working acceleration, braking, and real -time
• 9:28
  collision physics. They also had to build three
• 9:31
  computer -controlled AI opponents. Those opponents
• 9:33
  needed intelligent waypoint navigation to race
• 9:36
  against you on the track. And here's the real
• 9:38
  kicker for this route. They were completely forbidden
• 9:41
  from using any external 3D models. They had to
• 9:44
  generate all the 3D geometry from scratch using
• 9:46
  a library called 3 .js. Whoa. Imagine an AI generating
• 9:52
  a whole 3D world in minutes. It's basically dreaming
• 9:55
  an entire physics simulation into existence just
• 9:58
  using math. It's quite literally writing the
• 10:00
  math for a virtual reality. It uses another library
• 10:04
  called Canon .es to handle the vehicle physics
• 10:07
  engine. It has to handle speed -dependent steering
• 10:10
  responses and complex camera tracking. Right.
• 10:13
  The camera has to follow the player's car smoothly
• 10:15
  around sharp corners without clipping. So how
• 10:18
  did our two autonomous contenders actually perform
• 10:20
  here? Well, Kimi managed to generate a very clean,
• 10:23
  functional start menu. It built some decent coastal
• 10:26
  scenery with blocky palm trees. But it fundamentally
• 10:30
  failed on the core vehicle placement logic. When
• 10:32
  you start Kimi's game, your car is parked on
• 10:34
  the grass. It's completely off the actual asphalt
• 10:37
  racing track. Right, and it completely ruins
• 10:39
  the illusion of a racing game. Claude Opus 5
• 10:43
  .5, however, built a highly detailed winding
• 10:46
  track. It generated colorful sports cars and
• 10:50
  realistic textured asphalt. And it perfectly
• 10:53
  placed all four vehicles directly on the starting
• 10:55
  line for the race. Think about how wild this
• 10:57
  is for a second. Claude has no eyes. It doesn't
• 11:00
  actually know what a car or asphalt looks like
• 11:03
  visually. Yeah, it's essentially doing physics
• 11:05
  blindfolded, using only mathematical formulas.
• 11:08
  It uses text tokens to see the world, yet it
• 11:11
  perfectly placed four cars. Let's examine Kimi's
• 11:14
  car on the grass bug with that in mind. Is this
• 11:17
  a fundamental failure of the core game logic,
• 11:19
  or is it just a slight, completely understandable
• 11:22
  hallucination in mapping 3D coordinate geometry?
• 11:25
  It's mostly a coordinate mapping hallucination.
• 11:29
  Building 3D spaces purely through text code is
• 11:32
  incredibly abstract. Language models don't possess
• 11:35
  a visual latent space. Right. They rely entirely
• 11:38
  on mathematical grids to place objects. Kimmy
• 11:41
  got the z -axis or x -axis math slightly wrong
• 11:45
  for the spawn point. It placed the car to coordinate
• 11:48
  that happened to be grass instead of road. How
• 11:51
  should we weigh a single coordinate bug like
• 11:53
  that, especially against the fact that the underlying
• 11:55
  game code was structurally sound? We ultimately
• 11:57
  have to judge the final user experience. A structurally
• 12:01
  sound game where you spawn stuck in the grass
• 12:03
  feels broken. Claude managed to hold the entire
• 12:07
  3D spatial map in its active memory. It aligned
• 12:10
  the spawn coordinates perfectly with the complex
• 12:12
  generated track geometry that shows a much, much
• 12:15
  deeper level of spatial reasoning capability.
• 12:17
  It can juggle thousands of coordinates without
• 12:19
  losing track of the physical reality. Small coordinate
• 12:22
  errors break immersion, even if the math is sound.
• 12:25
  Exactly. Claude proved it can handle multi -layered
• 12:28
  spatial logic reliably. It doesn't just write
• 12:30
  code. It understands the spatial relationships
• 12:33
  within that code. OK, let's pull back from the
• 12:36
  specific code tests. We need to look at the macro
• 12:38
  landscape of AI coding right now. We have a final
• 12:41
  scorecard for this intense three -round duel.
• 12:44
  Claude Opus 5 .5 ultimately takes the crown today.
• 12:47
  It cleanly won two out of the three autonomous
• 12:50
  coding tests. It dominated the interactive web
• 12:52
  app and the complex 3D game. However, we absolutely
• 12:57
  cannot ignore Kimi K3 .1 Pro's performance. Kimi
• 13:00
  won the pure visual replication test outright.
• 13:03
  It was 40 % faster and significantly more accurate
• 13:06
  visually than the reigning champion. And we really
• 13:09
  have to remember the cost difference here. Kimi
• 13:11
  costs exactly 50 % less per API token the clock.
• 13:15
  Right. That's a massive budget advantage for
• 13:18
  scaled enterprise operations. If you're running
• 13:20
  thousands of autonomous tasks, that price cap
• 13:23
  is huge. So what's the ultimate verdict for you,
• 13:26
  the listener? If you're a founder or an engineer
• 13:28
  building an app right now, reach for Kimi for
• 13:31
  quick front -end visual replication projects.
• 13:34
  It's fast, cheap, and surprisingly accurate at
• 13:37
  matching static pixels. But you should definitely
• 13:41
  reach for Claude when dealing with complex data
• 13:43
  layout. If you need interactive logic or deep
• 13:46
  spatial reasoning, Claude wins. It just understands
• 13:49
  system architecture and human context much better
• 13:52
  than Kimi right now. There is one very important
• 13:55
  caveat to all of this data. These tests strictly
• 13:59
  evaluated visual and surface -level functional
• 14:02
  results. We didn't test the long -term code maintainability
• 14:05
  of these projects. That's the real hidden cost
• 14:09
  of AI -generated code. Is the code actually readable
• 14:13
  by a human developer six months later? Is the
• 14:16
  underlying architecture scalable? Or is it just
• 14:18
  a tangled brittle mess? We only evaluated the
• 14:21
  final playable results on the screen today. This
• 14:24
  brings up a really provocative thought to leave
• 14:26
  you with. Think about Kimi's performance in that
• 14:28
  very first replication test. It perfectly cloned
• 14:31
  a complex visual website in just 12 minutes.
• 14:34
  Yeah, they did this by automatically testing
• 14:36
  and comparing its own screenshots. It was essentially
• 14:39
  acting as its own fully automated front -end
• 14:41
  QA department. Exactly. So how long until front
• 14:45
  -end visual development is entirely commoditized?
• 14:49
  If a model that costs half the price can perfectly
• 14:51
  replicate designs today, What happens tomorrow?
• 14:54
  It's a scary thought. Will human developers soon
• 14:56
  only be needed for the deep, invisible system
• 14:59
  logic? It's something you really need to mull
• 15:02
  over as this technology accelerates. The surface
• 15:05
  -level pixels are getting much easier and cheaper
• 15:07
  to generate. The invisible logic is where the
• 15:10
  real human engineering value currently remains.
• 15:13
  Thank you for joining us on this Deep Dive.