Latest / AI Fire Daily / #115 Robin: Claude Sonnet 5.5 vs Opus 5.5 - I Tested Both on 7 Real Workflows

Transcript

#115 Robin: Claude Sonnet 5.5 vs Opus 5.5 - I Tested Both on 7 Real Workflows · AI Fire Daily

• 0:00
  Opus 5 .5 costs twice as much as Sonnet 5 .5.
• 0:05
  So why did the cheaper model win four out of
• 0:08
  seven real -world tests? Yeah, well, that is
• 0:11
  the multi -million dollar question. It really
• 0:13
  challenges everything we assume about paying
• 0:14
  a premium for AI. Welcome to the Deep Dive. We
• 0:18
  are breaking down a highly rigorous comparison
• 0:20
  today. Exactly. It's a seven -test head -to -head
• 0:23
  battle. We're looking at Anthropix Clawed Sonnet
• 0:26
  5 .5 versus Opus 5 .5. The stakes here are fascinating
• 0:30
  for you. It is not about which AI is objectively
• 0:34
  the best. It's about knowing exactly when that
• 0:37
  extra cost actually buys a better result. We're
• 0:40
  testing everything from marketing landing pages
• 0:42
  to heavy financial models. I think we need to
• 0:45
  establish the baseline cost first, you know,
• 0:47
  before we dive into the specific tests. Definitely.
• 0:50
  There's one crucial number you need to remember.
• 0:52
  Opus costs roughly twice as much as Sonnet. Right.
• 0:55
  It's $4 versus $2, and that's for 1 million input
• 0:59
  tokens. But the output tokens, that's where it
• 1:02
  really stinks. Oh, absolutely. Jumping to $20
• 1:05
  versus $10. Tokens, by the way, are just the
• 1:07
  basic building blocks of words that AI processes
• 1:10
  and generates. Right. Now, the testing ground
• 1:13
  rules here were incredibly strict, which I love.
• 1:16
  Yeah, they were. Both models use the exact same
• 1:18
  prompts. They both operated on the high effort
• 1:21
  setting. Both were run in parallel. And most
• 1:24
  importantly, they used API billing. Meaning,
• 1:27
  paying for exactly what you use instead of a
• 1:30
  flat monthly fee. Which gives us a much cleaner
• 1:32
  view of the actual compute cost. Exactly. Usage
• 1:35
  can vary wildly on standard $20 a month subscription
• 1:38
  plans. It kind of masks the underlying economics.
• 1:42
  We have an overarching thesis to guide us through
• 1:44
  these results. It revolves entirely around your
• 1:47
  definition of done. Yes. Do you have a very clear,
• 1:52
  rigid definition of what done looks like? Start
• 1:55
  with Sonnet. If you already know exactly what
• 1:57
  the final website should do, or if you know precisely
• 2:00
  how this spreadsheet should function, Sonnet
• 2:03
  usually has enough direction to get there. Exactly.
• 2:06
  But if you have an open -ended goal, try Opus.
• 2:10
  If you want the model to decide what good actually
• 2:13
  looks like. Right. If you need it to act as a
• 2:16
  thought partner to figure out the path, Opus
• 2:18
  has much more room to add value there. I have
• 2:20
  to admit, I still wrestle with prompt drift myself.
• 2:24
  I'm constantly second -guessing which model to
• 2:26
  use for my own work. Oh, it's exhausting, right?
• 2:28
  You want the highest possible quality? You want
• 2:31
  the highest possible quality. But you really
• 2:33
  don't want to burn money needlessly. Yeah. Especially
• 2:36
  on tasks that a cheaper model could handle in
• 2:38
  its sleep. So let me ask you this. Does paying
• 2:42
  double the price ever actually guarantee double
• 2:44
  the value in these raw outputs? Well, you'd think
• 2:47
  so. But the scoring rule we used actually proves
• 2:51
  otherwise. How so? If the outputs were incredibly
• 2:54
  close in quality, the cheaper model won. And
• 2:56
  here's the kicker. What's that? If Sonic could
• 2:59
  realistically reach the same quality with one
• 3:01
  or two extra follow -up prompts, and if it still
• 3:04
  cost less overall, that counted in Sonic's favor.
• 3:07
  So... Double the price rarely translates to double
• 3:10
  the real -world value. Precisely. Real AI work
• 3:13
  rarely ends after one perfect prompt, anyway.
• 3:16
  It's really about how much useful output you
• 3:18
  get for your time and money. So we've seen the
• 3:21
  overarching thesis. Let's look at the tests where
• 3:24
  Opus really shined. This is where its expensive
• 3:27
  taste actually justified the price tag. Right.
• 3:30
  Test 2 is a perfect example of this. They asked
• 3:33
  for a motion design showreel for a software product.
• 3:36
  Yeah, which gave both models massive creative
• 3:38
  freedom. They didn't just have to write a script.
• 3:41
  They had to research the brand. They had to decide
• 3:44
  how the finished piece should actually feel to
• 3:46
  the viewer. And Sonnet did much better than expected
• 3:49
  here, honestly. It created solid product UI demonstrations.
• 3:53
  It even added a decent AI -generated voiceover.
• 3:57
  And it built this incredibly cool visual transition.
• 4:00
  Oh yeah. where individual pixels form the company
• 4:03
  logo. Right. And that's impressive, technically.
• 4:06
  But Opus just pushed the creative side much further.
• 4:10
  It really did. It understood the assignment on
• 4:13
  a deeper level. It's about creative taste. It's
• 4:16
  like asking someone to paint a masterpiece versus
• 4:19
  painting by numbers. Opus paints the masterpiece.
• 4:22
  That's a perfect way to look at it. Its animation
• 4:25
  choices simply felt more polished. The copy fit
• 4:28
  the brand's tone far better. The rhythm and sound
• 4:31
  design worked more naturally together. It created
• 4:34
  an actual narrative arc. Yes. It even went out,
• 4:37
  found a real quote on the company website, and
• 4:39
  worked that quote seamlessly into the showreel.
• 4:42
  Sonnet cost about $14 here. Opus reached over
• 4:45
  $20. And normally, a $6 gap on a single run gives
• 4:49
  Sonnet a strong advantage. But you gladly pay
• 4:52
  more for Opus here. Exactly. The improvement
• 4:54
  came from pure taste. It came from subjective,
• 4:58
  nuanced, creative decisions that are incredibly
• 5:00
  hard to describe in a prompt. Test 3 was very
• 5:03
  similar. A deliberately vague 90 -day growth
• 5:05
  plan. Right. The goal was simply reaching 1 ,000
• 5:09
  podcast subscribers. Both models initially returned
• 5:12
  standard markdown text. So they both received
• 5:14
  the exact same follow -up prompt. The prompt
• 5:16
  just said, make it more visual and easier to
• 5:19
  read. And this exposed a massive difference in
• 5:22
  judgment. Sonnet did exactly what was asked.
• 5:25
  It created a usable visual plan. But the resulting
• 5:28
  page felt entirely crowded. There was way too
• 5:30
  much noise. You had subscriber targets, promotional
• 5:33
  tactics, traffic sources. Everything was competing
• 5:36
  for your attention on the screen at once. Opus,
• 5:38
  on the other hand, made much stronger planning
• 5:40
  decisions independently. It chose a very clean,
• 5:43
  structured Q4 timeline. It organized the strategy
• 5:46
  into clear month -by -month phases. It even added
• 5:49
  individual actionable tasks you could actually
• 5:51
  check off. It thought about how a human would
• 5:54
  use the document. Sonnet finished in 14 minutes
• 5:57
  for under $5. Opus took 20 minutes and cost over
• 6:00
  $9. But the prompt barely explained the final
• 6:03
  result. You were essentially asking the AI to
• 6:05
  help create the specification itself. And that's
• 6:08
  where the value is. When you don't know exactly
• 6:10
  what the output should look like, you're paying
• 6:12
  for the model's judgment. Exactly. Is taste or
• 6:15
  judgment something we can eventually just engineer
• 6:17
  into a sonnet prompt? I mean, if we just write
• 6:20
  a better prompt, can we save the money? Not without
• 6:23
  massive frustration. You can tell sonnet to change
• 6:26
  a specific font or move a box. Sure. But improving
• 6:29
  the entire rhythm, pacing, and subjective feel
• 6:32
  of a project takes way too many rounds of revision.
• 6:35
  You'd spend hours nudging it in the right direction.
• 6:38
  Got it. Creative intuition takes way too many
• 6:40
  prompts to fix manually. Exactly. You lose the
• 6:42
  cost savings immediately and lost time and patience.
• 6:45
  So, we've seen Opus act as a creative director,
• 6:48
  pulling these brilliant subjective choices out
• 6:50
  of thin air. But creativity is only half of the
• 6:53
  modern workflow. Right. What happens when the
• 6:56
  task is completely objective? When we just need
• 6:59
  the AI to follow strict instructions without
• 7:01
  hallucinating. Let's transition to the tests
• 7:04
  where Sonnet proved a major point. That a slightly
• 7:07
  flawed first draft is better than an expensive
• 7:10
  perfect one. Test 1 was building an immersive
• 7:13
  landing page. It was for a specific drink brand.
• 7:17
  Sonnet made an incredibly strong first impression
• 7:19
  here. It built layered typography in the background.
• 7:22
  It created product flavors you could switch between.
• 7:24
  It had scroll -driven animations. But there was
• 7:27
  one glaring problem. The hero section at the
• 7:29
  top of the page had an interactive 3D can. But
• 7:33
  the can itself was entirely invented by the AI.
• 7:36
  Yeah, it looked like a generic rendered approximation.
• 7:39
  was not the real branded product at all. Opus
• 7:42
  handled that specific part much better. It used
• 7:45
  the real brand imagery. It built a much more
• 7:47
  coherent story connecting the visuals to the
• 7:50
  actual product. The overall experience felt grounded,
• 7:54
  even if the hero section lacked the flashy visual
• 7:56
  impact that Sonnet had. But you have to look
• 7:58
  at the time difference. Sonnet took 28 minutes.
• 8:01
  Opus took 52 minutes. Opus also cost roughly
• 8:04
  twice as much. It landed around $13 for the run.
• 8:08
  And this brings us to the power of the cheap
• 8:10
  correction. We really need to explain why this
• 8:12
  matters. Sonnet's biggest weakness, that fake
• 8:15
  3D can, was easily fixable. You just use one
• 8:19
  simple follow -up prompt. Use the real product
• 8:21
  images instead of the 3D can. And because the
• 8:24
  context window is already built, that's a very
• 8:26
  cheap correction. It's much better than paying
• 8:28
  for another 24 minutes of heavy compute with
• 8:31
  Opus. Tests 5 and 6 really hammered this point
• 8:34
  home, too. Test 5 was an HTML explainer. Right.
• 8:38
  The prompt asked the models to explain what local
• 8:40
  AI models a specific PC could run. Opens explained
• 8:43
  the core technical idea much more clearly. It
• 8:46
  correctly noted that the model needs to physically
• 8:48
  fit inside the available graphics memory. It
• 8:50
  connected model size and GPU memory perfectly.
• 8:54
  Sonnet, however, was more visual. It used simple
• 8:56
  graphics. But the underlying reasoning was less
• 8:58
  clearly explained. The runs took roughly the
• 9:01
  same time, but Sonnet cost about half as much.
• 9:03
  Again, the missing explanation is so easy to
• 9:06
  fix. One more short prompt explicitly asking
• 9:08
  Sonnet to explain GPU memory closes the gap entirely.
• 9:12
  Test 6 was fascinating. An interactive AI news
• 9:15
  dashboard. They asked the models to collect current
• 9:17
  AI news from X and display it. Sonnet built a
• 9:20
  solid working product. It followed the spec.
• 9:23
  It had heat scores for the news, sorting and
• 9:25
  filtering, a calendar for upcoming events. But
• 9:27
  Opus added all this unprompted polish. It added
• 9:30
  post per hour charts. It created clearer trend
• 9:33
  signals. It added tags like rumor and shipped.
• 9:37
  It even built a progress indicator while the
• 9:38
  data collected. Visually, as a user, you would
• 9:42
  definitely choose Opus. But Opus took seven minutes
• 9:45
  longer. And it cost roughly six dollars more.
• 9:49
  I have to ask, if Opus is proactively adding
• 9:52
  cool features like progress bars, aren't we basically
• 9:55
  punishing it for overachieving? It definitely
• 9:58
  feels like it. I mean, extra initiative is great,
• 10:01
  but we have to look at the mechanics of real
• 10:03
  -world workflows. Right. It's financially useless
• 10:06
  if you already know exactly what features you
• 10:08
  want. You could just put, add a progress bar
• 10:11
  in the initial prompt for Sonnet and get it for
• 10:13
  half the price. Right. Unprompted features aren't
• 10:16
  valuable if you never actually needed them. Precisely.
• 10:20
  Sonnet closely follows the requested scope. Opus
• 10:23
  adds useful ideas you never explicitly requested,
• 10:26
  which is great for brainstorming, but bad for
• 10:29
  strict budgets. Up until now, we've assumed Opus
• 10:32
  is always slower and more expensive. Now we're
• 10:35
  going to look at the data that shatters that
• 10:36
  assumption entirely. Test 4 was building an investor
• 10:40
  pitch deck and a complex Excel model. They used
• 10:43
  a large set of mock company data. The presentation
• 10:46
  results were surprisingly close. Both decks followed
• 10:49
  a similar logical structure. Sections for the
• 10:52
  problem, product, customers, traction. Neither
• 10:55
  version had a massive design advantage over the
• 10:57
  other. But the spreadsheets were where the heavy
• 11:00
  lifting happened. Both were very strong. Both
• 11:03
  models used live formulas throughout the workbook.
• 11:05
  That matters immensely for financial modeling.
• 11:08
  Live formulas being spreadsheet calculations
• 11:10
  that update automatically when you change one
• 11:12
  input. Right. A workbook full of hard -coded
• 11:15
  numbers is incredibly painful to maintain. If
• 11:18
  you change a revenue assumption, you want the
• 11:20
  bottom line to update instantly. Sonnet went
• 11:22
  a little too far with the sheer number of tabs
• 11:25
  it created. But the raw quality of the financial
• 11:27
  logic was essentially tied. Then the usage numbers
• 11:30
  settled it completely. Opus actually finished
• 11:33
  faster and was cheaper on this specific run.
• 11:35
  Yeah, when two outputs are this functionally
• 11:37
  close, there is very little reason to pay more
• 11:40
  or wait longer. But how did Opus end up cheaper?
• 11:43
  We'll get to that in a second because test seven
• 11:46
  adds another piece to the puzzle. Test seven
• 11:48
  was building a YouTube resource guide, but they
• 11:51
  used a predefined custom skill. A custom skill
• 11:54
  is just a shaved set of specific instructions
• 11:56
  you reuse for common tasks. Both models received
• 12:00
  a video transcript. They had to turn it into
• 12:02
  a highly structured guide. This produced the
• 12:05
  closest outputs in the entire comparison. They
• 12:07
  had nearly the exact same structure, similar
• 12:09
  length, comparable detail. The custom skill rigidly
• 12:12
  defined exactly what a good guide contained.
• 12:15
  Opus had almost no room to flex its creative
• 12:18
  muscles or add extra value. Because of that rigid
• 12:20
  structure, Sonnet finished about twice as fast,
• 12:23
  and it saved roughly $1 .40 on the run. That
• 12:26
  sounds small, just $1 .40, but think about the
• 12:29
  mechanics of enterprise AI. Across hundreds of
• 12:33
  repeated automated workflows every day, that
• 12:35
  difference adds up incredibly fast. Let's look
• 12:38
  at the aggregate numbers across all seven tests.
• 12:41
  This is the real shocker, and it explains the
• 12:43
  pricing mystery we saw in the Excel test. Opus
• 12:45
  spent 46 more minutes working across these tests.
• 12:48
  It processed way more data. Whoa. Imagine dumping
• 12:53
  38 million more input tokens into a model and
• 12:57
  the final bill is only 16 bucks higher. I know,
• 12:59
  it sounds mathematically impossible. Especially
• 13:02
  with Opus charging double per token. But it works
• 13:05
  because Opus actually used far fewer output tokens
• 13:08
  overall. How does that happen mechanically? I
• 13:12
  mean, if it's doing more complex work, shouldn't
• 13:14
  it be generating more text and code? Well, it
• 13:17
  comes down to a concept called convergence. Convergence
• 13:20
  is how quickly and directly a model arrives at
• 13:23
  the correct answer. Okay, so a cheaper model
• 13:26
  might ramble. It might write inefficient, verbose
• 13:29
  code. Or it might require five back -and -forth
• 13:32
  correction prompts, each one churning out more
• 13:34
  output tokens. Well, the smarter model just hits
• 13:38
  the bullseye immediately. Exactly. Opus processes
• 13:41
  a massive amount of cheap input context. It synthesizes
• 13:45
  it perfectly and outputs a concise, highly accurate
• 13:48
  result. Oh, wow. So despite Opus having roughly
• 13:51
  double the per -token price, the total cost difference
• 13:54
  was incredibly small because it didn't waste
• 13:56
  expensive output tokens on mistakes. Does this
• 13:59
  mean purely counting tokens is a misleading way
• 14:01
  for us to estimate our project costs? Absolutely.
• 14:05
  Because runtime, output token efficiency, and
• 14:08
  how fast a model converges on the right answer
• 14:10
  all wildly swing the final bill. You can't just
• 14:13
  look at the pricing page. Yeah, runtime and output
• 14:15
  efficiency matter just as much as token price.
• 14:18
  You have to look at the whole picture, not just
• 14:20
  the sticker price. We'll be right back after
• 14:22
  a quick word from our sponsors. Welcome back.
• 14:27
  Let's synthesize these results and bring it all
• 14:30
  together. Sonnet ultimately secured a four to
• 14:33
  three victory across the seven tests. It won
• 14:35
  the landing page. It won the HTML explainer.
• 14:38
  It won the news dashboard. It won the resource
• 14:40
  guide. And Opus won the motion design. It won
• 14:43
  the 90 -day plan. It won the pitch deck. And
• 14:47
  Excel model. The ultimate takeaway here is profound
• 14:49
  for anyone building with these tools. Your prompt
• 14:52
  matters almost as much as your model choice.
• 14:55
  If Sonnet gives you a basic flat result and Opus
• 14:58
  gives you incredible polish, You need to look
• 15:01
  closely at why. Yeah. Did Opus actually solve
• 15:04
  a complex reasoning problem that Sonic could
• 15:06
  not? Or did Opus simply make aesthetic and structural
• 15:10
  decisions you never bothered to include in the
• 15:12
  prompt? If you know you want specific trend tags
• 15:15
  in your dashboard, if you want a specific layout
• 15:17
  or real brand images on your website, put them
• 15:20
  in the prompt. Right. Don't pay Opus to guess
• 15:23
  what you want. Or save them inside a reusable
• 15:26
  custom skill. The more clearly you define good,
• 15:30
  the more competitive the cheaper model becomes.
• 15:32
  It scales beautifully. Save Opus for when you
• 15:35
  truly need a thought partner. When you're staring
• 15:38
  at a blank page and you need the AI to figure
• 15:40
  out the path with you. Start with Sonnet for
• 15:42
  repeatable work. Clear specs, established skills.
• 15:46
  We highly encourage you to test both models yourself.
• 15:49
  Run them in parallel on your own recurring workflows.
• 15:52
  Track the real dollars. Track the quality. Understand
• 15:56
  how convergence affects your specific prompts,
• 15:59
  then reuse the winner next time. If cheaper models
• 16:02
  are getting this incredibly good at following
• 16:04
  strict templates and objective tasks, will the
• 16:07
  premium AI of the near future purely be a market
• 16:10
  for taste, intuition, and artistic judgment?
• 16:14
  Something to think about next time you hit generate.