Latest / AI Fire Daily / #546 Neil: GPT 5.6 Sol Review That Makes Terra And Luna Look Weak
Transcript
- 0:00You know, we spend so much time looking at AI
- 0:02benchmark scores. Yeah, we really do. We stare
- 0:05at these massive, complex leaderboards. We compare
- 0:08the abstract math scores. But we often forget
- 0:11to ask the only question that actually matters,
- 0:14two -sex silence. What happens when you finally
- 0:16put it to work? That is the real test, honestly.
- 0:19The real world is remarkably messy. Right. Benchmarks
- 0:23are just sterile laboratory environments. They
- 0:26measure raw potential, but they never actually
- 0:28measure execution. Execution is our mission today.
- 0:31Welcome to the deep dive. We are putting a highly
- 0:34anticipated model through a rigorous five -part
- 0:37gauntlet. We really want to see if it survives
- 0:39the mess. Today we're looking closely at GPT
- 0:425 .6 Sol. That's the flagship model of the heavily
- 0:45discussed Fable 5 generation. Yeah, and Fable
- 0:475 is getting serious attention in the industry
- 0:49right now. It really is. It can navigate ambiguous
- 0:52multi -step reasoning without requiring a human
- 0:56safety net. We're moving completely past the
- 0:58theoretical today. We're going to examine five
- 1:00incredibly specific real -world use cases. Passive
- 1:03spectrum of cognitive capability. Exactly. We'll
- 1:06explore HTML design, autonomous sales pipelines,
- 1:10and blank page website creation. We will also
- 1:13tackle the absolute geometric nightmare of fixing
- 1:16PDFs. And finally, we'll look at managing a multi
- 1:20-model workflows. We are testing visual design,
- 1:23automated research, rigid documentation, and
- 1:27pure logic. But before we jump into that first
- 1:29visual test, let's establish the baseline context.
- 1:32Right, the pricing tiers. Yeah, GPT 5 .6 actually
- 1:35comes in three distinct tiers. First, you have
- 1:38Luna. That's the lower cost option built for
- 1:41lighter text tasks. It runs at about 29 cents
- 1:44per run. Luna is great for generating a quick,
- 1:46dirty draft. Right, and then you step up to Terra.
- 1:49That's the middle option. Which costs 37 cents.
- 1:51Yeah, and it provides much more structural stability.
- 1:53And then we finally reach Sol. This is the strongest
- 1:56tier available. It's heavily optimized for deep
- 1:58reasoning. And it runs at 90 cents per task.
- 2:00So it's roughly three times the price of the
- 2:02Luna tier. This central question of this deep
- 2:05dive is whether that premium is actually justified.
- 2:08Exactly. Does paying 90 cents upfront save you
- 2:12$10 of human labor later? Let's find out. We're
- 2:15starting with a deeply foundational task. HTML
- 2:17generation. Right. That spatial logic we see
- 2:20every day in polished visual products. The first
- 2:23test they ran was generating an HTML presentation
- 2:25slide. And the instructions were incredibly strict.
- 2:28Very strict. It had to create exactly two presentation
- 2:31pages. It needed a strict 16 by 9 widescreen
- 2:35layout. It had to follow the provided brand colors
- 2:38exactly. Including a very specific gradient.
- 2:40Yeah. It also required embedded CSS and JavaScript,
- 2:44embedded CSS styling rules written directly inside
- 2:47the webpage file. The layout had to demonstrate
- 2:50a clear visual hierarchy and the company logo
- 2:53had to sit exactly in the bottom left corner.
- 2:56It's a rigidly defined design test. No external
- 2:59styling libraries were allowed unless absolutely
- 3:01required. And the prompt explicitly demanded
- 3:04perfect factual accuracy, no adding claims, no
- 3:09arbitrary numbers. Absolutely no hallucination
- 3:11was allowed. Right. Hallucination. When the AI
- 3:14confidently makes up fake information. Precisely.
- 3:17So they ran this identical prompt across all
- 3:20three model tiers. They used the exact same project
- 3:23files. Kept it completely objective. Yeah, totally
- 3:26fair comparison. Well, Luna handled the basic
- 3:29text generation perfectly well. It even got the
- 3:32brand colors right. But the final design just
- 3:34looked far too simple. It felt empty, almost
- 3:37like a wireframe sketch. Then they tried Terra.
- 3:41It cost slightly more than Luna. But the visual
- 3:44layout didn't feel clearly better. The spacing
- 3:46between the text blocks actually looked unbalanced.
- 3:49Yeah, it felt clunky and unprofessional. But
- 3:51then came the Sol model. Sol delivered a highly
- 3:54polished, presentation -ready result. It kept
- 3:57the text completely accurate to the source material.
- 4:00It followed the requested color gradient perfectly.
- 4:02It used much cleaner negative space between the
- 4:05elements. The visual hierarchy was immediately
- 4:07obvious to the human eye. The headline stood
- 4:09out beautifully against the background. The call
- 4:11to action was incredibly easy to notice. And
- 4:15the logo was anchored correctly in the bottom
- 4:17left. All of that for 90 cents. You know, I have
- 4:21to make a vulnerable admission here. Oh. I still
- 4:24wrestle with settling for rough AI drafts and
- 4:27wasting hours tweaking them myself. You are definitely
- 4:30not alone in that frustration. I just convince
- 4:32myself it's faster, but it never is. We often
- 4:35accept... a weak first draft because it feels
- 4:38like forward momentum. Right. But then we accidentally
- 4:40become full -time editors. We end up spending
- 4:43more time fixing the AI's structural mistakes
- 4:46than if we just started from scratch. It completely
- 4:49defeats the purpose of the automation. If I have
- 4:51to fix the margin spacing manually, I am not
- 4:54saving time. I am just changing my daily job
- 4:57description. Right. And a rough presentation
- 4:59draft simply creates another editing task. Exactly.
- 5:02Saul gave a surprisingly strong first version.
- 5:04It acts as a competent partner, not a burden.
- 5:08It significantly reduces the manual pixel pushing
- 5:10you need to do later. I hung up on one thing
- 5:13here though. Predicting the next word in a sentence
- 5:15makes intuitive sense to me. Sure. But why is
- 5:18layout so much harder for an AI than writing
- 5:21text? It fundamentally comes down to structural
- 5:23architecture. Language models are built to predict
- 5:27text in a straight linear line. Right. But graphic
- 5:30design... requires deep spatial logic. It requires
- 5:34understanding visual weight and negative space
- 5:37on a broad canvas. It's not just a sequence.
- 5:40No. Instead of stacking words, think of it like
- 5:43an architect working with a physical floor plan.
- 5:46That makes a lot of sense. The model has to calculate
- 5:48the gravity of the page. Exactly. It knows a
- 5:51heavy dark logo in the bottom corner needs breathing
- 5:54room. It needs negative space around it to keep
- 5:57the whole canvas from tipping over visually.
- 5:59It calculates the visual tension between the
- 6:02text blocks and the imagery. So design requires
- 6:05spatial reasoning, not just predicting words.
- 6:07It's a completely different cognitive muscle
- 6:09for the machine, and Sol clearly flexed it well.
- 6:12From spatial layout, we move straight into chaining
- 6:15data logic together. This is a big one. We're
- 6:17talking about connecting deep research, complex
- 6:20scoring, and targeted outreach in one continuous
- 6:23flow. This brings us directly to the second use
- 6:26case. connecting a platform like Clay to the
- 6:28model. Clay acts as the data extraction engine.
- 6:31Right. The core goal was to build a highly qualified
- 6:34automated sales lead list. The target criteria
- 6:37they set was highly specific. The model had to
- 6:40find U .S. fintech companies. They needed to
- 6:43have Series B funding. Series B funding, the
- 6:45second major investment round to scale business
- 6:47operations. Perfect. They had to have exactly
- 6:5050 to 200 employees. They also needed to show
- 6:53a clearly defined sales team. And display very
- 6:57recent signs of corporate growth. Things like
- 6:59recent venture funding, aggressive new hiring,
- 7:02or territorial expansion. The model had to find
- 7:05the company and confirm those complex details.
- 7:08Then it had to hunt down the specific VP of sales.
- 7:11It had to get a completely verified email address
- 7:14for that exact person. But the autonomous workflow
- 7:16didn't stop there. No, it had to score that specific
- 7:19lead. It assigned a score from 1 to 10. This
- 7:21was based on a very specific ideal customer profile.
- 7:24The ideal customer profile looked for companies
- 7:26handling large amounts of data. Companies that
- 7:29could genuinely benefit from automating their
- 7:31operations. The model had to logically explain
- 7:34that score in one short sentence. And then it
- 7:36had to write a custom first touch email. That
- 7:39email had to reference those exact recent growth
- 7:41signals it just found. Any company failing that
- 7:44strict filter had to be removed entirely. The
- 7:47final results were genuinely impressive. Sol
- 7:50successfully built the entire complex lead table.
- 7:53It returned five specific accounts with highly
- 7:56qualified scores from 8 to 10. It returned the
- 7:59complete actionable package. It grabbed the company
- 8:02name, employee count, and the latest funding
- 8:04rounds. It even found the contact person and
- 8:07verified their direct email. The outreach emails
- 8:10were highly customized and conversational. They
- 8:12referenced specific growth signals and likely
- 8:14sales needs for each unique company. It felt
- 8:17incredibly natural. Two secs silence. Whoa. Imagine
- 8:20scaling this to analyze thousands of companies
- 8:22in seconds. The sheer velocity of the workflow
- 8:25is staggering. It really is. What used to take
- 8:27a dedicated sales research team a full exhausting
- 8:30week is now a single automated prompt. It processes
- 8:34the entire pipeline instantly. But I want to
- 8:37pause and challenge this a bit. With all this
- 8:39automated research, How much do we actually trust
- 8:42the verified emails and funding data? That is
- 8:45the most critical question of autonomous workflows.
- 8:49AI can still pull heavily outdated data from
- 8:52stale databases. Right. People change jobs frequently.
- 8:56Funding rounds get misreported in the press.
- 8:59An experienced human must do a final spot check.
- 9:02You cannot blindly trust the machine. Trust the
- 9:05initial filter, but verify the final facts before
- 9:08sending. Exactly. You use the AI as an incredibly
- 9:11fast intelligence analyst. It sifts through the
- 9:14massive mountains of irrelevant data. But you
- 9:17still must inspect the final target list yourself
- 9:19before launching a real campaign. We've seen
- 9:21Sol follow incredibly strict visual and data
- 9:24instructions. Now what happens when we give it
- 9:27a completely blank page and ask it to make creative
- 9:29decisions? It takes us right into the third use
- 9:31case. building a complete one -page website from
- 9:34scratch. The prompt simply asked for a web design
- 9:37service aimed at small businesses. The main goal
- 9:39was to encourage visitors to book a free consultation.
- 9:42The target audience was small business owners
- 9:44who completely lack an in -house developer. The
- 9:47structural requirements were laid out clearly.
- 9:49It needed a strong hero section. A section explaining
- 9:52common small business problems with outdated
- 9:55websites. A dedicated services section. A simple
- 9:58process section explaining how the project actually
- 10:00works. It also needed a portfolio section with
- 10:03three sample project cards. A social proof section
- 10:06focused heavily on practical benefits. Clear
- 10:09pricing options. And a final, highly visible
- 10:12call to action. The design direction simply requested
- 10:15a clean, modern style. An open layout that is
- 10:19exceptionally easy to scan. and bento -style
- 10:22content blocks, bento -style blocks, neat grid
- 10:26-like square panels seen on modern dashboards.
- 10:28It also needed smooth scrolling and responsive
- 10:30design for mobile screens. Now, a blank page
- 10:33is completely terrifying for most generative
- 10:35models. Oh, absolutely. But Sol made excellent
- 10:38independent design decisions. It chose the typography
- 10:41combinations entirely on its own. It dynamically
- 10:44chose the bento card shapes. It decided on the
- 10:47internal padding, margin spacing, and color palettes.
- 10:50It created a highly visible contrasting consultation
- 10:54button. It even coded a browser -style visual
- 10:57graphic that made the digital service feel much
- 10:59more real. It turned the main customer problems
- 11:02into four easy -to -scan, beautiful cards. And
- 11:05it did all this without needing step -by -step
- 11:07design handholding. Right. It created a serious
- 11:10structural first draft, not just a rough visual
- 11:14mockup. The page obviously still needs real portfolio
- 11:17examples inserted later. Of course. It needs
- 11:20your final pricing numbers and heavily tested
- 11:22booking links. But the foundational web design
- 11:24is completely solid. It bypassed the empty screen
- 11:27problem completely. I'm still trying to wrap
- 11:29my head around this. How does the model avoid
- 11:31blank page paralysis when it isn't given exact
- 11:35hex codes or fonts? It uses deep underlying structural
- 11:38logic. Sol analyzes millions of modern websites
- 11:42during its initial training. It implicitly understands
- 11:45modern industry conventions. It knows how a bento
- 11:48layout should function visually and mathematically.
- 11:50So it just fills in the gaps. Yes. It fills in
- 11:53the creative gaps using established aesthetic
- 11:56patterns. It leans on industry patterns to make
- 11:59those unprompted creative choices. Precisely.
- 12:02It understands the underlying grammar of modern
- 12:05web design. Right. It knows that a heavy sans
- 12:08serif header pairs perfectly with 24 pixels of
- 12:11margin space. It knows what looks good because
- 12:15it intimately understands the statistical patterns
- 12:18of good design. Sol can build a beautiful website
- 12:20from a blank canvas. But can it handle the rigid,
- 12:24universally frustrating world of government PDFs?
- 12:27This is where automated data entry usually breaks
- 12:30down completely. Yeah. Manual government forms
- 12:32create endless frustrating systemic errors. A
- 12:35test here was filling out a completely blank
- 12:37W9 form. They used simple dummy data for the
- 12:40test. Name, business name, federal tax classification,
- 12:43and the physical address. They also included
- 12:45a dummy taxpayer ID number. A TIN. The model
- 12:49had to place every single text value exactly
- 12:51inside the correct visual field. It had to intentionally
- 12:54leave the signature and date fields entirely
- 12:57blank. Then it had to check that every value
- 12:59was perfectly inside the correct box. It had
- 13:02to autonomously fix any misaligned text. And
- 13:05finally, save it as a specific PDF file. PDF
- 13:09forms are notoriously easy for machines to completely
- 13:12misread. Very true. A business name lands awkwardly
- 13:16in the wrong field. Important numbers shift slightly
- 13:19out of place. One small geometric mistake makes
- 13:22the final document look incredibly unprofessional.
- 13:25And that problem scales terribly when you repeat
- 13:27the task across thousands of files. The results
- 13:31here were fascinating to look at. Sol correctly
- 13:33identified the official W9 form. It placed almost
- 13:36everything perfectly right. The name and address
- 13:38were perfectly situated in their boxes. However...
- 13:41There was a tiny, frustrating glitch. Just one.
- 13:44Yeah. One single part of the taxpayer ID number
- 13:46didn't sit cleanly inside its tiny box. It was
- 13:49slightly misaligned geometrically, bleeding over
- 13:51the line. But here is the truly interesting part.
- 13:54After receiving automated feedback, Sol readjusted
- 13:56the number's position. It successfully saved
- 13:59a much cleaner, perfectly aligned version. It
- 14:01actively fixed its own spatial error. The final
- 14:04form looked highly polished and professional.
- 14:06But the tester rightly noted, it still required
- 14:09that final human review before actual production
- 14:11use. Always. Why does a highly advanced model
- 14:14stumble over aligning a few numbers in a PDF?
- 14:18Because PDFs are not fluid dynamic text documents.
- 14:21They act more like rigid visual coordinate maps.
- 14:23Right, they are static. Placing text exactly
- 14:26inside a tiny box is a complex geometric challenge.
- 14:30It is not about reading comprehension. It is
- 14:32about plotting coordinates precisely on a strict
- 14:35X and Y grid. PDFs are visual maps, making text
- 14:38placement a geometric nightmare. Right. And legacy
- 14:41government forms were never designed for modern
- 14:44machine readability. No, they were designed for
- 14:46typewriters. Exactly. They are essentially digital
- 14:48photographs of paper documents. The AI has to
- 14:52mathematically guess where the invisible boxes
- 14:54actually begin and end. We have seen Saul work
- 14:57completely alone across visual design, sales
- 14:59research, and complex forms. Now let's look at
- 15:02the ultimate test of autonomous logic. Saul managing
- 15:06and critically reviewing another AI model's work.
- 15:08This takes us to the final, most complex test,
- 15:12a multi -model automated workflow. Giving each
- 15:15model a completely separate, distinct role to
- 15:18prevent systemic bias. Model 1 steps in first.
- 15:21It reads the complex project file. It reads the
- 15:24historical documentation and the test suites.
- 15:26It writes a detailed problem summary. It creates
- 15:29a technical plan to fix the software issue. It
- 15:32explicitly lists every underlying assumption
- 15:34used in the plan. Crucially, Model 1 is explicitly
- 15:37instructed not to change any files yet. Right.
- 15:39It simply creates the architectural blueprint.
- 15:42It proposes the technical changes for review.
- 15:44Then GPT -5 .6, Sol steps into the workflow as
- 15:48the senior reviewer. It independently compares
- 15:50Model 1's technical plan with the actual codebase,
- 15:52the tests, the documentation, and the current
- 15:55database. Sol's job is to ruthlessly hunt for
- 15:58logical bugs. It must catch missing steps and
- 16:00identify outdated instructions. It must clearly
- 16:02explain why each identified issue actually matters.
- 16:05It marks every finding as critical, important,
- 16:08or minor. And it has the absolute power to block
- 16:11the approval entirely. So what actually happened
- 16:13when they ran this workflow? Sol reviewed six
- 16:15dense project files. And it marked the entire
- 16:19workflow as blocked. It found highly critical
- 16:22bugs in the June 2026 financial data. Model 1
- 16:27had completely hallucinated some math and severely
- 16:31miscalculated the company's revenue. Model 1
- 16:33completely miscalculated the monthly recurring
- 16:35revenue. MR Predictable recurring monthly revenue
- 16:39from active customer subscriptions. Model 1 estimated
- 16:42the MRR at $4 ,450. So immediately caught the
- 16:47glaring error. It mathematically recalculated
- 16:50the correct MRR as just $600. It also forcefully
- 16:53corrected the average revenue per active customer.
- 16:56It brought the calculated average revenue down
- 16:58from $890 all the way to $120. How on earth did
- 17:03it find that massive discrepancy? Sol caught
- 17:06a fundamentally bad cancellation date rule in
- 17:08the code. Wow. Model 1 misunderstood when a customer
- 17:11technically churns based on billing cycles. It
- 17:14oversubtracted annual revenues incorrectly. The
- 17:16billing rule didn't match across the actual code,
- 17:19the tests, the RAIDME file, and the project documentation.
- 17:23Sol also found that annual subscription plans
- 17:25were incorrectly added at their full yearly price
- 17:28instead of being properly prorated. And the churn
- 17:31metric used the completely wrong active customer
- 17:35count. It rigorously checked Model 1's assumptions
- 17:37instead of blindly accepting the technical plan.
- 17:40It firmly rejected the incorrect cancellation
- 17:43rule. The workflow stayed safely blocked until
- 17:46everything followed the exact same mathematical
- 17:48rules. It successfully stopped a completely disastrous
- 17:51financial deployment. It caught both subtle coding
- 17:54mistakes and glaring project conflicts before
- 17:57any development continued. Yeah. I do have to
- 17:59ask though, why use two different models at all?
- 18:02It's a fair question. Why not just have Sol plan
- 18:04the update and review the code simultaneously.
- 18:07Because language models inherently develop massive
- 18:09logical blind spots. Separating the creation
- 18:12from the critique prevents the AI from being
- 18:15blind to its own logical errors. A single model
- 18:19will constantly justify its own mistakes because
- 18:21it strictly follows a single unbroken thread
- 18:24of logic. Splitting roles stops the AI from grading
- 18:27its own homework. Precisely. It creates a necessary
- 18:31adversarial check in the system. It forces the
- 18:34creative model to rigorously prove its logic
- 18:37against an independent, unforgiving auditor.
- 18:40We are going to take a quick break and then we
- 18:42will synthesize what all this means for you,
- 18:44sponsor poor. And we are back. Let's pull back
- 18:48and look at the big picture here. We have covered
- 18:50a massive amount of technical ground today. When
- 18:53we synthesize all these diverse findings, the
- 18:55core takeaway is actually about smart resource
- 18:57allocation and trust. GPT 5 .6 all costs roughly
- 19:01three times as much as Luna. 90 cents versus
- 19:0429 cents. But that 90 cents is a direct investment
- 19:07in systemic autonomy. For simple single -step
- 19:10tasks, you should absolutely use the cheaper
- 19:12models. Luna is perfectly fine for generating
- 19:15a quick text draft. But for multi -step workflows,
- 19:18the calculus changes dramatically. When you need
- 19:20strong historical context. When you need complex
- 19:22autonomous decision -making. When you rely heavily
- 19:25on deep structural logic. Paying a few extra
- 19:28cents for Sol makes perfect economic sense. It
- 19:31drastically reduces the manual human labor required
- 19:34to fix frustrating mistakes later. It fundamentally
- 19:37changes our psychological relationship with the
- 19:39machine. It shifts our professional role from
- 19:41tired editors to strategic directors. We finally
- 19:44stop fixing margin spacing and frustrating typos.
- 19:47We start guiding the larger structural outcomes
- 19:50and business strategy. Exactly. Before we wrap
- 19:53up this deep dive, I want to leave you with a
- 19:55lingering thought. We saw Saul independently
- 19:58review another model's code today. we saw it
- 20:02seamlessly recalculate complex recurring revenue
- 20:05with absolute mathematical precision. We saw
- 20:09it outright block a bad workflow that would have
- 20:12severely corrupted financial data. If Sol is
- 20:15already capable of doing all of this autonomously,
- 20:17if it can catch subtle logic errors that most
- 20:20junior developers would completely miss, how
- 20:23far are we from a future where AI handles the
- 20:26entire project lifecycle? A future where human
- 20:29beings are just the final signature on the approval
- 20:31form. It is a profound, slightly unsettling shift
- 20:35in how we work. I want you to actively think
- 20:38about your own daily workflows this week. Look
- 20:40at the rigid, highly manual tasks you are still
- 20:44doing right now. Which of those could easily
- 20:46be handed off to a multi -model autonomous review
- 20:49tomorrow? The foundational tools are already
- 20:51sitting right in front of us. It is just a matter
- 20:53of how we purposefully choose to apply them to
- 20:55our own friction points. Thank you for joining
- 20:58us on this deep dive. Stay curious, keep exploring
- 21:01your own workflows, and we will catch you next
- 21:03time.