Latest / AI Fire Daily / 🎙️ EP 263: Your Home is Now an AI Node & The 12M-Token "RAG Killer"
Transcript
- 0:00The next massive AI data center isn't being built
- 0:02in a remote desert. It's being installed right
- 0:05next to your AC unit, and it might just pay your
- 0:07utility bill. It is an absolutely wild concept
- 0:10to wrap your head around. Welcome back to the
- 0:12Deep Dive. I'm really glad you're joining us
- 0:14today. We have some truly fascinating ground
- 0:17to cover. The physical landscape of artificial
- 0:19intelligence is fundamentally shifting. We're
- 0:22looking at the hard limits of global computation,
- 0:25and we're seeing how engineers are quietly bypassing
- 0:28them. Yeah. Today we are exploring the distributed
- 0:31frontier of AI. We'll look at how supercomputers
- 0:34are moving into residential homes. We'll track
- 0:36the explosive agentic shift happening right now.
- 0:39This shift spans from simple Excel sheets to
- 0:41orbital satellites. And we'll unpack a massive
- 0:44new software breakthrough. It allows models to
- 0:47swallow 12 million words in a single pass. Let's
- 0:50start by looking at the physical infrastructure.
- 0:52We all know the AI boom is draining the power
- 0:54grid. Training these frontier models takes massive
- 0:57amounts of raw electricity. But building bigger
- 1:00power plants takes many, many years. So what
- 1:02if we just use the power capacity already wired
- 1:04into our homes? Right. And that is the exact
- 1:06premise of a startup called Span. They just announced
- 1:10a massive new infrastructure partnership. They're
- 1:13teaming up with NVIDIA and the home builder Polta
- 1:16Group. They're launching something called the
- 1:17XFRA Distributed Data Center. Yeah, the XFRA
- 1:21Distributed Data Center knows. I was reviewing
- 1:23the hardware specs earlier today. Each individual
- 1:26node is an absolute beast of a machine. They
- 1:28pack 16 NVIDIA RTX PRO 6000 Blackwell GPUs. Right,
- 1:33and they use the liquid -cooled server edition
- 1:35specifically. Exactly. They also include top
- 1:38-tier AMD EPYC processors. And they feature 3
- 1:42terabytes of RAM per node. beat. That is a staggering
- 1:46amount of local computing power. Why do we need
- 1:48three terabytes of memory sitting in a backyard?
- 1:51Well, it really comes down to loading massive
- 1:53parameter models natively. You need vast memory
- 1:56to hold these giant neural networks. So we're
- 1:58putting supercomputer level hardware directly
- 2:00into suburban neighborhoods. But the energy distribution
- 2:03is the real puzzle here. The copper wires in
- 2:06our streets have hard physical limits. How does
- 2:09a normal house power a massive supercomputer?
- 2:12This is where the smart panel technology comes
- 2:14in. The U .S. electrical grid was designed for
- 2:17peak theoretical loads. Most homes only use about
- 2:2140 % of their electrical capacity. Wow. Yeah,
- 2:24we have all this unused power just sitting there.
- 2:26SPAN's smart panel identifies that exact spare
- 2:29electricity dynamically. It monitors the home's
- 2:32power draw in milliseconds? Exactly. The panel
- 2:35detects when your oven or clothes dryer turns
- 2:38off. It then funnels that exact spare electrical
- 2:41capacity into the XFRA node. The node uses that
- 2:45diverted power to run complex cloud AI tasks.
- 2:48And the homeowner doesn't even notice the power
- 2:50shifting around. They don't notice a single thing
- 2:52changing. It operates entirely in the background
- 2:54of their daily lives. And the trade -off for
- 2:56the homeowner is incredibly compelling. In many
- 2:59of these markets, hosting a node brings massive
- 3:01benefits. You get completely free electricity
- 3:03and free gigabit internet. Span essentially pays
- 3:06your monthly utility bills for you. Yep. They
- 3:08cover your bills in exchange for using your wall
- 3:10space. They just tap into the spare electricity
- 3:12your house wasn't using anyway. It's like Airbnb
- 3:15for your home's unused electrical capacity. That's
- 3:18a perfect way to visualize the mechanism. And
- 3:20they're already rolling this infrastructure out
- 3:22right now. They're deploying them in new construction
- 3:25communities with Poltergroup. Think about the
- 3:28massive advantage in actual deployment speed.
- 3:31Instead of building one giant... 100 megawatt
- 3:34centralized data center, they can just deploy
- 3:368 ,000 of these distributed mini nodes. It is
- 3:39six times faster to build this way. And it comes
- 3:42in at one fifth of the traditional infrastructure
- 3:44cost. Two sec silence. I've been thinking about
- 3:47the broader grid impact here. Is this actually
- 3:49safe and scalable? Or does putting a supercomputer
- 3:52next to my dryer create localized grid failures?
- 3:55It is a completely valid engineering concern
- 3:58to have. But the system actually balances local
- 4:01loads dynamically. Span smart panels act like
- 4:04highly intelligent traffic cops. They ensure
- 4:06the node only draws power when the house absolutely
- 4:09doesn't need it. This prevents any localized
- 4:12transformers from blowing out. So it balances
- 4:14local loads instead of draining the central grid.
- 4:17Precisely. We've been so worried about massive
- 4:19data centers sucking the grid dry. Span is proving
- 4:22we can just use the capacity we've already built.
- 4:24It turns your home from a passive cost center
- 4:27into a revenue generating asset. Beat. So hardware
- 4:31is moving directly into our suburban homes, but
- 4:34the physical distribution of compute is pushing
- 4:36much further out. Hardware is distributing all
- 4:39the way into the vacuum of space, and it's backed
- 4:42by truly astronomical amounts of corporate capital.
- 4:45The financial scale of this shift is honestly
- 4:47hard to comprehend. We're seeing numbers that
- 4:50fundamentally redefine corporate spending. Anthropic
- 4:53just committed to a massive new global infrastructure
- 4:55deal. They're spending $200 billion on Google
- 4:59Cloud and AI chips. $200 billion over just five
- 5:02years? That is roughly $40 billion every single
- 5:05year. Yeah. That deal represents over 40 % of
- 5:08Google Cloud's entire revenue backlog. The global
- 5:11AI infrastructure war is completely exploding
- 5:13right now. They are buying up silicon and energy
- 5:16contracts at an unprecedented pace. But they
- 5:19aren't just looking at traditional cloud servers
- 5:21on Earth. Anthropic just announced an incredible
- 5:24partnership with SpaceX AI. They secured access
- 5:27to the Colossus One massive AI supercomputer.
- 5:30And here's the truly mind -bending part of that
- 5:34specific partnership. Both organizations are
- 5:37officially exploring orbital AI compute in space.
- 5:40We're talking about massive data centers. orbiting
- 5:43the planet. Whoa. Imagine scaling orbital AI
- 5:47compute across thousands of satellites. Right.
- 5:49It completely changes the physical limits of
- 5:51our infrastructure. The cold vacuum of space
- 5:53solves massive thermal cooling problems. And
- 5:56this endless hardware scale is powering a fundamental
- 5:58software shift. AI is no longer just a simple
- 6:01chat bot answering questions. It is evolving
- 6:03into an active, autonomous digital operator.
- 6:06Yeah, the agentic shift is fully underway across
- 6:08the entire industry. I want to unpack what an
- 6:10agent actually is. A chatbot just predicts the
- 6:13next logical word in a sentence. An agent sets
- 6:15goals, uses external tools, and executes actions
- 6:18autonomously. Exactly. OpenAI is reportedly building
- 6:21a dedicated AI phone right now. They expect early
- 6:24hardware production to start by the year 2027.
- 6:26It completes background tasks, understands your
- 6:28goals, and works like a true operator. It is
- 6:31not just a standard mobile device anymore. And
- 6:33we're seeing this deep agentic integration in
- 6:36the workplace, too. Anthropic just launched 10
- 6:38ready to use autonomous agents. They are built
- 6:41specifically for the finance and insurance sectors.
- 6:45These specific agents are doing highly complex
- 6:47professional work. They can screen dense KYC
- 6:49files for banking compliance. They can review
- 6:52massive corporate earnings reports in seconds.
- 6:54Wow. They can even build complex presentation
- 6:57decks completely from scratch. ChatGPT is also
- 7:00moving directly into our daily corporate workflows.
- 7:03It now works natively inside Microsoft Excel
- 7:05and Google Sheets. You can use it to build complex
- 7:08financial formulas automatically. It reads raw
- 7:11data, generates insights, and formats the entire
- 7:13spreadsheet. They're rolling out a free beta
- 7:15for paid users right now. And OpenAI is aggressively
- 7:18pushing this integration into higher education.
- 7:21They just introduced the ChatGPT Futures Class
- 7:23of 2026 program. Right. They gave 26 student
- 7:26builders $10 ,000 grants. And they gave them
- 7:30full, unlimited access to frontier AI models.
- 7:33It is the first generation to start and finish
- 7:36college with GPT. We're studying how students
- 7:39leverage frontier AI throughout their entire
- 7:42degree. We're watching the baseline of human
- 7:45productivity shift in real time. Beat, but this
- 7:49deep integration introduces some very strange
- 7:52new vulnerabilities. We're trusting autonomous
- 7:55agents with highly sensitive personal financial
- 7:57data. The security risks are evolving just as
- 8:00fast as the model capabilities. Researchers just
- 8:03exposed a completely new AI scam trick last week.
- 8:06They found a brilliant way to manipulate agents
- 8:08using hidden instructions. Yeah, the hidden malicious
- 8:11instructions were written entirely in Morse code,
- 8:14just simple dots and dashes buried deeply inside
- 8:16standard text files. And the terrifying part
- 8:18is that some agents actually followed them. The
- 8:21AI recognized the Morse code formatting perfectly.
- 8:23It then bypassed its safety filters and executed
- 8:26the hidden malicious commands. I still wrestle
- 8:28with prompt drift myself, so agents getting hacked
- 8:30by Morse code is terrifying. It sounds like a
- 8:33strange plot from a science fiction novel. But
- 8:35this is the harsh reality of deploying autonomous
- 8:38agents. Large language models process information
- 8:41through mathematical tokenization. They don't
- 8:44read English the way human beings do. Right.
- 8:46They see patterns, and Morse code is just another
- 8:48mathematical pattern. We are exposing ourselves
- 8:51to entirely new cryptographic attack vectors.
- 8:55And the traditional legal system is desperately
- 8:57trying to catch up. Apple just agreed to pay
- 9:00$250 million. They're settling a massive class
- 9:04action lawsuit regarding artificial intelligence
- 9:06right now. The underlying lawsuit was tied to
- 9:09claims about specific iPhone capabilities. Buyers
- 9:12claimed they were misled about Siri's newly advertised
- 9:15AI features. Apple agreed to the massive financial
- 9:18settlement to avoid a lengthy trial. They settled
- 9:22this massive case even though they admitted no
- 9:23official wrongdoing. Two sec silence. With agents
- 9:26acting autonomously in our spreadsheets and phones,
- 9:29who takes the fall when a Morse code hack successfully
- 9:32steals data? That is the defining legal question
- 9:35of this entire new era. The traditional paradigm
- 9:38of software security is shifting rapidly right
- 9:41now. Historically, the end user was heavily responsible
- 9:44for their own data security. But when the AI
- 9:47makes autonomous background decisions, the liability
- 9:50fundamentally moves. The legal burden falls heavily
- 9:53onto the developers building the underlying models.
- 9:57Ah, so security liability shifts entirely from
- 9:59user to the AI developer. Exactly. The tech companies
- 10:02building this intelligence have to legally secure
- 10:04it. They are creating autonomous actors, so they
- 10:07hold the ultimate responsibility. Sponsor. We're
- 10:10back. We've explored supercomputers sitting right
- 10:13next to our air conditioners. We've tracked autonomous
- 10:15agents running our daily financial spreadsheets.
- 10:18Beat. But for these complex agents to work effectively,
- 10:21they need vast memory. They need to instantly
- 10:23recall massive amounts of contextual data. And
- 10:26that brings us to an incredible new software
- 10:28breakthrough. It perfectly matches the explosive
- 10:31scale of the physical hardware we discussed.
- 10:33A company called SubQuadratic just launched a
- 10:37fascinating new AI model. They're calling this
- 10:39powerful new model SubQ. It's being described
- 10:43as a 12 million token memory hack killer. SubQ
- 10:47can cleanly swallow 12 million tokens in a single
- 10:51analytical pass. That is a truly incomprehensible
- 10:54amount of raw contextual information. It's like
- 10:57feeding an entire corporate code repository into
- 11:00one single prompt. Right. And the engineering
- 11:03team behind this model is absolutely world class.
- 11:06They hail from Meta. Google DeepMind and Oxford
- 11:09University. They've built an architecture that
- 11:11fundamentally changes the core processing economics.
- 11:14The massive cost difference is what really stands
- 11:16out to me here. It costs just $8 to run a massive
- 11:19context task on SubQ. That is a staggering price
- 11:23drop for working software developers. If you
- 11:25run that exact same massive task on traditional
- 11:27frontier models, it is wildly expensive and incredibly
- 11:30slow to process. It costs roughly $2 ,600 on
- 11:33those older legacy systems. And SubQ runs 52
- 11:36times faster than the current industry standard.
- 11:39Let's unpack exactly how they achieved this massive
- 11:41leap in efficiency. They use a highly optimized
- 11:44selective attention neural architecture. Traditional
- 11:47models calculate the relationship between every
- 11:50single word in a document. That requires massive
- 11:53computing power as the document gets larger and
- 11:56longer. Their system only looks at the token
- 11:58relationships that actually matter. It drops
- 12:01irrelevant contextual connections instantly.
- 12:04That is how they achieve such incredible, groundbreaking
- 12:07processing speeds. We should define a few important
- 12:10architectural concepts here. Flash attention
- 12:12is a widely used method to speed up AI memory
- 12:15processing. Right. It is a critical optimization
- 12:17technique for modern hardware architectures.
- 12:19And we also need to talk about traditional retrieval
- 12:21systems. The industry has relied heavily on complex
- 12:24RG pipelines for years now. These are systems
- 12:27that fetch outside data to help AI and... or
- 12:30questions. You build a vector database to search
- 12:32for small text fragments. Yeah, those traditional
- 12:35document retrieval systems are incredibly complex
- 12:37and fragile. But SubQ might make those complicated
- 12:40RRAG pipelines a thing of the past. You simply
- 12:44don't need to fetch small chunks of outside data
- 12:47anymore. You just feed the entire massive database
- 12:50directly into the working model. It's like stacking
- 12:53Lego blocks of data, but instead of building
- 12:55piece by piece, you dump the whole bucket and
- 12:58the AI instantly sees the final castle. That
- 13:00visual perfectly captures the leap in processing
- 13:02capability. And the verified benchmark performance
- 13:05of SubQ is simply incredible. Most standard models
- 13:09lose the plot as the input files get bigger.
- 13:12They suffer from the classic needle in a haystack
- 13:15problem. Right. They completely forget important
- 13:17information hidden deeply in the middle of the
- 13:20text. But SubQ hit 92 % recall at 12 million
- 13:23tokens. It remembers almost everything you feed
- 13:26into its working context window. Just for comparative
- 13:29context, we could look at the major industry
- 13:31competitors. Gemini 3 .1 Pro struggled significantly
- 13:35on similar massive multi -needle tests. The exact
- 13:38same degradation is true for OpenAI's GPT -5
- 13:41.4. They simply could not maintain accurate information
- 13:45recall at that massive scale. SubQ is absolutely
- 13:48crushing the established giants on these specific
- 13:51benchmarks. Yeah, they've already launched a
- 13:53specialized developer tool called SubQ Code.
- 13:55It's a command line interface agent built directly
- 13:58for professional software engineers. It can load
- 14:00your entire complex code base in just one single
- 14:03pass. And they are certainly not stopping at
- 14:0612 million tokens. The engineering team is officially
- 14:08targeting a 100 million token context window.
- 14:12They want to hit that massive milestone by the
- 14:14end of 2026. We have seen so -called transformer
- 14:16killers hyped up before. Alternate architectural
- 14:19models like Mamba tried to dethrone the standard
- 14:21transformer architecture. But SubQ genuinely
- 14:24feels like a permanent shift in the landscape.
- 14:26It's launching with a highly robust, production
- 14:29-ready developer API today. If you're a software
- 14:32engineer building agents, this is a massive win.
- 14:34Two sec silence. I have to wonder about the long
- 14:37-term architectural implications here. If an
- 14:41AI can perfectly recall 12 million tokens for
- 14:45eight bucks, does this completely kill the need
- 14:48for complex retrieval systems? It fundamentally
- 14:51rewrites how developers will build AI applications
- 14:53moving forward. You simply won't need clunky
- 14:56vector databases holding tiny fragments of text.
- 14:59You won't need complicated routing logic to find
- 15:02the right source document. You'll just load the
- 15:04entire required operational context directly
- 15:06into the working memory. Right. Massive context
- 15:09windows will just replace complex data retrieval
- 15:12pipelines entirely. It simplifies the entire
- 15:14software development process for engineers globally.
- 15:17Let's step back and look at the larger big picture
- 15:19here. The core physical bottlenecks of artificial
- 15:22intelligence are shattering simultaneously right
- 15:25now. We're seeing hardware limits being bypassed
- 15:27in incredibly creative and distributed ways.
- 15:30We are actively capturing the unused electrical
- 15:32capacity of our suburban homes. We are looking
- 15:34at deploying massive computing nodes in the cold
- 15:37vacuum of space. The physical deployment constraints
- 15:39that held us back are rapidly disappearing. And
- 15:42the software processing limits are evaporating
- 15:45just as fast today. New architectural models
- 15:48like SubQ can process entire repositories of
- 15:50human knowledge in seconds. We are rapidly transitioning
- 15:54away from the passive, simple chatbot era. We're
- 15:57actively unleashing highly autonomous, ubiquitous
- 16:00software agents into the real world. This digital
- 16:03intelligence is moving directly into our Excel
- 16:05sheets and our phones. It is running in your
- 16:08backyard, your devices, and in orbit. Beat. Which
- 16:12brings me to a final thought for you to ponder
- 16:14today. If your home's AC unit, your phone, and
- 16:17satellites in orbit are all continuously running
- 16:19massive AI agents that can instantly recall millions
- 16:22of data points, where does human decision -making
- 16:25actually sit in the loop? Thank you so much for
- 16:27exploring this deeply with us today. We will
- 16:29see you next time. You might be sitting right
- 16:31next to your air conditioner. Out to your own
- 16:33music.