Latest / Super Intelligence News and Report / Nasdaq's $2T October wipeout, Anthropic ROI tools, OpenAI agent rules
Transcript
- Super Intelligence News and Report — Nasdaq's $2T October wipeout, Anthropic ROI tools, OpenAI agent rules Host 1: So is the AI industry um about to just to just collapse under the weight of a 5.5 trillion dollar server bill? Because today we are looking at how big tech is desperately scrambling to prove that AI actually makes money. Host 2: Welcome to Super Intelligence News and Report for Saturday, October 31st. For For those of you following AI and business closely, well you know your time is valuable so we are skipping the basic definitions and diving right into the sources. Host 1: Yeah and we have a lot lot to cover you're leading on the money and policy front today right what is the biggest headline? Host 2: Yeah looking at the financial fallout there is a massive story this was reported not confirmed on on October 30th via Reuters basically the Nasdaq 100 is facing a two trillion dollar October wipeout it is down over 5.8% for the month. Host 1: oh wow two trillion Host 2: I I know right and it is directly tied to Meta Alphabet and Amazon because they all drastically raised their AI infrastructure spending forecasts I mean JP Morgan now expects 900 billion dollars in AI capex just in 2026 and and a staggering 5.5 trillion trillion dollars by 2030. Host 1: Wait wait 5.5 trillion just to put that in perspective for you guys listening that is roughly the GDP of Japan but entirely dedicated to AI servers by 2030. 2030 Host 2: exactly and why it matters is that the the market is clearly spooked investors are no longer just rewarding cloud growth like they used to you know they are looking at this huge debt funded mix and these highly uncertain payback timelines and they are finally demanding to see real ROI Host 1: well with that 5.5 trillion dollar number hanging over their heads it makes total sense that that buyers are panicking about ROI which actually perfectly explains our next our next headline on the products front from October 28th Host 2: Right, the Anthropic news. Host 1: Yeah, so so Anthropic just shipped their enterprise enterprise proof of value stack. It features these smart reports and pooled budgets and we're already seeing the impact because Jellyfish data shows spending on these exact tools grew 10 times and now makes up 2% of total engineering costs. Host 2: 10 times growth is wild. Host 1: But the real kicker here is how they are actually tracking that value. Host 2: Exactly. Host 1: Why it matters is that Anthropic is essentially using models to audit other models just to prove that ROI to corporate buyers. Host 2: Wait, so we have AI watching how humans use AI just to prove to the bosses that the AI is worth the the subscription cost? Host 1: Yeah, pretty much. I mean, it sounds like a complete recipe for some serious workplace surveillance tension. Host 2: It absolutely does. But I guess the pressure to justify that massive spend is forcing their hand. And speaking of the tension between oversight and autonomy, we have another major internal major internal shift. Host 1: Right. Going into research and tools. This was reported, not confirmed, by Reuters Reuters on October 29th. OpenAI completely rewrote its model spec. They basically retired their detailed style guidance entirely. Host 2: Oh, interesting. Host 1: So what did they replace it with? Host 2: They centered it around five core values. Faithfulness, harm minimization, honesty, human agency, and care. Host 1: Just those five? Host 2: The mechanics of that shift are actually very telling. Host 1: Yeah, and why it matters is that by stepping away from micromanaging the output style, their charter shifts the entire focus toward agents preserving oversight. It is a direct response to response to some recent internal monitoring weaknesses they have had. Host 2: Right. Especially as these models start acting more autonomously. But returning to the sheer cost of keeping all these autonomous models running we have have some major policy and chip maneuvers to look at. Host 1: Oh, right. The NVIDIA deal. Host 2: Yeah. October 30th filings reveal NVIDIA is going to pay one billion dollars for a 4.54% stake in Navra on November 6th. And this move actually clears execution for a massive nine billion dollar Brookfield AI factory contract. Host 1: Wait, hold hold on. Let me make sure I understand the mechanics here. NVIDIA isn't just selling chips anymore. Host 2: No. Exactly. Why it matters is that NVIDIA is operating as a financier architect here. They are basically acting like a bank to fund the very factories buying their chips. Host 1: Because Because the hyperscaler balance sheets are just totally totally strained right now. Host 2: Precisely. They are maxed out on debt. So NVIDIA is helping them set up special purpose vehicles or SPVs to bypass those strained balance sheets. Host 1: Ah, so they use these side companies to keep the massive infrastructure debt off off their main books and like hide it from Wall Street. Host 2: Yeah, pretty much. It keeps the money flowing so the building never has to stop. Host 1: Okay, let's unpack this. Looking at all these sources together, you really have to wonder, are these companies building AI to help humans or are they just deploying anthropic surveillance tools to justify JP Morgan's massive 5.5 trillion dollar capex estimates. Host 2: It is a valid question. Host 1: Right. I mean, it feels like taking out a second mortgage to pay the contractor and now you have to install surveillance cameras in the house just to prove to the bank that your family is actually living there. It's like the massive server bill is the tail wagging the AI dog. Host 2: Well, we connect this to the bigger picture you have to ask yourself a tough question. Host 1: What is that? Host 2: If debt markets are forcing infrastructure funding into special purpose vehicles just to keep the lights on will adherence to to OpenAI's new five value rule book inevitably take a back a backseat to pure financial survival? Host 1: That is definitely something for you to mull over as we hand things off to the main story. Host 1: Imagine you own this like really nice piece of commercial property, right? And you just want want to keep the local neighborhood dogs off your manicured lawn. Host 2: Sure. Yeah. Sounds perfectly reasonable. Host 1: Right. But in this highly specific, totally bizarre neighborhood, you can't just put up one standard no trespassing sign at the gate. You have to actually manufacture four completely different signs. Host 2: Oh man. Host 1: Yeah. And they have to be printed in four different languages, placed at all these different angles, and then you have to constantly patrol the perimeter. Host 2: Just to check if a new dog moved in, who, you know, only reads a fifth language. Host 1: Exactly. Exactly. It's exhausting. Host 2: And I mean, the stakes are significantly higher here than just some footprints prints on the grass. Host 1: Yeah. Host 2: If you miss even one of those signs, a dog doesn't just walk across your lawn, it scans the the foundational architecture of your building. Host 1: Wow. Okay. Host 2: It memorizes your structural structural engineering and then it uses that proprietary blueprint to build this slightly weirder, highly competitive replica right down the street. Host 1: Which is wild. But that structural theft, that has literally been the the daily reality for digital publishers for, for, I mean, the last two years. Host 2: It really has. Host 1: But we've got some breaking news today, October 30th, 2026, that completely shifts the landscape. Host 2: Huge shift. Yeah. Host 1: So the four largest US foundation model providers, that's OpenAI, Google, Microsoft, and and Anthropic, they have jointly agreed to honor a single unified robots.txt line to refuse AI training. Host 2: Finally. Host 1: Right. And the code is user agent colon underscore underscore usage underscore AI dash training. Host 2: It's a mouthful, but yeah. Host 1: It is. So today on the deep dive, we are pulling from this massive stack of SEO technical updates internal internet drafts from the IETF and a major legal amicus brief that was just filed by the the Wikimedia Foundation. Host 2: Lots of heavy source material today. Host 1: Oh, tons of it. But our mission is to basically figure out the reality of this new standard. So is this single line line of code actually the silver bullet publishers have been begging for? Host 2: To protect protect their IP, right? Host 1: Exactly. Or is it just this hollow PR victory that's masking a much much more existential threat? Host 2: Well, to answer that, we really have to look at the immense immense pressure that's been building up to this October 30th announcement. I mean, for years, web publishers have just been relying on these frantic DIY blockades. Host 1: Just patching holes on the ship. Host 2: Literally. They were maintaining these massive parallel exclusion lists. So they're blocking GPT bot for OpenAI in one file and then setting up Google extended commands commands in another. Host 1: Right. Host 2: And then they're configuring entirely different meta tags just to handle Bing. It was this fragmented, deeply inefficient mess. Host 1: And all of that that just to say no to having your your enterprise data ingested into an AI's brain. Host 2: Yeah. Host 1: While, you know, walking this crazy tightrope to avoid accidentally erasing your entire site from traditional web search in the process. Host 2: Which was a huge risk. Host 1: A massive risk. And let's be real, the AI labs, they aren't unifying now unifying now out of sudden corporate benevolence. Host 3: Oh, definitely not. Host 1: They are staring down the barrel of the EU AI Act specifically Article 50 of the act requires requires general purpose AI models to respect machine readable opt outs. Host 3: Right. Host 1: If they fail to comply with that code of practice, they face face these massive regulatory fines. And it's based on a percentage of their global revenue. And Host 3: And that regulatory threat is massive. But the actual catalyst for this specific rollout, it was actually driven by the infrastructure layer of the internet itself. Host 1: Wait, really? Host 3: Yeah. Host 1: How so? Host 1: so? Host 3: Yeah. So look into the key numbers in the timeline. It kind of reveals that the labs were effectively forced into this posture. We have the four major providers backing this one shared shared token now. Host 1: Right. Host 3: Right. But back on September 15th, 2026, Cloudflare rolled out a toggle that automatically writes a no training line for all of its users. Host 1: Okay, and Cloudflare sits in front of, what, nearly a fifth of all websites globally? Host 2: Exactly. So when they deploy a tool, tool, they aren't just making a polite polite suggestion. Host 1: Right. They are fundamentally altering the weather of the internet. Host 2: Yeah, because they're executing that block at the network level. Host 1: Okay. Host 2: Meaning the rejection happens before a crawler even reaches the publisher's host server. Host 1: Wow. Host 2: So when Cloudflare activated that toggle in September, it exposed this glaring vulnerability in Microsoft's architecture. Host 1: Oh, because Bing couldn't read it. Host 2: Exactly. Bing's crawling infrastructure fundamentally couldn't parse that specific automated network layer signal properly. Host 1: Yikes. Host 1: So Host 2: So Microsoft's crawlers were suddenly just blind to the de facto tooling adopted by a massive chunk of the web, like web, like overnight. Host 1: That is a huge problem for a search engine. Host 2: Massive. And that infrastructural blind spot basically forced their hand. And subsequently, the rest of the major labs had to standardize just to keep their crawlers functioning. Host 1: Okay, so a standard looks great on a press release, right? But I'm looking at the implementation schedule schedule here, and there is a staggering technical lag. Host 2: Yeah, it's not immediate. Host 1: Not at all. Like, if I manage a publishing platform and I inject that AI training line into my robots.txt file today, today, I am not protected right now. Host 2: No, you're really not. Host 1: Microsoft announced a rollout window stretching from December 15th, 2026, all the way to September 30th, 2027. Host 2: Because they have to phase out their their older directives, right? The no archive and no cache tags. Host 1: Exactly. And Bing won't even begin honoring the new preferences for active crawling until February 1st, 2027. And Google, they've only they've only committed to supporting it in, quote, early 2027. Host 2: Yeah, that timeline means there is absolutely no immediate change for the current foundational foundational training runs happening right now. Host 1: Wait, early 2027. That means there is absolutely no immediate change for the current training runs happening this very second. Host 2: Exactly. Exactly. Any site crawled in November or December of 2026 is still getting ingested under the old fragmented rules. Host 1: Which raises an obvious mechanical question. I mean, why does it take a trillion dollar tech conglomerate nearly a year to recognize a single standardized line of text? Host 2: Well, it comes down to the mechanics of crawl budgets and and these massive distributed cache architectures. Host 1: Okay. Host 2: Redexing the entire internet isn't just a switch you flip in a server room. These systems, they operate on historical globally distributed caches. Host 1: So they have a long memory. Host 2: Right. So when you introduce a new universal directive, the crawler has to systematically systematically revisit billions of domains. Host 1: Oh, wow. Host 2: It has to to clear the historical permissions cached on cached on edge servers worldwide and then write entirely new logic for for how the central index categorizes that data. Host 1: That sounds complicated. Host 2: It is. Flushing a memory cache of that magnitude without, you know, completely breaking current search functionalities it requires months of phased highly cautious deployment Host 1: Right. Okay well that timeline lag naturally opens the door door to a massive debate over what this agreement actually represents for the business ecosystem Host 2: Oh absolutely Host 1: like are we looking at a genuine foundational win here or just a clever stalling tactic so let me take the bold case here for a second kind of channeling Google search advocate John Mueller and the architects architects at the labs themselves. Host 2: Alright, make the case. Why should the enterprise web ecosystem view this as a victory? Host 1: Because it elegantly solves the SEO dilemma SEO dilemma that has literally been terrorizing terrorizing publishers. Host 2: The search visibility issue. Host 1: Right. Before this, if you blocked a crawler too aggressively at the server level, you risked just completely disappearing from traditional search indexes. Host 2: Which would just crater your organic traffic. Host 1: Exactly. Host 2: Right. Host 1: But this standard cleanly separates the two functions. So it gives publishers a broad sweeping choice in one step while perfectly preserving their core search visibility. Host 2: Okay, that makes sense. Host 1: And it also fulfills exactly what the EU Code of Practice explicitly demands, right? A primary universally recognized machine readable method for rights reservation. Host 2: True. Yeah. Host 1: And we are already seeing immediate practitioner uptake like Vanessa Fox. She's a major major SEO expert. She updated updated her auditing tools to detect this new user agent on the exact same day it day it was announced. Host 2: That's fast. Host 1: Right. The enterprise market is adopting it instantly because it just streamlines their entire workflow. Host 2: Okay, so the operational convenience is is undeniably strong. I'll give you you that. Host 1: But. Host 2: But the bear case reveals some pretty severe structural vulnerabilities. So for that perspective, we have to look at the look at the analysis from Nate Hake. Hake. He's an active participant in the in the IETF, which is the Internet Engineering Task Force. Host 1: Okay, what's his take? Host 2: He highlights a a crucial detail. This agreement is not a ratified global internet standard. Host 1: Wait, it's not? Host 2: No. It is merely an an individual internet draft authored by by Kevin Kelly over at Anthropic. Host 1: Oh, so they're bypassing the formal RFC process entirely. Host 2: Exactly. It's just a proposal the four companies agree to voluntarily. Operating totally outside the official standard setting bodies. Host 1: Wow. Meaning there are zero external audit mechanisms. Host 2: Yep. Host 1: And absolutely no enforcement penalties if they fail they fail to uphold it technically. Host 2: None. And the lack of enforcement is troubling, right? But the draft itself contains a rather alarming alarming loophole regarding user actions. Host 1: a loophole like what Host 2: so the current draft language effectively exempts AI preferences if a user explicitly Host 4: the system. system. Host 5: Wait, if it exempts user actions, does that mean a user can just bypass a publisher's protective block by explicitly typing like summarize the text at this exact life URL into a prompt window? Host 4: That is the exact vulnerability Heike is warning the community about. Host 5: That is insane. Host 4: It leads artists, creators, and you know enterprise data sets fully exposed to AI extraction as long as the end user initiates the request. Host 5: Because the AI can just point to the user prompt as justification for bypassing the robots.txt restriction. Host 4: Exactly. Exactly. Furthermore, if we look closer, Bing's specific rollout strategy the operational burden is heavily skewed against the publisher. Host 6: How so? Host 4: Well, Barry Schwartz at Search Engine Land, he outlined this trap perfectly. Microsoft is giving giving publishers independent choices for AI training and AI grounding. Host 5: Okay. Host 4: But they are leaving Bing's grounding default set to allow unless publishers actively change their settings by March 1st, 2027. Host 6: Of course. The default configuration favors the house. Host 4: Always does. Host 6: So if a business doesn't realize they need to proactively update their webmaster settings, their data is automatically opted in in for grounding. Host 4: Yeah. Host 6: Which points to a much larger and honestly more dangerous realization here. Host 4: What's that? Host 6: So this shared token the AI training line training line it only protects a publisher's static archive from being sucked into the next iteration of a foundational model right? Host 4: Right. Host 6: but it does absolutely nothing to stop an AI from scraping their live real time data today to answer a user's direct search prompt. Host 4: And that technical distinction, that brings us right to the absolute absolute core conflict dominating the tech industry right now. Host 6: The divide between training and grounding. Host 4: Exactly. The hard divide between training and what the industry calls RAG or retrieval augmented generation. Host 6: So to visualize the mechanics here for you, model training is like a law student locked in a a library. Host 4: Good analogy. Host 6: They're studying thousands of case files for months to prepare for the bar exam, right? Building complex complex internal knowledge, recognizing patterns. patterns. Host 4: Right. That's training. Host 6: But RAG or grounding is fundamentally different. It's an open book test. Host 4: Yep. Host 6: The student gets a specific question runs into the into the library pulls one specific case file off the shelf reads the exact paragraph and hands that copied information directly to the examiner. Host 4: And the October 30th pact it only addresses the studying phase. Host 6: Wow. Host 4: It stops the model from internalizing the book beforehand it does not stop the open book test from happening in real real time. Host 6: Right. Host 4: And this specific mechanical distinction is currently the centerpiece of massive legal battles. Host 6: Oh absolutely like on the exact same day this robots.txt pact was announced the Wikimedia Foundation filed a major amicus brief. Host 4: Right in the New York Times versus OpenAI litigation. Host 6: Yes that massive lawsuit and their brief is fascinating they are they are arguing that model training so the internal studying phase where the neural pathways learn how language works that that is a transformative fair use. Host 4: Interesting. Host 6: But they draw a really harsh legal line when it comes to RAG. Host 4: So where do they draw it? Host 6: They argue that fetching live information to redeliver it to a user operates completely differently they say RAG requires a license. Host 4: Ah I see because the legal theory in the Wikimedia brief argues that when a model engages in live RAG it acts as a direct market substitute for the original content. Host 6: Right it replaces the need for a user to ever visit the publisher's actual website. Host 4: Exactly. and that is why Microsoft splitting their Bing Webmaster tools into two distinct categories, AI training and AI grounding is is just so revealing. Host 6: It's an admission. Host 4: The The foundational labs are openly acknowledging in their own dashboards that the negotiation negotiation over RAG remains an entirely separate totally unresolved battlefield. Host 6: And the economic stakes here for enterprise publishers are just devastating. Host 4: Oh completely. Host 6: I mean if foundational models stop scraping the web for historical training data but they they aggressively crawl it for live rag grounding to power their chat bots publishers are looking at a fiscal nightmare. Host 4: How so break that down? Host 6: Well think about it AI bots constantly pinging host servers for live grounding requires massive amounts of compute Host 4: Right. So the server load goes up. Host 6: Exactly. The publisher's AWS bills spike just to handle all this machine traffic, but because the chatbot summarizes the answer perfectly for the end user, the publisher gets zero zero click through traffic to monetize. Host 4: Oh, wow. So they are literally paying the server costs to host the exact data that the AI is using to put them out of business. Host 6: Exactly. Host 4: Okay. Host 6: It's a vicious cycle. Host 4: And that economic reality completely rewrites the risk calculus calculus for any digital enterprise. Host 6: Totally. Host 4: A CEO might read the October 30th headlines, see this unified standard and assume, great, we have successfully shielded our business from AI extraction. Host 6: Right. We put up the sign. We're safe. Host 4: But they only opted out of the out of the historical memory. They did not opt out of the live real time extraction. Host 6: Which introduces a landscape of staggering unknowns. Host 4: So many unknowns. Host 6: Because even if a business successfully implements this new shirt token, token, we really have no idea how it holds up in a multi billion dollar copyright lawsuit. Host 4: No clue. Host 6: I mean we know robots.txt has been a voluntary voluntary web convention since the 1990s. Host 4: Right it's old school. Host 6: Yeah and it was originally designed to stop primitive crawlers from getting caught in like endless indexing loops. It was was not built to negotiate complex intellectual property rights. Host 4: Right. So the big question is will federal judges or the EU AI office actually treat a polite text text based request to a web crawler as a legally binding reservation of rights under modern copyright law? Host 6: Exactly. And that remains entirely uncharted territory for the courts. Host 4: Yeah. And technically speaking, there are still major logistical conflicts unresolved too. Host 6: Like what? Host 4: What actually happens if a site's shared token says no, but their specific Google extended tag says yes. Who wins that technical tug of war? Host 6: Well, current engineering guidelines suggest that if a crawler crawler finds a conflict between a specific company tag and the general shared token, the specific tag might override the general one. Host 4: Okay, might. Host 6: Might. Because algorithmic crawler behavior is notoriously unpredictable when faced with conflicting code at scale. Well, and that unpredictability assumes the the crawler actually respects the rules in the first place. Host 4: That's a huge assumption. Host 6: Right. The The big four labs agreed to the standard, but what about the open weight ecosystem? We have Meta, Apple, XAI and perhaps most importantly Common Crawl. Host 4: Right the massive non-profit scraping data set. Host 6: Exactly. It serves as the foundational bedrock for countless models globally globally if Common Crawl doesn't respect the token does the token even matter? Host 4: It's a fair question. And honestly it is reported not confirmed that some of the smaller highly aggressive labs are simply ignoring robots.txt directives entirely. Host 6: Just blowing right past the signs. Host 4: Yeah they're operating under the assumption that the data arms race is just far too competitive to play nice with polite opt outs. Host 6: Wow. and then you have to add the complexity of proprietary creator tools to that list of unknowns Host 4: oh like OpenAI's stuff Host 6: yeah OpenAI has been discussing initiatives like initiatives like media manager which are are these conceptual tools designed to let creators creators control their IP on a highly granular piece by piece level Host 4: okay so how does a broad site-wide robots.txt opt out interact with those direct licensing deals Host 6: that's the million dollar question if a major media conglomerate signs a multi-million dollar licensing pact with OpenAI but their overarching site architecture still broadcasts a universal no training token determining which system takes precedence requires highly complex complex backend negotiation Host 4: it sounds like it creates a severe strategic dilemma for any business Host 6: it really does because if you deploy this universal block with one line of code of code you might lose the leverage to go to Google or Microsoft and negotiate a customized licensing deal Host 4: right right you are essentially taking your data off the open market Host 6: exactly which is why anyone managing a digital footprint has to watch the calendar very very closely the deployment dates dictate the strategy here Host 4: yeah so enterprise teams really really need to flag December 15th 2026 Host 6: that's a big one speaker_C|> Host 4: it is that is when Microsoft's temporary grounding controls go live in Bing Webmaster Tools for sites relying on old directives directives Host 6: okay Host 4: next is February 1st 2027 that's when Microsoft actively begins honoring the new preferences in their crawl logic Host 6: got it we also need to track Google's specific implementation implementation update which they just broadly promised for early 2027 Host 4: right Host 6: and crucially the upcoming IETF interim meetings regarding the AI preferences preferences internet draft Host 5: we definitely need to watch their mailing lists Host 6: yeah to see if they address that controversial user action exemption and whether they can finally establish a shared standard vocabulary for grounding rather rather than just training before this whole thing becomes a locked in global standard standard Host 4: i mean the foundational plan Host 7: of the internet is is basically rewriting itself in real time, right? Host 8: It really is. Host 7: The decisions made in these next few months will dictate the rules of enterprise engagement for the next decade. Host 8: Which leaves us with a highly provocative thought to chew on regarding the architecture of the web itself. Host 7: Alright let's hear it. Host 8: If a universal do not train and do not ground signal eventually becomes as standardized standardized and you know legally binding as HTTPS does the open open web fracture entirely into two distinct distinct internets? Host 7: Oh wow two internets Host 8: Yeah, imagine a near future where we have one internet meant strictly for human eyes, right? Slower, heavily curated, visually complex, Host 7: okay Host 8: and running parallel to it, an entirely separate internet internet, locked away behind paywalls, heavily secured, and exclusively licensed to AI bots for their live rag feeds. Host 7: That is, a wild thought. Host 8: The era of a single unified web that humans and machines experience the exact same way it might have officially ended on October 30th. Host 7: The infrastructure has been permanently altered that's for sure. The only question left is who can afford to access the new pipelines. Host 8: We'll see you back here tomorrow at the same time and and please check the show notes for all sources.