Latest / B2B SaaS Talks with Fexingo: Enterprise Software, Sales Cycles, and Procurement / Why Enterprise Buyers Now Demand AI Output Insurance
Transcript
- Lucas: If these conversations are useful for what you're building or running, we keep this show ad-free by design. No sponsors, no pre-roll, no mid-roll interrupting the story. Luna: And honestly, that's pretty rare in B2B podcasting. Most shows in this space are underwritten by the very vendors we'd be talking about. Lucas: Right. So if you want to support that approach, the link is buy me a coffee dot com slash fexingo. Entirely optional, but it helps us stay independent. Luna: Appreciate that. Alright, Lucas — what's the new clause we're seeing in enterprise contracts? Lucas: AI output insurance. It's a requirement that the vendor carry a specific policy covering losses from ai generated content, recommendations, or decisions. Not just a general cyber policy — a standalone product that explicitly covers AI hallucinations, biased outputs, or erroneous predictions. Luna: So if my AI triage tool tells a nurse to discharge a patient who actually needs a CT scan, the hospital wants to know there's coverage specifically for that. Lucas: Exactly. And that's not hypothetical. In early 2025, a regional hospital chain in Ohio used an AI triage module from a major electronic health records vendor. The tool misprioritized a patient presenting with chest pain — flagged them as low risk. The patient went home, had a heart attack, and sued. Luna: I remember that. The hospital's professional liability policy denied the claim, arguing the error was a software defect, not a medical decision. Lucas: Correct. And the vendor's cyber policy excluded bodily injury. So the hospital was left uncovered for a seven-figure settlement. That case was the catalyst. Now procurement teams are writing AI output insurance into the contract as a non-negotiable. Luna: What does the policy actually cover? Is it like errors and omissions but for algorithms? Lucas: Essentially, yes. But with specific sub-limits for regulatory defense, recall costs, and third-party claims. The key distinction is that the trigger is the AI's output itself — not a data breach or a service outage. Some policies have a 'hallucination deductible' — the vendor pays the first fifty thousand dollars per incident before the carrier steps in. Luna: That's fascinating. And I imagine the premiums are scaling with model size and deployment context. Lucas: They are. A vendor embedding GPT-4-class models into a medical device might pay 3 to 5 percent of their software revenue for coverage. A company doing low-stakes content generation, like marketing copy, might pay 0.5 percent. The market is still very bespoke. Luna: Who's underwriting these? I assume not the same carriers that write D&O. Lucas: Specialty players mostly. One that's emerged is Ascend AI Risk — they launched in late 2024 and now have about 80 policies in force. They use a proprietary scoring model that looks at training data provenance, fine-tuning methodology, and guardrail architecture. Traditional carriers like Chubb and AIG are piloting products but haven't fully committed. Luna: So if I'm a B2B SaaS vendor with an AI feature, how do I even get a policy? Is there an application process? Lucas: Yes, and it's more rigorous than a standard cyber application. Underwriters typically require a third-party red-team report, an AI explainability audit, and documentation of your model card. They want to see your training data hygiene, your RLHF process, your output filters. Some even interview the engineering lead. Luna: That is a huge shift. A year ago, buyers were asking for SOC 2 and maybe a penetration test. Now they want you to open up your model governance. Lucas: And that's exactly what's happening. I've seen RFPs from financial services firms that include a fifteen-page AI insurance rider. They want the policy to name them as an additional insured, they want a thirty-day notice of cancellation, and they want a minimum coverage limit of ten million dollars per occurrence. Luna: How are vendors reacting? Are they pushing back or absorbing the cost? Lucas: Mixed. Larger vendors with established risk management departments are buying the coverage and treating it as a cost of doing business. Smaller startups are struggling — the underwriting process is expensive and time-consuming. Some are building the premium into their subscription pricing, effectively passing it to customers who require it. Luna: And the SEC is getting involved too, right? They've been looking at AI disclosures. Lucas: Yes. In March 2026, the SEC issued a sample comment letter to public companies asking them to detail how they assess and insure against ai related liability. That's forcing boards to ask whether their existing insurance programs have gaps. Many are discovering they don't cover 'AI output' as a distinct exposure. Luna: So this is moving from a procurement niche to a governance standard. What's the timeline look like? Lucas: I think by mid-2027, any enterprise software deal involving generative AI will include an AI output insurance requirement. It'll be as standard as a data processing agreement. The underwriters are building actuarial tables now based on the first couple of years of claims data. Luna: Are there any early claims that we can point to besides the hospital case? Lucas: A few. One involved an HR platform whose AI screening tool rejected candidates based on a proxy for age — the employer was sued for discrimination. The vendor's AI output insurance covered the defense costs, but the settlement was apportioned between the vendor and the employer based on their respective policies. That case is still working through the courts. Luna: So the insurance itself is creating new legal questions — like what shared liability looks like when both parties have overlapping coverage. Lucas: Exactly. And that's where brokers are adding value. They're designing 'AI liability towers' that stack the vendor's policy, the buyer's own policy, and sometimes a wrap-around umbrella. It's getting complex fast. Luna: What advice would you give to a B2B SaaS founder listening right now? Lucas: Start the underwriting conversation early — before you close your next enterprise deal. The process can take 8 to 12 weeks. Get a red-team report, document your model card, and talk to a specialty broker who understands AI risk. If you wait until a buyer asks for it, you'll be scrambling. Luna: And maybe look at Ascend AI Risk or one of the other dedicated carriers. It sounds like the traditional market still hasn't caught up. Lucas: It hasn't. But an interesting development: a few large self-insured captives are being formed by consortiums of SaaS vendors to pool AI risk. That might become an alternative for companies that can't get coverage in the commercial market. Luna: So we're seeing the early stages of a whole new insurance category being born, driven entirely by procurement requirements. Lucas: Exactly. And if the hospital case taught us anything, it's that the old policies weren't built for this. The industry is playing catch-up, and the buyers are forcing the pace.