Latest / B2B SaaS Talks with Fexingo: Enterprise Software, Sales Cycles, and Procurement / Why Enterprise RFPs Now Ask About Vendor AI Incident History
Transcript
- Lucas: So Luna, I was on a call with a procurement lead at a regional bank last week, and she said something that stopped me cold. She said, 'We're now asking every AI vendor for their incident history.' Not their security roadmap, not their SOC 2, but their actual track record of AI failures. Luna: Incident history, as in, 'tell me about the times your model messed up'? Lucas: Exactly. And it's not just her. I'm seeing this question pop up in more and more enterprise RFPs. It's a shift from 'what could go wrong' to 'what has gone wrong, and what did you do about it'. Luna: That's a big change. A few years ago, nobody would even think to ask that. They'd just want to know if the model had been 'validated'. Lucas: Right. And validation is still part of it, but now they want proof of lived experience. They want to see the scars. And this ties into a broader trend we've been covering all season — buyers wanting evidence, not assurances. Luna: So give me a concrete example. What does this look like in practice? Lucas: Sure. Back in early 2025, there was a well-known generative AI tool used in a Fortune 500 pilot for internal financial analysis. It hallucinated a few numbers in a summary report — made up a revenue figure that was off by millions. The company caught it before anything was published, but they killed the pilot and postponed their enterprise rollout by six months. Luna: And that was the kind of thing that used to stay internal. But now? Lucas: Now, it's a case study in procurement circles. That buyer added a whole new section to their RFP template: 'AI Incident History and Remediation.' They wanted to know about every material AI failure in the past three years, what the impact was, how it was disclosed, and what the vendor changed as a result. Luna: That's a lot to ask. Are vendors actually answering? Or are they hiding behind confidentiality? Lucas: Mixed, honestly. Some vendors have been surprisingly open, especially the ones with strong governance practices. They see it as a differentiator. Others are pushing back, saying incidents are proprietary or that they're under NDA with their clients. Luna: But if a vendor refuses to disclose, doesn't that raise a red flag? Lucas: It does, and that's the leverage the buyers have now. If you're competing for a deal and your competitor is willing to share their incident history, and you're not, you look like you're hiding something. Especially in regulated industries — finance, healthcare, public sector. Luna: So this isn't just a nice to have. It's becoming a gating question. Lucas: Exactly. And it's not just about the incidents themselves. It's about the disclosure culture. Buyers want to know if a vendor has a policy for public disclosure, like a security researcher disclosure policy, but for AI incidents. Are they proactive about notifying customers? Do they have a public incident registry? Luna: A public registry? I haven't seen many of those. Do they exist? Lucas: A few are starting to emerge. Some AI labs have published transparency reports that include model failures. But for most enterprise software, it's still a patchwork. That's why buyers are pushing for it in RFPs — they want to standardize it. Luna: And what about the SEC? I know they've been cracking down on ai related disclosures. Did that incident we mentioned trigger any of that? Lucas: In that specific case, no, because the company was a buyer, not a public company making false claims. But the SEC has been clear that if you're a public company and you're making claims about your AI capabilities, you better be able to back them up. And if there's a material risk, you may have to disclose it. Luna: So this is partly a legal and compliance issue, not just a procurement preference. Lucas: Right. And that's what's pushing this from a 'nice question' to a 'must answer' in many RFPs. General counsels are getting involved. They don't want to sign a deal with a vendor that might blow up on them six months down the road. Luna: What are the typical questions in this section of an RFP? Give me a sense. Lucas: So they're asking things like: 'Describe any material AI incidents in the last 24 months, including root cause, impact, and remediation.' 'What is your process for detecting and reporting AI incidents?' 'Do you maintain a public incident report? If not, why?' 'How do you handle third-party AI components that fail?' Luna: That last one is key — because most vendors are using third-party models, so they can't just say 'it's not our fault.' Lucas: Exactly. If you're building on top of a foundation model from someone else, you're still responsible for the output. Buyers want to know how you're monitoring for issues in the models you don't control. Luna: What about the vendors who are already facing this? Have you talked to any who've been through an RFP with these questions? Lucas: Yeah, I spoke to the head of product at a mid-size SaaS company that sells an ai driven analytics tool. They had a major customer ask for their incident history, and they realized they didn't have a formal process for tracking it. They had a few scattered docs and Slack threads, but nothing standardized. Luna: That's a scramble. How did they respond? Lucas: They had to go back and reconstruct everything from memory and emails. It took them two weeks to put together a response, and they probably missed some things. They lost that deal, by the way, to a competitor who had a proper incident log. Luna: That's a tough lesson. But it's a good wake-up call for any AI vendor listening: start tracking your incidents now, before you're asked. Lucas: Totally. And that's the practical takeaway here. If you're selling AI, you should be able to answer these questions today. If you can't, you're leaving revenue on the table. Luna: And if you're a buyer, what should you look for in a vendor's response? Lucas: Look for specificity, first of all. If a vendor says 'we have a rigorous incident response process' but can't give you an example, that's a red flag. Also, look for a pattern of improvement. Have they had incidents, but each time they've made concrete changes? That's actually a positive signal. Luna: So perfect isn't the goal. But learning from mistakes is. Lucas: Exactly. A vendor with a clean record might just be lucky or small. A vendor with a documented history of incidents and thoughtful remediation shows they've been tested and they've grown. Luna: That's a nuanced take. I like it. And it's a reminder that this trend is really about operational transparency, which we've seen across the board, from security roadmaps to financial viability. Lucas: Right. And it's only going to become more standard. I'd bet that within a couple of years, every enterprise RFP will have a section like this, and vendors without an incident history will be at a serious disadvantage. Luna: That's a bold prediction. But given the trajectory we've seen, it's hard to argue. Lucas: Yeah. And you know, it's funny — this kind of conversation is exactly why we do this show. We get to see these shifts happening before they become common knowledge. Luna: Absolutely. And you know, a lot of listeners have said they appreciate having these conversations available. And it's a small thing, but a handful of them chip in monthly through buy me a coffee dot com slash fexingo, and that's literally what funds making this many of these. Lucas: Yeah, and we don't have ads, so that's really the only way we keep this going. But it's not about pressure, just acknowledging that it helps if you find value here. Luna: Exactly. And now, back to the future of AI procurement — where do you see this heading next? Lucas: I think we're going to see more standardization. Maybe industry groups or the NIST will come up with a template for AI incident disclosure, similar to vulnerability disclosure frameworks. And buyers will start to expect that as a default. Luna: So if you're a vendor, it's smarter to get ahead of it now rather than wait for the mandate. Lucas: Absolutely. Start logging everything. Build your incident response playbook. And be honest when things go wrong. Because in the world of enterprise AI, your reputation for transparency is going to be worth more than any feature you can build. Luna: That's a great note to end on. Thanks for unpacking this with me. Lucas: Anytime. See you next time.