Latest / B2B SaaS Talks with Fexingo: Enterprise Software, Sales Cycles, and Procurement / Why Enterprise RFPs Now Demand an AI Risk Assessment
Transcript
- Lucas: So a few weeks ago, a Fortune 500 financial services firm updated its standard request for proposal template. Buried in the compliance section, they added a new mandatory deliverable: a completed AI risk assessment framework, mapping every AI feature in the vendor's product against five specific risk categories. Three of the six vendors who responded couldn't produce one. They were disqualified before the demo round. Luna: That sounds like it's becoming a pattern. We've covered so many new requirements—model cards, bias audits, data provenance. But an AI risk assessment feels like it wraps all of those into a single deliverable. Lucas: Exactly. And look, this show exists because of listeners who find these shifts useful for what they're building or selling. A small group of you already chip in monthly at buy me a coffee dot com slash fexingo, and that's literally what keeps this thing ad-free and going. So genuinely, thank you. Luna: Yeah, it's a big help. Now back to that RFP—what exactly were the five risk categories they defined? Lucas: Model integrity, data privacy, regulatory alignment, explainability, and incident response. Each category had a scoring rubric from one to four. If a vendor scored below three in any area, they were automatically excluded. That's the level of specificity we're seeing now. Luna: And I'm guessing this isn't just a financial services trend. Are you seeing it spread to other verticals? Lucas: Healthcare and insurance are the fastest adopters, but I've also seen it in manufacturing and even in some state government RFPs. The common thread is regulatory pressure—especially from the European AI Act, which introduces risk-tiering for AI systems. Any vendor selling into Europe now has to classify their AI as minimal, limited, high-risk, or unacceptable. That classification becomes part of the contract. Luna: So an AI risk assessment is essentially the vendor's self-classification plus their evidence for why they belong in that tier. That makes sense. But how should a B2B SaaS company actually build one of these things? Lucas: Right, that's the practical question. A solid assessment starts with a clear inventory of every ai driven feature in the product. Not just the obvious ones like recommendation engines or chatbots, but also things like automated email sorting, usage scoring, or anomaly detection. Then you map each feature to the risk categories. The financial firm I mentioned used the NIST AI Risk Management Framework as a reference. That's a good starting point. Luna: And then you need to evidence your compliance with each category. For model integrity, that might mean bias testing results. For data privacy, a data flow diagram showing where personal data goes. For explainability, maybe a technical documentation on how decisions are made. Lucas: Precisely. One mid-market insurance company we work with actually built a single-page scorecard that covers all five categories. They present it as a dashboard in the buyer's procurement portal. Their win rate on deals that require an AI risk assessment went from forty percent to over seventy percent in six months. Their competitors who didn't have one were just not invited to the next stage. Luna: That's a huge improvement. So it's not just a defensive document—it can be a competitive differentiator. Lucas: Absolutely. And it's not just about the document itself. The process of creating an AI risk assessment forces internal conversations that actually improve the product. You discover, for example, that your customer support chatbot retains chat logs for two years longer than necessary. Fixing that reduces both legal risk and storage costs. The assessment becomes a catalyst. Luna: What about cost? For a smaller SaaS company, building a comprehensive risk assessment sounds expensive. Lucas: It can be, but you don't need to start from scratch. There are open-source templates from the Open Group and from the Cloud Security Alliance. Also, the European AI Act actually provides a questionnaire for high-risk systems. You can adapt that. The key is to make it repeatable and updated regularly. It's easier if you embed it into your development lifecycle—every time you add a new AI feature, you update the risk assessment. Luna: But what about the buyers? Are they actually reading these assessments thoroughly, or is it just a checkbox? Lucas: Good question. From what I'm seeing, procurement teams are starting to treat AI risk assessments the way they treat SOC 2 reports—they do a high-level scan to see if it exists and looks reasonable, but they may not dive deep unless there's a red flag. However, a few sophisticated buyers, especially in regulated industries, have dedicated AI risk officers who scrutinise every line. I know of one large bank that sends the assessment to their third-party risk team, who then run independent tests on a sample of the vendor's AI models. Luna: So it's a bit of a spectrum. But if a vendor doesn't have one, they're essentially failing the first screen. That's become a dealbreaker. Lucas: Exactly. And I should mention that the timeframe for getting this together is shrinking. We've seen RFPs with a two-week turnaround from issue to submission. If you're not already prepared, you're scrambling. Luna: Let's talk about the scoring part. The financial firm used a one-to-four scale. What are some of the specific criteria that bump you up or down? Lucas: For model integrity, a one might mean you have no testing at all. A two means you have some basic unit tests. A three means you have a formal validation process with separate training and test sets. A four means you have an independent third-party audit. Similarly for data privacy: a two might mean you encrypt data in transit, a three means at rest too, and a four means you have a full data retention policy with automated deletion. Vendors that score a four in every category—those are the ones that move to the demo stage without questions. Luna: And the vendors that scored low—what were their common gaps? Lucas: The biggest gap was explainability. Many vendors couldn't clearly articulate how their AI models made decisions, especially for deep learning systems. The second biggest was incident response. They had a process for general outages, but nothing specific to AI—like what happens if the model starts producing biased output or hallucinating. The third was regulatory alignment: they didn't have a map of which regulations applied to which features. Luna: That explainability issue is interesting. I've heard that some vendors are now building what they call 'model fact sheets' to address it. Are you seeing that? Lucas: Yes, it's similar to the AI model card we discussed in episode 121, but often more detailed. A model fact sheet includes training data sources, accuracy metrics by subgroup, intended use cases, and known limitations. Some vendors combine it with the risk assessment into a single 'AI transparency document'. That's a smart move because it gives the buyer one place to look. Luna: So what's the one action item you'd recommend for a B2B SaaS company listening today who hasn't started this yet? Lucas: Start with an inventory. List every AI feature in your product—even the ones that are just a simple logistic regression. Then pick the highest-risk feature and draft a risk assessment for it using the NIST framework. That gives you a proof of concept. Then expand to the rest. And don't wait for a customer to ask—get ahead of it. Put the assessment on your website as a trust document. It signals to buyers that you take AI governance seriously. Luna: And what about the sales team? How should they position this in a demo? Lucas: Sales reps should lead with it. When a prospect mentions compliance concerns, they should say, 'We've already classified our AI features and have a risk assessment ready for your procurement team.' That changes the conversation from 'prove you're safe' to 'here's how we exceed your requirements.' Luna: I like that. It's proactive, not reactive. And it shortens the sales cycle, too. Lucas: Exactly. time to value is another factor. If you can hand over a completed risk assessment on day one, you skip weeks of back and forth. That's why buyers are starting to demand it—not just as a risk mitigation tool, but as a signal of vendor maturity. One procurement director told me, 'If you can't manage your own AI risks, how can you help us manage ours?' Luna: That's a powerful quote. It really drives home that this is now a table-stakes requirement, not a nice to have. Lucas: Right. And it's only going to become more common. With regulations like the EU AI Act coming into force over the next two years, we'll see this requirement embedded in nearly every enterprise RFP that involves AI. The vendors who prepare now will have a significant advantage. Luna: So to recap: inventory your AI features, create a risk assessment using a recognised framework, and make it a core part of your sales collateral. Lucas: And update it regularly. That's the part many companies forget. An assessment that's a year old loses credibility. Set a quarterly review cycle, and note the review date on the document. That shows buyers you're on top of it. Luna: Great advice. Thanks, Lucas. Lucas: Thanks, Luna. That's all for this episode. We'll be back soon with another deep dive into the evolving world of B2B SaaS procurement.