Latest / B2B SaaS Talks with Fexingo: Enterprise Software, Sales Cycles, and Procurement / Enterprise Buyers Now Demand a Vendor AI Model Registry
Transcript
- Lucas: So Luna, I was talking to a procurement director at a large European bank last week, and she mentioned something I hadn't heard before—at least not as a formal requirement. They're now asking every AI vendor to maintain what they call a 'model registry.' Luna: A model registry? Like an inventory of all the AI models the vendor uses in their product? Lucas: Exactly. But more than just a list. They want a living document that tracks each model's lineage—where the training data came from, when the model was last retrained, what version it is, how it's monitored for drift. Think of it like a software bill of materials, but for AI. Luna: And this is becoming a procurement requirement, not just a nice to have? Lucas: For this bank, it's now a contractual obligation for any vendor whose product includes machine learning. And I think we're going to see more of this. It's part of a broader push for AI transparency and governance. Luna: If these conversations are useful for what you're building or running, we keep this show ad-free thanks to listeners who support us. You can find us at buy me a coffee dot com slash fexingo. No pressure, just a simple way to keep the episodes coming. Lucas: Yeah, it really does help. So back to the registry—this bank's procurement team realized that they had no visibility into which models their vendors were using, how those models were trained, or whether they were subject to bias or drift. And with regulatory pressure from the EU AI Act, they needed a systematic way to audit. Luna: Right, because traditionally, when you buy software, you know exactly what version you're getting. With AI, the model can change under the hood without you even knowing. Lucas: Exactly. One of the vendors they work with—a compliance platform—had actually swapped out a sentiment analysis model mid-contract. The new model was more accurate overall, but it performed worse on certain dialects. And the bank only found out because a user noticed inconsistent results. Luna: So the registry would have flagged that change before it caused issues. Lucas: Exactly. Now, the vendor has to log every model update, including retraining cycles and data source changes. The bank can review the registry quarterly, and if a model introduces new risk—say, it's trained on data that wasn't properly anonymized—they can demand a remediation plan. Luna: What does this mean for a typical B2B SaaS vendor? Most of them are using multiple models—maybe one for recommendation, one for fraud detection, one for natural language processing. Do they need to track all of them? Lucas: If they want to sell to enterprises that are serious about AI governance, yes. And I think the bar is going to rise quickly. The bank I spoke to is already asking vendors to provide model cards—you know, that standardized documentation that includes intended use, performance metrics, and limitations. The registry is one step beyond that: it's operational, not just descriptive. Luna: It sounds like a lot of overhead for vendors, especially smaller ones. Lucas: It is. But there are tools emerging. Some ML platforms now have built-in registry features. And the larger vendors are starting to see it as a competitive advantage—if you can provide a clean registry, you're signaling that you take governance seriously. That can shorten sales cycles. Luna: What about models that are fine-tuned per customer? Do they need a registry entry for each customized version? Lucas: That's the tricky part. The bank's requirement specifically covers any model that affects the customer's data or decision-making. So if a vendor fine-tunes a base model on the bank's transaction data, that fine-tuned model needs its own entry. Version control becomes critical. Luna: That seems like it could get unwieldy fast. Lucas: It can. But the bank's approach is to focus on material models—ones that have a direct impact on regulatory compliance, financial risk, or customer outcomes. They're not asking for a registry of every experimental model in a research lab. Just the ones that are deployed in production and affect their business. Luna: So it's a risk-based approach. Lucas: Exactly. And I think that's sensible. You don't want to bury vendors in paperwork, but you do want visibility into what's actually running. The registry also helps with incident response. If a model starts producing biased outcomes, you can trace it back to the training data and version. Luna: Are there any standards emerging for what a registry should look like? Or is it every procurement team for themselves? Lucas: There are some efforts. The ML Commons has a model registry working group. And the EU AI Act is pushing for documentation requirements that effectively mandate a registry. But right now, it's still fragmented. The bank I spoke to built their own template based on the NIST AI Risk Management Framework. Luna: So if I'm a vendor listening to this, what's the one thing I should do today to prepare? Lucas: Start by inventorying your own models. You can't provide a registry if you don't know what you have. Map each model to its training data, its performance metrics, its update frequency. Even if you don't have a formal tool, a spreadsheet is better than nothing. Then, when a prospect asks, you can show you're thinking about it. Luna: And if a vendor doesn't have a registry, are they getting excluded from deals? Lucas: In this bank's case, yes. For their highest-risk AI use cases, they're requiring it as a condition of contract. For lower-risk use cases, they're asking for a roadmap. But I think within two years, it'll be standard for any enterprise procurement involving AI. Similar to how SOC 2 became table stakes for SaaS. Luna: Interesting. So the registry becomes part of the vendor's trust narrative. Lucas: Exactly. And it's not just about compliance. It's about operational maturity. If you can show a prospect that you have a disciplined model lifecycle—from development to deployment to retirement—you're demonstrating that you understand the risks of AI. That builds confidence. Luna: What about open-source models? If a vendor uses something like Llama or Mistral, do they need to registry that too? Lucas: Absolutely. The bank requires registry entries for all models, whether custom-built or open-source. In fact, open-source models can be trickier because you don't always have full lineage. The vendor has to document what they know and what they don't. Transparency is the key. Luna: That makes sense. I think this is one of those trends that's going to accelerate fast, especially as regulators start paying more attention. Lucas: I agree. And it's a good example of procurement driving governance in a way that regulators alone can't. When buyers demand transparency, vendors build it. We've seen it with security, with privacy, and now with AI. The model registry is the next frontier. Luna: Well, this has been really insightful. For anyone in B2B SaaS, I'd say start that inventory now. Lucas: Yeah, the early movers will have a real advantage. We'll keep tracking how this evolves. Thanks for listening.