Latest / The CTO Podcast with Fexingo: Technical Leadership, Architecture, and Engineering Org / How Salesforce Solved the AI Hallucination Crisis
Transcript
- Lucas: If these conversations are useful for what you're building or running, I want to be upfront about how this show stays independent. It’s just two operators talking shop, no ads, no sponsors. If today was worth a coffee to you, that's the link — buy me a coffee dot com slash fexingo. Luna: Yeah, it keeps the signal clean. And speaking of keeping things clean, let’s talk about the messiest problem in enterprise AI right now. Lucas: Which is confidence. You know the drill. A client asks their CRM for a summary of their last three contracts, and the model invents a clause that doesn’t exist because it sounded reasonable. That’s not a bug; that’s a feature of large language models unless you build guardrails around them. Salesforce has been wrestling with this specifically because their product is built on the premise of trust. If your assistant lies to you about your own customers, you’ve lost the value proposition entirely. Luna: Exactly. And unlike a chatbot for creative writing, where a hallucination might be funny, in sales or legal, it’s catastrophic. How did they actually solve the grounding problem without killing performance? Lucas: They stopped trying to make the model smarter and started making the context stricter. For years, the industry approach was to just feed more data into the prompt and hope the model filtered it. But as Salesforce scaled Einstein across billions of records, that broke. Their solution was a rigid Retrieval-Augmented Generation pipeline. Before any question hits the model, the system must first find the exact document snippets in their database that relate to the query. The model then acts strictly as a synthesizer, not a generator of new facts. Luna: So it’s less like a conversation and more like an open-book exam where the answers have to be cited? Lucas: Precisely. And here is the hard part: citation isn’t enough if the underlying data is messy. Salesforce realized they needed a layer of verification before the retrieval even happened. They implemented what they call a 'truth anchor.' The system checks the user’s permissions against the record owner’s permissions in real-time. If you don’t have access to a specific contract, the AI doesn’t just hide the answer; it actively refuses to process that part of the prompt. It forces the model to say, 'I can’t see that,' rather than guessing based on public patterns. Luna: That permission check sounds like it adds significant latency. Did that hurt the user experience? Lucas: It did, initially. Early benchmarks showed a delay of nearly two seconds per query due to the security audit step. That’s unacceptable for a UI that feels snappy. So they shifted the heavy lifting to the background. Instead of checking permissions synchronously for every word, they pre-calculate access scopes for the user’s session. The AI gets a masked view of the data universe before the model even starts thinking. It’s a trade-off: you lose some real-time flexibility, but you gain speed and security simultaneously. Luna: I love that. Masked views instead of live filtering. It reminds me of how databases handle indexing versus scanning. Lucas: Right. And another layer they added was feedback loops. Not just thumbs up or down, but structural corrections. When a sales rep corrects an ai generated summary, that correction isn’t just logged; it updates the weighting of the retrieval algorithm. If the model consistently pulls from outdated project folders, the system learns to deprioritize those sources over time. It’s continuous learning, but constrained within the boundaries of the verified data lake. Luna: Does that mean the model itself never changes, or does it retrain on those corrections? Lucas: The core model weights stay static. That’s crucial for stability. You don’t want your base language understanding drifting every time a junior rep makes a typo. What changes is the retrieval index and the prompt templates. Think of it as tuning the radio dial, not rebuilding the radio. The underlying voice is constant, but the station you’re listening to gets sharper based on usage patterns. Luna: That’s a much safer architectural pattern. It separates the intelligence from the knowledge base. But how do they measure success? Is it just about fewer complaints? Lucas: No, complaints are lagging indicators. They track 'citation fidelity.' Every time the AI gives an answer, it provides a link to the source record. Engineers monitor how often users click those links to verify the information. If the click-through rate drops below a certain threshold, it means the user doesn’t trust the source, even if the answer is technically correct. It’s a subtle metric, but it captures the nuance of professional trust better than raw accuracy scores. Luna: That’s fascinating. Trust is quantified by engagement with the proof, not just the conclusion. Do you think other enterprises are copying this architecture? Lucas: They’re trying, but most fail at the integration layer. Building a retrieval pipeline is easy. Connecting it securely to legacy ERP systems, crm platforms, and fragmented data warehouses is the nightmare. Salesforce had decades of cleaning their own data before they could even attempt this. Other companies are jumping into AI without fixing their data hygiene, so they end up with fast, confident nonsense. It’s the classic garbage in, gospel out problem. Luna: So the real bottleneck isn’t the AI model anymore; it’s the plumbing behind it. Lucas: Absolutely. The headline-grabbing part is the LLM, but the competitive moat is the data infrastructure that feeds it. Companies that treat AI as a plugin to their existing mess will get burned. Those that treat it as a lens through which they must clarify their own data will win. It’s a discipline problem, not just a technology problem. Luna: And that requires a different kind of CTO mindset. Less about buying the newest model, more about auditing the oldest records. Lucas: Exactly. You have to be willing to slow down the rollout to speed up the foundation. It’s counterintuitive when everyone else is racing ahead with shiny demos. But in enterprise software, speed is irrelevant if the output is unreliable. Reliability is the only currency that matters once you move past the pilot phase. Luna: Do you see this same pattern emerging in healthcare or finance, where the stakes are even higher? Lucas: We already do. In healthcare, the 'truth anchor' concept is being used to ensure medical recommendations are tied directly to peer-reviewed studies and patient history, not general web knowledge. In finance, it’s about tying trading suggestions to audited ledger entries. The pattern is universal: constrain the generative freedom, maximize the retrieval precision. It’s boring engineering, but it’s the only way to scale trust. Luna: Boring is good. Boring scales. Exciting breaks at scale. Lucas: That’s the takeaway. Everyone wants to talk about the magic of the model. The winners are the ones obsessed with the mechanics of the pipe. If you’re building AI products, stop asking 'how smart is it?' and start asking 'how verifiable is it?' That shift in question changes your entire architecture. Luna: Verifiability over intelligence. I’m going to steal that phrase for my next team meeting. Lucas: Please do. Because until we can prove where the answer came from, we’re just playing roulette with our users’ data. And that’s a risk no responsible leader should take. Luna: Agreed. Let’s see if we can get some leaders to start treating data hygiene as an AI strategy. Lucas: It’s a long game, but the early adopters of grounded AI are already pulling ahead. The rest of the market is still chasing the hype cycle. Luna: And the hype cycle always collapses. The ground stays solid. Lucas: Here’s to staying grounded. Thanks for listening.