Latest / The CTO Podcast with Fexingo: Technical Leadership, Architecture, and Engineering Org / How Salesforce Built the Agentforce Trust Framework
Transcript
- Lucas: The biggest risk in enterprise AI right now isn't that the models are dumb. It's that they're too good at what they do, combined with zero guardrails on what they're allowed to touch. When you give an AI agent access to your CRM, your billing system, and your customer support logs, you aren't just deploying software. You are delegating authority. Luna: And most companies are treating it like a feature toggle rather than a liability waiver. How are leaders actually structuring those boundaries? Lucas: Salesforce recently laid out their Agentforce Trust Framework, which is one of the few concrete attempts to solve this at scale. The core insight is that trust in traditional software comes from deterministic code execution. Trust in AI agents has to come from behavioral constraints and explicit permission scopes. They moved away from asking if the model is accurate, and started asking if the model is authorized. Luna: So they are decoupling capability from permission? That sounds like a fundamental shift in how we design APIs. Lucas: Exactly. In the Agentforce framework, every action an agent takes must be mapped to a specific permission set defined by the human administrator. The agent doesn't get blanket access to 'read and write customer data.' It gets access to 'update email preferences' or 'schedule follow-up calls.' If the agent decides a customer needs a refund, it can draft the request, but it cannot execute the transaction without a human checkpoint. Luna: That human-in-the-loop requirement is the bottleneck for efficiency, though. Does that slow down the workflow enough to make the agent useless? Lucas: It depends on the risk profile. For low-stakes tasks like summarizing meeting notes or updating ticket statuses, the agent operates autonomously. The permission scope is read-only or low-impact write. But for any action involving financial transfer, PII modification, or contract signing, the framework forces a pause. It’s not about stopping the agent; it’s about ensuring the human understands the context before the irreversible action happens. Luna: I see. So the architecture is really about defining the 'danger zones' in your data graph and putting walls around them. Lucas: Precisely. And the interesting part is how they handle visibility. Every decision the agent makes is logged in a structured audit trail. Not just the final output, but the reasoning path. Why did the agent decide to escalate this case? What data points did it weigh? This creates an explainability layer that satisfies compliance teams who are currently terrified of black-box automation. Luna: Explainability is huge for legal teams. But how do you enforce these rules technically? Is it just policy documentation, or is there code-level enforcement? Lucas: It’s code-level enforcement via a policy engine. The agent’s large language model generates a plan, but before any external API call is made, a separate validation service checks that plan against the permission sets. If the plan violates a constraint—like trying to export more than fifty thousand records—the validation fails immediately. The agent never sees the success of that action because it was blocked before execution. Luna: That’s a classic separation of concerns. Generation versus validation. It reminds me of how we handle database transactions, but applied to natural language intent. Lucas: It is exactly that. And it forces engineering teams to think in terms of intent mapping. You can’t just ask the model to 'help the customer.' You have to define what helping looks like in terms of discrete, permitted operations. It pushes the complexity from the inference stage into the planning stage. Luna: Which means the engineers need better tools for defining those plans. Do we have enough tooling for that yet? Lucas: Not really. Most current frameworks assume the developer will manually write the functions the agent can call. But as these systems become more autonomous, we’re going to need dynamic permission scoping that adapts to the context of the conversation. Salesforce’s approach is a starting point, but it’s still quite rigid. Luna: Rigid is safe, I suppose. Flexible is fast. Where’s the sweet spot for mid-market companies that can’t build custom policy engines? Lucas: They rely on platform providers to abstract the complexity. If you’re building on top of AWS Bedrock or Azure AI, you’re inheriting some of these guardrails, but you’re also inheriting their limitations. The key takeaway for CTOs today is that you cannot treat AI integration as a purely technical problem. It’s a governance problem first. Luna: Because if you mess up the governance, you don’t just have a bug. You have a lawsuit. Lucas: You have a reputational event that lasts years. Look at the recent cases where chatbots gave incorrect medical advice or biased hiring recommendations. Those weren’t failures of intelligence. They were failures of boundary definition. The agents knew how to speak, but no one told them what not to say. Luna: So the job description for an AI engineer is changing. It’s less about prompt engineering and more about policy engineering. Lucas: Absolutely. You need people who understand both the capabilities of the model and the regulatory landscape of your industry. A healthcare AI engineer needs different constraints than a retail AI engineer. One deals with HIPAA compliance, the other with consumer protection laws. The code might look similar, but the trust framework underneath is completely different. Luna: That specialization is going to create a new tier of senior roles. People who bridge law and engineering. Lucas: We’re already seeing that in the boardroom. Chief AI Officers are becoming standard titles, and their primary KPI isn’t accuracy. It’s incident rate and compliance adherence. They are measuring how often the agent asks for help, not just how often it succeeds. Luna: Asking for help is a metric? That’s counterintuitive for performance reviews. Lucas: It should be. An agent that constantly overreaches is dangerous. An agent that politely says 'I’m not authorized to do that, let me find a human' is valuable. It shows the system respects its own limits. That’s the hallmark of mature AI deployment. Luna: So, if I’m a CTO listening to this, what’s my immediate next step? Do I rewrite my entire API gateway? Lucas: Start with an inventory. Map every sensitive data type in your stack. Then map every potential AI interaction point. Ask yourself: if an AI agent accessed this data, what could it do? If the answer is anything worse than 'view,' you need a permission gate. It’s a simple exercise, but most companies skip it because they’re rushing to launch. Luna: Rushing is the enemy of trust. I agree. You build the fence before you let the dog out. Lucas: Exactly. And while we’re talking about building fences, I want to take a quick moment to talk about how we keep this network running. If these conversations about technical leadership and architecture are useful for what you're building or running, consider supporting us directly. Listener donations are what keep this show ad-free and independent, allowing us to dig into these deeper structural topics without corporate influence. You can find us at buy me a coffee dot com slash fexingo — that's F. E. X. I. N. G. O. Luna: It really helps sustain the research time we need to bring you these deep dives. Thanks for keeping the lights on. Lucas: Back to the architecture. Let’s look at the failure mode of this trust framework. What happens when the permission logic itself is flawed? Luna: Good question. If the rules are poorly written, the agent might find loopholes. Or worse, it might refuse valid actions, creating friction. Lucas: That’s the tension between security and usability. Salesforce addresses this with iterative refinement. Admins can review blocked actions and adjust permission scopes based on actual usage patterns. It’s not a static set of rules. It’s a living policy that evolves as the agent learns what the business actually needs. Luna: So it’s adaptive governance. The system learns from its mistakes, but only within the bounds of what the human approves. Lucas: Yes. And that requires a feedback loop that many organizations lack. They deploy the agent, set the permissions, and forget about it. But AI drift is real. Over time, the model’s behavior can shift, and old permissions might become risky. Continuous monitoring is essential. Luna: Monitoring the monitor. That adds another layer of complexity to the ops team. Lucas: It does. But it’s cheaper than a breach. The cost of implementing a robust trust framework is high upfront, but the cost of ignoring it is existential. As we move toward November 2026, we’re seeing the early adopters of these frameworks pulling ahead in trust metrics. Customers are choosing vendors who can prove their AI is safe, not just smart. Luna: Trust becomes a competitive advantage. That’s a powerful motivator for engineering teams who usually prioritize speed. Lucas: Speed is important, but sustainable speed requires trust. You can’t scale if you’re constantly putting out fires caused by autonomous errors. Building the guardrails allows you to scale confidently. Luna: So the lesson is: build the cage so the lion can run free. Lucas: Something like that. The cage defines the arena. Inside that arena, the agent can perform at its best. Outside of it, chaos. The CTO’s job is to design that arena carefully. Luna: And to make sure everyone knows where the walls are. Lucas: Always. Because once the walls fall, there’s no getting the data back.