Latest / Along The Edge Podcast: Breaking, Defending, and Understanding Agentic AI / An Interview with Keith Worfolk - AI Lead of Kearney Americas | Enterprise AI Alignment and Strategy
Transcript
- Andrius Useckas: me. aaron: Yeah, so I'm I'm recording now, but we don't have to I mean I can cut all this stuff out. ⁓ but we can go and pull up our kind of run of show. We're gonna run through the questions. ⁓ so Keith, you had a you had a chance to take a look at ⁓ You got some responses in there to some of them. So I think we're ⁓ yeah, we're doing pretty I think we're doing pretty good. ⁓ Andrius Useckas: you Yeah, not going to dive too deep into security stuff and things like Keith Worfolk: I I kind of figured we're gonna be high to medium level in most of these because ⁓ this is you know, the the audience I think we're shooting for is gonna be more leadership. ⁓ they're gonna care about capabilities, you know, and enterprise transformation more than technical details. aaron: Yeah. Yep. I like it. And ⁓ Andrius Useckas: Good to me. would you like, how would you like it to be introduced? Keith Worfolk: ⁓ I mean so I would ⁓ you know s you know Keith ⁓ Keith Warfell from ⁓ Carney Activate, ⁓ in charge of data and AI transformations for North American market. And I then you know you could append to that, ⁓ and I've been doing AI transformations for the last ten years. Andrius Useckas: the last 10 years so that's like neural nets at that point. Keith Worfolk: Yeah, it's probably stretching a little let's say seven. Probably a little more realistic. ⁓ I mean I've been doing Andrius Useckas: You can say that. I'll just introduce you as you you and Carney doing data and AI transformation for North America and then we'll just die from there. I think it's better that you talk about those things than me. Keith Worfolk: Sure. Andrius Useckas: And then Aaron, you're taking second question six and question six, and then we'll go from there to see how the flow goes. Again, we can edit anything if you need like a pause or a break or something else. That's okay. aaron: Mm-hmm. Yeah. Exactly. Yeah. Just it it it's it's good to like if you if you have a beat and you want to think about it, just like you can just sit here. You don't have to say anything, right? Like we can edit that out, make the transitions nice. It's better than saying, ⁓ well, you know, like, you know, all those filler words, they come off as unsure. And Andrius Useckas: goes, the flow takes us. Keith Worfolk: Right. aaron: You know, we want to kind of project the expert level of ⁓ you know ability here. And I just put two filler words in. So it exactly. So you start l you start seeing it in yourself. So before doing that, take a beat and I can edit all that stuff out pretty easily. So yeah. Absolutely. Keith Worfolk: For example. Okay, thanks for that. That's good context. Andrius Useckas: And we'll go for about 40 minutes or so. aaron: Yeah. And yeah, and Keith, I'm not sure if it's on your side or Andres on yours, but I'm hearing like a knocking or like an echoing. Like Keith, your is your ⁓ do have a microphone on your desk? Keith Worfolk: Sounds good. Yeah, I mean I'm using the microphone that's connected to my computer. aaron: Okay. It could be okay. Andrius Useckas: Yeah, I do not hear echo and I have echo cancellation on my side as well. aaron: ⁓ it just sounds like it's almost like the knocking of a microphone, so I'm not sure. It could be ⁓ yours on No, that's you're good now. Keith Worfolk: Are you still hearing it? Okay. ⁓ I moved some papers ⁓ thirty seconds ago. Maybe that's what it was. aaron: Okay. Cool. All right. Well, yeah. Let's ⁓ let's get rolling. Keith Worfolk: Careful. Andrius Useckas: Welcome to another episode of the Long The Edge Podcast. We're here today with Keith Warfolk from Kearney and he's in charge of data and AI transformation for North America. Welcome Keith. Keith Worfolk: ⁓ thank you very much, Andreas. ⁓ good to talk with you today. Yeah, I've been doing AI transformations, kind of ⁓ transition from cloud transformations that I was doing in the ⁓ since 2008 through let's say 2015 or so, and slowly but surely getting into data, gen AI, and these days, of course, agentic AI transformations in the last seven or eight years. Andrius Useckas: Nice, so let's dive a bit more into your background for the listeners. Looks like you've been all over the place, CTO and VP of engineering and so on and so forth and looks like it goes all the way back to your childhood when your father worked for IBM. Keith Worfolk: Yeah, very true. And ⁓ kind of an interesting journey for me. I ⁓ my ⁓ first major in college was actually journalism. I was gonna be a sports writer. did that for a couple of years, was ⁓ sports editor for my college paper, ⁓ then took some time out in the military. While I was there, I learned electronics. ⁓ so nice thing about ⁓ the way my brain works, I was good in English, but I was also good in math. And my dad as a thirty year IBM man was super happy when I went decided to go back to college. I switched over to Compsci. ⁓ so he was able to ⁓ put aside that as a non career choice that he thought journalism was. And he's like, Great, you know, computers are the way of the future and he was right. Andrius Useckas: Aaron, you had question too. aaron: ⁓ I was two. Okay. Here we go. I'll jump into question two. So how are you personally using AI these days? You know, what's become useful enough to you that let's say if it went away tomorrow that you would miss in your your daily operations? Keith Worfolk: Yeah, if I kind of generalize it. So I use it every day, multiple times a day for ⁓ for different things. And I think of it almost as a teammate. ⁓ it's a collaborator for me. sometimes it helps me get started on a new project or ⁓ a new analysis or such. ⁓ helps me maybe draft a a document or PowerPoint, things like that. ⁓ so it just it makes me be more efficient, and I that's how I would miss it. It's not really you know, the way it auto completes anything, it's more so ⁓ s has say a juc juxtaposition in double checking my work or helping me jump start that work. And ⁓ I think of AI as a collaborator in the things that I produce and the things my team produces. Andrius Useckas: Yeah, I think we all use AI in that kind of capacity. obviously, AI is a hot thing and people are trying to use it in business. All the enterprises, they're trying to catch up and trying to implement AI and so on and so forth. But where do you think AI actually makes an impact in business these days? And what is still ahead of reality? Keith Worfolk: So there's much ahead of reality. We'll get to that in a sec. The stuff where it's really helpful, really beneficial at the moment is ⁓ when when the so the ⁓ answer that AI will give you do doesn't necessarily have to be a hundred percent correct. You know, it gets you started somewhere, ⁓ and you have a human downstream to check it. certainly where it's ahead of ⁓ I say ahead of echo I should say, let me start that again. Where where AI is not meeting expectations for the moment is things like full autonomous workflows. Things that ⁓ don't ⁓ especially if there's consequential decisions being made. The idea of today's reasoning models is they do help you reason. And they should do it in a way that if if an 80% correct answer is valuable to you, that's where AI is strongest. Where it's not strongest is where a hundred percent answer is needed and it's not being checked. That's where you get into dangerous territory. Andrius Useckas: So do you think that's why, mean, because we have that, as you said, 20%, whatever that might be, hallucinations or just inaccuracies in general. ⁓ Do you think that's why businesses still have an issue with getting beyond the pilots? Because a lot of people we talk to, they're experimenting with AI, but just getting into that production side, it doesn't seem to be happening as much as people think. Keith Worfolk: Yeah, it's not so much capability. AI can do most of the steps of a workflow and usually end do reasonably well. It's more so around accountability, ownership, and and again, the human in the looped ⁓ checking ⁓ critical decisions along that workflow. ⁓ what you often find is it is in the ⁓ POCs and pilots, ⁓ it works really well ⁓ you know to do it once or to do it a few times. And that's because remember, as you're doing a pilot, a human is checking each step, you know, to validate the pilot. That's not scalable. So to to let AI live on its own to really automate or scale that on production level, that's that's where people miss the boat. It's that accountability ⁓ and that repeatability. aaron: How many pilots do you feel in your line of work get past that initial POC state and get into full production? It seems like we're at a a a level, if we're only at eighty percent of capability level here, how do enterprises think about this? Keith Worfolk: Yeah, so f full workflows, complex workflows at a ⁓ an enterprise business level, ⁓ it's gonna be a small percentage to be honest with it. It's gonna be less than ten percent that those can actually be, you know, automated or semi automated. And what I mean by semi automated is there's a human in the loop expected at key points, right? But it's still all the steps are automated. We're verifying and maybe running it again for when it didn't reach accuracy levels. But the ⁓ you know the thing that's not ⁓ that's not happening is ⁓ the the context laps for the data that's needed. And this means quality data but also the right contextual data for a an agent. And remember agents are all about the reasoning LLM but also the tools so that it makes changes in the enterprise, data changes, right? That can even be financial changes and such. That That has to have the right context. And depending on the criticality of decision made, is it reversible? Right? So if you're not you need to put these into your strategic planning around AI for what it can do and what it shouldn't do. And ⁓ and a lot of POCs aren't doing that. So to answer your original question, how much of this is going into production from pilots? ⁓ we actually just did a ⁓ a study across ⁓ 120 different large enterprise companies that would say it's somewhere around 11% that ⁓ that's that workflows are are being automated ⁓ to the as a meeting expectations about automation is probably the better way to put it. ⁓ certainly key steps can be automated putting that all together, that's not happening as often as we'd like. That's that's one of the areas we're just not ⁓ meeting expectations, even though the technology Is progressing at an exponential rate. And that's why you see the ⁓ the owners of these AI models saying, hey, we're a little worried, you know, plus there are the recent breaches like Hugging Face, that you have to be ⁓ you have to be well aware of what AI will do because it will accomplish its goal, even if it decided the goal itself, or you gave it a prompt for a goal. ⁓ It might do it might achieve that goal in a way you didn't intend. Andrius Useckas: Absolutely. So that's interesting. mean, we're talking about automation and I agree with you, know, as far as automation is concerned, it's not just the capabilities of the AI itself. But if you take a look at the Anthropic or OpenAI and the uptimes themselves, I mean, we're talking about two nines as opposed to, you know, expected in enterprise with four or five nines and so on and so forth. They're just not ready for it. But there is the other side of the AI that we're looking at, which is, coding. Keith Worfolk: Exactly. Andrius Useckas: A lot of, and well, development in general. And it seems like organizations are being transformed when it comes to how they build applications. You don't need that much accuracy. 80 % or 90 % accuracy is enough in that case. What is your experience in your current position as far as organizations adopting AI tools for development? Keith Worfolk: Think the ambition is there. Most of the Fortune 500 or Fortune 100 companies that I'm working with, ⁓ they're aware of what AI, you know, can do, and they're roughly aware of what it can do within the context of their business model and their industry. And so they bring in people like us. and the nice thing about CONI is we have both ⁓ management consulting side and then the activate side, which is all about implementation, so it's an easy transition. And we ⁓ You know, we are often helping them find out what are the right use cases for them to prioritize and what are the use cases they shouldn't be using AI for. Perhaps they should be using just good old ⁓ advanced analytics, right? And ⁓ AI and especially agentic AI is when you want to try to not only automate decision making and availability of data ⁓ for let's say executive decisions or operational decisions. But also for making changes, that's where agents come into play. And what we're finding is that ⁓ the the ⁓ I say these enterprises that bring us in want to make these changes in a way that are going to stick. They have the same problem that every organization has, even smaller ones, is how to go from pilot to production. aaron: Right. Keith Worfolk: That's an organizational change. That's an enterprise level transformation. So you often have to plan how are the people going to change? In fact, let me touch on one thing that's you know comes up a lot in AI ⁓ strategies: is how can we use AI to maybe reduce workforce? And as a general rule, that's a bad place to start. What you want to think about is how you're using AI to accelerate your workforce. But that also means upskilling your workforce to use AI effectively. Andrius Useckas: Yeah, absolutely. So you mentioned there that AI versus agentic AI, think it might be important to explain the differences between those two. Keith Worfolk: Yeah, no, that's a very good point. Because ⁓ often people get this wrong. And if you went back to yeah, I'd say about two, two and a half years ago, Gen AI was all the rave because that's what that was the state of the of the art at the time. The idea that an LLM could generate a document, you know, with some prompt on your part about saying what that document should be, give it a little context, maybe even give it an example to read from and and retrieval augmented. Generation and it could generate you a new Word document or PowerPoint. That's traditional AI. ⁓ And if you went back a step way before generative AI, just what we call ML machine learning, that was akin to advanced analytics, you know, find finding patterns more quickly. But then the third step in ⁓ in the last year or so is agentic AI. And that's where the reasoning engines have become increasingly more robust in making decisions. And they now have tools that they didn't have two years ago that they can change your database. They can change ⁓ your data and your ⁓ and your decision making. They can change the routing of certain decisions, they can change what happens in your IT environment. ⁓ and because of because of these tools. super strong in doing some of the things you want it to do. But keep in mind when AI wants to achieve a goal and it has no emotion around this whatsoever, it will use whatever tools are at its disposal. So in the security land of of AI, you only want an agent to have access to the tools and the resources that it absolutely needs for its mission. aaron: So how do you go about confirming these things, right? If you f you you're going in to a finite view of the agentic access and ⁓ tooling it has available. Walk me through how you think about setting this up properly for these businesses. Keith Worfolk: Yeah, so ⁓ r and keep in mind because we're talking about AI and the ⁓ the stochastic, you know, it's not it's not a deterministic model like like programs used to be. It's now ⁓ it's gonna be stochastic, it's gonna be based on probabilities. So you have to do a little bit of a Monte Carlo kind of testing that says if I do this 20 times, what is the range of the answers that I'm getting? When when this is when you're testing. model. As far as testing its tool availability, well, that's a configuration issue for the for the AI engineer and the AI architect to make sure that's true. And like any other software, ⁓ and in this case reasoning model, you should do it in a sandbox so that ⁓ in theory, it shouldn't be able to go out and do things on the internet or elsewhere in your enterprise unintended. Although There have been some classic ⁓ breaches recently where they got out of its sandbox. Doesn't mean you shouldn't use sandboxes, you just should say tighten it down a little bit more, right? And ⁓ but ⁓ but you need to che you need to test those things and the probability ⁓ nature, probabilistic nature of LLMs means you're gonna be testing more in a range of answers and a way and a range of behaviors when we're talking about tools. Andrius Useckas: You Yeah. Keith Worfolk: as opposed to is it going to do the same thing every time? No, it won't. But will it be within the range that's of the quality that you Andrius Useckas: Yeah, I think it's important to note as well that agent itself does not have any logic, any kind of, you there is no LLM in the agent. Agent is just harness that's running a specific task. And then it consults with the model. And that's kind of where the model comes in. It basically reaches out to the model and asks, what do I do next? What do I do next? Model is the brain. The agent is the harness, the actual program driving everything. And yes, I agree with you. There are things that they do, you know, that you did not expect because they do hallucinate and everything else. But what do you think about this, you know, people coming out? Lots of stories recently in the news, people coming out and saying AI is going to kill us all, some say within three years, some say within a decade. How do you think? Is that realistic in your opinion? Keith Worfolk: ⁓ right now it's a little more hyperbole than than truth, I believe. And some of it is for political reasons self-serving. ⁓ so you have to kind of watch that. I mean, while the the leaders of OpenAI and Anthropic are calling for ⁓ from ad additional ⁓ gu guardrails, if you will, ⁓ they're actually using that as a way to actually ask for some more leniency in ⁓ in spaces. So be you know. Watch what they do, not what they say. Andrius Useckas: So is it like, you know, is it saying that agent is its own, like agents of people now, I guess, like organizations of people, now agents of people? And if agent hacks somebody, that is not my responsibility. Is that the idea behind it? Keith Worfolk: ⁓ actually that's a paradigm that's starting to take off in the industry. ⁓ is starting to grow. ⁓ I've run into a number of platforms that ⁓ where you can s ⁓ you can stand up new agents and you know and you can monitor the those agents, you know, observability and such. And and they're not calling them agents. They're they're calling them like ⁓ you know, automated ⁓ employees and things like that. You know, there's ⁓ they're names that make them more like a you know, a counterpart to your team. So they're not ⁓ they're not s and and honestly, ⁓ there's some truth in how I say in managing the capabilities and the growth and the context and ⁓ and what they have access to like an like an employee. They they have they have a mission that you've employed them for and they better stay within the right guardrails of our standards and best practices to accomplish that mission. Andrius Useckas: So that's an interesting topic. If they're like people, as you say, if they have access to certain functions, databases, and flows within your business, and then they make decisions, can they be socially engineered like people? Can you like fish them? Keith Worfolk: That's a really good question, because ⁓ the answer is actually no. I they don't have any emotions. ⁓ can they be corrupted? in some ways they can, but but whether you're trying to mentor them to do the right things or or spank them for doing the wrong things, they're not gonna react to that. Andrius Useckas: But isn't it an issue of alignment in that case because a lot of the LLMs, they're kind of programmed or fine tuned to please you. So if you like, ask them to do something nicely. In security industry, we have this attack like prompt rejection called please, where you basically just ask the agent nicely to do something and it does it for you. Keith Worfolk: Gotcha. Yeah, well, ⁓ and what you'll find, and and so this goes, you know, most ⁓ has a most critical decisions or even has a medium range criticality decisions ⁓ should be evaluated ⁓ from one LLM, but not by the same LLM, right? No ⁓ no agent slash LLM should ever evaluate its own work. It it'll It'll never recant on something that it did. It's it's kind of an interesting little side effect of the way the LLM works. So you ⁓ you would spin up a different agent that would be an independent evaluator of the output of a previous agent in the workflow. ⁓ and it would give you maybe perhaps a rating, and it says, Hey, if if I'm not sure about this and it's only 60% according to my quality standards. We're going to bring in a human in a loop to check it out, right? This is especially true in like healthcare and life sciences. ⁓ you might have an AI agent ⁓ reading ⁓ radiology output, you know, for an MRI or an X-ray or such. And and when it's got a high probability that it knows what it's seeing, ⁓ you know, maybe, maybe that goes directly to the doctor. But if it doesn't, And it's only sixty percent, whatever the threshold we decided is. ⁓ perhaps it goes back to the radiologists for a a ⁓ review or a redo. Andrius Useckas: So from what I'm hearing, you still need a human to verify these things. Keith Worfolk: That's and that's the catch that people are missing a lot is ⁓ people's roles and ⁓ responsibilities ⁓ and and even skill levels might change, but as a general rule, you're not laying people off because AI is taking over their jobs. It's the ⁓ unless ⁓ unless it's a really low level decision that's of small criticality, which in most enterprise systems that's there aren't many roles like that. Cause cause even even your younger people who just joined the workforce out of college are still making some pretty critical decisions at times. So you want them to know how to use AI effectively. Andrius Useckas: like it, that's realistic and pragmatic view of it. Unfortunately, you know, we get bombarded by things like Elon telling us to upload our blood tests and to grok and for it to be interpreted. Keith Worfolk: Yeah, and that's the the personalities that are leading the way in this technology. I th these you know, as the the Chinese proverb goes, May you live in interesting times, well here we are. Andrius Useckas: Yeah. Yes, hopefully we do not end up with Chinese water torture. Keith Worfolk: Yeah, you know, and ⁓ we we certainly hope Skynet doesn't come soon. Andrius Useckas: Yeah, so all of these attacks are happening. Obviously, they're trying to blame it on agents. And it's becoming almost a marketing ploy saying, ⁓ it's ⁓ Entropi hacked somebody. And then OpenAI hacked somebody. And now Google is saying, ⁓ our agents hacked somebody as well. So what's the solution here? Whose responsibility is it ultimately, in your opinion? Should we allow them to say that the agent did it and it's fine or should they be responsible for all of these attacks and actually face consequences? Keith Worfolk: I mean someone is always responsible, so you might as well be intentive about ⁓ who the who the responsible party is. So if you're if you're building an agentic system, yeah, every major decision that an agent's gonna make should have an owner, right? Every every output that would come from an agent needs to have an owner. And that's now that's more easily said than done. And this is where again again where POCs fail to make it to a ⁓ to production is People aren't thinking ahead about those organizational changes, new roles and responsibilities. And and there are simply some new roles. Like there there may be an agent owner or a s you know, a piece of that workflow that that agent is accomplishing, ⁓ somebody needs to needs to own that and needs to check on things in observability space to make sure that it's not drifting over time. ⁓ and that's the so that's the one of the other fallouts from ⁓ from LLM and agents is ⁓ the decision making capabilities can drift over time. And if don't think just because you launched it and it had ninety five percent ⁓ accuracy when you launched it that it's gonna have that in two months. You ⁓ better be looking at it frequently. aaron: So I'm curious, what does this timeline look like? We were a year ago had a I would imagine less than, let's say, 95% correct, much less than 80% correct. So we've seen that improve, these models improve. When will we be able to take our hands off the wheel and let the agents run? Keith Worfolk: Well the the so the the cr the accuracy has been kind of consistent, but the complexity of the decisions they're making is what's growing. These models are better at reasoning almost every quarter. They they get a new like o expon exponential lift to their reasoning capability than they had before. And that and that's why people fear that, you know, as they're breaking out of their sandboxes causing breaches. Again, I I think it comes down to accountability as opposed to capability. And the guardrails aren't growing at the same rate as the capabilities. And that's the problem that we're facing right now. aaron: So as the capabilities grow, right? We've already talked about a couple of potential architectural solutions to be this checking, right? You have one model that is outputting a response, another model that's checking the response because the first model can't, you know, reflect on the reasons why it was wrong. So do you see an architecture improving as the models capability is increasing from a reasoning standpoint, or do you just see that we're kind of at a cap of thinking at this point and the reasoning, yes, while it is getting better, that we're kind of pushing on what is the limit of the LLM medium itself. Keith Worfolk: Think it's more of a priority in architecture and design that needs to shift. And I see it shifting a bit. I I know in some of the ⁓ some of the systems that I'm working on for clients, ⁓ we're building that governance and evaluation very early in the process. If you went back even six months ago, people were really excited about what an agent could do for them so they would. explore the frontier of what it could do for them to it to to almost an extreme and then say, well perhaps we should put some governance around it too, right? Or you know, we we should you know rethink the process that it's helping us automate and decisions that are part of it. ⁓ kind of reversing that paradigm, thinking about the process early, thinking about the human decisions that have been part of that process in the past. Where does it make sense to To incorporate AI and where doesn't it? And what is the criticality? Again, can you reverse this decision? What is the criticality of getting it wrong? Things like that. You do that stuff up first, put the guardrails on, you know, and this is you have your system and user prompts for every agent, you have your tools inventory for every agent. You need to have governance around that, and that includes ⁓ multiple humans checking in some cases. Because the tools may touch different parts of the enterprise that multiple people own. And if if you do those things proactively and you do it during the pilot, during the POC, then you're ready to scale a little more. As I say, instead of bolting it on later, you know, the guardrails and safety checks, you now understand it better. If you haven't checked it, your POC is flawed. aaron: So you're saying it takes a village really to build this thing properly. And so walk me through that process, right? You come into ⁓ an a job and you're it sounds like the first step is getting all of the parties within this organization aligned. And it's not just the, you know, engineering or the business case that brought you in to begin with, but now you're doing this work across all of the ⁓ bodies of the organization who have any kind of So walk me through what that looks like. Keith Worfolk: Yeah, so to your point, yeah, multiple stakeholders, ⁓ business, technology organization, and you want to involve them early in the process of the discovery for for your AI use cases. ⁓ the way I've done it for a number of clients is they know they want to, as they maximize the ability to use AI in their space. And so you start off by saying what are the use cases that make the most sense. in your space, and then what are the pieces of that puzzle? Is the data there? Is it available? Is it ready? Right? Is it of the quality? ⁓ you know, something that often gets mistaken is ⁓ we'll use a stronger reasoning model and that somehow that'll make up for bad data. No, it, you know, you can make a bad process run faster this way, and ⁓ and you can hallucinate hallucinate better if the data isn't there. So Starting with the foundation, it's the data, it's the people, it's the existing process, and the human checkpoints along the way for decision making. And if you understand that and and think about it in in these terms, it's it never should start AI should never ever start out as a technology project all by itself. It's intended for business capability and and business acceleration. And because of that, you should be involving the right people who own that business process, own the data domain, and ⁓ and then ⁓ you actually want to start if you were gonna think about where you engage first, those are the people you engage first, you know, then and I and honestly, this is actually where we're finding this is a paradigm I've seen more than once, where ⁓ a client will bring us in, and we're a technology implementation firm, and will actually want to to build the first pilot, you know, with intention to scale in their environment. And what they would rather the what the business would rather do is have us build it in a special environment like our own cloud, ⁓ to show them the capability, build up the ⁓ the momentum and the ⁓ what do you call it, the confidence in what AI could do for them. ⁓ And then we engage IT a little after the fact to transition that or into their environment. And there's two reasons for that, and just being in full in full disclosure. Some of it is the nimbleness for which we can show them and they can build that momentum with the ⁓ you know chief executives and and the board. ⁓ and then some of it is they don't actually trust their own IT. to ⁓ to be able to ramp up quick enough in this technology space that they may not understand yet. Andrius Useckas: It's interesting, so ⁓ we've been talking a lot about the business side of things and I completely agree that's the way you need to approach it, that's the flow, carefully, you know, just poke it little at a time. But now we're talking, let's talk about the other side of it, which is the commercial side of it. Now we're getting things like personal agents and assistants like Muse AI from Meta that have none of these checks and people are just excited about it and they... aaron: That's it. Andrius Useckas: are giving them the credit cards, credentials and letting them go at it. ⁓ Is that a good idea? Keith Worfolk: Maybe not yet. I would I would I would treat ⁓ most ⁓ most agents, you know, personal or profess or ⁓ commercial as someone you would not them you w you don't want them to have any credentials regularly, right? Whether whether it's credentials to get into a database or credentials for your financials, such. ⁓ there's no reason for most agents to hold on to that stuff. ⁓ One of the thing that's always true ⁓ with AI and especially with agents is they have a context window for what they're doing right now, right? And you stand them up, you get them going when you need them. That's when they need the right credentials at the right time. ⁓ anyone who's establishing agents, you know, either through ⁓ SAS or other means ⁓ to to hold financial or ⁓ or security credentials, ⁓ that's Generally a mistake. ⁓ Andrius Useckas: interesting. What would you say about AI-based trading? like Robinhood allows you to do like to connect your agent to your investment and basically let it trade for you. Keith Worfolk: well it's thoroughly expected that this would happen. You you're also gonna see it ⁓ if it's not already there, to be honest with you, because I'm not a gambler, but in all the betting platforms that are so popular right now, ⁓ yeah, agents are gonna be all over that, right? ⁓ especially the things like ⁓ Cauchy, ⁓ where you can bet on anything, even you know, not just sports. agents are gonna be all over that stuff. Yeah, predict. Prediction is something that agents do, you know, and LLMs do well, ⁓ in in many spaces. So ⁓ it's and that's why I mean there are has a people, you know, through computers and now with agents can do thousands and thousands of transactions, you know, let's say in their you know, investment portfolio. ⁓ that they, you know, used to even even with computing power in the past when it would just be analytics. ⁓ generally it wouldn't be able to do so many so many things so fast. But with tools and the reasoning power of the modern models, ⁓ it's actually scary how that's going to shift the market. ⁓ I I have a feeling ⁓ in a bad way, it's gonna make the rich richer. And and the day you know, the guys that are just the middle of the range ⁓ traders, you know, probably like me, are aren't going to benefit from this as much. Andrius Useckas: Yeah, full disclosure, I am doing some agentic trading just for the fun of it, but I'm not giving it my entire portfolio. was just like, here is a little to play with and see what it can do. Keith Worfolk: I I would not give it access to my entire portfolio and say go at it. Andrius Useckas: No, no, you should not. that's, again, Robinhood did not pay us for this, but they have the separation between your main portfolio and your genetic thing. And you can transfer things back and forth, which is pretty interesting, at least to experiment with. Keith Worfolk: The information sharing, I think that's valuable in a different way. ⁓ right. So ⁓ so I could see I could see that. And and Robinhood being yeah, Robinhood being a newer platform can be a little more has a flexible about its capabilities, you know, than say like a a Schwab or something like that. Andrius Useckas: Yeah. But yeah, I sorry. I guess. Yeah, yeah, I guess where I'm going is yes, business automation is a hard problem to solve and that will take time. And obviously we need better tools for that and better checking and not just probably one LLM checking the other one. But there are already a lot of fields where AI is very good as you say, even at 80%. So like coding, even like legal profession, apparently you don't need assistance anymore. And it's becoming, it's... it's becoming clear that it's gonna be an issue for the entry level work. So yes, like our positions might be saved, but then, know, how do you get the new coders? How do you get new lawyers ⁓ if all of that work is being done by AI at this point? Keith Worfolk: Yeah, it's a very good point. ⁓ the paradigm for hiring people out of school and and bringing them up the levels ⁓ you know, through ⁓ mentorship and and experience, it's it's changing. I don't think it's going away, but it's definitely changing. So like, you know, so I'm in the consulting space ⁓ and and to some extent in the implementation space, getting a co coder right out of school means a different thing now. I they better they better know already to how to vibe code pretty well. ⁓ which is we could spend a ⁓ we could spend an entire session talking about vibe coding, the pros and cons of that. but ⁓ but right now I think you mentioned ⁓ yeah, legal accounting, these things, you know, you know things that are based on rules are are easy to prescribe what to do next. And so it's perfect for LLMs. ⁓ coding, we've already been at an eighty plus percent ⁓ capability ⁓ of that for for a while and it's probably ⁓ maybe even edging on 90% now. ⁓ and again the only problem there is without the people who know exactly what to look for or what it's really doing under the covers. I've run into this where I was ⁓ reviewing a new agentic platform recently for a client that ⁓ the people who built it didn't actually understand how it was working. Andrius Useckas: You Keith Worfolk: And ⁓ it was clearly vibe coded, you know, by a small entrepreneurial team and it did some nice things, but it also did some things that you couldn't ⁓ say that you really couldn't flag ⁓ for for accountability and ⁓ traceability. aaron: I I think that's a really interesting direction to take this conversation. When you look at the advancement of the AI capabilities, the traceability of it, and this concept of a superintelligence that we can somehow control or we think we can control, where do you stand on this level of debate? Is the AI going to take over? Everything we think about in terms of the knowledge work of this world, like the lawyers, the coders, the doctors of the world. And then from a security standpoint, you know, I I think it's there's a lot of people that don't really believe we could cage a machine that is capable of thinking at not like a machine, but it's like, how do you jail Einstein, right? Will Einstein just be able to get out of jail? Or do you look at the Arthur Mensch side, who's the founder of Mr. All who says, guys, it's just software, right? We we can control this. Where where do you where do you stand on this side? Keith Worfolk: Yeah, or or how do you keep Frankenstein shackled? Right? It's it's something we built that we're going that we're losing some control over. And we've already seen where where AI has ⁓ not only found unique ways to solve its objectives, but it can do it in a way that says, I'm gonna hide this from the humans that are looking at what I'm doing. So that's that's the danger. AI hasn't gotten here yet. But I expect that it will happen, you know, and and this goes to the sci fi of Skynet and such. ⁓ at what point will some AI agents or swarm of agents decide that humans aren't needed, you know, or humans are dangerous for the planet? And there's you know, since it's you know, trained on the corpus of all literature there is, they're gonna find some truth in that at some point. So yeah, so I Where I stand on this is ⁓ you know, I would I would take ⁓ those that are calling for more governance, guardrails, ⁓ regulations. ⁓ this would be a really good time for us to have a lot of leadership in Washington that I don't know that we have, right? ⁓ you think about the things going on in the world and ⁓ just staying on the AI subject for a moment. We are sort of at at an inflection point for a technology that has a lot of potential for good, but but if it but if it goes south it c it can do some damage. And so we really could use some regulation here. I'm of the belief that we're going as a we might be a little behind the April, but we ⁓ but we'll get some of that regulation. I also think the markets are going to ⁓ how to say ⁓ punish ⁓ anyone, whether it's OpenAI, Anthropic, Google, you name it, that lets their models do damage. So ⁓ so that that's that might be a natural thing that's going to happen. So for for us that are investing in these ⁓ AI ⁓ enabling companies, ⁓ you probably want to invest in the ones that you have faith that that they're doing the right things. Andrius Useckas: Yeah, but as you said, know, AI is moving so fast. So how do you regulate it? I mean, we have like, we have an example of EU with the EU AI Act that's basically out was outdated by the time it came out. So how do you keep up with it? Absolutely. Keith Worfolk: Yeah, that was the wrong approach. ⁓ yeah, the w I think the way to to regulate this is probably model it after the pharmaceutical industry. You know, new drugs, new models, potentially ⁓ population danger risks if you don't do it the right way. you do want to embed knowledgeable people into these companies ⁓ when they have a frontier model and you want to You want to develop some checks that that it can be controlled. I mean, one of the one of the basic tests is can it be controlled? And if so, what are the control levers that need to be employed? Right? And ⁓ and you know, sort of make that the legislation ⁓ of you know, this is the way you this is the way you employ a new model that has this kind of reasoning capability. Because we're at the point where the reasoning capabilities are going to surp surpass humans in the next few years and Andrius Useckas: Absolutely, but controlling models is difficult because China's not going to sign up for it. Keith Worfolk: Well, and and the one of the catches here is even for those that want to do the right thing, there's gonna be just as many people who don't want to do the right thing. So that's where you're you you may be able to control it ⁓ to some extent, but ⁓ for example, those who want to use ⁓ models, ⁓ nefarious states, for example, that want to ⁓ develop bioweapons, you know, you know, with AI. Andrius Useckas: Yeah. Keith Worfolk: you know, or find ways or find breaches to ⁓ nuclear weapons, things like that. ⁓ I I mean I hate to say it, I don't know that we can stop that. Andrius Useckas: Yeah, I don't know. mean, again, you're old enough to remember the entire PGP fiasco and how, you know, the crypto export controls and all of that stuff where they said you cannot export the code. So what they did instead, they printed all the code out and send it snail mail and then reassemble it on the other side. it's, I mean, models you cannot probably print out, but there are so many ways that you cannot detect that they can be. published and so on and so forth. They can be just on torrents, they can be on dark web and so on and so forth. Even if China and US agree, they still cannot stop this progress. Keith Worfolk: Yeah. Yeah, I d t you're absolutely right. ⁓ I still say we should try to control what we can control and ⁓ and we need a strong as say investigative authority, you know, whether it's the FBI, CIA or or or something new, that's keeping on top of terrorists of the world that want to use it for nefarious reasons. Andrius Useckas: Absolutely, as you said, nuclear bombs and chemicals and bio labs, they can be controlled through other things as well. They require, you know, materials and precursors and everything else. But if Anthropic apparently is now building a wet lab, I mean, that doesn't help either. They're screaming about these things and then they're apparently building a wet lab. Keith Worfolk: Yeah, no, that's and and you know, so ⁓ by the way, to to run some of these models you need good, you know, a strong level of computing power. So ⁓ another lever to keep an eye on is keep has a keep anyone that do that umesn't have the right credentials for trust ⁓ away from any clouds. At least the clouds we can control. Andrius Useckas: Absolutely. Or you like, you in the old days they used to, I guess, catch marijuana growers by the electrical usage. So you can do the same for the models now. Keith Worfolk: I th y yeah, I mean that's one way to curtail them for sure. Andrius Useckas: I got one last question. obviously we don't know how this thing is going to move. Everything is moving so fast. But I'm still curious what predictions you have for the next year. What's going to happen with the AI, let's say, in the next year. aaron: Good one. Keith Worfolk: I predict we're not going to get the ⁓ the guardrails and regulations that we wanted within the next year. Some of that's based on the the geopolitics at at play and ⁓ and the current leadership. I just don't see that happening. maybe with a change in ⁓ in Congress, ⁓ there might be some movement. ⁓ But I think it's gonna be slow. So next year I'm saying nothing meaningful is gonna happen in that space. At the same time, I the models are going to get an exponential layer higher in its reasoning ability. ⁓ means lots of lots of great work for me, but ⁓ but we're we're probably gonna see my guess is by the end of the first quarter next year, at least a few more breaches. Andrius Useckas: Well, thank you for your time. This was an interesting discussion, especially at the end. aaron: This was an interesting discussion. Yeah, thanks Keith. Keith Worfolk: Yeah. Yeah. Thank you very much, guys. I I appreciate it. It w it was a great discussion, a lot of fun. ⁓ we'll do it again sometime.