Latest / Practical AI / AI Proficiency: From Users to Builders

Transcript

AI Proficiency: From Users to Builders · Practical AI

• 0:01
  Narrator: Welcome to the Practical AI Podcast, where we break down the real world applications of artificial intelligence and how it's shaping the way we live, work, and create. Our goal is to help make AI technology practical, productive, and accessible to everyone. Whether you're a developer, business leader, or just curious about the tech behind the buzz, you're in the right place. Be sure to connect with us on LinkedIn, X, or Blue Sky to stay up to date with episode drops, behind the scenes content, and AI insights. You can learn more at practicalai.fm.
• 0:35
  Now onto the show.
• 0:41
  Daniel: Welcome to another episode of the Practical AI Podcast. This is Daniel Whitenack. I am CEO at Prediction Guard, and I'm joined as always by my cohost, Benson, is a principal AI and autonomy research engineer. How are doing, Chris?
• 0:56
  Chris: I'm doing good today. How's it going?
• 0:58
  Daniel: It's it's going great. I'm continually impressed by just the the amazing AI work that's happening around, not only on on the coast, but in kind of the, I guess, the heartland, the the middle of the country where a lot of actually the large enterprises of of our country are are located. And we have a guest related to that today, but, but I do wanna remind folks, we're we're also involved and a sponsor of the Midwest AI Summit, which is coming up October 15 in Indianapolis. Really cool event, Chris. You were there.
• 1:37
  You saw what was going on. There's tables where you can sit down with actual AI practitioners and, you know, rather than just hear a bunch of talks, you can actually get feedback on what you're doing, suggestions, design, etcetera, and hear great talks. So recommend people check it out, Midwest AI Summit. You can use the code Practical AI 20 for 20% off. But we have an amazing, an amazing AI practitioner from from close by my area, more towards Cincinnati.
• 2:10
  Mike Lewis was on a previous episode with us. We got a ton of great feedback on that episode. He's chief AI architect at tier one performance. Welcome, Mike.
• 2:19
  Mike: Thank you, Daniel. It's great to be here again.
• 2:20
  Daniel: Yeah. Like I say, we got I I had multiple people comment on just the utility and and insights that they got out of our previous discussion. And just just for context, because I think it's so impressive what what what you're doing and what you're involved with. Could you just set a little bit of context for kind of the the types of projects that you work on? Like, give an example of kinda some of the types of companies that you work with in relation to AI initiatives?
• 2:50
  Mike: Yeah, sure. So, tier one performance is an end to end organizational performance and transformation partner. What does that mean? We kind of help some of the most of the largest organizations in the world rethink transformation. I would add in, though, that we have a nontraditional approach, a multi decade track record of success there.
• 3:17
  So, the types of clients we'd work with day to day would be Google, Eli Lilly, Takeda, Air Force, just most of the Fortune 100s. And we see everything related to transformation. So obviously, AI would just represent a sliver of the types of challenges we encounter as an organization day to day. But that sliver is my whole life, you know? So I don't really see all the other stuff.
• 3:47
  I kind of get a glimpse into it every now and then because more and more AI touches everything. But we are sort of a transformation partner, and so you can just imagine how the phone starts ringing when, you know, a massive disruptive technology is birthed into our lives whether we want it or not. Right?
• 4:09
  Daniel: Yeah. And a part of that, I think people are struggling with on the transformation side is there's just so many sources of information. There's so much noise around, like, what you should be paying attention to, what people are actually doing, what, you know, people are saying they're doing, what is hitting the news. Like, there's all sorts of a range of things, and obviously, a lot of that can be noise. And one of the things I've appreciated in interacting with you over time is that ability to kind of pick out some of that signal from the noise and develop some kind of structural ways of thinking about AI and, transformation in an actual real world organizational environment.
• 4:52
  In your kind of day to day, obviously, you're hearing a lot of things both from customers that you're working with, the the news cycle, the things from, you know, Anthropic, OpenAI, Hugging Face, whoever that that's coming out with things. What what does your process look like, or or how do you feel like your your rhythms day to day help you kind of isolate some of that signal, maybe kind of distill down some of what you should be paying attention to? Any any suggestions? I I think it's something on all of our minds, certainly on my mind.
• 5:25
  Mike: Yeah. Well, maybe I'm I'm very good at this because I am just the weirdo, oddball guy who stumbled into this at the perfect moment. I mean, Daniel, you know this about me, your audience might remember. I was a portrait painter for almost twenty years. I mean, from when I graduated college till 2016, I painted portraits, I owned a fine arts company and apprenticed a bunch of artists.
• 5:57
  And I only got into this because I'd heard about the DALL E model from OpenAI and applied to become one of the commercial arts beta testers, and I lucked into that. I got an email one night at dinner, I was like, Oh, wow, I just got access to this thing no one had heard of at the time. And I knew a little bit of Python and coding or whatever, so enough to fumble around and get things installed, and I fell in love with language models. I mean, almost instantly. I just couldn't believe all the new things you could make computers do.
• 6:29
  And this is before we really thought about them as chatbots or whatever. In my mind, it was like something you plug into code to just manage edge case weirdness to pick a path and make your app keep working. And context models were minuscule back there. I'm running things locally on my gaming computer. And then medical records companies start calling, and I build this book of business, and tier one acquires my company.
• 6:55
  Here's the only reason that's worth unpacking, is to say, I don't have this muddy, long history in the industry where I had opinions about everything or a solid way of thinking about it. It was all just fun toys for me. And then, I think the way I've made it through business my whole life, I've owned maybe a dozen businesses, all small, but is I've always just had a firm rule. It's like, do not concern myself with things outside of my sphere of influence. So, you know, there may be this big thing going on at the edge of the AI sphere, but the reality is like, if I if I don't have influence or if it doesn't impact, like, the way I'm interacting with my clients, if it's not, then I just ignore it.
• 7:38
  These things are not going away. I know for sure, whatever this is, it will be here probably forever. And in my experience, I'm a much better worker. I mean, can fool almost anyone into thinking I'm a smart guy when I'm using AI and I'm aligning context to whatever my work project is. I mean, it just works so much better using these tools.
• 8:02
  And I just have that conviction that's probably true for most people. So Daniel, my trick is really only focusing on what directly impacts the things going on in my day to day. And at tier one, what that is is a workforce enablement performance. Basically, not worrying so much about tools or whatever, but just outcomes and performance.
• 8:28
  Daniel: Yeah, that makes a lot of sense. Last time we talked, I looked up the date. It was 2024. Right? Yeah.
• 8:38
  Some some time in 2024. Obviously, a lot has has changed. It's impacting all of us in different ways. At that time, you we had a discussion about what's, I think, doable, scalable, etcetera. You you've, I know, spent a bit more time recently thinking about this concept of AI proficiency, and I know myself, you know, obviously, if you're doing AI things, you're searching them, you get targeted online.
• 9:10
  I'm all this all the time seeing these, like, ads for whatever it is, Harvard school, like, the engineering or what, becoming AI proficient in your business or whatever the, you know, the course or or thing is. And so there there's a lot of people trying to understand this because of, like you say, the wide reaching impact across across the workforce. So next week, our our company is doing an AI accelerator with one of our customers. I know that there's gonna be a whole range just from discussions from, of, quote, AI proficiency. But general like, other than thinking, well, I know that there's that range.
• 9:57
  Right? It's another step to go and say, well, in light of that range of proficiencies, what do I need to change for different proficiencies? How do I advance people through proficiencies? Do they even need to advance through different levels of AI proficiency? What is enough for some people?
• 10:17
  What what is too much? Right? So I don't know if you wanna help us launch into this topic. Maybe just starting with a little bit of definition of what we mean by proficiency would help us just so that we're all talking about the the same thing.
• 10:33
  Mike: If you don't mind, Daniel, I'd like to even go a step further back and just give you a sense for how and why I even started thinking about this. So if at tier one, we have most of the largest, we and three or four other companies are helping the largest companies in the world navigate this disruption. And we also see the same story playing out inside of every company. I mean, I could tell you 95% of the companies, I could walk through the timeline, and you'd say, Yep, I saw that ad. Fill in the blank, fill in the blank, fill in the blank.
• 11:07
  They've made all the same mistakes. It's kind of watching that happen sort of helpless at the periphery. As a consultant, you don't have a lot of control, and you don't have perfect ideas either. So everyone's just sort of watching this thing unfold and like, Oh, I really thought that was going to work, but that didn't stick. The big thing that I focused on early on was developing tools that help people learn and get good with AI.
• 11:31
  So I developed our solution archetypes framework where we kind of tried to pin down. We talked about that on the last episode. And then I built our super user habits index, which is basically, it's like, Hey, these are the 24 habits we see super users displaying. Then I built a coaching tool that people could use. And in my mind, I was like, I'm gonna put all this together and turn people into super users.
• 11:56
  And guess what? It just kinda didn't really happen. For some people, it did, but I couldn't help but feel like most of them, it was gonna happen anyway. And at the same time, I'm watching I'm at the same time, I'm sort of reading all the Substack articles on it, watching things unfold at companies, looking at my work history, you know, thinking through the people, looking at statistics we create or stuff in the literature. And then I think the breakthrough moment for me, I was invited to a bachelor party with some young guys who used to work with me.
• 12:31
  Never again. But I brought 200 pages of research citations and synthesis. So this is not 200 pages of research, it's 200 pages of the name of a paper, a URL, and maybe a sentence about why it might be relevant to my work just to read during that week. During the bachelor party. I thought I was probably going to need to pull away a lot.
• 12:55
  These are all younger guys. And I did, I read it all. And there were just patterns in there that answered a lot of questions for me. The breakthrough moment, with all of that just sort of happening within thirty to sixty days, that stew forming, I read a Substack article. It was Peter Yang.
• 13:16
  I'm kind of skimming my screen here for the details on it. But basically, it was called Your New Job is to Onboard AI Agents, How AI Native Companies Actually Operate. And they just had this way of thinking through the L0 through L3. So I didn't invent that. I read it in that subject.
• 13:34
  L0 through L3 classification. So L0, L1, L2, and L3. And the L0s are the people who aren't using it, and the L3s are the ones who are just making a difference across the entire enterprise, and we all know those, and then they labeled L1. L2, though, knocked me off my chair almost when they talked about the concept of a non technical builder. And they just gave a name to a thing that I just thought, Wow, this is something we need to really focus on.
• 14:08
  The idea of a non technical builder, we're all focusing on converting L zeros, people who are disengaged, and we'll unpack what that is, I think, we need to, because there's so much important stuff there. And they're trying to get people from L zero to L one, and L one is what everyone knows. It's the user. It's the one who's like, I couldn't live without this. And they kind of feel like it ends here, and then that leap, the big jump from like, okay, someone who uses AI to, wait a minute, no, this is a non technical builder.
• 14:36
  This is a person building durable solutions that emulate the company's work in an uncanny way. That jump is actually the one that now I sort of run around evangelizing, like, this is actually where you need to focus your energy. So, Daniel, I guess I unpacked all that to say, for me, I had the stew, I guess, is the way to say it, of the guy who didn't have a dog in the fight, built a company on accident, it was bought by tier one, now I'm here, I'm inside of all these companies trying to help them make sense of this thing. And then, you know, all these external inputs sort of just clashed. So now we, this is the framework we push and advertise and help ask people to think through.
• 15:24
  Sponsor: If you've been listening to the show over the past few months, you realize how transformative agentic AI is. Whether it's clot code or Hermes agent or your own custom software that you're deploying for operational efficiencies or as a product into the market. These systems are transformative and it's where the industry is going. But agents have a lot of risk associated with them. They have agency.
• 15:50
  They take action within your environment, within your infrastructure, within your systems. And that's why security teams and governance teams view it as risky to deploy these agents within high impact environments and industries like manufacturing, logistics, financial services, in the public sector. And that's why I've personally been spending my time working on a product called Prediction Guard with a great team of AI engineers. Prediction Guard is an AI control plane that you host within your own environment, whether that's on prem, hybrid, or in your cloud VPC, even air dapped scenarios. You can set custom AI policies or those that are aligned with NIST and OWASP standards, and those are then enforced automatically across every handshake, across every agent, whether that's being powered by your own self hosted models or models in your cloud environment like those from Azure or AWS Bedrock.
• 16:54
  And all of that telemetry goes back to your monitoring and observability system. I would love for you all to check out what we're doing. Schedule a demo and a call with our team to find out more at predictionguard.com/practicalai. Like I say, schedule a call today. We're happy to show you how this works and why we think it is so beneficial to the market.
• 17:18
  People already have this deployed in production, powering transformative agentic systems across a number of industries. So check us out at predictionguard.com/practicalai. That's predictionguard.com/practicalai.
• 17:38
  Chris: So, Mike, I wanted to ask you a question. As you are thinking about how you get the employees of one of your client companies moving from the L zeros up to the L threes, I want to throw another variable in, and that is the attitudes that people have, which are very varied in terms of there's always people who are excited. As you mentioned before, there are people that will naturally want to go to that L1, L2, and up to L3, but there are also people that are gonna push back hard on that. And so when you're kind of coming in and you're trying to start your process of helping them achieve that, how do you bring along the L0s that don't really want to move, or even the ones that have done a little bit, but they're also pushing back on that? Maybe they fear for their jobs or something like that.
• 18:29
  So there's that kind of L zeros and maybe some L ones. How do you get over that process as you're trying to bring everybody into this to kind of level up through those? Any thoughts? Can you kind of throw that in as you're talking about your process?
• 18:45
  Mike: Yeah. I love that you're asking me about L zeros, because the reality is we try to move an L zero. Here's the thing. The article I mentioned, the Substack article, I disagreed with 98% of what I read in that article. I just latched onto the concept of an L two, a nontechnical builder.
• 19:05
  That name resonated with me. The L0s in that article, at least at Ramp, the way I interpreted what I read, they were saying that, Hey, if you're an L0 at Ramp, of course, it's a software company. If you're an L0 at Ramp, that's grounds for dismissal. You have until this date for our system to assess you as an L1. And man, that just broke my heart, because it's kind of like, okay, wait a minute.
• 19:29
  It's terrifying. Let's back up. Let's back up. Name another technology that we would say that's true, but can you imagine you just go company wide? Hey, by the way, everyone, we want you to geek out about Smart Sheets and be great at Smart Sheets by the end of this year, or AWS, or just It's ridiculous.
• 19:46
  This is It's just another This technology like that. Yet, I think a lot of executives are scrambling and panicking, and they're saying, Hey, guess what? We're going to be AI enabled. We're going to be AI activated. Everyone in our company needs to speak AI by ex date, or there's not a place for you here.
• 20:04
  And so first of all, you remember I read 200 pages of science, and a lot of it was just about learning. It wasn't even about AI. So threat framing actually does slow down adoption. So if you back someone into a corner and say, Learn this, or there's not room for you here, they will not learn it as well as if you came to them and said, Let's figure this out together if it interests you. And I will acknowledge that there are some positions where you absolutely need an AI enabled individual.
• 20:34
  It's true. And so, there are cases where an L0's not gonna work in that role. But I think we're gonna realize in ten years, looking back at all this, that it's just not true in every role. And I think the one that hit me the hardest is my wife, probably the smartest person I know in the world. My wife, smartest person I know.
• 20:52
  And she has no use for AI in her job. She's a nurse, and she kinda just I know what she does. She pokes people in the arm with needles, and some other things, and there's medical records. Maybe someday it'll be useful to her. It's just really not right now.
• 21:08
  It would be in the way. And I started doing research on the concept of L zero, people who are disengaged, and I realized there are multiple buckets. So when we talk about an L zero, it's not a thing. So an L0 is not just someone who doesn't use AI. I found five major categories.
• 21:31
  So one is the performative user. They've been doing this. They've been nodding their head in all the trainings, the meetings. They've logged into the thing. They've submitted a chat, and they've gone about their job.
• 21:41
  And hey, they're still getting their job done. There's the disinterested. I think the moment when this hit me the hardest was when I was looking at all the research and I saw a chart that showed the adoption rates of personal computers in the office space, back when PCs were first released, and AI, and there's almost the exact same adoption curve over the same period of time. It looks like it's happening at the same rate. And when I saw that, I thought, how silly is it to think back to that?
• 22:14
  And one is in a panic because the people in their company haven't figured out personal computers now. It took care of itself because everyone wound up with one, and the same thing is happening with AI. Everyone went home and just created a chat GPT account or whatever, it's free. This is not true with your CRM. This is not true with smart.
• 22:32
  Like everyone didn't run home and create a smart sheet account because you know, like they just gotta have one. And so, you know, but the disinterested are still detached there. Okay. Here's another bucket. The too busy.
• 22:45
  I could give you 10 names at tier one. We almost don't want them running around trying to figure out AI because they are just so critical in their role. They're so good at what they do. It's like, it's like a, it's a don't break it or don't touch it. It ain't broke type role, and you know, who cares if they're using AI or not?
• 23:01
  And I think that is also really hard for the people who are here in communication, Use AI or you're out of here, and then they can look around and see the people who like, very often, it's some of the highest performers in the company who don't really care about it because they're in a rhythm. So those I kind of push past, but the next two, these are still buckets in L0. These are people who aren't using it, really matter. And there's something you can learn from these people. And I think when you say like, Hey, you're going to be doing this.
• 23:28
  You're going to be using AI at this company. They will just zip their lips, and they will nod their head, and they'll say, Yes. But here's the thing. First of all, we have the job fearful. And Chris, I think you this, you might have mentioned this earlier, job fearful.
• 23:42
  What do we do? What do we do about the job fearful? First of all, very often these people don't really understand how the tool work, and they're under impression that if they use AI to do their work, it's going to learn how to do what they do and replace them. And that's almost never true. I mean, just if you kind of know how these things work, I mean, maybe self driving cars.
• 24:01
  I could think of a few examples of like, the AI is actually learning from watching us, but my vote doesn't really impact whether or not it's gonna replace that profession someday. So I think with the job fearful, there's an opportunity to help them see like, Hey, you might be in the middle of a self fulfilling prophecy here. If you continue to push back against this, you might lose your job, but it won't be because the AI replaced you, because Joe, who's willing to use AI, will replace you. So that's one way to address an L0, but another one, and this is actually six out of 10 people that we talk to, and I've seen this number in research hover around sixty percent in more than one place, are the quality disappointed. This is where I think every executive's ears should turn on.
• 24:44
  They should lean forward in their seat, they should realize, do not dismiss these people. If they are resistant to AI, there could be some very valid reasons why. And I met a person at a company who, her complaint was, It's not good enough for what I have to produce for my work. And she kind of showed me, and she's just like, Humans do this better. And it was a marketing role.
• 25:07
  It was copy generation. It was some other stuff, and she was just right. And it may not stay true forever, but no one had ever actually just, no AI expert had ever just sat and listened to her, and I walked away from that conversation with just a brand new perspective on how much value there is to mine from the group of people who say, It's not ready to do this work yet. Or either it's not, or they need better tools or better training, but you can't really do that until you listen first. And I also don't think about it like a ladder.
• 25:39
  I don't think an L1 is better than an L0, and I don't think an L2 is better than an L1. Actually, I don't see it that way. I just see it as kind of a division within an organization for how different ways of thinking about the toolkit.
• 25:56
  Daniel: And maybe one follow-up question on that. I'm wondering, you mentioned maybe part of the, part part of the stumbling block here is the quality disillusion or or however you put it, you know, getting into these tools. I I sometimes I also wonder about the way in which, like, I would put myself into the category of the AI people. Like, yeah, I'm building an a product. Right?
• 26:21
  How much of this is us providing AI in a way that is just another thing that they have to add, like a different tool that they have to add in their workflow versus something that actually works in their workflow? And what I mean by that is, you know, if if I'm, to to give an example, you know, we're working with, an organization now. They have teams of translators. We have essentially a team of digital agents that helps translate with a with an agent harness, they use existing tools. They go through the, you know, drafting and quality checking and post editing.
• 27:01
  They do these cycles. But we we don't actually say, hey. Go to this different interface that you've never used and do things the AI way. The the sort of agents work and then stuff pops up in the translation management system that they already use. Right?
• 27:20
  And so what they see is, yes, there's change. Right? But it's actually like allowing them to do the thing that they want to do with the tools that they're using in a in a way that actually they they want to do that more. Now that may not always be possible. Like, there may need to be a shift of, like, how people you know, the interfaces that people use to do that.
• 27:45
  But I don't know. Do you you have any experience with that? Like, how much of this is we're asking people to use yet another tool, and they've already got tools that they like whether they're AI or not. Right? And how much of it is the actual AI output?
• 28:02
  I I don't know if you have any sense of that.
• 28:04
  Mike: No. I I hear I think I hear the the sort of the spirit of the question. And so this is where I land, you know, when it comes to, you know, who do we train to do what, and what do we give them, and how do we, you know? And honestly, at this point, I just feel like that is part of the noise to me in all the conversations. The main thing it all traces back to is executives are complaining, We're spending too much on all of these licenses.
• 28:29
  We don't even understand what we're buying. I can't measure. I mean, yeah, I see this or that anecdotal piece of data, but I can't measure the real impact. What is the value of this investment compared to the cost? And so we just say like, Hey, woah, let's zoom way out.
• 28:46
  Let's zoom way out, and let's think about where we should actually focus. I mean, if we've already said, Okay, L0, we understand some people just really aren't gonna be using it maybe ever, but definitely for a while. Okay? Let's not even think about them right now. We got the L ones.
• 29:02
  This is kind of who you're talking about, Daniel. It's the people who like believe it or not, it's the people who would say, you could pry it from my cold head hands. I could not live without this. You'd think, That must be like an L3 or whatever. No, no, actually it's just people using it, but we can look at that and we don't really understand the value there.
• 29:19
  I mean, they don't want to lose it, but they're just doing their email faster or they're interpreting technical documentation. But the company can't look at that and say, There's $4,000,000 we saved or whatever. So we say, That's noise too. That's going to happen. It's going to take care of itself.
• 29:37
  I wouldn't focus too much on that. When we talk about L2s, so at tier one, we have a process for assessing and identifying quality L2 candidates. Now, Daniel, I'm gonna work my way backwards to your actual question from here. So the idea is with an L2, this is a person who you might think, Oh, great. How do you assess them?
• 29:55
  So they have some aptitude for AI. No. We really don't even kind of start with any sort of technical aptitude assessment. What we care most about is company DNA. Do they understand the work in an uncanny way to the company?
• 30:07
  Do they know which cell of a spreadsheet to fight over? Do they know how to deliver the work in a way that the company wants to deliver? Is their work style uncanny to what we want? That is absolutely necessary before we screen anyone for L2 training candidacy. And here's the reason why.
• 30:28
  Right now, the wrong people are in the driver's seat. We call them the AI excited. The AI excited are the ones who are getting all the attention in the company. They're running down the hall screaming words like nano banana and methos class models, and everyone's just like, Well, they must be the person who should be building things, when in reality, what we see, Daniel, is these people who maybe by definition are often distractible and not very focused on the work, don't know exactly how to align an agent to work in a way that is absolutely uncanny to what the smartest and best and most qualified SMEs inside of that function would call good or accurate or what we want done. And so we say we want to start with, it doesn't need to be this me necessarily, but it needs to be someone who, in the Venn diagram, can replicate the work the company needs done in the way they do it, that honors the brand, and has the aptitude to learn these models, the interest.
• 31:34
  And though, Daniel, we only want one of those on each team. We find that it is not a one plus one equals two thing. Too many chefs in the kitchen, if you have two L twos on a team, they are not necessarily as good as just an L two, a few L ones, some L zeros. And we wanna make sure every L two, every non technical builder has an L three within arm's reach so that they can double check the work. The L2 can make it uncanny, but the L3 can make it scalable, durable, make a thing that won't break, or doesn't break laws, or violate governance policies.
• 32:11
  And Daniel, this is like your whole world. Right? So this is kind of what we tell. And the other nice thing about thinking about it this way is if you can see, well, we really only want one L2 inside of each high performing team. That also means we don't have to think about as many expensive licenses, and we can focus training on qualified candidates.
• 32:34
  And so if we're just letting the L1s happen naturally, we're saying, Okay, L0s will deal with them someday, but what can we learn from them now? And L2s are really the ones building the machines that we want to work in a way that honors the work. And think about it, how many tools have you used that's like, Who built this? Where did this come from? You know, these things tend to last for years or decades, and it better work right, you know?
• 32:59
  Anyway, particularly when you're building something that works at scale.
• 33:04
  Sponsor: If you're listening to the Practical AI Pod Podcast, I'm guessing that you value practicality, not just the hype around AI, which is why I think you should check out the Midwest AI Summit. This is an amazing event. I'm gonna be there this year. It's happening October 15 in Indianapolis. This is an event like no other I've been to.
• 33:26
  There's actually an AI engineering lounge where for free, you can go up and get expert advice from practitioners and get feedback on architecture, your design, your agentic harness, whatever you're looking at, you can get feedback on in real time in between amazing speakers that are on the main stage. So don't miss this event again, October 15 in Indianapolis, and you can use the code Practical AI 20 for 20% off. So go to midwestaisummit.com and grab your ticket today. Use code Practical AI20 for 20% off. Midwestaisummit.com.
• 34:08
  Chris: Mike, you got me pretty interested, and I'm really thinking about this kind of L2 process that you're describing. And I want to rephrase a little bit in my own words and kind of finish with a question from there, and that is it feels from what you're describing there that that L2 kind of has the knowledge of the value that the company is producing locked in their heads. And so they may not be the L1 that's running down the hallway screaming, Hey, AI is cool, we should do it, But they're the ones that are fundamentally and historically bringing that core value into the products and services the company's trying to produce, and that's kind of how it sounded to me. Whether or not they're into AI, they have that, So it seems like you're trying to get kind of the L2s to be able to best use these capabilities to enhance their ability to drive value creation in the company. Is that a good way to interpret that?
• 35:16
  Depending on what your answer is, can you kind of give me a course correction, or can you kind of go down that path and explain it more? Because I am pretty keen on that idea.
• 35:27
  Mike: Yeah, great. So, you know an L2 is a good one when the SMEs in the area don't complain about whatever AI tool they created, whatever its output is. You know that they are a good L2 candidate when they can align an agent so that when it creates outputs, it feels familiar to the type of work we do, and doesn't require a whole bunch of babysitting and handholding. But also, to back up a little bit, if you think about what that actually means to an organization, and in a minute, I'll give you an example of a project I'm on right now that I think would help you see the value of this instantly. A good L2 is also not only building tools that can address just repeatable work at scale, they're also converting tacit knowledge to documented process.
• 36:30
  Because once an AI model is aligned, now you have documentation. Maybe it's in code, but it is documented. And so that's a big problem in every industry right now. It's like, Oh, you know, the aging workforce, and what are we gonna do? Got don't get hit by bus guy over here, and if that's me, if we lose them, the plant shuts down, you know, kind of thing.
• 36:55
  And so I think in my mind, that's how you know you have an L2 there, that you don't have to worry about the tool they're building and whether or not it's gonna work well. But here's the other piece of it. And really, executives lean forward in their chair at this part of the conversation, because when we say, If you're wondering about the value of this investment, let's stop thinking about this as AI work. Really, the work of yesterday is going to look like the work of tomorrow. Believe it or not, the more things change, the more they stay the same.
• 37:33
  And I know it feels like everything is shifting beneath our feet, but the reality is we will be doing the same things before tomorrow. Whenever I talk to AI teams, it feels like what they want to build is agents. What they want to build is tools. But if you actually go look inside of the work at companies, they don't, they're not thinking about AI. They're thinking about this particular problem in front of them.
• 37:53
  So with one of my clients, let's just say one of the top four pharma companies in the world, they were sitting on a stack of, I can't remember how many thousand documents, they needed converted to look from to look like this to look like this. And I probably can't say too much more about that project, but it was a $4,000,000 job. It was a $4,000,000 job. They knew exactly how much it would cost to convert each one, and it requires SMEs. And an L2 took a look at it and said, This feels like a Claude skill.
• 38:25
  Had the Claude skill built within three hours, we drug one of their documents onto that skill, out the other end, squirted almost exactly what we were hoping these things would look like at the end of the process. Of course, everyone's jaw hit their desk. Wait a minute. You're telling me that was gonna be $4,000,000 It needs to be done. And now our Claude, which we've already paid for, you you just drag them on, wait, you can also run a thousand concurrently, and this whole thing could be done in hours?
• 38:56
  Like, wait, what? That is measurable, You know? And so we we kinda push people to thinking toward like, you know, you you're not gonna find out the value of this investment based on how many tokens people are spending or, you know, like, you're gonna find it when you start intentionally forming teams with a couple of strategic people in there who know how to spot an AI opportunity and just make the headache vanish. We call it doing their laundry. And so this is where the real money is, and this is where the real savings is.
• 39:28
  And Daniel, I know you know about this because I've heard you kinda almost complain about it. Like, you know, it's like, this stuff's not fun, but it's like, it's real work. Chris, is that helpful?
• 39:38
  Chris: It is. No. That that helps frame it very well for me. I appreciate that.
• 39:42
  Daniel: Yeah. And I I would be curious of your take on this, Mike. I think you're the right person to ask for a critique on some examples that I've I've been using even personally. But I've been I've been trying to use this example of, you know, how in, like, leadership retreats, you they always used to show, like, the the f one pit stop and how it advanced from, like, twenty seconds to two seconds. Right?
• 40:08
  And how that happened was, like, everybody knew knows their job in the pit lane. Right? And one guy's job is just to move the tire from here to there, and that's all they do. Like, that's their full responsibility. And it almost seems like in some of these use cases that I'm seeing at least, those kind of very targeted jobs or outcomes.
• 40:33
  Right? But like you're talking about this document from here to there. Right? That that needs to be done, and that is is maybe a good candidate for, however you frame it. A use of AI, an an agent to take care of it.
• 40:49
  How how are you word that? Like, that's a that's a thing. And and it kind of in my mind, it then it it doesn't remove the human, but it it actually kind of elevates, in some cases, the dignity of the human from being the person who's like, all you're gonna do every day is move the tire from here to there or to, you know, do this task. And certainly, recognize there is legitimate, you know, jobs will shift. Right?
• 41:18
  But, and, you know, people will will struggle around that. But it seems like now kind of the human you're not framing your team of humans as, hey. Don't go don't get out of your lane. Just move the tire from here to there, to actually being able to leverage AI in these individual tasks and you kind of coming into the orchestrator mode or the team principal or the the strategist mode and actually thinking about outcomes. Well, what what is the outcome I want here, and what are the individual things associated with it?
• 41:52
  Many of those which can be individual tasks that that AI accomplishes. I don't know if that that rings true at all. I've I've been trying to think about, yeah, how, people are very used to thinking now, I I think, or many people are very used to thinking of the one to one interaction between a human and, AI tool, let's say, or a chat interface, and maybe less familiar with this or or familiar with this way of thinking of maybe one human is actually aided by multiple different instantiations or manifestations of AI that help them work towards towards outcomes. But I don't know. Any critique on that rambling example of kind of the the the f one pit stop and some of these tasks?
• 42:43
  You know, what are the tasks in your in your, pit box that need to be accomplished, and what what of those are good candidates for AI? How do I think about managing this set of AI workers or tasks or whatever you however you might frame it?
• 42:59
  Mike: Yeah. One of the one of the things that popped in my head while I was listening to you, Daniel, which I always enjoy doing, is I'm very often thrown into rooms with people who live on a spectrum of attitudes. One of the big ones is just all the fear wrapped up in this. And I kinda always open with, hey, guess what? If you're worried about losing your job to I think there's thunder in the background.
• 43:26
  I don't know if you can hear that.
• 43:27
  Daniel: Yeah. It's definitely thunderstormy in the Midwest
• 43:31
  Chris: It's giving
• 43:31
  Daniel: the right answer extra trauma
• 43:33
  Mike: right there. Yeah, that's the fear part. No, really. When I go in a room, and I know there are people who are dwelling on that, hung up on that, I you know, I just open with, Guess what? You are losing your job.
• 43:46
  Your job is going to change, and that's true for all of us always, whether or not AI was invented or not. I don't know how many careers I've had, and I think the average person has seven. And even if you stuck with one, your job would change year over year in time. And if someone is uncomfortable with the concept of change, guess what? You're not cut out for the workforce.
• 44:07
  I mean, the reality is if my daughter was scared there's a monster under her bed, I wouldn't go and say, There's no monster under your bed. I'd say, You're stronger than a monster. Monsters aren't real. It's kind of like that in this job. It's like, Hey, adapt or die, and I don't think you're gonna die.
• 44:31
  And maybe some of this, it might sound a little heartless, but it comes out of the research, is that disruption is always, it turns out in hindsight, it was not as hard to adapt to as you thought it was going to be, and you adapt more quickly than you thought you could. And so, this is just documented truth. But it never feels that way. I think I'm just the weird bird who really likes the idea of uprooting my entire career and just trying something new. You know?
• 45:06
  I just have done that so many times, and I almost enjoy it. But then, I think the rest of what I heard you talk about, Daniel yeah. I addressed that. But remind me, what else? What was the the
• 45:19
  Daniel: other I think it's this idea of one element of the change, I think, is understanding that maybe your your position, which might be consumed by this task right now, might not be just come into work and do this task, but it it sort of elevates more to an outcome kind of creative orchestration mode where you're actually orchestrating work that needs to be accomplished. Like you say, the same functions, the same roles, like the same outcomes need to be accomplished in your in your business. Right? But, but, yeah, it it's it does take a different level of thinking to think about, well, how do I orchestrate these things to get the outcomes versus how do I do this task? Because I know how to get the task done.
• 46:09
  Yeah,
• 46:10
  Mike: I hear you. Maybe part of why I struggle with just rapid firing an answer to this is my mind darts back and forth from the hundreds of work environments and situations I've had to kind of inspect and solution for. Frame
• 46:28
  Chris: it for a moment, because I think I see where Dan's going, and I think you are the right person to answer this, and that would be if you go back to that analogy of the F1 pit stop that he was describing and the nature of the jobs that everyone had there, you're kinda looking forward. So we're kind of, what is our expectation of the future? At least I think one of the things that probably the three of us would agree on at some level is the fact that the human that's in that process, it may have these AI agents that are able to take over some of those very specific jobs along the way, and you're almost becoming the pit stop manager or orchestrator overall. So you're still doing the pit stop because that's the value of what your organization's doing, but the nature of the humans may change to better do it, to do it faster, and to grow rather than be kicked out of rather than loss of job. It's a change of job that's actually more valuable in a lot of ways.
• 47:30
  Could you talk a little bit about what that future might look like for those individual employees that are adjusting their pit stop job, but maybe moving a little higher in the abstraction layer to owning that whole process where they are valuable as humans and yet they're taking full advantage of the agents. Any thoughts, as you take companies through the process, how you would do that and what your expectation of the future will be on how that progresses.
• 48:02
  Mike: I don't think you're going to like my answer.
• 48:04
  Chris: It's all good.
• 48:06
  Mike: I've decided to stop worrying about it. I don't even think about it anymore. Maybe it's just because I don't fully understand the question. I think I do. I spent probably two years inside of all of these organizations trying to lead and organize and implement enablement and activation campaigns.
• 48:29
  And in the end, it just kind of felt like I was just watching water run down a river. It was gonna go down the direction it was gonna go. The river is just winding the way it winds. I think I'm always just searching for where can I actually make an impact? Where can I actually activate change in an organization that is measurable, something that we're all glad we did after we're done doing it?
• 48:54
  And when I come in and we try to just really analyze and think through, well, what's the impact of this gonna be after we do that? In the end, what it's gonna be is what it's gonna be. And people tend to adapt to these tools very quickly once they've used them once or twice and seen the result. And so in my mind, it's kind of like, I'm gonna do everything I can to get them to do that. But beyond that, if I can't get them to, I don't care.
• 49:19
  What I care about is, can I find that l two? Can we get them equipped and activated? And and and synergizing that's a better word than that. Effectively cooperating and contributing inside of their team in a way that like, I wanna know the difference before and after. And and so it's not just that, you know maybe the pit crew thing was a distraction for me because I was I was focused on like, well, it's faster now, and it drives me nuts when people say, I want things faster.
• 49:53
  At tier one, we say, like, faster's not always better, especially when it's a it's it's a system with a whole bunch of parts, and you've just created whiplash and angst for everything around that fast piece. You know? Like, nothing else is ready to go that fast, so that's not necessarily better. Cheaper? Sometimes that doesn't even matter.
• 50:11
  Believe it or not, a lot of the companies, that's that's not like the main thing we're talking about. Like but new emergent types of work. Interesting. So what can we do now that human minds just really couldn't do well before? And equipping these language models inside of really clever harnesses.
• 50:29
  So again, it's kind of like Daniel's world. Woah, new kinds of work. And so what's that mean for the product, or this department, or function, or whatever it is? So I guess I put that whole concept just right outside the periphery of what I can control, and so therefore, not something I worry about and I'm sort of done thinking about.
• 50:57
  Daniel: Yeah. That that makes sense. I actually do like that answer. I did.
• 51:00
  Mike: I did too.
• 51:01
  Daniel: I I think there there there are a lot of people and may maybe this is, I don't know, a a little bit too personal, but but certainly, like, a mode that I think all of us as humans get in is not being present in the moment and thinking about, of course, the past, you know, dwelling on that, but also the future and the all the stories that we tell in our head about how things might go. Right? Which won't there there may be elements of that that are true, and it's not of course, I don't mean that we shouldn't innovate and look at new ways of doing things. I just mean we often dwell on eventualities in the future that will that will never take take place. And there's dynamics that are happening that we need to dig into and think about how they matter in in the moment, which I I I really I really like that that perspective.
• 51:55
  As as we close out here, Mike, after kind of working across these different levels of proficiency, and I know also you're thinking about structured ways of thinking about the the actual things that can be done by AI, the outcomes, that sort of thing. What what's on your mind, you know, today as you're going into your next meetings as, like, the the the kind of challenges or the things maybe like, you had done this work to parse through all of these this research and and work on AI proficiencies. Right? What's that next kind of or is there a next kind of area in your mind where you're like, I don't I don't totally have a grip around this bit of it yet, but I'm really curious to dig in more and see if I can see if I can parse through some of it. Anything that comes to your mind as as we close out here?
• 52:51
  Mike: Yeah. I think I think the thing that's been most exciting to me as an idea or a rallying cry when it comes to the rank and file inside of organizations, or with leaders who look at me and say, Where is their energy? Where is their momentum? One idea that I've had lately, and I've seen pay off over and over and over is the reality inside of most of these massive organizations, mostly I'm talking about big business now, is that everyone's got a toolkit in place. Like, if you're, you know, Eddie Punch Clock, Procter and Gamble, you probably have access to one, two, or three AI tools, and they don't work well together, and they don't necessarily work well.
• 53:39
  And it is very easy to try to use them and just see it doesn't work super well for the thing that you need to get done and give up on it. And I think the thing I'm realizing is that there is massive, massive value in the people who are willing to lean against the brick wall and push and push and push. And believe it or not, you'll start to find ways to use these tools that are hugely informative in spite of the fact that they're so burdened by necessary governance and, you know, rules and restrictions, and why can't we turn that connector on and why can't you know? But, you know, when Claude tells you something's not possible, you know, ask it to get creative and, you know, and or, you know, think through, you know, alternate approaches. I think creative solutioning is going to be really, really valuable over the next two years, three, four, five years before all the wrinkles are ironed out, who knows, Terminators or whatever comes after that.
• 54:47
  But I think, yeah, the thing, Daniel, that I've been preaching lately with my trusted colleagues and with different leaders and organizations is do your best to to encourage that type of thinking and behavior. Like, okay, I know they aren't perfect. Figure out what you can get done, though. So
• 55:09
  Daniel: Yeah. That's that's, I think, a great rallying rallying cry to to end with. Appreciate you joining us again, Mike. We look forward to having you having you on again, to to learn from you in hopefully not too long again. But, appreciate you taking It's been a great conversation.
• 55:29
  Thank you.
• 55:36
  Narrator: All right. That's our show for this week. If you haven't checked out our website, head to practicalai.fm and be sure to connect with us on LinkedIn, X, or Blue Sky. You'll see us posting insights related to the latest AI developments, and we would love for you to join the conversation. Thanks to our partner, Prediction Guard, for providing operational support for the show.
• 55:57
  Check them out at predictionguard.com. Also, thanks to Breakmaster Cylinder for the beats and to you for listening. That's all for now, but you'll hear from us again next week.