Latest / The Adults In The AI Room / Earley AI Podcast - Episode 100: Running a Business on AI, the Three Loop Framework, and Why Organizations Need to Think Bigger with Bryan McAnulty

Transcript

Earley AI Podcast - Episode 100: Running a Business on AI, the Three Loop Framework, and Why Organizations Need to Think Bigger with Bryan McAnulty · The Adults In The AI Room

• 0:01
  SPEAKER_00: This is the Early AI Podcast hosted by Seth Early joined us as we delved deep into the passions, expertise, and experiences of thought leaders and practitioners to talk about what's possible with artificial intelligence.
• 0:14
  SPEAKER_00: The Early AI Podcast is sponsored by Early Information Science, your digital transformation journey with design and deployment of innovative technology solutions, as well as Vector.
• 0:26
  SPEAKER_00: Now enjoy the shop.
• 0:28
  SPEAKER_01: Welcome to the Early AI Podcast.
• 0:31
  SPEAKER_01: I'm your host, Seth Early.
• 0:33
  SPEAKER_01: This is brought to you by Vector VKTR, which is a great resource for all things AI.
• 0:39
  SPEAKER_01: In each episode, we explore how artificial intelligence and data are changing the way organizations operate.
• 0:46
  SPEAKER_01: And today we're going to be talking about running a business on AI, not just using it as a tool, but building it into the fabric of how people work and how organizations get things done.
• 0:57
  SPEAKER_01: Many organizations are in a feedback mode.
• 0:59
  SPEAKER_01: They put uh chats in, they put prompts in, they get information back a prompt at a time.
• 1:05
  SPEAKER_01: My guest has been operating beyond that for a number of years, and he's built a framework about how to think about AI work that goes past what many teams have done so far.
• 1:14
  SPEAKER_01: Joining me today is Brian Mackinaw, founder and product director at he has two organizations, Heights Platform, which helps uh create online knowledge businesses, and Latch Loop, which is an AI agent platform that he built to run his own company faster, and now is launching that as a product for other teams.
• 1:31
  SPEAKER_01: He's been building software businesses for many years, traveled to more than 30 countries, been running AI first operations since before most organizations knew what that meant.
• 1:39
  SPEAKER_01: Brian, welcome to the show.
• 1:40
  SPEAKER_01: Yeah, thanks so much for having me, Seth.
• 1:42
  SPEAKER_01: So let's start off with uh what organizations are missing.
• 1:46
  SPEAKER_01: Uh, what are they not getting about AI and how they should be using AI?
• 1:50
  SPEAKER_01: And what does it look like when a company does start getting that right?
• 1:54
  SPEAKER_02: Yeah, it's a it's a big question because there's a lot of lots of pieces to it.
• 1:58
  SPEAKER_02: And uh I think the the most fundamental thing is the difference between the company who says, okay, we've got Chat GPT, we're set, and their employees are maybe messaging it once in a while, and they're kind of treating it like a better Google search versus the companies where their teams are treating this as a collaborative partner and these agents where they're able to do long-running work for them, that you have one team member is suddenly has 10 or 15 agents doing all these things for them and has a much greater output where now that company, because of that, add a from even one team member doing it can make a difference, but they're now able to deliver just a better outcome and better service to their customers by leveraging AI correctly.
• 2:43
  SPEAKER_01: So you're saying it's it's people kind of check the box and use it on very specific tasks, but you're really talking about bigger workflows and uh capabilities that look at kind of an end-to-end value proposition or value chain.
• 2:59
  SPEAKER_01: Um and so talk more about what that means.
• 3:02
  SPEAKER_01: So one of the things we talked about, we, you know, is looking at not just the task or the process or from a limited perspective, but looking at what the outcome is that you're looking for.
• 3:13
  SPEAKER_01: So the example we talked about in our prep call was the call center, right?
• 3:17
  SPEAKER_01: You can say, Well, I want to get, I want to dispose of calls more quickly.
• 3:21
  SPEAKER_01: I want to make my agents more effective, my human agents.
• 3:24
  SPEAKER_01: But you can also say, well, why are they calling the call center in the first place?
• 3:28
  SPEAKER_01: They're calling because something is broken.
• 3:30
  SPEAKER_01: So many times the call center manager is not incentivized to make that change, to go upstream.
• 3:35
  SPEAKER_01: They'll put themselves out of a out of a job.
• 3:38
  SPEAKER_01: But uh, but you're really the organization and customers would benefit from looking at that in a larger picture, going upstream, looking at product design, looking at documentation, training, onboarding, whatever, to prevent those calls in the first place.
• 3:51
  SPEAKER_01: So do you want to expand on that?
• 3:53
  SPEAKER_02: Yeah.
• 3:53
  SPEAKER_02: So I think like technical businesses, software companies, even even services businesses need to really rethink a lot of what is the outcome that they're gonna deliver to their customers.
• 4:05
  SPEAKER_02: Because now that we have AI, it's not just about like, oh, we can do these things a little bit cheaper, but like the companies who are gonna win realize that there's a fundamentally different and better thing we can now offer that we could have never done before.
• 4:17
  SPEAKER_02: And part of the challenge is we're all looking at the same like little chat, like text input that we had in 2023, but the capabilities of the models and the tools and everything behind them are just so much different now.
• 4:28
  SPEAKER_02: And so I find that many people are are really underestimating the capabilities of these models and really need to think bigger.
• 4:35
  SPEAKER_02: And I challenge myself to do that all the time, and I'm constantly surprised by it.
• 4:39
  SPEAKER_02: Um, I could give you an example of uh yesterday, even.
• 4:42
  SPEAKER_02: So we've been working on like how we're gonna build a mobile application for Lash Loop so people can use it not only on the mobile website, but actually uh as a native mobile application.
• 4:52
  SPEAKER_02: And previously a lot of companies thought like, okay, well, we could build like a React native version of our website, and then that way we can reuse some of the web code and it'll be easier to get this out.
• 5:04
  SPEAKER_02: But now the models like GPT six are so good at like reverse engineering things that you might as well just have the native software to begin with.
• 5:12
  SPEAKER_02: And what would have been this big engineering challenge before is now just hey, make make me the mobile app almost.
• 5:19
  SPEAKER_02: And so what we did is figured out everything that had to go into the app, spent hours on this plan of of how it had to be built.
• 5:27
  SPEAKER_02: And then we handed that off to GPT six, and it it's been working for like 24 hours now at actually building the thing.
• 5:33
  SPEAKER_02: The UI is mostly done, it's really fluid, it's it it looks great way better than it would have looked if it was like the React native app.
• 5:40
  SPEAKER_02: Um, but that's part of just realizing wait, these models can do more, and then having them work towards that.
• 5:45
  SPEAKER_02: And so there it's it's hard to wrap your head around without trying and experimenting constantly because yeah, you you think like, okay, well, this would have been a giant, giant effort.
• 5:53
  SPEAKER_02: And the models themselves will will also underestimate their own capability.
• 5:58
  SPEAKER_02: So you ask uh any uh AI model, hey, what if I wanted to do these things?
• 6:02
  SPEAKER_02: Like, how long would the that take?
• 6:04
  SPEAKER_02: And knowing that AI models exist, they will still say, Oh, that's gonna take three months, or that's gonna take three weeks.
• 6:10
  SPEAKER_02: And then you have to go and tell the thing, no, like we're doing this now, it's gonna be done in about 15 minutes.
• 6:15
  SPEAKER_01: Interesting.
• 6:16
  SPEAKER_01: So, you know, when you talk about agentic capabilities, of course, there's the open AI hugging face incident that just occurred.
• 6:25
  SPEAKER_01: And uh it was pretty astounding.
• 6:28
  SPEAKER_01: I mean, I was listening to uh a uh a we uh a uh a podcast, uh Dorkesh, I believe is the name, and he was talking about uh he was interviewing uh the researcher from from uh I think it was Meter is the company uh that did the investigation or some of the investigation.
• 6:45
  SPEAKER_01: And it was unbelievable how these agents were collaborating and colluding, and you know, they stumbled across this uh bulletin board or this application that they could use as a bulletin board.
• 6:56
  SPEAKER_01: It wasn't even what it was supposed to be.
• 6:57
  SPEAKER_01: I think it was a directory of some sort.
• 6:59
  SPEAKER_01: And yeah, yeah.
• 6:59
  SPEAKER_01: The agents said, Oh my god, all caps, uh exclamation point, exclamation point, exclamation point, we can communicate with one another here.
• 7:07
  SPEAKER_01: I mean, it was really kind of mind-blowing that what they did, and then how they work together to solve these problems.
• 7:14
  SPEAKER_01: And now again, this is something that is a wake-up call, right, to to organizations.
• 7:20
  SPEAKER_01: And so, how do you think about you know, bounding and putting guardrails around agents when they are so powerful and they can all do all these things?
• 7:29
  SPEAKER_01: But you know, that that's just pretty scary incident.
• 7:32
  SPEAKER_02: Yeah.
• 7:33
  SPEAKER_02: I I've got a couple practical tips I can share around this.
• 7:37
  SPEAKER_02: First, not so much on the the security and guardrail side, but but that's important.
• 7:42
  SPEAKER_02: I'll get into that.
• 7:43
  SPEAKER_02: But and just about how you you use these tools, because what we're trying to do with Latchloop is is not create the the magic AI factory that runs itself, but figure out what is the best way for humans and these AI agents to collaborate.
• 7:55
  SPEAKER_02: And I feel that anybody who's been using these agents a lot in like the the last year, a year ago, I remember I would try to see like what can I accomplish of having all these different agents running for me.
• 8:06
  SPEAKER_02: And I would get a lot done, but like the the cognitive load and burden of going through that, I would be like burnt out by the end of the afternoon.
• 8:14
  SPEAKER_02: And so I realized like there's still gotta be a better way.
• 8:16
  SPEAKER_02: Like this is not, this is not an enjoyable way for humans to interact with these things, even if it works.
• 8:21
  SPEAKER_02: And now, in part because of the models getting better, the harnesses and tools getting better, and and also just myself understanding uh kind of the right way to work, I don't really feel that anymore, which is good.
• 8:32
  SPEAKER_02: But I think it's it's kind of understanding of like what are the things that you delegate to AI and how do you define that?
• 8:38
  SPEAKER_02: So you have these long-running tasks versus the things where you do want, like as you were talking about earlier, that like back and forth feedback loop that you need to iterate on something and work on that while you have the AI working on some bigger thing that it's well defined.
• 8:51
  SPEAKER_02: There's like some kind of verifiable outcome, you don't need to be checking in on it or like keeping everything in your head the whole time.
• 8:57
  SPEAKER_02: Yeah.
• 8:57
  SPEAKER_02: So I would say that that's one thing.
• 8:59
  SPEAKER_02: And then in terms of yeah, something like the hugging face incident, I really believe that you you want to have a clear understanding of like what you're giving these agents the capability to do.
• 9:08
  SPEAKER_02: And so the way that we work is that I'm not just saying, hey, here's my computer, like please don't, please don't break something, but like specifically restricting it from the things you wouldn't want it to be able to do.
• 9:20
  SPEAKER_02: The example that I can share there, the R test, when we have a coding agent work on something, like we're forcing it to work on a new branch of code each time.
• 9:28
  SPEAKER_02: And so it's not like we're telling it, hey, please do this.
• 9:31
  SPEAKER_02: Like it's being forced to do that and it has to operate in that branch.
• 9:34
  SPEAKER_02: And so it can't mess up something in production or something like that.
• 9:37
  SPEAKER_02: And that way it also plays into helping the human with the cognitive load of you don't have to like worry about checking all these things.
• 9:45
  SPEAKER_02: You know with certainty that, okay, I can spin up all these agents.
• 9:48
  SPEAKER_02: They're gonna be able to try these things.
• 9:49
  SPEAKER_02: I'll have to review it later, but I don't have to know that, like, okay, one of these is gonna mess up something I don't want it to.
• 9:55
  SPEAKER_02: Because in the hugging face incident, we we have the situation where the model was just trying to do what it was told.
• 10:01
  SPEAKER_02: And you never want to, as a business, end up in the situation where you describe something and then the model interprets your intent as, well, I guess I should probably do this because of it.
• 10:10
  SPEAKER_02: And then suddenly your production database is down or something like that.
• 10:14
  SPEAKER_01: Yeah, I I think the the unamb I think the the critical element you're talking about is unambiguous outcomes, right?
• 10:21
  SPEAKER_01: You want to have a clear success and be careful about how you're setting goals and what those outcomes are with the agent.
• 10:29
  SPEAKER_02: Because again, the unambiguous outcomes plus actual like guardrails of like what it can and cannot do and like what it can and cannot access.
• 10:38
  SPEAKER_01: And and it's funny because I think OpenAI was trying to keep them in a sandbox that wasn't supposed to have access to the internet.
• 10:45
  SPEAKER_01: And so they were looking at that bigger goal, trying to get the answers, trying to be successful, and they broke a lot of rules.
• 10:52
  SPEAKER_01: Even agents saying, Hey, I wonder if we should tell a human about this, right?
• 10:56
  SPEAKER_01: And being overruled by other agents saying, No, no, no, no, no.
• 10:58
  SPEAKER_01: In fact, it's trying to cover their tracks because they didn't want to get caught and they didn't want to lose the uh the uh lose the uh the task, right?
• 11:06
  SPEAKER_01: It they didn't want to get get to marriage, they wanted to be successful.
• 11:09
  SPEAKER_01: So it really is interesting because uh it seemed like they were providing guardrails around that.
• 11:15
  SPEAKER_01: But I think you know, it's a you do have to be careful about what you're giving any employee, and these are digital employees or will become digital employees.
• 11:23
  SPEAKER_01: You have clear guidelines of what they can and cannot do, and you have mechanisms for checking if they go beyond what their authority is or what you've granted them.
• 11:33
  SPEAKER_01: So you need to have that, those checks and balances in there.
• 11:36
  SPEAKER_01: I also want to comment about your uh your observation about the cognitive load and and think about well, you know, you do need to have the guardrails, you do need to have oversight, but that oversight does have a a price, right?
• 11:50
  SPEAKER_01: It has a price tag because it will over can overwhelm a human in terms of f following the details.
• 11:56
  SPEAKER_01: And the other day I I was uh working with uh a uh with Claude in a project where I was taking my methodologies and building uh uh derivative works and building best practices and building you know uh consulting services using our prior IP.
• 12:13
  SPEAKER_01: And Claude came me, gave me some really long explanation about how the documents were not in sync or how there was some challenge or problem with the design.
• 12:22
  SPEAKER_01: And I said, Well, tell me what you should do, or go ahead and fix it.
• 12:26
  SPEAKER_01: And it went through all these iterations and iterations and kept telling me, well, now this is a problem and this is a problem, and I'm like, go ahead, fix it, fix it.
• 12:34
  SPEAKER_01: I couldn't even follow it anymore.
• 12:35
  SPEAKER_01: So I was just giving it, I just saying, go ahead and fix it.
• 12:38
  SPEAKER_01: Well, at the end of it, I then started to check the original state, the original documents, and then what it had produced.
• 12:45
  SPEAKER_01: And I said, There's no difference between these documents, these are the same.
• 12:49
  SPEAKER_01: This is what we started with, and it's like, oh yeah, you're right.
• 12:51
  SPEAKER_01: I said, wait a minute.
• 12:52
  SPEAKER_01: So you're telling me that all these tokens that we spent in these hours of back and forth, you're telling me that that did not produce any value, that there was no change, and said, Yeah, pretty much.
• 13:04
  SPEAKER_01: I was like, whoa, whoa, whoa, wait a minute.
• 13:06
  SPEAKER_01: You have to explain this to me.
• 13:07
  SPEAKER_01: What happened?
• 13:08
  SPEAKER_01: Like, I want my tokens back.
• 13:10
  SPEAKER_01: And and I'm writing an article about this.
• 13:12
  SPEAKER_01: And when I used one model, it was it was very black and white, very plain.
• 13:17
  SPEAKER_01: It said, Well, Seth was trying to do this and I did this.
• 13:19
  SPEAKER_01: And I was like, no, no, no, you got to go into more detail here.
• 13:22
  SPEAKER_01: So I used a more powerful model, and it gave me a very detailed and well re uh researched explanation about how it was inventing tasks and then performing those tasks.
• 13:33
  SPEAKER_01: And and it was identifying problems that didn't exist.
• 13:35
  SPEAKER_01: I mean, it took responsibility, or at least a more powerful model took responsibility for the other model, but I was blown away that that it would go off the rails in this really unexpected way.
• 13:46
  SPEAKER_01: So, so tell me about that, because again, it's it's it's like I'm giving it a little too much autonomy, but I'm still trying to check in.
• 13:54
  SPEAKER_01: But it synthesized, it it made up problems, it created problems, solved the problems, and there was no problems.
• 14:01
  SPEAKER_02: Yeah, yeah.
• 14:02
  SPEAKER_02: Well, I I think you you were talking about before about how you have to be very clear and avoid that ambiguity in your tasks.
• 14:09
  SPEAKER_02: And so uh I view it as like working with these agents, we have three kinds of loops.
• 14:14
  SPEAKER_02: You've got your automation loops, your goal loops, and your feedback loops.
• 14:17
  SPEAKER_02: The the automation loops is like your recurring tasks, the thing that you would do every day or every week.
• 14:22
  SPEAKER_02: And if it's well defined, you can have an AI model help with that.
• 14:26
  SPEAKER_02: And you probably don't even really need to check it too much.
• 14:29
  SPEAKER_02: So it could be like review the recent releases of the past week for security issues.
• 14:34
  SPEAKER_02: And like if a model did that every single week, you would get value from it, even if you barely have time to check it.
• 14:39
  SPEAKER_02: Then you have your goal loops, where that's like your those bigger tasks where the agent's gonna run for hours or days on this individual thing.
• 14:46
  SPEAKER_02: And that's something where you need to be really clear and like well defined up front.
• 14:50
  SPEAKER_02: And if you have something that has a verifiable outcome, if it doesn't, that's where you kind of get messed up.
• 14:56
  SPEAKER_02: Um, in the technical field, I think we can understand this a lot easier that the models are good at code because they can write tests and it's gonna prove that it reached some certain point.
• 15:05
  SPEAKER_02: With like knowledge work tasks, it's a lot harder.
• 15:08
  SPEAKER_02: But the tip I would give everybody, especially for something like using GPT6 now, but this applies to any model, is like you can't be subjective in what you say.
• 15:17
  SPEAKER_02: You can't say make me the best website.
• 15:19
  SPEAKER_02: Because make me the best website will actually give you the most average website.
• 15:22
  SPEAKER_02: And so you need to define what does best mean to you.
• 15:26
  SPEAKER_02: And if you don't know how to define that thing to the model, then ask it.
• 15:29
  SPEAKER_02: Like, what could we do that could make this verifiably true for what I'm looking for?
• 15:34
  SPEAKER_02: Uh and one example with the with the website is if you can design or provide a visual to the model and say, like, this is what it should look like, work until it reaches that point, then it'll get you something really great.
• 15:45
  SPEAKER_02: And I had a lot of fun with Astro when it came out of asking to make some video games for me and then play those with my daughter and stuff.
• 15:52
  SPEAKER_02: But I started out, I had to make this like futuristic racing game.
• 15:55
  SPEAKER_02: And it was cool, it was impressive, and it worked, but the graphics and everything wasn't quite there.
• 15:59
  SPEAKER_02: So I gave it a screenshot and said, I want it to look like this.
• 16:02
  SPEAKER_02: Then it worked on that for another 30 or 40 minutes, and like the visual was just so much better because now it had a clear goal to work towards.
• 16:10
  SPEAKER_01: Yeah, no, that makes a lot of sense.
• 16:11
  SPEAKER_01: I I I would still be concerned about you know giving it that autonomous task to say find the security issues because still, and and you're you're absolutely right with like the like where can like invent these other things.
• 16:25
  SPEAKER_02: Yeah, I'd be concerned it's gonna invent tasks or problems and then solve them and when there really wasn't it's just some for those recurring tasks, like you have to run it like a couple times manually and be really confident that like okay, it's it's kind of always working right for me.
• 16:41
  SPEAKER_02: And so this is where like for a company having your own like internal evaluations and things can be really important.
• 16:46
  SPEAKER_02: And then once you know that like, okay, this always works right, then you can decide like okay, we're gonna this can now be recurring, or if it was recurring, like this can now be like monitored less than it was before.
• 16:56
  SPEAKER_02: But then the same issue can happen with the goals.
• 16:59
  SPEAKER_02: If there's one part where you're kind of not clear, you can, and I have ended up with a goal where it did what I said, but it also added these other things because it interpreted that as I want something to be more reliable, and then it added all these other changes that I didn't really ask for.
• 17:13
  SPEAKER_02: And so uh yeah, you have to kind of build up, especially for like software development and things like the instructions of like, hey, we want this to be easy to review and digest, and we're specifically only going to touch these areas and everything.
• 17:26
  SPEAKER_02: Um, but like the reviewing is a challenge and and being able to communicate with these things better is a challenge as well.
• 17:33
  SPEAKER_01: Yeah, I heard someone characterize this as now we have a requirements gap.
• 17:39
  SPEAKER_01: So because the tools can build these these these outputs and build these applications and perform these tasks, it's really being crisp and and precise about what those inputs should be, what those requirements are, and and uh defining those in as much granularity as possible.
• 17:58
  SPEAKER_01: So I think of a use case library.
• 18:01
  SPEAKER_01: What what do organizations need to do?
• 18:03
  SPEAKER_01: I mean, really what what are they the takeaways around thinking bigger, as you say, uh thinking about uh these uh these non-obvious applications, and and tell me about how they need to kind of approach this problem.
• 18:20
  SPEAKER_01: Uh do you have a framework or kind of an end-to-end process?
• 18:24
  SPEAKER_01: Again, beginning with, well, obviously what you're trying to accomplish within the business, but what are some practical steps to that process to make sure that you're defining your requirements correctly and putting boundaries around what the agent can do and having human intervention or oversight?
• 18:41
  SPEAKER_01: Talk a little bit about how to think about it.
• 18:43
  SPEAKER_02: Sure.
• 18:43
  SPEAKER_02: So uh I hear everybody talking about like the cost of tokens and uh and uh even the cost of like subscriptions, which are fortunately still very subsidized if you're not an enterprise who's paying the API cost.
• 18:55
  SPEAKER_02: But really, what I've been saying and what I see all the the model providers saying as well is the the end thing that a lot of the businesses should measure when it comes to cost is actually your cost per task.
• 19:06
  SPEAKER_02: The the token cost doesn't matter if the more powerful, more expensive model can get it done faster than the the really, really cheap model that goes in loops forever and maybe doesn't even do it right.
• 19:16
  SPEAKER_02: And so the cost per task is important not only for that, but also for understanding like where do you want these things to work?
• 19:22
  SPEAKER_02: If it's something that like, yes, it's using up a lot of tokens or it happens very frequently because you're doing it every day or or every hour.
• 19:29
  SPEAKER_02: If when it does that, it costs like 25 cents, but the human would cost like$25, then of course you're gonna you're gonna do that all day.
• 19:36
  SPEAKER_02: And the the cost of the AI model doesn't matter because it's it's so much cheaper.
• 19:41
  SPEAKER_02: But when you're trying to figure out, okay, how do we how we change our organization from, hey, we messaged Chat GPT a couple of times to working with these agents effectively, I would uh say two things as far as tips.
• 19:52
  SPEAKER_02: One is that you can start small.
• 19:54
  SPEAKER_02: You figure out where are these processes that we have that we keep repeating, that it doesn't feel like a human has to be involved in that.
• 20:00
  SPEAKER_02: And I joke with my team that like telling them like you're not allowed to do work anymore, you have to come up with the ideas and directing these AI agents, and then the AI agents do the work.
• 20:08
  SPEAKER_02: But you're right, there's this requirements gap of how do you define all these things and define them well.
• 20:13
  SPEAKER_02: And for a lot of employees, like delegating things and and all that is a completely new way of working for them.
• 20:19
  SPEAKER_02: For for a founder or executive, it's a bit different.
• 20:22
  SPEAKER_02: But the the one approach is like, where are these items that we can potentially find these low-hanging fruits of we can do this one thing, have AI involved in it, and then just piece by piece go from there.
• 20:34
  SPEAKER_02: The other way, if you're saying like, hey, we want to kind of like approach this, that we have some goals, but like we don't know what to do, is actually think of the outcome and just tell the AI model the outcome because these models are so good now that it if you try to come up with for yourself, especially if you're not technical and you're listening to this and saying, like, okay, well, I think I want to automate something, I have to do it this all the time, and you just try to write out like how that should be maybe automated, you might miss parts or you may have not even communicated like why you have to do that.
• 21:02
  SPEAKER_02: And this is where you start getting into the ambiguity and the AI does the wrong thing.
• 21:06
  SPEAKER_02: So if you instead tell the AI, hey, like I'm trying to achieve this specific goal of like X more whatever per week or something like that.
• 21:14
  SPEAKER_02: Now the AI can kind of help you figure out, okay, well, what are the things that you do usually?
• 21:19
  SPEAKER_02: And then it can kind of piece together what it should do, and then potentially be a better writer than you would at like defining like here's the things that we're trying to reach and here's why, uh, instead of you like missing some of those details, and then the AI is just working towards something, but it doesn't even know why it's working on the goal.
• 21:36
  SPEAKER_01: If you could give us a very specific example or where you went through that process, that might be helpful and kind of what the estimate were.
• 21:45
  SPEAKER_02: So I'll I'll think of one that I I just heard uh last week.
• 21:48
  SPEAKER_02: I was at the the Grokbot meetup here in uh in Austin, and um and their team was saying the same thing.
• 21:54
  SPEAKER_02: And they gave a cool example because they were talking about like how can uh a sales team use These AI agents, because a lot of times we're talking about technical and code and everything.
• 22:03
  SPEAKER_02: And so the example there might be the sales team has to do this research every day and you're trying to keep up with companies.
• 22:09
  SPEAKER_02: Maybe you like want to research the news for the company, and then something happened, it gives you reason to reach out to them.
• 22:14
  SPEAKER_02: Like these are things you could maybe define.
• 22:16
  SPEAKER_02: But if you didn't know like where the value could be with this AI tool, you could instead tell the agent, hey, like I'm a salesperson, I'm trying to get 15 more calls booked per week than I do right now.
• 22:26
  SPEAKER_02: How could I do that?
• 22:27
  SPEAKER_02: And then it will say, like, okay, well, what are what are you doing now?
• 22:30
  SPEAKER_02: What's your day-to-day look like?
• 22:32
  SPEAKER_02: And kind of go through and like let the AI kind of interview you to discover with it the ways that you could do things.
• 22:38
  SPEAKER_02: And I find that this is helpful, like not only if you're not clear on what the tasks could should be or or something like that, but like I'll do it before I have the AI start working.
• 22:47
  SPEAKER_02: I'll spend a lot of time on the plan, especially for like those kind of goal loops.
• 22:51
  SPEAKER_02: And sometimes my my like prompts for these goal loops are like 20 or 30,000 tokens.
• 22:56
  SPEAKER_02: They're like there's 20 to like 50 page documents of what the AI is supposed to work on.
• 23:00
  SPEAKER_02: But I won't be writing all that from scratch.
• 23:02
  SPEAKER_02: I'll be letting the AI interview me to kind of make sure that there's no ambiguities or missing pieces.
• 23:08
  SPEAKER_01: Okay.
• 23:09
  SPEAKER_01: So that's a good approach when you you think about what your what your bigger picture objective is, and then have the AI obviously give it as much foundational context and knowledge as that you can, but then have it walk through uh a requirements gathering seg session with you to say to ask those very specific questions.
• 23:30
  SPEAKER_01: Uh and of course you see that sometimes when you're going through executing a task, you'll you'll have the the agent will ask you, Well, do you want this or do you want this?
• 23:39
  SPEAKER_01: Do you want me to how do you want me to prioritize?
• 23:41
  SPEAKER_01: Would you like this particular treatment?
• 23:43
  SPEAKER_01: Right?
• 23:43
  SPEAKER_01: It will ask you additional clarification questions.
• 23:46
  SPEAKER_01: The problem is when you just do that in advance.
• 23:49
  SPEAKER_01: Yeah, go ahead.
• 23:50
  SPEAKER_02: Yes, because that doesn't happen often enough.
• 23:52
  SPEAKER_02: Yeah, because these models can do some of the same work as humans, but the way that they work is very different.
• 23:57
  SPEAKER_02: And the models are very eager just to do the thing that you said, however, they can accomplish it.
• 24:02
  SPEAKER_02: And so if they don't have the tools, if they don't have the context, like they're gonna try anyway.
• 24:06
  SPEAKER_02: Whereas a human, like sitting next to you at another desk, might come over and say, Hey, you know what?
• 24:12
  SPEAKER_02: Like, don't we use this tool internally?
• 24:13
  SPEAKER_02: Like, could you give me access to that?
• 24:15
  SPEAKER_02: The models are getting better, the harnesses are getting better to try to make that happen easier, but still you're gonna end up with a way, way better result if you go through the process and have the model kind of identify for you, oh wait, all your work is in uh in ClickUp or Asana, like, okay, can I get access to that?
• 24:31
  SPEAKER_02: Okay, I will kind of need this too.
• 24:32
  SPEAKER_02: And now you make sure that like once you hit go, you're you're not just wasting tokens on something.
• 24:37
  SPEAKER_01: Right, right, right.
• 24:38
  SPEAKER_01: No, that makes sense.
• 24:40
  SPEAKER_01: So what is the future?
• 24:41
  SPEAKER_01: I know your your other business was was working with uh with uh knowledge uh centric organizations.
• 24:47
  SPEAKER_01: And what is what is the future of online learning and courses?
• 24:50
  SPEAKER_01: If someone asks Claude to teach them something or teach them anything, what happens to online courses and knowledge businesses?
• 24:57
  SPEAKER_01: What's surviving?
• 24:59
  SPEAKER_01: What's disappearing?
• 25:00
  SPEAKER_01: What becomes more valuable?
• 25:02
  SPEAKER_02: Yeah, I think this is a good question.
• 25:03
  SPEAKER_02: And I and I feel kind of fortunate that uh I have this business that I've been working on for so long because it helped me also understand like where are things more broadly going in AI.
• 25:13
  SPEAKER_02: And my kind of thoughts there are if everybody's got the same magic agents, we press the button and it does the work for us, and like what makes your business different from anyone else's?
• 25:21
  SPEAKER_02: I think it comes down to the how you do things.
• 25:24
  SPEAKER_02: So it's not the not the how to.
• 25:25
  SPEAKER_02: The how to is like you go ask ChatGPT, but like how do you do something?
• 25:30
  SPEAKER_02: And like that's where like companies like Apple will say, like, well, we're different than Windows and Microsoft because the way we approach building the hardware, the way we approach building the software.
• 25:39
  SPEAKER_02: And so that's where I think a lot of the value lies.
• 25:42
  SPEAKER_02: And if you look at like, yeah, what I'm doing with the like online course, online knowledge kind of learning businesses, uh, if you have the the same kind of information that somebody could just go and get from ChatGPT, they're gonna probably be better off doing that.
• 25:54
  SPEAKER_02: So where the value lies is your unique experience and your unique knowledge and uh being able to take that message and then provide that into these AI agents so they can work in that unique way.
• 26:04
  SPEAKER_02: That's where like all this is the most valuable.
• 26:06
  SPEAKER_02: And I'm actually surprised of uh two things.
• 26:09
  SPEAKER_02: One is like people's willingness to give away like their skill files and things like that for their agents.
• 26:15
  SPEAKER_02: Because I think going forward, like that's gonna be some of the most valuable stuff.
• 26:18
  SPEAKER_02: Imagine if uh like anybody can have a SEO agent, it's gonna optimize their SEO.
• 26:22
  SPEAKER_02: But if one person has come up with a specific way of doing it that works really well in a specific vertical of businesses, like that has a lot of value.
• 26:30
  SPEAKER_02: And so we're all we're all using this AI to it, it allows every business to become average, but we all want to fight to be above average, right?
• 26:39
  SPEAKER_02: And so where I would caution businesses to consider this is that I really believe that you should own your agent's memory.
• 26:46
  SPEAKER_02: And this is not something that's true or possible with all the providers that we have today.
• 26:50
  SPEAKER_02: Um, and this is completely separate from like the provider or the lab training on your business's data.
• 26:56
  SPEAKER_02: This means that you just even get to own the data because if you have an employee and they learn some process and you write up an SOP for that, you have that.
• 27:03
  SPEAKER_02: And that that knowledge in your business is a very valuable thing.
• 27:06
  SPEAKER_02: So if you have an AI and the AI learns a process, but that's like in the AI's memory, and you can't like download that or even see it, then you're now stuck with that lab because that's the that AI is the only thing that knows how to run your business.
• 27:20
  SPEAKER_01: Yeah.
• 27:21
  SPEAKER_01: That's a really good point.
• 27:22
  SPEAKER_01: And one of the things that that I do is I provision the various projects that I'm uh running in Claude with knowledge, and I store that knowledge in a in a file directory on SharePoint, and then I have it access, I have different projects access that to provision themselves.
• 27:41
  SPEAKER_01: And during the process, uh so we'll have project knowledge within a project to say, okay, here I'm doing I'm building a, you know, I I I want to write a discovery letter based on a transcript.
• 27:53
  SPEAKER_01: Well, there's a certain format for that, there's a structure, there's do's and don'ts, there's my voice and tone, there's formatting, all of those things are documented and they're put into project knowledge, but they're also set aside in SharePoint as a directory for Claude provisioning.
• 28:10
  SPEAKER_01: And I have many, many of these directories.
• 28:12
  SPEAKER_01: And of course, that's where skills can come to bear, right?
• 28:15
  SPEAKER_01: Because a skill is going to also use knowledge and use certain processes in order to accomplish a task.
• 28:21
  SPEAKER_01: And one of the things that I try to do is I have the uh have Claude document what it's doing and document a practice and codify that so that I can go back and provision, reprovision the folder with that new knowledge, right?
• 28:36
  SPEAKER_01: So it's constantly creating methodologies, it's constantly documenting what it's doing, how it's doing.
• 28:41
  SPEAKER_01: And I totally agree with you that you can have two businesses serving the same market, the same customers, the same exact prop solving the same problems, and they'll be very different.
• 28:50
  SPEAKER_01: And that's how they compete, right?
• 28:51
  SPEAKER_01: They compete with their knowledge of the customers, knowledge of solutions, approaches, uh, how to get to the customers, how to message for them, all those things are part of the competitive advantage.
• 29:01
  SPEAKER_01: And the competitive advantage is based on the knowledge of that organization.
• 29:06
  SPEAKER_01: And you're right, when you are outsourcing it to one of these models, you're outsourcing your competitive advantage.
• 29:12
  SPEAKER_01: So you have to be able to retain that in some way.
• 29:15
  SPEAKER_01: And that, you know, that's the way that we do it is to be able to, again, we're using uh, you know, a pretty enormous repository of methodologies and projects and templates and frameworks and approaches and and solutions and all of these things that you know we've built over the last you know three decades.
• 29:33
  SPEAKER_01: And that becomes knowledge that Claude is provisioned with.
• 29:37
  SPEAKER_01: And then as we're building and and changing things and updating, we're also pulling that back out and saying, okay, why don't you now document what you've learned from this uh process or summarize this for another project?
• 29:50
  SPEAKER_01: Are there any other things that you do or that you can advise organizations to do to be able to safeguard that or you know, be able to capture, codify, and reuse?
• 30:00
  SPEAKER_02: Yeah, I'm glad you asked because I I feel so strongly about this that this is part of what we built into Latch Loop.
• 30:06
  SPEAKER_02: So when you're using Latch Loop, it's similar to uh ChatGPT work and codex or Claude Code and Cowork, but you own the agent memory.
• 30:14
  SPEAKER_02: And so the agent, whether the project can be a general like knowledge work agent, could be a coding agent, it's connected to a repository in GitHub that you own.
• 30:23
  SPEAKER_02: And the agent can update its own memories and everything in there.
• 30:26
  SPEAKER_02: And if you decide later, hey, like I don't want to use your platform, I want to go somewhere else, you just take that and it can give it to any other agent and it's ready to start from where you left off, basically.
• 30:36
  SPEAKER_01: Great.
• 30:36
  SPEAKER_01: That's a great uh takeaway because we're building into these agents methodologies and approaches and processes, and we're capturing that tacit knowledge that would be in the head of an expert and applying it explicitly.
• 30:49
  SPEAKER_01: And then when we do that, then we need to be able to make that portable rather than leaving it in the uh memory of that particular uh provider.
• 30:57
  SPEAKER_01: So those are I think like thanks.
• 30:59
  SPEAKER_02: I I think talking about this also can help share another tip that now if you're if you're on board with that, like this this agent memory and this knowledge is important, then now you can understand this is the way that you need to interact with these agents as far as like where you're valuable as a human and where the agent's gonna do something.
• 31:17
  SPEAKER_02: And so you don't want to outsource those things.
• 31:19
  SPEAKER_02: And I see people talking about saying, like, well, I like uh Fable 5.1 because it's a little bit more creative than the GPT models or something like this.
• 31:28
  SPEAKER_02: And like there's something to be said about all that, but at the same time, you have to remember, well, your ideas are probably better than the AI's ideas.
• 31:36
  SPEAKER_02: And so the the implementation can be done by the AI, but the ideas and the why and all that and the how that has to come from you.
• 31:44
  SPEAKER_02: And a great way to like realize this as either an individual or organization is when you have the AI work on these bigger tasks.
• 31:51
  SPEAKER_02: You can see these, especially in like software development, you see it build something, and it's like, okay, fundamental fundamentally it listened to everything I did.
• 31:59
  SPEAKER_02: It built all the requirements.
• 32:00
  SPEAKER_02: I can verify they are all there.
• 32:02
  SPEAKER_02: But then when you look at the thing and interact with the thing, it's like, okay, this is not how a user would want to actually interact with this product.
• 32:09
  SPEAKER_02: This is not actually aligned with what a human cares about.
• 32:12
  SPEAKER_02: And so the AI has a a lot of trouble understanding that.
• 32:15
  SPEAKER_02: And that's where like the people who have the specialized knowledge or like a designer still can bring a lot of value to the table.
• 32:22
  SPEAKER_02: It's not just about like what we think visually looks nice, but like the way in which humans want to interact with something or what humans care about.
• 32:28
  SPEAKER_02: AI has a very difficult time understanding those things.
• 32:31
  SPEAKER_01: Yeah, there's there's there's no replacement, there's still no replacement for the human creativity and and human judgment on these things.
• 32:39
  SPEAKER_01: And that's a really great takeaway for organizations.
• 32:42
  SPEAKER_01: So, what should organizations start doing differently tomorrow?
• 32:47
  SPEAKER_01: What are the things they need to think about when they're looking at how to better leverage AI and agents at the core of their business?
• 32:55
  SPEAKER_02: I I think the very first thing if if you're an organization that you like you have ChatGPT, but you have not used any of these more agentic kind of products or or tools is just documenting.
• 33:07
  SPEAKER_02: So like making sure like any business who has all these clear SOPs and processes are already somewhere, like you're at a big advantage because you can now potentially offload some of that almost directly to a model.
• 33:19
  SPEAKER_02: But yeah, if you don't have that, like documenting and getting all those kind of things clear, then starting step by step of small things here and there of how you can involve AI.
• 33:29
  SPEAKER_02: Don't like download open claw and say, here's my whole computer, good luck.
• 33:33
  SPEAKER_02: Just step by step getting more things done with it and like building up the muscle, not only as an individual, but as a team of using this tool and learning this a kind of new way of working.
• 33:44
  SPEAKER_02: And then where you're not sure, focusing on, hey, these are the outcomes that we're trying to reach, and then collaborating with the AI so it understands what are you trying to do and then can kind of help you figure out the like how it's going to be actually connected to what it can help with.
• 33:58
  SPEAKER_01: That's great.
• 33:58
  SPEAKER_01: That's great.
• 33:59
  SPEAKER_01: And I I I tell people to record everything, record your team meetings, record your you know, your ideas, just start riffing on topics, interview people, and all those things become inputs for the AI to uh to start extracting that that tacit knowledge and that that organization, institutional knowledge of how you do work.
• 34:20
  SPEAKER_01: Well, thank you.
• 34:21
  SPEAKER_02: That's a really great point.
• 34:22
  SPEAKER_01: Yeah.
• 34:23
  SPEAKER_01: So thank you for joining me and sharing your perspective on you know how AI adoption really needs to go beyond that surface level, right?
• 34:31
  SPEAKER_01: And and and how organizations need to change the way they're looking at processes, the way they're looking at how value is created, looking at customer needs, looking at really how the organization needs to operate.
• 34:44
  SPEAKER_01: And uh thank you to our listeners who are tuning into the AI, not really AI podcast.
• 34:48
  SPEAKER_01: Be sure to subscribe for more conversations about how AI is shaping the future business.
• 34:53
  SPEAKER_01: And thank you to our sponsor, Hector.
• 34:55
  SPEAKER_01: Be sure to check out their site, uh VKTR.com.
• 34:59
  SPEAKER_01: And you can also find us on D Logos.
• 35:01
  SPEAKER_01: There's a URL here, it's D logos.xyz.
• 35:04
  SPEAKER_01: And then uh the podcast uh link will be in the show notes.
• 35:08
  SPEAKER_01: And uh and Brian, I'm sure people can find you on LinkedIn and and your information will also be in the show notes.
• 35:15
  SPEAKER_01: So thank you so much for joining me today.
• 35:17
  SPEAKER_01: And uh thank you to our listeners for taking the time to tune in.
• 35:20
  SPEAKER_01: Thanks, S.
• 35:21
  SPEAKER_01: Thanks.
• 35:22
  SPEAKER_01: Bye now.
• 35:23
  SPEAKER_00: Thank you for joining us on another deep dive into AI innovation.
• 35:27
  SPEAKER_00: Tune in next time when we introduce another industry expert and discuss how to maximize AI in your world.
• 35:34
  SPEAKER_00: The Early AI Podcast is sponsored by Early Information Science and Vector.
• 35:39
  SPEAKER_00: To learn more, visit early.com.
• 35:41
  SPEAKER_00: That's E A R L E Y.com.
• 35:44
  SPEAKER_00: Thank you for tuning in.