Latest / Serious People Podcast / Michael Ducker: What Actually Makes You an Agent Builder

Transcript

Michael Ducker: What Actually Makes You an Agent Builder · Serious People Podcast

• [00:00:00] Michael Ducker: If you think of, like, the average sort of office worker, they've been at school for 20-some years before they get to that office job, which means we spent 20 years teaching them how to speak, how to write, how to think, how to use computers, And the crazy impact of AI is that it's changing all of those mechanics and giving us a whole new set of tools to go express our human behaviors in. And you've not had 20 years yet to learn. [00:00:22] Noah Levin: Welcome to Serious People, the show about doing serious work with AI. My guest today is Michael Ducker. Michael is the co-founder of Valet.dev and an expert on building agents. Michael and I get into the idea that AI is genuinely transformational, but it's also creating real disillusionment. And And the bigger idea underneath it all, a business is a collection of processes, and agents are becoming the way you encode those processes. So if you feel that there's a gap between what's suddenly possible in the world and what you feel like you can do yourself, this one's for you. I learned while I was doing some studying on you for this conversation that you are, in part, responsible for some of the emojis that I've used in my life. What did you do and what has it done to me? [00:01:06] Michael Ducker: We all have exciting careers, and I think that's the thing about being human is that we all get our own little chances to make impacts in the world. One of the opportunities I had was I worked at Twitter for a long time, back when it was called Twitter, And we had noticed, NTT Docomo had launched emojis in Japan, and Twitter's second-largest, uh, marketplace was Japan. So this is like right around the iPhone maybe come out the, you know, two... The iPhone came out, what? 2008. And what we noticed was when everyone from iPhone tweeted an emoji, it didn't work because Twitter was not an iOS platform. Twitter was a web platform that had an iOS app, and our website could not view these tweets. Our Android user could not view these tweets. It was only working on iOS apps. And so we went to Apple and said, "Hey, Apple, we have this big global web platform. You and us, we're friends. We're embedded into iOS. Could we get a like, soft license, like an email saying that you're okay with us, you know, grabbing the PNG files from the iOS app?" And putting them on the web and, like, putting, uh, your emoji everywhere. So we were tasked then with the problem of how do you generate, I think it was about 600 beautifully handcrafted icons to enable the rest of the world to experience emoji. And we went and built one of the world's first public open source color emoji sets, and we got that out in the wild. And it really spurred a lot of change in emoji. It became the mo- one of the most popular GitHub projects. It was used everywhere. If you go to Target and check out the checkout screen, it was all of the emojis that we had built. And I personally made some changes to the emojis to like make them better. Two of my favorite changes to the emojis that I made that I can claim credit for would be, at the time, all the airplanes had four engines, and in 2012, if you wanted to buy a four-engine plane, you couldn't. The A380 had just been discontinued. It did not exist anymore, a plane that had four engines. And so it felt very odd that our emoji, the art of the future, was showing the past airplanes. So we upgraded the airplane emoji to have two engines and to put the engine nacelles way up front of the wing where they belong, so it was aerodynamically correct. And now all the emojis copy that. Another example of an emoji we changed is the rugby ball that Apple had done was this like old school English rugby ball where it was like leather, and no one plays with that ball anymore. It's like using the old like soccer ball that would be like the hexagon, black and white. Like no one's used that for a long time. And we modernized the rugby ball to the current modern kind of swooshy format. And that's a nice one 'cause it just makes it now finally not look like a football [00:03:32] Noah Levin: I love that you have a mark on history in the form of the lineage of these emojis. And I think it's funny that you mentioned modernizing two of them because I'm using an emoji as part of one of the products that I'm building right now, and it's the floppy disk emoji. And I realize now that the floppy disk is... I don't even know what my son, who's two and a half, is gonna think a floppy disk was [00:03:53] Michael Ducker: It begs the question, like, what, what emojis are gonna go away because we adopt AI agents? [00:03:58] Noah Levin: Do they have to go away or do you just keep adding new ones forever? [00:04:02] Michael Ducker: I think they become anachronisms of the past. Like the fact we use a floppy disk for save. [00:04:08] Noah Levin: Mm-hmm [00:04:09] Michael Ducker: No one knows what that is [00:04:10] Noah Levin: Today's episode is sponsored by Valet. Our guest today, Michael Ducker, built Valet around the simple idea that it should be easy to turn a repeatable business process into an agent your team can use. Valet agents connect to the systems that you use at work, so you can chat with them in Slack or just about anywhere else, and they can do work that you would otherwise be doing yourself or not doing at all. I've been using Valet to prototype agents for clients. I love how easy they are to build and customize and integrate with my clients' systems. I've also been using them myself. One of my favorite use cases right now is prepping for meetings. I find myself in back-to-back meetings just context switching all day, so every morning a Valet agent reads my calendar and it researches who I'm meeting with and sends me a write-up in Slack so I have it handy. I want to thank Michael and the Valet team for sponsoring the Serious People podcast and taking a bet on us. Try making your own agent at valet.dev and see how Valet can help you own the business, not the busyness we're gonna talk a lot about, how to build an agent and what the pitfalls are and, I told you all the excitement that I'm hearing from people about AI and, how the more I put out into the world that AI is this transformational thing that I'm, riding the wave of, and you should too, I was getting a lot of, "Yes, yes, yes. Let's do it. Let's do it." And you said, "I was actually surprised to hear you say that." You mentioned something about AI disillusionment. Do you wanna say a little bit more about what you're seeing and, do you think both of these things can be true at the same time? [00:05:39] Michael Ducker: There is a lot of disillusion about AI in the world, and that is a true statement. People saying, "What is going on? I'm not ready for this yet." But then there's also the truth of it is transformational. I totally agree with you. So they're both true at the same time. But, maybe I'll give you the, bear case or at least the things that need to be addressed for people to, like, join us on this bandwagon of how exciting AI can be. The first is it takes a long time for humans to learn. If you think of, like, the average sort of office worker, they've been at school for 20-some years before they get to that office job, which means we spent 20 years teaching them how to speak, how to write, how to think, how to use computers, how to interact with other people. And the crazy impact of AI is that it's changing all of those mechanics and giving us a whole new set of tools to go express our human behaviors in. And you've not had 20 years yet to learn. And so the thing is there's a lot of, like, anxiety and emotion and friction about being asked as a human to, to change what you're good at. And obviously there are some people out there, you might call them early adopters, you might just call them, like, crazy people who love chaos, who lean into that, who are like, "Oh my gosh, change. What fun. I want to try all these new things, throw away everything that works and do something new." But I think a lot of people aren't in that position. They don't have the finances to go do that. They don't have the trust in society to go do that. And so AI can be really scary because it actually asks everyone to change their behavior overnight instead of giving them 20 years. The second thing that AI is difficult for me for is We as an industry, so I'm talking now as Michael Ducker, the Silicon Valley software guy, we have never rushed faster into something. We've never all of a sudden said, "Oh, this is the one way to do things," and moved with such groupthink with so many people participating. And the way that's been perceived is instead of a slow rollout where maybe your friend gets an iPhone and you still have your Flip Tac or your StarTAC and it taking three to five years for you to acquire the smartphone and realizing that you can now stream Spotify and have Uber, etc. AI happened all at once. Overnight, you launched your applications and now there was a chatbot on the right, and that chatbot wasn't actually able to do much. It couldn't really... It didn't really know about Microsoft Word or didn't know about your product. It just chatted with you. And a lot of the value of what you thought you were gonna get from it was a disillusionment because you would ask it things and it would not actually give you the right answer. And so there's a lot of trust that I think that got lost by our industry's overexuberance to put the generative text bits of AI everywhere. They're all, like, nervous that ChatGPT was taking all the traffic and they were gonna become a new Google monopoly, so if they put the chatbot in there, it would be better. And so there's a disillusionment of like, "Oh, it's no-- it's not that good," right? And I think I've seen, that a lot. And the third thing is it, it is incredibly impactful and I think that there's a real reason a lot of people in this, where the governments get involved and regulators. Like, there's a trust we built in society about how expensive it is to build things, how expensive it is to break things, that how expensive it is to automate and become a thousand people and all those things are going away overnight. And so we kind of are losing some of the implicit trust of how computers work. And I think that probably is the unsolved part of why AI is scary is I actually don't think we know as an industry how to make AI, quote, "safe." Anthropic recently launched a model called Mythos that they said that were able to find zero-day vulnerabilities and hack people's computers, over and over and over and over again in a way that no one's ever seen before. And so they said, "Okay, we can't release it." And then, recently they actually did release it in a model called Fable and they just disabled huge swaths of topics. If you ask it a computer question, it's like, "Sorry, I can't help you." Because they don't know how to make it safe yet. And I think that's why you get this disillusionment is like how can we have a technology that's so powerful that we don't as a c-community know yet how to like, even the experts know how to like fully contain or control or understand the impacts of [00:09:21] Noah Levin: All right, so the plan is, we're all doing AI. We have to do AI. We're not totally sure how to make AI safe. All your competitors are doing AI. You better bring AI into your business. Good luck [00:09:33] Michael Ducker: That's a high tension. So the people who get it are having so much fun because you can do so much, right? Like, especially if you put aside, like, give ... Like, I, like, I'll, I'll give you an example. I have given AI my Gmail password. You have given AI write access to your bank account. These are scary trust centers in our lives, money and the ability to log into any of our accounts. And we're already kind of like throwing caution to the wind and saying, "Let's try it out." And the whole industry is kind of encouraging us to do that because when you give your AI your Gmail, it can do your email for you, which is a huge bane. It takes a lot of time. Give your email your bank account, it can do payment ops and like manage your revenue for you. That's pretty nice. But we also still, you know, I think the cautious mindset is like there's no actual controls that are keeping that safe right now, and that, that's where you get a lot of the naysayers or like the negativity 'cause it doesn't take a lot of bad apples to, I think, ruin a party [00:10:25] Noah Levin: I think I have been learning about the trough of disillusionment and, I think I was trying to coin this term with you, the uncanny valley of agents or the agentic uncanny valley. There's this idea that agents just kind of work now. And, there's a lot of people who will YOLO their permissions for their personal lives, right? Because... And this is, this is in part, don't hack me, please, but this is, in part like how I live my personal life and how I'm running my business is I think it's so damn cool that it's 2026, and we're getting to use these things and that I'm alive for it. And so I don't really wanna wait, and I'm willing to take measured risk for myself. Pretty much anything you could delete of mine, I can find in, in the backup, you know, somewhere. So I'm willing to take a little risk on my own life, but, it doesn't work that way when you run a business, and it especially doesn't work that way when you run a business that has, people's lives on the line, people's money on the line, or high compliance risk or security risk. And so, what I've observed is that, I get excited reading something on AI Twitter and then try and apply it to my own life and realize that it's hard. But, you know, maybe just skip permissions and you can get it to work. But, when you try and bring it into a legal practice or a crypto company or a bank, you have to deal with these, messy gap between what's possible and what it takes to make it work in that environment. You're building a product that's meant to help bridge that gap. I'm building a service that's meant to help bridge that gap. And these two things I think are part of this emerging ecosystem of trying to make agents work in reality. It seems to me like that's part of the founding ethos of Valet is, helping to take some of that complexity and abstract it away. What is the founding story of Valet? Why are you building this? [00:12:07] Michael Ducker: So Valet is a platform for experts to express seeds of ideas of how to solve problems and allow you to hire them and join them into your team so those experts can participate alongside with you to, solve your critical business problems and automate your life. And you can imagine, like some examples of this can be very simple. Like, I wanna hire somebody who really understands media and understands buying ads on Facebook, and I wanna bring their expertise of how to buy ads on Facebook into my company so I can have better Facebook ads. You can imagine you might not have the most senior engineer on your team. You might be working with people who have less experience. Wouldn't it be great if you could hire their expertise into your team and embed their knowledge on how to run and stand up really great software practices so that you can scale your vibe coding over and over again to a much larger group of people? This articulation of this concept was not where we started. We started with recognizing that it's time for businesses to u-automate their business processes with AI. And if you think of a business, a business is a collection of processes. You have an aligned goal you're trying to get to, and everything you're trying to do is these processes that take inputs and create outputs that further that end goal. Those processes have been a mix of you hire humans who are experts at doing the work, and then you hire support humans who help make it more efficient for those experts to get their job done. You basically have that same shape, which is you have a bunch of people who are really good at solving the problem, and then you're hiring a ton of other people to basically build these efficiency machines to make sure that the actual work can get done. And when we saw last fall that the AI quality was moving from LLMs that generate text to LLMs that use tools and can complete work, we got really excited. We were like, "Wait a minute." Like, if a business is all processes, that means the role of working at a company is about to change. Like a generational change where the idea of the computer worker who uses Word and Excel is now a manager of all of their processes, and they're gonna almost like maintain them and grow them and make them better. There's a shift going on about what does it mean to be a worker. And when we dug into the tool chain of like the platform, the apps you can use to like become part of that new generation of workers, there was nothing, right? If you're a financial analyst today, your tool is a spreadsheet. What did financial analysts do before spreadsheets? They did it on paper, right? They didn't have a tool yet. I could not imagine some of my, peers and friends doing their job without the tool of a spreadsheet. And when we started digging into the problem, like what tool should exist, why don't they exist, we just learned that there was all sorts of problematic challenges around collaboration, around security, around safety, around access controls that made it difficult for many businesses, as I think you mentioned, Noah, in the intro to this, to even start using the tools. Like, they didn't feel ready or comfortable that their needs were being met such that they could deploy them. And so what you see is all these individuals figuring out on their own, in their own home environments on the weekends, "Hey, this thing works for me." And the one exception to that would be there's a small class of software companies in the world, basically take their best talent and say, "Hey, this AI thing looks really cool. Why don't you build, tens of millions of dollars of software for us, just for us, that allows us to have that sort of, that infrastructure we're missing?" That's what Valet is for. We're here to like plug in all those gaps, give you a platform that you can trust, that will solve some of these security challenges, that solve some of these problems of where data flows, that solve some of these problems about how it works with your existing users, and allow you that outcome of being able to hire these experts to join your team and to have that teammate that you get to talk with, that gets work done for you, get better and better and grow with you, as you run your business. [00:15:43] Noah Levin: I've been in enough of these, conversations with businesses that are trying to figure this out that I, I can really appreciate the gap between wanting to do it and actually getting started doing it. What do you think, for the people who are listening to this who are like, at the cliff edge, right? They're the person in the company who might be the change agent and who's the one who was, like, really eager to bring, a thing back to the team and get them signed onto it. What's the catalyst for, actually making it happen? Is it finding a use case that matters? Is it, being able to definitively say, "This, this risk is mitigated"? What sells that, transformation to the team that they're trying to bring along with them? [00:16:22] Michael Ducker: I've seen several patterns emerge. I'm sure you have too. The first step is getting permission to use the modern tools. And this is a harder step than I think a lot of people have to deal with, which is if you've approved ChatGPT on .com, figure out a way to get the desktop app. If you've approved Anthropic or claude.com, figure out a way to get the desktop app. Because these tools work best when they have access to your files and your computer. That's when they start to come alive. They start to become you. If they do not have access to your files, if they can't be you on the internet and use your web browser, it's really hard to come up with problems for them to solve. [00:16:58] Noah Levin: Already we're at a, a sticking point, right? So I'm gonna go to IT and say, "You've given me access to this, what was a cutting edge a year ago web service. I would like you to let me install their program on my computer so it can mess with my files, so it can browse our corporate internet." How do you persuade your IT person, that that's a risk worth taking or that, that, can be contained? [00:17:20] Michael Ducker: Luckily, Anthropic and Claude and OpenAI have heard these problems, and there's actually quite a few tools they provide built in that can help mitigate several of these risks. So you could sort of take them like the toolkit of, like, how do I set up my Claude to be safe? An enterprise plan of either of these products allows you to route what's called the inference through a proxy, so your IT can see and monitor and actually log everything that's happening in your tools, which gives you that realm of, "Hey, did I cause something to happen?" At least we can now understand what exactly happened and how to do it. The second is you can, configure these tools at an enterprise layer, like you would a desktop computer, to turn on or off certain functionality. And so an enterprise that's just getting started with AI might say, "You know what? I-I'm okay with using Claude desktop editing tools on your computer, but I don't want you to connect MCP servers up yet." And so you can actually disable that functionality. And while that's not what I would recommend in the future perspective, sometimes it's g-really good to iterate and build trust over time. The third step is really thinking about how are you trusting your employees? So there, there's always a human behind everything, and I think it's important in AI development that you're still holding your humans accountable to the work they're doing, in the same way we would hold a driver accountable to a car they're driving. And so it's important to know, do you actually have per human level credentials set up in your system, or are you sharing things into one password? You don't want to have everything work on shared credentials because that means you can't drive that human level accountability. But if you can say, "Hey, your credential was used in this way that did this thing we didn't like," who knows what it was? That gives you the actual real stick in the world, which is your employment with that company. And everything I've seen in every organization is that's the thing that works really well. It really comes down to, like, your human employment. You still want to be employed, so you have to-- you are then given the control to use the tool safely and appropriately for w-what your job is. [00:19:09] Noah Levin: Yes to all three of those. My experience has been that, there is a right way to configure Claude to work in a corporate environment, and it's not super easy to find that set of instructions and to know that that's the right one for you. And there's something about the way the product is evolving rapidly or just hasn't quite coalesced yet, where there's not just a one-click, make this safe for me. And that's causing a lot of adoption hurdles. There was a product person from, Anthropic, I think on, Lenny's Podcast at one point, who, said, sort of as an admission, sort of as a bragging right, that, there are parallel ways to do the same thing in Claude. And that's sort of the cost of moving as quickly on product as they are, is that, I-if you ask what is the web version for versus Cowork for versus Code for, there's not like a really clean answer for how these three products live totally separately for different use cases. It's kind of like, well, these things seem to be good, let's keep building them and let that shake out over time. I think the same thing is true on the settings panels, personally. [00:20:08] Michael Ducker: Yeah. I agree. A very complicated product. Like, the number of ways that you can add a skill or a plugin or an agent they just have routine loops, and there's a lot of definitions of what fundamentally kind of look like the same thing. They're rapidly growing products. Like, this is not a product surface area to sit back and expect to have something that's solved. Like the Claude or the OpenAI product you download today will not be the one you download in six months. [00:20:31] Noah Levin: You've just come out victorious from your conversation with IT, and you now have, Claude sitting on your desktop, of your computer. What's next? [00:20:38] Michael Ducker: The next step, and I think this is the step that people don't realize, is to align on how to share what you're learning with someone else. A lot of people try to use Claude in a work setting or OpenAI in a work setting the way you might do it on your personal laptop. Do some cool stuff and like kind of grow complexity. But the value that accrues of everyone learning together and going through this change together is so much greater if you start together. And so the things I generally recommend people set up is some form of shared file system from day one. And that can be on iCloud, it can be on Dropbox, it can be a GitHub repo where you're actually running Claude from a directory that's synced with your peers. And then what that does is it allows you to start building up skills and MCP configurations where only one person in your team has to do the work to figure out how to make this magical AI thing actually solve a problem. And then once you've solved that problem and written the prompt and given the tools, everyone else on your team can share that. And that, that is a pattern that software engineers have done forever. I mean, it's very common in software engineering that when you pull a repo, which is the way that source code is stored, it contains all the instructions on how to set up and use and test and actually manage that piece of software. Whereas in other fields, those SOPs end up being discrete documents that don't really work together. And so it is kind of a new skill to think about, like you're building this shared place that has everything it needs, self-contained, to get anyone new on your team. And I think a good test of this is imagine you're trying to hire a new person to your team. What would you give them? Like, how do you set them up to be just like you? And if you can just maintain that level of documentation, that allows Claude to grow with you and grow with your team. [00:22:12] Noah Levin: Um, I-- let's, let's make this as practical as possible because I think that what you just described, sounds more complicated than it is 'cause there are some m- proprietary words. So, when you say like a shared folder, are we talking like a Google Drive folder that's local on your computer? [00:22:27] Michael Ducker: It can be as simple as that. It can be as simple as if you use Google Drive or Dropbox or Box, make sure, like, have your team folder, and when you go to launch Claude, make that shared folder the working directory. So there's a way to, like, set which folder you're working on in Cowork and in Claude Code, and start with that folder. Because that means that everything you do that's in that everyone else will get as well [00:22:45] Noah Levin: And then when we talk about like, putting skills into that folder, a skill is [00:22:50] Michael Ducker: A skill is a saved prompt. So it's a, it's something that you've like rehearsed and done over and over again maybe, where you've like fine-tuned how AI responds to your instructions so that it's more repeatable than if you were to prompt it from yourself. One of the hardest things we learned in the past AIs that's carried forward is that the single one-word, one-sentence prompts aren't actually the way to get the best result. The best result from AI comes from giving it volumes and corpuses of relevant context. Every detail that's exactly needed to solve your task. The more information you give it that's related to your task, the higher value outcome and higher, higher quality outcome you're gonna get. And this is true for any human level of limit. This is in the software layers, you can actually give too much context and break it, but no human's gonna write a million words every prompt. So don't worry about overwhelming the AI with your human thinking. What the skill then is, is taking that example where you figured out maybe it might be 10 pages of instructions. How do you save that so somebody else can run it without having to rewrite 10 pages of work? [00:23:48] Noah Levin: I think one of the things that's most confusing to people who are just getting into skills is the idea that they're really just text files, right? I think we've done ourselves a huge disservice as an AI industry by, using Markdown and then not giving people an application on their computer that can open a .md file, [00:24:05] Michael Ducker: I know. Could not agree more with that. Oh, it's so scary. I still don't know the formatting. It's like how many hashtags make what is something I always ignore [00:24:14] Noah Levin: The metaphor here is it's like your agent wakes up and reads the new hire orientation manual from scratch every time it does a task. It's happy to read all of it, and then it's happy to do exactly what's in there to, to the best of its ability. And so instructing it as you would a new hire and maybe even being more pedantic than you would be for a new hire is a, like, is a perfectly valid way to approach a skill and create a library of skills [00:24:38] Michael Ducker: Quick hits on skills. One is they are just a, a macro. And what I mean by that is when you type /skill in any of the tools you use, they're literally copy and pasting the text into where you put that slash word. So it's just like your iPhone where you have, like, your misspelling and it's auto-correcting it based on what you saved in the dictionary. That's all a skill is doing. It's just copy and pasting all that text into your system. The second thing is that AI can write really good skills for you, and you can actually ask it. There's a really great technique where you ask the AI to interview you. A lot of the AI tools now have interview tools that are built into the harnesses that you use that know how to write questions to you and actually, like, treat you like you're the client, and they're the person you're hiring to get the job done. And that can really allow you to write way better skills. Because they have been trained on all the skills in the world and understand basically, here's a format that works. And so you'll get a much better skill generally by using the agent to actually write a skill first, which is like the generative concept, and then try to apply your skill. The third, another hot take or a third hot take is you can download these. You don't need to come up with them yourself. I use tons of marketing skills that I download all the time. I'm not a good marketer. I would love to be a better marketer. I go to skills.sh. It's an open source repository of skills. There's more than 100,000 of them, run by a really big tech company here in the Valley called Vercel, and so it's safe. They even give you security things around, like, which skills are doing what. But the important thing is you can get other people's expertise. So you can get a really good marketing copy, a really good design skill, a really good cold email skill, and just download it and save it. And then you have it. It provides that human lens on some of what you're doing that, that tunes these models, which are the average of everything. Again, if you just use the models without skills, you're gonna get the average. If you put a skill in there, you get to sort of have a little bit more opinionated take. [00:26:18] Noah Levin: One of my favorite things about the way AI, integrates knowledge is it, it can import something wholesale. You can just download a file and drop it in, or you can say, "Hey, read this, and then take the lessons from it. Take the version of it that applies to me and everything you do with me and write your own. Bring it into your world." So you can point it at a GitHub repo or, a file on your computer. You can point it at a book you don't really wanna read but you wanna extract the lessons from, and you can map that onto your own world and sort of import the knowledge without importing the file itself. I take a lot of pride in writing, really good proposals, and I wanna write a lot more of them. So I spent, an embarrassing amount of time on a proposal recently, about eight hours. And most of that time was me talking to my agent using Wispr Flow and, just like telling it verbally in way too many words how many pixels to move something up and down. And I got to an output that I was really proud of. And then I said, "Okay, that thing we just did is a skill. Not just the what, but also the why. And I want you to be able to recreate this conversation yourself the next time we start off a new proposal and get me much, much closer to that output. And then you can sort of iterate from there, mapping it back onto your own, process." [00:27:30] Michael Ducker: I love that example because every session you do at the end, you can actually reflect on, and you can reflect in a meditative sense, you can reflect in a learning sense, you can reflect on a store this and save it and do it again later. And I think once you learn that, that's actually one of the tricks to going from I'm using AI as skills and generating stuff on my laptop to where are my repeatable processes? How am I gonna ever turn this into a quote agent in the cloud or something that's running all the time? It starts by understanding, oh, I did something, I wanna save it. I wanna save that thinking and that process so I can reuse it again [00:28:01] Noah Levin: I'd love to ground this in an example if you have one. It could be a, an example that went really well or an example that went really not well. When we, we were talking, with your co-founder about the things that, make agents successful over the long run in a workplace, and, we used words like drift, to describe failure states. And so there's a bad version of this where an agent is, allowed to sort of meander, and there's a good version where an agent compounds over time. Do you have, like, a favorite example of this going really well or a favorite example of this not, and the lesson you would take from it? [00:28:36] Michael Ducker: I have a good example, and I'll start there, the Agent Builders Breakfast community that I run, it requires a good, community. It requires a good CRM. It requires contacting people. It requires sending events to people. It requires tracking who showed up. Like, just think of all the little bits and details to create the next great community. And traditionally, you would either buy a tool to do that or do that by hand. And I think most people start in a spreadsheet. I started in a spreadsheet, and I evolved a set of skills in Claude that would start task by task, starting to piece together, oh, like this person showed up to this event, let's mark them. Or this was the Luma, which is my event, ticket system. This is the CSV of their stuff now imported into the newsletter tool, so that I can, I can send you a newsletter. And all these tasks started building up, right? I started getting Claude very iteratively 'cause I kept running these every week. And at that point I said, "Wait a minute. Why am I doing this by hand at all? I now have everything documented. I've been practicing it. Let's try just running it as one sort of end-to-end flow." I give it a spreadsheet. I give it a photo of whose name tags were left at the event. Figure out the rest. And I now have it working such that I can take that photo of who didn't pick up their name tags, and I can take the spreadsheet from the event system, and every detail, the newsletters, the emails, the CRM, the LinkedIn requests are all automated. Now, that's a lot of work, but to get there, one of the techniques I had to do was I had to feed it back feedback. So every time it ran, I had to say, "Okay, review what happened in the session and do a technique called hill climbing." Hill climbing is where you feed results back to an AI who's writing a prompt to say, "Improve this skill." And it works for the first, say, 30% of how you can improve something. Like, it's not gonna get you all the way to the best something can be, but it's a very well-proven technique, and it's the first place to start. Feed it the results saying, "What did you learn from this session? Where did you get lost? What did you make a mistake on?" And rewrite your skills to address those problems. And that was a really great experience for me because I basically got to, like, run this every week, practice that hill climbing technique, and now I have something that I can just set and forget. I, I literally drag and drop the photos and the whole thing's done. [00:30:49] Noah Levin: L-let's make that really practical for people who are listening. So, I'm having a conversation with, Claude, in a session. Let's assume I'm in Cowork or I'm in Code, and, we're doing some task. A pr-pretty typical example for me would be, I'm writing some artifact. I'm writing an email or meeting notes, and I've got lots of feedback for it, right? I have edits. I get to the end of this task, and I've got an artifact that I like. What am I telling it to go through that hill climbing loop that you described? [00:31:14] Michael Ducker: Yeah. So the first step is you're telling it to create a skill because you need an artifact to hill climb. So you need to start by saying, "I need to create s- a representation of that session I did that I want to reuse." And then the next time you run it, so it does take several times in a row, the next time you run it, you need to invoke that skill. You need to say in your prompt, "Use this skill/whatever, do the task." And then you're likely, because it's your first time using the skill or your first time going through this process, gonna find that it gets stuck, or it needs to ask you questions, or maybe it just takes a really long time. And that's okay. Instead of asking to write a skill, ask it a different question. "Review the session I just had. How would you improve the skill to reflect the learnings you had, or the frictions, or the challenges or anything to make this better next time?" And it will go rewrite the skill, and now you repeat that loop. And I find in my personal experience, three loops or so, it starts working really well. You know, it depends on the complexity of your problem, depends on like how indeterminate what you're doing are. But generally, like you do it a just a few times. This isn't like you have to do it a ton of times. You get to something that really starts feeling like, oh, it's doing exactly what I want it to do. And at that point, that's really when other people can share it, right? Then you have a thing that's been tested by you that sort of started to feel like it's better, and you can put it out in the wild. And there are other techniques then for prompt improvements, um, that are beyond, I think, the scope of this podcast [00:32:34] Noah Levin: Yeah. But I think you, kind of get into another interesting area, which, we were talking about before we started recording, which is, that's where it bridges from single-player mode to, to multiplayer mode. So I, I think there's this confounding thing about, the way the form factor AI has taken on for a lot of us, which is we're all used to these like super, thin client cloud services, and we might not describe it as that. We just know we open our web browser and wherever we are, there's our email. Or we're used to opening a Google Doc and being able to collaborate and have other cursors right in there, and someone else is handling the complexity. Somehow with AI, all the magic seems to be happening, on my local laptop as long as I don't close it. And then there's, you know, the Mac Mini, meme of putting in the corner of your desk, the machine that's running your entire life. All of which kind of, seems very single-player mode, right? And so if you're, just nailing this, hill climb, thing, and you've got your own personal library of skills, and you're really eager to drag them into that shared folder so that everyone else has access, how do you keep that learning loop going and how do you make that, a multiplayer process [00:33:39] Michael Ducker: What I've seen is the current state of Anthropic and OpenAI's tools are they're single player. And so you have now graduated. If you are at this moment, you have now graduated from Claude or OpenAI being the only AI tool that you can use. Because there's a whole nother set of tools, particularly the one I'm building, valet.dev, which is one of, unfortunately many, it's a very busy space right now, where we are trying to build the software that allows, you to collaborate with other people on building agents. And so your skill evolves. Like your skill in this situation is no longer just a skill. It's a skill that's also gonna be connected to a tool. And I think going back, an agent is a s- instructions of what you want, an objective you're trying to get done, which would be the skill, with a set of tools and a feedback loop to know if it's got done or not. That is like the simplest version of an agent. And so when you start saying, "I want to share my skills" beyond maybe the Google Drive or Dropbox or GitHub version, but actually turn them into things that people are interacting with on your team, you've now graduated to being an agent builder. And agent building requires a new tool chain. With Valet, you can take any of those skills and turn them into Slack bots. And so you can literally copy and paste the file into Valet, and then we'll turn it into an interactive Slack bot that anyone on your team can talk to. And so everything you were doing in Claude all of a sudden graduates into the environment you're already chatting with. We're finding people do this with email. There's a tool called AgentMail that makes it really easy to set up an email for your AI skill, where you can give it to customers and say, "Oh, here's my knowledge base. Now you can just text it or you can just email it." And tools like the platforms like valet.dev enable you to now kind of create those experiences 'cause, because now, and this is why I think agent building is so exciting, even though you're just moving around a little text file that you've integrated with Claude, you've actually encoded a really intelligent piece of process. You've encoded something that's your opinion on how the world should work, that works within your tool chain and your processes, and now you want others to use it, and that's just, that's software. Now you're a software engineer [00:35:32] Noah Levin: We were talking about, with your co-founder on Slack, this idea that there's an analogy to, like, the way SaaS works and the way agents work. I thought this was really compelling. So I think we're all pretty used to SaaS products being sources of truth, right? So you have a roster of the people you work with, your employee directory, and, Workday is the embodiment of that in software, right? It's an operational version of that [00:35:56] Michael Ducker: It's like it's not real until it gets put into the website that you pay for. Otherwise, it's just an idea, which is like a thing I've heard from a lot of people. Like the sale's not closed until it's in the sales CRM [00:36:06] Noah Levin: It is the embodiment of that data. I thought the useful analogy was that an agent is sort of the embodiment of an SOP, of a process that happens oftentimes on that data. It's a way of thinking about this agent is not just, a sort of, set of capabilities, but it's actually like, it's a car that's about to go on a track, right? And things are gonna happen to it along that track, and you've defined that track, with your experience and adjust the track each time it runs. And one of the things I think is so hard to explain about agents, it kind of breaks my brain periodically, is when you encounter a new platform, like Valet, it's just sort of putty in your hands, right? You can shape it into so many things. When you're asked, like, what does it do? The question just kind of comes back, well, what do you want it to do? Do you have like, a favorite use case or five for, you've seen people use the platform for? Nobody knows what to do with cloud agents yet. [00:36:54] Michael Ducker: And I think that's because it's actually hard for individual work, the work that you're doing as a personal life just for yourself. You don't have that many business critical tasks that people are waiting for you with sub, you know, second response times to get an answer to. And it's that transition that we're about to go through of AI really entering the enterprise, where businesses, by being a nature of a collection of these processes, have thousands if not millions of these tasks they do every day. And so what we're seeing is, I think, people realizing that the personal experience of playing isn't fully translating to the work experience of getting things done because you personally are not spending every day trying to sell things and trying to, like, serve customers. You're trying to just, like, figure out your email, make it work, and make your calendar work. So really popular examples. I like framing these as what do you get started with? And I think the easiest thing to get started with is what I call a custom GPT, which is what OpenAI called it just in OpenAI. But it's where you embed a knowledge base, like a single document, and say, "Look, you can query this knowledge base, you can talk to it, you can ask questions." And this is really common and popular for compliance teams, for design teams when they want to talk about their brand guidelines, for legal teams, where they can basically just put a pointer to their existing Notion pages or Confluence pages, or just copy and paste all the files from Google Drive that they already use and give it as an answer of, "Okay, now you can just ask the legal team anything, and you'll get a pretty good answer." And one of the reasons I like this pattern is that the team that owns the concept, like the team that owns the brand guidelines, they get to maintain the agent and keep it up to date. And the end user just gets to treat them like they are a twenty four-seven always on, always responsive teammate. Another really common pattern that's really great to get started is to take a tool you use, like those system of records, and expose it as a Slack bot for everyone. A good example might be you only buy one seat for this expensive data tool, and you want everyone in your company to be able to have access to that tool, but it might cost, you know, thirty times more money to do that. And you don't want to spend all that money just to answer the random one-off question. And so what you do instead is you funnel questions through one person. They go open the web browser, look at the tool, then get the response back. With MCP, which is the connecting technology between AI and, and APIs, you can now expose all that data, that's your data, like it's your company data in that third-party tool you're paying by the seat- and build basically the ability for anyone to ask questions. And so you kind of democratize access to your company, which makes everyone make smarter decisions. It makes everyone hopefully better at their job. And there's no wait time, so it actually reduces the interrupts of the people who had that role of answering the field and all those questions [00:39:28] Noah Levin: The power of a good knowledge base that's easily available and easily queryable is like not to be undersold. And, for extra bonus points, once you build a knowledge base that you can access via query, other agents can access it too. And so you get this compounding benefit of, other agents become smarter and more context aware of what the right decision is or how to do a thing. And you can sort of daisy chain these things together, something that I'm very about in some of the products I'm building. [00:39:56] Michael Ducker: Another really great common example that I see is the ability to get a daily status report. It turns out that AI is phenomenally good at summarizing information. It can read and ingest faster than you could ever imagine you could read and ingest. It can find the patterns, it can find the outliers, and it can build reports for you. And you can make those reports very simple five bullets in Slack, or you can make them beautifully organic, dynamic websites with visualizations. Like, the sky's the limit. And you can do this every single day. And so one of the most common, AI sort of more advanced agents where you start connecting multiple sources together is just starting by saying, "Well, what happened in my company today? What do I need to know? What's different?" And this could be with your competitors, like, "What did my competitors do today?" An example of this that I did just literally an hour before here is we've had a lot of companies launch in our space in the last two weeks, and we went and grabbed every single ad they've put on Facebook, every single tweet from all the founders, every single part of their website, and we built like a comprehensive product marketing review, within thirty minutes that would've, that would've taken me at least weeks. Mean, this I rarely start people with that because I think it's both what they want, but it's also the hardest to get the trust around because it means that you're gonna trust the AI to write into your systems and actually deliver the value. And what we've seen in software is that this is starting developers first. A bug report comes in from a user, how do I turn it into a bug fix? And there are really great tools that predate my company that have been working on this problem for a while, and the state of the LLMs have gotten really good at coding. And this is, uh, generally a place I recommend people start. How do you post a blog post? How do you update your website? How do you fix a bug? You can do all of this just by talking to an agent that has access to your source code, has access to test your website in an environment so it can see if it did the right thing, and have access to the problem that encountered this. So like the, the end agent goals, you have all these things always doing things all the time, that are actually making an impact on your business. That's the place to get to. And you get there by kind of going through these stages and, getting comfortable with and learning how to use these tools and, inte-integrating them into your business. [00:41:47] Noah Levin: I love that and I, I learned so much from hearing the way you think about these things. [00:41:51] Michael Ducker: The last example you gave, I think that there's this ladder of complexity to the way you can, invoke the tools and build skills and build compound skills and build agents that are local and build agents that are remote. [00:42:04] Noah Levin: And, sometimes it feels like you're not doing the real thing unless you're doing the most current thing. And, one of the things I appreciate about the conversation around having a shared folder and just having a session and then encoding it as a skill is it's available to all of us right now, and it's actually building an agent, right? It's not pretend agents, it's not 2004 agents. It's actually a valid way of, you know, applying AI in your life. It's something that you and I do all the time. And it doesn't have to be, at the bleeding edge to be useful to you [00:42:37] Michael Ducker: In fact, one of the most chaotic parts of this technology wave is it's really hard to outrun the models. And so what we're finding is there's billions of dollars of tech-- money going into very successful startups whose lifespan is now being measured in months, not even years, of relevancy. Because they solved a problem, the LLM wasn't good enough, the last model, that literally goes away the next model release. And what we're seeing is that if you have a need for that level of quality at this moment, you should adopt those tools, and there's a lot of them, and I'm not gonna try to name any names. But if you're just trying to, like, play the long game here, like how do I learn a new behavior and adopt my use of computers to match to what the future's gonna be? All you need to learn is how to express what your desires are, what good looks like, and what tools you have access to, to use them. And let the AI and the LLM evolve over time to figure out how well they can solve those problems and how they actually will do it for you. But the minute you open up that box and start saying, "Well, I don't like how you did this thing. I don't like how you did this thing, and I'm gonna change it all," you are doing a very short-term optimization, is what we're seeing. My prediction would be in the long course of time, that is not the skill that's gonna be valuable [00:43:48] Noah Levin: If folks want to, hear more of what you think and talk to you, where should they go? [00:43:53] Michael Ducker: I'd love to invite people to build their agents and bring their skills to valet.dev. We give personal one-on-one support to all of our users. It's a really great easy-to-use platform. You can start with just a prompt, just copy and paste in one of those skills, and we'll get you started. Or you can hire one of our experts. We work with a lot of people who bring in their own perspectives on how to solve problems, and we allow you to use their ideas as seeds. So you can start with their idea and then make it your own before you ship it. So you're not really buying software anymore. It's not buying in this case. You're starting with. But you're not even installing somebody else's piece of software because you're getting the ideas that they have, and then you're expressing it to how you want it to be in your business. [00:44:27] Noah Levin: You're gonna find me in person, at Agent Builders Breakfast here in San Francisco weekly on Thursday mornings. You can sign up at agentbuildersbreakfast.com. [00:44:35] Michael Ducker: Noah, thank you so much for having me. This is amazing [00:44:38] Noah Levin:You're great. And I learn so much every time I talk to you, so I'm glad I got to record one of these conversations, and we'll do another one at some point soon. But, thank you for everything and, thanks for joining Serious People. Thank you for joining us for the Serious People podcast. If you like what you heard, subscribe on YouTube, leave us a review on iTunes, or sign up for our newsletter at seriouspeople.ai/podcast. When we're not podcasting, Serious People helps businesses put AI to work in their daily operations. Visit us at seriouspeople.ai to learn more. The Serious People podcast is sponsored by Valet.dev. Go try building your own agent today at valet.dev or click the link in the show notes. See you next week.