Latest / How I Work / Designing prompts that save you hours every week with Dropbox’s Morgan Brown
Transcript
- 0:01Speaker 1: What do you do when something as ordinary as dinner
- 0:04Speaker 1: turns into chaos? For Morgan Brown, vice President of Product
- 0:09Speaker 1: Management for AI at Dropbox, it was trying to work
- 0:13Speaker 1: out the cabs in his six year old son's meals.
- 0:18Speaker 1: Endless Google searchers stress at the dinner table, so he
- 0:22Speaker 1: built an app with no coding background, using AI to
- 0:26Speaker 1: solve it. That same problem solving mindset runs through his
- 0:31Speaker 1: work at Dropbox, where Morgan leads the company's AI products.
- 0:36Speaker 1: He's using tools like CHATDPT and Claude as sparring partners,
- 0:41Speaker 1: not just search engines, automating the grunt work, refining strategy documents,
- 0:46Speaker 1: even filtering AI research papers while he has his morning coffee.
- 0:52Speaker 1: In this conversation, you'll learn how to spot your hidden
- 0:55Speaker 1: time sinks, how to design prompts that actually work, and
- 1:00Speaker 1: how to use AI as a genuine thought partner. And
- 1:04Speaker 1: the result more bandwidth for the work and the life
- 1:08Speaker 1: that really matters. Welcome to How I Work, a show
- 1:19Speaker 1: about habits, rituals, and strategies for optimizing your day. I'm
- 1:23Speaker 1: your host, doctor Amantha Imber. I want to start by
- 1:31Speaker 1: talking about carb scan, which I read your post on
- 1:35Speaker 1: LinkedIn as something that you're doing on the side, Can
- 1:38Speaker 1: you tell me how this came about? Yes, because you're busy,
- 1:41Speaker 1: You've got like an important job at dropbox to do.
- 1:44Speaker 2: I'm actually a tapeline diabetic, so really important to manage
- 1:48Speaker 2: my blood sugar. I have, you know, an insulin pump
- 1:51Speaker 2: and blood glucose monitor and have been dealing with that
- 1:55Speaker 2: most of my adult life. But my son, who is six,
- 1:59Speaker 2: was also recently diagnosed with type one diabetes. And really
- 2:04Speaker 2: for little kids, it adds a ton of complexity to
- 2:08Speaker 2: your day because their bloodshair has to be monitored very closely.
- 2:11Speaker 2: And what I found was pretty much every meal turned
- 2:14Speaker 2: into grantically searching on Google for how many carbs are
- 2:18Speaker 2: in chicken nuggets or mac and cheese and trying to
- 2:22Speaker 2: add it up, and it really interferes with Hey, everyone's
- 2:24Speaker 2: trying to sit down and eat. And so I built
- 2:26Speaker 2: a little app that allows me and my family or
- 2:29Speaker 2: anyone that wants to try it to take a picture
- 2:32Speaker 2: of the food. I take that picture, I send it
- 2:35Speaker 2: to open Ai with a really detailed prompt, and then
- 2:38Speaker 2: reference calorie King, which is a kind of canonical source
- 2:42Speaker 2: of carbohydrate and other nutritional data on the web, to
- 2:45Speaker 2: give me carbohydrate estimates back for his meals, rather than
- 2:49Speaker 2: three or four different Google searches across a bunch of sites.
- 2:52Speaker 2: I get a very quick answer back that's really just
- 2:56Speaker 2: taken a ton of friction out of three times a
- 2:59Speaker 2: day or snacks included just kind of snapping photos. And
- 3:03Speaker 2: it built it using a vibe coding AI tools. You know,
- 3:06Speaker 2: I don't know a lot of code myself, and so
- 3:09Speaker 2: basically used AI to build new AI to solve the problem.
- 3:13Speaker 3: I love that.
- 3:13Speaker 1: I want to talk about the tools, and I love
- 3:15Speaker 1: that you've mentioned that you're not a coding guy, because
- 3:19Speaker 1: people might be listening going, well, that's easy for a
- 3:22Speaker 1: tech guy to say, so vibe coding, and it might
- 3:25Speaker 1: be worth just defining what vibe coding is for those
- 3:27Speaker 1: that don't know. And then I want to ask which
- 3:29Speaker 1: which are the tools you used.
- 3:30Speaker 2: The idea behind vibe coding is to use an LM
- 3:34Speaker 2: or an AI agent to describe the type of software
- 3:38Speaker 2: you want to build, and then have it build it
- 3:40Speaker 2: for you. And that can be how it looks, how
- 3:43Speaker 2: it works, and really you just use natural language to
- 3:46Speaker 2: describe it. And I think the reason they call it
- 3:48Speaker 2: vibe coding is that you don't have to be so
- 3:51Speaker 2: strict with kind of precise requirements and precise coding and
- 3:55Speaker 2: you know, making sure there's no typos, and so it's
- 3:57Speaker 2: really a very fluid creative process. And I used a
- 4:01Speaker 2: tool called Replet, which is a great tool for building
- 4:04Speaker 2: web applications, and built everything from the front end design
- 4:08Speaker 2: or I just explained what I wanted it to look like,
- 4:10Speaker 2: to the prompts that get sent to open AI, to
- 4:14Speaker 2: the way that it does the carb estimates and all
- 4:17Speaker 2: of that. So yeah, I've been working on it for
- 4:19Speaker 2: about a month and a half here and there, and
- 4:21Speaker 2: it's been fun to see it kind of come together.
- 4:23Speaker 3: That's amazing.
- 4:24Speaker 1: Did you use any other tools, like were using chat
- 4:26Speaker 1: chapte to help with how you prompted replic because I must.
- 4:31Speaker 1: I've played around with replet a little bit. I built
- 4:34Speaker 1: an app that had so many bugs in it and
- 4:36Speaker 1: then I just got sick of it and abandoned that
- 4:40Speaker 1: little project. But was it just Replet or were we
- 4:42Speaker 1: using other tools to help build that?
- 4:44Speaker 2: You know, caps can I definitely use chat gpt as
- 4:48Speaker 2: kind of my software thought partner, you know, so I
- 4:51Speaker 2: would go to chat gept and say, hey, I want
- 4:53Speaker 2: to write a prompt for an LLLM to give me
- 4:58Speaker 2: a really accurate carboh great estimate from a picture of food.
- 5:01Speaker 2: Could you give me some suggestions on how to write
- 5:04Speaker 2: a really great prompt? And it gave me a bunch
- 5:06Speaker 2: of detailed instructions. I kind of took that, worked on it.
- 5:10Speaker 2: Then I gave that to a Replet And actually, now
- 5:13Speaker 2: my current workflow is I have an idea for a
- 5:16Speaker 2: feature for carb scan. I go to chat GPT and
- 5:19Speaker 2: I say, I now have a project within chat GPT
- 5:22Speaker 2: that is just carb scan, so it has a whole
- 5:24Speaker 2: bunch of historical knowledge there. And I say, I have
- 5:27Speaker 2: this new feature, I'd like it to do X, Y
- 5:29Speaker 2: and Z. Can you help me write a really clear
- 5:33Speaker 2: product requirement and specification doc to give to Replet to
- 5:36Speaker 2: help me get this feature built and it will give
- 5:39Speaker 2: me that prompt and I take that and then I
- 5:41Speaker 2: paste that into Replet and I ask Replet to review
- 5:44Speaker 2: the requirements and then come back with a plan to
- 5:47Speaker 2: implement it, and then I approve the plan and it
- 5:49Speaker 2: starts to build.
- 5:50Speaker 3: So good.
- 5:51Speaker 1: Now, of course, building carbohydrate scanning apps is not your
- 5:56Speaker 1: day job working at dropbox is. Can you tell me
- 5:59Speaker 1: in the nutshell, what do you do at Dropbox?
- 6:03Speaker 2: My day job is I'm the Vice president of Product
- 6:06Speaker 2: Management for AI at Dropbox, where I'm responsible for the
- 6:10Speaker 2: AI products that we build at Dropbox to help people
- 6:13Speaker 2: be more effective at their job and with their work,
- 6:16Speaker 2: and so that means I'm primarily responsible for Dropbox Dash.
- 6:20Speaker 2: Dropbox Dash is an AI powered search knowledge management work
- 6:24Speaker 2: assistant that helps teams be more effective, get busy work
- 6:29Speaker 2: out of the way, and hopefully allow them to do
- 6:32Speaker 2: better work at more of the empowering, meaningful work that
- 6:35Speaker 2: we all set out to do every day that can
- 6:37Speaker 2: often get buried under a bunch of project management.
- 6:39Speaker 1: I would love to know about some of your AI workflows,
- 6:45Speaker 1: I guess, both in terms of how you use AI
- 6:47Speaker 1: to help with some of the grunt work, but also
- 6:50Speaker 1: how you use it to help augment your thinking as well.
- 6:53Speaker 1: So maybe let's start by talking about the grunt work,
- 6:56Speaker 1: the shallow work. Take me through some of the ways
- 7:00Speaker 1: that you're using AI. I'd love to get into real
- 7:03Speaker 1: specifics as to what you've designed for you.
- 7:06Speaker 2: One of the biggest challenges with my work is that
- 7:08Speaker 2: I'm constantly switching between all sorts of different contexts and
- 7:12Speaker 2: problems to solve. So it could be a product feature
- 7:15Speaker 2: over here, thinking about how to launch a new feature
- 7:18Speaker 2: over here, or communicate how feature works over here, looking
- 7:21Speaker 2: at the roadmap, reviewing something to ship out to customers,
- 7:25Speaker 2: working with our marketing team, and so constantly switching contexts,
- 7:29Speaker 2: and over the course of a week, I'll be in
- 7:31Speaker 2: dozens of meetings, and so one of the things I
- 7:34Speaker 2: do is I use AI to recap all of the
- 7:37Speaker 2: things that I've promised to follow up with for everyone,
- 7:41Speaker 2: all of the things that other people have promised to
- 7:43Speaker 2: get back to me on and big decisions or milestones
- 7:46Speaker 2: or risks to watch out for. So Dropbox uses Zoom,
- 7:49Speaker 2: which is kind of the video meeting software. It records
- 7:54Speaker 2: the meetings pretty much by defaults for the most part,
- 7:56Speaker 2: so I am able to take those transcripts and turn
- 8:00Speaker 2: those into summaries, you know, at the start of every week.
- 8:03Speaker 2: So it helps me reset context and stay kind of
- 8:06Speaker 2: on top of that. So that's one big use case,
- 8:08Speaker 2: and that's really helped with you know, I used to
- 8:10Speaker 2: write it all down in chicken scratch. My handwriting is terrible,
- 8:14Speaker 2: and try to go back through and remember who I
- 8:16Speaker 2: promised what to and so now that's very seamless and
- 8:20Speaker 2: just happens, you know, pretty much automatically for me.
- 8:22Speaker 4: So that's one big one.
- 8:23Speaker 3: Yeah, cool, What are a couple of others.
- 8:25Speaker 2: So another one of the things that I do all
- 8:28Speaker 2: the time is I'm reading documents constantly. So do these
- 8:31Speaker 2: product requirement documents or specifications or strategy briefs or memos
- 8:36Speaker 2: about to go to market approach? And I found that
- 8:38Speaker 2: over time, I have the same questions over and over.
- 8:42Speaker 2: Who are we building this for? What's the job to
- 8:44Speaker 2: be done? How do we know that this is the
- 8:46Speaker 2: right thing to build? Why does what we're proposing make
- 8:49Speaker 2: sense relative to everything else we know? And so I
- 8:52Speaker 2: basically built a prompt for DASH, which is our internal
- 8:57Speaker 2: AI tool, to help me pre read those documents and
- 9:00Speaker 2: evaluate those documents along the same sets of questions that
- 9:03Speaker 2: I ask all the time. So I've got, you know,
- 9:06Speaker 2: twelve questions that I ask all the time. I run
- 9:08Speaker 2: it through the prompt and it helps me surface areas
- 9:10Speaker 2: to look out for in the document, areas that I
- 9:13Speaker 2: might want to prove a bit deeper. And I found
- 9:15Speaker 2: it so effective that I actually just said emailed or
- 9:18Speaker 2: I messaged the prompt to my entire team and said, hey,
- 9:21Speaker 2: I'm using this myself. If you want to use it
- 9:24Speaker 2: before you send something to me, you know, it might
- 9:26Speaker 2: save us a few cycles.
- 9:28Speaker 4: So I just sent it to them. Now they use
- 9:29Speaker 4: it before they send it to me.
- 9:31Speaker 2: And so now I'm kind of taking that the next
- 9:33Speaker 2: step and really trying to think about, Hey, based on
- 9:36Speaker 2: this strategy or this approach, you know, what are some
- 9:39Speaker 2: second order effects that I may not be aware of.
- 9:42Speaker 2: What are some ways to potentially double down on this
- 9:45Speaker 2: or differentiate and make this an even stronger proposal. What
- 9:49Speaker 2: are some risks that may be hidden here that I'm
- 9:52Speaker 2: not considering? How might my CTO react to this? How
- 9:56Speaker 2: might our legal team react to this? How might our
- 9:58Speaker 2: go to market team react to it? So really trying
- 10:01Speaker 2: to basically roleplay and game theory out all of these
- 10:05Speaker 2: different inputs that would typically take weeks to get.
- 10:07Speaker 1: And when you're constructing a prompt like that, are you
- 10:11Speaker 1: typically using because what is called meta prompting where you're
- 10:15Speaker 1: you know what you described earlier, where you're working with
- 10:19Speaker 1: the AI to help improve the prompt or even write
- 10:22Speaker 1: the prompt, Like do you have a process when you're
- 10:25Speaker 1: I guess, creating one of those fundamental prompts in your
- 10:28Speaker 1: prompt library, and you want to make it really good.
- 10:30Speaker 3: How do you, I guess, build a great prompt?
- 10:33Speaker 2: Yeah, absolutely, And I think that is really a skill
- 10:36Speaker 2: that knowledge workers can really turn into a superpower now.
- 10:39Speaker 2: And so I think it's thinking about what is the
- 10:43Speaker 2: optimal amount of context you can give the LM to
- 10:47Speaker 2: give you the most relevant and best answer possible. And
- 10:50Speaker 2: so in my prompt that's basically what I'm trying to do.
- 10:53Speaker 4: Now.
- 10:54Speaker 2: You can't give it too much information or it kind
- 10:56Speaker 2: of gets overwhelmed. But the more that you can give
- 10:59Speaker 2: it that it is highly relevant, the better the answers
- 11:01Speaker 2: you get. And one of the main principles that I
- 11:04Speaker 2: use is that lms read things in context of everything
- 11:08Speaker 2: they've already read. So the initial pieces of information you
- 11:12Speaker 2: put into it are going to influence how it evaluates
- 11:16Speaker 2: the rest of it. And so, for example, one of
- 11:18Speaker 2: the tips that you often hear and I actually have
- 11:20Speaker 2: found to work really effectively is that when you're talking
- 11:22Speaker 2: about a domain specifically, it's important to state up front
- 11:27Speaker 2: that this is the domain you're talking about. So, for example,
- 11:29Speaker 2: if I'm working on positioning. I'm interested in how we
- 11:32Speaker 2: might position a dash feature. I'll say you are April Dunford,
- 11:38Speaker 2: author of this positioning book, or you are Al Reese
- 11:41Speaker 2: and Jack Trout of Positioning to Win. Help me position
- 11:46Speaker 2: this product feature. These are its capabilities, This is how
- 11:50Speaker 2: I think it could work. I think these are the challenges,
- 11:54Speaker 2: this is what's differentiated about it.
- 11:56Speaker 4: These are who it might compete.
- 11:57Speaker 2: With, and these are the ways where we would like
- 12:00Speaker 2: it to connect to the rest of our product suite.
- 12:03Speaker 2: And then it'll you know, kind of structuring it that
- 12:05Speaker 2: way where it's kind of a here's some context. Then
- 12:08Speaker 2: there's a very specific task or set of tasks that
- 12:11Speaker 2: you want to give it, and then you can give
- 12:13Speaker 2: it detailed directions around how you want it to accomplish
- 12:16Speaker 2: those tasks. So, for example, sometimes the lllms can be
- 12:20Speaker 2: very verbose, but I like very terse, very pragmatic answers,
- 12:24Speaker 2: So I always tell it no fluff, no hyperbole, consider
- 12:28Speaker 2: this in a very you know, you're not going to
- 12:30Speaker 2: hurt my feelings if you tell me it's a bad idea.
- 12:33Speaker 2: You know, just really trying to kind of get it
- 12:35Speaker 2: honed in with the language that I or the thought
- 12:38Speaker 2: process that I kind of want it to laid up,
- 12:40Speaker 2: and so context task output and then detailed instructions are
- 12:45Speaker 2: usually the format that I use to write these.
- 12:47Speaker 1: Still staying on the topic of I guess reducing or
- 12:50Speaker 1: eliminating some of the grunt work through AI, I'd love
- 12:54Speaker 1: to know if they're more on the automation or a
- 12:57Speaker 1: gentic AI side of the what some of the I
- 13:02Speaker 1: guess other tasks that you've been able to automate in
- 13:05Speaker 1: interesting ways would be.
- 13:06Speaker 2: I've really done a lot of automation around trying to
- 13:10Speaker 2: understand what's changing around me. So I have a daily
- 13:13Speaker 2: workflow that reviews all new kind of substack newsletters, important
- 13:18Speaker 2: tweets across a large domain of AI and mL X accounts.
- 13:23Speaker 2: It reviews Archive, which is the scientific paper repository for
- 13:29Speaker 2: new mL and artificial intelligence papers. It scans the whole
- 13:33Speaker 2: landscape YouTube, Spotify podcasts, and basically takes all of the
- 13:38Speaker 2: day's information about AI and mL reviews it all categorizes
- 13:42Speaker 2: it into world changing, interesting, incremental, and noise, and then
- 13:49Speaker 2: summarizes the top three quote unquote world changing potentials and
- 13:52Speaker 2: then summarizes it for me in the context of my
- 13:55Speaker 2: role at Dash where it kind of talks about hey,
- 13:58Speaker 2: things you might consider as the head of product for Dash. So,
- 14:01Speaker 2: for example, Anthropic the other day just announced that Claud
- 14:04Speaker 2: can handle now a million token context window, which effectively
- 14:09Speaker 2: means that for many reasonable size software codebases, it can
- 14:13Speaker 2: reason about the entire codebase, which is a pretty big
- 14:15Speaker 2: and important milestone for software development. And the other one
- 14:18Speaker 2: is I automate my personal email. I can't imagine you're
- 14:21Speaker 2: inbox Atmantha, but mine it gets hammered all the time.
- 14:24Speaker 4: And so I built a.
- 14:26Speaker 2: Little agent using Claud code to read my personal Gmail
- 14:31Speaker 2: every day, categorize it into newsletters, spam priority emails, summarize
- 14:38Speaker 2: the newsletters for me, flag the messages that I want
- 14:40Speaker 2: to respond to, and draft responses for those newsletters or
- 14:46Speaker 2: not for those emails. So really trying to kind of
- 14:49Speaker 2: like manage my inbound email automatically as well.
- 14:53Speaker 3: Wow, okay, I love the both those examples.
- 14:55Speaker 1: I want to dig a bit, Dafa into how you
- 14:57Speaker 1: went about creating this automation that summarizes all the AI
- 15:04Speaker 1: news of the day.
- 15:05Speaker 3: I mean that sounds so powerful. Can I buy that
- 15:08Speaker 3: from you?
- 15:09Speaker 1: Morgan?
- 15:09Speaker 3: Are you selling it?
- 15:11Speaker 4: You can link it in the show notes.
- 15:13Speaker 3: Wow amazing handed over Yeah Wow. Talk me through the
- 15:17Speaker 3: process of how you put that together.
- 15:19Speaker 2: First of all, I realized that there were a few
- 15:21Speaker 2: key sources of information that I was constantly referring to personally,
- 15:25Speaker 2: So you know, whenever I had a free moment, I
- 15:27Speaker 2: would go to archive and kind of browse the actual
- 15:31Speaker 2: published papers because that's the real source of truth on
- 15:34Speaker 2: the cutting edge. I would see a lot of popular
- 15:37Speaker 2: posts on x about kind of new research breakthroughs. I
- 15:42Speaker 2: follow some of the key people like the Sam Altman's
- 15:44Speaker 2: of the World and the Frontier Labs, and then there's
- 15:47Speaker 2: also several great substack newsletters around AIS that I read
- 15:52Speaker 2: as much as I can. But I just realized there
- 15:55Speaker 2: was no way for me to stay on top of
- 15:56Speaker 2: that volume consistently, and I just started with a prompt.
- 16:00Speaker 2: I told CHATGBT, I'd like to create a prompt to
- 16:02Speaker 2: scan the entire AIML space for important signals around research
- 16:08Speaker 2: and development and industry news that pertains to my role
- 16:12Speaker 2: at Dropbox. And I said, if for X, I want
- 16:15Speaker 2: to look across these accounts and accounts like it. So
- 16:18Speaker 2: I kind of gave it a seed list of accounts,
- 16:20Speaker 2: but asked it to expand where it could. I gave
- 16:23Speaker 2: it specific newsletters that I read on substack, but also
- 16:27Speaker 2: again asked to consider adjacents. And then I gave it
- 16:30Speaker 2: specific categories on archive where those papers are housed and
- 16:34Speaker 2: asked it to consider relevant ones that there. And then
- 16:38Speaker 2: I said, I'd like to really filter high signal, like
- 16:42Speaker 2: I don't want this to be a laundry list of
- 16:44Speaker 2: everything that happened that when actually helped me kind of
- 16:47Speaker 2: consume it any better. So I said, help me come
- 16:49Speaker 2: up with a scale to rank the innovations and the
- 16:53Speaker 2: news that's coming out in the day and kind of
- 16:54Speaker 2: categorize things as can't miss, these are like foundational important,
- 16:58Speaker 2: shifts increment and important, and then give me some synthesis
- 17:02Speaker 2: around hey, when you put these signals together, what does it.
- 17:06Speaker 4: Mean for my role?
- 17:07Speaker 2: So I fed that all just in basically talking to
- 17:11Speaker 2: chat GPT. I like to use the audio input to
- 17:15Speaker 2: kind of just stream of consciousness my thoughts into it.
- 17:17Speaker 2: Once I did that, I asked it to kind of
- 17:19Speaker 2: structure it into a prompt. I reviewed the prompt. Some
- 17:22Speaker 2: of the things I didn't like. I always try to
- 17:24Speaker 2: go back and forth with it. I think you should
- 17:27Speaker 2: kind of refine it, much like you with the copy
- 17:29Speaker 2: for a website or the script for something. And then
- 17:32Speaker 2: once I got the prompt to a pretty good spot,
- 17:35Speaker 2: I said, okay, let's run that for today and give
- 17:38Speaker 2: me the output.
- 17:38Speaker 4: And I got the.
- 17:39Speaker 2: Output and I was like, oh, this is a little
- 17:40Speaker 2: too long. I could be a little shorter. It looks
- 17:43Speaker 2: like we're underrepresenting, we're missing YouTube. Let's try to add
- 17:46Speaker 2: YouTube in so on and so forth. And then I
- 17:48Speaker 2: finally got it dialed in and then I said, great,
- 17:50Speaker 2: set this as a daily task every day five am Pacific.
- 17:54Speaker 2: Run it again while I'm having coffee. I just kind
- 17:56Speaker 2: of read through it.
- 17:57Speaker 1: Wow, Yes, I would love if you could that, and
- 18:00Speaker 1: I will pop that in the show notes. Let's talk
- 18:03Speaker 1: about how you use AI to augment your thinking. I recently,
- 18:08Speaker 1: I don't know if you know Bobb Johansson. He's a
- 18:10Speaker 1: futurist over in Silicon Valley and I interviewed him a
- 18:13Speaker 1: few weeks ago and he was talking about how, you know,
- 18:17Speaker 1: as many people have said, artificial intelligence is such a
- 18:20Speaker 1: bad brand name, and he said, if he was going
- 18:22Speaker 1: to name it, he would call it augmented intelligence. And
- 18:25Speaker 1: it's stuck with me, and I feel like I use
- 18:27Speaker 1: it a lot to augment my own thinking. But I
- 18:30Speaker 1: feel like most people talk about, you know, the efficiency
- 18:32Speaker 1: gains and the productivity gains. But I'd love to know,
- 18:35Speaker 1: for you, Morgan, what are the different ways or use
- 18:38Speaker 1: cases that you think about and use AI for to
- 18:42Speaker 1: improve your thinking.
- 18:43Speaker 2: I think a lot of people looks AI for answers,
- 18:46Speaker 2: but it's not really the best search engine. You know,
- 18:49Speaker 2: it can make things up, it can have outdated information.
- 18:52Speaker 2: I think it's much better to use it as kind
- 18:54Speaker 2: of a thought partner or sparing partner. And so I
- 18:57Speaker 2: think a lot of the techniques that you kind of
- 18:59Speaker 2: learn learn in business and brainstorming and communication and strategy
- 19:04Speaker 2: work really well with lllms. So the first thing I
- 19:07Speaker 2: try to do is again go back to, like what
- 19:10Speaker 2: kind of context does the LLLM need to know in
- 19:13Speaker 2: order to be a good thought partner for me? You know,
- 19:16Speaker 2: like you wouldn't go ask someone who works in sporting
- 19:20Speaker 2: goods about a medical condition, you know, and you're not
- 19:23Speaker 2: going to ask your doctor about how to score a
- 19:25Speaker 2: goal in soccer unless they used to play in a
- 19:27Speaker 2: university or something. But so, really, like, what does the
- 19:30Speaker 2: LLLM need to know to be successful. And so with DASH,
- 19:34Speaker 2: we have this concept called stacks, which are collections of
- 19:37Speaker 2: documents that you can create around a given topic, and
- 19:40Speaker 2: then dash chat uses that collection of documents to augment
- 19:45Speaker 2: its context and understanding of the problem space. But I
- 19:48Speaker 2: generalize this to chat, GBT or anthropic or any of
- 19:53Speaker 2: the other models where I try to say, Okay, what
- 19:56Speaker 2: does it need to know about me and the problem
- 19:57Speaker 2: I'm trying to solve first to be highly relevant, and
- 20:00Speaker 2: then go create that information. So, for example, one of
- 20:03Speaker 2: the things I've been that I started early in my
- 20:06Speaker 2: career was a Morgan's Operating Manual. So anyone that was
- 20:10Speaker 2: kind of going to work with me join my team.
- 20:13Speaker 2: It became really important when I became a manager to
- 20:15Speaker 2: help people understand, you know, here's what Morgan prioritizes, Here's
- 20:20Speaker 2: how he likes to communicate, here's the things that add friction.
- 20:23Speaker 2: It's like my operating manual that you would kind of
- 20:25Speaker 2: take a quick start guide for working with Morgan working
- 20:28Speaker 2: with an LM. That's one of the first documents I
- 20:31Speaker 2: give it are my operating principles, and it has things
- 20:34Speaker 2: like no fluff, what you don't know, say you don't
- 20:37Speaker 2: know no politics, you know that type of thing.
- 20:40Speaker 4: And then I've.
- 20:40Speaker 2: Started to build out those contextual documents. So, for example,
- 20:44Speaker 2: I've really been working lately on what are my core
- 20:47Speaker 2: principles as a person, as a father, as a husband,
- 20:51Speaker 2: what are the things that I care about in life generally,
- 20:54Speaker 2: and using an LLLM to codify those, and then I
- 20:57Speaker 2: can give it even domain specific things. So for example,
- 21:00Speaker 2: with carb scan, my priorities with carb scan are like,
- 21:03Speaker 2: accuracy is the most important thing, no false, confidence, delight
- 21:08Speaker 2: and speed are you know essential And so now it
- 21:12Speaker 2: has a working set of information to be a really
- 21:14Speaker 2: good thought partner. Once I do that, then I try
- 21:16Speaker 2: to use the techniques that we use to kind of
- 21:19Speaker 2: come up with good ideas all the time. So I'll say,
- 21:21Speaker 2: instead of asking AI for an answer about something, I'll say, Hey,
- 21:24Speaker 2: I have this problem I'm trying to solve. Let's do
- 21:26Speaker 2: some divergent thinking together. I want to be generative. Help
- 21:30Speaker 2: me be generative about the ideas here, So lead me
- 21:32Speaker 2: through a series of questions to help diverge and push
- 21:36Speaker 2: my thinking. So you know, in brainstorm, I use divergent
- 21:39Speaker 2: thinking to get as many options on the table. And
- 21:41Speaker 2: then once I feel pretty good about that, I'll start
- 21:44Speaker 2: to say, Okay, now let's now go through some conversion
- 21:47Speaker 2: thinking cycles.
- 21:48Speaker 4: Okay, now let's.
- 21:49Speaker 2: Drive down here, and then kind of through those questions,
- 21:52Speaker 2: I'll say, like, hey, is there a framework that could
- 21:56Speaker 2: help articulate how we're making these decisions? Like what are
- 21:59Speaker 2: some the implied principles about some of the decisions we're
- 22:03Speaker 2: making in this session? You know, is there a two
- 22:06Speaker 2: by two grid of you know, feature importance versus priority,
- 22:09Speaker 2: or user engagement versus monetization, or some other type of
- 22:13Speaker 2: way to kind of like help frame up the decisions.
- 22:16Speaker 2: And then look for like I mentioned earlier, like Okay,
- 22:19Speaker 2: now that we have this set of ideas, how might
- 22:21Speaker 2: we push them further? Are their power laws inherent?
- 22:24Speaker 1: Here?
- 22:25Speaker 2: Are there limiting steps, you know, things that must be
- 22:28Speaker 2: true for this to work? Like what are those? And
- 22:31Speaker 2: so that's really how I go back and forth with
- 22:33Speaker 2: them as a thinking partner. And then I try to
- 22:35Speaker 2: take the output of one. So if I do this
- 22:38Speaker 2: with chat GPT, I'll take its output and I will
- 22:40Speaker 2: give it to Claude and I will say I went
- 22:42Speaker 2: through this exercise of the chat GPT here's its recommendations.
- 22:45Speaker 4: What do you think.
- 22:46Speaker 1: So it's tempting to think that AI's biggest value is efficiency,
- 22:50Speaker 1: but Morgan.
- 22:51Speaker 3: Shows us that that is just the surface.
- 22:55Speaker 1: Because coming up we dig into experimentation, how tools like
- 22:59Speaker 1: chat JPT can simulate real world audiences, test hundreds of
- 23:03Speaker 1: ideas in minutes, and even reshape how you think about
- 23:07Speaker 1: what makes a strong hypothesis. We also get into the
- 23:11Speaker 1: risks of skipping humans all together, why speed can sometimes backfire,
- 23:16Speaker 1: and the surprising ways that Morgan stress tests his own ideas.
- 23:21Speaker 1: Stay tuned because this half gets very practical and very
- 23:25Speaker 1: eye opening. If you're looking for more tips to improve
- 23:31Speaker 1: the way you work can live. I write a short
- 23:34Speaker 1: weekly newsletter that contains tactics I've discovered that have helped
- 23:38Speaker 1: me personally. You can sign up for that at Amantha
- 23:41Speaker 1: dot com. That's Amantha dot com. I want to shift
- 23:49Speaker 1: into talking about experimentation, and but before we started recording,
- 23:53Speaker 1: I was showing you my very scribbled on dogged Coffee
- 23:58Speaker 1: of Hacking Growth, which I love this It came out
- 24:01Speaker 1: quite a few years ago now, but it's such a
- 24:03Speaker 1: great book, and I think about experimentation a lot. My
- 24:07Speaker 1: consultancy inventing part is what we do is we build
- 24:10Speaker 1: innovation capability for our clients. And you know, something I
- 24:14Speaker 1: have been thinking about in the last couple of years
- 24:16Speaker 1: is just how some of those fundamental principles of experimentation
- 24:20Speaker 1: and even some of the principles that like you know,
- 24:23Speaker 1: Eric Reaes and Steve Blank gave us with the lean
- 24:26Speaker 1: startup methodology and the great work in principles that you
- 24:31Speaker 1: wrote about in Hacking Growth.
- 24:32Speaker 3: How has your thinking evolved now with all these.
- 24:35Speaker 1: Different AI tools that we've got access to when it
- 24:38Speaker 1: comes to testing an idea.
- 24:40Speaker 4: Yeah, that's a great question, and you're right.
- 24:42Speaker 2: Your Hacking Growth, I think will be I think it's
- 24:44Speaker 2: eight years old at this point, and so I was
- 24:46Speaker 2: so excited to see.
- 24:47Speaker 4: That you had a copy.
- 24:48Speaker 2: When I reflect on it, I think the principles of
- 24:51Speaker 2: it still stand up very well. Like the first half
- 24:53Speaker 2: of the book kind of talking about, hey, you really
- 24:55Speaker 2: need to understand the problem deeply, using that understanding to
- 25:00Speaker 2: generate new potential ideas to test. And then the main
- 25:03Speaker 2: things that have changed are one, you can now go
- 25:06Speaker 2: through that loop much faster. Either you can kind of
- 25:09Speaker 2: work offline before you kind of move online. So, for example,
- 25:13Speaker 2: if you're going to test a bunch of headlines for
- 25:17Speaker 2: an advertisement or any email subject line. In the old days,
- 25:21Speaker 2: you would sit down, you would brainstorm with a team.
- 25:23Speaker 2: You'd come up with four or five. You kind of
- 25:25Speaker 2: debate which two you wanted to test. You would take
- 25:28Speaker 2: those two, you'd put them into your email software. You'd
- 25:30Speaker 2: send it to like ten percent of the list. You
- 25:32Speaker 2: would see which one was winning, and then you would
- 25:34Speaker 2: send that to the rest of the list, and then
- 25:36Speaker 2: you'd say, okay, for next time, this one one, and
- 25:39Speaker 2: so we'll use that as a kind of a starting
- 25:41Speaker 2: point to kind of move onto the next one. And
- 25:43Speaker 2: that was kind of the speed of the loop, which
- 25:46Speaker 2: was really gated by how much time it took to
- 25:48Speaker 2: do all that. Now today I can say I need
- 25:51Speaker 2: an email subject line chat you BT give me five immediately,
- 25:56Speaker 2: you get it instantly, and then you can start to
- 25:58Speaker 2: I think what's really interesting is you can start to simulate, hey,
- 26:02Speaker 2: how might this perform generally? You know, and the lllms
- 26:06Speaker 2: have so much context around the baseline industry open rates,
- 26:10Speaker 2: by email types, and you know, is to continue the example,
- 26:13Speaker 2: and so you can go through that learning loop much faster,
- 26:16Speaker 2: so that hopefully by the time you actually get down
- 26:19Speaker 2: to testing with that email, you've gone from maybe five
- 26:24Speaker 2: initial ideas that were kind of like generally concocted to
- 26:27Speaker 2: maybe you started now with twenty or one hundred ideas
- 26:30Speaker 2: which have gone through a series of you know, feedback
- 26:33Speaker 2: loops down to now two or three or five that
- 26:36Speaker 2: you feel really good about. And that takes you know,
- 26:38Speaker 2: thirty minutes versus you know, days and a couple of
- 26:41Speaker 2: meetings with your team and all of that. And so
- 26:43Speaker 2: I think that applies everywhere now. It applies to you know,
- 26:48Speaker 2: the subject lines of your email, your ad copy, the
- 26:51Speaker 2: landing page copy, the what shows up in your app.
- 26:54Speaker 2: To give you an example, with carb scan, one of
- 26:57Speaker 2: the things that I found coming back to it multiple
- 26:59Speaker 2: times a day was it was pretty sterile.
- 27:01Speaker 1: You know.
- 27:01Speaker 4: Every time he came back.
- 27:02Speaker 2: It said the same exact thing and said just you know,
- 27:05Speaker 2: know your carbs in a snap. And I was like, well,
- 27:08Speaker 2: this is my fifth time here today, you should know
- 27:10Speaker 2: a little bit something about me. So I just said,
- 27:13Speaker 2: you know, hey, I want to create it so that
- 27:14Speaker 2: when I come back, based on the time of day,
- 27:17Speaker 2: respond to that. So now when you come back at
- 27:19Speaker 2: dinner time, it's like dinner time, we've got you covered,
- 27:22Speaker 2: you know, get going with a snap, or like if
- 27:24Speaker 2: you go late at night, it's like late night snacking.
- 27:27Speaker 4: We've got your carbs counted. And that just took minutes.
- 27:29Speaker 1: I'd love to get into a bit more detail around
- 27:32Speaker 1: just your process for testing within an LLM without even
- 27:37Speaker 1: having to find a customer to open an email, and
- 27:41Speaker 1: maybe if you could walk me through an example, maybe
- 27:43Speaker 1: even with cabscan, Like let's just say you were testing
- 27:47Speaker 1: two subject lines for an email where the purpose of
- 27:51Speaker 1: the email.
- 27:51Speaker 3: Was get to get people to trial the app.
- 27:54Speaker 1: Can you talk me through like what that flow would
- 27:56Speaker 1: look like to test it, but just within an LLM.
- 27:59Speaker 2: Well, I think you get to get really wonky here,
- 28:02Speaker 2: so I'll try to kind of like break it down
- 28:04Speaker 2: a little bit. So one of the things that you
- 28:06Speaker 2: can do is you can ask the LLLM.
- 28:09Speaker 4: To simulate a persona.
- 28:11Speaker 2: So, for example, one of the things you can say is, hey,
- 28:14Speaker 2: you are a busy working mom with three kids. So
- 28:19Speaker 2: you use the carpskin example working mom, three kids, one
- 28:21Speaker 2: of them is a Type one diabetic. You typically eat
- 28:24Speaker 2: out a couple of times a week, but you know,
- 28:26Speaker 2: you're very on top of their diabetes regimen. You have
- 28:29Speaker 2: a bunch of mobile apps around kind of like managing
- 28:33Speaker 2: their blood glucose, and you know you're trying to make
- 28:36Speaker 2: the best informed decisions you can, but like you have
- 28:38Speaker 2: a very modern and hectic life, like we all do,
- 28:40Speaker 2: kind of schedule to the nines, and then you can
- 28:43Speaker 2: give it some specific prompts around, like from that persona,
- 28:46Speaker 2: which of these headlines might stand out in their inbox?
- 28:50Speaker 2: Which of these are the kind of more likely potentially
- 28:52Speaker 2: to click on? What might it be competing with around
- 28:55Speaker 2: that time of day? And then one of the things
- 28:58Speaker 2: I always like to do is ask the LM to
- 29:01Speaker 2: cite why it's giving me this information. So point me
- 29:05Speaker 2: to the blog posts that you're kind of referencing here,
- 29:08Speaker 2: point me to the study that you're using, or kind
- 29:11Speaker 2: of the ux principle, So trying to get as much
- 29:14Speaker 2: separate what it's hallucinating versus what I can go actually
- 29:18Speaker 2: kind of validate. So that's kind of the lightest layer
- 29:21Speaker 2: that you could do it. You can get deeper by
- 29:24Speaker 2: having it create full audiences of people like that. You
- 29:29Speaker 2: can say, imagine a representative pool of parents of type
- 29:34Speaker 2: one diabetics in this country, make up demographic psychographic behavioral information,
- 29:41Speaker 2: come up with a bunch of different personas, come up
- 29:44Speaker 2: with a distribution of those people, and now run this
- 29:48Speaker 2: subject line against that imagined population of people.
- 29:51Speaker 3: How accurate have you found it to be?
- 29:54Speaker 1: Like, have you ever done a split test where you've
- 29:57Speaker 1: recruited real humans and compared that to simulated audience.
- 30:02Speaker 2: I think it's all in the quality of the setup, right,
- 30:06Speaker 2: So the most important thing is if your assumptions about
- 30:10Speaker 2: the online audience. So, for example, if you're going to
- 30:13Speaker 2: test a x post and you're like, oh, I'm going
- 30:15Speaker 2: to build a synthetic audience with an LLM, if it
- 30:19Speaker 2: doesn't really map to your actual audience, then your results
- 30:22Speaker 2: are going to be, you know, just noise. But I
- 30:25Speaker 2: think there's some pretty interesting, you know, ways to kind
- 30:28Speaker 2: of like get that signal and get it pretty close.
- 30:31Speaker 2: So I would say I've had some where it's like
- 30:33Speaker 2: fully wrong and just doesn't resonate at all, and then
- 30:36Speaker 2: there are some where you're like, oh, that's actually that's
- 30:39Speaker 2: pretty good. So it's it's still more art than science.
- 30:42Speaker 2: I wouldn't put my job up against it quite yet,
- 30:45Speaker 2: but I do think it points to like an interesting
- 30:47Speaker 2: way to kind of accelerate ideation and kind of like
- 30:51Speaker 2: honing your thinking a bit further than kind of, you know,
- 30:54Speaker 2: just kind of scribbling, you know, down a couple of
- 30:57Speaker 2: ideas on a whiteboard.
- 30:58Speaker 1: So yeah, with that in mind, and when in an
- 31:01Speaker 1: experimentation or idea testing process, would you bring in real humans.
- 31:07Speaker 2: As soon as possible? Starting with hey, does the team
- 31:10Speaker 2: think this is a good idea? With dropbox and Dash?
- 31:13Speaker 2: We have some of our customer advisory boards and design
- 31:15Speaker 2: partners who we give them early access to our products
- 31:19Speaker 2: and ask for their feedback, or we show them mockups
- 31:22Speaker 2: and prototypes and ask them is this clear? How can
- 31:24Speaker 2: we make it better? And then from there you can
- 31:27Speaker 2: move up into bigger, scaled tests with larger groups of people,
- 31:32Speaker 2: quantitative surveys all the way up to ab tests and
- 31:36Speaker 2: beyond with personalization and so on. And so I don't
- 31:39Speaker 2: think it's lllms instead of humans. It's what is the
- 31:43Speaker 2: pre work that you can do to make the very
- 31:45Speaker 2: limited time and the very cost the expensive time with people,
- 31:50Speaker 2: you know, make it the most impactful as possible.
- 31:52Speaker 1: What else has changed when it comes to experimentation and
- 31:55Speaker 1: some of the concepts you were writing about years ago.
- 31:58Speaker 1: When it comes to test ideas in the age of AI.
- 32:02Speaker 2: The obviously answers, Hey, AI has changed a bunch of
- 32:04Speaker 2: the tooling and how we do it. But I actually
- 32:06Speaker 2: think maybe goes back to what you and I were
- 32:08Speaker 2: talking about earlier with the how to write like a
- 32:11Speaker 2: good prompt, which I think can be generalized into how
- 32:14Speaker 2: do I think about what's the right question to ask,
- 32:18Speaker 2: what's the right problem to solve? How good is our
- 32:22Speaker 2: thinking upfront in terms of the question we're trying to answer,
- 32:27Speaker 2: the problem we're trying to solve, and then the context
- 32:30Speaker 2: around that to do that really effectively, because the output
- 32:33Speaker 2: now is limitless, and so I think if I was
- 32:35Speaker 2: going to write packing growth again today, I would have
- 32:37Speaker 2: spent more time in how you think about framing up
- 32:41Speaker 2: and create hypothesis. Without that, the output is maybe just
- 32:45Speaker 2: more noise.
- 32:46Speaker 1: I'd love to finish on a piece of advice for listeners,
- 32:50Speaker 1: and you know, I feel like how I work listeners
- 32:53Speaker 1: fairly AI savvy, Like if you were to set them
- 32:56Speaker 1: one piece of homework something to do that would really
- 33:01Speaker 1: improve how they're working with AI right now, seeming they're
- 33:06Speaker 1: kind of you know, they're more than dabbling. What would
- 33:10Speaker 1: be one of the most powerful things that they could
- 33:12Speaker 1: just do this week?
- 33:14Speaker 2: Take thirty minutes and think about what's something that you
- 33:17Speaker 2: do all the time that you do kind of constantly
- 33:20Speaker 2: without even thinking about it. It could be checking email,
- 33:23Speaker 2: it could be reading documents, it could be approving expense reports,
- 33:29Speaker 2: it could be you know, there's a numerous things that
- 33:30Speaker 2: we all do constantly, and you know, kind of the
- 33:34Speaker 2: idea with carb skin is I didn't even realize I
- 33:36Speaker 2: was doing that many Google searches around dinner until I
- 33:39Speaker 2: sat down and kind of just thought about it, like
- 33:41Speaker 2: oh wow, just kind of like hit me. We see
- 33:43Speaker 2: that in work all the time that you know, when
- 33:45Speaker 2: we're working on DASH, we'll talk to someone like walk
- 33:48Speaker 2: us through your day or walk us through your week,
- 33:50Speaker 2: and someone will say, oh, every Wednesday, I get seven
- 33:55Speaker 2: emails from all of these different field agents about the
- 33:58Speaker 2: status of projects that are happening around the country. And
- 34:01Speaker 2: then I take all those I read all of them,
- 34:03Speaker 2: and then I have to put together a report to
- 34:05Speaker 2: send to our management team about what's on track, what's behind,
- 34:08Speaker 2: what's happening out in the field. I was like, great,
- 34:11Speaker 2: how do you do them? They're like, well, I get
- 34:13Speaker 2: the emails ever way for them to come in. I
- 34:14Speaker 2: read on my copy and Piete stuff into a new
- 34:16Speaker 2: document and then I write them. And I was like, Okay,
- 34:18Speaker 2: this is a great use case for like summarization and
- 34:22Speaker 2: that type of thing. And it just kind of the
- 34:24Speaker 2: light bulb goes off when you can get down at
- 34:26Speaker 2: that very specific thing. So yeah, start with I do
- 34:29Speaker 2: this a lot, It's important part of my job. And
- 34:32Speaker 2: then step back and say, how can I even if
- 34:35Speaker 2: you can't go end to end with the workflow, like
- 34:38Speaker 2: you might not be able to you know, connect to
- 34:40Speaker 2: your email inbox to read it all, but you know
- 34:43Speaker 2: what is a piece in there that you can let
- 34:45Speaker 2: AI kind of take a little bit off your plate
- 34:47Speaker 2: for you and then kind of use that as a
- 34:49Speaker 2: springboard to kind of work through the rest of your week.
- 34:53Speaker 1: Basically, Wulgan, it is vain such a joy chatting to
- 34:56Speaker 1: you and getting to pick your brain.
- 34:58Speaker 3: Thank you so much for spending some time with me.
- 35:02Speaker 4: Yeah, I Meanta, thank you so much. I really enjoyed
- 35:04Speaker 4: the conversation. Thanks for having me.
- 35:07Speaker 3: Here's what's stuck with me from Morgan.
- 35:10Speaker 1: The real leverage of AI isn't necessarily the answers, it's
- 35:15Speaker 1: better questions.
- 35:16Speaker 3: So this week, try some homework.
- 35:19Speaker 1: Take thirty minutes to notice something you do constantly, almost
- 35:22Speaker 1: without thinking, and then ask what part of this could
- 35:27Speaker 1: AI take off my plate?
- 35:29Speaker 3: You might be surprised at how.
- 35:31Speaker 1: Small experiments can lead to very big shifts. And if
- 35:35Speaker 1: you enjoyed this episode, you might want to go back
- 35:38Speaker 1: to my episode with Bobby Ohanson, the amazing Silicon Valley futurist.
- 35:42Speaker 1: We talked about reframing AI as augmented intelligence.
- 35:47Speaker 3: And it's a beautiful pairing.
- 35:50Speaker 1: To this interview with Morgan. Thank you so much for
- 35:55Speaker 1: listening to this episode. Hit follow on How I Work
- 35:59Speaker 1: so that you don't.
- 36:00Speaker 3: Miss what's next.
- 36:01Speaker 1: If you like today's show, make sure you get follow
- 36:04Speaker 1: on your podcast app to be alerted when new episodes drop.
- 36:08Speaker 1: How I Work was recorded on the traditional land of
- 36:11Speaker 1: the Warrangery People, part of the Cooler Nation.