Latest / How I Work / The prompting mistake almost everyone makes
Transcript
- 0:00Speaker 1: You ask AI to write something, it gives you back
- 0:04Speaker 1: a first draft that looks fine, but the moment you
- 0:07Speaker 1: read it properly, there's just nothing in there. So you
- 0:11Speaker 1: go back and forth with the AI, telling it what's wrong,
- 0:14Speaker 1: and half an hour later you think I should have
- 0:17Speaker 1: just written this myself. There's a research suggesting that we
- 0:21Speaker 1: lose close to one day a week doing exactly that.
- 0:26Speaker 1: Neo and I call this slop fixing, and in this
- 0:30Speaker 1: episode we get into why it happens and the two
- 0:33Speaker 1: ways of working with AI that decide whether it does.
- 0:38Speaker 1: You'll hear the difference between gun slinging and architecting, what
- 0:42Speaker 1: a proper brief actually sounds like, and how to have
- 0:45Speaker 1: a conversation with AI before you ask it to build anything.
- 0:50Speaker 1: By the end of this episode, you'll be getting version
- 0:52Speaker 1: one at ninety percent and not twenty percent. Welcome to
- 1:03Speaker 1: how IAI with me Doctor Amantha Imba and Neo Applin,
- 1:07Speaker 1: head of Inventium AI. Each episode we share one practical
- 1:11Speaker 1: way to use AI better at work and in life.
- 1:15Speaker 1: No fluff, no deech jargon, just things you can use
- 1:18Speaker 1: straight away. I think it's quite well known at this
- 1:22Speaker 1: point that AI has promised a lot of productivity savings,
- 1:27Speaker 1: but it is simply not delivering, and Neo I want
- 1:31Speaker 1: to talk about why and what we can do about it.
- 1:34Speaker 1: So we think about this problem and we've got a
- 1:38Speaker 1: name for it, and it is slop fixing, because if
- 1:43Speaker 1: you are not working well with AI, you are producing
- 1:45Speaker 1: a lot of AI slop. First, let's start by defining
- 1:49Speaker 1: AI slop for those that haven't come across it. What
- 1:52Speaker 1: is AI slop?
- 1:55Speaker 2: Different definitions for different people, but effectively, it's when you
- 1:58Speaker 2: get something out of AI and it doesn't feel real.
- 2:02Speaker 2: It doesn't feel it doesn't hit the mark, It doesn't
- 2:05Speaker 2: actually drive the conversation forward, the work forward. It's just
- 2:08Speaker 2: a average piece of work which has lots of generalities
- 2:12Speaker 2: in it that people read through and just go, yeah,
- 2:15Speaker 2: it's not quite there.
- 2:16Speaker 1: It's funny because it kind of looks good on the surface,
- 2:18Speaker 1: but when you get into it, there is absolutely no substance.
- 2:22Speaker 1: And what certainly we observe a lot of people doing
- 2:27Speaker 1: is they will prompt AI poorly. It will produce slop,
- 2:33Speaker 1: and if the person still has their critical thinking brain on,
- 2:36Speaker 1: they will recognize that and then they will just be
- 2:38Speaker 1: going back and forth and back and forth with the AI,
- 2:42Speaker 1: arguing with it, and we call this slop fixing. And
- 2:48Speaker 1: if you don't know how to prompt properly or are
- 2:51Speaker 1: not giving the AI enough context, you can spend hours
- 2:56Speaker 1: and hours of your week slop fixing, Which makes sense
- 3:00Speaker 1: then that we're not seeing these huge productivity gains. So
- 3:04Speaker 1: what's your experience of slop fixing and what you observe
- 3:08Speaker 1: in the many, many workshops you're running an inventive AI.
- 3:12Speaker 2: There's an argument with AI. People are working with it
- 3:15Speaker 2: and they go, you don't quite get me, and then
- 3:17Speaker 2: they argue with it, like no, no, you've got to do
- 3:19Speaker 2: more of this, You've got to do less of that.
- 3:21Speaker 2: It's almost like you're sculpting the work with AI, and
- 3:25Speaker 2: there's lots of backwards and forwards and a lot of
- 3:26Speaker 2: time to do this. Or people just give up and
- 3:29Speaker 2: they go, eh, it's fifty percent of the way there.
- 3:32Speaker 2: I've got a structure. That's fine. Now you just got
- 3:35Speaker 2: to go and clean that thing up myself. And so
- 3:36Speaker 2: that's slop fixing on my time on my computer. But
- 3:40Speaker 2: I don't know whether people are actually equating the total
- 3:43Speaker 2: amount of time has taken to work with the AI
- 3:45Speaker 2: and then do the cleanup versus just doing it themselves.
- 3:49Speaker 1: One hundred percent. And what research would suggest, because I've
- 3:52Speaker 1: heard slop fixing referred to as several different terms, is
- 3:56Speaker 1: that we spend nearly one day a week slop fixing.
- 4:00Speaker 1: And so what we want to talk about today is
- 4:02Speaker 1: how can we reduce the time that we are spending
- 4:05Speaker 1: fixing the slop. So let's start with one term. So
- 4:10Speaker 1: we're going to talk about a couple of terms that
- 4:12Speaker 1: we use at Inventium AI, and they are gun slinging
- 4:16Speaker 1: and architecting. So noil can you explain what gun slinging is.
- 4:21Speaker 2: It's what it's shooting from the hip and then not
- 4:24Speaker 2: quite hitting the bullseye at all times. So a lot
- 4:27Speaker 2: of people when they're working with AI, they'll just go,
- 4:29Speaker 2: I need AI to do some research on this, or
- 4:33Speaker 2: to analyze this thing, you know, or to write this
- 4:36Speaker 2: particular piece or whatever. But they don't give a lot
- 4:38Speaker 2: of background, they don't give a lot of context, they
- 4:40Speaker 2: don't give a lot of goals. There's not a lot
- 4:41Speaker 2: of wist. So you're not spending a lot of time
- 4:44Speaker 2: to work with that brief that you're giving AI to
- 4:47Speaker 2: get it to do the thing, and because you're not
- 4:49Speaker 2: giving it a great brief, just like you would with
- 4:51Speaker 2: any staff member. If you're not giving it a great brief,
- 4:53Speaker 2: they're not going to give you a great piece of
- 4:54Speaker 2: work out. So it's kind of shooting from the hip quick.
- 4:57Speaker 2: You can kind of quick shot from the hip with
- 4:58Speaker 2: your prompt and and then of course you don't hit
- 5:01Speaker 2: the bulls eye many times, and then you get a
- 5:03Speaker 2: fair bit of AI slot that you then got to
- 5:05Speaker 2: work and refine with. So that's gunslinging.
- 5:08Speaker 1: Okay, So we give a solution and we call it architecting.
- 5:14Speaker 1: So this is a really different way to work with
- 5:16Speaker 1: AI compared to what we observe a lot of people doing.
- 5:19Speaker 1: So can you explain what is architecting.
- 5:23Speaker 2: I'll give you an analogy of gun slinging and architect
- 5:25Speaker 2: just so you can get around you your head. You
- 5:27Speaker 2: wouldn't go to an architect or to someone who's going
- 5:30Speaker 2: to design a house and say build me a house,
- 5:33Speaker 2: because they'll build you a wonderful house and that's great,
- 5:35Speaker 2: but you go, no, no, no, no no. I wanted to
- 5:37Speaker 2: be a three bedroom house, not a four bedroom house.
- 5:40Speaker 2: I needed two car garage and I want to this,
- 5:44Speaker 2: and I want a larger kitchen, and I want a
- 5:45Speaker 2: Buttler's page. You haven't told the architect what you want
- 5:48Speaker 2: for your house, so why are you getting upset with
- 5:51Speaker 2: this house that you've now magically got. So gunslingers are
- 5:53Speaker 2: more like, build me a house, and then it's oh,
- 5:55Speaker 2: this is a crap house. Let's work on it. Ah,
- 5:57Speaker 2: you've got to do more with it. Whereas with architecture,
- 5:59Speaker 2: you'll go to an architect and you'll say, Okay, i've
- 6:02Speaker 2: got a family, and there's a family of four, and
- 6:05Speaker 2: I've got a dog, so we need a big enough yard.
- 6:08Speaker 2: And my husband he loves tinkering in the garage, so
- 6:12Speaker 2: we need to have a little workhopping area in the garage.
- 6:14Speaker 2: And so you're having a discussion about what you need,
- 6:17Speaker 2: and so you're more likely to get that first house
- 6:19Speaker 2: design really well and the other really good. The other
- 6:22Speaker 2: thing is when you're working with an architect, you'll ask questions.
- 6:24Speaker 2: You'll have a discussion going on, and you say, hey,
- 6:27Speaker 2: would it be better for me to do this? Or
- 6:29Speaker 2: is a light shining from the wrong direction for me
- 6:31Speaker 2: to have this window in this area, and things like that.
- 6:34Speaker 2: So it's it's a discussion with your architect as opposed
- 6:37Speaker 2: to just saying build me in house and then getting
- 6:39Speaker 2: grumpy that you've got a very generic, very bad house.
- 6:43Speaker 1: So can you perhaps talk me through an example of
- 6:48Speaker 1: using architecting, Like, what's the brief? How are you having
- 6:52Speaker 1: that discussion with the AI? What does that look like?
- 6:56Speaker 2: All right, let's look at a scenario something like I
- 7:00Speaker 2: don't know, I'm doing an onboarding plan for a new
- 7:03Speaker 2: staff member. Why not? Okay, if you just said, write
- 7:06Speaker 2: me an onboarding plan for a new what would you
- 7:10Speaker 2: like for a staff member? Like a marketing lead? Great,
- 7:14Speaker 2: a marketing lead, yep, okay, So give you a non
- 7:17Speaker 2: boarding plan for a marketing lead. You'll get a pretty
- 7:20Speaker 2: average one. But the first thing is is this a
- 7:23Speaker 2: digital marketing lead or are they dealing with posters and
- 7:26Speaker 2: those kind of marketing things. It's a really different marketing
- 7:30Speaker 2: role if you're doing digital versus non digital. You might
- 7:35Speaker 2: have one workforce one need for your business, which might be, Hey,
- 7:39Speaker 2: what we're trying to do is grow into international areas
- 7:42Speaker 2: and so therefore that's a very different kind of marketing
- 7:44Speaker 2: thing than grassroots marketing. So what you probably do is say,
- 7:48Speaker 2: look what I need is an onboarding to help me
- 7:51Speaker 2: get this marketing lead up to speed. Here's the things
- 7:53Speaker 2: that they're going to do. Here's what I'm trying to
- 7:55Speaker 2: achieve with this. So you're setting the goal, you're having
- 7:58Speaker 2: a conversation with AI about what you really want out
- 8:01Speaker 2: of this, and then what you might do is say, hey,
- 8:04Speaker 2: now this is kind of my goals, what I'm kind
- 8:06Speaker 2: of going for. What's best practice on this, what's the
- 8:09Speaker 2: best way to achieve this? So they are more innovative
- 8:11Speaker 2: ways that I could potentially get them across all their
- 8:14Speaker 2: needs and also to engage with the team and things
- 8:16Speaker 2: like that. And then AI might give you some suggestions.
- 8:19Speaker 2: You might agree with some, you might disagree with others,
- 8:21Speaker 2: but you then got a discussion going on, plus the
- 8:24Speaker 2: ability to be able to change things that it might
- 8:27Speaker 2: suggest that otherwise may not fit for you. So he
- 8:30Speaker 2: I might say, oh, you can do this, this and this,
- 8:31Speaker 2: and the second of those. You might say that doesn't
- 8:33Speaker 2: apply for us. We don't need that because we're doing
- 8:37Speaker 2: international digital marketing. And so you've suggested something that's got
- 8:40Speaker 2: like with posters and bricks and mortar, that's not us.
- 8:43Speaker 2: So you've got an ability here to refine your plan
- 8:47Speaker 2: with AI so it knows your goals what you're trying
- 8:49Speaker 2: to achieve, and then align what you want. And then
- 8:52Speaker 2: when you're happy that you've got this architect this blueprint
- 8:56Speaker 2: that you want if you like, then you'll get it
- 8:58Speaker 2: to build whatever you wanted to build.
- 9:00Speaker 1: And I think that's the key difference. Like when I've
- 9:02Speaker 1: seen you do architecting, and I know when I do
- 9:06Speaker 1: it myself, it's kind of the opposite to what I
- 9:09Speaker 1: see a lot of people doing, where they'll just go,
- 9:12Speaker 1: here's here's the stuff you need to know. AI, Now
- 9:15Speaker 1: produce the output. And then once you've got the output,
- 9:18Speaker 1: that's where you spend a lot of time arguing and
- 9:21Speaker 1: where a lot of the slot fixing comes in. But
- 9:24Speaker 1: you're saying, have a really long conversation or at least
- 9:29Speaker 1: a moderate length conversation with the AI before you even say, hey,
- 9:33Speaker 1: produce this output. Am I phrasing that correctly?
- 9:37Speaker 2: Yeah, It's not necessarily about the length. It's more about
- 9:42Speaker 2: the effort bias. So with Gunsley, you've got very amount
- 9:47Speaker 2: of a small amount of effort at the front, but
- 9:49Speaker 2: then you've got a lot of effort down the track
- 9:51Speaker 2: because it's then fixing, and often when you fix one things,
- 9:54Speaker 2: it breaks to something else and then it becomes quite
- 9:56Speaker 2: problematic and then you go, ah, bugg, I'll just go
- 9:57Speaker 2: and do it myself. Whereas with AR it's just a
- 10:00Speaker 2: little bit more time upfront making sure that it knows
- 10:03Speaker 2: what you're really trying to achieve. That goal is really important,
- 10:06Speaker 2: and then what matters to you whether it's like best practice,
- 10:09Speaker 2: or he's engaging with my team, or here's my different
- 10:11Speaker 2: client needs those kind of things, and then you're working
- 10:14Speaker 2: with that way. Then when you are sure that it
- 10:17Speaker 2: knows what you want, then go go and produce version
- 10:20Speaker 2: one of this thing. Version one is more likely to
- 10:23Speaker 2: nail what you need to be specific for what you need,
- 10:26Speaker 2: not there's kind of one on one general kind of stuff,
- 10:28Speaker 2: and it's going to be closer to rather than fifty
- 10:30Speaker 2: percent of the way there, it's probably going to be
- 10:31Speaker 2: eighty or ninety percent of the way there. So let's
- 10:33Speaker 2: clean up at the other end. So it's more about
- 10:35Speaker 2: spending a little bit more time upfront and then you've
- 10:39Speaker 2: got a much better result at the end.
- 10:41Speaker 1: Okay, are there any other final tips on how we
- 10:44Speaker 1: can become better architects with AI?
- 10:48Speaker 2: One of the ways to approach this is to work
- 10:50Speaker 2: with AI like you would a consultant, a staff member,
- 10:54Speaker 2: those kind of things. So we wouldn't just go, hey,
- 10:56Speaker 2: you do this. Instead, what you'd say is, here's what
- 10:59Speaker 2: I'm trying to check. Here's some background. What do you
- 11:02Speaker 2: think are there any better ways that you think that
- 11:05Speaker 2: we might be able to do this? Oh yeah, but
- 11:07Speaker 2: that doesn't apply for us. But that does. It's a
- 11:09Speaker 2: really good idea. Let's include some of that. And then
- 11:12Speaker 2: it's almost like you're planning things out together, you and
- 11:17Speaker 2: AI as a collaborative team here, and then when you're
- 11:20Speaker 2: comfortable that you know what it knows, that it knows
- 11:22Speaker 2: what you want, then you go, let's produce this thing together.
- 11:27Speaker 1: Love that. Thank you for giving us at a solution
- 11:31Speaker 1: to all the slot fixing that is consuming so many
- 11:35Speaker 1: people's days. How AI was hosted by me Amantha Imber
- 11:40Speaker 1: and Neo Applan. A big thank you to Martin Imba
- 11:42Speaker 1: who does our sound editing, and Jim Rubio for production support,
- 11:47Speaker 1: and thank you to John Kilby who composed the theme music.