Latest / Humanity Working / Is Generative AI Actually Good at Anything Meaningful?

Transcript

Is Generative AI Actually Good at Anything Meaningful? · Humanity Working

• 0:15
  SPEAKER_02: All right, so Paul, we're back with another uh review of the seven big questions articles, and uh this one I I I quite enjoyed a lot.
• 0:34
  SPEAKER_02: Is generative AI good at anything meaningful?
• 0:37
  SPEAKER_02: And while it's a provocative question, uh I think there's some deep meaning in the question itself that we should be asking about all these tools as they uh exponentially invade our our existence.
• 0:52
  SPEAKER_02: So tell me about the first point you you go about defining good, meaningful, is it good at things?
• 1:02
  SPEAKER_03: What is that eating?
• 1:03
  SPEAKER_03: Yeah, so what do we mean?
• 1:05
  SPEAKER_03: What do we mean by good?
• 1:07
  SPEAKER_02: Um Yeah, what do you mean by good?
• 1:09
  SPEAKER_03: Yeah.
• 1:09
  SPEAKER_03: Well, it's funny.
• 1:10
  SPEAKER_03: When I started getting asked this question, as a reminder, these these questions have come about from folks that are read the AI Ready Human, and it's sort of like they're they're follow-on questions that I've got at book events or other events that I've done from it.
• 1:29
  SPEAKER_03: And when I started getting this one, I I struggled initially with an answer because I I couldn't exactly figure out immediately what even the question what even the question meant, because I couldn't exactly figure out what we even mean by good.
• 1:59
  SPEAKER_03: Um you can say you can say AI is good at at predicting text, it's clearly good at that.
• 2:10
  SPEAKER_03: That's what it does, that's what it's built to do, and it's surprisingly good at it, and that let has led to this whole AI resolu uh revolution.
• 2:21
  SPEAKER_03: But but when people ask this question, what they really mean is it's is good as an indication of quality, whereby people that that know a field would judge it to be good in the context of their field.
• 2:48
  SPEAKER_03: So think of this as I talk about this in the article.
• 2:51
  SPEAKER_03: It could be a a wine that is uh it is 97 points or something.
• 2:59
  SPEAKER_03: It could be a movie that has a high critic score, right?
• 3:06
  SPEAKER_03: There are movies out there right now that that have extremely high audience scores and very low critic scores, right?
• 3:15
  SPEAKER_03: I'm what people meant when they were asking this question, they were they were talking about quality as judged by people that understand what quality is in the context of their field.
• 3:31
  SPEAKER_03: And I think that's a really interesting question to answer because it's entirely possible that something can be highly effective and highly useful and have strong business value without having that element of quality.
• 3:56
  SPEAKER_03: I think this is so important because AI is drifting into fields of creativity more and more.
• 4:06
  SPEAKER_03: And so the heart of creativity, or at least a strong piece of creativity, is quality, right?
• 4:14
  SPEAKER_03: Is a b there is in a world where there is no quality, is a world of diminished beauty, it's a world of diminished enjoyment, and all of those, uh, and all of those things.
• 4:27
  SPEAKER_03: So that's really what I wanted to explore.
• 4:28
  SPEAKER_03: And I wanted to explore whether AI was actually good at these things on its own.
• 4:38
  SPEAKER_03: So I'm not talking about the AI human partnership, I'm talking about AI on its own.
• 4:45
  SPEAKER_03: And maybe we'll get into reasons why how good AI is at things on its own is in uh is increasingly important.
• 4:55
  SPEAKER_02: You know what it reminded me of as you were talking about that is um this idea of like uh there's been a movement recent years of architectural design in uh uh uh restaurant and strip mall design where they're like very simple, muted colors, even places like McDonald's, you know, are whatever color the shopping center is, right?
• 5:24
  SPEAKER_02: And you you know barely differentiate them.
• 5:27
  SPEAKER_02: You know, obviously this means when businesses move in or out, it's still a useful building, there's a lot of reasons to do it, etc.
• 5:33
  SPEAKER_02: But is it beautiful?
• 5:34
  SPEAKER_02: Is it great?
• 5:35
  SPEAKER_02: Or was it way cooler when we had a pizza hut that looked like a hut?
• 5:40
  SPEAKER_02: You know, with a red roof and distinctive building.
• 5:44
  SPEAKER_02: We don't have to talk about the quality of the pizza, but the design of the building itself is now muted to this like simplified, very useful, utilitarian, good enough standard for a building to put a restaurant in.
• 5:59
  SPEAKER_02: And and it sometimes feels like if we didn't define good as being interesting, but rather defined good as being uh easy to replicate, yeah.
• 6:13
  SPEAKER_02: For example, well then your answer to the question is very different.
• 6:17
  SPEAKER_02: Is it good at something right?
• 6:19
  SPEAKER_02: So so that quality measure that you turn up or turn down in your expectation drives your answer quite a bit as well.
• 6:29
  SPEAKER_02: It does.
• 6:29
  SPEAKER_03: And I think that the I I think the reason that why I wanted to look at it in this way specifically is that um generative AI, the type of AI that we're interacting with on a day-to-day basis, is involved in the act of creation.
• 6:47
  SPEAKER_03: And whether it's creating an email or whether it's you know helping you create a long-form essay or whether it's creating an image, it's it's creating it's it's creating things.
• 7:02
  SPEAKER_03: And it's sometimes, many times in partners in partnership with a human being, but it is creating something.
• 7:07
  SPEAKER_03: And so it's a viable question as to whether that something it's creating is of quality.
• 7:15
  SPEAKER_03: And in many cases, the act of quality is not just replicating something, replicating something well, um, but it's it's also has an originality element to it, um, an element to surprise.
• 7:35
  SPEAKER_03: For example, you know, you do stand-up comedy.
• 7:39
  SPEAKER_03: If if you rocked up and recited a bunch of jokes from other people, aside from being ostracized from the stand-up community, you're you're not you're not doing something that is in any way original, and therefore you're not doing really doing something that is in any way interesting.
• 8:00
  SPEAKER_03: So there is this there's this sort of a connection between originality and quality and the creative process that generative AI is is is starting to enter that realm, so we have to we have to start asking ourselves questions about whether it whether it is producing stuff of quality, whether it's exp whether it's rational to even expect it to, um, and if it's producing 80-90% of the stuff that we consume, what the impact of that is if there is diminished quality.
• 8:40
  SPEAKER_02: Really interesting parallel on comedy is even if I delivered those that act extremely well, even if I could imitate the originator exactly and deliver, and the audience had a good time, they wouldn't realize necessarily that they were seeing something reproduced.
• 8:59
  SPEAKER_02: Or not that wasn't mine, right?
• 9:01
  SPEAKER_02: I mean, this is the the the joke-stealing thing is every audience is a new audience and they've never seen you before, they've never seen the person you ripped off before, so who cares?
• 9:10
  SPEAKER_03: Yeah, it's intriguing, isn't it?
• 9:11
  SPEAKER_03: I mean, the most rational if if you were not caught, the most rational thing for you to do would be to and all you wanted was the audience applause, and there wasn't any you know excitement in the creative process, the most rational thing for you to do would be to find a bunch of jokes that are killed and and and recycle them.
• 9:29
  SPEAKER_02: Yeah.
• 9:30
  SPEAKER_02: Interesting.
• 9:31
  SPEAKER_02: But that yeah, okay.
• 9:33
  SPEAKER_02: Interesting.
• 9:34
  SPEAKER_02: So so let let's uh let's focus on this human in the loop piece for a second, because um the key right now to ensuring that things are good, especially in the terms of generative AI, is still the critical thinking and the taste and the selection and the editing of things, right?
• 9:55
  SPEAKER_02: Um you know, for that to be the key, what is it that at some point we would make the distinction, or do you think that at some point we'll make the distinction and say, you know what, it is actually hitting the mark now?
• 10:15
  SPEAKER_02: Like is that even possible with generative AI given that it has to draw on what it's been fed?
• 10:24
  SPEAKER_03: Well, there's a few questions in there.
• 10:26
  SPEAKER_03: So let's start with where you ended.
• 10:29
  SPEAKER_03: Um is it is it possible for AI to hit a mark of uh a certain mark of quality if we think of quality in the way that that I was describing?
• 10:44
  SPEAKER_03: Um I don't think we definitively know the answer to that, to be honest.
• 10:50
  SPEAKER_03: Um and the reason that I that that I don't is that I you know I would never bet against um the innovations that are happening in uh in technology and the specifically the innovations that are happening in models and and uh the ability of models to improve over time and become ever more in quotes human-like.
• 11:16
  SPEAKER_03: Um and I've uh you know heard interviews with uh with people like uh Chris Columbus, for example, who's spoken about at some length about the concern that AI will be able to write uh movie scripts that are every bit as compelling as as the movie scripts that have written been written by the great screenwriters.
• 11:44
  SPEAKER_03: I think that's possi I think that's at least theoretically possible because um there is a language of movies, and if you master that language of movies, there's a language of sort of traditional storytelling, and if you master that language of traditional storytelling, that potentially you could you you could produce something that is uh uh that hits a certain quality mark.
• 12:11
  SPEAKER_03: But however, there's a real question as to whether you can produce something that has that uh genuinely innovative uh feel.
• 12:23
  SPEAKER_03: And that is a mark, that's a mark of quality.
• 12:25
  SPEAKER_03: The thing that's the thing that surprises you takes you in a different direction than you expected.
• 12:31
  SPEAKER_03: That could be it could be built in that it's that it's not really able to do that.
• 12:36
  SPEAKER_03: I mean, let's if we take a step back and we think about how AI is engineered, right?
• 12:45
  SPEAKER_03: The fundamental engine behind AI is about predicting the next word.
• 12:53
  SPEAKER_03: As simple as that.
• 12:54
  SPEAKER_03: I mean, when people say when people talk about AI as being glorified auto-complete, they are right.
• 12:59
  SPEAKER_03: I mean, it is basically I you ask it a series of questions, it continues the conversation in its answer, and it is asking itself, after, you know, through a whole bunch of different layers of a uh of a neural network and with a whole bunch of fancy tech behind it, it's saying, what's what's the best next choice word?
• 13:22
  SPEAKER_03: Well, best in that context is most likely.
• 13:26
  SPEAKER_03: So given that it's most it's looking for a most likely, the idea of it like surprising you and taking you in a different direction is is much lower than say a human artist who has mastered the, for want of a better term, has mastered the art of surprise.
• 13:48
  SPEAKER_03: So we'll be able to do um, you know, I've no I've no doubt that you could you could have it very soon say write the equivalent of an M-night Shalay M.
• 14:00
  SPEAKER_03: And how do you say that last word?
• 14:03
  SPEAKER_03: Shimalayan.
• 14:03
  SPEAKER_03: Write the equivalent of an M night shamelion movie.
• 14:07
  SPEAKER_03: But the real question is like, you know, after you've seen a thousand of those, is that are you now just replicating that and therefore it's not interesting anymore, right?
• 14:21
  SPEAKER_03: Is it gonna be is it gonna be Picasso in a world that didn't have Picasso?
• 14:28
  SPEAKER_03: I don't think so.
• 14:29
  SPEAKER_03: So it's that sort of uh it's those sorts of questions that have to be that have to be looked at.
• 14:37
  SPEAKER_02: It'll make Picasso for someone who's never seen Picasso.
• 14:41
  SPEAKER_02: Because it has.
• 14:44
  SPEAKER_02: Right.
• 14:44
  SPEAKER_02: So so that uh like as you're saying, I'm like, oh yeah, it's it's the same deal.
• 14:48
  SPEAKER_02: It's like it might write an incredible script because you've not read all the scripts.
• 14:54
  SPEAKER_02: And maybe you've not read Joseph Campbell and understand story structure, therefore it seems amazing.
• 14:59
  unknown: Yeah.
• 15:01
  SPEAKER_03: But is that truly quality?
• 15:03
  SPEAKER_03: I would argue I would argue not in the full sense of what we're discussing here.
• 15:09
  SPEAKER_03: Yeah, yeah, that's really interesting.
• 15:11
  SPEAKER_03: And then of course you asked about you asked about human in the loop as well.
• 15:14
  SPEAKER_03: Yeah.
• 15:15
  SPEAKER_03: Yeah.
• 15:15
  SPEAKER_03: Yeah.
• 15:16
  SPEAKER_03: What do you let's let's explore that.
• 15:18
  SPEAKER_03: What do you what do you want to think about regarding human and loop?
• 15:23
  SPEAKER_02: So right now, for the foreseeable future in the current paradigm that exists, obviously uh acknowledge that it's changing rapidly, but human in the loop is the key to quality.
• 15:34
  SPEAKER_02: So uh what is the are they then also the arbiter of of quality as a judgment for the AI?
• 15:49
  SPEAKER_02: And how long do you think it takes for that back and forth loop to wind itself down, if ever?
• 16:00
  SPEAKER_02: And I think I know the answer based on what we just talked about, but like you know, there's gonna be a point where we do trust it for things.
• 16:06
  SPEAKER_02: We're gonna continually train it, right?
• 16:08
  SPEAKER_02: So um barring any, you know, quantum computing limitations or something where the the processing power stops us, I I I see a point where almost the goal is to make ourselves obsolete if we want that to be better.
• 16:26
  SPEAKER_03: Yeah, right?
• 16:27
  SPEAKER_03: I mean, human in the loop is so interesting.
• 16:30
  SPEAKER_03: Um for folks that are um have not come across the concept, it's worth looking up.
• 16:37
  SPEAKER_03: It's used increasingly in the context of of agentic AI as well.
• 16:42
  SPEAKER_03: But human in the loop really is what it sounds like, right?
• 16:46
  SPEAKER_03: It is the idea that there's a whole bunch of AI happening and you need humans as part of the process.
• 16:54
  SPEAKER_03: Um and the main argument for human in the loop is really centered around, mostly centered around safety, right?
• 17:03
  SPEAKER_03: To make sure that it's not running amok and and that there are controls and all of this kind of stuff.
• 17:10
  SPEAKER_03: Um human in the loop has some challenges, by the way, in the sense that um the better the AI gets at it, uh, the less the human has to do, the less attention the human pays, and therefore the lower the the lower their ability to be able to police this whole thing.
• 17:30
  SPEAKER_03: Right?
• 17:30
  SPEAKER_03: I refer to it in the book, the AI Ready Human, as the sort of board security guard challenge.
• 17:36
  SPEAKER_03: Airlines have you know Airlines have had this with autopilot for a while, but but but if you're thinking about um it in the context of our everyday lives, self-driving cars are the obvious scenario, right?
• 17:49
  SPEAKER_03: The human in the loop, you're looking at the road, you've got your hands ideally on the steering wheel, it's it's driving, and then if it screws up, you jump in.
• 17:57
  SPEAKER_03: And you probably might have noticed if you got one of those cars that you you might drift off the more the more the car is doing.
• 18:04
  SPEAKER_03: So it's those, so there are some challenges associated with that, but you're dead right.
• 18:08
  SPEAKER_03: Human in the loop is an important element of quality because it's injecting, it's in this case, it's not um it's not the thing that is keeping AI on the rails in terms of safety or making sure that it's not doing anything dangerous.
• 18:24
  SPEAKER_03: It's the human that is injecting the human touch.
• 18:28
  SPEAKER_03: Now, what's really interesting in my work and and research for the book is that I found that um where the human is in the creative process makes a big difference in terms of that you know independent judgment of quality.
• 18:47
  SPEAKER_03: So, for example, if you're writing, let's let's say you wanted to write an interesting novel or essay or something along those lines, and you start by saying to AI, what do you think I should do?
• 19:03
  SPEAKER_03: And AI outlines, gives you your first draft, and now you start editing the first draft.
• 19:11
  SPEAKER_03: If you start with that, you end up with something often that's pretty good, even as you're going back and forth and back and forth with AI.
• 19:19
  SPEAKER_03: Um, and it can be pretty it can be pretty good.
• 19:21
  SPEAKER_03: But if you start from completely independently of AI determining what you want the finished product to look like, and then having AI refine or provide additional ideas or additional directions, the measure of quality is typically quite a bit higher.
• 19:43
  SPEAKER_03: So when it's creating something, I urge people to be highly engaged at the very start, ideally, even independently independent of AI at the very start.
• 19:56
  SPEAKER_03: And then that ensures a that the human creativity element is not lost and being bound and framed by AI, and then also to be highly involved and highly engaged at the end before you hit publish on anything, because that's where you're making sure that your voice is not lost.
• 20:20
  SPEAKER_03: So it's a it's not just it's not just in the loop, it's determining where you are and where in the creative process you're exerting control.
• 20:33
  SPEAKER_03: And it's really important to exert control at the start and at the end, not just because for the sake of it you want your voice, but in order to actually inject the quality that only a human can inject.
• 20:47
  SPEAKER_02: And I I think too that you capitalize then on the things that it actually is really good at, like formatting and standards and stuff like that, where you can get the benefits and and you could argue, oh, well, it is good at this thing.
• 21:02
  SPEAKER_02: That's not generative AI necessarily, but um I think that's a great point, is when you do things uh really matter.
• 21:08
  SPEAKER_03: Well, and ironically, you can you can develop a quality control loop with it.
• 21:13
  SPEAKER_03: So for example, um I use um it's actually with Claude, but when I'm writing stuff, I have a small uh Claude um project that I've created in Clock in Claude Co-Work, and its sole purpose, it's framed with everything that I've ever written, and its sole purpose is to flag when I'm I am drifting from my own voice.
• 21:39
  SPEAKER_03: So, in other words, it's like it's it's almost like a um an editor of a really high quality editor who will look at something and say, Okay, this does sound like you in paragraph one and two and four, but in paragraph three, it doesn't really sound like you.
• 21:57
  SPEAKER_03: What are you gonna do about it?
• 21:58
  SPEAKER_03: It's not fixing it, it's simply Diagnosing it.
• 22:01
  SPEAKER_03: And then I don't necessarily see that, but when it when it indicates it, I do see it.
• 22:06
  SPEAKER_03: So it's really interesting what the role of it is versus the role of what you are in this entire creative process.
• 22:15
  SPEAKER_02: Yeah, I I find it fascinating in that regard too, in terms of uh applying it very particularly into use cases like that.
• 22:28
  SPEAKER_02: And also one of the things that I found interesting with generative AI, particularly with uh with uh with Chat GPT, but somewhat with with Claude, is um I've had to be very clear when working on writing projects to make sure that it if it makes something up, it points that out very clearly.
• 22:51
  SPEAKER_02: Like not just not hallucinating, but literally like in summarizing an idea, it will tend to regurgitate a thought around what you feed it.
• 23:02
  SPEAKER_02: Like it sounds like you're trying to say XYZ, right?
• 23:05
  SPEAKER_02: Um and I I find that kind of interesting because in the course of writing you still can recognize your own voice somewhere and your own, like I I didn't write that sentence.
• 23:20
  SPEAKER_02: Yeah.
• 23:20
  SPEAKER_02: I'm sure I didn't write that sentence.
• 23:22
  SPEAKER_02: And it's a different I find it a different feeling than the hmm, I don't remember writing that, but I think I would have written that versus hey, did you make this line up?
• 23:31
  SPEAKER_02: And you check it, and it's like, yeah, I made yeah, I I invented that line based, you know, and it's just like it's really interesting interacting with the LLM in that way, I find.
• 23:42
  SPEAKER_02: Um but so so you called it uh I think a a master of mediocrity.
• 23:48
  SPEAKER_02: Is that is that accurate?
• 23:50
  SPEAKER_02: Like um Yeah.
• 23:52
  SPEAKER_02: What what do you think how do you think that plays into uh the general acceptance rate that we're dealing with right now, or maybe not acceptance rate, but the but the the uh having AI being put upon every single instance of everything and every company talking about it all at the same time.
• 24:14
  SPEAKER_02: Uh they probably wouldn't do that if there was some other tech that was just okay.
• 24:21
  SPEAKER_02: They wouldn't force feed it down everybody's throat.
• 24:24
  SPEAKER_02: You know what I mean?
• 24:24
  SPEAKER_02: Whereas this one feels like, yeah, it's just it's a master of mediocrity, but we're gonna put it everywhere anyway.
• 24:32
  SPEAKER_02: Like a sunk cost kind of thing, you know.
• 24:34
  SPEAKER_03: Well, this is this to me is actually one of the more the most interesting things as I'm I'm preparing for the next book where I'm looking at organizational factors associated associated with AI.
• 24:48
  SPEAKER_03: Which is that you can see why the appeal is there for enterprises.
• 24:56
  SPEAKER_03: And it extends, I think, beyond just simply, ooh, it means that I can, you know, potentially it means I don't have to hire as many people next year, or or indeed, you know, I can get rid of the I can get rid of these these people because AI is, in quotes, better than them.
• 25:13
  SPEAKER_03: I think that, you know, we've we've lived in a world, particularly in knowledge work, where um managers have been talking to employees for decades now about the concept of something we refer to as good enough, right?
• 25:32
  SPEAKER_03: And you underst it's completely understandable why that is, right?
• 25:39
  SPEAKER_03: There are many, many, many activities that happen inside a um inside an organization.
• 25:46
  SPEAKER_03: And not all of those activities have to be have to get an A plus, right?
• 25:53
  SPEAKER_03: Some of them a C is good enough, right?
• 25:57
  SPEAKER_03: If it's and if a C gets you to where you need to be, do you want to spend the extra, you know, ye six, three months, six months getting that getting the A for what purpose?
• 26:09
  SPEAKER_03: If it doesn't bring if it doesn't bring you any business value.
• 26:12
  SPEAKER_01: Yeah.
• 26:12
  SPEAKER_03: And so on looked at through that lens, the idea of like something being able to churn out stuff that isn't, quotes, good enough at speed and at pace and all of that kind of stuff feels very appealing.
• 26:32
  SPEAKER_03: But the question I think you have to ask yourself is um is good enough done efficiently going to be good enough when everybody can do that with the same level of efficiency?
• 26:54
  SPEAKER_03: It's like we're drowning in that online right now, right?
• 26:58
  SPEAKER_03: I mean, everybody can churn out AI-driven slop, but is your AI-driven slop going to be seen, right?
• 27:05
  SPEAKER_03: No, because it's just I I in the article I say it's you know we're drowning in a sea of C's, the letter C.
• 27:14
  SPEAKER_03: We're drowning in that, right?
• 27:16
  SPEAKER_03: And so the differentiator may become, in certain aspects of our business, may become higher quality again.
• 27:26
  SPEAKER_03: And how are you gonna get that?
• 27:27
  SPEAKER_03: You're gonna get that through the human involvement.
• 27:29
  SPEAKER_03: And so I think that this is, we don't know where this where the story ends, right?
• 27:35
  SPEAKER_03: We may be at a point where instead of us saying, let's just churn out even more of the mediocrity, we say, no, actually, there's gonna be a whole bunch of differentiation done on quality, and the secret to that differentiation is gonna be the human beings that are involved in the mix.
• 27:55
  SPEAKER_03: I think this might apply to it's gonna apply very much to our digital world, but I think it also might end up applying to a whole bunch of different areas where where people are command demanding higher quality, and they're gonna be judging quality based on something that doesn't seem exactly like everything else.
• 28:20
  SPEAKER_03: Um, you know, as you know, we've had Bill Kirsten on the podcast, and he talks about the beauty of human imperfection, right?
• 28:28
  SPEAKER_03: There's that sort of that the artisan touch is something that is I'm almost certain is going to be considered to be of high value.
• 28:40
  SPEAKER_03: But even throughout organizations, the sort of ability to be able to differentiate yourself based on marks of quality are going to be um going to become very important.
• 28:53
  SPEAKER_03: Like media, maybe you use it for customer service, right?
• 28:57
  SPEAKER_03: I I'm sure many people have noticed that it wonder that as we've increased the amount of AI involved in customer service, we've increased the amount of mediocrity associated with customer service.
• 29:08
  SPEAKER_02: Um and so Yeah, but it sounds like it's typing.
• 29:12
  SPEAKER_03: It makes me think it's working because it sound it has the type tells you something, and then and then it tells you something that's completely wrong.
• 29:21
  SPEAKER_03: And and it's like, oh, I'm sorry, let me correct that, you know, and all that kind of stuff.
• 29:25
  SPEAKER_03: So I get that it sounds like it's typing, and I get that it's gonna be better at it.
• 29:28
  SPEAKER_03: But and I get that I don't have to wait for, you know, 34 minutes to speak to a human being.
• 29:34
  SPEAKER_03: And I also get that much of what customer service was when done by humans was also terrible because people were paid really badly for it and you know, and trained poorly and all that kind of stuff.
• 29:44
  SPEAKER_03: But you could see it swinging all the all the way, the pendulum swinging all the way back to no, we're gonna pay our customer service reps well, we're gonna have human customer service with AI sitting behind them as a layer, and we're gonna do it because the human because the customer service layer is the human-to-human connection, and we're gonna invest heavily in the human-to-human connect connection because that is what our customers will determine as being high quality.
• 30:11
  SPEAKER_03: So you can see those sorts of things potentially starting to emerge as a reaction to where we've come to.
• 30:20
  SPEAKER_02: I can't even imagine if I've called a customer service line and an actual person answered first.
• 30:25
  SPEAKER_03: Yeah.
• 30:26
  SPEAKER_02: That would be crazy.
• 30:27
  SPEAKER_03: Wouldn't it be?
• 30:27
  SPEAKER_03: And how would that feel?
• 30:28
  SPEAKER_03: Especially if they picked up in like two rings and said, Hey, I'm here to help.
• 30:34
  SPEAKER_03: Yeah.
• 30:34
  SPEAKER_03: What can I do?
• 30:35
  SPEAKER_03: Wild.
• 30:36
  SPEAKER_02: Yeah, that'd be a that'd be a differentiator for sure.
• 30:39
  SPEAKER_02: Um so it strikes me that it, you know, if we had a time machine and we could go back to the launch of this really hitting the the public domain.
• 30:54
  SPEAKER_02: Uh I really wish that we would have started from the approach that your book takes to it instead.
• 31:07
  SPEAKER_02: Because we probably would have led the conversation much differently in the expectations and the rollout.
• 31:16
  SPEAKER_02: Yeah.
• 31:17
  SPEAKER_02: And the uh what happened in the first year or so there, where people were just laying off giant groups of people and saying, Oh, this is going to be great.
• 31:26
  SPEAKER_02: Uh, it's gonna be fine.
• 31:28
  SPEAKER_02: And it's still happening when it comes to code and you know, and regenerative code.
• 31:34
  SPEAKER_02: And and there was a great piece in the article there where I think somebody said they've never seen an increase in tech debt at this rate before.
• 31:42
  SPEAKER_02: Like it's it's an unbelievable pile of tech tech debt.
• 31:45
  SPEAKER_03: And by the way, this is research done by Google, so it's not just some, you know, it's it's it's it's it's real research into large amounts of of code that have been uh that's been created.
• 31:56
  SPEAKER_03: Yeah.
• 31:57
  SPEAKER_02: So like the code thing, the cognitive tax of reading a first draft that you didn't write, that we didn't calculate, all of those those things are contributing to um I hope uh a pretty heavy correction as this starts to level level out, maybe.
• 32:16
  SPEAKER_02: Um so I would I would want to know two things and then and then we can we can wrap it up.
• 32:21
  SPEAKER_02: First is um just because this has been on my mind lately uh and with the recent changes uh at at Apple, um I wonder what you think about if you think about the way that the iPad iPad iPhone before it were were rolled out under Steve Jobs as this absolutely game-changing, completely new thing.
• 32:47
  SPEAKER_02: What do you think would have been the difference in the way we perceived AI had it gone through that pathway as opposed to what we did?
• 32:58
  SPEAKER_02: Yeah.
• 32:59
  SPEAKER_02: Uh it has its benefits, obviously, the way we did it, with you know, which is uh open source idea and everybody gets it, you learn faster, etc.
• 33:06
  SPEAKER_02: But there's a there's a beauty to the way that jobs did it with Apple, too, which is they watched for a long time and then came out with perfection, kind of.
• 33:17
  SPEAKER_02: Um and then the the final piece I want you to address is what's your answer to the question?
• 33:23
  SPEAKER_02: Is generative AI good at anything meaningful?
• 33:27
  SPEAKER_03: All right, we'll hold that one.
• 33:30
  SPEAKER_03: Let's do the let's do the Apple question.
• 33:34
  SPEAKER_03: I think it is, you know, we're recording this um very shortly after the announcement around the change in leadership at Apple and the new CEO coming from the hardware division, which I think is very interesting.
• 33:46
  SPEAKER_03: Um and I don't know the guy, but probably right.
• 33:50
  SPEAKER_03: Um uh that it should that it should come from the hardware division.
• 33:56
  SPEAKER_03: Um yeah, I mean interesting in the age of AI, I think.
• 34:00
  SPEAKER_02: Very interesting in the age of AI.
• 34:01
  SPEAKER_03: Very interesting.
• 34:02
  SPEAKER_03: Yeah.
• 34:02
  SPEAKER_03: Um Apple is an is an intriguing company because um I think if you look at Siri, one would say that that's an example of um of not high quality.
• 34:24
  SPEAKER_03: Um and and I often wonder what I often wonder about the level of frustration that jobs might have had with Siri.
• 34:37
  SPEAKER_03: Um and how he might have um whether he would have either pulled it entirely and waited until Siri could be genuinely useful, um, or whether you know, much of his interest was in design perfection, not all of it, but much of his interest was in design perfection at the at the hardware level.
• 35:07
  SPEAKER_03: So um so I do wonder I do wonder about that, the nature of the of the company Apple and how it would respond to it.
• 35:17
  SPEAKER_03: But um but what I do think is particularly interesting for Apple specifically is that um part of what has differentiated Apple for a very long time has been a relentless focus on the human experience of interacting with their technology.
• 35:48
  SPEAKER_03: And I think that um as we think about what the human experience is with regard to AI, there's always been a ever since you know we started working with fire, humans' behavior has changed as a result of the technology that they've created.
• 36:20
  SPEAKER_03: Right?
• 36:21
  SPEAKER_03: The way by definition, right, you pick up a tool and it's useful, you work differently, right?
• 36:27
  SPEAKER_03: You start, you invent fire and you live differently, right?
• 36:31
  SPEAKER_03: So you do that's not new.
• 36:33
  SPEAKER_03: You do different things when technology arises.
• 36:37
  SPEAKER_03: But the f the real question now is that we've gone from dedicate largely dedicated tools that have dedicated functions to this one thing that does everything okay.
• 36:56
  SPEAKER_03: Right?
• 36:57
  SPEAKER_03: Yeah.
• 36:57
  SPEAKER_03: And uh everything within its realm.
• 37:01
  SPEAKER_03: Obviously, it's not particularly good at hanging up a painting right now, right?
• 37:04
  SPEAKER_03: But it does but but what of course once it combines with robotics, then it will, right?
• 37:09
  SPEAKER_03: But it but right now it it does everything in that digital realm kind kind of okay.
• 37:16
  SPEAKER_03: The question is, what does that do to us as humans when we're when we're interacting with it?
• 37:24
  SPEAKER_03: What is the medium-term impact of that?
• 37:28
  SPEAKER_03: And there's a question that I've been exploring for some time, which is that I would suggest that we should not assume that just because um we are um we have historically, as human beings, been very, very interested in this in this creative process and high quality, we should not assume automatically that it means that we will continue to be so, right?
• 37:59
  SPEAKER_03: Maybe we've talked a lot about how high quality might be the differentiator here, but it might also be that you know what, we just get so used to doom scrolling that we can't be bothered to seek out quality or look for quality or anything along those lines.
• 38:15
  SPEAKER_03: So I think it behooves Apple as the company that has really understood the interaction between humans and technology and what happens to humans as they're interacting with technology and what humans like and what humans don't like and that and the experience of interacting with technology.
• 38:34
  SPEAKER_03: It it behooves Apple to dive deeper into that and figure out what their AI strategy needs to be associated with that.
• 38:46
  SPEAKER_03: And I'm glad that one of these tech BMOFs has taken a bit more of a back seat in the early stages of this, um, because there is the potential there for them to come up with something that that where the entire experience of being alongside it is much more satisfying than what we do today.
• 39:12
  SPEAKER_03: Um, they can't wait too long, right?
• 39:14
  SPEAKER_03: Everybody's very locked in now to this idea of interacting with uh with these pieces of technology in other ways, but but there isn't there is an absolute opportunity there for an end-to-end kind of ecosystem thing where it becomes a little bit more invisible again, is genuinely helpful and doesn't get shit wrong all the time.
• 39:33
  SPEAKER_03: There's a you know, there's there's a there's a version of that that could be much more satisfying, much more helpful, but critically ideally results in higher quality overall, not just lots of random theoretical shortcuts interspersed with five-hour rabbit hole trips while you're trying to get AI to do something right that it will never get right.
• 39:58
  SPEAKER_03: Yeah, there's gotta be something better than that.
• 40:00
  SPEAKER_01: Yeah.
• 40:01
  SPEAKER_02: Yeah, yeah.
• 40:03
  SPEAKER_02: Well, even though we're immersed in it, you know, uh I think we're still at the front of it, which is which is really interesting to to watch.
• 40:11
  SPEAKER_03: And then you you're reasked your last question.
• 40:15
  SPEAKER_02: Yeah, so so the true the th the thing everybody's waiting to find out.
• 40:20
  SPEAKER_02: What's your answer?
• 40:21
  SPEAKER_02: Is is generative AI good at anything meaningful?
• 40:27
  SPEAKER_03: Yes, in niches.
• 40:30
  SPEAKER_03: So uh for example, um technical translation, pretty good now, right?
• 40:40
  SPEAKER_03: So in other words, the idea of being able to like take a long body of of documents that required a lot of human skill to do this, right?
• 40:49
  SPEAKER_03: To go through a uh uh large-scale document, translate it really authentically from one from one language to another.
• 40:59
  SPEAKER_03: It's pretty good at that now.
• 41:01
  SPEAKER_03: Uh another example that I come up with the uh you stick a human on a bunch of uh on a bunch of documents and ask it to pull references out of that out of those documents.
• 41:15
  SPEAKER_03: Um I didn't I never thought I'd mention the Epstein files in the in the context of our podcast, but that work that had to be done on the um uh on the Epstein files to publish them, yeah, it wasn't perfect, but imagine that being done without AI, right?
• 41:31
  SPEAKER_03: Without you know, given the millions and millions and millions of documents that were there and pulling uh pulling the right information out, redacting the right information, stuff like that.
• 41:40
  SPEAKER_03: So it's stuff like that, um it's it's better in most cases than you know tired, bored humans.
• 41:53
  SPEAKER_03: Uh but can it write a joke better than a human yet?
• 41:57
  SPEAKER_03: Not really, right?
• 41:58
  SPEAKER_03: Can it um so it's a it's it is good at at these things in niches, but what I want to really point out is that there is AI on its own, really good at some things, less good at humans at others, there are humans on their own, really adaptable and resilient and good at a lot of things, but you know, get tired and sick and drunk and things like that.
• 42:34
  SPEAKER_03: And then there is this sort of like well-done magic combo of the two, and in all fields, including creative fields, there is a there is an opportunity for the human plus AI combo to deliver something that neither could have done on its own.
• 43:00
  SPEAKER_03: The example that I often call out um was the Beatles song that was released a couple of um a couple of years ago.
• 43:12
  SPEAKER_03: Um struggling to remember the name of the song, maybe you can look it up while I'm finishing the story off here.
• 43:19
  SPEAKER_03: But the but um people will might remember that a c that a couple of years ago there was a in quotes a release of a new Beatles song.
• 43:31
  SPEAKER_03: And that worked because of the work that Peter Jackson was doing alongside um the now and then.
• 43:43
  SPEAKER_03: Now and then that's it, yes.
• 43:45
  SPEAKER_03: That worked because Peter Jackson was using some AI assisted technology to be able to fully extract and enhance the Lenin lyric.
• 43:57
  SPEAKER_03: You had the surviving beetle.
• 44:00
  SPEAKER_03: Going back in and playing their instruments alongside it.
• 44:06
  SPEAKER_03: You had Giles Martin going in, George's son, George Martin's son going in and acting as a human producer on this stuff, right?
• 44:18
  SPEAKER_03: Humans in control throughout all of this, but AI, absolutely essential to the creation of it.
• 44:26
  SPEAKER_03: And what you got was a song that, you know, if you go back and listen to that, I'm a Beatles fan, you go back and listen to that, whether you think it's a top-tier Beatles song or not.
• 44:36
  SPEAKER_03: It is definitely, it looks, feels, smells like a Beatles song, and it couldn't have existed without the humans, and it couldn't have existed without AI.
• 44:49
  SPEAKER_03: And so that is a that's a great example of where the two come together to produce something that would be impossible without either uh individually.
• 45:03
  SPEAKER_03: And I think that will that'll be maintained, particularly in the you know, in the creative era.
• 45:10
  SPEAKER_03: We've seen it again recently, if folks have seen Epic, the I was presently in concert movie, same kind of technology, same kind of deal, right?
• 45:18
  SPEAKER_03: And it's enabling things that could not that feel definitively human but would be would have been impossible otherwise.
• 45:24
  SPEAKER_03: So I think that's that that's sort of like the guiding light as to as to as to where this can happen, where by combining the two you can create greater quality than either than either um can individually.
• 45:39
  SPEAKER_03: But it also allows us to be able to look very clear-eyed as to what we do really well, what the technology does really well, and how those can combine together to create bigger quality.
• 45:50
  SPEAKER_03: I think that's the that's the that's the bigger picture of all of this.
• 45:56
  SPEAKER_02: Yeah, it's really the answer is our coexistence will make greater things.
• 46:04
  SPEAKER_03: And it's something we have to think really, really, really carefully about because you know you and I are gonna talk a lot more in coming weeks and months about agentic AI and what that means for all of us.
• 46:16
  SPEAKER_03: But let's not forget that as we move further into agentic AI, you only really gain the efficiencies by pulling the humans out of the loop more.
• 46:26
  SPEAKER_03: And so we have to think about this right now because the decisions that we make about how we pull humans out of the loop and when we pull humans out of the loop will have a definitive impact on things like safety and quality, as well as having an impact on things like efficiency, productivity more broadly, and so on.
• 46:48
  SPEAKER_01: Yeah.
• 46:49
  SPEAKER_01: Good stuff.
• 46:50
  SPEAKER_01: All right, Paul.
• 46:51
  SPEAKER_01: Well, until the next one.
• 46:52
  SPEAKER_01: Appreciate the uh conversation.
• 46:54
  SPEAKER_00: Alright, cheers, Matt.
• 46:57
  SPEAKER_00: Humanity Working is produced by Matt Neal.
• 47:00
  SPEAKER_00: If you liked this episode, please take a moment to rate and review us on your favorite podcast platform.
• 47:06
  SPEAKER_00: It makes a huge difference in helping others find us.
• 47:09
  SPEAKER_00: And don't forget, you can find all episodes and get our weekly newsletter at humanityworking.
• 47:17
  SPEAKER_00: See you next time.