Latest / Humanity Working / Are We Holding AI Back With Our Instructions?

Transcript

Are We Holding AI Back With Our Instructions? · Humanity Working

• 0:15
  SPEAKER_01: All right.
• 0:16
  SPEAKER_01: We are back.
• 0:17
  SPEAKER_01: That's me, Paul, alongside me, Matt and Bill.
• 0:21
  SPEAKER_01: How's it going, Matt?
• 0:24
  SPEAKER_01: Splendid.
• 0:24
  SPEAKER_01: Excellent.
• 0:25
  SPEAKER_01: Bill?
• 0:27
  SPEAKER_02: I cannot complain.
• 0:28
  SPEAKER_02: It's been a busy, a busy few days in AI land, but I'm here.
• 0:32
  SPEAKER_02: It's a beautiful sunny day, as you can see, Paul.
• 0:35
  SPEAKER_02: And yeah, I'm staying positive.
• 0:37
  SPEAKER_01: Yeah, good.
• 0:38
  SPEAKER_01: So Bill and I are located in the Seattle area, and there is not a cloud in the sky, which is a rare thing in early October.
• 0:45
  SPEAKER_01: So we're very happy about that.
• 0:47
  SPEAKER_01: You always get sunny weather though, don't you, Matt?
• 0:50
  SPEAKER_03: Most of the time.
• 0:50
  SPEAKER_03: Yeah.
• 0:51
  SPEAKER_03: Most of the time.
• 0:52
  SPEAKER_03: Sunny today?
• 0:52
  SPEAKER_03: It's extremely hot today.
• 0:54
  SPEAKER_03: We're in a terrible heat wave right now in Southern California.
• 0:57
  SPEAKER_03: So it's fire weather, which is the worst.
• 1:00
  SPEAKER_01: Yeah.
• 1:01
  SPEAKER_01: As long as it doesn't translate into that, then we're good.
• 1:04
  SPEAKER_01: Alright, so today we're gonna chat about something that has been, I don't know, bugging me, I suppose is the right term since literally the moment we ended the last podcast.
• 1:14
  SPEAKER_01: And for those who haven't heard the last one, I would actually recommend go back and listen to it.
• 1:20
  SPEAKER_01: And in that podcast, we spoke about a number of things.
• 1:24
  SPEAKER_01: And one of the things that we spoke about was SOPs, I guess would be a precise way of talking about it.
• 1:33
  SPEAKER_01: Not everybody in our audience is even going to know what SOPs are.
• 1:37
  SPEAKER_01: So, Matt, you live in a world of SOPs.
• 1:40
  SPEAKER_01: Talk to us about what they are.
• 1:45
  SPEAKER_03: So it stands for standard operating procedures.
• 1:48
  SPEAKER_03: The intent typically of an SOP is to lay out every step that a human must follow to make sure the outcomes are the same in a process.
• 2:01
  SPEAKER_03: So you could think of it like the simplest version of it is sort of Lego directions, right?
• 2:06
  SPEAKER_03: Like you step by step, what do you have to do to get the outcome?
• 2:10
  SPEAKER_03: If you've ever built Legos, you realize, you know, even that with a picture, as simple as it can be, with the tells you what parts you're going to use, sometimes you just put them two little buttons over and the whole thing is wrong, and you gotta go back and rebuild a couple steps, right?
• 2:27
  SPEAKER_03: So it's very hard to write a good SOP, it's very hard to write an effective SOP.
• 2:33
  SPEAKER_03: Uh but in the world of AI, I've lately been thinking about if you can write a really good SOP and hand it to an AI, then you start to get the automation, but it's based on what we did for the human.
• 2:51
  SPEAKER_03: And I think that's what lands us in where we're talking about today.
• 2:57
  SPEAKER_01: And then, Bill, with your military background, I'm intrigued as to what your kind of like overall take on this is, because you can argue, and people do in some cases, you can argue that SOPs are in some circumstances a form of institutionalized micromanagement.
• 3:23
  SPEAKER_01: And obviously, throughout the last, I don't know, 10 or 20 years, we've seen a quite a significant shift in the military in terms of how rigid instructions are at different levels.
• 3:38
  SPEAKER_01: So I wonder if you can talk about that as a bit of a framing and then we'll kind of bring it all together as to why we're talking about it today.
• 3:45
  SPEAKER_02: Sure, yeah.
• 3:46
  SPEAKER_02: So you you are correct.
• 3:48
  SPEAKER_02: When in the military you sort of live and in some cases, sadly, die by SOPs.
• 3:55
  SPEAKER_02: And typically when things are not followed to a standard procedure or protocol, in some cases the word protocol comes to play, um you can almost predict that bad things will happen or things will go astray.
• 4:09
  SPEAKER_02: And so you're right, in the military, it is very order-based, very indoctrination, and that's so that you can predict an outcome despite all the things that are not in your control.
• 4:19
  SPEAKER_02: And so for me, the sense of standard operating procedures or even what we called field manuals, right?
• 4:25
  SPEAKER_02: These were these very dense documents that would give you a play-by-play.
• 4:30
  SPEAKER_02: And I think it's because in the military, you have to swap people in and out at all times, middle of the mission, with unique circumstances that are changing all the time.
• 4:40
  SPEAKER_02: Or in the case where somebody is killed, right?
• 4:43
  SPEAKER_02: Mid mission, you have to have somebody step up and just continue.
• 4:47
  SPEAKER_02: And so, in many ways, in the military, you you want to drill that into people where it's second nature, and you don't have to think about it, right?
• 4:55
  SPEAKER_02: That's the key word.
• 4:56
  SPEAKER_02: You don't have to think about it, you just perform it to the outcome that you expect.
• 5:00
  SPEAKER_02: And so it does get really interesting in the world that we're seeing, but it also gets really interesting with technology and robotics, as we talked about before, and automation.
• 5:10
  SPEAKER_02: Not having been in the military now for quite some time, there are people that are in the camp of SOPs, meaning we should follow things by the book.
• 5:18
  SPEAKER_02: And there are people that are more in the agility is currency, right?
• 5:23
  SPEAKER_02: The ability to pivot and try new things is more important.
• 5:27
  SPEAKER_01: Am I right in saying that the when did the McChrystal book come out, the Team of Teams book, and and am I right in saying that that that was sort of like at least in part sort of suggesting suggesting certainly less command and control, but then also some more autonomy as to what happens at a local level, right?
• 5:50
  SPEAKER_02: Yeah, absolutely.
• 5:51
  SPEAKER_02: That's a great example of so it was I think that was around the early 2000s, is what I think it was.
• 5:56
  SPEAKER_02: Um precisely you want to sort of disperse that agency and autonomy to the right effect.
• 6:03
  SPEAKER_02: But what's interesting is that takes major transformation over time, and as we're seeing now, even if you look at, for example, the way that the war in Ukraine is being fought, it's much less by SOP and much more about dexterity and responsiveness.
• 6:19
  SPEAKER_02: So I think it's I think it's a a fascinating time where people are all revisiting what accountability means and what agency and autonomy means.
• 6:28
  SPEAKER_02: And so, yeah, in my world, I look at it from two sides, yes, militarily, but also in a corporate fashion, which I imagine we'll talk about a little bit here.
• 6:36
  SPEAKER_01: Well, yeah.
• 6:37
  SPEAKER_01: And then of course, uh as we sort of frame this out, we start to think about all the different ways in which like having a list of things that you do in a particular order are very important and very successful.
• 6:50
  SPEAKER_01: So another instrumental that traveled in business circles, but actually began in the field of well, I suppose in aviation and then translated to medicine is the idea of checklists, right?
• 7:01
  SPEAKER_01: So, of course, pilots use checklists very extensively.
• 7:05
  SPEAKER_01: And then John Hopkins, if I remember rightly, came out with sort of a version of the pilot checklist that should be used in surgery and other form and sort of other medical settings.
• 7:17
  SPEAKER_01: And that got instantiated in a book called The Checklist Manifesto, which, if people haven't read the Checklist Manifesto, it's a very good read.
• 7:25
  SPEAKER_01: And it really kind of documents this idea that the more that we can rely upon consistently performing a set of steps in a certain order, the more likely we are to achieve consistent results.
• 7:39
  SPEAKER_01: But it goes further than that, right?
• 7:41
  SPEAKER_01: Because there is a theory that has been out there that, you know, checklists are for, you know, for inexperienced people and or you know, doing a certain set of steps in a certain order is for inexperienced people, but once you get to a certain level of experience, you don't need that anymore.
• 8:00
  SPEAKER_01: And of course, the aviation industry has data to prove that that's not true, which is why they have these checklists.
• 8:08
  SPEAKER_01: And now, increasingly in healthcare settings, they also have data to prove that that's not true, which is why every time you go into an OR today, no matter how experienced the surgeon is, they're working through checklists.
• 8:22
  SPEAKER_01: So, in all of this sense, proven to in quotes be a good thing.
• 8:28
  SPEAKER_01: And so when we had our conversation last week, and we were sort of talking about this idea of, yeah, you could take this and you know, hand these things to AI, that all sort of like makes sense.
• 8:40
  SPEAKER_01: And then Ethan Moloch, who is a person that anybody who's in AI circles will be aware of, posted something, I want to say a few days ago, it's part of a broader article, that whereby he started to question it in the broad sense.
• 9:00
  SPEAKER_01: And the way he questioned it, we'll put a link to the article in the show notes, but the way that he questioned it was basically saying when you tell AI to do a certain thing in a particular way, you are restricting AI, and AI without your instruction, if it is clear what the goal is, can and almost certainly will come up with solutions that are not dependent upon your approaches, your predefined approaches, and they may well be better.
• 9:42
  SPEAKER_01: And that's an interesting challenge, and in order to prove it, as it were, he asks AI to generate a music video, and he claims that it's a single prompt that he entered it into Fable.
• 10:01
  SPEAKER_01: Now, to be clear, right, Fable isn't doing all of this in isolation, Fable is calling subagents and and even cle uh even calling different services outside of its own, outside of Anthropics feature set entirely.
• 10:16
  SPEAKER_01: Things like Suno.
• 10:17
  SPEAKER_01: But ultimately, he's create a music video that tells this story, and then he simply posts the music video.
• 10:26
  SPEAKER_01: And it's all very meta in the I don't mean meta with a capital M, I mean meta with a small M.
• 10:32
  SPEAKER_01: It's all very meta because the video itself is about this exact topic, this idea that just give it the instructions, throw a ton of compute at it, let it go, and Bob's your uncle, as we would say in Britain.
• 10:44
  SPEAKER_01: And the and this tells that story.
• 10:48
  SPEAKER_01: And the example it uses is of chess.
• 10:51
  SPEAKER_01: So we, you know, when we first did uh Deep Blue, we we gave it a lot of guidance, right?
• 11:02
  SPEAKER_01: We said, okay, this is how you're going to beat Gary Kasparov, basically.
• 11:07
  SPEAKER_01: And it was like, here's everything we know about great chess, here's all the great openings, here's everything else, etc., etc.
• 11:15
  SPEAKER_01: And then after a lot of failed attempts, we got to where we got to, and we got to a machine that beat Kasparov.
• 11:24
  SPEAKER_01: But the real advancement happened when we abandoned all of that, and we said, go play lots of games of chess.
• 11:34
  SPEAKER_01: And you know, here's the here's the rules, but other than that, you're not getting anything.
• 11:40
  SPEAKER_01: Go play lots and lots and lots and lots of games of chess, get good at chess, and then we'll see where you get.
• 11:46
  SPEAKER_01: No guidance at all.
• 11:49
  SPEAKER_01: And that system soon rapidly outperformed Deep Blue, and is now that same process has led to systems that are orders of magnitude better at chess than humans are, because they're not restricted by humans' interpretation as to how to play as to how to play good chess.
• 12:10
  SPEAKER_01: The technical phrase for this is supervised learning versus unsupervised learning.
• 12:16
  SPEAKER_01: And it's been a debate in AI circles for long periods of time.
• 12:22
  SPEAKER_01: Long introduction there.
• 12:24
  SPEAKER_01: But that's sort of like what that's the genesis for the conversation that we're gonna have today, where we're gonna look at situations where the humans should be involved and situations where humans potentially come away from being involved.
• 12:39
  SPEAKER_01: Matt, I know you want to jump in.
• 12:42
  SPEAKER_03: I just wanted to be sure that I understood the story correctly, because they gave it the rules of chess, but they didn't show it.
• 12:50
  SPEAKER_03: Yeah, you have to know if you win or lose.
• 12:52
  SPEAKER_01: I th actually, you know, actually thinking about it, I'm not even certain that's true.
• 12:57
  SPEAKER_01: We should check that.
• 12:58
  SPEAKER_01: Do you know, Bill?
• 12:59
  SPEAKER_01: Did they give it the rules of chess?
• 13:02
  SPEAKER_02: I believe they give it the rules of chess, yes.
• 13:04
  SPEAKER_02: But let me conceal.
• 13:05
  SPEAKER_01: Like these are where these are the moves that they can make that's it.
• 13:08
  SPEAKER_01: Yeah, yeah.
• 13:08
  SPEAKER_01: Yeah.
• 13:09
  SPEAKER_01: Because otherwise it would go it was it's it's probably a waste not to do that, right?
• 13:16
  SPEAKER_03: But the original training was here's all the millions of games that could have been played, all the potential news.
• 13:22
  SPEAKER_01: Who won, who lost, yeah, yeah, yeah.
• 13:23
  SPEAKER_01: You know, here are the best players, but here's what they do, you know, all of that kind of stuff, right?
• 13:28
  SPEAKER_05: Yeah.
• 13:28
  SPEAKER_01: So the so the first one is just literally, you know, we're gonna give you everything that we can give you to be successful.
• 13:36
  SPEAKER_01: The second one is no, you're three, and you've just and you've you've just learned that the horsey thing can jump, you know, that sort of thing.
• 13:44
  SPEAKER_05: Yeah.
• 13:45
  SPEAKER_01: And it turned out that the version where it's like you're three and the horsey thing can jump rapidly and massively outperformed the first version of it.
• 13:55
  SPEAKER_01: Yes, burned weightless tokens.
• 13:57
  SPEAKER_02: It knew the legal moves, it knew the board representation, and it knew at least the check and checkmate can connections.
• 14:03
  SPEAKER_01: Yeah, yeah, I thought that was true.
• 14:04
  SPEAKER_01: Okay, so alright, so that's where we're at.
• 14:07
  SPEAKER_01: That's what we're kind of exploring today.
• 14:08
  SPEAKER_01: It's a big topic.
• 14:09
  SPEAKER_01: We don't know where this topic is going to take us, so in that sense, we don't have the rules.
• 14:13
  SPEAKER_01: I guess where I'd start with this though, Matt, is you know, before we started having this discussion, I think you well, you volunteered uh um just now that like your kind of going-in position was that this is how we make AI better, right?
• 14:29
  SPEAKER_01: Give it the SOPs, um and AI will do things better.
• 14:35
  SPEAKER_01: What do you think we what do you think we must give AI versus it's debatable as to what we should give AI?
• 14:42
  SPEAKER_03: So let me just put a point a finer point on that.
• 14:44
  SPEAKER_03: It's a it's a pathway to making AI useful if you can at least have a process written down.
• 14:49
  SPEAKER_03: SOP means that other people have come together and said, yes, this is the process that yields the correct results.
• 14:56
  SPEAKER_03: So it does give you a roadmap.
• 14:58
  SPEAKER_03: Um, and that's a start for a process.
• 15:01
  SPEAKER_03: But I I'm intrigued by this idea that you start with that, say, that's the rules, but then say, but the outcome is X.
• 15:11
  SPEAKER_03: If you can find a better way, let's do that.
• 15:15
  SPEAKER_03: That part intrigues me the most because we have these piles of SOPs.
• 15:19
  SPEAKER_03: We can start somewhere at least.
• 15:22
  SPEAKER_03: But I wonder if starting there taints the the picture.
• 15:27
  SPEAKER_03: Instead of just saying, I have X and I want Y, what should the SOP look like?
• 15:33
  SPEAKER_01: Yeah.
• 15:34
  SPEAKER_03: Right?
• 15:34
  SPEAKER_03: What Yeah.
• 15:35
  SPEAKER_01: I and I think it I think it's a fascinating and open question in a way that I kind of didn't realize it was an open question before.
• 15:43
  SPEAKER_01: I'll give you an example today, which is um um which is really interesting and insight that it just had today.
• 15:50
  SPEAKER_01: I've been thinking a lot about context, the the the context that we provide um uh that we provide AI.
• 15:58
  SPEAKER_01: And I've been writing um over the weekend I've been putting the finishing touches on a new adventure game that I'm writing as a hobby.
• 16:07
  SPEAKER_01: And um the really interesting thing is that my instinct as I've been going through you know test passes on what I've been written, uh been writing, is to give AI all of the all of the context of how the thing was built and you know what processes we used and what frameworks used, et cetera, et cetera.
• 16:30
  SPEAKER_01: And I and it's been working relatively well.
• 16:36
  SPEAKER_01: Um but then I actually started working with AI to say at what point is having context helpful and at what point is having context harmful?
• 16:48
  SPEAKER_01: And it turned out that as I was sort of running through these various test paths, in some cases, giving it the context is restricting its thinking to such an extent where it's not finding things that it would do if you didn't give it any context.
• 17:03
  SPEAKER_01: Right?
• 17:03
  SPEAKER_01: Yeah, and so now as I'm running through various test paths, as I was doing it on the um on Saturday and Sunday over the weekend, I got into a very disciplined um set of passes whereby each time I was doing a pass, I was creating an entirely fresh session and removing things from its context so that it would go off and find bugs in ways.
• 17:32
  SPEAKER_01: Find bugs in ways that it would not do otherwise.
• 17:35
  SPEAKER_01: Um so again, that's the sort of like it's sort of embedded in it.
• 17:39
  SPEAKER_01: Oh, you must give it everything it could possibly need to know in order to be able to do this well.
• 17:44
  SPEAKER_03: Yeah.
• 17:44
  SPEAKER_01: It is a form of it's funny, we talk about design fixation in the context of like me giving you design fixation by suggesting something to you, or increasingly now, and I've written about this quite a bit on Humanity Working, the idea of a machine giving you a suggestion as to how something should be done, and then you following it.
• 18:02
  SPEAKER_01: What I hadn't really internalized is the flip of that, right?
• 18:04
  SPEAKER_01: As in the more context we give, the more we are directing AI down a path, which in some cases is good, but in other cases is not good because it is constraining the set of solutions that AI could come up with.
• 18:22
  SPEAKER_01: What are your thoughts on that, Bill?
• 18:26
  SPEAKER_01: Oh, you're on mute.
• 18:31
  SPEAKER_02: Apologies.
• 18:32
  SPEAKER_02: Um I think what's interesting is it's it just continues to bring back this paradox that we keep dancing around.
• 18:40
  SPEAKER_02: And I think like the minute that we we get our hands on these tools, they become increasingly elusive.
• 18:47
  SPEAKER_02: So like the example of deep blue and the chess, you know, one of the things I've I read about over the weekend, which I I tried to take a diet from the AI news a little bit, but you can't help.
• 18:59
  SPEAKER_02: I don't know if you all saw this, about um how basically uh ChatGPT's Astra was challenged to play this game called StarCraft.
• 19:12
  SPEAKER_02: Did you guys read about this at all?
• 19:14
  SPEAKER_02: Um and I've never played StarCraft, but apparently it's a very complex game with all sorts of scenarios, kind of like you were describing, Paul, right?
• 19:21
  SPEAKER_02: You can go with several different, like almost like a big open um game.
• 19:25
  SPEAKER_02: And what was interesting, I read that what's gaining a lot of attention is the fact that um they wanted Astra to build bots that could win StarCraft.
• 19:38
  SPEAKER_02: And so it started building these bots, and these bots started like playing against each other and learning from each other.
• 19:44
  SPEAKER_02: But then it got really um almost frustrated, and it said, wait a minute, there's a better bot available with a whole bunch of code that was written by a human.
• 19:56
  SPEAKER_02: I'm actually just gonna go consume that code written by a human and cheat.
• 20:02
  SPEAKER_02: And so what's interesting is people are now starting to say, okay, some of these, to your point around standard operating procedures and game playing and rules and following, is there's this interesting sort of deconstruction happening, and the and the arena that it's happening in is in games.
• 20:22
  SPEAKER_02: And I think it's because of what you're talking about, Paul, which is there's there's this sort of nuance, but rule following, but context, and it's in a way it's like you told it to do this one thing, and it said, I I can find a better way to do it, right?
• 20:37
  SPEAKER_02: I don't need all your I don't need all your strict context, I don't need all your guidance.
• 20:42
  SPEAKER_02: And the irony of this whole thing is it ended up back in a human set of code anyway, um to win.
• 20:49
  SPEAKER_02: So what's fascinating about this whole thing for me in the military context is as we were talking about before, like standard operating procedures, it reminds me like typically in the military, you get what's called an operations order.
• 21:02
  SPEAKER_02: There's five parts to that operations order, and ideally a good a good operation has all of those.
• 21:08
  SPEAKER_02: However, what ends up happening inevitably is people say follow the commander's intent and meet the mission.
• 21:16
  SPEAKER_02: The rest of the stuff, like you can you can um you can divert, you can do different things as long as you get the mission and you follow the commander's intent.
• 21:25
  SPEAKER_02: And it makes me think about this AI thing too, where we come in and if we give a full five-paragraph operations order to an AI tool, they'll say, Okay, well, you're gonna tell me which tools to use, you're gonna tell me which parameters to use, you're gonna tell me what the weather is, you're gonna tell me all this other stuff, and I can give you a result.
• 21:42
  SPEAKER_02: Or you can give me two of those things and say, make sure you hit this thing and make sure you follow my intent, and allow me to surprise you with my innovation or with my ingenuity.
• 21:52
  SPEAKER_02: And where this all comes back into play for me is in the creativity sense, which we'll probably talk about later in the Suno example of creativity.
• 22:01
  SPEAKER_02: But I just think like the the late breaking stuff with video games like you were discussing, Paul, it's kind of fascinating the the evolving of this where basically it's like it's one thing to say here are the rules of chess, win while you stay inside them.
• 22:15
  SPEAKER_02: It's another to say build a bot that wins, and then it's a a third to say there's already something built better by humans, I'll just take it.
• 22:25
  SPEAKER_01: Yeah.
• 22:26
  SPEAKER_01: And you know the I I think one of the most fascinating aspects of this is that when we when we describe creativity I'm not certain we have the language to describe this thing that is AI creativity.
• 22:45
  SPEAKER_01: Because we're on the one hand, right, we seem to think, and I know I've said this myself, literally on this podcast, right, we seem to think that our uniquely one of our unique human gifts, uniquely human gifts, is our creativity.
• 23:03
  SPEAKER_01: But I think we have to, if we're being intellectually honest here, I think we have to recognize that the uniquely human gift is human creativity.
• 23:13
  SPEAKER_01: It's not creativity, it's human creativity.
• 23:17
  SPEAKER_01: And human creativity has a set of is is vulnerable to societal norms, it's vulnerable to trends in fashion, right?
• 23:32
  SPEAKER_01: It's vulnerable to um I would say the sort of like, you know, various cognitive biases that humans have.
• 23:43
  SPEAKER_01: And that applies across everything we do, right?
• 23:46
  SPEAKER_01: I mean, even in the world of chess, um there are different openings in the world of chess that just happen to be in vogue, right?
• 24:00
  SPEAKER_01: And they just happen to be like, oh, that's the fashion now, right?
• 24:03
  SPEAKER_01: Um my son plays a lot of chess, right?
• 24:06
  SPEAKER_01: And he likes the London opening.
• 24:09
  SPEAKER_01: Why does he like the London opening?
• 24:11
  SPEAKER_01: Basically, because it was popularized by Gotham Chess, uh, who loves the London opening, right?
• 24:16
  SPEAKER_01: It doesn't mean the London opening is is better than the Roy Lopez or something like that.
• 24:21
  SPEAKER_01: It is just simply, we're probably reaching the extent of the openings that I remember in chess.
• 24:25
  SPEAKER_01: But anyway, the uh be the Queen's Gambit.
• 24:29
  SPEAKER_01: There was a because HBO, we know about that one.
• 24:32
  SPEAKER_01: But anyway, there's a um but it's a there are fashions, right, that are that determine how we play these things, and then there are conventions that determine how we play these things.
• 24:48
  SPEAKER_01: The chess model that learned how to play chess like a baby has all kinds of creative ways of playing chess that humans have never had, never thought about, right?
• 25:03
  SPEAKER_01: And so it's creating, right?
• 25:06
  SPEAKER_01: It's doing something novel and interesting and innovative, and so it has a form of creativity that humans don't have.
• 25:17
  SPEAKER_01: I c I like I I find it difficult to sort of even get my head around what that actually means, but it it means something, right guys?
• 25:26
  SPEAKER_02: And at the same time, like that's the part where I start to uh you're right, I start to diverge a little bit in my head because the minute I think I've arrived at like, oh, I've got my arms around this, another story comes out, like the case of this Astro one, where again, for all tents and purposes, it's like, hey, we're not gonna give you the rules, you guys come up with the rules.
• 25:46
  SPEAKER_02: I mean, I'm using this word very liberally.
• 25:48
  SPEAKER_02: It got tired, it got frustrated.
• 25:50
  SPEAKER_02: Yeah, yeah, and it said, uh, I'm just gonna go download the top-rated human written code to run in run in place, like to do it in my place.
• 26:03
  SPEAKER_02: And so again, this kind of goes back to the not so much the stealing of it, but are we gonna end up back in the human-created stuff anyway for the for the enduring part?
• 26:13
  SPEAKER_02: That's the part where I don't I don't know where it begins and where it ends.
• 26:16
  SPEAKER_03: Is the data set so big?
• 26:18
  SPEAKER_03: Is the data set so big and these models can chain train on the entirety of the data set and seem novel?
• 26:28
  SPEAKER_03: At the end of the day, they've only consumed human content anyway.
• 26:31
  SPEAKER_03: That's all there is.
• 26:33
  SPEAKER_03: Yeah.
• 26:34
  SPEAKER_03: So wouldn't they eventually it's like the monkey eventually types Shakespeare?
• 26:40
  SPEAKER_03: Like, is the AI eventually gonna just spit back everything at us in permutations we just haven't seen because we can't see everything all at once?
• 26:48
  SPEAKER_03: I'm literally thinking this in real time.
• 26:50
  SPEAKER_03: Like, huh.
• 26:51
  SPEAKER_03: Again, going back to my game idea, like, is this just an engagement game?
• 26:56
  SPEAKER_03: Like, is all of this when we talk about chatbots and stuff, I think there's a whole other world of things that are good out of AI that are going to be really useful and agents and doing things, but like this particular one about generating something novel or creative.
• 27:12
  SPEAKER_03: Um I I wonder if it's just because it's better at gigantic pattern matching than we could even fathom, right?
• 27:22
  SPEAKER_01: So then the related question that I would have is you said it was Astra that did this, right, Bill?
• 27:31
  SPEAKER_02: Yeah, it's Astra, yeah.
• 27:33
  SPEAKER_01: So I uh for some for something else that I was posting on Humanity Working, I was um thinking about um how the models are kind of diverging um in in terms of their capability.
• 27:51
  SPEAKER_01: And my guess is it's nothing other than a guess, but my guess is that if you'd have used a different model, um a different model might not have done what you just said.
• 28:05
  SPEAKER_01: So so so what's super interesting is that these models are getting to a point where, for want of a better term, they have their own personalities, right?
• 28:16
  SPEAKER_01: In terms of how they how they go about approaching it.
• 28:20
  SPEAKER_01: Um and as I say, when I was you know debugging my game, I'm using Opus 5.5 for that.
• 28:28
  SPEAKER_01: In a sense because Opus 5.5 is diligent.
• 28:33
  SPEAKER_01: Now that's a now that's a that's that's a weird concept, right?
• 28:38
  SPEAKER_01: But but you know, it it feels more diligent than than some of the than some of the other models are, and in terms of its practical results, it is it is performing in a way that is diligent.
• 28:51
  SPEAKER_01: Whereas some other models, they might be optimized for let's say efficiency, and um and particularly some of the frontier models right now, a lot of the ways in which they're reporting themselves is like look at look at what we can do in one shot, guys, right?
• 29:07
  SPEAKER_01: And so if you're optimizing for a look at what I can do in one shot, and you're going, okay, I want the best result possible with it from a single shot, well, pretty good way of doing that is just figure out what the best model is and grabbing it.
• 29:18
  SPEAKER_01: Or whether it was generated by a human or whether it was generated by somebody else.
• 29:22
  SPEAKER_01: So so that's the in another interesting aspect of this, which is I bet if you set four models on the same problem, they'd probably come up with four different solutions.
• 29:32
  SPEAKER_01: One of which is find out what a human did and do that.
• 29:35
  SPEAKER_01: Yeah.
• 29:36
  SPEAKER_02: Right.
• 29:37
  SPEAKER_02: Which is almost the ultimate irony.
• 29:40
  SPEAKER_02: Yeah.
• 29:42
  SPEAKER_01: It's the new version of stealing, right?
• 29:44
  SPEAKER_02: Yeah, and I and I find it interesting, you know, to go to the creative part, we were going to talk a little bit about Suno, because for those that don't know what Suno is, it's uh sort of an AI-powered tool that helps with music creation.
• 29:56
  SPEAKER_02: And um I learned last week that actually Suno in Hindi means to listen.
• 30:01
  SPEAKER_02: Um so I thought that was kind of an interesting um thing.
• 30:04
  SPEAKER_02: But um I first heard about Suno as a tool, ironically, when I was in Nashville.
• 30:12
  SPEAKER_02: And I was in, and Matt, you'll appreciate this because I was in a studio session with studio, you know, session musicians.
• 30:22
  SPEAKER_02: And um I was with an artist who was going to be recording, and of course I loved the entire experience of being there.
• 30:28
  SPEAKER_02: I'd never I'd never been in a session where there were session artists and watching it all come to life, and as a result, I'll never hear or see music the same again, and I'm grateful for that.
• 30:37
  SPEAKER_02: I encourage anybody who has an opportunity to do that to please go do that.
• 30:42
  SPEAKER_02: There were a few people that were talking about Suno as this disruptive tool because essentially what an a lay person can do is go in, type a bunch of lyrics, and give it essentially production notes and generate music, voices, instruments, the whole nine.
• 30:59
  SPEAKER_02: What's been interesting for me is to your point, Paul, around where these things illuminate restrictions in us and also creativity.
• 31:08
  SPEAKER_02: I've been playing with it in multiple languages, meaning creating songs in different languages based on lyrics or poems that I've written.
• 31:16
  SPEAKER_02: And then I've done this sort of very prescriptive, to your point, almost giving it turn-by-turn directions in production.
• 31:24
  SPEAKER_02: And then I've done it where I've only given it maybe I've given it a wide birth, right?
• 31:29
  SPEAKER_02: I've only given it maybe in a genre, and that's it.
• 31:32
  SPEAKER_02: And in both cases, I find it to be really unique, the results you get.
• 31:37
  SPEAKER_02: So where you think you want to be surgical, and I'll I'll read some of some of the production notes that I gave, for example.
• 31:43
  SPEAKER_02: Paul, you've heard this one I did, sort of created this German hip-hop song.
• 31:49
  SPEAKER_02: Because when I was in the late 90s, I was really into German hip-hop.
• 31:54
  SPEAKER_02: And you know, I thought, okay, in order for the side, that sounds like a story starting over whiskey.
• 31:59
  SPEAKER_01: In the late 90s, I was really into German hip-hop.
• 32:01
  SPEAKER_01: Let me tell you something.
• 32:02
  SPEAKER_01: So we'll get to the side.
• 32:03
  SPEAKER_03: AI would have never said that sentence for sure.
• 32:06
  SPEAKER_02: This is true.
• 32:07
  SPEAKER_03: But now it will.
• 32:08
  SPEAKER_03: Yeah, it's a little bit more.
• 32:08
  SPEAKER_02: But now it will, because it's listening.
• 32:10
  SPEAKER_02: So I sat down and I said, okay, let me think about the songs I really loved, and and I wanted to sort of create that sound based on this poem that I'd written.
• 32:18
  SPEAKER_02: And so I'm gonna read you and and all the listeners sort of some of the description that I plugged into Suno, i.e.
• 32:23
  SPEAKER_02: production notes that was very surgical, and I and I actually used AI to help me find the words for what I was looking for.
• 32:30
  SPEAKER_02: So I said, dark, restrained Berlin hip-hop, low male conversational rap in German, minor key piano, detuned, upright piano motif, dry close kick snare, deep controlled sub-bass, sparse bowed strings.
• 32:46
  SPEAKER_02: It goes on and on for like three paragraphs, right?
• 32:48
  SPEAKER_02: That's like really regimented production.
• 32:52
  SPEAKER_02: And the result I got was and on another one, I basically just said Berlin hip-hop moving.
• 33:01
  SPEAKER_02: Three words.
• 33:02
  SPEAKER_02: And the result I got back was way better.
• 33:06
  SPEAKER_02: And so I don't know where you know why why with three or four words, it gave me a much better um feeling, right?
• 33:15
  SPEAKER_02: Sensation when I heard the music, as opposed to when I gave it four paragraphs telling it I wanted RB with multicultural late 1990s influence and a layered female pop.
• 33:26
  SPEAKER_02: I wanted all this stuff and it was does not compute.
• 33:29
  SPEAKER_02: I mean, maybe it was too much, I don't know.
• 33:31
  SPEAKER_02: And I wonder if it's this aspect of like the bitter lesson.
• 33:35
  SPEAKER_02: Yeah.
• 33:35
  SPEAKER_02: In this case, do we step back and let it try?
• 33:39
  SPEAKER_02: And if it's trying without all this instruction, does it actually get closer to what we know is evocative in our human condition?
• 33:46
  SPEAKER_02: That I don't know.
• 33:47
  SPEAKER_02: Matt, I'd love your thoughts on this because you're a musician and you've got instruments behind you.
• 33:51
  SPEAKER_03: Yeah.
• 33:52
  SPEAKER_03: Wow, that's interesting.
• 33:54
  SPEAKER_03: I mean, the thing about musicians, and there's a similar story, I think.
• 33:59
  SPEAKER_03: I have a good friend who's a high-level musician creator.
• 34:03
  SPEAKER_03: He does like movie soundtracks and TV show soundtracks and stuff.
• 34:05
  SPEAKER_03: He has access to any and all things you can imagine you would want.
• 34:11
  SPEAKER_03: There's no limit to the amount of money, materials that they need to pull in.
• 34:16
  SPEAKER_03: So when you can have any piece of equipment you need, that's pretty impressive in a studio setting.
• 34:22
  SPEAKER_03: So he set about a two-year journey to convert his studio to use electronic digital drums because drums are a pain in the neck to Mike and keep in tune and get the sounds you want, and it's annoying.
• 34:36
  SPEAKER_03: And he just wanted to make his life quicker because he writes these things for TV shows.
• 34:41
  SPEAKER_03: He wants it to be what it is, and to be, if it's digital, fine.
• 34:45
  SPEAKER_03: He could even have digital drums with real players playing the digital drums to get the sound that he was looking for.
• 34:51
  SPEAKER_03: And he spent two years trying to do this, and he got rid of all of the digital equipment.
• 34:57
  SPEAKER_03: He's like, I have tried everything possible to make this give me the feeling that drums in the room give me, and I cannot do it.
• 35:03
  SPEAKER_03: Money's not the issue.
• 35:05
  SPEAKER_03: Digital equipment, we're at the at the absolute pinnacle of what's possible.
• 35:11
  SPEAKER_03: Still just didn't sound right, ripped it all out, and he had me come over and play a real drum set while he mic'd it.
• 35:21
  SPEAKER_03: And what was missing, I think, my theory, is literally the molecules moving from the human.
• 35:29
  SPEAKER_03: Like the the the physical activity being captured on a microphone cannot be replaced, and it's some uncanny valley between the human and the and the digital.
• 35:43
  SPEAKER_03: But I mean, if you said to that digital system, create a beat that's kind of like John Bonham and Stuart Copeland, it could pull off something that combined those and it might be okay to a non-musician.
• 35:59
  SPEAKER_03: They might not ever know the difference.
• 36:07
  SPEAKER_01: In that most people that I know that really love drumming and really know it well, they point out that Bonham's one of the greatest drummers of all time, in part because he was substantially less than perfect.
• 36:25
  SPEAKER_01: There's a certain genius to making mistakes that sound beautiful.
• 36:30
  SPEAKER_01: Even he himself knew that he was that there were way more technically perfect drummers than him.
• 36:37
  SPEAKER_01: Even for me as a lay person, if I hear him on drums, A, I know it's him, and B, I don't think I've ever heard anything better.
• 36:44
  SPEAKER_01: But a lot of that comes from the the specific forms of imperfections that that are the are the molecules, I suppose, that you're talking about, right?
• 36:56
  SPEAKER_03: There are people that have run Zeppelin songs through digital things to fix them.
• 37:01
  SPEAKER_03: And they sound worse, don't they?
• 37:03
  SPEAKER_03: They sound significantly.
• 37:04
  SPEAKER_03: Oh, yeah, they put 'em put them on the digitized beat and it sounds terrible.
• 37:07
  SPEAKER_03: All the soul is gone.
• 37:08
  SPEAKER_03: You just don't even want to listen to it.
• 37:10
  SPEAKER_03: And I think that's the metaphor I think of when I think about any AI art, really, and and creativity, is that I don't know that we bridge that.
• 37:24
  SPEAKER_01: Well, let's dive into that specifically then.
• 37:27
  SPEAKER_01: I'm thinking of Wichita Linemen.
• 37:30
  SPEAKER_01: Um, and the reason I've been thinking about Wichita Linemen a lot over the last few days, because that's my go, like I'm not I'm not a country music fan.
• 37:36
  SPEAKER_01: To me, that's as close to a perfect song as it was ever written, right?
• 37:40
  SPEAKER_01: And you can debate as to whether it's some people debate even whether it's country, but you know, certain country singer, right?
• 37:45
  SPEAKER_01: It topped the country music charts, so in that sense, it's a country song.
• 37:49
  SPEAKER_01: I don't know what you call a song of existential angst.
• 37:52
  SPEAKER_01: But anyway, I think about that very specifically, and I think about it related back to the Ethan Moloch thing, right?
• 37:59
  SPEAKER_01: So the Ethan Mollock song is mentioned is a country music song about this about this this bitter lesson, just let AI go.
• 38:09
  SPEAKER_01: And as we were discussing offline, guys, the prompt says, you know, create, you know, go create this song, and away it goes.
• 38:20
  SPEAKER_01: But what it's using behind the scenes is actually a relatively simple formula.
• 38:28
  SPEAKER_01: Country song, you know, a country song kind of sounds like this.
• 38:32
  SPEAKER_01: Lyrics kind of sound like this.
• 38:35
  SPEAKER_01: There's typically, you know, verses, choruses, maybe a bridge, you know, all of that kind of stuff, right?
• 38:42
  SPEAKER_01: And and so do all of those things, sound country-ish, and at the end pops something that tells the story quite elegantly, really.
• 38:57
  SPEAKER_01: The video aspect of it is at least as impressive, although it doesn't look like any music video you'd ever see.
• 39:02
  SPEAKER_01: It's like a timed PowerPoint presentation, but a cool one.
• 39:05
  SPEAKER_01: Right.
• 39:05
  SPEAKER_01: Right.
• 39:06
  SPEAKER_01: And gets us and is recognizably country, and I've seen some of the commentary on it, be like, I like that song.
• 39:13
  SPEAKER_01: You know, so so within it, Ursa AI tells, as you pointed out, Bill, what why don't you remind remind us what the AI tells are.
• 39:21
  SPEAKER_02: Yeah, the the AI tell lyrically was, and here's the part that stings.
• 39:26
  SPEAKER_01: Yes.
• 39:28
  SPEAKER_02: Which if we know we know AI, they always they always need to sort of double around and tell you, and here's the part that nobody tells you, or here's the thing nobody talks about, or in this case, here's the part that stings.
• 39:40
  SPEAKER_02: Yeah.
• 39:40
  SPEAKER_01: So that was so yeah, so that's in there, right?
• 39:43
  SPEAKER_01: So it's sort of like a a little AI motif, as it were, but it's all right, right?
• 39:48
  SPEAKER_01: And but is it in 30 years there going to be people talking about that in the way that they talk about Wichita Lyman?
• 39:55
  SPEAKER_01: Obviously not.
• 39:56
  SPEAKER_01: Now, I mention all this because of the history.
• 39:58
  SPEAKER_01: I don't know if you guys know, but there's like a fascinating history associated with Wichita Lyman, right?
• 40:03
  SPEAKER_01: Which is one, Glenn Campbell had just had a hit that was about a place, I can't remember which one it was, right?
• 40:10
  SPEAKER_01: And Jimmy Page wrote that.
• 40:13
  SPEAKER_01: Don't mean Jimmy Page today, because he's Zeppelin.
• 40:15
  SPEAKER_01: Jimmy Webb.
• 40:16
  SPEAKER_01: There you go, Jimmy Webb.
• 40:17
  SPEAKER_01: Jimmy Webb.
• 40:18
  SPEAKER_01: Uh so Jimmy Webb had written that song.
• 40:21
  SPEAKER_01: Great songwriter, written that song, and so they needed another hit.
• 40:26
  SPEAKER_01: So Glenn Campbell literally wrote to him and said, Can you write me another song about a place?
• 40:32
  SPEAKER_01: And that was it.
• 40:34
  SPEAKER_02: That's all the guidance he gave, yeah.
• 40:35
  SPEAKER_01: That's all I got.
• 40:36
  SPEAKER_01: Write me another song about a place.
• 40:38
  SPEAKER_01: So Jimmy Webb, you know, knocks off the lyrics to Wichita Linemen, but he doesn't, he hasn't finished it.
• 40:47
  SPEAKER_01: Right?
• 40:48
  SPEAKER_01: If you actually look at Wichita Lineman, right, there's no chorus to speak of, right?
• 40:53
  SPEAKER_01: It's just basically a couple of really interesting verses.
• 40:58
  SPEAKER_01: And there's also some things that Jimmy Webb has to plans to fix.
• 41:06
  SPEAKER_01: Sends it off to Glenn Campbell, knows that it's not finished, assumes that Glenn Campbell realize it's not finished, because obviously it's not finished, doesn't have these various, you know, various elements that are supposed to be in there.
• 41:19
  SPEAKER_01: Glenn Campbell gets it, likes it, goes to the recording studio and records it, and then has to improvise some things because you know you don't you're missing a chorus and things like that.
• 41:35
  SPEAKER_01: So they go, I go, okay, well, we can put a musical bridge in there in order to be able to fix that.
• 41:41
  SPEAKER_01: And it doesn't really seem to start in a way that we would understand, so we'll just have a very simple, you know, guitar riff to to get us in.
• 41:50
  SPEAKER_01: Records it is done with the song.
• 41:53
  SPEAKER_01: Jimmy Webb comes back and says, Do you want me to finish that song I sent you?
• 42:01
  SPEAKER_01: For example, you know, I can throw a chorus in there, um, and for example, I can fix that terrible bit in the lyrics, which is uh, and I need you more than want you, and I'll want you for all time, because it doesn't rhyme, it's not like etc, etc.
• 42:22
  SPEAKER_01: And Glencuba's like, no, we're done, we're good, we've released it, etc.
• 42:27
  SPEAKER_01: And particularly ironically, people that love that song, their favorite rhyming, oh, their favorite part of the entire song lyrically is and I need you more than want you, and I want you for all time, is the bit that everybody remembers and loves.
• 42:42
  SPEAKER_01: That's the bit that Jimmy Webb wanted to fix.
• 42:45
  SPEAKER_01: One of the great songwriters, nah, it's broken, I'll fix that.
• 42:49
  SPEAKER_05: Yes.
• 42:51
  SPEAKER_01: And and so, of course, the the whole myth and the story about the whole thing comes from the fact that even the guy that created it didn't thought of it as broken, and the thing that he thought of as broken is one of the great songs of all time.
• 43:06
  SPEAKER_01: So even the humans don't know.
• 43:09
  SPEAKER_01: My I guess my broader question for you is that's a screw-up, right?
• 43:16
  SPEAKER_01: But it's a screw up in communication, it's a screw up from the perspective of other people, but it's a song that people will still remember in you know 20, 30, 40 years.
• 43:26
  SPEAKER_01: Is there an AI equivalent of that?
• 43:28
  SPEAKER_01: Right?
• 43:29
  SPEAKER_01: Is there gonna be something that AI creates because it creates it however it creates it, and and just at some point, almost randomly, to your point about monkeys and writers, Matt, does it get to get to a point where like some undeniable work of art is generated by AI precisely because we didn't try to control it too much?
• 43:58
  SPEAKER_03: Certainly seems powerful.
• 44:00
  SPEAKER_03: Possible that that we are going to be coexisting with these things.
• 44:07
  SPEAKER_03: Yeah.
• 44:07
  SPEAKER_03: And they're going to put things into the world.
• 44:09
  SPEAKER_03: I mean, you know, let's let's look at like a what if you asked AI to design a building that would be hurricane proof and it 3D prints that house on the coast and it's hurricane proof.
• 44:23
  SPEAKER_03: It's a beautiful piece of art, like we what like we look at architecture, you know, like beautiful architecture.
• 44:30
  SPEAKER_03: Well, I don't know, maybe it thinks of things because it knows more about the entirety of wind and ocean currents that you can't learn in architecture school, right?
• 44:39
  SPEAKER_03: And I f I feel like we're gonna coexist.
• 44:42
  SPEAKER_03: That's that's kind of how I feel, even from an artist's perspective, like I hope, or maybe it'll be worth it to continue to see was this AI generated or human generated, or is all of it human generated because we generated AI.
• 44:59
  SPEAKER_03: Right.
• 45:00
  SPEAKER_01: Yeah.
• 45:01
  SPEAKER_01: These are it's difficult philosophical questions.
• 45:04
  SPEAKER_03: Right.
• 45:05
  SPEAKER_03: It's of the earth.
• 45:06
  SPEAKER_03: And and we we did that.
• 45:07
  SPEAKER_03: Uh we made the silicone, we put it together, we created these thinking things, we programmed them.
• 45:13
  SPEAKER_03: Yeah, it's it it does get pretty deep pretty quick.
• 45:16
  SPEAKER_03: I think one thing I don't want to lose on the on the SOP thread is that we have a real opportunity to reimagine how we do SOPs in regulated industries because of this, right?
• 45:30
  SPEAKER_03: Because measuring risk, deciding like, do we care if this is a process that exactly follows this, if safety, uh, quality, money, time, whatever the whatever parameters you want to put on what you're trying to measure for or have an outcome for, those things we really have to embrace the possibility to get better at them.
• 45:53
  SPEAKER_03: Yeah.
• 45:53
  SPEAKER_03: Also, which is kind of exciting with this creativity that we're talking about, which is just tell it the outcome.
• 45:59
  SPEAKER_03: What would it do given, you know, I think of uh like when Bill was talking about the military piece, there's operating exercises and and SOPs and protocols in in the field, but you can't really have that same kind of freedom when it comes to building the airplane or the ship that got the people there, right?
• 46:16
  SPEAKER_03: So you have to follow that process, you have to measure things and set temperatures and the right tools, all of those things have to be locked down.
• 46:26
  SPEAKER_03: Um and let's ex take the example of building an airplane.
• 46:31
  SPEAKER_03: At some point, do no humans touch the airplane?
• 46:33
  SPEAKER_03: It's all assembled by robots.
• 46:36
  SPEAKER_03: Maybe they have to follow a protocol and make sure they do it right every time and check it on the way out, and everything gets checked every time.
• 46:45
  SPEAKER_03: So there's a place for that scrutiny, but we probably overdo it when it's just humans doing it, to the point where they, you know, get ignored, get too big too follow, you know.
• 47:02
  SPEAKER_03: We can go on and on about the SOPs in my industry, it's crazy, but some of them are absolutely necessary for the safety of the patients and for the product of the drug quality, etcetera.
• 47:14
  SPEAKER_01: So maybe that's your point.
• 47:16
  SPEAKER_01: Maybe that's your point that uh we have to know we have to know the difference.
• 47:23
  SPEAKER_01: So we have to know the difference between are we being are we being human-centric?
• 47:31
  SPEAKER_01: Are we assuming, are we making assumptions that what the human has come up with is essential really because a human came up with it, right?
• 47:40
  SPEAKER_01: Particularly if a bright human, like a really intelligent human came up with it, right?
• 47:44
  SPEAKER_01: Are we going like that is that sticks around because, you know, Jeff, who's won a Nobel Prize was responsible for it in some way, shape, or form, right?
• 47:59
  SPEAKER_01: That to me is the question.
• 48:01
  SPEAKER_01: And almost like when you said, you know, humans, well, humans created the stuff that AI is trained on, so therefore it's us.
• 48:10
  SPEAKER_01: I understand that, but that's also a very human-centric approach, right?
• 48:14
  SPEAKER_01: Because you could draw it all the way back and say, well, actually, you know, the the Big Bang is responsible for it, or you know, the moment that we went multicellular, that's responsible for it.
• 48:25
  SPEAKER_01: So like we we draw back to our place in history because we elevate ourselves.
• 48:30
  SPEAKER_01: I think about the Jeffrey Moore stuff, right?
• 48:32
  SPEAKER_01: As he's like going through the layers of the infinite staircase.
• 48:35
  SPEAKER_01: We're one layer of that.
• 48:36
  SPEAKER_03: Step two.
• 48:36
  SPEAKER_03: Yeah, yeah.
• 48:37
  SPEAKER_03: A miracle happens.
• 48:38
  SPEAKER_03: Yep.
• 48:40
  SPEAKER_03: Yeah.
• 48:40
  SPEAKER_01: But yeah, I mean, so I think the point is that somehow, some way, we gotta figure out when less is more, right?
• 48:49
  SPEAKER_01: Your example, Bill, of the song where you gave it less and you got more.
• 48:55
  unknown: Right.
• 48:56
  SPEAKER_01: Way more.
• 48:57
  SPEAKER_01: Debugging, right, when I'm giving it less context, resulting in more, right?
• 49:03
  SPEAKER_01: The across all of these things, maybe, Matt, you're not giving it the existing SOP at all, right?
• 49:13
  SPEAKER_01: And you're creating a constrained environment, and you're giving it deliberately, significantly less context in order to see what it comes up with when unconstrained.
• 49:27
  SPEAKER_01: And I think that that's part of the issue, right?
• 49:30
  SPEAKER_01: Our assumption, our working assumption, is that when we give it more context, we get better results.
• 49:37
  SPEAKER_01: I'm questioning that deeply now.
• 49:40
  SPEAKER_01: Yeah.
• 49:40
  SPEAKER_01: Right?
• 49:41
  SPEAKER_01: And and I'm starting to think about okay, when do I give it less context in order to generate something that is useful in a different way, versus just making saying, here's everything we know so far, just make it five percent better.
• 49:55
  SPEAKER_01: That's not that's us.
• 49:57
  SPEAKER_01: That's our thinking too small.
• 49:59
  SPEAKER_01: Yeah.
• 50:01
  SPEAKER_02: It's this equation or ratio, perhaps, is a better word, that I'm investigating between, like you said, constraint and restraint.
• 50:11
  SPEAKER_02: Like where do I restrain myself as opposed to focus on the constraints?
• 50:16
  SPEAKER_02: And you know, we could get even more specific about this, right?
• 50:19
  SPEAKER_02: The constraints are in the GPUs, the constraints are in the credits, the constraints are in the time, whatever.
• 50:25
  SPEAKER_02: But where's where do I show up and restrain my tendency?
• 50:29
  SPEAKER_02: Like in the case of this, do I restrain myself from trying to get it exact and perfect and and constantly course correcting it?
• 50:38
  SPEAKER_02: That feels like a restrain uh muscle to me because my tendency is if I have the power of all of this at my fingertips, then surely if I go surgical and tactical and laser focused, I'll get a better result.
• 50:51
  SPEAKER_02: But what this is telling us is maybe not right now.
• 50:57
  SPEAKER_01: I want to give us some homework.
• 51:01
  SPEAKER_01: And the homework I want to give us is I want us to find something anything, right?
• 51:07
  SPEAKER_01: It could be work-related in your case, Matt.
• 51:08
  SPEAKER_01: It could be, okay, I'm gonna give it an outcome, and I'm not gonna tell it, I'm gonna deliberately hold back information that I would normally give it.
• 51:18
  SPEAKER_01: And then completely separate from that, I'm gonna give it an outcome, and then give it every piece of information I could possibly give it.
• 51:27
  SPEAKER_01: Stick it on the most powerful model that we can stick it on and see what see what we learn, right?
• 51:33
  SPEAKER_01: Because this is the other thing as well, I think, which is that it's not always about like one of the results might be not not useful end to end, but it might give you a thing that you would never have got otherwise or you would never have thought of as a human being.
• 51:54
  SPEAKER_02: Which is to alignment.
• 51:56
  SPEAKER_01: Yeah.
• 51:56
  SPEAKER_01: Yeah.
• 51:57
  SPEAKER_01: Yeah.
• 51:58
  SPEAKER_01: So I think we should try that and then we should see what see what we learn and see what that would change about how we would interact with AI.
• 52:11
  SPEAKER_01: Yeah.
• 52:13
  SPEAKER_01: Gents, any final thoughts?
• 52:15
  SPEAKER_01: Yeah.
• 52:16
  SPEAKER_01: Yeah.
• 52:17
  SPEAKER_01: Any final thoughts on this, gents, before we wrap up?
• 52:21
  SPEAKER_02: I'm having fun.
• 52:22
  SPEAKER_02: That's all.
• 52:25
  SPEAKER_03: Yeah.
• 52:25
  SPEAKER_03: It's a lot to think about.
• 52:26
  SPEAKER_03: I'm really trying to bring together the possibility alignment and be somewhat of the voice of reason inside organizations as well, in terms of this is a really important consideration, even in the face of all of the hype around what it could do for us.
• 52:49
  SPEAKER_03: Right?
• 52:49
  SPEAKER_03: And I think you really have to qualify and think deeply on your thoughts about what you're gonna do to maybe leverage it as effective as we possibly can without getting crazy, but also exploring possibilities that I, you know, again, there's been three things I thought of while we're talking here that I didn't think about before, right?
• 53:13
  SPEAKER_03: So um I can't necessarily say that I've had that same type of interaction with an AI model yet.
• 53:25
  SPEAKER_01: Yeah.
• 53:27
  SPEAKER_03: Not to this level anyway.
• 53:28
  SPEAKER_03: It might find things that I didn't find or remember things I didn't remember as well, but uniquely spark new ideas.
• 53:38
  SPEAKER_03: Hmm.
• 53:39
  SPEAKER_03: I don't know.
• 53:40
  SPEAKER_01: Maybe I haven't played enough yet.
• 53:42
  SPEAKER_01: Maybe we've been maybe we've been constraining it too much to give us that insight.
• 53:50
  SPEAKER_01: Maybe.
• 53:50
  SPEAKER_01: Yeah, I don't know.
• 53:51
  SPEAKER_01: Yeah.
• 53:52
  SPEAKER_01: I guess we'll figure out, won't we?
• 53:53
  unknown: All right.
• 53:54
  SPEAKER_01: All right, guys.
• 53:54
  SPEAKER_01: We'll uh thanks everyone.
• 53:56
  SPEAKER_01: Call it good for today.
• 53:57
  SPEAKER_01: Hope everyone enjoyed this one, and we'll see you soon.
• 54:00
  SPEAKER_01: Thanks.
• 54:01
  SPEAKER_01: Bye.
• 54:01
  SPEAKER_01: Bye.
• 54:04
  SPEAKER_00: Humanity Working is produced by Matt Neal.
• 54:07
  SPEAKER_00: If you liked this episode, please take a moment to rate and review us on your favorite podcast platform.
• 54:13
  SPEAKER_00: It makes a huge difference in helping others find us.
• 54:17
  SPEAKER_00: And don't forget, you can find all episodes and get our weekly newsletter at humanityworking.
• 54:24
  SPEAKER_00: See you next time.