Latest / aiEDU Studios / The Jagged Edge: Why Working With AI Is the New Human Skill

Transcript

The Jagged Edge: Why Working With AI Is the New Human Skill · aiEDU Studios

• 0:00
  SPEAKER_00: It's not about how smart an AI is.
• 0:04
  SPEAKER_00: It's not how smart humans are.
• 0:05
  SPEAKER_00: It's all about how well we as the humans can navigate that jagged edge.
• 0:10
  SPEAKER_00: Because the thing with the jagged edge is that it moves all the time.
• 0:14
  SPEAKER_00: I think that's the new human skill that we're gonna all have to get good at.
• 0:21
  SPEAKER_01: We're here in the uh the virtual studio with Wim Sweldens with Kiswi.
• 0:25
  SPEAKER_00: It's good to be back.
• 0:26
  SPEAKER_00: We've met before, we've spoken before.
• 0:28
  SPEAKER_00: I always love catching up with you.
• 0:29
  SPEAKER_00: So, two words about myself.
• 0:31
  SPEAKER_00: I'm sort of a technologist and innovator and business leader, working on a variety of different technologies.
• 0:38
  SPEAKER_00: We started my career at Bell Labs.
• 0:40
  SPEAKER_00: I'm currently working on a company that I co-founded called KISWE.
• 0:43
  SPEAKER_00: We're an interactive live streaming, distribution, production kind of capabilities using quite a bit of AI.
• 0:49
  SPEAKER_00: And I have been, as you know, musing a little bit about the use of AI and their concept of AGI, and particularly something that I like to call AIQ, which is how good are humans really working with AI, which I think is where the conversation needs to move for a moment.
• 1:05
  SPEAKER_01: And so you worked in the um in the telecommunications space.
• 1:09
  SPEAKER_01: If you had to try to find the closest analogue to the disruption the disruptive moment that AI sort of feels like today, would it would you point to is it the iPhone?
• 1:19
  SPEAKER_01: Is it the is it the internet itself?
• 1:21
  SPEAKER_00: I mean like w what's I I think sort of the I mean I'm a little older, so I've I've seen a few more things in in my life.
• 1:32
  SPEAKER_00: What what this period in technology makes me think of is sort of like the year ninety ninety-one or something like that, when sort of the browser came out and gopher came out and HTTP came out, and we're like, what is this?
• 1:48
  SPEAKER_00: You know, what is this?
• 1:50
  SPEAKER_00: And it sort of feels like that.
• 1:52
  SPEAKER_00: I mean, I've been in technology for like 40 years now or something.
• 1:56
  SPEAKER_00: iPhones, mobile networks, none, none of these things are the same caliber than than than this change we're going through.
• 2:06
  SPEAKER_00: And some have argued with me and might have a point that it might even be bigger than just the internet by itself.
• 2:13
  SPEAKER_00: So I remember I had a colleague around that time, or it must have been 92-93, who told me that everything that's not on the internet doesn't exist.
• 2:22
  SPEAKER_00: And that was sort of a controversial statement in '93, let's say.
• 2:26
  SPEAKER_00: And it sort of feels like that too.
• 2:28
  SPEAKER_00: Everything that's not an AI enabled or works with it or takes advantage of it is there.
• 2:37
  SPEAKER_01: You know, there's a lot of AI conferences now, you know, with like AI plus this.
• 2:40
  SPEAKER_01: What would be the moment where we stop talking about AI as this discrete thing and it just becomes just technology, right?
• 2:47
  SPEAKER_01: It's just like, I mean, I don't think there's like an internet conference.
• 2:49
  SPEAKER_01: I can't think of, you know, it'd be very silly to have a panel about the internet, as it were.
• 2:54
  SPEAKER_00: Yeah, yeah, yeah.
• 2:55
  SPEAKER_00: No, I I I've been thinking about that as well because so many things are branded AI this and AI that.
• 3:00
  SPEAKER_00: And and and even with some of the work we're doing in Kisby, I don't really want to call it our AI product anymore because AI should be everywhere in some sense.
• 3:10
  SPEAKER_00: So in in my mind, I've already sort of reached that point that I personally don't want to do that anymore.
• 3:15
  SPEAKER_00: But I think the world as a whole is not quite there yet.
• 3:19
  SPEAKER_00: But I think we'll see rapid adoption of AI, and therefore demoniker AI will sort of start to see.
• 3:25
  SPEAKER_00: Yeah, I mean it's which is I think a good thing.
• 3:27
  SPEAKER_01: A good thing, I guess, as long as as long as folks are brought around to it.
• 3:31
  SPEAKER_01: I mean, I think the and and I do want to come to come to your piece, which really was for me extremely formative.
• 3:37
  SPEAKER_01: Um because you you sort of pointed me away from this idea of you know thinking about AI as a sort of like technology that's implemented to you're sort of refocusing the lens on on the individuals within the an organization.
• 3:52
  SPEAKER_01: And and in your paper, you you you sort of deemed this AI Q.
• 3:56
  SPEAKER_01: The your is this is the artificial intelligence uh quotient?
• 4:01
  SPEAKER_01: Um yeah, can you just give sort of like a synopsis of what you wrote, but also maybe like what what led you to that insight?
• 4:07
  SPEAKER_01: I mean, was it was it personal experience at your business or or Yeah?
• 4:10
  SPEAKER_00: Okay, so a couple thoughts.
• 4:11
  SPEAKER_00: It was definitely sort of my personal experience using AI and working in it and working on sort of predecessors of AI years ago.
• 4:19
  SPEAKER_00: And it sort of came a little bit from two things.
• 4:22
  SPEAKER_00: First of all, my personal annoyance with the whole quest for AGI, which I still believe is silly.
• 4:28
  SPEAKER_00: I think that's a silly quest in my mind.
• 4:31
  SPEAKER_00: But then at the same time, the thing that really triggered me is this article that I read about the jagged edge.
• 4:36
  SPEAKER_00: Um, produced by, I think it was a uh yes, yes.
• 4:42
  SPEAKER_00: And and I'm like, yes, because I'd sort of been thinking about that as well.
• 4:45
  SPEAKER_00: The jagged edge is really this thing that that AI is just so good at some things, it just blows our mind how good it is.
• 4:52
  SPEAKER_00: And then you have a slightly different task that feels almost the same, where it's just so terrible at, and it makes the silliest and dumbest mistakes.
• 5:01
  SPEAKER_00: And and those two things are very close to each other.
• 5:03
  SPEAKER_00: So that's why, as a mathematician, I think it has a very high derivative.
• 5:07
  SPEAKER_00: I make a small change and the result is dramatically different.
• 5:10
  SPEAKER_00: And I started feeling the jagged edge myself.
• 5:14
  SPEAKER_00: I'm like, this works really well.
• 5:15
  SPEAKER_00: And then some of my colleagues would be using it as like, this is the stupidest thing ever.
• 5:19
  SPEAKER_00: This doesn't work at all.
• 5:20
  SPEAKER_00: And I'm like, what are you talking about?
• 5:22
  SPEAKER_00: And then I realized we were doing very similar things, but just a little different.
• 5:25
  SPEAKER_00: And I'm like, oh yeah, but it's hallucinating.
• 5:27
  SPEAKER_00: Here's the way to fix that, or it's being sycophantic, it's like, okay, here's the way to fix that.
• 5:31
  SPEAKER_00: And that sort of, I don't know why how it popped in my head one day is like, it's not about how smart an AI is, because AIs are clearly very smart and they quick quote unquote know everything.
• 5:42
  SPEAKER_00: It's not how smart humans are.
• 5:43
  SPEAKER_00: We've defined that a long time ago, and that has a lot of issues on its own right.
• 5:47
  SPEAKER_00: It's all about how well we as the humans can navigate that jagged edge.
• 5:52
  SPEAKER_00: Because the the thing with the jagged edge is that it moves all the time.
• 5:56
  SPEAKER_00: It moves, AI becomes more capable all the time.
• 5:58
  SPEAKER_00: So the edge moves in in the direction of AI could do more.
• 6:03
  SPEAKER_00: Great.
• 6:03
  SPEAKER_00: But it actually almost feels, and this is more of a feeling, that it becomes even more jagged.
• 6:08
  SPEAKER_00: As it becomes really, really good at certain things, it also sort of almost might regress in other things and becomes worse at those.
• 6:15
  SPEAKER_00: So our job to navigate and sort of tiptoe around it becomes only harder, and it becomes harder every day in a way.
• 6:23
  SPEAKER_00: And I think that's gonna be the new key skills that I know you're a lot into education, and what do we tell our students and all ages?
• 6:29
  SPEAKER_00: I think that's the new human skill that we're gonna all have to get good at.
• 6:33
  SPEAKER_00: So you're so your your statement is as AI gets better, the the peaks, as it were, get higher, but then that also means that the troths get more that the the the the uh intensity of those those changes are yeah, the troughs get get deeper in some sense, or or the the delta between the high and the low becomes higher, and and the difference between them might even become smaller.
• 6:56
  SPEAKER_00: As AI advances, it might become it might become more hard, it might become better at hallucinating, it might become better at being a sycophant, which is obviously one of the key components of AIQ is how well do you manage to detect that and compensate that behavior?
• 7:13
  SPEAKER_00: Because it's a big part of the job.
• 7:14
  SPEAKER_01: Because I think that there's I think a hope amongst some that in this period, you know, the AI is not it's still it's still improving and and and being you know maybe even quite nascent.
• 7:26
  SPEAKER_01: And you know, I think the hope might be that over time it we solve hallucinations or we solve some of the like it smooths out.
• 7:33
  SPEAKER_01: I haven't experienced that.
• 7:34
  SPEAKER_01: I think I I think Yeah I I think you're right.
• 7:38
  SPEAKER_01: It's it's uh the I can't think of what it's actually worse at, but I can think of the because it's getting really good at certain things, that also increases the leverage of how bad it the outcome could be.
• 7:52
  SPEAKER_00: Exactly.
• 7:53
  SPEAKER_00: You expect it to be really good at everything else, and it's and it's not in some sense, you know.
• 7:58
  SPEAKER_00: So yeah, I think the jagged edge and AIQ are not gonna go away.
• 8:01
  SPEAKER_00: It's not gonna go away.
• 8:03
  SPEAKER_00: And it's every day we are confronted with it.
• 8:05
  SPEAKER_00: It also has to do with obviously there's an enormous change management in everything we do, in every discipline.
• 8:11
  SPEAKER_00: In education, in every discipline of work, there's an impact.
• 8:15
  SPEAKER_00: And like any change management, there's people who like to be the Mavens and the early adopters, and I like to think myself amongst them.
• 8:23
  SPEAKER_00: There's people who sort of wait for the middle group to move, and there's people gonna be whole less like, oh, this is a bad idea, we should not use any of this because it's dangerous.
• 8:31
  SPEAKER_00: And they're not necessarily wrong because there's quite a few dangers and there's quite a few pitfalls.
• 8:36
  SPEAKER_00: And so we're all sort of going through it, and I think AIQ and and the definition and the awareness of the jagged edge will help us manage that path, which is gonna be a complicated path.
• 8:48
  SPEAKER_00: It's not gonna be a smooth path.
• 8:49
  SPEAKER_01: Not just sort of um buttering you up.
• 8:51
  SPEAKER_01: Like we even in the way that we talk to schools, we've started to use the language of change management, and it it's actually it's actually clicked for folks in a way that you know, talking about AI readiness or AI literacy, I think again, like you when you attach the word AI to it, people really get stuck on the technology part.
• 9:10
  SPEAKER_01: Like, oh, they think they think we're talking about AI.
• 9:12
  SPEAKER_01: When you talk about change management, it puts it into a frame that sounds familiar and that also I think people intuitively understand that change management is a human endeavor, and you can have change management around technology, around different processes.
• 9:27
  SPEAKER_01: You have five uh uh uh markers of of AIQ.
• 9:32
  SPEAKER_01: You talk about prompting, context awareness, anticipating failures, which I want to get to, and then verification and ethics and workflow integration.
• 9:40
  SPEAKER_01: Anticipating failures is interesting because you talked about the people who are really uh either afraid or or concerned about AI, and you said that's not actually necessarily a bad thing.
• 9:49
  SPEAKER_01: I mean, is is that because you actually need people within your organization who are sort of like pushing uh and uh uh uh almost almost like scoping out those those those potential risks so that you actually is that is that about mapping the that edge?
• 10:04
  SPEAKER_00: It's yeah, it's it's and I sort of experience that every day a little bit because I not just lead the AI front at work, but I also am involved with a lot of the AI change managers within our company, and you you get different reactions to it, and it somehow creates a little bit of tension.
• 10:20
  SPEAKER_00: Is it for people who are like, okay, they're AI advocates, and I'm sort of seen as one of them.
• 10:24
  SPEAKER_00: They're the AI holdouts, and it creates tension.
• 10:27
  SPEAKER_00: But in my mind, that is good tension.
• 10:28
  SPEAKER_00: That is the kind of tension you want in some sense, because it's just like you know, when you build a bridge, you gotta have tension in all the pieces of your bridge, otherwise your bridge is not gonna hold up.
• 10:37
  SPEAKER_00: You know, that's the whole point of civil engineering.
• 10:40
  SPEAKER_00: So I think it has to be healthy tension, it has to be tension that is in the open that you could talk about.
• 10:45
  SPEAKER_00: It can't be sort of hidden under the table or sort of passive aggressive behavior, that's sort of very, very non-productive.
• 10:51
  SPEAKER_00: But it's kind of AIQ gives you a language to talk about this tension.
• 10:56
  SPEAKER_00: You know, it's like, well, are we good at prompting?
• 10:58
  SPEAKER_00: It's like, well, we could get better, but are we good at understanding the jagged edge?
• 11:03
  SPEAKER_00: Are we good at detecting the pitfalls of it?
• 11:05
  SPEAKER_00: I mean, uh, because we might be great at writing prompts and we might get results that are nonsense because we are not good at detecting when it's hallucinating or when it's uh being sycophantic.
• 11:15
  SPEAKER_00: So in my mind, it's just it's a language that allows you to deal with change management, and there's people much better experts in change management and also the human aspects of it, you know, than than I am.
• 11:26
  SPEAKER_00: And this kind of both the the language and the change management, like you said, people recognize change management, they're familiar with it, can work together to sort of move us all in the right direction in my mind.
• 11:38
  SPEAKER_01: Wait, when you look at the And you have to you don't have to have a specific individual in mind, but when when you think about individuals at your business that either you've seen as moving faster than you might have expected, or or maybe slower than you might have expected, are there are there certain personas that have kind of uh uh surfaced any any any sort of common threads in trying to identify like who are those people?
• 12:02
  SPEAKER_01: Like is it is it just technologists?
• 12:06
  SPEAKER_00: Yeah, it's it's a I mean it might be a bit anecdotal, but but I think just reflecting on it a little bit.
• 12:13
  SPEAKER_00: There's offer obviously, I think being an early adopter of anything is a little bit of a personality trait.
• 12:18
  SPEAKER_00: Some people just generally like to buy new toys and new gadgets and things like that, and other people do not.
• 12:23
  SPEAKER_00: So there's a little bit of a personality.
• 12:25
  SPEAKER_00: So that's one part of it.
• 12:26
  SPEAKER_00: But I also obviously we do a lot of uh software development and software engineering.
• 12:30
  SPEAKER_00: We build large streaming systems used by millions of people and and that sort of needs to be super reliable and work globally and so on.
• 12:38
  SPEAKER_00: And so we've seen a couple things that, for example, some of our uh product managers, there are a product manager is typically somebody who will, I mean, uh set out a roadmap and define priorities and define sprints, maybe be a product order.
• 12:53
  SPEAKER_00: We've seen some of them sort of literally becoming prompt engineers, and in some sense, in the way in the old software technology telecom world, we used to call system engineers, people who write requirements.
• 13:08
  SPEAKER_00: I mean, uh back in the days where we used to build huge software systems for mobile networks and so on, we had product engineers who define priorities, we had system engineers, typically there were former developers who write very detailed requirements.
• 13:21
  SPEAKER_00: You know, if you see the documents that describe how a 5G network works, it's like an enormous uh uh document.
• 13:29
  SPEAKER_00: And then you have developers, people who then take those requirements and write code.
• 13:34
  SPEAKER_00: And those jobs are typically would then, as I'm sure you've seen as well over the last 20 years, would get often moved to different countries or low-cost regions and things like that, and they would become sort of more prescriptive type jobs.
• 13:47
  SPEAKER_00: What is happening now is that I think the key skill is that what we used to call the system engineering.
• 13:53
  SPEAKER_00: You can now write, say that's prompt engineering, or you can write to say the specification requirements.
• 13:58
  SPEAKER_00: Sort of in some sense, the source of truth of your system might not just be those system engineering documents, or might not just be the code, it might be your specifications.
• 14:09
  SPEAKER_00: That might be your your source of truth.
• 14:11
  SPEAKER_00: And that's sort of the key writing specifications and writing them well is going to become the key measurements of AIQ.
• 14:19
  SPEAKER_00: And we've seen now engineers, sort of people who used to write code, sort of move up the food chain and become our specification engineers in some sense.
• 14:28
  SPEAKER_00: And they so now they write code in English, think of it that way.
• 14:32
  SPEAKER_00: But we've also seen people who sort of typically are more on the product management or project management side, sort of come down and say, like, hey, wait a minute.
• 14:40
  SPEAKER_00: Instead of telling the developers what to do, I'm just gonna write the requirements or or the specifications and I'm gonna have an agent do it.
• 14:48
  SPEAKER_00: Now, there is danger in that too, because somebody who's who's not trained as a software engineer is not gonna have software engineering reflexes.
• 14:57
  SPEAKER_00: It's like, okay, I built this thing and it looks beautiful and it seems to work.
• 15:01
  SPEAKER_00: But is it secure?
• 15:03
  SPEAKER_00: Does it scale?
• 15:04
  SPEAKER_00: Is it robust?
• 15:05
  SPEAKER_00: I mean, uh, is there a risk of prompt injection or any number of things that can go wrong with it that somebody from an engineering background is gonna more have that mindset?
• 15:14
  SPEAKER_00: And we had a workshop in the office here in Belgium two weeks ago where I mean, everybody in the office were just eight of us, uh, got together and we started sharing our personal sort of journeys down the rabbit hole or not with different agents and so on.
• 15:29
  SPEAKER_00: And it was some some business people, some mixed people like myself, some coders, some product managers, a little bit of some operations people.
• 15:37
  SPEAKER_00: And we really came to to sort of a common understanding, just like, hey, there is a whole new way we can all start working when we share these things and we can really build this kind of specification layer of our of our system.
• 15:51
  SPEAKER_00: And so we started doing two things.
• 15:54
  SPEAKER_00: For some of our old systems, uh, we started a project which is still ongoing, so I can't say the results yet because we don't know, is where we say, we have a ton of code we wrote over the last five years.
• 16:04
  SPEAKER_00: Can we sort of reverse compile it into a number of specifications for which we can rebuild that code?
• 16:10
  SPEAKER_00: We can let's sort of throw the old code away.
• 16:12
  SPEAKER_00: Now that's a little scary, and we're trying it.
• 16:15
  SPEAKER_00: I'm not saying it's gonna work, but it's something that I think is worthwhile.
• 16:18
  SPEAKER_00: And then for things where we're building new stuff, where we're sort of starting from scratch on a new idea and new technology, we're like, yeah, we're just gonna write really good uh specifications, and that's gonna be the source of the truth, and then yeah, we'll throw our code away on a regular basis, which for an engineer that's a shocking thing.
• 16:37
  SPEAKER_00: We're gonna take code and throw it away.
• 16:39
  SPEAKER_00: Because I've written code as well, and code is like, I mean, it is it used to be like a sculpture, you know.
• 16:47
  SPEAKER_00: You start by figuring out the large framework and then you chisel at it month after month, and you build this piece of code, and it's it's a piece of art.
• 16:55
  SPEAKER_00: You take a lot of pride in it, and now somebody comes along and says, Hey, we're gonna throw your code away, and we're gonna start all over again.
• 17:00
  SPEAKER_00: Are you effort kidding me?
• 17:01
  SPEAKER_00: But that's literally what we're starting to do now.
• 17:04
  SPEAKER_00: So, so it is it is really a dramatic change in workflow.
• 17:07
  SPEAKER_00: And I think it was good we had an in-person workshop with the eight of us to just literally sync up as to the various directions we'd gone into.
• 17:15
  SPEAKER_01: On on the face of it, if someone were to describe like what like a technical writer were to do, you would think, well, surely that's the type of job that AI is going to replace.
• 17:23
  SPEAKER_01: I mean, people just that's it and write technical specifications about stuff.
• 17:27
  SPEAKER_01: It seems like the AI would be really good at that.
• 17:29
  SPEAKER_01: And yet I can also now see what you're saying, which is the ability to write with precision.
• 17:34
  SPEAKER_01: You know, I think prompt engineer is a misnomer because people think of prompts as sort of these very short, you know, pithy things that you're asking for.
• 17:40
  SPEAKER_01: But but really a prompt is more of like being able to articulate um like yeah, very precisely.
• 17:46
  SPEAKER_01: What do you want?
• 17:47
  SPEAKER_01: What what do you actually want to happen?
• 17:48
  SPEAKER_01: And that is um so so you actually believe then that this is that that like sort of like uh um let's say English or or or Dutch or whatever that the the language that you're using, is is it sort of are you seeing it actually replace specific coding languages in syntax as the dominant way that engineers are in creating and writing stuff?
• 18:08
  SPEAKER_00: Oh yeah.
• 18:10
  SPEAKER_00: Yeah, yeah.
• 18:10
  SPEAKER_00: I think that for for things that are new where we're starting from scratch, I mean, like you're never gonna start by writing code anymore.
• 18:17
  SPEAKER_00: I mean, and maybe a better word than prompt engineering is specification engineering in some sense.
• 18:22
  SPEAKER_00: Because you're right, people think of a prompt as a little message I I send in a chat bot.
• 18:26
  SPEAKER_00: That's not what what this is.
• 18:28
  SPEAKER_00: This is documents of pages and pages and pages that in English describe what the system will do in great detail.
• 18:36
  SPEAKER_00: And so that is gonna become a key key skill.
• 18:39
  SPEAKER_00: And there's still a debate a little bit that I have with with colleagues in other companies as well, is that is the source of truth going to be the specification document, which is like your MD file, which is mostly English, or is it is the code still the source of truth?
• 18:55
  SPEAKER_00: I mean, and then if you need a human specification, you just take the code and you will first compile it into an MD file.
• 19:02
  SPEAKER_00: I mean, or you ask it questions.
• 19:04
  SPEAKER_00: That's still there's still a lot of debate.
• 19:07
  SPEAKER_00: I think we're now within our company more going this the specification route, that the specification is the source of truth.
• 19:14
  SPEAKER_00: But I know colleagues at other companies uh uh are saying, no, no, the code is still the source of truth, but no human will ever look at it anymore.
• 19:22
  SPEAKER_00: But that and then the question is how do you do version control and and things like that?
• 19:26
  SPEAKER_00: That's a whole thing.
• 19:28
  SPEAKER_00: There's a whole whole GitHub now working on specifications and things like that.
• 19:32
  SPEAKER_00: So there are definitely projects for which DMD file is the programming language, I think that.
• 19:38
  SPEAKER_00: And it's weird because it's not deterministic.
• 19:41
  SPEAKER_00: You know, it's like you know, back in the days when compilers came out, if you had a compiler that was non-deterministic, you were dead.
• 19:47
  SPEAKER_00: Nobody would ever use it, you know.
• 19:49
  SPEAKER_00: But now we're like, yeah, yeah, it's non-deterministic.
• 19:52
  SPEAKER_00: But you know, as long as we write it tight enough, it's okay.
• 19:55
  SPEAKER_01: Because you could have the same and an MD file is a markdown file, it's essentially I mean, how would you describe it?
• 20:00
  SPEAKER_01: Effectively, it's sort of like a just like a like a text document, but with a little bit more.
• 20:04
  SPEAKER_01: It's just a structured document.
• 20:05
  SPEAKER_00: It's a structured English document with some tabulation and it's not a problem.
• 20:10
  SPEAKER_00: You're looking at like a word document, effectively.
• 20:12
  SPEAKER_01: And if you were to take the same markdown file and drop it into, let's say, three separate AI like language models, presumably you're not necessarily going to get identical code outputs each time.
• 20:26
  SPEAKER_00: For sure not.
• 20:26
  SPEAKER_00: You will give for sure it will not be identical.
• 20:28
  SPEAKER_00: Will it do the same thing?
• 20:30
  SPEAKER_00: Hopefully.
• 20:32
  SPEAKER_00: You know, I mean, and actually it that would be a good test because let's say you you write the specifications, you give it to three AIs, they will write different codes, obviously.
• 20:42
  SPEAKER_00: But is the code going to do the same thing?
• 20:44
  SPEAKER_00: And if it does the same thing, that means you've done a good job.
• 20:46
  SPEAKER_00: Because the other thing that obviously is happening is like, what happens with QA?
• 20:50
  SPEAKER_00: How are we going to test all this?
• 20:52
  SPEAKER_00: How are we going to have agents to test this?
• 20:54
  SPEAKER_00: This is actually a discussion we had this morning.
• 20:56
  SPEAKER_00: But we had written a code, we had written a test agent, it passed all the tests.
• 21:00
  SPEAKER_00: Well, great.
• 21:01
  SPEAKER_00: But the test agent had cheated because the test agent had read the code and knew what the code was going to do and sort of knew what the answer was going to be.
• 21:10
  SPEAKER_00: And I'm like, well, that's a terrible test agent, you know.
• 21:13
  SPEAKER_00: So we realized, like, whoa, whoa, whoa.
• 21:15
  SPEAKER_00: We need to write a coding agent who uses the specification and generates code.
• 21:20
  SPEAKER_00: Then we need to have a testing agent who has a completely different context.
• 21:24
  SPEAKER_00: It cannot look at the code.
• 21:25
  SPEAKER_00: That's the whole point.
• 21:26
  SPEAKER_00: It cannot look at the code.
• 21:27
  SPEAKER_00: It cannot look at the code, it can look at the same specifications, it can write the test cases and execute the test cases.
• 21:33
  SPEAKER_00: And then the two of them can sort of go back and forth.
• 21:36
  SPEAKER_00: And that was sort of an insight that, like, ah, yes, we can definitely do testing much better.
• 21:40
  SPEAKER_00: Just like you wouldn't want an engineer to test their own code.
• 21:43
  SPEAKER_00: That wouldn't make any sense.
• 21:43
  SPEAKER_00: Because you're going to be able to do that.
• 21:45
  SPEAKER_01: And I because I have your sub stack up.
• 21:47
  SPEAKER_01: And so we talked, so you talked about prompting, context awareness.
• 21:50
  SPEAKER_01: I mean, that's I think that's part of that like that specification document.
• 21:53
  SPEAKER_01: Anticipating failures, this is it's almost like this is the the new modality of the the process.
• 22:00
  SPEAKER_01: Problem solving and the engineering, as it were, that happens, rather than it's literally engineering the in the form of writing the code, the engineering you're describing is is is having an understanding of what the potential outcomes, the probabilistic like the space, the probabilistic space of outcomes that might happen, trying to anticipate that.
• 22:23
  SPEAKER_01: And then you say verification ethics.
• 22:25
  SPEAKER_01: And so when you're talking about verification, it's I think people assume that verification just means reading the output and making sure it's right, but you're actually describing an understanding of even how do you verify.
• 22:36
  SPEAKER_01: And then the workflow integration is like well, if you were going to now like you're basically talking about uh a new structure for the process of engineering, and so how do you design a new workflow so that if you have a QA engineer, well, well, what is their role actually?
• 22:54
  SPEAKER_01: Because it's as yeah, so you're saying they're not necessarily reading the code, they're perhaps designing the agents that are reading the code and thinking about how do you how do you almost blind the study, as it were.
• 23:07
  SPEAKER_00: Yeah, so okay, so a couple thoughts on that.
• 23:10
  SPEAKER_00: So so the the there's different agents involved in a piece of software.
• 23:13
  SPEAKER_00: There's obviously the one who writes the code, then you can have somebody to review the code, just like in in the in the old days, you had you had a developer, you had somebody else who reviewed the code.
• 23:22
  SPEAKER_00: You could have another agent review the code, like hey, your code is is wrong or has mistakes and so on.
• 23:27
  SPEAKER_00: That's not the testing agent, that's the code review agent.
• 23:30
  SPEAKER_00: Then you have a testing agent who doesn't get to see the code.
• 23:33
  SPEAKER_00: You don't get to see the code.
• 23:34
  SPEAKER_00: You get to run the code and you get to write test cases.
• 23:37
  SPEAKER_00: You know, if I put this as input, then this should be the output.
• 23:40
  SPEAKER_00: I mean, or if I click on this button, then that should happen.
• 23:43
  SPEAKER_00: I mean, that's what a test agent is gonna do.
• 23:45
  SPEAKER_00: Now, a test agent will do some simple unit testing, just like we do on each individual block.
• 23:51
  SPEAKER_00: It's gonna do some system testing.
• 23:52
  SPEAKER_00: Now, in order to do system testing, it's gonna need to have an agent who can log in and who can use a browser, who can use an app and do all those type of things.
• 23:59
  SPEAKER_00: So that will be the sort of uh user testing agent in some sense, just like you would hire a bunch of uh uh users to try out all the buttons on your app.
• 24:09
  SPEAKER_00: You know, now you have a bunch of agents doing that.
• 24:11
  SPEAKER_00: So there's probably three or four of those agents uh involved in this.
• 24:15
  SPEAKER_00: And then the other point, you you mentioned sort of the the third part of AIQ, the defaults and the verification.
• 24:21
  SPEAKER_00: So the faults is sort of like we were talking earlier of the whole troughs of the of the AI.
• 24:26
  SPEAKER_00: You know, now you're in the wrong corner of the jagged edge and it's done something silly and it's done something stupid, and and you're ready to sort of throw it out.
• 24:33
  SPEAKER_00: That's where the verification part comes in, which is related to the different agents having different contexts.
• 24:40
  SPEAKER_00: And I'll give you one example that actually happened the day I moved to Belgium, I think it was January 10, if I remember right.
• 24:47
  SPEAKER_00: There was a huge controversy in the news, and I just literally, quote unquote, got off the boat or got off the plane because one of the presidents of a university had gotten caught in a speech she gave at the opening of the academic year using a quote, I think it was an Einstein quote, that was hallucinated.
• 25:05
  SPEAKER_00: And the entire country was ready to nail her to the cross and vilify her.
• 25:11
  SPEAKER_00: Like, how dare she use AI for writing her remarks?
• 25:16
  SPEAKER_00: And of course, the presidents of the other universities got on TV and chimed in how preposterous this was.
• 25:22
  SPEAKER_00: And actually, I it prompted me to write another substack because I was kind of annoyed by it.
• 25:26
  SPEAKER_00: And I'm like, wait a minute.
• 25:27
  SPEAKER_00: No, no, no, no, no, you guys are missing the point.
• 25:29
  SPEAKER_00: Clearly, she made a mistake putting in these hallucinated quotes, obviously.
• 25:34
  SPEAKER_00: The mistake was not using AI.
• 25:37
  SPEAKER_00: That was not a mistake.
• 25:39
  SPEAKER_00: Actually, I have much more concerns with all the other presidents of the other universities who don't use AI at all.
• 25:44
  SPEAKER_00: I'm much more worried about these people.
• 25:46
  SPEAKER_00: Her mistake was that she didn't do verification properly.
• 25:49
  SPEAKER_00: She didn't have another agent that says, look at my speech, look at this quote, double check that you can find an authentic source where this person actually said that.
• 25:57
  SPEAKER_00: I mean, uh, and then she also didn't disclose that she used AI.
• 26:00
  SPEAKER_00: I think if you use AI, you should say, yes, I used AI.
• 26:03
  SPEAKER_00: Of course I did.
• 26:04
  SPEAKER_00: I mean, like, you didn't?
• 26:05
  SPEAKER_01: So I think that's what brings you to this AIQ.
• 26:09
  SPEAKER_01: You know, when you're talking about workflow integration, I mean, we're seeing this at our organization, um, and we're also seeing it in schools where you know a lot of teachers are are agonizing over, you know, how do we talk how do we deal with like academic integrity and students using AI to cheat?
• 26:23
  SPEAKER_01: And there was one, there was I was talking to one school, and one of the things that they've done, which very few schools have done, which is before they go into the process of like articulating here are the rules for how you're allowed to use AI or how you're not allowed to use AI, the teachers first go to the students and say, Okay, I want you to talk to me about what your expectations are about how I use AI.
• 26:47
  SPEAKER_01: And I've heard about it.
• 26:50
  SPEAKER_01: Apparently, this was done.
• 26:50
  SPEAKER_01: I've heard about this on enough classrooms that I think that I feel confident in the outcome.
• 26:55
  SPEAKER_01: Most students are not going to say, Oh, well, you just use as much of AI as you want.
• 26:59
  SPEAKER_01: We don't really care if all of your lessons are AI generated or if your feedback is AI generated, the students will say, Well, no, you're our teacher.
• 27:06
  SPEAKER_01: The whole point is that you're supposed to be, you know, really thinking about what you're teaching us.
• 27:10
  SPEAKER_01: And we want, if you're going to give us feedback, we want it to be authentic feedback.
• 27:15
  SPEAKER_01: And and that, you know, that's sort of often the first the first reaction of the teacher will say, Well, would you rather me sit down by myself in my office, sitting down and writing feedback hand in hand, and then just give you that?
• 27:26
  SPEAKER_01: Or would you rather I use the AI maybe to give the initial passive feedback and then use that time to sit down with you in person and talk through the feedback?
• 27:34
  SPEAKER_01: And the students will usually say, Oh, no, no, I'd rather have the chance to talk to you and hear the feedback live.
• 27:39
  SPEAKER_01: But I mean, to me, that's actually it's it's it shifts away from this idea of addressing you know academic integrity as a policy challenge, which is like as if there's some magical rule that if you write down the rule in the right way, suddenly it's gonna just make everything work.
• 27:54
  SPEAKER_01: It's like actually no, it's a workflow design challenge of okay, actually, you know, there's there's there's a way to design the process of when to use AI, when to like make sure that there actually is the human sort of like not in the loop even, but sort of just like the validation of human input.
• 28:11
  SPEAKER_01: Um but it is hard, right?
• 28:14
  SPEAKER_01: Like this is this is something that you have been feeling out as a leader organization.
• 28:20
  SPEAKER_01: Who have you enlisted as part of this?
• 28:22
  SPEAKER_01: Because what you're describing is when you talk about change management, this is this is really individuals, not just leaders, but also individuals within companies that are pulling like what has your strategy been to kind of you know evangelize?
• 28:36
  SPEAKER_01: Is it is it leadership top down, or have you found sort of allies, even like sort of like from the bottom up?
• 28:42
  SPEAKER_00: Yeah, yeah.
• 28:42
  SPEAKER_00: So uh so it's two different things.
• 28:44
  SPEAKER_00: So let's let's talk a little bit about sort of your comments on teachers and and and workflow in education, and and then I'll come back to sort of change management in the couple.
• 28:52
  SPEAKER_00: So on education, I think you and I spoke about this last time uh we spoke as well.
• 28:57
  SPEAKER_00: And I always use the example of my dad.
• 29:00
  SPEAKER_00: Um my dad who's retired now, he was a high school math teacher, and I actually was his student at some year, not recommended, but uh and he had a reputation of being one of the toughest teachers in in the high school, it would fail students left to right.
• 29:15
  SPEAKER_00: And this was the time when calculators and particularly programmable calculators came around.
• 29:19
  SPEAKER_00: And people say, Oh, no, no, no, no, don't use calculators because you'll never know how to you'll never learn how to calculate.
• 29:24
  SPEAKER_00: And his view was different.
• 29:25
  SPEAKER_00: He's like, no, here's my exam, and if you want to use a calculator, you can bring a calculator.
• 29:30
  SPEAKER_00: The only thing is that if you bring a calculator, I'm gonna give you a different exam.
• 29:33
  SPEAKER_00: It's gonna be a more difficult exam because if you just punch it into the calculator, you won't have learned anything.
• 29:38
  SPEAKER_00: So, and I remember I would always take the more difficult exam because I had a programmable HP calculator, which in like 1984 was like state-of-the-art compute power, and I would just program it and it would solve the the problem.
• 29:52
  SPEAKER_00: But we just learned a different way in some sense, and AI is not fundamentally different in that regard, uh, is that our skills shift as we can mechanize calculation like we did in the 80s, or we mechanize language as we do today, or we mechanize reasoning, quote unquote.
• 30:10
  SPEAKER_00: Um, then our skills move to a different uh territory of uh we use our brains to do other things.
• 30:17
  SPEAKER_00: So that's what I would sort of tell teachers.
• 30:19
  SPEAKER_00: And obviously, the job of a teacher is not to sort of regurgitate knowledge and to put to put facts and data in your brain.
• 30:29
  SPEAKER_00: The the point of a teacher is not to transfer knowledge but to transfer understanding, you know, it's like how do we really conceptualize and how do we create mental models that we can work with and talk about that like you and I are doing.
• 30:41
  SPEAKER_00: That's the point of a of a teacher in some sense.
• 30:43
  SPEAKER_00: Um, and yeah, I can see I have quite a few academic friends.
• 30:47
  SPEAKER_00: I mean, uh, I'm I'm I'm also sort of an honorary professor here, so I feel I need to show up at the local university once in a while.
• 30:54
  SPEAKER_00: And these are the topics that I think some academics have gotten.
• 30:57
  SPEAKER_00: They're like, yes, we get this, we know what our job is going to be, but but many teachers are still sort of struggling with it, which brings me back to the second part of your question.
• 31:05
  SPEAKER_00: How do you deal with the change management?
• 31:07
  SPEAKER_00: Is it top-down?
• 31:08
  SPEAKER_00: Is it bottom-up?
• 31:09
  SPEAKER_00: I mean, again, I'm not a change management expert.
• 31:12
  SPEAKER_00: There's people who have studied this, and there's a lot of like knowledge uh available on how to do it.
• 31:18
  SPEAKER_00: It's not about the CEO pounding the table, it's like we're all going to use AI.
• 31:22
  SPEAKER_00: That's not necessarily the way to do it.
• 31:23
  SPEAKER_00: It's not the way about everybody just thinking on their own and just figure out where we go.
• 31:28
  SPEAKER_00: It is a little bit of a push-pull uh between these two.
• 31:31
  SPEAKER_00: And to me, adopting AI is not different than any other massive change management, I would say.
• 31:39
  SPEAKER_00: And you see the usual adopters and resistors.
• 31:42
  SPEAKER_00: In our company, we have sort of, I mean, we're obviously progressing every day.
• 31:46
  SPEAKER_00: I think these workshops, I used to call it uh the thing we did two weeks ago.
• 31:51
  SPEAKER_00: I call it an AI work day.
• 31:53
  SPEAKER_00: It's sort of a day where we all work.
• 31:55
  SPEAKER_00: It's not just about talking, it's about about sort of show and tell.
• 31:59
  SPEAKER_00: And we all show each other.
• 32:01
  SPEAKER_00: We literally bring up our our our bots and our agents and our prompts and our specifications, and we show each other how our personal workflow has changed.
• 32:09
  SPEAKER_00: And there's a lot of aha moments that happen.
• 32:13
  SPEAKER_00: And I think that sort of individual progress, and then I like groups of about eight to 20 as a good uh sort of in-person group to sort of have that uh dynamic.
• 32:25
  SPEAKER_00: And then, yeah, we have people in our finance organizations who are one of the earliest adopters.
• 32:29
  SPEAKER_00: Like, we we need to hire a junior accountant, and like, well, we don't necessarily have the money, we can't find the right person.
• 32:35
  SPEAKER_00: And then our senior AI um uh finance person is like, oh, I work over the weekend with with Claude, and I have my junior accountant, I'm all sim.
• 32:44
  SPEAKER_00: And that's somebody who's obviously not a technical person who is a uh finance expert, but was able to come up with an agentic workflow to do a junior accounting job.
• 32:53
  SPEAKER_01: So I guess we have to talk about this now that you brought it up, which is something that has increasingly come to the I think public awareness is this idea that you know we don't really know how AI is going to impact the workforce.
• 33:04
  SPEAKER_01: It it does seem that at least in the short term right now, the most obvious way that it impacts is you know, displacing junior hiring to some degree.
• 33:14
  SPEAKER_01: And I it's it's there's been a lot of just sort of reflexive there's a lot of like fast twitch reflexes with a technology like this.
• 33:23
  SPEAKER_01: And so I'm trying to resist making broad statements.
• 33:27
  SPEAKER_01: I think we've seen it ourselves.
• 33:28
  SPEAKER_01: I mean, we're not we haven't like used it to displace anybody, but I think it has it has changed the way that we think about, you know, normally you might bring on an intern to do work, and it's it's hard to imagine what an intern might do that wouldn't actually be more uh Yeah, because a lot of that sort of like entry-level work is the type of stuff that even if I would even if an employee can't quite figure out how to use AI to do that thing, I think even the process of trying to figure out how to use AI to do that is important, right?
• 33:59
  SPEAKER_01: That's that's the learning experience.
• 34:01
  SPEAKER_01: But is that is this temporal or is this actually a structural shift in organizations?
• 34:09
  SPEAKER_00: It's it's something I've thought about as well, and I'm gonna also try to refrain myself from making broad uh statements because we just don't know yet.
• 34:17
  SPEAKER_00: I mean, I used to in some of the presentations I used to give, I used to say AI stands for all interns, meaning that that think of AI as just a lot of really smart interns that you've hired.
• 34:30
  SPEAKER_00: They're very inexpensive, they've read the entire internet, so they're very smart, but at the same time, they have no experience.
• 34:38
  SPEAKER_00: I mean, you'll tell them something, they'll go off the deep end and they'll they'll do things.
• 34:42
  SPEAKER_00: And your job is to sort of rein them in, to make sure that they go off in the right direction.
• 34:47
  SPEAKER_00: That's sort of fun.
• 34:48
  SPEAKER_00: Now, it is definitely true, and we've seen this as well, is that because of this all interns uh step, that it's like, wait a minute, what about a traditional uh professional path where you get a degree or you get an internship and you have to do some grunt work because that's what interns do, and therefore you you get an entry-level job and you sort of move into the uh into the professional world.
• 35:11
  SPEAKER_00: It feels like that traditional chain has been interrupted by the the all interns AI in some sense.
• 35:17
  SPEAKER_00: So the key question is I think what you're what are you gonna ask your interns to do?
• 35:20
  SPEAKER_00: You're not gonna give them grunt work anymore because the agents are gonna do all the grunt work.
• 35:24
  SPEAKER_00: So that that's gone.
• 35:26
  SPEAKER_00: So, but what are you gonna do?
• 35:28
  SPEAKER_00: You're gonna give them sort of junior jobs to say, like, well, you're not gonna go and train the next junior accountant because we're gonna have our senior finance person doing that.
• 35:38
  SPEAKER_00: But we're gonna ask you to show your AIQ skills.
• 35:41
  SPEAKER_00: That's what we're gonna ask you to do because an internship is an extended interview.
• 35:45
  SPEAKER_00: Uh, we're gonna say, well, how good are you at at writing prompts, or how good are you at detecting failures, or how good are you at putting a verification agent together, or how good are you at putting a workflow together?
• 35:55
  SPEAKER_00: And we're gonna ask you to do simple things that are not gonna make or break the company, but we're gonna ask you to show us your skills and we're gonna we're gonna teach you how to do this.
• 36:06
  SPEAKER_00: I mean, I think that's how I I would expect it to evolve.
• 36:09
  SPEAKER_00: Have we seen that yet?
• 36:12
  SPEAKER_00: A little bit, I would say, yeah.
• 36:13
  SPEAKER_00: Well, I think we've seen a little bit of that.
• 36:15
  SPEAKER_00: I mean, uh, and it changes how you hire and it changes how you uh find interns and how you how you teach them, but I think I don't see another way to go.
• 36:23
  SPEAKER_01: It certainly doesn't make sense to just continue hiring interns to do busy work for the sake of posterity.
• 36:29
  SPEAKER_01: Um You know, we've done a lot of these like vibe coding workshops.
• 36:33
  SPEAKER_01: You know, as I think that's been the big change is the um the barrier to entry just has gotten just low enough now that almost anybody, actually say anybody can, you know, within the course of an hour can really yeah.
• 36:47
  SPEAKER_00: Anyone who can write English, yes, and I think or some language, yes.
• 36:51
  SPEAKER_01: I don't know if I'm ready to say this as a blanket statement, but I think it's actually quite common that um the the correlation between like domain expertise and ability to build something cool, it's not totally um correlated.
• 37:04
  SPEAKER_01: It's it's you know, in some cases, actually in a lot of cases, the folks at these workshops who build the coolest things are the folks with the least experience.
• 37:13
  SPEAKER_01: But they have the they're just the fastest it just they just take on to it quickly.
• 37:17
  SPEAKER_01: And I and it's not necessarily even age.
• 37:20
  SPEAKER_01: In some cases it's somebody who is who is just uh you know has a different worldview or mindset or or in some cases they just you know they've they've been curious about something else and they just are are figure out how to channel the curiosity and actually often it's neurodivergent kids or people, people that, you know, I I mean I'm there there'd be it'd be really interesting to look at a body of research on this, but I have you know, we've done work with various nonprofits that that serve students with learning differences.
• 37:51
  SPEAKER_01: And one of my friends actually is um you know one of sort of like the leading uh you know thinkers and speakers in the um neurodiversity community.
• 37:59
  SPEAKER_01: And he and I have talked about this where there is this sort of anecdotally at least, there's something about you know people who just have a slightly different way of approaching or thinking about problems, and then they take on this tool, and sometimes that actually helps them kind of both figure it out, but then also go down the rabbit hole.
• 38:20
  SPEAKER_01: Because I think part of the challenge is you have to you have to get sucked in.
• 38:24
  SPEAKER_01: And for some people that's like a long weekend building, you know, some personal yeah, like um and often people will tell me I don't I'm curious if you have examples of this, but I I think often people will tell me there was like there's there's one project that they they started working on, and that becomes their entry point where you know they spent a few days just like oh, I wanted to build like a a tool to help me reserve camping spots at you know national parks because I always you know forget to wake up early enough to like get the spot, and it took me three or four days, and in that process they learned about you know uh sub agents and like you know what they done then they downloaded ClaudeBot because they realized they needed to have give it access to you know different sort of browsers and the calendar and um and it's hard to just like systematize that it's you know not everybody is gonna find that project as the the thing, yeah.
• 39:16
  SPEAKER_00: So a couple thoughts on that.
• 39:18
  SPEAKER_00: On the uh uh neurodiversity, I'm not an expert on this at all, so I'm gonna be but but I think I can certainly see that people who are good at details and people who are good at sort of tight specifications and and are very sort of uh uh uh used to particular habits of how things are supposed to be done can become very good sort of specification engineers because it it has that level of detail.
• 39:48
  SPEAKER_00: So I could definitely see that as a as a as a non-expert.
• 39:52
  SPEAKER_00: What what it what it what is happening for sure and related to that is that the concept of language becomes so much more important.
• 40:00
  SPEAKER_00: It's not like a language we're just talking because we're chatting and we're having a beer.
• 40:03
  SPEAKER_00: It's like no.
• 40:04
  SPEAKER_00: Now English is becoming more of a formal language and and definitions are important, and words are important, and semantics is important, just like it was for programming language.
• 40:14
  SPEAKER_00: If we're going to use natural language as a specification slash programming language, then being precise is going to become more important.
• 40:21
  SPEAKER_00: So we're gonna have to choose our words very, very carefully in some sense.
• 40:25
  SPEAKER_00: So that's part of AIQ.
• 40:27
  SPEAKER_00: I think in terms of what you said, in terms of uh skills and and also, I mean, what kind of jobs do you need?
• 40:34
  SPEAKER_00: Do you need fewer people in your company?
• 40:36
  SPEAKER_00: There's obviously every CEO in the world is looking at their workforce like, well, do I need all these developers?
• 40:42
  SPEAKER_00: You know, people are telling me AI can do this, maybe I can save money.
• 40:44
  SPEAKER_00: That's sort of what every CEO in the world is doing, and if they're not doing it, their board is telling them to do it.
• 40:50
  SPEAKER_00: And to be honest, I have been the voice in our company already quite some time ago.
• 40:55
  SPEAKER_00: This is like, well, maybe we should have fewer engineers.
• 40:57
  SPEAKER_00: And I have not become the most popular person in our company by saying that, because I say it publicly, I say it privately, I don't sort of say, oh no, I say, look, I I tell our engineers that maybe we should have fewer of you.
• 41:09
  SPEAKER_00: And we actually talk very openly at the workshop about this.
• 41:11
  SPEAKER_00: Because it's like, well, you've been saying we need fewer engineers, but we're the engineers, how dare you?
• 41:15
  SPEAKER_00: And I'm like, okay, let's talk about this.
• 41:17
  SPEAKER_00: Let's let's have a um and there's a couple there's one idea that popped in my head during the workshop, which I thought like you could say, like, hey, we need fewer engineers because agents could do what engineers are doing, so but maybe it's the other way around.
• 41:32
  SPEAKER_00: Maybe about no longer needing, let's say, software engineers, maybe software engineers can start doing everybody's job.
• 41:38
  SPEAKER_00: Maybe your software engineer can become your accountant or can become your salesperson or can become your marketing person because obviously they're not an expert in finance or marketing, but they're very good at writing formal language and building systems, you know, and I can definitely see some of our engineers already doing that.
• 41:55
  SPEAKER_00: I mean, like, hey, I'm not just gonna write code.
• 41:58
  SPEAKER_00: No, I'm gonna work together with our finance person and we're gonna together craft this finance agent because I'm good at formal definitions and specifications of systems.
• 42:09
  SPEAKER_00: That's what I'm good at.
• 42:11
  SPEAKER_00: I mean, whether I'm good at Python or not is no longer relevant.
• 42:14
  SPEAKER_00: So maybe everybody will be able to do that.
• 42:15
  SPEAKER_01: I mean, this is a really important uh conversation in in our space.
• 42:18
  SPEAKER_01: I mean, there is the the computer science teachers all over the country are, you know, I you know, I think there's, and I won't name them, but there are certain CEOs of very large uh companies at the frontier of AI who are who I think have been quite cavalier in the way that they've talked about uh computer science.
• 42:36
  SPEAKER_01: You know, I wouldn't teach my my kid computer science, you know.
• 42:39
  SPEAKER_01: I think AI is going to replace all computer scientists.
• 42:41
  SPEAKER_01: I think there's there's a lack of precision in that language, right?
• 42:44
  SPEAKER_01: Because because sure, it it I think it is undoubtedly I mean it is true.
• 42:48
  SPEAKER_01: It's not even a potential future, it is the it is the current state, which is AI is replacing the the act of writing code.
• 42:55
  SPEAKER_01: You know, very few people are now spending as much time writing literal lines of code as they were two or three years ago.
• 43:02
  SPEAKER_01: But if you actually abstract you know software engineering into computational thinking skills, yeah, I mean it almost feels like you know it's yes, almost everybody now is taking on this sort of computational thinking modality.
• 43:16
  SPEAKER_00: Exactly.
• 43:17
  SPEAKER_00: What does this mean by my college?
• 43:19
  SPEAKER_01: Because I I I worry that you know, we're we were we work in K-12, we're actually quite upstream of college where we don't necessarily have to answer the question of like, well, do you have every skill that you need to enter the workforce?
• 43:31
  SPEAKER_01: But once you get into, you know, post-high school, post-secondary, as we call it in the US, I'm not sure that I'm seeing universities shift as rapidly as you might expect that they should be, right?
• 43:41
  SPEAKER_01: Like I, you know, for example, I haven't seen a total overhaul of computer science programs to become more hybrid, liberal arts computer science where you're still learning to code, but then you're also taking philosophy.
• 43:55
  SPEAKER_01: I think I think there are schools where you can do that yourself, where you can sort of cobble it together and maybe do.
• 44:00
  SPEAKER_01: Double major or minor.
• 44:01
  SPEAKER_01: I mean, do you think that we actually need to be totally shifting the even just like the vocational training of people in computer science, or maybe it's the opposite?
• 44:09
  SPEAKER_01: Maybe it's like liberal arts needs to have more CS?
• 44:12
  SPEAKER_00: Yeah.
• 44:12
  SPEAKER_00: So there's a lot to unpack there.
• 44:16
  SPEAKER_00: So I remember sort of when I went through true education, there was a big thing.
• 44:21
  SPEAKER_00: It's like, oh, everybody should learn to program, you know, because that was sort of a thing 30 years ago.
• 44:26
  SPEAKER_00: I mean, uh now that's obviously absurd today.
• 44:29
  SPEAKER_00: Uh, and then we actually realized like, no, no, no, no, no, not everybody needs to learn how to program.
• 44:33
  SPEAKER_00: I mean, uh, I think everybody needs to know what programmers do, but but not everybody needs to be uh a software developer.
• 44:41
  SPEAKER_00: So sort of taking that same thought today is that uh what's going to happen to computer science as a discipline, that's gonna happen to software engineering as a practice is something I worry about personally too, because my son is uh is a software engineer.
• 44:57
  SPEAKER_00: I mean, he's a computer scientist, he's at the beginning of his career, I'm at the end of my career.
• 45:00
  SPEAKER_00: I don't for me, it doesn't matter so much anymore.
• 45:02
  SPEAKER_00: For him, it matters a great deal.
• 45:04
  SPEAKER_00: I mean, uh, so we we talk about this, about this a lot, but in some sense, the I think you called it cognitive science.
• 45:11
  SPEAKER_00: You need to be able to to to to educate how do you think, how do you design, how do you construct something that has you know a large-scale design and has yes.
• 45:22
  SPEAKER_00: So, do you need to know the the syntax of a programming language?
• 45:25
  SPEAKER_00: Obviously not.
• 45:26
  SPEAKER_00: Do you know the semantics of a programming language?
• 45:28
  SPEAKER_00: Maybe not, but you sure need to understand the semantics of English, and you need to understand the semantics of system engineering.
• 45:35
  SPEAKER_00: You need to be able to say what it is that a system needs to do and what it should not do and how you test it.
• 45:42
  SPEAKER_00: That ultimately is going to become become the the the new skill.
• 45:46
  SPEAKER_00: And there will be just like in anything, there is a pendulum, and sometimes it goes a little bit more one way and it's gonna have to come the other way, and it will eventually end.
• 45:54
  SPEAKER_00: And some of the the CEOs are being cavalier and are just being silly.
• 45:58
  SPEAKER_00: I mean, as low elite, they're just being silly.
• 46:01
  SPEAKER_00: I mean, uh, or they have this quest for AGI or silly things like that.
• 46:05
  SPEAKER_00: It's not what it's all about.
• 46:06
  SPEAKER_00: It's not what it's all about.
• 46:07
  SPEAKER_00: It's it's actually, in my mind, almost the most exciting time in humanity to be alive where we really get to push our brains and our minds and our creativity to things that we've never been able to do before.
• 46:18
  SPEAKER_01: And and just to like describe like like sort of like definitionally computational thinking, it's like this is it's a like problem solving where you're breaking down complex problems, like decomp decomposing those problems, identifying patterns, focusing on important details, abstracting them, developing step-by-step solutions, which is like algorithms.
• 46:34
  SPEAKER_01: Yeah.
• 46:35
  SPEAKER_01: Um and you can imagine in in social studies, in English, in philosophy, certainly in math, but also in computer science.
• 46:43
  SPEAKER_01: I mean, like you can you can actually think about computing digital thinking as a through line across most domains.
• 46:51
  SPEAKER_01: But I think historically we have been so siloed, right?
• 46:55
  SPEAKER_01: Like you you kind of have to you have to sort of self-select so early on.
• 46:59
  SPEAKER_01: And I remember for me, it was like, am I a STEM kid or am I a liberal arts kid?
• 47:03
  SPEAKER_01: And I was like, well, I like language, so I I guess I'm a liberal arts kid.
• 47:06
  SPEAKER_01: Yeah.
• 47:06
  SPEAKER_01: And it's too much to my chagrin.
• 47:08
  SPEAKER_01: I mean, I if I I still to this day, I mean, if only I didn't have a computer science class in high school, and there was nobody telling me, oh, you know, Alex, at least take one computer science class in college.
• 47:19
  SPEAKER_01: I mean, and if I had just had that, you know, it would have given me a fluency to be able to even just like approach my own continued learning.
• 47:27
  SPEAKER_01: I think so much, you know, because I think about like with Spanish, I took a couple years of Spanish, and now, you know, I when I go and travel in countries where that, you know, where Spanish is the primary language, I I have enough I have the I have a comfort level that allows me to then you know figure out what I need to figure out.
• 47:43
  SPEAKER_01: Yeah, yeah.
• 47:44
  SPEAKER_01: Please respond to that.
• 47:44
  SPEAKER_01: And then I want to talk about your son a bit more because it's so interesting.
• 47:47
  SPEAKER_00: I have a few thoughts on this.
• 47:49
  SPEAKER_00: That's all you you talk about about computational science and so on.
• 47:53
  SPEAKER_00: There's even a step before that.
• 47:55
  SPEAKER_00: It's like, you know, do we need to be able to calculate?
• 47:57
  SPEAKER_00: No, we don't need to be able to calculate because we have machines that calculate for us.
• 48:01
  SPEAKER_00: But we do need, and I often, when I teach classes, I often try to teach the kids at whatever age they are the concept of order of magnitude.
• 48:08
  SPEAKER_00: You know, just how big is this?
• 48:10
  SPEAKER_00: You know, how many people are in the US?
• 48:12
  SPEAKER_00: It doesn't matter if you if you're wrong by a factor, too, but you got to know how many digits has that number.
• 48:17
  SPEAKER_00: You know, is it a million, is it 10 million, or is there hundreds of millions?
• 48:20
  SPEAKER_00: Oh, it's hundreds of millions.
• 48:21
  SPEAKER_00: Okay, exactly how many, it doesn't matter.
• 48:23
  SPEAKER_00: You need to sort of think logarithmically.
• 48:25
  SPEAKER_00: You need to know how many.
• 48:27
  SPEAKER_00: It's like, hey, if I take this number and multiply it with this number, oh yeah, I'm not gonna be able to afford it, or it's not gonna work, or it's it's never gonna fit in a computer, or things like that.
• 48:35
  SPEAKER_00: So this kind of order of magnitude on large scale and and small scales like how many, how many significant digits does this have?
• 48:42
  SPEAKER_00: It doesn't that's kind of thing that I think we need to to teach kids, you know.
• 48:46
  SPEAKER_00: I mean, uh to say like, hey, quickly, how long would it take for me to travel to Mars?
• 48:50
  SPEAKER_00: It's like, is it gonna take uh how fast would I have to travel?
• 48:53
  SPEAKER_00: Is it gonna take hours or minutes or seconds?
• 48:55
  SPEAKER_00: I mean, just order magnitude.
• 48:57
  SPEAKER_00: So that's one thing.
• 48:57
  SPEAKER_00: And then the next step to that is then what we traditionally in high-minded terms call complexity theory, which is you know, about how complex is the problem and what is the what is the O of n of n square of a particular algorithm.
• 49:11
  SPEAKER_00: That's sort of very computer geeky speak.
• 49:14
  SPEAKER_00: But I think it's important that people understand that if I do this procedure and I have a million data points, it's roughly going to take a million times as long as one data point.
• 49:24
  SPEAKER_00: But if I have this algorithm and it's a quadratic algorithm and I use it on a million data points, it probably will never finish.
• 49:31
  SPEAKER_00: You know?
• 49:32
  SPEAKER_00: And so this idea of something being linear versus something else being logarithmic or being quadratic, which is considered very technical, is a concept that I think more people are gonna have to learn.
• 49:43
  SPEAKER_00: And if you take a programming class, one of the first assignments you have is to write a sorting program.
• 49:48
  SPEAKER_00: So you write a sorting program, you turn out it's it's it's quadratic and it doesn't work.
• 49:52
  SPEAKER_00: I mean, uh then you realize the smarter ways to do it and you can make it M log N and now it doesn't work.
• 49:57
  SPEAKER_00: And you can use it on a million or a billion data points, and it's no problem.
• 50:00
  SPEAKER_00: I mean, uh, that kind of thinking is gonna have to become general things that everybody in a curriculum would have to take classes.
• 50:07
  SPEAKER_01: What advice would you have to somebody?
• 50:09
  SPEAKER_01: And I was actually just talking to somebody who's um he's he's working at a law firm right now, he has like a sort of a um a case assistant, and his plan originally was to go to law school, and he's not even totally sold on being a lawyer, but he's like, well, that's the path I'm on.
• 50:24
  SPEAKER_01: And you know, he was thinking, well, you know, should I go and maybe go to a coding boot camp?
• 50:30
  SPEAKER_01: But I'm not sure if there are even jobs for engineers anymore.
• 50:34
  SPEAKER_01: And before I tell you what I had attempted to share with him, I'm curious, like, you know, yeah, like what do you have some sort of like generalized advice for for for kids that don't necessarily know what they want to do?
• 50:45
  SPEAKER_01: They're watching this space, they're seeing how much disruption is happening, they haven't quite figured out their passion yet.
• 50:51
  SPEAKER_00: Yeah, I think, okay, I don't necessarily have the magic bullet, but I get this question all the time from kids and from parents, and so on, just the last week is at least three times that this has come up.
• 51:02
  SPEAKER_00: And again, we don't always know yet.
• 51:04
  SPEAKER_00: I would definitely say, oh, I have to go into AI and I have to become a programmer.
• 51:08
  SPEAKER_00: No, that that for sure not.
• 51:09
  SPEAKER_00: I think you have to figure out in this world of the jagged edge, I'm gonna be on the human side of the jagged edge.
• 51:16
  SPEAKER_00: Like, what is the part that really I want to contribute?
• 51:20
  SPEAKER_00: You know, and it's not because of some specific knowledge that I have, but it is gonna become because of a specific skill that I have.
• 51:27
  SPEAKER_00: I'm gonna have the ability to massage this AI beast and to steer it and to rein it in where I need to to actually get something done.
• 51:36
  SPEAKER_00: Whether it is in a legal field or a financial field or uh a technical scientific field, I would still say follow your passion, follow the things that you're good at, you know, because the things that you enjoy to do are typically gonna be the things you're good at.
• 51:49
  SPEAKER_00: Uh, don't think of it as a traditional academic discipline in terms of I have to get a computer science degree or something like that.
• 51:56
  SPEAKER_00: Uh, I think you're gonna have to deal with this technology beast or monster or a stallion or whatever you want to call it that we've built in order to advance humankind.
• 52:08
  SPEAKER_00: You know, I mean, that's ultimately we want to advance the societies and our way of living to improve our general standard of living and our health and our mental health.
• 52:19
  SPEAKER_00: That's what we want to do.
• 52:20
  SPEAKER_00: And that's why we say, like, think of in the large world what piece you want to fit into, and then yeah, learn all these things.
• 52:27
  SPEAKER_00: Take a basic uh uh order of magnitude class, take a basic algorithms class, understand what complexity is, understand why some things work well on the computers and others don't.
• 52:37
  SPEAKER_00: Understand why sometimes you get the craziest answers.
• 52:40
  SPEAKER_00: Um and I don't want to call back.
• 52:45
  SPEAKER_01: It is worth coming back to because this is a moment where there's a lot of buzzwords floating around that don't mean very much.
• 52:52
  SPEAKER_01: And I found that talking about the jagged edge is actually very it's very poignant for people because it intuitively I it makes sense because I think people have had that simultaneously those experiences of like the thing that AI is very good at, as you say, and then also the things that it's like really horrifically bad at.
• 53:08
  SPEAKER_01: But I think putting it into the context of like that is actually the end goal, right?
• 53:12
  SPEAKER_01: It's like figuring that out and being able to not just figure out in a point of time, but developing the instinct to be able to sort of in real time figure it out sort of day by day as it evolves.
• 53:23
  SPEAKER_01: But you said something important, which is so find something you're passionate about, because that's going to be the most likely area that you're going to invest the energy into that with like the sort of the month the entrepreneurial mindset.
• 53:34
  SPEAKER_01: Like if you're giving advice to entrepreneurs, usually it's like make sure you're building a company and something that you're interested in.
• 53:39
  SPEAKER_01: You're doing if you're just doing it to make money, you're not going to succeed.
• 53:42
  SPEAKER_01: But then you also said, you know, make sure that you are pursuing some amount of um uh let like some way of building your computational thinking skills or sort of understanding of understand.
• 53:56
  SPEAKER_01: I mean you almost put it in math.
• 53:57
  SPEAKER_01: I mean, I wonder if math is like it would you say would you say math instead of computer science would be the sufficient and like stand-in?
• 54:05
  SPEAKER_01: I am like do you have to take a CS minor?
• 54:08
  SPEAKER_01: Could it be a math minor?
• 54:09
  SPEAKER_01: Could it be a physics minor?
• 54:10
  SPEAKER_00: Yeah, I mean, yeah, yeah.
• 54:11
  SPEAKER_00: It's it's it's a lot of it is common sense.
• 54:13
  SPEAKER_00: Like thinking in order of magnitude is also common sense.
• 54:16
  SPEAKER_00: You don't have to be a mathematician to do it, but just think about what is a logarithmic and what is a logarithm and what is an exponent, that or those those are good concepts to know for for everybody, uh in my mind.
• 54:28
  SPEAKER_00: I think the other thing is that somebody uh post me this uh this sort of thought experiment that that people like you and I could do.
• 54:36
  SPEAKER_00: It's like sit down and it's a thought experiment, and write down everything you know.
• 54:41
  SPEAKER_00: It's like, holy crap, I would take a while.
• 54:43
  SPEAKER_00: You know, I mean uh I I've forgotten a lot, but write down everything you know.
• 54:47
  SPEAKER_00: And just of course you're not gonna do it, but just imagine it.
• 54:49
  SPEAKER_00: You know, you you now write books and books and books and everything you know.
• 54:53
  SPEAKER_00: Okay, well, all of that really has no value because an AI will know all of this too.
• 54:59
  SPEAKER_00: Okay, if it has been written down, then it has written it somewhere, it has read it somewhere on the internet, it knows it.
• 55:05
  SPEAKER_00: Now think about everything that you can do that is not written down.
• 55:11
  SPEAKER_00: That is your skill.
• 55:12
  SPEAKER_00: You know, that is your skill.
• 55:14
  SPEAKER_00: Like the example that people always give is like riding a bike, you know.
• 55:17
  SPEAKER_00: I can't write down instructions on how to ride a bike, give them to you, and you read them and ride a bike.
• 55:22
  SPEAKER_00: You can't do it.
• 55:24
  SPEAKER_00: Can I have I taught my kids how to ride a bike?
• 55:26
  SPEAKER_00: Yeah, I figured out how to do it.
• 55:27
  SPEAKER_00: I mean, uh, but can I write it down?
• 55:29
  SPEAKER_00: No, I can't write it down.
• 55:30
  SPEAKER_00: You can read it, but it's a skill that is in your brain that we can transfer to other people.
• 55:35
  SPEAKER_00: Now, there's lots of other things like that.
• 55:37
  SPEAKER_00: And you talked about entrepreneurs like, hey, build a company that makes money, that's channel ID ID, but the companies that succeed are the ones that have consistent values, and the the thing that I come back to a lot, and this is more related to my personal work and study that I do on spiritualism, is like, what is your net worth as a human?
• 55:59
  SPEAKER_00: It's not your bank account, it's the sum of your values, you know.
• 56:04
  SPEAKER_00: And this is something that I think makes us uniquely human in some sense because we have values, we have shared values, we might have different values, you know, that makes us human.
• 56:13
  SPEAKER_00: And I think be able to, same with a company.
• 56:16
  SPEAKER_00: A company's gotta have values.
• 56:17
  SPEAKER_00: Like we are the best at this, and we do this that way.
• 56:20
  SPEAKER_00: And it could be the values could be on economic value, it could be on societal value, it could be on lots of different values.
• 56:26
  SPEAKER_00: But if you know your values and you align your workforce on your values, then you can be successful.
• 56:33
  SPEAKER_00: So that's is the part that is the the human part of the the human side of the jacket edge that I think is the fun part in some sense.
• 56:41
  SPEAKER_01: It must have been that that uh the conversation we had in New York where we talked about riding a bike, because I've I found that that analogy it clicks for people in a really important way because well, for a few reasons, yeah, exactly.
• 56:53
  SPEAKER_01: You ask people the I usually would literally do this, I'm like, well, how do you ride a bike?
• 56:56
  SPEAKER_01: And they and then once you get to the balancing part is where they get stuck, they're like, Well, you balance.
• 57:00
  SPEAKER_01: I was like, Well, how do you balance?
• 57:02
  SPEAKER_01: It's like, well, you you shift your weight, like how much do you shift your weight?
• 57:05
  SPEAKER_01: It's like, oh, okay, yeah.
• 57:06
  SPEAKER_01: And they quickly realize, like, yeah, there's just you feel it out.
• 57:09
  SPEAKER_01: You just have to get the feel for it.
• 57:10
  SPEAKER_01: I love and yeah.
• 57:13
  SPEAKER_01: But the other thing about riding a bike is you feel it out.
• 57:16
  SPEAKER_01: It doesn't take years to ride a bike.
• 57:19
  SPEAKER_01: It takes sometimes a few days, maybe a week, maybe two weeks.
• 57:22
  SPEAKER_01: But the other thing about riding a bike is you can you can ride a bike, but then you can look at someone who's a mountain biker.
• 57:27
  SPEAKER_01: You can look at someone who's uh, you know, in Belgium you have um, you know, street uh, you know, like road like road but road bikers.
• 57:33
  SPEAKER_01: Um the cyclists here, you know.
• 57:36
  SPEAKER_01: It isn't trivial to get to a place where you can ride in a Peloton, you know, inches, maybe even millimeters away from the the rider in front of you.
• 57:45
  SPEAKER_01: And so and so I think that there's a power there and like um it's AI does feel like that, where it's like anybody can, I really do believe anybody can be sort of like an AI.
• 57:55
  SPEAKER_01: What is it?
• 57:56
  SPEAKER_01: Uh uh uh James or Jason Lemkins.
• 58:01
  SPEAKER_01: Um Lemkin, this is like on uh 20 VC, is like one of the podcasts.
• 58:05
  SPEAKER_01: He was he's he's coined this new, so there's like prompt engineer, and he's like, Oh, okay, I think that we're gonna be talking about ADEs, agent deployment engineers, which I think was a bit of like almost like a tongue-in-cheek, like let's just continue making up these like sort of meaningless, um uh these meaningless like phrases, but but actually there's like something there, which is you he was basically talking about what you had described, which is this like the what was happening at your comp what is happening in your company is these like re rethinking about what does it even mean to like engineer.
• 58:36
  SPEAKER_01: And so you're you're basically all these folks at your company are various stages of learning to ride this bike, and then you're trying to get them to this place of like, okay, we need to ride in the Peloton together.
• 58:48
  SPEAKER_00: Yes, that's right.
• 58:49
  SPEAKER_00: Yeah, that's a good and haven't I mean, shame on me as a cyclist, I should have figured out these cycling analogies, but yeah, that is a good way to think about it.
• 58:56
  SPEAKER_00: You first have to ride your individual bike, and then you have to figure out how to ride in a Peloton or how to ride in a baseline because you're not gonna win the race unless you know how to ride in a baseline, which is a very particular skill.
• 59:06
  SPEAKER_00: And so maybe there's more to this analogy than I than I thought there was.
• 59:10
  SPEAKER_00: And it's it also reminds me of something that when I taught my kids how to ride a bike, I mean, it was very fascinating because I I personally, as a kid myself, had a very frustrating experience learning to ride a bike.
• 59:21
  SPEAKER_00: It was very frustrating for me, it was very frustrating for my parents.
• 59:24
  SPEAKER_00: So I just wasn't very good at it.
• 59:26
  SPEAKER_00: It took a lot more than a couple of weeks to to get it.
• 59:29
  SPEAKER_00: And I might have had a little trauma for me, it's like, oh, this was not a good experience.
• 59:32
  SPEAKER_00: So I was determined when I when my kids were of the age to I'm not gonna have a frustrating experience.
• 59:38
  SPEAKER_00: So, of course, I did some research.
• 59:39
  SPEAKER_00: I actually looked on the internet, which was kind of innovative those days, and I came up with this 10-step program like here are like step one, you do this, you balance, you do it, and so on.
• 59:49
  SPEAKER_00: And and then I said with my kids, we would practice every day for five minutes, never more than five minutes.
• 59:57
  SPEAKER_00: And any sign of any frustration on either the kid or the parent, we would immediately stop.
• 1:00:02
  SPEAKER_00: Oh, I don't feel like doing this.
• 1:00:03
  SPEAKER_00: Okay, we're gonna hope that's it.
• 1:00:05
  SPEAKER_00: And and then I would rate them.
• 1:00:06
  SPEAKER_00: It's like, oh, I think you're at a level two and a half.
• 1:00:08
  SPEAKER_00: And they're like, Oh, oh, that's really good, Dad.
• 1:00:10
  SPEAKER_00: What about tomorrow?
• 1:00:11
  SPEAKER_00: We'll try a level three.
• 1:00:12
  SPEAKER_00: And what I what I observed is that even though we only practiced five minutes a day, they would always improve between in between sessions.
• 1:00:21
  SPEAKER_00: They would never improve during uh uh an actual practice session, they would improve between sessions, like overnight, they would get better.
• 1:00:29
  SPEAKER_00: And I always thought it must be something in your brain that your brain at night is rewiring some neurons and it's given some stimuli about balancing a bike, and you don't know how to do it, but it sort of compiles it overnight, and the next day you're better at it.
• 1:00:43
  SPEAKER_00: Uh, and there's something maybe to learn from that, because I think something is because a bike is like a jagged edge.
• 1:00:49
  SPEAKER_00: If you don't do it right, it speaks to it.
• 1:00:51
  SPEAKER_01: And I know that we're coming up on time, and I think this is a really sort of beautiful way to end it for you know, for folks that are trying to uh you know find their path.
• 1:00:59
  SPEAKER_01: And so we know one of the things you said I think is important, there is no sort of right answer.
• 1:01:02
  SPEAKER_01: You're asked this sounds like a couple times a week, and so you've avoided just sort of giving something prescriptive.
• 1:01:08
  SPEAKER_01: And yes, it's like riding a bike, but it's also this consistency because it will build you know, like getting the feel for something is is not going to happen overnight.
• 1:01:17
  SPEAKER_01: You can you can get far enough where you can actually start to.
• 1:01:20
  SPEAKER_01: I mean, I think for me, the more the the the pieces getting it's really helpful to get to a place where you actually had built something just so you can see it's like you know, just just just to see that there's there's a magic here.
• 1:01:32
  SPEAKER_01: Um and the other thing that you you had talked about is sort of like find your passion.
• 1:01:38
  SPEAKER_01: And the important because and just to sort of connect the dots here is like because that's important because if you're working on something you're passionate about, that will be the thing that continues to bring you back over an extended enough period of time where you will start to have the benefits of sort of that compounding, yeah, sort of like compiling that's happening, like sort of in your brain.
• 1:01:57
  SPEAKER_00: And then maybe one final uh thought for my stuff, because you're in education, you said you're K-12 education, but with AI, education will never end.
• 1:02:07
  SPEAKER_00: There's no end to it.
• 1:02:08
  SPEAKER_00: Oh, now I've gotten my degree and I'm gonna practice it.
• 1:02:11
  SPEAKER_00: No, because the irony of AIQ is that every day your AIQ will go down, unless you keep it up.
• 1:02:18
  SPEAKER_00: Because AI will become smarter, the X edge will become more jagged.
• 1:02:21
  SPEAKER_00: So if you don't keep up, your AIQ, unlike your IQ, which presumably will remain relatively steady till maybe you get older, your AIQ, without doing anything, will go down.
• 1:02:31
  SPEAKER_00: So you constantly have to keep it up in some sense.
• 1:02:34
  SPEAKER_00: So that's why it changes the whole everybody's an intern in a way.
• 1:02:37
  SPEAKER_00: I'll be we're like I said, we're practicing, we're doing new projects, we're finding passions at every age and at every stage.
• 1:02:43
  SPEAKER_00: I know I really like the parting thought.
• 1:02:44
  SPEAKER_00: So maybe that's a parting thought.
• 1:02:47
  SPEAKER_01: I will I have a I have a lot of it's it's it's connected to some dots for me.
• 1:02:52
  SPEAKER_01: Um I'm gonna maybe I'll do some writing and I'll send I'll send you something.
• 1:02:55
  SPEAKER_01: Um we'll we'll have to set up another session.
• 1:03:00
  SPEAKER_01: It's very fun.
• 1:03:03
  SPEAKER_00: Thank you.
• 1:03:04
  SPEAKER_00: We'll find another time.
• 1:03:08
  SPEAKER_00: Hopefully we haven't looked different than it looks today, that's a guarantee.