Latest / Coffee with Developers / Kelsey Hightower - ZTA: Zero Token Architecture

Transcript

Kelsey Hightower - ZTA: Zero Token Architecture · Coffee with Developers

• 0:14
  SPEAKER_01: Hello, hello, hello.
• 0:15
  SPEAKER_01: And we have three more talks to go.
• 0:18
  SPEAKER_01: The first one now is half a talk, half a long QA.
• 0:22
  SPEAKER_01: That's what he said he wanted to do.
• 0:23
  SPEAKER_01: So we got Kelsey Haitar for you, back from the silence that he's done.
• 0:28
  SPEAKER_01: He hasn't given a talk for quite a while.
• 0:30
  SPEAKER_01: He was a distinguished Google engineer.
• 0:32
  SPEAKER_01: Anybody who had anything to do with data storage knew about him and still knows about him.
• 0:37
  SPEAKER_01: So we're very happy that he's here.
• 0:38
  SPEAKER_01: He's going to talk about zero token architecture, meaning, like, why do we forget the most basic things, how to optimize and burn tokens just for the sake of burning tokens?
• 0:48
  SPEAKER_01: So hopefully there will be some good information for you.
• 0:50
  SPEAKER_01: Please ask questions in the app so I can have something clever to ask because uh I can otherwise ask him non-clever things.
• 0:57
  SPEAKER_01: So we'll see about that.
• 0:58
  SPEAKER_01: And use the app for that, and we see you in a second.
• 1:01
  SPEAKER_01: And now I bring to the stage Kelsey Hightower.
• 1:05
  SPEAKER_01: So happy to have him here.
• 1:07
  SPEAKER_01: Thanks very much.
• 1:17
  SPEAKER_00: I promised myself I would never make slides again, and I don't know why, but I made slides just for you all.
• 1:23
  SPEAKER_00: Uh first thing, zero token architecture isn't really a thing.
• 1:27
  SPEAKER_00: I made it up out of spite.
• 1:30
  SPEAKER_00: As someone retires scrolling through LinkedIn, y'all seem to be having a lot of fun burning tokens.
• 1:37
  SPEAKER_00: And every once in a while I get the privilege of going to a conference and talking to people about what are you burning the tokens for.
• 1:45
  SPEAKER_00: And some of the things you all mentioned, I feel embarrassed for you.
• 1:52
  SPEAKER_00: And so the first thing is in my domain, the agentic world is probably a little bit more applicable because I'm seeing people starting to bring agents into things like workflows or things you would see in the infrastructure world.
• 2:02
  SPEAKER_00: And so the first thing I did is, since as an outsider, I had to get the terminology correct.
• 2:07
  SPEAKER_00: Okay?
• 2:08
  SPEAKER_00: So first thing is what is an agent?
• 2:11
  SPEAKER_00: And I ask people, and they just make up all kind of stuff.
• 2:15
  SPEAKER_00: But at the end of the day, it's something that performs a specific task.
• 2:21
  SPEAKER_00: And some of you all like to brag about how many agents you have running, like it's a competition.
• 2:26
  SPEAKER_00: I got 5,000 agents doing absolutely nothing.
• 2:33
  SPEAKER_00: And then I had to deal into the details of like an AI agent, because this is a little bit specific.
• 2:39
  SPEAKER_00: And so when you think about an AI agent, what is that?
• 2:42
  unknown: Right?
• 2:42
  SPEAKER_00: So it's something that has a model, a prompt, some tools, an MCP server, and some skills.
• 2:51
  SPEAKER_00: I'm like, damn, that's a lot of stuff to perform specific types of tasks.
• 2:57
  SPEAKER_00: And just so you know, that I did go, what I think is one of the best sources for this type of thing.
• 3:02
  SPEAKER_00: I just went straight to anthropic.
• 3:03
  SPEAKER_00: That's the place where you a lot of you seem to be burning your tokens.
• 3:07
  SPEAKER_00: It's also the people who seem to be in the driver's seat right now.
• 3:10
  SPEAKER_00: So I know what it's like to have a hype cycle, but it's also clear that there's always a few people really driving things, and there's no doubt that the things I'm seeing people do with Claude, it's worth the hype.
• 3:20
  SPEAKER_00: Not that I'm a big fan, but I can recognize when something is making momentum.
• 3:26
  SPEAKER_00: And so I digged in, it's like, let me read the documentation.
• 3:29
  SPEAKER_00: That's the thing you still do in AI, right?
• 3:30
  SPEAKER_00: You still read the docs.
• 3:31
  SPEAKER_00: Do y'all still read?
• 3:33
  unknown: No.
• 3:34
  SPEAKER_00: So you don't even read the docs.
• 3:35
  SPEAKER_00: Y'all ask Claude to tell you what's in the docs.
• 3:38
  SPEAKER_00: Boy, y'all gonna be in for some pain when people start injecting nonsense.
• 3:44
  SPEAKER_00: And I zoomed in, so I read the docs.
• 3:46
  SPEAKER_00: And this is the line that jumped out to me.
• 3:50
  SPEAKER_00: You can execute an agent and will optionally execute bundle code or load references to other files.
• 4:01
  SPEAKER_00: I fell out of my chair.
• 4:04
  SPEAKER_00: Five, ten years ago, if you told me that you had a process that could run arbitrary code, we would consider that malware.
• 4:14
  SPEAKER_00: We would try to remove all of this stuff.
• 4:16
  SPEAKER_00: We spent so much time getting to zero trust where you run things and you remove all implicit side effects.
• 4:24
  SPEAKER_00: That's actually really, really hard to do.
• 4:26
  SPEAKER_00: If you're really good at it, you can do formal verification of your software.
• 4:31
  SPEAKER_00: Then I wake up and y'all like, forget all of that.
• 4:34
  SPEAKER_00: Let it do whatever it wants, and we'll ask it not to.
• 4:42
  SPEAKER_00: And so I get it.
• 4:44
  SPEAKER_00: You need all of this stuff, and you need guardrails, and you need all of these things.
• 4:49
  SPEAKER_00: And I don't know y'all say you don't write the code anymore, but I've seen y'all markdown files.
• 4:54
  SPEAKER_00: Y'all writing a lot in those markdown files that I'm starting to think it would have been a little easier if you just wrote the first code in the first place.
• 5:02
  SPEAKER_00: But this talk isn't about whether you should AI to write code or not.
• 5:07
  SPEAKER_00: The thing I want you to think about is should you be running inference in a tight loop?
• 5:12
  SPEAKER_00: So now this stuff is moving into the agent realm, and I'm watching the agent, if you ship an agent with all of these things and you ask it to do inference for every part of that task, it seems to go against the very theory of automation to begin with.
• 5:27
  SPEAKER_00: You try to figure out a process and you make it efficient, and then you lower the cost to the point where some tasks can be put into the hardware.
• 5:37
  SPEAKER_00: TCP offloading.
• 5:39
  SPEAKER_00: There's so many things that we've gotten to the point of when it's expensive, we always find a way to make it cheap.
• 5:46
  SPEAKER_00: Uh expensive SQL query.
• 5:48
  SPEAKER_00: If you try to query a database and do a select or join that causes too much friction, what do we learn to do?
• 5:56
  SPEAKER_00: We cache the result.
• 5:57
  SPEAKER_00: So that way it's been a tight loop.
• 5:59
  SPEAKER_00: You're not going to destroy your database.
• 6:01
  SPEAKER_00: It makes no sense.
• 6:03
  SPEAKER_00: And then we also validate our inputs.
• 6:05
  SPEAKER_00: Why?
• 6:05
  SPEAKER_00: Because we don't want people injecting bad data into our database.
• 6:09
  SPEAKER_00: I know it's AI, so you're gonna allow this, but I'm gonna enjoy y'all relearning SQL injection and what that means is gonna be amazing.
• 6:15
  SPEAKER_00: And I look forward to the name that you give to these fundamentals, but I promise you the fundamentals will be the same.
• 6:21
  SPEAKER_00: So, what is this zero token thing that I made up?
• 6:25
  SPEAKER_00: Uh just being spiteful.
• 6:31
  SPEAKER_00: A zero token agent is probably like all the software you actually run.
• 6:35
  SPEAKER_00: Okay?
• 6:36
  SPEAKER_00: Now, what I'm gonna do, the reason why I wrote slides is I need to be very nuanced about what I'm talking about here.
• 6:42
  SPEAKER_00: Normally I would just have a discussion about this, but I want to remind some people because some people are new to this field, there's new to this industry, and they didn't have the benefit of going through the last 20 years to understand why some of this stuff actually makes sense, this AI stuff, some of it is attractive.
• 6:57
  SPEAKER_00: And I have to remind myself so I can have a little bit more empathy when I'm talking about this.
• 7:02
  SPEAKER_00: The other thing I want to keep in mind is I have a bit of a bias.
• 7:05
  SPEAKER_00: Machines have artificial intelligence, they don't have wisdom.
• 7:08
  SPEAKER_00: And the wisdom component is the experience piece.
• 7:11
  SPEAKER_00: The part where you ask Claude, should you watch a movie, and you blindly go or don't go because of Claude's opinion is insane to me.
• 7:19
  SPEAKER_00: Claude has never been to a movie theater, has never bought no popcorn.
• 7:25
  SPEAKER_00: And there are movies that you'll like because of who you are, not because you're like everybody else.
• 7:31
  SPEAKER_00: This is the thing that makes you human.
• 7:34
  SPEAKER_00: So intelligence plus experience is the thing you use to make decisions.
• 7:39
  SPEAKER_00: So if you're only making decisions based on intelligence alone, you're missing the point.
• 7:45
  SPEAKER_00: And so running an agent in a tight loop with only intelligence and no experience, I'm not surprised.
• 7:51
  SPEAKER_00: It's doing things that surprise you.
• 7:56
  SPEAKER_00: And throughout my career, there's one thing I've learned from the wisdom side of it.
• 8:02
  SPEAKER_00: You're not gonna need it.
• 8:04
  SPEAKER_00: You ever see an engineer working on something?
• 8:06
  SPEAKER_00: You're like, what are you doing?
• 8:07
  SPEAKER_00: Why are you adding all of that?
• 8:09
  SPEAKER_00: Just in case.
• 8:11
  SPEAKER_00: And we frowned upon that because if you start adding all of these just in case things, it's just a bigger attack surface.
• 8:17
  SPEAKER_00: It's more things that we have to update.
• 8:20
  SPEAKER_00: We went through this process of trying to figure out how to ship software that does what it needs to do and nothing else.
• 8:26
  SPEAKER_00: That was the benefit, that was an actual feature.
• 8:29
  SPEAKER_00: Now I get it, I respect the power of flexibility and having something that can actually react off of a feedback loop.
• 8:35
  SPEAKER_00: But remember the trade-off here.
• 8:37
  SPEAKER_00: That surface gets very big to the point where now you don't understand it.
• 8:42
  SPEAKER_00: So I'm gonna get into a use case.
• 8:44
  SPEAKER_00: Remember, I'm old, so someone at my age, I don't buy crypto tokens, I buy treasury bonds.
• 8:52
  SPEAKER_00: Your great-grandmother probably bought these.
• 8:54
  SPEAKER_00: Now, some of you will laugh because it's not an exciting thing, but the world runs off of treasuries.
• 9:00
  SPEAKER_00: Whatever value you think you have, put a multiple on it, and that's the treasury market.
• 9:05
  SPEAKER_00: And so for someone like me, I tracked the treasury market and the prices and all these things.
• 9:09
  SPEAKER_00: So I said, hey, let me try an AI agent to see if it can help me manage my treasury portfolio.
• 9:16
  SPEAKER_00: Okay, very simple task.
• 9:18
  SPEAKER_00: Now, some of you are like, Helsinki, that task is so simple, why would you use an agent for it?
• 9:22
  SPEAKER_00: This is exactly the question I've been asking y'all.
• 9:25
  SPEAKER_00: Some of the tasks y'all showed me, I'm like, why are you using an agent for it?
• 9:29
  SPEAKER_00: So if this looks ridiculous, take a look in the mirror.
• 9:35
  SPEAKER_00: So, all sophisticated investments, investors, this is how we manage our portfolios.
• 9:40
  SPEAKER_00: On a spreadsheet, I don't care what software is out there, you gotta have your portfolio on the spreadsheet in case the financial institution goes away.
• 9:49
  SPEAKER_00: I'm not sure this is gonna help you in court, but it do make you feel a little bit better when the spreadsheet matches your brokerage account.
• 9:57
  SPEAKER_00: And so on this spreadsheet, I have one of my treasury bonds, one of the serial numbers, I just made up a quantity.
• 10:03
  SPEAKER_00: And the thing about managing a treasury portfolio, when the interest rates go up, the price of your bond goes down.
• 10:10
  SPEAKER_00: It's weird, but we're not here for finance education.
• 10:14
  SPEAKER_00: And how would you manage this manually?
• 10:15
  SPEAKER_00: Think through the steps, right?
• 10:17
  SPEAKER_00: You will go and get your information about your treasury, and then you what you have to do is go get the current market rate to see what the interest rates are.
• 10:23
  SPEAKER_00: Do they go up if they go down?
• 10:25
  SPEAKER_00: You need to figure that out.
• 10:26
  SPEAKER_00: Now you could go to a website to get this information.
• 10:29
  SPEAKER_00: You may or may not trust the pricing data, and then once you have that information, you gotta calculate the current bond price.
• 10:35
  SPEAKER_00: So if you bought a bond for 4.75%, you need to calculate how much it's worth now if the interest rates go up, because now it's worth less.
• 10:42
  SPEAKER_00: And then you can update your spreadsheet with the new balance.
• 10:46
  SPEAKER_00: Simple, right?
• 10:47
  SPEAKER_00: Now, if you have any experience, you'd be thinking, Kelsey, you can solve this problem with a macro inside the spreadsheet.
• 10:54
  SPEAKER_00: Right?
• 10:55
  SPEAKER_00: You do not need to have Claude figure this out for you.
• 10:59
  SPEAKER_00: Right?
• 11:00
  SPEAKER_00: This is a common place where you don't need to burn any tokens to solve this problem.
• 11:04
  SPEAKER_00: You can do it right there in the spreadsheet.
• 11:05
  SPEAKER_00: But the problem is, do you really want your spreadsheet going to the internet to look at bond prices?
• 11:11
  SPEAKER_00: You could.
• 11:12
  SPEAKER_00: You could probably write an Excel macro to do this.
• 11:15
  SPEAKER_00: And you could solve this problem without using Claud.
• 11:18
  SPEAKER_00: But we're in the age of AI, we have to use it for everything.
• 11:21
  SPEAKER_00: I know some of y'all have token burning quotas at work, so you gotta burn tokens somehow.
• 11:27
  SPEAKER_00: So we're finna burn tokens or maybe burn tokens to solve this problem.
• 11:31
  SPEAKER_00: Now, the first thing I have to do is I have to go to the Treasury website.
• 11:35
  SPEAKER_00: Now I know some of you be like, you can just ask Claude where to go to get the bottom rates.
• 11:39
  SPEAKER_00: This is also insane to me.
• 11:41
  SPEAKER_00: There's only one official source for this data.
• 11:44
  SPEAKER_00: Maybe Claude knows that, maybe it doesn't.
• 11:46
  SPEAKER_00: But that's the decision you need to make.
• 11:49
  SPEAKER_00: And when you go there, you're presented with like this list of interest rates every day that give you the last set of interest rates that have traded on the market.
• 11:58
  SPEAKER_00: Now, once I have that, I can extract it.
• 12:01
  SPEAKER_00: And then you gotta do some math.
• 12:03
  SPEAKER_00: How many people want the college in here?
• 12:06
  SPEAKER_00: How many people understand this formula?
• 12:08
  SPEAKER_00: Aha! And then the three of y'all lying in the back.
• 12:13
  SPEAKER_00: But it's a very simple formula.
• 12:15
  SPEAKER_00: And the interesting thing about this formula this is a human judgment, nothing more.
• 12:20
  SPEAKER_00: Humans have agreed that this formula represents the fair price of this particular asset.
• 12:24
  SPEAKER_00: That's all it means.
• 12:26
  SPEAKER_00: There's no more opinionation other than humans said that's a fair rate when rates go up or down.
• 12:31
  SPEAKER_00: That's it.
• 12:32
  SPEAKER_00: And we represented this human decision into a mathematical formula.
• 12:38
  SPEAKER_00: There is no room for interpretation.
• 12:40
  SPEAKER_00: This is the agreed upon formula.
• 12:43
  SPEAKER_00: And so when you get all these things put together, you can do the math, and if interest rates are high, which they are now, you've lost a whole bunch of money if you sold those things.
• 12:52
  SPEAKER_00: Now, during this process, I actually wrote some code.
• 12:56
  SPEAKER_00: And I wrote some code to remind myself like, why do y'all want Claude generating a bunch of code for you all?
• 13:02
  SPEAKER_00: I was like, I gotta remind myself because this surely is an easy task to do.
• 13:06
  SPEAKER_00: How long would it take you all to write something like this if you had to write it from scratch?
• 13:11
  SPEAKER_00: Five minutes?
• 13:14
  SPEAKER_00: An hour?
• 13:16
  SPEAKER_00: Oh, people being honest.
• 13:18
  SPEAKER_00: How long would it take Claude to do it?
• 13:20
  SPEAKER_00: Five minutes?
• 13:22
  SPEAKER_00: Thirty seconds?
• 13:24
  SPEAKER_00: Alright, I'm gonna get with you afterwards, and I'm gonna see if you can do it, because I tried it and I ran out of tokens.
• 13:31
  SPEAKER_00: It tells you out of tokens, and you gotta upgrade your plan.
• 13:34
  SPEAKER_00: I wasn't giving anthropometry my credit card, but it did take longer than five seconds.
• 13:40
  SPEAKER_00: And I'll tell you why later, probably at a different event.
• 13:43
  SPEAKER_00: So I wrote the code myself.
• 13:44
  SPEAKER_00: Now in your mind, as a developer, you think it's easy.
• 13:48
  SPEAKER_00: In your mind, you're thinking how hard it could be.
• 13:50
  SPEAKER_00: Let me make this a little bit bigger.
• 13:53
  SPEAKER_00: And so in my mind, it looks like this.
• 13:56
  SPEAKER_00: Right?
• 13:57
  SPEAKER_00: Figure out what the bond portfolio is and update the spreadsheet.
• 14:03
  SPEAKER_00: And about, and I'm embarrassed.
• 14:06
  SPEAKER_00: It took me about six hours.
• 14:09
  SPEAKER_00: It took me six hours because it turns out calculating bond rates and finding all the right data in XML format and transferring it to the right format that you need.
• 14:19
  SPEAKER_00: And then the thing about calculating time, there's leap seconds and leap years.
• 14:24
  SPEAKER_00: And if you get that wrong, your bond prices will be off.
• 14:27
  SPEAKER_00: And there's some other things you need to adjust for, unless you understand the algorithm deeply, you will just be wrong.
• 14:33
  SPEAKER_00: And so when I tried to do this using, you know, the whole point is you take that algorithm or that mathematical formula and you have to turn it into code.
• 14:42
  SPEAKER_00: And that part was actually easy.
• 14:44
  SPEAKER_00: So the part that, you know, you look at that equation, I'm gonna show it to you.
• 14:47
  SPEAKER_00: That was actually pretty simple.
• 14:49
  SPEAKER_00: Like this.
• 14:50
  SPEAKER_00: This is simple.
• 14:51
  SPEAKER_00: Like calculating that bond rate was super simple, but all the nuance only an experienced person could do.
• 14:57
  SPEAKER_00: And so it reminded me of like, number one, writing code is hard mainly because there's no documentation for most of the libraries.
• 15:04
  SPEAKER_00: There's a discovery problem, what tools you should even use to do it.
• 15:08
  SPEAKER_00: And so now I kind of respect a bit more why people want things like MCP, because now we start to address the, I guess, the discovery problem.
• 15:16
  SPEAKER_00: Where's the endpoints?
• 15:17
  SPEAKER_00: What do the endpoints do?
• 15:18
  SPEAKER_00: Because trust me, the US Treasury doesn't have a RESTful API documented at all.
• 15:23
  SPEAKER_00: You end up screen scraping the website.
• 15:26
  SPEAKER_00: And at the end of it though, I did end up with a binary that can update a bond portfolio.
• 15:33
  SPEAKER_00: Now, if somebody was like, Kelsey, that's that's not an agent though.
• 15:36
  SPEAKER_00: Because it doesn't run autonomously, right?
• 15:40
  SPEAKER_00: Would you say that I got that part wrong?
• 15:41
  SPEAKER_00: It doesn't run on its own.
• 15:43
  SPEAKER_00: And I thought about it for a while.
• 15:45
  SPEAKER_00: I was like, oh, you might got me.
• 15:47
  SPEAKER_00: And I was like, okay, what would it take to make this thing run on its own?
• 15:52
  SPEAKER_00: And I thought long and hard about it.
• 15:55
  SPEAKER_00: And it turns out there's a thing on your computer that will run this agent every five minutes.
• 16:03
  SPEAKER_00: And this cost me no tokens at all.
• 16:07
  SPEAKER_00: And it has the same properties and the behavior.
• 16:09
  SPEAKER_00: Now you're looking at me like Kelsey Dow.
• 16:11
  SPEAKER_00: This, you're writing code like a caveman.
• 16:14
  SPEAKER_00: And I won't argue.
• 16:16
  SPEAKER_00: I will admit, this is probably not something that if I was in the industry at this moment, that I would ever see myself wanting to do again.
• 16:23
  SPEAKER_00: If you told me that a computer can actually do a good job at generating code and doing this for me, I might be interested in using it.
• 16:30
  SPEAKER_00: And I know most of you will tell me, Kelsey, you should just use something like this instead.
• 16:34
  SPEAKER_00: Go to the chat window, express this as a prompt, and you're gonna get the same result and behavior.
• 16:40
  SPEAKER_00: But if you did that and you let it deploy the agent for you, every time in that loop, it may sit there and burn tokens trying to discover what tool to call, or if the model changes, you may get different behavior than before.
• 16:53
  SPEAKER_00: And in that world, I'm gonna leave you with one thing.
• 16:56
  SPEAKER_00: If you're gonna go this route, fine, teach their own.
• 16:59
  SPEAKER_00: But I don't think it makes sense to be burning tokens in a tight loop.
• 17:03
  SPEAKER_00: You should be thinking about inferring once and then running the results anywhere.
• 17:09
  SPEAKER_00: There's an analogy that I think will help you with this.
• 17:11
  SPEAKER_00: Imagine if you wanted to have, I don't know, Beyonce make a new song.
• 17:16
  SPEAKER_00: If you wanted her to go to the studio every time you wanted to hear that song, that would be silly.
• 17:21
  SPEAKER_00: Number one, she ain't gonna do it.
• 17:24
  SPEAKER_00: And it takes a lot of work to set up a music studio, get all the producers, writers, songs, and the singers in one space.
• 17:31
  SPEAKER_00: But once you do it, it's a lot of work that goes into that, you should then export something that can then be played over and over again in a predictable manner without the same expense that it costs to make the song in the first place.
• 17:45
  SPEAKER_00: And that's the architecture I want to remind people of in computing.
• 17:48
  SPEAKER_00: Use the tools necessary to create the object, but in automation theory, it's just much better to run that thing in a predictable way.
• 17:56
  SPEAKER_00: And so I'm gonna leave you with one prompt, so maybe one day Cloud will support this functionality.
• 18:00
  SPEAKER_00: You should be able to give it some prompt like this like whatever it does, the last thing we should tell it is turn all of that into a shell script and remove all the dependencies that aren't necessary.
• 18:12
  SPEAKER_00: And if this thing is as good as you say it is, it should have no problem exporting the logic to something that cuts the dependencies and works the same way.
• 18:22
  SPEAKER_00: Thank you.
• 18:26
  SPEAKER_00: Now I left time for a QA.
• 18:28
  SPEAKER_00: Like, hey, what is the music person doing?
• 18:29
  SPEAKER_00: I got some time.
• 18:34
  SPEAKER_00: That's for them?
• 18:36
  SPEAKER_00: Oh.
• 18:36
  SPEAKER_00: Oh, y'all have a formula.
• 18:39
  SPEAKER_00: Now we're problems.
• 18:42
  SPEAKER_01: Yeah, uh, we had a few questions coming in.
• 18:44
  SPEAKER_01: Uh but I I have so much fun doing shell scripting again right now myself.
• 18:48
  SPEAKER_01: Because I I also like I get my agent, build things, then I got 15 folders with files, and I'm like, what is all that like?
• 18:56
  SPEAKER_01: It's just unbelievable.
• 18:58
  SPEAKER_01: You're just burning tokens.
• 18:59
  SPEAKER_01: Yeah.
• 19:00
  SPEAKER_01: Yeah.
• 19:00
  SPEAKER_01: In 30 years of my career, I got never actually got a promotion for spending money in uh for the company.
• 19:06
  SPEAKER_01: But we are in that place, aren't we?
• 19:09
  SPEAKER_00: Oh there's one thing I want to make sure.
• 19:11
  SPEAKER_00: This people don't get this joke.
• 19:13
  SPEAKER_00: So when I started my career as a system administrator, if you wasn't good at your job, someone will walk by your desk and threaten to replace you with a shell script.
• 19:22
  SPEAKER_00: So this is a callback to that.
• 19:24
  SPEAKER_00: If you catch yours burning tokens too much, you can also threaten to replace Claude with a shell script.
• 19:29
  SPEAKER_00: I just wanted to throw that out there.
• 19:31
  SPEAKER_00: He's like, I got it the first time.
• 19:34
  SPEAKER_01: So please use the app to ask questions because otherwise it's not gonna be on the feed, so we can't do that, that we just shout things because it's it's a different room here.
• 19:42
  SPEAKER_01: So um, one of them is a big one that I keep hearing all the time as well.
• 19:46
  SPEAKER_01: How do you recommend bringing managers back to reality?
• 19:49
  SPEAKER_01: We could stop there, but there's more information.
• 19:51
  SPEAKER_01: I feel like I often have to fight back against the AI salespeople and their demos promising AI will solve everything.
• 19:57
  SPEAKER_01: They see AI as a hammer and everything as a nail and don't have the technical skills to know any better.
• 20:04
  SPEAKER_01: How can you stand your ground as a technical person there?
• 20:06
  SPEAKER_00: I mean, I think you've got to have the right attitude.
• 20:08
  SPEAKER_00: For example, if I get a new tool, there's a chance I don't know how to use the tool effectively.
• 20:12
  SPEAKER_00: I'm admitting that today.
• 20:13
  SPEAKER_00: So, step one, you've got to get enough skill to be able to make a real assessment whether this thing is good or not.
• 20:19
  SPEAKER_00: So I leave myself a little grace period before I start complaining, typically.
• 20:23
  SPEAKER_00: Just not on the internet.
• 20:26
  SPEAKER_00: Uh and then I think the second part of it is you can just prove it.
• 20:29
  SPEAKER_00: We're supposed to be engineers, right?
• 20:31
  SPEAKER_00: So the way you prove it is if you say that AI is gonna solve a particular problem and you think there's a way to solve it in a better way without it, then you should be able to prove it.
• 20:42
  SPEAKER_00: And I think the proof comes into a situation where maybe you do have a session where you're trying to generate the code, and I know some of these things will get stuck.
• 20:50
  SPEAKER_00: Record your time.
• 20:52
  SPEAKER_00: And then ask yourself, is there a better way of doing it?
• 20:54
  SPEAKER_00: And if so, bring that data.
• 20:55
  SPEAKER_00: My guess is this thing will get better at generating code over time.
• 20:58
  SPEAKER_00: But in the case of that zero token piece around burning tokens unnecessarily, if I was at a company and I wanted to prove that to the management team, I would say, look, here's how much it costs to run these agents.
• 21:09
  SPEAKER_00: At some point you should be able to calculate every loop and how much it costs.
• 21:12
  SPEAKER_00: A dollar and fifty cents.
• 21:14
  SPEAKER_00: Okay, fine.
• 21:15
  SPEAKER_00: Do the math.
• 21:16
  SPEAKER_00: And then if I said, hey, Claude, export this thing without all the necessary dependencies, and we're still getting the same results.
• 21:23
  SPEAKER_00: Now I can just do math and say, hey, we're no longer spending a dollar and fifty cent for each of these loops at running every second.
• 21:29
  SPEAKER_00: And so if you're getting money back, I think the the manager would say you're still using AI maybe on the front end to create the thing that's being put in the loop.
• 21:37
  SPEAKER_00: But then you're removing the unnecessary dependency.
• 21:40
  SPEAKER_00: If someone was like, no, I'd rather you burn the tokens instead, that's a different conversation.
• 21:44
  SPEAKER_00: But that's the way I would approach that problem.
• 21:46
  SPEAKER_00: I would just actually go through the scientific method and prove it.
• 21:49
  SPEAKER_00: And then the last thing I'll say here is just be open to being wrong.
• 21:52
  SPEAKER_00: Right?
• 21:53
  SPEAKER_00: Maybe that agent is doing something that you can't do in an imperative loop or in a predictable way, and they want that flexibility.
• 21:59
  SPEAKER_00: Maybe they're reaching.
• 22:00
  SPEAKER_00: Unstructured data and they're outputting things in multiple ways that you don't want to program from scratch, then maybe it needs to be that way.
• 22:07
  SPEAKER_00: But I think you can actually find the truth at some point.
• 22:09
  SPEAKER_01: There is evidence that like structuring your data properly before you actually get the agent on it is the best saving you can do.
• 22:17
  SPEAKER_01: Work with clean information, work with clean tools, find the right thing to do rather than less.
• 22:23
  SPEAKER_01: I mean, it scares me that like most of the LLMs that we use, the free ones, are like get the first answer from Stack Overflow.
• 22:30
  SPEAKER_01: Whereas like as an as a senior engineer, you know the sixth or seventh answer was the right one.
• 22:35
  SPEAKER_01: The first one was the one that showed the how but not the why.
• 22:39
  SPEAKER_01: And we we have these tools.
• 22:41
  SPEAKER_01: I mean, the first LLMs that I used were code generators and like make the button blue and it started from scratch again, downloaded all the NPM modules.
• 22:49
  SPEAKER_01: Now we got modules that are more uh uh that are more modular, that basically just change the code as necessary.
• 22:55
  SPEAKER_01: So we're getting in a space where a conversation with an agent is like a conversation with a junior developer or with a with a pair programmer.
• 23:03
  SPEAKER_01: Do you think it will be that we have tools that optimize themselves and create the bash script, for example, automatically for us?
• 23:12
  SPEAKER_00: This is an area where I don't think that's far off, mainly because when you look at the what your compiler does, the code optimization that it does to run on a piece of hardware effectively, we kind of get into that domain already.
• 23:25
  SPEAKER_00: Sometimes your compiler may optimize part of your code out of it because it's unnecessary.
• 23:30
  SPEAKER_00: And most of us have grown to accept it.
• 23:32
  SPEAKER_00: Most people do not look at the generated output from their compiler, they just trust it.
• 23:37
  SPEAKER_00: But those compilers are built with a lot of intentionality, sometimes formal verification if they can if they can pull it off.
• 23:42
  SPEAKER_00: But the part that I actually want to address here is the part around it's like talking to a junior engineer.
• 23:48
  SPEAKER_00: And I think the unfortunate thing that I hope gets reversed because of all this productivity.
• 23:53
  SPEAKER_00: Talking to a junior engineer should not be about low-level syntax issues.
• 23:59
  SPEAKER_00: Talking to a junior engineer should be about things that a computer can't do.
• 24:03
  SPEAKER_00: The thing that you do when you get to a whiteboard and you reason about the customer base.
• 24:07
  SPEAKER_00: The part where you actually go and visit the customer and watch how they actually work.
• 24:12
  SPEAKER_00: And then you use wisdom, not just intelligence, to observe and honor their experience.
• 24:18
  SPEAKER_00: You capture all of those things and you revisit the whiteboard and ask yourself if there's a million ways of doing this, which one should we do?
• 24:26
  SPEAKER_00: And that's still to me the type of thing that a human should be much better than a computer app.
• 24:32
  SPEAKER_00: Can a computer mimic this behavior?
• 24:34
  SPEAKER_00: Absolutely.
• 24:34
  SPEAKER_00: So can a magic eight ball.
• 24:36
  SPEAKER_00: But I think you should try to optimize for the type of work that a computer can't do.
• 24:41
  SPEAKER_00: And so, yeah, if it can generate the bash script and run it in the background, okay.
• 24:46
  SPEAKER_00: Who cares?
• 24:47
  SPEAKER_00: But I hope that this productivity gives you all your time back, and I hope you all get raises for being 10x engineers now.
• 24:53
  SPEAKER_00: But I also hope you find the time to actually get to talk to each other again, and hopefully all of you get to experience what it's like to actually collaborate with people versus sitting in front of your computer hitting enter.
• 25:05
  SPEAKER_01: Which brings me to this question.
• 25:06
  SPEAKER_01: AI has just made me an angrier engineer, losing some of the joy in programming.
• 25:11
  SPEAKER_01: How do we keep that joy alive, or maybe are we beyond that phase?
• 25:15
  SPEAKER_01: Is angry our standard now?
• 25:16
  SPEAKER_01: Is that what you're saying?
• 25:17
  SPEAKER_00: No, just redefine it now.
• 25:19
  SPEAKER_00: So we've optimized work to what a person can do with the computer.
• 25:23
  SPEAKER_00: Now you have to change it.
• 25:24
  SPEAKER_00: Now people's gonna have to offer new ideas.
• 25:26
  SPEAKER_00: So complaining is one way to express feedback, no problem.
• 25:29
  SPEAKER_00: I can't fault you for that.
• 25:31
  SPEAKER_00: But what the culture should be should get redefined.
• 25:34
  SPEAKER_00: I remember when this agile methodology took over software engineering.
• 25:38
  SPEAKER_00: People were walking around with black belts but didn't know karate.
• 25:42
  SPEAKER_00: And then it became this cargo culted thing.
• 25:44
  SPEAKER_00: There's gonna be space there to redefine what engineering culture should be, especially if you're not gonna optimize it for taking Jira tickets and turning it into syntax.
• 25:54
  SPEAKER_00: If that's not gonna be the job anymore, and I'm assuming we still want engineers, what will they be doing?
• 26:00
  SPEAKER_00: So I think at some point, maybe this will be something that you do hand off to the machines.
• 26:04
  SPEAKER_00: You hand off a lot to the machines, like routing packets.
• 26:08
  SPEAKER_00: That stuff has already been delegated, and I think it's one of these things where I can understand the allergic reaction.
• 26:13
  SPEAKER_00: You spent a lot of time developing these skills.
• 26:16
  SPEAKER_00: You feel like a crafts person every time you walk up to that keyboard and format that code perfectly.
• 26:22
  SPEAKER_00: But that day may go away.
• 26:23
  SPEAKER_00: I'm pretty sure the person that used to neatly stack the punch cards had a system, and then that went away.
• 26:30
  SPEAKER_00: I'm not necessarily sold on it, but I do know how to deal with change.
• 26:34
  SPEAKER_00: And so for me, if I was in that situation, I would probably try to redefine the culture, give it a name so that other people can express it and practice it as well.
• 26:44
  SPEAKER_01: I'm actually happy that we have an opportunity for product managers to create click-throughs and MVPs that I as a developer don't have to write.
• 26:51
  SPEAKER_01: Because we went through the whole iteration of like until it needs to be productified, and why should that not be generated by an agent and somebody who's not a programmer?
• 27:00
  SPEAKER_01: I think that's just the danger that we think we're not in control anymore.
• 27:03
  SPEAKER_01: And maybe that's letting go is a good idea there.
• 27:06
  SPEAKER_00: Well, I will say this though.
• 27:22
  SPEAKER_00: It gets even worse when someone makes a decision on where to live or who to love.
• 27:27
  SPEAKER_00: This is this is insane.
• 27:29
  SPEAKER_00: And imagine when you start to delegate this to the judges, the lawyers, the teachers, the politicians.
• 27:35
  SPEAKER_00: If they're just sitting in front of a computer doing exactly the thing that you're doing, this is a problem.
• 27:42
  SPEAKER_00: This is why we are judged by our peers, not judged by computers.
• 27:46
  SPEAKER_00: And so that's the part that I'm a little worried about.
• 27:49
  SPEAKER_00: It's that judgment and decision making.
• 27:51
  SPEAKER_00: Remember, that's the whole point between the intelligence and wisdom piece.
• 27:56
  SPEAKER_00: It's the thing where it sounds good until you try it.
• 28:00
  SPEAKER_00: And then once we try it, we encode in our laws or in our practices that's not something you can do or should do because it's dangerous.
• 28:10
  SPEAKER_00: And that's the part that I hope we don't lose.
• 28:12
  SPEAKER_00: That's the part that makes us human and the reason why we all also have our own agency.
• 28:17
  SPEAKER_00: I am not looking for a cloud to design my life and to predict my future, but I don't care if it decides to help me along the way.
• 28:24
  SPEAKER_01: Yeah.
• 28:25
  SPEAKER_01: I mean, I I love when you talked about the malware thing, the amount of malware phishing attempts I had, like, here's some code, copy that into your terminal, and you're like, nah, not gonna happen, but then like, oh, it came from ChatGPT.
• 28:36
  SPEAKER_01: It's probably safe to copy in, right?
• 28:38
  SPEAKER_00: This seriously, I remember when I started my career, software that behaved in unpredictable ways, we called it malware.
• 28:47
  SPEAKER_00: We used to install a Norton antivirus on your computer, and it will watch for anything misbehaving, and then quarantine it unless you were being explicit about what it could do.
• 28:56
  SPEAKER_00: And that was like this thing, and then we had this whole thing around zero trust and this whole discipline around removing implicit trust from the system until we were being explicit.
• 29:07
  SPEAKER_00: And all of a sudden, there's a new thing that comes with some really cool capabilities.
• 29:11
  SPEAKER_00: I'm never gonna undermine it.
• 29:12
  SPEAKER_00: But boy, does it create a scenario that we have never practiced for in the last 30 years of software development.
• 29:18
  SPEAKER_00: That you have a thing that is designed to behave in many ways on its own terms.
• 29:24
  SPEAKER_00: And so now we're creating this illusion that we're gonna put it in the sandbox and we'll be fine.
• 29:29
  SPEAKER_00: You can do a lot of destruction from a sandbox.
• 29:32
  SPEAKER_00: You can steal all the data you want, you can affect other machines.
• 29:35
  SPEAKER_00: Sure, you can't copy files from the operating system underneath you, but you can do everything else.
• 29:40
  SPEAKER_00: So I think this is where we should probably be super careful with putting something that has no way to reason about its behavior and treat it as functionality that should become standard practice.
• 29:54
  SPEAKER_01: Um yeah, that's an interesting one.
• 29:55
  SPEAKER_01: In this AI era, uh, what's the role of senior leadership according to you, and what value does leadership offer?
• 30:01
  SPEAKER_01: Is the player coach role that's getting traction the only relevant role for current leaders?
• 30:06
  SPEAKER_00: I think a lot of leaders are senior leaders, senior directors, and junior humans.
• 30:11
  SPEAKER_00: Most of them are not really good at human development.
• 30:13
  SPEAKER_00: They've optimized for the wrong thing.
• 30:15
  SPEAKER_00: So I hope the senior leaders learn to become senior humans and everything gets a little bit better.
• 30:25
  SPEAKER_01: I agree.
• 30:26
  SPEAKER_01: I like humans.
• 30:28
  SPEAKER_01: Dogs are better, but humans are great too.
• 30:30
  SPEAKER_01: Thank you very much, Kelsey.
• 30:31
  SPEAKER_01: It was always fun to have you.
• 30:33
  SPEAKER_01: Thank you.
• 30:41
  SPEAKER_01: Good.
• 30:42
  SPEAKER_01: Another 10 minute break, and then it's gonna be the one and only Angie Jones up here.
• 30:47
  SPEAKER_01: So come back and make sure you have her because it's always an experience.
• 30:51
  SPEAKER_01: See you in 10 minutes.