Latest / Humanity Working / Lessons from an AI Startup

Transcript

Lessons from an AI Startup · Humanity Working

0:09
SPEAKER_02: When when you're innovating something, you're coming up with a better way to do something, and the deck is stacked against you because the world has a way that it worked before you, and you're coming in and you're trying to convince people to have a way to do things to change behavior.
0:25
SPEAKER_00: Alright, I'm here today with Zach Ratner.
0:28
SPEAKER_00: Zach is the author of the new book Grow Up Fast: Lessons from an AI Startup, which I think I've just about got in shot here.
0:37
SPEAKER_00: There we are.
0:39
SPEAKER_00: And uh we'll get into the details of that book a bit.
0:42
SPEAKER_00: Zach's pretty technical.
0:44
SPEAKER_00: He's got, I think, 18 Latin red and counting patents.
0:49
SPEAKER_00: Um you know that bit how your phone keeps service as you go down a freeway at 80 miles an hour?
0:54
SPEAKER_00: Well, Zach's at least partly responsible for that.
0:57
SPEAKER_00: If you've got a battery-powered hairdryer, you should be thanking Zach for that.
1:02
SPEAKER_00: Uh and today he is the co-founder and CTO of a company called Yembo.ai.
1:09
SPEAKER_00: And you can probably tell from the.ai that it's an AI company.
1:12
SPEAKER_00: They create AI-based virtual home surveys.
1:15
SPEAKER_00: Now, I'm really interested in this because uh Yembo is an AI company from the start.
1:20
SPEAKER_00: So I think that gives Zach a really interesting perspective on where AI is at and where it's going and what it means for employers and employees.
1:28
SPEAKER_00: So welcome to the podcast, Zach.
1:30
SPEAKER_02: Hi, Paul.
1:31
SPEAKER_02: Thanks for having me.
1:32
SPEAKER_02: Happy to be here.
1:34
SPEAKER_00: So, first, a little bit about the book.
1:37
SPEAKER_00: Um, couple of things.
1:39
SPEAKER_00: One, physically beautiful, by the way.
1:41
SPEAKER_00: So uh I understand from just talking to you now that that uh that the design of this of this thing is is thanks to Yembo, uh Yembo employee as well.
1:52
SPEAKER_00: Um But it's it's it's gorgeous to hold, gorgeous to pick up, and really interesting once you get uh once you get inside it.
1:59
SPEAKER_00: Um I was struck by the fact it's almost like in a way, it's almost like three books in one.
2:05
SPEAKER_00: There's really interesting biography in there.
2:08
SPEAKER_00: Um there's what I call almost like in the trenches guidance from a tech-focused startup.
2:14
SPEAKER_00: Um I run one and I was like, oh yeah, I remember that bit, I remember that bit.
2:18
SPEAKER_00: And then also, oh, I can do this differently, I can do that differently, which is great.
2:22
SPEAKER_00: Um and I think the one thing I would add as well is that um it would also apply really well to anybody that is in an innovation team inside a larger organization.
2:36
SPEAKER_00: Um you actually did that work at uh at Qualcomm for a period, and then you can kind of you can kind of tell that.
2:43
SPEAKER_00: Um and then there's a third element, which is uh really, if you like, a brief history of AI and and the thoughts on how you approach it.
2:52
SPEAKER_00: So I want to kind of start with that third area.
2:56
SPEAKER_00: Does that sound okay?
2:58
SPEAKER_00: Works for me.
2:59
SPEAKER_00: All right.
3:00
SPEAKER_00: So I think what's what's really interesting is that for many people, the story of AI I think goes something like this.
3:10
SPEAKER_00: It's science fiction, followed by almost out of nowhere, Deep Blue beating Kasparov, if you're old enough to remember that, a chess.
3:19
SPEAKER_00: Then there was that Jeopardy episode, and then uh uh and then suddenly Chat GPT, right?
3:26
SPEAKER_00: And so and when ChatGPT happened, all of a sudden it was like, oh, this is real and it's here, and it's not just a uh fancy commercial from IBM, it's like in and around us and among us.
3:39
SPEAKER_00: Um but in your book, particularly in your chapter around machine vision, you're focusing on a different piece of that history.
3:47
SPEAKER_00: And it's a part I remember pretty well.
3:50
SPEAKER_00: Um talk to me about um that period when uh when deep learning started to become something that was uh um that was being used more extensively, and then also what that meant for you personally in the field of machine vision.
4:08
SPEAKER_02: Sure.
4:08
SPEAKER_02: So if you wind back the clock, it is uh about 2015, and there's a popular academic benchmark, um, a competition that happens each year that uh I believe you're at Microsoft when Microsoft won it.
4:23
SPEAKER_02: So congratulations.
4:24
SPEAKER_02: Um and um, if you're not familiar with it, you can think of this like a thousand-way multiple choice test.
4:30
SPEAKER_02: So if I hold up a picture and I say, Hi, Paul, what's in here?
4:33
SPEAKER_02: Oh, that is a cat, that is a dog, that is a sofa.
4:37
SPEAKER_02: Um, it's a task that's relatively easy for people to do, but it can be quite complicated for computers because computers don't have concepts of what a thing is, they just see pixels.
4:46
SPEAKER_02: And then also most photos don't have just one thing in it.
4:50
SPEAKER_02: So you may have foreground and background, and a human can pretty easily tell like what is what is this of, but it's really hard to write like an algorithm around that.
4:59
SPEAKER_02: And the traditional way to do this was you would um look at the pixels, you would maybe put it in grayscale, make algorithms looking at edges and lines and things like that.
5:09
SPEAKER_02: And if you wanted to detect maybe a chair, you'd say, okay, a chair is going to have four legs and a seat and a back.
5:15
SPEAKER_02: And it was it worked kind of um with uh 70-ish percent accuracy in a lot of cases.
5:22
SPEAKER_02: But um, it was a ton of effort and didn't generalize very well.
5:26
SPEAKER_02: So if you made that great chair detector, well, I could rotate the chair 45 degrees, 90 degrees, and um, in a lot of cases break some of these algorithms.
5:34
SPEAKER_02: So what was happening was uh this concept called deep learning, which had been around for a while, but kind of um hadn't been applied in this way yet.
5:43
SPEAKER_02: And deep learning is biologically inspired.
5:46
SPEAKER_02: So if you look at what's going on, it's not like a carbon copy of what um a mammalian brain would do, but it borrows some concepts.
5:54
SPEAKER_02: And um the key difference that made it really powerful was the engineer no longer needed to manually hand code what those features were.
6:02
SPEAKER_02: You could just give a bunch of examples and say, these are chairs, these are cats, these are dogs, and the algorithms would learn over time by um going through the um going through the training process, and out would come a detector.
6:15
SPEAKER_02: And that was um, you have to think, remember what made this so powerful at that point in time was there were so many different things happening at once.
6:23
SPEAKER_02: So the math behind that had been known for some time, but um the compute, cost of compute was coming down with Moore's Law.
6:32
SPEAKER_02: Um, thanks to companies like Facebook and Twitter, we're uploading all of our data into the cloud.
6:37
SPEAKER_02: So we've got these companies with huge reams of data.
6:41
SPEAKER_02: The cost of compute is coming lower, lower, lower.
6:44
SPEAKER_02: And um these algorithms that have been theoretically possible for a while, but I would say practically challenging, it'd be hard to get like 10 million cat photos in 1993.
6:53
SPEAKER_02: It'd be easier to get it in 2013.
6:55
SPEAKER_02: Um, so what was happening was uh these algorithms took over and this benchmark where people were comparing um the image classification task and um the traditional pieces.
7:09
SPEAKER_02: If you looked at like the there's a competition called ImageNet, it's run every single year.
7:14
SPEAKER_02: And before deep learning came around, we were looking at like um 1%-ish um accuracy improvements year over year.
7:24
SPEAKER_02: And then deep learning comes along, fact uh it's like factor of 10, 10 years of progress, boom, in one year somebody comes along and wins.
7:30
SPEAKER_02: And what happened in the 2015 competition when Microsoft won is there was a benchmark that said humans are about 95% accurate at this task.
7:39
SPEAKER_02: Um, and then the submission comes along, and for the first time in human history, that threshold is breached by an algorithm, not by a person.
7:48
SPEAKER_02: So this is making waves in a relatively niche corner of um industry and academia, but the effects are profound.
7:56
SPEAKER_02: It means that computers are now better than people at identifying objects and images.
8:01
SPEAKER_02: And if you looked at the companies that were trying to utilize that and take that technology to certain markets, it appeared to me like it was very concentrated.
8:10
SPEAKER_02: Everyone wanted to make the next self-driving car company and a lot of um drone last mile delivery companies.
8:16
SPEAKER_02: But there was so much on potential that was just not really being looked at.
8:21
SPEAKER_02: Like computers are better than people at identifying objects and images, and it seems like nobody knows.
8:26
SPEAKER_02: Um and that's kind of one what ended up um having us take that core technology and bring it into the home service space.
8:34
SPEAKER_02: It's an area where people generally are not known for their high-tech usage.
8:38
SPEAKER_02: Um, but they have very real um workflow issues because it's hard to um drive to somebody's house and identify what they have to give them an accurate moving quote.
8:48
SPEAKER_02: Um, if you want to do things like uh construction estimates or um painting, junk removal, like any anytime someone needs to come to your house to give a quote, it's um kind of ripe for innovation where somebody could come along and accelerate that process and make a better customer experience, more accurate, um, fewer, fewer errors on both sides of it.
9:08
SPEAKER_02: So we wanted to um kind of go in a direction that we felt no one was really looking at yet.
9:14
SPEAKER_02: And by the time we had built out a business there, it would be sort of um like too late for someone else to come along because we've already um kind of established ourselves in that space.
9:24
SPEAKER_02: But there was this core technical innovation, and I wasn't really um like responsible for that spark.
9:31
SPEAKER_02: But I feel like um as someone who works in innovation, I'm sure you've seen this too, that innovators generally sit at the intersections of two things.
9:39
SPEAKER_02: And you don't usually need to be an expert in one of them, but being at that intersection, and this is happening in the AI space, this is happening in the home service space, and kind of positioning yourself at the intersection of the two is kind of like what made the the Yembo story possible.
9:55
SPEAKER_00: Well, that's actually one of the more exciting aspects of all of this, because I mean, to be clear, uh based on my understanding from your book, um here you were at that time not a deep expert in AI, but somebody who was interested by what was going on and was observing it and understood the the uh potential of it, but not not somebody who was deep into it.
10:22
SPEAKER_00: And almost equally interesting, nor were you somebody who who had 20 years of experience or background in home removals or or anything along those lines.
10:38
SPEAKER_00: What you did have was a connection into each of those things.
10:43
SPEAKER_00: And you can explain the second connection in a moment here, but the um but you had a connection into both of those things and the insight to see what uh to see why it was valuable, why it was valuable to create that intersection between those two things.
10:59
SPEAKER_00: So can you talk about that a little bit?
11:01
SPEAKER_02: Sure, yeah.
11:02
SPEAKER_02: So the the other the other side of the story that uh that you hinted at was my wife was working at a moving company.
11:07
SPEAKER_02: So by day I study deep learning, by night I come home, and my wife tells me about the just the operational work it takes uh to make a successful move.
11:16
SPEAKER_02: And she was working in an international department, which is uh pretty logistically complicated because not only do you have the packing of things into boxes, you also have customs forms and steamship lines and time zones and translations and all these things that just a local move wouldn't have.
11:31
SPEAKER_02: Um but I think the key is, like you had mentioned, I wouldn't call myself an expert in any one of those things.
11:38
SPEAKER_02: If you go to an AI conference, uh just go to C VPR, go to NeuroLibs, you'll see people that know more than I do.
11:45
SPEAKER_02: And if you go to a moving and storage convention, you'll see people that have been operating successful, profitable relocation businesses, and uh they they know more than I do.
11:56
SPEAKER_02: But I feel like um you kind of touched on this in the beginning when you mentioned my biography, um, someone who has patents in cellular protocols and wireless hairdryers and computer vision, that your capability to apply value to an area is not dependent on being like the smartest person in the room on that one area.
12:15
SPEAKER_02: I think the ability to learn, know what kinds of problems a piece of technology can and cannot do, and then understanding the real customer need is really what's valuable.
12:25
SPEAKER_02: And it's okay if there are other people out there that can um push the state of the art a little bit further or who understand um how the math works better.
12:34
SPEAKER_02: But I feel like when when you're innovating something, you're coming up with a better way to do something, and the deck is stacked against you because the world has a way that it worked before you, and you're coming in and you're trying to convince people that have a way to do things to change behavior.
12:50
SPEAKER_02: So you don't have a credential that you can come in and say, Oh, I have a PhD, so therefore you need to listen to me.
12:58
SPEAKER_02: You have to say, this way of doing things is 10 times better, and that's why it's worth doing.
13:02
SPEAKER_02: That you have to kind of um show me the goods.
13:05
SPEAKER_02: You can't just come in and uh and demand attention because of your accolades.
13:10
SPEAKER_02: And I feel like what that meant for us in the moving industry is we had to find this problem.
13:18
SPEAKER_02: We knew that AI could solve the accurate object detection issue, but there's a whole host of other things that you need to solve also to make that actually practical and useful to somebody.
13:28
SPEAKER_02: So the first off would be um, is the information actually there?
13:31
SPEAKER_02: If you need to go open up every drawer behind here and you can't give an accurate estimate without that, then who cares if you detected that there was a drawer, right?
13:38
SPEAKER_02: So we had to actually go through and say, hey, I'm gonna make some software.
13:42
SPEAKER_02: There was no AI in the very first version.
13:44
SPEAKER_02: And we said, I want to take a trained person, I'll save them the drive.
13:47
SPEAKER_02: They don't have to drive to the person's house, and uh, I'll give them the software for free or near free.
13:52
SPEAKER_02: And I'll ask them, can you let me know how accurate these jobs are?
13:56
SPEAKER_02: Like if you do them and you say, I can't, I can't do it, the information's just not there.
14:00
SPEAKER_02: And no amount of amazing technology can can solve for that.
14:04
SPEAKER_02: We had an early move too where somebody had actually already disconnected their power.
14:09
SPEAKER_02: Um they had they had uh they had relocated, but they left a lot of their items behind, and they were getting the move after the uh the power had been cut.
14:16
SPEAKER_02: So we get all these videos taken 8, 30, 9 o'clock at night, and they're just black.
14:21
SPEAKER_02: Uh you can hear someone shuffling around, and um the AI detects nothing because it's it's dark.
14:28
SPEAKER_02: And uh we mentioned that and uh we tell the we tell the mover, like, sorry, I have nothing for you here.
14:33
SPEAKER_02: And they said, Well, I thought you said the AI detects items.
14:35
SPEAKER_02: Um and that's when we realized the value of understanding like you are sort of like um like a messenger.
14:43
SPEAKER_02: You're you're bringing the the technology into this new space, but it's up to you to kind of frame it the right way.
14:50
SPEAKER_02: And don't put the customer in a position where they're depending on you in a way that you can't service them.
14:56
SPEAKER_02: So then we we built the lightness detector, and when you turn on the camera, we'll say, hey, turn on the lights.
15:00
SPEAKER_02: Like there is a technical solution to that.
15:02
SPEAKER_02: But I think the the thought behind it was you are responsible for the workflow.
15:08
SPEAKER_02: You're responsible for the output that's going to come out of this.
15:11
SPEAKER_02: And the AI is like a shiny, snazzy like way to get the job done, but it doesn't stop at the AI, that it stops at um the person wants to use the product because it's a better customer experience, because they're able to quote more jobs in a day, they can expand their business, focus more on building rapport with new clients and less on like tallying up carton counts and things like that.
15:32
SPEAKER_02: But it really came down to like understanding the value that we bring to the end user.
15:37
SPEAKER_02: And if you go and you talk about model architectures and how many coup de cores and all these like different uh technical details that engineers get really excited about, it's just um just irrelevant to the end user.
15:48
SPEAKER_02: So I feel like understanding like the real problem that you're trying to solve and then reaching into that tool belt and using the right tool for the job is really key.
15:58
SPEAKER_00: There's a story in the book that highlights, I think, uh, how how differently AI in quotes thinks to the way a human approaches this.
16:10
SPEAKER_00: And and therefore kind of how difficult the challenge is to get from let's say 70% good to a hundred percent uh a hundred percent accurate.
16:21
SPEAKER_00: Um I'm talking about the story where you uh around monitors and um and TVs.
16:27
SPEAKER_00: Because I think a lot of people with a rudimentary understanding of this would be like, well, just give it more pictures, give it more pictures, and and then everything will get better.
16:35
SPEAKER_00: Can you talk a little bit about what happened when you tr when you were trying to add information to your model in order to be able to uh to differentiate between those two?
16:48
SPEAKER_02: Sure.
16:48
SPEAKER_02: Yeah, I think this is a good example of um like understanding the limits of of the technology.
16:54
SPEAKER_02: So the thing moves so fast people think, oh, AI can do anything, but it's an algorithm and uh it has limitations.
17:00
SPEAKER_02: So imagine you have AI where someone scans video, quick 20-second room of a video of each room in their house, and we're gonna identify draw boxes around.
17:11
SPEAKER_02: Um here is a TV, here's a sofa, here's a lamp, and that goes into the moving quote.
17:16
SPEAKER_02: So this TV behind me is a good example.
17:19
SPEAKER_02: So our initial AI, we were pretty good at detecting these because a lot of people have them.
17:24
SPEAKER_02: But then we were expanding and we wanted to add the computer monitor concept because um sizes are different, they usually get packed a little bit differently.
17:33
SPEAKER_02: So even though they're big black rectangles, if you're a computer vision algorithm, if you're a mover, that distinction really matters.
17:39
SPEAKER_02: But now the problem is from the computer vision standpoint, they look very similar.
17:44
SPEAKER_02: It's really hard to tell a big black rectangle when you have two different labels you're assigning to it.
17:49
SPEAKER_02: So it kept getting confused.
17:51
SPEAKER_02: So in the earlier days, we were very close to perfect on TVs because if you look, my the one behind me is a good example.
17:59
SPEAKER_02: There's very high contrast, it's really easy to see.
18:02
SPEAKER_02: Um, so it's they're not usually like covered by clutter or things like that.
18:06
SPEAKER_02: And every time we saw it, the AI only knew one thing for it: TV.
18:10
SPEAKER_02: So we got it right.
18:11
SPEAKER_02: Then we bring in this concept of computer monitors, and now yeah, we get we stop calling the thing on the desk a TV, but then you also start calling this computer monitor because it's looking at like we talked before about the chairs, uh, the concept of like detecting the lines and all these things, they have very similar features.
18:31
SPEAKER_02: So we improved the AI, and then the overall accuracy went down.
18:36
SPEAKER_02: So then what's a what's a practitioner to do?
18:39
SPEAKER_02: I called this um the common sense engine.
18:42
SPEAKER_02: My AI team told me it was an ugly hack, but uh it worked.
18:46
SPEAKER_02: And what we had to end up doing was we would um look at the context around it and said, okay, if it's going to be hard to tell the TV itself because it's just a big black rectangle, let's look at um what are things that are around it.
18:58
SPEAKER_02: So we ended up having to detect fireplaces.
19:00
SPEAKER_02: No one ever moves a fireplace.
19:02
SPEAKER_02: But if you see a big black rectangle over a fireplace, then it's probably not a computer monitor.
19:07
SPEAKER_02: And if you see a big black rectangle on a desk, it's probably not a TV.
19:10
SPEAKER_02: And we've had to train based on all these examples, and some of them it was right most of the time.
19:14
SPEAKER_02: You had to kind of tweak it.
19:16
SPEAKER_02: But the idea being that there was this core AI technology that made everything possible, but it wasn't like a panacea that made everything work.
19:25
SPEAKER_02: You still had to go and understand the customers' needs and um kind of patch together these um common sense or ugly hacks, depending on how you look at it, and um come up with something that actually addresses the real customer issue.
19:38
SPEAKER_02: We also had a similar one too with um on trash day.
19:42
SPEAKER_02: We had a particular move that was like really, I think it was like a 4,000-pound move, which is typical for like a one or two bedroom home.
19:49
SPEAKER_02: And we called it like 8,000 or something way, way, way off.
19:52
SPEAKER_02: And it turns out uh we're debugging, customer mentioned it was way off.
19:55
SPEAKER_02: We're looking, bedroom looks fine, kitchen looks fine, like living room looks looks way off.
20:01
SPEAKER_02: Turns out it was trash day, and uh you look out the window on the first floor, you see trash can, trash can, trash can, trash can from all the neighbors down the street.
20:08
SPEAKER_02: Um, so we actually had to train the AI on, hey, this is a window, and if you see things through it, um then it's likely to like not actually be in that room.
20:16
SPEAKER_02: And again, that also has uh trade-offs and edge cases, and sometimes if there's like a cutout in a room, you do want to look at it.
20:23
SPEAKER_02: So we had to kind of um continuously improve based on the actual um problem that we were trying to solve, that there wasn't like a pure science answer to these kinds of things, but um, you had to kind of uh understand how to teach the AI to think and behave like a human would in that domain.
20:45
SPEAKER_02: And if you're building a self-driving car, you would be a fool to ignore trash cans because you might hit them.
20:50
SPEAKER_02: But in like in our use case, in that case, you wanted to.
20:53
SPEAKER_02: So that's where there's not like a one-size-fits-fits-all kind of solution.
20:56
SPEAKER_02: That real innovation is really like understanding the end problem you're trying to go through and building out things that uh that get you to that goal.
21:06
SPEAKER_01: Hey there.
21:07
SPEAKER_01: Glad you're still here.
21:09
SPEAKER_01: Even though this in every episode of Humanity Working is really interesting, science tells us that almost nobody can remain fully focused for more than 20 to 25 minutes.
21:17
SPEAKER_01: We don't want you to miss the important stuff.
21:19
SPEAKER_01: So here's a tiny little break for your brain.
21:21
SPEAKER_01: Oh, and while you're taking this break, here's another fact for you.
21:25
SPEAKER_01: If you do a one-day course on soft skills training in your organization, 70% of the information is lost in a week, and 90% in a month.
21:33
SPEAKER_01: That's right, only 10% of the information remains, and it's probably the most useless stuff, like the way your instructor kept sucking on his whiteboard marker and probably swallowed ink.
21:43
SPEAKER_01: Billion Minds knows that.
21:44
SPEAKER_01: So they've designed their soft skills development with one aim only to make sure that learning translates to behaviors that make your organization thrive.
21:52
SPEAKER_01: Every experience in Billion Minds is a learn do exercise of ten minutes or less.
21:56
SPEAKER_01: But by the end of each one, employees have tangible results they can bring to work.
22:00
unknown: Work.
22:00
SPEAKER_01: Not only that, but Billion Minds helps you as an organization optimize your infrastructure, policy, and culture to make sure that individual learning translates to your organization thriving.
22:10
SPEAKER_01: As a humanity working listener, you can get free access to our guide on how to build a culture of learning in your organization.
22:17
SPEAKER_01: It explains what works and what doesn't work based on years of research into how people actually develop in organizations.
22:23
SPEAKER_01: Download it today at billionminds.com/slash humanity working.
22:27
SPEAKER_01: All right, back to the episode.
22:31
SPEAKER_00: Yeah, I've been speaking to some folks recently about the self, you know, the fully autonomous self-driving environments and and I think one of the really interesting aspects of that as a challenge.
22:44
SPEAKER_00: Aside from all of the ethical things, you've probably, you know, seen and read about the idea of like a okay, if a car has got a split second decision as to whether it saves the driver or saves the passenger or saves the pedestrian, all those uh really challenging ethical considerations.
23:00
SPEAKER_00: One of the other challenges um is that we're going through a period right now whereby you've got uh humans driving cars as humans drive, and you've got AI driving cars as AI drives.
23:16
SPEAKER_00: Um and you spend any time in uh you know playing with the FSD capability on a Tesla, for example, it i these days it can do like it can it can do its job fairly well, but the ways in which it goes wrong are very counterintuitive to a human.
23:35
SPEAKER_00: Um and so of course there are other humans on the road that are now having to interact with or deal with the fact that there are other cars on the road that are behaving in a relatively counterintuitive way.
23:48
SPEAKER_00: Now, of course, humans are adaptable, we'll be aware we will we will figure that out.
23:51
SPEAKER_00: But it's really interesting when you kind of like combine in the same ecosystem humans that tend to get things wrong in very human ways and AI that tends to get things wrong in ways that to humans do not seem intuitive.
24:06
SPEAKER_00: Um I wanna I wanna move on to like the thing that everybody thinks of, or not everybody, but everybody who's not in this field tends to think of when they think of AI now, which is Chat GPT.
24:18
SPEAKER_00: And you know, the various other uh there are other LLMs available, but anyway, but um but um today the conversation really is all about when it comes to knowledge workers is all about prompt engineering, or in other words, how you ask AI the right questions.
24:37
SPEAKER_00: Now it's not something you talk about directly uh really in the book, but I did find myself wondering um how as an AI company you use technologies like that, and do you use it in a different way to to in quote regular folks?
24:54
SPEAKER_00: Um but I'd really love to get your views on how employees um who are just trying to be efficient and effective every day, how they can and should use AI almost like as an assistant to do that really well.
25:11
SPEAKER_02: Sure.
25:11
SPEAKER_02: Yeah.
25:11
SPEAKER_02: And I think when we started Yembo, and anytime we brought up AI, it felt like Terminator was the thing we're always being compared to.
25:19
SPEAKER_02: So now when we mention AI, people talk about ChatGPT.
25:21
SPEAKER_02: It feels like a step in the right direction because we talked about productivity and not the end of the world.
25:26
SPEAKER_02: Um but yeah, I think uh it's certainly powerful.
25:29
SPEAKER_02: I think one thing OpenAI nailed was the user interface, is um I didn't have to learn anything new.
25:34
SPEAKER_02: I can text my friends, I can text uh sent Slack soft at work, and the that interface is it works the same way.
25:41
SPEAKER_02: Um but I mean, like any technology, I think it pays to kind of go one level under the hood, and like you mentioned, they're LLMs.
25:49
SPEAKER_02: That means that they're um trained on a large corpus of data, and that means they do certain things well and certain things not well.
25:57
SPEAKER_02: Um I have found I probably use it 10 times a day for random small things here and there.
26:03
SPEAKER_02: Um I wouldn't say it's like a person.
26:06
SPEAKER_02: Like if I um combined all the work that I'm using Chat GPT for in a given day, I don't think there's like a job title that it's replacing.
26:15
SPEAKER_02: I think it's replacing the like um annoying interruption, like uh, hey, you got a minute, can you help me with this Google Sheet?
26:21
SPEAKER_02: Can you help me with this uh uh function I'm trying to write in my code?
26:24
SPEAKER_02: Can you help me with um like brainstorm 10 ideas for that?
26:28
SPEAKER_02: It's kind of taken out the I would say um some of the drudgery kind of work, but it's not like it's been everything that I can do.
26:35
SPEAKER_02: It's like 2% of this person and 1% of that person, and then we're kind of freed up to do other things.
26:40
SPEAKER_02: That's kind of just the state of where I've seen it today.
26:43
SPEAKER_02: Um to give some practical examples, um it's relatively good if you use the uh I've got the GPT for I pay 20 bucks a month or whatever for it.
26:53
SPEAKER_02: And um when you um upload files, it can do a pretty good job pulling out some insights there.
27:01
SPEAKER_02: And I think that kind of use case is pretty good for language models because it's not really like a black and white right and wrong thing.
27:10
SPEAKER_02: I think the wrong kinds of questions to ask, because again, the things trained on like most of Reddit and Twitter.
27:15
SPEAKER_02: And if you go read Twitter, what percent of that is accurate?
27:18
SPEAKER_02: Like, good lucky in answering that question.
27:19
SPEAKER_02: So if you go and ask it like a history test, it's not it makes up facts and things like that.
27:25
SPEAKER_02: Um, but if you have a spreadsheet around um here access logs to the server, anything look anomalous?
27:32
SPEAKER_02: Um, it can go sift through large amounts of data and pull out, hey, you had like uh this happened yesterday, you had uh like 10 requests per second from an IP address in Russia.
27:42
SPEAKER_02: It's like good to know if someone's accessing your server trying to do that.
27:45
SPEAKER_02: Um my editor for my book was telling me he uploaded his uh bank statements and said, Hey, any subscriptions I should cancel.
27:52
SPEAKER_02: I'm trying to cut down on my budget, be mindful of my spend.
27:54
SPEAKER_02: Um so I think things like that where it's making suggestions as opposed to like giving you a black and white answer are are pretty good use cases.
28:03
SPEAKER_02: Um if you are technical, I've had um pretty good success with things that are really hard to use.
28:10
SPEAKER_02: Um, things like regular expressions, bash scripts where the syntax is kind of funky.
28:14
SPEAKER_02: A lot of people ask questions about those things on the internet, which makes the training data.
28:18
SPEAKER_02: So it's pretty good at um answering those kinds of things.
28:22
SPEAKER_02: The latest and greatest technology, it's not so good at because it's got this knowledge cutoff.
28:26
SPEAKER_02: So if you um if you want to go learn how to like write code on the Apple Vision Pro headset, they'll probably just say, What is Apple Vision Pro?
28:34
SPEAKER_02: Um, but I think overall, um treating it as like an assistant who can be super patient, look at large swaths of data, and also someone that you can maybe use as like a coach or a brainstorming partner, I think has also been kind of helpful in terms of um we just uh we just had a trade show and um like I asked it a couple ideas around um here's the point I want to get across.
29:00
SPEAKER_02: Do you have any ideas around like swag I should give away or what the promotion campaigns might look like around it?
29:05
SPEAKER_02: And again, we still had like people executing, but it was sort of like uh a good uh sparring partner um where I wouldn't just 100% offload and say, Chat GPT, please, please go run this campaign for me.
29:18
SPEAKER_02: But if you say um, like here's here's what we did last year, we want to change it up a little bit, want it to be interesting.
29:24
SPEAKER_02: Can you give me like 10 ideas?
29:26
SPEAKER_02: I think that kind of like human-assisted human-in-the-loop AI, where it's kind of more of a back and forth than a one and done kind of thing.
29:34
SPEAKER_02: I think those are all the use cases that I feel like are practical here and here and now and not really science fiction anymore.
29:41
SPEAKER_00: Yeah, I think um Ethan Moloch uh in one interview at one point said um it's a bit like an intern that lies a bit.
29:49
SPEAKER_00: And I think that that is sort of sort of right, but it's not the it's not the full um it's not the full um aspect of it.
29:58
SPEAKER_00: And it's probably kind of disrespectful to interns that we find incredibly valuable anyway inside our company.
30:03
SPEAKER_00: Um But um I think that what you're drawing out there is something that I've seen um in our use of it as well, which is just that idea of being able to like provide suggestions and think of it in that way as well, can be really helpful because it may well be providing suggestions that you just haven't thought uh thought of.
30:25
SPEAKER_00: And it also kind of like takes the pressure off the idea that it's accurate.
30:29
SPEAKER_00: And so that's the thing that I I think people are starting to get their heads around.
30:34
SPEAKER_00: I would say to people if you wanna if you want to understand its accuracy or not, then spend just a little bit of time asking it about things that you know really deeply.
30:44
SPEAKER_00: And if the moment that you do and and then you just have it's a very quick gymnastic mental bit of gymnastics to go, okay, well, if it's getting all that wrong, then it's probably getting a bunch of other stuff wrong.
30:55
SPEAKER_00: So I shouldn't be using it as a means of of like um everything it says here about something I don't know about now becomes what I know about.
31:03
SPEAKER_00: Um that's that's a a less useful um usage of it.
31:08
SPEAKER_00: We use it um in collective brainstorming sessions as well.
31:14
SPEAKER_00: So we will have you know, if we've got like two or three or four people that were brainstorming on something, rather than we started by like asking it up front and then having that guide our thinking, but we found that to be too restrictive to our thinking.
31:27
SPEAKER_00: So now we do it kind of simultaneously.
31:30
SPEAKER_00: So it we'll say I'll come up with you know six ideas about something or other, we'll ask ChatGPT at the same time, and then we'll prioritize based on all of those inputs.
31:39
SPEAKER_00: And often it comes up with stuff that we haven't even thought of as a team, so which is great.
31:42
SPEAKER_02: That's do you ever use it for critiquing?
31:44
SPEAKER_02: That's another use case that we found is good.
31:46
SPEAKER_02: Like here's an email, here's the thing I'm concerned about it being misinterpreted.
31:50
SPEAKER_02: Can you give me like three different rewrites?
31:52
SPEAKER_02: Give me different suggestions where people like take from the AI, but I think you can give to it and ask it to refine.
31:59
SPEAKER_02: And that has also been a use case we've had pretty good success with.
32:02
SPEAKER_00: Yeah.
32:03
SPEAKER_00: A good example of that would be I I took an example of like some of the writing that I had done, and then I got it to critique my writing style.
32:11
SPEAKER_00: And now whenever I ask it to like generate a first draft or something, I use the specific critiques that it gave of my writing style and feed that to it, and then it comes back with something that sounds more like me in the first place, which is kind of useful.
32:23
SPEAKER_02: That's that's pretty awesome.
32:24
SPEAKER_02: Yeah, just for fun, after we uh after we finish the book, I uh I put it on um, I took a couple of chapters and said, like, write one-star Amazon ratings just to kind of see like if you were to give me something critical, what would you put in there?
32:36
SPEAKER_02: I didn't do anything about it, but uh it's uh you can like brace yourself for when you put something out in the world and you don't know what's gonna come.
32:43
SPEAKER_02: You can kind of uh think through what might happen.
32:46
SPEAKER_00: Well, that actually is a terrific segue um to one of my last questions here.
32:52
SPEAKER_00: So um in the book, um, I mentioned that the book, right, kind of has these, if you like, three three elements like the biography piece, the sort of in the trenches guidance, and then the history of AI.
33:04
SPEAKER_00: Um, I want to delve very briefly into one other uh aspect of it, which is this kind of in the trenches guidance thing.
33:12
SPEAKER_00: Um you can think of this almost as like a guide um for things to watch out for if you're uh if you're building a capability inside an organization or if you're building a new organization that has tech at the center of it.
33:30
SPEAKER_00: And you've got some really interesting aspects as to how you go about uh capturing the voice of the customer, how you go about capturing sentiment um inside your company.
33:39
SPEAKER_00: And you have this one artifact that I loved and I wanted to highlight um called the gripe doc.
33:44
SPEAKER_00: So you already talked about one-star reviews, uh, which uh for GPT, which I think is great.
33:50
SPEAKER_00: But tell me about the gripe doc and how that works.
33:54
SPEAKER_02: Sure.
33:54
SPEAKER_02: So I'm actually an optimist.
33:55
SPEAKER_02: I think uh the the thing that I've learned though is the first version of most anything, book, software, product, you name it, is gonna have kinks, it's gonna have work that needs to be done.
34:08
SPEAKER_02: And um what the gripe doc does is it just leans into that and helps you make a better product.
34:13
SPEAKER_02: So the way it works is um it's a mindset thing you need to get into first.
34:18
SPEAKER_02: That's why I put the the ugly, nasty gripe name in the title.
34:21
SPEAKER_02: And the mindset is we just finished a minimum viable prototype.
34:27
SPEAKER_02: It may become a product, or it's like a barely enough to work that I can uh start showing it, start getting feedback from clients, which I feel like a lot of startups um handle that part well.
34:38
SPEAKER_02: Is um I think we got Eric Reese telling us build, measure, learn, lean startup.
34:42
SPEAKER_02: Uh people generally, I think, understand the idea of like you want to race to get something out there and then iterate based on feedback.
34:49
SPEAKER_02: But what people I think overlook is what happens when you come out of that.
34:53
SPEAKER_02: So you do that, and then what happens next?
34:55
SPEAKER_02: And what the GRIPE doc does is um we found it was kind of hard to get like honest feedback from clients because we would we'd had it, we had our base product, we had some features we were looking to add, and we wanted to get initial feedback before we rolled it out.
35:11
SPEAKER_02: So, what do you do?
35:12
SPEAKER_02: Is you find your customers who are more generous with their time, who will like be willing to get on a call with you, who are willing to try stuff out.
35:18
SPEAKER_02: And they're usually friendly.
35:20
SPEAKER_02: So if you have something that's got bugs that doesn't quite work, they may sugarcoat it, they may be nice.
35:26
SPEAKER_02: Um, and then that sounds nice on the surface, but it's actually preventing you from getting valuable feedback that you would need to solve to make the product viable.
35:34
SPEAKER_02: So, what the gripe doc does, it's a little bit jarring the first time you do it, but we gave the client access, we got on a demo, we showed them everything, and then we shared um you get an email, Zach Ratner has shared a Google Doc with you.
35:46
SPEAKER_02: This product name sucks because dot dot dot.
35:49
SPEAKER_02: And then there's an empty Google Doc with like a bullet point.
35:52
SPEAKER_02: And what that does is it's just setting the mindset around, I know this is rough.
35:56
SPEAKER_02: I have some things that I want to work on, I just want to get your feedback and a gut check around what do you feel like it's missing.
36:02
SPEAKER_02: And by making it so stark, just saying, I know it sucks, you're um inviting the person to give you honest feedback.
36:12
SPEAKER_02: Where if I say, uh, I've been working so hard on this, the whole team like stayed up all night for the last three weeks, I'm really excited to show it to you.
36:19
SPEAKER_02: Like, even if it sucks, like I'm telling you that like uh how hard I worked, and now you're gonna be compassionate and you're gonna like sugarcoat the feedback.
36:27
SPEAKER_02: And it's just kind of cuts around that.
36:29
SPEAKER_02: So, what we do is we we share those.
36:31
SPEAKER_02: We usually give like one per client so they don't see each other's feedback, then we bring it together, and then we have our own ideas as well, and you can kind of prioritize based on that.
36:41
SPEAKER_02: And what we found is by doing that, when you do finally like show it to the world and come out of beta, the feedback is much more refined.
36:50
SPEAKER_02: So I feel like people maybe dodge the negative feedback, but if you kind of seek it out or if you use ChatPPT artificially create it, you can look at it kind of plan based around it and then decide what to do from there.
37:03
SPEAKER_02: So um I it does have limitations.
37:05
SPEAKER_02: Um I wouldn't use it all the time and everything.
37:08
SPEAKER_02: Um, my wife told me I'm not allowed to use it at home, so this is strictly a uh a work thing.
37:13
SPEAKER_02: Um but uh I think like incorporating that and making it easy for negative feedback to be um brought in at the place and time where you were like planning out the work to come next, we found it to be pretty helpful.
37:25
SPEAKER_00: Well, and use a variant of that even uh for improving your operations as well, which I think is is um a really useful um idea.
37:36
SPEAKER_00: Because oftentimes what happens is that you get to a certain point of operational, it's not even excellence, right?
37:44
SPEAKER_00: Operational okayness.
37:46
SPEAKER_00: And and particularly if you've gone through the process of operations being poor in your environment, you get to a point where it is okay, and people have worked really hard to get to that point, then it can become really hard to list all the ways in which things still suck and you still have to uh you still have to improve things.
38:03
SPEAKER_00: And so I love that.
38:05
SPEAKER_00: We're definitely going to use that inside our um our organization as well, just to just to kind of like reframe, uh, reframe it.
38:12
SPEAKER_00: And it gets into the heart of human psychology, right?
38:14
SPEAKER_00: It's a lot easier to tell, like, I do like your book, but let's say I thought your book sucked, it would be a lot, it would be a lot easier for me to say, oh, this is a great book, or not say anything, which is what customers mostly do, right?
38:27
SPEAKER_00: It's just they don't say anything or they don't use it or whatever.
38:30
SPEAKER_00: It's so hard to get that negative feedback.
38:34
SPEAKER_00: We we treat negative feedback when we get it like gold to be treasured, right?
38:39
SPEAKER_00: Because it means that somebody cares enough to tell you how some why something sucks or how something sucks.
38:46
SPEAKER_00: And it's really difficult to extract that.
38:48
SPEAKER_02: Yeah, and that's another insight I feel like we we learned kind of by accident, but that's if somebody is complaining about your product, that is a signal.
39:00
SPEAKER_02: It's an engagement signal.
39:02
SPEAKER_02: And what we found is if you're looking at a a prospect that may or may not buy your product, maybe they got a demo, maybe they had a trial, but they're at this decision point where they need to decide, am I going to use it, am I not?
39:14
SPEAKER_02: We saw that the people who complained are actually more likely to convert than the people who are like riffing off of a, oh, this is cool, it could be used here, it could be used there.
39:24
SPEAKER_02: Because there's conviction in that, right?
39:25
SPEAKER_02: They're trying to get something done and they're complaining about I ran into a roadblock here and ran into a roadblock there.
39:30
SPEAKER_02: It's really easy to like make castles in the sky and think about different ideas that don't really come with conviction.
39:37
SPEAKER_02: Um, so it was interesting because like on the surface, you'd want to talk to the optimist who's talking about the 20 different directions the company could go in.
39:44
SPEAKER_02: But actually the folks that are saying, I tried to do X and I ran into a problem.
39:48
SPEAKER_02: Can you help me get unblocked?
39:50
SPEAKER_02: are the ones that uh maybe the subject line of the email isn't as fun to read, but like it's actually where you should be spending more of your time.
39:57
SPEAKER_00: Yeah.
39:58
SPEAKER_00: All right, last question for you.
40:00
SPEAKER_00: Um, you're still pretty early in career.
40:03
SPEAKER_00: In fact, Zach, I'll say that, you know, like there was a uh when I I think it I think it was when I hit, I don't know, 28 or 30 or whatever, and and there was I was watching some tennis player on TV, it might have been Pete Sampras or somebody like that, and he won like seven majors.
40:17
SPEAKER_00: And I'm like, what have I done?
40:18
SPEAKER_00: I'm older than that, right?
40:20
SPEAKER_00: But uh but you've I kind of feel like that talking to you.
40:24
SPEAKER_00: Um, but you're you've accomplished so much already.
40:27
SPEAKER_00: Um you've got uh a good proportion of what would what would be considered to be a career going forward, but so much is gonna change in the workplace over the next say 20 years.
40:38
SPEAKER_00: So I'm curious as you look forward 20 years, um, you're probably not spending a lot of time doing that because you know so much is gonna change, but as you look forward 20 years, um what just guess?
40:51
SPEAKER_00: What do you think your world is gonna look like?
40:54
SPEAKER_00: Are you gonna be even doing something that could I that you could identify as a job?
40:58
SPEAKER_00: Do those even exist?
41:01
SPEAKER_00: What do you think when you when you picture Zach in what would that be, 2043?
41:07
SPEAKER_02: It's like so far out, sounds like a sci-fi number.
41:10
SPEAKER_02: Um does.
41:11
SPEAKER_02: Yeah, I think it's it's a good question.
41:12
SPEAKER_02: Yeah, and you're right.
41:12
SPEAKER_02: I don't spend too much time.
41:14
SPEAKER_02: I guess if you look 20 years back, this is kind of like iPod just came out, and you're asking me to describe what today looks like.
41:19
SPEAKER_02: So there's uh quite quite a room for change there.
41:22
SPEAKER_02: But I mean, I could talk about the the world I'd love to work in.
41:25
SPEAKER_02: Um I feel like we're seeing some very promising signals that we all have it, right?
41:31
SPEAKER_02: Like the if it's Sunday night and you're dreading work, what are the things that you're dreading?
41:35
SPEAKER_02: I'd love to see AI eliminate those aspects of it.
41:38
SPEAKER_02: Um it's usually, if you ask people, the more mundane, tedious, kind of boring things.
41:44
SPEAKER_02: And I feel like AI gives so much capability to people to create, to express themselves, to find those interesting intersections between two different disciplines or two different areas.
41:55
SPEAKER_02: And if we can, as a um like a workforce, spend more of our time there, um, pushing boundaries, creating new things, and less on um, oh, it's uh it's Friday, you got to fill out the timesheet and send that stuff in.
42:09
SPEAKER_02: I feel like humans have a pretty good track record of um advancing and building new things.
42:14
SPEAKER_02: And if I had like the most um tedious parts of my day that were bogging me down, just removed handles correctly, I think there will be lots of cool things that would happen.
42:24
SPEAKER_02: So I think from a workforce standpoint, that's what I'd love to see.
42:27
SPEAKER_02: And then lots of other areas that I think um could be innovated in.
42:33
SPEAKER_02: I would love to see an advancement in energy.
42:36
SPEAKER_02: Um, if you look at compute power, how that's come way down over time, disk storage.
42:41
SPEAKER_02: I mean, you can you can get a terabyte on your phone now, and that was thousands of dollars a couple decades ago.
42:46
SPEAKER_02: Look at um how many watt hours a battery can store.
42:50
SPEAKER_02: Um, it hasn't really gotten that much better since like the 90s.
42:53
SPEAKER_02: So I think um I'm hoping if we're that far out that someone solved that and maybe we finally get our much-promised jetpacks or our drones, we don't have to sit in traffic anymore.
43:02
SPEAKER_02: Um, but I think a lot of the time compute will get so powerful, we'll get limited by physics and by energy.
43:08
SPEAKER_02: And if you get to that point where um you're no longer limited by either of those, then maybe we'll be on the moon having this conversation.
43:15
SPEAKER_02: I don't know.
43:16
SPEAKER_00: Sounds fantastic.
43:18
SPEAKER_00: Thank you so much.
43:19
SPEAKER_00: That's a great point to leave it.
43:20
SPEAKER_00: Once again, here's the the book Um Grow Up Fast.
43:24
SPEAKER_00: Lessons from an AI startup will include a link to that in the uh in the show notes.
43:30
SPEAKER_00: Uh thank you for writing it.
43:33
SPEAKER_00: It was uh it was uh, as I say, really helpful for uh uh for me personally.
43:37
SPEAKER_00: Also reminded me, by the way, how useful it is or how much work you can get done if you have Radiohead playing at a low volume in your house.
43:45
SPEAKER_00: So that's so that was also very helpful too.
43:49
SPEAKER_00: Uh really appreciate your time today.
43:50
SPEAKER_00: Thanks a lot, Zach.
43:52
SPEAKER_01: Thanks so much, Paul.
43:53
SPEAKER_01: Humanity Working is produced and edited for Billion Minds by Matt Neal.
43:57
SPEAKER_01: Our interviewer is Paul Slater.
43:59
SPEAKER_01: Music is by Dolo Records.
44:01
SPEAKER_01: If you like this podcast, please subscribe, like, and recommend us to others.
44:05
SPEAKER_01: It helps us reach more people.
44:06
SPEAKER_01: If you want to learn more about how Billion Minds gets companies ready for the future of work by building adaptable, resilient employees, look for them on LinkedIn or visit their website at billionminds.com.