Latest / The Pragmatic Engineer / From Software Engineer to AI Engineer – with Janvi Kalra

Transcript

From Software Engineer to AI Engineer – with Janvi Kalra · The Pragmatic Engineer

0:00
SPEAKER_00: You interviewed at 46 companies.
0:02
SPEAKER_00: What did you learn about what the market is like,
0:05
SPEAKER_00: what interviewing is like,
0:06
SPEAKER_00: what the whole scene is?
0:07
SPEAKER_01: In terms of the space,
0:09
SPEAKER_01: there are the product companies,
0:11
SPEAKER_01: infrastructure companies,
0:12
SPEAKER_01: and the model companies.
0:13
I found it helpful to put companies in each category and figure
0:17
out which segment
0:18
SPEAKER_01: you're most excited about to help narrow down the options,
0:21
SPEAKER_01: given that there's so many AI companies right now.
0:24
SPEAKER_01: Product companies are the companies building on top of the model.
0:26
SPEAKER_01: Here I think of Cursor, Codium, Hebbia.
0:28
Infrastructure companies are the companies building the tools
0:31
to help AI product
0:33
SPEAKER_01: companies effectively use LLMs.
0:35
SPEAKER_01: So whole suite of these.
0:36
SPEAKER_01: There are the inference providers like modal, fireworks together.
0:39
SPEAKER_01: Vector database companies like Pinecone, ChromaDB, Weviate.
0:43
SPEAKER_01: Eval and observability tools like BrainTrust, Arise, Galileo.
0:47
SPEAKER_01: and a whole other suite of products.
0:49
SPEAKER_01: And then there's the model companies,
0:50
SPEAKER_01: which are the base of the ecosystem building the intelligence.
0:53
SPEAKER_01: You have the big tech companies like Google, Meta, building models.
0:57
SPEAKER_01: And then you also have startups like,
0:58
SPEAKER_01: or not startups,
0:59
SPEAKER_01: you have other smaller companies like OpenAI,
1:01
SPEAKER_01: Anthropic,
1:03
SPEAKER_01: building models as well.
1:03
SPEAKER_01: So that's all I kind of think.
1:05
SPEAKER_01: So for me,
1:06
SPEAKER_01: in trying to be more focused in my search,
1:08
I decided to focus on model and infrastructure companies
1:11
because I wanted to keep
1:12
SPEAKER_01: getting breath in what I was doing.
1:14
And I felt like the product companies were too similar to my
1:16
experience at Coda,
1:17
SPEAKER_01: which was phenomenal,
1:18
SPEAKER_01: but I wanted to keep growing.
1:19
SPEAKER_01: And that definitely,
1:20
the trade-off was that it's a bit more of an uphill battle
1:22
because the work that I
1:23
SPEAKER_01: had done was not as relevant to model or infrastructure companies.
1:27
What is AI engineering and what does it take to get hired as an
1:30
AI engineer?
1:31
John Lee Colroy is a software engineer turned AI engineer with
1:34
four years of
1:34
SPEAKER_00: experience,
1:35
SPEAKER_00: but already a very impressive background.
1:37
SPEAKER_00: During college,
1:38
SPEAKER_00: she joined the university incubator where she shipped four mobile apps of
1:41
SPEAKER_00: production to paying customers.
1:43
SPEAKER_00: She then interned at Google and Microsoft,
1:45
SPEAKER_00: joined Coda as a software engineer,
1:46
SPEAKER_00: became one of the first AI engineers at the company,
1:49
and then interviewed with 46 different AI startups and now
1:52
works at OpenAI.
1:54
SPEAKER_00: In today's conversation,
1:55
SPEAKER_00: we cover Jaume's decision making when she decided to join a startup after
1:59
graduating when she already had her turn offers from big tech
2:01
companies like Google
2:02
SPEAKER_00: and Microsoft.
2:03
SPEAKER_00: How Janvi became one of the first AI engineers at Coda,
2:06
despite being told no when she first volunteered to join Coda's
2:09
in-house AI team.
2:11
SPEAKER_00: What Janvi works on at OpenAI and why she thinks OpenAI moves so fast.
2:15
SPEAKER_00: And many more topics.
2:17
If you're interested in AI engineering or how to make that
2:19
transition from software
2:20
SPEAKER_00: engineer to AI engineer,
2:21
this conversation has lots of relevant tips and observations
2:24
coming from Janvi.
2:26
If you enjoy the show,
2:26
please subscribe to the podcast on any podcast platform and on
2:29
YouTube.
2:29
SPEAKER_00: So Janvi, welcome to the podcast.
2:32
SPEAKER_01: Thanks for having me, Gurgei.
2:33
You were in a good college in Dartmouth,
2:36
but you then got an internship at Google.
2:38
Now,
2:38
not everyone working at Dartmouth can get into a place like it
2:41
was very competitive.
2:43
SPEAKER_00: How did you get that internship and then the Microsoft internship?
2:46
SPEAKER_00: What was the interview process like?
2:47
SPEAKER_00: And what do you think helped you get your step in the door, basically?
2:52
Back then,
2:53
I didn't know anyone at Google or Microsoft,
2:56
so I applied through their portal.
2:57
SPEAKER_01: And I remember for university students,
2:59
SPEAKER_01: they asked you to write essays on why you want to work there.
3:03
So I remember in those essays talking about the things that I
3:06
had built outside of
3:07
SPEAKER_01: classes,
3:08
SPEAKER_01: as well as why I wanted to work there in particular.
3:11
SPEAKER_01: I was lucky to get picked up from the stack,
3:13
SPEAKER_01: to be honest,
3:14
SPEAKER_01: and then leak-coded to prepare for their interviews.
3:17
SPEAKER_00: So tell me about like preparing for lead code.
3:20
SPEAKER_00: I mean,
3:20
SPEAKER_00: these days it is somewhat commonly known,
3:22
SPEAKER_00: but there's,
3:22
SPEAKER_00: you know,
3:22
SPEAKER_00: two types of people,
3:23
SPEAKER_00: some engineers or college students,
3:25
SPEAKER_00: they roll their eyes saying,
3:26
SPEAKER_00: ah,
3:26
SPEAKER_00: this is,
3:26
SPEAKER_00: you know,
3:27
SPEAKER_00: pointless,
3:27
SPEAKER_00: it's not the job,
3:28
SPEAKER_00: et cetera.
3:29
SPEAKER_00: And then some people just,
3:30
SPEAKER_00: you know,
3:30
SPEAKER_00: like,
3:31
SPEAKER_00: it sounds like you just kind of like went through it,
3:33
SPEAKER_00: studied it,
3:34
SPEAKER_00: prepared it.
3:34
SPEAKER_00: How did that go?
3:35
SPEAKER_01: So for Google, that was in my sophomore year.
3:38
And I remember being surprised that I even got an interview in
3:41
the first place.
3:42
SPEAKER_01: So I wasn't studying actively before.
3:46
SPEAKER_01: I, when I got the interview, I asked a couple friends
3:50
SPEAKER_01: What do I study?
3:52
SPEAKER_01: I think they sent us a pamphlet of things to look at.
3:54
SPEAKER_01: And in that pamphlet, there was that green book.
3:58
SPEAKER_01: Because back then, neat code wasn't a thing.
4:01
SPEAKER_01: There wasn't blind 75.
4:02
SPEAKER_01: And so that green book, I'm forgetting what it's called.
4:06
But I remember buying that book,
4:08
locking myself in my room for three weeks.
4:10
SPEAKER_00: Cracking the coding interview, probably.
4:11
SPEAKER_01: Yes, cracking the coding interview.
4:13
SPEAKER_01: And just reading as many questions as I could back then.
4:16
SPEAKER_01: Yeah, you have it.
4:17
SPEAKER_00: Yeah.
4:19
I even have a version of it,
4:20
which was the white book that that was like 10,
4:23
10 years ago.
4:23
SPEAKER_00: But so the author of this was actually a Google interviewer,
4:28
SPEAKER_00: like I think 15 or so years ago,
4:30
SPEAKER_00: Gail Lackman McDonald.
4:32
And now she she actually sometimes I'm not sure if she does it,
4:36
but she used to.
4:37
SPEAKER_00: run training programs at companies at Uber.
4:40
SPEAKER_00: She came and she ran our training program on how to do coding interviews,
4:46
SPEAKER_00: what kind of signals to get,
4:47
SPEAKER_00: how to change it.
4:48
So it's actually really nice because she teaches the companies
4:50
on how to do it.
4:51
So then she can update the book and actually have up to date of
4:54
how it works at
4:55
SPEAKER_00: different companies.
4:56
SPEAKER_01: Wow.
4:57
She definitely changed the game in that I'm not sure how much
5:00
things were written
5:01
SPEAKER_01: down before that.
5:02
So I think she definitely paved the way for NeetCode and other
5:05
people to build on top of this.
5:08
SPEAKER_00: If you want to build a great product, you have to ship quickly.
5:12
SPEAKER_00: But how do you know what works?
5:13
SPEAKER_00: More importantly, how do you avoid shipping things that don't work?
5:18
SPEAKER_00: The answer, Statsig.
5:20
SPEAKER_00: Statsig is a unified platform for flags,
5:22
SPEAKER_00: analytics,
5:23
SPEAKER_00: experiments,
5:24
SPEAKER_00: and more,
5:24
combining five plus products into a single platform with a
5:27
unified set of data.
5:29
SPEAKER_00: Here's how it works.
5:31
SPEAKER_00: First, StatSick helps you ship a feature via FeatureFly or Config.
5:35
SPEAKER_00: Then it measures how it's working,
5:37
SPEAKER_00: from alerts and errors,
5:38
SPEAKER_00: to replays of people using that feature,
5:41
SPEAKER_00: to measurement of top-line impact.
5:43
SPEAKER_00: Then you get your analytics,
5:44
SPEAKER_00: user account metrics,
5:45
SPEAKER_00: and dashboards to track your progress over time,
5:48
SPEAKER_00: all linked to the stuff you ship.
5:50
SPEAKER_00: Even better,
5:50
SPEAKER_00: Statsik is incredibly affordable with a super generous free tier,
5:54
SPEAKER_00: a starter program with $50,000 of free credits,
5:57
SPEAKER_00: and custom plans to help you consolidate your existing spend on flags,
6:00
SPEAKER_00: analytics,
6:00
SPEAKER_00: or A-B testing tools.
6:02
SPEAKER_00: To get started, go to statsik.com slash pragmatic.
6:06
SPEAKER_00: That is S-T-A-T-S-I-G dot com slash pragmatic.
6:09
SPEAKER_00: Happy building!
6:11
SPEAKER_00: This episode is brought to you by Cinch,
6:13
the customer communications cloud trusted by thousands of
6:16
engineering teams around
6:17
SPEAKER_00: the world.
6:18
If you've ever added messaging,
6:19
voice,
6:20
or email into a product,
6:21
you know the pain.
6:22
SPEAKER_00: Flaky delivery and platform stack with middlemen.
6:25
SPEAKER_00: Cinch is different.
6:27
They run their own network with direct carrier connections in
6:30
over 60 countries.
6:31
That means faster delivery,
6:33
higher reliability,
6:34
and scale that just works.
6:36
SPEAKER_00: Developers love Cinch for its single API that covers 12 channels,
6:40
SPEAKER_00: including SMS,
6:41
SPEAKER_00: WhatsApp,
6:41
SPEAKER_00: and RCS.
6:43
SPEAKER_00: Now is the time to pay attention to RCS, rich communication services.
6:47
SPEAKER_00: It's like SMS, but smarter.
6:49
Your brand name,
6:50
logo,
6:50
and verified checkmark all inside the native messaging app.
6:54
SPEAKER_00: Built by Google,
6:55
SPEAKER_00: now rolling out with Apple and major carriers,
6:57
SPEAKER_00: RCS is becoming the messaging standard.
7:00
SPEAKER_00: Cinch is helping teams go live globally.
7:03
SPEAKER_00: Learn more at cinch.com slash pragmatic.
7:06
SPEAKER_00: That is S-I-N-C-H dot com slash pragmatic.
7:11
SPEAKER_00: And so how did your internships go at both Google and Microsoft?
7:16
SPEAKER_00: Must have been really exciting to like get it.
7:18
SPEAKER_00: Google was the first one, right?
7:19
It was a phenomenal experience and it was exciting for a couple
7:23
of reasons.
7:24
First,
7:24
it was just a privilege to get access to these code bases of
7:27
places that I admired.
7:28
SPEAKER_01: I remember when I was at Google,
7:29
SPEAKER_01: I was on the search team and I would use MoMA,
7:32
SPEAKER_01: their internal tool to find documents.
7:35
And so I remember so many weekends where I was just trying to
7:39
find company
7:40
documentation on how the search algorithm really works or comb
7:43
through the code
7:44
SPEAKER_01: beyond the code that I was touching to get a sense of,
7:46
SPEAKER_01: well,
7:47
SPEAKER_01: what makes Google tick?
7:48
So from an intellectual perspective,
7:50
it was just very exciting back then.
7:52
SPEAKER_01: Second,
7:53
you also learn a lot of technical things that you don't get
7:55
exposure to in college,
7:57
SPEAKER_01: like how to effectively operate in a large code base,
7:59
SPEAKER_01: the importance of writing unit tests.
8:01
SPEAKER_01: Now, when I look back, it's trivial.
8:03
SPEAKER_01: But for a college student back then,
8:04
I remember it was very important learnings that I really value
8:08
getting back then.
8:10
SPEAKER_01: And to me,
8:10
my favorite part was having access to people that were five or
8:14
10 years ahead of me
8:15
SPEAKER_01: in their career.
8:17
I remember over coffee chats asking many of them,
8:20
you know,
8:20
what career advice do you have?
8:22
SPEAKER_01: What are things that you loved in college that I should do more of?
8:26
SPEAKER_01: And some of the advice that I got really paved decisions that I made.
8:31
SPEAKER_01: So that was my favorite part of the internships.
8:35
SPEAKER_01: I would say in hindsight that given the big tech experience,
8:39
And startups are such different experiences and you learn so
8:43
much at each.
8:44
It would be more educational to do one startup internship and
8:48
one big tech
8:49
internship to get a very robust overview of what both
8:52
experiences are like very
8:53
SPEAKER_01: early.
8:53
SPEAKER_00: So like looking back now that you've done both Google and Microsoft,
8:56
SPEAKER_00: they were somewhat similar-ish?
8:59
SPEAKER_00: Is it safe to say?
8:59
SPEAKER_00: I mean, at the high level, right?
9:01
SPEAKER_00: Like we know every company and every team is different.
9:03
SPEAKER_01: Yes, at a high level.
9:04
What was different is I wanted my junior year to work on
9:07
operating systems because
9:08
at that point I had just taken a computer architecture class
9:10
and I loved it.
9:12
SPEAKER_01: And so I wanted to go deeper in the stack.
9:13
SPEAKER_01: So from a technical perspective, they were very different.
9:16
But from an experience of what do companies look like and how
9:21
do they work,
9:22
SPEAKER_01: which is a huge part of an internship,
9:23
SPEAKER_01: that part was similar.
9:25
SPEAKER_00: So what did you work on at Microsoft?
9:26
SPEAKER_00: Was that OS?
9:27
SPEAKER_01: Yeah, I was working on OS.
9:29
SPEAKER_01: Specifically, I was working on the Azure OS team.
9:32
It was a product that lets you interact with Azure blobs
9:37
locally from your file system.
9:39
SPEAKER_01: So it hydrates and dehydrates those blobs.
9:41
SPEAKER_01: You can think of it like Dropbox for Azure blobs.
9:44
SPEAKER_00: Yeah, nice.
9:45
SPEAKER_00: That is so cool.
9:47
SPEAKER_00: I mean,
9:47
both that you decided that you want to do something a lot less
9:51
conventional,
9:51
SPEAKER_00: you know,
9:51
SPEAKER_00: like not the usual SaaS apps or web apps or whatnot,
9:54
SPEAKER_00: and that you were able to make it happen.
9:56
SPEAKER_00: Did you express this preference when you got the internship?
10:00
SPEAKER_01: Yes,
10:00
I remember talking about my computer architecture class where
10:03
we built up a
10:05
computer from transistors and conveying how mind blown I was
10:08
from that experience
10:10
SPEAKER_01: and how I really wanted to work on operating systems.
10:12
SPEAKER_01: And then I was lucky that they put me on that team.
10:15
SPEAKER_00: That's awesome.
10:15
But I think there's a learning here of,
10:17
like,
10:17
you don't ask,
10:17
you don't get.
10:18
SPEAKER_00: So,
10:19
SPEAKER_00: like,
10:19
SPEAKER_00: again,
10:20
SPEAKER_00: I just remember when I was running,
10:22
SPEAKER_00: I set up our first internship at Uber in Amsterdam,
10:25
SPEAKER_00: so for that site.
10:26
SPEAKER_00: And,
10:27
SPEAKER_00: you know,
10:27
SPEAKER_00: like,
10:28
SPEAKER_00: once we made an offer to interns,
10:30
SPEAKER_00: like,
10:30
SPEAKER_00: you go through the interview process.
10:31
SPEAKER_00: But I also ask people, like, if they have preference.
10:33
SPEAKER_00: And most people just do not have preference.
10:35
So there is this interesting thing that if you do express your
10:38
preference,
10:38
SPEAKER_00: again,
10:39
SPEAKER_00: worst case,
10:39
SPEAKER_00: you know,
10:40
SPEAKER_00: you'll get whatever it would have been.
10:42
SPEAKER_00: But
10:43
SPEAKER_00: From the other side,
10:44
a lot of people often don't speak up and the people who are at
10:47
these companies,
10:48
SPEAKER_00: they really want to try this win-win,
10:50
SPEAKER_00: especially for internships.
10:51
The goal of an internship is to have a great experience and
10:53
companies would like you to return.
10:55
SPEAKER_00: It goes both ways, right?
10:56
SPEAKER_00: They evaluate you, but you also evaluate them.
10:58
SPEAKER_00: So they will actually do it.
11:00
SPEAKER_00: It's just a really nice learning.
11:02
SPEAKER_00: Like, yes, express what you're hoping for and it might just happen.
11:06
SPEAKER_01: Yeah.
11:06
And these companies have so much IP and so much that we take
11:11
for granted today,
11:12
SPEAKER_01: but are really hard technical problems that they have solved.
11:15
So it's just a treat to then go work on something that you
11:18
admire and get to
11:20
SPEAKER_01: actually see how that code works.
11:22
SPEAKER_00: Absolutely.
11:24
SPEAKER_00: Once you're in there,
11:25
SPEAKER_00: these companies are so amazing with how big they are,
11:28
SPEAKER_00: especially as an intern,
11:29
SPEAKER_00: a lot of doors are open.
11:30
SPEAKER_00: You can also just ask and they'll be super happy to do.
11:33
So then you'd made a very interesting decision because now you
11:36
were interned at
11:37
SPEAKER_00: Google,
11:37
SPEAKER_00: you're interned at Microsoft.
11:40
SPEAKER_00: A lot of students or new grads would be super happy with just having one.
11:44
SPEAKER_00: As I understand, you could have returned to either.
11:46
SPEAKER_00: And then you made the decision to not do that.
11:50
SPEAKER_00: Why?
11:52
SPEAKER_00: You know, Google, Microsoft, you love the teams.
11:55
Tell me about how you thought about the next step of what you
11:58
would like to do
12:00
SPEAKER_00: after you graduate.
12:01
SPEAKER_01: So I told you how I was having coffee chats at Microsoft,
12:04
SPEAKER_01: my junior internship with a bunch of mentors,
12:07
SPEAKER_01: mentioned that startups are a great experience as well.
12:10
SPEAKER_01: So I started to evaluate the big tech versus startup option.
12:14
SPEAKER_01: And I don't think it's black and white.
12:15
SPEAKER_01: I think there are really good reasons to go to both.
12:18
SPEAKER_01: The way I saw it,
12:19
the upside of going to big tech was first you learn how to
12:22
build reliable software
12:23
SPEAKER_01: for scale.
12:24
It's very different to build something that works versus build
12:28
something that works
12:29
when it's swarmed with millions requests from around the world
12:32
and Redis happens to
12:32
SPEAKER_01: be down at the same time.
12:34
SPEAKER_01: Very different skills.
12:36
SPEAKER_01: So that was one upside.
12:39
Different upside for big tech in general was that you do get to
12:43
work on more
12:44
SPEAKER_01: moonshot projects that aren't making money today.
12:48
SPEAKER_01: They don't have the same existential crisis that startups do.
12:51
And so they can work on things that,
12:53
you know,
12:55
great AR,
12:55
VR research is happening.
12:57
Back in the day,
12:57
I think Google was one of the best places if you wanted to do
12:59
AI research.
13:00
SPEAKER_01: There are also practical good reasons to go to AI.
13:03
SPEAKER_01: Big tech.
13:04
SPEAKER_01: I'd get my green card faster.
13:05
SPEAKER_01: I'd get paid more on average.
13:08
And the unfortunate reality,
13:09
I think,
13:10
is that the role does hold more weight.
13:12
People are more excited about hiring an L5 Google engineer
13:16
versus an L5 from a
13:17
SPEAKER_01: startup,
13:18
SPEAKER_01: especially if that startup doesn't become very successful.
13:22
With that all said,
13:23
though,
13:23
I think there are great reasons to go to a startup.
13:26
SPEAKER_01: And back then, this was hearsay based on what I heard from mentors.
13:30
But now,
13:30
having worked at a startup for three years,
13:32
I can confirm it's indeed true.
13:34
SPEAKER_01: First, you just ship so much code, right?
13:36
SPEAKER_01: There are more problems than people,
13:37
SPEAKER_01: and so you get access to these zero-to-one greenfield problems that...
13:42
you wouldn't necessarily get,
13:43
where Epic Tech may be where there are more people than
13:46
problems.
13:47
Second is the breadth of skills,
13:49
and this is not just in the software engineering space.
13:52
SPEAKER_01: Right from a software engineering space,
13:53
SPEAKER_01: maybe one quarter you're working on a growth hacking front end feature,
13:56
SPEAKER_01: and the next quarter you're writing Terraform.
13:58
But even in terms of the non-technical skills,
14:00
you get an insight into how the business works.
14:02
SPEAKER_01: And you're expected to PM your own work.
14:04
SPEAKER_01: So there's so much breadth over there.
14:06
SPEAKER_01: And you just get more agency in what you work on.
14:08
You get the opportunity to propose ideas that you think would
14:11
be impactful for the
14:12
SPEAKER_01: business and go execute on it.
14:14
So that breadth and learning opportunity to me was a huge
14:17
upside that got me very
14:18
SPEAKER_01: excited about startups.
14:19
It's just so nice to hear you summarize this because the
14:23
reality of what a lot of
14:24
SPEAKER_00: people do is they go to one company or the other,
14:26
SPEAKER_00: either big tech or startup,
14:28
SPEAKER_00: and then they're there for a long time.
14:29
SPEAKER_00: And then one day they might switch,
14:32
SPEAKER_00: but there's a lot of like sunk cost fallacy,
14:34
SPEAKER_00: you know,
14:34
SPEAKER_00: like you're used to this.
14:35
SPEAKER_00: So some people actually, after a few years, they might go back.
14:38
SPEAKER_00: to the same type of company.
14:40
And so I think there's relatively few people who see this with
14:44
such short and
14:46
SPEAKER_00: focused time difference to see the different upsides like you have.
14:50
SPEAKER_00: And as you said, so it sounds like the upsides did happen.
14:52
SPEAKER_00: So you went to Coda, right?
14:54
SPEAKER_01: Yes, I did go to Coda.
14:56
SPEAKER_00: And then how do things go?
14:58
So you mentioned some of the upsides,
14:59
I assume,
15:00
like that all happened there.
15:02
SPEAKER_00: But what other things?
15:05
SPEAKER_00: Sounds like things sped up there,
15:06
SPEAKER_00: actually,
15:06
SPEAKER_00: from a professional learning and also career experience.
15:10
SPEAKER_01: Definitely.
15:10
SPEAKER_01: I went there for growth and breadth,
15:12
and I definitely got that in terms of the opportunities that I
15:15
got to work on.
15:16
SPEAKER_01: So it was a phenomenal experience.
15:17
And I'm happy to dive into the specific work I did,
15:20
but overall,
15:20
just a phenomenal experience.
15:22
SPEAKER_00: But before we do,
15:23
SPEAKER_00: before the podcast,
15:24
we talked a little bit about how you thought about selecting a
15:29
startup because like
15:30
SPEAKER_00: you did go to Coda.
15:31
SPEAKER_00: But as I understand,
15:32
SPEAKER_00: this was not just I'm just going to,
15:34
SPEAKER_00: you know,
15:34
SPEAKER_00: like,
15:34
SPEAKER_00: oh,
15:35
SPEAKER_00: this looks like a good startup.
15:36
You actually thought about how to select a potentially great
15:41
startup that that
15:41
SPEAKER_00: would have that kind of potential growth potential.
15:44
What is your mental model and how did you evaluate and how did
15:48
you kind of rank the startups?
15:50
SPEAKER_00: What was your application process?
15:53
SPEAKER_01: So back then, I didn't have startup experience.
15:55
And I also went to a school on the East Coast where not many
15:57
peers around me were
15:58
SPEAKER_01: going to startups.
15:59
So I very much looked for where are places where I love the
16:04
people in terms of them
16:05
SPEAKER_01: being smart,
16:06
SPEAKER_01: people I can learn from,
16:08
SPEAKER_01: as well as being very passionate about the product.
16:10
Because I think you do your best work when you are passionate
16:12
about what you're building.
16:13
SPEAKER_01: So it was definitely something where I looked for from those two lenses.
16:18
SPEAKER_01: Today,
16:18
SPEAKER_01: after having been in Silicon Valley,
16:20
SPEAKER_01: though,
16:20
SPEAKER_01: for four years,
16:21
SPEAKER_01: I have a more robust rubric on what I look for.
16:25
SPEAKER_01: So that's definitely evolved since then.
16:27
Because one thing that's become super clear after living here
16:31
is that your career
16:33
SPEAKER_01: growth at a startup is very contingent on the startup growing.
16:36
SPEAKER_01: So then how do you choose which startup is going to grow?
16:38
SPEAKER_01: And that's a hard question.
16:40
SPEAKER_01: You know, venture capitalists spend all their time thinking about this.
16:44
SPEAKER_00: And today, what is your mental model?
16:46
SPEAKER_00: Or for someone who has a few years of experience a bit like yourself,
16:52
what would you advise for them on how to think about different
16:55
categories of
16:55
SPEAKER_00: startups,
16:56
SPEAKER_00: the kind of risk,
16:57
SPEAKER_00: the upsides,
16:58
SPEAKER_00: and so on?
16:58
SPEAKER_01: They're startups of all different sizes.
17:02
SPEAKER_01: And the smaller you go, the more risk there is.
17:04
SPEAKER_01: I think that's part of the game.
17:07
And that's what makes it exciting because you also have more
17:09
upside when there's more risk.
17:11
SPEAKER_01: That being said,
17:11
I feel very strongly that all engineers that take a pay cut to
17:14
go to a startup
17:15
should have an informed thesis on why they think that company
17:19
is going to grow
17:20
SPEAKER_01: during their tenure.
17:21
And how to actually assess growth is a hard question with no
17:26
right answer.
17:27
SPEAKER_01: But my current rubric is looking for four things.
17:32
SPEAKER_01: First, high revenue and steep revenue growth rate.
17:35
SPEAKER_01: Second, a large market where there's room to expand.
17:40
SPEAKER_01: Third, loyal, obsessed customers.
17:43
SPEAKER_01: And then fourth, competition, why this company will win in that space.
17:47
SPEAKER_01: And I'm happy to go deeper into any of those,
17:49
SPEAKER_01: but that's at least how I think about assessing different startups today.
17:53
And it's all relative because a startup that is pre-PMF will
17:58
have less revenue than
17:59
SPEAKER_01: a startup that is Series D 400 people.
18:01
SPEAKER_00: And then when you're like thinking about these four different things,
18:05
SPEAKER_00: so like we'll later get to your actual job search as well,
18:09
SPEAKER_00: but do you like try to find these things?
18:12
So for example,
18:12
you mentioned one thing about how customer obsession,
18:16
right?
18:16
SPEAKER_00: Like how much customers love it.
18:17
Like let's say you have a,
18:19
or there's a startup that you're kind of interested in.
18:21
SPEAKER_00: How do you evaluate that?
18:23
SPEAKER_00: Do you kind of look it up yourself?
18:24
SPEAKER_00: Do you put in the work?
18:25
SPEAKER_00: Do you try to somehow outsource it?
18:28
SPEAKER_00: What will work for you?
18:29
SPEAKER_01: Because there's no right answer here,
18:31
I think it's really important to do the due diligence yourself
18:34
because you're going
18:34
SPEAKER_01: to be the one that's responsible for your decision here,
18:37
SPEAKER_01: good or bad.
18:37
How I think about things like customer obsession is I look on
18:42
Reddit on YouTube to
18:43
SPEAKER_01: try to find real users.
18:45
For more SaaS companies where you may not have customers
18:48
writing about the product
18:49
SPEAKER_01: online,
18:50
I'd actually find companies that use that product and then go
18:53
try to talk to them
18:54
SPEAKER_01: and understand from the ground what
18:56
SPEAKER_01: What do people think about this product?
18:58
Especially if this is a product that I can't use myself because
19:00
it's not for
19:01
SPEAKER_01: customers,
19:02
SPEAKER_01: but for businesses instead.
19:03
SPEAKER_00: I love it.
19:04
And again,
19:04
I don't think enough people do this kind of due diligence and
19:09
they should.
19:10
SPEAKER_00: You know, one...
19:11
SPEAKER_00: I guess now a famous example is Fast,
19:14
SPEAKER_00: the one-click checkout startup,
19:15
SPEAKER_00: where they recruited actually,
19:17
SPEAKER_00: there were some ex-Uber folks there who I knew to some extent,
19:22
but a lot of people were recruited with a shiny diagram that
19:24
showed headcount
19:26
SPEAKER_00: growth.
19:26
SPEAKER_00: And a lot of people did not ask about revenue or
19:31
SPEAKER_00: or when they did, they were okay not hearing about it.
19:34
SPEAKER_00: And even the people who worked there for a while, they ignored it.
19:36
And then there were some people who actually asked about it and
19:38
they actually
19:39
SPEAKER_00: realized that something was off.
19:40
SPEAKER_00: But just following your framework,
19:42
SPEAKER_00: for example,
19:43
SPEAKER_00: some people who are a bit more diligent could have avoided.
19:45
SPEAKER_00: Same thing with customers, for example.
19:47
SPEAKER_00: There were not many.
19:49
And like one learning that I had back then and talking with
19:53
engineers who worked
19:53
SPEAKER_00: there and got burnt,
19:54
they all told me I wish I would have done a bit more due
19:56
diligence and not taken
19:58
SPEAKER_00: the CEO's word for it,
20:00
SPEAKER_00: but also asked for proof or say the same thing with revenue,
20:03
SPEAKER_00: runway,
20:04
SPEAKER_00: those kind of things.
20:05
Yeah,
20:05
I feel like,
20:06
you know,
20:06
at startups,
20:06
we're paid in equity a large chunk.
20:09
SPEAKER_01: And so you're investors.
20:10
SPEAKER_01: So you have the right to all this information.
20:13
And to me,
20:13
if a startup's not giving you that information,
20:15
that is a red flag in and of itself.
20:17
SPEAKER_00: Yeah,
20:17
SPEAKER_00: I feel maybe people should think about that if you join a startup,
20:20
SPEAKER_00: a bit like if you put in like a bunch of your money,
20:23
SPEAKER_00: like a significant amount of your savings.
20:25
SPEAKER_00: And when I did angel investing,
20:28
SPEAKER_00: if I didn't get information,
20:29
SPEAKER_00: I mean,
20:29
SPEAKER_00: you can still put it in and you can hope for the best.
20:31
SPEAKER_00: But I think that's called gambling, to be fair.
20:33
SPEAKER_01: Yeah.
20:34
SPEAKER_00: It is.
20:35
SPEAKER_00: And so that's okay.
20:36
SPEAKER_00: But then just be honest with yourself.
20:37
SPEAKER_00: If I'm not getting this information,
20:38
SPEAKER_00: I'm gambling my most valuable time and very valuable years of my life.
20:43
SPEAKER_00: And that's okay, right?
20:45
SPEAKER_00: It could work, but it's maybe not the smart way.
20:48
SPEAKER_01: Exactly.
20:48
SPEAKER_01: And as engineers, when we're recruiting, we're elite coding.
20:51
SPEAKER_01: We're doing system design.
20:52
SPEAKER_01: It's hard to carve out that time to do diligence.
20:54
SPEAKER_01: And so it's something I think we don't talk about enough.
20:56
SPEAKER_00: I will say that as a hiring manager, even as a manager,
21:00
SPEAKER_00: When you join a company and if you've previously done your due diligence,
21:04
SPEAKER_00: you will have a better start.
21:05
SPEAKER_00: People will remember you saying,
21:07
SPEAKER_00: oh,
21:07
SPEAKER_00: this is this person who actually cares about the business,
21:10
SPEAKER_00: cares about where it's going,
21:11
SPEAKER_00: cares about how they can contribute.
21:13
SPEAKER_00: So on day one,
21:14
SPEAKER_00: you're already not just seen as like,
21:15
SPEAKER_00: oh,
21:15
SPEAKER_00: you know,
21:16
SPEAKER_00: like new start or X,
21:17
SPEAKER_00: Y,
21:17
SPEAKER_00: Z,
21:17
SPEAKER_00: but like,
21:18
SPEAKER_00: oh,
21:18
SPEAKER_00: like this person has drive.
21:19
SPEAKER_00: Like, I think that comes across.
21:21
SPEAKER_00: And honestly,
21:21
SPEAKER_00: if a company is not happy,
21:23
SPEAKER_00: you just trying to understand the business,
21:25
SPEAKER_00: see how you can fit in.
21:26
SPEAKER_00: It's probably a red flag itself.
21:28
SPEAKER_00: Let's be real.
21:28
SPEAKER_01: Yeah, that's true.
21:29
SPEAKER_01: That's true.
21:30
SPEAKER_00: So at Coda,
21:31
SPEAKER_00: you joined as a software engineer and you then transitioned into,
21:35
SPEAKER_00: you know,
21:36
SPEAKER_00: I looked at your LinkedIn to AI engineer.
21:38
SPEAKER_00: How did that happen?
21:39
SPEAKER_00: And how did you make that happen?
21:41
SPEAKER_00: Because it sounds like you actually had a lot to do with it.
21:44
So if we rewind to end of 2022,
21:47
that was when ChatGPT came out and Coda was starting.
21:50
SPEAKER_00: Oh, yeah.
21:51
SPEAKER_01: Yeah, big milestone.
21:53
SPEAKER_01: And, you know, Coda saw the amount of love that this product was getting.
21:57
And Coda was starting an AI team with two engineers to build an
22:01
AI assistant to
22:01
SPEAKER_01: help you build your Coda documents.
22:04
At that time,
22:04
I asked that,
22:05
hey,
22:05
I'd love to be a part of it and got a very polite no.
22:09
SPEAKER_01: So I thought, no problem.
22:10
I'm just going to start building in this space anyway in my
22:13
nights and weekends
22:13
SPEAKER_01: because this technology is very cool.
22:16
SPEAKER_01: The first thing while I was learning was trying to answer to myself,
22:20
SPEAKER_01: how does ChatGPT even work?
22:23
And through that,
22:24
went down a rabbit hole of self-studying the foundations of
22:27
deep learning.
22:28
SPEAKER_01: So starting off with the very basics of what's a token,
22:31
SPEAKER_01: what's a weight,
22:32
SPEAKER_01: what's an embedding,
22:33
SPEAKER_01: to then understanding that,
22:34
SPEAKER_01: okay,
22:34
SPEAKER_01: LLMs are just next token prediction.
22:37
SPEAKER_01: Going through the history of different architectures,
22:40
SPEAKER_01: of how we went from RNNs to LSTMs,
22:42
SPEAKER_01: and then building my way up to the transformer and understanding that,
22:45
SPEAKER_01: okay,
22:45
it's positional encoding and attention that has allowed us to
22:48
scale up in such a
22:49
SPEAKER_01: good way.
22:50
What this did for me was to just give me some intuition of how
22:53
this technology
22:54
SPEAKER_01: works,
22:55
SPEAKER_01: which gave me a bit more confidence.
22:58
SPEAKER_01: After having built that foundation,
23:00
SPEAKER_01: I wrote about it in my blog,
23:02
SPEAKER_01: and so my team was aware that I was working on this as well.
23:05
SPEAKER_01: I started to build on top of these models.
23:08
SPEAKER_01: So I went to a lot of hackathons.
23:09
My favorite one was a way to learn languages while watching TV
23:15
because that's the
23:15
way that I learned Hindi and I wanted a way to practice my
23:18
Mandarin in that way.
23:20
SPEAKER_01: When I was building and...
23:22
SPEAKER_01: doing hackathons, I got a sense of how to actually use these tools.
23:26
SPEAKER_01: So after five months had passed,
23:29
SPEAKER_01: when I asked again to join the AI team,
23:31
SPEAKER_01: I got very much a heck yes,
23:33
SPEAKER_01: come join us.
23:33
We see that you truly care about this because you've been
23:35
working on it in your free time.
23:37
SPEAKER_01: And that's when the real learning started because
23:40
Hacking on top of models in your free time is very different
23:43
from trying to build
23:44
SPEAKER_01: it for production,
23:45
SPEAKER_01: especially because as engineers,
23:47
SPEAKER_01: our role is to build reliable systems,
23:49
SPEAKER_01: but you're dealing with stochastic models.
23:51
SPEAKER_01: So they're very much at odds with each other.
23:53
SPEAKER_00: And when you say hackathons,
23:55
SPEAKER_00: is this,
23:55
SPEAKER_00: was these like weekend hackathons,
23:57
SPEAKER_00: you know,
23:57
SPEAKER_00: the ones that anyone can attend,
23:59
SPEAKER_00: you register and like,
24:00
SPEAKER_00: especially they were popping up because of the AI,
24:03
SPEAKER_00: you know,
24:03
SPEAKER_00: like hype basically starting?
24:05
SPEAKER_00: Yeah.
24:05
SPEAKER_01: Yes, weekend hackathons.
24:06
SPEAKER_01: I also did.
24:08
So the project I was telling you about that was a language
24:12
learning tool that was
24:13
with an online hackathon for six weeks with this company called
24:17
Buildspace.
24:17
SPEAKER_01: Anyone can go join.
24:19
SPEAKER_01: And the way you win in this hackathon is not by what you build,
24:22
SPEAKER_01: but how many users you get or how much revenue you're generating.
24:24
SPEAKER_00: Oh, awesome.
24:25
SPEAKER_01: So it's such a fun way as an engineer to not just build something,
24:29
SPEAKER_01: but actually go and try to get people to use it.
24:32
SPEAKER_01: So it was a very fun learning experience.
24:33
And because it was all online,
24:35
they really encouraged us to build in public.
24:38
SPEAKER_01: And that in and of itself was a great learning.
24:41
I love it because I think a lot of times when a new technology
24:45
comes out and a lot
24:46
SPEAKER_00: of engineers,
24:47
SPEAKER_00: especially you had a day job and the people who have a day job,
24:49
SPEAKER_00: the biggest thing is like,
24:50
SPEAKER_00: hey,
24:51
SPEAKER_00: I can build something on the site,
24:52
SPEAKER_00: but what should I even build?
24:53
SPEAKER_00: I mean, you know, like it feels it's kind of pointless.
24:56
SPEAKER_00: Like you can do tutorials, but especially in this case, there's not many.
24:59
SPEAKER_00: And tutorials are kind of
25:01
not there so i love how you found a way to have a goal to enjoy
25:06
it to do you know
25:07
scratch your own itch as well and and combine it so like maybe
25:12
these online
25:12
hackathons or like hackathons happening around you it could be
25:15
a great way to do it
25:18
SPEAKER_00: And it sounds like it actually helped your professional,
25:20
SPEAKER_00: like,
25:22
helped even your company and your job because now knowing how
25:26
to use these tools
25:27
SPEAKER_00: was very much in demand.
25:28
SPEAKER_00: It still is.
25:30
But there were not many people who were,
25:31
like,
25:31
as enthusiastic and as self-taught.
25:34
One thing that I learned from that experience was don't wait
25:38
for people to give you
25:38
SPEAKER_01: the opportunity to do something.
25:40
SPEAKER_01: Just start working on it.
25:42
SPEAKER_00: I love this.
25:43
SPEAKER_00: This is such a good mindset.
25:44
SPEAKER_00: So when you join this team, so...
25:47
SPEAKER_00: Technically, did you become an AI engineer?
25:49
SPEAKER_00: What do you think even an AI engineer is?
25:50
SPEAKER_00: I feel is this kind of overloaded term.
25:52
SPEAKER_00: So I just love to hear like how you think about it.
25:55
SPEAKER_01: AI product engineer is building products on top of models.
25:59
And the work entails first a lot of experimentation of this new
26:04
tool came out,
26:05
SPEAKER_01: experimenting with what you can build to solve real customer problems,
26:09
SPEAKER_01: prototyping it,
26:11
SPEAKER_01: and then from there going and actually building it from production.
26:13
SPEAKER_01: So at its core, it's very similar to software engineering.
26:17
SPEAKER_01: There are some domain specific things like learning how to fine tune,
26:21
SPEAKER_01: learning how to write good prompts,
26:24
SPEAKER_01: learning how to host open source models.
26:26
SPEAKER_01: But in and of itself, the foundation is very much software engineering.
26:30
SPEAKER_00: Yeah.
26:30
SPEAKER_00: And I guess, you know, I guess evaluation is still is also a big one.
26:34
SPEAKER_00: Yes, that's a great one.
26:35
SPEAKER_01: Right.
26:35
SPEAKER_01: And get evals.
26:37
And then one thing that was really surprising for me to learn
26:40
when I talk with a
26:41
friend who works at a startup is how their test suite costs
26:45
money to run every
26:46
SPEAKER_00: time,
26:46
SPEAKER_00: the eval suite.
26:47
SPEAKER_00: They're like, I don't know how many, like $50 or something like that.
26:51
And it's like,
26:52
oh,
26:53
when I run my unit test,
26:55
it costs time and effort,
26:56
but it's free.
26:58
SPEAKER_00: It's just time.
26:58
And now you actually,
26:59
especially if you're using an API,
27:01
you have this cost,
27:02
which is...
27:03
SPEAKER_00: I think, refreshing and just a good way to think about it.
27:06
SPEAKER_00: And it just forces you to adapt.
27:09
SPEAKER_01: Yeah, for sure.
27:09
It's very interesting because there's no good way to measure
27:14
the accuracy of a
27:15
SPEAKER_01: non-deterministic model without using LLMs.
27:18
SPEAKER_01: And so at Coda, we used to use brain trust.
27:20
And it was so interesting and how the model is being used to
27:25
check whether or not
27:27
SPEAKER_01: it's working correctly.
27:30
As you're just going deeper and deeper into the AI field,
27:34
what were resources that helped you?
27:35
SPEAKER_00: Was it just pure self-learning?
27:37
SPEAKER_00: Was it going to the source of where the papers are?
27:41
This is a really ongoing question because the industry is not
27:46
slowing down and
27:46
SPEAKER_00: there's not many books or static resources out there.
27:51
SPEAKER_01: Yeah, very fair.
27:52
SPEAKER_01: Because things are changing quickly and there aren't static resources,
27:57
SPEAKER_01: at that time,
27:57
SPEAKER_01: and still true today,
27:59
SPEAKER_01: I found it most helpful to learn by just doing.
28:01
So even when I was on this team,
28:04
I'd go to a lot of hackathons,
28:05
internal to Coda and external.
28:07
I remember there was an internal hackathon at Coda where it
28:12
happened to line up
28:13
SPEAKER_01: with the day OpenAI released function calling for the first time.
28:17
SPEAKER_01: And so our team,
28:18
SPEAKER_01: we played around with the power of function calling,
28:20
SPEAKER_01: which is a very important tool,
28:22
by turning natural language prompts into appropriately
28:27
identifying what third-party
28:30
SPEAKER_01: code integration you should use.
28:32
SPEAKER_01: So for example, a user types in, how many unread emails do I have?
28:36
SPEAKER_01: And it should appropriately pull out the Gmail pack or Gmail third-party
28:40
SPEAKER_01: integration that Coda had.
28:42
SPEAKER_01: At that hackathon,
28:43
SPEAKER_01: playing around with embeddings from Pinecone to see,
28:46
SPEAKER_01: can I more accurately pull out the right third-party integration?
28:50
So that was one way through internal hackathons,
28:53
but there were also external hackathons.
28:55
I remember in SF,
28:56
when Llama 3 first came out,
28:58
they were hosting a fine-tuning hackathon.
29:02
SPEAKER_01: So I went.
29:03
The beginning,
29:03
they tell you what is fine tuning,
29:05
how to use it,
29:06
which is great.
29:07
Then there are a bunch of startups there that are building fine
29:09
tuning platforms.
29:10
SPEAKER_01: So they give you free credits to go fine tune.
29:13
And so then I remember building on top of Replicate and fine
29:16
tuning a way to turn
29:19
llama into coda formulas which is our equivalent of excel
29:23
formulas so learning by
29:25
doing to me was the most effective way when things are changing
29:28
so quickly and even
29:30
though hackathons are the most effective you know reading blogs
29:33
some paper twitter
29:34
to see what other people are doing did help there are a lot of
29:37
open source
29:37
companies i remember back in the day langchain had lovely
29:41
documentation on how to
29:42
SPEAKER_01: do rag when it was first getting popularized and so
29:45
SPEAKER_01: Reading what other people are doing,
29:47
SPEAKER_01: even though they're informally written,
29:49
SPEAKER_01: it's not a textbook,
29:50
SPEAKER_01: it's not a course,
29:51
SPEAKER_01: has been very informative as well.
29:53
SPEAKER_00: Nice.
29:54
SPEAKER_00: Well, yeah, I guess this is so new.
29:56
You just need to figure out what works for you and just try a
29:58
bunch of stuff and
29:59
SPEAKER_00: see what sticks.
30:00
SPEAKER_00: And also it changes, right?
30:01
SPEAKER_00: So like whatever works now, it might not be as efficient later.
30:04
SPEAKER_01: Totally.
30:04
SPEAKER_01: Yeah.
30:05
SPEAKER_01: And there are books coming up.
30:06
I remember you interviewed Chip and she has a lovely book on
30:08
how to build as an AI engineer.
30:11
Yeah,
30:11
she actually captured a lot of the things that are not really
30:14
changing anymore.
30:14
SPEAKER_00: So that's also changing.
30:16
SPEAKER_00: And I think,
30:16
SPEAKER_00: you know,
30:17
SPEAKER_00: we'll now see courses come out and Andrej Karpathy is doing some really,
30:22
SPEAKER_00: really in-depth courses if you have the time,
30:25
SPEAKER_00: which honestly,
30:26
SPEAKER_00: it doesn't sound like a bad time investment to do so.
30:29
SPEAKER_01: Yeah, exactly.
30:30
SPEAKER_01: With Zero to Hero.
30:31
SPEAKER_00: So at Coda,
30:32
what was your favorite project that you built using AI tools or
30:36
your favorite AI
30:37
SPEAKER_00: product?
30:38
SPEAKER_01: A project that's very close to my heart from Coda is Workspace Q&A.
30:42
SPEAKER_01: So maybe to set some context,
30:44
SPEAKER_01: at Coda,
30:45
a very common customer complaint was that I have so many
30:48
documents with my internal
30:50
SPEAKER_01: know-how of company documentation,
30:53
SPEAKER_01: but it's hard to find that document when I need it.
30:56
And about in November 2023,
30:58
RAG was getting popularized,
30:59
retrieval augmented generation.
31:01
And it struck our team that we actually had all the tools in
31:05
place to build a
31:06
SPEAKER_01: chatbot that would solve this problem.
31:09
SPEAKER_01: First,
31:09
SPEAKER_01: we had a team that had just redone our search index,
31:13
SPEAKER_01: and they put a lot of hard work into redoing that search index.
31:16
SPEAKER_01: Second, we had the infrastructure in place to call LLM tools reliably.
31:20
And third,
31:21
we had a chat bot that allowed you to,
31:24
in your Coda doc,
31:25
chat with an LLM.
31:28
SPEAKER_01: With those three things,
31:30
I was able to just glue them together in a couple days and
31:32
build a version one of a
31:33
chat bot that lets users ask questions about the content of
31:36
their workspace.
31:37
SPEAKER_00: Oh, nice.
31:38
SPEAKER_01: So I put that, you know, on Slack with a loom.
31:42
SPEAKER_01: And to my surprise,
31:42
SPEAKER_01: our CEO,
31:43
SPEAKER_01: Shashir,
31:44
SPEAKER_01: started taking interest in this and responding to that thread.
31:48
He saw a grander vision where Coda could create an enterprise
31:52
search tool.
31:53
So it's not just searching documents,
31:55
but all third-party integrations,
31:56
which Coda had a lot of.
31:58
SPEAKER_01: So ideally,
31:59
SPEAKER_01: you know,
31:59
SPEAKER_01: a sales team should be able to come in and say,
32:01
SPEAKER_01: what's my projected ARR for an account?
32:04
SPEAKER_01: And it pulls in from your Salesforce account.
32:08
SPEAKER_01: integration and answers that question for you.
32:10
SPEAKER_01: So that was exciting.
32:12
And he basically tasked a couple of us to experimentally prove
32:16
out that Coda could
32:17
SPEAKER_01: build this in four weeks.
32:19
SPEAKER_00: Oh, nice.
32:20
SPEAKER_00: And a good challenge.
32:21
SPEAKER_01: Yeah, it was a good, daunting challenge.
32:24
SPEAKER_01: It was me, my manager, the CTO, a designer, and a PM.
32:30
And it was short deadlines and high stakes because this was
32:33
going to be demoed to
32:34
SPEAKER_01: important people.
32:35
SPEAKER_01: So it was very much all hands on deck.
32:38
SPEAKER_01: On one day's notice, we flew to New York to hack together.
32:42
SPEAKER_01: And it was nights, weekends, whatever it took to make this work.
32:47
It was a very exciting time,
32:49
and I think a lot of blood,
32:51
sweat,
32:51
and tears behind it.
32:52
SPEAKER_01: But the TRDR is that it did go very well,
32:56
SPEAKER_01: and it became the birth of a second product for Coda called Coda Brain.
33:01
SPEAKER_01: From...
33:02
SPEAKER_01: January to June of 2024,
33:06
SPEAKER_01: we had a much larger initiative where 20 people were now working on it.
33:10
And it was taking that version two that that small team we had
33:14
built and making it
33:15
SPEAKER_01: a more robust thing,
33:16
SPEAKER_01: which is a very hard challenge in and of itself.
33:18
And the cherry on top was that Coda Brain was presented at
33:21
Snowflake Dev Day at the keynote.
33:23
So it was just a very exciting time to be a part of it from day
33:27
one and the world
33:30
SPEAKER_01: getting to see it at a large scale.
33:32
SPEAKER_00: Yeah,
33:32
so I'm just taking notes on how amazing it is that you joined
33:38
Coda as a new grad
33:40
SPEAKER_00: with no experience in AI engineering.
33:42
SPEAKER_00: And just frankly,
33:44
you had less experience than a lot of the experienced engineers
33:47
and software
33:47
SPEAKER_00: engineering.
33:49
SPEAKER_00: years of experience.
33:50
SPEAKER_00: But from the first day, you just kept track of the industry.
33:54
SPEAKER_00: You saw this exciting thing is coming out.
33:56
SPEAKER_00: ChatGPT, you tried it out.
33:57
SPEAKER_00: You were convinced this is going to be interesting and fun.
34:01
SPEAKER_00: You asked your manager when Coda started a team to join.
34:04
SPEAKER_00: They said no, and you just went and learned.
34:07
SPEAKER_00: And in a matter of few months,
34:08
you probably leapfrogged a lot of the people who were just kind
34:11
of waiting or not
34:13
SPEAKER_00: necessarily being as active as you are.
34:17
you you got onto this team as an early engineer and you know a
34:21
year later now when
34:22
20 people were working on this with coda you were still like
34:25
one of the earlier
34:25
ones so it just like shows me how like what you were saying not
34:30
waiting for
34:30
permission really pays off and you can just do things you can
34:34
learn things and
34:35
especially for for an innovative technology like ai and yeah
34:38
whatever we see next
34:39
it's actually valuable like the companies like a company will
34:43
value this kind of
34:44
thing because it helps them and they they want they desperately
34:46
need people like
34:48
SPEAKER_00: like you were in this case or other folks who are doing similar things.
34:51
SPEAKER_01: What is really cool is that it's so new.
34:53
So it definitely levels the playing field between all sorts of
34:57
seniorities because
34:58
SPEAKER_01: nobody knows what the right way is.
35:00
SPEAKER_01: And so we're all just figuring out together.
35:02
SPEAKER_01: And that's what makes it definitely more exciting.
35:04
SPEAKER_00: Yeah, I feel there's like two things here.
35:06
SPEAKER_00: Like if you're someone who already has some experience,
35:09
SPEAKER_00: may that be one year or 10 years or 20 years,
35:12
SPEAKER_00: that experience will eventually be applicable.
35:14
SPEAKER_00: Once you understand how this works,
35:16
SPEAKER_00: you know,
35:16
SPEAKER_00: you can take that past experience and see how it applies.
35:18
SPEAKER_00: And if you don't have experience,
35:19
it's actually not a bad thing because you're coming in with a
35:22
fresh mind and you
35:23
SPEAKER_00: will probably you will not have some of those biases of experience.
35:27
SPEAKER_00: you know,
35:27
SPEAKER_00: for example,
35:28
SPEAKER_00: a lot of software engineers who have like 10 plus years of experience,
35:31
they will know who build production system that unit testing
35:35
and automate testing
35:35
SPEAKER_00: is super efficient and a very good way to do stuff.
35:38
SPEAKER_00: Now with AI systems,
35:40
it's not necessarily the case when they're non-deterministic
35:42
and things like
35:43
for large scale system,
35:45
things like monitoring or checking out might be a better way.
35:48
I'm not sure which one it is,
35:49
but not having that bias could actually speed you up.
35:52
SPEAKER_00: So like either way,
35:53
it doesn't seem to be any downside in just figuring it out and
35:58
mastering this tool
35:59
SPEAKER_00: because it is a tool at the end of the day.
36:01
SPEAKER_01: Yeah,
36:02
SPEAKER_01: it's a new tool in our,
36:03
it's honestly a magical superpower because now it just unlocks
36:06
so many things that
36:07
SPEAKER_01: you can do on top of it.
36:09
SPEAKER_00: Yeah,
36:09
SPEAKER_00: but it feels a bit like,
36:10
SPEAKER_00: you know,
36:10
SPEAKER_00: the Harry Potter one,
36:11
SPEAKER_00: like when you watch the movies,
36:12
SPEAKER_00: like,
36:13
SPEAKER_00: you know,
36:13
SPEAKER_00: at first it sounds magical when you read the book,
36:15
SPEAKER_00: like you can do all these spells.
36:16
SPEAKER_00: But if you're a hardcore Harry Potter fan,
36:19
SPEAKER_00: you will know that there's only certain spells that you can do.
36:21
SPEAKER_00: And, you know, there's a certain thing that you need to say.
36:24
SPEAKER_00: And so there's a whole mechanic around it.
36:26
SPEAKER_00: And like for every fantasy book as well,
36:29
SPEAKER_00: when there's a magical world,
36:29
SPEAKER_00: like there are the rules and there's people who can master those rules.
36:32
SPEAKER_00: Yes.
36:33
SPEAKER_00: I feel it's a bit insane, right?
36:35
SPEAKER_00: At first it's magic, but actually it has the rules.
36:37
And once you learn it,
36:38
you can be this,
36:39
you know,
36:39
sorcerer who can...
36:41
Yeah,
36:42
exactly.
36:43
SPEAKER_00: This episode is brought to you by Cortex.io.
36:46
SPEAKER_00: Still tracking your services and production readiness in a spreadsheet.
36:49
SPEAKER_00: Real Microsoft Service is named after TV show characters.
36:52
SPEAKER_00: You aren't alone.
36:54
SPEAKER_00: Being woken up at 3 a.m.
36:55
for an incident and trying to figure out who owns what service,
36:58
that's no fun.
36:59
SPEAKER_00: Cortex is the internal developer portal that solves service ownership and
37:02
SPEAKER_00: accelerates the path to engineering excellence.
37:05
SPEAKER_00: Within minutes,
37:06
SPEAKER_00: determine who owns each service with Cortex's AI service ownership model,
37:10
SPEAKER_00: even across thousands of repositories.
37:13
SPEAKER_00: Layer ownership means faster migrations,
37:15
SPEAKER_00: quicker resolutions to critical issues like log4j,
37:17
SPEAKER_00: and fewer add here pings during incidents.
37:21
SPEAKER_00: Cortex is trusted by leading engineering organizations like Affirm,
37:24
SPEAKER_00: TripAdvisor,
37:24
SPEAKER_00: Grammarly,
37:25
SPEAKER_00: and SoFi.
37:26
Solve service ownership and unlock your team's full potential
37:29
with Cortex.
37:30
SPEAKER_00: Visit cortex.io slash pragmatic to learn more.
37:34
SPEAKER_00: That is C-O-R-T-X dot I-O slash pragmatic.
37:38
So then you had a really great run at Coda and then you did
37:43
something like you
37:44
SPEAKER_00: decided to look around the market and you blogged about this.
37:48
SPEAKER_00: You interviewed at 46 companies.
37:52
SPEAKER_00: Did I get that right?
37:53
SPEAKER_01: Yes, but there is context behind that.
37:56
SPEAKER_00: I'd love to understand how you went about interviewing,
37:59
SPEAKER_00: especially specifically for an AI position.
38:01
SPEAKER_00: What did you learn about what the market is like,
38:04
SPEAKER_00: what interviewing is like,
38:06
SPEAKER_00: what the whole scene is?
38:08
SPEAKER_00: And if you can give a little context on where you did this in terms of
38:11
SPEAKER_00: location-wise,
38:11
SPEAKER_00: types of companies,
38:12
SPEAKER_00: just to help us all understand this.
38:15
SPEAKER_01: Sure, maybe just by giving a little bit of context.
38:18
SPEAKER_01: It was over a six month period and the first half, I wasn't closing them.
38:23
I was getting no's as I was getting revved up on my leak coded
38:26
system design prep.
38:27
SPEAKER_01: After that,
38:28
SPEAKER_01: the interview process did take longer than I expected though,
38:31
SPEAKER_01: because the AI space is especially noisy right now.
38:34
And when I was trying to do my due diligence,
38:36
like we were talking about earlier,
38:38
There were often open questions that made me feel uneasy about
38:42
the growth potential.
38:43
SPEAKER_01: And the advice I got from a mentor was that if it's not, heck yes.
38:46
SPEAKER_01: And if you have savings, don't join.
38:48
SPEAKER_01: It's not fair to the company or you.
38:49
SPEAKER_01: So that was how I thought about this.
38:51
SPEAKER_01: In terms of...
38:53
SPEAKER_01: the space,
38:55
SPEAKER_01: it was clear that there are the product companies,
38:57
SPEAKER_01: infrastructure companies,
38:59
SPEAKER_01: and the model companies.
39:00
I found it helpful to put companies in each category and figure
39:05
out which segment
39:06
SPEAKER_01: you're most excited about to help narrow down the options,
39:10
SPEAKER_01: given that there's so many AI companies right now.
39:12
Could you give just an example of each,
39:14
especially with the infrared model?
39:15
I think it might be a bit,
39:17
I'm interested in how you're thinking about that.
39:19
SPEAKER_01: Yeah, product companies are the companies building on top of the model.
39:22
SPEAKER_01: Here I think of Cursor, Codium, Hebbia.
39:25
Infrastructure companies are the companies building the tools
39:29
to help AI product
39:30
SPEAKER_01: companies effectively use LLMs.
39:33
SPEAKER_01: So whole suite of these.
39:34
SPEAKER_01: There are the inference providers like modal fireworks together.
39:38
SPEAKER_01: Vector database companies like Pinecone,
39:40
SPEAKER_01: ChromaDB,
39:41
SPEAKER_01: Weviate,
39:42
SPEAKER_01: eval and observability tools like Braintrust,
39:44
SPEAKER_01: Arise,
39:45
SPEAKER_01: Galileo,
39:45
SPEAKER_01: and a whole other suite of products.
39:48
SPEAKER_01: And then there's the model companies,
39:50
SPEAKER_01: which are the base of the ecosystem building the intelligence.
39:53
SPEAKER_01: You have the big tech companies like Google, Meta, building models.
39:57
SPEAKER_01: And then you also have startups like,
39:59
SPEAKER_01: or not startups,
39:59
SPEAKER_01: you have other big,
40:00
SPEAKER_01: smaller companies like OpenAI,
40:02
SPEAKER_01: Anthropic,
40:03
SPEAKER_01: building models as well.
40:05
SPEAKER_00: I think it's a really good way to think about it.
40:07
SPEAKER_00: And again, I don't think many of us have verbalized it like this.
40:12
SPEAKER_00: This also goes back to not many people have necessarily gone through.
40:16
SPEAKER_01: I will say this is not something that I came up with myself.
40:19
Yash Kumar,
40:20
a mentor,
40:20
he pointed out that you should look at the space like this.
40:23
SPEAKER_01: And that's how I think about it now.
40:24
SPEAKER_00: Wonderful.
40:25
And what did you learn about each of these companies in terms
40:28
of the interview
40:29
SPEAKER_00: process,
40:30
SPEAKER_00: what the vibe was like,
40:33
SPEAKER_00: generally,
40:33
SPEAKER_00: and also how you personally felt about it?
40:35
Because as I understand where you were,
40:38
Coda,
40:38
we can put them in the product category.
40:41
SPEAKER_00: Sorry, the product company category.
40:42
SPEAKER_01: So for me,
40:43
SPEAKER_01: in trying to be more focused in my search,
40:47
I decided to focus on model and infrastructure companies
40:49
because I wanted to keep
40:51
SPEAKER_01: getting breath in what I was doing.
40:53
And I felt like the product companies were too similar to my
40:55
experience at Coda,
40:56
SPEAKER_01: which was phenomenal,
40:57
SPEAKER_01: but I wanted to keep growing.
40:58
SPEAKER_01: And that definitely,
41:00
the trade-off was that it's a bit more of an uphill battle
41:02
because the work that I
41:03
SPEAKER_01: had done was not as relevant to model or infrastructure companies.
41:07
SPEAKER_01: In terms of the vibe,
41:08
SPEAKER_01: I think all of them are shipping really fast,
41:11
SPEAKER_01: have really lean teams,
41:13
SPEAKER_01: and...
41:14
SPEAKER_01: are out to win.
41:14
SPEAKER_01: So it's a very exciting time to be looking at them.
41:17
Questions I would ask myself when I was trying to figure out is
41:20
this company viable
41:22
in the long run on the infrastructure side was are their
41:25
margins high enough given
41:26
SPEAKER_01: that so many of these inference providers
41:30
SPEAKER_01: are also paying for expensive GPUs.
41:33
SPEAKER_01: So what is the margins here,
41:34
especially when a good software business should have about 70%
41:37
gross margins?
41:38
SPEAKER_01: And how easy is it to build infrastructure in-house?
41:41
You know,
41:42
we know this,
41:43
but engineers are a hard group of people to sell to because...
41:47
SPEAKER_01: If it's not complex enough,
41:49
SPEAKER_01: if it's too expensive or doesn't work exactly how they want,
41:51
SPEAKER_01: engineers will just build it in-house.
41:53
SPEAKER_01: Google is a phenomenal example that's built so much in-house.
41:57
SPEAKER_01: So that's how I was thinking about the infrastructure companies.
41:59
SPEAKER_01: In terms of the model companies,
42:01
SPEAKER_01: I was just trying to get a sense of if they're training frontier models,
42:04
SPEAKER_01: can they afford to keep training them given how expensive it is?
42:08
SPEAKER_01: Are they staying ahead of the open source competition?
42:12
SPEAKER_01: Because if there are open weights that exist for a model,
42:15
no one's going to want to pay a premium to get the model from a
42:19
closed source
42:19
SPEAKER_01: provider.
42:20
SPEAKER_00: It's a sad reality.
42:22
SPEAKER_01: It is.
42:22
And I think that it's interesting because today product
42:25
companies are still willing
42:26
SPEAKER_01: to pay a premium for the best model,
42:29
SPEAKER_01: even though an open weight exists,
42:32
SPEAKER_01: as long as the closed source provider is ahead.
42:36
SPEAKER_00: Yes.
42:36
SPEAKER_00: And anyone who's nodding along,
42:38
when they'll find themselves evaluating an offer or a company
42:42
and trying to
42:42
SPEAKER_00: understand the margins,
42:43
SPEAKER_00: that's a hard one to do,
42:44
SPEAKER_00: especially as an engineer.
42:45
SPEAKER_01: Yeah, exactly.
42:46
Where did you get data or did companies answer some of your
42:50
questions on the unit economics?
42:52
SPEAKER_00: These are things that companies like to have under wraps,
42:54
even as someone who is covering sometimes these companies or
42:58
just interested in
42:59
SPEAKER_00: this space,
42:59
even publications like financial publications will just kind of
43:04
wave their hands
43:05
SPEAKER_00: because it is hard.
43:06
SPEAKER_00: Like this is the big question.
43:07
And these companies,
43:08
they want to hide these things from the casual observer for
43:12
sure.
43:13
SPEAKER_01: Exactly.
43:14
I think it's totally fair for a company not to share this
43:16
information until you get
43:17
SPEAKER_01: an offer because it is sensitive information.
43:19
SPEAKER_01: I do think once you have an offer,
43:21
it would be irresponsible for them not to tell you when you are
43:24
as an investor as
43:24
SPEAKER_01: well.
43:25
SPEAKER_01: And you sign an NDA, so you keep it to yourself.
43:27
SPEAKER_01: So I do think they should tell you.
43:28
SPEAKER_01: For questions or for companies in the inference space,
43:31
SPEAKER_01: I would just ask,
43:32
SPEAKER_01: you know,
43:32
SPEAKER_01: how much money do you spend on the GPUs?
43:34
And then how much revenue do you have to make rough back of the
43:38
envelope math of
43:39
SPEAKER_01: what those margins are like?
43:40
SPEAKER_01: Just get some sense of the business.
43:43
And then I also found it helpful to read some news providers
43:46
like the information
43:47
that does very good diligence on the economics behind different
43:53
startups in the AI
43:54
SPEAKER_01: space.
43:55
SPEAKER_01: And if I could,
43:56
I would try to also ask investors who have invested in these
44:00
companies or passed on
44:02
SPEAKER_01: investing in these companies because they see the deck come to them.
44:07
SPEAKER_01: So they have a lot more insight onto what the business really looks like.
44:11
You're talking like an investor or like how a senior executive
44:17
would do it,
44:18
which I love.
44:18
I think more people should be doing this,
44:20
by the way,
44:21
and not enough people are doing it.
44:22
SPEAKER_00: So it's just very refreshing to hear.
44:25
SPEAKER_00: And by the way,
44:26
SPEAKER_00: like the investor store is interesting because in my experience,
44:29
SPEAKER_00: investors,
44:30
SPEAKER_00: when you are
44:31
SPEAKER_00: applying to a company that they're an investor in,
44:32
SPEAKER_00: they actually want to help close great people.
44:34
SPEAKER_01: Yes, exactly.
44:35
SPEAKER_00: And they will happily connect.
44:37
SPEAKER_00: And then you also have a connection where a few years down the road,
44:41
SPEAKER_00: that investor might reach out to you saying,
44:43
SPEAKER_00: oh,
44:43
SPEAKER_00: I remember,
44:43
SPEAKER_00: you're a business-minded engineer.
44:46
SPEAKER_00: You know, like in the future, it's hard to tell.
44:48
SPEAKER_00: I think we were talking about this before,
44:49
SPEAKER_00: what will be in the future,
44:50
but there will be only more demand for software engineers who
44:53
not only know how to
44:54
SPEAKER_00: code,
44:54
SPEAKER_00: but are curious about the business,
44:56
SPEAKER_00: can communicate with users,
44:57
SPEAKER_00: et cetera.
44:58
SPEAKER_00: So you will now have a network, a stronger network.
45:01
SPEAKER_00: So there's only upside in doing your due diligence.
45:03
SPEAKER_00: It can actually help your career.
45:04
SPEAKER_01: That's true.
45:05
And I 100% agree with investors being very generous with their
45:09
time in wanting to
45:09
SPEAKER_01: chat with you and explain to you how the business works.
45:12
SPEAKER_01: So that's been something that's been really fun to do for sure.
45:15
SPEAKER_00: And then just going back to like,
45:18
SPEAKER_00: this is all great when you get an offer,
45:19
SPEAKER_00: but how did you get to getting an offer?
45:20
SPEAKER_00: Like what did you need to brush up on in terms of interviews?
45:23
SPEAKER_00: Was it the pretty typical,
45:25
SPEAKER_00: you know,
45:25
SPEAKER_00: tech interviews,
45:26
even though these were for AI engineering roles of the lead
45:29
code system design or
45:30
SPEAKER_00: for some AI specific things?
45:32
SPEAKER_00: You know,
45:32
what helped you go from initially you stumbled and you didn't
45:35
get too much to like,
45:36
SPEAKER_00: okay,
45:36
SPEAKER_00: you actually,
45:37
SPEAKER_00: like we're getting offers now.
45:39
SPEAKER_01: In terms of the interview process,
45:41
I definitely thought it was all over the place as the market is
45:47
trying to move away
45:48
SPEAKER_01: from leak code,
45:49
SPEAKER_01: but still asks leak code.
45:50
SPEAKER_01: So then you end up having to study leak code as well,
45:53
SPEAKER_01: unless you know exactly where you're applying to.
45:55
SPEAKER_01: So there were coding interviews, system design, and then projects.
46:00
Coding was a mix of data structures and algorithms where the
46:03
best way to do it is LeetCode.
46:06
SPEAKER_01: Luckily, NeetCode with an N now exists and he has phenomenal videos.
46:10
SPEAKER_01: So that was great.
46:11
SPEAKER_01: I believe in doing space repetition.
46:12
SPEAKER_01: So doing those questions a lot of times.
46:14
Then there were front end questions because I'm a full stack
46:17
engineer as well.
46:18
SPEAKER_01: And I found that there was this resource,
46:20
SPEAKER_01: the great front end,
46:21
that had lovely interview questions for the obscure JavaScript
46:24
questions they
46:25
SPEAKER_01: sometimes ask.
46:27
SPEAKER_01: On the back end,
46:28
SPEAKER_01: that one,
46:28
I just more relied on things that I had done at work for those
46:31
interviews.
46:32
SPEAKER_01: That's the coding part.
46:33
The system design part,
46:34
I thought Alex Hsu's system design,
46:36
his two books,
46:37
phenomenal.
46:38
SPEAKER_01: Just reading those,
46:38
SPEAKER_01: really understanding them,
46:39
doing them again and again until you understand why things work
46:44
a certain way.
46:44
SPEAKER_01: Honestly, I love system design interviews.
46:46
They're really fun because you learn things outside of the
46:49
domain that you're in as well.
46:50
SPEAKER_01: And then there are the third type of interviews,
46:52
SPEAKER_01: which is project interviews where go build something in a day.
46:55
SPEAKER_01: And those...
46:56
are probably my favorite out of all of them because you get to
46:58
show how passionate
46:59
you are about that specific product and you can actually show
47:02
what you can do.
47:03
SPEAKER_01: I do hope that as an industry,
47:05
SPEAKER_01: we move away from leak code and instead move to just project interviews,
47:09
SPEAKER_01: reading code,
47:10
SPEAKER_01: which has become way more important today,
47:12
SPEAKER_01: as well as debugging code.
47:13
SPEAKER_01: But I think we're kind of in the interim where as an industry,
47:16
SPEAKER_01: we haven't fully formed an opinion here.
47:19
And then most of these interviews was at the end of last year,
47:21
so end of 2024 or so.
47:24
SPEAKER_00: Were they remote or were somewhere in person already?
47:27
SPEAKER_01: It was between June of last year and a large chunk were remote,
47:35
SPEAKER_01: but there were definitely interviews in person as well,
47:38
which I enjoy because I was very much optimizing for companies
47:41
that are in person.
47:43
SPEAKER_00: Yeah, we'll see.
47:44
But I think we're sensing a trend or I'm sensing a trend that
47:47
in-person interviews
47:48
SPEAKER_00: might be starting to go back,
47:48
SPEAKER_00: at least your final rounds,
47:49
SPEAKER_00: which,
47:50
SPEAKER_00: by the way,
47:51
SPEAKER_00: it might not be a bad thing.
47:52
SPEAKER_00: I mean,
47:52
SPEAKER_00: it's interesting because before COVID,
47:54
SPEAKER_00: like when I spent most of my career there,
47:58
SPEAKER_00: it was just in-person.
47:59
SPEAKER_00: And there are so many upsides, right?
48:01
SPEAKER_00: You do meet the people, you do see the location.
48:03
SPEAKER_00: Oftentimes you meet your future teammates.
48:05
And for example,
48:06
for me,
48:07
I once in London,
48:08
I had two offers between two banks and
48:12
SPEAKER_00: And in one case, I met my future team, the whole team.
48:15
SPEAKER_00: And when I didn't meet my future team,
48:17
SPEAKER_00: it was just like they said,
48:18
SPEAKER_00: like,
48:18
SPEAKER_00: you will be assigned a team.
48:20
SPEAKER_00: And I actually chose,
48:21
SPEAKER_00: it was a lower salary,
48:22
SPEAKER_00: but I chose a lower salary because I really liked the people.
48:24
SPEAKER_00: And, you know, like we just kicked it off.
48:26
SPEAKER_00: It felt like a good connection.
48:28
SPEAKER_00: And back then I went to a recruiter.
48:30
So the recruiter negotiated the same salary for me,
48:32
which was kind of a win,
48:33
I guess.
48:35
SPEAKER_00: But like there are like,
48:36
SPEAKER_00: I know there's,
48:37
SPEAKER_00: you know,
48:37
SPEAKER_00: like it's always,
48:39
SPEAKER_00: we will hear people like mourning the end of,
48:42
SPEAKER_00: or fewer remote interviews,
48:44
SPEAKER_00: but there are all these upsides,
48:45
SPEAKER_00: which when you're committing to a place for so many,
48:47
SPEAKER_00: for hopefully many years,
48:50
SPEAKER_00: you want to have all that information.
48:51
SPEAKER_00: 100%.
48:52
SPEAKER_01: Definitely.
48:53
SPEAKER_01: I think it's energizing on both ends for sure.
48:55
SPEAKER_01: It's a great point.
48:56
SPEAKER_00: And so in the end, you joined OpenAI, right?
48:59
SPEAKER_01: Yes, I did.
49:00
SPEAKER_00: Congratulations.
49:01
SPEAKER_01: Thank you.
49:02
SPEAKER_00: And then can you share on what kind of general work you do at OpenAI?
49:06
SPEAKER_01: Sure.
49:07
SPEAKER_01: So I work on safety as an engineer at OpenAI.
49:10
And OpenAI's goal and mission is to build AGI that benefits all
49:14
of humanity.
49:14
On safety,
49:15
we focus on the suffix of that statement,
49:17
so benefiting all of humanity.
49:20
SPEAKER_01: Some things I work on are...
49:23
SPEAKER_01: A,
49:24
SPEAKER_01: small,
49:26
low latency classifiers that detect when the model or users are
49:31
doing things that
49:31
SPEAKER_01: are harmful so that you can block life.
49:34
So that means the training,
49:35
the data flywheel,
49:36
hosting these models to scale.
49:39
Second thing that I get to work on is measuring when the models
49:43
are being harmful in the wild.
49:45
SPEAKER_01: And there are a lot of dual use cases over here,
49:47
but really trying to get a sense as these models become more
49:50
capable and people are
49:51
SPEAKER_01: figuring out different ways to jailbreak them and exploit them.
49:54
What are those unknown harms that we don't know of with more
49:59
powerful models?
50:00
SPEAKER_01: And then
50:01
SPEAKER_01: distilling it into small classifiers.
50:03
SPEAKER_01: There's also on my team, a lot of safety mitigation services that we own.
50:07
And so part of our work is to integrate it with all the
50:10
different product launches.
50:11
SPEAKER_01: And as you know,
50:12
there are a lot of different product launches that definitely
50:14
keeps our team busy.
50:15
SPEAKER_01: And that's just the tip of the surface.
50:16
SPEAKER_01: There are a lot more things that we work on in safety.
50:18
SPEAKER_00: I mean,
50:19
SPEAKER_00: this is like,
50:20
it sounds very interesting because when I worked on payments
50:22
back at Uber,
50:23
SPEAKER_00: we had a team called Fraud and oh boy,
50:26
SPEAKER_00: they had so many stories.
50:27
SPEAKER_00: Like I just talking with them, I
50:29
like you would think you know like payments pretty simple like
50:33
oh you just need to
50:33
pay but then the edge cases are always interesting with every
50:37
every area and the
50:38
same thing i guess with l i mean elements are not as simple but
50:41
once you realize
50:42
how they work next token prediction it sounds pretty simple but
50:44
then the edge cases
50:45
and all the things that could go wrong etc and sounds like
50:47
you're kind of in the
50:47
middle of that like having a very like good vantage point and
50:51
actually in in in the
50:52
SPEAKER_00: details
50:53
you've now worked at coda you've you've interned at google and
50:56
and microsoft and
50:57
you talk with mentors about like what other places are what are
51:00
things that you
51:00
feel that are just very kind of distinctly different about open
51:03
ai compared to
51:04
SPEAKER_00: other companies
51:05
I think what makes OpenAI unique is the mix of speed and
51:11
building for scale.
51:12
SPEAKER_01: You know, at startups, you get that speed of iteration and it's so fun.
51:15
SPEAKER_01: And then at bigger places, you get to build for scale.
51:19
SPEAKER_01: But OpenAI is in a very unique spot where you have both at the moment.
51:22
SPEAKER_01: Things move really fast and you have huge amounts of users.
51:28
SPEAKER_01: The service that I work on, you know, gets 60K requests per second.
51:32
SPEAKER_01: And you just think...
51:34
SPEAKER_01: Normally you get one or the other, and it's really fun to get both.
51:39
Second thing that I think is quite unique for a company of this
51:44
size is the open culture.
51:45
People are very open to answering questions on why and how
51:49
things work a certain way.
51:51
So it's a great place to learn,
51:53
which is something I didn't realize from the outside.
51:56
And then third,
51:56
people are just very passionate about the mission,
51:59
work really hard.
52:00
SPEAKER_01: And I don't think this is unique to OpenAI in and of itself.
52:03
SPEAKER_01: All companies,
52:04
SPEAKER_01: I think where great work is happening,
52:05
SPEAKER_01: people are like this,
52:06
but it's just never a boring day in the office because people
52:10
care so much and are
52:11
SPEAKER_01: constantly shipping.
52:13
SPEAKER_00: Yeah,
52:13
SPEAKER_00: and then talking about shipping,
52:15
SPEAKER_00: like you've,
52:16
SPEAKER_00: I'm assuming you ship some things to production already,
52:19
SPEAKER_00: but how can we imagine a thing,
52:21
SPEAKER_00: a project,
52:21
SPEAKER_00: an idea making it into production,
52:23
SPEAKER_00: right?
52:23
SPEAKER_00: Like there's a very bureaucratic companies,
52:27
SPEAKER_00: you know,
52:27
SPEAKER_00: I don't want to like say old Microsoft,
52:29
SPEAKER_00: maybe not today,
52:30
SPEAKER_00: but where there's like,
52:31
SPEAKER_00: you know,
52:31
SPEAKER_00: like very strict planning process,
52:34
SPEAKER_00: then Jira tickets are created by the PM,
52:36
SPEAKER_00: the engineers have to pick it up,
52:38
SPEAKER_00: then someone else might actually deploy it.
52:40
SPEAKER_00: So like, this is the super, like,
52:42
SPEAKER_00: old school and slow and the reason why some engineers don't like it.
52:46
SPEAKER_00: What is it like?
52:46
You mentioned it's fast,
52:48
but what was your experience in getting things from idea to
52:52
production?
52:52
SPEAKER_00: And is it multiple teams?
52:54
SPEAKER_00: Can one person actually do it?
52:55
SPEAKER_00: Is it even allowed?
52:56
SPEAKER_00: I don't know.
52:57
SPEAKER_01: I think it's very much allowed and very much encouraged.
53:01
SPEAKER_01: There's been publications of how deep research came to be,
53:06
SPEAKER_01: where it was an engineer hacking on something,
53:08
SPEAKER_01: presenting it to larger C-suite and now becoming a full,
53:11
SPEAKER_01: very impactful product.
53:13
SPEAKER_01: So it's definitely encouraged, which I love.
53:15
I too have had a similar experience and it's very encouraged to
53:18
come with ideas and
53:20
SPEAKER_01: actually drive them forward.
53:22
SPEAKER_00: Just strictly from your perspective,
53:25
what do you think like one thing that stands out that OpenAI
53:28
can actually still
53:29
SPEAKER_00: ship so fast?
53:30
Because it feels it defies a little bit the laws of a growing
53:33
organization,
53:34
SPEAKER_00: which eventually slows down.
53:35
At one point,
53:36
I'm sure it will,
53:37
but there's no signs of this happening so far.
53:39
My observation is that the systems are built to enable you to
53:45
ship fast and they
53:46
SPEAKER_01: give engineers a lot of trust,
53:48
even though it comes with the downside of sometimes that can
53:51
lead to outages.
53:53
SPEAKER_01: To put this very concretely,
53:56
SPEAKER_01: When I joined, you can make static changes without an approval.
53:59
SPEAKER_01: So you have trust to go in and flip a flag to turn something on.
54:03
SPEAKER_01: That's no longer the case.
54:04
SPEAKER_01: You need one reviewer.
54:05
SPEAKER_01: The service that I get to work on has 60,000 requests per second,
54:09
SPEAKER_01: but you get to deploy with one review immediately.
54:12
So my observation is that there is truly trust put in engineers
54:17
to work quickly and
54:19
SPEAKER_01: not have a lot of red tape around shipping fast.
54:22
SPEAKER_00: Yeah,
54:22
I think this just goes with kind of onset expectation that
54:26
expectation will be very
54:27
high of the people who come in here because you cannot hire an
54:30
engineer who is used
54:31
SPEAKER_00: to,
54:31
SPEAKER_00: you know,
54:32
SPEAKER_00: being only doing a small part,
54:34
not used to thinking about the product and the business impact
54:37
and all those
54:37
SPEAKER_00: things.
54:38
SPEAKER_00: So I have a sense that
54:40
SPEAKER_00: what you're doing,
54:40
SPEAKER_00: it might be kind of a given for you,
54:41
SPEAKER_00: but in the industry,
54:42
SPEAKER_00: it might be more common to expect that engineers are just wearing,
54:45
SPEAKER_00: you know,
54:45
SPEAKER_00: we used to call it wearing more hats,
54:47
SPEAKER_00: but it's just like,
54:48
SPEAKER_00: it's just how it is.
54:48
SPEAKER_00: Like you do want to have,
54:51
SPEAKER_00: you know,
54:52
SPEAKER_00: like you're kind of merging a little bit of PM,
54:54
SPEAKER_00: a data scientist,
54:55
SPEAKER_00: an engineer all in one.
54:57
And these are the type of people who can actually make
55:00
something like OpenAI or
55:02
SPEAKER_00: similar companies like work so well with this many people.
55:05
SPEAKER_01: Yeah, and I just think with intelligence,
55:08
SPEAKER_01: Today,
55:09
SPEAKER_01: the roles between data science,
55:12
SPEAKER_01: engineering,
55:13
SPEAKER_01: backend,
55:13
SPEAKER_01: frontend,
55:14
SPEAKER_01: PM,
55:14
SPEAKER_01: blurs so much that each individual,
55:18
SPEAKER_01: whether you're in OpenA or not,
55:19
is expected to do more of that because you can get help from a
55:24
very capable model.
55:25
And I think that makes it very exciting for us because it means
55:27
that we can truly
55:28
SPEAKER_01: be full stack engineers and go from an idea to launch very quickly.
55:33
SPEAKER_00: Absolutely.
55:34
SPEAKER_00: So what are some things that you've learned about AI engineering,
55:37
SPEAKER_00: the kind of the realities of it?
55:39
SPEAKER_00: Because it's a very new field.
55:41
SPEAKER_00: And what are some surprising things that you didn't quite expect?
55:44
One thing that I've learned that I didn't realize coming in was
55:48
how much of AI
55:50
engineering is about building solutions to known limitations of
55:55
the model.
55:56
And then as the model gets better,
55:57
you scrap that work and build new guardrails.
56:00
SPEAKER_01: Let me give you an example from Coda.
56:02
SPEAKER_01: So pre-function calling days,
56:04
we wanted a way to get our model to take action based on what
56:09
the user said.
56:09
SPEAKER_01: Function calling didn't exist,
56:11
SPEAKER_01: so we prompted the model to return JSON,
56:13
SPEAKER_01: parse that,
56:14
SPEAKER_01: and actually deterministically call an action based on that JSON blob.
56:20
SPEAKER_01: Then OpenAI released function calling.
56:21
SPEAKER_01: Okay, scrap that and instead integrate with function calling.
56:24
But,
56:25
you know,
56:25
back in those days,
56:26
function calling was not very reliable.
56:27
SPEAKER_01: And now today we moved from function calling to the MCP paradigm.
56:33
SPEAKER_01: So things are changing very quickly and the models are getting better,
56:37
SPEAKER_01: but they're still not perfect.
56:39
SPEAKER_01: The moment you get more capability,
56:41
there are more engineering guardrails you need to build to make
56:43
sure they work
56:44
SPEAKER_01: reliably at scale.
56:45
SPEAKER_00: Yeah,
56:46
and I guess you need to become comfortable with throwing away
56:49
your work when the
56:50
SPEAKER_00: model is there.
56:51
You just need to not be as attached to it because I think
56:54
there's a little bit of
56:55
SPEAKER_00: this,
56:55
especially when you're used to like things not changing as much
56:57
as software
56:57
SPEAKER_00: engineering.
56:58
SPEAKER_00: So just, you know, like it's not a waste, it's a learning.
57:03
SPEAKER_01: Yeah, and it's just been easier now to, or cheaper to produce code.
57:08
And so you see this expansion and collapse phase happen a lot
57:13
in where you build a
57:13
SPEAKER_01: lot of features,
57:14
SPEAKER_01: see what works,
57:15
SPEAKER_01: and then collapse to what works and restart.
57:17
There's a lot of scrapping your work as the creation of code
57:20
becomes cheaper.
57:21
It's easier not to be attached when an LLM also helped generate
57:24
that code.
57:25
Yeah,
57:26
I think this will be a big change,
57:27
a good change,
57:29
once we get used to it.
57:31
SPEAKER_01: Yes, exactly.
57:32
SPEAKER_00: Now,
57:32
SPEAKER_00: when it comes to AI and junior engineers,
57:35
you're such an interesting example in the sense that you
57:37
started your career a
57:38
SPEAKER_00: little bit before AI took off,
57:41
SPEAKER_00: but you also transitioned with not decades of experience just yet.
57:48
What is your take on how Gen AI will impact new grads,
57:53
people who are still in college?
57:54
SPEAKER_00: Because, you know, there's two takes and they're both very extreme.
57:57
SPEAKER_00: One is the engineers with 10 plus years experience often just feel like,
58:01
SPEAKER_00: oh,
58:01
SPEAKER_00: I feel so sorry for these people.
58:02
SPEAKER_00: Yeah.
58:03
SPEAKER_00: They're not going to get jobs.
58:04
SPEAKER_00: Even if they get jobs, they're not going to depend on AI.
58:07
SPEAKER_00: They're not going to read the books.
58:08
SPEAKER_00: They won't know what it was exact back in our day.
58:11
SPEAKER_00: So there's this thing.
58:14
SPEAKER_00: And also,
58:16
SPEAKER_00: some people are generally worried that,
58:18
SPEAKER_00: well,
58:18
SPEAKER_00: you can now outsource so many things to AI.
58:21
SPEAKER_00: They're thinking,
58:21
SPEAKER_00: okay,
58:21
SPEAKER_00: maybe they can pick up things really quickly,
58:23
SPEAKER_00: but maybe they're never going to get to that depth.
58:25
SPEAKER_00: Now, I think they both are extreme.
58:27
I'd love to hear how you see it because you're kind of seeing
58:30
this firsthand.
58:31
SPEAKER_00: Yeah.
58:31
SPEAKER_01: Definitely.
58:32
SPEAKER_01: And you're right.
58:33
From my experience level,
58:34
I get insight into what both of those engineering lives are
58:38
like.
58:40
SPEAKER_01: And currently,
58:40
I'm not convinced that AI is going to be disproportionately
58:44
worse for junior
58:45
SPEAKER_01: engineers.
58:46
In fact,
58:47
I think that allows everyone to move higher into the stack
58:52
and...
58:53
SPEAKER_01: be more creative in what you're building.
58:55
You empower younger engineers to just do more,
58:59
propose ideas and actually ship that.
59:03
I do subscribe to the take that there will be people that use
59:06
AI to learn and then
59:08
SPEAKER_01: people that use AI to avoid learning.
59:11
I would say that there's actually room for both things to exist
59:14
and you should be doing both.
59:16
I personally think that when you're working on a greenfield
59:18
project trying to prove
59:19
SPEAKER_01: a vision that something should exist,
59:22
SPEAKER_01: why not skip the learning,
59:24
vibe code it to actually get a real product that you can
59:27
validate and then go run
59:29
SPEAKER_01: to build this for real as a new product line.
59:32
SPEAKER_01: But I don't think you should skip the learning when you're trying to
59:36
now build a robust system that you are the owner of because
59:40
when shit hits the fan
59:41
and you're in a sev ai doesn't help that much because it
59:44
doesn't work very well in
59:46
between at a high systems level and then you know reading logs
59:49
so when you own the
59:50
code i think you should use ai to learn to understand all the
59:53
edge cases of why
59:54
things work a certain way so i think there's room for both it's
59:58
going to be an
59:58
SPEAKER_01: important skill for us to learn when we should outsource
1:00:03
the doing versus when we should use AI to make ourselves better
1:00:07
and stronger engineers.
1:00:09
SPEAKER_00: Yeah.
1:00:09
SPEAKER_00: And I guess there's probably not too much harm.
1:00:11
SPEAKER_00: And if you don't understand it, spend some time to understand it.
1:00:15
SPEAKER_00: And AI will help you typically do this faster.
1:00:19
SPEAKER_00: So I'm not sure if this is to do with personality or curiosity, but I
1:00:26
SPEAKER_00: We've seen this before,
1:00:27
SPEAKER_00: by the way,
1:00:28
SPEAKER_00: like when at any time,
1:00:29
SPEAKER_00: but let's say 10 years before when I was like maybe a mid-level engineer,
1:00:33
SPEAKER_00: like I saw new grads join the workforce and we were now using,
1:00:37
SPEAKER_00: you know,
1:00:37
SPEAKER_00: higher level languages like JavaScript or Python or TypeScript.
1:00:40
SPEAKER_00: Or take the example of the recent,
1:00:44
SPEAKER_00: you know,
1:00:44
SPEAKER_00: a few years ago,
1:00:45
SPEAKER_00: like new grad engineers,
1:00:47
SPEAKER_00: they start with React.
1:00:48
And when you start with React,
1:00:50
JavaScript and TypeScript,
1:00:51
a lot of people who would
1:00:53
haven't studied computer science and didn't do assembly or C++
1:00:56
or these kind of
1:00:57
SPEAKER_00: things,
1:00:57
you can just learn React and you can just stay there and you
1:00:59
can figure out how to
1:01:00
SPEAKER_00: use it.
1:01:01
SPEAKER_00: But the better developers have always asked, why does it work like this?
1:01:06
SPEAKER_00: What happens?
1:01:07
SPEAKER_00: What is a virtual DOM?
1:01:09
SPEAKER_00: How can I do it?
1:01:09
SPEAKER_00: How can I manipulate it?
1:01:11
and and you look at the source code and i feel there's always
1:01:14
been the people who
1:01:15
do this and they're just better engineers eventually they can
1:01:17
debug faster they
1:01:19
they ask why and you know they're still sore so i think in this
1:01:22
new world we will
1:01:23
just have this and i don't i don't think this trait will will
1:01:25
die out in fact you
1:01:26
know like to me you're a great proof of you know you you go
1:01:29
deep you understand how
1:01:30
how the things work and then you think you decide like okay i'm
1:01:33
i'm gonna use it to
1:01:34
my advantage right now i just gonna want to go fast because i
1:01:36
know what i'm doing
1:01:37
SPEAKER_00: all
1:01:37
SPEAKER_01: Yes, I do think that's spot on and that we've had this in the past.
1:01:41
It will become even easier to outsource that intelligence and
1:01:46
in some sense be lazy.
1:01:48
So I think we'll have to just be more intentional as engineers
1:01:51
to make sure that we
1:01:52
SPEAKER_01: are actually going deep in cases where it really matters.
1:01:57
SPEAKER_00: And so far from what you're seeing,
1:01:59
SPEAKER_00: how,
1:02:00
SPEAKER_00: because you've seen before Gen.AI tools,
1:02:03
SPEAKER_00: you're now working at a,
1:02:04
SPEAKER_00: you've been a product engineer with AI,
1:02:06
SPEAKER_00: you're now working at a model company.
1:02:08
How do you think these tools will change the software engine
1:02:11
that you have been doing before?
1:02:13
SPEAKER_00: And how is it already changing your day-to-day work?
1:02:16
In terms of what doesn't change,
1:02:17
we still have code as the way innovation manifests.
1:02:21
SPEAKER_01: You go from idea to code to iterate.
1:02:23
SPEAKER_01: That's the same.
1:02:25
As engineers,
1:02:26
you still need to know how high-level systems work and design
1:02:29
them very well.
1:02:31
You have to debug code really well and you have to be able to
1:02:34
be really good at reading code.
1:02:35
SPEAKER_01: So to me, that all stays the same.
1:02:38
SPEAKER_01: What's changed,
1:02:39
SPEAKER_01: I think,
1:02:40
SPEAKER_01: is the division of responsibilities between PM designer,
1:02:43
SPEAKER_01: software engineer.
1:02:44
SPEAKER_01: I was talking to a friend at Decagon.
1:02:45
I think they were telling me there are 100 people and they
1:02:47
still don't have a
1:02:48
SPEAKER_01: designer because product is just expected to do the design as well.
1:02:53
SPEAKER_01: As a software engineer,
1:02:54
SPEAKER_01: this has always been true at startups,
1:02:55
SPEAKER_01: but now more than ever,
1:02:57
SPEAKER_01: you're expected to do product work as well.
1:02:59
SPEAKER_01: We talked about this earlier.
1:03:00
SPEAKER_01: What also changed is that software engineers become more full stack.
1:03:03
SPEAKER_01: You don't outsource...
1:03:06
work to another adjacent role like data engineer you're
1:03:09
expected to build those
1:03:10
data pipelines yourself i also think what's changed is that we
1:03:14
need to be better at
1:03:15
articulating our software engineering architectures and
1:03:19
thoughts because you are
1:03:20
expected to prompt models to do this and the engineers that
1:03:23
will be most efficient
1:03:25
SPEAKER_01: are the ones that can see the big picture
1:03:29
write a great prompt that also catches the edge cases and then
1:03:33
have the model implement it.
1:03:35
It's like the best engineering managers that are able to zoom
1:03:37
in and zoom out
1:03:38
SPEAKER_01: really well and being able to zoom out,
1:03:41
SPEAKER_01: prompt what you need to do,
1:03:42
but then zoom in when actually reading that code and catch
1:03:45
potential bugs instead
1:03:46
of just relying on the LLM to be 100% right in all cases
1:03:51
because there'll be unique
1:03:52
edge cases to the system that you're building that the LLM is
1:03:55
not aware of and you
1:03:56
SPEAKER_01: need to be able to catch that when you're reading the code.
1:03:58
SPEAKER_00: Yeah,
1:03:58
SPEAKER_00: I feel like if you have a mental model,
1:04:01
SPEAKER_00: and I see this so much when I'm using these tools,
1:04:03
SPEAKER_00: you know,
1:04:04
when I'm either like vibe coding or prompting or when I know
1:04:08
what I want to do,
1:04:09
SPEAKER_00: when it's in my head,
1:04:10
SPEAKER_00: I either like,
1:04:12
SPEAKER_00: because I know my code base or I know what I want to do,
1:04:14
SPEAKER_00: or I just sat down,
1:04:15
SPEAKER_00: I thought through it and I drew it out.
1:04:18
SPEAKER_00: I'm so fast.
1:04:18
SPEAKER_00: I'm great.
1:04:19
Like,
1:04:19
and I can switch between,
1:04:20
you know,
1:04:20
I might do an agentic mode to like generate it.
1:04:23
SPEAKER_00: Maybe I like it.
1:04:23
SPEAKER_00: Maybe I don't.
1:04:24
SPEAKER_00: Then I just do it by hand.
1:04:25
SPEAKER_00: It doesn't matter.
1:04:25
SPEAKER_00: Like I get there.
1:04:26
SPEAKER_00: Like I know where I'm going.
1:04:28
SPEAKER_00: But when I don't,
1:04:29
SPEAKER_00: I did this where like,
1:04:30
SPEAKER_00: oh,
1:04:30
SPEAKER_00: I tried to vibe code a game and I failed,
1:04:32
SPEAKER_00: not because I don't,
1:04:34
SPEAKER_00: I just didn't know what I wanted to do.
1:04:35
SPEAKER_00: Like,
1:04:36
SPEAKER_00: and,
1:04:36
SPEAKER_00: and,
1:04:36
SPEAKER_00: you know,
1:04:37
SPEAKER_00: when,
1:04:37
SPEAKER_00: when your prompt doesn't tell like,
1:04:39
SPEAKER_00: oh,
1:04:39
SPEAKER_00: do this,
1:04:40
SPEAKER_00: then I don't give it guidance.
1:04:42
SPEAKER_00: Yeah.
1:04:42
SPEAKER_00: Like it.
1:04:43
SPEAKER_00: Yeah.
1:04:44
SPEAKER_00: No, definitely.
1:04:44
SPEAKER_00: It wasn't the fault of, of the tool.
1:04:46
SPEAKER_00: It was just, you know, what, what, I don't know what I expect.
1:04:49
SPEAKER_00: Like how would this thing know,
1:04:52
SPEAKER_00: which it's,
1:04:53
SPEAKER_00: it's non-deterministic,
1:04:53
SPEAKER_00: but you need to give it some direction.
1:04:55
SPEAKER_01: Exactly, for sure.
1:04:57
SPEAKER_01: And I also,
1:04:58
SPEAKER_01: on that point,
1:04:58
think that it's great when you're zero-shotting and doing
1:05:01
greenfield work.
1:05:02
But today,
1:05:03
and I think this will change,
1:05:05
it's not the best at working in large codebases.
1:05:07
And as engineers,
1:05:08
you're always working in large codebases when you're building
1:05:11
things for prod.
1:05:13
SPEAKER_01: And so that part of our job hasn't changed of...
1:05:17
SPEAKER_01: being able to find the right place the code should go,
1:05:22
SPEAKER_01: use the right modules that exist,
1:05:24
SPEAKER_01: and piece it together in a larger code base when you're adding a feature.
1:05:28
SPEAKER_00: Yeah,
1:05:28
SPEAKER_00: and also just like simple stuff,
1:05:29
SPEAKER_00: which we take for granted,
1:05:30
SPEAKER_00: but like setting up the tools to like run the tests,
1:05:33
SPEAKER_00: to know how to deploy,
1:05:35
SPEAKER_00: to know how to control the feature flags,
1:05:36
to how safely put out something so it doesn't go to prod if you
1:05:40
want to A-B test,
1:05:40
SPEAKER_00: like which,
1:05:41
SPEAKER_00: you know,
1:05:42
SPEAKER_00: when you onboard,
1:05:42
SPEAKER_00: this is a pretty kind of given,
1:05:44
SPEAKER_00: but if you work at a place that has like microservices or whatever,
1:05:46
SPEAKER_00: like it's all,
1:05:48
SPEAKER_00: and I feel like there's so,
1:05:49
SPEAKER_00: so many other things.
1:05:51
But I love how you summarize what will not change because I
1:05:56
think that is really important.
1:05:58
SPEAKER_00: And I love how you brought up software architecture.
1:06:00
SPEAKER_00: I've been thinking about this recently.
1:06:02
SPEAKER_00: In fact,
1:06:02
I've started to read some really old software architecture
1:06:05
books because there are
1:06:07
SPEAKER_00: some ideas that I'm not sure will change.
1:06:09
SPEAKER_00: I want to assess this theory, but it might not change as much.
1:06:12
SPEAKER_01: What software architecture books are you reading at the moment?
1:06:15
SPEAKER_00: I've been going through the mythical man month.
1:06:17
SPEAKER_00: I've almost finished this.
1:06:19
SPEAKER_00: This is a real one.
1:06:20
SPEAKER_00: And then I have, so this is from the nineties.
1:06:23
It's called software architecture and it's by Mary Shaw and
1:06:29
David Garland and
1:06:30
SPEAKER_00: Grady,
1:06:31
SPEAKER_00: Grady Booch,
1:06:32
SPEAKER_00: who's a legend in software engineering.
1:06:34
SPEAKER_00: And I interviewed him.
1:06:35
He said that he thinks this is the single best software
1:06:38
architecture book.
1:06:38
SPEAKER_00: Now it's, it's very thin.
1:06:40
SPEAKER_00: And it's, I think, from 1995 or so.
1:06:44
SPEAKER_00: So I've just started to read it.
1:06:46
SPEAKER_00: But I'm interested in what 1996, which is 30 years ago.
1:06:50
SPEAKER_00: I'm just interested in what are the things that might have not changed.
1:06:53
SPEAKER_00: Clearly, some things will be dated, right?
1:06:55
SPEAKER_00: Like they're talking about Korba,
1:06:56
SPEAKER_00: which is like this old Java framework that we don't use anymore.
1:06:59
SPEAKER_00: But some of the other things, there's a lot of...
1:07:03
SPEAKER_00: reflection with civil engineering.
1:07:04
SPEAKER_00: And this book was written when there was no real software architecture.
1:07:07
So they tried to define it,
1:07:09
which I'm kind of thinking there might be some interesting
1:07:12
idea.
1:07:12
SPEAKER_00: So I'm interested on what has not changed for the most part.
1:07:17
SPEAKER_01: Yes.
1:07:18
SPEAKER_01: No,
1:07:18
that's a very nice approach of actually looking at the history
1:07:22
to see what's not
1:07:22
SPEAKER_01: changed from then to now,
1:07:24
SPEAKER_01: to then extend that line to what won't change in the future.
1:07:26
SPEAKER_01: I'd be very curious to see what you learn from that.
1:07:28
SPEAKER_00: Well, and also reinventing, right?
1:07:29
SPEAKER_00: Because I feel we will have to reinvent some parts of the stack.
1:07:32
SPEAKER_00: And I think it's important to understand.
1:07:34
Also,
1:07:34
I feel the past 10 years or so,
1:07:36
we've not talked too much about software architecture.
1:07:38
SPEAKER_00: So maybe there is a little bit to learn from other people's ideas.
1:07:41
SPEAKER_00: So to wrap up, how about we just wrap up with some rapid questions?
1:07:45
SPEAKER_00: I just asked a question.
1:07:46
SPEAKER_00: OK, let's go for it.
1:07:48
SPEAKER_00: So first of all, what is your AI stack?
1:07:49
SPEAKER_01: For coding, cursor.
1:07:51
For hackathons,
1:07:53
deep research to get a sense of what libraries already exist.
1:07:56
SPEAKER_01: ChatGPT is my default search and also my tutor.
1:07:59
And some internal tools to quickly do rag over company
1:08:02
documentations when I'm
1:08:03
SPEAKER_01: trying to find something.
1:08:04
SPEAKER_00: So what is a book that you would recommend and why?
1:08:07
SPEAKER_01: Book I'd recommend is The Almanac of Neville Ravikant.
1:08:11
SPEAKER_01: I've read it a couple of times.
1:08:13
SPEAKER_01: Big fan of the way he talks about building your life,
1:08:17
both from a very pragmatic way about how you should do it for
1:08:19
your career,
1:08:20
SPEAKER_01: but also in terms of just how to be happy.
1:08:22
And what is a piece of advice that made a big difference in
1:08:25
your professional career?
1:08:26
Don't wait for someone to give you the opportunity to go work
1:08:28
on something.
1:08:29
SPEAKER_01: Go work on it.
1:08:30
SPEAKER_00: Love it.
1:08:31
SPEAKER_00: So, Janvi, this was so nice to have you on the show.
1:08:34
SPEAKER_01: Thank you so much for even having me in the first place.
1:08:36
SPEAKER_00: Thanks very much to Janvi for this conversation.
1:08:38
SPEAKER_00: To me,
1:08:39
SPEAKER_00: talking with her was a great reminder of how in a new field like Gen.AI,
1:08:42
years of experience might be less relevant than teaching
1:08:44
yourself how to use these
1:08:45
SPEAKER_00: new technologies like Janvi has done so.
1:08:48
SPEAKER_00: It's also a good reminder of how it's never too late to get started.
1:08:51
Jamvi thought that she was late in 2022 because she was five
1:08:55
years behind every AI
1:08:56
SPEAKER_00: researcher who's been using transformers since it was released in 2017.
1:09:01
SPEAKER_00: And yet,
1:09:01
SPEAKER_00: Jamvi is now working in OpenAI,
1:09:03
the company that arguably made the most in utilizing
1:09:06
transformers and NLMs.
1:09:08
SPEAKER_00: For more in-depth deep dives on how OpenAI works,
1:09:11
SPEAKER_00: coming from the OpenAI team,
1:09:12
SPEAKER_00: and our practical guides on AI engineering,
1:09:14
SPEAKER_00: check out the Pragmatic Engineer deep dives,
1:09:16
SPEAKER_00: which are linked in the show notes below.
1:09:18
SPEAKER_00: If you enjoyed this podcast,
1:09:19
SPEAKER_00: please do subscribe on your favorite podcast platform and on YouTube.
1:09:23
A special thank you if you leave a review,
1:09:25
which greatly helps the podcast.
1:09:27
SPEAKER_00: Thanks, and see you in the next one.