Latest / The Pragmatic Engineer / Developer Experience at Uber with Gautam Korlam

Transcript

Developer Experience at Uber with Gautam Korlam · The Pragmatic Engineer

• 0:00
  SPEAKER_0: Vibe coding.
• 0:01
  SPEAKER_0: This is a brand new term.
• 0:02
  SPEAKER_0: What do you think it stands for?
• 0:03
  Vibe Coding is just you trying to figure out how the system should behave when
• 0:08
  you're prototyping.
• 0:09
  The good thing about Vibe coding is you're able to basically iterate much
• 0:13
  faster with this Agent Loops that now you're just focusing on how would I
• 0:16
  actually achieve my outcome rather than the exact way to do it.
• 0:19
  SPEAKER_1: You can always change those details later.
• 0:21
  SPEAKER_1: If you have the right abstractions, you could swap layers out.
• 0:23
  So I think that's changing the way people think about software because if
• 0:27
  you're someone who has taste and knows this is the outcome I want to see on the ux,
• 0:31
  you may not even need to know a framework.
• 0:33
  You should experiment with it till it feels right and of course you need to
• 0:36
  massage it to make sure it doesn't make it harder to all later on.
• 0:38
  I think a lot of the software today gets you to the initial,
• 0:41
  it looks great,
• 0:41
  but then how do you maintain it is basically sort of left for later time.
• 0:46
  And I think a lot of people who are still coding with AI are early in the
• 0:48
  prototype phase.
• 0:49
  Gotan Corlon worked at Uber for almost 10 years,
• 0:52
  joining X Android engine number eight and was the founding engine for the
• 0:55
  mobile platform team.
• 0:56
  He grew from software engineer two level,
• 0:58
  all the way to principal engineer and worked on developer experience and
• 1:01
  scalability challenges throughout his decade at Uber.
• 1:04
  Gotham is now the co-founder of guitar and a Gentech AI star that automates
• 1:07
  code maintenance.
• 1:08
  In today's conversation,
• 1:09
  we cover how Gotham acts onto the lead at Uber's job of monorepo,
• 1:13
  which had thousands of developers committing to it.
• 1:16
  In-house engineering tools like Monorepo,
• 1:18
  submit queue,
• 1:19
  local developer analytics and dev pods,
• 1:22
  AI's impact on software development.
• 1:24
  Why VI coding will spread and why got believes Junior does will thrive with AI tools.
• 1:29
  Uber's engineering culture is very unique even across tech companies and
• 1:32
  this epi is a perfect inside look of its early days.
• 1:36
  If you enjoyed the show,
• 1:37
  please subscribe to the podcast on any podcast platform and on YouTube.
• 1:42
  SPEAKER_0: Welcome to the podcast.
• 1:43
  SPEAKER_0: It's great to have you.
• 1:44
  SPEAKER_1: Oh, thanks so much Ka.
• 1:46
  SPEAKER_1: Glad to be here.
• 1:47
  SPEAKER_1: Thanks for having me.
• 1:48
  SPEAKER_0: It's good to reconnect.
• 1:50
  SPEAKER_0: We talked a lot while you were at Uber.
• 1:53
  You were on the platform team and your work touched a lot of the systems that we
• 1:59
  worked on.
• 2:00
  SPEAKER_0: Every engineer causes outages at Uber.
• 2:02
  SPEAKER_0: I did it, people on my team did.
• 2:04
  What was a memorable outage that you might have caused accidentally or deliberately?
• 2:10
  SPEAKER_1: So this is a funny story.
• 2:12
  Towards the end of my career there,
• 2:15
  I accidentally deleted the Java Monorepo Uber and there were thousands of
• 2:20
  devs committing this.
• 2:22
  Yeah,
• 2:22
  the Java monorepo,
• 2:24
  that was one of,
• 2:25
  we had two big back in Monorepo,
• 2:28
  right?
• 2:28
  SPEAKER_0: Go and Java.
• 2:29
  SPEAKER_1: Go and Java.
• 2:29
  SPEAKER_1: Yeah.
• 2:30
  So this was after we did the migration on almost everyone was using the Java.
• 2:35
  Yeah,
• 2:36
  it was pretty funny because what the story is essentially,
• 2:39
  I don't think many people know about this because I record pretty quickly,
• 2:42
  but I was trying to test something out like a test repo and I was like,
• 2:48
  Hey,
• 2:49
  I need to copy the GI push command.
• 2:52
  SPEAKER_1: We created the repos on push, so I copied the URL from the Java mon repo.
• 2:57
  I forgot to change it and then when I pushed,
• 3:00
  usually at Uber they prevent you from force pushing and it said,
• 3:06
  Hey,
• 3:06
  I can't force push and I didn't really read the message.
• 3:09
  Then I was like,
• 3:09
  why does this,
• 3:10
  I should be able to force push and I did with some special flags and there's
• 3:15
  only few people,
• 3:16
  maybe like three people at Uber who can force push with that flag because I was
• 3:20
  part of the platform team and we didn't want anyone else to do it.
• 3:23
  SPEAKER_1: And I pushed and then someone pings me on Slack.
• 3:26
  SPEAKER_1: Hey, so the repo is initial commit, what's going on?
• 3:32
  SPEAKER_0: So you were at a principal engineer at this point, right?
• 3:35
  You were one of the most senior people and you were there for a long time,
• 3:38
  right?
• 3:38
  SPEAKER_0: This is not some, you didn't know what was going on.
• 3:42
  SPEAKER_1: I mean that's why I had the permission to push.
• 3:44
  SPEAKER_0: So hold repo gone until you verdict minutes later.
• 3:49
  SPEAKER_1: Was it minutes?
• 3:49
  SPEAKER_1: It was tricky because we had to get it back from a backup.
• 3:51
  SPEAKER_1: Yeah, it was not too bad.
• 3:52
  SPEAKER_1: It was a few minutes and I think we had an incident review.
• 3:56
  It was luckily at a time when we had a submit queue so the queue would pause
• 3:59
  automatically and said,
• 4:00
  Hey,
• 4:01
  no work is lost.
• 4:03
  We had really good backups in place so we reworded quickly and I was like,
• 4:07
  ah.
• 4:07
  SPEAKER_1: Got to be very careful with those copy paste commands.
• 4:10
  SPEAKER_0: Wow.
• 4:12
  SPEAKER_0: Well, but I guess you tested the backup actually worked.
• 4:16
  SPEAKER_0: You should have said that.
• 4:17
  SPEAKER_0: It was just a test.
• 4:18
  SPEAKER_0: I wanted just to see if how it actually worked.
• 4:22
  SPEAKER_1: Yeah.
• 4:23
  SPEAKER_1: Chaos engineering,
• 4:24
  This episode is brought to you by sent buggy lines of code and long API Calls
• 4:28
  are impossible to debug and random map crashes are things no software
• 4:32
  engineer is a fan of.
• 4:34
  This is why over 4 million developers use sentri to fix errors in crashes and
• 4:38
  solve hidden or tricky performance issues.
• 4:41
  Centriq has debugging time in half,
• 4:44
  no more soul crushing,
• 4:45
  lock sifting or vague user reports like it broke.
• 4:48
  SPEAKER_0: Fix it.
• 4:49
  SPEAKER_0: Get the context.
• 4:50
  You need to know what happened when it happened and the impact down to the
• 4:54
  device browser and even a replay of what the user did before.
• 4:57
  The error Central will alert the right dev on your team with the exact broken
• 5:01
  line of code so they can push a fix fast or let AutoFi handle alert repetitive
• 5:05
  fixes so your team can focus on the real problems.
• 5:09
  Sent help monday.com reduce their errors by 60% and sp up time to resolution
• 5:14
  for nextdoor by 45 minutes per DAF per issue.
• 5:17
  Get your whole team on sent in seconds by heading to century.io/pragmatic.
• 5:22
  That is SENT y.io/pragmatic or use the code pragmatic on signup for three
• 5:30
  months on the team plan and 50,000 errors per month for free.
• 5:33
  SPEAKER_0: You reverted.
• 5:34
  SPEAKER_0: Thankfully nothing crazy happened, but this is a pretty big deal.
• 5:37
  SPEAKER_0: How can we imagine, what was the response of your manager?
• 5:43
  SPEAKER_0: The incident review?
• 5:44
  I mean yes,
• 5:45
  sure you were an experienced engineer,
• 5:46
  but come on,
• 5:46
  this was a big deal.
• 5:49
  So the good thing was since we had a summit queue and we can talk about it a
• 5:52
  little bit more,
• 5:53
  only thing it affected was the amount of time it would take for a commit to
• 5:57
  merge onto main.
• 5:58
  SPEAKER_1: So usually everything would get serialized.
• 6:00
  So all that it looked like was there was a 20 minute delay while we recovered
• 6:05
  everything and then everything just flowed naturally.
• 6:09
  So it was just like a random CI failure you would see,
• 6:12
  but we had so much recovery automated in place,
• 6:15
  it was almost a non-event and the only SLO we broke was the latency SLO,
• 6:19
  no reliability SLOs were broken.
• 6:20
  SPEAKER_1: So that was pretty good.
• 6:22
  All the systems we put into place in the many years actually worked to save me
• 6:26
  that day.
• 6:27
  And this is a good way to say into the next one,
• 6:30
  Uber's unique engineering stack.
• 6:33
  I worked at Uber,
• 6:34
  I worked for a lot shorter time than you,
• 6:36
  but we overlap.
• 6:37
  SPEAKER_0: You were there when I started and you were still there when I left.
• 6:41
  SPEAKER_0: Uber had a lot of unique systems.
• 6:43
  Can you talk about some of them and how they came about and why did Uber build so
• 6:49
  much of its stack?
• 6:49
  SPEAKER_0: Well Uber and also you built a lot of that stack, right?
• 6:53
  SPEAKER_1: Yeah, absolutely.
• 6:53
  So when I joined Uber in 2014,
• 6:56
  I was like Android engineer number eight I believe if I remember correctly.
• 7:00
  SPEAKER_1: And there was no unit tests.
• 7:01
  So I wrote the first unit test for the Android code base,
• 7:04
  then I set up the active factory.
• 7:06
  SPEAKER_1: Yeah, it was pretty funny.
• 7:07
  We were shipping so fast that people were just integration testing using
• 7:12
  their mobile phones.
• 7:13
  SPEAKER_1: It was just very, very fast growth.
• 7:15
  SPEAKER_0: This was in
• 7:17
  SPEAKER_1: 20 15, 20 14 is when I joined.
• 7:20
  So then I brought in Artifactory so we could share stuff and over time what
• 7:24
  ended up happening was a lot of the stuff you take for granted today,
• 7:28
  the cloud native SaaS products for observability or hosting source score
• 7:34
  were just not built for our scale back in the day.
• 7:37
  So we had to build a lot of stuff in-house to solve some very crazy problems.
• 7:41
  SPEAKER_1: So we had commits going in almost one every minute.
• 7:45
  I think,
• 7:46
  and that might not seem like a lot,
• 7:48
  but when you have thousands of engineers working on code basis where you can
• 7:51
  easily go into merge conflicts,
• 7:53
  that was a big deal.
• 7:54
  I hear a lot of companies saying it or at the time it was probably to say it's not
• 7:58
  built for our scale.
• 7:59
  SPEAKER_0: What was that scale?
• 8:01
  I mean you mentioned the one commit per minute,
• 8:03
  which I can actually attest to that build systems could not handle at least
• 8:07
  mobile build systems could not handle that,
• 8:09
  but what other scale was considered just way bigger than what commercial
• 8:14
  vendors supported?
• 8:16
  SPEAKER_1: Yeah, absolutely.
• 8:17
  So I think the build system part was definitely one because bills are taking
• 8:20
  us way too long.
• 8:22
  I think we embarked on a big app rewrite I think when you were there,
• 8:25
  I think you talked about it before.
• 8:26
  It was funny,
• 8:29
  I was actually in Amsterdam and that happened in between and then my boss
• 8:32
  calls me up,
• 8:33
  Hey,
• 8:34
  this build time is getting crazy.
• 8:35
  SPEAKER_1: We need to somehow bring it under control.
• 8:38
  SPEAKER_0: Oh that was you.
• 8:39
  SPEAKER_1: Yeah, I was there and it was almost an hour something plus.
• 8:44
  So if we actually seal analyze all the commits one by one to make sure there was
• 8:47
  no merge conflicts,
• 8:49
  it would just take forever.
• 8:50
  SPEAKER_1: And if it did not do that, our main bench would be red.
• 8:53
  We had this graph where it was like red,
• 8:55
  green,
• 8:56
  red green,
• 8:56
  red,
• 8:56
  green and it was impossible to work.
• 8:59
  There were so many teams trying to hit their deadlines and you were depending
• 9:02
  on so much work from the other teams like the networking library or the
• 9:05
  platform experimentation or a feature library from some other team.
• 9:09
  SPEAKER_1: It was just not possible to do this without Green Main essentially.
• 9:14
  So it's not just the bill system time but also the fact that when you push something,
• 9:20
  if you don't run tests against the other part of the code,
• 9:23
  then you're just not going to have a good time.
• 9:25
  SPEAKER_1: You will just be stuck reverting stuff.
• 9:27
  SPEAKER_1: Someone would be on call all the time.
• 9:29
  SPEAKER_1: So it was a combination of factors, it made to be very hard to work with.
• 9:34
  SPEAKER_0: And also we did do that on that project specifically.
• 9:36
  We turned off end-to-end test I think for two or three days to speed things up
• 9:40
  and sped things up.
• 9:42
  And then we had so many regressions took more time to,
• 9:48
  I think for two or three days we turned it off and for the next week everyone was
• 9:51
  fixing the regressions that sneaked in underneath.
• 9:55
  SPEAKER_1: Yeah, I think we got a request at some point.
• 9:57
  SPEAKER_1: Can you just turn off ci?
• 9:58
  SPEAKER_1: And I was like, what's going to happen if we actually did that?
• 10:02
  And I had to really talk people down saying you cannot just turn things off.
• 10:05
  SPEAKER_1: It's going to make things worse.
• 10:07
  SPEAKER_0: So there was the submit queue.
• 10:10
  Can we talk about what Submit Queue was and why it was unique at least
• 10:15
  initially to Uber even to this date,
• 10:17
  I'm pretty sure only a few companies have anything like that.
• 10:21
  Yeah,
• 10:21
  I think Submit Queue is essentially a way for you to guarantee a green main.
• 10:25
  So it's a way to kind of serialize your comments coming in and make sure that
• 10:28
  they play nicely with each other.
• 10:30
  There are some more open solutions now,
• 10:32
  but when we did at the time,
• 10:34
  we had actually a paper that we published.
• 10:37
  SPEAKER_1: It was fairly novel because we had very low tolerance for main being red.
• 10:43
  So we'd always want to guarantee a green main,
• 10:46
  which means we had to figure out how do we actually test changes coming in and
• 10:49
  make sure that the cross dependencies between different commit commits
• 10:53
  were considered.
• 10:54
  So it's just a way to kind of handle commits at scale and make sure that when
• 10:58
  everything merges,
• 10:59
  everything still builds with each other.
• 11:01
  And that's a trickier problem than it sounds because we had to test things in combination,
• 11:05
  discard some paths and do a lot of processing on the backend,
• 11:09
  but it was just one of the systems I think we had to build a ton more check and
• 11:12
  talk about as well.
• 11:13
  SPEAKER_0: I was the consumer on this side, I was just using it.
• 11:16
  SPEAKER_0: It was just a build system for me.
• 11:18
  But I didn't realize for quite a while that because my team,
• 11:22
  the 10 of us who were working on it,
• 11:24
  we didn't really have merge conflicts with each other.
• 11:27
  We were working in different parts of the code,
• 11:29
  but it took me a while to appreciate that when I was pushing something at any
• 11:34
  given time there would be maybe 10,
• 11:27
  20,
• 11:27
  30 people on the same code base in mobile working and each build would take 20,
• 11:27
  30,
• 11:27
  40 minutes.
• 11:45
  And then behind the scenes actually I understand only when I read the paper
• 11:48
  that you were looking checking is this code path in conflict?
• 11:52
  SPEAKER_0: If so, which one should we build beforehand?
• 11:54
  SPEAKER_0: There were all sorts of probability models.
• 11:56
  SPEAKER_0: There's a bunch of math in that paper as well, right?
• 11:59
  SPEAKER_1: There's a lot of ML models too.
• 12:01
  I think we had to estimate what would potentially cause a failure and
• 12:05
  speculatively try to go on paths that might be green to try to optimize and
• 12:10
  then backtrack if we didn't make that goal and we would make that model better
• 12:14
  over time.
• 12:15
  SPEAKER_1: So it was quite interesting.
• 12:16
  SPEAKER_1: I think if you didn't notice, that's a good thing.
• 12:19
  SPEAKER_1: It means that it worked as it's supposed to do.
• 12:22
  SPEAKER_0: One thing that Uber did as an engineering practice was mono repos.
• 12:27
  Can you tell us how it,
• 12:29
  Uber had multiple repos,
• 12:30
  but it all started with the iOS monorepo and I recall you were there.
• 12:35
  Can you tell us how decision was made to go from having a bunch of separate
• 12:40
  repos initially an iOS leader on Android and say,
• 12:43
  all right,
• 12:43
  let's just do one big repository and why did this happen and what happened next?
• 12:49
  So it's funny,
• 12:50
  I mentioned I think early on we didn't have tests at that point very,
• 12:54
  very early on we had two apps,
• 12:57
  rideranddriveronbothiOSandAndroid
• 12:59
  And we had another repository called library,
• 13:02
  which is the common component between the rider and driver app,
• 13:04
  which was just as a subpoint you would pull in and it was really painful to pull
• 13:08
  in as a submodule.
• 13:09
  So we said,
• 13:10
  Hey,
• 13:10
  let's set up some sort of cocoa parts artifactory thing where you could
• 13:13
  download stuff from somewhere.
• 13:16
  SPEAKER_1: But as we grew, we realized we couldn't put everything in one place.
• 13:19
  SPEAKER_1: We had a networking team, experimentation team, a analytics focused team.
• 13:25
  They all wanted their own sort of nice little playground so they could experiment.
• 13:30
  So we went off and had hundreds of repos for a while where everyone was just
• 13:35
  unaffected by everyone else could focus on their priorities and deliver.
• 13:38
  But then when you wanted to pull in these changes,
• 13:42
  let's imagine the networking library changed.
• 13:44
  SPEAKER_1: We had to update networking.
• 13:46
  SPEAKER_1: Then the analytics library that relied on it.
• 13:47
  SPEAKER_1: Then maybe the experimentation, maybe some other feature library.
• 13:51
  It was really painful to upgrade and pump and we were sort of on the hook
• 13:55
  because anything that the product teams don't do,
• 13:57
  the platform team ends up being the kind of catchall and you're like,
• 14:01
  Hey,
• 14:01
  this is not going to scale.
• 14:03
  I think someone on the iOS team was like,
• 14:06
  Hey,
• 14:06
  we should just go to Monte report.
• 14:07
  SPEAKER_1: We'll just take a weekend.
• 14:11
  SPEAKER_0: Famous lost words.
• 14:12
  SPEAKER_1: I know, but they took a while.
• 14:15
  It didn't take a weekend,
• 14:16
  but the iOS team did it and it was very effective for them.
• 14:20
  SPEAKER_1: It was a lot of pain initially and I think happened at the same time.
• 14:23
  SPEAKER_1: They're moving to a new language Swift as well.
• 14:25
  SPEAKER_1: So that was like double whammy.
• 14:27
  SPEAKER_1: They had to do both.
• 14:29
  They pulled it off and the gains in productivity were massive because you
• 14:33
  don't have to go bump libraries because we were not an open source.
• 14:38
  SPEAKER_1: Make sure the interfaces are nice for external consumption.
• 14:40
  It's one company,
• 14:41
  one code base,
• 14:41
  so I can't,
• 14:42
  they update my LI'S public API and just update all consumers in one shot.
• 14:47
  SPEAKER_1: It should be possible.
• 14:49
  SPEAKER_1: So mobile did it and Android quickly followed suit.
• 14:52
  SPEAKER_1: We actually froze the code for a weekend and we did everyone over.
• 14:56
  We moved everyone over and it was painful at the start because the build systems,
• 15:01
  as I mentioned,
• 15:01
  were not meant for hundreds of modules in the repository.
• 15:05
  SPEAKER_1: We have seen some changes to Google to make it better.
• 15:07
  And then after a while we migrated to a new build system,
• 15:11
  buck from Facebook at the time,
• 15:14
  and we wrote some auto conversion from the existing system to the new system.
• 15:17
  SPEAKER_1: So we had both at the same time.
• 15:20
  SPEAKER_1: That was really successful.
• 15:21
  SPEAKER_1: Once things were fast, the NPS improved again.
• 15:24
  SPEAKER_1: Monte was good.
• 15:25
  So the gist as monorepo non monorepo,
• 15:29
  if you don't have tooling,
• 15:30
  it's going to suck at bigger scale.
• 15:33
  SPEAKER_1: You have to invest in tooling.
• 15:35
  As your companies grow,
• 15:35
  typically this need becomes more and more obvious as things start slowing down.
• 15:40
  And then later we had the jam monorepo and we migrate over to Basil and we had
• 15:46
  the go monorepo.
• 15:47
  And I think what we realized with the Monorepo was that the main thing that
• 15:50
  helped us do was it was standardized,
• 15:52
  which meant to make a big change.
• 15:55
  One team could go do it,
• 15:56
  like a centralized team could just take the burden of updating this one
• 16:00
  networking library for everyone or updating Google Play services for everyone.
• 16:04
  SPEAKER_1: You just cannot do that in hundreds of repositories.
• 16:07
  SPEAKER_1: It's just very wasteful amount of work.
• 16:10
  SPEAKER_0: And what was the biggest pushback that the teams initially had?
• 16:13
  Because I remember the mobile ones,
• 16:15
  they actually happened and it was very practical,
• 16:18
  but then when the Java and the goal was happening,
• 16:21
  I remember a lot of teams were dragging their feet like,
• 16:23
  I'm not sure we want to do this.
• 16:24
  SPEAKER_0: It's going to slow things down.
• 16:27
  Do you remember what the biggest skepticism was and then how it turned out?
• 16:34
  Yeah,
• 16:34
  the biggest skepticism is usually right now before monorepo I could break an
• 16:38
  API and not worry about it.
• 16:40
  The tax is paid by someone eventually this is deferred,
• 16:44
  it still needs to get paid,
• 16:46
  but the monitor report,
• 16:46
  if I do it,
• 16:48
  I have to think about my interface design more and my bills are going to be
• 16:51
  slower with the argument with the new build system,
• 16:54
  we kind of said,
• 16:55
  Hey,
• 16:55
  the bills are not going to be slower,
• 16:57
  but people would always come up with reasons saying,
• 16:59
  Hey,
• 17:00
  if I did this upgrade then I had to do it for everyone.
• 17:03
  SPEAKER_1: So we said, okay, you know what?
• 17:04
  We will make it possible for you to just upgrade your part of the code in some scenarios,
• 17:09
  but otherwise we want to have everyone be standard.
• 17:11
  It does make things a little slower,
• 17:14
  but then as a business it makes a ton of sense because you might not see the
• 17:19
  macro picture on your team that hey,
• 17:21
  you might be slowing slow a little bit,
• 17:22
  but everyone else gets a huge productivity boost.
• 17:26
  They didn't have to spend the time you did on each and every one of those teams
• 17:30
  to get up to speed.
• 17:33
  SPEAKER_0: It is massive.
• 17:33
  SPEAKER_0: Is it safe to say that mono repos often?
• 17:36
  SPEAKER_0: Not always.
• 17:36
  They bring almost necessary friction where making certain changes will be
• 17:44
  painful because they should be painful because when they were not painful,
• 17:48
  they were masking something down the road of let's say I updated a dependency
• 17:52
  to something else and then it has to trickle down.
• 17:56
  Otherwise you'll have these problems,
• 17:58
  but they'll only come out in weeks or months later.
• 18:02
  SPEAKER_1: To be fair, you can also do stuff they thought them on April.
• 18:04
  I know companies like Amazon,
• 18:06
  Netflix too,
• 18:07
  multi repo setups,
• 18:08
  but they build tooling especially to handle those cases where you have
• 18:12
  golden version sets that work with each other.
• 18:14
  SPEAKER_1: It's a little bit like a monorepo without being a monorepo.
• 18:18
  So you can make both work,
• 18:20
  but repo is just generally easier in my opinion,
• 18:25
  just having construed both.
• 18:27
  SPEAKER_1: But if you want to go the other route, you absolutely can.
• 18:29
  SPEAKER_1: You still have to invest either way as your code base goes.
• 18:32
  SPEAKER_0: And what were some other novel slash innovative things?
• 18:36
  I mean it sounds like you had to invent some of these things at Uber because
• 18:39
  there was just not a solution that you could buy or it was just really
• 18:43
  expensive or not built for this use case.
• 18:47
  SPEAKER_1: Absolutely.
• 18:48
  So the other thing I think,
• 18:49
  which is interesting for us is as I mentioned,
• 18:51
  build times are a problem.
• 18:52
  SPEAKER_1: We only knew because we first had to measure it.
• 18:54
  I think back in the day,
• 18:55
  a lot of the measurement products for developer observability but just not
• 18:59
  built for the entire SST LC.
• 19:02
  So what I mean by that is you might have a CI check-ins back in the day that would
• 19:06
  tell you how long your bills would take,
• 19:08
  but you had no idea how much your indexing time would be in your IDE and we had
• 19:12
  very complex projects or we had no idea what was the time between pushing and
• 19:17
  code review.
• 19:18
  SPEAKER_1: So we built a system at Uber called local developer analytics.
• 19:21
  We call it LDA for short,
• 19:22
  which was a little demon that used to run on your machine,
• 19:26
  collect information about your system,
• 19:28
  what's your CPU usage,
• 19:29
  memory usage,
• 19:30
  but also integrate deeply into a lot of the CLI tools,
• 19:34
  the IDE.
• 19:35
  So we would for example,
• 19:36
  know when you open your project,
• 19:38
  which file you went to where,
• 19:41
  which part of the file you're actually coding the most,
• 19:43
  which files would have the most bugs,
• 19:45
  and then you could also see a funnel in a traditional product funnel,
• 19:48
  we care about developers as the end user,
• 19:52
  you would see analytics saying,
• 19:53
  Hey,
• 19:54
  there was a drop in the funnel.
• 19:55
  People could not create prs because this part of the build process would add a
• 19:59
  lot more often than the others.
• 20:00
  SPEAKER_1: So you would go and know what to target.
• 20:02
  And it was fairly novel because I gave it a few talks at a few different
• 20:07
  conferences and not even big companies had something like this back in the day.
• 20:10
  And I think we still use that today to power a lot of the dashboards and the deep
• 20:14
  analytics that we get from the developers workflows.
• 20:19
  SPEAKER_0: Yeah.
• 20:20
  SPEAKER_0: You've been a developer of productivity.
• 20:23
  You talk with a lot of people and know a lot of people,
• 20:25
  but what do most companies do?
• 20:27
  SPEAKER_0: Do they just ask developers how does it feel or?
• 20:31
  I think that's a very important thing to do for sure,
• 20:33
  because even if you have all the measurement,
• 20:34
  if the development doesn't feel right to you,
• 20:37
  you still want to ask that you want to have a survey,
• 20:39
  which we did.
• 20:40
  In fact,
• 20:40
  I remember when we started doing the surveys,
• 20:43
  I think we had an NPS score negative 50 something,
• 20:46
  and when I left it was positive eight or something like that.
• 20:50
  SPEAKER_1: So I call it as a win that we did a lot of effort.
• 20:53
  So you do your survey as the very first step,
• 20:55
  which we did,
• 20:56
  but then you measure the easier things like your time to review code or time to
• 21:01
  build code stuff that's out of the developers control you want to minimize
• 21:05
  and also time spent in meetings.
• 21:06
  Those are things that you could probably get easily,
• 21:08
  but the deeper stuff is only worth it if you see that those bottlenecks are
• 21:13
  adding up.
• 21:14
  If you're making changes but you're not seeing people are happier,
• 21:18
  you'll always see developer's complaint bill times are slow or ID indexing
• 21:22
  is slow.
• 21:22
  SPEAKER_1: That's when you know, okay, you need to measure things.
• 21:24
  SPEAKER_1: But now I think products like Cisco campaigns have better analytics.
• 21:28
  I think we've worked with JetBrains for example,
• 21:30
  in trying to make them understand,
• 21:32
  hey,
• 21:33
  these are some of our problems.
• 21:34
  SPEAKER_1: It'll be great to have solutions out of the box with these.
• 21:36
  And some of those did make it upstream into a products that everyone uses today.
• 21:40
  So we call that as we kind of had to forge our own paths and eventually all the
• 21:45
  vendors would catch up based on what the industry was doing.
• 21:49
  Yeah,
• 21:50
  but it's interesting how back then there was nothing and you kind of pinpointed,
• 21:55
  I guess one advantage that Uber always had is we had a lot of developers and
• 21:59
  there was actually a dedicated,
• 22:02
  what was your team's name?
• 22:03
  SPEAKER_0: Was it developer platform?
• 22:05
  SPEAKER_1: We were called developer experience.
• 22:07
  SPEAKER_1: They had multiple names.
• 22:08
  I think we were mobile infra first,
• 22:10
  then mobile developer platform,
• 22:11
  then developer platform.
• 22:12
  SPEAKER_1: That was the final name we had.
• 22:14
  SPEAKER_1: But funnily, actually the team was not that big.
• 22:17
  We had my team was about 10 people and we supported about a thousand engineers
• 22:21
  and that's roughly the breakdown.
• 22:23
  So you would see that speaking to peers in the industry,
• 22:27
  the team size are about the same.
• 22:29
  You have a very small code centralized team and then you have a much bigger
• 22:32
  product organization that they support.
• 22:34
  So it's very common and that means that you have to find ways to scale yourself
• 22:39
  and be effective.
• 22:41
  And then there was another project that I remember that was just taking off
• 22:44
  just in my last year.
• 22:46
  SPEAKER_0: It was called Dev pods.
• 22:48
  SPEAKER_0: What was that?
• 22:50
  So I think a lot of people might've heard the term cloud developer
• 22:54
  environments or developer environments in the cloud.
• 22:58
  So we call these dev parts and what it is essentially is a container of your code,
• 23:04
  your bill system artifacts,
• 23:06
  your IDE indices in the cloud.
• 23:09
  And what we did at Uber was fairly unique in the sense that we figured out some
• 23:14
  ways to make it good,
• 23:15
  really,
• 23:15
  really fast and have pretty much everything pre-IND index.
• 23:19
  It is one of the most loved products there because you could essentially
• 23:23
  multiplex yourselves now rather than trying to switch branches,
• 23:28
  re-index your code or context,
• 23:29
  which you could have 10 different parts working on different features and
• 23:33
  they could have everything warm.
• 23:35
  You could contact switch quickly,
• 23:37
  you could kick off a build and go do something else.
• 23:40
  SPEAKER_1: It was actually great.
• 23:41
  I think we put a lot of engineering effort to make that scale really well.
• 23:45
  SPEAKER_1: And it was a fairly small team as well.
• 23:46
  SPEAKER_1: Actually a bunch of those folks are still in Amsterdam now working on the
• 23:49
  SPEAKER_0: Product.
• 23:49
  And can you just explain,
• 23:50
  because I think most companies just do not have anything like that.
• 23:53
  SPEAKER_0: As a developer, what can I imagine?
• 23:56
  SPEAKER_0: Okay, I joined Uber depos is there, what does that mean for me?
• 24:00
  SPEAKER_0: What
• 24:01
  SPEAKER_1: Can I do?
• 24:01
  So when you onboard previously we used to have this boots strapping script
• 24:04
  you would run,
• 24:05
  it's funny,
• 24:06
  we do this now we have a bootstrap shell script that everyone just keeps updating.
• 24:11
  There was a shell script and then which usually worked well,
• 24:14
  but sometimes it crashed and then we would have to ping the on call of your team,
• 24:22
  but it was a pretty involved script.
• 24:23
  SPEAKER_0: It would take sometimes minutes to run out a lot of magic under the
• 24:28
  SPEAKER_1: Hood.
• 24:29
  And the problem with that is it's hard to keep it forward and quite compatible
• 24:32
  with your system updates on Mac.
• 24:34
  SPEAKER_1: So it was definitely not worth maintaining overtime.
• 24:37
  SPEAKER_1: But the deports is interesting.
• 24:40
  I think just when we started seeing some of these solutions like via coded
• 24:43
  remote or JetBrains projector become a little bit more mainstream,
• 24:47
  we said,
• 24:47
  Hey,
• 24:48
  why don't we move this development stuff into a container?
• 24:52
  SPEAKER_1: It should be easy, but it's actually much more than that.
• 24:55
  These containers can be huge multiple gigabytes because some of these
• 24:58
  artifacts and disease are huge.
• 25:01
  We had to make sure we also use compute efficiently,
• 25:04
  so we would have to have multi-tenant because developer workloads are very spiky.
• 25:08
  They don't always have a stable pattern like a server workload,
• 25:12
  which means that you had to understand when to scale up and down.
• 25:15
  We have developers in different time zones,
• 25:17
  so we have to also provision machines closer to them so the bail caches would
• 25:21
  be warm closer to where they're working.
• 25:24
  Otherwise you don't upload everything from a data center in the us but you
• 25:28
  might be working in India and the latency will pretty bad.
• 25:31
  SPEAKER_1: It's a lot more nuanced I think, than just spinning up a container.
• 25:35
  A lot of the solutions out there now are just focusing on content izing stuff,
• 25:39
  but not thinking through a lot of these other factors that are important.
• 25:42
  SPEAKER_1: So we went through and made a lot of improvements here.
• 25:45
  Some of that stuff,
• 25:46
  actually one of the funny things I remember is I think Jet Bains had this one
• 25:51
  shared index thing that they had a while back where you could download the
• 25:55
  index of the project,
• 25:57
  but I think the way they were doing it previously,
• 25:59
  you would have the full path of the system in the index.
• 26:03
  So you would download a de-normalized path and then you would normalize it to
• 26:07
  your system just populating all the absolute paths would take a really long time.
• 26:12
  We measured it was take an hour to actually index that shared index for chat pains.
• 26:18
  But then on DevOps what we did,
• 26:20
  we just said,
• 26:20
  Hey,
• 26:20
  everything's in our control.
• 26:21
  SPEAKER_1: Let's just put everyone at home user.
• 26:23
  So it'll be like closing the laptop and opening it again so we could just throw
• 26:28
  the cash there.
• 26:29
  SPEAKER_1: Everyone was at home user.
• 26:30
  So now no more time spent normalizing the cash,
• 26:34
  which meant we had a six second boot up,
• 26:36
  which is pretty insane for a dev environment.
• 26:38
  SPEAKER_1: So you would come in, you would say dev port, start something and done.
• 26:42
  You don't need to do any bootstrapping,
• 26:44
  not learn something,
• 26:45
  and you would have one flavor per monitor you to work on.
• 26:48
  So a data engineer or like a backend engineer or a mobile engineer would have
• 26:51
  their own flavor and that's very optimized for that workflow.
• 26:55
  So it's basically you go on and say Dev po start,
• 26:58
  and then in a few seconds you have your web,
• 27:01
  sorry,
• 27:01
  your local editor,
• 27:03
  you're using your local editor,
• 27:04
  but it's kind of connected magically from your perspective.
• 27:08
  SPEAKER_0: So you feel like you're working locally, but it's just all set up.
• 27:11
  SPEAKER_0: It works.
• 27:12
  SPEAKER_0: You can build immediately, you can run your tests, et cetera.
• 27:15
  Yeah,
• 27:15
  you would have lots of course as well,
• 27:17
  so you could run lot faster than running it locally.
• 27:20
  SPEAKER_0: That's pretty awesome.
• 27:22
  I previously did a coverage on,
• 27:24
  did a deep dive on cloud-based development environments I think two years ago.
• 27:27
  One thing I'm noticing is there's not much talk about these things even
• 27:30
  though they're just so darn efficient.
• 27:32
  Do you see kind of excitement in this space or is just AI right now blurring
• 27:38
  everything out?
• 27:39
  SPEAKER_0: To me this was a really big win?
• 27:43
  SPEAKER_1: Yeah, I think the environments are still good.
• 27:45
  SPEAKER_1: I think a lot of companies do want to use this for efficiency sake.
• 27:48
  The challenge usually comes with I think the understanding that I have
• 27:53
  talking to peers in industries,
• 27:54
  if you start to start with the solution rather than the developer workflow,
• 27:58
  Hey,
• 27:58
  we have a container environment,
• 27:59
  make it work for your developer workflow versus you have a particular way to work,
• 28:03
  let's make it work in a container or vm.
• 28:06
  SPEAKER_1: So we took the other approach and that is why we were so effective.
• 28:09
  SPEAKER_1: We were like, Hey, these caches have to be super fast load.
• 28:13
  Other stuff can come in a sync and not everyone would have that insight if left
• 28:17
  to their own devices.
• 28:18
  Teams might devise a docker file that has stuff,
• 28:21
  but then how are you getting it up to date as your team churns out more code?
• 28:25
  SPEAKER_1: Are your indices actually getting up to date?
• 28:27
  Otherwise you're spending a bunch of time after bootstrapping to e index
• 28:30
  stuff or redownload stuff.
• 28:32
  So you would push these updates every few hours completely transparently in
• 28:36
  the background.
• 28:37
  So it was very much like a golden path that we had to happen upon and that's why I
• 28:42
  think if you have that knowledge of the entire developer workflow,
• 28:46
  you can adapt it to each company's individual working style.
• 28:49
  If you give a general purpose thing,
• 28:50
  it'll work but may not actually work so well out of the box and you may not have
• 28:54
  the investment you may want to make to make it work for you.
• 28:56
  SPEAKER_1: Yeah,
• 28:57
  It sounds like there's no free lunch you need to put in the work and Uber had so
• 29:01
  many years of understanding investing and being there.
• 29:05
  Speaking of years,
• 29:07
  you spent nine years at Uber and when you joined you were an inch two,
• 29:11
  is that right or Yes.
• 29:13
  SPEAKER_1: I was an inch two when I joined one, I guess.
• 29:15
  SPEAKER_1: Yeah, what is the lowest level?
• 29:17
  SPEAKER_1: Whatever the lowest level was
• 29:20
  Level when you left,
• 29:21
  you were a principal engineer,
• 29:22
  which is there's not many principal engineers,
• 29:25
  several,
• 29:25
  a few dozen or something like that.
• 29:27
  SPEAKER_0: So
• 29:28
  SPEAKER_1: You were promoted actually Yeah, maybe a couple dozen.
• 29:29
  SPEAKER_1: Yeah, not that
• 29:31
  SPEAKER_0: Many out of the closer to two, 3000 people working in tech.
• 29:37
  And so you were promoted,
• 29:39
  I think I counted four or five times in nine years.
• 29:42
  SPEAKER_0: Can we talk about your journey?
• 29:45
  SPEAKER_0: How did you do it and what did you learn on the way?
• 29:49
  SPEAKER_1: Yeah, absolutely.
• 29:49
  I think one of the cheat codes is obviously joining early helps a lot,
• 29:54
  especially for getting on a rocket ship,
• 29:57
  but it's hard because when I joined we didn't have as many people.
• 30:00
  SPEAKER_1: The platform teams were tiny.
• 30:02
  SPEAKER_1: It was so small.
• 30:03
  It was funny because I jumped straight from entry-level to senior,
• 30:06
  so I skipped a level because the first year was just put through the paces of
• 30:11
  you need to get a lot of stuff done and I was shipping really fast.
• 30:14
  SPEAKER_1: So that helped obviously one jump up, but it gets harder over time.
• 30:18
  SPEAKER_1: I think your roles also change a lot over time.
• 30:20
  I think one of my good friends early on in the platform team said,
• 30:25
  Hey,
• 30:25
  if you can get into a niche and go deep,
• 30:28
  it can really help over long term,
• 30:30
  especially stuff that people may not want to do,
• 30:32
  like telpro platform,
• 30:33
  telpro tooling.
• 30:34
  Not everyone likes to do it,
• 30:35
  but if I said,
• 30:36
  Hey,
• 30:36
  I really like this because I'm passionate about it,
• 30:39
  that helps because then you become the go-to person for that aspect.
• 30:43
  So as the company grew,
• 30:44
  it was very obvious that hey,
• 30:45
  I had to scale myself to larger and larger efforts.
• 30:49
  So starting on mobile,
• 30:50
  then I started working on packet stuff and then on more holistic entire
• 30:55
  organization stuff.
• 30:56
  And that takes breaking your own comfort zone a lot,
• 31:01
  challenging yourself every year or two.
• 31:02
  So every two years I do some introspection and say,
• 31:06
  Hey,
• 31:07
  am I doing what I want?
• 31:08
  SPEAKER_1: Do I enjoy what I'm doing?
• 31:10
  SPEAKER_1: What can challenge me more?
• 31:12
  SPEAKER_1: And then I try to go for that.
• 31:14
  Usually the path opens up if you go that way rather than trying to chase a
• 31:17
  particular level.
• 31:19
  Then when you're pushing the boundaries,
• 31:21
  the next level becomes obvious then past a particular point.
• 31:25
  SPEAKER_1: It just also helps to have a lot of connections.
• 31:28
  SPEAKER_1: Social capital mentorship is a very important at a big company.
• 31:31
  So it definitely went and got some mentorship with some more senior folks
• 31:35
  that helped a lot to understand how these principal engineers operate and
• 31:40
  what's required.
• 31:41
  It's more than just the engineering,
• 31:43
  it's also the business side of things.
• 31:45
  How do we actually solve problems and focus on the business metrics and bring
• 31:52
  that to engineering rather than start from bottom up.
• 31:55
  And when you say you need to get some social capital and mentoring,
• 31:58
  it can sound a little bit of hand wavy to people of like,
• 32:01
  okay,
• 32:01
  but how did you get that?
• 32:04
  I'm not going to ask how you went about it,
• 32:06
  but it sounds like you didn't say,
• 32:07
  oh,
• 32:08
  I want to get social capital,
• 32:09
  but what were things that you did that you think actually helped build up and
• 32:13
  people saying,
• 32:14
  oh,
• 32:14
  like God,
• 32:15
  I really trust this person.
• 32:16
  SPEAKER_0: I know him, he's a go-to person, et cetera.
• 32:20
  Yeah,
• 32:20
  so one thing that helps is I have this habit of just helping people a lot.
• 32:25
  So when people come to me with a problem with their dev environment,
• 32:28
  I would just drop everything and say,
• 32:29
  Hey,
• 32:29
  let's try to figure it out.
• 32:30
  I'll spend a little bit of time box it and if it fixes the problem then I hire
• 32:36
  some social capital.
• 32:36
  SPEAKER_1: It kind of accumulates over time.
• 32:38
  They would send else to you and over time you can automate some of this stuff.
• 32:42
  And I think I was one of the early people on the platform team to have office server.
• 32:46
  So I would say,
• 32:47
  Hey,
• 32:47
  anyone can come in and just talk about your problems.
• 32:50
  SPEAKER_1: And we would just empathize a lot with everyone.
• 32:52
  SPEAKER_1: Say, Hey, I know this sucks.
• 32:54
  SPEAKER_1: We're working on it.
• 32:54
  It helps a lot to humanize the problem because developers usually being you
• 32:59
  on Slack and over time what happens is as you solve more and more problems,
• 33:05
  it's always good to attempt and if you can meet the expectation,
• 33:08
  just say,
• 33:08
  Hey,
• 33:08
  I can't solve this right now,
• 33:10
  then try to say,
• 33:11
  okay,
• 33:11
  that's not in my area,
• 33:13
  and we just try to help you out and that really helps build trust with the other side.
• 33:18
  And then when you go for mentorship,
• 33:20
  it's like,
• 33:20
  Hey,
• 33:20
  I clearly know you have the right sort of mindset.
• 33:25
  You have the right goals in mind or the right intentions,
• 33:27
  so I would love to help you go to the next level.
• 33:29
  SPEAKER_1: So that's usually what helps get very strong mentorships.
• 33:34
  And I can just plus one a little bit on the helping people because I was in Amsterdam,
• 33:39
  which is we say it's a distributed site,
• 33:41
  but we were just not in hq.
• 33:42
  We were a relatively small office compared to SF or even Palo Alto and headcount.
• 33:47
  And we felt a little bit isolated and sometimes a lot of times we would feel
• 33:52
  blocked by SF in reviews and anything.
• 33:57
  And after a while I knew that if I wanted to,
• 34:02
  you were one of the many people on the platform team,
• 34:04
  but I think either someone told me there was this thing,
• 34:08
  if you cannot cannot return anyone on the platform team try Gotham,
• 34:12
  he usually responds.
• 34:14
  And just thinking back,
• 34:15
  it was just well to me that I did get responses from sometimes and unexpected.
• 34:22
  SPEAKER_0: I didn't think I would get anything.
• 34:24
  I was just trying,
• 34:25
  but you did spend time on this and I just imagined that you had a lot of these
• 34:30
  things and I'm not saying you necessarily did it all the time and obviously
• 34:33
  people like me were doing it sparingly,
• 34:35
  but it built so much goodwill and also trust of like wow,
• 34:40
  I just felt that whenever you did go into a problem,
• 34:43
  you actually took it seriously.
• 34:46
  You kind of did it and as you said,
• 34:47
  often you would tell I cannot do this right now or it's a non-issue,
• 34:51
  et cetera.
• 34:52
  SPEAKER_0: So I didn't see everyone.
• 34:55
  SPEAKER_0: In fact, I saw pretty few people doing it.
• 34:57
  So it is interesting how it feels like something that would not scale,
• 35:02
  but I guess to some extent it did.
• 35:05
  SPEAKER_0: Is the starter mentality of do things that don't scale?
• 35:08
  Yeah,
• 35:09
  I think it definitely doesn't scale in all cases,
• 35:12
  but you can make it work.
• 35:13
  I think it's funny because some people won't even try because it won't scale.
• 35:18
  SPEAKER_1: My solution is like, hey, let's meet the person and see.
• 35:21
  SPEAKER_1: It may not even be an issue.
• 35:22
  Sometimes what you might discover is people are just finding it hard to find
• 35:26
  the right documentation because they just looked at the wrong guide and
• 35:30
  that's where they're coming to you.
• 35:31
  SPEAKER_1: Then you can just go make that fix and then you have much larger impact.
• 35:34
  But if you don't talk to people,
• 35:36
  you have these broken,
• 35:37
  you have the big picture,
• 35:38
  but then you lose touch of what's actually happening on the ground.
• 35:41
  SPEAKER_1: It's hard.
• 35:42
  So I think my policy was always like,
• 35:45
  Hey,
• 35:45
  I can always talk to someone,
• 35:46
  spend a few minutes and if it's a very esoteric thing I'll say,
• 35:50
  Hey,
• 35:50
  I can get back to you,
• 35:51
  but if it is not,
• 35:52
  I would rather just point you the right way.
• 35:55
  People joke when I left because they're like,
• 35:56
  Hey,
• 35:56
  we should go build gout,
• 35:57
  the LLM,
• 35:58
  because you would know the answers to pointed things at least so that we could
• 36:03
  find the right information.
• 36:04
  SPEAKER_1: So I was sort of like an encyclopedia of where to find stuff.
• 36:10
  And you were a principal engineer towards the end of your tenure at Uber,
• 36:14
  what is a principal engineer at Uber,
• 36:18
  and also previously we had an episode at Meta.
• 36:21
  SPEAKER_0: Meta has archetypes above staff engineer.
• 36:24
  Do you think you fit into an archetype or did you see archetypes at the
• 36:29
  principal level at Uber?
• 36:31
  SPEAKER_1: Yeah, I did listen to that episode.
• 36:34
  SPEAKER_1: It's a very good episode by the way.
• 36:35
  I think there is some resemblance to an archetype,
• 36:38
  although we don't actually call it that.
• 36:40
  So either you have depth in a particular area or you have a lot of breadth and
• 36:47
  sometimes it's a lot of internal influence or you might have a lot of external influence.
• 36:51
  SPEAKER_1: So it really depends on where you want to take your career.
• 36:55
  SPEAKER_1: I focus a lot on depth.
• 36:57
  As you can see,
• 36:57
  my specialization is data productivity and I have some amount of breadth
• 37:03
  because of all the social capital and talking to a lot of people usually makes
• 37:06
  you understand what are some of the business problems that Uber is going through.
• 37:09
  SPEAKER_1: So that was actually a good bonus for me.
• 37:11
  If I never talk to people,
• 37:12
  I won't even know,
• 37:13
  oh,
• 37:13
  these three teams are trying to solve the same problem,
• 37:16
  maybe we can centralize it or do it more efficiently.
• 37:19
  And I could come up with solutions and people would be like,
• 37:21
  Hey,
• 37:21
  how did you know that would work?
• 37:23
  SPEAKER_1: You just talk to people.
• 37:26
  And there were definitely lesser archetypes probably than meta I would say,
• 37:30
  but usually it's a bunch of breadth and depth and having people to sponsor you
• 37:35
  helps a lot.
• 37:36
  SPEAKER_1: So the mentorship angle is also part of that.
• 37:38
  If you're working with more senior engineers and they see you actually grow,
• 37:41
  they can give you feedback where you might be lacking and you might be able to
• 37:46
  work towards that goal.
• 37:47
  So I think that's one of the things I would recommend anyone who wants to go in
• 37:51
  that path.
• 37:51
  With principal engineering,
• 37:52
  it's a lot more you have to enjoy understanding how engineering meets
• 37:57
  business than just pure coding or just pure,
• 38:01
  what do you call it,
• 38:03
  not soft skill stuff.
• 38:04
  SPEAKER_1: A lot more soft skills need to be honed at that point.
• 38:08
  SPEAKER_1: It's like going and giving talks helps.
• 38:11
  And it's funny because when I was growing up,
• 38:13
  I was not very good at public speaking,
• 38:16
  but I made myself over time more comfortable.
• 38:19
  That helps also build external influence and a lot of the early folks that are
• 38:24
  using our product right now are people I met in the community and then they
• 38:28
  have that trust that they built over time.
• 38:30
  SPEAKER_1: So that helps
• 38:31
  SPEAKER_0: Quite.
• 38:32
  So this is a lot of people who are using your product right now at guitar,
• 38:35
  right?
• 38:36
  SPEAKER_1: Yeah.
• 38:37
  So I guess that's a good reminder of how,
• 38:41
  I guess when you're doing something,
• 38:43
  it might hopefully it'll help you outside of when you're at your current
• 38:48
  company and also help you while you're at the company.
• 38:52
  Yeah,
• 38:52
  my tip is generally if you really focus on the people side of things,
• 38:56
  you'll do really well in these higher levels because it's a lot of
• 39:00
  relationship management,
• 39:01
  both managing up and down and shielding your team from some of the noise.
• 39:07
  SPEAKER_1: And if you don't enjoy talking to people, it's harder.
• 39:10
  SPEAKER_1: You can still do it.
• 39:10
  There are people who do it,
• 39:11
  but it's a very unique path and not everyone can do it.
• 39:14
  But the most common path is usually understanding and spending time talking
• 39:20
  to people and figuring out solutions that are more creative over time.
• 39:23
  And that also makes you a problem solver,
• 39:26
  not just someone who can just code and finish up things up,
• 39:30
  but that's,
• 39:31
  of course,
• 39:32
  you can have different archetypes as you mentioned.
• 39:33
  SPEAKER_1: So it really depends on where you want to take things.
• 39:35
  But I guess at the principal entry level,
• 39:37
  it's kind of a given that you can code efficiently whenever and however it's
• 39:41
  just like you might not do it all the time by choice.
• 39:44
  SPEAKER_0: Right.
• 39:45
  It's funny because there was a graph someone sent me of number of years at the
• 39:50
  company and number of commits per year and number of reviews and there was a
• 39:56
  little graph,
• 39:57
  like an asymptotic graph,
• 39:59
  and there were two or three dots at the very extremes and then said,
• 40:03
  Hey,
• 40:03
  one of them is me,
• 40:04
  the other one is you.
• 40:06
  SPEAKER_1: Then I like, damn, I should ask my boss for a raise.
• 40:10
  SPEAKER_0: So you were still writing a lot of code and doing a lot of reviews.
• 40:13
  SPEAKER_1: I do write a lot of code even here at guitar, I write a ton of code.
• 40:16
  I love to review stuff because it really opens up your perspective of what
• 40:20
  else is happening.
• 40:21
  I like to comment on documents and understand where there are
• 40:25
  inefficiencies potentially,
• 40:27
  but just over time I think you have a muscle to just do it,
• 40:31
  but it does not mean that you write tons of code.
• 40:32
  SPEAKER_1: It just means that you have consistent
• 40:34
  SPEAKER_0: Higher
• 40:35
  SPEAKER_1: Impact code.
• 40:37
  So a really interesting topic is given that you are a principal engineer,
• 40:41
  which was either the highest level or maybe there was one level above it,
• 40:45
  but there weren't many levels above that.
• 40:48
  SPEAKER_0: How was it like for your manager to manage you?
• 40:51
  And I know you also talk with a lot of principal engineers and you have some
• 40:55
  thoughts on what are some tips to manage these very,
• 40:58
  very senior engineers.
• 41:00
  SPEAKER_0: Can you help explain your relationship with your manager?
• 41:02
  SPEAKER_0: Was it really a boss or more of a partnership?
• 41:07
  SPEAKER_0: What worked at this level?
• 41:09
  I think that's a really good question because I think as you grow in levels,
• 41:14
  you're more and more like a peer to your manager.
• 41:17
  You help your manager get stuff done too because you have broader influence.
• 41:22
  Sometimes it can be hard as an EM to push for a particular priority because
• 41:29
  there's not enough senior people looking back.
• 41:31
  It might be just your direct develop person that's sponsoring that,
• 41:34
  but having someone on the team that's senior enough can completely change thoughts,
• 41:38
  for example,
• 41:38
  to get some things practiced.
• 41:41
  One tip I will give managers is if someone's asking you,
• 41:44
  Hey,
• 41:44
  I'm at a senior staff or I want to become principal after the senior levels,
• 41:50
  if they come ask you,
• 41:51
  it's hard.
• 41:52
  You should probably not ask that question to a manager because at some point
• 41:57
  you'll have to figure things out.
• 41:59
  There's very few managers who probably can help you after a particular point
• 42:03
  because they maybe done it before or have a lot of experience at the company.
• 42:08
  But most managers tend to work at senior or senior plus level and past that
• 42:14
  it's more like they're almost like the principal engineer on the team is load
• 42:18
  balancing a lot of stuff technically so the manager can focus on upleveling,
• 42:23
  everyone else.
• 42:24
  So the relationship is very much like a peer more than like,
• 42:27
  hey,
• 42:28
  step boss.
• 42:29
  SPEAKER_1: But it does help to get feedback.
• 42:30
  I think their managers are a very good asset when you want to do
• 42:34
  non-engineering stuff,
• 42:36
  for example,
• 42:37
  you want to get stuff unblocked and the engineering angle is not working
• 42:42
  because you've tried to rationale it.
• 42:44
  SPEAKER_1: Some team they won't listen.
• 42:46
  Sometimes it's just got to take the,
• 42:48
  hey,
• 42:49
  this is the priority to get this done,
• 42:51
  unblock yourself.
• 42:52
  So that's what I would say for ems who might be looking at higher level engineers,
• 42:59
  basically give them agency,
• 43:02
  but of course check in often and make sure that they're unblocked.
• 43:06
  They can do wonderful things as long as your org is not getting in the way from
• 43:11
  them trying to actually have impact.
• 43:14
  Yeah,
• 43:14
  this is always interesting because I don't think we'd really talk about this
• 43:18
  too much just because there's not many of these engineers,
• 43:21
  but even when as was a manager,
• 43:22
  I actually promoted someone,
• 43:25
  actually two engineers,
• 43:26
  a level above where I was,
• 43:27
  right?
• 43:28
  I mean it was a bit easier for me because I saw examples,
• 43:30
  but it's always interesting and I feel it kind of goes hand in hand for these things.
• 43:36
  SPEAKER_0: Those promotions really help usually the manager as well.
• 43:38
  And as you said,
• 43:39
  for a lot of managers it's a first and there's,
• 43:42
  there's no playbook after senior or staff.
• 43:45
  SPEAKER_0: It's all unique.
• 43:46
  It's all based on the business needs for the company in that specific area.
• 43:52
  But if you need to keep up with the engineer,
• 43:54
  then they can pull you up as well with them because then you can become higher
• 43:58
  level and you can focus on bigger picture stuff.
• 44:00
  You hear a lot of stuff from an org or a company perspective from the engineers
• 44:05
  because just talk to so many people.
• 44:07
  Then you might have opportunities that you may not have if you're solely
• 44:10
  focused on your team.
• 44:11
  Before we jump back into the episode,
• 44:13
  I want to let you know that the audiobook version of my bestselling book,
• 44:16
  the Software Engineers Guidebook,
• 44:17
  is out now.
• 44:18
  You can get it on all major audiobook platforms like Spotify,
• 44:21
  audible,
• 44:22
  liberal fm,
• 44:23
  apple Books to Google Play,
• 44:24
  and also DM free MP three files.
• 44:27
  I started writing this book after a decade of working as a software engineer
• 44:30
  when I was working as an engineering manager for a few years already at Uber.
• 44:35
  Here's a review from the book from senior principal engineer Tanya Rayley,
• 44:38
  who is the author of the staff Engineer's Path from Performance Reviews to P
• 44:35
  95 latency,
• 44:43
  from Team Dynamics to testing ade di diversifies,
• 44:46
  all aspects of a software career.
• 44:48
  SPEAKER_0: This book is well named.
• 44:49
  SPEAKER_0: It's really does feel like the missing guidebook for the whole industry.
• 44:53
  You can get the book@engguidebook.com or search for the software engineer
• 44:56
  guidebook on your favorite audiobook platform.
• 44:59
  SPEAKER_0: I hope you'll enjoy it.
• 45:01
  One thing we just kind of mentioned as natural is you worked on a platform team,
• 45:05
  but when you started,
• 45:08
  I am pretty sure there was not yet a platform team and then there was a split
• 45:13
  called the platform program split.
• 45:15
  SPEAKER_0: It was a very famous email that I think later we saw.
• 45:19
  SPEAKER_0: I only read it.
• 45:20
  Can you talk about what was before there was platform and program teams and
• 45:24
  how did this split actually play out from your perspective?
• 45:28
  Yeah,
• 45:29
  so I remember when I was interviewing to join Uber,
• 45:33
  they asked me,
• 45:33
  Hey,
• 45:33
  what do you want to do?
• 45:35
  SPEAKER_1: You want to do some platform stuff?
• 45:37
  SPEAKER_1: I said, I want to build tools for developers.
• 45:38
  And they're like,
• 45:38
  okay,
• 45:39
  you're hired because no one wants to build tools for developers.
• 45:45
  The longer version is the platform teams sort of existed,
• 45:48
  we just used to call it,
• 45:49
  I think Mobile had it first,
• 45:50
  I think,
• 45:51
  but then the other ones,
• 45:52
  we just had unofficial platform teams.
• 45:54
  SPEAKER_1: They're working on core infrastructure pieces.
• 45:57
  The platform teams were essentially made so that the other teams could move
• 46:01
  faster and not duplicate stuff.
• 46:04
  There's a lot of stuff that goes into building features,
• 46:06
  and mobile was just very new back in the day,
• 46:09
  so things were just not as well figured out as they're today.
• 46:12
  So you had to build a ton of stuff internally,
• 46:14
  and if you're doing this on every team,
• 46:15
  then it's a lot of ways.
• 46:17
  So I think when we decided to the platform program,
• 46:21
  we said program teams don't need to worry about a lot of the underlying layers
• 46:25
  similar to a cloud native environment.
• 46:27
  You worry about your business logic,
• 46:30
  your product metrics,
• 46:32
  and we'll give you all the building blocks,
• 46:34
  which could be a mix of open source or wrapped in our own layer and our own architecture.
• 46:39
  SPEAKER_1: That makes so that you don't mess with the other features.
• 46:43
  Everyone has their own way of building banners or their own way of styling things,
• 46:48
  not at all consistent.
• 46:49
  For an app like Uber,
• 46:51
  you need to have a common design language,
• 46:52
  you need to use common components.
• 46:54
  Stuff you would do on a website or take for granted today just didn't exist on mobile.
• 47:00
  It had to be done that way to make things move faster so you don't have to
• 47:03
  rebuild another widget or rebuild another analytics processing thing and
• 47:08
  do something in the backend.
• 47:10
  So one pain point or I wonder how big a pain point is,
• 47:14
  but one thing that looked like challenging to me,
• 47:17
  I was not on the platform team.
• 47:18
  SPEAKER_0: I was your customer, but we always pinged you with questions.
• 47:22
  And as you mentioned,
• 47:23
  a year ago or so,
• 47:25
  there were 10 engineers for about 40,000 engineers.
• 47:28
  SPEAKER_0: What did support look like and what practices did you figure out?
• 47:33
  I saw a lot of practices like office hours on-call,
• 47:37
  rotations on where toing,
• 47:39
  et cetera,
• 47:40
  that I have not seen anywhere else.
• 47:41
  SPEAKER_0: The platform teams were advertising these things.
• 47:46
  SPEAKER_0: It was almost like, I don't know, having a vendor almost.
• 47:49
  SPEAKER_0: They were contracts, they were actually SLAs.
• 47:53
  It was published how fast you can expect things,
• 47:56
  where to escalate,
• 47:58
  how to tack something with high priority,
• 48:00
  et cetera.
• 48:01
  SPEAKER_0: I've never seen anything like this.
• 48:02
  SPEAKER_0: And how did you figure out support?
• 48:05
  SPEAKER_0: How important was it?
• 48:07
  SPEAKER_1: Sure.
• 48:07
  So the angle that I think we thought about was like,
• 48:10
  Hey,
• 48:10
  for everyone else,
• 48:11
  the end user is a customer for us,
• 48:13
  the developer is a customer.
• 48:14
  SPEAKER_1: Let's not treat it as another internal thing.
• 48:17
  SPEAKER_1: Let's really be customer obsessed.
• 48:19
  SPEAKER_1: That was the value we had to with it.
• 48:21
  SPEAKER_1: We extremely focused on the customer experience.
• 48:23
  Developer experience for us is the customer experience,
• 48:25
  which means imagine if your robot didn't show up,
• 48:28
  you'd be very upset.
• 48:29
  So if you're built,
• 48:29
  didn't finish exact same feeling,
• 48:32
  okay,
• 48:32
  I cannot ship my code.
• 48:34
  So we need to make sure that there is reliability and latency guarantees for
• 48:38
  that exact same reason.
• 48:40
  So all the product,
• 48:42
  I think like minded folks had very much hone in on product analytics.
• 48:46
  SPEAKER_1: We said, Hey, let's think about what is important for the developer.
• 48:49
  SPEAKER_1: So we asked them things like bill time.
• 48:51
  They said,
• 48:51
  Hey,
• 48:52
  I want to make sure my CA is reliable,
• 48:53
  has no flaky tests.
• 48:54
  Can you make sure the flaky tests are not blocking me because it's expensive
• 48:58
  with the bill cut,
• 48:59
  you have to chase the release strain and that's not fun.
• 49:02
  I'm sure you've seen your team complain about like,
• 49:04
  Hey,
• 49:04
  I had this flaky test,
• 49:05
  I had to resubmit this diff and that's not going to merge in time for the
• 49:09
  releasetrain
• 49:10
  SPEAKER_0: And just staying late.
• 49:11
  SPEAKER_0: I'm not sure.
• 49:12
  I think the release train left on Wednesday night,
• 49:15
  so Wednesday night was the cutoff or morning,
• 49:19
  I'm not sure,
• 49:20
  but I know people stayed longer on those days to catch the release train,
• 49:23
  meaning getting their PR as we call the merged on there.
• 49:28
  SPEAKER_0: Yeah, as you say, it was a big deal.
• 49:32
  I didn't actually realize that you were obsessing about it,
• 49:34
  but it kind of makes sense.
• 49:36
  SPEAKER_0: After a while it wasn't an issue.
• 49:38
  Yeah,
• 49:39
  when people complain a lot that,
• 49:40
  hey,
• 49:40
  I tried to catch the train,
• 49:41
  but then something was out of my control.
• 49:43
  SPEAKER_1: The infrastructure was flaky, so we missed this deadline.
• 49:48
  Now I'm on the hook as a platform team to ensure a heart fix happens,
• 49:51
  which means a bunch of other teams are going to get disturbed because they
• 49:54
  have to retest all their workflows.
• 49:56
  It has a lot of ripple effects,
• 49:59
  and this are not obvious unless you think about,
• 50:01
  oh,
• 50:01
  we cared about testing because the end user will see the impact if you don't
• 50:04
  test all these features again.
• 50:07
  SPEAKER_1: So for us, actually, the metrics are very important to have.
• 50:10
  We would have SLOs,
• 50:11
  we would publish them,
• 50:12
  we would try to keep them,
• 50:13
  we would review,
• 50:14
  we miss them,
• 50:15
  and then do incidents management and retrospective so we could avoid those.
• 50:19
  A lot of the good tooling came out of that was to ensure high reliability,
• 50:24
  low latency,
• 50:25
  which means you could ship product really fast as a company and the
• 50:29
  infrastructure would not get in that way.
• 50:32
  And that is why some of those metrics you see matter is because once we
• 50:35
  actively talk to the developers,
• 50:36
  we knew actually mattered to them.
• 50:38
  SPEAKER_1: That's what we focused on.
• 50:41
  SPEAKER_0: I'm just thinking about this, but is it safe to say that?
• 50:44
  I mean,
• 50:44
  I think wouldn't agree that the platform team or the developer experience
• 50:48
  team at Uber did a great job,
• 50:50
  but they did a great job because you ran it like a product team.
• 50:55
  As you said,
• 50:56
  you talk with the customers,
• 50:57
  you figure it out,
• 50:58
  how to map how they feel or what they care about into metrics.
• 51:01
  You then monitor those metrics and then you kept iterating on these things.
• 51:05
  SPEAKER_0: If I replace this with a paying customer, it is the same thing, right?
• 51:11
  SPEAKER_0: That's what a great product team does.
• 51:13
  SPEAKER_1: Absolutely.
• 51:14
  You have to write a product doc because then the incentives are good on both
• 51:18
  sides because if the developer doesn't like the product,
• 51:21
  then you should be like,
• 51:24
  okay,
• 51:24
  what do I do to make it better?
• 51:25
  So if the company is shipping high quality product,
• 51:29
  then we should also ship high quality product for our developers,
• 51:31
  and that's how we took pride in making sure that our developers could deliver
• 51:37
  to the end user.
• 51:37
  SPEAKER_1: So yeah, you're absolutely right.
• 51:38
  The metrics I think a lot of people might focus about at a high level may not be enough.
• 51:42
  SPEAKER_1: You're right, you have to talk to people, you have to look at the funnel.
• 51:45
  SPEAKER_1: Where are people dropping off?
• 51:47
  Things like the onboarding for example,
• 51:49
  for new engineers,
• 51:50
  where would they have difficulty?
• 51:51
  The dev parts for example,
• 51:52
  was like,
• 51:52
  hey,
• 51:52
  people would go look at these outdated docs and they would try to set up an
• 51:57
  environment and then just spend a lot of time.
• 52:00
  So having a depo just eliminates that,
• 52:03
  and then later you might have a situation where a bill failed and there's a
• 52:07
  very easy quick fix.
• 52:09
  SPEAKER_1: You should automate that.
• 52:10
  SPEAKER_1: You should not ask the user to go to it.
• 52:11
  If there's a linter failure,
• 52:12
  in my opinion,
• 52:13
  we fail because a link should not complain.
• 52:16
  SPEAKER_1: It should either fix or not.
• 52:19
  SPEAKER_1: Just telling them, Hey, this is a problem.
• 52:21
  SPEAKER_1: It is not going to do anything.
• 52:22
  You are so much pressure to ship product that if it's not blocking me,
• 52:26
  I will merge anyway,
• 52:27
  right?
• 52:28
  SPEAKER_1: So if it's blocking me, if it can be auto fixx, why can't you auto fix?
• 52:32
  SPEAKER_1: That's kind of how we took
• 52:34
  SPEAKER_0: Things.
• 52:36
  SPEAKER_0: I like that.
• 52:37
  SPEAKER_0: What is your take on measuring developer productivity?
• 52:40
  SPEAKER_0: Your team was called developer experience.
• 52:42
  SPEAKER_0: I'm sure developer productivity came in.
• 52:44
  This is a very heated topic these days it seems like for the past few years and
• 52:48
  there's a lot of back and forth of what can you measure,
• 52:50
  what can you not measure?
• 52:51
  SPEAKER_0: Should you measure diffs per engineer or per team?
• 52:56
  You probably thought a lot about this or have been in middle of this,
• 52:58
  what worked and what didn't?
• 53:01
  SPEAKER_1: Yeah, so I think the very first thing we measured was just sentiment.
• 53:04
  So talking to Dev,
• 53:05
  as I mentioned,
• 53:06
  the NPS survey was very easy because it was very quick signal to see if things
• 53:10
  are in the right spot.
• 53:11
  If everyone's super happy,
• 53:12
  then there's no problem,
• 53:13
  but that's really the case.
• 53:14
  SPEAKER_1: There's always problems that pop up.
• 53:19
  I think as you mature your measurement,
• 53:21
  I think a lot of the frameworks we see out in the open are sort of inspired by
• 53:25
  what Uber did early on and took some of the good picks out.
• 53:29
  I think when people talk about D per engineer,
• 53:32
  they're not really saying,
• 53:33
  Hey,
• 53:33
  you should commit more D per engineer is something preventing you from
• 53:37
  moving as fast as you could potentially.
• 53:39
  Because I can tell you that engineers,
• 53:42
  if basically nothing blocked them,
• 53:44
  would love to write code.
• 53:44
  SPEAKER_1: If you're an engineer,
• 53:46
  You want to ship code,
• 53:47
  you want to feel proud about what you shipped,
• 53:49
  there's no one will come in and say,
• 53:50
  Hey,
• 53:50
  I don't want to write these mini,
• 53:52
  I want to write code because it makes me happier.
• 53:56
  And what we were looking at those signals first for was,
• 53:59
  Hey,
• 54:00
  do you have too many meetings?
• 54:01
  SPEAKER_1: Do you not have focus time?
• 54:02
  SPEAKER_1: Because devs are usually a matter of focus time.
• 54:04
  SPEAKER_1: So when we improved that we saw focus time improved.
• 54:06
  I think when we had Covid,
• 54:08
  everyone was working from home,
• 54:10
  the dev engineer shot up and everyone thought that,
• 54:13
  Hey,
• 54:13
  people are working so hard.
• 54:16
  SPEAKER_1: It is amazing.
• 54:16
  It's just because people just didn't have too many meetings initially and
• 54:20
  over time and everything went to Zoom.
• 54:22
  SPEAKER_1: It kind of came back again down a little bit.
• 54:25
  It's usually a way to make sure that our infrastructure is letting people
• 54:31
  move as fast as they can.
• 54:33
  SPEAKER_1: And as you move roadblocks, you're focusing on the right things.
• 54:35
  And in fact,
• 54:36
  actually one of the things we measured is time for code review more than
• 54:39
  anything else,
• 54:41
  the time to review was the biggest bottleneck because as you mentioned,
• 54:44
  you would put up a PR in Amsterdam,
• 54:46
  might wait for review from sf.
• 54:47
  SPEAKER_1: That's already like a 10 hour delay
• 54:50
  SPEAKER_1: And I think the P nineties are pretty crazy.
• 54:53
  It would take days to get code reviewed and that would just be the biggest
• 54:57
  blocker more than anything to ship product.
• 55:00
  And honestly,
• 55:01
  CI time is a blip compared to amount of time it takes to review.
• 55:06
  So in the big picture,
• 55:07
  I think we work to improve turnaround time by nudging people by auto
• 55:11
  approving prs that had minor changes that semantically would not change the
• 55:15
  meaning or the behavior of the Yeah,
• 55:17
  that helped a lot because you didn't have to get another approval for making a
• 55:21
  comment fix,
• 55:21
  so you're doing a small stylistic change for example.
• 55:24
  SPEAKER_1: So that helped unblocked a lot of those scenarios.
• 55:26
  We had auto approve or auto land,
• 55:28
  which helped because if you review approved,
• 55:30
  then it would merge immediately so you don't have to rebate and worry about conflicts.
• 55:34
  A lot of these kind of small things can actually have a really big impact,
• 55:38
  but if you don't measure,
• 55:39
  you wouldn't know.
• 55:39
  So I think in terms of develop productivity,
• 55:43
  when you talk to your engineers,
• 55:44
  they might say,
• 55:44
  Hey,
• 55:45
  I feel less productive because I can't do something.
• 55:48
  Then your metrics should flow from that rather than just blindly taking a
• 55:52
  framework and saying,
• 55:52
  Hey,
• 55:53
  this is a framework in the industry,
• 55:55
  this should be it.
• 55:55
  SPEAKER_1: It may not actually work.
• 55:57
  SPEAKER_1: I mean it's not also the same for all kinds of developers.
• 56:01
  Is it safe to say that at Uber developer productivity was measured in the
• 56:08
  sense that there were these charts metrics,
• 56:10
  it showed this per engineer on your team,
• 56:13
  you could see it and how it changed over time,
• 56:15
  the time to code review,
• 56:17
  focus time,
• 56:17
  et cetera.
• 56:18
  But is it safe to say that these were measured with the intention of finding
• 56:23
  bottlenecks and what to improve and they were not measured with the
• 56:26
  intention of using them for performance reviews,
• 56:29
  promotion or figuring out who's the low performer?
• 56:33
  SPEAKER_0: Is that a fair assessment?
• 56:36
  SPEAKER_1: Yeah, that's the usual controversy.
• 56:38
  SPEAKER_1: People tend to misuse it.
• 56:40
  The guidance is always look at them at the high level and that is where a lot of
• 56:45
  these are aggregated and you can find ways to dissent enterprise people from
• 56:51
  bringing those up because you can easily catch bias.
• 56:54
  We had,
• 56:54
  for example,
• 56:55
  folks on the performance company that would be there just to catch bias that
• 56:58
  hey are just looking at tips or you're not looking at the quality.
• 57:02
  And sometimes people may not have as much output because they have other
• 57:05
  contributions they're making.
• 57:06
  SPEAKER_1: They might be writing docs or they might be influencing strategy.
• 57:09
  They might be working on unblocking teams or figuring out product and
• 57:13
  business goals and ing them to requirements and engineering.
• 57:17
  So it was definitely,
• 57:18
  I think initially people thought of it that way,
• 57:21
  but that was never the intention.
• 57:22
  SPEAKER_1: That was never the case.
• 57:23
  In my experience,
• 57:23
  at least on all the companies I've been,
• 57:25
  we never looked at it as anything more than,
• 57:27
  Hey,
• 57:28
  it's just good to know that hey,
• 57:29
  if you haven't had any prs,
• 57:30
  that's a red flag.
• 57:31
  SPEAKER_1: Potentially we had zero PRS in the entire year.
• 57:35
  SPEAKER_1: Explain yourself.
• 57:37
  SPEAKER_1: We will not.
• 57:38
  SPEAKER_1: We'll clarify with the manager, Hey, what's going on?
• 57:40
  SPEAKER_1: What are they doing?
• 57:41
  SPEAKER_1: But we never say that as a blanket that means nothing.
• 57:44
  So we always want to use this as a way to say if there are red flags,
• 57:48
  we catch it,
• 57:48
  that there might be something else going on.
• 57:50
  SPEAKER_1: But it was mostly used for unblocking teams from shipping fast.
• 57:56
  So it sounds like it's just like if you have data and you can gather it and that
• 58:00
  data can actually help you build the right things.
• 58:02
  I mean it will be silly not to do it and as you said,
• 58:05
  start with from what the pain points are,
• 58:07
  what people complain about.
• 58:08
  SPEAKER_0: I mean we always complained about, well that was there.
• 58:11
  It was always about code review and how long it took code review to take,
• 58:13
  and I don't think we ever measured it.
• 58:16
  I think I left by the time this was there,
• 58:18
  but if we would've measured it,
• 58:19
  it would've just really shown that our team was,
• 58:22
  we probably had our median was probably more than a day because we were
• 58:26
  waiting a team on the other side of the Atlantic and then we figured out how to
• 58:31
  cutthosetiesand
• 58:32
  SPEAKER_1: Quote is rarely a bottleneck, especially now with AI tools.
• 58:35
  SPEAKER_1: You could just generate a bunch of code and throw a pr.
• 58:38
  SPEAKER_1: That's rarely the bottleneck.
• 58:39
  You are bringing alignment or figuring out product requirements or making
• 58:43
  sure you're not breaking the experience takes the hardest.
• 58:46
  So just measuring output in terms of PRS is probably going to be harder as a
• 58:51
  justifiable thing.
• 58:52
  SPEAKER_1: After this new age of AI tools, people are just shipping so much more.
• 58:56
  We use a lot of AI tools at work,
• 58:58
  including our own,
• 58:59
  andweshipdonemorethanweusedtobackat
• 59:01
  SPEAKER_0: Uber.
• 59:02
  So you're now in the business of AI tools to help engineering productivity
• 59:09
  and engineers work better.
• 59:10
  SPEAKER_0: How do you see these tools changing how we do software development?
• 59:15
  I'm talking obviously about the usual suspects,
• 59:18
  the coding,
• 59:19
  auto completes,
• 59:20
  the copilots,
• 59:21
  also the agents and also everything else that's coming.
• 59:24
  SPEAKER_1: Yeah, absolutely.
• 59:25
  SPEAKER_1: So we are very thick in the space.
• 59:27
  Obviously the autocomplete is a huge boost because it in many cases helps you
• 59:32
  less type less but think more.
• 59:35
  SPEAKER_1: So I think if you probably use a bunch of these tools, I do too.
• 59:38
  It removes a lot of the grunt work,
• 59:40
  but it won't remove your thought process because the taste in what you want to
• 59:45
  do is still there,
• 59:45
  just not the how Sometimes it's like,
• 59:47
  okay,
• 59:48
  this makes sense,
• 59:50
  but I feel like a lot of the tools are either focused too much on their
• 59:53
  completion or they're only on the quote review.
• 59:56
  I think back from we learned things at Uber,
• 1:00:01
  we had a lot of inefficiencies and things we wished we could solve with some
• 1:00:05
  sort of intelligent layer.
• 1:00:06
  I was for example,
• 1:00:07
  going unblocking things like,
• 1:00:08
  hey,
• 1:00:08
  I knew exactly what would need to happen to unblock a build or so having some
• 1:00:14
  sort of an adjunct understanding of things are happening in the IDE,
• 1:00:17
  why did you type this character?
• 1:00:20
  That affects what's happening in code review and that affects what happens
• 1:00:23
  on deploy that affects what happens on your incident,
• 1:00:26
  for example.
• 1:00:27
  SPEAKER_1: There is nothing that's crosscutting.
• 1:00:28
  So that's kind of what we are trying to solve with qar is have this whole STLC
• 1:00:33
  under the purview and not just focused on the point solution of I'm just an ID
• 1:00:37
  code complete or I'm just going to be on your CI reviewing stuff.
• 1:00:41
  A lot of those solutions currently are I believe good starting points and my
• 1:00:46
  experience like how I was seen technology shifts happen.
• 1:00:49
  The much nicer or more important unlocks happen a little bit later into the
• 1:00:53
  cycle because since we're very early in the cycle with AI at this point,
• 1:00:56
  people are just experimenting stuff.
• 1:00:59
  But I believe that there's a huge value add for agents just because as you mentioned,
• 1:01:04
  there are some things that are so obvious when you think about it that hey,
• 1:01:07
  we wish we wrote a script to automate this for support or for documentation.
• 1:01:13
  Those can be potentially driven by agents but then still have the human in the
• 1:01:17
  loop when things need to be approved.
• 1:01:20
  SPEAKER_1: That's kind of how we think about it.
• 1:01:22
  SPEAKER_0: And as you're using these tools, how does it change your development?
• 1:01:30
  Obviously we're riding less code now,
• 1:01:33
  there's just more suggestions or there's also agents,
• 1:01:35
  but what does it allow you to do?
• 1:01:39
  SPEAKER_0: Are you just moving faster?
• 1:01:41
  SPEAKER_0: Are you actually having deeper thoughts?
• 1:01:43
  SPEAKER_0: I'm just interested in what it means for you.
• 1:01:45
  The current version of these agents that we know where it's a service cycle,
• 1:01:49
  we never know how long it'll go,
• 1:01:50
  but clearly it's only been no more than two years.
• 1:01:53
  SPEAKER_0: So
• 1:01:54
  Yeah,
• 1:01:55
  I think it allows you to explore more paths that you previously didn't have
• 1:01:58
  time to explore because you could experiment much faster and discard ideas
• 1:02:03
  much quicker because when you're prototyping,
• 1:02:05
  you don't need the full rigor.
• 1:02:07
  SPEAKER_1: You can put them behind a flag and then say, Hey, this looks good.
• 1:02:09
  I think people are calling it vibe coding these days like,
• 1:02:12
  Hey,
• 1:02:12
  this has the right vibe,
• 1:02:15
  we can experiment this.
• 1:02:16
  SPEAKER_1: Of course it's harder when you have a lot of other constraints.
• 1:02:18
  SPEAKER_1: You have to make sure the software is secure compliant.
• 1:02:22
  A lot of people are not thinking about that when prototyping,
• 1:02:25
  so it lets you prototype faster,
• 1:02:27
  which is great.
• 1:02:28
  And I think what's also happening is I feel now this all of talk about,
• 1:02:34
  hey,
• 1:02:35
  drone engineers are going to get replaced,
• 1:02:37
  but I think it's the opposite.
• 1:02:39
  My take is that junior engineers are going to thrive because they are coming
• 1:02:43
  with new knowledge,
• 1:02:44
  new ways of working with these tools and are going to be much more effective
• 1:02:48
  because they don't have bias of working a particular way.
• 1:02:52
  They're able to achieve things that,
• 1:02:53
  for example,
• 1:02:54
  if you take a long time to even wrap up doesn't mean that you shouldn't know the fundamentals.
• 1:02:59
  In fact,
• 1:03:01
  if you think about 20 years ago they had a DBA and A-H-T-M-L and a Java
• 1:03:09
  application engineer.
• 1:03:10
  SPEAKER_0: There was no data
• 1:03:11
  SPEAKER_1: Engineer and a
• 1:03:12
  SPEAKER_0: Webmaster sometimes,
• 1:03:13
  SPEAKER_1: Right?
• 1:03:13
  SPEAKER_1: There was not that
• 1:03:15
  SPEAKER_0: Jump no longer exists.
• 1:03:17
  No,
• 1:03:18
  there's no front end person,
• 1:03:19
  there's no mobile person,
• 1:03:20
  there's no data engineer,
• 1:03:22
  ML engineer.
• 1:03:22
  SPEAKER_1: There's so many more archetypes now or subcategories.
• 1:03:26
  I think with ai,
• 1:03:28
  the general engineer is going to see a rise again,
• 1:03:31
  which means I'm able to jump into for example,
• 1:03:34
  front end without not doing much front end because I can understand that okay,
• 1:03:38
  this is what I expect to happen and I would like to have this outcome with my software,
• 1:03:46
  which means I can then focus on the other parts of the problem.
• 1:03:50
  SPEAKER_0: You mentioned vibe coding and this is a brand new term.
• 1:03:55
  SPEAKER_0: What do you think it stands for?
• 1:03:56
  Is it just prototyping,
• 1:03:57
  is it just throwing out ideas or is it a bit more than that?
• 1:04:02
  I think it's vibe coding is just you trying to figure out how the system should
• 1:04:06
  behave when you're prototyping.
• 1:04:10
  The good thing about vibe coding is you're able to basically iterate much
• 1:04:14
  faster with these ENT loops that now you're just focusing on how would I
• 1:04:18
  actually achieve my outcome rather than the exact way to do it.
• 1:04:22
  SPEAKER_1: You can always change those details later.
• 1:04:24
  SPEAKER_1: If you have the right abstractions, you could swap layers out.
• 1:04:28
  So I think that's changing the way people think about software because if
• 1:04:31
  you're someone who has taste and knows this is the outcome I want to see on the ux,
• 1:04:38
  you may not even need to know a framework.
• 1:04:40
  You should experiment with it till it feels right and of course you need to
• 1:04:43
  massage it to make sure it doesn't make it harder day while later on.
• 1:04:46
  I think a lot of the software today gets you to the initial,
• 1:04:50
  it looks great,
• 1:04:51
  but then how do you maintain it is basically sort of left for later time.
• 1:04:55
  And I think a lot of people who are still quo with AI are early in the prototype phase.
• 1:05:00
  I don't think at enterprise scale that would work because I can imagine it'll
• 1:05:03
  probably mess with your entire abstraction you already put in.
• 1:05:06
  SPEAKER_1: So you have to really constrain it to work well.
• 1:05:09
  So white coding is great for prototyping at this point,
• 1:05:11
  but that's also why I think you can move a lot faster with less constraints
• 1:05:16
  becauseAIhasalotmorefreedomtoexplore
• 1:05:19
  SPEAKER_0: One of the problems that we're seeing.
• 1:05:22
  And if you're using it,
• 1:05:23
  you'll also see as there's this Adi Oman called it the 70% problem that vibe
• 1:05:29
  coding and these things that they get you started,
• 1:05:31
  but then it can get just really confused and stuck and people see this,
• 1:05:35
  people who are not really technical,
• 1:05:36
  they kind of get stuck and that's where experienced engineers really shine.
• 1:05:41
  SPEAKER_0: You said previously that you think junior engineers will do great there.
• 1:05:45
  SPEAKER_0: How do you think less experienced engineers can deal with this?
• 1:05:48
  That's one and then the other is this where we might see senior engineers
• 1:05:53
  actually thrive and spend more of their time?
• 1:05:57
  Yeah,
• 1:05:57
  I think the main differentiator there is when things go wrong,
• 1:06:01
  understanding why they went wrong,
• 1:06:02
  which means strong CS fundamentals,
• 1:06:05
  strong sort of system knowledge of hey,
• 1:06:07
  how does the entire thing work to end?
• 1:06:09
  SPEAKER_1: So senior engineers usually tend to have a better bigger picture.
• 1:06:14
  So if they adopt these new tools,
• 1:06:15
  actually they can get a lot more productive.
• 1:06:18
  I can tell from my own experience that adopting these tools,
• 1:06:21
  I'm able to get a lot more stuff done because I know for example,
• 1:06:24
  this is how I expect my system to work,
• 1:06:27
  which means I can delegate to,
• 1:06:29
  I used to give it to a more junior person to understand the system.
• 1:06:32
  Now I can just keep it to AI and I can check the work and I can come at it.
• 1:06:37
  But if you're very stuck in the ways and not want to use these tools,
• 1:06:42
  you can still do it.
• 1:06:43
  SPEAKER_1: It's going to be a lot slower and that's totally fine too.
• 1:06:45
  SPEAKER_1: It is just a matter of how quickly you want to ship.
• 1:06:47
  And for things that are not critical,
• 1:06:49
  you may want to still experiment to see what's out there because the things,
• 1:06:52
  the way they're going,
• 1:06:53
  the tools are getting better every day,
• 1:06:55
  which means over time a lot of the mistakes that are happening today will
• 1:06:59
  start to disappear.
• 1:07:00
  There'll be like other agents that will check the work of the first agent,
• 1:07:03
  so they're not going to be as many mistakes.
• 1:07:06
  You still want to have a human in the loop before you check in,
• 1:07:09
  otherwise you will not have full guarantee that whatever you're shipping
• 1:07:12
  with the user is going to be good.
• 1:07:14
  But you definitely want senior engineers to focus on the high level system
• 1:07:19
  level knowledge and the agent loops can be more nuanced and more focused on
• 1:07:25
  smaller parts of the puzzle.
• 1:07:27
  After nine years of Uber working on developer experience now co-founded a
• 1:07:32
  company called Guitar,
• 1:07:34
  how are you thinking about continuing this developer experience and also
• 1:07:38
  what's your approach to ai?
• 1:07:40
  SPEAKER_0: What are your bets on where the space is going to go?
• 1:07:43
  SPEAKER_1: Yeah, that's a good question.
• 1:07:45
  So when we co-founded guitar with my co-founder Ali and Raj,
• 1:07:49
  we basically saw that a lot of the inefficiencies we had tools were not
• 1:07:54
  available back in the day at Uber.
• 1:07:57
  SPEAKER_1: We saw because we had a golden path for developers.
• 1:08:00
  Now with agents,
• 1:08:01
  I think we talked about Deports previously like,
• 1:08:04
  hey,
• 1:08:04
  there's a golden path for a developer to get stuff done.
• 1:08:08
  So what if an agent had a golden path to get things done for a particular enterprise?
• 1:08:13
  So we are thinking of guitar,
• 1:08:15
  the agents we are building atar,
• 1:08:17
  we just are going to launch a new product pretty soon and you can check us out at
• 1:08:22
  guitar ai.
• 1:08:24
  The agent AI we build is actually inside the IDE on the code review system
• 1:08:30
  inside a deployment.
• 1:08:31
  SPEAKER_1: It's across the board.
• 1:08:32
  So there's no reason why you can't auto complete a deployment shock and not
• 1:08:35
  just be stuck inside ID and having an understanding of things like,
• 1:08:40
  Hey,
• 1:08:40
  how are things going in production?
• 1:08:42
  SPEAKER_1: Which part of the code is slow?
• 1:08:44
  SPEAKER_1: These dots are usually what engineers connect.
• 1:08:47
  The context is what flows through your brain,
• 1:08:51
  and if there's a standardized path for an agent to evolve software and
• 1:08:54
  maintain it,
• 1:08:55
  it becomes much easier to focus on the business value.
• 1:08:58
  And one of the things that people dread doing is,
• 1:09:00
  hey,
• 1:09:00
  maintaining stuff like updating libraries or fixing tech or adding coverage,
• 1:09:04
  we feel like a lot of that grant work can be done very efficiently with agents.
• 1:09:08
  So that's where our focus is that we don't want to be just take a code complete
• 1:09:13
  or within your point solution.
• 1:09:15
  SPEAKER_1: We want to have it across the entire stack of software development.
• 1:09:18
  And we have that experience working on this for many years,
• 1:09:21
  talking to peers in the industry,
• 1:09:22
  there's a lot of potential for disrupting the space and the stuff we wish we
• 1:09:28
  had that,
• 1:09:28
  hey,
• 1:09:29
  I wish someone was on call during this time to fix a small thing.
• 1:09:33
  We can make that happen because we can give the agent powers to unblock people
• 1:09:37
  when you might be asleep,
• 1:09:38
  which means you'll be on call less often.
• 1:09:40
  So we are thinking of it from a holistic developer experience standpoint and
• 1:09:45
  producing more code is just one part of it.
• 1:09:47
  It's about doing it reliably,
• 1:09:48
  maintaining it,
• 1:09:49
  and making it understandable so newer developers can understand what's
• 1:09:53
  going on.
• 1:09:54
  If we just let AI run amok on your code base and not have an idea of what's going on,
• 1:09:58
  then that's the recipe for disaster.
• 1:09:59
  So we feel like there's a lot of opportunity for making the entire experience
• 1:10:05
  end-to-end a much more standardized so that the golden path for developers
• 1:10:10
  and agents is really well laid out.
• 1:10:14
  Now when I'm listening to this one part of me is if I'm a business owner,
• 1:10:19
  I'm like,
• 1:10:19
  oh,
• 1:10:19
  that sounds great,
• 1:10:20
  because over time I might just have less developers have a really capable agent,
• 1:10:24
  and as a developer,
• 1:10:25
  obviously one thing that pops into your head is like,
• 1:10:28
  wow,
• 1:10:28
  we might have a smaller team.
• 1:10:29
  Hopefully I'll still be there,
• 1:10:31
  but I might have fewer colleagues or I might be fortunate one to get the cut.
• 1:10:36
  Let's just run with this,
• 1:10:38
  that this future is coming and we will have a lot more capable agents In this case,
• 1:10:44
  what do you think developers will do and what are sets that are worth
• 1:10:50
  investing in so that you will still be an efficient,
• 1:10:54
  a great developer in the future,
• 1:10:56
  seen as a highly efficient developer?
• 1:10:58
  10 years from now,
• 1:10:59
  your managers will be like,
• 1:11:00
  oh my god,
• 1:11:01
  we cannot do work without this person.
• 1:11:02
  We need this person here and we should probably give them a raise underpaid.
• 1:11:07
  Yeah,
• 1:11:07
  I think as you said,
• 1:11:09
  there might be less people in a business,
• 1:11:10
  but there'll be more businesses.
• 1:11:13
  SPEAKER_1: That's typically how a lot of these technology shifts go.
• 1:11:16
  If building software is easy,
• 1:11:18
  then the value unlock will be actually the taste and the experience for the
• 1:11:22
  end user.
• 1:11:23
  SPEAKER_1: What do we actually provide?
• 1:11:24
  If every website looks the same because you prototyped it in a website builder,
• 1:11:30
  then that's not going to be your differentiator.
• 1:11:33
  Even I think during the mobile era,
• 1:11:35
  a lot of apps came to do things like to-do lists or reminders or just scrolling
• 1:11:41
  social media,
• 1:11:42
  but then eventually some clients are great ux,
• 1:11:44
  they will not because they're just a joint use.
• 1:11:46
  So knowing what matters to your end customer is going to be differentiated
• 1:11:50
  for engineers to hone in on.
• 1:11:52
  SPEAKER_1: So that means understanding the business really well.
• 1:11:55
  SPEAKER_1: And then the other thing is how do you build something for scale?
• 1:12:00
  Essentially as you grow the business,
• 1:12:03
  you don't need as many engineers probably,
• 1:12:04
  and that's okay,
• 1:12:05
  and that might be fine because if you're at a smaller company with less number
• 1:12:08
  of engineers,
• 1:12:09
  if you have equity for example,
• 1:12:11
  you might have more of the pie,
• 1:12:12
  which means that's a good way to incentivize people to experi with more ideas.
• 1:12:18
  SPEAKER_1: And usually competition is good in that way.
• 1:12:21
  So you may not be bound by those jurisdiction that you have to always be in a big company.
• 1:12:26
  I think we'll start seeing smaller,
• 1:12:28
  more efficient teams who are going to be more productive and will deeply
• 1:12:32
  understand the product.
• 1:12:33
  And that's going to be,
• 1:12:34
  I think a good bit,
• 1:12:36
  compete with other products in that space.
• 1:12:38
  The taste,
• 1:12:39
  the art that goes into crafting great X or great end user experiences is going
• 1:12:43
  to matter more than how you wrote the code.
• 1:12:47
  Now if I'm thinking,
• 1:12:49
  I think it's good to think back that for example,
• 1:12:51
  even 15 years ago,
• 1:12:47
  10,
• 1:12:47
  15 years ago,
• 1:12:54
  we already had this WhatsApp.
• 1:12:57
  They were barely at 50 people when Facebook acquired them for $19 billion,
• 1:13:01
  which is crazy.
• 1:13:02
  SPEAKER_0: This was no way.
• 1:13:03
  SPEAKER_0: I know nothing, but clearly they were a very efficient team.
• 1:13:05
  Instagram,
• 1:13:06
  I think 12 or 13 people also,
• 1:13:09
  they built an app that was already 30 million users with that small,
• 1:13:12
  even today,
• 1:13:13
  that will be a big deal.
• 1:13:14
  So to get that skillset of you being able as an engineer,
• 1:13:20
  being able to do a lot with these tools with the small team,
• 1:13:23
  you mentioned taste,
• 1:13:24
  you mentioned art,
• 1:13:26
  what other things that are worth focusing on that should probably help do a
• 1:13:32
  lot more with less?
• 1:13:34
  SPEAKER_1: Yeah, I think having an understanding of the system layer is good.
• 1:13:37
  SPEAKER_1: So I think the scalability part we didn't cover.
• 1:13:39
  SPEAKER_1: So how does your app or software scale to remote users?
• 1:13:43
  SPEAKER_1: What parts of the system are less efficient?
• 1:13:45
  SPEAKER_1: More efficient?
• 1:13:46
  SPEAKER_1: Because all code might do the same things, but less or more efficiently,
• 1:13:50
  SPEAKER_1: And how do you maintain it?
• 1:13:51
  So that's the other challenge a lot of people run into as software systems grow,
• 1:13:56
  is it easy to upgrade because you have less external dependencies?
• 1:13:59
  SPEAKER_1: Is it harder because you have so many versions?
• 1:14:01
  The software might work the same today,
• 1:14:03
  but okay,
• 1:14:03
  three years down the line,
• 1:14:04
  if you can't upgrade and move faster,
• 1:14:07
  your competition is going to basically move faster than you because it
• 1:14:09
  started on a clean slate.
• 1:14:11
  SPEAKER_1: We see that all the time.
• 1:14:12
  I think there's a lot of legacy technology or database systems people are
• 1:14:15
  trying to move to modern systems.
• 1:14:17
  It's a big cost and that's very expensive,
• 1:14:20
  a lot of engineering time wasted.
• 1:14:22
  So if building for basically maintainability is going to be important as well,
• 1:14:27
  and building for either AI agent maintainability or human understanding so
• 1:14:32
  that you don't have to look at outdated docs or do an expensive migration.
• 1:14:37
  I mean,
• 1:14:37
  for example,
• 1:14:38
  if I could wave a magic wand and move all my micro repos to monitor repo and
• 1:14:42
  everything just worked,
• 1:14:43
  that'd be awesome.
• 1:14:44
  SPEAKER_1: I don't think any system can do that today yet.
• 1:14:47
  Maybe it would in the future,
• 1:14:48
  but if that's a reality because you designed your software to move that way,
• 1:14:52
  that's great.
• 1:14:53
  So having that knowledge is not because AI agents are fundamentally based on
• 1:14:58
  expert knowledge.
• 1:14:59
  So if your knowledge is understanding of your entire business,
• 1:15:03
  because they can't really tell you how should you write your code for end user,
• 1:15:08
  what matters to one business may not matter to the other.
• 1:15:11
  Some parts of the software might work well for one segment but not work so well
• 1:15:14
  for the other segment.
• 1:15:15
  So really understanding those business values will differentiate just AI
• 1:15:19
  driven code versus how do you actually use that code to make your business
• 1:15:23
  actually thrive.
• 1:15:25
  So again,
• 1:15:27
  it might not happen,
• 1:15:28
  right or not,
• 1:15:29
  or it might happen slower than we expect,
• 1:15:30
  et cetera.
• 1:15:31
  But what I'm hearing is even if it does happen,
• 1:15:34
  having expert knowledge helps having tastes and you kind of get taste from
• 1:15:39
  seeing things or failing things.
• 1:15:41
  So is it safe to say that it's probably a pretty good strategy to try your best
• 1:15:45
  to go and work at companies that are either building stuff at scale or they
• 1:15:49
  just have kind of taste,
• 1:15:50
  they're startups that are known to do these things so that you can build up one
• 1:15:55
  of the muscles of scale taste if you can switch routine them?
• 1:15:58
  Because in the future,
• 1:16:00
  I assume that some of the repeated,
• 1:16:04
  the successful people who have a pattern will probably have the pattern of like,
• 1:16:09
  oh,
• 1:16:10
  they worked at companies that did pretty groundbreaking stuff at their time
• 1:16:14
  and now that we have these tools,
• 1:16:15
  well,
• 1:16:16
  yeah,
• 1:16:16
  they're doing even more groundbreaking stuff and they probably build a
• 1:16:18
  bunch of their own experience.
• 1:16:20
  SPEAKER_1: The experience matters a lot knowing a lot of the stack helps.
• 1:16:24
  When I was at Uber,
• 1:16:25
  I didn't do much work on front end,
• 1:16:26
  but now I'm at guitar,
• 1:16:28
  I'm doing front end,
• 1:16:29
  I'm doing infra,
• 1:16:30
  I'm doing a bunch of stuff and I'm using agents to help me.
• 1:16:34
  We use our own agent to rewrite a bunch of our stuff,
• 1:16:36
  and I was like,
• 1:16:37
  okay,
• 1:16:37
  now I understand why this works that way.
• 1:16:38
  SPEAKER_1: I'm a general engineer now.
• 1:16:40
  SPEAKER_1: I have strong fundamentals that say I can see the system.
• 1:16:43
  SPEAKER_1: I know this is how it's supposed to work.
• 1:16:45
  Now I've uplevel myself without spending is kind of like that matrix movie
• 1:16:49
  where New York gets that upgrade to no kung fu,
• 1:16:54
  but doesn't mean that he can beat the master yet.
• 1:16:56
  SPEAKER_1: He still has to train up and understand
• 1:16:58
  SPEAKER_0: It.
• 1:16:58
  SPEAKER_0: Yeah, I like that.
• 1:17:01
  SPEAKER_0: So with that, but let's go with some rapid questions.
• 1:17:05
  I'll just shoot out some things and then you go,
• 1:17:08
  what is your favorite programming language and why?
• 1:17:11
  SPEAKER_0: Rust.
• 1:17:12
  SPEAKER_0: Rust.
• 1:17:13
  SPEAKER_1: Since when?
• 1:17:14
  Yeah,
• 1:17:15
  since guitar actually,
• 1:17:16
  I was initially like,
• 1:17:17
  oh my god,
• 1:17:17
  rust has a bad reputation for being hard,
• 1:17:19
  but it makes things so safe.
• 1:17:23
  It's hard to have,
• 1:17:24
  I think outages if you design it right,
• 1:17:26
  I mean you can still have them,
• 1:17:27
  but you can really understand the type safety is there to help you out,
• 1:17:31
  and you start treating it like your friend.
• 1:17:33
  SPEAKER_1: Then it unlocks a lot of stuff.
• 1:17:34
  SPEAKER_1: You
• 1:17:35
  Coming someone who you've spent a lot in Java and go,
• 1:17:39
  right,
• 1:17:39
  you've probably spent half your career in those.
• 1:17:42
  SPEAKER_1: Yeah.
• 1:17:42
  It's funny because when I graduated college,
• 1:17:44
  I told my friends I would never work in a company that does Java,
• 1:17:47
  and then I did Java for nine years.
• 1:17:51
  SPEAKER_1: So never say never.
• 1:17:55
  SPEAKER_0: What's an AI powered dev tool that you use?
• 1:17:57
  SPEAKER_0: And
• 1:17:59
  SPEAKER_1: We use our own agent thing quite a bit.
• 1:18:01
  SPEAKER_1: I call it Jimmy, which is a new agent.
• 1:18:05
  SPEAKER_1: Jimmy.
• 1:18:05
  Yeah,
• 1:18:05
  because guitar,
• 1:18:09
  we use the cursor as well for the auto complete,
• 1:18:12
  but we feel like Jimmy gives a lot more end-to-end stuff for us.
• 1:18:16
  So we've been using it pretty heavily,
• 1:18:18
  but you can also try it out hopefully soon.
• 1:18:20
  And then I use a bunch of other smaller CLO directly,
• 1:18:24
  obviously for question and answer stuff,
• 1:18:26
  how to play with some of the newer deep seek research models and stuff like that,
• 1:18:30
  because I've been getting good things about them for more deeper work,
• 1:18:33
  buthaven'thadachance
• 1:18:34
  SPEAKER_0: Yet.
• 1:18:34
  SPEAKER_0: What's a book that you'd recommend?
• 1:18:40
  My favorite book,
• 1:18:41
  I don't read too much,
• 1:18:42
  but I read this book early in my career called Headfirst Design Patterns,
• 1:18:46
  which is a little old at this point,
• 1:18:49
  but the patterns are really good because it really makes you understand how
• 1:18:53
  you can layer things.
• 1:18:54
  A lot of software today is just abstraction or abstraction or abstraction if
• 1:18:58
  you think about them.
• 1:18:59
  So really helps you understand what pattern makes sense,
• 1:19:02
  when and when to use it,
• 1:19:04
  how to transition between them.
• 1:19:05
  I would highly recommend that if you're trying to go into a more systems level
• 1:19:09
  approach and want to design software for maintainability,
• 1:19:12
  that's very crucial.
• 1:19:13
  SPEAKER_1: Making it pluggable, composable and replaceable under the hood.
• 1:19:17
  A lot of the migrations we did at Uber was zero downtime with no code fees.
• 1:19:22
  If we did the Java migration,
• 1:19:23
  there was no code fees,
• 1:19:25
  so that was accounting achievement.
• 1:19:27
  And IT banks these days still,
• 1:19:30
  they'll shut down their thing for some migration for a whole day on the
• 1:19:34
  weekend in 2025.
• 1:19:36
  SPEAKER_0: It's a big difference, as you say.
• 1:19:39
  SPEAKER_0: Well, it got some.
• 1:19:39
  SPEAKER_0: I really enjoyed this conversation.
• 1:19:42
  It was good to reconnect and just see how developer productivity empowering developers,
• 1:19:49
  you're still just running with it.
• 1:19:53
  SPEAKER_1: Yeah, I think this is the area I love working in this area.
• 1:19:56
  SPEAKER_1: I can't see myself doing anything else.
• 1:19:58
  SPEAKER_1: I've been doing it for 10 years.
• 1:19:59
  SPEAKER_1: I hope to do it for 10 more.
• 1:20:00
  SPEAKER_1: We'll see.
• 1:20:01
  SPEAKER_1: But I hope I can make more developers productive.
• 1:20:04
  I hope you also found this episode about Uber's internal tools develop
• 1:20:07
  productivity and AI's impact on software engineering.
• 1:20:10
  As interesting as I did,
• 1:20:11
  you can find Gotham on social media as linked to the show notes below,
• 1:20:14
  and check out his company guitar@guitar.ai.
• 1:20:17
  For more deep dives on Uber's engineering culture,
• 1:20:19
  check out the pragmatic engineering articles,
• 1:20:21
  LinkedIn,
• 1:20:22
  the show notes below.
• 1:20:23
  If you've enjoyed this podcast,
• 1:20:24
  please do subscribe on your favorite podcast platform and on YouTube.
• 1:20:28
  This helps more people discover the podcast and a special thank you if you
• 1:20:31
  leave a rating.
• 1:20:32
  SPEAKER_0: Thanks and see you in the next one.