Latest / The Pragmatic Engineer / Shipping projects at Big Tech with Sean Goedecke

Transcript

Shipping projects at Big Tech with Sean Goedecke · The Pragmatic Engineer

• 0:00
  SPEAKER_0: Successful projects, there's a narrow path to success and there's a million paths to failure.
• 0:04
  SPEAKER_1: A narrow path to success.
• 0:06
  SPEAKER_0: The default state of a project is to fail.
• 0:08
  SPEAKER_0: Projects only succeed because people drag them kicking and screaming to the finish line.
• 0:12
  SPEAKER_0: If you leave a project alone and nobody's worrying about whether it's going to ship, nine times out of 10, that project will not ship.
• 0:18
  SPEAKER_0: The ability to come up with a technical demo, moving mountains, one demo of like, "Hey, look at this thing happening," could cut through weeks of conversation about whether it's possible.
• 0:27
  SPEAKER_0: And if you just go off and produce that, you can really blow people out of the water.
• 0:31
  SPEAKER_0: In the context of a large company, being technically strong if you work on software is really a superpower because it makes you connected to reality.
• 0:37
  SPEAKER_0: It makes you connected to the real world, to the actual product that you're building.
• 0:41
  SPEAKER_0: It's sort of like a lot of the work these other people are doing around strategy, and marketing and figuring out what needs to be built, that's a huge multiplier.
• 0:48
  SPEAKER_1: How do you ship projects at big tech companies?
• 0:50
  SPEAKER_1: Sean Goedecke has worked as a staff engineer at two companies that employ thousands of software engineers.
• 0:55
  SPEAKER_1: He spent five years at Zendesk and now works at GitHub.
• 0:58
  SPEAKER_1: Sean wrote a blog post about this topic titled How I Ship Projects at Big Tech Companies.
• 1:03
  SPEAKER_1: The post went viral online and sparked a lot of heated discussion.
• 1:06
  SPEAKER_1: In this episode with Sean, we dived into what does shipping projects actually mean when you're working inside a larger company.
• 1:13
  SPEAKER_1: What is more important in order to be seen as someone who ships projects, your technical skills, an engineer, or your soft skills to navigate the organization?
• 1:20
  SPEAKER_1: The story of why Sean was considering looking for a new job when his manager told him to not worry too much about test failing in the staging environment.
• 1:27
  SPEAKER_1: How did GenAI tools change the ability for us engineers to ship projects?
• 1:31
  SPEAKER_1: Sean is a great person to talk about this as he's worked on the GitHub co-pilot team and now works on GitHub models.
• 1:37
  SPEAKER_1: Advice Sean has to succeed in a full remote team when working across time zones and a lot more.
• 1:43
  SPEAKER_1: If you enjoy the show, please subscribe to the podcast on any podcast platform and on YouTube.
• 1:48
  SPEAKER_1: Sean, welcome to the podcast.
• 1:51
  SPEAKER_0: Thanks.
• 1:51
  SPEAKER_0: It's great to be here.
• 1:52
  SPEAKER_0: I'm a big fan of all your stuff, going back many years, so really exciting to get to talk to you.
• 1:56
  SPEAKER_1: So, to start off, what is shipping?
• 1:59
  SPEAKER_1: In your post, you wrote, and I'm going to quote you, "I know it sounds extreme, but I think many engineers do not understand what shipping is even inside a tech company."
• 2:06
  SPEAKER_1: So, what is it?
• 2:08
  SPEAKER_0: My post came from this experience of seeing a lot of engineers around me and myself in the past, getting into this frustrating position where they were delivering code.
• 2:17
  SPEAKER_0: They were feeling happy about their output, particularly the volume of their output.
• 2:21
  SPEAKER_0: But they weren't getting the kind of recognition they felt they deserved.
• 2:25
  SPEAKER_0: And the management chain seemed happy or seemed unhappy, or even worse, just seemed totally indifferent to what they were doing.
• 2:32
  SPEAKER_0: So, that's the context in which I talk about shipping and the context in which I wrote my blog post.
• 2:37
  SPEAKER_0: So, in that context, I say that shipping is kind of a socially constructed fact, which sounds like this postmodern way of talking about things.
• 2:47
  SPEAKER_0: But it actually cashes out really concretely into the proposition that a project is shipped when your management chain believes that it's shipped and talks about it being shipped.
• 2:57
  SPEAKER_0: So, shipping means doing things that your management chain is happy with.
• 3:02
  SPEAKER_0: And I say management chain, because for small projects, this can be your manager and just your manager.
• 3:07
  SPEAKER_0: But for larger projects, it's usually a layer or two above your manager, particularly for staff roles.
• 3:13
  SPEAKER_0: And sometimes it goes all the way up to the CEO.
• 3:17
  SPEAKER_0: And in those cases, the metric of whether a project ships is, is the CEO happy with what you've done.
• 3:25
  SPEAKER_0: That can be a frustrating way to think about things.
• 3:27
  SPEAKER_0: I think for a lot of engineers, certainly the commenters on Hacker News found it that way.
• 3:32
  SPEAKER_0: I think partially because it relinquishes a lot of control.
• 3:35
  SPEAKER_0: It's not necessarily about the quality of your work, it's about how it's received.
• 3:40
  SPEAKER_0: But I do think you have a bit of a happier experience going into these projects with open eyes.
• 3:45
  SPEAKER_0: And I think you tend to be more successful if you're actually driving for this kind of organizational definition of shipping, rather than just saying, "I closed all the JIRA tickets, so the project's shipped.
• 3:55
  SPEAKER_0: I deployed the code to production, so the project's shipped."
• 3:59
  SPEAKER_0: That definitely does not mean that it's shipped.
• 4:01
  SPEAKER_0: You can deploy something to production and it can be a complete disaster.
• 4:06
  SPEAKER_1: This episode is brought to you by DX, a platform that helps organizations measure and improved developer productivity.
• 4:12
  SPEAKER_1: The DX team includes notable researchers behind the frameworks like DevEx and SPACE.
• 4:17
  SPEAKER_1: So, they're constantly asked by leaders, what metrics should we use to measure developer productivity?
• 4:22
  SPEAKER_1: To help simplify the landscape, DX just recently published the DX Core 4, a unified framework for measuring developer productivity that encapsulates DORA, SPACE and DevEx.
• 4:33
  SPEAKER_1: The DX Core 4 includes four dimensions, including speed, effectiveness, and quality that provide a focused set of metrics that help track productivity at all levels of the organization.
• 4:43
  SPEAKER_1: You can learn more about the DX Core 4 and how it's being used companies like Dropbox, Etsy, Verso, and Pfizer at getdx.com/core4.
• 4:52
  SPEAKER_1: Again, that's getdx.com/core4.
• 4:57
  SPEAKER_1: Something that feels wrong about this definition is it sounds like we've shared successfully if the management chain all the way up to CEO is happy and they're like, "All right, great job."
• 5:08
  SPEAKER_1: But it goes really against I think what a lot of us feel is shipping, which is does it work for customers, end users, the people who care about it.
• 5:16
  SPEAKER_1: This goes a little bit back to Agile, the original Agile manifesto, which talks about customers, doing stuff for customers.
• 5:24
  SPEAKER_1: How do you think that is connected?
• 5:27
  SPEAKER_1: Because clearly, I can see on one end if the customers are happy, the CEO is happy.
• 5:31
  SPEAKER_1: But you might have things where, I don't know, the customers are not happy and the CEO is not happy.
• 5:35
  SPEAKER_1: You've been doing this for quite a while, but surely you needed to reconcile this and also think about this fact.
• 5:41
  SPEAKER_0: I want to say a couple of things on this.
• 5:43
  SPEAKER_0: First, you touched on this in the question, but I do want to stress that in any remotely healthy company, your management chain wants customers to be happy.
• 5:52
  SPEAKER_0: They want customers to be happy because they want customers to be paying money.
• 5:55
  SPEAKER_0: They want more people to be using the service.
• 5:57
  SPEAKER_0: You should be aligned with your management chain's goals here.
• 5:59
  SPEAKER_0: If your management chain want customers to be unhappy and want the company to make no money, then you might ship and make them happy by deploying code.
• 6:07
  SPEAKER_0: Well, by doing whatever.
• 6:08
  SPEAKER_0: But you probably shouldn't be at that company.
• 6:10
  SPEAKER_0: You're definitely not going to have a fulfilling experience.
• 6:12
  SPEAKER_0: So, I think this idea that you are making your bosses happy instead of making customers happy, presupposes that you're working at a company where those things are in tension.
• 6:22
  SPEAKER_0: Certainly, in my experience, I've been lucky enough that at Zendesk and now at GitHub, people tend to want customers to be happy.
• 6:30
  SPEAKER_0: The other thing is sometimes your management chain has good reason to make decisions that are going to make customers unhappy.
• 6:36
  SPEAKER_0: And sometimes you don't have the context for that.
• 6:39
  SPEAKER_0: So, there's lots of reasons that are very important to the company.
• 6:43
  SPEAKER_0: We could spend the entire rest of the podcast listing them off and that might be a really interesting podcast.
• 6:47
  SPEAKER_0: But just to give you an example, sometimes a company is in trouble and needs to position itself as being in a particular SPACE.
• 6:54
  SPEAKER_0: Recently, AI is a good example of this.
• 6:57
  SPEAKER_0: So, you need to ship some kind of box-ticking feature in order to have a really positive impact to the success of the company.
• 7:06
  SPEAKER_0: Users might not like it, that's a shame.
• 7:07
  SPEAKER_0: It would be better if they did, but sometimes you need to ship it anyway.
• 7:11
  SPEAKER_0: I want to be clear that GitHub is not an example of this.
• 7:13
  SPEAKER_0: Our AI features are actually top-notch.
• 7:16
  SPEAKER_0: That's part of the reason I'm here.
• 7:20
  SPEAKER_0: Another really common reason why companies might want to justifiably ship something that users don't like is security and regulatory compliance.
• 7:28
  SPEAKER_0: If you've ever been at a company that's good, that's gone for say FedRAMP, you'll have to ship features that satisfy these [inaudible 00:07:37].
• 7:37
  SPEAKER_1: What is FedRAMP?
• 7:37
  SPEAKER_0: Oh, I'm sorry.
• 7:39
  SPEAKER_0: It's a US government regulatory set of requirements, I guess that make it so, as I understand it, the entities within the US government can use your software.
• 7:51
  SPEAKER_1: Yep.
• 7:52
  SPEAKER_1: Yeah.
• 7:56
  SPEAKER_1: It's a series of regulations if you want to do business.
• 7:59
  SPEAKER_1: And there's, I guess examples, not just this, but a lot of other regulations.
• 8:02
  SPEAKER_0: Yeah, exactly.
• 8:03
  SPEAKER_0: And some of these things, particularly in the realm of security controls, mean that you're compromising the customer experience a little bit and doing things that customers won't like.
• 8:12
  SPEAKER_0: Because for instance, you have to be more careful about the way you manage API keys.
• 8:17
  SPEAKER_0: You might have to make them expire over time.
• 8:19
  SPEAKER_0: Users really like having their API keys live forever.
• 8:22
  SPEAKER_0: It's very convenient, but you can't always do that.
• 8:26
  SPEAKER_0: Those are two reasons.
• 8:27
  SPEAKER_0: There's a million others.
• 8:28
  SPEAKER_0: But just to tie it back to the original question.
• 8:31
  SPEAKER_0: The fact that you're writing code that doesn't make customers happy might make you feel bad as an engineer, but it's not necessarily a counter to the interests of your company.
• 8:42
  SPEAKER_0: And there's lots of cases where I think it's quite literally your job, particularly as a very senior engineer, to suck it up and to prioritize the company's goals over your own goals.
• 8:56
  SPEAKER_0: And how you might be building the project if it was just your personal project and it didn't have the stakes that working at a large company has.
• 9:03
  SPEAKER_1: You touched on how as a senior engineer, very senior engineer, staff level engineer, it is your job to help the business, to understand the business as well.
• 9:14
  SPEAKER_1: What was the point where you recognized this, you learned this?
• 9:17
  SPEAKER_1: Because this is typically a learning that, I mean ideally it comes as you grow.
• 9:23
  SPEAKER_1: But what was the point where you realized that is my job now that I'm at this staff or above level?
• 9:30
  SPEAKER_0: Yeah.
• 9:31
  SPEAKER_0: I learned this one the hard way.
• 9:34
  SPEAKER_0: Sort of the story of mistakes.
• 9:37
  SPEAKER_0: So, I was a very enthusiastic and very motivated engineer early on.
• 9:43
  SPEAKER_0: I was just happy to be there.
• 9:44
  SPEAKER_0: I was really excited to not be in academia anymore, so I was absolutely absorbing things.
• 9:52
  SPEAKER_0: I cared a lot about what I was doing, which was good, but it pushed me into a couple of pathways that I think weren't super productive.
• 10:01
  SPEAKER_0: So, one of them was back at Zendesk, or at this point, six years ago, we had this suite of staging tests for the staging environment that were pretty unreliable for a whole bunch of reasons not worth going into.
• 10:16
  SPEAKER_0: But it was my holy mission for six months to keep them green and keep them up to date.
• 10:23
  SPEAKER_0: And I shipped fixes, I shipped improvements, I shipped reliability improvements to the tests.
• 10:27
  SPEAKER_0: I spent a lot of time and effort catching up to other engineers' work, where they shipped some feature that changed the UI, which changed the test selector.
• 10:35
  SPEAKER_0: This was something I cared a lot about.
• 10:39
  SPEAKER_0: And I felt like I wasn't getting a lot of rewards for it, but people seemed happy about it.
• 10:46
  SPEAKER_0: And then after about six months, I said something in a meeting where I complained about some feature we did that broke the test again.
• 10:55
  SPEAKER_0: And my manager said something like, "Oh, well, you just don't have to fix them."
• 11:00
  SPEAKER_0: And I was so mad.
• 11:01
  SPEAKER_0: I was so angry I left the meeting.
• 11:05
  SPEAKER_0: I think I actually left the meeting midway, which I've never done before or since.
• 11:08
  SPEAKER_0: I just got up and left and I went into another room.
• 11:11
  SPEAKER_0: This was in the office.
• 11:12
  SPEAKER_0: I went into another office and I updated my resume.
• 11:18
  SPEAKER_0: That was what I did.
• 11:20
  SPEAKER_1: Was this feeling of engineering craft is not, I don't know, the number one or what was it?
• 11:27
  SPEAKER_1: It just went against your core beliefs at that time?
• 11:30
  SPEAKER_0: I think I had just put in so much work that honestly a lot of it was a sunk cost thing.
• 11:36
  SPEAKER_0: I felt that it had to be important, because I had spent so much time on it.
• 11:39
  SPEAKER_0: In hindsight, I did think it offended my sensibilities to have tests that were constantly failing.
• 11:47
  SPEAKER_0: That felt bad as an engineer.
• 11:51
  SPEAKER_0: I was so mad.
• 11:54
  SPEAKER_0: And then I think a couple of days later, when I recovered, when I felt good again, it instilled in me this sense that you can care a lot about something and you can have good engineering reasons for caring about it, and the company might not care at all.
• 12:13
  SPEAKER_0: And if you're in that position, that's your fault.
• 12:16
  SPEAKER_0: That's not the company's fault.
• 12:19
  SPEAKER_0: You either have to deal with that and accept that you're not going to get rewards for doing this work that you're doing and not expect them, or you need to reprioritize so you are more in line with what the company wants.
• 12:34
  SPEAKER_0: That's sort of what I took from that experience.
• 12:36
  SPEAKER_0: And it took a couple more instances of me being invested in things and suffering this misalignment between what I wanted and what the company wanted.
• 12:45
  SPEAKER_0: But that was definitely the first kind of impulse.
• 12:49
  SPEAKER_0: And I think I was a mid-level engineer at that point.
• 12:51
  SPEAKER_0: I don't think I was even senior.
• 12:52
  SPEAKER_0: But that started the pathway for me to be thinking about these things in the way I do today.
• 12:58
  SPEAKER_1: But how did you reconcile this?
• 13:00
  SPEAKER_1: I can feel your frustration on why.
• 13:03
  SPEAKER_1: If you're a test suite, why I have a test suite, which is not green all the time, et cetera?
• 13:07
  SPEAKER_1: But in the end, how did you come to understand why either was not important or not as important as you thought for these staging tests specifically?
• 13:16
  SPEAKER_1: Or what was important?
• 13:17
  SPEAKER_1: What did the company care about that maybe you missed at the time?
• 13:21
  SPEAKER_0: I think I made the mistake that a lot of more junior engineers make of seeing something written down as a procedure or a rule and assuming that it was true.
• 13:32
  SPEAKER_0: So, I had thought that this social fact that you couldn't deploy with red tests.
• 13:37
  SPEAKER_0: So, we had to get the test green every time we deployed, which was a lot of work.
• 13:41
  SPEAKER_0: I thought that was an immutable fact.
• 13:43
  SPEAKER_0: And then sometime after, I got knocked back from this and I stopped maintaining the tests.
• 13:47
  SPEAKER_0: And they were red more often of the time.
• 13:50
  SPEAKER_0: We had this critical deployment that had to go out.
• 13:53
  SPEAKER_0: And the tests were red, the tests weren't green.
• 13:57
  SPEAKER_0: And I remember we were just like, "Oh, whatever.
• 14:00
  SPEAKER_0: We'll manually test the red cases.
• 14:02
  SPEAKER_0: We don't actually need to get the suite passing.
• 14:04
  SPEAKER_0: We'll just deploy the code."
• 14:05
  SPEAKER_0: And I was like, "Oh, yeah.
• 14:07
  SPEAKER_0: ", when the company needs code to go out, the company will find a way to get that code out.
• 14:13
  SPEAKER_0: And these rules that are in place to keep code quality up and stuff, those aren't hard rules.
• 14:21
  SPEAKER_0: Those are soft rules.
• 14:22
  SPEAKER_0: Those are rules that will bend if the company has a clear and present need to get something done around them.
• 14:30
  SPEAKER_0: I think that was the end of me worrying too much about those tests, I think.
• 14:38
  SPEAKER_1: Yeah.
• 14:39
  SPEAKER_1: And I'm just going to assume that these tests were more flaky.
• 14:43
  SPEAKER_1: They were less reliable, closer to end-to-end tests, which end-to-end testing suites are always a huge headache for every single company that does them.
• 14:51
  SPEAKER_1: And I always see companies investing a lot, and then scaling back and all those things.
• 14:56
  SPEAKER_1: So, there's also that part.
• 14:57
  SPEAKER_1: We're not talking about unit tests or test that signal.
• 15:01
  SPEAKER_1: If they break, there's 100% chance of regression or 99% chance of regression.
• 15:05
  SPEAKER_0: No, no.
• 15:06
  SPEAKER_0: End-to-end tests operating on a small set of accounts, such that one test could change the state of the account and fail a subsequent test.
• 15:13
  SPEAKER_0: Ruby tests, Capybara and Selenium, for anyone listening who's in a similar painful situation.
• 15:20
  SPEAKER_1: So, jumping back to shipping projects and production.
• 15:24
  SPEAKER_1: In your experience, what are things that get into the way of shipping to production, emphasis being both on shipping and production?
• 15:33
  SPEAKER_0: Yeah.
• 15:33
  SPEAKER_0: I mean this is another list of things we could go through for the rest of the podcast.
• 15:39
  SPEAKER_0: Because successful projects, there's a narrow path to success and there's a million paths to failure in my experience.
• 15:47
  SPEAKER_1: A narrow path to success?
• 15:49
  SPEAKER_0: Yeah.
• 15:49
  SPEAKER_0: Projects want to fail.
• 15:51
  SPEAKER_0: The default state of a project is to fail.
• 15:54
  SPEAKER_0: Projects only succeed because people drag them kicking and screaming to the finish line.
• 15:58
  SPEAKER_0: If you leave a project alone and nobody's worrying about whether it's going to ship, nine times out of 10, that project will not ship.
• 16:06
  SPEAKER_0: So, many, many things.
• 16:08
  SPEAKER_0: To get concrete, one common category of things is that it's some detail nobody thought of.
• 16:14
  SPEAKER_0: So, just to do a rapid fire of examples from my experience.
• 16:20
  SPEAKER_0: Hey, we got to a week before ship and we realized we've forgotten to do the security review.
• 16:25
  SPEAKER_0: Or we've gotten to before ship and we thought team X was going to build this thing, but actually they thought we were going to build this thing.
• 16:31
  SPEAKER_0: And this thing doesn't exist.
• 16:33
  SPEAKER_0: Or we were relying on this service, but this service is actually in beta.
• 16:36
  SPEAKER_0: It's not actually deployed yet.
• 16:39
  SPEAKER_0: It's slated to go out Q2 next year, sorry.
• 16:42
  SPEAKER_0: Or hey, this service is in production, but you want to run 1,000 requests a second through it and it can only support 10.
• 16:50
  SPEAKER_0: It's just not designed for your stuff.
• 16:52
  SPEAKER_0: So, it's just endless, endless, endless.
• 16:56
  SPEAKER_0: You're coordinating with some other team and there's a misunderstanding about who's delivering what.
• 17:00
  SPEAKER_0: And in big companies, projects need to go out on the order of five to 25 teams coordinating, sometimes.
• 17:11
  SPEAKER_0: There's a lot of cooks that go into shipping anything in a large company.
• 17:15
  SPEAKER_0: And it only takes two of them to be really badly misaligned to sink the whole project.
• 17:22
  SPEAKER_1: So, one thing I've observed at companies like this where it is really complex to ship, as you said, five to 25 teams collaborating together, which is a lot.
• 17:32
  SPEAKER_1: But again, if you worked at these larger companies, the ones that you have or similar size, it does happen.
• 17:39
  SPEAKER_1: But there's two ways I've seen people respond to this who are tech leads specifically.
• 17:43
  SPEAKER_1: One type of tech lead puts up their hand like, "Look, this is too much for one person to do.
• 17:49
  SPEAKER_1: Our TPM was not on top of it.
• 17:51
  SPEAKER_1: Technical program manager who typically coordinated.
• 17:53
  SPEAKER_1: It's not my fault.
• 17:55
  SPEAKER_1: What can I do?"
• 17:56
  SPEAKER_1: And then there's also the tech leads who don't have this response and they actually one way or the other go through it.
• 18:05
  SPEAKER_1: But what is your take on this?
• 18:06
  SPEAKER_1: Because when you do have such complexity, it can feel like a full-time job of just coordinating.
• 18:12
  SPEAKER_1: And there is some pushback that I hear from engineers, warranted or not, saying like, "Look, this is not software engineering.
• 18:18
  SPEAKER_1: This is project management and my main job is software engineering.
• 18:22
  SPEAKER_1: I would need someone else to do it."
• 18:25
  SPEAKER_1: When you were in this situation, what was your response, and also, especially keeping in mind that you do want to ship at the end of the day?
• 18:33
  SPEAKER_0: I think there's two coordination tasks that you are talking about there.
• 18:37
  SPEAKER_0: And luckily, they can be done in parallel or I don't think anything would ship at big companies, no matter what.
• 18:42
  SPEAKER_0: There's the task of making sure every team is actively working on the project and coordinating in communication, and agreeing that they're going to ship by a certain date.
• 18:53
  SPEAKER_0: That's what I see as more the TPM or program manager role of coordination.
• 18:59
  SPEAKER_0: But there's also ,in parallel, I think a purely technical sense in which these teams have to coordinate.
• 19:05
  SPEAKER_0: And maybe you've worked with different TPMs than I have.
• 19:08
  SPEAKER_0: But in my experience, you need an actual engineer on the team who is a full-time engineer.
• 19:14
  SPEAKER_0: Usually a senior or staff engineer to have the entire technical structure of the project in their head in order for something to ship.
• 19:20
  SPEAKER_0: And this is something I said in the blog post.
• 19:22
  SPEAKER_0: And this is something that actually was not controversial.
• 19:25
  SPEAKER_0: Everyone seemed to agree, which is that projects ship because one single person has the technical context in their head.
• 19:32
  SPEAKER_0: They don't ship because teams agree and manage to coordinate.
• 19:36
  SPEAKER_0: Teams agree and coordinate because one person among those teams has the whole flow in their head, and is able to anticipate problems and answer questions because of it.
• 19:47
  SPEAKER_0: And I think a lot of companies formalize that into having a DRI engineer, as well as a TPM and as well as all these other things.
• 19:55
  SPEAKER_0: There's a single engineer that's responsible for the technical success of a project.
• 20:01
  SPEAKER_0: And that's the person I'm talking to when I write my blog post, because that's the position I've been in a lot of times.
• 20:08
  SPEAKER_0: And you learn hard lessons.
• 20:11
  SPEAKER_1: And DRI being directly responsible individual?
• 20:13
  SPEAKER_0: Yeah, that's right.
• 20:14
  SPEAKER_0: Sorry, directly responsible individual, that's what it stands for.
• 20:18
  SPEAKER_1: So, I like how you're phrasing this.
• 20:21
  SPEAKER_1: But am I understanding it correctly that you're saying that if you're the tech lead on a project or you're an engineer on your project, aiming to be this operating at the staff plus level, regardless of your title?
• 20:35
  SPEAKER_1: You're like, "Okay, I want to know what's going on."
• 20:38
  SPEAKER_1: Then it is your job to understand the whole project, understand your team's work, how it ties with everything else, and understand all the parts, how it can go wrong and make sure that these are addressed at least.
• 20:49
  SPEAKER_1: And then if that's done, your part of the project should be fine.
• 20:52
  SPEAKER_1: There's a lot of coordination, communication that you cannot control but you can control, making sure, do I have this whole thing in my head, understanding it and having clarity and feeling good about it?
• 21:04
  SPEAKER_1: Is that roughly...?
• 21:05
  SPEAKER_0: Yeah, that's right.
• 21:06
  SPEAKER_0: That I think is what I'm saying.
• 21:08
  SPEAKER_0: The question of whether it's your job is an interesting one.
• 21:12
  SPEAKER_0: I don't actually know if people typically do this work because it's their job.
• 21:15
  SPEAKER_0: I think they typically do this work because they care about the project succeeding.
• 21:19
  SPEAKER_0: And they recognize that it's not going to succeed unless somebody steps up and thinks through the whole thing end-to-end.
• 21:27
  SPEAKER_0: But whether it's your job or not, if you are in this position of shipping these projects, it can feel like thankless work.
• 21:34
  SPEAKER_0: But in my experience, it is really not thankless work.
• 21:37
  SPEAKER_0: People do notice and people do talk about it.
• 21:40
  SPEAKER_0: And your name will come up eventually as somebody who can be relied on, which will lead you to getting more impactful work, which will lead you to promo packets and so on.
• 21:49
  SPEAKER_0: At least I've been lucky enough that that's happened to me.
• 21:52
  SPEAKER_0: So, I think whether it's your job or not, it can sort of be like perform for the job you want, not the job you have, maybe.
• 22:00
  SPEAKER_1: I'm going to plus one that.
• 22:01
  SPEAKER_1: Having been a manager on my team, I always remember people who help.
• 22:11
  SPEAKER_1: Either they bring up risks that are on projects, but they also bring solutions saying, "Oh, here's the risk.
• 22:16
  SPEAKER_1: Here's what I think I should do."
• 22:17
  SPEAKER_1: And when they do it enough times, I'm like, "Okay, this person, they're actually really good at whatever they're doing, whatever their level is."
• 22:24
  SPEAKER_1: And the lower the level, the more ammunition that is for the next promotion.
• 22:28
  SPEAKER_1: And also, on calibration meetings for promotions or performance reviews, it was very powerful argument when an engineer manager brought up, "Well, this person, they are de-risking the project.
• 22:41
  SPEAKER_1: The project shipped on time because of them, because they stepped in, even though it wasn't their job."
• 22:46
  SPEAKER_1: So, it does take, I think, a while to catch up.
• 22:48
  SPEAKER_1: But managers, usually, they want to boast about their direct reports at all these occasions, which is invisible to engineers.
• 22:59
  SPEAKER_1: I agree with you.
• 22:59
  SPEAKER_1: It's hard to go wrong with this.
• 23:01
  SPEAKER_1: So, talking about technology skills or hard skills, versus soft skills or politics.
• 23:09
  SPEAKER_1: A lot of what we talked about, shipping to production seemed like it's a bit of a soft skill.
• 23:15
  SPEAKER_1: Understand the business, know the priorities, make your management chain happy and these things.
• 23:21
  SPEAKER_1: But how do you think the technology skills fit into shipping?
• 23:26
  SPEAKER_1: You mentioned they're important, but can you give examples on how important they are or what comes first?
• 23:34
  SPEAKER_1: I'm trying to figure out if in your opinion or your experience, is one of them more important than the other.
• 23:41
  SPEAKER_0: I wrote a lot about political skill, because I think engineers typically understand that it's important to be technically strong.
• 23:47
  SPEAKER_0: And they typically don't understand that it's important to be aligned with the politics.
• 23:52
  SPEAKER_0: That's why my focus was there.
• 23:53
  SPEAKER_0: But if I had to pick one, I would say the technical skills are more important, because shipping projects, like shipping projects only succeeds... You can only ship a project when you understand that end-to-end.
• 24:08
  SPEAKER_0: And it takes a lot of technical skill to be able to understand a project end-to-end, and to be able to pivot, and to be able to answer questions that come in from various angles.
• 24:16
  SPEAKER_0: And one thing that I didn't write about a lot, but I think is really underrated, is speed in shipping projects.
• 24:22
  SPEAKER_0: Not just speed of implementation, but speed of reaction, speed of communication.
• 24:28
  SPEAKER_0: When something comes in out of left field, you need to have contingency plans right away.
• 24:34
  SPEAKER_0: You need to be communicating it right away.
• 24:36
  SPEAKER_0: When there's a block, you need to be addressing it right away because all these things add up.
• 24:40
  SPEAKER_0: Because there's so many things that can kill a project, the one grand theme that can kill a project is being too slow.
• 24:47
  SPEAKER_0: Because if you take an extra week or an extra two weeks, that's an extra two weeks that something can happen.
• 24:52
  SPEAKER_0: You really need to seize your opportunities and try and ship as quickly as possible before anything can go too badly wrong.
• 24:59
  SPEAKER_0: And you just don't have the firepower to do that without enough technical skill to be able to see through it end-to-end, which is a lot.
• 25:09
  SPEAKER_1: Yeah.
• 25:09
  SPEAKER_1: And reflecting on this, I'm just thinking back of the best tech leads I remember, especially when I was a manager on my team.
• 25:17
  SPEAKER_1: So, I think you saw it with the problem.
• 25:21
  SPEAKER_1: "Oh, here's a problem."
• 25:22
  SPEAKER_1: This team says they need another two weeks to build this stuff.
• 25:26
  SPEAKER_1: The tech lead that was okay was like, "Okay, let me talk to them, see if we can get it down to week and a half or a week, if we can reprioritize."
• 25:37
  SPEAKER_1: The awesome tech lead was like, "Why?
• 25:40
  SPEAKER_1: What is keeping them up?
• 25:43
  SPEAKER_1: I just looked at their code base and I don't really see a reason.
• 25:46
  SPEAKER_1: I'm actually talking with their engineer and they said they just need this small ticket done.
• 25:51
  SPEAKER_1: I already wrote the diff, it's ready to land for them.
• 25:54
  SPEAKER_1: We'll get them.
• 25:55
  SPEAKER_1: Block yourself for tomorrow."
• 25:57
  SPEAKER_1: And as you said, the person with the technical skill who can actually go there and understand and challenge.
• 26:04
  SPEAKER_1: And also, sometimes, especially if you're working at a company which has a internal open source policy, meaning you can make changes to their code base.
• 26:12
  SPEAKER_1: Just make the change and then the team, it's incredible how strong technical skill with the ability of getting things done can really just move things.
• 26:24
  SPEAKER_1: Have you observed similar things?
• 26:25
  SPEAKER_0: Yeah, no, I've seen that too.
• 26:26
  SPEAKER_0: I've seen that so often.
• 26:29
  SPEAKER_0: One thing I've seen is the ability to come up with a technical demo, kind of moving mountains.
• 26:34
  SPEAKER_0: One demo of like, "Hey, look at this thing happening," could cut through weeks of conversation about whether it's possible.
• 26:41
  SPEAKER_0: And if you just go off and produce that, you can really blow people out of the water.
• 26:47
  SPEAKER_0: One thing I say a lot is that in the context of a large company, which of necessity has many non-technical managers and non-technical stakeholders, being technically strong if you work on software is really a superpower, because it makes you connected to reality.
• 27:01
  SPEAKER_0: It makes you connected to the real world, to the actual product that you are building.
• 27:06
  SPEAKER_0: And it's sort of like a lot of the work these other people are doing around strategy, and marketing and figuring out what needs to be built, that's a huge multiplier on a tech company.
• 27:19
  SPEAKER_0: But it only starts multiplying when it locks into somebody with the technical skill to actually make it happen.
• 27:25
  SPEAKER_0: So, if you're that person just by having a lot of technical surface area, you can unlock the multiplicative factor of all these other people in your organization just by being willing to answer technical questions, to make technical demos, and to implement things quickly that are bothering them that they think are important.
• 27:46
  SPEAKER_0: You can really move an organization just by being technically strong, if you're willing to put that technical strength in the service of people who have thought hard about what needs doing, but don't have the technical skill to do it.
• 28:00
  SPEAKER_0: I see a lot of people who are so technically strong, but are pouring that technical strength into making the code a little bit cleaner or writing tests that are a little better.
• 28:09
  SPEAKER_0: And that can be really useful, but I think you can be doing so much more.
• 28:13
  SPEAKER_1: Can we zone in onto this specific thing?
• 28:16
  SPEAKER_1: Because I hear what you're saying, is if you are technically strong and you can surface in a way that is a lot more helpful to the business, visible to others.
• 28:26
  SPEAKER_1: But what I'm seeing is most software engineers are technically strong.
• 28:30
  SPEAKER_1: And as you say, they might be pouring it in some less visible things.
• 28:34
  SPEAKER_1: What might be some ways that I can surface this a little bit more?
• 28:38
  SPEAKER_0: I think the most impactful piece of advice I can give is just to go out and talk to people.
• 28:44
  SPEAKER_0: And that's a little spicy sometimes, because going out and talking to people circumvents some of the process that these large tech companies have in place by which communication goes through from project managers to designers, to engineers.
• 28:56
  SPEAKER_0: But if you're an engineer, and you think you have a lot of horsepower, and you're looking for a place to apply it, and you feel like the backlog on your team is not the most impactful place, sometimes the best thing you can do is to go and talk to your skip level.
• 29:09
  SPEAKER_0: Go and talk to your project manager, your product manager, your designer and just say, "What do you think needs doing?
• 29:16
  SPEAKER_0: What do you think I can work on?"
• 29:21
  SPEAKER_0: One thing that I do see engineers do this, and this is maybe not the most politically savvy way to use this superpower, but one way I have seen engineers use the superpower is to go to support, to their support counterparts who raise tickets and to invest that horsepower into making things better for support, fixing bugs that waste time for support.
• 29:40
  SPEAKER_0: This is not the kind of work that I think is going to be super impactful on a promo packet, but it is worth doing.
• 29:47
  SPEAKER_0: And I think a lot of engineers who already do that for support could be pretty well-served by taking that and also doing it for design and product, instead of waiting for these design and product plans to percolate through to epics, to issues, to scoping.
• 30:03
  SPEAKER_0: Because a lot of the time there are quick wins that you can just knock out quickly.
• 30:09
  SPEAKER_1: I like this.
• 30:10
  SPEAKER_0: It might make your manager unhappy and it might not be a sensible move, depending on the climate of your team to go around the process like that.
• 30:18
  SPEAKER_1: Yeah, I like this.
• 30:20
  SPEAKER_1: But I do think that if you're unable to go to customer support and fix a persistent issue on the side at all over a year or two years, then it is a good question of how effective can you be.
• 30:38
  SPEAKER_1: I've actually noticed that some of the most effective tech leads did do... Not on a regular basis, or they were able to do this.
• 30:45
  SPEAKER_1: So, I guess if you're trapped in looking for inspiration, this could be one way to try and also just figure out what the response is.
• 30:52
  SPEAKER_1: If your leadership team is like, "Oh, you should've not done that."
• 30:54
  SPEAKER_1: Well, do you really want to be working there when you're fixing, doing something delightful on top of the job that you have is taken like that?
• 31:03
  SPEAKER_1: Probably not.
• 31:04
  SPEAKER_1: Again, it depends on the external climate and all that.
• 31:07
  SPEAKER_1: But it can be.
• 31:09
  SPEAKER_1: I like the suggestion.
• 31:11
  SPEAKER_0: I think it's worth trying to understand why, if your leadership team comes back and says, "You should not be doing that."
• 31:16
  SPEAKER_0: Because yeah, maybe they're wrong, but maybe they're right.
• 31:18
  SPEAKER_0: Maybe there's some existential problem facing the company that you ought to be working on.
• 31:23
  SPEAKER_0: Instead, a lot of the time engineers can spin their wheels, it doesn't matter.
• 31:31
  SPEAKER_0: A problem affecting the company can be as serious as it gets and there can still be engineers spinning their wheels, because their team structure doesn't empower them to act.
• 31:39
  SPEAKER_0: I've seen that happen.
• 31:41
  SPEAKER_1: Yeah.
• 31:42
  SPEAKER_1: It's a good question on who's it down to.
• 31:44
  SPEAKER_1: Obviously, as an engineer, especially when you're senior or above, you should understand that.
• 31:47
  SPEAKER_1: But also, if you're not as experienced and the management team or your manager is not efficient in telling you this is really important, the company is on the line or our product is on the line.
• 31:59
  SPEAKER_1: Well, yeah, that's a little bit on them as well.
• 32:01
  SPEAKER_0: That's true.
• 32:02
  SPEAKER_1: Speaking of trust with your leadership team.
• 32:04
  SPEAKER_1: As a tech lead, it's really important that you do have trust with your leadership team.
• 32:08
  SPEAKER_1: And this sounds pretty vague, but it is important.
• 32:12
  SPEAKER_1: What are good ways you've seen of doing this?
• 32:15
  SPEAKER_0: So, by far, the best way to maintain trust with your leadership team is to ship, is to deliver, and is to deliver consistently.
• 32:24
  SPEAKER_0: So, if you are that engineer we talked about earlier who's saving projects, and de-risking projects and making sure things get across the line, that builds trust.
• 32:33
  SPEAKER_0: At minimum, it builds trust in leadership that you are technically strong.
• 32:39
  SPEAKER_0: And that goes a long way.
• 32:42
  SPEAKER_0: That gets you put on higher profile projects.
• 32:44
  SPEAKER_0: That gets you called in when something's important and starts this snowball.
• 32:51
  SPEAKER_0: But in order to do that, in order to ship on things that your management team cares about, I think you do have to maintain the capacity to do that.
• 33:02
  SPEAKER_0: So, what I mean by that is if you're at 120% on your team's JIRA tickets, and you're just burning down, and burning down and burning down, you're not going to have the capacity you need when something comes in that is more visible and is more higher profile.
• 33:17
  SPEAKER_0: So, I do think if you want to maintain this trust with your leadership team, one big part of that is to keep your regular working cadence at closer to 80% or 60%, and then kick it up to 120, 130 when something does come across.
• 33:31
  SPEAKER_0: And that way you can save your powder for when you really need it.
• 33:35
  SPEAKER_0: I think that's really important.
• 33:38
  SPEAKER_0: The other thing is to try and communicate like them.
• 33:41
  SPEAKER_0: So, I think a lot of engineers get the sense that talking like senior management is this kind of slimy thing, is this non-engineering value.
• 33:52
  SPEAKER_0: You're not a hacker if you use words like circle back or synergize.
• 33:57
  SPEAKER_0: And I think maybe that's true.
• 33:59
  SPEAKER_0: But if you want to build trust with your management, you need to play the game a little bit.
• 34:03
  SPEAKER_0: And you need to learn how they communicate and communicate back to them a little bit like that.
• 34:10
  SPEAKER_0: Just to make that really concrete, if you're shipping a project, the most important thing, almost more important than technically delivering is understanding what the project is for.
• 34:20
  SPEAKER_0: So, a project can be for box ticking for an audit.
• 34:24
  SPEAKER_0: It can be for attracting new users.
• 34:25
  SPEAKER_0: It can be for extracting more money from existing users.
• 34:28
  SPEAKER_0: It can be for satisfying some VP or CEO's dream of what a future product could be.
• 34:34
  SPEAKER_0: There's so many reasons and all of those mean you have to work differently.
• 34:37
  SPEAKER_0: Products for new users have to be really polished.
• 34:40
  SPEAKER_0: Products for large enterprise users don't have to be polished at all, but the requirements are usually really inflexible, stuff like that.
• 34:46
  SPEAKER_0: But you have to know what the project is for.
• 34:49
  SPEAKER_0: But management won't tell you that unless you can communicate in a way that they know how to communicate back to.
• 34:57
  SPEAKER_0: So, if you go to your manager and you say, "Hey, can I build a really crappy UI for this that is terrible to use?"
• 35:04
  SPEAKER_0: Every single manager in the world, particularly in writing is going to say, "No, of course not.
• 35:08
  SPEAKER_0: You have to build something really nice."
• 35:11
  SPEAKER_0: Because they're afraid that you're going to go and build something awful, and then point at them and say, "Hey, they told me to do it."
• 35:15
  SPEAKER_0: So, you really do have to pick up this crucial information.
• 35:21
  SPEAKER_0: You have to be able to communicate this, is it okay to make trade-offs with user experience, instead of saying, is it okay to build a crappy UI?
• 35:28
  SPEAKER_0: You have to be able to talk like that.
• 35:30
  SPEAKER_0: It's a hard lesson for some engineers to learn, I think.
• 35:34
  SPEAKER_0: But it's really important.
• 35:36
  SPEAKER_1: Yeah.
• 35:36
  SPEAKER_1: This is very interesting because it triggers memories for me where I was the tech lead on a project, which it didn't make sense for us to build it.
• 35:48
  SPEAKER_1: And I talked with my product manager and she confirmed like, "No, it doesn't make sense.
• 35:52
  SPEAKER_1: In fact, we're only doing this because there's an executive who struck some sort of deal with this company.
• 35:58
  SPEAKER_1: We needed to integrate something."
• 36:00
  SPEAKER_1: And she said, "Look, it makes no business sense.
• 36:03
  SPEAKER_1: We try to talk them out of it.
• 36:04
  SPEAKER_1: We go out details."
• 36:05
  SPEAKER_1: But it came down to something political, honestly, that they were saying, "Oh, we have a deal with this company, we cannot not do it," et cetera.
• 36:14
  SPEAKER_1: So, it became a little bit ego and she said, "Look, here's what's going to happen.
• 36:18
  SPEAKER_1: We need to build it.
• 36:19
  SPEAKER_1: There's no way out of it.
• 36:20
  SPEAKER_1: We need to do it as fast as we can.
• 36:22
  SPEAKER_1: We're going to retire it in a year.
• 36:23
  SPEAKER_1: We're going to have very, very few users of this thing relatively."
• 36:28
  SPEAKER_1: And the PM wasn't telling me.
• 36:31
  SPEAKER_1: She's like, "These are the constraints."
• 36:32
  SPEAKER_1: So, as a tech lead, I kind of turned to the team and told them like, "Look, here's the deal.
• 36:36
  SPEAKER_1: We need to build this.
• 36:38
  SPEAKER_1: We can do it quickly.
• 36:39
  SPEAKER_1: We can skip some of the tests.
• 36:41
  SPEAKER_1: It kind of needs to borderline work, but we don't need to pour much complexity."
• 36:45
  SPEAKER_1: And this was a complex project.
• 36:46
  SPEAKER_1: So, in the end, actually it helped us deprioritize a lot of the edge cases.
• 36:50
  SPEAKER_1: It was like, don't care, just add an error message.
• 36:52
  SPEAKER_1: Don't do this.
• 36:53
  SPEAKER_1: And in the end, actually what happened is we shipped it.
• 36:54
  SPEAKER_1: We had a ridiculously low number of users as we predicted and we were tired.
• 37:00
  SPEAKER_1: 12 months later, the code was deleted.
• 37:02
  SPEAKER_1: And on one end it was this box ticking exercise, which didn't make sense, but we had to do it.
• 37:10
  SPEAKER_1: And the shorter we did it, the better it was.
• 37:12
  SPEAKER_1: It was the first time I realized that you could do stuff that doesn't really make sense, but it was a necessity.
• 37:18
  SPEAKER_1: And we could have argued a lot more or dragged our feet about it, but in the end we're like, "You know what?
• 37:24
  SPEAKER_1: Let's do it quickly, ship it.
• 37:25
  SPEAKER_1: Thumbs up."
• 37:27
  SPEAKER_0: I'm really impressed by the trust relationship that you had with your team and with your product manager, that you could just be upfront about it and say, "Hey, we're doing this to check a box.
• 37:37
  SPEAKER_0: We can trade off user experience.
• 37:38
  SPEAKER_0: No problem."
• 37:40
  SPEAKER_0: That sounds like it made things easier.
• 37:43
  SPEAKER_1: It was a really close relationship with the product manager, where we were really kind of one unit.
• 37:49
  SPEAKER_1: It also helped that we were in a different office location from HQ.
• 37:53
  SPEAKER_1: I'm not sure if at HQ it would've been said this clearly, just because then people talk.
• 37:58
  SPEAKER_1: And I think it undermines leadership morale when you say... Basically, the leadership, we don't agree with leadership and we're right to do so.
• 38:06
  SPEAKER_1: And actually, the data showed that we're right to do so, but either way.
• 38:12
  SPEAKER_1: So, one thing that is changing software engineering a lot is GenAI.
• 38:17
  SPEAKER_1: It's now in a day-to-day tool chain.
• 38:19
  SPEAKER_1: Clearly, you're working on some tools that I know you cannot talk about at GitHub.
• 38:23
  SPEAKER_1: But a lot of software engineers are using tools like GitHub Copilot or some competitors to make their work easier.
• 38:32
  SPEAKER_1: But I'm wondering, because you're working both this tooling, you're using it, you're also seeing what developers use, what parts of the software engineering job do you think are becoming easier with GenAI tools, specifically with getting things done, right?
• 38:48
  SPEAKER_0: Yeah.
• 38:50
  SPEAKER_0: Well, one thing I said earlier was that the cardinal virtue of shipping projects is speed, is getting things done quickly, because so many things can go wrong when you delay.
• 39:02
  SPEAKER_0: And one thing GenAI is really good at is speed.
• 39:05
  SPEAKER_0: It's getting you 80% of the way and it's getting you volume for free.
• 39:13
  SPEAKER_0: And one, I'm a huge believer in GenAI for programming.
• 39:17
  SPEAKER_0: I use it every day.
• 39:18
  SPEAKER_0: I use it all the time.
• 39:20
  SPEAKER_0: I use GitHub Copilot.
• 39:21
  SPEAKER_0: I chat to Copilot in the editor.
• 39:23
  SPEAKER_0: When I'm on personal projects, I use ChatGPT.
• 39:26
  SPEAKER_0: And it's just the level with which you can go from zero to something is so good.
• 39:35
  SPEAKER_0: I do think it's the kind of thing where I think engineers that tend to work on one code base and tend to not work across a broad spectrum, can sort of underrate how powerful it is.
• 39:47
  SPEAKER_0: Because it's not going to be better than you at writing Ruby and Ruby on Rails in the code base that you've worked on for five years.
• 39:55
  SPEAKER_0: You are going to be better at it than that, for sure.
• 39:58
  SPEAKER_0: But it will be better then you probably at working on Golang in your company's load balancer repo that you haven't touched and are unfamiliar with.
• 40:07
  SPEAKER_0: It'll be better at you in all these domains where you don't have experience.
• 40:12
  SPEAKER_0: And because so much of shipping projects requires understanding code and writing, usually small bits of code in repos that you're not familiar with, I think GenAI can be this real difference maker there, where you need to build a page in your company's internal admin tool in order to ship a feature.
• 40:29
  SPEAKER_0: That probably doesn't have to be super polished.
• 40:32
  SPEAKER_0: That probably doesn't have to be 100% there.
• 40:35
  SPEAKER_0: GenAI can probably do most of that for you.
• 40:37
  SPEAKER_0: That's the kind of thing where you can save a day or two.
• 40:41
  SPEAKER_0: And that compounds over the course of a project.
• 40:46
  SPEAKER_1: So, we had Simon Wilson on the podcast, who is software engineer.
• 40:50
  SPEAKER_1: He's using a lot of GenAI.
• 40:51
  SPEAKER_1: And one interesting thing he said that GenAI makes him a lot more ambitious.
• 40:56
  SPEAKER_1: So, is it fair to say that with GenAI, if you have it at your workplace, which most companies will have at this point, as a software engineer, you can just be more ambitious and do either sign up or start doing these projects or just contribute to project, which before these tooling you might've been like, "Nah, too much time.
• 41:15
  SPEAKER_1: I'm not going to be able to get my head around it.
• 41:16
  SPEAKER_1: It's this code base that I've never touched, I've never programmed this, let me just leave it alone."
• 41:21
  SPEAKER_0: I personally haven't noticed this huge increase in ambition at work, just because typically the hardest parts of complicated software that has to be correct, the hardest part is that extra 20% on top of what AI can give you.
• 41:38
  SPEAKER_0: So, I would be reluctant, I think, to swan into let's say the load balancer, like GitHub's load balancer and say, "Hey, me and Copilot built this feature that I think is really cool."
• 41:49
  SPEAKER_0: I wouldn't expect a super positive response.
• 41:54
  SPEAKER_0: Where I have found the ChatGPT or GenAI lets me be more ambitious is personal projects for sure.
• 42:02
  SPEAKER_0: When I'm working on stuff by myself or learning stuff by myself, I almost feel like I can learn anything.
• 42:07
  SPEAKER_0: I almost feel like I can get into any area so much quicker than before, just because the ability to ask questions of Copilot, of ChatGPT.
• 42:19
  SPEAKER_0: The ability to just ask endless questions of someone that never gets tired of the questions, that always gives you considered answers.
• 42:27
  SPEAKER_0: That's sort of the dream, when you're learning something new.
• 42:30
  SPEAKER_0: So, I love it for that kind of work, and I love it for prototyping and spiking things out.
• 42:35
  SPEAKER_0: And I sat down the other weekend and I was like, what would it look like if you had five models playing Texas hold'em against each other?
• 42:42
  SPEAKER_0: Who would win?
• 42:43
  SPEAKER_0: And under a day, I had that working.
• 42:47
  SPEAKER_0: And turns out it's GPT-4.0.
• 42:51
  SPEAKER_0: It's just not something I would've done, building a Texas hold'em poker simulator to back this app.
• 43:01
  SPEAKER_0: I don't think I would've bothered doing it, because it would've taken multiple days.
• 43:05
  SPEAKER_0: It would've taken a lot of effort.
• 43:06
  SPEAKER_0: But with LLMs, I just kind of had it up in half an hour.
• 43:10
  SPEAKER_0: It was great.
• 43:13
  SPEAKER_1: And this is a little different question, but related to this.
• 43:16
  SPEAKER_1: Do you see parts of software engineering that are getting actually harder as a result of more GenAI being used, more people using GenAI or just because of these tools themselves?
• 43:28
  SPEAKER_0: Yeah.
• 43:28
  SPEAKER_0: I mean, I'm in a great spot to talk about this as somebody who's worked on Copilot and is working on GitHub models.
• 43:34
  SPEAKER_0: One part of software engineering that's getting harder is building these GenAI tools, which is increasingly becoming a large part of software engineering.
• 43:43
  SPEAKER_0: LLMs are really hard to make into working software.
• 43:48
  SPEAKER_0: The text, text modality is hard to work with.
• 43:51
  SPEAKER_0: Hallucinations is hard to work with.
• 43:52
  SPEAKER_0: The fact that it's all moving so fast.
• 43:54
  SPEAKER_0: It's a really fun area, but man, is it more difficult than the work I was doing before on so many different levels.
• 44:04
  SPEAKER_0: Even just the fact that many people you work with are struggling to keep up as well.
• 44:08
  SPEAKER_0: You just don't have this baseline level of understanding that I did, say at Zendesk when I was working on apps.
• 44:13
  SPEAKER_0: And we've had apps for years and everyone had been working on app stores for years.
• 44:17
  SPEAKER_0: Everyone knew what an app store was.
• 44:19
  SPEAKER_0: Now I'm at GitHub, working on LLMs and people are learning as we go like, what are evals?
• 44:25
  SPEAKER_0: What are logits?
• 44:27
  SPEAKER_0: What does it mean to put a grammar enforcer in front of the sampler when you're enforcing this JSON schema?
• 44:36
  SPEAKER_0: How does that work?
• 44:38
  SPEAKER_0: So many things.
• 44:39
  SPEAKER_0: The learning curve is just so steep.
• 44:40
  SPEAKER_0: So, that's definitely getting harder.
• 44:42
  SPEAKER_0: If you actually have to build these tools, man, that's harder than standard engineering crud work.
• 44:49
  SPEAKER_1: Well, I guess more and more of this over time, I would think it would leak over into software engineering.
• 44:55
  SPEAKER_1: In the sense that these days, if you're a backend engineer, you're kind of expected to understand how the front end works at a conceptual level.
• 45:04
  SPEAKER_1: If need be, okay, probably not your job, but every now and then you should be able to touch a little with CSS or HTML.
• 45:12
  SPEAKER_1: Same way, I would expect the term full-stack engineer is exploding for good reason.
• 45:17
  SPEAKER_1: And I would assume that in a few years, software engineers understanding things like, what does is RAG pipeline?
• 45:23
  SPEAKER_1: Or what does it mean to train a model versus to fine-tune it?
• 45:27
  SPEAKER_1: Again, you probably don't need to do it, but just knowing some of these and increasing more concepts will just be part of the day-to-day job, especially as more companies incorporate LLMs into their different projects.
• 45:39
  SPEAKER_0: Yeah, for sure.
• 45:40
  SPEAKER_0: Assuming we still have RAG pipelines, assuming we still have fine-tuning.
• 45:45
  SPEAKER_1: Probably keep changing.
• 45:45
  SPEAKER_0: I think some of those things are definitely still up in the air.
• 45:47
  SPEAKER_0: I mean even two days ago, OpenAI released a new way of fine-tuning models that isn't low rank adaptation, so who knows where it's going to go.
• 45:56
  SPEAKER_1: Exactly.
• 45:57
  SPEAKER_1: Well, it's an exciting space.
• 45:58
  SPEAKER_0: Yeah, sure is.
• 46:00
  SPEAKER_1: Another area that is very challenging and interesting is working across time zones.
• 46:06
  SPEAKER_1: You're doing this.
• 46:08
  SPEAKER_1: You're based in Australia, so I'm assuming you work with teams who are in vastly different time zones.
• 46:14
  SPEAKER_1: What is your experience working with teams who are in different time zones?
• 46:20
  SPEAKER_1: What works well and what doesn't?
• 46:23
  SPEAKER_0: Yeah.
• 46:23
  SPEAKER_0: Well, one thing I do want to say just on the topic of me being based in Australia, is a lot of people got angry about my post about shipping.
• 46:33
  SPEAKER_0: And said that they're going to plant their feet, and be really disagreeable, and tell their manager to screw off and do whatever.
• 46:40
  SPEAKER_0: I got to say that the calculus on that looks a little different when you're an Australian engineer, and you don't live in San Francisco and you don't have 20,000 jobs that you can walk to from your front door.
• 46:51
  SPEAKER_0: So, I think one thing about being in a different time zone is I think you don't have this... there are some areas in which your experience is not like your US peers, which definitely affects how you work.
• 47:04
  SPEAKER_0: And maybe makes you a bit more prone to do what the company needs, as opposed to planting your feet.
• 47:09
  SPEAKER_0: But in terms of my direct experience working across time zones.
• 47:13
  SPEAKER_0: At Zendesk, I worked with teams in different time zones, but my team was co-located, which was nice.
• 47:19
  SPEAKER_0: And then I had a bit of a rude awakening at GitHub.
• 47:22
  SPEAKER_0: I was hired as the first Australian engineer on my team.
• 47:26
  SPEAKER_0: Just me with the goal to grow this APAC group.
• 47:31
  SPEAKER_0: But I think two weeks after I was hired, there was a hiring freeze, as there were many hiring freezes around that time.
• 47:40
  SPEAKER_0: And so, for nine months, it was like five US engineers, and a US manager, and me in APAC doing my APAC thing.
• 47:51
  SPEAKER_0: And that was a very interesting experience.
• 47:57
  SPEAKER_0: It was a very lonely experience in some ways, but there were real upsides that I think are hard to anticipate if you haven't been in that position.
• 48:05
  SPEAKER_0: One of them is that my mornings were really busy, because everybody was closing up their day and I had to catch up on everything that had happened.
• 48:13
  SPEAKER_0: But my afternoons were almost all clear, so I was in this position where every afternoon I could do deep work.
• 48:22
  SPEAKER_0: I think that really set me up for success.
• 48:26
  SPEAKER_0: I do think I'm a pretty high agency engineer.
• 48:32
  SPEAKER_0: And GitHub was lucky that it got somebody like me who is happy maybe with a little bit less support.
• 48:40
  SPEAKER_0: I do think being the solo engineer for nine months in a different time zone, not for everybody.
• 48:47
  SPEAKER_1: No.
• 48:49
  SPEAKER_1: So, now having seen both co-located teams working with different times, is also you working by yourself.
• 49:02
  SPEAKER_1: When it comes to remote work, talking to engineering managers or people who are organizing this, what is the strategy that you've actually seen work the best and most efficient?
• 49:16
  SPEAKER_1: Am I incorrect to assume that it's going to be the co-located teams where there's multiple people?
• 49:21
  SPEAKER_0: Yeah.
• 49:22
  SPEAKER_0: Definitely don't just hire one person in a time zone, if you can.
• 49:28
  SPEAKER_0: Right now, we have a squad of APAC time zone engineers and a squad of US engineers, and I think that works well.
• 49:36
  SPEAKER_0: One thing I think that's really important for managers trying to do this, and I recommend all managers do this, I think every US company should open many hiring RAGs in Australia, please.
• 49:47
  SPEAKER_0: That would make me very happy.
• 49:48
  SPEAKER_0: But one thing that I think is a really good benefit of this approach and something we've done at GitHub the whole time I've been there, is to double down on the ability to have people working, follow the sun 24/7.
• 50:01
  SPEAKER_0: So, many, many, many times when we've had critical projects or critical bugs, we've had the US squad do a bunch of work and then hand off to the APAC squad.
• 50:09
  SPEAKER_0: And then the APAC squad hands off to the US squad.
• 50:11
  SPEAKER_0: And the result is that it sounds like a silver bullet, but you can basically 2X the speed when you're delivering some things, things that whether the handoff cost is pretty minimal, like bug investigations, that kind of thing.
• 50:26
  SPEAKER_0: And we've really been able to deliver, I think some projects that we just could not have delivered if we'd been restricted to working in the US time zone only.
• 50:36
  SPEAKER_0: Users as well.
• 50:36
  SPEAKER_0: When a user reports a bug in the afternoon, and they go to sleep, and they wake up and it's fixed, that feels really good.
• 50:43
  SPEAKER_0: And that's harder to do with NA engineers.
• 50:46
  SPEAKER_0: They have to work really late for that to happen.
• 50:48
  SPEAKER_0: But we got that for free.
• 50:50
  SPEAKER_0: Anytime an important bug came in, just got given to the APAC engineers.
• 50:54
  SPEAKER_0: And if we were able to solve it by the end of the day, it was just this really good user experience of like, wow, they literally fixed it overnight.
• 51:01
  SPEAKER_0: I wake up and it works.
• 51:03
  SPEAKER_0: That's amazing.
• 51:06
  SPEAKER_0: I do think one way in which GitHub is really well set up for this kind of work is that it's very async first.
• 51:11
  SPEAKER_0: And that's super important for managing these two time zones.
• 51:15
  SPEAKER_0: Because you can't all be in meetings, you need to be producing written artifacts and demos, these things that can speak for you and do work while you're asleep.
• 51:25
  SPEAKER_0: So, definitely, you need to hire people who are comfortable both writing and reading documents, and demos, and that kind of thing.
• 51:35
  SPEAKER_0: I think that reading part's important.
• 51:37
  SPEAKER_0: I think it's comparatively easy to hire people who are happy to write demos, to write memos and write RFCs.
• 51:42
  SPEAKER_0: I think it's slightly harder to find people who are happy to read RFCs, but that's almost more important on these distributed teams.
• 51:51
  SPEAKER_0: People need to be communicating async for it to work.
• 51:55
  SPEAKER_1: And you're now in, I think an increasingly enviable position, in the sense that you are working full remote at a team and a company that does support full remote.
• 52:04
  SPEAKER_1: And we're seeing data that the percentage of these job options is unfortunately just going down a bit.
• 52:13
  SPEAKER_1: Who knows if it'll stabilize or go up in the future, but right now it is going down some.
• 52:17
  SPEAKER_1: And the willingness of engineers wanting to do that is not going down.
• 52:22
  SPEAKER_1: I'd like to ask, in your experience, what are the practices and the traits that help you excel in a full remote environment?
• 52:30
  SPEAKER_1: And what do you think you needed to actually do this?
• 52:34
  SPEAKER_1: Because as you said, you worked for nine months by yourself, and you mentioned that it's not for everyone.
• 52:39
  SPEAKER_0: Yeah.
• 52:42
  SPEAKER_0: I mean, to sound like I see some people talk on Twitter these days.
• 52:47
  SPEAKER_0: I think part of it is that I'm built different.
• 52:49
  SPEAKER_0: And I mean that in a maybe not necessarily entirely positive way.
• 52:56
  SPEAKER_0: But some people I think need human interaction, need to be talking with people they work with, whether it's on Zoom or not.
• 53:03
  SPEAKER_0: I think I can go a long time communicating with people.
• 53:05
  SPEAKER_0: I think that's just how I'm put together, which makes it a really good fit for me first and foremost.
• 53:13
  SPEAKER_0: Because I know a lot of good engineers who I think could excel in this role flame out because they can't stand being by themselves.
• 53:22
  SPEAKER_0: And that's just not something that's bothered me.
• 53:24
  SPEAKER_0: So, I've had that obstacle removed, which is really nice.
• 53:29
  SPEAKER_0: Of course, then you still have to excel.
• 53:31
  SPEAKER_0: And I think what I've focused on, what I've done, well, we've talked about it.
• 53:37
  SPEAKER_0: You can go and read the blog post.
• 53:39
  SPEAKER_0: But the one sentence is if you're in this enviable position and you want to keep it, you have to align with what the company needs from you.
• 53:50
  SPEAKER_0: And you have to take what your job is really seriously.
• 53:53
  SPEAKER_0: I see a lot of engineers who have their own goals for how they want to work.
• 53:58
  SPEAKER_0: They want to have all their code perfectly clean, or they want to always have unit tests, or they want to work on front end stuff only.
• 54:05
  SPEAKER_0: And they push those goals up against the company's goals.
• 54:07
  SPEAKER_0: And that's fine.
• 54:08
  SPEAKER_0: You can do that and you can make those decisions.
• 54:12
  SPEAKER_0: But if you're in a spot that you really want to keep, like working fully remote for a company like GitHub that I love, I have a lot of reason to try and absorb the company's goals and to try and make them my own.
• 54:25
  SPEAKER_0: And I think that's broadly been successful and has put me in this position where I'm able to keep doing this.
• 54:31
  SPEAKER_0: And hopefully, I can keep doing this for some time to come.
• 54:36
  SPEAKER_1: I like this reflection, especially on the part of you can go a long time without communicating async.
• 54:45
  SPEAKER_1: I guess a lot of this will just be learning.
• 54:47
  SPEAKER_1: But is it safe to say that... did you know what it would be like to work proper full remote before you did it, or was it just something that you needed to go through to see?
• 54:59
  SPEAKER_1: Because as I understand at Zendesk, it was more in office and together with the squad.
• 55:04
  SPEAKER_1: And this was the first time where it was a proper full remote job for you.
• 55:08
  SPEAKER_0: Well, recall the timing.
• 55:10
  SPEAKER_0: I moved to GitHub in 2021.
• 55:13
  SPEAKER_0: And Australia, Melbourne had one of the longest, one of the most extreme lockdowns in the world.
• 55:22
  SPEAKER_0: So, for a decent amount of time before I moved to Zendesk, I'd been working co-located in a time zone with people, but I'd been working from my house.
• 55:30
  SPEAKER_0: I hadn't been working from the office.
• 55:31
  SPEAKER_0: I hadn't seen these people.
• 55:33
  SPEAKER_0: So, I think that was pretty unpleasant in a lot of ways.
• 55:36
  SPEAKER_0: But in hindsight, it did give me the confidence that I could do this remote role at a company like GitHub.
• 55:43
  SPEAKER_0: And I knew that I could work without going into the office.
• 55:47
  SPEAKER_0: And before I wasn't so sure how I would cope with that.
• 55:51
  SPEAKER_0: It turns out I cope really well, but I don't know if you can anticipate that ahead of time.
• 55:55
  SPEAKER_0: You kind of just have to try it and see.
• 55:58
  SPEAKER_1: On the flip side, a lot of us who have worked throughout 2020, 2021, have likely had that experience.
• 56:04
  SPEAKER_1: And I think people can reflect on how did it feel.
• 56:07
  SPEAKER_1: Could you do this for longer?
• 56:08
  SPEAKER_1: Did you actually thrive?
• 56:11
  SPEAKER_1: Because now some people thrive, some people hated it.
• 56:14
  SPEAKER_0: Yeah, for sure.
• 56:16
  SPEAKER_0: I do think it was very different being in the same time zone as people, from being alone in my time zone.
• 56:21
  SPEAKER_0: That was a big shift.
• 56:22
  SPEAKER_0: That was something that I think I got lucky that I ended up being a good fit for that.
• 56:28
  SPEAKER_0: Because had I known that was going to be the case, I might've been a bit more skittish about taking the role.
• 56:34
  SPEAKER_0: I probably still would've done it.
• 56:36
  SPEAKER_1: Oh, yeah.
• 56:36
  SPEAKER_1: Well, some things you just got to wing it.
• 56:38
  SPEAKER_1: And sounds like that's what you've did.
• 56:40
  SPEAKER_1: Let's close up with some rapid questions.
• 56:42
  SPEAKER_1: So, I'll just ask a question and just bird out whatever comes first to mind.
• 56:48
  SPEAKER_1: What's an interesting framework or library you've recently used and what did you like about it?
• 56:54
  SPEAKER_0: I mean, this is a silly answer, but Ruby on Rails, man.
• 56:57
  SPEAKER_0: I love Ruby on Rails.
• 56:58
  SPEAKER_0: It's not new, it's not sexy, but I am a huge Ruby on Rails believer.
• 57:04
  SPEAKER_0: And it's the most programmer-friendly way to build large web apps.
• 57:10
  SPEAKER_0: And its flaws are honestly endearing to me.
• 57:13
  SPEAKER_1: Is GitHub still using Ruby on Rails?
• 57:15
  SPEAKER_1: I heard that it was started off Ruby on Rails or it migrates through Ruby on Rails.
• 57:20
  SPEAKER_1: So, internally, it's being used or this is just using it on the side?
• 57:24
  SPEAKER_0: Oh, no, no.
• 57:25
  SPEAKER_0: GitHub is still in very large part, still a Ruby on Rails shop.
• 57:29
  SPEAKER_0: We run a lot of different things for a lot of different services.
• 57:32
  SPEAKER_0: Sometimes Ruby on Rails doesn't make sense.
• 57:34
  SPEAKER_0: But if you go to GitHub and you go through github.com and you interact on GitHub, almost every single thing you do is going through Ruby Review on Rails still.
• 57:42
  SPEAKER_1: That's a good fun fact reminder.
• 57:45
  SPEAKER_0: Yeah.
• 57:47
  SPEAKER_1: What is a book that you would recommend and why?
• 57:51
  SPEAKER_0: Well, one tech book is actually underneath my mic right now.
• 57:54
  SPEAKER_0: It's The Little Book of Deep Learning by Francois Fleuret, which was something I picked up when I was getting into LLMs and I wanted to have a bit of a better understanding of the maths that was backing them.
• 58:10
  SPEAKER_0: And it was just a really nice short primer on how it worked.
• 58:13
  SPEAKER_0: Didn't go into too much detail.
• 58:15
  SPEAKER_0: It was excellent.
• 58:16
  SPEAKER_0: But to recommend a book that I read every year, I got a shout out The Name of the Rose by Umberto Eco.
• 58:23
  SPEAKER_0: I read it every year.
• 58:24
  SPEAKER_0: It's an incredible book about an Italian monastery in the 1300s, where somebody commits a murder and then they have to do an investigation.
• 58:31
  SPEAKER_0: I think it's a very funny book, but it's a book about people doing their best in complicated systems, which kind of speaks to me and maybe some of the things we've talked about.
• 58:39
  SPEAKER_0: So, Name of the Rose by Umberto Eco.
• 58:40
  SPEAKER_0: Go read it.
• 58:41
  SPEAKER_1: Well, thank you very much for sharing all these details and being on the podcast.
• 58:46
  SPEAKER_0: No problem.
• 58:46
  SPEAKER_0: It was a lot of fun.
• 58:47
  SPEAKER_1: Thanks to Sean for the deep dive into the topic of shipping projects as a tech lead with the focus on how to do this at mid-size and above companies.
• 58:54
  SPEAKER_1: If you'd like to read more about Sean's thoughts on software engineering, check out his website or connect with him on LinkedIn, both linked in the show notes below.
• 59:02
  SPEAKER_1: If you've enjoyed this podcast, please subscribe on your favorite podcast platform and on YouTube.
• 59:07
  SPEAKER_1: And if you're interested in more tips on how to lead a project as a software engineer, check out the deep dives from The Pragmatic Engineer linked in the show notes below.
• 59:14
  SPEAKER_1: Thanks and see you at the next one.