Latest / The Pragmatic Engineer / Building Reddit’s iOS and Android app

Transcript

Building Reddit’s iOS and Android app · The Pragmatic Engineer

0:00
SPEAKER_0: AI coding tools.
0:01
SPEAKER_0: I'm interested.
0:01
SPEAKER_0: How much do you use the current ones?
0:03
SPEAKER_0: What do you use?
0:04
SPEAKER_0: What is working?
0:05
SPEAKER_1: I think Apple's built an Xcode one is perfectly fine.
0:08
SPEAKER_1: I use it a tremendous amount.
0:09
SPEAKER_1: I feel like chat GPT has officially replaced my knowledge of Git.
0:13
I can never remember how to use Git invocations properly,
0:16
so I just pop open a window and be like,
0:17
Hey,
0:18
can you please reeducate me on this?
0:19
I use a lot for that kind of like ad hoc things where I don't have an idea of where
0:22
to start.
0:23
SPEAKER_1: I've noticed that they're starting to do a lot more.
0:25
When you ask it a question,
0:26
it can generate a lot more complicated,
0:28
like it'll give you a integrated kind of web project or something like that.
0:32
I'm excited to look back into that,
0:33
but yet on the mobile side,
0:35
I've asked it a couple of specific wrinkly questions that I actually have to
0:39
solve on a day-to-day basis.
0:40
SPEAKER_1: I've never once gotten good code for those, but it's a good rubber duck.
0:43
If you're trying to break down a problem and carve it up into chunks and ask it
0:46
focused questions,
0:47
you might get decent code.
0:48
SPEAKER_2: So in personal projects, I've used Geminis built in Android studio.
0:52
SPEAKER_2: It's honestly pretty good.
0:53
SPEAKER_2: I don't use it to ask questions and have it write code for me.
0:55
SPEAKER_2: Honestly, I still don't believe that that's going to work very well.
0:58
SPEAKER_2: But as a very fancy, efficient, auto complete, it's fantastic.
1:02
Reddit completely rolled its I as an Android app starting in 2021 and did so
1:06
without most of us noticing any of it.
1:08
But why did they do it and how today we're real details by talking with three
1:12
members of Reddit's mobile platform team,
1:15
Laura Darcy,
1:16
Brandon Kolinski,
1:18
and Eric Koch.
1:19
We cover the sheer size of Reddit,
1:22
how they employ more than 200 mobile engineers have around two and a half
1:25
million lines of code for both iOS and Android,
1:27
more than 500 screens and hundreds of modules.
1:31
How Reddit changed their API from RESTER GraphQL and why this was more tricky
1:34
to do than most assumed it would be why Android chose Jetpack Compose when
1:39
composed was still an alpha and why iOS decided not to use Swift UI at the time
1:43
and why the iOS team is reversing course now and many more details.
1:47
If you're interested in how Fuller remote mobile engineering team operates
1:50
and makes decisions and want to understand the challenges of building
1:52
mobile apps used by tens of millions of users,
1:55
this episode is for you.
1:56
If you enjoy the show,
1:57
please subscribe to our podcast on any podcast platform and on YouTube.
2:01
SPEAKER_0: Welcome to the podcast all of you.
2:03
SPEAKER_1: Thank you.
2:04
SPEAKER_0: Thank you.
2:04
SPEAKER_1: Good to be
2:05
SPEAKER_0: Here.
2:05
So to start off,
2:06
can we get a sense of what is the scale and shape of the Reddit code base on Right
2:11
now we're talking about the mobile code base,
2:13
of course.
2:14
SPEAKER_2: Yeah.
2:14
SPEAKER_2: I guess I can start with Android.
2:16
There's some standard scale metrics,
2:19
I don't know,
2:19
two and a half or so million lines of code,
2:21
about 800 modules.
2:23
SPEAKER_2: But the thing that really surprises people is we have about 580 screens.
2:29
You think of Reddit and you think of the feed and the post detail screen,
2:33
maybe a handful of others,
2:34
but to this day,
2:35
I do not understand how we have this many screens or where they all go,
2:39
but we have a lot.
2:40
SPEAKER_2: I dunno, Brandon, you want to give the iOS take on that?
2:43
SPEAKER_1: The iOS side is actually, I think a pretty similar size.
2:45
We're behind on having a more specific concept of screen that I would trust,
2:49
but my favorite answer to this question is we're large enough that if you ask
2:54
me what scale we are,
2:55
I have to ask you clarifying questions of like,
2:57
well,
2:58
do you care about code?
2:59
SPEAKER_1: Do you care about specific kinds of targets?
3:02
We've got a thousand basal targets,
3:03
but I would have to take 15 minutes to categorize them or brently to figure it out.
3:09
SPEAKER_1: But yeah, it's big enough that we don't have to prove that it's too big.
3:14
And I guess one question for this type size of code basis,
3:17
for those who are not as familiar with,
3:19
let's say mobile is the compile time on backend and on web,
3:24
it's usually not a big deal,
3:25
although on TypeScript it could be,
3:27
but a mobile usually is these types of code bases.
3:29
A million line plus often would compile 10 minutes,
3:29
15,
3:32
sometimes even longer than that.
3:34
SPEAKER_0: What is your compile time from fresh?
3:38
SPEAKER_1: Again, this question causes me to make a face.
3:40
If you end up being a Reddit iOS developer that ends up being like,
3:43
Hey,
3:44
my build times.
3:44
I think if you do the entire app everything,
3:47
it's probably about 30 minutes,
3:49
but that'll cache a bunch of things,
3:51
and after that it's going to be way faster.
3:54
But if you are exposed to a 30 minute build time,
3:58
we're probably going to see you on a chart and be like,
4:00
hi,
4:00
why are you building the whole app?
4:01
SPEAKER_1: Would you like to be more productive?
4:02
SPEAKER_1: Can you try a couple of these different techniques?
4:05
SPEAKER_1: It's really hard to make that any faster.
4:08
SPEAKER_2: Yeah, on the end, Android side, incremental builds are not bad at all.
4:12
I think our clean builds are somewhere in the neighborhood of eight to nine minutes,
4:19
but that's with remote bill caching and a lot of things that we have to help
4:22
things out.
4:24
But just to clarify,
4:25
as you said,
4:26
random as well,
4:27
you wouldn't really do a clean build except maybe on your first day or when you
4:31
join or if you've been away from,
4:33
I don't know,
4:34
a long time or you do something.
4:35
SPEAKER_0: So it is just a rare event, right?
4:37
Most devs on day-to-day,
4:38
you're going to do small incremental bills that we're talking are not beyond
4:42
minutes and probably even faster.
4:43
And essentially we try to get people into select focused targets or I think we
4:49
have a couple of names for this because we've had it for a long time.
4:53
SPEAKER_1: We've had varying versions of this tech, like our playgrounds.
4:56
SPEAKER_1: We want you to take a selection of those targets and do a focus mode.
4:59
I think other places call it,
5:00
and those builds you'll vary from if you're in the feed area where you have a
5:05
tremendous amount of dependencies,
5:07
maybe that'll take 10 minutes,
5:08
but the small libraries that I might be working on,
5:11
it's like 30 seconds to a minute.
5:14
SPEAKER_1: It doesn't take any time at all If you're outside of the main project
5:18
And as context,
5:19
you mentioned the lines of code,
5:20
which we know is not a perfect metric,
5:22
but above a certain size,
5:24
it does give you a sense,
5:25
right?
5:25
Uber has a similar size,
5:27
for example,
5:27
and again,
5:28
a lot of startup will have a lot smaller.
5:31
SPEAKER_0: What is the size of a team that maintains all of this, right?
5:35
SPEAKER_0: All of Reddit's native apps?
5:37
SPEAKER_3: It's a great question.
5:39
SPEAKER_3: We have two dedicated platform teams for Android and iOS.
5:43
They're about 10 to 11 engineers a piece on a good day,
5:49
and they support about 200 mobile engineers across the company on different teams.
5:54
SPEAKER_0: Wow.
5:56
I want to say we about 20 feature teams and they are set up like a fairly typical
6:01
feature team with some Androids,
6:03
some iOS,
6:04
some web engineers,
6:05
maybe some dedicated backend engineers,
6:08
an em a pm,
6:10
a part designer,
6:11
that sort of situation.
6:13
But what's special about the platform teams is they tend to be homogenous.
6:17
SPEAKER_3: So everyone on the iOS platform team is an iOS developer.
6:20
SPEAKER_3: They're all together.
6:21
SPEAKER_3: Oh, wow.
6:23
So they develop a little bit more subject matter expertise to support all of
6:27
the feature iOS developers in their spaces.
6:30
Yeah,
6:31
because at most places,
6:32
the mobile platform team would have a mix of like 50% Android,
6:30
50%,
6:35
usually the ones I've seen.
6:38
SPEAKER_3: Yep.
6:40
We work more like sister teams instead of having one team that is both platforms.
6:45
We also have a web platform team and a UI platform team that owns the design system.
6:50
This episode is brought to you by graphite,
6:52
the developer productivity platform that helps developers create,
6:55
review and merge smaller code changes,
6:57
stay unblocked and ship faster.
7:00
SPEAKER_0: Code review is a huge time sink for engineering teams.
7:03
Most developers spend about a day per week or more review in code or blocked
7:07
waiting for a review.
7:09
SPEAKER_0: It doesn't have to be this way.
7:11
Graphite brings Stack Portal requests the workflow at the heart of the best
7:14
in class internal code review tools like companies like Meta and Google to
7:17
every solver company on GitHub.
7:20
Graphite also leverages high signal codebase aware AI to give the delivers
7:24
immediate actionable feedback on their portal requests,
7:27
allowing teams to cut down on review cycles.
7:30
Tens of thousands of developers at top,
7:31
companies like Asana,
7:33
ramp Teon and versa rely on graphite every day.
7:37
Start stacking with graphite today for free and reduce your time to merge
7:40
from days to hours.
7:42
SPEAKER_0: Get started at GT dev slash pragmatic.
7:45
SPEAKER_0: That is G four graphite for technology dev slash pragmatic.
7:49
This episode is brought to you by sentri buggy lines of code and long API Calls
7:54
are impossible to debug and random app crashes are things no software
7:57
engineer is a fan of.
7:59
This is why over 4 million developers use sentri to fix errors in crashes and
8:03
solve hidden or tricky performance issues.
8:07
SPEAKER_0: Sentri has Deb time in half.
8:09
SPEAKER_0: No more soul crushing, lock sifting or vague user reports like it broke.
8:13
SPEAKER_0: Fix it.
8:15
SPEAKER_0: Get the context.
8:15
You need to know what happened when it happened and the impact down to the
8:19
device browser and even a replay of what the user did before.
8:22
The error centric will alert the right dev on your team with the exact broken
8:26
line of code so they can push a fix fast or let auto fixx handle the repetitive
8:31
fixes so your team can focus on the real problems.
8:34
Sent help monday.com,
8:36
reduce their errors by 60% and spend up time to resolution for nextdoor by 45
8:40
minutes per death per issue.
8:43
Get your whole team on sent and seconds by heading to century.io/pragmatic.
8:47
SPEAKER_0: That is S-E-T-R-Y.
8:50
Do io slash pragmatic or use the code pragmatic on signup for months on the
8:55
team plan and 50,000 errors per month for free.
8:59
I have to ask this question because people are going to already asking it in
9:02
their heads.
9:03
SPEAKER_0: Why does Reddit have so many mobile engineers?
9:05
And by the way,
9:06
I did get this when I was at Uber as well,
9:08
and you've gotten this so many times before,
9:11
but again,
9:12
Eric,
9:12
you already mentioned the 580 screens,
9:14
which some people will just not really believe,
9:18
but we've all mentioned now more than a hundred mobile engineers,
9:21
which is a lot.
9:22
This will make you one of the largest native mobile engineering
9:27
organizations globally.
9:30
SPEAKER_0: Maybe not the single biggest one, but probably top 10.
9:33
SPEAKER_1: It's really scary when you say it that way
9:37
SPEAKER_0: In one team.
9:38
SPEAKER_0: That is right.
9:39
You will have,
9:40
I don't know,
9:40
Google if you add up,
9:42
they'll have a lot more combined but not in one app.
9:45
SPEAKER_3: Yeah, so we talked about the screens.
9:48
We have a lot of feature surfaces,
9:50
but there's also a lot of mobile developers working in areas like building
9:57
our safety features and building our eventing and our logging and our
10:01
experimentation platform.
10:05
So not everyone is focused on,
10:09
and our ads platform also is an entire organ to itself.
10:13
So while a lot of the bigger teams are teams like the feed and the post detail experience,
10:19
the things that many redditors know and love about Reddit,
10:23
there's an entire team just devoted to supporting mods.
10:28
So once you get into all of the different sorts of areas,
10:32
plus an entire Dov platform that is trying to extend the experience to third
10:36
party developers to actually build on Reddit,
10:41
it gets big fast.
10:43
The other big investment we saw,
10:44
I want to say like a year and a half to two years ago,
10:48
is we have a lot more,
10:52
we don't call them S steps,
10:53
but we actually have a lot more automation folks.
10:55
So a good portion of our actual builds are part of our test infra investment,
11:03
which a couple years ago we didn't have much test infra and now we have a wide
11:07
variety of test infra.
11:11
So they account for quite a few of the builds that we see running today too.
11:15
But by testing infra and automation,
11:17
is this end to test integration test some fancy AI stuff?
11:22
SPEAKER_0: What kind of mix?
11:22
SPEAKER_0: I'm assuming it is going to be a mix, right?
11:24
SPEAKER_3: I was about to say yes until you added the AI part.
11:28
Brandon,
11:29
Eric,
11:29
you want to talk about the different kinds of tests we have in our test pyramid?
11:33
I guess the other part of this is we did have some hackathon projects that did
11:36
some AI testing stuff.
11:37
They're just not something we use in tooling a lot,
11:39
but yet testing wise,
11:40
we have actually,
11:43
I think we pulled metrics for what our code coverage is,
11:45
but basically we ratchet to enforce unit test coverage in various areas.
11:50
We have definitely a lot of integration tests and we also use a lot of E two
11:56
stuff or black box testing for accessibility assertions and that kind of stuff.
12:03
SPEAKER_1: There's a lot that we actually do with it.
12:05
We also do weird stuff like I've been working for a while on using the UI
12:10
testing infrastructure to exercise the app and then try to connect it to
12:13
leaks to do memory leak analysis inside of our end-to-end testing.
12:17
So yes,
12:18
the answer to your question is we have all of those in a pretty good mix.
12:22
And is this simulators or do you actually have a device lab where the things
12:26
are deployed and you're holding your hat?
12:29
SPEAKER_0: So probably
12:30
SPEAKER_1: Well, yeah, I completely forgot that.
12:31
SPEAKER_1: We also have, we partner with a vendor to do that as well.
12:35
SPEAKER_1: Yes.
12:35
SPEAKER_1: So every kind of test,
12:37
SPEAKER_2: I think we've come a long way.
12:39
When I joined,
12:41
we didn't have coverage metrics,
12:43
but I want to say we were like maybe 2% unit test coverage and nothing beyond that.
12:50
So this is something that's really grown within the last,
12:53
I don't know,
12:53
two,
12:54
three years where I think our testing story is quite a bit better than it used
12:59
to be,
12:59
but I think we still have a long ways to go.
13:01
SPEAKER_2: Honestly.
13:02
I think we have the pyramid established,
13:05
but actually filling in all the bricks on the pyramid has some ways to go.
13:09
You've done way more testing than most engineering teams will do,
13:12
and especially startups starting out will not even dream of doing.
13:15
SPEAKER_0: So.
13:15
SPEAKER_0: What has been the benefit you put in a bunch of work, right?
13:19
SPEAKER_0: Because I do hear engineers asking why bother, right?
13:23
SPEAKER_0: It's going to be a lot of effort.
13:25
SPEAKER_0: They're hard to maintain.
13:26
SPEAKER_0: We know end-to-end tests, especially in mobile are slow, et cetera.
13:31
SPEAKER_0: What have you gotten out of this?
13:32
So if we go back a couple of years,
13:35
like Eric was saying,
13:37
we had almost 0% test coverage and we were having a lot of incidents in prod
13:42
that were easily preventable.
13:46
And so there was a whole shift left approach of trying to find those things
13:52
before our users were telling us about them at scale.
13:56
And so I think that our test info became a part of that,
14:00
but it was actually really hard to stand up in the first place,
14:03
and that was because we were in a tech debt situation that had really,
14:07
really long build times with no test infra in place yet.
14:12
So we didn't have a lot of capacity that we could say,
14:14
Hey,
14:15
we're willing to increase our build times to add tests in to be more protective.
14:21
So you have a kind of perfect storm of rapid growth in our engineering orgs,
14:27
so lots more people running builds.
14:29
Our builds are already really,
14:30
really long and we want to introduce all this test coverage and we're
14:34
constantly having incidents with the fact that we don't have test coverage
14:37
in place and we were mostly relying on manual QA from an overseas vendor with a
14:29
12 hour turnaround.
14:44
SPEAKER_3: So fixing things, ouch, really, really difficult.
14:49
So I think trunk based development,
14:50
so shifting we're testing and what artifacts we're testing,
14:54
introducing automation ideally in places that we felt had value first.
14:59
So if you're in a postmortem and you notice that a test could have resolved
15:04
this in a way that we would never have this problem again,
15:07
those are the tests that we really tried to evangelize doing first tests like
15:12
that issue or similar to that issue,
15:16
it's a lot easier to convince people those are tests that pull their weight
15:20
and are worth running.
15:22
But yeah,
15:23
we did have ratchets in place to just increase testing across the board and a
15:28
lot of those kind of contracting firms we brought in to actually help us build
15:32
tests and build a culture of testing,
15:37
SPEAKER_2: I will say.
15:38
SPEAKER_2: So the downsides you brought up are very real.
15:41
SPEAKER_2: We've had lots of issues with flaky tests.
15:43
People,
15:44
like I said,
15:44
we started with virtually no coverage and so now all these engineers who
15:49
maybe have never written a test in their life are now trying to test their code
15:52
that was not written to be very testable.
15:56
We ended up with a lot of bad tests and a lot of flaky tests and a lot of pain
16:00
around it.
16:01
SPEAKER_2: It is very real
16:04
Tech that exists anywhere regardless of whether it's in your app or your test
16:07
code or one of the little tidbits that was really helpful to me is we had
16:12
someone who joined Reddit after we'd gone through this process and we were
16:17
talking about some of our frustrations with the testing and he's like,
16:19
this is the first place I've worked at that added a ratchet and it worked.
16:25
SPEAKER_1: Yes, we have flaky tests.
16:26
SPEAKER_1: If you have tests, you'll always have some flak in there.
16:28
SPEAKER_1: That'll just be true.
16:28
SPEAKER_1: But you also have a lot, especially
16:30
SPEAKER_0: Mobile.
16:30
SPEAKER_0: Yeah,
16:31
SPEAKER_1: You also have a lot of tests.
16:33
And when Eric was referring to some of these iffy tests,
16:36
one of my favorite side effects of trying to build tests is you're at least
16:40
adding code paths to inject your dependencies and you can at least reach that
16:44
code and test,
16:45
which means even if you can write a bad one today,
16:47
you can probably write a good one tomorrow,
16:49
much simpler than starting from zero.
16:51
SPEAKER_1: The side effects have absolutely been worth it.
16:55
SPEAKER_0: So you're doing a lot of modern engineering practices.
16:58
One of the things that is the most modern and maybe the future,
17:02
there's a lot of pipe around it,
17:03
is AI coding tools.
17:04
SPEAKER_0: I'm interested how much do you use the current ones?
17:08
SPEAKER_0: What do you use?
17:09
What is working or how much does it help you or even if it helps you,
17:12
and I'm asking this because especially with native mobile,
17:16
these tools are not nearly as cutting edges as they are,
17:18
let's say on web or some of the JavaScript frameworks.
17:23
I'm just interested in today,
17:24
just honestly,
17:26
what are you getting out of it and what is working,
17:29
what is not really working?
17:33
SPEAKER_3: You too first,
17:34
SPEAKER_2: This is a little bit of a tricky one to answer.
17:37
SPEAKER_2: So in personal projects, I've used Gemini's built in Android studio.
17:41
SPEAKER_2: It's honestly pretty good.
17:42
I don't use it to ask questions and have it write code for me,
17:46
honestly,
17:46
I still don't believe that that's going to work very well,
17:49
but as a very fancy,
17:51
efficient,
17:52
auto complete,
17:52
it's fantastic.
17:54
SPEAKER_1: Yeah, I think Apple's built in Xcode one is perfectly fine.
17:57
SPEAKER_1: I use it a tremendous amount.
17:59
SPEAKER_1: I feel like chat GPT has officially replaced my knowledge of Git.
18:03
SPEAKER_1: I can never remember how to use Git indications properly.
18:06
So I just pop open a window and be like,
18:07
Hey,
18:08
can you please reeducate me on this?
18:11
I use a lot for that kind of ad hoc things where I don't have an idea of where to start.
18:15
I've noticed that they're starting to do a lot more like when you ask it a question,
18:19
it can generate a lot more complicated.
18:21
SPEAKER_1: It'll give you a integrated kind of web project or something like that.
18:26
I'm excited to look back into that,
18:28
but yet on the mobile side,
18:30
I've asked it a couple of specific wrinkly questions that I actually have to
18:35
solve on a day-to-day basis.
18:36
SPEAKER_1: I've never once gotten good code for those, but it's a good rubber duck.
18:40
If you're trying to break down a problem and carve it up into chunks and ask it
18:44
focus questions,
18:44
you might get decent code.
18:45
SPEAKER_1: You're always going to have to revisit it.
18:47
SPEAKER_1: But yeah, I think most of just the personal project bug standard ones,
18:53
I do wonder if these tools,
18:54
we will see a similar thing with what we're seeing with native mobile development.
18:58
SPEAKER_0: Native mobile development tooling is different than web than backend.
19:04
SPEAKER_0: We've always had, if you're not iOS, you're doing it right.
19:07
Xcode has different functionality,
19:09
Android Studio or JetBrains ID has and the challenges are just different.
19:14
So I feel what we might see certain teams using it and other domains might work
19:19
better and maybe mobile either catch up or be slower or who knows,
19:23
maybe it's a different fit.
19:26
SPEAKER_3: I think we are using it in a couple of ways though.
19:30
So I agree with Eric that rapid prototyping with the Android studio of Gemini
19:35
features are pretty cool.
19:36
That is a lot of fun to do on the side and for validating ideas before you
19:40
actually make them official Reddit ideas.
19:42
So is this fair for me to rephrase that for AI coding tools specifically for mobile,
19:47
it's somewhat useful.
19:49
Some people use it here and there,
19:50
but sounds like the bigger use case is you actually using this technology
19:55
LLMs and elsewhere to add to your workflows and eventually maybe even your
20:01
dev tools.
20:02
But it sounds like that's kind of a little bit more promising and exciting path.
20:05
SPEAKER_0: Did I sense this right?
20:07
SPEAKER_3: Yeah, I think that's right.
20:08
I think our test insights are also somewhat powered by AI now,
20:12
but we're not really using them as much as we could.
20:16
SPEAKER_3: I would love to see them more.
20:18
We had a lot of investment in test infrastructure,
20:22
but there's a lot of duplication in it.
20:24
And when a team goes and looks at how many tests they actually need and how many
20:28
are P zeros,
20:29
it's usually a very small subset of what is currently sitting there.
20:34
Being able to actually figure out what tests are pulling their weight post
20:38
ratchet is a great place I think that we might see AI in use.
20:46
SPEAKER_0: Let's talk about the interesting topic that we wanted to talk about.
20:50
SPEAKER_0: You did a really big tech mobile modernization starting from maybe 2021.
20:55
Can you tell me what was the state,
20:57
why did you decide to suddenly make a big change,
21:00
which as I understood,
21:01
also led to hiring more native engineers and how did it go?
21:07
SPEAKER_3: That's a big topic for us that we really like to talk about.
21:11
SPEAKER_3: Let's see, let me try and set the stage.
21:14
SPEAKER_3: So 2021, Reddit is growing as a company.
21:21
They are trying to establish themselves more as an international company
21:26
instead of a US domestic company.
21:30
And they're seeing the world shift from a web-based space to a mobile-based based,
21:39
especially overseas.
21:41
That led to a whole bunch of new strategy,
21:43
which led to a whole bunch of hiring across many,
21:47
many teams,
21:48
especially in the mobile space.
21:50
And so we start seeing a team that was probably in the 40 engineers,
21:50
50 engineers across both platform size and within two years we're around 200.
22:08
And that is the space in which you start from a place of having really,
22:14
really poor build times,
22:16
like two and a half hours type of a thing with no test infrastructure,
22:21
but all of these big goals.
22:24
And it's also around that time that platform teams really started to form officially,
22:29
Brenda,
22:29
you can correct me if I'm wrong here.
22:30
It was within that year or so where we kind of transitioned from a company like
22:35
many companies that kind of beg and borrow platform support from everybody
22:40
to keep things up and going to saying,
22:42
we need proper platform teams that are going to actually help us scale this way.
22:47
SPEAKER_1: I think that timeframe is about right.
22:49
Oh no,
22:49
actually,
22:49
that's definitely right because I was like,
22:52
this is my brief dip as an EM in this thing,
22:54
and I got to hire a couple people,
22:56
but we were kind of,
22:58
at least from my perspective on iOS,
22:59
we had always been about three people supporting however big the org was.
23:03
So the iOS platform team was three from the time that we had 10 or 15 engineers
23:09
until about the timeframe Lori's describing.
23:12
SPEAKER_1: And yeah, it was a lot.
23:15
SPEAKER_1: It was a tremendous amount of work.
23:17
SPEAKER_1: I can remember specific incidents about this.
23:19
But yeah,
23:20
it was really a time where we could finally start developing at least the
23:23
start of specific areas of expertise instead of platform engineers,
23:28
just being experts in everything and then having to pretend that it wasn't
23:32
stressful in context switchy.
23:34
So one of the things that happened though is we're hiring all this great
23:38
talent because we really want to invest in mobile and they're showing up and
23:43
onboarding is rough.
23:45
SPEAKER_3: They have never seen build times anything like this.
23:49
If you ask them what the tech stack is,
23:51
we're like,
23:51
what team are you joining?
23:52
It depends on what team you're joining and then we'll tell you what their tech
23:55
stack is,
23:55
maybe if there is one.
23:59
And so we're getting a ton of feedback from our existing developers and the
24:03
new people showing up that devex is not bringing them any joy.
24:09
Feature teams have these big ambitious new goals and the feature team hiring,
24:16
they join together,
24:16
they form a team and they can't execute because our code base is really slow
24:22
and they can't figure out how to actually deploy safely.
24:26
SPEAKER_3: All of the things are coming up at once.
24:30
SPEAKER_3: Yeah,
24:31
I was going to say one of my favorite,
24:34
actually least favorite I guess stories about this was when I joined.
24:37
So I guess I'll say I'm generally all four people on any team getting to know
24:42
the finer points of their tooling and the tech stack.
24:45
But there's one fact I strongly believe that not a single feature engineer
24:48
should ever know,
24:50
and that is at what point their CI provider decides that it's more likely that
24:53
a process is stuck in an infinite loop than that you're actually still trying
24:57
to build.
24:59
SPEAKER_2: But when I joined, we weren't so lucky.
25:00
I regularly had to retry PRS after they get timed out on CI after two and a half hours.
25:07
And a lot of these were from decisions that made a lot of sense when we were a
25:10
small team,
25:10
small code base,
25:12
and we just didn't have the staffing to readdress them as we grew.
25:18
SPEAKER_2: So yeah, we kind of reached a point where something had to be done.
25:22
SPEAKER_3: So it wasn't just DevX though.
25:24
SPEAKER_3: This was also bleeding out into the users and their experience.
25:28
So this is around the time where we probably had our all time worst stats on
25:33
mobile across both platforms.
25:36
SPEAKER_0: Didn't show the ratings or feedback,
25:39
SPEAKER_3: All of the above.
25:41
I started in 2021 and the week I started our Android startup time was 13
25:49
seconds at P 90.
25:52
SPEAKER_3: I think Brendan, our iOS was around seven to eight seconds, P 90.
25:56
SPEAKER_3: It was the worst of before Bolt.
25:59
I was pulled out of my onboarding the first week for a feed incident that had no observability.
26:06
SPEAKER_3: We couldn't figure out what was going on.
26:08
SPEAKER_3: Even that we were not showing feeds.
26:12
SPEAKER_3: So we were basically blind to the problems that we had.
26:16
SPEAKER_3: We were just constantly reacting to problems at that point.
26:21
I think that all of this together became a really good pivot point for the
26:29
company to say,
26:30
these are all becoming really big liabilities for us.
26:33
We need to actually have a plan for how we're going to resolve our devex issues
26:38
so that we can actually build what we want to build,
26:41
but also to dig ourselves out of hole that we were in from a existing tech debt perspective.
26:48
SPEAKER_3: It needed to be a pretty comprehensive plan.
26:54
So that kind of sets the stage for how we bring together a group of people to
26:59
define our tech stack and we're go from there.
27:02
SPEAKER_0: And then what was your journey in the modernization?
27:05
SPEAKER_0: So what were the things that you put in place?
27:08
We previously talked about before the recording about a monorepo,
27:12
which is an interesting one.
27:14
Early on,
27:14
executive leadership asked us for a prescription what was going to be our
27:20
kind big solve and ideally could we have a kind of umbrella effort that kind of
27:26
trademarked the entire approach.
27:28
And that's how we ended up with a branded name for our tech stack,
27:31
which is Core Stack.
27:33
SPEAKER_0: So this is for the new stack, right?
27:34
SPEAKER_0: Yes.
27:35
SPEAKER_3: This is our answer to everything and I'll very quickly, TLDR it.
27:45
SPEAKER_3: At the bottom of it, it is how are we going to organize our code?
27:48
We agree that we are going to organize it in a monorepo for each platform,
27:54
modularized primarily by feature.
28:00
SPEAKER_3: Above that we're going to use a modern programming language.
28:03
We're going to commit to using the modern programming language for the given platform.
28:11
We were already,
28:13
one thing that was helpful was that a lot of these particular choices were
28:17
already well underway.
28:18
They just needed to be blessed and pushed into the primary way we wanted
28:22
people to work,
28:23
which made it actually pretty easy to go down for most of these choices.
28:28
But we were in the middle of a GraphQL transition from a rest company,
28:32
so we committed to being a GraphQL first client and then a GraphQL only client.
28:39
SPEAKER_3: Above that.
28:40
We see a little bit of divergence across the two platforms,
28:44
but we settled on MVVM of some flavor.
28:49
SPEAKER_3: Do you want to explain why we chose MVVM, Eric Brenda?
28:57
SPEAKER_2: Sure.
28:58
I mean,
28:58
a lot of it was kind of forced by,
29:00
we haven't talked about this yet,
29:01
but our UI framework choice of compose.
29:05
So we were previously an MVP model view presenter architecture pattern,
29:10
which was even when I joined,
29:12
felt very out of date and a little odd,
29:15
but it was fine.
29:15
It is better to have consistency than to have a few things here and a few things there.
29:21
So we just kept going with it despite it not really being anyone's favorite.
29:27
But the issue was kind of forced when we chose Jet Pak compose because it is a
29:30
reactive framework,
29:31
so we can no longer use imperative architecture patterns to drive reactive framework.
29:35
So we kind of had to settle on something else,
29:38
which the industry has more or less settled on MVVM or MVI or some flavor of that.
29:48
SPEAKER_0: Sorry, just for those who don't know.
29:50
SPEAKER_0: So MVVM model view, view model, right, and MVP model view presenter.
29:54
SPEAKER_2: Yes.
29:55
SPEAKER_2: Yes, that is correct.
29:56
SPEAKER_0: And what is the difference between the two?
29:58
SPEAKER_0: Right.
29:58
SPEAKER_0: They sound pretty similar.
29:59
You have the model,
30:00
you have the view,
30:01
and then you have a third thing,
30:02
but they're different.
30:04
SPEAKER_2: So really what it boils down to is imperative or reactive.
30:09
There are several other differences,
30:11
but that was the main driver for us using a reactive UI framework.
30:15
SPEAKER_2: We were kind of forced to stop using imperative Logic to drive it.
30:20
SPEAKER_3: Brenda, how about the iOS side?
30:22
SPEAKER_3: We have,
30:23
SPEAKER_1: So iOS, this was a little bit more complicated.
30:26
So at that point we were evaluating Swift UI and kind of where we were at.
30:31
SPEAKER_1: I think our back deployment version on that was still iOS 14.
30:34
SPEAKER_1: So Swift UI was I think technically possible.
30:36
SPEAKER_1: I can never remember
30:37
SPEAKER_2: What it was actually 13 at the time.
30:39
SPEAKER_1: Oh, but Swift do.
30:40
I introduced some problems from that perspective,
30:42
and it also was just,
30:44
it was still pretty new.
30:45
SPEAKER_1: We're talking about 21.
30:46
SPEAKER_1: It was 3-year-old at that point and it's still pretty fresh.
30:51
SPEAKER_1: So we opted for a layer on top of basically collection view.
30:55
SPEAKER_1: We called Slice kit.
30:56
SPEAKER_1: I'm pretty sure we've had some blog posts about this.
30:58
SPEAKER_1: I can't remember exactly.
31:00
SPEAKER_1: We talked about that for a while.
31:02
SPEAKER_1: It had a lot of benefits.
31:03
There's some cool things that if you imagine wrapping a screen in a
31:09
collection view behind the scenes,
31:10
there's some really nice things like dynamic type was trivial to just get
31:14
that stuff working.
31:16
I think Conrad's pretty happy with how accessibility can work in some of
31:20
those areas,
31:21
but ultimately we started running into issues because writing a
31:25
declarative wrapper around D UI kit is tremendously complicated,
31:29
and it turns out even Apple is not perfect at doing it.
31:33
So if UI,
31:33
I would argue is what we wanted at that point,
31:35
at least from the code we wanted to write,
31:38
it wasn't just quite ready yet.
31:40
So we're revisiting some of that,
31:41
but I think we'll talk about that in a little bit.
31:43
SPEAKER_1: Yeah, it was a pretty legit journey though.
31:46
Yeah,
31:47
so we choose MVVM for both of our platforms with a slightly different answer
31:53
for what our UI framework is going to be.
31:55
We choose Compose on the Android side,
31:57
we choose Slice kit on the iOS side.
32:00
SPEAKER_3: One is an industry standard and one is an internal choice, let's say.
32:07
And then on top of that,
32:08
we build our new design system,
32:11
which is ideally going to solve for a lot of the feature delivery pain around
32:17
people constantly reinventing the wheel.
32:21
I think that at one point there were like 15 spinner controls in each client,
32:26
and there was a guy on Reddit that just continuously found new ones and posted
32:30
about them.
32:31
SPEAKER_3: I love that guy.
32:34
SPEAKER_3: But yes, so it wasn't particularly revolutionary except in small parts.
32:41
A lot of it had already been started and committed to,
32:44
but some of these projects had been going for three to six years already and
32:48
not finishing,
32:50
which had left us in a place of the worst of both worlds.
32:56
When I got here,
32:57
if you had an incident,
32:59
our client was half planted in the old rest world and half planted in the new
33:05
GraphQL world,
33:06
and you had to know which team and which project and which endpoint you were
33:11
talking about before you could solve for even where to look at the
33:14
observability for that element.
33:18
SPEAKER_3: And so you could vastly reduce and simplify for everyone.
33:24
Then a lot of things would get easy and our promise,
33:28
wait,
33:28
our promise for Core Stack was if you actually followed our simple
33:32
prescription platform teams would be here to build easy mode tools to
33:37
provide golden paths and to generally make it easy for our feature teams to
33:42
focus on building the best features instead of struggling with the tools and
33:46
the architecture.
33:49
So starting from the bottom,
33:51
in terms of the API,
33:53
you mentioned there was a transition from rest to GraphQL.
33:57
SPEAKER_0: Why did that happen and what did that mean for the mobile clients?
34:00
Especially that as we know for native mobile,
34:03
you can't just ship a new version and forget about the old versions.
34:07
You either need to have backwards support or you need to have some sort of kill switch,
34:11
which forces you update on WhatsApp is,
34:13
I think for people using it,
34:15
they will say saying,
34:16
oh,
34:17
this version of the app cannot be used.
34:19
SPEAKER_0: You need to download a new one.
34:20
SPEAKER_0: But until things like that are built, you can't really do that.
34:23
So what was the impact and how long did that take and how did you respond?
34:30
SPEAKER_1: Oh my gosh.
34:32
SPEAKER_1: It would be funny if I actually did the math on how long it took.
34:35
I think right now it's true that we're not in an active GraphQL migration,
34:39
but for most years of me being at Reddit,
34:41
there's been some kind of that migration,
34:46
we were tackling it early on,
34:49
one of the shout outs to Aiden who was in charge of our original iOS GraphQL
34:54
implementation is like that system is actually still in the code base today.
34:57
SPEAKER_1: So we got five years out of that thing at least.
35:01
But it provided a lot of just benefits in terms of contracts and knowing what
35:07
the mobile app is actually doing in which client is even doing a request.
35:12
One of the happy accidents I actually like is that because we have to name our
35:15
GraphQL requests in a specific way,
35:18
there are a couple of incidents that got solved in 15 minutes because we were like,
35:21
oh wait,
35:22
well,
35:22
that call is only deployed on Android,
35:24
so this is an Android specific issue.
35:26
SPEAKER_1: So the max thing is here, so iOS on-call people, you can take off.
35:31
SPEAKER_1: We don't have to worry about this.
35:32
But it was a lot,
35:37
but a lot of cool benefits,
35:38
and we got to build sort of our own Apollo internally that finally proved to me
35:44
that we definitely should just use Apollo instead of trying to roll our own.
35:46
SPEAKER_1: It turns out it's a complicated thing to build.
35:50
SPEAKER_0: And then just to hammer home the difference between rest and GraphQL.
35:54
SPEAKER_0: REST is it's APIs and then JSO, right?
35:58
And then you need to do all your validation with GraphQL,
36:01
you get more type safety,
36:03
do you?
36:04
SPEAKER_1: Yeah.
36:05
SPEAKER_1: Oh definitely.
36:05
SPEAKER_1: So the nice thing is you can go and read the GraphQL spec.
36:08
SPEAKER_1: It is actually somewhere on the internet.
36:10
SPEAKER_1: I actually had to read that a couple of weeks ago.
36:12
SPEAKER_1: Yeah, there's an explicit contract.
36:13
You're going to get types,
36:15
there's going to be a schema that enforces those types.
36:18
And if you think about a older website like Reddit,
36:22
you can still probably find some of our old school API documentation.
36:26
A lot of those endpoints are older than some of the people who listen to this podcast.
36:32
Like Reddit is 20 some years old,
36:34
and Eric alluded to this before,
36:36
there were a bunch of reasonable decisions that were made when we shipped
36:40
those after 10 years.
36:42
SPEAKER_1: Some of those contracts have to be reworked.
36:44
So GraphQL was a great opportunity to apply a layer on top of our old services
36:50
that established a clear contract and allowed people to actually
36:53
understand what clients want and what the backend needs to surface.
36:58
SPEAKER_1: I would never go back.
36:59
SPEAKER_1: Go ahead Eric.
37:00
SPEAKER_2: I'll say though, I think we took a little bit of a naive approach.
37:04
SPEAKER_2: We definitely have some learnings out of the migration.
37:07
Really what we did,
37:08
at least to start out was we took our rest API and did a one-to-one migration to GraphQL,
37:14
which is not how GraphQL is supposed to work.
37:16
We take these very flat models and we pull them over to GraphQL and then we
37:24
don't get that type safety really.
37:25
SPEAKER_2: We get some knowability safety, but that's about it.
37:28
So we still ended up with just massive amounts of client side logic that
37:32
didn't belong there.
37:34
And we still have a lot of these issues today around too much logic in the app or
37:39
things having to be redesigned in weird ways or even just over fetching.
37:44
A lot of times we fetch massive amounts of data that we never even touch just
37:49
because it was in the rest API.
37:51
SPEAKER_2: So we probably need it.
37:54
So we made some mistakes,
37:56
we have some learnings,
37:57
and it's the kind of thing we're still improving every day.
37:59
And then for this API migration,
38:02
the a i,
38:03
the backend a P change from rest.
38:04
So GraphQL,
38:05
how did it go in terms of,
38:06
was it the mobile teams telling the backend teams,
38:09
Hey,
38:09
this is what we need,
38:10
this is what the app uses,
38:12
and obviously there's a web team as well,
38:13
or was it more the other way around where the backend team is like,
38:15
okay,
38:16
we're exposing these things,
38:17
here's what you're getting.
38:18
Because it's always a question of who's leading in terms of you have the teams
38:23
who are closer to the users,
38:25
but there's also the backend teams which have a lot of considerations for
38:28
performance functionality,
38:30
maintainability,
38:30
that kind of stuff.
38:32
That's one of my favorite things about GraphQL actually,
38:35
is as long as you have some amount of collaboration,
38:39
you can usually come up with something that both people are happy about or
38:42
both sides are happy about.
38:44
I guess you have a query and you can query only for the very specific fields you
38:50
need and you can kind of form them into the data models the way you like 'em.
38:55
SPEAKER_2: The direct answer to your question is it depends on what team you're on.
38:59
SPEAKER_2: Sometimes the backend developers are leading it.
39:01
Sometimes the client side are,
39:02
sometimes there's a lot of collaboration,
39:04
but usually both sides can be happy just due to the flexibility of GraphQL.
39:10
I think initially the migration,
39:13
the web team probably started it,
39:15
but I feel like they pretty immediately pulled in platform people,
39:20
at least I'm sure Aiden was in there immediately as soon as Alex had the idea.
39:23
So I think it started from there,
39:26
but it pretty quickly we were in the room being able to ask questions and
39:29
provide feedback.
39:30
SPEAKER_1: For sure.
39:30
SPEAKER_1: Go ahead, Laurie.
39:32
SPEAKER_3: I was actually going to take a different answer on this one.
39:34
My entire experience at Reddit has been around a backend GraphQL team having
39:40
been established and really helping drive the migration away from rest and
39:44
our old infrastructure to a GraphQL and then a federated GraphQL model that
39:50
they have been evangelizing and they've been excellent partners to the
39:54
client teams,
39:55
but they have definitely driven a lot of that adoption and helped us build our
40:02
actual GraphQL experience as client engineers.
40:08
And some of that is because when we go out into the wider talent pool,
40:13
most people come in the door at Reddit not knowing GraphQL upfront.
40:16
That has changed over time,
40:18
but it was a lot more unusual in the time that we were growing.
40:22
Most people came in with a rest background,
40:24
they had maybe even written a Reddit app themselves before they joined the company.
40:30
And so I think that our journey here involves us learning how to write idiomatic,
40:36
GraphQL over time,
40:39
learning what happens when you don't.
40:43
And one of the other things I think that had an impact on this particular
40:48
project is our initial GraphQL implementation was slower than our old REST infrastructure.
40:56
It had a latency tax at the,
40:59
we got the flexibility and all the good parts we wanted,
41:02
but we had to take it with a latency tax upfront for a while.
41:08
And so when we transitioned the main Reddit clients to GraphQL,
41:14
they were noticeably slower on the same features.
41:16
We had been running on our rest infrastructure before,
41:18
and we had to decide if that was a trade-off we wanted to lean into and commit to
41:23
and work through or if we're going to rethink our approach.
41:27
Ultimately,
41:27
we took longer to actually move over to our GraphQL infrastructure because
41:31
we worked with the GraphQL backend team to reduce that latency and basically
41:37
make it much more performant.
41:40
SPEAKER_3: And so by the time our clients moved over, it was in a much better place.
41:44
But there was a point in time where it was frustrating for future teams to be
41:48
migrating in the direction of with a latency hit,
41:52
especially when we were really sensitive to our app being slower for users at
41:57
the time.
41:59
SPEAKER_0: Yeah, this is that migration.
42:01
SPEAKER_0: There's a comic about migrations at Google Man, CoreNet who drew it.
42:08
SPEAKER_0: It is like there are two paths to systems.
42:10
There's a path that doesn't work,
42:13
and the one that's being deprecated and it's the usual store and he's got a
42:16
request to whenever he went to translate it.
42:19
SPEAKER_0: Usually what happens with migrations, right?
42:21
SPEAKER_3: Yep.
42:22
But I asked both Brandon and Eric last week if they feel like Graph was the
42:27
right solution for our size at the time,
42:30
and what did you say?
42:32
I don't know if it was the right size at the time,
42:33
but I feel like this is such a great example of A,
42:36
one of my favorite things about Reddit and B,
42:38
why you should just hire tech experts and let them cook is like,
42:41
yes,
42:42
it was slower at the initial time,
42:44
but now because we have that,
42:46
we can do a bunch of code gen,
42:48
we can try to solve a bunch of different tech problems and there's a bunch of
42:51
optimizations that we can do because we have one thing,
42:55
we only have one kind of networking layer to maintain.
42:59
SPEAKER_1: Well, that's probably not true.
43:00
We probably have too many still,
43:02
but at least we can optimize the GraphQL one.
43:04
But because we took that or made that bet early and let people figure out how to
43:09
make it more performant,
43:11
I'm pretty happy with it.
43:12
SPEAKER_1: I'd do it again.
43:13
SPEAKER_4: Yeah.
43:14
SPEAKER_0: Nice.
43:15
SPEAKER_0: And then how did the architecture choices play out?
43:18
So you mentioned that on iOS you decided to go with an in-house UI framework at first.
43:24
SPEAKER_0: How did that work out and where are you right now?
43:29
SPEAKER_1: As ever?
43:30
SPEAKER_1: We are in a period of transition.
43:31
So I would still say that well,
43:34
actually no,
43:35
I would fight somebody about it.
43:36
It was absolutely worth it for one very big reason,
43:39
like a big chunk of the,
43:41
well,
43:42
actually,
43:42
let's talk about a few things.
43:44
One,
43:45
the feed code in our iOS app is a very,
43:48
very central component and it spreads its design choices,
43:53
Ithinkpretty
43:55
SPEAKER_0: Impressive.
43:55
So the feed code,
43:56
the feed is when you open it,
43:57
you see your feed or feed of different Reddits,
44:00
right?
44:00
SPEAKER_0: Or different subreddits.
44:01
When you think about either just popping open the app and seeing what we would
44:05
call your home feed,
44:05
like the automatically populated list of posts that we think you're really
44:10
going to like,
44:11
that infrastructure is actually shared between,
44:14
well,
44:14
a lot of it is shared between actually the subreddit feed.
44:16
If you look at a specific community,
44:18
and historically there was a pretty sort of nasty cluster of type
44:25
hierarchies in profiles as well.
44:28
Basically anything where you would show a vertical scrolling list of posts
44:31
is probably the same infrastructure before this.
44:34
SPEAKER_1: That was based in texture.
44:36
And if you're listening to this,
44:37
you probably don't remember that framework or what it was,
44:40
but it was something Facebook started and then Pinterest shepherded for a
44:45
long time that allowed you to do asynchronous and multi-threaded ui.
44:48
SPEAKER_1: So that created a tremendous number of complexity.
44:52
If you were remembering Lori's examples of somebody who comes in who doesn't
44:56
know a particular kind of pattern at Reddit,
45:00
nobody who we onboarded new texture.
45:02
SPEAKER_1: So we wanted to get out of that.
45:05
The first commitment to Slice kit was a massive win in that we just our backend
45:08
Apple's nice main thread UI land.
45:11
However,
45:12
we ran into a lot of issues where it's really hard to reuse the components.
45:17
SPEAKER_1: There's some boilerplate that we just can't quite crack.
45:20
So what we've been working on for a while and how we've been going about this is
45:27
I really strongly believe that if you're building infrastructure,
45:30
the new thing should at least be able to be hosted in the old thing.
45:34
So what we're doing is we're saying,
45:35
Hey,
45:36
if you're using Slice Kit,
45:37
we're going to figure out ways that you can put individual Swift UI based
45:40
cells in there and maybe you can easily AB test on those individual cells.
45:46
SPEAKER_1: So you can figure out is it performance, is it accessible?
45:49
SPEAKER_1: Does it meet all of your needs?
45:51
SPEAKER_1: And you can kind of roll out at your own pace.
45:53
And then we're essentially adapting that principle so that if you imagine
45:58
our vertical breakdown of course Stack,
46:00
we are replacing eventually Slice kit with Swift UI as we prove out areas of
46:05
this thing and make sure that,
46:08
for example,
46:08
swift DI list is a little bit perf suspect right now because it's really complicated.
46:13
SPEAKER_1: But essentially we're replacing it on a many year timescale.
46:17
But we're doing that by making sure that the teams who are currently
46:20
leveraging it have support for everything that they shipped in prod today,
46:24
SPEAKER_4: Today.
46:24
SPEAKER_4: Smart.
46:25
I think one of the interesting things about this is that when we gave our core
46:29
Stack prescription,
46:32
SPEAKER_3: A lot of us intended for it to be a starting place that we could make.
46:38
Once you separate the concerns out a bit and you have an established pattern,
46:42
we can build a roadmap away from that pattern if we decide that there's
46:45
something better out there.
46:47
But a lot of people kind of took it as,
46:49
oh,
46:49
when we have introduced all the new stuff,
46:51
we'll be good forever and we will never have to change our minds.
46:55
But a lot of the people who are trying to evangelize and build out this idea,
47:01
we're trying to plan for change for the long term because we knew that
47:05
anything that we chose in the past,
47:07
we chose with a reason.
47:09
We had reached a point where those particular choices were not holding up
47:13
over time and we needed to change them,
47:15
but we understood that we probably would be having the same conversations
47:18
ideally three to five years later about whatever we were doing now.
47:24
SPEAKER_3: So Slice, kit and compose were bets on both sides.
47:29
They had different risk profiles associated with 'em,
47:36
but we wanted to reserve the right to not have a one-way door there where we
47:40
couldn't change our minds later.
47:42
And even the Slice kit Choice had an intentionally revisit this choice on a
47:47
regular basis for the proper time to consider Swift ui.
47:54
SPEAKER_0: And how did it work on an Android?
47:56
You took a bet on compose and just like Swift UI compose back in 2021 was pretty
48:01
new right now.
48:03
SPEAKER_0: It's pretty accepted.
48:05
SPEAKER_0: It's a great way to build things.
48:06
In fact,
48:07
I'm not even sure there's really a competition of how to structure a moderately,
48:11
but back then this was not the case.
48:12
It was like,
48:12
here's a new framework,
48:16
we think it'll be good,
48:17
but you don't know.
48:19
So I want Eric to answer this question,
48:22
but one thing I want to to preface it with is when things are quite bad,
48:29
you have a lot more opportunity to just take bigger bets and get away with them
48:34
if they pay off.
48:36
SPEAKER_3: So Eric, why did we take the bet on compose as early as we did, given how
48:40
SPEAKER_2: Big?
48:40
SPEAKER_2: Yeah.
48:41
SPEAKER_2: I don't know how proud of this we should be.
48:43
Probably not super proud,
48:45
but when we started working on our first compose feature,
48:49
it was still an alpha we shipped while it was still in beta.
48:52
SPEAKER_0: Wow.
48:53
SPEAKER_2: Yeah.
48:54
The reason we were able to do this,
48:55
like Laurie said,
48:56
was that we knew we needed change,
48:59
and this looked like the next big thing.
49:01
At the time,
49:02
I was still on a feature team and I was working on a brand new feature,
49:05
like completely Greenfield,
49:08
perfect fit for trying something brand new.
49:10
SPEAKER_2: So I took a bet, I built the whole thing in Compose.
49:15
I used MVVM instead of MVP,
49:17
kind of introduced a whole lot of new patterns that seemed like the way to go.
49:22
And the reason that I was comfortable doing this was because I was also
49:26
willing to go back and rewrite it.
49:28
SPEAKER_2: If the bet didn't pay out, I was the sole developer on this feature.
49:33
SPEAKER_2: I completely owned it.
49:34
If it didn't work out,
49:36
if it was a terrible decision,
49:37
I was willing to own that and make it right.
49:40
SPEAKER_0: And just to confirm, because I think what people might miss is Reddit.
49:44
SPEAKER_0: It's a massive app.
49:46
SPEAKER_0: Large amount of users are using it.
49:48
This is not just your startup with a hundred users and this feature that you did,
49:53
was it,
49:53
it was a nice to have feature or not something critical or it was critical,
49:58
but you were just like,
49:58
okay,
49:59
I'm going to make it work.
50:00
It was,
50:01
I wouldn't call it critical,
50:03
so it's now gone,
50:05
but it was called Reddit talk.
50:06
I don't know how many people are familiar with that,
50:08
but it was the live chat room,
50:11
voicechatfeature
50:13
SPEAKER_0: Clubhouse taking off, right?
50:15
SPEAKER_2: Yes, exactly.
50:16
SPEAKER_2: Yes.
50:16
SPEAKER_2: Everyone was copying that for a while.
50:19
SPEAKER_2: So yeah, it was just kind of the perfect opportunity.
50:21
SPEAKER_2: So it was launched to a very small audience.
50:26
I think it was only two subreddits had the feature at first,
50:29
and then we were slowly scaling it as we proved it out.
50:33
SPEAKER_2: So it wasn't a super risky bet, but there was some risk there.
50:38
SPEAKER_0: Yeah.
50:39
SPEAKER_0: Well, reputation, right?
50:40
SPEAKER_0: Still,
50:41
It still only took less than a month to adapt talk as it first came about into
50:48
the golden example of what we ultimately decided was our very fairly
50:54
straightforward of tech Stack choices.
50:57
SPEAKER_3: And so it was easy to adapt.
51:00
So it basically proved out the Greenfield work as a starting point for our new
51:05
prescription was going to work just fine adapting some of our more legacy code,
51:13
our larger critical path features came with.
51:17
SPEAKER_3: They needed more than even our core Stack prescription was going to give.
51:21
SPEAKER_3: So they extended it in all sorts of interesting ways.
51:25
Now,
51:25
one thing that you mentioned that you're doing,
51:27
which is very interesting here,
51:28
server driven ui,
51:30
and this is interesting because we talked about this with Laurie,
51:33
but a lot of companies of a certain size,
51:35
especially when there's native mobile,
51:37
they will come up with something that might feel familiar to let's say either
51:42
React native or just something else with the goal of being able to drive some
51:47
of the UI backend changes.
51:50
SPEAKER_0: What was your approach here?
51:51
SPEAKER_0: Why did you do it?
51:52
SPEAKER_0: How did you build it and how do you feel about it?
51:54
So while it's not an official part of our core stack,
51:58
we definitely,
51:59
part of the core stack idea is that people can extend or experiment with
52:03
anything they think is going to work well on their surfaces because they're
52:06
not all the same.
52:07
We have massive feed surfaces and we have chat surfaces and we have profiles,
52:13
and they all really need different levels of complexity and capabilities.
52:19
SPEAKER_3: So we definitely had some teams try out server driven ui.
52:23
SPEAKER_3: I think Brandon, you might have a good example where it worked out well.
52:27
SPEAKER_1: Yeah, I think for a lot of those, tremendously complicated.
52:30
But one of the things that I think has been shipped for a real long time is our
52:33
reporter flow has our content reporting.
52:35
If you find something that you don't think should be on Reddit and you want to
52:38
report it,
52:40
I would say that has some server driven UI in it.
52:42
But since you're driving reporting reasons where you're really just trying
52:45
to ship a list of strings,
52:47
I think they chose a reasonable level of complexity where it makes sense for
52:52
that team.
52:53
SPEAKER_1: So it works really well, but other cases not so much.
52:57
SPEAKER_1: Eric, did you have an example of the nasty
52:59
SPEAKER_2: One?
52:59
SPEAKER_2: Yeah.
52:59
So the first place we tried out server driven UI was our biggest,
53:03
most important service,
53:04
which is maybe not where you want to start with experimentation generally.
53:09
SPEAKER_2: On the surface though, it made a lot of sense.
53:11
We had tons of logic in each client that went into determining how to display
53:16
posts within a feed.
53:18
Overly smart clients aren't a good thing,
53:21
and the logic didn't even match between clients.
53:23
There were times where you hold an Android and an iPhone next to each other and
53:26
the feeds would look different,
53:27
not intentionally,
53:28
just because we got the logic wrong on one or the other,
53:31
and nobody knew which one was right.
53:35
But despite all the good reasons that went into this,
53:39
it was a decision that we've ended up regretting to an extent.
53:43
And there were currently working to walk back from,
53:46
it turns out that even if the feed itself doesn't need the full post models to
53:51
be able to display correctly,
53:53
everything it links to still does.
53:55
So if we make the feed server driven UI and it no longer has any knowledge of the
54:01
backing models,
54:03
we just end up having to do double fetching.
54:05
We have to fetch the server driven UI definitions,
54:08
then we have to fetch the actual models.
54:09
And this just introduces double the opportunity for errors,
54:14
and it has led to lots of bugs that users see in production.
54:18
I don't know if you've ever used the app,
54:20
you've tried to tap on a post and nothing happens.
54:23
SPEAKER_2: That's because one or the other of the fetches failed.
54:27
SPEAKER_2: So we're working on it.
54:30
It's interesting because server driven UI is something that everyone
54:34
eventually goes to because of the native mobile apps.
54:37
Because the nature of mobile is if you only have client side logic,
54:41
you need to ship that code,
54:43
maybe put it behind a feature flag,
54:45
but it just takes,
54:46
if you want to add new business logic,
54:47
it will take you at least a week or two weeks or however long it takes.
54:51
So eventually everyone comes to the realization,
54:53
what if we just had a webpage wrapped into an app and that doesn't work,
54:58
but the next spec thing is back in ui.
55:01
What if we just have the business logic as A-J-S-O-N or something that we
55:05
download from the website and it should work?
55:07
But I've seen so many times where,
55:11
again,
55:11
most companies don't talk about this publicly,
55:13
but it becomes sour.
55:15
It just doesn't really work out because I think eventually what I've seen
55:19
realize is we need to be backwards compatible,
55:21
we need versioning,
55:23
we need to support older clients.
55:25
We're now running into,
55:26
oh,
55:26
we want this feature that we didn't think about back then and now we could add it,
55:29
but our older version doesn't have it.
55:32
What is your take on why is this not taking off in mobile land in general,
55:36
right?
55:36
SPEAKER_0: It seems there's just you come around and learn a lot of these things.
55:42
SPEAKER_3: I actually think that we shouldn't give up on the idea.
55:46
I think it's like any idea that we have had and failed at or wt,
55:56
you can say agile and it means different things to everybody.
56:01
You can say modularization and you can name just as many times that has failed
56:07
to deliver on its promises as it has worked.
56:12
SPEAKER_3: And so I don't actually think that there's a real problem with the idea.
56:16
I think there's a problem with an implementation that works for the shape of
56:22
the problem and we don't spend enough time making sure it is in this case,
56:27
I don't think we took enough time to make sure it was going to solve the Reddit
56:30
shaped problem as opposed to server driven UI makes sense,
56:35
but don't actually apply it to the actual reality of what our code base and
56:40
infrastructure looks like.
56:42
SPEAKER_3: There were parts of our infrastructure that were not ready for it.
56:45
We did not have an end-to-end commitment from everyone to work on the exact
56:50
same plan at the exact same time in order to support it.
56:55
But I've also seen that happen with trunk based development and other things
56:59
where it was like the fifth time you try,
57:01
you get it right and then everyone is really happy with it.
57:07
So I don't think it's necessarily the prescription that is the entire problem.
57:11
I think it's a underdeveloped implementation and plan in place to make sure
57:18
it can work in the cases that you want it to work.
57:21
SPEAKER_1: I want to double click on what Lori is saying with a specific example.
57:25
So just in case any product manager is listening who wants server driven ui,
57:29
the reason that managing that complexity is so important is that we know
57:33
server driven UI is possible and it can be done really well because web
57:38
browsers exist.
57:40
But that's the upper limit of this complexity and I don't have enough time to
57:43
build Chrome really,
57:44
really well.
57:45
So if you want server driven UI and you want the full feature that you're
57:49
specifying in a web browser,
57:50
well all you have to do is spend several billion.
57:53
If we control complexity well,
57:55
however,
57:56
we can definitely solve this problem.
57:58
SPEAKER_1: Just please no web browsers.
58:00
But you also see that WebView implementations in mobile are often not great
58:05
experiences for mobile developers.
58:07
SPEAKER_3: So
58:08
There are ways in which we've tried to bridge that at different times and
58:12
sometimes it really works and sometimes it does not work.
58:16
But I don't think we should stop trying on these ideas because I think we all
58:23
agree that our clients being simpler,
58:26
having less logic in them and being more nimble is like those are all
58:30
capabilities we want.
58:31
SPEAKER_3: We just haven't figured out how to deliver them well.
58:34
And in the meantime,
58:35
we have weekly,
58:36
sometimes more than once a week releases in order to keep the ability to
58:40
change production.
58:44
Nowhere near what our web platform can do in their continuous delivery pattern,
58:48
but it's pretty much the fastest you can deploy on mobile.
58:51
SPEAKER_1: Oh yeah.
58:52
There's also,
58:53
I feel like for Jetpack,
58:55
or excuse me for compose and for Swift UI because the atoms are different,
58:59
we might be in a case where it's easier to try some of the things using those modern,
59:04
excuse me,
59:05
modern frame works that are closer to DSLs.
59:08
Because if you think about a server driven UI thing,
59:10
it's probably really hard to do that.
59:12
SPEAKER_1: And UI kit, UI kit is not declarative.
59:14
SPEAKER_1: C qi is declarative.
59:15
SPEAKER_1: If you're shipping a declared spec, it probably will change a lot.
59:20
And so you've done a bunch of work over the last three or four years starting
59:24
from this modernization,
59:25
you move to compose on Android.
59:28
SPEAKER_0: Well, moving to Swift UI on iOS, added in the modern repo.
59:34
We didn't talk about it,
59:35
but you added code ownership and you usually did bunch of things that made
59:41
things better.
59:42
SPEAKER_0: In the end of course you onboarded so many new mobile engineers as well.
59:47
SPEAKER_0: Was this effort worth it?
59:48
How can you tell how this big modernization,
59:52
the core stack effort played off?
59:55
I think we have really good answers to this On the devex side,
59:59
I think we have really great answers for this.
1:00:01
On the user side,
1:00:04
I know that our sentiment has improved from our developer sentiment and we
1:00:11
are serving so many more developers than we were before and pretty much every
1:00:19
stability and performance metric has benefited from the changes that we've made.
1:00:25
SPEAKER_3: But who wants to go first?
1:00:28
SPEAKER_3: Android or iOS?
1:00:31
SPEAKER_2: Sure, I'll go.
1:00:33
Yeah,
1:00:33
I mean,
1:00:33
like Laurie said,
1:00:34
our developer surveys have shown that people are just far happier than they
1:00:37
used to be.
1:00:38
Those developer surveys were rough for a while and now they're,
1:00:42
overall they're pretty positive,
1:00:44
but a lot of things have gotten easier.
1:00:46
SPEAKER_2: Onboarding is easier.
1:00:46
SPEAKER_2: We don't have to coach people into the myriad of patterns that we have.
1:00:53
When I used to interview people,
1:00:54
they were all excited,
1:00:55
yeah,
1:00:56
Reddit sounds so cool.
1:00:57
SPEAKER_2: I really want to work here.
1:00:58
SPEAKER_2: What's your tech stack look like?
1:01:00
And I would tell 'em,
1:01:00
and I could just see their face like,
1:01:02
ah,
1:01:02
that sounds terrible.
1:01:04
SPEAKER_2: And that's not the case anymore.
1:01:05
SPEAKER_2: People are excited.
1:01:06
People want to work in this kind of code base,
1:01:09
so we've seen a lot of really good stuff come out of it.
1:01:12
Even the types of engineers that we can hire,
1:01:15
we can hire more junior engineers on the feature teams because you don't need
1:01:20
to know all this legacy knowledge that there's no reason to know anymore for
1:01:26
the most part.
1:01:27
And yeah,
1:01:29
there's obviously all the runtime user facing benefits as well,
1:01:32
but I don't know.
1:01:33
SPEAKER_2: I'm a developer.
1:01:33
SPEAKER_2: I care about developer experience.
1:01:35
SPEAKER_3: Brendan where the texture crashes.
1:01:39
I think iOS is a little lagging behind in that,
1:01:42
but I think well emphatically it was worth it if we wouldn't have done that.
1:01:49
I can't imagine how the heck anybody would be productive with the
1:01:53
expectations that we would've had,
1:01:55
but emphatically worth it.
1:01:57
And I think even in cases where I feel like the tech needs to get better,
1:02:02
it allows us to have a sort of specific path and a specific focus for people
1:02:07
working on this.
1:02:08
SPEAKER_1: I love that we now have iOS people who are the defacto GraphQL experts.
1:02:13
SPEAKER_1: That was not always the case.
1:02:15
And now as they iterate on that tooling and on those technologies,
1:02:20
everybody gets benefits so we can progress in that area independently while
1:02:25
I'm trying to figure out how we're going to get Swift UI into this thing.
1:02:29
SPEAKER_1: And structured concurrency also introduces a lot of wrinkles on this.
1:02:32
We have paths to activate these projects and I would much rather be in this
1:02:37
universe than the alternative.
1:02:40
SPEAKER_3: So I can give you some concrete examples of this.
1:02:44
We used to see Slack messages in our shared guild channels and you would have
1:02:48
to figure out what somebody was doing at a really detailed level because it
1:02:53
could be anything before you could help them.
1:02:55
And so it was actually really hard to mentor people,
1:02:57
especially in a remote first company.
1:03:00
You had to get a whole bunch of context before you could answer a question
1:03:03
about why their build was broken or what they were trying to achieve.
1:03:07
We have a much higher level of shared understanding when the question gets
1:03:11
asked at this point because most people are following golden ish paths most
1:03:15
of the time.
1:03:16
So there's a lot less frustration,
1:03:18
a lot less burnout on the people who are both the subject matter experts and
1:03:24
the people who are trying to become 'em.
1:03:28
We did a whole bunch of analysis of our devex improvements to try and prove out
1:03:34
not just by surveys,
1:03:36
but with some sort of quantitative analysis if they were worth doing.
1:03:41
I really wanted to be able to take the product approach of could we defend this
1:03:46
sort of effort?
1:03:47
We would defend a product initiative,
1:03:53
and we definitely saw that breaking up the monoliths and modularizing and
1:03:58
giving people stronger ownership of their code areas improved a ton of
1:04:03
productivity signals.
1:04:07
People were building new features like Reddit recap with half the amount of
1:04:11
people they did the previous year,
1:04:15
they were able to finish their features sooner so they would add these extra
1:04:20
bells and whistles to their actual deliveries,
1:04:23
which had unintended consequences.
1:04:26
They would introduce more animations and we'd be like,
1:04:31
where did those come from?
1:04:32
SPEAKER_3: Oh no, our performance.
1:04:36
But no,
1:04:36
all of those had real business benefits from a developer efficiency and
1:04:40
productivity perspective while increasing developer joy at the same time,
1:04:45
which was the kind of developer productivity we were after as platform team.
1:04:49
The feature from a user side,
1:04:52
pretty much every one of our metrics that was bugging us at the beginning of
1:04:56
this effort is in a much better and stable place,
1:04:59
and it very rarely sees regressions.
1:05:02
Our crashy rates benefited from going to Swift and Kotlin just from the null
1:05:09
ability perspective alone.
1:05:11
We see very few npe problems in production,
1:05:14
hot fixes associated with obvious avoidable problems like that.
1:05:20
I think we were up 1.5 percentage points on crash street rate,
1:05:25
which is a ridiculously great improvement on Android.
1:05:30
SPEAKER_3: iOS was always a little bit more stable.
1:05:31
They don't have quite as many devices in the wild,
1:05:36
but they also saw I think almost a 1% improvement over that sort of time period.
1:05:42
SPEAKER_1: I never remember the percentages.
1:05:43
SPEAKER_1: I only remember when the problem completely goes away.
1:05:48
So we are at a point of diminishing returns on investing in that space and we
1:05:52
can focus on,
1:05:55
we then focused on startup times.
1:05:56
Those are down under three to four seconds even in the P 90 plus sort of range.
1:06:03
And they've stayed there for years and people have noticed,
1:06:07
which allows us to basically move on to another problem we want to solve like
1:06:11
scroll performance or video performance and working with the video team and
1:06:15
stuff like that.
1:06:16
So by making these improvements and then putting policies into place to keep
1:06:23
them relatively stable in those places without big regressions allows
1:06:27
everyone to go redirect their time to what is the most pressing thing that is
1:06:33
in front of us either as a company or bothering our users the most and try and
1:06:37
focus on improving that space.
1:06:40
And we can do that in part because everyone is kind of a little bit more
1:06:46
similarly shaped,
1:06:48
which allows us to make a huge amount of assumptions.
1:06:51
We can assume that we can build a tool that will work for most of them most of the time,
1:06:55
which we now do.
1:06:56
We build a lot more tools and scripts and things to make things easy and we can
1:07:01
build observability such that when we actually go looking at what the
1:07:04
problem space looks like,
1:07:06
we can actually build a map of what it looks like as opposed to it looks like a
1:07:10
very strange world map with really strange boundary lines where you can't
1:07:14
actually see the whole problem because everyone is doing something different.
1:07:17
I want to double click on one of the things Laurie said in case there are any
1:07:20
perspective modernization enthusiasts in the audience.
1:07:26
We're not doing this because we think we have a better solution than the
1:07:30
feature teams.
1:07:32
I feel like there's a bunch of boxes at Reddit where I don't want you to have to
1:07:36
think about how to interact with our GraphQL service.
1:07:39
I just want you to think about your feature and I want you to be able to be
1:07:42
creative in the ways that she was describing.
1:07:44
I love the idea that you're adding animations and you're actually bringing
1:07:48
your creativity as an engineer to the features you're building.
1:07:52
SPEAKER_1: The reason we're doing this is not to take flexibility away from people.
1:07:59
It's to eliminate a bunch of boring things so that you can do the actual
1:08:02
important work.
1:08:03
If you're trying to do modernization,
1:08:05
do it in service to your stakeholders.
1:08:07
Do it in service to the teams that you support,
1:08:10
not because you think Swift is a cool language,
1:08:12
not because you think Swift UI is a cool thing because it has benefit to the
1:08:15
teams you support.
1:08:17
SPEAKER_1: That's a huge reason for us behind this.
1:08:19
SPEAKER_3: I agree with that.
1:08:20
I think that's one of the reasons why we focused on some of the places that
1:08:23
people do not gravitate to,
1:08:25
like dependency injection.
1:08:27
SPEAKER_3: Can we make that less painful for people?
1:08:29
Because nobody loves dependency injection with the possible exception of
1:08:33
Drew until he was in charge of it and now can never not be Anvil guy.
1:08:40
But we picked the places with pain points for our feature teams,
1:08:45
not the ones that we were trying to constrain them to Brendan's point,
1:08:52
And then you're in the enviable position of being a mobile platform team that
1:08:57
is actually iOS and Android,
1:08:59
so two pure native mobile platforms.
1:09:02
SPEAKER_0: It's quite rare in the industry.
1:09:03
And also working with a large native mobile engineering team,
1:09:06
which obviously you need a larger team for a platform team to make sense.
1:09:10
My question is what does it take for an iOS or Android engineer to work at a
1:09:17
platform team like this in terms of when you're hiring,
1:09:20
what are the traits that you're looking for?
1:09:22
Is it a lot of internal transfers I'm asking is because for a lot of mobile engineers,
1:09:27
this is a little bit of a dream to get their at some point if they're lucky
1:09:30
enough to have either their organization or to look for the select few
1:09:35
companies that do have this thing.
1:09:37
What are ways that mobile engineers who are doing either native or cross
1:09:41
platform can work towards skill sets or experience to increase their
1:09:45
chances to later join a team like this?
1:09:48
SPEAKER_3: I'll take this first.
1:09:51
So the people who have joined the teams in the last couple of years are a great
1:09:55
mix of internal transfers and going out into the hiring pool and hiring externally.
1:10:03
We often find people grow in their feature orgs to be a top performer and they
1:10:10
know everything about their scope in that space,
1:10:12
but they have frustrations that they would really like to get resolved by
1:10:17
platform solutions.
1:10:17
SPEAKER_3: They would also impact other teams.
1:10:19
They are great recruits and we have gotten some really great people come into
1:10:24
the platform teams who then just immediately turned around and helped solve
1:10:28
real problems at Reddit with us and helped us understand them and scale them
1:10:33
to everybody else.
1:10:34
So I can think of a couple people who've joined our teams that absolutely meet
1:10:38
that sort of pattern.
1:10:40
We've also gotten some really great talent out from other companies who've
1:10:44
brought in fresh ideas from other companies at scale as well as from startups
1:10:48
who are scrappy and Reddit is kind of messy and we often have zero to one
1:10:54
problems that it does not take at scale experience to solve.
1:10:57
It takes somebody looking at the problem and going,
1:10:59
I know exactly how to make this a hundred percent better and just going and
1:11:04
doing it.
1:11:05
So taking ownership over something and being practical and having first
1:11:10
principles is extremely important.
1:11:13
And then the last thing that I'm always looking for is,
1:11:17
so it's a little different between the platform teams here,
1:11:19
but I have a really robust rotation of devex.
1:11:23
Everyone on my team doesn't necessarily want to be devex all the time,
1:11:27
but they want to give back on the devex side at least part of their job,
1:11:33
at least 25% for everybody.
1:11:36
Some people want to completely specialize on the DevX side,
1:11:40
but a whole bunch of people actually really enjoy helping other engineers
1:11:45
work through their problems and such.
1:11:49
So I'm looking for that interest in giving back to other internal engineers
1:11:55
as well as thinking of end users at the same time.
1:11:59
But if they do join,
1:12:00
they get to work with awesome people like Brandon and Eric who probably have a
1:12:04
different perspective on what they're looking for on their platform teams.
1:12:09
SPEAKER_1: My joke answer is that I don't know why someone would choose to do this.
1:12:15
SPEAKER_1: I just sort of stumbled into it.
1:12:19
SPEAKER_1: It's very stressful and it's very hard.
1:12:22
SPEAKER_1: Having said that, I would not choose any other position I want to be.
1:12:26
SPEAKER_1: You'll drag me from this team kicking and screaming.
1:12:28
SPEAKER_1: I think Lori provided a good perspective.
1:12:31
I want to talk about what I think is the most important thing for an IC who is
1:12:38
trying to have a job on a platform team.
1:12:41
SPEAKER_1: You need to sit in the consequences of your decisions.
1:12:46
You need to,
1:12:47
in my opinion,
1:12:49
you should try to work at a tech company for a year or two and actually see what
1:12:54
happens after you ship a system and then the assumptions change and you have
1:12:59
to figure out how to keep this thing going.
1:13:01
There are systems that we have that have no joke,
1:13:04
been in our code base for five years and the platform team has been evaluating
1:13:09
whether to replace them or how to keep them going.
1:13:12
That's incredibly hard and I think we have very talented software engineers
1:13:18
across our iOS org,
1:13:21
but I think the big difference between folks who are working in features and
1:13:26
working this,
1:13:27
it's more reasonable for a feature team to take their code and after the
1:13:31
product doesn't work,
1:13:32
they just kind of delete it and go on.
1:13:33
SPEAKER_1: Eric mentioned the hawk is not in our code base anymore.
1:13:36
SPEAKER_1: Most things that I've had to write at Reddit are still in the code base.
1:13:40
SPEAKER_1: I wish they weren't.
1:13:41
But you have to understand,
1:13:42
and I think what you get out of that is you get a bunch of software design
1:13:46
intuition because you have to reevaluate your assumptions for an
1:13:52
incredibly long time.
1:13:53
SPEAKER_1: If you can do that, you're probably ready for platform stuff.
1:13:56
SPEAKER_1: But take your time.
1:13:58
SPEAKER_1: I don't want to burn you out before you're ready.
1:14:01
SPEAKER_1: This is hard.
1:14:02
SPEAKER_3: I want to interject real quickly there.
1:14:05
One of the things that Brandon mentioned that I think is really important,
1:14:07
and it definitely comes up in all of our interviews for platform roles,
1:14:13
what we're not looking for is somebody that just wants to go introduce
1:14:16
platformworkthattheyarenotgoingtousethemselvesordogfood
1:14:22
SPEAKER_3: And assume they can solve something without understanding the ground.
1:14:27
And so that is one of the reasons why we like internal transfers or even
1:14:31
feature team to platform team transfers or growth simply because if you take
1:14:37
product thinking in here,
1:14:39
we're just looking at a different set of users.
1:14:41
And if you do not take the feedback cycles from your developers,
1:14:45
you will absolutely miss on what you try and deliver to them as a platform team.
1:14:49
SPEAKER_3: And that is a problem that we're always looking for.
1:14:51
SPEAKER_3: Are we open to that feedback?
1:14:52
SPEAKER_3: Are we actually solving the problem that is the real problem at Reddit?
1:14:55
SPEAKER_3: Eric, what about
1:14:57
SPEAKER_2: You?
1:14:57
SPEAKER_2: Yeah, huge plus to that.
1:14:59
I don't think platform work is a great first job,
1:15:02
although we have someone on our team who's doing great at as the first job,
1:15:07
but I don't think that there's really a blueprint for becoming a platform engineer.
1:15:13
SPEAKER_2: Some people are jack of all trades because you need that to some extent.
1:15:16
You have to know how the build system works,
1:15:18
how to make a screen more performant,
1:15:21
just how the compiler works.
1:15:23
SPEAKER_2: There's just so much knowledge that you need.
1:15:26
SPEAKER_2: And then others are just specialists.
1:15:28
We have build experts,
1:15:30
we have performance experts,
1:15:31
we have people who just go really deep in a certain area.
1:15:35
So I don't know if there's a certain thing to do to become a platform engineer
1:15:40
other than having experience across a lot of different things and just
1:15:45
having an interest in it.
1:15:46
SPEAKER_2: I guess
1:15:47
I do think that remember that you're talking to a couple of people at,
1:15:52
I would entertain the argument that when I joined Reddit,
1:15:54
it was probably not an air quote startup,
1:15:56
but it was a startup kind of environment.
1:15:58
But both Eric and I started in at Reddit when the teams were small enough that
1:16:04
it was easier to get to that impact.
1:16:06
So one of the things if you're trying to get onto a platform team,
1:16:10
it's probably easier to go to a startup and get a bunch of that experience than
1:16:13
it is to go to a scaled platform where because our bar of what we expect our
1:16:19
engineers to do has gotten way higher.
1:16:23
My job now as a staff compared to my job,
1:16:26
I think I was a staff four or so years ago,
1:16:28
completely different,
1:16:29
completely different expectations.
1:16:31
So I think being in a smaller org where you can get exposed to a lot of those
1:16:35
different things that Eric is talking about is probably an easier path.
1:16:40
SPEAKER_1: But you should still apply.
1:16:41
You might not get accepted this round,
1:16:43
but I'd still love to talk to you about how we do this.
1:16:47
The last thing there is that I want to push back on platforms being elite teams
1:16:52
a little bit.
1:16:53
SPEAKER_1: We're not, we're no smarter than anybody else,
1:16:56
Mostly from the perspective of I'm always worried we're going to introduce
1:17:03
something that's going to reduce the psychological safety of this group.
1:17:07
SPEAKER_3: They need to be able to break in order to make big improvements.
1:17:12
And so having the ability to say you don't know an answer to something and not
1:17:17
having any sort of problems with that and being able to just really grow
1:17:22
through blameless postmortem culture is extremely important to platform teams.
1:17:27
Otherwise you always play it safe and you usually just continue to introduce
1:17:31
friction to teams instead of actually solving their real problems to help
1:17:36
them reach their maximum potential.
1:17:38
As feature teams,
1:17:39
it's too easy to be like,
1:17:40
we'll just add another lint rule so they avoid that foot gun and suddenly you
1:17:43
have a gazillion of them and everything is slowing down because of all of your checks.
1:17:50
So I think there's also just the brilliant a-hole sort of platform archetype
1:17:59
that we specifically are not interested in.
1:18:02
We like the brilliant,
1:18:04
humble really wants to work with other people and help them solve their
1:18:08
problems too.
1:18:10
SPEAKER_1: It is definitionally like a position of service.
1:18:14
SPEAKER_1: My job is to help engineers be more efficient.
1:18:17
It's not to ivory tower designs and then pretend that I'm smarter than anyone else.
1:18:21
SPEAKER_1: I'm really not.
1:18:24
SPEAKER_0: This is nice to hear.
1:18:25
SPEAKER_0: And I think it's just a good reminder.
1:18:26
SPEAKER_0: As companies grow, there's more opportunities.
1:18:29
So if you're lucky enough to work at a company,
1:18:31
which is hard to predict,
1:18:32
but if there are trajectories and when companies live up to expectations,
1:18:37
they will grow.
1:18:38
SPEAKER_0: It's just a lot of pass open up.
1:18:39
May that be going to a platform team,
1:18:42
may that be changing stacks,
1:18:43
may that be going to a leadership position or transferring teams working on a
1:18:48
lot of them,
1:18:49
it's just a lot easier.
1:18:51
And on the other hand,
1:18:52
companies that either shrink or stay kind of flat,
1:18:57
everything will be a lot more challenging and difficult there.
1:19:00
I would say that the friend,
1:19:03
your platform team is the best way to become a platform engineer someday.
1:19:08
If you have a platform team befriend them during hackathon weeks or do some
1:19:13
mentorship side projects.
1:19:14
Almost everyone has done something like that before they end up joining our team.
1:19:19
But also we really like having really good partners on the feature teams
1:19:23
themselves because they are very honest with us about what their real
1:19:26
problems are and they're the best source of that.
1:19:29
So there are also people who just never want to join a platform team because
1:19:33
they're really user focused and that's the end user,
1:19:37
the user they really,
1:19:37
really want to focus on with their career.
1:19:41
And they're also some of our very best partners because they tell us what
1:19:44
they're trying to achieve and we try and help them figure out how to do it.
1:19:50
They get early access to tech that's not ready yet if we have to find people who
1:19:56
want to use Swift ui so we can deploy it at scale,
1:19:58
so we can verify all of these issues.
1:20:01
If you're getting buddy buddy with us,
1:20:02
you're a hundred percent going to be the first person in my head of like,
1:20:05
oh,
1:20:05
well search is a great partner for us on the iOS platform of like,
1:20:08
yeah,
1:20:08
we'll get feedback from them and see what Alex thinks of this.
1:20:11
SPEAKER_1: And yeah,
1:20:13
Meanwhile ads is a really great litmus test because they have to monetize
1:20:16
every feature,
1:20:17
whether it's in legacy or modern,
1:20:19
and therefore they're very sensitive to slow migrations.
1:20:21
So yeah,
1:20:23
we go to different teams for different reasons and there's lots of ways to be
1:20:27
part of the platform answer even if you're not on a platform team or if you
1:20:30
choose to specialize in it.
1:20:33
SPEAKER_0: Nice.
1:20:34
So as closing,
1:20:35
let's just do some rapid questions where I'll shoot the question and then you
1:20:39
tell me what pops up.
1:20:41
SPEAKER_0: So what's the framework that you really like for usability or elegance?
1:20:47
SPEAKER_1: If we're choosing usability and elegance?
1:20:49
I have to pick Swift ui,
1:20:50
but I was thinking about this question and I think my answer is actually lib
1:20:54
GraphQL js.
1:20:54
That's the last framework that I interacted with that taught me a lot,
1:21:00
would recommend,
1:21:00
should read.
1:21:01
SPEAKER_2: I think for me it's composed.
1:21:05
SPEAKER_2: I love how maintainable in testable it is.
1:21:07
SPEAKER_2: It's just so much less boilerplate than anything I've used before.
1:21:10
Just really solid unidirectional design events work their way up,
1:21:14
state works its way down and all these principles are appealing for far
1:21:20
beyond just the UI layer.
1:21:22
SPEAKER_2: I know when you have a hammer, everything starts to look like a nail.
1:21:25
But if I'm being honest,
1:21:26
a lot of things have been looking like very appealing nails lately.
1:21:30
And at Reddit,
1:21:31
we do use it for both our UI and our presentations layer,
1:21:34
our presentation layer,
1:21:35
and it's been really great for us.
1:21:39
I do also compose,
1:21:40
the more I use it,
1:21:42
I was out of the actual IC world for a while while,
1:21:45
but I have side projects that I've been really enjoying it.
1:21:48
SPEAKER_3: Again, I picked a framework that is not a technology framework.
1:21:54
SPEAKER_3: I'm often thinking about the western typology of organizational culture.
1:22:00
SPEAKER_3: It's in the Accelerate book and I think about it a lot.
1:22:03
It basically says that all cultures will eventually settle into a
1:22:09
pathological power oriented culture,
1:22:11
a bureaucratic rule oriented culture or a generative performance oriented culture.
1:22:16
And when I first read about it,
1:22:20
I thought the whole answer was to keep moving people away from power cultures
1:22:24
through bureaucratic into generative.
1:22:27
SPEAKER_3: And that the best place for high performing teams was in generative.
1:22:31
But now I have kind of come around to,
1:22:33
you have to choose the right tool for the job.
1:22:35
And there's a reason why,
1:22:36
for example,
1:22:37
a QA team or a deployment situation might actually be more rules oriented and
1:22:46
have a lot more process around it than say how certain other teams could work.
1:22:53
So how do you actually work together when you have different cultures within teams?
1:22:59
SPEAKER_3: How do you get anything done when you have different incentives?
1:23:02
So that's something that is,
1:23:04
I think of every people problem as a system design problem.
1:23:09
And so my answer is there are three types and what is the inter between them?
1:23:16
SPEAKER_0: And then as closing, what's a book that you would recommend and why?
1:23:20
SPEAKER_1: I thought about this forever, and I have to choose the k and RC book.
1:23:25
I feel like it melted my brain a little bit when I really,
1:23:29
really understood that object oriented programming is just C structures
1:23:34
with function pointers inside of it.
1:23:35
SPEAKER_1: Go through that book, type out every code example they have in it.
1:23:40
And I promise at the end of it you'll be a confused programmer,
1:23:43
but you will be better.
1:23:46
SPEAKER_0: Can you repeat the book again?
1:23:47
SPEAKER_1: Sorry.
1:23:48
SPEAKER_1: The KRC book, the classic Kergan and Richie C book.
1:23:52
SPEAKER_1: Did I get those names right?
1:23:54
SPEAKER_3: GaN and Richie?
1:23:55
SPEAKER_3: You did.
1:23:57
SPEAKER_1: I have it in eight different forms.
1:23:59
SPEAKER_3: Is it 35, 40 years old
1:24:01
SPEAKER_1: And the book's probably older than me.
1:24:02
SPEAKER_3: Okay.
1:24:03
SPEAKER_1: Still works.
1:24:05
SPEAKER_3: It was a Bible for a reason.
1:24:08
SPEAKER_3: It is a good book.
1:24:09
SPEAKER_1: So good.
1:24:12
SPEAKER_3: So I read a lot.
1:24:14
I am a really big fan of taking a first principle and trying to get a whole bunch
1:24:19
of people to try it at scale in order to change one small thing.
1:24:25
SPEAKER_3: So I decided, my recommendation right now is Kent Beck's tidy first.
1:24:31
We read it as a team last year and had a whole multiday conversation about how
1:24:37
we could incorporate,
1:24:38
yeah,
1:24:38
it's a little cat book.
1:24:39
It's also extremely skinny,
1:24:40
so you're not investing too much,
1:24:42
but it has a lot of great ideas in it.
1:24:44
SPEAKER_0: It's skinny, but lots of good ideas.
1:24:47
So yeah,
1:24:48
I would suggest pick up the book and then just,
1:24:52
you can open it up to any part and just pick one thing and try it for a few weeks
1:24:56
and see if it changes your flow.
1:24:59
But that particular one I think has launched a thousand little ideas with our team.
1:25:06
I think for me,
1:25:07
this is where I start to lose my credibility as an engineer.
1:25:11
SPEAKER_2: Project Hail Mary is my fiction book recommendation.
1:25:15
SPEAKER_2: Fantastic.
1:25:15
SPEAKER_2: Couldn't put it down.
1:25:17
SPEAKER_0: Awesome.
1:25:18
Well,
1:25:18
thanks very much for this interesting conversation on what it actually
1:25:22
takes to build an app that is a lot more complex on the inside than a lot of
1:25:27
people assume who downloaded it and use it.
1:25:30
SPEAKER_0: This was great.
1:25:32
SPEAKER_3: Tons of fun.
1:25:32
SPEAKER_3: Thank you.
1:25:33
SPEAKER_0: Yeah, this
1:25:34
SPEAKER_3: Was lots of fun.
1:25:35
SPEAKER_0: I hope you enjoyed this.
1:25:36
Behind the scenes look into how Reddit redid the core parts of their native
1:25:39
iOS and Android apps over the course of several years.
1:25:42
Thanks very much to Lauren,
1:25:43
Brandon,
1:25:43
and Eric,
1:25:44
and you can find all of them in the show notes link below.
1:25:47
For more details on mobile engineering challenges,
1:25:49
you can find longer deep dives in the pragmatic engineer as linked in the show notes.
1:25:52
I also wrote a book titled Building Mobile Apps at Scale that contains more
1:25:56
pointers on this topic.
1:25:57
If you've enjoyed this podcast,
1:25:58
please do subscribe on your favorite podcast platform and on YouTube.
1:26:02
SPEAKER_0: This helps more people discover the podcast.
1:26:04
SPEAKER_0: And a special thank you if you leave a rating.
1:26:06
SPEAKER_0: Thanks and see you in the next one.