Latest / Inside DuckDuckGo / Duck Tales: How we use mental models and operational defaults to improve decision making (Ep.39)
Transcript
- 0:02SPEAKER_00: Hello, and welcome to DuckTales,
- 0:06SPEAKER_00: where we go behind the scenes at DuckDuckGo and discuss the stories,
- 0:10SPEAKER_00: the technology, and the people who helped build privacy tools for everyone.
- 0:15SPEAKER_00: In each episode,
- 0:16SPEAKER_00: you'll hear from employees' louder vision,
- 0:18SPEAKER_00: product updates, engineering, or approach to AI.
- 0:22SPEAKER_00: Today, you are going to hear about how we work,
- 0:27SPEAKER_00: I guess, or one facet of how we work, which is Using operational defaults.
- 0:34SPEAKER_00: And I have with me Mark, who is going to go to interview about operational defaults.
- 0:42SPEAKER_00: Mark, do you want to introduce yourself?
- 0:46SPEAKER_01: Yeah, Mark, I'm on the engineering team at DuckDuckGo.
- 0:51SPEAKER_00: Okay.
- 0:52SPEAKER_00: That's about all you need to know.
- 0:54SPEAKER_00: Yeah.
- 0:56SPEAKER_00: I'm on the product team if you haven't seen me before.
- 1:00SPEAKER_00: Okay, cool.
- 1:01SPEAKER_00: Well, let's jump in.
- 1:02SPEAKER_00: What even is an operational default?
- 1:06SPEAKER_01: Yeah.
- 1:07SPEAKER_01: So defaults are, I would say, a deceptively simple
- 1:15SPEAKER_01: We're always trying to leverage mental models to help us move more quickly
- 1:18SPEAKER_01: and think more clearly and things of that nature.
- 1:22SPEAKER_01: In that context, the default effect is seen as a negative right now.
- 1:26SPEAKER_01: And I think we've all seen that in apps and in different places.
- 1:30SPEAKER_01: Here we're trying to leverage the upside of that,
- 1:34SPEAKER_01: where we're basically taking any decision that you can make ahead of time
- 1:41SPEAKER_01: that Generally is true or has a right answer 80% of the time.
- 1:46SPEAKER_01: We're going to call that a default.
- 1:49SPEAKER_01: For that other 20%,
- 1:50SPEAKER_01: we want to leave room for our folks to be able to make exceptions,
- 1:54SPEAKER_01: ideally very thoughtful exceptions.
- 1:56SPEAKER_00: When you say default is a negative in the app,
- 1:59SPEAKER_00: you mean like,
- 2:00SPEAKER_00: you know,
- 2:00SPEAKER_00: if you've got an Android phone and the default browser is Chrome and
- 2:05SPEAKER_00: the default search engine is Google,
- 2:07SPEAKER_00: and it's really hard for another player like DuckDuckGo to break
- 2:11SPEAKER_00: through those defaults.
- 2:12SPEAKER_00: Is that right?
- 2:14SPEAKER_01: Yeah, exactly.
- 2:16SPEAKER_01: In Gabe's book of mental models, there's the default effect.
- 2:19SPEAKER_01: And again, it's generally negative.
- 2:20SPEAKER_01: But like I said, in this case, we're trying to use defaults as defaults.
- 2:25SPEAKER_01: As something that we can leverage to help us go quicker.
- 2:28SPEAKER_00: Yeah, yeah.
- 2:29SPEAKER_00: So how is it different than a rule?
- 2:32SPEAKER_01: I mean, it's conceptually the same thing.
- 2:35SPEAKER_01: I think rules are just more binary, right?
- 2:37SPEAKER_01: Like this is something you do or you don't do pretty much all the time.
- 2:42SPEAKER_01: Where here we're saying that these are things that are generally true.
- 2:46SPEAKER_01: Like I said, 80% of the time, it's just the number that we made up.
- 2:50SPEAKER_01: And that allows people to potentially...
- 2:54SPEAKER_01: Make faster decisions.
- 2:56SPEAKER_01: But we also want to encourage people to think about what they're doing
- 3:00SPEAKER_01: and not just go on autopilot.
- 3:01SPEAKER_01: And I think that's the danger of catching something as a rule is that,
- 3:06SPEAKER_01: yeah, people will just kind of blindly follow it by default.
- 3:09SPEAKER_00: Yeah, yeah.
- 3:10SPEAKER_00: I think that distinction is super important.
- 3:12SPEAKER_00: Like, first of all,
- 3:15SPEAKER_00: there are a lot of things where rules just have exceptions
- 3:21SPEAKER_00: and therefore they're not really good rules, you know?
- 3:24SPEAKER_00: Like you have to be pretty sure that there has to be a really good reason
- 3:28SPEAKER_00: to make something a rule because it's going to fit different situations differently.
- 3:32SPEAKER_00: And yeah, it's like once it becomes a rule, people like lose power to do anything differently.
- 3:37SPEAKER_00: So they don't think about it.
- 3:39SPEAKER_00: They don't feel empowered.
- 3:41SPEAKER_00: And like with the default,
- 3:43SPEAKER_00: there's still a burden of judgment because you can overturn the default,
- 3:48SPEAKER_00: choose a different path.
- 3:50SPEAKER_00: But the idea, I think, Would you say this is a fair characterization?
- 3:57SPEAKER_00: A, don't spend brainpower thinking about reinventing the wheel or thinking
- 4:01SPEAKER_00: about something that doesn't really warrant brainpower.
- 4:04SPEAKER_00: There's a good default.
- 4:05SPEAKER_00: And B,
- 4:08SPEAKER_00: just to nudge people in the direction of the best average answer.
- 4:15SPEAKER_01: Yeah, right.
- 4:15SPEAKER_01: It's leveraging decisions that we've deliberated over in the past and,
- 4:20SPEAKER_01: you know, have come to a good decision on, um, and exactly what you said.
- 4:24SPEAKER_01: We don't want folks reinventing the wheel.
- 4:26SPEAKER_01: That said,
- 4:27SPEAKER_01: we do want, I mean,
- 4:28SPEAKER_01: we hire really smart people at DuckDuckGo and we want to,
- 4:31SPEAKER_01: you know, leverage and celebrate that.
- 4:33SPEAKER_01: So yeah,
- 4:34SPEAKER_01: everybody's encouraged to,
- 4:36SPEAKER_01: um, Challenge the norm, so to speak, or, you know, pre-existing defaults.
- 4:41SPEAKER_01: And in fact,
- 4:41SPEAKER_01: that's a big,
- 4:42SPEAKER_01: big part of this whole concept is coming back and challenging these at times.
- 4:47SPEAKER_01: But yeah,
- 4:48SPEAKER_01: they're,
- 4:49SPEAKER_01: you know,
- 4:50SPEAKER_01: in a lot of cases,
- 4:51SPEAKER_01: it's, it would just be wasted effort to be rethinking this from scratch.
- 4:55SPEAKER_01: And as much as there are some that are,
- 4:57SPEAKER_01: that are rule or that,
- 4:58SPEAKER_01: you know,
- 4:59SPEAKER_01: might make better rules because they are more binary,
- 5:02SPEAKER_01: there are others that really don't make Great defaults either.
- 5:07SPEAKER_01: We had a decent example where it was like,
- 5:10SPEAKER_01: yeah, there was something that was true, say 50% of the time.
- 5:14SPEAKER_01: I don't remember what the actual number was, but it just didn't make a great default.
- 5:17SPEAKER_01: I honestly don't remember what it is,
- 5:19SPEAKER_01: but yeah, just to say that not everything falls into that framework.
- 5:24SPEAKER_00: Yeah.
- 5:25SPEAKER_00: Give me some examples of things that we do think are good operational defaults or
- 5:29SPEAKER_00: that we treat as such.
- 5:32SPEAKER_01: Sure, yeah.
- 5:33SPEAKER_01: I mean, we have a bunch,
- 5:35SPEAKER_01: you know,
- 5:35SPEAKER_01: one of the things that's so powerful about this is that I think we all do this all
- 5:39SPEAKER_01: the time.
- 5:41SPEAKER_01: So, for example,
- 5:42SPEAKER_01: at DuckDuckGo,
- 5:43SPEAKER_01: we try to prioritize quick wins,
- 5:45SPEAKER_01: which I guess the other thing I should say about defaults is
- 5:48SPEAKER_01: that they are somewhat self-evident,
- 5:51SPEAKER_01: like to the point where that's why I said they're kind of deceptively simple,
- 5:53SPEAKER_01: because...
- 5:54SPEAKER_01: There are things that you don't necessarily feel like you should have to put a lot
- 5:58SPEAKER_01: of time into, a lot of thought into.
- 6:01SPEAKER_01: But quick wins are one where,
- 6:03SPEAKER_01: you know,
- 6:04SPEAKER_01: we at the company, we've been, you know, trying to push people toward that, right?
- 6:08SPEAKER_01: Like the idea of a quick win is something that is relatively low effort, but has high impact.
- 6:14SPEAKER_01: And I mean,
- 6:15SPEAKER_01: you would saying it out loud,
- 6:16SPEAKER_01: you would say, of course, that's something we should prioritize.
- 6:18SPEAKER_01: But I think it's also important to acknowledge that people don't think that way.
- 6:24SPEAKER_01: Instinctually,
- 6:25SPEAKER_01: in a lot of cases,
- 6:26SPEAKER_01: they want to,
- 6:27SPEAKER_01: sometimes they want to work on,
- 6:28SPEAKER_01: you know,
- 6:28SPEAKER_01: the hardest thing or what they see as the most impact when,
- 6:32SPEAKER_01: yeah, maybe less would do.
- 6:35SPEAKER_01: So that's one.
- 6:39SPEAKER_01: One of my favorite ones,
- 6:40SPEAKER_01: and I think a lot of people at this company would agree,
- 6:43SPEAKER_01: is we do no meetings on Wednesdays.
- 6:47SPEAKER_01: So that's a very simple one that, yeah, you,
- 6:52SPEAKER_01: we don't,
- 6:53SPEAKER_01: I wouldn't call it a rule,
- 6:55SPEAKER_01: whereas we don't,
- 6:56SPEAKER_01: you know,
- 6:56SPEAKER_01: you can have a meeting on a Wednesday and sometimes you have to,
- 6:59SPEAKER_01: especially if it's with somebody external.
- 7:01SPEAKER_01: But generally, you shouldn't be scheduling meetings on Wednesdays.
- 7:05SPEAKER_00: See, that's interesting because I would have called that one a rule.
- 7:09SPEAKER_00: Specifically, I would have called the rule no internal planned meetings.
- 7:15SPEAKER_00: If you want to hop on real-time and talk to somebody internally about something
- 7:19SPEAKER_00: as it comes up, fine.
- 7:21SPEAKER_00: If you have an external call, we've
- 7:23SPEAKER_01: also accepted that.
- 7:25SPEAKER_00: But I try to enforce that one as a rule because I think that one,
- 7:28SPEAKER_00: if it's a default, people just don't observe it.
- 7:32SPEAKER_00: They break it way too much.
- 7:33SPEAKER_00: That's my opinion.
- 7:35SPEAKER_01: Well, yeah, but I think you just gave a few exceptions that are good, right?
- 7:38SPEAKER_01: Like, I'm not going to necessarily schedule something ahead of time,
- 7:42SPEAKER_01: but I will reach out to somebody and say,
- 7:45SPEAKER_01: hey, it would be quicker if we talked about this ad hoc.
- 7:48SPEAKER_01: So maybe, I mean, yeah, maybe it's splitting hairs, but.
- 7:52SPEAKER_00: Yeah, it's interesting because I think of those as part of, as like the rule is constrained.
- 7:56SPEAKER_00: So I guess making a rule is safer the more you constrain it.
- 8:02SPEAKER_00: And you could arguably tap a one operational problem and either make
- 8:07SPEAKER_00: a highly constrained rule that has exceptions and carve outs or a broader default.
- 8:15SPEAKER_00: And maybe sometimes both at the same time.
- 8:17SPEAKER_00: So yeah, okay, that's a good one.
- 8:19SPEAKER_00: Give me another example.
- 8:20SPEAKER_01: Yeah, I can give you another one.
- 8:23SPEAKER_01: So this one we came up with recently,
- 8:25SPEAKER_01: and that's working incrementally,
- 8:27SPEAKER_01: which again,
- 8:28SPEAKER_01: you would say,
- 8:29SPEAKER_01: and when I say working incrementally,
- 8:31SPEAKER_01: I basically mean,
- 8:31SPEAKER_01: you know,
- 8:33SPEAKER_01: we're trying to ship smaller features more frequently or even larger features
- 8:38SPEAKER_01: in smaller increments.
- 8:40SPEAKER_01: You're going back and you're You know,
- 8:43SPEAKER_01: making sure that you're on the right track,
- 8:45SPEAKER_01: et cetera,
- 8:46SPEAKER_01: which again,
- 8:46SPEAKER_01: sounds like obviously that's something that you would want to do,
- 8:50SPEAKER_01: but anyone who's ever collaborated with anyone on anything,
- 8:55SPEAKER_01: I think would,
- 8:55SPEAKER_01: would realize that, yeah, that's, I don't think that's a human default, right?
- 9:00SPEAKER_01: Like, I think we all,
- 9:01SPEAKER_01: you know,
- 9:02SPEAKER_01: want to have a task,
- 9:03SPEAKER_01: go build whatever we're supposed to build and then show the awesome thing
- 9:08SPEAKER_01: that we did.
- 9:09SPEAKER_01: And yeah, I think that's a good instinct to have.
- 9:12SPEAKER_01: But there are many cases where if you misunderstood something initially
- 9:15SPEAKER_01: or new information pops up, yeah,
- 9:19SPEAKER_01: it's super important to make sure that you're raising that up and collaborating
- 9:23SPEAKER_01: with others on.
- 9:25SPEAKER_00: So you said that we made that one recently.
- 9:27SPEAKER_00: Like, what does it mean to make an operational default?
- 9:30SPEAKER_00: How do we codify or communicate these?
- 9:34SPEAKER_01: Yeah, that's a great question.
- 9:36SPEAKER_01: And I don't think we've actually figured that out yet.
- 9:38SPEAKER_01: We have a lot of defaults across the company.
- 9:40SPEAKER_01: This one was communicated to several objectives.
- 9:47SPEAKER_01: But yeah,
- 9:47SPEAKER_01: it's not,
- 9:48SPEAKER_01: I would say it's written down,
- 9:50SPEAKER_01: but it's not,
- 9:50SPEAKER_01: there's no like internal glossary or internal Um,
- 9:54SPEAKER_01: directory of, of defaults or anything like that.
- 9:57SPEAKER_01: It's basically just a reminder for folks.
- 9:59SPEAKER_01: And again,
- 10:00SPEAKER_01: it seems like it's something that you should obviously do,
- 10:03SPEAKER_01: but, um, I think it's important to remind people of these things.
- 10:07SPEAKER_00: Yeah.
- 10:07SPEAKER_00: So like we, basically we write things down in the form of like messages and.
- 10:14SPEAKER_00: Uh, or, or really, I mean, anyway, but, um, and I think that can happen.
- 10:19SPEAKER_00: Maybe one thing that's worth saying is that that can happen at different levels of
- 10:22SPEAKER_00: the organization.
- 10:22SPEAKER_00: Like sometimes you have a group of five people working on an objective
- 10:25SPEAKER_00: and they might adopt their own defaults or ways of operating.
- 10:30SPEAKER_00: And then on the other hand,
- 10:31SPEAKER_00: once something,
- 10:32SPEAKER_00: once we like believe something to be, um,
- 10:36SPEAKER_00: sorry, I don't know if listeners can hear that, but my dog is drinking some water.
- 10:41SPEAKER_00: We'll see if that comes through.
- 10:43SPEAKER_00: Um, You know,
- 10:45SPEAKER_00: once we're really confident in something and that it's like broadly applicable
- 10:48SPEAKER_00: across the organization,
- 10:50SPEAKER_00: then we might communicate at like a higher level and you codify it
- 10:54SPEAKER_00: a little bit more, more strongly as a default.
- 10:57SPEAKER_00: Yeah.
- 10:57SPEAKER_01: And that's kind of where that's kind of where we are with this right now.
- 11:00SPEAKER_01: We've communicated to a couple of objectives.
- 11:02SPEAKER_01: You know, objectives end up having their own culture intentionally or otherwise.
- 11:07SPEAKER_01: And this is a, yeah,
- 11:08SPEAKER_01: an attempt to kind of test it on a smaller scale
- 11:11SPEAKER_01: before we start communicating it more broadly.
- 11:15SPEAKER_01: And I have one other I can give.
- 11:17SPEAKER_01: This one's much more engineering centric, but so we're all working on AI quite a bit these days.
- 11:24SPEAKER_01: And I think anybody who's ever worked with, well,
- 11:26SPEAKER_01: anybody that's ever used AI will tell you that or has observed that you don't get
- 11:31SPEAKER_01: the same answer twice, right?
- 11:33SPEAKER_01: It's, you know, so-called non-deterministic output.
- 11:37SPEAKER_01: So in order for us to build products leveraging AI,
- 11:41SPEAKER_01: It's really important that we have evaluations up front, right?
- 11:45SPEAKER_01: That you start with,
- 11:47SPEAKER_01: you know,
- 11:47SPEAKER_01: say a list of,
- 11:49SPEAKER_01: if I have question Y,
- 11:50SPEAKER_01: or sorry,
- 11:51SPEAKER_01: question X,
- 11:52SPEAKER_01: I'm going to get answer Y,
- 11:54SPEAKER_01: or at least something in the general, with the general gist of answer Y.
- 11:59SPEAKER_01: And yeah, because you can't always super accurately predict what's going to come out.
- 12:04SPEAKER_01: It's really important that before we start building something, you know, you have this eval.
- 12:11SPEAKER_01: Or evaluation data set so that you know when you're doing a good job or when you're not.
- 12:18SPEAKER_01: I think that one,
- 12:19SPEAKER_01: honestly,
- 12:19SPEAKER_01: that one's fairly close to a rule in my book in terms of
- 12:23SPEAKER_01: the engineering expectations we have.
- 12:25SPEAKER_01: But there are definitely cases where,
- 12:27SPEAKER_01: you know, it doesn't make sense or it's too much effort to start there.
- 12:31SPEAKER_00: Yeah.
- 12:32SPEAKER_00: And so like,
- 12:32SPEAKER_00: because that's a default,
- 12:34SPEAKER_00: even if people,
- 12:35SPEAKER_00: most people kind of think it's a best practice,
- 12:37SPEAKER_00: codifying it as a default means that if somebody decides not to do it,
- 12:40SPEAKER_00: they might get the question, why?
- 12:42SPEAKER_00: Are you sure?
- 12:43SPEAKER_00: And they have to like,
- 12:44SPEAKER_00: basically, if you don't use a default, you have to have a good answer for why.
- 12:49SPEAKER_01: Yeah, that's exactly right.
- 12:50SPEAKER_01: Yes, you have to have, yeah,
- 12:52SPEAKER_01: there's, we encourage people to make exceptions, but it's got to be well-reasoned.
- 12:58SPEAKER_00: Yeah, yeah.
- 12:59SPEAKER_00: Cool.
- 12:59SPEAKER_00: Cool.
- 12:59SPEAKER_00: Can you think of any defaults that we've adopted as an organization or on a team and
- 13:06SPEAKER_00: then like decided were the wrong default and changed them?
- 13:10SPEAKER_01: Yeah, I thought about this one a bit.
- 13:13SPEAKER_01: I can't think of one where we actually decided this was the wrong default,
- 13:17SPEAKER_01: but I can,
- 13:18SPEAKER_01: there are several that we have,
- 13:20SPEAKER_01: you know, potentially evolved over time and that continue to evolve.
- 13:26SPEAKER_01: The one, and yeah, this one is also super engineering.
- 13:30SPEAKER_01: But the one that comes to mind is,
- 13:32SPEAKER_01: you know,
- 13:33SPEAKER_01: initially all of our backend engineers were,
- 13:35SPEAKER_01: the default for all of them was to use Perl for our backend services,
- 13:39SPEAKER_01: the Perl programming language.
- 13:41SPEAKER_01: But we made exceptions there, you know, because no language is the right tool for every job.
- 13:47SPEAKER_01: So, you know,
- 13:48SPEAKER_01: if we were developing something that Perl just wasn't the right tool for,
- 13:53SPEAKER_01: we gave people the space to make a case to not use it.
- 13:58SPEAKER_01: And that was true for a number of years.
- 14:00SPEAKER_01: I think now I wouldn't call that a default anymore.
- 14:03SPEAKER_01: I don't think that that is the starting place for most folks,
- 14:06SPEAKER_01: like when we're developing a new service.
- 14:10SPEAKER_01: So maybe that's one that you would either say has evolved or maybe retired is
- 14:15SPEAKER_01: a better way to put it.
- 14:16SPEAKER_01: But yeah,
- 14:18SPEAKER_01: I can't think of one where I think we were actually outright wrong,
- 14:20SPEAKER_01: but definitely new information and things come in and we change them.
- 14:26SPEAKER_00: One that comes to mind to me that maybe went from not really having defined
- 14:31SPEAKER_00: a default intentionally,
- 14:33SPEAKER_00: but let the flow of the stream define itself was,
- 14:42SPEAKER_00: as some listeners may know,
- 14:43SPEAKER_00: our company is organized into a bunch of objectives,
- 14:47SPEAKER_00: like teams working towards some important goal for the company.
- 14:53SPEAKER_00: Those objectives are expected to hold what we call an assessment every so often
- 14:59SPEAKER_00: to kind of like redefine the strategy and the roadmap.
- 15:03SPEAKER_00: And we I think for the first maybe few years I was here,
- 15:08SPEAKER_00: we didn't have an explicit expectation on how often that happened.
- 15:12SPEAKER_00: But I think the sort of adopted default was.
- 15:16SPEAKER_00: Um, a pretty long period,
- 15:18SPEAKER_00: like maybe on average, I don't know, five months, six months, something like that.
- 15:25SPEAKER_00: And then at some point.
- 15:26SPEAKER_00: When you felt like it was important.
- 15:28SPEAKER_00: What's that?
- 15:29SPEAKER_01: I said, when you felt like it was important, it was very, I think, very subjective.
- 15:33SPEAKER_00: Yeah.
- 15:33SPEAKER_00: Well, and it was like easy to procrastinate if you were leading an objective,
- 15:37SPEAKER_00: you know, like,
- 15:38SPEAKER_00: cause it felt like a time,
- 15:39SPEAKER_00: like a thing you'd have to put time into and put yourself out there.
- 15:42SPEAKER_00: And so it just kind of was like getting long.
- 15:45SPEAKER_00: But in reality,
- 15:46SPEAKER_00: we observed that when people ran assessments more often,
- 15:50SPEAKER_00: we had better roadmaps and better decision making and,
- 15:54SPEAKER_00: you know,
- 15:54SPEAKER_00: sort of less leaving the track without, you know, the help to get back on the track.
- 16:01SPEAKER_00: And so at some point, we, I think, maybe said, like, let's default to every eight weeks.
- 16:05SPEAKER_00: And then we maybe brought that back to six.
- 16:07SPEAKER_00: Right now, it's six weeks as the default.
- 16:10SPEAKER_00: And I think that's been a nice improvement to make that explicit.
- 16:15SPEAKER_01: It's a good example of working incrementally over a different time horizon.
- 16:20SPEAKER_01: But yeah, for sure.
- 16:22SPEAKER_00: Cool.
- 16:23SPEAKER_00: Anything else you want to add, Mark, before we wrap this?
- 16:27SPEAKER_01: I don't think so.
- 16:28SPEAKER_01: I think, again,
- 16:30SPEAKER_01: I feel like I have to keep saying that a lot of these are self-evident
- 16:32SPEAKER_01: because it just sounds so obvious in a lot of cases.
- 16:35SPEAKER_01: But it is, yeah, I think it leads to better results.
- 16:40SPEAKER_01: And hopefully we're going to see that very soon in these objectives where we've introduced it.
- 16:44SPEAKER_01: And yeah.
- 16:45SPEAKER_01: They're really important to challenge them as well.
- 16:48SPEAKER_00: Yeah,
- 16:48SPEAKER_00: there's probably a lot of obvious things that people don't do on the regular because
- 16:53SPEAKER_00: of some cognitive,
- 16:55SPEAKER_00: you know, glitch or habit or, you know, your predictably irrational behavior.
- 17:02SPEAKER_00: So, yeah, just making it explicit, I think, can be powerful.
- 17:06Yeah.
- 17:07SPEAKER_00: Cool.
- 17:08SPEAKER_00: All right.
- 17:08SPEAKER_00: Well, thank you, Mark, and see you around.
- 17:12SPEAKER_01: Yeah, thanks.