Latest / Inside DuckDuckGo / Duck Tales: Why DuckDuckGo Added the ‘More’ Button for AI-Assisted Answers (Episode 1)
Transcript
- 0:02SPEAKER_01: Hi,
- 0:02SPEAKER_01: welcome to DuckTales,
- 0:03SPEAKER_01: where we pull back the curtain on DuckDuckGo to share stories,
- 0:07SPEAKER_01: tech,
- 0:07SPEAKER_01: and people building privacy tools for everyone.
- 0:10SPEAKER_01: Today, my guest is Tim Rebold, who leads AI development at DuckDuckGo.
- 0:18SPEAKER_01: Welcome, Tim.
- 0:19SPEAKER_01: Thank you.
- 0:20SPEAKER_01: Camille Baz Baz?
- 0:23SPEAKER_01: That's right.
- 0:24SPEAKER_01: Pretty good.
- 0:26SPEAKER_01: Baz Baz.
- 0:28SPEAKER_01: So we're going to talk about a new feature.
- 0:31that we just shipped on the old SERP at DuckDuckGo,
- 0:35which is the More button on Assist.
- 0:39Can you briefly explain what the More button does and why we
- 0:43decided to add it to
- 0:45SPEAKER_01: the AI Assist Answers?
- 0:47SPEAKER_00: Yep, sure.
- 0:49SPEAKER_00: Well,
- 0:49SPEAKER_00: SearchAssist,
- 0:49SPEAKER_00: first of all,
- 0:50is the AI-generated answer that goes on the top of results for
- 0:55about a fifth of
- 0:56SPEAKER_00: queries.
- 0:57That's whenever we think that the query is asking for
- 1:00information that can be
- 1:01SPEAKER_00: answered concisely.
- 1:05The main thing we pride ourselves on with Assist is that it is
- 1:08just the answer to
- 1:09SPEAKER_00: your question and nothing more.
- 1:11SPEAKER_00: It's unique in that way,
- 1:12SPEAKER_00: I think,
- 1:12across the AI answers industry in that it really tries to get
- 1:17the job done in a
- 1:18SPEAKER_00: very short amount of words.
- 1:20SPEAKER_00: But users,
- 1:21SPEAKER_00: for some queries,
- 1:22SPEAKER_00: were asking it to go deeper on topics,
- 1:24SPEAKER_00: and that's the answer to the more button question.
- 1:27SPEAKER_00: So it's a pretty simple feature.
- 1:30Right below the concise answer is a button that says more,
- 1:34and you click it.
- 1:35SPEAKER_00: And then it will go and expand its answer into more of a,
- 1:39maybe like a traditional answer that you see in an AI chat bot
- 1:42that has tables and
- 1:43SPEAKER_00: headings and it lays things out.
- 1:46SPEAKER_01: Very cool.
- 1:47And so what problem do you feel like users were experiencing
- 1:50that led to this future?
- 1:51SPEAKER_00: Um,
- 1:53Yeah,
- 1:54it was really just a request to go deeper on some percentage of
- 1:57topics.
- 1:58The short,
- 1:58concise answers are just what the doctor ordered for many,
- 2:02many of them.
- 2:04SPEAKER_00: But the click rate on the more button has been really good so far.
- 2:08SPEAKER_00: It's about 10% of people are clicking that button.
- 2:10SPEAKER_00: So around that portion of queries,
- 2:16the topic they asked about is they just want a little more
- 2:20depth with the answer.
- 2:22What was the biggest technical challenge you had in
- 2:25implementing this expanded answer?
- 2:29SPEAKER_00: Well, I don't know how far back you want to pull the curtain.
- 2:34SPEAKER_00: Your intro.
- 2:35SPEAKER_00: Full kimono, please.
- 2:36SPEAKER_00: Okay.
- 2:37SPEAKER_00: Well, I mean, the SERP is written in, largely written in Perl.
- 2:42SPEAKER_00: Perl has a, it is challenging to stream the response back in Perl.
- 2:49SPEAKER_00: So when you use an AI tool like DuckAI,
- 2:53SPEAKER_00: our chatbot tool.
- 2:54SPEAKER_00: The response will stream back,
- 2:57which means that every single set of words that it doesn't wait
- 3:01till the end of its
- 3:02SPEAKER_00: response to give it to you,
- 3:03SPEAKER_00: it gives you the response as it's going.
- 3:05And it's a much faster experience for users because they can
- 3:08start to read it as
- 3:09SPEAKER_00: the AI is generating word after word.
- 3:12SPEAKER_00: That's called streaming.
- 3:15SPEAKER_00: In Search Assist, we don't need to stream that concise response.
- 3:18SPEAKER_00: We wait until it's done.
- 3:20SPEAKER_00: We run some safety checks on it even.
- 3:23SPEAKER_00: And then we present the answer on the page.
- 3:25But with the more button,
- 3:27it would just be too long to wait to take that same approach.
- 3:31SPEAKER_00: So we needed to figure out streaming.
- 3:34SPEAKER_00: And in some cases, that's not a hard technical problem at all.
- 3:38In our particular environment and setup,
- 3:40that happened to be the hardest problem.
- 3:43SPEAKER_00: Second hardest problem is you need to come up with a prompt.
- 3:47that explains to the ai what type of response we're looking for
- 3:50um and that is you
- 3:53know our initial prompt as we've been saying is very focused on
- 3:58being concise but
- 3:59this one we needed to figure out the right words to tell the ai
- 4:02to say what type of
- 4:04answer we want out of an expanded answer and how we wanted to
- 4:06lay it out and talk
- 4:08to the user gotcha um and so how are you i mean it sounds like
- 4:14in the development
- 4:15SPEAKER_01: of this you were trying to balance
- 4:17the speed and simplicity people expect from DuckDuckGo with
- 4:23getting a more
- 4:24SPEAKER_01: comprehensive answer to their question.
- 4:28SPEAKER_00: Does that sound right?
- 4:31SPEAKER_00: It does sound right, yeah.
- 4:32SPEAKER_00: And I mean,
- 4:33it seems simple in retrospect that a simple button to click
- 4:35more was the answer,
- 4:36SPEAKER_00: but we
- 4:38SPEAKER_00: As we were building Search Assist,
- 4:40SPEAKER_00: we knew we wanted to give more in some cases to some answers.
- 4:46SPEAKER_00: And it was sort of a light bulb moment to just say,
- 4:49SPEAKER_00: well,
- 4:49the first step we can take is just to ship the user a button
- 4:52and the user can
- 4:52SPEAKER_00: decide when they want more.
- 4:55So that took the burden off of us to figure out which queries
- 5:00exactly deserve to
- 5:01SPEAKER_00: have more of an answer.
- 5:03SPEAKER_00: And then from a speed standpoint, that sort of solves that problem too.
- 5:07SPEAKER_00: because once they're engaged and clicking more,
- 5:11SPEAKER_00: they've presumably read the concise answer,
- 5:14SPEAKER_00: and the new answer is going to stream back,
- 5:16SPEAKER_00: so they're going to get to read something almost right away.
- 5:20SPEAKER_00: The speed piece there,
- 5:22we have a little bit more leeway to take a few seconds to
- 5:26deliver the answer,
- 5:27SPEAKER_00: whereas the concise answer is coming along with the SERP,
- 5:29SPEAKER_00: and people want a fast SERP,
- 5:31SPEAKER_00: so can't delay that.
- 5:34SPEAKER_01: What's been the feedback so far and what kind of data are we seeing?
- 5:40SPEAKER_00: It was the number one feature request before we built it.
- 5:45And it has largely satisfied that request that people asking
- 5:50for it has died down,
- 5:51SPEAKER_00: as you would expect.
- 5:52SPEAKER_00: And our overall,
- 5:55the main metric we use to track user satisfaction for this
- 5:59feature is the ratio of
- 6:01SPEAKER_00: people clicking the thumbs up over the thumbs down.
- 6:04that's the bottom of each answer it says was this helpful with
- 6:07those two options um
- 6:08so after we ship the more button the overall thumbs up to
- 6:12thumbs down ratio
- 6:13increased um noticeably so people seem to like it and uh if
- 6:19you're thinking about
- 6:20SPEAKER_01: the differences between duckduckgo what you would see on google
- 6:29SPEAKER_01: And then if you are using ChatGPT or like a perplexity,
- 6:34SPEAKER_01: now that DuckDuckO has more and is easily connecting to Duck.ai,
- 6:39SPEAKER_01: you know,
- 6:40SPEAKER_01: how would you place our offerings in that context?
- 6:46SPEAKER_00: Yeah.
- 6:47SPEAKER_00: Well,
- 6:48SPEAKER_00: when it comes specifically to these AI responses,
- 6:51I still think the main differentiator is that we play the 80-20
- 6:56rule really well
- 6:57SPEAKER_00: with the
- 6:59SPEAKER_00: length of the answer at first.
- 7:02SPEAKER_00: In most cases,
- 7:05giving the user the direct answer in a way that they don't have
- 7:09to spend a lot of
- 7:09SPEAKER_00: time parsing or reading is the preferred experience.
- 7:14SPEAKER_00: And we've heard users say that, specifically say that
- 7:18SPEAKER_00: Your answers are so much shorter.
- 7:19I didn't even know I wanted that,
- 7:21but that makes a lot more sense in most queries.
- 7:26And letting the user expand into this more button is the first
- 7:32step into some
- 7:34SPEAKER_00: queries do deserve a richer,
- 7:35SPEAKER_00: deeper answer.
- 7:37SPEAKER_00: It deserves more work from the AI to be able to do for the user.
- 7:44SPEAKER_00: So this more button is the first step along that path.
- 7:47But,
- 7:49you know,
- 7:49I think there's other tools that started there and they're
- 7:54coming back to...
- 7:57SPEAKER_00: what people want in a search engine, the core search engine experience.
- 8:00SPEAKER_00: They're having to add on to the core AI experience.
- 8:02SPEAKER_00: And we just started the other way around.
- 8:04We started with a great search engine that already gave a bunch
- 8:07of instant answers
- 8:08that were not AI-generated on stocks and sports and weather and
- 8:14things like that.
- 8:16So our search assist tool fits right into the wider SERP
- 8:23experience for people.
- 8:27SPEAKER_01: what do you,
- 8:29SPEAKER_01: how does this fit into,
- 8:31SPEAKER_01: you know,
- 8:32SPEAKER_01: the DuckDuckGo,
- 8:33SPEAKER_01: you know,
- 8:34SPEAKER_01: looking,
- 8:34SPEAKER_01: looking to the future here,
- 8:35SPEAKER_01: um,
- 8:37SPEAKER_01: how do you see this evolving?
- 8:38SPEAKER_01: What do you think is,
- 8:39SPEAKER_01: um,
- 8:41SPEAKER_01: is next for these kinds of,
- 8:44SPEAKER_01: for the,
- 8:44SPEAKER_01: for the search assist answer on DuckDuckGo?
- 8:49SPEAKER_00: Um, well, I think both, both of our AI tools will evolve, um,
- 8:56And DuckAI will get more and more ability to do things that
- 9:02search engines do.
- 9:04And a search engine will get more and more ability to lay out a
- 9:08search engine
- 9:08SPEAKER_00: results page,
- 9:09SPEAKER_00: a SERP,
- 9:10SPEAKER_00: with the benefit of generated content from AI.
- 9:17The difference right now is primarily the types of queries that
- 9:20people are bringing
- 9:21SPEAKER_00: to these two tools.
- 9:22SPEAKER_00: Whether or not
- 9:23SPEAKER_00: people's behavior merges into one tool is anybody's guess.
- 9:27SPEAKER_00: It's a very interesting question.
- 9:28But right now on the search engine,
- 9:31most of the queries are four words or fewer.
- 9:36you're not typing big,
- 9:38long prompts or asking long,
- 9:40complicated questions.
- 9:41So the search assist lives on the SERP and is built for those
- 9:47types of queries more so.
- 9:49SPEAKER_00: Whereas in DuckDuck.ai,
- 9:50SPEAKER_00: we see,
- 9:51SPEAKER_00: we don't see actually,
- 9:52because we don't see any of the prompts and that can be
- 9:55annoying from a building a
- 9:56SPEAKER_00: product standpoint.
- 9:57But we know from other data sets and other tools and benchmarks
- 10:01that when people go to AI tools,
- 10:04SPEAKER_00: they don't just type in four words,
- 10:06they're being a little bit more conversational in the way that
- 10:09they prompt the
- 10:10SPEAKER_00: model.
- 10:12SPEAKER_00: So that deserves a different,
- 10:14SPEAKER_00: yeah,
- 10:14SPEAKER_00: those queries are,
- 10:16SPEAKER_00: the user is asking something different,
- 10:17SPEAKER_00: so we're obviously we're gonna be,
- 10:19SPEAKER_00: the answer is gonna be differently natured.
- 10:24SPEAKER_00: One more thing on the future, I think.
- 10:29SPEAKER_00: So right now,
- 10:30SPEAKER_00: The search assist concise answer, it goes to the model one time.
- 10:35SPEAKER_00: One shot is the terminology in AI land.
- 10:40SPEAKER_00: And we give it context from the web,
- 10:43SPEAKER_00: and it has all the information it needs to answer the question,
- 10:46SPEAKER_00: and it comes up with a concise answer.
- 10:47SPEAKER_00: But it has one shot to do that.
- 10:50And it's up to us to feed the ALM with the right context to
- 10:55answer the question.
- 10:58But sometimes,
- 10:59you know,
- 11:00you've seen these deep research tools around probably where
- 11:06SPEAKER_00: You could imagine that for some queries,
- 11:08SPEAKER_00: it might want to search twice,
- 11:09SPEAKER_00: not just once,
- 11:10SPEAKER_00: but twice.
- 11:10So if I say,
- 11:11you know,
- 11:12who is older,
- 11:13person A or person B,
- 11:14it might do two searches.
- 11:16SPEAKER_00: How old is person A and how old is person B?
- 11:18And then it has all the information it needs to answer your
- 11:21question because
- 11:22probably nowhere on the internet was it written person A is
- 11:25older than person B.
- 11:26SPEAKER_00: You just have to do two separate searches.
- 11:29SPEAKER_00: That's sort of the next step where you could imagine that going much more
- 11:33SPEAKER_00: complicated into many,
- 11:34SPEAKER_00: many searches.
- 11:36SPEAKER_00: And maybe not just searches,
- 11:37but it could also use different tools other than search to
- 11:42gather information to
- 11:43SPEAKER_00: answer the question.
- 11:45SPEAKER_00: So at some point, we are starting to do this on DuckAI.
- 11:50And it's also interesting to start to do this on Search Assist
- 11:53to let the model,
- 11:54if the user...
- 11:56SPEAKER_00: tells us that they want more,
- 11:57SPEAKER_00: they're willing to wait in 10 seconds or even a minute for an answer,
- 12:02that we can use that time to have the LOM do a lot more things
- 12:07and do one thing
- 12:09SPEAKER_00: sort of in the loop to answer the question.
- 12:12SPEAKER_01: Got it.
- 12:12SPEAKER_01: That sounds fascinating.
- 12:14SPEAKER_01: People get better answers.
- 12:16SPEAKER_01: Final question.
- 12:17What's one thing about building AI search features that might
- 12:21surprise people
- 12:22SPEAKER_00: outside the industry?
- 12:26SPEAKER_00: Hmm.
- 12:28SPEAKER_00: A lot of our answers come from a cash.
- 12:30SPEAKER_00: I think maybe that's surprising.
- 12:35SPEAKER_00: Cash money.
- 12:36SPEAKER_00: Cash money.
- 12:37SPEAKER_00: Yeah, exactly.
- 12:39SPEAKER_00: Oh, with an E. I see.
- 12:40SPEAKER_00: Cash with an E. Yes.
- 12:41SPEAKER_00: Yes.
- 12:42SPEAKER_00: Yeah.
- 12:43SPEAKER_00: Wu-Tang was right.
- 12:44SPEAKER_00: The cash rules.
- 12:46SPEAKER_01: So in that context, you're not searching for a new answer.
- 12:50SPEAKER_01: It's something someone's already searched for.
- 12:51SPEAKER_00: Yeah, it's something that someone already has searched for.
- 12:55And what we do to make this experience quick is if someone is
- 13:01searching for
- 13:01SPEAKER_00: something,
- 13:02SPEAKER_00: even if they did not request a search assist answer,
- 13:06SPEAKER_00: If we notice it's being searched multiple times,
- 13:09we will proactively generate an answer in the background so
- 13:12that the next person
- 13:13that comes along to search that will have an answer delivered
- 13:16instantaneously.
- 13:16SPEAKER_00: Because when they come from the cache,
- 13:19SPEAKER_00: from the database of existing answers,
- 13:22SPEAKER_00: it's instant on page live.
- 13:25SPEAKER_00: I don't know.
- 13:26That surprised me just how important the role of a database of
- 13:30already generated answers was.
- 13:33SPEAKER_00: Um,
- 13:33that brings a lot of complications with storing those answers
- 13:35and invalidating them
- 13:36SPEAKER_00: at the right time.
- 13:37Um,
- 13:38you know,
- 13:39information changes,
- 13:39um,
- 13:41who is the current president or how old is this person?
- 13:43SPEAKER_00: Like those answers are not good forever.
- 13:45SPEAKER_00: They change.
- 13:46SPEAKER_00: Um, so there's those types of things, but I don't know.
- 13:50SPEAKER_00: Maybe if I had more, a little more time, I can go with a more surprise.
- 13:53SPEAKER_00: Sure.
- 13:53SPEAKER_00: I won't surprise you next time.
- 13:55SPEAKER_01: Uh, thank you, Tim, for joining us.
- 13:58SPEAKER_01: No problem.
- 13:59SPEAKER_01: Have a ducky day.
- 14:01SPEAKER_01: You too.
- 14:02SPEAKER_01: See ya.