Latest / Inside DuckDuckGo / Duck Tales: How we've made it easier to generate images privately in Duck.ai (Ep.33)
Transcript
- 0:03SPEAKER_00: Hi,
- 0:03SPEAKER_00: welcome to DuckTales,
- 0:05SPEAKER_00: where we go behind the scenes at DuckDuckGo and discuss the stories,
- 0:09SPEAKER_00: technology,
- 0:10SPEAKER_00: and people that help build privacy tools for everyone.
- 0:14SPEAKER_00: In each episode,
- 0:15SPEAKER_00: you'll hear from employees about our vision,
- 0:18SPEAKER_00: product updates,
- 0:19SPEAKER_00: engineering,
- 0:20SPEAKER_00: or approach to AI.
- 0:22Today,
- 0:24I'm here with Mate,
- 0:27and we're going to talk about image generation,
- 0:31in DuckDuckAI I will let Matej introduce himself but quickly
- 0:36I'll just say that if
- 0:38you haven't met me yet and Bia I'm on the product team here at
- 0:43DuckDuckGo and yeah
- 0:47Matej take it away tell us a little bit about yourself all
- 0:51right I'm Matej I've
- 0:52been at DuckDuckGo for a year and a half now and I'm working on
- 0:56DuckDuckAI and I
- 0:58SPEAKER_01: work primarily on
- 1:00SPEAKER_01: image generation last year from September until Christmas.
- 1:07SPEAKER_00: Awesome.
- 1:08Well,
- 1:08okay,
- 1:09so just to kind of get started,
- 1:11tell me a little bit about what that even means.
- 1:14SPEAKER_00: What is image generation in DuckDuck.ai?
- 1:16SPEAKER_00: And if you're able, I think a demo would be great.
- 1:22SPEAKER_01: Yep, definitely.
- 1:23SPEAKER_01: Let me share my screen here.
- 1:27SPEAKER_01: All right,
- 1:28so if you go to DuckDuckGo and you will see basically this
- 1:34screen and you probably
- 1:36notice that there's this option new image which is when you
- 1:39click on it then you
- 1:40type any prompt in here DuckDuckGo will try to generate an
- 1:43image for you so let's
- 1:45say show me an image cut on the windows so hopefully at the end
- 1:55of this we'll get
- 1:56SPEAKER_01: an image
- 1:57This is going to take a little while because image generation
- 2:00is quite a heavy process.
- 2:06SPEAKER_01: Yeah, this is the simplest form of it.
- 2:09SPEAKER_00: Okay.
- 2:10SPEAKER_00: Well, I see it.
- 2:12SPEAKER_00: It's coming together.
- 2:13SPEAKER_01: Yes, it's coming together and here's our little cat.
- 2:16SPEAKER_01: We can click on it,
- 2:17SPEAKER_01: zoom on it,
- 2:19download the image and copy it if we need it or we can keep
- 2:23iterating on it and ask
- 2:25SPEAKER_01: for adjustments in the image.
- 2:29SPEAKER_00: It's good.
- 2:29SPEAKER_00: I mean, it has like the right number of limbs.
- 2:32SPEAKER_00: There's nothing too crazy going on here.
- 2:34SPEAKER_00: That's correct.
- 2:35SPEAKER_00: I'm pretty impressed.
- 2:38Do you want to demo like any adjustments or anything else you
- 2:41want to point out
- 2:43SPEAKER_01: while your screen is being shared?
- 2:44SPEAKER_01: Sure.
- 2:45SPEAKER_01: We can try it.
- 2:46SPEAKER_01: Add another cast to the image.
- 2:50SPEAKER_01: Let's see what it comes up with.
- 2:52SPEAKER_01: Okay.
- 2:55SPEAKER_00: Cool, and what model is operating here?
- 3:03SPEAKER_01: So, we're currently using GPT-Image2 model.
- 3:09SPEAKER_01: We've been through multiple iterations of this,
- 3:12SPEAKER_01: and it's always been a trade-off between cost,
- 3:16SPEAKER_01: the output quality,
- 3:17SPEAKER_01: and the performance,
- 3:20SPEAKER_01: so how quick the model is.
- 3:24SPEAKER_01: We went through GPT-Image 1,
- 3:26SPEAKER_01: 1.5,
- 3:26SPEAKER_01: 1.mini and 2 Currently the GPT-Image 2 model is slower However,
- 3:33SPEAKER_01: the output quality is much,
- 3:35much better than previously You probably cannot see it in
- 3:38images like this However,
- 3:40if you would prompt it to generate a text Then you would see a
- 3:44clear difference
- 3:45between different models and GPT-Image 2 Follows instructions
- 3:49really well and
- 3:49SPEAKER_01: generates the text really well
- 3:53SPEAKER_00: Got it.
- 3:53So a bad job if you tell it to put some copy in an image is it
- 3:59doesn't get the letter.
- 4:00SPEAKER_00: It uses different copy.
- 4:03SPEAKER_00: It mangles the letters.
- 4:04SPEAKER_01: Is that right?
- 4:05SPEAKER_01: Exactly.
- 4:06SPEAKER_01: Exactly.
- 4:06SPEAKER_01: That's the case.
- 4:07SPEAKER_00: I've definitely been there.
- 4:09SPEAKER_00: But hey, look, this cat also has the right number of limbs.
- 4:11SPEAKER_00: I don't see anything problematic or fishy about this.
- 4:14SPEAKER_00: Yeah.
- 4:15It's kind of a softball query because there's plenty of CAD
- 4:20images in Windows Online,
- 4:22huh?
- 4:22SPEAKER_01: That's correct, yes.
- 4:25SPEAKER_00: A demo query.
- 4:26SPEAKER_01: Yes.
- 4:28SPEAKER_00: Cool.
- 4:30SPEAKER_00: Okay, awesome.
- 4:31SPEAKER_00: Well,
- 4:31SPEAKER_00: tell me a little bit,
- 4:32SPEAKER_00: like stepping back again,
- 4:34SPEAKER_00: tell me a little bit about why we even built Image Generation.
- 4:39SPEAKER_01: So I think from the outset we knew that we will have to at some point.
- 4:43SPEAKER_01: The reason for that is
- 4:45Most of the competitors or similar products already provide
- 4:48this feature.
- 4:50SPEAKER_01: We knew that we had the capability,
- 4:52but also we had some user feedback that this is a highly
- 4:55requested feature.
- 4:56SPEAKER_01: It doesn't match.
- 4:59SPEAKER_01: At this point in time,
- 5:00this is like a baseline feature that is expected by all the
- 5:03users for you to have
- 5:06SPEAKER_01: in any sort of AI product,
- 5:09SPEAKER_01: I would say.
- 5:10SPEAKER_00: Yeah, yeah, got it.
- 5:12SPEAKER_00: And,
- 5:13SPEAKER_00: all right,
- 5:15SPEAKER_00: so what were,
- 5:16SPEAKER_00: like,
- 5:18SPEAKER_00: did you run into any challenges when you were building this feature?
- 5:23SPEAKER_01: So I think challenges were not technical.
- 5:27Surprisingly,
- 5:29most of the challenges would come out of,
- 5:30for example,
- 5:31content moderation.
- 5:34SPEAKER_01: How do we prevent people from generating
- 5:39SPEAKER_01: For example, self-harm images.
- 5:43And then the second part that I've already mentioned is kind of
- 5:46how do we balance
- 5:47SPEAKER_01: the quality performance and cost at the same time.
- 5:51So those were the two challenges that I would say were that
- 5:56come to my mind as we
- 5:58SPEAKER_01: were building this.
- 6:00SPEAKER_00: Yeah.
- 6:01SPEAKER_00: Gotcha.
- 6:01SPEAKER_00: And like, what was the, what was the runtime on this work?
- 6:05SPEAKER_00: Like how long have you been working on this?
- 6:08So I started working on initial prototype in September last
- 6:12year and we shipped it
- 6:15SPEAKER_01: to production just before Christmas.
- 6:17SPEAKER_00: Got it.
- 6:18SPEAKER_00: And was it mostly you or was there a team of people working on it?
- 6:21In the beginning it was mostly me and then another engineer
- 6:24joined me for UI work.
- 6:27So I did mostly the back end and then a little bit of wiring on
- 6:31the front end.
- 6:33SPEAKER_01: specifically for like,
- 6:35you can also upload your own image and then ask the model to
- 6:38make adjustments to
- 6:39SPEAKER_01: it.
- 6:39SPEAKER_01: So this is the part that I was working on.
- 6:42SPEAKER_00: Yeah.
- 6:45Have we gotten any interesting or surprising feedback since
- 6:49it's been live?
- 6:51SPEAKER_01: So I wouldn't say any interesting,
- 6:54but what was surprising to me was that the reception was quite
- 6:57positive and I was a
- 6:58SPEAKER_01: bit worried when I built this feature because in general,
- 7:02SPEAKER_01: Image generation is perceived as a fairly harmful thing.
- 7:07SPEAKER_01: Depending on your intent, you can also do harmful things.
- 7:11SPEAKER_01: However,
- 7:12I was surprised that most of the people were positive and they
- 7:15appreciated to have
- 7:16SPEAKER_01: a privacy-focused way of generating images.
- 7:21Sounds like the hard work that you put into putting the right
- 7:25content generation
- 7:26SPEAKER_00: guardrails on it and so forth must have paid off.
- 7:31SPEAKER_01: I hope so, certainly.
- 7:33SPEAKER_00: Nice.
- 7:34So going forward,
- 7:36are there improvements that you guys are making on this
- 7:39feature?
- 7:40SPEAKER_01: So we've already made a bunch of improvements.
- 7:43This current iteration that you saw in the demo is an evolution
- 7:49of original implementation.
- 7:51SPEAKER_01: Previously, we had a separate mode just for image generation.
- 7:55SPEAKER_01: It was not integrated into your normal conversations.
- 7:59And this is one feedback that we've received and we pivoted
- 8:03away from the separate
- 8:04mode into having it inline in your regular chats Another piece
- 8:13of feedback we've
- 8:13SPEAKER_01: heard,
- 8:14SPEAKER_01: and I already mentioned this,
- 8:15we've gone through multiple models because people were asking
- 8:18for better output
- 8:22SPEAKER_01: Going forward,
- 8:23SPEAKER_01: I don't think we have anything in particular,
- 8:24SPEAKER_01: so if you have any feedback,
- 8:26SPEAKER_01: I'm happy to
- 8:28Read through it and put something on the roadmap There are many
- 8:32ways we could take
- 8:32this I can imagine we can add a freeform canvas and you can ask
- 8:41a model to make
- 8:41your image pretty Like you draw something with your free hand
- 8:44and you'll get a nice
- 8:45SPEAKER_01: image Or advanced editing,
- 8:48you mask out something and you ask a model to replace that bit
- 8:51of an image with
- 8:52SPEAKER_01: something else So there are many,
- 8:54SPEAKER_01: many different ways we can take it
- 8:57SPEAKER_00: Yeah.
- 8:59Going back to a second for the whole,
- 9:01like,
- 9:01to the inline versus a separate mode thing.
- 9:04SPEAKER_00: So that's,
- 9:06SPEAKER_00: like,
- 9:06SPEAKER_00: just in case it's not obvious that,
- 9:08SPEAKER_00: like,
- 9:09SPEAKER_00: it used to be that you had to start your chat in image mode.
- 9:13SPEAKER_00: And if you started with any other model or,
- 9:15SPEAKER_00: you know,
- 9:15SPEAKER_00: you were mid-chat,
- 9:16SPEAKER_00: you couldn't,
- 9:17SPEAKER_00: like,
- 9:18SPEAKER_00: jump into image generation,
- 9:19SPEAKER_00: right?
- 9:19So now if I'm,
- 9:20like,
- 9:20mid-chat on something,
- 9:22can I just ask it to generate an image and...
- 9:27SPEAKER_00: Like it's properly, it can jump between those modalities?
- 9:31SPEAKER_01: That's correct.
- 9:32SPEAKER_00: Yes.
- 9:33SPEAKER_00: Nice.
- 9:34SPEAKER_00: And then how does it,
- 9:35SPEAKER_00: so since we're using one model for image generation,
- 9:38SPEAKER_00: if I start chatting again,
- 9:40SPEAKER_00: how does the,
- 9:42SPEAKER_00: how does the,
- 9:42SPEAKER_00: does this just go back to the previous model that I was chatting with?
- 9:46SPEAKER_01: Yes, that's correct.
- 9:47So if you're in a conversation,
- 9:49there's in the background,
- 9:50there's essentially two models.
- 9:52SPEAKER_01: One is for your conversation, which will be GPT-5 mini or
- 9:58SPEAKER_01: Opus.
- 9:59And then there's another model that's specifically for image
- 10:01generation that's
- 10:02SPEAKER_01: always being called whenever you ask for an image,
- 10:04SPEAKER_01: which is GPTD Image2.
- 10:09SPEAKER_00: Got it.
- 10:09SPEAKER_00: And I can just jump back and forth willy-nilly.
- 10:11SPEAKER_00: Yeah.
- 10:11SPEAKER_00: Correct.
- 10:12SPEAKER_00: My image or how.
- 10:13SPEAKER_01: Okay.
- 10:14SPEAKER_01: Yes.
- 10:15SPEAKER_00: Gotcha.
- 10:19SPEAKER_00: And then how do we know whether, like, how does the model know?
- 10:22SPEAKER_00: Not how do we know, but how does DuckAI know whether
- 10:25to be,
- 10:27you know,
- 10:28interpreting my prompt as an image generation command or
- 10:33continue discussion?
- 10:35SPEAKER_01: That's an excellent question.
- 10:36SPEAKER_01: So we have image generation tool on our back end.
- 10:40SPEAKER_01: And basically it analyzes all the prompts that are coming in.
- 10:45SPEAKER_01: And if any of those is categorized as one that is requesting an image,
- 10:51Then it will trigger the tool and that tool will generate an
- 10:54image for you.
- 10:56SPEAKER_00: How good are we at that?
- 10:58SPEAKER_00: Are we perfect at classifying?
- 11:00Or do you think there are times right now when somebody wants
- 11:03to be editing the
- 11:03SPEAKER_00: image and they get a text response or vice versa?
- 11:07SPEAKER_01: I don't think we're perfect, but I think we're pretty good, I would say.
- 11:11SPEAKER_01: I would say about 90%.
- 11:15SPEAKER_01: We have a set of evaluations that we
- 11:18run on these tools and on various prompts and we keep adding to
- 11:22the data set that we have.
- 11:24SPEAKER_01: So we keep improving this and it's already pretty good.
- 11:29SPEAKER_00: Nice.
- 11:31SPEAKER_00: Okay.
- 11:32SPEAKER_00: One more question for you.
- 11:33SPEAKER_00: What is your favorite thing about image generation in DuckDuck.ai?
- 11:39SPEAKER_01: My favorite thing?
- 11:40I think like I had a good time with my wife when we were just
- 11:45goofing around.
- 11:46SPEAKER_01: We were generating
- 11:48SPEAKER_01: I think we generated like a demon llama in a coat or something.
- 11:51SPEAKER_01: I don't know,
- 11:51SPEAKER_01: like things that you would never think about,
- 11:53but it was just funny to keep working on the same image and
- 11:57what it would come up
- 11:58SPEAKER_01: with.
- 11:59SPEAKER_01: Another use case that I really love is generating logos.
- 12:02If you have an event you want to put on your,
- 12:05like a logo on your invite,
- 12:07then it's pretty easy.
- 12:10SPEAKER_01: It generates you and generally good enough results for you to use.
- 12:15Yeah,
- 12:15I actually had a good logo experience with DuckAI not too long
- 12:20ago.
- 12:21SPEAKER_00: I was talking to a friend,
- 12:25a child of a friend who with my daughter had made this like
- 12:32amazing like role
- 12:34SPEAKER_00: playing card game.
- 12:37SPEAKER_00: and named it and they wanted to create a logo for it.
- 12:43SPEAKER_00: And I was like, oh, let's just see how it does.
- 12:45SPEAKER_00: And I just on my phone,
- 12:46we were like literally on a hike and I just picked up my phone
- 12:49and told DuckAI what
- 12:51SPEAKER_00: the game was called and a few,
- 12:56SPEAKER_00: you know,
- 12:56SPEAKER_00: suggestions for what this
- 12:59person had in mind from the logo and like what came out was
- 13:02pretty great.
- 13:03SPEAKER_00: I was, I was impressed.
- 13:04SPEAKER_00: It was like a,
- 13:04SPEAKER_00: it was neat that you could do like that,
- 13:07SPEAKER_00: you know,
- 13:07literally like while hiking through a park somewhere in a
- 13:10matter of moments.
- 13:12SPEAKER_00: I think he was impressed too.
- 13:16SPEAKER_00: Thanks.
- 13:17SPEAKER_00: All right.
- 13:18SPEAKER_00: Well, anything else that you want to add, Matei, before we close?
- 13:23SPEAKER_01: Um,
- 13:24SPEAKER_01: Try using this feature.
- 13:27SPEAKER_01: Give us your feedback so that we can improve it.
- 13:29SPEAKER_01: We can keep working on it.
- 13:31SPEAKER_00: Yeah, for sure.
- 13:32SPEAKER_00: We actually read that, the feedback.
- 13:36SPEAKER_01: Yes.
- 13:36SPEAKER_00: All right.
- 13:37SPEAKER_00: Well, thanks for joining me today and thanks everyone for listening.
- 13:41SPEAKER_00: See you around.
- 13:42SPEAKER_01: Thanks for having me.
- 13:43SPEAKER_01: See ya.