Latest / Agentic Conversations (formally mlops.community) / The Future of Search in the Era of Large Language Models // Saahil Jain // MLOps Podcast #150

Transcript

The Future of Search in the Era of Large Language Models // Saahil Jain // MLOps Podcast #150 · Agentic Conversations (formally mlops.community)

0:00
So my name is sahil Jane, I'm an engineer at you.com and I
0:05
usually go for a mocha Frappuccino and Starbucks.
0:11
What's happening everybody? I am Demetrios your host and
0:14
today I have the immense pleasure of being joined by mi
0:20
mejor amigo. David Aponte, what's up, dude?
0:23
What up? Hello, everyone been a while
0:26
since we've been on here and we're going to be discussing
0:28
some really interesting questions.
0:30
Questions. What's going on with the search
0:31
landscape? How is it changing?
0:33
How is generate a generative AI affecting that landscape?
0:36
There is a lot of competition from the big companies to now
0:39
smaller companies such as u.com and we got a little preview into
0:43
some of the things that they're focusing on the challenges that
0:46
they're facing and how they're thinking about addressing it.
0:48
Also what they want to do to differentiate themselves and and
0:52
you know, make a name for themselves and we talked a lot
0:54
about search and and yes, what changes are happening in this
0:58
space. Then I There's a lot to say for
1:01
you.com and a lot of great stuff coming and so, yeah, I'm excited
1:05
to get into this dude. So, who is sahil?
1:09
Like, what is sake with Jane? He is an engineer at u.com, has
1:14
some experience at Stanford. We worked at sale, the Stanford
1:18
AI lab right now. Is thinking a lot about product
1:22
and I think that's very much in line with where a lot of people
1:26
in the space. Alright.
1:27
With the tech is there, but now how do we make A great product
1:30
out of it and you hear that all throughout the conversation, his
1:34
intuition on this stuff. Great, great, great, great
1:37
podcast, excited. Love how he really did, like he
1:41
has product in his blood. He thinks about the user
1:44
experience, any things, how can we make search different than it
1:47
is now and better than it is now?
1:49
And so he talked about that and he was able to go from this very
1:54
technical level of all right? We are thinking about you know,
1:58
like the Relevance, I love the trifecta that he said and you
2:03
really have to look at these trade-offs between what was it
2:06
relatively relevance and latency latency.
2:09
Especially in their case, where they're working a lot with large
2:12
language models, what's the throughput relevance and
2:15
latency? This was probably for me.
2:17
The greatest highlight was like the actual time or the objective
2:21
time and the subjective time that you have to wait for
2:24
something. And so the the real Jedi mind
2:28
trick that he talked about, Is that if you do have to wait, for
2:31
something for a long time? Can you give the user something
2:35
to do in the interim? Whether that's filling out a
2:37
survey or telling them something funny, maybe they a joke pops up
2:41
or something that occupies the user's attention.
2:47
So they don't feel like, wow, this is taking forever.
2:52
Yeah, and also we discussed a little bit about the
2:54
intersection of traditional classic information and
2:57
treatable and the transition to your modern day.
3:00
Large language, models are gendered remodels, and trying to
3:03
bridge the gap because there's a lot, right?
3:04
He discuss how information retrieval has its works.
3:07
It works really well in certain cases.
3:09
But now we have this new technology that is also really
3:12
gray and also discussing the outputs and thinking about how
3:17
there's some trade-offs in the type of outputs, right?
3:19
Or the, as your first send the generations.
3:22
What Generations you produce affects how the user will you
3:26
know engage with your website stay on it and do other things.
3:29
And so there's lots of interesting things to think
3:32
about outside of just, you know, performance.
3:35
Yep, dude. Well, I'm glad that you came on
3:37
here and you were able to do it like old times.
3:41
I know you're a fan of you and so it was only right to have you
3:45
get on here and interview. Yeah.
3:48
Well hopefully you guys enjoy this episode and for more
3:50
information, why Youyou it instead of Googling your it
3:54
sounds kind of funny. When I say like you you're a fan
3:57
of you tell you, I am I guess you.com, I'm a Fed.
4:03
There we go. All right, let's get into it.
4:10
Welcome everyone to the Apple apps podcast.
4:13
We are very excited to get today.
4:15
We have a very special guest from you.com, you know, the
4:18
search landscape is changing really quickly.
4:20
And there's so many advances happening especially in the NLP
4:23
space in generator edit about generative AI.
4:26
Let's just talk a little bit about your take on how the space
4:29
is changing and what are you guys doing to keep up with it or
4:32
compete rather? Yeah, yeah.
4:34
First of all. Thanks for having me and having
4:36
us on we really, really appreciate it.
4:39
Yeah, I think as you mentioned, Search right now is an extremely
4:42
Dynamic and interesting space. There's there's so much
4:45
happening and there's been a lot of advances in the last couple
4:48
of years. Like 55 by this years that have
4:50
really opened up a lot of new opportunities.
4:53
So I think one thing will see it, one of the biggest changes
4:56
and there's been a couple would be no we operated in a paradigm
4:59
of kind of you know, classical information retrieval.
5:03
So if we kind of look and I'll just kind of describe a couple
5:05
Transformations that have happened in search, you know
5:08
initially, you know what we think about and the Academic
5:11
discipline most language. Search, it's kind of this topic
5:13
or a discipline known as information retrieval and I'll
5:16
quickly just Define what that means.
5:18
So when we think about what information retrieval is in
5:20
search generally, it's really kind of characterized by a
5:23
couple properties one of which is it's often looking for, you
5:27
know, specific documents among kind of a large set of data.
5:30
So, usually you're searching across a large Corpus of
5:34
information. Search is usually a side like
5:36
you looking for the crest 10 documentary, 15 documents really
5:39
looking across a much. One constraint, set of documents
5:42
and often. These documents are very
5:43
unstructured in nature. And that also, you have query.
5:46
That's also kind of unstructured.
5:48
So this is usually like the Paradigm in which search
5:50
operates. So users are typing in anything,
5:53
you know, or asking a question and they expect to receive some
5:56
type of answer or licks. So classical information
5:59
retrieval is, you know, kind of primarily based on keywords on
6:03
the idea here. Being that, you know, you look
6:04
up a query and then you get kind of, you know, keywords and then,
6:08
you know, if you look at kind of historical search engines,
6:10
You'll get Blue Lakes. Essentially we get like a sea of
6:13
ten Blue Links, then you click on them.
6:15
And, you know, some of those links have pages that have
6:17
matches on this keywords and then we kind of started moving
6:20
more towards Neola formation retrieval.
6:22
So we started taking advantage of all the advances in NLP and
6:24
beddings. For example, Bert that came out
6:27
which was you know, I think around the 2017 this but then we
6:31
started out being able to embed documents very well and start
6:34
using, you know, techniques like semantic search to better
6:37
understand and match the meeting of a query to the meeting of the
6:40
talk. And that was, I think I did the
6:42
second changes shape that happened and this kind of had
6:44
multiplications on search one of which was in the, you know, the
6:48
retrieval stage but also on the re-ranking stage and then I
6:51
think now what we're seeing is even the next stage which is
6:53
kind of what I would think about more as you know, giving you the
6:56
direct answer. So when you search for something
6:59
you'd only just want lakes but you often want the exact answer
7:02
service to you. And I think chatty BT which is
7:04
probably the most notable consumer application that
7:06
started doing this in a very new way.
7:08
Has really kind of struck a chord with people.
7:10
And I think at you.com, we've also been thinking very deeply
7:13
about this and that's kind of what you chatted you.com is
7:15
about. Is, you know, with the
7:16
conversational AI, we can actually very quickly surface,
7:20
the answer for you and try to provide citations and kind of
7:22
keep improving in that role. It's such a better.
7:25
Yeah, yeah, yeah. And I think just to kind of
7:28
piggyback off the last thing where now we're getting direct
7:30
answers. I think that's also really
7:32
exciting for people because I feel like they can just ask
7:35
anything and get any answer. Help me with this.
7:37
Help me with that. You know.
7:39
Historically you had to kind of figure.
7:40
Figure out how to write that, right?
7:42
You had to really do some engineering to hook.
7:44
So what question you put into Google search to make sure that
7:46
you would get the right thing. And even then there's still ways
7:49
to mess up and get things that are not exactly what you're
7:52
looking for. So to be able to express exactly
7:55
what you want and get some sort of answer like you're talking to
7:57
a person. I think that is very different
7:59
from the way that we've typically interacted with these
8:02
traditional like information retrieval systems.
8:04
Right? Where you have to kind of their
8:06
very specific to the type of information that they store.
8:10
Yeah, exactly. And I just feel like it's such a
8:13
better experience that you don't have to go through different
8:17
links and be spammed with so many ads.
8:21
If we are just trying to get some kind of recipe on how to
8:23
make a vegan black bean burrito, right?
8:26
And you don't have to be hit with a million popups on how to
8:30
subscribe or please subscribe to my newsletter or this and that
8:34
and it just makes things a lot nicer.
8:36
So I see the user experience going.
8:39
That way for a lot of things I still feel like there is more of
8:44
like it's not all going to be like that, right?
8:47
And I don't know about you maybe you feel differently but I still
8:50
feel like there are plenty of use cases that you're not going
8:54
to just want an answer. You're going to want to explore
8:57
and get a bunch of different viewpoints on something before
9:00
you make your decision. Yeah.
9:03
Yeah I think that's a great point and I actually I totally
9:05
agree with you. I think that the key is to
9:07
really think about how to combine the two.
9:09
So we think about generate a i in advances and the ability to
9:13
provide kind of quick answers to queries or chats.
9:17
We also have to think about trustworthiness allowing shirt
9:19
making sure that people can ground the information, do their
9:22
own research, kind of the way we were thinking about it is we
9:25
recently actually yesterday had to launch.
9:27
What about you.com 2.0 and in you.com 2.0, we've essentially
9:31
introduced the idea of Kind of Cal, which is chat apps, and
9:34
links. So when you do a chat, you'll
9:37
basically, you know, get a chat response which will be you know,
9:41
fluid will somewhat you know, use a lot of these advances and
9:44
have citations as well. You'll be able to access the
9:46
links as well and you'll also be able to access you.com apps
9:49
which are essentially you can think of them as specific
9:52
information retrieval search constructions over, you know,
9:56
particular domains. So, for example, let's take a
9:58
program in query. If you look up something like
10:00
Roberta git, commit, you'll be able to see kind of Love, you
10:02
know, a generation and there might be some citations in that
10:05
generation. But, you know, the in general,
10:08
that generation will be kind of similar to what you might expect
10:11
from some of these other LMS, you'll also be able to see
10:14
links. So if you want to kind of go in,
10:16
read more, you can do that. And you'll also see a stack
10:18
Overflow app, which will allow you to kind of open the side
10:21
panel quickly, dig into a bunch of your content.
10:24
So that's kind of the way we're going about it.
10:26
I think it's very similar to kind of what you said is that,
10:28
you know, we want to be able to make sure that people are able
10:31
to ground themselves. And, you know, get the best out
10:34
of the tools that they use. I think it's a great idea
10:37
because it gives you like sometimes, maybe a direct
10:39
answer, you can click on places, you can go to look for further
10:42
information to learn. I think that is a really good
10:44
point that the interface matters, right?
10:46
Like, you know, I'm curious, you know, right now most people
10:48
think of just like that one little, you know, a tab where
10:51
you start typing your search. How do you think about
10:54
differentiating, what to give to users?
10:56
Because I'm sure there's lots of ways to figure out exactly what
10:59
they want. Do they want to direct answer
11:00
to? They want a lot of information.
11:02
How do You think about the trade-offs, they're giving them
11:04
too much or giving them die. Guess just enough which is very
11:07
similar to what you're talking about with I guess you.com 2.0.
11:11
Yeah, that's a great question. That's something we're actually,
11:13
you know, grappling with right now is you know how much
11:16
information do you want to give the user without overwhelming
11:18
them? I think historically.
11:19
We've also we always kind of been threading, the line between
11:22
you know giving too much information.
11:24
Then kind of sometimes getting too little.
11:26
So something I think we're still learning and we definitely at
11:28
least I don't have the answer for that but I think that's fair
11:31
enough but I think in General. Like, the the key is to kind of
11:34
be a bit nuanced and I do, you know, basically understand when
11:38
it makes sense to give them more information.
11:40
I want to make sense, not to, I think for certain queries, like
11:42
for example, when was Obama born in that case, the person is
11:46
probably looking for a year and maybe a citation.
11:47
Yeah. Don't need an entire me, tell
11:50
you a story about Obama's life but in some case, study looking
11:54
who is Barack Obama. You probably want more sources
11:57
that you can kind of dig into and, you know, richer set of
12:00
information. So I think it all depends.
12:02
Lil Bit on the user intent behind the query.
12:04
And that's something that, you know, is, is imported to think
12:07
about. So, it would search.
12:08
It just, there's just so many different types of use cases
12:11
that with, when it comes to Surge that it's almost harder
12:14
bucket, you know, answers for for, you know, one domain when
12:17
they're in the wind and one intent essentially, when there's
12:19
so many intense being served, another really interesting thing
12:22
that's happening. Now, that I think we'll start
12:24
seeing more search engines. Do is like, we think about Surge
12:27
and our search engine is a do engine.
12:29
The idea being that, you know, you're not only using it to
12:31
retrieve information but the You and take action.
12:34
And I think what we'll see is that, you know, a lot of search
12:36
engines, start moving in this direction as well.
12:40
And the idea, I'll give you an example, is, you can type in
12:42
something like, you know, generate an image and you'll get
12:44
served an app that, you know, you can use an existing, you
12:47
know, image generation tools, nice and kind of yourself.
12:50
So historically that hasn't really been the role of search
12:52
engines. But now that we're seeing all
12:54
these advances in generative AI, we're going to see, you know,
12:57
search engines kind of open up and allow you to really do
12:59
things that you wouldn't have been able to do before, which I
13:02
think is wrong. So, yeah, it's it's just going
13:06
to that next step. So you don't have to click
13:10
through something and you can just get what you want, done,
13:14
unless clicks and less steps and that makes complete sense to me.
13:19
I like that again. It goes back to that user
13:21
experience. Like, if I want to do something,
13:24
I want to do that thing as fast as possible.
13:26
Because, you know, some kind of notification might come up and
13:30
give me distracted. And then all of a sudden, I
13:32
forget You know what I was doing in the first place and I try not
13:36
to let that happen but it gets the best of me sometimes.
13:39
But so when you're talking about that, I mean there's some really
13:45
key challenges when it comes to search and you mapped out a few
13:49
of those, what are some other ones that you are constantly
13:52
thinking about? Yeah there's definitely a lot of
13:56
different challenges when it comes to search and I think I
13:59
can kind of go through some some you know, different challenges
14:02
and this is going to apply to you know, anybody building a
14:04
search engine and I think a lot of problems can be reframed as
14:07
search problems. So hopefully it's useful to
14:09
people who are not even working on surged.
14:11
We're working on other, you know, at other companies where
14:13
you know, they're working on search like problems.
14:15
I think one, you know challenge is relevance.
14:18
So in general, you know you want to be able to provide relevant
14:21
information for your user but relevance Is often subjective.
14:24
So, you know, you know, they're so what that means is that when
14:27
you're training machine Learning Systems, you need, you know,
14:29
ways of being able to annotate data, make sure that it's
14:32
aligned with human feedback essentially.
14:35
So the idea of relevance is kind of a one that's important and I
14:38
think it's very tight to evaluation.
14:40
So having very strong sets, evaluation of a structure is
14:42
really important to be able to kind of ensure that you can
14:44
provide highly relevant content as well as understanding user
14:48
feedback and being able to keep iterating on that getting a
14:51
second challenge. In general is latency.
14:53
Especially with a lot of these large language models, you know,
14:56
they can be quite slow and we need to think of ways in order
14:58
to you know reduce latency but also make the experience such
15:02
that you know, higher latency doesn't feel slower.
15:06
This could be you know, providing surveys of content.
15:08
Well other content is loading and we're kind of thinking a lot
15:10
about and make the user experience such that even if
15:13
certain responses are highly can see just because we're using,
15:16
you know, very powerful models in the background.
15:18
The user still is able to benefit and not really feel the
15:21
negative impact of that latency. So there's different creative
15:24
ways around it. I think the third challenge is
15:26
really about, you know, there's like, you know, challenges
15:29
around throughput and cost so thinking a lot about how to
15:32
optimize among those Dimensions user interface challenges.
15:36
So, you know what is the best user interface?
15:38
That's something we're thinking about.
15:39
We've now kind of, you know, moved more towards
15:42
conversationally. I had a conversational interface
15:44
and I think that's something that we've seen to have
15:46
resonated with users, but I do you going on, so I'll videos.
15:49
Yeah, yeah. Let's, let's start with that.
15:51
First run, talking about like relevant, And how subjective it
15:54
is something that came to mind. Was I lots of lots of ml
15:57
problems can be framed as a search or like a ranking problem
16:01
of sorts, right? Yeah and and there I, you know,
16:04
there are relevance depends on the domain.
16:06
I was thinking when I was working at benevolent AI where
16:08
we thought about biological relevance that while we were
16:10
searching for had some meaning to a drug discover, how do you
16:15
balance that? Because there's so many, you
16:17
know, ways to consider relevance to a user, is it collecting
16:21
additional Telemetry of like them?
16:23
Re-engaging with the application.
16:24
Like maybe that's a bit too deep into it but it seems like
16:28
relevance is a really tough thing to address.
16:31
What is you.com to doing to take care of that.
16:35
Yeah and that's that that's a good question.
16:36
It's constantly evolving the way we think about relevance but in
16:40
general so I think you're right. I think you mentioned.
16:43
One thing is about Telemetry so we we are also a private search
16:47
engines. So in our private mode, we
16:49
actually don't use any of this information for machine
16:51
learning. We do have a personal mode,
16:53
Where you can have user preferences and we do allow
16:55
people to write feedback. So for example, you can thumbs
16:58
up and thumbs down on the generation or a chat response
17:02
rather as well as apps that we provide.
17:04
So we have all this feedback that coming that's going from
17:07
users, that users are volunteering and giving us, as
17:10
well, as you know, personal preferences.
17:11
So some users, you know, sir like certain sources, more than
17:14
others and we try to allow them to have their preferences
17:16
reflected in their search results.
17:17
So relevance can also differ based on the individual, but in
17:21
general, I think it goes down into kind of two categories.
17:23
Cory's one of which I would think about as like, you know, I
17:27
guess preference, tripping feedback, so preference for the
17:29
relevance. This is basically when users
17:31
have preferences, how well, or be able to adapt their purposes
17:35
and do it properly. So, obviously filter bubbles,
17:37
Etc are also issues. So we don't want to, we want to
17:40
be mindful that as well. That's one bucket.
17:42
The second bucket would be, you know, thinking about what I
17:45
would call offline relevance. So, can we construct strong, you
17:49
know, death, so that's it. Evaluations ads in order to
17:51
Benchmark our systems. One example of this is we
17:54
actually partnered with a company called surge Ai and over
17:57
there we we ran a study and if you compare to you code which
18:00
which is our search under for developers to Google and we able
18:03
to find that on a lot of queries, were able to outperform
18:06
Google along different dimensions and we had users
18:09
basically, you know, rate which responds to down says more
18:12
satisfactory, which one help them save time and we found out.
18:15
We're able to save time with our apps of the way we structure our
18:18
content and our relevant essentially.
18:20
So that's kind of one ideas like offline data sets.
18:23
Running studies, you know, feedback, Etc.
18:27
Also, you know, the idea weekly supervised learning is important
18:29
here, can we cheaply and, you know, generate heuristics for
18:33
labels and then use those to Benchmark systems.
18:36
That's also a whole nother topic and then I think the third won't
18:38
be online. Feedbacks?
18:39
Just things like, you know, click signals, Etc.
18:42
Again, we're very focused on, you know, privacy.
18:44
So we do this in a way that, you know, best aligns with with
18:47
that. But, you know, understanding and
18:49
responding to the way users are using our product is Also
18:53
important. So I think those are just a
18:54
couple ways in which we think about relevance and a lot of it
18:57
comes down to the data. So I think that's something
19:00
that, you know, we talked about machine Learning Systems, you
19:02
can't not talk about the underlying data.
19:04
So we focus very much on that. So true.
19:11
I'm Emmanuel Mason machine learning engineer, at stripe and
19:15
author of building, machine learning, powered applications.
19:18
And if you don't want your machine learning models to
19:20
explode, well, you should subscribe to this podcast, dude,
19:29
there's one thing that I always think about and you tend to see
19:34
search and recommender systems and especially when it comes to
19:36
mlr ABS they're paired together. Or they are like brother and
19:40
sister type thing because of a lot of these things that you're
19:45
saying, like you can do surge and it's not that far of a
19:50
stretch to call it a recommender system.
19:53
And so what I am thinking about when you are talking about this
19:59
because you're using large language models very much for
20:03
search and recommending things to people a lot of recommender
20:08
systems. Are powered by feature stores.
20:11
When it is like, what I would consider the traditional way of
20:16
doing it, but you're doing it, it's like a whole new paradigm
20:20
of trying to do it. How do you look at those two and
20:24
how do you like, Reckoning style the way that you're doing it
20:27
with large language models compared to this way of doing
20:31
it, with a feature store and trying to just serve up?
20:35
Like if you read any Netflix blog in the Five years, how they
20:40
talk about it? Yeah, so if I understand your
20:43
question correctly, you're talking about, you know, there's
20:45
traditional recommender systems, which, you know, you kind of
20:48
generate features, some of these features may be, you know,
20:51
cough, is it features of a bunch of different ranking factors and
20:54
some of them, exactly, you know, just very simple stuff based on
20:58
user data and then you train models mapping these features to
21:02
a certain action versus now, with the Lord's language models,
21:06
where, you know, you provide prompts And you get output is
21:10
that kind of your question trying to understand.
21:11
Exactly, exactly. You I couldn't have said it
21:14
better myself. I'm glad next time.
21:16
I don't even need to come. You just pissed.
21:18
Interview yourself and then we'll be all good.
21:23
Yeah, it's a good question, and I think there is a place for
21:26
both, so I think I don't think it's an either/or.
21:30
I think in some ways there's different use cases for both.
21:33
So, for example, for recommender systems, you know, it, maybe, I
21:38
mean, we'll get to a state where a lot of these models are going
21:40
to be able to do that as well. But oftentimes you don't
21:43
necessarily need or want some of the randomness associated with a
21:47
large language model, when a very simple model, will have
21:49
very high fidelity. So I think that's one thing to
21:51
think about is the use case. So there is definitely a role
21:54
for both of those systems you know in my opinion far into the
21:58
future especially depending on the task and kind of the
22:01
guarantees you need. So for example in a taste like
22:04
healthcare I think we're much more unlikely to see you know,
22:09
large language models, governing Health interactions compared to
22:11
more of the traditional systems at least for the next two to
22:14
five years. I would say that.
22:16
That's one thing to think about is there's a place for both
22:18
depending on the use case and then I think the other aspect is
22:22
you're right, it is Very different philosophy when it
22:24
comes to both but I think both share kind of some of the same
22:28
principles and I think the same principle is around the
22:30
evaluation. So in both cases, you still need
22:32
to be able to evaluate the the models output.
22:37
So I think once you do that, then it becomes a question of
22:40
experimenting and the methods are different in both cases, but
22:43
the, you know, the overall way in which you evaluate them
22:45
should be, still be very standardized and similar.
22:48
So in some cases it is different but there are a lot of
22:50
similarities that we can carry over.
22:52
From ml Ops practices in the past, especially related to
22:55
evaluation metrics Benchmark, getting those types of things.
22:59
So there's a lot of carryover. Yeah.
23:02
And and sometimes even combine them to write.
23:04
Like I've seen treating system designs where there's like a
23:06
funnel of sorts, right? Where use a larger model more
23:08
powerful to provide you more accurate results and then
23:10
something smaller on top of that, to maybe, you know, reduce
23:13
it to like the top 10 or something like that or vice
23:15
versa. So, it seems like you have to
23:18
have a pretty involved infrastructure to support all
23:21
this to address these needs. You mention a couple of the
23:24
challenges with latency, right? Because at inference time,
23:27
traditionally many of these large language models are
23:29
cereal, right? You cannot, they're generating
23:31
tokens one at a time or one character want to know.
23:34
And so there there's a lot of research trying to find more
23:38
efficient ways to do that. Let's talk a little bit about
23:41
that if whatever you can discuss, what is you.com doing
23:44
to address some of these latency issues.
23:47
Yeah, so latency is something we're very focused on and it
23:51
said, it's a fascinating subject and I think as I mentioned
23:53
latency is actually there's this there's different components on
23:56
it comes to latency part of it is there's often I think I kind
24:00
of joke in some ways. I don't know if this is actually
24:02
the name of a real metric but I also thought I think it does
24:04
sometimes the actual agency which is you know time to you
24:09
know user to all the content they see and then perceived
24:11
latency which is what is the user field only to.
24:14
Yeah, that's sweet. Yeah, when it comes to search,
24:18
there's definitely, you know, sometimes differences between
24:20
the two which are very fascinating but I think when it
24:24
comes to, you know, actually and see which I think I'll just
24:25
group in the former bucket. There's definitely a lot of, you
24:29
know, things that, you know, we're working on part of it is
24:32
and I think so, I've actually given a talk.
24:35
If folks are interested with one of my co-workers Sam and we
24:37
talked a little bit about how we think about indexing and you
24:40
know, we talked a little bit about some of the steps we've
24:42
done, they're using spark Etc to you know you know efficiently.
24:47
Action and embed things for semantic search and buy things
24:50
any documents, or semantic search.
24:52
That's more of the indexing side, not the infant side, but
24:54
there it is latency that respect to.
24:56
And then on the search side, you know, part of it is having, you
24:59
know, you know, good infrastructure being able to,
25:02
you know, make sure that you're using the right amount of
25:05
resources in a way that's cost-efficient and I don't think
25:07
is any silver bullets, though. I don't really have any.
25:09
Yeah, exactly. And usually, it's a combination,
25:12
right? Like compression.
25:13
Some people do Network pruning. There's all knowledge.
25:17
As distillation like there's all sorts of tricks that people are
25:20
using and it seems like most companies using a mixture of the
25:23
two. Yeah, Gary Fisher has supported
25:25
to like using smaller one. The best Rider that I mind trick
25:28
in my eyes is what you were talking about when it comes to
25:32
having, somebody fill out a survey or do something while
25:35
they're waiting because then the perceived latency goes out the
25:39
window, right? It can be as long as it needs to
25:42
be, but if I'm preoccupied while I'm waiting, then I don't even
25:45
recognize that I'm actually Eating.
25:47
Yeah, and then sorry. The other.
25:49
The only other thing I wanted to talk about related to some of
25:52
the challenges is with throughput, right?
25:54
I think this is related to something we discussed earlier.
25:56
We're how much you want to give the user so you don't overwhelm
25:59
them. Are there any is any?
26:01
I guess noteworthy challenges between the trade-off and
26:05
latency and throughput at inference time with your models,
26:07
anything worth mentioning, I know that's been a challenge
26:10
especially with generative models, you know, Presenting
26:14
regenerating images or generating some sort of text?
26:17
I know there can be some challenges.
26:19
There are trying to find the right balance between the two is
26:22
that relevant at all for how you present results.
26:26
Yeah, in general I think there's probably a trade-off and when I
26:29
say throughput I think what you're talking about is also an
26:31
interesting concept and that's more about you know how much
26:33
information you provide to the user when I say through.
26:36
But I also think a little bit about you know, how many
26:39
requests are we getting per second from users and in I think
26:43
there is an interesting trade-off between latency
26:46
throughput and relevance. So then some ways these are, you
26:49
know, kind of three points on a triangle where you could end up
26:52
having very low latency. Then you end up having, you
26:55
know, lower latencies, Lula relevance because you're
26:57
probably using a distilled model or a smaller model.
27:00
Exactly. Yeah.
27:02
And then oftentimes, maybe that gives you higher throughput
27:04
because again, you're not using as much CPU or you don't need a
27:08
GPU. So there's always a trade-off
27:11
between these three different angles.
27:13
P relevance latency trade-off is really interesting and I think
27:18
yeah there's definitely a lot of ways in which you can get very
27:20
high relevant. So for example, you can use and
27:22
if you were working out of noetic company that is let's
27:26
just give it example of, you know, you're working at a
27:28
finance company and you're trying to understand and
27:33
classified documents into different classes, I don't know
27:36
you give somebody alone, you don't give somebody alone and
27:38
say that. In this case you have an
27:40
attending this example up, you have a None of you know
27:45
throughput at this case. So you need efficient solution.
27:48
You can often use a very simple model.
27:50
Like a classifier, I don't know. A very simple, you know, bird
27:53
based or distilled Bert base model.
27:55
And in that case, you may get low latency.
27:57
But the relevance may not be as good compared to using say, you
28:01
know, and opening iepi where you're essentially using GPT,
28:04
you know, 3.5 where, you know, you'll probably get better
28:07
results with the prompt, but you'll end up being slower.
28:11
So I think that is also often. Yeah.
28:13
Please get, I think sometimes throughput is usually not.
28:16
I wouldn't recommend people. Worry too much about that.
28:18
That's more function is success in some ways.
28:20
Like, you know, once you have success you to worry about that.
28:22
But really, and yeah, yeah. The trade-off is usually a more
28:25
relevant 14 people. Yeah.
28:27
And you were getting into the resources that are required to
28:29
do this, right? So, obviously, lots of gpus
28:31
required to train these models. Let's talk about some of the
28:34
challenges that you do cam is facing there with compute,
28:36
because there are again, more trade-offs there to.
28:39
You can use a smaller instance, maybe using pure CPUs you In
28:43
it's just there's challenges, they're right because you maybe
28:46
you use GPS but they're very expensive.
28:49
How is you.com addressing some of the challenges are thinking
28:52
through? What's the, how to prioritize
28:54
them for themselves? Yeah, it's a great question so I
28:58
can go into too much details around the the way you wish you
29:00
do but I can give a high-level a little bit about how we think
29:03
about some of these issues. Yeah but you're right I think it
29:05
comes down ultimately to the user experience so what can we
29:08
do to you know really Trace back from there.
29:11
I think one thing that we have noticed is that Peas are quite
29:14
expensive. So in general, you know, when we
29:17
can use CPUs, we do so I think CPU, you know, is important.
29:22
I think of, you know, one thing to think about what CPU usage is
29:26
often like CPU utilization. So a lot of times you need to
29:29
make sure that your most efficiently using the CPU so
29:31
exactly. Yeah, it will happen.
29:33
Like for example, if you're not careful, you don't account for,
29:36
you know, user queries that are really long, you might end up
29:39
spiking, you know, CPU utilization on those really long
29:42
queries because Your model ends up, you know, not bashing up
29:46
correctly, or iterating through all of it and you know, doing a
29:48
ton of operations. So I think those are kind of the
29:51
more tactical areas, we think about.
29:54
But I think at a high level, I would say in gpus are great and
29:58
they make a lot of sense and they you know, you can paralyze
30:01
and do things a lot faster and run heavier models.
30:04
But then you have to take into account costs and also yeah, the
30:07
limited amount of the GP there, still a place for CPUs, right?
30:10
And I know that a lot of companies are investing in
30:13
writing there. Kernels that can utilize our
30:15
better utilize CPUs because of how expensive gpus are right
30:18
now. And so there's like, how can we
30:20
reuse what we do? Or we already have a lot of
30:22
access to, but we have to do a lot of custom like Colonel
30:25
writing just to make it like you said dealing with different
30:27
sequence lens. Like there's all sorts of
30:29
challenges that I think are really interesting.
30:31
Just shifting gears a bit, you know this is where we're talking
30:34
about some of the technical details but let's talk about
30:36
you.com and what it's like working as an engineer at a
30:39
small start-up that's taking on big companies.
30:41
You know. It's a very Competitive space.
30:44
I mean, there's sort all sorts of news about this people that
30:47
don't even care about tech or are wondering, what's, what's
30:49
going on in this space. So, it just what you, what's it,
30:52
like being in the midst of like, I guess, like a lot of hype, but
30:55
a lot of its real to a lot of. It's really, you know,
30:57
actionable people are using it. So I feel like I must be
31:00
exciting. Yeah, I would definitely say,
31:02
it's very exciting. I think to work at a small
31:04
start-up or I guess it start up in general.
31:07
And I think, you know, one thing to think about is I think we
31:12
definitely you know our Trying to really build something that
31:15
will stand the test of time and is hopefully that, you know,
31:18
flash in the pan. So I think that's one thing to
31:20
think about a lot is you know, how do we make sure that our
31:22
fundamentals are good and at the end the day, you know it is
31:25
really matter what the hype is what really matters is.
31:28
You know, whether or not we can be useful for users and then
31:31
they'll keep coming back to us and, you know, they can find
31:34
Value and you know, it's differentiated value compared to
31:37
other products. I think that's really the thing
31:39
that we're going after. I think, you know, we've focused
31:42
a lot on developers in the past. And we found that with you code,
31:45
where were able to allow people to save time by, you know,
31:48
searching and then taking advantage of, you know, code
31:50
completion as well as, you know, stack Overflow apps Etc.
31:54
That allow you to access content quickly.
31:55
It's a very simple use case but that's something that has really
31:58
resonated with folks. And I think we're kind of
32:00
looking to do the same for more of a mass Market with you chat.
32:04
So really thinking about how to, you know, fulfill user, you
32:08
know, requests and and really provide differentiated value for
32:11
them. I think.
32:13
Yeah, the the joy of working as a servant iterating as well.
32:17
There are you mentioned some of the differentiators like one big
32:20
one is citations in fact that's like one I my wife's a lawyer
32:23
and one of the biggest I guess like critiques of some of these
32:28
language models is that they they sometimes provide wrong
32:31
answers or no, references to where they got these answers
32:34
from and for certain industries, that's critical.
32:36
You need to be able to cite a source, reference, to Source,
32:39
dig deeper into that. And I've II use you.com because
32:43
of that. That the fact that it provides
32:44
references. So what are some other things
32:46
that you think make you.com different from your traditional
32:50
search engine? Besides the obvious like
32:52
technology behind it. Yeah, definitely.
32:55
So I think there's definitely there's a couple of factors and
32:57
if we think about you.com in general and then you Chad, I'll
32:59
maybe go into both of them and I'll start with you chat.
33:02
So you Chad is our conversational interface and
33:05
this is really where we're seeing a lot of excitement and
33:08
we also seek Simon you.com in general, so it's about them.
33:11
But when it comes to you Chad I think you're right.
33:12
So one of them is The idea of being able to site content.
33:16
So when we show you a generation we show you you know a chat
33:18
response, we try to cite the content and we also are now
33:21
including, you know, web results as well as well as what we call
33:24
our you.com apps which have specific information from highly
33:28
curated sources. So this is all part of the idea
33:31
of being able to be more trustworthy and we still have a
33:34
long way to go. So we do have a disclaimer right
33:36
now which is we're still working and making sure that you know
33:39
the accuracy is, you know, is improving so user should be
33:42
aware and, you know, no, How to best use the tool but in general
33:45
I think being able to have a really trustworthy chat but it's
33:49
something that we're hoping will be a differentiator and we have
33:51
a lot of research and a lot of work to do to keep improving on
33:54
all these fronts. But that's maybe one reason in
33:56
the second one is I think I did too that this one the first one
33:59
but accessing a chat but in the context of search engine it's a
34:02
little bit different than texting it's a pot in another
34:04
environment. So the advantage here is that
34:07
you can then quickly toggle and switch modes because a lot of
34:10
times you're chatting and then you want to be able to die.
34:13
Beep. So we're trying to be able to
34:15
enable both of those modes of exploration.
34:17
So having a chat, but in the context of a search engine is
34:20
something that we think is unique and can provide value
34:23
adding the last one is accessibility and user
34:25
interface. So we're trying to focus a lot
34:27
on having a really nice user interface figuring out the best
34:30
way to not overload users with information.
34:33
Provided them you know tools to dig in deeply.
34:35
So those are kind of maybe three axes of differentiation of the
34:38
last one would be this concept of an open platform.
34:41
So the idea is that we have other people.
34:43
We can build apps which will eventually you know we should do
34:46
live in you are calm and we've had a couple partner.
34:48
So one example is a company called the area.
34:50
So they their shopping company, they have like reviews,
34:52
insights, whatever and that type of stuff and they have an app on
34:55
you.com. And the idea is that we'll have
34:58
all these you know, external folks.
35:00
Essentially you know building apps on you.com and will be able
35:04
to service their content. I'm hopefully allow them to
35:06
monetize as well and kind of grow the pie and really allowed
35:09
them to plug in whatever they build into a bridge
35:11
conversational interface. And that's maybe a little bit
35:14
more about the future is kind of building out an open platform
35:17
where it's not just us, but it's really an open Community, that's
35:21
building, you know, a new product together.
35:25
How do you see people? Like if somebody's listen to
35:27
this. And they want to go and start
35:29
playing around or they want to think about different use cases.
35:34
Like how do you see people using this?
35:37
What are some cool use cases? That you've either seen or
35:40
you've thought about and you just haven't had the time to
35:42
build yourself. Um yeah so you basically use
35:45
cases to build it just in general for search.
35:49
Yeah. Oh no on top like in this app is
35:52
an ecosystem that you're talking about?
35:55
Yeah yeah I think there's there's there's so much room
35:57
there so there's so much that so much unsolved territory in
36:00
Search and it really could be anything.
36:02
I mean, you can think of, you know, if you have a really cool
36:05
generate AI app and there's some queries that you think it should
36:08
be listed for, we can kind of take that into account and we
36:11
can build whatever you can. We can incorporate Are you build
36:14
into search results and present it to, you know, our users that?
36:18
And, you know, that's kind of you can think of it as an access
36:20
point that you have. So, there's the App Store on
36:22
Apple which is one access point to users, hopefully you.com will
36:26
be another access point which will allow you to kind of richly
36:29
integrate whatever your building.
36:31
And, you know, show it to users that would say, you know, things
36:33
like that. Really simple things like, you
36:35
know, a really good calculator, app would be, it would be great
36:38
one to add. That's something that, you know,
36:39
has been our backlog for a while, you know.
36:42
So it could be, you know, ask. Complicated or as simple.
36:45
There's a there's a space for a lot of different things.
36:46
Right now, the end and let's talk about, I guess, what would
36:49
be interesting to developers data scientist practitioners
36:52
that are, you know, looking to integrate you.com or a way to
36:56
leverage you.com into their day-to-day project?
37:00
So we think I'm thinking about AI tools for engineering.
37:02
Can we talk about some of the ideas that you.com has for that?
37:06
Yeah, yeah, I think right now is another shift that's happening
37:09
in engineering, is, you know, there's a lot of tools that can
37:12
make Engineers a lot more productive.
37:13
So historically, you know, there's always been tools that
37:17
we've used. I think the, you know, one of
37:18
the best ones in my opinion has been get any doubt collaboration
37:22
tools for engineers that just imagine collaborating with out
37:25
really good Version Control software.
37:27
So I think what we're seeing now is going to be may be as
37:30
important as that for for engineering.
37:33
So in some maybe in the future, will look back and the eye.
37:36
If you know, writing code without having access to code,
37:40
completion modules and, you know, a I will be as, you know,
37:43
foreign as the concept of working at a company without
37:45
Version Control, which is maybe a bold statement but it seems
37:48
like, you know, we are headed in that direction.
37:50
So I think one way is, you know, with you chat, you can type in
37:53
developer queries and you can get, you know, Rich output and
37:57
you can test whether on it compiles, I think the great
37:59
thing about me, Cody's case that I'm particularly excited about
38:01
is that it is very procedural. So you can quickly verify
38:04
yourself a lot of times. So, whether or not it works, you
38:07
know, obviously, you'll have to customize it to your context.
38:10
I don't expect no special yet certain companies, you can just
38:12
plug it in. It'll work out of the box, but
38:14
it can save you in a 50% of the time, Ian, what?
38:18
Library to import, you know, certain X and if it works, then
38:22
great, you know, you have starting position to build on
38:24
top of, that's one of the advantage of some of these
38:26
procedural types of queries is users.
38:28
Get kind of immediately, get used from them and they don't
38:31
have to be perfect. Yeah, it's I hope it's okay to
38:35
ask but one thing I was immediately thinking about was
38:37
you know, opening II and the way that they think about training,
38:40
these models is with a mixture of text and code, because
38:43
apparently there's some interesting relationship between
38:47
if it learns from Coda learns, maybe logical relationships, it
38:50
learns how to think procedurally, right?
38:52
But language is more open and sloppy but you need both to give
38:57
good answers right? And and I don't know if there's
39:00
other things to include on that that Also, you know, make it a
39:04
bit. It seems like it depends on
39:06
what's it like for each. One of the endpoints that
39:09
you.com to has, you may have different models, right?
39:12
That prioritize different types of responses is that okay to ask
39:16
about the nuances? Or is it is the idea that you'll
39:19
have one model that serves all of them?
39:21
I imagine that's not the case. That would be an extreme earful.
39:25
You're being so careful David. I love it.
39:29
Yeah. Yeah, we want to be want to be
39:30
mindful, you know, before things are proprietary.
39:32
So, So whatever you can disclose at a high level.
39:34
Again, it's just thinking about some of the interesting
39:37
challenges that they face their, like, what data to collect two,
39:40
to actually provide output. That like, you mentioned is
39:44
relevant to users. Because again, we're providing a
39:46
specific type of output depending on your output for
39:48
code for writing, for whatever the endpoints are.
39:53
Yeah, I think it's a great question and it's something to
39:55
think about a lot. And one thing, I'll I remember
39:58
is, you know, when we were running, a lot of these search
40:00
studies, in which we compared to you code to Google, So, a lot of
40:03
the feedback was, you know, we would sometimes show code
40:05
Snippets without context we would just kind of throw out,
40:09
you know, here's the answer and code.
40:11
This is the coast, I bet you should use and even if it was
40:14
correct, it would be a little bit starved them to users
40:16
because, you know, we do expect there to be some text contents.
40:19
That's something we've adapted towards.
40:21
So you don't just want Cody do want text with code unless
40:25
you're you know, And there's very few cases, I think where
40:28
you would just want to for the hanging coats at it and that
40:30
would be useful to you. You'd want some type of
40:32
explanation of what this does. So I think that is very
40:35
important. So I think, you know, there are
40:37
ways of seamlessly generating, it error including code Snippets
40:41
with like basically with text that we've done.
40:44
So if you do we do this right now.
40:46
So if you do certain queries and you chat, you should have both
40:49
code Snippets, you know, with proper formatting and like
40:52
syntax highlighting etcetera, incorporated into the response.
40:56
And we also stepped really have, you know, corpses information,
40:58
these apps that we have in which we use indexed, a lot of these
41:02
code Snippets along with the text associated with them.
41:05
And in that case, we can definitely order and, you know,
41:08
we would only searching against go Snippets but also against the
41:11
text, because a lot of time, if you're searching for something
41:14
you're often using natural language off of me, exactly.
41:17
You're debugging, you might be, you know, typing in code.
41:20
But I would say there's probably like a 70/30 split where most of
41:23
the time you're searching in language.
41:26
And then sometimes they're debugging and a lot of times the
41:28
bugging of Stack traces, it's not even really cool Snippets.
41:30
So the chance that you're actually typing in the code
41:33
snippet, maybe if you're looking for documentation and it's fine,
41:37
but in general, you are, you know, trying to understand and
41:39
use natural language. That's the way we interact.
41:41
In order to find text and code Snippets.
41:44
You're not typing in Coast night is looking for an exact match
41:47
when I killed stuff it with another code snippet.
41:50
So, yeah, we do think a lot about this and yeah, which we
41:53
try to incorporate the different elements.
41:56
And it seems like that will also shape the interface in some
41:58
ways, right? Like the how the outputs will
42:02
also be related to the type of interface that our user will you
42:05
know interact with you know you mention about these different
42:08
apps that are going to be provided having kind of a
42:10
variety of outputs, like some with references some text and
42:14
then some things that they can click on.
42:15
So it seems like there are lots of trade-offs to consider here.
42:19
Yeah, yeah, definitely dude. So then do you see a world and
42:23
forgive me if this is like what you're actively.
42:26
Lee trying to do because I may be a little slow.
42:28
But is it where you're going to be interfacing?
42:31
Also, with your, the bugs that are created or the logs and you
42:38
can just ask questions and then ideally the logs can go through
42:44
some kind of model or something where it's much more intuitive
42:48
or you can just ask the computer, why is my shit not
42:51
working and then it will tell you in a much more natural
42:54
language Are you saying that you know you're the user will
43:00
provide logs to the service and then try to query and understand
43:04
what's up for the end user if I'm putting something together
43:08
and I have my app or my just coding and for some reason what
43:15
I'm trying to do is not working and then I cannot debug it by
43:21
looking at the logs I can't figure out why is this not
43:24
working? And Then I could just use
43:26
natural language and ask, why is this not working?
43:30
And it will give me a more relevant answer as to why Sort
43:35
of like a plug-in in like editors sort of thing.
43:37
I guess like copilot would be wondering about, I don't know if
43:40
they look through logs as but. Anyways, yeah, something that
43:42
would be integrated into your coding environment to help you.
43:45
Like, you can query it in natural language.
43:47
What's real? Why is this not working?
43:49
That would be great. Yeah, I totally agree with you.
43:53
That, that would be great. We have thought a little bit
43:55
about this. So we do have the vs code
43:57
extension. Where you can basically
44:00
highlight a piece of code and then you right click, then you
44:02
can search it on you.com. So the turbines, the step We
44:05
haven't yet flow and we actually do have other extensions coming
44:09
out in which you'll be able to see code Snippets and
44:11
essentially do search within the editor, but I do think there's a
44:15
lot of work to be done there. I think, I think you're right
44:17
that a lot of times the access point for developers is the
44:20
editor. So we're taking a little bit
44:22
about that as well, but right now I, you know, we're focused
44:26
on, you know, you chat as well, but there definitely is a
44:30
Synergy between the two. So, you know, right now you can
44:34
still show returned from Mission outside and letter your little
44:36
bit limited you know by the types information in the UI that
44:39
you can show with in something like the S code.
44:42
And you know if it's very easy to move from BS code to a search
44:45
engine, it makes sense, but I think for certain things yeah
44:48
things that make sense to kind of show tips, copilot etcetera
44:51
within an editor and I think that provides a lot of value as
44:53
well. But yeah, the logs are
44:55
interesting statement. I think there's a lot of work to
44:56
be done there. Yeah, that would make life
44:59
easier. Yeah, I'm it.
45:01
Feels like there's so many different directions you guys
45:03
could go and because of the need Right?
45:05
Like a search engine people, use it to search anything, right?
45:08
And and it seems like this could spin out into lots of things.
45:11
How are you guys? Staying focused.
45:14
There's a lot of excitement in this sort of a picture for bike.
45:18
Isn't that why you allow people to do everything that you are
45:23
not actively doing? Because it's actually a point
45:25
build an app or so that you know we're not going to be able to
45:28
tackle 99% in the problems. We're just going to tackle the
45:31
most important to us and then let everyone else do it in more
45:34
of A decentralized way and grow the pie anyway.
45:38
Yeah. And you've answered it
45:39
perfectly. Yeah, that's smart.
45:42
That's is there. So before we finish, I really
45:45
want to know what has been Something that you've been
45:48
thinking about a lot lately and take it how you want.
45:53
It doesn't necessarily need to be technical but it can be
45:57
design-wise. It seems like you like design
46:00
and you're more. You have a lot of product sense
46:02
inside your blood, but when it comes to search or you.com, what
46:10
are some things that have just been?
46:13
They haven't been able to leave you alone.
46:14
Like you think about them when you're going to sleep, and when
46:17
you wake up, Yeah, I think there's probably a lot, but I
46:21
think I may be one of them would be around thinking a lot about
46:24
how to combine a lot of the advances, we're seeing an NLP,
46:28
with a lot of the traditional, you know, foundational knowledge
46:33
and information retrieval. So thinking a little bit like
46:35
what is the place of a lot of these techniques that have kind
46:38
of stood the test of time when it comes to, you know, searching
46:41
there's just give a simple example, you know, like tf-idf
46:45
p.m. 25, these types of techniques that have, you know,
46:48
worked kind of well distortedly as well as semantic search.
46:51
Just kind of, you know, based on, you know, embedding
46:53
documents and then doing nearest neighbor search like approximate
46:57
approximate nearest neighbor search.
46:58
So, there's this whole entire world of information retrieval,
47:02
Crystal formation, feeble neural information retrieval and you'll
47:05
re-ranking, you know, using document signals, you know, user
47:09
preferences, all this type of stuff in this world related to,
47:13
you know, large language models and especially of the generative
47:16
righty As they had a, we use both of these things together.
47:21
How do we kind of, you know, use a lot of the fluency of
47:24
generative Ai. And a lot of these large
47:26
language models, which can generate really fluid text, but
47:28
how do we also? Make sure they're grounded
47:30
using, you know information? We know to be true about the
47:32
world because in some ways, it doesn't make sense to, you know,
47:36
have a large language model have to encode, all of this knowledge
47:40
about the world. You know, there's so much
47:41
knowledge about the world, a lot of this changing especially when
47:43
it comes to live information. So in some ways it's clear that,
47:46
you know, you will also need knowledge Basis and you need
47:48
some way to combine the two in the most optimal pattern.
47:51
Letting that something that I think is probably really
47:53
interesting. And there's a lot that we're
47:55
doing to think they're, you know, how, like, you know, what
47:58
is the role so good. And how do they, how do they
48:00
work together? And there's been some really
48:02
interesting papers actually in the space, I've seen I've been
48:04
reading a few, we're like retrieval augmented models where
48:08
they in addition to having some out before the model, also have
48:12
some corporate information that they can also access.
48:16
And to me that was like, why? I thought that's what people
48:18
have been doing for a long time, but apparently, combining them
48:21
is a bit different than using two systems side-by-side, but
48:24
very exciting times. Lots of things going on any
48:28
closing thoughts, anything. You'd like to share with our
48:30
listeners about you.com about the space about anything,
48:34
anything that you're interested in.
48:36
Yeah. Yeah.
48:37
I think I guess the main thing would just be, you know, I would
48:40
just see you know, if you're interested definitely keep an
48:42
eye out on search right now. I think right now we're seeing
48:46
search you know in a more Dynamic State than it's probably
48:49
been in many previous years. So even right now I think today
48:52
and tomorrow Google and Microsoft are both coming out
48:55
with announcements in which they're releasing their own
48:58
versions of chat, interface has combined onto other search
49:02
engines or, you know, operating dependently So, I think it's a
49:06
really interesting space right now.
49:07
I think we still think that we have a lot of unique value to
49:10
add and we do it in maybe a different way, which I think
49:13
will be useful. But I would just say, you know,
49:15
if you're interested in search, this is definitely a fascinating
49:17
area to pay attention to, and if you're just in you.com you could
49:20
definitely reach out. So my email is, you know, sahil
49:23
at you.com so sah. I ll you.com definitely reach
49:27
out. If you ever want to talk about
49:28
search topics, always happy to discuss more or anything else.
49:33
And then, lastly, you know, if you're interested you can also
49:36
Reach out to us when we are hiring for you know, various
49:38
that the various areas. There we go.
49:40
And design, Etc, is definitely a definitely reach out through the
49:44
films like an awesome place to work.
49:45
Yeah, an exciting time to work in this space, I mean you're
49:49
competing with the Giants. You're also gaining all this
49:52
momentum, doing something that is super relevant and a Hot
49:55
Topic in lots of different areas outside of search as well.
49:58
But I also agree like searches. Something is awesome, is
50:01
happening in this space and it feels like in search for a
50:04
while. It was only Google.
50:05
They were the main competitor but now that's not the case
50:07
anymore. So it's going to be very
50:10
exciting to see what the future holds for search.
50:12
Yeah. Right.
50:14
Oh man. Well I think this is a good
50:16
place to end it. Thank you so much, so are you As
50:24
the lead engineer Tiki, people think.
50:26
I'm definitely not wanting other Engineers to have all this
50:30
relevant information and I will not give my name, which is Lena.
50:35
So definitely, I do not recommend this podcast.