Latest / Agentic Conversations (formally mlops.community) / All About Evaluating LLM Applications // Shahul Es // #179

Transcript

All About Evaluating LLM Applications // Shahul Es // #179 · Agentic Conversations (formally mlops.community)

0:00
Hi, I'm Shahul. I'm the co-founder of Exploiting
0:02
Gradients and I take my coffee black and without sugar.
0:08
Hello and welcome everyone to another edition of the ML OPS
0:11
Community Podcast. I am your host Dimitri OS and
0:15
today we are talking to none other than Shou S This man has
0:21
been doing very big things in the evaluation space.
0:24
He's also been absolutely killing it with the Open Assist
0:28
project. You may know him because he
0:31
created the Ragas project, which is an open source evaluation
0:36
framework for Rags or retrieval augmented generations he's got.
0:44
Such a pulse on the open source LLM ecosystem.
0:50
We had a deep dive on how to look at evaluation, where to
0:55
even start if you're thinking about Ohh man.
0:59
I'm trying to debug this. I don't know what is wrong.
1:02
Is it the open source model that I'm looking at?
1:05
Is it the prompts that I'm looking at?
1:07
Is it that I'm serving it on the wrong type of hardware or do I
1:13
need to check out? My vector database.
1:17
Do I need to look into that and my chunking it correctly?
1:20
Sparse, dense vectors? I'm just throwing out buzzwords
1:24
right now, but since he got into it because he actually knows
1:28
what he's talking about. And then we did a whole another
1:33
deep dive on the state of benchmarks.
1:38
Shameless gives us all kinds of insights and wisdom into this
1:43
world. Hope you enjoy this
1:45
conversation. And if you know one person, just
1:48
one person that you think. Would value this podcast.
1:54
It would be a huge help to me and the team.
1:59
You can share it with that person.
2:06
Shahrukh, it is great to have you on here.
2:09
I'm excited to talk to you and I think we have to start with the
2:12
big question, almost like the existential question.
2:17
What does evaluation mean to you?
2:21
Yeah. So evaluation means to me is a
2:25
way of really measuring and quantifying the performance of a
2:31
system. So whatever it may be can be
2:34
alums, it can be a traditional system or whatever.
2:38
And why we should evaluate is because if we want to make any
2:43
improvements on something, the first step is to measure and
2:46
quantify it in the 1st place. Then then only you will be to be
2:52
able to do any kind of iterations on top of it and see
2:56
if these iterations have made any positive or negative.
3:02
What country impact has these iterations made in with in in
3:06
your system so? That's what what is evaluation
3:10
and why we should do evaluation. Awesome.
3:13
So basically, what you can't measure, you can't actually fix,
3:19
and you've got to iterate as much as possible to make it
3:24
better, all right? And so when it comes to
3:29
evaluation, I mean, you wrote an awesome blog post that I saw and
3:35
I loved, and you talked about all the different ways that we
3:39
can evaluate, and I think that. It stood out to me.
3:44
There was a few things that stood out to me.
3:46
Probably the first one is. That we have all these
3:50
evaluation metrics, but it feels like sometimes when models come
3:56
out, they just say how they do against these evaluation metrics
4:01
and just saying how they do against evaluation metrics.
4:06
I've learned to. I'm almost like immune to that
4:09
now because it feels like every model that comes out, says Ohh.
4:12
We're state-of-the-art on these benchmarks or this or that.
4:17
Yeah, you felt that way too. And what do you think about
4:20
that, right? Even even as a guy who deals
4:24
with these open alms on a daily basis for a variety of purposes,
4:28
I have found it really hard to select an LLM for my own
4:33
application or my own usage. And these open LM leaderboards
4:38
are actually going to be less reliable than more reliable, if
4:42
any. And this can be because of maybe
4:46
an ongoing competition or race to produce the best model in the
4:49
leaderboard rather than producing the best useful model
4:54
in general. So it's a tricky situation.
4:57
As of now, everyone is trying to optimise for a particular leader
5:01
board and that optimization becomes a kind of overfitting to
5:05
the particular datasets or leaderboard itself and making
5:11
the model or even the efforts less useful in the long run.
5:16
So I will share the same feelings and it is something
5:20
which the open source community and that basically the LLM
5:25
community has to figure out really quickly so that people
5:29
can actually choose an LLM and actually know which LLM is at
5:35
least good for a particular task or something.
5:39
And rather than just trusting random leader boards which could
5:43
be blindly over philtre to the leader board itself.
5:46
Yeah, and let's just double click on that for a moment,
5:49
because why are these random leaderboards getting gamed?
5:53
Is it because some of the? Evaluation test sets are being
5:58
used in the training and then they're over fit for that.
6:00
So it's like the model basically knows the answer before it's
6:04
asked the question. Or are there other tricks that
6:08
are happening? So there are two reasons that
6:13
are mainly observed. The first one is the one that
6:16
you pointed out. These datasets and tasks are all
6:21
open, so people can use these datasets to fine tune them all
6:26
on these datasets. Which means that model has which
6:30
already seen that test set it. So it basically breaks the
6:35
preliminary assumptions of formulating a test set that
6:38
tests the model Should not have seen the test set before.
6:42
So that is a first problem and the second problem can be the
6:46
use of evaluation metrics that have poor correlation with human
6:53
judgement. Even these days many of these
6:55
metrics that we use thus does not really have high
6:59
coordination with human judgement when it comes to low
7:02
form answering. So basically when you say that
7:05
assets or even evaluation you will have different types of
7:09
datasets. One data set could have a simple
7:12
one word answer it could be one word question answering and some
7:16
other data set could be something like a loan from
7:19
question answer. So for short form question
7:21
answering, it is basically very easy to compare.
7:24
You know OK if this if you expect the answer as York City
7:28
and the model gives out something like New York City or
7:31
NYC, you you rather figure out some some metrics to see that OK
7:35
these answers were similar and hence the model was correct.
7:38
So for long form answering there are very few datasets available
7:43
and there are not really very good evaluation methods
7:46
available to do it for. For example, how do we evaluate
7:49
the capability of a model to write a summary or or or a long
7:54
story. So then they they introduced
7:57
different dimensions for example coherence, fluency, etcetera,
8:01
etcetera. And these dimensions are
8:03
evaluated on on the basis of mostly traditional metrics which
8:09
are not really highly coordinated with human
8:11
judgement. So that could be the reason why
8:13
the evaluation metrics itself feeds or even the third itself
8:16
fields. And the first one is obviously
8:19
the overfitting to the test set which should be avoided.
8:22
So has a very similar situation. Whenever they host a
8:27
competition, they have to evaluate these models and models
8:30
that people submit, right? So what Kaggle does?
8:33
Is that OK? Sure, the player should be able
8:37
to understand or have an estimate of the performance of
8:41
the model on leaderboard, but that estimate will all be final.
8:45
So there will be a public test set and a private test set.
8:49
So it ensures that the model are not all fit to the public test
8:54
set. So if you have a say if you have
8:57
100 sample test set that you are trying to optimise for, you
9:01
could train your model anyway and just optimise for that
9:04
particular data set itself just by measuring the performance 10
9:08
times, right? Yeah, but that's not the ideal
9:11
outcome. The ideal outcome is something
9:13
which is much more generalizable.
9:15
So that's why Carol keeps it private leaderboard and private
9:18
does it to avoid any of these situations.
9:21
So that's an effective thing that open source can adopt.
9:25
Yeah, it seems like. We're missing the other side,
9:29
which is the private test sets and.
9:32
I guess right now what's happening is each person who is
9:36
dealing with these models comes up with their own.
9:39
Private evaluation metrics to go against and it might just be
9:43
something as simple as does this work for my use case?
9:47
How much do I have to prompt it? Etcetera, etcetera.
9:49
And you were saying how you're working with a bunch of open
9:52
source LMS right now and it's really unclear.
9:55
I know that every time I mess around with open source LMS,
9:59
what the hardest thing to do is is recognise.
10:03
OK, I have a use case, let's say summarization like you were
10:08
saying, which is a bit harder to.
10:10
Evaluate And now I just want to know which one of the open
10:15
source models out there is the best that summarising long text.
10:19
You can't really get that information.
10:21
It's super obscured. I wish there was a way to just
10:24
say, alright cool, if you're trying to do this use case, use
10:28
this model. At this given time, it's the
10:30
best open source model that you've got, right?
10:33
I mean something like that would be really useful.
10:37
There is an initiative called HTML from Stanford and they're
10:41
doing a good work and I hope they continue it and improve the
10:44
evaluation as in a better way. But evaluating all these albums
10:49
itself is a task because it takes a lot of commute.
10:52
So you have to run these albums on all these different datasets
10:57
and then take the output and have a valuation metrics to
11:01
quantify the results. All that stuff so.
11:03
It's it's again computing density even in a way.
11:06
You know you have to have some kind of computer for sponsorship
11:09
to evaluate, such as a. Yeah, and it's funny because it
11:13
feels like it's almost like this paradox that as soon as an
11:16
evaluation test set becomes public, then it's no longer
11:21
useful. And.
11:24
So for me it's funny because the I've heard a lot of people say
11:29
that. Benchmarks are bullshit.
11:32
And that basically. When we talk about benchmarks,
11:37
especially with LMS, it shouldn't be used to gauge.
11:42
How well this model actually performs in the wild and I know
11:47
that you've undoubtedly had this and I'm sure a lot of people
11:50
that are listening have had this happen to them too where you
11:54
hear on Twitter, on LinkedIn, how there's this new model that
11:58
dropped and it's been so good at you know does it's
12:02
state-of-the-art soda on all these different metrics etcetera
12:05
etcetera. And then I go and ask it.
12:07
My favourite question to ask. It's like is the earth flat and?
12:12
It says yes, of course the Earth is flat because we have pictures
12:16
of it and you're like, whoa, this was trained on one too many
12:18
subreddit threads, even if we say the current benchmarks are
12:24
very limited as we discuss. So even if we improve that slide
12:28
of benchmarks, OK, let's say we have all these close set,
12:32
private set, test set and everything.
12:34
If you want to really use an LLM extensive info extensively for a
12:40
particular application, for a custom application, you better
12:44
evaluate it yourself in your custom data distribution,
12:47
because it's not really practical to evaluate LMS on all
12:52
these stuff individually and present it somewhere.
12:57
So benchmarks even if if it's done correctly, it will give you
13:02
an A ballpark idea of you know what is the performance of this
13:07
LLM or particular task or particular data.
13:10
All that's it. So it's a ballpark performance
13:13
generally that's I think that's the purpose of benchmarks or
13:17
even not really say that OK, it doesn't really guarantee a
13:20
performance on any particular application or any particular
13:24
data or anything. So if one is going to use any
13:29
element whether close or open source and building an
13:32
application on top of it, I think the person should figure
13:36
out some evaluation metrics and have it evaluated whenever they
13:42
make changes to the pipeline node itself.
13:44
The LLM pipeline can have multiple other components that
13:48
affect their LLM output other than their lab itself.
13:51
So whenever they change the part of pipelines, they should have a
13:57
valuation metrics figured out so that internally or they could
14:01
subscribe to any of the SAS solutions that are exposed and
14:05
evaluating and items and evaluate the LMS whenever they
14:10
make the change. So that they know that this is
14:12
something they can push to production without worrying
14:15
about it breaking or annoying customers.
14:19
And for you, what are some of the key cause you you map out in
14:24
your blog post a lot of different ways to evaluate
14:28
looms. How are you?
14:31
Recommending that we. Evaluate these like with which
14:36
standards and which ways of evaluating, because I imagine
14:40
it's not all of them if we could, and you do make this very
14:44
clear, A little bit of a tangent, but in the.
14:47
Blog posts. You also say how expensive and
14:50
time consuming it is to actually go through this process of
14:53
evaluating and So what would you recommend if we're just trying
14:58
to figure out what's the best way for us to evaluate it
15:00
against our own data and our own use cases, so.
15:04
You get it broke. The problem can be broken down
15:06
into different steps. OK, the first thing you need to
15:09
figure out is in your application what will be a
15:12
reasonable test set that. You have to evaluate.
15:17
So the test set should contain the expected distribution of
15:21
data which which you are exposing the Lambda for XAML.
15:25
If you are building a retrieve augmented system and the people
15:29
are always asking the system about summaries and stuff, you
15:33
should have some kind of prompts and user prompts that have these
15:37
kinds of distribution. OK, what is summary of this
15:40
document? What is number of that, all that
15:41
stuff. And then the next step should be
15:44
to figure out the dimensions, the aspects that people really
15:48
care about or even you really care about.
15:51
So when you get a summary, what are the aspects of summary that
15:55
you really care about? OK, so let's see if in my case
16:00
it could be fluency, coherency and abstract.
16:04
How much? How much level of information
16:06
from the actual content, Desktop abstract.
16:09
Capture all these steps. So let's say I have identified.
16:13
Now I have a test set and the second set I've identified some
16:16
aspects, some dimensions that I really care about or my users
16:20
and really care about. And then?
16:23
I have to figure out a method to quantify these aspects.
16:28
So this can be done this OK Quantifying the aspects can be
16:34
either done using some string match or you can use an LLM to
16:38
evaluate the element itself. So.
16:41
The I think the feature evaluation, evaluation, even
16:46
multi modal whatever multi modal audio or levels itself is.
16:53
So I think in future a will evaluate A systems but.
16:58
These evaluations will be formulated in such a way that it
17:03
cancels or it is not really affected by the biases that are
17:08
inherent to a systems, I think. I think in LM's case you could
17:13
either use LM itself to quantify these aspects.
17:16
How fluent is the output and how guarantees output?
17:23
Etcetera, etcetera. So now you have to have a figure
17:26
out a method to use LLM to quantify these output in a way
17:31
that it is not really affected by the biases of alums.
17:35
Some of the biases of alms like Elms usually only give
17:38
individual scores, like some of the things which I pointed out
17:41
in my blog. Yeah, you also mentioned, right?
17:44
There's one piece where you said LLM's prefer the output of other
17:48
LLM's rather than human generated content.
17:51
That's a funny thing, right? So so if you if you want to go
17:55
so this this was I think probably one of the experiments
17:59
so there was like some of the best golden annotated human
18:04
answers and then there was answer from some LMAO CBD
18:08
something CBD 3.5 or something and when evaluated LLM mostly
18:14
prefers answer from LLM itself because there is a bias to its
18:17
still. So it's basically.
18:19
Just subscribing to that style of answering, it's not really
18:23
looking at any particular, so if you give it to answers and ask
18:26
another which is the best, you don't really.
18:28
You're not really asking them to pinpoint to any aspect or
18:32
something, you're just asking which is the best and mostly
18:35
prefers style of a particular style of answering like humans
18:40
do. OK, we also do have some kind of
18:42
bias like that. So when it was also proved in
18:45
some of the experiments I think so there are many leaderboards
18:49
which allow humans to write answers.
18:52
So basically they they basically they show 2 answers from two
18:55
different albums and ask which answer was the best.
18:59
Well, what happens in that scenario is that the people who
19:03
are judging the answers usually only care about the state of
19:06
answering which. Which is not appropriate, right?
19:10
We also want to take into account complex things like what
19:14
was of how much factuality and other stuff.
19:17
People usually only jump to conclusion by just looking at
19:21
the state. I think another also suffers the
19:24
same kind of bias, but in that case of alums, it just prefers
19:29
it's on sale over the tail of even human or any other LR.
19:34
Yeah, it's almost looking at the answers and judging it not only
19:39
on one metric, and judging it on many different rubrics and
19:44
saying, OK, well. Writing itself, I like this one
19:48
better, but the actual facts or the way that it's presented, the
19:52
arguments, et cetera, et cetera, the flow you could.
19:57
See how it could get really out of hand quickly on how many
20:01
different ways you can judge the output.
20:04
And so again, going back to evaluating it, it's like what
20:08
are you evaluating it for and what is the most important for
20:12
you? And I think you brought up a
20:13
great point, it's, it all comes down to your use case and what
20:17
you're going to be seeing in the wild.
20:20
And so if you're mainly seeing summarization and you're mainly
20:23
seeing that your end users like to get their summaries.
20:28
In quick bullet points type answers, then you should
20:33
probably bias for those type of test set evaluation metrics.
20:38
Right. So there is another thing that I
20:42
wanted to mention, which is around.
20:45
Once you see that you have proper evaluation metrics and
20:51
the model is failing for some reason or the output isn't that
20:54
good, how do you debug that? What do you do?
20:58
Do you make a better RAG? Do you fine tune the prompts?
21:02
Do you fine tune the model? Do you get a different base
21:05
model? It feels like there's so many X
21:08
factors, and so where do you even start on that?
21:13
I think it. Comes down to first isolating
21:15
the error itself and you know, figuring out OK which part and
21:19
why it happens and then scrolling for it accordingly.
21:24
So there could be many reasons if.
21:27
From the alarm viewpoint, there could be many reasons.
21:30
From just another viewpoint, there could be many reasons why
21:32
it fails. For example, if you see Lama 2.
21:36
People are trying to, I think, fine tune Lava 24 core
21:41
applications, but. It is.
21:44
Not going to get much much better in code applications
21:47
because Lamma 2 was by default was not trained in much code
21:55
tokens as required. So basically the person, the age
21:58
of code tokens it has seen in the pre training itself.
22:02
Is. Limiting its knowledge and code
22:04
then you fine tune. Doing fine tuning on top of it
22:08
only will you know. Mean improve the.
22:11
It will improve the results, but it is also prone to overfitting.
22:14
So I would isolate the error like this and then then from my
22:19
conclusions and then figure out the next base.
22:23
So it's almost like you have to know a bit of, and this like
22:27
Lama 2 not being good at coding is news to me.
22:32
And I feel like I'm very deep in the subject.
22:34
But it's almost like you have to constantly be checking.
22:39
And I guess it's probably because I didn't read through
22:41
the whole Lama 2 paper and they may have said that.
22:44
And so if you're not up to date on each one of these models and
22:49
what the training data was that went into it, it can potentially
22:54
drive you crazy because you're like, OK, let's just grab Loma
22:57
two, everybody saying it's the best out there, and let's see if
23:00
we can fine tune it on our coding use case.
23:03
And then little do you know that it's never going to be that
23:06
good. Right, right.
23:08
So I think. Yeah, I think The thing is that
23:11
this is where you. Know.
23:14
And a guy who has good exposure to these things Northeast.
23:20
So so even if you know even if a person who is just trying out
23:26
these models really won't notice these stuff because OK, for that
23:30
person it's like just figure and trying out stuff.
23:33
OK, everyone else. Every model, OK, which is the
23:35
best model. Everyone says that this is the
23:37
best so this should be the best. But you'll have to dig through
23:41
stuff, especially when it comes to upon.
23:44
Jose Luis and using it in production or you know using it
23:48
in use cases, you know to really dig through it understand OK,
23:51
what are the methods that they used, how different is this from
23:57
any other olms and so on and so forth and then choose then make
24:02
a wise decision on which L to fine tune or use for your use
24:07
case in future. It's going to be crazy because
24:10
they are going to be, there are going to be many base, many
24:14
good. The lungs or you know,
24:15
foundation, Any models and then to just choose models from it is
24:20
itself going to be a hard task. We haven't even touched on the
24:26
idea of. The prompt injection or the
24:30
security of the models, which feels like all of the evaluation
24:34
metrics that I've seen, they don't really talk about that.
24:37
Like this model is able to get hacked 9 out of 10 times.
24:41
Nobody's going to say that we were the most secure model or
24:44
the least secure model. And things like, I know there is
24:48
a whole Reddit thread that came up around Dan or do anything now
24:53
and how there's that constant push and pull between making the
24:59
model less useful because it won't tell you things since
25:03
thinks it is bad and it shouldn't say that.
25:08
And then on the other side, making the model secure and not
25:12
telling you things because it actually is bad, right.
25:14
So I know there was a lot of tweets that were going around
25:17
and memes when Lama Two came out and it said how do I kill a
25:20
Kubernetes cluster or something like that, How do I, how do I
25:24
kill this? And Lama Two is saying things
25:27
like, no, I'm a language model. I'm not going to tell you how to
25:30
kill anything. And so.
25:33
So it's funny, you know? It's the.
25:35
I I. Really allow the work the Lama
25:41
to team did, apart from the kind of Aleman they did.
25:44
I think they spent a lot of effort in Element.
25:47
But Element itself is really, really hard and to get a good
25:52
reward, to produce a good reward model to get the model aligned,
25:57
that's a very, very hard task. I've worked in IT for open
26:00
assistant team and then there is.
26:04
It's like it's like a very. It's like a.
26:06
Data problem. There is very few data which is
26:08
available in open source that we can use to align language
26:12
models. So it needs a particular type of
26:15
feedback data. I think Opera has those feedback
26:18
data because people has these user interactions and people's
26:23
feedback from a long time and then they can use these people's
26:28
feedback itself to. Language models So what I might
26:33
do was fairly over aligned and and I'm a 2 chat to specific
26:39
Blood Glamour 2 doesn't have that problem fortunately so.
26:44
So I mean it's it's it's a very tricky problem and the current
26:47
LMD is looking at a way that OK, it is it is currently aligned to
26:53
I think subscribe to maybe a particular culture or something
26:56
say for XML element is element is subjective right.
27:00
So something which is offensive in where you belong to might not
27:05
be offensive in place I belong, right?
27:08
So there are these. Cultural differences and all
27:11
that stuff which is not currently taken into account in
27:13
language model alignment. It is currently just aligned to
27:16
a mainstream culture or whatever, so I think in future
27:20
I'll even would be something which can be even personalised.
27:26
To. That also could happen.
27:29
Ohh yeah. So I'm thinking potentially you
27:33
would get different alignment depending on where the companies
27:40
are based or the end user is based, depending on the cultures
27:45
that person is in. And so you're going to have all
27:47
of these. Maybe it's the same base model,
27:50
but it has different alignment because of where it's being
27:54
used, right? Exactly.
27:55
That that's that's the idea. So I think I mean in future will
28:00
be will be like that it should be something which.
28:04
It should be aligned. To where that it's serves the
28:07
population. In in the best possible interest
28:11
of community and yeah. And and it should be it's some,
28:17
it's Richard normally something which is just a mainstream
28:19
culture. So it is just that a model is
28:22
only subscribing to the mainstream culture.
28:24
But the moral can be, it can be, it can be aligned easily.
28:28
It can be aligned to different cultures, but just that.
28:32
If if you have N different aligned models to different
28:36
cultures, it means that you have to deploy in different models,
28:39
so it takes some kind of overhead in production.
28:43
But if it's worth in future, I guess it will happen.
28:47
Yeah, it's those trade-offs. And Speaking of trade-offs and
28:52
putting things into production, coming back to the evaluation
28:55
sets, you mention how costly it can be and how curating with
29:03
ground truth is expensive. It's time consuming and to
29:08
maintain that golden annotated test set against data drift is a
29:14
challenging task. Have you seen it done well and
29:19
what does that entail? Like what should we expect to
29:23
have to do if we wanted to do that?
29:25
Because something that I find fascinating is people thinking
29:30
that they can bring open source models in house and they don't
29:34
realise all this work that's going on with, like you said,
29:38
with Open AI or or with Claude behind the scenes.
29:42
Not only to host the model and to get the latency low, but also
29:46
to make sure that it's continuously being evaluated and
29:51
having these golden test sets come up.
29:54
So a. Tester Gordon Dessert should
29:59
have some kind of annotated best answer to it.
30:04
And that is mostly the hardest part of my Indian because
30:08
whenever you update your test set, you'll have to have a set
30:12
of people who can annotate the best answer to the given
30:16
question or best outcome to the given problem, let's say.
30:19
So how to overcome this situation is something which we
30:23
have been exploring. So there are dimensions, or
30:27
there are methods by which you could.
30:32
Effectively evaluate albums without a golden annotated
30:36
answer using an ALM assisted evaluation and avoiding avoiding
30:41
such biases from alms and identifying the rate dimensions
30:46
to evaluate and quantify. Can.
30:49
Be a. Solution to it and we have been
30:51
using this pretty effectively in retrieval, retrieval, augmented
30:55
generation systems. Yeah.
30:57
Hey everyone. My name is Aparna, founder of
30:59
Verizon, and the best way to stay up to date with.
31:02
ML OPS. Is by subscribing to this.
31:03
Podcast. So basically in retrieve log
31:08
when the generation systems the. Some of the aspects that we
31:13
really care about is how grounded is the answer to the
31:16
given given context. So if you give a context, even
31:20
if the moral laws much more than the given context, the model
31:23
should only limit itself to the given context.
31:26
So how can we quantify it? How can we quantify
31:29
hallucination? Do we need an annotated ground
31:32
truth to quantify hallucination, or can we do it with using
31:37
another alum and forming A paradigm so that we can quantify
31:41
it? Effectively so I think in future
31:45
again Ella will evaluate a systems and the power which I
31:51
see which is very difficult to evaluate without an annotated
31:55
down truth is factually correct quality of the answer.
32:01
Factuality is like it is a widely debated topic and we can
32:05
even relate to that with fake news detection.
32:07
It's a very it's. An age-old.
32:09
Problem. And it's very, very hard.
32:12
This sort of problem and nobody has solved it yet.
32:14
So The thing is that you know what is factual and what is not
32:19
factual and what is acceptable, what is not acceptable.
32:22
All these things are really complicated and that except.
32:27
That the. Emotion, which is really
32:28
complicated and which is. Which is also an age-old
32:34
problem. I think there are many
32:35
differences which we can evaluate without having much
32:38
human effort by using alarms itself to evaluate Elms right
32:43
adopted. So let's talk a minute about
32:46
open assist and what you've been doing over there and how you've
32:49
been overcoming some of these challenges with evaluation.
32:52
Has it been through the, like what I've heard it called?
32:58
So RLF has gotten very popular in the last six months, but I
33:03
also have heard it called like RL AIF.
33:07
So it's if reinforcement, reinforcement learning through
33:11
AI feedback instead of human feedback.
33:14
Have you been messing around with any of that too?
33:16
And this is really what we've been talking about, right?
33:18
Like getting the evaluation through the models, other AI
33:22
models and they they will tell you if it's good or bad alright.
33:27
So it's the basic idea of again aligning A models with A itself.
33:33
I believe strongly that this is going to be the future and if
33:36
you see if you see even for fine tuning test set curation by
33:42
human adaptation is very expensive.
33:45
So before we used to have say for example if you want to build
33:48
an any error name the needy recognition system, you
33:52
basically will have OK, thousand newspaper thousand documents and
33:56
then you have to upload these thousand documents into.
33:59
Zone turf, where technical Turk and then 100 people somewhere in
34:04
the world will annotate each entities in each document for
34:07
you. Now you don't really have to do
34:10
it. You can actually do the same
34:12
thing with maybe like 1 by 100th of the coast and using LMS
34:19
itself using LMS and even much much faster.
34:23
So I think in future a will generate.
34:28
The data that it needs to be trained, maybe with human
34:32
supervision, some some weak supervision and a will evaluate
34:37
and are laid a systems itself. Again, all these should happen
34:40
with at least some weeks, even supervision.
34:43
But I think the human in the loop, the number of humans in
34:46
the loop will be reduced to very very small percentage.
34:51
And as far as open assist skills, what have you been up to
34:54
on there so in open? Assistant I have been doing the
34:59
radio stuff we used. I used to work in reward
35:01
modelling and layman team where we work on aligning our trade
35:07
fine-tuned large alums and release it now I have been
35:13
involved in. Have been very interested in
35:17
data set you know. So for fine tuning I think after
35:21
playing with fire during all these past six months, what I've
35:23
really understood is that data is the king in fine tuning.
35:26
So if you know how to prepare the right data for fine tuning
35:30
with the right model, you'll have the you'll get the best
35:34
outcome just throwing some random data to model and fine
35:38
tuning. It's not really difficult to be
35:40
honest to fine tuning and model, maybe because I have been doing
35:44
it for a while, but. It's not really difficult to
35:47
throw a random some random data and fine tune a model.
35:50
Sorry, I was. I was going to say at the end of
35:53
the day, what it seems like is is not useful if you just throw
35:56
some random data at it, yes. Yes.
35:59
So basically if you want to do that, yeah, sure, you're free to
36:02
do that. And that's not really, it's not
36:03
really difficult. What is difficult is to produce
36:06
a high quality model and how to produce a high quality model.
36:09
It's basically garbage in, garbage out.
36:12
So if you have high quality data.
36:14
And if you have figured out methods.
36:17
By which you? Can produce high quality data to
36:21
fine tune algorithms. That's where the value is and
36:24
that's where I have been mostly working around these days.
36:27
Because we have with the fine tuning setup for even $70
36:32
billion to 1:00 and we are currently doing fine tuning for
36:35
70 model and mostly in there whenever I find some spare time
36:42
I spend on data preparation and filtering and all those
36:46
techniques. So that's something.
36:49
Which I have been a really contributing to Open Assistant
36:51
and we are. Planning to.
36:54
Release a 7 billion Lama 2 model this week.
36:59
Nice. So by the time this is out then
37:01
everyone will have probably heard about it and we'll leave a
37:05
link in the description and now I want to contrast that.
37:08
So basically you've been doing fine tuning with open assist,
37:11
but then you also have ragas which are a wave to evaluate
37:17
rags or the retrieval augmented generation which I think most
37:22
people are playing with much more than fine tuning.
37:26
Now in your mind since you lay in both realms, the fine tuning
37:32
in the rags realm. And just a quick side note, I
37:36
love the name ragas because is there in India.
37:41
There's ragas right? The singing are ragas, different
37:44
nails. Alright.
37:46
So rathers, actually we named it like RG, Retrieval Manager,
37:52
Rational Assessment and it came out to be Raga Summit.
37:54
It's a cool name and it's easy to pronounce.
37:57
So we thought, OK, why not? Yeah, yeah.
38:01
But there is in singing, right? Is there not ragas?
38:05
Yes, then singing that's. Awesome.
38:07
All right, so back on track? No.
38:10
When it comes to fine tuning versus rugs, you've been playing
38:16
in both fields. I've gotten the impression that
38:21
you don't want to fine tune unless you absolutely have to.
38:25
You want to try and see how far the Rogs can get you.
38:29
Right, so this is again some topic which I have been meaning
38:33
to write on SO. So I I.
38:36
Where the way I look at it as as I agree to your point like you
38:40
should not fine tune and until and unless it is absolutely
38:43
necessary but. Adding fine.
38:46
Tuning and retrieval. Augmented generation serves two
38:49
different purposes, so. The thing is.
38:52
That fine tuning is not really meant to inject any factual
38:58
information to the model. So if you have a model, let's
39:02
say even GBD 3. And let's say whatever, like
39:06
Lama two or something, it is strained on all these tokens and
39:09
all stuff, but you are going to use Lamma 2 on a private data
39:13
set that that resides on your enterprise.
39:17
Fine tuning doesn't really make sense in that situation because
39:21
by fine tuning you cannot really inject.
39:24
A new factual? Information to the model.
39:27
If you do that, that's called overfitting.
39:29
Why? Why do we fine tune?
39:31
Fine tuning actually enable instruction.
39:34
Fine tuning actually enables model to think in a certain way,
39:37
let's put it that way, think in certain way given certain
39:41
instructions. So even though the model has
39:43
internal knowledge about things, it really.
39:47
You don't really understand the way to think or you know,
39:50
understand the instructions correctly and output the answer
39:54
correctly. So fine tuning will enable you
39:57
to do it. For example, let's say if you
39:59
have a moral which you have pre trained on only.
40:04
Bio. Data which is in the lower
40:07
domain. OK, let's say we have this model
40:10
and this model has only seen the data which is which is a load of
40:15
enduring pre training. Most of his most of it it's
40:18
storage is now confined to the law domain.
40:21
You cannot really take that model and fine tune on something
40:25
like something like life sciences and expect it to
40:29
perform good. Why?
40:31
Because mine does not really help model.
40:35
Do you know up soak that kind of information?
40:37
Factual information. It helps model to thing in
40:39
certain way. For example, let's say you want
40:42
model to follow and call a particular function when someone
40:47
says something. Let's say I have a function or
40:50
play function is a good example of actually.
40:52
So they have this model, they have this base model and
40:55
whenever the user says something which could trigger a function,
40:59
they wanted the model to do it. They wanted to model the thing
41:03
in certain way. For example.
41:04
OK, what? Is the weather in Boston and I
41:06
have a function which which I also give a function which.
41:11
Can get the weather given the location.
41:14
So in this case the model gets two things, basically a function
41:19
which has an argument called place and I'm also concerned
41:25
with the model that asking what is the weather in Boston.
41:27
So in that case I want to model to think in a way and output
41:30
function of Boston. So that's a good example of fine
41:34
tuning. In that case, what will add the
41:35
predefined knowledge of what is a function and how does the
41:38
function look and what is the code and all that stuff but?
41:41
Model just we just nudge the model in a way that it it was
41:46
able to think and follow the instruction properly.
41:48
And this fine tuning can improve the benchmarks and performance
41:52
of the model because as we have seen, the model itself followed
41:55
the instructions in a much better way.
41:56
So when it comes to RAG, you cannot really do this.
42:00
OK, you cannot really inject any information to the model using
42:03
fine tuning you have to do even. You have the capability to fine
42:08
tune models. The places where you have to use
42:11
retrieve long window generation, you'll have to still continue
42:13
doing that, and that is something which works safely and
42:17
effectively. So these are independent things
42:20
and I think there is. Something I've been meaning to
42:24
write on it and there is a popular confusion.
42:27
OK, that fine tuning is an alternative to RG alternative to
42:32
fine tuning, but I don't look at it that way and I don't think
42:35
it's that way. They're.
42:37
Complementary in a way is what I'm understanding you saying.
42:40
And yes, just because you have one doesn't mean you are going
42:43
to not have the other. You still can fine tune and use
42:46
rags 100%. Yes, you gotta do it in use.
42:50
Tracks. I guess what I'm wondering is
42:53
what have you seen be a valuable use case besides this Open AI
42:59
functions use case that you're talking about for an open source
43:03
model and a company that wants to fine tune an open source
43:07
model. Why would they do that?
43:08
What are some use cases there? Like what's going to make me
43:11
think, OK now I need to just go and find tune this, OK?
43:17
So hmm, interesting. Question.
43:20
So let's say you have to yeah, you.
43:24
Are basically running a company. I'm just saying my own example.
43:27
So in my case I want to build a model that can evaluate LMS
43:32
itself. So in that case I want the model
43:35
to output things like OK, why, if this answer can be actually
43:40
attributed to the given context and why that happened.
43:43
All that stuff, so I want more interesting in a particular way,
43:46
so. The models the output should
43:48
look like OK this is the sentence one.
43:51
This is the first statement of the generated answer and this
43:54
statement can be attributed to this this part of the context.
44:00
Then it goes to goes and so basically the model is learning
44:04
to think and answer in a particular way.
44:06
So whenever you have those kind of application where OK, you
44:11
have a base or you even have which is already fine-tuned but
44:15
you want it to. And the capability of you want
44:19
to improve the capability in a certain aspect or certain
44:22
aspects, you could prepare the data in that particular way and
44:26
use fine tuning to improve the model performance in that
44:30
certain aspects. But that one really just just to
44:35
just to reinforce this at that part should not really be
44:39
injecting new factual information to the model.
44:42
So if let's say if I have a model which does not really know
44:45
who is. Popular days, there is no point.
44:49
In asking or fine tuning the model with all the things that
44:54
Socrates has said or whatever, because moral really doesn't
44:56
really understand who is it is in the 1st place because it has
45:00
never seen someone, some it has never really understood that
45:03
this guy assisted or whatever, all that stuff that happens in
45:06
the retraining stage. So that is the difference here
45:10
in that case. It's funny because I think a lot
45:13
of people, including myself. With the image generation models
45:18
and like the diffusion models, you think about fine tuning in a
45:23
different way which. I always thought about fine
45:26
tuning with like diffusion models as OK, I'm going to add
45:29
me Dimitrios to this model now so that I can call and say
45:34
Demetrio's meditating on a rock on the sun and then I would get
45:38
a picture of myself and it was image generated in the LMS. It's
45:43
a much different story, though, because it's you telling it, no
45:47
matter how many. Dimitrios notes you train you
45:51
fine TuneIn LM on it's not really going to kick U that OK,
45:56
Demetrius talks like this. For that use case, it's better
46:00
to use some kind of a rag and just say here some few shot
46:05
learning and say here use it speak like Dimitrios Demetrio
46:10
speaks like this. Here's a few context items and
46:13
then you get the output in that way.
46:16
What you're talking about with the fine tuning is that.
46:19
You want to. Give it almost like a new way of
46:26
looking at. The world or a new tool in its
46:31
tool belt to be able to ask it. Hey can you do XYZ which it
46:38
before it couldn't have done and now you have this new new way
46:43
that it can say Alright yeah now I understand what you mean when
46:47
you say XYZ which is. Like almost like a a different
46:52
job or a different. I am trying not to use the word
46:56
function because it goes back to the open AI one, but it feels
46:58
like it's a different way of executing.
47:03
It might getting that right and what?
47:05
Where am I wrong here? Alright, alright, I think.
47:08
What? Your what?
47:09
Your perception is right there. Maybe.
47:12
I'll add just more one example. Say that you have moral that
47:16
have already understood what is understood coding in Python in
47:21
pre training stage. So basically it knows all these
47:24
it has in its internal parameters.
47:26
All these correlation, all that stuff we we don't really know
47:29
what the model knows in its in the parameters.
47:32
We don't really know what the model.
47:34
That's great. So what?
47:35
What? Let's let's go with the best
47:37
assumption that model has learned many things like, OK,
47:39
what is the relationship between all these stuff, all these
47:42
words, all that stuff. But that doesn't make the model
47:46
follow instructions in a way. So let's say I'm coming in the
47:51
next day and saying OK. I'm just really signing Python
47:56
And I want to use. Semicolon.
48:00
At the end of every statement. In and write me a Python
48:04
programme to do XY to do this with all these steps.
48:09
So even though the model has an understanding about Python And
48:13
all that stuff, what is not able to understand how to think in a
48:17
way how to think correctly and how to think correctly so that
48:22
it serves your instruction the best way possible.
48:26
So that capability is what fine tuning brings and all that
48:29
internal knowledge about coding and stuff.
48:31
All that internal knowledge about Python.
48:33
Variables, emissions and all that stuff that's learned during
48:36
pre training stage. So initially we also nobody
48:39
really knew that OK fine during is this is what fine dust.
48:45
This is what I have seen happening and this is our best
48:49
possible research that this is the best possible thing that
48:53
research has given us right now. And This is why we use fine
48:58
tuning in why we should use when tuning with our current voltage.
49:04
Awesome, dude. Well, our time is coming to an
49:07
end. For anyone that is looking to
49:10
evaluate their rags more deeply, I would highly encourage you to
49:16
check out Ragas. As we mentioned in the episode,
49:18
it is awesome. It's some really cool work that
49:22
you're doing with that, and for anyone that wants to connect
49:26
with you, where's the best place to find you?
49:29
I am actually want to tell so you could correct me on Twitter
49:34
and and. With Radas, our aim is to
49:38
improve the reliability of LM applications, either with
49:42
evaluating, testing or railing all that.
49:45
So if you have any, if any of you are you hearing me has any
49:50
application, let me know and I'm always open to chat.
49:56
Excellent. Well, thank you.
49:58
So this has been wonderful, man. You taught me so much.
50:02
I appreciate it. Well and it there.
50:09
Hey, I'm. Vishnu, I'm a data scientist at
50:11
first hand and I definitely think that you should subscribe
50:15
to the ML OPS Coffee Sessions podcast.
50:18
It's the best podcast out there to stay on top of what ML OPS
50:21
actually is to talk to the true thought leaders in the space.
50:24
And oh by the way, Demetrius is absolutely hilarious.
50:28
What a weird guy. You should definitely subscribe
50:30
to the podcast.