Latest / Agentic Conversations (formally mlops.community) / Evaluating the Effectiveness of Large Language Models: Challenges and Insights // Aniket Singh // #248

Transcript

Evaluating the Effectiveness of Large Language Models: Challenges and Insights // Aniket Singh // #248 · Agentic Conversations (formally mlops.community)

• 0:00
  I'm Aniket Singh. I work as an AI engineer or
• 0:05
  prison systems engineer at Altium Souls.
• 0:09
  I think my coffee, usually with milk.
• 0:14
  What is up ML OPS community? I am back with another podcast,
• 0:19
  as always, your host Dimitri OS. Today we're talking with Aniket
• 0:24
  all about evaluating the LLMS themselves, but not from the
• 0:28
  benchmarking perspectives. We want to evaluate just how
• 0:33
  much they know how well they can do.
• 0:36
  I loved when he talked about the confidence scores, especially
• 0:39
  how he tuned these different confidence scores and which
• 0:42
  models are more confident and which models are less confident.
• 0:46
  Let's get into it with Aniket. As always, if you like the
• 0:50
  episode feel free to give us a little feedback and if you can
• 0:54
  share it with just one friend would mean the world to me.
• 1:04
  We should probably start with the back story on how we met,
• 1:09
  which was through. Which was through lattice flow
• 1:12
  actually, since we did the round table discussion and we were
• 1:18
  kind of like planning to do it. And we, I was like, we should
• 1:21
  also do another one, something related to that's not my job,
• 1:26
  something that I do on the side. Because you are, so you have.
• 1:32
  Can you explain to us and myself again what your job is and what
• 1:37
  your hobby is? Sure.
• 1:39
  So currently I work as a, we can say machine learning engineer or
• 1:45
  a vision systems engineer or an AI engineer, basically any, you
• 1:51
  can name any, anything you want. But at the EV battery
• 1:56
  manufacturing plant called Ultium Cells.
• 1:59
  And that's why I was invited to their own table discussion
• 2:03
  because we were discussing about the manufacturing and AI in
• 2:06
  manufacturing challenges related to that.
• 2:09
  But that's my day job. But at the same time, because I
• 2:14
  was very interested in research and especially LLMS and that's
• 2:20
  kind of my thesis. Also when I did my masters, my
• 2:24
  thesis was related to like Transformers.
• 2:27
  Transformer models, which wasn't LLMS were not a big thing back
• 2:32
  then when I did my masters. But I was always interested in
• 2:36
  this transformer model because it performed much better than
• 2:40
  any other model that we had in that time.
• 2:43
  You. Bet on the right horse.
• 2:44
  Yeah, yeah. And once ChatGPT came out with
• 2:49
  their API, I was like, we, we need to do something, we need to
• 2:53
  start doing some research in this.
• 2:55
  And I do enjoyed working on this on the side and ended up
• 3:00
  publishing few papers on this and I'm still working on few
• 3:03
  more. Yeah, awesome.
• 3:05
  So you've got your day job and then your night job and your
• 3:13
  night job has brought you into a bunch of cool realms.
• 3:17
  Specifically, we wanted to talk today about the ever hot topic
• 3:22
  of evaluating LLMSI know that there's been a bunch of people
• 3:27
  on here that have talked about evaluation for LLMS. We do it
• 3:32
  very much on the systems level, like how actually we've had.
• 3:39
  So taking a step back, we've done surveys on evaluation.
• 3:46
  We've done 2 surveys in the community on evaluation and how
• 3:51
  people in the community are looking at evaluating their LMS
• 3:56
  or and their systems. But you have a bit of a
• 3:59
  different take. So let's talk about that.
• 4:03
  So when these models started to come out and all the open source
• 4:07
  models started to come out, I would see a lot of like
• 4:11
  benchmarking specially. And since I wanted to do
• 4:15
  something within this LLM, right within the realm of LLM, I, I
• 4:20
  saw everyone is doing benchmarking and nobody actually
• 4:24
  figured out how to make a whole startup or application using
• 4:30
  LLM. So just yet, because it was
• 4:32
  still the very beginning phase of it and they were making a lot
• 4:36
  of changes to to the API and that'll break a lot of things on
• 4:41
  the application side, right? So we didn't want to get
• 4:45
  involved in any of that side like the LLM system evaluation
• 4:49
  or LLM benchmarking, because there are a lot of people doing
• 4:52
  it. So I thought of this different
• 4:58
  approach that people are using LLM to also see if they could
• 5:02
  replace some of the work that humans would do to make it more
• 5:07
  efficient or make it more cheaper, whatever you can say
• 5:11
  depending on the motive, right? So we thought why not just use
• 5:15
  LLM to see if we can replace some of the scenarios where
• 5:20
  human would come into play, right?
• 5:24
  Let's say auction, right? This is one where people go and
• 5:29
  bid on the things that they want to buy and you have to be very
• 5:33
  careful of how much you bid because you don't want to over
• 5:35
  bid. But humans in general usually
• 5:38
  are pretty good at it because they know what their budget,
• 5:41
  they know their what they can afford, how much the thing would
• 5:45
  cost unless they are very unfamiliar with the item.
• 5:48
  Like in most cases, humans are pretty good.
• 5:51
  But can LLMS do that? That was one of them.
• 5:55
  Another. Wait, can we can we double click
• 5:57
  on that one? Because that feels like
• 5:58
  something that also you could set up with just rules based
• 6:03
  system. I don't want to.
• 6:05
  Yeah. Bid outside of this number.
• 6:08
  Here's my budget. Yeah, technically rule rule
• 6:10
  based system can be done too, right?
• 6:13
  But we wanted to see like how LLM would make those those
• 6:18
  decisions, right? Rule based system or Mai system
• 6:23
  could do that too. But just to simulate, let's say
• 6:27
  there's an extension that we wanted to work on, but we
• 6:30
  haven't gotten the chance. Think of a scenario where you
• 6:33
  know the person's personality, right?
• 6:36
  You know how these persons would behave and you know kind of like
• 6:40
  what kind of people would come to the auction and you kind of
• 6:45
  simulate that auction beforehand to see how much profit you can
• 6:49
  make it, right? Some scenarios like that where
• 6:52
  you can could use the personality to dictate the
• 6:55
  outcome, right? So we did give personalities to
• 7:00
  all these LLMS, like we use blanching models to give them a
• 7:04
  personality, give them some back story, like hey, you are an
• 7:08
  accountant, you have been working for 10 years, 15 years,
• 7:11
  whatever. There was another model called
• 7:15
  Olivia who was an extrovert person who didn't care as much.
• 7:19
  She was more spontaneous. So all these models had a
• 7:24
  different characteristics that dictated a lot and that that was
• 7:29
  one thing that we got to see that LLMS actually behave
• 7:33
  similar to what the characteristics that we provided
• 7:36
  them with. Because of any characteristics
• 7:38
  that you gave them, did any go over budget?
• 7:41
  Oh yes, the extrovert 1 spontaneous one was going over
• 7:47
  budget every single time. And we had to do a little bit of
• 7:51
  work to like make her understand because renamed her Olivia.
• 7:55
  And we tried to like give her some clues like, Hey, you need
• 8:00
  to make sure that we were not telling her explicitly that this
• 8:04
  is your budget or this is how much you can spend, right?
• 8:08
  But at the same time, we were trying to make sure that they
• 8:13
  don't completely go crazy, especially with GPT 3.5.
• 8:16
  That's what that's what we noticed that they didn't care so
• 8:20
  much about math and the reasoning.
• 8:21
  But GPT four was a very different story.
• 8:24
  But we were able to achieve much better results just using chain
• 8:29
  of thought with GPT 3.5 as well. OK, so let's get into the
• 8:33
  differences between the models, but before we do, to give an
• 8:38
  overview of what you really work on.
• 8:40
  You're thinking a lot about evaluating the capabilities of
• 8:46
  the model and not so much about this system around the model and
• 8:54
  how it's set up and how the evaluation of the vector store
• 8:58
  retrieval is, or the evaluation of the final product or the
• 9:03
  final answer or the question answer pairs.
• 9:06
  That type of thing isn't really what you're looking at when
• 9:09
  you're talking about evaluating these models.
• 9:11
  You're saying, can it do what we do as humans?
• 9:15
  Are there tasks that we do that if you give it to an LLM and you
• 9:20
  give it a back story, it can also kind of do it or?
• 9:26
  Or not do it. Yeah, or not do it completely.
• 9:29
  So that's what we wanted to test basically because everyone was
• 9:32
  coming up with their version of testing the model.
• 9:35
  There are a lot of libraries now that can do that, but there
• 9:40
  weren't many papers that we're talking about this kind of
• 9:44
  evaluation. So that's that was one of the
• 9:47
  motivation to do something a bit different than what the trend
• 9:51
  is. And some of these different
• 9:54
  things that you tried to see if the LMS could do were bidding at
• 10:01
  an auction. What were some other tasks that
• 10:04
  you thought might be good to replicate?
• 10:06
  So there are other tasks that we are currently working on, but
• 10:11
  this is a very different type of assessment called stealth
• 10:14
  assessment. This is something we are working
• 10:16
  on and it's almost ready, should be available very soon.
• 10:22
  But what we basically did in this is we made them do lead
• 10:26
  code problems and not just like OK, solve this lead code
• 10:32
  problem, but actually using confidence scores that they
• 10:36
  provide with their answer, how confident they were and
• 10:42
  providing them with feedbacks if needed.
• 10:44
  If they ask for hints, we provided them with hints, but
• 10:47
  what we were doing is every hint or feedback would cost them
• 10:52
  something. They knew that there's a cost,
• 10:54
  there's a budget, but they didn't know explicitly that they
• 10:58
  were being tested on their budget management.
• 11:00
  That's why it's called stealth assessment because we
• 11:03
  explicitly, we are not making them aware of this situation.
• 11:09
  So this is a different paper that we are working on right
• 11:11
  now. I have to take a step back a
• 11:14
  little bit because this paper was inspired by another paper
• 11:19
  where we tested the models for Dunning Kruger effect, which is
• 11:23
  a very well known phenomenon. But there are a lot of
• 11:27
  controversies around that. Some accepted, some don't accept
• 11:30
  it, but we still wanted to see how because they learn from
• 11:34
  text, do they show similar behaviour or not right.
• 11:38
  So in in that paper we introduced the model confidence
• 11:43
  core with the answers to see how well their their confidence and
• 11:48
  the competence basically aligning.
• 11:51
  This can inspire the other paper.
• 11:53
  And the confidence score is just once they've done a task, they
• 11:57
  say, we think that this task is complete and we give it a number
• 12:02
  from 1:00 to 10:00. So we make LLM do this.
• 12:05
  We ask LLMS like, hey, so you solve this problem once you
• 12:10
  solve this, how confident you are out of one to 10 or 0 to one
• 12:16
  that you you did very well. So we had two different types of
• 12:20
  confidence metrics. So we were asking for absolute
• 12:23
  confidence and relative confidence.
• 12:26
  So the absolute confidence is how confident the model is in
• 12:29
  itself that it did a good job, right.
• 12:33
  And these relative confidence is because we tell the model that,
• 12:37
  hey, this is a test that we are doing between multiple models.
• 12:41
  How well do you think you did compared to the other models
• 12:45
  when they don't know what other models are doing?
• 12:47
  They have no idea. They just have information how
• 12:52
  good the model itself is. Especially in case of GPT 4, we
• 12:56
  could see that the model was very aware that it is one of the
• 12:59
  best model out there. So that was kind of interesting
• 13:02
  to see. I think all of these different
• 13:06
  LLMS have no problem with their confidence and that is one thing
• 13:11
  that they do not lack. That's what we thought that that
• 13:14
  was going to happen. But some models were kind of in
• 13:18
  a way aware or even less confident than they should have
• 13:22
  been. Like models like Cloudy was not
• 13:26
  confident even when the dancers were right.
• 13:29
  So it'll try to lay lay back a little bit and say that it
• 13:34
  doesn't feel very confident. But so I think it's trained in a
• 13:38
  way to not act way too confident maybe.
• 13:44
  But GPD 4 on the other hand, it's totally different story.
• 13:46
  It's so confident it it knows that I am the best model out
• 13:50
  there. And it will say it's.
• 13:53
  Yeah, yeah, yeah. That's classic.
• 13:56
  Now I've heard there are problems when you get into the
• 14:00
  scenarios of trying to rate things like one through 10.
• 14:04
  This has been on the evaluation of like LLMS as a judge.
• 14:10
  So in LLM rating another LL Ms output and saying is this
• 14:16
  correct And give it a score between like one through 10 on
• 14:19
  its correctness. And the problem there is that
• 14:24
  one through 9 for an LLM are basically all the same thing.
• 14:28
  And so really it should be binary.
• 14:30
  It should just be that zero to one.
• 14:32
  Did you see any of those correlations?
• 14:35
  Yeah, so we did two papers like this.
• 14:39
  So 1st paper, actually we had one to 10, but then we thought
• 14:45
  based on our professor's output, he recommended that we switch to
• 14:49
  0 to one and see if that helps. And we did see a difference
• 14:54
  because if we felt like it was much more accurate because it
• 14:58
  was thinking more probabilistically, which wasn't
• 15:01
  the case with one to 10, it would probably most likely give
• 15:04
  9 or 9.5 with GPT 4 especially and even GB is 3.5 actually.
• 15:11
  But when we switch to the 0 to one, it'll actually try to say
• 15:18
  0.980.92 or even sometimes go down all the way to 0.77.
• 15:24
  So we introduced feedback. Basically this feedback confirms
• 15:28
  that hey, your answer was wrong. So we wanted to see how well it
• 15:32
  aligns for the next question like is it going to reduce the
• 15:36
  confidence which we started to see especially in case of that
• 15:40
  02/1? So you were, let me just see if
• 15:44
  I understand what the experiment was.
• 15:46
  You were getting output and the LLM was scoring the confidence
• 15:52
  of it being right and then you would give it explicit feedback
• 15:57
  on that was right or that was wrong.
• 15:59
  Well then, And then you would ask it again, what's your
• 16:01
  confidence of it being right or wrong or you would have it
• 16:04
  regenerate. So think of two different cases.
• 16:07
  So there's feedback case and no feedback case, which both are
• 16:11
  independent to each other. So one case, we don't give them
• 16:14
  any feedback. We don't tell them if they were
• 16:16
  wrong or right at all. We just ask them the question,
• 16:19
  ask them to write their confidence for that answer that
• 16:23
  they just provided. In the other case, what we do is
• 16:27
  we tell them that hey, this time you were wrong, but it already
• 16:31
  gave us let's say 98% confidence.
• 16:34
  But now we ask them like how confident do you feel that you
• 16:38
  are going to do better in the next question?
• 16:40
  Then do this show that they went down on the confidence or not?
• 16:44
  And in most cases, especially with models like GPT 4 Cloud A,
• 16:49
  we would see that it would start to align the confidence and drop
• 16:52
  it to like 77%, right? So when they get to the next
• 16:58
  question and see the question and they answer it.
• 17:01
  Now in some cases when the questions are difficult because
• 17:03
  we had varying difficulty level of questions, right?
• 17:07
  So this way we could see if they are actually aligned, trying to
• 17:12
  align the confidence score with the how accurate the answer is
• 17:15
  or not. And so when you're asking it for
• 17:20
  its confidence on the next question, it hadn't seen, no.
• 17:25
  It doesn't. See the.
• 17:25
  Next question now. OK.
• 17:27
  And then later, one thing that is fascinating to me there is
• 17:33
  that it almost like knew that the questions were harder and it
• 17:40
  had less confidence in it. Is that what I'm understanding?
• 17:43
  Yeah, well, when it starts to mess up, when it starts to give
• 17:47
  wrong answers, I think it starts to realise that the questions
• 17:50
  are getting difficult now and the level is changing.
• 17:54
  But we don't tell them that, hey, you are in level 2 or Level
• 17:57
  3. It's all based on the question.
• 17:59
  So it does start to understand. And in some cases it just knows
• 18:03
  because it was probably trained on the exact question, right.
• 18:07
  But, but there is a lot of misinformation too.
• 18:10
  So we we kind of used data that would have a lot of
• 18:16
  misinformation and make things difficult forever.
• 18:18
  And so the data picking was done in a way that it it would be
• 18:25
  robust. Yeah, yeah.
• 18:27
  All right. So tell me some other takeaways
• 18:29
  from this. And this was just one of the
• 18:31
  papers, right? You, how many papers have you
• 18:33
  written, by the way? You've written a tonne.
• 18:36
  So the auction paper is one of the paper which is already
• 18:40
  published and the the model confidence score, the one to 10
• 18:45
  is also published an information journal.
• 18:48
  Actually in that paper, we used multiple models, not just one.
• 18:52
  We used a lot of open source models too.
• 18:55
  And then because of the feedback that we got and like people who
• 19:01
  do LLMS and like based on the understanding, we were like, OK,
• 19:07
  we need to do another one. And we ended up working on
• 19:11
  another paper that used only GPT 4 model because some of these
• 19:16
  models were not great enough to be tested for specifically to
• 19:21
  the USMLE related questions. That's what we worked on and we
• 19:24
  used zero to one in in that. And we have couple more papers
• 19:32
  that that's coming. The stealth assessment is one
• 19:36
  that's coming up very soon. There's another paper that the
• 19:41
  extension of the auction paper that we are, we are kind of
• 19:45
  working on that right now and trying to see if we can use more
• 19:51
  advanced features that we have like since they have become a
• 19:55
  multi model now. So we are trying to work on
• 19:58
  those things too, so. Oh cool, so are would you be
• 20:02
  feeding in photos of items and that type of thing?
• 20:06
  That's that's the plan and to see how well it adjusts based on
• 20:10
  the picture. If we change the make it not not
• 20:16
  as much attractive, does it actually drop the value or not?
• 20:20
  So those kind of things we are still in discussion with my
• 20:22
  professor how we can work on that paper because it hasn't
• 20:28
  completely, completely started yet.
• 20:30
  We do know that we want to work on it and we are making sure
• 20:33
  that we have the prereq done before we actually get into it.
• 20:39
  Yeah. And do you have thoughts in mind
• 20:42
  that where you're going to go next and how you're because if
• 20:46
  if I'm understanding your motives, you really want to see
• 20:50
  like how aware are these things? How can they understand?
• 20:56
  Like how much subtleties can they understand?
• 20:59
  And can we test for these different subtleties that they
• 21:02
  understand? So have you thought about other
• 21:05
  experiments on how to probe that?
• 21:09
  Now I'm thinking to move more into the applications and
• 21:13
  actually making them do an extensive task because a lot of
• 21:19
  the issues right now is using an LLM in production, right.
• 21:24
  We see a lot of issues and I have my own start up to where we
• 21:27
  see a lot of issues because of LLMS. So we, we are trying to
• 21:32
  now get into research of that so that we can improve our
• 21:37
  production LLMS, right. So that's where I'm going next.
• 21:41
  And especially using multimodal is multimodal and multi agent is
• 21:47
  what I'm looking into because a lot of the mistakes that NLLM
• 21:52
  makes with their output can be possibly improved using a
• 21:56
  different LLM, a different agent workflow right let's say.
• 22:02
  You explain that real fast. Yeah, yeah, sure.
• 22:05
  So let's say I'm trying to extract some information from a
• 22:12
  document, right? If I use just one LLM to do
• 22:18
  that, it's probably going to just skim through it and it's
• 22:22
  not going to do a great job. But when we use multiple agents
• 22:26
  and we make them take, hey, you will take care of this part, you
• 22:31
  will take care of this part, you will take care of this part and
• 22:34
  create a whole workflow. This could potentially improve
• 22:38
  how well the model will provide the results.
• 22:42
  And not just that, not just having let's say 10 different
• 22:46
  models to it, but having other models do cross check, making
• 22:50
  sure that the output given by the model is actually what is
• 22:54
  required by the user. So one model doesn't need to
• 22:58
  know exact motive. So let's say I need to extract
• 23:02
  all those information in Jason format, right?
• 23:05
  Because that's one of the thing that we do a lot, right?
• 23:08
  Extracting information from a picture or a document.
• 23:13
  And we want everything in Jason format.
• 23:16
  If LLM is completely focused to just extract and also give the
• 23:21
  Jason file, there's a good chance it might mess up because
• 23:25
  of the speed and hallucination and all that.
• 23:29
  So many things are there that could happen and make the output
• 23:34
  not as expected. But let's say if I ask multiple
• 23:38
  models to extract that information, have another model
• 23:41
  horse that information, another model that actually ends up
• 23:44
  giving the actual result that I'm looking for, and that model
• 23:48
  is fine-tuned or trained on exactly how I want the data to
• 23:54
  be for me. So this should, and actually
• 23:59
  Andrew Angie has even talked about it, that this actually
• 24:02
  does improve the performance of the model by a lot.
• 24:06
  And it does increase the cost, yes, but models are becoming
• 24:11
  cheaper and cheaper. So I think this is something
• 24:16
  that many startups or anyone working on NLMS with for
• 24:21
  application would be OK with if they're actually getting a
• 24:24
  reliable result. So my goal is now to focus on
• 24:27
  reliability of the model and how well they can understand the
• 24:32
  task to create that workflow. Well, it seems like there are
• 24:37
  many ways people are trying to create agent architectures right
• 24:44
  now because it is such a Greenfield technology, right?
• 24:47
  You just get to explore completely.
• 24:52
  Have you seen different architectures that work best?
• 24:56
  Like you were just saying Like if you give a an agent a task
• 25:01
  and it doesn't need to know about any of the other tasks, it
• 25:04
  just has to do that that works. Or if you're having multiple
• 25:07
  agents cross checking and validating and then one that is
• 25:11
  the final fine-tuned agent to really make sure that it comes
• 25:16
  out proper. That seems like one agent
• 25:19
  architecture that you've seen work.
• 25:21
  Have you seen other ones? So there are some libraries, I
• 25:27
  should say that are already out there.
• 25:29
  I think QAI is one of them. Another one would be the auto
• 25:34
  GPTI believe that came out a while ago.
• 25:36
  And there's there's few more. I think Microsoft has their some
• 25:40
  own version of it. So I have been testing some of
• 25:44
  them and my team is also working on testing some of these to see
• 25:48
  if they actually will get the job done.
• 25:50
  Or is is it actually not enough? So we are working on finding A
• 25:54
  use case that we could that we could have that we can also
• 25:59
  publish, right? And it's complicated enough that
• 26:02
  would actually make sense for multi agent to work on and not
• 26:06
  something like just summary, right?
• 26:09
  That that's very simple. So I want to work on something
• 26:12
  bit more complicated, but not at the same time.
• 26:14
  Like I think when GPT four came out, they had taxed GPT.
• 26:19
  Something that people would like to monetize that's already
• 26:22
  there. People are working on it.
• 26:24
  Not something like that, but something that would help just
• 26:27
  in general, like something related to document parsing is
• 26:33
  is what I'm looking into, but not exactly how I plan to do it.
• 26:38
  That's something we are still experimenting and trying to
• 26:40
  understand first and then we get to the actual application and
• 26:47
  building that application and basically publishing how we got
• 26:52
  to that level. How how did we actually achieve
• 26:54
  it? Did we have to do a lot of fine
• 26:56
  tuning? Did we have to just create a
• 26:59
  workflow with prompts So we need to work on that?
• 27:05
  Yeah. The the other piece that you
• 27:07
  mentioned like just getting a bunch of LLM calls and making
• 27:12
  sure to cross validate or get the consensus from a few
• 27:16
  different LLMS before it moves on to the next stage.
• 27:20
  And you have pros and cons for each of these, which I I'm sure
• 27:23
  you will address where it's like, yeah, you can make 10
• 27:27
  different LLM calls before it goes to the next stage.
• 27:30
  But that you have to think about price at this moment in time.
• 27:34
  As you mentioned, maybe later we don't need to think about the
• 27:37
  price, but for now you do have to kind of keep that in mind.
• 27:41
  The fine tuning piece, I think is a perfect example of when to
• 27:45
  actually fine tune when you want it to come out in Jason or when
• 27:49
  you want like Jason Validator. That's like one of the few times
• 27:55
  that fine tuning is brilliant. I've also heard about
• 27:59
  simulation, I think, where you just ask it to do what, or you
• 28:06
  ask it to like simulate what it thinks it should do.
• 28:10
  And so then it can give you like, all right, here's here's
• 28:16
  me simulating the task and you get, you get that going on.
• 28:23
  But I'm not well versed enough in the agent architecture to
• 28:28
  really like fully understand all the different ways that you can
• 28:33
  leverage these these agents. I'll talk a little bit about
• 28:37
  this one. I'm I'm still trying to learn
• 28:39
  this myself. Actually, we have a PhD student
• 28:43
  in our team who is amazing at these kind of workflows
• 28:47
  specially. So I wish she was here with me
• 28:50
  to answer this, but I will try to get to this a little bit
• 28:52
  based on my understanding, let's say auto GPT per SE, as much as
• 28:58
  I know what it does is it prompts itself.
• 29:00
  How would you do this? And then based on on those
• 29:05
  prompts, it creates multiple agents, right?
• 29:08
  Yes, this could help. But when we think of a
• 29:11
  production level LLM, we can't be doing that every time, right?
• 29:15
  Because it's going to have a lot of trial and error to get things
• 29:18
  done. It may think get things done
• 29:20
  right, but it's going to cost us a lot.
• 29:22
  But what I want to focus on with my paper is how would you do
• 29:29
  that? Fine tuning of not just the LLM,
• 29:32
  but like fine tuning of creation of those models, creation of
• 29:36
  those workflow for your use case.
• 29:39
  So focusing on that instead of just having prompts, creating
• 29:45
  prompts, and just assigning the prompts to each LLM, if that
• 29:51
  makes sense. Yeah.
• 29:52
  And I've heard that you have much more success when you make
• 29:56
  it very, very like pointed or almost like focused and closed
• 30:07
  system. You don't want to give it the
• 30:09
  ability to come up with whatever plan it wants.
• 30:11
  You want to really make it closed and have clear guardrails
• 30:18
  on it. Yeah.
• 30:19
  For now, that's what we are actually doing with our
• 30:22
  production LLM that we have, but I don't think that's just enough
• 30:28
  to make it even more reliable. Well, I would say 80 to 90% of
• 30:33
  the cases it does come up with reliable output.
• 30:37
  But the problem right now is one thing an open AI, open AI model
• 30:42
  will change and then it just messes up everything that we
• 30:45
  have planned for. So that that's what we I want to
• 30:48
  be able to fix with the with the research that I'm going to work
• 30:53
  on. It doesn't have to be open AI,
• 30:54
  it could be any model. And how do we make sure that we
• 30:59
  can get this process done more quickly is also one of my goal
• 31:04
  using any open source LM for said.
• 31:07
  Because if we are splitting task into multiple agents, we don't
• 31:11
  probably need GPD 4 level or GPD 4 O level agents.
• 31:17
  We could probably work just with Llama 7, right?
• 31:21
  And. We'll allow us to 7 billion,
• 31:24
  sorry. Yeah, yeah.
• 31:28
  So I'm trying to wrap my head around this and think about how
• 31:35
  that would work so that you can be model agnostic.
• 31:40
  And like, have you found success in that?
• 31:44
  Because it feels like the abstraction layer sometimes, you
• 31:47
  always have to be weary of the model that the underlying model
• 31:52
  and the capabilities that it it has.
• 31:55
  We do need to test 100% before thinking, OK, I'm going to go
• 32:00
  with this model because not all models are same.
• 32:04
  Like every model has their own capabilities, right?
• 32:08
  So it does seem like OK, GPT 4 is the answer to pretty much
• 32:12
  everything, except it messes up when we start to actually like
• 32:18
  put it in production. But first, it's bit expensive.
• 32:25
  It may not be that expensive for just few tokens or queries that
• 32:31
  you do. But when you're thinking about a
• 32:33
  production level LLM, it starts to get very expensive.
• 32:37
  So that's why for a smaller level task like when we're,
• 32:41
  let's just talk about document parsing for say, if I want to
• 32:44
  just parse a document, I, I don't have to have an LLM do
• 32:49
  everything pre processing we could do with OpenCV and other
• 32:54
  libraries, right? Then we can just maybe extract
• 32:58
  using Tensorflow like some kind of model that's our pre trained
• 33:02
  model, right? And once we extract, then we
• 33:05
  actually start to create a workflow with analytis to get to
• 33:10
  our results. So we don't have to have a very
• 33:15
  capable model, but a very capable workflow.
• 33:18
  This agent workflow that we have needs to be fine tune to our
• 33:24
  actual necessity, like what I actually want out of that
• 33:28
  document because I may not need everything out of that document.
• 33:32
  But at the same time, if I ask, hey, GPT 4, can you just get
• 33:38
  this from this document? And it's probably going to not
• 33:42
  always give that exact result when you just prompted to do
• 33:45
  that. So you're looking at it more in
• 33:49
  terms of pipelines and how you can create a robust pipeline as
• 33:53
  opposed to a robust prompt. Yeah.
• 33:57
  Yeah, makes. Sense So now my now my focus has
• 33:59
  shifted more towards getting to that level because I think
• 34:05
  that's where most like important results would come out of LLM
• 34:11
  because if you can't just use this in production, then there's
• 34:16
  almost no use, right? Yeah.
• 34:19
  Even if it does do all this stuff that we do as humans, or
• 34:24
  if it is as aware as we are and it has confidence scores and it
• 34:28
  knows it's going to get something wrong or right, that
• 34:33
  all doesn't really matter if at the end of the day you can't use
• 34:36
  it in production. Exactly.
• 34:38
  Exactly. Yeah, well, this has been
• 34:41
  awesome, dude. I appreciate you coming on here
• 34:42
  and chatting with me. And as always, I love talking
• 34:47
  about evaluation. And so I like the different spin
• 34:51
  that you bring to it, looking at the capabilities of the models
• 34:55
  themselves as opposed to the system or the benchmarks and
• 35:00
  hearing about your next steps for testing agents and
• 35:05
  evaluating the agents and looking at the agentic workflows
• 35:09
  as pipelines is super cool. Hopefully, once you've done
• 35:14
  that, you can also let us know how that goes and what some of
• 35:17
  the findings are. Once you get that paper out to
• 35:20
  I'll make sure to read it and give a little review and
• 35:25
  probably post about it. Sure.
• 35:27
  Thank you.