Latest / Agentic Conversations (formally mlops.community) / Thousands of Fine-Tuned Models

Transcript

Thousands of Fine-Tuned Models · Agentic Conversations (formally mlops.community)

• 0:00
  Slow data is more when I want to emulate a decision making
• 0:03
  process. A lot of things are beyond sort
• 0:05
  of prompting me, but if an agent or a system observes you for
• 0:09
  long enough, it can pick up on the intrinsic patterns of how
• 0:14
  and why you are making certain decisions.
• 0:19
  There's the whole small language model kick that you've been on.
• 0:23
  And then there's this other idea like I use as someone that works
• 0:29
  at a company, I have the internal uses of AI that
• 0:33
  hopefully I'm enabling with the agents.
• 0:36
  But then I also have just like when I interact with a chatbot
• 0:40
  directly and usually I'll interact with Gemini and ask it
• 0:44
  questions. Correct.
• 0:45
  There's different use cases where you're doing general
• 0:47
  purpose stuff. You go to Gemini, you go to Open
• 0:49
  AI, you know you want to summarize a document.
• 0:51
  Yeah, I don't message my colleague to summarize a
• 0:53
  document for me. But when I want my colleagues
• 0:55
  opinions on Asus compliant with what our company is doing, like
• 0:59
  if I'm going to adopt this and we're going to push this out, I
• 1:01
  go to my compliance officer because he knows information
• 1:05
  about the company that nobody else does.
• 1:07
  And that's that's what we want to capture in these models.
• 1:10
  Why do you need a small language model or why do you just need a
• 1:13
  fine-tuned model for that as opposed to some rag based system
• 1:17
  or something that is is a little bit more simple to set up.
• 1:24
  I could say, for lack of a better word, because I know
• 1:26
  you're trying to make like this fine tuning much more simple to
• 1:30
  set up. So I don't want to like, say
• 1:32
  that it's not simple, but it feels like if you're trying to
• 1:37
  fine tune small language models, you're adding a bit more
• 1:39
  complexity. Yeah, I think we've taken care
• 1:43
  of adding the more complexity part for you.
• 1:45
  So you don't have to really, you know, take the engineering
• 1:48
  overhead of fine tuning models. And I think fine tuning in RAG
• 1:51
  are complementary. At the end of the day, when you
• 1:55
  want to add static memory sources to an LLM to refer to
• 1:59
  during in France, you could do a RAG or you could do Energetic
• 2:02
  Memory Framework to supplement it.
• 2:05
  But the model itself, what should it be sending as a query
• 2:09
  to the RAG system? Is it aware of the overall
• 2:12
  context? One of the issues with Asian
• 2:14
  tech RAG is a lot of times because you're doing only
• 2:17
  semantic similarity, framing the right questions or framing the
• 2:20
  right intermittent questions becomes really difficult.
• 2:22
  And that's where you want to do fine tuning on large context.
• 2:25
  That's all your enterprise data you've collected so that it
• 2:28
  knows what right questions to ask for the RAG to go and
• 2:31
  retrieve memory. Because fine tuning you fine
• 2:34
  tuning models does not evolve as fast as RAG data sets evolve
• 2:37
  right RAG data sets. You can add new context every
• 2:39
  day, but then what questions should and data should be
• 2:42
  retrieved from the RAG system is what a fine-tuned model would do
• 2:45
  much better instead of if you just use like a vanilla based
• 2:48
  model. So then do you have some kind of
• 2:50
  framework that you think about with fine tuning certain data
• 2:56
  versus let's throw that into the almost like fast data, you've
• 3:01
  got the fast and slow thinking or you how do you look at like
• 3:06
  we're going to make that a fine tuning thing versus we're going
• 3:08
  to make that a rag thing? You do this is a RAG thing when
• 3:13
  there's a lot of factual information that needs to be
• 3:16
  recovered or that needs to be given out.
• 3:17
  So if you're doing customers, something like customer support,
• 3:20
  then you know, a RAG agent or a RAG system is really good
• 3:24
  because you can refer to similar semantic queries and then you
• 3:27
  can really quickly get around and then retrieve them and then
• 3:30
  sort of give it to the user. And as new use cases get added,
• 3:33
  as my database is growing, it's plugged into the RAG and I can
• 3:36
  recover them and I can give it. So I'd say yes.
• 3:38
  So that's where I use fast data. Slow data is more when I want to
• 3:41
  emulate a decision making process, right?
• 3:44
  When you want to sort of see, OK, what considerations go into
• 3:48
  making this decision. It's not really black and white.
• 3:50
  It's if you consider multiple variables as you make those
• 3:54
  decisions and fine-tuned models are really good at emulating how
• 3:58
  you would think to make those decisions.
• 4:00
  So what are the more mission critical operations you know?
• 4:02
  And for those we go for fine-tuned models.
• 4:05
  And then ideally, in the best case scenario, you wouldn't have
• 4:08
  them both working together because then the fine-tuned
• 4:11
  model can refer similar cases, draw inspiration from them, and
• 4:15
  then say, OK, this is what we've done.
• 4:17
  But the problem I have is completely new.
• 4:19
  So I'm gonna, I know what our framework and our embodiment of
• 4:22
  thinking is. Here's some things that people
• 4:24
  have been doing. Now, based on all this info,
• 4:26
  here's my suggestion or here's my recommendation because I like
• 4:29
  to think of models really as just humans.
• 4:32
  Like how would you do? Like, OK.
• 4:34
  And my, if you ask me, I have notes.
• 4:37
  I carry my notes with me everywhere.
• 4:38
  And my notes are like my rag. You know, I, I have written
• 4:41
  things written down. And when someone says, oh, let's
• 4:44
  talk about vaccines, Oh, sure, I have something written about
• 4:46
  these vaccines. I know you should do.
• 4:47
  But if somebody asks me what's your opinion on getting
• 4:50
  vaccinated? Ah, OK.
• 4:52
  It's something that I've thought about.
• 4:53
  It's something I've known, something I've gathered from
• 4:55
  multiple sources. And that's where my fine-tuned
• 4:57
  model kicks in. Right.
• 4:58
  So it's the different kind of problem statements you have,
• 5:00
  different approaches you would take.
• 5:02
  Thank you for getting this podcast banned on YouTube now
• 5:05
  for mentioning vaccines. I appreciate that one.
• 5:08
  That's. Not on purpose.
• 5:12
  OK. But we were also talking about,
• 5:14
  so I like this how you're thinking about the different
• 5:18
  decision making process and almost like what my values are
• 5:23
  versus there's this fast stuff that I can reference and I, I
• 5:28
  don't necessarily need to always have that in mind.
• 5:31
  I can reference it. I know where to go to find it.
• 5:33
  And we also mentioned about the, what were we calling it, like
• 5:38
  the AI workers, the AI workforce.
• 5:40
  We didn't really have a good term for it.
• 5:42
  Yeah, because it's, it's so difficult to put a term to it.
• 5:45
  Like what do you call them? It's almost everything.
• 5:48
  An AI assistant, An AI. Agent, workforce, and none of it
• 5:53
  sounds really appealing. And none of it sounds human.
• 5:56
  Yeah. And the issue with AI systems is
• 5:58
  there are low trust systems in a high trust environment, right?
• 6:02
  People don't like to read documents written by ChatGPT.
• 6:06
  Like we were talking about this earlier.
• 6:08
  You when when somebody sends a doc across to you and you see
• 6:11
  those double hyphens between the text.
• 6:13
  Or emojis galore. Galore and your internal
• 6:16
  classifier goes like Nah, I'm not going to pay attention, but
• 6:19
  then you ignore all the effort that went into writing the
• 6:23
  document. Yeah, well, it feels to me
• 6:25
  whenever I get one of those is like, oh, this person just
• 6:27
  phoned it in, they prompted it and now I have to be the one
• 6:31
  that spends my time on this jargon that got output.
• 6:37
  Yeah, yeah, it's so true. And and that's where I think
• 6:42
  anyway, it's coming back to the coming back to your point and
• 6:45
  I'll come back to this in a second is when you mentioned the
• 6:47
  AI agent AI workforce example, right?
• 6:51
  I think making it more human would solve sort of this problem
• 6:56
  of also making it more consumable.
• 6:59
  Because when when we talk about these sort of concepts of AI
• 7:04
  assisted work or AI assisted workflows or AI enabled
• 7:08
  workflows, you also want them to be not only think like a human,
• 7:13
  but also behave like a human, right?
• 7:15
  So a human would send you a Slack message about something
• 7:18
  that you asked. And the worst kind of humans
• 7:20
  would send you a Slack message that just says, hey, you around.
• 7:25
  I hate those I hate. Those, they don't give you any
• 7:27
  context or anything and it's just like, no I'm not fuck off.
• 7:33
  Anyway, sorry to derail that. No, and and I think a lot about
• 7:37
  building these AI agents and enterprise AI agents is about
• 7:42
  building the interaction design around them.
• 7:45
  You can build the best model like open AISG, PT-4 OS are
• 7:50
  they're great models, but their interaction design is, is not
• 7:54
  something that makes them makes people comfortable in working
• 7:58
  with them, right? We developed these biases
• 8:00
  towards them and we've sort of as a society developed this bias
• 8:03
  right now about documents generated by Chajibity that we
• 8:07
  will not read. And you, you said rightly, it
• 8:10
  almost feels like they're just throwing it my way and they want
• 8:12
  me to review it. They've not even taken the pain
• 8:14
  of deleting the emojis. So I think a lot about this
• 8:19
  workforce, while there's the technical part of it, there's
• 8:21
  also like a huge design aspect and the nomenclature aspect.
• 8:25
  Like what do we call an AI workforce enterprise agent
• 8:30
  thing? Yeah, it's, it's, it's, it's
• 8:32
  something that even we're still figuring out where we point the
• 8:36
  finger. But how do you plug in the small
• 8:38
  language models into that? And I've heard it referred to as
• 8:42
  almost like if you look at the jobs to be done framework and
• 8:46
  you look at what I do as a marketer, I'm moving around data
• 8:51
  and transforming data. And ideally you can get an agent
• 8:56
  to do some or the most amount of that data transformation,
• 9:02
  moving, presenting it in different ways.
• 9:06
  So enterprise agents or small language models working with
• 9:11
  agents, right? I think it's important to define
• 9:13
  what an agent is. An agent is essentially, I think
• 9:17
  in my opinion, three parts. Once you have the intelligence
• 9:20
  layer, which is more of the decision maker, then you have
• 9:23
  the memory layer which learns and remembers things.
• 9:26
  And the third bit is the actions.
• 9:28
  What can the agent do? Can it perform something?
• 9:31
  Can it read? Can it write?
• 9:32
  Can it make a phone call? I think it's, it's these three
• 9:35
  things together that sort of make an agent in any environment
• 9:38
  in any setting. Now, small language models fit
• 9:42
  into the intelligence layer of the agent, right?
• 9:46
  And small language models off the shelf are really not that
• 9:50
  intelligent. I mean, when I say small, I mean
• 9:52
  anywhere between, I'd say 1 to 7 billion parameters in size.
• 9:57
  And they're not really very intelligent of the shelf, but
• 10:00
  they're really good students and they're really, really good at
• 10:03
  absorbing knowledge if you teach them the right way, right?
• 10:07
  How you train. When somebody new joins your
• 10:10
  company or when you have an intern fresh of university
• 10:13
  coming in and you want to train them and how the organization
• 10:16
  works, they're fresh. They're ready to learn.
• 10:18
  A small language model is like that.
• 10:20
  If you feed it the right data of how you perform certain
• 10:25
  processes, you can fine tune it to emulate that behaviour.
• 10:30
  And it's important to use a small language model and not a
• 10:33
  larger one here. Because small language models
• 10:37
  are easily trainable, they require much less compute.
• 10:40
  You can fully fine tune them instead of just building low
• 10:43
  adapters on top of them, which you end up doing with the larger
• 10:46
  models, but smaller ones, you can do it entirely.
• 10:49
  That helps you instill much more deeper insights into the model
• 10:52
  itself. When you're doing these fine
• 10:54
  tunings, you can run RL on them as well.
• 10:56
  So if you want to do reasoning, thinking, you want to run some
• 10:59
  GRPO chains on it, you can do that effectively, cheaply and
• 11:05
  fairly fastly as compared to if you do it with a much bigger
• 11:08
  model. And for SLMS working in
• 11:12
  enterprises, it's essentially about capturing process
• 11:15
  knowledge. You just said as a marketer, a
• 11:17
  lot of your job involves moving data from one place to the right
• 11:20
  place and then deploying it. But you do a lot of thinking in
• 11:24
  the middle, right? And this thinking is in your
• 11:26
  head. But if an agent or a system
• 11:30
  observes you for long enough, it can pick up on the intrinsic
• 11:34
  patterns of how and why you are making certain decisions and
• 11:39
  then instill all of that knowledge into a model.
• 11:41
  And that's what we try to do when we fine tune small language
• 11:44
  models is, is instill these intrinsic patterns into them,
• 11:48
  something you cannot achieve with prompting.
• 11:50
  Like a lot of times people confuse that.
• 11:53
  Can we just prompt them? But then a lot of things are
• 11:56
  beyond sort of prompting. Well, and this is where that
• 12:00
  like high trust, low trust comes in because if I am going to let
• 12:05
  someone observe me at my job, first of all, that makes me
• 12:10
  uncomfortable just thinking about it.
• 12:13
  But second of all, I really want to make sure that I am not
• 12:21
  giving up trade secrets from 1 angle.
• 12:25
  And the other angle is that I'm instilling the right things into
• 12:28
  the model. True.
• 12:31
  I think it's I also hate being monitored like nobody wants to
• 12:36
  be seen what you're you know what you're doing when you're
• 12:38
  doing. But creepily, it's becoming the
• 12:41
  norm that we are letting sort of observers into our ecosystem.
• 12:46
  You know, when you go to a Google Meet or now even a Slack
• 12:51
  hurdle, there's this AI which just creeps in and it's
• 12:54
  listening and it's making notes. I'm like, so it's slowly
• 12:59
  sensitizing us to the fact that there's going to be these latent
• 13:03
  observers behind the scenes collecting data on them, and
• 13:07
  it's only going to, I'd say, grow from here.
• 13:10
  Yeah, we are. Microsoft launch products where
• 13:12
  they sort of record your screen and they collect information.
• 13:15
  You're even wearing the meta. Glasses.
• 13:17
  I'm wearing the meta glasses. Yeah, and it's becoming more
• 13:21
  popular, like pendants that will record your whole day.
• 13:23
  Exactly. So constant data capture is
• 13:26
  becoming a norm, but the most important part is where does
• 13:30
  this data go and what really happens with that data, right?
• 13:33
  And this is where we've got to be really, really careful and we
• 13:37
  need to give individuals whose data is being collected, the
• 13:41
  authority and the option to control where this data flows
• 13:45
  and how it's being used, right. And, and as we're, as we're
• 13:49
  building Prem, that's one of the things that we're doing very,
• 13:52
  very carefully is when we help you build these small language
• 13:57
  models or even medium sized language models built on your
• 14:00
  data. Our promise to you is that this
• 14:03
  data does not leave our ecosystem is only exposed to
• 14:07
  open source models that we host and we run.
• 14:10
  And then we have like a seven day delete policy after that,
• 14:13
  that once your models are trained and fine-tuned and you
• 14:16
  walk away with them, we delete all of the data from our
• 14:19
  infrastructure, from our logs. So then you know, and you're
• 14:22
  confident that it's not being used in a manner that you're not
• 14:25
  aware about. And that's the most scary part.
• 14:27
  I mean, not knowing where your data is going is, is worse than
• 14:31
  knowing that somebody is doing something mean with your data.
• 14:34
  And and that's what I think we're very careful about.
• 14:36
  Yeah. And so getting back to the
• 14:39
  capabilities that you're trying to unlock with this almost like
• 14:42
  virtuous cycle of observer, seeing how you're making your
• 14:47
  decisions, picking up these patterns and then creating
• 14:54
  workflows or agentic workflows out of it.
• 14:56
  Like how does, what's that next step of, OK, I see that you do
• 15:00
  XYZ whenever you pull up Google ads.
• 15:04
  Now you're creating a workflow, or you're fine tuning a small
• 15:09
  language model that can reason on Google Ads.
• 15:11
  Like, yeah, land the plane for me.
• 15:14
  Great. So I think this there's three
• 15:18
  things in building this virtual workforce that we talk about.
• 15:22
  It's data collection, it's model fine tuning or knowledge
• 15:26
  distillation in particular. And then it's using them in
• 15:29
  action. So we covered data collection,
• 15:31
  right. Data collection can happen on
• 15:33
  your Google meets, on your notes, on your Slack Notion
• 15:36
  process documents. We're.
• 15:38
  Gonna just screen just it's watching your.
• 15:40
  Screen. Just watching your screen
• 15:41
  exactly, taking screenshots, understanding the steps that you
• 15:45
  take to reach from point A to point B that look like success
• 15:49
  for you. That's the jobs to be done.
• 15:50
  And then how are you doing the jobs?
• 15:52
  That's data collection. There's data parsing, and then
• 15:54
  there's building the data sets, right?
• 15:56
  So imagine we've built a data set to fine tune LLM.
• 16:00
  Usually data sets to fine tune LLMS are conversational data
• 16:04
  sets. So they would be like, hey, I'm
• 16:05
  at this stage and on I see this. What am I doing next?
• 16:09
  And the user replies oh OK I would click on this.
• 16:12
  So it's it's a data set for a specific task or it's a data set
• 16:17
  for your whole day, or like what does that data set look like and
• 16:21
  how do you curate it? I think that data set is for
• 16:24
  your role and your job. So if you're a marketer and
• 16:29
  you're Demetrios, it would be Demetrios's model.
• 16:32
  It's Demetrios's way of operating when he's doing some
• 16:36
  work. So it's like your digital clone?
• 16:38
  My sidekick. It's your, your sidekick.
• 16:40
  It's your sidekick was always with you, right?
• 16:42
  So this data set really captures how does Demetrius work and then
• 16:47
  why he does the things that he does.
• 16:49
  So, so that's the data part of it.
• 16:51
  Then on the fine tuning part of it.
• 16:53
  So with Prem, we've built this product called Prem Studio
• 16:57
  wherein you can feed in all of this data and then we take care
• 17:01
  of building synthetic data for you that can be used to fine
• 17:04
  tune the SLMS and then also selecting which SLM is the best
• 17:09
  for you to fine tune and then running the jobs for you as
• 17:12
  well. So we do all of the ML OPS
• 17:14
  monitoring and sort of taking care that the GP us are
• 17:17
  provisioned, you know, all the different, if you're doing any
• 17:21
  RL, we take care of running RL experiments as well.
• 17:24
  So that way you can train a bunch of different open source
• 17:26
  models and then once you get them back, we evaluate them on
• 17:30
  some of the data set that we'd held out as our test data set
• 17:33
  early on. And the goal at the end of the
• 17:36
  whole training process, like what's, what's your training
• 17:38
  objective, right? Your training objective is that
• 17:41
  if given a situation and certain context about the situation as
• 17:46
  the input to the SLM, the SLM should predict the next step of
• 17:50
  what needs to be done, right? So that is what we do now.
• 17:55
  How does this go into action? Right until now, agentic
• 17:59
  interfaces have been really chat first.
• 18:02
  You know, you, you talk to an agent and it gives you some
• 18:04
  information back. But that's not, again, that's
• 18:06
  not how Dimitrios works. Dimitrios does stuff.
• 18:09
  He's more than just talking. I like that action oriented.
• 18:14
  So. So now the next part is having
• 18:18
  these action, having these sort of models deployed and giving
• 18:22
  them access to a set of tools. So if the set of tools that you
• 18:26
  want to run this model on Prem and then give it access to like
• 18:29
  your calendar, your Gmail, maybe your Slack, maybe your GitHub,
• 18:34
  right? And then you control the
• 18:36
  permissions of what the model can do for you.
• 18:39
  And then you just execute and you say, OK, now it's running.
• 18:42
  It's always on. We can give it an e-mail address
• 18:46
  or we can give it a Slack handle.
• 18:48
  It can join your Slack channel as one of your team members.
• 18:52
  And then in action, when you send it a message or when you
• 18:55
  send it an e-mail, it will emulate how you as an individual
• 19:00
  would be responding to it, right?
• 19:02
  And it would do that action if you send it an e-mail, it would
• 19:05
  read the e-mail. It would think how Dimitrios is
• 19:07
  going to respond to it, write back the response and send it
• 19:10
  back to you. And that's where we see it
• 19:12
  first. You can send it a document to
• 19:14
  review. Maybe somebody sent you a 5
• 19:16
  pager, you send it to him on Slack and say, hey, can you
• 19:19
  review this and tell me what you think about this?
• 19:22
  Or is this compliant? Or is this something that you
• 19:24
  know aligns with our company's goals, objectives?
• 19:27
  That's Phase 1 is where it's still conversational, but you're
• 19:31
  moving to multiple channels of conversation.
• 19:33
  You're going beyond just like 1 chat.
• 19:36
  You are. You can send it a voice note and
• 19:38
  it can understand the voice note and reply back to you in text,
• 19:41
  right? But it's how you would work with
• 19:42
  a Co worker. The next step after that, which
• 19:45
  we see coming in, let's say about a year from now is when
• 19:49
  you start giving it control over your systems and your tooling
• 19:53
  where the model can take control of a digital workspace and start
• 19:58
  doing things in browser on your screen using the tools that you
• 20:02
  use on a more day-to-day basis to perform more solid actions
• 20:06
  beyond conversation for you. And I think that's very
• 20:10
  important. It's I'm super excited about.
• 20:13
  All the work that's being done for computer use, for browser
• 20:16
  use, I think they're really, really great frameworks.
• 20:20
  And we see some really good models coming out as well.
• 20:22
  We've seen the agents come out to do it.
• 20:25
  But then a lot of the frustration of the people who
• 20:28
  are using these agents is like, he's not doing things the way I
• 20:31
  would do it or how I would do it.
• 20:34
  And that's what we want to solve for.
• 20:35
  And that's what we think is going to be the next evolution
• 20:37
  in this is are you sort of delegated tools to do stuff on
• 20:42
  your behalf it? Almost feels like you would need
• 20:44
  many small language models in a way that you're not only going
• 20:49
  to be using one for Demetrius and it feels weird referring to
• 20:55
  myself in the third person, but you're not going to have that as
• 21:00
  this is my small language model and it's my one that's been
• 21:03
  trained and retrained and continuously retrained on.
• 21:06
  It's almost like this is my Google Analytics small language
• 21:11
  model that my big brain model can call as a tool and or
• 21:17
  there's the web browser model that can be called as a tool in
• 21:21
  that way. Is is that kind of how you're
• 21:23
  you're thinking it's going to shake out?
• 21:26
  Yes, there would be two ways. There would be almost a
• 21:29
  hierarchical structure of models which would be experts in
• 21:33
  certain things. So you could look at it as a
• 21:35
  mixture of experts, but decoupled almost mixture of
• 21:39
  experts like in one. So we decoupled them and we have
• 21:41
  lots of small language models, but you'd almost have like the
• 21:44
  social media side of Demetrios, the social media expert.
• 21:48
  Initially when we are having a less amount of data, we would
• 21:51
  keep it confined to a singular model, right?
• 21:53
  Because then otherwise you, you become too thin across, you
• 21:57
  spread across different models. But as the volume of data grows,
• 22:00
  yes, you would want to split it out into different sort of
• 22:04
  individual language models that can take care of it.
• 22:07
  But on top of it, you may have your vision models which are
• 22:11
  parsing the screen and understanding what's on the
• 22:14
  screen. Where do I click?
• 22:15
  Where do I press? So that would be the sort of
• 22:18
  layering on top of it is to understand context, to
• 22:21
  understand different tools. Maybe we have something that can
• 22:24
  take control of your phone and do things on your behalf on the
• 22:26
  phone. So that would be the layer and
• 22:29
  then that would parse the information and then give it
• 22:31
  down to the ideal model depending on the application you
• 22:34
  are in that. OK, now how would Dimitrios
• 22:36
  maybe you know, do marketing on Telegram on his phone, he's in
• 22:40
  these five groups. What do we send?
• 22:42
  We've just launched a new. Tool.
• 22:43
  That's all it's. Always been about shilling.
• 22:46
  It's always. We're on Telegram.
• 22:49
  It's just shilling. Well, one other piece that I
• 22:52
  wanted to touch on was how you think about the differences
• 22:54
  between the workflows that are almost hard coded.
• 22:59
  And when you observe me doing something multiple times in a
• 23:04
  row, it's almost like you know that I'm going to do XYZ and
• 23:11
  there's that next best action. And you don't necessarily need
• 23:14
  to have the agentic capability be this open space that it can
• 23:19
  choose whatever it wants as an expect best action because
• 23:23
  you've seen me do it this same way five times.
• 23:26
  True. And so it can be a hard coded
• 23:28
  workflow versus oh, just go figure it out agent and then you
• 23:33
  have that reliability a little bit more certain.
• 23:39
  What you're talking about is almost like you can build
• 23:42
  process Markov chains where a Markov chain is essentially a
• 23:47
  set of events happening and then a probability connecting them
• 23:51
  saying, OK, this if these three things happen, what's the
• 23:53
  probability of us going into either of these one or two ways?
• 23:56
  And if if there are certain chains which are like not too
• 23:59
  divergent and they're always going straight, yeah, you can be
• 24:02
  certain that if these four steps were taken, this is what the
• 24:05
  next step is going to be. And I think that's going to be
• 24:08
  more on it's, it's going to, this is going to sound meta.
• 24:11
  But if, if you were to run an analysis on the data, not to,
• 24:17
  not to sort of emulate how you behave, but to understand, OK,
• 24:22
  what patterns do I see in this behaviour?
• 24:25
  I see a lot of noise or a lot of a different distribution for
• 24:29
  when you're working on Twitter because you're doing like 100
• 24:31
  different things. But then when you go to Google
• 24:33
  Ads, maybe there's just one workflow that you're always
• 24:37
  doing that and the distribution of data and that looks fairly,
• 24:40
  fairly just linear. It's very, it doesn't have a
• 24:42
  high amount of variance. It's really tight and in one
• 24:45
  place. So I think it would be
• 24:47
  interesting to run some experiments just on the data set
• 24:50
  that's collected to explore or to surface what these latent
• 24:55
  patterns are, and then maybe to see if we can control the
• 24:59
  inference space when the model is running in friends based on
• 25:03
  these inputs. Maybe there could be some work
• 25:05
  done with memory frameworks or some confidence scores being
• 25:08
  passed that. Hey, okay, if I'm at this step,
• 25:11
  has Dimitrios done something like in the past?
• 25:14
  How many times has he done something else?
• 25:15
  It's worth framing those questions, retrieving that
• 25:18
  information, and then also sort of taking the right step and
• 25:21
  controlling, controlling let's say our answer space during
• 25:24
  inference well. The other piece on this is that
• 25:27
  a lot of the things that I do are not just specifically done
• 25:31
  in the browser. And so there's taking data from
• 25:35
  the browser and then again, transforming it or downloading
• 25:39
  it to my local computer and then putting it on a program that I
• 25:42
  have. I'm thinking specifically about
• 25:44
  when I edit this video. Yeah.
• 25:46
  And I'm going to be taking the data and then putting it on to
• 25:51
  the Davinci Resolve. And then I do a few things.
• 25:55
  I'm going to import the different vocal tracks and the
• 25:58
  different videos, maybe do some color correction, but I'm also
• 26:01
  going to put them on one timeline is going to be just the
• 26:05
  full thing with a few edits. And then there's going to be the
• 26:08
  clips timeline. And so all of that is very
• 26:11
  menial work that I every time I do it, I'm like, I really wish
• 26:16
  at least like some kind of a template could be done here,
• 26:21
  because it's not necessarily that it's a template within
• 26:23
  Davinci Resolve. It's not as easy as that.
• 26:26
  It's true. It's like take these that are in
• 26:29
  this folder, upload them also upload these other the intro
• 26:34
  music and then create these different timelines and find the
• 26:40
  different clips. And then the clips are going to
• 26:42
  be verticalized. So there's all these things that
• 26:46
  I know could be templatized, But I guess the question that I'm
• 26:52
  trying to ask is there's the different abstraction layers
• 26:56
  that you play at. And that's just for one of my
• 26:59
  workflows. I imagine somebody that is a
• 27:02
  dentist that's trying to figure out their systems to work with
• 27:06
  and the insurance that you can pull from and trying to charge
• 27:12
  their client uses a whole lot of other systems, right?
• 27:16
  And so how can you enable the agents to take actions when the
• 27:22
  distribution of systems is so large?
• 27:25
  Yeah, I think when you move more into working with custom
• 27:29
  software it, it really starts getting challenging with less
• 27:35
  examples to train or fine tune a model on and then delegate to
• 27:39
  it. 100% responsibility to be autonomous and work on your
• 27:43
  behalf because you are making not just a big plan of
• 27:48
  execution, but at each step in the plan you're making a
• 27:51
  creative decision of what it should be like your video do I
• 27:55
  do like 3 seconds, what title font do I use?
• 27:58
  Colour grading or the lighting was bad that day and my other
• 28:01
  videos have this kind of colour grading.
• 28:02
  So you're making a lot of like these atomic decisions as you go
• 28:05
  along the process, right? And this is where I believe, I
• 28:09
  believe this is also a lot of creative work.
• 28:13
  Like there is creativity in every field, Engineering,
• 28:15
  medicine, video editing, it's a very creative process in such
• 28:20
  fields. I think it's really going to be
• 28:22
  human augmented agents that run. So you wouldn't give them
• 28:26
  complete authority and autonomity.
• 28:29
  Maybe you would make a plan for them.
• 28:31
  You would say, hey, OK, I want you to now edit this video.
• 28:36
  Here's the plan. I want you to do the next
• 28:38
  following 10 things. It says first it's okay, figure
• 28:41
  out a good title sequence. Then first 5 minutes I want you
• 28:45
  to focus on this, two on this, three on this.
• 28:47
  Maybe you give it that whole plan and then you let it go and
• 28:51
  do its thing. Today, we rely on these agents
• 28:54
  to make the plans for us as well.
• 28:56
  But I think it's it's not the right approach because when they
• 29:00
  fail and they fail miserably, a lot of times, there's not a lot
• 29:03
  you can do about it. So we need something where where
• 29:07
  me and the AI agent have like a common playing ground and it's
• 29:11
  something we can both read on. We can both riff on something.
• 29:13
  Maybe the agent can add to the plan and say, bro, I don't think
• 29:16
  these two steps are good. You want to add this so you vibe
• 29:20
  on the plan for a while and they're like, OK, now the plans
• 29:22
  ready, let me go and try and execute it.
• 29:25
  And maybe it's going to stop at some point.
• 29:27
  Maybe you get a notification saying, Hey, these I've done
• 29:29
  these four things. Do you want to review?
• 29:31
  And maybe this review step could be a part of the plan saying,
• 29:34
  OK, step one, Step 2, Step 3, review.
• 29:37
  Step three-step 4, review. So you work with it instead of
• 29:41
  just delegating and then going to bed and hoping it's done for
• 29:44
  some, for some domains you can do it, but it's not always like
• 29:48
  now you've got like coding agents, you've got cloud code
• 29:51
  would sort of go and you give it a sort of issue and it it
• 29:55
  creates APR for you. It writes the code for you, but
• 29:58
  you have to review it and you have to go go back and forth
• 30:01
  sometimes with it. And it would be the same with
• 30:03
  these sort of you bring the same analogies to these different
• 30:07
  spaces and these different tools, but it's just that these
• 30:09
  are more information sparse areas.
• 30:11
  They're more creativity rich areas.
• 30:13
  So we need to think of frameworks of how humans and
• 30:16
  agents would work together here. And I I guess the other thing on
• 30:21
  the information sparse piece is that a lot of this stuff doesn't
• 30:27
  have APIs that you can just ping.
• 30:29
  Yeah. And so integrating with these
• 30:32
  tools is so hard because it's not just all right, we have an
• 30:37
  API or an MCP server, these guys.
• 30:39
  And so how do you even think about that?
• 30:40
  And that's where a lot of the, I think people get excited about
• 30:44
  the browser use or the computer use because you can bypass the
• 30:48
  API and you just go into the GUI.
• 30:50
  However, I think anybody that's played around with computer use
• 30:54
  is like, yeah, it's cool, but it's also very hard to make
• 30:59
  consistently do things. True.
• 31:01
  So First off, I'm bullish on computer use and I completely
• 31:06
  acknowledge that the interfaces we have right now for two links
• 31:10
  as API MCPS, they are not for let's say consumer software or
• 31:16
  consumer products. They are more for SAS.
• 31:19
  All SAS tools have an API which is very easily consumable and
• 31:22
  because they were built for that purpose.
• 31:23
  But Premiere Pro probably doesn't have like a kick ass API
• 31:26
  you can edit a whole movie on. So because it's not made for
• 31:29
  that thing, it was never made to be used with an API, right?
• 31:33
  It was made for somebody to look at it, to see what's on it and
• 31:36
  then make those decisions. And I think this is where
• 31:40
  computer use will really become the norm for a lot of operations
• 31:44
  that we sort of work on with AI agents.
• 31:47
  And yeah, as somebody who is sort of always playing around
• 31:51
  with computer use, I, I completely acknowledge that
• 31:54
  sometimes it's too slow. Sometimes it rears off path.
• 31:57
  Sometimes it gets stuck on really dumb things.
• 31:59
  Like, bro, it's right there. Like, come on.
• 32:02
  Just just Scroll down the damn page exactly.
• 32:07
  And, and this is where sort of purpose specific training data
• 32:11
  is going to become super important.
• 32:13
  And this is where fine tuning will come into play.
• 32:15
  You, you cannot prompt it. You cannot always prompt it
• 32:18
  because your prompt will explore because there's so much context
• 32:20
  it has to go through, right? If you want long tail classes,
• 32:23
  it needs so much context. And this is where you want to do
• 32:25
  process fine tuning on this model.
• 32:28
  I see this and then I click here and then I do this.
• 32:30
  And then these are the chains. I think I, I, why am I clicking
• 32:33
  here? Why am I going there?
• 32:35
  You know, I, I also think there's going to be tool
• 32:38
  specific models that come out very soon.
• 32:40
  Computer use model fine-tuned on Photoshop, right?
• 32:45
  Imagine if somebody at Adobe sat down and collected and made this
• 32:49
  data. The query being, I want to, you
• 32:53
  know, add this hue to my image and give it a 1970s vibe, right?
• 32:58
  And then there's a screen recording of 5 minutes of a
• 33:01
  great designer doing this. That's your data point.
• 33:05
  So there'll be these process specific models or sorry,
• 33:08
  software specific models that would come out that would then
• 33:12
  help you do it. And then you can fine tune them
• 33:15
  further on your style. OK, this is the base model for
• 33:18
  Photoshop. I'm going to make a Demetrius
• 33:20
  version of the Photoshop based model because that's my style
• 33:23
  and my vibe. And then I make it run.
• 33:25
  Well, it, it also makes me think like, then where do you plug
• 33:28
  that in and how do you interact with it?
• 33:30
  Because we've seen and we know that the identification of the
• 33:38
  Internet and SAS is coming and it's almost already here in
• 33:42
  certain ways. Like if you're dealing with
• 33:43
  cursor, if you're dealing with lovable, you just talk to it.
• 33:46
  Yeah. And then it gets done.
• 33:48
  And so in Photoshop, I think Photoshop has kind of tried to
• 33:54
  do this, but it's more with image generation where you can
• 33:56
  generate an image. I haven't used Photoshop in a
• 33:59
  while, but I, I also, let's think about like Premiere Pro
• 34:02
  and I want to edit this or create a few snippets or
• 34:08
  whatever. I can I just talk to it and then
• 34:10
  it does it create a new timeline and then do this.
• 34:12
  Or I don't even need to be that specific because it already
• 34:15
  knows if I want to create clips, it's going to create a new
• 34:18
  timeline for each of those. But am I talking to Premiere
• 34:25
  directly? Am I talking to an overarching
• 34:29
  agent that then will use Premiere as a tool?
• 34:34
  Like where does that abstraction fit in, right?
• 34:37
  Because if you're thinking about it from the enterprise
• 34:40
  perspective, again, going back to we've been talking very
• 34:42
  consumerish. But if I now am in the marketing
• 34:47
  position and I'm trying to do these three things that require
• 34:52
  me to use a specific tool that we have at my company, maybe
• 34:59
  there's the API for that. Maybe not like you were saying,
• 35:02
  like SAS, there's great APIs because that's how we've been
• 35:05
  dealing with them, but maybe it's local to my own computer
• 35:11
  and that's how we have to use it.
• 35:12
  And so do I have my agent that is powering or my sidekick that
• 35:20
  is now going into each individual SAS tool and grabbing
• 35:24
  the data, transforming it, bringing it into the next SAS
• 35:26
  tool like or is it within the SAS tool I tell that tool do
• 35:31
  this? Yeah, it's, it's something we've
• 35:34
  been thinking about a lot. And I think where so we've come
• 35:39
  down to is that it's going to be an intermittent layer, which is
• 35:44
  more at an operating system level instead of the application
• 35:48
  level. You would have, let's say, a
• 35:51
  dynamic model orchestrator who would, given the task, of
• 35:56
  course, plan on what sort of models it would need to use to
• 36:01
  reach the final goal, right. So if, if on your screen there
• 36:06
  is Premiere Pro open or you wanted to eventually open
• 36:09
  Premiere Pro and do something that the planner would, this
• 36:13
  would be like an, an intermediate agent that's
• 36:15
  running in the middle. You would give it that
• 36:16
  instruction. His job would be OK.
• 36:19
  I, I first command, move the cursor, open Premiere Pro or use
• 36:23
  the CLI to open Premiere Pro. And then behind the scene, it's
• 36:26
  OK. Next step, load the Premiere Pro
• 36:28
  model memory, start using that for inference and then OK.
• 36:33
  Now look at the screen Premiere Pro model and tell me I want to
• 36:37
  achieve this goal. What do I do?
• 36:39
  What should I do next? And then once the task is done,
• 36:42
  that model gets offloaded. And then maybe you are you want
• 36:45
  to publish on YouTube. So like the YouTube specific
• 36:48
  browser use model gets loaded in and step one, open browser, go
• 36:51
  to youtube.com. Step 2GO Yeah, step three,
• 36:54
  create a title, create a title in Dimitrios style because it's
• 36:57
  fine-tuned on how you would give it a title.
• 37:00
  And that's where sort of this personalization.
• 37:02
  There's a whole thumbnail one. There's a whole yeah.
• 37:05
  So this it's going to be this family of models and and we're
• 37:09
  100% confident and that it's going to be a billion models
• 37:14
  fine-tuned for specific people and applications that are going
• 37:18
  to be live out there that will get hot swabbed at runtime and
• 37:21
  then be used for inference. right, right.
• 37:43
  Right. Bro, it's right there.
• 37:46
  Like that?