Latest / Agentic Conversations (formally mlops.community) / MLOps at the Crossroads // Patrick Barker & Farhood Etaati // #204

Transcript

MLOps at the Crossroads // Patrick Barker & Farhood Etaati // #204 · Agentic Conversations (formally mlops.community)

• 0:00
  I'm farhood. I don't spell out my lead.
• 0:05
  Last name is hard for you, I think.
• 0:10
  I'm from Iran, so I think I'm the first Iranian to like be on
• 0:16
  your podcast and I'm a data engineer slash ML OPS engineer
• 0:24
  ML OPS to be like I'm trying to implement MILFS here in
• 0:28
  Yorktown. It's it's an ad company in Iraq,
• 0:33
  and I like to have black espresso in the morning to just
• 0:42
  wake up. That's my main challenge.
• 0:45
  Yeah, I'm, I'm a man of tradition.
• 0:49
  Hi, I'm Patrick Barker. I am the CTO of Gitaros.
• 0:53
  We are building 1/3 of AI agents and I don't really drink.
• 0:58
  Coffee makes me crazy and I like shake really bad when I drink.
• 1:01
  I think I'm like allergic to or something.
• 1:03
  So I actually just eat a tonne of chocolate every morning.
• 1:06
  Just bars of chocolate. I eat like 2 bars of like dark
• 1:09
  chocolate a day. So.
• 1:13
  Boom, Shakalaka, we're back with another MLM's community podcast.
• 1:17
  I'm your host, Dimitrios, and today we got into it.
• 1:20
  It got a little bit of spicy because we had a nice little
• 1:25
  thread getting kicked off. And in that thread there were
• 1:29
  two some, there were two very opinionated people.
• 1:32
  So they were kind enough to agree to come on to this podcast
• 1:37
  here and talk about their feelings on the certain subject
• 1:42
  of fill ups and the evolution of Milos.
• 1:47
  What the difference between, quote unquote, LM OPS is if LM
• 1:53
  OPS is even a thing or if it's just a buzzword to get money
• 1:56
  from investors for a tool. We talked about all of it and I
• 2:01
  really felt like there was some insightful things that Patrick
• 2:04
  and Fought both said on each side.
• 2:08
  I found myself agreeing with both of them, even though they
• 2:10
  kind of were very different opinions and almost different
• 2:14
  sides of the spectrum on the opinions.
• 2:16
  I'll let you judge for yourself what you thought of it as a
• 2:20
  little bit of background with Fallout.
• 2:23
  He is the first dude that we've had on here from Iran, which is
• 2:29
  incredible in my eyes. We're getting people from all
• 2:31
  over the world. It just shows you how the
• 2:35
  community is so global and it is so cool to see what people are
• 2:41
  working on in each little nook and cranny around the globe.
• 2:46
  And of course, Patrick, Straight American.
• 2:48
  He's working with my buddy Dan Jeffries A1 of like the first
• 2:54
  people that joined the community back in the day in 2020.
• 2:58
  And he if you look up his virtual meet up, he did some
• 3:03
  really cool stuff on AI generated songs back in 2020
• 3:07
  before that was even popping off and it was a thing.
• 3:10
  So Patrick's working with him as CTO.
• 3:14
  Dan Jeffries is CEO of their new startup Kintaros.
• 3:17
  Before that, Patrick goes into what he was doing in the ML OPS
• 3:20
  world and then in the DevOps world and how these skills are
• 3:25
  quite different from where he finds himself today playing with
• 3:29
  agents and Fallout is a data scientist by trade.
• 3:34
  He's been doing data engineering and now he is killing it as an
• 3:40
  ML OPS role. And he took the side of I do not
• 3:43
  think LM OPS is even worth giving it any time or attention
• 3:48
  or mind. O let's let's just listen to
• 3:53
  this conversation and I would love to hear your thoughts and
• 3:56
  opinions by writing them in the comments.
• 4:00
  As always, the number one thing that I ask from you is to share
• 4:04
  this with a friend so that they can bask in the joy of listening
• 4:09
  to all of us. Let's Check out this bullshit I
• 4:19
  just got. Synthetic.
• 4:24
  For those who can't see me right now, I just turned my whole room
• 4:28
  into a disco, that is. It looks like a dog room from
• 4:34
  the movies. Have you all heard of Red Light
• 4:37
  Therapy? Oh yeah, it's not it's not the
• 4:40
  real red light therapy light. Yeah, I just got a red light
• 4:43
  therapy light. How's that working for you?
• 4:45
  Are you? Like, well, I just got it today,
• 4:47
  so I don't know yet. I feel like it's like part
• 4:51
  placebo, part actual therapy interpretation.
• 4:56
  I mean, there's like a little evidence for, right.
• 4:59
  I think they do use it sometimes in hospitals.
• 5:01
  Like ohh yeah. I think so.
• 5:03
  No, potentially. Yeah.
• 5:05
  Alright, I'll take it. Yeah, I'll tell you, I'll.
• 5:09
  Handle that, Yeah. So I'm going to be using it.
• 5:12
  Hopefully the occasional break from the desk and computer.
• 5:16
  I'm just going to stare into red lights and try not to go blind.
• 5:20
  But anyway, we're not here to talk about the latest and
• 5:24
  greatest biohacking. We are here because there was
• 5:29
  some things that were said on a thread in Slack and that was
• 5:32
  almost six months ago. There's been some updates, but
• 5:37
  the best part about it is that we all decided to get on this
• 5:41
  call and hash it out between each other.
• 5:44
  So I appreciate you 2 coming on here.
• 5:46
  And I know I guess it kind of started by Patrick.
• 5:51
  Dare I say, you said some outlandish stuff.
• 5:54
  Maybe we can kick it off there. And I know we were just talking
• 5:57
  about it right before we started and you were like, ohh, yeah,
• 5:59
  now I remember this thread. I have strong feelings about
• 6:02
  this stuff. So what's where are the strong
• 6:05
  feelings coming from, man? Break it down for us.
• 6:08
  Yeah, I think far ahead. I posted that LM OPS is more of
• 6:11
  just like a hype term for who see money.
• 6:15
  I feel differently. I think Dele Mops is like maybe
• 6:18
  almost a totally different field that is, you know, there's some
• 6:21
  overlap with them all OPS than what we traditionally did there.
• 6:24
  But I think, yeah, the majority of what I see kind of being used
• 6:28
  there is wildly different. So we kind of got into doing the
• 6:31
  thread and yeah, and now we're out of podcast so.
• 6:37
  Then we will share that thread too.
• 6:39
  But I almost yeah. So I'm very undecided on the
• 6:44
  topic. I will say that right now.
• 6:46
  But Farone, where? Where are you coming from?
• 6:49
  What's your stance on things? I actually think that alone is
• 6:55
  like a very good thing. It's like the obvious.
• 6:59
  It's pretty obvious that it has like given opportunity to many
• 7:04
  people to try new things and provide new services and optics
• 7:10
  that like that. But I think like Emma loves, at
• 7:14
  the core of it is like a mindset.
• 7:16
  It's a mantra. So I I cannot think like of LNG
• 7:24
  OPS as like something completely new, maybe another paradigm in
• 7:29
  the field. But I think like all these
• 7:32
  startups, all these people were like talking about it.
• 7:36
  Talking about it and talking about it are like blowing into a
• 7:41
  bubble. I've seen it too many times in,
• 7:45
  like other fields. First it was cloud.
• 7:48
  Yeah, cloud. It was great.
• 7:50
  But we we all remember the cloud hype.
• 7:53
  Like even the Xbox wanted to have games on cloud, like in
• 7:57
  2012. Come on, we cannot do that.
• 8:00
  So there's like this, we see optimism.
• 8:05
  I'm all for like technical people providing stuff, doing
• 8:10
  stuff, like talking about their challenges.
• 8:12
  But I think like at these moments the field gets hurt
• 8:17
  because some people are too opportunist.
• 8:20
  So at the end of the day, I'm pessimist about this stuff.
• 8:24
  It's kind of bubbles, yeah. So it's like over promising and.
• 8:28
  Under yeah. Deliver.
• 8:29
  Yep. What we're promising out there,
• 8:31
  delivering and hindering actual technical people who are like
• 8:35
  working with outlandish goals or outlandish claims and things
• 8:42
  like that. It's like I I get that we have
• 8:46
  to talk about Emma's for alms. I get that and I get that Even
• 8:53
  the term laps is a bit like it's only trying to get its feet.
• 8:58
  Like really, really recently that it's Yeah, if you talk with
• 9:04
  DevOps people, many of them like are so dismissive of the
• 9:09
  analogue term and even in companies, big companies, that
• 9:14
  the term is not the same like what Emma loves means in
• 9:18
  Microsoft, Google, Facebook does not map to what a medium sized
• 9:24
  company wants or things like that.
• 9:27
  So they feel this very young and trying to like have these
• 9:33
  special specialisations and not to look generally at problems we
• 9:39
  are trying to solve. It's like a it's like the gold
• 9:44
  rush. I think of it as a gold rush.
• 9:47
  Ellen is the current gold and we all know it provides so many
• 9:52
  possibilities. But L apps or things like that,
• 9:56
  even like a new paradigm like prompt engineering.
• 10:00
  I'm pessimistic about that. I know that people can do that,
• 10:05
  but I don't think like in five years time we have something
• 10:09
  like prompting. The main not.
• 10:11
  Just ignoring them pessimists. But I'm a bit like the sceptic,
• 10:18
  The sceptic where I think about this.
• 10:21
  It does feel a little bit like Rome.
• 10:24
  Engineering is an artefact that hopefully will go away.
• 10:28
  And I do feel your pain. On the LM OPS is already a
• 10:34
  branch of something that isn't exactly a branch of something
• 10:38
  else. It's like a branch of a branch
• 10:39
  that never existed. So where's the foundation There
• 10:43
  you could see that. I do feel like when you start to
• 10:47
  throw data into things it gets a little bit more tricky.
• 10:51
  SO11ML OPS in my book. Of course I'm talking my book,
• 10:57
  because otherwise, like what have what's the whole last 3-4
• 11:03
  years of my life been built on if not MLM's community?
• 11:07
  But I do feel like there is some definite distinct differences
• 11:12
  between DevOps and MO OPS. That doesn't necessarily mean in
• 11:16
  my eyes, that you can't be a great DevOps engineer and then
• 11:21
  easily transition into being a great ML OPS engineer.
• 11:24
  And so I feel like that is a really great way to make that
• 11:29
  transition. But I do 100% here you on two
• 11:32
  things that I want to mention. One is that the term ML OPS
• 11:38
  means something completely different for a gigantic
• 11:42
  company, these Fang companies that enterprises versus a
• 11:48
  midsize company and the needs that someone who is in that role
• 11:52
  because now we're starting to see these ML OPS engineering
• 11:55
  roles crop up and the needs for the people that are in these
• 11:59
  roles, it's wildly different depending on which company is
• 12:06
  asking you for that. So I will agree on that 100% and
• 12:12
  but the thing that I also want to agree on before we jump is
• 12:16
  the idea of LM OPS is now the hot buzzword of the moment.
• 12:25
  And so if you throw that onto your pitch or onto your whatever
• 12:31
  you're going to be talking about, it is easier received, I
• 12:39
  think, out there in the Internet and space and all that fun
• 12:44
  stuff. Yeah.
• 12:45
  I don't you know I I don't doubt that there's hope and there's
• 12:48
  obviously I I think it's good for the ecosystem of I put we
• 12:50
  want to have a lot of directions and then you know eventually
• 12:53
  like find the pieces that work because this is entirely new
• 12:56
  technologies So I don't disagree that there's there's a lot of
• 12:59
  noise and there's a lot of things being built that are
• 13:01
  completely useless. They're probably pulling people
• 13:03
  down paths that aren't good but this is a totally new space we
• 13:05
  need explored 5 out You know from my perspective I was in MLE
• 13:09
  of one medical you know before my current role and you know
• 13:12
  doing a lot of ML OPS you know you know ML kind of one our
• 13:16
  stuff allowed goal which was you know basically early you know
• 13:20
  piece stuff and anomaly detection you know a lot of like
• 13:23
  XG boost type stuff on on tabular data and the skill set
• 13:27
  that I had there which I feel like I got pretty good at you
• 13:31
  know my next role which has been found in a company we were
• 13:34
  building a Jeremy I space and I've you know worked with a
• 13:36
  bunch of founders Jeremy space and I feel like that skill set
• 13:39
  that I learned one medical resembles just not at all
• 13:42
  applicable to ella bobs. I love that.
• 13:45
  Hold on, break that down for me real fast because the skill set
• 13:48
  that you were doing, what was your day in and day out at one
• 13:51
  medical? And what's your day in, day out?
• 13:54
  Now I know that you're playing with agents now mainly, right?
• 13:57
  Yeah, yeah, so yeah, one medical, you know, there's a lot
• 14:01
  of like setting up the standard like ML 1.0 tool stack, running
• 14:04
  that stuff on Kubernetes, CML flow, you're you know, training
• 14:09
  NLP models like the early transfer models on, you know
• 14:13
  data using whatever library you wanted to use for training, then
• 14:17
  Sage Maker training or what, than serving it on Selden or
• 14:20
  deserve or you know any of those setting up sort of evaluation
• 14:24
  tooling for that stuff. Well, you know we're building
• 14:28
  models. You know one of my primary jobs
• 14:30
  there was sort of you know building models to detect things
• 14:34
  in text and and classify just doing you know multi class
• 14:37
  classification of of text. And I just feel like that's
• 14:42
  skill set moving from there, I went from there to you know,
• 14:45
  building on our laps and building applications on our
• 14:48
  labs and working with a bunch of different founders are building
• 14:51
  applications that allow that. I just didn't see like there's
• 14:53
  hardly any crossover, right. It's like ML Flow now has some
• 14:57
  like LM capabilities in it. So a lot of these tools are like
• 15:00
  pivoting towards that direction. But just having kind of like
• 15:03
  live that experience, I just I didn't see like oh what I
• 15:06
  learned at One Medical, just not a lot of it applied.
• 15:09
  I mean some of it as far as like how you evaluate things, you
• 15:12
  know, the the basic understanding of like how a
• 15:14
  transformer works and stuff. But I wouldn't necessarily call
• 15:16
  that specifically like MLPS maybe some of that.
• 15:19
  But so there's some bleed over. I just didn't see a lot of it
• 15:23
  all. But you feel like that is
• 15:25
  because you don't have to worry about like you're not serving
• 15:28
  the model, you're just hitting an API.
• 15:30
  Yeah, I mean there's three-part of that but even if you're like
• 15:33
  doing fine tuning on model and serving at you know it's that's
• 15:36
  also totally different just because of the scale and like
• 15:38
  how you fine tune Lems you know with Laura and whatnot that's
• 15:41
  like a totally different tool set than what we're saying with
• 15:44
  the with the early Transformers. So also felt like that didn't
• 15:47
  translate over even right. But you know the biggest use
• 15:50
  case that all EMS is definitely like JavaScript developer with
• 15:53
  an open AI key, right. And that's all they have.
• 15:57
  And now they're, they're building AI applications off
• 15:59
  that which we never really had before.
• 16:01
  Yeah, right. And there's it's really, really
• 16:04
  hard to get an AI application on LM's out to production.
• 16:07
  It's just it's astoundingly Challenger and we don't really,
• 16:10
  we didn't have any tools at the time to really help with that.
• 16:13
  So I do think a lot of the tools being built in the like space or
• 16:18
  actually really useful, it's just there's been a lot of noise
• 16:20
  to you got to like sit for. I I guess my last question for
• 16:25
  this round and Farhad I I want you to weigh in too, is don't
• 16:30
  you feel like the data pipelines and almost it's like the data
• 16:35
  engineering side of ML OPS is the same thing still with LM
• 16:39
  OPS? I actually think they are almost
• 16:43
  identical because when you have data like the prop, the the main
• 16:50
  problems that envelops like is trying to solve apart from like
• 16:56
  the model pipeline is like how you deliver the data
• 17:00
  transparently, secure and like reproducible.
• 17:05
  So this doesn't change whether you're training a black box like
• 17:11
  an LLM or like a very simple model like I don't know I guess
• 17:18
  SVN. So it doesn't change did, did,
• 17:23
  did the mantra. It's it's like the this
• 17:26
  paradigm. We are enabling envelops, we are
• 17:30
  enabling the never ending cycle from data to production.
• 17:36
  So it doesn't matter whether we like using the other or using a
• 17:40
  very simple model or even use like something like I don't know
• 17:46
  human in the loop models like at the end of the day we were
• 17:51
  trying to provide as much automation and as much security
• 17:56
  and transparency for like the companies benefit from.
• 18:00
  So I think that defines what envelops means.
• 18:06
  It adds another cycle to DevOps. That's because that means like
• 18:13
  wave when people, when they lose people like try to simplify
• 18:19
  envelops, They cannot like ignore this added cycle.
• 18:24
  This added cycle is at its core, it's like the definition of maps
• 18:28
  and it doesn't matter if whatever tool you are like,
• 18:34
  you're defining above that. So I think like after
• 18:38
  Transformers something new comes in like I don't know, maybe
• 18:43
  graph, maybe graph CNS like. Moba.
• 18:48
  Yeah maybe they become like norm again I don't know and see the
• 18:54
  things like that the the problem remains the same like very
• 18:58
  engineers but we we are not defined by our our technological
• 19:04
  stack. I think like, I think LM is like
• 19:07
  a technological stack. Like yeah, it's very good.
• 19:14
  It's very challenging. It's a new challenge that we are
• 19:16
  trying to solve problems via LM. We have never seen this before,
• 19:21
  but it's like let let me think from a step step behind and
• 19:29
  think about the future. Like what did we do when we
• 19:33
  first tried to solve the mill problem, the simple ML problem.
• 19:38
  OK. How can we map that to LOL
• 19:42
  actually think that there's like this void that people are
• 19:45
  feeling for LNG. There's not, There's not like a
• 19:50
  doubt about it. I don't have a doubt about that.
• 19:52
  But I think we should extend our current tools and current
• 19:57
  tooling, not separate them into this like very small specific
• 20:04
  surgical tools, because at the end of the day, this is not
• 20:09
  scalable I think for me. I think for me it just goes back
• 20:12
  to the user, what's what's most appealing to the user.
• 20:14
  And I think with all labs, we just really have a different
• 20:17
  like, user profile overall, Like there are some similar user
• 20:21
  profiles, right? But now we have like this whole
• 20:24
  whole new ecosystem that's opened up of like JavaScript
• 20:28
  developers, Rails developers, like all these other people were
• 20:31
  starting to use AI directly in their applications.
• 20:34
  So I do think we do need a new set of tools like cater
• 20:37
  specifically to those people who aren't necessarily like, you
• 20:41
  know, it's just not the same use cases we had prior, so.
• 20:45
  And that reminds me of a question that I wanted to ask
• 20:50
  you, Patrick, and that is really about how much of like how much
• 20:57
  are you going to benefit if you do understand ML OPS when
• 21:02
  working with LM's? Because it does feel like, I
• 21:04
  know you're you said, look, it's totally different things, it
• 21:10
  warrants a new name. I can 100% see that.
• 21:14
  I do feel like on one hand though, you're going to be able
• 21:21
  to accomplish what you want to. The more that you understand
• 21:25
  MLA's, the more that you understand and maybe it's not
• 21:29
  training a model. Maybe it is training a model
• 21:33
  with the like you were talking about with Laura is, I don't
• 21:35
  know. And that's kind of what I'm
• 21:36
  trying to figure out is how much of A power up is it going to
• 21:43
  give me If I like the borer ML that I understand.
• 21:48
  The more ML OPS that I understand, the more data
• 21:50
  engineering that I understand, the more DevOps that I
• 21:53
  understand all of these. Like if I can take from that
• 21:56
  because if it is generally a JavaScript developer or a Rails
• 22:00
  developer that is leveraging this like where wouldn't you
• 22:04
  think they want to go deeper to be able to accomplish their
• 22:08
  vision? Yeah, I think it depends on what
• 22:12
  you're doing. Yeah.
• 22:12
  I mean, obviously if you're fine tuning Laura, like there's
• 22:14
  there's a bunch of, right. I mean, I do still think we
• 22:17
  probably want specific tools and that's where we've seen like pop
• 22:19
  up in the ecosystem. I think that's what's effective
• 22:21
  is having tools like specifically for Lauras because
• 22:24
  they do have a specific set of problems.
• 22:26
  Not that you couldn't work that into a larger tool, I think it's
• 22:28
  possible. But what I think we've seen
• 22:30
  emerge is that it's easier to use a tool like specifically
• 22:33
  designed for that. So yeah, I mean if you're fine
• 22:35
  tuning in LM, I, you know, ML, OPS, City of history and that's
• 22:40
  going to be incredibly beneficial, right?
• 22:41
  Even though just to understand the basic concepts of what
• 22:44
  you're doing. But if you're a JavaScript or
• 22:46
  Rails developer and your job is just to put out an application
• 22:49
  that people are going to be using, I don't think it's super
• 22:52
  helpful. Like I think it's helpful
• 22:54
  obviously if you have that I just I you know it's it's a lot
• 22:58
  lesser. Yeah, I just feel like, how much
• 23:01
  value are you able to create if you're just doing it as a
• 23:08
  JavaScript developer, right? Like.
• 23:10
  How I think it's all about you, honestly.
• 23:13
  Yeah. Because LM are still just the
• 23:14
  textual interface, right? And that's all we have.
• 23:17
  I think what we're starting to see Gemini and one of the things
• 23:19
  we're definitely working on our house is like making albums be
• 23:22
  able to create UI elements, right.
• 23:23
  So you can actually start to create a web page, interact
• 23:26
  with. Because a lot of times that's a
• 23:28
  much better experience, right, to have an actual web page than
• 23:31
  to just have tax, which can get boring and you know, it can be
• 23:34
  harder to navigate, right? So, but you know, until we
• 23:37
  really have that stuff and that stuff working really well, I
• 23:40
  think having a JavaScript of all rebuild, a whip up an
• 23:43
  application that's really tailored towards, you know
• 23:45
  specific user and have all the bells and whistles of an
• 23:49
  application, I think that's incredibly bad.
• 23:53
  And so it feels like you're you're kind of arguing for the
• 23:57
  long tail of use cases that people are finding.
• 24:01
  And it doesn't matter the model under the hood per se.
• 24:06
  It's like, yeah, we can all have the same base model.
• 24:09
  Because my use case is such a sliver, and it's so unique that
• 24:13
  the use case itself is the valuable piece, not the data
• 24:17
  that is powering the model. Yeah.
• 24:20
  And there's different theories on this, but yeah, I mean if you
• 24:22
  look at like what YC's investing in right now, they're almost
• 24:25
  entirely investing. And you know the the first thing
• 24:28
  you mentioned there, which is you know developers just going
• 24:30
  after specific use case, applying lens to that, working
• 24:34
  that into business workflows that already exist and selling
• 24:36
  that and that's proven to already be quite lucrative,
• 24:39
  right. I think if you're going for the
• 24:41
  data play there, there are plays there.
• 24:42
  It's just it's, it's challenging to train a model that's going to
• 24:46
  be on par with GB 4, right. And we can fine tune models and
• 24:49
  specific areas that can do that. But T4 is just so far ahead that
• 24:54
  like you're probably gonna have, you know, hard time getting
• 24:56
  there and providing something as much value as just like, you
• 24:59
  know, contacts engineering on top of 254.
• 25:03
  So I thought you might have something to say there.
• 25:06
  Yeah, I actually agree with that that we like have to abstract a
• 25:11
  lot of layers like this is software engineering we're
• 25:14
  talking about. When we're talking about
• 25:16
  software engineering, we like we like to have like reproducible
• 25:21
  and measurable building blocks. So when the the challenge, the
• 25:26
  main challenge of ML OPS in our era at least is like all the we
• 25:32
  have we have to build those building blocks for our
• 25:36
  companies. So we're like, we're the
• 25:39
  abstraction like that means that I am the documentation.
• 25:43
  So like, yeah, we are the abstraction here and we provide
• 25:49
  that for others and to benefit from like.
• 25:54
  So when we're talking about a random JavaScript or even a
• 25:59
  random Python hobbies like trying to ride a bike, ride a
• 26:05
  bot or something like that, I understand that.
• 26:09
  I actually believe that it's a miracle of like engineering that
• 26:15
  what people are like using ChatGPT for this and that.
• 26:19
  Like it takes a lot because of my evil loves background.
• 26:25
  I understand them how much of an effort was put into it.
• 26:29
  So the abstraction is necessary. Yes it is necessary but do we
• 26:35
  need it in the LLM OPS space or should be talked about?
• 26:40
  OK, these these are like this is the layer of mylaps and this is
• 26:45
  the layer above that for whatever AI we are providing
• 26:48
  like what can we do, what can we do to separate these reliably?
• 26:56
  Have a blueprint for that because I I think like most of
• 27:01
  our us like traditional developed engineers we face this
• 27:09
  challenge too many times and there are there are still so
• 27:13
  many confusions about like how to actually solve this problem.
• 27:17
  That's I I had this talk with my one of my friends like I think
• 27:23
  that Emma Labs is not a single engineer should be a consistent
• 27:29
  of like 5 different engineers that you have to be a data
• 27:33
  engineer. You have to be a data scientist.
• 27:35
  You have to be a DevOps engineer.
• 27:37
  You have to know about back end, you have to know about a bit
• 27:40
  like about you know, you know BI is good.
• 27:45
  So at the end of the day this is like this.
• 27:50
  This is branching in itself. This is a paradigm.
• 27:53
  So we are like filling the void. The void will like become mature
• 27:58
  like used. You see the blueprint.
• 28:01
  But this type of like hyping things up LLM oops I don't know
• 28:08
  maybe and not in a few years we see something else OPS or
• 28:13
  something else Yeah we know what the how the the take hold or
• 28:17
  take Bros worked. We know that like bet three off.
• 28:21
  I can assure you it will happen web 3 apps I can assure you.
• 28:26
  So I think that we have to, as engineers, as fellow technical
• 28:33
  people, we have to like, make it easier for future engineers, for
• 28:38
  a future problem solvers, enable them to to enjoy the fruits of
• 28:45
  our labour. So this dislike, this this type
• 28:50
  of redefining the problem, the problem that was partially
• 28:56
  solved by another group, another dude, I don't know, it's it's
• 29:01
  like here it's me a bit because I think it's like the.com
• 29:06
  bubble. Everyone tried to had their own
• 29:09
  side. Yeah, many sites had this
• 29:13
  amazing service like Amazon, Google or things like that, but
• 29:19
  not all sides provide this speculation.
• 29:24
  I think that helps me a bit. This is Skyler.
• 29:28
  I lead machine learning at Health Rhythms.
• 29:31
  If you want to stay on top of everything happening in Mill
• 29:33
  OPS, subscribe to this podcast. Now.
• 29:38
  Now. Now.
• 29:41
  Now. Now.
• 29:44
  Now now. I don't doubt that I'm a big fan
• 29:51
  of bubbles. I think if you look at the
• 29:52
  research bubbles, you know, bubble like economies, you know,
• 29:57
  they perform better, right? So you have this big bubble, you
• 30:00
  explore a bunch of different areas, it pops, and then you
• 30:02
  kind of find out what's actually useful.
• 30:04
  But if you were to just go up gradually without the bubble,
• 30:07
  you actually go up slower than if you go up with the bubble and
• 30:09
  kind of go down and not. Right.
• 30:11
  So this is kind of a proven economic theory as well, right?
• 30:14
  So like, even youknowthe.com bubble, yeah, we had a bunch of
• 30:17
  nonsense on that. But, but you know exploring
• 30:19
  isn't necessarily nonsense. People just going out like
• 30:21
  trying things and doing different stuff and you know
• 30:24
  putting money into it. And I think particularly with I
• 30:26
  like we're under invested still. I think I think we're massively
• 30:29
  under invested. I think we are you know, with
• 30:31
  all mops as well like we're talking about you know,
• 30:33
  potentially like the evolution of human beings, right.
• 30:37
  Like I don't think you can be like too invested in this space.
• 30:40
  I don't think we could be trying too many things like this is
• 30:43
  this is the future potentially everything right.
• 30:45
  And I think like people are finally starting to see that
• 30:47
  we're getting the VC money into it and I think it's, I think
• 30:49
  these bubbles are great. I think they're fantastic, but
• 30:52
  that's that's just my view. Ride the wave is what he said.
• 30:58
  Ohh, well I'm all for calling bullshit on what it is if.
• 31:04
  If it's a spade, it's a spade, right?
• 31:06
  And I do feel like we need to. I like this idea of like, hey,
• 31:12
  we should cut through the noise, make sure that the future
• 31:16
  generations understand that. Yeah, there is some differences
• 31:20
  here, but here's the key points that you need to keep in mind.
• 31:24
  How much of that is going to serve you for this field?
• 31:28
  Whether it's Milos, whether it's DevOps, whether it's LLM, OBS.
• 31:31
  I I really appreciate that and like being a trusted source on
• 31:36
  just like this is way out of left field.
• 31:40
  But I'm going to give you a little antidote, Alright?
• 31:43
  If I were told that when I was a wee little lad, really, that not
• 31:50
  all drugs are created equal, maybe I would have been more
• 31:55
  receptive to believing what drug does what and I wouldn't have
• 31:59
  had to go and try for myself. Right and so.
• 32:02
  But it was a cool time, right? It was a good time.
• 32:06
  That's exactly. It was great.
• 32:07
  I'm glad I did it, but I wish, and I think I'm going to, with
• 32:12
  my children, sit them down and say, look, you know, got your
• 32:16
  weeds, you've got your psychedelics, you've got all
• 32:20
  these different things, they're very different breeds of drugs,
• 32:22
  just like you've got your ML OPS, you've got your LM mobs,
• 32:26
  you've got your DevOps, They're all very different breeds and
• 32:30
  there is some overlap. There's some similar feelings we
• 32:33
  could say, but I am leaning towards they're not exactly the
• 32:39
  same considering the type of when it comes to LM mobs, the
• 32:44
  type of person that you are seeing that is leveraging the LM
• 32:50
  tooling that's out there. I do see the profile that you're
• 32:53
  talking about. Patrick's come up quite a bit
• 32:56
  and it's almost like the builders, but that doesn't mean
• 32:59
  that it's only those people, right?
• 33:02
  Like I do think there's a lot of crossover with the platform
• 33:06
  teams that can be called ML OPS teams and they are leveraging
• 33:10
  LLM's too, and they're used to working in a certain way.
• 33:14
  Maybe it's just the fact that they like Python And so you have
• 33:19
  to keep that in mind too. Where's the bigger segment?
• 33:23
  I don't know, probably. I think there's a whole lot more
• 33:26
  front end developers than there are ML engineers.
• 33:29
  That's the thing that that I'm definitely seeing too.
• 33:32
  When I I've worked with a bunch of startup founders.
• 33:34
  It's like, you know, there are people doing the lower level
• 33:37
  work, but there's just so many JavaScript in here.
• 33:39
  There's so many Python in here. There's so many applications
• 33:42
  about those and they're all using this stuff now.
• 33:44
  And it's just a huge segment of the market that's been brought
• 33:47
  into the ML space. I do think we need to teach them
• 33:50
  some of the basics and there was a lot of people doing courses
• 33:52
  and whatnot to really teach them what to transformer is like how
• 33:55
  to do good AML stuff. And I do think that's probably
• 33:57
  useful to them, but it's a totally different market.
• 34:00
  We're saying that it's like nothing we've seen before.
• 34:02
  Yeah, what's interesting to me is different tools that are
• 34:07
  building for these personas. And I know there are LM tools
• 34:12
  that are coming out and they're saying, hey, we're just for the
• 34:14
  persona that loves TypeScript and that is the tool that we are
• 34:20
  building. We're not trying to accommodate
• 34:22
  any Python or anything else for that matter.
• 34:24
  It's just that TypeScript developer.
• 34:27
  If you are that person, then we're the tool for you.
• 34:31
  Yeah, Purcell layout, yeah. There you go.
• 34:34
  Versele AI is a perfect example of that.
• 34:37
  That is so true, the AI component of the vessel.
• 34:41
  And so, all right, now let's play a little game.
• 34:44
  I like to call. Did he actually say that shit?
• 34:49
  And this is going back to the thread that we that kicked off
• 34:54
  this whole thing. And now let's see, did I'm.
• 34:59
  I'm looking right at you, Patrick.
• 35:01
  And we're going to say, did he actually say that shit?
• 35:06
  Because I have to question you about some things that were said
• 35:09
  on that, here's here's what we are wondering.
• 35:15
  Prediction Gen AI will eat ML 1.0 in less than three years.
• 35:21
  Did he actually say that shit? 100 percent, 100% There's
• 35:24
  already companies there must you know, I will eat.
• 35:27
  I'd maybe probably sooner, honestly.
• 35:28
  Probably say two years. Oh my Explain because I cannot
• 35:33
  handle. This, I mean it's it's basically
• 35:35
  like hugging GPT, but you add like training into it, right?
• 35:38
  You know, we're getting good at teaching agencies tools, right?
• 35:41
  We're getting better, better at doing this and obviously we're
• 35:44
  we're starting to train foundational models.
• 35:46
  Yes, there's been some come out recently received paper called
• 35:48
  agent tuning that shows that if you train specifically on you
• 35:51
  know types of tools you can get much better and you can actually
• 35:54
  approach GPD 4 capacity for using tools as well.
• 35:58
  So it's like if you think about a model and the mill OPS one
• 36:01
  point ecosystem is just a set of tools then as soon as we get
• 36:05
  good at using tools ML 1.0 probably isn't needed for the
• 36:10
  most part. You know agents should be able
• 36:11
  to run at full stack. Obviously this depends on
• 36:14
  foundation models getting better, but I think we're seeing
• 36:17
  so much money that space, but I just can't imagine it does.
• 36:20
  I mean there's that's a big assumption there just because
• 36:23
  there's money in a space, ship's gonna get better.
• 36:26
  I think that's probably the there's a little bit of a delta
• 36:31
  in. That that's not, that's not
• 36:33
  true, right. But we are seeing a rapid rate
• 36:35
  of progress. So if that continues, I think we
• 36:38
  get there and if it doesn't? Definitely not going to be
• 36:42
  because of undercapitalization, that is for sure because we just
• 36:48
  saw it today or yesterday that Mistral got a massive second
• 36:53
  round whatever it was like 345 billion and they released, they
• 36:58
  recently released another model too.
• 37:00
  And so that's great. I mean, Patrick, I actually want
• 37:05
  to get your pulse check on it. Are you primarily playing around
• 37:09
  with the big models that you mentioned?
• 37:12
  Love Affair with GBT 4 and you also mentioned how you were
• 37:16
  using Gemini, but have you tried to play around with these
• 37:21
  Mistral models or the Lamas and see?
• 37:24
  Is there a fine-tuned version that will be my specific one for
• 37:31
  using tools or creating agents? Yeah, there's there's a number
• 37:35
  that have come out. There's also like Nexus.
• 37:37
  Don't forget the name of Nexus, Nexus Flow.
• 37:40
  I think they just they just came out with a ATAS model that
• 37:42
  shows. That it had to be a. 54 Yeah of.
• 37:44
  Course, that's something that they're never gonna let go of
• 37:48
  from the that'll work for, you know your.
• 37:51
  Ecosystem off? Yeah, the.
• 37:52
  Flow. Exactly.
• 37:53
  That's the only thing that's left after.
• 37:55
  As you say it, Jenny, I eats fucking hell last 1.0.
• 38:02
  Just the only thing that's left over is the flow artefact.
• 38:05
  The flow, yeah. But yeah, we're saying that I'm
• 38:08
  super bullish on, on tuning these these open source models
• 38:11
  towards tasks. It's one of the main things that
• 38:14
  focused on right now with my company and just seeing how good
• 38:17
  we can get these models at you know performing specific tasks
• 38:20
  because that's you know I think as soon as we accomplish sort of
• 38:23
  tasks and code John you know I mean we take the the model and I
• 38:27
  think most people do have you know executing tasks as a matter
• 38:30
  of generating code against specific APIs which is pretty
• 38:33
  common. But I think it's master that
• 38:36
  we're pretty close to Asian. I think that's you begin to
• 38:39
  approximate age. He did.
• 38:42
  He's throwing out words like AGI holy shit.
• 38:45
  I don't even know. This is the first in this
• 38:47
  podcast. Oh my God.
• 38:49
  We gotta go thank. You you've taken the red pill.
• 38:52
  You drank the kool-aid Man. Ohh, I I can't imagine.
• 38:58
  I still wanna break down like in your eyes.
• 39:00
  How does an agent effectively render ML 1.0 useless?
• 39:08
  Like, I still don't see that as clearly as you, and so I'm
• 39:12
  trying to like, pick that apart. Yeah, I mean I think it's, you
• 39:16
  know it's it's similar to probably what happens in the
• 39:18
  human brain and this is like in girl little dots here which is
• 39:22
  that, you know, we have our new cortex which is basically one
• 39:25
  algorithm that we know and it's it's like 1 pattern and it's
• 39:27
  this thick layer, you know on the on the top front of our
• 39:30
  brands. Then we have all these other
• 39:31
  areas that our brains that do specific things like you know,
• 39:34
  process visual data and process auditory data.
• 39:36
  Those are all different architectures, right.
• 39:38
  Like we developed convolutions basically from looking at how
• 39:42
  the brain processes visual information.
• 39:44
  That's how you came up with populations, right?
• 39:47
  I yeah, I I see kind of evolving in that way where we're gonna
• 39:50
  have this higher level brain that can probably begin to
• 39:52
  construct like lower level algorithms that are really good
• 39:55
  at processing things like audio data and visual data and they'll
• 39:58
  be better processing tabular, right.
• 40:00
  It's like Xgboost isn't gonna be replaced by a lot and probably
• 40:02
  anytime soon because it's so efficient, it's so small, it's
• 40:05
  so effective at what it does. But all of them will definitely
• 40:08
  be able to train Xbox models on tabular data.
• 40:11
  We're already doing that today, basically, right?
• 40:14
  And they start to be able to with more autonomy.
• 40:16
  Failed to have that light cycle. So as we see the autonomy of
• 40:19
  agents go up, which I think over the next year or two we're gonna
• 40:22
  see a huge jump in. Knock is pretty much every
• 40:24
  research lab is about just on that right now.
• 40:27
  So assuming they're, you know, they see fruits in their
• 40:29
  efforts, you know, we're gonna see agents get a lot smarter and
• 40:33
  then it's just applying agents that could do to us to, you
• 40:37
  know, the job of training models.
• 40:39
  But like, there's so much other shit besides training the model,
• 40:44
  right? Yeah, yeah.
• 40:46
  I mean I think, you know, like we talked about the data
• 40:47
  engineering side super hard and I, you know, some people
• 40:50
  consider that I'm off, some people don't.
• 40:52
  I'm kind of on the fence. I do think it's kind of its own
• 40:54
  thing that exists outside of ML OPS.
• 40:56
  It's not that different from all than it is for other things.
• 40:59
  But that part of it's probably the hardest part automate.
• 41:02
  But as far as like training models, evaluating models,
• 41:05
  serving models, that I think you could do pretty easily with
• 41:08
  agents within two to three years.
• 41:10
  Well, because it does feel like the data modelling is so context
• 41:18
  and you need the experts there that they model.
• 41:25
  Even if it can do it, does it understand everything?
• 41:28
  Like it just feels like they're very hard problem.
• 41:31
  I'm not saying it's. Important super hard problem on
• 41:32
  the data engineering side. Yeah, super hot and it's all
• 41:36
  streaming. A lot of times it's at high.
• 41:37
  Volume. Iran that are super complicated
• 41:41
  down the pipeline like that gets really hard for sure.
• 41:45
  Yeah, so mill 1.0 gets eaten. And I did hear.
• 41:50
  So when I was in the Berlin ML OPS community meet up last week,
• 41:55
  one of the guys there was talking about, hey, we're always
• 41:58
  talking about generative AI for this, generative AI for that or
• 42:03
  ML OPS for generative AI AKA whatever, Foundational Model OPS
• 42:09
  or LM OPS. But he's like what about
• 42:13
  generative AI for ML OPS? And yeah, I think that's exactly
• 42:18
  what you're. Saying, yeah, Abacus AI is kind
• 42:21
  of doing this. There's a couple of companies
• 42:22
  that are starting to to try to do this.
• 42:24
  I don't know how well it works. Yeah, but people are like
• 42:26
  approaching that space. I think that the like what the
• 42:34
  foundation models and things like that are solving is very
• 42:38
  admirable. But for me the the main
• 42:41
  challenge is like the whole MLC is the way Transformers and all
• 42:47
  these architectures are like working.
• 42:51
  The carbon footprint and the energy consumption of them is
• 42:55
  like very high and becomes higher and higher.
• 42:58
  And the main problem I see, the main obstacle that we have to
• 43:01
  solve eventually is like how to make these models an actual
• 43:09
  building blocks. So right now if I open AI like
• 43:13
  wants to train a new GT model it has to, it has to train from I
• 43:18
  don't know like from scratch I think.
• 43:22
  So when when we were talking about the the cost of this is
• 43:28
  like it will add up and add up and this is not engineering this
• 43:32
  is built force. We are currently we are like
• 43:37
  creating enough value to justify that but as time goes by this
• 43:43
  simple problems that LM is solving and hopefully complex
• 43:47
  more complex problems these these will like we have to solve
• 43:56
  this trade off how much is enough and how can we like go
• 44:00
  past from that. So this is the main challenge
• 44:02
  from, from my point of view might be like many of the global
• 44:07
  issues. So I know that many of the jobs
• 44:12
  or many of the things that these automations are like trying to
• 44:18
  get rid of are actually those those have their carbon
• 44:23
  footprint too. But in the end, we have to think
• 44:26
  we're seriously about it. Yeah, it's alright.
• 44:30
  There's a company out of MIT right now that that just got a
• 44:32
  bunch of funding that's solely focused on, yeah, how do we make
• 44:35
  these things? And I really felt like, and I
• 44:37
  think it's a huge problem. Yeah, if we want to use them at
• 44:40
  scale and use them the way you're saying, it's just
• 44:41
  building blocks put into everything.
• 44:44
  Yeah, we need to make it more efficient.
• 44:45
  And luckily, like that model of paper that just came out, you
• 44:48
  know, is wildly more efficient than the Transformer and as you
• 44:51
  know, linear context scaling, which is incredible.
• 44:55
  So we have yet to see if that can scale the same way
• 44:58
  Transformers Damp. But if it can, it's super
• 44:59
  promising. I mean, it's punching 2 eggs
• 45:01
  above its weight right now, so. Yeah, I think the, the
• 45:05
  interesting piece we had Jonathan on here like a week ago
• 45:09
  from Mosaic ML and he was talking about how his lab tries
• 45:15
  to reproduce pretty much any paper that comes out.
• 45:19
  And up until now it's been dismal at best because of how
• 45:26
  they don't hold up once you get to a certain what he was saying,
• 45:31
  the industry needs and the standards that you're expecting
• 45:36
  at the industry level. And so it's great.
• 45:40
  He was super kind when it came to praising the researchers that
• 45:46
  are trying to push the envelope. But he did say like, yeah, I'm
• 45:51
  not going out on Twitter and putting anyone on blast, but I
• 45:54
  am going to just caveat all these papers, be aware when
• 45:59
  you're reading them. It is very hard to actually see
• 46:04
  that being used at skill that we are used to.
• 46:11
  That's an indeed problem with evil in general because like
• 46:15
  when we are MLA's engineers we are like at this forefront of
• 46:21
  like two different paradigms fighting.
• 46:24
  We have like research minded people who are like try to
• 46:28
  absorb as much resources as they want, like they they are trying
• 46:33
  to get that knowledge they they want to know more.
• 46:38
  The research mindset is is about that.
• 46:40
  Give me as much as you can so I can think for one more 2nd.
• 46:44
  And the engineer mindset is like I have to like be resourceful, I
• 46:50
  have limited resources and I want a feasible solution.
• 46:53
  So MLA apps in general is like you're standing there and you're
• 46:58
  trying to appease both of these people like this is this is the
• 47:04
  main problem with e-mail like it.
• 47:08
  It is what it is like being published in papers just been
• 47:14
  instantly intrigued in the industry.
• 47:17
  So like we don't have this snag. It's very interesting in a sense
• 47:22
  like we are in a very interesting but living in a very
• 47:26
  interesting time. Something that might be looked
• 47:29
  upon black 100 years about AI animal, but in general it makes
• 47:35
  it super challenging for like providing stable and thoughtful
• 47:45
  growth. So many things seemed like
• 47:47
  accidentally bit like yeah this thing happened along the way and
• 47:51
  we did that and yeah, we're successful in that.
• 47:54
  Many of the AI companies, in Iran at least, are like that,
• 48:00
  so. Who knows?
• 48:03
  So I feel like we should put a disclaimer right now.
• 48:07
  You know how they say like non financial advice?
• 48:11
  Well this whole conversation is not engineering advice or not
• 48:16
  career advice I guess. Listen at your own risk, put any
• 48:19
  of this stuff into practise and it's been a blast talking to you
• 48:24
  guys. I really appreciate this.
• 48:25
  For everyone that wants to see the thread that kicked off this
• 48:29
  whole conversation, you can check the link below in the
• 48:32
  description. And I think that's it fellas.
• 48:35
  That's the pot. Thanks for having me.
• 48:39
  Thanks for having me. Yeah, hey everyone, My name is
• 48:46
  Aparna, founder of Verizon, and the best way to stay up to date
• 48:49
  with ML OPS is by subscribing to this podcast.