Latest / Agentic Conversations (formally mlops.community) / ProductizeML: Assisting Your Team to Better Build ML Products // Adrià Romero // MLOps Meetup #47

Transcript

ProductizeML: Assisting Your Team to Better Build ML Products // Adrià Romero // MLOps Meetup #47 · Agentic Conversations (formally mlops.community)

• 0:16
  Thank you all for coming. I am really excited to be able
• 0:20
  to talk to other about everything that you've got going
• 0:24
  on either. This is really cool to have you
• 0:27
  here man. Thank you.
• 0:29
  Yeah, I'm hi. Thanks for the opportunity.
• 0:33
  Yeah, so that everyone can be on the same page and we can know
• 0:38
  what exactly we're doing here today.
• 0:41
  Maybe you could give us a bit of background on that.
• 0:44
  And also just tell us like how did you get into Tech?
• 0:49
  Sure. So, I remember when I was
• 0:53
  finishing my bachelor's degree in electrical engineering that
• 0:58
  was in Barcelona. By the time electrical
• 1:02
  engineering was kind of like a popular and at the same time
• 1:05
  challenging Bachelors to to complete the job offer.
• 1:11
  The man was was quite High by the time that I started that
• 1:17
  Bachelors, but then I realized by the time that I was over with
• 1:21
  it. I realized that what I have.
• 1:25
  Ben sighing, what wasn't like the domain that it was like more
• 1:31
  attractive? That actually was computer
• 1:35
  science, right? And actually one of the key
• 1:40
  things and things that I'm really satisfied and really
• 1:44
  happy to to have had their is the, the research groups and the
• 1:51
  people that belong to that community.
• 1:58
  And yeah, from from those groups, I think that was my
• 2:02
  access to, to the, not only to the Technology field, but also
• 2:08
  to the AI and machine learning feel.
• 2:12
  And I'll tell you guys, the story is kind of my crash on ML.
• 2:18
  And after that, I'll give you some time.
• 2:21
  Demands used to think about your, your love story for.
• 2:25
  Or a male, but but in summary, there is this project.
• 2:31
  It's called the image net project.
• 2:34
  Some of you might have heard about this project, but
• 2:37
  basically group, a research group at Stanford collected,
• 2:42
  like 14 million of images, hand annotated collecting more than
• 2:48
  covering more than 20,000 categories.
• 2:52
  And from that data set days. started like the visual
• 2:57
  recognition challenge, the objective of that challenge was
• 3:03
  just to achieve like the highest accuracy on the on the visual
• 3:07
  classification task of 1000 categories.
• 3:13
  Those teams were offer like a like a batch data set of around
• 3:21
  1.2 million training images and at the end, the highest that you
• 3:27
  could get the higher study that you would rank on this
• 3:31
  challenge, right. And This was a story that
• 3:38
  basically by 2000 before 2012. More participants were using
• 3:44
  like traditional computer vision methods, like feature
• 3:49
  engineering and applying some classification, like algorithms
• 3:54
  with really shallow structures. And the, the important aspect of
• 4:02
  this challenge is that of around 2012.
• 4:06
  They some of the team's start to like using deep learning, right?
• 4:09
  And deep learning came into play and the first architecture that
• 4:14
  it was used was Alex net. These architecture what it
• 4:21
  attracted me like a lot. But by the time that I was
• 4:24
  reading this, and it's because, because they improved by around
• 4:30
  10%, the results that Additional methods were we're offering,
• 4:35
  right? That was obviously, thanks to
• 4:39
  gpus and this convolutional neural networks, but that was
• 4:45
  kind of my first impression of of AI.
• 4:48
  Wow. We can we, we can really dig
• 4:53
  into these Technologies and we can actually improve, right?
• 4:57
  The, these state-of-the-art by quite a lot, and by the end of
• 5:03
  the, I think it was around 2016. We reach to a point that this
• 5:11
  CNN architectures were even, like, improving, they were doing
• 5:16
  better than the human performance.
• 5:20
  Level, right? And that's actually like the
• 5:24
  the, the, the, the first trigger that, that I thought.
• 5:30
  Wow. This is actually going to change
• 5:35
  like, the, the way that we see deep learning, right,
• 5:39
  interacting with with our products and technologies that
• 5:44
  we use. How about if we move this
• 5:48
  domain? This Was as I was saying, just
• 5:53
  1000 categories from like regular daily objects, like cars
• 5:58
  cats animals, right? But then I started thinking,
• 6:01
  well, why don't we move this concept to other domains, right?
• 6:06
  For example, the healthcare domain and that was actually my
• 6:10
  first kind of interaction with this Technology's.
• 6:17
  I remember that by the end. Of the dispatcher's.
• 6:21
  I had to complete like a thesis Too Close.
• 6:27
  Like to end my, my bachelor's. And I basically took this idea
• 6:34
  of imagenet, but I was trying to solve other tasks or, like, for
• 6:39
  personal reasons. I had some motivations to
• 6:45
  relationship with Dermatology, and I thought, Okay, let's let's
• 6:51
  try to collect images of skin lesions and let's try to
• 6:55
  replicate what these guys have been doing with with the
• 6:58
  imagenet challenge. But let's do it now with skin
• 7:02
  lesions, right? And So I had I had few options
• 7:10
  to join some, some research groups.
• 7:13
  I guess we can differentiate between two types of research
• 7:18
  groups. Mostly.
• 7:20
  The first one is the fundamental research that they try to expand
• 7:24
  like technology from, for example, visual classification.
• 7:29
  Classification, tasks object detection and segmentation.
• 7:34
  So trying to improve the technology that That our people
• 7:39
  could use. And then the second type of
• 7:43
  researchers are like applied research, right guys?
• 7:47
  That take the technology that the first group of researchers
• 7:51
  have been developing, and they try to apply it so we can have
• 7:57
  like some impact on society, right?
• 8:00
  This two groups. They have to live together.
• 8:05
  So one really CR2 to progress, right?
• 8:12
  And this this was actually the the path that I took to use some
• 8:20
  existing Technologies, especially the convolutional
• 8:24
  neural networks, and I try to use this to build like a system
• 8:33
  that was able to classify skin. Asians taking, for example, with
• 8:38
  with your smartphone and be able to have like this first
• 8:42
  assessment of what that skin lesion could could be right.
• 8:54
  And then the second motivation was.
• 8:58
  Okay. I've seen like this imagenet
• 9:01
  challenge. I've seen that.
• 9:03
  It's feasible to collect a huge number of images and and all
• 9:11
  reasons to why a jump into this was the the Frameworks that were
• 9:17
  available by the time it was around.
• 9:21
  Teen. So carers, for example,
• 9:24
  tensorflow were available and gpus were also like a thing by
• 9:31
  by the time. And then you went in, you tried
• 9:34
  to put it out and get it into production.
• 9:39
  Exactly. I, the first kind of trust was
• 9:44
  just like a like a research publication that I did with with
• 9:49
  some folks at the floor. Dida Atlantic University and it
• 9:55
  became kind of like the pretty attractive business idea as
• 10:01
  well. And that's how I was Reach Out
• 10:03
  by startup called triage from, from Toronto, in Canada.
• 10:12
  And basically these guys wanted to bring all these deep learning
• 10:18
  Technologies to healthcare. And what it seems to be like the
• 10:23
  low-hanging fruit for medical imaging was this this kind of of
• 10:28
  problems, right? So, at the end, answering your
• 10:32
  question, yes. This we wanted to bring it to
• 10:37
  this. This this research idea into
• 10:41
  production by offering this service in, in a way that users
• 10:47
  were able to download the app and just snap a picture.
• 10:51
  And we were able to predict like in real time.
• 10:54
  What? That lesion could be right, then
• 10:57
  the fun part started. And you realized what?
• 11:01
  Yeah, that would take. Exactly.
• 11:03
  And then I spend like a couple of years playing with building
• 11:11
  machine learning Frameworks on the top of Open Source, like
• 11:16
  ones like Harris and tensorflow and then I got really
• 11:20
  interested. Done on building like a
• 11:23
  platform, right? And that's when I started like
• 11:26
  reading more about MN Ops. It was a pretty new thing by the
• 11:33
  time, but my objective was, we were like different teams in
• 11:39
  this startup, which we had like data scientists data Engineers
• 11:46
  that we had like a really big group of data management.
• 11:51
  Folks and obviously like the front-end and back-end guys.
• 11:56
  And it was pretty hard to communicate between each other.
• 12:03
  So I thought why don't we build like a platform that we can all
• 12:08
  like understand and we can use it like translation machine
• 12:13
  between all so it doesn't matter what your knowledge about ml.
• 12:21
  But you can see like deploy some models and train those models,
• 12:26
  right? So you wanted to create some
• 12:28
  kind of centralized platform and you're saying that there was a
• 12:33
  lot of this communication breakdown.
• 12:35
  Can you go a little bit deeper into that?
• 12:37
  Like, what was breaking down in the communication part, was it
• 12:41
  the database Engineers didn't understand what machine learning
• 12:45
  was or was it that the data scientist couldn't explain
• 12:49
  themselves properly. Mm-hmm.
• 12:52
  Yeah, I would start saying, for example, the conflict between
• 12:58
  what programming languages, each of the teams we're able to, to
• 13:04
  use. For example, we had like a lot
• 13:07
  of friction between requests from the data science team to
• 13:12
  the data engineering team and mostly because they were using
• 13:18
  like Ruby on Rails and and data. These are we're only using like
• 13:24
  python for example. So, every time that we had to
• 13:28
  deploy like a new model data, scientists are the ones
• 13:33
  responsible for like developing and pulling all the new data
• 13:38
  that the service was obtaining and training a new model
• 13:42
  optimizing, this model and then deploying it, right?
• 13:45
  So we had always to ask for those data Engineers 22.
• 13:51
  To deploy it for us. So in that sense, I was
• 13:55
  thinking, why don't we each exchange like communication
• 13:59
  through like a configuration file, for example, right.
• 14:03
  That it's almost like code-free, but still we can use it to
• 14:09
  communicate, like each other. So, that was kind of the
• 14:13
  objective with this machine learning platform that we worked
• 14:18
  on at this startup. Yeah, that Sort of sense and
• 14:21
  it's something that we hear so much, you know, like the data
• 14:24
  scientist, just having to throw it over the fence.
• 14:27
  And you have. Is it the devops team that has
• 14:30
  to then go and take care of it and do they even know how to
• 14:33
  take care of it? So exactly.
• 14:37
  So then maybe we can jump into just like how from there you
• 14:42
  decided you wanted to go into what you've now done and like
• 14:48
  help create a knowledge. Each base and of course to have
• 14:54
  everyone else understand. And for those of it's probably a
• 14:57
  good idea for you to break down. What exactly it is that you're
• 15:00
  doing with productionize ML and what you hope to achieve with
• 15:06
  it. Exactly.
• 15:08
  So I'm just thinking the idea of what I was explaining with this
• 15:13
  machine learning platform, the idea of letting everyone in your
• 15:19
  team to understand. Since what's the final goal and
• 15:23
  most of the times this final goal is to deploy your services
• 15:27
  or or bring your research idea in this case into production,
• 15:32
  right? And I thought that the best or
• 15:36
  like the low-hanging fruit was to do it through educational
• 15:40
  resources. All right.
• 15:41
  So everyone no matter in which theme you belong to and you are
• 15:47
  able to understand. What are the Other teams kind of
• 15:54
  needs are right? And especially in this field
• 15:58
  that it's quite like a there is like a lot of innovation going
• 16:02
  on. I thought that that what I had
• 16:06
  to build was like a educational tool that were that was offering
• 16:12
  this kind of knowledge to each of the teams.
• 16:16
  So the name of this so is product ML.
• 16:22
  And the objective overall is to enable the creation of new AI
• 16:29
  products. We target like, either, if
• 16:34
  you're part of the team or you're an individual
• 16:38
  contributor, we try to bridge this gap between, okay.
• 16:44
  I think I know what are the research ideas that I want to
• 16:50
  transfer. This form and turn it into a
• 16:53
  product, but I don't really know where to start, right?
• 16:59
  So prioritize and Mel wants to shorten the time from this is
• 17:05
  the idea that I want to build and I how I want to deploy this
• 17:11
  and wrap it, do like a packaging of the product and as I was,
• 17:21
  saying this is, The easiest way to, I thought that we could do,
• 17:26
  this is through educational resources, to keep everyone in
• 17:30
  your team aligned and improve in some way, the
• 17:35
  cross-functionality of, of the team.
• 17:38
  Yeah, and I'll throw in the chat right now, for anyone who wants
• 17:41
  to like, jump ahead and look at what it is that you've created,
• 17:46
  because I'm like, I'm blown away.
• 17:49
  You know that, I love it. That's why I got you on here,
• 17:52
  but I think I think it's it's Really important.
• 17:54
  This idea of the different stakeholders that you talked
• 17:57
  about, and the translation that needs to be done.
• 18:01
  Because so many times people come on here and they talk about
• 18:06
  how the ml Ops problem is an organizational problem and part
• 18:10
  of that is because people don't understand each other or they
• 18:14
  don't have the right access to this or that or they can't say
• 18:18
  what it is that they need to be done in a way that the other
• 18:22
  person understands the other. Teammate, that is doing things
• 18:25
  and I think there's parts of that that are really getting
• 18:28
  better as this becomes more mainstream.
• 18:31
  Right? But what you're doing is like a
• 18:34
  huge service to all of the rest of us because like you said
• 18:38
  you're getting everyone on the same page so that at least you
• 18:44
  can tell your teammates or you can tell the team at the very
• 18:49
  least, just go through and do this quick course, and you'll
• 18:53
  have an Standing of what I'm looking for.
• 18:55
  I will understand what you're looking for.
• 18:57
  We can speak the same language and you will know what I mean
• 19:00
  when I talk about this or that, so I really appreciate that you
• 19:04
  did this. I thought it was, it was
• 19:06
  incredible when I saw it and the idea of the stakeholders is huge
• 19:11
  and who needs what from machine learning, right?
• 19:16
  Like machine learning piece to each person is a different
• 19:20
  thing. Yeah, correct in That way, I try
• 19:25
  to like to break down each of the lectures with targeting like
• 19:29
  different teams and different audiences.
• 19:31
  Right? So, as you're saying, the first
• 19:36
  lecture, for example, like, the machine learning lecture, we
• 19:41
  tried to present some of the main algorithms, like, for
• 19:46
  supervised and supervised learning.
• 19:47
  So, and this lecture is mainly focused for like product
• 19:51
  managers, that might Not be like in a Enough full understanding
• 19:58
  of what these Technologies do. One require in terms of data and
• 20:04
  even for data management teams to write.
• 20:07
  Why do we need all these huge number of datasets?
• 20:15
  Right? What is behind those deep
• 20:19
  convolutional neural networks? Why do we need to represent?
• 20:23
  Resent or to cover all the classes.
• 20:27
  Such a. We don't suffer for from an
• 20:31
  classism. I'm balancing.
• 20:33
  Right? And I also for, for marketing or
• 20:38
  Business Development guys. Why is it taking like so long to
• 20:44
  train like a like a deep, convolutional neural network,
• 20:49
  things, things like that, and other sections, That just come
• 20:55
  to my man. For example, the the last one,
• 21:00
  for example, it's called productized it, right.
• 21:05
  This is especially useful for folks, like like like like me in
• 21:11
  the past who were purely focused on the engineering side of the,
• 21:19
  of the solution that we were willing and And this lecture
• 21:25
  resonates a lot along the lines of what is important or what is
• 21:33
  the product that I'm building? What are the business objectives
• 21:36
  that have? Because at the end, I believe
• 21:38
  that we should build our Technical Solutions considering
• 21:43
  that and that that means defining or answering some of
• 21:49
  the questions such as what is the benefit that I'm they From
• 21:54
  automating this process or making these predictions more
• 21:59
  accurate. Should I be carrying a lot about
• 22:03
  the accuracy of the model? Or for example, the coverage of
• 22:08
  classes? Because I don't know, certain
• 22:11
  customers or clients might care more about.
• 22:14
  Yeah, about the coverage of classes, that, for example, your
• 22:19
  classifier can have versus a really high.
• 22:23
  Performance and also like defining what's the MVP that you
• 22:31
  want to feel? This is something that
• 22:35
  engineer's my lack of understanding.
• 22:39
  Sometimes we sometimes care too much about.
• 22:43
  The, the accuracy, for example of the of these product and
• 22:50
  sometimes we lack of this. Business size, right?
• 22:55
  Yeah. I see some there's a great meme
• 22:58
  that Ron just put in the chat about.
• 23:02
  Basically. It's like for everyone that
• 23:05
  already clicked on it. It's like work, fine.
• 23:07
  And Dev. It's an Ops problem now.
• 23:09
  And you see the building blowing up in the background.
• 23:12
  I wanted to ask a few more questions before we maybe jump
• 23:16
  into the actual the course and your brainchild and I just think
• 23:24
  it's fascinating that you took a different approach to things
• 23:28
  because right now, we're seeing a ton of companies that are
• 23:31
  coming out, right? That are trying to help
• 23:34
  standardize the ml workflow. Or they're trying to help be the
• 23:39
  tool for X Y or Z and a lot of these companies we've had on
• 23:43
  here. They're in the community.
• 23:44
  They're great. But your idea of a platform was
• 23:48
  different. You said well, let's make it
• 23:51
  educational instead of Of a tool.
• 23:56
  And why exactly did you go about that?
• 23:58
  Just because it was going to be an easier, Endeavor.
• 24:00
  I'm sure it didn't end up being that easy.
• 24:03
  So what was it that made you think education?
• 24:05
  Here is probably going to be more powerful than creating some
• 24:10
  tool that does all of this for you.
• 24:12
  Yeah, that's that's a great question.
• 24:15
  So there is a there is this Chinese proverb that says, give
• 24:19
  a man a fish and you feed him like for the Teach a man to fish
• 24:26
  and you feed him like for a lifetime.
• 24:30
  So this is actually the the goal that I had with precise amount
• 24:38
  not to be like, solutions for a custom specific customer or
• 24:44
  specific, the problem that I have.
• 24:46
  But to see from the, from the outside and try to provide like,
• 24:53
  Road provide like some content that might extrapolate to two
• 24:59
  new products and new ideas. And after all, this is what I
• 25:04
  did in the past right at the end.
• 25:07
  I think Innovation comes also from seeing examples or being
• 25:13
  influenced by other Solutions. So this is what we try to do in
• 25:20
  productized. Ml just offer like some some um,
• 25:23
  some tools, some resources. So new products can come benefit
• 25:29
  from from it. Yeah, that makes total sense.
• 25:31
  I mean, it's it's a great way of doing it and it is also I feel
• 25:37
  like it's very versatile. Like you can get anyone on board
• 25:43
  with this. It's not like you need to go and
• 25:46
  try and get the CFO to buy a big product so that your Team can
• 25:54
  work more efficiently. It's like you can just use it as
• 25:56
  a Grassroots and people that hopefully, like what I hope for
• 26:00
  is that people that are on this call and people that are
• 26:02
  watching this later, on, in the future.
• 26:04
  They see this and maybe they've been having trouble at their
• 26:07
  company. Their current company.
• 26:08
  Maybe they're a lone wolf in their company and they are
• 26:11
  trying to have people understand.
• 26:15
  What is it? They are actually doing and they
• 26:18
  can show their colleagues this and and it will help everyone
• 26:22
  get on the same page and So I think you've done a great job at
• 26:26
  that. I want to ask one more question
• 26:28
  on the the content of it because there's a cool question that
• 26:33
  just came through the in the chat and it was back when you
• 26:36
  were working on your like the skin Legions.
• 26:40
  If you were training your data sets with different colored skin
• 26:45
  and how that worked and also because it's like there's
• 26:50
  generally a training on lighter. And so you get a bit of a bias
• 26:56
  and that kind of goes into the question that I wanted to ask
• 26:59
  around. You're in productized.
• 27:04
  Ml. There's nothing about like
• 27:07
  really ethics in there. Do you feel like you want to put
• 27:09
  that in there later? Yeah, that's that's definitely
• 27:13
  like a really good question. Regarding like the, the way that
• 27:16
  we were training the these Collision classifier.
• 27:20
  We were trying to sample from different like Skin tones, but
• 27:26
  it's true that there is like a lack of data like availability,
• 27:32
  right? For some skin tones.
• 27:35
  So some of the methods that we were using to this something
• 27:42
  that we touch on the during the course is the fact of datum.
• 27:47
  I went Fishin right just to be large those classes that Black
• 27:53
  from from images, in this case. So we try to expand those skin
• 28:02
  images with darker skin tone, for example, through those toes
• 28:09
  methods and regarding like the ethics in the course.
• 28:16
  This is actually what I see like, more AI products.
• 28:24
  Trying to integrate this aspects.
• 28:29
  So we should definitely add like a, like a section.
• 28:34
  We don't have any right now, but We coming, he's coming.
• 28:41
  Well, that's that's, that's also something the matter.
• 28:44
  Is that the course it's quite hard like to cover like all,
• 28:50
  what's going on, right? We're seeing for example right
• 28:54
  now. Like Transformers that they're
• 28:57
  aiming like to to take on convolutional neural networks
• 29:03
  for visual classification. Not just for that.
• 29:08
  Classification. But so this content will be
• 29:12
  extended and over time and hopefully, the community will
• 29:19
  will help also to, to make it like happen.
• 29:25
  Yeah. Yeah.
• 29:26
  And well, I think that's a perfect segue into another
• 29:29
  question that I had like, is there a way that people can get
• 29:32
  involved and participate in this?
• 29:35
  Because, as far as I know, it's like, Like, it's an open
• 29:38
  Collective right now, right? Yeah.
• 29:40
  That, that was one of the objectives when I started like
• 29:44
  this project. I think you probably know this
• 29:49
  with the ml Ops community that there is like a huge number of
• 29:53
  people with really talented people that are willing to
• 29:58
  contribute on the same way that at some point, we have borrowed,
• 30:03
  some of these Knowledge from Sac covered.
• 30:07
  For example, and forums like this at some point.
• 30:13
  I think we have to contribute back to the community the best
• 30:19
  way that I thought was through this through this platform and I
• 30:24
  chose the tool that is built with its get books.
• 30:29
  And this platform basically Works under the get method.
• 30:37
  Philologist. So it's linked to like a GitHub
• 30:42
  repo and anyone can actually open like a PR with changes on
• 30:49
  the content. And that's the way I see like
• 30:52
  external contributors to contribute to the existing
• 30:58
  content. Yeah, that's another awesome way
• 31:03
  of doing it because like you said, there's so many smart
• 31:06
  people out there and Probably got some of their wisdom to
• 31:10
  share. And so I encourage anyone that
• 31:12
  is out there watching and maybe you have some extra parts that
• 31:18
  you want to throw on there. Yeah, PR is our welcome.
• 31:21
  That's very cool. There's a quick question in the
• 31:23
  chat coming from Ron about how these models are actually
• 31:27
  packaged for production. I imagine he's talking about the
• 31:32
  skin lesion models that you were working on.
• 31:35
  Were you using docker? Sir with flask restful service
• 31:39
  python pickle files, dll tensorflow.
• 31:43
  Light tensorflow serving is. Am I correct in saying Ron?
• 31:48
  You'll tell us in the chat if it was for the, the skin lesion
• 31:54
  models. Are you talking about?
• 31:56
  Yeah. Yeah.
• 31:56
  He says, yeah. Yeah.
• 31:58
  So yeah, so the the way that we were working in this company the
• 32:02
  data scientists were training, those models using Kara's we
• 32:06
  were Converting the smallest into tensorflow and we were
• 32:10
  using D tensor form serving to create the docker images that we
• 32:17
  were hosting in the cloud. And then we were using a
• 32:22
  coordinate is 2 to bring those models into Productions in the
• 32:27
  form of an API that we were using in the front end.
• 32:37
  So one thing that jumps out at me as you explain that and as I
• 32:41
  look at, you know production eyes.
• 32:46
  Ml again is going through the The higher level like that being
• 32:53
  able to explain something like that, at a very high.
• 32:55
  Level is so useful. And then really nailing down and
• 32:59
  getting into the nitty-gritty. I think, is there a plan to go
• 33:05
  like super deep and talk about things in a lot of detail for
• 33:11
  the course? Or is that something that you
• 33:14
  want to just stay high level? So it can be something for
• 33:17
  everyone. Yeah.
• 33:19
  That's that's a great question. So, So I try to keep like this
• 33:23
  high level just to prioritize based on the each of the
• 33:28
  lectures were contributing to actually like create like
• 33:35
  products. So what is the impact that these
• 33:38
  Technologies or this methodologies that I'm I'm
• 33:43
  teaching in. This course are having on on the
• 33:46
  industry and on this Society. So it's All matter of
• 33:52
  prioritizing things, we will all obviously like extend this
• 33:58
  content over time, and not only that but I think I really
• 34:03
  important point to is to offer like Hands-On experiences,
• 34:08
  right? Something that it can complement
• 34:12
  the theory and this is something that we will be releasing really
• 34:17
  soon. I call it catalog.
• 34:20
  Ugh. And this catalog will include
• 34:25
  Jazz examples that you can download.
• 34:31
  It's like an end-to-end. Ml lifecycle for like a specific
• 34:36
  task and product. We will start.
• 34:39
  We have like a one of these catalog items will be like an
• 34:45
  urban sound classifier and we have others.
• 34:50
  For visual classification. Tasks.
• 34:53
  So with this, we try to inspire people to create some new
• 35:02
  products, based on the ones that we are offering.
• 35:07
  And this is the way that we will try to.
• 35:10
  Yeah. Expand the the existing course.
• 35:14
  Yeah, and for that like, how can I stay informed?
• 35:21
  I think you guys have you have like a slack Community or is it
• 35:24
  Discord? I can't remember.
• 35:26
  Yeah, we have a server on this chord that where we keep
• 35:33
  everyone like updated with the as soon as.
• 35:37
  We update like a section or we are like a new lecture on the
• 35:41
  course. We will keep everyone updated
• 35:44
  their. We also have like a Twitter
• 35:46
  account. And for now, you can also sign
• 35:50
  up in our newsletter, that we will try to send like updates
• 35:56
  every month with with this content as well as with these
• 36:01
  catalog. Items that I was, I was telling
• 36:06
  you before so nice. So as you look forward and
• 36:11
  you're thinking about what's coming next, I mean you talk to
• 36:15
  us about the Hands-On stuff. Is there anything else that you
• 36:18
  want to bring to the website and they get book?
• 36:21
  And is it just more collaboration with outside
• 36:25
  sources, or do you have anything else up your sleeve?
• 36:29
  Yeah, so we're trying to also reach like Other collaborators
• 36:35
  people that are really interested on.
• 36:39
  Yeah, educational Technologies and machine learning as well.
• 36:44
  So if you're definitely one of those, please reach me out and
• 36:52
  this is something extending like the team right now.
• 36:58
  I'm working with Some internally contributors that I'm really
• 37:05
  thankful to be working with with them.
• 37:08
  They're really talented. People and experience from
• 37:11
  software, engineering to Prague managers and ml Engineers that
• 37:17
  are helping me with this, with this content.
• 37:20
  So, as I was saying, yeah, the productized ml catalog.
• 37:26
  I see like a lot of really good examples, always, Trying to
• 37:32
  follow this line of not Building Solutions, but building tools or
• 37:39
  examples that people can take us as inspiration.
• 37:46
  Hmm. And you feel like it'll ever get
• 37:48
  to the point where you're going to be talking about certain
• 37:52
  tools? Or are you going to try to stay
• 37:56
  away from that? Hmm.
• 38:00
  I think it's pretty high. Hard without like I really deep
• 38:06
  knowledge off. The problem that you're trying
• 38:08
  to solve depending on the data structure that you're working
• 38:15
  with. I think it's pretty hard right
• 38:18
  to ultimate the tools that someone would use to create like
• 38:26
  a product, but instead, as I said, I think Humans work really
• 38:34
  well on getting Spire by things that going back to where we
• 38:40
  started this conversation. From with the idea of payment
• 38:43
  net and moving this idea of General visual classification
• 38:49
  task to more like a specific for for skin.
• 38:54
  Lesions classification. That's that's where we are
• 38:58
  aiming to with this. With these catalog.
• 39:04
  And as far as like, people that you want to see, get the most
• 39:09
  value from this, is it somebody that is that a huge Enterprise
• 39:14
  or is it someone like, like, when I look at it?
• 39:18
  It I look at it as someone at a startup that has a team, and
• 39:22
  they're not really functioning that.
• 39:23
  Well, there's maybe a little bit of dysfunctional, Team Dynamics
• 39:28
  and, and this can help everyone get on the same page.
• 39:31
  As I mentioned earlier. Is that the same vision that you
• 39:33
  have or you? Looking at it as more of
• 39:36
  something for everyone. Hmm.
• 39:38
  Yeah. So the two main all this
• 39:41
  audiences, I think that we are trying to Target here is teams
• 39:48
  coming from. Start up your even like big
• 39:50
  corporations, but also to like individual contributors here.
• 40:01
  There is like the point of of the pricing.
• 40:04
  For example, of the of the course.
• 40:08
  I really want this course, to promote the creation of new
• 40:13
  products. And I was thinking than most of
• 40:18
  the individual contributors, that they can not like, really
• 40:21
  afford like some of the prices that some educational courses
• 40:26
  are out there, right? So, but still These are really
• 40:31
  important like contributors. I'm not sure how many if you
• 40:35
  have done some statistics about the MLS Community if they're
• 40:39
  like purely like Indians or like they belong to Big corporations,
• 40:45
  but my point is a demon, like, small communities like of
• 40:51
  individual contributors. They have, we need to empower
• 40:55
  them right to with these tools and resources to really make
• 41:03
  something out of it. And that's why I'll try always
• 41:10
  to keep this this course free. And also, there is another
• 41:16
  reason for which there is this concept of public interest
• 41:21
  technology tool for for which we try to promote like technology
• 41:29
  on society. An offer like this knowledge to
• 41:35
  even like people are doesn't want to build like products for
• 41:39
  that purely. Just wants to understand what a
• 41:43
  I can really do and try to uncover this wrong label that we
• 41:49
  sometimes put on a i about like a black box, how?
• 41:56
  Netflix, for example, recommendation systems work or
• 41:59
  how Pinterest is suggesting us like pretty, like visually
• 42:05
  similar images through like image retrieval systems, or even
• 42:10
  like, when we see some of the deep fakes from, from Trump, or
• 42:15
  how do these, like, General, a generative, adversarial networks
• 42:20
  work, right. So these are the three main
• 42:23
  audiences that I see these course.
• 42:28
  Targeting. Yeah, that makes sense.
• 42:30
  And I think we should talk for a minute about that.
• 42:34
  Like right now it's totally free.
• 42:36
  Right? And you want to keep this free,
• 42:38
  but you want to have more built on it and it did you see it as
• 42:42
  being like certificates at some point, where it gets more
• 42:47
  specific and then you can get certificates from it.
• 42:50
  How does it? How do you envision that?
• 42:52
  Yeah, so I see a pro ties ml like dividing the efforts
• 42:57
  between like the course that I as I said, it will always be
• 43:02
  free. Well, we obviously accept some
• 43:05
  some donations through buy me a coffee.
• 43:08
  That's a beautiful platform, that I suggest that everyone to
• 43:12
  to check out. And the other thing is the, the
• 43:17
  the the other from that productized ml want to focus on
• 43:23
  is the catalog for Or which I think we will start like setting
• 43:30
  some some prices for, for this catalog of different tasks, and
• 43:35
  examples of tools and products. So we can maintain like both
• 43:43
  businesses, nice. And I'm seeing another question
• 43:48
  coming up here in the chat and it's a bit more specific
• 43:51
  question about how, you know, when you're Or ml model needs a
• 43:58
  new updated model. How do you know, once your
• 44:01
  productionize model needs to be updated?
• 44:05
  That's a great question. I would say it.
• 44:08
  Depends on the domain that you're trying to use your your
• 44:15
  model. The first steps sometimes.
• 44:20
  Just, you're just a Jesus have access like, a limited, like,
• 44:27
  coverage of the real domain that ideally, your product should
• 44:32
  work really well on, and on one of the aspects that having a
• 44:41
  system in production. The company can benefit from is
• 44:45
  the concept of data fly willing and Basically resonates around
• 44:53
  the idea of, by the time that you started pulling your
• 44:56
  services, you're not exclusively making predictions, but you are
• 45:01
  also collecting data from your users.
• 45:05
  So, here comes the, the point of that.
• 45:09
  You are actually sampling from the domain that you are one that
• 45:15
  you want to predict. Right?
• 45:17
  And here is the point that That as soon as you label these new
• 45:23
  data that your sampling and collecting from your you from
• 45:27
  your juices. That's the point.
• 45:29
  I think that you should be retraining your model.
• 45:33
  So we can get more accurate on the final objective that it's to
• 45:39
  solve those real cases. Yeah, and you mentioned
• 45:46
  something something here in the in the website about really like
• 45:51
  knowing also the kpis and what these kpis are and what you're
• 45:56
  really shooting for. What's the model trying to do
• 46:00
  don't like ever forget that higher level.
• 46:03
  So as you look forward and looks like we're kind of running out
• 46:06
  of time here the whole idea around production izing your
• 46:11
  machine learning. When you're looking forward into
• 46:14
  the, for the next year, we'll try and say or even less than
• 46:17
  that the next six years. How do you feel like things are
• 46:20
  evolving? Because like you mentioned
• 46:22
  before it's going so fast and it's at a speed that things are
• 46:27
  happening so quickly. Where do you foresee things
• 46:31
  going? And some like updates that
• 46:34
  you're going to need to make with the course or just some
• 46:38
  bets that you're taking that. You hope it will go one way or
• 46:41
  another. Yeah, so you're racist, question
• 46:45
  of adding some ethics on the course.
• 46:50
  So definitely. That's going to be like some of
• 46:53
  the priorities to add in the course things like offer like a
• 47:00
  less biased. For example, your products will
• 47:03
  be have mentioned the, the skin tone like aspect of of the skin
• 47:08
  classifier and other other. Other ethical sites, for
• 47:16
  example, there is this company that's called you that calm.
• 47:21
  That was the it was founded by Richard Sutter who is like a
• 47:28
  former Chief data scientist that say false.
• 47:32
  Anyways, the the the idea is to come up with products.
• 47:38
  This is like a search engine that tries to be less biased.
• 47:43
  Um, that doesn't only respect your privacy, but also your body
• 47:47
  is right. So, I see prioritize, I may like
• 47:51
  offering some of these lectures to another aspect or another
• 47:59
  lecture that I would like to see it like more topics regarding
• 48:04
  like explain ability of machine learning, right?
• 48:08
  Trying to move on from the idea of black box.
• 48:16
  Yeah. Yeah, and I'm sorry.
• 48:20
  I was wanted to say also like to touch on all the industries.
• 48:26
  For example, like healthcare and finance.
• 48:29
  I still like out of power and many new AI products.
• 48:36
  I think we'll follow those those lines.
• 48:39
  Mmm. Yeah, and it begs the question
• 48:41
  for me. Like, do you ever see what you
• 48:44
  do with the educational? Be?
• 48:46
  In more like vertical specific. Do you feel like you're going to
• 48:50
  be having educational product ties?
• 48:53
  Ml for healthcare industry or for fintech and for these
• 48:58
  different areas because it is a little bit different.
• 49:00
  Once you go into each one of these these there's nuances,
• 49:04
  right? Yeah.
• 49:06
  That's that's a great idea or way to say it.
• 49:11
  Maybe each of the lectures as you said they should.
• 49:16
  Be like more focused on each of the of the domains because data
• 49:21
  will look like completely different from one domain or one
• 49:26
  industry to the others. So as soon as we get a week
• 49:34
  span, like the team, I do see this like getting more deep
• 49:42
  going deeper on the each of the of the lectures and being More
• 49:46
  specific. Yeah.
• 49:49
  Well, you've done great work so far.
• 49:51
  I'm excited to see what you create next.
• 49:54
  Anyone out there who has anything that they want to add
• 49:58
  create a PR because it's just it's an awesome resource.
• 50:03
  I threw it in the chat and if you're listening from the
• 50:09
  future, you can check it out in the description below.
• 50:12
  I want to say thanks, either. This was coconu.
• 50:15
  Do Ha, ha ha. Thank you that messages for ya
• 50:20
  for organizing this meetups. You have like a beautiful
• 50:24
  community of there. So that's, that's amazing.
• 50:28
  Yeah, awesome man. Well, thanks again and everyone
• 50:31
  check out product ties. Ml right now.
• 50:36
  Get your whole team on it. Tell your p.m.
• 50:39
  Tell your data Engineers. Tell everybody tell everybody
• 50:42
  about it because I think it's a great thing that you're doing
• 50:45
  and And we'll be back. Thank you all for tuning in and
• 50:51
  thank you. Either, e, it's been awesome.
• 50:53
  Thank you. Bye.
• 50:54
  Bye.