Latest / Agentic Conversations (formally mlops.community) / Learnings from Live Coding: An MLOps Project on Twitch // Felipe Campos Penha // MLOps Meetup #63

Transcript

Learnings from Live Coding: An MLOps Project on Twitch // Felipe Campos Penha // MLOps Meetup #63 · Agentic Conversations (formally mlops.community)

• 0:16
  Now we're just getting right into this conversation with
• 0:20
  Felipe. As you can see.
• 0:22
  This man is a professional. You have some high quality
• 0:26
  sound. And look at that background.
• 0:28
  Look at this guy does not mess around.
• 0:30
  Around. We're doing some live coding and
• 0:33
  learnings that Philippe has had. He's started doing ml Ops live
• 0:41
  coding, a while ago and that's how you came up on my radar.
• 0:46
  That was something that I saw you doing because you posted
• 0:50
  something on LinkedIn, maybe or it was just like it popped up in
• 0:54
  my YouTube feed and I thought that's fascinating, like, I
• 0:57
  really like that and I know that You've been, you just got to a
• 1:03
  new job, but you've also been an astro, particle.
• 1:07
  Physicist, and you've gone from working as like, a data
• 1:13
  scientist or doing machine learning in a research setting,
• 1:16
  and then having to productionize machine learning.
• 1:19
  So, I want to talk about the differences that you see there,
• 1:21
  and I also want to talk about twitch and what you're doing on
• 1:26
  Twitch, how that whole story happened, I think.
• 1:30
  But before we get too ahead of ourselves, we should learn a
• 1:34
  little about you, and you can give us the whole background on
• 1:38
  how you were doing this astrophysics and then you got
• 1:43
  into machine learning and what that Journey looked like.
• 1:46
  Yeah. So my background is in physics
• 1:49
  and my bachelor's Master's and doctorate Alwyn Brazil from
• 1:53
  Brazilian universities University of São, Paulo, and
• 1:56
  University of Campinas. So in the doctorate, I focused
• 2:01
  on particle physics and in special wastrel particle
• 2:05
  physics. That's a mix of astrophysics and
• 2:08
  particle physics. And in that work, I had a lot of
• 2:13
  experience dealing with data. Like in that context no one
• 2:16
  called the data science. But it's kind of data science,
• 2:19
  right? It's getting data from
• 2:21
  accelerators and different Detectors of astroparticle
• 2:26
  physics so particles. And getting that data and
• 2:31
  analyzing it different ways. For example, one thing that I
• 2:34
  did was like a type of regression and the functions
• 2:38
  that I were actually that I was actually fitting, they were
• 2:41
  integrals like integral functions, so it was very
• 2:44
  heavily, very heavy computation Ali and I even used the universe
• 2:49
  cluster to make all this calculations.
• 2:52
  So it was kind of data science. Let's say some machine learning
• 2:56
  my basic machine learning stuff. Of and this is Astro particles
• 3:01
  that I'm talking about. They are like particles that
• 3:03
  come from different places in the universe, we don't really
• 3:07
  know exactly where each come from, but they come from, for
• 3:11
  example, stars that are exploding, supernovas things
• 3:14
  like that and they reach the Earth by accidents.
• 3:17
  And they, we detected here on the ground and there are many
• 3:20
  underground detectors like in the mountains, for example, in
• 3:25
  Italy. There's one that's famous called
• 3:27
  Gran Sasso Lab where they have a huge tank of water and they just
• 3:33
  keep waiting for some particles to Traverse that tank of water.
• 3:37
  And the the experiment that I'm most worked with, I mean, using
• 3:42
  data from it and trying to get inside spray, it was Ice Cube.
• 3:46
  That's a huge detector in Antarctica.
• 3:49
  That's actually one kilometer cubic.
• 3:52
  You know, it's because it's is the ice of Antarctica and I have
• 3:56
  some holes with detectors, And the whole volume of water ice
• 4:01
  actually adapted using as a detector is like one kilometer
• 4:07
  cubic. So it's huge.
• 4:09
  It's like it's even hard to imagine the size of that.
• 4:12
  Yeah. It's amazing.
• 4:14
  Yeah. So maybe something about how I
• 4:18
  transition right to the industry in the end of my Doctorate.
• 4:22
  Of course I was thinking about what I would do with my life.
• 4:27
  after investing so much from Academia, so much of my time on
• 4:32
  Academia and I applied for many postdoc positions things in
• 4:37
  Brazil were a bit hard in that moment because we had too many
• 4:41
  doctors like men, too many people with phds equivalent of a
• 4:45
  PhD with less like positions to fill There was a time that
• 4:53
  people spent time abroad, like I did this, I stayed one year and
• 4:56
  a half and the lower here in the US.
• 4:59
  So many people with very good CV competing for the same.
• 5:03
  Postdoc positions was really hard for me, then I realized I
• 5:06
  could enter the industry. Maybe then I applied first
• 5:09
  fragile physics company and this opened my mind because I never
• 5:16
  realized I could work in a company.
• 5:19
  So I Arted studying a bit of data science and some boot
• 5:24
  camps. You know, I heard about this
• 5:26
  years back and I started seeing that in Brazil companies were
• 5:31
  hiring for data scientists. Then I started investing in
• 5:34
  this, I moved to São. Paulo, starting started having
• 5:38
  many interviews and I ended up in a company called Neo a and I
• 5:43
  worked with them for three years.
• 5:45
  And I moved here to the West transferred to the Manhattan
• 5:50
  office that they had here. So tell me a little bit about
• 5:53
  how like, developing these models for companies with
• 6:00
  different from developing models in physics.
• 6:03
  You mentioned, like, you were doing data science a little bit
• 6:07
  and then you got into it more. And you start to realize, well,
• 6:10
  I can work for a company. How did that differ?
• 6:14
  Yes, I think the Dynamics, like, the day-to-day dynamics of work
• 6:18
  is very different because when you are in the Academia, Usually
• 6:22
  you have a supervisor and you'll meet with this person so many
• 6:26
  times. It's like weekly meetings.
• 6:29
  And the steps are is low, you know, I felt always that I was
• 6:34
  doing something you've very slow.
• 6:36
  I was working a lot during my doctorate but doing it in the
• 6:40
  slow pace in the industry. I felt that like everything is
• 6:45
  as people say a fire drill, you always need to do things so fast
• 6:50
  and to so things, as Because possible because there's a lot
• 6:53
  of money involved, right? Their customers on the other
• 6:57
  side, Then also the whole agile way of working.
• 7:04
  You don't have this in Academia. This is the first thing not
• 7:08
  talking about data science itself, just the Dynamics.
• 7:11
  The other thing in terms of data science, for example, I was
• 7:13
  doing regression as I explained during my doctorate, but the
• 7:18
  purpose of that regression was different was not to really
• 7:22
  predict so. Well, the phenomenon of course
• 7:27
  you want that but what you really we want is to discover
• 7:30
  the parameters of your functions that you are fitting.
• 7:33
  So for example, one parameter can be the mass of a particle
• 7:37
  and you really care about the mass of the particle in
• 7:39
  stablishing limits for the mass of that particle Wyoming data
• 7:43
  science. In the industry, you can have
• 7:45
  thousands of coefficients and regression in a linear
• 7:50
  regression for example and you don't care much about the
• 7:53
  coefficients to care more about. However, it's predicting.
• 7:57
  Although you care Coefficients to interpret your model but like
• 8:02
  your interpret, you will throw them away, you don't care
• 8:04
  anymore. You ran another prediction,
• 8:06
  another like you train again, you'll throw everything out
• 8:09
  because now you have a new model in physics.
• 8:12
  Like the parameters are really important.
• 8:14
  They have a physical meaning, they really mean something, so
• 8:18
  that it's a huge difference in mindset, I think, So, I'm always
• 8:25
  fascinated to hear about this idea of like the data scientist,
• 8:31
  who's going into operational machine learning.
• 8:34
  And we talked a lot about this in the community of like, is a
• 8:40
  jupyter notebook, ready for production or should you use a
• 8:44
  drip in her notebook in production?
• 8:46
  I think, is the better way of phrasing that question.
• 8:49
  And I know that you have some thoughts on that transition.
• 8:53
  Mission, you went through that transition going from like a the
• 8:56
  Lone Wolf, jupyter notebook, data scientists and then into
• 9:01
  operational ml, can you explain that process a bit and then
• 9:05
  after that I see there's some people in the chat that are
• 9:07
  mentioned and they are doing some live coding to so we'll get
• 9:10
  into that. But I want to hear about this,
• 9:13
  this idea of transitioning from a jupyter notebook to
• 9:16
  operational ML and were you using your jupyter notebook in
• 9:19
  production? Please say, no.
• 9:22
  And what that was like, Yes. So I watched your previous
• 9:27
  interview and I Heard lots of time, they're like proof of
• 9:33
  Concepts, how? Right?
• 9:35
  You'll see. Hell.
• 9:37
  And I think I lived there that. So, for almost 3 years, I was
• 9:44
  working only on plc's and most of the time it was on jupyter
• 9:49
  notebooks because the thought of business was always, Is like my
• 9:54
  customers, basically, they always wanted to see the value
• 9:59
  and try to roll out, more manual work, let's say, for example,
• 10:03
  spreadsheets to sales people, so they could try out that module
• 10:08
  on the fields first like, very mentally with spreadsheets, then
• 10:13
  maybe later as a problem, you know?
• 10:16
  So, and this takes a lot of time for people to really understand
• 10:20
  the Project's, understand the results.
• 10:23
  And really like have the buying of stakeholders.
• 10:28
  So I was living in that file. Let's say only in jupyter
• 10:31
  notebooks, but in many instances, I was like running
• 10:37
  code manually and sending it to a bucket for example, in AWS or
• 10:42
  the customer could read it from there.
• 10:44
  And at some points, I would schedule that job, but it was
• 10:49
  like at that level when I was leaving that company that I
• 10:53
  worked for three years to First draw by hand, I started
• 10:56
  organizing my code, a bit better to hand out my colleagues.
• 11:01
  And that was some kind of transition where I started
• 11:04
  paying attention to more software development or at least
• 11:07
  organizing better my code, you know.
• 11:11
  And then I had some peer that often employment.
• 11:15
  I invested a lot on, trying to learn these things by myself.
• 11:18
  As you said, they start is really hard to find sources.
• 11:23
  For that like there's not many places where you can start
• 11:27
  learning like Machinery, operations during software
• 11:30
  development for machine learning from from the start, like from
• 11:34
  the scratch. So I started doing this made
• 11:38
  myself, I started a job. This job was brief, was like,
• 11:42
  for four months that I had, but it was product-related and I
• 11:46
  started paying more attention to that working with people.
• 11:49
  In the same repository, at the same time.
• 11:52
  For example, I always use GitHub but it was like a drop box,
• 11:56
  right? You just throw things there in
• 11:57
  the master. Are now it's called name, right?
• 12:01
  And I started working really, with other people's Started
• 12:06
  paying attention. I had firm on my own choice with
• 12:10
  my manager. I had the role of being like the
• 12:13
  police of software development practices best practices.
• 12:17
  So, I was learning and also trying to enforce that, let's
• 12:21
  say inside the company. Wait, so did you, did you learn
• 12:25
  during this period between jobs that did you learn about
• 12:30
  software engineering? Just the basic fundamentals of
• 12:33
  software, engineering, or was it with that skew?
• 12:36
  Towards machine learning. That's, it's hard to send it was
• 12:41
  not linear, but from for some job interviews that I had.
• 12:45
  I did a lot of job interviews during the last year during the
• 12:49
  pandemic. And I knew that for some jobs, I
• 12:53
  had to know a lot of this software development side.
• 12:58
  Let's say and I invested on this myself just to be able to tell
• 13:02
  people that I was at least learning these things, you know.
• 13:06
  Because I don't like to lie your views.
• 13:11
  So I really was trying that done in around January this year.
• 13:16
  I started a project called greenhouse on GitHub, will start
• 13:21
  talking about this I guess because it's related to Twitch
• 13:24
  first. I wanted to to learn the wrist
• 13:27
  language ring, and I decided to make my own framework, or than
• 13:32
  plate, let's say, for projects for rust and I don't know why.
• 13:36
  Why I started going deep on this thing of making the templates of
• 13:42
  wrists for wrist development. And I didn't know much about
• 13:45
  wrist itself, but I wanted to like set up my Docker
• 13:50
  environment like my containers at everything before starting
• 13:53
  learning and I had this whole framework.
• 13:56
  What could go wrong? But you learn a lot because rest
• 14:02
  has a very that's a fixed way of doing things like the package.
• 14:10
  As many different pieces, you need to concatenate.
• 14:13
  Like you need to glue everything together very well.
• 14:16
  So I learned a bit about this, this product is not going on
• 14:20
  anymore. Now I'm focusing on python for
• 14:22
  that project started like that. Well, let's talk a little bit
• 14:26
  about your project and what made you start going into YouTube and
• 14:31
  twitch? And like, yeah, what was the
• 14:36
  Catalyst there? Yeah, so YouTube channel came
• 14:41
  before came first. Both are called data science, B.
• 14:45
  So, I have data science, bits on YouTube and data science bits on
• 14:49
  Twitch. The one on YouTube came.
• 14:51
  First and I was a way for me to maybe contribute to the
• 14:57
  community, of course, And also because I was struggling to get
• 15:03
  a job here in the US. If it also served as a portfolio
• 15:07
  for me, you know, Like, there's the mark Community thinking, and
• 15:12
  also more personal side that I also needed to show people.
• 15:17
  What was my level of knowledge and on YouTube was focused on my
• 15:21
  own statistics data science, career.
• 15:25
  Also, a lot of videos on performance metrics for machine
• 15:28
  learning, especially classification.
• 15:30
  It was a way to show people what I learned in this year's.
• 15:35
  And also was really cool to get feedback from the community.
• 15:39
  And getting questions answered an usually on LinkedIn people.
• 15:42
  Send messages about those videos so it was really, really
• 15:47
  interesting for me to get through this path and Then we
• 15:53
  switch was more or less by accident.
• 15:56
  Like let's say I saw some people in Brazil starting on Twitch.
• 16:01
  There's one colleague called tell he is the first like data
• 16:05
  scientists to really in Brazil start on Twitch and get more
• 16:09
  popular and I saw him like doing cool stuff there and getting
• 16:15
  feedback all the time. And I saw that movement and I
• 16:18
  wanted to join that movement, let's say and in Brazil I think
• 16:22
  the Each scene is really strong for coders right now.
• 16:26
  And I decided to start some videos very quickly, focus on
• 16:32
  that project called Greenhouse. That's a template for machine
• 16:36
  learning. Let's say the one specifically
• 16:39
  for Python, and I really loved it because on YouTube, I would
• 16:44
  usually weekly shoot some videos here with a similar setup with,
• 16:49
  but with a professional camera and there's a lot of working in
• 16:54
  editing the videos. I can doubt sounds good, the
• 16:59
  image cutting, like, yo, are all yours idiom and on Twitter, just
• 17:06
  turn on the camera. Put some colorful lights, I use
• 17:10
  my young camera in the laptop like I'm using here, so you just
• 17:17
  start coding and that's it. You don't need to add it in and
• 17:21
  you get feedback at the same time like really a bad time when
• 17:25
  you are making. In the live stream or get the
• 17:27
  ready feedback from from people that are watching that.
• 17:31
  So, it's very Dynamic. And I feel also, it's just
• 17:35
  something that I thought that during the pandemic, there was a
• 17:41
  change of mindset. I think the general audience
• 17:45
  like people were Watching so many webinars going like going
• 17:52
  watching, conferences online that they couldn't before.
• 17:56
  And there was a movement from like recorded videos on YouTube
• 18:02
  to live videos live streams on Twitch.
• 18:07
  I think people are moving from one platform to the other to see
• 18:10
  the live streams because inte, which is so Dynamic.
• 18:15
  It's almost like the YouTube life, but There's there's there
• 18:18
  are more elements there that make it really suitable for
• 18:22
  that. What do you think are the
• 18:25
  elements? Is it just like more engaged
• 18:27
  chat? Yeah, I think this is a major
• 18:30
  aspect because there's some like emojis that you can use there.
• 18:36
  It's called a motorist doesn't matter.
• 18:39
  Just some emojis. You can buy stuff, you can
• 18:42
  contribute financially like they're on the chat.
• 18:47
  You Can subscribe and contribute financially with your Prime
• 18:50
  account. You don't have to spend any
• 18:54
  additional money, you just select one streamer per month to
• 18:59
  be here, like preferred streamer.
• 19:02
  Let's say, and they redirect some money from subscription to
• 19:06
  the streamer. There are many elements like
• 19:08
  this and don't, which it's really about watching it life.
• 19:11
  Like, I don't like me personally.
• 19:14
  I don't have much patience of watching live streams that are
• 19:18
  stored there on the platform. You can watch later, Because
• 19:22
  there are so many elements that are the only makes sense when
• 19:27
  the live stream is happening, like people asking for the,
• 19:32
  there's something called rewards that when you watch you,
• 19:36
  accumulate some points with the hours that you watch that
• 19:38
  streamer and you can start buying stuff.
• 19:41
  You can buy, for example, something like I built, I play
• 19:45
  the harmonica line and then there's one reward.
• 19:49
  That's if you have, I think 1,000 points now.
• 19:52
  Now, you can ask me to play the harmonica during the live.
• 19:55
  Nice, it seems silly. But it's a way to stop some of
• 19:59
  the work that maybe people are tired and just play some stuff
• 20:04
  and put some people like, it's, it's just a playful when he of
• 20:11
  Dealing with it. So this you don't find on
• 20:14
  YouTube, you could have it. Of course, you can adapt, but
• 20:16
  which was made for that. One thing I find fascinating and
• 20:22
  the people in the audience who have done twitch or who have
• 20:27
  live coded, correct me if I'm wrong.
• 20:29
  I really liked seeing the non polished versions because I like
• 20:35
  seeing when you encounter bugs and knowing that it's Like okay,
• 20:41
  it doesn't all run smoothly the first time and knowing that it's
• 20:46
  not only me that is running into these roadblocks and then you
• 20:50
  get to see if it's someone that is is good at voicing what
• 20:56
  they're thinking, I think that's really important, right?
• 20:59
  Because you can't just run into a bug and then you go.
• 21:01
  Okay, let's let's figure this out and then you do something
• 21:05
  and you fixed the bug. It's like I want to know what
• 21:07
  you were thinking when you saw that.
• 21:09
  All right, let's go read the Boggs.
• 21:10
  Let's go figure out. Oh, it says this.
• 21:13
  So that must mean this and then you take someone on a journey of
• 21:18
  solving the problems with you. And I'd love to know your way of
• 21:24
  looking at that. And how you feel there too.
• 21:28
  Yes I think there are many types of codes.
• 21:31
  Streamers on Twitch, twitch was not made it for coding rights
• 21:35
  for gamers and people start doing their own stuff on
• 21:38
  Twitter, I have seen event People cooking on Twitch.
• 21:43
  So but about coding rain, like live coding.
• 21:45
  There are many types of people. There are some people that they
• 21:48
  just turn on the camera. They actually put themselves on
• 21:51
  mute and they say like Focus session, something like this,
• 21:56
  and they just show people what they are coding without
• 21:58
  interacting too much. Maybe on the chat only,
• 22:01
  sometimes they will turn on the microphone to say something but
• 22:04
  more focused. And then there is some people
• 22:07
  that vocalize infected what you're talking about.
• 22:10
  It's like the old deck, like rubber duck that people had on
• 22:15
  their desk. Like I will talk to that book to
• 22:19
  explain my ideas and maybe I will understand better.
• 22:22
  My thoughts my logic, so don't, which I personally do this type
• 22:29
  of streams, where I vocalize everything and I look at the
• 22:33
  documentation. And it's really nice because you
• 22:37
  lose that that fear of showing to people that you don't know
• 22:43
  everything. And then need to really look at
• 22:45
  the documentation. I'm really terrible at
• 22:48
  remembering syntax, so I Google a lot.
• 22:53
  And I always tell people that the most important thing is not
• 22:57
  remembering by heart. All this syntax is having used
• 23:00
  it before for example, in knowing where to wear or what to
• 23:04
  look for. Google.
• 23:05
  Yeah you know. So for example probably don't
• 23:08
  remember how let's say, just very simple thing how to join in
• 23:13
  a certain way with bundles but I know that it's available so I
• 23:18
  will just search on Google one minute and find out what's the
• 23:22
  same text and I don't think it's worth also having all these
• 23:27
  things by heart like this because probably in the future
• 23:31
  we will use another language and you have to learn how to The
• 23:35
  Syntax for that. So it's really important to know
• 23:37
  the concepts and I think I lost my track.
• 23:41
  My train of thoughts on this part, so they're getting back.
• 23:47
  I think I vocalize a lot and I think it's good for the entire
• 23:51
  audience to see the process. And maybe I should explain a bit
• 23:55
  about the project itself, right? Yeah.
• 23:57
  So it's called Greenhouse, it's a template for data, work, could
• 24:04
  be data. As data analytics data,
• 24:06
  engineering is going to be used for any python project, but it's
• 24:10
  more focused on this type of work.
• 24:13
  And I'm even making like, skeleton.
• 24:17
  Let's say, for placeholders for machine learning, training and
• 24:22
  prediction, but it's a template. So it's a code structure.
• 24:29
  Let's say that have many elements, for example, having
• 24:32
  containers, where you can run safely your code, that's why
• 24:35
  it's called the greenhouse because the greenhouse is, where
• 24:38
  you keep your plants and protect it from the environment, right?
• 24:43
  Hmm. During winter they survived but
• 24:46
  the greenhouse also receives things from the outside like the
• 24:50
  sunlight, right? So it's a container but it has
• 24:54
  some parts some ways of communicating so that's the
• 24:58
  matter further. Okay.
• 25:00
  Yeah. And it's really a template.
• 25:02
  So it's a code structure where you will look at some pieces.
• 25:06
  For example, where you I'm training, the model, you will
• 25:10
  replace that part for whatever you want.
• 25:13
  For your own on purpose, right? It's not the package that you
• 25:16
  call and their own codes. It's like a templates and gets
• 25:23
  help. Even has a button there to like,
• 25:25
  use this template when you click there and create a new
• 25:29
  repository you to copy out that code and it will start working
• 25:33
  on that. That's very similar to a project
• 25:36
  called Pachyderm by Apache. And other things like cat row or
• 25:43
  kiddo. Yeah, he dropped on black
• 25:45
  McKinsey. They are more.
• 25:47
  Let's say packages but I think it has a lot of similarity with
• 25:52
  Pachyderm and the reason I'm doing this right, yeah, I'm
• 25:56
  doing because there's so many other similar projects, right?
• 26:00
  It's because I'm not a machine learning engineer or machine
• 26:04
  learning operations specialist like bye.
• 26:09
  I like my background is not that, right?
• 26:12
  So I'm using that to learn a lot with the community with the
• 26:17
  audience and showing that process to the audience.
• 26:22
  Like I know best practices because I have worked with many
• 26:26
  different people, but I haven't may be implemented.
• 26:29
  Many of those things. So, and it's a very good
• 26:32
  opportunity. Defined the audience helps you
• 26:36
  or are they primarily learning? I think it's both.
• 26:41
  So there's some people there that I know they're there to
• 26:45
  learn but they also at the same time, for example, if I have
• 26:48
  some questions about how to use some specific package and trying
• 26:53
  this out the time, getting errors, some people go to the
• 26:57
  documentation ahead of time. Like, before I do that, they see
• 27:01
  the solution. They send it on the chat even
• 27:04
  though they are not. Also specially on this subject.
• 27:09
  There is Help out and the yes, firm, our resources like
• 27:15
  references for that. So they are learning a lot.
• 27:18
  There's some other people that are more experienced, but it's
• 27:21
  really interesting to see how they are influenced sometimes by
• 27:25
  what you are doing. So, for example, there's some
• 27:28
  colleagues on Twitch that they were watching me using Docker
• 27:31
  compose, and they never heard about this before, they never
• 27:36
  try to use. So on their own live streams,
• 27:38
  they started using that. Because they saw it was so easy
• 27:41
  to make a configuration file and start spinning like a jupyter
• 27:45
  notebook. That works on the content in the
• 27:48
  container, right? But exposed in a part.
• 27:51
  For example, just think simpler ways, maybe of doing things,
• 27:55
  people, learn and start reproducing.
• 27:58
  I do the same. I watch their live streams and I
• 28:01
  also start using packages that I saw there.
• 28:04
  For example, prefect, it's like a workflow package package
• 28:08
  similar to air. Whoa, I saw the that another
• 28:13
  live stream from a colleague and I started implementing this on
• 28:17
  the greenhouse project. So there's a few, like,
• 28:20
  logistical pieces that I'm wondering about a how are you
• 28:26
  deciding what projects or what problems to work on and then be
• 28:32
  how are you like Is are all the resources, your own.
• 28:38
  You're pouring all of your like, spare cash into the
• 28:42
  infrastructure. And and then like how do you
• 28:45
  decide on the data sets that you're using?
• 28:47
  How do you decide on the models? Are you doing all of that?
• 28:50
  Or is there any moments that you're going out and you're
• 28:53
  saying you're trying to crowd Source data?
• 28:59
  No matter, there's no crowd sourcing data yet.
• 29:02
  So My Philosophy during that project, Eject is usually there
• 29:07
  wouldn't be any actual data actual model in this type of
• 29:12
  project. Just because it's something for
• 29:15
  the other people to use for multiple purposes, right?
• 29:18
  My choice in the greenhouse project was to choose a very
• 29:23
  simple project. It could be the iris dataset.
• 29:26
  For example, I chose the Palmer Penguins data set, very
• 29:30
  simplistic, very small, very easy to understand, And it's
• 29:35
  there just as a placeholder as I said so you can replace by
• 29:39
  anything that you want. That's it.
• 29:43
  And then I start training models like by just in the data
• 29:47
  training models predicting based on that.
• 29:52
  Right? And then you can replace my
• 29:54
  anything. I'm not taking care about Cloud
• 29:57
  environments, for example, because the person that uses the
• 30:01
  project should connect it to whatever person wants.
• 30:05
  I may make some configuration files to make this easier for
• 30:09
  example. But that's not the purpose,
• 30:13
  right? This is just like a structure
• 30:15
  like at the greenhouses Lansing and you can connect with input
• 30:19
  data and output data. But with time I haven't reached
• 30:24
  that point yet, been reached with time, I want to make it
• 30:28
  easier to make an API for example with that project and I
• 30:33
  intend to have another repository maybe where I will
• 30:38
  use the greenhouse project for a more robust data set and then in
• 30:43
  this project I will maybe connect to some databases make
• 30:48
  it's a bit more complex. Try to use maybe a WS.
• 30:51
  As you know, to demonstrate how to use that projects as an
• 30:55
  example you know. Okay.
• 30:57
  Yeah. That makes sense.
• 30:59
  Now for those that are listening and they want to get started
• 31:04
  with it like any words of wisdom when it comes to live streaming.
• 31:11
  If live streaming. Yeah, so don't be afraid to
• 31:17
  start it. There are many streamers that
• 31:20
  are actually learning from scratch.
• 31:23
  I have seen so many learning python there.
• 31:27
  There are more front end people want, which need to be honest.
• 31:31
  But anyway, the many people are learning and they are showing
• 31:35
  the process of learning online and I guess they are learning
• 31:40
  probably faster. After getting help from other
• 31:42
  people faster, because of that. You may not, you may choose not
• 31:47
  to do that because you don't want to expose yourself like
• 31:50
  this if you know more you can for example do like myself to
• 31:57
  create a project and develop that project life or small
• 32:02
  projects, there's some data scientists there on Twitch that
• 32:06
  they are getting small projects. Like, I want to use that data
• 32:10
  set from cargo and I want to make a good model and I want to
• 32:14
  show everyone how to do this. They go on for a few lives but
• 32:19
  they five livestreams a show from beginning to end and that's
• 32:23
  it. Then the next project.
• 32:25
  So you need to be creative and don't be shy.
• 32:30
  For example why I hate live coding during interviews, you
• 32:35
  know, job interviews. I hate it.
• 32:38
  Like with all my Forest like I really hated and it's some kind
• 32:45
  of Sucks that I'm doing this on Twitch but for me it's about
• 32:51
  being more relaxed there. Try to be more relaxed and
• 32:56
  showing the process and all this while on a job interview there's
• 33:01
  a lot of pressure and you cannot Make a crappy code there because
• 33:06
  you will lose that job is different situation.
• 33:09
  So for people that hate life coding, like I do, it's
• 33:14
  completely different experience. I think and you can make it fun.
• 33:18
  You can play your harmonica. You can make sessions that are
• 33:22
  only about drinking some beer and talking about random stuff
• 33:26
  just to show out with your audience.
• 33:29
  It's very, very flexible. So, So I'm also thinking about
• 33:34
  how just today. I saw a post from one of our
• 33:39
  other guests Adam and he was mentioning how important it is
• 33:44
  for data scientists or machine learning Engineers to write blog
• 33:48
  posts and document their Journey.
• 33:51
  Do you feel like this is another way of doing that?
• 33:55
  For sure. Yeah.
• 33:57
  I also wrote a blog post on how Colonel for example.
• 34:02
  There is one difference that at least myself?
• 34:05
  I'm really perfectionist when I'm writing so it looks it took
• 34:10
  me a long time to write that blog post, it's short, but the
• 34:15
  shortest the harder actually, right.
• 34:17
  You want to polish with a lot? Make nice figures and all this.
• 34:23
  And I feel that for me that process Is choose low.
• 34:26
  You know, I learned a lot but it's too slow and you stay
• 34:31
  thinking about the same subjects for a long time.
• 34:34
  Trying to polish while live streaming is like, very Dynamic
• 34:39
  and very fast. Yeah.
• 34:42
  And was there a point where you felt like because you're doing
• 34:47
  this? It has had an effect on your
• 34:49
  career. Like did you get your most
• 34:52
  recent job? Because they saw that you had a
• 34:54
  twitch or Is it just that you were more comfortable in these
• 34:58
  live coding interviews? How has it affected you and end
• 35:02
  your career? Yeah.
• 35:04
  So I think YouTube had a more direct impact on my interviews.
• 35:08
  My job interviews because I did it first last year when I was
• 35:14
  looking intensely for prayer position and was really good.
• 35:19
  Because, as I said, it's served as a portfolio.
• 35:22
  So when I started doing YouTube, Choking on YouTube, right away.
• 35:28
  All my interviews at some point, there was a discussion about
• 35:31
  this. And the interviewer was always
• 35:33
  happy to see how I was interacting with the community
• 35:38
  how they liked one of the videos, things like that, you
• 35:41
  know, so it was really positive. Never like the decisive thing,
• 35:45
  like you are getting this job because of that was never like
• 35:49
  this. But was always a very positive
• 35:51
  element that I think was influencing the interviews
• 35:55
  there. Where's the lat?
• 35:56
  And for sure, I think is the inference to drop the having I
• 36:00
  got. Were you putting it on your
• 36:02
  resume? Yeah.
• 36:03
  Of course. I put it and so then people
• 36:06
  would go important. Yeah.
• 36:07
  Yeah. Blog posts.
• 36:09
  Anything maybe you don't specify exactly which blog post, but you
• 36:12
  say that, I have this repository where you can see all my texts,
• 36:18
  many times the interviewer won't go there and read or watch our
• 36:21
  videos, but just to go there and take a look.
• 36:25
  And see that there's something robust there.
• 36:28
  They are already very happy. So for anyone that struggling
• 36:32
  with getting a job this time, I think it's very good, very
• 36:37
  positive. And then, on Twitch, I think it
• 36:40
  didn't influence so much because it's more recent, but I think
• 36:44
  the inference is more in my career about what I'm learning
• 36:48
  from that experience, okay? Like, I think I said this
• 36:52
  already but like, exposing yourself.
• 36:55
  Showing your mistakes vocalizing, your thoughts.
• 37:01
  I think all of this make you a better programmer.
• 37:05
  I think the better maybe how to say someone that can explain.
• 37:12
  Well some Concepts, I think there are many positive sides on
• 37:16
  this. And I don't see any.
• 37:22
  Any downside of doing that can only see positive side.
• 37:26
  So I don't talk to me about when you run into problems or, or
• 37:33
  talk to me about one specific time that you've run into a
• 37:36
  problem and it was like a big problem and you got through it
• 37:43
  and this becomes like a job interview.
• 37:44
  Yeah I know I just think about like what if I were doing that
• 37:49
  and then I get to something where I get stuck in my
• 37:51
  potential. I spent an hour trying to just
• 37:54
  solve this small piece and I'm banging my head against the
• 37:57
  wall. Like I think nobody would want
• 37:59
  to actually watch me doing that but I imagine there are people
• 38:03
  and they really enjoy it. So I think one of my first live
• 38:08
  streams, I really was really frustrated because from
• 38:11
  beginning to end, everything went from like I started doing
• 38:14
  something, like cold was raking. I decided to go for another
• 38:19
  path, the other choice also was bad and it broke.
• 38:26
  I mean my way of dealing with this is explaining maybe
• 38:29
  explaining why I'm taking another path and in the end it
• 38:33
  says things didn't work. I always tell people that I will
• 38:36
  work a bit on the side offline, maybe just to not lose so much
• 38:44
  time. And next livestream, my always
• 38:46
  have the solution for that or it happens, sometimes that I turn
• 38:52
  on the camera run only like during the day.
• 38:54
  When I had more time, if I accepted this job that I'm
• 38:59
  working on right now, I will turn on my camera.
• 39:02
  Do like the other streamers like Butta.
• 39:04
  Me You'd say, Focus sessions and bring this work on that problem.
• 39:09
  Anyone can watch, I solving that problem, but it's more like
• 39:13
  aside session in a different time where people can just see
• 39:18
  what I'm doing. Just because I also want that
• 39:21
  like 99% of my coding that project happens life, I prefer
• 39:28
  to do it like that because it the project is about thats about
• 39:33
  learning and in And if that's project is useful, I will be
• 39:38
  happy at such times more about the learning experience.
• 39:42
  And so some other really cool projects that you've seen on
• 39:46
  Twitch. I mean, you mentioned there are
• 39:49
  some end-to-end projects that have been set up and they do
• 39:52
  like five sessions or however many sessions it takes, they set
• 39:55
  it up and then they move on to the next project.
• 39:58
  Is there anything that has been interesting for you as you are
• 40:03
  watching? Other Murs that you you could
• 40:07
  recommend to us, Yeah, so there's this channel
• 40:12
  unfortunately is in part because I actually tell me why, but this
• 40:16
  guy is the one that influenced me to start doing twitch.
• 40:23
  So he has many of these interesting projects.
• 40:26
  And actually, the most interesting one was not a
• 40:29
  project but was like focusing on a specific topic.
• 40:34
  Like it's not that class I think because I was not prepared,
• 40:38
  probably improvised but he wanted to show Concepts about
• 40:42
  SQL, you know, SQL. And I think he went on for two
• 40:49
  weeks or three weeks just about zero because he was saying, I
• 40:53
  think he's right, this is a really important topic for
• 40:57
  especially, for beginners to enter the market and data
• 41:00
  science. People usually avoid learning
• 41:03
  that But it's one of the most important elements that you need
• 41:08
  to learn before trying to get into data science.
• 41:11
  So yeah, I think about that and I think about the people that
• 41:19
  are starting from scratch. I like how you you said, there's
• 41:24
  a lot of people that are learning and they're using this,
• 41:27
  as their way of learning and potentially learning faster
• 41:30
  because they get help from the communities that they Start and
• 41:35
  can you talk to us a little bit just for these last couple of
• 41:39
  minutes, maybe about the community aspect of this, and
• 41:44
  why you feel? That is important.
• 41:47
  Yeah, so it's worth mentioning that like you have the slack
• 41:50
  channel for our machine learning operations Community.
• 41:54
  Usually on Twitch people have a Discord server starts similar to
• 41:58
  his leg. It's very similar to York,
• 42:02
  right? Iarc And I have one if you go to
• 42:07
  my twitch Channel, you'll find there, the link for the server
• 42:11
  to enter them and offline like outside of the this lot time.
• 42:18
  For for the live streams people interact a lot on this chat
• 42:22
  sexually. I follow me, I am part of many
• 42:27
  discard servers and I'm always taking a look there to see if
• 42:31
  there's some something interesting going on.
• 42:34
  Sometimes they even have their other live streams in Discord.
• 42:42
  So there's a whole Community, I think being built around twitch
• 42:48
  and I think it's being done through this card.
• 42:52
  I forgot about exactly where you wanted to get this question.
• 42:57
  I got a little that was perfect. I mean, it was the yeah, the
• 43:00
  role of community within twitch. Yeah.
• 43:03
  And I think people are helping each other a lot like this guy
• 43:08
  from Brazil, until he started, Is translating Brazil for data
• 43:13
  science, as some people followed him and I'm not the only one.
• 43:18
  And I feel that this people are really like, my friends.
• 43:21
  I always talk to Dan and we have some ideas about.
• 43:24
  I even discussed the idea of a start-up with one of this
• 43:30
  friends. Of course, it doesn't go go on.
• 43:33
  But anyway, just like interactions.
• 43:35
  That I know I didn't have before just because we are building
• 43:38
  this like as a community together.
• 43:42
  And my lives are in English. I started doing that in
• 43:45
  Portuguese, only because of that community, and I asked many
• 43:49
  people about the language. If it would hurt being an
• 43:52
  English people that were watching me, most of them said,
• 43:56
  it wouldn't hurt. So, I decided to transition to
• 43:59
  English some sessions, especially about the GitHub, has
• 44:04
  they have been in Portuguese for now but it's very flexible.
• 44:09
  People, don't stick too much with this rule.
• 44:11
  Scent, which is different from YouTube, and it's worth
• 44:14
  mentioning. Also, I didn't mention this, I
• 44:17
  usually have one session per week about git and GitHub, like,
• 44:21
  although my Greenhouse project has some get elements, of
• 44:25
  course, because of versioning, for example, sending bags to
• 44:28
  GitHub to take care of the continues.
• 44:33
  Development, right? Part of the project but there's
• 44:36
  one session focused only on Geeks and like Rhythm reviewing
• 44:42
  some basic concepts, I'm not a specialist on get and I know you
• 44:46
  can do very complex stuff you with get.
• 44:48
  So I'm trying to follow this path to of learning everything
• 44:52
  that I can combine these Amazing.
• 44:57
  I love talking to you about this.
• 44:59
  I'm fascinated by this whole community that is popping up
• 45:04
  there. I find it really important as we
• 45:08
  continue to move on with machine learning operations.
• 45:12
  Because if there are best practices and if there are
• 45:16
  experts that are they're sharing their knowledge, I love seeing
• 45:20
  that and I think it is totally needed in the ml Ops community
• 45:25
  and The Greater Community at large, right?
• 45:29
  So I think I'm. Yeah, go ahead.
• 45:31
  I think like Twitches becoming such a trend.
• 45:36
  That's even companies are following this.
• 45:40
  So, for example, there is three different GitHub channels on
• 45:43
  Twitch Like official twitch channels.
• 45:46
  There's one official GitHub. There's GitHub education, and
• 45:50
  there's GitHub Enterprise. This last one is just for
• 45:53
  seminars, and I think people from different companies.
• 45:56
  Disconnect there from time to time to watch some Seminars.
• 45:59
  The other two are really cool. Like, I have watched some things
• 46:03
  and its really interesting. There's one guy there that is
• 46:06
  like I don't know what this position was like the
• 46:08
  Evangelist, maybe it helps something, and he always
• 46:13
  interviews someone that has a project that's really focused on
• 46:17
  GitHub things. For example, those that use
• 46:20
  GitHub as a Central element and was really interesting to see in
• 46:27
  one live stream. For example, he's trying to help
• 46:29
  the person out with some GitHub, how to say, like, doing trying
• 46:36
  to recover, some stuff from the repository.
• 46:40
  And in the end, he said, okay, let's just delete the repository
• 46:43
  and clone it again. And I was really amazed though,
• 46:46
  I thought I was the only one that did that, but even the
• 46:49
  official guy from GitHub, the same that Because sometimes it's
• 46:53
  so complex that it's the best solution.
• 46:57
  Yeah, exactly. I just turn it off and turn it
• 47:00
  on. Delete it.
• 47:01
  It's fascinating to see this things happening live, you know?
• 47:04
  So you're saying this people, like, they are invincible, but
• 47:08
  you see some soft spots and you can feel better with it
• 47:12
  yourself. That you're not such a crappy
• 47:15
  colder because, like everyone like this mistakes.
• 47:20
  Yeah. Yeah, I find it fascinating.
• 47:22
  Fascinating and I think it is something that as you said,
• 47:26
  we're going to see more and more companies doing it and we're
• 47:28
  going to see more and more people doing it as especially in
• 47:32
  the ml ops world as it becomes more common.
• 47:36
  And there is more knowledge or there's more demand for people
• 47:39
  that need to know about this and since it is so complicated of a
• 47:46
  environment that we play in, I imagine we're going to get to
• 47:50
  see some people that are learning.
• 47:52
  And as you talked about that are starting from the beginning and
• 47:56
  then going up and then others that are really trying to
• 47:58
  decipher this end-to-end and how long that would take or just
• 48:04
  making sure that they show each step of the puzzle.
• 48:08
  So for me, it's super cool to hear about, I hope that everyone
• 48:12
  else listening also enjoyed Felipe.
• 48:15
  This has been excellent. Thank you so much, man.
• 48:17
  Any last words. Plus words.
• 48:21
  Yeah, so please follow me on YouTube on Twitch.
• 48:25
  Please visit the repository is called green apple pie - green
• 48:31
  house rent and you can find it in my profile.
• 48:34
  That's Philippi Pena. But of course, start your own
• 48:39
  channel on YouTube and twitch, twitch is really easy to start
• 48:44
  on YouTube. People want to have the perfect
• 48:46
  setup like the perfect camera and everything.
• 48:49
  On which is much easier and it can start interact.
• 48:52
  If you don't want to expose yourself on Twitch, at least try
• 48:57
  to visit to each, make your accounts connected with Prime
• 49:01
  and go there and see what's happening because I think it's
• 49:04
  really really intense started with the game developers.
• 49:08
  I think you can find many like cool game developers that are
• 49:12
  very Advanced like, real PlayStation games being
• 49:18
  developed live, you know? But about like machine learning
• 49:22
  data science will find very cool stuff happening there.
• 49:25
  And I think it's worth like being part of that Community.
• 49:31
  Has any special streaming software for it?
• 49:34
  It's obvious, it's open source. Okay, it's perfect.
• 49:38
  There's a tweet specific platform, but it's only for
• 49:42
  Windows and Mac. I think you cannot use on Linux,
• 49:46
  and Linux user and obvious. You can use in Windows Mac Linux
• 49:52
  if I'm not mistaken. So it's perfect.
• 49:55
  Nice, excellent Felipe. Thank you.
• 49:59
  Awesome stuff man. And I'll be checking you out on
• 50:02
  Twitch for sure. Everyone else go and watch him
• 50:05
  and we will see you next week for another ml Ops Community
• 50:09
  meet up. Thanks for joining us.
• 50:11
  Thank you. Take it.
• 50:12
  Easy Felipe, take it easy. Bye-bye.
• 49:49
  You can use in Windows Mac Linux if I'm not mistaken.
• 49:53
  So it's perfect. Nice, excellent Felipe.
• 49:58
  Thank you. Awesome stuff man.
• 50:00
  And I'll be checking you out on Twitch for sure.
• 50:03
  Everyone else go and watch him and we will see you next week
• 50:07
  for another ml Ops Community meet up.
• 50:10
  Thanks for joining us. Thank you.
• 50:11
  Take it. Easy Felipe, take it easy.
• 50:13
  Bye-bye.