Latest / Agentic Conversations (formally mlops.community) / Clean Code for Data Scientists // Matt Sharp // # 160

Transcript

Clean Code for Data Scientists // Matt Sharp // # 160 · Agentic Conversations (formally mlops.community)

• 0:01
  I am not sharp. I work at Shopify.
• 0:04
  I'm a day to developer there. I'm very generic title but
• 0:08
  essentially, machine learning engineer and I don't drink
• 0:12
  coffee or tea, so I'm water guy for sure.
• 0:17
  Welcome back to the ml. Ops Community podcast, I am your
• 0:21
  host Demetrios and I am here with one of the best in the
• 0:25
  business. A be.
• 0:26
  How's it going? A be great, there's a lot
• 0:29
  happening. Ain't like the tech world too
• 0:32
  many changes to many fruit Prince way too many
• 0:34
  announcements are, we had a wonderful conversation with
• 0:37
  Matt. Matt is a chemical engineer
• 0:40
  turned. A scientist turned the engineer,
• 0:42
  and right now he's leading, D top at Shopify.
• 0:47
  So, I ate think, I'll let dimitrios take on what were your
• 0:50
  favorite Parts Demetrius in the conversation?
• 0:54
  For those who don't know, Matt also is a bit of a LinkedIn
• 0:57
  influencer because of all of the Relations and post that he has
• 1:02
  up around clean code for data scientists T refers to himself
• 1:07
  as a recovering data scientist. And we got into that in this
• 1:11
  conversation, we also talked a little bit about the platform
• 1:15
  that Shopify created Merlin and the blog post that they wrote
• 1:19
  about it and how they're able to enable if there is a little bit
• 1:23
  of a tongue twister for you, the real-time recommender systems
• 1:28
  and the inbox Education and also, fraud detection, all that
• 1:33
  good stuff that you hear about for real-time use cases.
• 1:37
  They break down exactly how they're doing it.
• 1:39
  And also, I think the biggest takeaway for me on Merlin and
• 1:43
  the whole real time serving platform that they created was
• 1:47
  the ability to quickly iterate and get up and running with a
• 1:51
  Sandbox and then if you prove out some value, you can go and
• 1:56
  run with it. That was my takeaway.
• 1:58
  What about yours? Sofia me.
• 2:00
  I think I loved his advice on. What does clean code main for
• 2:05
  their scientists? I thought he had a hot take
• 2:08
  about those posts that blow up on LinkedIn and Twitter.
• 2:13
  Like, I've been a data scientist or a software engineer for x
• 2:16
  amount of years and I still Google how to do the most basic
• 2:19
  things in Python and his response to that was hilarious.
• 2:23
  It was like if you've been a data scientist for 12 years and
• 2:26
  you're still figuring out the basic building blocks of Vaughn
• 2:30
  and you have to Google that. You're not good at your job.
• 2:33
  So let's just be very candid about that and I thought that
• 2:36
  was a funny hot take that is a little bit contrarian to the
• 2:42
  things that we see and hear on social media.
• 2:45
  So, overall, I think the conversation was a good
• 2:47
  combination of talking about clean coding, coding practices
• 2:51
  for ideas like this and then we move to real-time streaming
• 2:55
  talking about World Landing. Then finally, to Ellen's in
• 2:57
  production in some of his work on Topic in some physical pain.
• 3:01
  And so, overall, there's something for everybody, he is
• 3:05
  coming out with a book to on lme's in production.
• 3:08
  I think that is actually the title of the book and it should
• 3:11
  be out soon. We, if it is out already, we
• 3:15
  will leave a link in the description if it's not out we
• 3:17
  will hopefully leave a link to the Early Bird version of it so
• 3:23
  that you can check it out. And without further Ado, I think
• 3:26
  it's time to get into this conversation if you like and
• 3:30
  Enjoy this podcast. It means the world to us if you
• 3:32
  just share it with one friend, one friend, that's all we ask.
• 3:35
  Because that is the way that we're going to spread the good
• 3:39
  word of ml Ops or as some people are calling it.
• 3:45
  These days L LM Ops or as other people are calling it.
• 3:50
  These days LMS in production, and We've been having all kinds
• 3:57
  of lme's in production conferences happening and I
• 4:01
  think by the time this bad boy comes out, we are going to
• 4:05
  either be gearing up for the LM in production conference, which
• 4:09
  you can find a link to in the description below or we're going
• 4:11
  to just have had it. Which you also can find a link
• 4:14
  to all the replays in the description below.
• 4:15
  So, depending on where you are at in time, when you listen to
• 4:19
  this, you can click on the link below and get everything about
• 4:24
  lme's in production. Your heart's desire.
• 4:27
  That's it for now, let's jump into the conversation with Matt.
• 4:37
  to start it off. Do you have a really big wall in
• 4:42
  front of you that you watch? You have a projector behind you
• 4:45
  and I'm guessing that it's your like home movie theater that you
• 4:48
  project onto a wall. It is, I knew it.
• 4:51
  Yes. So we Have this on the second
• 4:54
  floor across the living room. There's just this empty canvas
• 4:58
  and so yeah, we'll have home videos and stuff up there, so
• 5:02
  it's pretty nice. It looks so cool.
• 5:05
  Yeah. Well done, well done.
• 5:06
  All right, not great to have you here, I have been a fan for a
• 5:11
  while. You've been preaching the good
• 5:13
  word on LinkedIn about clean code for data scientists.
• 5:17
  I think it struck a nerve because occasionally, I'll see
• 5:21
  one of your post pop up with Thousands of likes and Views and
• 5:25
  all of that good stuff. So you are a bit of an
• 5:28
  influencer these days and I think there's a reason for that.
• 5:32
  Lots of people have either been a data scientist or work with
• 5:36
  data scientists that may or may not know the fundamentals.
• 5:41
  And so it's very important to get it, right?
• 5:47
  And you are trying to raise awareness about this.
• 5:53
  What brought you into this space?
• 5:55
  Like why did you decide that you were going to be the one to
• 5:58
  start talking about this? Huh.
• 6:01
  Well I guess I mean to start off like I used to be a data
• 6:04
  scientist myself, like I get it like when you're a data
• 6:08
  scientist, I was just like so many things to learn, you know,
• 6:12
  like statistics heavy just like analyzing data and just being
• 6:17
  able to answer stakeholders questions and answer the
• 6:19
  business values and all of these things that Oftentimes actual
• 6:24
  code portion is often seen as like well I got it working like,
• 6:30
  you know, that's fine. But like at least when I was at
• 6:33
  a descent there was so many errors and waste of time because
• 6:36
  like I had to go and fix some Thug or do other things.
• 6:40
  And like the when I was at there was there was no one around me
• 6:44
  who knew coding, you know? So like it was mostly just me
• 6:47
  and, you know, some other data scientist, you know, trying to
• 6:50
  hack things together. And so, And then like, you know,
• 6:55
  I got to a tech shop and like it's completely different,
• 6:58
  right? So like had a Shopify, for
• 7:03
  example, like when do you think they start expecting like a data
• 7:10
  scientist to start writing clean code?
• 7:13
  You know, and like the answer is like immediately, right?
• 7:17
  Like if you go in and you look at like our documents of saying,
• 7:21
  okay. Hey like Like you're a senior
• 7:23
  engineer. Like these are the requirements
• 7:25
  for your job, like, like, clean code is actually an Insight, you
• 7:29
  know, like being able to write clean code inside of our
• 7:32
  requirements and it starts as an intern, right?
• 7:36
  Like because, you know, like when you come in as an intern,
• 7:41
  they don't necessarily expect you to be able to solve problems
• 7:44
  but they do expect you to write clean code.
• 7:46
  So like it's very much, a fundamental problem that needs
• 7:51
  to be there right at the beginning.
• 7:53
  And I don't think a lot of data scientist like realize that one
• 7:56
  very quick question because I want to clarify this for
• 8:00
  listeners. There's a lot of conversation
• 8:02
  about writing clean could but what does that mean for you?
• 8:05
  Does that mean the use of functions or writing modular
• 8:09
  code? Does that mean fighting unit
• 8:11
  tests and integration tests while writing the code which or
• 8:16
  does it mean structuring all of those things?
• 8:18
  What are your top tips for? When you say, okay, this passes
• 8:22
  my Mark for clean code. So I clean code to me, like the
• 8:30
  definition, how easy it is to understand.
• 8:34
  And so, so the best way for you to gauge it is if you write code
• 8:39
  and you come back, six months from now.
• 8:41
  When you completely forgot about what you were working on, can
• 8:45
  you get caught up within a couple of seconds or minutes?
• 8:50
  Can someone else come in and read your code and understand
• 8:53
  it? Write Clean code is all about
• 8:55
  the communication. An aspect.
• 8:57
  It's very much a technical skill but it's who a lot more of the
• 9:03
  soft skill because the whole point of clean code is making
• 9:08
  sure that you can collaborate with other with other people,
• 9:13
  you know? So like, because you don't work
• 9:17
  in a vacuum, right? You like, you need to be able to
• 9:19
  communicate with analysts and other data scientists about the
• 9:21
  statistics and about how you're working with the code.
• 9:25
  And then you also need to To be able to go in.
• 9:29
  And hopefully, you're going to be working with data engineers
• 9:32
  and no engineers and other people.
• 9:35
  And they need to be able to talk.
• 9:37
  And then, like the code is really the communication piece
• 9:41
  of how this all works down in. So, like, when code is clean,
• 9:44
  it's communicate about. And how do you define clearly
• 9:48
  communicable code? Does that mean it's a Well
• 9:51
  commented or does that mean? You know you are using the right
• 9:55
  variables? Shows that the functioning is
• 9:59
  done in a way that makes sense. Instead of using very random
• 10:03
  variables, to be able to store things.
• 10:06
  How do you define or what are your three top things that you
• 10:09
  would say, are important for that?
• 10:11
  So, naming variables is definitely probably the largest
• 10:15
  aspect of writing clean code just because if the variables
• 10:19
  are well, written and well-defined like you just, you
• 10:25
  know, like Furious Is a model called cars rights just like,
• 10:29
  well, what could be in there and it gets I think a dataset that I
• 10:34
  often point to is I'm so I can my old company, we had a table
• 10:38
  called locations and like locations sounds like a great
• 10:43
  name. But you know like what does it
• 10:47
  mean, right? It can mean it.
• 10:50
  Because like we had locations for like Merchant addresses and
• 10:54
  then we also had online locations, which It's websites
• 10:57
  and just like it got really confusing.
• 11:00
  And the problem lies is that this locations table, like
• 11:03
  people put it up and it was actually zip codes and we never
• 11:07
  used it. And so like, but it was up there
• 11:10
  and I got integrated into a bunch of different things.
• 11:12
  And so like no one wanted to touch it and remove it and clean
• 11:16
  things up. But everyone would always try
• 11:17
  to, you know, merge onto that table and like, get their
• 11:21
  location data. And they thought the result
• 11:23
  Groove. And they would be like, wait,
• 11:25
  why isn't this working? Working, you know, what is my
• 11:27
  data analysis is not working? And it's just like, well, it's
• 11:31
  because you're using the locations table which, you know,
• 11:34
  isn't our locations? You locations aren't location,
• 11:38
  right? Like like you want, you know,
• 11:41
  Russian dress it, right? And so but like so a good name
• 11:44
  is both like I'm very like particular but it's also broad
• 11:49
  enough to include all the use cases of what you're trying to
• 11:51
  do and and kind of understanding that and getting a hang of good
• 11:56
  names. It is obviously really
• 11:57
  important. I think there's a common joke
• 12:00
  where it's just like there are, you know, the two hardest things
• 12:06
  and programming are naming cache invalidation and off by one
• 12:10
  errors. Not like because she's so yeah.
• 12:14
  So naming is definitely hard but I think oftentimes when you talk
• 12:18
  with data scientist so is that like they don't most of them
• 12:22
  don't necessarily come from acute, computer science,
• 12:25
  background, and so forth. A lot of the tips I give often
• 12:29
  involve just like how do you structure a project?
• 12:33
  So that way you can find the files, you need to know like how
• 12:36
  do you You know and because of that comes down to the
• 12:41
  communication, right? Because like if I'm given a
• 12:43
  project, a Ruby on Rails project like Ruby on Rails has its
• 12:47
  awesome structure where like you know you had scaffolding and so
• 12:50
  every project is exactly the same.
• 12:52
  It's really easy to find, okay? Here's where the JavaScript here
• 12:55
  is where the controllers are here's views are, it's really
• 12:59
  easy to go in and dig into them because they're all the same.
• 13:01
  But like, I like data science projects often don't have that
• 13:05
  type of scaffolding or other things like that.
• 13:07
  It's like, it's really The fine even like basic things and like
• 13:11
  where are the test. So like I do they even have
• 13:13
  Tesla. Okay.
• 13:14
  Now like this is a data science project.
• 13:15
  I forgot, you know, it's just like you know.
• 13:18
  And then like they have you know and just like a lot of the
• 13:23
  problems often stem from like you know, data scientists are
• 13:27
  always using jupyter notebooks, right?
• 13:30
  And but like they haven't learned the computer science
• 13:33
  aspects so they don't know how to like manage those and they
• 13:36
  often get a lots of different naming and And so, just like
• 13:39
  figuring out the best. And so a lot of my tips often go
• 13:43
  into that, I would like, okay. Like this is how you get the
• 13:47
  belt most out of your jupyter notebook.
• 13:49
  This is how you can communicate, more and others in subsets,
• 13:52
  dude, I love it. I mean, coming out hard on the
• 13:57
  data science projects and you went there.
• 14:00
  So I'm going to follow your lead and ask about jupyter, notebooks
• 14:05
  and things that you found because I actually We enjoy
• 14:09
  finding out about other people's processes when it comes to
• 14:13
  taking jupyter notebooks and all of the power that they present,
• 14:19
  in their phases of the machine learning lifecycle.
• 14:25
  But then, what do you do? Once you've reached the end of
• 14:29
  their life cycle and how do you get out of them as quickly as
• 14:32
  possible? And what do you recommend to the
• 14:34
  data scientist, then? If ever because some people I
• 14:38
  know, don't like to get out of them.
• 14:45
  You know, I've spent enough time in jupyter notebooks, I'm really
• 14:47
  not a fan, you know, everything from managing their kernels to
• 14:53
  like just you know, like the tricks you can play.
• 14:59
  This was actually I don't know if it's April Fool's but I was
• 15:03
  just like, hey, like this is a prank, I played where, you know,
• 15:06
  you go in, you can run some code and then you delete that cell
• 15:10
  and like no one's ever going to be able to understand why I like
• 15:13
  their jupyter notebook. Worksite.
• 15:15
  And you know doesn't work anymore just because you like,
• 15:18
  redefined, you know, something Imports, or things like that.
• 15:21
  And so generally how I like to use jupyter notebook though, is
• 15:26
  because like they are powerful, like you can iterate really
• 15:30
  quickly, you can go through And you can get things, you know,
• 15:34
  look at your data easier look, you know, that they are really
• 15:38
  powerful but so I like to use them.
• 15:41
  And I think this is really useful is essentially like I
• 15:46
  think people make their cells too small.
• 15:50
  You know, the they're often you know, you often want to make
• 15:53
  them do one thing but like I said, I wasn't a function like
• 15:57
  there's no restriction for to doing one thing.
• 15:59
  But so when I like to do is Like to think about cells, as I was
• 16:04
  like, if I was to put this into like a scripting project and
• 16:08
  inside of just pure python files, like a cell, would be a
• 16:12
  single file, you know, with with its separate Imports and doing
• 16:16
  its own specific thing. Usually like defining a class or
• 16:20
  more defining a group of functions that I can then like
• 16:24
  import into other other cells. And so, what makes this strategy
• 16:29
  powerful is generally like By the time you're, you know, each
• 16:34
  cell is encapsulated and you can be ensure that each cell will be
• 16:38
  able to run on its own. The the next powerful thing
• 16:42
  about doing this is that when it comes time to like communicating
• 16:46
  or maybe moving it into a more foundational, you know, python
• 16:50
  script or Library like it's just a matter of taking each cell and
• 16:54
  copying it into like script file and then you know, just making
• 16:59
  sure you adjust you know the Imports to The other things you
• 17:02
  need. And so, I find that strategy
• 17:07
  really powerful as far as transitioning between a jupyter
• 17:12
  notebook and an actual project as well as just like making sure
• 17:16
  things are clear and things always run think work.
• 17:20
  Well, I think this is a really interesting nerve that you're
• 17:23
  hitting which comes down to this difference between doing just an
• 17:27
  exploratory project where you're just exploring the detailed of
• 17:31
  at the Level trying to understand make sense, or if it
• 17:34
  worse, is when you're writing code for production leveling,
• 17:37
  why you can do it in anything, you can write it in a jupyter
• 17:41
  notebook, or you can choose to write the python code from the
• 17:44
  get-go, but I don't think that distinction is usually mentioned
• 17:50
  clearly which is, you know, there's no problem with using
• 17:53
  any tool. The problem is how you write at
• 17:57
  different stages of using that to so, while right?
• 18:02
  Lighting model code is a living more important.
• 18:04
  When you're writing production level code, do you think it's
• 18:08
  equally important when you're doing exploration as well as
• 18:11
  would you think now the cell code is?
• 18:13
  Okay, we can keep using the d, f da shape and just run that while
• 18:18
  we're doing exploration and maybe once we are writing the
• 18:21
  production level code at that, that doesn't really fly.
• 18:27
  You know I talk about this a lot I think clean code should be
• 18:30
  right at the very beginning, no matter what you're doing because
• 18:34
  exploratory data analysis, like the workflow, I see data analyst
• 18:39
  do all the time is. Ok I have a new exploratory data
• 18:43
  analysis, I create a new jupyter notebook and then the first
• 18:47
  thing they do is they go find their last analysis and they
• 18:50
  start copying the code over, right?
• 18:51
  Like, oh, I don't want to redo this again, right.
• 18:54
  And but like, That first code, they wrote being like, oh, this
• 18:58
  is just a quick script and it's really bad and then they go and
• 19:01
  they copy it over and then they're like trying to, you
• 19:03
  know, run it. And then like oh no I made a
• 19:05
  mistake and now they go and you know, they make fixes and then
• 19:09
  they make edits and like, you know, just like no, like just
• 19:13
  just go and then this is actually something I recommend
• 19:16
  is like, have a jupyter notebook.
• 19:19
  That is like four notes like actual notes, right?
• 19:23
  Like you're gonna go and and you're gonna Copy and paste.
• 19:26
  Okay, this function was really useful copy and paste it over
• 19:29
  give notes about what it does and like clean it up, you know,
• 19:33
  and then like when you go and you are going to copy code,
• 19:37
  you're first going to look at that notebook, that has all your
• 19:39
  clean code, you know, it's going to have all that all your
• 19:42
  reference stuff in it and and that's going to just and then
• 19:45
  you're copying over and things are just going to go a lot
• 19:48
  smoother when it comes to a quick analysis, right?
• 19:51
  Because like no quick analysis is ever like quick.
• 19:55
  Yeah, you know, and so like and so like it's almost more
• 19:59
  important to make sure your code is cleaned when it's when it's
• 20:03
  quick and when it's supposed to be exploratory data analysis
• 20:05
  because like one of the things software Engineers have learned
• 20:10
  is like you're, you know, when you're writing code like you
• 20:13
  yourself are reading in about 10 times more than your writing,
• 20:17
  you know, you're scanning up. You're looking up.
• 20:19
  Oh what did I set this out? Okay.
• 20:21
  What is this variable? And so like you know, every code
• 20:25
  is Read way more than it's ever written.
• 20:27
  And so like that is the heart of why, clean code, it's good and
• 20:30
  like clean code is way more valuable for you, right?
• 20:34
  Like it's future, you whether or not that future you earlier, I
• 20:38
  said, six months but like you will find Value five minutes
• 20:41
  from now. All right, like when you're
• 20:42
  coding and you're just looking back every like, oh well, what
• 20:46
  did I do up here? You know, what did I name that
• 20:48
  variable? You know, and so good.
• 20:50
  Clean code is useful and and this is often like When the
• 20:55
  values of actually working in an IDE versus, you know, just a
• 20:59
  general, jupyter notebook is IDs, come with all of the
• 21:02
  helpful, you know, code completion tips and other things
• 21:05
  like that. And so like I often encourage
• 21:08
  data scientist. Hey, go open up an IV, like,
• 21:11
  it's fine if you use, jupyter notebooks, and he loved that.
• 21:13
  But like, open it up and realize what tools and like, what things
• 21:17
  are there and ready for you out of the box, because it will
• 21:20
  definitely change your life. Yeah, it's so interesting that
• 21:23
  you point that out like the foundation and how just us as
• 21:27
  humans were so in a hurry all the time to get to the next
• 21:32
  thing or try and figure something out that we do these
• 21:35
  quick hacks and then we just start throwing something on top
• 21:38
  of these quick hacks and that foundational level is so flimsy
• 21:43
  that it can end up coming back and bite you in the ass real
• 21:47
  quick. And so I want to ask one more
• 21:52
  thing before we move on to the actually, So one of my most
• 22:00
  popular post on LinkedIn was about streaming data and
• 22:03
  real-time, and that seems to blow up.
• 22:07
  I want to ask you before we jump to the next one.
• 22:10
  What has been your most popular post on LinkedIn?
• 22:13
  This far. I imagine it's around something
• 22:17
  clean code. So, this is interesting, so, I
• 22:22
  think, my most popular post ever was taken down because it was
• 22:27
  too controversial and taken down by who you or LinkedIn dialing
• 22:33
  tone. Oh yeah, about clean cook about
• 22:36
  data scientists, some data science at LinkedIn got picked.
• 22:41
  So it was yes, yes. So it was it was, it was a very
• 22:45
  simple, very, very simple. All it was I was and I'm known
• 22:51
  to be a little controversial sometimes, but all the post
• 22:55
  said, was clean code is greater than clean data.
• 23:00
  And it kind of blew up and like, a lot of people were really
• 23:07
  angry with us. But like my, my, my basic, you
• 23:13
  know, thought process around this is just like clean data
• 23:16
  comes from clean coat, like all data, like ultimately comes from
• 23:21
  code. Like, you know, when you know,
• 23:23
  whether or not, you know, you're taking a video, you know,
• 23:26
  digital camera, like, all of that is, you know, code working
• 23:29
  on under the hood, to get the data and that you're putting
• 23:32
  into a database and data pipelines, and you're doing all
• 23:35
  these things. And so like, I was just this
• 23:37
  point or, you know, like they say like, you know, 80% of the
• 23:41
  data scientist job is like cleaning data and like everyone
• 23:46
  was always saying, hey, data is so so important.
• 23:49
  You have to have to clean it garbage in garbage out.
• 23:53
  And, you know, it's something that data scientists very
• 23:55
  understand very well, because when their data is bad, like it
• 23:58
  just ruins their day. And how that makes analysis way
• 24:01
  harder and makes training models really harder.
• 24:04
  You know, I'm but like they don't want clean code because
• 24:08
  that's work that is on them to do, but it's just like, but if
• 24:11
  they had clean code, their data would be cleaner, right?
• 24:15
  And like, this is something, you know, with things like what,
• 24:19
  Chad sanderson's pushing out with, you know, models and n
• 24:23
  apis and just, you know, having data contracts and things like
• 24:27
  that is just like, yeah. Like if you just had clean code
• 24:30
  and data contracts, and these other simple principles, like
• 24:33
  you're going to have clean data. Like it's just, it's gonna be a
• 24:37
  result of it. And so, anyway, so so yeah, it
• 24:40
  was just a very simple post, you know, clean code is more
• 24:45
  important than clean data and it just blew up and I think some
• 24:48
  people got mad and they took it down because of that.
• 24:50
  Like, but yeah, so that's so wild.
• 24:54
  That's I wonder if they took it down because of the comments
• 24:58
  potentially like being So you can LinkedIn doesn't want that
• 25:01
  to be the experience that people have on LinkedIn or if it was
• 25:05
  just some engineer at LinkedIn was like, fuck this and prints
• 25:09
  they went and went behind the scenes and were like delete
• 25:12
  post. Yeah, I don't, I don't know.
• 25:15
  It was up for a good while and like people even took
• 25:20
  screenshots of it so like I know it existed but like I can't find
• 25:23
  it anymore and I even had like a direct link to it and wow, that
• 25:27
  just returns a 44. So, I've been thinking about
• 25:32
  like re-exploring reposting it just because it's been taken
• 25:34
  down. I the cult like this was way
• 25:37
  back when when I was first, posting about clean code, like
• 25:42
  it would really upset a lot of data scientists and like no one
• 25:46
  thought it was important. But nowadays, like all
• 25:49
  influencers have found their like, oh, if I post some like,
• 25:52
  hey, like here's some good sequel, cleaning tips or like
• 25:55
  here's some good code to like the content does really well.
• 25:59
  Now, I see a lot of em. And so like I'm really glad that
• 26:02
  the community has changed and realize like for, you know, hey
• 26:06
  clean code is actually important for the better but like yeah,
• 26:08
  when I first started talking about clean code like the only
• 26:11
  people who I could satisfy was like you know old hardened you
• 26:16
  know software Engineers. Like yeah this is your finally
• 26:20
  saying what? I post a lot of hate crimes but,
• 26:22
  you know. Yeah.
• 26:24
  I would often have like some head of data science or some VP
• 26:28
  of data science or something. Like that.
• 26:30
  So, it doesn't matter. He just need to get it working
• 26:33
  but I don't see that as often. So, Wow.
• 26:39
  Wow, it's incredible. So, let's take a moment and
• 26:44
  recognize that this episode is brought to you.
• 26:46
  By --zl is soul is cloud is a purpose-built cloud, native
• 26:51
  Vector database that stores indexes and searches for
• 26:54
  billions of in Bedding, vectors to power enterprise-grade
• 26:59
  similarity, search recommendation, systems, llm
• 27:03
  apps and more. It's built on the popular
• 27:06
  open-source. Database Mill vez, which if you
• 27:09
  have not tried out, go check that out too.
• 27:12
  It's incredible but easier than Novus to get started with is Liz
• 27:16
  Cloud. It allows for easy integration
• 27:19
  with open a Ico here. Hugging face, Lang chain Lama
• 27:23
  index and more. It's makes it very simple for
• 27:26
  developers to get started. Building AI applications.
• 27:30
  That scale from the get-go. Try the new free tier adds a
• 27:34
  list.com. Cloud will leave a link.
• 27:37
  In the description. And now let's get back into this
• 27:42
  episode. I think now that people have
• 27:46
  moved from that stage of your people were like, can we
• 27:50
  actually derive value our if you know, building a machine
• 27:53
  learning or like a data science team?
• 27:55
  I think everybody has sort of moved where they have enough
• 27:59
  maturity to know. Yes, you can derive value, but
• 28:02
  you're not going to be able to drive value or if like bad
• 28:06
  practices or bad software. Engineering practices
• 28:09
  specifically speaking because you have to be able to write
• 28:12
  code that passes the level of at least some best practices on
• 28:17
  production that was software engineering.
• 28:20
  Yeah, I think this I guess just one more thing before we move on
• 28:24
  I just like like code is where the rubber meets the road for a
• 28:29
  data scientist and it's important to realize.
• 28:32
  I like a lot of data scientists think their job is reading white
• 28:36
  papers all day and now I'm like are you know like maybe at the
• 28:41
  end result is writing a right white paper like that is the you
• 28:45
  know Glory like that is, their job is to write white papers but
• 28:49
  But ultimately, no is like, it's the right code right there.
• 28:54
  They are programmers. Write like, ultimately, you
• 28:58
  know, like if they wanted manipulate data, if they're want
• 29:00
  to write machine learning models.
• 29:01
  If they want to do any research on data, like it involves
• 29:04
  writing code and that is where the rubber hits the road and
• 29:08
  like that. You know, it's just so important
• 29:10
  to harness their craft and and realize like, hey, yeah, you are
• 29:14
  a programmer, you're a very specialized programmer, that
• 29:17
  does a lot of statistics and does a lot of research.
• 29:19
  But Like your hand out. Like like that is, your craft is
• 29:25
  writing code, it's not reading and writing white papers because
• 29:28
  or whatever else, you may think it is, you know, talking and
• 29:32
  meetings about statistics notes. Like ultimately like you're like
• 29:37
  you the value you bring people as when you write code.
• 29:40
  Yes. You and you will be held
• 29:42
  responsible for that. So, it's cool that you've talked
• 29:45
  about it at school that you continue to preach the good word
• 29:49
  on it and you're doing. Hard work and I'm glad
• 29:51
  somebody's doing it. I want to go into this real-time
• 29:54
  streaming. As I mentioned, one of my best
• 29:56
  performing post on LinkedIn was talking about real time,
• 29:58
  streaming, and you all at Shopify, just put out, a blog
• 30:03
  post that goes really in-depth in to how you are doing
• 30:08
  real-time streaming. And one of the things that I
• 30:11
  think we can kick off the this part of this conversation about
• 30:15
  is Merlin and what exactly Merlin does and then It fits
• 30:20
  into the rest of your stack. Yeah.
• 30:22
  So Merlin is just the machine learning platform at Shopify and
• 30:32
  like really like when you when you look at it, it's it's really
• 30:39
  not much, but it's just a way for data scientists to quickly
• 30:45
  manage a machine learning project.
• 30:48
  Of course, And like I say it's not much but it's really cool.
• 30:53
  And so, like, Merlin the big thing with Merlin is it's built
• 30:56
  on top of re the main thing is is it makes parallel processing
• 31:01
  really, really easily. And so like you.
• 31:05
  So the basic principle is it has a head node which has a bunch of
• 31:09
  worker nodes. And so like, when you start a
• 31:12
  training job, it'll start spinning up worker nodes to
• 31:15
  process that data. And so it just makes working
• 31:19
  with really Really large data sets so much easier and so
• 31:23
  Merlin essentially just provides a bunch of command line tools so
• 31:30
  that way you can quickly spin up a project manager dependencies,
• 31:35
  you know, create your images and you know, has some boilerplate
• 31:39
  code for you to kind of do all this stuff and makes it easy.
• 31:44
  I'm specifically my work stream that I work on inside of Merlin
• 31:49
  is About making sure that we can do online inference.
• 31:54
  So, we, we provide a bunch of helper functions and things like
• 32:00
  that. So that way, you can quickly
• 32:02
  spin up a new rest API and new service with your machine
• 32:07
  learning model, and to be able to have it inside of the Shopify
• 32:11
  ecosystem, and to be able to, you know, get inference out of
• 32:15
  it. And so It's it's just really
• 32:19
  simple. You just end your terminal
• 32:21
  right? Merlin Services create and that
• 32:24
  creates a repo, it creates you know, simple, config files, easy
• 32:29
  enough. So that way like a data
• 32:32
  scientist doesn't have to go in and mess with Terror form or
• 32:36
  other things like that, but like, they can make sure that
• 32:38
  they have everything needed, you know, defined there, you know
• 32:42
  what gpus they need, how much, you know, CPU and memory.
• 32:45
  It takes care of all the auto, Only the one the recent projects
• 32:50
  I worked on was adding GPU auto-scaling to it.
• 32:55
  And so yeah so that's kind of what my team works on.
• 32:59
  We created a Merlin's pipelines library that just you can import
• 33:04
  and then it just has everything you need from getting data from
• 33:10
  our different data stores as well as you know, transforming
• 33:14
  it and getting it into however you need.
• 33:17
  As well as and and making sure that it's all distributed and
• 33:21
  using Ray out of the get go there now on version 2 and
• 33:25
  there's been lots of improvements but kind of working
• 33:28
  with with the team over there and things go.
• 33:31
  And so there's been kind of lots of learnings, like, one of the
• 33:35
  things we found is a lot of times like you know, data
• 33:38
  scientists end up often with just working on the head node
• 33:41
  and not taking advantage of the worker nodes and making sure
• 33:45
  things been out correctly. And so just making sure that we
• 33:49
  can make that easier. So there's one big thing that I
• 33:55
  want to know because as you know, I am a happy Shopify user
• 34:00
  with our funny shirts that we've been selling, which is I
• 34:05
  hallucinate more chat GPT. They've been discontinued
• 34:08
  because we can't just have them available all the time.
• 34:11
  We got to put a little bit of scarcity around it, but me as a
• 34:15
  Shopify. User.
• 34:19
  What are some of the things that like you're working on that?
• 34:22
  I as that end-user am seeing in my interactions with Shopify?
• 34:33
  So there's kind of a lot of cool projects happening under the
• 34:37
  hood. So so like one of the projects
• 34:44
  that the team has been working on a lot is supporting like
• 34:51
  not-safe-for-work filtering because you know Shopify
• 34:55
  supports lots of different businesses.
• 34:58
  And so you know when you go onto an app though, you don't You
• 35:03
  don't necessarily, you know, depending on who you are, right,
• 35:07
  like necessarily getting, you know, kind of an adult shop or
• 35:10
  lingerie shop, or like a violent, you know, like a gun
• 35:13
  shop or something like that. Maybe, you know, if you're just
• 35:17
  looking for Lego toys, you know, and something like that, like,
• 35:20
  air or water gun, you don't necessarily want real guns
• 35:23
  popping up, right? And so, having some sort of a
• 35:26
  filter there is something, the team's been working on improving
• 35:32
  and Ben. And so the.
• 35:35
  So those are some of the things. The other thing like we have
• 35:41
  Projects. Like so one of the cool projects
• 35:44
  I think is like you know recommendations for like new
• 35:50
  shop owners. So like you create a new shop
• 35:55
  like there's a lot that needs to happen, right?
• 35:58
  And like understanding you know the best past forwards over a
• 36:02
  job had a like an can be a confusing process.
• 36:06
  So like we have some models in place that help kind of
• 36:11
  Understand who you are, what type of shop you're trying to
• 36:14
  build and and then like make recommendation as you go and we
• 36:18
  try to follow what you're doing and continue to make the next
• 36:22
  best action recommendation. So that way you can build and
• 36:26
  grow more organically and so that's kind of cool.
• 36:31
  We also have other machine learning models that just, you
• 36:35
  know, are for internal. Use of understanding what we
• 36:39
  call the grid is just Like, you know, understanding our own data
• 36:44
  and understanding Trends. And you know, what, what should
• 36:48
  we do? Like, maybe we should we do
• 36:50
  different promotions, or should we do different Outreach house
• 36:53
  and markers, or cannot sit? Like we do have models
• 36:56
  internally that you know, help help the business due to
• 37:00
  business things. And so like, yeah, there's
• 37:03
  there's lots of different. There's lots of different
• 37:06
  projects, a Shopify, I think it That is like, you know, with
• 37:12
  online car, you know, e-commerce.
• 37:16
  Like there's so much that can be done and there's so much we can
• 37:20
  help our shop owners, become better at and so in all
• 37:24
  likelihood, I think the saying is saying, like, we're more
• 37:27
  likely to die from like, constipation, then we are from
• 37:31
  starving, right? Like we have, you know, trying
• 37:34
  to shove too much in and, and not focusing on the highest
• 37:38
  priorities. Then we are to be like Oh, we
• 37:41
  don't have anything to do, right?
• 37:42
  Like know, we have so many ideas floating around.
• 37:45
  There's so many cool projects that people come up with and so
• 37:50
  yeah, always trying to figure out what the highest priority
• 37:53
  wants are and what can be the most valuable, are always really
• 37:57
  useful. So yeah, our Shopify has
• 38:01
  invested a lot recently and to like open a.i. and we have a
• 38:08
  partnership with what G PT. And so like we Integrated it
• 38:12
  into a lot of different things that you might be able to see.
• 38:14
  So when you're creating your store and you want to start
• 38:18
  adding like products to it like there's a whole workflow process
• 38:25
  and inside of that you can actually add GPT
• 38:29
  recommendations. Right?
• 38:30
  So like it's like, hey I want to add sunglasses and if you add
• 38:35
  this to your workflow, it'll actually come up with a cool
• 38:37
  description of like what those sunglasses are and like, Kind of
• 38:41
  be a little bit scelzi or like, hey, like you're the coolest
• 38:45
  with these Shades and things like that.
• 38:46
  It helps kind of automate a lot of those different things, as
• 38:49
  well as, you know, like a, you can use it like, maybe
• 38:53
  auto-generate tags. So saying, like what, you know,
• 38:57
  who this is for, and things like that.
• 38:59
  And so, so yeah, lots of different.
• 39:03
  Lots of different projects happening inside Shopify to make
• 39:06
  your job easier. Yeah.
• 39:09
  Well I know we're going to get into a lot of that.
• 39:12
  Hopefully, I can convince somebody to Shopify to come and
• 39:16
  talk at the next lme's in production conference because
• 39:19
  that would be awesome. Maybe you maybe somebody else, I
• 39:21
  don't know. But I want to take these last
• 39:25
  couple minutes to talk about the book that you're writing.
• 39:27
  And what book are you writing first of all, and what made you
• 39:31
  want to write it? Half.
• 39:35
  So, we're writing a book lme's in production, which happened to
• 39:40
  be the same title as your last conference.
• 39:43
  So we fell in a great minds think alike.
• 39:47
  Yeah, exactly. Yeah.
• 39:48
  So this is something that's kind of been in the works for a while
• 39:51
  now in my co-author crisper. So he's kind of a legend in the
• 39:59
  NLP Community for say. He works at master Card.
• 40:04
  And so it really is a nice pairing because he comes with a
• 40:08
  lot of that natural language processing background.
• 40:13
  You know, he's worked as a translator, you know, several
• 40:16
  languages went in and became a data scientist and kind of
• 40:21
  learned a lot of that. And I kind of come from the
• 40:24
  other end of hey, like like one of the things I've always been
• 40:28
  focusing on in my career is like, how do I get a model into
• 40:32
  production? How do I make it easy for people
• 40:36
  to use? And actually usable, like, I
• 40:39
  like one of the things I hate the most is like when someone
• 40:42
  creates a cool model just sits and like their project workspace
• 40:47
  on their laptop or something like that.
• 40:48
  It's like no, I want like let's get this out to the world and
• 40:53
  and so yeah so we kind of come in from from both ends.
• 40:57
  And so yeah this this book islands and production it really
• 41:01
  focuses on How to put and work with and maintain these large
• 41:08
  language models and side of production.
• 41:11
  I think we because what kind of inspired this, I was like we saw
• 41:16
  a lot of people talking about writing LM books but a lot of it
• 41:20
  is more focused on the data. Scientists talking about the
• 41:24
  theory, it's really going into like Brazil, you train one of
• 41:28
  these massive models and you know, how do you deal with all
• 41:31
  those things? So we really wanted to take it
• 41:34
  the other way and so like our book is really more geared
• 41:37
  towards the end. They'll ease the software
• 41:39
  Engineers, you know the people who also want to work with these
• 41:44
  models but you know like kind of more get them into a more useful
• 41:49
  State and so so yeah we'll we're talking everything from you
• 41:55
  know. So like there are things we're
• 41:57
  going to be talking about in the book that you know like okay you
• 42:01
  know like what data sets. Do you need to train an llm?
• 42:07
  But like a data scientist is going to care about that
• 42:11
  question differently from like a machine-learning engineer,
• 42:15
  right? Like a machine learning engine
• 42:16
  resume or data, engineer cares about that in the aspect of
• 42:19
  like, how do I go and get that data, right?
• 42:21
  Like how do I build a data piping around it?
• 42:24
  Like how do I clean it or web scrape it or you know yeah
• 42:28
  things versus a data scientist is going to worry more about
• 42:31
  like, okay, like How do I clean it?
• 42:33
  How do I train it? You know, like and so we often
• 42:38
  have a lot of the same questions but our reasoning and purpose
• 42:40
  for for them are are different. And so, yeah.
• 42:44
  So this book is I think what's going to be really cool as.
• 42:50
  So, the end of the book we're going to have three different
• 42:53
  chapters on allow you to go and put an llm into production.
• 43:00
  So we're probably One chapter will be just like, hey working,
• 43:07
  you're gonna set up a cluster in the cloud you're gonna go in and
• 43:12
  you're going to deploy a model into the cloud.
• 43:14
  You're going to then you know create maybe like a phone app or
• 43:17
  something and you're going to be able to go interact with it and
• 43:21
  and and use it maybe you set up some sort of chatbot you know
• 43:25
  the other project were working on is, you know, like code
• 43:30
  completion like getting your own.
• 43:32
  Get up, copilot, something like that.
• 43:34
  Or you can go in, you can just get something, maybe set up on
• 43:37
  your laptop or again. You had it from the cloud
• 43:41
  before. So like, can you deploy an llm
• 43:45
  that you find tuned for that and kind of just use it on your own
• 43:49
  data, you know? So that way you can keep all
• 43:51
  your code, you know, prevent from pushing your code up to
• 43:55
  open a I so that way you have more control and making sure
• 43:58
  it's clean, and then the last one, I think is really going to
• 44:02
  blow people's mind. So like we've gotten, and
• 44:06
  there's definitely a smaller part of Creator.
• 44:09
  Two sets of few, one Cohen to read and things like that.
• 44:12
  But like we've gotten Ellen's working on a Raspberry Pi and we
• 44:16
  want to show you how you can do that as well.
• 44:18
  How you can get Ln on to the edge.
• 44:20
  And I think that's really cool because when we first started
• 44:22
  talking about this idea of her just like yeah lme's on the
• 44:25
  edge. Ha ha like like it's like it's
• 44:28
  not, you know, it's not even possible.
• 44:30
  Well, you know, several months ago and just like know you like
• 44:33
  he just have to have like tons of massive.
• 44:36
  Yeah. You things like that but like
• 44:38
  you know like with compression as well as Lama and other things
• 44:41
  like that, you can get a small, hello Lana, you know, 6 million
• 44:45
  parameters, some ruling parameters down onto a Raspberry
• 44:49
  Pi and get it running. And it's it may be really slow,
• 44:53
  but like we hope will be able to speed that up with like TPU
• 44:57
  stick line with coral or something like that.
• 45:00
  And I'm sorry. Yeah, we're pretty excited and
• 45:03
  about this book. So currently we can't wait I
• 45:08
  have a little bit information which is you've written the
• 45:10
  initials for chapter 2 and chapter 3 Alton three rain does
• 45:14
  the Early Access cover? So I think the Early Access
• 45:22
  comes out after you've written a third of your book and since
• 45:26
  we're going with Manning they have eat is what it's called
• 45:29
  MEAP, which gives people Early Access and so.
• 45:34
  So yeah. So we're supposed to be done
• 45:36
  with our side made 20 seconds so hopefully the the first Meep
• 45:44
  will be soon after that, by the time this podcast comes out
• 45:48
  approximately. See, Ricky around that time.
• 45:50
  Oh, okay, yeah. So yeah, if you're watching this
• 45:54
  populist go check out my book and we've hopefully it's out
• 45:57
  really soon. Yes, will leave a link?
• 46:00
  So do this is awesome. I look forward to reading it.
• 46:03
  Thank you so much for your time. This has been great.
• 46:06
  Awesome. Thank you.
• 45:48
  See, Ricky around that time. Oh, okay, yeah.
• 45:51
  So yeah, if you're watching this populist go check out my book
• 45:55
  and we've hopefully it's out really soon.
• 45:58
  Yes, will leave a link? So do this is awesome.
• 46:02
  I look forward to reading it. Thank you so much for your time.
• 46:05
  This has been great. Awesome.
• 46:06
  Thank you.