Latest / Agentic Conversations (formally mlops.community) / More than a Cache: Turning Redis into a Composable, ML Data Platform // Samuel Partee // Coffee Sessions #111

Transcript

More than a Cache: Turning Redis into a Composable, ML Data Platform // Samuel Partee // Coffee Sessions #111 · Agentic Conversations (formally mlops.community)

• 0:00
  Hey, I'm Sam and I love my coffee with a bunch of Splenda
• 0:06
  and black as night. It's a tree Sam.
• 0:11
  Par tea I don't even realise. His name is like party party
• 0:16
  that I oh yeah that's his website.
• 0:18
  Yeah do this guy wasn't party let's be honest.
• 0:20
  This was fun. I had a great time talking to
• 0:23
  him. You have some takeaways we had
• 0:26
  some good laughs what were your takeaways and then I'll talk
• 0:28
  about some laughs Yeah, Sam really awesome.
• 0:31
  Guy super, super nerdy. Super cool.
• 0:35
  I mean really just great to chat with him.
• 0:39
  He has a really extensive background high performance
• 0:41
  Computing and just all the latest greatest scalable
• 0:45
  training and friends. All these things, which I think
• 0:48
  makes them a very good fit for redis and the stuff that they're
• 0:52
  doing. I mean he's really leading a lot
• 0:53
  of their, their efforts. Pushing forward.
• 0:57
  The reddest platform to be more than just the web serving cash
• 1:01
  that we've known it as up to now.
• 1:04
  And one of my main takeaways was that it seems like a very
• 1:08
  natural progression for the platform actually II.
• 1:10
  See how they're evolving to be this way?
• 1:14
  I focus area native serving platform that does Vector
• 1:18
  similarity that does feature stored provides.
• 1:22
  Those kinds of functionalities. I think he's doing a good job of
• 1:26
  Thing and expanding those functionalities and making that
• 1:30
  a Natural Evolution for the platform as a whole.
• 1:32
  Yeah, I loved how he talked about when you asked him, what,
• 1:36
  or how did you decide to go in this direction?
• 1:38
  And it's like, well, the community was asking for it or
• 1:41
  they were already doing it without us, even thinking about
• 1:45
  it. And then we realized, oh yeah,
• 1:47
  maybe there's some overlap that we can go into and they started
• 1:50
  exploring it and then they got into it that way and red is for
• 1:54
  me is like incredible. We'll open source project.
• 1:57
  I love what they're doing. This episode is sponsored by
• 2:00
  redis. We go, we do a bit of a deep
• 2:02
  dive on. What redis is up to in the
• 2:05
  beginning of the episode. So if you are interested in
• 2:08
  redis, you're going to love it. If you're not interested skip to
• 2:10
  like halfway and you can just hear about Sam parties
• 2:14
  background and all this distributed computer stuff and
• 2:18
  distributed computing stuff. And then yeah we had some
• 2:23
  laughs. I mean it was, it was a great
• 2:25
  time with Sam. I really enjoyed it.
• 2:28
  I love the Rebrand that they're doing or like just kind of
• 2:32
  bringing this to the ml Community.
• 2:35
  I think a lot of people out there also are used to using
• 2:38
  redis and so it's not that big of a change and they're enjoying
• 2:42
  this happening to. So that's it man.
• 2:46
  Updates on our side from the community side.
• 2:49
  As I mentioned, a few episodes ago and I'm going to keep
• 2:53
  mentioning it, we're starting to do more in person events.
• 2:56
  Mihael is doing stuff in San Francisco.
• 2:59
  If anyone is in San Francisco and you want to meet and a
• 3:02
  coffee or beers or whatever. He's organizing.
• 3:06
  You can talk with him about that.
• 3:08
  And we've also got A lot of other places in this is, I'm so
• 3:13
  stoked. Dude, I don't know if you know
• 3:15
  how many places we've got like right now.
• 3:16
  There's people in Melbourne, Australia in Sydney, Australia
• 3:20
  people in Tel, Aviv people in Mexico, City are Cantina,
• 3:25
  Toronto, Canada Montreal, and then Europe all over the place
• 3:30
  in Europe. So I'm just using this as an
• 3:32
  excuse to go and travel and meet people in the ml Ops community
• 3:36
  in person. Hopefully this Melbourne in
• 3:39
  Australia, thing comes through because I've never been to Oz
• 3:42
  and I would love to go and you should come with me.
• 3:46
  I think it's closer to you than it is to me.
• 3:50
  Yeah it might be is it? I can never tell because it goes
• 3:54
  like around the globe where you go the other one was.
• 3:57
  Yeah. Right.
• 3:57
  Okay. Yeah now we're worldwide
• 4:00
  worldwide. Also I saw the pictures from was
• 4:04
  it Berlin? You guys had quite a showing.
• 4:06
  I mean, it means that stuff look like like a village honestly.
• 4:11
  Yeah. An SS supposed to be the know,
• 4:13
  what was what? We can't let Berlin beat us for.
• 4:18
  Let's just cool, dude. Berlin is very cool, so if
• 4:20
  you're in Berlin? Yeah, I mean, or if you want to
• 4:24
  organize something in your city, get a hold of us.
• 4:28
  If you there's probably something that's happening in
• 4:30
  your city, to be honest, we're all over the place we're making
• 4:35
  stuff happen. So let's do it.
• 4:37
  Let's do it. It's just so much fun to
• 4:39
  actually meet in person. Again after being inside for two
• 4:44
  years and now I forgot what it was like to have to hold eye
• 4:49
  contact with people and like it's scary.
• 4:52
  It's a very scary thing. Yeah I lost a lot of social
• 4:55
  skills and I'm relearning them as we go.
• 5:00
  So anyway, this is a long intro, let's get into the talk.
• 5:03
  People are like, come on. I just want to hear about redis,
• 5:05
  guys. Can you stop talking about this?
• 5:07
  How much how bad you are? Is social these days?
• 5:10
  And so we're going to get into it right now.
• 5:12
  Let's go, this is a great place to start which is how we just
• 5:20
  let into the conversation and that is redis is doing a lot
• 5:24
  more than it says on the tin. What's going on Sam?
• 5:28
  Bye. When did this start?
• 5:30
  And tell us a little bit about it?
• 5:33
  Yeah, it's a great question. I think the what we've been
• 5:36
  doing lately is really growing awareness to the fact that
• 5:39
  there's more to do These days with the redis than the
• 5:42
  traditional cash in that, you probably know red is for what
• 5:46
  red is has been traditionally used for is speeding up,
• 5:49
  websites beating up retrieval from your durable.
• 5:52
  Database people put in front of their durable database, they get
• 5:55
  faster retrieval times, because 20% of that data is cash and so
• 5:59
  you get that data like bastard. But these days, it's growing.
• 6:03
  So that's kind of the first layer of, you know, this German
• 6:07
  chocolate cake, if you will. The first layer is that open
• 6:10
  source. It's Community, you know, it is
• 6:12
  that really strong backbone where we have grown a use case
• 6:16
  for caching. It's been used in open source,
• 6:19
  Enterprise Etc, as that caching layer.
• 6:23
  And now the second layer that we're layering on top, it's
• 6:27
  really the module ecosystem. So if you think of redis Jason
• 6:31
  which turns it into a document database, write a search, which
• 6:34
  turns it into full text search capabilities.
• 6:37
  And now, with the addition of vector search, That's really the
• 6:41
  second layer, it's a composable database.
• 6:43
  You're able to put modules in that change, the functionality
• 6:47
  of redis and that's a really interesting part of it.
• 6:49
  And then the Third Kind of layer that is also really interesting
• 6:52
  is the fact that it's positioned differently now.
• 6:56
  So you see these platforms like feast and like tecton and
• 6:59
  feature form that are using it as an online feature store.
• 7:03
  And, you know, that's it's really interesting.
• 7:05
  It's changing the positioning of redis and where it's being used
• 7:10
  and And it's really because of what I like to say, is what's on
• 7:13
  paper? It's still the same cache,
• 7:16
  database. You're still getting the same
• 7:18
  retrieval times. It's just being used in a
• 7:21
  different use case because of that layer to that, I was
• 7:24
  talking about the ability to customize the capability and the
• 7:28
  API within the database. And we found that people really
• 7:31
  like that. To the point where we ended up
• 7:33
  releasing Reddit stack, which is essentially just the most
• 7:36
  popular modules that we have bundled into one container.
• 7:41
  With your face, open source redis.
• 7:43
  And people really like that experience because you can use
• 7:47
  it for all types of things. And it's one click to deploy.
• 7:50
  We're getting absolutely crushed with downloads on Docker,
• 7:54
  millions of downloads mostly because it's just easy to get an
• 7:58
  app started with this kind of database.
• 8:00
  You can really tune it and do any kind of activity you out
• 8:03
  with it. So he, there's been some really
• 8:05
  strong feedback from developers on that.
• 8:08
  Anyway, it's super impressive. If be perfectly honest I my
• 8:14
  background in redis is also more in the pure back.
• 8:16
  End back end standpoint, right? We're using it.
• 8:19
  Exactly. As you described as web apps
• 8:21
  beating up just really is like ash layer it.
• 8:25
  So when I heard about redis a, I like redis labs this this new AI
• 8:29
  focused initiative. I was a little bit confused to
• 8:32
  be perfectly honest. It didn't make it mediate sense
• 8:34
  to me. What is the vision here for?
• 8:38
  You know, you mentioned all these these new initiatives,
• 8:40
  right? Jace on the time series.
• 8:42
  The, the kind of feature stores, all these search capability,
• 8:45
  Vector search, Etc. What is the vision here?
• 8:48
  Like, what is the team? What is the team's Charter,
• 8:51
  right? Why is this like a natural next
• 8:53
  step in evolution? If you consider an evolution or
• 8:55
  maybe you consider it like a Justin orthogonal or maybe
• 8:58
  auxiliary. All right you know parallel
• 9:00
  Branch or is this like the next step for redis as a platform
• 9:04
  entirely? You talk about that?
• 9:07
  Yeah I think that's a great point so it's really A about
• 9:11
  that, German chocolate cake going back to that analogy, you
• 9:13
  know, layering on gay but I love that analogy.
• 9:16
  Yeah. It's it's giving people more
• 9:21
  opportunities to use it in different ways.
• 9:24
  So this is just another way that we think, you know, it can be
• 9:27
  used and this is really just because we've done research in,
• 9:31
  how are people using it? If you're looking at a people
• 9:33
  like doordash with their real time system doing route
• 9:38
  optimization, or recommendations, or Looking at
• 9:41
  over with Michelangelo and what they've done.
• 9:44
  A lot of these kind of BYO are bringing your own redis
• 9:48
  implementations. They are using it for a feature
• 9:52
  store into. That's when we noticed that's
• 9:54
  that's where the vision kind of started is.
• 9:56
  There's all of these companies, these large massive Enterprises
• 10:00
  that were asking us. Hey, can we use redis Enterprise
• 10:03
  for this? Or can we do this with redis and
• 10:07
  they needed expertise and saying, how can we get this
• 10:09
  done? And going back to because of
• 10:13
  what's on paper is just really you know single-digit
• 10:17
  millisecond latency these that allow you to retrieve features
• 10:21
  faster than other data stores. And so when we notice this and
• 10:25
  we saw this in the ecosystem with places like I mentioned
• 10:29
  AT&T or comcast ER you know these big Behemoth companies
• 10:32
  looking for ways to optimize their machine learning
• 10:36
  pipelines. We said we should probably build
• 10:39
  more features around We should probably explore this markets
• 10:44
  more and I think that's really where a lot of that initiative
• 10:46
  started is just supporting those people.
• 10:49
  Just like, we always have in the open source, you know, really
• 10:51
  supporting the people that are using the database and we wanted
• 10:54
  to grow into supporting. There's a IML developers that we
• 10:58
  saw using it for those use cases and especially those companies
• 11:02
  as well because there's obviously a market there.
• 11:06
  Is it easy to change the narrative around going from
• 11:10
  okay? Yeah yeah that's why.
• 11:12
  We're what does that shift look like?
• 11:28
  I mean jokes aside. I mean maybe maybe this isn't a
• 11:31
  joke with his reality aside. What is how does that narrative
• 11:35
  changed? And then going from from the,
• 11:37
  the, the cash for all all servers and web endpoints.
• 11:44
  And what not to? Now, this new and improved AI,
• 11:49
  you know, low-latency serving platform.
• 11:53
  It is a really great question and it's not easy.
• 11:56
  It is something that, you know, it's easy for when you talk to
• 12:01
  the door dashes or the Ubers because they already have an
• 12:04
  engineering team, that's Ortiz, deep that exactly.
• 12:08
  But not everybody has that. And so one thing that I've been
• 12:12
  focusing on, since I came aboard was making really easy to
• 12:16
  understand applications with it. So one of one example is this,
• 12:20
  this Vector Search application. It's a new thing for redis.
• 12:23
  It's basically the ability to take any kind of unstructured
• 12:25
  data. Use a Transformer whatnot, you
• 12:28
  know, some type of deep learning model created and bedding store
• 12:32
  that within Reavis and then do you know, even Hierarchical
• 12:36
  small navigatable worlds or something like that?
• 12:39
  Uh, nshn swe. Yeah, indexing methods, and also
• 12:44
  selection. And then of course, search
• 12:46
  methods to be able to say, find me the most similar embedding to
• 12:50
  this image and wedding or this text in wedding.
• 12:52
  And people immediately see with this demo, which you can check
• 12:58
  out on GitHub. I open sourced it.
• 13:01
  You can see immediately what it does is find me images that look
• 13:05
  like this. This t-shirt, find me a text
• 13:08
  that resembles, this text and those demos.
• 13:10
  What they do is they bring that value to something that even a
• 13:14
  consumer that's non-technical, can understand.
• 13:17
  And so being able to go through on your phone and say, you know,
• 13:20
  view similar by image or use some other by text that really
• 13:25
  brings it to life and you start going.
• 13:27
  Oh, and then when you make a full stack application like
• 13:31
  that, you know everything from react typescript front end all
• 13:34
  the way back down to earth. API back-end run a single page
• 13:38
  application, in a single Docker container people the engineers
• 13:42
  that see it start to recognize the value of the fact that you
• 13:45
  can bring a service like this up.
• 13:47
  With two Docker containers on ec2 instance, that has four be
• 13:51
  CPUs and I'm running that right now.
• 13:54
  It's, you know, it's survived an entire database conference on
• 13:58
  that tiny little ec2 server. And we had like 180 users on it
• 14:02
  at one point, which is a whole lot and it's not a huge data
• 14:05
  set. But the point They're being is
• 14:07
  it's something that you can really easily and quickly create
• 14:10
  value from and showing those really easy to deploy easy, to
• 14:14
  create applications with that, like consumer-facing realization
• 14:19
  value is the best way to bring that value of light.
• 14:22
  It kind of change that narrative I do think that's really
• 14:27
  powerful the and I saw some of the stuff they do open source
• 14:30
  and I think that's a really great way to evangelize.
• 14:34
  New use cases for technology. One thing though that I am
• 14:39
  fascinated by and I've always found a little bit tricky, maybe
• 14:42
  in the case of tools that are more like core pieces of
• 14:46
  infrastructure that are being used to demonstrate some
• 14:49
  functionality. And so, here's what I mean, when
• 14:51
  you talk about redis, like this is, this is like a, this is like
• 14:54
  an underlying Primitive like a core thing, that enables
• 14:57
  something else, right? This isn't.
• 14:59
  And when you build it even an application Focus, like demo
• 15:03
  around redis as a core building block, that's not the same thing
• 15:06
  as saying, hey I'm using tensorflow and I'm building a
• 15:10
  model, you know what I mean? It's there's like a much the the
• 15:15
  the translation from the tool and what it enables is much
• 15:18
  easier going from tensorflow to mnist model versus redis as a
• 15:23
  database to. Hey We enable something that you
• 15:26
  couldn't do before, right? So it feels like they're just
• 15:28
  more Hops and a consumers mind they have to make to understand.
• 15:33
  Oh this is possible because of rattus now because of all this
• 15:36
  like other stuff that, you know, like residues like one line,
• 15:39
  maybe an import statement somewhere and like, one hole
• 15:41
  somewhere in, you know what I mean?
• 15:42
  So how do you, how do you make that value prop more clear when
• 15:47
  you do these kinds of demos? Yeah, and that's exactly exactly
• 15:52
  what I've been toying with thinking about is really, you
• 15:55
  know, We go to these conferences when we talk to developers and
• 15:58
  we say things like this will enable you to do real time
• 16:02
  machine learning, this will have the sub s Layton sees that
• 16:06
  enable, you know, as you're typing in a search bar for
• 16:10
  machine learning recommendations to appear and to Engineers that
• 16:15
  understand how hard and complex doing activities like that or
• 16:19
  making a few guys. Like that are makes a lot of
• 16:21
  sense but to someone who, you know, Maybe as the purchasing
• 16:27
  power of p.m., who isn't maybe as technical, there is a layer
• 16:31
  that actually is really hard to understand sometimes that, you
• 16:34
  know, it is, it isn't the typescript react front end.
• 16:38
  You know, you're not seeing the spinning wheels and the button
• 16:41
  clicks, you know, it's it's about the speed of retrieval,
• 16:45
  which isn't something that you necessarily see you're not
• 16:47
  pulling up a flame graph in Chrome and saying see how much
• 16:51
  shorter this is then you know, X database because that's not
• 16:55
  Something that is going to bring a p.m. to the table.
• 16:58
  That is not something that is going to say.
• 17:00
  Hey guys, you know, look how much cooler or look how much
• 17:03
  faster this database is than another, what it's going to do
• 17:06
  is it's going to say, I have some really deep technical
• 17:11
  expertise in this area and kind of ostracised your solution for
• 17:15
  mother's because it has this one small difference.
• 17:19
  So when you make these applications it's it's about
• 17:22
  focusing on the pieces. That wouldn't ordinarily be.
• 17:25
  There if that application or that database wasn't present.
• 17:29
  So when we say things, like online future store and try to
• 17:31
  teach the difference between offline and online, talking
• 17:34
  about Ms retrievals, but putting them in the context of an
• 17:38
  application that someone can see, you know, when you know
• 17:41
  talking about that Vector search demo when you say by image and
• 17:45
  15 images appear almost instantaneously and you say,
• 17:49
  click it 100 more times and then retrieval speed will always be
• 17:52
  the same and then you show them an That is pointing directly
• 17:56
  from postgres and is trying to do the same thing and then they
• 18:01
  click it a hundred times and it appears twice there, you know,
• 18:04
  it has serious latency issues. That is like how you can make
• 18:09
  those discrepancies, really apparent people.
• 18:11
  But once again, that requires a lot of engineering effort just
• 18:14
  to put those demos together, so I'm still thinking of better
• 18:17
  ways to even show that value. So, you know, it's something
• 18:20
  we're absolutely working on. Well, Gratis is also in my eyes
• 18:24
  at least and I think The eyes of many the quintessential like
• 18:28
  Bottoms Up motion and maybe you're trying to change that
• 18:32
  too. But I feel like it's very easy
• 18:34
  to go and get your hands dirty with redis and figure out like
• 18:40
  is this right for my use case just because of the way that
• 18:44
  it's been positioned for ages and the way that it started as
• 18:48
  this very organic open source project and then like what
• 18:52
  you're talking about there is so true when it comes to the The
• 18:55
  engineers Engineers are going to understand the value of this,
• 18:58
  but then, as you start going higher up, it's Lost in
• 19:03
  Translation a little bit more. And so I can see how, like, you
• 19:09
  almost have to empower the engineers to be able to make the
• 19:13
  case for to their manager, or whoever it is that is you bought
• 19:19
  has that buying power. So, exactly.
• 19:23
  Yeah. And and this is a I think this
• 19:26
  is kind of a problem with a lot of the ml up stools, like, to be
• 19:29
  honest, it's not just redis, right?
• 19:31
  Like I've seen it with a few different tools.
• 19:34
  It's very easy for engineers to understand the value of it.
• 19:38
  But then, or a data scientist even.
• 19:41
  But then going up the food chain.
• 19:43
  It's much harder for them to like, explain like, translate
• 19:47
  from Geek and just nerd out on it to like business value.
• 19:52
  And so, how do you do that? And that's kind of the role.
• 19:55
  Owl. I feel like of the ml Ops where
• 19:58
  the ml tooling company, they have to have that Firepower and
• 20:02
  that like propaganda or promo or paraphernalia, to be able to
• 20:06
  give to, and empower the engineers.
• 20:09
  And so, but just like, kind of diving a little bit deeper into
• 20:14
  the, like, I really want to talk more about this Vector database
• 20:20
  and what is going on there. I know there's a lot of people
• 20:24
  that are pretty. Stoked about it, especially when
• 20:27
  I was mentioning it back in the day before.
• 20:29
  I think any official news came out because I heard beforehand
• 20:35
  because I got I got a connect it, read it.
• 20:38
  Yeah. I'm just so cool like that.
• 20:42
  The we were joking before about Sam and how he had to call his
• 20:49
  property manager and tell him that he was on a podcast.
• 20:53
  And a podcast that is Very popular for five people in the
• 20:57
  world. Oh yeah.
• 21:00
  Anyway, my grandpa told me his mother to of the exactly.
• 21:11
  So so the the vector database like just give us a background
• 21:16
  scoop on that because shout out to Jose Navarro who was very
• 21:19
  stoked on this when I told him about it.
• 21:21
  That's what I wanted to say. Like what's going on there?
• 21:25
  Is it for the people who haven't heard about it or know what
• 21:31
  you're trying to do there? Yeah, absolutely.
• 21:33
  So, first great question. So the, there's a lot of things
• 21:39
  out there. A lot of products out there that
• 21:41
  are pitching themselves as a vector database solution.
• 21:44
  So, if you're not familiar with a vector databases, essentially,
• 21:47
  it's the ability to store embeddings embedding this being,
• 21:51
  you know, essentially vectors list of numbers created by
• 21:54
  Machine learning models or Deep learning models that you can use
• 21:58
  to do similarity comparisons or distance, metrics to say how
• 22:02
  most similar is one vector to another.
• 22:04
  The vector databases allow you to do this intelligently through
• 22:08
  indexing and storage capabilities.
• 22:10
  That haven't necessarily been the focus of databases before.
• 22:16
  So one way that redis is adapting to this is through its
• 22:21
  redis search module and it's kind of a natural place for it
• 22:24
  since it is just a search. Don't want it enables you to do
• 22:27
  is index the keys. It's a key value store.
• 22:30
  So index, the keys within a redis to be able to search over
• 22:33
  them intelligently in prior to the vector data base value.
• 22:37
  It was tag-based text-based, full text search and a couple
• 22:41
  other you know Geographic search is a lot of different ways that
• 22:44
  you can use red a search. But this just adds one more
• 22:48
  powerful layer which is essentially turns redis into a
• 22:51
  vector database. You can store embeddings inside
• 22:54
  of redis have them in. Decks and searched over.
• 22:57
  And then what that enables you to do is have that Vector,
• 23:00
  search capability stored within redis used inside of your
• 23:05
  application. And the best part is that it
• 23:07
  really usually doesn't change your stack.
• 23:09
  When we talk to companies about this.
• 23:11
  When we talk to users about this they're like oh we're only using
• 23:13
  that as a cash you're saying, all we have to do is store these
• 23:16
  tiny vectors inside of our cash and then suddenly we get all
• 23:19
  those cool capability I'm like, yeah, that's exactly how it
• 23:23
  works and they're so excited about it.
• 23:25
  Because it is such an easy Improvement to their sites.
• 23:30
  We were talking to this one company with, it's like a 250 or
• 23:34
  something thousand products, their think it's toys.
• 23:40
  And what they wanted to do was have Pages where people said, oh
• 23:43
  I bought or I've viewed these products show me similar
• 23:47
  products but they wanted some more novelty in the way that
• 23:51
  they were doing it. They are doing very classical
• 23:53
  kind of data Centric. Way beforehand.
• 23:55
  Doing some clustering and Analysis like that, but what
• 23:59
  vectors, which allows you to do is, do you know, even waited
• 24:02
  Vector search to the point where you can say, give me the ability
• 24:05
  to buy the text attributes of this particular item, or these
• 24:10
  particular items and the image attributes of this particular
• 24:13
  item and or set of items show me a new set of items that resemble
• 24:19
  this item or set of items. And that is really powerful when
• 24:24
  you see it working in In real time, you start to realize how
• 24:28
  much better these search engines become these recommendations
• 24:32
  become because they actually are things you're interested in or
• 24:36
  you things that are really similar to that product.
• 24:39
  And that's been one of my favorite things about working at
• 24:41
  Red. A sexually is that this this new
• 24:44
  push towards the vector database solution side because it's one
• 24:47
  of my favorite uses of machine learning and deep learning
• 24:50
  models because it's really easy with things like the sentence
• 24:54
  Transformers. Is from hugging face, you can
• 24:57
  say modeled on code. I mean I'm sure you remember the
• 24:59
  days where you had to come up with a hook inside of your
• 25:04
  model. So that you know, as it was
• 25:05
  training, right? Going through the forward pass
• 25:07
  and copied out the buffer and then you can store that buffer
• 25:10
  in another place and like now it's just it's so much easier.
• 25:15
  You just say, you know, pull down this sentence Transformer
• 25:18
  encode. Boom.
• 25:19
  I have an embedding and it was it was not possible three or
• 25:23
  four years ago to say, You know have a pie George model and do
• 25:27
  the same thing. And so that's one area that
• 25:29
  really excited about because pretty much any piece of
• 25:32
  unstructured data, text audio video images, what have you?
• 25:36
  You can just feed to these models and then create
• 25:38
  embeddings and perform really powerful intelligent.
• 25:41
  Search is over them and all in, you know, sub tens of seconds of
• 25:47
  bills like Layton sees. Which if you're talking about
• 25:50
  inference pipelines, did you get into a 1?
• 25:52
  Second difference pipeline, it's pretty good.
• 25:55
  Like, that's that's good enough for most use, cases, sub, 100,
• 25:59
  milliseconds, you're killing it. You know, you're doing really
• 26:02
  well and into the tens. I mean, that's really fast.
• 26:07
  And so that's what I really like about this Vector database
• 26:10
  solution from Rhoda's. Yeah, it's up 10-year.
• 26:13
  Like, where do I sign on the dotted line?
• 26:16
  Yeah. You all my money right?
• 26:19
  We do. I get yeah.
• 26:21
  Where I give you money. Who am I standing?
• 26:23
  That's attractive. What about I mean?
• 26:29
  I we've had this guy David I think is David Berg from pine
• 26:35
  cone on here before and I know a ton of people out there are
• 26:40
  probably thinking, especially people who know about this
• 26:43
  space, the probably wondering like, what's the difference.
• 26:47
  Do you have? You looked at like how you do
• 26:49
  things, is it a fundamental difference?
• 26:52
  Whereas it features and benefits, that might be
• 26:54
  different. How do you look at that?
• 26:56
  Features the benefits. So, yes.
• 27:04
  I've looked into Pinecone. We V8, obviously, we've looked
• 27:08
  at the market and seen a lot of the different competitors that
• 27:11
  are out there and kind of the way that I'll structure it is,
• 27:15
  if you think about it. Like, you know, if you're
• 27:19
  thinking about just think of Cartesian grid and, you know,
• 27:22
  you have work, four quadrants, right?
• 27:24
  And it's like open source. Versus managed and then you have
• 27:28
  like you know the discrepancies between those two and really the
• 27:33
  difference between redis and a lot of those other Solutions.
• 27:36
  Is that redis, really has this great platform is already being
• 27:39
  utilized for a lot of different things outside of being a vector
• 27:43
  database solution. And so, just adding in the
• 27:45
  vector database solution just gives you the ability to say,
• 27:48
  oh, it's already in my stack of not, adding a dependency, I'm
• 27:51
  not adding a new piece and that's paying for a new service.
• 27:55
  I'm using my existing commit to AWS or am using my existing
• 27:58
  commit to redis or wherever and it's not like some new po that
• 28:03
  you have to draw up for your company.
• 28:05
  It's just already there and you can just keep using more of it.
• 28:10
  And so that's one really quick advantage that I see just on the
• 28:14
  business side of things is actually enabling this, which I
• 28:17
  don't think people actually talk about them now from them ah
• 28:19
  space is like you know sometimes is really hard to get up the
• 28:22
  other grown up your product actually getting somewhere and
• 28:25
  just He'd already be in the pipeline is such a huge
• 28:29
  Advantage because the second year at feature parity they're
• 28:33
  going to go with what they already have.
• 28:36
  You know and trust me it hasn't been doing this recipe for that
• 28:41
  year. Mihael knows better than
• 28:43
  anybody. He's an aw I was gonna say that
• 28:47
  like how many people there's there's a play right?
• 28:51
  And it's a great play to get on the Amazon Marketplace because
• 28:54
  of that, like now, People can allocate their Amazon budget
• 28:58
  towards a company that is in the marketplace and it's basically
• 29:02
  like their AWS spend. But they get this new tooling
• 29:05
  provider, I'm Emanuele, Mason machine learning engineer, at
• 29:09
  stripe and author of building, machine learning, powered
• 29:12
  applications. And if you don't want your
• 29:14
  machine learning models to explode, well, you should
• 29:18
  subscribe to this podcast. Exactly.
• 29:21
  And that's what we've seen. And so what I was talking about
• 29:26
  it, What's that kind of database conference?
• 29:30
  I tweeted out an alternate title to my talk.
• 29:32
  Which is do you really need another Vector database?
• 29:36
  Which you know it's there are a lot of great tools and they're
• 29:40
  definitely points where there are features that exist in other
• 29:43
  products that are better, or, you know, different in some way
• 29:46
  that maybe aren't reflected in the reddest Vector database
• 29:50
  solution. But what I can tell you is also,
• 29:53
  this is a primary point of focus.
• 29:56
  And the we have enough demand for this that it's going to be a
• 30:00
  fully featured platform in a matter of months.
• 30:04
  And it's going to be the best part about all of it is that it
• 30:08
  already is backed by all of the things that you get from
• 30:11
  Enterprise. So like the Geo distribution,
• 30:14
  the ability to use flash green wedding tables.
• 30:16
  Like you know, Godly big, you can just put it on flash and pay
• 30:22
  a fraction of the cost. If you can still stand a little
• 30:25
  bit, Lower latency and not even by much, because NVM he's got
• 30:29
  really fast levels of them. Then, you know, you can store
• 30:32
  two terabytes on nvme as long as you pick the right at AWS
• 30:36
  instance. So that's, that's something
• 30:39
  that, you know, we feel like because we already have a lot of
• 30:42
  those features adding in the vector data base side of things.
• 30:46
  In addition to that, you know, business side of things that we
• 30:48
  already talked about. We feel like we're pretty
• 30:51
  competitive in the space. It sounds like what you're
• 30:53
  describing is like the upsell model, right?
• 30:55
  Of Features being sold on platforms.
• 30:59
  And I think that's now, because of my past biases at all.
• 31:04
  I think that is a very powerful model, quite frankly.
• 31:09
  He's seen it. Work really well.
• 31:11
  And so, it's tried and true model.
• 31:14
  It's a tried and true model. So I hear where you're coming
• 31:17
  from, people will people will take a little bit worse, just
• 31:20
  for convenience, right? I mean this is, this is just
• 31:23
  human psychology in some sense. You can be any Your Craddock
• 31:27
  convenience, I think a lot of the times and also can get on
• 31:29
  the ground floor of it. You know, being able to say, hey
• 31:32
  can you guys implement this set of features in this kind of way?
• 31:35
  Because the way they did it didn't we didn't like and we've
• 31:38
  seen that too. They're like, oh well, we don't
• 31:41
  really like the way this operates.
• 31:42
  It's not doing this in parallel. Can you guys do it in a
• 31:45
  background thread or in parallel?
• 31:47
  Yeah, we can. So it's almost like this, the
• 31:50
  jobs model where, you know, the Zune came out first and he
• 31:53
  waited and saw how bad the Zune was.
• 31:56
  And then he came back later and did the iPad and everybody loved
• 31:59
  it or, you know, scuse me iPod as I am living in 2022.
• 32:04
  But you know what I mean, right? So you know in the zoo never
• 32:07
  caught on and not to say that this is the exact same
• 32:10
  comparison at all. But you know it gives you a
• 32:12
  mental model of being able to say, see something go-to-market
• 32:16
  be an incumbent in a different area and say hey I think we can
• 32:19
  tackle that area too. So let's change gears a little
• 32:24
  bit and you've done some stuff in the past which is pretty damn
• 32:30
  cool. I've been looking at it a little
• 32:33
  bit and I mean you you played around with distributive
• 32:38
  training or I shouldn't say you played around with you went deep
• 32:40
  on distributed training, hyper parameter, optimization and
• 32:44
  scalable inference. And now you're at redis and I
• 32:46
  almost feel like like is the Need for Speed in your blood.
• 32:51
  What's going on? Like, what, what thirst are you
• 32:54
  trying to quench? I mean, at the end of the day,
• 32:58
  like, I'm, I'm a nerd. Like I grew up building,
• 33:02
  anything are mostly taken apart. Honestly, and my Mom hated me
• 33:06
  for that taken apart, pretty much everything and not always
• 33:09
  putting it back together. I was fascinated with
• 33:15
  supercomputers by my first internship was at krei.
• 33:18
  It's the old, you know, since the 60s, one of the cray-1
• 33:22
  supercomputers are and I fell in love with the how that
• 33:25
  evolution. To create a research from
• 33:28
  Seymour cray and everything came to fruition and cray ink on
• 33:31
  repeating Garo and I really loved the way that you could
• 33:36
  just talk to people about. Yeah, it's like 80,000 MacBook
• 33:39
  Pros without a display strong together and people just seems
• 33:43
  high school and you're like, yeah, Isn't that cool?
• 33:47
  And I fell in love with how you could use that.
• 33:51
  For different problems. It started working on chapels.
• 33:54
  The parallel programming language.
• 33:56
  Um and then things like our to do, which is a distributed
• 33:59
  version of numpy which actually is backed by chapel and then
• 34:04
  that dovetailed into basically distributed feature selection,
• 34:07
  distributed hyperborean, rock musician.
• 34:10
  And that's just really because AI is started to say, hey, we
• 34:15
  should paralyzed most of what we're doing and all of a sudden
• 34:19
  I was working on a I just because I was in high
• 34:22
  performance Computing and this nerd love of I'm compute power
• 34:27
  suddenly became a valuable skill and then, you know, I keep went
• 34:34
  on to work on distributed training.
• 34:37
  Wouldn't just really deep research around improving
• 34:39
  numerical simulations with a. I eventually made a framework
• 34:43
  around that quilts Martin. And really, that's that's
• 34:47
  actually where I got exposed to read this but the Need for Speed
• 34:50
  that you're talking about, that's where I swear I fell in
• 34:52
  love with it is, you know, there's not many people that
• 34:55
  want to sit down. And try to optimize Collective
• 34:58
  Communications in MPI and give you no joy out of improving it
• 35:03
  by 5% and being like, yeah, I did that, like, look how much
• 35:07
  faster it is, but I do. And that's the type of that's
• 35:11
  been her van. Yeah.
• 35:13
  And I'm proud to say, Legends in a sense, it's interesting that
• 35:19
  you say that a lot of people aren't interested in that.
• 35:22
  But even when you look at something like G PT 3, right?
• 35:24
  Or any of these, fancy Dolly, 2's, Dolly, three, whatever.
• 35:29
  You know, these these these basically a lot of open eyes
• 35:31
  stuff all of their Demos in my mind.
• 35:35
  I think the most interesting stuff really is the advances
• 35:38
  they've made in infrastructure, right in.
• 35:41
  Oh yeah, approving. The speed at which things can be
• 35:43
  trained, really just Court in Structural advancements.
• 35:47
  It's not so much modeling or really anything core.
• 35:50
  Algorithmic is hopefully, how do you do this faster better?
• 35:53
  So it doesn't take 300 years for me to train this model, right?
• 35:57
  What they've done with kubernetes is fascinating.
• 36:00
  They've actually made kubernetes.
• 36:02
  I mean, if you think about the workload, managers high
• 36:04
  performance Computing, is the slurms PBS towards the things
• 36:06
  that have been there for ages. They basically took kubernetes
• 36:09
  and made it into, like, Morse learn.
• 36:14
  Like, you know, if you go read there, NG to 7,500 nodes or
• 36:18
  whatever it is that, you know, they had a honestly think it's a
• 36:22
  CS 500 host and a sure I don't really remember but, you know,
• 36:26
  it's a massive computer that they have tons of teams working
• 36:30
  on at the same time in orchestrating all of that
• 36:32
  through kubernetes. I mean, that is a feat.
• 36:35
  It is really, really hard to do that anyways, ever worth our
• 36:38
  kubernetes knows, it's not really the best scheduler in the
• 36:41
  world. It's actually not very good at
• 36:43
  that and slurm still kind of holds the The crown there is the
• 36:48
  Facebook research for large distributed models training.
• 36:51
  You can see in their logs, they're still using this learn.
• 36:54
  You know, they're hidden s info, every three minutes, you know
• 36:56
  that it is still slurm is still dominating that space.
• 37:00
  But what opening? I you're absolutely right the
• 37:02
  infrastructure that they've deployed and what they've really
• 37:04
  focused on it is fascinating what they've been able to do,
• 37:07
  huh? How do you feel about some of
• 37:10
  the tools out there right now? I mean, besides what you were
• 37:13
  just mentioning, but I know there's some people that That
• 37:16
  are like I think about October ML and then I also know about
• 37:21
  like the folks that run run a, I'm not doing yeah, they're
• 37:26
  doing stuff like that. Like it feels like there is a
• 37:29
  movement towards trying to help this.
• 37:34
  I know they're not like, Optimum L and run a iron, exactly
• 37:37
  competitors. But have you like, geeked out on
• 37:40
  those at all? And you have any thoughts?
• 37:43
  Yeah. I mean this is somewhere else
• 37:45
  bent. So so, so So much time.
• 37:47
  That's so nice. I love here.
• 37:49
  Exactly. Yeah.
• 37:50
  So determined that I don't know if you're familiar with the term
• 37:53
  day. I yeah.
• 37:54
  Oh yeah, they were they're pretty much all kubernetes
• 37:56
  pretty much all Docker based and I was leaving the team to
• 38:01
  basically give them on high performance Computing worth of
• 38:03
  managers that was what I was doing right before I left HP and
• 38:08
  so knowing slurm and knowing things like kubernetes in the
• 38:11
  differences. Became my job for a long time
• 38:14
  and so what I'll tell you is Is if you ever use kubernetes, you
• 38:18
  know, there's great apis for almost everything.
• 38:20
  You can just spin up a proxy in your opinion to do this or
• 38:23
  change this configuration, and deploy that column chart to do.
• 38:25
  X slurm does not do that. I mean it has a plug-in system
• 38:29
  but have fun writing to the C API so that you plug it in and
• 38:32
  have fun having that and I mess up whatever plug-in.
• 38:34
  The sysadmin of that sites already done to make it you know
• 38:37
  some different. You get the point right there is
• 38:40
  not the same capability and yes, learn does have a rest API for
• 38:44
  those of you that are going to come back and me.
• 38:45
  But you Watch jobs through that and it doesn't have in.
• 38:49
  It's very young to. This is not a sad, almost learn
• 38:53
  because I absolutely do. But the point there being
• 38:55
  there's discrepancies between the two that I think companies
• 38:58
  like runned out a, I are probably trying to address right
• 39:01
  now because if you know can take the scheduling ability of slurm,
• 39:05
  mix it with the deployment options and flexibility in apis,
• 39:09
  kubernetes given absolutely fantastic product.
• 39:12
  Yes, so you also wrote a paper I want to dive into I think really
• 39:19
  cool maybe a bit of background information on the paper and
• 39:24
  funny enough you were using redis for this paper, right?
• 39:27
  And it was before you were working at Red asthma.
• 39:29
  This might be where the love of red is came from it.
• 39:32
  Absolutely is. Yeah.
• 39:34
  So it's funny that, so yeah, the paper was actually, it was in
• 39:41
  collaboration with the national Center of atmospheric research
• 39:45
  and University. Victoria was some great
• 39:48
  collaborators, like Andrew shouts, got Bachmann Gustavo,
• 39:51
  Marquez, and all the great Folks at HP is it's a paper where we
• 39:57
  tried to use redis. As a online feature store, to be
• 40:01
  able to stream numerical quantities, out of a climate
• 40:04
  model, running it you know full global scale like so you know
• 40:09
  massive running on thousands and thousands of cores CPU, only do
• 40:14
  not reported to GPU. What we did was we stream these
• 40:18
  numeric features over to read it and then use redis day, I to run
• 40:21
  the hydros model. Almost numerical features to
• 40:23
  predict quantity called any kinetic energy.
• 40:27
  When we predicted that, we, then streams that back into the
• 40:31
  numerical model to distribute, Fortran code, and we were able
• 40:35
  to improve the resolution the, you know, this resolved quantity
• 40:41
  of the model because you know, how we were able to predict any
• 40:45
  kinetic energy from Training on a high-resolution version of the
• 40:49
  same climate model. Essentially, if you didn't catch
• 40:53
  that, the tldr is we improved the climate model in terms of
• 40:57
  its skill, let someone final models like, say because they
• 41:00
  avoid the term accuracy in terms of its skill because of this,
• 41:05
  injection of this value of any kinetic energy that is not
• 41:08
  typically present, in a low resolution version of this
• 41:12
  climate model and it's really, it's really fascinating.
• 41:16
  Look at the difference between the analytical parameterisation.
• 41:20
  So the machine learning Quran relations that will went into
• 41:22
  this model and what enabled? It was really the facts that we
• 41:26
  can spread a redis instance over 128 nodes and take up the, you
• 41:31
  know, 8 terabytes of memory and stream.
• 41:34
  The hundred gigabytes time steps redis because you can really
• 41:37
  just crush it with throughput if you have the systems to do so.
• 41:41
  And that was the realization. I made a while ago, when I was
• 41:44
  creating a framework called smart Sim, which Which is
• 41:46
  actually what we use in the paper to be able to do this.
• 41:49
  So yeah I've been deep in the HPC land for quite a bit and
• 41:55
  that was that was kind of dovetailing off.
• 41:57
  So my research that I also did in undergrad as well that sounds
• 42:01
  intense it isn't that description was superintendent
• 42:06
  is hardly on down. It's hard to like you know,
• 42:09
  bring it to it. There's a lot of research that
• 42:11
  went into it so it's it is hard to bring down a notch but yeah
• 42:15
  just just the The part that got me was just when you mentioned
• 42:18
  being able to scale across like 8 terabytes of memory, right?
• 42:22
  You mentioned like, basically a memory sort of compass.
• 42:24
  Wow, that's that's insane. Yeah, kind of sounds expensive.
• 42:28
  Oh yes. Who's paying for this shit?
• 42:30
  Yeah, it's a great question. HP, mostly at the time.
• 42:36
  But yeah, we were, I was really, really blessed to have access to
• 42:40
  just XE 50s. XX, you do XD systems the best
• 42:45
  supercomputers in the world. Old for the early part of the
• 42:48
  first six years of my career. I spent a lot of after work
• 42:53
  hours, just playing with core about not, you know, CS5
• 42:58
  hundreds in XE 50s and that was a huge blessing for me because
• 43:02
  it was just something I was thinking out over and I got to
• 43:04
  learn a lot. So as a as an HPC Guru, what are
• 43:10
  you? Okay?
• 43:12
  You as a very as someone who likes diving into this stuff
• 43:16
  what are Most excited about what like, what applications or
• 43:20
  future applications of the tech. Do you get really excited about
• 43:25
  and something that maybe we haven't seen yet or we expect to
• 43:27
  see in the future or, you know, if we, if we line up the pieces
• 43:30
  correctly, we're going to see this later and you can't wait
• 43:34
  for that future to happen. Well, I can't bail everything
• 43:38
  but if you, if you kind of put some pieces together.
• 43:41
  So I come from HPC land where I used redis and you know We are
• 43:46
  doing a lot of AI in ml a trellis and I think you can kind
• 43:50
  of put two and two together that there may be some things that we
• 43:54
  do in the future that might be similar to the things that we
• 43:57
  were prototyping. And you know showing at scale
• 44:02
  maybe on a smaller scale per se, maybe not this.
• 44:06
  Yes, 500 actually 50 scale. But you know, still distributed
• 44:09
  in large-scale and I am now with redis.
• 44:12
  So I think that's one area that I'm really excited about is
• 44:16
  being able to Do some more interesting things with red is
• 44:19
  that that kind of scale in the future on different types of
• 44:22
  Hardware to that wasn't a good, a good teaser that people aren't
• 44:26
  listening to doing and getting ready to sign up for it then I
• 44:28
  don't know, I don't know, I can't think of a better way to
• 44:30
  pitch a product than that. Yeah, that's incredible.
• 44:35
  Well, I wanna finish it up with, like, a little lightning round
• 44:38
  and these light your rounds. Usually go there, lightning, not
• 44:43
  so lightning. We could say.
• 44:44
  So, Try and answer this as short as you can.
• 44:49
  What was the last book you read? Oh blank.
• 44:53
  Mountain by well. Just finish, nice.
• 44:57
  All right, tell us about the last bug.
• 44:59
  You smashed the last bug? I smashed I was on the pier in
• 45:09
  in Where was that? Santa Cruz.
• 45:13
  I was in my car. I was on my hot spot and I was
• 45:19
  basically working on our python back-end for this Vector search
• 45:23
  Tab. And what happened was, I
• 45:26
  essentially created the embeddings wrong.
• 45:29
  I was using the wrong field name, and nothing was coming
• 45:31
  back and I was incredibly frustrated mostly because my
• 45:34
  internet connections terrible, but after I fixed it from IMG
• 45:38
  Vector to image vector We're perfectly and that was all my
• 45:43
  fault. The head of the hood, there you
• 45:47
  go. Stop learning from that and make
• 45:50
  sure you get the names, right? Maybe it's hard that yeah, don't
• 45:56
  and avoid using your hot spots when possible the there's also a
• 46:03
  that.remember, the Don, knuth quote, that the two hardest
• 46:07
  problems in computer science are naming and cache invalidation
• 46:10
  both of which That I think are relevant to you, right?
• 46:12
  Wow, that's really good. I should get that on board in my
• 46:16
  office or something tattooed on your arm, man.
• 46:19
  That's what you want. That's dedication.
• 46:22
  It's different. Every single member of the
• 46:24
  reddest Team. Every member is no, that's,
• 46:26
  that's your doctor normally requirement.
• 46:29
  That's it next off-site. Yes, that's so good.
• 46:38
  All right, so, What piece of technology?
• 46:43
  Are you bullish on? That might surprise people.
• 46:47
  I think I will say, feature Platforms in general and that,
• 46:53
  that, you know, I think that might surprise some people
• 46:55
  because I think some people see them as like, you know, working
• 46:58
  on a small margin. Maybe you're, you know, maybe
• 47:01
  not feeling as bigger roles they should.
• 47:03
  But I think they're really important and the larger
• 47:06
  abstractions that made things easier for people kind of like
• 47:09
  what I was talking about with the hugging face model.
• 47:12
  And code and making things like creating embeddings easier
• 47:16
  feature platforms. Make a lot of these distributed
• 47:19
  systems that we used to have to think, for about four days.
• 47:23
  You know, much simpler and especially when you're deploying
• 47:27
  multiple databases for multiple purposes.
• 47:29
  I think those Peach platforms can really help excellent.
• 47:33
  Last one, Sam. How do you want to be
• 47:35
  remembered? I was this a lightning question.
• 47:41
  We need three to five words for this one.
• 47:43
  No more than that, please. Yeah.
• 47:49
  As a nerd I guess, you know, I think that's it.
• 47:52
  I as a person who really loves technology and wants to learn
• 47:56
  more every day about it. Like and at the end of the day
• 48:00
  it's who I am. Excellent.
• 48:02
  Dude, we're going to end there. Thank you so much.
• 48:04
  If anyone wants to check out redis or hit Sam up your in the
• 48:08
  community slack. You're also on LinkedIn Twitter,
• 48:11
  all that fun stuff and it's redis, you know where to find
• 48:16
  redis. I mean, let's be honest.
• 48:17
  If you don't, it's Sorry D is shredded.
• 48:20
  Exactly and you make yeah cool thanks man.
• 48:26
  Oh hey thank y'all it's been awesome.