Latest / Agentic Conversations (formally mlops.community) / Context engineering 2.0, Agents + Structured Data, and the Redis Context Engine

Transcript

Context engineering 2.0, Agents + Structured Data, and the Redis Context Engine · Agentic Conversations (formally mlops.community)

• 0:00
  Biggest engineering part of right.
• 0:25
  So feature form got acquired by Redis.
• 0:29
  Does that signal the death of the feature store?
• 0:33
  It doesn't. It's funny because you made this
• 0:35
  LinkedIn post about this. And I was trolling you.
• 0:37
  You were trolling me. I knew I knew you were baiting
• 0:40
  me, so I didn't, I didn't take the bait.
• 0:42
  One like let's talk about where going back to ML OPS in general,
• 0:45
  like where did ML OPS work and where did it not work?
• 0:49
  I think the promise of ML OPS from the early days, which you
• 0:52
  remember, was that we were going to have models everywhere.
• 0:55
  We're going to have these like hyper intelligent apps where
• 0:57
  every time you're using an app, you would interact with like
• 1:00
  hundreds and hundreds of models. That was the idea.
• 1:02
  So you needed ML OPS to be able to operationalize all those
• 1:06
  models. The reality is, is that did
• 1:10
  actually happen, but not because of ML OPS.
• 1:13
  It happened because of. Transformers came.
• 1:17
  Yeah, and and Transformers like are I mean are old, but like
• 1:21
  specifically like like DPT style Transformers worked.
• 1:25
  And the nice thing about them is the bar to get them in
• 1:28
  production is weight lower because there's no you don't
• 1:30
  need to train them. They this kind of generically
• 1:33
  work. So the bar of how much Roy you
• 1:37
  need to justify doing it is so low.
• 1:40
  Yeah, and you can just hit an API.
• 1:42
  Exactly. So it's just like, so now where
• 1:46
  do you actually need or where does it make sense for a
• 1:49
  business to actually invest to maybe have an entire ML OPS
• 1:53
  team, which is an expensive team and it's an operational team.
• 1:57
  So it's like a team that's whole job is to help other teams do
• 2:01
  things. So if you start thinking of like
• 2:02
  this economics of where that would actually make sense, one,
• 2:06
  it tends to happen at scale. 2, the the use cases that we see
• 2:11
  very often, there's many, but the ones that come up a lot, 1
• 2:14
  is recommended systems. Yeah, which makes complete
• 2:17
  sense. Like if your recommendations are
• 2:19
  better, you make more money. There's a very short line
• 2:22
  between recommendations being better and money.
• 2:25
  And the other place you see is fraud.
• 2:27
  Same thing if you, yeah, screw up, you know, if your fraud is
• 2:31
  better, that literally equals money.
• 2:33
  And so and at scale like for every single user of your
• 2:36
  application or every single transaction that goes through.
• 2:39
  So that's where we've seen envelops work.
• 2:41
  And we're still seeing to this day, a lot of companies are
• 2:45
  deploying new feature stores, very doing, very existing
• 2:49
  feature stores. We're actually seeing some of
• 2:52
  the most demand we've ever seen for feature stores now.
• 2:57
  And I think that's, that's partly because that's forgotten
• 3:02
  a bit. And so when in the early days
• 3:04
  when people went and built their own, it's like the maintenance
• 3:07
  cost of those things is kind of a why did we build this thing
• 3:10
  when there are all these solutions out there?
• 3:11
  Yeah. And the maturity?
• 3:13
  Curve Maturity curve. You start to see that people
• 3:15
  understand the concept and they realize that it's not just like
• 3:19
  a Postgres database. Yeah, people are starting to
• 3:21
  really feel the problems and it's it's less like we need this
• 3:27
  thing because LinkedIn said so and like hype driven development
• 3:31
  and we're getting into like this actually is a huge pain.
• 3:34
  And if we solve this, there would be true ROI wait.
• 3:36
  Wait, there's there's things I want to know about the
• 3:38
  acquisition one being just like, what's it like to sell a
• 3:42
  company? How much of a pain is it?
• 3:44
  I've heard due diligence takes years off of your life.
• 3:49
  So in general, I mean raising around we've raised multiple
• 3:53
  rounds. Acquisition I would say is more
• 3:56
  intensive than raising around. It just it depends.
• 4:01
  I think it depends on a lot of things you can't really say.
• 4:04
  Anyway, I think the acquisition here it, it made sense for a lot
• 4:08
  of reasons. One being just as a team is like
• 4:12
  Adna match. I think of Redis.
• 4:15
  I think we, we, we Redis is known for usability, developer
• 4:20
  experience. It's like really has that dev
• 4:22
  love that we, we really think about a feature form, something
• 4:24
  that we obsess on. And I mean, the cool thing, the
• 4:29
  acquisition for us is like, we got to keep the whole team is
• 4:31
  there. So the feature form team came as
• 4:33
  a whole, which from the product continues, it will continue to
• 4:37
  exist. It's getting doubled down on.
• 4:39
  We're actually getting more resources to keep doing what
• 4:41
  we're doing and build on top of it.
• 4:44
  Our product enrich MCP is kind of transforming into something
• 4:49
  even bigger with contacts engine than what we're working on
• 4:53
  there. And I just think Redis is a
• 4:56
  company. It's just in such a cool spot
• 4:58
  because it's kind of moving and and up leveling from becoming
• 5:00
  just Redis, which again is is powerful.
• 5:04
  It's sorry, but it's like providing that platform layer
• 5:07
  the solutions on top and becoming this kind of AI and ML
• 5:11
  data platform and or a context platform.
• 5:15
  So I think it just was the right fit at the right time and it's
• 5:21
  just a really good match. And I mean, it's only been a
• 5:24
  month or so since it's happened. And like I'm wearing a red
• 5:28
  sweatshirt. I'm like, I really it just has
• 5:31
  been even better than I expected.
• 5:33
  Yeah, I'm stoked for you and hopefully we'll get to talk to
• 5:36
  you in two years and hear all about how you've evolved through
• 5:40
  the company and all that fun stuff.
• 5:42
  Yeah, it's, it's, it's, it's really awesome.
• 5:44
  And I think I think that's probably important part.
• 5:47
  I mean, my biggest thing of feedback is same with
• 5:49
  fundraising, like your partner, who you work with is all that
• 5:52
  matters because I also know a lot of companies that have been
• 5:54
  acquired. And the partner in the company
• 5:57
  that acquired? Yeah, partner.
• 5:58
  So I was thinking more like VC partner, but both like whoever
• 6:01
  like who, who are you working with?
• 6:03
  Who is like your champion there? Who is who cares about this?
• 6:08
  Are they investing in this? Are they not investing in it?
• 6:10
  How are they thinking about it? Because you can kind of get in
• 6:13
  your own head like what you want them to think and what is
• 6:16
  reality. And I think it's really
• 6:18
  important to go figure out what is reality, what is actually
• 6:20
  happening. So you make the right decision
• 6:22
  for yourself and your team and your invest and just everyone
• 6:24
  make sure that it's it's a all win.
• 6:27
  And, and unfortunately that rarely happens.
• 6:30
  And we, I think, are super fortunate that it did for us.
• 6:34
  Wait, so Redis already had a feature store, right?
• 6:37
  Or Redis was doing stuff? I know.
• 6:39
  I remember them talking a lot about feature stores back in the
• 6:43
  day. Yeah, feature form now is just
• 6:46
  folding into Redis. What does that path look like
• 6:48
  now that you're part of Redis? Yeah.
• 6:50
  So on the feature store part, how it looks is this.
• 6:53
  So Redis is always powered the online store.
• 6:56
  So a lot of companies come to us for like we're building a
• 6:58
  feature store, we're going to use one.
• 7:00
  Maybe we're going to use feature form, maybe we're going to use
• 7:04
  Feast, whatever they're going to, or if we're going to build
• 7:07
  something in house, we need an online store.
• 7:10
  And Redis is the best for a variety of reasons.
• 7:13
  But the general vision is like, well, why don't we meet our
• 7:18
  customers and prospects where we're at?
• 7:20
  If we're trying to solve this thing, why don't we meet them
• 7:23
  there and have a solution for them that's custom built with
• 7:26
  Redis? So rather than them having to go
• 7:28
  figure it out and just use this as a component, we can actually
• 7:31
  give them the full solution. And features and feature form
• 7:35
  traditionally was something that played nicely with everybody,
• 7:38
  right? It was one of those things where
• 7:40
  you said the engine, the compute, the storage, it doesn't
• 7:44
  really matter. We can just be on top of that.
• 7:46
  Yes. And that's so true.
• 7:48
  In fact, we still have customers on like other online stores
• 7:52
  today and that will continue to be true because Red, this is
• 7:55
  strategy, which I really appreciate and this is something
• 7:58
  that I again, really resonated with me and drew me to the
• 8:01
  company was we want to win on our merits.
• 8:04
  We don't want to win on like we own this thing.
• 8:06
  So therefore, you know, we're going to like shut off Cassandra
• 8:11
  support. It's actually no, let's get in
• 8:14
  and make it. I mean, the other thing with
• 8:16
  feature form on the flip side is because it's all unified, it
• 8:19
  makes it really easy to have like Cassandra next to Reddis
• 8:21
  and have both. So you don't have to kind of
• 8:23
  choose one online store or the other.
• 8:26
  Oh nice. And so then it's like, great,
• 8:27
  well, if we get people on feature form, if they just use
• 8:30
  feature form and pay for feature form, fine.
• 8:32
  I mean, it's still revenue for Redis, but our view and it's,
• 8:36
  it's showing to be true is actually now that it's easier to
• 8:39
  try Redis, you are more likely to one, just try it because you
• 8:43
  have it, you have a contract with us to get feature form and
• 8:47
  they will show you, we'll put like our kind of our money where
• 8:50
  our mouth is. We'll do a comparison on cost,
• 8:52
  on speed, on all these things and you'll see that we are
• 8:56
  actually the best online store for best.
• 8:58
  And if we're not, you know, for certain use cases, then that's
• 9:01
  fine too. Think they call that the Trojan
• 9:03
  Horse strategy. It it is, but I wouldn't say
• 9:07
  it's, I mean it's, it's sure like there is some aspect to
• 9:10
  that. But I think it's it's actually
• 9:12
  what I like is it's in in really good faith.
• 9:15
  It's like, yeah, like, yeah, we want to compete, we want to win.
• 9:17
  I mean, of course we do, like every company does, but it's
• 9:20
  not, it's like, let's just make it easier for them to compare,
• 9:25
  not lose due to just like, you know, walls that actually limit
• 9:30
  competition. Like, Oh well, like it's so hard
• 9:33
  for us to move over to this thing.
• 9:35
  So we can never, we can't, you know, we have to not only be
• 9:37
  bettering whatever we're using, we also have to justify the move
• 9:41
  cost. And now we don't have to, we can
• 9:42
  just be like, cool, what's actually better for you?
• 9:45
  And you can choose that. And so it's not just a Trojan
• 9:48
  horse. It's actually a simplicity to
• 9:49
  allow the best online store to win.
• 9:52
  And we believe that's us and and we've been building towards that
• 9:54
  for a long time. So talk to me about Redis and
• 9:58
  Elastic Cash because I'm not sure I fully understand what the
• 10:01
  difference is between the two. Elastic Cash and this isn't true
• 10:06
  for all the clouds. So I'll talk about Elastic Cash.
• 10:08
  Like for example, on Azure, we have AMR which is managed Redis
• 10:12
  and that actually is Redis. That's us of Elastic Cash.
• 10:18
  They have, there's a fork of Reddis on the older version of
• 10:22
  Reddis. And that's actually what's
• 10:26
  powering Elastic Cash. And there's a lot of things like
• 10:32
  availability, speed, etcetera, even just like general
• 10:35
  capabilities, Reddis, core Reddis, Reddis, we released
• 10:39
  Reddis 8 recently. Like core Reddis has really
• 10:42
  pulled away in a lot of places. The other thing though, isn't
• 10:47
  just Redis the, the, the kind of open source product that is also
• 10:52
  Redis the company. And I think one thing and
• 10:55
  everything that really is interesting to me about Redis
• 10:57
  coming in and then everything that really the vision that I
• 11:00
  really liked is think of like like feature form and feature
• 11:05
  stores. Like Redis has always just been
• 11:06
  like that database, the core database, which people
• 11:08
  developers have always used, we've always loved.
• 11:11
  And that continues to be true. But we're kind of like having
• 11:15
  this our almost like a platform moment.
• 11:17
  We're like, cool, what are people using Redis for?
• 11:20
  What more can we do to provide more value, maybe a higher
• 11:23
  levels of abstractions and solutions for both agents for ML
• 11:29
  via feature stores in a variety of like we have Flex which is
• 11:33
  like close stores in SSD, which is really powerful, especially
• 11:37
  actually for feature service cases where you might have
• 11:39
  massive data sets. Vector database.
• 11:42
  I remember back in the day we did a whole before LLMS came
• 11:46
  out, we had a vector search or some I think it was similarity
• 11:50
  search hackathon where we were using archive and it was with
• 11:56
  Redis. You know, what's funny is the
• 11:58
  use case with Rexis and actually a lot of the vector DBS, you
• 12:03
  know, people think of them As for RAG.
• 12:04
  But my sense, I actually don't know what's true at Redis, but
• 12:08
  I, I have seen actually just other companies in my sense is
• 12:12
  actually about a lot of revenue still comes from recommender
• 12:14
  systems for Vector DVS and not actually for Rag as people would
• 12:17
  think, which is kind of for features for what you'll do is
• 12:21
  you will generate like user embeddings or item embeddings
• 12:25
  and then you will see, you know, very simple thing might be I
• 12:28
  have an item embedding for like a, a shirt.
• 12:31
  If I do a nearest neighbor search on that I can get similar
• 12:35
  shirts and so they'll do candid generation and then re ranking.
• 12:39
  And this is like how 2 tower recommenders.
• 12:41
  Work. Yeah, I remember the two tower
• 12:44
  stuff that you mentioned back in the day and I totally had
• 12:47
  forgotten about that. Yeah, because I think funny
• 12:50
  enough, you were the first person to talk to me about
• 12:54
  embeddings and you had what, an embedding stores what you were
• 12:58
  calling it or? Embedding hub.
• 13:00
  Embedding hub? Dude that was what, like 2021?
• 13:03
  Yeah, we were like, I think fish from we were like one of the
• 13:06
  first factor databases, which is really funny.
• 13:10
  And it's because at that point I built like three of them.
• 13:12
  So I was just like, this needs to exist.
• 13:14
  What I would, I couldn't have guessed is that, oh, you know,
• 13:18
  in like 6 months or a year, this is actually going to be like
• 13:21
  super huge market because of Rag and LLM such hadn't even like
• 13:25
  crossed my mind. I built it again for access use
• 13:28
  cases because we were seeing people using feature source for
• 13:30
  our commander systems. So that's why we built it.
• 13:34
  And it's really funny how that worked out.
• 13:35
  And then we kind of bowed out because it became like a
• 13:38
  database thing and feature form itself.
• 13:42
  We just weren't like a true database company.
• 13:44
  We didn't really have that DNA, but I've always kind of wished
• 13:49
  we did and then kind of feel very fortunate now that like now
• 13:52
  like they're, you know, those people are to call away for me.
• 13:55
  So I understand the platform idea.
• 13:57
  I also know that in your talk when we were in London, you kind
• 14:01
  of broke down this vision of we've got all of these assets
• 14:04
  with Redis. And so now how can we unify them
• 14:08
  for the new generation of agents?
• 14:11
  Yeah, I've been calling it Context Engineering 2.0, which I
• 14:15
  don't know if I love that yet, but that's what I'm going with.
• 14:17
  It's catchy. It's catchy.
• 14:19
  I'll work with it. Yeah, you got to find, find the
• 14:23
  hype words, so you got to make new terms together.
• 14:25
  But, but really what I'm seeing is OK.
• 14:29
  So when we started with LMS, I think from the earliest point, I
• 14:34
  realized that the only thing, not the only thing, but one of
• 14:37
  the most important things is context.
• 14:40
  And when I think of context engineering in RAG, actually, I
• 14:43
  always felt like we over, we were too specific.
• 14:47
  Typically when we talked about RAG, people talk about RAG.
• 14:50
  We're like vector DB nearest neighbor look up on like
• 14:53
  paragraph chunks based on the user query.
• 14:55
  That's kind of like what people think when they think RAG.
• 14:58
  And but really the problem isn't that.
• 15:03
  The problem is I have a context window.
• 15:05
  It's yay big. I need to fill it with the most
• 15:09
  relevant and insightful information for the LLM to be
• 15:14
  able to do what it needs to do. And I needed to be also not not
• 15:20
  relevant because then they can throw again cause
• 15:22
  hallucinations. Now it just turns out that a
• 15:25
  really really good baseline way to do this is just grab a lot
• 15:30
  like a tax corpus and chunk it up and and kind of do rag.
• 15:35
  So RAG works naively just like weirdly well for how like kind
• 15:40
  of like cookie cutter of a solution it is.
• 15:43
  It kind of just works well for everyone.
• 15:46
  But that's just a baseline and we're clearly like moving past
• 15:50
  that. And the other parts of context
• 15:52
  have come up are like memory. It's a big one.
• 15:55
  And I think the newest frontier is structured data for agents.
• 16:00
  So it's kind of like kind of as MCP, but not really.
• 16:04
  I think it's a, it's actually a more general concept, which is
• 16:08
  how do I get, if someone asks? I mean, it could be something as
• 16:11
  simple as I'm building a support agent, someone's like, why is my
• 16:14
  order late? Go and find the order, find all
• 16:17
  the information about the order, which could be in like Postgres,
• 16:20
  like it could be in the structured database, get that
• 16:23
  into context because the agents going to need it to be able to
• 16:26
  solve the problem at hand. So we have these three things,
• 16:29
  unstructured data, memory, structured data.
• 16:32
  And today if we're going to do that, you would build like a rag
• 16:35
  pipeline here, then you would build like a memory service, and
• 16:39
  then you would deploy like some MCP servers.
• 16:42
  And these things don't really talk to each other or almost
• 16:45
  like kind of disjointed context. And they're all kind of work
• 16:49
  differently. And so my sense is we should
• 16:53
  build relationships between this context, like a memory can be
• 16:56
  associated with a document. There could be a row of data
• 17:01
  that kind of points to, you know, few paragraphs in the in
• 17:05
  the in the rag store. So almost like knowledge graph
• 17:08
  type of. Thing almost like knowledge
• 17:09
  graph. I don't like using knowledge
• 17:11
  graph purely because it's so overloaded in in this space.
• 17:14
  People mean they start thinking graph rag, which is not what
• 17:17
  this is and not how I would describe it.
• 17:18
  It's more just treating context as a holistic thing.
• 17:23
  There's all these indices and like different ways to, let's
• 17:25
  say, retrieve data, and you also need some sort of like semantic
• 17:32
  layer on top to understand that data.
• 17:34
  I kind of think of it. I've been thinking of it as what
• 17:37
  I wish I had was graph QL for agents.
• 17:41
  What's graph QL? Graph QL was kind of one of
• 17:44
  these things that was really high, but never I think fully
• 17:46
  took off. And graph QL is a different way
• 17:49
  of building like Rus AP is. Rus AP is a very like kind of
• 17:52
  input output graph. QL allows you to almost create
• 17:56
  and and unify a lot of data from different places and write kind
• 17:59
  of almost like ad hoc queries. And the reason why I didn't take
• 18:03
  off was this kind of complex to do.
• 18:05
  It could be slow and humans in general, like the especially
• 18:10
  engineers be like input outputs much clearer and cleaner to
• 18:12
  understand. The thing is, is does axis of
• 18:15
  freedom a graph QL gave us is extremely powerful for agents.
• 18:19
  Agents love having axis of freedom because they can
• 18:21
  actually take advantage of all of that.
• 18:23
  And the speed to like parse a graph QL query, you know,
• 18:26
  doesn't matter anymore because it's, I mean, the agents, the
• 18:29
  slow part, it's not the the the, yeah, typically not the data
• 18:33
  retrieval or the query. So graph QL I'm trying to wrap
• 18:36
  my mind around, but also how would that even work with
• 18:40
  agents? So how I imagine it is this you
• 18:43
  would define like let's say you're an e-commerce store, you
• 18:45
  have users, you have items, you have orders.
• 18:48
  These things are all. Yeah.
• 18:52
  There could be, yeah, very tabular, but they might also
• 18:54
  have documents in them, right? Like order might have
• 18:57
  descriptions or might have specifications or any other.
• 19:01
  There's a mix of things, but it's like you have attributes
• 19:05
  and you have relationships. So you can define this whole
• 19:08
  thing out. You can kind of define or
• 19:12
  explain somehow how to get to that data.
• 19:15
  Then you expose this knowledge graph or semantic graph to the
• 19:20
  agent. So agent can be like, cool, I
• 19:22
  can navigate and figure out what data exists.
• 19:25
  So I can go ask for exactly what I need to be able to resolve
• 19:30
  what's at hand. The nice thing there is there's
• 19:33
  two parts, one or there's many parts, but one part is control.
• 19:38
  You as the user can really control the contextual view that
• 19:43
  the agent has, which you can't do like Texas Sequel, which is
• 19:46
  like a whole nother thing. You can't really do it with API
• 19:50
  to MCP because you're just kind of stuck to every API has and
• 19:55
  the surface area there. And so this allows you to build
• 19:59
  a surface area that is tailored to agents with like the right
• 20:02
  kind of abstractions and semantic layer for an agent.
• 20:06
  But you see this almost being like an abstraction above all of
• 20:10
  that stuff that is behind the veil.
• 20:14
  It's a single pane and glass. So I'm calling it today, I call
• 20:17
  it, I'm calling it context engine, but it's a single pane
• 20:20
  and glass to all of your contacts or about structure
• 20:22
  data, unstructured data or memory.
• 20:23
  So you disconnect it, it speaks MCP, but that's this
• 20:26
  implementation detail. But the agent can connect to
• 20:29
  this thing and they can go and figure out, OK, what context
• 20:33
  exists in the world that I have access to and what does it mean,
• 20:37
  How is it related? How do I get it?
• 20:39
  They can go and retrieve what it needs in that way.
• 20:42
  And you're letting the agents figure out the semantic layer on
• 20:45
  top so that they understand, or is it humans that are kind of
• 20:48
  creating a document that says, well, this is this data, If
• 20:52
  you're getting these kind of queries, you should reference
• 20:54
  this. Yeah, it's written in Python, so
• 20:57
  you the the the developer would write it in Python.
• 21:01
  They could use an agent to help them do it, but I think it's
• 21:04
  really important that the humans kind of build this to control
• 21:09
  it. So I think that's a lot of what
• 21:13
  engineering becomes is, is, is that for agents, which is again,
• 21:16
  I think most of the, the biggest engineering part of agents for
• 21:20
  for builders becomes the engine like the context engineering.
• 21:25
  And this is my view of what context engineering looks like,
• 21:28
  at least for the next 5-10 years.
• 21:30
  Interesting. It's more the descriptions and
• 21:32
  the pointers and the understanding of the data.
• 21:34
  It's still being intimate with the data and understanding where
• 21:38
  and what you need, but it's not necessarily writing scripts or
• 21:43
  writing different functions to get to that data.
• 21:47
  It's describing that data in a way that an agent is going to be
• 21:51
  able to understand to get it when it needs it.
• 21:53
  For someone maybe like if you're listening to this and you want
• 21:56
  it to maybe see it more clearly, we feature form had it still has
• 22:02
  open source product called Enrich MCP which is widely used
• 22:06
  for and it it really captures that.
• 22:10
  Now Enrich MCP and how it was built originally is really
• 22:15
  generic. Now when you allow Redis to
• 22:20
  become a dependency, you get. So because now you get a vector
• 22:24
  database, you get all of the, the, the abstractions that Redis
• 22:27
  provides underneath the hood and the caching that you might need
• 22:31
  for AP is all of that stuff becomes part of it.
• 22:34
  And that's I think what context engine becomes is like this
• 22:38
  layer which maybe mimics or looks like enrich MCP or is that
• 22:42
  kind of a continuation of it but with the power of the Reddis
• 22:47
  database behind it? Yeah, almost like the Reddis
• 22:49
  ecosystem because it's, I really like how you've shifted my
• 22:53
  thinking from Reddis as this cache and just something that's
• 22:57
  super fast when you need it to. OK.
• 23:00
  Reddis has these other things though, and now you can look at
• 23:03
  it as an ecosystem. And how can we use all of the
• 23:07
  assets that Reddit Reddis has to leverage our agentic use cases?
• 23:12
  Yes, when you think of why people like Reddis, there's two
• 23:16
  things. One which we talk a lot about
• 23:17
  speed, which is like well understood Redis is in remain
• 23:22
  pretty much the fastest thing you can deploy for data.
• 23:25
  And the second is the founder is kind of a legend in his own
• 23:29
  right. Yeah.
• 23:30
  What was his name? Salvatore.
• 23:32
  Salvatore. That's it.
• 23:33
  The guy's. Like he's an employee of Redis
• 23:36
  again. Oh boy.
• 23:37
  Yeah, he's awesome. So that's the second reason.
• 23:40
  Or what was the? Second reason?
• 23:41
  No, that would be the third reason.
• 23:43
  The second reason is the abstractions that people that
• 23:46
  you have. It's like the simplicity.
• 23:50
  It's almost like when you look at Redis, you're like, oh, cool,
• 23:53
  I have all these types of things I can use like these different
• 23:55
  types of data structures that are built into Redis and I can
• 23:58
  pick and choose what I want. And it's really, really simple.
• 24:01
  It's all tax. It's just nice.
• 24:03
  Like the UX of Redis is always been really nice.
• 24:08
  And so now you have a really nice UX that's super, it's like
• 24:12
  Swiss Army knife and like it does so many things and they all
• 24:16
  kind of work really quickly. And we've read this cloud like
• 24:19
  now you get high availability and all this other stuff you'd
• 24:22
  need for a production setting. So you combine all that and then
• 24:25
  you build a layer on top, which is again, that kind of almost
• 24:28
  like data platform for agents. That's kind of where we're
• 24:30
  going. And I talked about context
• 24:32
  engine, which is a huge part of it.
• 24:33
  And I think a big, the point of context engine is, hey, agents
• 24:37
  give agents a single pin and glass to get to the data that
• 24:40
  they need to solve real problems.
• 24:44
  There are other things we do even to like our caching
• 24:47
  heritage, like we have lane cache, which is a lot of people
• 24:52
  have problems with spending too much money on tokens, right?
• 24:56
  Ones are very expensive. And when you go look at what
• 25:00
  people are using like your customer support for, you'll
• 25:02
  find that like a massive percentage of questions are like
• 25:06
  the same thing. 8020 Principle, but again.
• 25:09
  If you were to cash naively, you're like, oh, if this exact
• 25:13
  query gets written, it's not going to work for two reasons.
• 25:16
  One is if someone changes one word and doesn't, it doesn't
• 25:20
  match. 2 is if someone's like what's the weather today, then,
• 25:25
  well, you can't use the cached answer because yesterday the
• 25:28
  lever was different today. So what lane cache is is it uses
• 25:37
  actually it uses the vector DB, the vector DB under the hood
• 25:40
  again showing building these solution layers on top.
• 25:43
  And it enables us to actually see is this equivalent?
• 25:47
  And we look at a lot of different axes.
• 25:49
  There's negation, which is another thing, which is like
• 25:51
  something can be similar, but if you add the word not in it, it's
• 25:54
  actually the completely the opposite.
• 25:56
  If you're like, you know, what's the weather in San Francisco
• 25:58
  versus what's the weather in San Diego, It's like it looks pretty
• 26:01
  much the same. But that Francisco or Diego is
• 26:03
  completely changes the the context.
• 26:08
  And so, yeah, I mean, we're just doing well.
• 26:10
  I mean there's just so much happening at Redis and I think
• 26:13
  the, the vision is to continue. We've we've been the Swiss Army
• 26:16
  knife for the web age for things like sessions, rate limiting,
• 26:22
  just everything around making web apps reliable, fast at
• 26:27
  scale. I think now like the vision
• 26:30
  continues to be that and I think that's that is still a a huge
• 26:33
  part of our business, but also thinking of the future and where
• 26:36
  things are going. And I think the future of Redis
• 26:38
  is providing context to agents. What I am still caught up on is
• 26:44
  the single plane of glass and the let's say whether it's
• 26:49
  enrich MCP or it is this context engine, How do you basically
• 26:54
  give the data to the enrich MCP? Is it through tools you
• 26:59
  mentioned that enrich MCP, Obviously it speaks MCP, it's
• 27:03
  kind of in the name. So it's a dead giveaway.
• 27:06
  It's a giveaway. But is that just like on your
• 27:10
  side tools going into the different databases?
• 27:14
  Is it like? How does that actually work?
• 27:18
  Yeah. So there's Enrich MCP, but I'll
• 27:20
  actually talk about more like the broader vision of context
• 27:22
  engine. So think of how a rack pipeline
• 27:28
  works. You take documents in S3, let's
• 27:31
  say somewhere you transform them, which is just chunking,
• 27:36
  embedding, get them in a vector database.
• 27:40
  That thing you have in the vector DB is actually, if you
• 27:44
  zoom out a little bit, it's a, this is a materialized view.
• 27:47
  You're projecting documents in S3 into embedding space.
• 27:52
  And so that's a weird, you know, historically would have been a
• 27:54
  really weird transformation to do, but it's, it's the same,
• 27:57
  it's a materialized view. Online stores, features in
• 28:01
  online stores. These are just essentially
• 28:02
  materialized views. And so one aspect is just kind
• 28:06
  of materializing views that you want your agents to be able to
• 28:10
  see into Redis, making those queryable.
• 28:14
  Redis has a query engine called RQE, which powers the vector DB
• 28:17
  and the law of the other stuff, search, Redis search and other
• 28:20
  things. But if you project into Redis,
• 28:25
  you can then and then you explain what are these things
• 28:28
  and how they're related, how are they associated to each other.
• 28:31
  You they become accessible. But it's not just that, because
• 28:34
  some things you don't want to materialize into Redis for
• 28:36
  whatever reason, you can still support AP is calls or other MCB
• 28:41
  servers like backed. But when you're projecting into
• 28:44
  Redis, yeah, and then the agent is going to get that.
• 28:47
  Is that just through a tool called to Redis?
• 28:49
  It's like. Yes, it would be through context
• 28:52
  engine, but it would be backed by Redis.
• 28:55
  So it would just be like MCP in the end creates tool calls and
• 28:59
  another way to actually think about what enriched MCP is or
• 29:02
  the I guess thing behind it. I've talked about the graph QL
• 29:05
  thing. The other thing that I was
• 29:07
  thinking about was for a lot of use cases for MCP, it's really
• 29:12
  just data retrieval. And you find that it's actually
• 29:16
  very repetitive to write these tools sometimes because you have
• 29:19
  get user list users, get users by the, you know, you kind of
• 29:22
  end up with like a very repetitive code and and and and
• 29:27
  just a very kind of annoying to write code base and.
• 29:29
  Bloated. And it's bloated.
• 29:32
  But if you do it from the data first to define the schema first
• 29:37
  and then kind of attach the MCP stuff on top, you end up of MCP.
• 29:42
  It's still again becomes a set of tools, It becomes an MCP
• 29:44
  server, but it is data first. You start with the data and you
• 29:48
  define the MCP server from data first.
• 29:52
  That becomes tools as opposed to tool 1st and then trying to kind
• 29:55
  of map that to data. So the data first, as in you
• 29:59
  understand the data, you see what the data is almost like
• 30:03
  trying to tell you. And going back to this, maybe we
• 30:05
  can use the store example, online store as a perfect
• 30:09
  example where you've got different SKUs, you've got
• 30:11
  descriptions, but you've also got numbers and orders and all
• 30:15
  of that. You see that you also are
• 30:19
  embedding all of these descriptions or the SKUs or?
• 30:23
  You could embed things, but no, a lot of it is just like gets
• 30:28
  because that's The thing is like it's structured data.
• 30:30
  Not everything has to be like I've seen people do things like
• 30:33
  they have they put into Jason that they do like a nearest,
• 30:35
  like a like a rag on top of Jason documents, which is
• 30:39
  doesn't work for a variety of reasons.
• 30:41
  It's also just not natively how you would typically want an
• 30:44
  agent to access. That kind of data search has its
• 30:47
  place, and it's extremely powerful for agents, but it's
• 30:49
  not the answer to everything. I think we've gotten to the
• 30:52
  point where we have a hammer and everything looks like a nail.
• 30:55
  So it's not just embedding things.
• 30:58
  It's more like an ORM, if anyone's ever like, used an ORM.
• 31:03
  In fact, like Enrich MCP supports Sequel Alchemy
• 31:05
  directly. If you have a Sequel Alchemy
• 31:09
  ORM, you can actually just turn that into an enriched MCP
• 31:11
  server. It's an RM.
• 31:14
  If someone's not familiar is you would have your you would define
• 31:21
  your database schema as classes in Python.
• 31:24
  OK, and then you define what each field is and like the type
• 31:28
  and like the indices that you'd have on it.
• 31:30
  And then rather than writing SQL queries yourself, you kind of
• 31:34
  interact with it as you would in Python.
• 31:36
  And it under the hood, it always maps down the SQL queries, but
• 31:40
  when you're in the Python layer, it feels way more natural to
• 31:43
  work of classes and data objects and so inert, just kind of the
• 31:47
  same idea. It's much more natural for MCP,
• 31:50
  which is really just data retrieval, to think at the
• 31:53
  classes and data retrieval function layer than the writing.
• 31:58
  You know, tools that just run ad hoc queries or or whatever.
• 32:03
  One thing that I saw in the talks yesterday was that Jason
• 32:10
  consumes a ton of tokens and when you're trying to search or
• 32:13
  you're giving agents tools to search or just you're giving
• 32:16
  them access to data. Donae was talking about how they
• 32:21
  switched to CSV and it consumes way less tokens and therefore is
• 32:26
  faster and cheaper and all the benefits, right?
• 32:30
  In this case though, I'm I'm not sure how you would look at that.
• 32:36
  It wouldn't even be that you're giving the data like that to the
• 32:40
  agent, or you would. Could, but it doesn't.
• 32:43
  It's like what some people do that they'll dump an entire CSV
• 32:46
  to the agent. Yeah, like I've seen this
• 32:48
  before. People build this giant object
• 32:50
  and they'll just dump all of because they don't know how to
• 32:52
  make the agent go find structure data.
• 32:54
  So they'll just dump all the structure data that they think
• 32:57
  is relevant just into the prompt itself.
• 33:01
  And then in that case it becomes really important to like
• 33:03
  optimize those sort of things. But my sense has always been
• 33:09
  that rather than one shot queries where you just go and
• 33:14
  grab as much context as possible based on some query and just
• 33:17
  throw it into the context window.
• 33:19
  The the correct way to build agents for a lot of use cases is
• 33:22
  actually don't do that. Let give the agent an interface
• 33:26
  that window to go find the data it needs when it needs it.
• 33:30
  Like the agent is plenty intelligent, right?
• 33:33
  And nowadays LLMS are, are, are so smart and it's very rare.
• 33:38
  Like you're very unlikely to have a use case that you're
• 33:41
  building an agent for where the agent isn't smart enough.
• 33:44
  The issue has is almost always. The system.
• 33:47
  The context, yeah, it doesn't have the right information or
• 33:51
  data to be able to do something smart, even code generation
• 33:54
  right. Like a lot of people complain
• 33:56
  like, oh, like it, you know, uses too many try catches and we
• 33:58
  don't like doing that or, or our our style is different.
• 34:02
  And it's not like the agent's wrong.
• 34:04
  You know, the code that generates is likely good and,
• 34:07
  and and fine, but it doesn't have all of the the context
• 34:12
  about how you like your style guide and all these things that
• 34:14
  it needs to do well, or it's like, oh, it didn't, you know,
• 34:16
  it made-up a function or API that doesn't exist.
• 34:19
  Hallucinated that. And my sense is like, well, it's
• 34:22
  probably, it's like pretty much scheming your code and it's
• 34:24
  guessing what functions should be there.
• 34:27
  Chances are that function probably should be in your API
• 34:30
  if it hallucinated it. And if it can't be in your API,
• 34:34
  then that's more of, again, it's a context thing.
• 34:37
  There's some stuff that's missing.
• 34:38
  It doesn't really understand. It's not like if you didn't give
• 34:40
  it enough information it wouldn't be able to reason about
• 34:44
  your stuff. It just doesn't have that.
• 34:48
  And some of the problem is like, how do you get that to it and
• 34:50
  the way that's efficient without just dumping your entire code
• 34:52
  base in the context. Yeah.
• 34:54
  So when you say, hey, that function doesn't exist, you
• 34:57
  hallucinated it, it responds and says you're welcome.
• 35:00
  Yeah, well, I'm serious. Like I do this like sometimes it
• 35:03
  will hallucinate the function and I'll be like, oh, actually,
• 35:06
  you know, that function is actually probably fine having on
• 35:08
  a library. I just didn't think of it or
• 35:10
  didn't bother implementing it and I was be like go make it and
• 35:12
  and like. That's totally fine.
• 35:14
  Yeah, no mistakes. Don't mess up.
• 35:18
  OK, so I think there's a fascinating thread to pull on
• 35:22
  here with the ways that you're using search right and the ways
• 35:27
  that you're exposing the context, the right context at
• 35:30
  the right time. We heard it last night a bunch.
• 35:34
  How you, just because you have access, that doesn't mean you
• 35:40
  have like intelligence to the right type of answers.
• 35:46
  And I think you're, you're hitting on that point.
• 35:49
  And that's why Rag was a disaster, because it was like,
• 35:53
  Oh, well, we'll just embed everything, throw it in the
• 35:55
  vector store, and then we give access to the vector store and
• 36:00
  we give the LLM access to it and it's off to the races.
• 36:04
  But then you're like, OK, how much do we chunk?
• 36:07
  Where do we break the chunks? And it starts getting really
• 36:10
  confusing. And that's exactly why is that
• 36:13
  we're, we need to like almost dynamically be doing this.
• 36:16
  And instead of saying Rag is the solution, let's just give agents
• 36:22
  the search tool when they need it.
• 36:25
  Yeah, if they need it. And then we can also give them
• 36:28
  other tools that are going to be valuable at the same time.
• 36:33
  The the first iteration of Rag where alums that didn't run in a
• 36:36
  loop, which means that they weren't really able to like make
• 36:39
  tool calls and and and do all the things that you might want
• 36:42
  it to do what they would. So what you would do if you have
• 36:48
  one shot 1 pass is you're like, well, OK, I need to just put
• 36:52
  everything possible into this prompt.
• 36:54
  And so given that problem statement, brag was a good way
• 36:59
  to solve that. But now we're past that.
• 37:03
  Like we can really trust. I mean, yesterday when we were
• 37:06
  at the conference and there's some point where I needed to
• 37:09
  move from one library to another library and I literally let it
• 37:12
  run and it ran for 10 minutes and it came back and it had all
• 37:16
  my tests passing. I was shocked that this thing
• 37:19
  ran for 10 minutes and did it all for me and got it all
• 37:22
  working. So I mean, if you get an agent
• 37:25
  to run the loop, actually, I heard this thing from Anthropic
• 37:28
  where they're saying that the metric for tracking nowadays is
• 37:31
  more about how how long can you trust an agent to work
• 37:35
  independently, autonomously, without, you know, a human
• 37:39
  intervening and still making forward progress.
• 37:42
  What were they calling it? Yeah, last night they were
• 37:44
  talking about it was like interruption, amount of
• 37:48
  interruptions or something like that.
• 37:49
  And so if you interrupt it, then that trust and that metric is
• 37:55
  lower, right? Yes, I think I actually didn't
• 37:57
  see that talk yesterday, but I I think that it's all referencing
• 38:00
  the same thing. Yeah, that's a way to to track
• 38:02
  that is that they killed the job and they killed the job.
• 38:05
  That means that it started doing it circling off the rails or it
• 38:08
  just was got confused. And so that's a metric to track.
• 38:12
  And then if you're tracking that metric, they're saying it's
• 38:15
  doubling. I forget what they said.
• 38:16
  If it was every six months or every year, doesn't really
• 38:18
  matter. The fact that it's doubling is
• 38:20
  nuts because let's say today, you know.
• 38:23
  It's 10 minutes. Yeah, a few minutes, like 5
• 38:25
  minutes is what I got, yes. Or I got 10, but let's just say
• 38:28
  it's 5 minutes, OK, that's fine. But if we're at 30 minutes in a
• 38:33
  few years, like how different is that going to be?
• 38:38
  So and when you start thinking like, OK, well, I need something
• 38:42
  to run for 30 minutes, it's not like you can just leave it with
• 38:46
  thoughts for 30 minutes and it's just going to do something
• 38:49
  really smart. You kind of need to give it a
• 38:52
  lot of access to data, the Internet, all these things like
• 38:57
  how does pro work? Like the way Pro is so smart is
• 38:59
  that like like GPT Pro is that if you watch what it's doing,
• 39:02
  it's really just going and like searching the web.
• 39:05
  If you didn't let search the web and just sat there and thought
• 39:07
  and kind of in a loop, I might do better, but I just don't
• 39:10
  really think it would be a lot better.
• 39:13
  Yeah, it wouldn't be that useful.
• 39:14
  Yeah, it's, it's probably, it's more powerful to just give it
• 39:20
  access to. It's just context.
• 39:21
  It's all. I mean, it's, it's so funny
• 39:23
  because like we say the word so much, it almost sounds like it
• 39:26
  almost like doesn't sound like a real word anymore.
• 39:28
  But it's just all that matters is context.
• 39:30
  And if you can build proper context, you can get it to work.
• 39:35
  And especially now we know people are thinking about small
• 39:37
  models. There's a lot of things people
• 39:38
  are thinking about, but that will always be true.
• 39:42
  The better context you have, the better results you're going to
• 39:44
  end up with. So talk to me about MC pay.
• 39:49
  I would love to talk to you about M CP/M CP is like.
• 39:53
  MC Pay. MC pay is a is a dirty word in
• 39:56
  in in the in in the LinkedIn world, I think now.
• 40:00
  That's whenever I see you online, whether it's on X or
• 40:03
  LinkedIn, people talking about MCPI just insert that voice and
• 40:08
  I remember like, yeah, this is the type word of the moment.
• 40:13
  So MC pay it is. You know, The funny thing about
• 40:19
  MCP is really all it is, is like you're moving tools remotely and
• 40:23
  why that matters? I think the thing that people
• 40:25
  miss is like, oh, this thing's so simple.
• 40:26
  I already had tools, whatever. Like why are we doing this new
• 40:29
  MCP thing? We already had REST.
• 40:31
  The idea is, is this one like just tool discovery?
• 40:34
  All this, There's like all these concepts that exist in MCP but
• 40:38
  don't exist in REST. Some people would argue, well,
• 40:40
  couldn't we just build us on top of REST?
• 40:42
  The answer is probably, but for better or worse, we've all kind
• 40:46
  of standardized around MCP and that's important because getting
• 40:50
  entropic and open AI to agree on something a standard in having
• 40:56
  all the enterprise companies kind of come and accept it as as
• 40:59
  as the way forward, that rarely happens.
• 41:02
  So that's a big deal and what it's doing and it's almost this
• 41:06
  is it's such a minor change in some ways if it doesn't worth
• 41:09
  putting the tools in a in a server that that's the only way
• 41:13
  you could actually make something so monumental happen
• 41:16
  on the standards what MCP is actually doing.
• 41:20
  Let's think about what it actually means at scale, at
• 41:25
  enterprise scale to do MCP. Well, what it turns into is
• 41:31
  you're separating the agent from the context layer essentially
• 41:36
  and you are enabling teams to build and maintain in parallel
• 41:41
  MCP servers, which are really again the state access or tool
• 41:44
  access that any agent can come and connect to and get value
• 41:48
  from. So it's actually like an
• 41:50
  engineering. It's almost like microservices.
• 41:52
  Yeah, it's almost like databases like you now have access to my
• 41:56
  database. Or in another way, you could
• 41:59
  look at it as my knowledge, like here's my knowledge base, I can
• 42:03
  just have this for you. Yeah, I mean, yes.
• 42:06
  And and there's other piece of it which is like we how did we
• 42:09
  just before, well, we just put these tools directly into the
• 42:11
  agent. The problem is like
• 42:12
  organizationally that doesn't scale.
• 42:14
  No, like if you're doing many agents, you don't really want,
• 42:17
  you know, to be installing libraries or maintaining
• 42:19
  libraries and then trying to like kind of fit these into
• 42:21
  every single agent separately. Yeah, it's much cleaner to just
• 42:24
  say here connect to these five things that you have access to
• 42:27
  and we start thinking of off and discovery and all these other
• 42:29
  things. It, it just becomes a lot more
• 42:33
  scalable. It's suppose we have a large
• 42:36
  team to have all of these MCPN points and even in a small team,
• 42:42
  it's just like you're just creating a separation of
• 42:46
  concerns. There's the agent.
• 42:48
  How I think of it is like where's the agent?
• 42:50
  And like a land graph or whatever you use, there's kind
• 42:53
  of the, the agent framework, which is really explaining how
• 42:56
  the agents, what should be doing, which agents exist, how
• 42:59
  they should be thinking and what they should be connecting to.
• 43:02
  There's the context engine, which is again, providing the
• 43:04
  window to the data, which is typically backed by AP is data,
• 43:08
  maybe other MCPS. And that's, that's again, the
• 43:09
  thing that we're inventing and, and, and, and, and, and have at
• 43:14
  Redis. And then on the other side,
• 43:19
  there's the LM itself. And so those are the three
• 43:21
  components of an ejectic system where the agent workflow, the
• 43:25
  context workflow, and then the LM workflow and.
• 43:29
  That's it. So that's it.
• 43:31
  I mean, that's all it is. Like all an engine is, is it's
• 43:33
  a, it's a LLM running in some sort of like scheduled loop or
• 43:39
  orchestrated loop that gets data it needs to be able to solve
• 43:44
  hard problems. So it's really just taking this
• 43:47
  powerful thing, which is an LLM and turning it into actual
• 43:50
  enterprise solutions. To do that, you need to give
• 43:53
  access to ability to do things and ability to learn things.
• 43:57
  And then you need something to like.
• 43:59
  So you have multiple agents, multiple tasks, you just need to
• 44:01
  give it a system prompt. Perhaps you have a few different
• 44:04
  agents that create separation of concerns there, which is really
• 44:06
  interesting, but maybe conversation for another time.
• 44:10
  But that's it. It really is that simple.
• 44:13
  And so if you separate the MCP layer or just a context layer
• 44:19
  and endpoints and AP is away from the agent layer, it makes
• 44:24
  it way easier to be able to kind of iterate on these things
• 44:29
  separately each other. It's not like, oh, we added a
• 44:31
  new API, we want to change this. I need to go like make APR to
• 44:34
  every single agent or change this library and have all the
• 44:36
  agents update to this new. It's just much cleaner to just
• 44:40
  maintain the service and just say connect to this thing and we
• 44:43
  will make sure it's up to date and it provides these tools or
• 44:46
  access to this kind of data. Yeah, yeah.
• 44:48
  Like like APIs or like even just like separating storage and
• 44:52
  compute? Yeah, it is another way to think
• 44:55
  of it like you're separating storage and compute.
• 44:57
  You're separating the action, the intelligence, and the
• 45:00
  context. I like that.
• 45:01
  Yes. And and I think that's that's
• 45:05
  that's how you should build agents.
• 45:07
  And I think we're just handling the context layer amongst other
• 45:10
  things, but the context layer is personally what I'm I'm thinking
• 45:12
  about. We're good.
• 45:15
  I think we're good. I'm Sepe.
• 45:26
  I'm say I'm also not surprised you have multiple voices in your
• 45:30
  head knowing you as well as I do.