Latest / Agentic Conversations (formally mlops.community) / Real time features, AI search, Agentic similarities

Transcript

Real time features, AI search, Agentic similarities · Agentic Conversations (formally mlops.community)

• 0:25
  Feature platform is like how it started out, got like all kinds
• 0:29
  of usages. Now generally anything that goes
• 0:32
  into online metrics or rule engines are into context, not
• 0:36
  like this company's like Airbnb Drive.
• 0:38
  So the feature platform idea, I know there's like a few
• 0:43
  different ways of explaining it. How do you guys explain like
• 0:48
  what the feature platform is? Warren does a great job.
• 0:53
  He's the pro I'll try. Yeah, yeah, I know.
• 0:56
  I think there are different definitions of it.
• 0:57
  And I think the first thing I like to say when it comes to
• 1:00
  Cronon, one thing that Cronon does differently is that it, you
• 1:03
  know, it does the compute. So there's like feature stories
• 1:06
  that leave the compute onus on the users and then they'll like
• 1:09
  do storage more as like a classic KV store with some stuff
• 1:12
  around it. But what we saw and the reason
• 1:14
  why we built Cronon is that compute was really the hard
• 1:17
  thing that people struggled with.
• 1:18
  Like if you want streaming features, if you want to be
• 1:20
  doing streaming aggregations, if you want to be doing online
• 1:22
  serving and offline training of, you know, offline training and
• 1:25
  have consistency between those two things, you've got to be
• 1:27
  computing your features a very specific way.
• 1:30
  So for us, what the feature platform is, is takes raw data
• 1:34
  in, produces features, serves them online, serves them offline
• 1:38
  for online for inference, serves them offline for training, does
• 1:41
  both of those things in a very scalable way and exposes a very
• 1:45
  easy API to the user to do that to power those flows.
• 1:49
  And so, you know, that's the scope that we take a feature
• 1:51
  platform. And I think that's what made
• 1:52
  Cronon unique, and I think that's why kind of the open
• 1:54
  sources gotten some good adoption is because it kind of
• 1:58
  hits on the main pinpoint that people face.
• 2:00
  Yeah, that, and that was my big question, because I I know that
• 2:04
  there's the feature stores that do the compute.
• 2:08
  They don't do the compute. They don't even store anything.
• 2:11
  They just will help you orchestrate things.
• 2:14
  And so you have all of those different iterations of it.
• 2:18
  It came from Airbnb. What made you want to build it
• 2:21
  there? When I joined Airbnb there was
• 2:24
  already the earliest version of Kronon called zip line that like
• 2:27
  Warrant and Brad the other guy built.
• 2:30
  I remember reading that blog too, by the way, and I was
• 2:32
  looking for, there was no author on the blog and I was, I wanted
• 2:37
  to bring you guys on and I was so pissed because I'm like, who
• 2:42
  created this? And me being in Germany, I
• 2:45
  couldn't just like ask anybody, you know?
• 2:48
  So I was going nuts because I wanted to have you on the
• 2:52
  podcast, or at least on back in those days in 2020, we were
• 2:55
  doing like the virtual meet ups. Oh yeah.
• 2:57
  And it was like, this is a great blog.
• 2:59
  I want to know more about Zip Line because Zip Line did
• 3:01
  everything right. It wasn't just the feature
• 3:03
  store. Yeah, yeah.
• 3:04
  So when I joined, I think the very first meeting I was in was
• 3:07
  like, there was this like fraud team sitting in the room.
• 3:10
  And they're like, hey, we need to be able to engineer features
• 3:13
  and test them out in fraud very quickly because the nature of
• 3:16
  fraud is adversarial, right? Like someone figures out a way
• 3:18
  to overwhelm the systems and like, bleed money.
• 3:22
  And that was a big problem back then at Airbnb.
• 3:24
  And that was a big focus. So that's what like Crononos,
• 3:28
  the early versions of Cronon were built to solve that.
• 3:30
  Like to make it? All fraud.
• 3:32
  Yeah, yeah. People think of Airbnb as a
• 3:35
  travel company, but the fact of the matter is it moves a lot of
• 3:37
  money across a lot of borders. So it looks like, I mean,
• 3:40
  there's a whole payments company within Airbnb, which is kind of
• 3:42
  from the outside. You wouldn't necessarily think
• 3:44
  that. And so all those payments
• 3:46
  challenges that people have around payments fraud and trust
• 3:48
  and all those things, you know, I really have those exact same
• 3:51
  challenges. Basically that was the nature of
• 3:53
  the beast where, you know, they needed to be in a position where
• 3:57
  they can like fight fraud in a matter of hours, right?
• 4:01
  But people were trying to do a bunch of things there.
• 4:06
  They have to figure out the batch processing side of it or
• 4:09
  the spark. Then they have to like figure
• 4:11
  out the stream processing side of it.
• 4:12
  Then they figure out, you know, how to index it correctly so
• 4:15
  that they can serve it, you know, at scale.
• 4:17
  Then they have to figure out how to orchestrate it, you know,
• 4:19
  using airflow. You know, it's pretty mind
• 4:22
  bending just to get one signal all the way through to the
• 4:25
  application. And they have to do it with
• 4:27
  hundreds of signals, right? So imagine like the complexity
• 4:31
  and the number of components that would be involved.
• 4:33
  So and when you say signal, it's like basically a pipeline.
• 4:37
  Yeah, it's like a piece of information, like how many times
• 4:40
  a user used this credit card in the past, like use this IP in
• 4:43
  the past X days. It's a really simple piece of
• 4:46
  information, but like to engineer this to the application
• 4:50
  and to the model training process, they have to like write
• 4:53
  massive pipelines. Yeah, right.
• 4:55
  And hundreds of signals is like the lower end of things.
• 4:58
  It's usually thousands of signals.
• 5:00
  And yeah, yeah, these models are very greedy.
• 5:03
  Oh yeah. They need all the signals they
• 5:05
  can get. They get better, you know, the
• 5:07
  more you give it, the better it. Gets that then became Cronon.
• 5:11
  That then became Cronon. Yeah, and you bolted it on to
• 5:13
  zip line? We're, I mean we're largely
• 5:16
  taking Cronon and building zip line around it so.
• 5:19
  Basically the story is that you you built Cronon, you spun that
• 5:24
  out or you open sourced it from Airbnb, which I feel like Airbnb
• 5:30
  has open sourced other stuff in the past.
• 5:32
  Yeah, Airflow. Airflow.
• 5:34
  That's I knew there was some big data thing that came out of
• 5:38
  Airflow. Superset also.
• 5:39
  Oh yeah, Superset from both from Maxine.
• 5:42
  Yeah, both from. Maxine Yeah.
• 5:43
  So just declare that Cronon open source was actually in
• 5:45
  collaboration with Stripe. So before we open sourced, we
• 5:48
  had like a close partnership with Stripe, again another
• 5:51
  payments company basically and we collaborated with them on
• 5:55
  their repository for 6-8 months or so before going open source.
• 6:00
  So it's a jointly managed between the two companies.
• 6:03
  Well, it makes sense if it's very heavy on the fraud part
• 6:07
  that Stripe would be interested. Yeah, yeah, yeah.
• 6:10
  And then actually the journey with an Airbnb after payments is
• 6:13
  kind of interesting because search was the next big one that
• 6:15
  came along, right? So search personalization is
• 6:17
  another use case that's very sensitive to real time data, has
• 6:21
  very, very high volume and can move the needle on the business.
• 6:25
  So it's like worth pushing that forward if you can.
• 6:28
  And so that was kind of like the next big partnership that we
• 6:31
  had. And what was interesting is that
• 6:32
  once we were kind of working at search scale, we saw other
• 6:35
  interesting use cases within the company.
• 6:36
  You just start organically popping up.
• 6:38
  So like if you go to Airbnb and you see any listing, you'll see
• 6:41
  like a 4.8 stars average or whatever that's computed on
• 6:44
  Chronon because that team basically wanted that to be real
• 6:47
  time. You know, the same way that
• 6:48
  fraud and search want their signals to be real time and they
• 6:51
  wanted it to be like a windowed aggregation so that, you know,
• 6:53
  really, really old things fall out of the window and you see a
• 6:55
  more recent up to date view of that data to the user.
• 6:59
  And so it's the exact same data engineering challenge that ML
• 7:02
  faces, but just as a product facing, you know, metric.
• 7:05
  So they were like, well, Cronon's a great tool for this.
• 7:08
  And it like simplified their life from running this crazy
• 7:10
  Flink job to simply saying, hey, here's my data source.
• 7:13
  I want an average over this time window.
• 7:15
  And I want to, you know, plug it into this thing here.
• 7:17
  And it went from months to like a couple days to get that up and
• 7:20
  running. How did they find out about
• 7:21
  Cronon? They just were hearing others
• 7:24
  use it because I imagine things can get lost in an organization
• 7:27
  that big that. Was very interesting for us as
• 7:29
  Airbnb got bigger we no longer talk to everyone.
• 7:31
  Yeah, I believe it. Yeah, word gets out.
• 7:34
  I mean, you know, you struggle with the data engineering stuff
• 7:36
  enough and word gets out like, hey, there's this thing that'll
• 7:38
  make your life easier. So that was really cool for us
• 7:40
  to see we were happy with. That I think the search Usk
• 7:44
  succeeding played a big role in them finding us.
• 7:47
  OK. Because it was such a tough Usk
• 7:49
  that Airbnb like a lot of people thought like it wouldn't work on
• 7:52
  Kronan like very easily. But the fact that it worked like
• 7:56
  was very surprising and very impactful inside Airbnb.
• 8:00
  So and Holmes was very or like the team that did that US case
• 8:04
  was very close to search. Yeah, OK.
• 8:07
  Search is so funny to me because a lot of the stuff that we're
• 8:11
  doing now with AI and all of the agents or RAG, it's like this is
• 8:19
  a search problem. Even the context engineering
• 8:22
  now, it's like that feels like there's a lot of search going on
• 8:25
  here and you got to be good at search in some way, shape or
• 8:29
  form. Yeah, such as one of those
• 8:32
  things like where the number of signals is like in the order of
• 8:35
  thousands and search ends to be very complex, right.
• 8:38
  Some of these things include sequence modelling, LLM's like
• 8:44
  typical vector indexing for embeddings, all these kinds of
• 8:48
  things in like 1 monolithic system, right?
• 8:51
  The more sophisticated, you know, search setups look look
• 8:55
  like that and data engineering for that means like at least you
• 9:00
  know, 200 or 300 mini pipelines all like together pushing data
• 9:05
  into vector search. Yeah.
• 9:07
  Alright. So back to the story, you went,
• 9:10
  you open sourced Cronon with Stripe and then you were like, I
• 9:16
  guess it's time that we start a company and go all in on this,
• 9:20
  huh? So we, we saw Stripe use it and
• 9:23
  it wasn't very easy to get set up with Cronon and they already
• 9:30
  had expertise with Flink and Spark and all of these other
• 9:33
  technologies and it was still hard for them.
• 9:36
  So you're like, OK, if someone wants to use it, there needs to
• 9:38
  be a lot more work. And like, if someone was not
• 9:41
  familiar with any of these big data technologies, it's going to
• 9:44
  be very hard for them to like use Chronon at the end of the
• 9:48
  day, like Chronon does a good job of abstracting these
• 9:51
  technologies away from the user, right?
• 9:53
  Users write SQL like stuff and under the hood all of this is
• 9:57
  happening like the Sparks link and big table and like all of
• 10:01
  these technologies come up like even airflow, right?
• 10:03
  They don't have to even worry about Airflow like any of these
• 10:06
  technologies. They just like write their
• 10:08
  queries and these things happen automatically.
• 10:11
  It does that job. But like to set it up, you need
• 10:13
  to know how all of those things of.
• 10:15
  Course, because before you abstract it away, you've got to
• 10:18
  have a deep understanding of it, yeah.
• 10:20
  Basically it's a technology that can help a ton of different
• 10:23
  teams out there, we think. But right now it's like, you
• 10:25
  know, there's a lot of need to the role of the business partly
• 10:28
  is to is to productize it and make it, you know, very easy to
• 10:31
  adopt. And yeah, I mean, I think it's
• 10:34
  great that it has like this open core around it so that, you
• 10:37
  know, teams can adopt it with us.
• 10:38
  But then they have that like open source safety there, which
• 10:42
  means like no vendor lock in and kind of that that sort of easier
• 10:47
  to work with business model for larger enterprises.
• 10:49
  Thought you guys were crazy when you came out and you were doing
• 10:53
  this. I was like, but Feast is out
• 10:57
  there, which is kind of similar space.
• 11:01
  And I'm not saying you're crazy because you're competing with
• 11:03
  Feast, but because of what happened with Feast where it
• 11:08
  felt like it got bastardized a little bit and then and then
• 11:13
  also feather, same thing like LinkedIn release feather and
• 11:18
  that also got bastardized. So, so we did this, we did this
• 11:21
  exercise with an Airbnb, right? Because we don't want to just be
• 11:24
  open sourcing anything for any reason.
• 11:26
  We were like, OK, what you know, what is this doing in the
• 11:28
  market? What, what graph is this
• 11:30
  solving? And really what it came down to
• 11:32
  is kind of what we started off with, which is the compute
• 11:34
  stuff, right? The compute, the hardest part of
• 11:36
  the actual data engineering challenge for AI and ML FISA is
• 11:41
  an example of a something that adds a lot of value, like kind
• 11:43
  of downstream of that, but it doesn't solve that.
• 11:46
  That main problem and never try to feather is really
• 11:49
  interesting. If you look at the documentation
• 11:51
  for it, it talks about all those problems.
• 11:53
  But then if you go and look at the implementation, there's a
• 11:55
  lot of to do's that never really got done like streaming, like
• 11:58
  streaming aggregation, windowed aggregation.
• 12:00
  A lot of the compute stuff is where we were like, oh man, once
• 12:03
  you look a level deeper, it's like you kind of see that they
• 12:06
  never quite got there. And so we were like, OK, we are
• 12:08
  the only ones that have cracked this nut.
• 12:10
  And so we need to be out there and, and this is a very valuable
• 12:13
  sort of contribution to the community.
• 12:15
  Yeah, I guess Feather, they released it and they wanted to
• 12:18
  see if there would be any uptick and people using it and how that
• 12:23
  would and when they I I just don't think they got what they
• 12:26
  were looking for. Yeah, I think you had the right
• 12:27
  vision. I mean, you read that
• 12:28
  documentation and I think they hit the nail on the head on a
• 12:30
  lot of stuff. But I think when they got out
• 12:32
  there, there just wasn't enough of it there working really well.
• 12:35
  I mean, it took us a long time. To it took.
• 12:37
  Us really well, it's hard. I mean I see I see why it never
• 12:40
  got it was a grind for many, many years and then I think the
• 12:44
  opportunity to work with Stripe before getting out there into
• 12:46
  the open source gave us a chance to like really battle test it at
• 12:50
  a whole another company at scale and like iron out all these
• 12:53
  things. And so by the time we went open
• 12:54
  source, like the dream was real as opposed to I think, I think
• 12:58
  Feather had the vision, but I think by the time it got out
• 13:00
  there from what we saw, the implementation wasn't quite
• 13:03
  there. But yeah, I mean, I don't need
• 13:04
  to talk about about any of these tools like they.
• 13:06
  No, I don't think you're talking bad.
• 13:08
  If anybody's talking about it, it's me calling Ambassador Jon
• 13:11
  Snow over there. Yeah.
• 13:17
  They're all like, they're all very interesting technology.
• 13:18
  I think, I think we're unique in that like we took our like we
• 13:21
  took the time to like really make sure we nailed the hardest
• 13:24
  part of the problem. And I think that's what was
• 13:26
  unique about our open source. And so why was it so hard to get
• 13:31
  that working? There is a series of problems
• 13:35
  right that I think like, but I'll maybe talk about the two
• 13:39
  main ones I suppose. One is generating training data
• 13:44
  that is point in time, correct? Right.
• 13:47
  So people already have a series of like observations and labels
• 13:51
  associated with those observations.
• 13:53
  Like I sparked my card, like we're down the line, you know,
• 13:56
  someone said this is fraud and, you know, clawed money back,
• 13:58
  throw a charge back or whatever. So they have these observations,
• 14:01
  logs of observations. All companies have these and
• 14:05
  they're like, OK, when these observations happen, we need to
• 14:07
  create features at those points in time.
• 14:10
  So when I say point in time correct training set, like I
• 14:12
  have a new feature idea and I need to create those features at
• 14:15
  the observation point in history, that is an N cube
• 14:21
  problem essentially if you do it in my SQL way, right?
• 14:25
  If you write a SQL query saying like give me all the raw data at
• 14:29
  that point and then aggregate for every one of those
• 14:32
  observations, that blows up very quickly.
• 14:35
  And that's what people are doing.
• 14:36
  And they're like, OK, this is blowing up and I'm not going to
• 14:39
  figure out how to make this fast.
• 14:40
  I'll just use batch features. That's true.
• 14:43
  That's why real time is so hard. So.
• 14:46
  Hard. And if you put yourself into the
• 14:47
  user's shoes, right, it's like, OK, they're like I understand my
• 14:50
  data and anytime the transformation I want to run.
• 14:52
  And then it's like to express that you have to learn like
• 14:54
  Flink, Spark, I mean all the stuff that people are saying
• 14:56
  because like those technologies fundamentally were not built for
• 14:59
  these use cases. It's like it was really like if
• 15:01
  you look at the history of batch like OLOLAP, it's like it's
• 15:04
  pretty much BII mean it's like Spark and our Hive for or you
• 15:07
  know, MapReduce and Hive and Spark, all these things kind of
• 15:10
  are just evolutions of BI powering pipelines and.
• 15:14
  They're more good for, like, you know, every day I need to
• 15:16
  generate a report that someone's going to look at, right?
• 15:18
  For the, you know, executives and stuff, and now it's like the
• 15:22
  actual workflows for AI and ML, they cross these boundaries of
• 15:25
  streaming and batch of online serving, of inference and
• 15:27
  training. It's like they cross all these
• 15:29
  boundaries and no tool really took a step back and addressed
• 15:33
  that sort of user need in a first class way.
• 15:36
  Everyone started ducting stuff together and then that's where
• 15:39
  all these like nightmare data engineering projects come from.
• 15:41
  And I think, yeah, one thing unique about us is that we took
• 15:44
  took that step back and we're like, OK, what if we exposed the
• 15:47
  abstraction that the user's actually trying to do?
• 15:49
  Just here's the data, here's the transformation, I need it
• 15:51
  offline and display for model training and I need it online
• 15:54
  with these very strict requirements for a low latency
• 15:56
  inference. And like, what if we try to make
• 15:58
  that magic? Yeah.
• 16:00
  And. And yeah, I think it's hard, it
• 16:04
  turns out. It turns out there was a reason
• 16:06
  why nobody had done it. People have tried it.
• 16:09
  We have seen like many, many. There's A at.
• 16:11
  This problem. There's a graveyard full of
• 16:13
  people who have tried it. But Airbnb was a wonderful place
• 16:16
  for us to build it because it gave us it gave us what we
• 16:19
  needed to try it a few times and make some, like mistakes that I
• 16:22
  think if we were like, if we had turned back the clock seven
• 16:24
  years and started that startup, you know, from then, I don't
• 16:27
  think we would have had the leeway to make the mistakes that
• 16:29
  we needed to make to learn how to do this properly.
• 16:32
  So I think because we had like Airbnb as a place to get this
• 16:34
  right and then and Stripe as well as a place to get this
• 16:37
  right, that's what let us come out the gates was something that
• 16:39
  worked. Yeah.
• 16:40
  And that was huge for us. And it kind of felt like you
• 16:43
  could battle test it, but you could also recognize if it works
• 16:48
  in two companies that are very different, we could probably
• 16:52
  make it work in more and that's. Huge right and, and having the
• 16:55
  flexibility to do that was like that we were yeah, we were in a
• 17:00
  very unique place in time to to be able to pull that off.
• 17:04
  And Airbnb, you know, of course has the kind of like culture to
• 17:08
  to like excellent engineering culture to kind of support that
• 17:11
  and, and, and foster that and see the value in that and invest
• 17:13
  in those like long term projects, which is really rare.
• 17:17
  And. So now what are you thinking
• 17:19
  about now? It's the whole data life cycle,
• 17:23
  right? It's not just the feature part.
• 17:26
  Yeah. Oh yeah.
• 17:27
  Because I should preface this by you won the pitch competition
• 17:30
  for San Francisco. Congratulations.
• 17:33
  As promised, we are on a podcast.
• 17:37
  I don't have most pitch competitions.
• 17:39
  They give you like 100 grand. All this very fancy stuff your.
• 17:43
  Money. I would take this.
• 17:46
  You get a podcast, my friend. Let it be known Demetrius pays
• 17:49
  his debts. Yes, exactly.
• 17:53
  Yeah, Yeah. What are we?
• 17:55
  So what are we investing? I mean, it's very interesting.
• 17:56
  Now we're out there, we're working with customers, we're
• 17:58
  seeing what the actual needs are on the ground and we're seeing
• 18:00
  sort of where the main pain points are and, and, and how to,
• 18:03
  you know, get the most out of this technology that we started
• 18:05
  building. A couple of things are standing
• 18:07
  out. One is definitely governance and
• 18:09
  just, you know, like data privacy, data governance, having
• 18:14
  confidence in like these systems that are serving the most
• 18:18
  critical use cases at these companies.
• 18:20
  And so we're thinking a lot about that and how to make that
• 18:22
  part of the platform. And the other one is, yeah, I
• 18:25
  mean, we've seen people kind of still figuring out more
• 18:29
  foundational pieces of of data infrastructure.
• 18:31
  A lot of teams are trying to move to iceberg.
• 18:33
  A lot of teams are, you know, trying to move to open source,
• 18:35
  you know, sort of things from cloud specific offerings and
• 18:40
  we're thinking about that too. So we're kind of following where
• 18:44
  the actual needs are on the ground.
• 18:45
  We're very plugged into trying to be plugged into real value at
• 18:49
  real use cases, and those two things are standing out to us
• 18:52
  right now. It's interesting what you were
• 18:55
  talking about with Iceberg earlier, how everyone wants
• 18:59
  Iceberg but they're not sure how.
• 19:02
  Yeah. Yeah, for sure.
• 19:04
  It's hard. I mean, yeah.
• 19:06
  Where are they getting stuck? There's a lot, I mean series of
• 19:09
  choices, right? It's like you have an event bus
• 19:12
  or you have a database that's online and you need to wait to
• 19:15
  transfer that into your offline file system like SV or GCS.
• 19:20
  And from there, like you need to, once you have the iceberg
• 19:23
  files or whatever, you need a catalog that you could use,
• 19:27
  right? And depending on which cloud
• 19:29
  you're on, it catalog is not a solved problem, right?
• 19:32
  That's a very open problem. And that needs to play well with
• 19:34
  their existing data, right? If they're on Big Query, it
• 19:37
  needs to play well with their existing thing.
• 19:40
  And you need to have an engine that can like understand this
• 19:43
  Iceberg file well and work with the optimizations that Iceberg
• 19:47
  allows you to do, right? Iceberg has like a huge depth of
• 19:51
  like optimizations that you can like plug into which engines
• 19:55
  like Big Query do automatically under the hood, right?
• 19:58
  Just because you have some data in Iceberg doesn't mean you're.
• 20:01
  Getting the full benefit of having that data in ice work.
• 20:04
  Really. Yeah, it's not that
• 20:06
  straightforward. Like you need to be able to do
• 20:09
  clustering, do views, do all of these things that the defect
• 20:13
  offerings on things like Google Cloud simply don't support?
• 20:18
  So what do you have to do then? So people like try all kinds of
• 20:21
  things. They try Polaris, they try
• 20:23
  gravity and all this like burning problems.
• 20:24
  They write big their own catalog, right?
• 20:28
  And they like figure out how to get their data.
• 20:31
  They like move from Pubsub into GCS abro as it is and then like
• 20:36
  use some ETL to process that into par K and form iceberg out
• 20:41
  of it and then figure out this catalog thing and put it in
• 20:43
  there, right. How many people we don't think
• 20:46
  like a lot of people are successful with like doing this
• 20:48
  kind of a dance around their data infrastructure?
• 20:52
  So then is the idea to just be like, hey, potentially there is
• 20:56
  a world where you can hit zip line and you're instantly on
• 21:02
  iceberg and it is as easy as you would hope that it would be?
• 21:06
  Eventually we want to get there. So there is 2 levels to that.
• 21:09
  I think like 1 is like, I already have my data in pub sub,
• 21:13
  right? You take care of this dance of
• 21:15
  getting this into an iceberg and catalog and you take care of
• 21:18
  like leveraging the optimization, right?
• 21:21
  Then there are people who are like I have a service, I'm not
• 21:23
  logging anything. But if you, if you tell me if
• 21:26
  there is a library that I can log into, I will do it, right.
• 21:29
  So there is that level. Eventually we want to get there.
• 21:31
  This is a zip line library. You log it under the hood, we
• 21:35
  log it to Pubsub or Kafka and like create the Iceberg
• 21:37
  Warehouse and you can leverage it, right?
• 21:40
  So that's eventually where we want to get to.
• 21:42
  But first we are like trying to get to a point where you already
• 21:44
  have Pubsub. We are going to make Iceberg
• 21:47
  warehouse for you and. What about the engine?
• 21:49
  Because the hardest part is compute.
• 21:52
  Yeah. Well, so we started there,
• 21:53
  right? We started.
• 21:56
  With that wedge, which is a bit downstream of these problems and
• 21:58
  now we're working upstream, which is, you know, kind of kind
• 22:03
  of a funny journey. But I think somehow
• 22:05
  coincidentally not, not through any strategic planning of ours,
• 22:07
  kind of a good way to tackle this problem actually to start
• 22:10
  there in the middle kind of in between your like injection and
• 22:13
  your online AIML use cases and then kind of expand, expand from
• 22:17
  there. But I think, yeah, I mean, I
• 22:20
  think the cool thing is also like there's a lot of value and
• 22:25
  just people struggle with best practices for like how to, you
• 22:28
  know, do this ingestion, like write to these daily partition
• 22:30
  files in a certain way that it'll scale really well and plug
• 22:33
  in really well with downstream compute engines.
• 22:35
  So I think as we kind of put this stuff out and we'll be
• 22:37
  putting it out into open source, by the way, this will be not
• 22:40
  kind of vendor technology. It'll it'll kind of give people
• 22:45
  an example of like, hey, we've seen this stuff a couple of
• 22:47
  times. We've scaled this from like
• 22:48
  really bad practices to good practices at a couple of big
• 22:50
  companies. It's like we can start you off
• 22:52
  on a good path. I think there's a lot of value
• 22:54
  just in that as well. Starting on the right foot.
• 22:57
  Yeah, starting on the right foot.
• 22:58
  Like I can't tell you how many times we've migrated from like a
• 23:01
  huge monolithic Jason logging to like schema enforced, you know,
• 23:04
  smaller topics and things like that.
• 23:06
  And it, you know, it makes sense because to start with a
• 23:08
  monolith, because it's a lot easier to do that right now.
• 23:10
  But what if it was just as easy to start with the right thing?
• 23:14
  I think that would save so much pain for teams.
• 23:17
  But anyway, hell yeah yeah. I I see that completely.
• 23:21
  And so it's almost like you're doing the things that are the
• 23:25
  easiest just because they're, they're easy, even though, you
• 23:28
  know, later you're like, yeah, I'm just building my tech debt
• 23:31
  right now. Yeah.
• 23:32
  Yeah, yeah, I think that's a, it's a game that Nikhil and I
• 23:35
  like to play. It's like, how should this work?
• 23:38
  We looked at the data engineering problem for MLA.
• 23:41
  It's like that was the question that we asked ourselves.
• 23:43
  The answer was, well, very differently than how it works
• 23:44
  today. Yeah.
• 23:45
  And I think, I think with some of this like iceberg and
• 23:47
  ingestion stuff that we're seeing out there, it's like,
• 23:49
  well, this should be easier. And so we have some ideas on how
• 23:53
  to make it easier. All right.
• 23:55
  So you guys were also doing like agent stuff at Airbnb back in
• 23:59
  the day. Tell me more about that.
• 24:02
  Yeah. So the biggest thing at Airbnb
• 24:05
  that was using Kronon for to feed data into LLMS was customer
• 24:11
  support. Customer support is a big cost
• 24:15
  Center for Airbnb, right. Airbnb traditionally like
• 24:17
  implies people in US, right and pay pays a lot of like money for
• 24:23
  for that organization and the cost is like super high and like
• 24:27
  everything that they can save their translates to like profit
• 24:32
  or translates to lower prices, right.
• 24:34
  So all of that was a big driver and the biggest problem they had
• 24:40
  was populating context correctly.
• 24:44
  They already have the agent workflow figured out and the
• 24:48
  workflow has workflow decision points which are like either you
• 24:53
  know, give some data to LLM and see what it says and or it is
• 24:59
  more like gather data from around the organization from
• 25:02
  different surfaces and see what the user was doing in the
• 25:05
  previous day or previous week and figure out what they might
• 25:10
  want some. Features basically.
• 25:13
  It's basically features. It's contest engineering is what
• 25:16
  they call it now, but it's basically features.
• 25:18
  It is true. So they've started to build out
• 25:21
  something on their own using Lang chain and all of this like
• 25:24
  when they're prototyping, but there when it but when it came
• 25:27
  to prod it, you know, it was a natural thing for them to like,
• 25:31
  hey, let's like move all of these things to Cronon and have
• 25:34
  Cronon like drive the pipelines and the indexes and the
• 25:37
  endpoints that power this context.
• 25:40
  So that's kind of where it started.
• 25:42
  And search agent which I don't think is like out there yet but
• 25:46
  is another one that that was like those are the 2 main
• 25:49
  search. Agent.
• 25:51
  It's it's like, you know, you're chatting with agent that's
• 25:58
  helping you figure out how to book your next Airbnb and where
• 26:01
  to travel even right. So that's like the idea there.
• 26:05
  It's like a virtual travel agent kind of thing.
• 26:08
  Those are the 2 main use case at Airbnb.
• 26:12
  And so that was also just taking features from you and helping
• 26:16
  the search. Features and like in this one, a
• 26:19
  bit of embeddings too. I was going to say we should
• 26:22
  talk about embeddings as well. Yeah, talk about that.
• 26:24
  That's been an interesting one for us and coming out more and
• 26:26
  more recently. Yeah, even in places you
• 26:29
  wouldn't expect it, like fraud and personalization and search.
• 26:32
  Oh, really? Yeah, Rag was a big one, right?
• 26:35
  Like like in customer support for example, user says something
• 26:38
  and this something is about a particular set of policy and
• 26:42
  like this policy related to all this paragraphs in the policy
• 26:46
  documents. And there is huge corpus of like
• 26:48
  policy documents that we need to pull from, right?
• 26:51
  So that was all driven by RAG. So you know, it is like chunk
• 26:55
  these things and embed them and store them in a vector store and
• 26:59
  then pull it out, you know, whenever you need that
• 27:02
  information. And there is another set of
• 27:05
  things which is like users conversations, right as the
• 27:08
  thread is like happening with the host or with the support
• 27:11
  agent, the intent of the conversations keeps changing and
• 27:14
  that intent is captured as embeddings, right?
• 27:18
  So what you are trying to do is essentially like take this
• 27:20
  intent side and match it with the policy side, right?
• 27:24
  There is 2 embeddings and you are doing a dot product to like
• 27:26
  figure out what is the most relevant information that LLM
• 27:30
  needs to get to make a decision. Fascinating.
• 27:34
  So we were on both sides of this, but mostly on the user
• 27:37
  side because that's harder because you need to move a lot
• 27:40
  of real time data embedded and store it in a vector index and
• 27:44
  then surface it. Yeah.
• 27:46
  Yeah. And so this is like something
• 27:48
  that has some recent momentum in the Chronon open source, like
• 27:52
  first the native embedding support within Chronon.
• 27:54
  So it's not really features anymore.
• 27:56
  It's like features and embeddings now, which is pretty
• 27:57
  exciting. And there's a ton of value in
• 28:00
  like orchestrating more complex graphs of, you know, combining
• 28:03
  features and embeddings together into end to end work flows,
• 28:06
  which again, was pretty much prohibitively difficult before
• 28:10
  for all but the biggest, most sophisticated teams.
• 28:12
  But we're trying to make that just as easy as anything else in
• 28:14
  Chrono. But you're not creating
• 28:19
  embeddings out of features. Out of raw database, yeah, I
• 28:24
  mean, so we create embeddings out of message history and we
• 28:30
  treat a window of messages as a feature.
• 28:34
  OK. Right.
• 28:35
  And I mean, we treat that as a transformation like that
• 28:38
  windowing operation on the message stream as a feature
• 28:42
  essentially and feed that into an embedding model.
• 28:45
  Nice. Get an embedding out.
• 28:46
  User activities is another good example, right?
• 28:48
  Like user sequence modeling, where you take a bunch of a user
• 28:51
  activities. Again, you're usually
• 28:53
  aggravating like the last 50 or 100.
• 28:55
  Although now bigger windows have become an interesting thing that
• 28:57
  teams want to experiment with, you know, OK of user activities
• 29:01
  and turning that into a user embedding and using that to
• 29:03
  drive recommendations or personalized experiences.
• 29:06
  TikTok does 5000. 5000 embeddings. 5000 activities into
• 29:11
  an embedding. Right.
• 29:12
  And TikTok has the team to build that in front of.
• 29:14
  Not a lot of companies have access to that, even though they
• 29:16
  have potentially very high value use cases.
• 29:18
  And I think that's where we're offering a compelling, you know,
• 29:21
  open source based products here that can actually power that
• 29:25
  that kind of use case at scale.