Latest / Agentic Conversations (formally mlops.community) / Managing Small Knowledge Graphs for Multi-agent Systems // Tom Smoker // #236

Transcript

Managing Small Knowledge Graphs for Multi-agent Systems // Tom Smoker // #236 · Agentic Conversations (formally mlops.community)

• 0:00
  There's a name Tom Smoker. I was on a call yesterday where
• 0:04
  someone said is that a fake Zoom name?
• 0:06
  I was like, no, this is real. This is my genuine, genuine
• 0:09
  family name. Companies White House dot AI
• 0:13
  title. Yeah.
• 0:14
  Technical founder, CTO, how to make coffee is coffee flavoured
• 0:18
  water. It's just boring and black.
• 0:20
  It's yeah, about as basic as it gets.
• 0:22
  But I am from Australia so I actually do love like really
• 0:26
  good. Kind of like single origin
• 0:29
  stuff. I'm very very into puppy butt.
• 0:31
  Always black. Welcome back to the MLS
• 0:35
  Community Podcast. I am your host Dimitrios and
• 0:39
  today talking with Tom Smoker. I had a blast just thinking
• 0:45
  about knowledge graphs, learning how he is using them to make his
• 0:49
  rag systems more reliable, and really what he said that stuck
• 0:54
  with me and will stick with me. I'll probably end U quoting this
• 0:58
  every two out of three episodes. So you heard it here first.
• 1:04
  Be exactly wrong and that's what he's going for when he's
• 1:08
  creating his systems. He's trying to make sure that he
• 1:11
  can be exactly wrong instead of exactly right, which he breaks
• 1:15
  down into more detail. He talks about how he wants to
• 1:19
  be exactly wrong because there for the types of systems that he
• 1:24
  is breaking down, it is the start of creating a
• 1:30
  deterministic system with AI and with ALMS. And one way that he's
• 1:36
  found to do that is by using knowledge graphs.
• 1:40
  And of course, knowledge graphs. 1 piece of wisdom that he
• 1:45
  dropped on me with the knowledge graphs are that rice to a farmer
• 1:50
  is not the same thing as rice to a chef.
• 1:53
  And so you want to have a knowledge graph that can
• 1:55
  understand the differences and how these two things are linked
• 2:00
  but different. For those of you that have been
• 2:04
  wondering how to bring your rags up a notch, this is the
• 2:09
  conversation for you. However, he did warn at the end
• 2:13
  of the conversation that he's seen people dive into trying to
• 2:18
  optimise their RAG systems a little bit blindly.
• 2:22
  So he breaks down how you can make your RAG system 100%
• 2:28
  accurate and perfect flawless. But what kind of business value
• 2:33
  does that actually bring? What kind of like like, So what
• 2:36
  is basically the end statement there?
• 2:39
  Yeah, you created this chat bot that can tell you how many days
• 2:43
  of holiday you have left. But So what?
• 2:47
  And how much engineering time or resources went into having to
• 2:53
  really optimise it. So you have this chat bot that
• 2:57
  can tell you the hour policies when maybe just a search bar is
• 3:02
  sufficient. Let's get into it with Tom.
• 3:04
  If you liked it, as always, share with one friend or give us
• 3:10
  a star. Give us some feedback on
• 3:11
  Spotify. I'm reading all of.
• 3:13
  Them. We'll see you on the other side.
• 3:23
  When I read a paper on how to mitigate resignations and it was
• 3:27
  basically like 33 different techniques on mitigating
• 3:31
  hallucinations with different strategies and one of the
• 3:34
  strategies was using knowledge graphs.
• 3:37
  So it's probably good for us just to start with like what got
• 3:43
  you down the rabbit hole of knowledge graphs?
• 3:45
  And I think it was academia. And I know that we we also were
• 3:49
  talking about your shirt for anybody that's just listening,
• 3:51
  you've got on the archive shirt, you know, showing it proud.
• 3:55
  So give us a quick rundown on how you got deep down that
• 3:58
  rabbit hole. Yeah, yeah, yeah.
• 4:00
  So, yeah, Academia, absolutely. The Academy, I did a masters in
• 4:05
  electrical engineering and I got exposed to a specific thesis
• 4:09
  with a company that was a large mining company public called BHP
• 4:14
  when I was working in Australia. And my supervisors, Tim French,
• 4:17
  Way Lou, Melinda Horwitz were amazing.
• 4:19
  And they really started to suggest.
• 4:21
  And so Tim specifically was my primary supervisor and he's a
• 4:24
  logician. So he works in computer science,
• 4:26
  works in math, but he's a logician.
• 4:28
  So this was 2016, 2017 and he was like this representation,
• 4:33
  there's ontology, there's these things called knowledge graphs.
• 4:35
  Neophyte Jay was starting to pop off and it was like, let's
• 4:38
  understand and see what representation looks like as
• 4:41
  like this is when computer vision was really starting.
• 4:43
  So from there, and I won't bore you too much with the details,
• 4:45
  but from there really enjoyed it.
• 4:47
  Did a masters in comparing expert systems and ontologies
• 4:50
  and knowledge graphs. Enjoyed it, published my
• 4:53
  research, got myself into a PhD programme.
• 4:55
  Same supervisors now are doing a PhD in CS which did the same.
• 4:59
  Did knowledge graphs knowledge graph embeddings when embeddings
• 5:01
  were really starting? So I've been doing this for a
• 5:02
  minute. Yeah.
• 5:03
  So that was PhD started in 2018. Can I just point out that
• 5:09
  logician may be my new favourite word because it's basically like
• 5:13
  a magician but for logic. Is that how you rationalise it?
• 5:16
  It's how I think about it. He he's got some great quotes,
• 5:19
  too many to fit into this podcast.
• 5:21
  I got a lot of respect for him, but one of his best is that he
• 5:24
  doesn't code, doesn't code in language codes and pen and
• 5:26
  paper. And I always think about that a
• 5:28
  lot. So the purest form.
• 5:30
  But yeah, absolutely. It's like really, I think logic
• 5:33
  is a constantly relevant, you know, art and science, but very
• 5:38
  much like how do we think about what happens?
• 5:41
  And so I was actually, I did a undergraduate masters in
• 5:43
  electrical engineering, but actually minded in philosophy.
• 5:46
  And so the combination there was was really common in philosophy
• 5:49
  of logic, philosophy of science and representation.
• 5:53
  So tell me about knowledge graphs and how they are
• 5:56
  incorporated into rags because it feels like everybody's
• 5:59
  talking about them, feels like something that is coming U
• 6:03
  because of this mitigation of hallucinations.
• 6:06
  Or you can get better results in some cases.
• 6:08
  And I want to go down which cases those are, but just give
• 6:11
  us like the breakdown of what you've been seeing lately.
• 6:14
  Yeah, So the context for for White House is my cofounders and
• 6:18
  I were working in this space early last year building legal
• 6:23
  rack, right, which was information retrieval.
• 6:26
  But really like, hey, you can type a question in natural
• 6:28
  language that you can get an answer back.
• 6:30
  And we started to get these hallucinations.
• 6:31
  And I want to say off the top that, and I, I said this in my
• 6:34
  talk at, at KGC recently, LLM's only hallucinate or models only
• 6:38
  hallucinate. Like the amazing thing about
• 6:40
  generative AI is that they're creative.
• 6:42
  And I think Audrey Tabatha says it really well.
• 6:44
  It's a much broader quote and there's a tweet about it.
• 6:46
  You can find it. But L don't have a hallucination
• 6:49
  problem. LM assistants have a
• 6:51
  hallucination problem. And I think at the time he said
• 6:53
  that was really right, that they were happy with the
• 6:56
  hallucination until we're not. And so we would get people
• 6:59
  saying it's hallucinating all these edge cases.
• 7:01
  And I think at the time, like misattribution was a really high
• 7:04
  hallucination. So someone quoting someone else.
• 7:07
  But when you look at the models, like we had to rethink what we
• 7:10
  were doing because we were working in very specific, very
• 7:12
  high value legal cases, which was like, if, you know, if
• 7:16
  ioffer this person, this amount of money or this preset in this
• 7:18
  fund, you know, does this trigger any clauses?
• 7:21
  And so these questions can't be wrong.
• 7:24
  So we would like look the the risk of hallucination or the
• 7:26
  risk of error if everything's a hallucination is a problem.
• 7:29
  O we had to think about, well, how do we represent?
• 7:31
  More specifically, how do we create determinism for
• 7:34
  ourselves, even if that's scoped in?
• 7:36
  And again, because of my background, we came to knowledge
• 7:39
  graphs, we came to representation and and some
• 7:41
  level of like visibility and explain ability in the answer.
• 7:46
  So then you can adjust, constrain, and then repeatedly
• 7:49
  produce the results that you want.
• 7:53
  So this is the fascinating thing, the repeatability I think
• 7:56
  everyone is going to be on board with.
• 7:58
  Now when you talk about knowledge graphs, I had been
• 8:01
  vaguely familiar with things like Neo Fuji or I think there's
• 8:05
  other ones like Arango DB, and you get these basically
• 8:11
  knowledge graph database type things, tools that are out
• 8:16
  there. Is it just replacing a vector
• 8:20
  database with a knowledge graph database?
• 8:23
  Is that kind of how you see it or do you add both?
• 8:26
  What do you look at when you're designing the system?
• 8:29
  Yes, so we are at why our workflow tooling on top of
• 8:33
  infrastructure and and we work with a bunch of different
• 8:35
  infrastructure people. So there are graph databases
• 8:38
  near for Jay we work with we like I, I'd like to give a shout
• 8:41
  out to Kuzu DB out of Waterloo as well.
• 8:43
  A friend Prasad for they've got an amazing.
• 8:45
  They're like the doctor B for graphs and I think what they're
• 8:47
  doing is really cool. Use them, but we use vector as
• 8:50
  well. I, I think there's, there's
• 8:52
  semantic similarity in vectorization, especially
• 8:54
  embeddings and different type of embeddings, I think are really
• 8:56
  underexplored. And so for us, like information
• 8:58
  retrieval is a process. It's not a one off like it's a
• 9:03
  system. In our case, it's a multi agent
• 9:05
  system. And some of that agency is going
• 9:07
  to use vectors and semantic similarity and representation.
• 9:11
  Some of it's going to use graphs and knowledge graphs.
• 9:13
  So at its simplest form, you can you can replace a vector
• 9:17
  database with a graph database with a relational database.
• 9:21
  They're all similar in that place that you need to get
• 9:23
  information and you want to augment the the cause that you
• 9:26
  have. But I think if you look at it as
• 9:28
  a pipeline, as a system, I think different databases apply in
• 9:31
  different ways and at why how we use all of them.
• 9:34
  And I personally use all of them in terms of where they're most
• 9:37
  applicable. So how is this working?
• 9:39
  Do you have like a proxy that decides which database to go to?
• 9:44
  Right, Yeah. So I mean there are ways to do
• 9:46
  that. I mean you Langen has awesome
• 9:48
  stuff around around function calling and and and and tools
• 9:50
  and and, and how best to call from each one.
• 9:52
  And I think conditionally doing things like, you know,
• 9:56
  conditional logic in agents is is starting to be really
• 9:58
  promising. But for us it's actually like a
• 10:00
  sequential pipeline. So we embed, you know, chunk
• 10:03
  embed and then store things in vector databases and then we use
• 10:06
  that vector database to retrieve and then build a graph.
• 10:09
  So it's actually like it's in sequence in the way that we like
• 10:12
  as kind of an opinionated workflow.
• 10:14
  And so it's it's all part of a larger system and it's taking
• 10:19
  advantage of each of the benefits.
• 10:20
  Ultimately, what I want to be querying is something that gives
• 10:25
  me the same answer every time. We, we spent a lot of time
• 10:28
  trying to be exactly wrong, which I think is a cool term in
• 10:32
  this because I, I think you can be almost or probably right
• 10:34
  really quickly, but being exactly wrong is the first step
• 10:37
  to being exactly right. And so putting it in a graph for
• 10:40
  us was a good way to store and then see also like it's, you can
• 10:45
  kind of see the schema and then change the schema.
• 10:47
  And I think that's really powerful.
• 10:49
  What do you mean by exactly wrong?
• 10:51
  That is a funny phrase. I instinctively laugh, but yeah,
• 10:55
  not sure we understand it. I don't know if it's a broader
• 10:57
  term. It's it's what we talk about
• 10:59
  internally, which is like when people talk about determinism in
• 11:04
  anything in LMS. Again, they're generative, right?
• 11:06
  I don't think you can get determinism out of an L And I
• 11:08
  also don't necessarily think you'd want to.
• 11:10
  I think they're just amazing. I went to a talk the other night
• 11:12
  at free pick around creativity and, and, and the, you know,
• 11:17
  creator economy in when you're supported by, you know, these
• 11:21
  generative models. And I think it's amazing what
• 11:22
  can be done. But I think getting things
• 11:25
  exactly right is a really high, really, really hard ask.
• 11:28
  And I think it's difficult to guarantee, but I think it's much
• 11:30
  easier to get it exactly wrong. I think you can prove that
• 11:33
  really quickly. So for us, we can make a graph
• 11:35
  and look at it and be like, this does not represent what I want
• 11:37
  it to represent, right? And that's a step towards
• 11:39
  progress. Because even if I get it exactly
• 11:42
  wrong, that means my answer is going to be wrong every time.
• 11:45
  And that's for us, like at least like if you walk back, that's
• 11:48
  the step you need to be at because if you can, you can make
• 11:51
  it probably right. But then what does that mean for
• 11:53
  a customer? What does that mean for a
• 11:54
  product? What does that mean for
• 11:55
  reliability and consistently consistency?
• 11:57
  So exactly right is obviously the goal.
• 11:59
  I think it's a lofty and I think it's harder to prove exactly
• 12:02
  wrong. It's like, OK, now we have a
• 12:04
  framework for determinism and now we can iteratively improve
• 12:07
  it to get it where we want to. Ohh OK I see where you go with
• 12:12
  that now. This is fascinating stuff to
• 12:17
  think about, but how does the data and what you are doing in
• 12:24
  the system change? Because I'm very used to this
• 12:29
  idea of all right, we've got the embeddings or we've chunked it,
• 12:32
  we've got the embeddings, we've throw it in the vector store and
• 12:35
  all of that. Do you need to now be collecting
• 12:38
  new data? Do you need to have different
• 12:40
  sources? Like?
• 12:41
  How can you make sure that the knowledge graph is as robust as
• 12:45
  it can be? Yeah, we just put out an article
• 12:49
  about this, which is like this rudely.
• 12:51
  There's like a four square matrix about this, which is
• 12:55
  like, does the underlying data change or not?
• 12:57
  Like is the underlying data static or dynamic?
• 13:00
  And then does the schema static or dynamic, Right.
• 13:02
  And so knowledge graph scheme is really interesting.
• 13:05
  You can call them ontologies if you want.
• 13:07
  I think that the lines blur with knowledge graphs.
• 13:10
  I think the term has been taken, but there isn't a standard
• 13:13
  definition. There's a pretty good one from
• 13:15
  onto text that I like, but I think people use it in such a
• 13:17
  broad way because it's so accessible.
• 13:19
  People gonna see a graph and be like, oh, I get it.
• 13:21
  And better then it's very difficult to do in practise.
• 13:24
  So it's very accessible at a high level, very difficult and
• 13:27
  low level. So what, what changes there is
• 13:29
  like all the, all the specifics around it, but the way we think
• 13:32
  about it is create the schema, which is kind of like the types
• 13:36
  of your, what we call nodes and edges that can change.
• 13:40
  For example, that's like people place price like those, those
• 13:44
  broad categories. And then the underlying data, as
• 13:47
  you can imagine, like there's different people in your
• 13:49
  database, right? There's different prices in your
• 13:51
  database. So that's an example of changing
• 13:53
  and underlying data almost always changes.
• 13:56
  To give an example, it's like we work with like a veterinary
• 14:00
  radiology clinic, right? The, the diseases that can
• 14:04
  happen to an animal don't change that often.
• 14:07
  That's in the literature, right? The the patient specific data
• 14:11
  that can change a lot O those two are in combination.
• 14:14
  And so the, the scheme if a new disease is discovered, sure, if
• 14:18
  a new type of disease is discovered and that's different.
• 14:21
  And so it's constantly moving and shifting.
• 14:24
  All of this is to say that why we like using graphs for this
• 14:28
  particular problem is that the schemas are extensible.
• 14:31
  The data is extensible. It loads in query it back pretty
• 14:34
  easily. You have some level of structure
• 14:37
  that you don't necessarily get if you embed in light in space
• 14:40
  and then you get some level of flexibility and and change that
• 14:44
  you can't necessarily create easily in a relational database.
• 14:47
  And so that's where it's kind of the middle ground, which is
• 14:49
  useful. I hope that answered it.
• 14:51
  I'm not sure if that answers. If I'm understanding what you're
• 14:55
  saying correctly, it's basically that when you are able to use
• 15:00
  the knowledge graph over performs or it performs best
• 15:05
  when you have almost like this structured data and things that
• 15:12
  they're going to be changing, but also things that aren't
• 15:15
  changing. O That's where if you talk about
• 15:17
  this veterinarian and you have the diseases, but the patients
• 15:21
  can get new dogs or new cats, they can change their house,
• 15:25
  whatever it may be, their dogs get older.
• 15:28
  And so all of those things are changing.
• 15:30
  But then there's certain things, like the way that this disease
• 15:34
  manifests itself, that's not really changing.
• 15:37
  Yes. And so This is why we like the
• 15:40
  idea, we're promoting this idea, but it's also just makes our
• 15:43
  life a lot easier, which is like it's easy to make a smaller
• 15:46
  graph than it is to make a big one.
• 15:47
  And I spent a lot of time in academia and in in industry
• 15:51
  working on enormous large scale graphs in in conferences and
• 15:54
  then in practise. And the two situations I just
• 15:57
  described to you, if our goal is to create some structure that we
• 16:00
  can change when we're doing our retrieval, right, that's
• 16:03
  difficult in just pure, uh, just pure and beddings.
• 16:08
  And that's difficult in relational databases, right?
• 16:13
  Not, not, not doable. It's just difficult.
• 16:16
  What we like is being like, well, I'd rather just make 2
• 16:18
  graphs. I'd rather make one graph that
• 16:20
  has the diseases that I can iteratively bring in over time
• 16:24
  because I don't bring in all of them just to bring in the ones
• 16:26
  that matter to me. And then I'd also rather have a
• 16:28
  separate graph that is the patient information, two
• 16:31
  different schemes, two different graphs and then use them for to
• 16:34
  kind of logically compare because again, they have
• 16:38
  different dynamics. One is changing.
• 16:41
  The age is changing much more frequently than like the
• 16:44
  diseases that can happen change. So are you using knowledge
• 16:50
  graphs mainly to sanity check what's retrieved from the vector
• 16:54
  database? In a lot of ways and and and so
• 16:57
  that's one of the ways we can also go straight from CSV to to
• 17:00
  to graph, right. That's a level of structured
• 17:02
  data. We can go straight from.
• 17:03
  I I think LinkedIn recently put out a, a paper around the the
• 17:07
  GRABI like that semi structured data.
• 17:09
  You can go straight. We can go straight to graph from
• 17:11
  that. But when it comes to Saturday
• 17:13
  checking, yes, I really like that as a term, because if
• 17:16
  you're looking at unstructured data, if you're looking at APDF,
• 17:19
  it's not templated, right? It's just raw text.
• 17:21
  You're also looking at combining that PDF with the output of the
• 17:23
  LM, which is really useful. Yes, it's very useful to look at
• 17:27
  what it made and be like, hey, that's almost right, but here's
• 17:31
  where it's wrong. And then, you know, most, like
• 17:35
  most graphs, full acid or full crud, right?
• 17:38
  So it's like, whatever I want to do that.
• 17:39
  I can just delete this one if I want to.
• 17:41
  And now I have the repeatability, but I got rid of
• 17:44
  the thing that was off. Yeah, because I think if I
• 17:48
  remember correctly, the way that I saw knowledge graphs being
• 17:51
  used were to check for facts in the output.
• 17:56
  And so there was, it was, I think it was a smaller model
• 17:59
  that would just, it was trained to check for facts and then
• 18:03
  query and knowledge graph to see if those facts were true.
• 18:07
  It was clear knowledge graph or search the Internet.
• 18:09
  I think we're we're both of those.
• 18:11
  But basically, like if you said you know this person was alive
• 18:15
  in 1920 in your output, the smaller model plus the knowledge
• 18:20
  graph is going to be able to Fact Check that really easily.
• 18:24
  Yes, absolutely. I think, I think it's just, it's
• 18:27
  a broader term, awkward structured grounding, which I've
• 18:29
  seen recently, which I really like and was like, hey, we have
• 18:32
  this like external fact set like basically like make it like
• 18:37
  structure Wikipedia. So when I asked this question,
• 18:39
  it's like I'm getting the exact answer back that I want, right?
• 18:42
  Like we're working with, we're working with the customer at the
• 18:44
  moment who has like a like a HR customer care chat bot, right?
• 18:50
  And it's like when someone asks, I want to call someone about
• 18:53
  this problem, like which phone number do you bring in?
• 18:56
  And that could be a very difficult thing to find from raw
• 18:59
  text, right? And I think an interesting
• 19:02
  hallucination. And again, I use that word.
• 19:05
  I think it's difficult to use that word, as I've said, but an
• 19:07
  interesting hallucination that I've noticed is mean reversion
• 19:10
  that I see a lot. Like I think a lot of people
• 19:12
  think about hallucination as the edge, right?
• 19:14
  You lead and you know, you lead an LLM into a dark room and then
• 19:18
  say figure your way out. And then it's like it comes to
• 19:21
  this, like it just creates this whatever this magical, this new
• 19:24
  thing, whatever it may be. But if you look at something
• 19:27
  like code generation and you look at the language Solidity,
• 19:30
  which is heavily based on JavaScript, a lot of the
• 19:32
  hallucinations in generating Solidity that I've noticed, I
• 19:35
  haven't seen research on this is JavaScript syntax because it's
• 19:39
  reverting to the mean of what's most likely.
• 19:41
  And that seems really common with these models.
• 19:44
  Ohh fascinating to think about that.
• 19:47
  Like it's just going back to what it knows and.
• 19:49
  Yeah, or what countries knows it's likelihood, right.
• 19:52
  And so it's like, let's just find the most likely thing.
• 19:54
  And so in the in the the veterinary example, instead of
• 19:58
  instead of finding the rare disease that could be the value
• 20:02
  of the diagnosis, it finds a disease that is common to a
• 20:06
  similar animal. Naively, like and, and again, I
• 20:10
  want to be specific about the vectorization.
• 20:12
  I mean, longer index has some amazing stuff around this.
• 20:14
  Like you said, it's probably an infographic of theirs or or
• 20:17
  posted there that you're referring to.
• 20:18
  Well, the types of hallucination.
• 20:20
  There is many ways that you can improve this process and
• 20:22
  referring to knowledge graphs as one of them and one that we
• 20:24
  like. But this is not the don't want
• 20:26
  to suggest that it's the only way like what I describe as
• 20:29
  quite naive retrieval there. There's a lot of different ways
• 20:31
  to do it. But for us, yes, the control and
• 20:34
  the specifics and the scope is is really important.
• 20:37
  So then the LM is only hallucinating within the bounds
• 20:41
  of what we told it. One of the best hallucinations
• 20:45
  I've seen. I don't know if you've seen this
• 20:46
  but it was a tensorflow hallucination and there is a
• 20:51
  there if if you've used the Adam or Adam before intensive flu
• 20:55
  right? I saw a post on LinkedIn.
• 20:58
  It got a bit of attention, I think, but I, I don't know which
• 21:01
  model it is, and I don't want to say which model just in case,
• 21:04
  but it hallucinated Adam as John and then it got the error.
• 21:08
  There was like it, it got a compiler that was like, ohh,
• 21:10
  hey, there's no John, you know, method or property.
• 21:14
  And I thought that was really funny, but it was funny on the
• 21:16
  surface. But then like quite worrying in
• 21:20
  the absolute because then like an entirely unbounded lightened
• 21:24
  space with mean reversion is what we're dealing with.
• 21:26
  And so for us, we're like, look, let's just keep the scope really
• 21:29
  specific in terms of these things are going to hallucinate.
• 21:33
  That's their job. Let's make them kind of like
• 21:37
  still be the bumper car that they are, but put the put the
• 21:39
  guardrails in really specifically and that that, you
• 21:41
  know, graphs for us with those guardrails.
• 21:43
  I really like. This idea of basically
• 21:46
  tightening U this scope as much as possible so that you don't.
• 21:52
  Allow. The knowledge graph or the LLM
• 21:55
  or the embeddings to play in this infinite space, You're
• 21:59
  really trying to say, here's where it is, Just go look over
• 22:03
  here, it's going to be there, I promise, type thing.
• 22:07
  What are? Other ways that you can do that,
• 22:10
  not necessarily or what are other examples of you doing that
• 22:14
  I guess is a better way of putting that question.
• 22:16
  Yeah, no. I mean, I can talk to a bunch of
• 22:18
  ways outside of outside of graphs.
• 22:19
  I'm not sure if it's in the context of this, but like, I
• 22:21
  just want to make sure, like, you know, re ranking from cohere
• 22:24
  is is is great. There's the agentic workflows,
• 22:26
  which I'm happy to talk about. We do a lot of multi agent stuff
• 22:28
  internally, but like, you know, having checker agents, having
• 22:31
  conditional agents, I think works really well.
• 22:33
  Again, to basically anything. I, I said this in my job.
• 22:35
  Anything advanced drag from from Lamar index that I think is a
• 22:38
  really good way to start with this because they structured out
• 22:41
  really well. What does preprocessing look
• 22:42
  like? What's optimal chunking line?
• 22:44
  Chad has similar stuff like both of those groups have really
• 22:47
  explored this and seen this really well.
• 22:49
  So there's again, it's a process, not a system.
• 22:52
  And so there's a lot of incremental ways you can prove.
• 22:53
  But yeah, for us with the scope to give further examples of
• 22:58
  scope. Why we think?
• 23:00
  About smaller graphs rather than one big one, is that like a
• 23:04
  motivating question for us is like rice to a farmer is
• 23:07
  different to rise to a chef. So how do you put rice in a
• 23:10
  database if you want to create some level of consumer
• 23:14
  interaction? Because there's a buyer and a
• 23:16
  seller there, right? And so it's like, well, I can't
• 23:18
  just have a rice node or a rice cell.
• 23:21
  I have to have some other context.
• 23:23
  How do I represent it? And so for us it's like, well,
• 23:26
  scoping in this is the farmer and this is the chef schema than
• 23:32
  rice within that is in context I think is really good.
• 23:35
  So scoping in contextually, it is something that we work on a
• 23:39
  lot for this stuff. Give me more.
• 23:42
  Examples I love that the rice yeah farmer and right the chef
• 23:45
  and and of course when you think about it in the knowledge graph
• 23:49
  way it makes complete sense because you understand the
• 23:54
  relation between those two and. It.
• 23:57
  Also makes sense on why it would breakdown in a traditional
• 24:01
  relational database. Yeah, I'd.
• 24:03
  Probably should have said this at the top, but I mean, I've
• 24:05
  been a part of the graph and representation community for a
• 24:08
  long time, both in practise and, and and in and in pure and
• 24:12
  research. And they're a great use cases
• 24:14
  for large like ontologically aligned, well described
• 24:17
  knowledge graphs. And I use them a lot in my PhD
• 24:20
  around like free bases, your nails.
• 24:22
  I think they're great. I'm specifically talking about
• 24:25
  I'm trying to solve RAG problems and I'm trying to use this
• 24:27
  architecture so frag problems. So I want that to be clear that
• 24:31
  I think it's difficult to maintain those worldviews in
• 24:33
  this context. But another example would be I
• 24:36
  worked with a legal document and they had a definitions.
• 24:39
  A lot of legal documents had definitions page and they would
• 24:42
  say vehicular capacity, right? And my leg was just reading
• 24:46
  that, like what do you think the vehicular capacity is going to
• 24:49
  be? How many people?
• 24:50
  Fit in the vehicle. Cool, it's how many?
• 24:53
  Cars fit on a road. Ohh.
• 24:55
  Yeah. So it's and that's and so it has
• 24:57
  to be in in context. You can figure this out, but how
• 25:00
  do you, how do you put that context in?
• 25:03
  So it's like you need to tie that definition to that context
• 25:05
  every single time. And then you, as you start to
• 25:08
  try and represent the whole world, it is logical
• 25:11
  inconsistencies that happen really quickly.
• 25:13
  Plus like if you want to update over time, how does that work?
• 25:16
  So yeah, it's those sort of problems that we are like scope
• 25:20
  it, right, Scope in and then now you can talk about it as if
• 25:25
  you're in this context. Have you seen?
• 25:28
  Ways like when is the knowledge graph too small?
• 25:31
  Or is it ever too small? Yeah, I, I.
• 25:34
  Think that's a great point. So when I talk about small
• 25:36
  versus big, it's really like a very accessible way for people
• 25:38
  to think like, because I've done this before, I know all the pain
• 25:41
  points. Making a big knowledge graph is
• 25:43
  great in theory and so difficult in practise.
• 25:45
  The ROI, like there are some great use cases for it, but I
• 25:48
  think that some people stop before they build it properly.
• 25:51
  And so for me, when I see a small graph, it isn't a small
• 25:53
  domain. It's really like start with a
• 25:55
  well connected subgraph first, like almost like a just in time
• 25:59
  graph and then build that out as you go.
• 26:03
  So when is a graph too small is an interesting question because
• 26:05
  it's like the answer is when there isn't enough context to
• 26:09
  answer your problem, I would say.
• 26:11
  But I think that's often the point of the graph is, you know,
• 26:14
  efficient navigation and traversal.
• 26:15
  But really you only need a couple of notes to answer your
• 26:18
  question. Like we we think a lot about
• 26:21
  completeness and why graphs are useful for completeness.
• 26:24
  Like if you look at diagnosis, this would be an example where
• 26:27
  vector databases may struggle because it's not necessarily
• 26:31
  like a quantitative use, right? It will give you the best
• 26:34
  diagnosis back. But if your job as a, you know,
• 26:38
  as a medical professional is to look at all the available
• 26:41
  diagnosis and then make a decision based on patient
• 26:44
  history, then you actually want all of them back, not just the
• 26:46
  most likely the best ones. And so something like a graph
• 26:49
  like you can represent the entire thing and and so then you
• 26:53
  can get back the the full scope of information.
• 26:56
  So that's like a how small the graph this is.
• 26:58
  Like as long as it has all the diagnosis in it, then that's
• 27:01
  really as big as it needs to be at that.
• 27:03
  Again, just in time. It's just whatever answers your
• 27:06
  question. And are you creating?
• 27:09
  New knowledge. Graphs for each use case.
• 27:13
  Yeah, so a lot of times, yes, but we like what we've.
• 27:17
  And again, I don't want to be more willing for people to check
• 27:19
  us out, but very quickly, like our job is just to make this,
• 27:22
  how quickly can you make a graph and how quickly can you change
• 27:24
  it? But that's the whole process,
• 27:26
  which is like, don't sit off and wait.
• 27:28
  Like we can get it down to like a minute.
• 27:30
  Just toss a PDF into schema like however many pages you need.
• 27:32
  Just immediately see it be like, this sucks, change it, fix it
• 27:36
  and then iterate. And so yes, like growth
• 27:38
  management is something that we do, but when you look at trying
• 27:41
  to solve the, the edge cases shouldn't do that.
• 27:44
  But like the, the, the difficult problems in RAG, the ones that
• 27:46
  aren't immediately solvable, they are point solutions, right?
• 27:50
  It's not a one-size-fits-all. It's usually the bounds of what
• 27:52
  is represented a granularity or or a condition or its
• 27:55
  specificity. So yes, making a graph per
• 27:58
  thing. So our job is to make that as
• 28:00
  quick as possible. All right, real quick, let's
• 28:02
  talk for a minute about our sponsors of this episode making
• 28:05
  it all happen. Lattice flow AI.
• 28:08
  Are you grappling with stagnant model performance?
• 28:10
  Gartner reveals a staggering statistic that 85% of models
• 28:14
  never make it into production. Why?
• 28:16
  Well, reasons can include poor data quality, labelling issues,
• 28:20
  overfitting, underfitting, and more.
• 28:23
  But the real challenge lies in uncovering blind spots that lurk
• 28:27
  around until models hit production.
• 28:29
  Even with an impressive aggregate performance of 90%,
• 28:33
  models can plateau. Sadly, many companies optimise
• 28:37
  for prioritising model performance for perfect
• 28:41
  scenarios while leaving safety as an afterthought.
• 28:45
  Introducing Lattice Flow AI, the pioneer in delivering robust and
• 28:50
  reliable AI models at scale. They are here to help you
• 28:54
  mitigate these risks head on during the AI development stage,
• 28:57
  preventing any unwanted surprises in the real world.
• 29:01
  Their platform empowers your data scientists and ML engineers
• 29:04
  to systematically pinpoint and rectify data and model errors,
• 29:09
  enhancing predictive performance at scale.
• 29:12
  With Lattice Flow AI, you can accelerate time to production
• 29:15
  with a reliable and trustworthy models at scale.
• 29:18
  Don't let your model stall. Visit Lattice Flow Dot AI and
• 29:23
  book a call with the folks over there right now.
• 29:27
  Let them know you heard about it from the MO OPS Community
• 29:29
  podcast. Let's get back into the show.
• 29:33
  Yeah, so. Now let's go into the points
• 29:36
  that you mentioned on. When knowledge.
• 29:39
  Graphs aren't the most useful in the rag situations.
• 29:42
  Yeah, yeah. So, I mean, I, I use vector
• 29:45
  database rat like the vector database and semantic similarity
• 29:48
  works really well, right? And I'd also say that graphs the
• 29:51
  moment are, are really textual representations.
• 29:53
  And I think there's some amazing work, good friends with the team
• 29:56
  at Marco. They're doing some awesome,
• 29:58
  awesome work around custom embeddings and and
• 29:59
  multimodality, especially images.
• 30:01
  So I, I think that stuff is really useful for those, for
• 30:05
  those sort of embeddings. I would also say, yeah, I would,
• 30:09
  it depends on how specific you want to be.
• 30:11
  Because there's also a situation in which like you can change a
• 30:15
  graph schema, right? But if you already have a
• 30:17
  relational database with this information and you've already
• 30:20
  spent time describing that schema perfectly and you're
• 30:22
  problem is refer to this database really specifically, I
• 30:25
  think it is worth just using a relational database for that
• 30:28
  case. But a lot of pretty much all of
• 30:30
  the problems we work with graphs can be used for and I think
• 30:36
  vector databases can be used for just can't guarantee the
• 30:38
  reliability of it. And I think relational database
• 30:41
  can be used for a lot of that. It's just sometimes difficult
• 30:44
  and there's there's issues around natural language into
• 30:48
  query language, whether graph or or relational or otherwise.
• 30:51
  So I'd say that the, the problems that you need to be
• 30:55
  specific, reliable and change the schemer of an update that
• 30:58
  graphs work really well for. And then problems that you need
• 31:01
  to be just like high level, like customer responses or like help
• 31:06
  me book a trip somewhere or, or, or, you know, give me a recipe
• 31:09
  like it's great use cases for for vectorization.
• 31:12
  So I'd say that if if you're OK with the average being really
• 31:15
  good, I think that you're OK with affected databases in a lot
• 31:18
  of ways and. If I'm understanding this
• 31:22
  correctly, I if I've got my PDF documents, I'm throwing them
• 31:27
  into an embedding model and then putting them in my vector
• 31:31
  database. But I'm also throwing them into
• 31:34
  a knowledge graph. Yes, so.
• 31:36
  That's a great question. So what we actually do with the
• 31:39
  knowledge graphs is there are layer on top to every node that
• 31:42
  we have in our graph, whether that's like, you know, type
• 31:45
  person named Tom has a has a connection back to the chunk,
• 31:51
  the embedded chunk text where it came from.
• 31:54
  So this provenance from the embedding back to the
• 31:58
  representation. So they're not necessarily
• 32:00
  distinct. They're connected.
• 32:01
  And so you can think of the graph as like an indexing layer
• 32:04
  on top of the stored data. And the value of that is that
• 32:08
  again, with that scope, if you interact with the graph first in
• 32:11
  a deterministic way, then you just get back just the chunks
• 32:14
  that you want. I can't remember who it was at a
• 32:17
  I was at a is either a land chain posed or a land chain
• 32:19
  talk. Think it might have been Will
• 32:21
  from from land chain or it could have been Harrison talking about
• 32:25
  context poisoning and how if you set like a top K of 10, so you
• 32:31
  return back 10 chunks and three of those chunks have the answer
• 32:35
  in it and seven don't, you're at high risk of making it worse.
• 32:40
  Because if we believed it few shooting a model works, which I
• 32:43
  absolutely do, then you can fuse shot it wrong as easy as you can
• 32:47
  fuse shot right. And so for us, like indexing on
• 32:51
  top of the information that we actually care about means that
• 32:54
  the graph can direct it down to just those 3 chunks, or at
• 32:58
  least, you know, much more like concentrated retrieval process.
• 33:02
  So again, in a process like you can get back the graph, you can
• 33:05
  get back a summary of each node, you can get back the properties,
• 33:07
  but also we just have optional flags.
• 33:09
  So if you want the chunks as well, if you're the provenance,
• 33:11
  you are the full document, you want the document name, that's
• 33:14
  also like a, a process. So they're not necessarily
• 33:16
  distinct. I think you can think in a lot
• 33:18
  of times and we make different types of different people
• 33:20
  depending on the use case, but help them do the same.
• 33:22
  But I think if you think about it as an indexing layer on top,
• 33:24
  I think that's a useful way to think of and.
• 33:27
  So because of the. Order of.
• 33:30
  Operations that you were talking about, you're able to catch
• 33:34
  that. If we take that K10 example and
• 33:37
  you get the seven back that are kind of shit, you're able to
• 33:41
  catch that with the knowledge graph and be like, all right,
• 33:43
  well, let's philtre out these seven because it's already come
• 33:46
  out and we already know what the knowledge graph layer it is like
• 33:50
  a philtre in a way, yes. Yeah, that that's a, that's a
• 33:53
  use case that we use a lot and I think it's really useful.
• 33:56
  I I just, I think LM's are the coolest, coolest thing and I've
• 34:00
  been using LM for I, I like to say since before they were cool.
• 34:03
  I don't necessarily think they're cool now.
• 34:05
  I think Chad should be is probably cool now.
• 34:06
  I don't think L's necessarily. I've been using them since they
• 34:09
  were since they were models, since the terms came out, but I
• 34:12
  do not trust them to. There's a whole conversation
• 34:15
  whether they reason or not. But like when I build these
• 34:18
  systems and while them to be reliable, like I don't want to
• 34:21
  just like give an LLM a bunch of information and be like figure
• 34:25
  it out. It's like there's, there's
• 34:26
  mitigating strategies, but I just don't really trust it like
• 34:29
  that. So it's like, I just want to
• 34:30
  give the LM only the information it needs, no more at that point.
• 34:36
  And I'm pretty confident that it's going to fill in a really
• 34:39
  necessary part of a retrievable pipeline.
• 34:41
  Yeah. It's so funny that you mentioned
• 34:43
  that because how many times do we get the wrong answer because
• 34:48
  you've given it too much random context that has nothing to do
• 34:54
  with the actual output that we're looking for.
• 34:57
  And then like, of course it's going to hallucinate, right?
• 35:01
  Of course it's going to go off on something because especially
• 35:03
  if you're giving it in this case like 7 times or 7 / 10.
• 35:10
  Yep. Like 70.
• 35:12
  Percent is actual crap that you don't need.
• 35:15
  That is where that context poisoning comes in.
• 35:17
  That's. Me obviously like using for the
• 35:19
  sake of demonstration. I think if you, if you use
• 35:22
  ranking, if you, if you chunk really well, if you do it, I
• 35:25
  think you can mitigate that to that point.
• 35:27
  It's just that I know that even in theory and definitely in
• 35:29
  practise, in my experience like it, it, I can't trust it.
• 35:33
  Like I'm trying to give people reliable systems, right?
• 35:36
  So when I say, you know, 3 out of 10, the reality is it's
• 35:38
  probably, if you spend time really improving your, your rag
• 35:41
  process, I think it's much more likely than that.
• 35:43
  But I, it's not 100, right? And so the graph being exactly
• 35:47
  wrong, bringing it back to that, it's like, cool, if it's exactly
• 35:51
  wrong, then I know that I'm getting back the same 4 wrong
• 35:55
  ones every time, right? How do I improve that process?
• 35:58
  Instead of being like I have these mitigating strategies,
• 36:00
  it's like I have some level of determinism to fix it.
• 36:04
  O let's talk. About agents for a minute, What
• 36:06
  have you been seeing on that level and those use cases?
• 36:09
  Yeah. So we've been doing multi agents
• 36:12
  for for a while. And so we've actually been
• 36:14
  working on on a few of them. But I, I think that like there's
• 36:17
  a really interesting question that we talked about internally,
• 36:19
  like what is an agent? Because is it a Python wrapper
• 36:22
  around an open API, open AI API call?
• 36:25
  It's like probably, yeah. But also if you run a script, is
• 36:28
  that an agent like not an LA? And so it's like, what does it
• 36:31
  mean to be this? And I think the popularity is
• 36:33
  interesting. We work a lot on like structured
• 36:35
  grounding per agent. And this is how we solve a lot
• 36:37
  of our problems. Because I, I'm not of the
• 36:40
  opinion that you just give a natural language instruction to,
• 36:43
  to a system and then it just goes away and does that and
• 36:45
  nails it every time. It's like you want to break it
• 36:48
  down into its steps that you understand, you want to like
• 36:51
  decompose it down into steps and then and then orchestrate those
• 36:53
  steps. And there's some cool frameworks
• 36:55
  that help you do this. But a lot of the time, like we
• 36:58
  build what are effectively like functional programmes because
• 37:01
  like we want IO test, right? And I think programmes like,
• 37:05
  sorry, frameworks like we use Foster I Pidato instructor, like
• 37:08
  they're really useful to to build these systems out.
• 37:11
  But ultimately, like every time I make a call, I want to know or
• 37:17
  at least want to point test. And so breaking it down into
• 37:20
  agentic systems is actually a really good way to test and
• 37:24
  produce like, you know, well structured software, which we
• 37:28
  think about a lot like. So breaking it down into its
• 37:32
  components, making LLM's do as little as possible and then
• 37:35
  having many LLM calls is how we like to structure it.
• 37:38
  And I think that at its core is like a multi agent system.
• 37:41
  I was in this awesome framework. Said crew obviously are doing
• 37:43
  really cool work around like how do you do natural language
• 37:46
  instruction? But we use we definitely use
• 37:48
  line chain in some in some pieces.
• 37:50
  We don't use line chain for everything.
• 37:52
  And before I like love to use it for prototyping.
• 37:54
  There's some great parts of language that we use in
• 37:56
  production around Lang Smith, around their bedrock
• 37:59
  implementation is like a great way to test the different LMS.
• 38:02
  But when it comes to wrappers, I think they're really great for
• 38:05
  testing. When I put stuff out in
• 38:06
  production, I'm not yet ready to use the wrappers that are
• 38:09
  available. So I think that we have multi
• 38:11
  agent systems, but a lot of that is just well typed, well IO
• 38:16
  tested and well integration tested like different Python
• 38:19
  modules that wrap calls to specific LMS. And when you say?
• 38:23
  It's well tested. Is that basically, again going
• 38:26
  back to the reliability of it, that every time I ask this one
• 38:31
  question I'm going to get the same output means a few?
• 38:35
  Things so shout out to Lang Smith and how well they do
• 38:37
  provenance and that as a programme, I, I think he's
• 38:39
  underrated in terms of how well you can see the different LM
• 38:42
  calls and, and how that works. But I also mean literally to as
• 38:45
  in like it's π tested. So it's like, that is a process
• 38:49
  of like, I need certain coverage over my system.
• 38:52
  I don't like that have tested all of this, you know, well
• 38:55
  structured server and like so it's testing the IO based on
• 39:00
  questions and then breaking those down into modules.
• 39:03
  So then, for example, GBT 4 Omni comes out right and it's like,
• 39:08
  hey, forced to expensive 3.5 isn't consistent enough.
• 39:12
  This is like it's right in the middle.
• 39:14
  How do we know if it works? And then I can just change the
• 39:16
  pedantic model name and now I can just go into a dashboard and
• 39:20
  see every single step of my pipeline, conditional or
• 39:23
  otherwise, is this thing as performance is this thing.
• 39:26
  What's the latency like? So, so that's like a, an
• 39:29
  important start process that we do, but we're like in
• 39:31
  production, right? So that that requires like a
• 39:33
  level of tested software. So when I say tested, it's like
• 39:36
  this. I test to it, but there's also
• 39:38
  literal, you know, full test States and coverage around it.
• 39:42
  Yeah. Yeah, I like it.
• 39:44
  And when it comes to the agents you mentioned how each one you
• 39:50
  want to have this structured grounding, Yeah, per agent or
• 39:55
  per agentic workflow I think is what you mentioned.
• 39:57
  Yeah, breaks down exactly what that is.
• 40:00
  This is This is. I think this is really cool and
• 40:02
  I think about this a lot, but if I have a rag system, right, like
• 40:07
  if I need to, if I need to give every agent a little bit of
• 40:09
  information that doesn't already have and I can get each of those
• 40:12
  agents 95% accurate and I have a sequential workflow of five
• 40:16
  agents, that's 77% performance right on expectation.
• 40:20
  And I know that's not linearly distributed on in some cases
• 40:23
  fail every time, etcetera, etcetera.
• 40:24
  But like if you look at that 77 percent is awful, right?
• 40:27
  And if you think about it like, I don't know if I think people
• 40:30
  use the example, like what if my Slack message like didn't show
• 40:33
  up every now and then all like, what if I didn't trust like the
• 40:35
  e-mail protocols and hike did that?
• 40:37
  That's it's it. The, the level of reliability is
• 40:39
  really low. And 95 is a big ask even per
• 40:42
  agent, right? So what I think about a lot is
• 40:45
  like that rise to a farmer, rise to a chef, like, well, if I can
• 40:49
  get both of those exactly wrong and I can spend enough time kind
• 40:51
  of like fiddling with the inputs, outputs and the crud of
• 40:54
  each structured grounded agent with a schemer and a graph, then
• 40:57
  I can iteratively improve both of those to, you know, I, I
• 41:00
  guess maybe 100 it's impossible, but asymptotically like 99.
• 41:04
  And then now the propagated uncertainty is significantly
• 41:06
  less. And so when I think about multi
• 41:08
  agent systems, I think they are the future in a lot of ways
• 41:11
  because I work with workflows and agentic systems a lot, which
• 41:15
  is like, hey, I have a task that I need to do, break it down.
• 41:18
  And once you've broken it down, you have a multi agent system,
• 41:21
  right? So yes, I think the difficulty
• 41:24
  is we're about to see an explosion of, we just saw a year
• 41:29
  of Medium posts into no enterprises using this stuff,
• 41:33
  right. And it's like, and I think it's
• 41:35
  awesome and I love the exploratory nature of it, but I
• 41:38
  think we might see the same with agents, which is like, it's
• 41:41
  already not reliable enough per call to a model propagating that
• 41:47
  uncertainty or, and, and in my PhD decision making under
• 41:50
  uncertainty through Joseph Halpin.
• 41:51
  And, and I think it's like phenomenal thing that I care
• 41:53
  about a lot, but I feel like it's a massive risk, right?
• 41:57
  The turning point for me is going to be do I give my credit
• 41:59
  card to a booking agent and am I willing to just show up at the
• 42:05
  airport right once that happened or?
• 42:07
  Zeros in my bank account if I'm going to be doing that kind of
• 42:10
  shit. Yeah.
• 42:11
  So that's. It or am I gonna show up there
• 42:13
  like you booked the flight for tomorrow?
• 42:14
  Like, because what I want to say is like, hey, I'm not doing red
• 42:17
  eyes anymore, but I will get up early.
• 42:18
  I have this alliance membership, right?
• 42:21
  And and I'd prefer one stop over here.
• 42:23
  If I can't like how how do I like I should be able to write
• 42:25
  that as an instruction. The reason I don't is that I I
• 42:28
  don't trust the systems yet because I don't think each
• 42:30
  system, each agent is accurate enough.
• 42:33
  So what we work on is grounding each agent and then iteratively
• 42:37
  improving each agent, because then you can break down like a
• 42:41
  really good example. That's we're a long way away
• 42:43
  from ever getting this in, in, in through demo, let alone in
• 42:46
  practise. But like a global supply chain
• 42:49
  of like any sort of so it's like you have a shipping age and you
• 42:52
  have a procurement agent, you have a Providence agent, you
• 42:54
  have a you know, an employee agent, then you have those.
• 42:56
  Those are also different depending on which you know,
• 42:58
  nationally, where abouts you are in the world and how do you
• 43:01
  produce and how do you supply? That should be many different
• 43:04
  agents doing many different things, right.
• 43:05
  That is a completely automatable workflow in theory.
• 43:08
  Just the difficulty of that is so much in in practise.
• 43:11
  So that I look at is like the ultimate multi agent system,
• 43:15
  like international physical goods moving around as part of a
• 43:19
  supply chain. But each of those agents will
• 43:22
  have to be so specific and so accurate for you to be able to
• 43:26
  trust that at all. And yeah.
• 43:28
  Exactly. Like there's some talk about
• 43:31
  high risk. If shit doesn't work out, all of
• 43:35
  a sudden you've got a bunch of ships and shipping containers
• 43:39
  going to the wrong port or going to like, that's not good.
• 43:44
  Hey, we're about. To see I think I mean, yeah,
• 43:47
  Canada thing was already really interesting, but I, I think that
• 43:51
  if, if 2024 is the year of agents, which I've seen a few
• 43:54
  people say, I think we're going to see some mistakes before we
• 43:59
  see some really good systems. It's not physical systems,
• 44:02
  right? Because physical systems are
• 44:05
  easy to prove. But why I like agentic systems
• 44:09
  and it aligns with like the, the small graphs that we have is
• 44:12
  that then these specific person or specific domain expert who
• 44:17
  matters like who's in charge of procurement, It's like then they
• 44:20
  just deal with the procurement agent or agents and just the
• 44:23
  procurement graph or schema or grounding or, or knowledge and
• 44:27
  they don't have to deal with the rest of it.
• 44:29
  I think modularizing those systems is like a really good
• 44:31
  way to, to build these, these, these processes and automate
• 44:35
  these processes. But we're about to see people be
• 44:37
  like, again, I'll book this holiday for you.
• 44:39
  And it's like, you show up and there isn't a booking, right?
• 44:41
  That already happens. But these systems are so like,
• 44:45
  if you talk to an old head about this, like multi age, the the
• 44:48
  the multi agent system textbook that I started using three years
• 44:51
  ago to build these systems is from 2000.
• 44:54
  Like this is not a new thing, right?
• 44:56
  Like this has been around for a long time and I think the
• 45:00
  history doesn't repeat itself, but it rhymes keeps coming U or
• 45:03
  what's old is new again keeps coming up in this.
• 45:05
  And I think when you speak to like when I speak to like my
• 45:08
  supervisors about this, they're like, yeah, this is going to get
• 45:11
  it's already not good enough, but the the financial
• 45:15
  opportunity of automating larger workflows is such that people
• 45:19
  will do it. And it's easy to build more
• 45:22
  agents than it is to build a better agent right now.
• 45:25
  It's gonna get messy. Yeah, real quick.
• 45:27
  Sorry, I could talk. This for too long.
• 45:29
  We move on. Yeah, I.
• 45:30
  Well, I love this because I've had that feeling too.
• 45:33
  I made the joke like. When auto.
• 45:36
  GPT came out, it was like, yeah, everybody was talking about it.
• 45:39
  It was this huge thing. And I think people are easy to
• 45:42
  project themselves into the future and what life is going to
• 45:46
  be like when all of this works flawlessly.
• 45:49
  But to get there, that last 10% that we need to get there could
• 45:54
  take us longer than it took us to get here.
• 45:57
  And so I was making the joke of like, I've been playing with
• 46:00
  auto GPT and all that happened was I got like a $300.00 open AI
• 46:05
  bill, you know, like. Yeah, yeah, yeah, yeah.
• 46:08
  You, you made it good. You had a good post about this,
• 46:10
  right, Which is like more use actually equals more cost in a
• 46:14
  way that I, that I think people aren't talking about.
• 46:17
  And, and like I test a lot of different models locally.
• 46:19
  We, we use, I use open AI a lot through their API just because
• 46:23
  it's very easy to use. I think the models are great,
• 46:25
  but like there are so many different models, so many
• 46:26
  different settings, so many opportunities to fine tune.
• 46:28
  I think there's different levels of models right now for sure.
• 46:32
  But yes, a thought experiment for you because I've, I've asked
• 46:36
  a few people about this multi agent systems, I think in theory
• 46:41
  and currently in prototype, they don't break down in different
• 46:44
  places. They often break down in the
• 46:46
  same place, right, or the same place as the most uncertain with
• 46:50
  the booking example, right, there would be an agent that
• 46:53
  goes and gets the latest flights.
• 46:55
  There'd be an agent that goes and gets the latest hotels.
• 46:56
  There'd be an agent that goes and gets the latest goes and
• 46:59
  gets the latest, like, you know, car booking or whatever.
• 47:01
  There's an agent that you know, maybe take your nationality and
• 47:04
  make sure you can go there without a visa, whatever it may
• 47:05
  be, like automates this whole process.
• 47:09
  I wouldn't trust it because I don't think it's going to make
• 47:11
  the best decision with all that information.
• 47:13
  But if I could write my own agent and bring it to that
• 47:15
  system, maybe I would like would you, if you could control your
• 47:20
  right, you could see it, right. So we think a lot about this in
• 47:23
  a I like the master apprentice model.
• 47:25
  Jason left a really good post about this recently around here.
• 47:27
  And we do, we actually do this, which is like a lot of the, the
• 47:30
  people we work with, it's like it's report generation that they
• 47:32
  can review. So would you bring your own
• 47:34
  agent? Like if you could bring your own
• 47:36
  agent, would you? Do you think you would be more
• 47:39
  likely to use that system? And justice almost like train
• 47:43
  the other agents how to do it the other.
• 47:45
  Agents that they're all fully described as an API.
• 47:48
  Maybe like it's perfectly retrievable, it's small like.
• 47:52
  Is the problem that you don't get to control the middle of the
• 47:55
  workflow? Do you trust yourself more than
• 47:58
  you trust them? Yeah, no.
• 48:00
  I still am very much like you. Like I don't trust him at all.
• 48:06
  Yeah. And so.
• 48:08
  I would much rather, even if it's it's funny because it is
• 48:11
  very much like the time versus comfort or convenience tradeoff.
• 48:16
  And right now I am. I would rather spend the time
• 48:21
  and do it right than that tradeoff because it's the same
• 48:25
  with like. Self driving.
• 48:28
  Cars right? Yeah, I I feel like self driving
• 48:31
  cars. I am a little bit more apt to
• 48:34
  let the computer take over the wheel, but with the agents I I'm
• 48:38
  like I need more zeros in my bank account.
• 48:41
  Cool. Again, this is that's the answer
• 48:43
  I want. So a friend of mine called Linus
• 48:45
  Eckenstein, who's he been a friend of mine for a long time,
• 48:48
  but he's he's very popular on Twitter and this stuff was
• 48:51
  recently doing a talk here. I got to see him and not to tell
• 48:54
  too many tales out of school, but he has some.
• 48:56
  He's talking to his children and he was like, they went in a self
• 49:00
  driving car, right? And they were like, why are you
• 49:03
  driving? They're young, young children,
• 49:05
  right? So they were like and he's like,
• 49:08
  well, I need a steering wheel like I need the control.
• 49:11
  And they were like, you don't need the control.
• 49:13
  O what does the future look like with agents?
• 49:16
  If you and I are like, well, I'm not willing to give the Wheeler.
• 49:20
  But. If the new generation is willing
• 49:23
  to give the wheel up, then what are these systems look like?
• 49:25
  Because like, I don't trust that system, but does someone 10
• 49:28
  years younger than me trust that system?
• 49:29
  And are they a market? It's like, yeah.
• 49:31
  So I actually think expectation is really important with this
• 49:34
  stuff. And I think we may be at the
• 49:35
  stage where people are willing to do that as, as as a booking.
• 49:39
  I don't know. I, I think so Tim French, who
• 49:42
  was my main supervisor, who I talk about a lot.
• 49:44
  He, I don't know if this quotes his, but he was the first I
• 49:47
  first heard it from him. Which is like in, in academia,
• 49:52
  you write drunk and you edit sober.
• 49:54
  It's like the way to think about it, right?
• 49:55
  And then you go out for a beer and you're like, you write a
• 49:58
  bunch. So you remove inhibition, wake
• 50:00
  up the next morning, cold light of day.
• 50:01
  You look at it like this is the worst thing I've ever seen.
• 50:03
  And it's just much easier to edit than it is to, to write in
• 50:06
  a lot of ways, right. And, and I, I think of LM's as
• 50:09
  drunk riders a lot, right? They'll produce content no
• 50:12
  matter what through prompting. And then I'm happy to edit it.
• 50:14
  And so then if you, if you carry that through to a multi agent
• 50:18
  system to a booking, there's a level of drunk writing that I'm
• 50:23
  willing to edit, right? There is a level of information
• 50:26
  retrieval of cost of flights, cost of etcetera.
• 50:29
  So I I would be interested in a multi agent framework that is 2
• 50:34
  parts. One is my control and the 2nd is
• 50:37
  their control. And then personally I give and
• 50:40
  take because right now nothing's automatable because no one
• 50:43
  trusts it right as a broad point but some people trust it.
• 50:46
  There is a generation coming next who's going to trust it.
• 50:49
  There is a generation who has the most wealth who won't trust
• 50:52
  any of it, right? So, but what does it look like
• 50:55
  to personalise that? And I think that's a fascinating
• 50:56
  question of the sliding scale personally, of how much he
• 51:00
  trusts these systems. It's funny.
• 51:03
  That you mentioned that because I was literally today just
• 51:05
  listening to another podcast about how.
• 51:09
  Like sometimes. Removing yourself from your
• 51:12
  business can be the best thing to make the business grow.
• 51:16
  And it's almost like the the parallel here is like removing
• 51:20
  myself from this agent or all of these agents can be what
• 51:25
  actually makes it work and work better in that thought
• 51:29
  experiment. So I find that I find that
• 51:32
  fascinating. I do think that.
• 51:35
  You're always. Going to have to have the well,
• 51:39
  maybe not always You're going to want to see, you know, like
• 51:43
  you're going to want to see what's happening as it's
• 51:45
  happening just to kind of double check and know that it happened
• 51:52
  almost. And but I guess you could one
• 51:55
  part of seeing it could just be that you get the confirmation in
• 51:58
  your e-mail and you look at it and you go, alright, cool
• 52:02
  flights tomorrow then you show. Up to your first one and it
• 52:04
  works right. Again, I don't know how many of
• 52:07
  these going to work, but there there is a level, there's we all
• 52:09
  have a personal level of risk tolerance.
• 52:11
  And I think that there is interesting products that are
• 52:14
  going to be interesting systems in a friend of as a friend of
• 52:17
  mine who is post Series A I, I think I'm gonna leave them
• 52:22
  nameless just in case it's a problem.
• 52:23
  I don't think it would be, but they were like my job daily is
• 52:27
  I'm just a traffic controller, right?
• 52:29
  So like I don't do anything at anymore, right?
• 52:33
  And then deeply channel, but they've got to the point they're
• 52:35
  like, I'm really just a traffic controller, right?
• 52:37
  So I'm the most important person in the company because if I
• 52:39
  leave, everything crashes, but I don't do anything.
• 52:42
  And it's like I think about that with these systems.
• 52:44
  It's like, well, if you are just able, like what level of
• 52:47
  personal traffic control are you comfortable with?
• 52:49
  What level? Yeah.
• 52:50
  OK, so when you say removing yourself from your business,
• 52:52
  there's tears to it, right? So you remove yourself from the
• 52:54
  operations, but you're very necessary in terms of direction
• 52:57
  and control, whether that direction is immediate and in
• 53:01
  the moment or whether that direction those months ago
• 53:03
  through some level of SOP like I still think it's necessary to
• 53:06
  have that control and I think these systems are very similar.
• 53:10
  And where have you? Seen agents in your work
• 53:13
  actually working or have you found one?
• 53:17
  Best example. Was from GBT 3 and I think about
• 53:20
  this over and over and over and over and over again because it's
• 53:22
  like, how do you what we think a lot about like our North Star is
• 53:25
  getting people to production. And so we want to be the the
• 53:29
  factor for that. And I think the answer a lot of
• 53:30
  that time is some level of services and scoping in to get
• 53:33
  to the point. But like, what does it mean to
• 53:35
  put LM's in an existing revenue stream and speed it up?
• 53:39
  For example, when we talk about ideology, if if you produce a
• 53:42
  report faster and you get paid per report, then now it's like,
• 53:45
  well, if you have the master apprentice model, speed up the
• 53:48
  apprentices, right? Because there there was already
• 53:51
  some work being done in the legal example, speed up the
• 53:54
  speed up the apprentices. It was GPT three.
• 53:57
  It was a post and it was it was a user.
• 53:59
  I can't remember, but it was called writing GPT, right?
• 54:02
  This would have been early last year and it was an individual.
• 54:05
  I can't remember their name and it wasn't like a super popular
• 54:07
  post on medium, but it's going to be somewhere.
• 54:10
  I'm sure maybe, maybe it's gone really popular since I read it.
• 54:13
  But they were a photographer. They got early access to GPT 3.
• 54:16
  They were able to develop and they said I'm going to build
• 54:19
  several different. They didn't call them agents.
• 54:21
  They called it like almost like a board of directors.
• 54:23
  They were like, I'm going to prompt it and say, write me an
• 54:27
  article about this thing. Then I'm going to have a editor,
• 54:31
  an editor, right? And they're like, why can't I
• 54:32
  extend this? I'm going to have a grammar
• 54:34
  editor. I'm going to have a
• 54:36
  colloquialism editor. I'm going to have an EO editor.
• 54:38
  And then what I'm going to do is I'm going to say, I'm going to
• 54:41
  have another agent that is going to tell me where to put a photo,
• 54:45
  right? Where, where does the photo
• 54:46
  exist? So then they could be like, what
• 54:49
  are the top 10 washing machines of 2023, right.
• 54:52
  And three hours later, because at the end, it costs a bit of
• 54:55
  money, but three hours later, at the time, you'd have these like
• 54:57
  reinforced systems that would communicate with each other.
• 54:59
  And at the end they get a piece of writing that is colloquial,
• 55:04
  that is SEO optimised, that is well added and has good grammar.
• 55:07
  Then they would go out and take the photos.
• 55:09
  So it's an original piece of content now.
• 55:11
  Ohh, nice. And then they would get
• 55:12
  themselves to the top of like SEO blog stuff.
• 55:14
  And now they're making ad revenue.
• 55:16
  And so they can do this with anything they can do with
• 55:18
  washing machines, can do with tents.
• 55:20
  That was the coolest example I've I still haven't seen an
• 55:23
  example better than that because it was such a simple use and it
• 55:26
  was like, even if this doesn't work fully, like it doesn't need
• 55:29
  to be probabilistically, like it doesn't need to be perfect.
• 55:32
  They've created it. I think about this so much as
• 55:35
  like the board of directors, how do you do this?
• 55:38
  How do you reinforce? How often do you do it?
• 55:39
  But that to me was like a real trigger that I was, and this is
• 55:43
  before auto GT. So when that came out, I was
• 55:46
  like, this is going to explode. And then we never really saw it
• 55:47
  because of the probabilistic nature.
• 55:49
  But that was the coolest to me. That was like a real unlock.
• 55:54
  So talk to me. About why, how in these last
• 55:56
  couple of minutes and just. What it is?
• 55:59
  You're doing, because I from this conversation, I imagine a
• 56:02
  lot of people can gather that you know a bit about knowledge
• 56:06
  graphs. But it's I think the really
• 56:09
  interesting thing that you just said right there is you're
• 56:12
  highly focused on getting people to production.
• 56:15
  Yeah. Yes.
• 56:17
  So hot take. I don't know if people riding
• 56:22
  natural language and getting an answer back is the future of
• 56:25
  this stuff. I think a lot of people are
• 56:27
  like, I want to get my rag perfect.
• 56:28
  I don't think anyone really thought about what happens if
• 56:30
  they do. I think everyone stuck on that
• 56:31
  problem. And even if I think they get
• 56:33
  there, I'm not even sure what happened.
• 56:34
  But there are valuable business problems to solve, right?
• 56:38
  And as a business you want, I want to work on those problems.
• 56:42
  Like I, I've worked in, in tech for a long time, but I've worked
• 56:45
  in a lot of like heavy industry as a technical person, as a
• 56:49
  machine learning engineer and a large mining company.
• 56:51
  And so there are, they have a high reliability requirement.
• 56:55
  This valuable problem is interesting problems to solve
• 56:57
  there. So when we talk about getting
• 56:59
  people into production, we use knowledge graphs as a means to
• 57:01
  an end, as part of a broader workflow process to reliably get
• 57:07
  back the information that you asked for, right, consistently
• 57:10
  and reliably and overtime. And we do that through.
• 57:14
  Currently we have a closed beta, but it's going pretty well.
• 57:16
  We've had that out for a little while.
• 57:17
  We're releasing features like a couple of times a week.
• 57:20
  We're just listening to people talking to in our Discord.
• 57:23
  And so we're helping people and it's it's developer tooling for
• 57:26
  knowledge graphs. So it's helping people create
• 57:29
  these systems. It can give them more
• 57:30
  reliability. There's kind of broadly two
• 57:33
  types of production at the moment.
• 57:34
  One is like, I want you to improve my existing systems, but
• 57:38
  I don't know if there's that many people who actually have
• 57:40
  recused cases. Existing systems.
• 57:42
  I think a lot of people see us as a means to this is a
• 57:46
  difficult thing to say. It means to production, but not
• 57:49
  to, not to jump on the historical reference of that
• 57:51
  term, but there are way to get to a level of production because
• 57:54
  without a level of reliability, without a, without a low bar of
• 57:57
  reliability, we can't put this system in the wild.
• 57:59
  And so we work with graphs because we see them as the
• 58:04
  weakest part of the overall pipeline.
• 58:07
  And so we allow people to unlock that part, which then raises
• 58:11
  everything to the point that now we have a reliable system and,
• 58:15
  and so that's what we're working on.
• 58:18
  And that's how you. Can help people get into
• 58:20
  production. That makes a lot of sense, and I
• 58:22
  do. Man, it's.
• 58:24
  Like you're preaching to the choir here because I've felt so
• 58:27
  much of that Also where it's like alright cool the rag is
• 58:31
  fully optimised but turns out nobody actually uses the chat
• 58:35
  bot on our website so great job people haven't.
• 58:39
  Thought about that, like what does it mean to build the
• 58:42
  perfect chat bot? And it's like, do what really?
• 58:45
  What does it mean? And it's like, so for example, I
• 58:47
  worked with a large oil and gas company, but back when I was a
• 58:50
  consultant before I was doing this, this business and I was
• 58:53
  helping them build a leave policy like a leave, but someone
• 58:56
  could be like, how many weeks do I get off for paternity leave?
• 58:58
  It's give me the answer hallucinated like immediately,
• 59:00
  right? And they're like, and then
• 59:02
  everyone's like, Oh, this sucks. And so then it got scrapped as
• 59:04
  an internal project. They spent a bunch of money.
• 59:06
  We, we caught it early because I came in as a consultant when it
• 59:09
  was working. And I was like, this is never
• 59:10
  gonna work. By the way, if it did like, what
• 59:13
  did they solve? Yeah, it's like a.
• 59:15
  A little bit like not like it's not like a massive.
• 59:17
  So that I think that's an interesting way to think about
• 59:19
  this like if rag worked really well.
• 59:22
  So This is why we talk about multi agents because if they
• 59:27
  just ask you a question, get an answer and retrieval, I think
• 59:30
  he's like can be augmented really well.
• 59:32
  I love some of the search stuff coming out.
• 59:33
  I'm really looking forward to the the urban eye products that
• 59:36
  come out as a as a Mac user. But I do think that that's why
• 59:40
  agentic systems are the future, because if you can reliably
• 59:43
  create module after module after module after module, then you
• 59:46
  can just like Marcio Gillies to say this.
• 59:49
  It's like he was a lawyer for a while and he's like most white
• 59:53
  collar jobs are just like information retrieval and then
• 59:56
  reasoning and then information retrieval and then reasoning.
• 59:58
  And it's like that. You can kind of like you go
• 1:00:00
  away, you read some stuff, you make a thought, you give it a
• 1:00:02
  summary. And so like it's, it's workflow
• 1:00:05
  orchestration and it's these systems of creation that I think
• 1:00:08
  that if rag was 100% right, which means the agent is
• 1:00:12
  reliable. What does it look like to make
• 1:00:15
  a, you know, agent by agent composed workflow That I think
• 1:00:20
  is, is really interesting. And I and I think that part is
• 1:00:22
  why people are saying. Agents of the future.
• 1:00:25
  Those modular. Boxes.
• 1:00:26
  Yeah. And there's a lot that you said
• 1:00:30
  here I'm trying to take in. I think I'm gonna have to listen
• 1:00:32
  to this one a few times and just soak it up because it does feel
• 1:00:36
  like. There was some.
• 1:00:38
  Knowledge dropped on me here and I really appreciate all of the
• 1:00:43
  IT, the wisdom that you're bringing to this.
• 1:00:45
  It's super cool to see just that idea of OK, a chat bot can be
• 1:00:52
  useful, but how much business value is it actually bringing or
• 1:00:56
  what does it accomplish versus when you were talking about like
• 1:00:59
  the doctor getting? Their process.
• 1:01:02
  Speed up and I know there's a tonne of tools out there right
• 1:01:07
  now where they'll. Record.
• 1:01:10
  The conversation with the patient and the doctor and then
• 1:01:14
  like half or 90% of the doctor's time after they're with patients
• 1:01:19
  is just filling in all of the crap, all that paperwork that
• 1:01:23
  they have to do to get the insurance claims and whatnot.
• 1:01:26
  And so if you can speed that up, then they can see more patients,
• 1:01:30
  they can make more money. And that's actual, like there's
• 1:01:34
  very clear return on investment there.
• 1:01:37
  Not yeah. Like this idea of ohh, yeah, I
• 1:01:41
  now know that I have 11 holidays left.
• 1:01:44
  Yeah. It's like, I don't know, like I,
• 1:01:46
  I just, I think there are really valuable business.
• 1:01:48
  There's a lot of like win win automation opportunities in
• 1:01:53
  heavy industry. Definitely.
• 1:01:55
  I think Healthcare is a, a really, it seems to be an
• 1:01:57
  obvious one for it's well regulated industry.
• 1:01:59
  But you know, the people we speak to that we get purely
• 1:02:01
  inbound, like the variety of like, you know, earthquake
• 1:02:05
  preparation for hospitals in California and stuff.
• 1:02:08
  And like how valuable those problems aren't, how difficult
• 1:02:10
  they are to solve, but how, you know, in theory automatable they
• 1:02:12
  are and how modular they are is problems is like very
• 1:02:15
  interesting with this stuff. So yeah, I think those are the
• 1:02:19
  those are the interesting problems that that we really
• 1:02:21
  like to work on. I appreciate you saying wisdom
• 1:02:23
  as well. I'm not going to pretend that I
• 1:02:25
  I have a I've just, I work on this stuff a lot.
• 1:02:27
  So I have a bunch of comments. I want to thank Paco as well.
• 1:02:30
  He's the one who introduced us. And so that's genuine, genuine
• 1:02:33
  wisdom there. I, I think it is the type of
• 1:02:35
  person that I, I really enjoy speaking to, but yeah, that
• 1:02:38
  that's, that's how I see this shaking out.
• 1:02:40
  Maybe not loudly because I think it's difficult to write a post
• 1:02:44
  about it, but I think quietly and valuably is these processes.
• 1:02:48
  It's like, what was Linux gonna be when Linux released it?
• 1:02:52
  And it was like the new operating system.
• 1:02:54
  And what does it do now? Power every server that powers
• 1:02:57
  every GPU, the powers every, you know, like that.
• 1:02:59
  That's you think about these agent systems like those are the
• 1:03:02
  interesting problems I think. You said.
• 1:03:05
  Maybe quietly. I'm not going to write a post
• 1:03:07
  about it. And I just respond with hold my
• 1:03:09
  beer. Let's give back to you.
• 1:03:12
  Yeah, give back to you in a week.
• 1:03:14
  Let you know what I can do here. There's a lot of fodder for me
• 1:03:18
  to write some shit on LinkedIn about this.
• 1:03:21
  This is great, man. I really appreciate it.
• 1:03:23
  And I am going to take away this idea of be exactly wrong because
• 1:03:28
  if you can be exactly wrong, I really like that.
• 1:03:30
  That's where determinism starts and you get out of the weeds of
• 1:03:34
  this probabilistic. Well, it could be right, It
• 1:03:37
  might not be, but we'll see. That is huge.
• 1:03:41
  Some amazing. Use cases for probability.
• 1:03:43
  Some awesome stuff happening, but the problems I'm working on
• 1:03:46
  there, they can't be probabilistic problems.
• 1:03:48
  Excellent. Well, I appreciate this, Tom.
• 1:03:51
  Thanks so much for coming on here.
• 1:03:52
  If anybody wants to reach out to you, I know that you're on
• 1:03:54
  LinkedIn, so we'll leave your LinkedIn link in the description
• 1:03:59
  and or just go to why how dot AI and yeah find out discord server
• 1:04:06
  We got discord. We haven't newsletter, if you're
• 1:04:08
  not a Discord person, a newsletter is going well.
• 1:04:10
  CEO Cheers. Cranking out content.
• 1:04:12
  We're writing as much as we can about our learnings, doing our
• 1:04:15
  best to kind of build and show people what we're working on.
• 1:04:17
  So yeah, that is. Awesome, but I love it.
• 1:04:20
  Well, you've got yourself one new subscriber in me, so thank
• 1:04:24
  you very much. Thanks to Town for this.
• 1:04:26
  No. Worries man.