Latest / Agentic Conversations (formally mlops.community) / Unleashing Unconstrained News Knowledge Graphs to Combat Misinformation // Robert Caulk // #279

Transcript

Unleashing Unconstrained News Knowledge Graphs to Combat Misinformation // Robert Caulk // #279 · Agentic Conversations (formally mlops.community)

• 0:00
  Robert Calk I'm I'm the CEO of Emergent Methods and I take my
• 0:03
  coffee black. Knowledge graphs.
• 0:07
  We're talking knowledge graphs today with Rob.
• 0:09
  Welcome back to the Envelopes Community podcast.
• 0:11
  I'm your host, Dimitrios. And just as it says on the tin,
• 0:16
  we are all about them. Knowledge graphs.
• 0:19
  The conversation with Rob centred around how he ingests
• 0:25
  thousands of new sources, how he uses a combination of vector
• 0:31
  search, keyword search, LLMSAI and of course knowledge graphs
• 0:37
  to give two sides to every news story.
• 0:44
  I thought it was fascinating, especially when he went into the
• 0:46
  part about spinning up very small knowledge graphs for the
• 0:50
  end user and then blowing them up when the user is done with
• 0:54
  the query. As always, if you like it, tell
• 0:57
  one friend. That's all we ask.
• 0:59
  And last but not least, there's a really cool initiative going
• 1:02
  on in the ML OS community, which is you can, if you want, just
• 1:08
  meet somebody else in the ML OPS community.
• 1:10
  We're doing curated intros. So if you want to do that, reach
• 1:14
  out to me because you get to meet somebody else in the ML OPS
• 1:17
  community that is also looking for a new friend.
• 1:21
  And you can choose, do you want to meet somebody that's about
• 1:24
  your same seniority level or maybe higher?
• 1:27
  Maybe you want somebody like a mentor.
• 1:30
  And I think that is really cool. I got to give a shout out to the
• 1:36
  folks that are running that initiative.
• 1:39
  This is all the noise brainchild.
• 1:41
  We've got about 90 people doing it right now and only two people
• 1:48
  have not repeated in doing it after the second round.
• 1:53
  So we've done these twice already, two different rounds of
• 1:57
  introductions. Two people said no, they didn't
• 2:01
  do it, but that was only because they couldn't meet and so they
• 2:04
  didn't have the time for it. If you want to do it, hit me up.
• 2:06
  We'll make sure it happens. And let's jump into this
• 2:09
  conversation. Knowledge graphs and the
• 2:18
  ontology. You said there's there's kind of
• 2:20
  some hot takes you got. Start with saying, Hey, I'm Rob,
• 2:23
  I'm, I'm the founder of Ask News and you know, we're, we're
• 2:26
  building this, this data source, which is real time data source
• 2:29
  for news. And I'm sure I'll chat a bunch
• 2:31
  about that. But we kind of stumbled into
• 2:34
  knowledge graphs and stumbled into ontologies.
• 2:36
  And so, you know, probably our opinion has some, some temporal
• 2:41
  bias, right? Because depending on when you
• 2:43
  enter the one of these domains, you have some bias about the
• 2:47
  latest. It's kind of like if you're
• 2:50
  young and you, you are listening to a song that's actually a
• 2:53
  remake of a song from the 80s, but the first one you heard is
• 2:56
  the though the new one. And then you hear the original
• 2:59
  one. You're like, it's just not so
• 3:01
  good. It's kind of the same when it
• 3:03
  comes to, to ontology AI domains.
• 3:07
  And so we stumbled into, you know, should do we build an
• 3:11
  ontology for this massive news knowledge graph that we're
• 3:14
  building? Or do we go ontology free?
• 3:18
  And the at the end of the day, all the signs pointed to
• 3:21
  ontology free because of our end user and what they needed from
• 3:25
  it doesn't mean that the ontologies don't have a great
• 3:28
  use case still and always will. But when you, you enter this new
• 3:33
  world of AI interactions with AI interactions with people and
• 3:39
  trying to convey information and, and it's very different
• 3:42
  from interacting with a graph database in 2018 when it was,
• 3:47
  hey, you know, back then with an ontology.
• 3:50
  I've got my Trump is a part of this article, which is a part of
• 3:54
  this publication, which was published on this day in this
• 3:58
  month. That's like an ontology and you
• 4:00
  could potentially navigate through to get to other people a
• 4:03
  part of similar. But then when you're you're
• 4:06
  trying to convey information quickly to journalist or analyst
• 4:12
  or another LLM, you're missing some of the high resolution
• 4:15
  relationships. At that point, you're you know,
• 4:18
  Trump is a part of this article, which is a part of this
• 4:21
  publisher. And you know, Biden is a part of
• 4:23
  this article, which is also a part of the publisher.
• 4:26
  Or maybe he's a part of the same article.
• 4:27
  But you don't really know how Trump and Biden relate.
• 4:32
  And that's where we've stumbled into what if we go ontology free
• 4:36
  and let the relationships kind of form themselves.
• 4:40
  And so can you explain what the difference is there?
• 4:42
  So ontology is just basically all the metadata around the
• 4:49
  certain subjects or whatever it is that you're looking at.
• 4:53
  Yeah, so. You know, traditional ontology
• 4:55
  would be a very structured knowledge representation of your
• 5:00
  domain. Might be your company might be
• 5:02
  whatever domain you're trying to model and it by, you know,
• 5:07
  having the strict meditated definitions like you're saying
• 5:09
  it, it allows you to quickly traverse and quickly philtre
• 5:12
  efficiently really large swaths of data because you've you've
• 5:18
  done that metadata filtering and you've assigned relationships
• 5:21
  which are not assignable in any other way besides knowing that
• 5:24
  they have similar metadata. So you know, that's great, but
• 5:30
  it misses the but you have these very defined relationships.
• 5:33
  Trump is a part of this article. This article was published on
• 5:39
  this day. So published on is defined a
• 5:42
  part of is defined the article type the person.
• 5:47
  But what about like, you know, Trump defeated Kamala, right,
• 5:53
  defeated. Now you have to add a new
• 5:55
  relationship. You have to then add that into
• 5:57
  your ontology and be ready for it at all times.
• 6:00
  But then you kind of bring that to the limit case and it's just
• 6:05
  an unlimited number of possible relationships between people,
• 6:08
  organisations. You know, Kamala can be a part
• 6:11
  of the Democratic Party, Trump can be a part of the whatever
• 6:15
  MAGA. And so by kind of leveraging
• 6:20
  this high level rat reasoning and intelligence, which LMS have
• 6:24
  now ushered in, you can now start defining relationships in
• 6:28
  a high resolution. And of this, you know, there's a
• 6:31
  big weakness and we'll talk about that, but there's a big
• 6:33
  strength. The strength is with this high
• 6:36
  resolution, I can now look at a very small piece of my graph and
• 6:42
  quickly consume that information as an analyst or pass that to an
• 6:46
  LLM in a very, very prompt, token optimised way, right?
• 6:51
  Because instead of a full paragraph, which is like, oh,
• 6:53
  Kamala got defeated by Trump and then Trump joined MAGA, blah,
• 6:57
  blah, blah, blah, all these extra nouns and verbs.
• 6:59
  Instead, it's literally a Jason structure passed to an LLM.
• 7:03
  You've now reduced your token usage and you've, you've,
• 7:06
  you've, you've actually magnified the most important
• 7:09
  relationships for the LLM to move forward on to glean
• 7:13
  insights. So now you're just working in a
• 7:14
  much different world than the previous 2018 ontology
• 7:18
  structures. So it, it doesn't the, it's,
• 7:21
  it's almost just a different type of approach.
• 7:24
  But the guys that are using traditional knowledge graphs and
• 7:28
  ontologies, they haven't quite, you know, grasped this, this
• 7:33
  idea that maybe there is a new use case for for knowledge
• 7:36
  graphs that wasn't quite so useful back then.
• 7:39
  And so that's I mean, I don't know if that.
• 7:41
  Clarifies the question. Yeah, it's basically you're
• 7:44
  giving high definition relationships in a way as
• 7:49
  opposed to lots of noise and you have to philtre through all that
• 7:52
  noise to get what you're looking for.
• 7:55
  You're trying to say what are the most important relations
• 7:59
  that we have with whatever the subject is and then we can
• 8:04
  philtre from there. And I guess the main thing that
• 8:09
  comes to my mind is how you were saying you have almost infinite
• 8:13
  or infinite -1 amount of relations and edge cases where
• 8:20
  there's just that long tail of relations that you can have when
• 8:24
  you're doing it the ontology way.
• 8:26
  How does noun with no ontology void that?
• 8:34
  Avoid the? How does it avoid what?
• 8:36
  The the like infinite relations problem or do you still?
• 8:40
  Have that, yeah. So the infinite relations
• 8:43
  problem means that your database is just kind of screwed if
• 8:47
  you're going to try to model it with the same type of database,
• 8:49
  right? Because the database, the graph
• 8:52
  databases are not designed to handle an infinite number of
• 8:55
  relationships coming in because they they're indexing in a way
• 8:59
  that allows you to philtre. So you have to approach the
• 9:02
  problem from a different side if you're going to do something
• 9:04
  like this, which means a hierarchy of databases and some
• 9:10
  of them are not graph databases at all, right.
• 9:12
  And so and you know, for us, a lot, a lot of what we do is just
• 9:15
  storing a lot of high fidelity information and then building
• 9:19
  that into and then putting that into a graph database with a
• 9:23
  finite number of relationships and then traversing around.
• 9:27
  So you know, you're, you're looking at the, the world's
• 9:30
  largest news knowledge graph sitting on S3 more or less.
• 9:33
  And then using some traditional traversals or traditional
• 9:38
  indexing and retrieval methodologies and then finding,
• 9:41
  OK, this is the part I want because nobody wants to view a
• 9:44
  million relationships all in one graph.
• 9:46
  It's just useless. So that's, that's how you kind
• 9:49
  of solve it. Then you can cache it.
• 9:51
  And I, I don't, I mean, I don't work at all with mem graph, but
• 9:54
  I'll plug them. It's a great cache of a date of
• 9:57
  it. It's almost like designed for
• 9:59
  this use case because it's designed to be run in, in memory
• 10:03
  and that allows you to do very high performance cache movement
• 10:07
  in and out very quickly. So you're still interacting with
• 10:10
  the data as if you in the same the same type of queries, but
• 10:16
  you're not having to deal with that massive of infinite number
• 10:19
  of relationships if it makes sense.
• 10:22
  Actually I've heard Memgraph being mentioned before and the
• 10:27
  way that I think I heard it described was that it's almost
• 10:31
  like a duck DB of graph databases.
• 10:36
  So what are the benefits there of Memgraph is just small and
• 10:41
  fast and like why do you like it?
• 10:43
  Well, I like it because it's open source fully there.
• 10:47
  A lot of you get involved in Neo 4 J It's really, it's really
• 10:52
  geared at funnelling you to enterprise.
• 10:55
  And I come from an open source background.
• 10:59
  So I'm, you know, everything we do is open source and everything
• 11:01
  we consume is open source. And then getting, you know, if,
• 11:05
  if Neo 4 J's enterprise product really actually was compelling,
• 11:10
  then OK, I get it. But mem graphs open source is
• 11:14
  actually compelling and fully functional.
• 11:17
  There are some additional features as always, but
• 11:20
  generally speaking, like Quadrant, the base database that
• 11:23
  is open source is extremely usable in production to.
• 11:26
  So to me that's a big distinction.
• 11:28
  But the design, the underlying design, and I'm really not an
• 11:31
  engineer expert. I haven't even reviewed their
• 11:33
  source code or anything, but what I know from my bird's eye
• 11:37
  view of it and reading docs and community and interacting with
• 11:41
  their community, it's, yeah, they've really focused on
• 11:44
  performance. So using in memory.
• 11:46
  And so I mean, you know, the difference between storing stuff
• 11:49
  in memory and disc, it's going to be a lot faster.
• 11:51
  And Neo 4J keeps everything on disc.
• 11:54
  It's using cache most likely. But Mem Graph took kind of the
• 11:58
  reverse approach. It said everything's in memory
• 12:00
  and now if you want to try to store stuff on disc, it's kind
• 12:03
  of a feature. And people do want to do that as
• 12:06
  well for much larger graphs. But NUM NUM graph, I think kind
• 12:11
  of caught on to the idea that you don't actually need a
• 12:14
  massive graph. What you really want is the
• 12:16
  right graph. And that can be constituted of,
• 12:21
  you know, 100 nodes. But as long as it's the right
• 12:23
  graph and then you can traverse it and philtre it the right way,
• 12:27
  it becomes very valuable. And doing it quickly and on the
• 12:30
  fly, like we're doing loading stuff in, deleting it, like
• 12:33
  we'll just, we'll literally create a database and then just
• 12:36
  delete it, right? Because it's, when we're done
• 12:38
  with it, we're done with it. So it's, it's a really
• 12:40
  interesting use case of it. It fits really well in that
• 12:45
  paradigm. Yeah, it's ephemeral in that
• 12:49
  sense. OK, I like that.
• 12:51
  And it makes sense why it's in a way you can compare it to Duck
• 12:55
  DB. And because it's more like,
• 12:58
  yeah, small graphs, you don't need to boil the ocean.
• 13:02
  You just need the graphs that are important for you and for
• 13:05
  the time that it's important for you.
• 13:06
  And then you're done. And it almost reminds me of of
• 13:09
  Reddis too, how they took that approach of, hey, let's cash
• 13:13
  things, let's make it in memory, let's make it really fast and
• 13:17
  let's get you what you need. If that's your use case, then go
• 13:22
  for it. Yeah, exactly.
• 13:25
  No, I think, I think it's a really fun time to watch all of
• 13:27
  the different databases and services, choose their features
• 13:31
  and even us as a company trying to choose what we offer.
• 13:35
  And that's what I realised is the most difficult aspect of
• 13:38
  interacting in the AI industry right now is deciding, you know,
• 13:42
  what people appear to want, but what people might actually need.
• 13:46
  And that is a really difficult thing to connect.
• 13:48
  So I admire Mem Graph for kind of taking a slightly non
• 13:54
  conventional route a while ago, even before AI, they were
• 13:57
  pushing the sort of, but there's there's this other aspect of
• 14:01
  data graph DBS where maybe you don't even need them depending
• 14:05
  on what you're really doing. You can still do graph rag
• 14:09
  without a graph DB. And that's where I think
• 14:12
  there's, I'm really, I'm watching the, the ecosystem to
• 14:16
  see where that goes. Because you know, the, the Graf
• 14:22
  TV itself is very expensive to run.
• 14:24
  It is very computationally expensive.
• 14:27
  And if you can get away without it, it's, it's kind of like it's
• 14:31
  much better for Grafrag, especially for speed and stuff.
• 14:34
  So that's part of the industry. I don't know how it's going to
• 14:37
  go. I would love to watch and see
• 14:39
  what they do. But how do you even envision
• 14:43
  that happening? And what makes you think that is
• 14:46
  moving in that direction and that's a possibility in the
• 14:48
  future? I mean, at the end of the day,
• 14:51
  you can simulate a graph database with Postgres tables,
• 14:55
  right and and and link lists. It's it's more about what the
• 15:00
  graph DV does is it makes that multi hop query really nice and
• 15:03
  easy and fluid and and the UI is there and computationally
• 15:07
  effective. The index itself has prepared
• 15:09
  you for these multi hops, but it but truly all you're just
• 15:13
  connecting metadata on documents, right?
• 15:16
  And so you can you could build a graph DB at in Postgres.
• 15:20
  A lot of people do it. And if you don't require have
• 15:24
  that demand of these very complex queries, you might
• 15:28
  actually be better off with just Postgres, which is wild to think
• 15:31
  about, right? Because we're in this world of
• 15:33
  like everyone says, oh, vector DBS, graph DBS, no, no sequel
• 15:39
  DBS, all of the and then add the vectors to the no sequel, add
• 15:42
  the graph to the like. It's just like at some point you
• 15:46
  sometimes you can actually simplify things a little bit by
• 15:49
  just saying, Hey, this is my metadata.
• 15:51
  I have 4 points of metadata. I want to be able to do a couple
• 15:54
  hops. I don't need to do something
• 15:56
  wild. So maybe I just custom engineer
• 15:58
  it. Maybe that's how RAG goes,
• 16:01
  especially if RAG is becomes a super domain specific industry
• 16:07
  or not industry, but like use case of yeah, every time you
• 16:11
  need to engage RAG, you're doing it from with domain expertise of
• 16:16
  engineering behind it. And so in that case, if you had
• 16:20
  to do some engineering and maybe it made sense to not deal with
• 16:24
  them a really heavy database and instead do something more niche
• 16:27
  even with Reddis, I don't know. Yeah.
• 16:29
  It's so funny that it reminds me of that meme of the like the
• 16:33
  nitwit and the ninja and how it starts over here and it's like,
• 16:40
  I'll just use Postgres. And then there's the bell curve
• 16:43
  graph and it's like, no, we need vector database and graphic
• 16:47
  database and no sequel. And then on the other side, you
• 16:50
  have the ninja who's like, I'll just use Postgres, yeah.
• 16:53
  Exactly. Postgres is one of the best
• 16:56
  inventions I think of the last 20 years because no, I mean, I
• 17:01
  think it replaced Oracle, generally speaking.
• 17:03
  And Oracle was this miserable vendor lock in.
• 17:06
  I still have friends in the industry and their whole job is
• 17:11
  trying to migrate away from Oracle for years.
• 17:14
  Job security right there. It's it's absolute insanity.
• 17:18
  So yeah, I, I, I, I agree with this meme, but we do use Vectra
• 17:22
  database. I don't want to.
• 17:23
  I don't want to, you know, I hate on vector database.
• 17:26
  You're kind of. Talking in the future, this is
• 17:29
  where you're thinking maybe there's a potential here, let's
• 17:32
  see, and if you can, then great. But at this moment in time, you
• 17:37
  don't, and that's really good to call out it.
• 17:40
  It also reminds me when you talk about Oracle and migrating off
• 17:43
  of it. There was an episode of Acquired
• 17:47
  that I listened to on AWS and I think it wasn't until 2018 that
• 17:53
  they had fully migrated completely off of Oracle 2018
• 17:58
  for AWS and well I guess AWS or Amazon, it probably was because
• 18:03
  AWS probably started a lot on AWS stuff and so you can imagine
• 18:10
  how hard it must be for folks to get off of Oracle.
• 18:14
  So anyone that is out there and completely locked in Oracle,
• 18:18
  you're fighting a good fight. Yeah, join the Postgres world,
• 18:22
  Yeah. They're like you are, you're
• 18:26
  oblivious to all these pains that I have.
• 18:28
  So anyway, the let's talk about what you are actually doing as
• 18:32
  opposed to what is potential in a utopia world where we can just
• 18:37
  use Postgres for everything. Because it feels like, as you
• 18:41
  were saying, you have many layers of databases and the
• 18:45
  actual structure around that and the way that the queries go
• 18:49
  through each layer of these is kind of interesting to me.
• 18:53
  Yeah. There's a lot of engineering
• 18:55
  behind it, which is super fun to be a part of and to solve in
• 18:58
  real time. I mean, it one of the IT kind of
• 19:02
  if we back up to how we generate the data, it might help
• 19:05
  understand how the data is structured and then queried.
• 19:08
  We're doing a lot of processing of news and that each for every
• 19:13
  for each news article, we're extracting and enriching as much
• 19:17
  information as we can to build a synthetic representation.
• 19:21
  So we're, we're not actually dealing with the full text ever.
• 19:24
  We're going and we're extracting evidence, we're extracting the
• 19:27
  key people, we're extracting the the Geo coordinates right,
• 19:32
  related to the locations that are mentioned in the thing.
• 19:34
  We're extracting all sorts of stuff, including relationships.
• 19:38
  And so that happens for every single article.
• 19:41
  And that means, you know, one article might mention Joseph R
• 19:45
  Biden and the other article might mention President Biden
• 19:49
  and the other one might mention just Biden.
• 19:52
  And obviously that is one node or maybe it's not depending on
• 19:56
  what you want in your system, right?
• 19:58
  And so being able to choose how that those nodes are
• 20:01
  disambiguated is huge. But the generation of that data
• 20:04
  itself, since we're going ontology free and we're saying
• 20:08
  OK. You know, we're extracting a
• 20:10
  person from this article, He might have a different name from
• 20:12
  this article. We had to solve figure out how
• 20:15
  to do this and build up from it. You know that aspect.
• 20:18
  We fine-tuned Phi 3 Mini in order to build the graphs and
• 20:22
  I'm happy to chat about how we did that after.
• 20:24
  But basically Phi 3 Mini has one job and that's to just take a
• 20:30
  text and generate an ontology free graph.
• 20:34
  And so he it does it for this article, Joseph R Biden, this
• 20:37
  article Biden, this article President Biden.
• 20:40
  And then, you know, we store all of that information in, you
• 20:44
  know, a hierarchy of databases in some ways, right?
• 20:47
  A lot of UUI DS pointing to different UI DS.
• 20:50
  Maybe it's too complex, I don't know, but it seems to work quite
• 20:53
  well. But a lot of the data stored in
• 20:55
  S3. And so we're, we're doing
• 20:57
  retrieval on these UUIDS based on a lot of pretty traditional
• 21:02
  techniques, to be completely honest.
• 21:04
  I mean, we do have vector representations there, but
• 21:08
  keyword search is still really, really good.
• 21:11
  I think that's something that's maybe also a little contentious
• 21:16
  these days that maybe you don't actually need these dense
• 21:19
  vectors. You might not even need sparse
• 21:21
  vectors. So even with just basic keyword
• 21:25
  metadata filtering, you can get so far to just get to the base
• 21:28
  of OK, here's a, here's an article that has Biden, Biden,
• 21:32
  Biden. And then you can take those
• 21:34
  graphs and you can start doing your disambiguations on top of
• 21:37
  them and then put it into a, a mem graph.
• 21:41
  And then then you can do traversals where you're not
• 21:44
  dealing with three nodes, which are clearly all the same person.
• 21:48
  Now you have one node, but you managed it's like, it's almost
• 21:51
  like a decompression or a compression of information.
• 21:54
  So it's decompressed at that base layer.
• 21:57
  And then you have a decompression algorithm which
• 22:00
  inside of it is a disambiguation.
• 22:02
  And so that really helps with this with solving these, these
• 22:06
  big challenges on the the largest news knowledge graph in
• 22:09
  the world. Once you're an M graph, you can
• 22:12
  have all the fun you want with whatever queries you want and
• 22:15
  and then and you know, even philtre it down even further if
• 22:19
  you want. And so why is it that you were
• 22:24
  saying that the mem graphs were spun up and then blown up in in
• 22:32
  moments like where is that? What is the life cycle of the
• 22:34
  mem graph look like? Good question.
• 22:38
  So when our users come in and they want to interact with this,
• 22:41
  this knowledge graph, right, we're taking their query and
• 22:44
  we're saying, OK, how do we build a graph for this person
• 22:48
  from what we know here? So there is actually some AI on
• 22:51
  the, on the, on the incoming side, because we're, they're not
• 22:55
  using traditional query languages for the further the
• 22:57
  graph. They're just using natural
• 23:00
  English language. And so it's on us to say, OK,
• 23:03
  these are some important keywords we're going to use for
• 23:05
  filtering. These are some important
• 23:08
  concepts that we should keep in in place.
• 23:11
  And so then we, we combine all, we go fetch all that
• 23:14
  information, do all of the combination and we can actually
• 23:16
  put that into a mem graph instance for them right then and
• 23:19
  there. And then at that point, we can
• 23:21
  actually use traditional queries.
• 23:22
  And maybe it we also have already prepared a traditional
• 23:26
  graph query based on what they asked.
• 23:28
  So maybe they said, give me all of the relationships of Biden
• 23:32
  within 3 hops. That's it.
• 23:35
  And to end then tell me, tell me what's the biggest threat to
• 23:38
  Biden's administration based on three hops or something, right?
• 23:41
  Something that is clearly English language requires a lot
• 23:45
  of basic retrieval and the construction of some real query.
• 23:50
  And so then once it's in Memgraph, we can do those
• 23:52
  traditional, that actual query and spinning up the Memgraph is
• 23:57
  actually quite fast, right. So I know we kind of hated on
• 24:00
  graft DBS, but when we are at the scale of of what these users
• 24:05
  are looking for, they're not looking to build a massive
• 24:07
  graph. They want to know usually
• 24:09
  there's a time constraint as well.
• 24:12
  So you're only looking at the graph between, you know, now in
• 24:15
  the last 72 hours or even the last week.
• 24:18
  And no one would say I want the graph for the last two years
• 24:23
  because a, it would probably break them graph anyways.
• 24:26
  And so, but it also is just too much data to even parse and try
• 24:32
  to glean insights from. Usually what you want to know is
• 24:35
  like what was happening with Biden right before a certain
• 24:38
  event. So that would be an analyst
• 24:40
  saying, OK, leading up to Biden exiting the race, I want to know
• 24:45
  all of the relate, all of the people within 3 hops that said
• 24:49
  something related to Biden exiting.
• 24:51
  And then give me a summary of all of the those interactions
• 24:56
  and tell me who are the most likely individuals that
• 24:59
  contributed. To.
• 25:01
  Maybe influencing Biden or influencing the media or
• 25:04
  whatever kind of analytics insight you're after, That would
• 25:09
  basically we would take that and then have to do this whole
• 25:12
  decompression, get it here, extract it for you.
• 25:15
  And also even give them access directly to that mammograph
• 25:18
  instance to then maybe build their own queries, right?
• 25:21
  And as soon as they're done, we wipe it clear.
• 25:24
  We're done with it, right? We've interested at that point,
• 25:27
  there's nothing more for us to do.
• 25:29
  That graph will never need to be created again because someone
• 25:32
  else is going to have a slightly different query.
• 25:34
  And as you can see, this like decompression has so many, you
• 25:37
  know, I guess reasoning checkpoints that it most likely
• 25:42
  won't happen again. We do have some caching, but
• 25:45
  it's very very limited in this sense.
• 25:48
  Yeah. And so when you're loading up
• 25:51
  the information into the mem graph for the user to be able to
• 25:55
  interact with, how do you know what is enough if they don't put
• 26:02
  how many hops or also how are you not grabbing things that may
• 26:07
  or may not be relevant or may be relevant for the next query?
• 26:11
  Is that also just loaded in the in the time when someone says
• 26:16
  OK, and I want to go a little bit deeper over here, I'm going
• 26:19
  to poke around in this area. Then you just grab that data and
• 26:24
  serve it up. So you mean like modifying an
• 26:28
  existing graph? Is is what I'm saying.
• 26:30
  So there's exactly. It's like I modifying an
• 26:35
  existing graph or I want to know how you are creating the surface
• 26:45
  area of the graph because I I'm assuming that not everyone says
• 26:49
  give me 3 hops. Why do you have to?
• 26:52
  Do you have to say that or is that like one of the
• 26:54
  prerequisites? No.
• 26:56
  No, I mean, luckily we can kind of decompress from anything.
• 27:00
  It's on you to decide what level of structure and specificity
• 27:05
  you're after. It could be there's irrelevant
• 27:08
  things that are connected, but they're going to be connected.
• 27:11
  There's going to be a connection, right?
• 27:12
  Like you might have, if we're talking about this like
• 27:14
  presidential thing, it's like, OK, here's Biden floating here.
• 27:17
  And then and then like a couple hops over here, you've got some
• 27:22
  random sporting event that had nothing to do with it.
• 27:25
  But there was like a mention of something related to something
• 27:29
  in the anthem. And then just just this weird
• 27:31
  connection. It might be floating around.
• 27:34
  But in some ways that is actually what a lot of our
• 27:37
  analysts are after because that's where you get to
• 27:40
  something, a connection that's hard to get to through vector
• 27:44
  search because you've got this very, very dissonant connection
• 27:49
  set of connections. Like for example, one of our
• 27:52
  favourite clients is Love Justice.
• 27:55
  They do, they combat human trafficking.
• 27:57
  So it's an NGO in South Africa. And what they do is that like
• 28:01
  they'll say, OK, I know there was this incident of, you know,
• 28:07
  there was an incident of criminal activity at this shop.
• 28:12
  And this person reported an incident and the police are
• 28:15
  investigating it. But then this shop is also
• 28:18
  connected to this one person over here.
• 28:21
  And that person might be benign, but it turns out that they're
• 28:25
  also connected to something over here, which basically opens up,
• 28:29
  OK, why is this person, you know, you it really they follow
• 28:32
  this path through and all of a sudden they've kind of uncovered
• 28:35
  a hidden insight. And that allows them to then
• 28:38
  say, hey, this is somebody or something or some place or some
• 28:42
  product that we need to look into because it's weirdly
• 28:46
  connected. So that's, that's, that's
• 28:48
  essentially the one of the use cases you would use the
• 28:52
  knowledge graph for. But so the I've kind of drifted
• 28:57
  off the surface area, but now it's OK.
• 28:58
  It's kind of you get what you get.
• 29:00
  And you hope that that that raw representation and that this
• 29:04
  metadata filtering is going to get you enough of a surface area
• 29:07
  that then you can play a lot with once you're in mem graph.
• 29:10
  So maybe it's a weakness, maybe it's a strength.
• 29:14
  Yeah. It reminds me of that moment in
• 29:18
  the series The Wire. I don't know if you ever saw
• 29:21
  that, but there was. I can't remember in which season
• 29:24
  it was, but all of the police are sitting around in their
• 29:29
  stakeout room trying to figure out how the drug dealers have
• 29:35
  money or who is giving money to who when it comes to this drug
• 29:40
  dealer is supplying the streets with drugs.
• 29:43
  But then it's like the money is going into all of these accounts
• 29:47
  and then one of these accounts actually what's going on here.
• 29:50
  They're following the money and they see that it's going to a
• 29:53
  politician. And it feels like, damn, dude.
• 29:56
  If they just had Ask News, they could have made that whole
• 30:01
  episode like 30 minutes shorter. Exactly.
• 30:05
  That's that's funny that you say that actually, who's that guy?
• 30:08
  Matt Gaetz in the New York Times released an article when Matt
• 30:12
  Gaetz was it was the nominee for the head, the head of the
• 30:16
  Justice Department. They peep all of the The Dirty
• 30:20
  laundry was coming out as usual, but one of The Dirty laundry
• 30:23
  pieces from his interactions with minors was apparently the
• 30:28
  whole the investigation into him.
• 30:29
  They they built a Venmo payment knowledge graph.
• 30:33
  Did you see this? No.
• 30:37
  And it was just so damning. It was like, Gates pays this guy
• 30:43
  a lot and this guy pays all the miners a lot.
• 30:47
  And it was just, but then it was sometimes he would even pay the
• 30:52
  miners directly. And it was like, man.
• 30:55
  And then he would put little notes like for tuition.
• 30:57
  Like you couldn't make a better knowledge graph set of metadata,
• 31:01
  then use Venmo and add notes like for tuition from Gates to
• 31:06
  So people, I think people are missing the power of the graphs,
• 31:11
  especially people like that, like Gates.
• 31:14
  But you've got companies that are designed for using the power
• 31:17
  of graphs just to solve these criminal investigations.
• 31:21
  Like one of our advisors, Paco Nathan, I don't know, you know
• 31:24
  him, right? I think we've had.
• 31:26
  That's great. Yeah.
• 31:27
  Yeah, Paco's awesome. He's a legend in this space and
• 31:31
  just in AI in general. He really is.
• 31:33
  Did he talk about, did he tell you about what sensing is doing
• 31:36
  and what they're. Yeah, that's crazy.
• 31:38
  They're a lot of what they'll do is even like they, they use open
• 31:43
  source intelligence, they do a lot of disambiguation.
• 31:46
  And what they're able to do is open up insights through
• 31:51
  knowledge graph connections that are just totally hidden when
• 31:54
  tracking criminal activity. And some of that comes down to
• 31:58
  I'll even imagery for I think one of the examples he gave was
• 32:02
  like fishing protecting, protecting the oceans from
• 32:06
  overfishing and stuff. And you can use imagery and you
• 32:09
  can use, you know, information from ports and you can use boat
• 32:12
  registrations and you can connect all of that stuff.
• 32:14
  And all of a sudden you start to see this boat is doing something
• 32:16
  that is weird, is connected to this thing.
• 32:19
  And and then besides that might one of my favourite examples was
• 32:24
  the Silk Road investigation, how they found the the the creator
• 32:28
  of Silk Road, you know, silk. Road from Yeah, yeah, I know.
• 32:33
  I've for sure I've heard of Silk Road.
• 32:35
  The IT was a drug marketplace. Right.
• 32:40
  And it was based on Bitcoin and essentially the, the way that
• 32:44
  this, the feds, the FBI identified the the creator and
• 32:49
  found him was by mapping out all of the wallet addresses and the
• 32:54
  movement of money between wallets.
• 32:56
  And I think Cosmograph even has the, the Silk Road investigation
• 33:02
  available like on their website to visualise just to, and it's
• 33:06
  really cool. You see all the wallet addresses
• 33:07
  floating around and you can, and that's basically through these
• 33:10
  connections, they were able to say, OK, this wallet is showing
• 33:14
  up a lot in these very, very suspicious payment transactions.
• 33:20
  Or there is a wallet that's suspicious and some, you know,
• 33:23
  the, the way people glean information from knowledge
• 33:26
  graphs. Once you start seeing it, you
• 33:29
  realise, OK, there's a lot, a lot of power.
• 33:32
  And that is with ontology. So there's there's definitely
• 33:35
  still good use case for ontologies.
• 33:38
  And you mentioned how AI is on the front end when the user
• 33:41
  interacts and their queries, but then you also have AI on the
• 33:45
  back end when you're sending queries to an LLM.
• 33:49
  How does like where does everywhere in the besides the
• 33:54
  Phi 3, which is awesome use case of just fine tuning Phi 3.
• 33:59
  And that's very cool to hear you did that.
• 34:02
  Did you just still the when you fine-tuned it?
• 34:05
  How did you fine tune it? Oh, we have a blog post I can I
• 34:09
  can post as a comment on the on the podcast.
• 34:12
  But yeah, we we fine-tuned it. A lot of it was engineering the
• 34:16
  data set data set. We pretty much did a knowledge
• 34:20
  transfer from GPT 4 O. So which is was is not new.
• 34:25
  I think even back in the day, alpaca was like the first to
• 34:28
  show that, hey, you could actually steal some of this
• 34:31
  knowledge from ChatGPT or from GPT 33.5 back in the day.
• 34:37
  And now we're, you know, we, we basically did the same thing
• 34:39
  except with GPT 4 O to label a bunch of data engineer that data
• 34:43
  in a very diversified way to cover the parameter space as
• 34:47
  best as we can. The parameter space that we knew
• 34:49
  we would be looking in, which is, you know, ensuring you have
• 34:52
  topics of politics, sports, entertainment, ensuring you're
• 34:55
  hitting all the languages, ensuring that you're, you know,
• 35:00
  crossing all the different continents and countries and
• 35:03
  sources and stuff like that. So labelling that data set with
• 35:06
  GPT 4 O, ensuring it's diversified and then fine tuning
• 35:09
  5 three on it. And it turns out it outperforms
• 35:13
  cloud 3.5 and it's free and fast, right?
• 35:16
  So if we were to run on cloud 3.5, you'd get great results.
• 35:20
  But imagine putting 1/2 a million articles a day through
• 35:24
  cloud 3.5 that you'd pay 10 KA day.
• 35:28
  And I'm not even kidding. And so all of a sudden you've
• 35:32
  got this little tiny one that fits on basically a consumer
• 35:35
  graphics card and it's it's just pumping out clawed, clawed level
• 35:40
  quality. Of course, it's not going to do
• 35:41
  anything else, right? It's only going to build the
• 35:44
  graphs, but they're better than clawed graphs, which is fun, but
• 35:48
  OK. So to answer the, the end of
• 35:50
  your question, which is one of my favourite reasons or one of
• 35:53
  the actual main impetus for do taking this whole ontology free
• 35:58
  route was the, the conveying information in a very concise
• 36:02
  and token optimised way. So like you said, you know what
• 36:06
  happens on the end with the AI, Like why are you now we're
• 36:10
  trying to communicate to an LLM. So a lot of our users might not
• 36:14
  even need the mem graph component of this.
• 36:17
  So a lot of them will say, I need a graph of Biden's
• 36:22
  interactions and related to Hunter Biden or whatever.
• 36:27
  And so then we take, we build them this very, very concise
• 36:30
  representation of relationships. And then they'll take that as
• 36:34
  Jason or, you know, exit YAML. And then they'll just hand that
• 36:39
  directly in a prompt to an LLM and continue with their
• 36:43
  insights, right? So they'll say, maybe we're just
• 36:46
  part of their chain. We're not the beginning nor the
• 36:48
  end. We're just one component getting
• 36:50
  them that real time grounding. But instead of, you know, in
• 36:54
  order to build that relationship graph of Biden and Hunter,
• 36:58
  probably we had to read 1000, 2000 articles, distil them,
• 37:03
  enrich them, understand them, build relationships.
• 37:06
  And now it's all been condensed into maybe 3 or 400 tokens.
• 37:11
  And so they're, they're saving a tonne of money there.
• 37:13
  I mean, they're paying us because obviously they pay for
• 37:16
  the request, but they don't have to do any of that aggregation of
• 37:20
  information. You know, what if a Chinese
• 37:22
  outlet talked about Biden randomly, how would you have
• 37:24
  gotten that right? And doing all this in under a
• 37:28
  second too, and passing that to the next AI or to the next LM
• 37:33
  and then maybe a pending after. So like their prompt might look
• 37:36
  like this. So they they get the YAML or the
• 37:40
  relationship craft from us. They add it to the beginning of
• 37:43
  their prompt. And they say, here's the
• 37:44
  relationships related to the news of Biden and Hunter.
• 37:48
  And then here's my other context that I only I know about right
• 37:52
  in my own application related to this piece of whatever to help
• 37:56
  answer the user query. And here's another set of
• 37:59
  content. They're painting their own
• 38:00
  context. We're just a part of it.
• 38:02
  But now their context windows basically fully open.
• 38:05
  They didn't have to put a bunch of articles in it.
• 38:07
  They didn't have to preprocess all this text.
• 38:09
  It's. Just boom.
• 38:10
  Jason and so and the the LLM loves interacting with it.
• 38:15
  The other beauty, beautiful part of this is that this is the
• 38:20
  innovation of us connecting publishers in a very, very
• 38:23
  equitable way to LMS. Because now the original expression is
• 38:27
  nowhere to be found. There's no plagiarism concerns,
• 38:31
  but the publishers can still monetize the information that
• 38:34
  they've produced without worrying about leaking all of
• 38:37
  that information and leaking all of their not, sorry, not leaking
• 38:40
  the information, but not leaking the original expression that
• 38:43
  their journalist spent so much time writing.
• 38:46
  They don't like, they don't want to train new foundation models
• 38:49
  with their expression, with their real.
• 38:51
  With the truth, with the full style of writing.
• 38:53
  But they want to monetize it in this new AI world and they don't
• 38:57
  know how. So Ask News essentially locks
• 38:59
  them in and says, OK, come to us.
• 39:02
  If you surface, you get a royalty.
• 39:05
  And what surfaces is a synthetic contextual representation where
• 39:09
  like you've surfaced somehow, but in a lot of context with a
• 39:14
  lot of other information, you just contributed to this query.
• 39:17
  So you get a royalty, but there's nothing there that's
• 39:19
  going to get trained on or stolen or redisplayed on another
• 39:23
  website. It's only for communication to
• 39:25
  LLMS. Wow, wow, there's so much there and you're saying you're
• 39:31
  saying everything that I like to hear.
• 39:33
  There's so many cool things that you're talking about here and
• 39:38
  the So the first thing let me start from what my mind
• 39:43
  instantly jumped to was, Oh, you're just for a lot of your
• 39:47
  users use cases, it is. You're just another step in a
• 39:50
  DAG. Exactly.
• 39:52
  It's like go grab the context from ask News and then they can
• 39:57
  use that however they choose. Maybe it's by putting it into a
• 40:01
  context window for their LLM call, or maybe it's just to
• 40:06
  have. Some more information to play
• 40:10
  with when they want to go in and they want to play with that
• 40:14
  information. And then #2 is it is so nice
• 40:20
  that you're doing the heavy lifting to synthesise all that
• 40:25
  data and then find what the real signal is out of all of that.
• 40:31
  Because, yeah, thousands of news sources, I can imagine that some
• 40:35
  of them are really bad and some of them don't mean anything.
• 40:38
  And some of them are actually quite good.
• 40:42
  So how do you know which ones are good or bad?
• 40:46
  Or do all of the people get paid equally?
• 40:48
  So I can just I can start turning out shitty things and
• 40:52
  start getting paid on ask news because I'm not known for my
• 40:55
  quality. That's a good question.
• 40:58
  And that's, that's one of the most common questions we get.
• 41:01
  And, and that's all I like how people are aware of the, the
• 41:07
  dangers around potentially propagating misinformation.
• 41:11
  And this one is super interesting because we've just
• 41:14
  been in the mix of it and we've been observing it and running
• 41:17
  research on it. And so one of the, the outlets
• 41:22
  that people hate that they might stumble across is RT, right?
• 41:26
  It's like, oh, this is a Russian propaganda network.
• 41:29
  Everything they say is fake. But actually the, the beauty of
• 41:33
  what we're doing is this contextual representation.
• 41:36
  First of all, you can philtre out RT if you don't want, right?
• 41:39
  You can say, I don't want that or, but even when you include
• 41:43
  RT, if you take a contextual picture of French, German,
• 41:48
  English, Italian, Arabic and Russian outlets and Ukrainian to
• 41:53
  discuss the Ukrainian Russian war, all of a sudden the LLM is
• 41:58
  smart enough to detect what are the alignments and what is
• 42:01
  poking out and contradicting, right.
• 42:03
  And we're able to leverage the LLM intelligence at that point.
• 42:07
  So like the, the, the stories we write on Ask News, that's
• 42:10
  precisely what we do. We cluster a bunch of articles
• 42:13
  and then we actually ensure we do what's called diversity
• 42:17
  enforcement. So our team might show up, but
• 42:21
  knowing why, how RT relates to what on the same topic, to what
• 42:26
  the Ukrainian source says is actually the right piece of
• 42:29
  information to convey to the reader, because that shows up as
• 42:32
  a contradiction. It's like ORT says that no
• 42:35
  civilians were killed. Every other source, in every
• 42:37
  other language, in every other part of the world, it says that
• 42:41
  a bunch of civilians were killed.
• 42:43
  That's just, that's just a piece of information that we are
• 42:45
  relaying. It was a contradiction that the
• 42:47
  LLM was able to identify because it poked out like a sore thumb.
• 42:51
  Now you as the reader, you decide what to do with that
• 42:53
  information. It helps you.
• 42:55
  It reinforces that RTS might be reporting incorrectly.
• 42:58
  So if we remove RT from the whole interaction, now we're
• 43:04
  actually kind of biassing away from Russia.
• 43:06
  We're not giving Russia even a say.
• 43:09
  You know, sometimes what in an RT article there might actually
• 43:12
  be really important information to know about in Russia.
• 43:15
  You know, obviously there's that understanding that it's coming
• 43:20
  from a state-run megaphone and that that should be a part of
• 43:25
  what of of how you consume that information.
• 43:27
  But a lot of times they're reporting actual events that are
• 43:30
  are occurring and knowing what they say is important.
• 43:33
  So, you know, if you, if you want to start publishing a bunch
• 43:37
  of articles that might surface in these queries that can that
• 43:41
  that's perfectly OK. That's that that we would take
• 43:44
  in your public. Like if you have, if you want to
• 43:45
  monetize your sub, sub stack, you can do that, right?
• 43:48
  Yeah, you can come and add your sub stack and then we'll go and
• 43:51
  essentially build that synthetic relationship representation for
• 43:54
  you, leave your content there and then make it retrievable for
• 43:59
  the developer. And if you said something that's
• 44:01
  retrievable, they can play with it.
• 44:04
  The last point I would make on that, we have a tonne of
• 44:07
  filtering mechanisms. So we part of our enrichment, we
• 44:11
  check the reporting voice. So we'll check if it's
• 44:13
  persuasive, sensational, objective, investigative,
• 44:17
  analytical. So you could say I only want
• 44:19
  objective. Some of our analysts say I only
• 44:22
  want sensational because they actually want to see what the
• 44:25
  sensational side of it. Maybe they only want sensational
• 44:28
  Russian to just understand only what, because that's the part of
• 44:32
  the context that they need to paint in their next step, right?
• 44:35
  Like it's almost like a chosen bias with your philtre where you
• 44:38
  can say I only want this, I only want that.
• 44:41
  You can also philtre on page rank.
• 44:42
  This helps, right? Wait, that sentiment analysis is
• 44:46
  done by another model in the background?
• 44:50
  Yeah, actually Llama 370 B is is running.
• 44:52
  Llama 3.170 B is running basically the full synthetic
• 44:57
  representation, which includes, yeah, the reporting voice, the
• 44:59
  sentiment, the extraction of key evidence, supporting details,
• 45:06
  the any key people, stuff like that.
• 45:09
  That would be kind of the synthetic representation.
• 45:12
  So you know, Llama 3.170 B, you've played with it, right?
• 45:16
  I mean, it's, it's wildly capable and we see it works very
• 45:21
  well, especially when it comes to sentiment analysis, positive,
• 45:25
  neutral, negative. You see this like basically
• 45:31
  this, this massive shift when in comparison of the Llama models
• 45:36
  versus what sentiment analysis was before.
• 45:40
  And it used to be very, very difficult to get that right.
• 45:43
  We're still not perfect, but this was a step change.
• 45:47
  This was going from 50% of the time right to 85% of the time,
• 45:52
  right, Something where an analyst used to say, so many of
• 45:56
  our analysts and clients say you have sentiment analysis.
• 45:58
  I don't touch that like I've I've seen.
• 46:00
  I'm just seeing enough sentiment analysis.
• 46:02
  Thank you. Yeah, I've been.
• 46:03
  Burned by them enough that I just don't.
• 46:06
  Exactly it. So we have to convince them.
• 46:08
  It's like actually, we've done the research, we've done the
• 46:10
  cheques. It's not perfect, but it's very,
• 46:13
  very llama 3 has it's has a high level reasoning.
• 46:17
  What does that mean? When you have a higher level and
• 46:20
  you can do things like that, it's cool.
• 46:23
  And so maybe you explain that then, but I didn't catch it the
• 46:27
  first time around. It's these news sources.
• 46:30
  You're crawling everywhere you can to get the news sources.
• 46:34
  You're then using Phi 3 to figure out if there are nodes.
• 46:43
  And then when you figure out what the nodes are, you use
• 46:47
  Llama 3 to break it down. Close 1st, we aren't crawling
• 46:52
  everywhere. We're only crawling where we're
• 46:54
  allowed to crawl or where we have a licence source or where
• 46:58
  someone has signed up voluntarily and given us the
• 47:01
  ability to go behind the paywall.
• 47:03
  So we'll never go behind a paywall.
• 47:07
  Now the the construction of the information itself.
• 47:11
  Actually, we, when we get to the article, Llama does its
• 47:14
  extraction, Phi 3 does his extraction, Gleiner does his
• 47:17
  extraction. We, we didn't talk about it, but
• 47:20
  this is a very, very, this is a big pipeline.
• 47:23
  Gleiner is, is like our crown jewel at Emergent methods.
• 47:27
  It turns out it was like the 10th most downloaded model on
• 47:31
  Hugging Face this year. I found this out through Clem's
• 47:35
  post. Actually.
• 47:36
  He like, you know, they have the, the 21 days of hugging face
• 47:39
  or whatever, and day 2 was the most downloaded and I was
• 47:42
  looking through it's Gleiner. It's just an entity extractor.
• 47:46
  We're not even really that brilliant.
• 47:48
  We didn't invent the architecture.
• 47:50
  We just engineered a data set which was diverse and fine-tuned
• 47:53
  it. So we're kind of, we're fine
• 47:55
  tuners. We're not the architectural
• 47:58
  masters of Transformers. It turns out people, it works
• 48:02
  very well and people caught on, but it's really cheap.
• 48:05
  These are all super cheap models, which is why you can
• 48:07
  kind of Daisy chain them up because they, Oh yeah, Llama 3
• 48:10
  is pretty cheap. I mean, it's more most expensive
• 48:12
  one, but then FI 3 super cheap. Gleiner super cheap and that
• 48:17
  allows you to kind of really full build out that synthetic
• 48:21
  contextual picture of a single article and you do that for all
• 48:24
  of them so every single article we find we put it to that
• 48:27
  pipeline so that sits now go ahead and.
• 48:32
  Which? OK wait, so you have Llama 3
• 48:37
  going first and then it goes down the line?
• 48:40
  Or is it all at the same time like asynchronously?
• 48:44
  It depends. Some of them are async.
• 48:45
  If they can be async, like I think the graph and the glider
• 48:48
  are async at the same time 'cause they don't depend on each
• 48:51
  other. Yeah, Llama three goes 1st and
• 48:56
  it it could, because llama 3 gets you that original, that
• 48:59
  full kind of distillation of information, and then you can
• 49:02
  start processing on top of it, which is really nice.
• 49:04
  It's, it allows you to, a lot of these small models like Kleiner
• 49:10
  have a context window of 512 tokens.
• 49:13
  That's kind of the, at least the, the general pipeline and
• 49:16
  it's evolved. It's been really fun.
• 49:18
  We just get these free tools all the time, man, I don't know, I
• 49:21
  feel like every other month is Christmas because we're just
• 49:25
  sitting around running, you know what we were running Llama 213
• 49:30
  B, the Wizard LM. I don't know if you remember
• 49:33
  Wizard LM back in the day. I do.
• 49:36
  It was from Microsoft. They pulled it, but we were
• 49:39
  running that and then Llama 3 came out and we're like, all
• 49:41
  right, let's see, this seems smarter.
• 49:42
  Boom. And it was smaller and faster
• 49:45
  and smarter. You know, what does that do to
• 49:48
  the the ecosystem, right? I'm a tiny startup.
• 49:51
  We have a very small amount of funding and resources.
• 49:56
  In your opinion, are you seeing it because you, you, you're
• 49:58
  tapped in, You're seeing people, startups like me.
• 50:01
  It's got to have a massive ripple effect, right?
• 50:06
  Dude, it's it's so funny because I think there is on one hand,
• 50:11
  there is the you have to be almost, I'm not going to say the
• 50:17
  cutting edge, but you have to be testing all of this stuff out in
• 50:22
  a way because you are resources constraint.
• 50:24
  And then I see the other, there's another end of the
• 50:27
  spectrum where it's folks that are just trying to figure out
• 50:32
  the product itself. And once they figure out if they
• 50:35
  can make a product that someone loves, then they'll find out how
• 50:40
  to optimise on pricing and latency and all of that.
• 50:44
  And so you almost have two sides where there's the the pricing
• 50:49
  insensitive. And so they say, whatever, we'll
• 50:51
  just throw everything at Open AI or Claude because we want the
• 50:55
  best and that's the least of our worries right now.
• 50:58
  And then you have this side where it's like, I'm going to
• 51:01
  fine tune everything that I can to our specific use case and
• 51:07
  throw a lot of different models together and make it.
• 51:11
  And also like what you're doing with the databases and you're
• 51:14
  doing with how you're architecting things.
• 51:18
  It is so cool to see because it shows the possibility of what's
• 51:25
  out there. Let me ask you a question.
• 51:28
  It's kind of a side a tangent, but yeah, I've been wondering
• 51:31
  lately the pricing of open AI because you mentioned, OK, I'm
• 51:36
  just the pricing is going down and down and down.
• 51:39
  So why don't I just use that, that, that and for sure we use
• 51:42
  it. Don't get me wrong.
• 51:43
  Like I said, we, we to build to label our data, we use GPD 4 O
• 51:47
  and Claude. And for doing the high level
• 51:50
  forecast, we'll we'll run those through those because it makes
• 51:53
  sense. They're very smart, but I'm
• 51:55
  really scared as a start up about this pricing going down
• 52:00
  trend because I don't think that that's how it's.
• 52:03
  I think this is a race to the bottom of these companies
• 52:07
  willing to take a tonne of debt. So to subsidise those prices.
• 52:11
  What happens as soon as that debt that that fruitcake is
• 52:15
  done? All of a sudden, what do we do
• 52:19
  now? The prices go up and all of
• 52:20
  these businesses that like mine, even I'm worried about that have
• 52:25
  been exposed to A2 point $5 per million GPT 4 O token price and
• 52:31
  said I can build around that and build a real product.
• 52:35
  And then I start charging customers based on that ground
• 52:39
  level. All of a sudden in two years, if
• 52:43
  we're lucky and the gravy train's over, all of a sudden
• 52:46
  open AI is finally all of these companies say, listen, we can't
• 52:50
  do it anymore. All of our investors, Sequoias
• 52:52
  wants their money back. Microsoft Smith But now it's
• 52:55
  going to start ticking up, right?
• 52:57
  And then that's just going to put pressure.
• 53:00
  On us, do you think that's going to happen or can you alleviate
• 53:02
  some of my concerns? Yeah, it does feel a little bit
• 53:06
  like the days of the ride sharing when the ride sharing
• 53:09
  first came out, right, Exactly. You had lots of subsidised rides
• 53:13
  just to get people taking Uber and making it their go to.
• 53:19
  And then now if you look, Uber is not cheaper than taxi, that's
• 53:23
  for sure. So it does kind of have that,
• 53:28
  especially when you hear things like open AI lost 5 billion and
• 53:32
  they made 3 billion and you're like, well, why did they lose so
• 53:35
  much money? Oh, maybe it's because they were
• 53:37
  subsidising the inference, but that's the what is it the
• 53:42
  elephant in the room that I haven't heard a lot of people
• 53:45
  talking about. And so you, you could be right.
• 53:49
  I hope you're wrong and I hope what happens is, is that we just
• 53:54
  continue to get you build that muscle of using open source
• 53:58
  models so much that by the time Llama 5 comes out, you're like,
• 54:03
  yeah, I don't even need open AI. Or if I do need to hit an API,
• 54:10
  it's not really going to make a nominal difference on the cost
• 54:15
  because the ideally the majority of the businesses are maturing
• 54:19
  with the market and they understand.
• 54:22
  So that, that's my look at it. But I do think that there's the
• 54:28
  race to the bottom. The other thing that I, I was
• 54:30
  going to say is they can't go down forever.
• 54:34
  There's got to be a stopping point.
• 54:37
  And that's what it's, it's like when you have a running race and
• 54:45
  you have the finish line and you know how if you do it
• 54:49
  mathematically, every time you go from point A to point B and
• 54:54
  then you cut it in half, that could go on for Infinity.
• 54:58
  But in reality, at some point you cross the finish line.
• 55:01
  And that's what it feels like in reality.
• 55:03
  It feels like at some point we're going to hit a stage where
• 55:07
  you can't go lower because it just what are you going to do,
• 55:12
  charge free? That doesn't work.
• 55:15
  No, I'm scared. I think and I like what you said
• 55:18
  about building that open source muscle and we're trying our I
• 55:21
  think we are we're prepared. But yeah, I look at some clients
• 55:26
  who want to use these high level derivatives for a certain price
• 55:28
  and I'm thinking that like one guy said, I want a two year
• 55:32
  contract. I cannot forecast that far of
• 55:34
  open a is pricings and so that's hard.
• 55:37
  But no, the other part of the data front, the pricing of
• 55:40
  inference, the data itself is going to get more expensive.
• 55:44
  OK. And because that gravy train is
• 55:47
  also coming to a very abrupt end as you've probably watched the
• 55:50
  whole perplexity open AI legal fights between publishers and
• 55:57
  they're signing up with platforms, they're fighting
• 55:59
  others. The the cost of true good data
• 56:02
  is only going to go up from here.
• 56:04
  And you I think you've maybe seen on LinkedIn there are ads
• 56:07
  now for just like make 1600 bucks a week doing coding
• 56:11
  problems to make training data for us, right?
• 56:14
  I like that. I like the premise.
• 56:16
  I like what they're after. It's like pay someone to
• 56:18
  generate your data, get the data you want.
• 56:20
  It makes sense, but that doesn't equate to a 252 point $5 GPT 4 O
• 56:26
  $1,000,000 token. I'm.
• 56:28
  Sorry, it doesn't have expensive.
• 56:29
  Yeah, that's a very expensive one.
• 56:31
  Actually. I just saw that Uber opened up
• 56:34
  its gig economy or things that you can do as an Uber
• 56:40
  contractor. You can now give rides, you can
• 56:43
  deliver food, and you can label data that is their next
• 56:47
  frontier. So that's pretty wild.
• 56:50
  And then on the other side, you see LinkedIn that is trying to
• 56:53
  just farm the data by saying you're an expert in this field.
• 56:57
  Why don't you answer this question?
• 57:00
  And in that I have my favourite person that I follow on LinkedIn
• 57:05
  is this guy who just blatantly gives the wrong answers to all
• 57:09
  of those. So he's completely trying to
• 57:12
  data poison him. I really.
• 57:15
  Don't like that system? Yeah, but you you kind of read
• 57:19
  the answers and you go, this might make it past like this
• 57:22
  might actually work. You know, it's not like you kind
• 57:27
  of have to read it and know that he is data poisoning for you to
• 57:32
  understand what's going on. So yeah, that's the LinkedIn LLM
• 57:38
  that comes out. I'm not going to have much trust
• 57:40
  in because of that guy. Makes sense.
• 57:44
  I mean, with, with even LinkedIn data itself having value, maybe
• 57:48
  their LLM is going to be trash, but a lot of some of our
• 57:52
  competitors are people are companies like Exa.
• 57:55
  And I'm seeing I'm, I'm seeing these business models crop up
• 57:59
  and I think I'm missing something and I want you to tell
• 58:02
  me where what I'm missing. But the business model of EXA is
• 58:06
  we scrape the entire web without any concern, including LinkedIn,
• 58:11
  including threads, including Instagram and deliver it to you
• 58:16
  and structure it for you. And actually the tech is quite
• 58:19
  interesting. I like what they're doing.
• 58:21
  I think that it's very compelling, but how is it a
• 58:24
  sustainable business model when LinkedIn is going to be like,
• 58:27
  Nah, I'm good, you're not going to scrape our stuff anymore,
• 58:29
  right? How?
• 58:31
  How is that a a strong business? Model, yeah, and even X charging
• 58:37
  for API usage. And so all of a sudden you're
• 58:42
  now every time you're hitting an API, you're doing that.
• 58:45
  That is one of the biggest questions that I had for you is
• 58:49
  like, isn't it too long? Because all of, I'm assuming all
• 58:53
  of the news sources that you're getting are when it's published
• 58:59
  on a blog or a publication. And so isn't there too long of a
• 59:05
  gap between when something happens and when it gets
• 59:08
  published? Or I I know the gap is very
• 59:12
  short, but for someone to actually write something and
• 59:15
  then push publish is much different than me sending out a
• 59:19
  tweet. This is an awesome discussion I,
• 59:23
  I, I'm really enjoying this one There's different time scales of
• 59:29
  business insight OK and some you know, let's take forecasting for
• 59:34
  example. Typically forecasting is not the
• 59:36
  next 10 minutes in event forecasting where you're saying
• 59:41
  you want to forecast if Pete Hegseth is going to be removed
• 59:46
  as the nominee from for the Secretary of Defence.
• 59:51
  And that requires quite a lot of information into the archive
• 59:57
  actually, not even just the current.
• 59:58
  And you do want the latest data points for sure, because that
• 1:00:01
  helps you project forward. But understanding how those data
• 1:00:05
  points developed is equally important, if not even sometimes
• 1:00:08
  more especially if you're saying something like what is the S&P
• 1:00:12
  500? Is the S&P 500 going to close
• 1:00:15
  higher or lower in 2025? That doesn't really require the
• 1:00:21
  last tweet from Warren Buffett. It requires out an understanding
• 1:00:26
  of what the Fed is doing and understanding of the global
• 1:00:29
  economy and like a very high level contextual picture, which
• 1:00:34
  we can now do with LLMS, which is insane.
• 1:00:37
  And so that's a resolution where having 5 minute up to the last
• 1:00:42
  five minute news is very useful. But like you said, if you're
• 1:00:47
  like actually the, the best example is, yeah, a lot of
• 1:00:51
  prediction market users use us, but they can't use us for every,
• 1:00:56
  every prediction market because of what you just said.
• 1:00:59
  There are certain prediction markets where we're very usable
• 1:01:02
  and there are certain prediction markets where we're completely
• 1:01:04
  useless because they depend only on what happens on Twitter and
• 1:01:08
  only you need to be basically tracking Twitter and it'd be the
• 1:01:12
  fastest to Click to get the right prediction market price
• 1:01:15
  versus that P Tegseth example where you can make it 8 days in
• 1:01:19
  advance, 10 days in advance and do research.
• 1:01:22
  And so just seeing that there's that variety is really
• 1:01:26
  interesting. We, we like to specialise in
• 1:01:28
  structuring the news. We have a lot of analysts and
• 1:01:32
  firms like Love Justice using US for those sorts of
• 1:01:36
  investigations, but there are certainly applications where
• 1:01:39
  we're not the right 1. And I like that.
• 1:01:41
  I even think that it would make sense for us to reach out to
• 1:01:44
  another surface that the service that maybe specialises only in
• 1:01:49
  Twitter, right? I mean, we could use the Twitter
• 1:01:51
  API and we have some development on that.
• 1:01:54
  But maybe there's a better service that can really
• 1:01:57
  structure it in a more retrievable way in the way we do
• 1:02:00
  it with news. But there's they're focused.
• 1:02:03
  So plug. That in to your service and so
• 1:02:07
  you do have that real time. But I do like one thing that is
• 1:02:11
  clear, just like as we were talking with about Memgraph,
• 1:02:16
  right, Memgraph took steps and it made decisions on how they
• 1:02:22
  were going to do things. And it feels like you've done
• 1:02:25
  the same. And you said we are going to
• 1:02:28
  create our product for these use cases.
• 1:02:30
  And right now we aren't even going to worry about those use
• 1:02:35
  cases that need to be real, real, real, real, real time.
• 1:02:39
  And the latest, you know, tweets about whatever or the latest
• 1:02:43
  posts. If it's breaking, breaking news,
• 1:02:47
  that isn't necessarily where the value, the strong value comes
• 1:02:51
  in. It comes in more on the
• 1:02:53
  understanding of how the whole. Plays together exactly and
• 1:02:58
  there's also some consideration of factuality, right If it's
• 1:03:03
  just bumped to Twitter, there's a strong chance it might
• 1:03:07
  actually be untrue Some of our analysts really want us to track
• 1:03:09
  Telegram channels I. Was going to say that too.
• 1:03:12
  And you can't do. It especially like Ukraine war
• 1:03:15
  exactly that's where a lot of the real latest information
• 1:03:18
  comes from, but there's a lot of misinformation there too or may
• 1:03:22
  not even misinformation, but just just they're not sure what
• 1:03:26
  occurred. They're speculating, it's
• 1:03:28
  speculation and that can it shouldn't necessarily be written
• 1:03:32
  down as news. Just the fact that someone had
• 1:03:34
  to sit down and verify that this is something we want to put on
• 1:03:37
  the server as a news article, it gives it that additional level
• 1:03:41
  of credibility. Again, RT is doing it.
• 1:03:43
  So we take that a bit with a grain of salt, but there's
• 1:03:47
  something to be said for that difference in publishing versus
• 1:03:50
  just 200 character tweet or a message on Telegram.
• 1:03:54
  So I think that using that context smartly is good.
• 1:03:59
  You know, we we do. I think it would be really
• 1:04:03
  smart. Like Camelay, I don't know if
• 1:04:05
  you've heard of them. They're an agentic multi agent
• 1:04:08
  framework. They're playing with letting
• 1:04:10
  multi agents make each other and make more agents and then talk
• 1:04:13
  to each other and then come back and it's wild and what they just
• 1:04:17
  have a bunch of tools. To me, that really is the
• 1:04:19
  solution. Ask News is one tool that one of
• 1:04:22
  the agents might want to use, but then you have a Twitter
• 1:04:25
  tool, you have a, and then it can define that contextual
• 1:04:29
  picture, can say this is the news, this is the Twitter.
• 1:04:32
  Be careful, right? This is just Twitter.
• 1:04:34
  Let's let's, let's use, let's, let's use it for what it is and
• 1:04:38
  paint that contextual picture. I think that might be a good
• 1:04:42
  way. And maybe in the future we end
• 1:04:43
  up going the route of expanding. But I think you're right that
• 1:04:47
  one of our as a startup, we're really focused on getting news
• 1:04:50
  right. Yeah.
• 1:04:52
  One thing that you're doing is you're giving the whole picture
• 1:04:58
  and you're giving both sides and maybe some of these sides of the
• 1:05:01
  picture. Me as a consumer, I wouldn't
• 1:05:04
  necessarily go and seek out, but I am or I like to think of
• 1:05:08
  myself as open minded and I don't want to just read stuff
• 1:05:15
  that is coming from an echo chamber.
• 1:05:16
  And so I appreciate different viewpoints.
• 1:05:21
  And now how are you making sure that the stuff that you're
• 1:05:26
  grabbing isn't just fake news? Is it because you have a good
• 1:05:31
  solid graph of it and you can see?
• 1:05:35
  That's part of it. There's a lot of human in the
• 1:05:38
  loop. We have an editor in on staff,
• 1:05:40
  an editor in chief who is in our staff.
• 1:05:43
  He was editor of the Rocky Mountain News for 20 years,
• 1:05:46
  right? So this, this is someone who
• 1:05:50
  takes this seriously. There's a lot of human review of
• 1:05:53
  the data, a human review of the output, human review of what
• 1:05:57
  sources we're tracking. You know, if you submit, if you
• 1:06:01
  submit a sub stack and it's clearly just complete and utter
• 1:06:06
  trash. We, we, we won't, we won't
• 1:06:09
  include it. But it's, it's usually where our
• 1:06:11
  standards will allow a lot to come in.
• 1:06:14
  Not really, but because and only because once you get to the pic
• 1:06:20
  point where you're painting the picture as a user or as us, we
• 1:06:24
  are the user of our own service in some ways, you can really
• 1:06:28
  make strong decisions of how to put competing perspectives
• 1:06:32
  against each other. And like I said, you can philtre
• 1:06:35
  on objective reporting voice. You can detect when the voice of
• 1:06:40
  what was written was trying to evoke emotion and that isn't in
• 1:06:45
  a strong indicator that this is probably, there may be
• 1:06:49
  interesting details there, but that's an indicator that maybe
• 1:06:52
  we should leave this source with less credibility.
• 1:06:56
  You can even still pass it to the LLM, but you can just say
• 1:06:58
  label this, this is sensational, right?
• 1:07:01
  And now treat the sensational sources as they should be
• 1:07:04
  treated with less credibility. You can.
• 1:07:07
  It's a big grey zone of how you can approach that problem.
• 1:07:10
  It's a big experiment, to be completely frank, like the front
• 1:07:13
  news, the front page of Ask News.
• 1:07:16
  That is where we do that high level clustering.
• 1:07:19
  We're using our own service on the back end, but we're writing
• 1:07:23
  our own work, which is bringing in the the Chinese, Japanese,
• 1:07:27
  French, all of the different perspectives and in one.
• 1:07:31
  And that has been when we started, we didn't know if it
• 1:07:34
  would work very well. But what we've noticed over the
• 1:07:36
  past year, we've been tracking the Israel Palestinian conflict,
• 1:07:41
  which is maybe the most polemic topic that currently exists.
• 1:07:46
  And every time we write an article, not every time, but
• 1:07:50
  many times, I'll look at one and I'll say, I'm going to send this
• 1:07:53
  to someone I know who is very pro Israel and just ask them for
• 1:07:56
  their opinion. And I'm going to, or I'm going
• 1:07:59
  to send this to someone who's very pro Gaza and just ask them
• 1:08:01
  for the opinion. And honestly, they're, they,
• 1:08:05
  they always come back and say that's a decent representation
• 1:08:08
  because the way that it reports is following these editorial
• 1:08:12
  standards of, well, according to the, the Health Ministry of, of
• 1:08:19
  Palestine, this is what happened.
• 1:08:22
  And that's kind of what an Israel, Israel reader wants to
• 1:08:25
  see, right? Because they want to know what
• 1:08:28
  the, that health ministry is saying.
• 1:08:30
  And as long as you say that's who said it, this is where the
• 1:08:32
  information came from. It's, it's up to a pro Israel
• 1:08:36
  person to decide if they trust it or not.
• 1:08:37
  And same with the the Gaza side. If, oh, the government of Israel
• 1:08:41
  has said, you know, Benjamin Netanyahu came out and said this
• 1:08:45
  is what happened. Well, at least it's being
• 1:08:48
  reported in that fashion. And and then the contradictions
• 1:08:51
  are painted, right? It's OK.
• 1:08:54
  This the, the Israel government says this, the Palestinian
• 1:08:56
  government says this. And this is the contradiction,
• 1:08:59
  take it or leave it, right? And no point did we say anything
• 1:09:02
  in this was fact, nothing was true, nothing was false.
• 1:09:05
  But we've said these are where the alignments are.
• 1:09:08
  Believe it or not, a lot of times the Israel, you know, the,
• 1:09:11
  the stuff coming out of the Israel government actually
• 1:09:14
  aligns with some of the stuff coming out of the, the Gazan
• 1:09:17
  health Ministry because a lot of the facts are indisputable about
• 1:09:21
  where the thing occurred, right? There are journalists on the
• 1:09:23
  ground, there are pictures, there's a lot of aerial
• 1:09:25
  evidence. So the key alignments are very
• 1:09:29
  nice to see. And I think that helps those
• 1:09:32
  people when I send the polemic topics out.
• 1:09:35
  So I, I'm not saying we're solving it, but we definitely
• 1:09:38
  have found a very unique contextual representation.
• 1:09:42
  And you know, the, the final point of evidence for this
• 1:09:46
  methodology. One of our favourite clients,
• 1:09:50
  other favourite client is the University of Texas and they're
• 1:09:54
  doing research into misinformation detection and
• 1:09:57
  their new method, Credi RAG is a RAG based credibility system
• 1:10:03
  based on ASK news, but also using some wild techniques of
• 1:10:09
  building out social graphs between who said what and why
• 1:10:11
  they when they said it. And but ask news is this key
• 1:10:15
  component and they're finding that this is working really
• 1:10:19
  well, bringing in all this different, bringing competing
• 1:10:22
  perspectives for this. They'll basically they take a
• 1:10:25
  Reddit post that might be misinformation and they search
• 1:10:30
  ask News database to get all of the diversified viewpoints.
• 1:10:35
  And then they pass that plus the the Reddit to an LLM and you can
• 1:10:39
  classify it. You can say, OK, what does this
• 1:10:42
  seem to be misinformation? Does it seem to not be?
• 1:10:45
  And that works very well. Their accuracy is up to like 95%
• 1:10:51
  on the detection, which is up 25% from the previous
• 1:10:55
  state-of-the-art. So that's a really fun
• 1:10:57
  validation. I love bragging about it.
• 1:10:59
  I appreciate you giving me the moment for that.
• 1:11:00
  But yeah of. Course, it's so cool to see and
• 1:11:04
  I'm a firm believer that there's almost like three sides to every
• 1:11:09
  story. You know your side, my side, and
• 1:11:12
  the truth is somewhere in between there.
• 1:11:15
  So this in a way, is how you can get both sides of the story and
• 1:11:24
  you see where it matches up and then you see where the
• 1:11:26
  narratives take different turns for better, for worse.
• 1:11:32
  So that that's super cool that you give that because I I like
• 1:11:37
  your friends that you're sending these articles to want to feel
• 1:11:40
  like I'm at least getting the other side of the story, whether
• 1:11:44
  or not I believe it adds my choice or whether or not I
• 1:11:47
  choose to support it. I guess less than believe it is,
• 1:11:51
  Do I want to support that argument or another argument?
• 1:11:54
  But I want to at least know what are both of these arguments
• 1:11:58
  exactly? Something that I was thinking
• 1:12:01
  about too. Are you using podcast?
• 1:12:02
  Because it feels like that podcast transcripts could be a
• 1:12:05
  good news source too. We are looking into, we have an
• 1:12:11
  in the development environment, YouTube transcripts, but the
• 1:12:14
  legality of it of going public with it is some is the main
• 1:12:17
  blocker right now. There's a lot of, you know, you
• 1:12:22
  got to be very careful with how you use YouTube data because
• 1:12:24
  they're not really designed to be freely scraped like that.
• 1:12:30
  Yeah. So, yeah, I think we would
• 1:12:31
  probably need to sign up individual podcasts to really
• 1:12:34
  get them in. But there's so much good
• 1:12:36
  information being transacted on podcast.
• 1:12:38
  Like I completely agree. Even in news YouTube videos
• 1:12:43
  themselves, you know, I think you probably know of DW, right?
• 1:12:47
  The the German one. I find them to be very, you
• 1:12:51
  know, matter of fact straight. And I would love to get that
• 1:12:55
  information as another competing perspective from their, from
• 1:12:59
  their YouTube videos. But the the the scraping YouTube
• 1:13:03
  legality is a is a is one that's a bit hard to to handle.
• 1:13:07
  Interesting. Well, man, this has been
• 1:13:10
  fascinating. Before we go, I have to ask
• 1:13:13
  because you kind of talked about it back when we were mentioning
• 1:13:18
  the ontology. You said the way that you're
• 1:13:22
  doing, no ontology is beneficial in some ways, but you said for
• 1:13:28
  other ways it's not beneficial. What are the other ways for
• 1:13:31
  those people that have sat with us for this whole conversation,
• 1:13:36
  at least give them some kind of pay off and let them know what
• 1:13:38
  the other side of the story is because I know that that is that
• 1:13:41
  is also important. Like we can't paint it out to be
• 1:13:45
  all roses in tie dye, right? Yeah, a lot harder to handle
• 1:13:51
  more data. It'll become slower to do your
• 1:13:54
  traversals, it'll be in. Not only that, but harder to
• 1:13:57
  formulate your query in some ways, depending on if you're
• 1:14:01
  going to stick with this very free flowing ontology of
• 1:14:06
  ontology freeness. You can't really decide your.
• 1:14:09
  Your queries are harder to build because you don't have a
• 1:14:12
  definition a priori in order to decide how that query should
• 1:14:16
  propagate. So that would be the biggest
• 1:14:20
  limitation. However, with the raw data and
• 1:14:23
  like I said then moving into a mem graph, you could you can use
• 1:14:29
  our metadata to to build it out. It's just a matter of deciding
• 1:14:33
  what you need. But communicating to LMS, I find
• 1:14:37
  that to be the, the biggest difference is like the strength
• 1:14:42
  of the ontology is querying and being able to extract insights
• 1:14:47
  that way and getting really directly to where you need to go
• 1:14:50
  in that graph. Even using node to Vec in order
• 1:14:53
  to, you know, predict links and stuff like that.
• 1:14:56
  Doing things that are very, very creative, but passing really
• 1:15:02
  highly resolved information to an LLM to extract that, that
• 1:15:07
  might be where there's a slight weakness.
• 1:15:09
  So that would be, I think, the give and take.