Latest / Agentic Conversations (formally mlops.community) / arrowspace: Vector Spaces and Graph Wiring

Transcript

arrowspace: Vector Spaces and Graph Wiring · Agentic Conversations (formally mlops.community)

0:00
Because once you have data evaluation, you have
0:02
classification, you have search, you have a new set of tools that
0:05
you can, that you can implement with this, right?
0:09
And that's what graph wiring is about.
0:12
What do we do with this super nice, super new, cool tools
0:15
provided by AP Plexity to actually supervise, manage,
0:21
curate data sets for machine learning operations and large
0:25
language model operation? You've been working on a lot
0:33
over the last years. Can you break down for the
0:37
listener what you've been sinking your teeth into?
0:41
Yeah, absolutely. Very, very briefly.
0:44
It's like I've been actually experimenting a lot with drugs
0:48
since like March last year and I've run into some limitation of
0:55
vector search. But then like as I am like while
1:00
the rabbit old person, like probably a lot of developers and
1:04
engineers around, I started to dig a little bit into the
1:08
limitation of vector similarity in general.
1:12
So that's how I've actually started this project and
1:17
published this library called Aerospace that is an attempt to
1:21
make vector search more accessible and more powerful in
1:24
some sense, especially in the scope of text embeddings.
1:29
So like vector space, but not just like vector space, but
1:34
vector space with highly semantical connections.
1:41
So text embeddings for language, but also like in general any
1:46
kind of vector space that has an highly component of connectivity
1:51
between the features in their beddings.
1:54
And then yeah, I went, I went through with this and I started
1:56
writing a few papers. Now I am at the 5th paper since
2:00
October. I wrote two papers last year and
2:03
three more now, especially the last one, I'm very like happy
2:07
with that because I draw this connection between this way of
2:12
doing vector search and what I call graph wiring.
2:16
That is by basically the generic application of this particular
2:20
vector search and the way of measuring information that the
2:24
that is now like being proposed in January, that is called the
2:28
epiplexity. That is basically a new way of
2:34
looking into entropy and complexity.
2:39
So like this, this, this, this, this journey brought me from
2:43
like vector search, like trying to apply all these tools
2:48
connected to racks to like some more like interesting, like more
2:51
like higher level kind of concept.
2:53
But also like I'm trying now to build a set of tools based based
2:58
on these foundations. And I think they will be like
3:01
very, very helpful for everybody doing language model operations,
3:07
but also like engineer machine learning operations.
3:11
OK, so first of all, amazing name with Epiplex Cities.
3:16
Oh, it's not mine. It's it's a paper from from the
3:19
University of New York and Carnegie Mellon.
3:22
So they incredibly the, the I, I, I took the, the, all the
3:27
theoretical part from them and I, I just wired in what I've
3:32
been working on and the things just like aligned perfectly.
3:35
So I think we were like on the same wave somehow.
3:38
We were like, just like surfing in parallel at some point.
3:41
And in January, my paper on, on like one of my moon shots called
3:46
the topological Transformers. And this Epipex City paper just
3:49
came out at almost at the same time.
3:51
And I would say, oh, look, this is what I'm doing.
3:55
And yeah, OK, I have a measure for it.
3:57
Epipex City. Oh, that's great.
3:59
So what was hard about vector search and what did you do to
4:03
change it? Vector search has been developed
4:07
in the ears as as a purely geometrical kind of operation.
4:13
So you have like I dimensional vectors.
4:16
You just compute the distance between these two vectors.
4:18
You use your own like favorite distance metrics that can be
4:22
cosine, L2 or any other kind of metrics and you got a score out
4:26
of it, right? So these two vectors are are
4:28
X&X&Z distant, right? They're like applying this to
4:36
semantically dense. What I call semantically dense
4:40
embeddings is that some of the information that then I
4:45
demonstrated being part of the API Plexity framework basically
4:50
get lost in the in the folding or in the like reduction of like
4:56
the vector space into a geometrical space.
4:59
There is some part of structural information that is encoded in
5:03
my, in my, in my implementation, in my algorithm by the graph
5:09
described by the relationship between the features.
5:12
So you have column vectors, you, you compute the the connection
5:15
between these column vectors in the vector space and you get
5:18
this extra that I call topological spectral
5:21
information. And yeah, it's I went through
5:26
all these experiments and basically I found out in the end
5:29
that is demonstrated in the in the last three paper I published
5:34
that that I hope everybody, I mean, of some people is going to
5:39
to double check. And so I can, I can get some
5:43
very good feedback that basically some of some, some
5:46
piece of information during the bedding process get lost or in
5:50
like dimensionality reduction or in like, you know, in the way
5:54
like the the bedding process works by basically computing
6:01
only on the on the item space and totally forgetting the
6:04
feature space of the of the data set.
6:08
And that basically this can be fixed for geometric for for
6:13
these like geometrical vectors for geometrical embeddings by
6:16
increasing the number of dimension of the embeddings.
6:19
So if I started from 300 and the four dimension embeddings.
6:23
So it's like a usually like a kind of usual denoising out
6:27
encoder. And I, I, I actually run my test
6:31
comparing search using like cosine search on this limited
6:36
space embeddings and outer space, my library in the same
6:40
space, right? And I and I actually found out
6:42
that there is some there is some way of doing this search better
6:46
because there is some some kind of chunk of information called
6:51
that I call like topological information there get get lost.
6:54
So applying topological search to the same space I can achieve
6:58
better. There are different scores for
7:02
that in general, some kind of better search in in a more like
7:08
semantically meaningful search. I developed a score for for this
7:13
that is called the MRR top zero that is basically subtitled the
7:19
topological page rank. That is a way of measuring these
7:21
things. So I measured these things in
7:24
the geometric search and in the outer space search, topological
7:27
search and I found this gap. Recently I've done the same
7:30
thing like with larger embeddings.
7:32
So I've done the result the same thing with a 10/24 dimensional
7:36
embeddings and what it came out is that ah look, this piece of
7:41
information that got lost on the 384 out of the noising denoising
7:46
out encoder is actually preserved by this new way of
7:49
doing embeddings but with much higher dimension.
7:53
So basically what was happening is that I could do the same
7:56
level, the same quality of search of 1024 dimension
8:01
embedding using the 384 embeddings because Autospace
8:06
basically rebuild somehow the information that got lost in the
8:10
bending process through through what epiplexity measured as
8:16
generated information. So basically I was generating
8:19
information in the in the in the framework as as it is explained
8:24
by the framework of epiplexity. I was regenerating the
8:27
information that was lost by the bending process using my
8:32
algorithm Autospace. And the outcome is outer space
8:38
work better in topological terms, both in the 384 and in
8:41
1024 dimension space. So yes.
8:44
And it allowed basically a 384 space to almost work as a, as a,
8:51
as a space that that three times the number of dimensions there
8:54
was like a very like, oh, kind of discovery.
8:58
So this is fascinating to me because it's almost like you're
9:04
getting a bit of a cheat code. You're getting this extra
9:08
dimensionality for free. What are the downsides of it?
9:12
Because I imagine there's no free lunch.
9:15
No, absolutely. Yeah.
9:19
I mean, let's, let's take a step back in the general objective.
9:23
The general objective was we want a better, more fitter
9:29
search to apply rags on right? Because retrieval augmented
9:33
generation depends a lot from searching among documents.
9:37
So the initial hypothesis was that geometric search found very
9:42
well the first 345 top ranked documents, but starts to fail
9:48
very steeply after the 5th document.
9:50
And that's demonstrated with comparative test.
9:54
I wrote different blog post about this because that's how it
9:58
works. It just finds very well the top
10:01
three for five documents. But then it's performance just
10:05
like go down steeply for like the tail for the tail of the of
10:09
the of the ranking. So I said, OK, but retrieval,
10:13
augmented retrieval and it's something that is maybe as like
10:17
a better, more a smoother distribution, right?
10:20
We don't want like to have big performance on the 1st 3 and
10:22
then OK the rest whatever, whatever, whatever happens with
10:26
the rest, it's OK, right? No, because like Rag has to
10:30
maybe walk down different pathways to do some kind of
10:32
reasoning, get some chain of thought.
10:34
So you can start with like with the top three ranking results
10:38
and they have a great like outcome.
10:40
But then at some point you start looping and you start like get
10:43
stuck in your local minima because as many query you run on
10:47
the on the on your documents, you always return the same, the
10:50
same top 3454. So there's I said there should
10:53
be a way to actually make this the distribution of the ranking
10:57
like less like steep and allow the search to to look at the
11:04
lower ranks. So that if if you got stuck in
11:06
the top ranks with the top ranks, there is something maybe
11:09
got in this lock and minima on the top ranks just OK, let's go
11:13
and look at the lower ranks, right?
11:15
So that's actually exactly what topological search does.
11:19
There is these sliders in which you can basically modulate how
11:23
much geometric search and how much topological search you
11:26
want. So if you want like, OK, let's
11:28
start with pure cosine search. OK, slider 100% we look at the
11:34
top three, top five results. But then maybe you want OK, but
11:38
let's try to open the the reasoning, right?
11:40
Let's try to to to look a little bit on the edge of the
11:43
distribution a little bit. But just look at this tail.
11:46
What happens if I if I search like less, less popular or less
11:50
like top ranking results, right? So you can adjust this lighter
11:55
and you go down to almost like 40% of mix between like 40%
12:03
consigned search and 60% topological search.
12:05
And you start having like very good, still very good results.
12:09
But they somehow give a different point of view or a
12:11
different like kind of approach to the problem the the top
12:15
three, top four rank. Right, I like that.
12:17
And so you can just like tell your LLMS to just through to a
12:21
simple like MCP server. Oh, let's adjust my search.
12:25
I mean, I'm, I'm stuck in this loop at this point.
12:28
Maybe I should adjust my search. OK, let's put this ladder down.
12:31
Let's go. Let's look more into topological
12:34
kind of search. And you just like start
12:38
receiving very still like very highly related, highly
12:41
meaningful documents for the contest, but different ones,
12:45
right? So you can restart your chain of
12:47
thoughts and say, OK, this way, I, I reason this way, this way,
12:51
I reason this other way. What are the common patterns in
12:54
these things? And so on.
12:56
It's a way to let the LLM keep exploring on relevant data.
13:00
Like that, it's like when you do geometric, you basically go
13:04
depth first. If you adjust the topology, you
13:07
start opening the graph and starting to see like more like
13:10
breadth first, right? Yeah.
13:12
Now, you mentioned that there's maybe a way to implement this
13:16
with MCP servers. Have you seen any of the vector
13:19
stores or vector databases already incorporate these
13:23
algorithms into it? Would it be at that level of a
13:26
vector store, or is it something that you add on top of it?
13:29
Where does it play in the stack? It would be just a different
13:33
track for for running search. You can have like search
13:35
geometrical and search topological and you got no,
13:39
nobody implemented that. I have like some side project
13:42
trying to do these things like running comparative rugs,
13:46
installation 1 using traditional geometric vector space, search 1
13:53
using mixed hybrid topological geometric and things like that.
13:59
So we'll see the results soon enough.
14:01
I guess the fact is that geometric search is so well
14:04
established and so well optimised that obviously is the
14:08
first way to go for like 90% of A use case, maybe right up to
14:13
the point you need some very good reasoning or you start
14:16
having like very complex reasoning.
14:17
That's the point we are, we are reaching now with drugs, right?
14:20
So in the last 20 years or so and the last, like even before,
14:24
right, geometric search was perfectly good because, and it
14:28
was like I, and it becomes like, I optimize with this like HNSW.
14:32
Now we have like, so we have like these 7 layers of graphs
14:35
trying to, to look into like billions of records, right?
14:39
And it's great. But maybe we need to push these
14:42
things a little bit more now because we have like more
14:45
intelligence, a system to, to deal with, right?
14:48
So that's my that's my point of view.
14:52
Yeah, so is it? It's basically a.
14:57
If I'm understanding you correctly, you want to use the
15:01
tried and true algorithms until you saturate them and then reach
15:07
for something like this. Basically what I'm trying to do
15:11
now in, in experimental terms is that I add topological
15:14
information to the geometric search because in the end it's
15:17
like it's a way that sum, right? You have like an alpha that is
15:20
like the the geometric search and you have a beta that is the
15:23
the topological search component, right?
15:26
So through, through the aerospace to the library, you
15:29
can modulate these things until you see that you reach like
15:34
something better, maybe like just the marginal better that
15:37
that is just like a marginal 5% of 8% that I demonstrated in,
15:41
in, in my blog post. But that kicks the, the rags out
15:49
of his, of his local minimum, maybe right and, and allows him
15:53
to actually, OK, let's climb up the, the, the local minimum.
15:57
I, I'm, I'm running around now and let's get in, maybe reach
16:01
another local minimum. That tells me something
16:03
meaningful as well, and very much related, but different in a
16:07
way that I can re elaborate my chain of thoughts in some
16:11
different way. So it's that razor thin edge
16:15
because you don't want to introduce too much noise and
16:19
then have irrelevant data. Comes the fact is the problem
16:22
with joint similarity is that he has more noise if if it's run
16:26
like as a pure geometric search, because you just get stuff that
16:30
is similar geometrically, but not in meaning because there is
16:33
no semantical kind of structure that tells, oh, this is, this is
16:38
very, this is geometrically very similar.
16:40
But maybe it's, it's totally out of context, right?
16:42
That's why you get like if you run like a geometric search and
16:46
you and you ask for like the top 50 maybe records, the last 20
16:50
records are just like garbage documents or the last 2020
16:54
documents is just like garbage documents, like like gibberish
16:57
documents because that maybe they just happen to be close in,
17:03
in the in the geometrical space, but they are semantically
17:07
totally not relevant to what you are looking for, right?
17:10
And that's, and that's what topological search fix in the
17:12
1st place plus is allowed a better like a context that there
17:20
is a ratio that is the diluted is the the ad tailor ratio
17:23
right? In which like we measure the
17:27
distance in in terms of semantic, semantic distance
17:32
between the, the, the AD and the tails with with geometric
17:37
similarity. This, this is always unbalance.
17:40
Like the ad that tells you are are about something and the
17:43
tails are about something else. With topological search this
17:47
thing is a balance. So the add in the tails that
17:49
talk to the same semantic space because we inject like semantic
17:54
information in the in the search.
17:57
Now how does this play into memory?
18:01
Because I know you've been going down the memory for agents route
18:08
quite deeply also. Yeah, basically at some point
18:11
this became something like, OK, maybe we can do like search a
18:15
memory in the same structure, in the same data structure, right.
18:19
Because we we basically generate this this graph out of the bed
18:24
space. And this graph is a sort of
18:26
permanent memory of the semantic space because basically it
18:30
collects the invariance of of this semantic space.
18:33
So if you have like a bunch of legal documents about some
18:36
particular kind of or a bunch of philosophical or papers about
18:42
some kind of philosophical topic, right, you build the the
18:48
graph Laplacian out of this out of this embedded space of this
18:52
document and what the graph Laplacian represent.
18:56
There is like kind of a good kind of background work on this
19:02
by all the all the researchers that work on Laplacian
19:06
representation for for vector spaces that basically represent
19:10
the invariants of this space. So if the text embeddings, if
19:13
they were, if the vector space is a text embeddings.
19:16
So basically a reduction of all the all the meaning in in a, in
19:21
a given field of study, the graph Laplacian automatically
19:25
it's a representation of its invariants.
19:28
So you have like that somehow you get some like long term
19:31
memory There you have like a summary of the summaries of all
19:35
the documents you have in your vector space, all compressed in
19:38
a sparse matrix. Basically this part matrix is a
19:42
very interesting structure. I talked about it extensively in
19:46
my paper and in my blog post. It has incredible properties and
19:50
it's heavily used, but in the in the item space of the vector
19:55
space. So basically, let's build the
19:57
Laplacian on all the items of this vector space.
20:00
What Autospace does, it flips the concept.
20:04
By saying no, we, we, we want to look for invariants in the
20:08
feature space. So we want to look for
20:11
relationship between features. What is the relationship between
20:14
the colour of all these documents, the length of all
20:17
these documents, the sentiment in all these documents, right.
20:21
Not just like what is the the relation between the the colour
20:24
and the sentiment of documents A and documents B?
20:28
No, we want to look into the graph, the column vectors, the
20:32
feature. Graph, and you're doing it at a
20:33
document level, not at a chunking level.
20:37
Every. That's really depends how you do
20:38
your embeddings. If you chunk, if you chunk your
20:41
documents in your embeddings, you will come out with the with
20:43
the with the banded chunks, right?
20:46
I usually I started. The example I've used is the CVA
20:49
data set. That is a data sets of reports
20:52
of common vulnerabilities in, in in software and systems.
20:58
And usually these are like large report in in Jason format.
21:02
So they're just basically text file with a title, a
21:04
description. I just build my beddings by
21:06
passing the entire document. But in theory, yeah, if you have
21:08
like books, you can just like you don't like chunks the into
21:15
paragraphs at the time of the beddings.
21:17
And you can do like, yeah, you can just say, OK, then yeah, you
21:21
have to do the reconstruction, say which chunks belongs to the
21:24
same book, which chunks belong to the same paragraph, which
21:27
chunks belong and etcetera. But yeah, this works the same
21:30
both for documents and chunks. OK.
21:32
So sorry, I cut you off and derailed you a little bit.
21:35
You were talking about how it's flipping on it, on its head,
21:39
looking at the features as opposed to that.
21:42
And so how can the features tell you information?
21:46
That's what I'm I can't make the connection on.
21:49
That's the real semantic metadata that you're looking
21:51
for. Like when you look into what we,
21:54
there is like an infinite number of division for metadata, right?
21:58
But in the knowledge graph space, for example, that you
22:01
know very well and your your audience knows very well.
22:04
Obviously what you have is like it's a graph that goes on top of
22:09
the existing graph. So you have the graph of the
22:10
nodes, right? And then you have all the the
22:13
graph of your classes on top of the on top of your graph of the
22:17
nodes, right? And you call that metadata is is
22:20
the relationship between class of documents, is the properties
22:23
that connects documents in a way defined by the class level, by
22:28
the metadata level, not by the the instance item level.
22:32
Same thing for vector space. There is a space in the nodes
22:36
that are that is the space of the documents, the item space.
22:39
But then there is a space that connects the features because
22:43
each node as as is is feature vectors.
22:46
So we said this node is represented.
22:50
All the nodes of this space are represented by this feature
22:52
vector for dimension 0, the color or the length.
22:58
But there is another vector for the dimension 5 that is like the
23:03
another characteristics of the vector.
23:05
How these features connects together, So you see that you go
23:09
to a second order kind of layer, that is that I call it metadata
23:12
layer because that's exactly what it is conceptually, right?
23:15
And it turns out that there is information there that can be
23:18
used for search because you don't only search the node
23:22
space, you search the feature space, you can search the
23:25
feature space. And that's what Mathematicia
23:28
called the topological because this space gives you somehow the
23:32
third dimension, the Riemann space of the vector space.
23:36
Basically this is the space of the second derivative of the of
23:40
the vector space. And that's where a lot of cool
23:43
stuff is because that's where like all the structural
23:46
information is found. And that's the structural
23:49
information out of space injects in your query to find, oh, this
23:53
very super cool new results or results related to your query
23:58
that are not geometrically closed, but maybe they follow
24:00
different pathways down this morphology and and they're still
24:05
related, but may be forgotten by the geometric search.
24:10
So you're kind of blowing my mind here that that metadata has
24:16
any value. Exactly, that's what it was
24:19
forgotten. Basically, it was like left
24:21
there like an archaeological relic with no use right.
24:25
And no, it has so that I found this like last year as well
24:28
myself. And he was like, Mark, come on,
24:30
this is not possible that nobody looked into this thing before.
24:34
And they said, OK, let's try. I mean, I, I have, I mean, I'm
24:39
looking, I'm actually taking this, this time to, to walk down
24:43
this, this rag vector space kind of problems.
24:48
And, and that's true. That's, it's there.
24:50
And through the aplexity framework, I, I measured it like
24:55
last month, I ended up measuring what was missing by, by the
24:59
geometric search. And it's, it's what is called
25:04
structural information by the perplexity framework.
25:07
And it's measurable. It's almost like 2030% of the
25:11
total information. So every time we, we run a
25:13
search on the geometric search, we basically lost 2050, 2015,
25:17
20% of the information we could have used that is there.
25:21
But we, we do the, we didn't regenerate because we didn't
25:24
rebuild the graph Laplacian. And this is mathematically solid
25:30
as far as I could like investigate.
25:33
And I'm very, very, I would be very, very helpful to anybody
25:36
that will look into this to tell me, no, you're, you're not
25:39
right. I would be like the, the
25:41
happiest person if somebody founds the found some fault in
25:44
this reasoning. But at this point, it looks like
25:48
it works because I have run like 10s of experiments.
25:52
I have run 10s of in different settings like again like
25:55
shrinking or increasing the number of dimensions of the
25:58
embeddings and I hope I hope to be right.
26:03
So that's where you're getting these dense vectors for free,
26:10
basically. Yeah, we're getting information
26:12
out of the same dense vector space because we weren't looking
26:15
in the feature space before. Yeah, OK.
26:18
I'm, I'm starting to wrap my head around it.
26:21
And then the epiplexity, you should probably breakdown what
26:24
that paper is and what your paper was, because I know you're
26:28
referencing it quite a bit, but I'm not super clear on exactly
26:32
what it was. I think nobody is because it's
26:34
such a new thing and I, I guess they have only one citation for
26:40
now in, in scholar, in Google Scholar, because it's such a new
26:44
thing. So I, I, I, I don't know if I
26:46
cannot really. Maybe you can, you can actually
26:49
ask as a guest the people that wrote the paper.
26:54
But my, my basic understanding is that basically what is
26:58
Shannon information measure is what is what they called the
27:02
random entropy. But they said look if you in
27:06
general terms, like if the universe was a vector space, you
27:10
will have like you will, you will witness like Shannon
27:13
entropies. But they say look, but everybody
27:17
that computes something is not like is doesn't look at the
27:20
entire universe. It looks only at the problems
27:23
that are in his computing capacity.
27:27
So if I can compute up to certain level of for for up to a
27:32
certain time with a certain amount of computing power I can
27:36
reach, I can compute this this number of algorithms, right?
27:42
It is true that there are infinite algorithm out there and
27:45
they all like are under the law of of entropy.
27:49
But if I actually limit the, the investigation of only which
27:56
algorithms we can compute that is related to what Wolfram call
28:02
computational boundedness, I can I, I guess it is so is that we,
28:10
we can actually measure random entropy, but also something else
28:14
called structural information and epiplexity is basically both
28:18
of these thing instead of just looking at a random, a random
28:22
entropy, it looks both like bounding the the possible
28:28
computation, but to the observer to to the person that runs the
28:31
computation and to it's like computational power.
28:35
We can also highlight and measure this other thing called
28:40
structural information. So and in this like in this like
28:46
framework, the graph Laplacian in my outer space algorithm will
28:50
be the structural information part, while geometric search
28:54
deals only with like this very wide generic kind universal kind
28:58
of construct that is like the geometric space of of of the
29:01
vectors. OK, so I got this far in
29:05
understanding it. Your epiplexity is constraining
29:10
the space that you work with. Exactly because like, like it in
29:14
in physics, right? In, in, in, in modern, not, not
29:17
modern, but like in contemporary physics, you know something only
29:22
because you go and observe it, right?
29:23
So everything you see is limited by by by your detector.
29:28
So your eyes or you're like Andron Collider, right.
29:31
So anything is dependent of those server.
29:34
So if you actually does the same thing for information, you say
29:36
look, we cannot just like it's it's more like what like
29:40
relativistics physics does, right?
29:42
We don't look at the entire universe.
29:43
We look at the at the frame of reference.
29:45
So we have two bodies, they move at relative speeds to each other
29:50
and what happens depends on which observer you pick on and
29:54
how fast this observer is going in relation to the speed of
29:59
light, right. It actually, it actually does
30:01
the same apply the same thing to information saying look, you
30:04
cannot compute like the entropy of of the universe.
30:07
Yeah, you can, but it's not like a rule that applies to it, maybe
30:11
not the rule that do not apply to every observer in the
30:14
universe, because there are observer that go faster and the
30:16
observer that goes lower, right. So the same thing there are
30:19
observer that have a vast capacity for computing and there
30:26
are others that are like limited in terms of computing.
30:29
So for for for the observer, for one observer, it may happen
30:33
that's a problem is uncomputable.
30:35
What for another observer, it may happen that it is computable
30:37
because he has he has access to to more computing power and
30:41
maybe you can run the algorithm for more time because he lives
30:44
longer or because like whatever other reason, right?
30:47
And so if it actually try to bound every algorithm inside the
30:52
what is called the A model, I don't know.
30:59
I don't want to get too much in the day like a random work
31:02
model, that is, that is bound to the observer.
31:05
So you do this operation, you take the algorithm, you
31:07
encapsulate this algorithm inside a random worker that is
31:11
observer bounded. And then you compute the
31:15
structural information that is generated by this bounded class
31:18
between the observer and the and the algorithm.
31:22
It's very, very, it is greatly interesting to me.
31:25
So I hope it is forever. When you bind it, then you're
31:29
able to get a score on what the. If you want we we can actually,
31:34
we can actually take a look into into a ju Python notebook that I
31:37
wrote exactly to explain this point in my in my last paper if
31:41
you want if we have time. Busted out, Yeah, I would love.
31:44
That OK, I wish I will share it. No problem.
31:46
So basically this is the paper and everybody can can go take a
31:50
look at it. But if we if we scroll here on
31:54
the left, can you see it? Yes, there is like the notebooks
31:58
directory and there is like this 00 in this notebook.
32:03
Basically I try to to explain to myself what epiplexity is and
32:07
how it relates to my to my research and is the and is the
32:12
object of my of my latest paper. As you see, outer space feature
32:17
space graph Laplacian is structural information.
32:19
So the, the question was, is it true that that aerospace graph
32:24
Laplacian encodes like encapsulate structural
32:28
information as described by the by epiplexity, right.
32:31
That's the main question. And if we go through all the all
32:36
the steps of these notebooks, it will find that yes.
32:38
But very briefly, to know like just what Epiplexity measures,
32:44
we can just take a look to like this, these two, these two or
32:51
three first steps. Basically what it does first, it
32:54
computes the minimum length program to compute the graph
32:58
Laplacian from the from the vector space.
33:02
It's a, it's a, it's a measure of what is the minimal set of
33:07
bits that transform A vector space into a graph Laplacian.
33:10
Basically, it's a Kolmogorov measure for complexity.
33:14
It's Kolmogorov's measures for complexity, right?
33:17
So what is the minimal set of bits that maps my vector space
33:21
to the graph Laplacian? And that's the first part to
33:23
compute the playback city, the second one exactly.
33:28
And this year, the outer space pipeline as a prefix free
33:31
program, right? You want to compute the length
33:35
of the outer space program. So yeah, there is like it's a
33:40
mathematical construct that does this thing.
33:43
So what is the minimal number of bits to describe this problem,
33:46
this program basically. And that's the first step.
33:50
This is all described in the Apriplexity paper published the
33:53
6th of January. You can go and look into that.
33:56
The second step is is the is the is the wrapping.
33:58
I was telling you before, because we we have to make the
34:02
Laplacian of this vector space into a Laplacian constraints
34:07
Gaussian Markov random field that that's it.
34:11
It's already, I have probably already with the acronym myself,
34:13
but I tried to went through these things like line by line
34:16
and try to understand it. So we do these things, we
34:19
basically encapsulate these this system between the observer and
34:23
the and the algorithm inside this this shell of of Markov
34:27
chain of Markov chain model, right?
34:30
Yep. And in the end, we run a test
34:35
how much these things, they compose the original space and
34:38
compress the original space. Because yeah, basically from the
34:43
paper you can actually extract there are three tests that tells
34:47
you your epiplexity measure is correct.
34:51
Here they are. One is the compression test, 1
34:53
is the spectral gap test, 1 is the downstream lift test.
34:57
If all these three tests passes you are measuring, your
35:02
algorithm can be measured in terms of Apiplexity and.
35:06
Apiplexity, just so I'm clear, is giving you more information
35:14
on the randomness that you're getting from aerospace.
35:18
It's totally different from entropy.
35:22
Apiplexity basically adopts traditional information theory
35:25
to what we discovered with machine learning and neural
35:28
networks, right? Like traditional information
35:32
theory told you that you cannot extract more information that is
35:35
in the data because the, the, the, it's just not there, not
35:38
there, right? So it tells you that whatever
35:41
you do with the data, you will get entropy, you will lose the
35:43
information, right? But we demonstrate that is not
35:46
true because we have algorithm now that they do generate
35:50
information out of out of the vectors out of data in general
35:55
generating new structures. And this comes back to the whole
35:58
idea of, hey, if we look at the metadata and we find trends in
36:02
that, that actually gives us a denser vector.
36:05
Exactly. We, we will get the information
36:07
that is not in the vector space. So epipaxy they basically
36:10
demonstrate that we can generate additional information from
36:14
existing information. So that not is not, is not that
36:18
every, every data space, every data set is just like whatever
36:25
you do with it, you will going to lose the information you have
36:28
because you're going to compress it.
36:29
You lose information, you're going to the composite, you're
36:31
losing from information. Say no, look, now we have
36:34
algorithms that if you, if you take a bounded observer to the
36:37
algorithm, you can measure that these things generate
36:40
information that it's just like, just like a thermal loss.
36:44
There is not just like randomness taking over.
36:47
You can actually do the this is connected to to me what is what
36:52
are called the inverse problems, right?
36:55
When you have like a noise image and you want to denoise them,
37:00
this this is called an inverse problem.
37:02
You are generating information that is not there by looking at
37:05
the relation of the pixel. You can generate but non
37:08
generate, but you can extract information that is hidden there
37:12
by using algorithms, right? That's the same principle.
37:15
Whenever you do denoising, you generate information out of
37:18
something that is not supposed to have that information.
37:20
So when you do like super resolution, for example of
37:23
images, you are doing this thing right, you go from 200720P to
37:30
10/24. PS right you, you, you super,
37:34
you apply super resolution to to the image.
37:38
And that's exactly what Epiplexity measures.
37:40
It measure how much information is generated by the algorithm.
37:44
No, how much information is consumed by the algorithm.
37:48
And they demonstrated the mathematically, I mean, it's
37:51
very, it's a very young obviously framework.
37:53
So it's not as established. And it needs to be like double
37:57
check and tested. And I guess this is the first
38:00
algorithm that test itself against epipexity.
38:04
So Autospace is the first algorithm that on top of which I
38:10
computed, we computed information generation, the
38:15
amount of information generated by, by the, by the computation
38:19
computational process. Yeah.
38:23
And that's, that's something that say, oh wow, I connect to
38:26
some dots and that's, and that was the thing.
38:30
And that's actually, that's the very latest thing.
38:32
But if you look back to my previous papers, that is the
38:36
work the the stepwise a stair stairway that brings you from a
38:43
simple like search in a vector space kind of algorithm to a
38:49
more general algorithm. Because at this point, as as it
38:53
is in the in the abstract, basically outer space is generic
38:59
enough to provide. I mean, graph Laplacian applied
39:02
by outer space in the feature space is generic enough to
39:05
provide a good approximation or in good results for searching
39:11
classification and only detection, diffusion,
39:14
dimensionality reduction and that evaluation, all these
39:18
obviously my idea is that OK, but this is super cool.
39:21
We, we need to use this for like for LLMS and and machine
39:25
learning operation, right? Because once you have that
39:27
evaluation, you have classification, you have search,
39:31
you have a new set of tools that you can, that you can implement
39:34
with this, right? And that's what the graph wiring
39:37
is about. What do we do with this super
39:41
nice, super new, cool tools provided by Applexity to
39:45
actually supervise, manage, curate data sets for machine
39:50
learning operations and large language model operation?
39:54
And and you got some answers about this, but my the paper
40:00
that that deals about applications.
40:03
A machine learning application and largely whatever is the
40:05
previous one it talk that talks very extensively how these tools
40:09
can be used for in the in AI OPS or ML OPS pipelines.
40:16
Well, it does feel like you're doing this with data sets after
40:21
the model has been created. Have you also thought about
40:24
trying to go for the data sets that the models are being
40:29
created on? You mean the the bending model?
40:32
Yeah. So the training data.
40:35
Yeah, exactly. These things, there is a big
40:38
question marks in whatever we do with LLMS currently because
40:44
everything is connected to how we do embeddings.
40:46
That's why that's why it's very important to, and there is like,
40:51
there are now like teams and teams of engineers working only
40:54
on how do we move from raw text to embeddings, right?
40:58
It's, it's a field on its own. We have models with 4 billion
41:00
parameters only doing this at the moment.
41:03
So that's one, one thing if we go instead of the, you know, on
41:06
the numerical side and we said, let's look into pure numerical
41:12
data, right? Machine learning data
41:13
regressions, trees, decision trees and all that stuff we
41:18
find. I found out, I mean, or my
41:20
intuition is that it really, really matters how we do Fisher
41:25
engineering because from let's say we have like this mass
41:29
amount of raw data like coming from the large andrum Collider
41:33
or whatever other big machinery for physics or for like any
41:36
other kind of measurement you have, right?
41:39
Obviously you don't run your machine learning models on your
41:42
the totality of this data. We are talking about thousands
41:46
of terabytes, right? So what you do, you do some
41:49
Fisher engineering. So you run some models to reduce
41:52
your data into some manageable workable.
41:54
This, this is the same thing. It happens with satellite data,
41:57
right? The image you see on your screen
42:00
from the satellite, it's just like a model of all the raw data
42:05
that the, that the satellite collects and, and, and push down
42:08
to Earth, right? So it's really important how the
42:12
people at NASA, at ESA or whatever else, they, how they
42:15
model these, these algorithms to actually make this, they make
42:20
this data usable. My intuition is that if we try
42:25
to embed in, in some way or in a better way, we do now the
42:30
features relations, while we do feature engineering for, for
42:34
this data, we may have more powerful topological search
42:38
downstream. So as your question is really
42:41
relevant because it really matters how you treat the data
42:44
upstream. So but having, having having the
42:48
outer space, you can indeed measure how much structure and
42:52
information your feature engineering generates.
42:55
So you can compare if I generate these data sets using this
42:59
model, it it produce downstream 0 point something a perplexity,
43:06
right? Structure structural
43:09
information. What happens if the same data
43:11
we, I use a different feature engineering model and I compare
43:16
this, oh, look, this generate 1% more structural information.
43:21
You see that these things can work upstream and downstream
43:24
because my idea in the beginning started from, OK, let's apply
43:28
this to performances, large language model performances,
43:33
right? We can say, let's take the the
43:37
latent space and just like compute the Laplacian, the graph
43:40
Laplacian on the latent space, that that's that's what Ethan
43:44
Tropic they call, I guess, machinistic analysis.
43:47
So they go and and just like investigate, analyse the latent
43:51
space to find where the the best tokens are generated in which
43:54
subspace. That's exactly what you can do
43:57
with with the graph Laplacian. That's downstream.
43:59
But upstream, you can do what we were talking before, like let's
44:03
measure which best, which model does best feature engineering.
44:08
So you see that this is like pretty much like a quite kind of
44:12
effective point of view in terms of what we can do with the data,
44:17
how good is the data, how this data set will go if I had
44:20
something, what, how, how will this data set look like in six
44:27
months for now and things like that.
44:30
Coming back around, I'm not sure I fully understood how this
44:34
connects to memory with agents. OK that the graph Laplacian is a
44:42
permanent memory in the sense that he holds the he tells the
44:46
invariants of the of your context.
44:49
So basically it it tells you which pathways are possible from
44:55
one feature to another. I mean mathematician described
45:01
this thing as a three-dimensional space.
45:03
So for example, if you have like outliers in your in your feature
45:08
space, these outliers will be denoted with very high energy.
45:13
If you have like very connected kind of features, this feature
45:17
are denoted with the very low energy, right?
45:20
So and the graph applies and basically describes all the part
45:25
that you can take from a very connected feature to a loosely
45:31
connected features. So it basically it basically
45:34
constrained the way you can do reasoning somehow inside, inside
45:38
the inside the the feature space of the of the vector space.
45:44
OK, but we're not talking about agents remembering in the way
45:48
that it's like, oh, the agent understands that you like only
45:54
vegetarian food or? No, it's not about, yeah,
45:57
remembering scenes, what they're called scenes or what they're
46:00
called. Like they have different names
46:02
for that, right? It's not, it's not about that.
46:05
It's a different kind of memory, more like permanent long term
46:07
memory of a given context. But it can also be applied to
46:13
those because then if you have like a list of scene, a little
46:17
scenes that that your rags remember, you can do the same
46:20
thing because that's a vector space itself, right?
46:22
So you can, you can even bad things of this, of this, like
46:25
context memory. I don't know how you call it.
46:27
I don't remember how they call it and then do the same
46:30
operation on the context. So you can actually build your
46:33
memory and memories of your context, right?
46:36
That's, but that's a different kind of memory.
46:37
Indeed. It's not like this is more like
46:40
I I think they called it transient memory or something
46:44
like that. What, what the, what the vector,
46:47
what the graph Laplacians describe or define is the long
46:51
term permanent relationships inside a context of documents
46:56
that if you know it, it's a memory, right?
46:58
Because that's exactly what you want to remember.
47:01
If if you are an expert of that field somehow you want to
47:05
remember the invariants somehow and then connects your
47:07
invariants to to your to your application, or at least that
47:11
that's that's how I see the the the process itself.
47:16
Can you talk to me in a minute about this idea that you follow
47:22
on the discovery driven development?
47:26
Because I think that's also pretty fascinating on
47:29
understanding a little bit more on how you work and how you go
47:33
through and test some of these ideas.
47:35
Absolutely, yes. Thanks for the question.
47:37
Because that's I guess we have talked about the content and I
47:42
think the content may be interesting to people and I hope
47:45
so. But yeah, I mean going through
47:48
this, I also understood a lot about what I'm really doing here
47:53
like what what is my method right for, for doing this thing.
47:57
And my method is mostly based on intuition, like every based on
48:01
intuition that was great with large language models because
48:05
that's what large language model lack.
48:07
They don't have an understanding of the word that allows them to
48:10
be intuitive, right? Because they are constrained by
48:15
being, by their way of seeing the word, right?
48:18
They only see the word through language, through text.
48:22
So they cannot build the level of intuition that humans can do
48:25
because we have so many more senses, right?
48:27
So what I do is basically I try to leverage this very intuitive
48:30
part of of what I do. And I try to to inject this
48:34
intuition into into, into the process of multi agent kind of
48:42
research processes, right? So I work with different LLMS in
48:47
parallel and I just try to collect the thing that they do,
48:51
right? And that's just like try to
48:53
inject my intuition to correct what made I think that when I
48:58
think they're going a little bit out of the scope of what I'm
49:00
doing, right. So, yeah.
49:03
And this connects greatly with what happens in the meanwhile in
49:07
the in in scientific research, because in the meanwhile, we
49:10
have all these new papers published at at the level that
49:14
we have never seen before, right.
49:15
So you cannot really work in your own like lock down kind of
49:22
understanding of things. You always need to update this
49:25
understanding of things. Very Bayesian.
49:27
If if you if you, if you want, Right.
49:28
Yeah, it's Bayesian in the process, but also into listening
49:33
to what happens outside. And so actually when I found the
49:37
paper that connects, I instantly try to build new intuition of of
49:42
what I'm doing based on this new paper.
49:44
This happens with the Applexity paper and it's it is happened
49:49
also with another paper that was greatly impactful on me.
49:54
But that's a really long, another really long talk that
49:59
that I don't want. I mean, I, I think maybe you
50:01
don't have the time to start, but yeah, that's the that's the
50:03
process, right? You have your own Bayesian
50:04
process that goes on with your multi agent things.
50:08
But sometimes something from outside happens that is very
50:12
meaningful to what you're doing. And you cannot just keep going
50:15
on without including what what has happened at the in the
50:18
meanwhile, right? I said, OK, I'm, I build outer
50:21
space. I build what I call like this
50:24
super crisis experiments called the topological transformer.
50:26
There's a transformer architecture that works on
50:29
spectral indexing, like in topological search instead of
50:32
geometric search. That is very like kind of my my
50:34
moon shot. That is just like something I'm
50:37
not following at the moment. But that in experiment that I
50:40
did, I say, OK, but what what happens if if I look at
50:46
autospace in the framework of Applexity and and all this new
50:51
stuff came out right. So you can see how the method
50:55
somehow being so open to the outside, like putting in in
51:00
question, like asking question from things that comes from
51:04
outside what you're doing. It's fundamental because it
51:07
allows you to to stay in touch with what's happening outside
51:12
and to improve very much what you're doing.
51:15
Yeah, it's a kind of scientific science driven kind of
51:19
development somehow in this sense.
51:22
And how do you go about updating these frames of references?
51:27
Because I'm assuming you're constantly reading new papers,
51:30
you're constantly trying to build new intuition, but it
51:36
sounds like it's only occasionally that something will
51:38
hit. I have.
51:40
I have GitHub repos that are only text files with the with
51:44
the things like text that I've built using large language
51:48
model. I said OK this is good.
51:50
I put this in a text file. Inside this GitHub repository.
51:54
I have a file system that basically builds day by day on
51:58
top of this thing, and I have a bunch of text file that I didn't
52:02
open anymore, but they're still there.
52:05
But maybe there are like those 3 or 4 text files for each
52:09
directory that instead branched out into something else, right?
52:13
So I mean, I could go through to the entire history of what I'd
52:17
done because I have the entire file system of what I'd done
52:20
since like March last year. And yeah, and this contains the
52:26
prompts and the answers. So in theory, if you want to do
52:31
a meta research of what I'm doing, things that I have no
52:34
time of doing at the moment because of this, all the other
52:38
things I, I'm trying to carry on, I could actually do some
52:41
kind of which prompt worked best or which answer was the most
52:45
impactful in, in what I've done like in the, in the months
52:51
after, right. And all these kind of things.
52:53
Yeah, but it's just like basic prompt engineering prompt prompt
52:55
detecting, I guess. It's not.
52:57
It's nothing very anything. Very, very super fun.
53:00
There was one spicy question that I wanted to ask you.
53:04
If you inevitably have tried to play with vector search versus
53:11
just the giving an agent a tool, especially like a coding agent,
53:15
just being like, hey, grip and seeing what the differences are
53:19
in that. How do you compare those two
53:23
when you think to reach for one versus the other, if at all?
53:30
Yeah, basically what GRUB does is what is called lexical
53:33
search, right? That is also what BM25 does.
53:36
It basically just count the instances of the how many times
53:41
the word or the concept of the How is it called or the OR the
53:46
seed of the word is present, right?
53:49
That's called lexical search. That is part of geometric
53:51
search. Now you have all of this like
53:54
wave of ebrid search in the sense that they mix geometric
53:57
search with lexical search, right?
53:59
So what you're doing is to like geometric search because you
54:02
define your in the geometric search, you define your context
54:05
by distance among vectors. In the lexical search, you do a
54:10
more, more like statistical kind of things.
54:12
You count basically the statistical characteristics of
54:17
the words in the context. That is a pretty well good way
54:21
of establishing A context because the kind of words, the
54:25
the class of words, the families of words that you have in the in
54:29
a, in in a text is it's context like the definition, somehow the
54:33
definition of context. The problem is how you, you, you
54:37
measure these distances. So like, OK, you have like this,
54:40
this word appears 50 times in, in 1000 words, right?
54:47
Does it mean that that this word defines this context, right?
54:51
So you do OK, let's add some geometric search, some like
54:54
distance cosine L2 search to this.
54:58
And with these two things, you can tell us better which context
55:01
belongs that documents belongs to.
55:04
But then you go back to to the original problem of like running
55:10
vector search, geometric search on on that on that vectors you
55:14
build, right. So you go back that you lose all
55:17
the semantic semantical part that the features analysis
55:22
brings in. So yeah, that that's, that's
55:26
lexical search, but that's lexical analysis.
55:28
Yes, yes. Thorenzo, this has been great
55:31
man. I appreciate you coming on here.
55:34
Thanks a lot, Demetrius. Thanks a lot for what you do in
55:37
the community and it's great to have people that can work so
55:40
well, like building up communities and, and, and
55:44
products. So it has been great talking to
55:47
you.