Latest / Agentic Conversations (formally mlops.community) / The Variational Book // Yuri Plotkin // #253

Transcript

The Variational Book // Yuri Plotkin // #253 · Agentic Conversations (formally mlops.community)

0:00
My name is Yuri Plotkin. I'm the author of the
0:02
Variational book. You could find more at the
0:06
variationalbook.com or at the Variational on X.
0:10
And I like my coffee. I'm pretty simple.
0:12
Just black. Just black.
0:15
It doesn't even have to be good coffee, it just has to be black.
0:19
I drink all types of coffee. What is happening ML OPS
0:24
community, we are back for another podcast.
0:26
As usual, I am your host Demetrios.
0:28
And today, talking with Yuri, we got into the pedigree of models,
0:34
specifically all of these image generation models, and I really
0:38
appreciated how he broke down the lineage, opening my eyes to
0:44
the fact that there is a bit of standing on the shoulders of
0:50
giants. We knew it, but here he talks
0:53
about it very clearly, how one model inspired the next.
0:58
And it left us with being able to use text to video where we
1:04
are at today in this day and age.
1:07
So cool to see he's got a book out.
1:09
As he mentioned in the intro, go check it out if you want to
1:14
start getting into the architecture and understanding
1:20
of these models. He talks about how he was
1:25
inspired to write the book after just diving into reading so many
1:29
different papers and then seeing this thread, the common thread
1:33
they went through them and understanding the math and
1:36
understanding why things were the way they were.
1:40
A big caveat to this is that his book does not talk at all about
1:45
LLMS. He said there's enough conversation and literature out
1:49
there on those specific types of models, so he hit the other
1:54
ones. Hope you enjoy it.
1:55
And before we get in to our little conversation, I figured
2:01
I'd keep doing what I've been doing.
2:03
I'm going to start a habit of it and share with you one of my
2:06
favourite songs that I've been listening to these days.
2:09
I will say that this song 100% you got to watch on YouTube.
2:15
I'm a huge Spotify fan, I usually get all my music from
2:18
Spotify. But this is a live version of
2:21
the song called Hideout by Parcels and it is from their
2:27
2022 concert in Paris. Now let's have a listen.
2:34
Y'all can hear it That riff.
3:16
I don't know how you can listen to this and not start moving.
3:20
It's just too good. So there it is, folks.
3:36
Let's rock on in to the conversation.
3:40
Here we go. Right now.
3:50
Let's start with this though, I think that you've got an
3:55
interesting background. I want to get into some of it
3:58
because you were in biology, right?
4:02
And. Then.
4:03
You jumped into the AI scene. Can you talk to me about the
4:07
switch and what made you want to go from being a biologist to
4:14
generative AI ologist? Yeah, it's a great question.
4:18
People ask me, I think all the time, and I, I, I, I don't have
4:22
a clear answer, but I'll tell you, kind of as a kid, I always
4:26
loved biology. It was always something
4:28
fascinating about it. This is a personal kind of
4:33
story. When I was a kid, I used to love
4:35
ketchup and I haven't really shared it outside the family
4:40
circle, but I used to ask my grandparents if ketchup kills
4:45
germs and they would just lie to me to have me eat the food.
4:48
So yeah, I always had this like fascination or there's interest
4:52
in the, the biology side of things.
4:56
I, I really don't know where that comes from.
4:58
It's kind of like AI think it's just a little bit intrinsic, a
5:01
little bit curiosity. So that kind of manifested into
5:06
into me studying biomedical engineering for my undergraduate
5:10
and masters. And I loved it because it was a
5:13
good kind of hybrid between learning the biology and an
5:18
application and how it transfers to solving real world problems
5:22
from medical devices or nowadays it's a very computational.
5:27
So there's a lot of like problem solving analytics, as you could
5:32
imagine from the engineering side.
5:33
So I, I really enjoyed that because that was kind of
5:37
critical thinking aspect. At some point I transitioned to
5:41
the wet lab research and I really enjoyed that.
5:43
And yeah, there was kind of like a switch where I, I maybe got
5:48
back to some roots or got back to some math and started
5:51
studying computer science. And yeah, honestly, I haven't
5:55
looked back. It's been a very fascinating
5:58
topic. I love I, I think the, the
6:02
thread through kind of my trajectory in life, it's
6:05
learning. So the ability to learn and the
6:08
ability to develop new skills. I, I think in, in this context,
6:14
their heart skills, you know, the science, understanding the
6:17
science and, and yeah, there's just some something amazing
6:22
about I, I think the science is where, you know, it's like the
6:25
scientific method where you could test something and, you
6:27
know, it follows or, or some, some laws or, or some
6:32
observations or it's, it's very kind of systematic and I love
6:36
that reproducibility about the science.
6:38
So I think I'm digressing. But yeah, I, I, I think just
6:45
curiosity kind of led me to computer science.
6:48
And the more I started to learn, the more I kind of loved it and
6:54
the more it was just kind of, yeah, it took over, I suppose.
6:58
And, and, and the more you learn, the more you want to
7:02
know, the more you learn, the more you realise, you know, the
7:05
less you know. So I, I think it's just, it's
7:10
been kind of a natural evolution.
7:12
I, I, I think for me to get into the field.
7:15
So you have been writing a book recently, the Variational Book.
7:20
What is it and how did it come about?
7:24
So the variational book, it's about all things generative.
7:28
AIII discussed literally the last 10 years of generative AI
7:35
algorithms, not including LLMS. That's the one kind of elephant
7:40
in the room that I don't touch. But I, I, I basically derive and
7:44
explained logically all the steps in terms of how the
7:48
algorithms work, why they work a certain way, provide anecdotal
7:53
examples, more intuitive examples of, of, of the
7:58
rationale behind them. And it's a way initially it
8:03
started as a way for me to personally learn reading a lot
8:08
of papers, trying to disseminate a lot of the in between kind of
8:14
knowledge that was transferred in the papers.
8:17
And yeah, it kind of snowballed from a short kind of blog post
8:25
that I was going to write in terms of.
8:27
And then I just kind of kept writing and kept writing and
8:30
kept writing. So I'm pretty excited about the
8:33
book because it's very thorough and I think I provide a
8:38
different context and reference frame for describing these
8:44
algorithms and then also for the pieces that I include.
8:48
So if you can think, think of it as like a piece of a puzzle,
8:51
each publication has a small piece and sometimes it's hard to
8:57
fit those pieces together, how everything relates to each
8:59
other. A lot of these algorithms have
9:01
evolved over time and traditionally now you, you see
9:05
there's a lot of push on like text to image, text to video,
9:10
these diffusion models and these diffusion models actually have,
9:14
you could say have ancestral precedence and other earlier
9:18
models. So if we talked about, you know,
9:21
continuous normalising flows, you come on normalising flows,
9:25
time on score matching, you're talking about variational auto
9:28
encoders. These are all, I'd say, 10 years
9:32
old techniques that are still widely used and applicable and
9:36
very powerful methods, but yet they share similarities to these
9:40
newer methods. So I, I think the exciting part
9:43
about my book is I, I create that common thread that lets you
9:47
kind of in a stepwise fashion, understand things by starting
9:52
from Bayesian modelling to laden latent models to working your
9:57
way towards more generative methods and just what it's
10:02
about. I think it's a great resource
10:04
for any machine learning engineer, anyone in the field,
10:09
anyone who's an undergraduate or graduate student, because I
10:12
literally try to not leave any logical gaps.
10:20
So I tried to reduce the activation energy or the effort
10:24
required to learn it. It's still very technical, still
10:27
requires a lot of effort, but I think it's a very great kind of
10:32
procession of ideas that I'm able to explain and and provide.
10:37
Yeah. In terms of how it came about,
10:40
it's. Yeah.
10:41
It was started reading papers. It was more of a self
10:44
interested, I want to do, I want to learn myself.
10:47
It was a way for me to read papers, to try to understand the
10:52
papers. And I, I guess the saying goes,
10:54
you don't really understand it until you either teach it or you
10:57
write about it. So this was my way of gaining a
11:00
better understanding. And I think the byproduct has,
11:03
it's been several years, but the byproduct has been, has been, I
11:10
think a nice collection of, of, of writings that that fulfilled
11:16
in at least my initial interest. But also I think I'll provide a
11:21
lot of value to the wider audience or to to a lot of
11:24
people I think in universities and and in the field.
11:27
So it's almost like you have this family tree of the
11:30
different models. I really like how you put that
11:32
together. The thing that is a very strong
11:38
design decision on your part is not including LLMS in this book.
11:42
Why did you not do that? I didn't include LMS because
11:48
they're inherently a different, you could think about a
11:51
architecture from the machine learning side, although maybe
11:55
that's disputable at some level. But yeah, I, I focused on
12:00
Bayesian laden models and variational inference and that
12:06
kind of spilled over to a lot of these kind of newer techniques.
12:10
As I mentioned, diffusion models, score matching,
12:16
normalising flows, variational methods, a lot of more, how do
12:21
you compare 2 distributions or how do you train a model to fit
12:27
a distribution of data. So it's definitely it's, it's
12:33
definitely focused on more Bayesian kind of methodology.
12:38
Suffice to say, there's also a lot of other books that have
12:42
come out in the LLM space, both on the more theoretical side and
12:48
more the practical implementation of how you do
12:50
Laura and Rag and RLHF and so on and so forth.
12:54
So the spaces is is is is a little crowded.
12:59
So I thought there'd be a little bit more emphasis on more
13:02
traditional approaches that are still foundational.
13:05
And I think the theory, although it might be, albeit a little bit
13:09
different, still spills over to some of the modern techniques.
13:15
So there is something that in my eyes actually, no, you know what
13:24
I was going to mention. Did you see, Speaking of
13:28
diffusion models right now, I think last week or just a few
13:32
days ago there was a new open source, well, semi open source.
13:36
Because these days when it comes to models being open source, you
13:40
can't really believe any of that.
13:43
The semi open source model that came out, Flux, which is a
13:49
diffusion model. And it's super cool to see
13:52
because it has very high quality.
13:54
And it seems like just for me playing around with it for 15
13:58
minutes that it's easy to prompt and you can get some very high
14:02
quality images. So there's a lot of people that
14:06
are pushing the boundaries in that direction.
14:11
When you were reading about all these different models and
14:15
almost like the evolution of the models per SE, was there one
14:20
thing or a few different things that surprised you or stood out
14:24
to you? Specifically you mean in terms
14:26
of like their designs? You can take it, Yeah, take it.
14:29
Anyway, If there was anything or it was like, oh, this model
14:33
never actually got traction and whatever the in the story of the
14:37
models or the architectures or yeah, whatever the Speaking of
14:42
the designs, it could be that or on the math level or on The Who
14:48
created it, whatever jumped out at you and you were like, damn,
14:53
I didn't realise that. Yeah, I think, you know, I, I
14:57
think there's a common thread and if we, if we start where
15:01
let's say the, the initial conception of like variational
15:05
auto encoders, you encode some data into a latent space.
15:10
You and then you have this one step procedure where you sample
15:13
this latent space and you generate whatever you're trying
15:18
to generate your your synthetic data example.
15:21
I, I think what was interesting as you get into the weeds of the
15:25
evolution of the techniques is Gans again have a discriminator
15:32
and a generator or generator and discriminator again, the
15:36
generator again sampled in a one step process and you generate
15:41
your synthetic data sample. I'll bet the training algorithms
15:45
are a little bit different for, for those techniques and use
15:47
different types of objectives and approximations.
15:51
You could say what was interesting what so that was
15:54
when I started the deep dive into diffusion based models is,
16:00
is you have essentially you can think of it of a similar
16:04
analogous process, but it's just repeated many, many times for
16:08
the training and inference steps.
16:10
So for example, you start with a data sample and you just corrupt
16:16
it 1000 times and add noise at each iteration until you just
16:20
end up with some Gaussian noise. And the inference part is you
16:26
start without Gaussian noise and then you try to decorrupt it and
16:31
relearn what that synthetic or generate a synthetic example
16:35
that comes from the same data distribution as the original
16:38
training data. So you're relearning to generate
16:40
those images. The caveat there is these,
16:42
these, these models are extremely powerful and, and in
16:46
the in the fidelity and the quality of the samples.
16:50
But the bottlenecks are the inference where you have to do
16:53
1000 steps to generate how one example, and if you compare it
16:59
to let's say Gans, you could sample thousands of Gans in mere
17:04
minutes. And to do the same with the
17:06
fusion models. If we reference back to 2020,
17:10
the DDPM paper was a Seminole paper that was that, that that
17:17
would that would have taken hours because of the just
17:20
computational effort to do the inference.
17:22
So, and since that was really surprising because they, they
17:26
are very similar, but it's just the number of steps.
17:30
So the interesting part is the evolution of diffusion models.
17:33
The last few years have focused on the inference side, where
17:38
instead of 1000 steps, you could reduce it to 100 to 10.
17:42
Now they're trying to do one or two.
17:44
So I, I, I think that was very surprising and in, in the sense
17:49
that the techniques are very related, like you're sampling
17:52
the latent space in the very at high level abstracted sense.
17:57
But also in terms of just just in terms of like, I, I would say
18:06
maybe like in terms of, I was surprised in terms of how
18:09
they're very related in that sense.
18:11
And then they're just coming to more practical kind of
18:15
differences in terms of the inference side of things.
18:19
Yeah. So the nice piece about that is,
18:23
and I like that you brought up how we're starting to see an
18:27
evolution within diffusion models of cutting down those
18:31
steps because that is where the time consuming part comes in.
18:36
And I imagine a lot of us out there have played around with,
18:40
especially if you were playing around with Stable Diffusion in
18:43
2022 or whenever it came out, you would see the image kind of
18:47
form and then it would kind of get darker and darker and then
18:51
boom, it would be there. And it was like watching a
18:54
Polaroid. And those were those steps,
18:56
right? I think it was like every 15
18:58
steps or 20 steps. And you can put longer amounts
19:04
of, you could on the UI that you had, you could say how many
19:10
steps you wanted. And if you put too many steps it
19:12
would over bake sometimes. So it's funny how that is.
19:18
The thing in real life I've tried baking and I'm like.
19:23
Too many steps too goes back to that.
19:25
But the, the piece there is now how can we, it's almost like the
19:31
architects are trying to figure out a how to optimise it so it's
19:36
faster and B, how to make it so we can't really over bake it so
19:40
we can idiot proof it a little bit for people like myself to
19:43
not be able to come up with something and when we don't want
19:47
it. And one thing that I, I was
19:51
thinking about as you were talking about this, it just
19:53
reminded me of a huge takeaway that I got from reading Joe
19:59
Reese's book, The fundamentals of Data Engineering is how
20:04
useful it is for engineers to understand the tools and the
20:11
space that they play in and really understand what are the
20:14
pros and cons of each. And so as you were saying, well,
20:18
you know, you have gens that can be really fast, but they only
20:21
have one step and they don't have all of these cool like
20:24
generative properties as much as a diffusion model.
20:27
But a diffusion model is probably going to be a little
20:30
bit slower. So when you're architecting what
20:33
you need to architect, you can make your decisions based on
20:38
that because you can say, do we really need it to go through all
20:43
these steps or do we have the ability to wait one second or a
20:48
half a second? If not, and we just need to
20:50
classify a picture, then maybe it's better to use a different
20:54
type of architecture. We don't need this diffusion
20:56
architecture. And Joe was talking about it a
21:00
bunch in the book on understanding your compression
21:05
algorithms or understanding what databases are powerful in what
21:10
ways. And so it just reminded me
21:13
exactly of that. Yeah, I, I think, I think those
21:17
are good points. I would also add, I think it's
21:21
very exciting nowadays, at least from my perspective and vantage
21:25
point is a lot of this compression on the inference
21:29
side, I think has direct implications on the video
21:34
generation side and, and a lot of well, so, so I think the
21:41
current range is maybe around 5 minutes.
21:45
I, I, I don't know what each company's secret sauces, but
21:48
the, the, the, the time frame that you could generate a video
21:53
is still limited to certain degree.
21:55
But I think the exciting part is, is that those two things go
21:59
hand in hand. You reduce the inference time
22:01
and let's say for video generation, you can create more
22:04
longer videos. And from what I'm seeing, I
22:08
think there is a huge like there's hundreds of companies
22:12
I've talked to people before that are spawning trying to do
22:15
this video generation stuff. So it's, I think it's very still
22:20
early stages, but it's it's pretty significant in terms of
22:25
its applicability and in potential impacts on a lot of
22:29
industries. And maybe I'm in LA.
22:31
So I think a lot of entertainment industry is
22:34
cautiously scared, but very interesting.
22:39
If I'm understanding you correctly, it's because of the
22:43
ability for us to go down to one or two step generations with
22:48
these diffusion models. It's much easier to create
22:53
videos from these one or two step generations and then put
22:58
basically you start to create 24 frames and then you have the 24
23:03
frame per second video and you can add those up.
23:09
Is it different when it comes to 'cause I had always thought the
23:15
models that were being used to output the videos were
23:20
different, like a whole different architecture than the
23:25
models that were being used to output the photos?
23:31
I'd have to review it just so I don't, but for video generation,
23:35
it's usually units that are used and they they have some way of
23:41
incorporating the time domain elements of the.
23:44
The issue with video generation is, is you how you stitch
23:48
certain frames together or different scenes.
23:51
Yeah, for diffusion models it could vary.
23:57
It could be units as well, but I have to like take a look at.
24:01
I think it depends on the exact implementation.
24:05
So it goes back to what you were saying on how there is this
24:11
lineage or there's almost like a common thread that is going
24:16
through all of these models and you're really standing on the
24:19
shoulder of giants when you're doing anything with diffusion
24:23
models or units are coming from the the optimization and the
24:29
efficiency gains that you get with diffusion models also help
24:32
later on for the units and vice versa, I would imagine.
24:38
Yeah, I mean, I, I think, I think the from my kind of from,
24:43
from, from my just literature review and, and just the process
24:49
of writing the variation of book.
24:51
You definitely see a lot of I I would say I don't know what the
24:56
right quantity is or how to quantify, but you definitely see
24:59
a lot of spillover or, or translation and and and maybe
25:06
I'm not quite sure what the right verbage is, but you
25:08
definitely do see a lot of, I don't know, borrowing or
25:12
learnings throughout the literature.
25:14
Yeah. And like, maybe the best way to
25:17
phrase is like learnings from how earlier publications work
25:21
and, and stuff like that. I, in terms of some of the, in
25:29
terms of my writing for the book, I tried to stay away from
25:31
like specific architectural changes that might have been
25:35
published because those are very specific.
25:39
And I tried to focus on the core algorithms at hand.
25:42
So maybe I'm not the best person like like for example, in in
25:50
Gant training, there's or Gant training or any neural
25:55
architecture or any model training you could do specific,
25:58
let's say architectural changes, let's say you know, for for
26:03
Gans, they would do how do you incorporate style into the Gant
26:08
generation? So how do you precondition the
26:11
Gantt to generate certain types of styles or they would modify
26:15
the architectures that might increase, let's say fidelity
26:21
somehow. I tried to stay away from
26:25
architecture specific considerations because I think
26:28
those are less, those are more, those are more abundant.
26:33
And what I tried to focus on more is that is the core
26:37
algorithm in the logic used. And of course, if there's like
26:40
Seminole papers, for example, again, if we stick with Gans,
26:45
there's a truncation trick or style based generators or
26:48
Washestein Gans that improved either the objectives or or did
26:54
something in terms of the sampling.
26:56
For example, the truncation trick, not sampling from the the
27:00
ends of this distribution to prevent searching maybe rare
27:07
events or rare generation. So yeah, I, I, I, I, I tried to
27:14
stay away from architectural changes in in the Fort.
27:16
So I'm not maybe the best person to like comment on on like what
27:23
how things are have evolved or or how things have maybe yeah,
27:32
I'm not sure like like. No, I understand that talking
27:36
about all the different pieces of the ways that one is using
27:40
the other, Yeah, I think was the original.
27:42
Thing it's just, it's just like you could for example, if we
27:47
talked about LLMS, the original transformer paper had one layer
27:52
normalisation after I, I think the, the transformer block or
27:58
something like that. And then in hindsight they added
28:01
one before as well. So like those architectural
28:03
changes can have significant impact on the actual performance
28:08
of the method and algorithm, but I've stayed away from those a
28:14
little bit and try to make it more focused on the algorithms
28:19
themselves. And then and then of course once
28:24
one gets one, once someone gets assimilated or comfortable with
28:29
the algorithm approaches, then you can pull up archive or go do
28:33
a Google Scholar search to see maybe what the latest and
28:36
greatest approaches are. Yeah, so it's almost like
28:41
there's remixes of each algorithm to try and figure out
28:45
the best ways of optimising it. And.
28:48
Seeing how can it go from this diffusion model that was
28:55
originally taking thousands of steps to get to the image
28:58
generation to now it just takes one step and that's because of
29:02
some architectural changes that have happened.
29:06
Sure. Yeah, I mean some of the I can't
29:09
remember off the top of my head, but for example, there's
29:12
certain, there are certain public, there are certain
29:15
publications like the initial diffusion model started with
29:18
DDPM. That was a a big one that kind
29:21
of was like the breakthrough moment for those types of
29:23
modelings. Even though Diffusion itself was
29:27
published the idea like circa 2015, I think, but I'm going to,
29:34
I'm not going to say his name right, but.
29:38
That's always fun, man, pronouncing these names.
29:40
So the papers are a great time. Yeah, you want to.
29:43
Get home. I should, I, I should, I should
29:48
be able to figure out, Yeah, but like, yeah, but I guess there, I
29:58
guess it's interesting. The literature is filled with a
30:00
lot of these methods. To your point, there's
30:02
architectural changes as more algorithmic changes.
30:05
So from the diffusion side, architecture changes were in
30:09
terms of could include some type of distillation, whereas like
30:14
more algorithmic changes. There was a paper called DDIM
30:19
which is diffusion denoising implicit models where they
30:24
focused on changing the Markovian nature or today they
30:30
basically made the the inference process non Markovian and and
30:35
and in a high level way that allows you to reduce the number
30:40
of inference steps that are are required.
30:43
Whereas in the original diffusion papers, here it's very
30:46
conditional on the previous time step that which would we'd make
30:50
a very sequential process. Yeah, first of all, you said a
30:56
word that went right over my head.
30:57
What is Markovian? Markovian.
30:59
So and from a probabilistic sense like you, you can have a
31:04
first order Markov model where the probability of X of T is
31:10
given based on is conditioned on probability of XT -1, so the
31:14
time step before. So in this sense it's a first
31:18
order Markovian model because you're only conditioning on one
31:23
prior time step and the only thing that really, really
31:26
matters to predict the the future is the previous time
31:31
step. And you see Markovian types of
31:35
modelling is is very predominant in a lot of RL literature and
31:39
assumptions and a lot of machine learning basically simplifies
31:45
these types of sequential, sequential decision processes
31:50
using maybe a some mark type of mark of assumption, although
31:53
that's that's changed quite a bit I think.
31:57
How so? I mean you could think about if
32:01
we do, if we focus on diffusion models.
32:04
Let's say you have the diffuse, you have diffusion denoising
32:09
implicit models. So now the you're assuming a non
32:13
Markovian inference structure. So and without getting into the
32:21
math of it, you're you're basically not assuming that your
32:26
inference is dependent on just the previous time step, but it's
32:28
also dependent on the original for for these implicit models,
32:32
it's dependent on the original data sample as well.
32:35
You bake that into the probability distribution that
32:38
you're trying to model. So with all these diffusion
32:40
models is you're you're trying to learn at each sequential step
32:45
what the distribution looks like.
32:48
And if you take 1000 forward steps and you know what the
32:51
distribution looks like at each time step, then you could easily
32:56
take the reverse process back because you're just sampling
32:58
from a distribution. But in non Markovian diffusion
33:06
modelling, implicit models, for example, you're not, you're not
33:09
assuming a just one conditional what you're not, you're not
33:16
assuming Markovian past or Markovian model.
33:22
I'd have to maybe go through some of the math to kind of
33:24
maybe explain a little bit better.
33:26
But for example, also I, I, I think maybe I could be wrong,
33:32
maybe there's. But if you look at like language
33:34
modelling in terms of like attention mechanisms, right?
33:37
What attention does is initially you would have 6 sequential
33:41
generation, right? It's you're sampling from some
33:44
multinomial distribution and you're getting your next
33:47
character, your next token or whatever it is, but they're
33:50
biassing it with attention these days.
33:52
Now you have multiple, let's say hidden states, and then
33:55
attention is all you need. When the Transformers came out,
33:57
that changed what attention, how attention was defined.
34:02
So therefore, this is an interesting question.
34:08
And I don't know, I don't know how I, I don't really know the
34:13
theory behind it in terms of, let's say, how attention,
34:16
whether it's relatable to a first order Markov process or a
34:20
second or whatever it might be. There might, there's, there's
34:22
definitely some probably deep connections there.
34:25
And I would, I, I would be cautious and speaking on it on a
34:30
personal level. But yeah, there's, there's,
34:32
there's definitely some, some connections and that's kind of
34:37
focused on the interpretability of these models and how they
34:40
actually work, right. So I think probably in the next
34:44
few years we'll, we'll, we'll have more I think litter or
34:48
literature or publications on that.
34:51
So. Yeah, or one of the listeners
34:53
can just tell us in the comments.
34:56
Let us know. You said everything.
35:00
Well, hopefully they say you said pretty much everything
35:02
correctly, but. Of course, no, the if I'm
35:08
understanding it correctly and what I really like about that
35:12
and I appreciate you breaking down what Markovian is for me,
35:15
Markovian basically means just use the last step as your
35:19
reference to create the next step.
35:21
And the non Markovian is try and use the whole picture and what
35:27
we trained it, whether if it's the diffusion model, it's saying
35:32
there was 1000 steps from when we had a nice picture to when we
35:36
had perfect noise. Basically use that whole 1000
35:41
step journey as your reference. Not, not necessarily, not quite,
35:48
but just just to say that in a Markovian first order Markov
35:55
model, you're as you, as you mentioned, you're just assuming
36:00
that your future depends only on the previous state.
36:04
And that could be for diffusion models, reinforcement learning
36:07
or whatever might be basically in a denoising diffusion
36:13
probabilistic model, what you basically have is you have a
36:17
conditional dot distribution in the forward training pass.
36:21
You have your your surrogate QX of T conditioned on X of t -, 1.
36:28
So imagine you start from t = 0. You take a step, you're only
36:34
you're, you only care about your previous distribution.
36:38
What the non Markovian denoising diffusion implicit models do is
36:43
now they're learning a different distribution during inference,
36:48
and what they're learning is QX of T condition on XT -1 the
36:54
previous time step, just as before.
36:56
But they're also conditioning it in on the original image X of 0.
37:01
So now this. Is a different.
37:04
Yeah. So now you're conditioning on on
37:06
two different, you could say variables instead of one and the
37:11
distribution you learn is non is non Markovian in that sense.
37:16
Oh, I. See what, what the diffusion and
37:20
denoising implicit models show is that even non Markovian
37:23
distribute distributions can have the same marginals as the
37:30
original denoising diffusion probabilistic model did.
37:35
So the beauty of it is what they showed.
37:37
And, and I, I, I think a lot of this gets lost in translation
37:41
just by speaking about the mathematics.
37:43
But from a, a layman's term, I think what's pretty amazing
37:47
about these techniques is training nothing, nothing
37:51
changes. And in a lot of these models,
37:54
you don't really care about, you don't care about the training
37:57
portion of it because training if it takes 1000 runs or 1000
38:02
steps to do the to do the training, to do, to do the
38:06
noising steps, that's OK. But you care about the
38:09
inference, the reverse because you care about the generation
38:12
and you want that to be quick. So if you have again to our
38:17
discussion, 1000 steps, it's going to be very slow.
38:19
But these implicit models, because they form non Markovian
38:23
distributions, what they show is you could train the model the
38:28
same way. But when you do the inference
38:30
side, because those non Markovian in essence have this
38:35
is all just kind of like verbiage.
38:37
I have to kind of I would have to maybe this is like a selling
38:41
point like, you know, buy my book or subscribe because I go
38:45
over all the details of the math.
38:47
But in essence what that means is you're reducing your
38:49
inference time using these implicit non Markovian
38:54
distributions. And that goes into the fact that
38:58
you can have, you can generate the same marginals that your
39:01
distributions that you're interested for performing that
39:04
inference. Dude, So I'm going to start
39:08
using Markovian as a insult to people who are living in the
39:14
past. You know, like people who can't
39:17
let the past go. I'm going to say that's so
39:19
Markovian of you. They're just.
39:23
Training. They're living their life on in
39:25
the past and not not able to get over that last step.
39:31
Yeah. Yeah, you would.
39:33
You would. That would kind of make sense, I
39:38
think, from a nerd's perspective.
39:41
Yeah, I, I might just be the one that looks like an idiot by
39:44
saying that nobody's going to understand me.
39:46
I'm just going to be like, what the hell are you talking about,
39:49
man? And then I have to say, well,
39:51
you know, Markovian is actually when it's T -, 1, that's what
39:57
the Alps. And then I look like an idiot
40:00
because I can't actually explain why the what or what Markovian
40:05
is, but it. Does I mean you could get more
40:09
complicated. You could have second order or
40:11
third order. It's a little tricky because
40:12
then you can condition on two. So you could still have a
40:16
Markovian process, but it's not a first order side B.
40:19
We we could work on the on the. On the on the, on the burn,
40:25
that's. It you have, you'll have some
40:27
point and A and a computer science or two computer science
40:30
people working on on making sure that the delivery is good, but
40:34
also that that's factually. Yeah, factually correct.
40:38
And it actually makes sense. The analogy holds up.
40:41
So you know what else I was thinking?
40:43
Man, I would love to see what you were talking about, all
40:49
these different remixes of the models and specifically the
40:53
model architectures in like a lineage, family tree type of a
41:00
graph or visualisation to see, all right, here's all the Gans,
41:03
Here's all the different ways that we've tried to tweak Gans,
41:08
and then here's all the diffusion models or whatever it
41:11
may be. And you can climb up the tree
41:13
and see how diffusion models and Gans came from the same mother
41:18
or father or whatever it may be. Yeah, I think that would that
41:25
would be pretty powerful. That would require another some
41:29
some more effort that. No, I'm not expecting you to
41:32
put. That to it, but potentially I'll
41:34
have it to you on Monday. Yeah, totally.
41:38
I'll be expecting that before this comes out.
41:41
I imagine somebody that loves data visualisation has thought
41:46
about that and put it together potentially and showed some way
41:52
that all of this is linked. Because that's a huge takeaway
41:56
from what you're saying. And, and really your book is
41:58
that you can look to the past for inspiration and see how it
42:05
has given us the future or given us at least the present.
42:12
Yeah, I mean, I, I think that's definitely the case.
42:14
I I, I think actually to your point, what I try to embody in
42:20
this book is it's a little bit biassed because I cherry pick
42:24
for methods that worked and were well sited and more, you know,
42:29
rose to the top of the pile. But I actually essentially try
42:34
to do as you described as go through the evolution from
42:39
beginnings of Bayesian models and, and just Bayesian model
42:43
selection or something more old school towards more new
42:49
flavours, let's say diffusion models or, or text to video and
42:54
stuff like that. But yeah, I think it's, it's, I
43:00
think that's, I think it's a very good point.
43:02
And I, I think some of the things I try to do actually lend
43:06
itself towards that without actually having to read all the
43:10
landscape. I think the beauty every writer
43:14
on a personal level there that you know, they, I'm always
43:17
curious now, since I've gone into this journey, they always
43:20
talk about like some of the writers specific considerations,
43:24
like when do you know, when you finished, how do you stop?
43:28
And unfortunately, like the computer science literature is
43:32
so vast and there's so many great papers and there's so many
43:35
great topics. And when you try to cover a lot,
43:38
that basically means I just have to keep writing for another.
43:42
I don't know how long, but yeah. I mean, it's, it's, it's, it's,
43:51
it's, it's, yeah. It's, it's interesting.
43:54
It's interesting point, I'd say. And yeah.
43:59
You have the benefit of time too, now that you've been able
44:04
to see all of these different models that bubbled up and have
44:10
withstood the test of time. I think also is a huge factor.
44:16
It would probably be really hard to do what you're trying to do
44:20
now for the most cutting edge models because the all of these
44:28
variants are trying to take hold and we don't know which ones are
44:31
going to be the ones that will stand the test of time.
44:34
Yeah, yeah, there's definitely, I, I, there's definitely some
44:38
selective bias there in terms of how or, or some benefits I have
44:44
like withstood the test of time. I, I, I'd suppose for, for
44:50
allowing me to even like write this book because then, yeah,
44:54
it's hard to predict the future. I think maybe only a few voices
44:58
out there might have some reasonable yeah, competency to
45:03
to to try to act Nostradamus. Well, it does make a lot of
45:07
sense. You named it the variational
45:09
book and you're really looking at different variants of this
45:14
original model. Is that how where the
45:18
inspiration came from? Pretty much, yeah.
45:21
I initially focused on 1st. I'd say I'm I was, I was very
45:25
excited that the URL wasn't taken when I when I purchased
45:30
and and for anyone who wants to remove the the and just do
45:34
variational book and do a mimic copy, I I got that one too, so
45:38
they. Can't do it.
45:41
They can't do it. You got to, you got to be a
45:43
little bit more creative, but but yeah, yeah, it, it started
45:48
with variational inference and again, kind of variational
45:51
techniques, how you have some posterior distribution, some
45:56
data distribution that is theoretically unknown and how
46:01
you could model that distribution through a sample of
46:03
data and build some surrogate approximation to it.
46:07
So those techniques that I think I've been foundational for just
46:12
computer science in general, but yeah, they've, it's been pretty
46:16
cool because it starts with from the Bayesian side, just like I,
46:21
I start talking about if I go through my chapters like a base
46:26
theorem, which is a Seminole theory.
46:29
That is, I think it's easy to memorise, but it's very hard to
46:33
conceptualise what it actually is doing and what it actually
46:37
means. But once you conceptualise it, I
46:39
think it's extremely powerful because again, I think no matter
46:43
what the model is, I bet there's some interpretation that
46:47
incorporates some type of Bayes theorem.
46:48
But I go. Back conceptualise it.
46:54
So I mean, it's like you could memorise from a latent space
46:58
model, you could conceptualise it and, and this is some of the
47:01
things I talk about to help bring some intuition about it.
47:06
But from a, a latent space model, you always have variables
47:10
that you observe, which are your XS and then you have your
47:14
latents, which are things that you don't observe.
47:17
And you could think of both your latents and your and your
47:21
observed XS as forming some very complicated dependency graph
47:25
with connections of some which you might observe and some of
47:28
which you might not observe. And it's very, the goal is to
47:34
learn that joint structure of your probability of your X&Z,
47:41
your joint distribution. And, and, and when anyone said
47:44
it's a very interesting, actually it's a very interesting
47:46
question because a lot of people ask you to explain what
47:49
generative means. And I always go to like Bayes,
47:54
like, you know, joint distribution.
47:56
But if you don't know what that is, then it's hard to kind of
47:59
conceptualise what is generative.
48:01
Like you could generate stuff. I I think it's in the name but.
48:05
So that makes me think because you have these strong
48:09
dependencies with the the things you can see and the things you
48:13
can't see. It's almost like if you want to
48:17
find a certain part of latent space, sometimes you have
48:23
different things, like I'm thinking specifically with LLMS,
48:28
and sometimes you want to make the LLM perform better or do
48:33
something be more rigorous. And so you can say things like
48:37
think very hard or think through this clearly or things like
48:42
that. And it makes me wonder if that's
48:46
because there's these dependencies, so you almost zoom
48:50
to that part of latent space by using these trigger words.
48:57
Yeah, Yeah. I mean, I think that's
48:59
essentially that. I think that a lot of
49:01
interesting research, a lot of the big companies are looking
49:03
into that like their interpretability and that's the
49:06
whole thing. It's like prompt engineering,
49:09
like, yeah, you could do it, but explaining why it might give you
49:13
a better answer. I think that's the interesting
49:16
part. And to your point, it's you're,
49:19
you're you're potentially sampling from a different part
49:22
of that joint distribution or somehow depending on how you
49:25
would conceptualise it. But yeah, so it's a very
49:29
interesting topic. And a lot of the methods again,
49:31
stem from just these basic core ideas and from the variational
49:37
side, I think to your original question.
49:38
So I talked about model selection, I talked about
49:42
exponential families. They're very important in just
49:45
modelling in general and, and machine learning.
49:48
And then you could talk about, well, if you build some
49:50
approximate distribution with certain assumptions will be
49:53
called a surrogate. How do you, how do you measure
49:57
the difference between your surrogate and and your, and your
50:02
true data distribution? And, and essentially what that
50:04
is, is everyone post about, you know, the KL divergent is one
50:08
metric how you could do that and you could define your objective
50:10
that way. And then basically, you know, we
50:14
snowball into more complicated topics of how you model the
50:17
surrogate distribution, what assumptions you can make in the,
50:20
and a lot of these generative models, the, the, the underlying
50:25
theme there is, is that they make, they're able to model a
50:28
more complicated surrogate distribution.
50:32
And because it's more complicated, you can incorporate
50:34
a lot more of these dependencies that you might miss with like a
50:38
simple distribution. It analogously, it's like, and
50:44
not not to hate on linear regression, but linear
50:46
regression is a great, very powerful model, but has many
50:50
assumptions in terms of linearity baked into it.
50:54
If you want something more complicated, then you could do a
50:57
function approximation using the neural network and it can and,
51:01
and there's pros and cons to that.
51:03
So kind of similar in trajectory there where you can build a
51:07
simple surrogate and distribution, but then you can
51:09
work your way towards more complicated stuff.
51:12
And this is essentially what a lot of these generative methods
51:16
have done over the last how many number of years.
51:20
Yeah, not only that, it just goes back to the idea of
51:23
understanding what you're working with so you can know
51:25
what the benefits are or the the strengths and weaknesses are of
51:29
each model and how you can as AI can't tell you how many times
51:35
I've heard the question, how much machine learning do I need
51:38
to know to be an ML engineer? Or I'm coming from the data
51:42
engineering space and I want to get into ML.
51:45
What should I do? How should I go and learn the
51:48
depths of this? Or do I just need to understand
51:52
what people are saying or what they mean when they talk about
51:56
the different model architectures?
51:58
And it kind of reminds me of again, of that idea of the more
52:02
that you can understand on these model architectures, the better
52:05
you're going to be because you can suggest different
52:11
architectures and different models for the different use
52:14
cases and your needs. Yeah, 100% a shameless plug.
52:19
I wasn't saying go read the variational book to learn more
52:22
and more. I think theory that transcends a
52:24
lot of ML. But yeah, I think I think you're
52:28
that's like the like the hammer right on the nail.
52:34
In order to be able to make changes or proposed changes, you
52:37
have to understand, you know, it's very easy to come up with
52:40
an objective and say, you know, code it or seed and code and
52:44
say, well, OK, this is what we use.
52:46
But the question is, is why you're using it?
52:48
What are the implications? And a lot of I, I think that's
52:51
where some of the kind of like boundaries between maybe ML
52:56
engineering and, and, and theory and ML science kind of comes,
53:00
comes into play. And I think both are very
53:02
crucially important. So it's I, I think, yeah, I
53:07
think it's good to have both. Yeah, it's fascinating to me.
53:11
The more that you think about it, the more that you have to be
53:15
really working through so many different levels when it comes
53:20
to AI in industry. Because you not only want to
53:26
think about the infrastructure side of things and how to
53:29
optimise on that level, but also optimising the model and making
53:33
sure that you're choosing the right model for the job.
53:35
Are you choosing the right databases for the job?
53:37
Are you choosing the right data for the job?
53:39
All of that, but then even going up a level and recognising what
53:44
are we really even trying to accomplish here.
53:47
When I get tasked with making our user retention better and I
53:55
decide that I want to use machine learning for that, maybe
53:59
it's I, I say, all right, well, if we have a better recommender
54:03
system, then we can have better user retention.
54:06
You have to be able to think through the business side of
54:09
things all the way down to the technical side and the, the data
54:13
side and the, the machine learning side.
54:16
So you have you're playing at so many different layers.
54:21
I think, I think with some humility, it's always, you know,
54:23
we're always, we could always do personally better with all
54:26
those. And yeah, I think that's the
54:31
exciting part because some of that kind of manifests
54:35
differently as time evolves. But also that's something to
54:39
keep striving for and learning. And I think we could like on a
54:43
personal level, there's always way more to learn and way more
54:46
to understand. And, and yeah, every time I see,
54:53
see really cool new papers that I'm like, well, oh, I should
54:57
maybe incorporate that. And I'm like a one, on the one
54:59
hand, it's a dopamine rush 'cause you're like, oh, it's a
55:01
great new idea that I could learn, right?
55:04
And understand. And you kind of get into the
55:06
heads of how people are thinking about these things too, right.
55:09
On the, on the on the other side, it's like, oh, it's a lot
55:13
more work. Yeah, you have to balance it and
55:17
decide. Well, this has been awesome,
55:19
man. I appreciate you, Yuri.
55:21
Yeah, this is great.