Latest / Agentic Conversations (formally mlops.community) / PyTorch for Control Systems and Decision Making // Vincent Moens // #276

Transcript

PyTorch for Control Systems and Decision Making // Vincent Moens // #276 · Agentic Conversations (formally mlops.community)

0:00
Vincent Mines research engineer at Metab, and I usually take a
0:07
cappuccino when I get to work and a little bit later I take a
0:10
flat white. Welcome back to the Envelopes
0:13
Community podcast. I am your host Dimitrios, and
0:17
today we are talking with one of the people that makes Pie Torch
0:22
tick. This was a conversation that
0:26
went so deep into the compiler and really it was tips and
0:31
tricks for people who are using Pie Torch and they want to be
0:35
able to optimise their Pie Torch use to the Max.
0:39
I'm hoping that you are able to walk away with at least 9 tricks
0:45
that make you a better Pie Torch user.
0:49
I don't want to spoil any of this conversation because it was
0:53
all so good and I will not do it justice.
0:55
So let's get right into it. And for our recommendation of
1:01
the day, I'm bringing some classical music into your life.
1:09
Debussy, Claude Debussy. Just the epitome of smooth.
1:22
I'll see you all on the other side.
1:24
As always, if you liked this episode, share it with one
1:28
friend. I think I saw somewhere on
2:18
LinkedIn that you shared. Oh, here's a few ways that I've
2:22
seen could be better when people are dealing with Pie Torch.
2:26
Or here's a few tricks that you might not have thought about or
2:30
easy wins. I can't remember exactly how you
2:32
phrased it, but it was just like, you might want to do this
2:36
next time you're messing around in Pie Torch and you probably
2:40
remember what you posted. I don't remember because it's
2:42
been over six months, right? But tell us what that was.
2:46
Yeah, so it's actually funny because we we're gathering
2:51
together with the the folks in Pythorch had this outside in New
2:55
York. And when chatting with them, we
2:57
kind of realised that we had very different ideas of what was
3:02
the most efficient basically to take a tensor from RAM and send
3:05
it to CUDA. And if you look at the Pythorch
3:08
doc, we have this thing that we basically tell people, Oh, well,
3:11
if the tensor is in pin memory and you call dots to CUDA with
3:16
notebook in course 2, your transfer will be asynchronous
3:20
with respect to the host, which means that it's basically going
3:23
to be faster, you know, because you can send more than one
3:25
tensor at a time from CPU to Yeah.
3:29
So OK, that sounds good. Let's try to profile that.
3:33
You know, let's take a tensor like actually 2/3 or more
3:37
tensors and call dots, pin memory dots to Q dot with not
3:41
booking course true and compare that with just the bare tensor
3:45
to Q dot. You know, and you might be
3:47
surprised that the simple tensor dot TE CUDA or the list of
3:52
tensor dot TE CUDA might actually be faster than putting
3:56
your tensor in pin memory and then sending it to CUDA.
4:00
And we, we digged a little bit into that, but it wasn't like,
4:03
oh, well, that seems a little bit strange.
4:05
And so we looked at the, the documentation from our friends
4:09
at NVIDIA and all these kind of things.
4:11
And so we came up with this tutorial about proper usage of
4:15
PIN memory and what PIN memory actually is.
4:17
So PIN memory is a reserved part of your RAM where you're going
4:22
to put things, you know, on hold for a little while before you
4:24
send them to queue that you reserve that space and your
4:27
queue, that device knows that it has to gather the data from
4:30
there to to take it on on device.
4:34
The thing is copying your data from RAM to these special place
4:39
that is pin memory doesn't come for free.
4:40
It's a, it's a copy of the data. And any time you're sending data
4:44
from RAM to CUDA, it's going to transition through pin memory
4:49
whether you want it or not. So if you go dot pin memory,
4:51
you're doing it yourself. If you don't call pin memory,
4:54
it's going to happen under the hood like by either chunks of
4:57
data or things like that, but it's going to happen anyway.
5:00
So The thing is, if you call tensor dot pin memory, you're
5:03
doing something explicitly that would be done otherwise
5:06
implicitly. And so The thing is that the the
5:10
dog was the dog was a little bit misleading there.
5:13
So basically the definition of over engineering.
5:17
Yeah, exactly. And then and then I was chatting
5:21
with my colleagues and I said, oh, well, that's a surprising
5:23
fact. And some of them told me, well,
5:25
you know, we should not worry about that too much because
5:27
that's already pretty advanced. So we went on GitHub and we did
5:30
the experiment, you know, of looking at tensor dots, pin
5:33
memory dots to something. And we had a sort of results of
5:38
people doing that because the kind of kind of says you should
5:43
without explicitly because if you read between the lines, it's
5:45
sort of like this is a proper way of doing it, but you might
5:48
read it that way. And it's kind of that this
5:51
common belief that, yeah, it's a good thing to do it.
5:55
And and many people don't benchmark that obviously,
5:57
because we kind of told them to do that.
5:59
So we're like, OK, let's try to, you know, correct things a
6:02
little bit and write a proper tutorial where we experiment
6:06
things and and we try to give a consistent message.
6:09
So I invite you to to to check the tutorial.
6:11
It's full of infos on and, and interesting facts about that.
6:15
But the TLDR here is basically benchmark what you're doing
6:21
might be working for you for some reason.
6:24
And if it doesn't, so most of the time if you're sending from
6:27
CPU to Q dot just doing dots to Q dot with note booking was
6:31
true. We're already give you a pretty
6:34
decent amount of speed up that is sufficient, you know to to go
6:39
away with that get away with. So yeah, that's anything.
6:42
Have you found any other areas where you didn't necessarily
6:46
mean to direct people that way? Like here in the docks?
6:51
It was just a little ambiguous and so people took it for
6:55
granted and then felt like they should be doing it and where.
7:00
Yeah. So another example that comes to
7:02
mind is modifying or copying tensors in place.
7:08
It's something that people do all the time, like, you know,
7:13
add underscore something or, you know, a radio module with in
7:19
place supposed to you see that all of the time.
7:22
And The thing is that the gain in terms of performance is
7:25
marginal, if anything, and it's a pain to deal with that in if
7:31
you're writing a compiler, because you know, that's in by
7:34
torch right now, very heavy focused on Torch's compile and
7:37
compiling things where you have a single storage that happens to
7:40
pull into different things isn't very easy to deal with.
7:45
And so the official guideline is do not do things in pace, you
7:48
know, like it's probably not going to be faster.
7:50
And if you benchmark it, it's very likely that is not going to
7:53
be faster. And even if it is from the, if
7:57
you try to compile that thing, it's going to be a little bit
8:01
messy on our end. And and therefore also probably
8:04
on your end, you know, because a compiled graph might not be as
8:07
efficient as, as it could. So I think the compile guys are
8:10
super skilled and so they, they dealt with that that a long time
8:13
ago. But the official guideline is
8:15
basically trying not to do things in place, you know, it
8:18
might break things. It's not going to be faster.
8:21
Yeah, where folks are looking for that extra little lift,
8:26
that's probably not the place to be looking, yeah.
8:28
Exactly, Yeah. So where is the place to be
8:31
looking? Yeah, so now I'm going to like
8:37
put my hat on off the RL guy because I'm working on torture,
8:41
you know, the reinforcer training library.
8:43
And one thing that is very specific about RL is that we're
8:47
dealing with models that are that are rather tiny and that
8:51
are called very, very frequently.
8:53
So that basically means that we're heavily CPU overhead bound
8:57
in the sense that very simple things, you know, like getting
9:00
attributes in your module and things like that might take
9:03
crazy amount of time compared to the mathematical operations that
9:06
you're doing, like metrics and multiplications and things like
9:08
that. And in this case, sometimes when
9:13
I talk with people who are doing it and they don't believe me,
9:16
but simple things like calling module dot eval takes a crazy
9:20
amount of time because you're going to iterate over all those
9:23
sub modules that you have and and like set an attribute and
9:26
things like that. And and then that module is
9:27
written in such a way that getting attribute is already
9:31
kind of long. And so sometimes you see those
9:34
training loops in RL where people, they're doing module dot
9:37
train and they do something and they do module dot eval and they
9:40
do something else and then they redo module dot train and all of
9:43
these things introduced sometimes in overhead.
9:45
That's just like about 1015% of your own time.
9:50
And if you probably like, you're like, Oh my gosh, what am I
9:52
doing? Like I'm training my model for
9:54
basically 2 days. And in those two days, I have
9:57
like 4 hours that are spent just going modular, training modular.
10:02
And that seems totally crazy. And those like, if you're not
10:05
compiling, if we are in the eager side of things, a very
10:09
easy way to to throw that away is basically to create 2 copies
10:15
of your module. So we're going to create one
10:17
copy of your module with regular parameters, and then you're
10:20
going to associate a second copy.
10:21
There's a way to do that with what we call meta tensors, which
10:24
is, which are tensors that are not on any device, you know,
10:27
like they're kind of fake tensors.
10:29
Or you can, if your model is very small, you can just create
10:32
tensors that you are going to throw away later on.
10:36
So the next thing that you're going to do is that you're going
10:37
to take the parameters from the first module and you're going to
10:40
copy those parameters, you know, on the second module.
10:42
Now you'll have two module that two modules that points to the
10:45
same data storages and all the same parameters.
10:48
If you modify the parameters on the 1st module, then modify on
10:51
the 2nd module 2. And The thing is that now you
10:54
have those two modules, but they're different and so you can
10:57
call dot eval on the second one. And now you have one module that
10:59
behaves in eval mode and one module that behaves in training.
11:02
And then you just use those two things and rather than, you
11:07
know, in the same module calling dot eval dot train, you just,
11:10
you know, name the first one module underscore, train the
11:13
second one module underscore eval and you're done.
11:15
You know, like and then the magic happens And there's a lot
11:17
of things that you can do like that, you know, same thing with
11:19
module dot requires grad false, which people do, you know, to
11:23
detach parameters temporarily And then they do requires grad
11:26
true to re put the parameters in the graph again.
11:30
You can do that, you know, with this very simple trick I just
11:33
explained, you know, and create 2 copies of a model, one that
11:36
requires grad and the other one that doesn't.
11:38
There's another thing that you can do like that.
11:39
And usually, at least in a real world, again, the models are
11:42
tiny, cold, very frequently, etcetera.
11:45
It can bring you like a very decent speed up.
11:48
How much is too much of these copies?
11:50
Because I can see someone that is taking it overboard and
11:56
saying, all right, I'm going to do this.
11:57
And then all of a sudden there's a sprawl and they have 30 or 40
12:01
different copies. It takes 0, virtually 0 space in
12:06
memory. You know, like those things are
12:07
just like regular Python classes and there's no extra data being
12:12
copied anywhere. So like you can do as many as
12:16
you want. I I don't think there's any
12:20
upper bound that I know of. Well, so you did say and kind of
12:23
glossed over Tortoro. I want to know everything about
12:26
Tortoro and also it is interesting that you mentioned
12:30
these are being called frequently.
12:32
These are CPU bound. It feels like a whole different
12:36
world than most people or the hype plays in these days with
12:40
the LLMS, right? So yeah, give me the whole
12:44
trade-offs and what it is and where and why we use it.
12:48
Yeah, sure. So, so the, the story of torture
12:52
starts about four years ago when we had this idea that we wanted
12:56
to build some more domain libraries and we came up with
13:00
things like, you know, Torch Rec for recommendation systems and
13:03
Torch RL. You go for, for RL.
13:05
And the story goes that I was basically hired specifically to
13:08
do that thing here in London. And so I sat down and I started
13:13
reading documents, you know, of people at Meta and people
13:18
outside of Meta who had written about what they wanted to see,
13:22
you know, as an RL library. And there was a lot of things
13:25
like people were sometimes saying, oh, we just want
13:27
functionals, you know, things like value functions and these
13:30
kind of things that can be hard to do right, can be hard to do
13:34
efficiently. But if we have those
13:36
functionals, it's going to help us tremendously.
13:38
But we don't want anything to high level because like we're
13:41
scientists, you know, already the very research space.
13:44
So we're going to deal with that ourselves.
13:46
And then all the people who are like, Oh well, we want
13:48
distributed things, you know, like distributed repeat buffers.
13:51
So repeat buffers is basically a dynamic storage that we have in
13:53
our own. You know, you fill it, you empty
13:55
it all the time because your model is changing.
13:58
So the data you're looking at is changing with the model because
14:01
it's very dynamic. So they wanted some of that
14:05
discount solutions distributed across those.
14:07
So there was a lot of ask and nothing really consistent.
14:10
And then we saw a little bit and we said, well, wait a minute.
14:13
The thing is that RL is very heterogeneous.
14:16
In RL, you have things like Alphago, you know, like MCTS
14:20
kind of stuff, you know, that are very trendy, very
14:23
complicated. You have other things like, you
14:27
know, DQN to play Atari games. But you also have people who are
14:30
using RL to do, I don't know, portfolio management or
14:34
autonomous driving or robotics or you know, it goes a bit all
14:40
over the the space of applications.
14:42
That you know. And also like robots playing
14:46
soccer, this kind of thing. So really it's, it's really
14:48
crazy how many things you can do with RL.
14:50
But The thing is that if you're writing a library, I always
14:52
compare myself to my colleagues at Towards Vision, which I think
14:55
have a much easier life because they're like, oh, well, I know
14:58
exactly what's going to come in my modules.
15:00
You know, it's going to be an image and the output is going to
15:02
be something like a label or maybe, I don't know, something
15:05
that has to do with the image, you know, So the, the, the, the,
15:08
the space of possibilities is very limited.
15:11
You know, it's going to be an image or video plus maybe a
15:13
bonding box plus maybe a label. But you're never going to find
15:17
something as crazy as, I don't know, a balance sheet, you know,
15:22
or something like that in a row. You can have anything like the
15:26
data is basic anything. And the other thing is also RL
15:31
is not about because RL is not about the media.
15:33
RL is about the algorithm. And the algorithm basically
15:36
means that what you call a policy, you know this module
15:40
that is making decisions in your environment, it can be very
15:45
different depending on the algorithm.
15:46
The PPO that we use and process for RHF has a policy that
15:51
outputs an action probabilistically, you know,
15:53
given the certain probability distribution, and it also
15:56
outputs the log probability of that of that action conditioned
16:00
on the distribution. And that's not the case for
16:02
DQNDQN doesn't output actually an action.
16:05
DQN outputs a value that then you transform into an action and
16:08
you don't have this concept of log probability of the action.
16:11
So you have two very similar things, you know, 2 basic
16:13
modules that are aimed at making a decision in the environment
16:18
and they have a very different signature.
16:20
And we're like, OK, how are we going to build a library around
16:23
things that have kind this kind of this common flavour of being
16:29
a policy or a repeat buffer that stores data or, I don't know,
16:34
like a value function when the signatures are so different.
16:38
And that, that seemed like an impossible problem to solve.
16:40
And then we said, OK, what we're going to do is that we're going
16:43
to say, OK, any policy in torsional is going to receive
16:47
sort of a dictionary as input. It's going to do its job and
16:50
it's going to output a dictionary.
16:52
And once we have abstracted what the signature of the policy is,
16:56
we can say, OK, the policy just goes there in the pipeline and
16:59
something with the repeat buffer.
17:00
The repeat buffer receives a dictionary and does its own
17:02
business with, with with that dictionary.
17:04
And when you sample from that repeat buffer, you're going to
17:07
get to know the dictionary. And like, OK, that seems like a
17:10
pretty decent way to go forward, but dealing with dictionaries is
17:14
not very convenient if you have two dictionaries.
17:16
Like if I, if I have an environment and I do 100 steps
17:19
in this environment, and then I try to stack those dictionaries
17:22
together and represent them contiguously in memory, I might
17:26
use something like pie tree. And pie tree is definitely like
17:30
a proper solution to do that. But it's not very intuitive how
17:32
we're going to call pie tree over your set of dictionaries to
17:36
stack things together. And so we thought, OK, why don't
17:40
we come up with another abstraction that allows us to
17:43
easily stack those dictionaries together.
17:46
And we came up with this thing, this thing called tensor dict.
17:48
So initially tensor dict was just a class living in in
17:51
Torshaw that basically behaves like a dictionary but has some
17:56
extra features from torso tensor.
17:58
And those extra features are like I was saying, those
18:00
stacking tensor Dicks together, reshaping them.
18:03
But also other stuff like sending a tensor Dick from your
18:06
CPU to CUDA or from CUDA to CPU, which we do a lot in error
18:10
because we need to save things, you know, in the repeat buffer,
18:14
Other things like communicating between the two different nodes.
18:18
So it has a lot of distributed features and it has a lot of
18:22
like utils like if you have a tensor date, you can call tensor
18:24
date dot bytes. And then you're going to get the
18:27
amount of data that is contained in your tensor date.
18:29
So that you can say, oh, well, if one single element that I get
18:32
here is like one MB pig and I need to store 1 million of them,
18:37
my repair buffer needs to be as big as that.
18:39
Do I have enough RAM or don't I you know, to make this kind of
18:43
decisions? Also, if you attend something
18:45
that contains Llama 3.2, you can look at that and just call the
18:48
bytes and you're going to know how big is your Llama model
18:50
basically. So it has like all all of these
18:54
features. So the story is that we had this
18:56
thing living in Torshaw and the early users were like, man, this
18:59
is amazing. Like I'm using that all of the
19:02
time, but not only in RLA, you know, things like, you know,
19:05
unsupervised learning or semi supervised learning and they're
19:09
like. Because it had that modular
19:12
ability or it it was able to have that flexibility.
19:16
Since you said we're just going to have these dictionary types
19:20
and we don't really care much about how you're using it, just
19:25
we give you everything you need to use it how you want to use
19:28
it. Yeah, exactly.
19:29
And from the beginning, we really tried for Tensor to be
19:31
distinct from RL. Like there was probably one or
19:35
two bad decisions I made at the beginning, you know, in terms of
19:38
designing Tensedict. And I quickly like backtracked
19:40
and said, no, no, this is too, this is overfitted to RL.
19:44
Like we're not going to do that because like it, it doesn't feel
19:47
right. And so those users came to us
19:50
and said, man, you should, you should do another library with
19:54
Tensedict, which we did. So low tensile date is like like
19:57
the it's own thing, separate from Torshire.
19:59
Obviously there's a big chunk of the community using tensile date
20:02
that is like that are users from Torshirell also because of like
20:06
probably historical reasons. But today the users of Tensile
20:10
Date are also like a lot of people that are working in Gen
20:14
AI specifically like, you know, different models and things like
20:19
that. Tensor D can also store non
20:21
tensor data, which is very helpful like again, if you're
20:24
doing for instance portfolio optimization, which as I
20:27
mentioned before, you know, because you can store like, I
20:30
don't know, headlines that go with your financial data or
20:32
things like that. You can basically store anything
20:34
in a tensor D if you do tensor D dot CUDA.
20:37
Obviously if you have a string, the string will not be sent to
20:39
CUDA. That doesn't mean anything
20:40
because it's on the tensor, but all the tensors will behave like
20:44
as expected. And the other use case for
20:46
tensor D is also tensor D I'm talking about about tensor D to
20:50
store data, but you can also use tensor D to store parameters.
20:53
And then again, to come back to RL, we started using tensor D
20:58
for another kind of things, which are functional calls.
21:01
So functional calls are basically that you take the
21:03
parameters out of your module, you store them in a data
21:06
structure, and then you can do things like stacking parameters
21:09
together and calling a vectorized map.
21:12
So instead of looping over different configurations of your
21:14
model, execute all the configurations of the model all
21:17
at once over the stack of parameters, which is much faster
21:20
because things are vectorized using Tensudic.
21:23
That is very easy because again, I if I find 5 different models,
21:27
I asked Tensudic to gather the parameters, I stack those
21:31
parameters together in a continuous, in a set of
21:33
continuous tensors. And then I just called my model,
21:36
functionally dispatched those things.
21:39
And that's much faster, you know, than looking over things.
21:42
So yeah, that that's kind of the story of tortion.
21:44
And that's what made, you know, tortion something that is very
21:46
useful all across the board, you know, for our use cases.
21:50
So what I'm always surprised and, and happy about is when I'm
21:54
talking with users and people come to you and they say, oh,
21:57
I'm using Torch Hall to do drug design.
21:59
And you're like, this is amazing.
22:01
Like I, I have no idea you could do drug design with that thing.
22:03
And, you know, I truly kind of gives me the impression that we
22:09
did, we did a good job, you know, in, in providing a library
22:12
that could be used, you know, for very different, a very
22:14
different set of things. But it does sound that that was
22:17
almost one of the design principles that you had in mind
22:21
was how can we make sure that this is not overfit on that RL?
22:25
And did you, if you remember, because I know it's four years
22:30
ago, how did you figure out that this is what we're looking for
22:34
and this is the abstraction layer that we want to play in?
22:36
Yeah. So that that's a good point.
22:38
Definitely RL libraries all have this kind of solution where they
22:43
have a data container that they use, you know, to carry data and
22:46
even some of them, I guess by chance also maybe tensor DT is
22:50
kind of like the generic name. So you, you might find your
22:54
other tensor DT in other libraries.
22:57
I guess that's here. The, the, the, the specificities
23:01
of tensor DT is really how it blends within the Pythroid
23:04
ecosystem. So compared to other solutions,
23:07
we really tried in so deep to have like this one to one
23:11
equivalence with Torch of the tensor.
23:13
So think about this. You have your single tensor and
23:16
you're writing an optimizer for a single tensor.
23:18
So you read the Adam paper, for instance, and you're like, you
23:20
go to the algorithm section and you're like, OK, you know, Kim
23:26
is giving me like this, this example of how to write my
23:29
optimizer in the line by line for one particular tensor.
23:32
So I'm going to rewrite that in, in Pytorch for one particular
23:35
tensor. And then you write this function
23:38
and instead of passing a tensor, you pass a tensor, you know, and
23:42
all those mathematical operation, you know, like tensor
23:45
dot epsilon times epsilon plus blah, blah.
23:49
You do that with a tensor dict and it's going to work like you
23:53
can literally write a, a quote for a single tensor and just
23:57
pass tensor dict instead. And things are going to work
23:59
out-of-the-box. And that's really like what
24:01
distinguishes tensor dict from existing solutions in other RL
24:05
frameworks, which really build a class for RL use cases and which
24:10
is great like because they don't need to think about anything
24:13
else. I guess that in our case we're
24:15
like, OK, let's try to invest, you know, in making that things
24:18
bigger and little bit more ambitious for broader community,
24:22
which I think is probably interesting to talk about here.
24:25
So you started working on this four years ago in 2020 and that
24:30
was right at like this, this explosion in my mind of folks
24:35
recognising that there was really something here.
24:38
And the developer experience was nice.
24:40
You just kind of walked us through how you were thinking
24:43
about creating abstractions. And it sounds like you were
24:47
servicing people inside of Facebook first, right?
24:51
And then or or inside of Meta, but then for a greater use case
24:59
you recognised, yeah, there, there's something here.
25:02
How much of the developer experience played a part in your
25:07
mind as you're building versus just we want it to do what it
25:11
needs to do because it, it really sounds, as you're
25:14
explaining it all, it really sounds like you're thinking
25:17
through those two vectors. You're thinking through it needs
25:20
to do what it needs to do, but also we will.
25:23
We want to make it so people don't have to go and create new
25:26
Stack Overflow threads because now this tensor doesn't line up
25:30
and we made it overly complicated.
25:33
Yeah. So first I would say that Torch
25:36
L is a little bit bigger in the sense that we really developed
25:39
it for your open source community.
25:41
It's something that I, I really cherish a lot.
25:44
So that let me backtrack a little bit.
25:47
I actually have a background in medicine.
25:49
I used to be a medical doctor like back in the days, and then
25:52
I did a PhD in your science. So I'm, I'm, I kind of have a
25:56
scientific background if you want.
25:57
And I really have this strong belief that it's through
26:03
collaborative development that we can move things forward.
26:06
And I think that's what, what also motivates a lot.
26:08
You know, the, the, the Pytorch folks, about 50% of the
26:11
collaborations in Pytorch are from the open source community,
26:14
you know, so we know that just us is not enough to bring
26:20
Pytorch at full speed. And I think it's the same in
26:23
science, you know, like a single lab will not discover anything.
26:27
It's because you have multiple people publishing and talking
26:29
with each other that you build knowledge, you know, and, and
26:34
I'm, I'm really convinced that I'm really proud to work for a
26:37
company that that puts so much emphasis on open source.
26:41
You know, I, I think that's really amazing.
26:43
And I think it's a winning bet for many reasons.
26:46
But anyway, so yeah, Torchell is very open source oriented.
26:50
Now regarding developer experience, I, I think what made
26:53
the success of Pytorch was really that, well, first the
26:58
eager mode story, you know, that Pytorch was, you didn't need to
27:03
compile a graph to execute Pytorch.
27:05
And so you could like put bricks and prints and things like that,
27:08
or you put if statements and they, they walk onto the bug.
27:11
So that was very handy. But the other thing was Pytorch
27:15
had included within a lot of stuff like NN dot module or
27:20
optimizers and and it's all packaged in.
27:23
So once you install Pythorch, you have all of those tools that
27:25
are presented to you and they're, they're very Pythonic.
27:28
They're very easy to use. We once talked with a very high
27:35
end user who told us what I love about Pythorch is that in 15
27:40
minutes, you can get started, you know, like you just need one
27:44
longest tutorial and it's going to go through, you know, and in
27:48
the module and data orders and and optimizers, and you can
27:53
build your first model and train that on Ms or something, you
27:57
know, and, and, and you're done, you know, like 15 minutes and,
27:59
and you have understood the, the basics, which I find the basic,
28:03
like it's mind boggling, if you think about it, that, that we're
28:06
talking about, one of the things that most people think is the
28:08
one of the most complex achievements of humankind,
28:11
namely AI, And you can learn so much about that in, in such a
28:16
little time. You know, it's a, it's kind of
28:18
nicely done. It's, it's not trivial.
28:20
I think that we're, we, we developers try to keep that in
28:25
mind, you know, this kind of white or spirit when we develop
28:28
new stuff. So I talked about, and that's
28:30
definitely what I'm thinking about.
28:32
I'm like, let's try to find the most by Torchy API or something
28:35
that will be easy to use and does not require a learning
28:40
curve that is too long and does not like change the way you
28:44
think about things too much. Another thing for instance, that
28:47
we recently open sourced is distributed tensors, no D
28:50
tensors. And the idea there is that you
28:52
have one tensor that you Shard, you know, across multiple nodes
28:56
that is represented as a single object.
28:58
And when you manipulate that object, you're actually
29:01
manipulating the sharded version of that.
29:05
I think it's a brilliant idea, you know, like, because it
29:07
simplify things a lot for you. You're looking at that thing and
29:10
like, is it a tensor? Is it a tensor?
29:12
I don't care. I can write code for that.
29:14
It's going to work out-of-the-box, whether it's
29:16
distributed or not. You know, like this kind of
29:18
thing, I think is like super duper handy.
29:22
And, and I, I would bet that this is the kind of stuff that
29:25
is going to guarantee, you know, pipe or chooser ability on the
29:28
long term. And when would that be useful?
29:31
Specifically, the distributed. Any time you're writing a code
29:35
and you're like, oh, well, I want to apply this operation,
29:38
you know, across different nodes.
29:41
And you know, for instance, you, you, you just want to sum to
29:45
tensors and you want your code to work either on a single node
29:49
or on multiple nodes. Well, you the tensor one post
29:52
tensor 2. And if those two things are the
29:54
tensors, it's going to work, you know, as it works, you know,
29:57
with a single tensor on a single machine.
29:59
So basically, if you want to unit test your code, you don't
30:02
need to have like if I'm in the distributed setting do this and
30:06
if I'm not to do that, you know, this kind of thing that are very
30:08
clunky. And then you read that thing in
30:09
like, Oh my gosh, what's happening there?
30:11
And when you're exploring the code, it's very difficult here.
30:14
You kind of abstract that that distributed factor away and
30:16
you're basically separating things.
30:19
You know, a little bit like I was saying, like what I was
30:21
saying with Torch Rally, you know, I was saying, well, we
30:23
have one policy and does different things and then you're
30:26
like and start to abstract that away and separate things and you
30:29
know, get the nice API where you don't need to care too much
30:32
about that. I think this is the same thing
30:34
with with the sensor in my understanding.
30:36
It really sounds like you're looking at probably these the
30:41
most common use cases or the most common ways that people are
30:45
utilising this and recognising what someone is looking for as
30:52
far as the output. Or basically like looking at the
30:57
steps that they're doing and then saying to yourself, can we
31:02
make these 10 steps 5 and can we still keep all of the usability
31:08
and all of the knobs that if somebody wants to go and they
31:11
want to figure things out, they can, but there's probably some
31:15
bloat there. And so how do we get rid of the
31:17
bloat? Yeah, there's that.
31:20
And there's also like, obviously, you know, we're
31:24
obviously working very cosy with a lot of folks in the community,
31:27
you know, like within Meta, you know that we have like Lana team
31:31
and Jenny I, but also outside of Meta, a lot of those big
31:35
companies and startups are using Pytorch and are collaborating
31:39
with us, you know, in implementing stuff.
31:41
And so we kind of have a lot of feedback from users who are
31:45
sending us and we're very, you know, aware of that and really
31:48
seeking for that activity, you know, going to people and
31:50
saying, OK, what are the pain points right now?
31:53
Like if, if you had, if you had the magic wand, you could like
31:57
solve something in bite or make something easier, what would you
32:00
do? And usually it balls on to what
32:01
you were saying, you know, like burger plate code.
32:04
I, I, every time I'm doing this, I'm going in the, in the dark or
32:08
in some tutorial and copy pasting those 5 lines.
32:10
If people tell you that it's not good, like that means that
32:13
you're missing a point. There's something easier that
32:16
you that you should provide as an API and and we we try to
32:20
engage with people to understand what those things could be.
32:24
So that brings up another really great point is where have you
32:30
seen some challenges that you haven't gotten to fix or like
32:36
debugging challenges or things that you just feel like, oof,
32:42
these are some hairy problems. We haven't made the developer
32:45
experience perfect here yet, but we are maybe going to or we're
32:49
thinking about it. And here's a a few best
32:52
practises that you can share with us right now that you've
32:55
seen others doing. I, I mentioned a couple of times
32:59
already, Torso compile, I think torso compile is getting to a
33:02
stage right now where it's very mature, you know, and actually I
33:06
was really surprised in RL, you know, I, I, I, I tried to use
33:11
it, you know, about six months ago for the various things that
33:14
we have. And we saw some amazing speed
33:18
UPS when using compiled correctly.
33:21
I think that the real learning curve here was how to use
33:23
compiled correctly. So there are two sides to that
33:26
story. The first side is compile is
33:29
being actively developed. So there are, you know, the way
33:32
compile works in a nutshell is that it looks like at your code,
33:37
it goes through your code and try to interpret your Python
33:40
code to to compile it. And if you have things that are
33:43
not accounted for by compile because the operation is not
33:46
registered, compile is going to graph break.
33:48
So it's going to say, OK here I don't know what to do.
33:50
I'm going to fall back on Python And everything that comes before
33:53
that is going to be 1 graph and everything that goes after that
33:56
is going to be another graph. Obviously assuming that
34:00
everything that comes after has the graph break.
34:03
No, you, you compile those two things and you have a graph
34:05
break in the middle. So the game for both the
34:09
developers and the user is to have as few graph breaks as
34:12
possible. So the first thing is from the
34:14
developer side, what we like is having people coming to us and
34:17
say, hey guys, I looked at, you know, the report of my compile
34:21
code and it seems that you did not account for that.
34:24
A good example is something that we pinpoint to to our our
34:28
friends side in the compile team a few weeks ago, which was, you
34:31
know, there is this thing called Hydra.
34:33
You probably know of Hydra. We know it's a it's a library to
34:36
do configurations of, of experiments and things like
34:39
that. Hydra realised on something
34:41
called Omega con Omega con has like this Dictionary of configs,
34:44
you know, things like that. And when you call a config a
34:48
config object dot something. So when you get an attribute of
34:51
that config, it was gruff breaking.
34:53
You know, it was it's that that's not very good because
34:56
sometimes people doing their code, if config dot clip grad,
35:00
then clip the grad, you know, these kind of things.
35:03
But that means that every time you reach that point in the
35:05
ground, you're going to graph break and not ideal.
35:08
Maybe you, you, you don't want to do that.
35:10
So we realise that they were there were graph breaks at this
35:12
point and that was slowing the code, you know, for the users
35:15
and, and we fixed that. So right now our friends to go
35:19
from towards Dynamo, fix that thing And, and if, if you have a
35:22
hydra config that has been called in your graph, it's not a
35:25
good graph break anymore. So that's on our end, but on the
35:28
user end, there might also be the, it might also be the case
35:30
that like, OK, I'm going to report to them that this thing
35:34
and that thing could be optimised.
35:35
But in the meantime, I'm going to try to find a workaround, you
35:38
know, find a version of my code that does not 'cause this kind
35:40
of graphic. And so we have something called
35:43
Torch underscore logs. So if you Google that torch
35:45
logs, you're going to find, you know, all the flags that you can
35:48
put. So this is basically an
35:49
environment variable that you that you set before running your
35:52
code. And it basically tells Pythors
35:55
to tell you all of the things, for instance, that cause graph
35:59
breaks. So it's going to tell you at
36:02
this line in your code, it's something that was not
36:04
registered. So I'm going to put a graph
36:06
break there. And you read that and you're
36:07
like, OK, that seems like something I could do
36:10
differently. You know, typical example is an
36:14
if statement or this kind of things.
36:17
An if statement can be sometimes for return with like tors dot
36:20
wire, you know, and that's probably going to help, you
36:23
know, you to to to to solve your graph break.
36:26
But you have a lot of other things that you can see.
36:28
You know, the other thing that that tors dot compiled does is
36:32
that it puts a certain number of guards on your code.
36:34
So when you execute the code the first time within tors dot
36:37
compiled, it's going to say, OK, I have this kind of inputs, you
36:39
know, for instance, an integer, and there's somewhere in the
36:43
code if the integer equals 5, you know, I'm like making this
36:48
up, but it could be the case, you know, if this integer is
36:50
five, do something and then it puts a guard on the integer and
36:53
says, OK, before executing the compiled code, I'm going to
36:56
check that the integer is five. If the integer is five, I don't
36:59
need to put any graph back because I'm happy I can do a
37:00
single graph. But then you change your integer
37:03
and the integer becomes 4. Then compiler is going to do 2
37:06
things. The first thing is that it's
37:07
going to put a graph break, you know at the if statement.
37:10
And the other thing is that it's going to recompile things you
37:12
know for you and do a second version of the compiled code to
37:14
run this particular use. Here is where the integer is not
37:17
4. Now The thing is you can look at
37:20
that, you can look at what the guards are and what the
37:22
recompiles are and you can try to solve these things and say
37:26
and be proactive basically in making compile happy.
37:29
So it can be a little bit of work, but once you get to work
37:32
with it, it becomes very intuitive.
37:33
Actually, I, I was really surprised because I was like, Oh
37:36
my gosh, they're really asking me to do other stuff.
37:38
But it became a little bit funny.
37:39
You know, I, I, I actually like to optimise my models and make
37:42
them as performance as possible. So, you know, it, you, you have
37:46
this boost of dopamine, you know, you, you rerun your model,
37:49
it's two times faster just because you rewrote one line of
37:51
code. They're like, Oh my gosh, this
37:52
is fantastic. So in a real, by doing this kind
37:55
of stuff, we got models to run sometimes 6 to 8 times faster
38:00
than what they used to without compile.
38:02
So that that was very, very rewarding.
38:05
Well, even just talking about that environment variable, what
38:09
was that one again? Torch underscore logs.
38:12
So that is incredible because you can see where you need to
38:18
change things around. And it's almost like that
38:22
cliche, you can't track what you don't measure.
38:25
And so if you're not seeing where it's breaking, then how
38:31
are you supposed to know how to make your code better and how to
38:34
compile it better? What is the best way to be a
38:39
productive community member? And when we see there is
38:44
something that for sure could be better, bring that to the team
38:50
so that it at least is seen and hopefully actioned on.
38:56
Yeah, well, very simply GitHub issues, you go on Pythorch like
39:00
Pythorch and you submit an issue.
39:02
If it's not Pythorch, it's similar to library like Torch
39:04
Vision. Go on Torch Visions GitHub and
39:07
submit the issue. And one thing also is, well, try
39:11
to be reasonable with what I'm going to say, but people are
39:15
very busy. Everyone's busy.
39:16
You're busy, I'm busy, everyone's busy.
39:18
So keep in mind that if people do not answer, it's not
39:22
necessary that they have anything against you or they
39:24
don't think that this problem is important to address or
39:27
something. It's probably just that it
39:30
slipped through or they're busy doing something else.
39:33
So if there's something that annoys you and you submit an
39:36
issue and no one has answered or no one has moved the needle,
39:39
there are two things that you can do.
39:42
First thing is ping them. You know, you wait two weeks, 3
39:45
weeks. And if you really see that
39:48
nothing's moving, you just say, hey, guys, like, you know, you
39:51
type the person who's like assigned to that issue and you
39:54
say, hey, guys, I was wondering how are things moving over
39:56
there? It's usually better to do that
39:58
than just dropping it, you know, because you're not helping
40:02
anyone by just saying, Oh, well, I'm going to try to find
40:05
something else, you know, And the other thing is.
40:08
You can be an active contributor and you can have a look at that
40:11
and try to figure out, you know, do a little bit of homework, not
40:14
necessarily fixing the issue, but dig a little bit more, you
40:17
know, and say, OK, I'm going to try to narrow it down for them,
40:21
you know, because one thing is saying if I execute that code is
40:27
broken. The other thing is to do the
40:28
extra step, you know, of going through the code and understand
40:31
what's going on. It's sometimes a little bit
40:33
harder with the compiler because it's very intricated.
40:36
And even myself like finding, you know, what we call the
40:39
minimal reproducible example is not an easy in compile to be
40:44
honest. It's, it's sometimes very, very
40:45
difficult because it depends so much on the whole stack of
40:50
things that you're putting together that producing this
40:53
minimal reproducible example isn't easy.
40:56
But fortunately the guys from torture compile have a lot of
40:59
tooling to understand what's going on.
41:00
So if you submit an issue with compile, they will probably come
41:03
back to you and say, install that thing or put towards the
41:07
logs. You got something and tell me,
41:09
you know, copy paste the, the print of what you get there.
41:12
And then you're going to put, copy paste and it will be like,
41:14
OK, it's this line of code. Can you tell me what this line
41:16
of code is doing? You know, and like this
41:17
conversation can can keep on going and then eventually will
41:21
you will be able to to solve that.
41:23
So the the, the, the take a message here is really, if
41:27
you're dealing with Torjot compile and you have an issue
41:31
and you cannot find a minimal reproducible example, don't
41:34
worry about that. Submit that issue, Let people
41:37
know and start the conversation as already as you can.
41:40
And and there's probably something that can be fixed too.
41:44
What do you think of when you think of a very well put
41:49
together GitHub issue and let's take for Torch RL?
41:53
What really makes you see an issue and go, wow, that is nice
41:58
that they went through all of that.
42:02
Yeah. So what I'd like to know is for
42:05
personal curiosity, but also sometimes it's important for the
42:09
issue what you're trying to do. Just saying, if I run this, it
42:13
breaks. OK, but what was your intention?
42:16
You know, like for instance, if I create the subject and I put
42:19
this apartment or equals blah, blah, it breaks and blah blah.
42:23
Doesn't really make sense. You know, if you look at the
42:25
door string, it's not something that is expected.
42:26
And you're like, OK, So what was it that you were trying to do?
42:30
Because maybe we can help you. Maybe what you're trying to do
42:33
is not like the way we thought about it, and maybe that's
42:37
totally valid to do it that way. We just didn't think about it.
42:39
But you know, like try to explain what's the broader
42:42
context. Once you have done that, if you
42:44
can, try to find a minimal reproducible example saying if I
42:48
run Python myscript dot π, it breaks at this point is not
42:52
going to help me because I don't necessarily want to install a
42:56
new virtual environment with all of the dependencies that you
43:00
have on the same machine that you have, you know, like a
43:02
Docker image or something. And then try to rerun the exact
43:05
same thing to see where the properties.
43:07
So if you can, you know, with a minimal amount of dependencies
43:11
and minimal amount of code, we run the problem and find what
43:14
the problem was. That's much better.
43:16
Obviously, it's not like I was saying with the compiler,
43:19
sometimes it's, it's just not possible And, and that's OK.
43:22
But just like, let us know, you know, I tried to, to do a
43:25
minimal, minimal representational example,
43:28
couldn't do it in like 30 minutes or so or 30 minutes or
43:32
whatever. So here's the issue.
43:34
And that's, that's all right. We, we, we can, we can find a
43:37
way. And the last thing is, if you
43:39
want and you are interested, try to dig in the code.
43:41
You know, look at first the doc like make sure that it's not
43:45
documented already. Like if there's a huge warning
43:48
in the doc strings that say you should not be doing that and you
43:50
do that like that's, that's not perfect.
43:53
So read the doc of the of the class that's breaking and these
43:56
kind of things. Try to see if it's already
43:58
documented. Try to see if there's not
43:59
another issue that documents that that problem.
44:02
And finally, if you can well, help us identify the problem
44:07
like that, we don't have to. But, you know, that's, that's
44:09
already optional, obviously. But I know that a lot of users
44:12
like to do that because they like to understand what the
44:14
country doing. And that gives them the sense of
44:17
mastering your library better. And so that has some value, I
44:21
think, for users to, you know, to to do this kind of extra
44:24
step, but it requires time sometimes.
44:26
So, yeah, yeah. Talk to me about testing pie
44:30
torch models. I know that we wanted to get
44:32
into this and I, I'm wondering a little bit about if you yourself
44:38
have any tips and tricks because up until now this has been just
44:42
a treasure trove of tips and tricks.
44:44
I'm learning a tonne, so thanks for that.
44:47
But when it comes to libraries or frameworks with testing the
44:52
specific PIE torch models, is there anything that you have to
44:56
tell us that you can enlighten us with?
44:59
Yeah, sure. So in in the Pytorch code base,
45:04
people usually use the built in unit test library.
45:07
But in in torch vision, in torch L, we like to use pytest.
45:11
So you in torch vision you have for instance a set of
45:13
transforms, you know, to transform your image from colour
45:17
to back grayscale and this kind of things.
45:21
We have the same kind of transforms in Torch L to do
45:23
common operations like, you know, reshaping, permuting the
45:27
dimensions, sending data from 1 device to the other,
45:30
discounting. So all of those transforms share
45:33
kind of the same signatures. You know, they usually you want
45:36
to execute them on single thread or multiple thread.
45:39
You want them to be, to be used maybe in the repair buffer
45:43
because we want those transforms, you know, either to,
45:45
to play with the environment when you're collecting data, but
45:47
also to transform the data when you're sampling it from the data
45:51
set. So we want those, those things
45:52
to work in various scenarios. So what we have is that we have
45:56
an abstract class that tests the transforms and we ask the users,
46:00
the developers, sorry, the contributors, when they develop
46:03
a new transform to basically copy paste that base class and
46:07
write an example of that for the transformer that they're
46:11
implementing. And I got to make sure that the
46:14
transform ticks the boxes that that it needs to, you know, like
46:18
it needs to work in a single thread, it needs to work in
46:20
multiple threads, it's to work with a Reaper buffer, it needs
46:23
to work as a module, etcetera, etcetera.
46:25
So we, we really ask people to abide by this rule and and, and
46:30
use that abstract kind of checklist.
46:34
And we do the same in Tensor Dick.
46:36
So in the Tensor Dick library, we have the best class called
46:39
Tensor Dick, you know, and it has all these nice features.
46:42
But one thing I didn't talk about is that there's a data
46:44
class version of that, which is called tensor class.
46:46
So you decorate your data class, instead of doing as data class,
46:50
you do as tensor class. And then magically your data
46:54
class now has a lot of attributes like that's to, you
46:57
know, to send to a certain device.
46:59
Or you can add tensor classes together or you can do some very
47:03
fancy stuff like you would do again, you know, with a regular
47:05
tensor. So when you write test, we have
47:09
all these battery of tests for tensor dict.
47:11
And what we do is that we iterate the test over tensor
47:14
dict and then we iterate the test over a tensor classes.
47:18
And then we do that again for another type of tensor dig that
47:21
stores H5 databases. So we we have all of those says
47:27
that we basically decline over the values instances of density
47:32
that we had and and that proved to be very useful to catch edge
47:35
cases, you know, to say, Oh, well, we're designing this new
47:39
method, you know, like for instance, this arithmetic
47:41
operation that was not implemented before.
47:43
Let's test that arithmetic operation over all the tensor
47:47
classes that we had. And then we do that.
47:49
And then usually we catch, oh, well, you know, here there's a
47:51
warning that is being raised. Here's the this behaviour is not
47:55
really properly defined with, for instance, an H5, typically
48:00
with a regular tensor leak, Tensor leak dot to CUDA is
48:02
you're going to send your tensor D to CUDA.
48:04
And yeah, that makes sense. You know, it was a Tensor D, it
48:06
is still tensor leak. Now, if your tensor leak is
48:08
containing an H5 database and you send that H5 database to
48:11
CUDA, what's the expected behaviour?
48:14
Like what, what should that thing do?
48:17
So, you know, by testing things systematically, we kind of like,
48:21
Oh yeah, I did nothing about that, what should we do there?
48:24
And then it pushes you to write to the proper dog that goes with
48:27
it, etcetera, etcetera. So try to be systematic in your
48:30
test. That's basically the idea, you
48:31
know, implementing a new feature, writing a new block of
48:35
tests, and just saying I'm done with it, I'm happy.
48:38
Usually if you don't have a systematic way to test things
48:41
and your project is ambitious, it's not going to cut it.
48:45
And then from the contributor side of things, that's probably
48:50
something I should add here, which is if you're working on
48:52
your feature, the proper way to do that is first to raise an
48:57
issue on the repo. To go on the repo, raise an
49:00
issue and say, I have this idea about your feature.
49:02
What do you guys think about that?
49:04
Because it might be the case that you're going to work a week
49:06
on that feature. You're going to think, oh, it's
49:07
marvellous. Everyone needs that.
49:08
And either it already exists, or people don't think it's actually
49:13
that useful because there's already a way to do that, or.
49:15
Whatever so. Yeah, engage with with the
49:19
maintainers and, and try to, you know, seek goodbyes and once
49:23
you're done with that, don't hesitate to ask how should I
49:25
test that? Because then they were going to
49:27
go to you and say, Oh well, you know we, we have this way of
49:29
testing things typically, like I was saying in Tesoli torch
49:32
value, we have this battery of tests that we run automatically.
49:35
Here's here is where the dock is here is how you should do it.
49:38
Here is an example of APR that does what you want to do.
49:42
And by gathering that information, you can save
49:45
yourself actually a lot of time and save the reviewer a lot of
49:48
time because that person is going to have to, you know,
49:51
review a code, then tell you, Oh well, you need to modify that.
49:54
And that's, you know, sometimes spent for everyone.
49:57
So if you can avoid that, that's better.
50:00
What are the things you're most excited about working on in the
50:05
next six months when it comes to Pytorch?
50:08
Yeah. So I'm, I'm, I'm really excited
50:11
about 10 Sodic. I, I, I think that we really
50:15
have like an API that it's so generic.
50:20
And, and the, the, the reason why I'm really excited about
50:23
that is that I'm looking back, I, I try, sometimes try to do
50:26
that exercise, you know, putting myself like six years ago and
50:29
looking up by torch in it's early days.
50:31
And I'm thinking if we had had tensoric at that time, what
50:36
would have gone differently? Like what kind of thing would
50:39
have been easier to code? And there's a lot of things like
50:42
functional codes would have been easier, you know, but I don't
50:46
know if you notice, but in Pythorch, when you're building
50:48
an optimizer, you actually don't pass a state deck, but you pass
50:51
a list of parameters. That's kind of weird because you
50:55
know, when you represent your parameters to serialise them,
50:58
you build a state deck. So why don't we use the same
51:00
thing for both? If you had had tensor deck at
51:02
the time within Pythorch to bite your ecosystem, we would
51:05
probably have said a state deck is a tensor deck and the
51:08
optimizer takes a tensor as input.
51:11
I guess. I hope so, You know, and that
51:12
would have been like, you know, everything is like unified.
51:16
Where would I say represent parameters always as a tensor
51:19
date. And that simplifies things in
51:21
terms of writing the optimizer, in terms of passing things, in
51:24
terms of serialising your optimizer or your model.
51:28
You know, things would have been, I think probably better
51:32
for some things. And I come from a field where
51:36
people do a lot of fancy stuff like high order derivatives and
51:39
things like that. And again, I think when I work
51:41
with them, so I think it's so much easier to do some stuff.
51:45
So I'm really excited about that.
51:47
And the other thing is one of the things I'm I'm working on
51:51
right now is basically to to work on the planning side of
51:55
things. So, you know, planning is
51:57
something that is super important, obviously in our
52:01
health, you know, to play games like Go or chess.
52:05
Also in robotics, you know, your robots needs to plan the kind of
52:09
the, the movements it's going to do if you don't want it to crash
52:11
into a row or something. But also obviously in the, in
52:14
the realm of LLMS and things, planning is becoming, you know,
52:19
always more important with, you know, strawberry and all this
52:22
kind of things. We, we, we definitely saw that
52:24
there's a huge trend there too. And I think torchial through it.
52:29
So it has like the right API to solve this kind of problems
52:33
because we deal very easily with things like representation of
52:37
tree of thoughts and this kind of things.
52:40
And I'm really excited about seeing where we can go in that
52:43
space using this kind of primitives.
52:45
So that's going to be part of my focus in in the next month.
52:49
But yeah, hopefully we we will be able to ship some exciting
52:53
stuff in the future. Are there any frameworks or
52:56
libraries or anything that you're using that we should know
52:59
about? Let me think about that.
53:04
And while you're thinking it's just because I'm subscribed to
53:08
this like tools dot dev or something like a dev tools dot
53:12
something newsletter. And one of the coolest things is
53:16
that they just have a bunch of random new frameworks and
53:20
libraries and tools that come out every week.
53:23
And it's like, oh, wow, 90% of them are not useful for me, but
53:29
I still see them and know that they're out there.
53:31
And it's kind of like, oh, cool, I guess that seems like it's a
53:34
novel way of doing things, right?
53:36
And so it's interesting when that happens.
53:38
And I know that whenever we have meetups, some of the most
53:44
passionate conversations amongst the people in the meet ups are,
53:49
have you seen this tool? Oh, it has this feature that you
53:52
can do and you should really check it out next time.
53:55
And so that's almost like some of the fun stuff that happens
53:58
when you get together with people.
54:01
Yeah. So one thing I'm I'm using a lot
54:05
for all my experiments in that I mentioned before is Hydra.
54:09
I really think that it simplifies a lot, you know,
54:12
experimenting with things. So basically there the one very
54:16
nice thing about Hydra and one of the reasons I, I it's used a
54:19
lot in the research community, but I think it's underused.
54:22
And what I'm trying to say there is with Hydra one very powerful
54:26
things that can easily do, you know, sweeps off your parameter
54:31
configuration. So you can easily say, OK, I
54:34
want to try learning rate of 0.10.010.001, you know, and it's
54:40
just one line comment. You know, when you're running
54:42
our experiment, you just say try those three different things and
54:45
then you can sync that with your one DB account and it's going to
54:48
run the three experiments and then you get the results too.
54:51
And it's amazing you get that and and you can look at that and
54:54
you're like, OK, this one is better than this other one.
54:57
Another thing that's we're using a lot is this library called
55:01
submitted. And that's something that I'm
55:03
using also heavily to run experiments and submitted.
55:06
The idea there is that it's, if you know about Slurm, you know,
55:08
on a cluster, you're basically telling Slurm, OK, run those
55:11
different experiments, submit it as a way to write this kind of
55:15
stuff, but in Python. And so you can very easily
55:18
interact with your model and say, OK, I'm going to create
55:21
this instance of my model, but you're going to run that with
55:23
these different set of parameters and it becomes very,
55:26
very handy and much more intuitive for people who are
55:29
used to use Python. So basically you can like
55:31
configure things to go with slurm in a very flexible way
55:35
without having to digest the whole like slurm duck or
55:40
something like that. Which I've heard it is very
55:43
painful sometimes. Yeah, it's not always obvious,
55:46
especially if you look at things like what's defined as a job and
55:50
as a task and like, it can be, Oh my gosh, what are we talking
55:54
about? Or how many GPUs per node per
55:56
task? And you're like, oh, I'm using
55:58
my mind around that. So hopefully somebody can help
56:01
you know, to to to simplify these things so you know.
56:04
And for Hydra you are using it with another like experiment
56:09
tracker such as one DB or ML flow and that.
56:12
Yeah. So they're complimentary.
56:14
Yeah, they're totally complimentary.
56:16
Yeah, they're actually Hydra integrates very, very well with
56:19
with these councils. Yeah, fascinating.
56:21
All right, cool. Well, I got to check that out
56:23
more. I haven't seen that, man.
56:25
Vincent. Awesome dude.