Latest / Agentic Conversations (formally mlops.community) / Bringing DevOps Agility to ML// Luis Ceze // Coffee Sessions #121

Transcript

Bringing DevOps Agility to ML// Luis Ceze // Coffee Sessions #121 · Agentic Conversations (formally mlops.community)

0:00
I'm really says, I'm co-founder and CEO.
0:02
A doctor Mao and I drink french press coffee black.
0:06
I'm not espresso drink ourselves, dude.
0:09
I just gotta say it. We're matching today and we did
0:11
not plan this at all. You said that I didn't notice,
0:16
but you're absolutely right. Oh my God.
0:17
We are, that's literally the same shade of green that's
0:20
embarrassing. And it's not the most common
0:23
shade of green. And so for those that are just
0:25
listening, me, Kyle and I are both wearing t-shirts and it's
0:28
both like this mint. Green color and we just talked
0:33
with Lewis says a who wow, he was special, man.
0:39
This was really cool and I love how everything like he just must
0:43
see the world and bring everything back to like
0:46
Computing. It feels like everything he
0:49
sees. He's like oh yeah that's like
0:51
the same as when we're doing that with computers.
0:54
Yeah, yeah. Yeah I love it.
0:55
I love it. I thought it was fantastic.
0:58
I always I was totally nerding out.
1:00
Things and have a conversation just as he was going down these
1:02
you know, these rabbit holes of like talking there's Hardware
1:05
targets and how that relates to you know, their efforts at
1:07
optimal tying that into the broader vision of what we should
1:11
do. What's the next Frontier
1:13
basically? Bio bio based computation and
1:17
how we can leverage different, you know that for doing solving
1:22
computational problems. And then of course, yeah how it
1:24
tied into at the very end, you know, just talking about books
1:28
that he reads. And how that also Is is a
1:30
metaphor for computing. One thing that I absolutely
1:33
loved from him and his vision, I feel like he's got a really
1:38
strong Vision. He's got a really great way to
1:39
articulate things. I was geeking out listening to
1:42
you all seeking out and how he mentioned you don't he like he
1:49
sees a future where you don't need to think about
1:51
infrastructure. You should just be able to do
1:54
what you got to do and infrastructure happens.
1:58
And this is interesting because Kate on who is also in
2:02
Washington and has he's leading Union dot AI.
2:07
He told me that same vision and it feels like that.
2:11
If there's something there, if he's seeing it, Lois is seeing
2:15
it. People are understanding that
2:18
there is a lot of complexity underneath the hood and that
2:22
most data scientists or machine learning Engineers when they
2:25
start deploying things or even when there before.
2:30
Deployment process. They shouldn't have to be
2:32
worrying about. What's the most efficient way
2:34
for me to do this? That's absolutely right.
2:39
And and that is a trend, I think in a lot of technological
2:43
Paradigm shifts birth starts with, everyone needs to know
2:46
everything and then as the number of options and things
2:52
that you have to keep track of, right?
2:54
In terms of how many different hardware systems, you need to
2:57
accommodate how many different software systems you need to
2:59
calm down. Wait, how many of them modeling?
3:01
Framers means accommodate. There comes this question of
3:03
that's like the unbundling of the ecosystem right where
3:06
everything is just going all over the place.
3:09
And there's this re bundling that happens when someone
3:11
decides wait a second. Like I don't feel that random
3:14
person starting as an Emily tomorrow should have to worry
3:17
about. Oh do I have to make applying an
3:19
x86 computation architecture? Or is this, you know, this is
3:22
something else or am I using a and Nvidia GPU or am I gonna you
3:27
know Intel CPU vs. XYZ thing?
3:30
That makes a ton of sense to me that that's just something that
3:33
we shouldn't have to deal with. And most practitioners should
3:36
focus on the things that they want that.
3:37
They really need to be focusing on which for a lot of purposes.
3:40
Just the modeling, the data concerns, right?
3:44
And I'm all in favor one favor of this movement.
3:47
So what else is interesting about that is we had on the
3:51
podcast? I think like two months ago,
3:53
this guy named David and he works at LexisNexis and he had
3:58
around 12 p.m. HDs really working on low level
4:02
things in the company to help with these with speed and
4:07
compiling and especially when it comes to data and he was talking
4:11
about how he does not want a machine-learning engineer, doing
4:15
those kind of tasks. And he also really wants it to
4:19
be the most efficient. And so what it feels like Louis
4:23
is doing and Caton and I'm sure there's probably others out
4:26
there that I don't know about. Is they're trying to bring That
4:31
benefit to the world without you needing the 12 phds to be
4:36
working on it. So I love that he's doing it.
4:40
I love that. We got to speak a little
4:41
Portuguese with him. Well I got to speak a little
4:45
Portuguese. I got the airport from you say.
4:46
I was listening to Portuguese. I was there you go.
4:49
I enjoyed that very nice language.
4:52
Very nice. So last thing that I will
4:55
mention is we're doing all kinds of local meetups.
5:00
So if you are interested in doing a local meet up, get a
5:05
hold of us, Mikael is organizing the local meetups in San
5:09
Francisco. We got all over the US, they're
5:13
happening all over Europe, they're happening in India,
5:16
they're happening and super cool.
5:18
Which I think we should just do it.
5:21
Have a little Excursion to the first one in Australia.
5:25
It's happening there in Sydney. So wow dude.
5:29
Yeah, actually in Africa to I think they're going to start
5:31
doing it in Nigeria and potentially South Africa so.
5:36
Wow, yeah we're having it happen, man.
5:39
This is worldwide man. Yeah that's it.
5:44
That's what we're trying to do. Here.
5:45
You in simple mr. Worldwide.
5:50
That's the only sentence. The only way that I can be
5:53
compared to that that guy not as cool.
5:57
Well, as him in any other way, but in your community, you are
6:01
as cool in your small Niche. There it is.
6:04
There it is. So yeah, so get out us.
6:07
And without further Ado, let's jump into the conversation that
6:10
we just had with Louis. If you want to find out more
6:13
about the local, meetups you can find us on meetup.com or just
6:18
jump in the slack and there's probably a city Channel named
6:22
after the city that you're in. Alright.
6:24
That's all. We're just talking about how we
6:30
got to keep this fun and fresh and make it interesting because
6:34
there is boring content out there.
6:36
So I got to ask you at least I did a little research before we
6:39
hopped on this and you were on a podcast not too long ago and you
6:43
professed that ml else does not exist.
6:45
So I gotta stay like I took that personally, you coming out of my
6:49
life blood man, what's going on here?
6:53
Explain yourself. Thank you.
6:59
Well, you see when you said that it's just so true that the
7:02
internet, you know, confirmed it broke.
7:07
Yeah, you're back. You're back.
7:08
Your voice. All right, so CeCe said that
7:29
like I everyone you know thanks for having me the podcast.
7:32
You know I can tell this is gonna be fun and interesting so
7:35
I want to qualify that what I mean.
7:37
Now let's let me put my academic hat.
7:39
Like we're talking about just anything about this for a while,
7:42
right? So just a bit of a bit of my
7:43
history that's relevant to our discussion here.
7:45
I've been doing work in the context of, you know, high
7:48
performance Computing, various forms as started in the blue
7:51
jean project doing linear algebra and Hardware
7:54
accelerators and you know how to design for life.
7:57
Sciences, you know. And then, you know, for most of
8:00
my academic career of done, working hard, or soft with cool
8:02
design for emerging applications.
8:05
And I think using machine learning as a workloads to
8:08
motivate this, this research for for a while, right?
8:12
So now that means that, of course, as an academic, I'm
8:15
going to have strong opinions, you know?
8:17
But now as a start-up CEO, breeding a product, I'm also
8:19
going to have strong opinions about what I think, you know,
8:21
this is a market opportunity and and how we actually get machine
8:25
learning which has proven too. Sure that he can solve a lot of
8:28
awesome problems section. Permeating in all of the
8:31
interesting applications. We can think of involving
8:33
computers today. Have a machine learning
8:34
components, right? So give me this ring reaching
8:36
the level of maturity that we have to be very principled about
8:39
how we deal with it. How we deployed, how do we make
8:41
machine learning models and integral parts of our
8:45
applications, right? So I've been challenging
8:47
applications and how to qualify with Dimitri started with the ml
8:50
abscess that. So but Wednesday, the here's how
8:52
I think about it, right? So machine learning models, they
8:56
have code and they have Data, right?
8:58
So in together you know date perform you know computations
9:01
that honestly we didn't even know how to express with just
9:03
code in the past. This is the magic of this is
9:06
using a ton of data to guide. How a blob of code behaves to do
9:09
amazing, things like computer vision.
9:11
That's natural language processing and so on.
9:14
Now when I say the no Lobster, is it should not exist.
9:17
What I really mean is that a lot of what people call in a lobster
9:21
today, and we should talk about it, because there's so many
9:23
definitions of I'm a lapsed that can make it confusing.
9:27
Exist. Just because we cannot treat
9:29
machine learning models. As if you were, were any other
9:32
piece of code. Once the mold is ready to be
9:34
deployed. I fundamentally believe that for
9:37
machine learning deployments reach a level of maturity that I
9:40
think he needs to get were to continue permeating all
9:43
applications. We need to be able to treat
9:44
machine learning models, as if you were in the other piece of
9:47
code, okay? Once you're at that point, why
9:49
not call it that valves? Like if you do it right and you
9:52
make a machine learning model, once it's ready for deployment
9:55
like an integral part of applications.
9:57
And Is devops. I think we're in a good place.
9:59
So that's that's what I really mean.
10:00
All that said the process of getting from a problem you want
10:05
to solve collecting the data and producing machine learning
10:07
model. That is actually ready to be
10:10
deployed. That needs the flow that special
10:13
we can call that analog. So that would be okay with me.
10:16
I just don't want to confuse the model creation Parts with once
10:20
you deploy them all this partying application with MLS.
10:23
Does that make sense now our do that?
10:26
Yeah yeah, yeah. Breathing easier.
10:28
Now the meters are you reading easier now, having he's
10:31
unfrozen, right? So now he's okay.
10:33
That's right. Yeah, he unfroze.
10:39
Yeah. It sounds like what you're
10:43
saying here is that it's more about like the semantics.
10:45
It's kind of like semantics argument.
10:47
Like there's still the devops, which is a devops piece but, but
10:52
but fundamentally there's this additional component.
10:55
Maybe, maybe we're conflating the term ml.
10:57
Ops to Encompass more than it really should be.
11:00
Should be subsuming. Namely, some of this more, like
11:02
operationalization of just resources and things like that,
11:06
which is more of a traditional software, engineering problem.
11:09
Yeah, and also operation station, but also integration
11:12
with the application, right? So remember that.
11:15
We have to be careful. Not to think that the machine
11:17
learning model is entire application, right?
11:19
So of course, Machinery Engineers, they are at a
11:22
premium. They're doing awesome.
11:23
Things is what we cannot forget that what makes machine learning
11:26
model, amazing? Is when it's embedded in
11:28
application that people use and pay money for, and creating the
11:32
businesses around it as well on machine.
11:34
Learning model, does not exist in isolation.
11:36
There's a bunch of stuff around it.
11:37
That makes it useful enriches people's hands, right?
11:39
So, and I think that's the parks.
11:41
That is a huge opportunity today.
11:43
Because the way I think about in part of this is obvious is that
11:47
there's all this wonderful machine learning Innovation to
11:49
happen, like it's just incredible to see every day.
11:51
We see a new machine learning model, that does amazing things,
11:53
right? So but then you have this
11:55
application to the other hand. If people actually use this It's
11:57
a huge gap between those. And then this Gap involves a lot
12:01
of manual, engineering, involves a lot of work to actually put
12:04
them all into production, right? So, and we should improve both
12:08
sides, of course, we should continue making the process of
12:10
creating machine learning models easier.
12:12
So, we continue to accelerate Innovation there and create sort
12:16
of like this pool of awesome things that you can put into
12:18
applications. But also you need to work on
12:20
putting a very systematic process to turning that into
12:22
application. So you know, it actually reaches
12:25
a broad audience, right? So And if you don't mind like I
12:28
love to go a bit deeper on the deployment side, you know?
12:31
So I in terms of my background like I can talk for longer than
12:35
you probably want me to about mother deployment.
12:38
We can, we can talk a lot about the machine learnt core machine
12:41
learning side as well because they have some passions there,
12:43
we can discuss. But let's let's talk a bit about
12:45
the deployment side. So, you know, it's it's
12:48
interesting to go through the exercise of thinking about what,
12:51
why are we not treating machine learning models as if you were
12:54
any other piece of code today? First is, it's highly dependent
12:58
on the hardware, right? So you know, naturally, you
13:02
know, machine learning models, fundamentally become high
13:05
performance, linear, algebra code that links to run well and
13:08
that involves using the hardware to the maximum extent possible,
13:12
right? So that the way it's being done
13:14
so far, basically, boils down to very talented, programmers
13:18
writing, low-level code, to make very tight Loops using, you
13:22
know, specific data types, and uses Specific Instructions in
13:25
processors, or in GP. They're in accelerators to keep
13:28
it has a Sheen that, you know, added Peak as or as as high as
13:32
possible, then he goes. A lot of Arts is encode, right?
13:34
So and that typically, if needs to evolve with the model, right?
13:40
So as new models, do new things, there's new operators that have
13:43
to go and generate new very hard, independent code.
13:45
So, to pop pop, what does this mean from?
13:49
Let's almost like from an industry perspective, we saw
13:52
really tight coupling between the hardware and the software
13:55
stack on top of it. So our Probably, you know,
13:57
Nvidia is where it is today in machine learning, because they
14:01
did a fantastic job with Cuda Cuda made it a lot easier for
14:04
folks to go and program. A parallel machine that a GPU
14:08
is, you know, parallel computers have been notoriously hard to
14:10
program, right? So this is a decades-old problem
14:13
in Academia in a decades-old opportunity in Industry to,
14:17
right. So starting from the
14:18
supercomputing days all the way to machine learning today,
14:21
right? So and this tight coupling is so
14:24
is so strong that I like to make an analogy.
14:26
It looks like the Mainframe industry in the 70s and 80s.
14:31
Like, if you bought IBM software, if you wanted IBM
14:33
software, you had to buy IBM hardware.
14:35
And if you want an IBM Hardware, you had to buy it be insulted
14:39
because it's the only thing that was available for that harder,
14:41
right? So unfortunate to a large extent
14:43
that exists in in machine learning today, right?
14:47
So, every harder vendor has its own set of system.
14:49
Software for machine learning, they have to maintain and I quit
14:53
a lot of fragmentation. The industry created a major
14:55
complexity in deploy, Machine learning models, right?
14:58
So once you put it in production to, that was going to put them
15:01
all into production today. You have to go and figure out
15:04
what is the target that's going to run, which libraries are
15:06
going to use? Or maybe if there's a compiler
15:08
available, you're going to go and use that compiled and try
15:10
it. Is it run?
15:11
Faster? Does it work?
15:12
You're going to validate it. And all of this creates the this
15:16
coupling that once you create a version for machine, learning
15:19
model to be Department specific heart attack, you got to stick
15:21
with it or you're going to do the work to support all of the
15:24
other hundred targets that you need to support, right?
15:26
So, Um, and so this is so manual today that, you know, I think it
15:33
stiffens how people Department really models either they go
15:35
suboptimal, you just go in the clouds, pick any sense and you
15:39
scale. Whatever instance you got there
15:41
or you live with poor performance on the edge or
15:44
sometimes you don't even deploy them all to the edge because you
15:46
don't reach the level performance that you need to
15:48
deploy that model, right? So and he said say that's a lot
15:52
of it boils down to part of dependence.
15:54
That, you know, we will talk about how we, how we make the
15:56
Much more portable and avoid the need for for human tuning by
16:02
taking a fundamentally, a compilation approach to machine
16:04
learning, as opposed to a translation Library, pin the
16:07
protract. So any Senator going to like you
16:10
don't aboard, the audience going deeper and deeper than this, but
16:12
it's a like Hardware dependency machine learning, is something
16:15
that is, I think this is is a serious problem that we need to
16:19
solve. And once you decouple model from
16:21
harder thing, you're going to create opportunities on both
16:23
sides. Right?
16:24
So. Yeah and the end.
16:26
There's only really What you're saying and fundamentally?
16:28
I mean, if I could paint like really broad Strokes here,
16:31
talking about just maybe different like we're getting new
16:34
abstraction layers around some of these Concepts, right, this
16:36
interface between the hardware in the software.
16:38
And yeah, I mean, I remember back in college, you know,
16:42
taking parallel, programming classes going through the
16:44
exercise of, you know, Cindy kind of like programming models
16:47
doing stuff like actually having to code up, Cuda kernels.
16:50
And I think a lot of people that enter the field today don't
16:53
recognize how far we come even at this point.
16:56
Compared to those earlier abstraction or the others
16:59
earlier. Kind of Primitives that we had
17:00
to deal with to do anything of sensible from a programming
17:03
standpoint parallel, programming standpoint?
17:05
Yes. So well said me like I?
17:07
Yeah, thank you for acknowledging that we made a ton
17:09
of progress, but even to this day like you know, folks pretty
17:12
much the only mother still think about you know what we're what
17:15
hard are you running? You know you think about
17:17
performance early on when the model creation side imagine a
17:20
world where you this abstraction Evolution goes even further in
17:24
you when you create a machine learning model, all you think
17:26
about This is what is the best work that I can do with the data
17:29
that I have to create the very best model and not think about
17:32
the heart of infrastructure and lets the system do that for you.
17:36
Right? So yeah you know, it's a think
17:39
it's interesting that you know, a lot of Machinery Engineers.
17:42
You know when you see resumes they say they talk about the
17:44
framework, there was nothing else to talk about so we can
17:46
write Cuda code and cried suppose.
17:48
First goes like hmmm. No, I wish that all those brain
17:50
cells went to know all the cognitive Cycles which to
17:54
creating a better model and not having to think about You should
17:57
even think about what the heart is running.
17:59
One should even think about Cooter C++.
18:00
It doesn't make any sense, right?
18:02
So I think it's not even even last Cindy and threading, what
18:06
kind of? But people have to do that
18:07
today, right? So, right, right.
18:09
I mean, yeah, some some smart people did a lot of the legwork
18:12
to make it easier for the next generation and maybe that's, I
18:15
mean this, I think ties really nicely into what your maybe
18:17
outgoing mail and even the efforts of your project in the
18:20
company and is actually pretty nice segue.
18:23
Because in a sense, one of the things that that The optimal
18:27
seems like it was born out of TVM, right?
18:29
And I actually remember using TVM in some of my own work back
18:33
in the day. Great, nice Library.
18:35
Yeah, no, no, I'm just like, it's really nice library, and
18:38
I'd love to just maybe dive a little bit into talking about
18:41
the motivations behind that effort, and how and then
18:44
eventually we can get into talking about how that evolved
18:45
into which, what October was probably start for a few of us
18:50
out there, just like, can you explain what?
18:52
TVM is? Yeah, absolutely.
18:54
Yes. So today, yeah, so TPMS. Since
18:56
for tensor virtual machine, right?
18:59
So it is a VM, a virtual machine analogy because you have a clean
19:02
interface over, you know, that abstracts away Hardware details,
19:06
and you can map machine learning models into that interface and
19:09
then whatever comes underneath. It should be should not be a
19:13
concern for folks creating models, right?
19:15
So, I noticed TVM started out of an effort, probably maybe six to
19:21
eight years ago. Maybe more in the question that
19:23
we are asking then was the following.
19:25
We have lots of machine learning models, we have The part of
19:27
targets there is this, you know, how to depend a software stack,
19:31
you think about how would we think about intermediate
19:33
representation, that you can translate models in, to do a
19:36
bunch of optimizations to make those models run well on
19:39
whatever how to Target you have in the general way.
19:41
Such that you can extend to new harder targets and then
19:44
progressively lower decks to custom codes for a given model
19:48
on a given Hardware. So this is why you wouldn't call
19:51
it. A library approaches video
19:52
compilation approach like we early on as in the first one,
19:55
says the question like can we Turn machine learning model,
19:57
execution from an interpreted process, which is interpreted
20:01
modern called library to modeling and fresh, binary code
20:06
out for that model on the harder that you chose.
20:08
So why am I emphasizing these? I promise.
20:11
It's not to board the audience's that once you go from, you know,
20:14
Library brace interpretive approach to a fresh cold
20:17
generation, but you can do a lot of things that are very
20:19
powerful, right? You can do high-level graph
20:22
optimizations of your model like you can fuse operators, say a
20:26
The pie with the convolution you can do them together so you get
20:30
you know, locality benefits you can do a lot of choice of like
20:34
how should you order the computation in a way that is
20:37
going to better match the hardware capabilities.
20:39
It in, as you go down. How do you automatically choose
20:42
specific harder? Instructions that are available
20:44
on the Honda processor or in the Jeep you that you're running
20:47
there? Might not be available in
20:48
others, right? So if you think about tense
20:51
record today's is are you know essentially hard educated her
20:54
units they can do Matrix multiply for example.
20:56
Oh, but to use that, you have to set up your code, just right.
20:59
But now what if you were automatic, that's the question
21:01
that we are asking. So that's what we do, we divvy
21:03
em. So we took a compiled, a
21:06
compilation based approach and couple that with some special
21:09
sauce that at that time was okay.
21:11
So, when you're compiling machine learning model to given
21:14
harder Target, that's literally billions of ways in which you
21:16
can compile a given model on a given harder Target.
21:20
The question is, how do you pick the very best option, okay?
21:23
So without without trying them all because to try You know, a
21:27
billion combinations of code choices compilation of
21:30
measurement and trying you talking about centuries of
21:32
computer more like just impractical.
21:35
That's when we use machine learning for machine, learning
21:37
code generation and to pick in a using predictive way, how should
21:41
you make these choices compile your code.
21:43
So I know this is a long long intro but I wanted to emphasize
21:48
what makes the game special then?
21:50
And I think was a huge driver of extraction because we not only
21:54
made it very effective in generating code but also making
21:56
It easy to extend to other Frameworks is to extend to other
21:59
Hardware to other Hardware targets.
22:01
We fast forward to, you know, three years at 34 years after
22:05
creation. You know, we're started October
22:08
Mao to game was about to become an Apache software Foundation
22:11
project had been using production for a while, you
22:14
know, at Big hyper scalars and harder than to start adopting it
22:18
because it supported all the major Frameworks and you made it
22:21
easy to extend to Me Harder targets and fast-forward to
22:23
today. Now, to p.m. supports pretty
22:25
much all the major heart attack. It's the folks care about and it
22:29
supports all the major Frameworks and given the
22:32
landscape of machine learning compilers and system software in
22:36
general TV and build Bridges to all of the other pieces.
22:39
And we really more bridges to other compilers for machine
22:42
learning as well. Such that we offer.
22:44
One Stop Shop for folks to come in, and get your models to run
22:47
really well in hundred targets. So, Is that okay intro.
22:51
You have to make sense? Yeah, no.
22:53
I think, I think it's awesome. I mean, if I could even just
22:56
like, you know, just say it even in 11, just maybe pause here to
23:00
say that. I think that there's so many
23:02
details behind what you just said.
23:04
But for people, that that are maybe newcomers to like data
23:08
science machine learning. I think the, the key point is
23:12
that when you do something as simple as, like, Matt mole or,
23:15
you know, convert 2D in your tents, your your pie torch code
23:18
and your your tensor tensorflow. I heard that there's a lot of
23:21
optimizations that have to happen in the background to make
23:24
this run efficiently. So you're not there waiting
23:27
forever for something actually run to completion and that's I
23:30
mean that's really one of the core pieces.
23:31
I think what you're discussing here is like all that extra
23:34
level of optimization needs to happen to make something
23:36
efficient to make it tractable. Right?
23:38
From a computation standpoint, which is like the really is a
23:41
lot of detail here that people are able to disregard because of
23:45
things like TVM, which is really astounding, it's right?
23:48
And until you turned your point. Thank you.
23:49
Yeah. Yeah.
23:50
So very well said is about, you know, raising the level of
23:52
abstractions such that you don't have to worry about performance
23:54
early on, right? So you can go and use these Matt
23:57
Moore convolutions and not worry about, how is it going to run
24:00
fast. And to be clear here, running
24:04
fast is not a nice to have in machine learning.
24:06
You know, making the most of the heart is an enabler.
24:08
Otherwise it's just, you know, the amount of computation
24:10
involved in in doing one, inference is just incredible
24:14
compared to anything else of using computers rights though.
24:17
So I'd love to go even a little bit.
24:20
Maybe to give more context here. Where does something like a TV?
24:23
Em sit in the ecosystem of other efforts have been made to
24:27
provide these harder based compiler optimizations.
24:30
When I think of something like a tensor RT or Onyx runtime,
24:33
right? What you're trying to, maybe,
24:35
maybe tackle someone with similar problems or with maybe
24:38
different approaches. Where does it kind of sit in
24:39
that whole ecosystem? Yeah, great great question and
24:43
TDM integrates with unexplained time and destroyed.
24:45
See the way to think about this is that so Onyx when time is
24:50
Made. It was made to be generally, no
24:52
support a broad set of models and also some sort of hard has,
24:55
it does take, you know, primarily a library based
24:58
approach by having a very rich library of basic operators that
25:01
are pure optimized for harder targets that gives you some
25:03
portability there. But the real question here is,
25:06
is it can you, can you do better by generating fresh code for
25:11
your whole model or parts of your model on the harder that
25:14
you chose at that? Give at a specific time.
25:16
The way we integrate with Onyx runtime is creating we call an
25:19
execution. Vidor where the mother comes
25:21
into an excellent time. And when profitable when makes
25:24
sense it generates fresh codes, okay?
25:27
So intense for Artie. But what it is is invidious
25:30
answer to machine, learning compilation tool to a large
25:33
extent and it's great of course because Nvidia created 100, they
25:37
understand the harder properties, right?
25:39
So in they have a long history of writing high performance code
25:42
for their own hardware and that means they can do a fantastic
25:44
job. But to a large extent has 40 is
25:47
also specialized to the classes of models that Nvidia cares, you
25:52
know, cares about of course because of its customers, right?
25:54
So, where does CVM come in? And we do have, you know,
25:58
collaboration with this. Alrighty folks in showing that
26:02
it's the best of all worlds approach.
26:04
Like, so the way we, it, we do not wear not dogmatic.
26:07
By the way, I want to make it very clear that, you know, we
26:09
created TVM, we invest a lot of resources interviewing, but it's
26:12
not that we see the world, I should be only 2 GM, know our
26:15
Here is to make models, run really fast on the how difficult
26:18
is because it's Saves the World, you know, because it reduces the
26:22
energy consumption of computers and makes people's life easier,
26:24
right? So the reason I'm saying this is
26:26
because we think a complementary approach.
26:28
So if you take him although and you wanna and you have ten ten,
26:33
it happens to be in a video Hardware.
26:34
You what we do is we can show that okay.
26:37
Parts of her mother should go to 10:30 and parts of all the could
26:39
go through TV M and then you pick the vast of each and then
26:41
combining producer, better model.
26:43
Does that make sense? So, and I would say That in
26:46
cases, that UVM tends to offer an advantage, or emerging models
26:50
or things in the long tail that having received as much
26:53
attention because those haven't been optimized by hand as much,
26:55
right? So, this is why I would say,
26:57
they're very complimentary and, but an offer parallel paths to
27:01
define the model. So, to answer your question, me
27:03
how I know it was with might have been a long answer but no,
27:07
no, absolutely. I think that, that makes a lot
27:09
of sense. So this I think also fits nicely
27:12
into, you know. So TVM was the Genesis and in,
27:15
he sounds like TDM. Still forms.
27:16
The core of a lot of October ml, but optimal maybe he's taking
27:19
this one step further where we going with with that.
27:22
Yeah. Where you guys going with that?
27:23
Great. Thank you for asking.
27:25
So you know what would be a doctor Mao is machine learning
27:28
deployment platform, right? So what we want their the
27:31
highest level back to the early conversation, we want to bring
27:34
develops agility to machine learning deployments.
27:36
What does that mean? It means offering hard during
27:39
dependence. So you give your mother can
27:41
report. You never have to think about.
27:43
You don't even need to think about whether You're going to
27:46
have to, you're going to use a library based approach a
27:48
compilation based approach or if it's if it's nvda going to use
27:51
could be Nan or 10:32 p.m. like we abstract all of that away
27:54
such that you have the you know a flow that looks like modeling
27:58
and Deployable portable high performance function code to run
28:02
out that you can you can move around to different harder
28:05
targets. So if you're going to deploy it
28:06
in the cloud, you could deploy in a given GPU.
28:08
But then if you want to scale it up and the instance is not
28:11
available. You can go and run on a
28:12
different GPU and still make the most out of it, right?
28:15
All right. Well, now this is how to make
28:17
the whole process of exploring how your model runs on different
28:21
hardware and produce vulnerable versions for each every, you
28:24
know, Target that you choose. We support all major clouds TCP
28:29
as you're in AWS. We also support some Edge
28:33
targets, though. Our primary primary Strike Zone
28:36
today. You know, with customers is, is
28:38
in cloud deployments. And then you can, if you can
28:41
evaluate, you know how different versions of your model during
28:44
deployment by actually. No, give you the ability to
28:47
create multiple versions of your modern compare, how they do
28:48
respect each other price performance trade-offs, you
28:51
know, things like say that you what you want is a model to run
28:56
difference be at 100 milliseconds or last you don't
28:59
care if it's if you care if it's the more than 100 seconds but
29:02
you don't care if it's you know 15 minutes X is just cause I
29:05
kind of realize. Then after that what you care
29:07
about space throughput per dollar for you to do that today,
29:10
you'd have to go do manual work in a bunch of differences,
29:13
figure out which one you should do it for.
29:15
So, Ultimate all of that introduces, the ability to
29:18
choose that. So it abstracts away.
29:20
All the trade-offs, are you think about, you know, batching
29:23
and concurrency and threading is going like it just tries a bunch
29:25
of combinations are as well and then he puts it in a package to
29:28
can go and deploy it in the cloud or locally via python will
29:32
or daughter. So library in a bunch of
29:34
different packaging. So when I guess, you know,
29:37
stepping back and seeing one line is modeling and Deployable
29:40
code out and then everything you got in between tears of
29:42
benchmarking and, and, you know, evaluating Multifamily
29:46
properties and so on and ultimately the part that
29:48
honestly today's kind of done by hand in a lot of what people
29:51
call them a lava flows, right? So a lot of the just a lot of
29:55
folks in model deployment and fully aware of that indeed
29:58
there's a lot of great stuff being done there so the part of
30:00
you wanted to do presume that we have with with our history and
30:04
our technology. Our technology here is to cover
30:07
the part that is assumed to be done by a very talented
30:11
engineer, which is producing the thing that you're going to
30:13
deploy. And that's the thing that you
30:14
want to talk to mate, right soul.
30:15
Um, in the process of doing that we have, since you guys like
30:18
what is the GM's Road here TPMS is an important part, which is
30:22
not the only part of the platform, right?
30:23
So the platform here can use tense variety and unexplained
30:27
time and you know part torches native Native stack tensorflow,
30:31
native stack and we support all, you know, pretty much all the
30:34
major framework for Frameworks for modern generation and so on.
30:39
Does that, does that give you a general view, what it is?
30:41
Yeah, yeah. Absolutely.
30:43
And I think that I'd looks like, maybe go even one step further
30:46
here by zeroing in on. So as you said, there's a lot of
30:50
players in the deployment space and people have certainly
30:53
recognize that it's it's a big problem area for a lot of
30:56
Emily's and ml teams and data teams.
30:59
And I'm curious you know, when you talk about one of the big
31:03
value adds to something like an opto malware you you have your
31:06
very deliberate because of your research history of TV.
31:09
The one I used very deliberate about this Hardware software
31:11
abstraction how you can optimize for different Hardware targets,
31:14
and you're able to be super, super fine, grained about the
31:16
level of control you given. Hey, I'm Uprising for this this
31:20
Hardware Target. I'm I'm, you know, these latency
31:22
concerns like this, throughput, maybe, you know, you talk about
31:24
cost per dollar kinds of considerations.
31:27
I wonder which of the kinds of groups that drive the most value
31:30
from these that level of specificity in the
31:33
considerations, right? Because there's also one can
31:35
make that argument that there's this whole class of companies
31:38
that don't even have. A single model production yet
31:40
they're just trying to get something out the door, they
31:42
don't care. If it's 50 milliseconds or 25
31:45
milliseconds or like 300 milliseconds, they just need
31:48
something that they can hit inference, you know, they can
31:50
run inference and points with how do you, where does it sit in
31:53
that? Yeah, kind of that, if that
31:55
value chain. Yeah, I know, I love, I love
31:57
that that question. So I would say that we offer,
32:01
you know, harder Independence like we could performance right
32:03
and Automation and really good performance, right?
32:06
So in with these three value the property, Here, depending, on
32:11
the end user, they get value in different ways, right?
32:13
So for the high maturity users, what they care about is
32:17
performance and hydro Independence, right?
32:19
So because, you know, they're models will run faster.
32:22
So they meet they save money running at scale in the cloud
32:25
and they give you heart Independence.
32:27
He gives them more ability to scale and automation reduces,
32:31
you know, the time that they're expensive Engineers spend to get
32:34
those models ready. So that's how they derive that,
32:36
right? So, but now the product, as you
32:38
see, you know, Probably you. If you've been following, we've
32:41
been doing to the. We are, you know, going more and
32:44
more towards broadening the set of users that would extract
32:47
those see value in what we're doing, right?
32:48
So in what we're doing in here is automation, you know,
32:51
decreasing the complexity of getting your model ready for for
32:55
deployment. How does it fit in folks that
32:57
are early in the machine Learning Journey while?
32:59
So for folks is still creating the model, figure out how much
33:01
you learn is going to solve their problem.
33:03
We don't cater to those users, right?
33:05
So we cater to users that once they have model candidates are
33:07
going to put into production. We're going To alleviate all of
33:11
the complexities that typically involves handing off the model
33:15
from a data scientist or machine engineer to a software engineer
33:18
to go and figure out how to deploy.
33:20
All right. So by shortening that time from,
33:23
you know, weeks or months, two hours, think also closes the
33:26
loop of English, like they can deploy it quick here, quicker.
33:29
They can collect data and then figure out what is the next
33:31
iteration of their model early on.
33:33
Right? So anyways the direct answer to
33:35
your question is for folks that are early in the journey and
33:37
less mature. It's really Just about
33:39
automation. They probably don't care as much
33:41
about about performance stand. Unless the model doesn't run
33:45
fast enough to be even be viable, then they're going to
33:47
care about performance, right? So but I would say that what
33:52
they what they care about disability to do to deploy it in
33:55
a way that costs less and takes less time.
33:57
So these are bigger players than like these didn't mature players
34:00
in the ecosystem that maybe companies that have bigger
34:03
resources that are able to, you know, that have already invested
34:06
the time and the effort to build out machine gunner Pipelines.
34:09
Now we're like hey we want to run this on this Hardware device
34:13
or the mobile this Edge Edge Edge device.
34:16
That's really where it seems like a lot of the values being
34:18
derived. This is Skyler.
34:20
I lead machine learning and health rhythms if you want to
34:22
stay on top of everything happening in ml.
34:24
Ops subscribe to this podcast right now is again the high
34:28
maturity folks who see the value immediately because of performs
34:31
a hard Independence in the lower maturity, you know, users of
34:36
machine learning will see value more in the automation right
34:39
there. Oh, of of just we have if you
34:42
have a chance to take, take a look at our CLI demo that we
34:45
have out there. You can see without writing a
34:47
line of code, you can get your model in an endpoint deployed
34:50
and then iterate on it and starting your laptop in the move
34:52
it to the cloud without changing your code at all.
34:54
So in that case, you know, anything about performance, all
34:57
you care about is that, hey, you've done all this automation
34:59
of migrating your code, from a local version to the cloud
35:02
version or two different verses in the cloud.
35:04
And I want I want to think to your point there.
35:06
Oh, sorry. Yeah.
35:07
On the, on the high mature users.
35:08
One thing that's That it is important to highlight here is
35:11
that we spent a lot of efforts in making very easy to integrate
35:16
with existing devops flows, right?
35:19
So that means living with, you can have a model in a
35:21
repository, you can call in your CI CD flow.
35:24
We have an API that you can send them, although download the
35:28
container back and deploy it in a way that does not disrupt
35:31
existing flows. Because if we did, that is just
35:34
becomes a hard sell for the high maturity users to just go and
35:38
and and and have to change how they do things just because they
35:41
want to use our technology, right?
35:43
So we like to think that once they do try it and they see the
35:46
performance of the portability going to go and want to change
35:49
their their flows. But we are working hard to make
35:53
the API, be very easy to integrate to it.
35:55
So so do you mainly talked with s.res and people that are doing
36:00
devops or are you talking more with machine learning Engineers?
36:04
Yeah so it's a it's a good question.
36:06
We talked to to both and again, depends on the level Of
36:09
maturity, right? So the high maturity, the high
36:13
maturity companies deploying and high maturity amount users,
36:16
rights of tend to have either sometimes dedicated seeing to a
36:20
male deployment. So those were the things we
36:22
talked to is some others that have.
36:23
It's just part of the higher-end, you know, more
36:26
machine, learning focused, s res but you know, for the lower
36:31
maturity ones, it seems more the folks that are doing machine
36:34
learning engineering and they have their hands in deployment.
36:37
So then we talked to those in maturity Customers.
36:39
So so you've got one book behind you which is start with why I
36:45
have to ask why Bill dr. Mel well you made some things
36:51
that were interesting to me before about if we have this
36:54
optimized, if we can make things go faster, it's better for the
36:57
environment. It's just better for people
36:58
because they're not sitting around waiting for whatever
37:01
their Hardware to do the hard work.
37:04
But I'm wondering about your personal journey into In this
37:10
product yeah I know. Great question.
37:13
I'm glad you noticed that I promise was not on purpose, you
37:15
know. So we do we did send this book
37:17
to everyone in the company by the way.
37:19
So I really love that book credits.
37:21
Our CMO David miss, you know, who also loves his book is like,
37:24
you know, everyone is completed read this book.
37:26
So our, our vision here is to make AI sustainable and
37:30
accessible. And I Want to Break these two
37:33
words, to say to an accessible down, just to explain the
37:35
originally. So sustainable.
37:37
As in, you know, Faster, more energy, efficient using last
37:40
natural resources, but also sustainable in terms of human
37:43
effort, right? So using natural resources, just
37:45
the right thing to do. As I said, if you look at, as I
37:48
mentioned before, if you look at the amount of Computer Resources
37:51
going to machine learning is just is.
37:52
The fastest-growing category of computing have like in the
37:55
desert very meaningful fraction of Cycles in the cloud and eh
37:59
that is running very tight Loops Computing linear algebra.
38:03
We got to make them really, really fast, right?
38:05
And use less energy because it's only going to grow given where
38:08
machine learning is going. So make more signal from their
38:10
point of view and then making more sensible from a, from a
38:13
human effort, point of view just accelerates Innovation, right?
38:16
So then you want to have human human brains were too precious.
38:19
Things to focus about the problems they want to solve and
38:21
lies about, you know, how they can actually going to put in
38:24
production. And then on the accessible side,
38:27
remember, making machine learning sustainable and making.
38:30
It is the same one accessible to do.
38:32
Good, right? So, on the accessible site
38:34
accessible to as many people as possible, that means that you
38:37
have just more people creating Nations that are compelling and
38:41
solve problems and relevant to the world.
38:43
And then excessive was inaccessible as many devices as
38:45
possible because, you know, by making enabling folks to run a I
38:50
wear it should be run, can enable new applications and more
38:56
Innovation as well. So, you know, from it'll
38:59
believer that, you know, a, I can create very, very compelling
39:04
applications that create new Industries.
39:07
But also now on the personal, I have a passion for biology, and
39:11
I've always had a passion for biology.
39:13
Octo ml octave here comes from octopus and amazing organisms.
39:17
That rewrite, their RNA is a lot of interesting parallels with
39:19
just in time, code generation, and so on.
39:22
And I've done a bunch of research in the past and
39:23
molecular biology and Engineering molecular biology
39:25
for computing purposes. And reason mentioning this, we
39:29
seems like, it's not like crazy talk, right?
39:31
Well, now, there is an issue. This is that fundamentally?
39:35
I don't think that without a, i machine learning, you can make
39:38
sense of Or engineer with Biology.
39:41
And I feel that's just the next wave of human evolution is being
39:44
able to manipulate biomolecules and engineer them and understand
39:48
them to make life better to create new types of things that
39:52
make life better, right? So.
39:54
And it's so complex that it's beyond our ability to use first
39:57
principles engineering. So we have to use AI machine
39:59
learning, but the levels of performance that he needed a
40:04
level of accessibility to run very, very confidential.
40:08
Any models depends on. You can infrastructure, very
40:11
efficient and sustainable. So now you go deep into how I
40:14
think of it like by creating this platform and maybe more
40:16
people to go and build is very sophisticated models and run
40:19
them in a viable way. We're going to enable folks are
40:23
going to engineer biology and they send biology much better.
40:25
So that's so cool to think about it.
40:28
And you see the Netflix documentary about the octopus
40:33
ice yet? My teacher octopus.
40:35
Yeah. Absolutely.
40:35
It's a great know if you It's super anxious.
40:39
You should you should if you are the audience is interested in
40:42
that. So look at how octopus rewrites
40:46
the fundament. That one of the fundamental
40:48
reasons why they have such complex cognitive processes is
40:52
because they be right, there are n a like this L don't normally
40:57
you know this is wrong with that is its DNA to RNA to proteins,
41:01
right? Messenger, native proteins.
41:03
So most of the time you you change the proteins that you
41:07
that you express which Whatever happens in your in your
41:11
metabolism, right by just regulatory process that block
41:15
and unblock parts of your DNA. But in an octopus's, what they
41:18
do to actually get it from DNA to RNA to go and rewrites.
41:21
The RNA molecule itself before it gets translated to proteins,
41:25
which is very, very unique. It's very similar to, you know,
41:29
a just-in-time compiler, that is its run is generating code that
41:32
then you're going to go and run, right?
41:33
So let's click very geeky aside here.
41:37
So, so, So here's the killer question.
41:42
Do you eat octopus? Oh boy.
41:46
You know, I used to eat a lot more octopus.
41:48
Now, every time I consider it, I think about how intelligent they
41:52
are and it's hard. It's getting harder and harder.
41:54
Do you know, especially know.
42:14
They are. I must admit, it's, it's hard
42:18
for me to opted to eat octopus today even though I'm not
42:20
vegetarian, you know, so yeah, I can imagine.
42:23
But anyway, we must comment and then I wanted to ask which is
42:31
why I really appreciate the fact that you're calling out.
42:34
This idea of making things more sustainable with machine
42:37
learning because I feel like it always gets overlooked when you
42:40
have like the new large language model that comes out via.
42:44
Amount of resources that in took to train that and that it took
42:49
to actually like create that, nobody talks about the effect on
42:54
the environment of that. Yeah, that's right.
42:57
You know, I'm glad you brought that up because there's a lot to
43:00
talk about there. You can probably have a whole
43:01
episode on this, you know, a whole session on that.
43:03
But I want to comment on a few things.
43:06
First time, it's important to realize that a lot of the people
43:09
talk about today is the resources goes to training
43:12
models, right? So, but for every very For
43:15
models that he trained. There is successful, you're
43:17
going to do a lot of inference that you do more ties, the cost
43:19
of training to a very large. That means that it's okay to
43:22
pay, you know, of signal amount of computation early on.
43:25
If you can amortize it over, you know, a large number of
43:28
instances. But if it's also has to be
43:29
cheap, remember because any dominates the dominates where
43:33
the Cycles go, right? So, and for any popular model
43:36
does useful things, guess what you're going to do?
43:38
Inference trillions of time. So you got to think about that.
43:40
So people don't think about this in general.
43:43
The second thing is, I hope I'm not going to sound like an
43:46
activist here, but I think that a little, it's important to make
43:50
these huge computational jobs to train models to be a success as
43:55
accessible customer as many people as possible in all be
43:58
behind, you know, huge pay walls, right?
44:00
So, so here's what I end when you turn a language model.
44:04
So here's here's a quick thought experiment, right?
44:06
So, if you train a language model for English, right?
44:10
So, and everyone who speaks English would benefit from this
44:14
but then, You know everyone speaks English, also kind of on
44:16
the English language so we shouldn't be good if it one who
44:18
speaks English could use this model without having to pay for
44:21
it, right? So this why I'm glad that people
44:23
are more and more during this open and free language models.
44:27
But it's that's only the beginning because now for you to
44:30
be able to make use of these models computation has to be
44:32
cheap enough, right? So and that's where I think
44:36
making it very, very officious, not just about reducing the cost
44:39
in beautiful in the first place, but also using the cost of
44:41
people, to even use it, right? So, And it's great to see all
44:46
the progress in, you know, in foundational models, where you
44:50
can train a large model and then use it and specialized for
44:53
different use cases and great to see both, you know, commercial
44:56
and open source approaches to that.
44:58
So I just want to go back to something that we talked about
45:01
previously, which I'm really glad.
45:03
We actually we touched on, which was your octopus.
45:06
You have these this, these dualities on the one hand,
45:08
you're like a hardware software compilation ml Guru, on the
45:11
other hand, you have this very interesting.
45:14
Of research interest in things like biology and molecular
45:18
biology. And I mean, this has, it seems
45:20
like that's very different ramifications, right?
45:22
I mean, when I spent some time, looking into what that meant in
45:25
the context of your research, it was Concepts around using DNA as
45:29
a form of storage and kind of the high information density of
45:32
fundamentally, you know, biological mechanisms, right?
45:36
And and things like that. So, I'm curious.
45:38
How do you tell me more how you bridge that Gap from something,
45:41
which is more Hardware software, you know, ultramel which Is
45:44
clearly more just focused on these types of concerns and how
45:48
that eventually will evolve or can tie into what may be the a
45:52
longer term goal is around using biology and interesting ways for
45:55
computation and things like that.
45:58
Oh yeah, no, even got the goose bumps.
45:59
His thanks for asking that question.
46:00
So I, well, like if I, if I zoom out the way I see, it is just
46:04
about time scales, right? So, if ignoring ignoring Oxford
46:09
mail and TVM, and even the specific use of DNA data storage
46:12
as in a research on that, Even I can talk about competing as
46:16
well, but what I'm interested in fundamental is, how do we build
46:20
better Computing machines, you know, that our mortgage
46:23
energy-efficient that enables you to, you know, really reach
46:26
the next level of things that you can do with machines, to
46:29
make, you know, human life better and imprudent.
46:31
Let's say that the next wave of, of human evolution even right?
46:36
Because humans design machines and make you my life's better,
46:38
the least, but, so how do you make computers better?
46:41
Well, as a computer architect, you have to look at you know,
46:44
History of silicon based Electronics that's about to
46:47
reach its limits. So, you have to think about
46:49
Alternatives, and there's many Alternatives forward, including
46:52
different types of electronics, like 35, materials,
46:55
superconducting logic, Quantum, Computing and molecular based
46:59
approaches, right? So, and, you know, the way I
47:03
think about this is a computer architect is that, you know,
47:06
we're going to have a mix of these things.
47:07
I don't think Electronics is ever going to go away because
47:10
it's hard, engineer bow but is we change limits fundamentally.
47:12
So if you're going to go going to think about the next, Wave of
47:15
evolution with Computing machines, you have to think
47:18
about alternative compute methods in alternative data,
47:20
storage methods and so on. So, and then now how do I reach
47:26
these two things? Well, AI as I said before a, i
47:29
machine, learnings is so fundamentally.
47:30
How you think about Computing? The means you have to build good
47:33
machines for it, right? So, and also, you can use that
47:36
to to design the next wave of machine.
47:38
So that's the bridge here, right?
47:40
So, using computers that exist today, we create tools and
47:44
Enables us to do research and explore new ways to do
47:47
computation and DNA data storage is one example, right?
47:50
So the motivation for DNA data storage early on was we did this
47:57
really long absolutely awesome and research project between UW
48:02
and Microsoft's are about 77 years ago or so asking the
48:06
question of. Okay, so if you could look at
48:08
the data grows any ability to store data, there's a growing
48:10
Gap here and even in the vast assumptions of storage, Vice and
48:15
ability to store data for all of the mainstream storage devices
48:18
that gap, which to grow. So, you need to find, you know,
48:20
a more first of all viable approach and second, the
48:24
sustainable approach to it, we arrived at DNA because you know,
48:28
it's fundamentally a couple few atoms per bit, right?
48:31
So has been hardened by Nature. So it's durable it's dense and
48:36
you know it's always going to be relevant because it's not going
48:39
to go up so late, you're always going to care about reading DNA
48:42
because it's relevant to to to life, right?
48:45
So then when we started looking at this and know, how do you
48:49
encode digital dating to DNA and writing to the molecules and
48:52
reading it back? There's machine learning
48:53
everywhere, like our encoder. How to map B.
48:57
Two nucleotides is a learned based approach when you read DNA
49:01
back, it's a giant, you know, really, really messy, you know,
49:06
set of signals that have to go and decode back into basis and
49:08
that back to this, that use a ton of machine learning.
49:11
And all of that actually, is so big that the ability to run that
49:14
He was helping our ability to the research that like, let's go
49:16
to make it faster. So, in fact, we use TVM and with
49:20
a lot of always done. We used to actually make those
49:23
algorithms faster but the crazy loop there is that then what we
49:27
did was, okay. So let's see if we can compute
49:30
some machine learning Colonels directly in chemical form.
49:34
So there is a paper that we put out last year, doing similarity
49:38
search like you know, k-nearest neighbors like algorithms a
49:41
molecular form and that's how we went full circle.
49:44
I will be demonstrating how to do we actually day in biology,
49:47
demonstrating the lab, including a bunch of images in DNA and
49:50
showing how you can actually perform similarity, search
49:55
indirectly in DNA forms. So, wow, that does that.
50:00
That like crazy, you might have been a crazy, but it's all about
50:02
time scales, right? So, making computers faster
50:04
today and thinking about how we can make computers faster and
50:07
better, you know, for the next wave.
50:09
So, yeah, because I mean, there's all that sounds
50:13
astounding. I would love to read this paper.
50:14
Earth when the you just you just described the one question that
50:18
I have though. I mean, just for all the good
50:22
properties of the biological kind of substrate that you're
50:26
using namely DNA. The one question I have is how,
50:30
you know, the question of program program programmability
50:33
right, which is something that you described even Electronics
50:36
has the big advantage over other other sort of mechanisms for
50:38
doing this. How does that, you know, how
50:41
does that play into? Because when I imagined
50:43
something, when I think of something I DNA, I think of like
50:45
wet lab work, not as easy to program or manual labor kind of
50:48
involved there. How do we bridge that that part
50:51
of the problem machine learning and automation Yeah, it says I'm
50:58
actually being being serious here at, you know, it's really
51:01
hard to design any molecular systems, just with first
51:04
principles, the way you design, you know, a mechanical system or
51:07
even Electronics, or even a Computing system today.
51:10
So, you have to use, which is not first principles, but, you
51:14
know, explore what you need and then do a search process to
51:17
arrive at a at a design that does what you need to do.
51:20
So that fundamental is machine learning and then how you
51:24
evaluate and Getting the lab is by using automation wet lab
51:29
robotics to do experiments as well, because we just use humans
51:32
to go to the white line. You just don't get the
51:34
throughput that you needs to validate the designs, fast
51:37
enough and turn the design cycle, right?
51:40
So it's interesting is no different than, you know, even
51:44
think about how Beauty machine learning models today.
51:46
It's an iterative process that the faster turnaround the better
51:49
and more information, it gives the more arrive at a, at a
51:53
design. So there's another parallel
51:54
they're creating machine learning models.
51:56
Designing molecular systems today is very much, you know,
51:59
process of search finding a good one, evaluating learning and in
52:02
trying the next one, right? So I think that that's great.
52:06
I mean, the need for the future where I could maybe know, 10 15
52:10
years. So now, you guys are talking
52:11
about the wet lab Ops, you know, how do you, how do you design
52:14
better molecular systems and create?
52:17
That's, that's, that's good. That's going to be the sister
52:22
and you're going to say no, that doesn't exist.
52:26
You would have come full circle, right?
52:27
So we're gonna we're gonna play that clip.
52:30
We're gonna, we're gonna actually reject the clip you
52:32
like, remember when you said that and know that, that was
52:36
fun. Yeah, so, anyway, well, this
52:39
this is the us when we talked about a number of different
52:42
topics here. They're all, I think comes back
52:43
to ability to create better machine learning models and
52:47
deploy them and make good use of them, right?
52:48
So I want to make sure they're actually touching some of the
52:51
key things that you all wanted to touch on.
52:52
So you're so I don't distract you too much here.
52:54
So yeah, but once One other maybe one question that I had
52:59
because you know you are a professor also like you know in
53:02
your background I'm curious. There's been certainly this this
53:06
wave of Enlighten the last few years of people from Academia
53:10
going into industry, either getting poached by big companies
53:13
or starting companies. Can you talk a little bit about
53:16
people for what that transition, you know, being a Serial
53:19
entrepreneur, in your case from Academia into industry and you
53:23
know, what are the skills that translate or how is it
53:25
different? Friends and how to serve, think
53:27
about that transition? Yeah, I know this.
53:31
It's a pretty big question. So try, I'll offer some thoughts
53:34
here, right, first. You know, there's, there's a lot
53:38
to learn in beauty Beauty and business, right?
53:40
So, you know, it's say that in Academia, your focus, the end,
53:44
the say, the output of the work is a fantastic student is going
53:47
to go and change the world but also papers are going to change
53:50
how people think and this is very much about how do you
53:55
create Wait, some surprise going to make coffee.
53:58
That's going to motivate people to think differently, right?
54:00
So, where as an industry, we would need to do to build a
54:03
successful business, is build a product that people are going to
54:05
pay money for and actually make a successful, right?
54:07
So some of that is similar, which is about, okay?
54:11
So you need to have some differentiation sometimes it's a
54:14
piece of technology. Normally, coming from Academia,
54:16
you have some different issues technology.
54:18
Sometimes it's a set of people that think very differently, you
54:21
know how to do things better than anyone else, right?
54:23
Sometimes just a market opportunity that you Smoother,
54:26
and you have to move as fast as you can, right?
54:28
So that's what station for me from, you know, Academia to now,
54:33
I miss them essentially, you know, my signal, I'm full-time.
54:36
A talk to him about being full-time, in October for the
54:38
past three years or so, still spend a little time in Academia
54:41
because it's just part of how who I am and lots of a job.
54:45
The, the things that, you know, I realized that I was very, very
54:50
different is for you. To be a product organization.
54:53
It's all about turning any If anything I would say most
54:57
successful businesses today, even if they started from a
55:00
technology Foundation eventually has to become product
55:03
businesses, you know, because that's what people pay money
55:05
for, right? So, and that each active process
55:08
of finding design your product. And so on, I don't think it's
55:11
similar to how we do research at all.
55:14
Right? So and you have to come to, you
55:16
have to contain understanding that.
55:17
Hey, that's that's how that's how business is for.
55:20
Can being very diligent about the project and not lat, you
55:23
know, the shininess of Technology distract you from the
55:26
fact that you doing a probably people should be money for is
55:28
something that academics we do need to think hard about right?
55:31
So but now to your earlier point of industry and I don't feel use
55:36
the word poaching but I'd say no luring academics to do think
55:41
it's a system for motivation for that, right?
55:43
So All of them, just go to do research and industry.
55:45
It's great to see, you know, companies that have built very
55:49
successful businesses investing in future research, like,
55:51
historically a lot of good stuff happened.
55:53
That way, look at that, you know, you know, I'm Bell Labs
55:56
deck deck, research labs, and IBM research, Microsoft
55:59
research, you know. And now, Google like there's
56:02
this is, this is naturally, it's great, right?
56:04
So, I think that has to be done, right, right?
56:06
Because if you get all of the, you know, the faculty that are
56:11
very productive, doing research with your students into When
56:13
they stop making students, then wait, so who are you going to
56:17
hire with phds five ten years from now, right?
56:20
So I think industry is now finding its balance and being a
56:22
lot more flexible. In letting you know, people
56:25
straddle like, no be part of not getting your part in industry
56:28
and keep both going. I think it's coming from
56:31
realization that. If you don't do this, you don't
56:32
keep producing PhD students. You know, you're not going to
56:35
have folks too high in the future that level of, you know,
56:40
at that level of depth. So, that's, that's one point.
56:44
But now, when, when industry hire faculty would say to do to
56:48
do product often is because, you know, it's in the high-tech
56:51
industry. Something that, you know, you
56:52
beauty products around, you know, technologies that are
56:55
pushing the state of the art. And that's what I particularly
56:57
find. Exciting.
56:58
I don't think definitely not the only way to build companies,
57:01
obviously. But if it's what I think an
57:03
appeal to a lot of folks that let's say, especially older
57:06
faculty like me that have done have had their impact in writing
57:11
papers and graduating students. I see building a company.
57:13
Any out of research that starting Academia is the third
57:16
form of impact, right? So, it is this some sorts no
57:19
answer to your question but more thoughts, you know?
57:20
So what do you say to the people that feel like Academia is out
57:26
of touch when it comes to the work that they're doing and the
57:31
impact that they could be having on industry?
57:34
It's like almost like there's two parallel worlds there and
57:37
I've heard some people say, well it's because Academia isn't
57:40
exactly Earth Academia doesn't have.
57:43
Of full range on all of the data that the industry has.
57:49
So how can they be making all this impact?
57:53
It's a great question. I think ties back a bit to tell
57:56
me how asked as well, so the way I tie, this one question first,
58:01
is that some faculty, get really attracted to music by the way
58:03
because you have access to things for research that you
58:06
wouldn't have in Academia often. It's the data set.
58:10
This is through used to be through in life sciences now,
58:13
it's being Through in computer science, like you have access to
58:16
how the world do search, for example, you know, there's no
58:18
way to have access that in Academia, right?
58:19
So, now, actually being out of touch, I would say that I can
58:26
see where the sentiment comes from, right?
58:27
Because often Academia try to think about a world that might
58:29
not be how the world is and solve problems in that imaginary
58:32
world. You know in some cases actually
58:35
leads to great ideas that this come back right?
58:38
In some cases it doesn't what worries me is.
58:40
When Academia This research In, you know, with intention of
58:46
having packed in Industry when it's out of touch with industry.
58:50
So that's the thing, that's the worrisome.
58:52
I do not mind when you think about here is a let's say a more
58:56
long Horizon Research. That's not directly motivated by
58:58
an industry problem. With motivated by fundamental
59:00
technology Trends. That's good.
59:02
He might be seen as out of touch with industry today but
59:04
eventually in this is going to catch up because it's a
59:06
fundamental technology Trend, but now if you look at them make
59:08
research and you try to motivate an industry and their motivation
59:11
is wrong, that's bad, there's some research like that.
59:13
And so those are the words that we should call out and then, you
59:16
know, hopefully, you know, they'll come to grips with that,
59:18
right? So, does that make sense?
59:20
So I and it's interesting to see to I would say a lot of
59:27
technology companies being more and more open about, you know,
59:31
opening up data, sets opening up infrastructure for Academia to
59:34
use. This is all ways of actually
59:37
avoiding scenarios where Academia has to be speculate too
59:40
much in what industries actually doing and all the Everything is
59:44
starting to pay off, right? So yeah, yeah, so quick
59:49
rapid-fire for you before we finish this I want to ask you
59:56
what the last book you read was Whoa, but what was it again?
1:00:03
Oh boys, in the boat. Oh yeah, dude, you know, dude,
1:00:07
you know about this book right now so it's about the I think
1:00:12
1936 Olympics, the Rowing Club here and there is of Washington
1:00:17
dead, end up, winning the Olympics, being completely
1:00:19
Underdog, and then winning. So the stories like winning
1:00:22
Against All Odds fun but the whole story about it.
1:00:25
So please repeat of history. In this one of us, history is
1:00:28
part of that too. You know, for example, there was
1:00:29
a massive, Sandstorm that happening, the US had no idea,
1:00:33
there was surprising, but the key thing there is About how you
1:00:37
know for you to win that for successful crew, team to
1:00:41
actually make it, you have to have these these Rovers like be
1:00:45
completely in sync with their bodies and how they think.
1:00:47
And so like, is it going? It's all about parallel
1:00:49
Computing in the end, but we so you think, if you don't think
1:00:52
this perfect synchrony in lockstep perfectly perfectly,
1:00:57
you know, synchronized you don't you don't make it right?
1:01:00
So in every single ounce of efficiency matters there, And I
1:01:06
think there's an interesting parallel here with how
1:01:08
eventually machine learning models running computers that
1:01:10
everything seems like just perfectly right for it to be
1:01:13
efficient, you know? And when I was reading about the
1:01:18
level of that thing and refinement, that is team went to
1:01:23
to reach maximum Peak efficiency in rowing.
1:01:26
First of all, it's an incredible story of human achievement
1:01:29
honestly but I also putting like a pure science hats like oh my
1:01:34
God, held his buddies at that level.
1:01:35
To see every single thing matter and they synchronize.
1:01:38
The perfect degree made me think of how computers work to self.
1:01:41
That's incredible. I love how you bring the
1:01:43
parallels back to computers. And so what is a company in your
1:01:49
in Washington, right? Washington State.
1:01:51
What is it company or startup that you've seen?
1:01:54
That has impressed you lately. One of his Runway ml.
1:02:00
Mmm. Yep, we had Brandon on here, a
1:02:03
few months ago, who is their lead ml engineer?
1:02:07
They are they're killing it. That's it's soaked the video
1:02:10
editing them out. That's, I think, you know that
1:02:13
one, that is really fantastic. So here's, so here's another
1:02:17
one, a alpha by, oh, so just to bridge computer science and and
1:02:22
biotech with you're doing is using a mix of old saying,
1:02:26
Synthetic. Biology protocol.
1:02:28
The problem you want to solve is that what is the protein-protein
1:02:31
interaction? Like how do you because it's
1:02:33
important for antibody. Development is important for,
1:02:36
you know, diagnostic throughout development.
1:02:38
So, on and they had this machine learning + synthetic biology
1:02:42
approach to use East through a lot of work to produce proteins,
1:02:46
and then measure how they stick, or don'ts and read that with DNA
1:02:50
sequencing and use a lot of machine learning to produce a
1:02:53
data sets. That informs the folks designing
1:02:55
vaccines and Design. Antibodies and so on, that's
1:02:58
really impressive with data and, you know, and putting in
1:03:00
bringing together synthetic biologist machine learning,
1:03:03
folks, and folks doing what lab Automation and so on.
1:03:06
Wow, yeah. All right, last one for you,
1:03:08
Louise, how do you want to be remembered?
1:03:11
Wow, that's wow, that's a good question.
1:03:18
I think in bringing the right people together, so buting
1:03:23
teams, I guess. Using, yeah, and it doesn't
1:03:28
matter in what context I feel like, you know, it's impossible
1:03:31
to do anything big without having a team.
1:03:34
Nobody can do big things alone, right?
1:03:35
So, and I guess being somewhat, remember, it's not that I have
1:03:38
aspirations of having a legacy of not like, but I was like, if
1:03:41
people give me a, somebody was able to bring the right people
1:03:43
together. I'll be very happy so I like
1:03:47
that, I got goosebumps. Well, thank you so much, Lisa.
1:03:50
This is super cool. Come on, jeez on.
1:03:53
She's moving to Obrigado. The more two emotional people
1:03:58
saving for our podcast, it total been proposed a Optimum.
1:04:05
Thank you very much guys. Awesome chat.
1:04:07
Hope to meet you in person and you know coaches on this
1:04:09
conversation is in a pub sometimes so yeah, absolutely.
1:04:14
We're starting a local meet up in Seattle so you are maybe I'll
1:04:19
have to drop by and absolutely I'm you don't have to talk.
1:04:23
I love. I love to join and bring some
1:04:25
other. CO2 and help help spread the
1:04:27
word here. So there we go.
1:04:29
Awesome.