Latest / Agentic Conversations (formally mlops.community) / Intelligence & MLOps // Karl Fezer // MLOps Podcast # 148

Transcript

Intelligence & MLOps // Karl Fezer // MLOps Podcast # 148 · Agentic Conversations (formally mlops.community)

0:00
I am Karl fazer. I'm an AI developer evangelist
0:04
at Lockheed Martin and I been I've got a new like coffee
0:10
grinder, that grinds it right before it drips coffee for me.
0:14
So, I'm really lazy in the morning but I take it like black
0:17
with a little bit of unsweetened non-dairy creamer.
0:22
What's up everybody? Welcome back to another ml.
0:24
Ops Community podcast, I am your host Demetrios and I'm here with
0:28
Abby. What's going on, Abby?
0:30
Interesting conversation today. Yeah, this one really win very
0:36
philosophical for a moment and I love talking about, that reminds
0:39
me of my college dorm room, smoking, a bunch of weed, and
0:43
philosophizing on the meaning of life, These are topics.
0:46
I think most of us have discussed about.
0:49
I think we started to talk about imparted cognition vices we
0:53
talked about how to define intelligence, which I think at
0:58
this point time is an important. Relation to have again after the
1:02
superintelligence book. And I think with all the work in
1:06
Foundation, I mentioned, Rita, Potter's, I think it is a really
1:10
good time to have that conversation once again.
1:12
Yes. So for anyone that has not read
1:15
it, check out his paper all about defining intelligence.
1:20
I feel like he's we are very Kindred Spirits when it comes to
1:23
Carl. He was talking about how he
1:26
wrote that paper, because this idea of artificial intelligence
1:29
Agents was starting to gain traction again.
1:31
If there are so many different people that are calling it, so
1:35
many different things or have so much expectation around it.
1:40
And it's really not clear what we mean by intelligence and he
1:45
broke it down in the paper, he broke it down in the
1:47
conversation today and then we went on a few philosophical
1:51
tangents but we brought it back around because at the end of the
1:54
day, he's doing a lot of ml Ops stuff and especially inside of
1:58
his company. He's a van Liza ng/ml Ops and he
2:03
had some really cool thoughts on where we're going when it comes
2:07
to the future of these large language models with respect to
2:11
ml. Ops the takeaway that I loved
2:14
was that he was saying look we've been able to just make
2:20
people more efficient for the last since Computing began.
2:24
That's really been what we've do been doing.
2:27
I think we might as well just get into the conversation.
2:29
What do you Gabby, subscribe to our newsletters, get into a
2:33
slack and keep listening to the podcast.
2:37
Give us glowing reviews. We love to read reviews.
2:40
We've received one or two recently comments on our
2:48
YouTube. Yes, people comment on the fact
2:52
that I should never play musical instruments again in my life.
2:56
So if that is, you listening telling me to not play musical
3:00
instruments, thank you. We love those.
3:01
Kind of comments and reviews. Also keep them coming.
3:06
It's awesome to hear it. Alright, let's get into it.
3:14
So I bought it started very starting because basically
3:17
you've been in the center of the A high communities for so long.
3:21
And we are served in the process of trying to preserve on it.
3:25
So I want to get your tips and learn from you.
3:30
You lost season one of the arm developer broadcast.
3:33
What do you think is the key to a successful podcast?
3:39
That's a good question. I actually don't know if I'd
3:44
call that a successful podcast, especially compared to this one
3:48
like it was something our marketing team is like hey we
3:53
should do a podcast. I was like great, I'll do a
3:54
podcast because I'd always wanted to do one.
3:57
I was getting into like music production so I had all this
4:00
stuff and I was like I'll do the soundtrack will do everything
4:03
myself. If so it was mostly like a
4:05
production learning experience, but in It's really just
4:08
something I really like doing because we had this startup like
4:11
ecosystem since I was working arm like we had our startup
4:15
Partners who focus on ML on the edge and I was like, Hey it'd be
4:18
great to chat about all of them because they each did like
4:20
something very specific and that's kind of what their
4:22
startup focused on like a gin poles.
4:24
For example, we talked about transfer learning but there is a
4:27
lot of commonality between managing a community and
4:30
managing a podcast because it's like regular Cadence.
4:34
And this is mostly like what I've read in theoretical and And
4:38
some of the things that make them both work.
4:41
Well, are like a community to follow them and the podcast
4:47
axis, like this kind of nice regular Cadence metronome that
4:52
they're like, oh, I shouldn't basically these things that
4:54
remind people to stay involved and stay in touch, like ml
4:58
office is still pretty new. And like, they're like, more and
5:03
more people want to be learning about it.
5:04
So building this kind of culture of like a Since, and things.
5:09
And that's kind of what gets people to stick around.
5:12
It's like, you bring them in with the podcast and they stick
5:14
around and the same things work with, like, developer relations
5:17
anywhere. It's like, you do an event and
5:19
then you're like, hey, stay with us, because we'll give you beta
5:23
access or like free Hardware, or credits.
5:27
Yeah, GPU credits, like that kind of stuff.
5:30
And I 100% agree with you on that.
5:33
It also helps people feel like they know you and they get to
5:37
know like what this community is about through the community
5:39
Through the podcast and they get to know like what the vibe of
5:42
the community is also in a way. And so I want to talk a little
5:47
bit about this open-source toolkit that you have for AI
5:50
that you've have played around with.
5:53
But first, the big question that's going through my head
5:56
right now and has been since I jumped on this call with you is
6:00
which of these like five instruments behind you do play
6:03
the most Oh the longest is probably the guitar, the guitar
6:10
back from high schools that a Gibson Epiphone.
6:13
So it's the it's like the white label Gibson.
6:16
It's a good eye. I was like it was high school
6:19
and I was like I shouldn't spend money on a Gibson.
6:21
All buttoned up at five and this one I've actually like The
6:26
internal like pick up. It's like an acoustic electric
6:30
the internal pickup died like 10 years ago so I ended up cutting
6:34
a hole and putting a new one in and like filling it in with
6:36
like, wood putty. So it's kind of weird
6:39
Frankenstein guitar at this point.
6:40
Next time that happens. I'm just going to put in like a
6:43
weird synth like pick up and just run it into pure mini
6:48
Earth. All right, so tell tell me about
6:51
this. Open source toolkit.
6:54
Oh yeah. So currently I'm at Lockheed
6:57
Martin as an internal developer advocate so I don't do anything
7:01
external anymore which part of me kind of Misses because a lot
7:04
of it was like I love doing events.
7:06
It's like a party for all my favorite developer friends but
7:10
now it's internal so it's like it's got a plus has its pros and
7:15
cons but it is I mean it is kind of like this controlled box and
7:18
Lockheed Martin's a big Company that I'm never going to get
7:21
bored. And it means like it's easier to
7:23
stay in touch with people because we're all on the same
7:26
slack and they can't leave because they work with me so I
7:29
can I think can't hide them. You can't ride?
7:32
Yeah. Yes, I do a lot of work with
7:36
like a sister team called the AI Factory, and what they do is
7:39
focus on basically in for Ops and Emma lamps and they manage A
7:44
bunch of servers and software stat mostly open source stuff
7:48
like Cube close. Probably, the biggest thing,
7:50
underlying everything that we work on and kind of and then our
7:55
their job is to manage it and kind of keep it deployed and
7:58
managed access and kind of make it as easy as possible for our
8:02
machine learning Engineers to hop on that.
8:05
So it's, the onboarding ramp is really as easy as possible and
8:09
then as an advocate I'm adjacent to them to try to help get more
8:13
people. To use it.
8:14
And also teach them how so you're just going around inside
8:18
the company. And I have a ginger dealing with
8:21
machine, learning engineers and potentially, Sometimes some data
8:23
Engineers or data scientists, and you're telling them.
8:26
Look, this is the best thing that we've got right now.
8:30
You don't have to worry about getting anybody from procurement
8:34
involved on this new tool that you want to buy.
8:37
And adding it, see how we can hook it up to what we've got
8:41
already. And so if you want the Easy 021
8:45
for your project, use the platform that we Supply.
8:50
Yeah, 100%. There is some other stuff in
8:53
other parts of walking Martin like that are like paid for
8:55
tools. And so, we're kind of some of
8:58
those. We also try to support, but
9:00
we're also trying like the AI factories, relatively new
9:03
compared to those words like internal Hardware offering
9:08
managed by internal people running pretty much all open
9:11
source tools miles, kind of the best of all worlds.
9:17
And yeah, I try to get them on. Also, a lot of times it's not
9:21
machine learning Engineers or like data science specifically,
9:24
sometimes we do like upskilling with like McCaleb engineers and
9:28
like, hey, I should know about artificial intelligence and
9:31
machine learning because there are applications and sometimes
9:35
those teams work together. So that's also a lot of fun.
9:38
It's people who were kind of sometimes even new to
9:41
programming, just getting started with machine learning,
9:43
so, Dude, that is so cool. So speaking about evangelizing,
9:48
are there things that you would say are effective ways of
9:53
evangelizing within a company like if I wanted to go around in
9:58
my company and try and get people on the same page as me to
10:02
use this cool platform that I have.
10:05
What are ways of doing that? I've heard some of our guests
10:08
talk about like Lee and Fitzpatrick.
10:11
She comes to mind. She's like, head of data at the
10:15
financial times and she spoke about this push-pull and really
10:19
it was for her. You can't push people into using
10:23
anything, you have to kind of entice them and show them the
10:28
benefits without forcing the benefits on them.
10:31
Do you have evangelizing? Internal evangelism tricks that
10:35
you use? Yeah, so internal evangelism is
10:40
very much like external evangelism and the Biggest like,
10:46
preventative for people, learning a new skill or
10:49
technology. It's still like the time and
10:51
having to learn something new and like, you still have to
10:54
convince people that it's worth their time, through various
10:58
reasons and like, cost is normally not a big reason, he's
11:02
is probably one of the best performances like less of a good
11:06
reason. Because most of the time, if
11:09
you're learning, you're like I don't care about performance.
11:11
I just want something to work and to be easy.
11:14
So yeah it's kind of any can't push people into it and people
11:18
especially internally if you're like sorry we're not using this
11:22
talk technology anymore. We're shifting over here.
11:24
They still get very mad when you kick them off their stuff so
11:28
like making that he's as easy to transition as possible like
11:33
transparency, like you said like whole reason you do a podcast or
11:37
people can get to know you like you got they have to trust you
11:41
and evangelism. I talked to Rabbi and Africa.
11:45
Once and she like she we really had good conversations around
11:49
evangelism. I'm like, yeah, it's basically
11:51
the same thing. It's like you're being very like
11:55
you're supposed to be a source of trust and you're trying to
11:58
convince people to do something and they're like, I don't know
12:00
if it's worth it, I'm not buying into it so far.
12:04
So like it's trust and you kind of just show them your process.
12:10
One thing I started recently and we're playing around with his
12:13
like Colonel office hours, but since it's all internal, we're
12:17
like well you're just going to watch me build something.
12:20
And when it breaks, you're going to want to serve it.
12:22
Try to fix it and so far it's like an experiment.
12:26
But it's like, yeah, show me, you're a person and things are
12:29
hard, and it's still learned. We're all learning stuff.
12:32
I like to make things a little bit funny.
12:35
I like I I was like an improv kid in high school so I really
12:42
like doing that. But I always build funny silly
12:46
weird projects like trick voice-activated trick or treat
12:49
dispensers or I built a dog snorts.
12:55
Detection thing really early in the pandemic.
12:58
When I was like I need some reliable source of data and my
13:02
fiance is a French Bulldog. So it always just sit at my feet
13:04
and just snore, like a chainsaw is like that's that's a great I
13:08
can tell when the dogs asleep or not and it's like silly stupid
13:13
problems but I Use like real techniques to like detect that
13:17
you can use the same thing to Tech chainsaws in the would
13:19
probably actually the same model.
13:21
Okay, let's get into your paper. Yeah, let's talk about this
13:25
because I know that when we reached out to you and we wanted
13:29
to get you on here, you thought we want to get you on here
13:32
because of the Lockheed Martin, but really, I was more
13:35
interested like you came into my ecosystem through your paper, so
13:39
maybe you can give us a bit of context on What you wrote and
13:44
why you wrote it, I think it was early in the pandemic, I was at
13:48
arm and this was with a co-worker, Andrew Schloss who'd
13:51
been an arm, he wouldn't he been around for at arm like 20 years?
13:56
So he was one of the people who'd been around there, the
13:58
longest. So it's backgrounds, like
14:01
definitely more heavy on the engineering.
14:02
And we both just started, clicking around like philosophy
14:05
and artificial intelligence. And I was on the marketing side,
14:09
I like the hype cycle of artificial intelligence, and
14:13
people Talking about it and just kind of like this disposable
14:16
term got us really. I wouldn't say upset but we were
14:21
very focused on. Let's define what people mean
14:25
when they say artificial intelligence and by that, let's
14:28
define what people mean when they say intelligence because
14:32
that's just a huge. It's a big, I mean, it's really
14:36
a philosophical question. Still in terms of what is
14:40
intelligence, so we kind of went into the different ways.
14:42
We know how to measure it. Um and then different like
14:46
scientists and philosophers Through The Years who've, you
14:49
know, peeked into like cognitive science and things like that and
14:52
cognitive philosophy. And how can we ground?
14:55
Some of the things we say and put a metric to it?
14:57
In terms of? What do we mean when we say
15:00
intelligence and what are the attributes that Define
15:02
intelligence? And so we kind of dived into
15:04
like two different aspects like there's problem-solving, there's
15:07
communication, there's like free thinking there's like Like
15:14
abstract thought is 1. So all those kind of building
15:19
together into this nebulous blob that we call intelligence and
15:22
then throw it at like machine learning products.
15:26
Let's break it down. Yes, eating three different
15:29
terminologies used. One is intelligence and I know
15:33
tearing when he wrote his paper he was like oh machines are
15:36
going to be intelligent and this is my definition of what makes
15:40
something intelligent. The second is what we call
15:43
artificial. Should intelligence and that's
15:45
also sort of murky based on like the 1958 meetings and then the
15:49
1990 works and all of that and the other is a new term.
15:53
That would be hearing which is basically machine intelligence.
15:56
That is served trying to create a more defined definition.
16:01
What do you think is the difference between them which
16:03
intelligence do you think? Are we closer to and what's the
16:08
one intelligence that we absolutely will never be able to
16:11
touch? Oh, that's a, I don't think I
16:13
can say. Anything about never being able
16:15
to touch because I think anybody involved with artificial
16:18
intelligence or machine learning like the constant goal was, you
16:22
know, mimicry of human intelligence.
16:24
The question is like when or how and like, will I see it?
16:28
So I can't say never and four terms of some things.
16:32
So the first one was Alan turing's definition of
16:37
intelligence and that was mostly if I remember, right?
16:42
It was kind of, if you can't tell, The difference between
16:44
something that's intelligent and this machine, then it is
16:48
intelligent. So it's like this intelligence
16:50
by kind of assumption or proxy and that one, I don't like
16:57
because it's too philosophical. And it's too much in the eye of
17:02
the like Observer and human are, like, humans are not great
17:09
observers of reality. It also slides into like a
17:13
little bit. Of salope is MM.
17:15
Sobs ISM solves ISM where I don't I don't believe anybody
17:19
else on this planet is intelligent and you can't prove
17:23
me wrong. So and like that's also the flip
17:26
side of that, op observable intelligence, which is
17:29
important. And I think that's an important
17:33
characteristic, but it's also like you end up in the uncanny
17:35
valley of like it seems intelligent until it gets a
17:40
little bit too intelligent. And then you're like, oh, that's
17:42
definitely a computer. It's too good at math.
17:44
At and then it slides up a little bit it's like animation
17:46
uncanny valley. So it's a harder thing to like
17:50
lock down just because there's all this other things going on
17:53
with it. And it's it's not focused on
17:55
like A specific task or capability of the unit.
18:01
In question, it's more about the Observer and like, what they
18:04
think seems intelligent and that one's weird.
18:10
Then there's the like, the 1960s and 1990s.
18:13
And, like, the following AI Winters, those are I think those
18:18
are, like, I think those and the weird thing is, the one in the
18:22
60s, like that basically laid the groundwork for, where we are
18:26
now. Now, because all they just kind
18:27
of theorized a bunch of techniques and eventually
18:31
computers caught up. So we could actually do it at
18:34
the eye that that's, that's kind of, I don't know if it's kind of
18:37
like the science of philosophy, kind of thing or like, really.
18:42
I think it kind of, more of a marketing thing because they're
18:45
like, hey, we can do the, it's almost science fiction, but they
18:48
grounded it on like biomimicry, kind of things like neural
18:52
networks and genetic algorithms. And I think Big fuzzy logic came
18:57
out of there, but basically a lot of the concepts we use now
19:01
and actually, no can be used for machine learning came out of
19:05
those ideas. Then there's also like
19:08
knowledge-based systems Everyone, my name is aparna
19:15
founder of arise. And the best way to stay
19:17
up-to-date with ML Ops is, by subscribing to this podcast,
19:27
dude. Well, there's some interesting
19:29
you said here where it's talking about how its biomimicry and
19:34
that's fascinating to me. Because I just think about how
19:37
little we know about our own brains and the our Own minds,
19:41
really, in how things work. There is like the brain side of
19:44
things that matter. But then there's that mind that
19:48
each one of us is inside our own heads and there are so many
19:53
questions there that are not answered and I feel like are
19:57
very far from being a turd and so on one hand you can do this
20:02
biomimicry where it is like taking some of this scientific
20:09
idea. And then just putting more
20:12
compute behind it or saying, right?
20:15
We're going to, we're going to try and do this but in a way
20:18
that has more power. But I think about the idea of
20:23
like, oh are we just in our minds?
20:28
Is it a radio station that we're plugging into and our thoughts
20:33
are not actually coming from us and so that's like some
20:37
philosophical shit right there, right?
20:40
And then, You've got and so then if you think about that and what
20:44
are the implications on machine learning and until artificial
20:47
intelligence, if that is the case.
20:49
And and then I also think about just the idea of how many of us
20:56
are Like mental health is a huge thing these days, right?
21:02
And that mental health is so big and so many people have their
21:08
battles with mental health. Makes me think that we know so
21:12
much less than we think we do about the mind and then us
21:17
expecting that. We're going to be able to do
21:20
anything with artificial intelligence makes me a little
21:25
bit, like, that's A tall order. Yeah, I 100% agree with all of
21:33
that. There's a lot of things that are
21:36
like our own. We understand the basic like,
21:39
Concepts and building blocks of the brain which is like neural
21:42
networks. And like G, PT 3 is, let's just
21:46
take billions of those and throw it all the information in the
21:50
unit like it at all. Actually, like the Wikipedia and
21:53
see what we get and there is kind of this.
21:55
It is a definitely a Brute Force approach and like a million.
22:00
Million Champions. He's on typewriters can now make
22:03
Shakespeare like we proved that Sofia but of what we know about
22:08
it and like, explain ability, like it's also like quickly
22:11
Beyond getting beyond our scope where we're building these
22:14
things that are too big almost to understand and that's why I
22:17
explain abilities coming into play.
22:19
So at least, we can try to understand that things that were
22:21
building, if not our own minds. You also like the thing about I
22:28
always feel like the more we learn about the brain, the less
22:31
I believe in free will because it's like, oh, yeah.
22:34
And there's this, it's your history and your like nature
22:38
combined together to and your environment to kind of shape
22:42
like how you view the world and like you're like, that's kind of
22:45
cognitive psychology is based on your own thought, patterns shape
22:50
how you exist in the world and I mean that's kind of why I mental
22:54
health exist in like more less do drug therapy do more talk
22:57
therapy. E, let's try to change the way
22:59
we think about things or like, you know, take deep breath and
23:02
live. You don't understand any of
23:03
this. And it's, we do have this
23:05
ridiculous amount of control over our own mental state.
23:09
If like, at least someone was to, I don't know about like It's
23:13
Tricky. It takes work.
23:14
It's like you could spend your whole life just working on
23:17
trying to be a good human. Let alone trying to build a good
23:20
fake one. If so like it's a full-time job.
23:25
Just existing almost in a lot of cases I think.
23:28
Are called, like gurus and like the Dalai Lama, like, basically,
23:31
all the images like, I'm just going to be a good human.
23:34
Yeah, I'm going to work on myself and that is a full-time
23:37
job. Yeah, yeah.
23:39
It's just an interesting you bring up dualism like there's
23:44
like religious connection aside. I I've always liked the idea of
23:50
dualism because it's it allows for the unexplainable.
23:54
And I think that's probably a better approach to things
23:57
complicated. Plex like intelligence in
23:59
science. It's like we're going to focus
24:02
on the things. We do know and allow this huge
24:04
amount of like we're just not going to be able to understand
24:08
it whether it's like could be quantum entanglement who knows?
24:11
Like maybe we are like brains living in a jar on another
24:15
planet and these are just our Meat Puppets that we walk around
24:18
with and and then we're going to wake up in the jars and they be
24:20
like well I was wrong the whole time.
24:25
But like, you know, it's there's so many things that we do know
24:28
and then every time we learn something new, it just opens up
24:30
a whole new thing of these are all the things now we don't.
24:34
So it is really interesting and I do like dualism and
24:37
philosophy. I was like oh dualism school, it
24:39
gives you room for Consciousness and the brain and like the
24:42
content does Consciousness, affect the brain or is it from
24:45
something else, because it adds like extra Unobservable things
24:51
that I guess they're entertaining for philosophers.
24:53
Yes, you can never prove them wrong.
24:57
Yeah. Like unicorns exist in
24:59
epistemology. I think the biggest thing I took
25:01
away from like studying it was you remember things because
25:04
they're relevant and Salient like see aliens.
25:07
Like you can understand it where, you know, things because
25:11
they're relevant and say way it and you remember them because
25:14
they're relevant. So salience is, you can
25:17
understand it because you're never going to Really remember
25:20
or know something if you don't understand it first.
25:25
So I think that comprehension is kind of this Mist Park if we're
25:29
trying to like mimic human intelligence and then also
25:33
relevant. So like relevance is mostly just
25:37
a filter to filter out all the unwanted stuff that we don't
25:40
care about and with machine learning and like deep learning
25:43
specifically, we have to build that kind of relevance model for
25:48
it by By like bounding like that's how it works.
25:51
It's like bounding boxes and color Court coding.
25:54
Like, look this stuff, this Parts the relevant part, ignore
25:57
everything else. I think the Salient is the,
26:02
like, the understanding of the world is still not really there
26:07
in a lot of cases, like most of the time when I see machine
26:10
learning and production, it's it's like adds a sensing tool.
26:15
We're like in terms of like On the way to being like a fully
26:20
intelligent creature, you've got to like sensory environment and
26:23
then you have to understand it and act.
26:26
And a lot of times, you can get around this, especially in like,
26:30
reinforcement learning with simulated environments where you
26:32
can actually just kind of encode everything and you don't worry
26:35
so much about the sensing part and you focus on the
26:39
interaction. The understanding is still a big
26:42
question. I think in those cases, it's
26:46
like II think it's just a question General, like how it
26:50
and a lot of times it's more correlation is not causation.
26:54
And is that really understanding or did they take the wrong
26:56
lesson away? So I think we have to I really
27:03
like reinforcement learning specifically because it's kind
27:06
of building this model of the world and I think like
27:12
theoretically it, see that quite happened yet and it does It's
27:19
very dependent on the engine, like the reward function and
27:24
like that. And like every time I see like
27:26
chatty pt3 do they've got like a reinforcement learning function
27:31
based on like, oh, the person liked my answer.
27:35
This must be the right thing to say.
27:36
I'm like, that's exactly how you you go for the wrong way, with,
27:41
like machine learning like, oh, they really liked it, it must be
27:45
the right thing and like, that's not, that's not how we should do
27:48
it. Should be.
27:48
More. A like we need an empirical
27:51
better, empirical gaze and I think the question is, what is
27:55
that? And then once we have that, like
27:58
entropy is probably a good one in like Bayesian probability and
28:02
entropy of the universe and like understanding the environment.
28:04
Like there are mathematical things that kind of do that,
28:08
which is Interesting. But like I don't see them a lot
28:14
in practice and I think without looking at those things in terms
28:18
of like focusing on our understanding, we're not going
28:21
to get good learning results. Like we're very much like focus
28:26
on this focus on this focus on this.
28:28
All you didn't learn the thing. I yelled at you about for like,
28:31
3,000 epochs like, what's wrong with you and instead, like it's
28:35
more and like, like watching babies, screw up and children.
28:41
Like they spend the first like, arguably decades just wandering
28:45
around the world and like trying to understand it but like,
28:48
still, look at everything. Yeah.
28:50
I mean, I'm still doing that. But like, I think there's this
28:55
this lack of understanding in terms of In general and I see a
29:00
lot of things there. I was reading recently about
29:03
like there's been this slow push of like doing simulation first
29:08
and then moving in, like those like algorithms into robotics
29:13
second. We're like we have this world.
29:16
That is easier to understand. We're not worried about weird
29:18
sensing techniques. Just think we sort of have
29:21
mismatched expectation of where artificial intelligence is
29:25
supposed to be. Is it supposed to be accurate or
29:28
is it? Used to model the human behavior
29:33
to model the human behavior. Then you have that leeway to
29:36
give the wrong answers and still be super confident to buy at not
29:40
learn a thing because they're plenty of suffering.
29:42
People who feel like I'm Kurt. Yeah, they learn from us.
29:45
That's search. Yeah, I think that's the problem
29:49
really is like we're trying to mimic human behavior.
29:51
My questions like, should we were bad at some stuff?
29:55
Um, and like, computers are very good at things, I think.
29:59
I think there's been this focus on it also like the real world.
30:03
We it's still really hard to teach computers how to live in
30:07
the real world, like autonomous driving.
30:09
It's kind of been around for a while now, but it's really
30:13
tricky and if it goes wrong pretty bad.
30:15
So it's like it's very careful things.
30:21
There's also like the other side is like oh autonomous vehicles
30:25
should just have cameras because humans just Just have eyes and
30:30
that is good enough for us. I was like, I don't know if it
30:32
is and if you can put on like white are and all kinds of other
30:35
things on a car, why don't you? Um, like there's things machines
30:41
are good at like Matt, the things our people are good at
30:45
and I think the thing is, is the success is like, at least in
30:51
terms of expectations for now, is figuring out where that line
30:54
Falls and there is like this, it constantly keeps coming.
30:59
Yup. Like the singularity and like oh
31:01
artificial general intelligence is right around the corner and I
31:04
think I was just reading yesterday's.
31:05
It's like 20 30. It's going to be here and I'm
31:08
like I don't know if it's gonna be that soon and also are you
31:13
shooting for the wrong thing? I think that's why I like the
31:18
new, the new push into like, hybrid to intelligence.
31:23
So like you have a human in the loop or especially with like
31:26
machine learning development. At least you have a human in the
31:29
root Loop and say, this is wrong.
31:32
Like, Darwin AI is really good at this.
31:35
I think this is kind of their technique, is they're not
31:38
presenting information to like a lay person.
31:41
They're presenting information to like a doctor who, then
31:45
they're kind of augmenting. Being a specialist and helping
31:50
them in terms of scaling their time.
31:52
And you know, what is it like paying attention basically
31:57
because people get tired and we stopped paying attention after
31:59
so long. But computers.
32:01
Don't. They're effectively like they
32:03
always like this. Yeah, they're always on and
32:05
they're basically making the same kind of decision any time
32:08
and part of that is like making sure they make the right
32:10
decision. But if you put a human in the
32:11
light Loop and augment this intelligent, I think that is a
32:14
much better focus than Replacing humans or mimicking humans.
32:20
I think it augmenting humans is probably the right approach and
32:24
better so coming back to the idea of the infrastructure
32:30
around machine learning and especially when you're looking
32:34
at this new form of machine learning, do you see?
32:38
I mean, I know you mentioned the open-source toolkit that you
32:41
guys are using relies heavily on Cube flow.
32:44
Is it in your mind? If you were to think about two
32:51
years or five years out, are there going to be completely new
32:57
tools that were using? Because there are completely new
33:01
ways of doing AI or is what we're using right now on the
33:06
like the developer tool side, or the ml offside, going to morph
33:12
into something new. And so you'll still be using
33:15
Cube flow. It's just going to be the next
33:18
version of cube flow, takes into account more, foundational
33:22
models and more data and whatever.
33:25
More of the I guess data is not the word I was looking for but I
33:30
think you understand what I'm trying to get at right?
33:33
Yeah. Um I think I what I like seeing
33:41
is A slow progression and like, I don't think you flows going to
33:48
go anywhere. I also like two or four years
33:52
ago said, like, oh hi torch is definitely going to replace
33:55
tensor flew High torches, so much easier to have but there's
33:58
a lot of other things at play, in terms of like out with the
34:01
old and in with the new kind of mentality and like tents flows,
34:04
got a little huge amount of support behind it and like we
34:08
still, I still use it and because like, and it works with
34:11
things. Like, whatever I want to do with
34:14
tensorflow and cute flow. Now, it kind of works with and I
34:18
think that's, its kind of, and like so we were like the way we
34:23
used to do it, like, way back in the day, like, my first project
34:27
was like hand coding warming. That that was one way to do, I
34:32
think, I believe I've never done it myself, but there was a
34:35
neural network class for the, like, you can do it yourself.
34:37
Just in math. Equations and code neural
34:39
networks. Like no.
34:40
Yeah, that's awful. Nobody wants to do that.
34:42
The other other one is like hand programming, like Mike does
34:46
neurons or Yeah, and just making them run through and then,
34:52
without a framework, and the next came framework.
34:54
So it's like, he's, it's the same Principle as, and just
34:56
getting easier and easier and easier to do.
35:01
Let the machine do, the machine Focus part where it's like,
35:03
yeah, I want to do this five thousand times and you give me
35:05
the best result and that's what like distributed training is,
35:08
and that's gotten easier. So the tools are shaping up to
35:12
make it easier for the people who know what they want to do to
35:16
do what they want to do as quickly as possible.
35:17
Like This is my goal. I want you to recognize this and
35:24
we don't even we barely even need to worry about like what
35:27
kind of layer model were using. Like there's been so much work
35:30
and development into that in terms of what works and what
35:33
doesn't. Now we can kind of, there's
35:35
almost a shelf of like here start here, you're going to
35:37
transfer learning, do hyper parameter, tuning across the
35:40
stack and give me the best model and like those things didn't
35:43
exist before and it's all built on the same stuff.
35:46
There's just more options, Q flow And like pipelines.
35:49
It's like you're getting into that better.
35:52
Like let's take all this Dev Ops stuff, you got to change your
35:55
code, you gotta update stuff. Let's put that into like machine
35:59
learning approach. So that again is like it's it's
36:02
really putting machine learning in production with this like
36:06
robust set of tools that software Engineers know all
36:08
about. And you know, deep learning
36:11
researchers are like I don't Source control.
36:13
What are you talking about? I just want to see you recognize
36:16
the cat and I win. So, like, it's these, it's this
36:20
good bit of practice. I think the things I see still
36:23
are like Edge. Deployments are still hard, it's
36:26
really due to, like, cloud-based stuff's getting super easy.
36:30
But like, if I want to get this thing here, you still have to
36:34
figure out those connections. So, like those things will get
36:37
easier like multiple models running it, like orchestration
36:43
of multiple models, it's going to get easier.
36:44
So it's, I think it's just going to be building layers, and
36:47
layers and layers. And so this is fascinating
36:50
because Abby and I were just talking on Tuesday to Alex who
36:55
wrote a book on dynamodb and his whole thing that he mentioned
37:00
was that these days databases have been able to use some of
37:06
the cloud Primitives and make things so much easier because
37:11
they're leveraging these Cloud Primitives in your eyes.
37:14
It's almost like that. Same idea is going to be just
37:20
extrapolated out more. It's like the parts that we
37:24
recognize give us more advantages are just going to get
37:30
easier and better and faster and everything that you would want,
37:34
right? And then from there, we're going
37:38
to see More exploration or ideas going from someone's mind to
37:47
reality much quicker. And so that time I think some
37:52
guys that doordash I'm doing all kinds of name-dropping right
37:55
now. Just forgive me for that but we
37:57
talked to some guys it's great Jordache, they were mentioning
38:00
that they called it the velocity of ml or basically for them.
38:05
It's how long does it take to go from idea to production and so
38:11
You're just seeing and if I'm getting this correct, it's that
38:16
you're taking everywhere that we can get gains on.
38:23
The efficiency of this are just going to be maxed out more than
38:28
we can imagine. Yeah, 100% And doordash is
38:32
actually really good name to drop because their routing stuff
38:34
is very good. Well, I think I was reading some
38:38
of the things they publish. I was like, oh, this is amazing.
38:41
You can Get me a sandwich and teach me about machine learning.
38:44
It's just perfect, they are, they're great team, shout out to
38:48
hey and who's behind their machine learning platform.
38:50
But so that's a fascinating one because then it's not like
38:54
you're disrupting. I mean, one thing that I keep
38:58
going back to is there has been an explosion of ml Ops tools in
39:02
the last three to five years and these ml Ops tools are they
39:10
going to Be around in another five years.
39:15
That's like my big question. Yeah that's um Probably not all
39:20
of them. There's probably going to be
39:22
some like coalescing and submerging in like the big
39:26
companies are probably going to lie mean like H, ve just got
39:29
Pachyderm. Right.
39:30
Yeah. So like it's going to be things
39:32
like that where they kind of end up clustering.
39:35
And you end up with less options and like the same thing
39:37
happened, like, you know, ten years ago Frameworks.
39:40
It's just whenever there's a new opportunity, there's someone has
39:45
in their head, like this is the best way to do it and for better
39:48
wear. Sometimes I Right.
39:49
And sometimes they're wrong. So like they're probably they're
39:52
not all going to be around but they're all probably going to.
39:55
We're all going to learn something from it.
39:57
And at the end we hopefully have like the best of all of that and
40:01
like, the best proven methods because a lot of times like,
40:06
startups and companies are a lot like experimentation anyway and
40:11
it's just like, yeah, they're going to try their path and this
40:14
other company is going to try their Pat and whoever wins is
40:18
like the Tway and win but either way, we learn.
40:23
Yeah. And when you're playing around
40:24
with projects and definitely like open source, tools, are you
40:28
thinking about that a lot like as you're looking into what's
40:32
out there on the market? How does that play into your
40:36
like personal projects or just the infrastructure that you feel
40:43
like you're going to invest time into?
40:44
Because going back to the beginning of the conversation
40:46
when you're trying to evangelize?
40:49
A certain tool or a certain product to people at your job
40:53
and the biggest friction that you're getting from people is
40:56
like why should I put in the time to learn this?
40:59
How can you justify to yourself that this is a good use of my
41:03
time? And how do you choose which ones
41:06
are? Yeah.
41:08
It's I think one thing I got in the habit of doing and haven't
41:12
done lately is just like every week, trying new tool and just
41:16
try to evaluate as quickly as possible like at the same time
41:19
you're not going to learn. Oh, well, well that fast but you
41:23
can at least Like, I talk to Saudi once at like a developer
41:27
relations conference in the like you got about 20 minutes for
41:31
someone to convince someone to learn to use your stuff.
41:34
And I'm like, that sounds about, right?
41:36
It's like about as fast as getting a pizza and you're like,
41:40
if you're not into it in like 20 minutes, then you're going to go
41:44
back to your old ways. Like you've got a really went
41:47
building, it a like a developer Focus tool.
41:51
You've got to make it really Li like sing and like work, right
41:57
out the gate or else someone you're going to lose, someone's
41:59
time. The me the best tools have like
42:05
this dis. What's it's like a design
42:07
thinking approach. There's a freight.
42:10
It's like an iterative deepening of complexity.
42:13
So like it's a very shallow at first and my small way and stuff
42:17
done. And like you give the small wins
42:19
but like as you learn more, you understand how much you can
42:23
actually get out of it. Um, and how deep it can be.
42:27
I didn't like Karis is a really good example, because it was
42:30
like, hey, let's make tensorflow easier and like that.
42:32
It was great. And then, it's just, if you want
42:35
to do more, you just go into the tensorflow part.
42:39
And I like, that's to me. Those are the best tools where
42:43
they start off, really simple. But your, if you want to dig
42:46
into it, you can. And like those to me have the
42:49
most promise where it's like a lot of things, especially like,
42:54
Now, I see a lot of no code or low code stuff and they're like,
42:58
oh yeah, we're going to make it great for you in the back end,
43:00
and I'm like, but what if I want to do something that you didn't
43:03
set up for me, it's like you prepared a buffet and you're
43:07
like, I want something off the menu and they're like, no sorry
43:09
we don't do that. So like those things might not
43:12
stick around as long but like the stuff with like the
43:15
foundation, those those probably stick around longer and like I'm
43:20
over built on open source stuff. So completely, we had a great
43:24
Great conversation or meet up over the summer.
43:28
And the presenter was saying, yeah, the best part about using
43:32
vertex AI is that it's managed service and then configure like
43:37
and the worst part about using vertex AI is it's a manager
43:41
vintage service. Yeah.
43:42
So that's a good gotta. Yeah.
43:45
You gotta have your trade-offs there.
43:46
I think the thing is, is like, and you kind of mention this
43:50
before, it's Regardless, if machine learning is getting open
43:55
up to new and new more and more people, and that's kind of the
43:58
goal of all these tools and like it's meant to, like, augment
44:03
stuff that you want to be doing and I think actually like Adobe
44:08
has been doing a lot of really, I don't use it.
44:10
But like if I were an artist or something like they're doing
44:13
tons of cool stuff, too, like, hey, let's use machine learning
44:16
to help you do the stuff you want to be doing if you're a
44:19
photographer or if you're making Music or there's a lot more
44:24
writing tools out. Like I used to separate podcast
44:27
tool. Yeah, they're hot actually wear
44:30
them. So I think that's the kind of
44:33
stuff. Like, the more we see it in
44:36
non-engineers hands, the better it is for us because it's all
44:40
like building and building and building and yeah, eventually
44:45
you'll be able to just Build me a thing to recognize a cat and
44:50
let me know when it's angry. So like we're pretty close to
44:54
that. She's but it in like it's it's
44:58
kind of whose problems are you solving and how big are they?
45:01
And that's really the focus of all these tools.
45:06
So, what do you think is the one thing?
45:09
Would it be the product? Would it be their penetration in
45:14
the community itself? Would it be very good?
45:17
Understanding of your customers are building on top of work
45:20
would be the one keeping factor for the companies that make it
45:25
in the next five years in the envelope space.
45:31
That's a good question. I think they're all important,
45:33
but I do think it's like, sometimes it's you win by
45:38
getting a big customer and then you can afford to keep doing
45:41
stuff. I do think it's to me
45:43
personally, it's like a developer relations per person.
45:45
I have to say, it's the product like it, like to me.
45:49
That's what draws me in. That's not who always gets to
45:53
make the decision. I do think Community is
45:56
important because without Community, you don't know your
45:58
customer or Or the customer can't get help, or they feel
46:04
like they're doing it on their own, which is like, I don't
46:06
know, like human existentialism but like, that's what the
46:09
community is for. So like sometimes there are
46:12
products that I think it really is a balance between all three,
46:16
but for me, it's got to be like the product.
46:19
This excellent. Dude, this has been awesome.
46:22
I did not expect the conversation to go in so many
46:25
directions that it went in. Its, it's really cool to get to
46:29
talk to you about philosophy about the meaning of
46:34
intelligence and then wrap it all up with a little ml Ops and
46:39
where we're going in the next couple of years.
46:41
So this has been an absolute pleasure to talk to you.
46:45
Carl, thank you so much man. Yeah, awesome.
46:48
No thank you. This has been a lot of fun.
46:51
This is Skylar, I lead machine learning at Health rhythms, if
46:55
you want to stay on top of everything happening in ml.
46:57
Ops, subscribe to this podcast now.
47:02
Now, now, now now now Now.