Latest / Agentic Conversations (formally mlops.community) / Updated The Evolution of ML Infrastructure // Sakib Dadi // MLOps Podcast #140

Transcript

Updated The Evolution of ML Infrastructure // Sakib Dadi // MLOps Podcast #140 · Agentic Conversations (formally mlops.community)

0:00
So I'm sake of Dottie on the vice president with Bessemer
0:03
Venture partners and if I'm not making a cup of Joe and I can
0:07
access at home, I'm having a filtered soul from Phil's
0:10
cotton. What's up?
0:12
Everyone I am your host feature as Brinkman and we are back for
0:16
another edition of BML Ops Community podcast.
0:20
Today we have the pleasure of talking with an investor who's
0:25
been invested in devtools for quite a bit.
0:29
You You will hear a little bit different tone for me today
0:33
because I almost wanted to play Devil's Advocate.
0:36
I know that in the investing World, it feels that web 3 is
0:43
out and generative, Ai and foundational models are in.
0:48
So I took the stance asking saqib what he thought about
0:53
that, and almost pushing out the idea that ml Ops as a Sector to
1:00
invest in is dead, and this is coming from conversations that
1:07
I've been having with people who are trying to invest or
1:11
different VCS and what they're looking at.
1:15
I thought it was fascinating how he pushed back on that complete
1:19
idea and he said, you know what, it's not dead.
1:22
So let's get into the full conversation.
1:25
But before we do, let me give you a bit of background and bio
1:28
on crew succubus. Keep is the vice president at
1:32
the San Francisco offices of esenberg quite a famous venture
1:37
capitalist firm. He primarily focuses on early
1:41
stage investments in developer platforms data infrastructure,
1:45
and machine learning. He's been involved with
1:47
bessemer's investments in prefect.
1:51
T' Cole, I'd Periscope data, which was acquired by census,
1:57
DM, which was acquired by GitHub and launch Darkly before,
2:03
joining Bessemer sake, worked in product at Via.
2:08
Go go and international marketplace for buying and
2:11
selling tickets for Live Events. And I also want to give a huge
2:17
shout out to our sponsors Wallaroo.
2:20
While the root is a For designed to be your control room for
2:25
production machine, learning to facilitate deployment
2:28
management, observability monitoring and optimization of
2:32
models in a production environment.
2:35
While the veru caters to a i-team's small or large working
2:40
in an integrated fashion, in your data environment via sdks
2:45
UI or an API, you can install Wallaroo on your choice of
2:51
eight. U.s. as your gcp or on-premise
2:53
and deploy your models and pipeline to your environment in
2:57
seconds. Once in production, you can keep
3:00
your models running in a production environment and up to
3:03
date through capabilities, such as hot model switching A/B,
3:07
Testing Shadow deployment, anomaly detection, and model
3:11
monitoring. So get your hands on and grow
3:16
your skills by downloading and installing the free community
3:19
edition of Wallaroo. I'm going through the Hands-On
3:22
tutorials that they've got and we will leave the link to
3:26
everything you need to know in the description below.
3:29
There is also a community slack channel that you can check out
3:33
and you can get help and share your projects and ideas with
3:37
like-minded, folks will leave a link to that.
3:40
Also, in the description, check out, Wallaroo.
3:43
And thank you folks, for sponsoring this episode.
3:52
Let's just start with how you even got it to investment.
3:55
Let's go right to the beginning and you started with ML
4:01
investing even like when it wasn't even a thing, right?
4:05
So you've got quite the track record.
4:08
How did it all? Conspire it's it was very much
4:12
an accident. Actually I'd never thought I was
4:14
going to end up in investing it really actually happened.
4:18
Gosh. Call it six almost seven years
4:20
ago. Now, at this point I was an
4:21
engineer by training study Material Science which was very
4:25
researchy as well as also business and I knew I wanted to
4:28
do something at the intersection of both.
4:30
And at first, I thought that meant doing something in startup
4:32
world and I tried a couple things out over the years but
4:36
wasn't really enticed by anything.
4:37
And as I was graduating from school and my graduate program,
4:42
I just know, I have no idea what I want to do, but you know, let
4:45
me go explore and check a couple things out and it just so
4:48
happened that Customer actually recruited on campus.
4:50
No, no for investors. Yeah.
4:52
And I just wandered into an information session and like I
4:55
was like, yeah, this sounds actually really interesting
4:57
because to talk to really smart folks.
4:58
Every single day. We're starting cool,
5:01
interesting. New businesses around technology
5:03
and get to learn about new spaces and that's this.
5:06
How I guess got sucked into this role.
5:08
And it sort of naturally found my way into a couple of
5:10
different areas some of which touched materials.
5:13
But also some that were more technical on the software side
5:15
and particular just gave him my background.
5:17
And so that's sort of how like Shouldn't a machine learning
5:19
infrastructure, investing in the machine learning space came
5:22
about. So you have seen a lot of
5:25
companies pchew what are the ones that you regret not
5:30
investing in Oh yeah, what's my anti portfolio?
5:35
Yes. I mean the one that sticks out
5:37
right now because every machine learning practitioner, I talk to
5:41
talks about it is hugging face. I mean I think hugging faces
5:45
build, such an incredible community and product.
5:48
The ecosystem in many different ways and we're stage agnostic
5:51
investors. So I hope that we can correct
5:53
our mistake at some point in the future.
5:55
But for now, for sure it's the one that very often, I think
5:58
about is climate at come to share the story with us.
6:01
I don't think we fully appreciate She added how big
6:03
ultimately think face was going to become as part of this
6:06
ecosystem and how transformative it really would be cop weight.
6:09
And which round did you pass on? This was the everyone.
6:14
Yeah. Yeah.
6:15
Pretty much the, a is, the one that sticks out in my mind in
6:18
particular. Yeah.
6:20
Because there was all the numbers were right.
6:22
Then there was a lot of growth and I think I did not fully
6:26
comprehend. How big transformers were going
6:29
to be and how much of a standard it was becoming machinery.
6:32
Ecosystem and it truly has been, right?
6:35
And I think a lot of the growth for I'll call it machine
6:39
learning and it's 2013, 2014 deep learning imagenet, kind of
6:43
days to today has come about because of the standardization
6:47
around Transformers architecture.
6:48
And a lot of people pushing in that one direction, as you
6:50
generally, see with technology shifts, but I like it takes some
6:53
amount of standardization to say, hey, we're all
6:56
consolidating around. This one thing and all of our
6:58
work is going to be pushing forward.
7:00
This one, this one sort of Of the world or perspective on the
7:05
world, rather than fighting over like competing different
7:08
standards, almost like Blu-ray and HD DVD, right?
7:11
That's where the maybe more modern canonical example,
7:14
relative to AC and DC power. But I think that's sort of
7:18
applies in this case, as well too.
7:20
So Transformers are today. What do you feel like is
7:23
tomorrow? Because I know a lot of your
7:25
thoughts and basically, your brain power centers around, what
7:29
is the next thing? Do you feel like 20?
7:32
T to basically 2022. 2022 has been Transformers, does it have
7:37
a long Runway and it's going to be until 2030?
7:40
Or is there something else that you feel like will replace this
7:43
eventually? It certainly feels like it has a
7:46
very long Runway from my vantage point right now from talking
7:49
with folks from my perspective in speaking with practitioners,
7:54
a lot of it is how do we make use of this?
7:56
Like this is really interesting technology and we don't have the
7:59
talent and sometimes the time necessarily to make use of all
8:03
these interesting machine learning models that are coming
8:06
out of research other technology companies.
8:08
And so how do we take those models and apply it for our use
8:11
cases and help manage and scale its infrastructure to serve our
8:15
customers better? So it is starting to become a
8:18
lot more perhaps something that you might have perceived many
8:21
years ago. As you started the podcast in
8:23
the community about the machine learning operations of it all.
8:25
When actual production izing these models?
8:28
Yeah, completely. So that is the regret.
8:33
What is the company that you wish was around?
8:36
But you haven't seen it yet. The company that I wish was
8:40
around, but haven't seen it yet, it's funny because I when I was
8:43
a working product at a portfolio company of ours, company called
8:47
view, go go the ticket Marketplace space and they
8:49
bought StubHub it turned out to be more of like a data science
8:51
rule honestly. And there were a lot of
8:54
companies that I wish existed to help me facilitate by roll back
8:57
then. And in all honesty actually
9:00
think quite a bit of them have been founded today, there's a
9:03
lot we built our own in-house data, labeling product.
9:05
We built our own experiments framework as well too and I can
9:08
name a bunch a bunch Of companies in both of those areas
9:11
in particular, and across the whole lifecycle.
9:14
So I wouldn't say there's a company that I wish had been
9:18
founded. I feel like there's actually a
9:19
lot of folks working on these problems, and he actively trying
9:22
to solve them today. And whether it be in the
9:24
labeling side, experimentation side or even though the
9:27
monitoring site as well to, there is a lot of activity going
9:30
on, on all parts of the spectrum ecosystem.
9:34
And so give me your current It's on the envelopes market and how
9:42
things look through your eyes because I find it fascinating
9:46
actually, I just saw the funniest meme today about
9:49
someone who was at a soccer game, or a football game, and
9:54
they had on one of the team's jerseys.
9:57
It was the yellow team and then they, the yellow team was
10:02
getting scored on and they were getting beat.
10:04
So the person took off that Jersey and boom, it was the red
10:08
team's Jersey. And the caption around.
10:11
It said, like VC's changing from crypto to foundational bottoms.
10:17
Yeah. Like creative AI, or generative
10:19
AI. I know exactly which one you're
10:22
talking about. It's a World Cup as well too.
10:24
Yeah, topical. Yeah, no, I think there's
10:27
definitely a lot of heat right out in machine learning for
10:30
sure, especially if it's a girl around machine learning models.
10:32
Is it on warranted, though? I don't necessarily think so.
10:36
I think it's very warranted. I mean, some of the progress.
10:39
Would be invalidation models is just startled, you Bessemer.
10:42
We've been fortunate to invest in companies like Jasper, the
10:44
eye, for example, Witcher really leveraging, these to great
10:47
effect. And the growth there has been
10:50
astounding because the results are so spectacular, and that's
10:54
in just one use case of marketing and there's a whole
10:58
universe of use cases that haven't been uncovered yet, but
11:01
I think we'll be that being said, this, is it happening in
11:04
the next two years, as all of this Innovation going to happen
11:07
over the next two years. And that's It and it's going to
11:09
be closed. Shut book?
11:11
No, I don't think so. I think this is going to be a
11:13
much longer transition than perhaps a bunch of people
11:17
flooding into the ecosystem right now.
11:19
Might think these things do take time and it's not just about
11:23
hey, we can build these big models, but how do we actually
11:25
make use of them? And how do we enable not?
11:27
Just the Ubers Netflix is B, Dance of the world to really
11:31
productionize machine learning, but even the smaller startups to
11:34
the mid-sized companies to everyone in between.
11:37
Yeah, it's so from my Eyes. It feels like there's been a bit
11:42
of a transition from ml Ops having the hype, especially
11:49
through the lens of an investor. And now it's this generative, a
11:53
eye movement and foundational bottles and I wonder if a do you
11:58
feel like it's similar? Like ml Ops has been around for
12:01
a while now and we haven't necessarily seen this explosion
12:06
or the adoption. That potentially people thought.
12:09
Was going to happen when it comes to the tools.
12:11
Now the practitioners, I definitely think there is a huge
12:14
movement going on in that we can attest to buy how many people
12:18
are in the envelopes Community slack and just how many people
12:21
are doing you know, UPS these days as their day job.
12:25
But when it comes to the tooling adoption, I don't know if it has
12:30
blossomed quite like many had bet on.
12:35
Am I mistaken in thinking that I don't think you're mistaken in
12:40
thinking it hasn't bloomed in the way that the people
12:43
expected. And again, this is perhaps the
12:45
same as what's happening it in generative.
12:48
A I like in 10, 20 x number of years out.
12:51
I struggled to find a world in which we are not.
12:55
Everyone is interfacing with a foundation model in the open
12:59
a.i. codex or whisper or G, PT, 3, 4, whatever series we might
13:05
be on. We will all be interfacing with
13:07
them in some way. That transition doesn't just
13:11
happen. Super quickly, boom, explosion
13:13
overnight, and then over a year or two it happens, these do take
13:15
time, I mean, I think the machine learning tooling space
13:18
in particular, maybe in college 2016, 17, and 2018, certainly
13:22
experienced that sort of hype cycle and is now definitely a
13:25
little bit of the trough of disillusionment over the past
13:28
couple of years. But there is this steady
13:30
adoption that you can see from your own Community as well and
13:32
when I speak with practitioners it's very evident that people or
13:36
companies whether they be those. ER, Ubers and Netflix is to the
13:40
smaller startups are actually starting to adopt these tools in
13:42
open source, or through commercial relationships as
13:45
well. And actually start to utilize
13:47
them to put models into production that benefit their
13:49
operations and benefit their users as well too.
13:51
So we aren't there on the generative space just yet.
13:54
I think we will experience some of that in the machine.
13:56
Learning off space and tolling space there certainly did.
13:59
But now we are seeing some affect the benefits of that
14:02
height. A lot of people came in, but a
14:03
lot of people started building companies around it and some of
14:06
those companies are starting to Garner of you.
14:08
Option. Yeah, what I find fascinating
14:11
about the to because I see them as different markets, although I
14:15
did just this week, see someone another VC posted like the
14:20
generative AI type map or deciphering the market today.
14:26
I yeah. And and the labs was part of the
14:29
generative AI landscape now and I'm like all right let's just
14:32
throw it all in there to why not?
14:34
So yeah, I find that hilarious its species I love you all.
14:40
But sometimes you take real big swings and that was one that I
14:44
was like, come on. Let's be realistic here anyway,
14:48
not just like when I was resting Demetria.
14:50
Yeah, I mean, you got to go out there and you got a gardener
14:56
that hype 2 or try and like position yourself.
14:58
Because I do understand now that you want to get the inflow of
15:03
people that are creating the best companies for tomorrow.
15:08
You ought to be talking to them. And a great way to talk to them
15:12
is by creating that mind maps, I should say, or the market maps
15:17
on the market that beasts. It's funny because a lot of
15:20
folks, I think have similar criticisms of the maps.
15:23
And I don't disagree by the way, like I've published some I've
15:27
participated in putting together Market Maps.
15:29
I think it does help you box out the world of a landscape and
15:33
help you like fully understand it.
15:36
And appreciate though this is where everyone fits is how How
15:38
things fit together but still rarely is like the real world
15:42
actually on a two-dimensional like box.
15:45
It's a lot Parisienne complicated.
15:47
So it's a good first, first try at it perhaps, but certainly is
15:51
thought I would call that like, truth of the world or the
15:54
reality of the world. Yes.
15:56
So one thing know, when I look at the numbers of the generative
16:00
AI space and let's just take Jasper because you mentioned
16:03
them before they have serious Revenue.
16:06
Like I think it's pretty well though.
16:08
Road to most people that they went from zero to 50 million
16:12
plus in 12 months or something like that, which is just a lot
16:17
of money coming in. And then you look at the ml up
16:21
space, I do not know any of the labs company that has had that
16:25
quick of an adoption at a revenue growth may be data
16:30
bricks but they were back in the day and that's like the one
16:35
company that's an outlier that isn't necessarily an envelope
16:38
It's company. So how do you reconcile those
16:43
two worlds? Because I know at the end of the
16:46
day you are trying to get the return on investment.
16:49
Yeah. It's a really interesting point
16:51
and the data Brooks story actually to use that example was
16:56
not so up into the right. So early, I mean the spark
17:00
project got founded. Believe it was 2009. 2010 is
17:04
really when the initial paper was released and I want to say
17:07
it took Until about 2014 2015. 21 you really start.
17:12
If you go back and like start doing the Googling of, oh, when
17:14
did spark really start taking off it, took probably five years
17:17
from that initial paper to like, oh wow, people are really
17:19
talking about this. The open source growth is
17:21
exploding and we see that in a lot of infrastructure software
17:23
companies, whether they be in machine learning or outside of
17:26
it. I mean, Hoshi Corp.
17:27
It was not necessarily A wow. Factor into terraformer launched
17:31
from day one and these were the most important projects vault in
17:34
terraform, excuse me. They started with a completely
17:36
different projects and drag that actually a major.
17:38
A part of the adoption and business Commercial Business of
17:41
Hoshi Corp today. And those projects actually took
17:43
a very long time and many years to grow an adoption.
17:46
Then once they started to turn the commercial screws on it,
17:48
then they the growth accelerated and continues to compound
17:52
dramatically but investing in infrastructure software
17:55
requires, I mean, having to go to a certain extent, a little
17:58
bit of faith that you can build this community build this
18:00
adoption and then monetize that successfully got the road and if
18:04
you're able to go through those first couple steps of getting
18:06
the adoption on, If you can figure out the right business
18:09
model to convert it, then the often accelerate very quickly,
18:13
but they take a lot longer to develop, then what might look
18:17
like a consumer application in some ways, right?
18:19
Like an Instagram, it blows up virally these things.
18:22
Do have some certain amount of viral growth but they just take
18:25
a longer a long time to develop. Yeah, I see that.
18:29
So it's more like the Jasper's of the world that the generative
18:32
AI stuff is in that latter bucket where it does have that.
18:38
That virality right from the get-go, but the infrastructure
18:41
stuff. You're not expecting it to blow
18:45
up overnight and you're not thinking that the revenues are
18:50
going to be high even, two three, four, five years after
18:55
they've their Inception. It totally depends on Project.
18:59
I think the thrust of which is saying is right?
19:00
I mean a lot of Founders in the infrastructure space is funnier
19:03
like, oh yeah, we're doing the big Hacker, News launch we're
19:06
doing for posting on product hunt We're going to do this big
19:09
launch and like when you ask him, what the plan of how to
19:12
develop and expand the community and adoption is long term.
19:16
It's a little bit of a tougher question and that actually is
19:19
the right question for every founder to answer, because it's
19:22
not just that one launch and I'm sure you know, this having done
19:25
the podcast. Now for a while, it's just ready
19:27
drumbeat of pounding, the table for the vision that you have,
19:31
for the product. You have as getting people
19:33
around the table and involved in the community to help push that
19:36
forward. That's not a, just a fun.
19:38
Time launched kind of thing. This is a long burn of getting
19:42
folks, from trying it out to adopting it to them, telling all
19:47
their friends about it as well too because it's transformed
19:49
their work. That's really hard long work to
19:52
end up doing. Yeah.
19:54
That's community building. That's the fun part - yes, it's
19:57
the funnest a very hard part as well too.
20:00
Yeah, I think the hardest part about it is putting out a tweet
20:04
or LinkedIn post to very low Reception, you get one like or
20:10
something and then you or you put out the podcast or whatever,
20:15
to just try and get people into your community and see the way
20:18
that you see things and it just goes out to five people and one
20:23
of them's your mom or somebody like that.
20:27
So I want to talk about two things.
20:31
I want to talk about the intersection because we have
20:34
been mentioning the foundational models.
20:37
And And the ml Ops peace and how you see moving forward
20:42
foundational models and ml Ops playing together.
20:45
Like what does that look like if your eyes as we move forward,
20:48
are we going to start seeing more of these gigantic models
20:53
being used for more use cases? And is it going to become a
20:59
necessary skill? That lots of ml Engineers need
21:03
to now have in their back pocket that they should be thinking
21:06
about deploying. These gigantic models or is it
21:09
something that will still stay fairly Niche and fairly vertical
21:14
for the Jasper? AI type companies?
21:18
I think it's going to become more commonplace and when you
21:21
speak with practitioners in our portfolio owner and the Bessemer
21:25
Network, there's so much interested in utilizing them.
21:28
Yeah. And I think it will be a core
21:32
part of the model selection process for any Enterprise.
21:37
And so, I think it's now just becoming a hoarder will become
21:43
important for everyone to be familiar with them and be able
21:45
to utilize Foundation models. It's upright I think it just
21:48
accentuates the need for machine learning operations, though.
21:51
And 4m, a lot stools. Need a lot of companies, say
21:53
yes. Some of them will be able to
21:55
send their data out, do an opening I, but a lot of them
21:57
still want to leverage open source model like a bloom fine
21:59
tune in for their use case and then apply it within their
22:02
product and manage it themselves.
22:04
So I think that just highlights that there's still going to be a
22:07
big tool a gap or a Big Tool in need people to actually use
22:10
these models that some of your competitors or other companies
22:13
in adjacent areas are using as well.
22:15
To Consumers will just demand and expect that from you, so
22:19
it'll become bored. Hey, everyone, my name is aparna
22:24
founder of arise. And the best way to stay
22:27
up-to-date with ML Ops is, by subscribing to this podcast.
22:34
Yeah, and when you think about differences, now, when you're
22:39
starting to bring these models in house, what are some things
22:42
that you feel like people need to be aware of where people
22:47
should be looking at? Or what are some tools that
22:51
you've been seeing or companies that you've been seeing that are
22:53
trying to address that specific problem of.
22:56
Hey, look the world's moving this way.
22:59
Yeah, everyone's going to be using A foundational bottles.
23:02
And right now the way that things are set up, its hard,
23:06
know there's quite a few companies, I think one company,
23:09
it's not in the best mood for folio, but that I've always
23:11
bought it by our from afar as well to log with hugging face
23:13
has been snorkel, which very much started with data labeling
23:17
and his stool expanded its purview into other areas as well
23:20
too, and made a really big the product announcement and the
23:23
ventilation model, space leveraging them for data
23:26
labeling but also for building bombs and working with
23:29
Foundation models as well too. And I think Type of shift from
23:32
vendors is something that we will expect as well.
23:34
These are one tool in the tool box along with many other tools
23:38
that you might have. And it's great because what
23:43
snorkel is now enabling is, oh, you have a certain amount of
23:47
data will help you label it. Fine tune in for your model of
23:50
this particular use case and now this just opens up the
23:53
floodgates for not just these Google's and Meadows and Ubers
23:57
and the Flux Of The World To leverage machine learning.
24:00
But all the start, Ups and mid-sized companies that maybe
24:03
before you need huge Corpus is of data.
24:05
But now with larger language models and much fewer shot
24:08
approaches on learning on that, you can leverage them to Great
24:11
efficacy as well. And so those types of products
24:13
that may be fit or work within what might be your existing ml
24:18
tooling workflow but help you leverage this.
24:20
New tool in the toolbox I think are really interesting and
24:23
really valuable and are there other companies that you feel
24:29
like or spaces? B.
24:32
That you feel like are going to become outdated unless they
24:37
pivot like. Potentially it's you're saying,
24:40
oh this whatever orchestration tool was very popular back in
24:44
the day when we needed that but maybe there is a space that you
24:48
feel like as we move more towards this reality, that
24:51
you're speaking of we're not going to need as much of certain
24:56
tools. And so they're going to have to
24:58
figure out where they sit in the market as this comes about.
25:02
About. Yeah I mean for the
25:03
orchestration space in particular where investors and
25:05
prefect sin just hearing from some of those customers in that
25:08
team. There's certainly no shortage
25:09
and orchestration so it was just continue to be an issue there.
25:12
Can't think of anything as I look at like my own internal
25:15
Market map in my own brain of what the machine learning
25:18
operation space will look like. There's nothing that I could say
25:22
will become totally outdated, think a lot will continue to
25:25
become more and more important. Now there was certainly a lot of
25:31
proliferation. Tools, maybe call it four, five,
25:34
six years ago or so and there will likely be some
25:37
consolidation their broadly into a few buckets around data,
25:40
preparation model, preparation, validation, and selection.
25:44
And then call it deployment and monitoring and governance and
25:47
those perhaps particular Niche Tools in each of those broader,
25:51
buckets of IQ 2011, to do something that's a little bit
25:54
more broader for offering, but all of those things with
25:57
Foundation models will still continue to be important and
26:01
interesting and always become even more important, right?
26:03
I mean, certainly the monitoring and governance piece I think is
26:06
going to become very significant overtime, as he's like larger
26:09
more black boxy, looking models, start to get adopted more
26:14
broadly. Yeah, that's so true.
26:16
That is one piece that I think people are going to recognize,
26:20
they cannot leave without and it is already happening.
26:24
There's a lot of understanding of monitoring.
26:27
I think the nuances of monitoring is really difficult
26:30
investors. We often say, Oh, a space is too
26:33
crowded. Like this is too competitive,
26:34
but sometimes that's actually a really good thing.
26:37
Like if there are a lot of folks who are trying to solve this
26:40
problem, perhaps it also means that there is a really big
26:43
problem to be solved, so sometimes it can be construed,
26:46
very poorly, but taking a step back, I think sometimes it could
26:49
be a really positive thing as well too.
26:51
I try to spend as much time, you know, parse through the perhaps
26:54
analogies and really get to the ground.
26:56
Truth of sort of what's happening in a space and why
26:58
it's important. So, speaking of monitoring
27:01
companies, is there have been a few people in envelopes
27:04
community that have approached me and said, yeah, I'm thinking
27:06
about creating a new company and I wanted to be something around
27:10
monitoring and I try with all my might to discourage them from
27:16
starting another monitoring company and it feels like and
27:22
this is what I would love to get your perspective on because it's
27:24
not only monitoring companies but there are like data labeling
27:28
company that comes out. But They're not getting funded
27:33
anymore, that whole explosion that we talked about if it was
27:37
2016 or if even if it was 2020, that company might be funded.
27:42
But now it's not getting funded. And is it because the novelty
27:48
has worn off. And there's already the set
27:50
players in the space and so since no one is coming at it
27:53
from, no one is doing monitoring in this super novel way.
27:58
It's just like an incremental betterment to the are
28:02
Re-established players. That's why they're not getting
28:04
funding. Yeah, I think in some of those
28:08
instances one, I would say there are definitely monitoring and
28:11
data labeling companies that are still getting funded.
28:13
I mean, I think just a few weeks ago, V7 announced that they had
28:17
reached out to V7. So there are certainly companies
28:19
that are getting funded and I can't speak to all the instances
28:23
of ones that might be getting funded versus ones that might
28:26
not. But I think in situations where
28:29
there is already a crowded Market, investors will ask
28:31
themselves. The question, what is truly
28:33
different about this company? And this approach?
28:35
And why should I invest in this one versus the X number that
28:39
might have been funded and are already out there in the wild as
28:42
well too and to some of what we talked about earlier that come
28:46
crowded competitive landscape can mean that there is a lot of
28:50
interest from buyers and clearly I think of the monitoring and
28:53
labeling spaces that's true. But the bar of why this one
28:58
versus the other ones is much much higher.
29:00
And so you do have to come with a A novel approach for something
29:03
that's really unique and differentiated relative to
29:06
others for at least myself. For example, say okay, this is
29:09
the one I really want to pack invest on because I think
29:11
talking to practitioners this approach is actually going to
29:14
resonate with them really well. And if you don't necessarily
29:17
have that founding the next monitoring company that's doing
29:19
something similar to what a bunch of other companies might
29:22
be doing regardless of whether it's in machine learning or
29:25
elsewhere. I think just becomes
29:26
incrementally harder. Yeah, so there are, we can talk
29:31
all Because now if you want or we could go in to dig into
29:35
specific deals, but I am fascinated with what are some
29:40
things that have pushed you over the edge to invest in companies?
29:46
Recently that made you think okay this is innovative and this
29:49
is worth me putting millions of dollars into when you first
29:53
started investing and you're like oh teams and markets and
29:56
then there's this bell curve of like oh you then you think about
29:59
all these other things like business model like Everything
30:02
going on underneath the hood and then as you get to the real
30:05
Epiphany point the Jedi. Yeah.
30:07
You there's the Jedi that's still says, it's teams at Big
30:10
markets and honestly the great teams figured out the really big
30:13
Market opportunities and I think it's really simple to say, but
30:18
it's actually really hard to find and evaluate.
30:20
Just these extraordinary teams, who are doing really interesting
30:24
things and you can just see have a vision for the world.
30:26
That's so different than what someone else might have pissed
30:29
you in the hundreds or Help thousands or however many
30:33
pitches you might be hearing but this one person really compels
30:36
you with their vision of the world and what is going to look
30:38
like in the next ten. Twenty ten twenty years.
30:40
I mean the one the folks who really sort of embody that for
30:44
me in particular in the orchestration spaces prefect and
30:47
Jeremiah Chris and the team there I think are just Stellar
30:51
Founders and Stellar product, folks, and really are so deep in
30:56
the data science in machine, learning spaces in particular
31:00
that they just shared this vision for a world.
31:02
World of orchestration and the way that people interact with
31:05
data, that I found incredibly compelling and it was a big
31:08
reason as to why we wanted to invest in them.
31:10
Despite the crowded landscape of orchestration, these are shared
31:14
a vision for the world that was so different than other folks
31:17
and clearly have the expertise to do so.
31:19
So you're calling yourself a Jedi time Hotel Demetrius, but I
31:22
certainly hope I'm more on the Jedi side that my, then I am on
31:25
The Apprentice side. That's awesome.
31:28
Yeah, I think at the end of the day, it doesn't matter.
31:30
You recognize the Orton thing, our teams and markets.
31:34
And when you're looking at the market of orchestration, despite
31:39
being crowded, I mean, we've got air flow which every single data
31:43
engineer, and machine learning engineer has touched at some
31:46
point in their life. I think yet, there are these
31:50
other tools like prefect or dags or managed airflow even and
31:55
that's not even to mention the machine learning specific tools,
32:00
right? The orchestration.
32:02
In machine learning specific ones like the flights or the
32:06
wars in Mo and that kind of stuff.
32:08
And so you're looking at it, you're saying this Market is
32:14
huge. What do you do to like, how do
32:17
you calculate that? How do you figure out that
32:20
market is the market is big as the you think?
32:23
Yeah, exactly. I mean, the simplest ways to
32:26
look at it or the simplest and potentially most accurate way.
32:30
These are generally Not 100% accurate, very rarely, 100%
32:35
accurate, and are more directionally correct in
32:37
anything, but it is definitely about the number of potential
32:41
users multiplied by the potential Revenue.
32:44
You might be able to get from those users.
32:46
That's generally what constitutes Market size and by
32:49
the, of course, inquire requires a lot of assumptions on our end
32:52
like how much do you actually think you can generate for a
32:55
user? How many users do you actually
32:56
think? There might be.
32:57
And in these early Nason markets, the first question of
33:01
how much You can generate per customer or per user is, maybe
33:05
not as important as like, how big, or how many of these folks
33:09
will, they're actually be when we first were looking at
33:12
developer platforms back in 2008-2009.
33:15
The very common refrain was there actually weren't that, all
33:18
that many developers. And you probably weren't going
33:20
to be able to charge all that much from them.
33:22
So now lo and behold, like that's actually very false say
33:25
that turned out to be wrong. And in the number of companies
33:28
in the developer platform world have emerged taking advantage of
33:31
both of those. Two vectors of growth neutral,
33:33
Leo and centigrade. Hey, Judy yacht 01 starkly and
33:37
Hoshi Corp as well to. Those are all examples of how
33:41
those things came to fruition. I suspect, we'll see exactly the
33:44
same thing not just for orchestration, but for the
33:46
machine learning on Space, broadly, the number of data
33:49
scientists and mlas is growing. So so, so rapidly.
33:52
And then 10, 20 years, I personally believe it's going to
33:55
be an order of magnitude larger than it is today.
33:57
And every single company in the world is going to be influencing
34:00
machine learning. Whether it be for For your
34:02
users. And I externally facing product
34:04
feature or something that's more internal and automating your
34:07
work internal to the organization.
34:09
So that's fascinating because it makes me think the market for
34:14
envelopes as we're talking about markets, right?
34:17
And it is very much at a it's like at a plateau right now but
34:22
that is just a natural occurrence in.
34:26
It is still got a lot of room to grow.
34:29
It's just going to grow once there's some Surety that happens
34:33
in the space. So I'd like to push back up one
34:36
point that it is at a plateau. So I think maybe from an
34:39
investor or hype perspective, it looks like it's at a plateau,
34:42
like the outside looking in. But, you know, something you
34:45
said earlier, where if you talk to folks who look at the
34:48
community, the MLS community in particular, these things are
34:51
growing and it may not look like again, this sort of hyper growth
34:55
just exponential curve up into the right.
34:58
But it's a slow and steady growth and if anything that
35:01
Kate's even more health of the community that people are slow
35:04
or in a systematic way or being drawn into leveraging.
35:08
Machine learning actually utilizing it for real
35:11
production, use cases. And when you talk to
35:13
practitioners, whether they be the best from portfolio company,
35:16
you are you talk to vendors themselves?
35:19
The adoption is growing on every single Vector.
35:21
So think there is a lot of growth happening in the
35:24
ecosystem and it might not be. So like top of Mind, perhaps
35:27
like some other areas but I certainly see from my
35:30
conversations with folks that It's happening.
35:32
Machine learning is happening and I struggled to think of a
35:34
world where in 10-15 years machine learning is not common
35:39
with everywhere. And in some ways it almost is
35:42
already, right? A lot of the consumer
35:43
applications, we interface with certainly have machine learning,
35:46
is part of the feature set or the core product in some way,
35:49
but I think that will just continue to be the case, not
35:52
just for Consumer, use cases, but also Enterprise has, and
35:55
we're just going to see that proliferate over the next
35:57
decade, plus b, and maybe a little bit more down Down on
36:02
this. Now, what I find interesting to
36:05
from your Viewpoint when you look at machine learning and you
36:09
look at the different use cases that are coming through.
36:11
And you, look at the nature of how companies are doing machine
36:17
learning, and there seems to be like there's one camp or there
36:24
are certain companies that are trying to attack vertical
36:28
problems. Like let's say, oh there's a
36:32
Recommender systems for e-commerce and we can add
36:36
machine learning to that or even I mean, Jasper AI is a great
36:39
example of this. This is copywriting and we're
36:42
using machine learning with it. Those are the vertical
36:46
Solutions. And then you have these more
36:50
what I feel like are the ml Ops companies that are selling two
36:54
teams that are doing, much more than can be verticalized.
36:59
Yeah, I just I wonder. Is it going to be the mix of
37:04
that or are we going to go totally vertical?
37:08
It is that going to be more of the other side but I would love
37:12
to hear your thought on it. I think I think as with most
37:16
things, the reality is probably somewhere in the middle or like
37:19
it will be a mixture of vertical Solutions as well as also
37:23
infrastructure solutions to enable the builders.
37:25
And yes, there's certainly some instances like Asian particular.
37:28
Jasper was a marketing copy and we have others in our portfolio.
37:32
Optimal Dynamics, which helps you optimize Trucking routes or
37:35
something that was totally pen and paper before today and
37:38
instances where there isn't necessarily the machine learning
37:41
Talent or the ability to hire those folks.
37:43
You might go with a more vertical solution or verb or
37:45
vertical Solutions might emerge there in much the same way like
37:48
Salesforce is at some level a database, but you don't really
37:52
interact with it. In the same way that you might
37:55
like a traditional sort of relational database is a little
37:58
bit, abstract it away for that kind of user but there is Still
38:02
really interesting things happening in the database world
38:04
as well to that people are building other applications on
38:06
top of as well. And I think in a very similar
38:09
way, we'll see things whether they be for particular verticals
38:13
or segments, clear, be marketing, or Trucking Logistics
38:15
or something similar to be Commerce.
38:18
For example, in your recommender system to we're building.
38:22
We're giving you the building blocks and the builders can
38:25
create new novel things that perhaps don't fit those tight
38:29
little squares of Market, Max or buckets.
38:31
Tubular verticals and build something that's fairly creative
38:34
and novel in those use cases. I mean take the developer
38:37
platform example as well. Like there is twilio of course
38:40
in those whole ecosystem of applications built on top of the
38:43
toilet, you also the attentive of the world.
38:45
For example, perhaps of all in that bucket, most notably so I
38:48
think there will be both and it's sort of a matter of what
38:50
fits your segment or which fits your your end customer better.
38:54
Yeah. I love that and I can imagine
38:57
lots of Engineers out there thinking to themselves.
39:00
No way in hell. It's going to be all Vertical
39:02
that that means that their job will become Obsolete.
39:06
And so, I don't think that is happening anytime soon.
39:10
And I really enjoy this view of Salesforce is a database, but
39:17
that doesn't mean that there are not tons of Innovations
39:20
happening every day with databases.
39:24
There's, I mean, there's a new database that comes out every
39:26
day so it is incredible to see that now When you're looking at
39:33
or where you are thinking about the future and I know I kinda
39:37
asked you this one already, but I want to hear it again.
39:40
Like what are some pieces that you feel like if I was just out
39:46
there and I was trying to figure out what company that I want to
39:51
build, where would you tell people to go?
39:53
Look, where would you be focusing your attention right
39:58
now? Personally, I am a big fan of
40:00
like enabling The builders to go and build things, certainly a
40:03
lot of interest or excitement that I've heard from some of our
40:08
largest portfolio companies. Do some of our medium-sized one
40:11
has been around actually novel storage techniques.
40:13
We're hearing a lot of interest in particular around Vector data
40:16
bases of passing, those structured data through model
40:19
and storing the relevant feature from them and not just doing
40:22
things like semantic search. But then also building things
40:25
I've recommender systems and other ml power features that has
40:28
actually not. Historically been very easy in
40:30
your traditional database system.
40:31
As well. It's a very hard tractable
40:34
database problem that cannot be solved immediately overnight.
40:38
But this one of these more basic building blocks that if it
40:43
starts to manifest and proliferate in adoption.
40:45
I think we'll see an incredible transformation across the
40:48
ecosystem of people, being able to actually leverage and use
40:51
machine learning at scale. And a way that wasn't
40:53
necessarily possible before and there are actually a few
40:56
companies that are already started doing that so perhaps it
40:58
might not be the best place for me to start a company.
41:00
But it's certainly an area that I am.
41:01
Really excited about and I think it's really interesting.
41:04
Yeah, it's funny. You mention that we just got
41:06
done with our Vector search hackathon not a week ago or two
41:12
weeks ago and we did it with in partnership with redis because
41:16
they put out a vector search database and so it's been really
41:22
cool to see the explosion of just that whole movement.
41:27
I mean, you've got what Pinecone, there is.
41:29
Also a small vase, Yes, Novus, yeah.
41:35
So there's a ton of that are coming out in that space and it
41:37
is really interesting to see all the use cases that they have.
41:42
And I like your view of it on how useful it becomes as we move
41:50
into this new paradigm that you were.
41:53
We were talking about for the first half of this conversation.
41:57
There are also a whole other second, a subset of sectors of.
42:00
You haven't even talked about. If you perhaps the one that's
42:03
most or at least I see is most top of mind for folks and
42:05
perhaps. Because so top of mind for me as
42:07
well, to is biotech, the applications of some of these
42:09
larger models. For biotechnology use cases,
42:12
just ain't going to be incredibly transformative for
42:14
the world in a very good way and beneficial way and enabling
42:18
early-stage companies there, take a model off the shelf or
42:21
apply it for your Corpus of data is going to be huge opportunity
42:25
and Company to be built in facilitating that Yeah, it'd
42:30
help us live longer. I cannot wait and get all this
42:33
be healthier live longer. Yeah.
42:35
I think it'll be tremendously valuable for Society, for sure.
42:40
It's going to give us so many more sake.
42:42
Dude, this has been incredible. I love talking with you, and
42:47
gotten to dive into what you're thinking before we go.
42:52
I want to hear a little bit. If you're cool with it, just
42:55
about how you usually are structuring.
42:58
Deals. What are you go into?
42:59
What do you look at? When you are investing in
43:02
companies? As far as like, the structure
43:06
goes? Are you always thinking about
43:09
trying to do? Follow on rounds?
43:11
I know you said you're basically round agnostic, so you can do
43:14
whatever you want. Do you like that idea of doing
43:19
follow-up rounds? Do you like to have that locked
43:22
in with a Founder? What does that look like?
43:25
Yeah, so Bessemer we are a state.
43:28
Age, agnostic firm, invest everywhere from seed to grow
43:31
crops in the Salient example is twilio.
43:33
Or I believe we LED their seed round which back in the day who
43:36
much smaller with 125 thousand dollar check invested over 75
43:41
million dollars in life of that company until I went public.
43:44
And that is what Bessemer we like to do is invest really, and
43:47
follow on at lead rods until exit.
43:49
And we tend to be very thematic as a firm supporter go, very
43:51
deep into a particular area of focus, the best of the best
43:54
company regardless of stage, and so the machine learning and
43:57
software Structure spaces to invest at the C Series a and
44:02
even Series be stages and support the company's all the
44:04
way through until exit. So hopefully Bernie one who's
44:08
building, whether it be early stage or what may seem like a
44:11
little bit later stage and growth.
44:13
We're excited to have a conversation and would love to
44:16
speak with anyone building for some of these problems that we
44:18
talked about today and building for machine learning in the
44:20
future as well too. And now that we know you are a
44:25
sucker for infrastructure tools. And enabling the builders I
44:30
think that is going to make it much better for anyone out there
44:33
who is thinking about enabling Builders and what they do.
44:38
So what about when you're looking at a company?
44:41
How long does it generally take you?
44:42
And how has it changed over the last two years from the time
44:47
that you first meet with that founder to the time that you
44:51
invest? Yeah by let's see the process
44:53
can take anywhere from days to years.
44:57
I've had some founders There we met them and know them for many
44:59
years before we actually entered a formal partnership and really
45:03
invested in good part of their cap table and Summers been much
45:06
shorter and two days weeks, long process, certainly over the past
45:10
couple of years it was on the shorter and but it wasn't the
45:12
longer end, but we were going to meet always to meet our Founders
45:15
timeline on fundraises. My one gratuitous prized piece
45:19
of advice for Founders would be to really take the time to get
45:22
to know your investors. And I don't say that just
45:24
selfishly so that we can have more time on a process but
45:27
really Lee, you're spending probably seven to ten years with
45:31
this person potentially on your board.
45:33
And that's a really important relationship to have.
45:36
And you can't develop trust overnight.
45:38
And in the span of a few days, it's good to be able to have
45:41
open dialogue, before you sign the term sheet.
45:43
And before you really enjoy her, enter the formal partnership
45:47
there, so that you know who's going to be on your board and
45:49
who you're working with. Have you ever had to fire a
45:53
Founder here? The buyer of a we've had to part
45:55
ways with our Founders every so often.
45:57
Yes, it has thankfully, it's happened, mutually for
46:00
recognizes for the good of the company and sometimes
46:03
unfortunately, it has had to happen, but in those instances I
46:05
think we've been able to do it in a in an agreeable way.
46:09
So I'm thankful to for having that experience thus far.
46:12
Yeah, and it's interesting on that because I've from what I've
46:17
heard obviously I don't know too much about your world.
46:21
Your day-to-day. But those companies that really
46:25
are doing a nosedive, those are the ones where you just kind of
46:29
say, all right, you know what we can't do much about this and
46:33
sometimes it's because of a Founder sometimes it's because
46:36
of a market or whatever and you don't necessarily have the
46:41
ability to spend, as much time with those Founders or those
46:44
companies. And and those ones maybe are the
46:49
ones that need Need the most help, but it just takes up too
46:53
much time. Is that a correct assumption?
46:56
I would actually disagree with that really disagree with that
46:59
actually. And it's not always in the
47:01
instances where things aren't going well that they might
47:05
mutually part ways with the founder, who is the CEO and they
47:08
transition into another part of the business.
47:11
Sometimes it is what you had a different level of scale.
47:13
Sometimes you need to shift leadership to another person who
47:16
might be well suited to take the business to the next level and
47:19
there's multiple examples of that in the Extra software
47:22
space. And I think the most Salient
47:23
most recent one that is really well known or Hoshi core.
47:26
Right? Where Mitchell and Armand,
47:28
really? The founders and creators of the
47:29
projects and Dave came in and CEO honest.
47:32
Impossible, job there, anything that was beneficial for the
47:35
organization company as a whole. So there's a lot of those
47:38
instances of course and in terms of where we spend our time, we
47:42
spend our time, a lot with companies that it might always
47:45
be up into the right rear. So a so rarely is it always very
47:49
open to the right with no Kinks along the growth curve.
47:51
ER of these are often, very nonlinear stories and are very
47:55
bumpy along the way, so we don't all orphan and I hope none of
47:59
our other period or even any companies, certainly not.
48:01
He spent a lot of times with companies where it may not seem
48:04
like, it's always going right in one or two quarters or a year or
48:09
so. But, you know, his Venture
48:10
investors were trained to take a long-term perspective on things
48:13
and so we're partnered for the long term and the long term
48:16
outcome of the company. Whether it may look like a local
48:19
minimum at the time, but we want to help go.
48:21
Every single one of our companies.
48:23
Yeah. I imagine there's Founders that
48:25
can pull a rabbit out of the hat and you've seen that happen
48:29
where maybe they do have those bumpy quarters or bumpy years
48:33
and then they pull it off and it is back up into the right?
48:38
And then there are other ones that for whatever the case may
48:41
be, it's not their own doing, it's probably more the market,
48:44
it just doesn't get there and they have to abandon the company
48:48
or the company goes under. That inflection point or have
48:56
you seen anything that you can pass along to companies that may
49:00
be going through a rough patch right now?
49:03
Yeah. And in particular there's
49:05
definitely a rough patch happening right now.
49:07
I think we're starting to see a lot of Enterprise pulled back on
49:09
spending and it's obviously affecting some of our own
49:11
portfolio companies as well too and it sounds very easy to say.
49:15
But I think it's true. These periods of contraction can
49:18
often craft some of the best companies and so the best
49:20
Companies will emerge from this opportunity.
49:22
It takes a lot of persistence to get through it, but we just
49:25
published a, a podcast with the founder of StubHub Indiegogo, my
49:29
former employer actually and he talks about navigating the
49:32
complex water is of the past. Two decades, having started both
49:36
of those businesses and seen covid totally decimate.
49:39
The Live Events industry and have to survive that but the
49:42
they've come out stronger as a result of it.
49:44
So yes, again, these Journeys are not quarters long or years
49:49
long, these are Take many years to be a founder and see these
49:53
things all the way through. And so, the piece of advice, I
49:56
give all of my portfolio company Founders is really built with
49:58
company for the long term and with these local sort of
50:02
minimums along the way this happening in 2022 2023.
50:06
I firmly believe in particular on the machine learning space
50:08
and the broader infrastructure software space a lot of
50:11
incredible work that's happening and we're going to empower a lot
50:13
of users to do their work better and consumers to experience
50:17
products in a much better way, whether it be through machine
50:19
learning or Churros code, we're helping a lot of books and the
50:23
focus, and vision should always be on that.
50:25
On this build a business around optimizing Bryant users and our
50:28
end customers. Excellent, sake bed.
50:32
Thank you so much for coming on here.
50:35
If someone is starting a company or is looking to fundraise, best
50:39
way to get in touch with you. What is your favorite way to
50:42
talk your reach out to me at LinkedIn?
50:44
Feel free to send me an OD ADI, @bv p.com.
50:47
I am, I'm around my email, so feel free to reach out, and I'm
50:52
amazing. Thank you so much for coming on
50:54
here, vetted, really taking a different opinion and approach.
51:00
Play Devil's Advocate with me. I love it and I love the work
51:04
that you've been doing. I love chatting with you.
51:06
Whenever I get the chance, it's always enlightening.
51:09
Always fun to chat with the juice.
51:11
Thanks so much. My name is Autumn ostraca and
51:14
I'm head of machine learning engineering at origami energy
51:17
here in the UK. And I'm bothering you today to
51:20
recommend, you subscribe to the ml Ops committee podcast to keep
51:24
up to date with all of the brilliant goings-on, within the
51:27
community, and get a glimpse of what some of The greatest Minds
51:31
across the amount of space are up to.