Latest / Agentic Conversations (formally mlops.community) / ML Unicorn Start-up Investor Tells-IT-All // George Mathew // MLOps Coffee Sessions #126

Transcript

ML Unicorn Start-up Investor Tells-IT-All // George Mathew // MLOps Coffee Sessions #126 · Agentic Conversations (formally mlops.community)

0:00
Everyone, my name is George Mathew.
0:02
I am a managing director at Insight partners and I love my
0:07
coffee cold brew. Ideally Nitro.
0:13
All right it's a little bit of a different one today.
0:15
You've Got Just Me Myself. And I along with George and
0:19
we're talking primarily about how he sees the ml Ops
0:24
ecosystem. I love talking with investors.
0:26
If you haven't realized this yet, I've done it quite a few
0:29
times but Like to see what their thesis is, are on the MLM space,
0:34
how they're seeing tools coming through, where these tools are
0:37
valuable what tools are valuable to them.
0:40
And George Mathew is one of the best in my eyes.
0:44
He has done some incredible investing in the last two years
0:49
at his time, at inside Partners. Some of the companies invested
0:53
in have been weights and biases data bricks Fiddler, our
0:58
sponsors of the The community shout out to Fiddler, gotta love
1:01
them, astronomer the managed air flow solution.
1:06
Killing it. Landing a i, let's talk about
1:08
who George is for a minute. Before we get into this
1:11
conversation, I said it before he's managing director at inside
1:15
Partners before that. He did a little stint at a drone
1:19
tech company and before that, he was the CEO.
1:24
Oh, and president of all turrets company, you've probably heard
1:28
of they're doing some Even stuff in the data space.
1:32
He took all tricks from the state that they were in and
1:40
scaled them up to its IPO. And before that, let's just keep
1:45
going back. He was at sap before that he
1:48
held a leadership position at Salesforce when they were super
1:51
small. So you guys been around the
1:54
block, he's a bit of a legend and I am thrilled to have
2:00
learned from him. We talked all about his thesis
2:03
is around the mL of space. We also talked a bit about what
2:07
he feels is. So enticing about Enterprise
2:10
software and why he's so enamored by it.
2:14
And last but not least, we look at the way he likes to
2:19
structures deals and how he thinks about companies when he's
2:24
reviewing them. It's incredible to see his idea
2:28
in his vision for these Generationally enduring
2:32
companies. That's what I came away with.
2:35
I'd love to hear what you come away with.
2:37
Though they give us a grant or just let us know, right?
2:40
Me it would mean the world to me if you like and leave us a
2:45
review. If you're listening to this on
2:47
Spotify or apple and if you are on YouTube hit that subscribe
2:51
button, it does a ton to get this out to more people.
2:55
Let's get into this conversation with George.
3:03
So I want to know this because we are going to be talking a lot
3:06
about the future. Do you think we're ever going to
3:08
get to a point seeing as how? There's a lot of this ml
3:12
prompting, and a lot of stuff right now it's all the rage with
3:15
Dolly and GTP 3. Are we going to get to a point
3:18
where we can just prompt infrastructure to happen?
3:21
I can tell whoever that I want x amount of pods in my career, and
3:27
he's cluster and that will just happen.
3:29
We're getting Oh, sir, for sure. And it's an interesting question
3:33
that you're asking there. Because if you think about the
3:35
nature of how a serverless distributed computing has now
3:39
arrived at scale, what the thing that people don't really talk
3:42
about is how much intensity there is a particularly from a
3:46
system, reliability engineering standpoint to keep that
3:49
infrastructure going. So one of my favorite companies
3:52
recently that we invested into is a company called Shoreline
3:55
data and shorelines entire business is helping this exact
3:59
problem. Demetrius which is how do you
4:02
Auto? Wait, the challenges of standing
4:05
up and managing infrastructure at scale.
4:08
Hey, this things about to run out of disk like how do we
4:11
proactively deploy AI Bots to be able to handle the complexity of
4:16
distributed? Yes, but yeah, moment in time
4:19
where almost a moral equivalent of large language models used
4:24
for infrastructure deployment at scale is bit by bit Brick by
4:28
Brick coming together as we speak.
4:30
That's what I mean, it just feels like we're going to go
4:33
there, right? It feels like we can't not be in
4:37
a place where it's going to be that easy to just prompt
4:40
whatever all of my different services to work together and it
4:45
will be less of a headache than it is now.
4:47
Yeah, it's funny that we're starting.
4:49
This is the ml Ops discussion going directly into the
4:53
infrastructure dialogue. First, and probably work her way
4:55
back and said, well office in a little bit, but this is not
4:58
necessarily A new idea but it's actually one that can be better
5:04
implemented at scale with what we have today with on
5:07
microservices server, liske brunette, he's distributed
5:10
architecture but let's play this back a little bit even though I
5:16
don't know 15 years ago if we think about where the scale even
5:21
10, 15 years ago, a product like Roku.
5:25
As a great example of the last generation of infrastructure as
5:28
a service, you can can define a whole series of, you know, call
5:34
it dials in terms of what the workload should look like for
5:37
the application that you might be deploying this case early on
5:40
in heroic is life cycle, they were very strong at Ruby on
5:44
Rails applications, right? If you wanted to just simply
5:47
deploy a Ruby on Rails application that had called the
5:50
temperature dials enabled so that you can build and deploy an
5:54
application skill like crooked did an incredible job for
5:57
developers at that moment. And I think In a moment now
6:02
where we have even more scalable infrastructure in front of us
6:06
than we've ever had in human history.
6:09
But at the same time, the complexity of managing is
6:13
exactly is profound and so kudos to companies that have been
6:18
really attacking this problem. Big and small certainly you look
6:22
at what's happened with Hoshi, Corp, and terraformed is an
6:25
example of a big company that's attacking the space, but also
6:30
Lots of incredible startups that are earlier in the journey like
6:33
Shoreline data. I think it is something that's a
6:35
natural, stroke substrate to how the systems of the next decade,
6:40
emergent of all, I love it. All right, so over the last 20
6:45
months, basically, what since covid started, you've been
6:48
deploying Capital at Insight partners and you've gone pretty
6:54
wild like you've deployed and I think I did a little research
6:58
and it's been 21. Is probably more now maybe
7:02
correct me if I'm wrong, but I want to know where these
7:05
companies on your radar back when you were at all Turks, or
7:10
was it something that you decided?
7:15
Once you started becoming more and more in this space and you
7:19
started seeing more deals and you started creating a thesis
7:22
that I want to talk to you about in a second, or did you already
7:25
have them in mind and it was just a natural shift?
7:29
Yeah, they weren't Tall on the radar.
7:31
When I was certainly president of alteryx many of those
7:33
companies didn't even exist at that moment.
7:36
Some of them are certainly on my radar when I was building parts
7:40
of Casper a particularly when we were looking at a lot of image
7:43
video processing and how much the deployment around deep
7:47
learning models would be influential for really
7:50
understanding what anomaly detection and similar use cases.
7:54
Look like in video image related applications of these first sort
7:59
of that portion. Test resistance.
8:01
But I think timing is always fascinating.
8:03
And I think in this case, what we started to see particularly
8:07
in ml Ops was a moment where this important disaggregation
8:12
was occurring. In two fundamental areas, one is
8:16
core ml Ops tool chain itself and just how much more important
8:21
key elements of that, a mobsteel chain were and driving best
8:25
practices in terms of how and then we'll practitioner could
8:28
consume elements that will change.
8:30
The most effectively. And if you look at the
8:32
investment thesis, it certainly shows right?
8:35
How we've really been focused on the experience around, some of
8:38
the best of breed Solutions are on machine learning operations.
8:42
The second was frankly, just the emergence of the modern data
8:45
stack because the importance of em Olaf's, and how you can now,
8:49
build higher functioning, machine learning applications at
8:53
scale, using the Hema Labs Frameworks that we see today
8:57
could not have been possible, a lot of ways with Having some
9:01
level of taming understanding of the underlying data particularly
9:05
on the structure game. So, yeah.
9:07
And so you started to see that specifically with a structured
9:10
and semi-structured data, not only with snowflake data breaks
9:14
and dromio and just the number of tools that have emerged
9:17
around the data Lake itself. But then the higher functioning
9:19
capabilities that were built affect the on top of the data
9:22
Lakes. So today we're into that
9:24
opportunity. Just how challenging it has been
9:29
to manage. Video to manage acoustic wave
9:33
forms to manage imagery and some of the most interesting machine
9:38
learning at this point, is actually happening on call it
9:41
that. But we have historically called
9:43
unstructured data. And this is where that
9:46
opportunity was timing was great, because he started to see
9:49
the rise of this ml Ops toolchain.
9:53
You started to see this great foundation of the modern data
9:56
sack, that's supporting all kinds of data.
9:59
But still Quite figured out this sort of unstructured data realm
10:02
as well and so they were just great investible companies at
10:07
that moment. So I feel in a lot of ways
10:10
blessed by two things, one was just fact that was there and
10:13
secondly, that there was a forward leaning wonderful
10:18
organization in inside partners, that has historically just been
10:23
long on the transformational nature of software at scale.
10:27
We've been investors in software for well over. 25 years.
10:31
And this moment in time, at least for as you talked about 2
10:35
meters with this infrastructure, being ready with the now
10:38
emergence of modern data, stack on top of that infrastructure
10:41
and certainly I'm a loves tool chain that I just mentioned a
10:44
moment ago, all these enabled, the timing to work itself out to
10:48
make those Investments. Now, we still have to play
10:52
forward, we still have to ensure the the amazing companies that
10:56
it's had an opportunity to invest into thematically in the
10:59
space can grow. Prospered can build incredible
11:01
businesses. This next half a decade and
11:04
certainly that's the other part of our Focus right now.
11:06
So, one thing that amazed me, the last time we chatted was I
11:11
think I do just as much at talking, is anyone in the mobs
11:16
Community, about how much of a mess ml Ops is right now and how
11:20
things are very unclear book. When we chatted it felt like you
11:24
really had an understanding of things and how they were
11:27
congregating and I was blown away.
11:30
Away by that. Because for me, I look at it and
11:32
I say, oh yeah, especially when I was trying to do my research,
11:35
on the comparison side of the ml Ops Community website, where we
11:40
put together a comparison of different monitoring tools or
11:44
feature stores or deployment tools, and it felt to me like
11:49
deployment tools. For example, we have tools that
11:54
are specifically geared toward speeding up your underlying I'm
12:00
sure something like, a Desi data or octo ML and they do
12:04
deployment but they're like they're there and they're their
12:07
main value prop is something completely different.
12:10
And so it's hard to say, oh, that's a deployment tool, but
12:13
when I talk to you, it was a different story.
12:15
Like, it felt like you had things more clear at least than
12:18
I did. And so I'd love to break down
12:20
when you talk about how you're looking at the space, What are
12:23
the different areas that you see?
12:26
Yeah, so it was us as a firm, a lot of Chaffee who Allah of work
12:30
and everything related to apply day.
12:33
I myself to a party number of us were all thinking through this
12:37
next generation of machine learning and machine learning
12:41
operations. And what we derived was the
12:44
following, there was still components in areas that I think
12:48
you've described a few of them out of two by two, that was
12:50
going to be important. One was just where it stood in
12:53
the overall tool chain itself and I think we broke it down
12:56
into three big areas of the tool chain.
12:58
One was Data prep, right? In a preparation, all the
13:01
necessity of being able to do the work for dinner preparation
13:05
to was model development right? And then three was then model
13:10
governance and security and everything we invested into
13:13
could be put into those three big sub categories of machine
13:18
learning operations, and then there was like the Nuance that
13:20
you called out, which is ultimately, was it call it on
13:25
the application layer? Like the Roscoe's and the
13:28
weights, and biases, and the Fiddler Of the world or was it
13:31
in the algorithmic layer? Where the Desi's and the neural
13:34
Magics? The Octo ml's and the Run AI is
13:37
all congregated and I think that gave us a lot more clarity
13:39
because if you think it through great in that realm of what the
13:45
capabilities provide from an application layer specifically
13:49
when it came to delivering data preparation model development
13:53
and model governance and security and then where did that
13:57
algorithmically are particularly for model optimization to get
13:59
the end. Tool chain all the ways.
14:01
Any and that's only because I'm drawing it out in a more linear
14:04
sequential fashion in reality. It is much more to do with the
14:07
end of the day. But if I just drop this sort of
14:09
tail end of models coming into deployment, what we loved about
14:13
the companies that you just mentioned that, we're just
14:16
talking about right now, the algorithm Claire, every one of
14:18
them was taking a different way to really think about improving
14:22
how models showed up at scale in large-scale production
14:27
infrastructure and in the case of Desi it was just model
14:30
estimation. And that was just something that
14:33
don't Eddie Lonnie and myself. Just looked at and we're like
14:35
every model that goes into production is to going to need
14:39
just a stronger way to prune. The model tree to be able to
14:44
understand what is it that is compacted from a production
14:49
ization standpoint. Then the full life cycle of the
14:52
model that might be developed for the use case.
14:54
And so I think what we're seeing here is a lot more clarity now
14:59
in So what you can invest into particularly as I create the
15:04
sort of nuance between got Oliver, declare an application
15:06
layer, and then cut across a specifically data prep, the
15:11
model development, and the government security surrounding
15:13
it. And that's generally how we've
15:15
taken sort of the investment thesis to mahlab State, and I
15:19
would say there's just one fine-grained Nuance that.
15:21
We've also noticed, is that the tools that are now required for
15:27
structured data are somewhat different Aunt than unstructured
15:31
data. I'll give you a great example
15:32
this right? I love that.
15:34
I don't know if we need a feature store for anything
15:37
related to talk about future as an above it but we certainly
15:40
don't need a feature scorer. For unstructured data, right?
15:43
Because ultimately you have most that captured inside of your
15:48
labeling and annotation infrastructure there, you don't
15:51
need a separate features for at that moment in time.
15:53
But at the same time, you might be less reliant on labeling an
15:57
annotation when you're dealing with mostly Data where there is
16:01
some feature management. That's absolutely necessary.
16:03
And whether you get it from a feature assure, you that it
16:06
inside your database, that's an investment thesis.
16:08
That we can talk a little bit more about, but there's nuances
16:11
there in terms of how you deal with the shape of the data
16:15
that's actually coming into your analysis.
16:18
Yes. And it's so interesting to look
16:20
at and I've been saying this for a while and I have a half
16:24
written blog post that I never released, but it was something
16:27
along the lines of there will never be Like, it's so easy to
16:31
say that, there's a modern data stack, it's really difficult to
16:34
say that there's a modern ml stack, Andre, it's because of
16:38
exactly what you say. We're whether maybe you're doing
16:42
Robotics and that's a whole different Beast than if you're
16:47
doing unstructured data. And even if you're doing
16:49
unstructured data, around autonomous vehicles compared to
16:52
unstructured data when it comes to just like predicting cancer
16:56
in x-rays or whatnot. And so, those two Are so
17:00
fundamentally different in the SLS that you need.
17:03
Are so complex and they're each their own.
17:07
They have their own set of needs that you.
17:11
I have a hard time seeing one stack that will be for all of
17:16
these and that's not even when you get into something like the
17:20
structured data and you start to look at the time series
17:22
forecasting. We're going to start to look at
17:24
just like recommender systems, all of that and I'm a firm
17:27
believer in that theory that we're going to see.
17:30
A lot of segregation and I almost wonder if it's going to
17:33
be something along the lines of now, there's like the term ml
17:38
Ops is too broad, even though it is like the niche of a niche.
17:42
Right now, it feels if you throw everything under Emma Ops it
17:46
could be too broad of an idea there but I'm well.
17:51
And you notice that I never have said ml op stack and that's
17:54
exactly to that point. Yeah, made right because
17:58
ultimately what will See over the next half a decade, is that
18:02
it is truly a tool chain that there will be elements that
18:06
would be swapped in and outside of that.
18:09
She will chain based on the use cases that are necessary to
18:12
solve for. And you know, as much as we'd
18:16
like to think that mobs is Nishi, it's not it's actually
18:19
the foundational element will fuel an entire generation of
18:24
software Innovation and I think it will speciate it will
18:27
differentiate pretty remarkably. Particularly in this next half a
18:31
decade, totally. There's going to be delivering
18:33
and that's pretty obvious. It's already forming and we're
18:36
just seeing the beginnings of it and later those splinters will
18:39
Branch out and we'll see what it actually, what comes to pass.
18:44
So the other piece that I was wondering about this, like when
18:48
you're looking at the ml Ops companies or you're just looking
18:53
at ml tooling companies, is there a certain section or piece
18:59
of Chain that you're interested in right now that yeah, maybe
19:04
you have been paying a lot of attention to and you're like wow
19:08
this is really heating up. Yeah, let me mention at least
19:12
one that I really was excited by this past year and how that's
19:17
gone. And then use that as a jumping
19:19
Point, feature excitement, is building as a look through
19:23
spaces, it was evolving, it was phenomenal to see the tools were
19:28
beginning to be much more bill. It's for the machine learning
19:30
practitioner at their core and the particular challenge of how
19:36
you handle everything related to model building specifically
19:41
experiment, tracking the Version Control the hyper parameter,
19:45
tuning workloads. Those were things that was
19:49
fascinating to see that there could be a possibility to build
19:52
another set of delightful tools to the same realm of delight
19:57
that we had built companies like alterations blow the last
20:00
generation self-service analytics for the machine
20:03
learning practitioner weights and biases really stood out two
20:06
years ago, transitioning, operator to investor and
20:10
thinking a lot about the future of really this entire generation
20:15
personas and particularly the memo practitioner Priscilla an
20:18
emerging in this space and what the sort of software tools for
20:21
me for them and even more compliant even about it was the
20:26
fact that Lucas and team pretty far along in the depth of Of
20:30
what an experienced D4 product there, and proven itself out
20:33
quite nicely in the last two years, by all measure, which
20:37
devices become a delightful tool that become essential to how
20:44
machine learning practitioner works very consistently.
20:47
We get to hear that many folks in the space and so thinking
20:50
about it in that level of delight and that level of just
20:56
fundamental I can't live without this capability.
20:59
I think a lot about that in terms of the next series of
21:02
Investments and into areas that I'm unpacking and try to
21:06
understand a little bit more. Is any thinking mentioned it
21:10
idea of Time series, but just broadening that view a little
21:13
bit more to be sure. It's like, where does this realm
21:15
of streaming data, kind of play? And how do we functionally,
21:19
build better tools to be able to do machine learning driven
21:23
analysis, particularly against her, and also involves data
21:27
management conversation. And also Paul's a good
21:30
opportunity from a machine-learning tooling
21:33
standpoint and so that's an area that I'm very excited by.
21:36
I haven't paid any investments in that area yet, but it's
21:38
something that I'm actively looking at what's there.
21:41
And what could be exciting? Particularly the snacks at half
21:44
decades? That's one call it some segment
21:47
that's exciting. The second one that's even more
21:50
profound is that think we also alluded to this like the move
21:55
that we're now seeing to go from.
21:59
Mostly the world that we've lived in to date around training
22:04
and being able to use supervised deep learning up to this moment.
22:09
And now starting to see some incredible benefits and
22:14
opportunities particularly with unsupervised deep learning.
22:17
And if we think through just the patterns of where large language
22:23
models have evolved typically in this last 18 months ago,
22:27
forwarded to Next, 18 months, we're going to be at a moment
22:30
where sometime in the middle of 2023 will reach a trillion
22:38
parameter large language model, right?
22:40
I don't think we can actually comprehend the impact of.
22:45
I don't know what that even means for Humanity.
22:47
You when we reach that level of parameterisation and Alive,
22:50
large language model at scale and then you layer in what's
22:54
possible. In terms of prompt engineering,
22:56
that can be incredible applications.
22:58
That are effectively built on top of these large language
23:01
models. I'm wondering for you.
23:04
Personally, what fascinates you so much about enterprise
23:07
software. Yeah, I'll start with kind of
23:11
enterprise software, but on a quickly dive into back to the
23:14
areas that we've been talking about.
23:17
I think, for me personally, what I've learned in my experience of
23:22
being in a press awful person, I guess I can now say that, I have
23:24
Enterprise upper versus having spent time at Person sap and you
23:30
got the tracks until all tricks. Yeah.
23:32
It's at least in the in the blood.
23:33
At this point I've been remarkably fascinated by the
23:38
ability for enterprise software to not only drive tremendous
23:43
value in terms of how it enables all kinds of companies across
23:48
all different Industries to build a scale at a rate that
23:51
they've historically, never been able to without the use of that
23:54
suffer. But it would have been even more
23:56
fascinated by is how much Free software can continue to reach
24:01
reform itself, transform, and almost three format itself in
24:04
every generation. And that's what keeps me
24:07
continuously. Excited because what I saw with
24:11
sap and Salesforce is actually not what we saw with
24:15
self-serving products like alteryx antenna.
24:17
Yeah. When they got to scale.
24:19
And certainly now we're seeing right now in the way that suffer
24:22
is consumed in scaled, so that that keeps someone who's been a,
24:27
you know, build, Owner and operator.
24:29
And now investor tremendously excited by the potential of what
24:33
we're discussing here. But what I love even more now is
24:36
the tremendous human impact. The positive human impact that
24:40
software has had on society and I saw little inklings of it.
24:44
Even at all tracks in particular.
24:46
I would remember when when I would, I would go to our user
24:50
conferences. They were incredible experiences
24:53
of users whose lives were basically transformed by the use
24:57
of software like alteration, And this particular example, I
25:00
remember a dad who was also a data analyst and he came to me
25:04
and he said, I just want to give you a hug and I'm like, I don't
25:07
know you, but why do you want to give you a?
25:10
I just want you to know that the work that you and your
25:12
Associates is done at all. Trikes has enabled me to take
25:15
into my son's baseball game on Time, on a Friday afternoon
25:19
which I've only been able to do for years because I was always
25:23
wrangling with data as the last thing I had to do on a Friday
25:26
afternoon and it was just part of my job.
25:28
And you just made it easier for me to have a home life.
25:32
Again, when most of my work life was consuming time that would
25:37
have taken up in this case to make it to my son's baseball
25:39
game and those examples keep coming at you, right?
25:42
And when you build beautiful software that really serves
25:46
people. The way we had done in this last
25:48
generation and certainly some incredible Founders are doing it
25:51
this year, our nation. That's what the transformative
25:54
effect of software is on a very personal level and that Level
25:59
becomes scaled out at a societal level and that's what keeps me
26:03
incredibly excited about everything related to software,
26:06
even to this day. It's so funny, man, because I
26:10
was just talking to my buddies after earlier today.
26:12
And he's a data analyst, and he told me how at his new company
26:17
that he's working at. He's using alter aches, and she
26:20
didn't even know that I was going to talk with you.
26:22
But yeah, it's like, oh, dude, it's so good.
26:24
Oh, it's so nice to use this. I am so happy with the tool.
26:28
And just starts singing the Praises of it.
26:30
And then I told him like it's funny you mention that because
26:33
I'm talking to George and George had something to do with that
26:35
back in the day. And so I mentioned that he was
26:38
like tell him thanks. So yeah, you can't make that
26:43
stuff up. You can't buy that gratitude and
26:45
frankly, you can't have a higher purpose in life than to Delight.
26:49
Someone at that level, just their own personal
26:53
transformation. Being driven by a software
26:55
experience. And look, if you give me build
26:58
one of those, Generational tools.
26:59
It is a profound personal and professional experience to go
27:02
through it by happen to at least see.
27:04
Twice certainly with Salesforce when I was early in my career
27:07
and and then again at all tricks but yeah searching for the next
27:12
opportunity at least now is an investor is frankly what I love
27:15
her. Yeah so you went from president
27:18
of Anthrax and you did a transition into drone Tech with
27:24
cats Bree, what prompted that transition.
27:27
Yeah. It It's a wild ride going into
27:30
caspari. It was an incredible experience
27:32
for me personally, because what I wanted to learn more about was
27:37
just additional ways that enormous amounts of data could
27:41
be leveraged. And at that time coming, CEO
27:43
caspari was incredible to see was not only this sort of piece
27:48
of Hardware that caspari built to be able to serve these.
27:53
These cases mining companies, insurance companies to be able
27:55
to collect imagery photograph. We stitch that together from 2D
28:00
to 3D layer in insights, but really, it was those insights
28:04
that got me excited. Because guess what?
28:05
All those insights were using deep learning at that moment.
28:09
And so, we were some of the earliest purveyors of just,
28:11
apply deep learning at scale fact, that you can Ensemble, the
28:14
series of CNN's or CNN's Ellis TMZ in a way that you could now
28:20
perceive what the state and condition of.
28:23
In this case. Physical assets were and how
28:26
they basically changed over time.
28:28
Using not with detection classification, techniques from
28:31
a deep learning standpoint. It just taught me that there was
28:34
like other ways to think about data than for my mostly
28:37
structured Kate experiences really, being part of the
28:40
subjects and alter its prior and that could Escape me another
28:45
kind of lens to think about the world of data least.
28:48
And I would say, I was after the four years at all tricks, I was
28:52
so enamored with what was happening.
28:54
Like, it was either becoming a Founder CEO.
28:58
In the world of machine learning or to be able to help the next
29:03
generation of founder. CEOs, in terms of how they were
29:05
building their Journeys and it was a very deeply personal
29:08
decision to not be as a Founder be co-founder and to be an
29:13
investor because I just never thought that there would be an
29:16
opportunity to still be as strongly passionate about
29:19
everything that I just mentioned a moment ago and do it from the
29:23
investment side of the world versus doing it from.
29:26
No interest is the company building side.
29:28
And it turned out like, at least with the opportunity that
29:31
insight and the partnership at insight has afforded me to do
29:34
particular in these last two years, is to be that profoundly
29:37
passionate about the future of machine learning in particular,
29:40
and, and lean into it and so far that is played out pretty well.
29:46
So there are a lot of Founders in the MLS community and I'm
29:49
owner if we can jump into some specifics of the deals and how
29:53
you go about looking at companies that you invest in the
29:56
first thing that we should. Probably figure out what size?
30:01
What round, what check size, what stage are you looking at?
30:06
Yeah, we're where we have a lot of flexibility at inside, is
30:11
that we are multi stage in nature and that has helped us be
30:16
able to write earlier checks and many companies that are core
30:21
growth Equity. But at the same time, we're also
30:24
not affirm that. Once you write the series a
30:27
check that you have to go. Find long-term capital somewhere
30:30
else, particularly as you're a high-growth concern in the
30:32
space. So that multi-stage approach has
30:36
helped us as a firm as we kind of work with the best Founders
30:41
in the world as they build their respective Journeys.
30:44
And for us, will we generally try to do is we try to be
30:49
focused on the series a, and Beyond.
30:51
And that's where present Focus has been.
30:54
We don't generally do seed stage checks, Try to do is support
31:00
incredible seed stage investment, firms companies that
31:05
have done really well as investor in the seed stage and
31:09
participate in the future deal flow to be future Czech writer
31:13
particularly in the series, a series B, and B are.
31:15
So if you look through where I mean, most of more recent
31:22
Investments, particularly in the envelopes and the amount of data
31:25
stack, it was It was a B's and C's mostly centered around a
31:31
little bit more focus on A's and a little bit more focus on sees
31:36
because that happened to be where the opportunities were at
31:39
the moment. But we're more generally stage,
31:42
agnostic, and this point and I generally try to be a little bit
31:47
more domain-specific in my staging us asleep.
31:50
Okay, I like that. Now, let's talk some numbers and
31:53
some kpis, and breakdown the deals we chatted before.
31:58
I appreciate it an email that I got from you because I love
32:01
looking at how investors think about community and how a
32:06
community has value. And we chatted about what kind
32:09
of different signals. There are four communities like
32:13
issues opened and just how people are responding in slack.
32:18
If there's an active slack, like active members, you've also got
32:22
the amount of contributors that aren't on the team.
32:25
There's all kinds of cool stuff that you can look at cat.
32:29
But what are some just in general as you're evaluating kpi
32:33
is not necessarily community-related.
32:36
What are the kpis that you're looking at when you're
32:38
evaluating a deal? Yeah, and let me mention a few
32:42
not Financial metrics and then we can talk a little bit about
32:44
the financial metrics. Did non Financial metrics,
32:47
particularly for an open-source company, particularly in the
32:49
early stages, do you have to go beyond just the GitHub stars?
32:54
And again, important but you have to really And how much
33:00
stickiness and repeatability you're seeing in your open
33:03
source product and usually that goes into two more intensive
33:08
metrics and more Colour understanding of with
33:10
communities around it. We generally do.
33:12
Look at how many pull requests are we talking about how many
33:15
contributors there are inside and outside the organization?
33:19
How active is that level of contribution?
33:21
That's occurring. How quickly are they iterating
33:24
and working through the natural issues that would rear gin and
33:27
open source? Project that might be coming
33:29
about and then once they get past that point of just
33:33
understanding the basics of where open-source traction is
33:37
what I try to do is I try to at least get into the first
33:41
dialogue of what commercialization look like,
33:44
right, what is it natural path to commercialization from this
33:48
incredible Groundswell of community forming around around
33:53
that particular open source project.
33:55
So one example, like use this Is was meta Bates, right?
33:59
So when we invested into metal base, it was just very clear
34:03
that the usage and adoption. Particularly in terms of the
34:06
number of downloads, the number of pull requests, the number of
34:09
GitHub Stars, the amount of activity that was in the open
34:14
source community. Be something that could be very
34:17
readily commercialized in the Years, subsequent to the
34:20
investment that we have made, and that I should turned out to
34:22
be quite sure. Right?
34:23
Because that a base had such a strong ability to attract That
34:29
bi user that just needed that capability of the dashboarding,
34:33
visualization. That metal base provides in a
34:35
much more easy, open source, Deployable manner than called it
34:39
last generation tools that were in market.
34:42
And that's where metal base was finding a lot of incredible
34:44
traction and increasingly that's where the commercialization
34:48
efforts of metal-based kind of really found incredible traction
34:51
as well. So I think I would like to see
34:55
that level of understanding of we're in this It's an open
34:58
source solution can find Community LED growth.
35:02
Before you get to commercialization, when you get
35:05
to commercialization, Dad, a whole set of metrics kind of
35:08
layer on top. And here that commercialization
35:11
metric comes down to how much are you bringing in a set of you
35:16
logos? For instance, for new customers
35:19
that are being acquired on a quarterly basis, in your really
35:21
tracking you logo acquisition as one key metric.
35:25
The second is if you have now acquired the logo, What is the
35:28
annual contract value of that logo?
35:30
You've Acquired and frankly, does that, in no contract value
35:34
have an opportunity to expand over the course of the lifetime
35:39
of that subscription where you know that one dollar can become
35:42
$1.50 $1.75 within the first year, few years of Landing, the
35:49
first dollar in the customer so that metric is the grocery
35:53
tension versus the Net retention.
35:55
So you're looking at trying to find businesses that have That
35:58
dollar attention some of the best businesses that we have
36:00
happened to invest into believe time are now reaching 200, % ND
36:06
are two hundred percent net dollar attention.
36:08
So every time they land a customer with a dollar that
36:13
dollar is generating another dollar.
36:16
So it's like doubling within 12 months of Landing that customer.
36:20
So incredible metric to understand the health of a
36:23
business. So, I mentioned the logos, I
36:25
mentioned the ndr And grocery tension as well alongside of
36:30
that what you're generally looking for is a health of net,
36:34
you are creation, like how much new a are you generating on a
36:38
quarterly basis and then alongside of that, are you doing
36:42
it? Efficiently?
36:43
What is your cost of customer acquisition effective?
36:45
Look like when you put that entire picture together, you
36:49
have a good understanding of how healthy you business is today.
36:54
Importantly, we're could go in the future that makes Good
36:58
investment. That's the kind of early view
37:01
on. How do you take?
37:04
Commercialization from a community that may or may not
37:09
have an open-source underpinning associated with it and play it
37:12
forward as investable pieces. I love this idea of how it
37:16
expands like the contract value expands and really looking at it
37:21
like, in laymen's terms, I just think people are so stoked using
37:25
the tool that they want to use more of it.
37:28
Basically at the end of the day and you can break that down and
37:31
say all right, here's the numbers Associated and Some of
37:35
the best deals you've got two hundred percent expansion and
37:38
that is that, that's a great tool.
37:41
It's sticky people enjoy it and they want more of it.
37:44
And so you've got something there's something that's
37:46
happening behind the scenes and so I'm wondering success.
37:49
So I won't name the company. Who's got the to protect the
37:52
innocent 200% idea as we speak, probably guests among the stuff
37:56
is we invested into but just to be precise on this, that's
37:59
exactly right. And you saw that in every
38:02
generation of tools prior to it, right?
38:04
If you think about why Salesforce in the early day was
38:08
so incredibly powerful was that once people in this case sales,
38:14
managers and sales team started to use Salesforce.
38:16
Even at a departmental level it had just the natural ability to
38:20
expand within that organization. Even if it didn't have fire
38:23
Enterprise up front, if you looked at all, Trace in Tableau,
38:27
particularly in the 20 2010 timeframes, what you saw was
38:31
companies that had a net dollar, North of 144.
38:35
So for every dollar that alters the Tableau were generating at
38:39
that present time your later, they were expanding 40 cents on
38:44
that dollar. And why was that do is yours is
38:46
exactly what you just said. It's like, people were
38:49
generationally delighted as the analyst to use a product.
38:52
Like Tableau use, a product like alteryx to solve their needs as,
38:55
as day-to-day data analyst. And so finding those delightful
39:00
experiences has a tremendous impact.
39:04
Packed on where company growth and Company's ability to
39:08
uniquely differentiate themselves in the space layout
39:12
in every software category. And frankly that has not changed
39:15
for 20 years, you find delightful software, and those
39:19
companies become the Seminole companies, that their
39:21
perspective generations. And so, how much does the last
39:25
12 months or the next 12 months valuation multiples, play into
39:30
the deals that you're making. Yeah, let's let's talk about
39:33
valuations for a I think what we see right now is a reversion to
39:37
the mean, as I think about it, and why is that reverse to
39:40
happening? Look valuations, got overheated.
39:43
They just did across the board. And look, we don't even have to
39:46
pick on private companies, just look at the public company comps
39:51
in terms of multiples early and where they were 12 months ago.
39:55
Heck even six, seven months ago versus where they are now and
39:58
so, does that mean these are bad companies?
40:02
But does it actually doesn't at all.
40:04
It just means that the cost of capital was quite low, what was
40:07
the best place? You can invest into when callous
40:10
capital is low. It says she's high-growth.
40:12
Where's that sort of highest growth?
40:14
That's possible in the investable universe?
40:17
Will turns out that stack, where is it intact turns out that's in
40:21
software? Where is that in software?
40:22
And kind of see how that plays out, that's why the snowflakes
40:26
of the world, even as a public company had the sort of comps
40:30
that they had. But now, as we think about both
40:32
public and private companies, He's we've reverted back to
40:36
where things were even well before the sort of upcycled, in
40:39
terms of valuations were, but it doesn't change the reality that
40:43
you still have to build a great business.
40:44
It doesn't change the reality. That amazing companies that are
40:49
forming. Now, are generational in nature
40:54
and they will still be great companies.
40:56
Regardless of how the mid term and short term Cycles are for
41:02
evaluation. So what I try to focus on with
41:06
the founders that I have the honor of working with is, how do
41:11
we build enduring companies? That last not only in the good
41:17
times, but also in the challenging times, for least, in
41:21
this case, valuations, but transcend that to just build
41:25
great companies. And I think the ones that are
41:27
now focused today on, More efficient growth that it's not
41:34
okay, to spend your way to Top Line growth, but you have to
41:38
just get better efficiency, better repeatability in terms of
41:42
how you bring your products to Market at scale.
41:45
Those are the companies that will do perfectly fine and
41:48
Beyond perfectly fine, particularly as we exit
41:52
whatever. This multi quarter, perhaps even
41:55
multi-year downturn different stuff.
41:58
So as far as As you being an operator, do you feel like you
42:03
have a sixth sense in when you're looking at a CEO and when
42:09
you're talking to them, can you, or is it just so far across the
42:12
board that you can't really pick if someone's going to be an
42:17
incredible operator, to build those endearing companies?
42:20
I think you get a sense of who are going to be great, Founders
42:26
and frankly, founder operators that have longevity.
42:30
But that's a lie. I don't have enough of a sense
42:34
of fine-tuning of these as an investor to read really
42:38
understand who were perhaps the generational founder and CEO.
42:42
That could be present. That what I try to do is just
42:45
really understand is their longevity, right?
42:48
Is there going to be someone here that will really continue
42:53
to carry the flag long after all the hype Fades away and you
42:57
still have to build in the middle.
42:59
Yeah. Really, you have to do it over a
43:02
decade and that's what I try to have a conversation with
43:06
Founders. Particularly, as we're engaging
43:09
early, I just want to just have the discussions like, hey you
43:13
realize this is there's a another decade you have ahead of
43:17
you on this. You're just getting going to
43:18
leave a series, a series B Investments, and the ones that
43:23
have internalized, that the ones that have literally come to an
43:26
understanding that is their normative reality.
43:30
For a decade. Those Founders, I believe have
43:34
just generally greater longevity than the ones that like shocked
43:39
by the fact that I just said that because I think generally
43:41
speaking the ones that want it, find it a faster outcome aren't
43:47
going to be able to deal with the hardships of the challenges
43:49
of building. A great company.
43:51
That's good. Yeah.
43:52
Yeah. That makes complete sense when
43:55
the going gets tough then they're looking for the easiest
43:59
way out a You moron this? If you've got an extra minutes,
44:03
I know we're hitting up on time so I don't want to keep you if
44:05
you have a hard stop right now. But I would love to pick your
44:08
brain about a few more of these details on how you structure
44:11
deals and what your preference is, what you like to see.
44:14
And I'm wondering about like liquidation preferences for
44:17
startups that you've invested in.
44:18
What do those look like? So generally, we try to keep it
44:22
as simple as possible on this Demetrius.
44:25
And I think the reason we try to do this in the simplest way
44:28
possible of least on call. The growth-oriented
44:32
opportunities that we focus on is that if your liquidation
44:37
preferences, how you return money and capital at scale,
44:40
particularly on the early stage Investments like that you're
44:43
probably someone's in the Rock conversation, right?
44:46
So on the earliest stage heels were interesting opportunity has
44:49
like a lot of Runway ahead of them.
44:52
We haven't at least, I personally haven't done anything
44:55
beyond the 1X liquidation. Preference, now, later stage
44:59
deals where there's There's a pivot.
45:01
There's a restart. There's some things to Think
45:03
Through. There's there's reasons that you
45:05
want us to introduce structure, and there's good reasons that
45:08
structure comes into deals, but I change channel and try to shy
45:12
away from structure on earlier deals, early being called the
45:17
series a series because I look, everything still being figured
45:20
out. Like, why would you want to
45:22
place this sort of structure on top of the founder, working
45:25
through the Journey of figuring out how they get their company
45:28
to scale? I think there's moments.
45:30
Structure is good and I think there are plenty of
45:33
opportunities to put structure particularly into later stage
45:36
opportunities. But personally, as an investor,
45:41
my rule of thumb is been try to stay away from particularly
45:44
multiples of breath liquidation preferences, particularly on and
45:48
earlier stage investment and talk me through how many days.
45:52
It takes to go from that first meeting with founder to the term
45:56
sheet and how has that changed since?
45:58
Because I heard it was real hot back in the day. a a k 6 months
46:01
ago and now I've heard it's cooled off a little Yes, it has
46:06
the way it's cooled off right now, is that time as far as
46:10
getting to a term sheet, is just generally longer.
46:15
But I will say, what hasn't changed for us.
46:18
Personally, as a firm is our focus on spending worth of right
46:22
amount of duration. Necessary to get to the right
46:26
decision with not only Powers but just the whole Nuance of
46:30
making investment decision. So we would have still done.
46:35
Work, like, we don't go rifle off the term sheets without
46:39
doing the work and that has been pretty consistent for us and
46:41
maybe other firms have had different experiences.
46:45
But for us, at least, we take the time necessary to do the
46:48
work to conduct the right level of due diligence before putting
46:53
a term sheet in front of any investment decision.
46:56
And so for us, there is a minimum time box involved in
46:59
that, and we don't take first meetings instead of term sheets
47:03
that has not been at least my time.
47:05
It so far and I have never happened.
47:09
And I ever happened and more power to the amazing folks who
47:12
can pull that off. But yeah, that's just not,
47:15
that's not at least my personal super router.
47:17
Yeah. Yeah.
47:18
It's a little bit of a crapshoot when you do it like that.
47:21
But wow and your words, my friend your words about my life
47:27
because actually you gotta be wondering about that.
47:31
But what if it's working for him?
47:33
I who am I to judge some? I think the Founders end up
47:36
finding the best investors and I think more often than that
47:41
incredible Founders want discipline investors around them
47:45
that that are there for the Long Haul because they're there for
47:47
the Long Haul. And I think that despite all of
47:51
the noise that was in the market during this massive up cycle and
47:56
the frothiness that was there. Ultimately, as we've now started
48:00
to more than anything reverted, back to the meeting and seen the
48:03
highest come to read. Active lows.
48:06
What has been the constant? Is that, as I said, great
48:11
investors and great Founders, find a way to meet for the
48:14
longevity of building incredible companies and at least in the
48:18
case of enterprise software, you have to think about it in terms
48:23
of a decade, not in terms of, hey, how could I flip this in a
48:26
year or two and see what happens?
48:28
Yeah, it's not a quick house renovation.
48:31
Flip that you can get in and get out of with Scott.
48:34
Don't go. Real estate.
48:35
If you want to do that, it's plenty of activity, but great
48:39
flock together in real estate, but that's not how software
48:42
Works. No.
48:43
And so, do most that your meetings come from referrals, or
48:47
are you taking meetings from random cold, the Outreach?
48:52
Most of my meetings come from referrals and there's a reason
48:54
for that. Most of my meetings come from,
48:57
honestly, our own sourcing capabilities.
48:59
Like it will be outbound interesting to just about any
49:02
and every one that's doing incredible work in the software
49:05
universe. And so, most of our own quality
49:09
ability to continue to grow, and scale comes from our own ability
49:12
to outreach to the best Founders, and entrepreneurs and
49:16
the world. And I would say when there is
49:19
activity of these, Terms of investable opportunity inbound.
49:25
It's still referrals are probably the best approach every
49:29
user in the reason I say that, like at least with a referral,
49:33
someone else is vouching for the thesis, the founder, whatever it
49:38
might be, someone says putting at least their reputation in
49:41
front of the conversation. And I think I just generally
49:46
like to see referrals if it's not something that we're already
49:49
album, reaching out to ourselves.
49:51
Excellent. Excellent.
49:52
This has been super cool. If anyone wants to get ahold of
49:56
you then I'll tell them to reach out to me and I'll bet it and
49:59
then pass it along to tell you because hugely appreciative
50:02
interest. Yeah that would be amazing.
50:04
I love the way that you're looking at things, I love your
50:06
thesis on and just the inside Partners like how you see Emma
50:11
lobs, I really think it's super cool to see who your how you are
50:18
really taking apart the space and putting deep thought.
50:22
Each one of these different moving pieces and so I'm excited
50:27
to see where it goes. I really want to thank you so
50:30
much for coming on here and talking to me about this and
50:34
really School in me and not only on how you're seeing things but
50:37
also on the art of the deal which could say.
50:41
So thanks again, George and we will see you later more.
50:46
We're all learning together and certainly appreciate this
50:49
opportunity to learn in this journey with you.
50:51
And these promises Awesome podcast to be sure something
50:53
again, I'm looking forward to Future conversations.