Latest / The Pragmatic Engineer / Building Windsurf with Varun Mohan

Transcript

Building Windsurf with Varun Mohan · The Pragmatic Engineer

0:00
I think a lot of people talk about how we're going to have way
0:02
fewer software
0:03
SPEAKER_00: engineers in the near future.
0:04
I think it feels like it's people that hate software engineers,
0:07
largely speaking,
0:07
that say this.
0:08
SPEAKER_00: It feels pessimistic,
0:09
SPEAKER_00: not only towards these people,
0:10
SPEAKER_00: but I would say just in terms of what the ambitions for companies are.
0:13
I think the ambitions for a lot of companies is to build a lot
0:16
better product.
0:16
And if you now give the ability for companies to now have a
0:20
better return on
0:20
SPEAKER_00: investment for building technology,
0:22
SPEAKER_00: right?
0:23
SPEAKER_00: Because the cost of building software has gone down.
0:25
SPEAKER_00: What should you be doing?
0:26
You should be building more because now the ROI for software
0:29
and developers is even
0:30
SPEAKER_00: higher because a singular developer can do more for your business.
0:33
SPEAKER_00: So technology actually increases the ceiling of your company much faster.
0:37
Winsurf is one of the popular IDs us software engineers use
0:40
thanks to its AI coding capabilities.
0:42
But what are the unique engineering challenges that go into
0:44
building it and how
0:45
SPEAKER_01: could tools like Windsurf change software engineering?
0:48
SPEAKER_01: Today I sat down with Varun Mohan, co-founder and CEO of Windsurf.
0:51
We talk about why the Windsurf team build their own LLMs and
0:55
how LLMs for text are
0:57
SPEAKER_01: missing capabilities necessary for coding,
0:59
SPEAKER_01: like fill in the middle.
1:00
SPEAKER_01: How Windsurf uses a mix of techniques for many cases,
1:03
SPEAKER_01: like to solve for search,
1:05
SPEAKER_01: how they use a combination of embeddings and keyword based searches.
1:08
Why latency is your number one challenge and how incorrectly
1:11
balancing GPU compute
1:12
load and memory load can lead to higher latency for code
1:15
suggestions popping up.
1:16
How Varun thinks the software engineering field will evolve and
1:19
why he stopped
1:20
worrying about predictions like 90% of code will be generated
1:23
by AI in six months.
1:24
If you want to understand the engineering that goes into these
1:27
next generation IDs,
1:28
SPEAKER_01: then this episode is for you.
1:30
If you enjoy the show,
1:30
please do subscribe on any podcast platform and on YouTube.
1:34
SPEAKER_01: Welcome to the podcast.
1:36
SPEAKER_01: Yeah, thanks for having me on.
1:38
SPEAKER_01: You've recently launched GPC 4.1 support in Windsurf,
1:43
SPEAKER_01: which by the time this is out,
1:46
SPEAKER_01: it will have been a few weeks.
1:47
SPEAKER_01: But what are your initial impressions so far?
1:50
SPEAKER_01: And in general,
1:52
SPEAKER_01: when you introduce a new model,
1:54
how do you evaluate how it's working for the coding use cases
1:57
that we all use?
2:00
SPEAKER_00: Yeah,
2:00
SPEAKER_00: maybe I can talk about the second part,
2:02
SPEAKER_00: and then I can talk about GPC 4.1 and the other models afterwards.
2:07
Basically,
2:08
internally,
2:09
these models have these non-deterministic properties.
2:12
They sometimes perform differently in different tasks in ways
2:14
that are unexpected.
2:16
You can't just look at a score on a competitive programming
2:19
competition and decide,
2:20
SPEAKER_00: hey,
2:20
SPEAKER_00: it's going to be awesome for programming.
2:23
SPEAKER_00: And,
2:23
SPEAKER_00: you know,
2:23
SPEAKER_00: interestingly about the company,
2:25
SPEAKER_00: maybe this is,
2:25
SPEAKER_00: this is going to be helpful context.
2:27
SPEAKER_00: A lot of us in the company previously worked in autonomous vehicles.
2:30
SPEAKER_00: And I think in autonomous vehicles,
2:32
SPEAKER_00: we had a similar type of behavior where you had a piece of software.
2:36
SPEAKER_00: The software was very modular, lots of different pieces, uh,
2:39
SPEAKER_00: So each piece was machine learning driven.
2:42
So there was some non-determinism and it's very hard to test it
2:45
in the real world,
2:46
right?
2:46
Actually,
2:46
it's much harder than it is to test,
2:48
I guess,
2:48
windsurf out in the real world.
2:49
It's much harder to test autonomous vehicle software out in the
2:52
real world,
2:52
SPEAKER_00: because if you ship bad software,
2:54
SPEAKER_00: you have the chance of hurting a lot of people.
2:56
SPEAKER_00: Right.
2:56
SPEAKER_00: Hurting a lot of people,
2:57
SPEAKER_00: hurting the,
2:58
SPEAKER_00: you know,
2:58
SPEAKER_00: I don't know,
2:58
SPEAKER_00: just the general public infrastructure.
3:00
SPEAKER_00: Right.
3:01
SPEAKER_00: So in that case,
3:02
we needed to build really good simulation evaluation
3:04
infrastructure in autonomous vehicles.
3:06
SPEAKER_00: And I guess we brought that over here as well,
3:08
SPEAKER_00: where,
3:08
SPEAKER_00: hey,
3:09
SPEAKER_00: if you want to test out a new model,
3:10
we have evaluation suites and the evaluation suites not only
3:14
test end to end
3:15
SPEAKER_00: software performance,
3:16
SPEAKER_00: which is to say you give a high level task.
3:18
What is the pass rate of actually completing the high-level
3:21
tasks on a bunch of unit tests?
3:23
SPEAKER_00: It also tests retrieval accuracy,
3:25
SPEAKER_00: edit accuracy,
3:27
SPEAKER_00: redundant changes,
3:28
SPEAKER_00: all these different parts of a model that are like negative behavior.
3:30
SPEAKER_00: Because for our product, it not only matters that you pass a test,
3:34
It also matters that you didn't go out and make 10 steps that
3:37
were unnecessary
3:37
because the human is going to be waiting on the other end for
3:40
all of those changes.
3:41
SPEAKER_00: So we have metrics for all of these things.
3:43
And we're able to put each model through,
3:45
I guess,
3:46
a suite of tests that give us metrics.
3:48
And that's the way we decide,
3:49
hey,
3:49
this is a good model for our end users.
3:52
SPEAKER_00: And that's the high-level way that we go about testing.
3:55
And these tests,
3:56
they sound great in theory,
3:57
but in practice,
3:58
what does it look like?
4:00
SPEAKER_01: I'm going to assume you're going to have,
4:01
SPEAKER_01: we can imagine us engineers who've been writing code,
4:06
SPEAKER_01: probably not autonomous vehicles,
4:07
SPEAKER_01: but similar ones.
4:08
SPEAKER_01: We know our unit tests, our integration tests.
4:11
SPEAKER_01: If you do mobile, you know your end-to-end tests.
4:13
SPEAKER_01: I'm assuming this will be a little bit...
4:15
different but with some similarities like do you actually like
4:18
code some scenarios
4:19
you have like example codes example prompts and and then i
4:24
assume you can do a bit
4:25
of that but then what else and you know how does this all come
4:28
together and like
4:28
how can i imagine this test suite is it like one big giant blob
4:32
that runs for i
4:32
SPEAKER_01: don't know how long
4:33
Yeah,
4:34
one of the aspects of code that is really good is it can be run,
4:37
right?
4:37
SPEAKER_00: So it's not like a very touchy-feely kind of thing.
4:40
SPEAKER_00: In the end, a test can be passed.
4:42
SPEAKER_00: So what we can do is we can take a bunch of open source repositories.
4:45
We can find previous pull requests or commits that actually not
4:49
only add tests,
4:50
SPEAKER_00: but also add the implementations correspondingly.
4:54
SPEAKER_00: And what we can do is instead of just taking the commit description,
4:57
SPEAKER_00: we can remake what the description of the commit should have been,
5:00
SPEAKER_00: like a very high level intent.
5:02
SPEAKER_00: And then from there,
5:03
SPEAKER_00: it becomes a very,
5:04
SPEAKER_00: I guess,
5:04
SPEAKER_00: programmatic problem,
5:05
SPEAKER_00: which is to say,
5:06
SPEAKER_00: hey,
5:06
SPEAKER_00: like,
5:07
SPEAKER_00: first of all,
5:08
SPEAKER_00: find the right files that you need to go and make changes to,
5:10
SPEAKER_00: right?
5:11
SPEAKER_00: There is a ground truth for that, right?
5:12
Because the base code actually has a set of five,
5:15
10 files that changes were made to.
5:17
SPEAKER_00: Then after that, what is the intent on those files?
5:19
SPEAKER_00: You can actually go from the ground truth backwards,
5:22
SPEAKER_00: which is that you know what the final change was from the actual code,
5:26
SPEAKER_00: and you can have the model generate that intent.
5:29
And then after that,
5:30
you can see if the edit,
5:32
given that intent,
5:32
is correct.
5:33
So you now have three layers of tests,
5:35
which is that,
5:36
hey,
5:36
did I retrieve the right things?
5:38
SPEAKER_00: Did I have the high-level intent correctly?
5:40
SPEAKER_00: And is the edit performance good, right?
5:42
SPEAKER_00: And then you can imagine doing much more than just that, but...
5:45
at a high level now you know just from a pure commit or a pure
5:49
actual ground truth
5:50
piece of code you now have multiple metrics that you can go
5:52
about and then
5:53
obviously the final thing you can actually do is run the code
5:55
right so it's not
5:56
just like a you know when you when you measure some of these
5:58
chat products they
6:00
actually you know the evaluation is a little bit different
6:02
which is to say the
6:03
evaluation is you give it to multiple humor humans in a blind
6:06
test in an a b test
6:07
SPEAKER_00: and you ask them
6:08
SPEAKER_00: Which one did you like more?
6:09
SPEAKER_00: Obviously,
6:10
SPEAKER_00: for us to quickly evaluate,
6:11
SPEAKER_00: we can't be giving it to tens of thousands of humans in a second.
6:16
SPEAKER_00: And with this now,
6:16
SPEAKER_00: within minutes,
6:17
we can get answers to what is the performance on tens of
6:20
thousands of repositories
6:21
SPEAKER_00: and tests,
6:22
SPEAKER_00: basically.
6:22
This episode is brought to you by Modal,
6:24
the cloud platform that makes AI development simple.
6:27
SPEAKER_01: Need GPUs without the headache.
6:29
SPEAKER_01: With Modal,
6:30
SPEAKER_01: just add one line of code to any Python function and boom,
6:33
SPEAKER_01: it's running in the cloud on your choice of CPU or GPU.
6:36
SPEAKER_01: And the best part, you only pay for what you use.
6:40
With sub-second container starts and instant scaling to
6:42
thousands of GPUs,
6:43
SPEAKER_01: it's no wonder companies like Suno,
6:45
SPEAKER_01: RAMP,
6:45
SPEAKER_01: and Substack already trust Modal for their AI applications.
6:49
SPEAKER_01: Getting an H100 is just a pip install away.
6:52
SPEAKER_01: Go to modal.com slash pragmatic to get $30 in free credits every month.
6:56
SPEAKER_01: That is modal.com slash pragmatic.
7:00
SPEAKER_01: This episode was brought to you by CodeRabbit,
7:02
the AI code review platform transforming how engineering teams
7:05
shift faster without
7:06
SPEAKER_01: sacrificing code quality.
7:09
SPEAKER_01: Code reviews are critical, but time consuming.
7:12
CodeRabbit acts as your AI co-pilot providing instant code
7:15
review comments and
7:16
SPEAKER_01: potential impacts of every pull request.
7:19
SPEAKER_01: Beyond just flagging issues,
7:20
CodeRabbit provides one-click fixed solutions and lets you
7:23
define custom code
7:24
SPEAKER_01: quality rules using AST graph patterns,
7:27
SPEAKER_01: catching subtle issues that traditional static analysis tools might miss.
7:31
SPEAKER_01: CodeRabbit has so far reviewed more than 5 million pull requests,
7:35
SPEAKER_01: is installed on 1 million repositories,
7:37
SPEAKER_01: and is used by 50,000 open source projects.
7:40
Try CodeRabbit free for one month at coderabbit.ai using the
7:44
code pragmatic.
7:46
SPEAKER_01: That is coderabbit.ai and use the code pragmatic.
7:51
I really like how much engineering you can bring in because
7:55
it's code and because
7:56
we have repositories because you can use all these things that
8:00
feels to me it gives
8:01
a bit of an edge to like some of the other use cases just as
8:04
you mentioned.
8:05
SPEAKER_00: No, I think you're totally right.
8:07
SPEAKER_00: Like we think about this a lot of like,
8:09
what would have happened if we were to pick a different sort of
8:11
category entirely?
8:13
SPEAKER_00: It's just, I think the ground truth is just very hard.
8:16
SPEAKER_00: You don't even know if the ground truth is great, right?
8:18
In some ways,
8:19
in some cases,
8:20
for all we know,
8:20
the ground truth is not good.
8:21
SPEAKER_00: But in this case,
8:22
SPEAKER_00: I think it's a lot easier because of the verifiability or kind of,
8:26
SPEAKER_00: if you have a good test,
8:27
SPEAKER_00: it's a lot more easy to verify software.
8:30
And can you give us a sense of what is the team behind Windsurf
8:34
and also how
8:35
SPEAKER_01: complex this thing is?
8:36
SPEAKER_01: And how did it even come about?
8:37
SPEAKER_01: Because for all I know,
8:39
SPEAKER_01: you know,
8:40
SPEAKER_01: like a few months ago,
8:41
SPEAKER_01: when this podcast started,
8:42
SPEAKER_01: there was no Windsurf.
8:43
SPEAKER_01: There was Codium.
8:44
SPEAKER_01: We actually...
8:45
SPEAKER_01: talked a bit about what Codeam was a little bit different.
8:47
SPEAKER_01: And then out of nowhere, boom, Winster comes out.
8:50
SPEAKER_01: A week later,
8:51
SPEAKER_01: already in the pragmatic engineer,
8:52
SPEAKER_01: about 10% of people that we surveyed were already using it,
8:56
SPEAKER_01: which was,
8:56
SPEAKER_01: I think,
8:57
SPEAKER_01: the second largest usage of tools and people were enthusiastic about it.
9:01
SPEAKER_01: But I assume there's more to this.
9:03
SPEAKER_01: It didn't just come out of, you know, like nothing, right?
9:05
SPEAKER_01: Right.
9:06
SPEAKER_00: Yeah, so happy to talk a little bit about our story and summarize it.
9:11
SPEAKER_00: So we started the company now close to four years ago,
9:13
SPEAKER_00: which is substantially before,
9:15
SPEAKER_00: I guess,
9:16
SPEAKER_00: the co-pilot and Chai Chi PT sort of moment.
9:19
SPEAKER_00: A lot of us at the company,
9:20
SPEAKER_00: as I mentioned,
9:21
SPEAKER_00: previously worked,
9:21
SPEAKER_00: I would say,
9:22
SPEAKER_00: on these hard tech problems,
9:24
SPEAKER_00: you know,
9:24
SPEAKER_00: AR,
9:24
SPEAKER_00: VR.
9:26
SPEAKER_00: autonomous vehicles.
9:27
SPEAKER_00: And I guess at that point,
9:28
what we started building out and we had a different company
9:30
name at that point,
9:31
SPEAKER_00: it was called Exa function.
9:33
SPEAKER_00: We started building out GPU virtualization systems.
9:36
So we built out systems to make it very fast and efficient to
9:39
run GPU based workloads.
9:40
SPEAKER_00: And we would enable companies to run these GPU workloads on CPUs.
9:44
And we would transparently offload all GPU computations to
9:47
remote machines.
9:48
SPEAKER_00: And that could be CUDA kernels all the way down to full-on model cost.
9:53
SPEAKER_00: It was a very low-level abstraction that we provided people.
9:57
SPEAKER_00: And so much so that if the remote machine died,
9:59
we'd be able to reconstruct the state of what was on that GPU
10:02
and another GPU.
10:03
And the main use case we targeted were these large-scale
10:06
simulation workloads for
10:08
these deep learning workloads that a lot of these robotics and
10:10
autonomous vehicle
10:10
SPEAKER_00: companies had.
10:11
SPEAKER_00: And we thought, hey, the world was going to look like that in the future.
10:14
SPEAKER_00: A lot of companies would be running deep learning workloads.
10:17
SPEAKER_00: What ended up happening was in the middle of 2022,
10:19
SPEAKER_00: I think Text DaVinci 3 sort of came out,
10:22
SPEAKER_00: which was,
10:22
SPEAKER_00: I guess,
10:23
SPEAKER_00: the GPT-3 sort of instruction model sort of came out.
10:28
SPEAKER_00: And I guess that changed a lot of our priors,
10:30
SPEAKER_00: like both me and my co-founders' priors,
10:32
which is to say we thought that the set of models that would
10:35
run were going to look
10:35
SPEAKER_00: a lot more homogenous.
10:37
SPEAKER_00: If you were to imagine in the past,
10:39
SPEAKER_00: the number of different models that people would run was very diverse.
10:42
People would run convolutional neural nets,
10:45
recurrent neural nets,
10:46
LSTMs,
10:47
graph neural nets.
10:48
SPEAKER_00: There was a whole suite of different types of models.
10:51
SPEAKER_00: We thought in that case,
10:52
SPEAKER_00: hey,
10:52
SPEAKER_00: if we were an infrastructure company,
10:54
SPEAKER_00: we can make it a lot easier for these companies to run these workloads.
10:57
SPEAKER_00: But the thing is,
10:58
SPEAKER_00: with TextMG3,
10:59
we actually thought that actually there would be a
11:01
simplification of the set of
11:02
SPEAKER_00: models that would run.
11:03
Why go out and train a very custom BERT model if you could go
11:06
out and just ask a
11:07
SPEAKER_00: very large generative model,
11:08
SPEAKER_00: is this a positive or negative sentiment?
11:10
SPEAKER_00: And we thought that that was where the puck was going.
11:12
SPEAKER_00: I guess for us, we believe in scaling laws and all these things.
11:15
If it's this good today,
11:17
how good is a much smaller model going to be in two years?
11:19
SPEAKER_00: It's probably going to be way better.
11:21
SPEAKER_00: So what we decided to do was actually focus on the application layer,
11:24
SPEAKER_00: take the infrastructure that we had,
11:25
SPEAKER_00: and actually build on an application.
11:27
SPEAKER_00: And that was what Codium was.
11:28
SPEAKER_00: So we built out extensions in all the major IDEs.
11:32
SPEAKER_00: And very quickly, we were able to get to that point.
11:35
And we actually did train our own models and run them ourselves
11:38
with our own inference stack.
11:39
And the reason why we did that is,
11:40
at the time,
11:41
the models were not very good.
11:42
SPEAKER_00: The open models are not very good.
11:43
And also,
11:44
for the workload that we had,
11:45
which was autocomplete,
11:46
it was a very weird workload.
11:48
SPEAKER_00: It's not very similar to the chat workload.
11:50
SPEAKER_00: Code is in a very incomplete state.
11:52
SPEAKER_00: You need to fill in code in the middle of a line.
11:54
SPEAKER_00: There's a bunch of reasons why this workload is not very similar.
11:56
SPEAKER_00: And we thought we could do a much better job.
11:58
So we provided that because of our infrastructure background
12:00
for free to basically
12:02
SPEAKER_00: every developer in the world.
12:03
SPEAKER_00: There was no way to pay for the product.
12:05
SPEAKER_00: And then very quickly, enterprises started to reach out.
12:08
We were able to handle the security requirements and
12:10
personalization because the
12:11
SPEAKER_00: companies not only care about,
12:12
SPEAKER_00: hey,
12:13
SPEAKER_00: it's fast,
12:14
SPEAKER_00: it's free,
12:14
SPEAKER_00: but is this the best code for my company,
12:16
SPEAKER_00: right?
12:17
SPEAKER_00: And we were able to meet that workload.
12:19
SPEAKER_00: And then fast forward to today, and I know that this is a long answer,
12:23
What we felt was agents in the beginning of last year would be
12:26
very huge.
12:27
SPEAKER_00: The problem was the models were not there yet, right?
12:30
SPEAKER_00: We had internal,
12:31
we had teams inside the company building these agent use cases
12:34
and they were just
12:35
SPEAKER_00: not good enough.
12:36
But the middle of last year,
12:37
we were like,
12:37
hey,
12:38
it's actually going to be good enough.
12:39
SPEAKER_00: But the problem is the IDE is going to be a limitation for us.
12:42
Because VS Code is not evolving fast enough to enable us to
12:46
provide the best
12:47
SPEAKER_00: experience for end users.
12:48
SPEAKER_00: In a world in which agents were going to write 90% or 95% of software,
12:52
SPEAKER_00: developers would still be in the loop.
12:54
But the way they would interact with their IDs would look
12:56
markedly different.
12:57
SPEAKER_00: And that's why we ended up building out Windsurf in the first place.
13:01
We thought that there was a much higher ceiling on what IDEs
13:03
could provide.
13:04
SPEAKER_00: And with the agent product,
13:05
SPEAKER_00: which is Cascade,
13:06
we were able to deliver what we felt was a premier experience
13:09
right off the bat
13:10
SPEAKER_00: that we couldn't have with VS Code.
13:12
How large is the team who's working on Windsurf and how complex
13:15
is Windsurf as a product?
13:17
SPEAKER_01: I'm not sure how much we can quantify it.
13:20
SPEAKER_00: Yeah.
13:22
I try to be pretty modest with some of these things,
13:27
but just to say we're a pretty small team.
13:29
SPEAKER_00: So right now the engineering team is a bit over 50 people.
13:32
SPEAKER_00: Maybe that's large compared to other startups.
13:37
But if I were to say compared to other large engineering
13:40
projects in the grand
13:41
SPEAKER_00: scheme of things,
13:41
like one of the books that I read a while ago was this book
13:44
called Showstopper,
13:46
SPEAKER_00: right?
13:46
SPEAKER_00: And it's this book about how Microsoft built Windows NT, right?
13:50
SPEAKER_00: And it's a much larger team,
13:52
SPEAKER_00: obviously,
13:52
SPEAKER_00: but operating systems are a very complex piece of software.
13:55
SPEAKER_00: But my viewpoint on this is that this is a very,
13:58
SPEAKER_00: very complex piece of software in terms of where the goalpost is.
14:02
SPEAKER_00: Which is to say, I would say the goalpost is constantly moving, right?
14:06
One of the goals that I give to the company is that we should
14:08
be reducing the time
14:09
SPEAKER_00: it takes to build applications by 99%,
14:11
SPEAKER_00: right?
14:12
SPEAKER_00: And I would say pre-Windsurf, it was probably 20.
14:15
SPEAKER_00: And post-Windsurf, it was probably over 40.
14:17
SPEAKER_00: But we are very far from 99, right?
14:20
SPEAKER_00: We're still like, you know, a 60x away from 99, right?
14:23
Like if there's a 60 units of time and we want to make a one,
14:27
we're quite far.
14:28
So in my head,
14:29
there's a lot of different engineering projects that we have at
14:31
the company.
14:32
SPEAKER_00: In fact,
14:32
like I would say over maybe close to half of the engineering
14:35
team is working on
14:36
SPEAKER_00: projects that have not seen the light of day.
14:39
And that's an interesting decision that I guess we've made
14:42
because I think we
14:42
SPEAKER_00: cannot be embracing incremental.
14:45
We're not going to win and be a valuable company to our
14:48
customers if all we're
14:49
SPEAKER_00: doing is changing the location of buttons.
14:51
I think people will like us for great UI,
14:53
but that cannot be the only reason why we win.
14:56
SPEAKER_01: No, I love it.
14:56
I mean,
14:57
this is,
14:57
you know,
14:57
when you're a startup,
14:58
I think you need to say really big.
15:00
SPEAKER_01: You cannot just do incremental.
15:01
SPEAKER_01: You can do incremental later.
15:02
SPEAKER_01: Hopefully you're going to get there.
15:04
And what are some interesting numbers that you can share about
15:07
the usage of
15:08
SPEAKER_01: Windsurf or the load that you're handling?
15:10
SPEAKER_01: Because I'm assuming this is just going to, it's pretty easy to tell.
15:14
SPEAKER_01: It will keep going up, right?
15:16
SPEAKER_01: Yeah.
15:16
SPEAKER_01: That's an easy prediction.
15:17
SPEAKER_01: Yeah.
15:18
SPEAKER_00: No, I think you're right.
15:19
SPEAKER_00: So one of the interesting numbers I can,
15:21
SPEAKER_00: I can,
15:21
SPEAKER_00: I can,
15:22
or a handful of numbers is within a couple months of the
15:24
product existing,
15:25
SPEAKER_00: we had like well over a million developers tried the product.
15:28
SPEAKER_00: So it's been growing quite quickly within pricing coming out.
15:31
We've,
15:31
we've reached over within a month,
15:33
we reached over sort of eight figures in,
15:35
in ARR.
15:36
SPEAKER_00: And, and,
15:38
SPEAKER_00: I think all of those are kind of interesting metrics,
15:40
SPEAKER_00: but also on top of that,
15:42
SPEAKER_00: sort of we run our own model still in a lot of places.
15:44
Like you could imagine the fast passive experience is
15:46
completely our own model.
15:48
A lot of the models to go out and retrieve parts of the code
15:51
base and find relevant
15:53
SPEAKER_00: snippets are our own models.
15:54
And that system processes well over sort of 500 billion tokens
15:57
of code every day right now.
15:59
SPEAKER_00: So that system itself is huge.
16:01
SPEAKER_00: It's a huge world that we actually run.
16:04
SPEAKER_01: Yeah,
16:04
and I guess the history of Windsurf is interesting once I
16:10
understand that you've
16:11
SPEAKER_01: actually been building your own models for quite the time.
16:14
You've not just started here because I think for most
16:17
engineering teams that would be daunting.
16:19
SPEAKER_01: And also it's just a lot of time, right?
16:22
It's not something that you would just like,
16:23
it's harder to do from scratch.
16:25
SPEAKER_00: I'll say that because nothing's impossible here.
16:28
SPEAKER_00: I totally agree with you.
16:28
SPEAKER_00: I think,
16:29
SPEAKER_00: you know,
16:30
one of the weird things is because of the time that we started
16:33
and the fact that we
16:34
SPEAKER_00: were like in the very beginning,
16:35
SPEAKER_00: first of all,
16:35
SPEAKER_00: we had the infrastructure background,
16:37
but we were first saying we need to go out and build an
16:40
autocomplete model.
16:41
The best model at the time that was open source end of 2022 was
16:44
Salesforce Coget.
16:47
SPEAKER_00: And I'm not saying it was a bad model.
16:48
SPEAKER_00: It was awesome that Salesforce did open source that model,
16:51
SPEAKER_00: but it was missing a lot of capabilities that we needed for our product,
16:54
SPEAKER_00: right?
16:54
SPEAKER_00: It was missing fill in the middle,
16:56
SPEAKER_00: which feels like a very,
16:57
SPEAKER_00: very obvious capability,
16:59
SPEAKER_00: but the model...
16:59
SPEAKER_02: Fill in the middle?
17:00
SPEAKER_00: What is that?
17:01
SPEAKER_00: So the idea of fill in the middle is basically,
17:04
SPEAKER_00: if you look at the task of writing software,
17:05
SPEAKER_00: it's very different than chat.
17:07
SPEAKER_00: And maybe an example of what chat is,
17:09
you're always appending something to the very end and maybe
17:11
adding an instruction.
17:13
But the problem for writing code is you're writing code in ways
17:15
that are in the
17:16
SPEAKER_00: middle of a line,
17:17
SPEAKER_00: in the middle of a snippet of code.
17:19
SPEAKER_00: Yeah, or function, that kind of stuff, yeah.
17:21
SPEAKER_00: In the middle of a function.
17:22
SPEAKER_00: And the problem there is actually there's a lot of issues that pop up,
17:25
SPEAKER_00: which is to say,
17:26
SPEAKER_00: actually,
17:26
SPEAKER_00: the tokenization.
17:27
So these models,
17:28
when they consume files,
17:29
they actually tokenize the files,
17:30
right?
17:31
SPEAKER_00: Which is to say they don't consume them byte by byte.
17:33
SPEAKER_00: They consume them token by token.
17:35
SPEAKER_00: But the fact that the code,
17:36
SPEAKER_00: when you write it at any given point,
17:38
SPEAKER_00: doesn't tokenize into something that looks like in distribution.
17:41
SPEAKER_00: I'll give you an example.
17:42
SPEAKER_00: How many times do you think in the training data set for these models,
17:45
SPEAKER_00: does it see instead of return RETU only without the RN?
17:49
SPEAKER_00: Probably never.
17:51
SPEAKER_00: It probably never sees that.
17:52
SPEAKER_00: So it's completely out of distribution.
17:54
SPEAKER_00: But we still need to,
17:55
SPEAKER_00: when we see RETU,
17:56
SPEAKER_00: predict you are going to do RN space a bunch of other stuff.
17:59
SPEAKER_00: It sounds like a very small detail,
18:01
SPEAKER_00: but that is actually very important if you want to build the product.
18:04
And that is a capability that cannot be slightly post-trained
18:07
onto the models.
18:08
It's actually something where you need to do a non-trivial
18:10
amount of training on
18:11
SPEAKER_00: top of a model or pre-train to get that capability.
18:14
SPEAKER_00: And it was table stakes for us to provide that for our users.
18:16
So that forced us very early on to actually build out our own
18:19
models and figure out
18:20
training recipes and make sure we could run them at massive
18:22
scale ourselves for our
18:24
SPEAKER_00: end users.
18:24
And what are other things that are unique in terms of building
18:28
models for code as
18:30
SPEAKER_01: opposed to the usual text models?
18:33
SPEAKER_01: I can think of things like the brackets, for example, in some languages.
18:37
SPEAKER_01: Maybe this is just naive.
18:38
SPEAKER_01: You have seen so many more.
18:40
SPEAKER_01: So what makes code...
18:43
What makes it interesting slash worthwhile to build your own
18:47
model for code?
18:49
SPEAKER_00: Yeah, I think what you said is definitely like one thing.
18:54
SPEAKER_00: The fill in the middle capability, I would say another thing.
18:56
SPEAKER_00: Another thing you can do is code is like quite easy to,
19:00
SPEAKER_00: and quite easy is maybe,
19:01
SPEAKER_00: you know,
19:02
SPEAKER_00: an overstatement,
19:03
SPEAKER_00: but quite easy to parse,
19:04
SPEAKER_00: right?
19:05
SPEAKER_00: You can actually ASD parse code.
19:06
You can find relationships of the code because code is a system
19:10
that is like evolved over time.
19:12
You can actually look at the commit history of code to build a
19:15
knowledge graph of
19:15
SPEAKER_00: the code base.
19:17
SPEAKER_00: And you can start putting... Do you do that?
19:19
SPEAKER_00: What?
19:20
SPEAKER_00: Do you do that?
19:21
SPEAKER_00: Yep.
19:22
SPEAKER_00: Yeah.
19:22
SPEAKER_00: Yeah, yeah.
19:23
SPEAKER_00: We look at the previous commits.
19:24
And one of the things that it enables us to do is build a
19:27
probability distribution
19:28
SPEAKER_00: of the code base of conditional on you modifying a piece of code.
19:31
SPEAKER_00: What is the probability of you modifying another piece of code?
19:34
SPEAKER_00: So there is...
19:35
You know,
19:35
when you get into the weeds,
19:36
code is very information dense,
19:39
right?
19:39
SPEAKER_00: It's testable.
19:41
SPEAKER_00: There's a way that it evolves.
19:42
SPEAKER_00: People write comments,
19:43
SPEAKER_00: which is also cool,
19:45
SPEAKER_00: which is to say once a pull request gets created,
19:47
SPEAKER_00: people actually say,
19:48
SPEAKER_00: I didn't like this code.
19:48
So there's a lot of signal on what good and bad looks like
19:51
within a company.
19:52
And you can use that actually as a way to automatically make
19:56
the product much
19:57
SPEAKER_00: better for companies,
19:58
SPEAKER_00: right?
19:58
SPEAKER_00: You know,
19:58
SPEAKER_00: one of the things that I think all of us were talking about,
20:01
SPEAKER_00: I would say like a couple of years ago,
20:03
SPEAKER_00: and I guess we've been here in this space quite a long time.
20:06
SPEAKER_00: I know a couple of years is not a very long time in most categories,
20:09
SPEAKER_00: but in this category,
20:10
SPEAKER_00: it's dinosaur years.
20:11
SPEAKER_00: Yeah.
20:12
SPEAKER_00: So,
20:13
SPEAKER_00: you know,
20:15
one of the things that I think is kind of interesting is we're
20:18
in the beginning,
20:19
SPEAKER_00: we were saying,
20:20
SPEAKER_00: hey,
20:20
people would write all these guidelines and documentations on
20:23
how best to use the product.
20:24
SPEAKER_00: But the interesting thing is code is such a treasure trove.
20:26
You can go out and probably make a good first cut on what the
20:30
best way to write
20:31
SPEAKER_00: software is inside JPMorgan Chase,
20:33
SPEAKER_00: inside Dell.
20:34
SPEAKER_00: You can go out and do that by using the rich history inside the company.
20:37
So there's a lot of things that you can start doing
20:39
autonomously as well,
20:40
if that makes sense.
20:42
SPEAKER_01: Yeah.
20:43
SPEAKER_01: One thing I'd love to get your take on, on how it might have changed.
20:47
SPEAKER_01: A year or two ago,
20:48
SPEAKER_01: when using,
20:50
SPEAKER_01: when Copilot started to become more popular,
20:52
SPEAKER_01: again,
20:52
SPEAKER_01: on earlier version,
20:53
companies like Sourcegraph and others have started to build
20:56
other capabilities.
20:57
SPEAKER_01: There was this debate of, would it be worth fine tuning a platform
21:03
model on my company's code base talking about large companies
21:06
let's talk jp morgan
21:07
or those others and you know there are two trains of thoughts
21:10
one said like oh it's
21:11
probably worth it because our code is so unique it might be
21:14
worth it and some other
21:15
people will think like it might not be worth it because uh it
21:19
might be too resource
21:20
intensive the models are too generic did you try this out and
21:23
where did you land in
21:24
this because i i never got an answer to you know what happened
21:27
like what was worth
21:28
SPEAKER_01: it what was not worth it in the end
21:30
SPEAKER_00: So for what it's worth, we did try it out.
21:33
SPEAKER_00: We built out some crazy infrastructure to go out and try it out.
21:36
I guess this will be the first place where I talk about the
21:38
actual infrastructure.
21:39
SPEAKER_00: We built out systems.
21:41
SPEAKER_00: So transformers have these many layers, right?
21:43
SPEAKER_00: And if you were to imagine,
21:45
SPEAKER_00: when we actually enable companies to self-host,
21:47
SPEAKER_00: at some point in the past,
21:49
we were enabling companies to self-host the system and the
21:52
fine-tuning system as well.
21:53
SPEAKER_00: So at that time... You built this out.
22:00
SPEAKER_00: fine tuning.
22:01
SPEAKER_00: And the way that that actually worked was actually quite crazy,
22:05
SPEAKER_00: which was to say,
22:05
SPEAKER_00: okay,
22:07
where do you get the capacity to fine tune a model if you're
22:09
already running it for inference?
22:10
SPEAKER_00: Like the company may not want to give you so many GPUs.
22:13
SPEAKER_00: So we just said,
22:14
SPEAKER_00: hey,
22:14
SPEAKER_00: why don't we use the preemptible time,
22:17
SPEAKER_00: which is to say when the model is not running inference,
22:19
What if we actually go out and do back props on the transformer
22:23
model while this is happening?
22:24
SPEAKER_00: And then what we found was,
22:26
SPEAKER_00: oh,
22:26
SPEAKER_00: the back props take a long time,
22:27
SPEAKER_00: and it might cause downtime on the inference side.
22:30
So what we enabled it was we enabled the back prop to be able
22:33
to be preemptible on
22:34
SPEAKER_00: every layer of the transformer.
22:36
So if that's to say,
22:37
let's say you send an inference request,
22:39
and it's doing back propagation.
22:41
SPEAKER_00: And it's on layer 10.
22:42
It'll just stop at layer 10 and it will continue after your
22:45
inference request completes.
22:46
SPEAKER_00: So we built a lot of crazy systems to actually go out and do this.
22:50
SPEAKER_00: I guess here's the thing we found.
22:51
SPEAKER_00: We found that fine tuning was a bump,
22:54
but it was a very modest bump compared to what great
22:56
personalization and great
22:58
SPEAKER_00: retrieval could do.
22:59
SPEAKER_00: That's what we found.
23:00
Now,
23:00
does that mean fine tuning in the future is not going to be valuable?
23:03
SPEAKER_00: I think actually per person fine tuning could actually work quite well.
23:07
I think though maybe some of the techniques that we do it are
23:10
going to need to change.
23:11
SPEAKER_00: And here's the way I like to look at it, right?
23:13
For any time a system,
23:14
you know,
23:14
you build a system,
23:15
there are many ways to improve it.
23:16
SPEAKER_00: Some of them are much easier than other ways.
23:19
SPEAKER_00: And you can imagine there's a hill to climb for everything.
23:21
SPEAKER_00: And some hills are much easier.
23:23
And the right strategy to do when a hill is much easier and it
23:25
provides a lot of
23:26
value is climb that hill fully before you go out and do
23:28
something that's a lot harder.
23:30
SPEAKER_00: Because when you do the thing that's a lot harder,
23:32
you are like adding some amount of tech debt if that's not the
23:34
right solution,
23:35
SPEAKER_00: right?
23:36
What I described to you in terms of like the solution of doing
23:38
backprop on a layer
23:39
SPEAKER_00: by layer basis,
23:40
SPEAKER_00: it's a cool idea,
23:41
but you could imagine it added a lot of technical complexity to
23:44
the software that
23:45
might've been unnecessary if we thought that purely doing
23:48
better retrieval
23:49
SPEAKER_00: going to be much better.
23:49
SPEAKER_00: So there's this like,
23:50
SPEAKER_00: I guess there's this tightrope to kind of,
23:52
SPEAKER_00: you know,
23:52
SPEAKER_00: balance on top of on how you how you decide these things.
23:56
SPEAKER_01: Now,
23:57
SPEAKER_01: I was asking around,
23:58
SPEAKER_01: I've been using Windsurf as well,
23:59
SPEAKER_01: but I'm not a very heavy user,
24:00
SPEAKER_01: but I have been asking around more heavy users.
24:02
SPEAKER_01: And one of the biggest criticisms,
24:05
SPEAKER_01: both of Windsurf,
24:05
SPEAKER_01: but also of every tool in this area,
24:08
SPEAKER_01: has been like,
24:09
SPEAKER_01: look,
24:09
SPEAKER_01: I start off,
24:10
SPEAKER_01: it's good.
24:11
SPEAKER_01: It works good.
24:11
SPEAKER_01: I have a relatively small complex.
24:13
My project grows either because Windsurf generates code or is
24:16
just a big project.
24:17
SPEAKER_01: And after a while, it starts to struggle with the context.
24:20
SPEAKER_01: Maybe it doesn't see...
24:22
SPEAKER_01: you know, parts, it gets confused, et cetera.
24:25
SPEAKER_01: And clearly as an engineer,
24:28
SPEAKER_01: understand that it is going to be a problem of like,
24:30
SPEAKER_01: you have a growing context window.
24:31
SPEAKER_01: You still want to have similar quality.
24:34
SPEAKER_01: How do you tackle this challenge?
24:37
SPEAKER_01: What, what progress have you made?
24:39
SPEAKER_01: And like,
24:39
SPEAKER_01: I think this is a bit of a million dollar question in the sense of like,
24:42
SPEAKER_01: if,
24:42
SPEAKER_01: if we could somehow have a solution for this,
24:45
SPEAKER_01: we would be better off.
24:46
SPEAKER_01: Where have you gotten?
24:47
SPEAKER_01: Yeah.
24:48
SPEAKER_01: On this, I'm assuming this is a pretty common challenge and typical.
24:52
SPEAKER_00: I think it's a very hard problem.
24:54
SPEAKER_00: You're totally right.
24:55
SPEAKER_00: There's a lot of things that we can do,
24:56
SPEAKER_00: which is to say,
24:57
SPEAKER_00: obviously,
24:58
we need to work around the fact that the models don't have
25:00
infinite context.
25:01
SPEAKER_00: And when they do have larger and larger context,
25:03
SPEAKER_00: you are paying a lot more and you take a lot more time.
25:06
SPEAKER_00: Right.
25:07
SPEAKER_00: And developers usually a lot of the time don't really want to wait.
25:09
SPEAKER_00: And, you know, one of the things that we have for our products.
25:12
SPEAKER_01: We hate waiting.
25:13
SPEAKER_00: Yeah, exactly.
25:14
But one of the things that we have for our products that we've
25:16
learned is if you
25:16
SPEAKER_00: make a developer wait,
25:17
SPEAKER_00: the answer better be 100% correct.
25:19
And I don't think we're at a time right now where I can
25:21
guarantee you with like a
25:23
SPEAKER_00: magic wand that all of our cascade responses are 100% correct,
25:27
SPEAKER_00: right?
25:27
SPEAKER_00: I don't think we're at that right now.
25:29
So there's a lot of things that we need to do that are almost approximations,
25:32
right?
25:33
SPEAKER_00: How do we keep a very large context?
25:35
But despite that,
25:36
we have chats that are so long that how do you accurately
25:38
checkpoint the past
25:41
SPEAKER_00: But that has some natural lossiness attached to it, right?
25:44
SPEAKER_00: And then similarly,
25:46
SPEAKER_00: if the code base gets very large,
25:47
SPEAKER_00: how do we get very,
25:48
SPEAKER_00: very confident that the retrieval is very good?
25:50
SPEAKER_00: And we have evaluations for all of these things, right?
25:52
This is not something which we're like shooting in the dark and
25:54
being like,
25:55
SPEAKER_00: hey,
25:55
SPEAKER_00: YOLO,
25:56
SPEAKER_00: let's try a new approach and like give it to half of our users.
25:58
SPEAKER_00: But I think you're totally right.
26:00
SPEAKER_00: There's no...
26:01
SPEAKER_00: I don't think there's like a complete solution for it.
26:03
SPEAKER_00: What I think it's going to be is like a mixture of a bunch of things,
26:06
which is to say much better checkpointing coupled with better
26:09
usage of context length,
26:11
SPEAKER_00: much faster LMs and much better models.
26:13
SPEAKER_00: So it's going to be, it's not going to be, I think a silver bullet.
26:15
SPEAKER_00: And by the way, that could be tied with, Hey, you know,
26:19
understanding you know understanding the code base much better
26:23
from the perspective
26:24
of the code base already existed able to use the knowledge
26:26
graph right able to use
26:28
a lot of the dependencies within the code base a lot better so
26:30
it's a bunch of
26:31
things that i think are going to multiply together to solve the
26:33
problem i don't
26:33
think there's going to be like one silver bullet that makes it
26:35
so you're going to
26:36
be able to have amazingly coherent conversations that are very
26:39
very long basically
26:41
to be fair as an engineer it kind of this might feel weird but
26:44
it makes me feel a
26:45
SPEAKER_01: bit better than
26:47
We're actually back to talking about like engineering step by
26:49
step as opposed to like,
26:50
SPEAKER_01: okay,
26:51
SPEAKER_01: you know,
26:51
SPEAKER_01: having these,
26:52
SPEAKER_01: you know,
26:52
SPEAKER_01: it feels like you get a new model.
26:53
Not now,
26:54
but early on when we got a new model,
26:55
it was like,
26:56
oh my gosh,
26:56
it's magic.
26:57
And it took a while to understand how it works,
27:00
how it's broken down,
27:01
et cetera.
27:02
SPEAKER_01: You did mention your infrastructure.
27:04
Can you talk a little bit about how we can imagine your
27:07
hardware and backend stack
27:09
SPEAKER_01: if I was to join Windsurf as an engineer?
27:13
SPEAKER_01: Is it going to be a bunch of cloud deployments here and there?
27:17
SPEAKER_01: Do you self-host some of your GPUs?
27:19
SPEAKER_01: Because a lot of AI startups who are smaller or more modest,
27:23
SPEAKER_01: they're just going to be a platform as a service.
27:26
It sounds like you might be at the scale where maybe you're
27:28
outgrowing this as well.
27:30
SPEAKER_00: Yeah,
27:31
SPEAKER_00: I think we might have just never done kind of,
27:33
SPEAKER_00: you know,
27:34
SPEAKER_00: buying off the shelf stuff in the early part of the company.
27:37
SPEAKER_01: Your background, I keep forgetting this.
27:40
SPEAKER_00: Yeah,
27:40
SPEAKER_00: but even more than the background,
27:41
SPEAKER_00: I think there were cases where we could have and maybe should have.
27:44
SPEAKER_00: One of the reasons why...
27:46
We also didn't was very quickly,
27:47
we got brought into working with very large enterprises.
27:50
SPEAKER_00: And I think the more dependencies you have in your software,
27:54
it just makes it harder and harder for these larger companies
27:56
to integrate the technology,
27:57
SPEAKER_00: right?
27:58
SPEAKER_00: Like they don't want a ton of sub processors attached to it, right?
28:01
SPEAKER_00: We recently got FedRAMP compliant, FedRAMP high compliance.
28:04
SPEAKER_00: We're the only AI software assistant with FedRAMP high compliance.
28:08
And the only reason why that's the case is we've kept our
28:11
systems like very tight.
28:14
SPEAKER_01: For these compliances, I did some, but not specifically FedRAMP.
28:17
SPEAKER_01: What do you need to prove that you are this compliant?
28:21
Yeah,
28:21
I think basically you need to map out the high levels of all
28:26
the interactions.
28:27
You need to be very methodical about releases and how the
28:30
releases make it into the system.
28:32
You need to be very methodical about where data has persisted
28:35
at a layer that is
28:36
SPEAKER_00: probably much deeper than SOC 2.
28:38
SPEAKER_00: I think going through the SOC 2 versus FedRAMP...
28:41
SPEAKER_01: I did talk to you and that was already pretty long.
28:46
SPEAKER_00: It's impressive that you did this as a startup slash scale.
28:49
SPEAKER_00: Congrats.
28:50
SPEAKER_00: Yeah.
28:51
SPEAKER_00: One of the reasons why was,
28:52
SPEAKER_00: I guess,
28:52
like one of our first customers that were a large enterprise
28:55
was like Dell,
28:55
SPEAKER_00: right?
28:56
SPEAKER_00: Which is like not a usual first large enterprise.
28:59
SPEAKER_00: And I guess- For a startup, no.
29:00
SPEAKER_00: For a startup, definitely no.
29:01
SPEAKER_00: So it forces down a path of how do we build very scalable infrastructure?
29:05
How do we make sure our systems work at a code base that is 100
29:08
plus million lines of code?
29:09
SPEAKER_00: What does our GPU provisioning need to look like for this larger team?
29:13
SPEAKER_00: It just forces us to become a lot more,
29:14
SPEAKER_00: I guess,
29:15
SPEAKER_00: operationally sound for these kinds of problems,
29:18
SPEAKER_00: if that makes sense.
29:19
SPEAKER_01: Yeah.
29:20
SPEAKER_01: And how do you deal with inference?
29:24
You're serving the systems that serve probably billions or
29:28
hundreds of billions,
29:29
SPEAKER_01: well,
29:29
SPEAKER_01: actually hundreds of billions tokens per day,
29:31
SPEAKER_01: as you just said,
29:32
SPEAKER_01: with low latency.
29:33
SPEAKER_01: What...
29:35
SPEAKER_01: Smart approaches do you do to do this?
29:37
SPEAKER_01: What kind of optimizations have you looked into?
29:40
SPEAKER_00: Yeah, I mean, like a lot, as you can imagine.
29:43
SPEAKER_00: One of the interesting things about some of the products that we have,
29:45
SPEAKER_00: like the passive experience,
29:46
latency matters a ton in a way that's like very different than
29:49
some of these API providers.
29:51
SPEAKER_00: I think for the API providers,
29:52
SPEAKER_00: time to first token is important,
29:54
SPEAKER_00: but it doesn't matter that time to first token is 100 milliseconds.
29:57
SPEAKER_00: For us, that's the bar we are trying to look for.
30:00
Can we get it to sub a couple hundred milliseconds and then
30:04
hundreds of tokens a
30:04
SPEAKER_00: second for the generation time?
30:07
So much faster than what all of the providers are providing in
30:10
terms of throughput as well.
30:12
SPEAKER_00: just because of how quickly we want this product to kind of run.
30:14
SPEAKER_00: And you can imagine there's a lot of things that we want to do, right?
30:17
SPEAKER_00: How do we run?
30:18
SPEAKER_00: How do we do things like speculative decoding?
30:20
SPEAKER_00: How do we do things like model parallelism, right?
30:22
How do we make sure we can actually batch requests properly to
30:25
get the maximum
30:26
SPEAKER_00: utilization of the GPU all the while not hurting latency,
30:30
SPEAKER_00: right?
30:30
SPEAKER_00: That's an important thing.
30:31
And one of the interesting things just to give some of the
30:34
listeners some a mental model,
30:36
SPEAKER_00: GPUs are amazing.
30:37
SPEAKER_00: They have a lot of compute.
30:39
SPEAKER_00: if I were to draw an analogy to CPUs,
30:42
SPEAKER_00: GPUs have over sort of two orders of magnitude more compute than a CPU,
30:46
SPEAKER_00: right?
30:46
SPEAKER_00: It might actually be more on the more recent GPUs, but keep that in mind.
30:50
But GPUs only have an order of magnitude more memory bandwidth
30:53
than a CPU.
30:54
So what that actually means is if you do things that are not
30:56
compute intense,
30:57
SPEAKER_00: you will be memory bound,
30:59
SPEAKER_00: right?
30:59
So that necessarily means to get the most out of the compute of
31:02
your processor,
31:03
SPEAKER_00: you need to be doing a lot of things in parallel.
31:05
SPEAKER_00: But if you need to wait to do a lot of things in parallel,
31:08
SPEAKER_00: you're going to be hurting the latency.
31:09
SPEAKER_00: So there's all of these different trade-offs that we need to make.
31:13
to ensure a quality of experience for our users that we think
31:15
is high for the product.
31:17
SPEAKER_00: And we've obviously mapped out all of these.
31:19
SPEAKER_00: We've seen how,
31:20
SPEAKER_00: hey,
31:20
SPEAKER_00: like if we change the latency by this much,
31:23
SPEAKER_00: what is this change in terms of people's willingness to use the product?
31:26
SPEAKER_00: And it's very stark, right?
31:27
Like a 10 millisecond increase in latency affects people's
31:30
willingness to use the
31:31
SPEAKER_00: product materially,
31:32
SPEAKER_00: right?
31:33
SPEAKER_00: It's percentage points that we're talking about.
31:34
So these are all parts of the inference stack that we've needed
31:37
to optimize.
31:38
SPEAKER_01: And...
31:39
Is latency important enough or does the location factor factor
31:43
into this?
31:44
Physically how close people using Windsurf are to wherever your
31:50
server and then
31:51
SPEAKER_01: your GPUs are running.
31:52
SPEAKER_01: Do you need to worry about that as well?
31:54
SPEAKER_00: You do need to worry about that.
31:55
SPEAKER_00: Speed of light starts mattering.
31:58
interestingly you know this is not something i would have
32:00
expected but we do have
32:01
users in india and interestingly the speed of light is not
32:04
actually what is
32:05
bottlenecking their uh their performance it's actually the
32:09
local network so just
32:10
the time it takes for the packet to get from maybe like from
32:13
their home to the
32:14
major isp is actually is somehow there's a lot of congestion
32:18
there and that's the
32:19
SPEAKER_00: kind of stuff that we need to kind of deal with um but
32:22
SPEAKER_00: By the way, that is something that we just cannot solve right now.
32:24
SPEAKER_00: So you're totally right.
32:25
SPEAKER_00: The data center placement matters.
32:27
SPEAKER_00: Like for instance,
32:28
SPEAKER_00: if you force a data center in Sydney and you have people in Europe,
32:31
SPEAKER_00: they're not going to be happy about the latency.
32:33
So we do think about like where the location of our GPUs are to
32:36
make sure that we
32:37
SPEAKER_00: do have good performance.
32:38
But there are some places where there are some issues that even
32:40
we can't get around basically.
32:42
SPEAKER_01: Now,
32:43
SPEAKER_01: the last time I heard this complaint before Windsurf,
32:45
SPEAKER_01: because this came up with actually,
32:47
SPEAKER_01: again,
32:48
someone who's using Windsurf and other tools a lot said that
32:51
specifically for one
32:52
SPEAKER_01: of the tools,
32:53
SPEAKER_01: he can tell that the data centers are far away because it's just slow.
32:58
Cloud development environments had the exact same thing because
33:01
they were similar,
33:02
right?
33:02
SPEAKER_01: Like this was,
33:03
SPEAKER_01: I'm not sure they're as popular right now,
33:04
SPEAKER_01: but there was a time where it looked like it might be the future.
33:06
SPEAKER_01: You just log on to your
33:08
SPEAKER_01: remote environment, which is running on CPUs or GPUs somewhere else.
33:11
SPEAKER_01: And again,
33:12
SPEAKER_01: I think it might have to do with,
33:14
SPEAKER_01: as does when you're typing,
33:15
SPEAKER_01: like when I'm using it,
33:17
SPEAKER_01: I mean,
33:17
SPEAKER_01: I'm just used to,
33:18
SPEAKER_01: like I do want sub-second,
33:20
SPEAKER_01: probably like a few hundred milliseconds.
33:22
SPEAKER_01: I just noticed, right?
33:23
SPEAKER_01: You feel it's slow and it just bothers you.
33:25
SPEAKER_01: Like it just...
33:28
SPEAKER_00: No, I agree.
33:29
SPEAKER_00: I think if I had to,
33:30
SPEAKER_00: if I had to even see every time I typed a keystroke,
33:33
SPEAKER_00: like a couple hundred milliseconds later,
33:34
SPEAKER_00: the key would show up,
33:35
SPEAKER_00: like I would rage quit.
33:37
SPEAKER_00: That would be a terrible experience for me.
33:39
SPEAKER_01: How do you deal with indexing of the code?
33:42
So you're going to be indexing,
33:43
you know,
33:44
depends on the code base,
33:45
it'll be more or less.
33:45
But if you add it up,
33:46
I'm sure we're talking billions or a lot more in code.
33:49
SPEAKER_01: And for your enterprise customers,
33:51
SPEAKER_01: you might actually have,
33:53
SPEAKER_01: you know,
33:53
SPEAKER_01: the hundreds of millions or even more lines of code.
33:57
SPEAKER_01: Is there anything like novel or interesting that you're using?
34:01
Or is it just kind of tried and proven things,
34:04
for example,
34:05
that search engines might use?
34:06
SPEAKER_01: Sure.
34:07
SPEAKER_00: It's a little bit of both, to be honest.
34:09
SPEAKER_00: And what I mean by that, it's not a very clean answer.
34:12
SPEAKER_00: We do try approaches that are embedding based.
34:15
SPEAKER_00: We have approaches that are keyword based on the indexing.
34:18
SPEAKER_00: Interestingly,
34:19
SPEAKER_00: actually,
34:20
SPEAKER_00: one of the approaches that we've taken that's very different than search,
34:22
SPEAKER_00: and maybe actually systems like Google actually do this,
34:25
SPEAKER_00: is we not only actually look at just the retrieval,
34:29
SPEAKER_00: we do a lot of computation at retrieval time.
34:32
SPEAKER_00: So what that means is, let's say you want to go out and ask a question.
34:35
SPEAKER_00: You know,
34:36
one of the things that you can go out and do is like ask it to
34:38
an embedding store
34:39
SPEAKER_00: and get a bunch of locations.
34:41
SPEAKER_00: What we found was the recall of that operation was quite low.
34:44
And,
34:45
you know,
34:45
one of the reasons why that happens is embedding search is a
34:49
little bit lossy.
34:50
Like,
34:50
let's say I was to go to a code base and ask,
34:53
hey,
34:53
give me all cases.
34:55
SPEAKER_00: where this function, this Spring Boot version X type function was there.
35:00
I don't think anyone would believe embedding search would be comprehensive.
35:03
SPEAKER_01: Yeah.
35:03
SPEAKER_00: Right?
35:04
SPEAKER_00: Because it's just not,
35:05
SPEAKER_00: like,
35:05
you're taking something that is very high dimensionality and
35:08
reducing it to
35:08
SPEAKER_00: something very low dimensionality.
35:10
SPEAKER_00: Yeah.
35:10
SPEAKER_00: without any knowledge of the question.
35:12
SPEAKER_00: That's like the most important thing.
35:13
So it needs to somehow encode all the possible,
35:17
be relevant for all the possible questions.
35:19
SPEAKER_00: So instead,
35:19
what we decided to do is take a variety of approaches to
35:22
retrieve a large amount of data,
35:24
SPEAKER_00: and that it could include the knowledge graph,
35:25
SPEAKER_00: that it could include the dependencies from the abstract syntax tree,
35:28
SPEAKER_00: that could include keyword search,
35:31
SPEAKER_00: that could include embedding search,
35:32
SPEAKER_00: and you kind of fuse them all together.
35:33
SPEAKER_00: And then after that,
35:34
we throw compute at this and actually go out and process large
35:38
chunks of the code
35:39
SPEAKER_00: base at...
35:40
at inference time and go out and like say,
35:43
hey,
35:43
these are the most relevant snippets.
35:45
SPEAKER_00: And this gives us much higher precision recall, right?
35:48
SPEAKER_00: On the retrieval side to actually go out.
35:50
SPEAKER_00: And by the way,
35:51
that is like very important for an agent because imagine if an
35:53
agent kind of like
35:54
SPEAKER_00: doesn't have access and doesn't deeply understand the code base.
35:57
All the while the code base is much larger than the context
35:59
length of what an agent
36:00
SPEAKER_00: is able to take in,
36:01
SPEAKER_00: right?
36:01
SPEAKER_00: So we've,
36:02
SPEAKER_00: you know,
36:02
optimizing the precision recall of the system is actually
36:04
something that we spent a
36:05
SPEAKER_00: lot of time and built a lot of systems for.
36:08
SPEAKER_01: It's interesting because it feels like you're,
36:10
SPEAKER_01: it shows how,
36:11
SPEAKER_01: well,
36:11
SPEAKER_01: A,
36:12
SPEAKER_01: it's code,
36:12
SPEAKER_01: so you can more easily work with it,
36:14
SPEAKER_01: especially with certain keywords,
36:15
SPEAKER_01: for example,
36:16
SPEAKER_01: on some languages.
36:17
SPEAKER_01: I can imagine that you can even just,
36:18
SPEAKER_01: you know,
36:18
you can even list all the keywords that are pretty common and
36:21
you can decide if
36:21
SPEAKER_01: it's a keyword or if it's something special.
36:24
SPEAKER_01: And if it's a keyword, you can already just like do it.
36:27
And it's interesting how you can combine the kind of old school
36:30
or old school
36:31
SPEAKER_01: before LMs and then add the best parts of LMs,
36:34
SPEAKER_01: but not forgetting about the,
36:36
SPEAKER_01: you know,
36:36
SPEAKER_01: what worked before.
36:41
SPEAKER_01: I wonder if there's any other industry that has this,
36:45
that we do have this lower dimensionality space in terms of the
36:49
grammar and all
36:50
SPEAKER_01: these things.
36:51
SPEAKER_01: We understand the usage pretty well.
36:52
SPEAKER_01: And then the users are power users who actually,
36:55
SPEAKER_01: the same people use it who could actually build this tool.
36:59
SPEAKER_00: Yeah,
37:01
I feel like Google's system is probably ridiculously complex
37:04
and sophisticated for
37:06
SPEAKER_00: obvious reasons,
37:06
SPEAKER_00: just because for one,
37:08
they've been doing this for so long and obviously they've been
37:12
at the top for such
37:13
SPEAKER_00: a long time.
37:13
SPEAKER_00: And then also on top of that,
37:15
The monetary value they get from delivering great search is so
37:19
high given ads that
37:21
SPEAKER_00: they are incentivized to throw a lot of compute even at the query time,
37:25
SPEAKER_00: right?
37:25
SPEAKER_00: To make sure that the quality of suggestions is really good.
37:28
SPEAKER_00: So I assume they're doing a lot of tactics there.
37:30
SPEAKER_00: Obviously, I'm not privy to all the details of the system.
37:34
SPEAKER_00: So I don't know.
37:35
SPEAKER_01: Well,
37:35
SPEAKER_01: it's interesting because I would have agreed with you until recently,
37:38
SPEAKER_01: but there are some search engines that are doing really good results.
37:42
So I wonder if Google is less focused on the haystack and the
37:47
needle and maybe more
37:48
SPEAKER_01: on revenue,
37:48
SPEAKER_01: or maybe they're doing it as unvisible.
37:50
SPEAKER_01: I'm sure they're doing an amazing job,
37:51
SPEAKER_01: by the way,
37:52
SPEAKER_01: behind the hood,
37:52
SPEAKER_01: but I wonder if some of that knowledge has commoditized,
37:54
SPEAKER_01: but we'll see.
37:56
SPEAKER_01: But moving on from indexing, in terms of databases,
38:00
What kind of databases do you use and what challenges are they
38:02
giving you?
38:03
SPEAKER_01: Like,
38:03
SPEAKER_01: again,
38:03
I'm assuming you're not just going to be happy with like the
38:07
usual let's know
38:08
SPEAKER_01: everything in Postgres or do you actually?
38:12
SPEAKER_01: You might be able to.
38:12
SPEAKER_01: I don't know.
38:13
It sounds like these days Postgres can be used surprisingly
38:16
well for even embeddings.
38:17
SPEAKER_00: Yeah, you know, I think we do a combination of things.
38:21
SPEAKER_00: So we do like some amount of local indexing.
38:23
SPEAKER_00: We do some remote indexing as well.
38:26
SPEAKER_00: Local indexing?
38:27
SPEAKER_00: On the user's machine.
38:28
SPEAKER_00: Nice.
38:28
SPEAKER_00: On the user's machine.
38:29
That helps us get,
38:31
in some ways,
38:31
the benefit of that is it helps you build up.
38:33
SPEAKER_00: If you were to say, hey, you have some understanding of the code base.
38:37
The problem is that understanding changes very quickly as the
38:39
user starts changing code,
38:41
SPEAKER_00: starts checking out new branches.
38:42
And you don't want to basically say all of your information
38:45
about the code base you
38:46
SPEAKER_00: need to throw away because of that.
38:47
So it's good to have some information about the user's history
38:51
and what they've
38:52
SPEAKER_00: done locally stored.
38:54
In terms of remote,
38:55
I think it would be a lot simpler than people would imagine.
38:57
SPEAKER_00: One of the complexities of our product,
38:59
SPEAKER_00: the reason why the product is very complex,
39:01
SPEAKER_00: is actually the fact that we need to run all of this GPU infrastructure.
39:05
SPEAKER_00: Right now,
39:05
SPEAKER_00: that's actually a large chunk of the complexity,
39:07
SPEAKER_00: because if you were to look at our QPS,
39:09
SPEAKER_00: our QPS is high,
39:10
SPEAKER_00: but it is not like tens of thousands of QPS.
39:13
SPEAKER_00: right?
39:14
SPEAKER_00: Actually,
39:14
SPEAKER_00: it's not,
39:15
SPEAKER_00: it doesn't need to be that high because in some ways,
39:18
like actually each of the queries that is happening is actually
39:21
a really expensive query.
39:22
SPEAKER_00: It's doing trillions of operations remotely.
39:24
So actually the complexity of the problem is how do you
39:27
optimally do that process,
39:29
right?
39:29
SPEAKER_00: So we can actually get away with things like Postgres.
39:31
SPEAKER_00: Like we're not,
39:32
SPEAKER_00: in fact,
39:33
I would say I like to keep things pretty simple if it's
39:35
possible to keep things
39:36
SPEAKER_00: very simple and
39:37
SPEAKER_00: We should not be rolling any type of our own database.
39:40
SPEAKER_00: Like I think databases are very, very complex pieces of technology.
39:43
SPEAKER_00: I think we're good engineers,
39:44
but we're definitely not good enough to kind of like on the
39:47
side build our own database.
39:48
SPEAKER_01: And then for local industry, what database do you use?
39:52
Yeah,
39:52
we have our own combination of like sort of like SQL based database.
39:55
We have a local SQL database and then like some sort of
39:58
embedding databases as well
39:59
SPEAKER_00: that we store locally as well.
40:02
SPEAKER_01: What is your view on the value of embedding databases?
40:05
SPEAKER_01: This has been an ongoing debate for the past, since Chad GBC became big.
40:09
SPEAKER_01: Again, there were two schools of thoughts.
40:11
One is we do need embedding databases because they can give us
40:15
vector search.
40:16
They can give us all these other features that LLMs and
40:19
embeddings will need.
40:20
And the other school of thought is,
40:21
well,
40:21
let's just expand relational databases.
40:23
SPEAKER_01: We add a few extra indexes and boom, we're done.
40:26
You're more of a user of this,
40:28
but you're a heavy user at Windsurf and Codium.
40:33
SPEAKER_01: what pros and cons are you seeing?
40:35
SPEAKER_01: I'm just trying to get you to like go to one direction or the other.
40:38
SPEAKER_00: It's a good question.
40:39
SPEAKER_00: So my,
40:40
SPEAKER_00: our viewpoint on embeddings are probably that they are not,
40:43
SPEAKER_00: they don't solve a problem by themselves.
40:45
SPEAKER_00: They actually just do not.
40:47
SPEAKER_00: So the answer is going to be mixed.
40:49
SPEAKER_00: And then the question is, why do we even do it in the first place?
40:51
SPEAKER_00: Right.
40:51
SPEAKER_00: And I think it really boils down to it's a, it's a recall problem, right?
40:55
SPEAKER_00: When you want to do good retrieval, you need the input of,
40:58
SPEAKER_00: to what you're willing to consider to be large and high recall, right?
41:02
SPEAKER_00: And if you were to think about it,
41:03
the problem is if you only have something like keyword search
41:07
and you have a very,
41:07
SPEAKER_00: very large sort of code base,
41:10
SPEAKER_00: actually what happens if the user typos something?
41:13
SPEAKER_00: right?
41:13
SPEAKER_00: Then your recall is going to be bad.
41:15
So the way I like to think about it is each of these approaches,
41:17
keyword search,
41:18
right?
41:19
Like sort of knowledge graph-based retrieval,
41:21
all of them,
41:22
they're all in different circles.
41:23
And what you're trying to do is get something where the union
41:26
of these circles is
41:27
SPEAKER_00: going to give you the highest recall ultimately for the retrieval query.
41:31
And I think embedding can give you good recall because it is
41:34
able to summarize or
41:37
actually able to distill somewhat a semantic information about
41:40
the chunk of code
41:41
SPEAKER_00: the AST or the file or the directory and all this other stuff.
41:44
SPEAKER_00: So what I would say is it's a tool in the toolkit, right?
41:47
It's not like you cannot build our product entirely with an
41:50
embedding system,
41:51
SPEAKER_00: but also does the embedding system help?
41:52
SPEAKER_00: I think it actually does help, right?
41:54
SPEAKER_00: It does improve our recall metrics and our precision metrics.
41:58
SPEAKER_01: So I talked with your head of research,
42:01
SPEAKER_01: Nicholas Moy,
42:02
SPEAKER_01: and he told me about a really interesting challenge that you're facing,
42:05
SPEAKER_01: which he called the split brain situation.
42:08
He was basically saying that it's almost like the team and
42:11
everyone on the team
42:12
SPEAKER_01: needs to have two brains.
42:13
SPEAKER_01: One is just being aggressively in the present,
42:15
SPEAKER_01: shipping improvements as you go,
42:16
but also then do a long-term vision where you're building for
42:19
the long run.
42:20
SPEAKER_01: How...
42:22
SPEAKER_01: Do you do this?
42:23
SPEAKER_01: How do you start doing it and how do you keep doing it?
42:25
SPEAKER_01: You did mention earlier that half the team is working on other stuff,
42:28
SPEAKER_01: but do you split people so people focus on short-term,
42:32
SPEAKER_01: long-term?
42:33
Or does everyone,
42:35
including you,
42:35
juggle these things in your head day-to-day?
42:38
SPEAKER_01: It's an interesting one.
42:40
SPEAKER_00: Yeah, I don't want to give myself that much credit here.
42:42
SPEAKER_00: I think our engineers probably should be given most of the credit here.
42:46
SPEAKER_00: But I think in terms of maybe company strategic direction,
42:49
SPEAKER_00: both me and my co-founder,
42:50
SPEAKER_00: the CTO of the company,
42:53
SPEAKER_00: we try to think a lot about how do we disrupt ourselves?
42:57
SPEAKER_00: Right.
42:58
Because I think it's very easy to get into the state where,
43:01
hey,
43:01
I added this cool button.
43:02
I added this like way to control X with a knob and you keep
43:06
going down this path
43:07
SPEAKER_00: and your users get very happy.
43:09
SPEAKER_00: But what happens if tomorrow I told you users don't need to do that?
43:12
SPEAKER_00: And it's an amazing experience and it's like a better experience.
43:16
SPEAKER_00: Users are going to feel like, why do I need to do this?
43:18
SPEAKER_00: So here's the thing.
43:20
SPEAKER_00: Users are right up to a certain point.
43:23
SPEAKER_00: right?
43:23
They will not be able to see,
43:24
like,
43:26
by the way,
43:26
if they can,
43:27
then we should not be doing this.
43:28
They will not be able to see exactly what the future solution
43:30
should be,
43:31
right?
43:31
SPEAKER_00: If our users can see the future solution better than we can,
43:34
SPEAKER_00: like,
43:35
SPEAKER_00: we should just pack up our bags and leave,
43:36
SPEAKER_00: right?
43:37
SPEAKER_00: At that point, like, what are we actually doing here?
43:39
So I think basically,
43:40
you know,
43:40
you have this tension here where you need to build
43:43
SPEAKER_00: features to make the product more usable today, right?
43:46
SPEAKER_00: And our users are 100% right.
43:48
SPEAKER_00: They understand this.
43:49
They face pain through many different axes that we don't,
43:52
and we should listen to them.
43:53
SPEAKER_00: But also at the same time,
43:55
we might have an opinionated stance on where coding and where
43:58
these models and
43:59
SPEAKER_00: where this product can go that we should go out and build towards.
44:02
And we should be expounding a large amount of our engineering
44:06
capital on that.
44:09
SPEAKER_01: Can you talk about some kind of bets that you're having?
44:13
SPEAKER_01: Not that you're giving away everything,
44:15
SPEAKER_01: but some promising directions that might or might not work out.
44:18
Or even in the past,
44:19
some promising directions that maybe did not work out.
44:21
SPEAKER_00: Yeah, I'll tell you a lot of them.
44:24
SPEAKER_00: So we failed a lot.
44:25
SPEAKER_00: And I think failing is great.
44:28
And one of the things that I tell our engineers is like
44:30
engineering is not like a
44:31
SPEAKER_00: factory building,
44:32
SPEAKER_00: right?
44:33
SPEAKER_00: It's actually,
44:33
SPEAKER_00: you know,
44:34
SPEAKER_00: you have a hypothesis,
44:35
SPEAKER_00: you go in and you shouldn't be penalized if you failed.
44:38
SPEAKER_00: Actually, I love the idea of, hey, an idea sounds interesting.
44:42
SPEAKER_00: We tried it and it didn't work because we at least learned something.
44:45
SPEAKER_00: And learning something is awesome.
44:46
SPEAKER_00: And I'll give you an example.
44:47
The agent work that we did for,
44:49
we didn't even start beginning of last year.
44:50
SPEAKER_00: We started even before beginning of last year.
44:52
SPEAKER_00: It was not working for many months.
44:54
SPEAKER_00: And actually,
44:55
SPEAKER_00: Nick Moy was working on,
44:56
SPEAKER_00: who you probably spoke with,
44:58
SPEAKER_00: was the one who was working on some of this stuff.
45:00
SPEAKER_00: And for a long time, a lot of what he was doing was just not working.
45:04
SPEAKER_00: And you would come to us and we would say, okay, fine.
45:07
SPEAKER_00: It doesn't seem like it's working.
45:08
SPEAKER_00: So we're definitely not going to ship this, but let's keep doing it.
45:11
Let's keep working on it because we believe it's going to get
45:14
better and better.
45:14
SPEAKER_00: But it was failing for a long time, right?
45:17
SPEAKER_00: We came out with a review product,
45:19
SPEAKER_00: right,
45:20
SPEAKER_00: in beginning of last year or around then called Forge for code reviews.
45:24
We thought it was kind of useful internally at the company and
45:27
we thought we could
45:27
SPEAKER_00: continue to improve it.
45:29
SPEAKER_00: People did not find it that useful.
45:30
SPEAKER_00: Right.
45:31
SPEAKER_00: It was not actually that useful.
45:32
And,
45:33
you know,
45:33
this was we were going in with the assumption code reviews take
45:36
a long time.
45:37
SPEAKER_00: What if we could help people?
45:38
And the fact of the matter was the way we thought we could help
45:41
people wasn't
45:41
SPEAKER_00: actually material enough for people to want to take on this new tool.
45:45
SPEAKER_00: Right.
45:46
And there's a lot of things that that sort of obviously that
45:49
we've tried in the
45:50
SPEAKER_00: past that just didn't work the way we the way we thought it did.
45:53
SPEAKER_00: And,
45:53
SPEAKER_00: you know,
45:54
SPEAKER_00: for me,
45:54
I think I would be totally fine if 50 percent of the bets we
45:57
make don't work.
45:58
SPEAKER_01: Yeah.
45:58
SPEAKER_01: And it's a lot of startups say that.
46:01
SPEAKER_01: And then after a while,
46:02
SPEAKER_01: what I noticed is as a company becomes bigger,
46:04
SPEAKER_01: I saw this as Uber,
46:05
SPEAKER_01: it's actually not really the case.
46:07
There's like failures kind of on paper it's embraced,
46:11
but actually it's not.
46:12
SPEAKER_01: So I think,
46:12
SPEAKER_01: you know,
46:12
SPEAKER_01: like there's this tricky thing that when it's actually meant,
46:15
SPEAKER_01: like it's awesome.
46:16
Otherwise people just like start to like polish things and make
46:19
things look good
46:20
SPEAKER_01: when they're not pretend that it's not a failure,
46:22
SPEAKER_01: but it was a success.
46:23
SPEAKER_01: And we're just walking away that kind of stuff.
46:25
SPEAKER_01: So yeah.
46:25
SPEAKER_01: It's nice to see that you're doing it.
46:27
What was the thing that turned the agents around,
46:30
which then I assume became Cascade?
46:32
SPEAKER_01: Like, was it a breakthrough on your end?
46:35
SPEAKER_01: Was it the models getting better?
46:36
SPEAKER_01: Was it a mix of something else?
46:38
SPEAKER_00: Yeah, I think it was a handful of things.
46:39
SPEAKER_00: So I'll walk through it.
46:41
SPEAKER_00: So first of all, the models got better.
46:42
SPEAKER_00: 100% the models got better.
46:44
SPEAKER_00: I think even with all the internal breakthroughs we had,
46:46
SPEAKER_00: if the models hadn't gotten better,
46:47
SPEAKER_00: we wouldn't have been able to release it.
46:48
SPEAKER_00: So I don't want to trivialize that matter.
46:51
SPEAKER_00: It was huge.
46:52
The two other pieces that were quite important was our
46:55
retrieval stack was also
46:56
SPEAKER_00: getting better and better,
46:57
SPEAKER_00: which I think enabled us to work much better at these larger code bases.
47:00
SPEAKER_00: I guess table stakes, it's quite good at zero to one programming.
47:04
But I think the thing that was groundbreaking to us was our
47:08
developers on a complex
47:10
SPEAKER_00: code base were getting a lot of value from it,
47:13
SPEAKER_00: right?
47:13
SPEAKER_00: And I would say something quite interesting,
47:15
which is that ChatGPT by itself wasn't incredibly
47:19
groundbreaking to our developers.
47:21
SPEAKER_00: inside the company.
47:22
SPEAKER_00: And that's not because ChatGPT is not a very useful product.
47:25
SPEAKER_00: ChatGPT is a ridiculously useful product.
47:28
It's actually just because you need to think about it from the
47:30
perspective of
47:31
SPEAKER_00: opportunity cost and how much more efficient you get.
47:33
SPEAKER_00: Our developers, a lot of them have been developers in the past.
47:36
SPEAKER_00: They are quite, I think we do have an exceptional engineering team.
47:39
They were used to how to use Stack Overflow and all these other
47:42
tools to get what they wanted.
47:44
SPEAKER_00: So, but
47:45
SPEAKER_00: Suddenly,
47:46
when the model had the capability to not only understand your
47:49
code base and start
47:50
SPEAKER_00: to make larger and larger changes,
47:51
SPEAKER_00: it changed the behavior of the people inside the company.
47:55
And not only make changes,
47:57
we built systems to very quickly edit the code,
47:59
right?
47:59
SPEAKER_00: The ability to edit code,
48:01
we built the kind of models to take a high-level plan and make
48:05
an edit to a piece
48:06
SPEAKER_00: of code very fast.
48:07
So all of these together made it so that this was a workflow
48:10
that our developers wanted to use.
48:12
SPEAKER_00: We had the speed covered.
48:14
SPEAKER_00: We had the fact that it understood the code base well.
48:17
And then we also had massive model improvements to actually be
48:19
able to call these
48:20
SPEAKER_00: tools and make these iterative changes.
48:22
SPEAKER_00: I don't want to diminish that.
48:24
SPEAKER_00: You have all of these and suddenly now you have a real product.
48:28
I've been meaning to ask you this,
48:29
but how is the team using Windsurf to develop Windsurf?
48:35
SPEAKER_01: Because you're doing it, right?
48:36
SPEAKER_01: You just told me how you're doing it.
48:40
SPEAKER_01: From two perspectives.
48:41
One,
48:41
from the technical feasibility,
48:43
I'm assuming you're not going to work on the exact same...
48:48
code base or you have a fork or something like that,
48:49
or a build or something like that.
48:51
And then the other hand on like,
48:53
you know,
48:54
do you kind of force people to dog food?
48:55
SPEAKER_01: Do people just do it?
48:56
SPEAKER_01: Do people get stuck on certain versions?
48:58
SPEAKER_01: Do they turn on features for themselves, et cetera?
49:00
SPEAKER_01: Yeah.
49:01
SPEAKER_00: So the way we do it is we do have like an insider's developer mode.
49:05
SPEAKER_00: So this enables us to test new features.
49:07
I guess anyone at the company should be able to create a
49:09
feature and then deploy it
49:10
SPEAKER_00: to everyone internally.
49:11
SPEAKER_00: And now we have a large number of developers who will get feedback.
49:14
SPEAKER_00: We have an ability for our own developers to dog food new releases.
49:17
SPEAKER_00: We can have our own developers say, I hate this thing.
49:20
SPEAKER_00: Please don't ever do this.
49:21
And it's nice because then we don't need to give it to our own developers,
49:24
but other developers.
49:26
SPEAKER_00: So
49:26
SPEAKER_00: I think we have this tiered system at the company.
49:29
SPEAKER_00: We have our own sort of release.
49:30
SPEAKER_00: We have Next,
49:31
which is future looking products that we are releasing that are
49:35
a little bit more raw.
49:37
SPEAKER_00: And then we have like the actual release that we give to developers,
49:40
SPEAKER_00: which we're willing to A-B test things,
49:41
but we're not willing to A-B test things in such a way where we
49:43
give people a
49:44
SPEAKER_00: comically bad experience just to A-B test them.
49:47
SPEAKER_00: It's bad because people are using this for their real work.
49:49
So if you're using it for your real work,
49:51
we don't want to be hurting you.
49:52
SPEAKER_00: Right.
49:53
So I think one of the things that's quite valuable to us is
49:56
probably this is a this
49:57
SPEAKER_00: is you would think this is a failure mode for our company,
50:00
SPEAKER_00: which is that we use Windsurf,
50:01
SPEAKER_00: largely speaking,
50:02
SPEAKER_00: to modify large code bases.
50:04
SPEAKER_00: Right.
50:04
SPEAKER_00: For obvious reasons,
50:05
because I think our developers aren't building these toy apps
50:07
over and over again.
50:08
SPEAKER_00: But crazily enough,
50:10
one of our biggest power users inside our company is actually a non-developer.
50:13
SPEAKER_00: He leads partnerships.
50:15
SPEAKER_00: He's never written software before.
50:16
SPEAKER_00: And he routinely builds apps with Windsurf.
50:20
SPEAKER_00: Right.
50:20
SPEAKER_00: And he's one of our biggest users inside the company.
50:23
SPEAKER_00: And we've used this to actually replace buying other SaaS tools.
50:26
SPEAKER_00: And he's actually even deployed some of these tools inside the company.
50:28
SPEAKER_01: What function is this person in?
50:31
SPEAKER_00: It's partnership.
50:32
SPEAKER_00: So I'll give you an example of some of the tools.
50:35
SPEAKER_00: These are not complex pieces of software,
50:36
SPEAKER_00: but you would be surprised at how much they actually cost.
50:38
SPEAKER_00: They're six figures in cost because it's bespoke software, right?
50:42
SPEAKER_00: I'll give you an example.
50:43
SPEAKER_00: You have a quoting tool.
50:44
SPEAKER_00: So the idea of a quoting tool is you have a customer.
50:46
SPEAKER_00: The customer has this size.
50:47
SPEAKER_00: They're in this vertical.
50:49
SPEAKER_00: They want this kind of deal.
50:51
SPEAKER_00: Here's the way it would look.
50:52
SPEAKER_00: Here's the amount of discount we're willing to give them as a customer.
50:56
And usually these systems are really systems that you would
50:59
need to pay a lot of money for.
51:01
SPEAKER_00: And the reason is because,
51:02
SPEAKER_00: I don't know,
51:04
SPEAKER_00: there's no reason for us,
51:05
SPEAKER_00: for our developers to go out and build this internally,
51:07
SPEAKER_00: right?
51:07
SPEAKER_00: It's a big distraction from going out and building our product.
51:10
SPEAKER_00: But now,
51:11
SPEAKER_00: on the other hand,
51:12
SPEAKER_00: you have a domain expert in the person that actually runs partnerships.
51:16
SPEAKER_00: He doesn't know software, but he knows this really well, right?
51:19
SPEAKER_00: And because of that, he's able to create these apps really quickly.
51:22
SPEAKER_00: And granted,
51:22
SPEAKER_00: we do have a person inside the company that looks at the app,
51:25
SPEAKER_00: makes sure that it logistically makes sense.
51:28
SPEAKER_00: It's secure, can be deployed inside the company.
51:30
SPEAKER_00: But these are more ephemeral apps, right?
51:32
SPEAKER_00: They're quite stateless.
51:33
SPEAKER_00: If you were to look at the input output of this app,
51:36
SPEAKER_00: it is not as complex as like,
51:38
SPEAKER_00: let's say,
51:38
SPEAKER_00: the Windsor project,
51:39
SPEAKER_00: right?
51:40
SPEAKER_00: Right.
51:41
SPEAKER_00: Yeah.
51:42
We now have this like growing set of people inside the company
51:44
that are not
51:45
developers that are getting value from which we found a little
51:47
surprising too.
51:49
SPEAKER_01: Yeah,
51:49
and can you also give maybe just some other examples of what
51:53
you think it might be
51:54
SPEAKER_01: your place?
51:54
SPEAKER_01: The reason being this,
51:55
I'm actually really interested in this because I do hear a lot
51:58
of people either on
51:59
SPEAKER_01: social media or CEOs saying that SaaS apps could be the end of it.
52:02
And I've always been skeptical for the reason that there's two
52:05
types of SaaS apps.
52:05
SPEAKER_01: And most of the SaaS apps I see,
52:07
SPEAKER_01: for example,
52:08
SPEAKER_01: Workday,
52:09
SPEAKER_01: which is an HR platform,
52:10
SPEAKER_01: and they will have hosting,
52:11
SPEAKER_01: they will have business rules,
52:13
SPEAKER_01: they will update to some extent with regulations and all that stuff.
52:15
SPEAKER_01: So they do a lot of stuff that is...
52:17
SPEAKER_01: The UI, I know we can trivialize, but it's a lot more than that.
52:21
SPEAKER_01: And then there are a few of these simpler ones.
52:24
SPEAKER_01: I don't want to put names,
52:25
but there's a polling app where internally inside the company
52:28
you can poll.
52:29
SPEAKER_01: It has state, but it's relatively simple.
52:31
SPEAKER_01: You can see behind it.
52:33
SPEAKER_01: I could build it,
52:34
but I just don't want to deal with authentication to host it
52:38
inside the company.
52:39
SPEAKER_01: Yeah.
52:40
SPEAKER_01: It's already there.
52:41
SPEAKER_01: And then there's the ones you mentioned that are stateless.
52:43
SPEAKER_01: So like what kinds of SaaS tools do you see that you're replacing?
52:47
SPEAKER_01: And you might see other companies potentially using,
52:50
SPEAKER_01: you know,
52:50
like tools like this actually have with one dedicated helper
52:55
developer build it internally,
52:58
SPEAKER_00: bring it in house.
53:00
SPEAKER_00: Yeah.
53:00
SPEAKER_00: You know,
53:01
I think it's hubris to believe that products like Workday and
53:04
Salesforce are going
53:05
SPEAKER_00: to get replaced by this.
53:05
SPEAKER_00: I think you're totally right.
53:07
SPEAKER_00: These products have a lot of state.
53:09
SPEAKER_00: They encapsulate business workflows.
53:11
SPEAKER_00: There's actually for a product like Workday,
53:12
probably compliance that you need to do because of how business
53:15
critical the system is.
53:16
SPEAKER_00: So this isn't the kind of system that this would replace.
53:19
It probably falls in the latter two categories and probably
53:22
even just the last one,
53:23
SPEAKER_00: which is to say kind of these stateless systems...
53:25
that don't do rights to the most business critical parts of
53:29
your databases.
53:31
It's probably actually those kinds of systems that very quickly
53:34
can get replaced.
53:35
And I would say there's a new category where think about the
53:37
amount of software
53:38
that would benefit a business that just isn't getting created
53:41
that now could get created.
53:42
SPEAKER_02: right?
53:42
SPEAKER_00: Because,
53:43
and the reason why that software couldn't get created is a
53:45
company couldn't be
53:46
SPEAKER_00: created that would be able to sustain itself,
53:48
SPEAKER_00: that would have an economic,
53:49
SPEAKER_00: a business model that would justify it existing.
53:51
SPEAKER_00: But now,
53:52
SPEAKER_00: since the software is very easy to create,
53:54
SPEAKER_00: these pieces of software are going to proliferate,
53:56
SPEAKER_00: right?
53:56
SPEAKER_00: And,
53:57
One of the things that I'd like to talk about for software is
54:00
there's a little bit
54:01
SPEAKER_00: of a,
54:01
SPEAKER_00: we've been,
54:02
SPEAKER_00: you know,
54:02
SPEAKER_00: because the cost of building software was a lot higher,
54:05
SPEAKER_00: right?
54:05
SPEAKER_00: Of simple software was a lot higher, right?
54:08
SPEAKER_00: Right now for a front end,
54:09
we have to admit it's gotten a lot cheaper to build a basic
54:12
front end system,
54:12
SPEAKER_00: right?
54:13
SPEAKER_02: Radically.
54:14
SPEAKER_00: Radically cheaper.
54:15
So I think the way I would sort of look at it is for these kind
54:20
of systems,
54:21
right?
54:22
SPEAKER_00: What are you really paying for when you pay a SaaS vendor?
54:24
You're not only paying for your product,
54:26
you're paying for the maintenance.
54:28
SPEAKER_00: You're paying for the fact that actually,
54:30
SPEAKER_00: you know,
54:30
this company actually is building a bunch of other features
54:33
that you don't need.
54:34
SPEAKER_00: And the reason why is because they need to support a bunch of customers,
54:37
SPEAKER_00: but you're still paying for that R&D,
54:39
SPEAKER_00: right?
54:40
You're paying for their sales and marketing,
54:41
a bunch of other stuff there.
54:43
SPEAKER_00: So my viewpoint is,
54:44
SPEAKER_00: if you can build custom software for yourself,
54:47
SPEAKER_00: that is not very complex,
54:49
SPEAKER_00: but helps you in your own business processes,
54:50
SPEAKER_00: I think that might proliferate.
54:52
SPEAKER_00: inside companies.
54:53
And that might actually cause a whole host of kind of companies
54:57
that fall into that category.
54:58
It is simple business software that feels largely stateless to
55:02
kind of have trouble
55:03
SPEAKER_00: unless they like kind of reinvent themselves.
55:06
SPEAKER_01: Yeah,
55:06
SPEAKER_01: and I guess,
55:07
SPEAKER_01: you know,
55:07
SPEAKER_01: one obvious reinventing that could happen later is once this happens,
55:10
SPEAKER_01: let's just continue this thought of,
55:11
SPEAKER_01: like,
55:12
SPEAKER_01: companies are building a lot of internal software.
55:15
SPEAKER_01: They might start to have some similar problems of,
55:18
SPEAKER_01: let's take,
55:19
SPEAKER_01: you know,
55:19
SPEAKER_01: three,
55:19
SPEAKER_01: five years of that under-road maintenance,
55:22
SPEAKER_01: storage,
55:23
SPEAKER_01: maintenance.
55:25
SPEAKER_01: compliance,
55:25
SPEAKER_01: just going through it,
55:27
SPEAKER_01: if they're working,
55:28
SPEAKER_01: reevaluating if it makes sense to actually bring it into something.
55:31
So like this could create a lot of new opportunities for other
55:34
software businesses
55:35
SPEAKER_01: or software developers or,
55:37
SPEAKER_01: you know,
55:37
SPEAKER_01: maybe these companies or maybe a new job role in software engineering,
55:40
SPEAKER_01: which is,
55:40
SPEAKER_01: you know,
55:41
SPEAKER_01: I'm now specialized in,
55:42
SPEAKER_01: I've built so many of these apps and I can help you with them.
55:45
SPEAKER_01: Who knows?
55:46
SPEAKER_00: No,
55:46
I think a lot of people talk about how we're going to have way
55:51
fewer software
55:51
SPEAKER_00: engineers in the near future.
55:53
I think it feels like it's people that hate software engineers,
55:58
largely speaking,
55:58
that say this.
56:00
SPEAKER_00: It feels pessimistic not only towards these people,
56:03
SPEAKER_00: but I would say just in terms of what the ambitions for companies are.
56:06
I think the ambitions for a lot of companies is to build a lot
56:09
better product.
56:10
And if you now give the ability for companies to now have a
56:13
better return on
56:14
SPEAKER_00: investment for building technology,
56:16
SPEAKER_00: right?
56:16
SPEAKER_00: Because the cost of building software has gone down.
56:19
SPEAKER_00: What should you be doing?
56:20
You should be building more because now the ROI for software
56:23
and developers is even
56:24
SPEAKER_00: higher because a singular developer can do more for your business,
56:27
SPEAKER_00: right?
56:28
SPEAKER_00: So technology actually increases the ceiling of your company much faster.
56:32
SPEAKER_00: Yeah.
56:32
SPEAKER_01: And I'm going to just double click on that because like,
56:34
SPEAKER_01: you know,
56:35
SPEAKER_01: you're,
56:35
SPEAKER_01: you have been building Windsurf and you've been building these tools,
56:38
SPEAKER_01: but you've also worked with the team.
56:40
SPEAKER_01: In fact,
56:40
SPEAKER_01: with the same team,
56:41
SPEAKER_01: even before these tools today,
56:44
SPEAKER_01: one of your,
56:44
SPEAKER_01: you know,
56:45
SPEAKER_01: solid engineers who was a solid engineer four years ago,
56:48
SPEAKER_01: how has their work changed now that they have access to Windsurf,
56:52
SPEAKER_01: Agentic,
56:53
SPEAKER_01: you know,
56:54
SPEAKER_01: Cascade,
56:54
SPEAKER_01: all these other tools,
56:57
SPEAKER_01: including,
56:57
SPEAKER_01: you know,
56:57
SPEAKER_01: like ChatGPZ,
56:58
SPEAKER_01: et cetera.
56:59
SPEAKER_01: What's changed?
57:00
SPEAKER_01: And then not just your engineering,
57:01
SPEAKER_01: but also the team that you had four years ago,
57:04
SPEAKER_01: you know,
57:04
SPEAKER_01: that was doing work.
57:05
SPEAKER_01: How has their work changed in terms of,
57:09
SPEAKER_01: I don't want to point you in any direction,
57:10
SPEAKER_01: but I'm just interested in like what you would say,
57:12
SPEAKER_01: how is that team different in what they do or how they do,
57:15
SPEAKER_01: how much they do?
57:16
SPEAKER_01: Yeah.
57:16
SPEAKER_01: Yeah.
57:17
SPEAKER_00: I think there's maybe a couple of things.
57:19
So first of all,
57:20
the amount of code that we have in the company is quite high.
57:22
SPEAKER_00: It now dominates what a single person knows at the moment.
57:26
SPEAKER_00: So in the beginning of the company, that's not the case.
57:28
SPEAKER_00: So actually,
57:28
this is something that I can't point to because the company is
57:31
necessarily quite small.
57:33
SPEAKER_00: Right now,
57:33
I would say there's more fearlessness to jump into a new part
57:39
of the code base and
57:39
SPEAKER_00: start making changes.
57:41
SPEAKER_00: I would say in the past,
57:43
SPEAKER_00: you would more say,
57:44
SPEAKER_00: hey,
57:45
SPEAKER_00: this person has way more familiarity with this part of the code.
57:48
SPEAKER_00: That is still the case.
57:49
SPEAKER_00: Right.
57:49
SPEAKER_00: When you say familiarity,
57:51
SPEAKER_00: now it is,
57:52
SPEAKER_00: it's like understanding the code,
57:53
SPEAKER_00: but this person also knows where the dead bodies are,
57:55
SPEAKER_00: which is to say,
57:56
SPEAKER_00: Hey,
57:57
SPEAKER_00: when we're all,
57:58
SPEAKER_00: you know,
57:58
SPEAKER_00: you did X and you got Y that happened.
58:01
SPEAKER_00: And that means you always should do Z. Right.
58:03
SPEAKER_00: Yeah.
58:04
SPEAKER_00: And there are still people like that at the company.
58:06
SPEAKER_00: And I'm not saying that that is not valuable,
58:08
but I think now engineers feel more empowered to go out and
58:12
make changes throughout
58:13
SPEAKER_00: the code base,
58:14
SPEAKER_00: which is actually awesome.
58:15
And the second key piece is our developers now go to the AI
58:18
first to see what value
58:20
SPEAKER_00: it would generate for them before making a change.
58:22
SPEAKER_00: which is something which I would say in the autocomplete days,
58:25
you would go out and type it and you would get a lot of
58:27
advantage from autocomplete
58:28
SPEAKER_00: and the passive AI.
58:29
But now the active AI is something that developers more and
58:32
more reach towards to
58:33
SPEAKER_00: actually go out and make changes at the very beginning.
58:36
SPEAKER_01: I'm interested in how this will change software engineering.
58:42
SPEAKER_01: I noticed both things on myself.
58:43
SPEAKER_01: I still code and I do my side projects,
58:47
but I always drag my feet of getting back into the context of
58:50
the code that I wrote,
58:51
SPEAKER_01: which was,
58:51
SPEAKER_01: you know,
58:52
SPEAKER_01: I kind of forgot part of it,
58:53
getting back into the language because I use multiple languages
58:56
because they're
58:56
SPEAKER_01: side projects.
58:57
SPEAKER_01: And AI, it does help me just jump into it.
59:00
SPEAKER_01: I no longer have the thing.
59:02
SPEAKER_01: And sometimes, yeah, I just prompt the AI saying, what would you do?
59:05
SPEAKER_01: I just want to know.
59:06
SPEAKER_01: And then if it looks good, I do it.
59:08
SPEAKER_01: If it not, I just scrap it.
59:10
SPEAKER_01: Maybe I prompt it or sometimes I just like,
59:12
SPEAKER_01: nah,
59:12
I'm just going to do it because either I didn't give it right instructions.
59:15
SPEAKER_01: Like,
59:15
SPEAKER_01: you know,
59:17
SPEAKER_01: there is this thing,
59:17
SPEAKER_01: especially when you're working on stuff,
59:19
SPEAKER_01: you know,
59:20
SPEAKER_01: the code base,
59:21
SPEAKER_01: you've onboarded,
59:21
SPEAKER_01: you know what you want to do.
59:23
SPEAKER_01: But I think it really helps.
59:25
SPEAKER_01: It helps me at least with the effort.
59:27
SPEAKER_01: Sorry, with the...
59:29
SPEAKER_01: With the thing that wouldn't take much creativity,
59:33
SPEAKER_01: but it would just be time,
59:34
SPEAKER_01: a drag,
59:35
SPEAKER_01: figuring out the right things,
59:38
SPEAKER_01: finding the right dependency,
59:40
SPEAKER_01: changing those things,
59:40
SPEAKER_01: that kind of stuff.
59:42
SPEAKER_00: I think you're exactly right.
59:43
SPEAKER_00: I think this reducing friction piece is something that is,
59:48
it's kind of hard to quantify the value because it makes you
59:51
more excited to do more.
59:53
I think software development is a very weird profession,
59:57
and I'll give you an example of why.
1:00:00
SPEAKER_00: It's weird,
1:00:00
SPEAKER_00: and a lot of people would think,
1:00:02
SPEAKER_00: oh,
1:00:02
SPEAKER_00: this is a very easy job,
1:00:03
SPEAKER_00: and I actually think it's quite hard on you mentally,
1:00:07
SPEAKER_00: and I'll walk you through what I mean by that.
1:00:09
SPEAKER_00: You're doing a hard project.
1:00:10
SPEAKER_00: You sometimes go home with an incomplete idea.
1:00:14
SPEAKER_00: The code didn't pass a bunch of tests,
1:00:17
SPEAKER_00: and it just bothers you when you sleep,
1:00:19
SPEAKER_00: and you need to go back and kind of fix it,
1:00:22
SPEAKER_00: and this could be for days.
1:00:24
SPEAKER_00: And for other jobs, I don't think you kind of feel that, right?
1:00:28
SPEAKER_00: It's a lot more procedural potentially for other types of jobs.
1:00:32
SPEAKER_00: I'm not saying for every job,
1:00:33
there are obviously jobs where there's a massive like problem
1:00:35
solving component.
1:00:36
SPEAKER_00: But that just means that this kind of,
1:00:39
SPEAKER_00: you do get a fatigue if you,
1:00:42
SPEAKER_00: you know,
1:00:42
SPEAKER_00: at some point,
1:00:43
SPEAKER_00: even the easy things,
1:00:44
SPEAKER_00: just forcing you to do new easy things,
1:00:46
SPEAKER_00: it adds some amount of mental fatigue.
1:00:48
SPEAKER_00: And I think you now have a very powerful system that you now trust.
1:00:53
that should ideally reduce this fatigue and be able to do a lot
1:00:57
of the things that
1:00:58
SPEAKER_00: are in the past high activation energy and do it really fast for you.
1:01:02
SPEAKER_01: Yeah,
1:01:02
this is really interesting because I was just talking with a
1:01:05
former colleague of
1:01:06
SPEAKER_01: mine who had a few months where he just wasn't producing much code.
1:01:10
SPEAKER_01: Really good engineer, really solid.
1:01:12
SPEAKER_01: And at the time, I didn't know why.
1:01:15
SPEAKER_01: And he didn't tell me.
1:01:17
SPEAKER_01: And then he kind of snapped out of it.
1:01:19
SPEAKER_01: But we're just talking.
1:01:20
SPEAKER_01: He said that actually he was at a really bad time in his life.
1:01:24
lots of stress in a relationship and at home with family,
1:01:28
all these things.
1:01:29
And he said that he just realizes how mental of the game
1:01:32
software engineering is.
1:01:34
SPEAKER_01: He at work, he just couldn't get himself to, you know, get into the zone.
1:01:38
SPEAKER_01: We know how it is, especially before AI tools.
1:01:40
SPEAKER_01: And, uh,
1:01:41
SPEAKER_01: What you said, I'm starting to get a bit of an appreciation.
1:01:44
SPEAKER_01: And the fact that I remember,
1:01:45
SPEAKER_01: you know,
1:01:46
SPEAKER_01: stressful,
1:01:46
SPEAKER_01: but I couldn't turn off like,
1:01:48
SPEAKER_01: you go home,
1:01:48
SPEAKER_01: you're having dinner,
1:01:49
you're still thinking about how you would change that or why
1:01:53
it's not working.
1:01:54
SPEAKER_01: It's
1:01:55
SPEAKER_01: I don't think we'll be able to go onwards,
1:01:59
SPEAKER_01: but I think for listeners,
1:02:00
SPEAKER_01: it's worth thinking about how weird it is.
1:02:03
SPEAKER_01: I think it's good to reflect on it because it is unique.
1:02:07
SPEAKER_01: For so many jobs,
1:02:08
you can actually just put down your work and leave the office
1:02:11
and you cannot continue.
1:02:12
SPEAKER_01: And that's it.
1:02:13
SPEAKER_01: I cannot even think about it because all your work is there.
1:02:16
And also how these tools might just change it for the better in
1:02:20
many ways and maybe
1:02:21
SPEAKER_01: just in weird ways that we don't expect in others.
1:02:24
SPEAKER_00: No,
1:02:24
SPEAKER_00: I think you're totally...
1:02:25
SPEAKER_00: This idea of...
1:02:26
SPEAKER_00: I think this is why finding amazing software engineers is very...
1:02:30
SPEAKER_00: It's rare.
1:02:31
SPEAKER_00: It's rare.
1:02:31
Because these people are people that I guess have gone through
1:02:35
this and are willing
1:02:35
SPEAKER_00: to put themselves through the idea of like,
1:02:37
SPEAKER_00: hey,
1:02:38
all of the learnings that I had from the lowest level to the
1:02:40
highest level,
1:02:41
SPEAKER_00: and then willing to go down to the weeds...
1:02:44
SPEAKER_00: to kind of make sure you solve the problem.
1:02:47
SPEAKER_00: It's a rare skill.
1:02:48
SPEAKER_00: It's that,
1:02:49
SPEAKER_00: you know,
1:02:49
SPEAKER_00: you would imagine,
1:02:50
SPEAKER_00: hey,
1:02:50
SPEAKER_00: this is something that everyone would be able to do,
1:02:52
SPEAKER_00: but it like takes a lot of dedication.
1:02:54
SPEAKER_00: And as you pointed out,
1:02:55
SPEAKER_00: it's like this,
1:02:56
SPEAKER_00: you know,
1:02:57
SPEAKER_00: for an activity that is not a very normal activity.
1:03:00
SPEAKER_01: Yeah.
1:03:01
SPEAKER_01: Well,
1:03:01
SPEAKER_01: going back to engineering challenges and decisions,
1:03:04
SPEAKER_01: one super interesting thing that I've been dying to ask you is,
1:03:09
SPEAKER_01: you did mention in the beginning that,
1:03:10
SPEAKER_01: you know,
1:03:10
SPEAKER_01: like it's...
1:03:12
When you started Windsurf,
1:03:14
you realized Visual Studio Code is not there where it should be.
1:03:18
SPEAKER_01: However, you started by forking Visual Studio Code, right?
1:03:22
SPEAKER_01: Do I know that right?
1:03:23
SPEAKER_01: That's exactly right.
1:03:24
Can you tell me the pros and cons of doing this as opposed to
1:03:28
building your own editor?
1:03:29
SPEAKER_01: And I'm aware that there are some downsides of doing this.
1:03:32
SPEAKER_01: There's some licensing things.
1:03:33
SPEAKER_01: So that's one part of the question.
1:03:34
SPEAKER_01: The second part of the question, why did you think that forking...
1:03:38
SPEAKER_01: is the right move to build a much better,
1:03:41
SPEAKER_01: much more capable thing of whatever VS Code was back in the day.
1:03:46
SPEAKER_00: Yeah.
1:03:46
SPEAKER_00: So just maybe some clarifications just on terminology.
1:03:50
SPEAKER_00: VS Code is a product that is built on top of Code OSS,
1:03:56
SPEAKER_00: which is basically the open source project.
1:04:01
SPEAKER_00: I did not know that.
1:04:02
SPEAKER_00: Yeah, because VS Code has proprietary pieces on top of the open source.
1:04:08
SPEAKER_01: I do know that, and a lot of people don't know that, actually.
1:04:10
SPEAKER_00: Yeah, exactly.
1:04:11
So I guess one of the things that we actually did was we wanted
1:04:15
to make sure we did this right.
1:04:16
SPEAKER_00: And what I mean by that is when we actually built our product,
1:04:20
SPEAKER_00: we did fork code OSS,
1:04:22
SPEAKER_00: but we did not support any of the proprietary pieces that Microsoft had.
1:04:26
And we never actually provided support for those,
1:04:30
not through a marketplace or anything.
1:04:31
SPEAKER_00: We actually use an open marketplace that is completely fine.
1:04:34
SPEAKER_00: And this,
1:04:35
SPEAKER_00: by the way,
1:04:35
this forced us to actually have to build out a lot of
1:04:38
extensions that people needed
1:04:39
SPEAKER_00: and bake it into the product.
1:04:40
SPEAKER_00: I'll give you an example.
1:04:41
SPEAKER_00: For Python language servers, we actually now, we have our own version.
1:04:45
right for remote ssh we have our own version for dev containers
1:04:48
we have our own
1:04:48
version so this actually forced us to get a lot tighter on what
1:04:52
we need to do and
1:04:53
we never took i guess a shortcut of hey let's go out and do
1:04:57
something that we
1:04:57
shouldn't be doing um because hey we work with real companies
1:05:01
we work with real
1:05:01
developers and why should we be putting them in that position
1:05:05
right i guess we we
1:05:06
SPEAKER_00: kind of took that position and um
1:05:10
SPEAKER_00: So that was the positioning we had.
1:05:13
SPEAKER_00: Obviously,
1:05:13
SPEAKER_00: there were some complexities,
1:05:14
but this just caused us more engineering effort before we
1:05:17
launched the product.
1:05:18
We did launch the product with an ability to connect it to
1:05:21
remote SSH and do all
1:05:22
SPEAKER_00: this other stuff.
1:05:22
And we did have internal engineering effort to actually go out
1:05:26
and do that.
1:05:27
SPEAKER_00: Now, the question might be, why even fork VS Code?
1:05:32
I think it's because it's a very well-known IDE where people
1:05:38
have workflows.
1:05:40
There are also many extensions there that people rely on that
1:05:45
are extremely popular.
1:05:46
SPEAKER_00: An IDE is not just the place where you write software.
1:05:51
It's also the place where you attach a debugger and do all
1:05:53
these other operations.
1:05:55
SPEAKER_00: And we didn't want to reinvent the wheel on that.
1:05:57
We didn't think we were better than,
1:05:59
I guess,
1:05:59
the entire open source community on that,
1:06:02
right?
1:06:02
SPEAKER_00: In terms of all the ways you could use the product.
1:06:04
And I'll give you an example of maybe how we're trying to be
1:06:08
pragmatic here.
1:06:09
SPEAKER_00: We didn't go out and try to replace JetBrains with this product.
1:06:12
SPEAKER_00: We actually put all the capabilities of Windsurf into JetBrains,
1:06:15
SPEAKER_00: into what's called a Windsurf plugin.
1:06:17
SPEAKER_00: And this is where our goal is to meet developers where they are.
1:06:21
And meeting VS Code developers where they are means we should
1:06:24
give them a familiar experience.
1:06:25
Meeting JetBrains developers means we should give them a
1:06:28
familiar experience,
1:06:28
SPEAKER_00: which is actually use JetBrains.
1:06:30
SPEAKER_00: Now, a question might be, why didn't we fork JetBrains?
1:06:32
SPEAKER_00: And the answer is two reasons.
1:06:33
SPEAKER_00: First of all, we can't.
1:06:35
SPEAKER_00: It's closed doors.
1:06:36
SPEAKER_00: Second of all,
1:06:37
the answer is actually because Jeopardens is actually a
1:06:39
fantastic IDE for Java
1:06:41
SPEAKER_00: developers and in a lot of cases,
1:06:44
SPEAKER_00: C++ and Python developers.
1:06:46
SPEAKER_01: And so far as PHP as well, PHP Storm, if you ever meet them.
1:06:49
SPEAKER_01: That's exactly right.
1:06:51
SPEAKER_00: They have one for almost every single language.
1:06:54
SPEAKER_00: For every single language.
1:06:55
SPEAKER_00: And the reason is because they have great debuggers,
1:06:57
great language servers that actually think are not even present
1:07:00
on VS Code right now.
1:07:01
SPEAKER_00: Like if you were a great Java developer,
1:07:03
SPEAKER_00: most of them and probably 80 plus percent right now use Intelligent.
1:07:07
SPEAKER_00: on the market.
1:07:08
So the question there is like,
1:07:09
I think as a company,
1:07:10
our goal is not to be dogmatic.
1:07:12
Our goal is to build the best technology and provide it and
1:07:15
democratize it and
1:07:16
SPEAKER_00: provide it to as many developers as possible.
1:07:18
SPEAKER_01: No, I love it.
1:07:18
SPEAKER_01: And this is actually,
1:07:20
I was talking with one of your software engineers who did
1:07:23
mention an interesting
1:07:24
SPEAKER_01: challenge because of just this,
1:07:25
the fact that you do have a JetBrains plugin and then you have
1:07:28
the ID and now
1:07:30
SPEAKER_01: apparently you're sharing some binaries between the two.
1:07:32
SPEAKER_01: Can you talk a little bit about that engineering?
1:07:35
SPEAKER_00: Yeah.
1:07:35
So this was actually an engineering decision we needed to make
1:07:38
a couple months into
1:07:38
SPEAKER_00: starting working on Codium,
1:07:40
SPEAKER_00: which was that,
1:07:41
SPEAKER_00: hey,
1:07:41
SPEAKER_00: we're going to go out and build a VS Code extension.
1:07:43
SPEAKER_00: That's what we started out with.
1:07:44
SPEAKER_00: But very quickly, the next step is let's go implement it in Jeopardy.
1:07:48
SPEAKER_00: The problem is if we need to duplicate all the code,
1:07:51
SPEAKER_00: it's going to be really,
1:07:52
SPEAKER_00: really annoying for us to support all this.
1:07:54
So what we decided to do is actually go out and build almost a
1:07:57
shared binary
1:07:58
between both that we call the language server that actually
1:08:00
does the heavy lifting.
1:08:01
So the goal there is hopefully we're not just duplicating the
1:08:05
work in a bunch of places.
1:08:06
And this enables us to support many,
1:08:08
many IDs from an architecture standpoint.
1:08:11
SPEAKER_00: And that's why we were able to support not just JetBrains,
1:08:13
SPEAKER_00: Eclipse,
1:08:14
SPEAKER_00: Vim,
1:08:15
SPEAKER_00: all of these other IDs that are popular without much lift.
1:08:22
SPEAKER_01: Okay.
1:08:23
SPEAKER_01: I need to ask you about MCP.
1:08:26
SPEAKER_01: You have started to support it, which is really cool.
1:08:28
SPEAKER_01: I play around with it and I think it's a good first step.
1:08:31
SPEAKER_01: What is your take on MCP, especially with the security worries?
1:08:36
SPEAKER_01: And also, where do you see MCP going right now?
1:08:38
SPEAKER_01: I think it's a bit of an open book,
1:08:39
but you are probably a bit more exposed to this than most
1:08:42
listeners will be.
1:08:44
SPEAKER_00: You know, I think it's very exciting.
1:08:46
SPEAKER_00: I have some, maybe one concern, but let me start with the exciting part.
1:08:51
And the exciting part is now it's democratizing access to
1:08:54
everything inside a
1:08:55
company or everything a user would want within their coding
1:08:59
environment for our
1:09:00
SPEAKER_00: product in particular.
1:09:01
SPEAKER_00: Obviously, there are other products.
1:09:02
SPEAKER_00: Maybe it can help you buy goods and grocery and stuff like that.
1:09:06
Obviously,
1:09:06
we're not that interested in that case,
1:09:08
but that is a nice thing.
1:09:11
One of the other things that it lets companies do is they can
1:09:13
implement their own
1:09:14
SPEAKER_00: MCP servers with security guarantees,
1:09:17
which is to say they can implement a battle-tested MCP server
1:09:21
that talks to an
1:09:22
internal service that actually does auth and all these other
1:09:25
things for the end user,
1:09:27
SPEAKER_00: and they can own the implementation of that.
1:09:29
SPEAKER_00: So there's a way for companies now to enable us to use,
1:09:33
SPEAKER_00: to interact with their internal services in a secure way.
1:09:37
SPEAKER_00: But you're totally right.
1:09:38
There could be a slippery slope where this causes everyone to
1:09:41
have immediate access
1:09:42
to everything in a right-based fashion that could have negative consequences.
1:09:46
SPEAKER_00: But the thing I'm particularly maybe a little bit worried about,
1:09:51
SPEAKER_00: and it's not worry,
1:09:52
SPEAKER_00: it's more so the paradigm itself.
1:09:54
SPEAKER_00: Is MCP the right way of encapsulating talking to other systems?
1:10:00
Or is it like actual workflows of developers going and
1:10:04
interacting with these systems?
1:10:05
SPEAKER_00: And I'll give you an example of that.
1:10:06
SPEAKER_00: One of the problems with MCP is it forces you to hit a particular spec.
1:10:11
SPEAKER_00: And you know this, actually the best spec is flexibility, right?
1:10:15
SPEAKER_00: It's flexibility.
1:10:16
SPEAKER_00: And,
1:10:17
SPEAKER_00: you know,
1:10:17
SPEAKER_00: if you ask these systems now to integrate with another,
1:10:20
SPEAKER_00: like you ask an LOM,
1:10:21
SPEAKER_00: like a GPT-4-1 or a Sony,
1:10:23
SPEAKER_00: hey,
1:10:23
SPEAKER_00: you know,
1:10:24
SPEAKER_00: build an integration to this system,
1:10:26
SPEAKER_00: to a Notion.
1:10:27
SPEAKER_00: It will do it zero shot now.
1:10:29
SPEAKER_02: Yep.
1:10:29
SPEAKER_00: So you could build an MCP server that is particular,
1:10:33
SPEAKER_00: that only lets you have access to two things in Notion,
1:10:35
SPEAKER_00: or the models themselves are capable of doing a lot.
1:10:38
SPEAKER_00: And it's like, how much do you want to constrain versus have freedom?
1:10:41
SPEAKER_00: And then also there's the corresponding security issue too.
1:10:43
SPEAKER_00: So like, look, it's awesome that we have access to it.
1:10:46
SPEAKER_00: Is this the final version?
1:10:48
SPEAKER_00: I don't know if this is the final version.
1:10:49
SPEAKER_01: Yeah, I'm going to rephrase it and let me know if you think I'm off.
1:10:56
But when you set up,
1:10:57
for example,
1:10:58
I'm building a web project and I'm using Node.
1:11:00
SPEAKER_01: And I have my packages JSON that specify what packages I'm going to use.
1:11:05
SPEAKER_01: Now, on my machine, I will have a lot of packages installed.
1:11:07
SPEAKER_01: But for each specific project,
1:11:09
SPEAKER_01: I'm going to be very clear of what I want to use,
1:11:12
SPEAKER_01: what package,
1:11:14
SPEAKER_01: maybe a subset of it.
1:11:15
SPEAKER_01: And, you know, like...
1:11:17
Right now,
1:11:17
it feels to me that the current version of MCP,
1:11:19
it just lets me connect everything.
1:11:21
SPEAKER_01: And I can't really say that,
1:11:24
SPEAKER_01: for example,
1:11:24
SPEAKER_01: on this project,
1:11:26
SPEAKER_01: I actually want you to only talk to this table in my database.
1:11:29
I don't want you to access all the other stuff because it's a
1:11:33
proud database and I
1:11:34
SPEAKER_01: have a test table there.
1:11:35
SPEAKER_01: That kind of stuff, right?
1:11:37
Are we talking about this granularity and figuring out what
1:11:40
would actually help me
1:11:42
SPEAKER_01: as an engineer be productive?
1:11:46
SPEAKER_00: No, it's an interesting point.
1:11:47
SPEAKER_00: So you're totally right.
1:11:49
You want these systems to have access to a lot of things so
1:11:51
that you can be productive.
1:11:53
SPEAKER_00: All the while,
1:11:53
you want to be imperative and very instructive on what systems
1:12:00
they should have
1:12:00
SPEAKER_00: access to internally.
1:12:02
SPEAKER_00: But the problem is people are very,
1:12:04
SPEAKER_00: I'm not going to say lazy,
1:12:05
SPEAKER_00: but it is annoying if you have 50 services and you need to tell it,
1:12:07
SPEAKER_00: you need to do this,
1:12:08
SPEAKER_00: you need to do that,
1:12:09
SPEAKER_00: you need to do this.
1:12:10
And what can very quickly happen is people don't and they get
1:12:13
mixed results or it
1:12:14
SPEAKER_00: has negative consequences.
1:12:16
SPEAKER_00: Look, I think we're figuring this out.
1:12:18
I think the whole industry is kind of figuring this out,
1:12:20
what the right model is.
1:12:22
And maybe it actually is a lot of engineering that needs to get
1:12:25
done post the MCP server,
1:12:27
SPEAKER_00: which is to say the MCP server provides a very free flowing interface,
1:12:31
but there's a lot of like kind of understanding of the server
1:12:34
to who the user is,
1:12:35
SPEAKER_00: what service they're trying to touch,
1:12:37
SPEAKER_00: what code base they're in.
1:12:38
And there's like proper access controls that are implemented
1:12:41
afterwards that helps
1:12:42
SPEAKER_00: you kind of like do that.
1:12:44
SPEAKER_01: I'm thinking these languages are not really popular,
1:12:47
SPEAKER_01: but when I started programming,
1:12:49
SPEAKER_01: I used C Sharp.
1:12:50
SPEAKER_01: And in C Sharp, for the classes, you had keywords.
1:12:52
SPEAKER_01: You know, you have classes, but you couldn't just access them.
1:12:55
SPEAKER_01: You had public classes, which everyone can access.
1:12:57
SPEAKER_01: You had protected classes.
1:12:58
SPEAKER_01: You actually had internal classes that were inside the module.
1:13:01
You had private classes,
1:13:02
which were not accessible unless you were a child class.
1:13:04
And these were just keywords of how,
1:13:07
what module can access what parts of your code.
1:13:10
SPEAKER_01: inside the code base.
1:13:10
SPEAKER_01: And we,
1:13:11
SPEAKER_01: back then,
1:13:12
SPEAKER_01: this was like the 2000s,
1:13:13
SPEAKER_01: we spent a lot of care deciding who can access what and how,
1:13:17
SPEAKER_01: even though technically you could have just,
1:13:19
SPEAKER_01: everyone could have chalked up with everyone,
1:13:20
SPEAKER_01: but we decided this was,
1:13:22
SPEAKER_01: you know,
1:13:22
SPEAKER_01: evolution of a few decades that it wasn't a good idea.
1:13:24
SPEAKER_01: So I'm wondering if we're going to get there,
1:13:27
SPEAKER_01: for example,
1:13:28
SPEAKER_01: with MCP,
1:13:28
we might reinvent some parts of it because that didn't come up
1:13:31
because like,
1:13:32
SPEAKER_01: you know,
1:13:32
SPEAKER_01: like someone thought it was...
1:13:35
SPEAKER_01: like just lick their finger,
1:13:37
it was because we needed it to organize large amounts of code
1:13:40
back then when we
1:13:41
SPEAKER_01: didn't have the tools that we have today.
1:13:43
SPEAKER_00: No, I think you're right.
1:13:44
SPEAKER_00: I think some primitives are missing right now, for sure.
1:13:47
SPEAKER_00: It's too freeform right now.
1:13:49
SPEAKER_01: It's going to be super exciting though,
1:13:50
SPEAKER_01: because we are seeing that it is going somewhere,
1:13:53
SPEAKER_01: maybe MCP,
1:13:53
SPEAKER_01: maybe not,
1:13:54
SPEAKER_01: and we're in the middle of it.
1:13:55
SPEAKER_01: You know, who knows?
1:13:56
Some people listening to it might actually influence the
1:13:59
direction of this new
1:14:00
SPEAKER_01: thing that we're going to use in like five years from now.
1:14:02
SPEAKER_01: That's awesome.
1:14:03
SPEAKER_01: Yeah.
1:14:04
SPEAKER_01: What is your take on this 70-30% of mental model for AI tools?
1:14:09
SPEAKER_01: This is something that comes up every now and then,
1:14:10
SPEAKER_01: especially with folks who are less technical,
1:14:13
that today they can prompt AI tools from Winsurf to Lovable and
1:14:18
others of like,
1:14:19
SPEAKER_01: hey,
1:14:19
SPEAKER_01: generate this idea that I have.
1:14:21
SPEAKER_01: And they do a good job of the one shot or the tweaking.
1:14:25
SPEAKER_01: And then the last 30%,
1:14:26
SPEAKER_01: especially when they're not experienced software engineers,
1:14:29
SPEAKER_01: they just get a little stuck or hopelessly stuck.
1:14:33
Do you observe this with Winsterfusers or this is not really a
1:14:38
thing when people
1:14:39
SPEAKER_01: are pretty technical and developers?
1:14:42
SPEAKER_00: Yeah, I think we do have non-developers that use the product.
1:14:45
SPEAKER_00: And I do think the level of frustration for them...
1:14:47
SPEAKER_00: And by the way,
1:14:47
SPEAKER_00: my viewpoint on this is not just let them be frustrated.
1:14:50
SPEAKER_00: I would love to help them.
1:14:52
SPEAKER_00: But the level of frustration when they have a problem is much higher.
1:14:55
SPEAKER_00: And the reason is because for you and I,
1:14:58
when we go and use this and it gets into this degenerate state
1:15:01
where it goes out
1:15:02
and it tries to make a change and it does a series of changes
1:15:04
that doesn't make sense.
1:15:05
SPEAKER_00: Our first instinct is don't just like...
1:15:08
SPEAKER_00: do it 10 more times when five times it didn't work.
1:15:10
SPEAKER_00: It's probably like,
1:15:11
look at the code and see what step didn't work and refer back
1:15:13
to the step that works,
1:15:14
SPEAKER_00: right?
1:15:14
SPEAKER_00: Like debugging principles.
1:15:16
Because,
1:15:16
but that's,
1:15:16
by the way,
1:15:17
the reason why we do that is we understand the code.
1:15:19
SPEAKER_00: Yeah.
1:15:19
SPEAKER_00: We can like go back into the code and kind of understand it.
1:15:22
SPEAKER_00: But you're right that for developers that can't,
1:15:25
SPEAKER_00: they're kind of in a state of helplessness.
1:15:28
SPEAKER_00: And I deeply empathize with that.
1:15:31
And it's like,
1:15:32
it's our job to figure out ways that we can make that a lot better.
1:15:35
SPEAKER_00: Now, granted, right?
1:15:37
SPEAKER_00: Does that mean we make our product completely catered to non-developers?
1:15:40
SPEAKER_00: No, that's actually not what we do.
1:15:41
SPEAKER_02: Yeah, of course not.
1:15:42
SPEAKER_00: Are there principles from that that we can take that help both groups?
1:15:46
SPEAKER_00: Because I think for us,
1:15:47
we do want to get to a state where these systems can be more
1:15:49
and more autonomous.
1:15:50
A real developer needs to go out and needs to fix these issues
1:15:54
all the time when they prompt it.
1:15:55
It also just means we're getting,
1:15:56
we're farther and farther away from being autonomous as well.
1:15:59
SPEAKER_00: So that's kind of the way we think about it.
1:16:02
SPEAKER_01: But I do think as an industry,
1:16:03
SPEAKER_01: and this is,
1:16:04
SPEAKER_01: you know,
1:16:05
SPEAKER_01: there's engineers who like the coders and then the non-coders.
1:16:09
SPEAKER_01: There is a question that needs to be asked of,
1:16:12
SPEAKER_01: Do we eventually need to understand what the code does?
1:16:14
SPEAKER_01: Do you need to be able to read the code?
1:16:15
SPEAKER_01: Because, for example, when I was at university, we studied assembly.
1:16:18
SPEAKER_01: Now,
1:16:18
SPEAKER_01: I never really programmed assembly beyond the class,
1:16:22
but I have since came across assembly code and I'm not afraid
1:16:26
to look at it.
1:16:27
Now,
1:16:27
again,
1:16:27
I'm not saying I'm the expert,
1:16:29
but you can go all the way down to the stack.
1:16:31
SPEAKER_01: And I think there is something to be said that,
1:16:33
SPEAKER_01: you know,
1:16:33
SPEAKER_01: we're now adding a new level of abstraction that...
1:16:36
SPEAKER_01: As a professional,
1:16:37
SPEAKER_01: it will always be helpful to be able to look through the stack,
1:16:40
SPEAKER_01: you know,
1:16:40
SPEAKER_01: sometimes all the way to the networking logs or the packet,
1:16:45
SPEAKER_01: not often,
1:16:45
SPEAKER_01: but just knowing where to look and eventually where to go.
1:16:48
SPEAKER_01: So this might be more of a philosophical question,
1:16:50
SPEAKER_01: because I think a lot of people don't want,
1:16:52
SPEAKER_01: they just think,
1:16:52
SPEAKER_01: okay,
1:16:53
SPEAKER_01: we can just use English for everything.
1:16:54
SPEAKER_01: But it does translate into a level,
1:16:57
SPEAKER_01: which is programming languages,
1:16:58
SPEAKER_01: translates into the next level and so on.
1:17:01
SPEAKER_00: I think you're right.
1:17:02
SPEAKER_00: So here's my take on it.
1:17:04
SPEAKER_00: We are going to have a proliferation of software.
1:17:06
SPEAKER_00: Some of the software will be built by people that don't know code, right?
1:17:10
I think it feels simplistic to say that that is not going to happen,
1:17:14
right?
1:17:14
SPEAKER_00: And we're already seeing it in real time.
1:17:16
SPEAKER_00: But here's the thing.
1:17:17
SPEAKER_00: It's almost like when you think about the best developer that you know,
1:17:21
even if they're a full stack developer,
1:17:23
they probably understand when the product is slow.
1:17:26
It's because there's some issue with the way that this
1:17:27
interacts with the operating system.
1:17:29
If there's some issue with the way that this interacts with the
1:17:31
networking stack,
1:17:32
it's the ability for this person to kind of peel back layers of
1:17:35
abstraction to get
1:17:36
SPEAKER_00: to ground truth.
1:17:37
SPEAKER_00: That is what makes a great developer, a great developer.
1:17:40
SPEAKER_00: And these people are more powerful.
1:17:42
SPEAKER_00: They're more powerful in any organization.
1:17:43
You know that you can take these people and put them on any
1:17:45
project and it's just
1:17:47
SPEAKER_00: going to be a lot more successful with them.
1:17:49
SPEAKER_00: And I think the same thing is going to happen,
1:17:50
SPEAKER_00: which is that some set of projects,
1:17:52
it is going to be fine if the level of abstraction you deal
1:17:55
with is the final
1:17:56
SPEAKER_00: application plus English and a spec,
1:17:59
SPEAKER_00: right?
1:17:59
SPEAKER_00: For some other set of applications,
1:18:01
SPEAKER_00: it's,
1:18:01
SPEAKER_00: you know,
1:18:02
SPEAKER_00: actually a developer will go in,
1:18:03
SPEAKER_00: but there's going to be some gnarly natures,
1:18:05
SPEAKER_00: right?
1:18:05
SPEAKER_00: It's going to interact with the database.
1:18:06
SPEAKER_00: It's going to have some hype.
1:18:07
SPEAKER_00: It's going to have like performance related issues.
1:18:10
And you're going to have an expectation that the AI and the
1:18:12
human can go down the
1:18:13
SPEAKER_00: stack and the human can reason about this.
1:18:15
Yeah.
1:18:16
SPEAKER_00: And I think these people are always going to be really valuable.
1:18:20
SPEAKER_00: Similar to how I think actually like our best engineers can,
1:18:23
SPEAKER_00: if I ask them to go and look at the object dump of,
1:18:26
SPEAKER_00: of,
1:18:27
SPEAKER_00: of like a C++ program and actually understand,
1:18:30
SPEAKER_00: Hey,
1:18:30
SPEAKER_00: actually,
1:18:31
SPEAKER_00: you know,
1:18:31
SPEAKER_00: here's a place where we're,
1:18:33
SPEAKER_00: here's a function.
1:18:34
Here's a place where we're seeing a massive amount of like
1:18:36
contention and we need
1:18:38
SPEAKER_00: to go out and fix this.
1:18:38
SPEAKER_00: Right.
1:18:39
SPEAKER_00: And,
1:18:40
SPEAKER_00: If the developer didn't understand the sort of fundamentals,
1:18:45
SPEAKER_00: they would be much worse at our company because of that.
1:18:48
SPEAKER_01: Yeah,
1:18:48
SPEAKER_01: I wonder if an analogy might be that a car mechanic,
1:18:50
SPEAKER_01: you know,
1:18:51
SPEAKER_01: car mechanics evolved over time.
1:18:52
SPEAKER_01: Like my dad used to,
1:18:53
we used to have like these old school cars where he would take
1:18:56
apart the engine.
1:18:57
He would take the whole thing apart and then put it back
1:19:00
together over a weekend.
1:19:01
SPEAKER_01: Like all the parts lay in, I remember.
1:19:03
And of course,
1:19:04
by the time,
1:19:04
you know,
1:19:05
I got to owning a car,
1:19:06
I could change the oil.
1:19:08
And now I have an electric car,
1:19:09
which is,
1:19:10
you know,
1:19:11
like there's not as many moving parts.
1:19:12
SPEAKER_01: However, someone who understands...
1:19:15
SPEAKER_01: how cars work,
1:19:16
SPEAKER_01: how they're built,
1:19:16
SPEAKER_01: how they evolved,
1:19:18
SPEAKER_01: they will always be more in demand for special cases.
1:19:21
SPEAKER_01: For example, I just had my 12-volt battery die in my electric car.
1:19:24
SPEAKER_01: I had no idea there was a 12-volt battery,
1:19:26
SPEAKER_01: but apparently I talked with someone who is in this and like,
1:19:28
SPEAKER_01: yeah,
1:19:28
it's from the gas cars and this is why and this is the reason
1:19:30
and this is how the
1:19:31
SPEAKER_01: new version will evolve.
1:19:33
SPEAKER_01: So like...
1:19:35
Clearly,
1:19:36
the majority of people might not need it eventually,
1:19:38
but there is that expertise.
1:19:40
SPEAKER_01: Plus,
1:19:40
these are the people who understand everything who will often
1:19:42
take innovation
1:19:43
forward because they understand what came before and they
1:19:46
understand what needs to come.
1:19:48
SPEAKER_00: You're totally right.
1:19:49
Maybe one other thing that I would want to add to what you
1:19:52
basically said is when
1:19:54
SPEAKER_00: you look at what great computer scientists and software engineers do,
1:19:58
I think they're great problem solvers given understanding sort
1:20:02
of a high level sort
1:20:04
SPEAKER_00: of business case or what the company really wants to do.
1:20:08
SPEAKER_00: And there are people that can distill it down.
1:20:09
And I think that skill is actually what I think boils down to
1:20:13
when you meet great engineers.
1:20:14
SPEAKER_00: It's not just like you tell them about a feature.
1:20:16
SPEAKER_00: You tell them about an issue.
1:20:18
SPEAKER_00: a desired outcome.
1:20:19
SPEAKER_00: And they will go out and find any way possible to go out and get to that.
1:20:24
SPEAKER_00: I think that's what great engineers are.
1:20:25
SPEAKER_00: They're problem solvers.
1:20:26
SPEAKER_00: And that's always going to be in demand.
1:20:28
SPEAKER_00: Now, is the person that builds the most boilerplate website
1:20:33
SPEAKER_00: And that is the only thing they are excited to do in the future.
1:20:36
SPEAKER_00: That person's skill set is going to be depreciating with time.
1:20:40
But that's a simplistic way of looking at it because if they
1:20:43
were a software engineer,
1:20:44
SPEAKER_00: they should know how to reason about systems.
1:20:46
SPEAKER_00: They should be good problem solvers.
1:20:48
SPEAKER_00: I think that's the hallmark of software engineering as a whole.
1:20:52
SPEAKER_00: And they will always have a position out there, in my opinion.
1:20:55
SPEAKER_01: Now,
1:20:56
SPEAKER_01: since you started to build Windsor or even Codium,
1:20:58
SPEAKER_01: how has your view changed on the future of software engineering?
1:21:01
SPEAKER_01: And we've touched on a few things,
1:21:03
SPEAKER_01: but have there been some things like before and after,
1:21:07
SPEAKER_01: now you're thinking about things differently?
1:21:10
SPEAKER_00: I think that timelines for a lot of things,
1:21:12
SPEAKER_00: I'm less scared of them,
1:21:14
even though I think a lot of them are supposed to come out as
1:21:18
scary numbers.
1:21:19
I think recently Dario from Anthropic was 90% of all committed
1:21:23
code is going to be AI generated.
1:21:24
SPEAKER_00: I think the answer to that is going to be yes.
1:21:26
SPEAKER_00: My question after that is, so what?
1:21:28
SPEAKER_00: So what if that's the case?
1:21:29
SPEAKER_00: Developers don't only spend time writing code.
1:21:32
SPEAKER_00: I think there's this fear that comes from all this stuff.
1:21:36
SPEAKER_00: I think AI systems are going to get smarter and smarter very quickly.
1:21:40
SPEAKER_00: But look,
1:21:42
SPEAKER_00: when I think about what engineers love doing,
1:21:44
SPEAKER_00: I think they love solving problems,
1:21:46
SPEAKER_00: right?
1:21:46
They love collaborating with their peers to find out how to
1:21:49
make solutions that work.
1:21:51
SPEAKER_00: And I think when I look at the future,
1:21:53
SPEAKER_00: it's more like things are going to improve very quickly,
1:21:55
but I think people are going to be able to focus on the things
1:21:57
that they really
1:21:57
SPEAKER_00: want to do when they're developers.
1:21:59
SPEAKER_00: Not like the nitty gritty details that,
1:22:01
SPEAKER_00: as you said,
1:22:01
SPEAKER_00: you go home and you're like,
1:22:03
SPEAKER_00: I don't know why this doesn't compile.
1:22:05
SPEAKER_00: I think that will,
1:22:06
a lot of those small details for most people are going to be a
1:22:10
relic of the past.
1:22:11
Well,
1:22:11
I'll tell you,
1:22:12
I'll give the idea to decide why people are stressed.
1:22:16
SPEAKER_01: And they're going to say,
1:22:17
SPEAKER_01: some listeners will say,
1:22:18
SPEAKER_01: well,
1:22:18
you're in an easy position because you're in the middle of an
1:22:20
AI company building
1:22:21
SPEAKER_01: all these tools,
1:22:21
SPEAKER_01: which is the future,
1:22:22
SPEAKER_01: right?
1:22:23
SPEAKER_01: And you're going to be fine for the next few years.
1:22:25
And they're thinking,
1:22:26
I'm sitting at a B2B SaaS company where I'm building this software.
1:22:31
And my employer is thinking that these things make us 20% or
1:22:35
25% more efficient.
1:22:36
SPEAKER_01: And they're going to cut a quarter of a team.
1:22:38
And I'm worried,
1:22:39
A,
1:22:40
if it's going to be me,
1:22:40
B,
1:22:41
the job market is not that great.
1:22:42
And I get it that I can be more productive with these things,
1:22:45
but I still need to find a job.
1:22:47
SPEAKER_01: And that is the,
1:22:48
SPEAKER_01: you know,
1:22:48
SPEAKER_01: like not everyone will verbalize this,
1:22:50
SPEAKER_01: but this is the thing that gives people,
1:22:52
SPEAKER_01: you know,
1:22:53
SPEAKER_01: when they're hearing Dario talk about the 90%,
1:22:54
SPEAKER_01: they're thinking,
1:22:55
SPEAKER_01: oh,
1:22:55
SPEAKER_01: damn,
1:22:55
SPEAKER_01: my employer will say like,
1:22:56
SPEAKER_01: okay,
1:22:57
SPEAKER_01: Joe,
1:22:57
SPEAKER_01: we don't need you anymore.
1:22:59
SPEAKER_00: Yeah.
1:22:59
SPEAKER_00: The problem is,
1:23:00
SPEAKER_00: I don't know what,
1:23:01
SPEAKER_00: maybe this is like,
1:23:02
SPEAKER_00: I don't know if this is like a real good answer,
1:23:05
SPEAKER_00: but that feels like the employer is being like irrational because,
1:23:08
SPEAKER_00: okay,
1:23:09
SPEAKER_00: let me provide the take here.
1:23:10
If the B2B SaaS company that is not doing well needs to compete
1:23:14
with other B2B SaaS companies,
1:23:15
SPEAKER_00: If they reduce the number of engineers that they have,
1:23:18
they're basically saying their product is not going to improve
1:23:20
that quickly
1:23:21
compared to a competitor that is willing to hire engineers and
1:23:24
improve their
1:23:24
SPEAKER_00: software much more quickly.
1:23:25
I do think consumers and just businesses are going to have much
1:23:29
higher expectations
1:23:30
SPEAKER_00: for software.
1:23:31
SPEAKER_00: So the demand for software that I buy is way higher.
1:23:34
SPEAKER_00: I don't know if I've noticed this.
1:23:36
I feel bad when I buy a piece of software that looks like it
1:23:38
did a couple of years ago.
1:23:40
SPEAKER_00: That's like this ugly procurement software.
1:23:42
SPEAKER_00: Yeah.
1:23:42
SPEAKER_00: And these days you don't have a... I hear you.
1:23:45
SPEAKER_00: I see the short term of like,
1:23:49
SPEAKER_00: are there employers that look at this and they're like,
1:23:52
SPEAKER_00: this is an opportunity to cut?
1:23:53
SPEAKER_00: I think these employers are being really, really short-sighted.
1:23:56
SPEAKER_01: Yeah.
1:23:56
SPEAKER_01: And I think I'm getting a little bit of hope from even other industries.
1:23:59
SPEAKER_01: There was a time where people...
1:24:01
SPEAKER_01: writers were being fired left and right.
1:24:02
Like I'm not saying software writers,
1:24:03
but like just like old traditional writers.
1:24:06
And now there's a big hiring spree from all sorts of companies
1:24:08
of hiring writers
1:24:09
SPEAKER_01: because it turns out the AI is kind of,
1:24:11
SPEAKER_01: you know,
1:24:12
SPEAKER_01: it's a bit bland and a great writer with AI is way better than without.
1:24:16
SPEAKER_01: I think same for software engineers.
1:24:18
SPEAKER_01: So that's also a bit of my message for anyone listening.
1:24:20
SPEAKER_01: It was just good to hear from you.
1:24:22
SPEAKER_01: Exactly.
1:24:22
SPEAKER_00: When you have a competitive market and you add a lot of automation,
1:24:26
SPEAKER_00: automation is great,
1:24:27
SPEAKER_00: but what you actually need to compare is automation with a human.
1:24:31
And if that's way more leveraged,
1:24:33
then you actually should compete with that.
1:24:35
SPEAKER_00: That's like the game theoretically optimal thing to do.
1:24:37
SPEAKER_01: And actually that's the tool that you're building right now,
1:24:40
SPEAKER_01: which I think is one of the reasons that I like to use it.
1:24:43
SPEAKER_01: It doesn't feel that it's like trying to do anything.
1:24:45
Instead of me,
1:24:45
it's doing it with me and making me way more efficient as an engineer.
1:24:49
SPEAKER_01: So to wrap up, I just have some rapid questions.
1:24:51
SPEAKER_01: I'm just going to ask them and then you can shoot the answer.
1:24:55
SPEAKER_01: So I've heard that you're really into endurance sports,
1:24:58
SPEAKER_01: long distance running,
1:24:59
SPEAKER_01: cycling,
1:24:59
SPEAKER_01: and you do just a lot of it.
1:25:01
SPEAKER_01: Now,
1:25:02
A lot of people are thinking,
1:25:04
well,
1:25:05
I'm pretty busy with my job,
1:25:06
with coding,
1:25:06
et cetera.
1:25:07
SPEAKER_01: I don't have as much time for sports.
1:25:08
SPEAKER_01: How do you make time for sports?
1:25:09
And what would your advice be for someone who is like actually
1:25:12
want to get in a lot
1:25:13
SPEAKER_01: better shape while being a software engineer and busy with your work?
1:25:16
SPEAKER_00: So I will say this,
1:25:17
SPEAKER_00: like since the company that has gone down drastically,
1:25:19
SPEAKER_00: but my previous company,
1:25:21
SPEAKER_00: I still worked a ton.
1:25:23
SPEAKER_00: I worked at an autonomous vehicle company.
1:25:25
SPEAKER_00: I would bike over like 150 miles an hour.
1:25:27
SPEAKER_00: uh, a week, like rigorously, like probably close to 160, 170.
1:25:31
SPEAKER_00: Um,
1:25:32
SPEAKER_00: I think it's just interestingly,
1:25:33
SPEAKER_00: it's I,
1:25:34
SPEAKER_00: for,
1:25:34
SPEAKER_00: for,
1:25:35
SPEAKER_00: for an activity like this,
1:25:36
SPEAKER_00: I actually got Zwift.
1:25:37
SPEAKER_00: So like this way to bike indoors,
1:25:40
SPEAKER_00: uh,
1:25:40
SPEAKER_00: and I would just be able to knock out like 20 to 25 miles in an hour,
1:25:43
SPEAKER_00: uh,
1:25:44
SPEAKER_00: like at home.
1:25:45
Uh,
1:25:45
and,
1:25:46
and the benefit there is like,
1:25:47
now I can come back from work very quickly,
1:25:49
do a ride.
1:25:50
SPEAKER_00: And then, you know, on the weekends,
1:25:52
SPEAKER_00: on a Saturday,
1:25:53
SPEAKER_00: I would just dedicate being able to do potentially like 70,
1:25:56
SPEAKER_00: a 70 mile loop somewhere.
1:25:57
SPEAKER_00: One of the lucky things for me is I'm in the Bay area.
1:26:00
SPEAKER_00: So there's a lot of like amazing places to ride a bike,
1:26:03
SPEAKER_00: um,
1:26:03
SPEAKER_00: like that have hills and stuff like that.
1:26:05
SPEAKER_00: So I think it's easy to carve out this time,
1:26:07
SPEAKER_00: but you kind of,
1:26:08
SPEAKER_00: you know,
1:26:09
SPEAKER_00: you need to make the friction for yourself a lot lower,
1:26:11
SPEAKER_00: right?
1:26:11
SPEAKER_00: I think if I needed to, I would never go to a gym or like rigorously.
1:26:16
SPEAKER_00: I think I'm, I'm not the type of person that would like, you know,
1:26:18
SPEAKER_00: I would just find a way to not do it.
1:26:20
But if it's literally at home,
1:26:22
right next to where I sleep,
1:26:24
I'm going to find a way to do it.
1:26:25
SPEAKER_01: Sounds like just make it work for you.
1:26:27
SPEAKER_00: Yeah.
1:26:28
SPEAKER_01: And what's a book that you would recommend and why?
1:26:31
You know,
1:26:32
there was a book that I read a long time ago that I really enjoyed.
1:26:35
SPEAKER_00: It's called The Idea Factory.
1:26:37
It's basically about how Bell Labs kind of like innovated so
1:26:40
much while being a
1:26:41
SPEAKER_00: very commercial entity.
1:26:43
And it was very interesting to see some of like the great
1:26:45
scientists of our time
1:26:46
SPEAKER_00: working at this company,
1:26:48
SPEAKER_00: providing so much value.
1:26:48
SPEAKER_00: So like information theory, Claude Shannon worked there, right?
1:26:52
The founding of the transistor happened sort of like Shockley
1:26:56
and all these people
1:26:57
SPEAKER_00: kind of were there too.
1:26:58
And just hearing how a company is able to straddle the line
1:27:01
between both was really exciting.
1:27:03
SPEAKER_01: Yeah.
1:27:04
SPEAKER_01: And I hear that, you know, OpenAI got inspired by Bell Labs a lot.
1:27:07
SPEAKER_01: Their titles are coming back.
1:27:09
SPEAKER_01: And I think I actually, I personally want to read more about that.
1:27:12
SPEAKER_01: So thanks for recommendation.
1:27:13
SPEAKER_01: Well, thank you.
1:27:14
SPEAKER_01: This was great.
1:27:15
This was super interesting and just love all the insights that
1:27:19
you shared.
1:27:20
SPEAKER_01: Yeah.
1:27:20
SPEAKER_01: Thanks a lot for having me.
1:27:21
I hope you enjoyed this conversation with Varun and the
1:27:23
challenges that the
1:27:24
SPEAKER_01: Windsurf team is solving for.
1:27:25
One of the things I enjoyed discussing was when Varun shared
1:27:28
how they have a bunch
1:27:29
SPEAKER_01: of features that just didn't work out,
1:27:30
SPEAKER_01: like their review tool,
1:27:31
SPEAKER_01: and then they celebrate failure and just move on.
1:27:34
I also found it fun to learn how any developer can roll out any
1:27:37
feature they built
1:27:37
SPEAKER_01: to the whole company and get immediate feedback,
1:27:40
SPEAKER_01: whether it's good or bad.
1:27:41
SPEAKER_01: For more deep dives on AI coding tools,
1:27:43
SPEAKER_01: check out the Pragmatic Engineer Deep Dives link in the show notes below.
1:27:47
SPEAKER_01: If you've enjoyed this podcast, please consider leaving a rating.
1:27:49
SPEAKER_01: This helps more listeners discover the podcast.
1:27:52
SPEAKER_01: Thanks, and see you in the next one.