Latest / Agentic Conversations (formally mlops.community) / A Candid Conversation with the CEO of Stack Overflow

Transcript

A Candid Conversation with the CEO of Stack Overflow · Agentic Conversations (formally mlops.community)

0:00
The first answer will be a stack overflow link.
0:02
You're growing and profitable. Yes, we work with everybody,
0:05
Open AI and Google and all the cloud providers.
0:07
At some point, the AI taps out. That's why we want to be able to
0:11
integrate into these tools. So we want to make sure we are
0:14
where users are. The big portion of what we're
0:15
doing now is expanding well beyond Q&A.
0:21
Alright, you just told me something very important.
0:26
You're growing and profitable. Yes, we are growing and
0:28
profitable despite, you know, all the the viral headlines
0:32
around our specifically just our Q&A, which is an interesting
0:36
stat to look at because obviously in the age of AI,
0:38
you've got people asking a lot of simple questions on let's
0:42
call it Chachi PT, any AI tool, right?
0:44
And that's I think it's like it's, it's more it's just a
0:46
function of being a content site on the Internet.
0:48
I think a lot of content sites now had that same point around
0:51
who's creating the new knowledge.
0:53
And so for us, it's thankfully like the increment because of
0:56
the experts in the platform etcetera.
0:59
And the stand has always been about let's have the bar really
1:01
high and let's create this amazing corpus of cumulative
1:07
information and knowledge that is going to be very useful,
1:09
including for all the AI models who now work with us in an
1:12
official capacity by licensing our data.
1:15
Yeah, I was going to ask about that.
1:16
Basically any coding helper yeah is powered in one way, shape or
1:22
form by models that are powered by the Stack Overflow answers.
1:27
That's right. That's right.
1:28
So, so specifically just to finish the earlier thought, so
1:31
people come to our site to ask all the incrementally difficult
1:34
questions, but all the simple questions now are also, you
1:38
know, are being asked on places like ChatGPT, but they're also
1:41
now in a we're empowering people to ask those questions living on
1:44
our own platform through multiple mechanisms, which I'll
1:46
talk to you about AI and non Q&A related.
1:49
So we want to, we want to accept all questions, but at the same
1:52
time keep the Q&A corpus very, very sort of pristine and like
1:56
incrementally kind of accurate. So it's very useful for these AI
1:58
providers. So to answer your second
2:00
question, yes, all the AI providers work with us
2:04
officially a few years ago when chat CPT came out in late 2022,
2:08
we noticed, you know, all the AI companies and LLM labs or AI
2:12
labs had used our data to train or pre train their models.
2:15
So we soon put up a bunch of anti scrapers on our site and
2:19
they, you know, they, they all sort of wanted to work with us
2:21
because we generate obviously very high quality data.
2:25
We even worked with the the process team to actually think
2:27
about. We took an open source model and
2:29
fine-tuned it with our data. And now we actually published
2:31
these results with something called Pro LLM dot AI, where our
2:35
stacked data is used to rate all the latest foundational models.
2:41
I love that. That's so valuable, especially
2:43
because the data isn't, it's like public but not public.
2:48
And so you have these models that as soon as they come out,
2:51
you can get a quick gauge at how good they are on coding tasks.
2:55
Exactly right. So with that information, we
2:57
noticed, you know, something like a 30 to 40% improvement in
3:00
accuracy of answers. And so either if people were
3:05
using the data already, then they now realize they have to
3:07
officially license it through us, which is now a requirement.
3:10
Attribution is very important because that's part of our
3:12
license. You need to attribute back to
3:14
the sources so it recognizes our users, etcetera.
3:17
And ultimately, we're able to contribute back to the community
3:20
by building new features, which I'll talk to you about.
3:23
So that, yes, we work with everybody open AI and Google and
3:26
all the cloud providers and the, and the, you know, all the AI
3:29
labs and they officially license our data.
3:31
And they're, they're more the everyday we're seeing new
3:34
companies come into the, into the, you know, to the limelight.
3:37
So we're happy to work with them.
3:38
Yeah. There's something that is to be
3:40
said about the experience of Stack Overflow versus just
3:44
asking one of these answer engines.
3:46
Yeah, and it's along the lines of there's more than one way to
3:50
skin a cat. And so when I go to Stack
3:53
Overflow, I get a much more robust answer because I can see
4:00
various opinions on things and I can get the nuances a lot more
4:04
versus, oh, here's how it's done, or oh, let me just go try
4:09
and do that. If it's a coding agent and then
4:11
it tries it and it doesn't work because some package isn't
4:14
updated or something, and then you have to go through that
4:16
loop, etcetera. Right, right.
4:19
No, exactly. I think the the depth of the
4:21
answers where people over a 1617 year period have, you know,
4:26
started by writing their first, I don't know, basic programming,
4:29
you know, kind of like their first Hello world program all
4:32
the way to, you know, using more advanced languages over time.
4:36
And then as things have abstracted higher and higher, I
4:38
think people have you see that journey, right?
4:40
And so that depth of what is the right answer to canonically
4:44
right answer to a question and how that's changed over time.
4:48
You know, the accurate answer from six years ago may have been
4:51
very good, but now we've got a more elegant answer that gets,
4:54
you know. That's so true.
4:57
And the right answers bubble up because somebody tries something
5:02
and it doesn't work. And then they come back and they
5:04
put in the thread, hey, this is this doesn't work anymore.
5:06
That is right and. So you have that almost like up
5:09
to date answer. Exactly, and your point around,
5:13
you know, the depth of the answer is very useful because
5:16
what we do notice is that. And the reason why I think we're
5:19
very excited to work with all the AI companies is when AI,
5:22
let's say, does a lot of the, let's call it the grunt work and
5:25
they're kind of the, you know, the, the craft.
5:27
And that's good because it basically is all about
5:29
abstraction. Let's remove like the, the kind
5:31
of the millennial tasks. That's all good.
5:33
However, it does have a complexity Cliff issue.
5:36
At some point the AI taps out or it doesn't have the answer to a
5:40
question because it's specific and contextually relevant.
5:43
It doesn't have it based on the fact that nobody's actually come
5:46
across that issue and actually like produced a viable answer,
5:49
which all shows up on Stack Overflow.
5:51
So that's why we want to be able to integrate into these tools
5:55
and what we call knowledge as a service go where the users are.
5:57
So previously the interface was go to Google and type in a
6:01
question. Thank you.
6:02
How do I spin up a Kubernetes cluster?
6:04
The first answer will be a stack Overflow link.
6:06
You click on that, you come to our our our page and so on.
6:09
Now people use Google, but they also use a bunch of AI tools to
6:13
go and search for answers to your earlier point.
6:16
And so we want to make sure we are where users are.
6:18
So by many of our partnerships are about also not only
6:22
licensing our data, but also being able to empower users to
6:26
ask questions straight into stack overflow from those
6:28
places, right or and and and addition also attribute back to
6:32
stack Overflow from their from their AI, you know, their chat
6:37
box, because that way people come and look at the more
6:39
detailed answers to your earlier point also around here.
6:42
Excuse exactly how you do it, why you're why we're suggesting
6:44
this sort of solution. So that's the the full loop that
6:47
we're trying to do kind of enable plus obviously on our own
6:50
website, being able to basically bring people.
6:53
And we just launched this this week, which is Stackoflow dot AI
6:56
on the site where people can bring their LLM chat straight
7:01
into this and drop it in there. It imports it and then it looks
7:05
at the corpus of 15,000,000 questions and answers that we
7:07
have and then provides A relevant answer.
7:09
Or if none of that works, then you can post a question to the
7:11
community either in one of our now newly formed live chat rooms
7:15
with humans, human experts, or in our traditional Q&A or even a
7:19
discussion board nowadays. Yeah.
7:21
So I think it's a good moment to just mention the what everyone
7:26
thinks of as they look at Stack Overflow now and they're like,
7:29
oh, well, the questions are going down because of this.
7:32
You rightly so kind of talk about yeah, probably the basic
7:37
questions are getting answered by the answer engines.
7:41
The more nuanced discussions are still happening.
7:44
Can you explain what you've been seeing as far as questions being
7:48
asked? Is it more in depth questions?
7:51
Is it always more topical questions that are just not in
7:56
the corpus of data and so you you don't have them and they're
7:59
encountering them? They're very, if you take a step
8:01
back, we've historically only had Q&A and the Q&A, again, the
8:05
mission and purpose of Q&A was what is the almost like the
8:08
encyclopedia, the, you know, the Library of Congress for all
8:12
things software programming, right?
8:14
So it literally has that. Super valuable too, by the way.
8:17
I was going to tell you before the most valuable form for
8:20
developers by far, but. Yes, exactly.
8:23
And that's why it's been useful for all the AI companies to
8:26
leverage. So with that Library of
8:28
Congress, it's like, you know, if you have the same book like 6
8:30
times, like it's not useful. Like what we're really trying to
8:33
do is like say what is the next book, you know and so on and so
8:36
we want to keep adding new books into that.
8:39
So previously, so one out, one out of every two questions were
8:42
closed on Stack Overflow in just AQ and A.
8:45
So the reason for that is because they were all duplicate
8:47
questions, right? So with my own first experience
8:50
using Stack Overflow, when I even before I joined the company
8:53
was like, Hey, this is questions been answered already, like, you
8:55
know, slap on the wrist, right? And and while that can be harsh,
8:58
the whole that was very deliberate by the original
9:00
founders of the company to say, let's create a Library of
9:02
Congress. That's truly like the cumulative
9:05
effects, cumulative effect of all the works in the world
9:08
around this topic. So, so that yes, it is now all
9:12
just, you know, what is the something that's never been
9:14
answered before? I'm a, you know, an expert on
9:17
Python And here's an additional book.
9:20
You know, Python, a lot of questions have been answered in
9:22
the world, but I would say, let's say something newer, you
9:24
know, maybe it's Transformers, maybe it's even something even
9:26
even more more current and one of the latest technologies in
9:29
AI. And so that goes additional
9:31
books keep getting added to the library, but that's the Q&A form
9:35
factor. But a big portion of what we're
9:36
doing now is expanding well beyond Q&A.
9:39
So back in March, we had an AMA with our community members and
9:44
we talked, we jointly created this now vision, which is to be
9:46
the most vital source for technologists period, right.
9:51
So the source is well, how we want to do that is to make sure
9:54
that we cultivate community. We want to power learning and
9:57
unlocking growth in people's careers at their work to be
10:00
productive and so on. And the way we want to do that
10:03
is especially with what's going to happen in the world, there's
10:05
going to be a lot of job displacement, etcetera.
10:07
We want to make sure that you any question actually also is
10:11
fine on stack of flow and maybe not in Q&A, which is why that's
10:13
why we've opened up, you know, hundreds of chat rooms on the
10:16
site which are live human chat. Now, that's not exactly novel,
10:19
but the fact that you've got experts in these places because
10:23
you can't remember, people have been building expertise on the
10:26
side. They're they've got reputations
10:28
on the side. I have a million points on
10:30
Python. OK, so you know what you're
10:31
talking about because you know, you're in the school of Hard
10:34
Knocks and for you to be in the Python chat room to be able to
10:37
help another developer out when they have an issue could be one
10:41
of the newer AI topics. It could be an AI specific chat
10:44
room. How do I prompt on this?
10:45
Why is my prompt not whatever it may be?
10:48
So chat discussions, coding challenges because coding
10:52
challenges like how do we help a young developer?
10:54
Because you know, those folks are going to struggle in this
10:56
environment most likely because companies, etcetera, unlikely to
11:00
dive straight in and hire a bunch of junior developers only
11:03
because a lot of the AI agents are pretty capable to do the,
11:06
the, let's say the simpler, you know, you work flows.
11:10
So all these ways that I just described, we're broadening the
11:13
site to be much more multi dimensional to realize our
11:16
mission and to help our developer community and
11:18
technologies community. So that's so yes, questions in
11:22
Q&A are incremental, but we are also now opening up all these
11:24
other avenues where any questions should be fine.
11:26
You ask a question, it either shows up in Q&A discussions or
11:29
chat and, and, and all that will be helpful, you know, through
11:34
different mechanisms for the community.
11:35
Yeah, I really like the idea of Ask an Expert because that is
11:38
something that with the ML OPS community, we've had in mind.
11:42
And it's almost the way that I feel like the ML OPS community
11:47
was valuable or is valuable to folks because you you have
11:51
different experts from different walks of life.
11:53
And we used to joke back in the day that the ML OPS community
11:57
Slack was like Slack overflow. And it was almost like real time
12:01
Slack overflow. And what I've seen when it comes
12:04
to the questions that folks are asking in the ML OPS community
12:07
Slack, is that those basic questions that you can really
12:11
answer easily, they're not being asked as much anymore.
12:14
What you do have are more nuanced questions and things
12:17
that you really want to talk to a human about and you want to
12:21
get a human opinion on. It's not like what's the, what's
12:25
the answer to X, right? You want this more nuanced
12:29
discussion. You want to hear opinions.
12:31
You don't want to hear an answer.
12:33
Correct, exactly right, which is why we opened up something
12:35
called discussions on the side. So they're a lot less stricter
12:39
in terms of the Q&A is meant to be the canonical answer to a
12:42
question. This is more of like, look, you
12:45
can ask whatever you want. What is what is the discussion
12:48
around the topic? You know, should I be using XYZ
12:51
wipe coding tool versus the other wipe coding tool?
12:54
What are the pros and cons of X versus Y whatever you want,
12:57
right. So that's I think it's less
12:59
about it's a, it's a conversation versus our
13:01
discussion versus here's the accurate answer to the question.
13:04
Exactly. There's so much Gray area that
13:07
you have to cover and you have to hear all the different
13:09
opinions from folks and you can't just say this is how you
13:12
do. It.
13:13
Because if you're looking for, this is how you do it.
13:15
You have it there, it's closed, or maybe it's getting updated.
13:19
As accepted answer or you know, and so on right in the Q&A area.
13:23
Exactly. I think this is probably a good
13:25
moment to talk about what types of questions you've seen as far
13:29
as like the amount of AI questions or the way that
13:34
questions have changed over time.
13:36
Yeah, yeah. I think that that I think we
13:38
sort of covered in that they're more specific and they're more,
13:42
you can call them like, you know, incrementally sort of very
13:44
complex relative to the standards of the questions that
13:47
now that that will also change with what I described, because
13:49
you could ask that repeat question and chat and
13:51
discussion. So we're sort of in between both
13:54
those. So in in specifically in Q&A, we
13:56
see the more complex questions. We see increasingly number of AI
14:00
related conversations happening on that site or questions around
14:04
certain AI talk technologies. So we have, you know, everything
14:07
from, you know, NVIDIA CUDA tags to, you know, Tensorflow to, you
14:12
know, any, any of the people who are here at this conference, you
14:14
know, they all have content on the site, right.
14:16
So that's all I would say. There's obviously an increasing
14:18
number of developers learning about those technologies because
14:22
they realize that's where the world's headed.
14:24
And so that is naturally the place where that happens.
14:27
So, and then, but even for the person who is early in their
14:30
programming journey or developer journey, they're learning the
14:33
fundamentals, which we very much encourage because it's like, you
14:36
know, you know, you know, you want to know when you are using
14:39
a calculator that 2 + 2 is not 6 right at some point.
14:43
So like you want to learn the fundamentals of math.
14:45
So in the similarly, we encourage people to, to learn
14:47
all that and do it on stack because you will learn through,
14:51
you know, going through this journey of learning on stack.
14:53
So the questions are complex on Q&A, but again, simpler on chat
14:57
and discussions. And and yes, they're
15:00
increasingly AI oriented on the Q&A side.
15:03
I've got to know because I asked you before we hit record.
15:07
Why don't you build a cursor or a windsurf with all of this
15:12
information? Yeah.
15:14
So I think we are, it's, I don't think there's actually now a,
15:17
it's kind of the same question. I sometimes get around why
15:20
haven't we built our own LLM, right.
15:22
So why can't we just go do that? And so the answer is that I
15:25
think we have certain points of view.
15:26
For example, I believe the LLM game is fairly commoditized.
15:30
So over time, they're all going to sort of get to the same
15:34
level. Maybe one will get to AGI and
15:36
all those sort of father predictions that people have.
15:38
And there's also the open source topic because like when this
15:42
knowledge just becomes ubiquitous and then the question
15:44
is, you know, you spent, let's say a few $100 million training
15:47
these models and the whole ecosystem is running on that.
15:50
So the question is like, you know, how do we want, we want to
15:53
just serve our community the best way we can.
15:55
We have a great knowledge base. The user interface seems to have
15:58
changed. So we want to sort of play the
16:00
substrate layer at the moment where we are powering all these
16:03
AILLM providers, AI labs, even the AI agents.
16:08
And I can we should talk about our enterprise scenario, which
16:11
is like a very fascinating thing to see how enterprises are using
16:15
Stack Overflow inside their companies, which has given us a
16:18
very sort of key insight. So all this suggests that we,
16:22
for AI to work, you need a very trustworthy foundation of
16:28
knowledge from experts. And it comes down to when we
16:32
talk to our developers, developers, every year we do a
16:35
developer survey, as you're aware.
16:37
And, you know, that's our survey, yes.
16:39
I'm a big fan of that because you get so much information from
16:43
folks and there's so many diverse opinions.
16:47
Again, that's I think the beauty of Stack Overflow is the diverse
16:50
opinions, the willingness to share.
16:52
That's right. So, so the key insight that to
16:55
answer your question, the key insight from our this year's
16:56
developer, a survey that just closed that we have not yet
16:59
released. What I'll share it with you is
17:01
that not surprisingly the AI adoption for coding related
17:05
tasks has gone up. So it's gone up roughly over the
17:07
past three years from let's call it 60% of our users up to about
17:12
80% of our users. So not surprising everybody's
17:14
using AI tools at least trying or interested to try it.
17:17
But what is very surprising is the trust level around what's
17:24
coming out of these AI tools. I would have expected to go up,
17:27
but actually it has gone down. So for the past three years, it
17:30
has been 40% of the users trust what's coming out of AI tools to
17:35
40% year 2, which is last year. They would trust AI tools to 29%
17:40
this past year. All that this year.
17:42
Spaghetti code is coming to roost now.
17:44
Finally, three years later. So it is quite counterintuitive,
17:47
but at the same time quite fascinating.
17:48
That was the one data point that stood out to us.
17:50
Like the trust factor is really, really key.
17:54
And you know, in a world where you've got a lot of AI slop and
17:57
if you go to the Internet, you, you know, you're going to see
17:58
all sorts of AI generated content.
18:00
So there's going to be very few places where you actually go to
18:03
get very credible, authentic, human curated knowledge.
18:08
Even if you use AI to sort of do things, but you need the human
18:12
to make sure it's accurate and validated and, and you know,
18:15
it's actually trustworthy ultimately.
18:17
So, so that is, that's what we longed onto saying.
18:22
That's the problem that speaks to our strengths.
18:24
Our strengths are people trust us, they trust our brand, they
18:28
trust our community. It's in fact it's really hard in
18:31
some ways to engage with our community because it's actually
18:33
really, really the standards are very high.
18:36
So you know, we're not going to be bought by commercial
18:38
interest. You know, it's like it's one of
18:39
these things it's like a very independent, like a Switzerland
18:41
type of place. So that I think is one of our
18:43
primary strengths. So yes, could we go and build a
18:47
cursor or, you know, something like that perhaps.
18:51
But you know, it's not exactly. We could, you know, we never say
18:53
we wouldn't do it at some point. But at the same time, it's not
18:57
something that we're focused on now because even I think there
18:59
is, it's very early in that world of trying to figure out is
19:02
it a, you know, is it an ID tool?
19:04
Is a command line tool? Is it a, you know, is it an
19:07
agent? Yeah.
19:09
And for what use cases? So we're very.
19:11
So I don't think that world's been sorted yet.
19:14
And I think it's still, you know, there's a lot of movement
19:16
between, I mean, you know, people have very low switching
19:20
costs between tools. That's just, you know, move from
19:22
one to the other. Yeah, a lot of folks use two or
19:24
three. Right.
19:25
So, yeah, so I think we always will be open to it and our AI
19:29
functionality on our public site now is maybe a step in that
19:32
direction to say, look, get the best of the humans expertise on
19:36
the site plus leveraging AI search and discovery and all the
19:40
things that you know are good about that.
19:41
But also over time, if we're able to capture the users
19:44
context, you know, be able to perform tasks over time.
19:47
But you know, that's not exactly a current focus.
19:49
But the the idea is to power the ecosystem through that
19:54
trustworthy knowledge intelligence layer, if you will.
19:57
And we should talk about the enterprise, which is another
19:59
very key data point. But I want to get to that.
20:01
I do think that it's smart to almost like be patient here
20:06
because you're right, two years ago we were having the same
20:10
conversation about why aren't you building an LLM, you have
20:12
all this data, you should make your own.
20:14
Now we realize that would have been a horrible.
20:16
Decision, right? Totally.
20:18
And that is not where you're going to have the delta, so or
20:23
the alpha, I guess I should say it.
20:26
In, in financial terms, yeah, yeah, yeah.
20:30
Exactly. Sure.
20:32
You can see where I play. I, I don't necessarily do the
20:35
CEO stuff. I don't have to go to the
20:36
earnings calls like that. Like like you.
20:39
I, I like that you took that approach.
20:41
Like, hey, let's be patient now when you're looking at ways to
20:46
make the company more interesting, you've put a big
20:52
bet on the enterprise stuff. Can you talk to me about what
20:55
you're doing there and how that even looks?
20:57
Totally. So the two sides of our company,
20:59
1 is obviously a public site which we've spent a bunch of
21:02
time on the way in which we obviously we're building all
21:05
these new features etcetera that I explained.
21:07
And the way our business model that supports that is our data
21:10
licensing business, which is to all the big AI labs as we talked
21:13
about, which is a very fast growing area for us where they
21:16
get access to the knowledge for for a fee, a recurring fee and
21:19
then advertising obviously. So that's one side.
21:22
The second part of our company is the enterprise version of
21:25
Stack Overflow that's used in literally thousands of companies
21:28
around the world. And that's actually for some
21:30
reason less known in the ecosystem because our brand is
21:33
so prevalent on the public platform that people just sort
21:36
of they look at. That's why, you know, I joke
21:38
about this like 1 chart that shows up the number of questions
21:40
and it's like, OK, that's Oh my goodness, it's the, it's the end
21:42
of the world. Not really because that is a
21:44
component. It's what we're well known for.
21:47
But our company is like multi dimensional in nature.
21:50
You know, we've got this B to C or B to D rather business
21:53
developer, you know, business developer.
21:54
And then we have this B to B business, which is all around a
21:58
private knowledge intelligence layer inside your company that
22:02
allows you to have human curated knowledge that will always stay
22:06
up to date and upgrade by your subject matter experts inside
22:10
the company. And now most recently what we
22:12
discovered is that all our largest customers are using this
22:16
very, very accurate knowledge base, which is stack of flow
22:19
internal to power their AI agents.
22:22
So what we started doing is that we noticed our AP is of all our
22:26
customers were red hot. And we when we talked to our
22:28
customers, they said, what are you doing exactly?
22:30
And they're like surfacing it in search results inside the
22:33
company. They're surfacing it using it in
22:34
AI assistance. As an example, Uber built
22:38
something called Uber Genie. It was one of our customers,
22:40
Uber, and Uber Genie leverages the thousands of questions and
22:46
answers within Uber's internal version of Stack Overflow and
22:50
surfaces it into that sorry powers AI assistant, which then
22:54
goes into that Slack channels and answers a whole bunch of
22:57
questions from users automatically that are all like,
23:00
you know, questions about the same stuff, how do I reset my
23:02
password at Uber and so on. And so it drives a tremendous
23:05
amount of productivity inside the company because it's tapping
23:08
into this very accurate knowledge base.
23:10
And knowledge bases are underestimated because you know
23:14
you know you have a lot of knowledge inside the company,
23:15
but either it's like hard to find or it's out of date.
23:18
Right. We all can sympathize with the
23:21
fact that maybe there's some stuff in Jira, maybe there's
23:24
some stuff in Confluence, maybe then somebody came and they have
23:27
a Notion database and then there's, Oh yeah, all that
23:30
stuff. No, that's actually on Git book.
23:32
And you've got this sprawl across all these different
23:36
areas. And I imagine that that type of
23:39
thing, building an abstraction on it and having the ability to
23:42
pull from wherever the answer is, if it's in Stack Overflow or
23:46
if it's in Ocean, whatever it may be, you have that and you
23:49
can get that answer. That's a huge unlock because I
23:53
can't tell you how many times I've spent searching for
23:57
something and I'm like, I swear it was.
23:59
Somewhere. Name was like around this and
24:02
you're typing in keywords and you're not finding it.
24:04
Yeah. Totally.
24:05
It's like, it's a very, very common frustration.
24:08
And that's why customers since 2019, I, I came on as ACEO in
24:12
2019. So, and partly was to solve this
24:15
problem. So inside companies, they had
24:17
this huge issue around developer productivity employees like, you
24:21
know, spending a lot of time just asking the same stuff and,
24:23
you know, tapping on the shoulder of the subject matter
24:26
expert always. And so exactly like you said
24:29
this, now Stack Overflow internal ingest information from
24:32
other knowledge sources that are all like out of date or or, you
24:36
know, getting out of date and creates this area on Stack
24:40
Overflow through the subject matter experts that we
24:41
automatically identify can automatically make sure that the
24:45
information is constantly up to date, much like our public
24:47
platform has done for 16 years. So we had that secret sauce of
24:51
what makes the curation engine really work, right with
24:54
reputation points and all the things.
24:56
And then that with when you plug it into an MCP server, you can
25:00
do all sorts of amazing things because you're getting this
25:02
really accurate knowledge stream that's powering, you know,
25:06
whatever, you know, could be your agent to perform an action,
25:09
could be assistant to provide information, whatever it may be.
25:11
Yeah. We were talking to Ben Young
25:13
yesterday who is the CTO of Source Craft and he was
25:16
mentioning one of the biggest unlocks for their agent amp is
25:22
giving it a little bit of a nudge on where to look right.
25:25
And so I imagine that's why people are, that's why you saw
25:29
the API is a red hot because the agents then if you just say, Oh
25:33
yeah, check Stack Overflow for that answer or get the context
25:38
from Stack Overflow, that could be a reason that all of a sudden
25:42
these agents are performing better and they're able to just
25:46
start looking around stack overflows for a lot of
25:48
different. Stuff that is right and we, you
25:50
know, and it's very apparent because a lot of our customers
25:54
are in financial services and healthcare and retail and tech
25:56
companies globally, right? And so they all have realized
26:01
that this is one of the few credible sources that they're
26:05
experts have spent time making sure it's accurate.
26:08
And so absolutely it goes back to trust and it's the same, the
26:12
same way we help AI agents on the public platform train off of
26:16
our data. Now with the commercially using
26:20
partnerships. Similarly here, those AI agents
26:23
inside companies in the enterprise are leveraging the
26:27
platforms data to be truly value added inside companies.
26:31
You know, ultimately these agents, AI agents and assistants
26:33
need to show ROI and there's going to be a tremendous amount
26:36
of pressure to do that. I think we're probably second or
26:38
third now in terms of phases around people's attempts to
26:42
deploy some AI related, let's call it technology inside
26:46
companies. So, and now they're like, look,
26:48
OK, we've figured out the dependencies, our privacy and
26:51
great data and it has to be secure and you know, and so on.
26:54
And we don't want our stuff getting out of our company to
26:56
other company, our competitors. Like all the things that people
26:58
have been debating over the past three years are finally, I think
27:02
people have figured out, OK, you need like a very accurate
27:05
knowledge base and you want that to be very seamlessly accessible
27:08
through an agent that can do a lot of great things.
27:11
So I think it's just like it's, it's timing.
27:13
It's in patients, especially when so much is happening in a,
27:16
in a kind of a hype cycle. You just have to know, you know,
27:20
if you've been through it a few times, you will know like you
27:22
know, the, the, the, you know, like the through of the loose
27:24
disillusionment and when people actually, you know, understand
27:27
the reality of what it means to actually deploy things.
27:29
It also is something that avoids the bus problem.
27:34
If somebody gets hit by a bus and they have that information
27:37
in their head, that's a big problem.
27:39
But if it's there on the internal Stack Overflow.
27:41
Exactly right, exactly right, though that's a very, very
27:44
common use case why people bring us in in the enterprises.
27:47
Let's make sure because you know, people are constantly
27:49
moving in and out of companies. People are bringing in new
27:52
people. You know, people are changing
27:54
teams. So when all that happens, say
27:57
Dimitrios is, is like answers, you know, on a certain topic
28:02
that he's an expert on are codified and they're built on
28:05
over time by other experts in the company.
28:07
But the fact is that you're the expert on the topic.
28:09
That's your intelligence is now in the company.
28:12
Companies like knowledge repository and irrespective of
28:16
what you end up doing with your own career, that the company
28:19
continues to benefit from investing in you, right, by
28:21
investing in you. So that's so the company should
28:24
benefit because that's just the right trade because you're
28:25
actually ultimately it's all about people and what they bring
28:28
to the story. So yeah.
28:29
Last thing that I wanted to ask around the questions was do you
28:36
see a lot of very obscure languages getting more questions
28:42
because these things like the not Python or TypeScript type of
28:48
questions, I imagine they don't have the same amount of data and
28:52
training data that the LMS can't answer those or they probably
28:56
hallucinate more when they're talking about.
28:59
New. Totally.
29:00
Totally. Languages, all that stuff. 100
29:01
percent, 100% and you will see some of that in our developer
29:04
survey that will come out in a few weeks and we'll share that
29:07
publicly. But yes, absolutely the that's
29:09
always been the Stacko flow. You know, we know what's
29:13
happening in the ecosystem even before we all like, you know,
29:16
realize it, because it's already like the number of questions and
29:18
certain topics have been sort of increasing.
29:19
So the new languages, you know, couple years ago was about Rust
29:22
and you know, swelt and you know, all these things.
29:25
So as they keep sort of progressing, it happens on the
29:27
platform first because somebody has to figure out how to use
29:30
this in the spirit of abstracting, you know, and going
29:34
up, you know, kind of up the stack, so to speak.
29:36
How do you make it more and more elegant, etcetera, as you're
29:38
aware. So those things absolutely are
29:40
the incremental sort of questions, the net new etcetera
29:43
because the world's questions on let's say JavaScript have let's
29:46
say vastly been answered, right. So the what's more to be
29:50
answered is like the more the incremental things and also the
29:52
dynamic of who is asking these questions is also quite
29:56
fascinating. Maybe I'll leave you with that.
29:58
Is that the one of the I mentioned earlier, the trust
30:00
score has gone down to 29% from AI related topics, even though
30:05
adoption of AI has gone up. And we'll, one of the
30:07
fascinating things we noticed in the demographic is that the more
30:10
experienced the programmer, the less they seem to trust AI,
30:15
which is interesting, right? But that is what we notice also,
30:19
it's another sort of quick tidbit from our developer survey
30:22
is that the, the kind of the, the, the experience level age
30:25
group has gone up and the trust level has gone down.
30:28
I suspect the younger developers are all like rushing straight
30:31
to, you know, the kind of the, the easy button of like, let's
30:35
use the agent to a bunch, do do a bunch of things, which I think
30:38
is good. But at the same time, again, we
30:40
would encourage people to do both, you know, learn the
30:42
fundamentals and do that. And so the expert programmers
30:46
are the, let's say the more experienced ones are the ones
30:48
who are spending time thinking about what is the latest
30:51
programming language. And, you know, what is, what
30:54
should I ask? How do I really sort of fully
30:56
understand? How does this hook up into the
30:58
rest of, you know, what libraries I should be using?
31:00
All the things you need to do. And why is this more elegant
31:02
than the previous version of the same problem?
31:05
Yeah. Awesome, dude.
31:07
I think that's it. Is there anything else that you
31:09
wanted to touch on that we didn't hit?
31:11
I think we we covered, we covered everything.
31:14
I think we maybe one thing would just be I think we are in the
31:17
spirit of real realizing our vision, vision to be the most
31:20
vital source for technologists and all those dimensions that I
31:23
mentioned. Whether that's creating
31:25
community in an age of AI, which is going to be very disruptive
31:28
and high change environment. Whether that's powering learning
31:31
for, you know, new developers who are looking to learn on the
31:34
side through our challenges. And you know, we'll do all sorts
31:36
of other things like hackathons potentially in the future, plus,
31:40
you know, for them to build their reputations and then for
31:42
that to unlock people's careers or in their productivity inside
31:45
companies through our Enterprise Products.
31:48
You know, that's what, you know, we're building this with the
31:49
community, you know, you know, we want to do this with them.
31:52
So I think, you know, my ask would be to your audience to
31:55
give us a lot of suggestions on, you know, how they would like
31:58
us, how they'd like us to evolve based on their own needs, right?
32:02
Because you know, we are moving from just being AQ and a site to
32:06
much more of a multi dimensional company.
32:08
And yes, Q&A has changed, the form factor has changed relative
32:12
to AI and you know, but for us to really work with the
32:15
community to build the future of Saco flow.
32:16
So we're excited about that.