Latest / a16z AI Policy Brief / Measuring AI: What Policymakers Can Learn from Benchmarks

Transcript

Measuring AI: What Policymakers Can Learn from Benchmarks · a16z AI Policy Brief

0:00
SPEAKER_02: Welcome back to the a16z AI Policy Brief.
0:03
SPEAKER_02: I'm Matt Peralta.
0:04
SPEAKER_02: AI benchmarks are becoming increasingly central to policy debates,
0:08
SPEAKER_02: but beyond the headlines, what does AI benchmarking mean in practice?
0:11
SPEAKER_02: What is measured?
0:12
SPEAKER_02: Who measures it?
0:13
SPEAKER_02: And how quickly can a benchmark adapt to the pace of model development?
0:16
SPEAKER_02: Today, I'm joined by two people who are building the tools and the partnerships
0:20
SPEAKER_02: to advance AI evaluation technology and its use in both the private sector and in
0:24
SPEAKER_02: the government.
0:25
SPEAKER_02: Ryan Krishnan,
0:26
SPEAKER_02: Co-Founder and CEO at VALS,
0:28
SPEAKER_02: and Glenn Parham, who is the company's head of public sector.
0:31
SPEAKER_02: We discuss what policymakers can learn from AI benchmarks.
0:34
SPEAKER_02: Drawing on their research and work evaluating the latest AI models,
0:37
SPEAKER_02: Ryan and Glenn explain why static tests quickly become obsolete,
0:40
SPEAKER_02: why credible evaluations depend on deep domain expertise,
0:43
SPEAKER_02: and why independent testing is essential to producing evidence
0:46
SPEAKER_02: that policymakers can trust.
0:48
SPEAKER_02: We also explore how a growing market of evaluators is moving the field
0:51
SPEAKER_02: beyond evaluation by vibes and toward more sophisticated,
0:55
SPEAKER_02: rigorous, data-driven measures of real-world performance.
0:59
SPEAKER_02: Here's our conversation.
1:00
SPEAKER_00: When we started the year, I think there was almost evaluations by vibes.
1:04
SPEAKER_00: And I think now in the coming months and by the end of the year,
1:06
SPEAKER_00: I think there'll be a lot more sophistication across the board.
1:09
SPEAKER_00: For enterprises,
1:10
SPEAKER_00: they're now,
1:11
SPEAKER_00: I think,
1:11
SPEAKER_00: transitioning more from token maxing to actually understanding where they're going
1:14
SPEAKER_00: to see ROI from adoption.
1:16
SPEAKER_00: And so benchmarks and evals will drive more precise business decisions for a lot
1:21
SPEAKER_00: of these people.
1:22
SPEAKER_00: And I think from the U.S.
1:22
SPEAKER_00: government side,
1:23
SPEAKER_00: it's clear that they're pushing really hard to gather more evidence.
1:27
SPEAKER_00: And that may go on to inform policy and legislation,
1:30
SPEAKER_00: but I think in the base case,
1:32
SPEAKER_00: it will form a more complete picture of what the capabilities and risks are
1:35
SPEAKER_00: of these models.
1:36
SPEAKER_01: The wrong answer here from a policy lens would be for the government to say,
1:39
SPEAKER_01: all right,
1:40
SPEAKER_01: this cybersecurity benchmark,
1:42
SPEAKER_01: this version of it,
1:42
SPEAKER_01: this is the thing that everyone rallies around from now until the end of time,
1:47
SPEAKER_01: because that'll be outdated in, what, six weeks, right?
1:52
SPEAKER_02: Ryan, co-founder and CEO of VALS, Glenn, head of public sector at VALS, welcome to the podcast.
1:57
SPEAKER_00: Thanks for having us, Matt.
1:58
SPEAKER_02: So, can we just start at the very beginning,
2:01
SPEAKER_02: which is, can you tell us a little bit about your company?
2:03
SPEAKER_00: Yeah, happy to start there.
2:05
SPEAKER_00: Yeah, so we started Val's with what was a very simple realization.
2:08
SPEAKER_00: It was just that these new AI models were coming to market quickly.
2:12
SPEAKER_00: It was clear there was going to be a competitive landscape of models.
2:15
SPEAKER_00: And in the same token,
2:17
SPEAKER_00: the benchmarks that people were using to describe what was newly capable
2:20
SPEAKER_00: the models were not really keeping up with the pace of the frontier.
2:23
SPEAKER_00: And so we felt there'd be value there being a new company that existed solely
2:26
SPEAKER_00: to solve this evaluation and benchmarking challenge.
2:29
SPEAKER_00: And that required them to build Really high quality,
2:33
SPEAKER_00: realistic,
2:34
SPEAKER_00: challenging benchmarks,
2:35
SPEAKER_00: and then run them independently to score model progress at the time.
2:38
SPEAKER_02: So when you say the benchmarks that were in the ecosystem
2:42
SPEAKER_02: to date weren't really keeping up with the field, what do you mean by that?
2:45
SPEAKER_00: Yeah, I mean,
2:46
SPEAKER_00: benchmarks are very well understood in the research landscape as a mechanism
2:50
SPEAKER_00: to drive model progress.
2:51
SPEAKER_00: And existing benchmarks,
2:52
SPEAKER_00: they looked a lot like exams you might give to students, like multiple choice exams.
2:58
SPEAKER_00: And that can get you a certain degree of signal on what models know and what they can do.
3:04
SPEAKER_00: But it's not quite like giving models an actual legal task or a coding task
3:08
SPEAKER_00: and seeing if they're able to do it at the equivalent standard of a human lawyer
3:11
SPEAKER_00: or programmer.
3:13
SPEAKER_02: So talk a little bit more then about,
3:15
SPEAKER_02: given that challenge, what the value is of Val specifically.
3:20
SPEAKER_00: Yeah, the value for us is that we're able to build these sets of benchmarks
3:23
SPEAKER_00: and our techniques around simulating what models can actually do with real tasks.
3:28
SPEAKER_00: So as if you're a human sitting in a chair doing a programming task or writing
3:34
SPEAKER_00: a financial brief,
3:35
SPEAKER_00: can the models substitute and do that work meeting the set of expectations that
3:40
SPEAKER_00: the humans have in practice?
3:41
SPEAKER_02: So what are you seeing in the information that you are producing?
3:47
SPEAKER_02: What are your benchmarks showing about the ecosystem that you're helping people
3:51
SPEAKER_02: to review and see more deeply?
3:53
SPEAKER_00: Yeah, the most obvious one is we're able to see the rate of model progress and
3:58
SPEAKER_00: the frontier shift with successive model releases.
4:01
SPEAKER_00: What's also been interesting to track is the differences in capabilities
4:04
SPEAKER_00: between closed source and open source models.
4:07
SPEAKER_00: Also looking at some more factors around cost and latency.
4:10
SPEAKER_00: It's no longer the case that people are willing to spend Mm-hmm.
4:14
SPEAKER_00: Mm-hmm.
4:28
SPEAKER_02: It sounds like the work of a sizable company,
4:30
SPEAKER_02: but you're still in, I think the relatively early days.
4:34
SPEAKER_02: So can you talk about what the company looks and feels like?
4:52
SPEAKER_00: It's quadrupling and maybe we'll double again by the end of the year.
4:55
SPEAKER_00: We'll see.
4:56
SPEAKER_00: So, yeah, I think that the mandate is pretty broad.
4:59
SPEAKER_00: You can imagine there is a really wide surface area of places people want to use AI models.
5:05
SPEAKER_00: The economy is very diverse with many sets And so there's kind of an unlimited set
5:10
SPEAKER_00: of things that we can evaluate the models for.
5:13
SPEAKER_00: On the flip side of this,
5:14
SPEAKER_00: what you're seeing is there's actually also an exploding set
5:17
SPEAKER_00: of intelligence options.
5:19
SPEAKER_00: You have more foundation model labs and neo labs getting funded.
5:22
SPEAKER_00: And so with all that,
5:27
SPEAKER_00: what you're seeing is that it's actually more challenging than ever to keep up
5:38
SPEAKER_00: with this dynamic landscape of models and products that are coming out.
5:42
SPEAKER_00: And so we took a bet pretty early on to build our benchmarks in a way
5:45
SPEAKER_00: that we could reproducibly run them over time without any human intervention.
5:50
SPEAKER_00: And that's allowed us to actually keep up with this really rapid clip
5:53
SPEAKER_00: of model release and product releases happening.
5:55
SPEAKER_02: I've been surprised just in the conversations that we've had just in the last couple
5:58
SPEAKER_02: of months, I think.
5:59
SPEAKER_02: I think we met each other maybe a couple of months ago.
6:03
SPEAKER_02: It seems like in every conversation we have,
6:05
SPEAKER_02: you're talking about some new thing that you're able to evaluate,
6:08
SPEAKER_02: which just seems extraordinary for a company that's at the size that you're at.
6:11
SPEAKER_00: Yeah, I think we've invested a lot of time in building out this infrastructure
6:14
SPEAKER_00: that allows us to run the valuations at scale in a short period of time.
6:18
SPEAKER_00: In some instances, we'll get access to models six hours before they go live.
6:23
SPEAKER_00: And so that means we need to run tens of billions of tokens and really maximize
6:27
SPEAKER_00: the rate limits to pull out signal out of the models in a very short period of time.
6:30
SPEAKER_00: We never want to be slowing down model release timelines.
6:34
SPEAKER_00: We want to be just keeping up with the pace of the frontier.
6:37
SPEAKER_00: And I think being a private company allows us to invest in
6:40
SPEAKER_00: the underlying technology rather than be an inhibitor to progress.
6:45
SPEAKER_02: And is that really the timeline that you're capable of doing these evaluations on,
6:48
SPEAKER_02: like six hours from getting access to a model?
6:51
SPEAKER_00: Yeah.
6:51
SPEAKER_00: I mean, there was a time when every time we had an early access model,
6:55
SPEAKER_00: someone was going to pull an all-nighter.
6:58
SPEAKER_00: Was that someone new?
6:59
SPEAKER_00: Yeah.
7:01
SPEAKER_00: Me or Langston, the CTO, we had, you know, Yerba Mate stash that we'd go through.
7:06
SPEAKER_00: And so now the infrastructure's stabilized,
7:08
SPEAKER_00: we have a larger team,
7:09
SPEAKER_00: we're able to do this rapid kind of valuation in a much more scalable way.
7:14
SPEAKER_00: But yeah, ideally we'd get 48 hours early access.
7:16
SPEAKER_00: Sometimes it can be as close as six hours.
7:18
SPEAKER_02: And you've just very excitingly announced your Series A. Congratulations.
7:24
SPEAKER_02: I'm interested in with that, with the new investment coming in, what is that enabling you to do?
7:32
SPEAKER_02: What are you excited about being able to do with your increased capacity?
7:35
SPEAKER_00: Yeah, I mean, there's some tactical things we're announcing with the Series A.
7:39
SPEAKER_00: We have this new enterprise product, which anyone can sign up and use, called Valsmith.
7:44
SPEAKER_00: We're also doing complete redesign of our site and an update to the Vals Index,
7:49
SPEAKER_00: which aggregates our benchmarks.
7:51
SPEAKER_00: And so that's shifting from answering the question,
7:54
SPEAKER_00: can models answer this question, to can they actually do this task expected of them?
7:59
SPEAKER_00: So that's Val's Index 2.0.
8:00
SPEAKER_00: But now it's also allowed us to invest more heavily in some of the deeper questions
8:05
SPEAKER_00: and model evaluation, which folks in D.C. and people in Silicon Valley want to know.
8:10
SPEAKER_00: So in particular,
8:11
SPEAKER_00: I'll call out we're building a benchmark in cybersecurity,
8:14
SPEAKER_00: which is, you know, I think a really topical conversation area today.
8:20
SPEAKER_00: Self-improvement benchmarks are seeing which models have a propensity
8:23
SPEAKER_00: to actually create the next version of themselves without human intervention.
8:28
SPEAKER_00: Mental health benchmarks are seeing the impact of these models on people under the age of 18.
8:33
SPEAKER_00: And then also doing some work around a military benchmark, law of armed conflict.
8:37
SPEAKER_00: So these are kind of early pieces of work that we're announcing.
8:40
SPEAKER_00: There's also some experiments around distillation that we're releasing on our Ballast.ai site.
8:45
SPEAKER_00: But evolving very quickly, investing more.
8:48
SPEAKER_02: So can we talk through one of those in a little bit more detail,
8:51
SPEAKER_02: like whichever one you think kind of showcases the work that that you're able
8:54
SPEAKER_02: to do most effectively,
8:57
SPEAKER_02: like a cybersecurity benchmark,
8:58
SPEAKER_02: for instance,
8:59
SPEAKER_02: or another,
8:59
SPEAKER_02: if you want to choose another,
9:00
SPEAKER_02: what what are you actually evaluating in your benchmark?
9:04
SPEAKER_00: Yeah, so we released an initial benchmark called CyberBench.
9:07
SPEAKER_00: It builds in a lot of the methodology that people are using around a benchmark called CyberGym.
9:12
SPEAKER_00: And so we've collected a set of open source vulnerabilities and evaluated models
9:17
SPEAKER_00: on both their offensive and defensive capabilities.
9:21
SPEAKER_00: And then on the defensive side,
9:26
SPEAKER_00: if they're given a repository and a vulnerability,
9:28
SPEAKER_00: can they actually patch it to resolve that,
9:31
SPEAKER_00: to maintain the functionality while removing the vulnerability?
9:34
SPEAKER_00: Mm-hmm.
9:34
SPEAKER_00: And so that's given us an interesting picture to see
9:36
SPEAKER_00: whether models have more offensive or defensive capabilities.
9:39
SPEAKER_00: We are, depending on the timing of this release,
9:43
SPEAKER_00: going to have a new benchmark called SREbench,
9:45
SPEAKER_00: where we're seeing whether models can actually decompile an executable binary.
9:49
SPEAKER_00: And it's kind of a primitive in cybersecurity to see
9:52
SPEAKER_00: if you can expose vulnerabilities in the source code when you don't have access
9:55
SPEAKER_00: to it.
9:56
SPEAKER_00: And then there's some new work we'll have shortly after in the cyber ranges concept.
10:01
SPEAKER_00: So seeing the offensive capabilities when we give models a realistic And
10:04
SPEAKER_00: so can you talk a little bit about what those findings are?
10:24
SPEAKER_00: Yeah, I can share some preliminary sense.
10:27
SPEAKER_00: I think on the whole,
10:28
SPEAKER_00: it's very clear that the cyber capabilities
10:30
SPEAKER_00: of these models is advancing incredibly rapidly.
10:34
SPEAKER_00: I think there is some positive signal that the guardrails
10:37
SPEAKER_00: that foundation model labs have set up is I think that approach can work
10:40
SPEAKER_00: if done correctly.
10:41
SPEAKER_00: And if we're able to submit these refusals to maligned actors
10:43
SPEAKER_00: an appropriate degree,
10:44
SPEAKER_00: then we will be able to effectively moderate the defensive
10:47
SPEAKER_00: and offensive capabilities of the models.
11:03
SPEAKER_02: Some people are more concerned about cyber capabilities of proprietary models.
11:06
SPEAKER_02: Other people are more concerned about cyber capabilities of open source models.
11:10
SPEAKER_02: What is the role of the evaluation ecosystem in evaluating open versus closed?
11:15
SPEAKER_00: I think it's essential for evaluation to happen in both groups.
11:19
SPEAKER_00: I think to selectively evaluate closed source models
11:22
SPEAKER_00: because there's some domestic company behind them misses a significant risk vector
11:28
SPEAKER_00: in the open source models that are released.
11:31
SPEAKER_00: And so we can kind of trivially evaluate both open and closed source models and to do that.
11:36
SPEAKER_02: And is there a difference in the challenge of doing the evaluation work
11:40
SPEAKER_02: with open versus closed, given that open is by definition open?
11:43
SPEAKER_02: And I assume that means it's easier to access.
11:45
SPEAKER_02: How does accessing proprietary compare?
11:49
SPEAKER_00: There are sometimes challenges around getting the direct model versus
11:54
SPEAKER_00: a guardrailed version of the models with the closed source providers.
11:57
SPEAKER_00: With an open source model, we can host it ourselves.
11:59
SPEAKER_00: We can hit the weights directly and see the generations.
12:03
SPEAKER_00: And so there's an irrefutable way that we can evaluate it.
12:06
SPEAKER_00: Thankfully,
12:07
SPEAKER_00: we've found good partnership in the closed source providers and
12:10
SPEAKER_00: so haven't had any issues in evaluating the cyber capabilities of the models.
12:14
SPEAKER_02: Glenn, let's bring you in here if that works.
12:17
SPEAKER_02: You're the head of public sector.
12:19
SPEAKER_02: Your job entails a lot of conversations with people in Washington,
12:24
SPEAKER_02: D.C. You're talking to policymakers.
12:27
SPEAKER_02: You're talking to government officials and agencies.
12:29
SPEAKER_02: What are the things that you're hearing in those conversations?
12:32
SPEAKER_01: Well, I mean, it's incredible.
12:33
SPEAKER_01: I think over the past couple of years,
12:35
SPEAKER_01: my time in DC,
12:36
SPEAKER_01: I didn't see that demand signal from government to care about AI evals.
12:40
SPEAKER_01: But now we are starting to see that, especially in domains like cybersecurity.
12:44
SPEAKER_01: So I would say at this moment, cybersecurity is the talk of the town.
12:48
SPEAKER_01: There's a lot of indexing on that.
12:50
SPEAKER_01: So you'll see kind of the traditional cybersecurity agencies like CISA
12:53
SPEAKER_01: or NSA take great interest in understanding model capability,
12:58
SPEAKER_01: not just for their own sanity and their own sake,
13:01
SPEAKER_01: But they work with the private sector, right?
13:02
SPEAKER_01: And they want to communicate these potential vulnerabilities to the people
13:08
SPEAKER_01: who manage critical infrastructure, financial institutions, etc.
13:11
SPEAKER_01: So for a third party like Val's to come in and actually provide
13:15
SPEAKER_01: that independent analysis is really huge.
13:19
SPEAKER_01: And on top of cybersecurity, Ryan already touched on this.
13:22
SPEAKER_01: One of the things,
13:22
SPEAKER_01: especially in Congress right now,
13:24
SPEAKER_01: that we're seeing a lot of discourse around is like AI child safety.
13:28
SPEAKER_01: I would say that A16z.com
13:45
SPEAKER_01: I'm curious.
14:02
SPEAKER_02: You said role of an independent evaluator.
14:04
SPEAKER_02: You guys are independent in a couple of different ways.
14:07
SPEAKER_02: One is you are not a government benchmark tool, at least not currently.
14:11
SPEAKER_02: Maybe the government would use you to do benchmarking at different points in
14:14
SPEAKER_02: the future, but you are not like the official government benchmark tool.
14:18
SPEAKER_02: And you are also not funded by the organizations that are getting benchmarked.
14:22
SPEAKER_02: And now in the ecosystem,
14:23
SPEAKER_02: there's a question about to what extent We should enable people to or permit people
14:28
SPEAKER_02: to grade their own homework.
14:30
SPEAKER_02: Do we want independence in various different ways?
14:33
SPEAKER_02: I'm wondering if you can talk about the role
14:35
SPEAKER_02: that you see Val's potentially providing in both of those forms of independence.
14:38
SPEAKER_02: Like we could have lots of different relationships between a benchmarking ecosystem
14:43
SPEAKER_02: and the government.
14:44
SPEAKER_02: The government could develop its own model.
14:45
SPEAKER_02: It could procure a third party like you guys.
14:48
SPEAKER_02: It could just point to a third party ecosystem where there are a lot of competing benchmarks.
14:52
SPEAKER_02: So I'm curious about that question and then also about
14:58
SPEAKER_01: Yeah, I mean,
14:59
SPEAKER_01: from my perspective,
15:00
SPEAKER_01: there's this concept of like these IVOs,
15:02
SPEAKER_01: these independent verification organizations that are getting a lot of traction
15:06
SPEAKER_01: and potential legislation drafts and just kind of in the policy circles in DC,
15:12
SPEAKER_01: which would kind of serve as those third party independent organizations that
15:16
SPEAKER_01: the government's able to tap to either assess model capabilities
15:20
SPEAKER_01: or different AI systems.
15:22
SPEAKER_01: And I find that analogous to These are the third party assessment organizations that
15:28
SPEAKER_01: the government has been using for over 15 years.
15:31
SPEAKER_01: The idea that the government accredits an organization to be an assessor,
15:35
SPEAKER_01: but it's a private entity and that company is able to go in
15:39
SPEAKER_01: a whole marketplace forms around that.
15:42
SPEAKER_01: So really tapping in the best of the private sector to go compete.
15:46
SPEAKER_01: So I think VALS, you know, for us, like we're going to see how that language evolves.
15:51
SPEAKER_01: It's still,
15:52
SPEAKER_01: as you know,
15:53
SPEAKER_01: like being formulated in specifics,
15:55
SPEAKER_01: but I think we see ourselves as at least partially a contributor.
15:59
SPEAKER_01: Maybe it's as an IVO or whatever ends up,
16:01
SPEAKER_01: you know, emerging as like that third party assessment organization.
16:06
SPEAKER_02: And how does that model look from your standpoint versus VALS?
16:10
SPEAKER_02: Government running its own benchmark and not making the process open
16:14
SPEAKER_02: to third-party private sector evaluators.
16:17
SPEAKER_01: I mean, so from my experience, I was in the Pentagon leading generative AI.
16:21
SPEAKER_01: We tried to do benchmarks and it was a lot of great efforts,
16:25
SPEAKER_01: but you're just not able as a government employee to move and be as agile to meet
16:31
SPEAKER_01: the moment of how quickly AI is changing, how many new models are coming online.
16:36
SPEAKER_01: And so the benefits of like tapping vendors,
16:39
SPEAKER_01: private companies like Vals or other organizations is that we have that agility,
16:44
SPEAKER_01: right?
16:44
SPEAKER_01: We're able to pull an all-nighter or, you know, do whatever it takes to create benchmarks.
16:49
SPEAKER_01: We partner with the experts, don't get it wrong.
16:51
SPEAKER_01: Like we need to make sure we're tapping the right people who know that domain,
16:56
SPEAKER_01: whether it's in the military or cybersecurity, whatever it may be.
17:00
SPEAKER_01: But, you know,
17:01
SPEAKER_01: I think it's important that an organization that is creating benchmarks is able
17:05
SPEAKER_01: to meet the pace at which models are coming out and capabilities are developing.
17:10
SPEAKER_02: What about on the other prong of independence?
17:12
SPEAKER_02: So I think even just in the last few weeks,
17:15
SPEAKER_02: I think I've seen more and more attention on the question of independence.
17:19
SPEAKER_02: Model developers funding benchmark organizations,
17:22
SPEAKER_02: and it could either be their own in-house benchmark,
17:25
SPEAKER_02: it could be that they're funding external organizations.
17:28
SPEAKER_02: I saw one recent proposal that would permit that practice to continue but would cap
17:33
SPEAKER_02: the funding that a developer could provide.
17:35
SPEAKER_02: I think it's essential.
17:43
SPEAKER_00: It is not a surprise to people who have done work in machinery that there's
17:48
SPEAKER_00: an incredible importance to having a held out test set,
17:51
SPEAKER_00: which is actually run through an evaluation administered by a third party.
17:55
SPEAKER_00: I think I've been surprised to see, but...
17:59
SPEAKER_00: Maybe shouldn't have been that there have been a bunch of instances of companies
18:02
SPEAKER_00: who have just handed over test sets to financial model labs for them
18:05
SPEAKER_00: to run themselves.
18:07
SPEAKER_00: And it should be no surprise that those self-reported results are a weaker signal
18:11
SPEAKER_00: on model performance over time.
18:13
SPEAKER_02: And can you say for the non-technical folks out there,
18:16
SPEAKER_02: including me, what is handing over a test set?
18:19
SPEAKER_00: It's akin to if you're taking the SAT exam and College Board decides to send home
18:22
SPEAKER_00: the test packet and you trust that student to self-proctor their exam
18:26
SPEAKER_00: and pass back their score.
18:29
SPEAKER_00: The right way to do it and the way that our admission system trusts is that you go
18:33
SPEAKER_00: to a testing facility,
18:34
SPEAKER_00: you have it proctored,
18:35
SPEAKER_00: and the tests are sent to College Board to do the scoring process.
18:39
SPEAKER_00: And so I think there's a very clear parallel where
18:41
SPEAKER_00: the foundation model labs will be most accurately evaluated.
18:45
SPEAKER_00: And if there's a company that's actually running the third party test themselves
18:49
SPEAKER_00: and developing those tests for everyone's sake.
18:54
SPEAKER_00: A lot of these labs are very rational as well.
18:56
SPEAKER_00: I think they understand the need for having more third-party testers.
19:00
SPEAKER_00: Actually,
19:00
SPEAKER_00: Demis wrote an article a couple weeks ago,
19:02
SPEAKER_00: which has gotten a lot of attention and buy-in from folks
19:05
SPEAKER_00: across other foundation model labs.
19:08
SPEAKER_00: Jack Clark went One of the Anthropa co-founders.
19:10
SPEAKER_00: He wrote this paper on market-based regulators.
19:13
SPEAKER_00: And so he presented this world in which self-regulation,
19:16
SPEAKER_00: as kind of is dominant today from Foundation Model Labs,
19:19
SPEAKER_00: isn't going to hold up long-term.
19:21
SPEAKER_00: And you see this apples to oranges comparison And so there's a good precedent
19:24
SPEAKER_00: in market-based regulators who have a mandate from the government,
19:28
SPEAKER_00: are incentive-aligned with the population itself,
19:30
SPEAKER_00: but can choose to invest very heavily in
19:54
SPEAKER_02: I'm curious how the two of you see this market evolving generally.
20:00
SPEAKER_02: We have been thinking about benchmark concepts for,
20:02
SPEAKER_02: I think, probably aggressively for six, eight months, something like that.
20:06
SPEAKER_02: And at the beginning of that period,
20:07
SPEAKER_02: which is obviously not very long ago,
20:09
SPEAKER_02: when we would say, um,
20:12
SPEAKER_02: One approach to these issues would be to have companies be required
20:15
SPEAKER_02: to publish benchmarks on some of these key issue areas and then stand
20:19
SPEAKER_02: behind their work.
20:19
SPEAKER_02: And so you could publish a poor score,
20:21
SPEAKER_02: you could publish a low cybersecurity benchmark,
20:23
SPEAKER_02: but that's going to probably affect your competitive position in the market.
20:27
SPEAKER_02: And if you're looking to compete aggressively,
20:31
SPEAKER_02: either for consumers generally or in what Glenn's focused on,
20:35
SPEAKER_02: like for government procurement,
20:37
SPEAKER_02: It's going to be challenging for you in the ecosystem if your performance is poor.
20:41
SPEAKER_02: And when we had some of those initial conversations,
20:42
SPEAKER_02: people would say like, oh, yeah, but you can't evaluate.
20:45
SPEAKER_02: There's no cybersecurity benchmark out there.
20:47
SPEAKER_02: It seems like that has rapidly shifted from like, you know, even a fairly short time ago.
20:53
SPEAKER_02: So how do you guys how are you guys seeing the development of this ecosystem and
20:58
SPEAKER_02: the level of robustness of the benchmark ecosystem relative to the scale of
21:04
SPEAKER_02: the potential problems?
21:05
SPEAKER_01: I think that there is going to be a great incentive for a lot of companies
21:12
SPEAKER_01: to evaluate their systems,
21:13
SPEAKER_01: their models,
21:14
SPEAKER_01: and publish their benchmarks and the results pretty publicly,
21:18
SPEAKER_01: whether they're trying to sell to the government or the private sector.
21:20
SPEAKER_01: So I think we're creating that incentive structure to be able to prove
21:23
SPEAKER_01: that you empirically are meeting whatever requirements or threshold
21:25
SPEAKER_01: that maybe your customer is requiring.
21:40
SPEAKER_00: I think when you look at cyber,
21:42
SPEAKER_00: there's some really high quality work that's been done by the UK AI Institute, UKAC,
21:48
SPEAKER_00: where they've constructed some cyber ranges and been able
21:51
SPEAKER_00: to very keenly track model progress in cyber for the last year.
21:55
SPEAKER_00: Unfortunately,
21:56
SPEAKER_00: now those ranges are saturated and so we're not able to delineate model performance
22:01
SPEAKER_00: in the last few months.
22:03
SPEAKER_00: And so I think what we've seen now is a pretty fragmented landscape
22:07
SPEAKER_00: of cyber valuations
22:08
SPEAKER_00: where there are really high quality valuations developed There are some non-profits
22:15
SPEAKER_00: and startups looking to make smaller environments that they can run evaluations over
22:19
SPEAKER_00: and report back findings to the labs.
22:21
SPEAKER_00: And I think naturally there are some folks in the U.S.
22:23
SPEAKER_00: government who are trying to pull these various signals But I think
22:27
SPEAKER_00: the fragmentation that I described also has some natural consequences in
22:31
SPEAKER_00: that we're not able to see consistent evaluation across a lot
22:35
SPEAKER_00: of these model providers.
22:37
SPEAKER_00: When you look at this issue of containment, for instance, we have infrastructure Mm-hmm.
22:41
SPEAKER_00: Mm-hmm.
22:41
SPEAKER_00: Mm-hmm.
23:07
SPEAKER_01: Another trend that I see kind of evolving with the marketplace of evaluations
23:12
SPEAKER_01: and benchmarks is just a diversification of the domains itself.
23:16
SPEAKER_01: I think as different regulations,
23:19
SPEAKER_01: executive orders,
23:20
SPEAKER_01: laws come online in America,
23:22
SPEAKER_01: actually all over the world, we're starting to see things come online.
23:25
SPEAKER_01: They're going to call for companies to be able to prove how their models perform
23:30
SPEAKER_01: on not just,
23:31
SPEAKER_01: you know,
23:31
SPEAKER_01: national security implications and cybersecurity,
23:34
SPEAKER_01: but also like in California,
23:36
SPEAKER_01: things like discrimination,
23:38
SPEAKER_01: you know, kind of more civil rights or civic minded benchmarks and evaluations.
23:43
SPEAKER_01: So.
23:44
SPEAKER_01: It is fascinating to understand the extent to which
24:07
SPEAKER_02: The work that you're doing pushes back on this real concern
24:11
SPEAKER_02: that I think people have,
24:12
SPEAKER_02: policymakers have,
24:13
SPEAKER_02: even in some of my conversations within our firm,
24:15
SPEAKER_02: some people,
24:17
SPEAKER_02: even though we're AI optimists,
24:18
SPEAKER_02: we have concerns about, which is there are these risks that we can't measure.
24:22
SPEAKER_02: And it I assume that there are things that are hard to measure,
24:25
SPEAKER_02: but it wouldn't have been intuitive to me before I started having conversations
24:31
SPEAKER_02: with you guys that you would be able to measure performance
24:34
SPEAKER_02: in areas like mental health or child safety or cybersecurity.
24:39
SPEAKER_02: And so I guess I'm curious about How the conversations with policymakers would go
24:46
SPEAKER_02: if they say, we want you to measure, look at this other thing.
24:48
SPEAKER_02: I don't know what the next thing coming down the pike would be,
24:50
SPEAKER_02: but like, look at, we're concerned about this thing.
24:54
SPEAKER_02: I'm curious about how responsive you feel like you can be to whatever the concern of
24:59
SPEAKER_02: the policymaker public might be.
25:01
SPEAKER_00: Yeah, I wouldn't say the work we're doing is easy.
25:04
SPEAKER_00: And I think if anyone has done research or seen this landscape,
25:09
SPEAKER_00: it's not easy to build a really high quality evaluation.
25:12
SPEAKER_00: But I think what you've seen is that in research,
25:15
SPEAKER_00: evaluation and generation have been two pillars that have been very symbiotic.
25:19
SPEAKER_00: And in order to do better at one, you have to do better at the other.
25:22
SPEAKER_00: And so over the last,
25:24
SPEAKER_00: I'd say,
25:24
SPEAKER_00: three years,
25:24
SPEAKER_00: there's the There's been a disproportionate investment in generation
25:28
SPEAKER_00: the former foundation model labs and comparatively little in evaluation happening
25:33
SPEAKER_00: in academia or nonprofits.
25:35
SPEAKER_00: And so I think our existence as a company and the places where we've chosen
25:39
SPEAKER_00: to invest have shown that actually It is possible to measure and pull out of signal
25:43
SPEAKER_00: in these models in a way that maybe policymakers are not familiar with.
25:48
SPEAKER_00: And I think a continued investment on our part and in other players
25:51
SPEAKER_00: to pull out more signal will actually reveal we can rival a lot of
25:54
SPEAKER_00: the work happening within Foundation Model Labs.
25:56
SPEAKER_02: I don't want to be too naive about it.
25:58
SPEAKER_02: I assume there are questions that a policymaker might bring to you
26:01
SPEAKER_02: where you would say, we really can't measure that.
26:03
SPEAKER_02: That's really hard.
26:05
SPEAKER_02: Are there things that are in that domain to you where you say, like,
26:08
SPEAKER_02: we understand the concerns that you have,
26:09
SPEAKER_02: but really it's very difficult to develop an evaluation that's responsive?
26:14
SPEAKER_00: I think that as a broad category,
26:16
SPEAKER_00: what's really hard to measure is things that we don't have a good verifier for.
26:22
SPEAKER_00: So we don't have a good sense of how do we grade whether or not a response is good
26:26
SPEAKER_00: or bad or risky or not risky.
26:29
SPEAKER_00: And so, for instance, You can look to biosecurity.
26:31
SPEAKER_00: For instance,
26:32
SPEAKER_00: if we need to run a wet lab experiment to see whether a model is producing a protein
26:37
SPEAKER_00: that could be dangerous or not dangerous,
26:39
SPEAKER_00: that's going to be very challenging for us to build an evaluation around.
26:42
SPEAKER_00: Or if there are subjective elements where it's hard to uniformly say that
26:47
SPEAKER_00: a model is doing a good or bad job, that's harder for us to do in practice.
26:50
SPEAKER_00: But I would say it's hard and not impossible.
26:54
SPEAKER_02: Glenn, I'm curious about your experience with that,
26:56
SPEAKER_02: because I assume a lot of your questions with government officials end up
27:00
SPEAKER_02: with people saying, what about this?
27:02
SPEAKER_02: Can you do this?
27:02
SPEAKER_02: And what about that?
27:03
SPEAKER_02: Or, you know,
27:04
SPEAKER_02: you say you're able to develop an evaluation that does this,
27:06
SPEAKER_02: but can we really trust it?
27:08
SPEAKER_01: What are those conversations like for you?
27:10
SPEAKER_01: Really the talk of the town right now,
27:11
SPEAKER_01: you're seeing it come out of the White House,
27:13
SPEAKER_01: out of Congress,
27:14
SPEAKER_01: is distillation,
27:15
SPEAKER_01: allegations of distillation,
27:17
SPEAKER_01: you know, Chinese labs engaging in distillation from U.S. labs.
27:24
SPEAKER_01: I would say that is likely the number one question I get.
27:28
SPEAKER_01: That is...
27:30
SPEAKER_01: Difficult,
27:30
SPEAKER_01: because from a research perspective,
27:32
SPEAKER_01: and Ryan and the researchers can chime in on more on this,
27:35
SPEAKER_01: you know, we can't necessarily prove, causally prove that distillation occurred.
27:41
SPEAKER_01: We can surface signals of it and provide evidence of that.
27:45
SPEAKER_01: But, you know, the government wants to know, you know, definitively, like, do you have proof?
27:48
SPEAKER_01: Yes or no?
27:49
SPEAKER_01: Like, is this evidence that distillation occurred?
27:52
SPEAKER_01: So I would say,
27:53
SPEAKER_01: you know,
27:53
SPEAKER_01: with that,
27:54
SPEAKER_01: the best we can do is say, hey,
27:56
SPEAKER_01: we're going to give you some signals of this,
27:58
SPEAKER_01: what you do with that and how you fuse that
27:59
SPEAKER_01: with other intelligence you may have already about potential distillation.
28:04
SPEAKER_01: That's on you.
28:05
SPEAKER_01: But we're just a resource for them to rely on and source from.
28:09
SPEAKER_02: I think there are a few different visions of how
28:12
SPEAKER_02: a benchmarking ecosystem could play out in practice.
28:14
SPEAKER_02: I think the skeptics think it's going to be a race to the bottom,
28:18
SPEAKER_02: that developers will be looking for,
28:20
SPEAKER_02: to use your SAT example,
28:21
SPEAKER_02: like easier and easier SATs and more access to more information about them so that
28:26
SPEAKER_02: the scores become less and less meaningful data about it.
28:30
SPEAKER_02: A16z.
28:53
SPEAKER_02: In those two visions,
28:54
SPEAKER_02: race to the bottom,
28:54
SPEAKER_02: race to the top,
28:55
SPEAKER_02: what do you think that the ecosystem,
28:59
SPEAKER_02: by which I mean both on the policy side and then on the industry side,
29:03
SPEAKER_02: what does it need to do to get us more likely on the race to the top track?
29:08
SPEAKER_00: Yeah, I think, I mean, I can speak to what we are trying to do at VALS.
29:11
SPEAKER_00: I think that we're still in a space in which not many people are doing work like we are.
29:16
SPEAKER_00: And so we should just strive to do the best at it.
29:19
SPEAKER_00: And I think that really comes in the form of transparency.
29:22
SPEAKER_00: Can we And so the more we can strive to do that,
29:24
SPEAKER_00: I think policymakers,
29:25
SPEAKER_00: the industry themselves will trust our approach in the way that we've done things.
29:38
SPEAKER_00: I think the other part that you see this also play out in places like finance
29:42
SPEAKER_00: and other industries that have market-based regulators,
29:45
SPEAKER_00: there needs to be some process by which they get audited over time,
29:49
SPEAKER_00: either by the government or by folks in industry.
29:52
SPEAKER_00: And so I would invite other folks to also critique our methods and also work with us
29:57
SPEAKER_00: to make sure we are held to the highest possible standards.
30:00
SPEAKER_01: Yeah,
30:00
SPEAKER_01: I think the wrong answer here from a policy lens would be for the government to say,
30:05
SPEAKER_01: all right,
30:06
SPEAKER_01: this cybersecurity benchmark,
30:07
SPEAKER_01: this version of it,
30:08
SPEAKER_01: this is the thing that everyone rallies around from now until the end of time.
30:15
SPEAKER_01: That'll be outdated in, what, six weeks, right?
30:17
SPEAKER_01: So I think it's important that the government recognize just how dynamic
30:22
SPEAKER_01: and fast-paced everything is and do what they've done with other things.
30:27
SPEAKER_01: I keep going back to cybersecurity with NIST.
30:30
SPEAKER_01: Laying out the framework,
30:31
SPEAKER_01: the risk management framework that so many industries rely on for cybersecurity.
30:37
SPEAKER_01: They're not overly prescriptive in saying you must use XYZ services and whatnot.
30:43
SPEAKER_01: It's a more abstract, dynamic framework that people are able to rely on.
30:48
SPEAKER_01: And so similarly,
30:49
SPEAKER_01: when it comes to AI benchmarks,
30:51
SPEAKER_01: I think the government should definitely defer and tap
30:55
SPEAKER_01: in private partners like ourselves and others to And not be overly prescriptive
31:00
SPEAKER_01: because it'll just become outdated overnight.
31:04
SPEAKER_02: So you mentioned transparency.
31:05
SPEAKER_02: And that's,
31:06
SPEAKER_02: I think,
31:06
SPEAKER_02: also something that's part of the one of the talk of the town issues,
31:10
SPEAKER_02: as Glenn is describing.
31:11
SPEAKER_02: There are proposals on the table now for government benchmarks
31:14
SPEAKER_02: to not only be choose one particular benchmark,
31:17
SPEAKER_02: but also to not be public about the benchmark, about the methodology underlying it.
31:23
SPEAKER_02: How are you guys transparent about your methodology?
31:26
SPEAKER_00: Yeah, we have two main vehicles to incentivize transparency.
31:30
SPEAKER_00: The first is we have blog post style information on every one of our benchmarks,
31:37
SPEAKER_00: which helps more general audiences get a sense of what's being measured.
31:40
SPEAKER_00: And the second is we make sure We published a research paper with every benchmark
31:43
SPEAKER_00: to make sure we're digging the technical details of how we created it,
31:46
SPEAKER_00: how we make sure what we're measuring is actually important and high signal.
31:50
SPEAKER_00: And then relatedly, we open source a lot of our infrastructure and sample questions and rubrics.
31:56
SPEAKER_00: To make sure that people can audit.
31:58
SPEAKER_00: And if they have critiques, they can make suggestions or come back to us.
32:02
SPEAKER_00: So I think this helps a lot.
32:04
SPEAKER_00: I think the other thing just to also note is that we don't have any benchmarks
32:08
SPEAKER_00: that aren't listed publicly on Vals.ai.
32:11
SPEAKER_00: And so I think that helps people gain a lot of clarity
32:15
SPEAKER_00: that there aren't other mystery things that we're measuring or training data sets
32:20
SPEAKER_00: that we're supporting foundation model labs with.
32:22
SPEAKER_00: What you see is what you get with LSAT AI and the benchmarks that we support.
32:26
SPEAKER_02: Glenn, is the transparency that you provide,
32:29
SPEAKER_02: does that end up being a meaningful thing in your conversations
32:32
SPEAKER_02: with government agencies?
32:33
SPEAKER_01: Absolutely.
32:34
SPEAKER_01: I think because it really boils down to the ability for government
32:37
SPEAKER_01: or government partners to reproduce our results, right?
32:40
SPEAKER_01: They want to make sure that, hey,
32:42
SPEAKER_01: if we were to import your open source infrastructure and your benchmarks,
32:45
SPEAKER_01: we should get pretty much identical results.
32:47
SPEAKER_01: And that's something So there are lots of people out there who are skeptical about
32:50
SPEAKER_01: the merits of benchmarks.
32:52
SPEAKER_01: When you hear criticism from them,
32:54
SPEAKER_01: what are some of the common critiques that you hear and how do you respond to them?
33:15
SPEAKER_00: Yeah, I think the first one that I seemingly get a lot is this belief
33:20
SPEAKER_00: that we're not actually running the results that we host on valance.ai,
33:23
SPEAKER_00: that we're taking numbers that labs have run themselves and then mirror hosting them
33:28
SPEAKER_00: on our site.
33:29
SPEAKER_00: That's actually not the case.
33:31
SPEAKER_00: Every number you see on a benchmark on valance.ai,
33:34
SPEAKER_00: we've actually submitted the query to the model.
33:36
SPEAKER_00: So those are numbers that we have independently run and can stand behind.
33:43
SPEAKER_00: So it takes more work and effort,
33:44
SPEAKER_00: but it means that we have real signal on what the models can do.
33:51
SPEAKER_00: What I see a lot of is that intelligence is kind of this mystical substrate
33:55
SPEAKER_00: which applies uniformly across all domains and tasks at once.
34:00
SPEAKER_00: I think there are ways in which machine learning and AI techniques can generalize
34:05
SPEAKER_00: across many domains, But we still see that intelligence is jagged.
34:09
SPEAKER_00: And by that, I mean,
34:10
SPEAKER_00: it isn't necessarily obvious what the best model to use is in any given task
34:15
SPEAKER_00: or application.
34:16
SPEAKER_00: And that's even more the case when you consider other factors like cost,
34:19
SPEAKER_00: latency, or sovereignty of the model.
34:21
SPEAKER_00: And so this jaggedness,
34:22
SPEAKER_00: it means that it's actually very hard to predict what the right intelligence is
34:26
SPEAKER_00: to use in any given task or application.
34:29
SPEAKER_00: And that's borne out in the benchmarks we create on balance.ai.
34:31
SPEAKER_00: Mm-hmm.
34:32
SPEAKER_02: Glenn, how about you in your conversations in D.C.?
34:34
SPEAKER_02: What are some of the common criticisms that you hear?
34:36
SPEAKER_01: I just want to double tap on what Ryan just said.
34:39
SPEAKER_01: I've experienced that a lot going across the U.S.
34:41
SPEAKER_01: military where there was an expectation of, well,
34:45
SPEAKER_01: I've used this chatbot in,
34:46
SPEAKER_01: you know, Personally at home to help, you know, write a poem or something.
34:50
SPEAKER_01: Now I want to try to use it for air refuel optimization in the South China Sea.
34:55
SPEAKER_01: Why isn't it as good?
34:56
SPEAKER_01: And that kind of,
34:57
SPEAKER_01: I call like this domain transfer fallacy, like,
35:00
SPEAKER_01: no, don't assume that it's going to uniformly be good at all tasks and all domains.
35:06
SPEAKER_01: So, yeah, definitely agree with that.
35:08
SPEAKER_01: But more broadly,
35:11
SPEAKER_01: I've been shocked,
35:12
SPEAKER_01: to be honest,
35:13
SPEAKER_01: over the past couple of years,
35:15
SPEAKER_01: people just not believing in the utility or the need for evaluation or backsliding.
35:20
SPEAKER_01: Benchmarks of AI itself,
35:23
SPEAKER_01: because there's this idea and presumption that
35:25
SPEAKER_01: the models can always just improve themselves.
35:29
SPEAKER_01: And maybe at one time, at one point that that will be the case.
35:34
SPEAKER_01: But the models that we have access to and certainly the models that
35:37
SPEAKER_01: the government has access to today just are not should not be trusted
35:42
SPEAKER_01: to just magically improve themselves and correct for all mistakes.
35:46
SPEAKER_01: And so I think,
35:48
SPEAKER_01: you know,
35:48
SPEAKER_01: part of the campaign of us being in DC is convincing people,
35:53
SPEAKER_01: like, actually, yeah, these models aren't perfect.
35:57
SPEAKER_01: They may seem perfect.
35:58
SPEAKER_01: They may seem very intelligent.
36:00
SPEAKER_01: But just like all other software,
36:02
SPEAKER_01: all other hardware,
36:03
SPEAKER_01: when it comes to,
36:04
SPEAKER_01: you know,
36:05
SPEAKER_01: the whole industry of test and evaluation,
36:08
SPEAKER_01: we need to apply the same rigor when it comes to these AI systems.
36:13
SPEAKER_02: I think we met for the first time a couple of months ago
36:15
SPEAKER_02: and you were already talking about a strategy with government.
36:20
SPEAKER_02: It's bizarre because it seems like you have already accelerated so fast and been
36:24
SPEAKER_02: so engaged on the government side,
36:26
SPEAKER_02: but it really was an idea that you had not very long ago.
36:29
SPEAKER_02: So can you talk a little bit about what that was that was kind of germinating
36:33
SPEAKER_02: in your head about what this public sector part of your business might look like and
36:37
SPEAKER_02: then how you put it in place and how you went to bring Glenn on to lead that effort?
36:43
SPEAKER_00: Yeah, I mean,
36:43
SPEAKER_00: I think it was a place that I had speculated on when we started Vals,
36:47
SPEAKER_00: that this future might emerge when there needed
36:49
SPEAKER_00: to be third-party valuation companies like Vals.
36:52
SPEAKER_00: I think it wasn't quite sure how that would develop in practice.
36:57
SPEAKER_00: And so, yeah,
36:58
SPEAKER_00: I think we met maybe a month or two ago when we were starting to see conversations
37:02
SPEAKER_00: in DC emerge around what risk testing or pre-release testing might look like at
37:07
SPEAKER_00: the federal level.
37:08
SPEAKER_00: And so it seemed like the right time for us
37:09
SPEAKER_00: to take some expertise we'd built around model valuations to a new DC audience.
37:14
SPEAKER_00: And then thankfully, Glenn and I have been buddies for a while now.
37:17
SPEAKER_00: He had this great project, GovBench AI.
37:20
SPEAKER_00: And so it seemed natural for him to join VALS and lead this effort as our head of public sector.
37:24
SPEAKER_02: I couldn't believe when we had that conversation.
37:26
SPEAKER_02: I'm probably misremembering the dates a little bit,
37:28
SPEAKER_02: but it felt to me like we had that initial conversation on Thursday,
37:31
SPEAKER_02: and you were like, I'm thinking about bringing in this guy to lead this effort.
37:34
SPEAKER_02: And then I think maybe we exchanged emails on Monday, and you were like, Glenn's on board.
37:38
SPEAKER_01: Here's Glenn in Balzada.
37:42
SPEAKER_00: Yeah, it was a very fast clip.
37:43
SPEAKER_00: And I think for what it's worth,
37:44
SPEAKER_00: Glenn also saw a different version of this from DC,
37:47
SPEAKER_00: whereas I was seeing the vantage point for Miss F,
37:50
SPEAKER_00: if you want to share some view on that, Glenn.
37:52
SPEAKER_01: You know,
37:52
SPEAKER_01: my background,
37:53
SPEAKER_01: I was technically for Generative AI in the Pentagon for a couple of years.
37:56
SPEAKER_01: And so my job was going around looking at all these different vendor offerings,
38:00
SPEAKER_01: people trying to sell the military things with respect to Generative AI.
38:04
SPEAKER_01: And one of the main takeaways I had in my tenure was we just did not have good benchmarks.
38:10
SPEAKER_01: The benchmarks and results that these vendors would,
38:12
SPEAKER_01: you know,
38:13
SPEAKER_01: cough up and brag about were all,
38:15
SPEAKER_01: you know, self-attested, self-made, and just were not
38:22
SPEAKER_01: So in my time exiting and in the next phase,
38:27
SPEAKER_01: I created this nonprofit called GovBench that really wanted to just focus
38:32
SPEAKER_01: on creating government specific benchmarks.
38:35
SPEAKER_01: And so when I met up with Ryan and we talked last year,
38:38
SPEAKER_01: we just both were totally on the same page of like,
38:42
SPEAKER_01: we need to make sure we're creating AI benchmarks.
38:45
SPEAKER_01: Given the amount of money going into buying AI products spent by the government,
38:52
SPEAKER_01: we need to get these AI benchmarks, right?
38:55
SPEAKER_01: And so, yeah,
38:56
SPEAKER_01: just a couple of months ago,
38:57
SPEAKER_01: I came on board,
38:59
SPEAKER_01: I talked with Ryan and Just
39:09
SPEAKER_02: in the last couple of weeks,
39:11
SPEAKER_02: you guys announced an agreement with Casey, which is fantastic.
39:13
SPEAKER_02: They're arguably the lead safety evaluation organization,
39:17
SPEAKER_02: public or private in the United States.
39:19
SPEAKER_02: What do you think was so compelling to them about your offering?
39:24
SPEAKER_00: Yeah, I think first order,
39:26
SPEAKER_00: we have so regularly been able to keep up with the pace of model releases.
39:31
SPEAKER_00: And so we're able to turn around evaluations for them in a very, very short period of time.
39:36
SPEAKER_00: And so I think they respected that pace of response.
39:42
SPEAKER_00: I think the second is the places that we've chosen to spend time and invest in the methodology.
39:49
SPEAKER_00: So these conversations about cyber,
39:51
SPEAKER_00: but also future work around distillation,
39:54
SPEAKER_00: I think has been compelling for them as they look to gather more evidence
39:57
SPEAKER_00: and understanding of what's going on with these models.
39:59
SPEAKER_02: You guys are at such an exciting moment.
40:01
SPEAKER_02: I mean, I think you can feel it and hear it in how you talk.
40:05
SPEAKER_02: You have this tool that is so responsive to the moment, again,
40:09
SPEAKER_02: to policymaker concerns and also just concerns that people on the street,
40:12
SPEAKER_02: I think, have about AI.
40:13
SPEAKER_02: You've just announced your Series A.
40:16
SPEAKER_02: What are you excited about in the next few months?
40:19
SPEAKER_00: I think what's been interesting to see is we've been doing evaluations for some time.
40:22
SPEAKER_00: I was actually doing research in this before CHPT came out.
40:26
SPEAKER_00: And it feels like in a very short period of time,
40:28
SPEAKER_00: now this concept of evaluations benchmarks has gone mainstream.
40:32
SPEAKER_00: And when we started the year, I think there was almost evaluations by vibes.
40:35
SPEAKER_00: We see people just try a few questions and use that to qualitatively get a sense
40:40
SPEAKER_00: of what was newly capable.
40:42
SPEAKER_00: And I think now in the coming months and by the end of the year,
40:44
SPEAKER_00: I think there'll be a lot more sophistication across the board.
40:47
SPEAKER_00: I think in foundation model labs,
40:49
SPEAKER_00: they're already starting to rely more on third-party testing
40:52
SPEAKER_00: to actually convince people that their billions of dollars
40:55
SPEAKER_00: in investment is amounting to real value.
40:57
SPEAKER_00: For enterprises, they're now...
40:59
SPEAKER_00: I think transitioning more from token maxing to actually understanding
41:02
SPEAKER_00: where they're going to see ROI from adopting AI.
41:05
SPEAKER_00: And so benchmarks and evals will drive more precise business decisions for a lot
41:10
SPEAKER_00: of these people.
41:11
SPEAKER_00: And I think from the U.S.
41:11
SPEAKER_00: government side,
41:12
SPEAKER_00: it's clear that they're pushing really hard to gather more evidence.
41:17
SPEAKER_00: And that may go on to inform policy and legislation,
41:20
SPEAKER_00: but I think in the base case,
41:22
SPEAKER_00: It will form a more complete picture of what the capabilities and risks are
41:25
SPEAKER_00: of these models.
41:27
SPEAKER_00: And so we're excited to plug in for each one of these tracks and support them
41:30
SPEAKER_00: with what we've built.
41:31
SPEAKER_01: Yeah.
41:32
SPEAKER_01: And, you know,
41:33
SPEAKER_01: only being here two months,
41:34
SPEAKER_01: I've just seen the pace at which Vals moves, which has just been incredible.
41:38
SPEAKER_01: And then also just coincidentally and seeing how the,
41:41
SPEAKER_01: you know,
41:42
SPEAKER_01: the discourse in D.C.
41:43
SPEAKER_01: has evolved, you know, across the board, across Congress, across the White House.
41:48
SPEAKER_01: It's been, you know, really incredible and a great opportunity for Vals at this moment.
41:53
SPEAKER_01: I would say over the next couple of months,
41:55
SPEAKER_01: given the concerns of the government with respect
41:57
SPEAKER_01: to specifically cybersecurity concerns of U.S.
42:02
SPEAKER_01: versus Chinese models,
42:04
SPEAKER_01: I think it's great that VALS is in a position to help inform the U.S.
42:10
SPEAKER_01: government on some of the various threats that exist in real time
42:14
SPEAKER_01: as new models come out.
42:16
SPEAKER_01: We've already seen great demand from government where they're very grateful for
42:20
SPEAKER_01: the insights we're able to provide.
42:22
SPEAKER_01: And then I think,
42:23
SPEAKER_01: you know, AI,
42:24
SPEAKER_01: child safety and mental health,
42:26
SPEAKER_01: like I mentioned before, huge issue, bipartisan support.
42:31
SPEAKER_01: And I think over the next couple of months at the federal level and at a lot
42:34
SPEAKER_01: of different state levels,
42:36
SPEAKER_01: we're going to just see a lot of demand for the results that we're able to,
42:40
SPEAKER_01: that are going to come out from the different research and studies
42:43
SPEAKER_01: that we're conducting right now.
42:45
SPEAKER_01: And yeah, really excited to help inform policymakers.
42:49
SPEAKER_01: Data Beats Vibes.
42:50
SPEAKER_02: Ryan, Glenn, thanks so much for coming on the podcast.
42:53
SPEAKER_02: Thanks so much for having us, Matt.
42:54
SPEAKER_01: Awesome.
42:54
SPEAKER_01: Thanks.
42:55
SPEAKER_01: Appreciate you.