Latest / Newcomer Pod / Has AI Hit a Wall? A Debate from the Cerebral Valley AI Summit
Transcript
- 0:03Welcome to the Cerebral Valley Podcast.
- 0:06I'm Eric Newcomer. I'm here with Max Child and
- 0:08James Will Struman, my Cerebral Valley Co host.
- 0:10Hey, guys. Good morning, Eric.
- 0:13Hey, James. Hey, Eric, happy to be here.
- 0:15Excited to talk to you guys. Great.
- 0:18We survived the Cerebral Valley AI Summit.
- 0:21It was the week before Thanksgiving, we all checked out
- 0:24for Thanksgiving, and we're back here to give you the audio
- 0:27rundown of the most important things that happened.
- 0:30If you couldn't, couldn't be there.
- 0:32Max and James, why don't you start off favorite moment from
- 0:36the event off the top of your head?
- 0:39I think my favorite moment has to be Dario Amade from Anthropic
- 0:43at the end refuting the the case that we don't have to worry
- 0:47about AI because it's just math by pointing out that all neurons
- 0:53in everyone's brains are just math and therefore we should be
- 0:56afraid of the math because the math might produce a Hitler for
- 0:59example, who just had a bunch of math going in his brain.
- 1:01Or he said Hitler was just math, basically Hitler.
- 1:04Was just math, I know he's a one liner.
- 1:07Team, I don't think was happy he said it that way.
- 1:09I did as a moderator, try to be like, oh, that it makes sense to
- 1:13me what he's saying. I have also had this thought in
- 1:15my mind whenever people say something is just math, because
- 1:17you could say anything good or bad in human history is just
- 1:20math. Nuclear bombs are just math.
- 1:22Hitler is just math. Everything's just math
- 1:24fundamentally, because all physics is just math.
- 1:25So I thought that was a good point because I always thought
- 1:28that's been a very dumb take, that we shouldn't be scared of
- 1:30AI because it's just math. James Highlight.
- 1:33Sure. I mean, the first talk of the
- 1:35day I thought was great. You interviewed Alexander Wang
- 1:38from scale. I thought he was like, you know,
- 1:41letting us in a little bit on the the wall discussion in AI
- 1:46foundation model training because he sees it from all of
- 1:49these AI foundation model companies needing more data.
- 1:52And then there's we, it seems like on pre training we are
- 1:56hitting a wall on data. I think it was the clear message
- 1:58very. We'll debate that in in a
- 2:00second. I think that that clearly
- 2:02defined the day. I'd say my, I mean it's the
- 2:06friends we made along the way. I mean, I love everybody who,
- 2:08who was there and I think like just good group of founders and
- 2:12like getting everybody in a room and hanging out.
- 2:14There's a lot of fun, man. When I started hosting events
- 2:17with you guys, like I did not know anything about the events
- 2:20business. Then last year what we had two
- 2:22cerebral valleys and I got married and you know, now we've
- 2:25done 3 events this year. I was like, I'm a big believer
- 2:28in events and it's like, oh, we had great chairs.
- 2:30So everybody sat close to each other in the front and like it
- 2:34felt like standing room only and like, so that was, yeah, I like
- 2:38the vibes. I thought the vibes were really
- 2:39good. People were dialed in.
- 2:41I almost wanted to say like go downstairs and like talk to each
- 2:44other. Like you don't have to sit here
- 2:45and watch everything, but obviously if you're putting on
- 2:48on stage stuff, you're happy people want to watch.
- 2:50So yeah, I thought it was it was a good, good vibes.
- 2:53It was probably my audience. Was dialed in.
- 2:55The audience was dialed in. I think people wanted to hear
- 2:58from the panelists, which which is great.
- 3:00That means we have a great panel.
- 3:02This episode is presented by Breck's The Financial Stack that
- 3:05founders and VCs Can truly bank on.
- 3:09Imagine what your founders could do with their runway if they had
- 3:11a banking solution that had no minimums, no transaction fees,
- 3:16and 20 times the standard FDIC protection.
- 3:19Plus they could earn an industry leading yield while maintaining
- 3:23access to funds whenever needed. Brex simplifies financial
- 3:26services for start-ups so they can focus on building.
- 3:30Connect your portfolio to the financial stack that one in
- 3:32three US venture backed start-ups already use.
- 3:35Check out brex.com/banking Dash Solutions.
- 3:42So yeah, we're starting off James, you started to hint at
- 3:44it. Alexander Wang, the CEO of Scale
- 3:47AI, started off the day and really framed a conversation for
- 3:52the whole conference, which is have we hit sort of a wall, you
- 3:56know, and obviously a wall, you know, is a somewhat broad term,
- 4:01but he basically. You explain the wall for people
- 4:03because I feel like we're using this for an.
- 4:05Accident. Yeah, yeah.
- 4:06Insiders here, but what is? The Wall.
- 4:08The wall is this idea that foundation models aren't going
- 4:12to get smarter. You know that fundamentally
- 4:14everything in AI world depends on sort of the power of models.
- 4:20And the ways the models are supposed to get smarter are
- 4:22either they throw more NVIDIA GPUs at it basically, and just
- 4:27get bigger and that they're the money's there, it seems.
- 4:30So the money's there to spend on the GPUs, Nvidia's producing
- 4:32them, Amazon and others are trying to get in the game.
- 4:36And so there's this real question of like, OK, given you
- 4:39have access to GP US and money to spend on them, will you be
- 4:42able to improve the models? And nobody really knows besides
- 4:45Open AI, Anthropic and a handful of people.
- 4:48And so there's a lot of skepticism that that's going to
- 4:50be enough. So they need sort of new
- 4:53approaches. And then there there's also this
- 4:55question of given that Open AI seemingly as like inhaled all of
- 5:00YouTube and like everything else on the Internet is, is there
- 5:03really more data to have, even if they're manually using
- 5:06companies like scale to to find more like, are they really going
- 5:09to train, train these models? So the the wall is this building
- 5:13of the foundation models has has that tapped out?
- 5:16And I mean, Max and James, yeah. What did?
- 5:18How did you take Alexander's response to that question?
- 5:22I mean, he was the first speaker, as you said.
- 5:25I would say had I just watched him, I would have thought, oh,
- 5:28we're definitely hitting a wall. Like he basically said it
- 5:32without saying it, as in, yes, it's actually quite a struggle
- 5:35to get these models to the next generation to scale up the data.
- 5:39You know, everyone's talking to me about how we can do this, how
- 5:42we can clean the data, how we can create more synthetic data,
- 5:44this kind of stuff. Like it was a very short case
- 5:47for the next generation of models, which obviously worried
- 5:50me because I'm very excited about the.
- 5:52Next, I mean he said it seems to be the case that we've hit a
- 5:55wall on pre training. So the large cluster training on
- 5:58huge amounts of Internet data that seems to have genuinely hit
- 6:01a wall, but we haven't hit a wall on progress in AI.
- 6:03So, James, do you want to explain this progress in AI
- 6:06Point? Yeah.
- 6:07I mean, I think he's making a distinction here that maybe we
- 6:11have hit a wall on pre training performance.
- 6:14So maybe we've run out of data or you know, just adding compute
- 6:19doesn't seem to be getting the gains that we would expect see
- 6:22or seen in prior generations. But I think he's making the
- 6:25claim that that doesn't mean we're going to see diminishing
- 6:29model performance, let's say next year beyond because we will
- 6:33be able to improve the models with test time, compute or
- 6:37synthetic data like Max. Book a little bit too human, you
- 6:40know, reinforcement learning. I think some of it is happening
- 6:43in sort of this post training where you built the model and
- 6:46you're sort of cleaning up around the edges, getting better
- 6:49data. And so it's sort of like, well,
- 6:52with scale we can get past the wall.
- 6:54But the classic foundation model approach, you know, basically
- 6:57attention is all you need through a lot of, you know, run
- 7:00a bunch of transformer models and that's it.
- 7:02That approach you know the argument is in.
- 7:05A wall what sort of makes sense in a a sort of philosophical way
- 7:10that the models are about as good as they're going to get in
- 7:14terms of giving like quick answers to questions right?
- 7:17It's like we've scraped all the data on the Internet and we've
- 7:19scraped every video on YouTube. So we basically collected all of
- 7:22this sort of artifacts of human intelligence that you can right
- 7:25fairly quickly. And so these models are like
- 7:28about as good at quick answers as you could possibly get right.
- 7:30And that's, but what everyone's saying and what you guys are
- 7:34sort of alluding to with this post training concept is the
- 7:37idea of thinking for longer periods of time and sort of
- 7:40ruminating on what a good answer to a question would be.
- 7:43And what they call, you know, chain of thought and sort of
- 7:46iterating on that is where the next generation of games will
- 7:49come, which analogizes very similarly to to humans in many
- 7:53ways, right? Like very, very, very few humans
- 7:55can give like a quick answer to sort of such a broad array of
- 7:58questions as the current generation of models.
- 8:01But we obviously have Einstein's and Oppenheimer's and, you know,
- 8:06great literature, of course, you know, the Shakespeare's and and
- 8:09Jane Austen, where they had years, you know, or months and
- 8:12years and decades to sort of think about things and then put
- 8:15out these kind of finished products that moved the human
- 8:18race forward. Right.
- 8:19And it kind of seems like we're hitting this wall at like, yeah,
- 8:22this thing can instantly give you an answer at about the
- 8:24maximum level of human intelligence.
- 8:27But where the big picture, the Super intelligence, the next,
- 8:29the next generation will be will be give the model a really
- 8:32interesting problem. Let it think about it for a day
- 8:34or a week or a month, and then you'll get something really,
- 8:36really good out of it, which I think is kind of an interesting
- 8:39metaphor. Podcast listener, you get an
- 8:41advantage over the conference attendee and that we can skip to
- 8:43the end of the conference where we had sort of the rebuttal.
- 8:46Literally, you know, first session we've seems to have hit
- 8:50a pre training wall last session, Dario's like Dario
- 8:53Amade, the anthropic CEO, you know, says basically we haven't
- 8:57hit a wall. I was among the first to
- 8:58document the scaling laws and the scaling of AI.
- 9:01Nothing I've seen in the field is out of character with what
- 9:04I've been I've seen over the last 10 years or leads me to
- 9:07expect that things will slow down.
- 9:09I think one challenge with his point of view is I don't think
- 9:13he, it still seems like we need new ideas or new approaches to
- 9:19continue to improve things, right?
- 9:21It's, you know, like. Neither Alexander or Dario are
- 9:24saying, yeah, just throw a compute and scale and this will
- 9:28continue to work. I just think they're both saying
- 9:29the same thing, which is we need to do things slightly
- 9:33differently than we've done them before.
- 9:35But we have the visibility into that that we feel confident it
- 9:40will continue to, the AI output will continue to improve.
- 9:44But Dario is more on the side of the model companies have like
- 9:46good ideas and sort of approach will continue to put out, you
- 9:51know the pace. Maybe maybe you felt you felt
- 9:53like Alexander was like, I don't know.
- 9:55I mean, I think we have an idea of how this could could improve,
- 9:58but we don't actually. It's not like guaranteed.
- 10:01I mean, I, I mean, they're both talking their books in a little
- 10:03way. I mean, I do think Anthropic is
- 10:06pretty dependent on being one of the one of or the strongest
- 10:10model companies. Well, I think just to wrap this
- 10:12discussion, maybe I'll let Goat see as well, because I think he
- 10:16a big portion of his discussion was also are we hitting this
- 10:19mystical wall And he sees it from the data and infrastructure
- 10:22side. His take was it's just really
- 10:25effing expensive to train the next generation of models.
- 10:29And so possibly one of the reasons we're hitting the wall
- 10:32is that the next order of magnitude of cost is maybe not
- 10:35sustainable for any company that's, you know, even the fan
- 10:39companies potentially or certainly any company below the
- 10:41level of the fan companies. So I think he was a little bit
- 10:45short the next generation of scaling really driving new new
- 10:49models. But as you said, the kind of
- 10:51cherry on top of the the scaling wall discussion was Dario being
- 10:53like no full throttle like ready to go.
- 10:56Then like the day after the conference, he stuck a
- 10:58partnership with Amazon. I don't know.
- 11:01My, my sense is data bricks, you know, has seen the progress of
- 11:06Facebook's llama and basically said, oh, that's a pretty good
- 11:09open source model. I don't know how much we need to
- 11:11be building our own models. So I think his strategy has been
- 11:15more like we're never going to be competitive relative to all
- 11:18the spending. But I don't, it seems like there
- 11:21are lots of signs that like the hyperscalers are going to spend
- 11:24to build the model as long as they see evidence that that
- 11:28spending will produce an improvement on the trajectory
- 11:32we've seen so far. And that's sort of where this,
- 11:34you know, it becomes like what are they seeing?
- 11:36Are they seeing signs that all that spending will deliver
- 11:40something? I think the next big hitter that
- 11:43you know you would like to dig into given your kind of
- 11:47relationship there is Martine Casado, right?
- 11:49The the A-16 VC who has gotten deep into the kind of political
- 11:54intrigue was probably the biggest anti California
- 11:58legislation proponents, SB1047 and I think specifically had an
- 12:03insight in what they look for in companies that are taking
- 12:07advantage of this new generation of models.
- 12:08So what was your take on Martine?
- 12:10Yeah. I mean, I think he had this
- 12:11line. You know, it's, it's just like,
- 12:14well, there actually. I'll, I'll flag two things, one
- 12:17a little broader. It was sort of like what's
- 12:19investable in AI, like who has the mode?
- 12:22And his take was sort of, you know, we don't know.
- 12:25And so we're going to invest in everything.
- 12:26And I, you know, joke that that was my sense of the Andreessen
- 12:30Horowitz approach to investing, which is just like, let's cover
- 12:33it all and like, because you don't know what's going to be
- 12:36sort of the future. So I, I thought that was a
- 12:38funny, I mean, he didn't really like push back on my
- 12:41characterization. More specifically, I thought he
- 12:45did have an, I mean, I thought he was great overall and I, I
- 12:47even that part, they might be right, just like index AI is the
- 12:51right approach. But he sort of made the point
- 12:53that he thinks in sort of business applications, there's
- 12:57sort of a strong chance that what's going to differentiate
- 13:01companies is the same stuff that it differentiated them in the
- 13:03soft software or the SAS era. It's just like know your
- 13:07customer, know how to sell to them, you know, know, you know,
- 13:11have both sides of the marketplace, like have great
- 13:14integrations in the space. You know, it's not necessarily
- 13:16like have the best model, you know, have the the best coders
- 13:21in, you know, I don't know what call center, you know, call
- 13:25center AP is like if you're going to build the next
- 13:28generation of software company, it might not be you have sort of
- 13:33the hardest core engineers. It's that you sort of know call
- 13:36centers, you know what their customers want, you know how to
- 13:38sell to them. You have a strategy, you start
- 13:40to build integrations. I don't know, what do you guys
- 13:42think about that idea? I mean sort of a core persistent
- 13:47question, Silicon Valley, which is like how much do you need a
- 13:50technological edge or can you just be a great sort of sales
- 13:53product company? Yeah, I mean, I don't think any
- 13:58company has had like a sustainable software edge for
- 14:02the long term. It's like my odd take on Silicon
- 14:05Valley. This is something I was arguing
- 14:06with a lot of VCs with about a year ago when we were
- 14:08fundraising where they're like, you know, do you guys have like
- 14:11the best generative AI for voice games?
- 14:13And I was like, no, like, obviously Open open AI and
- 14:16Entropic have the best generative AI.
- 14:18Like, you know, and I just think similarly to we just forget our
- 14:22history so quickly, like, you know, you didn't have to have
- 14:25the best cloud software to build a cloud based business.
- 14:27You didn't have to have the best, you know, database to
- 14:30build a business with databases. You know, you didn't have to
- 14:32have the best operating system to build an applications
- 14:34business on Windows or Mac or whatever.
- 14:36You know, I mean, there's, there's so many layers of
- 14:38technology that end up being commoditized and there's a few
- 14:41big winners and then folks who are building the applications on
- 14:44top often end up still being massive successes, but they're
- 14:48not driving these technological advantages.
- 14:50And these technological advantages often end up being,
- 14:53you know, capital and scale and kind of economies of scale
- 14:56advantages, which is I think where we're seeing the open AIS
- 14:59and the anthropics just kind of running away from everyone is
- 15:01there's this like winner take all cycle of having the best
- 15:04model means you get all these customers, means you get all
- 15:06this VC money, which means you get to pour it all into GPUs,
- 15:09which means you get the next generation best model.
- 15:11And, you know, after two turns of that wheel, you're, you're,
- 15:15you're gone. No one can ever catch you,
- 15:16basically. Well, that's the hope though.
- 15:18We've talked about open source models have been surprisingly
- 15:21able to catch up and so. Well, Facebook definitely is
- 15:24like trying to be the third person in that game basically
- 15:26right with with Llama. The Android, yeah.
- 15:29Martine, Martine also mentioned, which I thought was interesting
- 15:32that he sees opportunity for small models, right.
- 15:36And I think we've seen seen this a lot in the last year that
- 15:39there are companies who are able to compete by having small but
- 15:45very specifically focused vertical models, right, like
- 15:49music generation or image generation or video generation.
- 15:52Like it does seem that those that's a more fertile ground for
- 15:56startups to train models in those creative fields.
- 16:00Maybe, and I don't know, maybe we'll see that that all gets
- 16:03kind of eaten up by Open AI and Anthropic as well.
- 16:08But so far right now, it does seem there's a lot of start-ups
- 16:10succeeding by training these smaller creative models.
- 16:14And presumably a theme we've talked about on this podcast,
- 16:17models that are optimized for the cost.
- 16:19It's like if you're focused on video or voice and you're doing
- 16:23it everyday, you're like, oh, you know, it's not just about
- 16:26getting the best performance so we can show off.
- 16:28It's like doing it in a way that is sustainable and people can
- 16:31build businesses on top of or you can build a business on top
- 16:35of. Another, another thing that
- 16:36really stood out to me from our teens, this interview with you,
- 16:41Eric, was this kind of question of like, what are, what is
- 16:45agents? What are, what are we going to
- 16:46see in the next year from agentic experiences?
- 16:49And he had like I thought a very good definition, or at least
- 16:54like 2 definitions of agents, right?
- 16:56One being that agents are, you know, what we think of human
- 17:00agents like customer support agent or travel agent, right?
- 17:02Something that you can communicate with, it's sort of
- 17:06replicates that experience of talking to a human to accomplish
- 17:09a task being kind of bucket A and then bucket BB being.
- 17:14What a lot of people are excited about now, but we haven't seen
- 17:16really yet is the ability for these models or agentic models
- 17:21to go and take action on be your behalf over long periods of
- 17:25time. And I think that will be
- 17:27something we see more and more is allowing these agentic
- 17:31systems to go, you know, use your computer, use your web
- 17:34browser and just go do things on the Internet for you for for
- 17:38hours and hours at a time, Maybe even coding or using other just
- 17:43like human like interfaces, like a browser or a code IDE, right?
- 17:48And, and actively using them without human input.
- 17:52So I'm excited for both those experiences.
- 17:54Curious if you guys had thoughts on on his discussion of agents.
- 17:58I mean, this sort of leads into my voice AI panel, but basically
- 18:01everyone said we're going to pass the audio touring test next
- 18:04year, IE end of 2025, saying we could be on the phone with a
- 18:08customer support agent and not know if they're a human or not.
- 18:11The majority of the time. I buy that.
- 18:14Obviously, James, you and I have been prototyping a lot of voice
- 18:16AI game stuff recently and it's unbelievably realistic.
- 18:19And I think that customer support is, in the end, fairly
- 18:24intellectually not very complicated.
- 18:27And so you could easily throw a fairly simple AI voice agent at
- 18:30customer support and handle 80% of support calls to the Apple or
- 18:35the AT&T phone numbers or whatever, like ASAP.
- 18:38The voice discussion I thought was super interesting.
- 18:41I mean, sort of give us a sense of like, you know, this was a
- 18:46larger panel, the sort of characters on it and including
- 18:50yourself and sort of where each of you sort of sits in the, you
- 18:53know, what motivates each of you in terms of voice and then what
- 18:56you thought was interesting out of it.
- 18:58Yeah, I mean, so voice AII do think even though I'm very
- 19:02biased is a is a hot topic this year and I think we'll be going
- 19:05forward because talking to other people is obviously a huge
- 19:09portion of human civilization and I think we're now able to
- 19:11replicate that. And so the people on the panel,
- 19:14we had two kind of tool slash infra companies.
- 19:17We had Cartesia, which does voice output, IE synthesized AI
- 19:22voices. And then we had Deepgram, which
- 19:25is mostly known for speech recognition, but is also adding
- 19:29kind of large language models and also voice output.
- 19:32And then on the kind of product and application side, we had
- 19:34Character, which is famous for letting people chat with Tony
- 19:38Stark in a web browser. But what was interesting here
- 19:41was they've added a voice mode where you can actually call Tony
- 19:44Stark and have a conversation with him for as long as you
- 19:47want, or you can type to him and he'll chat back to you.
- 19:50And then there was myself hosting a panel.
- 19:52We make voice AI games that are powered by, you know, chatting
- 19:55with a game show host or, you know, medieval times convincing
- 20:00a a knight to go slay a dragon or something like that.
- 20:02Two interesting things about the panel really stood out to me.
- 20:05One was, as I alluded to earlier, everyone's saying we're
- 20:08going to pass this audio version of the Turing Test next year.
- 20:11I as in. You're going to be able to talk
- 20:13to an AI and not know it's an AI for extended periods of time in
- 20:16the next 12-14 months, whatever everyone's take was.
- 20:20Basically this is going to revolutionize like the phone
- 20:23call as a meta concept, right? Like everything you can imagine
- 20:27doing a phone call for that isn't calling your friend or
- 20:30your mom or whatever is going to be completely revolutionized by
- 20:33the fact that a company is not going to need a person on the
- 20:36other end of the line. Whether that's customer support
- 20:40or, you know, just getting the, the hours of a business or
- 20:43ordering from a restaurant or, you know, any, anything you
- 20:46could do a phone call for, we're no longer going to need humans.
- 20:50And that's going to be, I think, a huge, huge impact on the sort
- 20:54of cost structure of a ton of different companies.
- 20:56But also it might, it might make people a lot more likely to call
- 21:00these companies to go solve their problems.
- 21:01That was interesting. And then the second interesting
- 21:03thing more on the consumer side, which is where where I focus is
- 21:07the character AI CPO Aaron Teague who spoke, said when they
- 21:11added the voicemail, they thought people were going to be
- 21:13calling these characters all the time, but they actually
- 21:15discovered that a lot of what people want to ask fictional
- 21:18characters is is pretty embarrassing.
- 21:20Like she specifically said in our, in our preview that she had
- 21:26wanted to ask Martin Luther King why he had cheated on his wife
- 21:29as such a, an amazing, like interesting figure in history.
- 21:32And she said, I'm, I obviously would never ask the real Martin
- 21:35Luther King that question, but I really wanted to know from this
- 21:38character version of, of Martin Luther King.
- 21:40And so I asked him this question, and he gave this
- 21:42really nuanced, thoughtful answer about how he, you know,
- 21:47he was on the road all the time. And it was a really complicated
- 21:50existence. And he didn't have the best
- 21:51relationship with his wife. And, you know, he had failed
- 21:53morally in some ways while still being a civil rights leader,
- 21:56which I thought, I don't know. I thought that was a really
- 21:57interesting anecdote. Her point, though, was that
- 22:00people don't want to say this stuff out loud.
- 22:02They don't even want to hear themselves say this when they're
- 22:05speaking to an AI character. And so what's actually very
- 22:07popular with their voice mode is typing these kind of
- 22:10embarrassing questions in a box, whether on your computer or your
- 22:13phone, and then hearing the voice output from the character,
- 22:17because that creates this more kind of emotive experience and
- 22:21feedback from the character. So I thought that might be like
- 22:24the beginning of a kernel of insight of how people want to
- 22:26interact with the AI characters. You know, maybe that's just
- 22:29specific to their company, but I don't know.
- 22:31I got a lot out of that discussion and I thought it was
- 22:33really, really interesting. A a, a big question that you
- 22:37guys are probably facing as you place games like those sort of
- 22:42reasoning engines, language models allow for these sort of
- 22:45unstructured interactions where it's, you know, I ask, you know,
- 22:49Martin Luther King a question gives me an answer.
- 22:53But do you think consumers generally want that or do they
- 22:57want sort of a, you know, a narrower path?
- 23:00You know, it's like, well, in some ways in the the traditional
- 23:03video game world, it's like, do they want all these sort of like
- 23:06open world games or do they want sort of to be LED along sort of
- 23:10a plot? I don't know.
- 23:12Do you think we're like taking the wrong thing from what LLMS
- 23:15can do where it we're giving up on sort of the structure because
- 23:20we can't, you know? I would just make the point that
- 23:22like the there's a lot of things you can do in games, but the
- 23:26mistake probably is to try to do everything at once with LLMS,
- 23:29right? So you could like design all the
- 23:32assets with LLMS or you could have like these NPCS that you
- 23:36can talk to, or you could even design the quests themselves
- 23:39with LLM. All of those could be
- 23:41interesting if you if you do them really well or in
- 23:43isolation. I do generally think though,
- 23:46that to Eric's point, we at Volley definitely lean into the
- 23:50more structured experience site. Yeah, Just, you know, as a one
- 23:54liner, I think we believe people want more structure.
- 23:57We think people want a goal and something they have to do at a
- 24:01mission, you know, to pursue at a game, and that LLMS can make
- 24:05that experience more fun and more natural.
- 24:07I, I, I agree that's what we are focused on in volley.
- 24:10I think that obviously from characters success, people also
- 24:14just want open-ended companionship in, in their
- 24:17characters and they and they want to create like never ending
- 24:21friendships that they can just kind of, you know, mold and
- 24:26adapt however they want on any given day.
- 24:28So yeah, I think there is a value in the unstructured
- 24:33aspects as well. I mean, to really step back, you
- 24:37know, I'm just I feel like I'm faced with this question of do I
- 24:41think AI is going to improve a lot next year?
- 24:43Like how much are we sort of high on our own supply or not?
- 24:46You know, you see lots, you know, I was very skeptical of
- 24:50self driving cars along the way. And now I'm an extreme bull
- 24:54wrote like very positive case and newcomer on the Free Press.
- 24:59But it's like, you know, you Silicon Valley is loves bubbles
- 25:04and like everybody involved sort of is, has an incentive to sort
- 25:08of lean into them and just sort of speculate, you know, sort of
- 25:11to do sort of the next step, but can be wrong on sort of timing.
- 25:17So yeah, I, I guess throughout this conversation, I'm sort of
- 25:20riddled, you know, by this debate we had on stage between
- 25:25sort of have we had a wall or not?
- 25:28And I guess one thought I had listening to you guys is this
- 25:31sense that like 1 bull case for AI is honestly that the open
- 25:39text chat is really hard for people to get value out of.
- 25:44You know, it's like if you just have this spot, it's like, why
- 25:47don't I use AIA lot? It's like, well, because I have
- 25:49to sort of come up with a plan and figure out what I want.
- 25:52And I think sort of tying a couple pieces together.
- 25:54It's like Martin's point. It's like unique companies that
- 25:57build a very clear use case, like what you're saying about
- 26:00games. You need to give people sort of
- 26:01a road map to use it. I do think, you know, even if
- 26:06the reasoning doesn't improve as much as we want, next year,
- 26:09there's just going to be a lot of like successfully leading the
- 26:13horse to water. Or it's like showing the
- 26:15consumer or the business like how how to use the AI we already
- 26:20have today in a much more productive way.
- 26:24Yeah, I 100% agree. I think that these tools are
- 26:29amazing, but they're very unstructured, to your point.
- 26:32And you really have to be sort of a creative person to sort of
- 26:37get value out of these AI. You have to imagine what it
- 26:40might be able to do and then keep throwing, you know,
- 26:43spaghetti at the wall until it does something amazing in many
- 26:45cases. Obviously one of the most
- 26:47overused line in Silicon Valley, But you know, the future is
- 26:49here. It's not just not evenly
- 26:50distributed yet. Like everybody doesn't know
- 26:52about it yet. There's so much stuff this
- 26:55current generation of models can do that people don't take
- 26:57advantage of. And so much of that is going to
- 26:59be integrated into newer types of applications or making
- 27:03existing applications much better or just new types of user
- 27:06experiences that that allow people to get more value out of
- 27:09these AI tools we already have. And as a lay user using open AII
- 27:13can be like, clearly I wanted to have a repository of my stories
- 27:17and do stuff based on it, but I don't have access to a team of
- 27:21engineers to sort of like go do that.
- 27:23And if, you know, a product is built that allows that, then
- 27:26even these use cases that I can sort of imagine will be sort of
- 27:29more at my fingertips than there are now anyway that that got
- 27:33very zoom out, but I, I think you know.
- 27:35That's a good. Point let's let's do our sort of
- 27:36like we're not going to hit every panel in such great depth,
- 27:40but I do think there were great Nuggets out of them.
- 27:42And people maybe don't want to listen to the full conversations
- 27:45on YouTube, which by the way, all these conversations are on
- 27:48YouTube. You don't need to hear our
- 27:49characterization of it. You can go and we encourage you
- 27:52to go listen yourself. But this is maybe a, you know,
- 27:55the a looser way to inhale everything that happened.
- 27:59Max, you want to kick off with sort of our digest of some of
- 28:02the other talks. Yeah, we're going to do a
- 28:04section called the lighting round where we bang through 5 or
- 28:076 panels in 30 seconds or less, maybe 60 seconds if we're really
- 28:11getting into it. Going to go in the order of the
- 28:13discussion and throw it to whomever has the the juiciest
- 28:17Nuggets nugget or the hottest take from that discussion.
- 28:20So first, Tim Tully gave a big presentation on AI in the
- 28:25Enterprise. I think, Eric, I'm going to
- 28:27throw it to you. What's the one liner from Tim
- 28:29Tully's presentation? Enterprises are hedging their
- 28:32bets. They're not.
- 28:34They've learned. They've learned from AWS and
- 28:37Microsoft and the hyper scalers and they don't want to go all in
- 28:39with anybody. They're going to like, you know,
- 28:41we sort of saw it at the Goldman talk, Marco Argenti at Cerebral
- 28:45Valley New York. It's like smart enterprises are
- 28:48trying open AI, They're doing anthropic.
- 28:50They're using small models and they're not going to like go
- 28:52deep with just one model. They're going to make sure that
- 28:54nobody has leverage over them and they're diversifying.
- 28:58He also called out that Anthropic had had made a huge,
- 29:01huge gains against open AI in the last year, which I think is
- 29:04super important to to know. And, and he also mentioned that
- 29:10that coding was the number one, I think by far use case for the
- 29:13enterprise in terms of these, these models.
- 29:15And maybe, maybe that goes hand in hand with Anthropic doing
- 29:19well against open AI in the last year.
- 29:21And this was Menlo, where Tim Tully is an investor, is an
- 29:25investor in Anthropic, but they'd commissioned this huge
- 29:28survey of enterprises to get the data.
- 29:30So just hopefully just reporting back the data they found.
- 29:34All right, next one all. Right next one we had a biology
- 29:37panel which was moderated by your newcomer colleague
- 29:42Madeline. The quick take from the biology
- 29:45panel is that models are going to make drug discovery much
- 29:48better. I think, you know, the public
- 29:50will believe it when we get the drug, but it seems like people
- 29:53in the industry are super optimistic that especially small
- 29:56targeted models will help them discover new drugs.
- 29:59Obviously, we've talked on this podcast before that, you know,
- 30:02human regulatory burdens will remain, but finding new drugs
- 30:07could be miraculously important to human beings.
- 30:10Dario and Anthropic has made a similar case that he thinks
- 30:13biology is an area where models are going to be
- 30:16transformational. So a lot of optimism from that
- 30:19world. All right, nice.
- 30:22Next one one of James's panels. Train, tune or turnkey models.
- 30:26Yeah, we gave you a big one. This is broad.
- 30:29Big fan of alliteration. James What's?
- 30:31What are the key moments here? This panel represented different
- 30:35parts of like train tune or turnkey right.
- 30:37You had glean that was like, OK, we deliver you a solution.
- 30:40He's like, it works on day one. You have core weave on the other
- 30:44end of the spectrum, which is like we're going to help you
- 30:46sort of build your own models and like get you access to
- 30:51compute and then together, which is sort of in between, which is
- 30:54we're going to sort of have a little more of a heavy-handed
- 30:56approach. I don't know, Did they see the
- 30:59world differently or did you feel like there was some
- 31:02consensus on what companies want?
- 31:05And I think the main consensus here is that they're all trying
- 31:07to figure out how to get into non-native AI enterprises,
- 31:11right? So they, you know, I mean, maybe
- 31:14glean is is kind of the furthest along there, but core weave and
- 31:20together they, you know, they primarily work with AI labs that
- 31:24need to train models and fine tune or host the model by API,
- 31:29right. But like, what are the use cases
- 31:31for the enterprise? I think like there's some
- 31:34interesting things. I think there's an example that
- 31:37together has where they worked with Washington Post to allow
- 31:42you to kind of chat with their archive of, of articles.
- 31:45So that what kind of requires this fine tuning use case?
- 31:48And I think that's like a big question for the enterprise,
- 31:50Like where does it make sense to to allow fine tuning or to use
- 31:54fine tuning to improve model performance?
- 31:58It's, you know, when you have proprietary data that isn't
- 32:01trained in the foundation models.
- 32:04Next one, how to train your robot.
- 32:07I thought this was super entertaining.
- 32:08James, you moderated this one. Robots in the home, at work.
- 32:13What? What did we learn about the
- 32:14future of AI robots? I I thought that Jonathan Hurst,
- 32:18he's the chief robot officer at Agility Robotics, was super
- 32:22interesting. He's building humanoid robots
- 32:26primarily for warehouse use cases.
- 32:28And he just made a lot of great points about how hard it will be
- 32:31to introduce humanoid robots into into more consumer
- 32:36environments like the home. A they have to like be able to
- 32:40navigate, you know, every aspect of, you know, a house,
- 32:44including, you know, getting upstairs and, you know, figuring
- 32:48how to if you're if they're going to help you unload your
- 32:51car or, you know, take the groceries out and put them in
- 32:53the cabinets right in the refrigerator.
- 32:55Like all of that is like very specific use cases that need to
- 32:59need to work Florida State. But like, more importantly,
- 33:02there's a big safety element of having, you know, 100 LB a £150
- 33:07robot walking around your home with your kid.
- 33:10Every every dystopian movie in the world warrants.
- 33:13Us. Yeah, but it's not, I mean, it's
- 33:15almost like the, the, the bigger risk is just kind of not that
- 33:21they will go get evil and go, they'll kill you.
- 33:24It's just that they'll fall over and crush you or something.
- 33:27You know, I think that was an interesting point.
- 33:29Of course, the constant irony of all of this is that we drive
- 33:32around in heavy metal cars that because they're human
- 33:34intelligence powered, we're fine with insane collateral damage.
- 33:37And maybe the odds are, you know, robots in your house
- 33:40probably aren't going to kill more people than cars.
- 33:44Do well. We had the.
- 33:45We had the, the head of onboard AI at, at Waymo, also Srikanth
- 33:52and he, he made some great points about just how hard it is
- 33:56to collect data at, at these long tail kind of environments
- 34:02for, for robotics. Like, you know, they, there's
- 34:06just constantly new scenarios that they're seeing that, you
- 34:08know, you wouldn't experience in 10,000 lifetimes or something if
- 34:12you were just a normal human driver.
- 34:14And you know, how do you, how do you handle that?
- 34:17Is is a super interesting question in robotics?
- 34:20All right, last one, Marissa Meyer, I got to interview.
- 34:24Highlight. Then yeah, the legendary head of
- 34:27search at Google, CEO at Yahoo, she is building an AI photo
- 34:32powered app, which is sort of like, I don't know, Instagram
- 34:36for groups or something still early on that app.
- 34:39So I think maybe the more interesting part of the
- 34:40discussion was her takes on Google having run search there
- 34:44for a while. I think Google's AI summaries
- 34:47replacing its links to websites was a big, you know, topic in
- 34:51the world right now. And also kind of upstarts like
- 34:54perplexity or open AI adding search to their chat interface
- 34:58as kind of a disruptive force against Google.
- 35:00I think are are really interesting trends we're seeing
- 35:04and her take. I mean, she's very loyal to
- 35:08Google, I would say, but I would also say that she said a lot of
- 35:11things that would scare me a lot if I worked at Google or were an
- 35:15investor in Google. Like she basically said, yeah,
- 35:18IA huge portion of what I used to type in the Google search
- 35:21box. I just type into ChatGPT.
- 35:25I assume a lot of other people are doing that too.
- 35:28She also on the kind of supply side, IE the websites that
- 35:32Google indexes, she said, yes, I'm sure more and more of that
- 35:36is being generated by AI already and not really adding new value
- 35:40to the Internet, meaning that sort of the kind of information
- 35:44that Google sits on top of is degrading or at least not
- 35:47improving in the last couple years, which is a big problem
- 35:50for Google. And then she was like, yeah, I
- 35:52mean, the kind of AD model doesn't work in a world as well
- 35:55where you just want the answer to your question rather than 10
- 35:57possible answers and maybe three of them are sponsored links or
- 36:00something. As a media person, yeah, the
- 36:02idea that they're just, like, strangling the creation of the
- 36:06information that they rely on is, you know, an ongoing,
- 36:09fascinating dynamic. Yeah, I thought it was a pretty
- 36:13short case for the core business of Google because she was like,
- 36:16the only way you get out of this is by being like a mediator,
- 36:18shopping and Ticketmaster and stuff and just directly
- 36:22accessing like inventory of goods from Google.
- 36:25And I was like, are these companies going to let Google do
- 36:28that? And she was kind of like, no,
- 36:29probably not. Like it's really hard was.
- 36:32The way it. Almost suggests you know.
- 36:35Google clearly isn't going this way, but it suggests Google
- 36:37should be on the side of getting people to other websites.
- 36:40It's like, oh, what do we do? Well, we get people to other
- 36:42websites, but instead they've decided like, no, we need to
- 36:46keep people on our page. Isn't the obvious biggest
- 36:49oversight so far that they just haven't created a Gemini
- 36:52assistant like within Google search like the?
- 36:55Just like a separate product instead.
- 36:57Of just trying to take over Google, just put it in.
- 36:59No, not even a separate project. I'm just saying like right now,
- 37:02yeah, you get these Gemini summaries at the top, but like
- 37:06it's not a chat bot. Like why not just put a chat bot
- 37:08into that that search experience?
- 37:10Like that's what people want. Just like side by side or just.
- 37:13I think so, yeah. I mean it.
- 37:15Or I feel like people want I. Bet they're working on that.
- 37:18I mean Open AI is doing the opposite right?
- 37:20Open AI is putting a search engine in a chat bot.
- 37:23Right. Yeah, I think it's just really a
- 37:25struggle to disrupt their entire business.
- 37:27I mean, again, the word disrupt is really overused, but it's
- 37:29like they have the greatest ad business of all time and putting
- 37:33a chatbot in the Google search box is probably quite bad for
- 37:36that business in many ways. And because you just can't show
- 37:39as many ad sponsored links. And so I'm sure they're working
- 37:43on what you're saying, James, but I, I would imagine they're
- 37:45very, very scared of destroying their entire business by
- 37:47launching this kind of product. So TVDI thought that was super
- 37:52interesting discussion. I don't know, Eric.
- 37:54Wrapping it up, final takes. Final takes, I mean, I'd say
- 37:58we're 4 cerebral valleys in in two years.
- 38:01At the first one, we were extremely bullish.
- 38:04It was early. We saw the promise of chachi BTI
- 38:06think felt like super excited. I think you know by the second
- 38:12one that you're in November last, so November 2023, it was
- 38:16sort of like the big questions of AI and sort of these these
- 38:20like big debates about like the world.
- 38:25I have a point about this. I have a point.
- 38:27About this, you jump in. Sure.
- 38:28I mean, I think when we started this conference series, it was
- 38:33March 2023, right? And it was only a few weeks, I
- 38:37think after GPT 4 had launched, right.
- 38:42But at that time, there was like a very large Overton window of
- 38:46what was being discussed. Like a lot of the people at the
- 38:49conference, like we're very scared that like in the next two
- 38:52to three years, you know, we would be all all be dead or, you
- 38:57know, that that AI was going to take over the world.
- 39:00And then other people, you know, were kind of like really focused
- 39:05on the foundation models and, and how, you know, they were
- 39:08just going to be, you know, take all the all, all all the value
- 39:12in the ecosystem, right? There's just like a massive
- 39:14Overton window of what was possible.
- 39:16And I think we've just seen that narrow like we've seen now, you
- 39:19know, more agreement in some sense, like here are the safety
- 39:22risks, kind of what the the risk profile is.
- 39:25And, and here's, you know, kind of where the opportunity lies.
- 39:29It's not just all within foundation models.
- 39:31And, and I think like just a general, you know, understanding
- 39:34of maybe, you know, what pace of advancement we're going to see
- 39:38in this industry. You know, not like it's not
- 39:40super clear to any of us, like whether we've, you know, exactly
- 39:44how powerful GPT 5 will be and when it will be arrive or
- 39:48something. But at least we have like a
- 39:49pretty good understanding of what is possible in the space,
- 39:53Whereas I think even a year and a half ago that wasn't the case.
- 39:57I don't know. I'm very, I'm very mixed.
- 39:59I think we, we sort of say two things at once.
- 40:01Like we say, OK, AI today is really good, but we need to sort
- 40:06of give people a road map how to use it.
- 40:08I think that's true. But I do think there's a reality
- 40:10in our heart of hearts that you sort of alluded to.
- 40:13They're all waiting for Chachi PD5 or, you know, a superpower
- 40:17to O one or anthropic to leap ahead and really get this like
- 40:21moment again where we're like, Oh my God, it's so smart.
- 40:24And I do think that's what the wall discussion is all about.
- 40:27We're we're sort of thirsting for another sort of jump, you
- 40:31know, leap. And I feel like it hasn't
- 40:34happened again. Nothing has happened.
- 40:37Sort of like the first ChatGPT launch.
- 40:40Yeah. I mean, I think the key question
- 40:42is, was ChatGPT like the iPhone, you know, and everything else
- 40:47was the Palm Pilot? And even 17 years later, we
- 40:51still basically have an iPhone that I mean, I had that first
- 40:53iPhone in 2007. It's it's not so different.
- 40:56I was a BlackBerry, like, yeah. But I'm just saying there's only
- 41:00one iPhone moment, right? You could say like, OK, the
- 41:02iPhone 4 was like where they really got it all, like working
- 41:05super well. Or, you know, you could always,
- 41:07you could pick your favorite iPhone, but there still was only
- 41:09really one iPhone, right? I mean, you know, like and was
- 41:13Chachi BT it like was that the Rubicon like or is there kind of
- 41:16one more next big leaf, right, or is there a couple more?
- 41:21Right. I think that my take is there's
- 41:23a lot more to come. I mean.
- 41:24OK, there's a lot because if there's more Leafs of this scale
- 41:27to come in the near future, the world is gonna be so fucking
- 41:31crazy compared to the the life that we've had to date that I
- 41:35that is the sort of big open question, right?
- 41:37Is is there more of these transformational moments coming?
- 41:41Because if so, like. I think.
- 41:43We've already in a new era. We've.
- 41:44Talked a a lot about them today and I'm I'm bullish on them in
- 41:48the next one to two years. You know it's like it's these
- 41:51agentic models it's it's that go off and do things on your
- 41:54computer. It's like Max is talking about
- 41:57you know you're on phone calls with AI is much more often than
- 42:00you're on with with humans and that opens up a lot of new use
- 42:03cases. I think, you know, some of these
- 42:05small models like video and music, I think they're just
- 42:07going to get better and better and we'll be able to make, you
- 42:10know, Hollywood quality movies and amazing, amazing music
- 42:15personally, kind of for our own use.
- 42:18So I don't know, I think all of these could be like massive
- 42:21changes to sort of our day-to-day and society.
- 42:23And you know, we, we're, we're, we're, we're getting there
- 42:26pretty quickly. Great.
- 42:29All right. Well, we'll see where we are.
- 42:31We'll take state. You know, Cerebral Valley will
- 42:34probably have one to announce for the summer and certainly
- 42:36we'll be back in San Francisco in November.
- 42:40So we'll, we'll keep keep an eye on it.
- 42:43Sounds good. Thanks.
- 42:45Thanks guys. Yeah, see ya.
- 42:46See ya.