Latest / The Index Podcast / Jake Brukhman: The Truth About Decentralized AI Training - CoinFund
Transcript
- 0:13SPEAKER_00: Hey everybody, welcome to the index.
- 0:15SPEAKER_00: I'm your host, Alex Kahaya, and we've got Jake Burkham here, who is the founder and CEO of CoinFund.
- 0:24SPEAKER_00: We know each other well.
- 0:24SPEAKER_00: He's been on the show in the past.
- 0:27SPEAKER_00: Really excited to catch up with Jake.
- 0:29SPEAKER_00: I think it was maybe two years ago that you introduced me to Bagel and I was kind of just starting to get really interested in AI.
- 0:37SPEAKER_00: I mean, this was this had to have been almost two years ago at this point.
- 0:41SPEAKER_00: And one of the things I've noticed about you is that you have been kind of on the bleeding edge of investing and finding like really interesting opportunities in this space.
- 0:51SPEAKER_00: So I'm excited to have you here and kind of catch up because a lot has happened since then.
- 0:56SPEAKER_00: Um thanks for taking the time.
- 0:58SPEAKER_01: Yeah, thanks for having me out.
- 0:59SPEAKER_01: It's good to be here.
- 1:00SPEAKER_00: So paint a picture for me back then of where we were, and then get into like the State of the Union of where we are.
- 1:10SPEAKER_00: Because I do think I just feel like we're in a different universes.
- 1:14SPEAKER_01: Yeah, I mean, two years ago is like a lifetime.
- 1:17SPEAKER_01: Well, I think in 2024, I don't think like the idea of agents was very widespread.
- 1:22SPEAKER_01: I definitely think some people were talking about them, but most people had not had, you know, this kind of um experience hands-on with them that made it real for them.
- 1:37SPEAKER_01: I remember like the first time that started to kind of become real for me.
- 1:40SPEAKER_01: I think it was in 2025, around tax time, when I was trying to get like, you know, that very early agent that OpenAI put out to try to like help me with my crypto taxes.
- 1:53SPEAKER_01: And it just like was not working very well.
- 1:55SPEAKER_01: And that was really like the first time I I had a hands-on experience with them.
- 2:01SPEAKER_01: But I think since then, you know, most people went from prompting AI in a back and forth like chat to actually being involved with an agent and and being and having an agent like do something um useful for them.
- 2:19SPEAKER_01: So I think on the agent side, we've gone you know a long way.
- 2:24SPEAKER_01: And then on the kind of infrastructural side, you know, about two years ago was when I started my journey with Pluralis and really, really getting bullish on the idea of decentralized training.
- 2:39SPEAKER_01: And two years ago, and before that, you know, most people were saying that this is just kind of like a crazy idea.
- 2:48SPEAKER_01: It first of all, you know, would not be supported by consumer hardware and you know, was essentially impossible and stupid to a couple of things.
- 3:00SPEAKER_01: So one is just a lot of research and technological progress around being able to actually put training onto lower-end hardware and consumer hardware and solve some of the bottlenecks that were originally cited as the reasons why this couldn't work.
- 3:20SPEAKER_01: And, you know, from an investment perspective, two years ago, a lot of investors were looking at in, like especially in you know, the crypt the decentralized AI world, they were looking at GPU networks and inference networks.
- 3:34SPEAKER_01: And I think, I think it's coming, it's becoming a little bit more obvious now that training is actually the core bet here, right?
- 3:45SPEAKER_01: And and some of you can see some of that in the fact that there's now a lot more discourse out in the world around companies needing to train their own models.
- 3:55SPEAKER_01: You have big firms like Kirkland and Ellis spending$500 million on their own model.
- 4:01SPEAKER_01: You have things like the Tinker API, you have companies like uh our company Prime Intellect essentially saying the training stack should be ripped out of these open labs and become a module, right?
- 4:13SPEAKER_01: And it should be open and accessible to everybody, um, and and thus opening up training to kind of a larger, wider set of enterprises.
- 4:22SPEAKER_01: And then, of course, you have um many examples in decentralized AI of companies who are now going into training.
- 4:30SPEAKER_01: We, you know, shoots.ai, which is um kind of the most prominent uh subnet on BitTensor, you know, had some traction and inference, but are now you know talking about something called Parallax, which is a decentralized training strategy.
- 4:47SPEAKER_01: Um you see uh certain companies starting to offer post-training and starting to generate revenue from post-training uh for uh for real customers, etc.
- 5:01SPEAKER_01: So where we have gone from in the last two years is you know went from this stuff is stupid and is never gonna work to actually this is one of the kind of key ways that you know infrastructural AI is gonna come to market.
- 5:17SPEAKER_01: And and there's some really exciting opportunities to save people money, right?
- 5:23SPEAKER_00: One, there's so much to unpack there.
- 5:25SPEAKER_00: Prime Intellect is such a cool company, and they just raised a pretty big round.
- 5:30SPEAKER_00: I think I can't remember the the the headline, but it was what like 50 million or something like that at a at a pretty large valuation.
- 5:38SPEAKER_00: And they've got revenue.
- 5:39SPEAKER_00: Uh, the thing I've liked about like liked about them is from the very beginning, they were talking about open intelligence and open intelligence as being something really important for for enterprises.
- 5:49SPEAKER_00: And I think that that is like what we're seeing open intelligence and open source AI and open source AI stacks, intelligence stacks have its moment now, which is kind of feeding into this whole thing around training, right?
- 6:00SPEAKER_00: Like everybody realized when Mythos and Fable 5 got restricted even temporarily that they really needed a different they whatever whatever investments you're making as a business in your intelligence stack, it better be flexible and you better have complete control over it.
- 6:15SPEAKER_00: You you better be able to switch models kind of at the drop of a hat.
- 6:19SPEAKER_01: Yeah.
- 6:19SPEAKER_01: So so by the way, um Prime Intellect's Series 8 was a$130 million race.
- 6:25SPEAKER_01: I just I had to look it up quick to remember the exact number, but you know, at Unicorn plus valuations.
- 6:31SPEAKER_01: And you know, you bring up a really, really important point.
- 6:34SPEAKER_01: This like event that happened a few weeks ago where the government just cut access to Fable 5, I think was a very instrumental and instructive event that you know where enterprises realize like, hey, you know, if we put this concentration risk into this one or two, these one or two companies that are just kind of beholden to be shut down, then you're actually putting your whole business, you know, in a very precarious position.
- 7:02SPEAKER_01: And that's just sort of the tip of the iceberg, because as the discourse continues, you know, we learn that these companies behave in in sort of monopolistic ways.
- 7:13SPEAKER_01: They suck in customer data and they learn from it, and then they they have this Microsoft Softtian kind of strategy of launching competing products, right?
- 7:22SPEAKER_01: And you see that especially in Anthropic, where you know they're launching coding agents and then design agents and then uh health and government and et cetera, et cetera.
- 7:32SPEAKER_01: And so, you know, the really cogent argument for enterprise is you know, you have, you know, if you start to rely on these big centralized players buying the story that kind of the marginally better, most expensive model is sort of the only way that you can do AI, then you're also sort of betting kind of your proprietary data, right, in the future of your business.
- 7:58SPEAKER_01: And I think for most enterprises, that makes them very uncomfortable.
- 8:03SPEAKER_01: I think there's a huge discourse around privacy, individual privacy and proprietary information kind of privacy for enterprises.
- 8:12SPEAKER_01: There's at the same time a market of open models that are primarily today being funded and issued by Chinese labs.
- 8:22SPEAKER_01: I feel like Meta is now going away from that, although we also have NVIDIA coming into the market with fresh funding for open models.
- 8:29SPEAKER_01: And what these models are showing is that you can have essentially comparable performance, and some people even think that they will surpass frontier performance uh at some point soon, and you can have incredible cost savings while doing that, and you can control the weights.
- 8:46SPEAKER_01: With the NVIDIA offering, it's also like awesome that not only are they open sourcing the weights, but they're also open sourcing setup, which has really not been practiced before.
- 8:56SPEAKER_01: So you can see like the data that Mimatron is is uh trained on, you can see cleaning pipeline, you can see the setup and so on and so forth.
- 9:04SPEAKER_01: So I think that's really, really awesome for open source.
- 9:08SPEAKER_01: But what I want to leave you guys here today is like with there's this predominant narrative that open source is the answer.
- 9:14SPEAKER_01: However, you know, thinking critically about that, how can it be the answer when it doesn't have a business model?
- 9:21SPEAKER_01: So you have sort of the same problems and open source, open way type of AI models as you do in open source software, except they're like gajillions of dollars worse, right?
- 9:34SPEAKER_01: Because it takes you know$10 million,$100 million, you know,$500 million eventually, right, to train sort of something on the frontier.
- 9:44SPEAKER_01: So who makes that sustainable, right?
- 9:48SPEAKER_01: Um and so my answer to that question is that decentralized AI training is actually better than open source models for the reason that it financializes those things and gives them a sustainable way of making money.
- 10:06SPEAKER_01: And here's how that works: a bunch of people come together, you might just own a GPU, you might own some capital that rents a GPU, and you participate in a decentralized training of a big competitive or at least commercially viable model, and you own some of the weights, right?
- 10:23SPEAKER_01: So this is the idea is um unextractability.
- 10:27SPEAKER_01: These models, they're not owned by a single party, and actually they're not open weight models.
- 10:33SPEAKER_01: And in this case, that's a good thing, because what it means is that for someone to be able to uh get the output of this model, they need to go pay the network.
- 10:43SPEAKER_01: So the network has and the model has a has a business model.
- 10:46SPEAKER_01: Now we're talking, Alex, about, and this is like old numbers now because I looked this up like over a month ago, but you know, about a month ago, I was estimating that uh on a on a five-year kind of horizon, and this is when the run rate of foundation model inference was something like$60 billion, which has now probably grown to over like a hundred billion since then because anthropic put in new numbers, right?
- 11:12SPEAKER_01: But if you kind of like extrapolate the growth like from a month ago, you know, you you you come to a market that's like a trillion dollars of inference revenue over five years, and now a month later, maybe it's even maybe it's like a trillion and a half, you know, or or whatever it is, right?
- 11:28SPEAKER_01: We can we could do the math again.
- 11:29SPEAKER_01: But what I'm saying is there's no opportunity in that market for open models, right?
- 11:34SPEAKER_01: Because open model inference is not monetized, it's being used kind of locally, and I think that's an important function.
- 11:41SPEAKER_01: But there's also another segment of the market that wants to pay for really, really great models that they also own, that you feel good contributing your data to because you know that you are partially an owner of the revenue of this model.
- 11:57SPEAKER_01: And you know, these decentralized models, if they live in the network, they also capture the associated revenue from people inferencing them.
- 12:06SPEAKER_01: So if you if you look at this like trillion dollar five-year revenue opportunity, you capture, you know, 5%, 10%, you know, you're talking a$50 to$100 billion kind of opportunity.
- 12:18SPEAKER_00: Yeah, I I thought a lot about this issue because you know I spent a lot of time in infrastructure like looking at hard hard hardware and data centers.
- 12:24SPEAKER_00: And so when I've been seeing these headlines from OpenAI making like trillion dollar data center commitments, right?
- 12:30SPEAKER_00: You start getting really worried about what's gonna happen when those dominoes fall apart.
- 12:35SPEAKER_00: But you also, and I was just talking to a friend this morning, we were making some content for uh the Corbett's YouTube channel, and I was talking about how yes, open source models are super powerful and they give you a lot of control, but you still need the underlying infra.
- 12:47SPEAKER_00: Like you still have to, I mean, there are companies paying hundreds of millions of dollars for just like what are those superpods from NVIDIA, right?
- 12:56SPEAKER_00: Like they're just rolling them out because they have you have to have that kind of capital to really train at scale.
- 13:01SPEAKER_00: Um, and so this these networks you're talking about are kind of an answer to that problem.
- 13:06SPEAKER_00: I do think that like people are monetizing open models in different ways, right?
- 13:10SPEAKER_00: You have like open router and things like that, but if you're gonna try to create an open source model, you need significant compute, and that only comes with like tons of capital.
- 13:21SPEAKER_00: And that's a huge barrier to entry.
- 13:24SPEAKER_00: And so these networks are a are an answer to that.
- 13:27SPEAKER_00: And I've been thinking about this the similarities, I like this is the convergence I've been kind of waiting for for crypto and like AI, where it actually might work and it might have a really net positive effect for humanity, is these network effects that could get created around an incentivized network and data and training.
- 13:48SPEAKER_00: It'll be interesting to see how it plays out.
- 13:50SPEAKER_00: Another thing that's really interesting is going back to my comment earlier about open source kind of having its moment where people are realizing they need to start having more control and just in general and ownership over their intelligence stack.
- 14:01SPEAKER_00: USV had um a really cool post uh about this thing they call like the Rebel Alliance.
- 14:06SPEAKER_00: I don't know if you saw it, but it's their it's like their thesis, right?
- 14:09SPEAKER_00: And it's exactly crypto to me.
- 14:11SPEAKER_00: Like when I see it, I'm like, oh yeah, DeFi, you know.
- 14:15SPEAKER_00: Um, you know, it's a composable open stack of technology that, you know, with like certain specialists that can do good things really well around specific kinds of strategies or whatever.
- 14:25SPEAKER_00: And it just feels similar to me when you look at it.
- 14:28SPEAKER_00: It's not exactly the same, but I think I wonder if like with these networks being successful that you're talking about, you'll start to see the same kind of speed of the market moving around this stuff.
- 14:41SPEAKER_00: Like, I don't like something, I don't know exactly what the shape of it will be, but it's something like you know, the DeFi wars where like all this TVL would like flow over here all of a sudden.
- 14:49SPEAKER_00: Like, I I wonder if it'll be very similar to that.
- 14:51SPEAKER_00: So I'll pause there.
- 14:52SPEAKER_00: I have some other questions for you, but uh I wonder if you've like had that thought or it just opinions on that, like the similarities and the kind of overlap.
- 14:59SPEAKER_01: Yeah, no, I mean, I mean, that's exactly kind of in the abstract, the way that like the way that some decentralized AI is approaching like winning the AI game, right, is is is in the abstract very similar to some of the processes that we've seen in crypto.
- 15:16SPEAKER_01: And essentially, um, you know, crypto and blockchains, you know, they're kind of making the case that you have this like coordination software.
- 15:26SPEAKER_01: And if you can coordinate like the long tail of people, then you can you can beat like decentralized incumbents in various ways, right?
- 15:34SPEAKER_01: So this is the same sort of idea here is that okay, you have these large, um, powerful, well capitalized, like better capitalized than most companies in the world, AI labs, and they have completely cornered the market on high-end compute for their own purposes.
- 15:53SPEAKER_01: Yep.
- 15:53SPEAKER_01: Their own purposes is being locked into a race, it's it's sucking up everybody's data, it's competing with each other, it's jacking up prices, and it's ultimately like monopoly, right?
- 16:04SPEAKER_01: And centralization.
- 16:05SPEAKER_01: And what you know, decentralized AI is saying is that you know there's a there's an Achilles heel in that strategy, which is that it this whole strategy like makes this assumption that like everyone needs this high-end compute.
- 16:21SPEAKER_01: But actually, you know, people are smart, and if they can go and optimize the algorithms to work on other kinds of compute, they can unlock the long tail.
- 16:31SPEAKER_01: And you know, I've seen I've seen research with some very interesting graphics that say, like, look, if you kind of pool together all of the Mac, you know, the Apple silicon and all the 4090s in the world right now, like that is like much bigger than the biggest data center that any company has.
- 16:51SPEAKER_01: And if you can actually have an algorithm that trades on that, you can train it, you can actually train a bigger model.
- 16:57SPEAKER_01: The problem is just coordination, the problem is incentives, the problem is getting people together into that position.
- 17:05SPEAKER_01: And this is why crypto, and I should say, you know, maybe like token incentives in particular, right, are kind of a very powerful ingredient of decentralized AI because they are the coordination layer that helps people do this.
- 17:18SPEAKER_01: Like we have networks today.
- 17:19SPEAKER_01: I have on my screen right now, I'm I'm literally like the GPU that I'm on on this Mac is also in a decentralized network that is trading a model and earning tokens, right?
- 17:31SPEAKER_01: And that to give them credit, that's the iota trader from Macrocosmos, right?
- 17:36SPEAKER_01: Let's go.
- 17:36SPEAKER_01: You know, so this is like possible, it's just it just needs to it needs to kind of get out there.
- 17:42SPEAKER_01: And of course, there's all these like technical details we can talk about, but what's actually possible, what's the state of the art, like blah, blah, blah.
- 17:49SPEAKER_01: Uh, but uh, but let me like finish with one thought here, yeah, which is that the journey that we've gone through over the last two years is sort of from like decentralized training is impossible to okay, it's possible, but it's you know, it's like super expensive, to actually realizing from data that it's actually not it might actually be cheaper, right?
- 18:14SPEAKER_01: The the reason is that if you're training in a data center, you have like half of your total cost of ownership of the data center is like cooling and maintenance facilities and maintenance, all the stuff.
- 18:24SPEAKER_01: And when you train in a swarm on on these on Macs, let's say, right, you just don't have those costs, right?
- 18:31SPEAKER_01: So as long as you can scale, as long as you could coordinate the compute, you actually save a bunch of money on you know, sunk CapEx, you save a bunch of money on facilities maintenance, you sit you you don't have cooling, right?
- 18:42SPEAKER_01: And so it actually turns out that that decentralized training could be cheaper.
- 18:47SPEAKER_01: We have um there's a paper from Gradient Network which shows that reinforcement learning is like embarrassingly parallel, and when you it's like a perfect fit for a decentralized network that lowers its costs 10 times.
- 18:59SPEAKER_01: There's a paper from Zero Gravity rate to the same effect.
- 19:02SPEAKER_01: And so people are like, okay, fine, maybe it's cheaper under circumcircumstances, but it's like way slower.
- 19:08SPEAKER_01: You're like, well, if it's cheaper, then you can still like make the unit economics work if it's like a little bit slower.
- 19:14SPEAKER_01: And it is a little bit slower today, as far as like what I've seen.
- 19:18SPEAKER_01: But like, look, we're in version zero.
- 19:20SPEAKER_01: Like, we're gonna be optimizing the heck out of these things.
- 19:22SPEAKER_01: Hardware is getting better on the on the home desktop, right?
- 19:25SPEAKER_01: Like, there's all these reasons why it's actually not going to be slower.
- 19:29SPEAKER_01: And so now they're like, okay, fine, it works, it's cheaper, it's maybe not slower, but it'll never scale, right?
- 19:37SPEAKER_01: Like that this is the stage that where we are.
- 19:40SPEAKER_01: Like it'll never scale now.
- 19:41SPEAKER_01: And I'm here to tell you that here's my like bold prediction for a show.
- 19:46SPEAKER_01: Next six to nine months, we're going to train and like a four to five hundred billion parameter model on decentralized networks, you know, Nematron size, like super exciting.
- 19:58SPEAKER_01: It's gonna happen, right?
- 20:00SPEAKER_01: And once once, once you get there, like I think you're in the in the zone of commercial viability for decentralized models, and and we'll see.
- 20:08SPEAKER_00: It's perfectly timed your prediction for the collapse of the current market.
- 20:12SPEAKER_00: Because, you know, I'm joking, but I I've been saying this on the show and in private that you know it does feel very bubbly out there, and with in and there's a lot of debt and not and like the revenue has to match exactly for them not to like have a problem, right?
- 20:28SPEAKER_00: And I don't know the details like the intricacies of the business, but like back in the napkin, you're like, man, this is kind of crazy.
- 20:33SPEAKER_00: Um, and the one thing that I I've been hypothesizing that could like kick that first domino is something like what you're talking about, like a different kind of AI that's way cheaper, doesn't require all the GPUs.
- 20:45SPEAKER_00: It's gonna hit the market and take the market by storm because it's performant and cheap, and all these enterprises are gonna want to use it, and the market will react instantly, right?
- 20:55SPEAKER_00: Like it's gonna react very fast.
- 20:57SPEAKER_00: Like Deep Seek was just like the canary in the coal mine.
- 21:00SPEAKER_00: And they realized like Deep Seek, yeah, it was cheaper and it was good, but like it still needed all the GPUs, right?
- 21:05SPEAKER_00: You still need all that infrastructure.
- 21:07SPEAKER_00: Yeah, but something that doesn't need that, maybe it's like cognitive AI, maybe it's a decentralized training network like what we were talking about, but something like that comes and it blows up the whole space.
- 21:16SPEAKER_01: Yeah.
- 21:17SPEAKER_01: Look, I think you're right.
- 21:19SPEAKER_01: Uh let me let me try to paint you a picture of how I think it would it might go.
- 21:23SPEAKER_01: So um, you know, today when people talk about the Fable incident and we need alternatives, like like out there in the broader world, like people's answer is open models.
- 21:36SPEAKER_01: Like decentralized AI or decentralized AI training is not, broadly speaking, a narrative, despite my trying and getting a bunch of op-eds into different places, and even an editorial letter to The Economist this week, right?
- 21:51SPEAKER_01: But that is not a narrative today.
- 21:53SPEAKER_01: Now, what makes that a narrative?
- 21:55SPEAKER_01: I think what makes it a narrative is when you know, one day, you know.
- 22:00SPEAKER_01: A product comes out and they say, Look, guys, you want to counteract the centralizing power of AI and Dario Amade's Anthropic.
- 22:08SPEAKER_01: Join this GPU network, contribute your GPU cycles.
- 22:13SPEAKER_01: You can use your H100, or you can use your 4090, or maybe you can even use your MacBook Mac Studio at home, bring whatever you can, and suddenly over a weekend, this process crowdsources like, I don't know,$100 million worth of compute.
- 22:35SPEAKER_01: This is not crazy.
- 22:36SPEAKER_01: Like we've seen ICOs raise billions of dollars.
- 22:39SPEAKER_01: We've seen like there's a lot of people with home GPU hardware these days.
- 22:43SPEAKER_01: There's a lot of prosumers, there's a lot of enthusiasts.
- 22:45SPEAKER_01: Apple's even MacBook Pros have GPUs that are becoming extremely.
- 22:50SPEAKER_00: We know they'll run the software.
- 22:51SPEAKER_00: We know they'll run the software, right?
- 22:53SPEAKER_00: Like we've seen this play out many times in crypto, right?
- 22:56SPEAKER_00: Like people installing open source software to run a validator.
- 22:59SPEAKER_00: Or like Helium, right?
- 23:00SPEAKER_00: Like how many nodes were out there?
- 23:01SPEAKER_00: Like millions at one time.
- 23:02SPEAKER_01: Oh absolutely.
- 23:04SPEAKER_01: Definitely millions, right?
- 23:06SPEAKER_01: Yeah.
- 23:06SPEAKER_01: And so and so over a weekend, you suddenly get this phenomenon where, like, hey, like if I want to train a commercially viable size model, hundreds of billions of parameters, I can go and kind of like propose to a network that maybe like compute comes together.
- 23:21SPEAKER_01: Boom, I have this hundred million dollar equivalent, like GPU equivalent fundraise.
- 23:27SPEAKER_01: I mean, that sounds like what we sort of went through during the ICO times, during you know, token sale times, right?
- 23:34SPEAKER_01: And I think that might be a huge signal.
- 23:37SPEAKER_01: Like when people realize like if I want to train a large competitive model, I don't actually have to work at a big lab, right?
- 23:46SPEAKER_01: And I don't actually necessarily have to own a giant data center with expensive hardware.
- 23:51SPEAKER_01: All I got to do is I gotta come to this network with a phenomenal idea and basically have these people believe in the idea and kind of want to speculatively back it.
- 24:02SPEAKER_01: And then over a weekend I could get$100 million worth of compute.
- 24:05SPEAKER_01: Like that signal is I think is just game-changing.
- 24:10SPEAKER_00: So how do how do you so let's talk?
- 24:11SPEAKER_00: You mentioned privacy earlier, and this is the thing that keeps coming up in the back of my mind, you know, when we talk about putting the data on this network.
- 24:17SPEAKER_00: Like, how do we handle that problem?
- 24:19SPEAKER_00: Because it's it's not an easy problem, right?
- 24:21SPEAKER_00: To do to maintain.
- 24:22SPEAKER_00: And because this is the one of the big concerns too with this narrative and why people are talking about open open models, right?
- 24:27SPEAKER_00: It's like, I want to protect my data, I want to train it on, you know, maybe in-house.
- 24:32SPEAKER_00: So if they're big enough, they'll make the investment in the GPUs and try and scrape together whatever they can to do that training.
- 24:37SPEAKER_00: How do we protect data of these companies?
- 24:40SPEAKER_00: I just feel like that's going to be a barrier to adoption.
- 24:43SPEAKER_01: Yeah.
- 24:43SPEAKER_00: I mean, or are there specific use cases you think you can bootstrap the market without solving that problem?
- 24:48SPEAKER_01: Well, certainly the latter, because I don't think privacy is like a requirement for like decentralized AI to create something, create many different kinds of things that could be very useful, or like open, you know, open models or or just like smart models, right?
- 25:03SPEAKER_01: And they don't have to address the the question of privacy head-on.
- 25:06SPEAKER_01: The question of privacy is also very nuanced, right?
- 25:09SPEAKER_01: Like what kind of privacy do you mean?
- 25:11SPEAKER_01: So if you're like uh you know an enterprise and you have like enterprise critical proprietary data, you absolutely can't send that to anybody.
- 25:20SPEAKER_01: Um then I think the centralized AI still helps because the centralized AI will create foundation models that are open too.
- 25:27SPEAKER_01: And you'll be able to take those foundation models and you'll be able to fine-tune them in ways that are advantageous to you, and you don't have to necessarily you can do that in-house, or you know, there's schemes out there that allow kind of more private training as well.
- 25:42SPEAKER_01: I think they're slow and early, but people do do this kind of thing with TEs, right?
- 25:48SPEAKER_01: So that's one thing to say.
- 25:49SPEAKER_01: The the other thing to say is like there's also just user privacy, right?
- 25:54SPEAKER_01: So we have we live in a time where Venice has made quite a case for um an alternative product here where people kind of people's identity are protected from you know from the from the model issuers.
- 26:09SPEAKER_01: Now, if you think about how that actually works, right?
- 26:11SPEAKER_01: Like I can't like someone has to still go and engage the anthropic API.
- 26:16SPEAKER_01: And I think that's Venice, right?
- 26:18SPEAKER_01: So if I'm interacting through Venice, but I'm putting my personal information into the chat, like I'm actually not protecting my privacy necessarily.
- 26:28SPEAKER_01: Again, there's a lot of nuances about what gets protected.
- 26:32SPEAKER_01: If you need a strong level of user privacy, then unfortunately you're gonna have to get a local model, right?
- 26:37SPEAKER_01: You just you just have to do that.
- 26:39SPEAKER_01: If you if that's this if that's the bar of privacy that you need, like you're gonna have to get a local model.
- 26:44SPEAKER_01: The good news is that hardware at home is becoming much more powerful.
- 26:50SPEAKER_01: I don't know if it's becoming cheaper necessarily, but but it's becoming much more powerful, right?
- 26:56SPEAKER_01: You know, you can get a DGX Spark, you can get a 49, you get a little cluster of 4090s from like tiny box or not a price that most people you know can afford.
- 27:06SPEAKER_01: But if you really need that privacy or if you're in a business context, you know, that's a that's an alternative here.
- 27:11SPEAKER_01: Again, Apple hardware is getting better.
- 27:14SPEAKER_01: We have these rumors about uh you know the 1.5 gigabytes of of unified uh fast memory and the I think it's like the M7, right?
- 27:26SPEAKER_01: This came out this week, right?
- 27:28SPEAKER_01: So so there's a lot, you know, there's this hardware floor that keeps rising and enabling people to locally inference bigger and bigger and smarter and smarter models.
- 27:37SPEAKER_01: And then from the software side, you have kind of like the open models.
- 27:42SPEAKER_01: Now, people are saying that some of the recent models here, they're almost as good as Fable and they're open weight, right?
- 27:48SPEAKER_01: So now if you have enough hardware today, you can do this.
- 27:51SPEAKER_01: Now five years from now, you're gonna have even more hardware, you're gonna have even smarter open models.
- 27:56SPEAKER_01: So you'll be able to do that.
- 27:58SPEAKER_01: And then I think there's also like when we get to a point where the models are just so big and so smart that they have to be distributed, hosted in a network.
- 28:08SPEAKER_01: I think there's still privacy technology that can create.
- 28:11SPEAKER_01: You know, there's like FHE, there's TEs, there's like all these things.
- 28:15SPEAKER_01: Today, those things are very slow.
- 28:16SPEAKER_01: Today, those are not going to be like mostly practical, practically available to most users, right?
- 28:21SPEAKER_01: But over time, I think it gets better and there's hope.
- 28:25SPEAKER_01: So I think there's solutions today, and I think there's different levels of privacy, different customers of privacy.
- 28:31SPEAKER_01: Privacy is not a requirement to make DAI kill it.
- 28:35SPEAKER_00: You know, I'm I think you're probably right about that.
- 28:38SPEAKER_00: I think there's got to be a beach head with some use cases that they that these things will take off.
- 28:43SPEAKER_00: But I also think just the excitement about the scalability, like once you hit that thing, that space where it's like commercially viable, a ton of people will flock to it.
- 28:51SPEAKER_00: I mean, like I sit here and talk about privacy and how enterprises care about it, and they do, but then we saw OpenClaw just like rip right when it at launch, right?
- 28:59SPEAKER_00: It had tons of security problems.
- 29:01SPEAKER_00: Nobody cared because it was so useful.
- 29:04SPEAKER_00: They just used it, right?
- 29:05SPEAKER_00: Like it got product market fit like out of the gate, and it was just such a novel, game-changing experience.
- 29:10SPEAKER_00: And this could be at that level when you talk about the magic of being able to just like broadcast and get like a hundred million dollars worth of compute.
- 29:19SPEAKER_00: I want to briefly just have you touch on some of the companies that you want to highlight that you guys have invested in or that you just like in this space, because like not a lot of people.
- 29:30SPEAKER_00: I just feel like you've always had an edge, like the alpha on this stuff.
- 29:34SPEAKER_00: I can't remember the other company that you sent me, but it was when they were just Alexander Long's company.
- 29:38SPEAKER_00: What's his what's what are they called again?
- 29:40SPEAKER_00: Yeah, Pluralis.
- 29:41SPEAKER_00: You guys backed Pluralis when it was just him in like a research paper, right?
- 29:47SPEAKER_00: Like, and and they've done some really amazing work.
- 29:50SPEAKER_00: So give us the the list of like very briefly of who who some of these guys are that you got that you're excited about.
- 29:57SPEAKER_01: Yeah, so I think you know, let's talk about training, let's talk about inference.
- 30:02SPEAKER_01: So in training, I think Alex's team at Pluralis has done just incredible, incredible work.
- 30:08SPEAKER_01: The thing that differentiates their work is that they're really the first team to be able to create a training framework that breaks up the model.
- 30:17SPEAKER_01: So, in other words, model parallel training.
- 30:19SPEAKER_01: Like every single decentralized AI training that came before that is a training where still the whole model lives on every node and is therefore limited by you know kind of the hardware of the node.
- 30:34SPEAKER_01: What model parallel training does is we can break up the model so now we can have much smaller hardware requirements for each of the nodes, and the nodes coordinate with each other to do the training.
- 30:45SPEAKER_01: And what Alex's team has done is they've shown some of the first, now multiple model parallel training runs.
- 30:52SPEAKER_01: And now the question is like, can we scale this?
- 30:55SPEAKER_01: Right?
- 30:56SPEAKER_01: Um, Bagel, who you mentioned earlier in the call, you know, they are, I think, the first company to do a decentralized training on not LLMs, but diffusion models.
- 31:09SPEAKER_01: So, in other words, models that generate like images and videos, and what they're what they're seeing is that you can create efficiencies and cost savings by being able to do that.
- 31:18SPEAKER_01: Jensen is actually the first decentralized training company we ever invested in.
- 31:22SPEAKER_01: It was in March of 2022.
- 31:25SPEAKER_01: They were like way earlier than like everybody else on this topic.
- 31:29SPEAKER_01: And their kind of approach was around verifiability.
- 31:32SPEAKER_01: Now, today we, you know, they they sort of evolved into um, you know, a more uh kind of fulsome suite of decentralized AI services.
- 31:42SPEAKER_01: But on that topic of verifiability, I think this is a very prescient moment because we're seeing a lot of discourse about things like sleeper agents or models that even though they're open, they might be backdoored or they might send your information somewhere, or you know, they have secret compartment, like you don't have any guarantees, in other words, of like this is the scary thing with the with the Chinese models, right?
- 32:08SPEAKER_01: Like you know, like so people say, but but the the the common need, right, is like a verification, it's it's like a robust verification that the training process has been done correctly.
- 32:19SPEAKER_01: Jensen is the you know, kind of one of the one of the primary researchers of this question in the world and how that might work, and how that could involve you know zero-knowledge proofs, and how that could involve kind of deterministic processes that would be needed to like replicate these trainings and check them, right?
- 32:38SPEAKER_01: And so I think you'll see uh some of those things.
- 32:42SPEAKER_01: Prime Intellect, right, has again just raised$130 million series A.
- 32:48SPEAKER_01: And it's really about democratizing and opening the training stack.
- 32:52SPEAKER_01: They were actually the first company to demonstrate uh decentralized trading of a it was first, it was a 10 billion parameter model, and then it was, I believe it was a 100 billion parameter like later, using a kind of data parallel approach.
- 33:05SPEAKER_01: And really, just these guys like put an existence proof into the world that, like, hey, this stuff is actually possible, right?
- 33:12SPEAKER_01: So I'm super proud and excited to be like a prime intellect investor just for that reason, for them just breaking this record in a way, right?
- 33:22SPEAKER_01: Of like showing the world that this is possible.
- 33:25SPEAKER_01: So I talked a lot about like the training side, right?
- 33:27SPEAKER_01: What we're also seeing now in the market is a lot of people coming back to the inference side and saying, like, look, is the centralized inference actually good?
- 33:37SPEAKER_01: Why might it be actually good?
- 33:39SPEAKER_01: Well, decentralized inference could be good if it unlocks supply where we usually like don't have supply.
- 33:45SPEAKER_01: It could be good because it's cheaper, or it could be like better distributed in some in some ways, right?
- 33:51SPEAKER_01: But really, like it comes down to the question of like in a decentralized network, can we do things more cheaply?
- 33:58SPEAKER_01: And theoretically, the answer is yes.
- 34:00SPEAKER_01: Theoretically, it's like, well, 4090 is a cheaper GPU than an H100, and it also is cheaper to run.
- 34:07SPEAKER_01: And so if you can take the same model, you know, that you run here, and you can like adjust an algorithm to run it here, like then the unit economics becomes better, right?
- 34:19SPEAKER_01: And so the big question for companies coming to market right now, like is sharded inference on decentralized networks like really, really possible?
- 34:29SPEAKER_01: And then there's also decentralized networks that just run high-end hardware and you know do the inference the old way, but using like token incentives and things like that.
- 34:38SPEAKER_01: And so I'm very interested in this like sharded inference piece.
- 34:42SPEAKER_01: In a comms set of 76 companies, Alex, there are six companies that work on sharded inference.
- 34:48SPEAKER_00: You know, it's really interesting.
- 34:50SPEAKER_00: You're you're connecting another dot for me that I that I was thinking about earlier this morning.
- 34:54SPEAKER_00: I heard that anthropic is essentially removing Fable 5 from the subscription plans.
- 35:02SPEAKER_00: So like your max, you're you can't you can't use it.
- 35:04SPEAKER_00: You have to use it only via API.
- 35:06SPEAKER_00: And the reason they're doing it is because they don't have enough compute.
- 35:11SPEAKER_00: Like they literally are like, we when we have more compute, we'll we'll put it back.
- 35:15SPEAKER_00: But for now, you got to pay more and use the you know, the per paper use model.
- 35:19SPEAKER_00: What you're talking about could potentially be an answer to that.
- 35:22SPEAKER_00: And and maybe maybe that's the catalyst.
- 35:25SPEAKER_00: Maybe that's another catalyst here that like pushes decentralized training and inference to the mainstream is the simple fact that all the resources are consumed, right?
- 35:36SPEAKER_00: Like there's literally not enough for you know, with you've got XAI, Anthropic, and OpenAI just gobbling all of the GPUs and all the power up.
- 35:45SPEAKER_00: What are you maybe they just run out and and there's nothing left?
- 35:49SPEAKER_00: And so then you have to look to these decentralized networks.
- 35:52SPEAKER_00: Yeah.
- 35:52SPEAKER_00: It's kind of crazy to think about, but it this is the most massive infrastructure build out of human history.
- 35:57SPEAKER_01: Yeah, I completely agree, right?
- 35:59SPEAKER_01: I mean, like maybe like a year, year and a half ago, everyone was like, oh, this compute you know, shortage, blah, blah, blah.
- 36:06SPEAKER_01: And like in my mind, I was like, well, people will manufacture more GPUs, there's gonna be more supply.
- 36:11SPEAKER_01: And I think the story has been like, no, the compute remains scarce, the growth of AI has been tremendous, prices are like going up.
- 36:20SPEAKER_01: Even old GPUs are still like super val and more valuable than newer GPUs in some cases, right?
- 36:26SPEAKER_01: Because of because of these like demand effects that are that are happening.
- 36:30SPEAKER_01: And so, you know, someone told me like a year ago that people at Google, I don't know who, just a rumor, I guess, but don't quote me on it.
- 36:41SPEAKER_01: But like, you know, they were kind of thinking like, okay, do we have to go out onto decentralized networks or at least like distribute compute in other ways, like if demand keeps going?
- 36:53SPEAKER_01: They seem to be in the camp that yes.
- 36:55SPEAKER_01: And then more recently this year, Google is actually implementing Di-Loco, which is the first decentralized AI training algorithm that was partially invented at Google in their production processes for like robustness of data centers, right?
- 37:11SPEAKER_01: So you they had this like on their blog you know a couple of months ago.
- 37:14SPEAKER_00: That is out, that is some alpha, dude.
- 37:15SPEAKER_00: I really feel like that's something people aren't thinking about, you know, they're not realizing.
- 37:20SPEAKER_01: So so these advancements that we're making in decentralized AI, 100% are also going to flow into these data centers, but for slightly different purposes, for like robustness, for you know, like redundancy of these processes for distribution.
- 37:40SPEAKER_01: Also, also like Alex, like think about it.
- 37:42SPEAKER_01: You you know, you you like live in a world where now you have all these like different players with different data centers.
- 37:48SPEAKER_01: You have Elon saying, maybe we'll be we're gonna be able to like take the GPUs and all the Tesla fleet, and when you're sitting there charging, you know, your your Tesla is gonna be like participating in a GPU network.
- 38:00SPEAKER_01: He's also building data centers in space.
- 38:02SPEAKER_01: So, in other words, you have all these like disparate sources of GPU capacity.
- 38:07SPEAKER_01: I wonder how you coordinate all of them.
- 38:09SPEAKER_01: Could it be a uh you know, decentralized trading algorithm of some kind, right?
- 38:14SPEAKER_01: So I just I'm very, very bullish, if nothing else, that this technology will be very used.
- 38:21SPEAKER_01: But I think the the you know, kind of the investment opportunity, the commercial opportunity is for some of these companies employing these algorithms to literally create a better dynamic for models.
- 38:33SPEAKER_01: I call them public models.
- 38:34SPEAKER_01: They're publicly owned, they're not necessarily open.
- 38:37SPEAKER_01: You could create open models as well if you wanted to, but these publicly owned models they coordinate the creators, the participants, the owners, the users in a way that everybody shares the revenue, and then you have solved some of these like monopoly effects that you see.
- 38:55SPEAKER_00: We gotta we gotta get you in front of Congress to start talking about this.
- 38:58SPEAKER_00: I'm like, I'm like dead serious.
- 39:00SPEAKER_00: You know, we had uh Amadei was there, like kind of I felt like fear-mongering around open source AI.
- 39:07SPEAKER_00: And I don't think he's like a bad person or anything like that, but I do think he has some incentives to try to not have open source AI running in, you know, uh, because it hurts his margins.
- 39:19SPEAKER_00: And I just want to see, you know, a rational voice that's saying, like, I know actually like open intelligence running on decentralized networks or not, like just basically anything that that is in that stack is important.
- 39:32SPEAKER_00: And I keep thinking about like the these ideas that some politicians are coming up with to just like own huge chunks of anthropic and open AI.
- 39:41SPEAKER_00: Like, how about just make them open source their models?
- 39:44SPEAKER_00: Like make it a public good.
- 39:45SPEAKER_00: Like your idea about public, it's like a public it's a road, right?
- 39:49SPEAKER_00: It's like it's like everyone should have access to it.
- 39:51SPEAKER_00: And you you can pay tolls for sure, but it should be publicly owned.
- 39:56SPEAKER_00: Some of it should, anyway.
- 39:57SPEAKER_00: I'm not against like for-profit companies, but if you're gonna go down that that route, you know, don't own 50% of the company, just open source the models or somehow try to give ownership of some of the infrastructure to people.
- 40:08SPEAKER_01: Yeah, I mean, look, it's tough.
- 40:10SPEAKER_01: It's like we have to balance, on the one hand, the idea that these this is technology is like so important that it has to have checks and balances in a public sense.
- 40:23SPEAKER_01: And on the other hand, you know, we live in a in a capitalist world where these things also require a lot of capital to create and and sustain, right?
- 40:33SPEAKER_01: And so how do you find the balance between those two things?
- 40:36SPEAKER_01: To me, the answer is public AI models on open decentralized networks, they have the best of both worlds.
- 40:44SPEAKER_01: They have good alignment, they have good um transparency, right?
- 40:50SPEAKER_01: They can give you guarantees, you can produce open models, you can produce proprietary models for businesses, right?
- 40:57SPEAKER_01: And then on the other hand, they are also valuable.
- 40:59SPEAKER_01: They're financialized, they create revenue, right?
- 41:02SPEAKER_01: They're sustainable.
- 41:03SPEAKER_01: They're not just like open source, open weight, where we're hoping someone like NVIDIA will come in and put in$20 billion to fund the thing.
- 41:11SPEAKER_01: And that's in many different areas, like that's really like the crypto theme, right?
- 41:17SPEAKER_01: Of of why blockchains and decentralized networks are kind of a better economic choice that balances those two sides of things.
- 41:28SPEAKER_01: And you know, I think our reality today is we are not in that operating system.
- 41:32SPEAKER_01: We're in the anthropic, open AI, big tech, etc., big defense company, et cetera, like operating system.
- 41:39SPEAKER_01: But we really should be in something a little bit more public, publicly owned.
- 41:44SPEAKER_00: And I think we'll get there.
- 41:45SPEAKER_00: I think the tide is turning.
- 41:46SPEAKER_00: Listen, uh, we're at we're at the end of the show here.
- 41:48SPEAKER_00: We're top top of the show.
- 41:49SPEAKER_00: I always ask this what have I not asked you that I should have asked or that you want to talk about?
- 41:54SPEAKER_01: Well, ask me how crypto's going.
- 41:57SPEAKER_00: Do we have to talk about that?
- 42:00SPEAKER_00: How's crypto going?
- 42:01SPEAKER_00: Great.
- 42:02SPEAKER_00: Doesn't feel great to me.
- 42:03SPEAKER_00: It's awesome.
- 42:04SPEAKER_01: I think crypto's going great.
- 42:05SPEAKER_01: I think I know I know we've gone through like a tough like couple of years here.
- 42:10SPEAKER_01: We're going through a transitional period.
- 42:12SPEAKER_01: We're going through a maturation process that is like very, very painful for some of the early participants and founders of crypto and investors of crypto, some of which are not here anymore.
- 42:23SPEAKER_00: We're here.
- 42:24SPEAKER_01: I, however, uh we're here.
- 42:28SPEAKER_01: But I'm I'm very bullish, and the reason I'm very bullish is I do see that we actually have more crypto adoption than ever before in enterprise, in institutions.
- 42:38SPEAKER_01: We have legislation.
- 42:40SPEAKER_01: We might have, we might, we might, if we're lucky, we might get a little bit more legislation.
- 42:44SPEAKER_01: We have kind of the government sort of supporting this process.
- 42:48SPEAKER_01: Asset tokenization is happening, stable coins are happening, RWA is happening.
- 42:54SPEAKER_01: And if you want some of that, you know, early innovative craziness that you know we've come to love at crypto so much, go to check out decentralized AI.
- 43:02SPEAKER_01: There's some really interesting projects that are very early stage and whose opportunities are very big because you know AI is big.
- 43:11SPEAKER_01: And I don't think I'm feeling very bad about crypto at all.
- 43:15SPEAKER_01: I think the base founders are building right now.
- 43:18SPEAKER_00: I agree.
- 43:19SPEAKER_00: And I think if you're feeling bad about crypto and you're in crypto, go over to Baxis, I'm an investor, or go over to uh uh gotcha sports.
- 43:27SPEAKER_00: I'm not an investor, but I just had him on the podcast and just play the gotcha machines.
- 43:31SPEAKER_00: Like just it'd make you it's actually so have you done that?
- 43:35SPEAKER_00: It's so much fun.
- 43:36SPEAKER_00: They're uh they're hilarious.
- 43:38SPEAKER_00: Like it's like totally addicting.
- 43:39SPEAKER_00: You like you like hit a button, you put a hundred bucks in, you get like a random bottle, or on gotcha sports, you can get like a baseball card back.
- 43:46SPEAKER_00: I I had um I had the the um founder of Baxis on the show, Todd, and we did it live, like his gotcha machine live, and I won like a thousand dollar bottle of like bourbon.
- 43:58SPEAKER_00: It was nuts.
- 43:58SPEAKER_00: And then I did the same thing with the gotcha sports guy, and the gotcha sports guy's like bootstrapped, been in business for a couple months, and has done like a couple hundred K in revenue, like out of the gate, and it's just like people flipping baseball cards, you know.
- 44:11SPEAKER_00: But they're RWAs, they have the similar, he works with collector crypto, and so he's like got them in a vault and everything like that.
- 44:16SPEAKER_00: And it just feels like a real thing instead of like you know, high NFTs kind of didn't pan out, like really, in my opinion.
- 44:24SPEAKER_00: But this is like there's a real market for this already, you know, people trade and collect these assets, and it was fun.
- 44:30SPEAKER_00: So, anyway, my my point is if you're feeling down on crypto, try to go have some fun, touch some grass, we'll get we'll get through it.
- 44:37SPEAKER_00: And there's a lot of cool things to I think this intersection of AI and blockchains and these incentivization of these networks, there's something real there that's beyond stable coins and trading of assets.
- 44:48SPEAKER_00: It's gonna be something really meaningful.
- 44:51SPEAKER_00: And it could spur at what you're talking about, a truly publicly owned network for AI.
- 44:58SPEAKER_00: And I think that that's really important.
- 45:00SPEAKER_00: I keep hammering on this.
- 45:01SPEAKER_00: Open intelligence is a human right.
- 45:03SPEAKER_00: The only way you get there is through open source stacks and open networks like this, because the infrastructure problem, that's the big piece where you need to get like$100 million in compute.
- 45:13SPEAKER_00: If you don't have that and you have all the other open source tools, it doesn't matter, right?
- 45:16SPEAKER_00: Like you're just gonna be screwed.
- 45:18SPEAKER_00: Um, but if you can get both, now you can get super intelligence that people can own.
- 45:22SPEAKER_00: And that's that's important.
- 45:24SPEAKER_01: That's what I'm here for, Alex, is super intelligence that I can own.
- 45:28SPEAKER_01: Let's go.
- 45:29SPEAKER_01: Thanks for coming on the show.
- 45:30SPEAKER_01: It's always a pleasure.
- 45:32SPEAKER_01: Thanks for having me.