Latest / Razib Khan's Unsupervised Learning / Nikolai Yakovenko: the Singularity is not here
Transcript
- 0:00SPEAKER_01: This podcast is brought to you by the Albany Public Library Main Branch and the
- 0:04SPEAKER_01: generosity of listeners like you.
- 0:09SPEAKER_01: What is a podcast?
- 0:16SPEAKER_01: God, Daddy, these people talk as much as you do.
- 0:26SPEAKER_01: Razeeb Khan's Unsupervised Learning.
- 0:29SPEAKER_00: Hey, everybody.
- 0:29SPEAKER_00: This is Razeeb with the Unsupervised Learning Podcast.
- 0:32SPEAKER_00: And I'm here to talk to Nikolai Yakovenko.
- 0:36SPEAKER_00: He is a returning guest.
- 0:37SPEAKER_00: I can't even count how many times he's been on the podcast the last couple of years.
- 0:42SPEAKER_00: And the main reason he is on is we often talk about artificial intelligence,
- 0:46SPEAKER_00: AI,
- 0:46SPEAKER_00: AGI,
- 0:48SPEAKER_00: PDU,
- 0:48SPEAKER_00: all of these things that have been kind of in the discourse floating around.
- 0:52SPEAKER_00: And, um, you know, it's just a big deal right now, uh, in my opinion.
- 0:55SPEAKER_00: And therefore I think it's a useful talk to Nikolai as a practitioner in the field.
- 1:00SPEAKER_00: He's been in the field,
- 1:00SPEAKER_00: uh,
- 1:01SPEAKER_00: in various ways for a decade,
- 1:03SPEAKER_00: uh,
- 1:03SPEAKER_00: ex Davidia,
- 1:04SPEAKER_00: uh,
- 1:05SPEAKER_00: you know,
- 1:05SPEAKER_00: way back he was at Google.
- 1:06SPEAKER_00: He was also at Twitter when it was like called Twitter.
- 1:09SPEAKER_00: And, uh, more recently he has been an entrepreneur, um,
- 1:12SPEAKER_00: in the AI as well as crypto fields.
- 1:15SPEAKER_00: You know, he has his hands in a lot of different things.
- 1:18SPEAKER_00: He is right now the CEO of deep news with a Z.
- 1:21SPEAKER_00: It's a really great application that helps summarize the news and keeps you abreast
- 1:25SPEAKER_00: of what's going on in the world.
- 1:27SPEAKER_00: So you'll get it on your phone, download the app or get it on the browser.
- 1:30SPEAKER_00: We'll have the links there.
- 1:31SPEAKER_00: We'll talk a little about, a little bit about what, um,
- 1:35SPEAKER_00: Building deep news has kind of taught Nikolai about AI as a practitioner in the
- 1:40SPEAKER_00: current moment as opposed to the theory of what's going on.
- 1:45SPEAKER_00: Before we get into that, Nikolai, it's been a couple of years.
- 1:49SPEAKER_00: We did do an informational interview.
- 1:51SPEAKER_00: podcast a while ago,
- 1:53SPEAKER_00: but I do have new listeners,
- 1:55SPEAKER_00: believe it or not,
- 1:56SPEAKER_00: so some people might not actually know what AI is and why it's a big deal and why
- 2:01SPEAKER_00: we keep talking about LLM's large language models.
- 2:04SPEAKER_00: Can you just tell the listener real quickly,
- 2:06SPEAKER_00: okay,
- 2:07SPEAKER_00: in the generality,
- 2:08SPEAKER_00: this is AI,
- 2:09SPEAKER_00: and then we have these large language models specifically,
- 2:11SPEAKER_00: and why they became a big deal over the last couple of years,
- 2:15SPEAKER_00: and why they are
- 2:18SPEAKER_00: so much more powerful than the other types of AI that we had had in the past.
- 2:23SPEAKER_00: And yeah, a lot of it has to do with neural networks.
- 2:27SPEAKER_00: A lot of it has to do with unexpected scaling properties, et cetera, et cetera.
- 2:31SPEAKER_00: Let's just get a little bit into that in the beginning so that people know a little beyond the jargon.
- 2:37SPEAKER_03: Yes, I resume.
- 2:39SPEAKER_03: So I guess we'll just say AI is simple in that you're saying,
- 2:46SPEAKER_03: hey,
- 2:46SPEAKER_03: I want the machines to make human-like decisions or to do sort of human-like things.
- 2:53SPEAKER_03: And the biggest sort of insight was that after decades of sort of doing a lot of
- 2:59SPEAKER_03: design and procedurally explaining the problem,
- 3:01SPEAKER_03: that sort of stuff,
- 3:02SPEAKER_03: essentially these were small data problems,
- 3:04SPEAKER_03: right?
- 3:06SPEAKER_03: You remember when big data was a buzzword.
- 3:08SPEAKER_03: Well, it was mostly bullshit.
- 3:09SPEAKER_03: But it turns out that like actually a lot of data,
- 3:13SPEAKER_03: even noisy sort of like medium quality data was the better way to train these AIs
- 3:19SPEAKER_03: the way they're trained.
- 3:20SPEAKER_03: But most most famously,
- 3:21SPEAKER_03: like you said,
- 3:21SPEAKER_03: the LLMs,
- 3:22SPEAKER_03: the large language models is you give it a huge amount of text.
- 3:26SPEAKER_03: Like, do you do some quality checks, whatever, we can get into that.
- 3:29SPEAKER_03: But relatively straightforward,
- 3:31SPEAKER_03: you just sort of take all the decent stuff on the internet,
- 3:33SPEAKER_03: for example,
- 3:34SPEAKER_03: and you train the model on a very simple task of predicting the next word.
- 3:38SPEAKER_03: And some people would even refer to that as unlabeled data, but I mean, it is labeled, essentially.
- 3:44SPEAKER_03: But you're just doing the most rudimentary task,
- 3:46SPEAKER_03: predict the next word,
- 3:48SPEAKER_03: predict the next sentence,
- 3:48SPEAKER_03: that sort of thing.
- 3:50SPEAKER_03: And it turns out that that does scale incredibly well,
- 3:54SPEAKER_03: although some are arguing now that we're running into some scaling issues.
- 3:57SPEAKER_03: So we'll sort of get into that in a second.
- 3:59SPEAKER_03: But basically,
- 4:00SPEAKER_03: it's the culmination of this work where you said,
- 4:03SPEAKER_03: hey,
- 4:03SPEAKER_03: you know,
- 4:04SPEAKER_03: whether it's a chess game and you just predict the next move and the model sort of,
- 4:07SPEAKER_03: you know,
- 4:08SPEAKER_03: basically fits a neural net,
- 4:09SPEAKER_03: which is basically like a universal function approximator.
- 4:12SPEAKER_03: It's something that can sort of model anything, let's just say.
- 4:15SPEAKER_03: Let's not worry about that for too much.
- 4:17SPEAKER_03: Well,
- 4:17SPEAKER_03: it's like,
- 4:17SPEAKER_03: well,
- 4:18SPEAKER_03: that is the best way to train,
- 4:19SPEAKER_03: you know,
- 4:19SPEAKER_03: a robot to play chess,
- 4:22SPEAKER_03: you know,
- 4:22SPEAKER_03: something to play poker,
- 4:24SPEAKER_03: but mostly these language models.
- 4:26SPEAKER_03: Now,
- 4:26SPEAKER_03: the games do have one important caveat,
- 4:28SPEAKER_03: which is sort of the crux of this problem,
- 4:30SPEAKER_03: where,
- 4:31SPEAKER_03: you know,
- 4:32SPEAKER_03: the first really good models for something like chess or poker would learn from,
- 4:36SPEAKER_03: you know,
- 4:36SPEAKER_03: games of good players and learn to imitate them.
- 4:39SPEAKER_03: And believe it or not,
- 4:39SPEAKER_03: you can actually scale it to where they play better than the original games they
- 4:42SPEAKER_03: were imitating.
- 4:43SPEAKER_03: But then they were like, hey, since the...
- 4:45SPEAKER_03: Since it's a constrained game,
- 4:48SPEAKER_03: you can sort of have it play itself,
- 4:50SPEAKER_03: you know,
- 4:50SPEAKER_03: introduce the variants,
- 4:51SPEAKER_03: try different moves,
- 4:52SPEAKER_03: and basically the computers will play each other and you have this like runaway
- 4:54SPEAKER_03: success problem.
- 4:56SPEAKER_03: Now,
- 4:56SPEAKER_03: obviously,
- 4:56SPEAKER_03: as you can imagine,
- 4:58SPEAKER_03: that has not really happened with other models,
- 5:01SPEAKER_03: particularly language.
- 5:02SPEAKER_03: Although we do do some of that now, which I think we'll get into as well.
- 5:07SPEAKER_03: Is that a pretty good start?
- 5:08SPEAKER_00: Yeah, yeah.
- 5:09SPEAKER_00: And so I think one of the...
- 5:14SPEAKER_00: you know, questions or one of the topics that I do want to touch on.
- 5:17SPEAKER_00: And there's been a few things like understanding AI,
- 5:19SPEAKER_00: I think wrote,
- 5:20SPEAKER_00: uh,
- 5:20SPEAKER_00: that substack,
- 5:21SPEAKER_00: uh,
- 5:21SPEAKER_00: by Tim Lee,
- 5:22SPEAKER_00: uh,
- 5:23SPEAKER_00: wrote about,
- 5:23SPEAKER_00: uh,
- 5:24SPEAKER_00: you know,
- 5:25SPEAKER_00: the kind of like the second wave of connectionism that kind of started around 2010.
- 5:30SPEAKER_00: Um,
- 5:30SPEAKER_00: so just,
- 5:31SPEAKER_00: you know,
- 5:31SPEAKER_00: I read,
- 5:32SPEAKER_00: uh,
- 5:32SPEAKER_00: some stuff about connectionism and neural networks,
- 5:35SPEAKER_00: I think like early two thousands.
- 5:37SPEAKER_00: And, um,
- 5:38SPEAKER_00: I think it was by Gary Marcus, who is right now kind of has a second career as an AI naysayer.
- 5:44SPEAKER_00: Yeah, he's got a – he's the number one hater at the moment.
- 5:48SPEAKER_00: He is the hater.
- 5:48SPEAKER_00: He is the anti-pope.
- 5:50SPEAKER_00: And he kind of like pooh-poohed the future of neural networks.
- 5:56SPEAKER_00: They had a big – they had like a period in the 70s or 80s, I guess, that they were a big deal.
- 6:00SPEAKER_00: And then they kind of like ran into problems.
- 6:03SPEAKER_00: And then they,
- 6:04SPEAKER_00: you know,
- 6:04SPEAKER_00: deep learning,
- 6:05SPEAKER_00: another term associated with these neural networks,
- 6:07SPEAKER_00: these layers and stuff like that.
- 6:09SPEAKER_00: This all kind of comes to the fore in the last decade and then LLMs.
- 6:14SPEAKER_00: And I guess from an outsider's perspective, one of the things that is notable is it seems to me how –
- 6:21SPEAKER_00: How unpredictable this has been insofar as you read what people said 10 years ago
- 6:27SPEAKER_00: or even five years ago,
- 6:28SPEAKER_00: they're very,
- 6:29SPEAKER_00: very far off compared to where they are right now,
- 6:32SPEAKER_00: I feel.
- 6:33SPEAKER_00: And, you know, for example, NVIDIA famously went all in on GPUs.
- 6:40SPEAKER_00: And that's why they're a big deal because artificial intelligence, really compute intensive.
- 6:45SPEAKER_00: It runs on GPUs and Intel.
- 6:48SPEAKER_00: I'm reading articles about how Intel,
- 6:50SPEAKER_00: the national champion a decade ago,
- 6:52SPEAKER_00: is possibly going to get a government bailout because they're not very –
- 6:57SPEAKER_00: optimized to manufacture gpus uh so there was no anticipation a decade ago even in
- 7:03SPEAKER_00: a place like intel where okay like they're supposedly smart people they're
- 7:06SPEAKER_00: supposedly smart executives but they had no idea um how did this i mean am i am i
- 7:11SPEAKER_00: wrong in terms of the lack of foresight uh did some people internally kind of see
- 7:16SPEAKER_00: it and it didn't just get out like give me a give me a um you know clarity on that
- 7:21SPEAKER_03: Well, it's very complicated.
- 7:22SPEAKER_03: I think it's tying into sort of too many things to address, to be honest.
- 7:25SPEAKER_03: But let's try a couple.
- 7:26SPEAKER_03: I mean, so first of all, yeah, it's hard to predict the future.
- 7:30SPEAKER_03: I think we know that when people get it right, but it's kind of random.
- 7:33SPEAKER_03: I mean, I think I think of it a little more as a trader.
- 7:36SPEAKER_03: where you can look at things like momentum and extrapolation, but you can't extrapolate very far.
- 7:41SPEAKER_03: So even take something like the weather predicting hurricanes,
- 7:44SPEAKER_03: which by the way,
- 7:45SPEAKER_03: that has gotten way better than 20 years ago by like a tremendous amount.
- 7:49SPEAKER_03: But just take that problem, right?
- 7:51SPEAKER_03: You can sort of,
- 7:53SPEAKER_03: okay,
- 7:53SPEAKER_03: if the hurricane's building,
- 7:54SPEAKER_03: you can say it might get bigger,
- 7:55SPEAKER_03: it might get smaller.
- 7:57SPEAKER_03: If it's on a path, you can see the momentum.
- 7:59SPEAKER_03: So I don't really necessarily think NVIDIA did anything more than that.
- 8:03SPEAKER_03: I mean, obviously Jensen,
- 8:05SPEAKER_03: organizationally put enough resources behind it, even when I was at NVIDIA.
- 8:09SPEAKER_03: I mean, certainly most people were not working on... I mean, it's a GPU company.
- 8:13SPEAKER_03: Most people were not working on deep learning.
- 8:14SPEAKER_03: I mean, it was a big thing by, you know, people were doing other problems and this and that.
- 8:19SPEAKER_03: And, you know, I mean, famously...
- 8:22SPEAKER_03: The GPUs were used to train ConvNets,
- 8:26SPEAKER_03: which is a technology nobody uses anymore,
- 8:28SPEAKER_03: but they were probably the first big breakthrough for deep learning for computer vision.
- 8:32SPEAKER_03: And that was like one of 10 or 12 problems that they supported.
- 8:35SPEAKER_03: And it was not anyone's number one problem.
- 8:39SPEAKER_03: You know,
- 8:39SPEAKER_03: if you want to if you want to sort of believe their narrative or sort of these just
- 8:43SPEAKER_03: so stories,
- 8:44SPEAKER_03: it's like,
- 8:44SPEAKER_03: well,
- 8:45SPEAKER_03: you know,
- 8:45SPEAKER_03: they supported 12 things and it worked out.
- 8:47SPEAKER_03: Right.
- 8:47SPEAKER_03: So it's kind of a classical problem where you invest into multiple things.
- 8:50SPEAKER_03: And often the biggest one isn't the one that is as cool.
- 8:53SPEAKER_03: I mean,
- 8:53SPEAKER_03: even when I was at Google 20 years ago,
- 8:55SPEAKER_03: the two the two sort of non Google problems that Google supported back then was Google,
- 9:00SPEAKER_03: not Alphabet.
- 9:00SPEAKER_03: Right.
- 9:01SPEAKER_03: The two problems that they were very supportive was the,
- 9:03SPEAKER_03: you know,
- 9:04SPEAKER_03: the genomics like Trekventor project and the self-driving project that at the time
- 9:09SPEAKER_03: was run out of Carnegie Mellon.
- 9:11SPEAKER_03: Both of those sort of ended up working out and being big,
- 9:13SPEAKER_03: but in very different ways and very different trajectories.
- 9:16SPEAKER_03: And I think if you follow the path of those things, they've crossed several times.
- 9:21SPEAKER_03: I mean,
- 9:21SPEAKER_03: certainly,
- 9:21SPEAKER_03: there was definitely a time not long ago when self-driving things seemed way more
- 9:25SPEAKER_03: significant than some of the genomic stuff.
- 9:27SPEAKER_03: Okay, maybe that also got dragged out.
- 9:28SPEAKER_03: That's taken longer.
- 9:29SPEAKER_03: But now you have Waymos.
- 9:31SPEAKER_03: I think a lot of the genomics stuff is...
- 9:34SPEAKER_03: you know it's just going to take longer as well uh you know we need to see once
- 9:37SPEAKER_03: more people or whatever i mean you're the expert on that so it's with research
- 9:41SPEAKER_03: nobody knows nobody knows and and the problem also is when you're a researcher you
- 9:46SPEAKER_03: know like how do you get out how do you get resources and should you get resources
- 9:51SPEAKER_03: you know like deep learning was already well into the upswing so there was a lot of
- 9:54SPEAKER_03: momentum when we were there so we did well but you know
- 9:59SPEAKER_03: Our big LLM,
- 10:00SPEAKER_03: which is basically our attempt to replicate and improve on GPT-3,
- 10:05SPEAKER_03: or even GPT-2,
- 10:05SPEAKER_03: I guess,
- 10:07SPEAKER_03: is a big deal to get resources for that.
- 10:09SPEAKER_03: We got it.
- 10:10SPEAKER_03: My boss deserves a lot of credit.
- 10:11SPEAKER_03: But it was a very big deal to get time on NVIDIA's internal computer cluster.
- 10:15SPEAKER_03: And in order to scale it that big, to train...
- 10:19SPEAKER_03: 11 billion parameter model which was briefly the largest model ever trained i mean
- 10:23SPEAKER_03: it was a very big deal to set that up to have the people motivated to do it the
- 10:25SPEAKER_03: resources so you know you you can't you you just can't know and then obviously
- 10:30SPEAKER_03: after the fact i mean a year or two later it's like oh my god this is the most
- 10:34SPEAKER_03: obvious decision of all time you know
- 10:37SPEAKER_00: Yeah.
- 10:37SPEAKER_00: Well, so, you know, we've been talking about high level abstraction here.
- 10:41SPEAKER_00: And,
- 10:42SPEAKER_00: you know,
- 10:42SPEAKER_00: I want to loop back a little bit about the technology at some point in terms of
- 10:46SPEAKER_00: like the more macro social,
- 10:48SPEAKER_00: you know,
- 10:50SPEAKER_00: implications.
- 10:51SPEAKER_00: And mostly I want to get your take.
- 10:53SPEAKER_00: Let me sort of add one more thing real quick.
- 10:55SPEAKER_03: I mean, yeah, just because I'm saying this sort of we'll talk about this more in a bit.
- 10:59SPEAKER_03: But I mean, this happens all the time is.
- 11:02SPEAKER_03: You know, me and some of my colleagues, we got very excited about things.
- 11:05SPEAKER_03: Imagine having the first billion parameter model and it can do cool things.
- 11:08SPEAKER_03: It can do things like,
- 11:09SPEAKER_03: wow,
- 11:10SPEAKER_03: you can get it to write like synthetic Wikipedia articles,
- 11:12SPEAKER_03: you know,
- 11:13SPEAKER_03: something amazing like that.
- 11:14SPEAKER_03: Like given the sources, it would write a 500 word Wikipedia stub at the top.
- 11:17SPEAKER_03: That was like kind of okay.
- 11:19SPEAKER_03: You know, so you're looking at that and no one's really done that before.
- 11:23SPEAKER_03: And you think it's amazing.
- 11:24SPEAKER_03: And it's easy to think I certainly fell into this quite a bit.
- 11:27SPEAKER_03: It's like, oh, my God, it's not that long until, you know, blah, blah, blah.
- 11:30SPEAKER_03: You can do this.
- 11:31SPEAKER_03: You can imagine all of these applications.
- 11:32SPEAKER_03: Right.
- 11:33SPEAKER_03: But like, hey, those applications took several years.
- 11:37SPEAKER_03: The truth is until OpenAI did ChatGPT, no one was really that actively working at it.
- 11:41SPEAKER_03: And still to this day, it hasn't exactly replaced human writing.
- 11:45SPEAKER_03: The one place where the AI probably has replaced human work quite a bit is on the
- 11:48SPEAKER_03: image generation side,
- 11:50SPEAKER_03: you know,
- 11:51SPEAKER_03: but that maybe has a lot more to do with like,
- 11:53SPEAKER_03: and that's actually not even that big of a model.
- 11:56SPEAKER_03: And it just maybe has to do with us being lazy and we need illustrations.
- 11:59SPEAKER_03: And most of us aren't good enough to draw things.
- 12:01SPEAKER_03: So the truth is, what I'm saying is that's why you see all of this hype and bust and hype and bust.
- 12:08SPEAKER_03: And it's very schizophrenic because when you see this technology,
- 12:11SPEAKER_03: you immediately say,
- 12:12SPEAKER_03: hey,
- 12:12SPEAKER_03: it's going to do these amazing things.
- 12:13SPEAKER_03: It's going to change the world.
- 12:15SPEAKER_03: And it often does.
- 12:16SPEAKER_03: But sometimes it doesn't at all.
- 12:17SPEAKER_03: And sometimes it takes a lot longer.
- 12:19SPEAKER_03: Or maybe you show it and people don't believe it.
- 12:21SPEAKER_03: So final thought is, for example, you see this with the AGI excited people.
- 12:25SPEAKER_03: I mean,
- 12:25SPEAKER_03: how many people do you know,
- 12:26SPEAKER_03: especially on the young side,
- 12:28SPEAKER_03: you know,
- 12:28SPEAKER_03: who like have been saying now for like maybe even two years,
- 12:31SPEAKER_03: AGI is already here and that kind of stuff.
- 12:33SPEAKER_03: You can sort of convince yourself, but it's obviously not true.
- 12:36SPEAKER_00: Many, many, many false positives, actually, false starts or false alarms on that front, actually.
- 12:42SPEAKER_00: So I will just really quickly, you know, I'm in various group chats.
- 12:45SPEAKER_00: I've heard various things and things I've heard people.
- 12:48SPEAKER_00: I mean, in the fall of 2022, this is a real pandemic.
- 12:51SPEAKER_00: quote i'm not gonna say who this person is but they you know are like a silicon
- 12:54SPEAKER_00: valley like micro influencer and you know one of the group chats they were just
- 12:59SPEAKER_00: like oh they don't think humanity is gonna be around in six months this was like
- 13:02SPEAKER_00: december 2022 we're still here a year you know a year and a half past that sell by
- 13:06SPEAKER_00: date you know other people uh have been you know i i recently had a recently had a
- 13:14SPEAKER_00: um it was like october october 20th i was having dinner with a friend and she was
- 13:18SPEAKER_00: uh
- 13:19SPEAKER_00: This might be the last day.
- 13:20SPEAKER_00: AI's coming tomorrow, they say, at Anthropic or OpenAI or whatever.
- 13:25SPEAKER_00: Someone said that, okay?
- 13:27SPEAKER_00: So we're still here.
- 13:29SPEAKER_00: So I'm at this point jaded and just like whatever about that stuff.
- 13:32SPEAKER_00: I do want to ask you, I want to talk about
- 13:35SPEAKER_00: How you make an application on top of this backbone, like how you did deep news.
- 13:40SPEAKER_00: But before that, let me just ask you really quickly.
- 13:42SPEAKER_00: So there's Suno that does music,
- 13:45SPEAKER_00: Midjourney,
- 13:46SPEAKER_00: and Dolly and all these that do stable diffusion or whatever that do the visual stuff.
- 13:52SPEAKER_00: There are different types of AI than an LLM, which is the tech stuff, right?
- 13:56SPEAKER_00: Yeah.
- 13:57SPEAKER_03: Yes, so they all have a text language component, actually.
- 14:02SPEAKER_00: Well, because there's a command prompt, right?
- 14:06SPEAKER_00: Say again?
- 14:07SPEAKER_00: Well, I mean, there's a command prompt, and then that propagates certain lyrics or whatever, right?
- 14:12SPEAKER_03: Well,
- 14:12SPEAKER_03: I mean,
- 14:13SPEAKER_03: the lyrics one really has a strong language component,
- 14:15SPEAKER_03: but even the image one,
- 14:16SPEAKER_03: I mean,
- 14:16SPEAKER_03: you kind of have to describe an image and it has to build a space,
- 14:20SPEAKER_03: which is sort of like the point is it still operates the image somewhat through the
- 14:24SPEAKER_03: text space.
- 14:26SPEAKER_03: But yeah,
- 14:26SPEAKER_03: the song,
- 14:27SPEAKER_03: by the way,
- 14:28SPEAKER_03: that song,
- 14:29SPEAKER_03: that long song that plays for your other podcast,
- 14:32SPEAKER_03: is that Suno Generate it?
- 14:34SPEAKER_00: yeah and a lot of people are really annoyed by it but i i don't know i kind of like
- 14:40SPEAKER_00: it i mean this is let's just say that there's very polarized responses i did cut it
- 14:45SPEAKER_00: down to one minute for the for the intro but yeah it's it's i like the song it
- 14:50SPEAKER_03: definitely feels to me like a little bit uncanny valley though because since you
- 14:54SPEAKER_03: can't really quite tell it's like you're like what's going on here this song is
- 14:58SPEAKER_03: like kind of keeps going
- 15:00SPEAKER_03: Like Indian classical music or something?
- 15:02SPEAKER_03: I can't tell.
- 15:03SPEAKER_00: Yeah, we'll drop in a sample right here of the song so the people know what you're talking about.
- 15:28SPEAKER_01: Two voices rise above the static, dissecting issues diplomatic.
- 15:33SPEAKER_01: With mics at hand, they pray for storm of clashing views and norms transform.
- 15:41SPEAKER_00: And now, okay, now we should be back.
- 15:43SPEAKER_00: But yeah, I'll...
- 15:45SPEAKER_03: hopefully people know what what uh it reminds me of i mean i'm sure this was partly
- 15:50SPEAKER_03: the intent is um if you remember that scene from uh like whatever midnight in paris
- 15:55SPEAKER_03: one of those movies or whatever when you go and you go to like the like the the you
- 15:59SPEAKER_03: know the homeless poet you know and you tell him hey like write me a poem about
- 16:04SPEAKER_03: like this couple and use the word pineapple you know
- 16:07SPEAKER_00: Yeah.
- 16:07SPEAKER_00: Yeah.
- 16:08SPEAKER_00: No, that's very similar to what it's like.
- 16:10SPEAKER_00: That was very similar to what, to what I did.
- 16:12SPEAKER_00: Yeah.
- 16:12SPEAKER_00: So, I mean, those are, I mean, look, I'm using it.
- 16:15SPEAKER_00: Um,
- 16:16SPEAKER_00: you know,
- 16:16SPEAKER_00: uh,
- 16:17SPEAKER_00: I'm not like,
- 16:17SPEAKER_00: I bought like for one podcast that I did,
- 16:20SPEAKER_00: uh,
- 16:21SPEAKER_00: you know,
- 16:22SPEAKER_00: I got,
- 16:22SPEAKER_00: I bought like a,
- 16:24SPEAKER_00: some sample of,
- 16:25SPEAKER_00: from some library.
- 16:26SPEAKER_00: I didn't do that with the Sudo,
- 16:27SPEAKER_00: although I was paying for Sudo for a while just to,
- 16:29SPEAKER_00: but then I got bored really quickly with making songs.
- 16:31SPEAKER_00: I'm not a songwriter.
- 16:32SPEAKER_00: I will say real quickly, uh,
- 16:35SPEAKER_00: I do tell it to write a 1,000-word – I do tell Claude and I think it's like 01,
- 16:41SPEAKER_00: the newest one of ChatGPT,
- 16:42SPEAKER_00: to write a 1,000-word essay about Indo-Europeans by Razim Khan.
- 16:48SPEAKER_00: And it's still definitely – people should still be paying for the subscription for
- 16:52SPEAKER_00: what I write because it's like – I'll put it like senior year of high school,
- 16:55SPEAKER_00: but a good student,
- 16:57SPEAKER_00: not a bad one.
- 16:58SPEAKER_00: But still, it's very, very pedestrian.
- 17:01SPEAKER_00: There's like a –
- 17:02SPEAKER_00: There's a riz in there that my writing has, and also more details, more content.
- 17:07SPEAKER_00: So it's still not up there.
- 17:09SPEAKER_00: But you know what?
- 17:09SPEAKER_00: They're not actually bad summaries.
- 17:12SPEAKER_00: If you're totally naive, they're okay getting you up to par.
- 17:16SPEAKER_00: So some of this writing stuff is getting good.
- 17:18SPEAKER_00: And then you have the visual stuff, which I do it sometimes.
- 17:23SPEAKER_00: I use AI to do some of the images for my posts that people don't often notice, and sometimes they do.
- 17:30SPEAKER_03: Yeah, I mean, I think images, you know, people are just used to that style and they're okay with it.
- 17:34SPEAKER_03: I mean, I think people are having a lot of fun with it these days.
- 17:37SPEAKER_03: But anyway, we can keep going with this conversation.
- 17:39SPEAKER_03: But I think to bring it back a little bit,
- 17:41SPEAKER_03: you know, to bring back the weave, you know, as a, you know, as, as, as our, yeah.
- 17:46SPEAKER_03: Like to say,
- 17:47SPEAKER_03: uh,
- 17:48SPEAKER_03: is,
- 17:48SPEAKER_03: uh,
- 17:49SPEAKER_03: you know,
- 17:49SPEAKER_03: it's,
- 17:50SPEAKER_03: you don't really know the development of these things,
- 17:52SPEAKER_03: you know,
- 17:52SPEAKER_03: and as,
- 17:53SPEAKER_03: as researchers,
- 17:53SPEAKER_03: we get very excited.
- 17:54SPEAKER_03: I'm sure the Suno people are thrilled once they got even like Suno 0.1 working and
- 17:58SPEAKER_03: they should be,
- 18:00SPEAKER_03: um,
- 18:01SPEAKER_03: by,
- 18:01SPEAKER_03: you know,
- 18:01SPEAKER_03: how far away is it in writing compelling songs?
- 18:05SPEAKER_03: I mean, probably,
- 18:06SPEAKER_03: You know, you never know.
- 18:07SPEAKER_03: I mean, maybe it's around the corner.
- 18:08SPEAKER_03: Maybe it'll be a niche thing.
- 18:10SPEAKER_03: You know, who the heck knows, you know?
- 18:12SPEAKER_00: Yeah.
- 18:12SPEAKER_00: So, okay.
- 18:13SPEAKER_00: So you,
- 18:14SPEAKER_00: you have,
- 18:14SPEAKER_00: you know,
- 18:16SPEAKER_00: since we first started talking,
- 18:18SPEAKER_00: you really,
- 18:18SPEAKER_00: you know,
- 18:19SPEAKER_00: gone through the whole process of,
- 18:20SPEAKER_00: I mean,
- 18:21SPEAKER_00: you're beyond an MVP.
- 18:22SPEAKER_00: You have,
- 18:22SPEAKER_00: if you go to deepnews.com with a Z or,
- 18:25SPEAKER_00: you know,
- 18:25SPEAKER_00: get on your phone,
- 18:26SPEAKER_00: you guys can check out what I'm saying,
- 18:27SPEAKER_00: but you have an app,
- 18:28SPEAKER_00: you have an application,
- 18:29SPEAKER_00: you have something real that's going on.
- 18:31SPEAKER_00: And, you know, basically the way I understand,
- 18:34SPEAKER_00: use it because I don't understand it because I didn't develop it,
- 18:37SPEAKER_00: is that there are summaries of various news events that are happening.
- 18:41SPEAKER_00: I often use it to actually search for something specific, and then I get the summarization off that.
- 18:47SPEAKER_00: So these are kind of based on the underlying distribution of stories,
- 18:50SPEAKER_00: and you have rankings based on Twitter influencers and all sorts of other things as
- 18:54SPEAKER_00: well that go into it.
- 18:55SPEAKER_00: But it's basically using artificial...
- 18:57SPEAKER_00: artificial intelligence to kind of like munch through the news and kind of digest
- 19:01SPEAKER_00: it for you that's one way i think of it um what did you learn though uh about how
- 19:07SPEAKER_00: to create applications how to actually build a business off this new technology
- 19:11SPEAKER_00: that you know uh is very exciting but we've only done a few things with it that i
- 19:16SPEAKER_03: can see yeah i mean when we started i just wanted to see if we can do it because i
- 19:19SPEAKER_03: know that we could um
- 19:21SPEAKER_03: But as it went further along,
- 19:24SPEAKER_03: we're like,
- 19:25SPEAKER_03: hey,
- 19:25SPEAKER_03: we can actually use this to replace something like the New York Times or like a newspaper.
- 19:30SPEAKER_03: Because I think it has some real advantages.
- 19:36SPEAKER_03: You know, the most obvious is that it's looking at a lot more sources.
- 19:39SPEAKER_03: So it's going to be wider, you know, faster.
- 19:41SPEAKER_03: You know, it can write articles when you sleep.
- 19:43SPEAKER_03: But I think I think especially where the media has gone recently,
- 19:47SPEAKER_03: it's just they're well,
- 19:50SPEAKER_03: they're not just biased.
- 19:51SPEAKER_03: It's kind of like you can get the unbiased version, if that makes sense.
- 19:55SPEAKER_03: So, for example.
- 19:57SPEAKER_03: you know,
- 19:57SPEAKER_03: if I want to read about fed policy or I just want to know what's going on with the
- 20:01SPEAKER_03: fed policy,
- 20:02SPEAKER_03: like the New York times will give me that information,
- 20:04SPEAKER_03: I guess,
- 20:05SPEAKER_03: but I'm going to get it through like Paul Krugman's lens,
- 20:07SPEAKER_03: you know?
- 20:09SPEAKER_03: And yeah.
- 20:10SPEAKER_03: And, and I literally can't get it in any other way.
- 20:12SPEAKER_03: Whereas I,
- 20:14SPEAKER_03: I kind of just want to see like, A, what's going on?
- 20:16SPEAKER_03: And two, okay, what is everyone saying?
- 20:18SPEAKER_03: What's sort of the, you know, I guess the synthesis of it, right?
- 20:22SPEAKER_03: And I think for whatever reason, the media has had a lot of difficulty with that.
- 20:27SPEAKER_03: I mean, you saw that especially in the election, but a lot of other things also.
- 20:30SPEAKER_03: So I think what's great about the AI is, again, it's fast.
- 20:34SPEAKER_03: It looks at a lot of sources.
- 20:35SPEAKER_03: It can look at things internationally.
- 20:36SPEAKER_03: All of these things that humans really can't do,
- 20:38SPEAKER_03: but I just think in practice,
- 20:40SPEAKER_03: it just generates better news in many ways.
- 20:43SPEAKER_03: It gives you,
- 20:44SPEAKER_03: yes,
- 20:44SPEAKER_03: the personalization that you want,
- 20:45SPEAKER_03: which is huge,
- 20:46SPEAKER_03: but also it just gives you nice,
- 20:49SPEAKER_03: clean headlines that get to the point,
- 20:51SPEAKER_03: which I think is a lot of people want,
- 20:52SPEAKER_03: and they struggle to get that from the media.
- 20:54SPEAKER_03: You know,
- 20:54SPEAKER_03: I mean,
- 20:55SPEAKER_03: yeah,
- 20:56SPEAKER_03: nobody that I know would even consider turning on notifications on their phone,
- 21:00SPEAKER_03: for example,
- 21:00SPEAKER_03: on a news app.
- 21:01SPEAKER_03: And if they did, you either get way too little or way too much.
- 21:04SPEAKER_03: You know, it's like you can and you can build it to be great from a human perspective.
- 21:10SPEAKER_03: I mean, Apple kind of does that with Apple News to the extent has a lot of human intervention.
- 21:14SPEAKER_03: They do a pretty good job.
- 21:15SPEAKER_03: But I think I think starting with the AI is just sort of, you know, the obvious way.
- 21:21SPEAKER_03: Now, obviously,
- 21:23SPEAKER_03: It sometimes writes bad headlines.
- 21:24SPEAKER_03: Sometimes it misses important facts.
- 21:26SPEAKER_03: Sometimes it misses context.
- 21:27SPEAKER_03: It doesn't tie stories together.
- 21:29SPEAKER_03: Generally speaking,
- 21:30SPEAKER_03: the articles are quite short,
- 21:31SPEAKER_03: although for bigger ones,
- 21:32SPEAKER_03: especially with O1,
- 21:33SPEAKER_03: it's now writing three,
- 21:34SPEAKER_03: four paragraph articles that I think are quite good.
- 21:37SPEAKER_03: You know, but these things will improve.
- 21:39SPEAKER_03: It's not like I think it's more than useful now.
- 21:42SPEAKER_03: I think it's very useful.
- 21:43SPEAKER_03: In some areas, it lacks the context.
- 21:46SPEAKER_03: Like people, you know, correctly point out that some of the biology stuff, it's a little fuzzy on.
- 21:50SPEAKER_03: Like the LLMs, we can sort of explain why.
- 21:52SPEAKER_03: I think that says the training data,
- 21:54SPEAKER_03: but the LLMs are not very good at explaining or understanding biology.
- 21:59SPEAKER_03: But that's an improve.
- 22:01SPEAKER_00: And so they'll probably improve.
- 22:03SPEAKER_00: So why not just,
- 22:05SPEAKER_00: well,
- 22:05SPEAKER_00: actually,
- 22:06SPEAKER_00: I kind of know,
- 22:06SPEAKER_00: but let me just ask you,
- 22:07SPEAKER_00: why not just use ChatGPT to query the news?
- 22:10SPEAKER_00: Like they have a limit on their, you know,
- 22:13SPEAKER_00: how late the data is and other things like that, right?
- 22:15SPEAKER_00: Like how did you modify it?
- 22:16SPEAKER_00: Like how, what did you find that you had to do?
- 22:18SPEAKER_03: Well, I think that's almost coming at it from like, like the wrong direction in a sense, right?
- 22:23SPEAKER_03: You know, because, and I think this is where people get stuck.
- 22:26SPEAKER_03: They think,
- 22:27SPEAKER_03: oh,
- 22:28SPEAKER_03: you know,
- 22:28SPEAKER_03: they just think procedurally,
- 22:29SPEAKER_03: well,
- 22:29SPEAKER_03: I take A and I add it to B.
- 22:31SPEAKER_03: And if the thing is super intelligent,
- 22:32SPEAKER_03: it's just going to work.
- 22:33SPEAKER_03: by,
- 22:33SPEAKER_03: you know,
- 22:34SPEAKER_03: we're actually,
- 22:35SPEAKER_03: you know,
- 22:35SPEAKER_03: building a fully fledged news service,
- 22:37SPEAKER_03: which means we have to think about,
- 22:38SPEAKER_03: okay,
- 22:39SPEAKER_03: like you want to organize things by topics.
- 22:41SPEAKER_03: You want to have top sources.
- 22:43SPEAKER_03: You want to write things in this sort of article and headline format that people are used to.
- 22:47SPEAKER_03: That therefore you can, you can work on that.
- 22:50SPEAKER_03: You can, you can, you can train it.
- 22:52SPEAKER_03: You can deal with specific issues.
- 22:53SPEAKER_03: Like a lot of times a big story,
- 22:56SPEAKER_03: Like it's really easy to write top stories from a week ago.
- 22:58SPEAKER_03: You have all the social proof, all the best sources, all the best explainers.
- 23:01SPEAKER_03: And actually, if you see some of our competitors, that's like pretty close to what they do.
- 23:04SPEAKER_03: As a demo, you can say, wow, we have great news.
- 23:06SPEAKER_03: We have this great summary,
- 23:07SPEAKER_03: you know,
- 23:08SPEAKER_03: perplexity or someone in their demo will be like,
- 23:10SPEAKER_03: we did a great job of explaining the open AIs like Sam Altman drama from a year ago.
- 23:14SPEAKER_03: It's like, great.
- 23:15SPEAKER_03: No one's going to read about that today.
- 23:17SPEAKER_03: Yeah, that's not news.
- 23:19SPEAKER_03: It's almost history.
- 23:20SPEAKER_03: It's history.
- 23:20SPEAKER_03: Yeah.
- 23:21SPEAKER_03: And they're a search company.
- 23:22SPEAKER_03: So that makes sense for them to focus on that.
- 23:23SPEAKER_03: They would be good at that.
- 23:24SPEAKER_03: That's what you would expect.
- 23:25SPEAKER_03: But news is sort of its own animal.
- 23:27SPEAKER_03: And I think a lot of the stuff we do is really not qualitatively that difficult.
- 23:31SPEAKER_03: But you have to think about it in terms of solving the problem.
- 23:33SPEAKER_03: You know, like, what do you want an actual fully fledged, fully automated newspaper to look at?
- 23:38SPEAKER_03: Which means, yeah, it's going to be automated.
- 23:40SPEAKER_03: You're not going to have humans making judgments.
- 23:41SPEAKER_03: You're not going to have humans moving things around.
- 23:43SPEAKER_03: You're not going to be like,
- 23:44SPEAKER_03: oh,
- 23:44SPEAKER_03: have people propose things,
- 23:46SPEAKER_03: you know,
- 23:47SPEAKER_03: and and sure,
- 23:49SPEAKER_03: you know,
- 23:49SPEAKER_03: I mean,
- 23:49SPEAKER_03: if the problem is super easy,
- 23:50SPEAKER_03: you could just feed in all the tweets and get all the answers.
- 23:52SPEAKER_03: But for example,
- 23:53SPEAKER_03: for example,
- 23:53SPEAKER_03: the chat interfaces,
- 23:54SPEAKER_03: you know,
- 23:55SPEAKER_03: like a lot of people thought that we would just be going to GPT or custom GPTs or
- 24:00SPEAKER_03: whatever and just asking questions all day.
- 24:02SPEAKER_03: No one does that.
- 24:04SPEAKER_03: I mean, they do it for some problems.
- 24:05SPEAKER_03: I use it for some problems,
- 24:06SPEAKER_03: but people aren't using it that much,
- 24:09SPEAKER_03: especially now that the novelty has worn off.
- 24:11SPEAKER_03: No one is going to chat GPT and ask them like, hey, what's happening in the Middle East?
- 24:15SPEAKER_03: It's like not an experience that people want.
- 24:17SPEAKER_03: You know, people people want to see stories and headlines and details and citations and sources.
- 24:22SPEAKER_03: And, you know, so that's what you have to build it as a real product.
- 24:27SPEAKER_03: And I mean, I would say the, you know, 95% of the work is not the LOM writing the story.
- 24:32SPEAKER_00: Okay, because that's done for you by, so you are using right now an OpenAI API.
- 24:40SPEAKER_03: Yeah, or we can use a XAI Grok API, which we've been using a little bit.
- 24:46SPEAKER_03: You'll see some of those stories.
- 24:46SPEAKER_03: I mean, we can sort of swap it out for anything.
- 24:48SPEAKER_03: But I mean, the general flow is not really that complicated, but you have to take all the sources.
- 24:53SPEAKER_03: You have to sort of kick off these events,
- 24:54SPEAKER_03: either continuously or in some sort of clock,
- 24:56SPEAKER_03: where you try to basically generate a hypothesis,
- 24:59SPEAKER_03: essentially.
- 25:00SPEAKER_03: It says, OK, you have this cluster of people who are talking about something that may be related.
- 25:05SPEAKER_03: Great.
- 25:06SPEAKER_03: Write a sample story and then evaluate.
- 25:08SPEAKER_03: Is it newsy?
- 25:09SPEAKER_03: Is it new?
- 25:11SPEAKER_03: Is it interesting?
- 25:12SPEAKER_03: If so, okay, great.
- 25:13SPEAKER_03: Then put more of your resources into writing like a proper story.
- 25:17SPEAKER_03: And then, you know, you have this, I mean, internally we call it deduplication.
- 25:21SPEAKER_03: It's like a very like nerdy way of putting it, but really, you know, just sorting, right?
- 25:25SPEAKER_03: I mean, you're going to have, we write about 2,000 unique stories per day, maybe a little more now.
- 25:31SPEAKER_03: But if you discounted the uniqueness, it's like 100,000 a day.
- 25:35SPEAKER_03: or something crazy like that.
- 25:37SPEAKER_03: Like a lot of stuff is rejected because it's not newsy or not really a story.
- 25:41SPEAKER_03: And a lot of things are, you know, there's, okay, we already wrote this, you know?
- 25:45SPEAKER_03: So, you know, is it a big enough update to write something new?
- 25:48SPEAKER_03: Do you replace?
- 25:48SPEAKER_03: It's actually very difficult.
- 25:51SPEAKER_03: Like all the time,
- 25:51SPEAKER_03: even now,
- 25:52SPEAKER_03: you know,
- 25:53SPEAKER_03: we'll have a really,
- 25:53SPEAKER_03: really good version of a story and then you check back two hours later and then we
- 25:57SPEAKER_03: replaced it with a version that's frankly worse sometimes.
- 26:00SPEAKER_03: So the AI has to make that evaluation and sometimes it makes a mistake.
- 26:04SPEAKER_00: So you're, uh, okay.
- 26:06SPEAKER_00: So I'm looking at it.
- 26:08SPEAKER_00: You said there's like a thousand new stories that you're creating summaries for.
- 26:11SPEAKER_00: This is just your small company.
- 26:13SPEAKER_03: I think that that's really kind of misleading.
- 26:15SPEAKER_03: Like, like that's where I think people go wrong.
- 26:19SPEAKER_03: They're like, take this and summarize it, you know?
- 26:21SPEAKER_03: But it's like, no, these are just, they're just writing stories, you know?
- 26:24SPEAKER_00: So, yeah.
- 26:24SPEAKER_00: So a comparison is,
- 26:27SPEAKER_00: New York times has an average of 150 articles daily and about 65 blog posts.
- 26:35SPEAKER_00: So yeah.
- 26:36SPEAKER_00: So we're getting to bigger.
- 26:38SPEAKER_00: We're like 2000, right?
- 26:39SPEAKER_00: Yeah.
- 26:41SPEAKER_00: Yeah.
- 26:42SPEAKER_00: You're drawing from all these like ultimate, like the training data are like all these different sources.
- 26:47SPEAKER_03: Yeah.
- 26:47SPEAKER_03: We're looking at like, you know, more than 50,000 sources and, you know, and, and across topics.
- 26:54SPEAKER_03: And we've added a bit of automatic translation, but we're going to do a lot more of that.
- 27:00SPEAKER_03: So,
- 27:00SPEAKER_03: for example,
- 27:00SPEAKER_03: if you want coverage,
- 27:02SPEAKER_03: you know,
- 27:02SPEAKER_03: in Spain or France or Brazil or whatever,
- 27:04SPEAKER_03: I mean,
- 27:05SPEAKER_03: you get decent coverage from the English language sources,
- 27:07SPEAKER_03: but obviously you would have much better coverage if you,
- 27:10SPEAKER_03: you know,
- 27:10SPEAKER_03: if you took the best sources in those languages.
- 27:13SPEAKER_03: Yeah.
- 27:13SPEAKER_03: But by the same token,
- 27:15SPEAKER_03: like,
- 27:15SPEAKER_03: you know,
- 27:16SPEAKER_03: people are probably interested,
- 27:17SPEAKER_03: what are they saying about America or about Trump in Japan?
- 27:19SPEAKER_03: So you can cover that as well.
- 27:22SPEAKER_03: That's a little more of a...
- 27:24SPEAKER_03: of a niche um right so so yeah i mean something like new york times which is the
- 27:28SPEAKER_03: biggest news group okay i didn't know that but that makes sense they're doing about
- 27:31SPEAKER_03: 200 stories a day roughly speaking so we're doing an order of magnitude bigger yeah
- 27:36SPEAKER_00: not not a huge difference but you know um noticeable noticeable uh and and you know
- 27:41SPEAKER_00: and you like rolled this all out like what like a year and a half two years no i
- 27:45SPEAKER_03: mean really less than that i mean we've really only been live for about a year um
- 27:49SPEAKER_03: maybe like a little bit longer as like a very crappy demo by
- 27:53SPEAKER_03: Like as a product you can use for about a year, a little bit less than that.
- 27:57SPEAKER_00: So deep news is already a product.
- 27:59SPEAKER_00: You already have that.
- 28:02SPEAKER_00: Kind of like juxtaposed against that,
- 28:04SPEAKER_00: I want to talk about some people are talking about a plateau,
- 28:07SPEAKER_00: like a plateau in performance or a plateau in AI right now.
- 28:11SPEAKER_00: What do they mean by that?
- 28:13SPEAKER_00: Like what are the benchmarks?
- 28:14SPEAKER_00: Sure.
- 28:14SPEAKER_00: Sure.
- 28:15SPEAKER_03: Yeah.
- 28:15SPEAKER_03: So, I mean, I think it speaks to a very true thing, right?
- 28:19SPEAKER_03: Which is that at sort of at best, at best, even when, um, let's just say you have infinite data, right?
- 28:27SPEAKER_03: You know, and the model can scale to an infinite size.
- 28:30SPEAKER_03: Um,
- 28:31SPEAKER_03: When you're in a very data-rich environment,
- 28:36SPEAKER_03: generally speaking,
- 28:37SPEAKER_03: you would have improvement in a linear metric grow at whatever,
- 28:42SPEAKER_03: like the log or the square root or something like that.
- 28:44SPEAKER_03: It's never of the model size.
- 28:47SPEAKER_03: It would never, ever, ever be linear.
- 28:49SPEAKER_03: That's impossible.
- 28:51SPEAKER_03: So so you had these sort of so at a rough level,
- 28:54SPEAKER_03: you had these sort of projections and you can think of different X and Y scales.
- 28:57SPEAKER_03: They don't really matter for a second.
- 28:58SPEAKER_03: You know,
- 29:00SPEAKER_03: let's just think on even the very simple task of perplexity of basically predicting
- 29:03SPEAKER_03: the next token,
- 29:04SPEAKER_03: which obviously gets,
- 29:05SPEAKER_03: you know,
- 29:06SPEAKER_03: gets confusing data quality and blah,
- 29:08SPEAKER_03: blah,
- 29:08SPEAKER_03: blah.
- 29:08SPEAKER_03: But take something like that, like, you know, for a while that just kept increasing, you know.
- 29:14SPEAKER_03: It's like if you have a bigger model,
- 29:16SPEAKER_03: if you train for longer,
- 29:18SPEAKER_03: or if you add more data,
- 29:20SPEAKER_03: these were for a while,
- 29:21SPEAKER_03: I guess the first popular version is something called the Chinchilla Rules,
- 29:24SPEAKER_03: which is like from a Facebook paper,
- 29:27SPEAKER_03: from a model people don't even use anymore.
- 29:28SPEAKER_03: But this idea that you have these sort of three things and this is how they scale.
- 29:34SPEAKER_03: Well, the issue that people are pointing out is that we're getting to the point where
- 29:39SPEAKER_03: yeah,
- 29:39SPEAKER_03: adding more data or more compute isn't really helping as much,
- 29:43SPEAKER_03: like just the curve is slowing down in these improvements.
- 29:47SPEAKER_03: I think that is both true and somewhat misleading.
- 29:50SPEAKER_03: I mean, I think you pointed out the metrics.
- 29:52SPEAKER_03: The problem is ultimately we don't care about predicting the next token.
- 29:55SPEAKER_03: We care about, you know,
- 29:57SPEAKER_03: solving real problems and um and a lot of these human metrics have both been gamed
- 30:03SPEAKER_03: and they get saturated once you're getting like 95 you're really getting into an
- 30:06SPEAKER_03: issue of okay are the examples labeled wrong did the model train audit etc so
- 30:11SPEAKER_03: that's been a big issue for a while um another metric you really like for a while
- 30:15SPEAKER_03: was this lmsys where people put up their own questions but it was relative but now
- 30:19SPEAKER_03: that's clearly was being gamed in in both you know on purpose and not you know i
- 30:25SPEAKER_03: think i think
- 30:26SPEAKER_03: You know,
- 30:26SPEAKER_03: one would argue that the task we probably care more about is like real difficult
- 30:30SPEAKER_03: problems that people face.
- 30:32SPEAKER_03: But it's becoming like more difficult to, you know, to collect those.
- 30:37SPEAKER_00: Yeah,
- 30:37SPEAKER_00: I mean,
- 30:38SPEAKER_00: one of the things that artificial intelligence is supposed to do is put people out
- 30:42SPEAKER_00: of work.
- 30:42SPEAKER_00: Translators are having issues.
- 30:44SPEAKER_00: Some illustrators are definitely having issues.
- 30:47SPEAKER_00: Yeah, I'm seeing the unemployment rate 4.1%, which is, I mean, that's what I was called.
- 30:51SPEAKER_00: That was called full employment when we were kids.
- 30:53SPEAKER_00: I mean,
- 30:53SPEAKER_00: today we've had lower unemployment since the recovery from the 2010,
- 30:58SPEAKER_00: and it's never really changed except for the one pandemic spike,
- 31:01SPEAKER_00: but that was temporary and aberration.
- 31:03SPEAKER_00: So we have low unemployment, and this artificial intelligence is...
- 31:06SPEAKER_00: you know,
- 31:07SPEAKER_00: cannibalizing some jobs we know,
- 31:08SPEAKER_00: but it,
- 31:09SPEAKER_00: it hasn't seemed to have a major impact yet,
- 31:11SPEAKER_00: which is not shocking to me,
- 31:12SPEAKER_00: but some people,
- 31:13SPEAKER_00: you know,
- 31:14SPEAKER_03: it hasn't been around for very long,
- 31:15SPEAKER_03: you know?
- 31:15SPEAKER_03: And,
- 31:16SPEAKER_03: and yeah,
- 31:17SPEAKER_03: I mean,
- 31:17SPEAKER_03: that's kind of what you would expect with,
- 31:19SPEAKER_03: I mean,
- 31:19SPEAKER_03: so far,
- 31:19SPEAKER_03: yes,
- 31:19SPEAKER_03: it's,
- 31:20SPEAKER_03: it's acted very much like any other technology where it's going to be very
- 31:23SPEAKER_03: disruptive in some sections.
- 31:24SPEAKER_03: Overall, you don't really see it in the numbers and over time things change a lot, you know?
- 31:28SPEAKER_03: I mean, you look at, I mean, how much as, and then Thompson's favorite example, um,
- 31:34SPEAKER_03: How much has something like Facebook transformed not just advertising,
- 31:37SPEAKER_03: but like even businesses that get created,
- 31:39SPEAKER_03: you know?
- 31:39SPEAKER_03: I mean, okay.
- 31:40SPEAKER_03: And for a while, it was online businesses.
- 31:42SPEAKER_03: All these direct-to-consumer stuff was really created by Facebook in a sense.
- 31:46SPEAKER_03: Because you can, you know, target different demographics.
- 31:49SPEAKER_03: You know,
- 31:49SPEAKER_03: famously,
- 31:49SPEAKER_03: Dollar Shave Club was the first one,
- 31:51SPEAKER_03: at least in sort of the...
- 31:52SPEAKER_03: He's like,
- 31:52SPEAKER_03: hey,
- 31:53SPEAKER_03: you can advertise just to men of a certain age,
- 31:57SPEAKER_03: certain income range in certain cities.
- 31:58SPEAKER_03: Like, great.
- 31:59SPEAKER_03: You know?
- 32:00SPEAKER_03: And obviously, even IRL businesses constantly...
- 32:04SPEAKER_03: do nothing but try to post on on instagram and tell you to not even to review them
- 32:08SPEAKER_03: but to about them so i mean these things have like weird and like longer you know
- 32:14SPEAKER_03: more factors i mean i think it's you know it would both be i think silly to think
- 32:18SPEAKER_03: that ai won't have a big influence at the same time like why would that happen
- 32:21SPEAKER_03: quickly you know except for like you said if your job is being an illustrator okay
- 32:26SPEAKER_03: or or or github right uh not github the stack overflow right like their traffic is
- 32:31SPEAKER_03: down like 90 right
- 32:33SPEAKER_03: Yeah, that makes sense.
- 32:35SPEAKER_03: You know, Stack Overflow, the website for, you know, people fixing each other's coding problems.
- 32:40SPEAKER_00: Yeah.
- 32:41SPEAKER_00: Yeah, I have used it.
- 32:44SPEAKER_00: I have to say, like, I still use it a little bit, but I really switched over to Claude and 01 a lot.
- 32:54SPEAKER_03: And you're not really a full-time developer.
- 32:58SPEAKER_03: Like, all the developers I know pretty much use these tools a ton now.
- 33:02SPEAKER_00: Yeah.
- 33:03SPEAKER_00: Yeah.
- 33:03SPEAKER_00: It's like, it's just, it's part of our daily workflow, right?
- 33:06SPEAKER_00: It's part of our, it's just like, you can't, I can't imagine what it'd be like without it.
- 33:09SPEAKER_00: So you asked, you asked about O1.
- 33:11SPEAKER_00: I don't use O1 all the time.
- 33:13SPEAKER_00: I use probably like half the time, you know, I pay for it, but it's slower.
- 33:16SPEAKER_00: Yeah.
- 33:18SPEAKER_00: Can you just,
- 33:19SPEAKER_00: like,
- 33:19SPEAKER_00: tell me,
- 33:19SPEAKER_00: I mean,
- 33:20SPEAKER_00: like,
- 33:20SPEAKER_00: you know,
- 33:20SPEAKER_00: obviously don't know what the difference really is in a technical sense.
- 33:23SPEAKER_00: How is O1 better than ChatGPT 4.0?
- 33:27SPEAKER_03: O1's a very interesting use case because that's where I think a lot of these things come up.
- 33:32SPEAKER_03: We've been talking a bit abstractly,
- 33:33SPEAKER_03: but I think it's good to get into the details,
- 33:35SPEAKER_03: at least on a specific example.
- 33:36SPEAKER_03: So...
- 33:38SPEAKER_03: 01, when it came out, it was supposed to be this great model.
- 33:40SPEAKER_03: It's going to be a lot more expensive, and I'll explain why in a second.
- 33:44SPEAKER_03: But it's supposed to be a lot better, and it launches on LMSys.
- 33:47SPEAKER_03: At first, it has a score that's not shockingly higher, but was significantly higher than GPT-40.
- 33:52SPEAKER_03: And then people voted it down to below GPT-40.
- 33:56SPEAKER_03: I think somewhat on purpose,
- 33:57SPEAKER_03: but somewhat like,
- 33:57SPEAKER_03: yes,
- 33:58SPEAKER_03: on a lot of queries,
- 34:00SPEAKER_03: it's not noticeably better,
- 34:02SPEAKER_03: as you're somewhat pointing out.
- 34:03SPEAKER_03: And it's much slower.
- 34:05SPEAKER_03: Now, what...
- 34:07SPEAKER_03: I do still think it's potentially a transformational technology for us.
- 34:11SPEAKER_03: For us, story writing, it makes a very big difference.
- 34:13SPEAKER_03: But let's just talk about what it does.
- 34:15SPEAKER_03: So the problem is this.
- 34:18SPEAKER_03: We've had this long-term problem for years now with the LLMs,
- 34:24SPEAKER_03: where the way they're trained is they're just predicting the next word.
- 34:27SPEAKER_03: And they write, they write the next word.
- 34:29SPEAKER_03: And this is a very linear process, right?
- 34:30SPEAKER_03: The problem with this is that if you make a mistake, you can't fix it.
- 34:34SPEAKER_03: That's really the problem.
- 34:36SPEAKER_03: So you can alleviate this with a million different tricks.
- 34:39SPEAKER_03: Obviously, the most obvious is beam search.
- 34:41SPEAKER_03: They don't actually write a word.
- 34:42SPEAKER_03: They sort of do like a look ahead and they sort of find the best sort of, you know, multi-step thing.
- 34:47SPEAKER_03: Right.
- 34:48SPEAKER_03: Kind of like almost you would think of us as humans.
- 34:51SPEAKER_03: We probably do some look ahead as well when we're walking or going somewhere, driving a little bit ahead.
- 34:56SPEAKER_03: You know what I mean?
- 34:58SPEAKER_03: But still, you're kind of taking irreversible actions like when we're having a conversation right now.
- 35:02SPEAKER_03: Um, I can't, we can't go back and edit it.
- 35:04SPEAKER_03: That just wouldn't really make any sense.
- 35:07SPEAKER_03: But,
- 35:08SPEAKER_03: but when you write,
- 35:08SPEAKER_03: I mean,
- 35:09SPEAKER_03: when you write an article,
- 35:10SPEAKER_03: you do that,
- 35:10SPEAKER_03: you write a draft and then you edit it and you show it.
- 35:12SPEAKER_03: Maybe you come back later, you think about it again, you change it.
- 35:15SPEAKER_03: Right.
- 35:16SPEAKER_03: So, so the, the, the, the, the,
- 35:20SPEAKER_03: how how they implemented is almost not that important what is important is that
- 35:23SPEAKER_03: they gave the model this ability because you can engineer it yourself we've been
- 35:27SPEAKER_03: doing that with deep news for a while for quite some time um yeah from the very
- 35:32SPEAKER_03: beginning what we've done is do this sort of chain of thought kind of stuff um
- 35:35SPEAKER_03: we've been doing that from the very beginning where even with gpt 3.5 you would
- 35:40SPEAKER_03: have it write the original version of the article and you're like hey
- 35:42SPEAKER_03: like rewrite it with this thing, like find the salient facts, right?
- 35:45SPEAKER_03: Because I mean, think about writing, writing an article, perfect example.
- 35:49SPEAKER_03: It makes like logical sense to write the headline first.
- 35:52SPEAKER_03: But when you finish writing the article, you may want to change the headline.
- 35:56SPEAKER_03: Actually, as you got into the details, you want these things to match.
- 36:00SPEAKER_03: So already before ourselves, we had a prompt chain where we would sort of rewrite two or three cycles.
- 36:05SPEAKER_03: The problem is this has problems.
- 36:06SPEAKER_03: You can't rewrite it 10 times.
- 36:07SPEAKER_03: It actually gets super weird.
- 36:09SPEAKER_03: So we found out it was to rewrite it two or three times.
- 36:11SPEAKER_03: But we had to sort of essentially do this.
- 36:13SPEAKER_03: The idea with O1 is to essentially at a high level do that internally.
- 36:17SPEAKER_03: It makes more sense for the model to be trained to do that on its own.
- 36:21SPEAKER_03: That's fundamentally why it takes longer.
- 36:24SPEAKER_03: You can think of it as rewriting things.
- 36:26SPEAKER_03: You can think of it as looking in two directions as opposed to just linearly only.
- 36:32SPEAKER_03: But yes, on a lot of problems, especially on easy problems, you don't really need it.
- 36:37SPEAKER_03: It's not necessarily better.
- 36:39SPEAKER_03: And also the other thing is how would,
- 36:40SPEAKER_03: you know,
- 36:41SPEAKER_03: how do they necessarily generate the right training data,
- 36:43SPEAKER_03: right?
- 36:44SPEAKER_03: It's not that obvious how the rewriting should happen.
- 36:46SPEAKER_03: I mean, that's something we have literally no training data at all.
- 36:49SPEAKER_03: So they had to do some work.
- 36:51SPEAKER_03: I mean, some people will say, will give more abstract ways.
- 36:54SPEAKER_03: They'll say, oh, you give it like thinking tokens.
- 36:56SPEAKER_03: It basically can just,
- 36:57SPEAKER_03: and at some level,
- 36:58SPEAKER_03: this is true,
- 36:59SPEAKER_03: you know,
- 37:00SPEAKER_03: just by having a bigger budget for certain answers that matter more.
- 37:03SPEAKER_03: you know, you can sort of like think harder, right?
- 37:05SPEAKER_03: The classic example is you give it as complicated math equation and you're like, give us the answer.
- 37:10SPEAKER_03: Well, if it just gives you the answer, it's not thinking very hard.
- 37:13SPEAKER_03: Whereas if you work through the problem,
- 37:15SPEAKER_03: you're essentially giving it a bigger compute budget to solve the problem.
- 37:18SPEAKER_03: So even like very simple instructions like that were helpful,
- 37:20SPEAKER_03: but obviously be more structured around it,
- 37:23SPEAKER_03: giving it examples of solving problems,
- 37:25SPEAKER_03: simulating that,
- 37:25SPEAKER_03: et cetera,
- 37:26SPEAKER_03: you know,
- 37:26SPEAKER_03: makes a whole lot of sense.
- 37:27SPEAKER_03: So I think, I think, I think one was actually a pretty big success.
- 37:33SPEAKER_03: For complicated tasks,
- 37:34SPEAKER_03: like again,
- 37:34SPEAKER_03: now it's writing three,
- 37:35SPEAKER_03: four paragraph articles for us,
- 37:36SPEAKER_03: which are like massively more intelligent.
- 37:39SPEAKER_03: It's making fewer mistakes.
- 37:41SPEAKER_03: It's taking something.
- 37:42SPEAKER_03: This is an example.
- 37:43SPEAKER_03: You'll have a bunch of tweets about like the polls during the election,
- 37:46SPEAKER_03: say like the last weekend,
- 37:47SPEAKER_03: you know,
- 37:47SPEAKER_03: are the polls good for,
- 37:48SPEAKER_03: you know,
- 37:49SPEAKER_03: Kamala or not,
- 37:49SPEAKER_03: right?
- 37:50SPEAKER_03: And you would see a bunch of tweets and it would synthesize that into like a very
- 37:52SPEAKER_03: nice three,
- 37:53SPEAKER_03: four paragraph thing that was accurate,
- 37:55SPEAKER_03: intelligent,
- 37:56SPEAKER_03: tied well together.
- 37:57SPEAKER_03: So I think before a system like that, you couldn't do that.
- 37:59SPEAKER_03: You're not asking it to do that.
- 38:02SPEAKER_03: Like to your point,
- 38:03SPEAKER_03: for a thousand words or even for 200 words,
- 38:06SPEAKER_03: I think you would expect it to do a much better job than GPT-40.
- 38:11SPEAKER_03: Yeah.
- 38:12SPEAKER_03: But it's not as polished, right?
- 38:14SPEAKER_03: GPT-40 is being polished.
- 38:16SPEAKER_03: So that's like closer to its optimal, whereas O1 is like probably like extremely not optimized.
- 38:21SPEAKER_03: Yeah.
- 38:22SPEAKER_00: Do you guys use Claude at all for deep news, just out of curiosity?
- 38:25SPEAKER_03: We've experimented with it, I think.
- 38:28SPEAKER_03: And again, on our specific use case, it's weird.
- 38:30SPEAKER_03: So the first model, I forget which one, the Sonnet 3, whatever.
- 38:35SPEAKER_03: Well,
- 38:35SPEAKER_03: the first one that was like beating GPT on LMSYS,
- 38:38SPEAKER_03: which is a bit slow and very expensive,
- 38:41SPEAKER_03: was very good.
- 38:42SPEAKER_03: And then the next one,
- 38:43SPEAKER_03: which was faster and people liked it for coding,
- 38:46SPEAKER_03: I thought was actually very bad at writing news articles,
- 38:48SPEAKER_03: but mainly because its default tone was like too chatty.
- 38:51SPEAKER_03: It was like turning everything into like a Reddit post.
- 38:53SPEAKER_03: And I'm like, that's what we want.
- 38:55SPEAKER_03: By the way, we give it the same instructions.
- 38:56SPEAKER_03: We ask,
- 38:57SPEAKER_03: you know,
- 38:58SPEAKER_03: the articles to be written in a factual and sort of,
- 39:01SPEAKER_03: you know,
- 39:02SPEAKER_03: like an objective matter,
- 39:04SPEAKER_03: kind of like in a style of something like Bloomberg,
- 39:05SPEAKER_03: et cetera.
- 39:06SPEAKER_03: And it just wasn't able to follow that instruction for whatever reason.
- 39:09SPEAKER_03: So I think as we saw,
- 39:11SPEAKER_03: we've seen quite a bit of differentiation there,
- 39:12SPEAKER_03: specifically between GPT and Claude,
- 39:17SPEAKER_03: where they'll start optimizing for different things.
- 39:19SPEAKER_03: Yeah.
- 39:21SPEAKER_03: You know, Claude optimized for code.
- 39:23SPEAKER_03: I think a lot of people who code like it,
- 39:26SPEAKER_03: you know,
- 39:27SPEAKER_03: but even for GPT,
- 39:28SPEAKER_03: like the version that you're getting in the sort of user interface is different
- 39:33SPEAKER_03: than the version I'm getting in the API.
- 39:35SPEAKER_03: And I can't tell you why, but it's, you know, that it's different.
- 39:38SPEAKER_00: Well,
- 39:38SPEAKER_00: I will tell you guys,
- 39:39SPEAKER_00: tell you listeners out there,
- 39:41SPEAKER_00: when Claude tries to,
- 39:43SPEAKER_00: or GPT tries to write like me,
- 39:45SPEAKER_00: it's a more pedestrian,
- 39:47SPEAKER_00: straight ahead,
- 39:49SPEAKER_00: fact-rich essay than Claude,
- 39:51SPEAKER_00: which is much more writerly.
- 39:52SPEAKER_00: I don't know why.
- 39:53SPEAKER_00: Yeah, it's kind of bullshitty, yes.
- 39:56SPEAKER_00: Yeah,
- 39:56SPEAKER_00: Claude has lower data density in its responses,
- 39:59SPEAKER_00: but it seems more like that would be something I write,
- 40:02SPEAKER_00: if that makes sense.
- 40:03SPEAKER_03: Yeah, it's like more conversational, which can be good, but I think for us is like not good at all.
- 40:09SPEAKER_03: You know, you have very serious topics and I would almost like try to make fun of them.
- 40:14SPEAKER_00: Yeah, yeah.
- 40:15SPEAKER_00: So, you know, there's been all sorts of things that have happened over the last couple of years.
- 40:19SPEAKER_00: And I want to like, you know, talk about some of them in terms of what's going on.
- 40:22SPEAKER_00: This guy like Leo Aschenbacher, I think.
- 40:28SPEAKER_00: uh, Austin Brenner, Austin Brenner, he wrote, um, you know, he follows me on Twitter.
- 40:34SPEAKER_00: I should have him on to explain, but he wrote something called situational awareness for open AI.
- 40:38SPEAKER_00: And then, you know, he's basically fired from open AI.
- 40:41SPEAKER_00: And he was like on all these podcasts, like door caches podcasts.
- 40:44SPEAKER_00: And, um, it somehow got to Donald Trump.
- 40:48SPEAKER_00: Uh, and he talked, he alluded to situational awareness.
- 40:52SPEAKER_00: Um,
- 40:53SPEAKER_00: There have been arguments about whether – I mean,
- 40:56SPEAKER_00: look,
- 40:57SPEAKER_00: I haven't listened to the whole thing,
- 40:58SPEAKER_00: but Dario Amidai of Anthropic has like a five-year roadmap or four-year roadmap
- 41:02SPEAKER_00: about AGI and all this stuff.
- 41:06SPEAKER_00: This is like percolating into the policy sphere.
- 41:10SPEAKER_00: Elon Musk, who is – let's just call him the first friend.
- 41:14SPEAKER_00: um right now he's very involved in policy making in the trump world and he is a
- 41:20SPEAKER_00: little bit of what we would call a d-cell uh he had a famous falling out with open
- 41:24SPEAKER_00: ai uh mostly because he's not sam altman and crew or well it was a non-profit they
- 41:29SPEAKER_00: were going to turn into a profit obviously and he's he's executed on that now sam
- 41:33SPEAKER_00: altman's pushed everyone else out uh since we last talked it's just him uh uh and
- 41:38SPEAKER_00: um
- 41:39SPEAKER_00: Anyway,
- 41:40SPEAKER_00: so we have someone who has opinions on artificial intelligence,
- 41:42SPEAKER_00: Elon Musk,
- 41:43SPEAKER_00: at the center of power,
- 41:44SPEAKER_00: right?
- 41:45SPEAKER_00: So we've been talking about this as technology.
- 41:48SPEAKER_00: We've been talking about this private sector.
- 41:50SPEAKER_00: But there is a national security geopolitical aspect.
- 41:54SPEAKER_00: It looks like China and the United States are the ones that are really producing
- 41:58SPEAKER_00: these models,
- 41:59SPEAKER_00: these LLMs,
- 42:00SPEAKER_00: that are pushing the frontier of artificial intelligence.
- 42:05SPEAKER_00: And are you worried...
- 42:08SPEAKER_00: You're in the field as a business owner.
- 42:11SPEAKER_00: You're looking for partnerships.
- 42:13SPEAKER_00: You get some visibility, so you have to get a general sense.
- 42:18SPEAKER_00: Are people worried about the government getting involved?
- 42:22SPEAKER_00: I will tell you, I have heard people in the Bay Area tell me
- 42:27SPEAKER_00: I mean I'm not going to say who it is,
- 42:29SPEAKER_00: but someone who's not a trivially – not an unserious person told me that the Trump
- 42:37SPEAKER_00: administration was thinking about trying to nationalize open AI.
- 42:39SPEAKER_00: I'm just going to say that.
- 42:40SPEAKER_00: That was something else, and I was like, how would you even do that?
- 42:43SPEAKER_00: There are some laws.
- 42:44SPEAKER_00: There are some things that you could probably try, but why pick open AI?
- 42:48SPEAKER_00: Why not pick anthra?
- 42:49SPEAKER_00: There's just all sorts of questions, and it seems to me – okay, this is what I'm going to say.
- 42:54SPEAKER_00: For a couple of years, I just want the government to keep its nose out of it.
- 42:57SPEAKER_00: That's my opinion because I don't think people understand what's going on.
- 43:00SPEAKER_00: On the other hand,
- 43:02SPEAKER_00: some people might say that's very naive and maybe we could create some monstrosity
- 43:07SPEAKER_00: and it'll be like – what is that –
- 43:10SPEAKER_00: Mila Jojovic is in those series Resident Evil,
- 43:14SPEAKER_00: like some evil corporation could create some monster,
- 43:18SPEAKER_00: artificial intelligence monster.
- 43:19SPEAKER_00: Maybe some people would say that.
- 43:20SPEAKER_00: But anyway, I've been talking for a while.
- 43:21SPEAKER_00: What do you think about that in terms of policy people,
- 43:26SPEAKER_00: the government and laws getting involved in artificial intelligence?
- 43:30SPEAKER_03: Yeah, I mean, as you can tell from what you're saying, it's all very abstract, right?
- 43:35SPEAKER_03: I mean,
- 43:36SPEAKER_03: the main thing is I've never seen something with as much fluff and buzzwords as
- 43:41SPEAKER_03: this whole discussion.
- 43:44SPEAKER_03: But at the same time, I think it's also inevitable, you know?
- 43:46SPEAKER_03: Yeah.
- 43:48SPEAKER_03: I think,
- 43:48SPEAKER_03: I think even if it's just the next technology,
- 43:52SPEAKER_03: um,
- 43:52SPEAKER_03: as you can see that the government will meddle in the technology,
- 43:56SPEAKER_03: it sort of is what it is.
- 43:57SPEAKER_03: I think that's not something that's a big deal.
- 43:59SPEAKER_00: The government's going to get involved a hundred percent.
- 44:01SPEAKER_03: And I, and I think, you know, being a libertarian about it, isn't going to change anything.
- 44:06SPEAKER_03: You know, I think the government will get involved.
- 44:08SPEAKER_03: I think at the same time, people will do their thing.
- 44:11SPEAKER_03: You know,
- 44:11SPEAKER_03: I think both can be sort of forces that are true at the same time as sort of a yin
- 44:15SPEAKER_03: yang kind of thing.
- 44:16SPEAKER_03: Right.
- 44:17SPEAKER_03: Um,
- 44:17SPEAKER_03: Like,
- 44:18SPEAKER_03: you know,
- 44:19SPEAKER_03: the government tries to ban NVIDIA chips,
- 44:21SPEAKER_03: so people just sneak them in their luggage,
- 44:22SPEAKER_03: you know,
- 44:23SPEAKER_03: and yeah,
- 44:24SPEAKER_03: the government tries to censor things so they get other stuff like it's just I
- 44:28SPEAKER_03: think the government can encourage and discourage things,
- 44:31SPEAKER_03: suppress them to some extent.
- 44:32SPEAKER_03: But I think, you know, completely dictating is like it's not competent enough to do that.
- 44:36SPEAKER_03: Right.
- 44:37SPEAKER_03: But at the same time, of course, it's going to have some sort of some sort of policy.
- 44:40SPEAKER_03: Right.
- 44:41SPEAKER_03: And I think it
- 44:44SPEAKER_03: if candidates should be at some sort of federal level,
- 44:46SPEAKER_03: like to change the focus from AI for a second,
- 44:49SPEAKER_03: you know,
- 44:50SPEAKER_03: as part of some of the election stuff,
- 44:52SPEAKER_03: I mean,
- 44:52SPEAKER_03: you saw states on both sides,
- 44:53SPEAKER_03: both like very blue states and very red states try to like push legislation to,
- 44:58SPEAKER_03: you know,
- 44:59SPEAKER_03: to regulate like Facebook at the state level.
- 45:00SPEAKER_03: And it's like,
- 45:02SPEAKER_03: Like, what are you doing?
- 45:03SPEAKER_03: It's not going to happen.
- 45:04SPEAKER_03: It's not going to work.
- 45:05SPEAKER_03: Like even doing it at the national level is already challenging enough.
- 45:08SPEAKER_03: But like, OK, so different states are going to have like different.
- 45:12SPEAKER_03: That's just not where we're going, you know, and I'm not against localism at all.
- 45:16SPEAKER_03: But it just doesn't really make sense for Idaho to have different Facebook laws than New York.
- 45:21SPEAKER_00: Yeah.
- 45:22SPEAKER_03: It's just kind of a little bit silly.
- 45:24SPEAKER_00: So I think it is silly because they're not there.
- 45:26SPEAKER_00: They're in cyberspace.
- 45:27SPEAKER_00: They're not like physical space.
- 45:28SPEAKER_03: And no one no one really thinks of them as different places.
- 45:31SPEAKER_03: You know, like once you're online, you're online.
- 45:34SPEAKER_03: Yeah.
- 45:34SPEAKER_03: Yeah.
- 45:35SPEAKER_03: When you're in a different country, when you go to Argentina, you use the Internet.
- 45:38SPEAKER_03: You are aware that you're maybe like in a different space and it's not quite the same Internet.
- 45:42SPEAKER_03: You know?
- 45:42SPEAKER_03: Yeah.
- 45:44SPEAKER_03: But if you're if you cross the border from California to Nevada and the rules change,
- 45:48SPEAKER_03: like unless we're talking about online gambling,
- 45:49SPEAKER_03: it doesn't really make sense.
- 45:51SPEAKER_03: Yeah.
- 45:53SPEAKER_03: So I think it's good that the government has people who are just technically competent.
- 45:59SPEAKER_03: You know, I think that's a good thing.
- 46:01SPEAKER_03: I think I also know a bunch of AI policy people just socially.
- 46:05SPEAKER_03: I have no idea what they're doing.
- 46:09SPEAKER_03: I find their reading to be very complicated and fluffy as well, even though they do this for a living.
- 46:14SPEAKER_01: Yeah.
- 46:15SPEAKER_03: So I think that there's a...
- 46:17SPEAKER_03: you know, it's a little bit, it's a little bit putting the cart before the horse, right?
- 46:22SPEAKER_03: I mean,
- 46:23SPEAKER_03: the stuff doesn't,
- 46:24SPEAKER_03: to your point,
- 46:24SPEAKER_03: hasn't really transformed the economy,
- 46:26SPEAKER_03: except,
- 46:26SPEAKER_03: you know,
- 46:27SPEAKER_03: putting a few very specific sectors kind of down 90%.
- 46:30SPEAKER_03: And, you know, that is going to happen with any technology.
- 46:33SPEAKER_00: So yeah, I mean, we're gonna, I mean, we're gonna go through multiple hype cycles.
- 46:37SPEAKER_00: You know,
- 46:38SPEAKER_00: I feel like we're not like on the downswing,
- 46:41SPEAKER_00: like we're not having like think pieces like,
- 46:43SPEAKER_00: oh,
- 46:43SPEAKER_00: AI has been a bust.
- 46:44SPEAKER_00: Because it hasn't.
- 46:45SPEAKER_00: I mean,
- 46:45SPEAKER_00: I think,
- 46:46SPEAKER_00: you know,
- 46:47SPEAKER_00: as we already said,
- 46:48SPEAKER_00: it's already impacted our lives and we're not like that atypical in terms of just
- 46:53SPEAKER_00: using it in our day to day.
- 46:54SPEAKER_00: You've created a business with it.
- 46:56SPEAKER_00: There are other people.
- 46:56SPEAKER_00: I mean, I know other people personally who have created various businesses.
- 47:00SPEAKER_00: Usually they're like chatbots, assistants, like these sorts of things.
- 47:03SPEAKER_00: Yeah.
- 47:04SPEAKER_00: Yeah.
- 47:06SPEAKER_03: And here's the question.
- 47:07SPEAKER_03: I mean, are any of them really making big money?
- 47:09SPEAKER_03: I mean, we know the answer.
- 47:10SPEAKER_03: They're not.
- 47:11SPEAKER_03: I mean, there's a few small exceptions.
- 47:13SPEAKER_03: So look, in that way, I think the most obvious analogy is to the chip industry, you know?
- 47:18SPEAKER_03: So have you read Chip Wars?
- 47:22SPEAKER_03: Yes, yes.
- 47:23SPEAKER_03: Yeah, very good book.
- 47:24SPEAKER_03: I think a lot of your readers have.
- 47:25SPEAKER_03: If not, I would highly recommend it.
- 47:27SPEAKER_03: With the one recommendation that read the second half first,
- 47:32SPEAKER_03: because otherwise you're going to be stuck in a lot of interesting stories from the 50s,
- 47:35SPEAKER_03: 60s and 70s,
- 47:35SPEAKER_03: which are awesome,
- 47:37SPEAKER_03: but like don't have that much bearing on today.
- 47:38SPEAKER_03: But anyway, great.
- 47:40SPEAKER_03: And actually,
- 47:40SPEAKER_03: if you look at those stories,
- 47:41SPEAKER_03: I mean,
- 47:41SPEAKER_03: you know,
- 47:42SPEAKER_03: all of these fledgling chip companies,
- 47:43SPEAKER_03: you know,
- 47:44SPEAKER_03: when they were interesting,
- 47:45SPEAKER_03: I mean,
- 47:46SPEAKER_03: they were constantly going out of business and running out of money.
- 47:48SPEAKER_03: running to the government, doing deals, having international stuff with Japan at the time.
- 47:53SPEAKER_03: So I think a lot of these stories are actually kind of familiar if you look at it, right?
- 47:56SPEAKER_03: Where if you look at these companies, it's obviously a huge industry.
- 47:59SPEAKER_03: These companies are worth a lot.
- 48:01SPEAKER_03: None of them are like printing cash, really, partly because they're in competition.
- 48:04SPEAKER_03: I mean, even OpenAI and Anthropic are like still losing money.
- 48:08SPEAKER_03: um doesn't mean we wouldn't all you know line up to give them money as shareholders
- 48:13SPEAKER_03: but you know it's just one of those things where it's still and i think that's
- 48:18SPEAKER_03: probably why you know the government is going to have some hand to play like it's a
- 48:22SPEAKER_03: very weird market where even as a new entrant you basically have to throw a lot of
- 48:26SPEAKER_03: money at it to be competitive but you know but but like to be competitive in an
- 48:31SPEAKER_03: industry that frankly doesn't even make money like for example tiktok
- 48:35SPEAKER_03: spend billions of dollars to break into the social media game.
- 48:39SPEAKER_03: That is a very profitable business.
- 48:42SPEAKER_03: Now,
- 48:42SPEAKER_03: you can argue whether they did it for the money or for the clout,
- 48:46SPEAKER_03: but they're making a lot of money through ads now.
- 48:48SPEAKER_03: like a tremendous amount.
- 48:50SPEAKER_03: Yeah.
- 48:50SPEAKER_03: Whereas in AI,
- 48:51SPEAKER_03: like it's almost like fair to say that,
- 48:53SPEAKER_03: you know,
- 48:53SPEAKER_03: the refrain is,
- 48:54SPEAKER_03: you know,
- 48:54SPEAKER_03: NVIDIA is the only company that makes any money on AI.
- 48:57SPEAKER_03: And so far that has been true.
- 48:59SPEAKER_03: Like NVIDIA is making all the money, basically.
- 49:02SPEAKER_03: Maybe some of the server companies, but certainly not the downstream companies.
- 49:06SPEAKER_03: But it's okay, because it's early.
- 49:07SPEAKER_03: They're fighting for market share, and it's worked out very well for OpenAI.
- 49:12SPEAKER_03: At this point, Claude can release a better model, and it wouldn't really matter.
- 49:16SPEAKER_03: They're not going to really cut into OpenAI's market share.
- 49:21SPEAKER_00: We've talked about this.
- 49:23SPEAKER_00: Really,
- 49:24SPEAKER_00: the artificial intelligence current hype cycle,
- 49:26SPEAKER_00: current era,
- 49:27SPEAKER_00: has only really produced one brand name that the public recognizes,
- 49:31SPEAKER_00: and that's OpenAI.
- 49:31SPEAKER_00: That's ChatGBT.
- 49:33SPEAKER_00: Like nerds like us, no Anthropic and all these other companies.
- 49:37SPEAKER_03: One caveat, one caveat.
- 49:38SPEAKER_03: So if you look at the,
- 49:40SPEAKER_03: I wish I had the chart,
- 49:40SPEAKER_03: but if you,
- 49:42SPEAKER_03: you know,
- 49:42SPEAKER_03: again,
- 49:43SPEAKER_03: Claude and GPT have really kind of forked a bit,
- 49:46SPEAKER_03: which is always going to happen when you have sort of close competitors,
- 49:49SPEAKER_03: almost through randomness,
- 49:50SPEAKER_03: they end up focusing on different things and you end up with like,
- 49:52SPEAKER_03: you know,
- 49:53SPEAKER_03: you know one company's making like a saturn and one company's making a volvo you
- 49:57SPEAKER_03: know almost to like randomness right like they find different pockets right um and
- 50:02SPEAKER_03: so so so chat gpt open ai is completely dominant in the consumer space by anthropic
- 50:10SPEAKER_03: is very is very um is very competitive in the api space so their api revenue is
- 50:15SPEAKER_03: something like half of open ais which is
- 50:17SPEAKER_03: Great.
- 50:18SPEAKER_03: Whereas their,
- 50:18SPEAKER_03: you know,
- 50:19SPEAKER_03: their their consumer revenue is like one one thousandth of open AIs,
- 50:23SPEAKER_03: like basically zero.
- 50:25SPEAKER_03: So we've kind of fallen into this thing where probably largely because of the
- 50:29SPEAKER_03: developers and the vibes,
- 50:31SPEAKER_03: you know,
- 50:31SPEAKER_03: they have,
- 50:33SPEAKER_03: I would say,
- 50:33SPEAKER_03: yes,
- 50:34SPEAKER_03: while most people in the public have heard of ChatGPT,
- 50:37SPEAKER_03: they have not heard of Anthropic and they haven't heard of Perplexity.
- 50:39SPEAKER_03: I think on the sort of API developer user side,
- 50:43SPEAKER_03: I mean,
- 50:43SPEAKER_03: Anthropic is,
- 50:44SPEAKER_03: I mean,
- 50:44SPEAKER_03: it's the more famous,
- 50:45SPEAKER_03: it's the more popular one in Berkeley,
- 50:47SPEAKER_03: but it's also just well-known among developers.
- 50:50SPEAKER_00: Yeah.
- 50:50SPEAKER_00: And like perplexity, you mentioned perplexity.
- 50:52SPEAKER_00: A lot of listeners are not going to know what perplexity is.
- 50:55SPEAKER_00: Can you tell them what that is?
- 50:56SPEAKER_03: Yeah.
- 50:57SPEAKER_03: So perplexity also started pretty early on.
- 50:59SPEAKER_03: They were doing a bunch of other things before,
- 51:00SPEAKER_03: but basically where they've settled into is they are sort of the Google competitor
- 51:05SPEAKER_03: as a search engine.
- 51:05SPEAKER_03: So it's just an AI search engine.
- 51:06SPEAKER_03: That's all it really is.
- 51:08SPEAKER_03: Um,
- 51:09SPEAKER_03: It actually licenses their results from Google,
- 51:12SPEAKER_03: which some people think is like a big gotcha,
- 51:14SPEAKER_03: but it's not.
- 51:14SPEAKER_03: Basically,
- 51:15SPEAKER_03: it's taking very similar results,
- 51:17SPEAKER_03: top 30 results from Google and organizing in a very different way.
- 51:21SPEAKER_03: You know, I use it, you use it.
- 51:22SPEAKER_03: A lot of people we know use it.
- 51:23SPEAKER_03: And it's good for,
- 51:24SPEAKER_03: you know,
- 51:24SPEAKER_03: I wouldn't say I would use it all the time,
- 51:26SPEAKER_03: but for like researchy type queries,
- 51:28SPEAKER_03: they do give responses,
- 51:29SPEAKER_03: generally speaking,
- 51:30SPEAKER_03: that are better than Google.
- 51:31SPEAKER_03: Yeah, at least in some cases.
- 51:34SPEAKER_03: And some people like the UI and have a nice app and they're getting into more stuff.
- 51:39SPEAKER_03: But that's the main thing.
- 51:41SPEAKER_03: But,
- 51:41SPEAKER_03: you know,
- 51:42SPEAKER_03: I mean,
- 51:42SPEAKER_03: I looked at I tried to find their search volume traffic on just like stats in the
- 51:47SPEAKER_03: US and it's like it doesn't even register.
- 51:49SPEAKER_03: Neither does serve GPT.
- 51:51SPEAKER_03: So then OpenAI basically tried to compete directly with perplexity and.
- 51:55SPEAKER_03: I mean, when's the last time you heard someone even mention that?
- 51:58SPEAKER_03: So all of these search competitors,
- 52:01SPEAKER_03: they're there,
- 52:02SPEAKER_03: but they haven't really made anything closely resembling to them than Google.
- 52:05SPEAKER_03: Now,
- 52:05SPEAKER_03: obviously,
- 52:06SPEAKER_03: Google has started using a lot more search,
- 52:08SPEAKER_03: a lot more AI in their searches,
- 52:10SPEAKER_03: which I think recently has gotten better.
- 52:12SPEAKER_03: I've been having some lower back problems, some sciatica, and so I've been Googling about that a lot.
- 52:18SPEAKER_03: And I think Google has started doing a better job of surfacing good Reddit threads and
- 52:24SPEAKER_03: pulling out answers and things like that.
- 52:25SPEAKER_03: And, you know, would they have done that if not for competition for complexity?
- 52:28SPEAKER_03: I don't know.
- 52:29SPEAKER_03: Maybe, maybe not.
- 52:31SPEAKER_03: I think their problem is,
- 52:32SPEAKER_03: again,
- 52:33SPEAKER_03: as you said,
- 52:34SPEAKER_03: ChatGPT launched at the right time with the right product,
- 52:36SPEAKER_03: really hit the thing,
- 52:38SPEAKER_03: and developed a consumer brand that is still popular,
- 52:41SPEAKER_03: even if people don't use it much.
- 52:42SPEAKER_03: I'm sure many people are like me, have the consumer $20 a month and never use it.
- 52:47SPEAKER_00: Yeah, and I don't drink Coke or Pepsi very much, but I know them.
- 52:52SPEAKER_00: It's at that stage where if you do know something about AI and you're just an average civilian,
- 52:58SPEAKER_00: You can blurt out that name.
- 53:00SPEAKER_03: Exactly.
- 53:02SPEAKER_03: You may not use head and shoulders either, but if someone's like, hey, here's a bunch of shampoos.
- 53:07SPEAKER_03: You're in Thailand and they have crazy lettering on them.
- 53:11SPEAKER_03: You might take the head and shoulders.
- 53:13SPEAKER_03: Same as you may drink a Coke,
- 53:14SPEAKER_03: even if you don't normally drink Coke,
- 53:16SPEAKER_03: but you're like,
- 53:16SPEAKER_03: I don't know what the hell these other things are.
- 53:17SPEAKER_03: Exactly.
- 53:18SPEAKER_03: So, so they, so I think perplexity tried, I mean, this is just my outside analysis.
- 53:24SPEAKER_03: They tried to get some sort of foothold and,
- 53:26SPEAKER_03: you know,
- 53:27SPEAKER_03: in the consumer search space and I'm not sure they've made much progress,
- 53:31SPEAKER_03: but I think the product's pretty good.
- 53:33SPEAKER_00: Yeah.
- 53:34SPEAKER_00: Yeah.
- 53:34SPEAKER_00: Um,
- 53:34SPEAKER_00: so I guess like I want to close,
- 53:37SPEAKER_00: I want to,
- 53:37SPEAKER_00: um,
- 53:38SPEAKER_00: close kind of back with,
- 53:40SPEAKER_00: uh,
- 53:41SPEAKER_00: with deep news and as you're developing it,
- 53:45SPEAKER_00: um,
- 53:45SPEAKER_00: look,
- 53:46SPEAKER_00: I mean,
- 53:46SPEAKER_00: your startup,
- 53:47SPEAKER_00: uh,
- 53:47SPEAKER_00: you know,
- 53:48SPEAKER_00: You know, computes always an issue.
- 53:51SPEAKER_00: The underlying models are changing somewhat because I'm assuming you didn't initially start out with 01.
- 53:57SPEAKER_03: Well, no, we started with the GPT three and a half because that was the only thing available.
- 54:01SPEAKER_00: Yeah, yeah.
- 54:03SPEAKER_03: Look, we've ridden that wave.
- 54:04SPEAKER_03: I mean, that part has improved and you knew that from the beginning.
- 54:07SPEAKER_03: Yeah.
- 54:07SPEAKER_03: But I think,
- 54:08SPEAKER_03: look,
- 54:09SPEAKER_03: I mean,
- 54:09SPEAKER_03: I'll tell you the main thing that we're dealing with now,
- 54:11SPEAKER_03: which,
- 54:11SPEAKER_03: I mean,
- 54:11SPEAKER_03: you know this,
- 54:12SPEAKER_03: but,
- 54:12SPEAKER_03: you know,
- 54:13SPEAKER_03: the issue is awareness,
- 54:14SPEAKER_03: right?
- 54:14SPEAKER_03: We have the same issue essentially as perplexity and everybody else,
- 54:17SPEAKER_03: which is that,
- 54:18SPEAKER_03: you know,
- 54:18SPEAKER_03: people have heard of the New York Times or they've even heard of X or Twitter,
- 54:22SPEAKER_03: even if they're not.
- 54:24SPEAKER_03: We love X, but I mean, it's a confusing place to get your news because you get the whole everything.
- 54:29SPEAKER_03: Yeah.
- 54:30SPEAKER_03: Yeah.
- 54:32SPEAKER_03: But yeah, of course, like, you know, we don't have a brand.
- 54:34SPEAKER_03: And I think there have been a few news aggregators,
- 54:38SPEAKER_03: which I guess we sort of are and sort of are not,
- 54:40SPEAKER_03: that have developed their own brand,
- 54:41SPEAKER_03: but not that many,
- 54:42SPEAKER_03: right?
- 54:42SPEAKER_03: Like Drudge was big for a while until they went bad.
- 54:46SPEAKER_03: Tech meme is sort of a thing, you know?
- 54:48SPEAKER_03: So I think there's just a lot of...
- 54:51SPEAKER_03: I think that we are,
- 54:53SPEAKER_03: you know,
- 54:54SPEAKER_03: trying to figure out how to,
- 54:56SPEAKER_03: you know,
- 54:57SPEAKER_03: how to how to reach people as a consumer product.
- 54:58SPEAKER_03: And I think the product is better than people think.
- 55:00SPEAKER_03: You know, it's quite it's quite compelling as is not not even to mention, you know, the obvious value.
- 55:06SPEAKER_00: I mean, I think it adds value.
- 55:07SPEAKER_00: I mean, you know, I'm biased, but like I think it definitely adds value.
- 55:09SPEAKER_00: And so, you know, we're talking about the AI revolution.
- 55:13SPEAKER_00: We kind of went on abstract.
- 55:14SPEAKER_00: You know, I pushed it to abstract layer sometimes, but the reality is it's here.
- 55:19SPEAKER_00: there are products that people are using your people are using your product there's
- 55:23SPEAKER_00: other things related to translation services i mean that's just like a a gimme that
- 55:27SPEAKER_00: people are using i know that colin wright uh reality last name i mean he had an
- 55:31SPEAKER_00: editor he got rid of an editor he just uses gpt now to fix things you know that's
- 55:37SPEAKER_03: great yes and look and again i mean not not knowing colin at all i can i can
- 55:41SPEAKER_03: imagine why the reason is probably not the cost
- 55:44SPEAKER_03: It's probably not even the quality.
- 55:45SPEAKER_03: It's like, hey, this editor, like you don't have to email somebody.
- 55:48SPEAKER_03: Like they're not going to be late.
- 55:50SPEAKER_03: Convenience.
- 55:52SPEAKER_03: And consistency, right?
- 55:53SPEAKER_03: So I remember when I was at Google 20 years ago,
- 55:56SPEAKER_03: the head of search,
- 55:57SPEAKER_03: Amit Singhal,
- 55:58SPEAKER_03: he would always,
- 55:58SPEAKER_03: you know,
- 55:58SPEAKER_03: one of the things he would always say to the group was,
- 56:01SPEAKER_03: okay,
- 56:01SPEAKER_03: if there was like a,
- 56:02SPEAKER_03: you know,
- 56:02SPEAKER_03: like a,
- 56:03SPEAKER_03: you know,
- 56:03SPEAKER_03: like a,
- 56:03SPEAKER_03: like a,
- 56:04SPEAKER_03: you know,
- 56:04SPEAKER_03: the mechanical Turk,
- 56:05SPEAKER_03: the man inside the machine.
- 56:06SPEAKER_01: Yeah.
- 56:07SPEAKER_03: Your job is ranking the results.
- 56:09SPEAKER_03: But I'm going to do this fast.
- 56:11SPEAKER_03: Yeah.
- 56:12SPEAKER_03: and consistently and that's it and that was worth doing you know and it still is so
- 56:18SPEAKER_03: i i think look i mean i think to sum some of this stuff up you know to for the
- 56:22SPEAKER_03: famous you know churchill quote right i mean this is the end of the beginning you
- 56:26SPEAKER_03: know this is still early um you know we're seeing some great companies and
- 56:30SPEAKER_03: definitely a few very popular niche products um
- 56:34SPEAKER_03: A lot of very useful tools.
- 56:35SPEAKER_03: I think I'm not a fan of destroying old grade businesses like Stack Overflow,
- 56:40SPEAKER_03: but I think it is showing that some things are better and are being replaced.
- 56:46SPEAKER_03: And then we'll see.
- 56:47SPEAKER_03: I mean, this will take years.
- 56:51SPEAKER_03: And along the way,
- 56:52SPEAKER_03: I think the part that I'm excited about is really that second stage of a new
- 56:55SPEAKER_03: technology where you start doing things you couldn't imagine before.
- 56:59SPEAKER_03: You know, right now, still mostly, hey, AI is better at this than this other thing.
- 57:04SPEAKER_03: And I think the one case we've seen where it's a thing we didn't know we needed is
- 57:07SPEAKER_03: obviously the image generation.
- 57:08SPEAKER_03: We didn't know we needed to generate all these images and illustrate our pictures.
- 57:12SPEAKER_03: I mean, we sort of had it, but it's different now.
- 57:13SPEAKER_03: The vibe is completely different.
- 57:15SPEAKER_03: You're not like, ooh, I'm getting rid of my illustrator.
- 57:17SPEAKER_03: It's like, now I can just add little pictures to make my point.
- 57:21SPEAKER_03: But that's pretty small.
- 57:21SPEAKER_03: I mean, it's not nothing, but...
- 57:24SPEAKER_03: you know, imagine if Suno wrote amazing songs and you just want to listen to them all day long.
- 57:28SPEAKER_03: I mean, it's, it wouldn't, it wouldn't just, it wouldn't be about replacing your Spotify.
- 57:32SPEAKER_03: It would be like a new experience.
- 57:34SPEAKER_03: And then that way, I mean, you know, do I think we're there with deep news?
- 57:38SPEAKER_03: I mean, maybe not, but I certainly, you know, I like the autoplay in the morning.
- 57:43SPEAKER_03: Now,
- 57:43SPEAKER_03: instead of listening to a podcast or something like that,
- 57:46SPEAKER_03: I'll just press play and it'll just start reading me news stories.
- 57:48SPEAKER_03: And I pause when I find it interesting.
- 57:50SPEAKER_03: Maybe at times I wish it dragged it out a little bit more,
- 57:54SPEAKER_00: it's an interesting way to start the day yeah i mean it's just these sorts of tools
- 58:00SPEAKER_00: these sorts of apps they're kind of conveniences that slowly like step by steps
- 58:04SPEAKER_00: kind of scaffold our daily just experience of life and we don't even uh you know i
- 58:09SPEAKER_00: feel like you know a couple years from now even five years from now there's gonna
- 58:13SPEAKER_00: be all these sorts of things we're not gonna we're not gonna think about it we're
- 58:16SPEAKER_00: not gonna second guess it's gonna be part of our lives and i think one of the uh
- 58:19SPEAKER_03: Well, flipping through TikTok, right?
- 58:20SPEAKER_03: Like, I don't use TikTok.
- 58:21SPEAKER_03: I refuse.
- 58:22SPEAKER_03: But I guess I got the same experience through YouTube and Instagram.
- 58:26SPEAKER_03: And, like, I don't know.
- 58:28SPEAKER_03: YouTube especially, you'll go and you'll start flicking through things for 10 minutes.
- 58:32SPEAKER_03: I didn't know I wanted that.
- 58:34SPEAKER_03: If it was banned, like, Google would fight against the ban because they'll go somewhere else.
- 58:39SPEAKER_03: And you have people who emerge.
- 58:41SPEAKER_03: Like, who's the imam that answers questions that I just...
- 58:48SPEAKER_00: I forget his name, but he's really funny.
- 58:50SPEAKER_00: Yeah, he's great.
- 58:52SPEAKER_00: He gets really dumb Zoomer questions.
- 58:54SPEAKER_03: And he's sort of yelling at them.
- 58:56SPEAKER_03: It's like a comedy routine.
- 58:57SPEAKER_00: He's like, man, what do you think?
- 58:58SPEAKER_00: What are you asking that for?
- 59:00SPEAKER_00: He'll be like, isn't Halal to eat pig in a game?
- 59:04SPEAKER_00: He's like,
- 59:06SPEAKER_03: what it's a game i think for that one he was like well like he's like look look if
- 59:14SPEAKER_03: it's just energy for your character just go ahead and do it don't overthink it but
- 59:17SPEAKER_03: he's you love pig so now you want to go and order a bacon cheeseburger of course
- 59:21SPEAKER_03: that's haram you know
- 59:23SPEAKER_00: Yeah, that sort of stuff is funny.
- 59:26SPEAKER_03: It's great.
- 59:27SPEAKER_03: And again,
- 59:27SPEAKER_03: kind of like the companies that got created around Facebook,
- 59:30SPEAKER_03: you have these personalities and media styles and really language that's being
- 59:34SPEAKER_03: created around the internet and TikTok,
- 59:35SPEAKER_03: right?
- 59:36SPEAKER_03: So I think AI has had some influence on that for sure, but it's still early stages, right?
- 59:42SPEAKER_03: And in that way, I don't blame people.
- 59:44SPEAKER_03: So I'm obviously not a doomer, but I don't think their concerns are insane.
- 59:49SPEAKER_03: Right.
- 59:49SPEAKER_03: Like it is going to transform the society.
- 59:52SPEAKER_03: And maybe we should be like the Amish.
- 59:54SPEAKER_03: And, you know, what do they say?
- 59:55SPEAKER_03: Right.
- 59:55SPEAKER_03: The Amish aren't against society.
- 59:57SPEAKER_03: They just technology.
- 59:59SPEAKER_03: They just sort of like go one by one.
- 1:00:00SPEAKER_03: Like, do we want this one or not?
- 1:00:02SPEAKER_03: And individual communities make different choices.
- 1:00:05SPEAKER_03: And maybe that's what people want.
- 1:00:07SPEAKER_03: Maybe.
- 1:00:07SPEAKER_03: I mean, you see this in Japan, right?
- 1:00:09SPEAKER_03: Yeah.
- 1:00:09SPEAKER_03: Japan is very advanced and very technological in some ways,
- 1:00:13SPEAKER_03: not others,
- 1:00:13SPEAKER_03: but they explicitly as a society like decide they want certain things done not by machine,
- 1:00:19SPEAKER_03: even though they can.
- 1:00:20SPEAKER_03: I don't know how they reach these conclusions.
- 1:00:22SPEAKER_03: I would be very interested to find out.
- 1:00:24SPEAKER_00: Yeah.
- 1:00:25SPEAKER_00: Yeah,
- 1:00:26SPEAKER_00: well,
- 1:00:26SPEAKER_00: I mean,
- 1:00:27SPEAKER_00: I'm as excited as – I think I am as – yeah,
- 1:00:32SPEAKER_00: I am as excited about this technology as when I first talked to you.
- 1:00:36SPEAKER_00: I definitely left – I don't know if I was a doomer,
- 1:00:40SPEAKER_00: but,
- 1:00:40SPEAKER_00: I mean,
- 1:00:41SPEAKER_00: I'm a little bit more skeptical of a lot of the doomer stuff.
- 1:00:43SPEAKER_00: And I can't, like, put a finger on why, but I think it's just – Well, I haven't had a good record.
- 1:00:48SPEAKER_03: I think listening to them for two or three years, I think –
- 1:00:53SPEAKER_03: I think generally people have trended away from them for good reason.
- 1:00:58SPEAKER_03: And also, here's the thing.
- 1:01:00SPEAKER_03: It's the same reason that all the people who hate Elon are going to lose in the long run.
- 1:01:06SPEAKER_03: It's just not the American way.
- 1:01:07SPEAKER_03: Being a hater is not very American.
- 1:01:10SPEAKER_03: It's not an accident that people like Gary Marcus,
- 1:01:13SPEAKER_03: who at one point maybe did some useful things,
- 1:01:15SPEAKER_03: I don't really know.
- 1:01:16SPEAKER_03: But just being a hater and shouting at clouds is just not fun.
- 1:01:20SPEAKER_03: It's not interesting.
- 1:01:21SPEAKER_03: Yeah.
- 1:01:22SPEAKER_03: It's not even that entertaining, you know, unlike the Imam, who's very entertaining.
- 1:01:25SPEAKER_00: Yeah.
- 1:01:26SPEAKER_03: Yeah.
- 1:01:27SPEAKER_00: All right.
- 1:01:28SPEAKER_00: Yeah.
- 1:01:29SPEAKER_00: Yeah.
- 1:01:29SPEAKER_00: He is.
- 1:01:29SPEAKER_00: He is entertaining.
- 1:01:30SPEAKER_00: It's that's a definitely like interesting spinoff of like what.
- 1:01:35SPEAKER_00: what these new technologies are doing too,
- 1:01:36SPEAKER_00: in terms of kind of like the,
- 1:01:38SPEAKER_00: um,
- 1:01:39SPEAKER_00: you know,
- 1:01:39SPEAKER_00: traditionalist,
- 1:01:40SPEAKER_00: I mean,
- 1:01:40SPEAKER_00: there's all these things that we have part of our culture and they're being reborn
- 1:01:44SPEAKER_00: in different ways by the internet.
- 1:01:45SPEAKER_00: They have been by the internet,
- 1:01:47SPEAKER_00: you know,
- 1:01:47SPEAKER_00: and now the internet and now you have artificial intelligence and,
- 1:01:51SPEAKER_00: um,
- 1:01:52SPEAKER_00: you know,
- 1:01:53SPEAKER_00: I'm going to end here,
- 1:01:54SPEAKER_00: but it's just like 2,500 years ago with alphabetic writing,
- 1:01:58SPEAKER_00: uh,
- 1:01:59SPEAKER_00: which like allowed like much cheaper writing and,
- 1:02:02SPEAKER_00: you know,
- 1:02:02SPEAKER_00: like,
- 1:02:03SPEAKER_00: writing plays and all these other things easily.
- 1:02:06SPEAKER_00: That was a big, big discussion point in places like ancient Greece.
- 1:02:11SPEAKER_00: Like, how is that going to transform how we think and how we operate in the world?
- 1:02:16SPEAKER_00: And it did transform how we thought and how we operate in the world.
- 1:02:19SPEAKER_00: I mean, it wasn't bad, like some people thought, but it was a change.
- 1:02:25SPEAKER_00: And so I think this is like, to me, like, I feel like this is similar.
- 1:02:29SPEAKER_00: Okay.
- 1:02:30SPEAKER_00: Yeah.
- 1:02:30SPEAKER_00: There's going to be a lot of jobs.
- 1:02:32SPEAKER_00: uh, that are going to get destroyed.
- 1:02:34SPEAKER_00: I think that that's just reality, but I can see it myself.
- 1:02:37SPEAKER_00: I can see jobs that are like,
- 1:02:39SPEAKER_00: I don't see the point of that now or something,
- 1:02:41SPEAKER_00: but like the unemployment rate is still really low.
- 1:02:44SPEAKER_00: So people are getting new jobs.
- 1:02:46SPEAKER_00: You know, it's just like the magic of economics is still, still working in America.
- 1:02:50SPEAKER_03: I mean, it's just a lot more dynamic, right?
- 1:02:51SPEAKER_03: I mean,
- 1:02:51SPEAKER_03: obviously you,
- 1:02:52SPEAKER_03: you do have so,
- 1:02:53SPEAKER_03: some familial things,
- 1:02:54SPEAKER_03: you know,
- 1:02:54SPEAKER_03: um,
- 1:02:55SPEAKER_03: like,
- 1:02:55SPEAKER_03: like,
- 1:02:56SPEAKER_03: like,
- 1:02:56SPEAKER_03: like being a funeral director,
- 1:02:57SPEAKER_03: apparently you're sort of in that cast or you're not even in America.
- 1:03:00SPEAKER_03: Um,
- 1:03:01SPEAKER_03: Like no one starts a new funeral home.
- 1:03:03SPEAKER_03: It's always someone you inherit from your relatives.
- 1:03:06SPEAKER_03: And there's many jobs like that.
- 1:03:07SPEAKER_03: But I think most people in America, I mean, what are the stats?
- 1:03:10SPEAKER_03: They change careers like four or five times and people do all kinds of different things.
- 1:03:14SPEAKER_03: So yeah, as usual, don't bet against American dynamism.
- 1:03:19SPEAKER_00: Yeah, I think that's right.
- 1:03:20SPEAKER_00: I think that's right.
- 1:03:21SPEAKER_00: Well, you know, we're closing out 2024.
- 1:03:23SPEAKER_00: I will talk to you again in 2025 for sure to see where we are.
- 1:03:26SPEAKER_00: Hopefully...
- 1:03:30SPEAKER_00: you know,
- 1:03:31SPEAKER_00: I'm kind of liking,
- 1:03:32SPEAKER_00: I'm kind of liking this pace a little bit because there's nothing crazy that's
- 1:03:37SPEAKER_00: super disruptive.
- 1:03:38SPEAKER_00: And there's just like new technologies and uses that are coming up.
- 1:03:42SPEAKER_00: You know,
- 1:03:42SPEAKER_00: we had,
- 1:03:43SPEAKER_00: like outside of the LLM,
- 1:03:44SPEAKER_00: like the standard LLMs with the text,
- 1:03:46SPEAKER_00: we have like Sudo,
- 1:03:48SPEAKER_00: we have Midjourney,
- 1:03:49SPEAKER_00: we have all these companies and all these,
- 1:03:51SPEAKER_00: you know,
- 1:03:51SPEAKER_00: people doing things.
- 1:03:52SPEAKER_03: It's diversified and it's definitely slowed down for sure.
- 1:03:55SPEAKER_03: And in that way, I think that the whole like,
- 1:03:58SPEAKER_03: That's where when we started,
- 1:03:59SPEAKER_03: I don't think that the people talking about the slowdown are completely wrong.
- 1:04:02SPEAKER_03: I think there's truth to what they're saying.
- 1:04:05SPEAKER_03: I think the fact that these metrics are not helpful or you can even imagine on some
- 1:04:10SPEAKER_03: questions the 01 versus the GPT-4 costs a thousand times less is the same.
- 1:04:16SPEAKER_03: It does suggest that we're not just going up and to the right the way we were.
- 1:04:19SPEAKER_03: But it doesn't mean that there isn't a huge amount of progress.
- 1:04:22SPEAKER_03: It's just going to happen at a different pace with different stuff and...
- 1:04:25SPEAKER_03: You know, like we could be in a little bit of a local plateau, but I agree with you.
- 1:04:28SPEAKER_03: The slowing down, I think, is good, and I think it's reassuring.
- 1:04:31SPEAKER_03: I think it's overall a good thing, but it just also just is, you know.
- 1:04:36SPEAKER_03: So to calm down and then think about like, oh, like, is AI actually going to be helpful here?
- 1:04:42SPEAKER_03: Or maybe it's not, you know.
- 1:04:43SPEAKER_03: I think something that happened with a lot of my friends in Silicon Valley,
- 1:04:46SPEAKER_03: a lot of the more pessimistic type people,
- 1:04:48SPEAKER_03: they thought that AI was going to ruin their companies.
- 1:04:50SPEAKER_03: If you take a company like Asana or something like that,
- 1:04:58SPEAKER_03: they were like,
- 1:04:58SPEAKER_03: okay,
- 1:04:58SPEAKER_03: we're going to have half of our engineers work on AI.
- 1:05:04SPEAKER_03: Because we have to, right?
- 1:05:05SPEAKER_03: And I think a lot of those more crazy overreactions have been sort of petering out,
- 1:05:09SPEAKER_03: which I think is probably good.
- 1:05:11SPEAKER_03: Companies should probably do what they're doing and keep abreast of this and use it
- 1:05:15SPEAKER_03: where it's useful and not really worry about that everything's going to be
- 1:05:19SPEAKER_03: disrupted tomorrow.
- 1:05:20SPEAKER_00: Yeah.
- 1:05:21SPEAKER_00: Yeah.
- 1:05:22SPEAKER_00: All right.
- 1:05:22SPEAKER_00: Well,
- 1:05:22SPEAKER_00: on that note,
- 1:05:23SPEAKER_00: um,
- 1:05:23SPEAKER_00: I will let you go and we will,
- 1:05:25SPEAKER_00: we will check in,
- 1:05:26SPEAKER_00: uh,
- 1:05:26SPEAKER_00: in 2025,
- 1:05:27SPEAKER_00: uh,
- 1:05:28SPEAKER_00: just for people finding you on the internet.
- 1:05:29SPEAKER_00: Um, I'll link to your, your Twitter, but obviously deep news, uh, you know, you got that site.
- 1:05:34SPEAKER_00: That's your, your, your main project, your, your baby right now.
- 1:05:37SPEAKER_00: So,
- 1:05:38SPEAKER_00: Check that out, everybody.
- 1:05:39SPEAKER_00: And it's been great talking to you and just keeping up on this field.
- 1:05:43SPEAKER_00: And I,
- 1:05:43SPEAKER_00: you know,
- 1:05:44SPEAKER_00: I do think it is important for people to understand what's happening because it is
- 1:05:48SPEAKER_00: going to affect our lives and this next generation a fair amount.
- 1:05:51SPEAKER_03: Yeah, I think so.
- 1:05:52SPEAKER_03: And then maybe next time we can take some questions ahead of time.
- 1:05:54SPEAKER_03: I'm curious what people are thinking.
- 1:05:56SPEAKER_00: Yeah, that's a good idea.
- 1:05:58SPEAKER_00: All right.
- 1:05:59SPEAKER_00: I'll talk to you later.
- 1:06:00SPEAKER_03: All right.
- 1:06:00SPEAKER_03: Take care.
- 1:06:07SPEAKER_01: This podcast for kids.
- 1:06:36SPEAKER_01: This is my favorite podcast.