Latest / From First Principles / Roman Concrete, Brain “Cognitive Legos,” DeepSeek, and Econophysics (EP. 21)

Transcript

Roman Concrete, Brain “Cognitive Legos,” DeepSeek, and Econophysics (EP. 21) · From First Principles

• 0:00
  Hello internet, this is your captain speaking Lester Nare joined as always by my co-host and our resident PhD newly nicknamed some physics nerd Krishna Chowdhury Courtesy of young Jamie.
• 0:13
  Yes, how's it going?
• 0:15
  Um, it's been an interesting week.
• 0:17
  It's been it's been pretty incredible, right?
• 0:19
  I I got a DM And it was a screenshot of your face.
• 0:24
  Yeah, and Joe Rogan's face Yeah, and and I thought it was AI
• 0:30
  I thought it was just some deep fake.
• 0:32
  And it took us a minute to find it.
• 0:34
  So kudos first and foremost.
• 0:36
  We're back for season two.
• 0:38
  We're super excited.
• 0:39
  This week has been a little bit crazy because young Jamie, the producer over at the Joe Rogan Experience, decided to reference our Nuno Lurero Christmas special episode in the middle of a UFO segment, no less.
• 0:52
  Yeah.
• 0:53
  And that was very unexpected.
• 0:56
  Yeah, it was pretty incredible.
• 0:58
  And he called us some physics nerds.
• 1:00
  Yeah.
• 1:01
  Point of order.
• 1:03
  I am no physics nerd.
• 1:04
  And it was a compliment to me.
• 1:06
  It was a compliment.
• 1:06
  It was a compliment.
• 1:08
  It was a compliment.
• 1:09
  And you're an honorary physics nerd.
• 1:10
  I feel like you've learned a lot.
• 1:13
  I mean, I've learned a lot.
• 1:16
  So if I can quote Navier Stokes' equation off rip, you know, it's not too bad.
• 1:21
  Exactly.
• 1:22
  We're going to dive in.
• 1:23
  We have a great episode today.
• 1:25
  We have a new segment that we're going to try out for in between one of our stories.
• 1:30
  And we're going to talk about a variety of cool topics.
• 1:33
  We're going to start off with Roman Concrete.
• 1:37
  And then pivot over to our favorite institution, Princeton.
• 1:40
  Go Tigers!
• 1:41
  For a new cognitive Lego.
• 1:45
  Yes.
• 1:45
  Which I love Legos.
• 1:47
  We can see from the Lego that's behind us here.
• 1:49
  Yes, but this is the Legos that make up our thoughts and our abstract ideas in our head.
• 1:55
  I think it's a very cool story, and it's very cool research.
• 1:57
  Minority Report Inbound.
• 1:59
  Yes.
• 1:59
  There was another deep seek moment.
• 2:02
  Took over the internet.
• 2:03
  They did a January's back to back.
• 2:05
  I think our one was January last year.
• 2:07
  Yep.
• 2:08
  And now some new, something about constrained manifolds.
• 2:13
  Yeah.
• 2:14
  Manifold constraints.
• 2:15
  Very complicated.
• 2:16
  And we will end our story with our former classmates, favorite character.
• 2:21
  Finance, bro-ology.
• 2:23
  It's our first economic story.
• 2:25
  Yeah.
• 2:26
  That'll be interesting.
• 2:27
  And as always, we're going to get into the weeds of the research papers, the methods, the experimental design.
• 2:34
  You might learn something today because this is from First Principle.
• 2:54
  Okay, our first story is about Roman concrete.
• 2:59
  Have you ever been to Rome?
• 3:00
  I have not.
• 3:01
  I was just in Italy, but I did not get the chance to partake in the life of the Roman.
• 3:06
  Yeah, Rome is...
• 3:08
  honestly, one of my favorite cities that I've ever been to because the amount of history that's there and the continuous civilization, you know, it's, it's been around for a millennia now and you can see buildings that are literally 2000 years old that I can scarce believe they built 2000 years ago.
• 3:28
  It's actually like pretty incredible.
• 3:30
  And all of that is because of something called Roman concrete.
• 3:34
  Okay.
• 3:35
  Um,
• 3:36
  There's been a breakthrough because a recent excavation in Pompeii has actually led us to figure out how they made that Roman concrete.
• 3:47
  And it's been elusive for...
• 3:49
  to even material scientists in the modern day, how the Romans made that and how they could build such buildings that last 2,000 years and are still used today.
• 4:00
  Okay?
• 4:00
  It's pretty incredible.
• 4:02
  I think I heard about that on one of the tours.
• 4:04
  Oh, the concrete is da-da-da.
• 4:06
  Yeah, yeah.
• 4:07
  I mean, it's really quite something if you think about it.
• 4:10
  So there was this thing called the Roman Architectural Revolution, okay?
• 4:14
  It's this widespread use of Roman architecture, right?
• 4:17
  Previously unused forms of architecture.
• 4:20
  They used to have arches, vaults, domes.
• 4:23
  Before the Romans, the Greeks never made this kind of stuff.
• 4:27
  But because of concrete, we can actually do it.
• 4:29
  This is the Pont du Gard in France.
• 4:32
  What we just saw was the semicircular arches that were in the theater of Marcellus.
• 4:37
  This is all by Augustus, the Emperor Augustus.
• 4:40
  Yeah, he's the guy who took over right after Caesar got stabbed.
• 4:43
  Right.
• 4:43
  Right.
• 4:44
  Two Brutus?
• 4:46
  Yeah, A2 Brutus was Caesar, and then Augustus said, you know, friends, Romans, countrymen, lend me your ears.
• 4:54
  But Caesar...
• 4:55
  But Brutus is an honorable man, right?
• 4:58
  Great, great.
• 4:59
  One of my favorite Shakespeare plays.
• 5:01
  So...
• 5:02
  They're building all this crazy, crazy stuff, right?
• 5:05
  40 AD, this is the time of Jesus.
• 5:08
  The crown jewel is really the Pantheon, which was built by Emperor Hadrian in 126 AD.
• 5:14
  That building is still intact, and people can go inside, and there's no damage to it.
• 5:21
  It's pretty insane.
• 5:22
  2,000 years after it's built, the Pantheon's dome is still the largest unreinforced concrete dome in the world.
• 5:32
  That's today?
• 5:33
  Today.
• 5:34
  That's actually pretty incredible, especially given the scale of architecture projects that are happening.
• 5:38
  Yeah, yeah, yeah.
• 5:38
  Yeah.
• 5:39
  It's pretty insane.
• 5:40
  I think the Superdome was, like, coming close, but then, like, Hurricane Katrina happened.
• 5:45
  But this is a single dome.
• 5:47
  It's a concrete dome with a hole on the top.
• 5:49
  Right.
• 5:50
  It's massive.
• 5:51
  Yeah.
• 5:51
  And it's just still around entirely made out of concrete.
• 5:55
  We have our next photo where it shows you can see it from the Roman skyline.
• 6:00
  That's the concrete dome.
• 6:01
  Right.
• 6:02
  And it's still it's that's 2000 years.
• 6:04
  That's so crazy.
• 6:06
  Yeah, that dome is 2,000 years old and it's still alive to this day.
• 6:10
  And all of this has to do with the fact that the Romans invented Roman concrete.
• 6:16
  Okay?
• 6:17
  And we still can't figure out how they did it.
• 6:20
  Okay.
• 6:21
  But we're getting close.
• 6:22
  To me, that's—how do we not know?
• 6:25
  Yeah.
• 6:26
  I mean, we're going to find out why it's so difficult, right?
• 6:30
  In 2017, there was a paper in the American Mineralologist from the University of Utah, and they actually showed that when this concrete got in touch with marine ecosystems, so with saltwater,
• 6:42
  it would actually strengthen itself.
• 6:45
  It was like a smart material where it was like repairing itself.
• 6:49
  If there was a crack in the concrete, unlike modern concrete, where the thing just like, you know, you got to decommission it, you got to tear it down, build a new bridge.
• 6:57
  This stuff was actually self-reinforcing.
• 7:00
  Okay.
• 7:00
  So that was in 2017.
• 7:01
  Already we've got like a smart material that's better than modern day concrete.
• 7:06
  Okay.
• 7:07
  So the question is,
• 7:09
  How do you do that?
• 7:10
  Yeah, yeah.
• 7:10
  Because we see it in superhero movies when they get the thingy.
• 7:13
  Or the Foundation, the Apple TV show.
• 7:17
  One of the, spoiler alert, one of the characters is this sentient robot that's lived on forever.
• 7:23
  And has a self-healing kind of.
• 7:26
  Yeah, these Romans are doing it 2,000 years ago.
• 7:29
  So it's like, what's going on?
• 7:31
  So we want to figure out how it is.
• 7:33
  The first clue comes from a guy named Vitruvius.
• 7:36
  He actually wrote the first architecture book in the world back in 30 BC.
• 7:40
  It's called De Architectura.
• 7:42
  Yeah.
• 7:43
  And he described a way of mixing the concrete.
• 7:47
  So what you do is you take something called quick lime, which is just calcium oxide.
• 7:52
  That's something that you get when you take limestone, you heat it up.
• 7:56
  Limestone is calcium carbonate.
• 7:57
  That's like a sedimentary rock that you get from, you know,
• 8:01
  Millions of years of sedimentation from all of these ancient sea creatures with like seashells.
• 8:06
  Okay.
• 8:07
  You pack up a bunch of seashells, you get limestone.
• 8:09
  You heat it up, you get quick lime, which is CAO, calcium oxide.
• 8:13
  Okay.
• 8:14
  Okay.
• 8:15
  And when you combine that in excess water over months, you get some stable form of lime putty.
• 8:21
  And that's the process that he described earlier.
• 8:25
  in his book, De Architectura.
• 8:28
  Okay.
• 8:28
  Yeah.
• 8:28
  Got it.
• 8:29
  Okay.
• 8:29
  So the defects that they found, whenever you look at this Roman concrete in like super detail, you'll find these millimeter grain to centimeter grain defects of lime.
• 8:42
  It's like full lime.
• 8:42
  So it's like not fully mixed in.
• 8:44
  Okay.
• 8:45
  Okay.
• 8:45
  And...
• 8:47
  It was just thought that all of this lime that's not fully mixed in is kind of a defect.
• 8:52
  Okay?
• 8:54
  Then in 2023, a paper from MIT comes out.
• 8:58
  Okay?
• 8:58
  This is from the Masich Lab.
• 9:00
  And this describes a new form of mixing.
• 9:03
  This is hot mixing.
• 9:05
  So in the previous one...
• 9:06
  In Vitruvius, in his document, that's 2,000 years old.
• 9:10
  I mean, the guy was there, presumably.
• 9:12
  So he's describing how he saw the concrete being made.
• 9:15
  He said it's cold mixing.
• 9:16
  This calcium oxide.
• 9:18
  Yeah, which is you get the calcium oxide, you mix it with water, and then you mix the other stuff.
• 9:24
  Okay?
• 9:25
  What the MIT guys are saying is, no, actually what we do is we get the lime class.
• 9:31
  We get that lime.
• 9:31
  We don't mix it with water first.
• 9:33
  We mix it with volcanic ash and all of these other ingredients, and then we put in the concrete.
• 9:39
  And that order is very important.
• 9:41
  Order of operations.
• 9:42
  Yes, because I told you in chemistry, chemistry, this is basically materials chemistry, right?
• 9:47
  Yeah.
• 9:47
  It's like cooking.
• 9:48
  The order of operations is very, very important.
• 9:51
  Yes.
• 9:51
  Okay.
• 9:51
  Yes.
• 9:51
  So –
• 9:53
  What the MIT 2023 paper says is that we're going to do hot mixing, which is mix the lime with the volcanic ash, mix it together, and then put in the water.
• 10:02
  Okay?
• 10:02
  Okay.
• 10:03
  And then these chunks of lime, they're going to be the reason why the Roman concrete actually can self-heal.
• 10:11
  Okay.
• 10:12
  Okay?
• 10:13
  It's by design.
• 10:14
  Okay.
• 10:14
  It's not a defect.
• 10:15
  Okay.
• 10:15
  Okay?
• 10:16
  Okay.
• 10:17
  So the central problem.
• 10:19
  Right.
• 10:19
  Who's right?
• 10:20
  Yeah.
• 10:21
  Right?
• 10:21
  Right.
• 10:22
  Hot or cold.
• 10:23
  Like, who is right?
• 10:24
  Who's the guy who actually can do it?
• 10:28
  Like, is it the hot mixing or is it the cold mixing?
• 10:31
  Right?
• 10:32
  Can I take a guess?
• 10:33
  Yeah.
• 10:35
  Actually, no.
• 10:36
  Go ahead.
• 10:37
  It's the MIT guy.
• 10:39
  It turns out.
• 10:39
  It turns out the original author 2,000 years ago.
• 10:42
  Right.
• 10:47
  I think got it wrong, but there might be a good reason for it, and we'll get to that at the end.
• 10:51
  Okay.
• 10:51
  Okay?
• 10:52
  Okay.
• 10:52
  So the paper comes out last month in Nature Communications, photo number 12, and this is the paper, an unfinished Pompeii construction site.
• 11:06
  Okay?
• 11:07
  Okay.
• 11:07
  Okay.
• 11:09
  That's the key.
• 11:10
  Okay.
• 11:10
  Pompeii is the key.
• 11:12
  Okay.
• 11:12
  Okay?
• 11:13
  I went to Pompeii, I think it was two years ago.
• 11:17
  Actually, I did go to Pompeii.
• 11:19
  Oh, you did go to Pompeii.
• 11:20
  Isn't it incredible?
• 11:21
  Yeah.
• 11:21
  Pompeii is an intact Roman city that has been buried...
• 11:30
  Under a volcanic eruption.
• 11:32
  Mount Vesuvius erupted in 79 AD.
• 11:36
  And it buried an entire volcanic city in moments.
• 11:41
  And just like, you know, within a few hours.
• 11:46
  And when you say city, it's, you know, particularly for the time, it is so unbelievably large.
• 11:54
  Yeah.
• 11:55
  And organized and choreographed.
• 11:57
  Yeah.
• 11:57
  It's like hard...
• 11:59
  It was a really surreal and bizarre feeling being like, oh, that street goes all the way over there.
• 12:06
  Yeah.
• 12:07
  And this one goes all the way over here.
• 12:08
  And it's a grid system.
• 12:09
  It's really impressive.
• 12:10
  Yeah.
• 12:10
  And there's a shopping district.
• 12:12
  Yes.
• 12:12
  And there's like intact restaurants.
• 12:15
  Yes.
• 12:15
  And you can tell what the function was because it looks just like...
• 12:20
  Today.
• 12:21
  Yeah, yeah.
• 12:21
  There's a mall.
• 12:22
  Yeah.
• 12:23
  You know, it's insane when you think about it, right?
• 12:26
  It is.
• 12:26
  And the advantage of studying a site like Pompeii is that what you've got is a city that was 2,000 years old, one of the beach cities of the Roman Empire, and it's frozen in time.
• 12:42
  And just like every other city, there's going to be construction sites.
• 12:45
  Right.
• 12:48
  Okay.
• 12:48
  Okay.
• 12:49
  You see what's happening.
• 12:50
  So if there's a construction site, you can go and look and see how they were mixing the concrete.
• 12:56
  Yes.
• 12:56
  That's the key.
• 12:57
  Yes.
• 12:58
  We can see how the sausage was made.
• 12:59
  Exactly.
• 13:00
  Because much like in all time periods in human history, we're always building something.
• 13:04
  Yeah.
• 13:04
  Yeah.
• 13:05
  Exactly.
• 13:06
  And, you know, like they're renovating a building or they're building a new building or something like that.
• 13:11
  So they found a construction site and that's the focus of this paper.
• 13:15
  OK, got it.
• 13:15
  OK, they found an active construction site and what they found was the dry pre-mixed piles.
• 13:22
  So not lime that's been sitting in water and cold mixing, but actually lime that's been put in.
• 13:29
  And mixed with volcanic ash.
• 13:31
  Okay.
• 13:31
  And then it's waiting to be mixed with water.
• 13:33
  Ah, yeah, okay.
• 13:34
  Okay.
• 13:34
  Got it, got it, got it.
• 13:35
  Got it.
• 13:35
  And the reason why this matters chemically, right?
• 13:40
  We want to talk about, like, what's the difference between the hot mixing and the cold mixing?
• 13:44
  Yeah, like, you know, and I can kind of maybe into it a little bit, but it seems to be really fundamental.
• 13:49
  Yes, it is.
• 13:49
  It is very fundamental.
• 13:51
  So the idea is whenever water reacts with calcium oxide...
• 13:56
  It gives off a lot of heat.
• 13:59
  Okay.
• 13:59
  Okay?
• 14:01
  That's the hot mixing part.
• 14:02
  Okay.
• 14:02
  Okay?
• 14:03
  Now, if you do it the way Vitruvius said all those years ago in that book, then you take the lime, you mix it with the water, and because it takes such a long time, the heat is just going to dissipate into the water.
• 14:15
  And so that's the cold mixing.
• 14:16
  Okay.
• 14:17
  If you do it when it's hot, right?
• 14:20
  If you do it when it's hot, then...
• 14:23
  you've got a mixture of the lime and the volcanic ash that you're mixing together.
• 14:29
  That's going to give off a lot of heat.
• 14:32
  It's going to reach temperatures nearing 200 degrees Celsius.
• 14:36
  Okay?
• 14:37
  And in that chemical reaction, what's going to end up happening is it's going to sort of plaster everything together, and that's actually going to preserve chunks of lime
• 14:47
  Before they fully dissolve into the rest of the mixture.
• 14:51
  I see.
• 14:51
  I see.
• 14:52
  So there's sort of this timing function where there's a window where it can solidify prior to distribution.
• 15:01
  Yes.
• 15:02
  But only at certain temperatures.
• 15:04
  Yes.
• 15:04
  Got it.
• 15:05
  Exactly.
• 15:06
  Yeah.
• 15:06
  And because it's happening so fast, the chunks of lime don't have enough time to really diffuse and become part of a homogenous mixture.
• 15:15
  I see.
• 15:15
  And so having these grains of lime in your concrete is actually a good idea.
• 15:21
  I see.
• 15:22
  Okay?
• 15:22
  Okay.
• 15:23
  And the reason for that— I was going to say—
• 15:26
  And the reason for that, we're going to sort of get into a little bit later, right?
• 15:30
  Okay.
• 15:31
  That's actually key to the self-reporting.
• 15:36
  healing process this kind of random distribution of this lime inside the mixture that solidifies in this case at high temperature prior to it all kind of getting distributed like into a muck allow has some derivative effects that create uh favorable outcomes exactly from a construction perspective
• 15:57
  Yeah, from a construction perspective, right?
• 16:00
  So they found these dry mix piles and they're like, okay, this is great.
• 16:03
  That means that we were kind of right in that there's lime mixed with volcanic ash and it's dry.
• 16:09
  It's not in the form of the concrete, okay?
• 16:12
  Let's look at some of the smoking gun microstructure that they found, okay?
• 16:17
  When they looked at the unfinished walls and then they looked at the raw material—
• 16:22
  What you see is this really brittle microstructure where you're getting chunks of lime, you're getting microscopic bits of volcanic ash, and the two are exactly the same, meaning the raw material and the unfinished walls are...
• 16:38
  The walls that have just been created.
• 16:40
  Right.
• 16:40
  Right?
• 16:41
  Unfinished means there's no plaster, which means it was just done.
• 16:43
  Yes.
• 16:44
  Those have the same material properties.
• 16:47
  So this is, again, helping to verify the hypothesis that it is unfinished.
• 16:53
  Yes.
• 16:53
  Because when you look at it, when it's really small, the structure of it matches.
• 16:58
  Because if it was finished, that microstructure would look different.
• 17:01
  Yeah.
• 17:02
  Well...
• 17:03
  No, hold on.
• 17:03
  Okay.
• 17:05
  The fact that it's an unfinished wall means that it's very recently been made.
• 17:09
  Gotcha.
• 17:10
  Okay?
• 17:10
  Yes.
• 17:11
  And so what that means is that that construction...
• 17:15
  Happened from this pile.
• 17:17
  I see what you're saying.
• 17:17
  You see what I'm saying?
• 17:18
  It didn't have like... Because over time... Yeah, because finished means that they put a plaster on it.
• 17:23
  And then like now the wall is smooth.
• 17:25
  Like if you've been inside the Pompeii buildings, there's like paintings and stuff like that.
• 17:29
  This is a new wall, right?
• 17:30
  And the raw material is right there.
• 17:32
  So you can connect those two and be like this...
• 17:36
  Was going to go become that.
• 17:37
  Yeah, I gotcha.
• 17:38
  You know what I mean?
• 17:38
  Yes.
• 17:39
  And it's right next to each other.
• 17:40
  Yes.
• 17:40
  And because the microstructure is exactly the same, you can say that actually, yeah, they did dry mix.
• 17:45
  Yeah.
• 17:47
  And then do the hot mixing and then make that wall.
• 17:49
  Right?
• 17:50
  That's the idea.
• 17:51
  Got it.
• 17:53
  Okay.
• 17:54
  The other really cool technique that I thought they used was something called stable isotope mass spectroscopy.
• 18:01
  Okay?
• 18:02
  Here's the idea.
• 18:03
  So...
• 18:04
  You know, in nature, you've got isotopes.
• 18:07
  Isotopes are elements that have different numbers of neutrons versus protons.
• 18:12
  And so they're going to be heavier.
• 18:15
  Like, for example, oxygen 18 versus oxygen 16.
• 18:21
  Same chemical properties, but it's just heavier.
• 18:23
  Okay.
• 18:24
  Which means if something gets hot, right, the heavier oxygen is going to move slower.
• 18:29
  Okay.
• 18:30
  And so in any reaction, there's going to be more lighter oxygen that's going to do the reaction versus the heavier stuff.
• 18:38
  Right.
• 18:39
  Because it's moving faster.
• 18:40
  The lighter oxygen is moving faster.
• 18:42
  So you can actually calculate, you can do an experiment, and you can figure out from mass spectroscopy how much of each isotope is in the substance that you're looking at.
• 18:53
  Got it.
• 18:54
  Does that make sense?
• 18:55
  It does.
• 18:55
  It does.
• 18:56
  Right.
• 18:56
  And so that's what they did.
• 18:57
  What they looked at was how much of carbon 13 there is and how much oxygen 18 there is.
• 19:04
  And what they found was that the slope of these lines, if you look at photo 16, right, it's characteristic of that water limited high temperature fractionation.
• 19:16
  Oh, I see what you're saying here.
• 19:18
  So the gray triangles are what they found.
• 19:22
  Mm-hmm.
• 19:23
  At the site in Pompeii.
• 19:25
  And what you're looking at, the red box, is where most of those gray triangles are.
• 19:31
  And that's exactly what would happen if you pre-mixed.
• 19:34
  And then you put water.
• 19:35
  That's that quick lime.
• 19:36
  Yeah, that's that quick lime.
• 19:37
  Exactly.
• 19:38
  And so the slaked lime samples, that's the cold mixing.
• 19:42
  That's the stuff that you want to use for finishing.
• 19:45
  Because it's smooth, and you can plaster it onto the wall, and then you can put mosaics and light paintings.
• 19:50
  That's one of the things stuff.
• 19:51
  and the ornate stuff but if you want the structure you want you want the concrete that actually self-heals you actually want to do it with the hot stuff this is so like and so it's like there there was also this understanding of the dual use at these difference at these different that's what's insane that that's really that's really cool and and here's here's actually like what's what's sort of happening with the the quicklime
• 20:13
  Okay.
• 20:14
  And I haven't gotten into how exactly it would work, right?
• 20:18
  Like, why would these chunks of lime heal concrete?
• 20:23
  Mm-hmm.
• 20:23
  Right?
• 20:24
  Mm-hmm.
• 20:24
  Here's what's happening.
• 20:25
  So, suppose you've got concrete and you've got a crack.
• 20:28
  Okay.
• 20:29
  Now, you've got a crack in the concrete.
• 20:31
  Okay.
• 20:32
  There's going to be water that goes in into this concrete.
• 20:35
  If it's normal concrete, that water is going to push out more and push out more and degrade the whole thing.
• 20:40
  But if you have chunks of lime, calcium oxide, right?
• 20:43
  Yes.
• 20:43
  That's going to dissolve into the concrete and fill, sorry, dissolve with the water and fill the crack.
• 20:50
  And become a kind of putty that like seals that crack over time.
• 20:55
  It's like an extended release.
• 20:57
  Exactly.
• 20:58
  So you have these like pellets.
• 21:00
  You have these pellets of like glue that get activated.
• 21:03
  Right.
• 21:04
  Only when.
• 21:04
  Only when there's a crack and the water goes in and then it seals that crack.
• 21:08
  That's so.
• 21:11
  What's so amazing about that is.
• 21:15
  They clearly had intuited that.
• 21:17
  It's insane.
• 21:18
  And what could be perceived as imperfection initially actually had these unintended consequences that actually made the underlying structure more robust.
• 21:33
  And just like the open-mindedness to think about stuff and try things to get to that level.
• 21:39
  Because in this day and age, we'd be like, oh, no, it's not.
• 21:41
  The mixture is.
• 21:42
  Yeah.
• 21:44
  And that's what we assumed for the longest time.
• 21:46
  We assumed that, like, this Roman mixture was just, like, not good.
• 21:49
  Good.
• 21:49
  You know?
• 21:50
  But, no, it might have been by design that they did this.
• 21:55
  And it's, dude, it's so insane.
• 21:58
  And this actually makes sense, right?
• 22:00
  Because Roman-inspired concrete...
• 22:02
  This could be a huge thing for us environmentally because Roman-inspired concrete, this is the Pantheon again, it's lasted for thousands of years.
• 22:11
  So even though in the process of creating this concrete, it might use more CO2 than, let's say, the more environmentally friendly concrete, the biggest carbon footprint...
• 22:24
  When building something or the biggest carbon footprint of a building is the process of building it, not the materials themselves.
• 22:33
  If you build it once and it lasts for 2000 years, that building is the greenest building on the planet.
• 22:41
  I would argue that the Pantheon, which is still in use, is the most environmentally friendly building because it was built once and then it's been used for 2000 years.
• 22:50
  Right, right.
• 22:51
  Right.
• 22:51
  Yeah.
• 22:52
  Yeah.
• 22:52
  It, it, it's, it's really hard to, because also the scale of it.
• 22:59
  Right.
• 23:00
  And I mean, it rained for three days here and everything's falling apart.
• 23:06
  Oh my gosh.
• 23:06
  Yeah.
• 23:07
  Yeah, yeah, yeah.
• 23:07
  You know what I mean?
• 23:08
  Yeah, it was.
• 23:10
  And there's potholes everywhere.
• 23:12
  It's just the worst, right?
• 23:13
  So it would be really cool if we could actually start replicating this ancient technology.
• 23:18
  Right.
• 23:18
  And then start using it.
• 23:20
  And so Admir Masic, who's the lead author of this study, he actually does have a startup called DMAT.
• 23:29
  that is trying to make sustainable and durable concrete solutions inspired by ancient Roman concrete.
• 23:35
  That's fantastic.
• 23:36
  Okay, so he's using this knowledge that he's gaining from all the research that he's doing on ancient Rome to commercialize eco-concrete.
• 23:45
  And they've actually got pilot projects in the US and in Europe right now.
• 23:50
  Shout out, Admir.
• 23:52
  Well done.
• 23:52
  Yeah.
• 23:53
  Well done.
• 23:54
  When everyone thought it was nothing to see here.
• 23:56
  Yeah, yeah.
• 23:57
  And he kept at it, right?
• 23:58
  And he proved Vitruvius wrong.
• 24:00
  Now, there might be a good reason that Vitruvius was wrong, okay?
• 24:04
  And it's because Vitruvius actually wrote his architecture textbook in like 30 BC.
• 24:12
  But the architectural revolution that we talk about, the Pantheon, Pont du Gard, and all that, that happened about 100 years after.
• 24:18
  So it could be that in those 100 years, 100 years is a long time, right?
• 24:23
  Even in ancient Rome.
• 24:24
  So they could have actually figured out this new technique of hot mixing after he wrote that textbook.
• 24:30
  Fair.
• 24:31
  And, you know, given how fragile...
• 24:35
  any textbooks are maybe somebody else wrote another textbook but it's just been lost to the agents didn't make it right so that's probably one of the reasons the other reason the other reason could be that it was just like translated wrong from like ancient yeah you know roman whatever latin i guess they were using yeah um so there's a lot of reasons but this is a pretty good like nail in the coffin and in the
• 25:01
  cold mixing vitruvian theory theory you know yeah right right and like we're pretty sure we're right that it is hot mixing and and the the the group at mit has actually replicated that the the architecture right and they've actually started a whole company with this new type of concrete so it's clearly working i was gonna say the way the way you know something is working is when the uh the capital markets yeah get involved yeah that's when you know it's uh
• 25:27
  It's transcended.
• 25:29
  That's fascinating.
• 25:30
  I remember walking around in Pompeii and just being like, this is crazy.
• 25:34
  Yeah, dude.
• 25:35
  I was so impressed with Pompeii.
• 25:37
  And this makes sense.
• 25:40
  I know there's always a lot of memes about, you know, the story about the Romans made concrete that we can't figure out.
• 25:46
  And one of the things about this story that's perseverance is really such a fundamental aspect of like scientific research.
• 25:53
  Because one, like you're constantly battling just to maintain like to maintain funding, to continue to do the search.
• 25:59
  And like people give up after a while if you can't meet some kind of quarterly annual whatever metric that is independent of just the curiosity of it.
• 26:08
  Yeah.
• 26:09
  So kudos to the team.
• 26:10
  It's always hard for us to give big ups to MIT.
• 26:14
  But I think U.S. college rankings had them at number one and us at number three.
• 26:17
  I think this is fake news.
• 26:19
  That's totally fake news.
• 26:21
  This is nonsense.
• 26:22
  Yeah.
• 26:22
  This is absolute nonsense.
• 26:26
  Tiger Nation number one.
• 26:27
  We're going to get to Tiger Nation, speaking of which, in our second story.
• 26:32
  But before we do so, we want to do our new segment, The Rundown, where we're just going to hit on a couple of other recent breaking research stories that have come out this week that we're not going to go deep into but are still fascinating nonetheless.
• 26:48
  And we are going to start with the moon rehearsal, which happened on December 20th in preparation for...
• 26:57
  This is the sort of Artemis II final dress rehearsal.
• 27:01
  NASA successfully completed a full launch day simulation for the first crewed lunar mission in half a century.
• 27:09
  So if you don't think we went to the moon before...
• 27:12
  We at least might go to the moon now.
• 27:14
  Let's see.
• 27:15
  This is a collaboration between both NASA and the Canadian Space Agency.
• 27:20
  We have astronauts Christina Koch, Victor Glover, Jeremy Hansen, and Reed Weissman.
• 27:25
  It's going to be a 10-day mission, which is super exciting.
• 27:29
  And they're taking off, I believe it is February 5th.
• 27:32
  So it's coming soon.
• 27:33
  Yeah, it's going to be great.
• 27:34
  It's basically like Apollo 8.
• 27:36
  They're not landing on the moon, but...
• 27:40
  They're not landing on the moon, but they're actually going to be going around the moon and then coming right back.
• 27:47
  And one of the coolest things about this is it's going to be the farthest that humans have ever been from planet Earth.
• 27:55
  Really?
• 27:55
  Yes.
• 27:56
  Because even in the Apollo missions, they went around the moon, but...
• 28:01
  The guy who stayed in the lunar capsule, or sorry, in the command module, rather than the two guys who went down in the lunar module, the guy who stayed in the command module, Michael Collins in Apollo 11, for example, was pretty close to the surface of the moon on the other side.
• 28:18
  He was on the far side on his own.
• 28:19
  Yeah.
• 28:21
  With these guys, these guys are actually overshooting, and they're going to go really far out around the back.
• 28:27
  They'll catch a wider orbit.
• 28:28
  Yeah, yeah.
• 28:28
  So they're going to be the farthest humans ever from Earth.
• 28:33
  And it's going to be—I'm really excited to see the photos that they get.
• 28:36
  It would be cool to see, like, are they going to be so far that they can get the moon and the Earth in the same shot?
• 28:43
  It's the pale blue dot V2.
• 28:46
  That would be insane.
• 28:46
  That would be really, really cool.
• 28:48
  Because we don't really have a photo like that of our family, like Earth and the moon together like that.
• 28:56
  Actually, we do.
• 28:56
  That's not true.
• 28:57
  We do have that from there's this Japanese satellite that's quite far away, right in between the sun and the Earth.
• 29:04
  Okay.
• 29:05
  And every once in a while when the moon goes in front of the Earth, it gets a photo.
• 29:09
  It doesn't count.
• 29:09
  There's no people in there.
• 29:10
  Yeah, exactly.
• 29:11
  This would be like a photo taken by a human being.
• 29:14
  It's us.
• 29:14
  It's us.
• 29:14
  That would be really cool.
• 29:15
  Not the fruit of our loins, which is technology.
• 29:18
  Yeah.
• 29:19
  Yeah.
• 29:19
  So we'll be on the lookout for Artemis 2.
• 29:21
  We actually have Artemis behind us.
• 29:25
  Over one of our shoulders.
• 29:26
  Oh, over the other shoulder.
• 29:28
  Oh, it's this camera now.
• 29:30
  We're still getting used to the three cameras set up here.
• 29:32
  Yeah.
• 29:33
  But it took me not a reasonable amount of time to build.
• 29:38
  So that's our story number one in the rundown.
• 29:39
  Number two is the cellular cleanup crew.
• 29:44
  This came out January 7th.
• 29:47
  It's about iDigs, iDags,
• 29:50
  which are deleting disease-causing proteins in nature chemistry.
• 29:56
  So scientists have found a way to supercharge our cells, natural recycling systems, to completely destroy cancer-shielding enzymes.
• 30:05
  And we've talked about T-Rags and the whole immune system and the immune response in our past episodes in Season 1.
• 30:15
  Some really big deep guys that are great on this.
• 30:16
  That's what kind of caught my eye on this one.
• 30:18
  Yeah.
• 30:18
  So we have Max Planck Institute involved in this, led by George Winter.
• 30:22
  And the idea here is that most drugs today act as inhibitors that just like block, right?
• 30:28
  Yeah.
• 30:29
  They block the site.
• 30:30
  They don't really, you know, activate.
• 30:33
  But –
• 30:35
  This is now recognizing and destroying the targets entirely.
• 30:41
  Yeah, and it's using the body's own machinery.
• 30:45
  Right.
• 30:45
  That's, I think, what's key.
• 30:46
  Right.
• 30:47
  It's like it's basically priming these proteins and not actually, like, taking care of the problem themselves, but, like, tagging them for the body's own trash collectors, these E3 ligases, to do it themselves.
• 30:59
  And it's always good to just, like, have the body do it.
• 31:02
  Right.
• 31:02
  Because...
• 31:03
  The body has a whole bunch of dynamics.
• 31:05
  Let's talk about biology.
• 31:06
  There's all these variables.
• 31:07
  Yeah.
• 31:08
  There's a lot of variables.
• 31:09
  Yeah.
• 31:10
  If you can help the body rather than just do the task yourself, it's always better.
• 31:14
  Right?
• 31:14
  So the analogy was if standard drugs are like putting a lock on a door to keep a villain inside, IDEX are calling in a demolition crew to remove the entire room, making sure the threat is gone for good.
• 31:27
  Yep.
• 31:27
  Again, this is all happening like this is this week.
• 31:29
  This is this week.
• 31:32
  Third rundown, the chat CBT moment.
• 31:35
  But for sleep, Jan 6, Sleep FM, this new thing out of Stanford Medicine that is diagnosing 130 diseases while you sleep.
• 31:45
  It's hard to avoid AI in stories all the time because it kind of just is what it is.
• 31:49
  But this story is...
• 31:52
  Fascinating.
• 31:53
  Sleep FM was trained on 600,000 hours of physiological data to be able to spot subtle mismatches between your brain, your heart, and your breathing signals.
• 32:03
  Yeah.
• 32:03
  And from that, it can then make predictions about a variety of diseases.
• 32:07
  Yeah.
• 32:07
  And what really got to me was it only needs to analyze one night of sleep data from you.
• 32:13
  To figure something out, right?
• 32:15
  It's trained.
• 32:16
  So it's a pre-trained AI, kind of like the pre-trained transformer models that we use in LLMs and things like that.
• 32:23
  So this is an AI that's been pre-trained with, as you said, 600,000 hours of physiological data, but then all it needs to classify you is just one night.
• 32:33
  And then it'll tell you one of the 130 diseases that you could have.
• 32:37
  It's pretty insane.
• 32:39
  Being able to read biological data through like these like sort of derivative signals in such a short, limited amount of like you need such a limited amount.
• 32:49
  Yeah, yeah, yeah.
• 32:49
  You could just like go and go, you know, to a facility that will monitor all of these different things while you sleep for one night.
• 32:56
  And then that's it.
• 32:57
  And you got it.
• 32:58
  Yeah.
• 32:58
  Yeah.
• 32:59
  unbelievable access to information.
• 33:01
  What will we do about it?
• 33:02
  Tune in next week.
• 33:04
  Story number four is now an optics and nanotech story.
• 33:08
  This was today it actually came out.
• 33:11
  The Silica Breakthrough, high-tech lenses for common glass.
• 33:16
  This is out of Harvard, Federico Capasso.
• 33:19
  So traditionally, flat metal lenses require this exotic and expensive materials to bend light.
• 33:24
  You know, we know as you get smaller in scale, things become more expensive.
• 33:28
  We see this with chips and all these other things.
• 33:30
  And so they've sort of perfected this single clean path of light without having to use, like, they've perfected this ability to do what used to take massive scale and massive cost with basically off-the-shelf glass.
• 33:44
  Yeah, yeah.
• 33:46
  It's pretty insane.
• 33:47
  I mean, it's going to be way cheaper, right?
• 33:49
  Massive scale, lower cost.
• 33:52
  And we use Glass in a whole number of applications.
• 33:56
  And one of the key things for me that I really liked about this story was that you can use all of the engineering that we've developed to make chips and silicon semiconductor industry stuff we can now use to make this Glass, right?
• 34:12
  That's one of the reasons why it's so cheap is because we can leverage all of this infrastructure that we've already built for all of our computing.
• 34:19
  Yeah.
• 34:20
  And now we're making glass out of silica.
• 34:22
  It's always great when we don't have to rebuild our industrial base in order to be able to enable and access new technology.
• 34:29
  It's like discovering you can build a high-performance jet engine with the standard parts from a local hardware store, which is fascinating.
• 34:36
  Our last quick rundown story of the day before we get back into our main stories is this observance.
• 34:44
  Ability to watch plants breathing really, really, really well.
• 34:50
  This is in Environmental Science, January 7th.
• 34:53
  Stomata Insight, a real-time window into crop survival.
• 34:57
  So the idea behind this is people are trying to improve how we do certain aspects of agriculture.
• 35:02
  The better we can monitor and see stuff, the better we can do things to sort of impact outcomes from a crop perspective.
• 35:09
  Yeah.
• 35:09
  Yeah, it's like a methods paper that is like, you know, now that we have a way to measure
• 35:16
  the stomata opening and closing on the bottom of a leaf, we can figure out what are the genes that actually control these things.
• 35:22
  And then, well, can I, you know, CRISPR these genes to then make a drought resistant crop that doesn't open when it's, you know, too dry, things like that.
• 35:33
  This is a new imaging system that lets science watch the microscopic mouths of plants as they breed in high definition out of University of Illinois Urbana Champaign, Joseph D. Crawford and Andrew D.B.
• 35:44
  Leakey.
• 35:45
  Um, very cool.
• 35:47
  I love plants.
• 35:47
  I love the idea of like, you know, having plants around is always super fun and fantastic.
• 35:52
  And that's just a little brief rundown.
• 35:53
  We can't always cover every story, but we're trying to illustrate there's so much happening.
• 35:59
  There's so much going on that does actually translate into real world.
• 36:03
  It's not all just abstract.
• 36:04
  Yeah.
• 36:05
  It's not mathematicians kind of going back and forth.
• 36:07
  Exactly.
• 36:07
  About God knows what.
• 36:09
  But now my favorite, another story about the good old Princeton Tigers.
• 36:15
  Yes, the best.
• 36:17
  This is actually from the Princeton Neuroscience Institute, which is the building where I did my senior thesis back in the day.
• 36:25
  I was a part of the physics department, but my senior thesis, I was really interested in biological physics.
• 36:34
  And so there was a professor there who was interested in physics and how to sort of use physical physics.
• 36:42
  intuition to talk about the brain i was particularly focusing on the retina this is talking about much higher order abstraction um task demand like thought and and really abstract things but there is a very nice thread of physics and mathematics in this story and it is going to get kind of technical but we're in for the ride right like that's that's why you guys are here yes
• 37:11
  It's really cool.
• 37:13
  Okay.
• 37:14
  The core problem is something called compositionality.
• 37:19
  Okay.
• 37:20
  It's a key in biological intelligence and it's something that AI is actually not very good at.
• 37:25
  Okay.
• 37:26
  And something that the authors used was, you know, if you already know how to bake bread, you can use this ability to bake a cake.
• 37:35
  Why?
• 37:35
  Because you can actually take parts of your learning when you baked bread, you know, how to mix stuff, how to turn on an oven, how an oven works.
• 37:46
  And you can take those models together and then figure out, OK, with a cake, it's like not that much different.
• 37:53
  Fine.
• 37:53
  There's like a little bit of frosting.
• 37:55
  There's like all this like other stuff.
• 37:57
  But you can take the Legos, the cognitive Legos that you've built when baking bread.
• 38:02
  Yeah.
• 38:03
  And then use that with some new stuff to figure out the ability to bake a cake from scratch.
• 38:10
  Okay?
• 38:12
  AI is not very good at this.
• 38:14
  Okay.
• 38:14
  Okay?
• 38:15
  Like...
• 38:17
  AI is like in terms of this generalized intelligence that we kind of are after.
• 38:24
  Whatever that means.
• 38:24
  Right?
• 38:24
  Whatever that means.
• 38:25
  It has no clear definition.
• 38:26
  But I know I'm being facetious.
• 38:27
  You know what I mean?
• 38:27
  I'm being facetious.
• 38:28
  It's like it's good at certain tasks because you've trained it on certain tasks.
• 38:32
  It's seen the data for those certain tasks.
• 38:35
  Right?
• 38:35
  But as soon as you sort of bring it out of its comfort zone, the performance decreases by a lot.
• 38:42
  okay and for a really long time it's been kind of a dilemma this question how can a neural network right that's defined by neurons connected to each other so you've got these synaptic connections you want it to be stable enough so that it doesn't forget old skills right right but you also want it to be plastic enough you want it to be
• 39:09
  on the move enough that it can actually rapidly assemble new skills okay so it's this dilemma between stability and plasticity plasticity meaning the ability to change and we as as humans have really mastered that tightrope we really have right we have than every any other thing yeah i mean really see
• 39:29
  Yeah, I mean, we have millions of years of evolution to thank for that.
• 39:33
  But in terms of AI, whenever you have AI, there's this problem called catastrophic forgetting.
• 39:39
  Okay.
• 39:40
  And that's the idea when you learn a new task.
• 39:43
  And when you're learning a new task, it totally obliterates your ability to do some old tasks.
• 39:48
  So in this case, you've got a dog-cat classifier.
• 39:51
  And then you train it to, you know, figure out how birds look.
• 39:55
  And then all of a sudden, it can't figure out what a dog and a cat look.
• 39:59
  This is an exaggeration because nowadays we have learning models that can do much better.
• 40:03
  But at the end of the day, this idea of catastrophic forgetting is still very much a problem.
• 40:08
  Right.
• 40:08
  If you want to expand to just generalized ability to solve tasks.
• 40:13
  Okay?
• 40:13
  Mm-hmm.
• 40:14
  Now, in cognitive science, we have another problem, which is called the binding problem.
• 40:19
  And that's the idea of how does the brain dynamically bind different attributes to the same thing without requiring a single neuron for all of the specific stuff?
• 40:31
  And here's what I mean.
• 40:32
  Over there, you've got a person who's looking at a rolling ball.
• 40:35
  The ball is red, and it's moving to the right, right?
• 40:38
  Now, there's three attributes here that I can immediately see.
• 40:41
  There's motion.
• 40:42
  It's moving to the right.
• 40:43
  There's color.
• 40:44
  It's red.
• 40:45
  And there's form.
• 40:45
  It's a ball, and it's rolling, right?
• 40:48
  Now, the brain doesn't have a single neuron for red rolling ball to the right.
• 40:58
  Okay, because that would be insane.
• 41:00
  Right.
• 41:01
  Okay, I don't have like a single neuron for every little object and every little idea.
• 41:06
  What the brain does is somehow combine all of these ideas and bind them together into a single idea of a rolling ball that's moving to the right.
• 41:17
  Okay.
• 41:17
  Okay, and that's called the binding problem.
• 41:19
  Okay.
• 41:19
  Okay?
• 41:20
  It's a problem in neuroscience, and we don't quite know mechanistically inside the brain how these neurons are actually doing it.
• 41:28
  Okay?
• 41:29
  You can talk about it.
• 41:30
  Right.
• 41:30
  Like, oh, there's an idea of a red.
• 41:33
  But at the end of the day, it's like, what is the neuron doing?
• 41:36
  Right.
• 41:36
  Because these are just cells in my brain.
• 41:39
  And it is incredible because it's a highly efficient neuron.
• 41:44
  highly sustainable architectural system.
• 41:49
  Yes.
• 41:50
  Yeah.
• 41:50
  And like, that can learn anything.
• 41:51
  It would be good to know how it works.
• 41:53
  Yes.
• 41:54
  For a whole number of reasons.
• 41:54
  Yes, exactly.
• 41:55
  And that's what the key discovery of this paper that came out of nature is.
• 41:59
  Okay.
• 42:00
  What they figured out is the,
• 42:02
  neural basis at a systems neuroscience level of these cognitive legos okay and they used something called low dimensional shared neural subspaces we're going to get into what that is but there's a lot of math involved and some very cool physics intuition which is why i really love this paper because it reminds me of like sort of why i got into like physics and neuroscience and the beauty of that in the first place so
• 42:28
  What we're really doing is we're solving the problem of this binding problem using very cool mathematics.
• 42:35
  And it might actually give us insights into how to train AI better.
• 42:39
  Which makes sense because ultimately if we understand how to solve the binding problem, that's immediately going to be architecture you want to implement into any kind of artificial system that you want to replicate human intelligence.
• 42:48
  Yes, exactly.
• 42:49
  And AI is, at the end of the day...
• 42:52
  For a long time, it's been inspired by human cognition.
• 42:57
  So there's a reason why they're called neural networks, right?
• 43:01
  Okay.
• 43:02
  So let's talk about some mathematics now.
• 43:06
  Okay?
• 43:06
  Here we go.
• 43:08
  So...
• 43:11
  Imagine you've got a bunch of neurons.
• 43:13
  Okay.
• 43:13
  We've got about, I think, 80 billion neurons.
• 43:16
  We're not going to talk about that many neurons.
• 43:18
  But what you can think about when we think about what is a brain state –
• 43:23
  Okay?
• 43:23
  Okay.
• 43:24
  Let's first talk about what is a brain state.
• 43:26
  A brain state would be a description of all of your neurons and which ones are active and which ones are not active.
• 43:35
  Okay.
• 43:36
  For example.
• 43:37
  Right?
• 43:37
  Like a zero one.
• 43:38
  Like a map of a zero one state.
• 43:39
  Yeah.
• 43:39
  That's one way of thinking about it.
• 43:40
  Just a simple one.
• 43:41
  Another way would be like what's the firing rate of each of these neurons.
• 43:44
  Right?
• 43:45
  And is it?
• 43:47
  Yeah.
• 43:47
  But at the end of the day, it's something like is this neuron active or not?
• 43:50
  And then how is it connected to all the other stuff?
• 43:52
  Okay.
• 43:52
  But the state of the brain at any given point, like right now when I'm doing this podcast, there's certain neurons that are turning on and off, right?
• 44:01
  And my brain is moving from one state to the other as I speak, as I go through what I'm trying to talk about and things like that, okay?
• 44:10
  And given we've got so many neurons, we run into this curse of dimensionality.
• 44:16
  And that means that if you've got a lot of neurons...
• 44:22
  Your brain state lives in a high dimensional space.
• 44:26
  Here's what that means.
• 44:26
  I was going to say, yeah.
• 44:27
  Okay.
• 44:28
  So suppose I only have three neurons.
• 44:31
  Okay.
• 44:32
  Imagine a brain with only three neurons.
• 44:34
  That brain is going to live in a 3D space because the X axis is going to tell you how neuron one is behaving.
• 44:42
  The Y axis is going to tell you how neuron two and the Z neuron three.
• 44:47
  So where I am here is.
• 44:49
  at any given moment, is going to tell me what this three-neuron brain is doing.
• 44:54
  Now imagine a thousand neurons.
• 44:56
  I've got a thousand different axes, right?
• 45:00
  And each state in this thousand-dimensional space is a brain state.
• 45:07
  Okay?
• 45:08
  The curse of dimensionality is that if there's unstructured data all over the place, then the amount of data that is required to figure out where I am is going to be, like, ridiculous.
• 45:17
  Yeah, yeah, yeah.
• 45:17
  Okay?
• 45:18
  So clearly the brain doesn't do that.
• 45:20
  And we know that it doesn't do that because— That would be, like, more of a brute force approach.
• 45:23
  Yeah, it would be, like, just a really terrible way to do any kind of learning, anything like that.
• 45:29
  Highly inefficient.
• 45:29
  Highly inefficient.
• 45:30
  It would take, like, longer than the—
• 45:32
  Lifespan of the organism to actually like get any learning done, get any cognition done.
• 45:37
  Okay.
• 45:38
  So instead, what the brain does is correlate these neurons.
• 45:43
  Okay?
• 45:44
  Because neurons are connected to each other.
• 45:46
  So if one of them fires, then it's going to influence its neighbors to fire, and so on and so forth.
• 45:51
  So the neural activity is actually constrained in what's called a low-dimensional manifold, a manifold you can think of as a surface.
• 45:59
  And here you can think, let's go back to our analogy of the three-neuron brain.
• 46:04
  Okay?
• 46:05
  You've got a three-neuron brain, neuron one, neuron two, neuron three.
• 46:07
  But let's say neuron three is actually correlated to how neuron one and neuron two do.
• 46:12
  Okay?
• 46:14
  My dimensionality is actually not three dimensions, because if I know what neuron one and neuron two are doing, I immediately know what neuron three is doing, because these two, let's say, are input to them.
• 46:26
  And so neuron three's activity is on a plane.
• 46:32
  And so instead of three dimensions, I'm really only talking about two dimensions.
• 46:37
  And my brain states are confined to this manifold in a higher dimensional space.
• 46:43
  The manifold being this lower dimensional 2D structure.
• 46:46
  The idea is because of the correlation that you know exists, you can abstract the complexity to...
• 46:55
  a simpler structure because if ball goes up like if neuron one goes if you know when neuron one goes up neuron two goes right you you don't you can now use yeah yeah i i know that information right yeah yeah and that becomes sort of like i can rule out all this other stuff that otherwise would have been in the probabilities okay yeah exactly yeah you see
• 47:15
  See what I'm saying?
• 47:15
  So now you're constraining your brain states to a low dimensional manifold in this very high dimensional space.
• 47:22
  So even though you have a thousand neurons that you're dealing with, maybe the brain actually only lives on a much lower dimensional surface in this thousand dimensional space.
• 47:32
  Just like the three brain, the three neuron brain was only living in a 2D surface.
• 47:37
  You could even imagine a 1D surface where it's like a line.
• 47:40
  Yes.
• 47:41
  Right.
• 47:41
  Yes.
• 47:41
  And so all three of them are connected in some way and there's no, there's no independence.
• 47:46
  It's just like, all you have to do is figure out where you are on the line rather than where you are in 3d space.
• 47:51
  Okay.
• 47:51
  So,
• 47:52
  The central insight here is that if we have two different cognitive variables, remember, we're talking about the binding problem, right?
• 48:00
  That's what we're trying to solve.
• 48:01
  We had the red and we had the ball and we had the motion.
• 48:03
  These are three different things.
• 48:06
  That we don't store in individual neurons.
• 48:08
  Exactly, right?
• 48:09
  So we've got to figure out a way for the brain as a state to store this information.
• 48:14
  Okay?
• 48:15
  So the way we can do this is if we have a giant...
• 48:20
  high-dimensional space, but we can constrain ourselves to manifolds.
• 48:25
  What if there's two different manifolds for the two different variables that we're trying to store?
• 48:31
  Okay.
• 48:32
  Okay?
• 48:33
  Yes.
• 48:33
  That's the idea.
• 48:34
  Okay.
• 48:34
  The key idea is you can have in your, let's go back to our three-neuron brain, you can have orthogonal planes.
• 48:43
  You can have planes that are perpendicular to each other.
• 48:45
  And I've got a photo to show this.
• 48:50
  which is one plane, and then I can have a shape plane, let's say.
• 48:55
  And that's the second plane, or a motion plane.
• 48:58
  And that's the second plane.
• 49:01
  So now, here's what we would do.
• 49:03
  Your brain is somewhere in 3D, right?
• 49:05
  In this 3D space.
• 49:07
  But what a downstream circuit can do is look at all of these neurons and then figure out what is the shadow of my brain state on one plane?
• 49:17
  That's going to tell me the color.
• 49:19
  The shadow of my brain state on the other plane?
• 49:21
  That's going to tell me the shape.
• 49:24
  You see?
• 49:25
  So I can keep track of a bunch of different information.
• 49:29
  Yeah.
• 49:30
  By constraining myself along these manifolds.
• 49:33
  The intersections of different manifolds.
• 49:36
  That's the idea.
• 49:37
  That's – because the idea is that like for lack of a better way to put it, it's – I'm still grokking.
• 49:42
  But like the – you have a high – a very high dimensional space of potential – potentialities.
• 49:51
  Yeah.
• 49:51
  Yeah.
• 49:52
  But you have a very acute space, whether it's on a single manifold or at the intersection of the manifolds, which is actually what you care about.
• 50:00
  And that surface area, the amount of information or surface area is fundamentally smaller and shorter to traverse than navigating through the entirety of that larger multidimensional space.
• 50:12
  Yeah, yeah.
• 50:12
  It's kind of like you can't fully have unstructured space.
• 50:17
  just like brain space right right like your brain states have to have to adhere conform to some conform to some order in order to actually like learn and like like get ideas and like keep track of ideas right so what you could do is you could have a brain state that's in this high dimensional space and then you can have another brain circuit that's somewhere else yes that is looking at this brain state and then what it does is calculate okay where along this plane am i yeah yeah
• 50:45
  And then another, the color circuit is going to be like, actually, I care about this other plane.
• 50:49
  So where along this other plane am I?
• 50:51
  Right?
• 50:52
  And then the binding problem is kind of solved.
• 50:55
  Right.
• 50:56
  Right.
• 50:56
  Right.
• 50:57
  Because you don't, you now have an ability to store the color.
• 51:08
  the shape, and the motion along these men.
• 51:12
  With the same number of neurons.
• 51:13
  With the same number of neurons.
• 51:14
  I don't need individual neurons for red, for blue, for this, that, right?
• 51:18
  And I don't need an individual neuron for red ball, red square, red star, blue ball, blue square, blue star, right?
• 51:26
  I can now, with the same number of neurons, all I have to do is just calculate where I am on this plane and this plane.
• 51:32
  Yes.
• 51:32
  You see what I'm saying?
• 51:33
  Yeah, I do.
• 51:33
  It's very cool.
• 51:34
  Yes.
• 51:35
  Please continue.
• 51:36
  Sorry.
• 51:36
  But yes.
• 51:36
  Yes.
• 51:37
  Okay.
• 51:38
  So it's cool to think about.
• 51:40
  Yes.
• 51:40
  But what truly impressed me about this paper is how they actually went about proving this.
• 51:46
  Right.
• 51:46
  Because this is quite like very abstract.
• 51:49
  Yeah.
• 51:50
  Right.
• 51:50
  So like how are we actually doing this?
• 51:52
  Prove it.
• 51:53
  Yeah.
• 51:53
  Exactly.
• 51:55
  So they used two adult Reus monkeys.
• 51:57
  The Reus monkeys are chosen because the prefrontal cortex, the neocortex of a Reus monkey is very similar to a human neocortex.
• 52:05
  Okay.
• 52:06
  So, you know, for it's better than, for example, a lot of the other model organisms that we use like rodents because...
• 52:14
  This brain can be involved with rule switching, abstract categorization.
• 52:18
  So it's very close that way.
• 52:19
  And we can actually try to probe these higher order abstract ideas.
• 52:24
  Right.
• 52:25
  And the stimuli that the subjects were responding to were of two kinds.
• 52:32
  OK, there's shape and then there's color.
• 52:35
  The shape that they were responding to is a bunny versus a tee.
• 52:39
  Right.
• 52:41
  Okay, and the bunny ears became a T. Okay, got it.
• 52:44
  Okay, so that's the sort of shape axis that we're trying to discern, right?
• 52:50
  And then on the other hand, we have the color axis, which is green versus red.
• 52:54
  And one of the things that they were very careful about is that these axes are circular.
• 53:01
  They don't have edges.
• 53:03
  Okay.
• 53:03
  Okay.
• 53:03
  So you smoothly go from a T to a bunny back to a T and from green to red back to green.
• 53:09
  Okay.
• 53:10
  The reason why this is important is because you don't have any edge effects.
• 53:13
  Right?
• 53:14
  Like if you have like a square, let's say, then there might be some artifact from being in the corner here because there's nowhere to go over on this side.
• 53:23
  So your brain state might be doing something weird, right?
• 53:25
  Yes.
• 53:25
  But here in this case, you've got a circle, and then you've got another circle.
• 53:28
  There's no edges.
• 53:29
  Yes.
• 53:30
  So every point in the circle is equivalent.
• 53:32
  Yes.
• 53:32
  Right?
• 53:33
  Yes.
• 53:33
  So we're really trying to figure out the topology of this manifold that we're trying to look at.
• 53:38
  Yes.
• 53:38
  Right?
• 53:39
  And the manifold turns out to be a donut.
• 53:41
  It's a Clifford Taurus because it's a circle.
• 53:44
  which is a circle like this way, and then a circle around the donut, and then like through the...
• 53:50
  There's two circles in a donut.
• 53:52
  So you're constraining your stimulus to a torus.
• 53:56
  To a torus.
• 53:57
  Okay?
• 53:57
  So...
• 53:58
  presumably the brain should also have some kind of representation of this Taurus.
• 54:04
  And the task that they're doing is the following, okay?
• 54:06
  So you've got a stimulus, the subject looks at the stimulus, and then he does a saccade, which is the eyes go in one particular direction based on whether it's a tea or a bunny or if it's green or if it's red, okay?
• 54:19
  Okay.
• 54:20
  So the subjects are trained to categorize shape, in which case it's a bunny or a T, and they respond in one axis.
• 54:30
  So the axis being like, you know, upper left, lower right.
• 54:34
  Mm-hmm.
• 54:35
  That's going to be the shape axis.
• 54:37
  And then there's going to be another task, which is color, and they're going to do the opposite direction, which is upper right, lower left.
• 54:43
  Okay?
• 54:43
  That's key.
• 54:44
  So we need to understand that.
• 54:46
  And task design is everything.
• 54:48
  Okay?
• 54:49
  In a lot of these systems, neuroscience, behavior, and abstraction experiments, I remember when I was in my PhD, like, we used to spend entire lab meetings just discussing the task.
• 55:05
  how to probe these very weird questions that we're asking, right?
• 55:09
  Because once we have the task, then we can figure out how to analyze what the neuron was doing in this task versus this task and so on and so forth, right?
• 55:17
  So the task is as follows.
• 55:19
  There's a shape axis, which is, let's say, this way.
• 55:22
  And then there's a color axis, which is this way, okay?
• 55:26
  And then what they have to do is they have to switch the axes, right?
• 55:30
  Okay?
• 55:30
  Okay.
• 55:31
  So before, the shape axis was upper left, lower right.
• 55:34
  The color axis was upper right, lower left.
• 55:36
  And then now what they have to do is they have to say, actually, I'm going to respond to color, but using the shape axis.
• 55:45
  Okay.
• 55:46
  Okay.
• 55:46
  Okay?
• 55:47
  Okay.
• 55:47
  So let's go through this again.
• 55:50
  Okay.
• 55:50
  Okay?
• 55:50
  Yeah, yeah, yeah.
• 55:51
  With shape...
• 55:53
  If I see the shape one way, I'm going to go to the lower right or the upper right.
• 55:57
  If I see the color, I'm going to go to the upper left or lower left.
• 56:02
  But now in the third task, I'm mixing the two.
• 56:06
  Okay?
• 56:06
  So I'm looking at color, but I'm actually responding with the motion of the shape.
• 56:13
  Okay.
• 56:14
  Okay?
• 56:14
  Yes.
• 56:15
  Here's the key hypothesis, okay?
• 56:18
  If this cognitive Legos is correct, then the brain should learn that mixing task, but it shouldn't learn it as a brand new problem.
• 56:29
  It should be just...
• 56:31
  It should be a cognitive Lego of the shape task from the motion side and a cognitive Lego from the color task combining them together.
• 56:40
  Yes.
• 56:41
  The neural representations of those two should be resilient.
• 56:46
  Okay.
• 56:46
  Okay.
• 56:47
  Like it does.
• 56:48
  It's not ephemeral.
• 56:50
  It's not.
• 56:51
  When you say resilient, how do you mean?
• 56:52
  What I mean by resilient is the same neurons.
• 56:55
  Yes.
• 56:55
  That were responding to color.
• 56:57
  Yes.
• 56:57
  Are now going to still be responding to color.
• 56:59
  Yes.
• 56:59
  The same neurons that were responding to this axis are still going to be responding to this axis.
• 57:03
  Yes.
• 57:04
  But the subject is still is going to be doing a completely new task.
• 57:07
  Right.
• 57:08
  Right.
• 57:08
  Using the same old neurons.
• 57:11
  Yes.
• 57:11
  Using the same sort of procedure, right?
• 57:14
  The color processing module developed for one task and the response, the axis motor response for another task, it's going to just snap them together like Legos.
• 57:26
  Literally like Legos.
• 57:27
  Yeah.
• 57:27
  And then that gives it the ability to then...
• 57:31
  do the derivation without ascribing more neurons to do exactly yeah yeah so it's it's it's learning something but it's actually it's it's it's able to learn because it's able to snap older older learn habits together but at the neural level yes the neurons are now working together yes yes yes so yes the the idea is if you okay i'm gonna pause this one this one is this one is
• 57:58
  This one's weird.
• 57:59
  This one's weird.
• 58:00
  This is weird.
• 58:01
  Neuroscience is really cool to think about that way.
• 58:04
  I just... Yeah, let's pause and really think about this.
• 58:07
  Because I know we have a lot to cover, but I'm really trying to...
• 58:11
  The point is, right, you have these... A neuron has a learning about whether it's shape or color in this context on these axes.
• 58:20
  Yeah.
• 58:21
  And let's say, as an analogy, the color...
• 58:26
  is a red Lego brick, and the shape is a blue Lego brick.
• 58:34
  And so rather than saying when... Rather than having a new yellow Lego brick, you can just take those two
• 58:43
  put them together now and they are now able to get to this, all this reversed response task that you're talking about.
• 58:50
  This like combined, this, this, yeah.
• 58:52
  And the, and the key is that we're combining two different modalities here.
• 58:57
  We're, we're combining it's, it's, it's, it's,
• 59:01
  responding to color right but the response is with the shape response it's it's using the the the the axis that it's using it used to use for shape yeah it's a totally different task but it is able to reuse an old a previously paved path for a different uh use case without losing the ability to still do the old thing yeah
• 59:28
  And also not losing the ability to continue to do the new thing.
• 59:30
  Yes, exactly.
• 59:31
  And one of the things they did was they randomized, like, which task they were doing.
• 59:35
  They found that the performance didn't decrease at all.
• 59:39
  And that's something that, like, biological brains do all the time.
• 59:42
  But the key thing is what they can do is they can actually do simultaneous single-unit recordings of five different brain regions at the same time.
• 59:52
  Okay.
• 59:52
  So this is...
• 59:52
  Okay.
• 59:53
  And this is where the neurons come in, right?
• 59:55
  So we can now do electrophysiology, which is literally putting electrodes into the brain and then listen to thousands of neurons at the same time.
• 1:00:04
  I think it's on the order of a thousand in this study.
• 1:00:06
  From five different brain regions, the parietal, the lateral prefrontal cortex, the frontal eye fields, the temporal cortex, the striatum.
• 1:00:16
  Right.
• 1:00:16
  So this is where there's sensory motor information, reinforcement learning, pre-motor information.
• 1:00:24
  There's an executive hub, which is the prefrontal cortex, which is really the task rule.
• 1:00:28
  It's trying to figure out which task am I on in the first place.
• 1:00:31
  Right.
• 1:00:32
  And that is what is really cool.
• 1:00:35
  OK, is the fact that we could use all of these things.
• 1:00:38
  And so how do we actually make sure that like how do we show that these cognitive Legos are actually working?
• 1:00:45
  How do you demonstrate existence?
• 1:00:46
  Yes.
• 1:00:47
  How do you demonstrate that there's these manifolds and all this other stuff?
• 1:00:50
  Here's what they did.
• 1:00:51
  Okay.
• 1:00:51
  So you get a classifier that is trained on color.
• 1:00:55
  Okay.
• 1:00:57
  Okay.
• 1:00:57
  In task, in the color task.
• 1:01:01
  And what you do is you say, okay, now in this new task where it's trying to find color, but the motor response is completely different.
• 1:01:10
  Okay.
• 1:01:11
  When you say motor response.
• 1:01:13
  Meaning in the first task, the color response was upper left, lower right.
• 1:01:16
  Oh, I guess.
• 1:01:17
  But my motor response now in the second task is same color, but now I'm going in the opposite axis.
• 1:01:22
  Yes, yes, yes.
• 1:01:22
  Right?
• 1:01:23
  If I train a classifier on the color task and I want to decode the color, meaning I look at the neurons and I try to guess the color.
• 1:01:33
  Was the color red or blue?
• 1:01:35
  No way.
• 1:01:35
  Right?
• 1:01:36
  Green?
• 1:01:36
  No way.
• 1:01:37
  And I don't retrain the classifier.
• 1:01:39
  Right.
• 1:01:39
  I just now test it on the second task.
• 1:01:43
  Yes.
• 1:01:43
  Which is a totally different motor thing.
• 1:01:45
  Yes.
• 1:01:45
  It can still guess the color.
• 1:01:47
  No way.
• 1:01:47
  You see?
• 1:01:48
  This is the key.
• 1:01:49
  This is the key.
• 1:01:50
  Yeah, yeah.
• 1:01:51
  Now it clicked, right?
• 1:01:52
  Yes.
• 1:01:52
  This result is very profound because it means that the geometry of the representation in this high dimensional space is stable, right?
• 1:02:01
  That's actually, no, that, no, that is, and the way they got to, it's different.
• 1:02:08
  I love these neuroscience.
• 1:02:09
  People are so clever.
• 1:02:11
  Yeah.
• 1:02:12
  Because that's what that means.
• 1:02:13
  If you've trained a classifier here, and then now this classifier is not trained on this second task.
• 1:02:19
  Right.
• 1:02:19
  But because the neural representation is stable, I can take all of the learning I've done here and just apply it here.
• 1:02:25
  And I can guess...
• 1:02:26
  If it was looking at green or red or whatever.
• 1:02:28
  Which means that underlying manifold structure is the same.
• 1:02:32
  It's like a real thing.
• 1:02:34
  It's a thing.
• 1:02:35
  It's like a real mathematical thing in this high dimensional space.
• 1:02:41
  Wow.
• 1:02:41
  Very cool, right?
• 1:02:42
  This one is very good.
• 1:02:43
  And they did the same thing for the shape.
• 1:02:45
  Yes.
• 1:02:45
  They did the same thing for the direction of the motor and all that other stuff.
• 1:02:49
  That's like a really big deal.
• 1:02:50
  Mm-hmm.
• 1:02:52
  Yeah, I thought it was very, very cool.
• 1:02:55
  This is why everyone was rushing to get out their manifold stories early this year, because it seems to be a confluence of... Yeah, it is.
• 1:03:01
  It is kind of cool, because the next story we're going to talk about is about manifolds as well, right?
• 1:03:05
  Oh, my God.
• 1:03:06
  One of the other cool things that they did was, so, you know, if like they have a compression, they actually what they found was.
• 1:03:13
  Sorry.
• 1:03:14
  Yeah.
• 1:03:15
  This is cool.
• 1:03:16
  So cool.
• 1:03:17
  It's cool.
• 1:03:18
  They had also this thing called the compression index.
• 1:03:20
  What they found was like whatever the task is not yet.
• 1:03:24
  Oh, sorry.
• 1:03:25
  But like whatever the task is relevant for, like suppose it was paying attention to color.
• 1:03:29
  Yes.
• 1:03:29
  Then the shape manifold would actually like shrink.
• 1:03:31
  Yeah.
• 1:03:32
  Okay.
• 1:03:33
  Okay?
• 1:03:33
  Like, if you look at the neurons that were doing the shape stuff, they wouldn't be as active anymore.
• 1:03:39
  Because the subject is paying attention to the color first.
• 1:03:43
  And then...
• 1:03:43
  So even the manifolds themselves are dynamic in that sense.
• 1:03:48
  Yes.
• 1:03:48
  Right?
• 1:03:48
  The neurons that are in this plane are now a little bit shut off because they're not as important right now.
• 1:03:53
  Right?
• 1:03:54
  Or right now.
• 1:03:55
  You know, in this particular task.
• 1:03:57
  Yes.
• 1:03:57
  I mean, it's kind of like muscles, right?
• 1:03:59
  Very much.
• 1:04:00
  Like, if you're not using them, they kind of get a little smaller.
• 1:04:02
  Yeah.
• 1:04:02
  But then they're still kind of there.
• 1:04:04
  Yeah, but this is like, you know, on the order of minutes that they're like responding.
• 1:04:08
  Because the subject knows which task is happening.
• 1:04:13
  Happening in real time.
• 1:04:15
  And then subsequently which one to then prioritize or have active, in a more active state.
• 1:04:22
  Yeah.
• 1:04:24
  Very cool.
• 1:04:25
  It's very cool to see just how our brain has done this.
• 1:04:30
  Millions of years of evolution, and we're getting very close to solving stuff like the binding problem, which is a very big deal.
• 1:04:38
  And this has a lot of implications for artificial intelligence.
• 1:04:41
  I mean, immediately.
• 1:04:42
  Because catastrophic forgetting is very much a thing, right?
• 1:04:45
  Right.
• 1:04:45
  So there was actually this...
• 1:04:47
  There's a new way of trying to figure out how to...
• 1:04:53
  not do catastrophic forgetting so if we go to photo 15 right there's something called orthogonal subspace learning which is the idea that if i want to train on one task task a and then i also want to train on task b what i can do is i can make the two subspaces orthogonal inside my neural network and now when i do back propagation which we're going to get into later but it's just
• 1:05:24
  The model is going to train for task A, and the way that the model is going to train for task B are going to be orthogonal.
• 1:05:30
  Yes.
• 1:05:30
  So they're not going to interfere with each other.
• 1:05:32
  Yes.
• 1:05:32
  And you can enforce that constraint.
• 1:05:35
  I get you.
• 1:05:35
  Like, if I'm on task A, I'm only going to move along this plane in my parameter space.
• 1:05:41
  Yes.
• 1:05:41
  And if I'm trying to do task B, I'm only going to move along this plane in my parameter space.
• 1:05:45
  Right.
• 1:05:46
  And that way I don't forget how I did task A.
• 1:05:48
  Right.
• 1:05:48
  If I now all of a sudden start trying to learn task B.
• 1:05:52
  It's putting in roads and traffic lights into the pathways of how the processes proceed in a way that they don't interfere with each other.
• 1:06:04
  Mm-hmm.
• 1:06:05
  This is so good.
• 1:06:06
  So, yeah, it's really cool.
• 1:06:08
  I mean, there's this new AI architecture called Orthogonal Low-Rank Adaptation.
• 1:06:13
  Oh, Laura.
• 1:06:14
  Or Oh, Laura.
• 1:06:16
  It very recently came out.
• 1:06:17
  I think it was last year where they're trying to do this.
• 1:06:20
  Because the orthogonal idea, this like orthogonal subspace learning, conceptually, that maps to the same thing you were just talking about earlier about when you have...
• 1:06:29
  plane, this plane, and this plane.
• 1:06:33
  That's the orthogonal.
• 1:06:34
  And they're applying it now in the context of training.
• 1:06:38
  But it... We're able to experimentally see this.
• 1:06:41
  See this.
• 1:06:42
  That's what's so cool.
• 1:06:43
  That is... And like, straight up confirm it.
• 1:06:47
  Right.
• 1:06:47
  That is very good.
• 1:06:48
  I thought it was a very cool paper.
• 1:06:49
  I mean, it's hard to beat
• 1:06:52
  Princeton.
• 1:06:52
  Yeah.
• 1:06:52
  At anything.
• 1:06:53
  I mean, maybe football.
• 1:06:55
  You might get us in football.
• 1:06:57
  Like all the time.
• 1:06:58
  But when it comes to being the leaders in a variety of hard sciences, my brain is currently having a lot of orthogonal subspace.
• 1:07:10
  Yeah.
• 1:07:11
  Yeah.
• 1:07:12
  Yeah, every time we get on this podcast, I have to do that.
• 1:07:15
  A lot of orthogonal subspace learning.
• 1:07:18
  But it's very cool.
• 1:07:19
  And I think, you know, in terms of...
• 1:07:21
  So this is not just for fundamental science, right?
• 1:07:25
  You might be asking, okay, like...
• 1:07:26
  Cool guys.
• 1:07:27
  Cool guys, right?
• 1:07:28
  But, you know, cognitive inflexibility is one of the symptoms of psychiatric disorders like schizophrenia and ASD, right?
• 1:07:38
  And, you know...
• 1:07:40
  Given this kind of fundamental research, it could be that that inflexibility comes from the failure to orthogonally separate and compress subspaces in your head.
• 1:07:50
  So you can imagine like non-invasive neuromodulation therapies that would like orthogonalize.
• 1:07:58
  Sub spaces.
• 1:07:59
  Like make it more perpendicular.
• 1:08:00
  You have these headbands on that would be... Yeah, yeah.
• 1:08:03
  That would like...
• 1:08:04
  Exactly.
• 1:08:04
  That would do the thing.
• 1:08:05
  Yeah.
• 1:08:06
  But the point is there's a...
• 1:08:07
  I mean, that's far out.
• 1:08:08
  But like you got to get to this kind of fundamental research.
• 1:08:11
  You have to understand that the manifold is there.
• 1:08:13
  Yeah.
• 1:08:13
  Yeah.
• 1:08:15
  Yeah.
• 1:08:15
  And I think fundamentally what I really liked about this paper is because it provides a physics of meaning.
• 1:08:23
  It's like meaning is literally like geometry in your brain.
• 1:08:26
  Like your brain is living in this high dimensional subspace and there's like a geometry underlying it.
• 1:08:35
  Mm-hmm.
• 1:08:37
  that gives stuff meaning.
• 1:08:40
  I thought that was very cool.
• 1:08:41
  It's sort of the whole, this is why math is discovered, not invented.
• 1:08:46
  It's even in our brain.
• 1:08:47
  Right.
• 1:08:47
  Yeah.
• 1:08:48
  Very cool.
• 1:08:49
  Fascinating.
• 1:08:49
  This is our story number two out of Princeton neuroscience.
• 1:08:57
  I'm really struggling.
• 1:08:58
  I'm normally not like this.
• 1:08:59
  That story really kind of...
• 1:09:01
  I'm going to have to grok that one for a minute, but I have a good follow-up for it now because we are going to go into story number three, which is another deep seek moment.
• 1:09:11
  Um,
• 1:09:12
  Jan 1, they came into 2020.
• 1:09:14
  January 1st, dude.
• 1:09:15
  Just like all guns blazing.
• 1:09:17
  Guns blazing.
• 1:09:19
  It's all over the news, especially if you're in tech, et cetera, and all these other spaces.
• 1:09:23
  Yeah.
• 1:09:24
  But there's something about manifold constraints, and they broke the scaling laws, and everyone's hair on fire.
• 1:09:31
  How are they doing it?
• 1:09:33
  Ref, somebody, please.
• 1:09:35
  Yeah.
• 1:09:36
  Well, they're trying to beat scaling laws, right?
• 1:09:38
  Because they're GPU poor at the end of the day.
• 1:09:43
  Scaling laws are very effectively more compute, more parameters, better performance.
• 1:09:51
  If you have bigger, better stuff, you spend a lot of money.
• 1:09:53
  More chips, bigger, better, faster chips means you can get to the next stage of...
• 1:09:58
  Whatever you're trying to scale up.
• 1:10:00
  Yeah, yeah, exactly.
• 1:10:01
  But there's a caveat, right?
• 1:10:04
  Which is at massive scale, you've got like hundreds of billions of parameters.
• 1:10:07
  I think now we're getting to trillions of parameters.
• 1:10:09
  You're going to hit something called a stability limit.
• 1:10:12
  Okay.
• 1:10:12
  And in deep neural networks,
• 1:10:15
  It's all about signal propagation, right?
• 1:10:18
  Which is like, there's a signal of did I get something right or did I get something wrong?
• 1:10:22
  And as that propagates through my network, it's very chaotic, right?
• 1:10:27
  And it can lead to amplification, vanishing gradient problem, exploding gradient problem.
• 1:10:31
  These are stuff that we're going to talk about.
• 1:10:33
  So...
• 1:10:34
  There's some kind of limit, right?
• 1:10:35
  There might be some kind of like thermodynamic limit to how much we can do.
• 1:10:42
  And the solution is what DeepSeek is purporting to have found.
• 1:10:47
  Okay.
• 1:10:48
  It's something called manifold constrained hyperconnections.
• 1:10:51
  It's a new way to train models more efficiently and actually, like, straight up better.
• 1:11:01
  Okay?
• 1:11:01
  But it's a very small hack that they're doing.
• 1:11:03
  And it's actually quite simple.
• 1:11:05
  The hack that they're...
• 1:11:07
  The little hack that they did is, like, really simple.
• 1:11:10
  But...
• 1:11:11
  Apparently, it's a very big deal.
• 1:11:12
  Okay.
• 1:11:13
  Okay.
• 1:11:13
  So let's get into it.
• 1:11:14
  Okay.
• 1:11:15
  Neural networks.
• 1:11:16
  First, let's just talk about neural networks real quick.
• 1:11:18
  Okay.
• 1:11:19
  Neural networks are a giant mathematical function that basically it's fancy linear algebra.
• 1:11:24
  Okay.
• 1:11:25
  This is a transformer architecture from the very famous paper.
• 1:11:31
  Attention is all you need.
• 1:11:33
  Right.
• 1:11:33
  Sorry.
• 1:11:33
  I repeated after him.
• 1:11:34
  I almost got through the whole episode without doing it.
• 1:11:37
  It's hard to unlearn 15 years of friendship, please.
• 1:11:38
  Yeah, yeah, yeah.
• 1:11:40
  Well, so all of this is showing all of these arrows are basically numbers going from one thing to the other.
• 1:11:46
  And then each block is effectively a matrix.
• 1:11:49
  You're multiplying matrices, then you add stuff, then you multiply a matrix, then you do a threshold, then you add stuff.
• 1:11:55
  It's a giant mathematical function at the end of the day.
• 1:11:58
  And each of the building blocks...
• 1:12:01
  of these are something called an artificial neuron.
• 1:12:05
  Okay?
• 1:12:06
  And an artificial neuron is effectively kind of like a biological neuron in that it sums up its inputs, and then if that input is above a certain threshold, well, depending on the activation function, but let's just say it's above a certain threshold, then it lets it through.
• 1:12:22
  If it's below a certain threshold, then it doesn't let anything through.
• 1:12:25
  So that's that non-linearity part.
• 1:12:27
  Okay?
• 1:12:29
  And when we train a network, we've got these billions of parameters, billions of neurons that are all connected together.
• 1:12:35
  And what we want to do is something called back propagation.
• 1:12:38
  So you've got a network with weights, which are how these neurons are connected.
• 1:12:44
  What you do is you initialize the network with a bunch of random stuff.
• 1:12:48
  Usually it's like random weights.
• 1:12:50
  You can get more complicated that way.
• 1:12:53
  What it's going to do is it's going to have a guess at an answer.
• 1:12:56
  It's going to compare its guess to the real thing, compute a loss, which is how different was my guess from the real thing, and then...
• 1:13:06
  propagate that back through the network and change the weights such that the next time it's going to be a bit closer to the correct answer.
• 1:13:14
  That's what back propagation is.
• 1:13:16
  Literally, the movie Tenet is a movie about humans back propagating in like really.
• 1:13:21
  It's like I just made that connection in my head right now, but that's literally what they were doing.
• 1:13:25
  It's funny.
• 1:13:26
  That's so funny, actually.
• 1:13:27
  Yeah.
• 1:13:29
  So.
• 1:13:30
  So that's effectively what backpropagation is.
• 1:13:32
  And really what you're doing is like the chain rule in calculus where you take the change here and you multiply it.
• 1:13:39
  You take a derivative here and you multiply it to the next layer, to the next layer, to the next layer.
• 1:13:42
  And that's how you calculate how much you want to change your weights by.
• 1:13:46
  Okay.
• 1:13:46
  It's a product of these.
• 1:13:50
  derivatives right okay um something called the jacobian matrix we don't have to we don't have to get into what this jacobian matrix is but here is the key thing okay effectively whenever you're doing back propagation the amount that the weights here change is based on a product of all of these numbers right now if the numbers that you're multiplying are a bit less than one
• 1:14:15
  Let's say 0.9.
• 1:14:17
  But there's 100 layers.
• 1:14:19
  0.9 times 0.9 times 0.9 100 times is something like 0.00003.
• 1:14:26
  This is the vanishing gradient problem.
• 1:14:29
  Interesting.
• 1:14:30
  Okay?
• 1:14:31
  The idea is the stuff that's changing, the weights that are going to change, if I do this very naively, is only going to be towards the output.
• 1:14:43
  Okay.
• 1:14:44
  Of my neural network.
• 1:14:45
  All of the earlier layers that I have, like, you know, it goes input and then there's something, something, something, something all the way to output.
• 1:14:53
  My change, my backpropagation is never going to make it to these early layers.
• 1:14:58
  Because I've just been multiplying a number that's slightly smaller than one, smaller than one.
• 1:15:02
  I mean, in effect, I just used the idea of 0.9 to the 100, right?
• 1:15:06
  Sometimes it's 0.5, and then you're really screwed, right?
• 1:15:09
  So that's the vanishing gradient problem, if you're really bad at it.
• 1:15:14
  We solved this already, but I'm just letting you know.
• 1:15:16
  This is still kind of a problem.
• 1:15:18
  You couldn't just say, oh, backpropagating, we're fine.
• 1:15:20
  It's like, well, only if it's not greater than 1.
• 1:15:23
  Yeah.
• 1:15:23
  And also...
• 1:15:25
  So here's the problem that happens if that number that you're multiplying is a little bit smaller than 1.
• 1:15:30
  If it's larger than 1, then you have the opposite problem, right?
• 1:15:33
  Because 1.1 to the 100, 1.1 times 1.1 times 1.1 is something like 14,000.
• 1:15:35
  And so you have an exploding gradient.
• 1:15:36
  Mm-hmm.
• 1:15:41
  Right.
• 1:15:42
  Where like one, the system gets really chaotic.
• 1:15:45
  It's kind of like a chain reaction in like nuclear physics.
• 1:15:49
  Yes.
• 1:15:49
  Right.
• 1:15:50
  And there have been historical fixes like the long short term memory LSTM model.
• 1:15:56
  It used something called the constant air carousel.
• 1:15:58
  But it's it's that thing was limited to sequential data.
• 1:16:02
  OK.
• 1:16:03
  Right.
• 1:16:03
  Along came in 2015 ResNet.
• 1:16:05
  OK.
• 1:16:06
  Residual network.
• 1:16:07
  OK.
• 1:16:11
  Okay.
• 1:16:11
  Okay.
• 1:16:12
  They said, I'm not only going to have a feed forward layer.
• 1:16:15
  So it goes layer and then it does something goes to the next layer.
• 1:16:18
  I'm also just going to pass along the input.
• 1:16:22
  Oh, okay.
• 1:16:22
  As in just the same thing.
• 1:16:24
  Yes.
• 1:16:24
  So, so the next, the next part of the network, the next block is going to get my original input plus whatever computation I've done.
• 1:16:33
  Right.
• 1:16:33
  Okay.
• 1:16:33
  Your, your output from your part of the chain plus what you had as your input to get to that.
• 1:16:37
  Yeah.
• 1:16:37
  Yeah.
• 1:16:38
  And that's that residual connection.
• 1:16:39
  That's, that's why it's called the resnet.
• 1:16:41
  Okay.
• 1:16:41
  Okay.
• 1:16:41
  I like to just call it the skip.
• 1:16:43
  The skip.
• 1:16:44
  Okay.
• 1:16:44
  The skip connection.
• 1:16:45
  Okay.
• 1:16:45
  All right.
• 1:16:46
  And the skip connection is very cool because this is basically doing something called Euler's method, which is something that you learn in calculus.
• 1:16:53
  The idea is like if you've got a differential equation and you want to figure out where it's going, all you do is you say, okay, well, the slope is this way.
• 1:16:59
  I'm going to make a step, but I'm going to keep going in that direction and then make a step and then keep going in that direction.
• 1:17:04
  Right?
• 1:17:04
  So...
• 1:17:06
  That identity mapping, what it does is it ensures that you're not going to have that vanishing gradient problem.
• 1:17:12
  Right.
• 1:17:12
  Right?
• 1:17:13
  Because even if it's like 0.99 to the... You're still adding... Like, it's still 1...
• 1:17:18
  plus or minus something, right?
• 1:17:20
  One plus or minus something.
• 1:17:21
  And you can still have an exploding gradient because you could still have that plus, plus, plus.
• 1:17:26
  But then you have stuff like batch norm and layer norm that can keep that in check, right?
• 1:17:31
  So this kind of solved that, the ResNet.
• 1:17:33
  The ResNet problem kind of solved that, okay?
• 1:17:36
  Yes.
• 1:17:36
  The problem with the ResNet is you want to do better.
• 1:17:39
  You always want to do better.
• 1:17:41
  Okay?
• 1:17:41
  In this case, I've got one residual connection.
• 1:17:44
  It's like a single-lane highway.
• 1:17:45
  All I'm doing is I'm taking the input, I do a bunch of computation, and then I add it to the input again.
• 1:17:50
  Right?
• 1:17:51
  This thing is a single-lane highway.
• 1:17:53
  I'm not doing anything fancy here.
• 1:17:55
  Low bandwidth.
• 1:17:55
  Low bandwidth.
• 1:17:56
  I'm not doing anything fancy.
• 1:17:58
  2024-2025, a paper by ByteDance, their AI, which is the TikTok company.
• 1:18:04
  Yes.
• 1:18:05
  Right?
• 1:18:05
  Yes.
• 1:18:05
  Yes.
• 1:18:07
  They come up with something called hyper connections, which is what if we can start doing something fancy with that?
• 1:18:14
  You know, that skip connection.
• 1:18:16
  Instead of just like copy pasting.
• 1:18:18
  Right.
• 1:18:18
  What if I do something fancy in that lane?
• 1:18:21
  Right.
• 1:18:21
  Okay.
• 1:18:22
  And this is what they did.
• 1:18:23
  So on the left hand side is your original ResNet, which is just copy paste, copy paste, do some computation, copy paste like that.
• 1:18:29
  Yes.
• 1:18:30
  On the right hand side, what they're doing is you can imagine instead of a single lane highway, you've got multiple lanes and you're mixing traffic.
• 1:18:37
  Okay.
• 1:18:37
  Okay.
• 1:18:38
  Okay, so you've got all of this information, and you want to find the optimal way to send this information to the next place, right?
• 1:18:47
  This was the ByteDance idea from HyperConnections.
• 1:18:49
  This was a paper in 2024, 2025, right?
• 1:18:51
  This is good.
• 1:18:53
  So the idea is to always have more and more computation in all of these little segregated spots because then that makes your model better, right?
• 1:19:01
  Right.
• 1:19:01
  The more sort of...
• 1:19:03
  Fine tuning you're doing here and there and here and there.
• 1:19:06
  Yeah.
• 1:19:06
  Yeah.
• 1:19:07
  You're getting better, right?
• 1:19:07
  The model output is going to be better.
• 1:19:10
  Letting it run and then trying to like send back a conclusion that has now had these intermediate steps where there's no feedback loop.
• 1:19:19
  Yeah, there's no, and there's no, it's kind of just lazy.
• 1:19:22
  Yeah, yeah.
• 1:19:23
  Right?
• 1:19:23
  It's just like copy-paste.
• 1:19:24
  Yeah, yeah.
• 1:19:25
  Well, but what if the next step could use some more finagling, right?
• 1:19:30
  What if I could finagle this part to do a little bit better here, right?
• 1:19:35
  So that I get a better answer all the way at the end.
• 1:19:38
  Right?
• 1:19:38
  That's the whole idea.
• 1:19:39
  Makes sense.
• 1:19:40
  Right?
• 1:19:40
  So it's that multiple lanes kind of traffic merging.
• 1:19:44
  And the point is during training, we figure out what the rules of the traffic merging are, right?
• 1:19:49
  Mm-hmm.
• 1:19:49
  Okay, the problem is kind of the same of what we had earlier, which is if you're...
• 1:19:58
  The whole point of this skip connection was to make sure that I don't blow up and I don't vanish.
• 1:20:04
  But now, if I don't have that identity, I could start blowing up and vanishing here on this end.
• 1:20:12
  Right?
• 1:20:12
  It's the same problem that I had earlier.
• 1:20:14
  I gotcha.
• 1:20:15
  Okay?
• 1:20:16
  So, so...
• 1:20:19
  It's hard to use, these hyperconnections.
• 1:20:21
  Because they were just being let loose a little bit.
• 1:20:24
  Like, maybe there was a wide road, but not lanes.
• 1:20:29
  Yeah.
• 1:20:29
  In between what, like, would be, like, lanes for different things to go down.
• 1:20:33
  It was just like, ah, just everyone, like, the wide road, let's all go.
• 1:20:36
  And if, like, and if the highway itself is now, like, magnifying the amount of traffic.
• 1:20:40
  Right.
• 1:20:41
  Then you're running into the same problem that you were earlier.
• 1:20:43
  That makes sense.
• 1:20:44
  Right.
• 1:20:45
  Okay.
• 1:20:45
  And that is where this particular paper comes in.
• 1:20:48
  Okay.
• 1:20:49
  With something called a manifold solution.
• 1:20:51
  Okay.
• 1:20:51
  Okay.
• 1:20:52
  The idea is what you do is you say, okay, remember that hyperconnection, which is a way to change the...
• 1:21:03
  how we're merging this traffic stuff, right?
• 1:21:06
  That thing is just another matrix.
• 1:21:09
  Okay.
• 1:21:09
  That thing is just another block of numbers that's going to tell me how much of this vector is going to go into that one, how much of this vector is going to go into that one.
• 1:21:19
  And the problem is that if this matrix amplifies or de-amplifies, then I get into that vanishing or exploding problem, right?
• 1:21:27
  So what if I constrain that matrix to basically never do that?
• 1:21:32
  And that's what this is.
• 1:21:33
  They constrain the matrix to be doubly stochastic.
• 1:21:36
  And what that means is the sum of all the rows is one.
• 1:21:39
  The sum of all the columns is one.
• 1:21:41
  The question is, how do you make a matrix do that?
• 1:21:44
  It's not really a trivial way to do it.
• 1:21:47
  But so it's kind of a brute force algorithm, something called the Sinkhorn-NOP algorithm.
• 1:21:54
  All you do is you basically say, okay, take this row.
• 1:21:57
  Divide by the sum.
• 1:21:58
  So now everything sums to one.
• 1:22:00
  Next row, divide by the sum.
• 1:22:01
  Okay?
• 1:22:02
  And then you do it for the rows.
• 1:22:03
  Now you go along the columns.
• 1:22:05
  Then you go for the rows again.
• 1:22:06
  Then you do the columns.
• 1:22:07
  And one by one, it's like you're folding a pastry over and over again until you get a very nice...
• 1:22:14
  Matrix where all of the rows and all the columns sum up to one.
• 1:22:17
  The key is once you do that, you're no longer going to blow up or shrink.
• 1:22:24
  And that's the key insight.
• 1:22:26
  That's it.
• 1:22:26
  That's it.
• 1:22:27
  It's just an iterative brute force method to constrain the hyperconnection.
• 1:22:31
  That's it.
• 1:22:32
  That was the idea.
• 1:22:33
  That was the idea.
• 1:22:35
  It's like, you know, I want to do this hyperconnection stuff, but I don't want to deal with all of the nonsense of blowing up and all of that stuff.
• 1:22:43
  So I'm just going to put in a little hack where I make sure that these hyperconnections
• 1:22:48
  are constrained so that the sum is one.
• 1:22:51
  And so I'm not stretching.
• 1:22:53
  I'm not shrinking.
• 1:22:54
  So every time I'm not like doing 0.9 times 0.9, it's always one times one times one.
• 1:22:59
  I'm just mixing.
• 1:23:00
  I'm mixing it in a really nice way.
• 1:23:04
  i'm i'm not mad sometimes simplicity it's a very simple hack it's one of those things where some people are probably looking at that paper and being like i can't believe you've done this yeah it's it seems pretty simple now um again within it's not obviously yeah yeah but obviously but like but um this is very important for deep seek though right because deep seek um bingo
• 1:23:29
  It's hard for them to get GPUs.
• 1:23:32
  There was this very recent news that actually came out a month ago where like two dudes got caught with $160 million.
• 1:23:42
  This is, yeah, $160 million worth of GPUs that they were trying to smuggle into China.
• 1:23:48
  Yeah.
• 1:23:49
  This was a month ago.
• 1:23:50
  They're never getting out.
• 1:23:51
  So they're desperate.
• 1:23:52
  Yeah.
• 1:23:52
  Right?
• 1:23:53
  And that's – this is – so for context, right, and I believe this started under the Biden administration, there was sort of this doctrine about how do we win the sort of great power war for AI?
• 1:24:02
  And the idea was because people really buy into the scaling laws as like the –
• 1:24:08
  most accessible lever for us to pull and like the fastest way to sort of accelerate the progress.
• 1:24:15
  We continue to double down on domestic chip production.
• 1:24:20
  We have the lead on that because of the likes of people like NVIDIA who just have something no one else has.
• 1:24:26
  And we control all exports.
• 1:24:30
  of these, not only to countries like China, but a whole list of others.
• 1:24:35
  And there was this really big conflict when Trump administration came in because the argument was being made that, uh,
• 1:24:42
  That is not solving a problem.
• 1:24:45
  Yeah.
• 1:24:46
  And then it's also eliminating the ability to generate revenue that can then be applied and put in other places.
• 1:24:51
  Yeah.
• 1:24:51
  And so the Chinese and the research community there are having to be clever with not enough because we haven't seen Baidu or any of these larger –
• 1:25:04
  tech companies within China get to that point.
• 1:25:08
  They're getting, they're trying to get there.
• 1:25:09
  They're trying to get there, they're trying to get there, but they have not yet.
• 1:25:12
  And that's still the bottleneck.
• 1:25:13
  Yeah.
• 1:25:13
  And I think, I think very recently the Trump administration said that they're going to allow Nvidia to sell to China, but they're,
• 1:25:23
  They'd have to pay like a 25% tariff that would go to the government.
• 1:25:27
  So the government would like share in the profits, something like that.
• 1:25:30
  I'm not quite clear.
• 1:25:32
  But in any case, what we're trying to do is like choke out the Chinese AI industry by starving them of GPUs.
• 1:25:40
  So –
• 1:25:41
  the DeepSea guys have to be very clever in how they implement this.
• 1:25:45
  Because here's the idea, right?
• 1:25:47
  Modern GPUs, like the NVIDIA H100s, they're characterized by a massive disparity between GPUs,
• 1:25:56
  how much they can compute, so the compute capability, which is a number of flops, floating point operations, and their memory bandwidth in terabytes per second.
• 1:26:04
  This is called the memory wall, which refers to the latency and the energy costs associated with moving data from memory, which is this high bandwidth RAM, to your spam registers, which is the thing that the computing part of the GPU has access to.
• 1:26:22
  And so it's really slow to transfer stuff,
• 1:26:26
  And if you're starved of GPUs, you need to come up with clever ways to, if you want to do something very cool, like this manifold hyperconnection thingy, thingy, thingy, but you don't have a lot of GPUs, you got to get clever, right?
• 1:26:40
  And that part of the paper I actually thought was really cool, right?
• 1:26:44
  So you've got this bottleneck, right?
• 1:26:45
  And if you want to naively do something like this hyperconnection stuff...
• 1:26:52
  Before, you just had a single lane.
• 1:26:54
  Now you've got, let's say, four lanes.
• 1:26:57
  That's four times the compute that you have to do because you have to figure out.
• 1:27:02
  You have to read your input.
• 1:27:04
  Then you have to churn the numbers four times.
• 1:27:08
  And then that's expanding a lot of...
• 1:27:12
  Compute.
• 1:27:13
  Compute cost.
• 1:27:14
  It's four to five times higher than the original ResNet.
• 1:27:18
  Right.
• 1:27:19
  Okay?
• 1:27:19
  And when you're starved with GPUs, four to five times higher input-output is not something that...
• 1:27:25
  It's too inefficient.
• 1:27:26
  Yeah, you can't just buy like 500 more.
• 1:27:28
  Right?
• 1:27:29
  So what DeepSeek did was reduce that computational overhead to something that was negligible.
• 1:27:35
  Six to 6.7%.
• 1:27:38
  Really?
• 1:27:38
  Instead of four to five times.
• 1:27:40
  And here's how they did it.
• 1:27:41
  They used this language called TileLang.
• 1:27:44
  Okay.
• 1:27:44
  Okay?
• 1:27:45
  And what they're doing is literally overcoming the memory wall by something called kernel fusion.
• 1:27:50
  So whenever you're doing AI...
• 1:27:54
  Let's say you do like some kind of operation.
• 1:27:56
  You have to like start up a kernel.
• 1:27:58
  Yes.
• 1:27:59
  It's just like, it's kind of like a processing unit.
• 1:28:01
  Yes.
• 1:28:01
  Okay.
• 1:28:02
  And individual kernels are started up for individual tasks.
• 1:28:06
  Like there's matrix multiplication, there's the activation and stuff like that.
• 1:28:10
  But what these guys did was customize it so that all of the math is done in a single kernel.
• 1:28:16
  faster arithmetic and the intermediate results don't go to memory back and forth so you don't like you don't like do you don't like go to memory get some data do something put it back in memory go somewhere else put it back it's kind of like if you're like a chef like an industrial chef you don't go to the fridge all the time right it's like you get all your ingredients you put it on your stovetop the fridge is like your high bandwidth memory yeah
• 1:28:38
  The stovetop is like your SRAM, which is the countertop.
• 1:28:42
  So it's limited, but you want to do everything you can in this little time.
• 1:28:48
  And the kernel is your instructions.
• 1:28:50
  But if you can make your kernel very complicated and clever, then I don't have to go visit the fridge all the time.
• 1:28:58
  And that's exactly what they did.
• 1:29:00
  Reduce the global memory reads by a huge amount.
• 1:29:04
  That's fascinating.
• 1:29:06
  Which then opens up the bandwidth on memory.
• 1:29:10
  Exactly.
• 1:29:11
  So you're not constantly going back to store and retrieve in time as you're doing any number of different operations or processes.
• 1:29:18
  Yeah.
• 1:29:18
  And the proof is in the pudding.
• 1:29:20
  They've got a smoking gun.
• 1:29:21
  They used a 27 billion parameter model.
• 1:29:23
  The loss is lower.
• 1:29:24
  The gradients aren't blowing up.
• 1:29:28
  So it is very cool.
• 1:29:30
  Oh.
• 1:29:31
  But it's kind of insane that it was such a simple trick.
• 1:29:33
  It was just like, oh, just make the matrices add up to one.
• 1:29:37
  Just make sure it doesn't blow up.
• 1:29:39
  Yeah, no, that was the whole point of ResNet.
• 1:29:41
  And they just did it again.
• 1:29:44
  It's not in the same way.
• 1:29:45
  But like...
• 1:29:46
  Still, you know, the concept is very similar.
• 1:29:49
  And it's very physics intuition as well, because, you know, in quantum mechanics, for example, this reminded me a lot of quantum mechanics, because in quantum mechanics, you have, for example, quantum states, and then you have something called operators that operate on these quantum states, like a time evolution operator would take your quantum state now and tell you what happens sometime later and sometime later and sometime later, right?
• 1:30:10
  Yeah.
• 1:30:12
  And these operators have a condition that they have to be unitary, meaning the probability has to add up to one.
• 1:30:20
  Okay.
• 1:30:20
  Because all of the outcomes need to be all of the outcomes.
• 1:30:23
  You can't have an operator that expands probability so that the sum of all probabilities is two.
• 1:30:29
  But there's a very central physical insight, right?
• 1:30:32
  It's like conservation of probability.
• 1:30:33
  You also have conservation of energy.
• 1:30:35
  In this case, they have like conservation of norm or something like that.
• 1:30:38
  You know, it's like a very simple insight.
• 1:30:42
  And this goes back to the way you kind of always talk about and see the world, which is like you can apply the mental model that we have as physicists into other domains.
• 1:30:52
  Yeah.
• 1:30:53
  And it gives you, it's like a way of thinking about stuff and systems.
• 1:30:57
  Yeah.
• 1:30:57
  And, you know.
• 1:30:59
  order and progress and and it because it it's funny that we again did the prince and neuro story yeah followed up by this one because they're there it's like two different types of manifolds yeah in this case the manifold is like constrain it into the size yes right there it was like a a giant smooth sort of manifold here it's like just make it small and and the what is and what was driving the
• 1:31:27
  you know, with the sort of deep sick story, it's, it's obviously very much focused on like a commercial outcome.
• 1:31:35
  The insight was, uh,
• 1:31:37
  not particularly, uh, not didn't have a huge wow fact.
• 1:31:42
  The outcome had wow fact, not the intuition.
• 1:31:45
  Yeah.
• 1:31:45
  Whereas I feel like with the, the Princeton neuroscience story, like the intuition intuition.
• 1:31:49
  Yeah.
• 1:31:49
  That was cool.
• 1:31:50
  It was really, really cool.
• 1:31:51
  Yeah.
• 1:31:52
  We, I know every, a lot of the AI community and the sort of tech community, uh,
• 1:31:58
  None of them knew what hyper...
• 1:32:03
  Manifold constrained hyperconnections.
• 1:32:06
  Everyone was like, what is...
• 1:32:07
  I don't know what manifolds are.
• 1:32:09
  I don't know what constrained means in this context.
• 1:32:11
  I don't know what hyperconnections are.
• 1:32:12
  But now you do, right?
• 1:32:13
  Hyperconnections is the mixing of the traffic.
• 1:32:16
  The manifold constraint is to just say the thing that is mixing the traffic better not amplify the traffic or shrink the traffic.
• 1:32:23
  The traffic better be the same.
• 1:32:25
  This is obviously going to make models.
• 1:32:27
  Everyone's going to start in different ways and capacities using this.
• 1:32:30
  Exactly.
• 1:32:31
  It's going to make models much, much better.
• 1:32:33
  Yeah.
• 1:32:34
  And it's also like kind of a shift in the paradigm of just like it's just compute, compute, compute, right?
• 1:32:41
  Because right here, this is algorithmic innovation.
• 1:32:44
  Yes.
• 1:32:44
  It's acting as a substitute for compute.
• 1:32:46
  You bring up a really good point because that's been the...
• 1:32:50
  People sort of say the argument is like, oh, the Americans are just being fat and lazy or being dumb and lazy in this context because they just want to throw money at the problem.
• 1:32:59
  And then they're not actually doing any of the variety of areas of fundamental research.
• 1:33:03
  I mean, some people argue that you can't just scale without – this is the Jan LeCun.
• 1:33:07
  Yeah, Jan LeCun says this a lot.
• 1:33:09
  Yeah.
• 1:33:11
  whatever you want to define as AGI by scaling LLMs, you need some or some subset, even Dario at Anthropoc kind of says the same thing.
• 1:33:19
  You need some subset of other insights or breakthroughs around it.
• 1:33:24
  But, you know, this is a perfect example of identifying that there's a lot of green space still.
• 1:33:31
  Yeah.
• 1:33:32
  In these areas that people might not be focusing on because the rat race around being the first AGI has people just like, what is the easiest way to get there quickly?
• 1:33:41
  Yeah.
• 1:33:41
  Throw money at the problem.
• 1:33:42
  Yeah.
• 1:33:43
  And so we don't get clever.
• 1:33:44
  Very interesting.
• 1:33:45
  Again, the US-China nexus is not going anywhere.
• 1:33:48
  Nope.
• 1:33:49
  It is what it is.
• 1:33:50
  Yep.
• 1:33:51
  And so we've hit Rome.
• 1:33:54
  We went really far back in time.
• 1:33:57
  Then we went real, real deep on the brain.
• 1:34:00
  Yeah.
• 1:34:02
  And then we kind of went a little bit, we went shallow seek on AI, the fake brain.
• 1:34:11
  And now we're going to move to our last story of the day, which we have aptly titled Econophysics.
• 1:34:19
  That's right.
• 1:34:20
  As you can see, we are donned in our finance bro garb.
• 1:34:24
  Yeah.
• 1:34:25
  I got my Jackson Hole, you know.
• 1:34:27
  Yeah.
• 1:34:28
  Bunch of finance bros there, let me tell you.
• 1:34:30
  Last time I went.
• 1:34:32
  And I have no idea where this is going to go.
• 1:34:37
  Yeah, this one, I was pleasantly surprised.
• 1:34:41
  And it caught my eye because this was a paper in PRL, physical review letters.
• 1:34:47
  This is the story journal that has published, you know, superconductivity, the black hole papers, Einstein, Wheeler,
• 1:34:56
  cosmic microwave background and then a universal law about stock prices and the impact that orders have on the price of an asset and i was like what is this doing in a physics journal okay so here's the central question okay um you know when when you trade stocks
• 1:35:20
  Stonks.
• 1:35:20
  Stonks.
• 1:35:21
  Yeah.
• 1:35:22
  When you trade stocks, right, the trade itself affects the price of the asset.
• 1:35:28
  Okay?
• 1:35:29
  The question is how does that price change?
• 1:35:33
  Okay?
• 1:35:33
  There's something called the square root law, which says that if I put in an order for a certain amount—
• 1:35:42
  The amount that the price is going to change as a response to that order is proportional to the square root of the size of my order.
• 1:35:49
  Okay?
• 1:35:50
  And it's sort of just been observed, but this was a paper that was trying to figure out, is it a square root?
• 1:35:56
  Meaning, is the exponent a one-half, or is it one-fourth, or is it like 0.4, is it 0.6?
• 1:36:03
  What is it?
• 1:36:03
  Okay?
• 1:36:04
  And...
• 1:36:05
  The field of econophysics is actually quite old.
• 1:36:09
  Okay, all the way back in 1900, there was this guy, Louis Bachelier.
• 1:36:15
  He wrote his senior thesis, his PhD thesis, La Speculation.
• 1:36:20
  That's my French.
• 1:36:21
  I was going to say it.
• 1:36:25
  But it's a physics thesis about speculation in the markets.
• 1:36:31
  He actually predated Brownian motion.
• 1:36:34
  Like this is 1900.
• 1:36:35
  So this is before Einstein's 1905 paper on Brownian motion.
• 1:36:39
  Yes.
• 1:36:40
  Derived the diffusion equation.
• 1:36:41
  It's a precursor to Brownian motion.
• 1:36:43
  All of this stuff just to get rich off speculation.
• 1:36:46
  Right.
• 1:36:48
  We love it.
• 1:36:49
  And that was sort of the birth of like trying to use statistical mechanics and insight from physics to try and talk about agents in a stock market, in a market with, you know.
• 1:37:01
  people that are trying to buy and sell stuff.
• 1:37:03
  Yes.
• 1:37:03
  Okay?
• 1:37:05
  In the 1990s, there was a guy, Eugene Stanley, he actually figured out something called the inverse cubic law of price returns, which means if you plot the returns that you get and the probability distribution of the returns based on, like, all of the assets everywhere, right?
• 1:37:24
  Yeah.
• 1:37:25
  It goes down like returns to the negative three power.
• 1:37:30
  Okay.
• 1:37:31
  Okay?
• 1:37:31
  Meaning like if you want ten times the returns, the probability of getting ten times the returns is one divided by ten to the three compared to just getting the same return.
• 1:37:41
  Does that make sense?
• 1:37:42
  Yes, it does.
• 1:37:43
  It does.
• 1:37:43
  Okay.
• 1:37:44
  So, and it's the same for like whether you wait one day, whether you wait two days, whether you wait three days.
• 1:37:50
  It's kind of weird.
• 1:37:51
  That is a little weird.
• 1:37:52
  Like the probability distribution remains the same.
• 1:37:54
  Yeah, I don't like that.
• 1:37:55
  Okay.
• 1:37:56
  Another one was the probability density of transaction sizes.
• 1:38:00
  If you look at all of the transactions and the New York Stock Exchange, in the Paris Exchange, in the London Stock Exchange, and you plot the trade size on the x-axis and like how many trades happen at a certain size, at another certain size, then the exponent is negative 2.5.
• 1:38:22
  Okay?
• 1:38:22
  Yeah.
• 1:38:22
  Which means that no matter what stock market you're looking at, it's the same trend.
• 1:38:28
  I don't like that.
• 1:38:29
  It's really weird.
• 1:38:30
  Yeah, I don't like that.
• 1:38:31
  Right?
• 1:38:31
  And whenever physicists look at something like this, it reminds them of something called universality.
• 1:38:37
  Okay?
• 1:38:38
  Universality means there's some weird underlying physics that's happening.
• 1:38:42
  Yes.
• 1:38:43
  That is invariant of the thing that is...
• 1:38:47
  involved in the data collection.
• 1:38:51
  Here's what I mean.
• 1:38:52
  Like in universality, what we a lot of times think about is phase transitions.
• 1:38:56
  So if you think about gases or any substance that's going from liquid to gas, and it's at something called the critical point, which is where the liquid and the gas phase are like sort of same, same, right?
• 1:39:12
  And you plot something like the specific heat of this substance, okay, as you change some parameter, like temperature or density or something like that.
• 1:39:24
  Regardless of whether you're using helium or oxygen or this or that, the shape of that curve is going to be exactly the same, which means there's some underlying physics that is invariant of...
• 1:39:37
  The actual mechanics of the particles.
• 1:39:40
  It goes itself.
• 1:39:41
  Yeah.
• 1:39:41
  There's something much deeper.
• 1:39:43
  Yes.
• 1:39:43
  Okay?
• 1:39:43
  So whenever we see stuff like that, we're like, there's something.
• 1:39:47
  Okay?
• 1:39:47
  And that's, I think, why this made it to PRL.
• 1:39:50
  That makes sense.
• 1:39:51
  Okay?
• 1:39:51
  That makes sense.
• 1:39:52
  I think that's why this made it to the physical review.
• 1:39:54
  Okay.
• 1:39:55
  So the empirical law is that square root law.
• 1:40:01
  Okay?
• 1:40:02
  It's that if I...
• 1:40:05
  I put in a buy order of some size, let's say X, and then I put in and then somebody else puts in a buy order that's four times, right?
• 1:40:16
  My buy order is going to move the price sum amount.
• 1:40:20
  But their buy order is not going to move the price by four times.
• 1:40:23
  It's only going to move the price by twice what mine moved.
• 1:40:26
  Okay.
• 1:40:26
  Because they did four and the square root of four is two.
• 1:40:29
  If somebody put in nine times by my order, it would only move the price by three.
• 1:40:34
  And it kind of makes sense because, you know, if some asset is like trading at $10 and then I put in like I want to buy something at like $10.1.
• 1:40:48
  Right.
• 1:40:49
  There's not going to be a lot of people who are willing to sell it to me.
• 1:40:53
  So the price is really going to go to 10.1.
• 1:40:56
  Right.
• 1:40:56
  But if I if I put in an order for like I want to buy like a bunch, a bunch at like 12 or something.
• 1:41:03
  Right.
• 1:41:04
  There's going to be a lot of sellers that are going to be really keen.
• 1:41:07
  And so the price is not going to move that much because a bunch of sellers just like fill that order.
• 1:41:11
  Yes.
• 1:41:12
  Right.
• 1:41:12
  So that's the intuition behind it.
• 1:41:14
  Okay.
• 1:41:15
  The idea is that it's nonlinear in that sense, right?
• 1:41:19
  The more volume that I want to buy or sell, the less the price is going to get impacted.
• 1:41:23
  It's not linear.
• 1:41:24
  Linear would mean that it's directly proportional.
• 1:41:26
  Here, it's like a diminishing return type thing.
• 1:41:29
  Yes.
• 1:41:30
  Okay.
• 1:41:31
  So...
• 1:41:32
  The question is, is this truly universal, right?
• 1:41:35
  Yes.
• 1:41:35
  Now, in order to actually measure something like this, right?
• 1:41:39
  I mean, there's a lot of people who think that it's not totally universal, right?
• 1:41:43
  Maybe it depends on the microstructure.
• 1:41:45
  Like, there's inventory risk models, like how much inventory is out there in the market.
• 1:41:50
  Like, if there's large traders, like what if there's like a giant whale, you know, when we talk about Bitcoin whales?
• 1:41:55
  Yeah, yeah, yeah.
• 1:41:56
  If there's a market where there's only big orders, right?
• 1:42:01
  No one's buying, like, small stuff.
• 1:42:02
  Right.
• 1:42:03
  And it doesn't even matter if you're buying small stuff.
• 1:42:05
  Then does that change the dynamics of the square root law, right?
• 1:42:08
  Okay.
• 1:42:09
  Which makes sense.
• 1:42:10
  Which totally makes sense.
• 1:42:11
  But the universality hypothesis says that no, it's still 0.5.
• 1:42:15
  Regardless of any of those.
• 1:42:17
  The exponent is always going to be square root.
• 1:42:19
  Okay.
• 1:42:19
  Okay.
• 1:42:20
  So, um...
• 1:42:23
  To discern between whether it's one or the other, you have to get some kind of very good data, okay?
• 1:42:29
  Because normally when we want to analyze data, there's these things called meta orders, right?
• 1:42:34
  Like there's a hidden parent order, and then there's anonymous like child orders.
• 1:42:38
  Like if I want to buy like 10 stocks of a particular asset...
• 1:42:43
  I could just put in an order for 10 or I could be like, okay, I want to buy two and then I'm going to buy three and then I'm going to buy two and then I'm going to buy three.
• 1:42:50
  That's four different little child orders for one big meta order.
• 1:42:54
  Right.
• 1:42:54
  Which is that you bought 10.
• 1:42:55
  Yeah.
• 1:42:56
  Yeah.
• 1:42:56
  And in normal sort of trading in the data that you get if you want to analyze this kind of stuff.
• 1:43:03
  You only have access to the little orders.
• 1:43:05
  You don't have access to the guy who's, right?
• 1:43:08
  Because each of these people have like a virtual server.
• 1:43:13
  So each of the little orders might have like a virtual server ID.
• 1:43:18
  And so it's going to be like pretty impossible to reconstruct who did which one, right?
• 1:43:22
  Right.
• 1:43:22
  Okay.
• 1:43:23
  So this is where the Sato Kanazawa study came.
• 1:43:26
  that was just published in PRL in 2025, and December 2025 comes in.
• 1:43:31
  Strict universality.
• 1:43:34
  They show that this is a very strict universality, and it's a complete survey of the Tokyo Stock Exchange.
• 1:43:39
  So the Tokyo Stock Exchange, what they did was they got a bunch of data from that stock exchange for eight years, so from 2012 all the way to 2019.
• 1:43:49
  And the key innovation is they figured out a way to stitch together the effective trader IDs, so the big meta orders, from these virtual server IDs.
• 1:44:00
  They kind of de-anonymized it.
• 1:44:01
  Yes.
• 1:44:02
  Yeah.
• 1:44:02
  It's still anonymous, but they can pool them together directly.
• 1:44:07
  to recognize a single trader.
• 1:44:10
  And the way they did it, the reconstruction method is actually pretty cool.
• 1:44:14
  So here's what they did.
• 1:44:14
  They developed a heuristic that said, okay, I'm going to aggregate IDs such that if server A submits an order and then server B cancels that order, the only way that's possible is if both of these servers are connected to the same trader.
• 1:44:31
  So every other time...
• 1:44:33
  I'm going to treat that, right, as it's pretty good.
• 1:44:37
  That's really good.
• 1:44:38
  And so you get these effective trader IDs.
• 1:44:41
  You get the trader desk, right, in these, like, firms.
• 1:44:44
  Yes.
• 1:44:44
  Right?
• 1:44:45
  And this allows them to analyze the behavior of all the active traders across all liquid stocks, all 2,000 stocks for nearly a decade.
• 1:44:55
  It's like a God view.
• 1:44:57
  Of the market.
• 1:44:58
  I love that.
• 1:44:59
  Of the Tokyo Stock Exchange, right?
• 1:45:00
  Yeah.
• 1:45:01
  Yeah.
• 1:45:01
  Very cool.
• 1:45:02
  And that's clever.
• 1:45:03
  And again, it doesn't take a large leap to get that connection.
• 1:45:08
  Yeah, yeah, yeah.
• 1:45:09
  But they did the due diligence, right?
• 1:45:10
  Yeah, yeah.
• 1:45:10
  And they actually did it.
• 1:45:11
  Yeah, yeah.
• 1:45:12
  And so when they actually go through and do the math, they find that the mean exponent...
• 1:45:18
  is about 0.489, very close to 0.5.
• 1:45:22
  The standard deviation is 0.071.
• 1:45:25
  And there on the left, you can see all of the different stocks they use.
• 1:45:28
  Toyota, NTT, I don't know that stock.
• 1:45:32
  But all of them are the same line.
• 1:45:36
  Yes.
• 1:45:36
  Yes.
• 1:45:37
  And they're all centered at 0.5.
• 1:45:38
  And if you look across all 2,000 stocks, the spread is quite big, I'll admit.
• 1:45:42
  It's like from 0.4 all the way to 0.6.
• 1:45:45
  But the mean of that spread is very sharply at 0.5.
• 1:45:48
  Okay?
• 1:45:48
  Mm-hmm.
• 1:45:50
  So—
• 1:45:52
  From this data set, they can actually falsify some of the earlier models.
• 1:45:55
  There's this one model called the GGPS model, which is the, I don't know, GoBikes, Gopi Krishnan, Pluro, Stanley model from 2003.
• 1:46:04
  This is the model of inventory risk.
• 1:46:07
  This is the one that's saying that, like, okay, market makers, they provide liquidity, but they face the risk, right?
• 1:46:12
  Because, like, if I'm a market maker, meaning, like, I have, like, I'm the one, if you're trying to sell, I'll buy from you, and if you're trying to buy, I'm going to,
• 1:46:21
  I'll sell to you and I'm going to make money on the spread.
• 1:46:24
  Yes.
• 1:46:25
  But if the price like goes off, then I'm just left with the bag, the bag.
• 1:46:30
  Right.
• 1:46:31
  And I, that I didn't want, or maybe I did want.
• 1:46:33
  Okay.
• 1:46:34
  So they charge a premium and that price impact is going to compensate for that risk.
• 1:46:39
  Okay.
• 1:46:40
  So what if, what if this stuff's and the scaling argument is the size of the trades follows a power law, but to limit their risk,
• 1:46:48
  They're going to adjust the price to balance the probability of large orders.
• 1:46:53
  Okay?
• 1:46:54
  And there's another sort of numerical coefficient that comes in beta that you can figure out, which is sort of the probability of these large orders.
• 1:47:02
  And they say that the alpha, which is the exponent of my price impact, the one half that we're talking about here, the square root, is related to this beta.
• 1:47:12
  So these guys plotted those two, and there's no correlation.
• 1:47:16
  It's flat.
• 1:47:16
  It's flat.
• 1:47:18
  Okay?
• 1:47:19
  So this 2003 model, this GGPS model, completely wrong.
• 1:47:22
  Nonsense.
• 1:47:23
  Yeah.
• 1:47:24
  Okay?
• 1:47:24
  Okay?
• 1:47:25
  Like most things in economics.
• 1:47:27
  But okay.
• 1:47:29
  You know?
• 1:47:29
  There's something wrong with the mic.
• 1:47:32
  So zero correlation there.
• 1:47:33
  There was also another fair pricing efficiency model.
• 1:47:36
  That's crazy.
• 1:47:37
  Yeah.
• 1:47:38
  Yeah.
• 1:47:39
  Zero correlation.
• 1:47:40
  Right.
• 1:47:40
  Yeah.
• 1:47:40
  So there's this FGLW model, which is the Farmer Gehrig Lilo Weiland Broic model.
• 1:47:47
  That also was falsified.
• 1:47:49
  I'm not going to get into that.
• 1:47:50
  But it's about market efficiency and fair pricing.
• 1:47:53
  So I was reading this and I was like, okay, I would like to know mechanistically, though, what's going on.
• 1:47:58
  Right.
• 1:47:59
  Okay?
• 1:47:59
  I'm a physicist.
• 1:48:00
  It's like, what's actually going on?
• 1:48:01
  Right.
• 1:48:02
  Right?
• 1:48:02
  Because there's clearly some underlying...
• 1:48:04
  Yes.
• 1:48:04
  The underlying physics is there.
• 1:48:06
  That's there.
• 1:48:06
  There's some explanation.
• 1:48:07
  That's none of these other things.
• 1:48:09
  Yes.
• 1:48:09
  Okay.
• 1:48:10
  So...
• 1:48:10
  The victor comes from something called latent order book models.
• 1:48:14
  This is from Bouchard, Donier, and Mastro Mateo in 2015.
• 1:48:20
  Bouchard actually holds a PhD in theoretical physics from ENS, which is like the top university in Paris.
• 1:48:27
  And his analogy is reaction diffusion in chemical reactions.
• 1:48:33
  Imagine you've got two different types of particles.
• 1:48:36
  You've got a bi-particle and you've got a cell particle.
• 1:48:38
  If they meet up, they annihilate.
• 1:48:41
  Right.
• 1:48:41
  So they they they they poof out of existence.
• 1:48:45
  Right.
• 1:48:46
  The bi particles are on one side and the cell particles are on the other side.
• 1:48:50
  OK. And they all want to get to each other.
• 1:48:53
  But because of the nature of the reaction in the middle, there's not going to be any.
• 1:48:57
  Because if they were in the middle, they would annihilate.
• 1:49:00
  Right.
• 1:49:00
  So you're going to get a density profile that is roughly linear.
• 1:49:04
  Right.
• 1:49:05
  On the right-hand side and the left-hand side.
• 1:49:07
  And I think the next picture shows this very well, which is the density profile being very much linear.
• 1:49:13
  So you're asking for a certain price above the current price, and that's in the red.
• 1:49:19
  So this is like how much I would sell for.
• 1:49:22
  Yes.
• 1:49:23
  And then in the blue is how much I would buy for.
• 1:49:25
  Mm-hmm.
• 1:49:26
  And it's roughly linear, right?
• 1:49:28
  Okay.
• 1:49:29
  So now imagine with this linear sort of density profile of my order book.
• 1:49:37
  These are the order books, right?
• 1:49:39
  Suppose some order of size Q comes in.
• 1:49:44
  Okay?
• 1:49:45
  How much of it is going to go away?
• 1:49:49
  It's going to be that triangle.
• 1:49:50
  Okay?
• 1:49:51
  Right.
• 1:49:52
  And what's the area of the triangle?
• 1:49:53
  The area of the triangle is one half times the length and the width.
• 1:49:59
  Right.
• 1:50:00
  So it's the total size is my area.
• 1:50:04
  And that's proportional to the price impact squared.
• 1:50:10
  So the price impact is proportional to the area square root.
• 1:50:15
  Yep.
• 1:50:17
  It's strictly universal.
• 1:50:19
  It's like it makes sense.
• 1:50:21
  It has to do with the shape of the order book.
• 1:50:25
  Right.
• 1:50:25
  Right.
• 1:50:25
  The area is the price impact squared.
• 1:50:28
  Yes.
• 1:50:28
  Yes.
• 1:50:29
  And so the price impact is the area square root.
• 1:50:32
  And the area is the size of your order.
• 1:50:35
  So can we all go home?
• 1:50:36
  It's, like, pretty simple.
• 1:50:37
  I thought that was, like, kind of cute.
• 1:50:39
  No, that is – because it – you know, there's always a lot of hand-waving.
• 1:50:45
  Yeah.
• 1:50:45
  The thing I love about this is, like, no, but – Yeah, how does it work, though?
• 1:50:49
  Show me.
• 1:50:49
  Like, yeah.
• 1:50:50
  Don't – well, like, we have to, you know –
• 1:50:53
  some risk of that.
• 1:50:54
  Yeah, no, it's like, no, just, oh, it's okay.
• 1:50:56
  It's just an area.
• 1:50:58
  That's why you guys don't want to talk about it, because you're trying to take your premiums, because you know, anyway.
• 1:51:05
  This is actually really cool, econophysics.
• 1:51:07
  Yeah, and it makes sense now that it was in PRL.
• 1:51:11
  It's a pretty cool study, like the way that they batched the orders together to figure out what the trading desk was doing, things like that.
• 1:51:19
  I think it was a really cool study.
• 1:51:21
  um you know the practical impact i don't know somebody's probably going to make billions in slippage well yeah based on this or it's probably going to be priced in because now this is now that this it's it probably already is priced in because like the economists don't actually care about like the the finance guys don't actually care about the underlying like oh why is it doing it's like oh it's already been an empirical observation so they're gonna put it in their algorithms anyway so you know i i don't know if like
• 1:51:47
  You're going to make a lot of money off this insight, but it's a cool insight nonetheless.
• 1:51:50
  This is not financial advice.
• 1:51:52
  No.
• 1:51:53
  Yeah.
• 1:51:54
  Don't go do experiments.
• 1:51:55
  Hey, I wonder if I buy 100 Bitcoin.
• 1:51:57
  What's going to happen?
• 1:51:59
  We're just some physics nerds over here.
• 1:52:01
  Yeah.
• 1:52:01
  Yeah.
• 1:52:02
  So don't come cry.
• 1:52:05
  Because I think you can find there's structure all around us.
• 1:52:08
  Yeah, exactly.
• 1:52:09
  Yes, very good.
• 1:52:10
  There's structure all around us.
• 1:52:13
  It's accessible, but sometimes it may be a little opaque or discreet.
• 1:52:20
  In terms of how we engage and move around in the world.
• 1:52:23
  Yeah, yeah.
• 1:52:24
  And then how it becomes obvious.
• 1:52:26
  That's a very good point.
• 1:52:26
  There is structure all around us.
• 1:52:28
  I like that.
• 1:52:29
  I mean, they're all of our stories.
• 1:52:31
  We started with concrete, right?
• 1:52:34
  And that's pretty good with structure.
• 1:52:36
  Then we went to cognitive Legos and the structure of manifolds.
• 1:52:38
  Yeah, in our brain.
• 1:52:40
  In our brain and biological beings.
• 1:52:42
  Then we went to the similar kind of structure for manifolds and constrained hyperconnections.
• 1:52:48
  in the the human brain we're trying to replicate in silica and then the structure of this whole thing people view as like you need to be galaxy brain to beat the markets and all of this it's like it's actually just it's just a area of a triangle
• 1:53:08
  We hope you guys are excited.
• 1:53:10
  This season two, we have a lot cooking.
• 1:53:13
  As you can tell, we've tried to do some updates and upgrades.
• 1:53:18
  Not a perfect execution, but we've learned a lot in this first episode.
• 1:53:22
  We are trying to do longer overlays now with better viewability because we do know many of you want to actually digest the graphics and see it.
• 1:53:30
  For our listeners who are listening on the pod, we are a video podcast.
• 1:53:33
  We're available on Spotify.
• 1:53:34
  We're available on Spotify.
• 1:53:36
  We're available on YouTube for our full-length episodes and clips all over the socials.
• 1:53:40
  If you really want to kind of grok what we're talking through, especially some of the stories in today's episode, the graphics are super, super helpful.
• 1:53:47
  Christian, you always do a good job of trying to verbalize what's being seen, but there's so much detail that's lost in it.
• 1:53:53
  And I did mean to bring this up at the beginning of the episode, but there were too many new variables that I was worried about that I forgot to.
• 1:54:00
  However, if you made it to the end, you get to know this before anybody else.
• 1:54:03
  Our new website has just gone live and there's a sort of a couple of interesting things.
• 1:54:09
  So obviously, if you want to support the pod, you can go there and do so.
• 1:54:11
  But all of our episodes are there.
• 1:54:13
  And...
• 1:54:14
  We are backlogging all of the research papers for season one.
• 1:54:17
  And on an ongoing basis, we'll be putting all of the research papers with key takeaways, the related episodes, all in one place.
• 1:54:25
  So you can listen to us while you're on your commute to work or while you're in the middle of the gym workout.
• 1:54:31
  And if you're like, I actually am really kind of, I kind of really picked my brain.
• 1:54:35
  I want to dig in more.
• 1:54:36
  You have an easy access to the source, the sort of nature paper that it's from or the journal, the publisher, impact factor.
• 1:54:44
  We have some really, really cool ideas for how we can start to do some interesting things with the leaderboard and rankings.
• 1:54:51
  I won't leak too much.
• 1:54:53
  Yeah.
• 1:54:54
  We also did get confirmation for our first on-site podcast.
• 1:55:00
  I'll zip the lip.
• 1:55:01
  Yeah, yeah, yeah.
• 1:55:02
  That's it.
• 1:55:03
  It's all you get.
• 1:55:04
  But you can only get it if you go to the end of the podcast.
• 1:55:06
  Or maybe we'll sprinkle it in the middle.
• 1:55:07
  Who knows?
• 1:55:08
  So you got to listen because you never know what you're going to get.
• 1:55:11
  This was a good, long, two-hour epic.
• 1:55:14
  Oh, wait.
• 1:55:15
  We need to tell the listeners if they made it this far what to comment.
• 1:55:19
  Ah, yes.
• 1:55:19
  Yes.
• 1:55:20
  Okay.
• 1:55:20
  This is good.
• 1:55:21
  If you have an idea, I'll let you cook on it.
• 1:55:25
  I don't know if I...
• 1:55:28
  Yeah, if you guys write down finance and physics.
• 1:55:35
  No, that isn't.
• 1:55:36
  No, no.
• 1:55:37
  Don't do that.
• 1:55:37
  Don't do that.
• 1:55:38
  That's so bad.
• 1:55:38
  We're going to edit that out.
• 1:55:43
  This is why I don't do that.
• 1:55:44
  So I don't know why you gave me the fucking, you know, like.
• 1:55:48
  You're the promo guy.
• 1:55:49
  Come on.
• 1:55:50
  Dude, dude.
• 1:55:54
  My brain, you fried my brain so hard today.
• 1:55:56
  Yeah.
• 1:55:57
  We've been working like really hard trying to get all of this set up for this season.
• 1:56:01
  So maybe let's do this.
• 1:56:05
  There's structure all around us.
• 1:56:07
  Yeah.
• 1:56:08
  And a pun on that or something like that.
• 1:56:10
  Because last time...
• 1:56:12
  There were some comedians.
• 1:56:13
  Some of y'all were hilarious.
• 1:56:15
  We still have to put a compilation for that feedback.
• 1:56:19
  But there's structure all around us or something like that.
• 1:56:21
  Or something along those lines.
• 1:56:23
  The last thing I'll say is...
• 1:56:26
  We are hoping to, in short order, put a submissions page on that new website that I mentioned.
• 1:56:32
  So if you have a peer-reviewed research paper in a journal that has an impact factor that is greater than we will tell you soon, but it's going to be pretty high just because we have to kind of keep things there.
• 1:56:45
  Please send it to our submission page, or if there's just something you're interested in, send it there too.
• 1:56:49
  If it is not in a journal, please.
• 1:56:51
  that has a significantly high impact factor.
• 1:56:54
  We will not cover it.
• 1:56:55
  It is not personal.
• 1:56:57
  It may be amazing.
• 1:56:58
  And if it is amazing, it will ultimately end up in one of those journals and we will ultimately cover it.
• 1:57:03
  We are just the two of us.
• 1:57:04
  We're trying to do what we can.
• 1:57:05
  But we know so many of you send us stuff.
• 1:57:07
  We're trying to start the process of creating an ecosystem and community to begin to do so.
• 1:57:12
  So...
• 1:57:14
  A lot of fun, exciting things.
• 1:57:16
  Go ahead.
• 1:57:16
  Yeah, and I was just going to say, like, the one thing that I was going to say with the impact factor stuff is, like, we cover papers from even, like, lower impact factor.
• 1:57:28
  To me, peer-reviewed journals.
• 1:57:30
  Okay?
• 1:57:31
  It's got to be peer-reviewed.
• 1:57:34
  We need reviewer number two.
• 1:57:35
  Yeah, yeah.
• 1:57:37
  Archive doesn't work, guys.
• 1:57:38
  Yeah.
• 1:57:39
  Sorry.
• 1:57:39
  It's not today.
• 1:57:41
  Yeah.
• 1:57:42
  But, hey, if you support the pod and help us grow...
• 1:57:44
  Maybe one day.
• 1:57:45
  Yeah.
• 1:57:46
  We'll expand.
• 1:57:47
  We'll have a big board behind us where we can get the infographics from the infographic team.
• 1:57:52
  And we'll have the 3D animations like MKBHC.
• 1:57:55
  We'll be scaled down to the nanoscale.
• 1:57:59
  Also, we have a blank wall behind us now because of the new angle.
• 1:58:02
  If you have anything you would like to put on the wall...
• 1:58:04
  please DM us.
• 1:58:06
  As always, I am your host, Lester Nare, joined by the one, the only, our resident PhD, Krishna Chowdhury.
• 1:58:14
  We really appreciate you guys joining us and coming back.
• 1:58:17
  And if you're a new fan who heard about us on Rogan, you're in for a good treat.
• 1:58:23
  This is From First Principles.