Latest / What's Your Problem? / What Claude Shannon Figured Out

Transcript

What Claude Shannon Figured Out · What's Your Problem?

• 0:15
  Speaker 1: Pushkin.
• 0:20
  Speaker 2: Hey, Happy New Year. We're very happy to be back,
• 0:23
  Speaker 2: and i have one request before we start the show.
• 0:26
  Speaker 2: I'm asking you a favor, and the favor is this,
• 0:30
  Speaker 2: would you please send us an email to problem at
• 0:35
  Speaker 2: pushkin dot fm and tell us what you like about
• 0:39
  Speaker 2: the show and what you don't like about the show,
• 0:41
  Speaker 2: and specifically what kinds of things you want to hear
• 0:44
  Speaker 2: more of, and perhaps what kinds of things you don't
• 0:47
  Speaker 2: want to hear Again, it's problem at Pushkin dot fm.
• 0:51
  Speaker 2: I'm going to read all the emails, so thank you
• 0:54
  Speaker 2: in advance for sending them. Claude Shannon is this huge
• 1:00
  Speaker 2: figure in the history of technology. He's one of the
• 1:02
  Speaker 2: key people who worked at Bell Labs in the middle
• 1:05
  Speaker 2: of the twentieth century and really came up with the
• 1:08
  Speaker 2: idea that made modern technology possible. But I'm going to
• 1:13
  Speaker 2: be honest with you, I never really understood what Claude
• 1:18
  Speaker 2: Shannon figured out that was such a big deal. But
• 1:21
  Speaker 2: the people who know about technology, who know about the
• 1:23
  Speaker 2: history of ideas, they say Shannon's a giant. Claude Shannon
• 1:28
  Speaker 2: is like the nerds. Nerd he's the techno intellectuals, techno
• 1:32
  Speaker 2: intellectual and so For today's show, I wanted to understand
• 1:37
  Speaker 2: what did Claude Shannon figure out and why is it
• 1:40
  Speaker 2: so important.
• 1:41
  Speaker 1: For the modern world.
• 1:48
  Speaker 2: I'm Jacob Goldstein, and this is What's your problem. My
• 1:51
  Speaker 2: guest today is David Shay. David is a professor of
• 1:55
  Speaker 2: electrical engineering at Stanford. He has studied Shannon for decades.
• 2:00
  Speaker 2: He teaches Shannon's work to his students, and David used
• 2:05
  Speaker 2: Shannon's work to make a breakthrough in cell phone technology.
• 2:09
  Speaker 2: And that breakthrough, that breakthrough that came to us via
• 2:12
  Speaker 2: Shannon and Shay, it affects every phone call we make.
• 2:17
  Speaker 2: David and I talked about Shannon's key insights and about
• 2:20
  Speaker 2: how David's own work built on Shannon, and we also
• 2:23
  Speaker 2: talked about the big chunk of Shannon's life that was
• 2:26
  Speaker 2: taken up with juggling and riding unicycles and building mechanical toys.
• 2:31
  Speaker 2: But to start, we talked about how in the middle
• 2:34
  Speaker 2: of the twentieth century, Bell Labs wound up driving so
• 2:37
  Speaker 2: much technological innovation.
• 2:42
  Speaker 1: Yeah, so Bell Labs was the research lab of AT
• 2:47
  Speaker 1: and T. Aightenh at that time was the phone company. Okay,
• 2:52
  Speaker 1: nowadays we have many phone companies. We have Verizon, we
• 2:55
  Speaker 1: have T Mobile, et cetera. But those days there was
• 3:00
  Speaker 1: only one phone company, and that's a monopoly. So a
• 3:04
  Speaker 1: monopoly needs to justify its existence.
• 3:07
  Speaker 2: Huh. So it doesn't get broken by the government.
• 3:10
  Speaker 1: It doesn't get broken up. Of course, it eventually got
• 3:12
  Speaker 1: broken up, but at that time it was a monopoly.
• 3:16
  Speaker 1: And so one way of justifying its existence is to
• 3:21
  Speaker 1: say that. Okay, he says to the American people, to
• 3:24
  Speaker 1: the government, that we will always spend a certain percent
• 3:29
  Speaker 1: of our revenue on this research lab called bow Labs,
• 3:35
  Speaker 1: and whatever bow Labs come up with is kind of
• 3:38
  Speaker 1: our contribution, know only to our bottom line, but also
• 3:43
  Speaker 1: to technology of the country.
• 3:47
  Speaker 2: So they have this sort of public mission to prevent
• 3:49
  Speaker 2: the government from breaking them up.
• 3:52
  Speaker 1: Yeah, and so therefore it also allows researchers a very
• 3:56
  Speaker 1: free reign to do research that not necessarily tied to,
• 4:02
  Speaker 1: like say, a particular business unit. Okay, So they can
• 4:06
  Speaker 1: be very creative. And that's the atmosphere about so bad
• 4:10
  Speaker 1: Labs attracted a bunch of very smart people because smart
• 4:14
  Speaker 1: people wants to work on their own problem, not the
• 4:16
  Speaker 1: problem that the manager gives them. Yeah, okay, that's the
• 4:20
  Speaker 1: that's one characteristic of smart people. And so yeah, that
• 4:23
  Speaker 1: was the heydays of Bat Labs. Lots of smart people
• 4:27
  Speaker 1: inventing amazing stuff. Laser was invented there, information theory, the
• 4:32
  Speaker 1: transistor was invented there. Sort of almost all the foundation
• 4:37
  Speaker 1: of the information age. Yeah, where there's hardware, algorithm, software
• 4:44
  Speaker 1: is in some sense all have the roots at Ball Labs.
• 4:47
  Speaker 1: So that was the contribution to mankind. Actually, I should say, no,
• 4:50
  Speaker 1: only to America.
• 4:52
  Speaker 2: So Shannon gets there at this time, right, he's there
• 4:56
  Speaker 2: with you know, when they're inventing certainly the transistor.
• 5:01
  Speaker 1: What's he do?
• 5:02
  Speaker 2: Tell me about his his work there? When he gets there,
• 5:05
  Speaker 2: what's he working on?
• 5:07
  Speaker 1: Yeah? So I think Shannon always have his own agenda, right.
• 5:14
  Speaker 1: We know for a fact that he has been interested
• 5:17
  Speaker 1: in the problem of communication, that idea of having a
• 5:22
  Speaker 1: grand theory of communication, even back in nineteen thirty eight,
• 5:27
  Speaker 1: I think thirty seven thirty eight, because he wrote a
• 5:29
  Speaker 1: letter at that time to a very famous person named
• 5:33
  Speaker 1: Venera Bush. Yeah. Vera Bush is very famous, as he
• 5:37
  Speaker 1: was I think president of MIT or dean of MIT,
• 5:41
  Speaker 1: and then he became sort of a scientific advisor to
• 5:43
  Speaker 1: the president, and so he wrote a letter to Venera
• 5:47
  Speaker 1: Bush in nineteen thirty eight and say, hey, you know what,
• 5:49
  Speaker 1: I'm really interested in this question of how to find
• 5:53
  Speaker 1: one theory that unifies all possible communication systems. There's so
• 5:56
  Speaker 1: many different communications system out there, but I think there's
• 5:58
  Speaker 1: something at the heart of every system, and I'm trying
• 6:01
  Speaker 1: to get to the heart.
• 6:02
  Speaker 2: And like nobody had thought of it in that way, right.
• 6:05
  Speaker 2: It seems like part of his part of what Shannon's
• 6:09
  Speaker 2: such a big deal is like as I understand that
• 6:11
  Speaker 2: people it was like, you know that people understood like
• 6:13
  Speaker 2: they were trying to figure out how to make the
• 6:15
  Speaker 2: phone work better, and they were trying to, you know,
• 6:18
  Speaker 2: make movies be clearer or whatever. But there wasn't this
• 6:20
  Speaker 2: idea that you could abstract it until Shannon came along.
• 6:25
  Speaker 1: And the reason is very simple, actually, because if you
• 6:29
  Speaker 1: have a physical system, then you want to build right
• 6:32
  Speaker 1: what do you see right? You say, hey man the
• 6:35
  Speaker 1: video for example, i'm seeing you right now, I'm not
• 6:37
  Speaker 1: seeing you very clearly, have to say yes.
• 6:40
  Speaker 2: I'm in a closet, a closet.
• 6:42
  Speaker 1: Right Then I would say, how to try to improve
• 6:46
  Speaker 1: the imbage? Maybe I can try to, you know, fix
• 6:48
  Speaker 1: this pixel or do some filtering of your noise. So
• 6:52
  Speaker 1: I'm very tied to the very specific details of the
• 6:56
  Speaker 1: specific problem. Because why I'm the engineer. I need to
• 6:59
  Speaker 1: improve the system, not in ten years, but tomorrow. You know, tomorrow.
• 7:03
  Speaker 2: You don't need a theory of the system. You just
• 7:05
  Speaker 2: want to creer picture.
• 7:07
  Speaker 1: Yeah, yeah, I mean I'm in the weeds, right, I'm
• 7:10
  Speaker 1: in the weeds. And Shannon, because of his training, and
• 7:15
  Speaker 1: also because of the atmosphere of a place like bout Labs,
• 7:18
  Speaker 1: could afford to st step back and just look at
• 7:22
  Speaker 1: the broader forest as opposed to the details of specific trees.
• 7:26
  Speaker 2: So so, okay, Shannon's big idea comes out in this
• 7:30
  Speaker 2: paper he publishes in nineteen forty eight. The paper is
• 7:34
  Speaker 2: called a Mathematical Theory of Communication. It's like his great work.
• 7:38
  Speaker 2: Tell me about that paper.
• 7:40
  Speaker 1: So that paper is actually a very interesting paper. In fact,
• 7:44
  Speaker 1: when I teach information theory, I teach from the paper itself,
• 7:49
  Speaker 1: because I thought it's an amazing way not only of
• 7:53
  Speaker 1: learning information theory, but learning how to write a scientific
• 7:56
  Speaker 1: paper properly. Huh okay. And you know, not everyone does
• 8:01
  Speaker 1: research and information theory, but everybody has to write uh
• 8:04
  Speaker 1: huh okay. Every researcher has the right to express their
• 8:08
  Speaker 1: ideas to the peers and to the audience. So in
• 8:12
  Speaker 1: that paper, very interesting. The first paragraph of the paper. Okay,
• 8:15
  Speaker 1: it's already very interesting because typically when people write a
• 8:19
  Speaker 1: paper nowadays, they tell you, oh, how great my invention is.
• 8:22
  Speaker 1: It's going to change the world. Every paper is going
• 8:24
  Speaker 1: to change the world. But in fact, his first paper
• 8:28
  Speaker 1: paragraph focused on telling you what his paper is not achieving.
• 8:31
  Speaker 1: Ha ha, I mean that's a master. That's the masters, right,
• 8:36
  Speaker 1: I mean, how many papers that you read nowadays tells
• 8:39
  Speaker 1: you in the beginning, Hey, you know what, guys, expectation
• 8:43
  Speaker 1: management here, this paper is not about this. Hey, don't
• 8:47
  Speaker 1: get your home. Yeah, exactly, That's exactly what he did.
• 8:50
  Speaker 1: Expectation management nowadays, that's what today we will call it
• 8:55
  Speaker 1: expectation management. And now those days, I guess he just
• 8:58
  Speaker 1: calls it honesty. And his whole point was often people
• 9:03
  Speaker 1: associate information with meaning, okay, and then he said in
• 9:08
  Speaker 1: this we ignore meaning, we ignore meaning. Huh okay. So
• 9:15
  Speaker 1: that was the first thing he did, which is brilliant
• 9:18
  Speaker 1: because once you high information with meaning, then he will
• 9:22
  Speaker 1: never be able to make any progress. It's just too
• 9:24
  Speaker 1: difficult and too broad and too vague a problem.
• 9:27
  Speaker 2: Everybody gets stuck on this idea of meaning and what
• 9:30
  Speaker 2: is meaning? And he's like, forget about meaning. So we're
• 9:33
  Speaker 2: gonna forget about meaning. What is left?
• 9:37
  Speaker 1: Yes, Actually, the biggest I think breakthrough of that paper
• 9:41
  Speaker 1: is to really focus on the thing that matters and
• 9:46
  Speaker 1: cut away a lot of stuff that really doesn't learn
• 9:50
  Speaker 1: that it doesn't matter, but it doesn't matter in terms
• 9:53
  Speaker 1: of solving the communication problem, the communication. So then he said, okay,
• 9:58
  Speaker 1: what is the communication problem? The communication problem is the
• 10:02
  Speaker 1: following is that there are multiple possibilities of a word,
• 10:07
  Speaker 1: and my goal is to tell the receiver destination which
• 10:11
  Speaker 1: of the multiple pospiity is the correct prosperity.
• 10:15
  Speaker 2: Yeah, and so in language, it's basically it's a finite set.
• 10:19
  Speaker 2: Language is a finite set. It's very large. But if
• 10:21
  Speaker 2: we're speaking and we both know that we're speaking English,
• 10:25
  Speaker 2: then essentially you are hearing the words and decoding them,
• 10:28
  Speaker 2: and you know that it is a series of words,
• 10:30
  Speaker 2: and you just have to figure out which words I
• 10:33
  Speaker 2: mean like that for example, Yes, like that? Okay, so
• 10:37
  Speaker 2: that's the frame he builds then.
• 10:39
  Speaker 1: Why okay, all right, Then once you have this framing, right,
• 10:46
  Speaker 1: then you can ask the question, Okay, what is the
• 10:49
  Speaker 1: goal of communication? The goal of communication is to communicate
• 10:53
  Speaker 1: as fast as I can, right, And the natural question
• 10:59
  Speaker 1: is why is there a limit on how fast I
• 11:04
  Speaker 1: can communicate to you? Because if there's no limit, then
• 11:08
  Speaker 1: amazing world. Right, we can communicate so fast it's.
• 11:11
  Speaker 2: Like instant telepathy. It's like you instantly beat me every
• 11:15
  Speaker 2: thought in your head.
• 11:16
  Speaker 1: Yeah, okay, exactly. The natural question it has once you
• 11:18
  Speaker 1: set up this finite set, as you mentioned, is okay,
• 11:22
  Speaker 1: given these finite sets, is there a limit on how
• 11:25
  Speaker 1: fast I can communicate to you? And so that was
• 11:29
  Speaker 1: the question that was the heart of the paper, which
• 11:31
  Speaker 1: is to so he formulated this notion of a capacity.
• 11:37
  Speaker 1: That communication system is like a pipe. It's like you're
• 11:43
  Speaker 1: pushing water through this pipe, and the size of the
• 11:46
  Speaker 1: pipe limits of how fast you can push water through it.
• 11:50
  Speaker 1: And now, justly in communication, there's this notion of a
• 11:54
  Speaker 1: size of the pipe, which is called a capacity. And
• 11:57
  Speaker 1: you figured a way of computing this capacity for different
• 12:02
  Speaker 1: communication medium, any communication medium, you can actually compute a
• 12:07
  Speaker 1: capacity for that community, and that limits how fast you
• 12:11
  Speaker 1: can communicate information over that medium, whether that medium is wireless,
• 12:16
  Speaker 1: over the air or over the widline. Like I'm talking
• 12:19
  Speaker 1: to you, I communicate over the air, I talk to
• 12:22
  Speaker 1: my WiFi. The wi Fi goes through some copper cabo,
• 12:25
  Speaker 1: some optical fiber. H He's a physical medium, but he
• 12:29
  Speaker 1: can compute a capacity for each of these different mediums.
• 12:34
  Speaker 2: And I know that part of the paper looks at, say,
• 12:44
  Speaker 2: redundancy in various modes of communication and on related note patterns. Right,
• 12:52
  Speaker 2: there's this whole section of the paper where he looks
• 12:54
  Speaker 2: at the frequency with which letters occur in English and
• 12:59
  Speaker 2: kind of builds builds an idea around that. Tell me
• 13:02
  Speaker 2: about those pieces.
• 13:03
  Speaker 1: Of the paper. Yeah, so let's talk with the word redundancy.
• 13:07
  Speaker 2: Yeah, that comes off right.
• 13:09
  Speaker 1: No, no, no, no, no, no no, that's not only
• 13:11
  Speaker 1: not the wrong word, but it's actually the most important word.
• 13:14
  Speaker 1: I would say almost because you go back to the
• 13:20
  Speaker 1: question to the thing I was talking about, which is
• 13:22
  Speaker 1: how fast you can communicate? Right, So what he discovered
• 13:26
  Speaker 1: was actually there's no limit on how fast it can communicate.
• 13:28
  Speaker 1: You can always communicate very fast. But what the guy
• 13:31
  Speaker 1: can hear is gibberish, and he cannot really distinguish what
• 13:35
  Speaker 1: you're trying to say is like so much noise in
• 13:37
  Speaker 1: the system, okay, that he cannot really figure out what he's.
• 13:39
  Speaker 2: Saying, even if you're face to face, right, even if
• 13:41
  Speaker 2: you're face to face, you're not going to over the
• 13:43
  Speaker 2: phone or whatever. If you talk too fast, the listener
• 13:46
  Speaker 2: won't understand because you're going to.
• 13:47
  Speaker 1: And anybody who goes to a crazy professor's lecture would
• 13:51
  Speaker 1: know about this, where the professor just keeps on talking
• 13:55
  Speaker 1: and million miles per hour and the students, the sister
• 13:58
  Speaker 1: and nobody understood the thing, and the professor cours the
• 14:01
  Speaker 1: day when it's finished. So so basically he's what he's
• 14:05
  Speaker 1: saying is that, hey, you know what to make sure
• 14:09
  Speaker 1: that the information goes through reliably, reliably, that's the first
• 14:13
  Speaker 1: word you need to introduce, redundancy, redundancy in your message, okay.
• 14:21
  Speaker 1: And what you figured out is in some sense the
• 14:24
  Speaker 1: optimal way of adding redundancy, because you know, you can
• 14:28
  Speaker 1: always be stupid in adding redundancy. For example, I can
• 14:32
  Speaker 1: keep on repeating the same word one hundred times to
• 14:35
  Speaker 1: you and then you probably get it, and then I
• 14:37
  Speaker 1: move on to the next word. I cannot move on
• 14:39
  Speaker 1: the next word, but that would take me one hundred
• 14:41
  Speaker 1: times slow. Yes, right, and so that's not a very
• 14:47
  Speaker 1: smart way of adding redundancy. So what do you figured
• 14:49
  Speaker 1: out is an optimal way of edding redundancy so that
• 14:52
  Speaker 1: you can communicate reliably and yet at the maximum what
• 14:57
  Speaker 1: he calls capacity limit. And that was a totally amazing
• 15:02
  Speaker 1: actually formulation of the problem and highly non obvious. And
• 15:08
  Speaker 1: I think that is some of the amazing contribution of
• 15:12
  Speaker 1: this guy, Shennon.
• 15:12
  Speaker 2: Yeah, it's optimization. He optimizes communication across any channel where
• 15:20
  Speaker 2: you're balancing efficiency or speed and reliability. That is the tradeoff,
• 15:27
  Speaker 2: and he figures out how to optimize for that trade off.
• 15:31
  Speaker 1: Yes, yes, he figured out how to optimize that trade off.
• 15:37
  Speaker 1: But that tradeoff turns out to be very interesting. Uh huh.
• 15:43
  Speaker 1: It's a very interesting tradeoff. So typically when we think
• 15:46
  Speaker 1: about tradeoff, we think about like a smooth curve, right,
• 15:50
  Speaker 1: as when you tune something that you can get better performance.
• 15:54
  Speaker 1: But what he showed was that there's kind of like
• 15:57
  Speaker 1: a cliff effect, Okay, And the cliff effect is that
• 16:02
  Speaker 1: if you communicate below this number called capacity, then you
• 16:08
  Speaker 1: can always engineer system to make your signal the communication
• 16:13
  Speaker 1: as reliable as you want. Huh so reliable, that's completely clean. Wow.
• 16:18
  Speaker 1: Whereas you communicate above this number of capacity, then there's
• 16:23
  Speaker 1: nothing you can do to make a signal clean. It's
• 16:25
  Speaker 1: just completely gibberish. Huh. So it's a very sharp tradeoff
• 16:30
  Speaker 1: that he identified. It's not a smooth tradeoff.
• 16:33
  Speaker 2: And if you're running the phone company, that's exactly what
• 16:35
  Speaker 2: you want to know, right, So then you can tune
• 16:37
  Speaker 2: it all the way to capacity and then not try
• 16:41
  Speaker 2: and tune it anymore after that, because it's not going
• 16:43
  Speaker 2: to get any better.
• 16:44
  Speaker 1: Correct, And that's the goal of sixty years of engineering
• 16:46
  Speaker 1: to achieve his vision, his vision nineteen forty eight. It
• 16:51
  Speaker 1: took people around sixty years to get to that implement
• 16:54
  Speaker 1: his vision.
• 16:56
  Speaker 2: Well, so you are part of that story, right, Let's
• 16:59
  Speaker 2: let's let you walk onto the story now. So you
• 17:03
  Speaker 2: tell me about your work and how Shannon's work. You know,
• 17:07
  Speaker 2: how you built on Shannon's work. Tell me about how
• 17:10
  Speaker 2: you built on Shannon's work.
• 17:13
  Speaker 1: Yeah. So I did my PhD in the nineties. In
• 17:16
  Speaker 1: the nineties. My advisor was a Shannon student, and so
• 17:23
  Speaker 1: I learned information theory him. Okay, Now, at that time,
• 17:27
  Speaker 1: information theory was almost a dead subject. Okay. When I
• 17:31
  Speaker 1: was a PhD student, the first thing my advisor told me,
• 17:35
  Speaker 1: maybe following Shannon, is hey, don't work in information theory. Wow,
• 17:40
  Speaker 1: you'll never find a job. You never find a job
• 17:42
  Speaker 1: with this stuff. Okay, that's a tough moment.
• 17:44
  Speaker 2: That must be a tough moment for.
• 17:46
  Speaker 1: Pretty tough, yeah, because at that time, there's not much
• 17:50
  Speaker 1: progress made in the theory, and there's no killer applications either.
• 17:55
  Speaker 1: There's no very killer applications that need all this sophisticated
• 17:58
  Speaker 1: information theory. Okay. So it's like a dead field.
• 18:02
  Speaker 2: Was there a while when people used it to like
• 18:05
  Speaker 2: whatever make landline phones work better, like in the fifties
• 18:08
  Speaker 2: or something with where people like, oh great, now we've
• 18:10
  Speaker 2: got this theory and we can make the phone work better.
• 18:14
  Speaker 1: Yeah. So the thing is that the solutions that people
• 18:20
  Speaker 1: come up with to achieve these capacity limits is very complicated, okay,
• 18:24
  Speaker 1: and the electronics that technology is just not enough to
• 18:27
  Speaker 1: build these complicated circuits. So information theory have had not
• 18:31
  Speaker 1: a very significant applic impact in the fifties, sixties, or
• 18:35
  Speaker 1: even seven sounds.
• 18:36
  Speaker 2: So it's like one of those cases where the theory
• 18:38
  Speaker 2: is just too far ahead of the technology.
• 18:42
  Speaker 1: To be useful it. Yeah, and so people can start
• 18:45
  Speaker 1: losing interest in the theory is that, yes, this is
• 18:47
  Speaker 1: a bunch of maths. It's not impacting the real world,
• 18:50
  Speaker 1: and so students are drifting away from the field. But
• 18:53
  Speaker 1: there's still always a few students, okay, who are just
• 18:56
  Speaker 1: so enumorated by the theory that they keep on pursuing it.
• 19:01
  Speaker 1: And my advisor is one of the leading professors in
• 19:04
  Speaker 1: this area, and he would have like one student every decade,
• 19:08
  Speaker 1: every decade to do research in if.
• 19:11
  Speaker 2: You were that student, you were and I was.
• 19:14
  Speaker 1: Not that student, And I was not that student, Okay.
• 19:17
  Speaker 1: At that time, that slot was already taken by an
• 19:19
  Speaker 1: earlier student who was ways more than me, who's ways
• 19:22
  Speaker 1: more than me. And that's that he was that he
• 19:25
  Speaker 1: was a student of the decade in information theory. Okay. Now,
• 19:30
  Speaker 1: so I was assigned to work on some other problems okay,
• 19:32
  Speaker 1: completely and related Okay, But anyway, the point though, is
• 19:35
  Speaker 1: that when I graduated, something happened, okay, And that was
• 19:40
  Speaker 1: the beginning of the wireless revolution. That was the time
• 19:45
  Speaker 1: when only a million people have cell phones, and those
• 19:49
  Speaker 1: cell phones I don't even remember. It's like gigantically break yeah.
• 19:53
  Speaker 2: Like there's that famous scene from the movie Wall Street, right,
• 19:56
  Speaker 2: that's the one that everybody talks about where it's like
• 19:58
  Speaker 2: bigger than a brick. People say brick, but it's actually
• 20:01
  Speaker 2: bigger than a brick. It's like a big hardback book
• 20:04
  Speaker 2: or something.
• 20:05
  Speaker 1: Yeah. And actually those days, because there's some few of
• 20:07
  Speaker 1: six post it's like a prestige. It's like it's prestige
• 20:11
  Speaker 1: to have this brick. Yeah, okay, yeah.
• 20:14
  Speaker 2: You couldn't get that brick. You had to be rich
• 20:16
  Speaker 2: to get that brick. Yeah.
• 20:18
  Speaker 1: Yeah. And so the wireless revolution was happening because people
• 20:21
  Speaker 1: realized that hey, you know what, be able to communicate
• 20:24
  Speaker 1: anytime anywhere is really viable, and so people are now
• 20:28
  Speaker 1: getting interested. And at that time, what people realize is
• 20:31
  Speaker 1: that whoa this wireless physical media, it's really tough to
• 20:37
  Speaker 1: communicate over because the bandwidth is so limited and the
• 20:41
  Speaker 1: noise is so much. Right, FCC was limiting the bandwidth
• 20:45
  Speaker 1: allocation to these applications a lot.
• 20:47
  Speaker 2: Aha, and so Communications Commission the government wasn't letting wireless
• 20:52
  Speaker 2: companies use much bandwidth for.
• 20:54
  Speaker 1: Cel phone yeah, because all the bandwi most of them
• 20:56
  Speaker 1: are allocated for military purposes and there's only very little bandwidth
• 20:59
  Speaker 1: allocated at that time for civilians, and so those bandwidth
• 21:03
  Speaker 1: were auctional to companies with a very high price, and
• 21:06
  Speaker 1: so it became very important to be very efficient in
• 21:10
  Speaker 1: using this very expensive property. Aha, Okay, and then people realize, hey,
• 21:16
  Speaker 1: if we want to be really efficient, then we need
• 21:19
  Speaker 1: a theory which is about efficiency. So people start thinking, okay,
• 21:23
  Speaker 1: all right, so information theory was dead, but now it's
• 21:26
  Speaker 1: going to come back to life because we have this
• 21:29
  Speaker 1: really important problem, really expensive spectrum that was allocated by SEC,
• 21:32
  Speaker 1: and we want to squeeze as much of it as possible.
• 21:35
  Speaker 2: As much communication. We need a sort of mathematical theory
• 21:38
  Speaker 2: of communication, if you will.
• 21:41
  Speaker 1: And that was the renaissance of information theory, spurred by
• 21:45
  Speaker 1: this amazing technology of wireless, which took us from one
• 21:49
  Speaker 1: million phones to ten billion phones.
• 21:53
  Speaker 2: Today everybody has one point one.
• 21:55
  Speaker 1: Phone, and information theory play a big role in that revolution.
• 22:06
  Speaker 2: In a minute, how David used quad shannon It's nineteen
• 22:09
  Speaker 2: forty eight paper to come up with an idea that
• 22:12
  Speaker 2: we all use every time we make.
• 22:14
  Speaker 1: A phone call.
• 22:24
  Speaker 2: Let's talk for a moment about your your role, right,
• 22:27
  Speaker 2: like you actually played played an important role there.
• 22:31
  Speaker 1: Yeah, so I was at Ball Labs. Uh huh, just
• 22:34
  Speaker 1: like Claude. So it's like Claude to Yeah. Yeah, So
• 22:37
  Speaker 1: I spent one year at ball Lapse as a so
• 22:40
  Speaker 1: called postdoc right after my PhD, before I moved to
• 22:44
  Speaker 1: Berkeley to become a professor there. I spent one year there.
• 22:48
  Speaker 1: And that's what people were talking about that time of
• 22:49
  Speaker 1: beat Labs. Hey, this new thing, wireless information theories come
• 22:53
  Speaker 1: back to life. We can try to use information theory
• 22:57
  Speaker 1: and adapt it and extend it to this wireless communication problem.
• 23:01
  Speaker 1: And so that's when I said, whoa this information theory
• 23:05
  Speaker 1: I learned from Bob Gallagher. Finally there's a place to
• 23:08
  Speaker 1: use it. Finally I can actually make a living, make
• 23:13
  Speaker 1: a living out of it. And like what my advisor
• 23:15
  Speaker 1: told me, is not dead, it's come back to life. Yeah,
• 23:18
  Speaker 1: and so that's sort of my start in the field.
• 23:22
  Speaker 1: And uh yeah, so I did, I, you know, invented
• 23:25
  Speaker 1: a bunch of stuff and actually apply this connect this
• 23:29
  Speaker 1: information theory to the real world. And uh, every time
• 23:32
  Speaker 1: you use a phone, you're using my algorithm, which is
• 23:36
  Speaker 1: based on the theory of information. Huh.
• 23:39
  Speaker 2: And so you're you're that's a cool thing to be
• 23:42
  Speaker 2: able to say. First of all, that's a very good
• 23:44
  Speaker 2: flex your algorithm, it's the proportional fair scheduling algorithm, right, yes, yes,
• 23:52
  Speaker 2: what is that? What's it do?
• 23:54
  Speaker 1: All right? So I should tell you a little bit story.
• 23:57
  Speaker 1: I think the story is, uh, and then I'll tell
• 23:58
  Speaker 1: you what it does. Okay. So I went to So
• 24:02
  Speaker 1: that was the end of nineteen ninety nine, around nine
• 24:05
  Speaker 1: ninety nine. So I was doing all this information theory
• 24:09
  Speaker 1: stuff at Berkeley, writing many papers. But then I always
• 24:13
  Speaker 1: have a thought the back of my mind, which you say,
• 24:16
  Speaker 1: is this stuff going to be useful? And so I
• 24:18
  Speaker 1: went to a I decided to go to a company,
• 24:20
  Speaker 1: a wireless company who actually build these things and see
• 24:22
  Speaker 1: whether this theory can be used. And the company went
• 24:24
  Speaker 1: to is called Quaker.
• 24:26
  Speaker 2: Okay, I've heard of Quaker.
• 24:28
  Speaker 1: You've heard of Quaker, but at that time it was
• 24:30
  Speaker 1: a small company, it was not very big, Okay. And
• 24:34
  Speaker 1: at that time they have this problem they're working on. Okay,
• 24:37
  Speaker 1: which is the following. All right. So in wireless communication,
• 24:41
  Speaker 1: there's a concept COUD based station okay, and the base
• 24:45
  Speaker 1: station serves many cell phones in the vicinity of the
• 24:49
  Speaker 1: bay station. It's cost sout.
• 24:51
  Speaker 2: Is it like a tower? Is what we would call
• 24:52
  Speaker 2: it a tower?
• 24:54
  Speaker 1: That's right, it's always on the tower. There's there's some
• 24:56
  Speaker 1: electronics there. Yeah, and that's how the bay station is
• 25:00
  Speaker 1: supposed to beam information to many phones, and many phones.
• 25:03
  Speaker 2: You still see them. You see them when whatever on
• 25:05
  Speaker 2: top of a big building or when you're driving down
• 25:07
  Speaker 2: the freeway. Right, that's what you're talking about.
• 25:09
  Speaker 1: That, Yeah, that's right. And sometimes on fake trees.
• 25:12
  Speaker 2: I love the fake trees in New Jersey.
• 25:14
  Speaker 1: They love the fake New Jersey. That's right, New Jersey,
• 25:18
  Speaker 1: fake trees. Yes, So at that time they would look
• 25:22
  Speaker 1: at this problem, which is, hey, okay, my bandwidth is limited,
• 25:27
  Speaker 1: but I have many users to serve. Yeah, okay, how
• 25:30
  Speaker 1: do I schedule my limited resource among all these users? Right?
• 25:34
  Speaker 1: Because I only have one total bandwidth. And so at
• 25:37
  Speaker 1: that time people think, okay, maybe something simple. I give
• 25:41
  Speaker 1: one end of the time to the end user. Right,
• 25:43
  Speaker 1: so the five users, I serve this user for a
• 25:46
  Speaker 1: little bit, and I served the second user for a
• 25:48
  Speaker 1: little bit, and served user for.
• 25:50
  Speaker 2: The ideas, you're switching really fast. You're just like switching.
• 25:53
  Speaker 1: Switching really fast, yeah, exactly. And then when I went there,
• 25:56
  Speaker 1: I said, okay, good is the problem is a good problem.
• 25:59
  Speaker 1: And I said, hey, instead of fixating on this particular
• 26:03
  Speaker 1: scheduling policy, why don't we do any Shannon thing, a.
• 26:08
  Speaker 2: Card Shannon thing. You thought of, your thought of it? Yeah, okay.
• 26:12
  Speaker 1: The clause shaming thing is what is to look at
• 26:15
  Speaker 1: the problem from first principle, Uh not reassume a particular
• 26:21
  Speaker 1: solution or a particular class solution even and ask ourselves
• 26:26
  Speaker 1: what is the capacity of this whole system, and how
• 26:31
  Speaker 1: do I engineer the system to achieve that capacity? Okay?
• 26:35
  Speaker 1: And it turns out that if you look at the
• 26:37
  Speaker 1: problem this way, then it turns out that the optimal
• 26:40
  Speaker 1: way of scheduling is not the one that they will
• 26:43
  Speaker 1: try and design. And the reason is because in wireless
• 26:47
  Speaker 1: communication there's a very interesting characteristic which is called fading. Okay, okay.
• 26:55
  Speaker 1: When I talk to you over the air, the channel
• 26:59
  Speaker 1: actually goes up and down, strong and weak, strong and weak,
• 27:02
  Speaker 1: very rapidly. What I mean is when I send an
• 27:05
  Speaker 1: Electromnett signal from the from the base to the phone,
• 27:10
  Speaker 1: that signal get amplified and attenuate it very rapidly. It
• 27:17
  Speaker 1: goes up and down.
• 27:18
  Speaker 2: Okay, Okay, can we say it gets stronger and weaker.
• 27:21
  Speaker 2: Can we say it gets stronger.
• 27:22
  Speaker 1: And stronger and weaker? Okay? Yes, And so the alcomal
• 27:26
  Speaker 1: way that Information SERVI it has to do is actually
• 27:29
  Speaker 1: not divide the time into slots blindly, but really try
• 27:34
  Speaker 1: to SKATEU a user when the channel is strong.
• 27:40
  Speaker 2: Ha.
• 27:42
  Speaker 1: And then from that on I designed a scheduling album
• 27:45
  Speaker 1: from which is more practical by sort of leverage of
• 27:48
  Speaker 1: this basic idea from information theory.
• 27:51
  Speaker 2: And so so the base station is basically monitoring the
• 27:55
  Speaker 2: strength of the incoming signals from all the different phones
• 27:59
  Speaker 2: and saying, Oh, that one's strong, I'm gonna grab that one.
• 28:02
  Speaker 2: Oh that one's strong, I'm gonna grab that one.
• 28:03
  Speaker 1: That's what's happening, correct? Correct?
• 28:05
  Speaker 2: And how does that I mean I get in a
• 28:07
  Speaker 2: kind of big first principles way sort of analogously, it
• 28:10
  Speaker 2: follows from Shannon. But is there anything sort of specific
• 28:14
  Speaker 2: in Shannon that leads you to this algorithm?
• 28:20
  Speaker 1: So remember Shannon is a very general theory. Yeah, Okay.
• 28:27
  Speaker 1: It basically says that given any communication medium or any
• 28:31
  Speaker 1: communication setting, yeah, you can try to calculate this notion
• 28:37
  Speaker 1: of a capacity. So the very general theory, what I
• 28:41
  Speaker 1: did was to apply it to a very specific context,
• 28:44
  Speaker 1: which is this base station serving multiple user setting. Yeah,
• 28:50
  Speaker 1: and then apply his framework to analyze the capacity of
• 28:54
  Speaker 1: that system.
• 28:55
  Speaker 2: Huh.
• 28:57
  Speaker 1: And in the process of analyzing the capacity, you can
• 28:59
  Speaker 1: also figure out what is the optimal way of achieving
• 29:04
  Speaker 1: that capacity. Remember you mentioned capacity is really an optimization problem,
• 29:09
  Speaker 1: and Shannon was able to solve this optimization problem in general.
• 29:13
  Speaker 1: But now I specialize it in some sense to this
• 29:16
  Speaker 1: pretty specific setting, except that the setting is used by everybody.
• 29:21
  Speaker 1: But at that time, it was like, you know, research
• 29:24
  Speaker 1: is about timing, and I was there at the right
• 29:27
  Speaker 1: place at the right time. Because Qualcom turns out to
• 29:32
  Speaker 1: completely dominate the entire third generation technology. So when I
• 29:37
  Speaker 1: was able to convince them that, hey, your way of
• 29:40
  Speaker 1: doing things is no good. This way suggested by Shannon's
• 29:44
  Speaker 1: actually far better. Please use this way. It took me
• 29:48
  Speaker 1: a few months, but I was able to persuade them
• 29:50
  Speaker 1: to implement it, and then it got into the standard
• 29:53
  Speaker 1: through the domination, and then every standard after that uses
• 29:58
  Speaker 1: the same basic, the same algorithm. So it was good because,
• 30:02
  Speaker 1: as I said, I'm at the right place at the
• 30:04
  Speaker 1: right time. You know, when you try to contribute to engineering.
• 30:07
  Speaker 1: It's too late if the system is built, because people
• 30:09
  Speaker 1: don't want to wreck change the whole system to accompany
• 30:12
  Speaker 1: or new idea. But it was very early in the
• 30:15
  Speaker 1: design phase.
• 30:17
  Speaker 2: So so okay, So you made this breakthrough in wireless
• 30:21
  Speaker 2: communications using Shannon's work. Were there similar breakthroughs in, you know,
• 30:26
  Speaker 2: in other domains?
• 30:29
  Speaker 1: Any communication median right, it could be optical, fiber, it
• 30:33
  Speaker 1: could be DSL modem, YESL modem, underwater communication. Almost all
• 30:40
  Speaker 1: these communication systems are now designed based on his principle.
• 30:45
  Speaker 1: So as impact of this theory is kind of global,
• 30:50
  Speaker 1: it's the entire communication landscape.
• 30:54
  Speaker 2: There's a story I read in about Shannon that when
• 31:00
  Speaker 2: he is developing information theory, he he takes a book
• 31:05
  Speaker 2: off the shelf and he reads a sentence to its
• 31:07
  Speaker 2: actually his wife, and it's something like the lamp was
• 31:10
  Speaker 2: sitting on the and she says table and he says, no,
• 31:14
  Speaker 2: I'll give you a clue. The first letter is D
• 31:16
  Speaker 2: and she says desk. And when I heard that story,
• 31:21
  Speaker 2: what I thought of was large language models, Like that
• 31:24
  Speaker 2: sounds exactly like a large language model, And so I'm
• 31:28
  Speaker 2: just fishing, I'm just curious, like does his work matter
• 31:34
  Speaker 2: for machine learning, large language models, et cetera or no.
• 31:40
  Speaker 1: Yeah, so that's a very interesting point. Now I'm not
• 31:44
  Speaker 1: an expert by any means in AI or large language models. Yeah,
• 31:48
  Speaker 1: I'm not a professional researcher in that area. But I
• 31:52
  Speaker 1: think you can actually see some commonality, right, is that
• 31:56
  Speaker 1: you know these models in some sense, they don't care
• 31:58
  Speaker 1: about meaning either.
• 32:00
  Speaker 2: Yeah, very good, Very good. Yeah.
• 32:03
  Speaker 1: Right, Actually I've just came to my discussion is very
• 32:08
  Speaker 1: interesting because it's really just patterns. It's just which patterns
• 32:13
  Speaker 1: are more likely than other patterns. Right. The example you
• 32:16
  Speaker 1: gave about desk and is basically about patterns, and information
• 32:20
  Speaker 1: theory is really analyzing sort of the number of possible
• 32:23
  Speaker 1: patterns in some sense. So there is definitely a philosophical connection,
• 32:30
  Speaker 1: I believe, starting from Shannon to these large language models.
• 32:35
  Speaker 2: So let me ask you about one other, and this
• 32:36
  Speaker 2: is one that you are professionally involved in cryptocurrency and blockchain.
• 32:44
  Speaker 2: You have studied it and you started a company.
• 32:48
  Speaker 1: Right.
• 32:49
  Speaker 2: Is there a connection between Shannon's work and cryptocurrency.
• 32:53
  Speaker 1: Yeah, So what attracts me to work in this area blockchain,
• 32:59
  Speaker 1: is that blockchain actually has one very common philosophical connection
• 33:05
  Speaker 1: to information theory, which is a following in blockchain. The
• 33:09
  Speaker 1: problem is not communication per se. It's called consensus. Okay,
• 33:14
  Speaker 1: it's a different problem, but it's essentially allow a bunch
• 33:18
  Speaker 1: of users at different places to come to an agreement
• 33:22
  Speaker 1: on something. Okay, yes, Now, the goal of designing blockchain
• 33:28
  Speaker 1: is really to be so called for tolerant, tolerant, which
• 33:33
  Speaker 1: means for torerant, which means that even if say one
• 33:37
  Speaker 1: third of the users are bad guys and send you
• 33:40
  Speaker 1: some gibberish message, you can still the rest two third
• 33:46
  Speaker 1: people can still come to an agreement. Okay, all right,
• 33:51
  Speaker 1: So you look at this problem, it's actually not that
• 33:53
  Speaker 1: different from communication information theory because it's kind of combating.
• 33:58
  Speaker 2: The bad guys are the noise that the good guys
• 34:00
  Speaker 2: at the signal.
• 34:01
  Speaker 1: And the good guys at the signal, and they try
• 34:03
  Speaker 1: to reintroduce redundancy, okay, to help them to fight against
• 34:08
  Speaker 1: these bad guys.
• 34:09
  Speaker 2: And there's an optimization problem where like the more redundancy
• 34:12
  Speaker 2: you have, the sort of slower the system is, the
• 34:14
  Speaker 2: more ponderous.
• 34:16
  Speaker 1: And so you tried an optimization problem is to try
• 34:19
  Speaker 1: to figure out what is the optimal number of bad
• 34:22
  Speaker 1: guys that you can tolerate and your system still works.
• 34:26
  Speaker 1: That is the analogous to the capacity problem. So I
• 34:29
  Speaker 1: find the philosophical connection very appealing, and so that's sort
• 34:35
  Speaker 1: of one reason why I got attracted to work into
• 34:37
  Speaker 1: this area.
• 34:39
  Speaker 2: Why do you think more people don't know about Shannon?
• 34:42
  Speaker 2: Like all of the sort of intellectuals in technology say,
• 34:48
  Speaker 2: he's like one of the great thinkers of the twentieth century,
• 34:54
  Speaker 2: but most people have never heard of him. Why do
• 34:57
  Speaker 2: you think that is?
• 34:59
  Speaker 1: So? Shannon was actually a very shy person, very shy person.
• 35:06
  Speaker 1: He hates publicity, He hated when people interview him. You remember, right,
• 35:13
  Speaker 1: it's basically a very modest person. Remember the first paragraph
• 35:16
  Speaker 1: I talked you about. Yeah, he tells you what he
• 35:19
  Speaker 1: is not accomplishing. Yeah, And so he's a very modest,
• 35:22
  Speaker 1: very shy person, not into publicity. And I think that
• 35:26
  Speaker 1: sort of impact not only himself, but also everybody who
• 35:32
  Speaker 1: works in that field. Uhha, and dob this as kind
• 35:36
  Speaker 1: of like a metric, right that Hey, we should all
• 35:38
  Speaker 1: be modest, because what look at this guy who accomplished
• 35:41
  Speaker 1: so much and he's still so motist. Who are we?
• 35:44
  Speaker 1: Who are we? Right? So, as a result, the field
• 35:48
  Speaker 1: doesn't really sell himself very well. The marketing engine, the
• 35:53
  Speaker 1: marketing DNA is not there. Yeah, and so people don't
• 35:56
  Speaker 1: know about him.
• 35:58
  Speaker 2: So I want to talk for a minute about the
• 36:00
  Speaker 2: rest of Shannon's life. He writes this huge paper when
• 36:03
  Speaker 2: he's in his early thirties, eventually goes on to be
• 36:07
  Speaker 2: a professor at MIT, and he seems to spend a
• 36:11
  Speaker 2: lot of his career juggling, writing a unicycle, building mechanical toys,
• 36:17
  Speaker 2: building games, and he never, you know, does sort of
• 36:21
  Speaker 2: great influential work again, and I'm curious, you know, what
• 36:25
  Speaker 2: do you what do you make of that? How do
• 36:27
  Speaker 2: you sort of fit his whole career together?
• 36:31
  Speaker 1: So there's a single there's a theme that unifies all
• 36:34
  Speaker 1: this in my mind, which is playfulness. Because in his mind,
• 36:39
  Speaker 1: research is really about puzzles. Uh, he doesn't understand something.
• 36:46
  Speaker 1: It's like a puzzle to him, and he's trying to
• 36:49
  Speaker 1: figure out the pieces of the puzzle. Information theory was
• 36:53
  Speaker 1: like that the puzzles. He sees all these real war systems,
• 36:56
  Speaker 1: they seem to all share some community, but nobody understood it.
• 36:59
  Speaker 1: So there's a puzzle, and it's always thinking about the puzzle.
• 37:02
  Speaker 1: And finally his paper basically solve that puzzle. So everything
• 37:06
  Speaker 1: to him is playfulness. I think it's playing. There's a
• 37:09
  Speaker 1: game puzzle and needs a soft to puzzle, and that
• 37:13
  Speaker 1: is mine. That's how it's my work. So although it
• 37:15
  Speaker 1: seems very different things that he did pre and post
• 37:20
  Speaker 1: inflamation theory, but it's actually in my mind quite strongly monouns.
• 37:29
  Speaker 2: We'll be back in a minute with the lightning round.
• 37:40
  Speaker 2: So I read that you recently asked people at your
• 37:45
  Speaker 2: company to give five minute talks. I'm curious why you
• 37:50
  Speaker 2: did that. That's interesting to me. Why'd you do that?
• 37:53
  Speaker 1: So just short to talk the harderest to give. So
• 38:00
  Speaker 1: you can't explain an idea in five minutes, then I
• 38:04
  Speaker 1: think your idea is actually not very good.
• 38:06
  Speaker 2: Ah, that's good.
• 38:09
  Speaker 1: Most good ideas you can get the point to across
• 38:12
  Speaker 1: in five minutes. Remember, I'm an information theorist by training,
• 38:16
  Speaker 1: so communication to the limit is what I'm passionate about.
• 38:21
  Speaker 2: If you had to give a five minute talk, what
• 38:25
  Speaker 2: would it be about?
• 38:27
  Speaker 1: How about Shutton? I guess he's my hero. He's my hero.
• 38:34
  Speaker 2: So one you talked about the importance of timing in
• 38:40
  Speaker 2: research of not only finding the right problem, but finding
• 38:44
  Speaker 2: the right problem at the right time. Right, both in
• 38:46
  Speaker 2: terms of Shannon's work and in terms of your work.
• 38:51
  Speaker 2: You know, you're also a professor and you know a manager, Like,
• 38:55
  Speaker 2: how do you help other people find the right problem
• 38:58
  Speaker 2: at the right time?
• 39:01
  Speaker 1: Yeah, finding the right problem at the right time is
• 39:04
  Speaker 1: probably the most difficult because you know, time is everything. However,
• 39:12
  Speaker 1: this is hard to teach. What we try to do
• 39:16
  Speaker 1: is to be ready. So one very famous information theorist
• 39:21
  Speaker 1: told me this. He said, you know, everybody will get
• 39:25
  Speaker 1: lucky at some point in time in the career. However,
• 39:30
  Speaker 1: most people, when they get lucky, they're not ready, so
• 39:34
  Speaker 1: they don't realize that they get lucky, and so they
• 39:37
  Speaker 1: missed the opportunity. They went a different direction. Luck tells
• 39:40
  Speaker 1: you should go this way, but you went the other way.
• 39:42
  Speaker 1: Lost it.
• 39:43
  Speaker 2: That makes me so scared.
• 39:45
  Speaker 1: And so what I teach my students is always be ready.
• 39:49
  Speaker 1: It's like your muscles. You have to be always training
• 39:52
  Speaker 1: your muscles so that when you are lucky, you can
• 39:54
  Speaker 1: capitalize on the lucky.
• 39:58
  Speaker 2: Do you So you talked about Shannon's playful nature like
• 40:02
  Speaker 2: he was a juggler. He wrote a unicycle You do
• 40:05
  Speaker 2: anything like that? You have any weird hobbies.
• 40:10
  Speaker 1: No, No, the only weird hobby is I love to
• 40:14
  Speaker 1: talk to people like.
• 40:15
  Speaker 2: You fair, you love going on podcasts. That that is
• 40:19
  Speaker 2: the juggling of the twenty first century. Who's your second
• 40:24
  Speaker 2: favorite underrated thinker?
• 40:27
  Speaker 1: My advisor Ah Gallagher? Well gallaghery. He taught me how
• 40:33
  Speaker 1: to think about research because you learned from Shannon and
• 40:38
  Speaker 1: I learned from him.
• 40:40
  Speaker 2: And if you boil down what your advisor learned from
• 40:44
  Speaker 2: Shannon and what you learned from your advisor, what would
• 40:47
  Speaker 2: it be? What did you learn?
• 40:49
  Speaker 1: Yeah, and learn about taking a very complicated problem and
• 40:54
  Speaker 1: strip it down to the essential and then formulate a
• 40:59
  Speaker 1: problem around that and solve it. That's an art. It's
• 41:04
  Speaker 1: not something you can to convert it into a mathematical
• 41:07
  Speaker 1: formula and teach students. It's just based on intuition, experience.
• 41:13
  Speaker 1: And that's what Shannon talked my advisor, and that's what
• 41:17
  Speaker 1: my advisor taught me, and that's what I try to
• 41:20
  Speaker 1: teach my students. Really, teaching is not really about giving
• 41:24
  Speaker 1: the follower is really just learning by examples. I observe
• 41:27
  Speaker 1: what he does, and then my students observe what I
• 41:31
  Speaker 1: do as I interact with them. And hopefully this art
• 41:34
  Speaker 1: will carry on from generation to generation.
• 41:38
  Speaker 2: Finding the essence of the problem.
• 41:41
  Speaker 1: Yeah, David.
• 41:50
  Speaker 2: She is a professor at Stanford. Today's show was produced
• 41:54
  Speaker 2: by Gabriel Hunter Chang. It was edited by Lyddy Jean
• 41:58
  Speaker 2: Kott and engineered by Sarah Bruguier. You can email us
• 42:01
  Speaker 2: at problem at Pushkin dot FM. I'm Jacob Goldstein and
• 42:05
  Speaker 2: we'll be back next week with another episode of What's
• 42:08
  Speaker 2: Your Problem.