Latest / Broken Record with Rick Rubin, Malcolm Gladwell, Bruce Headlam and Justin Richmond / Broken Record Presents: A Musical Episode of Solvable

Transcript

Broken Record Presents: A Musical Episode of Solvable · Broken Record with Rick Rubin, Malcolm Gladwell, Bruce Headlam and Justin Richmond

0:15
Speaker 1: Pushkin. Hey, y'all, this is justin Richmond. I hope you're
0:22
Speaker 1: enjoying a nice and safe holiday season. We're looking forward
0:25
Speaker 1: to coming back to you with new episodes of Broken
0:27
Speaker 1: Record in twenty twenty one, starting on January twelve. But
0:31
Speaker 1: in the meantime, we wanted to gift you an interview
0:33
Speaker 1: Malcolm Gladwell did for another podcast called Solvable. Solvable is
0:38
Speaker 1: hosted by Pushkin CEO Jacob Weisberg and occasionally Malcolm as well.
0:43
Speaker 1: In this episode, Malcolm speaks with a software engineer who
0:46
Speaker 1: solved awkward video conference calls by reducing the delay time
0:50
Speaker 1: that occurs enough that musicians can now jam over Zoom
0:53
Speaker 1: in real time. Enjoy this episode. Here's Malcolm Gladwell. Was Solvable.
0:59
Speaker 1: This is Solvable. I'm Malcolm Gladwell. Occasionally we would get
1:03
Speaker 1: to go on break out rooms on Zoom and just
1:06
Speaker 1: discuss our feelings about what's happening at the time, and
1:11
Speaker 1: when the lockdown began, we were all devastated because singing
1:15
Speaker 1: was our life. Sam is twelve years old and like
1:18
Speaker 1: many kids, he's struggling with the restrictions COVID nineteen has
1:21
Speaker 1: imposed on his life. There's a lot of talking to
1:24
Speaker 1: news about the impact of COVID on classroom education. But
1:28
Speaker 1: for Sam and his friends, what they're missing is after school.
1:33
Speaker 1: Ragotsi was like a sanctuary to go to at the
1:36
Speaker 1: end of the school day. Rogatsi is a boy's choir
1:39
Speaker 1: based in California. Sam's been singing in the group for
1:42
Speaker 1: seven years. In pre COVID times, he would rehearse twice
1:46
Speaker 1: a week for several hours each day, but all of
1:50
Speaker 1: that came to a halt when COVID arrived in the US.
1:53
Speaker 1: The choir was basically stuck. If you've ever tried to
1:57
Speaker 1: sing something like Happy Birthday during a zoom meeting, you
2:00
Speaker 1: know what I'm talking about. Making music sync up with
2:03
Speaker 1: other people in real time over the Internet is really tough.
2:08
Speaker 1: The connect he delays, what they call latency, can be
2:12
Speaker 1: pretty awkward. Give us the technical explanation of the problem
2:16
Speaker 1: your son was facing. It's rooted in latency. Fundamentally, the
2:21
Speaker 1: time that it takes the sound you make to reach
2:24
Speaker 1: another musician that you're performing with. You need to have
2:28
Speaker 1: a latency of about less than twenty five milliseconds. Otherwise,
2:32
Speaker 1: the asense of the pulse or the beat of the
2:35
Speaker 1: music is just lost across those musicians. Mike Dickie is
2:39
Speaker 1: a software engineer, and after months of seeing his son
2:42
Speaker 1: Sam bummed out about not singing, he realized he could
2:46
Speaker 1: do something about it now for far less than the
2:49
Speaker 1: cost of a year of private music lessons. Dickie created
2:52
Speaker 1: a tool that allows groups of musicians to make music online.
2:57
Speaker 1: And it might be just the beginning. So we could
3:00
Speaker 1: have real time cheering and fan reaction in a basketball game. Yeah, absolutely,
3:06
Speaker 1: Performing in rehearsing with others in real time is really
3:09
Speaker 1: difficult or even impossible for many people. But this problem
3:13
Speaker 1: is solvable. I talked with Mike Dickie about Virtual Studio.
3:21
Speaker 1: It's a low cost device that works with the open
3:24
Speaker 1: source tool called jack Trip to make remote music collaboration
3:29
Speaker 1: possible in real time. Here is our conversation. Mike tell
3:34
Speaker 1: me about the first time this that Rugazzi rehearse using
3:38
Speaker 1: virtual Studio. What was that like? At first? It was
3:40
Speaker 1: a bit terrifying progress out. Let's progress down, okay and
3:47
Speaker 1: keep go. I thought of all the things that could
3:53
Speaker 1: go wrong, and I knew that this was all very
3:56
Speaker 1: new and untested, and I was also worried just of
3:59
Speaker 1: what people would think of me and my son if
4:02
Speaker 1: this was a huge disaster. But when we when we
4:06
Speaker 1: started getting people online and it's started to work, and
4:10
Speaker 1: the boys started to sing right. It was the first
4:26
Speaker 1: time they were singing together in a long time, and
4:29
Speaker 1: I could see through the zoom the faces on the
4:33
Speaker 1: different boys start to light up. I could see the smiles,
4:35
Speaker 1: I could see it in their eyes, and I could
4:37
Speaker 1: see it in the director's eyes, thinking, hey, this actually
4:41
Speaker 1: is working. That experience of seeing people and having them
4:52
Speaker 1: so happy and having their experience with others impact their lives,
4:59
Speaker 1: that's really what led me to put more and more
5:01
Speaker 1: time into this and ultimately to devote all my time
5:04
Speaker 1: towards it. Prior to the lockdown, people must have thought
5:08
Speaker 1: about this problem and tried to solve it. Were there
5:11
Speaker 1: existing technical solutions to the problem of how to sing remotely?
5:17
Speaker 1: There were, thankfully. I looked at several of them when
5:20
Speaker 1: I started out, and I came across this small project
5:25
Speaker 1: from out of Stanford University called jack Trip, which is
5:28
Speaker 1: an open source technology that was developed over a year
5:31
Speaker 1: ago by the Computer Music department. It seemed very promising.
5:35
Speaker 1: It had a high quality, lossless audio. It focused on
5:39
Speaker 1: low latency and over the Internet, and there were a
5:42
Speaker 1: lot of people already using it. The challenge that I
5:45
Speaker 1: found with that is that it was a command line tool.
5:49
Speaker 1: You had to open a terminal on your computer. In
5:52
Speaker 1: many cases, you had to build it from the source code,
5:54
Speaker 1: the C plus plus source code, and then you had
5:57
Speaker 1: to know the command lines and figure out the right
5:59
Speaker 1: command line options in order to use it with other people.
6:03
Speaker 1: That's great for somebody who's very technically oriented, but in
6:06
Speaker 1: my son's chorus, I was trying to figure out how
6:09
Speaker 1: how can we roll this out to a large number
6:11
Speaker 1: of young boys that don't have this advanced technical expertise.
6:16
Speaker 1: And it was actually a challenge that I thought I
6:19
Speaker 1: was well suited for because I understood the technical side
6:22
Speaker 1: and I also had the product background in order to
6:25
Speaker 1: try to make something that was fairly difficult to use
6:30
Speaker 1: fairly easy for most of the people that were out there.
6:32
Speaker 1: So we're used to is it a normal sonic collaboration,
6:39
Speaker 1: a conversation up with people singing, We're already used to
6:42
Speaker 1: a degree of latency. We've incorporated that into our definition
6:47
Speaker 1: of normal. You don't have to you don't have to
6:50
Speaker 1: take latency to zero to make it sound normal. In
6:52
Speaker 1: other words, Yes, it's all about tricking your brain. Sound
6:55
Speaker 1: travels at roughly one foot per millisecond, and so if
7:00
Speaker 1: you're ten feet away from someone, it takes you ten
7:03
Speaker 1: milliseconds to actually hear something that they say or a
7:05
Speaker 1: note that they play on their instrument, and ours have
7:09
Speaker 1: evolved to adapt to that. And when we play music
7:13
Speaker 1: with a group, our brains are used to having people
7:16
Speaker 1: that are a certain distance away. Once they get further
7:19
Speaker 1: than twenty five feet away from you, it becomes really
7:21
Speaker 1: hard for you to stay in sync with them. So
7:23
Speaker 1: the trick here with these software solutions is simply to
7:27
Speaker 1: get under that twenty five threshold with a way that
7:32
Speaker 1: the sound is being transmitted exactly. And there's many steps
7:36
Speaker 1: along that path when you think about it. There's there's
7:38
Speaker 1: the path from let's say you're singing. There's the path
7:41
Speaker 1: from your voice your vocal chords to the microphone. Then
7:45
Speaker 1: there's the path from the microphone to your computer or
7:47
Speaker 1: whatever audio device you're using. There's the process of digitizing
7:51
Speaker 1: that audio, turning it from an analog sound into digital
7:55
Speaker 1: samples that can be buffered and transmitted electronically. And then
7:59
Speaker 1: there's the process of sending all that data from your
8:01
Speaker 1: computer to somebody else's computer and then essentially reversing the
8:05
Speaker 1: process in order to get into the headphones that someone's
8:08
Speaker 1: listening to you on. And each of those steps adds latency,
8:11
Speaker 1: and the key is just minimizing the latency of as
8:15
Speaker 1: many of those steps as possible along the way. Can
8:18
Speaker 1: you give a layperson's explanation of what we mean when
8:23
Speaker 1: we talk about buffer size? Sure? Buffer size is similar
8:27
Speaker 1: to if you're a shipping company and you're trying to
8:29
Speaker 1: send something from China to the United States. You have
8:32
Speaker 1: to load all of these containers onto a ship and
8:36
Speaker 1: then send that ship across the ocean to its destination port.
8:40
Speaker 1: All those containers then get unloaded and delivered to someone else.
8:44
Speaker 1: So each of those containers could be considered a buffer.
8:47
Speaker 1: The ship itself could also be considered a buffer. So
8:49
Speaker 1: you're kind of grouping things into these larger buckets and
8:52
Speaker 1: then transmitting them from one location to another. Describe virtual
8:57
Speaker 1: studio to make what is this thing that you have created?
9:01
Speaker 1: So the device it's basically using a Raspberry pie microcomputer
9:06
Speaker 1: or a single board computer that fits in the palm
9:08
Speaker 1: of your hand, and that this board only costs about
9:12
Speaker 1: thirty five dollars, and so I combine that with a
9:16
Speaker 1: sound card from a company called Hi Fi Berry, which
9:21
Speaker 1: is extremely low latency, and it plugs directly into this
9:24
Speaker 1: single board computer, so you don't have to go through
9:27
Speaker 1: a USB bus, which by itself adds latency by the way,
9:31
Speaker 1: it just talks directly to the CPU and other components
9:35
Speaker 1: on that board, and that allows it to achieve a
9:38
Speaker 1: latency as low as about one millisecond. But when I
9:41
Speaker 1: realized that I could do that, and I tested it
9:43
Speaker 1: and measured it and everything else, then I realized, Hey,
9:48
Speaker 1: this is really possible. We can really do something with this.
9:51
Speaker 1: Wait back up for a moment. So there is this
9:54
Speaker 1: thing Jack Trip, which comes out of Stanford. How do they,
9:59
Speaker 1: without without getting two swallowed up in the technical but
10:04
Speaker 1: explain to a computer dummy like me, how they propose
10:08
Speaker 1: to solve this latency problem. So you have this lag
10:12
Speaker 1: how does Jack Trip solve that problem? There's really no
10:15
Speaker 1: rocket science to it. What Jack Trip does is it
10:18
Speaker 1: just tries to take the audio packets, to extract them
10:21
Speaker 1: from the chip sets that are in your sound card
10:25
Speaker 1: and transmit them over the internet as quickly and as
10:28
Speaker 1: efficiently as possible. A large part of that is just
10:31
Speaker 1: having small buffer sizes, so jack Trip effectively just tries
10:35
Speaker 1: to minimize the overhead of transmitting this data over the networks.
10:40
Speaker 1: How long did it take you to build the first
10:44
Speaker 1: kind of prototype of this? Building the raspberry pie itself
10:48
Speaker 1: was fairly easy. It just took a few days to
10:50
Speaker 1: convert that over. And I had a few people helping
10:54
Speaker 1: test things with me. And I remember the first time
10:57
Speaker 1: I tested it myself, just as a loop back to
11:00
Speaker 1: the server and back to hear myself, and I was
11:03
Speaker 1: certain I did something wrong because it was instantaneous, or
11:07
Speaker 1: at least to me, it sounded instantaneous, and I thought, now,
11:10
Speaker 1: this can't possibly be sending. But yeah, the more I
11:14
Speaker 1: dug into it, the more I convinced myself that this
11:16
Speaker 1: was really happening. Were you Do you remember what? Were
11:18
Speaker 1: you singing or speaking? Do you remember why? I can't
11:21
Speaker 1: say you don't want to hear anything? No, I was
11:25
Speaker 1: just I was whistling a lot. For usually I whistled
11:29
Speaker 1: from my test. Sometimes I do try to sing if
11:32
Speaker 1: I know nobody's listening or in the room. If you
11:34
Speaker 1: do try to sing, what do you try to sing,
11:36
Speaker 1: I sing about my dog. This is like didn't what
11:41
Speaker 1: is it that Alexander Graham Bell said to his assistant
11:44
Speaker 1: on the first telephone transmission. Was it like, you know,
11:47
Speaker 1: doctor Watson, come quick or something like that? This is
11:51
Speaker 1: like up there with that? And when was that when
11:55
Speaker 1: you did that first test and you couldn't believe it?
11:57
Speaker 1: When was that? That was probably in May? Oh wow,
11:59
Speaker 1: this whole thing went really quickly. Yeah. Yeah. We shut
12:03
Speaker 1: things down here in California in mid March, late March.
12:07
Speaker 1: And I started out just you know, some evenings I
12:11
Speaker 1: would look at what's out there and spend a little
12:13
Speaker 1: time here and there. And the amount of time I
12:15
Speaker 1: spent grew more and more every week, until finally I
12:20
Speaker 1: started really putting things together and building things and testing things.
12:23
Speaker 1: By may, So, walk me through how Rigazzi does their rehearsals. Now,
12:29
Speaker 1: how do you put this into practical service? Well, interestingly,
12:34
Speaker 1: Rigassi still starts all of their rehearsals with Zoom, and
12:38
Speaker 1: we still use that as a tool to carry the
12:40
Speaker 1: video and connect everyone visually with one another. And then
12:45
Speaker 1: they in parallel to running Zoom, they run jack Trip
12:49
Speaker 1: and they use jack Trip for the audio and that
12:52
Speaker 1: means that the video is very heavily delayed compared to
12:56
Speaker 1: the audio. It takes a lot longer for you to
12:57
Speaker 1: see something versus when you hear it. And one of
13:01
Speaker 1: the downsides of that is that conducting, we've learned, is
13:04
Speaker 1: just not possible. You can't conduct visually, in particular with
13:09
Speaker 1: the gesticulations that you would normally see a conductor doing
13:12
Speaker 1: throughout a performance. So what the conductors have done and
13:17
Speaker 1: learned over time is that they have started conducting through audio.
13:23
Speaker 1: They give audio cues throughout a performance to either become
13:26
Speaker 1: a little bit louder, a little bit softer, a little
13:28
Speaker 1: bit faster, and when to cut off. All these things
13:32
Speaker 1: ended up moving from the visual side to the audio.
13:35
Speaker 1: So we still we still use the video just to
13:38
Speaker 1: help everybody seeing one another, helps give them a little
13:42
Speaker 1: bit more sense of presence. So everyone does. Every boy
13:45
Speaker 1: in the choir have to have his own little virtual studio. Yes,
13:50
Speaker 1: early on a major donor that allowed us to distribute
13:54
Speaker 1: this out to the entire chorus. It's about one hundred
13:56
Speaker 1: and fifty dollars for the device, and that doesn't include
13:59
Speaker 1: the accessories, which would be the microphone and some cables.
14:03
Speaker 1: So altogether it's about two hundred dollars. Did you when
14:07
Speaker 1: you set out to solve this particular rob you quit
14:09
Speaker 1: your old job to focus on this? Was that was? That?
14:15
Speaker 1: Was that a big step? Did that seem like a
14:16
Speaker 1: rash step? It still seems like a rash step. It was.
14:21
Speaker 1: It was certainly a monumental and life changing step for me.
14:25
Speaker 1: Um I also realized that this isn't something that's unique
14:28
Speaker 1: to Ragatzi. This is a problem that's really general across
14:31
Speaker 1: the entire world. And to me that was that was
14:34
Speaker 1: one of those moments where I realized that I could
14:38
Speaker 1: do something to help, and I may not ever have
14:41
Speaker 1: an opportunity to have an impact like I could at
14:44
Speaker 1: that moment. How do people I mean, so you have
14:48
Speaker 1: this thing that's now being used by Rigazzi, how do
14:51
Speaker 1: you get the Rigazzi cam't be the only choir out there?
14:54
Speaker 1: Or people who might I mean, have I come from
14:56
Speaker 1: a family where three members of my family are in
14:58
Speaker 1: church choirs and they've been grounded since March. I don't
15:01
Speaker 1: think they think it's even possible to be able to
15:05
Speaker 1: sing together. How have you gotten the word out about
15:07
Speaker 1: virtual studio? Different mechanisms I'm certainly in the Within the
15:11
Speaker 1: Bay area itself, we had a number of connections to
15:13
Speaker 1: other choruses and performing arts organizations. We've also been working
15:18
Speaker 1: a lot with Chorus America, which is a great organization
15:21
Speaker 1: of many choruses across the country. So we're slowly getting
15:26
Speaker 1: the information and the word out that this is possible
15:29
Speaker 1: through through not just our mechanism, but different mechanisms. So
15:34
Speaker 1: it'll be interesting to see as this progresses over the
15:37
Speaker 1: next several months. But wait, I would go even further.
15:41
Speaker 1: There's a whole category of live experiences which could be
15:44
Speaker 1: recreated using this. So I'm thinking, I mean, I don't
15:47
Speaker 1: know whether you're a basketball fan, but imagine an NBA
15:50
Speaker 1: game where everyone who's watching, every fan who's watching the game,
15:54
Speaker 1: has one of these devices in their house, So we
15:59
Speaker 1: could have real time cheering and fan reaction in a
16:03
Speaker 1: basketball game. We can recreate a reel. Right now, they're
16:06
Speaker 1: just doing fake crowd noise. Right it's a total phony experience.
16:11
Speaker 1: But if I have if I elimit, eliminate the latency
16:14
Speaker 1: for fans to react to a sporting event, I've I've
16:18
Speaker 1: got a virtual I've got a realistic virtual fan experience.
16:22
Speaker 1: Do I not. Yeah, absolutely, Honestly, I haven't even thought
16:25
Speaker 1: about sports. But I think you're totally correct, Mike, Mike,
16:30
Speaker 1: this is this is you can this is the future
16:33
Speaker 1: of this, This is That's the first thing I thought
16:37
Speaker 1: about when when I was thinking of because this is
16:39
Speaker 1: this huge because every single I don't know why I'm
16:42
Speaker 1: going on like this, but every single sporting event right
16:45
Speaker 1: now and that goes on television is suffering huge ratings declines.
16:49
Speaker 1: And that is because the experience of watching sports doesn't
16:52
Speaker 1: seem real. And if you're telling me, if it's this easy,
16:55
Speaker 1: I just have to I gotta buy it. I gotta
16:57
Speaker 1: buy a device if I want to play. But people
16:59
Speaker 1: are they were they were paying one hundred and fifty
17:01
Speaker 1: bucks for their ticket to the to one game before.
17:04
Speaker 1: I mean, it's not like people aren't used to investing
17:08
Speaker 1: in a fan experience. Yeah, it's a it's a great point.
17:10
Speaker 1: I think what one challenge we have is just with
17:13
Speaker 1: the video side of it is we can make the
17:15
Speaker 1: audio very quick, but you're still going to see things
17:18
Speaker 1: delayed on the video side. Potentially, if the delay of
17:21
Speaker 1: the video is the same for everyone who's watching, that
17:25
Speaker 1: may not be an issue, like they'll still hear each
17:27
Speaker 1: other in sync with what's happening. Yeah. Yeah. What's fascinating
17:31
Speaker 1: to me is how you start with a very very
17:35
Speaker 1: specific and very very personal problem, which is your son
17:39
Speaker 1: is bombed out that he can't sing with his chorus,
17:43
Speaker 1: And now you know, we're six months later and you're
17:46
Speaker 1: thinking about recreating the way in which we experience live music,
17:51
Speaker 1: and I'm going on and on about rescuing the NBA,
17:55
Speaker 1: And is it I mean, it's this sort of funny
17:58
Speaker 1: and fascinating picture of how innovation works. It is. It's
18:03
Speaker 1: It's amazing, isn't it. Yeah, that whole process of innovation
18:06
Speaker 1: being this this thing that builds and builds and changes
18:10
Speaker 1: over time and ways you don't even expect it to. Yeah.
18:13
Speaker 1: That's That's what I've always loved about technology. It's always
18:16
Speaker 1: drawn me into the world of entrepreneurism and creating products.
18:21
Speaker 1: And now I have the opportunity to do that in
18:23
Speaker 1: a way that involves something I really love, which is music,
18:27
Speaker 1: but also is helping people. Where could you go to
18:30
Speaker 1: listen in on something that's been performance has been conducted
18:34
Speaker 1: or recorded over virtual studio. The Rigassi Voye Chorus has
18:38
Speaker 1: recorded and published a number of our rehearsals online on YouTube,
18:43
Speaker 1: so the best place to see a few examples is
18:46
Speaker 1: probably at Rigazzi's YouTube channel. One last thing, can you
18:49
Speaker 1: give us a little sense of what listeners can do
18:52
Speaker 1: if they want to make use of this new idea
18:55
Speaker 1: or virtual studio or they want to spread the word.
18:57
Speaker 1: Do you have any practical some practical words of advice
19:00
Speaker 1: to our listeners. Yes, I'd suggest going to our website
19:04
Speaker 1: at jacktrip dot org, where we have a lot of
19:07
Speaker 1: different resources and people bowl who are volunteers available to
19:11
Speaker 1: help answer questions. Mike Dickie is co founder and chairman
19:34
Speaker 1: of the Jack Tripp Foundation and the creator of Virtual Studio.
19:38
Speaker 1: And we're hearing the Ragatzi Boys Choir from California rehearsing
19:43
Speaker 1: from over thirty locations at once using jack Trip and
19:48
Speaker 1: Virtual Studio. Be sure to check out our show notes
19:52
Speaker 1: to learn more about online music collaboration and ways you
19:56
Speaker 1: can get involved. Next week, Unsolvable, just ahead of Thanksgiving,
20:01
Speaker 1: retired NASA astronaut Scott Kelly. We'll talk about how social
20:05
Speaker 1: isolation is solvable even when feels like you're trapped alone
20:10
Speaker 1: so many miles and miles away from those who love most.
20:13
Speaker 1: I hope you'll join us. Solvable is brought to you
20:18
Speaker 1: by Pushkin Industries. Our show is produced by Camille Baptista,
20:22
Speaker 1: Senior producer jos and Frank. Catherine Gerardo is our managing
20:27
Speaker 1: producer and our Executive producer is Miilabell. Special thanks to
20:31
Speaker 1: Kobe Gilford, Heather Fane, Eric Sandler, Carle Migliori and Kadisha Holland.
20:38
Speaker 1: I'm Malcolm Bobber