Latest / The Innovation Show / Understanding the Complexity of Modern Technology with Samuel Arbesman | Overcomplicated

Transcript

Understanding the Complexity of Modern Technology with Samuel Arbesman | Overcomplicated · The Innovation Show

0:00
Before we start, I wanna thank our sponsor Kyndryl, with a unique blend
0:03
of AI powered consulting, built on unmatched service capability.
0:07
Kyndryl helps leaders harness the power of technology for smarter
0:10
decisions, faster innovation, and a lasting competitive edge.
0:14
You can find our friends at Kyndryl at KYND R YL Kyndryl.com.
0:19
Why did the New York Stock Exchange suspend trading without
0:23
warning on July 8th, 2015?
0:26
Why did certain Toyota vehicles accelerate uncontrollably
0:30
against the will of the drivers?
0:32
Why does programming inside our airplanes occasionally surprise its creators?
0:38
After a thorough analysis by top experts, the answers still elude us.
0:43
We don't understand the software running our cars or our iPhones or
0:48
our smartphones, but here's a secret.
0:50
Neither do the geniuses at Apple or the Phds at Toyota.
0:55
Well, not perfectly anyway.
0:57
No one, not lawyers, doctors, accountants, or policymakers fully
1:01
grasps the rules, governing our tax returns, our retirement accounts,
1:06
and our hospital medical machinery.
1:08
The same technology advances that have simplified our lives have made the systems
1:13
governing our lives, incomprehensible, unpredictable, and overcomplicated.
1:18
Today's guest offers us fresh, insightful field guide to
1:22
living with complex technologies that defy human comprehension.
1:27
It is a pleasure to welcome back the author of Overcomplicated Technology
1:32
at the limits of comprehension.
1:34
Samuel Arbesman, welcome back to the show.
1:36
Thank you so much.
1:37
Great to have you back, man.
1:38
I have the Arbesman tome behind me.
1:41
There you can see two books on the shelf behind me.
1:44
We've covered the Half-Life of Facts.
1:45
Brilliant episode.
1:46
Great feedback on that, Samuel.
1:48
And also this is Up Next and followed by his latest book, the Magic of Code.
1:52
Next week, Sam, I thought we'd start with your overview of the book.
1:56
So.
1:57
For those people who go TLDL , too long, didn't listen, what's the main
2:02
point from the book Over complicated.
2:06
Yeah, I mean, so the main point is that when we think about technologies,
2:11
obviously our technologies have gotten more and more complex over
2:14
time, more and more interconnected.
2:16
And I feel like each of us recognize this.
2:20
We're like, , we don't understand how our phones work or how various
2:23
other technologies work, but the truth is it's that, that issue of.
2:27
Not understanding these technologies.
2:29
It's not just for the users.
2:30
It is increasingly for the people who actually built these systems or even
2:34
just work with them and the experts who work with 'em on a daily basis, that
2:37
these technologies are increasingly incomprehensible due to the accretion over
2:42
time, the kind of interconnection of all the different systems together yielding
2:46
all these unanticipated consequences.
2:48
And so we, yeah, we are increasingly getting into this realm of
2:52
incomprehensible technologies.
2:53
thought about how.
2:55
You must have had a moment.
2:56
You know that character from The Simpsons that goes, ha ha, that guy
3:00
where when COVID hit, we saw the resilience and the complexity of so
3:05
many of our systems and the lack of resilience of those systems as well.
3:08
And you wrote it before the COVID pandemic, which I thought was, was
3:12
thought was an interesting one, but I mentioned in the intro those three
3:16
instances, I thought we better cover those because people will be coming on.
3:19
Well, what were they about?
3:20
I don't quite remember what happened in the Stark Exchange in 2015, for example.
3:26
Yeah, so this was yeah.
3:28
So back, I guess now, 20 years ago the the stock exchange had an issue where I
3:36
think like certain parts of it went down.
3:38
It was also a, it was actually interesting where I think the New York
3:41
Stock Exchange went down alongside a number of other, computational systems,
3:46
like right around the same time.
3:48
And people actually thought initially that this was just some sort of
3:52
coordinated attack where like people, it was like some sort of cyber attack.
3:55
And then it turns out upon further examination, wasn't, just buggy code.
4:01
And it happened to be that like when you have enough bugs and large
4:04
technologies, eventually you'll have things that simultaneously go down.
4:07
And so yeah, this is, it was one of those instances, and I don't remember the exact
4:11
details, but it was one of these things where I think there was this simultaneous
4:15
failure, which is kind of crazy to
4:16
And the other one was the Toyota car.
4:18
And Toyota, they're a client of mine.
4:20
They're gonna hate me for, for bringing this up, but it's quite interesting.
4:23
I find it quite interesting that.
4:25
Even these companies that we laud as great innovators, become susceptible
4:28
to this, particularly as we move more and more towards code and we
4:32
kind of see code as a science, but as you show in the book, it's more akin
4:37
to complexity and biology at times.
4:41
Yeah.
4:41
So, I mean, right when we think of cars, we think, oh, like it's
4:44
this big piece of machinery.
4:45
And it is, it's big and complex, but it's also increasingly just a
4:50
huge computer program on wheels.
4:52
Like these are computers.
4:54
On wheels.
4:54
Like they, they have, I think depending on the situation, like they can be like tens
4:58
of millions of lines of computer code.
5:00
Which is often used as a rough proxy for complexity of code.
5:03
Which means of course when you have that amount of computer code, it
5:08
is very difficult for any single human to keep it all in their head.
5:11
And so what happened with Toyota vehicles, this is back in I think like
5:15
the, maybe like 2008 timeframe, if I remember correctly that people noticed
5:22
that occasionally certain Toyota vehicles would accelerate, unexpectedly.
5:28
This is called un unanticipated acceleration.
5:31
And then occasionally when these cars would just un accelerate
5:35
uncontrollably they would crash.
5:37
And in certain cases, actually people died.
5:38
It was actually a really serious issue.
5:40
And so there was a court case to try to understand what was going on.
5:44
And it was one of these situations where.
5:47
From what I understand, it wasn't like you could point to a single like line
5:52
of code of like this kind of thing is actually the reason for the failure.
5:56
It was rather that when the system reaches this level of complexity, in this case,
6:02
it sounds like they all, they actually built a system in such a way that they did
6:05
not actually adhere to certain, I guess like responsible kind of coding practices.
6:09
But it was a situation where when you build something that big and don't
6:13
necessarily do it as well as you could have, the incomprehensibility of these
6:18
technologies is almost inevitable.
6:20
And so it wasn't like, oh, you could point to like, oh, this was the reason for
6:22
the failure or some other kind of thing.
6:24
It was just when you have a system this large that's the inevitable result.
6:28
And I think.
6:30
This kind of speaks to also just more broadly how we think
6:33
about failure in systems.
6:36
We want like, like there's this deep seated human desire to say, okay,
6:41
here's the reason for the failure.
6:43
Here's the person that did something wrong.
6:45
And sometimes we can actually do that.
6:47
But as these systems become bigger and bigger, we often can't do that.
6:52
It's just it's an accumulation of a lot of different decisions and
6:57
the accumulation of the increasing complexity of the system, whether it's.
7:02
Software.
7:03
It could be some sort of mechanical system.
7:04
It could be the combination of lots of different things.
7:06
It could be even like a legal system.
7:08
We can talk about that as well.
7:10
That you get that there is this kind of inevitable result of like, okay,
7:14
when these things get so big, you are going to have reduced understanding.
7:17
And as a result, there's going to be failures.
7:19
There's going to be bugs.
7:20
There's going to be a gap between how we think the system actually should
7:22
operate and how it actually does operate.
7:24
And we often don't know this until we see the failure and then we
7:28
can use that failure to better understand what's actually going on.
7:31
And we can talk about how bugs can be kind of, and failures can be used productively,
7:35
but it's it's a very weird sort of mindset that we increasingly need to use when
7:41
we deal with these complex technologies, which is that it's not always going to be
7:45
Okay, here's the single point of failure.
7:46
It's going to be rather, it is accumulation of a lot of things
7:50
that lead us to this point of inevitability and incomprehensibility.
7:53
I thought about again, how you were ahead of the curve here with
7:57
understanding how, and this is a very important term, how code or how
8:03
jargon, how the law system accretes.
8:07
So this term accretion is important one, but ahead of the point
8:11
of where we are with ai, where the AI is now coding itself.
8:14
So.
8:15
We know that concept of the ghost in the machine where the, the machine will do
8:19
something that it wasn't even coded for.
8:22
And nobody exists anymore in the company who wrote that originally line of code.
8:25
And it's like Frankenstein code or as it's called, spaghetti
8:28
code that accretes over time.
8:31
And I mentioned this to a guy who works in a, a long longtime veteran in telcos.
8:36
And he told me, he said, yeah, he remembers at times looking for the
8:40
guy who wrote the original code.
8:42
And he's like, on anybody know where Bob is?
8:44
And he goes, is Bob even still alive?
8:46
And he's like, Bob's like 87, like retired 20 years.
8:49
And they're like, Bob, sorry about this man.
8:52
Do you remember that line of code?
8:53
And inevitably one person will remember it, but that person
8:56
then passes on and there's nobody handing that baton from one.
9:00
Generation to the next.
9:02
There's a whole lot of complexity in there that we're not even
9:06
ready for at this point.
9:07
Where we're going to this jump to now AI writing the code so there's
9:11
no even human hands on it anymore.
9:13
There's a lot in there, Sam, but I think starting with the term
9:15
accretion, we can build on that and bring back in the point you said
9:19
there about the law, the legal system.
9:22
Yeah.
9:23
And so accretion is simply just this process of, of a system kind of
9:27
growing and pieces being added to it a little bit over and over, over time.
9:32
And when we think about complex technologies, that is how they grow.
9:37
People want to add features, they want to add complexity, they want
9:40
to handle additional situations.
9:43
And the truth is, each of those decisions by itself is quite rational.
9:47
You say, okay, here's the system as it is.
9:49
We want it to haKyndryle this new thing or some new, some, I don't know, new
9:53
type of software or whatever it is.
9:55
And then you add some additional feature and so it, it slowly
9:57
but surely accretes over time.
9:59
The problem is though well one, sometimes you're just
10:02
putting things as placeholders.
10:03
So like things are like okay, here's good enough for now.
10:05
And then of course it never gets updated or it never gets modified or
10:08
fixed, and then suddenly you're just adding more and more on top of it.
10:11
But then you end up with these systems that are still essential
10:15
for getting things to work.
10:17
But right.
10:18
The people might be long retired, they might be dead.
10:20
And so this is like legacy code legacy systems where these things that are
10:24
very old or no longer being updated or are no longer being fully under
10:29
understood, are still essential for these systems to be run and Right.
10:33
Exactly.
10:34
And when you were talking about those kinds of situations, I was talking to
10:36
someone who, he used to work at Los Alamos National Labs, which is kinda like this
10:41
national lab where they do these large scale com computational simulations
10:44
around like nuclear explosions.
10:46
And he said he would like occasionally come across chunks
10:49
of code that would have comments around it saying, do not touch this.
10:52
No one knows what it does.
10:53
And like, and it just, that is, and it sounds horrifying and weird, but the truth
10:57
is that's probably much more the rule rather than the exception where we just
11:01
have lots and lots of things that are still necessary for the system to run.
11:06
But And yeah, it's because this slow steady accretion of functionality and
11:11
additional features or whatever it is.
11:13
And then of course, then you have, I mean, you're connecting
11:15
the system to another system.
11:17
So the newer thing is still reliant on the older thing.
11:19
And then, yeah, you get this massive amount of complexity.
11:24
And and then we can talk about this more, but our brains are not designed to
11:27
handle this kind of thing, but especially with the law, going back to what you
11:30
were saying about these legal systems, we add after law or regulation after
11:36
regulation, each one might seem completely rational and completely reasonable, but.
11:43
These different rules are all interacting and they're often interacting in ways
11:48
that no one who is building this system, whether it's a legal system, whether
11:51
it's a more tech, more traditional technological system anticipated.
11:55
And so, yeah, then you get these massive unanticipated consequences.
11:59
And yeah, it's like with the legacy stuff, and you can see this kind of in
12:01
the lead up to the Y 2K situation where people were not quite sure what was gonna
12:07
happen when the year 2000 rolled around.
12:09
'cause s 'cause the, a lot of computer systems only held two
12:11
digits instead of four digits.
12:13
And so they would think it was actually 1900 instead of the year 2000.
12:16
A lot of these systems were developed seventies or whatever
12:20
earlier or a long time ago.
12:22
And yeah.
12:22
And they had to often bring people out of retirement and say, okay,
12:25
like, help us figure out these things because no one was around who had
12:28
actually built those systems any longer.
12:31
I thought we'd mentioned why that's important and there's a piece I pulled
12:35
from the book here that this, we might miss this, that we're at this point where
12:41
it's a digital economy and so much of our infrastructure is dependent on code
12:47
or on a digital economy, and you said The relationship between our human made
12:52
systems and the natural world means that each of our actions now has even more
12:57
unexpected ramifications than ever before, rippling not just to every corner of our
13:02
infrastructure, but to every corner of the planet and sometimes even beyond.
13:06
The totality of our technology and infrastructure is becoming the equivalent
13:10
of an enormously complicated vascular system, both physical and digital that
13:15
pulls in the earth's raw materials and emits roads, skyscrapers, large
13:20
populations and chemical effluent.
13:23
Our technological realm has accelerated the metabolism of the earth and
13:27
done so in an extraordinarily complicated dance of materials.
13:31
Even changing the glow of the planet's surface.
13:34
Absolutely love that.
13:36
But there's a really important point there, and I think this is a point that
13:40
we miss because we're so busy and we have a shifting baseline of technology
13:46
that we just become accustomed to and we're onto the next one then all of
13:49
a sudden, including the race to ai.
13:51
I thought you might share some thoughts on that.
13:53
I think many of us have this sense that like there's us,
13:58
there's humans, or there's nature, and then there's technology.
14:01
And the truth is, first of all, throughout history, humans have always been affecting
14:06
our environment in the world around us.
14:08
But at this point, increasingly we are in this weird sort of like
14:13
co-evolutionary situation where we have modified the world itself at this point
14:18
is this massive technological system.
14:20
And so the the writer Kevin Kelly, he refers to, he talks about the technium,
14:25
the entire ecosystem of technology.
14:28
we are embedded within that.
14:29
And of course what that means is when this thing is massively, massively complex
14:34
that we don't, and we're in it, but we don't necessarily fully understand it.
14:37
And so the the computer scientist, Danny Hillis has talked about this idea that
14:41
we've moved from the enlightenment when we thought we could apply our rationale
14:45
to the world around us to understand it.
14:48
And of course that still pays a lot of dividends and we should not discount that.
14:52
But we've moved increasingly to what he refers to as the entanglement where
14:56
everything is so hopelessly interconnected that we can no longer fully understand it.
15:00
And right when you think about these sociotechnical technical systems.
15:04
Both our infrastructure, our software, the code that runs all
15:09
these different kinds of things.
15:10
Logistical systems.
15:11
We were talking, you mentioned before, with the pandemic.
15:14
All these things are these, it's a, it's this massively complicated vascular
15:20
system that really is a technology.
15:24
It is this massive technology.
15:26
No one person has built this entire thing.
15:28
It's emerged in this very distributed kind of way, but now we are forced to contend
15:33
with it and really try to understand it.
15:35
And in many ways we have to look to it in the same way that and biologists
15:38
try to understand ecosystems.
15:39
We now have to understand our technological ecosystem in order to really
15:43
understand how these things actually work or sometimes increasingly don't work
15:48
You delve a little bit into complexity, the science of.
15:50
Complexity and I thought it was important.
15:52
Not everybody understands the difference between complicated and complex,
15:56
and you do give it a bit of time in the book as well, but you use them
16:00
interchangeably, but they have different meanings and you mentioned that kind
16:03
of Gordian knot of the entanglement.
16:06
I think it's worth explaining that so everybody's on the
16:08
same page as we progress.
16:09
And to, to be clear, I think different people might distinguish between
16:12
complicated and complex in different sorts of ways, and certainly in the book
16:16
I do often use them interchangeably.
16:18
Especially also just due to the fact, like if I use the exact same word over and
16:21
over, it would be very annoying to read.
16:23
So I have to have a little bit of synonyms, but I do want to
16:25
distinguish between, right, in terms of complication versus complexity.
16:30
I when thing is, when something is complicated, it has, it might
16:33
have a lot of interconnected parts.
16:35
And so , I discuss it in the context between, I, I think I use the example
16:38
of a whole bunch of buoys in the water that are all interconnected.
16:43
And so you, you take all the buoys, you, and there's like a whole bunch of ropes.
16:46
You throw 'em out, you throw 'em on the dock or on the pier or whatever.
16:50
the interrelationships is very complicated, like trying to see, okay, how
16:54
this one is connected to this other one.
16:56
it, it requires a lot of explanation and like description
16:59
to understand what that is.
17:00
But that system is static.
17:03
It's not, it doesn't have sophisticated behavior.
17:06
It just has this very convoluted structure.
17:09
a system is complex, not only does it have a lot of parts, but, due to
17:15
the interactions of those parts, it yields a whole set of behaviors that
17:20
are, that require sort of a different mode of thinking to understand.
17:24
So like if you took all those buoys.
17:26
all those different connections and then threw them in the water.
17:28
And then now with waves and things like that, you now suddenly have feedback.
17:31
You might have some sort of weird emergent behavior where they all move together in
17:34
a way that you couldn't fully anticipate.
17:36
And so those kinds of like non-linear feedback and emergence and maybe
17:43
some sort of concerted behavior.
17:44
It's the behavior coupled with the interactions and the connections that is
17:49
the sort of comp the kind of complexity and complex things that I'm talking about
17:53
here and of course all these systems that we were talking about so far.
17:58
not just complicated in terms of, oh, like this is a really
18:00
hard thing to keep in my mind.
18:02
They are also in, they're also very complex because not only do I need
18:06
to keep the bits and pieces in my mind, I need to understand how they
18:10
all interact and then how those interactions yield other interactions.
18:13
Then of course, you have a second order thinking and third order, and
18:15
it becomes this weird non-linear, highly complex process and behaviour
18:20
There's a book I'm reading as well, I can't see it here.
18:23
I thought I had it.
18:24
It's called human Errors by Nathan Lents.
18:27
And it's about all these errors we have in our genome and as
18:32
humans, you know, having a tailbone that's no longer needed anymore.
18:36
And you do touch on this later on that just as nature has that as we have
18:41
junk, DNA, that we think is junk, we don't know actually quite what it does.
18:46
It could be very valuable that we don't know what it does at all.
18:49
That this happens in code as well, which I thought was remarkable,
18:52
that somebody might go, don't, you can't remove that line of code.
18:56
You might be trying to clean up the spaghetti code, but it does
18:58
something and it will have a knock on effect like the buoy.
19:01
That's really important to understand because people go, well, why
19:04
don't we just start from scratch?
19:05
And you're like, eh, it's not that easy.
19:09
Yeah.
19:09
And so this is this idea of like the Kluge, which is like this thing like you.
19:15
build a system, it works.
19:17
But oftentimes you're building in certain parts in a way that are not always the
19:23
cleanest or where like things work.
19:26
But as you try to actually interrogate the system, you realize that these
19:30
things that you think maybe are not as relevant are actually incredibly
19:35
vital for the system to operate.
19:36
And so, yeah, and especially within software going back to like this accretion
19:40
process, like these things that you might think are old or no longer relevant are
19:45
actually still vital for the system to operate even if you don't necessarily
19:49
fully understand them and Right.
19:50
And like in terms of like evolution, We have many things
19:53
in our evolutionary history.
19:56
Like we look at ourselves like we are the like we are the cumulation of
20:01
many things that maybe are no longer quite relevant but are still there.
20:05
They might be relevant, they might not be.
20:07
But we have to recognize that kind of thing.
20:09
And so I think I mentioned the example in the book of when so where we used to live
20:17
the backyard, there were these trees that had these giant spikes on them, basically.
20:22
Like they were kind, kind of,
20:23
frightening
20:24
Honey Locust
20:26
I think it was honey
20:27
does Honey locust.
20:28
Yeah.
20:28
And yeah, the honey locust tree.
20:29
And so it turns out and I was doing a little digging into why, there are
20:33
these spikes and biologists think that the reason they're there is was to
20:39
actually prevent the giant megafauna of North America from eating the leaves
20:44
or eating the fruit or whatever of the tree, or eating the bark, maybe.
20:47
I don't know what it was.
20:47
But of course, these giant megafauna, like these, like massive sloths or
20:51
whatever, like they've been extinct for tens of thousands of years.
20:54
But I mean, but evolution takes time and so these trees haven't gotten the message.
20:59
So that kind of thing is still part of the genetic information of of the tree,
21:05
even though it's no longer relevant.
21:07
And of course you see this kind of thing with with code and technology more
21:12
broadly where it's just like these things accumulate, they're maybe not relevant,
21:15
maybe they are, we don't really know.
21:17
Yeah.
21:17
And of course, yeah, going with like junk, DNA, we now realize that like a
21:20
lot of that is not actually junk, but systems, they have all these kind of
21:25
accumulations of things, bits and pieces.
21:27
There's been optimization processes that we're not fully under, fully clear on.
21:30
And as a result, yeah, you end up with these things.
21:32
You don't fully understand them.
21:33
They be klugy, they kind of work.
21:35
We don't really know.
21:35
And this is actually one of the, so one of the other features of a lot of
21:38
these technologies we've talked about accretion and interconnection, one of
21:42
the other forces that kind of causes these systems to become very complex.
21:48
Is the idea of contending with edge cases and kind of the
21:50
weirdness of the world itself.
21:52
The evolutionary process with within biology will add bits and pieces to
21:57
handle, oh, this situation or this other exception, or this other kind of thing.
22:01
It will not necessarily result in something beautiful or pristine or clean
22:07
or optimize or we'll say it might be optimized, but it might also, it might
22:11
not necessarily be a kind of like a simple elegant kind of result, but it works.
22:15
It gets the job done.
22:16
And the same kind of thing happens in technology where, especially when you're
22:20
dealing with the messiness of the real world, especially the messiness of
22:24
the human made, real world, you have to contend with all these edge cases.
22:27
And as a result, you end up with a very complex system.
22:30
And so I give the example of like building a calendar application.
22:33
Where like you would think, oh, it's like very easy, like I'm
22:36
gonna build a calendar app, it's gonna have 365 days in a year done.
22:39
And of course that's not quite true.
22:41
To actually handle our current Gregorian calendar, you have to realize, okay,
22:46
not only are there leap years, which are not every four years, they're actually
22:49
like it's every four years on, unless it's every a hundred years, which is
22:53
also not divisible by every 400 years.
22:56
Then you have to, which is like a weird kind of algorithm you have to include,
22:59
then you have to deal with maybe time changes and the weirdness of time zones
23:02
and all these other kind of things.
23:03
And then suddenly you end up with this massively complex piece of
23:06
software because it's dealing with all of these weird, messy edge
23:10
cases of reality and to solve it.
23:13
Yeah, you would think, okay, maybe I can make some like very clear algorithm, but
23:17
no, you often have to just put lots of like weird if statements and other kinds
23:21
of things and it gets all very, very complex and hard to understand as well.
23:24
I love that when you talked about the edge cases, I heard one which was
23:28
incredible, which was self-driving cars that once the self-driving car didn't
23:33
know what to do because a truck went past full of traffic lights, transporting
23:39
them from one place to the next and the car was like going it was not compute.
23:44
Explodes, it didn't
23:46
:). Oh, I love that.
23:47
Yeah.
23:47
And then like, yeah, like finding these edge cases and these things right.
23:51
, They're often super interesting and I feel like with a lot of, and
23:54
ai, like the current AI systems.
23:56
And the interesting like thing is with this, with the book over complicated, so
23:59
it came out in 2016 before the current big wave of complex LLMs, but increasingly.
24:06
The way we begin to understand these LLMs is by finding yeah, these weird edge
24:10
cases and exceptions to understand, okay, like how are these things actually doing?
24:14
What they're doing?
24:15
How are they actually processing information?
24:18
And and actually anyway, we'll discuss this next week, but like the magic
24:20
code actually to discuss a lot of these things around edge cases and
24:23
glitches and using these as windows to understand our systems much more.
24:28
But oftentimes, right, you have to haKyndryle the edge cases, then
24:32
you also increasingly realize that oftentimes you have not haKyndryled
24:36
all the edge cases, but you don't realize this until yet another edge
24:40
case comes to the floor and you're like, oh my God, I hadn't realized this.
24:43
And I believe I discussed this idea it's a term from physics, this idea of
24:48
a system that is robust, yet fragile.
24:51
And the idea behind this and maybe I'm using the term a little bit more
24:54
metaphorically, but when a system is robust, yet fragile, it means that
24:58
it's robust to all the things that we anticipate it being able to handle,
25:02
but incredibly fragile to basically anything outside of that distribution.
25:06
Outside of any of the things that we've built the system to handle.
25:09
And, and is though, is we often don't realize that a system is robust, yet
25:14
fragile until we are confronted with that fragility until some bug creates some sort
25:20
of catastrophic cascade or whatever it is.
25:22
And of course, we see these things more and more with where it's yeah,
25:26
some weird buggy version of windows or some weird update, like causes weird
25:33
cascading suddenly, all the airlines have gone down for like several hours
25:37
and we see these kinds of things like they happen not infrequently.
25:40
And yeah.
25:42
And it's it's very
25:43
humbling.
25:44
that intellectual humility is the common thread I see
25:48
through the three books in that.
25:50
That's where it starts from.
25:51
And then you just bring it into different realms.
25:53
So you bring it from actual facts and knowledge into the digital realm.
25:58
And then you go into the code realm next, which we'll talk
26:00
about next week, which is nice.
26:02
And it's great.
26:02
That's why I love doing these series like this back to back because it's a privilege
26:06
to get an author like this, to to do it, but also to see the progression of their
26:11
thinking over time, which is really nice.
26:13
But there was a couple of things.
26:16
When you talked about the complexity, the buoys tied together,
26:20
I thought about, well, that's really important to understand.
26:22
You said, for example, a piece of code goes down over here, it doesn't
26:25
seem like a big deal, but then something over here doesn't work.
26:28
You talked about Gmail, for example, happened Gmail, and they were like,
26:31
oh, what the heck's happening here?
26:32
And it was connected to some cloud that they, somebody coded it for that
26:36
didn't, shouldn't have been to, so there's all this klugy code or spaghetti
26:41
code in there and those people move on, maybe hired somewhere else, et cetera.
26:46
But then there's the world where you build upon that.
26:49
So you go the world opening up APIs, the interoperability of code seems
26:57
like a great thing, which it is.
26:58
It needs to be accessible so it can connect to everything else, but
27:02
there's a huge cost there and there's a trade off with cost and Failure.
27:06
And again, these are things we don't really stop to think about.
27:10
And why I like the book is that you, it's stuff that you just wouldn't think of.
27:14
And as you said, until it happens, you, and we don't actually foresee this.
27:18
We don't scenario plan, we don't red team, we don't chaos
27:21
monkey like Netflix would do.
27:23
So maybe we'll talk about the interoperability and that trade off.
27:27
This is one of those things where on the one hand, building
27:30
systems that are interoperable, sounds like a, an alloyed good.
27:34
Like you, oh yeah, you want this kind of thing.
27:36
Like you want systems to be able to pass information and update in certain ways.
27:41
And, that's great.
27:43
But of course it come, it does come with trade offs as you mentioned, which is
27:46
that when one system maybe does something wrong or one system is updated in a way
27:53
that the other system can't anticipate, can have things that, that then cascade
27:58
through a system and cause failures that you might not have recognized,
28:02
like inter, so interoperability, right?
28:04
These kinds of things always come with trade-offs and and that's fine.
28:08
And I think for me, like going back to the intellectual humility, it's more
28:11
just a matter of recognizing that when we build our systems, we have to recognize
28:16
that it's going to come with trade-offs.
28:18
These systems might be more sophisticated, but also as a result,
28:22
they might be more conducive to failure.
28:26
And so, and you mentioned like , the cast monkey system within Netflix, way it
28:30
operates is actually trying to actually reduce the amount of unexpectedness and
28:37
lack of understanding there, because oftentimes when these systems go down,
28:43
you're not gonna fully understand the entire shape of the system.
28:45
So really the best way to sometimes understand how these systems actually
28:49
should operate is to, in this case, , almost like inject bugs,
28:53
inject failures into the system.
28:55
And so the way the Chaos Monkey system within Netflix works is it like
28:59
periodically just will, randomly take down different subsystems to make sure that
29:03
when there is some unexpected failure, the system is as robust as possible.
29:07
So the engineers are able to actually build in a certain amount
29:10
of resilience and robustness.
29:13
But , when we have this kind of interoperability, it also
29:20
highlights, two interesting things.
29:22
One is the fact that sometimes, going back to what we were saying earlier,
29:26
like for failure is not a person.
29:29
It's not, oh, someone did something wrong.
29:31
It's more these two things, which on their own can behave perfectly fine.
29:37
When they interact, there are going to be unexpected consequences.
29:40
And so maybe you could say, oh, then it's the fault of the person
29:42
who created the interconnection.
29:43
And of course, I'm sure , you can always place blame, but you we also have to
29:47
recognize that to a certain degree, these are systemic kind of failures.
29:50
They're not just like individual choices that are causing these failures.
29:53
The other aspect of these systems and in terms of like the interoperability
29:57
is actually also related to somewhat how we think about open source software.
30:01
And I discussed this a little bit more in my next book as well, but
30:04
oftentimes we think, oh, like you're building on open source software.
30:07
A lot of people have been looking at it.
30:08
It should be really well understood.
30:11
And and sometimes that is true, but many times systems that we think are
30:17
really well understood they're actually integral for lots of other systems, like
30:22
the piece of open source software, but it might be, it might not be maintained
30:26
by, I dunno, thousands of people.
30:28
It's maintained by one guy who's he doesn't have enough time to haKyndryle
30:31
these kinds of things, or he built it and then moved on to something else.
30:34
But it's still essential for a lot of these other systems and.
30:38
see this where there was what was it?
30:40
The the heart bleed bug.
30:42
I think that might have been the one that I mentioned where it was like,
30:44
it was one of these failures where it turned out it was like something
30:47
related to the internet infrastructure.
30:50
And it was like, yeah, this one little bit of code or this piece of software that was
30:54
essential for all these different things.
30:56
And it was like this one guy built it a long time ago and it propagated.
31:00
And for me, actually, one of the takeaways there I think is to also
31:02
just realize that we need to really just invest more into like maintenance
31:09
and support for these kinds of things.
31:11
If they are essential and if they're being used by all these different systems
31:14
and like they're then we should really.
31:17
Actually them in a way that kind of befits 'em, like all the companies
31:21
that maybe use these kinds of things should actually pay it forward.
31:24
Or actually this could pay it back.
31:25
Like actually allow these systems to be maintained properly.
31:28
But overall, I would just say the interoperability kind of goes hand in hand
31:32
with this intellectual humility of like, yeah, when these things interconnect our
31:37
understanding is going to be reduced.
31:38
It doesn't necessarily therefore mean that we don't understand it entirely or at all.
31:42
And we can talk about this more about understanding is really
31:44
this kind of like spectrum.
31:45
It's not like an either like full understanding or nothing.
31:48
But we have to recognize that as these systems become more interoperable
31:52
or just increasing in size and are built over time, our understanding is
31:56
going to be reduced., And that's fine.
31:58
We just need to confront that with, with open eyes.
32:00
And it brings us, then you teed us up earlier on to the idea of the
32:04
biological limits of human comprehension.
32:06
So whether we like to admit that or not, it is very true.
32:10
We have limits and one of the things that's been bugging me like
32:14
a buzzing fridge is the idea that.
32:18
It's gonna get worse the more we outsource to the machine.
32:21
So digital dementia, where we do more and more of our thinking.
32:25
If you get ChatGPT to write for you and you're not going through the struggle
32:29
of the writing, you're not using that muscle, that muscle atrophies,
32:32
that brain atrophies, et cetera.
32:33
But you say, as our systems become more complex over time a gap begins to grow
32:38
between the structure of these complex systems and what our brains can handle,
32:43
whether it's the entirety of the internet or other large pieces of infrastructure.
32:46
Understanding the whole is no longer even close to possible and hand in
32:52
hand with this becomes what happened post-industrial revolution of our drive
32:57
towards specialization and the lack there of, of T workers or generalists.
33:05
Yeah.
33:06
And so starting with kinda like the biological limits of like
33:10
our ability to understand, right?
33:11
These systems, the way we build these systems is based on right,
33:14
huge number of interconnected parts.
33:16
They're all doing things.
33:17
They're, there's feedback, there's nonlinear phenomena.
33:22
These are not things that we evolved to think about.
33:26
And, the truth is very quickly our brains break down.
33:30
And I think I actually even discussed like a whole bunch of different examples
33:33
of the things our brains are good at, the things that computers can handle.
33:35
And so, yeah, like and there's like a classic paper looking at like how
33:38
much, like how many bits of information we can hold in our short-term memory.
33:41
And it's not hundreds or thousands, which of course is still much
33:44
smaller than these systems.
33:45
It's seven like plus or minus two.
33:48
And so like.
33:49
that shows we are very limited.
33:51
The speed of our neurons is , not particularly fast.
33:55
There's different estimates about like how much we can actually
33:57
hold in our long-term memory.
33:58
So that actually might be reasonably powerful.
34:01
But especially if you look at being able to hold, I was mentioning
34:04
before, like these like nonlinear feedback kinds of things.
34:06
Like these are things where systems are feeding back on themselves.
34:10
There's all that's related to this idea of like recursion where something
34:13
is calling itself and building things more and more complicated.
34:17
also occurs in language where you can say like I thought that she thought
34:21
that, he thought that I thought like about this, or whatever it is.
34:23
And of course you can't do that more than a few times before your kind of
34:26
brain breaks down in terms of trying to understand a sentence like that.
34:30
A computer ha should have no issue with actually parsing that kind of sentence.
34:34
And so you can see that once you have multiple different layers of
34:37
feedback our brains are not good at that, but these systems can keep
34:43
on doing these kinds of things.
34:44
And I think there needs to be a recognition, right?
34:46
That we are building these systems in a way that we are not
34:49
particularly good at understanding.
34:51
Now the question becomes what do we do with that?
34:53
Like how do we still try to understand these systems?
34:57
And this kind of goes to what we were talking in terms of understanding, like
34:59
understanding is not a binary condition.
35:01
It's not okay.
35:01
I either understand this thing with perfect, like perfect
35:04
fidelity and understanding every aspect of it, or I'm living in
35:08
complete and total utter ignorance.
35:11
You can be somewhere in between.
35:12
And in fact, of our lives of understanding things, that's where that's where we are.
35:16
And I think trying to figure out how we can move close, like closer to the more
35:22
complete understanding and away from the complete utter ignorance the key.
35:26
And so whether or not it's like saying, okay, I have I can understand
35:31
how this module connects to this other piece, or I can understand the
35:34
broader shape of what's going on.
35:37
That can be helpful.
35:38
Going back anymore.
35:39
You were talking about with AI of like, on the one hand, you're right,
35:42
it's like reducing our understanding of these kinds of things and maybe
35:44
we're no longer thinking about them.
35:45
That being said, think going back to trade-offs if we say to ourselves,
35:50
okay, we will never understand these systems, but can I use tools to better
35:55
understand the underlying technology?
35:57
So it's one of these weird things where on the one hand , I don't
35:59
understand this technology.
36:01
But then I'm gonna use another technology to actually understand
36:04
the original technology.
36:05
And that also comes with trade-offs because maybe you'll
36:08
get an incomplete understanding.
36:09
Maybe the technology you're using as a tool, of this like
36:12
technological microscope itself, you might not fully understand.
36:15
And so there's always going to be trade offs.
36:16
But I do think.
36:18
can sometimes use these tools, these additional technologies,
36:21
to move us a little bit closer to more complete understanding.
36:24
We might not ever get there, but we can do these kinds of things.
36:27
And so one of the examples I give this is like well before our current
36:30
era of AI was I think Google wanted to better understand the energy
36:35
efficiency of their data centers.
36:36
And they used what was then probably reasonably complex machine learning
36:40
what probably now is like extremely rudimentary, but they recognized
36:44
that they were using this technique to actually understand the systems
36:47
that they themselves could not fully and completely understand.
36:50
And that's fine.
36:52
and I think that can actually be very useful.
36:54
Oftentimes though when we build these systems or build a mental model
36:59
for these things it's going to be a simplification of the overall system.
37:02
The question becomes how useful is that model?
37:05
Which is actually one of the things I also discussed in the magic of
37:07
code about, like thinking about complexity and managing complexity.
37:10
But yeah, this is one of those, one of those things where I think we have to
37:17
recognize that like the technologies we already have, like they're not,
37:20
it's not like, oh, we are increasingly going to get to the point where we're
37:23
not gonna understand these technologies and we have to grapple with that fact.
37:26
have been there for a long time.
37:28
the problem though is for many of us, especially if you're not
37:31
dealing with these systems on a daily basis, you've been shielded
37:34
from that level of complexity.
37:35
And I don't think I included this in overcomplicated, but it was around the
37:39
time when the Apple Watch first came out.
37:44
There was this article in the it was in the Wall Street Journal, I think like the
37:47
style section talking about, okay, are people still gonna buy mechanical watches?
37:51
Are, is everyone just gonna start buying smart watches?
37:53
And they interviewed this one guy who was a big fan of mechanical watches and of
37:57
course there people are still buying them.
37:59
And he said, yeah, of course I want a mechanical watch.
38:01
Like I think about a mechanical watch it and all of its
38:03
sophistication, it's so wonderful.
38:05
As opposed to the smartwatch, which is just a chip now.
38:08
Like I will not deny that a mechanical watch is sophisticated, but like a
38:12
smart, like a chip, a smartwatch is like, it's orders of magnitude more
38:15
complex than a mechanical watch.
38:16
But for many people they're shielded from that level of complexity.
38:21
one of the things that I actually try to argue for is like finding ways to
38:26
peek under the hood to actually kinda reveal a little bit of that complexity
38:30
because increasingly we are being shielded from these kinds of things.
38:34
Like you look at like an iPad which is just like, I don't even know
38:37
how the files are stored versus you look at like early computers.
38:40
, Like my family's first computer was the Commodore Vic 20
38:43
connected to your television.
38:44
Used these like cassette tapes to store information and the way you got
38:49
computer programs is like, oftentimes you just enter the code in yourself.
38:52
And, I have vague memories of my father doing this kind of thing.
38:55
And you could see the clear relationship between the code being entered, like the
38:58
text being entered and what was happening on the screen, sometimes resulting in a
39:02
game or something cool more often than not, some weird buggy thing or whatever.
39:05
And you could see that clear relationship and.
39:08
And I think for many of us though, we've been shielded from that kind of thing.
39:11
And so one of the things I argue is like finding ways to at least
39:14
reveal a tiny bit of that complexity.
39:17
'cause otherwise going to think, oh, this thing is actually really
39:20
simple until some failure or some cascading catastrophic issue.
39:25
And then we're confronted with the fact that, oh no, this systems
39:26
are actually much more complex than we might have realized.
39:28
often.
39:29
Even on as a reality thing that, you know, what, what we see colors and all that.
39:35
It's just an interpretation by the lens we have.
39:38
And you, you mentioned it as well, and I went down a rabbit hole of
39:41
looking at , how technology actually bolsters our level of understanding.
39:46
But one of the things I, found in the history was we went from having
39:50
just two cones to trichromacy.
39:53
So we now were able to observe more colors . But I often think about
39:57
that, the way we perceive things is essentially like the desktop on a
40:01
computer versus the MS dos green writing.
40:04
It's just all these ones and zeros on the screen, et cetera.
40:07
But we'll move on for that.
40:09
But the point I wanted to just lean on there was the, that term abstraction,
40:13
which is, , I just assume that in a way so I can just actually get on with it
40:18
that I can level up my understanding.
40:20
But I mentioned this that it became even more apparent to me after our
40:24
last conversation and today was that.
40:27
The importance of using the technology to actually level up.
40:31
Because of that limited, the magical number seven plus or minus two, because
40:36
we have a limit in our understanding.
40:37
We can actually use the technology to get above that.
40:40
So we don't have to hold all that in working memory and we can then make
40:43
connections and patterns at a different level that we couldn't make before.
40:47
And I think that's what technology can do for us if we use it for that.
40:52
But one of the problems I have, and I don't know if you've had this and
40:57
it's why I have that library behind me and probably why you have it
41:00
is the burden of knowledge and the burden of knowledge is a real thing.
41:05
Where, say for example, your new book came out, right?
41:08
So I was like going, I cannot interview him on his new book until
41:11
I've read his previous books to see where he is coming from, right?
41:14
So I have it on the level of this show, and that's why I cover older
41:19
books and people are like going, oh, why are you covering that book?
41:21
And I'm go, well, I kind of need to understand where that that theory
41:25
came from and if the person's still with us, if they're still live,
41:28
definitely getting them on the show to be able to bring those two things
41:31
together and to record that knowledge.
41:33
So you don't even have to read the book anymore and you can maybe fast track
41:37
the burden of knowledge for yourself.
41:39
There's a lot in there, but I thought that burden of knowledge thing is
41:41
important to understand and how then tech can help us get over that.
41:46
Yeah, so this idea of the burden of knowledge is right, like in order to,
41:49
and actually going, going back to the scientific kind of stuff that we discussed
41:53
last week about the halflife of facts.
41:54
It's like oftentimes, like in order to make discoveries at the frontier, you have
41:58
to learn everything up until that point.
42:01
There are things that have become obsolete or there, I'm going
42:04
back to the halflife facts.
42:05
You don't necessarily have to learn, like the cutting edge alchemy in order
42:08
to make chemistry discoveries, but you have to learn a whole bunch and means
42:13
that it takes a long time and a lot of training to really get to that frontier.
42:19
And, especially in science as we.
42:23
Learn more and more and our body of scientific knowledge increases that
42:28
creates an increased burden to actually learn all these different kinds of things.
42:32
And also in terms of our technologies, like how we build these technologies also
42:36
requires a huge amount of understanding.
42:38
Like, especially as these technologies get more and more complex, you have
42:41
to understand the bits and pieces of different things and certainly the
42:44
framework that it was used in some piece of software or whatever it was.
42:48
And abstraction can help with that kind of thing, where you can say,
42:50
okay, here's the thing as it is.
42:52
I've abstracted away the details.
42:54
I don't necessarily have to worry about everything that kind of came before it.
42:57
problem though is sometimes when abstraction layers break down
43:00
and you actually are forced to confront all, like all of those
43:03
kind of tho those pieces as well.
43:05
. And so, the burden of knowledge, whether you're thinking about
43:07
technological advancement or scientific advancement, Does, right?
43:11
It is this increasingly pressing thing where it is harder and
43:15
harder to create new knowledge, new technologies, whatever, because you're
43:21
often required to learn more and more to get there at that frontier.
43:25
There are ways of sidestepping and going back to this abstraction, but it is yeah,
43:29
it is a really big issue and then of course it goes hand in hand with, there's
43:35
this increasing amount of knowledge that's required or increasing amount of
43:37
understanding, our brains are finite.
43:40
Like, it's not like one of these things where, okay, I can spend, a hundred years
43:44
understanding this kind of thing, or I can absorb millions of pages of things.
43:49
Like you can't do that.
43:51
Like there either you don't have enough time, you don't
43:53
have enough mental capacity.
43:54
It's just not possible.
43:55
And so, right then it becomes, can I use.
43:59
Tools, increasingly computational tools and technologies to help me
44:06
grapple with the burden of knowledge.
44:07
Whether it's increasingly complex technology or one that has lots of
44:10
different pieces together, whether it's scientific knowledge and grappling
44:14
with different bits and pieces from different sub fields or entirely
44:17
different fields to make new discoveries.
44:19
You need tools to , help you better understand these kinds of things.
44:22
But as a result way that like, I want to like peek under the hood, but sometimes we
44:27
feel like we're increasingly distant from our technologies, you sometimes are, using
44:32
these kinds of things in a way that's mediated by these assistive tools as
44:37
opposed to getting into the nitty gritty.
44:38
So there's always going to be these sorts of trade-offs and we
44:41
just need to be open about them.
44:42
I mentioned there, the specialization, and you touched on this, the idea of
44:46
being a generalist, that you have so much knowledge, but actually when you
44:51
mentioned, for example, the Apple Watch, just being a, oh, it's just a chip.
44:55
Well, it made me think of what you said about the autonomous, or not even an
44:58
autonomous vehicle, but an electric car, that it takes so many different
45:01
disciplines, different specialists to come together and coordinate and
45:07
neurodiversity to see that same challenge from different angles with different
45:10
specialties to bring it together in order to develop that type of thing.
45:14
But oftentimes then to think about that is very difficult when you
45:19
bring people, and they're all seeing it from their own perspective, the
45:22
whole time, their own specialization.
45:24
But then the generalist, as you said, is aware of all those different disciplines,
45:30
but also then able to zoom up above them and see the common denominators,
45:34
the overlaps, the Venn diagrams between all these different things.
45:38
But that is not very valued, but extremely valuable in today's society.
45:43
I know I'm jumping a little bit ahead here, but I thought it's an important
45:47
thing because I know many of our listeners would be in that bucket and
45:51
would appreciate the shows, and, and it's why I try and draw from different
45:54
elements, but there's common denominators in all this stuff that is important for
45:58
innovation or change or transformation.
46:00
And going back to burden of knowledge, like as you require huge amounts of
46:04
specialists to deal with these systems.
46:06
You require lots of expertise and it requires a lot of effort.
46:09
The interesting thing about, about these complex technologies
46:11
is exactly what you're saying.
46:12
They often draw on many, many different areas of expertise.
46:16
And so there's huge amounts of specialization involved, means
46:20
and large, no single expert can haKyndryle all these different things.
46:23
And so then the question becomes, can we actually gain a better sense of
46:27
understanding of these technologies?
46:28
And the, the key is, someone who has a more kind of thinking.
46:35
Sometimes people call these like T-shaped individuals where they might have a series
46:38
of expertise in one area, but that's the vertical part of the T and then the
46:41
horizontal part of the T allows you to jump across lots of different domains.
46:45
And whether that's being able to more easily translate
46:49
different types of jargon.
46:51
Understanding exactly what you were saying before of kinda like
46:53
mental models that can be useful for multiple different domains.
46:57
I actually do think increasingly as our technologies become interdisciplinary in
47:03
the different domains that they touch on, we need these kinds of generalist thinkers
47:08
or these kinds of t-shaped individuals.
47:11
The problem though is how do we create them?
47:13
How do we support them?
47:14
How do we kinda valorize them in today's society where I think
47:17
expertise is often very, very valued.
47:20
Not to the detriment of generalists, but it's very, very hard sometimes
47:23
to figure out how to cultivate them.
47:25
And David Epstein had a wonderful book range where he tried to actually like
47:28
talk about the value and the power of of people with more generalist
47:33
thinking and how to get these people, build them and them in the world.
47:37
'cause I think we need more of them, but especially when it comes right to building
47:41
these technologies we need more of them.
47:43
And actually, one of the things I discuss is I remember my mother told
47:48
me about when she was in, what was it?
47:51
The Girl Scouts there were these different badges you could get.
47:55
You could, I dunno, get a badge for learning how to like go camping or who
48:01
knows, all these different, but then there was one badge called the dabbler badge
48:06
and it was like getting like just doing a little bit of lots of different things.
48:11
And she told me that she earned the dabbler badge.
48:13
And it was like, and I love the fact that like there was this reward or
48:16
there was this recognition of like, knowing a little bit about lots of
48:19
different things is actually valuable.
48:21
That in itself is something that we need to actually value and and recognize.
48:26
And I think.
48:28
Increasingly as a society, we have to recognize like we need more
48:30
of these dabbler, we need people who are like, we need dabbler
48:33
badges for these kinds of things.
48:34
'cause those are the kind of generalists who they will need to
48:36
work in conjunction with experts.
48:38
It's not an either or.
48:39
It's not just, oh, we only need experts, or we only need generalists.
48:42
But we have to recognize that cultivating generalists as well as actually
48:47
inclu incorporating them into our organizations is very, very valuable.
48:52
'cause oftentimes, like when you have an organization that is just full of
48:55
experts, like they're dealing with their very specific thing and there is
49:00
no one then in that organization that has the luxury or the mandate to say,
49:04
oh, let's take a little step backward.
49:05
Let's figure out how these things interconnect or
49:07
sometimes fail to interconnect.
49:09
Or are there other ideas that we should be exploring?
49:12
Are there ideas that we can use in this import export business of
49:16
knowledge that we can bring to bear?
49:19
And yeah.
49:20
So we need to find ways to actually recognize, yeah, that this is a very
49:23
valuable part of knowledge creation, of managing complex technologies.
49:28
As opposed to just saying, either there's gonna be people who know everything
49:32
or there's gonna be people who are experts in some very specific thing.
49:35
Like we need to build people who have not knowing everything.
49:40
'cause then it's like a dilletent 'cause then they really actually
49:43
don't have any deep expertise.
49:44
So I think that t-shaped individual or so where you actually do have an expertise,
49:48
but you are also very comfortable jumping into lots of different domains and
49:51
then providing that sort of connective tissue to different experts that is
49:56
something that we really need more
49:57
And just a note to our audience, David Epstein will be on the show next year.
50:01
We're gonna be doing a version of the Sam episode with him covering his books,
50:05
the Sports Gene and Range, and I think he might be working on a new one in the
50:09
meantime, just like you did Sam, as well.
50:11
I'll have to, I have to code in for an extra week there.
50:14
It's like the Y 2K code.
50:16
I'll have to make space for an extra book for people.
50:19
But I was intrigued and I loved, I absolutely loved what you talked
50:24
about, the difference between physics thinking and biological thinking.
50:28
I absolutely loved this.
50:30
And the contrast of those and the mirror of those within a.
50:34
Innovation.
50:34
In innovation literature, we talk about the need for exploration and exploitation.
50:39
Much like you said, the girl guides that type of roaming and exploring
50:44
versus being a soldier and very focused on a certain challenge.
50:49
But that difference is beautiful and I'd love you to take us through it.
50:54
Yeah.
50:54
So I'm sure there, there are gonna be many physicists and biologists that
50:58
will view this as an oversimplification.
51:01
But the way to think about this kind of thing is oftentimes the
51:07
physics mode of thing, whether or not it's always physicists doing
51:09
this kind of thing, but the physics mode of thinking involves like.
51:13
Going back to abstraction, abstracting away the details, saying, okay,
51:16
this system is very complex.
51:18
But I can ignore certain details and try to see, okay, here are the
51:24
broad shape of what's going on.
51:26
And that kind of thing is really, really powerful to say.
51:28
Okay.
51:29
Like looking at the world and saying, okay, throwing a ball the
51:33
way the moon orbits the earth.
51:37
The I don't know, the way the tides work, like these things are all like
51:40
wildly different, it turns out we can abstract away the details and say
51:45
they're actually all being involved.
51:47
Like they're, there's gravity, like gravity is the thing
51:49
that kind of understands the parabolic arc of a baseball.
51:52
And trying to help us explain the tides, help us explain the
51:55
orbits of the planet, so the orbit of the moon or whatever it is.
51:59
And this is incredibly powerful, that kind of like new Newtonian
52:02
approach of saying, okay, here's all these widely disparate phenomena.
52:06
Let's actually try to understand them in some sort of unifying way,
52:10
this kind of unifying approach that ignores certain details.
52:14
And I would say that's like the physics mode of thinking.
52:17
The biological mode of thinking is saying no.
52:20
In fact, sometimes we we need to like revel in all the details because
52:24
the details are actually really, really important to understand.
52:27
Okay.
52:27
What is like the specific evolutionary history for how we got to where we are
52:31
and why this thing does this in biology?
52:34
And of course you can see both.
52:36
And I would say like the biological mode of thinking in physics maybe is
52:39
like where people were just trying to like collect lots and lots of
52:43
different particles when they're like understanding particle physics.
52:46
That would be a more biological mode in physics and in, and conversely the, a
52:49
physics mode in biology would be like.
52:53
Darwin and saying, okay, actually evolution explains a whole host of
52:57
disparate things that we actually see.
52:59
being said, this kind of the detail oriented mode, which is more akin
53:05
to collecting lots of different bits of information and then
53:08
slowly but surely building up a complete picture of what's going on.
53:13
That works really, really well when you have a really sophisticated system
53:19
where the details actually matter.
53:21
And increasingly, those are our technology.
53:23
Those are our technologies.
53:24
We think, okay, our technologies, they are, they're human made.
53:28
They're going to be entirely rational.
53:29
And yeah, sometimes they are.
53:31
But when you have a really complex technology that has an almost
53:36
organic level of complexity.
53:37
It's evolved over time.
53:38
We have legacy, like we're actually using biological terms.
53:40
, And at that point it becomes, oh, maybe we should actually use the techniques
53:44
of biological thinking or even the way in which biologists interrogate
53:48
complex biological systems to actually interrogate complex technologies.
53:52
And often when I give a talk about overcomplicated and complex technologies
53:57
I'll show this massively complex chart.
54:00
It looks like a circuit diagram and it turns out it is a diagram of the
54:03
metabolic processes within a cell.
54:05
So it is a biological system but it looks like a complex technology.
54:10
And of course conversely complex technology are increasingly
54:12
looking like biology.
54:13
And you see this actually with a lot of AI systems where we understand
54:17
the general principles of how these systems are trained, but
54:21
the result is incredibly complex.
54:23
And then it requires interrogating them using almost like biological,
54:28
like field biology kind of principles of weird exceptions why these
54:31
things do these weird situations.
54:33
Trying to slowly but surely build up a picture of why these LLMS or other AI
54:38
systems are doing what they're doing.
54:40
Because oftentimes that is going to be much more valuable than saying,
54:45
oh, like I'll just abstract the way the details and this is how it works,
54:47
because that's not how it works.
54:49
In biological systems or at least using the biological mode of thinking,
54:54
abstracting the way the details misses so much of the system's behavior
54:59
that you can't actually a afford to ignore those kinds of details.
55:02
I have a lovely quote, Sam, right at the end of the book that I'd love to share.
55:06
And before I share that as our final message, I'd love you to
55:08
think about your final message, and , I hope this might tee you up.
55:12
Actually, it's beautifully written.
55:13
You said There is a whimsy and beauty in the complicated and the unexpected.
55:18
A glittering and shimmering technological network with its branching gossamer,
55:23
web of links and interactions is unbelievable in its complexity.
55:27
And sometimes even if we don't understand every part and every hole,
55:31
an imperfect grasp can be enough.
55:33
We can walk humbly with our technology.
55:37
I love that.
55:37
That was the end of chapter five, but I just thought it was a nice finale
55:42
and beautifully written, but over to you maybe as your final message for
55:46
this part two on overcomplicated.
55:49
Yeah.
55:49
Thank you.
55:50
Yeah, and I think Yeah, this message of humility is one of the things that
55:56
I think a lot about, whether it's understanding our science or technologies
56:00
that oftentimes, like going back to what I was saying we're not gonna have
56:04
complete ignorance, we're not also not gonna have complete understanding.
56:07
Our understanding is going to be somewhere in between.
56:10
oftentimes when we are confronted with technologies we don't fully
56:14
understand, often zoom to two extremes.
56:17
One of fear in the face of the unknown, or awe in the face of this
56:21
thing is like so unbelievably complex.
56:23
It's almost like the mind of God.
56:24
And of course, like it's not the mind of God, it's like the mind of, I
56:27
dunno, Google or like some other thing.
56:29
It's built by imperfect people.
56:31
And in the fear in the face of the unknown.
56:34
Yeah, these systems can be complex, but oftentimes both fear as well as
56:38
awe like this kind of undue reverence, they cut off questioning and any
56:43
sort of attempt to fully or further understand the system, what's going
56:48
on, we might never fully understand it, but we need to actually keep on trying.
56:52
And so for me that kind of productive middle ground is humility, which is
56:57
saying, okay, I might not f ever fully understand these things and that's okay.
57:01
And actually I talk about how in the era before the Enlightenment certainly in the
57:07
Middle Ages there was this understanding of like, oh yeah, there are certain things
57:09
that humans might never fully understand.
57:11
And of course, with the advent of the enlightenment, we set that aside and I
57:14
think that's, that is good, that we should actually keep on trying to understand
57:18
our technologies aspects of the world.
57:20
These things are very, very important.
57:22
But we still might need to recognize that yeah, there's going to be limits.
57:25
Humans are incredibly finite creatures and so we should keep on trying to
57:31
understand things but also recognize that it's okay if we don't fully
57:35
understand it in all of its details.
57:37
And so for me, as long as we have this productive humility of never stop trying
57:41
to actually further understand our technologies or the systems we built,
57:44
or aspects of the world around us, but also recognize that we are going
57:48
to be finite and so we might need to grapple with that kind of thing, then
57:51
I think we are going to be much better positioned to actually engage with the
57:57
true complexity of the world around us as opposed to saying like, oh no, this is
58:01
something we can never fully understand.
58:02
Or actually sometimes thinking that it's actually a lot simpler than it truly is.
58:06
And so as long as we maintain that productive humility in the face of
58:11
even the technologies we ourselves have built, I think we'll be able to
58:15
build them better as well as try to actually understand them more and more
58:19
productively.
58:20
And Sam, for people who want to find you, your substack, your books,
58:23
et cetera, where's the best place?
58:26
Yeah, so the best place, if you go to my website, Arbesman.net, that will
58:29
have links to my substack as well as a lot of my writing and my books.
58:33
And that's probably the best way to
58:35
reach me.
58:35
author of Over Complicated Samuel Arbesman.
58:39
And don't forget Up Next Magic of Code next week.
58:42
Thank you for joining us.
58:43
Thank you so much.
58:44
Thanks again to our sponsor, Kyndryl.
58:46
With a unique blend of AI powered consulting, built on unmatched
58:49
managed service capability.
58:51
Kyndryl helps leaders harness the power of technology for smarter
58:54
decisions, faster innovation, and a lasting competitive edge.
58:57
Find out more about Kyndryl at K-Y-N-D-R-Y-L Kyndryl.com.