Latest / Mechanical Engineering Made Simple / (#136) Why Reliability Predictions Fail in the Real World: Designing Systems That Actually Last

Transcript

(#136) Why Reliability Predictions Fail in the Real World: Designing Systems That Actually Last · Mechanical Engineering Made Simple

0:00
Welcome back, design junkies, to Mechanical Engineering Made
0:02
simple. If the podcast brings you value,
0:04
help support the show with a like a follow or leave a comment
0:06
below. You know, usually when we talk
0:09
about engineering, there's this, this underlying assumption,
0:12
almost like a comforting blanket of absolute precision.
0:15
Oh yeah, the illusion of the perfect equation.
0:18
Right, because we're taught from our very first university
0:21
physics class that the universe operates on solvable equations.
0:26
I mean, you have a physical load on a steel beam, you calculate
0:29
the stress, you look up the material properties to find the
0:31
yield point, and the math just points a definitive finger and
0:34
says you know there is your safety factor.
0:37
It's clean. It's totally binary.
0:38
Exactly. Safe or unsafe, Broken or not
0:41
broken. And that is the dream we all
0:43
want to live in, right? We crave that visibility.
0:46
We want every single aspect of the physical universe to fit
0:49
neatly into our carefully formatted spreadsheets, but the
0:54
moment you step out of pure static mechanics and you wade
0:56
into the world of reliability, prediction and failure rates.
0:59
Oh man, it just falls apart. It shatters.
1:02
That pristine mathematical machinery just shatters.
1:05
You are suddenly navigating a landscape defined entirely by
1:08
uncertainty, these statistical ghosts and really diagnostic
1:13
Muddy Waters. Which brings us to today's
1:16
mission. We are taking a massive stack of
1:18
sources today. We've got articles, historical
1:20
data, research papers on statistical inference, and we're
1:24
doing a deep dive into the messy, misunderstood, and
1:27
frankly incredibly frustrating world of reliability prediction.
1:30
It's a massive pinpoint for so many teams.
1:33
It really is. We are looking directly at a
1:35
pain point. I know you, the listener, have
1:37
felt in your bones. You're sitting at your desk.
1:40
You design what you think is a flawless system.
1:42
You run the numbers and the math swears up and down that this
1:45
mechanical valve, or you know, this electronic control unit
1:49
will last for a decade without a single hiccup.
1:51
And then you ship it. You ship it and six months later
1:54
it fails catastrophically in the field and you were left staring
1:58
at your screen just wondering what went wrong.
1:59
Did manufacturing mess up? Did the customer abuse it?
2:02
Well, most of the time, I hate to say it, the failure wasn't a
2:05
fluke of the universe. The failure happened because the
2:09
way engineers are taught to interpret ferry rate data is
2:13
it's fundamentally flawed from the ground up.
2:16
Wait, flawed how? Like the math is wrong.
2:18
No, no, the flaw is almost never in the actual arithmetic you
2:21
did. You probably carried the one and
2:23
divided perfectly correctly. The trap lies in the
2:26
philosophical interpretation of what those database numbers
2:28
actually represent. I see.
2:30
We have this collective industry habit of treating statistical
2:34
probabilities as if they are hard physical laws and they just
2:39
aren't. So today we are breaking down
2:41
the biggest real world issues you run into, we are calling out
2:45
the deadly misconceptions that keep ruining designs, and we are
2:48
going to build a step by step approach to actual practical
2:51
optimization so you can stop designing for a fantasy and
2:55
start designing for reality. That's the goal.
2:57
Let's start at the very beginning of the design phase.
2:59
Right before we even touch a calculator, we have to look at
3:01
where our numbers actually come from.
3:03
Because treating a failure database like a universal truth
3:07
is, well, it's like trusting a car's estimated Hwy. miles per
3:12
gallon rating while you're driving up a mountain towing a
3:14
boat in an absolute Blizzard. Yeah, the environment and the
3:18
operator change the fundamental reality of the machine.
3:21
The origin of the data is everything.
3:23
So let's talk about those origins.
3:24
Where do engineers usually get these numbers?
3:27
Traditionally, engineers lean on a few heavy hitter databases.
3:30
These are massive volumes of historical data that have shaped
3:34
reliability engineering for decades now.
3:37
Like the military handbooks? Exactly.
3:39
The granddaddy of them all is the USMILHD BK217.
3:43
This is the famous military handbook for electronic failure
3:47
rates. It uses incredibly complex
3:49
regression models based on, you know, part complexity, junction
3:52
temperature, the operating environment, packaging.
3:54
But what is really fascinating about MIL 217 is that if you
3:57
track its successive issues over the years, the published failure
4:01
rates keep dropping. They get lower and lower.
4:03
Well, my first instinct is to say that's just because the
4:06
technology is getting better, right?
4:08
I mean, manufacturing processes improve, clean rooms get
4:11
cleaner, so the components themselves are genuinely more
4:13
reliable. And that is certainly part of
4:15
it. Better manufacturing and more
4:17
rigorous screening definitely play a huge role.
4:20
However, it is equally likely that they simply reassessed
4:24
their earlier data and realized they were being wildly
4:26
pessimistic. Oh, interesting.
4:28
Yeah, early reliability data often included failures that
4:32
weren't actually the component's fault at all.
4:34
Either way, it demonstrates that this quote UN quote hard data is
4:38
actually a moving target. OK, so that's electronics.
4:41
What about mechanical? Stuff moving away from
4:43
electronics, you have NPR D-95. That's the non electronic Parts
4:47
Reliability data book. This is basically the Bible for
4:51
mechanical, hydraulic, and pneumatic parts.
4:53
And how does it differ from the Military Handbook?
4:56
Well, unlike MIL 217, which leans really heavily on
4:59
mathematical models and theoretical stress formulas, NPR
5:03
D-95 is grounded entirely in actual field data.
5:07
It simply logs the number of recorded failures against the
5:10
millions of operating hours in various real world environments.
5:13
OK. And then there is a Retta,
5:15
right, the offshore reliability data.
5:16
Yes, a retta is a big one. When I look at the stakes
5:19
involved with a Retta, it really puts everything into perspective
5:22
for me because this database is managed by a consortium of
5:25
massive oil companies. We are talking BP, Shell State
5:29
Oil and they are tracking safety related equipment in some of the
5:32
most brutal total environments on earth.
5:34
Offshore oil rigs are no joke. Exactly.
5:36
Firing gas detection systems, emergency shutdown valves,
5:40
massive industrial compressors. If a part fails in an ARETTA
5:43
data set, it's not a minor inconvenience, It's potentially
5:46
A catastrophic environmental disaster or a loss of human
5:49
life. Right, and those extreme stakes
5:51
dictate the rigor of the data collection.
5:54
Aretta is incredibly specific because the oil and gas industry
5:57
absolutely demands it, but the modern engineer rarely just
6:00
looks at one single source anymore.
6:03
Use aggregators. Exactly The source material we
6:05
are examining today heavily references fared at .3.
6:10
Think of this as a massive synthesis engine.
6:13
It aggregates over 20 different data sources, including the ones
6:16
we just talked about, and it provides synthesized ranges of
6:20
failure rates for components. And this is exactly where the
6:23
illusion of precision shatters for the engineer who's just
6:26
trying to build a reliable system.
6:28
Because you open Ferridip where you look up a standard
6:30
component, say a commercial signal transformer or a basic
6:34
hydraulic check valve, you want one number to plug into your
6:37
reliability block diagram. So give me the number.
6:40
Right, right. But instead you see that the
6:42
failure rates from these synthesized sources span one or
6:45
two full orders of magnitude for the exact same component.
6:48
Which is terrifying. A variance of two orders of
6:51
magnitude is staggering. It means one highly respected
6:54
engineering source is telling you a part will fail 100 times
6:57
more often than another equally respected source claims. 100
7:01
times. Yes.
7:02
So for an engineering manager trying to calculate a system's
7:05
mean time in between failures or MTBF to, you know, guarantee a
7:11
warranty period, that level of variance is paralyzing.
7:14
It makes the entire exercise feel like statistical astrology.
7:18
If the baseline data has 100 to 1 margin of error, how can
7:22
anyone justify the cost of adding a redundant safety
7:25
system? Why does this massive
7:27
discrepancy even exist? Are the databases just poorly
7:31
managed? No.
7:32
The databases are managed as well as they can be honestly,
7:35
but they're subject to 4 hidden variables that completely warp
7:39
the data behind the scenes. When you just pull a raw number
7:41
from a column, you are totally blind to these factors.
7:44
OK, let's break these down. What's the first hidden
7:46
variable? The first hidden variable is the
7:48
component source. This refers to the degree of
7:50
screening and quality assurance applied before the part ever
7:53
reaches the field. Makes sense?
7:55
Imagine a standard integrated circuit.
7:57
The one bought off the shelf for a cheap consumer toy relies on
8:00
the exact same underlying silicon technology as the one
8:04
bought for a deep space satellite.
8:05
Right, the silicon is the silicon.
8:07
Exactly, but the satellite part undergoes 100% environmental
8:11
screening, intense thermal cycling and factory burn in
8:15
testing. So the week links the parts with
8:19
microscopic manufacturing defects.
8:21
Yeah, those were already thrown in the trash before the data
8:24
collection clock even started ticking.
8:26
Precisely so. The space agency's data will
8:28
show an incredibly low failure rate, while the toy
8:32
manufacturer's data will show a much higher one, even though the
8:34
fundamental component design is identical.
8:37
OK, that's variable one. What's the second hidden
8:39
variable? The second is circuit
8:40
tolerancing, and this is entirely about the skill and the
8:43
effort of the design engineer who integrated the part, because
8:47
components rarely just explode or snap in half.
8:49
Right, they just age. Yes, their internal parameters
8:51
just drift over time, like a resistor's omic value might
8:55
slowly increase. Now, if the surrounding circuit
8:58
was designed to be highly tolerant, meaning it can still
9:01
function perfectly fine even if that resistor drifts by 10%, no
9:05
failure is ever perceived by the user and nothing gets logged.
9:09
But if a lazy or rushed designer built a brittle, highly
9:14
sensitive circuit, that exact same 10% drift causes the entire
9:18
system to crash. And then?
9:20
The maintenance tech logs it as a quote UN quote resistor
9:24
failure in the database, when in reality the resistor was just
9:27
aging normally. The failure was a brittle system
9:30
design, making a perfectly adequate component look
9:33
terrible. That happens constantly.
9:35
The database doesn't know the circuit was brittle, it just
9:37
tallies another tick mark against the resistor.
9:40
That's wild. OK, what's the third factor
9:42
warping the data? The third is reliability growth.
9:46
This is the positive feedback loop between field experience
9:49
and the manufacturing line. Like iterative improvement.
9:51
Exactly. Say a major telecom company
9:54
tracks field failures of a router.
9:56
They identify a specific weakness in a solder joint, and
9:59
they update their manufacturing process.
10:01
The batches produced in year 3 are inherently vastly more
10:05
reliable than the batches produced in year 1.
10:07
I see where this is going. Right.
10:09
If a database indiscriminately mixes early life cycle data with
10:14
late life cycle data, the resulting average is a
10:17
completely misleading fiction. It represents neither the bad
10:21
early parts nor the good late parts.
10:23
It's like judging a professional athlete's current skill level by
10:27
averaging their stats from when they were a clumsy rookie with
10:30
their stats from their championship year.
10:32
It gives you a number that doesn't actually describe their
10:35
current reality at all. That's a great analogy.
10:37
It completely flattens the reality.
10:39
OK, so we have component source circuit tolerance and
10:42
reliability growth. What is the 4th variable?
10:45
The 4th is perhaps the most insidious, and I'll tell you it
10:48
drives reliability engineers completely crazy.
10:51
It is maintenance philosophy. Maintenance philosophy.
10:54
How does that warp the database? Well, in many complex industries
10:58
when a massive system goes down, technicians do not have the time
11:01
to sit there with a multi meter and find the exact microscopic
11:04
point of failure. The factory floor is losing
11:06
thousands of dollars a minute. Right, the pressure's on to get
11:09
the line moving. Exactly.
11:11
So they use trial and error replacement.
11:12
It's often called shotgun maintenance.
11:14
The system stops, so they just swap out circuit board A.
11:17
They turn it on, still broken. They swap out circuit board B.
11:21
It works. The line is back up.
11:22
Oh man, let me guess. The technician fills out the
11:25
paperwork and and writes down that both Board A and Board B
11:28
failed. Yep, they both get logged as
11:31
replacements, which data scrapers almost always interpret
11:34
as failures. Board A was probably in perfect
11:37
working condition. It was just a victim of
11:40
diagnostic guessing. But now the database has
11:42
artificially inflated the failure rate of Board A.
11:45
In some systems, this artificially inflates failure
11:47
data by absolutely massive amounts.
11:50
So if I am sitting at my drafting table and I know that
11:52
these databases are polluted by diagnostic guessing, brittle
11:56
circuit designs, and vastly different screening processes,
12:00
why do we even open these books? I mean, if the data is this
12:03
muddy, aren't we just guessing? We use them because we have to
12:06
anchor our designs to something, but we have to radically change
12:09
how we view that anchor. A component failing in the field
12:12
is a system level event. It is deeply related to the
12:16
component itself, sure, but a massive proportion of field
12:19
failures are actually due to software edge cases, human
12:22
operators doing things they shouldn't, unforeseen
12:25
environmental interference, or multiple components drifting
12:28
simultaneously in ways the designer didn't anticipate.
12:31
So the data is just. The data from Mill to 17 or
12:35
Erada is a guide. It is a relative indicator of
12:38
risk. It is not an absolute, legally
12:40
binding guarantee of how your specific design will behave in
12:44
the real world. It's a compass, not Agps with
12:47
turn by turn directions. It points you toward N, but you
12:50
still have to navigate the rocks yourself.
12:52
That is the perfect way to look at it.
12:53
It tells you the relative contribution of different parts
12:55
to your total failure risk. It tells you that a complex
12:58
mechanical relay is inherently riskier than a solid-state piece
13:02
of silicon. But taking a raw number from a
13:04
table and treating it as a sacred promise to your customer
13:07
is a fundamental, dangerous error.
13:09
OK, let's say we accept that reality.
13:11
We accept that the data exists in these massive, terrifying
13:14
ranges. We are looking at a factor of 50
13:17
to one between the highly optimistic telecom data and the
13:21
deeply pessimistic military data, but at the end of the day,
13:24
I still have to pick a single number to plug into my
13:27
reliability block diagram to see if the system works.
13:30
You do. And this brings us to a massive
13:32
mathematical trap that catches so many engineers off guard.
13:36
The averages trap. It is a classic trap, and it
13:38
stems from a basic misunderstanding of how human
13:41
intuition fails when dealing with statistical orders of
13:44
magnitude. Let's set up the scenario.
13:46
I'm looking at my database range.
13:49
The most optimistic source says my hydraulic valve had a failure
13:52
rate of 0.1 per million hours, and the most pessimistic source
13:56
says it's 1.0 per million hours. My intuition, and honestly the
14:00
intuition of almost anyone who has ever taken a basic math
14:03
class, is to just split the difference.
14:04
Take the arithmetic mean. Which is what everyone does at
14:07
first. Right.
14:07
You had 0.1 and 1.0, you get 1.1 / 2 and you get an average
14:12
failure rate of 0.55 per million hours.
14:17
It feels fair, it feels safe. It feels logical right up until
14:20
you look at what that arithmetic mean actually represents
14:23
structurally. Think about the numbers.
14:26
Your arithmetic mean of 0.55 is more than five times larger than
14:31
the optimistic lower estimate, but it is only about half of the
14:34
pessimistic upper estimate. Wow, OK.
14:37
The center of gravity is completely skewed.
14:39
It is heavily biased toward the larger number, yes.
14:43
When your data range spans an order of magnitude or more, the
14:46
larger value absolutely dominates the arithmetic mean.
14:50
By using simple addition and division, you are essentially
14:53
allowing the worst case scenario to drag the entire prediction
14:57
almost entirely into its own territory.
14:59
So you aren't really finding the middle.
15:01
Not at all. You are no longer designing for
15:03
the average case, you are accidentally designing for a
15:05
slightly mitigated worst case scenario.
15:07
So what is the mathematically correct way to find the middle
15:09
ground when the extremes are so far apart?
15:12
You have to use the geometric mean.
15:14
Instead of adding the 2 numbers and dividing by two, you
15:17
multiply the 2 numbers together and take the square root.
15:21
OK, so sqrt 0.1 * 1.0 gives you a geometric mean of roughly
15:27
0.316. Exactly.
15:29
That feels like a subtle difference, 0.316 versus 0.55,
15:35
but let's breakdown why 0.316 is a vastly superior representation
15:40
of reality. It is superior because of the
15:42
geometric mean averages the orders of magnitude rather than
15:46
the raw linear values. Look at the proportional
15:49
relationship. 0.316 is roughly 3 times larger than the optimistic
15:54
estimate of 0.1 OK, and the pessimistic estimate of 1.0 is
15:58
roughly 3 times larger than 0.316.
16:01
It's it's perfectly in the proportional middle of the
16:04
multiplier spectrum. Oh I see.
16:05
It balances the ratio on both sides.
16:07
Exactly. When you are dealing with
16:09
failure rates, which inherently pave logarithmically across
16:12
different stress environments and quality levels, the
16:14
geometric mean provides the most credible, balanced and
16:18
defensible baseline. This makes so much sense, and
16:21
it's exactly how the Faraday .3 database is structured to help
16:25
engineers. It doesn't just give you the
16:27
wild extremes, it provides data in three distinct columns, the
16:32
minimum reported rate, the maximum reported rate, and the
16:35
geometric mean. Which is incredibly helpful.
16:38
Right, because if you are designing a system and you have
16:41
no specific compelling reason to assume it will be abused in an
16:45
extreme environment, or conversely pampered in
16:49
hermetically sealed clean room, the geometric mean is your most
16:52
mathematically sound starting line.
16:54
It is the only rational starting point, and once you have that
16:57
geometric mean baseline, you then apply specific multipliers
17:01
based on the reality your product will actually face in
17:03
the field. The databases provide these
17:05
modifiers. Give me an example of a
17:07
modifier. Let's look at quality
17:08
multipliers. If you procure a standard
17:11
commercial off the shelf item, your multiplier might be 2.0.
17:14
You take your geometric mean and double it, acknowledging the
17:17
risk of lower quality control. Makes sense?
17:20
But if you pay the premium to have those exact same items
17:23
subjected to 100% screening and factory burn in, your multiplier
17:27
might drop to 0.4. You are drastically shrinking
17:31
your expected failure rate because you've paid to weed out
17:34
the weak links. And the environmental
17:36
multipliers are even more extreme, aren't they?
17:38
Oh, absolutely. If your electronic component is
17:40
going to live its life in a dormant situation, say sitting
17:44
on a warehouse shelf or idling in a backup system, the
17:47
multiplier is 0.1. You shrink the baseline failure
17:51
rate to 1/10. Because it's practically asleep.
17:54
Right, but if that exact same component is destined for a
17:57
mobile ground environment like being bolted to the rattling
18:00
chassis of a military Jeep driving over dirt roads, the
18:03
multiplier is a massive 4.0. You are quadrupling the baseline
18:09
rate just based on the vibration and thermal shock.
18:12
Exactly. You tailor the geometric mean to
18:14
your specific reality. OK, I want to push back on this
18:16
a little bit, or at least channel the anxiety of a lead
18:19
engineer who is terrified of a product recall.
18:23
Bring it on. If I am designing a mission
18:25
critical system, say a medical device or an aviation control,
18:29
shouldn't I just use the absolute maximum failure rate
18:33
from the database? I mean if I use the geometric
18:36
mean, aren't I designing for a quote UN quote average day when
18:40
my ethical responsibility is to guarantee safety on the worst
18:43
possible day? That is a very noble, very
18:46
common intuition, and if you follow it, you will destroy your
18:49
own project. Really.
18:51
Destroy it. Completely.
18:52
Think about it. If you take the absolute worst
18:55
case pessimistic failure rate for every single resistor,
18:58
capacitor, valve and seal in a complex system, and you sum them
19:01
all up, your predicted system level failure rate will be so
19:05
impossibly astronomical that the system will look entirely
19:09
unviable on paper. Oh, the math will tell
19:11
management that the machine is guaranteed to explode the second
19:14
you turn it on. Precisely.
19:15
And to counter that fake mathematical crisis, you will be
19:18
forced to add ridiculous convoluted levels of redundancy.
19:21
You will add backup pumps for your backup.
19:23
Pumps, which adds weight and cost.
19:25
Immense weight, staggering cost, manufacturing complexity, and
19:29
ironically, it introduces entirely new unforeseen failure
19:34
modes. You end up over engineering the
19:37
system to death because you modeled a fantasy where every
19:40
single component was simultaneously experiencing its
19:43
worst possible Day in History. You essentially build a fortress
19:47
when you only needed a storm shutter.
19:49
Beautifully said. You use the geometric meme
19:51
because it represents the central tendency of a
19:53
component's behavior across a massive normal distribution of
19:57
operational lives. And you just scale up the parts
19:59
you know will suffer. Right.
20:01
If you know for a fact that a specific subsystem is genuinely
20:04
going to be pushed to its absolute thermal limits, you use
20:07
the environmental multipliers to scale that specific subsystem
20:10
up. But you never assume the maximum
20:12
published database value as your starting baseline, because, as
20:15
we discussed earlier, those maximum values are heavily
20:18
polluted by diagnostic replacements and brittle legacy
20:21
designs. OK, that makes perfect sense.
20:23
We have successfully navigated the database minefield.
20:26
We know how to pick a realistic baseline using the geometric
20:28
mean and targeted multipliers. We have our prediction on paper.
20:32
Easy part is done. Right, because paper doesn't
20:35
survive contact with the real world.
20:37
Eventually you have to build a physical prototype, put it on a
20:40
test bench, and prove your numbers.
20:42
And this is where we run into a psychological and statistical
20:45
trap that ruins careers. Let's talk about the zero
20:48
failure fallacy. Oh, this is a big one.
20:50
This is the arena where statistical inference clashes
20:53
violently with common sense. Human intuition is terrible at
20:57
statistics, and nowhere is that more obvious than in physical
21:00
reliability testing. Let's lay out the absolute basic
21:03
math 1st to get a point estimate of a system's meantime between
21:08
failures. In TBF, the formula is
21:09
wonderfully simple. MTBF is simply the total
21:12
accumulated test hours divided by the total number of observed
21:15
failures. Very straightforward.
21:17
So if your company builds 10 prototype hydraulic pumps, and
21:21
you run each of them on a test bench for 1000 hours, you've
21:24
accumulated 10,000 total test hours.
21:27
If two of those pumps break during the test, your math is
21:30
10,000 / 2. Your point estimate MTBF is 5000
21:34
hours. That is the raw point estimate.
21:37
But here's the critical flaw in treating that number as gospel.
21:40
OK, if you walk back into the factory, grab 10 different pumps
21:44
from the exact same manufacturing batch, and ran
21:47
that exact same thousand hour test again, you almost certainly
21:50
wouldn't get exactly 2 failures at the exact same time.
21:53
Right. You might get one failure, you
21:55
might get 4 failures, you might get none.
21:57
Because of random sampling and the inherent chaos of physical
22:00
variants, the only way to find the true absolute MTBF of a
22:06
specific batch of pumps would be to literally run every single
22:10
pump you manufactured until it physically broke.
22:12
Yeah, which is obviously a terrible business model because
22:15
then you have zero products left to sell to your customers.
22:18
They're all breaking in your lab.
22:20
Exactly, which is why the entire field of reliability relies on
22:24
statistical inference. We test a tiny sample, and we
22:28
attempt to make a definitive statement about the entire
22:30
untested population. But, and this is the rule that
22:33
gets ignored, that statement must absolutely always be
22:36
accompanied by a confidence level.
22:37
And this brings us to the 2 main types of tests, time truncated
22:41
and failure truncated. Let's define those real quick.
22:44
Sure. If you decide beforehand we're
22:47
going to run these pumps for exactly 1 month and then shut
22:50
them off regardless of what happens, that is a time
22:53
truncated test. If you decide we're going to run
22:55
these pumps indefinitely until exactly three of them break and
22:58
then we stop, that is a failure truncated test, right?
23:02
But let's look at the absolute most dangerous, intoxicating
23:04
scenario for an engineering team.
23:07
What happens when you test a component?
23:09
You get absolutely 0 failures. It is the moment every team
23:12
cheers, and it is the moment the reliability engineer starts to
23:15
panic. Because the math breaks, Yeah.
23:17
Say we run our ten prototype UMS for 1000 hours each, 10,000
23:21
total hours. None of them break, They run
23:23
perfectly. We plug that into our simple
23:25
MTBF formula. Total hours 10,000, number of
23:29
failures zero. 10,000 / 0 is well, mathematically it's
23:33
undefined, but conceptually it's Infinity.
23:36
Does that mean the pump has an infinite MTBF?
23:38
Have we invented a machine that defies entropy and will last
23:41
forever? Of course, not every engineer
23:44
knows it won't last forever. But without a firm grasp of
23:47
statistical tools, management often looks at a zero failure
23:51
test report and assumes they have a near perfect bulletproof
23:55
part. They assume the risk has been
23:57
completely eliminated. It reminds me of someone testing
24:00
a bulletproof vest by firing exactly 0 bullets at it,
24:05
observing that there are no holes and declaring it
24:07
impenetrable, or flipping a coin 0 times, watching it sit
24:12
perfectly still on the table and assuming it will never ever land
24:16
on tails. You haven't proven that the part
24:18
won't fail, you simply haven't observed the inevitable failure
24:21
yet. Your sample size and time is
24:23
just too small. That's spot on.
24:25
To interpret a test with 0 failures or any incredibly small
24:29
number of failures, assuming we're dealing with a constant
24:31
failure rate, we have to leverage the chi square
24:34
distribution. The chi square test.
24:36
Yes, the Chi square test is a statistical tool that allows us
24:39
to calculate the confidence limits of our MTBF based on the
24:42
quote UN quote degrees of freedom present in our test
24:45
data. OK, degrees of freedom is one of
24:48
those textbook terms that makes people's eyes glaze over.
24:51
Let's ground it in the context of MTBF testing.
24:55
The number of degrees of freedom is essentially a measure of how
24:58
much actual failure data you captured.
25:01
The standard rule is that the degrees of freedom is simply
25:04
twice the number of failures you observed.
25:06
Correct. If you observe 3 real failures,
25:09
you have 6° of freedom to work with in your statistical tables.
25:12
But what happens when you have 0 failures?
25:14
2 * 0 is 0. Right, and you can't have 0° of
25:17
freedom and still do the math if the test is time truncated,
25:21
meaning you just ran it to a predetermined clock limit and
25:24
stopped without seeing anything. Break statistics forces a
25:27
penalty on you a. Penalty.
25:29
Yes, you have to artificially add failures to the math to
25:33
calculate the lower limit of your MTBF.
25:35
Specifically, the degrees of freedom is calculated as two
25:38
times the number of failures, +2.
25:40
So 2 * 0 is 0 + 2 is 2. Why does the math force you to
25:44
add these phantom failures? It takes into account the harsh
25:47
statistical reality that had you kept the test running for just a
25:51
few more hours, or even a few more seconds, that first failure
25:54
might have finally occurred. You stop the clock, so you have
25:57
to assume the worst about what was just around the corner.
26:00
Let's walk through the example from our source material to see
26:02
how brutal this penalty actually is in practice.
26:06
Consider a manufacturer who builds 100 electronic control
26:10
units. They put all 100 on a test bench
26:12
for 50 hours. Total accumulated test time is
26:16
5000 hours. 0 units fail. Sounds like a great test.
26:20
It looks like a massive success. The team wants to declare an
26:23
MTDF, but they want to state it with a 60% confidence level.
26:27
They want to be 60% sure the true MTBF is at least a certain
26:31
number. OK, let's do the actual
26:33
translation of the math. The formula for the lower single
26:36
sided confidence limit of MTBF dictates that we take two times
26:40
the total accumulated test time and divide it by a specific Chai
26:43
square value. 1st we have to find that Chai square value.
26:47
Right. Our desired confidence level is
26:49
60%. In statistics, our alpha is 1
26:53
minus the confidence level. So 1 -, 0.6 gives us an alpha of
26:57
0.4. And as we said, because this is
27:01
a time truncated test with 0 observed failures, our degrees
27:05
of freedom is 2. So we go to our standard chi
27:08
square statistical table. We look up the column for an
27:11
alpha of 0.4. We drop down to the row for two
27:14
degrees of freedom. The table gives us a value of
27:16
roughly 1.83. Now we take that value and apply
27:20
it to our test data. We take two times the total test
27:22
times of 2 * 5000 gives us 10,000.
27:25
We divide that 10,000 by our Chai square penalty value of
27:28
1.8. Three 10,000 / 1.83 gives us an
27:31
MTBF of roughly 5464 hours. I want every engineer listening
27:36
to pause and let the reality of that math sink in.
27:39
You tested a massive batch of 100 components.
27:42
You ran them for 50 hours. Not a single one broke or even
27:45
hiccup. A perfect run.
27:46
Your intuition is screaming that this is an incredibly reliable
27:50
product, but the cold hard statistics dictate that even at
27:55
a relatively weak 60% confidence level, you have only legally
27:59
proven an MTBF of about 5400 hours.
28:03
The math essentially took your 10,000 accumulated hours and cut
28:07
your proven lifespan in half just to be safe.
28:10
It's a massive haircut, and if you change the parameters the
28:13
reality gets even harsher. Suppose your massive government
28:17
contract legally required you to prove an MTBF of 20,000 hours.
28:21
Before they cut the check. You run your 5000 hours 0
28:25
failure test. What statistical confidence do
28:28
you actually have that you've met their 20,000 hour
28:30
requirements? Oh wow, we can work the formula
28:32
backward to find out. Exactly.
28:33
You take two times the total test time 10,000 and divide it
28:36
by the required MTBF of 20,000. That gives you a required Chai
28:40
square value of 0.5. So you take the 0.5, you go back
28:44
to the Chai square table for two degrees of freedom and you find
28:47
the corresponding alpha is about 0.61 -. 0.6 gives you a
28:51
confidence level of 40%. Yes, 40%.
28:54
You ran a perfect zero failure test.
28:56
Yet statistically you can only be 40% confident that your
29:00
product actually meets the 20,000 hour requirement.
29:03
More likely than not, you haven't met it.
29:06
That exposes the absolute illusion of 0 failures.
29:09
And the incredible danger of small sample sizes.
29:13
It takes a staggeringly massive amount of accumulated test hours
29:17
to demonstrate high reliability with high confidence.
29:20
Which flows perfectly into the next massive hurdle in
29:23
reliability engineering. Because since a company cannot
29:26
afford to run a test bench for 50 years without going bankrupt,
29:29
how do a manufacturer and a buyer actually sit across a
29:33
table and agree that a part is reliable?
29:35
It's a huge negotiation. How do you?
29:37
Negotiate a test that satisfies both parties.
29:40
This is the battle between producer risk and consumer.
29:42
Risk, right? This is the core of fixed time
29:44
demonstration tests, and it is a delicate negotiation of
29:47
statistical exposure. Whenever you test a small sample
29:51
to judge a massive population, someone is taking a risk.
29:54
Let's define the producers risk first.
29:56
The producers risk is a manufacturer's nightmare.
29:59
It is the mathematical probability that a perfectly
30:02
good factory batch of components, A batch that
30:05
genuinely meets the required MTBF, will just happen to yield
30:09
a few unlucky pessimistic samples that fail the test,
30:13
causing the entire good batch to be unfairly rejected by the
30:16
buyer. You built 1,000,000 great pumps,
30:19
but the five you randomly pulled out of the crate for the test
30:21
bench happened to have minor defects and the buyer cancels A
30:25
multimillion dollar contract. Exactly.
30:27
It's devastating. Conversely, the consumers risk
30:29
is the buyer's nightmare. It is the probability that a
30:32
genuinely terrible, defective batch of components happens to
30:36
yield a deceptively optimistic sample that sails through the
30:39
test without a hitch, causing the buyer to accept garbage.
30:42
You buy 1000 safety valves that pass the lab test, you install
30:45
them in your chemical refinery, and they all start failing in a
30:48
week. To explain how insanely tricky
30:51
it is to balance these two risks, our source material uses
30:54
a brilliant analogy. I want to expand on this bad egg
30:57
scenario because it perfectly illustrates the trap.
31:00
Let's say you were managing a high end restaurant and you
31:03
consider an acceptable failure rate from your farm supplier to
31:06
be 1 bad egg in every 10,000. That is the baseline you agree
31:10
on. OK, one in 10,000.
31:11
To test this quality without breaking every single egg, you
31:15
decide to randomly sample six eggs from the delivery every
31:18
week. If you get 0 bad eggs in your
31:20
sample of six, you pay the supplier and accept the batch.
31:24
On the surface, to a Lehman, it seems like a perfectly
31:27
reasonable spot check. If the true underlying failure
31:30
rate is indeed one in 10,000, the statistical probability of
31:33
passing that 6 egg test is incredibly high.
31:36
It sits at about 99.9%. So the producers risk is tiny.
31:41
The farmer is happy because good eggs will almost always pass.
31:44
But let's say a disease hits the farm.
31:46
The quality plummets overnight. The failure rate skyrockets from
31:50
one in 10,000 to 1 bad egg in every 1000.
31:53
The product is now 10 times worse than what you
31:56
contractually agreed to buy. What happens to the consumers
31:59
risk? You are the restaurant manager.
32:02
If you calculate the probability of pulling 0 bad eggs in a
32:06
random sample of 6 when the true reality of the batch is 1 bad
32:10
egg per thousand, the math reveals that you will still pass
32:15
the test over 99% of the time. That is mind blowing.
32:19
The suppliers product degraded by a factor of 10 and your
32:22
weekly test is still giving them a passing grade 99 times out of
32:26
100. You are serving bad eggs to your
32:28
customers and you don't even know it because your test is
32:30
blind to the shift in quality. It's completely blind.
32:32
This is the fundamental problem of discrimination in statistical
32:35
testing. A simple short duration 0
32:38
failure test on a small sample is a mathematically terrible
32:42
discriminator. It happily passes excellent
32:44
batches, but it also happily passes terrible batches.
32:48
It lacks the statistical resolution to tell the
32:49
difference. So how do you fix it?
32:51
In order to protect the consumer, to ensure that the
32:53
probability of acceptance drops sharply the moment the failure
32:56
rate rises, you cannot rely on a tiny 0 failure sample.
33:00
You have to vastly increase both the test time and the allowed
33:03
number of failures. So to actually catch the drop in
33:06
egg quality, you can can't just test six eggs and look for 0
33:09
failures. You might have to test 6000 eggs
33:12
and mathematically allow for, say 2 failures to establish a
33:16
tight confidence interval. Exactly.
33:18
You need volume to create discrimination.
33:20
The source material points out just how punishing this reality
33:23
is for high reliability industrial systems.
33:26
How punishing are we talking? Well, if you want to demonstrate
33:29
a one year MTBF with acceptable balanced risks, say a 10%
33:34
producer risk of rejecting a good batch and a 10% consumer
33:38
risk of accepting A batch that is 3 times worse than promised,
33:41
you might have to put a single item on a test bench for three
33:44
entire years. Three years.
33:46
No engineering firm on earth has three years to test a single
33:51
component before bringing it to market.
33:53
By the time the test finishes, the technology would be
33:56
completely obsolete. Your competitors would have
33:59
already released 3 new versions. So how do we escape this time
34:02
trap? Is there a shortcut?
34:03
The shortcut is a brilliant statistical method called
34:06
sequential testing. Instead of rigidly fixing the
34:09
total test time and the target number of failures in advance,
34:13
you accumulate test hours dynamically and track the
34:15
results on a specialized graph. OK, how does the graph work?
34:18
The graph features 2 parallel diagonal lines stretching upward
34:22
and to the right. These lines create 3 distinct
34:25
zones on paper, an accept zone at the bottom right are reject
34:29
zone at the top left and a continue testing zone right down
34:32
the middle between the lines. OK, let's visualize this.
34:35
I'm the tester. I have my graph.
34:37
I plot accumulated test hours on the horizontal X axis.
34:40
Every time the clock ticks, my pen moves to the right.
34:42
But every time a component actually fails on the bench, my
34:45
pen steps up 1 unit on the vertical Y axis.
34:48
Correct. As long as your jagged plotline
34:51
stays trapped in that middle corridor between the two
34:54
parallel boundaries. The math says you don't have
34:57
enough confidence yet. You must continue testing.
35:00
So you just keep running the batch.
35:01
Right. However, if your components are
35:04
incredibly reliable, your pen keeps moving horizontally to the
35:08
right without stepping up. Eventually, your line crosses
35:11
the lower diagonal boundary into the accept zone.
35:14
The moment that happens, you stop the test and accept the
35:17
batch. You pass early.
35:18
Oh, that's great. And conversely, if the batch is
35:20
terrible and you get a rapid cluster of failures right out of
35:23
the gate, your pen shoots vertically, crosses the upper
35:27
boundary into the reject zone, and you immediately fail the
35:30
batch and stop testing. That is incredibly elegant.
35:33
It essentially acts as a statistical triage.
35:35
It saves immense amounts of time because if a batch is
35:38
spectacularly good or horrifically bad, the test
35:42
recognizes the extreme trend quickly and shuts the whole
35:45
thing down. You only get stuck in a long
35:47
drawn out test if the batch is borderline and the mass
35:50
genuinely needs more data to decide.
35:52
Yes, decades of military and industrial experience show that
35:56
sequential testing is significantly faster on average
36:00
than equivalent fixed time tests.
36:03
But, and I must stress this caveat, statistical
36:06
demonstration of any kind, whether fixed or sequential, can
36:09
still hit a brick wall of unreality when dealing with
36:12
truly highly reliable systems. Like what if you are designing a
36:17
deep sea submarine valve or a satellite guidance system where
36:20
the required MTBF is 50 years? You simply cannot demonstrate
36:24
that statistically in a laboratory environment.
36:27
The math demands more hours than you have in a human lifetime.
36:30
Which raises a very uncomfortable question for
36:32
anyone managing an engineering team.
36:35
If you are setting U hysical tests for your new product, are
36:38
you designing tests that actually prove the reliability
36:40
of your system or are you just designing a test that is easy to
36:43
pass so you can check a compliance box and ship the
36:45
product? That's $1,000,000 question.
36:47
Because if you are just testing 6 eggs, you aren't proving
36:50
anything, you're just creating a paper shield to hide behind when
36:53
things break. That is the harsh reality of the
36:55
industry. You must understand the
36:57
operating characteristic curves of your specific test plan to
37:00
know what level of bad quality you are secretly agreeing to
37:04
accept. But this entire intense
37:07
discussion we've just had Chai square math, the Poisson
37:10
distribution for 0 failures, the parallel lines of sequential
37:13
testing, Every single bit of it has been hiding a massive,
37:17
glaring, dangerous assumption. The elephant in the room
37:21
Everything we just talked about strictly relies on the
37:23
assumption that failure rates are constant.
37:25
Yes, it assumes that components fail completely randomly, like a
37:28
lightning strike or the radioactive decay of an isotope.
37:31
It assumes the part has absolutely no memory of how old
37:34
it is. But we live in a physical,
37:35
physical universe governed by friction and entropy.
37:38
Things wear out. Rubber seals degrade and crack.
37:41
Steel bearings lose their lubrication and grind themselves
37:44
to dust. Aluminum wings experience
37:46
microscopic metal fatigue every time they flex.
37:49
A mechanical pump absolutely do not have a constant random
37:52
failure rate over a 20 year lifespan.
37:55
It gets physically weaker every single day.
37:57
Which shatters the constant failure rate assumption entirely
38:01
and brings us to the most famous concept in reliability
38:04
engineering, the bathtub curve. The bathtub curve.
38:09
If you track the failure rate of a large population of items over
38:12
their entire lifespan, the resulting graph often looks like
38:15
a cross section of a bathtub at time 0, right off the assembly
38:20
line. The failure rate is actually
38:21
quite high, but it rapidly decreases.
38:24
This downward slope is called infant mortality.
38:27
This is the phase where all the manufacturing screw ups reveal
38:30
themselves. The bad solder joints, the
38:32
microscopic cracks in the plastic, the improperly torqued
38:35
bolts, the parts that were destined to die young fail
38:39
almost immediately. Yes, once those weak links are
38:41
weeded out of the population, the curve flattens out.
38:44
You hit the bottom of the bathtub.
38:45
This is the useful life period here.
38:47
The failure rate is relatively low and for a time constant
38:51
failures in this flat zone are genuinely random.
38:53
A sudden massive power surge and operator dropping a wrench on
38:56
the machine and act of God. But nothing lasts forever.
39:00
No, it doesn't. Eventually you reach the far
39:02
side of the bathtub. The curve begins to rise slowly
39:06
at first and then incredibly steeply.
39:09
This is the wear out phase. The physical degradation finally
39:12
catches up. So if a failure rate is actively
39:15
changing over time, decreasing at the start, flat in the
39:19
middle, and exponentially increasing at the end, how on
39:23
earth do we model it? Our chi square equations from
39:26
earlier are completely useless here because they assume a flat
39:29
line. Enter Wolodi Weibull, a Swedish
39:31
engineer who in 1951 presented A mathematical distribution that
39:35
revolutionized the field. The relationship between failure
39:38
rate and time can almost always be described by the Weibull
39:41
formula. And super flexible, right?
39:42
It is incredibly versatile because it uses up to three
39:45
flexible parameters to bend and fit almost any shape of data
39:48
curve. There is the location parameter,
39:51
denoted by gamma, which simply shifts the entire curve left or
39:54
right, establishing the starting time zero.
39:57
There is the scale parameter denoted by ETA, which acts
40:00
somewhat like an MTBF, stretching or compressing the
40:04
curve along the time axis. But the absolute star of the
40:08
show, the parameter that tells you the true story of your
40:11
component, is the shape parameter.
40:13
The shape parameter to noted by the Greek letter beta.
40:16
If I understand this correctly, beta is the metric that tells
40:19
you exactly where you were sitting in the bathtub.
40:21
That is the perfect way to visualize it.
40:24
If you gather your failure data, run the math and find that your
40:27
beta equals exactly 1.0, it means your failures are purely
40:32
random. The failure rate is perfectly
40:34
constant. The Weibull distribution
40:37
gracefully mathematically reduces the simple exponential
40:40
distribution we used earlier for chi square.
40:42
You are sitting safely in the flat bottom of the bathtub.
40:45
So logically, if a beta of exactly 1 means a constant rate,
40:49
what does it mean if beta drops below 1, say 0.5?
40:52
If your beta is less than one, the math is telling you that
40:55
your failure rate is actively decreasing over time.
40:58
You are sliding down the left side of the bathtub.
41:00
You are trapped in the infant mortality phase.
41:02
Meaning you have bad parts. Exactly.
41:05
Physically, this implies you have severe quality control
41:08
issues on your manufacturing line.
41:10
You are shipping defective units.
41:13
The standard engineering response to a beta less than one
41:16
is to implement a strict factory burn in period.
41:19
You run the components in the factory for a week so that all
41:22
the early failures happen on your time, not the customer's
41:25
time. And conversely if beta is
41:27
greater than one. If beta is greater than one,
41:29
your failure rate is increasing. You are climbing the steep right
41:32
side of the bathtub. Your components are experiencing
41:35
active wear out, fatigue, or material degradation.
41:38
OK, so like a gear grinding down.
41:40
Right. A beta of, say, 1.5 indicates a
41:43
slowly increasing failure rate, perhaps due to gradual
41:46
mechanical wear on a bearing. But a beta of three 3.0 or 4.0
41:51
indicates a very rapid, highly pronounced wear out mechanism.
41:54
It means that almost all of your components are going to survive
41:57
for a while, and then they are all going to suddenly fail at
41:59
roughly the exact same age. OK, here is where it gets really
42:02
interesting for the people actually doing the work.
42:05
Conceptually, beta is brilliant, but practically, how do we
42:08
actually extract beta from the real world?
42:12
An engineer has a cardboard box full of broken UMS an A
42:15
clipboard with a list of the exact dates and times that each
42:18
failed. How do we turn a list of dates
42:21
into a viable beta Arameter? Historically, before the era of
42:25
ubiquitous computing, this was done visually using graphical
42:28
probability plotting. Engineers used highly
42:32
specialized double logarithmic graph paper.
42:35
You plot the cumulative percentage of failures on the Y
42:37
axis against the time to failure on the X axis.
42:40
There's a massive catch there. You cannot just use raw
42:43
percentages. If you are testing 10 items and
42:46
the very first one breaks, you cannot just put a dot at 10%.
42:49
Right, because saying 10% of the infinite total population has
42:52
failed based on a sample size of one failure is mathematically
42:56
arrogant. You have to adjust for the small
42:58
sample size. So you use statistical median
43:01
ranking tables. These tables provide adjusted
43:04
percentage values that account for the uncertainty of your
43:06
specific sample size. Once you look up your adjusted
43:09
ranks and plot these points on the specialized log log Weibel
43:13
paper, the magic happens. It really is like magic.
43:16
If the underlying physical failure mechanism follows a
43:19
Weibull distribution, those scattered dots will visually
43:22
align to form a perfectly straight diagonal line.
43:25
And the physical steepness of that line.
43:27
The slope is beta. The slope of the drawn line is
43:31
exactly your beta parameter. You can literally measure it
43:33
with a protractor in some setups, and the point where that
43:36
drawn line intersects a specific horizontal reference axis gives
43:40
you ETA, your scale parameter. Once you have those two numbers
43:43
pulled from the paper, you can calculate the true dynamic MTBF
43:47
and confidently predict the probability of survival for any
43:50
given hour in the future. Plotting dots on logarithmic
43:53
paper with a ruler sounds wonderfully tactile, but we live
43:56
in the digital age, nobody's doing this by hand anymore.
44:00
The source material outlines 2 primary computer based methods
44:04
for curve fitting this data, Least squares and maximum
44:08
likelihood when I'm sure we don't just gloss over these as
44:10
magic black boxes. Yeah, they are very different
44:13
mathematical approaches. Least squares is the older,
44:17
simpler computational method. It essentially takes your data
44:20
points and tries to draw a straight line through them by
44:23
measuring the horizontal distance from each actual point
44:26
to the proposed line. And it's square, yes.
44:29
It squares those distance values to remove negative numbers, adds
44:33
them all up, and adjusts the line until that total sum is as
44:36
small as possible. It literally minimizes the
44:39
square of the errors. However, because of how it
44:41
calculates those distances linearly, it tends to
44:44
artificially favor the data points that occurred earlier in
44:47
time. Meaning an early failure might
44:49
skew the slope of the line more than a late failure.
44:53
It's a bit rigid. It can be very rigid.
44:55
The vastly more robust and modern method is called maximum
44:59
likelihood estimation, or MLE. Instead of measuring physical
45:03
geometry on a graph, MLE looks at the pure statistical
45:06
probability. The computer algorithm generates
45:09
millions of hypothetical Weibull lines.
45:12
Millions of them. Millions.
45:13
For each line. It asks the mathematical
45:16
question if this specific line represents the true nature of
45:19
the universe. What is the exact numerical
45:22
probability that we would have observed this specific set of
45:25
broken parts at these exact times?
45:28
It iterates until it finds the one specific line that maximizes
45:32
that total probability. It finds the reality that is
45:35
most likely to have produced the garbage in your cardboard box.
45:39
And Emily is particularly crucial because it can handle
45:41
sensor data, right? Sensor data is the lifeblood of
45:44
real world reliability. A sensor data point represents
45:47
an item that was part of the test, but did not actually fail
45:50
by the time the data was collected.
45:52
Like maybe the test clock ran out and the part was still
45:54
humming along. Exactly, Or maybe a technician
45:57
accidentally dropped the part and broke it, removing it from
46:00
the test for a reason completely unrelated to its internal
46:03
reliability. Lee Scores struggles with this,
46:06
but maximum likelihood beautifully factors in the fact
46:09
that these surviving items existed and endured stress up to
46:12
a certain point. It gives them credit for
46:14
surviving, which provides a much more accurate, less pessimistic
46:17
estimate of true reliability. So let's tie this back to the
46:20
engineer's daily life. If I assume a constant failure
46:23
rate for a mechanical hydraulic pump and I do all my Chai square
46:26
math and I proudly present a slide to my boss declaring a
46:29
5000 hour MTBF. But the physical reality is that
46:32
the pumps internal seals are actively degrading, meaning it's
46:35
true. Weibull beta is 1.5.
46:38
My Chai square presentation isn't just slightly off is it?
46:41
It is completely, dangerously useless.
46:43
It is worse than useless. It is a mathematical lie that
46:47
will cost the company millions. You used a statistical tool
46:51
designed to predict random lightning strikes to model a
46:54
system that is bleeding to death from 1000 cuts.
46:57
Your Chai square calculation assumes that the pump is just as
47:00
healthy and robust at our 4000 as it was at our one.
47:03
But it's not. Not at all.
47:05
A true beta of 1.5 guarantees that the pump is vastly more
47:09
fragile and likely to fail at our 4000 because the seals have
47:12
been wearing away every single day.
47:14
You're blind to the wear out because you chose the wrong
47:17
equation. This perfectly highlights why
47:19
understanding the physical failure modes is infinitely more
47:22
critical than just crunching numbers in a vacuum.
47:25
Which leads us to one of the most fascinating counter
47:27
intuitive traps in all of engineering analysis.
47:30
This is where data actively lies to you if you don't understand
47:33
the physical context. Let's explore the continuous
47:36
process illusion. This phenomenon is a trap that
47:39
catches even seasoned reliability veterans.
47:42
It is a phenomenal example of how an entire system can
47:45
camouflage its true nature. To explain this, we have to look
47:49
at the filament lamp example from the text light bulbs.
47:52
Let's scale it up to make it vivid.
47:54
Imagine we construct a massive state-of-the-art skyscraper On
47:59
opening day. We install 10,000 identical
48:02
brand new light bulbs throughout the ceilings.
48:04
OK, 10,000 bulbs. Now we know from basic physics
48:07
that an incandescent light bulb does not fail randomly.
48:11
It has a very distinct, well documented wear out
48:14
characteristic. The tungsten filament literally
48:16
burns away over time. It's Weibull beta is usually
48:20
well in excess of 2.0. Correct.
48:22
A light bulb is a textbook wear out item.
48:25
They all survive for a very predictable long period of time.
48:28
Then, as the filaments thin out and become fragile, they start
48:31
failing very rapidly, clustered tightly around their design life
48:35
limit. So in our new skyscraper, for
48:37
the first few months, it's smooth sailing.
48:38
The maintenance crew is bored, maybe a handful of defective
48:41
bulbs, burnout, early infant mortality.
48:43
But as we approach the one year mark, suddenly it's an absolute
48:47
bloodbath. The wear out hits.
48:48
Exactly. Bulbs are burning out by the
48:51
hundreds every single day. The maintenance team is running
48:53
ragged with ladders. If you plotted the failure
48:56
distribution of that original batch, it would look like a
48:59
massive steep bell curve localized right around that one
49:03
year wear out age. But here's where the illusion
49:05
takes hold. The skyscraper doesn't stay
49:07
dark. Every single time a bulb burns
49:10
out, a maintenance worker immediately replaces it with a
49:12
brand new one. Fast forward 3 or 4 years into
49:15
the future, the building no longer contains 10,000 bulbs of
49:19
the exact same age. Right, because of the constant
49:22
staggered replacements. Yes, the building now contains
49:25
10,000 bulbs of wildly different ages, forming a completely
49:28
homogenized mixture from brand new to nearly dead.
49:32
So what happens to the failure rate of the building as a whole
49:35
macro system? Because the individual bald ages
49:37
are completely randomized across the ceilings, the number of
49:40
bulbs failing in any given week stabilizes.
49:43
It becomes approximate constant 50 bulbs fail in January, 48
49:47
fail in February, 52 fail in March.
49:50
To the building manager sitting in the basement looking at a
49:52
spreadsheet of weekly maintenance expenses, it looks
49:55
exactly like a constant failure rate.
49:58
The failures appear completely random and evenly distributed
50:01
over time. And that is the terrifying part.
50:04
A population of physical items governed by a strongly
50:07
increasing failure rate. A massive wear out mechanism
50:11
suddenly appears to the macro observer as a constant random
50:15
failure rate system. If a new reliability engineer
50:18
came into that building, recorded the time intervals
50:20
between maintenance calls, and ran a Weibull analysis on that
50:23
mixed data, the computer would spit out a beta of exactly 1.0.
50:28
The engineer would look at the math and happily conclude light
50:31
bulbs fail purely randomly. They don't wear out, which we
50:35
know is a physical absurdity that violates the loss of
50:38
thermodynamics. This is exactly why I constantly
50:40
warn teams. You must separate your failure
50:43
modes, and you must ruthlessly track individual item histories
50:47
from their specific time 0 not just track system level alarms.
50:50
It's about data hygiene. Yes, if a factory manager is
50:54
replacing rubber pump diaphragms during routine maintenance and
50:58
the text just scribbles diaphragm failed on Tuesday in
51:01
the logbook without recording the exact serial number and
51:04
installation date of the previous diaphragm, the data is
51:07
poisoned. The manager can never run a
51:09
Weibull analysis to see if the rubber is degrading prematurely
51:13
because the timeline is just broken.
51:14
The data is hopelessly mixed. The manager will just see a
51:17
steady, constant trickle of failures on the line and falsely
51:20
assume it is a random, unavoidable fact of life.
51:23
To do real, actionable reliability analysis, you must
51:27
preserve data hygiene. You need the unique identity of
51:30
the item, its exact operational time to failure, or its exact
51:34
accumulated time of replacement if it survived.
51:36
Without that granular discipline, statistical analysis
51:39
isn't just impossible, it is actively misleading.
51:42
OK, we have covered a staggering amount of ground today.
51:45
We tore down the illusion of precision in the databases and
51:48
expose the variables hiding in the margins.
51:51
We navigated the arithmetic mean trap.
51:53
We shattered the zero failure fallacy and felt the sting of
51:56
the Chai square penalty. We explored the brutal reality
52:00
of producer and consumer risks using bad eggs, and we use the
52:03
viable distribution to respect the physical reality of the
52:06
bathtub curve. And finally, we uncovered how
52:09
wear out disguises itself as randomness.
52:12
Which brings us to the ultimate question.
52:14
How do we turn this mountain of theory into a step by step
52:16
practical approach? How do we actually optimize a
52:19
design? And why do our paper predictions
52:21
so rarely match the messy reality of the field?
52:24
This is where we distill everything into an actionable
52:27
philosophy for the drafting table.
52:29
Step one is acknowledging the databases for what they are.
52:32
Use Faradip. Use Mill 217, but respect their
52:35
extreme limits. Never take an arithmetic average
52:38
of extremes. Use the geometric mean to
52:41
establish a mathematically credible baseline that balances
52:44
the orders of magnitude. Then apply your environmental
52:47
multipliers thoughtfully. But always treat this final
52:50
baseline number as a relative indicator of approximate
52:53
capability, not a legally binding contract with the
52:56
universe. Step 2 is defining your failure
52:59
modes. This is arguably the most
53:01
important physical step. The source material hammers home
53:04
the point that predicting a single monolithic total failure
53:07
rate is largely useless if you don't know how the system is
53:09
failing. Exactly.
53:11
Let's look at a critical safety shutdown valve in a chemical
53:14
plant. The database prediction says it
53:16
will fail once every five years. That number alone tells you
53:20
nothing about risk. If 90% of those predicted
53:23
failures manifest as spurious trips, meaning the valve
53:26
accidentally triggers and safely shuts down the plant when there
53:29
is no actual emergency, that is annoying.
53:32
It cost the company money and lost production, but nobody's
53:35
injured. The system failed safely.
53:37
But if 90% of those failures manifest as a failure to
53:41
respond, meaning the reactor is overheating, the alarm is
53:44
blaring, and the valve is physically jammed open, that is
53:47
catastrophic. The plant explodes.
53:49
The system failed dangerously. The total baseline failure rate
53:52
is exactly the same in both scenarios, but the real world
53:55
impact is vastly different. This is why you must perform a
53:58
rigorous failure mode and effects analysis or FMEA.
54:02
You have to take that total failure rate and mathematically
54:04
portion it into specific physical modes.
54:07
Short circuit failed to release blocked port, spurious trip.
54:10
Only by dividing the risk can you design the correct
54:12
protective architecture. Step 3.
54:15
Use your failure rates for relative comparisons, not
54:18
absolute promises. If we accept everything we've
54:21
talked about today, that failure rates are perhaps the least
54:24
precise engineering parameter in existence, subject to massive
54:27
hidden variables, what good is the final MTBF number on the
54:31
data sheet? It is an architectural design
54:34
tool. You do not look at a predicted
54:36
MTBF of 50,000 hours and promised the client 50,000
54:39
hours. You use that calculated
54:41
prediction to make internal engineering decisions.
54:44
You compare design prototype A against design prototype B.
54:48
You ask the math. If I spend the budget to add a
54:51
redundant cooling pump to this subsystem, how much does the
54:54
overall system availability improve relative to the original
54:57
design? The absolute numbers on the
54:59
paper might be off by a factor of 3 compared to reality, but
55:02
the relative percentage contribution of each part to the
55:05
total system risk is usually quite accurate.
55:08
It acts as an X-ray. It highlights where the fragile
55:10
bones are. Even if he can't tell you the
55:12
exact date the bone will snap. It guides your redundancy and
55:15
architecture decisions. It forces you to allocate your
55:18
budget where it actually matters.
55:20
And finally, Step 4, you must realize where the true benefit
55:25
of reliability engineering actually lives.
55:27
Didn't live in the upfront paper prediction does it?
55:30
The prediction is just buying a ticket to the game.
55:32
The primary benefit is reliability growth.
55:34
It is everything that happens after the system hits the field.
55:37
It is the relentless continuing analysis.
55:40
It is meticulously tracking the field data, maintaining perfect
55:44
data hygiene so you can run a proper Weibull analysis,
55:47
realizing your pumps beta is actually 1.5, performing a
55:50
physical root cause analysis on the degraded seals, and issuing
55:54
a design change to use a different polymer in the next
55:56
manufacturing run. True practical optimization is
55:59
an endless feedback loop of design testing, field data
56:02
collection, and corrective action.
56:04
The initial paper prediction just gets you into the ballpark.
56:07
Growth wins the game. So to synthesize this entire
56:10
journey for the engineer listening, we avoid the trap of
56:13
the arithmetic mean because it maliciously biases toward the
56:16
worst case scenario. We rely on the geometric mean to
56:19
respect logarithmic variance. We never ever fall for the
56:23
intoxicating 0 failure fallacy because we know Chai Square
56:27
proves that small sample sizes with 0 failures only guarantee
56:31
mediocre, heavily penalized reliability.
56:34
We deeply respect the producer and consumer risks inherent in
56:37
any physical test, recognizing that demonstrating high
56:40
reliability requires either impossible amounts of time or
56:44
clever sequential testing. And we use the Weibull
56:47
distribution to honor the physical reality of the bathtub
56:49
curve, ensuring we never treat a grinding wearing gear like a
56:53
random bolt of lightning. That is the comprehensive,
56:55
mature approach to the discipline.
56:57
It requires abandoning the warm comfort of binary pass fail
57:00
engineering and fully embracing the nuanced, often uncomfortable
57:03
reality of statistical confidence.
57:05
This has been an incredibly deep dive.
57:07
I feel like we have fundamentally dismantled and
57:10
rewired how we look at MTBF data sheets and testing protocols.
57:14
But before we sign off, I want to leave the listener with a
57:16
final thought, a provocation to Mull over based on everything
57:19
we've unpacked today. We've clearly established today
57:21
that failure rates are the least precise parameter in our
57:23
toolkit, and that statistically demonstrating high reliability
57:27
for critical systems is nearly impossible within a reasonable
57:29
time frame or budget. This raises A profound
57:33
structural question for our industry.
57:35
Are our rigid contractual reliability requirements
57:38
actually stifling true technological innovation?
57:41
If massive organizations demand absolute statistical certainty,
57:45
forcing engineers to mathematically prove decades of
57:48
MTVF on paper before a new system ever sees the light of
57:51
day, are we just forcing teams to design for the test rather
57:55
than designing for reality? Are we trapping ourselves in a
57:58
cycle of using conservative heavy legacy designs just
58:01
because they have historical database entries, while actively
58:04
ignoring superior innovative designs simply because they lack
58:07
the millions of accumulated test hours required to appease the
58:10
Chai Square penalty? That is a brilliant, unsettling
58:13
point to end on. Designing for the test versus
58:16
designing for reality. The friction between innovation
58:19
and statistical fear. If you got something useful out
58:22
of today's episode, hit follow and drop a like.
58:25
It helps grow the show and keeps the engineering knowledge
58:27
flowing.