Latest / MBA Training AI / ADAS perception stacks: why sensor fusion fails first

Transcript

ADAS perception stacks: why sensor fusion fails first · MBA Training AI

0:00
Host: Welcome to the Leaders Insights Podcast.
0:07
Host: Today's episode, how Addis perception stacks turn sensors into real-time driving decisions.
0:12
Host: Every time you tap the brakes in a modern car, a computer already decided you should,
0:18
Host: milliseconds before you did.
0:20
Host: So walk me through what's actually happening in that black box.
0:23
Expert: Right.
0:24
Expert: So Addis, Advanced Driver Assistance Systems, the tech behind lane keeping and automatic
0:28
Expert: braking, is basically a relay race.
0:31
Expert: Raw data comes in from sensors, gets handed off through several stages, and out the other
0:36
Expert: end pops a decision.
0:38
Expert: Break, steer, do nothing.
0:40
Expert: The whole thing runs dozens of times a second.
0:47
Expert: Three main types.
0:48
Expert: Cameras, which see color and text but hate fog and glare.
0:52
Expert: Radar, ooh radio waves that measure distance and speed, works in bad weather, but sees
0:57
Expert: the world as fuzzy blobs.
0:59
Expert: And Lidar, laser pulses that build a precise 3D map.
1:03
Expert: Brilliant but expensive.
1:05
Expert: Each is blind in a different way, which is the whole point.
1:08
Host: Why is that the point?
1:09
Host: Sounds like a weakness.
1:11
Expert: Because you fuse them.
1:13
Expert: Sensor fusion is the stage where you combine those feeds into one picture.
1:17
Expert: The camera says, there's something ahead.
1:20
Expert: The radar says, it's 40 meters away, closing at 30 kph.
1:24
Expert: And together, you get confidence neither had alone.
1:27
Expert: It's like triangulating a rumor.
1:29
Expert: One source is gossip.
1:31
Expert: Three agreeing sources is a fact.
1:33
Host: And if two sensors disagree?
1:35
Expert: Then you've found the hard part.
1:37
Expert: Fusion has to arbitrate.
1:39
Expert: Usually it waits by context.
1:40
Expert: Trust radar for distance.
1:42
Expert: Trust the camera for what the thing actually is.
1:45
Expert: But conflicts are exactly where systems fail.
1:48
Expert: The infamous cases where a car drove into a white truck against a bright sky.
1:53
Expert: Radar saw open road.
1:55
Expert: Radar saw a stationary object and got overruled as probable clutter, like an overpass.
2:00
Expert: Bad arbitration.
2:01
Expert: Fatal outcome.
2:03
Host: So after fusion, what happens to that combined picture?
2:06
Expert: Object classification.
2:07
Expert: A neural network.
2:10
Expert: Software trained on millions of examples to recognize patterns.
2:13
Expert: Labels everything.
2:14
Expert: Pedestrian, cyclist, car, plastic bag.
2:18
Expert: And it assigns probabilities.
2:20
Expert: Never certainties.
2:21
Expert: That's 91% sure that's a person.
2:24
Expert: The system lives in that gap between 91 and 100.
2:27
Host: That gap sounds terrifying at 100 kph.
2:30
Expert: It should.
2:31
Expert: And here's the uncomfortable bit.
2:33
Expert: These models are trained mostly on the situations engineers thought to collect.
2:38
Expert: The edge cases.
2:39
Expert: A kangaroo.
2:40
Expert: A person in a chicken costume.
2:42
Expert: A couch on the highway.
2:44
Expert: Those are the ones that break classification because the networks never seen them.
2:48
Expert: MIT Technology Review has run good coverage on how the long tail of rare events is where
2:55
Expert: autonomy stalls out.
2:57
Host: OpenAI put out numbers on perception model accuracy recently.
3:00
Host: What do those say?
3:01
Expert: They've claimed detection accuracy north of 99% on benchmark datasets.
3:08
Expert: But flag that OpenAI is an AI lab selling models.
3:12
Expert: So that's a vendor figure on their chosen benchmark.
3:14
Expert: Not a wet Tuesday on the M25.
3:17
Expert: Cross check against independent work.
3:20
Expert: O'Reilly radar's write-ups make the point that benchmark accuracy and road accuracy
3:24
Expert: are different animals.
3:26
Expert: Because the real road doesn't send you clean, labeled test images.
3:30
Host: Fine.
3:31
Host: Classification's done.
3:32
Host: The car knows what's around it.
3:34
Host: Then?
3:35
Expert: Decision logic.
3:36
Expert: The final leg.
3:37
Expert: It takes the labeled scene and predicts what everything will do next.
3:41
Expert: That cyclist might swerve.
3:43
Expert: That car might merge.
3:45
Expert: Then plans your response.
3:47
Expert: This is where milliseconds matter most.
3:49
Expert: The full-chain, sensor-to-break command has to finish in roughly 100 milliseconds.
3:55
Expert: Faster than you can blink, which takes about 300.
3:58
Host: Where in that pipeline should an exec worry most?
4:01
Expert: Not classification.
4:02
Expert: That's where the flashy demos live.
4:05
Expert: Worry about fusion and the training data behind it?
4:08
Expert: A system that classifies beautifully but arbitrates conflicts badly.
4:12
Expert: Or was trained on a thin slice of reality, will pass every demo and then surprise you
4:16
Expert: in the rain.
4:18
Host: So what's the one thing they should actually do Monday morning?
4:21
Expert: Ask your vendor for their failure data, not their accuracy data.
4:25
Expert: Specifically, how the system behaves when two sensors disagree, and what edge cases it's
4:30
Expert: never been trained on.
4:32
Expert: If they only show you the 99% and can't talk fluently about the missing 1%, that
4:37
Expert: 1% is your liability.
4:40
Expert: The gap is the product.
4:42
Host: This is for today's episode.
4:43
Host: TechCrunch AI, Ars Technica AI, O'Reilly Radar, KD Nuggets, The Decoder, OpenAI, Vendor,
4:51
Host: AI Lab.
4:52
Host: That's it.
4:53
Host: The AI Decision Tools are live at MBA-training.com.