Latest / Agentic Conversations (formally mlops.community) / How to Build Production-Ready AI Models for Manufacturing // [Exclusive] LatticeFlow Roundtable

Transcript

How to Build Production-Ready AI Models for Manufacturing // [Exclusive] LatticeFlow Roundtable · Agentic Conversations (formally mlops.community)

0:07
What is up good people of Earth? How y'all doing out there?
0:11
I am very excited to be with you Today.
0:15
We are going to be talking all about how to build production
0:19
ready AI models for manufacturing, something that
0:22
has been coming up over and over and over again when I talk to
0:28
people about AI use cases. And so hopefully by the end of
0:32
this, you're going to have some of your questions answered, but
0:35
maybe you're going to have some food for thought and you're
0:37
going to be chewing on some of these questions for a few days
0:40
afterwards. And I want to bring out our
0:44
Esteemed Guesses guests right now, But before we do that, I've
0:50
got a few announcements on the MLS community side of the house.
0:54
In case you didn't know, we do have a podcast that I recommend
0:59
you all listen to. We're going to be putting stuff
1:01
like this and other great episodes onto the feed.
1:06
You can check it out. There's a little QR code for you
1:08
that I'll throw up in the left hand corner.
1:12
But the most fun that we've been doing is ohh, here's a better,
1:18
there's a better QR code for you.
1:20
So hopefully you can check out the podcast there.
1:23
But what I have the most fun with is presenting you the merch
1:30
that we've got. We've got shirts that say I
1:32
hallucinate more than Chad. GT Get yours now.
1:35
It is on limited edition run. And with that, we're gonna bring
1:41
out our guests. Let me get that off.
1:43
Hit that QR code if you want to check out the shirts.
1:45
Otherwise, if you want something a little more tame, you can go
1:49
with my prompts or my special sauce.
1:52
We've got all kinds of fun merch out there.
1:55
And last announcement before we get rolling with this roundtable
2:01
session is on June 21st, June 25th, June 25th in San
2:09
Francisco, we are having our first in person conference.
2:13
If you want to join, hit me up, let me know.
2:16
It's going to be San Francisco June 25th.
2:19
We'll also be streaming one of the tracks onto this platform
2:25
that you are on right now. So we will make sure that you
2:29
are all updated. Enough of the announcements.
2:33
We've got a lot of ground to cover and I want to get right to
2:39
it and bring out our guests. Let's get Paval out here.
2:44
Where you at? And we've also we're also joined
2:47
today by Mohan and N kids. Where you all at A they're they
2:51
are what's going on guys? Hello.
2:54
Hey, the only I do guitar intros, but this time I didn't
3:01
because I thought the QR code of our merch was funny enough to
3:05
have some fun. Now let's jump into the topic at
3:10
hand. I'll lead us through all kind of
3:12
guide us and guide the conversation, but I'm by no
3:15
means an expert in this topic and so I'm here to learn from
3:19
you all. I would love to start with a
3:27
little bit. Ohh, I just realised we're
3:29
missing our our 4th to make it a foursome.
3:32
There he is. Hey, sorry, I didn't mean to
3:34
keep you out. Keep.
3:35
Having no worries. So let's let's start with a
3:40
little bit of like 10 second intros so everybody can know who
3:43
you are and what you're doing and why you are on this virtual
3:48
roundtable stage. And Mohan, since you're big on
3:53
my screen right now, you can start it off.
3:56
Thank you. Thank you.
3:58
It's a pleasure to be here. First of all, thank you for
4:00
having me so in background. I've been working in the
4:05
computer vision slash machine learning AI space for the last
4:09
25 years. About nineteen of those were in
4:13
the manufacturing industry, in the semiconductor manufacturing
4:16
industry. And other than that I've also
4:19
worked in robotics, fintech and insurtech.
4:22
And yeah, that in a nutshell is. Me So.
4:25
However, today my name is Paula Paul.
4:27
Bit less experience than one, so not 25 years, but we've been
4:32
working, I would say probably more than 10 years on the topics
4:36
of safety, robustness of our models, more from the academic
4:40
background that I trade. This is where I did my PhD,
4:43
after which we started company with two other professors that
4:48
tries to bring some of the kind of research and knowledge we
4:52
accumulated out in practise to help actually deploy these
4:56
things in a. Safeway OK, so yeah, yoga and
4:59
vice mega and part of Schneider Electric SA.
5:02
I hop where I'm responsible for strategy innovation.
5:05
I'm hanging around in the field. I'd like moan for the last 25
5:09
years and also we're starting in the space of generate DVI.
5:16
Actually lots of work in LVMS, large vision models, but also
5:21
doing what's in the classically, say today you would say old
5:25
school machine learning stuff, but it's still important to have
5:28
it. And yeah, my teams were working
5:30
on next generation solutions like teaching LGBT time and
5:35
space. Hello, I'm gonna keep I work as
5:39
a vision engineer, e-mail engineer at Altium cells which
5:45
is one of the EV battery manufacturing plant here in
5:50
Ohio. I don't have much experience
5:52
like everybody else here but I did my masters last year and
5:58
been working for almost a year. Well, fellas, when it comes to
6:03
industry and Heavy Industries, AI and ML use cases are they're
6:09
very prevalent, but I always want to know more about them.
6:14
I think there are some common use cases that you have probably
6:18
seen and some uncommon use cases that I want to hear about.
6:23
So any kid, do you want to start us off with what some of the use
6:28
cases that you've been playing around with are?
6:30
Sure. So I mean most of the use cases
6:32
for us, I mean there are multiple spaces where we use AI,
6:38
but especially for us, we use it for battery inspection.
6:42
We do the electrode inspection actually made it's in the this.
6:48
So we do surface level inspection, dimension level
6:51
inspection, but for surface we use the AI for the most part.
6:55
Alright then, what about you Yergin?
6:57
How you been playing with it besides teaching based which
7:03
sounds like a song? I mean I I gotta be honest with
7:05
you that does sounds like something out of a sci-fi novel
7:08
too. But what are?
7:10
What are some other use cases? But they men battery health as
7:14
Ankit satus plays a big role for Schneider as when we are
7:18
manufacturing UPS for data centres.
7:21
So keeping this stuff alive is is super important thing.
7:26
So understanding the cells health during a live, how you
7:30
can manage charge and reach us. So is a huge area of business
7:35
for us. But other big business areas as
7:38
it comes to Heavy Industries is how do you control carbon,
7:42
carbon emissions, carbon reduction.
7:44
So working on carbon digital twins and then incorporating
7:49
this with process optimization and electrification logics and
7:54
machine learning programmes is does a lot to gain.
7:58
And we find out there's lots more to understand because of
8:04
people don't even know where all of the GHG is coming from.
8:07
And so that's this is too big areas are US battery health,
8:12
sustainability, CHE admissions where we do lots of work.
8:16
My view is a bit different. So Aniket, you know, his view is
8:21
in the manufacturing of the cells and you're and you were
8:24
talking of what happens during the monitoring of cells as
8:28
they're used in these data centres and their and their
8:30
light span. My experience has been more than
8:34
the semiconductor manufacturing industry where chips are made.
8:38
And so there are two pieces to this.
8:41
AI is used first in the development of the process
8:44
itself, because that's a very, very difficult journey.
8:48
And that tends to be the hardest set of problems that we solve to
8:52
help the TSMC's and the Intels and the Samsungs of the world
8:56
build their process to figure out what will work when they
9:01
manufacture their next Gen nodes right at the critical dimension
9:05
nodes as Moore's Law scales. And then the other part, which
9:10
is volume production, when once the process is optimised, then
9:14
you're going to volume production.
9:16
And there you're doing much more of a monitoring of the process.
9:19
So you sample some and you make sure that in both of these
9:23
cases, where AI comes in is to discover known knowns, which
9:30
means that you have some kinds of defects and you know that you
9:32
can live with some statistical process control of how many
9:36
defects per wafer or per cell or per 100 cells or whatever.
9:40
Then you've got the known unknowns and then you've got the
9:43
unknown unknowns. And so you've got to be able to
9:46
be able to capture all of these things in the process so that
9:50
the right actions can be taken in the manufacturing process.
9:54
Anything else you want to add before we jump to the first
9:57
question? Ohh, I think the there are a lot
10:02
of small answers and subtleties that we can unpack in some of
10:07
these disciplines, but happy to jump on and get into the
10:11
details. Cool, very cool.
10:14
Well, let's let's get your take on this panel because and maybe
10:19
we can go into some of the challenges that you've seen as
10:23
you've worked with many different use cases and many
10:27
different customers in this field.
10:29
What are they struggling with? Yeah.
10:31
I think maybe to follow up on Mohan.
10:33
So, So what we try to do at lattice flow is kind of be in
10:37
this part of the manufacturing process.
10:39
So this is like once there is already a production line and
10:42
then you would want to set up and quality control on top to
10:47
try to figure out what are the known knowns, known unknowns as
10:50
well as the unknown unknowns. And this is one of the challenge
10:53
where there is a process associated with kind of figuring
10:57
it out. And this is at this point very
10:59
manual. So you would rely on domain
11:01
experts that have bought the machine learning expertise as
11:05
well as the domain knowledge. And this is important because
11:08
often when we work with some of the kind of applications, you
11:12
would, for example, do preventive maintenance, you
11:14
would look at the defects. And if you show them to me, I
11:17
have really no clue like you would show like this is how it
11:19
looks correct. This is how it looks incorrect.
11:22
And I just don't see the difference.
11:24
And there are people who have 10 years of experience who can see
11:27
it from the image or maybe actually you cannot even solve
11:30
it from like one signal. You would need multiple signals
11:33
put together. And this is all the information
11:36
you need kind of also at the design phase to be even able to
11:39
design such system. OK, So the main thing that I I'm
11:45
wondering about when you have these different use cases and,
11:49
and I appreciate that each one of you has a very specific
11:53
piece. I know that depending on the
11:58
machine learning use cases that you do have and what you are
12:01
doing with AI, sometimes you're really trying to optimise for
12:06
certain pieces like latency. And if you're using LM, you
12:10
can't really do that. If you are however, trying to do
12:14
like predictive maintenance and figure out which piece needs to
12:18
go off of the construction line or sorry, like the the line, the
12:24
conveyor line, I can't think of the word right now, but you
12:27
don't necessarily need really low latency for that either.
12:30
So what are some of these constraints and things that you
12:34
are optimising for as you're thinking through your AI
12:38
problems? Are there different pieces that
12:40
you're saying, all right, well, we're when we think about
12:43
trade-offs, we think about tradeoffs like this.
12:46
And maybe you're going all through this one over to you
12:49
first, and whoever wants to jump in afterwards can.
12:52
See it heavily depends on the application.
12:56
When we have ultra low or zero latency applications today in
13:03
specifically in computer vision related use cases where we
13:07
talking total time to make it a Zishan 50 milliseconds inference
13:14
time for AI sub 20 milliseconds because that's what many people
13:19
don't understand is it takes time to acquire an image or
13:22
multiple images that they need to be interpreted.
13:26
You need to send signals to control systems.
13:28
Control systems needs to actuate stuff.
13:31
So people when they hear 50 milliseconds, I am forever 50
13:34
milliseconds for AI. No, we don't have so we have 10
13:37
to 20 milliseconds. So it puts you in in a whole
13:41
world of pain to figure out the pure hardware setup for that.
13:47
It's not as one big area. Then when it comes to other
13:51
scenarios where time is of less importance, its accuracy, how
13:57
precisely can you predict scenarios?
14:00
How accurately can you make distinctions between?
14:03
This is still within the realm of acceptable and this is where
14:08
it starts to be not acceptable. We also seeing lots of safety
14:12
use cases where the safety of human beings plays an enormous
14:17
part area. So somebody enters an area where
14:22
he's not supposed to be. If he doesn't leave the sound
14:24
within a given time frame, you have to stop production.
14:28
And then it's it's for people looks like stop production is
14:32
not a big deal a whole it get the guy out, throw the switch
14:36
and it runs again. And most processes it's not that
14:40
way. It's not you just flip the
14:42
switch and it runs again. It's one of my best examples.
14:45
Take a paper machine. These things are between 4 and
14:49
600 metres long and if you have to stop that machine, it's not
14:54
flipping a switch. You have to clean it, restart
14:57
it, it's ours to get it and running O.
15:01
These are applications now where different types of accuracy
15:06
plays an immediate role. And then this other applications
15:09
where time is of essence and, and then when we going into the
15:15
first big area around generative solutions, uh, what it becomes
15:21
there is can you manage the quantity or the amount of
15:24
hallucinations? So how accurate in how little
15:28
hallucinations you actually produce when you create content.
15:34
And I think more than was speaking about the semiconductor
15:37
industry, we have, for example, a use case around generating new
15:40
geography, which is super important because this is
15:43
literally how you run all the stuff on the chip and you want
15:47
to make it as small as possible and so on.
15:50
So you want to hallucinate ideally 0 and that this is 3 big
15:56
areas which different implications which really bother
16:00
us today. Yeah, Yeah, you're good.
16:02
Makes some excellent points and I could just expand on one or
16:06
two of those points. So in a manufacturing process,
16:13
if you've got too many false positives, you'll stop a line
16:16
and, and, and it may trigger other processes like Yogan was
16:20
saying, which if you trigger too often, it's very expensive.
16:25
And yet if you miss certain manufacturing defects, you can
16:30
create a massive loss of material because you've
16:33
manufactured defective material. So it's really walking that
16:36
tightrope. And here's where the initial
16:40
process of figuring out the statistical process control of
16:45
where those limits are, O one thing I've learned is there's
16:47
nothing called 0 or 100, OK, so it's all a question of, you
16:51
know, how many errors you can live with, how often do you do
16:54
them? And figuring out statistical
16:55
process controls in the manufacturing process.
16:58
So to Jurgen's point, there's a cost on both sides and it's a
17:04
fine line that we have to walk and we have to set that up at
17:09
the point where the processing is developed and then make sure
17:12
that that is followed throughout the as the process is run in
17:18
time. That's brilliant to think about,
17:21
especially when it comes to this idea of like it's not black or
17:26
white, you have a lot of grey area, there's no 0 or 100.
17:29
And so you have to be thinking through these different
17:32
trade-offs. And if you're stopping the
17:36
production line, that's great, especially if you make sure that
17:40
there's not faulty products that are going out.
17:44
But if you're stopping it too much, that's not great.
17:46
And you have to really answer to people who might be a little
17:51
more hawkish on, hey, let's just let's stop it.
17:53
But then what you're going to say is people's lives are on the
17:56
line. So you can't be playing loose
18:00
fast and loose with somebody who is who's got their life there.
18:05
So Next up that I wanted to, I wanted to just dive into like
18:10
the idea of going from proof of concept to production and maybe
18:15
educate. Can you talk to us about some of
18:17
your experiences, if you've had any of like you've seen it in as
18:22
a POC and then when you wanted to take something to production,
18:26
how is that and what are some challenges that you've had
18:28
there? So this is commonly one of the
18:31
biggest challenges actually proof of concept because in in
18:35
especially battery industry, you don't want to deploy any models
18:39
right away just because you think it's working, because it
18:42
takes months to make sure that your model is robust enough to
18:46
be deployed on the line. Because it's not just about
18:50
millions of dollars that you're going to lose in material, but
18:53
also it's too risky to leak anything to the cars that those
18:58
batteries going to go to. So this is one of the biggest
19:05
challenges that I think we do have.
19:08
So we spend a lot of time in the proof of concept and also doing
19:13
like secondary inspections instead of actually doing a live
19:16
inspection before deploying the model in the live for making
19:21
actual judgments. And is there certain
19:26
infrastructure? And I know one of the big
19:28
questions that people have is around the teams and like how
19:32
the platform looks, what kind of teams are responsible for the
19:36
models when they go out? Once you get that green light,
19:39
there's so many different stakeholders that are involved
19:42
in these questions, right? And so how do you make sure that
19:47
everyone is on board and everybody understands what's
19:50
going on with these AI models that are going out?
19:53
First of all, like in our use case, like when we do the proof
19:58
of concept and we when we know that we are ready, we have to
20:01
let the production team. Know that this this new model is
20:05
going in place and then they take care of like dear part who
20:10
they need to confirm, like they they need to notify that this
20:14
change is happening. But even while the change is
20:17
happening, it's usually not as a primary change because there is
20:20
another model that's actually working to make sure that the
20:24
secondary model is there. And then we compare the results.
20:29
So like our analytics team would be the one comparing to make
20:32
sure that OK, this is actually doing a better job than the
20:35
current model. So now it's time to get it on
20:40
the light. So then we have to notify the
20:42
people who need to know and there is a chain of command.
20:47
Of course it does take some time, but it's very important
20:51
that we go through that step. Yeah, along that and there's a
20:54
great question coming through in the chat from Richard.
20:57
There's also an awesome one from Hattie that I'm going to ask in
21:00
one minute, but the one from Richard is perfect.
21:03
I want to throw this to everyone.
21:06
Are we looking at purely AI ML solutions or hybrid where the
21:11
legacy hardware and techniques are used In Sync with AI?
21:16
How does AI integrate with what is in play already?
21:20
Safety and emergency response is what I'm thinking about
21:23
primarily. I can talk a little bit about
21:26
this. So not everything is AI, Yeah, I
21:31
is used, but we also use rule based system which is the OG of
21:36
inspection still and a lot of dimensions we have to stay with
21:42
a rule based methods. So and in many cases we use rule
21:47
based and then AI after that or both models at once doing the
21:54
judgement because rule based is pretty fast.
21:56
Usually we don't have inference speed issues with rule based.
22:01
SO sometimes both models are working and then we are using
22:05
judgement of both of these models.
22:07
Sometimes they're using being used as a combination.
22:11
So the let's say there is a surface defect that's visible to
22:17
the rule based model. It's then sent to the AI model
22:22
to confirm OK it is actually a defect or not instead of just
22:26
sending everything to AI because that's going to be time
22:29
consuming. And 2nd it it's going to be very
22:33
difficult to send everything to AI.
22:35
Yeah, so there's two class two groups of faults happen.
22:38
Yeah. So the not everything has and
22:41
must be a I mean it's not here just a beauty contest to just
22:45
strictly. So we are using lots of sensors
22:49
or intelligent sensors, which make things easy when you can
22:53
have little variables you put on workers in largely Windia
22:58
entering someones not supposed to enter, they just get notified
23:02
and that's very effective and especially also low error rates
23:06
to Mohan's words per previously. And the other thing we are doing
23:10
now is to marry the what's called first principle or
23:14
natural or world with the data I world is pins physics and form
23:19
neural networks. It was a natural evolution step
23:22
because if you just go purely data, what these algorithms do
23:27
is they optimise and they always try to find the maximum.
23:31
The reality shows when when you show this, let's see this
23:34
optimised solution to an engine. Here he's going to say to you
23:38
why this is a perfect solution, but I will never going to do it
23:41
because I'm 0 Margin for error is the critical thing is how you
23:45
teach systems margin for error because this is what clever
23:50
engineers this is like. I leave myself always a bit of
23:53
wiggle room in case something goes a little bit off track.
23:58
We're not having immediately disaster.
24:00
We just keep it going. And this is what it is in many
24:03
processes. When you look space thick, Elon
24:07
Musks a fourth launch and yes, the thing was falling halfway
24:13
apart on reentry, but it still it had enough margin for error
24:17
to get it to a successful splashdown in the Indian Ocean.
24:22
And this is what pure database systems don't understand.
24:26
They don't understand the concept of what they were.
24:29
And is that so? This is a fascinating idea
24:31
because it's like in software, you have a lot of wiggle room.
24:37
And in this type of scenario, when you're in the industry, you
24:41
don't have that wiggle room and you can't do things where you
24:45
have the 0 margin for error. Is that what I'm understanding?
24:50
So there's very little environment where you truly have
24:53
0 margin for error. So every engineer tries to give
24:58
himself a space where you can operate in, but this space is
25:04
often differently defined or does not necessarily cohere
25:09
with. You run an optimization
25:11
algorithm which say I'm trying to climb the mountain as high as
25:14
I can. So they would say if you take
25:17
Mount Everest, would they go to 8843 metres?
25:22
No, they would say I'm. For me, 7 1/2 thousand is good
25:25
enough. If I'm sticking around there,
25:28
I'm safe and I can manage it. If I'm going to the top, my
25:33
margin for error was literally zero.
25:35
And and seeing is that many, many industries is it's exactly
25:41
what they're managing the margin for error to keep the process
25:45
going because continuity in production is more valuable than
25:51
short periods of peak production and then long prolonged periods
25:57
of time where you are not running at all because you break
26:00
something. And that's what these guys are
26:02
factoring in. And this is teaching this
26:05
machines isn't isn't an easy task.
26:09
Yeah, maybe just to follow up on your again, I think this is an
26:11
excellent point because essentially exactly if you just
26:15
take the data, you throw it in, you take like the latest
26:18
greatest optimization algorithms.
26:19
They just don't have this notion of optimising and adding the
26:23
error for the margin. And not only that, but it's also
26:27
these algorithms are optimised for the average case, for the
26:30
expectation over the data set. But this is again not something
26:34
that you want to optimise for. You actually want to optimise
26:37
for the corner cases that will lead to like production lines
26:40
stopping. And this is where the margin of
26:42
error is really important. And this is very, now again,
26:46
would like to go and actually start thinking about how do I
26:49
maybe change the algorithm? How do I add this margin of
26:52
error for these things to actually work well in practise?
26:55
And maybe one more thing to also follow up on what anarchists had
26:59
said in the original question. So it is definitely often these
27:04
kind of systems of systems where the EML algorithm is like one
27:08
piece that maybe has actually replaced some original algorithm
27:13
that now can be optimised more. And this is also something that
27:16
often when going from POC production, not always is kind
27:21
of so clear because then you get problems like, OK, if I change
27:24
this algorithm in the future, now I have other things that
27:28
depend on it. And maybe the other parts
27:32
actually learn to kind of manage the mistakes the algorithm
27:37
makes. And now maybe I improve my
27:39
algorithm and actually the whole system breaks even though my
27:41
algorithm is better. And that's because it's just not
27:44
this like one small box. It's a bigger system.
27:48
Man, that is, yeah, it's like you're you're taking one step
27:52
forward, but two steps back in a way by making that algorithm
27:55
better. So you have to be careful.
27:57
Yes. And then this also goes back to
27:59
like how are we making it better again?
28:02
Is it like is it strictly improving over what we have
28:05
previously such that we really avoid these regressions or is it
28:09
this on average battery if I look at this traditional
28:13
metrics, how we evaluate these models from like purely machine
28:18
learning perspective? So the chat is awesome right
28:21
now. I gotta give a shout out to
28:22
everyone that is asking questions.
28:26
Syed, what's up? Assad is in the chat asking
28:28
about in the realm of quality control, how can AI systems be
28:33
trained to handle the vast variability in defect types and
28:40
severity seen across semiconductor slash wafers and
28:44
EV components? And as as a manufacturing
28:47
company mean, so we manufacture a lot of our components of
28:51
ourselves. So we we have encountered what I
28:56
call predicting the unknown because things can fail
29:02
mathematically in many, many, many, many ways which we can
29:05
never well, we never have enough data to train anything to
29:08
predict it. O we made a conscious decisions
29:12
a while ago where we said we forget the failure pattern
29:15
training because it's never going to work.
29:17
You will encounter every day something new which you haven't
29:21
seen, and you'll be just retraining, retraining,
29:25
retraining all the time. So we said, OK, let's train the
29:29
algorithms on how the ideal state is supposed to be.
29:32
Says OK, this is perfect. And Parliament mentioned it
29:36
earlier. It's not always easy to define
29:39
this, but saying OK, this is the ideal state, this is how it
29:41
should look like. Everything within a certain
29:45
margin of error which does not look like this is bad.
29:48
And the advantage of that approach that is you can, you
29:52
don't need to know the bats beforehand.
29:54
You can either. What has a retro labelled them?
29:58
So if you if you encounter something you've never seen, you
30:02
can label it and add it to your foundation.
30:04
So the next time you would not only create a KFW nation from
30:08
good, you could also say, yeah, we have this in this case and
30:12
this triggers this, this kind of fix.
30:15
But the advantage of this is it keeps the system stable and
30:18
robust and you can deal with all sorts of forms of abnormal
30:23
abnormalities which you have never encountered before.
30:26
So, so that's the take we took in since we, we switched this
30:30
and we also found we have way lesser issues with labelled
30:33
training examples reference cases, because the, the amount
30:38
of good production data that we have outperforms the bad data by
30:44
I don't know, a million to one or even more.
30:47
So that is how we literally switched and tackled the the
30:51
challenge of predicting the unknown.
30:54
Yeah, so it it sort of boils down to a system design problem.
30:58
So as you're gonna mentioned, you know, the points yoga
31:01
mention are very important. So if you try to map out all the
31:06
failure modes, then I think it's an impossible task because
31:09
there's always going to be something new that pops up, some
31:11
variation of something that you haven't seen before or what have
31:14
you. And machine learning models,
31:16
unfortunately, as all mentioned, they are function wrappings,
31:20
which means that they're on distributions and the long tails
31:24
of those distributions are very, very poorly learned because
31:27
there isn't data there for them to learn, right?
31:30
So, so really this is about system design.
31:33
You've got to build what we call anomaly detectors that just
31:38
detect as Yorkin mentioned from the ideal state that deviation.
31:42
And so there's far anomalies that are easy to detect.
31:45
Any kind of simple detector is going to detect that it's the
31:48
near anomalies to the boundaries of the statistical process
31:51
control that become really hard. And that's where that's that's
31:55
where the challenge lies. And that's where you need to
31:58
build a system designed quite carefully.
32:01
And if you do that well then and you have the right feedback loop
32:05
within the manufacturing system, that means that anything that
32:07
you detect in that window, you must get it QC D right.
32:11
And you must be able to QC it with root calls.
32:14
So you not only QC what you're detected, but also you're able
32:17
to root cause that to the manufacturing process itself.
32:20
And then you can close that loop.
32:22
Then you can maintain sort of a stable process as you
32:27
manufacture. Nonetheless, having said this,
32:30
surprises popping now and then and it leads to those late
32:34
nights where everyone has got to get up and get going to figure
32:38
out what the hell tracking here, right?
32:40
And so that happens every now and then in, in, in, in these
32:45
high stakes manufacturing environment where you've really
32:48
got to solve that and get that process and control ASAP.
32:52
This is so we always love hearing about those late nights
32:56
and 3:00 AM calls, especially on Sundays, but it does feel like
33:02
there's so I want to address one thing tonnes of questions about
33:06
LM's in the chat. I'm going to get to that.
33:08
We'll have a whole LM session section about that.
33:11
The Gen AI hype train will arrive.
33:15
Don't you worry about a thing. But before we do, staying on
33:19
this idea of just like complex systems and system design
33:26
challenges in designing the systems, it is something that
33:30
you were saying, Pavel, they don't work in isolation, right?
33:33
So you have your model that's only one piece of the pie.
33:37
And what are some of these challenges of integrating AI
33:42
into the equipment, into the processes?
33:45
When you've gone about doing that you, you mentioned before,
33:52
maybe the model starts working really well, but everything else
33:57
has almost been picking U the weight or it's been carrying the
34:03
model. And so now when the model starts
34:05
working better, things start to not work as well.
34:09
And especially when you're dealing with, I feel like when
34:11
you're dealing with things in real world, it's not bits and
34:15
bytes, right? It is actual physical materials
34:20
that you have to deal with. How do you think about some of
34:23
these system challenges? I think the definitely one of
34:28
the big challenges here is, or at least traditionally have,
34:32
have been trust. Because if you have these
34:34
systems that have been designed by engineers, by humans, you
34:39
kind of kind of look at how you can look inside.
34:43
You can understand this is how the decision is made.
34:46
This is why it was made. Maybe now This is why it's
34:48
failing. And I was saying, let's praise
34:50
this little box with the eye. You better get like kind of
34:56
better return on investment than just now saying, OK, I don't
34:59
actually know what's going on. So you really have to kind of
35:02
have very good reason why you are replacing it.
35:04
And there are there are very good reasons why, why companies
35:07
are doing this, but then you have to somehow get the trust
35:11
back. And, and this is again like one
35:14
question is the system design and the other one is just very,
35:17
very thorough evaluation. And this is also what unit had
35:20
mentioned. These are not things that
35:22
typically happen overnight. Like you have to be very careful
35:25
about how you set up the process.
35:27
There is like a backup system typically that is you are not
35:29
running it on one model, you are running it on multiple models.
35:33
And when whenever there is a disagreement, you would also
35:36
start flagging things and things like this.
35:40
I do like one thing that you mentioned, Aniket, about how you
35:43
have analysts that are looking at the performance too and
35:47
you're seeing is it actually doing better than what our
35:51
baseline is or what we had before.
35:54
One question that came through is what are some of those KPI's
35:57
and metrics are used to evaluate the success of if something is
36:03
is going better than before? And this actually, I mean, of
36:07
course I could see a couple of things which we get from like
36:10
confusion. Resources are common, but you do
36:13
have to actually have a human look through it and not just
36:16
numbers because you are comparing the images.
36:20
Of course there's no way to do all the images, but the ones
36:24
with the less confidence may be the ones where we are having
36:27
more issues. Which one is performing better
36:31
and is it still performing good on the ones that we are not
36:34
having issues? Because we can't just just
36:35
because the model starts to perform better on let's say a
36:39
new type of defect that we start started to see and we trained
36:43
well on and but then starts to perform bad on the other ones.
36:48
So we still have to make sure that the ones that were already
36:51
good are still good and the new ones are, is it actually
36:56
performing better? So there's a lot of manual work
36:58
in this process other than the mathematical numbers that we get
37:04
just because of how the industry is.
37:07
We need to be sure we can't have just rely on the numbers.
37:12
You're gonna wanna throw one over to you because there was a
37:15
pretty hairy use case that you mentioned beforehand about this
37:21
low latency use case. I think it was computer vision
37:25
though. Latency.
37:27
Now, would you say those are some of the hardest ones that
37:32
you've had to deal with where you're trying to incorporate AI
37:35
or ML into it? Or if not, which ones are some
37:39
of the hairiest problems? Yeah, it's it's a time when this
37:42
one's from an AI perspective, it's still computer vision.
37:45
If you have a well trained model and it's not that much of a big
37:53
deal. And the things you need to solve
37:56
is how you eradicate transfer times to because you still have
38:01
multiple players, you have a camera or a camera system, you
38:05
have control system in place. How do you get and you have you
38:09
need to execute your AI model somewhere as close as possible.
38:14
So how you eliminate the time in between, you can get the
38:18
individual components run very, very fast.
38:22
It's just how you stitch it together.
38:24
So often it's it's not the individual parts, it's how you
38:27
stitch it together. But we have other problems where
38:33
the pure challenge on the AI side is reaching dimensions and
38:40
which which are right now if you would say with standard means
38:46
that really solvable, you would hear the likes of NVIDIA saying,
38:51
ohh, you couldn't use our super chips and we can bring
38:53
lithography from days to minutes and hours.
38:58
Yeah, but this is still often not the answer.
39:01
It's just throwing more compute power on.
39:03
It doesn't solve the actual problem on hand.
39:06
And this brings us back to the mathematical role of problems
39:10
solvable in ON or not. And so this is same where it
39:17
comes really into it. And then we also encountering
39:20
scenarios where the algorithms we have today available just
39:26
don't cut it or they cannot replicate the concept.
39:31
For example, what what I always have to smile is when the
39:35
generator if you I people speak about how they intend to teach
39:40
reasoning to and generative model and they're quoting Yachin
39:44
of Fort is the answer to reasoning.
39:47
When I see it, I said, look just by changing from an programmatic
39:53
programming language into a natural language programming
39:56
language. This has nothing to do with
40:00
teaching a system gene fault or reasoning.
40:03
It's just you provide instructions in a different
40:07
format, but it's still providing precise.
40:12
Instructions to assist, and you can see that you can very easily
40:19
circumvent or prove them wrong by making some tiny alterations.
40:24
And when there's the classic examples in in computer vision,
40:28
then you change one single pixel in an image of a cat.
40:33
And for the human eye, it's still 100% a cat.
40:37
But for the computer vision, it's already a doc.
40:40
And so handling these things where you have ultra minor
40:46
changes in assuring that still if you use human perception, you
40:52
can easily determine, yeah, that's still a cat.
40:54
Doesn't matter if that bloody pixel is changed or not.
40:57
It's for it's for systems massive problems.
41:01
And it they always make you think like him.
41:05
And I know there's a lot of marketing going around say,
41:09
yeah, yeah, no problem, you have it under control.
41:12
And, and we're seeing it in some very simple programming,
41:18
generative programming examples. It's by all means not simple
41:23
because following the human, the human understands the process
41:27
and then writes the code to fit the process.
41:32
Making a machine understand the process is unsolved territory.
41:38
Today we're trying to break it down, making the chunks small
41:42
enough. But look at Morgan's example of
41:45
manufacturing a computer chip. It's 1700 steps which have to be
41:52
synchronised and sequenced. Try to ask a GPT right metre
41:57
control programme which synchronises the 700 steps you
42:02
get is simply not working. And so this is the things where
42:08
it gets really, really complex and where we reaching limits of
42:12
machine comprehension. Now I think here again makes a
42:16
lot of good points. From my perspective, I think
42:20
it's about both the machine learning models, the
42:23
foundational models and, and, and these large language models,
42:27
large language vision models, They definitely have all kinds
42:33
of problems, right? But the question is, can we as
42:37
humans introduce enough inductive biases to find small
42:41
but controlled domains to use them impact fully in the
42:45
industrial, right? So it's almost as if you're
42:48
working around this very rough tool that it's very powerful,
42:54
but it's got all kinds of problems and you figure out a
42:56
way. No, no, no, I can cut my thing
42:58
by just doing this there. So today we are in that kind of
43:02
a situation. We we bring in all these machine
43:05
learning scientists that try to figure this out day in and day
43:08
out by building systems and trying to figure out how to make
43:11
these rough and ready things that on their demos they
43:18
amazing, right? You can see a demo that just
43:20
looks like ohh yeah. We saw world hunger yesterday.
43:23
But the reality is very different.
43:25
So I think, I think to Yorgen's point, there is a place where if
43:32
we can build the right guardrails, we can still get a
43:34
lot of value out of these things today, even though they're far
43:37
from perfect. And that's what you were talking
43:40
about, protecting against hallucinations and, and really
43:44
finding a way to use it. So I think the current state is
43:47
very clear. These things are highly
43:49
unreliable. And if you want to use that, you
43:51
better put a lot of guardrails and protection mechanisms in
43:54
place. But but let's see how these
43:57
things evolve. I think there's just a lot of
43:59
science and engineering and hardware and compute getting
44:03
thrown at it and, and, and a lot of smart people working across
44:07
the globe. So yeah, let's see how this
44:10
evolves. And our challenge, I think, is
44:12
to stay on top of this. It's so insane, right?
44:15
Then the number of new models, a number of new approaches, some
44:18
good, some not so good, and both staying on top of them, but also
44:22
the cost of staying on top of them.
44:24
So very expensive, you know, and new multimodal models come out
44:27
to figure out if something is an actual improvement or not for
44:31
our use case, right? So it's an interesting world we
44:35
live in, but we diverge away from manufacturing.
44:37
But. That that's a great point that
44:40
it's not just the cost of like the API calls, but the human
44:44
cost of trying to spend time and figure out if it's worthwhile to
44:49
put a team on this and implement it or at least mess around like
44:54
Aniket was saying, do a POC with it and then see if they can
44:58
graduate up to being in production.
45:00
So there there's also there's a few other valid points that I
45:05
wanted to call out here. In these complex systems, we're
45:08
by no means at any point where we can say to an LM, OK, go
45:13
figure out this whole very complex system and optimise each
45:18
step. But Mohan, what I'm hearing from
45:20
you is you're, you're saying if we can limit the scope as much
45:25
as possible, then potentially we can optimise maybe one small
45:32
step or half of a step or .5 or .3 of a step, that type of
45:38
thing. So I I do like that idea.
45:41
No, since we're on the topic and I think everyone in the chat has
45:46
been asking for it, we should start to talk about LLM's and
45:50
how you all are using them, if at all with and especially like
45:55
I know there were some use cases that you had mentioned you were
45:59
going about using them. Like what?
46:01
What are you using if at all? Because I heard from my friend
46:08
Paco the other day and he was talking about with a company
46:13
that he was working with that was in the heavy industry.
46:17
You would think that a lot of this predictive maintenance and
46:22
the stuff that we've been talking about for the last 45
46:24
minutes would be the highest ROI and the most important things
46:29
that they can do with AI. And he said that when he went
46:34
into the company, the biggest ROI and the lowest hanging fruit
46:40
was just digitising PDFs and letting those be searchable,
46:47
which is debatable if it's even AI at all.
46:51
And so I, I would love to hear if you all have found that on
46:55
the admin side, there's been areas to leverage AI use cases.
47:02
So men obviously by order of our CEO, we had to look into LM's
47:08
and how we can apply them and believe the obvious use cases
47:14
where it actually the lowest value is in automating simple
47:19
things like we get lots of emails, customer service gets
47:24
lots of emails, going through the emails and creating an
47:28
appropriate response which would have otherwise taken days or
47:31
weeks before it has been processed through The xx hens
47:36
who have to take a look on it. So there is use cases around
47:41
chatbot automated answer generation where these things
47:44
working extremely well because it's what they are designed for.
47:49
The very nice create very nice textural answers when we start
47:55
to bring it into the industrial world than you see like when
48:02
everybody was thinking coding, creating code with L&M's should
48:07
be a super easy thing to do. And then in the first instance
48:12
for your, let's say, private coding examples, it works fairly
48:18
well. But when you lift it on an
48:21
industrial scale where repeatability, maintainability,
48:26
consistency in language plays a role, than this whole story
48:32
becomes very, very different because the nature of the auto
48:35
encoder specifically in combination with Rag is he tends
48:40
to continuously change his answers.
48:42
So if if I ask you 10 times over time, the question right me a 10
48:48
second wait loop, my likelihood that I get 10 different answers
48:52
for the same task is extremely high.
48:56
Now translate this, you have field service engineers, service
48:59
engineers going in the field. They're using these tools and
49:06
when these guys are not dumb, they know how to fix things.
49:08
So if if he asks the tool tell me how to fix it and he asks it
49:12
five times and he get 5 different answers and it brings
49:16
it back to the point of trust bubbles that earlier if you give
49:19
him five different answers will he is the guy going to trust the
49:22
system or will you just stop using it?
49:25
As I'm, I'm reverting back to what I have in my head because
49:28
this is more reliable and more repeatable than following a
49:31
system which tells me every 5 minutes something just in the
49:35
way how the probabilistic mathematics fall.
49:38
It guides it through differently, and so ESS charming
49:43
as it is on first glance, using these spins on industrial scale,
49:49
you tap into the probabilistic nature of the algorithms and
49:54
therefore the likelihood that they give you for the same
49:57
question multiple answers is very high.
50:00
And this creates a huge issue because many as I said in in if
50:06
you control process control systems PLC, you don't, you
50:11
don't wanna have 10 peel C's with 100 different programmes
50:16
for the same task. So that's what we are finding.
50:19
And this is where we are now have to be literally do open
50:23
heart surgery on the GPS to again contain to Mohan's words,
50:30
put the guardrails in place that it's not going to go ballistic.
50:34
Yeah. One question that was coming
50:36
through the chat that I think a lot of people have is, are you
50:40
fine tuning directly? What?
50:42
What if so like what does that look like?
50:44
It's, I imagine it's not just hitting an open AI API.
50:50
So maybe breakdown the use case Mohan for us and then tell us if
50:53
you are using an in production or if you're like you're going a
50:57
little bit more like it's fun, it might work, but it's very
51:03
dangerous. Yeah, I think, I think honestly
51:06
you're gonna covered most of the important points here, but I'll
51:09
just add something from my perspective.
51:14
So whenever there's a copilot use case, it can be quite
51:19
useful, which means that think of a copilot use case where it's
51:24
helping you to do some tasks and it's A and you as a human is
51:29
other master and you control the, you know what that end
51:33
result is. So a lot of copilot use cases
51:36
are seeing a lot of traction from GPS where hallucinations
51:39
are caught by the human. And yet for a majority of the
51:43
time, if it gets it right, then it's useful to the humans to go
51:47
faster, to do more things, etcetera, in terms of actually
51:53
deploying this in production. So we've got think of the world
51:56
as the development world where we actually develop things in
52:00
house before we go to production.
52:02
So improving the pace of development and internal
52:05
processes, I think they're very much in place today in terms of
52:09
production, you know, building the right guardrails becomes,
52:14
becomes a bit challenging to do it right off the bat.
52:20
So it's, you're not quite sure. Suddenly a new LLM pops up,
52:25
suddenly a new multimode model pops up.
52:27
Suddenly you want to try that and then it's just as Wild West.
52:30
So we've got some limited sort of production deployment type
52:35
situations, but we are very careful because for us, trust is
52:39
important. If we deploy something and and
52:42
and it leads to some errors and some obvious hallucinations,
52:46
then our customers are just highly unlikely to trust
52:49
anything right then. Then you, you break trust, not
52:52
for that particular stake, you break trust across the whole,
52:56
whole product. So, so production wise, I think
53:01
it's in our world. And right now I work in
53:04
Insurtech. It's not.
53:07
We are, we are there in bits and pieces where we can control the
53:11
destiny very well, but we're not there as a standalone multimodal
53:16
model or an LLM that's that's deployed in production.
53:19
So since you brought up the topic of trust, I think there is
53:23
an interesting question around trust between traditional mill
53:32
versus trust between looms And have you seen Pavel in your
53:37
experiences people just finding that with traditional ML since
53:42
we've had more time to stress test it, there is a bit more
53:46
trust there and you see things going out into production.
53:50
What is what is your take on like the trust with one versus
53:55
the other? Maybe I would just say like and
53:59
because the LMS are new hype, like everybody knows about them,
54:02
but this is not a new problem. Like we had the same problems
54:06
five years ago. Because it does boil down to
54:09
there are people using systems And if these systems have been
54:12
consistent answers as you're going mentioned, because like I
54:16
turned on my light or like the camera shifts like 1° to the
54:20
left and suddenly it's not a defect.
54:22
I will just not be very happy with the system and and that
54:26
user is at the end. And maybe all the people who
54:29
have Teslas also seen this like there is a software update
54:32
suddenly like think work differently than before and I
54:35
have to adjust. And I think with LM, they're
54:38
just more pervasive because it's, it's people have more
54:41
access to it. And the main thing I would add
54:44
here is it's, it's a challenge of kind of building even the
54:47
framework to evaluate the systems in the 1st place.
54:51
And kind of coming from academic background, this is also
54:54
something we, we are very interested in.
54:57
And we would be also open sourcing and kind of initiative
55:01
around this to actually build the framework where anyone can
55:05
go and kind of check the model against kind of set of kind of
55:10
properties and benchmarks that these models should satisfy.
55:14
You're you're gonna open source that did I just hear you say
55:16
that? Yes.
55:17
Alright, we will make sure that everybody finds out on you do
55:21
that is super cool to hear. I feel like we are hitting up on
55:25
time so I want to be very conscientious that you all are
55:27
awesome for doing this and thank you so much.
55:31
There's been some incredible questions that we didn't get to
55:34
all of them. So forgive me for not being able
55:36
to ask them all. If anyone is listening in the
55:40
chat and wants to continue the conversations with any of these
55:44
incredible panellists, hit them up on LinkedIn.
55:48
I think they're all, they're all active on there.
55:51
So it would be great to continue the conversation.
55:53
And as I mentioned at the beginning, feel free to join us
55:59
in California for the AI Q Con, AKA the AI Quality Con Friends.
56:06
Would love to see you there folks.
56:09
This has been a lot of fun and hopefully you learned some stuff
56:14
and you have more questions when you're thinking about AI in
56:18
industry. Let's keep the conversation
56:22
going. I'll see you all later.
56:23
A huge thank you to everyone of you for coming on here and
56:27
talking with me and enlightening me on these very important
56:31
topics.