Latest / Agentic Conversations (formally mlops.community) / ML and AI as Distinct Control Systems in Heavy Industrial Settings // Richard Howes // #243

Transcript

ML and AI as Distinct Control Systems in Heavy Industrial Settings // Richard Howes // #243 · Agentic Conversations (formally mlops.community)

0:00
But my name's Richard. I work for CBR right now as a
0:05
product manager. I focus on data science products
0:08
primarily and my coffee is usually taken black after a long
0:13
run so hit you hard with the caffeine at the beginning of the
0:17
day. Hey everybody, we're back for
0:20
another ML OBS Community podcast episode.
0:22
I'm your host, Dimitrios. Talking with Richard today, we
0:27
took a few of the learnings that he has had over the years
0:30
working in the heavy industry and tried to apply them to how
0:37
we can build more reliable systems with AI and mill at the
0:43
core. We went on so many tangents, I
0:46
had a blast. But one huge takeaway is his
0:50
idea of control systems and how we want to make sure that when
0:56
we are working with a data scientist, they go and they do
1:00
their exploratory data analysis. You give them all of the most
1:08
important information that they will need.
1:11
As he likened it to context stuffing of an LM really,
1:17
because you are less likely to fail when whoever it is that
1:24
you're working with the key, the other key stakeholder, in this
1:28
case, the data scientist, understands thoroughly what they
1:34
are trying to accomplish. They're going to be much better
1:38
at getting that outcome if they understand what it is.
1:44
We've heard it time and time again, but he breaks it down
1:47
eloquently. Let's get into it with Richard.
1:50
And as always, if you like this one, share it with one friend or
1:55
write us on Spotify. They can write in stuff now and
1:59
I'm reading all of them. Some of them are pretty funny so
2:02
feel free to comment. And I got thick skin.
2:07
Say whatever you want, O dude, you've got a violin back there.
2:20
What's the story with that? Because you know if you.
2:22
Finish. Yeah, I, I picked it up like a
2:27
year ago. I needed to get up off the desk,
2:31
right. And my daughter, she's always
2:33
playing the guitar and the harmonica, so naturally started
2:37
jamming children's songs to start.
2:41
But it's a good start. You get the the basics down.
2:45
The violin goes so nicely with the guitar that it's not the
2:52
first instrument that I would expect.
2:54
You're like, hey, my, my daughter is jamming the guitar.
2:57
You know what we need in that like Bob Dylan song that she
3:00
likes to play is different violin solo.
3:03
Yeah, totally. Let's grab the middle.
3:07
Yeah. Yeah.
3:08
Well, maybe, Yeah, maybe you're more on the bluegrass side of
3:12
things. I guess in that style of music
3:15
you can get a fair bit of violin.
3:18
That's where I'd like to get into.
3:20
I'm not quite there yet. They're pretty, pretty
3:23
complicated compared to where my skill level is, but it's pretty
3:26
fun around the campfire. You just background a few little
3:29
things, you know. Yeah, busted out.
3:33
That's so good. Well, man, you're working on
3:35
some pretty wild stuff with AI and ML in Heavy Industries.
3:40
Can you break down what the state of AI and ML is right now
3:45
and what you've been working on? Yeah.
3:48
So there's a tonne of stuff going on in in Heavy Industries.
3:52
And by Heavy Industries, I mean like oil and gas, forestry,
3:54
manufacturing, those things with lots and lots of equipment in
3:57
there. And you see anything from the
4:01
embedded type devices like IoT doing fancy measurement of
4:07
sounds to monitor safety and in a workplace to the enterprise
4:10
application, which, you know, we've seen lots of chat bots,
4:14
we've seen lot of, you know, Q&A Reg systems to try and help us
4:17
understand our data a little better.
4:20
But the common ones out there, you know, I, I think almost
4:24
everybody's kind of doing them. They're like preventative
4:27
maintenance. Let's like try to predict before
4:30
machinery needs to fail, right? QA, QC processes around
4:36
compliance for the regulatory bodies.
4:38
Like that's kind of a complicated endeavour,
4:41
especially with energy and emissions and carbon reduction.
4:45
The regulations change frequently.
4:46
So keeping on top of that's challenging.
4:50
Wait, what does that mean? It's basically like your use it
4:55
how? How does ML AI get incorporated
4:58
into that QA? So, you know, first regulations,
5:03
regulations come from legislation, right?
5:05
Like from some government body. And then the regulators are
5:10
usually third party government agencies who regulate the
5:14
industry, say like a boiler pressures and safety vessel kind
5:19
of situation. You have the auditors coming in
5:22
to see if you've done the maintenance and check out your
5:26
procedures and stuff like that. So it's kind of difficult with
5:30
the rapidly changing industries like, you know, carbon emissions
5:33
to keep up with policy changes. So, you know, using a simple
5:38
Radbot to help you understand the regulations and maybe queue
5:42
up changes kind of helps you advise on what to do next for
5:48
your policy procedures. You know, the change management
5:50
in these industries is pretty substantial.
5:53
So when changes come down the line, it's usually a big to do
5:56
right. We have a whole team of people
5:58
managing that for us. So it's important to get it down
6:01
to the people who are doing the work as soon as possible.
6:06
So that's the QA, QC kind of thing, quality checking and all
6:12
of the compliance and stuff. You also mentioned embedded
6:16
systems. I mean the predictive
6:17
maintenance I've heard a hunt about before because it can be
6:22
so useful when you're working with machines that cost more
6:26
than a yacht. Yeah.
6:28
And the coordination of them, right?
6:29
Like so if you had a like a gas plant, for example, and you're
6:34
going to do a turn around in June, you're trying to plan what
6:36
kind of stuff needs to come out and what kind of stuff needs to
6:39
go back in. You wanna really time your
6:43
equipment replacements and updates that one time when the
6:46
equipment's down for the year, right?
6:48
If you if you mess that U, you're going to your is going to
6:51
be down again and then you're losing money, right?
6:55
So you really want to keep on top of what needs to be replaced
7:00
when and keep track of the patterns on failure, right?
7:05
Most of that stuff comes from the control systems themselves
7:08
actually, so get a sense of what the performance of the equipment
7:12
is. To explain control system for
7:17
for myself and maybe some people that are listening that might
7:20
not know what it is. Yeah, for you see them every
7:23
day, the most basic ones like control the temperature of your
7:27
room, right? You got the thermostat, you got
7:30
the, you know, element, heating element, maybe the baseboard.
7:34
Optimise the temperature to 20 C and you know, cycle the the
7:40
element on and off to get to the right temperature.
7:43
That's one system, but typically in like a processing facility,
7:47
there's, you know, 10s or hundreds of those working
7:51
together to give you some some output.
7:54
So it's kind of modular in in the design and have the
7:59
industries and it's like that for a reason, because then you
8:01
can test each of those control loops individually to make sure
8:06
that they're working. One thing that fascinates me
8:09
with these types of use cases is that it's almost like you
8:13
mentioned you're using AI for almost this point in time.
8:19
It's not like it's a continuous prediction.
8:23
It is. As you mentioned, we have all of
8:26
this equipment that's coming offline.
8:28
We need to predict which one or five or ten need to stay with us
8:33
and we need to replace and the other ones that need to go out.
8:36
But if I'm understanding it correctly, it's not like every
8:40
day you're predicting this one might go offline tomorrow.
8:43
So you don't need to build your infrastructure and your system
8:47
in a way that it's constantly making predictions.
8:50
Is that correct? Yeah, usually, you know,
8:54
preventative maintenance is a batch process, you know, whether
8:57
you're doing it monthly or annually.
9:00
There's stuff that you do every day, too, But you know, when
9:03
you're scheduling a big retrofit or overhaul to make sure your
9:07
system's running as you expect it to be, that's usually done
9:11
once a year, hopefully just once a year.
9:15
You know, you know, I'd be going down too much.
9:17
But yeah, it's really important to nail down the critical
9:21
components and their failure points just to make sure you're
9:25
operating at an optimal space within your processes.
9:29
Pretty substantial investment and just taking things down
9:32
because you're going to be checking all the safety systems
9:34
at the same time. I haven't seen an application
9:39
there with machine learning and AI to help you identify what the
9:42
critical components are. Or am I missing any?
9:47
I imagine that'll come. So that is the predictive
9:51
maintenance on the batch schedule.
9:53
You're feeding in all this stuff, trying to figure out the
9:56
anomalies, you're trying to know what has happened in the past
10:00
and then just decide if something needs to be upgraded
10:03
or not. You also mentioned though there
10:05
are some embedded systems. What does that look like?
10:11
Out of that comes from vendors typically, like the companies
10:15
that I work for, we typically wouldn't develop our own
10:18
embedded systems, which would be an IoT device, a sensor.
10:24
If you bought a turbine, those would come with embedded systems
10:28
to measure the performance of that machine.
10:30
So they're getting smarter and smarter.
10:33
I mean, embedded controllers themselves have AI embedded
10:37
right in their architecture now. So it's it's kind of getting out
10:41
of control, not out of control. It's just interesting to see the
10:45
development overtime, right, because I've seen anything from,
10:51
you know, vacuum tube relay systems to fully automated
10:56
dispersed systems over like a giant geographical area now.
11:00
So it's the change in control systems themselves is quite
11:05
extravagant in the last 20 ish years. 30 years.
11:10
When we had Poco on here and he was talking about the heavy
11:15
industry and how he was consulting with a company that
11:18
was in this field. And I was asking him about the
11:23
different use cases. And he said that by far the most
11:28
valuable things that they were doing with AI were ingesting
11:32
PDFs and then being able to understand the paperwork better.
11:37
And he was like, it's, it's much more contrary to what you would
11:40
think. Yeah, You would think that,
11:41
yeah, this predictive maintenance or maybe there's
11:44
this tiny ML going on and you're doing some cool stuff with the
11:47
machinery. But it was more valuable to get
11:52
all of these PDFs ingested and then get some computer vision on
11:56
them than it was to have crazy stuff going on with the big
12:01
machinery. And I'm guessing that's because
12:04
the machinery is very expensive and it's very high risk if
12:08
you're doing anything on there that could potentially mess up.
12:12
Is that how you look at it? Sort of.
12:16
I think it's reliability is a big one and we're hitting that
12:20
one pretty hard with the quality drum lately.
12:23
It's we need to know what the control systems are and the
12:26
heavy industry has a pretty decent way to document on these
12:31
systems so that everybody understands it, right?
12:34
Like the electrician, the plant operator, the plant manager,
12:38
they can look at the set of documentation they've developed
12:41
and understand the system for their use.
12:45
Nice. So we use that a lot to kind of
12:47
get into the control side. But I think you hit the nail on
12:50
the head with the non equipment related stuff.
12:55
The differentiator in I think heavy industry specifically is
12:59
going to be your data, right. The technology is going to be
13:03
out there available for others to buy and use and whatever.
13:06
There's open source stuff that we can all use.
13:09
The models are being produced by other people who we have no
13:12
impact in. So the best thing we can do in
13:15
heavy industry is just mine. Understand and store that data
13:20
somewhere so that we can use it later.
13:23
Once these technologies have developed to a point where they
13:27
are reliable and we can just start rolling them out, trusting
13:30
that they'll work. They're, you know, regulations
13:34
will probably come in and help us with that a bit too.
13:38
Hmm. And what do you feel like are
13:39
some challenges of getting good data in these industries?
13:44
And specifically like, which types of data are you talking
13:48
about and what are the challenges?
13:52
Well, the challenges are like you said, the PDF.
13:55
So we've got everybody I think in every company has the
13:59
mountains of SharePoint sites, the distributed, you know,
14:02
folders all over the place, the stuff on desktops and whatnot.
14:06
So like reining that in, that's gonna be where the gold mine is.
14:10
It's, it's everywhere, right? Everybody's got their little
14:14
source of data on their computer or in a SharePoint site
14:17
someplace. So getting that into a usable
14:21
format like a knowledge graph. I've seen pretty slick
14:25
applications of been recently playing around with Neo 4 J They
14:29
have a lot of stuff that gets you going real quick.
14:32
So now the quicker we can kind of adopt these newer techniques,
14:37
the faster we're going to be able to implement and test the
14:40
AI use cases. But in terms of the equipment
14:43
side of things, the assets and the equipment is like the number
14:47
one piece of data that you need, right?
14:49
So you might have this big vast plant with control systems all
14:53
over the place. But if you don't know where to
14:56
get the manual for this or that or how to look something U
15:00
quickly because you're troubleshooting a plant issue,
15:03
you're going to lose production time.
15:05
So knowing your assets, knowing where to access the information
15:09
to solve your problem right now, that's the number one gap.
15:13
I don't think too many people have like the giant repository
15:18
of equipment manuals sitting around.
15:21
Like I remember back in the day, operating plans, you'd have like
15:24
a stack of manuals, lot big, but that was just like the six or
15:27
seven key pieces, right? So getting access to those is
15:34
probably the biggest challenge. And that feels like it's such an
15:37
easy fix, right? And so me naively saying it's
15:42
easy, I imagine if there are people that are listening that
15:45
this is their day in, day out, they're like, yeah, easy, right?
15:49
Because you would think, OK, you're just put it in some kind
15:51
of a database and you're able to search for that.
15:56
But what makes it hard? Why can you not get those
16:00
manuals and then just like put them in one spot?
16:04
Ah, just a few housekeeping things really.
16:08
Right. So we all have, well, when
16:10
something is built, you have like the preconstruction
16:13
drawings, once it's built, those are as built drawings.
16:16
Sometimes you don't update your pre built with your as built.
16:20
So you know exactly what you installed and where and how,
16:24
right? Like how that whole thing went
16:25
down. So, you know, keeping up with
16:28
your documentation is a challenge.
16:30
And usually, to be honest, it's we just open your phone, Google
16:35
the piece of equipment which requires you to look at the
16:38
serial number, the equipment manufacturer and things like
16:41
that. So it's kind of cumbersome.
16:43
And there are some solutions you know, I've seen, you know,
16:46
barcodes or simple things like that.
16:49
But I haven't seen anything fancy yet with say computer
16:53
vision or any even repository of this information.
16:56
I haven't, I haven't seen that yet.
16:59
Be a good pet project. What?
17:02
Yeah, yeah, it's, it sounds very familiar.
17:06
Like I think I've seen something on people being able to take a
17:12
video of a machine and then they get a walkthrough or they get
17:17
told like what it is. But that feels also like a bit
17:21
of a Twitter demo. Like it sounds cool in practise
17:25
or in theory, but then in practise it falls down and you
17:29
don't actually have that. And then you end up, like you
17:33
said, you have downtime, which costs a lot of money.
17:36
And so you better make sure it's working if you're trying to use
17:39
this new fancy tech. That's what I mean by augment
17:43
people's or like the operators, whoever's current work.
17:47
Like if we can give the operation staff, the maintenance
17:51
staff access to any of the manuals, the maintenance
17:55
information about particular piece of equipment, we could
17:58
probably get to a solution quicker.
18:00
And those downtime, quick hustle times of the day.
18:06
You can say the same thing about our data and AI world too,
18:11
right? Like how many people know about
18:12
all the different vendors out there, what they can do for you.
18:16
And if you have one, like now you have to go to their website
18:19
and decrypt their docs and stuff like that.
18:24
You had a, you had a fella on just recently about docs and
18:28
like, I liked what he was throwing down.
18:31
Make it quick and easy, man. What are the things the people
18:33
who are reading this doc need to know?
18:35
Let's get it to him quick so they can move on with their day.
18:40
Yeah, yeah. I give him the dopamine hit of
18:42
feeling like they've done something.
18:44
And that is fascinating to think about how with software
18:50
development specifically, you have a whole team that is
18:55
keeping like the upkeep of the software.
18:59
And so you've got this whole team on the hardware side of
19:02
things too and you have the same piece of it.
19:06
Like there's still documentation, it's just since
19:09
it is probably a little bit more hardware like physical world,
19:14
you can get caught in this space where the documentation is more
19:19
in the physical world and less easily searchable.
19:24
I think some pretty interesting things we're going to see in the
19:27
near future as the modelling behind these systems to help us
19:30
understand them a little better. I think we might have briefly
19:34
mentioned it offline, but they're made-up all, all these
19:37
control citizens are made-up of blocks of PID controllers.
19:41
They're just a controller similar to your thermostat,
19:44
right? Simple.
19:46
So that we can test them modular, so that we can port
19:48
them from one place to another. But when you have bunch of these
19:52
together, it's, you know, we understand how these systems
19:56
work from an engineering perspective.
19:57
You know, we get the mechanical engineer and the petroleum
19:59
engineer and they can understand how the refinery works.
20:03
But are there any hidden issues with the thing we've built?
20:08
I think I can maybe help us identify the like system
20:12
degradation over time or control systems that aren't performing
20:16
as well. I think it might be really good
20:18
at that although. We're not quite doing that with
20:23
AI right now, right? Like we're all about reasoning
20:25
and understanding, getting some text back.
20:28
But, you know, if we can kind of think of these systems as maybe
20:32
a different language, we can get them to help us understand them
20:36
like the physical world a little better.
20:39
So this is fascinating and how would you go about that?
20:45
There's a few papers out there already that you know, I think
20:47
they were back in 2020, I forget the name of them actually,
20:52
sorry. But they were using these
20:54
traditional PID loops to see if we can understand is this system
20:58
performing good or not? And what would be some
21:02
mitigations to fix it up the parameters, the the weights and
21:07
this control PID loop. And they're quite successful on
21:11
like a small scale. But getting that to like the
21:15
plant level to understand all of the interconnectedness, that's
21:19
where it fell short and didn't work.
21:22
So, you know, is there a solution?
21:26
I think there is. We as humans can do this.
21:28
So I mean overtime, I think we'll get there.
21:32
Yeah, You have to have such a deep understanding of the Lant
21:38
and how everything works together.
21:40
And so it's not like just one model for one little piece
21:44
that's one movement. It is ideally all of the
21:49
movements together that yeah, conducting an orchestra, not
21:55
just like playing the violin. Basically, yeah, basically,
21:58
yeah, totally. And that's where all of like the
22:01
the key documentation comes into play.
22:03
They have this this diagram that everybody kind of knows about.
22:08
It's called a piping and instrumentation diagram.
22:10
And on there is your whole plant and every single control loop or
22:16
system or sensor, and it clearly articulates exactly how that
22:21
thing is working. You know, it's useful just to
22:25
make sure that the whole team, like the maintenance staff, the
22:28
management, the operators, they all understand it, especially
22:32
the controls people, right? The automation folks.
22:36
You need everybody to work together to make sure these
22:39
systems work well. And that diagram is one that
22:43
kind of glues all of them together.
22:46
And I haven't seen a similar diagram in like the AI or or
22:51
software space as much something that, you know, almost every
22:55
stakeholder who's doing something on this system can
22:58
relate to, umm, to kind of help them integrate into the system
23:04
itself, right? Like what am I doing?
23:06
What am I offering this thing? Ohh, I see where you're taking
23:09
this. Hold on, let me see if I
23:10
understand this because there's a huge jump there that I like
23:16
where you're going with it. It's basically saying in the
23:19
plants everyone speaks the same language, which is this diagram.
23:24
Everyone knows if there's something going on, they can
23:27
refer to the diagram and check out at least how the system
23:32
works and where the electrical parts are, where the plumbing
23:37
is, whatever it may be that is pertinent to their concerns and
23:41
their needs. But you want to generalise that
23:46
type of diagram and say, if we have these Milai use cases,
23:53
where's that diagram for what everyone, because there are so
23:56
many stakeholders in these ML AI use cases.
24:01
So you have the whatever the DevOps team, the dev Secops
24:06
team, you have the platform team or the data scientist, you have
24:10
the ethicists or the whatever regulation, and everyone has
24:16
their questions. Everyone wants to understand how
24:18
the system works and so you can have one of those types of Rd
24:25
maps. Not necessarily Rd maps, just
24:26
diagrams on the system. Yeah, the end to end system,
24:31
what's going in, what's going out and what happens in between.
24:34
It's like our typical ETL thing, right?
24:36
I love UTL for that. You can generalise it to any,
24:39
but the key point to get that diagram is to understand what
24:45
you're optimising at each stage. So it's essentially your boxy
24:48
diagram. Everybody knows boxy diagrams,
24:51
it's just has standards wrapped around it.
24:54
What means what? So we do need that end to end
24:58
thing and we need each group, each person in the group to
25:02
understand what they're optimising for, what's my
25:05
objectives and have those clearly articulated in the
25:09
diagram so that others can see and contribute to it.
25:12
And you know, it identifies dependencies.
25:15
If we're trying to troubleshoot the system, it allows us to have
25:18
a common point to reference me. Like, yeah, it's failing there.
25:22
This is the potential reasons why.
25:25
Wow, I had never seen something like that.
25:28
I do like it a lot. I wonder if it since the hard
25:32
part is how technically detailed do you make it for someone who
25:36
has a business stakeholder? The business stakeholder would
25:41
have their own set of inputs, I would think, right?
25:43
Like they'd be probably sponsoring the programme,
25:46
wouldn't they? So they're the ones that are
25:48
hopefully helping us identify the outputs at least.
25:52
Like what are my expected outputs?
25:53
Do I need inference to happen right now in a cup in a couple
25:58
seconds or under a second? Or can it wait?
26:00
Is can it be done in batch? What should it look like?
26:03
What are the you know, it can help us understand.
26:05
Just like a gas plant, you don't wanna be putting out crude oil
26:09
at the output. You want something refined that
26:11
I could put in my gas tank, so that's where the business
26:14
stakeholder would come in. It's like at the higher level,
26:18
what's going in, what's going out, you know, are you guys
26:21
overcooking this solution or not?
26:24
So each stakeholder would have an input, not necessarily a
26:27
technical one, but they would still have something and they
26:30
should be able to understand, you know, at least the different
26:33
stages and about what's going on in there, I would expect.
26:39
It feels like what you're, what you're talking about is much
26:43
more conclusive or it is much more detailed because you have
26:49
what the business stakeholders are looking for, for the output.
26:53
You also have the technical side of things and how that looks,
26:57
but you're able to have a strong representation of the whole
27:02
system, especially if it's a complex system.
27:05
If it's one of these, you know, just make an LLM API call and
27:09
then you're good. It may not be needed, but when
27:13
it is a very complex system, you want to have a deep deep
27:19
understanding and have a strong diagram that everyone can
27:23
everyone can reference. Yeah, especially because, well,
27:29
I hope every single one of these products that we build, well
27:32
collectively we but they have business impact, right.
27:35
If something goes wrong, it's not going to be good.
27:38
You know, somebody you could have risked everything, you
27:41
know, society environment that profit all of that.
27:45
So having a diagram like that will help you identify the
27:49
critical control components, right?
27:51
Like during your ML OPS or LOPS processes, what are these things
27:55
that we're looking at that would indicate a failure of some kind
28:00
and you know, back to your business stakeholder, what would
28:03
they have input over it? Well, those would be numbers
28:06
front and centre that we're really concerned about.
28:08
If these conditions go over their prescribed values or
28:13
whatever, we have an emergency situation, meaning we have to
28:16
either revert to a different version, do some triage.
28:22
I mean, you know, we've probably all been there, done that, but
28:26
knowing what those control points are, having a process
28:29
that is helping you through that like an emergency response plan,
28:34
right? We have those in heavy industry
28:36
for every single potential case. In the same way, we probably
28:40
should have something like that for these products that are
28:45
being delivered, right? And mass volume, you need to
28:47
know right away and what to do about it.
28:51
And everybody needs to know what to do, or what's being done at
28:54
least. It's like this levelling field
28:58
where it's clear, just as you said with the factories you have
29:04
the electricians, you have anybody that is trying to figure
29:09
out what's going on, they can reference this diagram.
29:12
And so you could potentially have something like that with
29:15
your AI project or your AI product and anyone who is trying
29:21
to figure out what's going on with it or what it looks like
29:25
under the hood, they can reference this like one source
29:28
of truth, hopefully. Yeah, yeah.
29:31
And those are usually, you know, you might say that's the docs
29:35
that you know, you typically find for a product.
29:38
You might, it might be some of the, you know, artefacts that
29:41
are being developed throughout the development and planning and
29:44
stuff like that, stuff that the product manager engineering
29:46
teams might build. You might be like, oh, we have
29:48
these already. But you know, if if you were to
29:52
get, say this was some regulation like you were
29:55
regulated to, just like you were in a plant to know your data
29:59
flows, to know how much information you're pushing
30:01
through a given pipeline to mitigate risk, right?
30:05
I don't think many people would be able to answer those
30:08
questions and like a succinct, succinct manner, right?
30:12
One after another. What's first, What's second?
30:14
What's third? What's 4th?
30:17
In order to understand that you need the whole team in the room.
30:20
Whereas if you have a, yeah, you know, diagram like this with
30:24
proper O's on what to do, it's already covered as part of our
30:28
like compliance and risk mitigation plan is just to have
30:31
this. So everybody has a common
30:32
understanding what's going on around here because I just some
30:39
feedback from colleagues kind of informally now, nobody's really
30:43
sure what A&M's really doing, right?
30:46
We're taking data from someplace go to model cool.
30:49
We put it through the model, we got this output.
30:51
Cool. And then they, you know, they
30:54
can help wrap a process around it to make a good decision, but
30:58
it's still a black box for the most part.
31:00
And if you can bring comfort to them, you know, being like, I
31:04
did my due diligence, I got these diagrams, I got, what does
31:08
a shutdown look like? What happened?
31:09
What are the catastrophic things that can happen potentially, if
31:13
you can show and demonstrate that you've thought these things
31:15
through, we've got some risk mitigation and place.
31:19
Here's how it's being done. Get a little bit more buy in
31:23
from that perspective. Um, now I'm mostly focusing on
31:27
that because of the industries that I'm in, right?
31:29
Like there's, there's risk if we replace $1,000,000 chiller and
31:35
we didn't need to, right? Like we do that for preventative
31:39
maintenance, trying to predict it and it's wrong and we just
31:43
trust it. Well, not great.
31:46
I do like the idea of how you're saying that anyone.
31:51
You could basically go to anyone and ask what's going on here and
31:56
they should be able to know because you have that diagram
31:59
and you have these like standard operating procedures.
32:02
And also you're able to identify what the key metrics are that
32:07
everyone cares about and what the metrics are that you're
32:11
going to be watchful of. So in case something does go
32:17
wrong, you know that, alright, our metrics, if they're starting
32:21
to show these numbers, we need the escape route.
32:26
We need to figure out like, what are we doing rolling back what,
32:29
what's going on? It's not just some kind of a, a
32:32
call to the engineer, the one engineer that built the system
32:38
in the middle of the night. And it's like, hey, things
32:42
aren't working. And then that one single point
32:44
of failure, right? Like can be a liability for
32:49
sure. O yeah, I I like this
32:53
organisational piece because it shows the maturity and also it
33:00
gets you the ability to champion for things with a little bit
33:05
more transparency within your organisation if you are trying
33:09
to bring some use case to see the light of day.
33:14
Yeah, totally. If you can start with these kind
33:16
of basic things, you can be pretty vendor agnostic when
33:19
you're choosing the actual solution, right?
33:21
Like if you're at the very beginning stages of some use
33:24
case, you have an awesome idea for your company, you're going
33:27
to draw it up, try to get buy in and sell that idea so we can
33:31
start getting it funded and build it succinctly.
33:37
Being able to relay this kind of information brings comfort to
33:41
people. It's like, yeah, you, I'm gonna
33:42
give you a bunch of money to build something.
33:45
I kind of trust you more with this money because you've kind
33:48
of thought through what could happen and what should happen.
33:53
And that's kind of common in our, you know, product
33:56
development life cycle. We do all sorts of requirements
33:59
gathering and things like that, but when you bring in the
34:04
engineering folks, the data scientists, and they can also
34:07
back you up from a common point of view, it kind of gives you a
34:11
little bit more backing, right? It kind of shows that we're kind
34:14
of together on this. We all kind of understand it and
34:17
where it could fail. Yeah, get everyone on the same
34:20
page. I am fully on board with that,
34:23
right? Real fast.
34:24
I want to tell you about the sponsor of today's episode, AWS
34:27
Tranium and Influencia. Are you stuck in the performance
34:30
cost trade off when training and deploying your generative AI
34:33
applications? Well, good news is you're not
34:37
alone. Look no further than AW's
34:40
Tranium and Influencia, the optimal infrastructure for
34:44
generative AI. AWS Tranium and Influencia
34:48
provide high performance compute infrastructure for large scale
34:54
training and inference with LM's and diffusion models.
34:58
And here's the kicker, you can save up to 50% on training costs
35:04
and up to 40% on inference costs.
35:06
That is 50 with A5 and A0. Whoo, that's kind of a big
35:12
number. Get started today by using your
35:15
existing code in frameworks such as Pytorch and Tensorflow.
35:19
That is AWS Tranium and Influencia.
35:22
Check it out. And now let's get back into the
35:24
show. There is another direction that
35:28
I wanted to take this because you mentioned stuff about like
35:32
satellite imagery and external statistical data and market
35:37
trends and working with that. I know it's a hard left that
35:42
we're taken from the organisational peace and the
35:46
system diagrams and this product management, but can you go down
35:51
that rabbit hole for a minute? That's a lot of information,
35:55
hey, Like satellite imagery. Well, not just satellite
36:00
imagery, but what's on the top, but what's underneath.
36:03
We do like seismic. There's seismic exploration and,
36:07
you know, there's Lidar data for the forestry industry and things
36:10
like that. So being able to understand the
36:13
environment, those are all your inputs for these Heavy
36:16
Industries, right? Like the raw resources.
36:18
And typically we've done, you know, spot studies, you know, do
36:22
a survey from here to there to try to understand if there's
36:26
opportunity for, you know, say, logging or oil exploration or
36:30
whatever. I haven't personally built
36:33
anything along the lines of satellite imagery, but I have
36:36
some colleagues that sure have and they get right excited about
36:39
it just because you can understand things so much
36:44
faster, right? Having your typical analysts
36:47
kind of reviewing pictures or videos or just core samples and
36:51
stuff like that. It's laborious and boring.
36:54
You know how much coffee you have to drink to get that done.
36:57
But pushing it through some AI applications and machine
37:01
learning, like especially vision, we can kind of
37:04
understand these things at scale and at least get closer to the
37:08
intended outcome versus before. We're just augmenting people in
37:12
general. Yeah.
37:15
So this is these are use cases where you're trying to figure
37:19
out if a certain site you can do X on it.
37:24
And maybe it's logging, maybe it is building something, maybe it
37:28
can be putting UA wind turbine if.
37:32
And so you're getting all this data and normally the human
37:39
would have to manually look through all this data to figure
37:43
out if it is a viable place to go logging or to build a wind
37:48
turbine. But now you're this use case is
37:52
another one of those high value use cases because you're able to
37:58
augment the human's ability with machine learning.
38:01
Is that what I'm understanding? Yeah, pretty much it's
38:05
essentially anywhere where there's like copious amount of
38:08
information that's kind of a bear to deal with, like the
38:12
controls data we were talking about earlier or you know,
38:15
hectares and hectares of Lidar about a forest which what kind
38:21
of species are there? Is there any disease?
38:24
You know, they can, you can help learn more about the forest and
38:28
it's kind of ecosystem just just by firing a few algorithms
38:33
through and interpreting it on the other side with, you know,
38:36
subject matter experts who can also validate the output of this
38:40
machine learning and kind of augment it with their
38:42
professional opinions and whatnot.
38:44
So it's not like we're removing any of the professionals.
38:47
We need them. They're the ones with the stamps
38:50
usually, right? Like this is a quality
38:53
assessment. Here's the cheques and balances
38:55
we've done, all that sort of thing.
38:57
It's just allowing us to do a lot more, a lot quicker.
39:00
And from what it sounds like to me is that a lot of this stuff
39:05
that is happening it, it's not necessarily one off, but it's
39:11
not like it requires a lot of infrastructure behind it for the
39:17
technology side and like the mill side.
39:19
Am I wrong in saying that? Not a lot of infrastructure.
39:24
You mean like we're doing, you know, batch here or there.
39:26
It's not a continuous kind of integration situation that
39:31
you're. Yeah, exactly.
39:33
Yeah, yeah. So like.
39:34
You're not needing to figure out like the surveying, low latency
39:39
surveying, you kind of are doing it almost all offline and that
39:45
makes it a little bit easier to deal with.
39:48
Yeah, totally. There's some like real time
39:52
drilling applications and like the oil and gas industry that
39:55
kind of help help them navigate drilling to the reservoir.
40:00
And then like a low impact kind of manner.
40:03
I know of a few applications using AI directly on the drill
40:05
heads to measure the pressures and temperatures and figure out
40:09
what what's going on down there. Is it what I think it is?
40:12
You know, kind of deal. And so in those cases, that is
40:18
100% like just signal data that's coming, just sitting down
40:23
in a hole. And yeah, it's trying to put
40:27
together like you're seeing it on a screen, but you're also,
40:30
you've got a sidekick that's saying like, yeah, all looks
40:33
good to me. Or maybe you might wanna
40:36
reassess. Yeah.
40:39
Yeah, exactly. Yeah, Most of them, like AI, ML
40:44
use cases that I've seen quite a big value revolve around the
40:48
assets themselves, right? Like the turbine that comes with
40:53
the I, the IoT sensors that allow us to, you know,
40:56
understand safety and whatnot. The enterprise ones which you
41:01
know, you find typical of prioritising capital investments
41:05
or the project management assistant kind of thing.
41:09
Those are all kind of they're, they're less interesting to me.
41:13
They have a bit of impact, but where the real fruit's gonna be
41:17
coming from in the near future is the, the assets, the
41:19
equipment, the how. How does, how do we make these
41:22
things work better than they have in the past?
41:24
Or how can we augment the people, maintaining and using
41:27
them to do it more effectively, efficiently, protect our planet,
41:32
protect the people, that kind of thing?
41:35
To this is so cool to think about.
41:37
What is something that I did not ask you yet that you wanted to
41:43
talk about? There's a few like lessons that
41:47
I've learned from these Heavy Industries and they're like
41:50
mostly back office type things, but they've helped me out of
41:53
some pretty big problems, right? When you're first formulating
41:57
products or trying to find the best use case, really, right.
42:02
A lot of times my academic partners or you know, super
42:07
smart folks, the data scientists, they know all the
42:09
algorithms and whatnot, feeding them as much.
42:14
It's like an LLM, right? You want to feed as much context
42:16
as you possibly could fit into the window to get it give you
42:21
the best answer, right? So I always try to feed the
42:26
people I'm working with as much information about the system,
42:29
the business process or whatever as possible.
42:33
And the concept of a control philosophy is really hit me hard
42:37
lately. So in in these big gas plants or
42:41
facilities or whatever, you have a control philosophy.
42:43
It's just a document. It's like a requirements
42:45
document kind of thing, but it outlines your operating
42:49
principles, a control narrative, like what's happening from start
42:53
to finish kind of thing. Has all the diagrams in there,
42:57
like that piping and instrumentation diagram we
43:00
talked about earlier has what do we do if things go wrong in
43:04
there? You know, all sorts of stuff
43:06
about scalability and data quality, like help them
43:11
understand what's important from the business perspective.
43:13
Because you know, when you send the data scientists off to do
43:17
EDA on some database, that database might have hundreds of
43:20
tables and like so much stuff, how are they supposed to
43:23
decipher it all? We're going to run some
43:25
algorithms to find out what's important now.
43:29
So I like to bring the control philosophy to the table to help
43:33
narrow that scope. And like, here's your
43:36
optimization objectives. This is what we're trying to
43:38
achieve. And then let them loose on it as
43:41
well, so they they start adding to it so that we have this body
43:44
of knowledge about a specific problem.
43:47
And I've used a lots of times. Sometimes that particular idea
43:52
is successful and we start doing something about it.
43:55
And other times the data scientists or engineers are
43:57
like, no, like that. We can't even get data for that.
44:01
Or, you know, you're kind of out to lunch with some of these,
44:05
like the control narrative. That's not going to happen.
44:07
That's getting from A to B is not possible, right?
44:11
Like how are we supposed to extract all of this data from
44:15
like a remote station when there's no connectivity there?
44:19
I mean, it doesn't make any sense.
44:21
So bringing that control philosophy to the table that
44:24
just kind of narrows the scope and narrows the understanding
44:27
and for the team kind of set it set me up for success quite
44:31
often. So at the end of the day, you
44:34
want to make sure that whoever it is, you're working with, the
44:37
data scientist, let's say in this case, they understand as
44:42
much as they can about the problem so that when they do
44:45
create the solution, they're creating the proper solution.
44:49
And this goes back to, I think something that I've heard a
44:55
tonne on here, but I typically reference one of the first
44:58
conversations we had with Dan Sullivan on how he mentioned he
45:05
spent think 2 weeks or three weeks in one of his jobs, one of
45:10
his early jobs as a data scientist trying to improve the
45:13
F1 score. Only to emerge from this cave 3
45:18
weeks later and find out that the whole solution that he
45:22
proposed, it didn't matter if it was 100% accurate or if the F1
45:26
score was whatever, because the solution itself was so far off
45:32
the mark that it it wasn't valuable.
45:35
And so you're trying to, I like this idea of giving them as much
45:40
context, just basically context stuffing in their context window
45:46
and making sure that they understand here's the most
45:49
important pieces. And this is what would be a huge
45:54
outcome for us. This is what we're trying to get
45:57
to. And they can also stress test
45:59
any of your assumptions on, Oh, well, this data we don't have,
46:04
or this data is not clean, or this data we just can't get even
46:07
if we wanted to, those types of things.
46:12
Yeah, exactly. It allows you to challenge each
46:15
other's ideas pretty easily. That's kind of the whole point
46:18
of that piping and instrumentation diagram that
46:20
allows you to debate amongst the professionals what is the best
46:26
and get people to have a say at the table from a common point of
46:29
view. Because I found it's pretty
46:31
relative. A lot of these business
46:33
problems, you know, sometimes they're based on experience,
46:37
sometimes you're brand new, you're coming into it fresh, so
46:40
you have a different perspective than the guy who's been there
46:43
for 20 years doing that process. So if you're going to have both
46:46
of those guys at the table, they better have a common viewpoint,
46:51
like something they can reference and be like, yeah, we
46:52
need to change this or that so they don't feel like they're
46:55
battling each other. It's just it's just this
46:57
document. We're just trying to get this
46:58
document as accurate as possible and then we can build.
47:04
It's real easy to pull the trigger and start building, but
47:07
building in the wrong direction will cost you, you know, time,
47:10
money, if we put it into the production customers probably.
47:14
Yeah, trust, Yeah, trust with your customers for sure.
47:18
So this is the control philosophies.
47:20
What else did you want to talk about?
47:22
I've been interested in that, this quality conference that
47:26
you're kicking off and you know, we see regulations popping out
47:31
here and there around data and AI and privacy and all this
47:35
stuff. I'd like to dig a little bit
47:38
down into quality, I guess, you know, are we concerned just
47:44
about the AI, just about the ammo, just about the data
47:47
platform? AI quality is really about like
47:52
the outcomes, isn't it? Yep, exactly.
47:57
That's what. And so I think for me, I don't
48:01
know if you've been feeling this, but data quality has
48:04
become something very popular in the data engineering space
48:09
because there's been so much pain around it for the data
48:13
engineers. Every data engineer I think in
48:16
existence has had to go and try and debug some kind of a
48:20
dashboard or they've been approached by an analytics
48:24
engineer and they say like, hey, what's going on with my data
48:27
here? This dashboard isn't working.
48:29
And then the data engineer has to go and figure out why that
48:32
is. And that's a data quality issue.
48:36
So it resonates pretty well when it comes to data engineers.
48:42
Now, AI quality is something that for me, it feels like if
48:47
people haven't had experience with it yet, they will soon.
48:51
Especially when we're putting all these random agents into
48:55
production or we're putting all these rag chat bots into
48:58
production and you hear the horror stories and you're trying
49:03
as much as you can to make sure that you don't land on the front
49:07
page of the Internet. But how do you make sure that
49:13
you can properly derisk that scenario, right?
49:19
And so that for me is the the AI quality piece and and ML is
49:25
this, it's in the same boat, right?
49:27
And a lot of this is organisational, being able to,
49:33
like you're talking about, like, explain to stakeholders, if we
49:37
implement this type of technology, we are at risk in
49:46
these different ways because of whatever it may be.
49:50
Like, beyond just hallucinations, it can be that
49:55
some nefarious actor is trying to do some prompt injections.
50:00
And. Yeah.
50:01
Exactly. Ready for that?
50:03
Yeah. Or you don't understand the
50:05
problem completely and now the input isn't what you expected
50:08
and it's not really been working.
50:10
I kind of liked, yeah, John Cook's approach with knowledge
50:13
graphs applied to the business processes so that the engineers
50:18
and the technical people can get access to that data, as well as
50:22
all the other business benefits of a knowledge graph for your
50:26
business. I think things like that will
50:29
see creeping up more and more as as we see compliance being a
50:34
real thing or even risk to the business outcomes be a thing,
50:38
they're going to look for solutions to quantify and
50:42
document their business. Because like I said before, the
50:45
data is the the differentiator. You know, you might have this
50:48
technology or that model or whatever, but the data is your
50:51
secret sauce for your business. It's going to be interesting to
50:55
see how it levels the playing field.
50:58
But if you don't document those well enough, how can you expect
51:01
your AI and ML to operate on top of your business process, which
51:05
might be fragmented Excel sheets, right?
51:08
So getting that somewhere is, I think, the key to quality here.
51:13
It's so funny you mentioned that and it reminds me of a piece,
51:18
something that I saw somewhere online that was referencing how
51:26
you've got this rag chat bot just talk with your data use
51:34
case, but it turns out your data is absolutely shit.
51:38
Like the documentation hasn't been updated in years so the
51:42
chat bot is basically worthless. Yeah, yeah, it's kind of cool.
51:47
I think it's actually a real plus to have these things kind
51:50
of show up and be in our face because like been fighting for a
51:55
long time on data quality and sometimes the business doesn't
51:58
want the data quality to be improved, right?
52:01
I don't know how many people I've had this experience, four
52:04
where you do this analysis, you're like, ohh, cool.
52:06
We can save a lot of money if we do X or Y.
52:10
You take it to the business and they're like, yeah, but we're
52:13
just gonna put that under the carpet and not talk about it
52:15
again, right? So, you know, having these
52:18
things come up front and centre I think is kind of important.
52:21
It makes us ask these questions of like, Oh yeah, I guess we do
52:24
need to do that that analysis, or we do need to document that
52:28
data about whatever process is being used by this machine
52:32
learning algorithm so that we can feed it good data.
52:35
What a great point because it's so in your face.
52:39
Yeah. So in your face, like if you're
52:40
gonna try to get an ML algorithm to classify something and to say
52:45
four classes or something, but the input is always changing.
52:49
Like we're always getting numbers, letters, anybody can
52:52
put in whatever. Like you put an essay in there
52:54
if you want to, you know, we're just going to end up with
52:57
garbage. So understanding these things
53:00
and getting the business on board to go back to that
53:03
whatever platform is creating that data and be like, no, we
53:05
need 4 things in there. And now we don't need this mill
53:09
algorithm to do this classification anymore because
53:12
we are on top of our data quality.
53:14
Yeah, you don't need the Band-Aid downstream.
53:17
Yeah, the Band-Aid, yeah, we're using ML and AI as a Band-Aid
53:20
sometimes, but in a positive light, I guess, right.
53:23
If it if it brings together the conversation and an ultimate,
53:26
you know, resolution of that data quality problem, I think
53:29
that's a plus. One of the reasons why I am so
53:34
bullish on data engineers and I am so empathetic to the data
53:40
engineering job in general because man, they got it hard.
53:45
They yeah, definitely have some job security, that is for sure.
53:50
Yeah, just a bit. It's like, well, I got all your
53:54
data, So what are you gonna do? But but they also have to know.
53:59
Because now. Yeah, where it goes, where it
54:03
flows, who's using it. But like the the problem with
54:08
that is usually they don't own the data, right?
54:10
They have no control over a source applications way to
54:14
process or gather the information, right?
54:17
So a lot of the times it's the please and thank you process.
54:21
Work with me kindly. Please don't change your
54:23
attributes crazily or your schema or whatever, right?
54:29
Yep, poor data engineer. And if you do, yeah, man, you're
54:34
preaching to the choir. That's why we are making Data
54:38
Appreciate Data Engineering Appreciation Day.
54:42
Think I mentioned this once on like it.
54:45
It definitely needs to be a thing if it is not, but go and
54:50
say thank you to your data engineers that you're working
54:53
with because they just get, they get hammered, man.
54:57
Yeah, if you report on KPI's or use reports or whatever you're
55:02
doing, just thank you, Did engineer.
55:06
Yeah, at the end of the day, your decisions matter.
55:08
Who touched Ada? Yeah, yeah, I like that.
55:14
Ohh. Well, dude, Richard, this has
55:15
been awesome, man. Thank you for coming on here and
55:19
explaining this idea of control philosophies and the standard
55:23
operating procedures. Also just the big diagrams,
55:27
maybe some people are putting it into practise already and if
55:30
they are or they're doing something similar to this, just
55:33
calling it a different name, I would love to hear from those
55:36
people and how they have been finding it.
55:40
It feels like it is something that a very mature data slash
55:45
AIML team would be doing, so I hope that somebody's doing it
55:50
and those people that are doing it would love to hear from.
55:54
Yeah, me too. I it's just such a fundamental
55:57
practise in physical engineering of machines and stuff like that,
56:02
that I think that we could learn something from that in this data
56:05
and AI space because we also want to build reliable systems,
56:09
really reliable systems. That's a good way to do it.
56:12
That's a great one. Awesome dude.
56:15
Well, thanks for coming on here. Yeah.
56:17
Thank you so much.