Latest / Upon Further Inspection / Episode 14 - Stood the Test of Time (featuring Clay White)

Transcript

Episode 14 - Stood the Test of Time (featuring Clay White) · Upon Further Inspection

0:00
Nick: Hey, I'm Branden.
0:01
Nick: Hey, and I'm Greg.
0:02
Branden: This is upon further inspection.
0:04
Branden: We got Clay white with us today.
0:23
Greg: so, let me ask you this.
0:25
Greg: Yeah.
0:26
Greg: So can you share a little bit with our listeners about your current activities and then also, how did you ever get started in the world of mechanical integrity?
0:36
Clay: Okay, yeah, I'll give you a bit of background.
0:39
Clay: So my degree is in welding and metallurgy from a and m.
0:42
Clay: I really hadn't worked too much in the oil industry, even, you know, before going to, to college.
0:49
Clay: I did end up taking a, job that kind of got me into materials, corrosion, engineering side of things.
0:56
Clay: early on in about my junior year in college, I had a job at, the Tennessee Gas Transmission Company, which Tenneco, in Houston.
1:07
Clay: They had a lab where they did failure analysis.
1:09
Clay: And so that was kind of my segue into getting interested in corrosion and materials.
1:15
Clay: And, uh, found it kind of interesting and fascinating and I, you know, I always worked on cars and built stuff and, and encountered a couple of failures early on and tried to try, was trying to figure out why in the world it felt.
1:26
Clay: And so I guess that was kind of my catalyst for getting into the corrosion materials side of things.
1:31
Clay: after coming out at, out of a and MI took, took a job in a materials lab for a company called FMC, which owned, you know, 30, 40 different different companies including
1:42
Greg: Yeah,
1:42
Clay: wellhead divisions and mining divisions and a ordinance division out in California.
1:47
Clay: And, so I, I actually worked in a, in a materials lab doing failure analysis and that kind of stuff for the first five or six years of, of my career.
1:57
Clay: and then, I took a job with Exxon.
1:59
Clay: Coming back from there in chemicals.
2:01
Clay: and then, had a few other stents, but ultimately ended up doing, you know, inspection, supervision, starting at Exxon and, coercion materials.
2:10
Clay: And actually they kind of needed more of a mechanical engineer.
2:13
Clay: So I really did more mechanical engineering early on as a fixed equipment engineer.
2:17
Clay: And then, about that time, that was late eighties and then early nineties I started in with API and meeting our friend John Reynolds, recruited me, to help him on, I don't remember what aspect we were working on, 5 81, but I had, I helped, head up one of the, um, section writeups around, uh, corrosion and ran the subgroup on that.
2:39
Clay: And, so, goes back about the same timeframe when I met, Reynolds.
2:43
Clay: Okay, Len, and then that was about
2:46
Greg: 92, 93, about that timeframe.
2:49
Clay: Yeah.
2:49
Clay: Yeah, yeah.
2:50
Clay: About, about when we started should, maybe shortly after we started, 5 81, kind of, RBI, work and approaches.
2:57
Clay: Mm-hmm.
2:57
Clay: At, at one point I actually wrote a version before the API 5 81 program was released through DNVI actually wrote, tried to follow a similar pattern and wrote a, an RBI program that, a company called AB Tech ended up using at a couple of sites.
3:14
Clay: I don't know if they still, I think that actually is still in use, today, but I haven't kept up with that one.
3:20
Greg: Yeah, I remember seeing that in the field.
3:22
Greg: yeah.
3:22
Greg: A lot.
3:23
Greg: And when we bid on that original A-P-I-R-B-I contract at DNV, app Tech was one of the bidders as was a SME.
3:34
Greg: And, what is it?
3:36
Greg: Lawrence Livermore Labs or somebody like that.
3:38
Greg: don't hold me to that, but, yeah, it was somebody out, out there, you know, west Coast.
3:43
Branden: Yeah,
3:43
Greg: yeah,
3:44
Branden: why did you end up going and writing a separate, well, there
3:46
Clay: wasn't, it wasn't separate at the time.
3:48
Clay: There wasn't any competing software at the time, so what we were looking to, to try to start to implement it, we had a, I guess it goes back to, when, when AB tech kind of recruited me to, to, to work for'em and do this.
4:01
Clay: we knew the A-P-I-R-B-I tool was coming, but it was still not released yet, and it wouldn't have been released for another two years or so.
4:10
Clay: I think the first version that we actually tested in trial.
4:15
Clay: So it was, it was just in advance of anything being available.
4:18
Clay: So I wrote the mechanical side of it, and a guy named a, a process engineer, fantastic guy named John Young, wrote, he was a, a very senior, process engineer.
4:27
Clay: I think his, most of his career and experience was with, Dow or DuPont.
4:31
Clay: I don't remember which now.
4:32
Clay: But he, he wrote the, the consequence side model.
4:36
Clay: I wrote the likelihood side model, and we paired it together.
4:39
Clay: So it was just a spreadsheet based version.
4:41
Clay: some of the data that was available from the early work at 5 81, we incorporated some of those concepts and, and then, and then we, and then it kind of, you know, developed a life of its own.
4:52
Clay: But it probably predated the API release by about two years, I guess, give or take, something like that.
5:00
Greg: Would you consider it qualitative, quantitative, semi quantit.
5:04
Clay: semi-quantitative.
5:05
Clay: It was, you know, we were actually trying to calculate the probability of failure, to, to a degree.
5:12
Clay: But the, the likelihood or the likelihood of failure in that case, not a true probability, but, but we were actually using real corrosion rates and data and tolerance, damage tolerance of the equipment, to, to come up with a, likelihood value for the equipment.
5:29
Clay: And, on the consequence side, you know, we reason all the same kind of parameters, but it wasn't anything as sophisticated as the 5 81 model.
5:36
Clay: Try, you know, calculating, you know, blast pressure and, you know, areas and, flammable effects and, and all for five different kinds of fire, damage.
5:47
Clay: You know, we didn't, we didn't get into it that.
5:49
Clay: It wasn't that detailed or sophisticated, but it, it led it down the same path and, or a similar path to coming up with a, a risk based on a likelihood consequence model and then, and then, the inspection side of it.
6:02
Greg: Yep.
6:03
Greg: I certainly remember seeing it around and it seems like it, it got a little traction on the A SME side too.
6:09
Greg: I'm not so sure if some of the work there didn't morph out on the Oh, I dunno.
6:14
Greg: Yeah, it could, could've been because Ron Leonard was heavily involved.
6:18
Greg: He was, Ron was our first side.
6:20
Greg: Yeah.
6:20
Clay: Yeah, Ron.
6:21
Clay: Ron, that promoted it.
6:23
Clay: one of the first, RBI assessments we did was for Ron at his, massive, chemical plant up there.
6:29
Branden: Yep.
6:30
Branden: since you were in charge of the mechanical side, how did you go about the likelihood piece?
6:34
Branden: what was your starting point?
6:36
Clay: Well, it was, it was, the damage rate, either expected or measured damage rates.
6:40
Clay: and then comparing that to a, a model for the location and what the damage tolerance of the equipment based on its design was.
6:49
Clay: So we were, we were doing some, some math in terms of, you know, how thin of an area could we get, before it would potentially fail.
6:57
Clay: And as we got closer to failure, we were essentially signing a damage factor similar to what a 5 81 did.
7:04
Clay: So mm-hmm.
7:05
Clay: that's how we did it.
7:06
Branden: Okay.
7:07
Branden: I was just curious because I, I wasn't sure if you'd gone down the GFF route and had kind of a starting point'cause you've been, you've been on, on record being pretty, vocal about gffs as well.
7:16
Branden: Yeah,
7:17
Clay: I'm not a huge fan of the, of the, uh, life fraction model for a number of reasons.
7:23
Clay: you know, later of course, when I've given several presentations at 5 81 trying to promote, some changes and, several of them are kind of based on the problems with the GFF.
7:34
Clay: it's not, it's not a particularly accurate way to determine a probability of failure.
7:40
Clay: So when you start with a, number.
7:42
Clay: And we, and if you know the background right, we, we haven't, in API, we have not updated those numbers in 20 years.
7:49
Clay: And when we started 20 years ago, we didn't really have particularly good data.
7:53
Clay: We were using a couple of different sources for data, but it was more akin to offshore equipment, number one.
7:58
Clay: And number two, it wasn't particularly well calculated to produce an actual GFF.
8:03
Clay: If you, if you look at the, and for those that may listening, that may not know, GFF is generic failure frequency.
8:09
Clay: So they're basically trying to take an actuarial approach like you would on, um.
8:15
Clay: say a life insurance, right?
8:16
Clay: Where you're looking at an average lifespan of a, of a person and then you add factors relative to, to risk factors essentially that would, shorten or extend that lifespan, right?
8:29
Clay: So that's kind of the similar approach to what we did with the GFF and the damage factors, with 5 81.
8:35
Clay: But, you know, it doesn't take a lot of time to realize that the GFF numbers can't be right.
8:40
Clay: You know, you've got the exact same GFF number for multiple different types of equipment, right?
8:46
Clay: And, if you look at the magnitude of the GFF, it's not really in line with what we expect for failure frequency.
8:53
Clay: And when you compare actual, frequencies, I tracked.
8:57
Clay: 12 sites, towards the end of my career, which we didn't really finish talking about, but it was, I, I ended up at, Phillips 66 and I had it at the time of 14 or 15 different sites.
9:07
Clay: And so one of my jobs, for them at corporate as the director for their mechanical integrity programs was to publish, uh, uh, essentially kind of a high level roll up version of what kind of risk we had and how we were performing, you know, on MI programs and whatnot.
9:23
Clay: And, so we had a very detailed metric package where we were looking at all different types of failures and causes, how many clamps you had, how many leaks you had, those kinds of things.
9:34
Clay: And, comparing that data.
9:37
Clay: But, you know, from 12 different refineries, I knew the equipment counts how many different types of pipe and vessels we had, and then what, how many failures we had comparing that to either the GFF or the GFF plus a damage factor.
9:50
Clay: We were, we were using 5 81, at, at at, at the end.
9:54
Clay: it, you didn't get a particularly good comparison.
9:56
Clay: It was, it was probably, almost a order of magnitude off on a predicted damage rate, so predicted failure rate.
10:04
Clay: so I knew, we needed to think about a different way of doing this.
10:07
Clay: So
10:08
Branden: conservatively order of magnitude or non conservatively.
10:11
Branden: And did you see a large difference?
10:12
Branden: No.
10:12
Branden: It was just
10:12
Clay: too, it's too conservative.
10:13
Clay: Yeah.
10:14
Clay: It's, it's too conservative.
10:15
Clay: Yeah.
10:15
Clay: It's too conservative.
10:16
Branden: And were you seeing specifically
10:17
Clay: for piping?
10:18
Branden: Specifically for piping?
10:19
Branden: Yeah.
10:20
Branden: So that was gonna be my question is were, were you seeing large differences, right?
10:23
Branden: So you said all the gffs are, are the same, right.
10:25
Branden: Except for tanks.
10:26
Branden: And you were seeing that between the different equipment types, the different component types, that there are vastly different failure rates that that are Yeah.
10:36
Clay: From what we were experiencing Yeah.
10:38
Clay: Yeah.
10:38
Clay: so the answer then is do we, do we try to go back and try to fine tune the GFF or do we or we change the approach or basis, you know, which, some people are advocating how to go to a more of a true probabilistic model, which I like, it's just, it's more data, more time, more energy to kind of produce that, that level of a, of an answer.
10:56
Clay: And then of course, more recently, what's kind of gotten me, keyed up about AI is that one of, one of our prime contractors, for our industry is said, you know, well, we, we need to change the entire basis.
11:07
Clay: We ought to make it based on ai.
11:09
Clay: And when you base it on ai, you won't be able to regulate it.
11:12
Clay: And that's, that's what prompted the, the last presentation I gave in November, 2024,
11:19
Greg: clay, just for reference, as I was at DNV at the time, and I'm not defending the generic fed frequency approach in any way.
11:28
Greg: but, we got, we smashed together data from different sources, and two of the primary ones were the ARITA database.
11:35
Greg: ARITA database.
11:36
Greg: Yeah.
11:36
Greg: And, and the Marshall.
11:37
Greg: database, which was actually from the fossil fuel power industry.
11:41
Clay: Right, right.
11:42
Clay: Yeah.
11:42
Clay: Came to, ultimately they still produce a Marsh McClellan report?
11:45
Clay: Right?
11:46
Greg: yeah, the marsh McClenan But, but the other thing too, there was, I, I remember when you probably remember this too.
11:51
Greg: We were putting those tables together.
11:53
Greg: there was a lot more extrapolation and interpolation, in my opinion, on piping than there was on pressure vessel.
11:59
Greg: Well,
12:00
Clay: yeah.
12:00
Clay: And then piping too.
12:01
Clay: You know, the other thing they, we never did in on piping is piping was excluded from the, the absolute validation, uh mm-hmm.
12:09
Clay: Results and the mm-hmm.
12:09
Clay: Relative validation results.
12:11
Clay: We didn't, they didn't do anything on piping, so there was no, there was no smoothing or adjustment to any of the data for, for piping.
12:20
Clay: So it kind of, it kind of wasn't considered when we did those two.
12:25
Clay: reports, right, right.
12:25
Clay: From vessels and tanks towers.
12:27
Clay: Well,
12:28
Greg: and in fairness to you, AI wasn't anywhere near as prevalent as today, and neither was computing horsepower, and now with probabilistic and things we can do with computers, we can do that stuff a lot faster instead of telling the computer compute and come back the next morning and get the answers.
12:43
Greg: That was, that's
12:44
Clay: been my example, Greg.
12:45
Clay: I remember many, many times, the last thing I did before I went home at night was punch the calculate for on a, on a unit level.
12:52
Clay: Right.
12:53
Clay: Because I knew it was gonna be four, four hours and you hope to God you didn't have critical errors in it when you came back in the next morning.
13:00
Clay: You had to do it again the next night, you know?
13:02
Clay: Yeah.
13:02
Greg: And now we can run a few thousand Monte Carlo simulations and Oh, I know.
13:06
Greg: In a split second, you know, it's no,
13:08
Clay: the, the data, I mean the API 5 81 approach has stood the test of time.
13:12
Clay: Right.
13:12
Clay: it's done well.
13:13
Clay: It's just not particularly accurate.
13:16
Clay: And more of my passion in the last couple years has been trying to influence some change, improvement around the piping piece specifically because, you know, early on.
13:25
Clay: A lot of folks, you know, tried to do piping RBI and it, it didn't particularly work well.
13:31
Clay: So I think most people kind of abandoned it and went on, you know, like I did at multiple sites and, we tried it and didn't seem to work particularly well, so we limited it to just the vessels and, primary pressure vessels and we excluded pipe and we stayed on a half-life, kind of approach for piping.
13:49
Clay: But now in the last five years, I know, I know of at least four majors that are using, using RBI for, for piping, but, in looking at their approach on it, I was still concerned about, you know, the answers they might be given because of the specific approach they'd taken for piping.
14:07
Clay: You know, you were, they were still looking at averaging, you know, multiple components as a circuit and then plugging that data in, right?
14:15
Clay: Mm-hmm.
14:16
Clay: Mm-hmm.
14:17
Clay: And I've, I've showed in presentations at API, that that's just not conservative and
14:21
Greg: representative circuits, some people would.
14:23
Greg: Yeah.
14:24
Clay: Yeah.
14:24
Clay: AB absolutely.
14:25
Clay: And, if you stop and think about it, you've got, in any pipe circuit, if you've analyzed much data on piping, you know, that, you may have, you know, 20, 30, 40, whatever, TMLs on a, on, but they're across multiple different components.
14:37
Clay: Every component's got a different T men, even though the pressure rating for it was all class one 50 or whatever.
14:43
Clay: Mm-hmm.
14:43
Clay: But, you can't compare a corrosion rate and the root of a 3000 pound DRT to, you know, to, the 12 inch main pipe.
14:53
Clay: You know?
14:53
Clay: Mm-hmm.
14:54
Clay: Life, it, it just doesn't work well when you're trying to, analyze risks that way without recognizing the, the kind of the limiting factors, either the higher corrosion rate locations or the shorter remaining life locations.
15:06
Clay: So, and we've showed that data mathematically Right.
15:08
Clay: At API meetings.
15:09
Clay: we've showed that you, you can come up with a non-conservative future inspection relative to an actual predicted failure on an individual component within a circuit if you were just averaging it across the board without any other considerations.
15:25
Clay: Right.
15:25
Clay: So,
15:26
Greg: right, right.
15:27
Greg: You know, this is really not off base.
15:29
Greg: but, Clay, you and I used to have some of the, some conversations I used to really love when we were both together back in the day at equity.
15:35
Greg: But, this is something that's always bugged me, is working in the area of risk and risk-based inspection and me primarily fixed equipment reliability is how come we don't see more organizations when they come up with what they think is a good technical approach.
15:55
Greg: How come they don't go back and test it on past failures to see how well it would've done?
16:03
Greg: I don't see a lot of that.
16:04
Greg: Do you maybe I off?
16:06
Clay: No, no.
16:07
Clay: I, I think you're absolutely right.
16:08
Clay: Part of the, part of the reason is, you know, I don't know if it's a human nature thing or whatever, we don't.
16:14
Clay: We don't always learn well from our mistakes.
16:17
Clay: We don't always capture the data necessary to come back and, and revisit that in the future.
16:22
Clay: You know, we don't keep good data.
16:23
Clay: In some cases, they intentionally kind of purged it from their history, you know?
16:28
Clay: Mm-hmm.
16:28
Clay: I remember looking at a, trying to research a, a couple of failures and, the site had actually intentionally purged all of the data related to that failure.
16:38
Clay: I don't know if it was a, if it was a legal concern or they just didn't like that his past history and, wanted to make sure that nobody could dredge that buck up.
16:47
Clay: I, I don't know, Greg.
16:48
Clay: There's, yeah, that's always kind of
16:49
Greg: baffled me.
16:50
Greg: Same thing in RCM.
16:52
Greg: You know, people come along with all these big ideas that they want to charge you a gazillion dollars to implement and buy the software.
16:59
Greg: It's like, well, you, you knew when this failure occurred and you know everything around it.
17:05
Greg: Why didn't anybody go back?
17:07
Greg: And test.
17:08
Greg: 'cause there are times when we really, I think, do know everything around it.
17:11
Greg: Yeah.
17:12
Greg: And anyway,
17:13
Clay: yeah.
17:14
Clay: Cer certainly if you're running an RBI program and you have an unpredicted failure at your site, right?
17:19
Clay: Mm-hmm.
17:20
Clay: Where you, there's a reason you failed to predict it.
17:22
Clay: Right.
17:22
Clay: That's, we used to take, I used to specifically take metrics on, on this kind of stuff for, for years to try to improve our programs.
17:29
Clay: I certainly would advocate, you know, going back and, and looking at why was it missed, you know, for mm-hmm.
17:36
Clay: From an RBI approach or just a, you know, just a standard inspection approach.
17:39
Clay: What did you fail to predict?
17:41
Clay: was it, you didn't, didn't have the right operational data, you didn't have the right, the CML allocation or location specific kind of a problem.
17:49
Clay: For you failed to predict something that that would've allowed you to stay in front of it as a potential before it actually physically failed.
17:56
Clay: Right.
17:56
Clay: So,
17:57
Greg: yeah.
17:57
Greg: I, I actually have a theory about that and one of the best tools we have for continuous improvement, because I don't care how good the initiative is, there will eventually be a failure and that for sure, and, and we will learn from that failure to improve the program, but we've gotta capture the data and, and go after it intentionally, to make it happen.
18:20
Clay: One, one of my favorite quotes is, if you fail to learn from history, you're destined to repeat it.
18:24
Clay: Right?
18:25
Clay: So, yes, sir.
18:26
Clay: The other one that's a big issue to me too these days is, is around, kind of the, the current.
18:33
Clay: Trends to try to right size CMLs.
18:36
Clay: I, I liked your version, Greg.
18:37
Clay: You actually coined that phrase back in the day when we were at equity.
18:41
Clay: Remember the, that first work I did, you know, looking in and applying some statistics.
18:47
Clay: Back.
18:47
Clay: man, that goes back to about 2000 or 2001 maybe.
18:52
Clay: you know, a lot of people were, were talking about CML reduction, but, but you had coined the phrase, I guess either right?
18:59
Clay: Siding or right sizing or our
19:01
Greg: CML optimization was optimization.
19:04
Clay: Yeah.
19:04
Clay: I think that's the word.
19:05
Clay: We, we may add
19:05
Greg: more, we may take away, but, and we, and,
19:08
Clay: and we did, obviously we recommended in a couple cases you add a lot, a lot more.
19:12
Clay: Yeah.
19:13
Clay: The, but still, you know, that true, that seems to be more in vogue these days actually than when you and I were starting to do that kind of stuff back in, in early 2000.
19:21
Clay: But, these days, like I, every site I've gone to seems like they've had some kind of visit from a company trying to, trying tos true.
19:30
Clay: Trying to, and, and then in those cases it was more reduced than to, to, right size I, I suppose, but I have seen some really scary stuff on that one.
19:40
Clay: when I've gone back and pulled the old data compared, it, it really, really concerns me because, you know, if you're applying, you know, whatever formula you've got, you know, either you're just looking at it and deciding, I can, I can remove some of these.
19:54
Clay: Or you're basing it overly basing it on some, statistical routine, right?
19:58
Clay: looking at the data, how, how much can I reduce this and still come up with essentially the same equivalent, average likelihood, right?
20:07
Clay: for that equipment.
20:08
Clay: I've, I've seen some stuff that really worries me, to be honest.
20:12
Clay: So
20:12
Branden: same here, clay.
20:14
Greg: any pitfalls, clay, that you could point out?
20:16
Clay: well, one for sure.
20:18
Clay: if you're looking at it purely mathematical, right?
20:21
Clay: You think about, the math approach, right?
20:23
Clay: You, if you're, if you're deriving statistics based on existing data, what it precludes is, is do you have the right data?
20:31
Clay: What's the accuracy of that data?
20:33
Clay: And did you take the data in the right locations relative to the damage mechanism?
20:37
Clay: So those are, those are three reasons that just, just a data approach is not good.
20:41
Clay: You really want to make sure that you've got good quality data.
20:46
Clay: But, but even before that, that you've done a DMR, you understand what damage mechanisms are likely to occur, and you had a, a good inspection strategy relative to where the damage is occurring, right?
20:57
Clay: So if you don't have good data, you may have missed the worst case location.
21:00
Clay: Yeah,
21:01
Branden: entirely.
21:02
Clay: You know, the, the DMR is
21:04
Branden: a really important piece there.
21:05
Clay: That's absolutely, it's the start for so much of our work that really needs to be done on, all this stuff.
21:11
Clay: Right.
21:11
Clay: So, so, good, good.
21:12
Clay: Starting with a good DMR and then, you know, the thing that I've seen from, I've done lots and lots of auditing, in my career But, what I've, what I've seen from auditing is that, you know, a lot of sites will go through the, the DMR, but then they've never actually adjust their inspection program relative to what the DMR says.
21:33
Clay: They, they put the DMR on the shelf and not actually adjust their inspection plan.
21:39
Clay: Right.
21:39
Clay: that's not good.
21:40
Clay: That's, and that, but that seems to be a very common thing across the industry is they'll check the box DMR done, but they don't ever go back and change their strategy relative to the, to the DMR.
21:51
Clay: Now, obviously that's the, that's where the rubber meets the road.
21:53
Clay: Right.
21:54
Clay: And, and you don't, if you don't adjust where you're inspecting and why or how you're inspecting for damage, you're not gonna take the benefit of the DMR.
22:05
Branden: That's, I mean, for me, when you start talking CML optimization, and I, I talk about a lot with folks, it, the first thing, I call it a susceptibility review, right.
22:13
Branden: Because bringing back the RBI piece of susceptible locations, right.
22:17
Branden: you gotta figure out are the CMLs in the susceptible areas, and you can only do that from the dmr.
22:23
Greg: Yeah.
22:24
Greg: If your CMLs
22:24
Branden: aren't in the right area, there's no point in trying to add or remove.
22:28
Branden: It's moving them and then Exactly.
22:30
Branden: And then you gotta track'em for five years and see what data you can get.
22:34
Branden: You gotta have a couple data points in there.
22:36
Branden: So to, to go through the optimization effort.
22:39
Branden: Half of it, if people have, don't have, haven't gone down the path far enough.
22:43
Branden: Half of it is just doing a DMR and reviewing at that level.
22:47
Clay: Right, exactly.
22:48
Clay: Of course, you know, changing, adding CMLs and redoing drawings and re a circuit, you know, all of that is tedious work, right?
22:57
Clay: Yeah.
22:58
Branden: Yep.
22:58
Clay: It's, I know you can do it in the back end, probably a lot quicker.
23:02
Clay: I, I only really know how to do it from the front end of, of a couple of programs and it's tedious and time consuming and, it's not easy to accomplish, right?
23:11
Clay: So you, it, it costs, costs you to do it and, but it's the right step.
23:16
Clay: if you're trying to analyze the data you have on an existing circuit and you haven't done that step first, you're likely to get bad data.
23:21
Clay: You're gonna get at least for sure, kind of one tailed data.
23:24
Clay: You're only gonna see half the picture if you haven't been inspecting in the right location.
23:28
Clay: So that's, yeah, that's one of the biggest fallacies I see.
23:31
Clay: The other side of it is if you're just doing the statistical analysis, you ignore the fact that maybe I don't have all the right data in the right locations.
23:38
Clay: you know, people don't think through.
23:41
Clay: A lot of times I, I believe, how to do a good statistical analysis on the data, right?
23:47
Clay: And, and one of the first things you want to do is you want to determine, do I have a, based on what statistics you're trying to run, do I have a population of data that supports the statistical approach I'm doing?
23:59
Clay: Right?
23:59
Clay: So if I don't have, enough data to actually derive a statistical significance, then I'm, I'm averaging, you know, very few numbers, right?
24:10
Clay: Like, so in, in the extreme it's like, what's the slope of a line between two points?
24:14
Clay: You've got no clue.
24:15
Clay: It, it's not, it may not be anything close to a straight line, right?
24:19
Clay: So you really have to understand that piece of it.
24:21
Clay: It has to be based on the right.
24:23
Clay: Kind of damage mechanisms and taken in the right location.
24:26
Clay: Then you, on your statistics, you really need to look at if you're using those to, to do, like a sampling statistic, which I'm a pretty big fan of, of using.
24:34
Clay: You gotta make sure you have statistically significant population of data in order to derive the, the statistics that it's based on.
24:41
Clay: otherwise you're just gonna get garbage and you, you'll never know it's garbage.
24:44
Clay: It will give you an average corrosion rate, right?
24:47
Clay: Mm-hmm.
24:47
Clay: You can do, you can always get you an average corrosion rate.
24:50
Clay: Mm-hmm.
24:50
Clay: You know, with two, two data points.
24:52
Clay: And if you're just blindly plugging this stuff into a program and getting an average, and then determining a, sampling statistic and a confidence interval, you can calculate all of those too without having the right amount of data in the right location.
25:05
Clay: So it'll still give you an answer.
25:06
Clay: And that's the scary part, right?
25:08
Clay: Yeah.
25:08
Clay: You don't understand that.
25:10
Clay: Yeah.
25:11
Branden: Well, what do they say?
25:11
Branden: data can say anything you want it to.
25:13
Clay: Sure.
25:14
Clay: Or, or the way we used to say it, you know, data doesn't lie, but liars configure, you know?
25:20
Clay: There it is.
25:21
Greg: I did actually, somebody, I think you asked, Branden about an example.
25:26
Greg: like Clay, I won't get any specifics, but I've seen what can happen when you turn a data scientist loose without any context and without understanding at least some understanding of why we do A DMR and what it means or whatever.
25:44
Greg: I've seen that go seriously awry.
25:47
Greg: And on the other hand, I've seen.
25:50
Greg: When you take the time to collaborate with the knowledgeable corrosion materials engineer specialist and the data scientist, man, some beautiful, powerful things could happen.
26:01
Clay: Sure, sure.
26:01
Clay: I'm a, I'm a full believer in, in running statistics on, on the data and looking at can I reduce the CMLs?
26:07
Clay: That's, it's a very valid approach, but you've gotta think about all the things we just talked about, right?
26:12
Clay: Do you have, the right location?
26:13
Clay: Am I getting the right data?
26:14
Clay: You know, you may not even have the right, the right corrosion rate kind of assigned'cause you're not looking in the right location.
26:20
Clay: So you have to do those things to do this.
26:22
Clay: Well, a tedious part of the job, obviously is going back and re-looking at what strategy was employed at the time for the data that you're collecting or trying to analyze, right?
26:34
Clay: Mm-hmm.
26:34
Clay: So if you go mm-hmm.
26:34
Clay: Pull up an iso, understand the damage mechanisms, and then meticulously think through where that damage was more likely to occur.
26:42
Clay: is it a six o'clock position or 12 o'clock position on a horizontal run pipe?
26:46
Clay: or how much effect could I have at small dead legs or, or in high flow areas, and look at the strategy that was employed on that circuit and then start to make adjustments.
26:59
Clay: Look at the data that they collected, and then make some adjustments to those TMLs.
27:03
Clay: That, that is a very tedious process and it's not always really accomplished with, well, with an inspector.
27:11
Clay: Now, if you've got enough experience and you know enough about damage mechanisms and maybe with a bit of some consultation from somebody that, that's, more of a corrosion materials guy, certainly you can do that.
27:21
Clay: But, you know, we don't have enough corrosion materials engineers in our industry to start with, let alone for working at that level of detail.
27:28
Clay: So, it is, it's not easy to do, I can tell you that from doing it.
27:33
Clay: I did a review of all the injection and mix points at one site here recently and help them configure A RBI approach for it, which we had to write a complete set of rules for how to do that.
27:47
Clay: but, looking at each one of their ISOs and where they were collecting data, made multiple recommendations on changing CML locations.
27:56
Clay: and that actually resulted, this is a plant, it's a major, refiner and it's a, they've been collecting this data for a long time, but they had missed some key areas relative to what kind of damage could occur.
28:08
Clay: And, from that, they were able to go back and they had, I think last time I, they talked to me about it.
28:14
Clay: They, they said they had found seven near misses.
28:18
Clay: they were close enough to failure that they classified it as a near miss.
28:22
Clay: I don't know what number of CMLs they ultimately, you know, were, trimmed it to, I gave them some recommendations, but they were gonna go back and, and re-look at it.
28:29
Clay: But that's a key step is to go back and really understand, where do you have your CMLs and what were you looking for and do you have the right techniques and strategy,
28:39
Greg: you know, clay, I remember something you and I were both connected to, and I won't get into any geographic or company names, but of a heater that failed and, I remember that unit had, RBI done on it like about, I don't know, four or five years previously.
28:57
Greg: And the site determined, like we were talking with'em about, you know, it's been three to five years since you last did the RBI.
29:05
Greg: Do you think it would be a good RI think it would be a good idea was my recommendation to redo.
29:10
Greg: The, the DMR read, Hey, let's see if operating conditions are still the same and everything.
29:15
Greg: Well, so fast forward, a few years later, and, the heater burns down and we went back and looked at that first RBI study that was done years earlier.
29:26
Greg: We don't know who did the DMR.
29:29
Greg: And when we looked at the corrosion rate that was being used for the main header going into the heater, and this was in high temperature sulfur service, they had used three mils per year.
29:39
Greg: The corrosion rate to model that header, and you looked at the dichotomy curves, it was 35 mils per year.
29:47
Greg: But when you looked at the IDMS data, the corrosion rate was one and a half mils per year.
29:54
Greg: My theory is that whoever assigned that three mils per year, and this is just a theory, I don't know, looked at that and said, well, if the IDMS is saying one and a half mils per year and we put in three, we're probably safe.
30:05
Greg: You know, probably safe.
30:06
Greg: Maybe that happened, maybe that didn't, but I, I kind of feel, and again, only God knows what would've happened that if we'd have redone the DMR at that time, which means we would've had to revalidate the operating and the corrosion materials engineer, a real specialist would've looked at that data and caught it.
30:24
Greg: Yeah.
30:24
Greg: Yeah.
30:25
Greg: That's just, you know what's, I think you remember that either.
30:28
Clay: I don't remember exactly that one, but certainly that's happened a lot of times.
30:33
Clay: yeah, it's interesting to me, Greg, if you remember back in the early days when we were doing RBI, you know, we kind of had a, it was a bit of an unwritten rule, right?
30:41
Clay: we would do, and I, I still do this, I will use a.
30:46
Clay: Either the worst case of the calculated rate or estimated rate or actual measured rate, I will start somebody off using the worst case that forces them essentially to go inspect it based on that worst case likelihood, right?
31:03
Clay: Mm-hmm.
31:03
Clay: Until they prove that, that I don't have that kind of a rate.
31:07
Clay: Right?
31:08
Clay: Realistically, you're challenging them Yeah.
31:09
Clay: And allows'em, so you, but, but that starts you out a little bit conservative.
31:14
Clay: Clearly, if you're using an, you know, an estimated or a calculated rate to start out your RBI on, it can, it can tell you to do a lot more inspections than maybe you have been doing in the past, but.
31:27
Clay: And unless you can really prove that your data is good, your corrosion rate data is good, meaning you've anticipated those damage mechanisms, you've collected the data in the right areas, and you have a good quality, you know, IDMS program, and you're doing all the right things for, measuring, corrosion rate, you know, to me you always start conservative and, force, the, the site to, to come to grips with that and make sure that that's not correct.
31:52
Clay: Right.
31:52
Clay: Yes.
31:53
Clay: We don't seem to do that much anymore.
31:55
Clay: we seem to either base it purely on whatever corrosion date data you have, whether or not that was a good program or they'd put the CMLs in the right location, or you're maybe basing it on somebody's opinion and they may or may not have ever worked in that unit, that type of unit.
32:11
Clay: You know, in industry it's, it's a little scary at times for sure.
32:15
Branden: you mentioned, QA and data assurance and getting data in, and, you know, there's a lot of data being moved around.
32:23
Branden: There's a lot of data that we're taking in the industry now.
32:26
Branden: that kind of leads us to a conversation around ai.
32:29
Branden: I'm curious, where do you see the future for AI in our industry?
32:35
Branden: You've, you've worn a lot of hats.
32:37
Branden: You've seen a lot of good, you've seen a lot of bad, yeah.
32:39
Branden: Where do you think there is a, I mean, there has to be a place, we can't say that there isn't a place for artificial intelligence to be able to support, especially supporting on the Yeah.
32:49
Branden: Global version of refining and chemicals.
32:52
Branden: Where do you see it fitting in?
32:53
Clay: it's probably almost unlimited in how it could fit in and, and in terms of where it should fit in or what we should be concerned about is, you know, is if you're ultimately, if you are, using an AI routine in order to, to calculate anything in terms of like a risk or a likelihood of failure or a consequence of failure or when I should be reinspecting this equipment.
33:18
Clay: You know, number one, it's incumbent upon you to understand any of the biases and what that is based on what those are actually being based on.
33:27
Clay: can you get all the way down into the details of how it's specifically calculated?
33:30
Clay: I mean, when I do, I, I use AI now to do a lot of data analysis and, but I'm very specific about it.
33:39
Clay: I either have it use a model that I created, or when I let it run on its own, I ask it to show me the calculations and considerations that it came up with for doing that analysis.
33:50
Clay: Right.
33:50
Clay: And then I kind of look through for key things.
33:52
Clay: I'll give you an example, a quick one, and then we'll finish talking about this.
33:57
Clay: so I I plugged in a circuit that I was having some, problems with, and I wanted to see what kind of results I'd get from ai.
34:04
Clay: So I plugged it in.
34:05
Clay: I was using chat, GPT, I think fourth or fifth version.
34:09
Clay: Anyway, so I fed it all the data off of spreadsheets, and I said, you know what?
34:13
Clay: What would you assess the, you know, the average corrosion rate to be and, the confidence interval for that corrosion rate and, for use, to represent the circuit.
34:22
Clay: Right?
34:23
Clay: And, and are there any concerns?
34:25
Clay: So it came back, gave me all the calcs and said, you know, your average rate is this, your confidence interval is this.
34:29
Clay: And,
34:29
Greg: and,
34:30
Clay: you know, and because I knew the data that I'd given it, it had only a couple sets of readings, right?
34:35
Clay: So it wasn't something that had, you know, 10, 15 surveys.
34:39
Clay: It was only a two or three surveys of, of data.
34:42
Clay: And so I said, did you consider this some of this data, some of these data points.
34:47
Clay: Were based on only two readings.
34:50
Clay: You know, did you consider the, the statistical significance of the population of data for these calculations?
34:56
Clay: and Chad GPT came back and says, oh, great idea.
35:00
Clay: We should do that.
35:01
Clay: You know, if you're depending upon me to think of every problem that could occur within, you know, a complicated set of statistical analysis, you, you've made a mistake to start with, right?
35:12
Clay: So I happen to know that one because I looked at the data, so, you know, you've gotta be really careful about it and, asking open-ended questions without constraining it to, data resources that you want it to pull from.
35:25
Clay: you know, the biggest mistakes in AI these days are often made because it's, it's pulling bad references off the internet, right?
35:32
Clay: that's the key to me is I think we'll use AI more and more obviously, but you have to, again, like any program, you have to understand the bias and the limitations in, in those programs.
35:42
Clay: yeah.
35:43
Clay: And, if you're gonna hang, you know, your company's livelihood either in, producing chemicals or whatever it is you're producing, it's incumbent upon you, especially when people's lives are at stake.
35:54
Clay: Literally.
35:55
Clay: You need to understand it and make sure that you're getting good answers from it.
35:59
Clay: So that was the genesis of a concern from a presentation I gave.
36:04
Clay: At, the fall uhmi executive leadership meeting, it was very clear that AI was going to come and be used for things like RBI type analysis or other type of, RCM or whatever type, type of analysis.
36:21
Clay: So we knew it was coming, but the key is you have to understand it.
36:25
Clay: And, and I think to a degree we can regulate it.
36:28
Clay: you can't necessarily check every reference, but there's a couple ways that I think we can do this.
36:33
Clay: And I think an industry, not just us, but not just all industry, but I think a lot of different industries are struggling now with how do I regulate ai.
36:41
Clay: But you have, like any program, like we've already said, you have to understand the basis for it, and you have to make sure that you're, that it's correct or, or you've gotta devise a way to test and check it.
36:49
Clay: So one of the, ultimately as it doesn't matter if you're using RBI today, very few people could quote all the nuances of how the RBI is calculated today in their plant, right?
37:00
Clay: And, which is another problem.
37:01
Clay: But, you know, failure to understand, that's no worse than failure to understand how AI is generating your answers, that you're, that you're hanging your future inspection dates and potential failures on either, right?
37:13
Clay: So you have to understand that.
37:15
Clay: A couple, couple ways that we could do this.
37:17
Clay: of course, you know, one I suggested that we have either the company that's providing it for you maps out the math on all the equations and provides it to the company they just sold it to.
37:29
Clay: number two, the company, whoever's writing the software, takes on some financial risk of anything that happens that'll help fix a lot of problems.
37:36
Clay: They'll be a, they'll be a hell of a lot more careful than, when they're, when they're creating this stuff.
37:40
Clay: Right.
37:41
Clay: And then the third way I suggested we do this is, look at, creating a compendium of example, problems that have been worked out, either known failures, where we've worked out all the details in the math, in an RBI approach, for comparison that you could run through.
37:57
Clay: any new AI approach or any new, you know, qualitative or quantitative, approach worked.
38:03
Greg: Examples.
38:04
Clay: Yeah.
38:04
Clay: Worked essentially a worked example or a marquee data set, if you will.
38:08
Clay: Mm-hmm.
38:08
Greg: Some
38:08
Clay: people call it, and you could certainly do it not only from that, but you could do it from QRA.
38:13
Clay: So if you, if you had a very detailed quantitative risk assessment performed, you know, and there's bunches and bunches of those, we could actually use that as the basis for a full set of, of marquee data examples to, test the problem, test the new software solutions that are being generated.
38:30
Clay: I mean, those were my initial thoughts on how we might, three ways we might approach doing this.
38:34
Clay: Mm-hmm.
38:35
Clay: To make sure we were getting good quality data.
38:37
Branden: that's a neat idea.
38:38
Branden: the marquee data set.
38:40
Branden: if you want to come up with some new model, some new method, right.
38:43
Branden: Of, of determining failure.
38:45
Branden: Time, time till failure.
38:47
Branden: 'cause that's essentially what we're trying to do.
38:48
Branden: Or time till inspection or risk
38:49
Clay: consequence too.
38:49
Clay: Yeah.
38:50
Branden: Yeah.
38:50
Branden: Any of it.
38:50
Branden: Right.
38:51
Branden: But the final result is an inspection or a Right.
38:53
Branden: An action, time till action.
38:55
Branden: And having a data set there of those known failures, and then testing it to see, does it miss any of those, feed it all, provide that full data set of all the information that's required or that we have, and that we know.
39:07
Branden: That's a neat idea.
39:09
Clay: Hmm.
39:09
Clay: Yeah.
39:10
Clay: ultimately you could score it depending upon if it's, it's a quantitative result, you could actually score it relative to the actual results.
39:16
Clay: Right.
39:17
Clay: And, and then you could kind of look at, okay, you know, if I'm not getting very close to this, there's gotta be a reason why, what, what's the difference in the calcs and why?
39:25
Clay: But, you've gotta almost now have a marriage right between.
39:29
Clay: The, the company's producing the, the AI solutions, and then, and you wanna make sure that they're not just, creating the solution that fixes that one incident, right.
39:39
Clay: That where they weren't very accurate.
39:41
Clay: You know?
39:41
Clay: So you've gotta be really careful about that too.
39:43
Clay: But, but, ultimately we've got to come up with a way to, make sure that we're getting good reasonable answers that are, you know, giving you a, a good degree or margin of confidence in, in the answer on what it's telling you is the potential future failure date or when you need to reinspect.
40:02
Greg: So you're saying transparency
40:03
Clay: is important?
40:05
Clay: I think so.
40:06
Clay: I don't like the idea of the black box.
40:08
Clay: Right?
40:08
Clay: So that was what I titled The presentation when I gave it, and I gave a couple of examples.
40:13
Clay: I gave a AI example, on, on Tesla, the first fatality.
40:19
Clay: Mm-hmm.
40:19
Clay: I used that one.
40:20
Clay: because essentially, the guy was trusting the car to drive for him.
40:25
Clay: a tractor trailer pulled across the path, the radar system that, that it employed for their autopilot, was aimed down lower.
40:32
Clay: So it went under the tractor trailer that was completely across the freeway.
40:36
Clay: Never saw it.
40:37
Clay: Hmm.
40:37
Clay: And the camera system, the tractor trailer box trailer was painted white.
40:42
Clay: and based on the sky, it couldn't distinguish well enough to identify it as a problem.
40:48
Clay: so that was one.
40:49
Clay: there's been like 47 fatalities, on, on Right.
40:52
Clay: Teslas, you know, from their version of the autopilot.
40:55
Clay: And they claim it shouldn't be driving the car for, you need to pay attention, you know, as you're the operator of the car.
41:00
Clay: Right?
41:01
Clay: Right.
41:01
Clay: And, and now Tesla of course, has cameras, it watches, and it knows everything, every adjustment, everything you're doing, including what you're physically doing when they, at the time of an accident with their black box now.
41:12
Clay: So, the other one was a military example, where they were testing AI to, identify threats and then, develop a missile firing solution.
41:20
Clay: And then it, and it required an operator intervention to agree to it and authorize the launch of the missile.
41:27
Clay: Right.
41:28
Clay: Or not.
41:29
Clay: But pretty interesting stuff for sure,
41:31
Branden: man.
41:32
Branden: Yeah.
41:32
Branden: Some of that, the hallucinations, is what I think they've called it, where it just starts making things up and
41:37
Clay: Oh, yeah.
41:38
Branden: But the reasoning, I mean, you can't, can't argue with some of that reasoning there, flawed reasoning, but at least it's reasoning, which is interesting.
41:45
Clay: Yeah.
41:45
Clay: it's definitely a bit scary, you know?
41:46
Clay: I, I've realized now that, that I get more predictable and better results, you know, when I provide the frame of references that I want the AI to use.
41:54
Clay: Right.
41:54
Clay: it's a bit more of a program than I thought.
41:56
Clay: I first started thinking about AI as a, you know, I just log in and ask it a question, right.
42:02
Clay: And, and let it.
42:03
Clay: Search the world and to come up with good answers.
42:05
Clay: Right?
42:06
Clay: But it can't understand the difference between fake data and real data.
42:12
Clay: it can't judge, I don't think, a better approaches versus a worse approach.
42:17
Clay: And so what I've done more recently is I, you know, I, I code, code in, so I've got a whole series of things I code in, including reference documents, considerations that I want it to use for doing statistical analysis or whatever I'm doing.
42:30
Clay: So it's got a whole series of instructions, so almost like a program parameters, if you will, for what it needs to consider.
42:38
Clay: And that, that, that kind of keeps it more grounded in something I understand or know where it's coming from.
42:43
Clay: Right?
42:43
Clay: So, I don't know if you'd ever be able to get that out of a third party producing, say an RBI tool, right?
42:49
Clay: Dunno if they're willing to do that, and then maybe with their clients they're willing to share it.
42:52
Clay: They certainly should be able To do that with a client that's bought their software, they ought to be able to Right.
42:58
Clay: Identify that.
42:58
Clay: But I think the best way is some independent checking on the quality of the data and answers you're getting relative to something, in my case, relative to re what I consider to be reality, actual known failures or, or worked out QRA kinda worked out, examples.
43:14
Clay: Yeah.
43:14
Clay: I think are the best way to do it.
43:16
Branden: Yeah.
43:17
Branden: what's your favorite, prompt hack that you've been using lately?
43:21
Branden: Like prompt hack?
43:23
Branden: Yeah, maybe not hack, but your favorite, your favorite prompt that you're using to help.
43:26
Branden: Like mine right now is I'll ask it something and I'll, and then I'll follow it with, and ask me any questions that you need to make sure that we get a quality response.
43:35
Branden: And then it gives me 5, 10, 15 questions that I go back and I answer, which helps hone in what it, what I'm asking.
43:42
Branden: Yeah,
43:42
Clay: yeah.
43:43
Clay: You know, I, you probably have a lot more experience at it than I do at this.
43:46
Clay: You know, I try to frame up the problem as fully as I can and then provide, data and or documents that I think are reasonable examples, for what I'm trying to do.
44:00
Clay: And then I'll tell it always to list out, show the math that it's basing it on, and then I'll go through that to look for any problems or issues.
44:10
Clay: That's how I discovered the, in the one case I gave earlier, that, that it wasn't looking at calculating any statistical significance on the data population.
44:20
Clay: That it was using to, to derive the averages and, and, z critical, Z value and, and confidence intervals.
44:27
Clay: So, you know, you're right.
44:29
Clay: Asking it more questions.
44:30
Clay: That's a good holistic question for sure to ask.
44:33
Clay: you know, I don't know how to get around the problem.
44:34
Clay: Like it didn't recognize itself that it should have checked for statistical significance.
44:40
Clay: I knew it should have, but, but that's'cause I've done it a bunch, but, you know, I, there's a lot of stuff I don't know to ask it.
44:46
Clay: So how do you get around that?
44:47
Clay: I don't know, Branden.
44:48
Clay: That's a good question.
44:49
Clay: Yeah.
44:50
Branden: John Reynolds, we had him on a little bit ago, and, he says he uses it.
44:54
Branden: he's dabbled with, I think probably something like chat GPT or something, and his take on the whole thing is that it's a really good grad student.
45:02
Branden: He views it as a really good grad student where you can give it a task, it can go do it, but you still have to go back and check it just to make sure.
45:08
Clay: Yeah, yeah, absolutely.
45:10
Branden: Well, good, I appreciate your time.
45:11
Branden: Appreciate you coming on and talking and, you've got a lot of experience.
45:15
Branden: You've worn a lot of hats, you've got a lot of opinions.
45:17
Branden: it's really nice being able to sit down and talk with you.
45:19
Branden: So, yeah, enjoyed it.
45:20
Branden: Thanks for having me.
45:21
Branden: This was
45:21
Greg: great, clay.
45:22
Greg: This was great man.
45:24
Branden: Thank you for listening to Upon Further Inspection, a Mechanical Integrity podcast.
45:28
Branden: This episode was co-created by Inspectioneering, and CorrSolutions.
45:33
Branden: Our producers are Nick Schmoyer, Jocelyn Christie and Jeremiah Wooten.
45:39
Branden: This podcast is for informational purposes only and does not constitute legal or professional's advice.
45:45
Branden: Listeners should seek their own qualified advisors for guidance.
45:49
Branden: If you enjoyed this episode.
45:51
Branden: Please join us next time wherever you listen to your podcasts.
45:54
Branden: Until then, stay safe and stay informed.