Latest / Breaking the ICE / Episode 7 - Demystifying AI in Enterprise Software with CTO Al Robertson

Transcript

Episode 7 - Demystifying AI in Enterprise Software with CTO Al Robertson · Breaking the ICE

• 0:02
  SPEAKER_02: Hello everyone and welcome back to the Breaking the Ice podcast.
• 0:06
  SPEAKER_02: We are on episode seven of season one.
• 0:10
  SPEAKER_02: Um extremely exciting episode today.
• 0:13
  SPEAKER_02: I am joined once again by Andrew Passfield.
• 0:16
  SPEAKER_02: Andrew, welcome.
• 0:17
  SPEAKER_02: Nice to have you back.
• 0:18
  SPEAKER_02: Thank you.
• 0:19
  SPEAKER_02: Yep.
• 0:20
  SPEAKER_02: And uh very excitingly, we are also joined by Al Robertson.
• 0:24
  SPEAKER_02: Al, welcome to the podcast.
• 0:26
  SPEAKER_02: How are you doing today?
• 0:28
  SPEAKER_00: Thanks, Elliot.
• 0:28
  SPEAKER_00: Yeah, good.
• 0:29
  SPEAKER_00: Thank you.
• 0:31
  SPEAKER_02: So just for a bit of a background, uh Andrew and Al were uh two of the founders of Ice Insure Tech Limited, the company that was merged with Nordic Insurance Software earlier this year to form Ice Tech.
• 0:44
  SPEAKER_02: Uh Andrew's now the chief commercial officer, and Al is the chief technology officer of Ice Tech, the uh the grouped entity here.
• 0:54
  SPEAKER_02: Uh so we are slowly but surely uh acquiring senior leadership members onto this podcast.
• 0:59
  SPEAKER_02: Uh maybe we'll do a full round table at some point.
• 1:02
  SPEAKER_02: Uh but uh nevertheless, great to have you both on here.
• 1:05
  SPEAKER_02: Um Al, how are you doing today?
• 1:08
  SPEAKER_02: Yeah.
• 1:08
  SPEAKER_02: Good thanks.
• 1:09
  SPEAKER_02: Uh looking forward to this.
• 1:10
  SPEAKER_02: Yeah.
• 1:11
  SPEAKER_02: Andrew?
• 1:11
  SPEAKER_01: Yeah, I'm uh excited with uh with Al.
• 1:15
  SPEAKER_01: We've made a lot of progress in the whole sort of AI and and software development lifecycle.
• 1:20
  SPEAKER_01: So uh I think uh I think Al will give you some good insights and I'll uh I'll chip in as appropriate.
• 1:26
  SPEAKER_02: Yeah.
• 1:27
  SPEAKER_02: And this is your second time on.
• 1:28
  SPEAKER_02: So you're you're now a pro.
• 1:30
  SPEAKER_02: Uh no mistakes like last time, right?
• 1:34
  SPEAKER_02: Try my best.
• 1:35
  SPEAKER_02: Okay.
• 1:36
  SPEAKER_02: Okay.
• 1:36
  SPEAKER_02: Well, like I we'll we'll we'll save the banter uh because we're we're very fortunate to have Al on here, uh, our chief technology officer.
• 1:45
  SPEAKER_02: And Al, I think we need to launch straight into the conversation around AI.
• 1:49
  SPEAKER_02: Um uh, you know, we we managed to get a minute and 55 seconds into this without saying AI, so kudos to us.
• 1:56
  SPEAKER_02: But I want to spend you know some time today demystifying AI within the context of of our landscape and enterprise software.
• 2:05
  SPEAKER_02: Um, I think that there is a tremendous amount of buzz and hype.
• 2:11
  SPEAKER_02: And because it is a complicated subject, I think there is the the commercialized terms that you know Andrew and I use, and then there's the the actuality of what's happening behind the scenes.
• 2:26
  SPEAKER_02: And I want to dig into that, you know, in a little bit more detail for the audience.
• 2:30
  SPEAKER_02: So I guess one place to start out would be under that umbrella and under your stewardship of our engineering department, uh how is AI being leveraged?
• 2:46
  SPEAKER_02: So, you know, what does AI mean from an engineering standpoint?
• 2:50
  SPEAKER_02: Uh and then maybe by extension, what does it not mean?
• 2:53
  SPEAKER_02: Or or or where is it not applicable within the framework of engineering?
• 2:58
  SPEAKER_00: Yeah, okay, let's let's stop there.
• 3:00
  SPEAKER_00: And actually, I think it'd be good to have a framing um of what we're going to talk about today because our engineering team is uh using AI, embedding it within our products, and that's fine.
• 3:13
  SPEAKER_00: That's for our customers um to get the benefit.
• 3:17
  SPEAKER_00: But the key point of today is where we use AI to augment our software development lifecycle.
• 3:24
  SPEAKER_00: Um, and that's really been a massive focus for me over the last probably um 15 months or so, um, when I when I think back to it, and and it's been something that um we've taken the engineering department of of circa 80 people through through here.
• 3:42
  SPEAKER_00: So um in terms of SDLC, as we'll call it, you know, fundamentally the the steps that we're going through are the same steps that they always were.
• 3:54
  SPEAKER_00: So we have um you know requirements and specification creation, we look at the architecture, um, we do the builds, we verify the changes that we're making, and and then we release, etc.
• 4:07
  SPEAKER_00: The difference is that AI is taking a key role in some of those steps that it wasn't taking before, and it's being an assistant to the engineers.
• 4:18
  SPEAKER_00: Uh so one of the major changes, uh, and my team will be be well aware that I've said this probably a thousand times, is is the concept of shifting left.
• 4:28
  SPEAKER_00: So the engineers now are not focused on the build execution like they were, we're leaving that to the AI.
• 4:37
  SPEAKER_00: What we're focused on is uh on gathering all of the requirements and the specification and documenting the architecture, so we're removing any ambiguity from from our actual um uh execution of the build process so we get the best results at the end of it.
• 4:58
  SPEAKER_00: And that includes really giving as much context as possible to get the best result at the end of the day.
• 5:07
  SPEAKER_02: Yeah, I mean again, you know, to frame this within the context of uh software development lifecycle, it's it's it's an additive process, right?
• 5:16
  SPEAKER_02: And the more tight correlation between each of the components of the SDLC, the less you have to loop back, right?
• 5:26
  SPEAKER_02: So from if if we just zoom in on the the actual build process and deploying AI there, has that provided and then you think about the connecting points.
• 5:38
  SPEAKER_02: So what feeds into the build process and what comes out of the build process?
• 5:42
  SPEAKER_02: With what's feeding in, does that provide more capacity to have engineers or architects um supporting the requirements process?
• 5:54
  SPEAKER_02: Or correspondingly, has that created a bit of a um a bottleneck on that, you know, sort of requirements piece because the engineering uh output is is complemented by AI.
• 6:11
  SPEAKER_00: So I think it's exactly that.
• 6:13
  SPEAKER_00: The the engineers are now focused on the requirements gathering to a level that we never were before, and the specification detail to a level that we never used to, because we used to just hand the requirements and a specification to an engineer, and there would be a lot of implicit um uh information that might not be spelt out because we would expect them to read between the lines, make good informed decisions.
• 6:45
  SPEAKER_00: Um, and we're moving away from that world to try and say, Do you know what?
• 6:49
  SPEAKER_00: We are writing everything down because we we're gonna hand it over to Ashley, an artificial intelligence agent, to go and deliver that implementation of the build for us.
• 7:03
  SPEAKER_00: So the build used to be the most expensive thing in the whole SDLC process, it's now not, is the reality.
• 7:11
  SPEAKER_00: So, yes, we are focused on on uh those requirements gathering.
• 7:16
  SPEAKER_00: It it takes me, it takes me back.
• 7:18
  SPEAKER_00: You know, I've spent um I've I've we worked on a number of products over the last uh six months where I've been spending uh an hour a day for six weeks fully working on documenting uh 30, 40, 50 page requirements documents, you know, product requirements documents as we're calling them, that I never would have done to the same level um uh uh uh two years ago is reality.
• 7:47
  SPEAKER_00: So we're really focused on that.
• 7:49
  SPEAKER_00: But if we if we do focus on that and then and then understand all of the complementary documents that go with it, talking about architecture, design, implementation nodes, etc.
• 8:00
  SPEAKER_00: The final specification should provide everything that an agent knows to go and build maybe a segment of that.
• 8:07
  SPEAKER_00: And um, and it's evolved over the last year, but it's now um a process where, given enough experience, you can be confident in the the result that you're gonna get.
• 8:20
  SPEAKER_00: And a key part of it is that we introduced verification um that uh is is key, and we'll probably talk about that in a bit more detail later.
• 8:31
  SPEAKER_02: Yeah, I just wanted to just a couple more thoughts on the on the the transition from the PRD into the build.
• 8:39
  SPEAKER_02: Naturally, there's upstream change that happens once the engineers are more involved and the requirements themselves need to change in order to be processed by uh AI tooling.
• 8:50
  SPEAKER_02: Um has it been a net positive, or is it a bit of a double-edged sword as well?
• 8:55
  SPEAKER_02: Like what what's products feedback?
• 8:57
  SPEAKER_02: And full disclosure, we had Casper, our chief product officer on a few weeks ago.
• 9:01
  SPEAKER_02: He didn't he wasn't pulling his hair out about anything.
• 9:03
  SPEAKER_02: Uh, so I think I think in general they're okay with the changes, but um, has has there been has it just been a sort of a net positive from a product standpoint, or um is there teething that goes along with that as well because the process changes?
• 9:18
  SPEAKER_00: Well, I think both of those are true for sure.
• 9:21
  SPEAKER_00: Um, I think we're definitely at the stage of being in a net positive.
• 9:25
  SPEAKER_00: Um uh there's definitely been teething, and the reality is is is we had to undertake uh a lot of investment, and those those really formed two areas.
• 9:37
  SPEAKER_00: The first was that we invested heavily in our people.
• 9:40
  SPEAKER_00: Now that was allowing them the time and the space to be able to learn how to use AI, and we we did a number of things um in that area.
• 9:50
  SPEAKER_00: One thing that we did near the back end of last year was create a community of practice for our AI champions, as we call them.
• 9:59
  SPEAKER_00: Now that's that's 22 engineers out of about 80 uh across the business.
• 10:05
  SPEAKER_00: So it's um it's a large number of them.
• 10:08
  SPEAKER_00: And there are representatives from every team that we've got, and these are the guys who are well, we again we're giving the time and the um uh in their in their daily um daily work to be investigating, to be researching, to be testing and and trying out and and trying to share best practice ideas within the group, you know, aligning between the group of of which direction that we're gonna go in.
• 10:35
  SPEAKER_00: And then those same champions are responsible for disseminating all that information to each of their teams.
• 10:42
  SPEAKER_00: So making sure that um everyone in engineering is is getting up to the same bar, which is uh to be a competent, you know, agentic engineer.
• 10:55
  SPEAKER_00: Um uh and they're going through sort of pair programming exercises, they're passing on all that learned information, um uh they're doing presentations, uh they're getting uh people on the team to to present what they've done and allowing everyone to share and could critique and stuff.
• 11:14
  SPEAKER_00: So it's uh it really is giving them the time to learn a new skill, because that's that's huge.
• 11:21
  SPEAKER_01: So I was gonna I was just gonna say I thought what was quite interesting when when we've talked through this is um you know it's a different way of working for people to learn.
• 11:32
  SPEAKER_01: So people have been coding for you know five, ten, fifteen, twenty years, um, but also the I guess the the sort of differences in in different people, the way they threw themselves into it, or some people were less keen to throw themselves into it, and trying to just bring everybody up to the same level.
• 11:54
  SPEAKER_01: And I think that that sort of idea of champions um seemed to really work.
• 11:58
  SPEAKER_01: And and you and I know we had some some sort of doubters and and we had some very early adopters um and some successes, we had some some learnings, um, but it has been uh uh I guess a big part of of our sort of evolution and that training and investment in people.
• 12:16
  SPEAKER_01: Yeah.
• 12:16
  SPEAKER_01: But not everybody, I guess the point I'm making is not you know, it's not like you're going on a training course, giving someone a manual, and and two weeks later they're a competent AI developer.
• 12:27
  SPEAKER_01: So it's been quite interesting to watch.
• 12:29
  SPEAKER_02: Yeah, I mean I think I think the I think the you know a a fair analogy is agentic AI is is such a powerful tool, right?
• 12:39
  SPEAKER_02: I mean you can compare it to powerful, you know, industrial changing tools of years past.
• 12:46
  SPEAKER_02: There's going to be early adopters and there's going to be um there's going to be folks that are are so proficient at the pre-industrialized way of doing it that they that that it's more efficient for them to continue on that way.
• 13:02
  SPEAKER_02: But eventually, you know, the sort of the technology curve wins and and and everybody does industrialize.
• 13:09
  SPEAKER_02: And uh I'm sure there's a day where I'll be vibe coding the majority of our back end, but uh let's hope not.
• 13:16
  SPEAKER_02: Not in our time.
• 13:17
  SPEAKER_00: Not in your I mean you can do that in your own time.
• 13:22
  SPEAKER_00: Yeah, we really had um some people uh who came and said, you know what?
• 13:29
  SPEAKER_00: I love my job.
• 13:31
  SPEAKER_00: AI is spoiling it for me because it takes some of the the craft, and you know, a lot of a lot of engineers see what they do as a craft, and they're they're using all of their tools that they've got available in their workshop and selecting the right one and coming up with an elegant solution to a problem.
• 13:48
  SPEAKER_00: That that's uh that's that's the truth.
• 13:51
  SPEAKER_00: Um the reality is the world is changing, and to become a really um a valuable agentic engineer is where we're trying to encourage all of our guys to go to.
• 14:04
  SPEAKER_00: Some of them flew at it, some of them didn't.
• 14:07
  SPEAKER_01: I think what what also interesting is, you know, and we've had some feedback from customers that that are sort of just expecting code happens now.
• 14:16
  SPEAKER_01: I think, you know, absolutely, I think I've been there for all 1,000 times you've mentioned the sort of shift left, but but you're sort of swapping.
• 14:25
  SPEAKER_01: So we might have spent a day specing and five days coding.
• 14:29
  SPEAKER_01: Metrics are, but it you might be doing three or four days, and then the uh the the code is is getting built through AI.
• 14:37
  SPEAKER_01: So it's a net gain, as you said earlier, it's not completely disappearing because some of that work is is is shifting left.
• 14:44
  SPEAKER_01: But I think the other thing that that we've talked about is the quality of output, and and and it you mentioned output, the quality of output goes up.
• 14:52
  SPEAKER_01: So you've told me in the past, you know, we're very competent and very disciplined in writing sort of uh unit tests and everything, but the level of testing and and tests that you get out of AI is so much better than uh and the level of coverage that you get um than someone manually, so the quality goes up as well.
• 15:13
  SPEAKER_00: I think that's really key.
• 15:14
  SPEAKER_00: Uh yeah, anecdotally, I have um an AI process to go and scan a number of our key repositories and provide metrics for our board packs.
• 15:24
  SPEAKER_00: And one of them from the last month was 3,300 new um unit and integration tests from the written that that we we just wouldn't have had that um a year ago.
• 15:35
  SPEAKER_00: It would have been a tenth of that, I think.
• 15:38
  SPEAKER_00: Um uh what you know we used to have a number that we thought satisfied the problem, and now we can get 30 or 40, which is great to prove the way that we work now.
• 15:50
  SPEAKER_00: We work in a test-driven development fashion for those that um that understand that, i.e., when we're when we're doing the build of the process, generally we we write the tests first, and then we write the implementation to satisfy those tests with the knowledge that of what we've got to do.
• 16:11
  SPEAKER_00: And that needs a huge amount of coverage to test or to make sure that all of all of those scenarios are tests, and that gives a a way well a less of a bias than if you write the the implementation first, you can the a the AI can write tests to make the implementation work, which isn't necessarily um what you want from your requirements, which is why we write write the tests first.
• 16:40
  SPEAKER_00: So just doing that is um is really positive.
• 16:44
  SPEAKER_02: Yeah.
• 16:45
  SPEAKER_02: The um I I just wanted to sort of expand on that thought a little bit, right?
• 16:51
  SPEAKER_02: Because I guess one of the positives of agentic engineering is the the breadth of what you can accomplish is much wider, right?
• 17:03
  SPEAKER_02: I mean you can you can deliver ten times the amount of features that a human could with with age, I'm just you know, for sake of of illustration.
• 17:12
  SPEAKER_02: But there's a human, because there's a human component, there's still heavy, heavy human components to you know that feature leaning in and coming out of it.
• 17:25
  SPEAKER_02: Has it created uh or highlighted uh the sort of opportunity costs that go along with this?
• 17:33
  SPEAKER_02: Because the agents can deliver, again, 10 new features, but but we don't necessarily have capacity, or it isn't necessarily the right thing to deploy all of those features into the product, right?
• 17:46
  SPEAKER_02: Some things that are delivered shouldn't leave the cutting room floor necessarily.
• 17:51
  SPEAKER_02: So, you know, we we have the ability now to prototype and test and and do all these wonderful things, but uh does it then also start to create that downstream pressure from a testing and deployment standpoint of what should actually go in and and and in what order of priority should they go in?
• 18:08
  SPEAKER_00: Yeah, uh again, great, great question.
• 18:11
  SPEAKER_00: So so fundamentally here in the SDLC, we have three major categories of um of capability.
• 18:21
  SPEAKER_00: One is human, one is an agent, i.e.
• 18:25
  SPEAKER_00: an AI-driven um uh component, and one is good old-fashioned code.
• 18:32
  SPEAKER_00: Um and what we're trying to do is um combine those uh those three things uh as as um as well as we can to get uh an efficient process.
• 18:44
  SPEAKER_00: And what's happened over the last year is we used to have really just human um uh doing this.
• 18:52
  SPEAKER_00: Uh the eight, we've come in and we've added a lot of uh agentic capability to automate as much of the process as possible, and it's pointing out where we've got bottlenecks on humans.
• 19:04
  SPEAKER_00: Um and that might be that we need to uh spend more time on the requirements and the conveyor belt of of getting requirements and getting things aligned with customers still takes the same amount of time, but the engineers are now waiting for it, and then the other side of the conveyor belt, you've got quality assurance and people that are going through and asserting that that what's been um specified is actually delivered.
• 19:30
  SPEAKER_00: If if we're if we were trading 10 tickets a week and we're now doing 50, it's not that level, but if we were, then obviously uh it it's it's just too much for them.
• 19:41
  SPEAKER_00: So there's a there's a continuous evolution on trying to improve all aspects of this because we're on a journey here where we were three months ago, six months ago, nine months ago, every every period in this really short timeline in the in the overall scheme of things is quite different and is is evolving, it's approving, and we're talking a lot about what's gonna happen in the next six months, and that and that'll be interesting.
• 20:07
  SPEAKER_01: I think um if people that are listening to this um will be sitting there saying, Well, you keep talking about a genetic AI and and uh agentic engineering.
• 20:20
  SPEAKER_01: Do you want to talk through the tools that we're using and and how you sort of came about that that sort of choice?
• 20:26
  SPEAKER_01: Because I think people will be interested in that.
• 20:28
  SPEAKER_00: Yes, uh sure.
• 20:29
  SPEAKER_00: So so uh our fundamental um tool that the majority of the company is now using is Clawed Code.
• 20:37
  SPEAKER_00: We have quite a few people also using Codecs.
• 20:40
  SPEAKER_00: Um so that's the open AI version.
• 20:43
  SPEAKER_00: Some people, uh a small number of people are primary codex users, but a lot of people use codecs and the AR models in an adversarial sort of context.
• 20:53
  SPEAKER_00: So we we might have Claude do, let's say, a a build and then have Codecx review it without all of the information that that Claude has sort of built up in its process so we can take a fresh look at it.
• 21:08
  SPEAKER_00: It's based on different training data.
• 21:12
  SPEAKER_00: All of these models are different training data, different inputs, different reinforcement learning sort of stances on it.
• 21:19
  SPEAKER_00: So it's great to have an adversarial model uh look at it.
• 21:23
  SPEAKER_00: So but Claude Code is is the main one.
• 21:25
  SPEAKER_00: We've gone, we've tried a number of different uh harnesses, and a harness is basically everything but the model, is the way I would describe it.
• 21:34
  SPEAKER_00: Um and and we have the option of using different models for the different um intelligence levels and thoroughness that that that we need, and and the engineers are used to selecting those manually, and actually what we're doing now more is trying to automate when we um uh when we use, let's say, a different agent with a different um model to do a certain task.
• 22:00
  SPEAKER_00: Like I have a verification agent very similar to what I've just talked about that that verifies every step of the work that was just being done, and it takes a fresh look at it and it points out um where there might have been gaps, or or it might give you a warning to say, okay, do you know what this is this is passing, but have you thought about this, this, and this that you didn't get from from your first view?
• 22:25
  SPEAKER_00: So, yes, we're burning more tokens, but um uh we're getting better quality at the end, which is which is our primary goal.
• 22:32
  SPEAKER_01: Tokens is is interesting because again, I think you know, people think this stuff comes for free.
• 22:38
  SPEAKER_01: Um, you know, we've put a lot of investment in training our people.
• 22:42
  SPEAKER_01: We're also spending a lot of money with with companies that we didn't spend before, where traditionally we were, you know, um uh just sort of hand coding.
• 22:50
  SPEAKER_01: So I think you know, people do need to understand how the whole life cycle works and comes together, not just that you're gonna get estimates cut in in half or or or cut in into quarters, just because you know there's an agentic element to it being done.
• 23:05
  SPEAKER_01: There's a cost to that agentic element being done, there's a cost to that training, there's a cost of spending more time up front specifying the detail that you talked about earlier.
• 23:14
  SPEAKER_00: And actually, there's a another cost because right at the beginning we talked about our investment.
• 23:19
  SPEAKER_00: Um, the other significant investment that we've made is in what what I call the agentic layer.
• 23:26
  SPEAKER_00: So that is all of the the markdown and skills to allow agents to to work within our code base and write the code, applying all of our standards, giving it abilities.
• 23:41
  SPEAKER_00: These are general generally in the skills space.
• 23:44
  SPEAKER_00: Um uh we have about 300 skills now um that we've that we've built up and we are continually modifying.
• 23:52
  SPEAKER_00: So all of that is an investment that takes a lot of time and it allows us to automate and it allows us to um apply standards uh and and give it abilities that it wouldn't have out of the box.
• 24:05
  SPEAKER_00: Because as I as I always sort of say to newbies, um when we're dealing with uh these fantastic frontier models these days, they basically have the knowledge of of the compressed internet.
• 24:17
  SPEAKER_00: So they are extremely smart, knowledgeable, and capable, but they know absolutely nothing about our code base because it's private domain, and we have to tell them about it, and we have to tell it to to look in this and go and investigate that, and that's part of our um our our other investment in making sure that we get the best quality by learning how AI works and um uh and investing in the things that allow us to deliver more.
• 24:48
  SPEAKER_02: Yeah, and I I think I mean you've you've both covered most of the points on that investment, but I think something else, Al, is and you can attest firsthand to how much has changed over the past 15 months and forecast that out into the future.
• 25:04
  SPEAKER_02: There's going to be continuous change.
• 25:07
  SPEAKER_02: And and you know, it obviously starts with you as the CTO, but it emanates downward.
• 25:12
  SPEAKER_02: Models change, tools change, you know, there's going to be new tools that come out, particularly because the pace of change is so much more rapid now.
• 25:22
  SPEAKER_02: There are going to be tools that come out over the next 12 months that we don't even know about that are going to infiltrate what we do.
• 25:29
  SPEAKER_02: And what underlies that is the same careful process that you've undertaken to get where we are today.
• 25:38
  SPEAKER_02: So, you know, every time something comes out that's better, the net output is, yes, eventually better, more efficient, you know, higher quality, but the same level of investment has to go in to get people using it the right way and make sure processes adopt it the right way.
• 25:57
  SPEAKER_00: Spot on.
• 25:57
  SPEAKER_00: I think I think we're seeing this as continued in continuous investment.
• 26:01
  SPEAKER_00: Um it's got the full support of uh everyone on RSLT.
• 26:07
  SPEAKER_00: Um uh the whole business is trying or is on the journey to becoming AI native, and uh certainly the engineering department is um, I would say uh I would say because I'm the CTO, but but we're at the forefront of that, um, trying to trying to push hard.
• 26:25
  SPEAKER_00: Um but we're doing it for good reasons and we're getting good results.
• 26:30
  SPEAKER_00: So I don't think anyone's um really thinking thinking um we're doing the wrong thing.
• 26:37
  SPEAKER_02: No, absolutely not.
• 26:38
  SPEAKER_02: I mean, the and and just to just to sort of state the obvious here, but the competitive landscape is growing by the day as well, because these tools are accessible to everyone.
• 26:48
  SPEAKER_02: Now, you know, we benefit from the industrial knowledge that we've obtained over the last 20 years servicing particular markets, uh, and we benefit from having a robust enterprise platform, and we're not starting from scratch like my vibe coding uh startup on the side.
• 27:07
  SPEAKER_02: But uh, you know, the the the reality is it the it's becoming a much more democratized process because of the uh the output, the potential output of of uh of these uh agentic tools.
• 27:24
  SPEAKER_02: So I want to pivot uh to a bit of a hardball question, um, Al, because um I don't well, I don't want to scare you off from coming on again because I think there's lots of other things we can talk about.
• 27:35
  SPEAKER_02: But you know, I and and we have we have talked about obviously the amount of time uh that we've invested in this, but the reality is when you think about AI uh or or sort of agentic engineering versus human engineering, it's implied that it's cheaper and faster.
• 27:55
  SPEAKER_02: Right?
• 27:56
  SPEAKER_02: I know tokens are expensive, but they're not as expensive as people.
• 28:00
  SPEAKER_02: Help demystify that because that that really isn't the case.
• 28:03
  SPEAKER_02: I I mean again, you know, if there's a very narrow slice of that, that's the truth, but but help paint the full picture here of why you can't simply just say something costs less because AI wrote it.
• 28:17
  SPEAKER_00: Yeah, I think um I think there's uh a number of a number of aspects um to that because uh obviously the human is the most expensive part of this process now.
• 28:28
  SPEAKER_00: As I talked about earlier, we've got the human, we've got the agent, and we've got the code.
• 28:32
  SPEAKER_00: You're going from one extreme of the human being the most expensive, the agent in the middle with with token burning, and then code, which is essentially once it's written, is pretty much lightning quick and it's free.
• 28:45
  SPEAKER_00: Um, so the idea is is can we add as much to our code base as possible to support the software development lifecycle, but still have humans in the loop where they add the most value?
• 28:59
  SPEAKER_00: Um and and we've talked about those those areas.
• 29:02
  SPEAKER_00: So you're gonna use humans for for judgment.
• 29:06
  SPEAKER_00: That's an absolutely um critical uh critical thing that we can't expect the AI to do yet without giving it um all of our gains knowledge the the you know thirty years in in in insurance and I know all aspects of our product, so it's quite hard to replicate that um uh in the um uh in in an AI judgment process.
• 29:32
  SPEAKER_00: Um there's uh you know architecture and system design, we can get the AI to augment um humans, uh, which is which is what we're trying to do, and maybe come up with more ideas.
• 29:44
  SPEAKER_00: But at the end of the day, again, the humans always gonna um m take the judgment call on these things because of the context that that we have, and that's certainly gonna be true for the for the foreseeable future.
• 29:57
  SPEAKER_00: So um yeah, we definitely need the humans in there.
• 30:02
  SPEAKER_00: We are very capable beings to adapt to any task very uh well to pick things up.
• 30:09
  SPEAKER_00: We all know that the AI needs a lot of investment uh to to do it as well as we do it.
• 30:14
  SPEAKER_00: So we try to use the the humans on the the parts that are um uh I I guess that the AI can't do as well um as as we can, and uh we can't just deliver by writing some code.
• 30:32
  SPEAKER_02: And and Al, I mean, you know, planting the seed here for hopefully a future conversation about uh RAS, but uh is it is it also fair to say that I I mean quality matters, reliability matters, safety and security matter, and I don't think anyone in the marketplace would be comfortable outsourcing that to an agent.
• 30:55
  SPEAKER_02: Right.
• 30:56
  SPEAKER_02: And and and so again, because um because the breadth of code has increased, because the the breadth of features has increased, it it then puts more of a human capital requirement on ensuring that as you build a more complicated machine, the connections don't break.
• 31:21
  SPEAKER_00: That's uh that's an interesting, again, an evolving picture.
• 31:25
  SPEAKER_00: Um I think you'd have to have been on Mars if you hadn't heard of the um the frontier models that are uh hacking and jailbreaking and using a number of techniques to to achieve to to to break um through security layers that that just something that wasn't wasn't talked about six months ago.
• 31:44
  SPEAKER_00: So so we've got a a moving sort of um frontier from from that side that there is going to be more intelligence out there to to to need to add security, but then we are gonna be using intelligence to try and help us one um uh fix these issues, fix bugs as they come in.
• 32:06
  SPEAKER_00: You know, we are we are we're busy investing in factories at the moment where where we can spot bugs happening by looking at logs and sending them through a whole workflow to try and deliver a fix at the end of it, with only having human in that that workflow process, definitely before it's merged, but hopefully not not needing it at all unless the the sort of the logic determines you know what I need someone to look at this point to decide what to do with it.
• 32:39
  SPEAKER_00: But you know, we're we're working on that to try and try and deliver a a more reliable platform.
• 32:45
  SPEAKER_00: We'll we are using AI to try and help with our um performance optimizations on on the scalability side of stuff, and then you know, we're definitely working towards uh an optimal mechanism where uh we can patch when there when there are sort of third-party libraries that we might use where there is a vulnerability spotted, we want to be able to create a patch and be able to deploy it for testing within a very, very small window and automate as much of that.
• 33:14
  SPEAKER_00: So um, you know, working on again factories for delivering that sort of capability.
• 33:20
  SPEAKER_00: So yes, one metric is is moving, and we're trying to, I guess, catch up.
• 33:26
  SPEAKER_00: So we're all trying, but we're all using AI to try and get there.
• 33:28
  SPEAKER_02: So I guess just to put a bow on it, I mean, within the tech landscape that we operate in today, where does AI not fit within the SDLC?
• 33:39
  SPEAKER_00: So it doesn't fit um where you can do things more efficiently um without using AI.
• 33:45
  SPEAKER_00: Uh it um so we don't want to be burning tokens to deliver a process.
• 33:52
  SPEAKER_00: We could deliver that in good old-fashioned code.
• 33:54
  SPEAKER_00: Um, you know, that that just doesn't make sense at all.
• 33:56
  SPEAKER_00: The other thing about AI, it's it's generative AI that we're using, and by its very nature, there's an element of non-determinism about it.
• 34:06
  SPEAKER_00: So it might do it might um behave one way today, and you get a different answer tomorrow.
• 34:13
  SPEAKER_00: And and where we need a defined process to say, or or whatever it is, to say this is this is the logic that we want to flow, uh, you know, that's where we would uh not use AI.
• 34:26
  SPEAKER_02: So that yeah, I think I I I think that's a hidden cost, right?
• 34:29
  SPEAKER_02: That doesn't get highlighted very often, but it's just probabilities.
• 34:34
  SPEAKER_02: And and you can and at the end of the day, I mean you can get things to as close to non-zero as possible, but it still is probabilistic, and there still is the possibility that the outcome varies.
• 34:48
  unknown: Agreed.
• 34:48
  SPEAKER_02: Regardless of how tight you yeah, how tight you prompt it, like it still could produce something different.
• 34:54
  SPEAKER_00: Yeah, internally, we've we've talked about um yeah, a level of maturity signal to say, you know, you need to recognize where AI isn't the right solution because it's not throw AI at everything and um and you get the answers that you want always.
• 35:11
  SPEAKER_00: So you need to go through that evolution to almost then be able to make that judgment cool yourself, I think.
• 35:17
  SPEAKER_02: I think that's a good lesson, just tying this back to industry as well.
• 35:20
  SPEAKER_02: Um, because we're gonna have you know folks from industry that are that are following along as well, right?
• 35:25
  SPEAKER_02: Um there is there is certainly, especially at the time and place of of current maturity, uh, there's a time and a place for uh, you know, delivering things agentically, and there's a time and a place for uh having humans.
• 35:42
  SPEAKER_02: And I obviously, you know, the the world of of travel and travel assistance, which is near and dear to me, uh you know, there still is such a human touch and human element to to that industry that um there are certainly slices that are apt for innovation, but but there are also very important slices um that need to be human-led.
• 36:06
  SPEAKER_02: Um so Al, you thank you.
• 36:09
  SPEAKER_02: I mean, you've been extremely gracious with your time.
• 36:11
  SPEAKER_02: I have one more question, and it's it's just sort of open-ended.
• 36:15
  SPEAKER_02: Um, what what comes next if you think you know, three, six months down the road?
• 36:21
  SPEAKER_02: Um, what what are you focusing on?
• 36:23
  SPEAKER_00: So I'm really pleased you didn't say three or six years because um uh I'd have been stargazing, definitely.
• 36:29
  SPEAKER_02: Then then you then you might have heard what's happening if you're on Mars, right?
• 36:33
  SPEAKER_02: I mean they may have uh data transmission at that point in time, yeah.
• 36:37
  SPEAKER_00: I think um the focus for us right now on the SDLC side is um to evolve um our processes as I was talking about earlier.
• 36:49
  SPEAKER_00: So so right to the beginning, you know, we started off with with um generating requirements and um and generating a plan and then and then building it.
• 37:01
  SPEAKER_00: And then what we're doing is is taking those simple processes and making them more and more complex and and dealing with um uh many different actors that can come in and feed into the uh into the software development lifecycle process.
• 37:17
  SPEAKER_00: Like, wow, there's a new security issue um that's just been raised from our from our sort of commercial platform that can notify us.
• 37:25
  SPEAKER_00: What that needs to go in.
• 37:27
  SPEAKER_00: Wow, I can I can look through the logs in in the 180 environments that that we have running.
• 37:34
  SPEAKER_00: I can notice um errors coming from those logs, I can feed that into the loop.
• 37:39
  SPEAKER_00: I've got clients with requirements, I've got um maintenance to do.
• 37:43
  SPEAKER_00: These these can all be um factories that can help us deliver a more efficient process using AI where it's needed and using code where it's not, and most importantly, making the humans the most uh do the most important work that only humans can do, which is fundamentally using their their knowledge and their judgment.
• 38:06
  SPEAKER_00: Andrew, what are your thoughts?
• 38:07
  SPEAKER_02: What are what are your thoughts on that?
• 38:09
  SPEAKER_01: Yeah, no, I was gonna um I was gonna mention an example we talked about uh before how with within one of our products where where we made some underlying sort of technology changes and and and upgrades and and our sort of claims team with uh with a few of the grads took took many months to to sort of process that upgrade.
• 38:30
  SPEAKER_01: Uh in in the other products, you had AI um lead that.
• 38:36
  SPEAKER_01: So some of those sort of necessary sort of maintenance tasks and and and sort of, if you like, technical debt pieces.
• 38:44
  SPEAKER_01: Um we've we've already seen examples where it's just changed and you know allow people to perhaps focus on more value add, you know, changing some of the underlying libraries.
• 38:56
  SPEAKER_01: Obviously, vulnerabilities are important, but some of the sort of libraries that we use just to move to the next versions to keep up with technology isn't adding a lot of value to the person, isn't adding a lot of value to our products, but is a necessity to make sure you've got supported products.
• 39:13
  SPEAKER_01: And and that one was intriguing to me that that we had a small, small bunch of of pretty junior guys but spent many months upgrading those libraries uh in our claims platform, and then we automated it all in our policy platform and left people focused on on sort of client deliveries.
• 39:32
  SPEAKER_00: True.
• 39:32
  SPEAKER_00: Another thing as well as as maintenance, which I completely agree with, um, is that we've been able to tackle some issues that maybe we couldn't have justified previously from uh an investment perspective.
• 39:48
  SPEAKER_00: So we've recently um delivered a um a migration module that would have taken many man years to build without AI, and um we delivered a framework and an implementation to do a a midterm migration of uh of a book of business um on the policy side.
• 40:12
  SPEAKER_00: Uh and it was fully built with with AI.
• 40:16
  SPEAKER_00: I mean, most of our engineers have not been writing code since the beginning of of this year, but this this one was done with a big um you know six-week uh discovery, sort of agentic discovery process where we were creating huge amounts of um knowledge, requirements, specification, and then went through a build process and then with went through sort of many iterations to to deliver it.
• 40:42
  SPEAKER_00: Uh and it's uh it's actually something that I'm I'm really proud of in terms of a a delivery, and we've been wanting to do it for many years, but we've just never been, you know, we've never had, let's say, a two-year window to say, right, we want to do that, and we were able to do it in in a matter of months.
• 41:00
  SPEAKER_00: So stuff like that is um is uh great for for companies like us.
• 41:07
  SPEAKER_01: And completely reusable, of course.
• 41:09
  SPEAKER_01: Whereas before we were doing a lot of sort of person-dependent sort of handcrafting of code.
• 41:16
  SPEAKER_01: Now we have a much more robust, scalable, uh, auditable sort of function.
• 41:22
  SPEAKER_01: Um, and and in our world, as as you know as well as us, uh Elliot, um, migrations, whether they're at renewal or or sort of the the midterm big bang migrations are are pretty critical to our customers and one we always get right.
• 41:37
  SPEAKER_01: But now we have uh uh some tooling to make that um make that more professional.
• 41:43
  SPEAKER_02: Well, I think it just it gives us the confidence to to try this same process in other domains, right?
• 41:51
  SPEAKER_02: I I think it it was a success, uh an overwhelming success.
• 41:55
  SPEAKER_02: And and now I think we can look at other areas uh where, again, it's something that was put on the back burner, you know, now is there potential to be able to deliver it, or areas where we wouldn't even touch it previously, we wouldn't even think about doing it previously that we can now deliver.
• 42:14
  SPEAKER_02: So it's tremendously exciting, but again, just to sort of put the capstone uh on the episode, there there's a there's a human component to all of these things as well, right?
• 42:25
  SPEAKER_02: So um, you know, we we still we still are constrained by humans, um, by time.
• 42:34
  SPEAKER_02: Ultimately, you know, time still moves at the same pace every day, right?
• 42:38
  SPEAKER_02: So um there is that opportunity cost of of what we can actually do and and what the impact is to the business.
• 42:45
  SPEAKER_01: Um and and humans remain critical, you know, and that's the point.
• 42:49
  SPEAKER_01: You know, there's a lot of a lot of uh news stories that AI is going to replace humans, you know.
• 42:58
  SPEAKER_01: In our world, you know, we're not replacing these developers.
• 43:01
  SPEAKER_01: We're definitely augmenting.
• 43:03
  SPEAKER_01: Um we obviously uh joke internally that um uh we now have multiple more engineers that just all call clawed, but it does um it does allow us to sort of upscale.
• 43:18
  SPEAKER_02: It's funny.
• 43:18
  SPEAKER_02: I was reading a TechCrunch article a couple weeks ago where there's been this mass boomerang effect.
• 43:23
  SPEAKER_02: I mean, you know, uh all of these companies that were leading in layoffs have all just rehired loads of engineers by the thousands.
• 43:31
  SPEAKER_02: Um, so I absolutely I I think I think there's the hype.
• 43:35
  SPEAKER_02: Look, the the technology still has to mature, it needs to industrialize in order for us to realize uh, you know, what it's actually going to be and how it's going to fit.
• 43:46
  SPEAKER_02: Um, it's just encouraging to to know where we are, you know, in that journey.
• 43:52
  SPEAKER_02: Uh gentlemen, it's been a pleasure.
• 43:54
  SPEAKER_02: Uh hopefully this is one of many to come uh in in conversation with both of you.
• 44:00
  SPEAKER_02: Uh really an illuminating discussion.
• 44:02
  SPEAKER_02: Uh so thank you so much for your time.
• 44:04
  SPEAKER_02: And until uh next time.
• 44:07
  SPEAKER_02: Take care.
• 44:08
  SPEAKER_02: Thanks, guys.
• 44:08
  SPEAKER_02: Thanks, Daddy, and then we're gonna go to the hospital.