Latest / CXOTalk / Autonomous Software Development at Enterprise Scale: Inside a 1,000-Developer Pilot (with Blitzy) | CXOTalk #918
Transcript
- 0:00The percentage of all the completed work done autonomously
- 0:04by Blitzie, it ranged from 80 and 95%.
- 0:08Depending in the use case. We accelerated development
- 0:12velocity of between 5:00 and 10:00 X.
- 0:15The human is not writing the code.
- 0:17The human is directing a platform on how to write the
- 0:20code. That's a huge change in
- 0:22paradigm. That's Enrique Ibarra, CIO and
- 0:26Head of Business Transformation at GNP, Mexico's largest
- 0:31insurance company. Our conversation covers AI
- 0:34adoption, change management, and autonomous software development.
- 0:40Our main operational system is a mainframe based map with IBM
- 0:44components. System has been running for a
- 0:46little bit over 20 years. The initial rationale for
- 0:49modernizing this application was basically cost.
- 0:53That was one of the concerns, but it was not the only one.
- 0:56In a few years from now, it's going to be much harder to get
- 1:01COBOL resources. I mean, if you go to the
- 1:04universities, students don't learn COBOL in the universities.
- 1:07There's no real interest. That's going to be an issue I
- 1:10guess in the future to in terms of how do we give longevity to
- 1:14this asset. We started a few years ago
- 1:17incorporating the typical coding Co pilots.
- 1:22We, we incorporated several informally.
- 1:26We started to provide all of our developers with Co pilots
- 1:30without formally measuring their increasing productivity.
- 1:33We just basically use it, find it useful, take advantage of it,
- 1:37and that's being expanded among the all of our developer base
- 1:42and we have roughly about 1000 developers by looking at
- 1:45different industry solutions, we find we basically learn about
- 1:49blitzing the value proposition sounded too good to be true.
- 1:55It was a little bit like magic is like you give me the
- 1:58specifications, give me the code and we will autonomously
- 2:02generate everything. It was very, it was very
- 2:04attractive. So we went and visited blitzing
- 2:08their their offices in Boston to learn, you know, what was their
- 2:12vision, what was their product? I mean, just meet with, you
- 2:15know, founders on the management and we like the approach.
- 2:19It was very aligned with what we're trying to achieve in the
- 2:23in, you know, here in GMP. So we decided to execute a pilot
- 2:28that will sufficiently evaluate, you know, the abilities of Blitz
- 2:32C in our own environment and with our own code.
- 2:36Tell us about that pilot and what were your goals for the
- 2:41pilot? The goals of the pilot were to
- 2:44evaluate, you know, the capabilities of Blitz C as a as
- 2:48a software development platform. So what we did is we selected
- 2:53one an existing system that that we have real system that
- 2:57actually had a, you know, a number of requirements that had
- 3:01to be executed anyway. And what we decided is to test
- 3:06different types of use cases using this existing system.
- 3:10So one of them was a back end migration.
- 3:14I mean the back end was written in an old version of Java, Java
- 3:178, and we needed to modernize. Got back into Java 21, that was
- 3:23very clear. Same thing with the front end.
- 3:25Front end was developed in a very old version of Angular.
- 3:28I think it was Angular 11, I might be wrong and we needed to
- 3:32upgrade it to one of the latest versions.
- 3:34We also wanted to test a specifically use case to build a
- 3:39new feature autonomously that is providing the right prompt to
- 3:44the platform of you know, describing the new functional
- 3:49feature in business terms that we wanted to build and having
- 3:52the system built the new feature for us.
- 3:55And then a fourth use case where we wanted to do the remediation
- 3:59of a number of security vulnerabilities that the system
- 4:02had. So we thought that, you know,
- 4:05with that breadth of use cases, it will be a nice spectrum to to
- 4:10basically test the capabilities of the system in a real life
- 4:15environment, which was our environment connected to our Git
- 4:18lab repository where all of our code resides and connecting it
- 4:23to our CICD pipeline. Basically, that's what we wanted
- 4:26to test. Why didn't you just give these
- 4:28small teams the opportunity to vibe code their way into this
- 4:34modernization? That was perfectly possible.
- 4:37Most of our teams, they're using, you know, Co pilots to
- 4:41basically work faster and have the Co pilots, you know,
- 4:46partially developer, they're trying to develop and test what
- 4:49they're doing. But this is a different type of
- 4:52platform. I mean, there's no, there's no
- 4:54ID initially here. It's just a platform.
- 4:57You provide a very detailed prompt of what you want to
- 5:00achieve and then the system using its own internal logentic
- 5:05architecture basically autonomously creates all the
- 5:10different software changes or creates the new software that
- 5:13needs to be incorporated into your project.
- 5:15That's what we wanted to test, yes.
- 5:17I mean, we could have given this to a team, you know, given
- 5:21different not only one, but you know, even several types of
- 5:24copilots. And we're working that way
- 5:26already. But we have never worked with an
- 5:29autonomous platform. And the idea was to explicitly
- 5:32test a platform that has a different, you know, paradigm of
- 5:37usage than the rest of the byte coding tools.
- 5:41So you were really looking at shifting the strategic value of
- 5:46technology development? Right, exactly.
- 5:51What does autonomous development mean in practical terms at GNP
- 5:56and and how does Blitzie fit along with the other tools in
- 6:01your AI code generation stack? We are interested in increasing
- 6:07agility as much as if it is feasible.
- 6:11That has been our, you know, our objective for for many years.
- 6:14So if we're in adapting methodologies, incorporating
- 6:18tools, you know, streamlining our CICD pipeline in order to
- 6:23try to achieve agility, we keep improving, trying to make
- 6:27changes to improve agility. We want to change the role of
- 6:31humans in the software development process.
- 6:34We want to do it to achieve agility by coding.
- 6:37And you know, all the tools that assist work groups are great.
- 6:41I mean, they do provide value, but we wanted to test a new
- 6:45generation of or or a new type of tools for generating software
- 6:50in an in an autonomous way. That's a value proposition of
- 6:53bleed. See, it looked, sounded very
- 6:55attractive to us and that's what we wanted to test.
- 6:58What's the value of that is, you know, you have to learn a new
- 7:03skill, which is being very good at prompt engineering.
- 7:06You have to be, you have to create very precise and complete
- 7:11prompts for for the platform. But if you do, and that's the
- 7:14skill that you know, I've seen that our engineers have learned
- 7:18fast. Once you get the right prompt,
- 7:21then you get the right results in the speed.
- 7:24That was very impressive. What about enterprise
- 7:27requirements such as security, governance, architectural
- 7:32standards? How does this approach support
- 7:37the corporate technology requirements that are needed?
- 7:41You definitely need to be very specific regarding your own
- 7:46corporate guidelines. We do have our own guidelines,
- 7:48you know, technical guidelines. We have our own security
- 7:52guidelines too, regarding characteristics that code has to
- 7:55meet and the type of test we want to execute on the on the
- 8:00components that we develop. But what we found is that, you
- 8:04know, those specification, those guard rails, those are part of
- 8:10the input that you provide to a platform like Blitzing.
- 8:13So you not only provide you know the functional specs and the
- 8:17intention of your project with all the functional you know
- 8:20specifications, you also provide all the technical and security
- 8:24guidelines too. What results have you seen
- 8:27across development? Velocity, Quality of code?
- 8:31Cost Hours saved. We were very pleased with the
- 8:36results that we got. We roughly were able to
- 8:41accelerated development velocity of between 5:00 and 10X in terms
- 8:46of engineering velocity and the percentage of all the completed
- 8:51work done autonomously by Blitzie, it ranged from between
- 8:5680 and 95%, depending in the use case.
- 8:59For example, use cases that you basically want to upgrade from
- 9:04an old version of Java to a new version of Java.
- 9:07And you do have to provide the guidelines of how to do the
- 9:10upgrade. It's not, it's not as simple as
- 9:12just mentioning just upgrade it. Just upgrade from May to 21 and
- 9:16I'm done requesting. Now you do have to provide a lot
- 9:18of information on how to do it. But those type of projects, we,
- 9:23we basically saw that, you know, the, the percentage of
- 9:28autonomous completion was close to 100% front end.
- 9:34You know, modernization of front ends is different.
- 9:36Those, it's a little bit more tricky, but we got around 80%
- 9:40but and an 80%, you know, the, what basically the team told me
- 9:44is 80% is just wonderful. I mean, and so I mean we still
- 9:48have to do 20% and for the reminder 20%, they do by coding,
- 9:54bringing their IDs, they're bringing their copilots and they
- 9:57finish the reminder 20%. And you know, overall that
- 10:02accelerates the process a lot. So they're using Blitzy to do
- 10:06the heavy lifting and then using lighter weight tools to do the
- 10:11fine tune polishing. Exactly to basically complete
- 10:16the project. Enrique, how did the developer's
- 10:20jobs, roles and daily work change as a result of this shift
- 10:27to autonomous development? The role of humans changes.
- 10:32You still need developers. If you're going to develop a
- 10:36system, you need to provide technical guidelines, you need
- 10:40to provide guidelines related to the platforms where the software
- 10:44is going to run. And an end user cannot do that.
- 10:47I mean, in in our case, we, we deploy our systems mainly in
- 10:51Google Cloud. So you need to know about the
- 10:54technical features and then and, and the technical settings that
- 10:57you need to enable in the in the cloud platforms in order for the
- 11:01system to run. An end user is not going to do
- 11:04that. So you still need a program, but
- 11:06the role changes completely. I mean, it's the human is not
- 11:10writing the code. I mean, the human is directing a
- 11:14platform on how to write the code.
- 11:16So that's, I mean, that's a huge, you know, changing
- 11:20paradigm. How did the developers react?
- 11:22You have to be careful with the change management and the human
- 11:27resistance to change is something that you cannot
- 11:30overlook in general. Some of them were skeptic at the
- 11:34beginning, You know, it was like this sounds too good to be true.
- 11:39As we started work with the platform and getting results,
- 11:43their attitude changed very rapidly and they started to get
- 11:48interested and they started to see value.
- 11:51They were intellectually challenged by, you know, this
- 11:55new type of work. So they not only accepted it,
- 11:58but they were very motivated in, in testing the platform in, in
- 12:03testing the different prompt, you know, techniques to try to
- 12:07achieve better results from, from the platform.
- 12:10And I think that overall the group that so far has worked
- 12:14with Blitzi at GMP, they're they're excited about it and
- 12:18they're very enthusiastic about, you know, basically expanding
- 12:23the use of Blitzi to the rest of of our software development.
- 12:26What advice would you give to other CIOs on transforming their
- 12:31engineering organization to be AI native in this same way?
- 12:36You don't just flip a switch to full autonomy.
- 12:40That doesn't doesn't work that way.
- 12:42You have to build trust through a faced human in the loop
- 12:46approach. You need to target the friction.
- 12:50You know, we need to deploy agents to 1st to handle high F4,
- 12:56low risk friction points first. Such as you know, this
- 13:01modernization projects of basically upgrading the
- 13:04programming languages or writing system documentation for
- 13:09example, or generating test suites.
- 13:13This type of tasks that generally are low friction.
- 13:16Then you need to shift the engineering mindset.
- 13:19We have to train our engineers to transition from being
- 13:22creators to editors and orchestrators.
- 13:25It's a different role, and the human leader's job now becomes
- 13:29defining the prompt, reviewing the architecture, and validating
- 13:34the AI's execution. You are a technologist, You're
- 13:38also a business person. Can you describe the strategic
- 13:43benefits that this approach and the faster development speed
- 13:48unlocks for GNP? Development speed is isn't just
- 13:53about writing code faster. I think it on lots different
- 13:57strategic advantages and 1 is like first to market innovation.
- 14:03I mean, you know, we can design, deploy and iterate on like new
- 14:09insurance products in weeks rather than in months or
- 14:12allowing us to capture market demand before our competitors
- 14:16even react. That's one of the goals that we
- 14:19have. The other benefit and the other
- 14:21objective that we have as a company is improving customer
- 14:24experience. So this speed and this agility
- 14:27empowers us to continuously ship digital improvements, so like
- 14:33instant claim processing or, or seamless onboarding and
- 14:38basically try to meet the, the high expectations of the modern
- 14:41consumer who, who is, you know, basically they're more demanding
- 14:45all the time. It shifts our technology
- 14:48organization from simply maintaining the business to
- 14:53actively dictating the pace of the Mexican insurance market.
- 14:57That's that's our goal. As you plan to roll this type of
- 15:02agentic product development out across the company, what are
- 15:07your thoughts? What are your concerns?
- 15:09What's the approach that you're taking?
- 15:11We are now a process of expanding the use of the Blitzy
- 15:16platform. We have incorporated seven
- 15:19additional teams now that they're going to be trained and
- 15:22they're going to be Twitter, they're going to be some hand
- 15:24holding of working with them between Blitzy engineers and the
- 15:29our engineers that have already gone through the different
- 15:32projects the last four months. So we're going to gradually
- 15:36start expanding the use and what we are going to measure and make
- 15:40sure that it gradually happens and becomes a reality is that
- 15:45our speed of execution basically improves at a very noticeable
- 15:50and exponential rate. And gradually we will continue
- 15:53to expand. I mean initially we have seven
- 15:55groups. Once this groups are
- 15:57sufficiently mature, they will continue working in this fashion
- 16:00and they will continue incorporating groups.
- 16:03We think that in the two year time frame we will be able to
- 16:06change the paradigm in in the company.
- 16:09And how many developers do you have?
- 16:11Currently we use in between our own developers and external
- 16:14developers from software factories.
- 16:16We have around 1000. Ideally we should not rely on
- 16:20external developers in the short to medium term and we will only
- 16:24keep internal employees basically working with
- 16:28platforms. So speed was the driver, but
- 16:31ultimately you will also be reducing cost.
- 16:35Yes, definitely. Any final advice on how
- 16:39engineering leaders can get started or should get started
- 16:43with autonomous development? You can easily pick within your
- 16:46organization a use case. I mean, either a very old system
- 16:51that need to be modernized or a system that is giving you a lot
- 16:54of problems, you know, during regular operation because this
- 16:58has a lot of bugs or has a lot of security vulnerabilities.
- 17:02And try these type of tools. I mean, you know, big bigger use
- 17:06case. Pick an existing system and just
- 17:10try it and you know, sandbox it, do it carefully, you know,
- 17:14select the right initial set of engineers to work in this
- 17:18project to make sure that they will be able to to do the
- 17:22transition and to understand this new paradigm.
- 17:25Test it, see how it goes. And you basically elaborate from
- 17:29there. You just presented a very
- 17:31practical and wise textbook on AI adoption.
- 17:36I think it's just common sense, but thank you.
- 17:39Enrique Ibarra, thank you so much for taking time to speak
- 17:43with us today. No thanks to you, Michael.
- 17:45It was an honor. Thank you.