Latest / CXOTalk / Building trust in AI: A CIO's perspective | CXOTalk #869
Transcript
- 0:00How can Chief Information Officer survive and thrive
- 0:05amidst the turmoil, madness, and opportunity of AI?
- 0:11Today on episode 869, we're getting advice from two of the
- 0:17world's foremost CIO advisors, Tim Crawford and Isaac Saccolic.
- 0:23Isaac, let me jump to you first. How can early adopters create
- 0:30effective AI strategy for and we're talking to CIO's.
- 0:35Very first thing I think of when I think about AI strategy
- 0:39actually is our own governance. You know, I need to make sure
- 0:42that my staff understands what data they can work with, that
- 0:46the data is got enough volume, it's cleansed enough to be using
- 0:51inside some kind of AI model that I'm going to use it with,
- 0:55that people understand what some of our high level objectives are
- 0:59in terms of what we're trying to go after.
- 1:01So I'm actually starting with governance as opposed to just
- 1:04going after and trying out new things.
- 1:07I think the second thing, and this is where a lot of
- 1:09organizations do fall off, is I'm going to talk about change
- 1:12management very early on. A lot of the early successes in
- 1:17AI has been happening in the IT groups.
- 1:20If I want to start bringing AI into my sales and marketing
- 1:23customer support groups where there's a lot more value, I need
- 1:27to be thinking about how I'm going to educate the groups,
- 1:30where I'm going to give them some leeway to use some tools
- 1:33and work with them. So I'm going to really be
- 1:35thinking about change management much early in AAI strategy, then
- 1:39maybe I was able to get away with in the past.
- 1:42And then lastly, I'm looking for value other than just
- 1:47productivity. There's a lot of talk about AIO
- 1:49let's us be more productive. I can't fund transformation
- 1:53efforts or AI for that matter, on just productivity
- 1:56improvements. So I want to hear where is it
- 1:59transforming our business, want to hear about where workflow is
- 2:02going to be improved, in what ways, what value what, where
- 2:05we're driving quality. So I'm looking for value beyond
- 2:08just productivity to be part of my strategy.
- 2:12Tim, thoughts on this set of issues?
- 2:15Isaac has just raised a bunch of topics from innovation to
- 2:21culture and how AI is different from traditional IT.
- 2:27Thoughts on this? You have to think about the end
- 2:30in mind when you start down this path.
- 2:33But but as part of the prerequisites for AI, you have
- 2:38to start to break things down. You know, Isaac talked about
- 2:40transformation. I think about it similarly, but
- 2:43I use a couple of different terms in defining different
- 2:47types of AI initiatives. One type is around innovation
- 2:53and the other type is around efficiency.
- 2:55And as Isaac mentioned, you know, efficiency becomes kind of
- 2:58hard because how are you really driving toward efficiency and
- 3:03what metrics are you using and how are you actually measuring
- 3:06that? There's a lot of gamification
- 3:08that goes into it and a lot of question around whether you can
- 3:11actually achieve reasonable efficiency gains considering the
- 3:16amount of effort you're putting in.
- 3:18As opposed to innovation efforts, which have a much
- 3:23higher output and much higher value to the organization than
- 3:28efficiency in the long run. And so therefore, they tend to
- 3:31have lower hurdles to get over because efficiency is something
- 3:35that technically you could do with more people, but you're
- 3:39using technology as a means to, to advance your position.
- 3:43Innovation is a definition. I'm looking at those types of
- 3:47efforts is these are things that you couldn't do any other way.
- 3:51These are insights you could not get by throwing more people at
- 3:54the problem. And then the last component that
- 3:57you have to think about before you even start to lift a finger
- 4:00on AI is you have to start thinking about your data
- 4:03strategy. You have to have a comprehensive
- 4:06data strategy in place because if you don't have good data
- 4:09going in, you're going to have bad results coming out.
- 4:12And so this is going to require a different way of thinking for
- 4:16a lot of organizations in terms of just simply how they think
- 4:19about data, not just where the data is and how they can start
- 4:22to bring it together. But also, as Isaac mentioned,
- 4:26where's that governance? How are we start to think about
- 4:28governance? There's a lot of regulation
- 4:30that's coming about over 400 components across just the US
- 4:36states alone, something like 40 in California.
- 4:40So there's a lot to get your arms around there.
- 4:44And I think you have to start with the end of mind and be real
- 4:47methodical about how you traverse down this path.
- 4:50On the efficiency side, I think we're at a point where CIOs can
- 4:55start looking top down and start reinventing workflow.
- 5:00How are we hiring people? What does this mechanical
- 5:03operation look like if we built it from the ground up using data
- 5:09analytics, AI, automation from the ground up?
- 5:12A lot of what we've been doing over the last two years in
- 5:15fitting Ain has been taking a piece of that process or taking
- 5:20a process that we haven't done well and saying we're putting in
- 5:22a technology or a step in here that's going to either make it
- 5:26more efficient or more scalable or improve quality.
- 5:29We're really at a point where we can start reinventing our
- 5:32workflow. That's going to rattle a lot of
- 5:35heads. We're always used to how things
- 5:38are running today. We're always used to what our
- 5:40job is in that function. And so that flows back into what
- 5:45I was saying earlier around change management.
- 5:47We need programs for our organization, training programs,
- 5:51learning programs so that they start understanding and
- 5:54embracing what their work is going to look like as AI is
- 5:58brought in more fundamentally into processes.
- 6:01And when I think about innovation, Tim, I go back to
- 6:05the days when we launched mobile technologies.
- 6:08In the early days of mobile, it was, you know, we were taking
- 6:12web interfaces and slapping them on a small screen and say, hey,
- 6:16we have a mobile site. And so the, you know, the
- 6:18technology got a lot easier the to do mobile first capabilities.
- 6:24We started seeing app stores come out, we start seeing mobile
- 6:27first capabilities and now we start reinventing the user
- 6:31experience. We haven't seen that so much
- 6:34with AI yet. Even our agents, the genetic A
- 6:37is that we're talking about and bring workflow and there's most
- 6:40of the talk around that and most of the value we're seeing is
- 6:44inside our organization. But look over the next year or
- 6:47two and saying how are we going to do something completely
- 6:49different now that we can put an agent in place and how does that
- 6:53change the customer experience? If you're watching on Twitter,
- 6:58pop your question onto Twitter using the hashtag CXO talk.
- 7:02If you're watching on LinkedIn, just ask your question, share
- 7:06your comment in the LinkedIn chat and ask these guys pretty
- 7:12much whatever you want. It's a great opportunity.
- 7:14You can get free consulting right now.
- 7:17So take advantage of it. So you're you're both describing
- 7:21attributes of IT, culture, deployment and so forth that are
- 7:28not really unique or new Culture is a human nature phenomenon and
- 7:36change management and all of that.
- 7:38Is there anything that's really new about about AI that changes
- 7:44the CIO role that has a big impact here that's different
- 7:47from what we've had in the past? We have to get more comfortable
- 7:51with how we trust technology and also how we bring automation
- 7:57into the fold. I mean, even with technology, a
- 8:00lot of applications still rely on a human component.
- 8:04And with artificial intelligence and some of the the newer
- 8:09technology that's coming, coming out, it requires us to get more
- 8:13comfortable with automation in ways that we haven't had to deal
- 8:17with in the past. And that is new.
- 8:19That's very new because we haven't necessarily had to get
- 8:23focused on our processes such that we need to make sure that
- 8:28they are sound, they are clear. And this touches into some of
- 8:32the things that Isaac mentioned around change management.
- 8:35You know, do we have good processes in place and good
- 8:39change management in place? And if you just turn around and
- 8:41say, oh, we're just going to automate it, that's a very
- 8:44dangerous decision to make. And so one of the things that's
- 8:48new is it's causing us to kind of go back and rethink how we do
- 8:52things to speed it up to ensure that we are getting more
- 8:56accurate output and ensuring that we're not automating bad
- 9:00processes. And what that means is that
- 9:03there's a cultural shift to embrace technology more fully
- 9:08than we ever have. And that is change.
- 9:11That's A, and it's a, it's a hard cultural change for a lot
- 9:15of organizations that have maybe been more accustomed to,
- 9:18especially as you get to larger organizations more accustomed to
- 9:22a more incremental change. You know, we're making these
- 9:26small improvements that doesn't suffice anymore.
- 9:29You have to make tectonic improvements.
- 9:31You have to make transformational improvements
- 9:34because that's exactly where your competition is going.
- 9:37And if you don't make those, you'll be left in the dust.
- 9:40One of the differences, Michael, is that it's happening to
- 9:43employees in our back offices a lot faster than what automation
- 9:48did. You know, their jobs are
- 9:50changing and it's looking like the things I used to do, I maybe
- 9:54no longer have to do it at all or do it fundamentally different
- 9:57than I've ever done before software development.
- 10:00I mean, if you asked any of us 4-5 years ago could machines
- 10:04code for us, we probably would have said no.
- 10:06I have a blog post around this now somewhere around 20 to 30%
- 10:10of code being written by AI as being accepted inside being
- 10:16pushed and being pushed into production being accepted by the
- 10:19development group. So it's changing fundamentally
- 10:22how our employees are working. And for some, you know, they
- 10:26embrace it, they learn it. Or for a lot of other people,
- 10:29they're still scratching their heads and saying, what do I want
- 10:32to do now that AI is being able to do some of the things that
- 10:36I'm doing? I think of that as a writer,
- 10:38right? Am I writing for people or am I
- 10:40writing for for LLMS that are going to bring my data in his
- 10:45content and use it to answer questions?
- 10:47So it's a, it is a fundamental shift.
- 10:49And then with the CIO role, I mean, we've seen this a little
- 10:52bit in the past, but it's getting harder.
- 10:54You know, every time there's a whole new discipline that we had
- 10:57to learn, you know, whether it was digital or data and now AI,
- 11:02if we didn't learn it as ACIO enough and build up enough
- 11:05competency around us to lead these areas, the board, this the
- 11:10executive group would look at and say, we're going to go hire
- 11:13a chief digital officer. We're going to go hire a chief
- 11:15data officer. Now we're saying we're going to
- 11:18go hire a chief AI officer. And learning AI is fundamentally
- 11:22a little bit more difficult. We have to scale our ability to
- 11:25learn from the organization. So I think the the the goal for
- 11:30CIOs need to move really fast and make sure that our
- 11:34lieutenants and our high potentials are getting the
- 11:37enough leadership and getting out there and learning enough so
- 11:40that they can inform me as ACIO about how to invest and not
- 11:44meet. Going back to them and saying
- 11:46this is the area that I want to focus on.
- 11:49On Twitter, Mike Boyson says that to take advantage of these
- 11:57AI opportunities, he says you would need completely new
- 12:01business models to weave with AI, and those business models
- 12:06will drive what your data needs are.
- 12:10And then on LinkedIn, Greg Walters says that this is a long
- 12:18question, so I'm going to shorten it.
- 12:20I'm going to ask you guys to ask short questions so it's easier
- 12:23for the moderator that's me to to ask.
- 12:26So I'm going to shorten this. He says we're transforming from
- 12:29manual thinking to artificial thinking.
- 12:32Does this mean we will proceed by ignoring silos and
- 12:35hierarchies to change everything and ensure fully integrated
- 12:39organization wide adoption of AI And, and I think the the common
- 12:45thread here is the power of AI to drive disruption.
- 12:50But what kind of changes do we as CI OS need to make in order
- 12:54to take advantage of these opportunities?
- 12:58It kind of comes back to is AI going to drive changes to
- 13:01organizational structure and and how we think about organizations
- 13:05and operate as organizations. I mean, let's not forget an
- 13:08organization is a living being, right?
- 13:11It's an evolving living being and it changes over time and
- 13:15culture is just one component of that.
- 13:17I do think that AI is and it all we already have examples of how
- 13:22it is changing how we think about structures within an
- 13:25organization. Has it gone so far as to break
- 13:28down the silos broadly? Not yet.
- 13:32I think at some point that might start to happen.
- 13:35I mean, you could envision of position that we get to sometime
- 13:39down the road where an AI type of solution, magical mystical
- 13:46solution looks at how we work as an organization starts to learn
- 13:51how we engage with customers, how we engage with employees,
- 13:54what the market dynamics are, what the environmentals are from
- 13:58geopolitics to weather, and is able to actually tell us how to
- 14:04run our business more efficiently.
- 14:07That's actually something that we can envision and get to.
- 14:11And then kind of kind of building off of Isaac's earlier
- 14:15point. Now just imagine you need
- 14:17software to be able to do it. And that same system can then go
- 14:20and start to build some of this software.
- 14:23There's already talk amongst vendors, big, huge vendors with
- 14:29really big complicated applications and platforms
- 14:34around. Can we use AI to essentially
- 14:36rewrite the entire platform and what, how can we learn from the
- 14:42way people work to improve upon that?
- 14:45And then kind of feed that into the software.
- 14:48So the short answer is not yet broadly, but we can completely
- 14:54envision that. I do think that AI is already
- 14:56having an impact though on org change.
- 14:59Join our community, go to cxotalk.com Sign up for our
- 15:06newsletter so we can notify you of other live shows.
- 15:09We have a question now from Arsalan Khan on Twitter relating
- 15:15to this. And since you're the one who
- 15:18first brought up culture, let me address this to you.
- 15:20He says who decides what's a good process versus versus a bad
- 15:24process? Most standard operating
- 15:26procedures become archaic since people find other ways of doing
- 15:31things that are not even documented.
- 15:34And I'll ask you the Isaac to keep this pretty short because
- 15:37with the questions are stacking up.
- 15:39Our notion of process is changing dramatically.
- 15:41We used to think of linear process and linear handoffs.
- 15:45We used to think about how to automate pieces of this.
- 15:48And you know, what's changing is our ability to put intelligence
- 15:53to handle all the nuances in complex decision making.
- 15:57We want to be able to ask questions and knowing that the
- 16:01analytics, the data behind it is far more complex than what can
- 16:05go into a linear process. And I'm going to ask a question,
- 16:09you know, given what's happening in the United States over the
- 16:12last few weeks, how should I evolve my supply chain?
- 16:15What are my options? This is not a linear process
- 16:18anymore. This is about having the right
- 16:20data in place to make tactical decisions as things change and
- 16:24evolve quickly. And going back to what Tim was
- 16:27saying, I mean, we we may not be able to do strategic oriented
- 16:30questions at that scale just yet, but we certainly can get
- 16:34our data ready for that. And I think that's a big
- 16:36challenge for organizations to think think about today.
- 16:39Martin Davis asks this question. He says how does a CIO avoid
- 16:47pilot purgatory with AI? So who wants to explain briefly
- 16:53Pilot Purgatory and then give the antidote to Pilot purgatory?
- 16:58I just read a statistic that said 69% of companies are
- 17:03running 10 or more PO CS and 10% are doing 50 or more of and to
- 17:09some extent I, I don't have an issue with that.
- 17:11I mean, I think pilots and POC's, you know, not all of them
- 17:16are going to make it into production.
- 17:17A lot of them are learning exercises.
- 17:20A lot of them are really fishing into, into the underlying data.
- 17:24Can the data provide enough of a backing to support the AI, the
- 17:30hypothesis around the AI? And even when it does, does the
- 17:35AI that you've created drive enough business value to put it
- 17:39into production? So when you put that all into
- 17:41play, it's not surprising to me at least that a good number of
- 17:45these are not making it into production.
- 17:47I think the real issue is that organizations don't have a
- 17:51process to make that happen when they really having a winning
- 17:55PSCA pilot that's delivering. Do they have the change
- 17:58management in place to impact the people who are going to be
- 18:01affected by it? Can they bring that AI out at
- 18:04scale? Can they manage the operations
- 18:07around the model OPS and the ML OPS that go into monitoring a
- 18:11model and making sure that's still relevant?
- 18:13So I think there's, you know, you do so much work just to
- 18:16figure out what video seeds to go after, what data to go after.
- 18:20But even when you have a willing formula, you haven't done enough
- 18:23of production Alizing to figure out how to make it do that at
- 18:26scale. Tim, maybe you can help me make
- 18:29sure that I understand what Isaac said.
- 18:33So let me read that back to you and you tell me if this is
- 18:36right. OK.
- 18:37So I think what Isaac just said is there's a lot of people
- 18:41playing with AI. They don't really know what
- 18:43they're doing. They haven't thought through.
- 18:45Their organizations aren't ready.
- 18:46They don't have the data ready. But you know they're spending
- 18:50money because, well, it's not clear why.
- 18:52It's absolutely true. And every time I turn around, I
- 18:56see another survey, another some more anecdotal information that
- 19:01comes in that's basically supports that.
- 19:04Now, I don't want to, I don't want to come down on the idea of
- 19:08experimentation. I mean, having a culture of
- 19:11experimentation, especially within a technology org is
- 19:14really, really important. And that's important because you
- 19:17want people to use their imagination.
- 19:20You want people to bring that creativity in because ultimately
- 19:23that's what leads to change within an organization and
- 19:27potential ways for your organization to differentiate
- 19:31yourself from your competition. And so CI OS know that, see
- 19:35leading CI OS know that they need to find ways to really kind
- 19:38of explore these new areas. But as Isaac said, you do need
- 19:44to be somewhat methodical with it.
- 19:47And as we've gone through the last 12/18/20, four months,
- 19:51there's been a, a market shift where the bar has gone up that
- 19:55before you even start an AI experiment, you need to have
- 19:59some very clear business outcome and you need to understand how
- 20:03it ties to one of your company's objectives.
- 20:06And typically that falls into one of three camps.
- 20:09It's either around customer experience, employee experience,
- 20:12or business operations and supply chain.
- 20:14It's one of those 3. So I think that the thing
- 20:17Michael to to consider there and and to Martin's original
- 20:21question is you just have to think about what you're doing
- 20:24and what makes the most sense. But still don't don't make sure
- 20:29that you don't put everything together and expect it to be
- 20:33buttoned up from the start. You still have to experiment,
- 20:36but you need to more quickly come to an outcome to determine
- 20:40whether this is a project that has legs or whether it's
- 20:43something you need to to pull the RIP cord on.
- 20:46Tim, I like that part of it. I think that's what companies
- 20:49miss a lot is how long do you give a team to experiment And
- 20:53even if the experiment is a little bit open-ended, when do
- 20:56they come back with their vision?
- 20:59How is it aligned to the strategy?
- 21:00I mean, you can give people some time to go out and learn what
- 21:03the data is telling you, but you can't give them a mile long
- 21:07runway anymore. The expectation is that an ROI
- 21:11that a, an AI is going to be delivered in about 8 months.
- 21:13That ROI is going to be measured in about 13 months.
- 21:17That's pretty quick. I mean, we couldn't do apps at
- 21:19that rate 10 years ago. And we're saying we're doing all
- 21:23this data work, we're doing all this change management work.
- 21:25We're putting all this AI in place.
- 21:26We still have a skill issue here and we're trying to get to to
- 21:29value between 8 and 13 months as sentiment from boards and CE OS
- 21:34are looking at and saying, you know, the honeymoon's over, I
- 21:37better start seeing some value from all this investment I'm
- 21:39making. Liz Martinez on LinkedIn is
- 21:43screaming out during this conversation.
- 21:46Business case. Business case.
- 21:48Business case. I agree there needs to be some
- 21:52connection to a business outcome.
- 21:55I would just caution on using the phrase business case And the
- 21:59reason why, and we might be agreeing or disagreeing on this,
- 22:03Liz, is because a lot of times when people think about the
- 22:06business case, they put together this multi page document that
- 22:12could be 20-30 pages defining why they want to make this
- 22:15investment into this experiment and what the potential outcome
- 22:19is. And that's not what we're
- 22:21talking about. We're talking about these very
- 22:23quick wins. But I agree it needs to have
- 22:26some connection to the business as I mentioned earlier.
- 22:29So as long as that business case is very short and direct and to
- 22:34the point and as you start down the path, you can show
- 22:39connection kind of closing those connecting the dots, if you will
- 22:43to the outcome in short order in the time frame that that Isaac
- 22:47mentioned, then I'm great. I'm good with that.
- 22:50Just don't look for the fully like put together business case,
- 22:54business plan. The three-year plan, the full
- 22:56PNL No, I Tim, I use a one page vision statement and you know,
- 23:02and it's got, you know, information around customers
- 23:04around it. It's got information on
- 23:06strategy. You have to project the
- 23:08timeline, you have to project what OK, Rs you're impacting
- 23:13very important. You got to have a value
- 23:14proposition for the end user or customer that's benefiting and
- 23:19anyone wants it could come reach out to me.
- 23:21I'll be able to share it with anybody who's interested in it.
- 23:23But I agree with you, it's got to be simple.
- 23:25And most importantly, what we used to call business case is
- 23:29really about alignment, making sure that people who are working
- 23:32on this, making sure that stakeholders and making sure
- 23:35executives know what the objective is.
- 23:37I'm going to bring 2 questions together, and this is from
- 23:40Joseph Puglisi and Ashish Parulikar.
- 23:47I hope I have your name correct. And Joseph says speak to the
- 23:52need for governance. What lessons have you learned
- 23:56around early efforts to use AI? And Ashish says how can the
- 24:02company communicate its AI strategy and progress to
- 24:06employees and stakeholders that are inclusive and get adoption?
- 24:13AI governance is an extension over what we've been doing with
- 24:16data governance. There's some nuances to it that
- 24:19are important around areas around bias, around ethics and
- 24:23AI. But the, the main lesson learned
- 24:26I, I find is that we treated data governance and data
- 24:29security very separate tracks from our innovation and very
- 24:33often lagging. So we'd figure out what the
- 24:36innovation was around data, we'd cleanse our data, and then we
- 24:39start looking at what are the governance implications around
- 24:42it. But with AI, you need to flip
- 24:45that. And even with data, the advice
- 24:47I've given to groups I work with is, you know, I do everything in
- 24:51agile. I have my data scientists
- 24:54working on the model. I have my application developers
- 24:56working on the user experience. I need my data governance
- 24:59specialist there. I need my AI governance on that
- 25:02agile team to make sure that as we're working through this
- 25:06problem, we're doing things that align with compliance and
- 25:09regulation. As Tim mentioned earlier, these
- 25:11are changing very rapidly. There's a lot of depth to them,
- 25:14so not everybody has the answers around them, but I'm putting
- 25:17people who are experts around this right on my team so that I
- 25:20can do governance in parallel with my innovation.
- 25:23Couple of things to add in into that second question briefly is,
- 25:28number one, make sure you have good relationships with your
- 25:32head of legal, head of audit, if you're the CIO.
- 25:36Those are some of the first relationships you should be
- 25:39making within your C-Suite, within your peer network.
- 25:44And then this the second part of the question around
- 25:47communicating this out. I mean, many organizations have
- 25:50had success with creating a governance body or a council.
- 25:55And So what that does is 2 things.
- 25:57One, it starts to bring different ideas and perspectives
- 26:00into the mix. So you get those different
- 26:03personas engaged, but it also has a dual purpose, which is
- 26:07starting to communicate those shared objectives out to the
- 26:11rest of the organization. And so by bringing that
- 26:15together, it's not just sitting with the CIO or one individual,
- 26:19whether it's the head of audit, head of compliance, whomever,
- 26:22depending on your industry, but rather it becomes a shared
- 26:25responsibility. And I think that's probably the
- 26:27best way to go at this stage of the game.
- 26:30I'm going to add one person to that and that's the CHROI think
- 26:34is really important to have there when we talk about change
- 26:37in people's jobs and that changing that quickly.
- 26:40And a tip for CIOs, if you're looking for budget around
- 26:44training and learning and development programs, a lot of
- 26:47enterprises have that with the CHRO, right?
- 26:50And so bring them into the conversation around how AI is
- 26:53going to impact the business. And we have a question now on
- 26:58data from Michelle Clark on LinkedIn.
- 27:01And let me just remind everybody that this is your opportunity to
- 27:07ask your questions to get free consulting.
- 27:10If you're watching on LinkedIn, pop your question into the chat.
- 27:13I urge you to put these guys on the spot.
- 27:15These are two smart guys and ask the ask the hard questions, not
- 27:20to me, but to them. Michelle Clark says, how do you
- 27:23get clean data? How do you know that your
- 27:27medical diagnosis data, for example, isn't full of racial
- 27:33and gender bias? And this is really, really
- 27:36important for every business. Usually been a lot of financial
- 27:40services industry companies that have been using these data
- 27:44platforms watch longer with a lot more skill than in other
- 27:48industries. And because so many of us are
- 27:50investing in AI and looking at our data, we need to have those
- 27:54platforms in place. If you don't know if your data
- 27:56is cleansed enough, or is healthy enough, or is
- 27:59trustworthy enough, the go look at the platforms and see how to
- 28:02tune them for your type of data. Completely agree.
- 28:05Use the technology to your advantage.
- 28:08They're getting far more sophisticated now where where
- 28:10they're actually showing ongoing real time dashboards about the
- 28:14health of your data. So leverage that.
- 28:17And I would just add, I mean, you have to also bring the human
- 28:20component into the mix too, which is help people understand
- 28:24why things like bias and how they're bringing that data, how
- 28:29they're putting data into these systems, how it starts to impact
- 28:33things downstream. Let's go back to Twitter.
- 28:37A really important question from Naya on Twitter who says What
- 28:43are the top three challenges organizations face when
- 28:47pioneering AI apps? There's two sides to this, OK.
- 28:53One area you're going to see a lot of pioneering is built
- 28:57directly into the platforms that people are already using.
- 29:00So CRMCHRO platforms, they are all putting agents in place.
- 29:07They're all trying to say put more data in our environment or
- 29:11port more data accessible in my environment because I'm going to
- 29:14bring the AI to you. And, and so the, the real
- 29:18question there is understanding whether your data is ready for
- 29:23this is one of the things I don't see in some of the
- 29:25platforms is they can go, you know, do sales forecasting or
- 29:31they can go do assistance around hiring, but they're not telling
- 29:34you enough about whether the data in your platform is ready
- 29:38for that. Ask your people, all right, this
- 29:41is a great place to engage people, let them go, use the
- 29:44platforms that you've already sanctioned to try out these
- 29:47agents and come back and saying what?
- 29:50Where's it delivering value? Where's the data need to be
- 29:52improved on and go from there when it comes to building your
- 29:56own agents, right? And I know a number of companies
- 29:59that are starting to explore this.
- 30:01It really comes down to a build buy discussion and thinking
- 30:04about where are you having proprietary data, proprietary
- 30:09value to customer that you're going to start investing in,
- 30:12building out an AI agent with such good data, that's going to
- 30:15really be a game changer for you.
- 30:17And so that's what I'm looking for first is where is there
- 30:20something that's going to be a game changer in our industry or
- 30:24in our company or because I have data that's worth investing in
- 30:28building up that capability? I'm going to counter Isaac a
- 30:31little bit on this one because I do agree with some of the
- 30:34points, but one one thing is it's people, people, people is
- 30:39one of the biggest challenges that organizations have.
- 30:42And the lighthouse companies that started to roll their own
- 30:46with AI learned very quickly. They just do not have the
- 30:50capability to truly do that and maintain it because let's let's
- 30:54keep this in mind. It's not a one and done project.
- 30:58And So what you're seeing now is companies starting to adopt AI
- 31:03technology built into those existing products as as Isaac
- 31:07mentioned. The one thing that I will
- 31:09counter with is I would highly recommend at this point that
- 31:14organizations, CI, OS and organizations think about buy
- 31:19over build, especially with this technology and where it's, it is
- 31:23today. And the reason for that is there
- 31:25are so many variables and parameters and very, very, very
- 31:30few organizations, IT organizations or line of
- 31:33business organizations are truly capable of getting their arms
- 31:37safely around the technology. You potentially run into more
- 31:41risk than reward when you start to go down the build path unless
- 31:46you can truly get your arms around it, which few can.
- 31:49So I would encourage buy over build at this stage.
- 31:53Of course, that's going to that's going to vary depending
- 31:56on the particular use case and scenario and organization we're
- 31:59talking about. I agree today is a buy over
- 32:02build, but what CI OS have to keep in mind is the build is
- 32:06going to get easier and cheaper. You know, we saw that with
- 32:09mobile technologies if you built too early, you were building a
- 32:12lot of proprietary code and getting into technical debt
- 32:15around that and getting into user experiences that needed to
- 32:18get rewritten. It's going to get easier to
- 32:21build your own agents. It is but the, but the other
- 32:24piece to this that that hasn't come up in our conversation is
- 32:28the adoption piece. And we're already seeing a lot
- 32:31of dragging of the feed and just not really good adoption numbers
- 32:37of just simple things like Co pilots within your existing
- 32:41productivity applications. So this kind of gets back to the
- 32:44people and the the cultural pieces, but adoption of these
- 32:47tools is going to be really key. Ashish Parolekar comes back and
- 32:52he says are organizations adjusting their Okie Rs OK, OK,
- 32:58R sorry and metrics to assess the impact of AI on business
- 33:05outcomes or how are they planning to measure the ROI?
- 33:11This comes directly back to the question of alignment between AI
- 33:18initiatives and business strategy that you both alluded
- 33:22to. Tim, you want to jump on this
- 33:24one first. It depends on which, OK R, which
- 33:28particular strategy you're talking about, because in some
- 33:30cases, it shouldn't impact them whatsoever.
- 33:33Whether you use a blue technology, a yellow technology,
- 33:36red, green, it doesn't matter. What matters is the business
- 33:40outcome. And so there are different ways
- 33:43that you're going to accelerate or hinder your business, but you
- 33:47need to stay focused on what that outcome is, whether it's AI
- 33:50or not. When you get into some of the
- 33:53operational pieces and you start to want to measure things like
- 33:58before and after of bringing new applications or technology into
- 34:02the mix, I could see those changing.
- 34:04But you need to distinguish between those business strategy
- 34:08pieces and the ones that are kind of more in the weeds.
- 34:12And that's a really big differentiator to answer that
- 34:15question. Isaac, here's another question.
- 34:17Let me direct this one to you. This is from Chris Peterson on
- 34:21Twitter. And he says regarding POC's, to
- 34:27what extent are legal and audit functions actually ready to be
- 34:32involved in the development and innovation and not just be
- 34:37reactive after the POC? So, in other words, to what
- 34:41extent are legal and audit functions involved and should be
- 34:45involved? They're understaffed to be able
- 34:48to keep up with the space of change.
- 34:50The technologies aren't transparent enough for them.
- 34:55That might change when you look at what's happening with agents
- 34:59being able to take natural language input and be able to
- 35:02share that natural language thinking.
- 35:05A natural language output where when it has a conversation with
- 35:09another agent, we're now starting to be able to create a
- 35:13an audit trail in English that an auditor or somebody in legal
- 35:18can start following. So it's still early in this
- 35:20path, but because we're taking our services, moving off APIs
- 35:24and moving into natural language interfaces and they're sharing
- 35:27their thought streams and where they're passing other questions
- 35:31to other agents, we're going to start being able to have a lot
- 35:33more audit controls and legal controls around it.
- 35:36They have to be right up there at the front of the line.
- 35:39So at the beginning of the conversation, audit and legal
- 35:42has to be part of the conversation as part of your
- 35:45data strategy. But the other thing is there's
- 35:47been a lot of concern around the black box nature of these LLMS,
- 35:52and that's starting to change. We just saw Open AI come out and
- 35:56start to share some of the reasoning.
- 35:58And I think Deepseek is driving some of this.
- 36:02But now you're starting to see companies like Open AI that are
- 36:05saying, OK, we're going to actually expose some of the
- 36:08reasoning that went into the answer that we gave you from
- 36:11your prompt. And I would expect to see more
- 36:13of that. This is from LinkedIn from Jason
- 36:16Gutierrez, who says quick wins take smaller bytes, bytes, BYTES
- 36:22as well out of the problem you're trying to solve.
- 36:25And here's the question, what KPI are you trying to influence?
- 36:31Is your dev team skilled enough to deliver an AI app quickly, or
- 36:37are they still upskilling? Which raises the very important
- 36:42question of talent. Do we you guys spoke about build
- 36:48versus buy technology earlier. What about talent and developing
- 36:53talent in house versus going out on the market and recruiting
- 36:58people who have those skills? How do you think about that
- 37:01balance that? Skill set has changed from
- 37:05knowing as much as you can and how to do something into knowing
- 37:11what to do and whether what's being built is being built
- 37:16securely and robustly and, and, and has high performances.
- 37:20It's a shift in mindset into knowing how to ask the right
- 37:25question than knowing how to roll up your sleeves and getting
- 37:28something done. And that's, that's perplexing,
- 37:31particularly for us engineers and those working in IT to be
- 37:34able to think about this. But that's the nature of what AI
- 37:39is allowing us to do. It's not just around, can am I
- 37:42more productive? It's am I able to do things that
- 37:47I wasn't able to do because AI is providing the assistance
- 37:50around it. Last summer, I did coding for
- 37:53the first time, but I didn't write a line of code.
- 37:55I had AI write the line of the code.
- 37:56And I just took the person asking the questions and saying
- 37:59I need help to be able to do this function.
- 38:03How can I get some code to be able to do this?
- 38:04And after 5 or 6 prompts being able to do this.
- 38:07So I think the same thing is happening inside our IT
- 38:11organization. I think there's a real question
- 38:14for CIOs in particular. I saw a data point from McKenzie
- 38:18saying that IT is more than twice advanced using Gen.
- 38:22AI than other departments. But the question is, you know,
- 38:25when we use the word productivity, when we use the
- 38:27word capability and we start asking the question around ROI,
- 38:32we're not showing where that's delivering value.
- 38:35We're going to be asked to give cost up.
- 38:37Jason Gutierrez comes back to both of you guys, maybe Tim you
- 38:41what you can grab this and he says sure, but doesn't that
- 38:46necessarily mean more OpEx? There isn't A1 size fit all for
- 38:52every organization. I think one of the things that
- 38:54that I back away from is trying to get too granular on that
- 39:00answer. I think what's important for the
- 39:03CIO, one of the things I'm looking at is how do I start to
- 39:07measure my organization's value and impact to my business.
- 39:12And when I say my business, I don't mean a particular
- 39:15department of the the company. I mean to the business and our
- 39:19company's customers. And so I'm looking at how I tie
- 39:24what we do as an organization with NIT, with our business
- 39:29partners in HR, with our business partners in finance and
- 39:33operations and engineering. And I'm putting together OK Rs
- 39:36and metrics around our impact and performance against those
- 39:42objectives that we all share. That's where you have to start.
- 39:47Now you can delve further into developer productivity and call
- 39:52center productivity and and that's great.
- 39:55But when you start to get further into the weeds, that's
- 39:59where a lot more variables come into play.
- 40:01And you have there, there is no one-size-fits-all answer at that
- 40:04point. Isaac, we have a question from
- 40:08Derek Butts, and it relates to risk and security.
- 40:13And he says AI risk guidelines and standards are continuously
- 40:18being developed. Can you recommend any AI
- 40:23frameworks to securely roll out and mature the value of AI tools
- 40:28for your business operations without increasing the risk
- 40:33across your business culture? So I think really fundamentally
- 40:38we're talking about risk culture and rolling out at scale.
- 40:46When you're doing innovation, when you're driving change, you
- 40:50by definition are increasing risk, right?
- 40:53There's no ways around that. If you want to stay doing what
- 40:56you did today and continue to perfect it, that's, you know,
- 41:00you're going to be in that box. That can be disruptive.
- 41:03So as soon as I'm innovating, as soon as I'm looking at any new
- 41:07capability or bring change to my organization, I'm taking on
- 41:10risk. And the question is, you know,
- 41:13are you taking smart parallel measures around that risk?
- 41:17Are you identifying it? Are you bringing the right
- 41:20people? And who are going to ask the
- 41:22questions from a risk management perspective?
- 41:25Tim's brought up legal, audit, security.
- 41:28Am I bringing those people into the right conversations so that
- 41:32we're asking those questions earlier?
- 41:33What's going to be the impact if this data leaks?
- 41:36Should we be using this data for this perspective?
- 41:39How are we securing this data? Is our data masks so that if
- 41:42anybody comes into our organization and starts using
- 41:45that data in a new way, that data is masked for PIA
- 41:48information? These are the kinds of things
- 41:50that are building blocks when we're doing innovation that we
- 41:54have in place so that we can do those things securely.
- 41:57Tim, this is something that's orthogonally related, and this
- 42:02comes from R Salon Khan on Twitter, who says, how do you
- 42:08address shadow IT from frontline employees versus executives?
- 42:15Should we encourage shadow IT? And the reason I think it's
- 42:19orthogonally related is because one of the traditional arguments
- 42:23against shadow IT, of course, has been, oh, we're going to
- 42:28increase our risk footprint. So what about shadow IT?
- 42:32And especially in this age of AI where everybody's using ChatGPT.
- 42:38I actually support Shadow IT and there are a lot of folks that
- 42:42that think it is a dirty word or it can be problematic for the
- 42:46organization. But you have to go back and
- 42:48understand why is shadow IT coming into your organization to
- 42:52begin with. And typically it's because
- 42:54someone isn't getting what they need in the way they need it.
- 42:57That's the simple answer. The the longer answer may have a
- 43:01lot more components to it when it comes to AI.
- 43:04Yeah, there's sure there there's potential for invoking more risk
- 43:09into the equation. However, if you put the right
- 43:13guardrails in place, you actually can enable shadow IT in
- 43:17that creativity, especially in business units that understand
- 43:20the business better than your organization.
- 43:23And you can embrace that. And organizations have been
- 43:27highly successful in creating a culture around shadow IT.
- 43:31So it becomes more Co development, it becomes more
- 43:36collaborative in nature as opposed to us and them.
- 43:40But it starts with the culture and then just kind of fans out
- 43:43from there. Yes, of course you have to think
- 43:46about the data governance pieces.
- 43:49Yes, of course you have to think about cybersecurity and and
- 43:52those kinds of components. And you also want to think about
- 43:55simple things like how many different versions of generative
- 43:58AI do you want running in your organization and all the data
- 44:02that goes behind it. So all of those do come into
- 44:05play, but I would start with what is your current stance with
- 44:09regards to shadow IT and then let it fan out from there.
- 44:12Let's go to another question. This is from Mark P McDonald,
- 44:17WHO who's been a guest on this show.
- 44:20He's distinguished Vice President and a research fellow
- 44:24at Gartner. And he says who are the AI
- 44:30leaders, companies that we can follow that CICIO see as the
- 44:36leaders right now that we can study on who are doing AI well?
- 44:42Any thoughts on that, either of you?
- 44:44There are some companies that are doing some amazing things.
- 44:47Can I talk publicly about them? No.
- 44:50And the, the unfortunate piece to that is because we are in the
- 44:53very early days, they're using this as differentiating for
- 44:57their business strategy. And so they're finding ways to
- 45:00really kind of change the game, not just change the the chess
- 45:04pieces, but changing the whole game and using a very different
- 45:08approach with it. So we're not quite there yet.
- 45:12You do see some public examples of smaller examples of how AI is
- 45:16being used. You know, everything from
- 45:18cogeneration to summarization. You know, I think go back to the
- 45:22earlier conversation about legal, you know, being able to
- 45:25summarize the body of work around legal.
- 45:29I mean, that's, that's a massive opportunity.
- 45:33So there are some of those opportunities that come into
- 45:36that efficiencies space, but the ones that I'm familiar with, the
- 45:39really big demonstrable ones that that would be kind of those
- 45:43lighthouses, those beacons of opportunity to follow in their
- 45:47footsteps, those are pretty close to the vest still.
- 45:50Is it close to the vest because they have figured out the magic
- 45:57silver bullet? Or is it close to the vest
- 46:00because they're crying in the corners and don't want anybody
- 46:05to see it? You know, there's probably some
- 46:10of the crying in the corners too, to be honest.
- 46:14No, hey, listen, I I understand nobody wants to cry out in
- 46:17public, right? I mean we.
- 46:18Want to get back to? Lick our wounds.
- 46:20Yeah. I mean, let's face it, you got
- 46:22it. You got to have a few cuts to to
- 46:25have success, right. And the examples that I was
- 46:29thinking of, two examples that I'm thinking of, they're pretty
- 46:33significant examples. But again, I kind of wish I
- 46:37could even give you some context, but it I don't know how
- 46:40to do that without exposing who it is.
- 46:43Look, I'm looking for the small examples, OK?
- 46:45And the reason they're slow to announce them is their fear of
- 46:49people coming in, using the technology to find the gaps and
- 46:53what the AI isn't doing well. And then they're going to be in
- 46:55the headlines for the thing it shouldn't have been doing in the
- 46:58first place. So they're rolling these things
- 47:00out slowly. But I'm starting to see customer
- 47:03facing agents come out. I've got one in an e-mail this
- 47:06morning from a bank about being able to go through an agent
- 47:10around a car loan. We hate going for car loans.
- 47:13It's a horrible experience. So when you start seeing that
- 47:15being publicized, little things that people are doing, that
- 47:19agents are starting to help, well, you know that some of that
- 47:22is changing in the marketplace. Tim, Jason Genovese comes back
- 47:27and he says regarding Shadow IT, which you're in favor of.
- 47:31He says as Shadow IT steps in, at what point does enterprise
- 47:35architecture become a concern? And really quickly.
- 47:38Please work because we're running out of time.
- 47:40Enterprise architecture has to still be there.
- 47:43I will say a lot of enterprise architecture organizations get a
- 47:47little too big for their britches and so there has to be
- 47:50a bit of a check and balance there.
- 47:53But I think if you're if you're taking the right approach with
- 47:56EA in such a way that it becomes more modular and more of a
- 48:00framework as opposed to a heavily over architectured
- 48:04structure, then it's accommodating for shadow IT.
- 48:08Tim, here's one I'll direct to you.
- 48:10And this is from Ashish Bhatak and he on LinkedIn.
- 48:15He says buying AI may come with a cheaper cost, but what about
- 48:19the fear of data privacy and security as the data is governed
- 48:24by the AI service provider? What are the key parameters to
- 48:28be kept in mind when choosing an AI service provider?
- 48:33This kind of touches into what I was going to respond to an
- 48:36earlier question, which is why many companies are choosing to
- 48:40buy and use AI built into their existing enterprise applications
- 48:44as opposed to building. Because those companies
- 48:47understand the challenges around the data management, data
- 48:51governance, and they're able to put the right guardrails in
- 48:54place. Most enterprise organizations
- 48:57don't necessarily know how to navigate that.
- 48:59This is all new for them, but those enterprise application
- 49:03vendors actually have the, the girth or or scale to be able to
- 49:07do it for many different companies.
- 49:10And so a good way to think about this is I can kind of build off
- 49:14the the backs of those organizations as opposed to
- 49:18having to figure out how to do the basics myself.
- 49:21And unfortunately, there's a lot of assumptions that IT
- 49:25organizations make. We saw this in data centers
- 49:27thinking, oh, my data center is super secure.
- 49:30Guess what? Of all the data centers I have
- 49:32done assessments on, they are not as secure as you think they
- 49:36are. And so the problem is you have
- 49:38to take those assumptions out. And unless you're willing to do
- 49:41that culturally, it's really hard to ensure that things are
- 49:45buttoned up to the same degree. Isaac, Jason Genovese just wants
- 49:50to be clear that he disagrees. Well, I'll say this to to to
- 49:56both of you that he disagrees with Shadow IT.
- 50:00He thinks there's too much risk with data exfiltration.
- 50:03If you're going to allow it, you must have appropriate security
- 50:06controls in place. And he does agree, however, that
- 50:10responsible sandboxing of apps for testing or a PAPOC is
- 50:15necessary. You just can't forgo security
- 50:18controls and it's a shame we're not in a bar 'cause we could all
- 50:21step out and fight. I don't think we're fighting.
- 50:24I don't mean. Well, OK, I'm trying to
- 50:26instigate a fight then it's not well and it's not working.
- 50:29I don't, I don't agree that shadow IT is a good thing.
- 50:32But what I think CIOs have to understand is that shadow IT and
- 50:36now shadow I, I is happening, OK, that they can't put up the
- 50:40walls and prevent it from happening.
- 50:43And then you can go back to Tim's comments and saying, OK,
- 50:45what can we learn from what people are trying to do that
- 50:48we're not servicing well or they're not learning enough in
- 50:52terms of what are the risks or what are the technologies that
- 50:55we've already put out in place that that they can go out and
- 50:58use. So I think the real question is,
- 50:59well, how is CI OS monitoring for this and responding to it?
- 51:03All right. So you're on Jason's side.
- 51:04I'm on Tim's side because personally I think that if an
- 51:07organization has shadow IT, it means that the the the people
- 51:12out in the trenches are not getting what they need and so
- 51:15they're bypassing and they're just doing it themselves.
- 51:17But that's why I say I wish we were together and we could go
- 51:19out and fight about it. Just to be clear on that, I'm
- 51:22not saying shadow IT free for all is OK.
- 51:26So just just so we're clear, I'm talking about a managed
- 51:29approach, an integrated managed approach to shadow IT.
- 51:32I'm not talking about the free for all that many people think
- 51:34it is. So Tim makes clear that he's not
- 51:40talking about technology run amok and he won't allow it.
- 51:45As a CIO, you have a responsibility to the
- 51:48organization, right? And so on one hand, you need to
- 51:52make sure and, and make hard decisions around what you do
- 51:56and, and how you support the organization and how you engage
- 51:59the organization. And so that's why I think shadow
- 52:01IT can be really powerful in a collaborative sense.
- 52:05But you're right, I mean, if it's run amok, just going off
- 52:10and doing whatever they want to do and however they want to do
- 52:12it, sure, that's risky. That's not a good thing, but
- 52:16again, that requires some some mature thinking from a
- 52:20leadership standpoint to be able to get to that point.
- 52:23Martin Davis, who says AIAI as a service has the potential to
- 52:29turn all business apps into simply databases with an AI
- 52:34layer above to handle all business logic in one place, no
- 52:38longer siloed to functions, purposes, etcetera.
- 52:43How should we prepare and plan for this future?
- 52:47And I think it's a great place to and, and so Isaac, very
- 52:52quickly, how should we prepare for the AI future that lies
- 52:57before us all? Very quickly, please.
- 52:59It's a great vision. I don't think we can get there
- 53:03that easily. We've been having a vision of a
- 53:05utopian connected system for a very long time.
- 53:10I think to execute on that vision really comes down to
- 53:13developing your people. The the technology is changing
- 53:17so rapidly. When I talk about
- 53:19transformation, transformation evolves every 18 to 24 months.
- 53:24This is just the next wave of transformation with its own
- 53:27languages, with its own risks. And I think it really comes down
- 53:31to having the leaders in place that are looking for the
- 53:34opportunities, where to spend time on where to do the
- 53:38experiments and how they're going to evolve.
- 53:41Not just the cost equations and the efficiencies that we're
- 53:44working on, but how are we going to really change our business
- 53:47model and evolve because AI is a brand new capability for us to
- 53:52consider. Tim, you're going to get the
- 53:53last word. I actually agree.
- 53:55I think one of the things that that you need to think about is
- 53:59how can you accelerate the rate in which you're transforming.
- 54:02Transformation is an ongoing process.
- 54:05It's not a, it doesn't have a start and stop to it, but you
- 54:08need to look at how you can accelerate not just the
- 54:10technology you're using and the innovation you're using, but
- 54:13also how your organization both within IT and and outside is
- 54:17evolving too. So that's going to require
- 54:20change in terms of the people, the personas, the skills,
- 54:23rescaling, upscaling is going to be heavily engaged in this
- 54:28process as well as the relationships you have both
- 54:31inside and outside the organization.
- 54:33That's a that's a huge remit for CIOs and it's a different remit
- 54:37than they necessarily have had at this scale, at this pace in
- 54:41the past. All right.
- 54:42We have covered a lot of territory today.
- 54:47In this hour, I want to thank Tim Crawford and Isaac Sukullak.
- 54:52Thank you both so much for coming back and spending your
- 54:55time with CXO Talk. Really, really appreciate you
- 54:58both. Thanks for having me.
- 54:59Thanks for having me Mike. Great show.
- 55:01And thank you to everybody who watch.
- 55:03You guys are incredible audience, the questions and the
- 55:06insights that you have. But before you go, join our
- 55:11community. Go to cxotalk.com, sign up for
- 55:16our newsletter so we can notify you of other live shows.
- 55:20You see the discussions we have. So join us now.
- 55:25In two weeks, we are speaking with AT&T's President of
- 55:31Consumer. She owns the lion's share of
- 55:34AT&T's revenue and business, so join us.
- 55:39Then you can ask her questions and share your comments.
- 55:44And with that, I hope everybody has a great day and we will see
- 55:48you. See you again next time.
- 55:49Take care, everybody.