Latest / Technology Now / Can AI make itself more efficient?

Transcript

Can AI make itself more efficient? · Technology Now

• AUBREY LOVELL Michael, how are you today. MICHAEL BIRD I am good. The sun is shining. AUBREY LOVELL Well I have a question for you, just kind of leading into our topic of the day. do you plan things backwards or forwards? Do you start from the result and work out how to get there? Or do you start with what you've got and try to work out how to get to your desired outcome? MICHAEL BIRD I am very much… I start with the result and work my way back. I am, that's the hill I'll die on is that is the best way to do things. AUBREY LOVELL Yep. You know, I think I'm the same. I definitely find myself reverse engineering things. It just, for me, it makes sense. So we definitely have that in common. And this week, we're actually going to be exploring why starting from the end and working backwards can sometimes be the best way to approach a problem. I’m Aubrey Lovell MICHAEL BIRD I’m Michael Bird AUBREY LOVELL And welcome to Technology Now from HPE. AUBREY LOVELL So Michael, efficiency in AI has been a pretty hot topic for a while now, right? And everything from energy efficiency to the speed at which we get a result when we ask a question. Bu t here's the problem, right? Things are often connected behind the scenes and changing one thing can lead to unexpected consequences elsewhere. MICHAEL BIRD But presumably, this is something that could be prevented if it’s thought of during the design stage? AUBREY LOVELL Well I mean, you’d hope so, but part of the issue here is learning how to try and predict unexpected consequences, right? MICHAEL BIRD Yeah, I also imagine that some things can't be like decoupled either, like they're intrinsically linked within a system. AUBREY LOVELL Absolutely, I mean if a model requires lots of data to train and run it's going to need lots of power too And of course the opposite is true, but the topic is as you might expect far more complicated than that so, to find out more, I know you'll be excited about this, I was joined by John Fry, Senior Director and Chief Technologist of Sustainable Transformation for HPE. But before we get to John's interview, Michael, I believe you have something to tell us about? MICHAEL BIRD Yes I do Aubrey, because we are going to take a look at one of the most famous examples of unexpected inefficiency in a system. Yes Aubrey, it's my turn this week. It is time for Technology Then. MICHAEL BIRD Right, I want to start by debunking a common claim which states the QWERTY keyboard, you know the keyboard you probably have in front of you, was designed to be deliberately inefficient to slow typists down. It is said that this was because it helped prevent jams on typewriters for people who were typing too quickly. Now, a re you a fast typist, Aubrey? AUBREY LOVELL I would say I'm probably average. I don't know, I've never really thought about it. What about you? MICHAEL BIRD You and I are of a similar vintage, I would say. I am not a touch-typer. I am embarrassingly still like a little two-finger henpecker, maybe… maybe some other fingers get involved occasions but generally I’m pretty analogue with my typing. Well, we discussed frequency analysis a few weeks ago when we talked about ciphers and we're going to be returning to it now because frequency analysis of letter pairings published by the University of Chicago, Illinois shows that the most common pairs of letters are the following T-H, H and E, A and N, I and N and E and R. And if you have a look at your keyboard in front of you, you can see that you can type all of these pairs incredibly quickly because they use different fingers. So if the keyboard was designed to slow you down, it clearly wasn't designed particularly well. In fact, the keyboard was actually designed for telegraph operators transcribing Morse code, specifically American Morse code. So for example the letter Z or Z was shown as four dots, the letter S by three and the letter E by one . This would make it very difficult to tell the letter Z apart from an S, followed by an E. By placing the letters next to each other on the keyboard, the operator could hover over them and once the context of the message showed which was correct, they could then press the right keys. the problem with this is that while the keyboard was great for Morse code, it was not great for general typing. In fact, a much more efficient keyboard was designed in 1932 by a man called August Dvorak, which put the most commonly used keys in the same place and was designed in a both left-handed and right-handed formation to favour the most dominant hand when typing. Aubrey... I have shared with you a picture of this which we'll link to in the show notes so you can check it out. What do you think of this keyboard? All the vowels are in the same place, don't you notice that? AUBREY LOVELL I feel like it, I feel like this would hurt. Like my hands hurt looking at it. It's so spaced out though, right? MICHAEL BIRD Yeah. I sort of, without even typing on it, I sort of feel my wrists getting really sore. MICHAEL BIRD Anyway, a study by the US Navy in the 40s showed that it was significantly better than the traditional QWERTY keyboard, but unfortunately by then the market had already locked into the QWERTY model . This market lock-in has left us with a keyboard that works but is by no means the best option out there. AUBREY LOVELL But it really is just fascinating, right, that the inefficiencies actually arose because the keyboard was designed for a system, right, that we no longer use. So even though we now view it as inefficient at the time, like we've been saying, it was the best model going. And that focus on efficiency really hasn't changed. However, the way people approach it very much has, right? So to find out more about that, I spoke with John Frey, Senior Director and Chief Technologist of Sustainable Transformation for HPE, and I started by simply asking him how efficiency is viewed in the world of technology JOHN FREY When we think about overall technology efficiency, we've concluded over the 25 years of doing this work that there's really five levers and we call them levers because they're interconnected. Equipment efficiency, energy efficiency, resource efficiency. So all the things beyond IT, the cooling and power conversion and such. And those were where we started for about 15 years, but then we recognize the applications play a huge role. So we... came up with software efficiency in our point of view on that. And finally data efficiency and although AI is very old, recently, of course, it's become a really hot topic. And as the topic started coming up, we asked ourselves, does anything about AI shift our five levers of efficiency or not? And what we came to the conclusion was, no, they're all still applicable and in fact, some of them, are even much more applicable, particularly as we start thinking about things like generative AI, where we have these large language models, the software efficiency piece takes on a disproportionately large piece. And then even when we start thinking about data efficiency, because so many of these AI solutions require a tremendous amount of data, how do we make sure we're using the right data and training appropriately and those sorts of things? The other thing we found from an efficiency perspective is often these new systems, when we think about a genetic or generative AI, they take higher performing hardware, much more performance, yet a lot more energy consumption, therefore a lot more heat output. So we have to balance those nuances, the complicated algorithms, and as I said before, just a tremendous amount of data. So all of those have efficiency implications. And because those levers are interconnected, solving for one doesn't get the job done. You really have to look across. And finally, the other thing I would say from a slightly different take on efficiency is it's being reported that only about 5 to 10 % of current AI pilots ever make it into production. So from an overall efficiency perspective, we have a whole lot of pilots going on consuming a tremendous amount of resource, but yet those aren't making an end to production. So from an efficiency perspective, we've got a lot of room to grow there as well. AUBREY LOVELL Absolutely and I know you mentioned the five levers of efficiency kind of you know we're talking about this broadly But you didn't mention generative AI and I kind of want to go back to that because it's something that a lot of people are hearing probably Most is like one of the hottest topics in AI What is particularly challenging with generative AI versus some of the other types of AI? JOHN FREY Generative AI, because the initial belief was the larger the model and the larger the data training set, the more accurate the model could be. These are very, very large models. They take weeks or months to train on hardware platforms that can be a hundred million to over a billion dollars. So they are very resource intensive. And by the way, then once you've trained the model, it's only as good as the data that it had up to that training point. So if new information comes out that you want to incorporate in the model, you have to do more work there. The other thing that is slightly challenging about that is almost anything we know about the model efficiency, the model size, the resources it takes, are forensic guesses looking backward. The other thing that we're running into is models were continuing to grow almost a logarithmic climb up in this quest for accuracy. And it was believed for a long time that that's the only way to get a more accurate model. And then we saw some models, some small language models start coming out that change that paradigm or challenge that paradigm. And all of a sudden we're offering a fair degree of accuracy with a much smaller model. Yet the model growth continues. In fact, Uptime Institute published a paper on AI density that said, and I'm just going to read the quote, “presently there's no evidence of a profitable business model for state-of-the-art AI products costing hundreds of millions or billions of dollars to develop. Or that scaling” dot dot dot “the models another ten times to a hundred times would improve the financial viabilities of these offerings” “Yet even without a solid business case, there is no sign of the surge in investments in ever larger AI models slowing.” That's a dynamic we've got to solve for. W e're using a tremendous amount of resources for larger models when in fact, their accuracy and their financial viability isn't proven at this point. So that poses an interesting challenge in generative AI particularly when we're starting to see small language models that can be every bit as effective. AUBREY LOVELL So does that mean people are going to have to choose between using large language models and these small language models? Or are they able to work together to answer a question? JOHN FREY So we're already starting to see even the model developers start to shift that mindset from bigger is better to perhaps variety is better and use the most appropriately sized model for the task at hand. In fact, one of the latest models that just was released in the last few weeks actually auto throttles the input to the most appropriate internal model that it has. If you ask a simple question, it's gonna use a much smaller model to answer that partially to get the asker a really quick answer. where some of the more complicated questions take longer to produce the output. So even from a consumer use perspective or a user satisfaction perspective, being able to use smaller models sometimes to answer those simpler questions makes the user happier because they ask a question and get an almost instantaneous answer. So all of that wraps together to say, How do we balance efficiency, user satisfaction, the financial viability of these models? And that's still being solved for as we go. AUBREY LOVELL Absolutely, and I think to your point right you're seeing obviously there's a long way to go and there's gaps to be filled But what we're kind of seeing is that shift as well as it being so accessible Just to on a normal consumer base and normal laptops. we were kind of talking about different types of AI. And one word we keep hearing over and over again is agentic AI, right? So as this becomes more common, what is the difference between agentic AI and tool AI? JOHN FREY Yeah, great questions all. Agenic AI is a term that you hear a lot of, but a lot of times people don't understand what it is. And very simply, it's an AI tool that can do multi-step problem solving or multi-step processes. If you think about the way most of us interact with, a large language model or a generative AI model today, we ask a question and get output. Then we say, refine your answer, or that's not what I was looking for, and we ask another question. So it's input, output, input output with a human involved every step of the way, where a agentic AI can actually take an input and then do multiple things. And in fact, sometimes even take an action that doesn't involve the human being involved in that. And it can also, because it can do that fine tuning, it can get more nuanced answers and can learn from experience as well. So agentic, part of the reason people are excited, is because it allows the intelligence to do more things. But let's remember behind all of this is a mathematical or a set of mathematical equations that are happening. There's not actually human reasoning behind the scenes. These are still tools that involve some human interaction. AUBREY LOVELL And what does that look like from a sustainability perspective, John? JOHN FREY So when we use agentic from an efficiency perspective, you want to put in things like guardrails, give the tool itself some parameters to stay within, for example think about the unintended consequences and take those out of the equation if you can. So a great example of this is, and I think you've talked about it on an earlier podcast, is Antonio Nearly, an AI solution that we built to interact with customers at customer events and things. And it's an avatar of our CEO, Antonio Neary. And part of when they first trained that system up, they trained it on every press release that HPE has ever released. Well, what does every press release have at the bottom of it? A legal statement. So when they were first working on Antonio Nearly and allowing it to actually do some agentic types of things, one of the guardrails they put in as they learned from experience was tell it to ignore that statement at the bottom of every press release because it gave that statement the same weight as what was in the press release, which was completely unintended, but that's the way press releases are written. AUBREY LOVELL That’s the sort of thing you just don’t think about unless you’ve made that mistake before so those guardrails are really important. And I guess, if you’re putting those in place to stop it wasting time considering material which isn’t needed, you’re also improving the efficiency of the system right? JOHN FREY So from an efficiency perspective, some would say, well, actually if the tool can take multiple steps on its own, you're generating more resource use by that inference process again and again this self-diagnosis and intervention capability of agentic AI is partially where we're seeing the return on investments really come to play because we can solve a much bigger problem before that ever happens. Or in a customer satisfaction perspective, if we can take some action and keep a problem from occurring from a customer perspective as well, there's huge financial benefit to that as well. So for us then, how we've learnt to look at AI efficiency, and particularly as agentic has become more prevalent, is think about the entire solution. Because often the customer buys an optimized piece of hardware, and they put it in a data center they hope is optimized, and then they buy an application that they hope is optimized. And so we're optimizing the piece parts, but that doesn't necessarily tie it all together. So we're teaching our developers internal to HPE and also suggesting to our customers, that they design AI solutions from a solution perspective and understand the interplay between the pieces. You can have a very inefficient solution that has optimized pieces if you're not using the pieces effectively. We’re helping them think about all those implications because As these solutions become more more common, we need the license to operate in the areas where customers want to run these solutions. So we've got to think of this optimization and efficiency from the beginning and then all the way through. AUBREY LOVELL Got it. It's almost like, you know, when you think about city planning, right? Or when they're building roads, you want to build a road and add lanes for the predictive population, not the population of today, right? So I love that there's so much thinking in the solution build up front of what that could look like to mitigate all of these things and also free up resources as well. JOHN FREY Absolutely. And you know, one of the other challenges is over 80% of the data that people want to train these systems on is unstructured. So how do we get that data that wasn't in a database or in a structured format and get that into solution so we get the benefit of that knowledge, but get past some of the challenges of dealing with unstructured data. So again, if you think about that from the beginning of the design, you can adapt to that, but that's… that's another challenge that comes into play in these systems. AUBREY LOVELL True. And I think there's, you know, another side of this too. I mean, it's exciting that agentic can kind of have its own personality and do these things without necessarily a human being involved all the time and make its own decisions. But you have to kind of wonder as well, h ow do we ensure that they're doing what they're supposed to be doing and not misbehaving? JOHN FREY Yeah, it's a great question. part of what we have to acknowledge out of the gate here is that we're on a huge learning curve. So one is anticipating the unintended consequence. And history can teach us a lot about unintended consequences as well. And where there are life safety implications or environmental implications, you put more guardrails on those things so that you prevent that. You get a human involved in an intervention. And often these eugenic systems can go to some point and then a human's got to get involved again. And it's not just the unintended consequences of the system itself, but it's in the use of the system as well. And so one of the things that HPE has been very focused on, and I think Kirk Bresnicker has been on this podcast before, talking about it's the responsible AI attributes that go hand in hand with efficient AI. We tell our customers you need both. You have to have sustainable AI, but it's also got to be responsible. And so don't decouple those from one another either. So focus on the efficiency of the solution and then on the efficiency of what you can do with the solution to drive new efficiencies as well. AUBREY LOVELL That makes a lot of sense. You want to make sure you have everything covered, all bases, all solutions kind of wrapped together, end to end, essentially. Okay, so I do have one more question. Can you use AI to make itself more efficient? Right? I think we have to ask the question, right? If they're thinking on their own and they're doing all these things, can they essentially improve themselves? JOHN FREY Absolutely, yes, and it's happening today. And a great example of this is one of the early adopters of digital twins that I know has been talked about on the podcast before is for data centers itself and data centers running AI workloads. And there's a variety of reasons for that. Some is there are some parameters you can set and allow the agentic tool to control cooling and other attributes within that itself and so it can even predict things. The other thing that we find is often for some of that mechanical equipment, if you bring it up to temperature slower, it's more efficient than if you rapidly have to bring it up to temperature. So we can even predict times looking at weather patterns and things when you're gonna need additional cooling and bring it into bear slowly. Or if you're talking about shifting workloads or adding workloads, what are the implications of that in the data center environment that's already there? And using a digital twin, you can run a variety of those scenarios to operate more efficiently. In fact, we have customers that are doing that today, and that's a big part of HPE's digital twin practice and our solution set. So that's a great example of using AI to make AI more efficient. MICHAEL BIRD Always love having John on the show. I always think he's got such wonderful explanations. I don't know about you, but one of my big reflections from that interview was, well, I've never heard of the phrase small language models before. Had you before that interview? AUBREY LOVELL Not really. I knew what he was talking about, but I just never heard it in that context. MICHAEL BIRD The concept of using the right size model for the task at hand, I think that's what John said. It sort of feels a bit reminiscent of the way that the processors and our computers have changed over the last like a couple of decades some laptops from some manufacturers, they are they have like individual parts of the processor for specific tasks, which relates to what John was saying. You have an AI model for a specific task at hand because it's a way more efficient way of doing it. Whereas processors from 20, 30 years ago, it was just about pure grunt. Like how much power could you shove through this thing? And that meant that laptops would burn through their batteries very quickly and would run very hot. Whereas nowadays, laptops don't burn quite so hot and they can sit on their batteries for quite a while. So I thought that was quite fascinating because I think it feels like we're maybe in that stage of AI AUBREY LOVELL For sure. And I think we touched on that a little bit, right, where we were talking about how it's so consumable now. Like there was a time when AI could only run a certain way on a certain type of, you know, massive computer and only accessed by so many people. And now it's like, everybody has access. It is super scalable. And these solutions are kind of bespoke to what you need, and when you need it MICHAEL BIRD Yeah, and I think that the point you made at the end about using AI to make predictive workload optimization, it makes sense. But there is also an element of like, well, yeah, of course that makes so much sense to use AI for things like that. Like the things that AI is good at, let's use it for that. AUBREY LOVELL For sure. And I just really love talking to John, right? I mean, he's such an interesting guy. He's done so much. And, you know, he's really been in the industry for a really long time. So I thought, you know, to close out the conversation, it would be really interesting to finish up with the major AI breakthrough he would love to see in his lifetime. JOHN FREY I think there's lots of opportunities to use AI as an adjunct to bridge existing gaps. I think of folks that have mobility challenges or even neurological challenges, and how do we use an AI tool to allow them to interact with their environment around them in a much more natural way? Huge progress has been made over time, but really opening those opportunities. How do we do things like explore space when the number one challenge is, is how do you get a human to live in a space environment? My early career, I worked for NASA during the days of the shuttle program. And so how do we unlock some things like that, that pose great opportunities perhaps for new materials to be developed or whatever, that today the challenge is we have to have the human involved directly. Perhaps AI can enable us to solve some of those challenges as well. I don't even think we've thought of all the uses that we could use AI for from a productive perspective. Again, I think the responsible AI has to be hand in hand with the efficient part there, but I'm excited to see where it goes. MICHAEL BIRD Okay that brings us to the end of Technology Now for this week. Thank you to our guest, John Frey, And of course, to our listeners. Thank you so much for joining us. AUBREY LOVELL If you've enjoyed this episode, please do let us know. You can rate and review us whenever you listen to episodes. And if you want to get in contact with us, send us an email to technology now at hpe.com, subject title line to the moon. And don't forget to subscribe so you can listen first every week MICHAEL BIRD Technology Now is hosted by Aubrey Lovell and myself, Michael Bird This episode was produced by Harry Lampert and Izzie Clarke with production support from Alysha Kempson-Taylor, Beckie Bird, Alissa Mitry and Renee Edwards. AUBREY LOVELL Our social editorial team is Rebecca Wissinger, Judy-Anne Goldman and Jacqueline Green and our social media designers are Alejandra Garcia, and Ambar Maldonado. MICHAEL BIRD Technology Now is a Fresh Air Production for Hewlett Packard Enterprise. (and) we’ll see you next week. Cheers!