Latest / Technology Now / How to power AI: smarter cooling and the future of energy | Cullen Bash
Transcript
- CULLEN BASH Frontier was the first exascale system on the market, and when we push that to its very limit, it, consumes 30 MW of power. And at that time in, there were other, AI infrastructure, AI supercomputers that were deployed at that time. They were around the same order of magnitude. Fast-forward to today, three, four years later, 2025, the most capable AI supercomputer that was, commissioned consumes, 300 megawatts power. So we've gone an order of magnitude increase in power consumption over, just three, three short years and that slope, that curve, that increase, uh, is not leveling off. SAM JARRELL Wow. No, actually quite the opposite of leveling off. I still remember when Frontier debuted and what a moment that was for it to break the exascale barrier. But since then, we've had even faster supercomputers like El Capitan And while systems like these are the world’s largest computers, tackling complex scientific research, AI more broadly still presents significant energy demands. But where is all that extra power gonna come from? MICHAEL BIRD Yeah, that's a very, very good question, and I think it's something that, as a society we're battling with at the moment. because also, it's not just data centers that need this energy. It's everything, isn't it? It's hospitals, office blocks, our homes, our cars. And if demand outstrips supply, we could face some very serious issues. So, in today's episode, we're gonna be looking at how efficiency might just be the answer. to the energy crisis in tech, I’m Michael Bird SAM JARRELL I'm Sam Jarrell And welcome to Technology Now from HPE. MICHAEL BIRD Sam, I don't need to tell you that data centers and artificial intelligence need energy, a lot of energy, and as AI continue to scale, the source of this energy is gonna become a very, very serious issue SAM JARRELL Yeah, and I think in some ways it already is becoming a bit of an issue because if there's a limited supply of energy and if tech is demanding more we have to find somewhere to get that supply, be that a new power source or a way to use what we already have more efficiently MICHAEL BIRD Yeah, exactly. And that means we need, more innovation and people putting more time and effort into finding ways to solve the energy problems, not just efficiently or even sustainably, but also economically SAM JARRELL Of course. And because no organization is going to willingly adopt something which is, like financially damaging. MICHAEL BIRD Yes. Yes, that's very true. Now, this episode is the fourth installment in our miniseries with HPE Labs celebrating 60 years of innovation, and we thought the best person to talk about this would be Cullen Bash, Deputy Director of HPE Labs, as he has been working in this space for decades. And when I met with him, the first thing he told me was what he's seen change in that time. CULLEN BASH we're celebrating 60 years of innovation at HPE Labs, which is pretty amazing. and I've, been with the organization for 30 years, so 30 of those 60 years. And, the question's interesting because in some ways nothing has changed, and and in some ways almost everything has changed, right? we still take energy in its purest form, stored form, whatever that may be, the sunlight, wind, coal, natural gas, and we convert that to electricity. We did that 60 years ago. The means may have been a little bit different, but same process. and then we deliver that electricity to where it needs to be ultimately used. We deliver it as AC current, right? we then drop it down to DC current, and we use that to try to do useful work. That hasn't changed. What's really interesting though, that's changed in the last, at least 30 years that I've been around this is the percentage of electricity that's being consumed by IT infrastructure relative to everything else that consumes electricity is growing in proportion significantly. And, Lawrence Berkeley National Lab, had a report that came out two years ago. and it projected based on where things are going in AI, consumption, but it was a more generic IT view all the way across, the sector. they estimate that 10 to 25% of energy consumed by 2028 is gonna be in IT infrastructure and data centers. That means that the interaction between the demand for energy and the supply has changed significantly, dramatically in the past 30 years. And that creates challenges and it creates opportunities MICHAEL BIRD Which means it's more important for us to consider how technology is using energy. CULLEN BASH we're now a holistic view of energy. So we think about it from its generation, like I said before, through distribution, through its use in data centers, through its capture, and technology can be used across all of those areas, right? In terms of how we generate it,generating it now through other means than we were in the past. We have a lot more options for how we generate that energy. and, not all of them are useful or appropriate for all situations. So how do we match IT and energy and consumption? How do we match demand and supply? And so bringing AI into this story as well, AI is using probably more energy than traditional data centers were using. So what's the impact that AI has had on energy usage? Well, in 2022, HPE, along with our partner, AMD, and our customer Oak Ridge National Lab, released . Frontier was the first exascale system on the market, and when we push that to its very limit, it, consumes 30 MW of power. And at that time in 2022, there were other, AI infrastructure, AI supercomputers that were deployed at that time. They were around the same order of magnitude, 30 megawatts of power. and fast-forward to today, three, four years later, 2025, the most capable AI supercomputer that was, commissioned consumes, 300 megawatts power. So we've gone an order of magnitude increase in power consumption over, just three, three short years and that slope, that curve, that increase, uh, is not leveling off. There's every reason to believe that future systems are gonna continue on that same path, which ultimately not sustainable. And then th- those are the systems that we use to, to run models, But, you know, what are we gonna run on them? Well, we use those systems for model training, and, the highest-end models today are trained off of data that essentially is all the knowledge of human history, of parameters in those models that need to be worked out. And so the, the highest-end supercomputers can run for months in order to train these models once we have them trained, we use them inference and, and that's gonna continue that power consumption curve even beyond model training. Are we seeing, similar, energy increases from, organizations that aren't training AI models just, businesses? is IT consuming more energy because there's, there is more of it? Well, there's training, right? And we see these, these these big models that are being trained, trillions of parameters, using, as much or all of the data that we've generated in humanity throughout our entire existence. that's being used to train these models. That takes an enormous amount of power, hundreds of megawatts for months at a time from these big systems. But when they're trained, we have to use them in in some way, right? And using those models, think about a chatbot. you might chat with one of the, the largest models out there, but it's fairly well-contained. Your chat may not take forever. There's a moderate amount of energy use. But with agentic AI and with the proliferation of agentic AI, we now have agents that are able to access those models, and we have millions of agents, potentially billions of agents accessing those models. And if they're all accessing these large language models that consume a lot of energy when it comes to inference, not training, but inference, then that adds up rapidly. So overall, whether it be enterprises or hyperscalers or what have you, energy consumption is increasing, and it's gonna continue to increase as we rely more and more on these large language models. MICHAEL BIRD and I think what's interesting is, how actually energy usage within organizations, used to be separate from the IT budget. But increasingly that's becoming- less and less the case. energy is becoming something that the IT teams are having to think about and consider CULLEN BASH So when we first started thinking about data centers and energy efficiency, back in about, 2000, so 26 years ago. And we would talk to, CFOs, we would talk to CTOs and CIOs, and the CIOs certainly were only interested in the IT portion of the problem because they didn't pay the energy bill, Who paid the energy bill? Well, the CFOs paid the energy bill, or the COOs paid the energy bill, right? And so we were split-brained. who do we talk with? And so trying to, manage energy at the data center level when various different entities were involved in that was extraordinarily difficult and energy back then, we were talking about data centers a few megawatts. When data centers are now, tens of megawatts to 100 megawatts, now it's materially important for the company, and the CEO has to thinking about it So that's where things starting come together and where opportunities come together because once you start thinking about things holistically, we can start tackling the problem, not by piece parts, but holistically. holistically. MICHAEL BIRD you talked about this, holistic view, when it comes to energy in the past. can you explain what exactly you mean by that? CULLEN BASH By holistic view, we mean, thinking about the whole energy flow from the beginning to end, so from its generation through distribution through use it's not just about energy consumption and the IT infrastructure, it's about how we efficiently capture the heat that's, resulting from that, that electricity consumption. The way in which we capture that heat is important because we ultimately wanna be able to reuse that heat for other purposes, right? That's getting the most out of every joule, of energy that we can throughout that entire process. And so to get, large value out of energy reuse, we need to keep the temperature of that waste heat, if you wanna call it waste heat as high as possible, right? We don't want low temperature waste heat because it's not that useful. So that means that the waste heat recovery system and the cooling system and the IT system become intimately connected because they're... the IT system provides the heat, the cooling system provides the way to efficiently capture it, and the waste heat recovery system communicates with the consumers of that heat to make sure that it's at the right temperature for them to be able to do something useful with. MICHAEL BIRD Can we talk cooling? Because, with the rise of AI, we're hearing a lot about, direct liquid cooling CULLEN BASH Direct liquid cooling has been around for quite some time, even before the rise of AI, and, we've had that in ... I mentioned Frontier earlier, first exascale system. That's 100% liquid cooled. There's no air moving through that, machine at all, and, why is that interesting? Well, it's interesting because capturing that heat in liquid form allows us to transport that energy in a way that's much more efficient than if it was an air-cooled device. It also, of course, allows us to pack more GPUs and more CPUs into a smaller volume than we can with air cooling, which is gonna be important for, AI inference and AI training. our next generation of systems think about them as a group of blades in a rack they all, receive the same amount of, liquid volume flow rate, but we're now investigating would it take for us to be able to vary that flow rate through each individual blade in the system according to the needs of that blade in the system? If we could do that, that would allow us to save energy 'cause we're only ... moving cooling resources where they're needed, not uniformly distributing them throughout the entire machine. Controlling that becomes, more difficult, and so what we've been looking at is can we use AI, can we use reinforcement learning to help us manage those actuators? So AI is producing more of the energy that we have to deal with, but we're also looking at it as a potential way to solve these challenges that have come up, and that's where the duality of AI comes in that I think is really, really interesting. MICHAEL BIRD Which leads really nicely onto the fact that, I think you said before, HPE Labs starts researching ideas long before the market is ready for them can you give some examples? CULLEN BASH I started, about 28 years ago. I came from the business units, and I started actually in the product design organization. Spent about three years there, which was fantastic because we got to learn, really the basics of product design, how market forces impact that, how to work with, product managers, and who ultimately decide what features are in, what are out and when I joined, the business unit, we were developing, our PA RISC 8000 processor. It was projected to come back at, 45 watts of power but It came back at 90 watts. So that created a sea change in how we think about cooling design way back 30 years ago, and I joined HP Labs, about three years after that to explore research in cooling technologies and my group at the time were looking at every possible way we could think of to cool computer systems. This was back in 1998, 1999. We looked at liquid cooling. We looked at two-phase cooling. We looked at evaporative spray cooling. We turned computers into refrigerators and looked at using vapor compression refrigeration to cool chips. a lot of that technology wasn't ready because the power dissipation of CPUs in particular, started to flatten out because of Dennard scaling But as power consumption started to increase because Dennard scaling, stopped, all of those things started to come back in, in play again once we finished exploring every possible way we could think of to cool chips, we started to think about data centers. but, if we think about the data center, it's part of a system. It's IT infrastructure and it's facilities infrastructure, And we started this project called Integrated Management, where we intimately connected then the IT infrastructure with the cooling infrastructure, which had never been done before, and we found that we could get massive savings by doing that. This was back in 2010, right? And then what we did is we thought, well, we really need to go out to the supply side and generation, and so we put a photovoltaic array on top of our data center, connected it to our IT infrastructure, and we created the first data center in the world that consumed no net energy from the public utility grid. It was grid connected, but we tried to manipulate demand and workload so that we were taking advantage of the time when we had the sun out. We didn't have batteries back then. That was 2012. 2012. and now we're in 2026, and much of what we did back then is, is coming to fruition today. MICHAEL BIRD so looking forward, what are you expecting to see change over the next five or 10 years as it relates to, how data centers and our IT infrastructure uses energy? CULLEN BASH the path we're on right now is not sustainable. If we continue to, to go, uh, and follow the curve the way it's moving, as I said, by 2028, according to Lawrence Berkeley, electricity in particular from IT, will be 10 to 25%. That's not sustainable, so we have to level that curve. And in the next 5 or 10 years, techniques for leveling that curve, people are looking at them, their software, their hardware. within HPE and at Labs, we're focusing on that whole stack, but on the hardware side, we're developing, accelerators that are based on analog technology, not digital technology like GPUs and CPUs and FPGAs, but analog technology that manipulates voltages and currents using, resistors and the like we're using that to do computation now. We're using light to do computation. Light is in the analog domain as well. And what we're finding is that for specific workloads like AI inference we can solve them using, orders of magnitude less energy. So we need ideas like that to come to market so that we can start to flatten out that curve. And I think the other thing I would suggest is on the supply side, we need more efficient ways to generate energy that is, less environmentally intense. And so as an example of that, I know this is, may not seem entirely environmentally friendly, but, small modular nuclear fission reactors. they don't emit carbon, but they do emit waste that we have to deal with. but that's gaining, traction across the globe because they're smaller devices that can be deployed with IT infrastructure, and then there's always work in fusion energy and then we're working on within Labs applying AI technology to accelerate the, development of fusion technology with many of our partners. And so that's what we're looking forward to in 5 to 10 years as well. MICHAEL BIRD Cullen, thank you so much for joining us on Technology Now Really appreciate your time. CULLEN BASH All right. Thank you SAM JARRELL Wow. I always feel, almost overwhelmed but, very grateful whenever we have on someone from HPE Labs because they just make me very excited about the future whenever you hear them speak so passionately. And this topic in particular has been so sensitive in the news and in communities around the world, that it is encouraging to hear all of the work that is being done, even just within our own organization around addressing the issue of the energy consumption. to me, it's exciting that we're even doing research into things like nuclear fusion as an option, and that there's other methods within existing methods, like direct liquid cooling, to be more sustainable, rather than just always creating something that new. We're innovating within what we already have too. it's encouraging. MICHAEL BIRD Yeah. the sort of line that I wrote down that I thought was quite a, a good takeaway was, something along the lines of, "We need to be generating more energy, but less environmentally intense." So to your point, nuclear fusion, small nuclear reactors, taking advantage of, solar, wind, whatever, thi-things that can be generated locally, geothermal. because, the demand for energy isn't going to decrease, it's going to increase. So we need to figure out a way of generating that energy, because how we generate some energy at the moment is from a finite source. and once that runs out, there isn't any left. And so actually finding a way that is more sustainable, that is renewable, I suppose, is a sensible long-term solution from a sustainability perspective, but also from a financial sp-perspective. one of the things that I thought was really interesting was, some of the examples he gave as to some of the innovations that they've been thinking about. And, um, the one that I think is really, really nerdy, but if you think about it makes so much sense, was basically using AI to, dial in the taps, for different flow rates for different components. Because, you know, this concept of, like, not everything needs to be cooled at the same rate all the time. So if you can use, AI, some sort of machine lear-learning algorithm, to basically just dial in the different flow rates for different components. Um, it's little things like that, that you think, "Oh, yeah. Actually, that, that is really clever, and that does make so much sense when you think about it." SAM JARRELL That's true. it's a little bit ironic. Like AI is driving a higher energy demand, but it's then also giving researchers new tools to optimize cooling, improve efficiency, and potentially accelerate some breakthroughs in energy technologies themselves. So it's both the challenge and part of the solution? MICHAEL BIRD Yeah, Cullen said at the top of the show, by 2028, between 10 and 25% of all energy will be consumed by IT infrastructure and data centers. So if you can make something 1% more efficient that is a huge amount of energy you're saving SAM JARRELL That is very true. and as we talked about at the top of the show, organizations don't want to take on something that is gonna be financially damaging to them, but the energy conversation is a very large financial conversation these days. So even saving 1%, given how fast this curve is accelerating on usage and where those costs are now living, will be a huge game changer, I think, for a lot of businesses as well as, governments and organizations around the world MICHAEL BIRD energy is a big chunk of, money that businesses and organizations are having to spend. So actually now, not just the CFO and COO, the CEO cares about energy usage in an organization. So it's made its way up the rungs. And so It'll have an actual impact on the company's bottom line if you can be, if you can use this energy more efficiently SAM JARRELL I agree. this is a conversation that has become so wide, and beyond, CIO, CEO, CFO, C-suite in general, the communities around the world are starting to care more and more about this. And, I think increasing we're seeing the intersection of, the AI discussion, and what it means for the world become more and more prevalent. And so this is one of those moments too where, maybe the, ROI, while it can be financial and it can also be related to, the energy itself, there's also an ROI for the world in that we figure out some of these big conversations around sustainability and around energy generally, right? MICHAEL BIRD Yeah, yeah, totally. Now, this has been quite a-an information and number-dense episode, so I wanted to ask Cullen one s-quite simple question. What is the most important thing that people and organizations should know when it comes to energy and AI? CULLEN BASH for energy and AI, they need to think about things, holistically, it's not just a matter of focusing on their specific infrastructure. it's about thinking more holistically about where they're getting their energy from and, how they're, dissipating that energy in a way that's most efficient. And if they have the means, and the opportunity, how are they able to reuse that energy into something, that can continue that flow and continue the usefulness as it goes through its IT lifespan? efficiency is important, but the other thing to focus on is how can we use these fantastic tools that are coming, these AI tools that are evolving rapidly over time to help us solve some of these efficiency, challenges as well? SAM JARRELL Okay that brings us to the end of Technology Now for this week. Thank you to our guest, Cullen Bash And of course, to our listeners. Thank you so much for joining us. MICHAEL BIRD If you’ve enjoyed this episode, please do let us know – rate and review us wherever you listen to episodes and if you want to get in contact with us, send us an email to technology now AT hpe.com and don’t forget to subscribe so you can listen first every week. Technology Now is hosted by Sam Jarrell and myself, Michael Bird This episode was produced by Harry Lampert and Eva Higginbotham with production support from Alysha Kempson-Taylor, Nik Damarell Beckie Bird, Alissa Mitry, and Jenessa Ayache. Our theme music was composed by Greg Hooper. SAM JARRELL Our social editorial team is Rebecca Wissinger, Judy-Anne Goldman and Jacqueline Green and our social media designers are Alejandra Garcia, and Ambar Maldonado. MICHAEL BIRD Technology Now is a Fresh Air Production for Hewlett Packard Enterprise. (and) we’ll see you next week. Cheers! SAM JARRELL Bye y’all