Latest / Technology Now / Inside the UK's fastest supercomputer: Isambard AI
Transcript
- SAM JARRELL Good morning, Michael. You look like you're out on location again. MICHAEL BIRD Yeah, Sam. That's right. I've escaped my home studio and I'm standing next to one of the most powerful supercomputers in the world. How cool is that? SAM JARRELL Wow. Okay. Okay. You've caught my interest. Tell me more about this supercomputer. MICHAEL BIRD Well, Sam, this right here is Isambard AI, the UK’s fastest supercomputer. I’m Michael Bird SAM JARRELL I'm Sam Jarrell MICHAEL BIRD And welcome to Technology Now from HPE. MICHAEL BIRD Well, Sam, I've come to moderately cloudy Bristol in the southwest of the United Kingdom to visit the most powerful supercomputer in the country. And this is a video episode, so if you wanna see the supercomputer for yourself, make sure to check us out wherever you watch your podcasts as well, of course, as wherever you listen to them. SAM JARRELL and you absolutely should. 'cause it looks really cool. And this episode is in two parts, because there is far too much for us to cover in just one episode about Isambard AI. So today we are looking at the project itself, and next week we'll be looking into how you run something of this scale efficiently. Now, Michael, when we say it's the most powerful supercomputer in the UK, just how fast are we talking? MICHAEL BIRD Great question, Sam. Isambard AI is a Mod Pod – a modular data centre – and it’s run by the Bristol Centre for Supercomputing who have said that it is the 11th fastest supercomputer in the world and the 9th fastest for public supercomputing and, get this, the internal network speed is 200 Gigabits per second which is about 2,000 times faster than the average home broadband speed of 100 megabits per second and we have, as always, linked to these stats in the show notes.iii SAM JARRELL Oh my. Well, you certainly wouldn't have any lag while playing any games on an Isambard. MICHAEL BIRD I don't think so. SAM JARRELL So, Michael, who are you interviewing today? MICHAEL BIRD Well earlier on, I got a guided tour of Isambard AI from Professor Simon McIntosh-Smith, Director of the Bristol Center for Supercomputing who has worked on the project since its inception. MICHAEL BIRD So Simon, we are standing here at the entrance to Isambard ai. Uh, what exactly are we looking at here? SIMON MCINTOSH-SMITH we're now inside the Isad ai. Supercomputer compound. Yeah. You'll probably notice around you there's lots of physical security. So have lots of big fences, lots of barbed wire, loads of cameras. That's because what we're about to go in and see is one of the most powerful supercomputers anywhere in the world. MICHAEL BIRD Wow. Okay. And, um, what exactly is this project all about? SIMON MCINTOSH-SMITH Well, Isambard, AI is the first big part of the UK government's AI research resource program funded by the department for science, innovation and technology, DS I. And it's providing a national resource for researchers and companies to be able to run their AI workloads at scale in the UK for the first time. MICHAEL BIRD Amazing. Alright, well let's, let's make our way in. Um, and so, so this is, this is called Isambard AI. What, like, what, what exactly is it that we're go about to go into? SIMON MCINTOSH-SMITH What we're gonna go into, actually the first thing you'll see is it's in a, what looks like a big metal box. And this is the, the purpose built data center. It's actually a modular data center, which is special kind of data center that's actually prefabricated, offsite, uh, and then it's brought here in different components and reassembled, we actually reassembled this whole data center in just 48 hours MICHAEL BIRD So this is Isambard AI. What exactly is, Isambard AI? SIMON MCINTOSH-SMITH Yeah. Isambard AI, it's the UK government's first large part of its. AI research resource program. Uh, and this is a large AI optimized supercomputer that's built specifically for enabling UK researchers and companies to be able to run their AI workloads at scale for the very first time. MICHAEL BIRD Amazing. so this is part of the UK government's, uh, national AI strategy SIMON MCINTOSH-SMITH That That's right. Um, it's all funded by the Department for Science, innovation and Technology, or DCIT, uh, and they're investing hundreds of millions of pounds to ensure that the UK has access to the AI infrastructure, the AI capability that we need as a nation, MICHAEL BIRD so what problem, uh, is, is solving that other computers or facilities around the UK can't. SIMON MCINTOSH-SMITH Well, one of the, the things the government was really pressed to enable was the ability to train large language models in the UK and have a sovereign AI capability. So the compute would be here, the data would be here, the software would be here, and you needed a big enough supercomputer to optimize for AI to do that. And so Isambard AI has actually got nearly five and a half thousand of the recent Nvidia GPUs to enable that training of the, the large, largest large language models MICHAEL BIRD And and you sort of alluded this, but um, this was built incredibly quickly. SIMON MCINTOSH-SMITH government realized this was gonna be very important, they asked us how quickly could you do it? And fortunately, we had a bit of a head start because we were already building. A smaller general purpose supercomputer with HPE called Isambard three. And for that we were using a modular data center for the first time and we actually built a whole site from scratch in just four months. So from a car park to a site with a data center on it and a computer in passing all its tests in just four months. And for us, that was transformational. So we, we'd done that sort of thing. In the past, it had taken three or four years to get that down to three or four months. Is absolutely transformational. So we pitched the idea of building, using, about AI using modular data centers so that we could do it very quickly. We already had the site, we already had the power here, and we knew the modular data center would let us do it fast. So in fact, the whole thing start to finish from digging the first hole in the ground for the site to putting the data center in. Having the supercomputer inside, it switched on and in production, which means running real user problems. It was just 14 months. That's one, four months, just over one year to do that MICHAEL BIRD And when you say modular data center, like how, how much of it is, is modular? You know, does it, does this whole room just turn up on the back of a lorry SIMON MCINTOSH-SMITH . So the piece we're in was one big piece and it came over on six giant trucks, actually sailed across the channel on on the ferry, and then drove around the motorway to get here. Uh, so it was fully tested in the factory before it shipped. Uh, and then it could be reassembled incredibly quickly, just two days, 48 hours to put the whole data center together on site. MICHAEL BIRD Alright. Can we, um, can we go and have a look inside? Should we have a look? SIMON MCINTOSH-SMITH Yeah. MICHAEL BIRD Is that right? If I open up one of these? Big cabinets, they look like big wardrobes. SIMON MCINTOSH-SMITH But inside, when I open up the doors, you'll see lots and lots of equipment. Now we don't tend to have the flashing lights anymore like we used to on supercomputers of old. But instead, you'll see there are lots and lots of pipes, and these are pipes, not cables, and they're, they're blue and red, and that kind of gives you a hint as to what they're for. Now, all modern super computers at this scale are direct, liquid cooled, so they produce so much heat, they're using so much power. But the only way to cool 'em down effectively and, and keep the thing very compact is with water. So we actually have, this is water flowing around the system. It's a water glycol mix actually. Um, but that's keeping it cool. And in fact the blue pipes are the cool water coming in from outside. And you'll see the, where that comes from later on. And the red pipes are the hot water coming back off the equipment, which goes back out to be. MICHAEL BIRD So, so there's a, there's a temperature difference. Can I feel the difference? SIMON MCINTOSH-SMITH well, even the blue pipe is you can, you can touch it. Even the blue pipes are warm. So in some ways it's actually warm water cooling. Yeah. Okay. Which I know sounds a bit like an oxymoron, but it is warm water. So, so it comes in warm, but it comes out even warmer. And it's just that temperature difference is the thing that's providing the cooling effect. MICHAEL BIRD And so these, these are all individual blades. SIMON MCINTOSH-SMITH Yes. So these vertical, these big vertical metal, uh. Structures, which if I were to pull one of those out, each one of those would have lots of GPUs on it. And the GPUs -are the engine of an AI supercomputer. Um, and there are eight GPUs on each one of those blades. And there are 55 blades in each one of these cabinets. That's 440 GPUs in one cabinet. Now they do use quite a lot of power to energize all of those. In fact, this one cabinet. It's about a third of a megawatt 330 kilowatts. MICHAEL BIRD So, so how many GPUs are in this whole facility? How many cabinets are there? How many GPUs in total? SIMON MCINTOSH-SMITH We have 12 of these compute cabinets. There's 5,280 GPUs in this room, and this room is not very big. It's only 12 by 12 meters, but that can be up to four megawatts of computing power because it's so dense enabled by that liquid cooling. MICHAEL BIRD Yeah, this, um, this, this doesn't look like a, a sort of a server cabinet is what I'm expecting. You know, most server cabinets, uh, the compute is sort of horizontal. This is all vertical. Is this just for absolute density? SIMON MCINTOSH-SMITH This is for density. And in fact, what you can't see here, but on the back is all the networking and the networking actually runs horizontally. And then these two kind of mesh together, the compute blades slide into and connect with the, the networking blades that are on the back so that all that together is to give us this maximum density. One of these cabinets weighs 4,000 kilograms or four metric tons. MICHAEL BIRD And is the networking water cooled as well? SIMON MCINTOSH-SMITH Absolutely. Everything is watercolor fact you, you can't really tell, but that's very quiet. We are just having a normal conversation. Yeah, very quiet in here. There's a very faint hum that's going on, but that's actually from the pumps that are pumping the water around. The networking is water cooled. Even the power supplies are water called. So there are no fans in here whatsoever. MICHAEL BIRD Yeah. I'm quite used to a, like a screaming data center for all the fans. This is quite a nice environment. SIMON MCINTOSH-SMITH we don't need ear protection or anything like that, and we can have a conversation. MICHAEL BIRD Okay. So, um, we have something behind us. I'll, can we, um, yeah. Talk through what we have here. SIMON MCINTOSH-SMITH So this is, we talked about those blades, those vertical structures that we pulled out one of the cabinets, and this is what one of the blades look like. So it's quite long. It's surprising really just how long this thing is. And you can see there are these interesting looking things with pipes on it. So it's 1, 2, 3, 4 GPUs here, and there's another 1, 2, 3, 4 on that end. And the four actually grouped together into one server, or we call them a node. This is one node on one end and another node on the other end, and we've got eight GPUs in total. Down the middle is where all the networking goes. There's actually one network endpoint for each GPU to eight GPUs, actually eight network connections. Those are 200 gigabit. HP slingshot for each one. So there's lots and lots of network connectivity. The pipes are where the water goes for the cooling MICHAEL BIRD and the pipes are bright green SIMON MCINTOSH-SMITH The pipes are green. Actually, the water has been dyed green deliberately in case there's a leak, which makes it much easier to spot. And in fact, the dye also fluoresces under a, under a, uh, an artificial light. And you can see the copper plates with the little copper pipes. It's all part of the, the cooling loop. So that we can get the, the heat extracted from the GPUs into the water and outside to be, uh, cooled down again. MICHAEL BIRD and you've told me that I can't touch this 'cause this is probably worth more than my house. SIMON MCINTOSH-SMITH one of these would get you a decent flat somewhere. Absolutely. So these are very, very valuable things. But in the spirit of old fashioned tv, here's one I prepared earlier, and we do have one of the GPUs. And that one you can touch, that one doesn't MICHAEL BIRD So it's about the size of my, my, uh, my tablet that I was holding moments ago. I mean, so much computing here, and as you said, it's one of the fastest in the world. SIMON MCINTOSH-SMITH Yeah. Currently is about, AI is actually ranked number 11 on the top 500, which is sort of the league table of international supercomputers. It's the most powerful in the UK by a long way today. There are other AI research resource computers coming and, and national supercomputers, but today Isambard AI is about 10 times more powerful than the second most powerful computer in the country. So a lot of the UK's compute power is in this room right now MICHAEL BIRD Wow. What sort of things, projects, being run on the, um, on the system. SIMON MCINTOSH-SMITH So we we're now up to over two and a half thousand users, uh, and over 600 different projects all running on Isenberg ai. And it's across a very wide range of anything and anything you can imagine to do with, um, ai. Some of the projects are, for example, training large language models, uh, which might have a unique set of data going into them. So for example, we have one project training a. A large language model based on UK legal case law, and also other UK languages. So not just English, but also say Welsh and Scotts Gallic. So there's training type project. There's lots of projects in sort of health space where we're looking at things like automatically analyzing images of, of, uh, moles on the skin to try and determine if they're cancerous or not. Uh, and lots of projects. Doing things like, can we automatically optimize or discover new kinds of drugs? Which might be something that will treat cancer, it might be a vaccine. We've got some projects looking at vaccines for dementia. So this is the kind of thing you, in five or 10 years time, we might be able to go to the doctors and get a vaccine that will protect us from say, Alzheimer's. And that was developed on Isambard today. MICHAEL BIRD That's really exciting. So how, how do people come and run these projects? SIMON MCINTOSH-SMITH the allocation process is actually, uh, run by the people who funded the system. So that's part of the UK government, uh, called DCIT. So they and UKRI, who's our, our National Research Council, they run regular calls. So they say, we're looking for projects in this kind of area. Please submit a proposal. Uh, lots of researchers and companies as well, by the way. Can submit proposals with their ideas. And if the government likes what they've suggested, they'll award them the time as a grant. So as a, as a free grant on the system, and these have been anything from smaller projects, 10, 20, 50,000 G hours, we now have big projects being allocated on the system. They could be up to a million GPU hours. So that's the sort of thing that if you are buying that time in the cloud, that would be costing you millions of dollars. You can get that as a grant to run a, an exciting project. On is and about AI today. MICHAEL BIRD And can you, can you talk through any breakthroughs that have happened on the Isambard AI project? SIMON MCINTOSH-SMITH I mentioned the project looking at pictures of skin cancer. it had, uh, a hypothesis that some of these algorithms had bias built in that where they work better on pale skin and not so well on darker skin. And they've actually already proven that is the case and they've already, uh, improved the training data so that it's more diverse, rerun the algorithms and they've got a lot better. So one of the first breakthroughs that we've had was, um, already starting to significantly improve how accurate these algorithms work on a, a wider range of skin color. MICHAEL BIRD So, um, we talked about the, these GPUs and the technology here, but technology moves on very quickly. Um, so are you already thinking of what's next? SIMON MCINTOSH-SMITH Yeah, I, I'm always thinking about what comes next. So Isen about AI is actually the fourth in the line of different supercomputers that we've been building. We had isenberg one, two, and three now isen about ai we're already considering. What would we do next? Uh, the government's very keen actually to try and get significantly more compute than this even for ai, uh, by 2030. So there's a lot more to come. And we've got space nearby and more power nearby where we could build even bigger versions of Isambard nearby very quickly. We're even considering things like, um, different sorts of architecture that maybe will be optimized specifically for AI inference, and maybe not just everything running on the same GPUs, but different kinds of hardware, different kinds of problems. We've even got some early discussions about possible quantum computing projects that would be, uh, connected to AI in some way. So there's, there's always something that we're thinking about for what will come next. MICHAEL BIRD could you just add more modules to this data center, assuming there was space. SIMON MCINTOSH-SMITH space in, in theory, you could, I mean, this one's really been designed as a specific size for a specific computer, but the, this approach of having modules, this, this one is essentially four modules wide with a module on the end for storage and other, other air cord equipment on the end. But you could have designed one that was five, six modules wide. You can design them with that are one module at a time, but you have them next to each other and there are other sites that have been doing things like that. So it's a very flexible, very agile, very adaptable, uh, approach. It's kind of more flexible than a bricks and mortar building where once you built it, it's quite a big job to change it and knock a wall down and build extensions and things like that. So it's, it's a very flexible, adaptable approach using modular data centers. MICHAEL BIRD Simon, thank you so much for your time. Thanks so much for showing us all of this incredible equipment. Uh, it's lovely to have you on technology now. SIMON MCINTOSH-SMITH It's been a pleasure to be here and really nice to welcome you into Isambard AI. SAM JARRELL Wow, interesting discussion. First things first though, Michael, I wanna understand the mod pod a bit more – it's kind of fun to say, right? The Mod Pod. But, like, from what I understand, that’s what sets this apart from other supercomputers... MICHAEL BIRD Yeah. I mean, as Simon said, um, it's a modular data center, so it can be built really, really quickly. So I think, as Simon said in the interview, I think this was. Uh, 13 months, 14 months to build it. And it would've taken about three years if it was sort of traditional bricks and mortar. Um, but the whole thing is that you could just sort of spec it up. It essentially arrives in the back of a lorry. It's a bit more to it than that, and they sort of plonk it down. and it's all modular, so it's just a much faster way of doing it this was basically a car park before, this was before this data center appeared here. This and yeah, I think it's just about the sheer speed and I mean. The amount of power, the amount of, uh, GPUs and computing power just in this space, It’s sort of ... it’s quite incredible- SAM JARRELL (jokingly) 11th, fastest supercomputer in the world, but probably the fastest one period to put together, right? MICHAEL BIRD Yeah, yeah. Absolutely. Yeah. so the cool thing is, um, and I'll open this door again 'cause I like that I can keep doing this. Um, so yeah, as he said, so we have, um, multiple blades in this rack. I think it was something like 55. I'm looking down to my producer, Harry, just to double check. 55. It gives me a thumbs up. So 55 blades in here. Each of these blades has eight GPUs in them. They're all water cooled. So yeah, it's incredible. Like the, every single little piece, every single little bit of technology has been thought about to really just like hone and refine and make it as efficient as possible. SAM JARRELL Do you know why the cooling liquid glows? MICHAEL BIRD So I think they, they add it in because I think it's, it's water with, with some special coolant in it. Um, but I think they put the green dye in it. so they can see it so they can, you know, see if it's, see if it's flowing, see if it's leaked. SAM JARRELL it's kind of fun. It was very sci-fi. MICHAEL BIRD It's very sci-fi, very sci-fi looking. SAM JARRELL I love that. I also really appreciated some of the discussion around what this supercomputer will actually be used for. The main projects being like the multimodal health foundation model trained on the NHS data and then the cancer screening. To me, that's the kind of stuff that you want to hear about supercomputing and AI actually doing to be a force for good in the world. But which ones stood out to you? MICHAEL BIRD well, Simon talked, Simon and I talks about the, the cancer screening. So, um, screening for moles, uh, that sort of cancer screening. So, um, I'm guessing, analyzing that image data, um. I just think that's cool. I think to your point, you wanna be a force. We wanna be a force for good. And, um, just to think that like, again, all of this stuff is, is changing the world, it's quite a humbling place to be, quite humbling to be stood next to it. Um, and the fact that there are so many different, uh, projects running on this. it's only really been running for, um, about, you know, six, six to 12 months, I think. Um, so there's so much more opportunity, so much more that's gonna come from this. MICHAEL BIRD anyway, Sam, lemme just say, um, we, we heard from Simon about the, how the project came to be and how it works, but looking forward, I also wanted to ask if there was anything that we should be looking out for in the future SIMON MCINTOSH-SMITH for Isambard AI. I think we're literally only just starting. I think the most exciting projects that will run on this system haven't even been imagined yet. And in five years time, we'll look back and, and think, boy, we were just scratching the surface there. And now look at what's been possible. I think there'll be some amazing, uh, breakthroughs for health. I think there'll be some amazing breakthroughs in science and engineering. We have some projects running, doing amazing things in in FinTech, so I think we can't imagine today what will come out of it, but it's gonna be very exciting. SAM JARRELL Okay that brings us to the end of Technology Now for this week. Thank you to our guest, Professor Simon McIntosh-Smith And of course, to our listeners. Thank you so much for joining us. MICHAEL BIRD If you’ve enjoyed this episode, please do let us know – rate and review us wherever you listen to episodes and if you want to get in contact with us, send us an email to technology now AT hpe.com and don’t forget to subscribe so you can listen first every week. Technology Now is hosted by Sam Jarrell and myself, Michael Bird This episode was produced by Harry Lampert and Izzie Clarke with production support from Alysha Kempson-Taylor, Beckie Bird, Alissa Mitry, and Janessa Ayache. Our theme music was composed by Greg Hooper. SAM JARRELL Our social editorial team is Rebecca Wissinger, Judy-Anne Goldman and Jacqueline Green and our social media designers are Alejandra Garcia, and Ambar Maldonado. MICHAEL BIRD Technology Now is a Fresh Air Production for Hewlett Packard Enterprise. (and) we’ll see you next week. Cheers! SAM JARRELL Bye y’all