Latest / CXOTalk / Can AMD Change the Future of AI in 2025? | CXOTalk 884
Transcript
- 0:00AI infrastructure is a critical and rapidly evolving part of the
- 0:05artificial intelligence landscape.
- 0:08I'm Michael Krigsman, and today on CXO Talk Episode 884, we're
- 0:14exploring this topic with Mark Papermaster, Chief Technology
- 0:19Officer and Executive Vice President at AMD.
- 0:23Mark overseas an extensive portfolio of hardware and
- 0:27software. So let's get into it.
- 0:29I've been very fortunate because I've had the chance for the last
- 0:3313 1/2 years to be the CTO of AMD and an incredible inflection
- 0:40point in the industry. So myself and Lisa Sue were
- 0:44recruited at that time because AMD was a storied Silicon Valley
- 0:48company. It's over 55 years old and had
- 0:51such promise, but was facing challenges at that time.
- 0:54And so it was an opportunity to jump in right as AI was starting
- 0:59to come out. Think about, think about 2012
- 1:01when you were starting to 1st get natural language processing
- 1:05where it could have a much higher accuracy rate than other
- 1:08techniques using to, you know, actually be effective use of
- 1:13neural net engines and, and to be able to be that first promise
- 1:18of what was yet to come so that the timing was perfect.
- 1:22AMD had great building blocks, CPU technology, GPU technology,
- 1:27other, other accelerators and the know how to put them
- 1:31together to address a specific markets, everything from, you
- 1:36know, supercomputers down to, you know, PCs to embedded
- 1:39devices. So it's been a phenomenal
- 1:41opportunity. There's just great, great talent
- 1:44at AMD and, and, and like I say, it's just been a been a dream
- 1:48job. The technology world is going
- 1:52through another inflection point today with generative AI.
- 1:58What does that mean for AMD? The real inflection point was
- 2:04indeed generative AI. We we saw the promise coming of
- 2:08what what could be. And so that's why we started
- 2:10investing and making sure you know, our technologies would
- 2:14would be there, both hardware and software.
- 2:16But generative AI was such a fundamental inflection, Michael,
- 2:19because it made AI accessible to the masses.
- 2:22Now you could go into that started all of course, was
- 2:26ChatGPT. And so think about when in, in,
- 2:31you know, November 2022, ChatGPT comes out and suddenly it's a
- 2:36conversation you're, you're putting in tokens or questions
- 2:41into this ChatGPT and it's accessing supercomputing to give
- 2:46you, you know, answers that you, you, you, you never thought of
- 2:50getting that type of intelligence out of a computer
- 2:52before. So it really opened up AI and,
- 2:56and honestly, the supercomputing underneath it to the masses.
- 3:00And what have we seen? And, and what's hard to believe?
- 3:03It's just been several years since then, Just such a
- 3:07explosion of capability, more and more accuracy from those
- 3:11large language models and, and ChatGPT and, and, you know, all,
- 3:17all of the Grok and, and, and the rest of the models there.
- 3:21But you're also seeing now the shift to inference.
- 3:24So now people have really started to realize how you can
- 3:28deploy AI and fundamentally change most every process that
- 3:32we've dreamed of. So this shift to inter
- 3:35inference, this expectation of accuracy that you were just
- 3:41describing, what does that do to AMD in terms of your technology,
- 3:49your, your and your and your strategy?
- 3:52The need for more accuracy and now the shift of inferencing,
- 3:55meaning, you know, millions and millions and more users already
- 3:59ChatGPT is running, you know, 400 to 500 million users
- 4:04interactions per week. I mean, it's just stunning.
- 4:07And what I'd have to say is it means for us and frankly our
- 4:10peers in the industry, is it the need for the computation to
- 4:14support that is just growing exponentially.
- 4:17I call it this insatiable demand for more computing.
- 4:21And it needs to be efficient because it you can't, you can't
- 4:25just throw just more and more engines at it because it, it, it
- 4:29would burn more power than we have available.
- 4:32And so it takes, Michael, a lot of innovation to be able to
- 4:37drive forward that kind of computing capability in a smart
- 4:41way. And it really takes
- 4:43understanding how those software algorithms work with the
- 4:47hardware, because otherwise you can't, you can't truly optimize.
- 4:50So you were strategy then involves both the hardware and
- 4:57the software, the interaction. So you're, you're broadening,
- 5:01you have broadened the portfolio, could we say from
- 5:06your traditional business, which was more narrowly focused on the
- 5:10processors? We've always been a hardware and
- 5:12a software company. But but we were, you know,
- 5:15capital H hardware in small S software for for years, software
- 5:19was very, very important, but always an enabler.
- 5:23Now in this AI era, the software is equal or more important,
- 5:28frankly, than than the hardware because, you know, we, we we
- 5:32can't let off at all on hardware.
- 5:34As I said, they that those engines have to be more and more
- 5:37efficient at every generation. But to unlock that power, you
- 5:42have to understand where are the AI algorithms going?
- 5:46What are the what are the techniques that could be used to
- 5:48make AI more efficient? Can can we use different matrix
- 5:54math methods to make it more efficient?
- 5:57Are there algorithms? Think about the you know, some
- 5:59of your listeners may know of of the change when the transformer
- 6:02was introduced in the AI map model and and flash attention.
- 6:06These are all techniques in the algorithms to allow AI to be
- 6:10more accurate and and more efficient and the hardware has
- 6:15has to match up with that. So it takes a very, very tight
- 6:18collaboration of hardware and software.
- 6:21And that's what makes it hard, you know, to to bring
- 6:26competition, you know, to the market in AI.
- 6:29You know, I'll tell you, Michael, for AMD, we had to earn
- 6:32our way to get a seat at the table with these largest of
- 6:36companies that are that are creating the cutting edge large
- 6:39language models, the large language models, the new
- 6:42advancements in AI. We had to prove that that we're
- 6:45that player, we can bring that competition and that got us
- 6:49access with their developers and that allows us to make sure that
- 6:53our road map is absolutely competitive in AI going forward.
- 6:58What does winning for AMD mean in this market today as you've
- 7:05been describing it? We are very tight in our
- 7:07communications with with her CEO leadership and her
- 7:11communications is always driving us to be the very best that we
- 7:15can. When we enter a market, we
- 7:17intend to rapidly gain share. We want to be a a leader in that
- 7:21market. The analogy I'll give you
- 7:23Michael is what we did in in CPU servers.
- 7:26When Lisa and I joined, we actually had to actually exit
- 7:30the server market because we didn't have a sufficiently
- 7:33competitive X86 CPU. And So what we did was laid a
- 7:37strategy. Let's get a leadership CPU.
- 7:39Let's, let's, let's go win that market because it's clear that
- 7:43so many workloads were moving to the cloud, moving to the data
- 7:47center. And so we did exactly that.
- 7:49And we, it, it, it doesn't happen overnight when you do
- 7:51chip design. So it took, it took, you know,
- 7:54literally five years to get that new leadership CPU.
- 7:58It's called our Zen family of processors.
- 8:01But look at what's happened and the server market.
- 8:03We went from virtually 0 share in 2017 when we launched that to
- 8:08roughly 40% market share right now versus the, the, the, the
- 8:14incumbent leader, Intel that, that, that had that dominant
- 8:16share. And that's exactly what we want
- 8:18to do in, in GP US and CPUs and GPUs, actually they work very,
- 8:23very closely together. And that's what we want to do in
- 8:26AI. And that's what we've already
- 8:27started, I mean, which when we launched our first AI oriented
- 8:31processor, AI optimized processors, it's called the
- 8:34Instincts 300 MI 300 December of 2023.
- 8:38So first year production 2024, we went from again virtually 0
- 8:43revenue in these data center GPUs to $5 billion in one year.
- 8:48Fastest product ramp ever at AMD by far.
- 8:52And so it you know, so that was like, you know, again, zero
- 8:55share to about 5%. It's a huge market, but we won't
- 8:59let up. I mean, we'll do just that CPU
- 9:01journey I described to you. Was it it?
- 9:04It took generation after generation of listening to
- 9:07customers, putting out products, hearing, having that seat at the
- 9:10table and then improving the product, the hardware and
- 9:13software every generation. That is exactly what we're doing
- 9:17for data center GPU and frankly it's what we're doing across our
- 9:20portfolio because our whole portfolio from PCs to embedded
- 9:24device to gaming graphics, they're all AI accelerated.
- 9:29Go to cxotalk.com, Subscribe to our newsletter.
- 9:33We have incredible shows coming up, so check them out.
- 9:37How do you work with the model makers with open AIS and others
- 9:44in this world to optimize what you're doing against what they
- 9:49ultimately need? We had to beef up our skills in
- 9:52this area. You can't show up at an open AI
- 9:56or a meta, you know, or folks that are just absolutely steeped
- 10:00in, in the, the fundamentals, you know, down to all of the
- 10:04details of what it takes to create optimized a large
- 10:08language models handling these, you know, massive tasks that the
- 10:14generative AI, you know, massive scope, I should say, the
- 10:17generative AI takes on because, you know, think about the, the
- 10:22billions to trillions of parameters that are going into
- 10:24these large language models, Michael.
- 10:26So the, what we had to do and what we did was to mode match.
- 10:31So we took our brightest software leaders and we brought
- 10:35them to the table. You don't come in with a
- 10:37marketing team. You don't come in, you know,
- 10:39with a waving of hands. You come in with your deepest
- 10:43and and steepest of technical experts and then we grew that
- 10:48expertise, that software expertise through organic hiring
- 10:52and also acquisitions. And so at this point we have,
- 10:56you know, very skilled teams that can sit down and really go
- 10:59through and understand where are the bottlenecks.
- 11:02How can we hardware software that we provide the enablement
- 11:06across those CPUs, GPUs and now racks, you have to build that up
- 11:10into, you know, rack scale expansion to, to be able to
- 11:14handle the training and inferences for these largest
- 11:17LMMS. And so that is a muscle that we
- 11:21have built up over the last two years and it, and we had to do
- 11:25it very, very quickly. And we have to be incredibly
- 11:28agile. If there's one constant, as you
- 11:30work with the, you know, companies like Open AI and Meta
- 11:32and the rest, if there's one constant, it's change.
- 11:36We'll, we'll sit down with them and, and, and they'll say, well,
- 11:38guess what? We're going down this path.
- 11:40We found a better way. There's a better algorithm.
- 11:43You know, here's what we need to do different.
- 11:46And we were always an agile company, always able to be quick
- 11:49on our feet. That's one of the, the stalwarts
- 11:52of our, of our AMD culture. What we put that to test as we
- 11:55work with those, those AI model companies and, and they're,
- 11:59luckily we're good at it. So we react quickly and adapt
- 12:02and meet their needs. Folks, right now you can ask
- 12:06your questions of Mark Papermaster.
- 12:09If you're watching on LinkedIn, just pop your question into the
- 12:12chat. If you're watching on Twitter X,
- 12:16use the hashtag CXO Talk and get your questions answered.
- 12:22Mark, is there a cultural dimension to this change?
- 12:27You mentioned developing a new set of muscle memory muscles.
- 12:32Is there a cultural change that has to go on when you go through
- 12:36this type of inflection? We went through a cultural
- 12:39change at AMD first overall that's been frankly the fuel of
- 12:44the whole turn around. If you just look at AMD over the
- 12:46last 10 years, we've grown dramatically.
- 12:48Michael and I attribute our culture as a big piece of that
- 12:53and and the change in culture was to really focus first of all
- 12:58on execution. When you put out a product road
- 13:00map and you talk to customers and you listen to them.
- 13:03So you built in features that you know they need, you know,
- 13:06can differentiate your product and and make it better than the
- 13:09competition. You have to deliver that when
- 13:12you said you would with quality and become that dependable
- 13:16supplier. And so that is just been a
- 13:19maniacal focus. When I got here, I started
- 13:21working with the rest of the engineering team on really re
- 13:24engineering our engineering processes so we could be that
- 13:27repeatable engine of getting out new leadership, innovative
- 13:31technology cycle after cycle. And as Lisa stepped up from
- 13:36she's running all of our businesses in AMD and in 2015
- 13:42became ACEO of the company. And so if you think about
- 13:48actually as late 2014, she became the CEO and what she
- 13:52brought was this amazing focus across the entire company on a
- 13:59focus on customers, listening to customers, a focus on delivering
- 14:04products that really make a difference.
- 14:06That's that execution engine. And then thirdly, just
- 14:09simplicity. Let let's not be a, a, a company
- 14:12that's caught up in complexity, but really simplifying how we do
- 14:16things to make sure that we're the most efficient that we can
- 14:19be. So that was a, that was a change
- 14:21from a company standpoint. And then as AI, this explosion
- 14:25of AI, as you say, back to the release of the first generative
- 14:30AI. And now what we've seen is such,
- 14:32you know, a massive inflection industry that, that that's still
- 14:36going, that's causing yet again new muscle to the company.
- 14:40That new muscle is 1 like what I said earlier, being as much a
- 14:46software company, if not more than we are a hardware company.
- 14:50So that's been a change. And then secondly is just speed
- 14:56at which we move. I I have been in the industry 4
- 15:01plus decades and I have never moved at the velocity that we
- 15:06are now. And as I look across our whole
- 15:09company, we are moving at a faster velocity than we ever
- 15:12have. And when I look across the
- 15:14entire industry I see everyone moving at a faster pace.
- 15:18AI is accelerating the rate and pace at which innovation is
- 15:23delivered to the market. We have a question coming in
- 15:26from LinkedIn and again, folks, when else will you have the
- 15:30chance to ask Mark Papermaster, he's the CTO of AMD, pretty much
- 15:35whatever you want. So take advantage of it.
- 15:38And this is quite, this question is from Preeti Narayanan.
- 15:42And she says as the cost of running large scale AIAI models
- 15:49in public cloud environments continues to rise, many
- 15:53enterprises are re evaluating high performance on Prem or
- 15:58hybrid infrastructure options. Yet bare metal deployments bring
- 16:04their own challenges in terms of complexity and maintenance.
- 16:08From AM DS perspective, how should CI OS and CT OS navigate
- 16:13this trade off and are we approaching a point where hybrid
- 16:18AI infrastructure becomes the strategic norm?
- 16:21One size doesn't fit all. We think about that with our
- 16:24product portfolio. We're going to offer choice.
- 16:26We offer a broad ecosystem so our customers can think about
- 16:33really, really having choice, not a bestowed, you know, not,
- 16:36not, not just here's your only, you know, AI solution.
- 16:39Here's here's, you know, a rack that our competitors putting out
- 16:43there and, and, and not giving you the ability to really tailor
- 16:47it as you need. We're, we work with OEMs and we
- 16:52work with hyperscale. So what you're starting to see
- 16:54is a bit of a dichotomy. You're getting massive rack
- 16:58scale designs in the hypers hyperscalers that are supporting
- 17:02these largest of large language models.
- 17:05And so where you have significant training needs or
- 17:09large scale inferencing that you're doing, you're probably
- 17:12going to continue to run those on the cloud.
- 17:16And by the way, because of the efficiency that we're driving
- 17:19the industry, the cost per tokens going down.
- 17:22Now the cost of the computing is going up because we're adding
- 17:26more and more capabilities, but the actual cost per token is
- 17:30going down. But it's still, it's an
- 17:34expensive bill because all of industries bringing on more and
- 17:37more users because people are starting to deploy AI, they're
- 17:41running AI infra scene in most every process that they that
- 17:44they have. Consumers are using more and
- 17:46more AI in their daily lives. So that demands going up.
- 17:50How are businesses thinking about that?
- 17:52I do see them moving to a hybrid.
- 17:54I do see them, as I said a moment ago, using the cloud for
- 17:58those big tasks that they have. But they're starting to tailor
- 18:03and fine tune models to their business.
- 18:06They're harnessing the data they have and and so then you're not
- 18:10you're not needing the the LLM that you can ask it anything.
- 18:14You're creating your own large language model and in some cases
- 18:18small language models really tailored to more point task.
- 18:23And that's allowing businesses to develop the and to be able to
- 18:28support more economically on Prem and frankly lower latency.
- 18:33It's faster because it's right there at the point of, of the
- 18:35factory floor, the point of, of where the users are, are seeking
- 18:39those, you know, those immediate answers.
- 18:42So I, I, I do see a, a, a, a hybrid approach.
- 18:45I see a dichotomy and, and there's a third leg of that.
- 18:49And that's actually the embedded devices.
- 18:51So you have on Prem, you know, and you have what you're running
- 18:56in the cloud. And then thirdly, the edge for
- 18:59AI is really going and that's where you're literally embedding
- 19:02the AI engines at the point of data acquisition so that it's,
- 19:07you know, it's just immediately providing smarts to the to the
- 19:11process that's being controlled. Other guests on CXO Talk have
- 19:15also said that they believe that the future is smaller but more
- 19:21tightly focused models for particular domains and
- 19:25applications. It's obvious, right When you
- 19:28step back, you think about it's like, OK, as you start deploying
- 19:31AI, you know it will get tailored, it will get, it will
- 19:34become more efficient. People have to drive the cost
- 19:37down for point applications. We saw this coming at AMD, you
- 19:41know, quite some time ago. And we leveraged the fact that
- 19:43we have that breadth of portfolio across, as I said,
- 19:46everything from supercomputers. We have the number one and #2
- 19:49supercomputer in the world, right in an AM, DCPUS and GPUs.
- 19:53We're growing in, in the data center with both CPUs and GPUs,
- 19:57but also smart PCs, Michael. So we were the first to
- 20:00introduce AI, deep AI acceleration for copilots.
- 20:06In the PC and that's been a really growing piece of our
- 20:10portfolio, but also we just introduced with our latest
- 20:15version of gaming graphics leveraging AI there beautiful
- 20:19upscaling and graphics or leveraging the embedded AI.
- 20:23And then think about our acquisition of Xilinx IN2022A
- 20:27leader in embedded devices, programmable Gator Grays along
- 20:31with the the embedded X86 business that we have all of it
- 20:35AI accelerated. So we're, we, we are absolutely
- 20:39seeing this coming and expect to see our continued growth of AI
- 20:43deployments as people understand how to harness their data and
- 20:47bring inference, effective inference applications.
- 20:51Let's take some more questions. We have some from Twitter right
- 20:55now from X and this is from Ricardo Dionda.
- 20:59And he says, how does AM DS internal IT organization act as
- 21:07Customer 0 for your AI solutions?
- 21:12Are there examples you can share where internal adoption directly
- 21:16influenced product development? We're a technology company.
- 21:20Shame on us if we didn't make ourselves Customer 0.
- 21:24But we have actually been doing this for for the last years and
- 21:28starting about four years ago, we actually raised the
- 21:32visibility of IT significantly in being customer 0.
- 21:36So our CIO is directly engaged. Hausmach will engage with
- 21:43customers, he'll understand their needs, he'll share with
- 21:45them how we are deploying AI, and he's also supporting our
- 21:49engineering teams as they're running our compute systems.
- 21:53And you know, to the extent where we don't think about
- 21:56bringing out a new data center GPU, a new data center CPU, a
- 22:00new PC, we don't even think about bringing it to market
- 22:03until RIT has already exercised or already deployed it first to
- 22:08the, the users, the, the compute users at AMD and that we've
- 22:13tested, we build, we build clusters of data center
- 22:16computation first. We do deployment of of hundreds
- 22:20of PCs before it goes out to the market.
- 22:22And and likewise across our embedded applications, it's a
- 22:26little bit harder on than embedded applications, but it's
- 22:29not hard at all for PCs through our data center computing
- 22:34products to be customer 0. And that's exactly what we do.
- 22:38In fact, I'll add, we've actually speed our chip design
- 22:41process up. That's what we do.
- 22:42We develop chip designs and and the hardware and software has to
- 22:45work effectively. So what do we do in the chip
- 22:48design process? We've we are using AMD and just
- 22:51think about, you know, there is billions of transistors.
- 22:55Our latest AI chip has 154 billion transistors.
- 23:00Actually our newest one now has over 180 billion transistors.
- 23:04How do you get all those transistors laid out across the
- 23:07silicon in in the mass vast wave of interconnect that you have to
- 23:11put this together, It all has to be perfect.
- 23:14You can't have one transistor, one connection.
- 23:16That transistor doesn't work. AI it turns out is very, very
- 23:19effective at helping ensure that we not only have the most
- 23:22optimum implementation, but it's helping our test processes and
- 23:25getting the coverage to make sure that no defects can can
- 23:29escape through our manufacturing test line.
- 23:33And and so it's it's been very, very effective not just in
- 23:36engineering, but even our business practices.
- 23:38So being customer zero also brings direct benefits as you
- 23:44speed the internal AI applications success efficiency.
- 23:49Let's grab another question. You can see we the audience is
- 23:53an incredible audience and really smart.
- 23:56So I I always defer to their questions in front of my own.
- 24:01And this is from Elizabeth Shaw who says, so what's the
- 24:05Technology Strategy? Is it thinking of all the
- 24:07product lines or families as one system distributed over
- 24:13different domains? When you think about a product
- 24:16portfolio like we have at AMD, you have to be a flexible in how
- 24:22you chart the direction. What do I mean by that?
- 24:25Each product itself has to stand alone in its own category.
- 24:30So CPU has, you know, X86 Zen line of CPUs.
- 24:34It has to be the best X86 CPUs that are out there.
- 24:39Same thing with our GPUs. They have to be that, you know,
- 24:42the best engines that, that are out there.
- 24:44The same thing across our, our, our gaming graphics, our
- 24:47embedded devices. So you have to think about all
- 24:52the building blocks our, our road map has to make sure a
- 24:55hardware and software, the, the, the software enablement that we
- 24:59are best of breed. And then when you, when you put
- 25:02it together, what we have is a strategy of how they are
- 25:06deployed, how we tailor that to our products that meet the
- 25:10customer needs, how many CPU cores they need.
- 25:13How do you need to tailor it when it's being used for
- 25:16database processing versus that same CPU is being used as a head
- 25:21note, a controller for a a vast number of GPU's that it's
- 25:25managing 2 totally different use cases.
- 25:28And likewise that that applies across each of our portfolios.
- 25:31You have to think through the use case, the dominant use cases
- 25:35of our customers, making sure you're optimized.
- 25:37And then there's the other third step, and that is we have to
- 25:40think across our whole portfolio, how can we get
- 25:43synergy for AI? What we've done is we have one
- 25:47software enablement stack, ROCM. That's our name of our software
- 25:51enablement and that is, is going to be the the, the top level of,
- 25:55of making it easy to deploy AI across whether it be our data
- 26:00center GPU's, our CPU's or our gaming graphic devices or
- 26:05embedded devices. And that's an area that we're
- 26:08incredibly focused on right now in 2025 and really opening up
- 26:12that capability to users. You can now run ROCM on Window
- 26:15our AMD based Windows PC's. You can now run rack em on the
- 26:20latest of the Radeon graphics cards and and as well have it
- 26:25optimized. Even CPUs are people don't
- 26:28realize CPUs are an essential part of AI processing and we
- 26:33have them well enabled as well. This is from Chris Peterson who
- 26:38says on the technology side, where does AMD see the GPU to
- 26:46GPU interconnects going On the biz side, what's AM DS plan if
- 26:53the AI industry pivots massively in algorithm design or the power
- 26:58water sustainability just can't keep up with demand.
- 27:02So 2 questions very, very quickly.
- 27:04GPU to GPU interconnect and then the larger set of business
- 27:08issues. We have previously are gone
- 27:11about that with using our proprietary Infinity
- 27:13architecture, which has been proven through our generations
- 27:16of CPUs and and GPUs today. But we have a commitment in AMD
- 27:22to an open ecosystem. We think it's extremely
- 27:25important that you know again that you don't just have a a few
- 27:28bespoke offerings because it's a a wall garden approach.
- 27:32And so we support and we're a founding member of Ultra
- 27:35Acceleration Link. And what Ultra Acceleration Link
- 27:38is, we donated the kind of rotocol we used to connect their
- 27:42GPUs and now it's out there, not an AMD control of a control with
- 27:46multiple companies that are running the UA Link consortia
- 27:49that are ensuring that that other switch vendors can play
- 27:53other competitors of ours that are, are creating their own
- 27:56accelerators. They can use this ultra
- 27:59acceleration link standard and use the same switches and, and
- 28:03you know, connectivity solutions that are out there.
- 28:05Again, we are committed to an open ecosystem and that is our
- 28:09strategy for for GPU to GPU links.
- 28:11The second questions of course critically important, you know,
- 28:15what do we do as as the demand for energy consumption goes up
- 28:21and up and up to accommodate our insatiable demand for more AI
- 28:25computation? You know, my short answer is
- 28:27innovation. Now there's just the, the demand
- 28:30is just so high. You look at, you know, the
- 28:33hundreds of billion dollar market opportunity we're looking
- 28:36up there at, at, at, you know, by, you know, just in the next
- 28:38few years, by 2028, the market size is going to be that large.
- 28:43And so that's an incredible pull for innovation.
- 28:46When you have a market that large, what it, what it means is
- 28:50we're going to, we're going to continue to innovate.
- 28:52How do we make these GPU's more efficient?
- 28:55And we're going to, you know, how do you make that GPU and CPU
- 28:58work more efficiently together? But even more than that, you
- 29:03know, how do you, how do you support where you have
- 29:05algorithms that aren't changing so quickly, so you don't need
- 29:08that programmability of that GPU and CPU.
- 29:11We support as well tailored devices and, and custom devices
- 29:15that our customers could, could work with us on.
- 29:17So I think we're going to see that whole range.
- 29:20You know, I'll just say just to show our commitment and we just
- 29:24hit a milestone for CPU and GPU computation for these most
- 29:30demanding AI and supercomputing workloads.
- 29:33We set out in 2020 that we would drive a 30X improvement by 2025
- 29:39S in a five year period 2020 to 2025, we'd have a a 30X
- 29:44improvement in performance per Watt energy efficiency.
- 29:47And we actually declared just a couple weeks ago with the advent
- 29:50of our new MI 350 series, we not only hit that 30X improvement
- 29:55and efficiency, we actually hit 38 X.
- 29:59So we we do that we put out measurable milestones to incent
- 30:04our own team to be efficiency. We work with professors and
- 30:07universities to make sure it's not marketing fluff, it's it's
- 30:10real efficiency that's going to be delivered at the end of the
- 30:13day. And so now we're on to our next
- 30:16milestone. Sam Nassinger leads this for us
- 30:18as just put together again working with the community,
- 30:21working with professor so we can measure it a rack efficiency
- 30:25gain. So as I told you these now scale
- 30:27it rack level of how you put these AI compute clusters
- 30:31together and we've committed to a 20X improvement at rack level
- 30:36by 2030. So that's our next energy
- 30:38efficiency milestone that we're focused on.
- 30:41You've spoken about significant changes in potential or, or the
- 30:48innovation necessary to manage potential, significant changes
- 30:53in algorithms. You've spoken about the need to
- 30:57for sustainability. Are there other big issues that
- 31:02you have on your radar right now that you're focused on?
- 31:07We don't have a choice, Michael. We have to look at the entire
- 31:10landscape. And so we worry everyday.
- 31:14We worry everyday first on disruptive technologies are
- 31:18coming. We have a a great research team.
- 31:21They're constantly look ahead at looking ahead at what's what's
- 31:24coming in terms of our computation, how we can
- 31:27interconnect that computation to be more efficient.
- 31:30Our networking technologies, we've got very, very innovative
- 31:34networking through our acquisitions of Violinx and
- 31:37Pinsando, just super technology of how we how we can network
- 31:41these AI competition. And now with our acquisition of
- 31:44ZT Systems, the landscape includes the rack design and,
- 31:48and driving that innovation. So we, we call it holistic
- 31:52design. It has to be cross disciplinary
- 31:55of all of these groups, hardware and software working together to
- 32:00not only design the next generation, but look one and two
- 32:04generations beyond that. What's disruptive, what's
- 32:07coming? We're partnering with others on
- 32:09on quantum computing. You know, I won't get in the
- 32:11debate as when does quantum go mainstream?
- 32:14Quantum will start off as an accelerator.
- 32:16It'll be, you know, bespoke applications that can really
- 32:19benefit from quantum, but we're going to be there already our
- 32:22Xilinx based Fpgas in our portfolio are used to, to as
- 32:28controllers in most, most every quantum computer out there
- 32:32today. And we also, you know, work with
- 32:35the rest of our portfolio to be that CPU and GPU complex that
- 32:40can work with those quantum accelerators, just an example.
- 32:44But it's really the entire landscape, Michael, that we have
- 32:47to be looking out again, not just today's products, but the
- 32:52next generation and generation beyond.
- 32:54Again, from Chris Peterson, AMD, NVIDIA and others seem to be
- 32:59competing relatively head to head in terms of overall design
- 33:03for AI. What about plans to compete with
- 33:07niche players like Sarah Brus or others that have wildly
- 33:11different solutions? So I'm interested really in in
- 33:14both your competition with AMD as as well as with the other
- 33:19niche players that Chris Peterson mentions.
- 33:21We are a mass producer of computing technology and so it
- 33:25doesn't behoove us to be, you know, the the first to market in
- 33:30a niche application, Cerebus, Cerebus.
- 33:32I give that team a lot of credit.
- 33:35Andrew D Roz and team do do a great job of a wafer scale
- 33:40integration and there's certain tasks that when it fits in that
- 33:45model and can leverage that, that, you know, flow processing,
- 33:50data flow processing to a wafer scale it it's going to be
- 33:55certainly have the advantages, but it's not, you know what it
- 33:57it is specific workloads that can really benefit there.
- 34:01And so they're doing well in, in that we'll watch that if the, if
- 34:04workloads that can fit in that kind of application growing, I
- 34:08could say the same about any other the startups that are
- 34:11focused in certain areas. We watch it closely.
- 34:14And if we hear from our customers that that's a gaining
- 34:17traction that more and more models could leverage that
- 34:19we're, we're, we're going to, you know, bake those approaches
- 34:23in our plan. But we, we, we have to make sure
- 34:27that we're listening to as well, given the role AMD plays in the
- 34:30industry, that we are really looking and, and making sure
- 34:34that we're one and two steps ahead of mass scale AI
- 34:38computation. Two of the core themes that you
- 34:41have mentioned are execution and to simplify it, keeping your ear
- 34:48to the ground on what's happening in the industry and
- 34:51where things are going. Do I have that right?
- 34:54Absolutely. I wouldn't manage my day-to-day
- 34:58anything any different than I do right now and that I have in the
- 35:01entire time I've been in the CTO role at AMD and that is
- 35:04prioritize having that ear to the ground.
- 35:07So I prioritize customer visits and I fly to customers as are
- 35:12set up, you know, virtual meetings like you and I are
- 35:14having today and, and very much make sure I'm in listen mode.
- 35:19I don't go them in and just beat my chest and say, you should be
- 35:22using AMD. Here's how we can give you the
- 35:24better total cost of ownership. We can help save you money.
- 35:27I do mention that I, I leave that in for sure, but that's not
- 35:31the reason I'm, I'm making that call.
- 35:33I'm also, I'm first understanding what challenge are
- 35:35they facing? And I want to make sure that,
- 35:37that our portfolio can address that and then I can educate them
- 35:41and, and show them where we can bring them advantage.
- 35:45And likewise, we follow the competition.
- 35:47I, I run a regular process within AMD, which is just our,
- 35:52our competitive review. So we look at any announcements
- 35:55that our competitors are making because we don't ever want to be
- 35:59arrogant. Like we can never have an
- 36:01attitude in AMD that, oh, we, we're just better than everyone
- 36:05else. We have the right way.
- 36:06What other people are doing is wrong.
- 36:08The converse we look and we we are constantly asking ourselves,
- 36:13do we have that best practice? Is there, Is there if there's a,
- 36:17a better approach someone else had, let's top that yet again.
- 36:20Let's let's use that to spur our innovation on to make sure that
- 36:25that we don't find ourselves in any way disadvantaged.
- 36:29And on this topic, Christine Lofgren from LinkedIn asks about
- 36:36leadership practices that enable you to be responsive, to
- 36:42innovate and do the things that you're describing.
- 36:45We often look past how important fundamental communication skills
- 36:50are. I mean, we, we, you think that
- 36:52sounds right. Like of course you have to
- 36:53communicate well. Well, actually I'll tell you
- 36:56it's paramount. Like, you know, I have, I have
- 36:59spent 4 decades trying to hone my communication skills and
- 37:02communication means two way. You think you're listening.
- 37:05Are you really listening? Did you play it back to the
- 37:09person who is talking to you to make sure you got it right?
- 37:12And, and that applies externally and internally.
- 37:16It also is just how do, how do you communicate what you're
- 37:19doing in in your North star of the team or is, is the whole
- 37:22engineering team aligned on the priorities that we have and,
- 37:26and, and how we achieve those goals that that takes an
- 37:30investment in, in excellent communications.
- 37:34And then with our customers, are we really articulating in our
- 37:37value proposition and, and, and what it is we're about in AMD
- 37:40and how we can deliver, deliver them value.
- 37:43So across the panacea, internal and external, I I think
- 37:48communications vitally important.
- 37:52It has to be married with a sound strategy to win.
- 37:56And so if you take the time to really develop a strategy that
- 38:00will be make it clear for all the investments we make, how
- 38:04we're differentiated, how we bring value, how we're going to
- 38:07win. And you marry that with
- 38:09excellent communication that that's that's how you went.
- 38:13Do you have any quick advice on communicating well for business
- 38:18executives? Seems like an obvious topic, but
- 38:21you've just emphasized the deep importance of it.
- 38:24You have to put the time in it. So it comes easy to me now
- 38:27because I've been doing this for, for, for decades, but it
- 38:31didn't always come easy. I, I, I invested the time to, to
- 38:34hone my skills at, which means you have to put yourself in
- 38:38uncomfortable situations, force yourself to be where you are not
- 38:43comfortable. And again, that can be internal
- 38:46communications or external communications, but get get out
- 38:50of that zone that you've been living in.
- 38:53Stretch yourself, hone your skills, get feedback.
- 38:56Lisa Sue always says feedback's a gift.
- 38:59It really is like put your thick skin on, get feedback from those
- 39:03that you know, will tell you the truth, not, not butter you up
- 39:06and tell you, Oh yeah, yeah, that was great.
- 39:08You know, get unfiltered feedback.
- 39:10And then lastly, in this age of AI, use AII use AI every single
- 39:15day to help me and give me information to have better
- 39:19decisions, but also in communication, proofreading what
- 39:22I did, what was not clear, what was the tone of that
- 39:25communication? I just wrote it.
- 39:27It it's that right hand person. You know that that it's, by the
- 39:32way, getting better and better every generation.
- 39:35That's there to help you. Isn't that amazing that you,
- 39:39given your role as CTO of AMD, just referred to the IT as a
- 39:49they essentially. Why?
- 39:51Because you use it that way. And so when you're deploying AI,
- 39:55you're thinking about like, oh, I need help.
- 39:57Do I get this person to help me, that person to help me, or do I
- 40:00want the AI to help me? That is how I think about it.
- 40:03I think that's how most people are starting to think about it.
- 40:05I use numerous models every single day.
- 40:08And absolutely, that's certainly how I think about it, Michael.
- 40:11Have you used some of the new reasoning approaches that are
- 40:15out there and and the the research inferencing
- 40:18capabilities are out there. Have you, have you tried any of
- 40:21those capabilities? Yeah, all the time.
- 40:24And now one of the hardest challenges I face is figuring
- 40:29out what's the right model to use for the right task.
- 40:34And if you use the wrong reasoning model, you'll end up
- 40:37sitting there for 20 minutes while it gives you back the
- 40:39wrong answer. Absolutely.
- 40:41How you deploy, how you prompt, how you iterate, So important.
- 40:45You asked me a question earlier about how does algorithmic
- 40:48change? How does that affect AMD now
- 40:50that we took in as we hit, as we hit the reasoning and, and as
- 40:55well as a gentic processes that spawn off, you know, many, many
- 40:59tasks. I can tell you it had a huge
- 41:02change in terms of how our computation engines are
- 41:05deployed. So when you go through
- 41:06reasoning, it means that you're going through multiple loops of
- 41:10inference, you're doing the forward chaining of your
- 41:13machine, you know, your, your machine learning processing
- 41:17again and again and, and again to get more accuracy, but
- 41:21primarily also bigger context windows you're building on.
- 41:24I ran AI, ran a, an inferencing loop and I got these kind of
- 41:27answers. Well, I want to remember that,
- 41:29but I want to build on that and I want to use maybe a different
- 41:32expert or different kind of model.
- 41:33I'm going to, I'm going to build on that.
- 41:35It turns out that's like a database that you're having to
- 41:39manage of all that context. And so it needs great CPU's,
- 41:42which we have at AMD. And So what we're seeing is
- 41:45more, more and more having that strong CPU paired with the GPU
- 41:51to, to be able to be effective on these new modes of, of
- 41:55inferencing with the reasoning and in the research.
- 41:59And then likewise agentic AI, because agentic AI, you can code
- 42:03it up to handle a number of tasks.
- 42:05It can go out and run, you know, get to API, the application
- 42:08programming interfaces and spawn other tasks.
- 42:12And those what we're we're finding are often on CPU's.
- 42:15So it's it's really driving a different mix and the
- 42:18computation for AI. Are you helping guide your
- 42:23software technology partners? The model makers in terms of
- 42:29best practices for offloading certain kinds of tasks to the
- 42:34CPU versus the GPU and when to go back and forth and so forth.
- 42:38When you think about that interaction that we have with
- 42:42the model developers, they're the deep experts of the model
- 42:47itself. We're the deep experts of of
- 42:49that communication, you know, the communication turn tuning
- 42:54across the different compute devices and how to mode match
- 42:58that with the algorithms. So it's very much a give and
- 43:01take, a brainstorm that you know is, is pretty incredible to see.
- 43:07We have a question from LinkedIn changing gears entirely here
- 43:11from Doctor Anker who Pattier and he says what are AM DS plans
- 43:17for future expansion, Any plans with Southeast Asia?
- 43:22We are constantly expanding. We have grown across Asia.
- 43:29I mean, before, you know, when I think about what we've done and
- 43:34we've, we've been in Malaysia and Singapore for many, many
- 43:38decades. I think we've been in Malaysia
- 43:4050 years. And so you know, we continue to
- 43:44expand worldwide and including Southwest Asia, India, we're
- 43:48going to be, we're at 8000 employees today, 8000 engineers
- 43:54and we're on a pace to be at 10,000 by 2028.
- 43:57So across, you know, and and you know across Taiwan, China, you
- 44:04know, we look at where the other geos are.
- 44:06So it's not just there. We're equally expanding in
- 44:11Europe and you know we are, we are an example of a global
- 44:18company, a multinational company, MNC.
- 44:21We serve a global market and so our engineering force and our
- 44:25and our sales force are absolutely global.
- 44:28This is from Vishal Bhargava and he says on LinkedIn he says
- 44:33which research and inferencing products are we talking about?
- 44:38Can you please share specific names?
- 44:41Oh, when I'm talking about research products, I'm talking
- 44:43about literally like open AI research.
- 44:45So you can get a research subscription with open AI and
- 44:49you and, and there's a, it costs more, but you get you, you truly
- 44:54have, you know, that that researching capability I was
- 44:57described and, and you can get the similar functions with, you
- 45:01know, anthropic and other models that are out there.
- 45:04Ricardo Deonda on Twitter X comes back again and he says
- 45:09what are the biggest challenges AMD faces in managing machine
- 45:14identities across hybrid and multi cloud environments,
- 45:18especially with the rise of AI driven automation?
- 45:22Identity is a huge focus across computing.
- 45:25I mean, we do an authentication every time we talk to any other
- 45:29computer to make sure that we know it's a valid entrusted
- 45:34compute device. So I would I would probably need
- 45:37a little bit more context to answer your question the best.
- 45:41How do geopolitical considerations and the drive for
- 45:45supply chain resilience directly influence AM DS, AI, product
- 45:51strategy, R&D investments, and global manufacturing
- 45:56partnerships? And I'm not trying to make this
- 45:58up a political conversation. Our supply chain, like everyone
- 46:01in the industry has had to become very agile.
- 46:04And so is, is there's tariffs which apply to certain products
- 46:09and certain sourcing locations. We need to have the agility to
- 46:14manoeuvre as best we can to mitigate those impacts.
- 46:17Otherwise the cost of our products would go up.
- 46:20And so we, we do that. We have a fantastic supply chain
- 46:23team. What Kayvon and his group do
- 46:27are, are very flexible. We, we are again, we're a global
- 46:30company and we have global manufacturing.
- 46:32And so we've been agile just like I said, we have to be agile
- 46:36to adjust our products and the list of our customers and
- 46:39optimize them for computation. Turns out, in the current
- 46:43geopolitical, our supply chain has to be equally agile.
- 46:47On LinkedIn, Christine Lofgren comes back and says What key
- 46:51trends or developments, both within AMD and the wider tech
- 46:55industry, do you expect to shape a IS trajectory over the next 12
- 47:00months? One is there's more and more
- 47:03inferencing you're just going to such innovation on models
- 47:06already. I'm so impressed with those some
- 47:08startups or some businesses have been able to do with small
- 47:11language models that are that are really pointed at tasks.
- 47:14And I just think we're just at the absolute beginning, the tip
- 47:18of the iceberg of tailored models that that bring
- 47:21innovations for, you know, you know, very unique tasks that can
- 47:27be incredibly efficient. So I think model development,
- 47:31both tailored in small language models, but also to allow us to
- 47:35to scale the accuracy on the of the large language models is, is
- 47:40hyper scalers are driving to artificial general intelligence.
- 47:43So I think we're going to see just continued innovation across
- 47:47that spectrum. And then likewise, the base
- 47:50technology itself, how we put it together, you're just going to
- 47:54see tremendous innovation to make it more power efficient
- 47:57across everything from supercomputing to the smallest
- 48:01devices or tremendous innovation going on in this area.
- 48:05Everything from materials to new types of transistors, how the
- 48:08memory is is connected more closely and more efficiently to
- 48:12the compute devices, networking and on and on.
- 48:15Mark, you mentioned artificial general intelligence, AGI, Can
- 48:20you, you're in a very unique position working with these
- 48:24model developers. Do you have thoughts on the
- 48:29trajectory of the models going forward, say over the next year?
- 48:34Where do you think the world will end up, say a year from now
- 48:38or nine months from now? We can see nine months.
- 48:42I don't know we can go much beyond that nine months to a
- 48:45year, but but when you look at that time frame, you're seeing a
- 48:49continuation of build, you know, of, of of more and more accuracy
- 48:53coming out of these models, larger context, being able to
- 48:56handle, you know, much more thoughtful answers.
- 49:00Particularly like I talked about earlier with these reasoning
- 49:03techniques that are being applied, that trend will
- 49:05absolutely compute, you know, that, that trend will actually
- 49:07continue over the next year as well as more and more compute
- 49:11capability needed to support that.
- 49:14The, the, the tough question is five years from now, where are
- 49:17we, you know, five years from now, what actually is that
- 49:21definition of AGI? Do we hit it five years from
- 49:23now? Lots of debate out there, you
- 49:26know, and, and I'll, I'll leave that to others to prognesticate
- 49:30what is that definition of AGI and, and the precise date in
- 49:34which we hit it. What about agentic AI that
- 49:37you've mentioned several times? It seems like that is such an
- 49:42important architecture, if we can call it that, that it's not
- 49:45going away anytime soon. Thoughts on that?
- 49:48I talked to the CIO of a of a major, major bank and she was
- 49:52describing me the progress they've made in agentic AI.
- 49:55I mean, they, they, they've got almost the entire company
- 50:00enrolled in deploying AI and what they're, you know, every
- 50:04week they're finding point applications that can just be
- 50:08speed up by creating an, an agentic AI and, and think about
- 50:12that. That's a ripe area.
- 50:13Banks handle transaction processing.
- 50:16I mean, so much of their time is, is spent on well defined
- 50:20tasks, well defined data. And so if you can create an
- 50:24agentic AI that has, you know, clear goals, clear sets of data,
- 50:29you know, clearly defined AP is to, to connect different tasks
- 50:34together, it's going to drive, you know, a very, very quick hit
- 50:38of efficiency. And I, I, I do think we're,
- 50:41we're going to start seeing that, you know, more and more,
- 50:44we're certainly doing that internal to AMD, looking for
- 50:47processes where a Gentek AI can speed our productivity.
- 50:51And it leads Michael to the, you know, you asked me to keep it
- 50:53short, but I'll just so keep it very short.
- 50:56But that it's a gentic AI that I would urge people to think about
- 51:01where, where can you make people really more productive?
- 51:04Like do do people want to be spending their time where it is
- 51:08very, you know, very much a, a process that a gentic AI could
- 51:13do for you. Lean in, get that agentic AI to
- 51:17complete those tasks to free up the time for the innovation, the
- 51:21creativity, you know, the really, you know, human driven
- 51:25opportunities that we have in front of us as a society.
- 51:28That is going to take some time to diffuse that kind of
- 51:33knowledge and the ability to breakdown processes because it's
- 51:36one thing folks like yourself who are really advanced
- 51:39technologists who understand the link between the technology and
- 51:43the business process and being able to decompose, but there's a
- 51:46maturing process across industry that will need to take place.
- 51:50Absolutely. It's actually again, cultural as
- 51:52well as as well as technical. And so therefore it, it does
- 51:56take some time, people have to get their heads around it.
- 51:58They have to build the technical acumen to be able to deploy.
- 52:03But I think it's going to happen much more quickly than we ever
- 52:06anticipated. Mark, as we finish up advice for
- 52:11CIOCTOS inside large organizations in the enterprise
- 52:16at this point in time today and to generalize based on you talk
- 52:20with lots of customers, what should they be really focused
- 52:23on? One, I'd say if you're not
- 52:25moving more quickly than you ever have before, something's
- 52:28wrong because the, the pace of change, you know, really is, has
- 52:33really inflected over the last, you know, several years and will
- 52:37continue and, and, and, you know, the period that we could,
- 52:41any of us could project. So I think 1 is, you know, if
- 52:45you're not comfortable embracing change, you have, you have to
- 52:48adapt. And I don't mean change without
- 52:51Mana's risk. One of the things that we drove
- 52:52in our AMD culture is take the risk, but manage it.
- 52:55Make sure you have checks and balances.
- 52:57Don't, you know, don't walk off the edge of a Cliff without, you
- 53:01know, a, you know, a rope attached to keep you from
- 53:05falling in the, the chasm. So I think that, you know,
- 53:08that's one is, is really be receptive to change, but manage
- 53:11it to ensure that, you know, you're controlling your, your
- 53:14risk. And and the second I would say
- 53:19that that comes with that is stepping back and examining
- 53:24what, what is your core value property?
- 53:26What is the key things you're doing?
- 53:28And in an AI era, can it be, can can you really re architect how
- 53:33you do things? Are there the very fundamentals
- 53:36and processes by which you run your business that can be, that
- 53:40can be re architected in the AI area?
- 53:43So both, both those would be the probably the two biggest piece
- 53:47of of the advice I would share to, to CI OS and heads of
- 53:51infrastructure out there. And I would say help AMD in our
- 53:56quest to create a competitive environment where there's an
- 53:59open ecosystem and competition out there, not one dominant
- 54:03supplier. I think that's bad for everyone.
- 54:05And personally I want faster chips so I can render video and
- 54:09do all the things I do faster and faster and faster and with
- 54:12with not quite so much heat. That, that, that's unending
- 54:17quest. I mean, that's what
- 54:18technologists like myself, we have no choice.
- 54:20We have to. We have to stay focused there
- 54:22every single day. I'm Mark Papermaster, Chief
- 54:25Technology Officer and Executive Vice President at AMD.
- 54:28Thank you so much for taking time to be with us.
- 54:32I'm very grateful to you, Mark. My pleasure, Michael.
- 54:35Thank you. And thank you to everybody who
- 54:37watched and for your great questions.
- 54:39Before you go, go to cxotalk.com, subscribe to our
- 54:44newsletter. We have incredible shows coming
- 54:47up, so check them out and we'll see you again next time.
- 54:50Thanks so much everybody.