Latest / Wired to Build / Transparency, Scale & the AI Arms Race: Redefining Data Center Delivery with Matt Caldwell

Transcript

Transparency, Scale & the AI Arms Race: Redefining Data Center Delivery with Matt Caldwell · Wired to Build

0:00
Hey, what's going? On everyone, welcome to Wire to
0:02
build, a data-driven, technology focused podcast digging into how
0:05
building owners and operators use technology to manage the
0:08
full life cycle of their projects.
0:10
I'm Nick Caravella, your short runner and curator, and I'm
0:13
excited you're joining us today to talk about the people driving
0:15
the change in our industry. Yeah, well, today I want to
0:19
welcome you back to another exciting episode where I'm
0:21
joined by Christy Hunt and Matt Caldwell from Hyper Solutions.
0:25
Welcome back, Matt. How's it going?
0:27
It's going good. It's been a great two weeks
0:29
since our last discussion. Studio 2 weeks, it's 20 minute
0:32
break in between recordings, but that's where the magic happens.
0:36
So I know we're going to be jumping back in.
0:39
I wonder, you know, do you want to tease up the audience to
0:42
share a little bit about you real quick so we can get back
0:44
into the races? Yeah.
0:45
So, I mean, I expanded upon this in Episode 1, but Matt Caldwell,
0:50
I've been in the data center space for 20 years, spent the
0:53
bulk of my career with Schneider Electric, did four years at
0:55
Siemens, managing Heal at Packard, worked at Train for
0:59
four years, and for the past eight months, I've been at Hyper
1:02
Solutions. Thanks.
1:04
And so Hyper Solutions kind of fits in a really unique niche.
1:07
Last episode we talked a lot about just the data center
1:11
industry, where it is to get today and where it came from as
1:14
far as 20 years back when you first ran in the industry and
1:17
the recurring themes as you understood, it really came out
1:20
to be around scalability and and transparency because we went
1:26
from talking about kilowatt racks all the way up to MW racks
1:30
and now up to GW data centers and just the whole shift that
1:34
occurs between all of that. So I'm curious and and maybe
1:38
this gives us the right jumping end point, how has this changed
1:42
driving supply chain constraints as it applies back to hyper and
1:46
the the value you guys create? And what does the supply model
1:51
really look like for the developers and hyper scalers in
1:55
the space looking to get the equipment to power their
1:58
building? Yeah.
2:00
And I touched a little bit about the the hyper business model in
2:03
Episode 1. So you can go back and take a
2:06
look at that. But what that gives us the
2:09
ability to do is scale in a way that other manufacturers that
2:14
are constrained by the fault 4 walls of the facility are not
2:17
able to do right. And if we're looking at the
2:21
rapid expansion of cloud computing and AI, which the
2:25
forecasts are just astronomical, the old way of doing it just is
2:29
not going to keep up with demand.
2:31
You can expand capacity by putting a shovel in the ground
2:35
in a capital expense, but there's going to be limits there
2:40
and you're really limited by the four walls of the building.
2:42
Whereas the hyper solutions business model is scalable and
2:48
doesn't have the same constraints that a traditional
2:51
manufacturing model would have. One of the things that we talked
2:54
about on on our prep was about up leveling manufacturing.
2:59
Can you explain what that means? Yes.
3:02
So I think what we're doing in hyper solutions currently is
3:05
going to be extrapolated across the entire data center industry.
3:10
What I was talking about as far as transparency and being able
3:13
to see where, where the equipment is in the production
3:17
process, having a high degree of certainty that equipment's going
3:20
to arrive when it was it you were told it was going to arrive
3:24
is going to become table stakes in in the years to come.
3:28
And same thing on the on the scale of scalability aspect.
3:32
I think the industry is going to start to realize that
3:35
traditional manufacturing approaches where you're, you're
3:38
taking a single order, you're trying to cherry pick a bill
3:41
slot out of existing manufacturing line isn't going
3:45
to serve the customers that are building GW data centers on the
3:49
scale of manufacturing has to ramp up to meet the demand.
3:54
Right now the bottleneck is on the power side, but once we get
3:58
through that, once the industry shifts from, you know, utility
4:04
power to on site generation and eventually nuclear, the
4:07
bottleneck is quickly going to become, OK, well, how do we get
4:11
it? The massive amount of equipment
4:13
that we need to operate these data centers, or I should say to
4:17
build these data centers. And that's that's where I think
4:21
some of the things that the hyper is doing are going to need
4:24
to be implemented across the industry in order for us as an
4:28
industry to, to, to meet the demand.
4:31
So you know, what you're talking about is this kind of
4:33
decentralized model and, and you know what the power is, right,
4:38
with the power of power, funny with the power of that is.
4:42
So how does, how does a decentralized model improve
4:45
speed, reliability and scale? And then maybe touch on some of
4:48
the challenges and and why transparency is so important and
4:52
maybe even almost a superpower. Yeah.
4:54
So, so staying on the on the scale and the speed aspect at
5:00
this point in the data center life cycle or industry life
5:06
cycle, I should say, if you're still talking about lead times
5:10
and you're just bidding stuff out, you've missed the boat,
5:13
whether it be hyper, another partner you really need to
5:17
partner with, with companies ahead of time.
5:20
Look at long term capacity agreements, look at your
5:23
pipeline holistically over the next 2-3, five years and start
5:27
talking to manufacturers well ahead of time.
5:30
I've seen this with smaller data centers.
5:33
I shouldn't say smaller that that's probably not the the
5:36
right term. Newer data centers that are
5:38
getting into the space and looking at it as a traditional
5:41
construction project and they get to the point where their
5:45
contractor is looking to procure equipment and it's a week and a
5:49
half RFP cycle and they're putting it out on the street and
5:53
they're not getting the responses that they want.
5:56
And the reason reason is all the players that have been in the
5:59
space for a while have long term relationships with the major,
6:03
major suppliers in this space and know how to forecast their
6:09
projects appropriately, set expectations with the
6:13
manufacturers and have a two way relationship.
6:17
The days of, hey, I'm building a large data center, so
6:21
everybody's going to line up and, and meet my demands are
6:25
over, right? I, I, when I was with a major
6:30
OEM for, for chillers, I was sitting across the table eating
6:33
a steak with a customer and I, I jokingly said, shouldn't you be
6:37
buying me the steak 'cause you need my production spots more
6:43
than I need your PO? And it was kind of tongue in
6:45
cheek. I mean it, you know, obviously
6:47
as the OEM sales guy, like you're expected to pick up the
6:50
check, but like it wasn't, it wasn't lost on him either.
6:54
Like he it's, it's true, right. Like if you're working for a
6:58
major OEM, getting a PO is, is easy, that that's the easy part,
7:02
getting the company to build the equipment, getting securing
7:06
those build slots internally. That's where the knives come
7:09
out. And and you got to kind of fight
7:11
for, for that capacity for your customer because you're talking
7:15
about a fixed, fixed pie and everybody's looking for a slice
7:18
of it. So maybe I talked to her in
7:20
circles there for a second. But back-to-back to your
7:23
original point of, you know, speed and scale, you need to be
7:29
looking at this from, you know, a, a wider lens and looking at
7:33
it from a relationship standpoint, which drives out the
7:36
the lead time conversation. It's more of a capacity
7:39
discussion. And then on the scale side,
7:42
that's where the business model of hyper solutions has an
7:46
advantage of not being constrained by the four walls by
7:50
being able to replicate our business model over and over
7:54
again and kind of recap what we went into detail in the first
7:58
episode. We have a private manufacturing
8:01
ecosystem. It's built on hyperspace, which
8:03
is a digital platform and then that process is implemented at
8:10
various distributed manufacturing facilities.
8:14
Now the scale aspect of it, if we were to come close to
8:19
reaching the capacity of the 9 column satellites that we have
8:22
today, we have we're in some phase of pre qualification on 49
8:28
additional manufacturing facilities.
8:31
So instead of when we reach the limit of our capacity or if a
8:36
customer comes to us and says, hey, I've got this large demand
8:39
for equipment coming in the next 3-6 months, 12 months, we can
8:44
scale up additional facilities. It takes us two or three months
8:48
to scale up an additional factory.
8:51
So that means going in training the facilities, training the,
8:55
the staff on our process, making sure everything's up to speed,
8:59
going in there, doing a, a, a practice run to make sure that
9:02
everything's going good. And then having hyper personnel
9:05
on step on hand to oversee the process and the quality control.
9:09
Because at the end of the day, Hyper's responsible for the
9:12
process, the quality and the service associated with our
9:16
equipment. It's not that we're offloading
9:19
responsibility to a third party contract manufacturer.
9:23
These are all Hyper facilities that just happen to be owned by
9:28
basically a franchise a a manufacturing partner of hypers.
9:35
And I like that distinction because I think that was
9:37
something that I was maybe even getting a little confused myself
9:40
of like where, you know, where hyper starts and stops And and I
9:43
love that, you know, you kind of be where the buck stops if
9:47
there's any questions around the equipment pipeline, where things
9:50
are at, how they're being manufactured, ensuring the
9:52
quality and ensuring that they're get delivered.
9:54
And, you know, I think it was really cool when you started to
9:57
go. He gave the Amazon analogy.
9:59
So if, like, I could track my toothpaste and I can know
10:01
exactly where it is, and then I can almost get too much
10:04
information. And you touched on this, too,
10:06
where like balancing the difference between transparency
10:09
and noise. And I can think of the Amazon
10:12
order where, like, it starts in Kansas, flies over Denver, goes
10:15
to California, and then it gets delivered to me in Denver.
10:17
And then I start asking questions like why this
10:19
happened. I think that's the aspect of
10:21
where transparency probably creates noise, but how has it
10:26
been to really start to create transparency into this equipment
10:30
journey and what has that changed about owners
10:33
expectations and trust that maybe is different than working
10:37
with like a large OEA? Yeah.
10:39
I mean, that's a huge differentiator for hyper, right?
10:42
That, that hyper space model is so foreign to the data center
10:46
industry that our clients that are utilizing it and see it are
10:51
like, I, I want this not only for, for your equipment, but why
10:55
isn't everybody doing this right now?
10:57
Like I want to know, I want visibility into where it's at in
11:01
the production, production cycle, right?
11:03
And that level of visibility to and trust and assurance that not
11:08
only are you telling me it's going to arrive on site, but you
11:10
can tell me exactly where it's at in the production process.
11:13
So you know, this, this lot of equipments and you know, being
11:17
manufactured, this unit is that process, you know, stage 3 in
11:21
the production process like that level of transparency and being
11:26
able to tell them, hey, you know, if that piece of equipment
11:29
cannot move to the next step in the process that gets flagged in
11:33
our system, we see that as red, OK, we need to go look at this.
11:38
Why, why couldn't it go to the next stage?
11:41
We look at rectifying that, but that is transparent to our
11:43
clients as well. So we can tell them, hey, you
11:47
know, this is what happened. Steve dropped the wrench.
11:49
It, you know, screwed something up.
11:50
We had additional part on hand. This is our process to
11:53
rectifying it. It's not going to affect your
11:55
your delivery time. And yeah, maybe the first,
11:59
second, third time they, they call them, they say, hey, why
12:01
did it turn red? But eventually there's a level
12:03
of trust and and comfort in knowing that even if there's a
12:07
hiccup along the way that, you know, I like what you said
12:09
there. The buck stops, stops with hyper
12:12
like we we're going to own that. That's not us sending this out
12:16
to a third party and and relying on them and their process to
12:20
figure out how to build it. We own the supply chain for all
12:24
the critical components, which that's an area we should
12:27
probably dig into, right? Because there's a misnomer in
12:29
the industry that the reason that production or the
12:35
bottleneck in the production of electrical equipment is through
12:38
the critical components manufacturers like the Breakers.
12:42
Let's just key in on that, right?
12:44
That is not the bottleneck. It's in final assembly.
12:47
You can get Breakers. And the unique thing about Hyper
12:51
is that all of our equipment is UL listed with multiple
12:54
manufacturers. So let's just say as an example,
12:58
a customer has a preference for Square D Breakers and we
13:03
standardize on Square D Breakers.
13:04
We deliver 100 PD US with Square D Breakers and then all of a
13:08
sudden there's an issue. Let's just say for whatever
13:11
reason Schneider can't deliver Breakers anymore.
13:13
Unlikely to happen. But if that were to happen, we
13:16
could go to the customer and say we can source our Breakers from
13:20
Siemens, swap out that component, deliver the next
13:23
batch of PD US with Siemens Breakers without missing a step,
13:28
without impacting the cost, without impacting the delivery
13:32
dates. We can pivot in a way that if
13:34
you're going to OEM, they're locked into one manufacturer of
13:40
Breakers, whereas we have agility in our our ability to
13:44
swap between components and all of our equipment is has
13:48
redundancy built into its design.
13:50
I, I think you just really hit like the magic of the sauce
13:53
there with like flexibility is part of how we can move as
13:57
quickly as we, we can. And it's not, you know, I see
14:01
this on the equipment level, which is how you said of like,
14:05
you know, hey, this part's not coming through, but we can go
14:07
and solve this problem with this.
14:09
But then there's also just from the managing the capital
14:12
construction, you know, there's other delays out there.
14:15
It's not all equipment related. And so when I got site A and
14:18
site A has fallen behind, but you know what, everything was
14:20
going to get delivered on time there, but site B is actually
14:24
ahead of schedule and I need a free up equipment.
14:27
It starts to really allow owners the creative flexibility to plug
14:31
and play and, and move dots around.
14:34
And I think that this, you know, honestly, with the, the red,
14:37
yellow, green light that was so exciting is that traditionally
14:40
when you find out something's not going to be on site, it's
14:43
the day you're on site expecting the darn thing.
14:45
And then it's like, well, why can't I mobilize my careers?
14:47
I got a whole bunch of people here like what the heck is going
14:50
on versus the proactive communication that can allow the
14:53
management teams and the leadership on the construction
14:55
teams and the owner teams to say, OK, this is going to shift.
14:59
How does this impact my critical path?
15:01
And if it does, what does that mean for this project?
15:05
And I'm looking elevating up to the program level.
15:07
OK, stuff's going to slip on site a site BS behind a neat
15:11
stuff. Let's do that shift.
15:12
Let's do that pivot. And I feel like it allows for
15:16
such executive flexibility with managing these huge, huge spends
15:21
on all of this equipment to help keep projects moving as
15:25
seamlessly as possible. Is that, you know, is that what
15:29
you're seeing where you're ending up getting into this like
15:31
juggle that's happening with these big orders that you have?
15:35
Yes. And something we were just
15:37
talking about as far as agility sparks something in my head.
15:40
So I want to pivot a little bit away from the construction side
15:43
onto the design side. So these companies or data
15:47
center builders have been conditioned to design their data
15:52
centers around the equipment that's available today, right?
15:55
So they're going in, they're saying what equipment's
15:58
available in the market, I'm going to design around what
16:01
equipment I can get on site as opposed to if I had a blank
16:05
sheet of paper, how would I design this, right?
16:07
And that's another unique aspect to Hyper Solutions, which is
16:11
we're having conversations with clients that are saying if I had
16:16
a blank sheet of paper, I would not be designing this the way
16:20
that I am designing it today around the off the shelf
16:24
equipment that's available today.
16:26
What I would do is I design it this way, this way, this way we
16:30
have the unique ability to take something from a napkin sketch
16:35
to AUL listed piece of equipment and build it at scale in a
16:41
matter of 20 to 30 weeks. And we've done this with a
16:44
client who said I cannot get, you know, 2-3 MW generators on
16:50
site in the timeline that I need.
16:52
I could use 500 kba generators and I just need a combiner box.
16:57
But the combiner box that's available today would be the
17:00
size of a semi truck. I need it in this unique
17:03
footprint and I need it in the left to right configuration for,
17:07
for the load bank testing or whatever else.
17:10
And we took that and designed it into AUL listed piece of
17:14
equipment in 20 something weeks and was able to basically put a
17:20
unique skew on that for that customer that they can buy that
17:24
100 times over and over and over again.
17:26
So that doesn't work with a customer that's, you know, a
17:31
legacy enterprise site that's, you know, a healthcare, the
17:35
hospital, whatever that has a five MW data center and they
17:39
just need one piece of equipment.
17:41
If we're going to do that, if we're going to put the
17:42
engineering resources and effort into it, it needs to be
17:46
something that's being built at scale.
17:48
And that's where our target customer comes in of, you know,
17:52
the hyper scalers and like that next level of wholesale
17:55
collocation clients that we're working with because we're
17:57
willing to put in the time and effort on the front end to
18:00
partner with those clients. Design A transformer, panel,
18:05
board, genset, combiner, box, whatever, whatever they can
18:08
dream up, right? Like they just take a a pencil
18:11
and start sketching out of one line and a general idea.
18:15
I just got another example of that as well.
18:17
But yeah. And then and then build that out
18:20
at scale, right? Trying to think of how much I
18:24
can share here without violating an NDA.
18:26
But we had a, we had a client at our manufacturing facility in
18:30
Grand Rapids. They shared with us that they
18:33
had an issue during commissioning.
18:35
And they're like, you know, it'd be really nice if the PDU
18:38
shipped with this. And I can't get really into the
18:40
details of what exactly that was.
18:42
But within 48 hours, our engineering team had the, you
18:47
know, drawings of what this might look like and in back in
18:51
front of that client. And they're like, where did you
18:54
come up with this? And we're like, we just designed
18:56
it in 48 hours and we could ship this with every PDU.
19:00
And they're like, and I know because I've worked at large
19:03
OEMs, like just getting the resources in house at a large
19:07
OEM, you have to go through a whole business case on, you
19:10
know, why should we dedicate these, these engineering
19:12
resources for this? Put it, you know, put a team
19:15
together, have meetings about meetings about meetings about
19:17
eventually maybe making a decision.
19:19
Whereas working for a smaller company like Hyper, we are super
19:22
agile. Hey, is this in the best
19:25
interest of the customer? Great, just go do it.
19:28
Take whatever resources you need, do it.
19:30
Let's let's go help the customer.
19:32
And that's, that's one of our core pillars of hyper.
19:35
It's it's do the right thing always.
19:36
But it goes back to my early days at APC, which was the, the
19:41
rank order in which you're supposed to make a decision.
19:44
And that was isn't in the best interest of the customer, the
19:47
company, the team, and then yourself in that order, right.
19:50
And I really feel like what we're doing now is embodies
19:54
that, that vision of, you know, putting the customer at the
19:57
center of everything that we do. And that's where like it, it's
20:01
man you, you told my Horror Story of how the the business
20:05
cases and the myriad of meetings, just a ultimately a
20:08
decision because these are these are these are questions that
20:13
require just a little bit of creativity and the right people
20:17
in the room to be able to make the right decision to keep
20:19
things moving. And we get caught up and the the
20:23
weeds of running the business to avoid ourselves for a like.
20:27
From being able to answer that solution, that problem right
20:30
away and where, you know, not to oversimplify, but you know, I
20:34
think there's beauty and Lego and understanding the different
20:38
parts and how they connect and how they block and stack and
20:40
then saying, listen, you're all master builders.
20:43
Here are the tools to me, like with the story you're telling me
20:47
gives me like Hyper has this like Lego like approach to no.
20:51
We understand all the different parts in pieces and how they fit
20:54
together and you understand your problem.
20:56
And together we can really quickly iterate, build out a new
20:59
solution and even start to deploy it and as fast as the 20
21:03
something weeks, as you said, which is incredible.
21:06
And it really highlights, I think the pace at which this
21:09
industry is moving and the level of creativity and opportunity
21:13
there is for a smaller, smaller players, if you will, to be
21:18
really making a massive difference in this market, which
21:22
is so cool. To build on that Lego block
21:25
approach, a lot of times there's a divide between engineering and
21:30
manufacturing, right. You have the engineers designing
21:34
it and then they hand that off to manufacturing to build it.
21:37
And in Hyper those two functions are much more integrated.
21:41
We are designing to build. And what I can give you a
21:45
specific example of that is we do all of our equipment with bus
21:49
bar construction that can only be assembled a certain way.
21:53
So that when it goes out and it's being manufactured or, or
21:57
assembled by our our manufacturing partners, you
22:00
can't bend a, a piece of conduit and force it into somewhere that
22:05
it's not meant to go. All those bus bars can only be
22:08
assembled a certain way. And that has that certain Lego
22:11
block approach that like not only do we have a standard
22:14
process for building it, but it's designed in a way that it
22:18
cannot really be built in any other way other than the right
22:22
way. So I want to shift a little bit
22:24
and get your crystal ball out. 15 years ago, we wouldn't have
22:28
been talking, We wouldn't have imagined sharing cat videos of,
22:31
you know, cats with laser beams as as a thing, you know, But
22:34
there's so there's also other shifts in the in the industry,
22:37
things like liquid cooling and the future of modular delivery
22:42
talked about nuclear. Where do you think the industry
22:45
is headed and where and how? What things need to change in
22:50
order to kind of keep pace, especially with AI driven
22:53
demand? Yeah, so, so let me, before I
22:56
get my crystal ball out and look forward, I'm going to rewind 3
23:00
years. I'm at a DICE event in New York
23:03
and there's a panel discussion on liquid cooling.
23:06
And the five panelists are all very bearish on liquid cooling.
23:11
They're saying, yeah, you know, it's the question was when is
23:15
liquid cooling going to be adopted at at scale?
23:20
And the answers were 15 years, 20 years, 30 years down the
23:25
road. So at the end of the panel
23:28
discussion, the question, you know, and question and answer
23:31
session, they're like, does anybody have a question?
23:33
I raise my hand and I get the mic and I go, first of all, I'm
23:37
going to take the under on all your bets there from a timeline
23:40
perspective. But let me rephrase that
23:43
question. What is the rack density that is
23:47
going to force the industry to go to liquid cooling?
23:51
And the answers were 40 kW, 50 kW, 70 kW, whatever the answers
23:57
were where you reach the limits of physics on when or how a data
24:02
center could actually be be cooled with there.
24:04
And obviously that was probably just before, you know, the AI
24:10
explosion and all of a sudden Nvidia's throwing out these
24:14
crazy numbers for racks. So, you know, within a year of
24:18
that discussion, 100 kW was readily accepted as the, the,
24:23
you know, benchmark of OK, we're we got a design around for for
24:26
at least 100 kW per rack. And I think the last NVIDIA
24:33
announcement a couple months ago was their new design is 600 kW
24:38
per rack. And I've heard rumors of 1 mega
24:43
MW racks coming down the road. I think there's a
24:46
misunderstanding on on why racks need to be this highly dense,
24:52
right? There's this perception and I, I
24:55
got to admit like, you know up until fairly recently I was
25:00
like, why can't they just spread the racks out like what what's
25:03
the problem like if you got a one MW rack, why couldn't you
25:06
just make 1000 kW racks like that?
25:10
That seems pretty simple, right? The the the challenge is that
25:14
these GPU chips act as a single solitary unit and there's
25:20
latency issues with having the chips too far apart.
25:23
So they all need to be in a single location in order for
25:27
them to work in in in the way that they are intended to.
25:32
I've heard, and this is not based on any sort of actual
25:36
research, this is just something I heard secondhand, whatever,
25:39
but those connections between the the GPU chips are currently
25:43
copper. If you could figure out a way to
25:45
turn those connections into fiber, you may be able to spread
25:48
out the the chips and the heat load a little bit more without
25:52
the latency issues of, you know, the copper connections between
25:55
the chips. That statement is so far out
25:58
over my skis. If anybody is listening to this
26:02
and they want to refute that comment and tell me exactly
26:05
what's going to actually happen, I would love to hear some actual
26:08
data on what's, what's going on there.
26:10
I'm not, I'm not hanging my hat and my ego or reputation on that
26:14
comment. But that just seems like
26:16
something that if somebody can solve for that, if that's the
26:20
reason why we're getting these super high density racks and we
26:23
could figure out a way to distribute the heat load a
26:26
little bit more that may. I mean, we're probably never
26:29
gonna go back to 10 kW racks for an AI training application.
26:33
But even if we're, we're approaching one MW racks and we
26:36
could spread that out to 1000 kW racks.
26:40
That makes sense in my mind. So that that is my unsolicited,
26:47
I guess it was solicited uneducated thought on, on, on
26:51
where that may go. But overall, I mean, I if you, I
26:54
just listened to an interview with Andy Jazzy on on Jim
26:59
Cramer. I think it aired last night.
27:01
And he was saying even with the $100 billion a year investment
27:06
that they're making into data centers, they're not even
27:09
scratching the surface of the demand.
27:10
The demand is outstripping supply across the industry.
27:13
So when I hear people talking about is this another
27:20
aioranother.com dot bomb type situation where, you know, we're
27:26
getting all ramped up here and we're all excited and eventually
27:29
the bottom is going to fall out, I actually think this is the
27:31
exact opposite. And here's my reasoning behind
27:34
that. In the early 2000s where we had
27:37
pets.com and, and all this excitement around the Internet
27:41
and, and you know, Internet based companies, there was an
27:45
overbuilt overbuild of the infrastructure, right?
27:49
You're building a data Center for the next 30 years and
27:53
anticipating the demand for the entire life cycle of that demand
27:57
of that data center. So pets.com builds A10 MW data
28:01
center and utilizes 50 kW of that.
28:05
And then they basically flame out and, and everything falls
28:09
apart. But they built that
28:10
infrastructure, they built the, the power of the cooling, the
28:13
fiber network, which became the backbone of, of the Internet and
28:17
everything that was built on it. But it, there was a lag.
28:20
There was 5-10 years where we're just basically consuming that
28:25
that build that got that got started in you know, 2000, 2001
28:30
where we're at today is the exact opposite because the the
28:34
vacancy rates across the data center industry are low single
28:38
digits. So what that means is that if
28:42
you are thinking about building a data center and you have land
28:45
and power and fiber that'll be pre leased before you put a
28:48
shovel on the ground. People are, if you can validate
28:51
a business model that we can build this data center and you
28:54
have credibility that it can be built in the next 18 months,
28:58
somebody's going to grab that to the point that it's almost
29:03
impossible to build a retail collocation data center now
29:07
because I was listening to a panel discussion with for one of
29:12
the actually said it on stage. So it's public.
29:14
It's age 5. It was David Dunn and he was
29:17
saying that they're building A50 MW data center in Virginia with
29:22
the intent to sublease it out in say 5 MW chunks.
29:25
I forget the exact number, but you said before they were able
29:29
to finish construction, one of the big hyperscalers came in and
29:32
gave them a single lease for the entire facility.
29:34
And even wholesale Co location companies are, you know,
29:38
intending to build something that they're going to operate.
29:41
And once they have a powered shell, a hyperscaler comes in
29:44
and just buys the asset and just builds it out themselves.
29:46
So it's, it's really an arms race right now.
29:49
And we're even getting to the point that this is becoming a
29:53
international. I, I mean, it's almost like a
29:56
Sputnik moment at the, at this point in time where, you know,
30:00
it's US versus China when it comes to the AI arms race.
30:04
And Europe's kind of they're, they're, they're trying to,
30:06
trying to play. We'll see if they can catch up.
30:08
But really, I mean, it's going to come down to a fundamental,
30:14
you know, what does the world look like in 20 years?
30:17
Everybody's already got enough, enough nuclear bombs to end the
30:20
world 10 times over. So the next pivot point in this
30:25
global posturing for who's who's going to be the major player in
30:30
the next couple decades is really going to be based upon
30:33
who owns the data, who owns the, the AI, the capacity to compete
30:40
globally. That arms race is not likely to
30:44
end anytime soon. It's it's going to be private
30:47
investment, it's going to be public investment.
30:49
It's going to be public resources and regulation
30:53
aligning with the goal of, you know, building this AI
30:58
infrastructure that's going to position the US versus other
31:02
international adversaries in a way that keeps us competitive.
31:06
Now I'm I'm just going way off on a tangent here, but I'll keep
31:09
going. It's a great tangent.
31:11
Then then we got to look at the, you know, the social
31:14
implications of this and, and my, my Co founder of hyper
31:20
Dynastrato loves to use the phrase universal basic AI
31:24
instead of universal basic income, right.
31:27
And the, the idea there is that if AI is going to replace a
31:31
number of jobs, instead of providing universal basic
31:37
income, just ensure that AI is in the hands of everybody who is
31:43
displaced by it, right? And what that, what that really
31:48
means is all those jobs, all those low level, you know, white
31:54
collar jobs that are being displaced by AI, put the tools
31:58
in the hand of those individuals to level up, to start a
32:02
business, to operate at a higher level within their organization.
32:06
Enable them to be contributing members of society, even though
32:10
though that administrative task of entering data into a
32:14
spreadsheet 8 hours a day no longer exists.
32:17
I feel like this crystal ball that I'm looking in.
32:20
Thank you for that analogy. I'm, I'm getting really deep
32:22
there, you know, 10 years from now, the idea of sitting there
32:27
and manually entering data or manually doing some of these
32:30
mind numbing processes, people are going to look back and be
32:34
like, I can't believe we had people doing that, that that is
32:37
such a waste of talent and creativity when they could be
32:42
creating more cat videos with lasers.
32:44
I mean, that's, that's really the highest level of human
32:47
productivity I can think of. But.
32:49
Never enough, never enough. Oh my goodness, Yeah, no.
32:53
And there's a couple of points I want to just slide back to and,
32:56
and number one, yeah, there's never enough cats and laser
32:59
beams. But you, you mentioned, you
33:03
know, AI, the AI bubble and, and not really acknowledging it.
33:08
And I, I think there's a, there's a part of that to unpack
33:10
where I do think there is an AI bubble, but it's not about the
33:15
infrastructure at all. The AI bubble, as I see it, is
33:19
really in all these like what we call AI wrapper solutions that
33:22
are out there. Everybody's saying like, oh, I
33:24
can do this. And now it's an AI tool.
33:27
And what I've found from leveraging some of those AI
33:31
tools and leveraging, say, ChatGPT, if you will, I can
33:35
build myself an AI just as quickly as these companies are
33:39
to do the task that they're selling.
33:41
And so I think the air is going to get really thin really
33:45
quickly on these people who are building these like very purpose
33:49
built, general solutions where there's then going to be people
33:53
who are experts in activities who can build a tailored GPT
33:58
very quickly, very easily, and can do it today to meet their
34:03
needs. And a great example is how we
34:05
even use it on this podcast. And I could go out and use AI
34:08
tools that are off the shelf and pay someone who built their AI
34:12
solution. I sat for a few days crafting A
34:15
GPT back and forth. I'm going to need to pay a dime
34:19
beyond what my subscription is for open AI to do that.
34:22
And so that's where the bubble to me lies is these folks who
34:26
are making a quick one and and God bless them for making a
34:29
quick one. But their model I do not feel is
34:32
going to be sustainable because everyone is going to be able on
34:37
under this universal AI idea. Everyone's going to be able to
34:40
generate a tool. And so to me, AI is like the
34:44
next level of a public utility. I do agree, like everybody, if
34:47
we're going to have world domination of AI race and US is
34:51
going to have the best AI in the world, then everybody in the US,
34:55
especially you being the democratic nation that we are,
34:58
should have access to that AI so we can all be creating value for
35:01
our society. And I think that that's, you
35:03
know, that's really cool. So I see like, yeah,
35:05
infrastructure that's not going anywhere.
35:07
In fact, we got to be building more yesterday.
35:10
But the AI bubble to me is really around the AI wrappers as
35:13
a service kind of thing. Or to me they're they're just
35:16
building generic tools that can be deeply specialized.
35:19
And the deeply specialized is always going to win in my head.
35:22
Yeah, I'm going to put in my my strategy hat from Business
35:26
School and say that the business model that you're talking about
35:29
doesn't really have any sort of mode around it.
35:31
They're, they're basically doing something that people could do,
35:36
but they're doing it more efficiently because they're
35:38
utilizing AI tools. And eventually people are going
35:40
to realize like I, I can do that on my own without your help.
35:43
And that's going to fall apart. Back to the infrastructure piece
35:47
of it. The mode around Amazon,
35:50
Microsoft, Meta and Google is a mile wide and a mile deep.
35:55
Like in order to compete with these guys that are investing
35:59
billions of dollars into data centers like you don't there's,
36:04
it's almost impossible to reach the scale that you could even
36:08
compete with that. I think core weave is probably
36:11
the most notable exception of somebody that's taking a stab at
36:16
building that scale of, of infrastructure that you could
36:20
even play in the sandbox with, with an Amazon or a Microsoft.
36:24
And and they're never going to be the same size, but they may
36:27
get a seat at the kids table at some point.
36:30
Yeah. And I think that's too like I'm
36:32
glad you said, it's like there are companies that are
36:36
positioned to to re point and re pivot.
36:39
There was so much energy going out of the Bitcoin space and
36:42
they were cannibalizing GPU usage.
36:44
And so now they're sitting on this mountain, the GPU's that
36:47
they can repurpose for AI and we're watching that happen where
36:49
everybody's kind of re pointing. So I'm really curious to see, I
36:52
don't want to go down a speculative journey of Bitcoin,
36:55
curious to see how that's going to shape out.
36:57
But it is just, it's so fascinating to think about then
37:01
the consolidation of power to these businesses and then how
37:04
they have to work with these sovereign entities of whether it
37:08
be the States, whether it be China, UAE and so on.
37:12
And how they then get it gets round right down to the meat of
37:15
it is how do we, you know, rid out of public political podcast.
37:19
But how could we start to talk about data privacy, data
37:22
control, all the things that are really important on a personal,
37:27
corporate and even public level. And making sure that, you know,
37:33
AI for good is ultimately kind of what we're trying to achieve
37:37
because with all of that data and all of that compute in the
37:40
hands of just anybody is a lot of power.
37:44
And that's very interesting. And it's a double edged sword
37:47
because on one side, you're entirely correct.
37:51
There needs to be regulation, data privacy, everything else.
37:54
But if the pendulum swings too far to that direction on that
37:58
side, now you're getting left behind.
38:00
And I think, you know, US is probably somewhere in the
38:03
middle. I think the European Union is
38:06
putting enough regulation in place that's squandering an
38:10
opportunity to to leverage AI, whereas China's the other end of
38:15
the spectrum, basically anything goes state operated there.
38:19
They're going to go balls out and you may have to add to that,
38:23
but they're, they're, they're going to, they're going to go
38:26
full throttle on the because there, there is no personal
38:31
freedom over there. They're just going to do
38:33
everything in the best interest of the state and it's all
38:35
operated that way. So as a democracy, as you know,
38:39
the United States, we got to find some sort of middle ground
38:43
that I think is, you know, stakeholders in the process
38:46
having a voice and and really it comes down to educating the
38:51
masses on what it is we're actually doing.
38:54
Because you go into these, you go to Northern Virginia, go to
38:58
data center event there. And you know, there's picket
39:01
lines and not in my backyard and people taking pictures of their
39:04
signs of posting it on Facebook. And you know, every everybody at
39:07
the conference recognizes the irony of that.
39:10
But you know, it's, it's also a fair, fair discussion where if
39:16
we're just going in and we're dropping another 500 MW data
39:20
center in somebody's backyard, that has an impact to them.
39:22
There's, there's noise, there's disruption, there's, there's
39:26
issues there. And I, I think that especially
39:30
as in you, you would Christy, you'd ask like my crystal ball
39:32
where I see things going forward, we're we're going to
39:34
end up nuclear. That's that's a given.
39:36
It's a it's a when, not if. I could see a world where a
39:41
Microsoft or an Amazon builds a nuclear facility and uses the
39:49
waste heat from the data Center for productive uses in that
39:53
community around it provides free Internet for that
39:56
community, maybe even provides financial incentives.
40:00
If you're within a certain distance from the reactor and
40:04
data center, you get some sort of subsidy.
40:06
If you're further out it, it comes less and less, but really
40:10
incorporating that data center into the community and endearing
40:14
themselves to the population. As opposed to, you know, just
40:20
going in and dropping it in and asking for forgiveness later or
40:23
just kind of shoehorning it in there.
40:25
And, you know, that's reaching the outer points of my or my
40:30
expertise in the data center space.
40:32
But I see enough of it that, you know, these data center
40:34
developers are are are running it or that's part of their
40:39
business at this point. Jeff Hub Hughes from DC Blocks
40:43
was speaking at on than in Atlanta and he was saying that a
40:48
good portion of his time is spent talking to the local
40:52
community boards and politicians and everything else to educate
40:56
them and, you know, dissuading some of the the negative
41:01
pressure on on on data centers. And while, while it's it's fresh
41:05
in my mind, his keynote speech was incredible because he was
41:08
talking about, he gave real numbers from 2021 to 2025 and
41:14
his prediction at the beginning of the year and how far he
41:17
missed it by the end of the year on the upside.
41:21
And then he'd recalibrate it for the following year using the end
41:25
of the year from the previous year and again, miss it again.
41:27
And he's like, I've missed it the past four years, five years.
41:31
The data center demand has outstripped their prediction
41:34
every single year. You know, they're talking about
41:36
the Southeast particularly, but there's a boom across the board
41:41
and particularly in that region. Yeah, no.
41:43
And I can't remember if I've ever said it on this podcast,
41:45
but I'll say it's like the 8 because I'm with you entirely.
41:47
Like nuclear went from being uncertain maybe 8 months ago to
41:51
like, if anything, it definite it's happening and everybody's
41:54
trying to figure out the regulatory around it.
41:56
But to me, it's like this is we're, we're fast on the track,
42:00
building on the moon. There is no shortage of where
42:03
this is going to stop. And I feel comfortable saying
42:07
statements like that and knowing that as we continue to grow and
42:11
as there continues to be demand, there's going to be a shortage
42:14
of land. We're going to end up innovating
42:15
the way we transfer information. We're going to end up innovating
42:18
the way we cool. We're going to end up innovating
42:20
a whole heck of a lot of things that almost anything is
42:23
possible. I mean, we're building
42:24
underwater, probably going to, you know, build right next to
42:26
Atlanta. So I hope they don't have
42:27
permits or any regulatory agencies down there, but we'll
42:30
find out. It's an exciting market that
42:33
it's creating such exciting challenges and it's moving so
42:37
quickly. And what I love is just worth
42:40
working with these teams. They're so collaborative.
42:42
Everybody is really coming into these rooms truly understanding
42:46
the deep amount of expertise everybody brings.
42:48
And it, it feels like we're really all working together in
42:51
ways that sometimes different than working with like a, a
42:55
hospital client who like, they know what they want and that's
42:58
the only thing that they want versus, hey, we want you to be
43:02
coming into the room with us figuring out what does it mean
43:05
to deliver a better data center, deliver them more quickly with
43:08
more certainty and transparency and scale.
43:11
That's one of the things I love about the data center space is
43:14
that the pie is growing so rapidly that everybody's retty
43:19
open about the challenges that they're facing and the solutions
43:23
that they're imlementing to resolve those issues because the
43:29
demand outstrips supply to the point that there's really no
43:33
competition. When I was at train, I was
43:36
saying like JCI is not my competitor.
43:39
Our our production capacity is, is the cap.
43:42
And even within the data center space, like there's so much
43:45
demand out there that if they can find the land and power,
43:48
they're going to get the business.
43:50
And then how they build the data center isn't really even the
43:54
secret sauce anymore. So if somebody finds out, oh,
43:57
they're doing it this way and it's slightly better and they
43:59
implement it, it's not going to ruin that company's business
44:02
because that, that differentiator is not going to
44:05
make or break their business. There's a collective idea that
44:10
the rising tide that we're all experiencing is rising, raising
44:14
all boats. And I, I just, I love that about
44:17
this industry, just the collaboration and openness on,
44:21
on innovating because we have to, because it's, we're not
44:24
building data centers the same way we were five years ago.
44:26
That'll that'll continue to change too, right?
44:28
It all evolved. So as you wrap things up, come
44:32
up to the end, what is your final piece of advice for
44:36
owners? And it doesn't necessarily need
44:37
to be about the supply chain. Maybe how they can get more
44:41
videos of cats with laser beams, could be the pineapple thing
44:44
with pizza. But what would your advice be
44:47
for owners? Yes.
44:48
So if I'm an owner, if I'm putting myself in the shoes of
44:52
somebody who's building at scale multiple 100 plus MW data
44:57
centers, I'm I my advice would be to partner early and
45:03
establish those partnerships for the long term.
45:06
And this goes back to something I said, maybe it was in the last
45:09
podcast, But if you, if you start down this path and you
45:14
wait until the very end and you just put out a generic RFP and
45:19
you're looking for bids, you're going to get the, the scraps
45:23
from everybody else who's put in the time and to build a
45:26
relationships with the suppliers.
45:28
And I'm not even saying hyper's the only option here.
45:31
I'm saying go out vet, vet the partners.
45:34
And it doesn't, it doesn't have to be just one either.
45:36
I'm not saying like put all your eggs in one basket.
45:38
I'm saying go out, find 3 suppliers for your mechanical
45:42
equipment, find 3 suppliers for your electrical gear, find 3
45:46
suppliers for your in controls right and develop long term
45:50
relationships with them. Openly share your pipeline,
45:54
openly share who else you're working with, openly share what
45:59
your requirements are as far as lead time and and cost and you
46:04
know everything else around the project.
46:06
And you need to establish those partnerships early on and have
46:10
visibility into what every components capacity is to
46:16
support that project, because what you don't want to do as an
46:18
owner is over commit and not be able to deliver, right.
46:21
So when you're entering into that discussion with let's just
46:25
say it's a single tenant and has built a suit, make sure you've
46:29
got line of sight into availability of all the
46:33
components that are going to go into that.
46:34
You don't have to cut orders ahead of time, but at least have
46:38
visibility to, I have a fairly good idea that when they sign
46:42
this lease, I'm going to be able to get all this equipment in the
46:45
timeline that I've committed to. And then communicate that early
46:48
to your partners that that this is the path that we're going
46:52
down. You can expect, you know, even
46:54
just say LOI or a handshake agreement that hey, we're
46:58
probably going to go with you on this.
47:00
You might want to get your ducks in a row as far as your
47:02
production capacity that you've already told us you have.
47:06
But do not wait until in the 11th hour where the design has
47:11
been 100% buttoned up and you've got an RFP that you throw out on
47:15
the street and you give everybody 14 days to respond to
47:18
it because you're doing yourself a disservice there.
47:21
You're going to you're going to pick the lowest cost or what's
47:25
available out there in the industry and it's going to be a
47:28
challenge for you and you're not going to get the best, best
47:32
equipment and best relationships out of that type of engagement.
47:37
Yeah, it's, you know, I was at a panel Once Upon a time and it
47:40
wasn't even about data centers. It was the one of the Denver
47:44
water directors and said, you know, innovation starts by
47:48
asking who else do I need at the table?
47:50
And I think your advice kind of completely resonates with that
47:54
is like to be able to go fast and you to be able to trust
47:56
others. AI did a really good job with
47:59
the the recap here with their crystal ball of saying, you
48:03
know, transparency isn't about being a feature, It's the
48:06
foundation of trust in infrastructure delivery.
48:09
And I don't think it could have been more spot on.
48:11
So kudos, kudos to the intelligence for kicking that
48:14
one up. But really want to just thank
48:16
you for joining us today. This has been an amazing and a
48:19
lightning conversation. Always enjoy your time together.
48:22
But you know, thanks for being on the show.
48:24
Appreciate you being here. Yeah.
48:25
Thanks for your time and look forward to seeing you at the
48:27
next Data Center conference. Yeah, have a good one.
48:30
So Christy, another one in the books heading into our
48:33
second-half of Season 2. How did you feel today's episode
48:37
went? That was so much fun.
48:39
Matt has a really great presence.
48:42
And then what I really appreciated was he was already
48:46
walking the trail ahead of us, right?
48:49
He knew where we were going with questions asked like, well,
48:51
don't need to ask that. He's already answered it, but
48:54
just, you know, it was great getting the perspective of how
48:57
data centers have have evolved, how quickly it's evolving, and
49:02
just the kind of creativity with transparency and scalability and
49:06
partnership and how important that's going to be as we enter
49:10
our new AI world. Yeah.
49:12
And I think what's it's really cool kind of juxtaposing this
49:15
against Kaylin's episode from last month where, you know, just
49:18
kind of understood her journey and how, you know, Matt kind of
49:23
really under underlines, bolds and, and, and double clicks on
49:26
this of just the amount of opportunity that's in this space
49:29
and the, the amount of room for anybody of any expertise to
49:33
contribute some value. It just is such a, it's a fun
49:36
and exciting industry that has just new challenges everyday.
49:41
And you to me, I think really under underlining what the the
49:46
AI suggested as our recap line of what transparency not being
49:49
just a feature is. It's so interesting to think
49:54
about, you know, what does that mean to build true business
49:58
partnerships in true collaboration instead of this
50:03
coming from the traditional ways of things of like, well, here's
50:06
my responsibility and I'm lucky. Throw the ball over the fence to
50:08
you. There's this this power of being
50:12
transparent that allows everybody to know what's going
50:15
on. You know, this industry is full
50:18
of surprises, but being able to be open and commutative about
50:22
those surprises early usually makes them a lot less painful.
50:26
So a lot to think on coming off of today's episode.
50:30
Excited to continue to get to meet Matt, you know, throughout
50:33
these conversations and throughout these conferences
50:35
that'll be at them, you know, in the second-half of the year.
50:38
Sounds great, I look forward to the next one.
50:40
Yeah. So it's, yeah, it's crystal ball
50:42
being what it is. We are recording the next
50:44
episode to Mario. So we're going to take a little
50:47
bit of a different take. Still talking about
50:49
infrastructure. For those of you who want to
50:51
peek at next next month's episodes, we're going to be
50:54
bringing in Itsic from for M Analytics and talking about
50:58
infrastructure as a whole, where they spent a lot of time
51:02
actually mapping and understanding where
51:05
infrastructure is. And this goes beyond just
51:08
digital infrastructure. Think railroads, think roads,
51:10
bridges, all the different bits that go into the utilities that
51:15
we enjoy on a regular day basis in our modern society.
51:18
So before you get too excited about that, I want to make sure
51:21
I plug Matt and let in his business.
51:23
Be sure to check him out on LinkedIn, check out Hyper
51:26
solutions, learn more about hyperspace.
51:29
Think about what that can mean for your business model.
51:32
And then if you like the episode, share it with a friend,
51:34
Sarah, with someone who else wants to listen and you know,
51:37
like and follow, subscribe, all that kind of fun stuff.
51:39
It helps us know you know what you're getting out of this and
51:41
what you enjoy. And if you want to stay in the
51:44
loop of all the things we're doing over here at Avocado, be
51:47
sure to visit our website and follow the Avocado Toast
51:51
newsletter. You can tell and there's
51:53
shortage of puns, cats or laser beams on this side of the fence.
51:55
So thanks for tuning in today, everyone excited and stay wired
52:00
to build. We'll talk to you next time.
52:01
Have a good one.