Full Transcript for American datacenters might be getting dirtier, but at least they'll be efficient

data centers. Brandon: Right. Tobias Mann: That's not so simple because you can't just shut off the machines. You gotta shut down the workloads. You have to let them get to this to a checkpoint where they can save. And so the the platform is built down to a granular level where you can have this demand signal come in and it automatically figures out what workloads can be shut down right away, which ones have to be kept running, which ones can be migrated to a different data center, kind of chasing the sun, so to speak. Brandon: Mm-hmm. Tobias Mann: in order to keep the grid stable. and and and one of the big benefits that NVIDIA sees in all this is again it comes back to money. I mean it it sounds really great for the you know you and I who are worried about brownouts during a hot summer day, but from NVIDIA's perspective, if the utility knows it can claw back, say 30% of the power that's been allocated to the data center, They might green light say a hundred and fifty megawatt data center build out, knowing if they know if they're not quite sure about say fifty megawatts of that capacity. Brandon: Right. Tobias Mann: Whereas they might not have said, you know, we can give you a hundred, but we can't give you a hundred and fifty. You know, so there's you know, there's that and every Brandon: Yeah. Tobias Mann: kilowatt that gets opened up is a opportunity to drive revenues for NVIDIA. Brandon: Right, right. And of course, you know, I think as you mentioned toward the end of your story, this is not also an entirely benevolent move on NVIDIA's part either, right? But this is creating a lot more lock in for them, right? Like you said, companies using other chips, other other designs like AMD kind of setups and everything, they're gonna have to probably invent their own version of this, you know, because NVIDIA is creating an opportunity for people to to get more more hardware more locked into NVIDIA's ecosystem, basically. Tobias Mann: Right, right. you know, the the other the other example that they gave, this was working in collaboration with Lambda Labs, was this what is it called? DSX Max LPS. And it was, you know, this example of where by knowing where all of the power is going in the data center, you can run a whole lot closer to your the the the actual capacity available to the facility. Cause usually traditionally Brandon: Mm-hmm. Tobias Mann: you'd might run at 80% of the available capacity because if you had too many things click on, or you had failures for redundancy, you didn't want to end up in a situation where you were pulling more than you actually had access to. And so Brandon: Right. Tobias Mann: this visibility lets them run closer to that edge. Now from an economic standpoint, somebody buying into this says sees that they can get nine nodes run GPU nodes running in what you or 19 nodes running in what used to only be able to support 16, that's a huge economic you know driver and and and a very attractive one to go with NVIDIA now. And so there's this Brandon: Mm-hmm. Tobias Mann: they're building this walled garden around this this AI factory push where it's like, well, it's an NVIDIA AI factory. So you can cram more GPUs into the same me number of megawatts that you would have, and you can probably convince your utility to give you more megawatts than they otherwise would because we have it so tightly integrated that you know it won't be a problem. Whereas if you go with AMD or any of these other AI chip startups, you might get some of those features from from Schneider and and others, but it might only work with their equipment. It might not be as feature rich. And so it's it then becomes just a you know a gamble to go with somebody else other than NVIDIA. This is the sales pitch here. So Brandon: Right, right. Tobias Mann: and and then you know, once you you build this in DSX spec NVIDIA AI factory, are you going to risk putting in AMD GPUs? now NVIDIA says that they're going Brandon: Yeah, that won't talk or to the you know, yeah. Tobias Mann: yeah, go ahead. Brandon: I was gonna say that w that these these in you know, these GPUs that might not talk to the rest of the stack now, right? Because it's Tobias Mann: Right. Brandon: yeah. Tobias Mann: And so th there there there is there's definitely some walled garden building going on here and some lock in. I don't know how long it will last though, because I think Nvidia recognizes that That kind of lock-in is potentially problematic from a regulatory standpoint, but also they want everybody building to their spec. And so in some cases, they have opened up the technology so that everybody can have like 80% of it, and then they get like the 20% on top of extra feature stack with their stuff is enough for them to be okay. They did this with their rack. designs with MGX. So it th we might see something like that happen from them just to get as more data centers buying into the design. But it's it's hard to say at this point. They may just use it to block out competition. Brandon: Yeah, we'll see, right? I mean, but it seems like in general this might have a a net positive effect on data center energy footprints, right? We're talking running more efficiently, you know, getting more out of each kilowatt, out of each megawatt. This is this is a a net positive for energy concerns when it comes to data centers, it seems. Tobias Mann: I think I think this is absolutely the right direction that we should be going. Historically, and I know Dan has written about this quite a bit over the last couple of years. Historically, these power saving features don't get used. they're Brandon: Mm-hmm. Tobias Mann: there and they they look really promising, but when you have a data center that's serving customers, there is a great deal of concern about meeting your SLAs. And so the risk inherent to enabling these kinds of power savings features has you know always been that you might not, you know, you might lose performance, you might

lose, you know, introduce latency, you might not be able to reach your your SLAs, and so you might end up not being able to charge your customers for the the work that has been done. And the power but the power consumption pro prior to what, 2022, 2023 has been pretty benign relative to what kind of the data centers we're talking about now. Brandon: sure, yeah. Yeah, this is a huge increase. It is interesting that we have kind of like this opportunity for AI to solve a bit of the problem that didn't exist before AI came along and and and caused this, right? So I guess we'll see. There's some good news here, but on the flip side of that, we have some recent announcements out of the EPA that make the data center energy problem, at least in the United States, seem like it's gonna get a heck of a lot worse. Dan, you covered a story in late August about some proposed changes with Public reporting requirements for air air pollution, is that is that correct? Dan: That's right, yes. The story was that the EPA was proposing to eliminate basically federal rules that would require... participation in minor source air permits is what it's called but what it basically boils down to is to whether there's any kind of public notification about things like data centers that are going to be causing pollution and whether there's any feedback on that, whether there's a proposed whether there's any say, the public gets any say on it. Brandon: Mm-hmm. Dan: So at the moment this thing exists as part of the Clean Air Act, I believe, but they want to undo that and basically it will go back to state level, so it will be up to the local authorities as to whether they actually bother to do that when there's a new data centre or something being built. Brandon: Right. So we're talking about when it we're talking about sort of sort of s sort of minor or small emissions issues. This is we're talking about like data centers that are building their own on site energy production facilities, correct? I mean, this isn't talking about grid connection or new construction being done for existing grids that data centers are gonna connect to, or is that covering both here? Dan: Yeah, I mean it's basically causing anything which might add extra pollution into the area. Brandon: Mm-hmm. Dan: it possibly doesn't include things like power stations and things which is where the miner comes in. But a Brandon: Right. Dan: lot of data centers these days are generating their own power and effectively Brandon: Right, right. Dan: having small on-site power stations in addition to the backup power generators that they all have. And so it's just a kind of... They pitched it as a rolling back of power, know, deregulation, removing Brandon: Mm-hmm. Dan: unnecessary regulation, know, Brandon: Yeah. Dan: all these woke rules that people have to follow. Brandon: Yeah. We Tobias Mann: I I feel like the the Yeah, Brandon: wouldn't want communities. Go ahead, Tobias. Tobias Mann: I was gonna say that I feel

like the the residents of Boxtown, Tennessee probably would disagree with all of this already. Brandon: Mm-hmm. Tobias Mann: you know, the the the the emissions coming off of these quote unquote portable gas generator plants, that they're portable in the sense that they're they're a extra wide load that can be hauled behind a s a a large semi-tractor. and what if you looked at the Colossus data center that SpaceX AI built what a year and a half ago. they Brandon: Mm-hmm. Tobias Mann: had 30 some of these running because they were waiting on a 150 megawatt interconnect to the the utility. Dan: I think it was more than that at one point, wasn't it? Brandon: Yeah, I think I think it ramped up. I think there's maybe fifty or more there now. Yeah. Tobias Mann: Yeah, yeah, it's increased now. but they you know the the the the challenge that happened was for a certain period of time, I think it might have been about a year, they could run these these gas generator plants without all of the air handling equipment that would make them more you know, less polluting, and cut down on kind of the smog generating materials. And so they they just didn't. They're just like, Well, you know, we're not required to. Brandon: Right. I mean that's the thing. There's there's no requirement. Yeah, exactly. That's the thing is there's no requirement that these companies or that data center operators who are bringing on their own own site power have to choose the the cleaner version, right? There are there are multiple kinds of gas turbines available. Some, like you said, include pollution mitigation technology, but there's nothing that says you have to use that and they're more expensive. So if you're a data center operator that's trying to keep their bottom line up, you're probably not gonna opt for these if you don't have to. And now if you don't have to report to the local community the potential emissions from this thing, there's even less of an incentive to do that. Dan, do you know if this rule has been finalized or is it still the sort of thing that's in discussion? Dan: This was only a month ago, I don't think I haven't heard about it being finalized. Brandon: Okay. Dan: We heard of it because of a pressure group called the Environmental Protection Network, which is actually made up of former EPA employees who were Brandon: Mm-hmm. Dan: raising the alarm about this because they were saying that apart from the... greenhouse gases, extra greenhouse gases that might be emitted by these things. There's actual pollution, know, the nitrogen oxides that generate smog and also, Brandon: Mm-hmm. Dan: you know, there can be heavy metals and other pollutants in there which are not very nice to breathe in. Brandon: No, definitely not. I mean, and so that's that's still kind of in the in the maybe this will make things worse category. But in the in the actually making things worse category, I don't know if you guys had heard this week that so there was a G twenty energy meeting in Houston, that was going on and the Environmental Protection Agency, which I mean, is barely even worthy of that name anymore. they decided this week to announce that they were eliminating these Biden era greenhouse gas requirements for power plants, basically getting rid of restrictions on on power generation facilities that capped their greenhouse gas emissions, right? And they they basically were saying in the in the EPA's press release, they said, you know, this it was gonna be a cost-saving measure, it was gonna save $310 billion. and there were some additional proposals they rolled out to eliminate, quote, Every remaining greenhouse gas standard for the power sector, unquote. And that they said would save an additional $370 billion in direct compliance costs, which, you know, fantastic, right? but now they're saying that this is gonna probably cause an increase in coal production for power sector use by a factor of 10, right? so basically we're we're we're being reintroduced to the era of soot covered cities and smog and, like you said, heavy metals and all this in the name of essentially. I think a lot of this is probably gonna be rolled out to support the a AI industry, right? I mean that's what the power push is for right now. So why is the American government deregulating pollution requirements if for no other reason than to ensure that we win this supposed AI race that we're in with China right now? Dan: Yeah. Tobias Mann: Th yeah, this is one of those that the US government is sometimes disconnected from the realities of the power sector. Brandon: Mm-hmm. Tobias Mann: I I I you know, there there is the potential that we'll see h increased coal use. I'm not sure that that will necessarily be true. One of the factors here is that getting coal out of the ground and maintaining those facilities is just getting really expensive in general relative to other sources of electricity. But but the other part of this is that power utilities in the United States have been burned so many times on speculative markets that they're really conservative to the point of being infuriating to the AI sector. They don't want to add capacity based on demand that hasn't materialized yet. Because if they do the ratepayers end up paying some of it, but they end up eating a lot of the cost as well, 'cause they can't put you know, they can't push the entire cost under the rate payer. It's just not possible to do that. And so while w you know, the the vision of, you know, Blade Runner and and the, you know, the the blackened sky and the the, you know the perpetually orange smog, you know, is is is is a visceral one. at this point the materi the fuel costs have gotten to such that I don't know that this is going to necessarily have the negative impact that we fear, but also I don't necessarily think it will have the economic impact the US government is pushing with this. the grid operators yeah Brandon: Yeah, I mean this Trump has been touting bringing

back coal for a long time, right? And he's gonna he's gonna res you know, cause a resurgence in the coal industry. And it hasn't happened, right? Because like you said, it's prohibitively expensive. A lot of the mines have already shut down, or getting the remaining coal out is more and more work. You know, yeah, I I hope you're right. Dan: Sure, sure. But some of those coal burning power plants were due to have closed down and basically the demand for energy has meant that they're being kept online instead of being shut down. Brandon: Right. Right. And if they are still online and they're now less beholden to environmental regulations, I mean that's that's probably a net negative, especially now if there's at least enough demand to keep them fired and keep them fired with less for new requirements. And also on again, right, I'm sure this touches on site power generation for data centers too. Right. If you're if you're, you know, building your own data center and you're throwing up some gas turbines. You're I mean, no one's gonna be throwing up coal burning generators at a data center, I don't think, anytime soon. Tobias Mann: One would help. Brandon: Yeah. But, you know, I mean they do mention natural gas in this too, right? So they're they're basically saying that they wanna they've they've eliminated clean energy requirements for natural gas turbines too. So, you know, even if you p theoretically wanted to get these cleaner ones, that might get harder if there's a there's already a huge weight for turbines. Tobias Mann: Right. Brandon: that the queue is getting bigger. Right. So I can't imagine that everyone's gonna like, well, we really wanna burn this this gas responsibly. So we're gonna get in the longer line for cleaner turbines that are now necessarily not gonna be in as high demand because there's less requirements that we have to do this. So Dan: Yeah, they've been actually repurposing old jet engines to turn them into the kind of turbines that can be used to generate energy. So, there's a... Brandon: Yeah, and as we all know, jet engines are incredibly environmentally efficient. Tobias Mann: There there there is also there's also an an interesting play here. the the race to get cac compute capacity up is s you know so intense right now that the cost you know choosing the most efficient fuel or energy generation mechanism is in some cases secondary to getting it Brandon: Mm-hmm. Tobias Mann: deployed in the first place. And so w we've also seen adoption of natural grass. Fuel cells. So similar to what you see and have seen in NASA spacecraft, but instead of hydrogen Brandon: Mm-hmm. Tobias Mann: and oxygen running on these things, it's predominantly natural gas and reacting with with oxygen to form CO2. the greenhouse gas emissions are roughly similar to gas turbines, but the pollutants tend to be fairly smaller because you're not combusting anything. So You don't necessarily have as much kind of smog producing output from these these systems. The downside is that they're not particularly power dense and so you need a ton of these fuel cells and they're expensive. Brandon: Right. Tobias Mann: But if you can't get turbines and you want your capacity now, that's what people are doing. Brandon: Yeah. Yeah. that would be nice if more of those became more of an option, right? I mean are they do you know what how they are to produce compared to turbines? I mean the turbine Q is big, but are these fuel cells you know can they ramp up production quick enough that they're a a better option? Tobias Mann: This th this is an open question right now. There's I think what one or two companies in the US that that actually make these things. Brandon: Okay. Tobias Mann: Bloom Energy is the big one and you know, comp big neo clouds like Nebius are deploying them in in in pretty large volumes in order to to supply power to their data centers because they you know, they wanna deploy in areas that have good connectivity but are constrained on power and so they're Brandon: Mm-hmm. Tobias Mann: There w you know, it's oftentimes in those situations easier to get a natural gas line out to the building, than it is to get, you know, transmission lines hooked up. Brandon: Sure. Yeah. Dan: Yeah, yeah, in fact there was a report out just this week that the ability to build extra infrastructure, the grid, is actually holding back on data centers. It's outpacing the rate at which the utility companies and the grid operators can actually add and build new lines to supply the power. Brandon: Yeah, I believe it. I believe it. I mean there's there's there's a lot of projects happening in a lot of very remote areas, you know, that require a lot of transmission. Up in my part of the in my part of the state, I'm in Michigan, in my area, you know, there's no one actually discussing building a data center, but a lot of people are worried about it. And those of us who who, know this stuff are pretty much like, you need to relax. There's no transmission out here. And also beyond that, there's no space for someone to build their own co-located power facility. You know, there's a lot of protected land, a lot of a lot of occupied space and the zoning and everything, you know, doesn't allow it. But there are a lot of areas like mine where there is space, but no transmission lines. And yeah, those areas are ripe for the picking, but you're gonna need to supply your own power. So Tobias Mann: Her Hermantown, Minnesota, right outside of Duluth on on Lake Superior. Lots wet, plenty of power, cheap land, not Brandon: Mm-hmm. Tobias Mann: popular at all. Brandon: With data center operators? Yeah. Yeah, totally. And that's the thing is I think a lot of people they Tobias Mann: No no, not popular with the s the residents of that area. So yeah. Brandon: don't they don't want them here. I mean, people don't want them here. People don't want them there. No one wants these things in their backyard. but increasingly I feel like we're running out of space to do that. I don't know. We'll see what happens. Dan, I guess you're just lucky you're not here. dealing with all this deregulation. you know, hopefully the air stays a little bit cleaner over over where you're at. You know, and I hope Tobias is right that we don't have a whole bunch of coal plants going up in the near future. Dan: Yeah, we've got rid of all our coal plants over here, but there is a shortage of energy and we've got the most Brandon: Yeah. Dan: expensive energy in certainly in Western Europe, if not the whole G7 in the UK. And there's no shortage of data centres. Tobias Mann: No. Brandon: Yep. Dan: Plenty of data centers and I know of at least three new ones that are being built within about a 20 mile radius of where I live, which is just to the north of London. So they're coming. It's just, you know, and people are worried about where the energy is going to come from to supply them. Brandon: Totally. Tobias Mann: I I I I guess here here's a here's a good place for us to to to place the bet. Which which happens first? small modular reactors or space space data centers at scale? Dan: Hahaha Brandon: my god. Yeah. I don't want to take a bet on either side of that equation because Tobias Mann: Ha ha ha Brandon: I mean, I don't know that I'll be alive to see either one happen. It's you know, I would love I would love it if small modular reactors won out because that would mean that there would be more safe nuclear in more places. but but I jeez, I don't see it happening anytime soon. Tobias Mann: If you thought the Dan: Ah-hah! Tobias Mann: NIMBY for data centers was bad, just wait till you attach nukes to the slides of them. I mean Brandon: Yeah, yeah. I can't imagine people will like it even if it is relatively safe compared to you know, we're not in the three mile island Chernobyl area of nuclear power anymore. But I don't think very many people have really bothered to to learn to understand that. So Well, no matter which way it swings, we will be here at the kettle to tell you all about it, and we hope you'll tune in and not be too miserable by the end of our future episodes. Thanks for listening.