Why AI Hits a Cooling Wall, Not a Chip Wall: 140 kW Racks

Artificial intelligence runs into cooling because almost all the electricity that computing consumes turns into heat. An AI rack gives off 100–140 kW, while air can remove no more than 50 kW per rack. On a 5,000 kW data center, classic chillers spend 10.22 million kWh a year on cooling; direct-to-chip cooling, about 1.3 million.

People talk about neural networks in terms of brains and money: what the chips cost, whose model is smarter. Hardly anyone talks about the physics. Yet every breakthrough in computing is paid for in megawatts that have to be brought to the site and then carried away as heat.

I build cooling for industry and data centers, so this is the side of AI I look at. Below: what is changing in heat density, how much electricity goes to cooling, why in Russia everything comes down to grid capacity, and what a data center owner should do before the project starts.

Why is AI growth limited by heat removal rather than chips?

AI growth is limited by heat removal because heat density in the server room has grown tens of times over, while air as a heat carrier has stayed the same. A typical plant shop floor where we install cooling gives off 3–5 kW of heat per square meter. A data-center rack on the same footprint puts out 30–40 kW, and an AI server rack a full 100.

Flagship systems go further still. An NVIDIA GB200 or GB300 NVL72 rack (Grace Blackwell architecture) gives off 120–140 kW, a single chip 1,200–1,400 W. Once a rack goes beyond 50 kW, air physically cannot remove the heat, and the conversation turns straight to liquid cooling.

Hence the main conclusion. You can buy chips in a month, but bringing megawatts to a site and removing the same amount of heat from it is a multi-year project. So in the race for AI, people hit the limit of the power grid and the cooling plant before they hit the limit of silicon.

How much electricity does a data center spend on cooling?

The cooling share of data-center electricity depends on the scheme and the generation of the site: from a few percent to a third or more. According to an industry review by Encor Advisors, the most efficient hyperscale sites (the largest cloud data centers) spend about 7% of their energy on cooling; less efficient enterprise sites, more than 30%.

We modeled one and the same site: 5,000 kW IT load, 70% average load, Moscow, year-round operation. The servers consume 30.66 million kWh a year, other infrastructure 1.53 million. Cooling adds 1.3 to 10.22 million kWh on top, depending on the scheme.

Cooling scheme	Cooling, million kWh per year	Total site electricity, million kWh per year	Cooling share	PUE
Chillers without free cooling	10.22	42.41	24%	1.45 and above
Packaged chillers with free cooling	5.57	37.76	15%	1.25–1.35
Centrifugal chiller and open cooling tower	4.70	36.89	13%	1.20–1.30
Direct-to-chip cooling, closed cooling tower	1.30	33.49	4%	1.06–1.12

PUE (power usage effectiveness) is the ratio of all the electricity a data center uses to the electricity of its IT equipment. The ideal is 1.0; the industry average is 1.6–1.8. The closer to one, the less electricity goes to anything other than computing.

A difference of a few tenths of PUE is big money. On a data center with 10 MW of IT load, moving from PUE 1.6 to 1.3 takes 3 MW off the meter around the clock, or 26 million kWh a year. At RUB 10 per kWh that is about RUB 263 million per year.

Why has grid power become the main constraint for data centers?

Grid power has become the main constraint because data-center demand is growing faster than the grid. In Moscow, data centers today draw about 1 GW, and by 2028 they will need 3.1 GW. There is no spare capacity, what is left is reserved, and the capital's power reserve margin in 2026 is forecast below 5%.

Under these conditions every kilowatt taken off cooling goes to computing. When the grid is short, that is direct revenue for the operator. A grid connection (connection to the network, with every kilowatt of capacity paid for) costs RUB 47,000 per kW, and up to RUB 100,000 in some regions.

What matters here is not annual consumption but the peak. A 5,000 kW cooling plant with packaged chillers holds a summer peak of 1,667 kW, and the grid connection is sized on it. Direct-to-chip cooling brings the peak down to 360–410 kW, a gas absorption chiller to 230 kW. That frees 1,260–1,440 kW: 25–28 racks of 50 kW that can go into computing on the same grid capacity.

Our strongest example is a court complex in St. Petersburg: 12,000 kW of cooling, four absorption chillers on natural gas, project under way. On the Petrograd Side there is simply nowhere to get that much electricity, and with electric chillers the project would not have happened. I explain how gas frees capacity for racks in Absorption chiller for data centers when grid power runs out.

What does direct-to-chip liquid cooling change?

Direct-to-chip liquid cooling removes the refrigeration machine from the scheme: AI servers do not need chilled water. A cold plate with channels is pressed against each chip, and coolant flows through it. NVIDIA GB200 and GB300 NVL72 accept inlet water up to 45 °C (class W45 in the classification of ASHRAE, the American engineering society), and the die stays at 80–90 °C.

Water that warm does not need a compressor to cool it; a closed cooling tower is enough. It delivers 29–31 °C water even in hot weather because it works on the wet-bulb temperature, and the chip receives 32–40 °C. On a 5,000 kW site the peak of the whole system drops from 1,667 to 360–410 kW, and annual consumption from 10.22 to 1.3 million kWh.

The rest of the world has worked this way for a long time. NREL, a US Department of Energy laboratory, cools its computers with 24 °C water without mechanical chillers at PUE 1.036. The scheme is covered in detail in Liquid cooling for a data center without a chiller.

To be honest about ourselves: we do not yet have completed projects with liquid cooling of servers. We offer it, and we have a project in the works where our cooling towers first cool the refrigeration machines and then the same towers switch to liquid cooling. That is an evolution of the existing loop, without replacing the whole scheme.

What changes for data centers in Russia?

For Russian data centers the main difference is that the climate works for us, while our calculations are weaker than they could be. For half the year it is colder outside than in the server room. Free cooling (cooling with outdoor air) lets the compressors run about four months instead of twelve.

A typical Russian data center is cooled by packaged chillers with built-in free cooling, about 500 kW each, on the roof. One site in Moscow has fifty of them. In the US, instead of a scattering of machines, they build a large central chiller plant, and before that they compare dozens of options on 15-year total cost of ownership.

Here, cooling is chosen on the price of the machine, because in the cost estimate the chiller is the fattest line. But the American magazine FacilitiesNet puts the first cost of the equipment at 5–10% of the life-cycle cost of a chiller plant. By our calculations it is up to 15%, depending on the scheme. The rest goes to electricity, make-up water, service and spare parts.

AI is already helping cooling itself. In 2016 Google and DeepMind put a neural network in charge of data-center cooling, and the energy used for cooling fell by up to 40%. In July 2026 Johnson Controls (YORK) presented a reference design for AI data centers on gas-fired cooling: 44% less electricity for cooling and up to 97 MW of capacity freed on a 1 GW site. These are the manufacturer's modeled results; on a specific site the numbers will differ.

What should a data center owner do now?

A data center owner should choose the cooling scheme before paying for the grid connection, not after. The main argument for gas, liquid cooling and cooling towers is the grid kilowatts they save. If the capacity has already been bought in full, that argument is spent. At a webinar I asked the owner of a 5 MW data center why they had not done it this way. The answer was: "Well, we had already bought it."

The sequence I recommend:

1. Find out the rack power now and three years out. If it goes beyond 50 kW, air cannot remove the heat, and the project is designed for liquid cooling from the start.

2. Find out how much electrical capacity the site will get and what a kilowatt of grid connection costs. If it is not available or too expensive, run the numbers on a gas absorption chiller or trigeneration.

3. Check whether treated water is available and whether there is waste heat nearby: a gas engine plant, a boiler house, steam. Water decides between a dry scheme and a cooling tower; heat decides the payback of an absorption chiller.

4. Compare at least three schemes on 15-year cost of ownership, not on the price of the chiller. Use seasonal efficiency: a data center runs at peak 5–7 days a year.

5. Design the loop for warm water. The higher the temperature the equipment tolerates, the more hours a year the compressor is not needed at all, and the easier it is to move to liquid cooling later on the same cooling towers.

A comparison of five schemes on one site, with money and payback, is in Data center cooling: five options for 5,000 kW. All articles on the topic are in Data center cooling.

Key takeaways

1. Almost all the electricity that computing consumes becomes heat. An AI rack gives off 100–140 kW; air removes no more than 50 kW.

2. On a data center with 5,000 kW of IT load, cooling takes from 24% of the electricity with chillers without free cooling to 4% with direct-to-chip cooling.

3. Moscow is short of power: about 1 GW now against 3.1 GW of demand by 2028. Every kilowatt taken off cooling goes to computing.

4. Direct-to-chip cooling and a gas absorption chiller cut the peak of a 5,000 kW cooling plant from 1,667 to 230–410 kW and free room for 25–28 racks of 50 kW.

5. The cooling scheme is chosen before paying for the grid connection and on 15-year cost of ownership, not on the price of the machine.

FAQ

How much heat does an AI server rack give off?

An AI server rack gives off about 100 kW, a flagship NVIDIA GB200 or GB300 NVL72 rack 120–140 kW, and a single chip 1,200–1,400 W. For comparison: a square meter of a plant shop floor gives off 3–5 kW, a typical data-center rack 30–40 kW.

Can AI servers be cooled with air?

Only up to 50 kW per rack. Above that density air physically cannot remove the heat, and you need liquid directly on the chip. Some racks on liquid and some on air in the same hall is a normal solution.

What is a good PUE for a data center?

The industry average PUE is 1.6–1.8. Schemes with free cooling and cooling towers achieve 1.20–1.35, direct-to-chip cooling 1.06–1.12. The NREL laboratory in the US holds 1.036 without mechanical chillers.

How much water does data center liquid cooling use?

On a data center with 5,000 kW of IT load, direct-to-chip cooling with a closed cooling tower uses about 40,000 m³ of water a year, a scheme with an open cooling tower and a chiller about 63,000 m³. Consumption is lower because below +5 °C outdoors the spray water is switched off, and for 3–5 months a year the tower runs dry.

Does Cold Energy have liquid-cooled server projects?

No completed projects yet. We have a project in the works where cooling towers first cool the refrigeration machines and then switch to liquid cooling. We make closed cooling towers with finned heat exchangers ourselves, including for nuclear power plants.

Sources

• DeepMind AI Reduces Google Data Centre Cooling Bill by 40%, DeepMind blog, 20 July 2016.

• Encor Advisors, Data Center Cost: cooling share of energy use in data centers of different classes.

• FacilitiesNet: the first cost of equipment is about 5–10% of the life-cycle cost of a chiller plant.

• NVIDIA GB200 and GB300 NVL72: allowable inlet water temperature up to 45 °C, ASHRAE class W45.

• NREL ESIF, a US Department of Energy laboratory: direct liquid cooling without mechanical chillers, PUE 1.036.

• Johnson Controls (YORK), July 2026: reference design for gas-fired cooling of AI data centers, modeled results.

• Cold Energy experience and calculations: site with 5,000 kW of IT load, tariff RUB 10 per kWh.

Need a cooling solution for your site? Get a consultation