Cold Plates and Immersion: Direct Liquid Cooling in the Best Practices Guide for Energy-Efficient Data Center Design

The guide is called "Best Practices Guide for Energy-Efficient Data Center Design," and its cover carries "Revised July 2024." Its acknowledgments say the work "was supported by the U.S. Department of Energy's Federal Energy Management Program." Data center operators "should consider implementing Direct Liquid Cooling of high-density servers," the guide says, and it gives that way of cooling a section of its own.

Its disclaimer says the views and opinions of the authors expressed in it "do not necessarily state or reflect those of the United States Government or any agency thereof." Parts of the guide follow, not the whole.

Short Design Cycles

"Data center spaces can consume many times as much electricity as standard office spaces," it says, and with such large power consumption they are prime targets for energy-efficient design measures that can save money and reduce electricity use. But the critical nature of data center loads, it adds, puts many design criteria, "chiefly reliability and high-power density capacity," far above energy efficiency.

"Short design cycles often leave little time to fully assess efficient design opportunities or consider first cost versus life-cycle-cost issues." That can lead, the guide says, to designs that are "simply scaled up versions of standard office space approaches," or that reuse strategies and specifications that worked "good enough" in the past "without regard for energy performance."

A design cycle too short to weigh first cost against life-cycle cost can repeat what was built before. For a hall meant for high-density racks, what was built before may be a way of cooling that no longer fits.

Heat at or Very Near Its Source

"Direct liquid cooling refers to a number of different cooling approaches that all share the same characteristic of transferring waste heat to a fluid at or very near the point the heat is generated, rather than transferring it to room air and then conditioning the room air." Liquid cooling "can serve higher heat densities and be much more efficient than traditional air cooling," it says, "as water flow is a much more efficient method of transporting heat." Energy efficiencies will be realized, it says, when such systems allow "the use of a medium temperature chilled water supply," and by reducing the size and power consumption of fans serving the data center.

On the IT loop side, liquid-cooling fluids "can range from treated water to glycol-based solutions to dielectric fluids (nonconductive)." Several liquid-cooled solutions are considered hybrid solutions, in which the liquid captures "the majority (but not all)" of the heat load and traditional air cooling removes the rest. Some others "capture practically all the heat load without the use of any fans, which is the most energy-efficient." Most approaches involve a cooling distribution unit, a CDU, which interfaces with the facility cooling loop and provides cooling liquid at the temperature, pressure and chemistry appropriate for the IT equipment.

Cold Plates and Immersion

Liquid-cooling technologies, the guide says, "can be categorized into three technology classes." One is a localized air-to-liquid heat exchanger, a coil placed at the back of a server rack. With cold plates, "Standard fin-based heat sinks on chips are replaced with cold plates that have liquid flowing through channels to remove heat," a solution that generally involves a CDU. With immersion, electronics are submerged in a dielectric fluid. In single-phase immersion the fluid is pumped around the electronics by a CDU. In two-phase immersion an engineered dielectric fluid removes heat by changing from liquid to gas, and condenses back to a liquid "in a passive cycle."

"The cold plate and immersion classes are considered direct-liquid-cooling technologies." High-performance computing data centers, the guide says, "have been early adopters of direct liquid cooling due to rack power densities (where densities of 60 kW per compute rack were observed in 2013, and recently surpassing 125+ kW per compute rack)." The sentence names no source for those densities.

A rack that draws that much is as much a question for the electrical side of a campus as for its cooling. The heat a cold plate carries away is power the campus first had to deliver.