Microsoft documents its liquid-immersion cooling efforts

Last week I told you about an immersion-cooling firm called LiquidStack being spun off from its parent company, the China-based server vendor Wiwynn. The story mentioned how Microsoft was experimenting with immersion cooling, and now Microsoft has pulled back the curtain on the whole show.

It’s been trying out immersion cooling for two years but is now going full throttle, at least at its Quincy, Washington, data center. Situated in the middle of the state, the city of Quincy is tiny—just 6,750 as of 2010—but the Columbia River cuts through it, making it ideal for a hydropower-based data center, and there are several data centers in this tiny town.

The company said the initiative started with the same observation everyone has about data-center cooling: With CPUs drawing more than 300 watts per chip and GPUs drawing up to 700 watts per chip it needed something more effective to carry off heat.

“Air cooling is not enough,” said Christian Belady, distinguished engineer and vice president of Microsoft’s data center advanced development group in the company’s Innovation blog. “That’s what’s driving us to immersion cooling, where we can directly boil off the surfaces of the chip.”

Microsoft uses the Wywinn DataTanks, which they said are each about the size of a couch, filled with equipment submersed in dielectric fluid made by 3M that is non-conductive of electricity. The liquid has a boiling point of just 122 degrees Fahrenheit, a temperature that CPUs and GPUs under load will hit real fast. The bath boils off and is caught by coils that run through the tank and enable the vapor to condense. The condensed liquid then falls right back into the tank.

Microsoft said that so far its immersion cooling has reduced the power consumption of servers by between 5% and 15%. Under high demand, the servers can even be overclocked, where the processor runs at above its rated speed, without risk of overheating.

Change of Atmosphere

There are other benefits besides more efficient heat transfer, Beladay noted. Because the hardware is immersed in the dielectric fluid, the hardware is protected from the corrosive effects of humidity and oxygen in the data center, leading to fewer mechanical failures in the system.

Microsoft discovered this with its Project Natick, an experimental data center sunk off the coast of Scotland. Project Natick is exploring the potential of underwater data centers that are easily deployed and can operate for years on the seabed sealed inside submarine-like tubes without any onsite maintenance by people.

The underwater data center is filled with dry nitrogen instead of normal air. One of the findings of Project Natick is that the servers on the seafloor experienced one-eighth the failure rate of replica servers in a  data center on land, and preliminary analysis indicates that the lack of humidity and corrosive effects of oxygen were primarily responsible for the superior performance.

The immersion tanks could provide the same kind of protection. “With immersion, you have a similar thing,” Belady said. “Essentially, you are displacing oxygen and moisture.”