IM Engineer Study Notes
← Back to list
Infrastructure & Cloud
#데이터센터#액침냉각#직접칩냉각#PUE#열관리
Last updated · 2026-10-20

Data Center Cooling (Air, Liquid, and Immersion Cooling)

1. Overview

Data Center Cooling is the thermal-management discipline that, in an environment where IT equipment such as servers and accelerators converts all consumed power into heat, captures the generated heat and safely rejects it to the outside so that semiconductor devices stay within their allowable temperature, while minimizing the energy, water, and space consumed in doing so. Cooling is far more than "keeping gear from failing"; it is a central design axis that simultaneously governs a data center's power efficiency (PUE), achievable rack density, total cost of ownership (TCO), and carbon emissions.

The background to cooling emerging recently as a standalone design challenge is the dramatic rise in power density per rack. Space filled with traditional web and DB servers drew roughly 5–10 kW per rack, and that level of heat was comfortably handled by air cooling alone, pushing cold air up through the floor of the machine room. But since the generative-AI boom, AI racks packing dozens of GPUs/accelerators that each consume more than 700 W have appeared, and racks exceeding 40–130 kW have become common. For example, a configuration binding 72 GPUs into one rack, like NVIDIA's GB200 NVL72, pushes a single rack well past 100 kW — a heat load that air physically struggles to remove.

The key here is the physical fact of the difference in heat-transfer capacity between air and liquid. For the same flowing volume, water can carry roughly 3,000 times more heat than air, so the amount of fluid and the fan/pump power needed to remove the same heat differ fundamentally. The harder air cooling blows air, the sooner it hits the limits of noise, fan power, and airflow-path design, and in practice economics and stability degrade sharply near 20–30 kW per rack. Thus cooling in the high-density era is moving not toward "blowing air harder" but toward "bringing liquid closer to the heat source," and this shift is why liquid cooling and immersion cooling have risen to mainstream discussion.

The second reason cooling matters is energy/water efficiency and sustainability. A data center's power-efficiency metric, PUE (Power Usage Effectiveness), is defined as "total facility power ÷ IT equipment power," with an ideal lower bound of 1.0. It would be 1.0 if there were no cooling/distribution losses at all, but a typical real-world data center sits around 1.5, while a highly optimized hyperscaler drives it down to the 1.1–1.2 range. In other words, a PUE of 1.5 means that when IT uses 100, cooling and ancillary systems use an extra 50, and the largest chunk of that extra is cooling. If merely changing the cooling method improves PUE by 0.1–0.3, at a large site that moves billions of won in annual electricity bills and a substantial amount of carbon emissions.

Against this backdrop, the purpose of this article is to systematize data-center cooling from the viewpoint of where heat is received and by which fluid it is carried how far, and to treat in depth the principles and trade-offs of air, liquid, and immersion cooling, along with efficiency metrics such as PUE and WUE and the latest trends.

2. The Heat-Flow Structure of Cooling and Key Metrics

The starting point for understanding data-center cooling is to draw out "the chained heat-transfer path by which heat generated at the chip is ultimately rejected to the atmosphere (or to water/district heating)." Because heat flows only from hot to cold, each stage needs a medium (air, chilled water, refrigerant, outside air) to create a temperature difference, and the fans, pumps, and compressors that move those media become cooling power itself. The overall structure diagram below shows the typical path from heat source (IT equipment) to external heat rejection, bundling the air-side and liquid-side systems together.

flowchart TB
  subgraph IT["Heat source: IT equipment(racks)"]
    CHIP["CPU/GPU die(hundreds of W)"]
    RACK["servers·racks(tens to 100kW+)"]
    CHIP --> RACK
  end
  subgraph ROOM["Indoor heat-removal stage"]
    AIR["Air: CRAC/CRAH + hot/cold aisle"]
    LIQ["Liquid: cold plate(D2C) · rear-door HX · immersion tank"]
    RACK --> AIR
    RACK --> LIQ
  end
  subgraph LOOP["Heat-transport system"]
    CDU["CDU(coolant distribution unit) · secondary chilled loop"]
    CHW["Chilled Water primary loop"]
    AIR --> CHW
    LIQ --> CDU
    CDU --> CHW
  end
  subgraph REJECT["External heat rejection"]
    CHILLER["chiller/compression · cooling tower · free cooling"]
    REUSE["waste-heat reuse(district heating, etc.)"]
    CHW --> CHILLER
    CHW --> REUSE
  end

As the figure shows, cooling consists of a chain of ① receiving heat at the chip/rack → ② first removing heat indoors → ③ a chilled-water/refrigerant loop carrying heat out of the building → ④ rejecting it outside (or reusing it). Whether air or immersion, the chain itself is the same; only whether ①–② are done with air or liquid distinguishes the method. If any single stage in the chain is a bottleneck, heat from the prior stage accumulates and device temperature rises, so cooling design is the work of balancing the heat budget across the whole chain.

The key metrics for evaluating this structure need their meaning made clear in prose. PUE, as defined above, is the representative metric for overall facility efficiency, but it is sensitive to season and site — favorable in cold winters, worse in hot summers — so it should be read as an "annual average" and not overtrusted as a single value. WUE (Water Usage Effectiveness) is the amount of water consumed to process 1 kWh of IT power (L/kWh); using an evaporative cooling tower improves PUE but consumes a lot of water, creating a trade-off in which WUE worsens. That is, power and water are in a relationship where pressing one pops the other out, so in water-scarce regions chasing PUE alone can breed conflict with the local community. Beyond these, ERF (Energy Reuse Factor), which measures heat-recovery rate, and the ASHRAE TC 9.9 guidelines (recommended roughly 18–27 °C) setting the room temperature/humidity standard serve as the common language of design and operation.

Category Air Direct-to-Chip (D2C/DLC) Immersion
Heat-transport medium air chilled water (via cold plate) dielectric fluid (direct contact)
Suitable rack density around ~20 kW 40–100 kW+ tens to 100 kW+
Relative efficiency (PUE contribution) low high very high
Adoption difficulty / retrofit easy medium (piping·CDU) large (tanks·fluid·maintenance)
Main risk density limit leakage fluid cost·PFAS·service access

The table is only a summary to aid comparison; why each method has these characteristics is explained by principle in the next section.

3. Principles and Types by Cooling Method

A. Air Cooling and Its Limits

Air cooling is the oldest and most common method: cold air is supplied into the machine room, server fans draw that air into the equipment to carry heat out, and the warmed air is gathered again and cooled by a cooling coil in a cycle. In practice, hot-aisle/cold-aisle containment is the baseline: racks face each other so that the cold-air-intake fronts are grouped together and the hot-air-exhaust rears are grouped together, with the aisle physically closed off. This reduces bypass and recirculation — where cold and hot air mix and efficiency drops — so the same chiller can remove more heat. Air handlers split into CRAH, which uses chilled water, and CRAC, which uses refrigerant directly; in regions with cold outside air, free cooling (economizer), which turns off compression cooling and uses outside air directly (or indirectly), saves a great deal of power.

The virtue of air cooling is that its design and operation are mature, it is easy to retrofit, and the risk of catastrophic incidents such as leaks is low. But because of air's low heat capacity noted above, it grows rapidly disadvantaged as density rises. Removing more heat requires blowing air faster and in greater volume, and raising air velocity makes fan power surge roughly with the cube of velocity, while noise and vibration also grow. It is also hard to fully prevent recirculation of hot air from the rack rear back to the front, so above 20–30 kW hot spots persist. Ultimately air cooling remains reasonable for mid-to-low-density workloads spread "wide and low," but structurally does not fit AI racks packed "narrow and high."

Intermediate techniques for pushing air cooling even slightly toward high density are also widely used. The Rear-Door Heat Exchanger (RDHx), which mounts a water-cooled coil on the rack's rear door, cools the hot air leaving the rack right at the door and returns it to the room, reducing whole-room air-conditioning load while being relatively easy to add to an air-cooled floor — making it a realistic option for retrofit-style densification. In addition, details of airflow design — the raised floor approach sending cold air beneath the floor, the return plenum approach gathering hot air into the ceiling for exhaust — govern cooling efficiency even for identical equipment. Thus, however simple it looks, air cooling varies widely in performance depending on how thoroughly the principle "do not mix cold and hot air" is implemented.

B. Liquid / Direct-to-Chip Cooling (DLC)

The core idea of liquid cooling is to "strip out air, that inefficient intermediate medium, and bring liquid directly against the chips that generate the most heat." Direct-to-Chip (D2C or DLC) mounts a metal cold plate tightly onto the CPU/GPU and flows coolant through it to take heat directly. The warmed water gathers at a CDU (Coolant Distribution Unit) placed per rack or per row, exchanges heat with the building's primary chilled-water loop, and then circulates again after cooling. Here the CDU separates the secondary loop that touches the servers (cleanly managed coolant) from the building's primary loop, acting as a buffer that localizes water-quality, pressure, and leak risk.

D2C's strength is that it can handle AI-rack-level high density while coexisting with existing air-cooled infrastructure. Because the cold plate directly takes a large share of chip heat, the air-conditioning load left on the floor shrinks and PUE improves noticeably. However, it is hard to cover every component (memory, power delivery, storage, etc.) with cold plates, so typically part of the heat is still handled by air in a hybrid form. Also, piping and quick-connectors added per server make leak detection/shutoff design essential, and connector fastening quality during maintenance governs reliability. Even so, D2C is regarded as the realistic first step in moving "from air to liquid" and has been adopted almost as a de facto standard in recently built AI data centers.

Looking more closely at the D2C system, the key is that the CDU separates, via a heat exchanger, the secondary loop (TCS) that touches servers directly from the primary loop (FWS) circulating the whole building. The secondary loop protects the chips by circulating clean coolant with tightly managed water quality/pressure in a narrow temperature band, while the primary loop connects to cooling towers, chillers, and free cooling to finally send heat outside. Separating the two loops means a micro-leak on the server side does not spread to the whole-building chilled-water system, and conversely water-quality fluctuations in the primary loop do not touch the chips directly. The detail diagram below shows how these two loops, the CDU, and leak detection interlock.

flowchart LR
  subgraph SERVER["servers·racks(secondary loop TCS)"]
    COLD["cold plate · tight to GPU/CPU"]
    QC["quick-connector · leak sensor"]
    COLD --> QC
  end
  subgraph CDUBOX["CDU(coolant distribution unit)"]
    HX["plate heat exchanger"]
    PUMP["circulation pump · flow/temp control"]
    LEAK["leak detection · auto shutoff valve"]
    QC --> HX
    HX --> PUMP
    PUMP --> QC
    LEAK -.monitor.-> QC
  end
  subgraph FACILITY["building(primary loop FWS)"]
    CHILLER2["chiller · free cooling"]
    TOWER["cooling tower · external rejection"]
    HX --> CHILLER2
    CHILLER2 --> TOWER
    CHILLER2 --> HX
  end

The detail diagram shows that the CDU is not a mere pump box but the center of safety and control that actively regulates flow and temperature and instantly shuts off leaks. Keeping the secondary loop's supply temperature above the dew point to prevent condensation, and adjusting flow to match load so as not to spend excessive pump power, are the crux of operational optimization.

C. Immersion Cooling

Immersion cooling is the most radical method, submerging the entire server board in a non-electrically-conductive dielectric fluid so that the fluid directly takes heat from every component. Because it passes through neither air nor cold plate, the heat-transfer path is the shortest, and with no need for fans, noise and fan power vanish. Immersion splits into single-phase, where the fluid does not boil and carries heat only by convection, and two-phase, where the fluid boils at the chip surface to take heat as latent heat and then returns to liquid in a condenser. The two-phase method uses the enormous latent heat of vaporization near the boiling point for extremely high heat-removal efficiency, but as environmental/regulatory issues with the fluorinated fluids used (e.g., certain PFAS families) have surfaced, material choices are being reshaped.

Immersion is theoretically the most favorable in density/efficiency and has the potential to pull PUE down into the low 1.0s, but its adoption barrier is also the highest. Fluid tanks that greatly increase floor loading, the initial/refill cost of expensive dielectric fluid, the unfamiliar maintenance workflow of submerging and extracting, and the inconvenience of optical connectors, labeling, and inspection are all challenges. Accordingly, immersion is currently adopted mainly for ultra-high-density/special purposes (cryptocurrency mining, some HPC/AI pilots), and it is reasonable to see general commercialization as proceeding gradually alongside standardization, fluid regulation, and supply-chain maturity.

Single-phase and two-phase immersion have different operational characteristics. The single-phase method circulates fluid with a pump without boiling it and sends it to a heat exchanger, so its structure is relatively simple and fluid evaporation/loss management is easy, while its heat-removal density is lower than the two-phase method. The two-phase method enables ultra-high-density heat removal as the fluid boils up at the chip surface and absorbs enormous latent heat, but sealing/condenser design is tricky, sealing so that vaporized fluid does not escape is important, and the regulatory risk of the fluorinated fluids noted above is especially pronounced in this method. Material compatibility — verifying that server components, cables, and sealing materials do not degrade after long contact with the fluid — is also an immersion-specific design challenge, and the fact that existing air-cooled components often cannot simply be submerged is a factor slowing generalization.

D. Heat Rejection and Waste-Heat Reuse

The three methods above dealt with "how heat is captured," and the captured heat must ultimately be rejected outside the building. Traditionally, water is evaporated in a cooling tower to throw heat away or a chiller's compression cooling cools it, but these are the chief culprits of power (chiller) and water (cooling tower) consumption. So recently, free cooling — which halts compression cooling and cools naturally in cold regions/seasons — has become a baseline strategy, and going further, attempts to reuse captured heat rather than discard it are spreading. For example, cases of supplying a data center's warm effluent to nearby district heating, greenhouses, and swimming pools are increasing, mainly in Northern Europe, and this is quantified by the ERF metric. Waste-heat reuse carries large significance from the professional-engineer viewpoint as a shift in thinking that turns cooling from a "cost" into a "resource."

Worth noting here is the fact that liquid cooling is close to a precondition for waste-heat reuse. The hot air that air cooling exhausts is low in temperature and dispersed, so its reuse value is low, whereas the warm effluent captured by D2C/immersion is gathered at relatively high temperature, making it advantageous for secondary uses such as district heating. Thus the shift to high-density/liquid cooling is not merely a change in "how to cool" but also opens the possibility of redefining the data center as one axis of energy infrastructure that supplies heat. However, reuse only holds if there is a demand point (nearby heat demand), so it is a design variable that must be considered from the site-selection stage.

4. Comparison and Application Cases

The difference among the three methods ultimately originates in the "distance between heat source and coolant." Air cooling passes through many stages — chip → heat sink → air → chilled water — demanding a temperature difference and blowing power at each stage, whereas D2C skips one or two stages via chip → cold plate → chilled water, and immersion is essentially direct via chip → fluid. The shorter the distance, the lower the power to capture the same heat, so efficiency improves; but the closer liquid is brought to the heat source, the more the costs of leakage, fluid management, and maintenance difficulty grow together. That is, efficiency and operational complexity are in exchange, and which method is the "right answer" depends on rack density, site (outside air, water resources), whether retrofitting, and organizational capability.

To get a feel with concrete cases and figures: first, hyperscalers have disclosed cases of lowering annual-average PUE into the 1.1s by optimizing outside-air free cooling and hot-aisle containment to the extreme, a result combining site selection (cool climate) with operational automation. Second, in Korea, Naver is known for applying cooling designs that actively use outside air at its "GAK" facilities in Chuncheon, Gangwon and in Sejong, raising power efficiency. Third, AI-dedicated racks (around 100 kW per rack) are essentially impossible with air cooling, so D2C is adopted by default, and 72-GPU-class rack configurations being designed on the premise of liquid cooling are a representative example. These three cases each show different drivers: "operational optimization," "site/outside-air utilization," and "liquefaction forced by density."

The implication the professional engineer must not miss in this comparison is that the cooling method cannot be separated from power, site, and network design. For example, if rack density is raised and liquid cooling introduced, floor area shrinks but the power-intake capacity per rack and the distribution/breaker design must grow together, and high-density packing shortens GPU-to-GPU communication distance, working favorably for network efficiency. Conversely, lowering density and keeping air cooling simplifies cooling but requires wider floor space and longer cabling. That is, "how to cool" is one bundle of decisions with "how densely, where, and with what power to build," and optimizing cooling in isolation throws the whole design out of alignment.

5. Deep Dive: Latest Trends in High-Density, Sustainable Cooling

The recent flow of cooling technology converges on "from air to liquid, and then toward saving water and carbon too." First, as the AI demand surge makes D2C liquid cooling the default for new-build data centers, standardization of reference designs and of connectors/leak detection spanning servers, racks, CDUs, and piping is advancing rapidly. Demand to retrofit existing air-cooled-only floors to liquid cooling is also growing, bringing intermediate-stage technologies like the rear-door heat exchanger (RDHx), which mounts a water-cooled coil on the rack's rear door, back into the spotlight.

Second, as regulatory/social pressure on water use strengthens, WUE has risen to an evaluation axis no less than PUE. In water-scarce regions, compromises are increasing in which an air-cooled chiller or closed-loop approach is chosen instead of an evaporative cooling tower to save water while spending slightly more power, giving rise to a site-by-site "power vs. water" optimization problem. Third, as some fluorinated fluids used in two-phase immersion come under PFAS (perfluoroalkyl substances) regulation, the material supply chain is being reshaped — manufacturers scaling back or pivoting their related fluid businesses — so the regulatory and long-term-procurement risk of the fluid must always be reviewed together when adopting immersion. Fourth, reusing captured heat for district heating, etc. (ERF) and carbon-aware operation that shifts workloads to lower-carbon-intensity time windows are being combined with cooling/power strategy. However, these trends vary greatly by region and operator, so a prudent attitude of confirming concrete figures and regulatory timing against the latest sources is needed.

6. Considerations and Implications

Because data-center cooling is not a single technology choice but a multi-objective optimization problem entangling density, site, cost, and sustainability, the following must be judged comprehensively from the professional-engineer viewpoint.

  • Density-based method selection (application strategy): The cooling method must be decided by power density per rack, not by "fashion." Mid-to-low density (~20 kW) is adequately served by air cooling + free cooling, AI workloads exceeding 40 kW call for reviewing D2C, and ultra-high-density/special purposes call for immersion; design to the density band, while mixing by zone even within one site is realistic.
  • Trade-off of power, water, and cost: Choosing evaporative cooling for PUE alone worsens WUE, and using an air-cooled chiller to save water increases power. The balance point between TCO and sustainability must be found by jointly factoring in the site's climate, water resources, electricity rates, and carbon regulation; optimizing a single metric becomes a trap.
  • Reliability / operational risk management: The moment liquid is introduced, leakage becomes the greatest risk, so leak detection, auto shutoff, redundant piping, and no-downtime maintenance procedures must be reflected from the design stage. Immersion also entails operational risks of maintenance workflow, workforce skill, and fluid procurement, so an organization's operational capability must be included as a variable in cooling-method selection.
  • Outlook and linked technologies: Cooling evolves interlocked with power (intake/distribution/ESS), network (high-density packing), and semiconductors (chip heat). Since waste-heat reuse, carbon-aware operation, and water saving are likely to become standard under regulatory and ESG pressure, governance is required that elevates cooling from a "cost center" to part of an energy/carbon strategy.

References

In one line: Data-center cooling carries a heat chain from chip to outside using air, water, and dielectric fluid; AI-driven densification shifts its center from air cooling to direct-to-chip (D2C) and immersion, and the crux is balancing power, water, and carbon through PUE, WUE, and waste-heat reuse.