Rack-Level Liquid Cooling for AI & High-Density Data Centers
As AI and HPC workloads drive higher GPU power and rack densities, cooling increasingly needs to be designed at the rack level, rather than relying entirely on room-level air conditioning.
Rack-level liquid cooling brings liquid cooling infrastructure closer to the source of heat, allowing high-density racks to manage significantly greater thermal loads while reducing dependence on large volumes of room airflow.
This approach can include direct-to-chip cooling, coolant distribution units (CDUs), manifolds, rack-level coolant loops and heat-rejection infrastructure.
Want to know more? Contact us.
What Is Rack-Level Liquid Cooling?
Rack-level liquid cooling is a cooling architecture designed to remove heat within or directly around the server rack, rather than relying primarily on air circulation throughout the data center.
A typical architecture can connect:
GPU / CPU → Cold Plate → Server Manifold → Rack CDU → Facility Cooling Loop → Heat Rejection
The exact configuration depends on the equipment, rack density, cooling requirements and facility design.
Why Is Rack-Level Liquid Cooling Important?
Modern AI servers can generate substantially more heat than conventional enterprise equipment.
As rack power increases, traditional room-level air cooling can become increasingly difficult and inefficient.
Rack-level liquid cooling can help organizations:
• Support higher rack densities
• Manage high-power GPUs and CPUs
• Improve heat-transfer efficiency
• Reduce dependence on airflow
• Scale cooling alongside compute
• Prepare infrastructure for future AI workloads
The closer cooling gets to the heat source, the more effectively high-density thermal loads can be managed.
How Does Rack-Level Liquid Cooling Work?
A rack-level system typically consists of several connected components.
1. GPU or CPU
High-performance processors generate the majority of the thermal load.
2. Cold Plate
A liquid-cooled cold plate transfers heat directly from the processor into the coolant.
3. Server Manifold
Coolant is distributed between the rack's servers through supply and return connections.
4. Coolant Distribution Unit
A CDU manages coolant flow, temperature and heat transfer between the IT cooling loop and facility cooling infrastructure.
5. Facility Cooling Loop
Heat is transferred from the rack-side system into the facility cooling system.
6. Heat Rejection
The captured heat is ultimately rejected through chillers, dry coolers, heat exchangers or other facility systems.
Rack-Level vs Room-Level Cooling
Traditional data centers primarily cool equipment by controlling the temperature of the surrounding room.
Rack-level liquid cooling takes a different approach.
The two approaches can also be combined.
Hybrid cooling can use liquid cooling for the highest-power components while air cooling handles remaining equipment.
Rack-Level Liquid Cooling for AI & GPU Infrastructure
AI infrastructure is one of the key drivers behind rack-level liquid cooling.
High-density GPU racks can concentrate substantial electrical power and thermal load into a relatively small footprint.
Cooling therefore needs to be evaluated alongside:
• GPU platform
• GPU quantity
• Rack power
• IT load
• Coolant temperature
• CDU capacity
• Facility cooling capacity
• Heat rejection
• Future rack expansion
GPU density, power density and cooling capacity need to be designed as one system.
What Is a Rack CDU?
A Coolant Distribution Unit (CDU) is an important component in many rack-level liquid cooling architectures.
The CDU manages the flow of coolant between the technology cooling loop and the facility cooling loop.
Depending on the design, it can control:
• Coolant temperature
• Flow rate
• Pressure
• Heat transfer
• Monitoring and controls
CDUs can be deployed at different levels, including rack, row or facility configurations.
What Types of Rack-Level Liquid Cooling Are Available?
Direct-to-Chip Cooling
Cold plates transfer heat directly from GPUs and CPUs into liquid.
This is increasingly relevant for high-density AI and HPC servers.
Rear-Door Heat Exchangers
A heat exchanger attached to the rear of a rack removes heat from the server exhaust before it enters the data-center environment.
Immersion Cooling
Servers or components are placed directly into dielectric cooling fluid.
Hybrid Cooling
Liquid cooling manages high-power components while air cooling handles lower-power equipment.
The right architecture depends on rack density, server design, facility infrastructure and deployment objectives.
Can Rack-Level Liquid Cooling Be Added to an Existing Data Center?
Potentially.
Rack-level liquid cooling can be considered for both new data centers and selected retrofit projects.
However, an existing facility needs to be evaluated for:
• Available cooling capacity
• Water or coolant infrastructure
• Power availability
• Rack design
• Floor loading
• Pipe routing
• CDU placement
• Heat rejection
• Controls and monitoring
A retrofit should therefore evaluate the entire cooling path, rather than simply installing liquid-cooled servers.
Does Rack-Level Liquid Cooling Use Water?
Not necessarily.
The coolant used at the rack level and the facility's overall water consumption are separate considerations.
A system can use closed-loop liquid cooling, while the facility-side heat rejection may use dry coolers or other water-efficient technologies.
For water-constrained locations, the complete architecture should be evaluated from:
Chip → Rack → CDU → Facility Loop → Heat Rejection
Is Rack-Level Liquid Cooling Better Than Air Cooling?
Not universally.
Air cooling remains effective for many data-center applications.
Rack-level liquid cooling becomes increasingly attractive when rack power and thermal density exceed what conventional air cooling can efficiently manage.
The right decision depends on:
• Rack power
• GPU density
• IT load
• Existing infrastructure
• Climate
• Water availability
• Energy objectives
• Future expansion
The objective isn't to replace air cooling everywhere.
It's to use the right cooling architecture for the required compute density.
Rack-Level Liquid Cooling for High-Density Compute
As AI and HPC workloads become more powerful, thermal management is moving closer to the rack and ultimately to the processor itself.
A modern architecture can connect:
GPU → Cold Plate → Rack Manifold → CDU → Facility Cooling → Heat Rejection
This creates an integrated thermal path capable of supporting increasingly dense computing environments.
The rack is becoming a critical part of the data-center cooling architecture.
Frequently Asked Questions
What is rack-level liquid cooling?
Rack-level liquid cooling removes heat from high-density servers using liquid cooling infrastructure located at or near the rack.
What is the difference between rack-level and direct-to-chip cooling?
Direct-to-chip describes how heat is captured at the processor. Rack-level cooling describes the broader cooling architecture serving the rack.
Does rack-level liquid cooling require a CDU?
Many liquid-cooled rack architectures use a CDU, although the exact configuration depends on the system design.
Can rack-level liquid cooling support GPUs?
Yes. It is particularly relevant for high-density GPU and AI deployments where traditional air cooling becomes increasingly challenging.
Can rack-level liquid cooling be retrofitted?
Potentially, but the existing facility's power, cooling, piping, rack and heat-rejection infrastructure should be evaluated first.
Is rack-level liquid cooling waterless?
It can be part of a water-efficient or waterless cooling architecture, depending on the facility-side heat-rejection system.
Planning High-Density AI or HPC Infrastructure?
Rack-level cooling should be designed alongside GPU capacity, power, networking and facility cooling, rather than treated as a standalone equipment decision.
Whether you're planning a new AI data center or upgrading existing infrastructure, the right architecture starts with the rack's actual thermal and power requirements.
Need to Evaluate Rack-Level Liquid Cooling?
Want to know more? Contact us.
