NVIDIA Blackwell’s Power Consumption Spurs Cooling Needs; Liquid Cooling to Hit 10% by 2024

The demand for high-speed computing is on the rise, and as a result, more effective cooling solutions for AI servers are becoming increasingly important. According to a recent report from TrendForce, nVidia is gearing up to launch its next-generation Blackwell platform by the end of 2024. This new platform is expected to drive the adoption of Liquid Cooling solutions in AI server data centers, potentially pushing the penetration rate to 10%.

The NVIDIA Blackwell platform, set to officially launch in 2025, will replace the current Hopper platform and become the dominant solution for NVIDIA’s high-end GPUs. In fact, it is projected to account for a whopping 83% of all high-end products. The B200 and GB200 models, designed for maximum efficiency, are high-performance AI server models that consume over 1,000 W of power each. The HGX models will house 8 GPUs, while the NVL models will support either 36 or 72 GPUs per rack. This significant increase in GPU density will undoubtedly fuel the growth of the liquid cooling supply chain for AI servers.

One of the key factors driving the need for liquid cooling solutions is the increasing thermal design power (TDP) of server chips. For instance, the TDP of the B200 chip is expected to reach a staggering 1,000 W, rendering traditional air cooling solutions inadequate. The TDP of the GB200 NVL36 and NVL72 complete rack systems is projected to reach 70kW and nearly 140kW, respectively. These power levels necessitate advanced liquid cooling solutions to effectively manage the heat generated by these powerful chips.

TrendForce predicts that the GB200 NVL36 architecture will initially utilize a combination of air and liquid cooling solutions. However, due to higher cooling demands, the NVL72 will primarily rely on liquid cooling. The current liquid cooling supply chain for GB200 rack systems consists of five major components: cold plates, coolant distribution units (CDUs), manifolds, quick disconnects (QDs), and rear door heat exchangers (RDHx). The CDU, in particular, plays a critical role in regulating coolant flow to maintain rack temperatures within the designated TDP range, preventing damage to the components. Vertiv is currently the main supplier of CDUs for NVIDIA AI solutions, with other companies like Chicony, Auras, Delta, and CoolIT also undergoing continuous testing.

In 2025, NVIDIA plans to target cloud service providers (CSPs) and enterprise customers with a range of AI server configurations, including the HGX, GB200 Rack, and MGX. The expected shipment ratios for these configurations are 5:4:1, respectively. The HGX platform will seamlessly transition from the existing Hopper platform, allowing CSPs and large enterprise customers to adopt it quickly. The GB200 rack AI server solution, on the other hand, will primarily focus on the hyperscale CSP market. TrendForce predicts that NVIDIA will introduce the NVL36 configuration by the end of 2024 to quickly enter the market, with the more complex NVL72 expected to launch in 2025.

TrendForce forecasts that in 2025, GB200 NVL36 shipments will reach 60,000 racks, with Blackwell GPU usage totaling between 2.1 and 2.2 million units. However, there are a few variables that could impact the adoption of the GB200 Rack by end customers. For instance, the NVL72’s power consumption of around 140kW per rack requires sophisticated liquid cooling solutions, which can be challenging to implement. Additionally, liquid-cooled rack designs are better suited for new CSP data centers and involve complex planning processes. Some CSPs might also prefer to avoid being tied to a single supplier’s specifications and instead opt for HGX or MGX models with x86 CPU architectures. Alternatively, they may choose to expand their self-developed ASIC AI server infrastructure for lower costs or specific AI applications.

In conclusion, the launch of NVIDIA’s Blackwell platform in 2025 is set to revolutionize the AI server market. With its high-performance GPUs and increased power consumption, the demand for effective cooling solutions, particularly liquid cooling, is expected to soar. However, the adoption of liquid-cooled rack designs will depend on various factors, including the specific needs of end customers and the complexity involved in implementing these solutions. It will be interesting to see how the market evolves and whether liquid cooling becomes the go-to choice for AI server data centers in the coming years.

Background Information

---

About nVidia:

NVIDIA has firmly established itself as a leader in the realm of client computing, continuously pushing the boundaries of innovation in graphics and AI technologies. With a deep commitment to enhancing user experiences, NVIDIA's client computing business focuses on delivering solutions that power everything from gaming and creative workloads to enterprise applications. for its GeForce graphics cards, the company has redefined high-performance gaming, setting industry standards for realistic visuals, fluid frame rates, and immersive experiences. Complementing its gaming expertise, NVIDIA's Quadro and NVIDIA RTX graphics cards cater to professionals in design, content creation, and scientific fields, enabling real-time ray tracing and AI-driven workflows that elevate productivity and creativity to unprecedented heights. By seamlessly integrating graphics, AI, and software, NVIDIA continues to shape the landscape of client computing, fostering innovation and immersive interactions in a rapidly evolving digital world.

  

Latest Articles about nVidia

Technology Explained

---

Blackwell: Blackwell is an AI computing architecture designed to supercharge tasks like training large language models. These powerful GPUs boast features like a next-gen Transformer Engine and support for lower-precision calculations, enabling them to handle complex AI workloads significantly faster and more efficiently than before. While aimed at data centers, the innovations within Blackwell are expected to influence consumer graphics cards as well

Latest Articles about Blackwell

---

CPU: The Central Processing Unit (CPU) is the brain of a computer, responsible for executing instructions and performing calculations. It is the most important component of a computer system, as it is responsible for controlling all other components. CPUs are used in a wide range of applications, from desktop computers to mobile devices, gaming consoles, and even supercomputers. CPUs are used to process data, execute instructions, and control the flow of information within a computer system. They are also used to control the input and output of data, as well as to store and retrieve data from memory. CPUs are essential for the functioning of any computer system, and their applications in the computer industry are vast.

Latest Articles about CPU

---

GPU: GPU stands for Graphics Processing Unit and is a specialized type of processor designed to handle graphics-intensive tasks. It is used in the computer industry to render images, videos, and 3D graphics. GPUs are used in gaming consoles, PCs, and mobile devices to provide a smooth and immersive gaming experience. They are also used in the medical field to create 3D models of organs and tissues, and in the automotive industry to create virtual prototypes of cars. GPUs are also used in the field of artificial intelligence to process large amounts of data and create complex models. GPUs are becoming increasingly important in the computer industry as they are able to process large amounts of data quickly and efficiently.

Latest Articles about GPU

---

Liquid Cooling: Liquid cooling is a technology used to cool down computer components, such as processors, graphics cards, and other components that generate a lot of heat. It works by circulating a liquid coolant, such as water or a special coolant, through a series of pipes and radiators. The liquid absorbs the heat from the components and then dissipates it into the air. This technology is becoming increasingly popular in the computer industry due to its ability to provide more efficient cooling than traditional air cooling methods. Liquid cooling can also be used to overclock components, allowing them to run at higher speeds than their rated speeds. This technology is becoming increasingly popular in the gaming industry, as it allows gamers to get the most out of their hardware.

Latest Articles about Liquid Cooling