Blackwell Shipments Set to Boost CoWoS Capacity, Anticipated 70% Surge by 2025

According to a recent report from TrendForce, nVidia’s Hopper H100 is finally experiencing a reduction in shortages in the first quarter of 2024. This is great news for gamers and tech enthusiasts who have been eagerly awaiting the availability of this high-performance GPU. But that’s not all, as the report also reveals that NVIDIA’s H200 from the same platform is expected to gradually ramp up in the second quarter, followed by the introduction of the Blackwell platform in the third quarter. The Blackwell platform will initially target data center customers in the fourth quarter.

While the Blackwell platform is projected to account for less than 10% of the total high-end GPU market, it is still a significant development in the industry. The chips on the Blackwell platform, such as the B100, have a die size twice that of the H100, indicating a substantial increase in processing power. As supply chain integration progresses, the Blackwell platform is expected to gain momentum in 2025.

TSMC’s CoWoS, which plays a crucial role in the production of these advanced GPUs, is projected to see a substantial increase in capacity. By 2024, the total capacity is expected to grow by 150%, with NVIDIA’s demand occupying nearly half of this capacity. Additionally, the evolution of NVIDIA’s GPU platform will see an increase in HBM capacity. The H100 currently utilizes 80 GB of HBM3, while the 2025 B200 will feature a whopping 288 GB of HBM3E per chip. This represents a significant 3-4 fold increase in capacity.

In terms of adoption, industry giants like Microsoft, Meta (formerly Facebook), and AWS are leading the way by being the first to adopt GB200CSPs. These companies are expected to exceed a combined shipment volume of 30,000 racks by 2025. On the other hand, Google seems to be prioritizing the expansion of its TPU AI infrastructure.

What sets the Blackwell platform apart from traditional approaches like HGX or MGX single AI servers is its whole-rack configuration. NVIDIA has integrated its own CPU, GPU, NVLink, and InfiniBand high-speed networking technologies into this platform, offering a comprehensive solution for AI workloads. The GB200, which is part of the Blackwell platform, comes in two configurations: NVL36 and NVL72. Each rack features either 36 or 72 GPUs, with the NVL72 being the primary configuration NVIDIA is pushing for. However, due to the complexity of its design, the NVL36 will be introduced first by the end of 2024 for initial trials to ensure a smooth market entry.

Looking ahead, TrendForce predicts a significant rise in AI server demand from tier-2 data centers and the government, reaching 50% in 2024. NVIDIA’s FY1Q25 data already indicates an increase in demand from enterprise customers, sovereign AI clouds, and tier-2 data centers. In 2023, hyper CSPs accounted for approximately 65% of the global high-end AI server market demand, but with the easing supply of GPUs in the first half of 2024, server brands like Dell and HPE are expected to expand their demand share to 19%. Additionally, tier-2 data centers and sovereign AI clouds will see their combined demand share rise to 31%, driven by the sensitive and privacy-focused nature of AI training and applications. This segment is expected to play a crucial role in driving the growth of AI server shipments in the future.

Background Information

---

About Dell:

Dell is a globally technology leader providing comprehensive solutions in the field of hardware, software, and services. for its customizable computers and enterprise solutions, Dell offers a diverse range of laptops, desktops, servers, and networking equipment. With a commitment to innovation and customer satisfaction, Dell caters to a wide range of consumer and business needs, making it a important player in the tech industry.

  

Latest Articles about Dell

---

About Google:

Google, founded by Larry Page and Sergey Brin in 1998, is a multinational technology company known for its internet-related services and products. Initially for its search engine, Google has since expanded into various domains including online advertising, cloud computing, software development, and hardware devices. With its innovative approach, Google has introduced influential products such as Google Search, Android OS, Google Maps, and Google Drive. The company's commitment to research and development has led to advancements in artificial intelligence and machine learning.

  

Latest Articles about Google

---

About Microsoft:

Microsoft, founded by Bill Gates and Paul Allen in 1975 in Redmond, Washington, USA, is a technology giant known for its wide range of software products, including the Windows operating system, Office productivity suite, and cloud services like Azure. Microsoft also manufactures hardware, such as the Surface line of laptops and tablets, Xbox gaming consoles, and accessories.

  

Latest Articles about Microsoft

---

About nVidia:

NVIDIA has firmly established itself as a leader in the realm of client computing, continuously pushing the boundaries of innovation in graphics and AI technologies. With a deep commitment to enhancing user experiences, NVIDIA's client computing business focuses on delivering solutions that power everything from gaming and creative workloads to enterprise applications. for its GeForce graphics cards, the company has redefined high-performance gaming, setting industry standards for realistic visuals, fluid frame rates, and immersive experiences. Complementing its gaming expertise, NVIDIA's Quadro and NVIDIA RTX graphics cards cater to professionals in design, content creation, and scientific fields, enabling real-time ray tracing and AI-driven workflows that elevate productivity and creativity to unprecedented heights. By seamlessly integrating graphics, AI, and software, NVIDIA continues to shape the landscape of client computing, fostering innovation and immersive interactions in a rapidly evolving digital world.

  

Latest Articles about nVidia

---

About TSMC:

TSMC, or Taiwan Semiconductor Manufacturing Company, is a semiconductor foundry based in Taiwan. Established in 1987, TSMC is a important player in the global semiconductor industry, specializing in the manufacturing of semiconductor wafers for a wide range of clients, including technology companies and chip designers. The company is known for its semiconductor fabrication processes and plays a critical role in advancing semiconductor technology worldwide.

  

Latest Articles about TSMC

Technology Explained

---

AWS: Amazon Web Services (AWS) is a cloud platform powered by Amazon that enables users to access cloud computing services, such as storage, data analytics, and distributed computing. It offers users the ability to utilize both on-demand and pay-as-you-go computing services, making it a great option for the computer industry. It offers a wide range of services with great flexibility for a variety of uses. It can help companies build powerful web and mobile applications, run large-scale analytics, quickly provision servers and other services, design sophisticated architectures for data storage, and more. AWS provides access to a wide range of services such as virtualization, storage, database, monitoring, analytics, and other services that can help organizations increase agility, manage complexity, and remain on the cutting edge of technology. Many big and famous organizations use AWS services to give them a competitive edge, and more and more companies are turning to this service for their computer needs.

Latest Articles about AWS

---

Blackwell: Blackwell is an AI computing architecture designed to supercharge tasks like training large language models. These powerful GPUs boast features like a next-gen Transformer Engine and support for lower-precision calculations, enabling them to handle complex AI workloads significantly faster and more efficiently than before. While aimed at data centers, the innovations within Blackwell are expected to influence consumer graphics cards as well

Latest Articles about Blackwell

---

CoWoS: CoWoS, or Chip-on-Wafer-on-Substrate, is a recent advancement in chip packaging that allows for more powerful processors in a compact size. This technology stacks multiple chips on a silicon interposer, enabling denser connections and improved performance. Developed for high-performance computing, CoWoS promises faster processing, lower power consumption, and the ability to pack more processing power into smaller devices.

Latest Articles about CoWoS

---

CPU: The Central Processing Unit (CPU) is the brain of a computer, responsible for executing instructions and performing calculations. It is the most important component of a computer system, as it is responsible for controlling all other components. CPUs are used in a wide range of applications, from desktop computers to mobile devices, gaming consoles, and even supercomputers. CPUs are used to process data, execute instructions, and control the flow of information within a computer system. They are also used to control the input and output of data, as well as to store and retrieve data from memory. CPUs are essential for the functioning of any computer system, and their applications in the computer industry are vast.

Latest Articles about CPU

---

GPU: GPU stands for Graphics Processing Unit and is a specialized type of processor designed to handle graphics-intensive tasks. It is used in the computer industry to render images, videos, and 3D graphics. GPUs are used in gaming consoles, PCs, and mobile devices to provide a smooth and immersive gaming experience. They are also used in the medical field to create 3D models of organs and tissues, and in the automotive industry to create virtual prototypes of cars. GPUs are also used in the field of artificial intelligence to process large amounts of data and create complex models. GPUs are becoming increasingly important in the computer industry as they are able to process large amounts of data quickly and efficiently.

Latest Articles about GPU

---

HBM3E: HBM3E is the latest generation of high-bandwidth memory (HBM), a type of DRAM that is designed for artificial intelligence (AI) applications. HBM3E offers faster data transfer rates, higher density, and lower power consumption than previous HBM versions. HBM3E is developed by SK Hynix, a South Korean chipmaker, and is expected to enter mass production in 2024. HBM3E can achieve a speed of 1.15 TB/s and a capacity of 64 GB per stack. HBM3E is suitable for AI systems that require large amounts of data processing, such as deep learning, machine learning, and computer vision.

Latest Articles about HBM3E