AI Server Demand Surge to Propel 2024 Market Value to $187B; Dominates Server Market

In the fast-paced world of AI servers, it seems that the demand for advanced technology is showing no signs of slowing down. According to TrendForce’s latest industry report, major cloud service providers (CSPs) and brand clients are expected to continue their high demand for AI servers well into 2024. This is great news for companies like TSMC, SK hynix, Samsung, and Micron, who have been gradually expanding their production to meet this growing need. As a result, shortages in the second quarter of 2024 have significantly eased, leading to a decrease in lead time for nVidia’s flagship H100 solution from a whopping 40-50 weeks to less than 16 weeks. Now that’s what I call progress!

TrendForce estimates that AI server shipments in the second quarter will experience a nearly 20% increase compared to the previous quarter. In fact, they have revised their annual shipment forecast up to a staggering 1.67 million units—a remarkable 41.5% year-over-year growth. It’s clear that major CSPs are putting their budgets towards procuring AI servers, which is causing a bit of a squeeze on the growth momentum of general servers. While the annual growth rate of general server shipments is a modest 1.9%, the growth rate for AI servers is skyrocketing.

In terms of market value, AI servers are making a significant contribution to revenue growth compared to general servers. TrendForce projects that the market value of AI servers will exceed $187 billion in 2024, with a growth rate of 69%. This means that AI servers will account for a whopping 65% of the total server market value. That’s quite an impressive figure!

When it comes to AI chip suppliers for AI servers, NVIDIA takes the crown with the highest market share—approaching 90% for GPU-equipped AI servers. On the other hand, AMD’s market share is only about 8%. However, if we consider all AI chips used in AI servers (GPU, ASIC, FPGA), NVIDIA’s market share this year is around 64%. It’s clear that NVIDIA is a dominant player in this space, but it will be interesting to see if other competitors can catch up.

Looking ahead, TrendForce predicts that the demand for advanced AI servers will remain strong through 2025. One of the key drivers of this demand is NVIDIA’s next-generation Blackwell platform, which is set to replace the Hopper platform as the market mainstream. This transition will also drive the demand for CoWoS (Chip-on-Wafer-on-Substrate) and HBM (High Bandwidth Memory). For example, NVIDIA’s B100 chip will be double the size of the H100, requiring more CoWoS. The production capacity of major supplier TSMC’s CoWoS is estimated to reach an impressive 550-600K units by the end of 2025, with a growth rate approaching 80%. It’s clear that the future of AI servers is looking bright!

In terms of memory, the mainstream H100 in 2024 will be equipped with 80 GB HBM3 (High Bandwidth Memory 3). However, by 2025, main chips like NVIDIA’s Blackwell Ultra or AMD’s MI350 are expected to be equipped with up to a whopping 288 GB of HBM3E (High Bandwidth Memory 3e), tripling the unit usage. With such strong ongoing demand in the AI server market, the overall supply of HBM is expected to double by 2025. It’s incredible to think about the amount of memory that will be powering these advanced AI servers.

All in all, it’s an exciting time for the AI server industry. With high demand from major CSPs and brand clients, eased shortages, and impressive growth rates, the future looks promising. And with NVIDIA’s Blackwell platform on the horizon, we can expect even more advancements and innovations in the world of AI servers. It’s safe to say that the AI server market is one to watch!

Background Information

---

About AMD:

AMD, a large player in the semiconductor industry is known for its powerful processors and graphic solutions, AMD has consistently pushed the boundaries of performance, efficiency, and user experience. With a customer-centric approach, the company has cultivated a reputation for delivering high-performance solutions that cater to the needs of gamers, professionals, and general users. AMD's Ryzen series of processors have redefined the landscape of desktop and laptop computing, offering impressive multi-core performance and competitive pricing that has challenged the dominance of its competitors. Complementing its processor expertise, AMD's Radeon graphics cards have also earned accolades for their efficiency and exceptional graphical capabilities, making them a favored choice among gamers and content creators. The company's commitment to innovation and technology continues to shape the client computing landscape, providing users with powerful tools to fuel their digital endeavors.

  

Latest Articles about AMD

---

About nVidia:

NVIDIA has firmly established itself as a leader in the realm of client computing, continuously pushing the boundaries of innovation in graphics and AI technologies. With a deep commitment to enhancing user experiences, NVIDIA's client computing business focuses on delivering solutions that power everything from gaming and creative workloads to enterprise applications. for its GeForce graphics cards, the company has redefined high-performance gaming, setting industry standards for realistic visuals, fluid frame rates, and immersive experiences. Complementing its gaming expertise, NVIDIA's Quadro and NVIDIA RTX graphics cards cater to professionals in design, content creation, and scientific fields, enabling real-time ray tracing and AI-driven workflows that elevate productivity and creativity to unprecedented heights. By seamlessly integrating graphics, AI, and software, NVIDIA continues to shape the landscape of client computing, fostering innovation and immersive interactions in a rapidly evolving digital world.

  

Latest Articles about nVidia

---

About Samsung:

Samsung, a South Korean multinational conglomerate, has established itself as a global leader in various industries, including electronics, technology, and more. Founded in 1938, Samsung's influence spans from smartphones and consumer electronics to semiconductors and home appliances. With a commitment to innovation, Samsung has contributed products like the Galaxy series of smartphones, QLED TVs, and SSDs that have revolutionized the way we live and work.

  

Latest Articles about Samsung

---

About SK hynix:

SK Hynix is a important South Korean semiconductor company known for its innovative contributions to the global technology landscape. Specializing in the production of memory solutions, SK Hynix has played a vital role in shaping the semiconductor industry. With a commitment to research and development, they have continuously pushed the boundaries of memory technology, resulting in products that power various devices and applications.

  

Latest Articles about SK hynix

---

About TSMC:

TSMC, or Taiwan Semiconductor Manufacturing Company, is a semiconductor foundry based in Taiwan. Established in 1987, TSMC is a important player in the global semiconductor industry, specializing in the manufacturing of semiconductor wafers for a wide range of clients, including technology companies and chip designers. The company is known for its semiconductor fabrication processes and plays a critical role in advancing semiconductor technology worldwide.

  

Latest Articles about TSMC

Technology Explained

---

Blackwell: Blackwell is an AI computing architecture designed to supercharge tasks like training large language models. These powerful GPUs boast features like a next-gen Transformer Engine and support for lower-precision calculations, enabling them to handle complex AI workloads significantly faster and more efficiently than before. While aimed at data centers, the innovations within Blackwell are expected to influence consumer graphics cards as well

Latest Articles about Blackwell

---

CoWoS: CoWoS, or Chip-on-Wafer-on-Substrate, is a recent advancement in chip packaging that allows for more powerful processors in a compact size. This technology stacks multiple chips on a silicon interposer, enabling denser connections and improved performance. Developed for high-performance computing, CoWoS promises faster processing, lower power consumption, and the ability to pack more processing power into smaller devices.

Latest Articles about CoWoS

---

FPGA: Field Programmable Gate Arrays (FPGAs) are a type of technology used in the computer industry. They are integrated circuits that can be programmed to perform specific tasks. FPGAs are used in a variety of applications, including digital signal processing, networking, and embedded systems. They are also used in the development of artificial intelligence and machine learning algorithms. FPGAs are advantageous because they can be reprogrammed to perform different tasks, allowing for greater flexibility and faster development times. Additionally, FPGAs are more energy efficient than traditional processors, making them ideal for applications that require low power consumption.

Latest Articles about FPGA

---

GPU: GPU stands for Graphics Processing Unit and is a specialized type of processor designed to handle graphics-intensive tasks. It is used in the computer industry to render images, videos, and 3D graphics. GPUs are used in gaming consoles, PCs, and mobile devices to provide a smooth and immersive gaming experience. They are also used in the medical field to create 3D models of organs and tissues, and in the automotive industry to create virtual prototypes of cars. GPUs are also used in the field of artificial intelligence to process large amounts of data and create complex models. GPUs are becoming increasingly important in the computer industry as they are able to process large amounts of data quickly and efficiently.

Latest Articles about GPU

---

HBM3E: HBM3E is the latest generation of high-bandwidth memory (HBM), a type of DRAM that is designed for artificial intelligence (AI) applications. HBM3E offers faster data transfer rates, higher density, and lower power consumption than previous HBM versions. HBM3E is developed by SK Hynix, a South Korean chipmaker, and is expected to enter mass production in 2024. HBM3E can achieve a speed of 1.15 TB/s and a capacity of 64 GB per stack. HBM3E is suitable for AI systems that require large amounts of data processing, such as deep learning, machine learning, and computer vision.

Latest Articles about HBM3E

---

Substrate: The technology substrate is a layer of material that provides a foundation for the components of a computer system. It is the foundation upon which the components of a computer system are built. It is usually made of a material such as silicon, which is a semiconductor material. The technology substrate is used to create the circuits and pathways that allow the components of a computer system to communicate with each other. It is also used to create the physical structure of the computer system, such as the motherboard, memory, and other components. The technology substrate is essential for the functioning of a computer system, as it provides the necessary pathways for the components to communicate with each other. It is also used to create the physical structure of the computer system, such as the motherboard, memory, and other components.

Latest Articles about Substrate