TSMC Forecasts 1.5-Year Scarcity of Compute GPUs, Awaiting Relief

Reports of a shortage in compute GPUs used for AI and HPC servers have been circulating in recent months, as the demand for GPUs to power generative AI applications continues to skyrocket. TSMC, one of the leading chip manufacturers, has acknowledged that the main bottleneck in GPU supply is its CoWoS packaging capacity, which is utilized by nearly all players in the AI and HPC industries. While TSMC is expanding its CoWoS capacity, it anticipates that the shortage will persist for approximately 1.5 years.

Mark Liu, Chairman of TSMC, explained, “It’s not a shortage of AI chips; it’s a shortage of our CoWoS capacity. Currently, we are unable to meet 100% of our customers’ demands, but we strive to support around 80%. We view this as a temporary situation, and once we expand our advanced chip packaging capacity, it should alleviate the issue within 1.5 years.”

TSMC is responsible for producing the majority of processors that power popular AI services, including compute GPUs like AMD’s Instinct MI250 and nVidia’s A100 and H100, as well as FPGAs, specialized ASICs from companies like d-Matrix and Tenstorrent, and proprietary processors from cloud service providers such as AWS’s Trainium and Inferentia, and Google’s TPU.

It’s worth noting that compute GPUs, FPGAs, and accelerators from cloud service providers all utilize HBM memory to achieve optimal bandwidth and rely on TSMC’s interposer-based chip-on-wafer-on-Substrate packaging. While other outsourced semiconductor assembly and test (OSAT) companies like ASE and Amkor also offer similar packaging technologies, TSMC seems to be receiving the majority of orders, making it difficult for them to keep up with the demand for their packaging services.

Industry analysts suggest that OSATs are less inclined to offer advanced packaging services due to the significant capital investment and higher financial risks associated with it compared to traditional packaging. For instance, if there’s an issue with a mainstream processor on an organic substrate, an OSAT would only lose one chip. However, if a package carrying four chiplets and eight HBM memory stacks encounters a problem, the losses could amount to hundreds or even thousands of dollars. Given that OSATs do not generate substantial margins from producing chiplets, they are more cautious about expanding their advanced packaging capacity, despite the higher costs involved.

Like its competitors, TSMC is investing billions of dollars in upcoming advanced packaging facilities. Recently, the company announced plans to spend approximately $2.9 billion on a packaging fab, rumored to be operational by 2027.

“We are rapidly increasing our capacity,” stated C.C. Wei, CEO of TSMC, during the company’s earnings call earlier this year. “We expect some relief from these constraints next year, likely towards the end of the year. […] I won’t provide you with the exact number [of processed wafers capacity], but CoWoS capacity will double in 2024 compared to 2023.”

Source: Nikkei

Background Information

---

About AMD: AMD, a large player in the semiconductor industry is known for its powerful processors and graphic solutions, AMD has consistently pushed the boundaries of performance, efficiency, and user experience. With a customer-centric approach, the company has cultivated a reputation for delivering high-performance solutions that cater to the needs of gamers, professionals, and general users. AMD's Ryzen series of processors have redefined the landscape of desktop and laptop computing, offering impressive multi-core performance and competitive pricing that has challenged the dominance of its competitors. Complementing its processor expertise, AMD's Radeon graphics cards have also earned accolades for their efficiency and exceptional graphical capabilities, making them a favored choice among gamers and content creators. The company's commitment to innovation and technology continues to shape the client computing landscape, providing users with powerful tools to fuel their digital endeavors.

  

---

About Google: Google, founded by Larry Page and Sergey Brin in 1998, is a multinational technology company known for its internet-related services and products. Initially for its search engine, Google has since expanded into various domains including online advertising, cloud computing, software development, and hardware devices. With its innovative approach, Google has introduced influential products such as Google Search, Android OS, Google Maps, and Google Drive. The company's commitment to research and development has led to advancements in artificial intelligence and machine learning.

  

---

About nVidia: NVIDIA has firmly established itself as a leader in the realm of client computing, continuously pushing the boundaries of innovation in graphics and AI technologies. With a deep commitment to enhancing user experiences, NVIDIA's client computing business focuses on delivering solutions that power everything from gaming and creative workloads to enterprise applications. for its GeForce graphics cards, the company has redefined high-performance gaming, setting industry standards for realistic visuals, fluid frame rates, and immersive experiences. Complementing its gaming expertise, NVIDIA's Quadro and NVIDIA RTX graphics cards cater to professionals in design, content creation, and scientific fields, enabling real-time ray tracing and AI-driven workflows that elevate productivity and creativity to unprecedented heights. By seamlessly integrating graphics, AI, and software, NVIDIA continues to shape the landscape of client computing, fostering innovation and immersive interactions in a rapidly evolving digital world.

  

---

About TSMC: TSMC, or Taiwan Semiconductor Manufacturing Company, is a semiconductor foundry based in Taiwan. Established in 1987, TSMC is a important player in the global semiconductor industry, specializing in the manufacturing of semiconductor wafers for a wide range of clients, including technology companies and chip designers. The company is known for its semiconductor fabrication processes and plays a critical role in advancing semiconductor technology worldwide.

  

Technology Explained

---

AWS: Amazon Web Services (AWS) is a cloud platform powered by Amazon that enables users to access cloud computing services, such as storage, data analytics, and distributed computing. It offers users the ability to utilize both on-demand and pay-as-you-go computing services, making it a great option for the computer industry. It offers a wide range of services with great flexibility for a variety of uses. It can help companies build powerful web and mobile applications, run large-scale analytics, quickly provision servers and other services, design sophisticated architectures for data storage, and more. AWS provides access to a wide range of services such as virtualization, storage, database, monitoring, analytics, and other services that can help organizations increase agility, manage complexity, and remain on the cutting edge of technology. Many big and famous organizations use AWS services to give them a competitive edge, and more and more companies are turning to this service for their computer needs.

---

chiplets: Chiplets are a new type of technology that is revolutionizing the computer industry. They are small, modular components that can be used to build a variety of computer systems. Chiplets are designed to be highly efficient and cost-effective, allowing for the creation of powerful and complex systems without the need for large, expensive components. They are also highly customizable, allowing for the creation of systems tailored to specific needs. Chiplets are being used in a variety of applications, from high-end gaming PCs to embedded systems and even supercomputers. They are also being used to create powerful AI systems, allowing for the development of more advanced and intelligent machines. Chiplets are revolutionizing the computer industry, allowing for the creation of powerful and efficient systems at a fraction of the cost.

---

GPU: GPU stands for Graphics Processing Unit and is a specialized type of processor designed to handle graphics-intensive tasks. It is used in the computer industry to render images, videos, and 3D graphics. GPUs are used in gaming consoles, PCs, and mobile devices to provide a smooth and immersive gaming experience. They are also used in the medical field to create 3D models of organs and tissues, and in the automotive industry to create virtual prototypes of cars. GPUs are also used in the field of artificial intelligence to process large amounts of data and create complex models. GPUs are becoming increasingly important in the computer industry as they are able to process large amounts of data quickly and efficiently.