NVIDIA’s MLPerf Training Results Reveal Remarkable Performance and Flexibility

nVidia’s accelerated computing platform continues to push the boundaries of performance, as demonstrated in the latest MLPerf Training v4.0 benchmarks. In fact, NVIDIA has more than tripled its performance on the large language model (LLM) benchmark, based on GPT-3 175B, compared to its record-setting submission from last year. How did they achieve this remarkable feat? By utilizing an AI supercomputer featuring 11,616 NVIDIA H100 Tensor Core GPUs connected with NVIDIA Quantum-2 InfiniBand networking.

But it’s not just about scale. NVIDIA’s full-stack engineering and extensive optimizations have played a crucial role in achieving these impressive results. The scalability of the NVIDIA AI platform allows for faster training of massive AI models like GPT-3 175B, opening up significant business opportunities.

For instance, NVIDIA highlighted in its recent earnings call how LLM service providers can turn a single dollar invested into seven dollars in just four years by running the Llama 3 70B model on NVIDIA HGX H200 servers. This assumes a price of $0.60 per million tokens and an HGX H200 server throughput of 24,000 tokens per second. These numbers demonstrate the tangible value that AI performance can bring to businesses.

So, what makes the new NVIDIA H200 Tensor GPU so powerful? Building upon the strength of the Hopper architecture, it boasts 141 GB of HBM3 memory and over 40% more memory bandwidth compared to its predecessor, the H100 GPU. In its MLPerf Training debut, the H200 extended the H100’s performance by up to 47%, pushing the boundaries of what’s possible in AI training.

But it’s not just about hardware. NVIDIA’s software stack has also undergone numerous optimizations, resulting in up to 27% faster performance compared to just one year ago using a 512 H100 GPU configuration. This highlights the significant impact that continuous software enhancements can have on performance, even with the same hardware.

One notable achievement is the nearly perfect scaling delivered by NVIDIA’s work. As the number of GPUs increased by 3.2 times, going from 3,584 H100 GPUs last year to 11,616 H100 GPUs in this submission, the performance also scaled accordingly. This demonstrates the effectiveness of NVIDIA’s approach in maximizing performance gains with increased hardware resources.

LLM fine-tuning is another key industry workload that enterprises are increasingly focusing on. And once again, the NVIDIA platform excelled in this area. Scaling from eight to 1,024 GPUs, the largest-scale NVIDIA submission completed the benchmark in a record-breaking 1.5 minutes. This showcases the platform’s ability to handle LLM fine-tuning efficiently and effectively.

Stable Diffusion v2 training performance was also significantly accelerated by up to 80% at the same system scales as the previous round. These improvements are a result of the continuous enhancements made to the NVIDIA software stack, demonstrating the synergy between software and hardware in delivering top-tier performance.

When it comes to graph neural network (GNN) training based on R-GAT, both the H100 and H200 GPUs performed exceptionally well at small and large scales. The H200 delivered a 47% boost in single-node GNN training compared to its predecessor, further highlighting the powerful performance and high efficiency of NVIDIA GPUs across a wide range of AI applications.

NVIDIA’s AI ecosystem is thriving, as evidenced by the participation of 10 NVIDIA partners in the MLPerf benchmarks. Companies like ASUS, Dell Technologies, Fujitsu, Gigabyte, Hewlett Packard Enterprise, Lenovo, Oracle, Quanta Cloud Technology, Supermicro, and Sustainable Metal Cloud have all submitted impressive results, underscoring the widespread adoption and trust in NVIDIA’s AI platform throughout the industry.

MLCommons’ work in bringing benchmarking best practices to AI computing is crucial. By enabling peer-reviewed comparisons of AI and HPC platforms and keeping up with the rapid changes in AI computing, MLCommons provides valuable data that can guide important purchasing decisions.

And the future looks even more promising with the upcoming NVIDIA Blackwell platform, which promises next-level AI performance on trillion-parameter generative AI models for both training and inference. With NVIDIA’s continuous innovation and commitment to pushing the boundaries of AI computing, we can expect even more achievements in the near future.

Background Information

---

About ASUS:

ASUS, founded in 1989 by Ted Hsu, M.T. Liao, Wayne Hsieh, and T.H. Tung, has become a multinational tech giant known for its diverse hardware products. Spanning laptops, motherboards, graphics cards, and more, ASUS has gained recognition for its innovation and commitment to high-performance computing solutions. The company has a significant presence in gaming technology, producing popular products that cater to enthusiasts and professionals alike. With a focus on delivering and reliable technology, ASUS maintains its position as a important player in the industry.

  

Latest Articles about ASUS

---

About Dell:

Dell is a globally technology leader providing comprehensive solutions in the field of hardware, software, and services. for its customizable computers and enterprise solutions, Dell offers a diverse range of laptops, desktops, servers, and networking equipment. With a commitment to innovation and customer satisfaction, Dell caters to a wide range of consumer and business needs, making it a important player in the tech industry.

  

Latest Articles about Dell

---

About Fujitsu:

Fujitsu is a important Japanese technology company for its wide array of computing solutions. With a history dating back to 1935, Fujitsu excels in producing personal computers, laptops, and tablets that combine innovation and reliability. In addition to consumer-focused products, Fujitsu is a key player in enterprise solutions, offering servers, storage systems, and data center services. The company's emphasis on quality, advanced features, and IT services has solidified its position as a significant player in the global computing industry.

Latest Articles about Fujitsu

---

About Gigabyte:

Gigabyte Technology, a important player in the computer hardware industry, has established itself as a leading provider of innovative solutions and products catering to the ever-evolving needs of modern computing. With a strong emphasis on quality, performance, and technology, Gigabyte has gained recognition for its wide array of computer products. These encompass motherboards, graphics cards, laptops, desktop PCs, monitors, and other components that are integral to building high-performance systems. for their reliability and advanced features, Gigabyte's motherboards and graphics cards have become staples in the gaming and enthusiast communities, delivering the power and capabilities required for immersive gaming experiences and resource-intensive applications

  

Latest Articles about Gigabyte

---

About Lenovo:

Lenovo, formerly known as "Legend Holdings," is a important global technology company that offers an extensive portfolio of computers, smartphones, servers, and electronic devices. Notably, Lenovo acquired IBM's personal computer division, including the ThinkPad line of laptops, in 2005. With a strong presence in laptops and PCs, Lenovo's products cater to a wide range of consumer and business needs. Committed to innovation and quality, Lenovo delivers reliable and high-performance solutions, making it a significant player in the tech industry.

  

Latest Articles about Lenovo

---

About nVidia:

NVIDIA has firmly established itself as a leader in the realm of client computing, continuously pushing the boundaries of innovation in graphics and AI technologies. With a deep commitment to enhancing user experiences, NVIDIA's client computing business focuses on delivering solutions that power everything from gaming and creative workloads to enterprise applications. for its GeForce graphics cards, the company has redefined high-performance gaming, setting industry standards for realistic visuals, fluid frame rates, and immersive experiences. Complementing its gaming expertise, NVIDIA's Quadro and NVIDIA RTX graphics cards cater to professionals in design, content creation, and scientific fields, enabling real-time ray tracing and AI-driven workflows that elevate productivity and creativity to unprecedented heights. By seamlessly integrating graphics, AI, and software, NVIDIA continues to shape the landscape of client computing, fostering innovation and immersive interactions in a rapidly evolving digital world.

  

Latest Articles about nVidia

---

About Oracle:

Oracle Corporation is a important American multinational technology company founded in 1977 and headquartered in Redwood City, California. It's one of the world's largest software and cloud computing companies, known for its enterprise software products and services. Oracle specializes in developing and providing database management systems, cloud solutions, software applications, and hardware infrastructure. Their flagship product, the Oracle Database, is widely used in businesses and organizations worldwide. Oracle also offers a range of cloud services, including Infrastructure as a Service (IaaS), Platform as a Service (PaaS), and Software as a Service (SaaS).

  

Latest Articles about Oracle

---

About Supermicro:

Supermicro is a reputable American technology company founded in 1993 and headquartered in San Jose, California. Specializing in high-performance server and storage solutions, Supermicro has become a trusted name in the data center industry. The company offers a wide range of innovative and customizable server hardware, including motherboards, servers, storage systems, and networking equipment, catering to the needs of enterprise clients, cloud service providers, and businesses seeking reliable infrastructure solutions.

  

Latest Articles about Supermicro

Technology Explained

---

Blackwell: Blackwell is an AI computing architecture designed to supercharge tasks like training large language models. These powerful GPUs boast features like a next-gen Transformer Engine and support for lower-precision calculations, enabling them to handle complex AI workloads significantly faster and more efficiently than before. While aimed at data centers, the innovations within Blackwell are expected to influence consumer graphics cards as well

Latest Articles about Blackwell

---

GPU: GPU stands for Graphics Processing Unit and is a specialized type of processor designed to handle graphics-intensive tasks. It is used in the computer industry to render images, videos, and 3D graphics. GPUs are used in gaming consoles, PCs, and mobile devices to provide a smooth and immersive gaming experience. They are also used in the medical field to create 3D models of organs and tissues, and in the automotive industry to create virtual prototypes of cars. GPUs are also used in the field of artificial intelligence to process large amounts of data and create complex models. GPUs are becoming increasingly important in the computer industry as they are able to process large amounts of data quickly and efficiently.

Latest Articles about GPU

---

HPC: HPC, or High Performance Computing, is a type of technology that allows computers to perform complex calculations and process large amounts of data at incredibly high speeds. This is achieved through the use of specialized hardware and software, such as supercomputers and parallel processing techniques. In the computer industry, HPC has a wide range of applications, from weather forecasting and scientific research to financial modeling and artificial intelligence. It enables researchers and businesses to tackle complex problems and analyze vast amounts of data in a fraction of the time it would take with traditional computing methods. HPC has revolutionized the way we approach data analysis and has opened up new possibilities for innovation and discovery in various fields.

Latest Articles about HPC

---

LLM: A Large Language Model (LLM) is a highly advanced artificial intelligence system, often based on complex architectures like GPT-3.5, designed to comprehend and produce human-like text on a massive scale. LLMs possess exceptional capabilities in various natural language understanding and generation tasks, including answering questions, generating creative content, and delivering context-aware responses to textual inputs. These models undergo extensive training on vast datasets to grasp the nuances of language, making them invaluable tools for applications like chatbots, content generation, and language translation.

Latest Articles about LLM

---

Stable Diffusion: Stable Diffusion is a technology that is used to improve the performance of computer systems. It is a process of spreading out the load of a system across multiple processors or cores. This helps to reduce the amount of time it takes for a system to complete a task, as well as reduce the amount of energy used. Stable Diffusion is used in many areas of the computer industry, such as in cloud computing, distributed computing, and high-performance computing. It is also used in gaming, where it can help to reduce the amount of time it takes for a game to load. Stable Diffusion is also used in artificial intelligence, where it can help to improve the accuracy of machine learning algorithms.

Latest Articles about Stable Diffusion