Meta dismisses rumors of benchmark tampering with Llama 4, igniting tech community debate.

Last week, Meta launched its new Llama 4 models, which come in different categories for specific uses. We have the Llama 4 Scout, Llama 4 Maverick, and Llama 4 Behemoth, all integrated with Meta’s new AI platforms.

Each model has its dedicated features based on its intended use. The Scout model is designed for use with a single nVidia H100 graphics card. It offers a context window for 10 million tokens. The Maverick, on the other hand, is larger than the Scout and aims to match the processing power of GPT-4o and DeepSeek-V3.

Now, let’s talk about the Behemoth, the widest of the three models. It utilizes over 288 billion different parameters and Meta claims it surpasses the capabilities of GPT-4.5 and Claude Sonnet 3.7 in STEM benchmark tests.

However, after the unveiling of Meta’s new LLM models, rumors started swirling that the company may have used dedicated training models to achieve the best possible results in benchmark tests. These accusations originated from sources in China, suggesting that Meta had a dedicated team training the new models to gain an advantage in specific benchmark tests, even though their real-world capabilities might be more limited.

These rumors quickly spread to other social media platforms like X and Reddit, with some tests seemingly supporting the idea. The dissemination of this information prompted Meta to issue a statement, firmly denying the rumors as baseless and unrealistic.

These rumors gained traction when it was discovered that the benchmark models used on the LMArena platform were not the same versions that were publicly available. Meta justifies this by explaining that the version used on the platform was still experimental and it usually takes a few days for updates to roll out across all testing platforms.

In conclusion, while rumors may circulate, it’s important to rely on verified information and statements from the company itself. Meta has categorically denied using any unfair advantages in benchmark tests, and we should take their word for it until proven otherwise.

Background Information

---

About nVidia:

NVIDIA has firmly established itself as a leader in the realm of client computing, continuously pushing the boundaries of innovation in graphics and AI technologies. With a deep commitment to enhancing user experiences, NVIDIA's client computing business focuses on delivering solutions that power everything from gaming and creative workloads to enterprise applications. for its GeForce graphics cards, the company has redefined high-performance gaming, setting industry standards for realistic visuals, fluid frame rates, and immersive experiences. Complementing its gaming expertise, NVIDIA's Quadro and NVIDIA RTX graphics cards cater to professionals in design, content creation, and scientific fields, enabling real-time ray tracing and AI-driven workflows that elevate productivity and creativity to unprecedented heights. By seamlessly integrating graphics, AI, and software, NVIDIA continues to shape the landscape of client computing, fostering innovation and immersive interactions in a rapidly evolving digital world.

  

Latest Articles about nVidia

Technology Explained

---

LLM: A Large Language Model (LLM) is a highly advanced artificial intelligence system, often based on complex architectures like GPT-3.5, designed to comprehend and produce human-like text on a massive scale. LLMs possess exceptional capabilities in various natural language understanding and generation tasks, including answering questions, generating creative content, and delivering context-aware responses to textual inputs. These models undergo extensive training on vast datasets to grasp the nuances of language, making them invaluable tools for applications like chatbots, content generation, and language translation.

Latest Articles about LLM