FuriosaAI Introduces Energy-Efficient RNGD AI Processor at Hot Chips 2024

At Hot Chips 2024, FuriosaAI is unveiling RNGD—pronounced “Renegade”—a new AI accelerator that promises to shake up the world of large language models (LLMs) and multimodal model inference. If you’re wondering what that means, think of it as a supercharged engine designed to power the latest advancements in artificial intelligence, particularly in data centers where efficiency and performance are paramount.

During a presentation that felt more like a tech pep rally than a dry corporate update, Furiosa co-founder and CEO June Paik shared the nitty-gritty details of RNGD, including a first look at the fully functioning RNGD card. With a thermal design power (TDP) of just 150 watts, this chip isn’t just about raw power; it’s about doing more with less. It leverages memory technology like HBM3, which stands for High Bandwidth Memory 3, a type of memory that allows for faster data transfer. This means RNGD is built to handle the heavy lifting of demanding LLMs and multimodal models without breaking a sweat.

But let’s take a moment to appreciate the feat that is RNGD’s development timeline. In an industry where bringing a chip from concept to reality can feel like waiting for a new season of your favorite show, Furiosa managed to get the first RNGD silicon samples from TSMC in May, boot the hardware within a week, and start running industry-standard Llama 3.1 models by June. That’s a pace that would make even seasoned engineers raise their eyebrows.

So, what’s next for RNGD? According to Paik, the focus is now on refining the software stack as they ramp up production. This is crucial because, as anyone who’s ever tried to tune a new gadget knows, the hardware is only as good as the software that runs it. Furiosa’s previous chip, launched in 2021, set the stage with impressive performance metrics, and they’re aiming for similar success with RNGD.

Here’s where it gets interesting: RNGD is currently generating about 12 queries per second while running the GPT-J 6B model. While that sounds solid, it’s just the beginning. As they tweak and enhance their software, that performance metric is likely to rise. They’ve been intentionally low-key about their targets, opting not to add to the noise of bold claims that often litter the tech landscape. After all, with a team that’s 95% engineers, the focus has been on building rather than marketing.

If you’re at Hot Chips this week, swing by the Furiosa booth. The team is eager to engage with the AI community, answer questions, and gather feedback. It’s a refreshing approach in a field that often feels dominated by hype. They’re also showcasing the first live demo of RNGD, which is sure to be a highlight for anyone interested in the future of AI hardware.

As we look ahead, we can expect more updates on RNGD’s performance benchmarks, availability, and other developments in the coming weeks. In a world where tech announcements can sometimes feel like empty promises, it’s refreshing to see a company like FuriosaAI take a measured approach, focused on delivering tangible results. So, stay tuned—this is just the beginning for RNGD, and it’s shaping up to be an exciting ride.

Background Information

---

About TSMC:

TSMC, or Taiwan Semiconductor Manufacturing Company, is a semiconductor foundry based in Taiwan. Established in 1987, TSMC is a important player in the global semiconductor industry, specializing in the manufacturing of semiconductor wafers for a wide range of clients, including technology companies and chip designers. The company is known for its semiconductor fabrication processes and plays a critical role in advancing semiconductor technology worldwide.

  

Latest Articles about TSMC