TensorRT-LLM Boosts GeForce RTX Performance Fourfold, Unleashing Striking Speed

Generative AI, powered by Geforce RTX and nVidia RTX GPUs, is revolutionizing personal computing. With dedicated AI processors called Tensor Cores, these GPUs are bringing the power of generative AI to over 100 million Windows PCs and workstations. Today, generative AI on PC is getting a significant boost in speed with the introduction of TensorRT-LLM for Windows. This open-source library accelerates inference performance for the latest AI large language models, such as Llama 2 and Code Llama. NVIDIA has also released tools to help developers optimize their LLMs, including scripts, optimized open-source models, and a reference project that showcases the speed and quality of LLM responses.

TensorRT-LLM now offers acceleration for Stable Diffusion in the popular Web UI by Automatic1111 distribution. This results in up to 2x faster performance for the generative AI diffusion model compared to previous implementations. Additionally, RTX Video Super Resolution (VSR) version 1.5 is now available, enhancing the image quality of streamed video content by reducing compression artifacts and sharpening edges and details.

TensorRT-LLM acceleration significantly improves the performance of LLMs, enabling more sophisticated use cases like writing and coding assistants that provide multiple auto-complete results simultaneously. It also benefits integration with other technologies, such as retrieval-augmented generation (RAG), where an LLM is paired with a vector library or database to deliver more targeted answers.

To demonstrate the capabilities of TensorRT-LLM, an example was given using the LLaMa 2 base model. When asked about how NVIDIA ACE generates emotional responses, the base model returned an unhelpful response. However, when combined with RAG and recent GeForce news articles loaded into a vector library, the same LLaMa 2 model provided the correct answer much faster with TensorRT-LLM acceleration. This combination of speed and proficiency offers users smarter solutions.

TensorRT-LLM will soon be available for download from the NVIDIA Developer website, along with TensorRT-optimized open-source models and the RAG demo project. This empowers developers to leverage the benefits of accelerated LLM inference.

Diffusion models, like Stable Diffusion, are widely used for creating artistic works. However, the iterative process of image generation can be time-consuming on underpowered computers. TensorRT addresses this issue by accelerating AI models through various techniques, resulting in faster inference and improved efficiency. With TensorRT acceleration, Stable Diffusion now runs twice as fast, allowing users to iterate more quickly and spend less time waiting for results.

For video streaming enthusiasts, RTX VSR version 1.5 is a game-changer. This AI-powered pixel processing technology enhances the quality of streamed video content by reducing compression artifacts and enhancing details. The latest version introduces improved models that accurately preserve details during the upscaling process, resulting in sharper and crisper images. Additionally, it now de-artifacts video played at the display’s native resolution, further enhancing the visual experience.

RTX VSR 1.5 is available now for all RTX users in the latest Game Ready Driver and will be included in the upcoming NVIDIA Studio Driver. This update is part of NVIDIA’s commitment to advancing AI-enabled applications and games, with over 400 already available to consumers.

As we enter the AI era, RTX continues to supercharge its evolution, bringing unprecedented advancements to gaming, creativity, video, productivity, and development.

Background Information

---

About nVidia: NVIDIA has firmly established itself as a leader in the realm of client computing, continuously pushing the boundaries of innovation in graphics and AI technologies. With a deep commitment to enhancing user experiences, NVIDIA's client computing business focuses on delivering solutions that power everything from gaming and creative workloads to enterprise applications. for its GeForce graphics cards, the company has redefined high-performance gaming, setting industry standards for realistic visuals, fluid frame rates, and immersive experiences. Complementing its gaming expertise, NVIDIA's Quadro and NVIDIA RTX graphics cards cater to professionals in design, content creation, and scientific fields, enabling real-time ray tracing and AI-driven workflows that elevate productivity and creativity to unprecedented heights. By seamlessly integrating graphics, AI, and software, NVIDIA continues to shape the landscape of client computing, fostering innovation and immersive interactions in a rapidly evolving digital world.

  

Technology Explained

---

Geforce: Geforce is a line of graphics processing units (GPUs) developed by Nvidia. It is the most popular GPU used in the computer industry today. Geforce GPUs are used in gaming PCs, workstations, and high-end laptops. They are also used in virtual reality systems, artificial intelligence, and deep learning applications. Geforce GPUs are designed to deliver high performance and power efficiency, making them ideal for gaming and other demanding applications. They are also capable of rendering high-resolution graphics and providing smooth, realistic visuals. Geforce GPUs are used in a variety of applications, from gaming to professional workstations, and are the preferred choice for many computer users.

---

LLM: A Large Language Model (LLM) is a highly advanced artificial intelligence system, often based on complex architectures like GPT-3.5, designed to comprehend and produce human-like text on a massive scale. LLMs possess exceptional capabilities in various natural language understanding and generation tasks, including answering questions, generating creative content, and delivering context-aware responses to textual inputs. These models undergo extensive training on vast datasets to grasp the nuances of language, making them invaluable tools for applications like chatbots, content generation, and language translation.

---

Stable Diffusion: Stable Diffusion is a technology that is used to improve the performance of computer systems. It is a process of spreading out the load of a system across multiple processors or cores. This helps to reduce the amount of time it takes for a system to complete a task, as well as reduce the amount of energy used. Stable Diffusion is used in many areas of the computer industry, such as in cloud computing, distributed computing, and high-performance computing. It is also used in gaming, where it can help to reduce the amount of time it takes for a game to load. Stable Diffusion is also used in artificial intelligence, where it can help to improve the accuracy of machine learning algorithms.

---

Tensor Cores: Tensor Cores are a type of specialized hardware designed to accelerate deep learning and AI applications. They are used in the computer industry to speed up the training of deep learning models and to enable faster inference. Tensor Cores are capable of performing matrix operations at a much faster rate than traditional CPUs, allowing for faster training and inference of deep learning models. This technology is used in a variety of applications, including image recognition, natural language processing, and autonomous driving. Tensor Cores are also used in the gaming industry to improve the performance of games and to enable more realistic graphics.