From NVIDIA: 2024-10-09 11:00:32

Large language models powered by NVIDIA enable organizations to derive deeper insights and create new applications. NVIDIA regularly optimizes cutting-edge community models like Meta’s Llama and Google’s Gemma to improve performance by 3.5x in less than a year and deliver 4x more performance in the MLPerf Inference benchmark compared to previous generations.

The latest improvements in NVIDIA’s Blackwell platform have increased performance by 3.4x on the H100 model in MLPerf over the last year. This marks a significant advancement, with current peak performance on Blackwell being 10x faster than just one year ago on Hopper, showcasing tremendous progress in a short period.

NVIDIA’s ongoing work includes developing TensorRT-LLM, a dedicated library for accelerating large language models efficiently on NVIDIA GPUs. By optimizing variants of Meta’s Llama models and leveraging advanced parallelization techniques with NVLink and NVSwitch, NVIDIA ensures optimal performance for demanding models like Llama 3.1 405B that require multiple GPUs for fast responses.

Different parallelism techniques like tensor and pipeline allow LLMs to balance low latency and high throughput based on application requirements. Tensor parallelism can deliver over 5x more throughput in minimum latency scenarios, while pipeline parallelism boosts performance by 50% for maximum throughput use cases, showing the importance of selecting the right technique for each scenario.

NVIDIA’s constant software tuning and optimization lead to significant performance gains across its architectures, enabling customers to create more capable models and applications with less infrastructure. As new LLMs and generative AI models emerge, NVIDIA continues to optimize them on its platforms to enhance ROI and facilitate easier deployment with technologies like NIM microservices and NIM Agent Blueprints.



Read more at NVIDIA: What’s the ROI? Getting the Most Out of LLM Inference