NVIDIA dominated MLPerf Training v5.1 tests, winning all seven categories for AI training performance, including large language models, image generation, recommender systems, computer vision, and graph neural networks. NVIDIA was the only platform to submit results on every test, showcasing the power and versatility of its CUDA software stack.
The NVIDIA Blackwell Ultra GPU architecture, featured in the GB300 NVL72 rack-scale system, delivered over 4x and nearly 5x performance compared to the previous Hopper architecture in pretraining and fine-tuning tasks. The architectural improvements of Blackwell Ultra, including new Tensor Cores and training methods, contributed to these impressive gains.
NVIDIA introduced NVFP4 precision calculations in MLPerf Training, marking a significant advancement in performance. By adopting FP4 precision for LLM training, NVIDIA achieved faster computations while maintaining accuracy. The platform submitted results using FP4 precision, showcasing its commitment to innovation and performance excellence.
NVIDIA set a new record in Llama 3.1 405B time-to-train, completing the task in just 10 minutes with over 5,000 Blackwell GPUs. The platform scaled efficiently, achieving a 2.7x faster result than the previous round. NVIDIA also demonstrated improved performance per GPU, achieving a 45% faster training time with 2,560 Blackwell GPUs.
In addition to winning all existing benchmarks, NVIDIA set new performance records in the Llama 3.1 8B and FLUX.1 benchmarks. These benchmarks showcased the platform’s capabilities in training compact LLMs and state-of-the-art image generation models. NVIDIA’s performance on these new benchmarks further solidified its leadership in AI training.
NVIDIA’s ecosystem partners, including ASUSTeK, Dell Technologies, and Lenovo, contributed to the MLPerf Training round, highlighting the broad industry support for NVIDIA’s AI technologies. With a focus on innovation and rapid performance increases, NVIDIA is driving advancements in AI training, inference, and intelligence levels.
Read more at NVIDIA: NVIDIA Wins Every MLPerf Training v5.1 Benchmark