Inference performance is crucial for AI factory economics. Higher throughput means more tokens produced quickly, increasing revenue and lowering costs. The NVIDIA GB300 NVL72 system, powered by Blackwell Ultra architecture, set records on MLPerf Inference v5.1, delivering 45% more DeepSeek-R1 inference throughput than previous systems.

Blackwell Ultra architecture enhances AI compute and memory compared to Blackwell, setting new performance records on all data center benchmarks in MLPerf Inference v5.1. Full-stack co-design, including hardware acceleration for NVFP4 data format, plays a key role in delivering these results. NVIDIA TensorRT Model Optimizer software quantized models for improved accuracy and performance.

Disaggregated serving technique optimized distinct language model inference workloads for higher overall throughput. This technique contributed to record-setting performance on the Llama 3.1 405B Interactive benchmark. NVIDIA also introduced submissions using the NVIDIA Dynamo inference framework, showcasing market-leading performance with lower total cost of ownership.

NVIDIA partners, including major cloud providers and server makers, achieved great results using Blackwell and Hopper platforms. These partners, such as Azure, Dell, and Oracle, contributed to the market-leading inference performance available from the NVIDIA AI platform. This translates to lower TCO and better ROI for organizations deploying advanced AI applications.

Read more at NVIDIA: NVIDIA Blackwell Ultra Sets the Bar in New MLPerf Inference Benchmark