Nvidia Blackwell vs Hopper GPUs: Performance Comparison

NVIDIA’s Blackwell GPU architecture is designed to be a significant leap over the previous Hopper generation, particularly for AI workloads like large language models (LLMs) and generative AI. The performance gains are substantial and multi-faceted, stemming from fundamental architectural changes rather than just incremental improvements.

Performance Comparison (AI Workloads)

NVIDIA has demonstrated significant performance improvements with Blackwell, especially in real-world AI benchmarks for LLMs.

AI Compute Performance (FLOPS):

Blackwell (B200):

1. 20 PFLOPS (FP4) / 9 PFLOPS (FP8) / 4.5 PFLOPS (FP16/BF16) per GPU.

Hopper (H100):

1. 4 PFLOPS (FP8) / 2 PFLOPS (FP16/BF16) per GPU.

Advantage: Blackwell offers a 4.5x increase in FP8 performance (and introduces FP4 for even higher numbers) per GPU compared to H100, and a 2.25x increase at FP16/BF16.

LLM Training Performance (MLPerf Benchmarks):

1. Llama 3.1 405B Pre-training: Blackwell delivered2.2x greater performance compared to Hopper at the same scale (e.g., 512 GPUs). A 2,496 Blackwell chip cluster completed this in 27 minutes.
2. Llama 2 70B LoRA Fine-tuning: An 8-GPU DGX GB200 NVL72 system delivered2.5x more performance than a DGX H100 system.
3. GPT-3 175B Training: Up to2.0x performance improvement per GPU with Blackwell compared to Hopper.
4. Overall: NVIDIA consistently reports Blackwell providing2x to 2.5x performance gains per GPU for core LLM training tasks compared to Hopper in MLPerf benchmarks.

LLM Inference Performance:

For extremely large models (e.g., 1.8 trillion-parameter GPT-MoE), a single Blackwell GPU can achieve up to15x higher inference throughput than an H100 GPU in real-time generation of output tokens.

This is heavily aided by the new FP4 precision and the enhanced Transformer Engine.

System-Level Performance (GB200 NVL72 vs. DGX H100):

A full GB200 NVL72 rack (combining 36 Grace CPUs and 72 Blackwell GPUs) delivers 1.4 ExaFLOPS of AI performance (FP8) and 30 TB of fast memory.

This translates to up to a30x increase in throughput for large LLMs like Llama 3.1 405B compared to an equivalent Hopper system.

Power Efficiency:

While individual Blackwell GPUs (e.g., B200) have higher power consumption (up to 1000W vs. 700W for H100), the performance per watt for large-scale AI is dramatically superior.

NVIDIA claims up to a25x reduction in energy consumption for LLM inference with Blackwell systems compared to Hopper at equivalent performance levels. This means significant savings in operational costs and a reduced carbon footprint for AI data centers.

Conclusion:

NVIDIA Blackwell represents a strategic evolution designed specifically for the rapidly expanding scale and complexity of AI. While Hopper was revolutionary, Blackwell builds on that foundation with a multi-die design, significantly larger transistor counts, vastly increased memory bandwidth, specialized hardware for ultra-low precision AI (FP4), and a more robust interconnect fabric. These advancements result in multi-fold performance increases (ranging from 2x to 30x depending on the specific AI workload and system configuration) compared to Hopper, particularly for the most demanding LLM training and inference tasks, all while dramatically improving power efficiency at scale.