Technology
NVIDIA Blackwell GPUs: what changes for training
NVIDIA Blackwell GPUs change large-model training mainly at rack scale. The GB200 NVL72 packs 72 Blackwell GPUs and 36 Grace CPUs into around 120 kW and presents itself to software as one machine holding 13.4 TB of HBM3e. Official MLPerf Training v5.0 numbers show 2.2x on Llama 3.1 405B pretraining against H100.