NVIDIA Introduces a 4-Bit Pretraining Methodology Using NVFP4, Validated on a 12B Hybrid — 2026-05-18

Impact Vector: AI Tools · May 18, 2026 · 3m

## Short Segments Today, NVIDIA unveils a groundbreaking 4-bit pretraining methodology using NVFP4, validated on a 12-billion-parameter hybrid Mamba-Transformer model. This development could redefine efficiency in AI training. Coming up, we'll explore how this innovation could change the landscape of large language model training. ## Feature Story NVIDIA has introduced a new 4-bit pretraining methodology using NVFP4, marking a significant advancement in AI model training. This approach was validated on a 12-billion-parameter hybrid Mamba-Transformer model, trained on an unprecedented 10…