NVIDIA Open-Sources TwoTower Architecture, Achieves 2.42x Text Generation Speedup

According to Beating, NVIDIA has open-sourced Nemotron-Labs-TwoTower, a discrete text diffusion architecture designed to accelerate large language model text generation. The dual-tower design decouples a frozen autoregressive model that preserves reasoning capability from a trainable denoising tower that generates text in parallel. Using only 1/12 of baseline pretraining data on a 30B-parameter Mamba-Transformer mixture-of-experts model, TwoTower retained 98.7% of baseline quality while increasing generation speed by 2.42 times without additional memory overhead.
Disclaimer: The information on this page may come from third-party sources and is for reference only. It does not represent the views or opinions of Gate and does not constitute any financial, investment, or legal advice. Virtual asset trading involves high risk. Please do not rely solely on the information on this page when making decisions. For details, see the Disclaimer.
Comment
0/400
No comments