llms.blog
Nvidia

Nvidia's Nemotron 4 aims for a trillion parameters, but China is already there

Nvidia is building Nemotron 4, a new family of open-weight AI models meant to challenge the strongest freely available models, according to reporting by The Information cited by The Decoder. The largest model in the family is planned to reach at least one trillion parameters, twice the size of Nvidia's current Nemotron 3 Ultra. To train it, Nvidia has tripled its cloud spending on in-house model development to 28 billion dollars through 2031. The earliest the models could ship is this fall. C

axc
axc
1 min
Nvidia's Nemotron 4 aims for a trillion parameters, but China is already there

Nvidia is building Nemotron 4, a new family of open-weight AI models meant to challenge the strongest freely available models, according to reporting by The Information cited by The Decoder. The largest model in the family is planned to reach at least one trillion parameters, twice the size of Nvidia's current Nemotron 3 Ultra.

To train it, Nvidia has tripled its cloud spending on in-house model development to 28 billion dollars through 2031. The earliest the models could ship is this fall.

China is already there

Illustration of the AI model scale gap between US and Chinese labs

A trillion parameters would be a milestone for a US open model, but not for the field. China's labs already operate at that scale and beyond. Moonshot AI's Kimi K3 uses 2.8 trillion parameters, and DeepSeek V4 Pro uses 1.6 trillion. On the Artificial Analysis Intelligence Index, Nvidia's Nemotron 3 Ultra scores 38 points and trails Moonshot's Kimi K3 at roughly 60.

A tension with its own customers

The move also puts Nvidia in tension with its own customers. The chipmaker recently announced a 10 gigawatt infrastructure partnership with OpenAI, yet Nemotron 4 would compete directly with OpenAI's own models in the open-weight arena. Nvidia is also among the signatories of a petition opposing new regulation of open models, a position that aligns with selling more GPUs as companies choose to self-host.

Sources

The Decoder: https://the-decoder.com/nvidias-nemotron-4-aims-for-one-trillion-parameters-a-scale-chinese-labs-already-surpassed/

The Information (via The Decoder): https://www.theinformation.com/articles/nvidia-open-source-ai

NvidiaOpen ModelsAI Hardware
ShareX (Twitter)LinkedIn

Written by

axc

More to read

  • Hybrid SSM-Transformer Architectures: How Interleaving Attention and Recurrence Solves the State-Retrieval Trade-Off

    Hybrid SSM-Transformer Architectures: How Interleaving Attention and Recurrence Solves the State-Retrieval Trade-Off Autoregressive language models face a fundamental tension between inference efficiency and long-context retrieval capacity. Pure Transformer architectures scale quadratic computational complexity during sequence prefill and linear key-value (KV) cache memory consumption during autoregressive token generation. Conversely, pure State Space Models (SSMs) and linear recurrent neural

    1 min

  • Study: Why Labor-Saving LLMs Incline Scientists to Do More Work Less Well

    A theoretical study published by researchers from Princeton University, the University of Washington, and collaborating institutions models how large language models alter researchers' time allocation across projects. The authors find that by reducing time friction across different stages of the research lifecycle, AI assistants increase the opportunity cost of researcher time, creating economic incentives to publish a higher volume of less thoroughly refined papers. The paper, titled The unint

    1 min

  • Memory Shortage Drives Nvidia AI Server Prices Up Over 15%

    Nvidia has notified major customers that prices for server systems containing its artificial intelligence accelerators are increasing by more than 15% in many configurations, according to reports from Bloomberg and Fortune. The price adjustments stem from severe supply constraints and rising costs across dynamic random-access memory (DRAM) and high-bandwidth memory (HBM) modules. The price increases will apply to server systems scheduled for delivery starting in early 2027, covering platforms p

    1 min