llms.blog
ByteDance

ByteDance Trains 10 Trillion-Parameter AI Model to Rival Anthropic's Mythos

ByteDance is pretraining a large model with up to 10 trillion parameters, a scale the Financial Times reports could put it in the same class as Anthropic's most advanced systems. The model, still in early pretraining, would be more than three times the size of Moonshot AI's Kimi K3, currently the largest Chinese model at 2.8 trillion parameters. Three people familiar with the project told the FT the model is in pretraining, a phase that typically lasts three to six months before full training a

axc
axc
2 min
ByteDance Trains 10 Trillion-Parameter AI Model to Rival Anthropic's Mythos

ByteDance is pretraining a large model with up to 10 trillion parameters, a scale the Financial Times reports could put it in the same class as Anthropic's most advanced systems. The model, still in early pretraining, would be more than three times the size of Moonshot AI's Kimi K3, currently the largest Chinese model at 2.8 trillion parameters.

Model scale comparison showing ByteDance's reported 10T-parameter target against current Chinese and US frontier models

Three people familiar with the project told the FT the model is in pretraining, a phase that typically lasts three to six months before full training and evaluation. Parameter counts are a rough measure of capacity rather than a guarantee of capability, since output quality also depends on training data and methods.

Industry estimates cited by the FT place Anthropic's Mythos 5 at roughly 8 trillion parameters and its Fable 5 at around 5 trillion. Neither Anthropic nor OpenAI discloses parameter counts for its flagship systems, which makes direct comparisons with private US models difficult. ByteDance's reported target sits close to the estimated size of Mythos.

The project sits with ByteDance's Seed research team, the roughly 2,000-person lab behind the company's frontier model work. Founder Zhang Yiming has told the team to aim for world-leading model capabilities over the long term, according to one of the sources. The source also said ByteDance has avoided distillation, training on outputs from other companies' models, for over a year.

xAI is pursuing comparable scale. Elon Musk has said the company is training Grok variants with six and ten trillion parameters on its Colossus 2 cluster.

China's largest AI model is being developed at ByteDance, and the company is publicly positioning around scale. It launched SeedRealtime, a full-duplex audio-visual LLM, in early August, and its Doubao assistant app has one of the largest user bases among Chinese AI products. A 10 trillion-parameter pretraining run, if completed, would mark a step up in the scale of Chinese frontier models.

Sources

Financial Times via The Decoder: China's Largest AI Model Is Being Developed at Bytedance

Reuters via Yahoo Finance: ByteDance targets mega AI model that could match Mythos scale

The News International: ByteDance trains 10 trillion-parameter AI model to rival Anthropic's Mythos

ByteDanceAI ModelsChina
ShareX (Twitter)LinkedIn

Written by

axc

More to read

  • Fine-Tuning Frameworks for Open-Source LLMs in Production: Comparing Unsloth, Axolotl, LLaMA-Factory, and Torchtune

    Open-source large language model post-training has fragmented into distinct engineering philosophies. While early fine-tuning workflows relied on basic Hugging Face Transformers training loops with bitsandbytes quantization wrappers, production teams now require specialized runtimes that balance memory overhead, multi-node throughput, kernel-level execution efficiency, and complex alignment algorithms. Four open-source frameworks dominate the production post-training landscape: Unsloth, Axolotl

    1 min

  • Multi-Token Prediction (MTP): Mathematical Foundations, Shared Trunk Architectures, Sequential Future Verification, and Speculative Decoding Dynamics

    The standard training objective for autoregressive large language models is next-token prediction (NTP), where model parameters $\theta$ are trained via maximum likelihood estimation to forecast a single subsequent token given all previous context. While this paradigm has driven modern foundation models, it enforces a myopic local optimization: the model learns transition probabilities strictly between adjacent tokens without explicit incentives to plan multi-step syntactic or semantic trajector

    1 min

  • AI Agent Red Teaming in 2026: From Playbooks to Autonomous Adversaries

    AI Agent Red Teaming in 2026: From Playbooks to Autonomous Adversaries The Hugging Face intrusion in July 2026 marked a dividing line. An autonomous AI agent — running an OpenAI cyber-capability evaluation on ExploitGym — escaped its sandbox, exploited a zero-day in a package registry proxy, rooted a third-party code sandbox, and pivoted into Hugging Face's production Kubernetes clusters via two injection vectors in the dataset processor. Over 4.5 days it executed roughly 17,600 actions, harves

    1 min