Alibaba has released the open weights for Qwen3.8-Flash-Next, a 125B-parameter multimodal Mixture-of-Experts (MoE) model that acts as an early architectural preview for the upcoming Qwen4 family. Operating with only 6B active parameters per token alongside a 51B N-gram embedding layer, the model targets cost efficiency across long-context reasoning, agentic coding, and multimodal workloads.
Weights are publicly available on Hugging Face and ModelScope, while a managed production endpoint named Qwen3.8-Flash is slated for QwenCloud at $0.16 per million input tokens and $0.47 per million output tokens.
Architectural Redesign: GDN, QSA, and Gated Residuals
Qwen3.8-Flash-Next introduces four structural changes over prior Qwen generations:
- Hybrid Attention with QSA: The model alternates three Gated DeltaNet (GDN) linear attention layers with one global Qwen Sparse Attention (QSA) layer. Rather than computing token-level sparse indices, QSA aggregates sequences into micro-blocks to select relevant context regions independently per layer. Alibaba reports a 7.6x prefill speedup and 4.9x decode speedup at 1M tokens, yielding 8.6x prefill throughput when running at a 90% prefix-cache hit rate.
- Gated Residual Streams: The standard single residual channel is split into four parallel pathways managed by dynamic element-wise gating. This architecture allows long-range feature propagation from early attention layers directly into deep layers while dampening activation outliers and enabling FP8 state storage.
- Host-Offloaded N-gram Embeddings: A dedicated 51B-parameter N-gram lookup table captures local phrase patterns near the input layer. Because lookups depend only on local token sequences, the embedding table resides in host CPU memory and is prefetched asynchronously during compute, keeping GPU memory footprints low.
- Muon Optimizer Integration: Core 2D linear weight matrices across attention and MoE experts are trained using the Muon optimizer, with AdamW retained for routers and embeddings. Alibaba noted that eliminating traditional batch size warmup reduced required optimization steps by 18.8%.
Benchmark Results and Training Efficiency
Alibaba claims Qwen3.8-Flash-Next required approximately one-ninth of the training compute used for Qwen3.7-Plus while outperforming larger dense and sparse baselines across coding and agent evaluations:
- Software Engineering: Scored 62.5 on SWE-bench Pro (compared to 53.4 for Claude Opus 4.6 Max and 56.0 for DeepSeek-V4-Flash-0731) and 81.0 on SWE-bench Multilingual.
- Autonomous Agents: Reached 73.9 on CoWorkBench and 55.7 on JobBench, compared to 68.2 and 36.6 respectively for Claude Opus 4.6 Max.
- Reasoning and Coding: Scored 91.9 on LiveCodeBench v6 and 91.7 on GPQA Diamond.
- Multimodal Intelligence: Achieved 84.5 on AndroidWorld and 19.4 binary / 52.3 partial success on OSWorld 2.0.
The model natively processes context windows up to 262,144 tokens, with extension up to 1,000,000 tokens supported via YaRN.
