Experimenting with Qwen3.8-Flash-Next on NVIDIA GB300 NVL72 for Agentic Coding

Alibaba has released the model weights for Qwen3.8-Flash-Next, a preview of the upcoming Qwen4 architecture intended for developer experimentation and evaluation. The multimodal mixture-of-experts (MoE) model contains a 125B-parameter main model and an additional 51B N-gram embedding component, with 6B parameters activated for each token. It provides a native 262,144-token context window, which can be extended to 1M tokens with YaRN.

NVIDIA offers best-effort Day 0 functional support through SGLang, vLLM, and NVIDIA TensorRT LLM. The model has also been validated for inference on NVIDIA GB300 NVL72, with post-training recipes available through NVIDIA NeMo AutoModel and NVIDIA NeMo RL.

Architectural innovations for long-context inference

Qwen3.8-Flash-Next targets high-volume, context-intensive workloads such as agentic coding, document processing, and tool-driven workflows. As context windows expand, attention computation and KV cache memory can become bottlenecks. The model addresses these challenges with a hybrid architecture that combines Gated DeltaNet (GDN) and Qwen Sparse Attention (QSA).

Three out of every four layers use GDN, which continuously compresses historical context into a fixed-size recurrent state. This prevents KV cache growth as sequence length increases. The remaining layer uses QSA to retrieve specific information across the full context.

Earlier sparse-attention approaches often rely on token-level indexers that become more expensive as context length grows. QSA instead divides the sequence into micro-blocks, estimates the importance of each block, and selects only the most relevant regions. This reduces attention, computation, and indexing overhead in each layer, supporting the alternating GDN and QSA design.

Alibaba’s published benchmarks indicate that QSA can improve the efficiency of 1M-token workloads. Compared with full attention, its attention kernel achieved speedups of up to 7.6x during prefill and 4.9x during decoding. In a cache-heavy online-serving test using a 1M-token context length and a 90% prefix-cache hit rate, Qwen3.8-Flash-Next delivered 8.6x the prefill throughput of Qwen3.7-Plus.

Figure 1. Overview of Qwen3.8-Flash-Next, showing three GDN layers and one QSA layer with MoE to reduce memory and computation for large-context inference.

Running Qwen3.8-Flash-Next on NVIDIA GB300 NVL72

The GB300 NVL72 uses a rack-scale architecture that integrates 72 NVIDIA Blackwell Ultra GPUs into one platform. Its 72-GPU NVIDIA NVLink domain supports all-to-all communication at 130 TB/s. This helps avoid bottlenecks that can occur when expert traffic must pass through conventional off-the-shelf networks.

On NVIDIA GB300 NVL72, Qwen3.8-Flash-Next delivers more than 16K tokens per second per GPU and more than 200 tokens per second per user. These rates support experimentation with agentic coding applications at high throughput and low latency.

Figure 2. A Pareto curve showing Qwen3.8-Flash-Next achieving peak throughput above 16K tokens per second per GPU on NVIDIA GB300 NVL72.

The model also runs on local NVIDIA hardware, including NVIDIA DGX Station, NVIDIA DGX Spark clusters, and workstations equipped with four NVIDIA RTX PRO 6000 Blackwell Max-Q Workstation Edition GPUs. Developers can prototype and evaluate agentic coding workflows locally, then scale the same model to GB300 NVL72 for production serving.

Post-training and inference options

Developers can fine-tune Qwen3.8-Flash-Next for domain-specific applications with NVIDIA NeMo AutoModel, a PyTorch-native fine-tuning library that provides Day 0 support for Hugging Face checkpoints. Existing checkpoints can be trained directly without model conversion, using either full supervised fine-tuning (SFT) or memory-efficient LoRA fine-tuning.

Reinforcement learning is supported through NVIDIA NeMo RL recipes.

Several inference stacks are available for different deployment requirements. SGLang, vLLM, and TokenSpeed provide open-source inference recipes for developers who need greater control over performance on NVIDIA-accelerated platforms.

Accessing Qwen3.8-Flash-Next

The model is available through QwenCloud. Its weights can also be downloaded from Hugging Face or ModelScope.