Scaling Agentic Coding Workflows with Meta’s Muse Spark Open on NVIDIA GB300 NVL72

Meta has introduced Muse Spark Open, an 816B-parameter mixture-of-experts (MoE) model designed for coding. The model activates 42B parameters per token and supports a 1M-token context window. Trained specifically on NVIDIA GB300 NVL72 Blackwell Ultra systems for agentic coding workflows, it is designed to improve first-attempt accuracy, tool calling, and codebase understanding across repositories.

The 1M-token context window can contain an entire project, including complex dependencies, legacy code, and thousands of files, reducing the need to divide a task into smaller pieces.

As a reasoning model, Muse Spark Open also supports audio and video perception in addition to image and document understanding. It processes what it sees, hears, and reads as part of a unified workflow. Meta-generated NVFP4 checkpoints, optimized for NVIDIA Blackwell Ultra, reduce the memory footprint of the 816B parameters and use native FP4 Tensor Core acceleration. Moving fewer bytes per token can increase tokens per second per user, while the released memory can support a larger KV cache for concurrent long-context sessions. These optimizations are intended to increase throughput and reduce cost per token.

Multimodal understanding for agentic coding

General coding models are often optimized for single-shot tasks, such as generating a function, writing a unit test, or suggesting an inline fix. These tasks typically have a defined scope, limited context, and an immediate answer.

Agentic coding involves longer, multistep workflows. Planning a multifile refactor, navigating thousands of files, recovering from failing tests, and coordinating subagents requires sustained reasoning over time.

Muse Spark Open is designed for these workloads. It can plan, act, observe, and self-correct while using tools, evaluating outputs, and refining its work. Goal conditioning helps maintain direction, while context compaction retains information needed during long-running sessions. Its 1M-token context window can hold large projects, including dependencies, legacy code, and thousands of files.

The model can also search the web, including through multimodal browsing, and create and coordinate subagents for decomposable tasks. Its computer-use capabilities operate across desktop, mobile, and browser environments. Before each action, it generates separate commentary describing what it is doing and why.

Muse Spark Open uses a 1.8B-parameter perception encoder to process interleaved text, images, video frames, and audio without separate pipelines for each modality. This allows it to reason over screenshots, diagrams, error logs, and video walkthroughs within the same agent session.

Its sparse MoE architecture activates only 42B of the 816B total parameters for each token. The model uses top-eight routing across 256 experts, reducing serving requirements compared with a dense model of equivalent overall scale.

Performance on NVIDIA GB300 NVL72

The NVIDIA GB300 NVL72 uses a rack-scale architecture that integrates 72 NVIDIA Blackwell Ultra GPUs into one platform. Its 72-GPU NVIDIA NVLink domain provides communication bandwidth of 130 TB/s. This is intended to reduce bottlenecks that can occur when expert traffic crosses conventional networks.

Muse Spark Open routes each token to eight of 256 experts, making communication bandwidth an important factor in the throughput achieved by a deployment.

Agentic workloads transform single-turn interactions into multistep processes that reason, invoke tools, coordinate subagents, and carry growing context from one turn to the next. On NVIDIA GB300 NVL72, Muse Spark Open delivers up to 50K tokens per second per GPU and up to 600 tokens per second per user for agentic workloads. These figures support production deployments of large-parameter models with high throughput and low latency.

In testing on four NVIDIA Blackwell GPUs with speculative decoding, Muse Spark Open achieved more than 50K tokens per second per GPU and more than 600 tokens per second per user, with an acceptance length of 6.

Deployment options

NVIDIA provides several inference stacks for Muse Spark Open:

• SGLang provides a serving option with control over deployment configuration.
• vLLM supports serving and configuration on NVIDIA accelerated computing platforms.
• TensorRT LLM provides compiled engines tuned for NVIDIA GPUs.
• NVIDIA Dynamo provides deployment recipes for all three stacks.

NVIDIA Dynamo can separate prefill and decode across dedicated GPU pools, preventing compute-intensive prompt processing from slowing token generation. Its KV cache-aware routing directs requests to workers that already contain the relevant prefix, avoiding repeated precomputation as context grows during long agentic sessions.

Post-training with NeMo AutoModel and Megatron Bridge

Developers can post-train Muse Spark Open on domain-specific datasets using NVIDIA NeMo AutoModel. The PyTorch-native training library supports Hugging Face checkpoints without requiring model conversion. It includes support for full supervised fine-tuning and LoRA fine-tuning, enabling experimentation across industry-specific applications.

For large-scale production training, NVIDIA Megatron Bridge can convert Hugging Face checkpoints to Megatron format, allowing developers to use the associated training optimizations.

Getting started

Muse Spark Open NVFP4 weights are available from Hugging Face. NVIDIA also provides a downloadable NVIDIA NIM for production deployments. The NIM can be launched with a single command and automatically selects a serving backend and precision based on the detected hardware.