NVIDIA Nemotron 3.5 Super VL Supports Specialized Multimodal Agents

Always-on agents are increasingly used for long, complex enterprise workflows. A single request may require reading a document, checking multiple sources, applying company policies, and calling several tools.

NVIDIA Nemotron 3.5 Super VL is an open, customizable multimodal model designed for these workflows. Its 120-billion-parameter mixture-of-experts (MoE) architecture activates 12 billion parameters per token, combining model capacity with inference efficiency. On a single GPU, NVIDIA reports up to five times higher throughput for agentic tasks than other open models in the same size category.

The model combines reasoning, instruction following, and tool use with text, image, document, and video understanding. This allows teams to handle text and visual tasks with one model instead of deploying separate text and vision-language models or transferring context between them.

Developers can post-train Nemotron 3.5 Super VL with NVIDIA NeMo using their own data, tools, policies, and workflows. The model can then serve as a foundation for specialized enterprise agents.

Why use Nemotron 3.5 Super VL for specialized agents?

Agents must understand both their assigned tasks and the environments in which they operate. A customer-service agent needs to follow return policies, research agents must connect evidence across sources, and document agents need to interpret tables, charts, and page layouts.

Nemotron 3.5 Super VL combines reasoning and visual understanding for workflows such as:

• Coding: Test and validate code, analyze errors, and recommend fixes.
• Web research: Search websites, connect information, and verify answers.
• Document intelligence: Extract information from tables, charts, and page layouts to create summaries and reports.
• Cybersecurity: Identify vulnerabilities and triage threats for remediation.
• Video understanding: Extract information for search, content review, and incident analysis.

Within a system of models, Nemotron 3.5 Super VL handles specialized, multistep tasks requiring reasoning, tool use, and text and visual understanding together. Smaller models such as NVIDIA Nemotron 3.5 Lightning can handle high-volume text tasks, while NVIDIA Nemotron 3 Ultra is intended for complex planning and orchestration.

Customizing the model for domain data, tools, and policies

Specialized agents must learn an organization’s terminology, tool interfaces, operating procedures, and policy requirements. Organizations in cybersecurity, finance, healthcare, and other industries are post-training Nemotron 3.5 Super VL for production workflows.

Examples include:

• Scale AI post-trained the model for automated vendor credit processing at Skydance Corporation. The resulting workload is expected to reduce inference cost by 40x and latency by 5x.
• Ramp is using the model to convert receipt images into structured JSON containing merchant information, dates, and payment data. Ramp expects up to 50% higher throughput and 10x lower latency.
• AI21 post-trained the model for Alice’s threat intelligence and vulnerability analysis, improving accuracy by 40%.

Teams can use NVIDIA NeMo open libraries for supervised fine-tuning (SFT), low-rank adaptation (LoRA), and reinforcement learning (RL). NVIDIA NeMo AutoModel supports SFT and LoRA for domain-specific behavior, while RL can improve performance in agentic environments that represent organizational workflows.

NVIDIA is also releasing open datasets on Hugging Face. They include agentic tool-use and conversational data, finance reasoning and tool-calling datasets, instruction-following and chat data, long-context transduction and translation data, fairness data, and 45 vision-language datasets. These resources can supplement proprietary data for SFT and RL.

For multimodal training, NeMo Data Designer includes resources for chart question answering and long-document understanding. NVIDIA NeMo RL and NVIDIA NeMo Gym support reinforcement learning and environment-based evaluation.

Organizations can adapt and deploy the model’s open weights in supported local, data center, or cloud environments.

Combining text and visual understanding

Nemotron 3.5 Super VL handles text, images, documents, and video while also providing reasoning and instruction following. Its visual training covers charts, tables, optical character recognition (OCR), multipage documents, and video understanding.

A two-stage SFT pipeline trains the language and vision components using visual evidence, reasoning, chat, safety, long-context data, and agent trajectories. For document intelligence, the model can interpret a chart, connect it with a table elsewhere in the same document, and summarize both.

Training data generated with NeMo Data Designer helped improve performance on complex charts. On CharXiv, a chart-reasoning benchmark, Nemotron 3.5 Super VL scored 80.8%, compared with 63% for Nemotron 3 Nano Omni. With Python code interpretation, it reached 84%.

The model was developed with support from Nemotron Coalition organizations. Datasets from Perplexity contributed to mid-training, while environments from Prime Intellect supported the development of teacher models. Nemotron 3.5 Super VL is also among the leading open models on the OCRBench v2 leaderboard.

Accuracy and efficiency for agent tasks

Evaluations of Nemotron 3.5 Super VL include Hermes, LangChain, OpenClaw, OpenCode, Pi, Codex, and Claude. On Hermes Bench and LangChain behavioral evaluations, it scored higher than Nemotron 3 Ultra:

For always-on agents, throughput, token usage, tool calls, and the number of steps needed to reach a correct result all affect task completion time. NVIDIA reports that Nemotron 3.5 Super VL uses throughput and token efficiency to complete coding tasks up to four times faster than other open models in its class in an evaluation of 10,000 tasks on NVIDIA B200 GPUs.

The model was also optimized for local operation on NVIDIA DGX Spark. An independent evaluation by EXO Labs placed Nemotron 3.5 Super VL on the Pareto frontier for accuracy and task completion time among models that fit on a single DGX Spark with 128GB of memory.

Model architecture and inference techniques

Nemotron 3.5 Super VL combines a strong base model with a hybrid architecture, speculative decoding, context reuse, and mixed precision. These techniques reduce inference work across repeated agent calls.

Building the base model

Nemotron 3.5 Super VL was distilled from Nemotron 3 Ultra. Ultra’s coding capabilities and the SkewReverse loss used during distillation helped improve coding accuracy.

During post-training, multi-teacher on-policy distillation (MOPD) combines guidance from specialized teachers across text and vision. The model generates responses, and the teachers score them for video reasoning and tracking, vision tool use, and alignment.

Additional training on agent actions, code repositories, and software engineering tasks improved agent performance. Nemotron 3.5 Super VL outperforms Nemotron 3 Super on SWE-bench and Terminal-Bench across multiple evaluation setups.

Continued pretraining with natural and synthetic data also improved Indic-language capabilities. This stage added 10 billion Indic tokens from a 12-language subset of Nemotron-pretraining-Indic-v1 and natural Indic text. The complete dataset contains about 2.8 trillion tokens across 22 Indian languages and was developed with IIT Madras and its AI4Bharat lab.

LiLiCorr speculative decoding

LiLiCorr improves parallel speculative decoding by evaluating token candidates jointly across an entire draft. A lightweight network computes compatibility scores in one pass, producing more coherent drafts and leaving only a low-cost sequential selection step.

On DGX Spark, LiLiCorr is the recommended drafter for low-latency Nemotron 3.5 Super VL inference. It delivers up to 2.3 times faster token generation than decoding without speculation.

NVFP4 and mixed precision

NVIDIA Model Optimizer supports post-training quantization and quantization-aware distillation (QAD). The NVFP4 recipe uses 4-bit weights and activations on NVIDIA Blackwell architecture. The same checkpoint can run on non-Blackwell platforms with 4-bit weights and 16-bit activations, with precision selected separately for sensitive components.

NVFP4 reduces model size, while QAD helps preserve accuracy at lower precision. During QAD, the quantized model learns to match the outputs of the original BF16 model. QAD also supports storing the key-value (KV) cache in NVFP4 instead of FP8 while meeting accuracy targets. Because QAD occurs during training, it adds no inference overhead.

Multi-token prediction

Multi-token prediction (MTP) gives the model an internal drafter that proposes several tokens for the main model to verify in one forward pass. Accepting multiple tokens at once reduces the number of compute-intensive verification steps. A dedicated MTP training phase further aligned the drafter with the final model.

Reusing context across agent turns

Agents repeatedly send system instructions, tool definitions, and conversation history to the model. Prefix caching reuses computation from these shared inputs.

The serving optimizations preserve both attention caches and Mamba recurrent states. Additional Mamba checkpoints reduce the replay needed to recover inputs for speculative decoding, while internal checkpointing avoids extra model passes during prompt processing.

Production deployment with NVIDIA NIM and NVIDIA Dynamo

NVIDIA NIM packages models with optimized inference engines, validated configurations, and standard APIs for cloud and data center deployment. Enterprise support is available through NVIDIA AI Enterprise.

NVIDIA Dynamo provides distributed inference optimizations, including KV-cache-aware routing, KV cache management, and disaggregated serving. On 16 NVIDIA B200 GPUs at 50 tokens per second per user, NVIDIA reports that KV-aware routing with session affinity and KV cache offload can serve up to eight times as many users while reducing first-token latency by 85%.

Getting started

The release includes base, post-trained, and NVFP4-quantized checkpoints, along with training recipes, data, and reinforcement-learning environments. Developers can use the technical report, Hugging Face, or ModelScope for model information and deployment instructions. Hosted access is also available through OpenRouter.

Partners support post-training, inference software, agent frameworks, cloud platforms, systems integration, hardware, infrastructure, and hosted inference services. Listed partners include Scale AI, Ramp, AI21, Baseten, Databricks, Prime Intellect, LangChain, OpenHands, Amazon SageMaker JumpStart, Google Cloud Gemini Enterprise Agent Platform, MSFT Foundry, OCI Enterprise AI, Dell, HPE AI Services, Red Hat, CoreWeave, Fireworks AI, and Together AI.