Mistral Large 4: Architecture, Benchmarks, and Deployment Options

Mistral Large 4: Architecture, Benchmarks, and Deployment Options

Mistral Large 4, also known as Le Chonk, is a multimodal and multilingual Mixture-of-Experts (MoE) model from Mistral. It accepts text and images as input and generates text. The model has approximately 1T total parameters, 49B active parameters, and a context window of 1M tokens.

Mistral provides two official variants:

1. FP8
2. NVFP4

GGUF versions are also available from Unsloth and the llama.cpp team. The model is distributed under the Mistral Ecosystem Open-Weight License, an MIT derivative that places limitations on model-as-a-service offerings.

Mistral Large 4 is designed for applications including cybersecurity, coding agents, and multimodal understanding. This article covers its capabilities, architecture, benchmark results, inference options, and fine-tuning support.

What is new in Mistral Large 4?

Mistral Large 4 combines a large pool of experts, sparse attention, and a Pixtral vision encoder.

For each token, the language model selects a small group of experts. Separately, its attention indexer selects the earlier token positions that should receive attention. These mechanisms provide two forms of sparsity: one selects feed-forward networks, while the other selects context positions.

All available checkpoints are listed in the Mistral Large 4 collection.

Cybersecurity capabilities

Mistral Large 4 scores 82% on the Artificial Analysis Cyber Index test. The evaluation asks a model to reproduce a real vulnerability in open-source software and then patch it. Mistral reports that several closed frontier models score near zero on the same test because they refuse to attempt the task.

Reproducing a vulnerability is often part of fixing it, making this capability relevant to defensive security work. Mistral's internal testing also found the model useful for malware analysis, vulnerability prioritization, and writing detection rules, although it was not explicitly trained for those tasks.

The model is primarily positioned for defensive applications. Vulnerability research and incident response can resemble attack activity in model prompts, which may cause closed-source filters to block legitimate work. API-dependent teams may also lose access during an incident. With an open-weight model, users control both the deployment and the applicable usage policy.

Mistral Large 4 still declines harmful requests. On cyber prompts from JailbreakBench, StrongREJECT, and AgentHarm, its average refusal rate was higher than that of any other open-weight model.

Open weights also support downstream applications built around a user's codebase, crash logs, and malware samples. For example, TRL can be used to fine-tune smaller models from traces generated while Mistral Large 4 patches code. Reproduce-then-patch tasks have verifiable rewards, making them suitable for reinforcement-learning environments. Open weights also allow researchers to measure refusal behavior directly instead of relying solely on provider descriptions.

Architecture

Mistral Large 4's text decoder has 72 layers, a hidden dimension of 6,144, and 64 attention heads. Its attention blocks combine Multi-head Latent Attention (MLA) with a DeepSeek Sparse Attention (DSA) indexer.

MLA maintains a compressed key-value representation, while DSA selects up to 2,048 key positions for each query.

The decoder follows a 3:1 RoPE-to-NoPE pattern. Three decoder layers use RoPE, and every fourth layer uses NoPE. The model also shares key-value state across layers.

The first decoder layer uses a dense feed-forward network. The remaining 71 layers use MoE blocks. A softmax router selects four of 128 experts for each token, while one shared expert runs alongside them. This produces five active experts per token.

For rotary positional embeddings, Mistral uses LeRope, a YaRN-scaled RoPE implementation with one learned multiplier per frequency.

Vision encoder

Mistral Large 4 uses a 1.6B-parameter Pixtral encoder. Pixtral has 30 layers, a hidden dimension of 2,048, and 16 attention heads. It uses 14 × 14 patches and a patch-merge projector with a spatial merge size of 2.

Benchmark results

As of October 29, 2026, Mistral Large 4 was the fifth-best open-weights model on the Artificial Analysis Intelligence index. Its reported results cover coding, cybersecurity, multimodal understanding, and agentic workflows.

For coding, the model achieves:

• 61.7% on DeepSWE v1.1
• 59.4% on SWE-Atlas-QnA
• 28.3% on Terminal-Bench 4
• 49.8% on the Coding Agent Index

The Coding Agent Index result places it ahead of DeepSeek V4 Pro 0813 and Qwen3.8 Max.

For knowledge work, Mistral Large 4 scores 59.9% on AutomationBench and 1,393 Elo on AA-Briefcase. These benchmarks evaluate business workflows and long-horizon professional tasks.

Cybersecurity results include:

• Artificial Analysis Cyber Index: The model ranks among the top five models globally on an independent evaluation of vulnerability discovery and remediation.
• Vulnerability reproduction and patching: The model achieves 82%, the highest reported score on this test within the index.
• Cybench: The model solves 93% of 40 challenges drawn from security competitions.

Mistral Large 4 also records 93.3% attack resistance on Lakera's B3 benchmark, which evaluates resistance to prompt-injection attacks. Internal testing reported useful performance in malware analysis, vulnerability prioritization, and detection-rule generation.

Inference support

Mistral Large 4 has day-one support for Transformers, vLLM, and llama.cpp. GGUF quantizations are available from the llama.cpp team. The model can also be deployed remotely with Inference Endpoints or used through serverless inference providers.

Transformers

The Transformers implementation provides AutoModelForMultimodalLM, while AutoProcessor prepares conversations and image inputs. Mistral Large 4 supports low, medium, and high reasoning effort. The default is high. Reasoning cannot be disabled, so setting reasoning_effort to none still uses high.

Upgrade Transformers before running the model:

pip install --upgrade transformers

A distributed inference example is shown below:

import os
from transformers import AutoProcessor, AutoModelForMultimodalLM, DistributedConfig

distributed_config = DistributedConfig(
    tp_size=int(os.environ["WORLD_SIZE"]),
    ep_size=int(os.environ["WORLD_SIZE"]),
)

model_id = "mistralai/Mistral-Large-4-1T-A52B"
processor = AutoProcessor.from_pretrained(model_id)
model = AutoModelForMultimodalLM.from_pretrained(
    model_id,
    distributed_config=distributed_config,
)

text_messages = [
    {
        "role": "user",
        "content": "Who are you?"
    }
]

multimodal_messages = [
    {
        "role": "user",
        "content": [
            {
                "type": "image",
                "image": "https://huggingface.co/datasets/huggingface/cats-image/resolve/main/cats_image.jpeg",
            },
            {
                "type": "text",
                "text": "How many cats are in the image?",
            },
        ],
    },
]

for messages in [text_messages, multimodal_messages]:
    inputs = processor.apply_chat_template(
        messages,
        add_generation_prompt=True,
        tokenize=True,
        return_dict=True,
        return_tensors="pt",
        reasoning_effort="medium",
    ).to(model.device).to(model.dtype)
    input_len = inputs["input_ids"].shape[-1]

    outputs = model.generate(**inputs, max_new_tokens=100)
    if int(os.environ["RANK"]) == 0:
        print(processor.decode(outputs[0][input_len:], skip_special_tokens=False))

Save the script as script.py and launch it with:

torchrun --nproc-per-node 8 script.py

vLLM

vLLM supports the NVFP4 Mistral Large 4 checkpoint. This version has a smaller memory footprint and requires NVIDIA Blackwell GPUs. The model also includes a DSpark drafter for additional speed improvements with vLLM.

vllm serve mistralai/Mistral-Large-4-1T-A52B-NVFP4 \
  --config-format mistral \
  --load-format mistral \
  --tensor-parallel-size 8 \
  --decode-context-parallel-size 8 \
  --enable-expert-parallel \
  --dtype bfloat16 \
  --max-model-len 131072 \
  --max-num-seqs 16 \
  --max-num-batched-tokens 2048 \
  --gpu-memory-utilization 0.85 \
  --enable-prefix-caching \
  --enforce-eager \
  --mm_processor_cache_gb 0.0

After the server starts, a request can be sent through the OpenAI-compatible API:

curl -s http://127.0.0.1:8000/v1/chat/completions \
  -H 'Content-Type: application/json' \
  -d '{"messages":[{"role":"user","content":"What is 17*23?"}],"max_tokens":512}'

llama.cpp

llama.cpp and Llama App support Mistral Large 4. Llama App can be installed with:

curl -LsSf https://llama.app/install.sh | sh

Specify the quantization suffix after the model ID when starting the server:

llama serve -hf ggml-org/Mistral-Large-4-1T-A52B-GGUF:Q2_K

GGUF quantizations derived from the FP8 checkpoint are available in the GGUF repository. A separate repository contains the version converted from the official NVFP4 model: Mistral-Large-4-1T-A52B-NVFP4-GGUF.

When the server starts, the built-in WebUI is available at localhost:9931. The interface supports image inputs and adjustment of the reasoning level. The server can also be queried through its API:

curl http://localhost:8080/v1/chat/completions \
    -H "Content-Type: application/json" \
    -d '{
        "messages": [
            {"role": "system", "content": "You are a helpful assistant."},
            {"role": "user", "content": "Write a limerick about python exceptions"}
        ]
    }'

An image-text request can use a base64-encoded image:

curl -s http://127.0.0.1:8080/v1/chat/completions \
    -H "Content-Type: application/json" \
    -H "Authorization: Bearer <your-key>" \
    -d '{
        "model": "muse-spark-open",
        "messages": [
            {
                "role": "user",
                "content": [
                    {
                        "type": "text",
                        "text": "What is in this image?"
                    },
                    {
                        "type": "image_url",
                        "image_url": {
                            "url": "data:image/png;base64,..."
                        }
                    }
                ]
            }
        ],
        "temperature": 0
    }'

The llama.cpp team provides the following quantizations: IQ2_XXS, Q2_K, Q4_K_M, NVFP4, Q8_0, and BF16 for further conversions. The 2-bit quantizations run on a single Apple Silicon Ultra computer with 512 GB of unified RAM. The other versions require distributed inference. Multiple NVIDIA or AMD cards can also be used.

With Q2_K on a single M3 Ultra, reported prefill throughput was approximately 90 tokens per second, with generation at 15 tok/s. On 4 × RTX6000 Pro, the reported results were:

• NVFP4: 32K context size, prefill at 130-590 t/s, generation at approximately 10 t/s
• IQ2_XXS: 800K context size. At short context lengths, prefill reached 500 tok/s and generation reached 30 tok/s. With long contexts, performance fell to 129 tok/s for prefill and 3 tok/s for generation.

Fine-tuning with TRL

Mistral Large 4 has day-one fine-tuning support in TRL. Upgrade transformers, trl, and peft before starting:

pip install --upgrade transformers trl peft

The following example trains the model for multilingual reasoning. The resulting fine-tuned model can reason in the language specified by the system prompt.

The script runs the FP8 variant on a node with 8 × B300 GPUs. A complete example, including a Slurm launcher, is available in the TRL examples repository.

## torchrun --nproc-per-node 8 sft.py
import os

from datasets import load_dataset
from peft import LoraConfig
from transformers import AutoModelForImageTextToText, DistributedConfig
from trl import SFTConfig, SFTTrainer

world_size = int(os.environ["WORLD_SIZE"])
model_id = "mistralai/Mistral-Large-4-1T-A52B"
model = AutoModelForImageTextToText.from_pretrained(
    model_id,
    distributed_config=DistributedConfig(fsdp_size=world_size, ep_size=world_size),
    dtype="auto",
)

dataset = load_dataset("HuggingFaceH4/Multilingual-Thinking", split="train")
dataset = dataset.map(lambda x: {"chat_template_kwargs": {"reasoning_effort": "high"}})

trainer = SFTTrainer(
    model=model,
    args=SFTConfig(
        packing=True,
        max_length=4096,
        assistant_only_loss=True,
        learning_rate=1e-4,
        lr_scheduler_type="cosine",
        warmup_steps=10,
    ),
    train_dataset=dataset,
    peft_config=LoraConfig(
        r=16,
        lora_alpha=32,
        target_modules=r".*language_model.*\.(q_proj|kv_a_proj_with_mqa|kv_b_proj|o_proj|gate_proj|up_proj|down_proj)",
    ),
)
trainer.train()

Inference Endpoints

Mistral Large 4 is available through Inference Endpoints with a deployment recipe for 8 × NVIDIA H200 GPUs. The recipe runs vLLM with FP8 weights distributed across all eight GPUs using tensor and expert parallelism. It also uses the model's DSpark draft head for faster generation through speculative decoding.

The configuration supports up to 131K tokens of context and up to 16 simultaneous requests. Reported results include:

• Single request: Time to first token of approximately 0.6 seconds for a 1K-token prompt, with generation at approximately 65 tok/s. Chat prompts reached 120-138 tok/s when DSpark drafts were accepted.
• Long prompts: Four 16K-token prompts were prefilled at approximately 4,300 tok/s in total, with a time to first token of 8-9 seconds.
• Under load: Approximately 480 tok/s of total generation across 1,000 ShareGPT conversations.

Deployment can be started with the hf CLI:

hf endpoints catalog deploy --repo mistralai/Mistral-Large-4-1T-A52B --name mistral-large-4

Once the endpoint is running, its OpenAI-compatible API can be queried using the endpoint URL and a Hugging Face token. The reasoning_effort field accepts low, medium, or high, with high as the default.

curl "$(hf endpoints describe mistral-large-4 --json | jq -r .status.url)/v1/chat/completions" \
    -H "Authorization: Bearer $(hf auth token 2>/dev/null)" \
    -H "Content-Type: application/json" \
    -d '{
        "model": "mistralai/Mistral-Large-4-1T-A52B",
        "messages": [
            {"role": "user", "content": "Write a limerick about python exceptions"}
        ],
        "reasoning_effort": "medium"
    }'

The generated answer is returned in choices[0].message.content, while the reasoning is returned in choices[0].message.reasoning. The endpoint also supports tool calls, structured outputs, image inputs, and streaming through standard Chat Completions fields.

Dell Enterprise Hub

Dell AI Servers with NVIDIA H200, B200, GB200, or GB300 GPUs can use the dell-ai CLI. Install it with:

pip install dell-ai && dell-ai login

An optimized Mistral Large 4 recipe can then be deployed on a platform such as an NVIDIA B300 node:

dell-ai models deploy --model-id mistralai/Mistral-Large-4-1T-A52B-NVFP4 \
    --engine docker \
    --platform-id xe9780-nvidia-b300 \
    --gpus 4

Additional release information is available through the Dell Enterprise Hub.

Availability

Mistral Large 4 weights are available in the Mistral Large 4 collection. GGUF versions support local deployment, while Inference Endpoints provide a hosted deployment option.