Accelerating Vision-Language Models with LFM2.5-VL-DSpark

Accelerating Vision-Language Models with LFM2.5-VL-DSpark

Liquid AI has released LFM2.5-VL-DSpark, an experimental draft model for the LFM2.5-VL-3B vision-language model (VLM). It adds a speculative decoding path that increases memory usage slightly while improving inference speed without changing output quality.

Key results include:

• Faster inference: Decode speedups of up to 3.13x on-device and 2.66x on an H100, with end-to-end improvements of up to 2.62x and 2.27x, respectively.
• Limited memory overhead: The drafter adds 280M parameters, or 8.9% of the 3B target model.
• Day-one framework support: Integrations are available for llama.cpp, MLX-VLM, and SGLang.

Speculative decoding for vision-language models

The vision drafter uses the same architecture as the text-based LFM2.5-DSpark drafters. It captures hidden states from a fixed set of tapped layers and uses them to draft a block of k candidate tokens.

Before those layers, image patches and text tokens are projected into a shared representation. As a result, the drafter receives hidden-state vectors with the same dimensionality regardless of whether the input comes from an image or text. The inference algorithm remains unchanged from the text models.

Training and architecture

The model follows the DSpark training recipe, using a mixture of vision-language supervised fine-tuning data weighted toward the workloads it is expected to serve.

Ablation experiments across three, four, and five layers led to a simplified, attention-only drafter with four layers and a block size of nine. The model was trained for 10 epochs on the final data mixture, with acceptance measured after each epoch. Acceptance improved as more training tokens were added before reaching diminishing returns. At inference time, a block size of eight or nine is recommended, depending on the hardware.

The resulting drafter has approximately 280M parameters and increases the deployed model's parameter count by 8.9%.

Inference speed on CPU and GPU

LFM2.5-VL-3B-DSpark includes support for llama.cpp, MLX-VLM, and SGLang.

Measurements cover on-device and GPU inference. Both configurations use a DSpark block size of eight and are evaluated on six vision tasks based on the MMSpec benchmark: general visual question answering, text VQA, image captioning, chart VQA, complex reasoning, and multi-turn conversation.

On-device inference

With MLX on an M5 Max, decoding is 2.30x to 3.13x faster depending on the task, while end-to-end latency improves by 1.56x to 2.62x.

With llama.cpp on an M3 Ultra, decoding improves by 1.57x to 2.14x, and end-to-end performance improves by 1.30x to 1.77x.

GPU inference

On an H100, the drafter delivers decoding improvements ranging from 2.04x to 2.66x, with end-to-end improvements of 1.64x to 2.27x.

Limitations for vision workloads

In large language models, prefill is primarily compute-bound, and its cost grows subquadratically with prompt length. Vision-language models add the cost of processing the image through a vision encoder before the language backbone handles the resulting visual tokens alongside the text prompt.

Edge devices have substantially less compute than datacenter GPUs, so prefill can represent a larger share of end-to-end latency. Measurements of time to first token and decoding on Apple silicon and an H100 illustrate this difference. The per-core GPU neural accelerators in the M5 narrow the gap.

Speculative decoding accelerates only the decoding stage. It does not speed up vision encoding or prefill. When those stages account for much of the total runtime, even a substantial decoding improvement produces a more limited end-to-end gain. This follows Amdahl's law: overall acceleration is constrained by the portion of the workload that is not optimized.

Using LFM2.5-VL-DSpark

SGLang

SGLang requires a build with DSpark support for LFM2 targets, provided in PR #40651. Launch the target model with the draft model attached:

python -m sglang.launch_server \
  --model-path LiquidAI/LFM2.5-VL-3B \
  --speculative-algorithm DSPARK \
  --speculative-draft-model-path LiquidAI/LFM2.5-VL-3B-DSpark \
  --speculative-draft-attention-backend flashinfer \
  --speculative-dspark-block-size 9 \
  --disable-radix-cache

The server exposes an OpenAI-compatible endpoint at http://localhost:30000/v1. The block size is read from the draft model's config.json. A baseline run uses the same command without the three --speculative-* flags.

llama.cpp

llama.cpp requires the build provided in PR #29339:

llama-server -m models/LFM2.5-VL-3B-F16.gguf \
  --mmproj models/mmproj-LFM2.5-VL-3B-F16.gguf \
  -md LFM2.5-2.6B-DSpark-F16.gguf \
  --spec-type draft-dspark --spec-draft-n-max 8 --spec-draft-n-min 0 \
  -fa on -ngl 99 -c 8192

MLX-VLM

MLX-VLM requires the build provided in PR #2280:

mlx_vlm.server --model LiquidAI/LFM2.5-VL-3B --draft-model LiquidAI/LFM2.5-VL-3B-DSpark

For MLX-VLM, the block size is read from sidecar metadata, and n-max is limited to that value.

Speculative decoding is exact. The target model verifies every proposed token, so greedy output matches the output produced by the target model alone. Per-response timings fields report draft_n and draft_n_accepted.

Model availability

The vision DSpark draft model is available in Safetensors format and GGUF format.