LFM2.5-DSpark Delivers Up to 3.2x Faster Inference

Liquid AI has released DSpark draft model checkpoints for three models in the LFM2.5 family: LFM2.5-1.2B-Instruct, LFM2.5-2.6B, and LFM2.5-8B-A1B. The checkpoints add speculative decoding, which slightly increases memory use while improving decoding speed without changing output quality.

Reported results include:

• Up to 3.18x higher throughput on a GPU and up to 2.87x on-device.
• An average 57% reduction in function-calling latency for LFM2.5-2.6B.
• Support for llama.cpp and SGLang, with LFM-compatible DSpark integrations available upstream.

How DSpark works

LLM decoding is traditionally memory-bound. Much of its latency comes from streaming model weights from DRAM into SRAM rather than from intensive computation.

Speculative decoding uses a lightweight draft model to propose candidate tokens. The target model then verifies those candidates in a single forward pass, allowing the cost of loading its weights to be shared across multiple tokens.

Several speculative decoding approaches have been proposed, including EAGLE-3, DFlash, and DSpark. DSpark combines three components:

• A DFlash-style parallel backbone, conditioned on context features from the target model and producing hidden states for all draft tokens in one forward pass.
• A lightweight sequential head, modeled as a Markov chain between neighboring tokens. It adds inter-token dependency and improves acceptance rates at later positions.
• A confidence-scheduled verifier, which estimates each token's probability of surviving verification and removes low-confidence suffixes when checking them would cost more than it saves.

Training and architecture

The DSpark training recipe uses a larger and more diverse mixture of supervised fine-tuning, chat, code, and function-calling data. The initial draft models are simplified, attention-only models with five layers and a block size of nine.

Each draft model was trained for 15 epochs over the full dataset. The selected checkpoint was the one with the highest acceptance rate, rather than the lowest loss.

The resulting draft models contain approximately 300 million parameters each:

Quality parity

With greedy decoding, a draft token is accepted only when it matches the target model's distribution. If the token is rejected, the target model's own token replaces it.

As a result, the emitted sequence is identical to baseline greedy decoding by construction. Benchmark results based on pass@1 or exact match therefore remain unchanged.

Inference performance on CPU and GPU

The LFM2.5 DSpark draft models support llama.cpp and SGLang from release. The llama.cpp implementation builds on the official codebase and uses experimental Metal kernels. The SGLang implementation builds on the official SGLang DSpark implementation.

On-device throughput was measured with llama.cpp and Metal on an M4 Max MacBook Pro, using FP16 GGUF weights and up to 256 output tokens. GPU throughput was measured with SGLang on a single H100 80 GB using BF16. Both configurations used a DSpark block size of nine, a batch size of one, and a temperature of zero. Five benchmark datasets were evaluated.

LFM2.5-2.6B

Across multi-tool scenarios, DSpark reduced latency by an average of 57% for LFM2.5-2.6B.

LFM2.5-1.2B-Instruct

Acceptance rates varied more across datasets for LFM2.5-1.2B-Instruct, causing speedups to differ by as much as 52% depending on the underlying text distribution.

LFM2.5-8B-A1B

LFM2.5-8B-A1B achieved higher acceptance rates than the two dense models. However, its average on-device improvement was 18%.

The difference is attributed to the current MoE implementation in llama.cpp's Metal backend. Verifying multiple tokens activates more experts, which creates more weight traffic than a single decode step.

Using LFM2.5-DSpark

SGLang

Running DSpark with SGLang requires an SGLang build with DSpark support for LFM2 targets, available through PR #31041. Launch the target model with the draft model attached:

python -m sglang.launch_server \
  --model-path LiquidAI/LFM2.5-2.6B \
  --speculative-algorithm DSPARK \
  --speculative-draft-model-path LiquidAI/LFM2.5-2.6B-DSpark \
  --speculative-draft-attention-backend flashinfer \
  --disable-radix-cache --mem-fraction-static 0.75 --port 30000

The OpenAI-compatible endpoint is available at http://localhost:30000/v1. The block size is read from the draft model's config.json. A baseline run uses the same command without the three --speculative-* flags.

llama.cpp

Running the models with llama.cpp requires the corresponding build, available through PR #27383:

llama-server -m LFM2.5-2.6B-F16.gguf \
  -md LFM2.5-2.6B-DSpark-F16.gguf \
  --spec-type draft-dspark --spec-draft-n-max 10 --spec-draft-n-min 0 \
  -fa on -ngl 99

The block size is read from sidecar metadata, and n-max is clamped to that value. Speculative decoding is exact: the target model verifies every proposed token, so greedy output matches the target model running alone. Per-response timings fields report draft_n and draft_n_accepted.

Model checkpoints

DSpark checkpoints are available in Safetensors and GGUF formats:

• Safetensors: LFM2.5-2.6B-DSpark, LFM2.5-1.2B-Instruct-DSpark, and LFM2.5-8B-A1B-DSpark
• GGUF: LFM2.5-2.6B-DSpark-GGUF, LFM2.5-1.2B-Instruct-DSpark-GGUF, and LFM2.5-8B-A1B-DSpark-GGUF