Granite Switch 4.1 Combines Chat, Decisions, and Speech at One Endpoint

Granite Switch 4.1 Combines Chat, Decisions, and Speech at One Endpoint

Introduced last spring, the Granite Switch project provides a modular approach to model development. Granite Switch 4.1 extends that approach with a Decision adapter and the fastest automatic speech recognition (ASR) model on the OpenASR Leaderboard. The models can handle natural-language responses, general and specialized decisions, and speech-to-text tasks through one hosted endpoint.

What Granite Switch 4.1 Includes

The Granite-Switch-4.1-8B and Granite-Switch-4.1-30B models include three main components:

1. A new Decision aLoRA for general-purpose decision-making, such as v1/systemone.
2. A specialized Guardian aLoRA for safety-related decision tasks. On average, it outperforms Jev 1.13, Microsoft-Decision-1, GPT-Luna-Decisions, and Clef-Flash on those tasks.
3. Granite-Speech-5.0-Turbo-CTC, embedded for transcription and identified as the fastest ASR model on the OpenASR Leaderboard.

These technologies, together with the underlying base model, are composed into one model. Their combined configuration uses approximately the same amount of GPU VRAM as the base model alone, allowing the additional capabilities to run without requiring more GPUs.

Granite Switch adapters use cache-efficient Activated LoRAs. This allows the key-value (KV) cache generated by the base model to be reused for subsequent adapter-based decisions. Granite Switch 4.1 also incorporates contributions to vLLM for faster low-rank adapter inference and includes a co-located ASR encoder. As a result, the models can switch between chat, decision, and speech requests through the same endpoint.

Combining General and Specialized Decision Models

Decision models are also available through earlier Granite Libraries releases. That collection contains specialized decision-model adapters and other task adapters that can be combined into custom Granite Switch models.

General decision models such as Jev are designed for broad use, while specialized Granite Library adapters can outperform general-purpose models on certain classification tasks. Because adapters share the base model's KV cache, adding specialists alongside a general decision adapter has little additional cache cost. This provides broad coverage from the generalist adapter while retaining higher accuracy for selected tasks.

In a comparison involving Granite-Switch-4.1-30B, which used a frozen Granite base and four specialized aLoRAs, and Jev-27B, which used a frozen Qwen base and one generic decision LoRA, Granite-Switch-4.1-30B completed a five-step real-world retrieval-augmented generation (RAG) workflow faster.

Building Custom Granite Switch Models

Granite-Switch-4.1-8B and Granite-Switch-4.1-30B are distributed with predefined adapters. Users can also compose their own Granite Switch model by combining adapters from Granite Libraries.

The project plans to provide an Adapter SDK for training adapters in Mellea, along with tutorials and additional Granite Switch support. Custom adapters could target specialized classification workflows, other language tasks, or entirely different applications.

For example, a custom 3B-parameter Granite Switch model was used to make decisions while navigating the classic game Doom. It operated at 60 to 100 decisions per second, compared with Jev's reported 10 decisions per second. The Granite Switch model could also narrate the gameplay and respond to questions while playing.

Resources

• Granite Switch 4.1 Model Cards
• Speech-to-Decisions Demo
• Granite Switch GitHub Repository
• Original Granite Switch and Granite Libraries Release
• aLoRA versus LoRA Colab notebook

Footnote

1. Typesafe: Introducing System One Models and Jev