Beam: Reflection’s 501B Open Model for Customizable, Large-Scale Deployment

Beam: Reflection’s 501B Open Model for Customizable, Large-Scale Deployment

Published October 16, 2026

Beam, developed by the US-based lab Reflection, is an open mixture-of-experts language model with 501 billion parameters, 23 billion active parameters per token, and a 1 million-token context window. It accepts and generates text and is released under the Apache 2.0 license.

The model targets coding, reasoning, and agentic workloads. It can run on privately managed hardware, internal infrastructure, or cloud systems, and can be adapted to specific use cases. Beam includes a Rust-based inference harness, a native Transformers implementation, and support in vLLM and SGLang. Recipes are available for inference, supervised fine-tuning, and reinforcement learning.

Overview

• 501B-parameter sparse MoE model with 23B active parameters per token
• 52 layers, 256 experts per MoE layer, and 8 routed experts per token
• Text input and text output with a 1M-token context window
• BF16, FP8, and NVFP4 checkpoints
• Reasoning that can persist across turns with preserve_thinking
• Tool calling through a dedicated grammar
• Reasoning levels: low, medium (default), high, xhigh, and max
• Response template included in the tokenizer
• DFlash2 speculative-decoding drafter model
• Day-0 support in Transformers, SGLang, and vLLM
• Full-parameter reinforcement learning support through Miles
• Rust-based custom harness
• Apache 2.0 license

Model Architecture

Beam is a text-only model designed for long-context reasoning and agentic tasks. Its sparse mixture-of-experts design activates only part of the network for each token. This gives it the computational behavior of a smaller model while retaining the parameter capacity of a larger one.

The first three layers use dense SwiGLU multilayer perceptrons. The remaining 49 layers use mixture-of-experts blocks.

Attention

The decoder has 52 layers and a hidden size of 5,120. Most layers use sliding-window attention over the previous 512 tokens. Every sixth layer, along with the final layer, attends to the full context. This produces 43 sliding-window layers and 9 global layers.

Sliding-window layers use rotary positional embeddings, or RoPE, because relative position is most important within a short window. Global layers do not use positional encoding. Instead, each query is multiplied by a slowly increasing scaler:

(1 + (pos + 1) / 2048) ** 0.1

This scalable-softmax query scaling acts like a temperature adjustment, helping softmax retain precision as the context grows rather than becoming too flat.

Attention uses grouped-query attention with 48 query heads and 8 key-value heads. Queries and keys use RMSNorm. A learned sigmoid gate, calculated from the layer input, scales every channel in every head before the output projection.

Mixture of Experts

The 49 MoE layers each contain 256 experts. Every expert is a SwiGLU MLP with an intermediate size of 2,560 and approximately 39 million parameters. Together, the experts contain 98.4% of the model’s parameters.

The router uses a group-limited approach based on DeepSeek-V3. It scores all 256 experts with a sigmoid, divides them into eight groups of 32, retains the four groups with the highest combined top-two scores, and then selects the top eight experts within those groups. Beam uses the same group sizes as DeepSeek-V3 but does not include a shared expert. Expert outputs are combined using normalized router weights and scaled by 2.5.

Only eight experts run for each token, so approximately 23B parameters are active during computation. All 256 experts remain in memory, meaning deployment requires memory for a 501B model while computation operates at roughly the speed of a 23B model.

The BF16 checkpoint occupies approximately 1 TB in Safetensors format. The NVFP4 checkpoint stores most parameters at 4-bit precision for NVIDIA Blackwell GPUs, while retaining some parameters in BF16. FP8 checkpoints are available for Hopper and Ampere GPUs.

Residual Stream

Beam uses depth-dependent residual weights rather than adding every sublayer output with a fixed weight of one.

The input embedding is treated as step 1. At step k, the model adds the sublayer output with weight 1/sqrt(k) and retains the existing stream with weight sqrt((k-1)/k). The squares of these two weights sum to one, keeping the stream at a consistent scale through the network. After all 104 additions, each sublayer output and the input embedding have the same weight in the final hidden state.

Every sublayer is normalized before and after processing. Final logits use a tanh softcap at ±30, calculated in FP32. The implementation also uses FP32 for the router, the SwiGLU product, normalization, RoPE, query scaling, and the softcap.

Long-Context Operation

Beam’s hybrid attention reduces key-value cache growth. Each layer stores:

2 x 8 KV heads x 128 dimensions = 2,048 values per token

If all 52 layers attended globally, the cache would grow by 208 KiB per token. Instead, only the 9 global layers retain the complete sequence. The 43 sliding-window layers keep fixed 512-token windows, so they stop growing after those windows are filled.

The resulting cache growth is approximately 36 KiB per token, about one-sixth of the naive rate.

These figures are per sequence, in BF16, using the Transformers dynamic cache.

Benchmark Results

The following results were provided by the Reflection team for the BF16, FP8, and NVFP4 versions of Beam.

Hardware Requirements

The NVFP4 checkpoint is the most compact deployment option. It reduces the BF16 model from approximately 1 TB to about 280 GB. Blackwell GPUs support NVFP4 natively, while older GPUs such as the H200 do not. FP8 checkpoints are intended for hardware older than Blackwell.

A B200 provides 180 GB of HBM3e. Two B200 GPUs provide 360 GB, enough for the Beam weights but with limited space remaining for activations and the key-value cache. Four B200 GPUs, providing 720 GB, leave more room for long contexts and concurrent requests.

Beam can also run on:

• An 8xB200 node
• Two B300 GPUs with 288 GB each
• Eight RTX Pro 6000 GPUs

Inference Providers can be used to test Reflection’s model without Blackwell hardware.

Inference Support

Beam has support in Transformers, SGLang, and vLLM.

Transformers

Install a current Transformers release:

pip install -U "transformers>=5.20.1"

The text-generation pipeline provides a simple way to run the model:

from transformers import pipeline

pipe = pipeline(
    "text-generation", model="reflection/Beam-1.0-501B-A23B", device_map="auto"
)

messages = [
    {"role": "user", "content": "Write a limerick about Python exceptions."}
]
output = pipe(messages, max_new_tokens=2000)
output[0]["generated_text"][-1]

The pipeline returns the complete chat. The final message is Beam’s response, already parsed through the response template.

For lower-level access, load the model and tokenizer directly:

from transformers import AutoModelForCausalLM, AutoTokenizer

model_id = "reflection/Beam-1.0-501B-A23B"
tokenizer = AutoTokenizer.from_pretrained(model_id)

model = AutoModelForCausalLM.from_pretrained(
    model_id, dtype="auto", device_map="auto"
)

Chat and Reasoning

messages = [
    {"role": "system", "content": "You should only answer with a number."},
    {"role": "user", "content": "What is 17 * 23?"},
]

inputs = tokenizer.apply_chat_template(
    messages,
    add_generation_prompt=True,
    tokenize=True,
    return_dict=True,
    return_tensors="pt",
    reasoning_effort="medium",
).to(model.device)

input_len = inputs["input_ids"].shape[-1]
outputs = model.generate(**inputs, max_new_tokens=2000)

tokenizer.parse_response(
    outputs[0][input_len:], prefix=inputs["input_ids"][0]
)

Beam’s tokenizer includes a response template. parse_response converts raw output into a message dictionary containing separate fields for reasoning, the final answer, and tool calls. Because the parsing rules are included with the tokenizer, the resulting structure matches the format used during training. The chat template can also pre-write part of the assistant turn, which the parser expects to see.

Tool Calling

def get_weather(city: str, unit: str = "celsius"):
    """
    Get the current weather for a city.

    Args:
        city: The city to look up.
        unit: Temperature unit, "celsius" or "fahrenheit".
    """
    ...

tools = [get_weather]
messages = [
    {"role": "user", "content": "I land in Paris tonight. Do I need a coat?"}
]

inputs = tokenizer.apply_chat_template(
    messages,
    tools=tools,
    add_generation_prompt=True,
    tokenize=True,
    return_dict=True,
    return_tensors="pt",
).to(model.device)

input_len = inputs["input_ids"].shape[-1]
outputs = model.generate(**inputs, max_new_tokens=1000)

message = tokenizer.parse_response(
    outputs[0][input_len:], prefix=inputs["input_ids"][0], tools=tools
)
message.get("tool_calls")

The tool definitions must be passed to parse_response as well. Beam writes tool-call arguments inline using its XML-style grammar. Tool schemas allow the parser to distinguish values such as numeric and string arguments and cast them to their declared types.

Multi-GPU and Multi-Node Deployment

Beam’s configuration defines an expert-parallel plan. Expert parallelism places each expert on one device, while attention, dense layers, embeddings, and the language-model head are replicated across devices. These replicated components account for approximately 8B parameters, or 16 GB in BF16.

import os

from transformers import AutoModelForCausalLM
from transformers.distributed import DistributedConfig

model = AutoModelForCausalLM.from_pretrained(
    "reflection/Beam-1.0-501B-A23B",
    distributed_config=DistributedConfig(
        tp_size=int(os.environ["WORLD_SIZE"]),
        ep_size=int(os.environ["WORLD_SIZE"]),
        tp_plan={},
    ),
)

The model can be launched with torchrun across 16 H200 GPUs. Each GPU then stores approximately 78 GB of weights. With eight GPUs, each device would store approximately 139 GB, leaving little space on a 141 GB H200. The number of devices must divide evenly into the 256 experts.

vLLM

The following command serves the BF16 checkpoint with tensor parallelism across 16 GPUs, such as two 8x H200 nodes, and exposes an OpenAI-compatible API:

vllm serve reflection/Beam-1.0-501B-A23B \
  --tensor-parallel-size 16 \
  --served-model-name beam

DFlash2 speculative decoding can improve generation speed. A smaller drafter proposes up to 15 tokens, which Beam checks in one forward pass:

vllm serve reflection/Beam-1.0-501B-A23B \
  --tensor-parallel-size 16 \
  --speculative-config '{ "method": "dflash", "model": "reflection/dflash2-drafter-preview", "num_speculative_tokens": 15 }'

Both commands provide an OpenAI-compatible endpoint at http://localhost:8000/v1.

SGLang

On Blackwell hardware, the NVFP4 checkpoint can run on four B300 GPUs with DFlash2 speculative decoding:

python -m sglang.launch_server \
  --model-path reflection/Beam-1.0-501B-A23B-NVFP4 \
  --served-model-name beam \
  --tp 4 \
  --attention-backend trtllm_mha \
  --page-size 64 \
  --context-length 1048576 \
  --mem-fraction-static 0.85 \
  --enable-unified-memory \
  --speculative-algorithm DFLASH \
  --speculative-draft-model-path reflection/Beam-1.0-501B-A23B-DFlash2 \
  --speculative-dflash-block-size 8 \
  --tool-call-parser beam --reasoning-parser beam \
  --host 0.0.0.0 --port 30000

On Hopper GPUs, the FP8 checkpoint can run on eight H200 GPUs. The tensor-parallel size changes to 8, and the attention backend changes to fa3:

python -m sglang.launch_server \
  --model-path reflection/Beam-1.0-501B-A23B-FP8 \
  --served-model-name beam \
  --tp 8 \
  --attention-backend fa3 \
  --page-size 64 \
  --context-length 1048576 \
  --mem-fraction-static 0.85 \
  --enable-unified-memory \
  --speculative-algorithm DFLASH \
  --speculative-draft-model-path reflection/Beam-1.0-501B-A23B-DFlash2 \
  --speculative-dflash-block-size 8 \
  --tool-call-parser beam --reasoning-parser beam \
  --host 0.0.0.0 --port 30000

Inference Providers

Beam is available through Inference Providers powered by Baseten, Fireworks, and Together.

import os
from openai import OpenAI

client = OpenAI(
    base_url="https://router.huggingface.co/v1",
    api_key=os.environ["HF_TOKEN"],
)

completion = client.chat.completions.create(
    model="reflection/Beam-1.0-501B-A23B:auto",
    messages=[{"role": "user", "content": "What is the capital of France?"}],
)

print(completion.choices[0].message)

Inference Endpoints

Beam recipes are available for deployment through Inference Endpoints using the FP8 or NVFP4 checkpoints. It can also be deployed with the hf CLI:

hf endpoints catalog deploy \
  --repo reflection/Beam-1.0-501B-A23B \
  --name beam

Evaluation Tasks

Five informal evaluations tested Reflection Beam-501B-A23B at low and high reasoning effort through the Reflection platform. These were task demonstrations rather than formal benchmarks.

The low-reasoning long-context response correctly captured the code behavior and all six citations but did not match the parser’s expected format. The low-reasoning math response returned the correct answer but omitted the requested justification. The high-reasoning response supplied a correct roots-of-unity proof.

Post-Training Beam

Beam can be adapted for specific use cases through supervised fine-tuning and reinforcement learning.

Supervised Fine-Tuning with TRL

The TRL SFTTrainer can be used with LoRA adapters, allowing the base model’s 501B parameters to remain frozen. In one example, Beam was adapted to reason in the language requested by the user using the HuggingFaceH4/Multilingual-Thinking dataset.

The setup combined model sharding at load time, expert parallelism, FlashAttention 4, and sequence packing. LoRA fine-tuning fit on one node with eight B300 GPUs and reached approximately 19,000 tokens per second. The equivalent H100 configuration used four nodes.

After one epoch, Beam reasoned in the requested language on 16 of 16 held-out prompts, compared with 3 of 16 for the base model. This evaluation measured language adherence rather than reasoning correctness.

GRPO with Miles

Reflection and RadixArk created a BF16 GRPO recipe in Miles. It supports either full-parameter updates or LoRA adapters. Megatron handles training, SGLang generates rollouts, and Miles publishes updated weights to the rollout engines after each update.

The recipe runs GRPO on DAPO-Math-17K and evaluates on AIME25. A rollout proceeds as follows:

1. Sample: 32 prompts are passed to the model with 8 responses each. Two SGLang engines serve rollouts on 16 GPUs each.
2. Score: The boxed answer is checked against the label, with a soft penalty for outputs near the length limit.
3. Update: One optimizer step is performed on all 256 samples using group-relative advantages with truncated importance sampling and no KL or entropy penalty.
4. Train in parallel: Tensor parallelism is 8, expert parallelism is 8, and data parallelism is 8, with sequence parallelism and full recomputation.
5. Optimize: Adam uses a constant learning rate of 1e-6. Gradients are in BF16, master weights are in FP32, and optimizer moments are in BF16. Optimizer state streams through node-local NVMe, requiring 2 TB of scratch space per trainer node.
6. Evaluate: AIME25 is evaluated every five rollouts with eight samples per problem. A checkpoint is saved every 20 rollouts.

The next rollout is generated while the current rollout trains, leaving responses one update behind. The --resume option continues from the last checkpoint. execute-colocate runs the same recipe on eight nodes, with each GPU alternating between training and serving.

LoRA Training

GRPO can also run with LoRA on four 8x H200 nodes, using 16 GPUs for training and 16 for rollouts. The recipe trains rank-16 adapters on Beam’s attention, dense MLP, and expert layers while leaving the base weights frozen. Only adapter weights are synchronized with SGLang, and optimizer state remains in GPU memory rather than being offloaded to NVMe.

python scripts/run_beam_lora.py execute \
  --model-dir /root/models/Beam \
  --data-dir /data/datasets \
  --output-dir /root/shared_data/beam-run \
  --num-rollout 100

Demonstrations

Native macOS Application

One demonstration used Beam with the open-source Pi coding harness to create BeamSpeak, a native macOS menu-bar dictation application. The application uses Parakeet with MLX for local speech recognition.

The requested application included:

• A compact recording indicator without a main window or account
• ⌘⇧Space to start and stop recording
• Elapsed time, microphone level, and transcription status
• Local speech recognition with a Hugging Face model, preferably Parakeet v3 through MLX
• Model persistence between recordings
• Pasting into the focused application when Accessibility permission is available
• Clipboard fallback that preserves newer clipboard changes and never presses Enter
• Permission handling, cancellation, silence detection, errors, a two-minute recording limit, and temporary audio cleanup

The saved run included follow-up prompts, and the final source included later Codex repairs.

Multiple-Agent Programming

Another experiment paired Beam with the open-source Pi coding agent and compared three configurations:

1. One agent working alone
2. Five persistent peers collaborating through a shared Git repository
3. A lead agent delegating work to asynchronous subagents

The harness ran on Hugging Face jobs, with inference through Reflection’s platform API. The tasks came from ProgramBench, where agents recreate programs from compiled executables and documentation without access to source code or evaluator tests.

Each configuration used the same 8M API-token ceiling and a two-hour solver limit per task. Implementation snapshots were evaluated against fixed hidden-test suites from the ProgramBench-Tests dataset.

The initial experiment used five tasks. The published figure shows a post-hoc subset consisting of jq, xsv, FIGlet, and hexyl. ripgrep was excluded from all three configurations because grading its saved artifacts took too long. The published results retain all 15 original episodes.

Incident-Triage Decision Model

A further experiment examined whether a coding agent could turn an incident-triage policy into a small decision model. The task was to read a developer incident and select the responsible team, severity, and whether someone should be contacted immediately.

Using Unsloth, the workflow fine-tuned Qwen3.5-0.8B with a LoRA adapter and decision head. It first created a synthetic dataset of 2.3k incidents divided by incident cluster into training, calibration, and testing groups. Training ran for three epochs with a 4-bit backbone on one H100 80GB GPU in the Science Slurm cluster.

On 320 synthetic test tickets, the fine-tuned model got all three decisions correct on 76.25% of examples, compared with 4.69% for prompted zero-shot inference. Mean ticket latency decreased from 1,218.06 ms to 102.58 ms, an 11.9x reduction.

Summary

Beam is a 501B open mixture-of-experts model with 23B active parameters per token, a 1M-token context window, and Apache 2.0 licensing. It is available in BF16, FP8, and NVFP4 formats, with support in Transformers, vLLM, SGLang, and Miles. Its architecture combines hybrid attention, grouped-query attention, depth-dependent residual connections, and group-limited expert routing for long-context reasoning and agentic workloads.