FLUX.3 Image Klein Comes to Diffusers with Text-to-Image, Editing, and Layout Control

FLUX.3 Image Klein Comes to Diffusers with Text-to-Image, Editing, and Layout Control

Published October 15, 2026

Black Forest Labs' FLUX.3 Image Klein is available in Diffusers from launch. The model supports text-to-image generation, multi-reference image-to-image generation, and layout control through bounding boxes included in the prompt.

Two open checkpoints are available in the FLUX.3 Image Klein collection:

The distilled checkpoint generates a 1024 × 1024 image in about 2 seconds on an H100. NVFP4 and FP8 weights can be loaded with from_single_file, and Diffusers includes LoRA training scripts for text-to-image and image editing.

FLUX.3 is distributed as a Modular Diffusers pipeline, allowing developers to inspect, remove, or add pipeline blocks.

Getting started

Install the latest versions of diffusers and transformers. The latter provides the Qwen3-VL text encoder used by the model.

pip install -U diffusers transformers

The distilled checkpoint can then be loaded as follows:

import torch
from diffusers import ModularPipeline

pipe = ModularPipeline.from_pretrained("black-forest-labs/flux-3-image-klein")
pipe.load_components(dtype=torch.bfloat16)
pipe.to("cuda")

pipe(prompt="a red fox", output="images")[0].save("fox.png")

Image generation and editing

Text-to-image

FLUX.3 responds best to prompts that describe the subject, setting, lighting, composition, and visual style. The distilled checkpoint uses 16 inference steps by default and produces a 1024 × 1024 image. The height, width, and num_inference_steps arguments can change these defaults.

prompt = (
    "A wide-angle wildlife photograph of a red fox standing alert in a snow-covered forest clearing at golden hour. "
    "The fox is centered in the frame, its thick orange winter coat contrasting with the fresh white snow, breath "
    "visible in the cold air. Snow-laden pine trees fill the softly blurred background, and low warm sunlight rakes "
    "across the scene from the left, casting long blue shadows."
)

fox = pipe(
    prompt=prompt,
    generator=torch.Generator().manual_seed(0),
    output="images"
)[0]
fox.save("fox.png")

Image-to-image

Reference images are passed through the image argument. The prompt should describe the desired output, including the intended change.

prompt = (
    "The same red fox in the same snow-covered forest clearing, now at night under a full moon: cool blue "
    "moonlight, deep shadows, stars visible through the pine branches, the fox's eyes catching the light."
)

night = pipe(
    prompt=prompt,
    image=fox,
    generator=torch.Generator().manual_seed(0),
    output="images",
)[0]
night.save("night.png")

Multiple references can be passed as a list. They are identified in the prompt as <ref_image_0>, <ref_image_1>, and subsequent identifiers in the order in which they are supplied.

red_fox = pipe(
    prompt="a red fox",
    generator=torch.Generator().manual_seed(0),
    output="images",
)[0]

bench = pipe(
    prompt="a wooden bench",
    generator=torch.Generator().manual_seed(0),
    output="images",
)[0]

combined = pipe(
    prompt="The fox from <ref_image_0> sitting on the wooden bench from <ref_image_1> in a snowy setting.",
    image=[red_fox, bench],
    output="images",
)[0]

combined.save("combined.png")

Layout control

Layout control places generated content in specified regions. A JSON list is appended to the caption. Each region includes an id, a bbox, and a desc, while the caption refers to the region with its identifier.

Bounding boxes use the format [y_min, x_min, y_max, x_max], with the vertical coordinate first. Coordinates are normalized to a 0-1000 scale, so the same box can be used at different image sizes.

layout_prompt = (
    "Minimalist graphic illustration featuring a black silhouette of a person <silhouette_1> centered against a "
    "solid, vibrant chartreuse background <background_1>. The figure is captured in a dynamic, mid-stride running "
    "pose, facing toward the left side of the frame. "
    '[{"id": "background_1", "bbox": [0, 0, 1000, 1000], "desc": "A flat field of neon yellow-green with a subtle '
    'paper texture and fine, organic grain."}, {"id": "silhouette_1", "bbox": [150, 150, 850, 850], "desc": "A '
    "black silhouette of a person in motion, made of dense stippled ink. The leading edges are solid; the trailing "
    'arm and leg dissolve into coarse square pixels and scattered dots."}]'
)

runner = pipe(
    prompt=layout_prompt,
    generator=torch.Generator().manual_seed(0),
    output="images",
)[0]

For reference-based edits, a region can specify both its source location, src_bbox, and destination location, tgt_bbox. Regions with identical source and target boxes remain fixed.

from diffusers.utils import load_image

knight_image = load_image("knight_orig.png")

edit_prompt = (
    "In <ref_image_0>, move the miniature grey amigurumi knight figure <knight_1> upwards and to the left along the "
    "yarn cliff <cliff_1>. Keep the rest of the image completely unchanged, preserving the blurred yarn backdrop "
    "<backdrop_1>, the large knitted dragon <dragon_1>, and the fiery orange thread <fire_1>. "
    '[{"id": "knight_1", "from": "ref_image_0", "src_bbox": [500, 150, 850, 350], "tgt_bbox": [194, 55, 544, 255], '
    '"desc": "A miniature amigurumi knight crocheted from thick grey woolen yarn."}, '
    '{"id": "backdrop_1", "from": "ref_image_0", "src_bbox": [0, 0, 1000, 1000], "tgt_bbox": [0, 0, 1000, 1000], '
    '"desc": "A soft, out-of-focus backdrop of cream and indigo knitted yarn."}, '
    '{"id": "cliff_1", "from": "ref_image_0", "src_bbox": [300, 0, 1000, 450], "tgt_bbox": [300, 0, 1000, 450], '
    '"desc": "A steep cliff of stacked yarn skeins in mustard, terracotta, and indigo."}, '
    '{"id": "needle_1", "from": "ref_image_0", "src_bbox": [420, 250, 550, 450], "tgt_bbox": [420, 250, 550, 450], '
    '"desc": "An oversized polished steel tapestry needle."}, '
    '{"id": "dragon_1", "from": "ref_image_0", "src_bbox": [100, 600, 950, 1000], "tgt_bbox": [100, 600, 950, 1000], '
    '"desc": "A large, fluffy knitted dragon in terracotta and mustard yarn."}, '
    '{"id": "fire_1", "from": "ref_image_0", "src_bbox": [350, 400, 650, 650], "tgt_bbox": [350, 400, 650, 650], '
    '"desc": "Curled puffs of fiery orange angora thread suspended in mid-air."}]'
)

moved = pipe(
    prompt=edit_prompt,
    image=knight_image,
    generator=torch.Generator().manual_seed(0),
    output="images",
)[0]

Base and quantized checkpoints

Base model

The base checkpoint uses 50 steps and classifier-free guidance. It is also the checkpoint intended for fine-tuning.

pipe = ModularPipeline.from_pretrained(
    "black-forest-labs/flux-3-image-klein-base"
)
pipe.load_components(dtype={"vae": torch.float32, "default": torch.bfloat16})
pipe.to("cuda")

pipe.guider = pipe.guider.new(guidance_scale=5.0)  # optional
image = pipe(prompt=prompt, output="images")[0]

Single-file checkpoints

Black Forest Labs provides the transformer as a single file in bf16, fp8r (rowwise FP8), and nvfp4 formats. The transformer can be loaded with from_single_file, while the remaining components are loaded by name.

from diffusers import BFLQuantizationConfig, Flux3Transformer2DModel, ModularPipeline

model_id = "black-forest-labs/flux-3-image-klein-base"
variant = "fp8r"  # or "nvfp4"

transformer = Flux3Transformer2DModel.from_single_file(
    f"https://huggingface.co/{model_id}/blob/main/variants/{variant}/model.safetensors",
    config=model_id,
    subfolder="transformer",
    quantization_config=BFLQuantizationConfig(compute_dtype=torch.bfloat16),
    dtype=torch.bfloat16,
)

pipe = ModularPipeline.from_pretrained(model_id)
pipe.load_components(dtype=torch.bfloat16)
pipe.update_components(transformer=transformer)
pipe.to("cuda")

For the unquantized single-file version, use https://huggingface.co/{model_id}/blob/main/model.safetensors and omit quantization_config.

LoRA fine-tuning

FLUX.3 supports both text-to-image and image-to-image generation, so the base checkpoint can be used to train custom LoRAs. Diffusers includes DreamBooth LoRA scripts for both tasks. Training is also supported through Ostris' AI Toolkit.

The following example trains a tarot-card style with the multimodalart/1920-raider-waite-tarot-public-domain dataset and the trtcrd trigger word:

accelerate launch train_dreambooth_lora_flux3_klein.py \
  --pretrained_model_name_or_path="black-forest-labs/flux-3-image-klein-base" \
  --dataset_name="multimodalart/1920-raider-waite-tarot-public-domain" \
  --caption_column="caption" \
  --instance_prompt="trtcrd" \
  --output_dir="trained-flux3-tarot-lora" \
  --mixed_precision="bf16" \
  --resolution=1024 \
  --use_aspect_ratio_buckets \
  --train_batch_size=1 \
  --rank=16 \
  --lora_alpha=16 \
  --learning_rate=1e-4 \
  --lr_scheduler="constant" \
  --max_train_steps=1500 \
  --validation_prompt="a trtcrd of a cat, tarot style" \
  --validation_epochs=25 \
  --seed=0 \
  --push_to_hub

The following options reduce memory use:

--gradient_checkpointing \
  --cache_latents \
  --use_8bit_adam

--gradient_checkpointing recomputes activations during the backward pass, reducing VRAM use at the cost of some speed. --cache_latents pre-encodes images and releases the VAE. --use_8bit_adam stores optimizer states in 8-bit format.

For instruction-based editing, train_dreambooth_lora_flux3_klein_img2img.py trains on reference image, target image, and instruction triplets. The reference image is identified in the prompt as <ref_image_0>.

Runtime memory and speed

Memory

For systems with less memory, users can load a quantized transformer, run the text encoder separately and release it after encoding, or enable automatic CPU offloading.

from diffusers import ComponentsManager, ModularPipeline

manager = ComponentsManager()
manager.enable_auto_cpu_offload(device="cuda")

pipe = ModularPipeline.from_pretrained(
    "black-forest-labs/flux-3-image-klein",
    components_manager=manager,
)
pipe.load_components(dtype={"vae": torch.float32, "default": torch.bfloat16})
## No pipe.to("cuda"): the manager handles device placement

Speed

The following end-to-end timings use the distilled checkpoint with 16 steps. Results are averages from three runs after a warmup.

Measurements were taken on a single NVIDIA H100 80GB without offloading and with the default attention backend.

A reference image roughly doubles generation time. Reference tokens are included in the same sequence as the text and generated image. A 1024 × 1024 reference adds 4,096 tokens to every transformer call.

Architecture

FLUX.3 is a latent-space flow model. Its denoiser receives latent representations and produces latent outputs, which a VAE then decodes into pixels. The model processes three token types:

1. Text tokens
2. Tokens for the noisy image being generated
3. Optional reference-image tokens

From pixels to tokens

FLUX.3 uses the same VAE architecture as Flux.2. The FLUX.2 VAE compresses an image by 8× in each spatial direction into 32 channels. Every 2 × 2 patch in the latent representation is packed into one 128-channel token.

As a result, one token represents a 16 × 16 pixel patch. A 1024 × 1024 image becomes a 64 × 64 grid containing 4,096 tokens. A reference image passes through the same VAE encoder and packing process while retaining its own dimensions.

From prompts to tokens

The text encoder is Qwen3-VL, a vision-language model. FLUX.3 uses hidden states from eight of its 36 layers, specifically layers 4, 8, through 32, and stacks them. Each text token therefore becomes an 8 × 2560 = 20,480-dimensional vector before a linear layer projects it to the transformer's width of 3072.

This provides features from multiple depths of the language model rather than only its final layer. Prompts are padded to a multiple of 80 tokens, and the transformer also attends to the padding. The pipeline therefore accepts one prompt per call.

Two processing stages

The transformer processes tokens in two stages:

1. Per-mode blocks: Each stream passes through five blocks with separate weights. Text, noisy image, and reference-image streams each have their own blocks.
2. Joint blocks: The streams are concatenated into [text | image | reference] and passed through 28 blocks that share one set of weights.

In the first stage, each stream attends only to itself. In the second stage, all streams attend to one another, allowing image tokens to use text and reference information.

At the end of processing, text and reference tokens are discarded. Only image tokens pass through the output layers, which predict how they should move toward a clean image.

Inside a transformer block

Each block applies attention and the MLP in parallel to the same normalized input. Their sum is added to the residual stream. LayerNorm has no learned scale or shift of its own. Instead, the timestep embedding supplies the scale, shift, and gate through modulation.

Queries and keys are RMS-normalized and receive rotary position embeddings, or RoPE. In a joint block, text, image, and reference tokens share weights but receive separate modulation values.

def block(x, shift, scale, gate):
    h = layer_norm(x)
    h = (1 + scale) * h + shift
    q, k, v = to_q(h), to_k(h), to_v(h)
    q, k = rope(rms_norm(q)), rope(rms_norm(k))
    out = attention(q, k, v) + mlp(h)
    return x + gate * out

Token-specific timesteps

FLUX.1 and FLUX.2 use one timestep embedding for every token. FLUX.3 varies the modulation according to token type.

Noisy image tokens use the current timestep, t. Text and reference tokens always use timestep 0, indicating clean context that should not be denoised. The distilled checkpoint is also given the timestep to which each step will jump. This second timestep embedding is added to every token's modulation, allowing its 16 steps to make larger jumps.

Four-dimensional positions

Attention does not inherently encode token positions. RoPE adds positional information by rotating queries and keys according to token position.

FLUX.3 uses four position axes: (t, h, w, l), representing timestep, height, width, and sequence length. Each axis receives one quarter, or 32 dimensions, of each 128-dimensional attention head.

Text tokens are arranged along l, while images and references use two-dimensional (h, w) grids. References are separated from the output image along l but retain the same spatial coordinates. This gives the model a precise interpretation of corresponding locations, which supports src_bbox and tgt_bbox editing.

Denoising

FLUX.3 uses flow matching. Generation starts with pure noise, and the transformer predicts a velocity at every one of the 50 steps. The scheduler moves the latent representation along that velocity toward a clean image.

The model can also produce an estimate of the final image at each step. In one example, the fox's pose and lighting are established by step 10 while the noise level remains 0.97. Snow-covered branches and fur texture appear during the final steps.

The scheduler uses a time shift of 6.93, keeping the noise level high for most of the run. Forty-four of the 50 steps occur above σ = 0.5.

Modular Diffusers

FLUX.3 is implemented as a Modular Diffusers pipeline. Workflows can be listed and their blocks inspected without loading model weights.

from diffusers import Flux3AutoBlocks

blocks = Flux3AutoBlocks()
print(blocks.available_workflows)
## ['text2image', 'image_conditioned']

print(list(blocks.get_workflow("text2image").sub_blocks))
## ['text_encoder', 'denoise.input', 'denoise.prepare_latents', 'denoise.set_timesteps',
## 'denoise.prepare_rope_inputs', 'denoise.denoise', 'denoise.after_denoise', 'decode']

get_workflow() resolves automatic branches and returns the sequence of steps that runs. The workflow documentation also exposes its components, inputs, defaults, and outputs.

Encode once, generate multiple images

The text encoder can be removed from the image-generation pipeline. A prompt can be encoded once, the text encoder released, and the cached embeddings reused for multiple generations.

import torch
from diffusers import ModularPipeline

model_id = "black-forest-labs/flux-3-image-klein-base"

blocks = ModularPipeline.from_pretrained(model_id).blocks
text_blocks = blocks.sub_blocks.pop("text_encoder")

text_pipe = text_blocks.init_pipeline(model_id)
text_pipe.load_components(dtype=torch.bfloat16)
text_pipe.to("cuda")

state = text_pipe(prompt=prompt, negative_prompt="")

prompt_embeds = state.get("prompt_embeds")
negative_prompt_embeds = state.get("negative_prompt_embeds")

del text_pipe
torch.cuda.empty_cache()

image_pipe = blocks.init_pipeline(model_id)
image_pipe.load_components(dtype={"vae": torch.float32, "default": torch.bfloat16})
image_pipe.to("cuda")

for seed in range(3):
    image = image_pipe(
        prompt_embeds=prompt_embeds,
        negative_prompt_embeds=negative_prompt_embeds,
        generator=torch.Generator().manual_seed(seed),
        output="images",
    )[0]
    image.save(f"{seed}.png")

Adding a block inside the denoising loop

Modular Diffusers can also insert custom blocks into an existing workflow. The following example records the model's estimate of the clean image at every denoising step.

from diffusers.modular_pipelines import (
    Flux3AutoBlocks,
    InputParam,
    ModularPipelineBlocks,
    OutputParam,
)

class Flux3PreviewStep(ModularPipelineBlocks):
    model_name = "flux3"

    @property
    def inputs(self):
        return [InputParam("latents"), InputParam("noise_pred")]

    @property
    def intermediate_outputs(self):
        return [OutputParam("x0_previews", type_hint=list)]

    @torch.no_grad()
    def __call__(self, components, block_state, i, t):
        if i == 0:
            block_state.x0_previews = []
        sigma = t / 1000
        block_state.x0_previews.append(
            block_state.latents - sigma * block_state.noise_pred
        )
        return components, block_state

blocks = Flux3AutoBlocks()
denoise_loop = blocks.sub_blocks["denoise"].sub_blocks["text2image"].sub_blocks["denoise"]
denoise_loop.sub_blocks.insert(
    "preview",
    Flux3PreviewStep(),
    1,
)

pipe = blocks.init_pipeline(model_id)
pipe.load_components(dtype={"vae": torch.float32, "default": torch.bfloat16})
pipe.to("cuda")

out = pipe(
    prompt=prompt,
    generator=torch.Generator().manual_seed(0),
    output=["x0_previews", "images"],
)

Resources

• Models: FLUX.3 Image Klein collection
• Documentation: FLUX.3 in Diffusers
• Fine-tuning: DreamBooth LoRA scripts
• Prompting: Black Forest Labs prompting guide