{ "@context": "https://schema.org", "@id": "https://aggr.aymericbeaumet.com/items/huggingface-co/2026-10-15-welcome-flux-3-image-klein-in-diffusers/#webpage", "@type": [ "WebPage", "ArchiveComponent" ], "articleBody": "[Black Forest Labs](https://huggingface.co/black-forest-labs) FLUX.3 Image Klein is here, and it runs in Diffusers from day one. One model does text-to-image and multi-reference image-to-image, and it adds something new, the **layout control**. You write bounding boxes into the prompt to say where each part of the image should go, or which parts of a reference image to move.\n\nBoth open checkpoints are in the [FLUX.3 Image Klein collection](https://huggingface.co/collections/black-forest-labs/flux-3-image-klein):\n\n| Checkpoint | What it is | Defaults |\n| ---------------------------------------------------------------------------------------------------------------- | --------------------------------------- | ---------------- |\n| [`black-forest-labs/flux-3-image-klein`](https://huggingface.co/black-forest-labs/flux-3-image-klein) | Time step distilled, for fast inference | 16 steps, no CFG |\n| [`black-forest-labs/flux-3-image-klein-base`](https://huggingface.co/black-forest-labs/flux-3-image-klein-base) | Base model | 50 steps, CFG |\n\n[![a red fox in a snowy forest (text-to-image), the same fox re-rendered from that image (image-to-image), a black stippled running silhouette on a chartreuse background, drawn inside its labeled bounding box, and a knitted knight moved from a \\\"from\\\" box to a \\\"to\\\" box next to a knitted dragon (layout editing)](https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/flux3/hero.png)](https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/flux3/hero.png)\n\nYou can head on to the [Hugging Face space](https://huggingface.co/spaces/black-forest-labs/flux-3-image-klein) and generate some images right now!\n\n## TL;DR\n\n- One model for text-to-image, multi-reference image-to-image, and bounding-box layout control.\n- The distilled checkpoint makes a 1024 × 1024 image in about **2 seconds** on an H100.\n- NVFP4 and FP8 weights load with `from_single_file`, for smaller GPUs.\n- LoRA training scripts for both text-to-image and image editing.\n- Ships as a [Modular Diffusers](https://huggingface.co/blog/flux-3#modular-diffusers) pipeline, so you can pop blocks out or add your own.\n\n## Contents\n\n- [Getting started](https://huggingface.co/blog/flux-3#getting-started)\n- [What FLUX.3 can do](https://huggingface.co/blog/flux-3#what-flux3-can-do)\n- [Base model and quantized variants](https://huggingface.co/blog/flux-3#base-model-and-quantized-variants)\n- [LoRA finetuning](https://huggingface.co/blog/flux-3#lora-finetuning)\n- [What it costs to run](https://huggingface.co/blog/flux-3#what-it-costs-to-run)\n- [The architecture](https://huggingface.co/blog/flux-3#the-architecture)\n- [Modular Diffusers](https://huggingface.co/blog/flux-3#modular-diffusers)\n- [Try it](https://huggingface.co/blog/flux-3#try-it)\n\n## Getting started\n\nInstall the latest `diffusers` and `transformers` (for the Qwen3-VL text encoder):\n\n```bash\npip install -U diffusers transformers\n```\n\nLoad the distilled checkpoint and generate an image:\n\n```python\nimport torch\nfrom diffusers import ModularPipeline\n\npipe = ModularPipeline.from_pretrained(\"black-forest-labs/flux-3-image-klein\")\npipe.load_components(dtype=torch.bfloat16)\npipe.to(\"cuda\")\n\npipe(prompt=\"a red fox\", output=\"images\")[0].save(\"fox.png\")\n```\n\n## What FLUX.3 can do\n\n### Text-to-image\n\nFLUX.3 responds best to a full scene description with the subject, the setting, the light, the composition, and the style. The [official prompting guide](https://docs.bfl.ai/guides/prompting_summary) from Black Forest Labs has more tips. The distilled checkpoint generates a `1024 × 1024` image in `16` steps by default. Pass `height`, `width`, or `num_inference_steps` to change them.\n\n```python\nprompt = (\n \"A wide-angle wildlife photograph of a red fox standing alert in a snow-covered forest clearing at golden hour. \"\n \"The fox is centered in the frame, its thick orange winter coat contrasting with the fresh white snow, breath \"\n \"visible in the cold air. Snow-laden pine trees fill the softly blurred background, and low warm sunlight rakes \"\n \"across the scene from the left, casting long blue shadows.\"\n)\n\nfox = pipe(\n prompt=prompt,\n generator=torch.Generator().manual_seed(0),\n output=\"images\"\n)[0]\nfox.save(\"fox.png\")\n```\n\n| [![A red fox standing alert in a snow-covered forest clearing, lit by low golden sunlight from the left](https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/flux3/t2i.png)](https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/flux3/t2i.png) |\n| ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |\n| Text to image |\n\n### Image-to-image\n\nPass a reference image (or more) with the `image` argument. Write the prompt as a caption of the image you *want*, with the change included.\n\n```python\nprompt = (\n \"The same red fox in the same snow-covered forest clearing, now at night under a full moon: cool blue \"\n \"moonlight, deep shadows, stars visible through the pine branches, the fox's eyes catching the light.\"\n)\nnight = pipe(\n prompt=prompt,\n image=fox, # from the previous text to image example\n generator=torch.Generator().manual_seed(0),\n output=\"images\",\n)[0]\nnight.save(\"night.png\")\n```\n\n| [![The fox from the previous image, relit in cold blue light](https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/flux3/i2i.png)](https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/flux3/i2i.png) |\n| --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |\n| Image to image |\n\nWith several references, refer to them in the prompt as ``, ``, ... in the order they are passed to `image`:\n\n```python\nred_fox = pipe(prompt=\"a red fox\", generator=torch.Generator().manual_seed(0), output=\"images\")[0]\nbench = pipe(prompt=\"a wooden bench\", generator=torch.Generator().manual_seed(0), output=\"images\")[0]\n\ncombined = pipe(\n prompt=\"The fox from sitting on the wooden bench from in a snowy setting.\",\n image=[red_fox, bench],\n output=\"images\",\n)[0]\n\ncombined.save(\"combined.png\")\n```\n\n| [![A red fox sitting on dry grass](https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/flux3/i2i_fox.png)](https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/flux3/i2i_fox.png) | [![A slatted wooden bench against a plain grey wall](https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/flux3/i2i_bench.png)](https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/flux3/i2i_bench.png) | [![The same fox sitting on the same bench, with snow on the ground](https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/flux3/combined.png)](https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/flux3/combined.png) |\n| -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |\n| ``: a red fox | ``: a wooden bench | The fox on the bench |\n\n### Layout control\n\nAdd a JSON list of regions after the caption. Each region has an `id`, a `bbox`, and a `desc`, and the caption mentions it as ``.\n\n> Boxes are written as `[y_min, x_min, y_max, x_max]`, **y first**, on a 0–1000 scale. Because the scale is normalized, the same box works at any image size.\n\n#### Placing regions in a new image\n\n```python\nlayout_prompt = (\n \"Minimalist graphic illustration featuring a black silhouette of a person centered against a \"\n \"solid, vibrant chartreuse background . The figure is captured in a dynamic, mid-stride running \"\n \"pose, facing toward the left side of the frame. \"\n '[{\"id\": \"background_1\", \"bbox\": [0, 0, 1000, 1000], \"desc\": \"A flat field of neon yellow-green with a subtle '\n 'paper texture and fine, organic grain.\"}, {\"id\": \"silhouette_1\", \"bbox\": [150, 150, 850, 850], \"desc\": \"A '\n \"black silhouette of a person in motion, made of dense stippled ink. The leading edges are solid; the trailing \"\n 'arm and leg dissolve into coarse square pixels and scattered dots.\"}]'\n)\n\nrunner = pipe(prompt=layout_prompt, generator=torch.Generator().manual_seed(0), output=\"images\")[0]\n```\n\n| [![Diagram: the caption and the two region descriptions on the left, the two boxes on a 0–1000 grid in the middle, and the generated image on the right, where a black stippled runner fills the silhouette box on a chartreuse background](https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/flux3/bbox-layout.png)](https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/flux3/bbox-layout.png) |\n| ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |\n| Each region's `desc` is drawn inside its `bbox` |\n\n#### Moving regions within a reference image\n\nFor edits, each region says where its content comes **from** (`src_bbox` in the reference) and where it goes (`tgt_bbox` in the output). A region with the same two boxes stays where it is.\n\nThis example moves the knitted knight up and to the left, and pins the backdrop, the cliff, the needle, the dragon, and the fire:\n\n```python\nfrom diffusers.utils import load_image\n\nknight_image = load_image(\"knight_orig.png\")\nedit_prompt = (\n \"In , move the miniature grey amigurumi knight figure upwards and to the left along the \"\n \"yarn cliff . Keep the rest of the image completely unchanged, preserving the blurred yarn backdrop \"\n \", the large knitted dragon , and the fiery orange thread . \"\n '[{\"id\": \"knight_1\", \"from\": \"ref_image_0\", \"src_bbox\": [500, 150, 850, 350], \"tgt_bbox\": [194, 55, 544, 255], '\n '\"desc\": \"A miniature amigurumi knight crocheted from thick grey woolen yarn.\"}, '\n '{\"id\": \"backdrop_1\", \"from\": \"ref_image_0\", \"src_bbox\": [0, 0, 1000, 1000], \"tgt_bbox\": [0, 0, 1000, 1000], '\n '\"desc\": \"A soft, out-of-focus backdrop of cream and indigo knitted yarn.\"}, '\n '{\"id\": \"cliff_1\", \"from\": \"ref_image_0\", \"src_bbox\": [300, 0, 1000, 450], \"tgt_bbox\": [300, 0, 1000, 450], '\n '\"desc\": \"A steep cliff of stacked yarn skeins in mustard, terracotta, and indigo.\"}, '\n '{\"id\": \"needle_1\", \"from\": \"ref_image_0\", \"src_bbox\": [420, 250, 550, 450], \"tgt_bbox\": [420, 250, 550, 450], '\n '\"desc\": \"An oversized polished steel tapestry needle.\"}, '\n '{\"id\": \"dragon_1\", \"from\": \"ref_image_0\", \"src_bbox\": [100, 600, 950, 1000], \"tgt_bbox\": [100, 600, 950, 1000], '\n '\"desc\": \"A large, fluffy knitted dragon in terracotta and mustard yarn.\"}, '\n '{\"id\": \"fire_1\", \"from\": \"ref_image_0\", \"src_bbox\": [350, 400, 650, 650], \"tgt_bbox\": [350, 400, 650, 650], '\n '\"desc\": \"Curled puffs of fiery orange angora thread suspended in mid-air.\"}]'\n)\n\nmoved = pipe(prompt=edit_prompt, image=knight_image, generator=torch.Generator().manual_seed(0), output=\"images\")[0]\n```\n\n| [![Left: the reference with the knight's source box. Right: the output with an arrow from the source box to the target box; the pinned regions are outlined in white](https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/flux3/knight.png)](https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/flux3/knight.png) |\n| ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |\n| The knight moves from its `src_bbox` to its `tgt_bbox`, and the rest of the scene stays put |\n\n## Base model and quantized variants\n\n### Base model\n\nEverything above uses the distilled checkpoint. The base checkpoint runs 50 steps with classifier-free guidance, and it's the one to fine-tune (see [LoRA finetuning](https://huggingface.co/blog/flux-3#lora-finetuning)).\n\n```python\npipe = ModularPipeline.from_pretrained(\"black-forest-labs/flux-3-image-klein-base\")\npipe.load_components(dtype={\"vae\": torch.float32, \"default\": torch.bfloat16})\npipe.to(\"cuda\")\n\npipe.guider = pipe.guider.new(guidance_scale=5.0) # optional\nimage = pipe(prompt=prompt, output=\"images\")[0]\n```\n\n### Single-file format checkpoints\n\nBlack Forest Labs also ships the transformer as a single file, in `bf16` and in two quantized formats: `fp8r` (rowwise FP8) and `nvfp4`. Load any of them with `from_single_file`, then load the other components by name so the bf16 transformer isn't downloaded too.\n\n```python\nfrom diffusers import BFLQuantizationConfig, Flux3Transformer2DModel, ModularPipeline\n\nmodel_id = \"black-forest-labs/flux-3-image-klein-base\"\nvariant = \"fp8r\" # or \"nvfp4\"\n\ntransformer = Flux3Transformer2DModel.from_single_file(\n f\"https://huggingface.co/{model_id}/blob/main/variants/{variant}/model.safetensors\",\n config=model_id,\n subfolder=\"transformer\",\n quantization_config=BFLQuantizationConfig(compute_dtype=torch.bfloat16),\n dtype=torch.bfloat16,\n)\n\npipe = ModularPipeline.from_pretrained(model_id)\npipe.load_components(dtype=torch.bfloat16)\npipe.update_components(transformer=transformer)\npipe.to(\"cuda\")\n```\n\nFor the unquantized single file, use `https://huggingface.co/{model_id}/blob/main/model.safetensors` and drop `quantization_config`.\n\n| Variant | Fast kernels on | Other GPUs |\n| ------- | ------------------------------------------------------------------------------ | ----------------------------------------------------------- |\n| `fp8r` | Compute capability 8.9+ (Ada, Hopper, Blackwell) | Weights are dequantized each step: lower memory, no speedup |\n| `nvfp4` | Blackwell, with `flashinfer-python` and `flashinfer-cubin` at the same version | Same as above |\n\n## LoRA finetuning\n\nFLUX.3 works for both text-to-image and image-to-image, which makes it a great base for custom LoRAs. `diffusers` ships DreamBooth LoRA training scripts for both tasks (you can also train with [Ostris' AI Toolkit](https://github.com/ostris/ai-toolkit)).\n\nHere is a text-to-image example that learns a tarot-card style from the [`multimodalart/1920-raider-waite-tarot-public-domain`](https://huggingface.co/datasets/multimodalart/1920-raider-waite-tarot-public-domain) dataset, triggered with `trtcrd`:\n\n```bash\naccelerate launch train_dreambooth_lora_flux3_klein.py \\\n --pretrained_model_name_or_path=\"black-forest-labs/flux-3-image-klein-base\" \\\n --dataset_name=\"multimodalart/1920-raider-waite-tarot-public-domain\" \\\n --caption_column=\"caption\" \\\n --instance_prompt=\"trtcrd\" \\\n --output_dir=\"trained-flux3-tarot-lora\" \\\n --mixed_precision=\"bf16\" \\\n --resolution=1024 \\\n --use_aspect_ratio_buckets \\\n --train_batch_size=1 \\\n --rank=16 \\\n --lora_alpha=16 \\\n --learning_rate=1e-4 \\\n --lr_scheduler=\"constant\" \\\n --max_train_steps=1500 \\\n --validation_prompt=\"a trtcrd of a cat, tarot style\" \\\n --validation_epochs=25 \\\n --seed=0 \\\n --push_to_hub\n```\n\n> To reduce memory usage, add a few optimization flags:\n>\n> ```bash\n> --gradient_checkpointing \\\n> --cache_latents \\\n> --use_8bit_adam\n> ```\n>\n> `--gradient_checkpointing` recomputes activations in the backward pass (lower VRAM, slightly slower), `--cache_latents` pre-encodes the images and frees the VAE, and `--use_8bit_adam` keeps the optimizer states in 8-bit.\n\nA couple of knobs are worth experimenting with. The grid below trains the same tarot style with different LoRA ranks and optimizers; the adapters are on the Hub at [`diffusers-modular/flux3-tarot-lora`](https://huggingface.co/diffusers-modular/flux3-tarot-lora). More training features and configurations are documented in the [DreamBooth FLUX.3 README](https://github.com/huggingface/diffusers/blob/main/examples/dreambooth/README_flux3_klein.md).\n\n[![FLUX.3 tarot-style LoRA across ranks and optimizers: the base model with no LoRA, then rank 16, rank 8 and rank 32 (AdamW, lr 1e-4) and a Prodigy run, on two held-out prompts](https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/flux3/tarot-lora-sweep.png)](https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/flux3/tarot-lora-sweep.png)\n\nFor instruction-based editing, the image-to-image script (`train_dreambooth_lora_flux3_klein_img2img.py`) trains on (reference image, target image, instruction) triplets, where the reference is addressed in the prompt as ``.\n\nSee the [DreamBooth FLUX.3 README](https://github.com/huggingface/diffusers/blob/main/examples/dreambooth/README_flux3_klein.md) for all options and the full image-to-image example.\n\n## What it costs to run\n\nIn this section, we will talk about the memory consumption and the speed of generation of the models.\n\n### Memory\n\n| name | peak allocated (after load) GiBs | peak allocated (after inference) GiBs |\n| --------------- | -------------------------------- | ------------------------------------- |\n| nvfp4 (RTX6000) | 13.4 | 15.3 |\n| fp8r (H100) | 14.35 | 16.27 |\n| bf16 (H100) | 18.80 | 20.72 |\n\nIf you have less memory:\n\n- Load a [quantized transformer](https://huggingface.co/blog/flux-3#base-model-and-quantized-variants).\n- Run the text encoder separately and free it. It only runs once per image, before denoising (see [Encode once, generate many](https://huggingface.co/blog/flux-3#encode-once-generate-many)).\n- Turn on automatic CPU offloading, which moves each component to the GPU only while it runs (we didn't benchmark this)\n\n```python\nfrom diffusers import ComponentsManager, ModularPipeline\n\nmanager = ComponentsManager()\nmanager.enable_auto_cpu_offload(device=\"cuda\")\n\npipe = ModularPipeline.from_pretrained(\"black-forest-labs/flux-3-image-klein\", components_manager=manager)\npipe.load_components(dtype={\"vae\": torch.float32, \"default\": torch.bfloat16})\n# no pipe.to(\"cuda\"): the manager handles device placement\n```\n\n### Speed\n\nEnd-to-end timings for the distilled checkpoint (16 steps), averaged over three runs after a warmup:\n\n| Workflow | Output size | Time | Peak memory |\n| ----------------------------------------------------- | ----------- | ------ | ----------- |\n| Text-to-image | 1024 × 1024 | 1.93 s | 23.8 GiB |\n| Image-to-image, one 1024 × 1024 reference | 1024 × 1024 | 3.87 s | 23.8 GiB |\n| Layout editing (the knight), one 1360 × 768 reference | 1360 × 768 | 4.10 s | 23.8 GiB |\n\n> Single NVIDIA H100 80GB, no offloading, default attention backend.\n\nA reference image roughly doubles the time. Reference tokens join the same sequence as the text and the image being generated, so a 1024 × 1024 reference adds 4,096 tokens to every transformer call\n\n| [![Stacked bars of tokens per transformer call: text-to-image 4,256 tokens; image-to-image 8,272; layout text-to-image 4,336; layout image-to-image 8,800](https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/flux3/tokens-per-workflow.png)](https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/flux3/tokens-per-workflow.png) |\n| -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |\n| Stacked bars of tokens per transformer call: text-to-image 4,256 tokens; image-to-image 8,272; layout text-to-image 4,336; layout image-to-image 8,800 |\n\n## The architecture\n\nLet's open the model up, one piece at a time. The model takes three kinds of tokens:\n\n1. **text**\n2. the **noisy image** being generated\n3. and optional **reference images**\n\n### From pixels to tokens\n\nFLUX.3 is a latent-space flow model for image generation. This means that the denoiser takes latents as inputs and produces latents as outputs. We use a VAE to decode the latents back to pixels. FLUX.3 uses the same VAE architecture as Flux.2.\n\nThe FLUX.2 VAE compresses an image 8× in each direction into 32 channels, and then every 2 × 2 patch of that latent is packed into a single token with 128 channels.\n\nSo **one token covers a 16 × 16 pixel patch**, and a 1024 × 1024 image becomes a 64 × 64 grid of 4,096 tokens. The noisy image starts as random noise of exactly that shape. A reference image goes through the same VAE encoder and packing, and keeps its own size.\n\n### From prompt to tokens\n\nThe text encoder is Qwen3-VL, a vision-language model. FLUX.3 doesn't just take its last layer. It reads the hidden states of **eight** of its 36 layers (layers 4, 8, …, 32) and stacks them, so each text token becomes a 8 × 2560 = 20,480-dimensional vector before a linear layer projects it to the transformer's width of 3072.\n\nThis gives the transformer features from several depths of the language model, not just its final layer. The prompt is padded to a multiple of 80 tokens, and the transformer attends to the padding too, which is why the pipeline takes one prompt per call.\n\n### Two stages\n\nThe transformer processes the tokens in two stages:\n\n1. **Stage 1: per-mode blocks.** Each stream goes through 5 blocks with its own weights. Text has its own blocks, the noisy image has its own, and the reference images have their own.\n2. **Stage 2: joint blocks.** The streams are concatenated into one sequence, `[text | image | reference]`, which goes through 28 blocks that share one set of weights.\n\nThe two stages differ in *who can attend to whom*.\n\nIn stage 1, each stream only attends to itself, so each kind of token gets refined on its own terms first. In stage 2, everything attends to everything, where the image learns from the text and from the references. For comparison, FLUX.2's double-stream blocks also give text and image separate weights, but let them attend to each other from the start. FLUX.3 keeps the streams fully apart first, then merges them.\n\nAt the end, the text and reference tokens are simply dropped. Only the image tokens go through the output layers, which predict how to move them one step toward a clean image.\n\n### Inside one block\n\nEvery block, in either stage, has the same shape. Attention and the MLP run **in parallel** on the same normalized input, and their sum is added back to the residual stream.\n\nEach block starts with a LayerNorm that has no learned scale or shift of its own. The scale, shift, and gate come from a timestep embedding instead (this is called *modulation*). Inside the attention, the queries and keys are RMS-normalized and get rotary position embeddings (RoPE), which we look at below.\n\nIn a joint block, all tokens share the same weights, but text, image, and reference tokens each get their own scale, shift, and gate.\n\n```python\ndef block(x, shift, scale, gate): # shift, scale, gate come from the token's timestep embedding\n h = layer_norm(x) # no learned scale or shift\n h = (1 + scale) * h + shift\n q, k, v = to_q(h), to_k(h), to_v(h)\n q, k = rope(rms_norm(q)), rope(rms_norm(k))\n out = attention(q, k, v) + mlp(h) # parallel, MLP is SwiGLU\n return x + gate * out\n```\n\n### Every token gets its own timestep\n\nIn FLUX.1 and FLUX.2, one timestep embedding modulates every token. In FLUX.3, the modulation depends on the kind of token.\n\nThe noisy image tokens are modulated by the current timestep `t`. The text and reference tokens are always modulated by timestep **0**, which means \"clean, nothing to denoise here\". So even a reference that looks almost the same as the output is marked as given context, not as something to generate.\n\nThe distilled checkpoint is a *flow map*. It is also told the timestep each step jumps to. That second timestep gets its own embedding, which is added to every token's modulation, so each of its 16 steps can make a larger jump.\n\n### Positions in four dimensions\n\nAttention on its own doesn't know where a token is (this phenomenon is called permutation equivariance). RoPE fixes that by rotating each query and key by an amount that depends on the token's position. FLUX.3 uses four position axes, `(t, h, w, l)` (timestep, height, width, length of sequence), and gives each one a quarter (32 dimensions) of every 128-dimensional attention head:\n\n| Tokens | t: timestep | h: height | w: width | l: length of sequence |\n| --------------------------- | ----------- | --------- | -------- | --------------------- |\n| text token *i* | 0 | 0 | 0 | *i* |\n| image token at (row, col) | 0 | row / 2 | col / 2 | 0 |\n| reference *k* at (row, col) | 0 | row / 2 | col / 2 | −(*k* + 1) |\n\nText tokens line up along `l`, like words in a sentence. The image and the references are 2D grids in `(h, w)`, separated from each other along `l`. Because a reference uses the same `(h, w)` coordinates as the output, \"the same spot in the reference\" has a precise meaning to the model, which is what the `src_bbox` / `tgt_bbox` editing relies on.\n\n### The denoising loop\n\nFLUX.3 is a flow-matching model. Generation starts from pure noise, and at each of the 50 steps the transformer predicts a *velocity*. This is the direction from noise toward a clean image. The scheduler takes one step along it.\n\nAt every step you can also read off the model's current guess of the final image. Decoding that guess at a few steps shows how the image takes shape:\n\n| [![step01](https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/flux3/traj_step01.png)](https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/flux3/traj_step01.png) | [![step05](https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/flux3/traj_step05.png)](https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/flux3/traj_step05.png) | [![step10](https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/flux3/traj_step10.png)](https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/flux3/traj_step10.png) | [![step15](https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/flux3/traj_step15.png)](https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/flux3/traj_step15.png) | [![step25](https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/flux3/traj_step25.png)](https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/flux3/traj_step25.png) | [![step50](https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/flux3/traj_step50.png)](https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/flux3/traj_step50.png) |\n| ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |\n| Step 1 | Step 5 | Step 10 | Step 15 | Step 25 | Step 50 |\n\nThe fox, its pose, and the light are already in place by step 10, while the noise level is still 0.97. The snow-laden branches and the texture of the fur only appear in the last few steps. FLUX.3's scheduler uses a large time shift (6.93), which keeps the noise level high for most of the run: 44 of the 50 steps happen above σ = 0.5. We built this figure by adding one small block to the pipeline, which brings us to Modular Diffusers.\n\n> If you are new to flow matching, we have a beginner [friendly tutorial](https://huggingface.co/spaces/ariG23498/flow-matching) and a [video](https://youtu.be/hXQZ6lyRCsI) for you to understand how it works.\n\n## Modular Diffusers\n\nFLUX.3 ships as a [Modular Diffusers](https://huggingface.co/docs/diffusers/main/en/modular_diffusers/overview) pipeline.\n\n### See the workflows\n\nYou can list the workflows and print the exact blocks of each one, without loading any weights.\n\n```python\nfrom diffusers import Flux3AutoBlocks\n\nblocks = Flux3AutoBlocks()\nprint(blocks.available_workflows)\n# ['text2image', 'image_conditioned']\n\nprint(list(blocks.get_workflow(\"text2image\").sub_blocks))\n# ['text_encoder', 'denoise.input', 'denoise.prepare_latents', 'denoise.set_timesteps',\n# 'denoise.prepare_rope_inputs', 'denoise.denoise', 'denoise.after_denoise', 'decode']\n```\n\n`get_workflow()` resolves the auto branches and gives you the plain sequence of steps that will run. `print(blocks.get_workflow(\"image_conditioned\").doc)` prints the components, inputs, defaults, and outputs of that workflow.\n\n### Encode once, generate many\n\nPop the text encoder out, encode the prompt once, free it, and generate from the cached embeddings.\n\n```python\nimport torch\nfrom diffusers import ModularPipeline\n\nmodel_id = \"black-forest-labs/flux-3-image-klein-base\"\n\nblocks = ModularPipeline.from_pretrained(model_id).blocks\ntext_blocks = blocks.sub_blocks.pop(\"text_encoder\") # `blocks` now starts at the VAE encoder\n\ntext_pipe = text_blocks.init_pipeline(model_id)\ntext_pipe.load_components(dtype=torch.bfloat16)\ntext_pipe.to(\"cuda\")\n\nstate = text_pipe(prompt=prompt, negative_prompt=\"\") # no guider here, so ask for the negative embeddings\n\nprompt_embeds = state.get(\"prompt_embeds\")\nnegative_prompt_embeds = state.get(\"negative_prompt_embeds\")\n\ndel text_pipe\ntorch.cuda.empty_cache()\n\nimage_pipe = blocks.init_pipeline(model_id) # no text encoder to load\nimage_pipe.load_components(dtype={\"vae\": torch.float32, \"default\": torch.bfloat16})\nimage_pipe.to(\"cuda\")\n\nfor seed in range(3):\n image = image_pipe(\n prompt_embeds=prompt_embeds,\n negative_prompt_embeds=negative_prompt_embeds,\n generator=torch.Generator().manual_seed(seed),\n output=\"images\",\n )[0]\n image.save(f\"{seed}.png\")\n```\n\n### Add a step inside the loop\n\nThe denoising figure above came from one extra block that records the model's guess of the clean image at every step:\n\n```python\nfrom diffusers.modular_pipelines import Flux3AutoBlocks, InputParam, ModularPipelineBlocks, OutputParam\n\n\nclass Flux3PreviewStep(ModularPipelineBlocks):\n model_name = \"flux3\"\n\n @property\n def inputs(self):\n return [InputParam(\"latents\"), InputParam(\"noise_pred\")]\n\n @property\n def intermediate_outputs(self):\n return [OutputParam(\"x0_previews\", type_hint=list)]\n\n @torch.no_grad()\n def __call__(self, components, block_state, i, t):\n if i == 0:\n block_state.x0_previews = []\n sigma = t / 1000 # timesteps are sigmas scaled to [0, 1000]\n block_state.x0_previews.append(block_state.latents - sigma * block_state.noise_pred) # x0 = x_t - sigma * v\n return components, block_state\n\n\nblocks = Flux3AutoBlocks()\ndenoise_loop = blocks.sub_blocks[\"denoise\"].sub_blocks[\"text2image\"].sub_blocks[\"denoise\"]\ndenoise_loop.sub_blocks.insert(\"preview\", Flux3PreviewStep(), 1) # between \"denoiser\" and \"after_denoiser\"\n\npipe = blocks.init_pipeline(model_id)\npipe.load_components(dtype={\"vae\": torch.float32, \"default\": torch.bfloat16})\npipe.to(\"cuda\")\n\nout = pipe(prompt=prompt, generator=torch.Generator().manual_seed(0), output=[\"x0_previews\", \"images\"])\n```\n\n## Try it\n\n- **Models:** [FLUX.3 Image Klein collection](https://huggingface.co/collections/black-forest-labs/flux-3-image-klein)\n- **Docs:** [FLUX.3 in Diffusers](https://huggingface.co/docs/diffusers/main/en/api/pipelines/flux3)\n- **Fine-tune:** [DreamBooth LoRA scripts](https://github.com/huggingface/diffusers/blob/main/examples/dreambooth/README_flux3_klein.md)\n- **Prompting:** [Black Forest Labs prompting guide](https://docs.bfl.ai/guides/prompting_summary)\n\nBuilt something with FLUX.3? Share your LoRAs and pipelines on the Hub. We'd love to see them! 😎\n", "authors": [], "captured_at": "2026-10-15T18:36:21.248140220+00:00", "category": "ai", "content_html": "

Black Forest Labs FLUX.3 Image Klein is here, and it runs in Diffusers from day one. One model does text-to-image and multi-reference image-to-image, and it adds something new, the layout control. You write bounding boxes into the prompt to say where each part of the image should go, or which parts of a reference image to move.

\n

Both open checkpoints are in the FLUX.3 Image Klein collection:

\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n
CheckpointWhat it isDefaults
black-forest-labs/flux-3-image-kleinTime step distilled, for fast inference16 steps, no CFG
black-forest-labs/flux-3-image-klein-baseBase model50 steps, CFG
\n

\"a

\n

You can head on to the Hugging Face space and generate some images right now!

\n

TL;DR

\n
    \n
  • One model for text-to-image, multi-reference image-to-image, and bounding-box layout control.
    • \n
  • The distilled checkpoint makes a 1024 × 1024 image in about 2 seconds on an H100.
    • \n
  • NVFP4 and FP8 weights load with from_single_file, for smaller GPUs.
    • \n
  • LoRA training scripts for both text-to-image and image editing.
    • \n
  • Ships as a Modular Diffusers pipeline, so you can pop blocks out or add your own.
    • \n
\n

Contents

\n\n

Getting started

\n

Install the latest diffusers and transformers (for the Qwen3-VL text encoder):

\n
pip install -U diffusers transformers\n
\n

Load the distilled checkpoint and generate an image:

\n
import torch\nfrom diffusers import ModularPipeline\n\npipe = ModularPipeline.from_pretrained("black-forest-labs/flux-3-image-klein")\npipe.load_components(dtype=torch.bfloat16)\npipe.to("cuda")\n\npipe(prompt="a red fox", output="images")[0].save("fox.png")\n
\n

What FLUX.3 can do

\n

Text-to-image

\n

FLUX.3 responds best to a full scene description with the subject, the setting, the light, the composition, and the style. The official prompting guide from Black Forest Labs has more tips. The distilled checkpoint generates a 1024 × 1024 image in 16 steps by default. Pass height, width, or num_inference_steps to change them.

\n
prompt = (\n    "A wide-angle wildlife photograph of a red fox standing alert in a snow-covered forest clearing at golden hour. "\n    "The fox is centered in the frame, its thick orange winter coat contrasting with the fresh white snow, breath "\n    "visible in the cold air. Snow-laden pine trees fill the softly blurred background, and low warm sunlight rakes "\n    "across the scene from the left, casting long blue shadows."\n)\n\nfox = pipe(\n    prompt=prompt,\n    generator=torch.Generator().manual_seed(0),\n    output="images"\n)[0]\nfox.save("fox.png")\n
\n\n\n\n\n\n\n\n\n\n\n\n
\"A
Text to image
\n

Image-to-image

\n

Pass a reference image (or more) with the image argument. Write the prompt as a caption of the image you want, with the change included.

\n
prompt = (\n    "The same red fox in the same snow-covered forest clearing, now at night under a full moon: cool blue "\n    "moonlight, deep shadows, stars visible through the pine branches, the fox's eyes catching the light."\n)\nnight = pipe(\n    prompt=prompt,\n    image=fox, # from the previous text to image example\n    generator=torch.Generator().manual_seed(0),\n    output="images",\n)[0]\nnight.save("night.png")\n
\n\n\n\n\n\n\n\n\n\n\n\n
\"The
Image to image
\n

With several references, refer to them in the prompt as <ref_image_0>, <ref_image_1>, ... in the order they are passed to image:

\n
red_fox = pipe(prompt="a red fox", generator=torch.Generator().manual_seed(0), output="images")[0]\nbench = pipe(prompt="a wooden bench", generator=torch.Generator().manual_seed(0), output="images")[0]\n\ncombined = pipe(\n    prompt="The fox from <ref_image_0> sitting on the wooden bench from <ref_image_1> in a snowy setting.",\n    image=[red_fox, bench],\n    output="images",\n)[0]\n\ncombined.save("combined.png")\n
\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n
\"A\"A\"The
<ref_image_0>: a red fox<ref_image_1>: a wooden benchThe fox on the bench
\n

Layout control

\n

Add a JSON list of regions after the caption. Each region has an id, a bbox, and a desc, and the caption mentions it as <id>.

\n
\n

Boxes are written as [y_min, x_min, y_max, x_max], y first, on a 0–1000 scale. Because the scale is normalized, the same box works at any image size.

\n
\n

Placing regions in a new image

\n
layout_prompt = (\n    "Minimalist graphic illustration featuring a black silhouette of a person <silhouette_1> centered against a "\n    "solid, vibrant chartreuse background <background_1>. The figure is captured in a dynamic, mid-stride running "\n    "pose, facing toward the left side of the frame. "\n    '[{"id": "background_1", "bbox": [0, 0, 1000, 1000], "desc": "A flat field of neon yellow-green with a subtle '\n    'paper texture and fine, organic grain."}, {"id": "silhouette_1", "bbox": [150, 150, 850, 850], "desc": "A '\n    "black silhouette of a person in motion, made of dense stippled ink. The leading edges are solid; the trailing "\n    'arm and leg dissolve into coarse square pixels and scattered dots."}]'\n)\n\nrunner = pipe(prompt=layout_prompt, generator=torch.Generator().manual_seed(0), output="images")[0]\n
\n\n\n\n\n\n\n\n\n\n\n\n
\"Diagram:
Each region's desc is drawn inside its bbox
\n

Moving regions within a reference image

\n

For edits, each region says where its content comes from (src_bbox in the reference) and where it goes (tgt_bbox in the output). A region with the same two boxes stays where it is.

\n

This example moves the knitted knight up and to the left, and pins the backdrop, the cliff, the needle, the dragon, and the fire:

\n
from diffusers.utils import load_image\n\nknight_image = load_image("knight_orig.png")\nedit_prompt = (\n    "In <ref_image_0>, move the miniature grey amigurumi knight figure <knight_1> upwards and to the left along the "\n    "yarn cliff <cliff_1>. Keep the rest of the image completely unchanged, preserving the blurred yarn backdrop "\n    "<backdrop_1>, the large knitted dragon <dragon_1>, and the fiery orange thread <fire_1>. "\n    '[{"id": "knight_1", "from": "ref_image_0", "src_bbox": [500, 150, 850, 350], "tgt_bbox": [194, 55, 544, 255], '\n    '"desc": "A miniature amigurumi knight crocheted from thick grey woolen yarn."}, '\n    '{"id": "backdrop_1", "from": "ref_image_0", "src_bbox": [0, 0, 1000, 1000], "tgt_bbox": [0, 0, 1000, 1000], '\n    '"desc": "A soft, out-of-focus backdrop of cream and indigo knitted yarn."}, '\n    '{"id": "cliff_1", "from": "ref_image_0", "src_bbox": [300, 0, 1000, 450], "tgt_bbox": [300, 0, 1000, 450], '\n    '"desc": "A steep cliff of stacked yarn skeins in mustard, terracotta, and indigo."}, '\n    '{"id": "needle_1", "from": "ref_image_0", "src_bbox": [420, 250, 550, 450], "tgt_bbox": [420, 250, 550, 450], '\n    '"desc": "An oversized polished steel tapestry needle."}, '\n    '{"id": "dragon_1", "from": "ref_image_0", "src_bbox": [100, 600, 950, 1000], "tgt_bbox": [100, 600, 950, 1000], '\n    '"desc": "A large, fluffy knitted dragon in terracotta and mustard yarn."}, '\n    '{"id": "fire_1", "from": "ref_image_0", "src_bbox": [350, 400, 650, 650], "tgt_bbox": [350, 400, 650, 650], '\n    '"desc": "Curled puffs of fiery orange angora thread suspended in mid-air."}]'\n)\n\nmoved = pipe(prompt=edit_prompt, image=knight_image, generator=torch.Generator().manual_seed(0), output="images")[0]\n
\n\n\n\n\n\n\n\n\n\n\n\n
\"Left:
The knight moves from its src_bbox to its tgt_bbox, and the rest of the scene stays put
\n

Base model and quantized variants

\n

Base model

\n

Everything above uses the distilled checkpoint. The base checkpoint runs 50 steps with classifier-free guidance, and it's the one to fine-tune (see LoRA finetuning).

\n
pipe = ModularPipeline.from_pretrained("black-forest-labs/flux-3-image-klein-base")\npipe.load_components(dtype={"vae": torch.float32, "default": torch.bfloat16})\npipe.to("cuda")\n\npipe.guider = pipe.guider.new(guidance_scale=5.0)  # optional\nimage = pipe(prompt=prompt, output="images")[0]\n
\n

Single-file format checkpoints

\n

Black Forest Labs also ships the transformer as a single file, in bf16 and in two quantized formats: fp8r (rowwise FP8) and nvfp4. Load any of them with from_single_file, then load the other components by name so the bf16 transformer isn't downloaded too.

\n
from diffusers import BFLQuantizationConfig, Flux3Transformer2DModel, ModularPipeline\n\nmodel_id = "black-forest-labs/flux-3-image-klein-base"\nvariant = "fp8r"  # or "nvfp4"\n\ntransformer = Flux3Transformer2DModel.from_single_file(\n    f"https://huggingface.co/{model_id}/blob/main/variants/{variant}/model.safetensors",\n    config=model_id,\n    subfolder="transformer",\n    quantization_config=BFLQuantizationConfig(compute_dtype=torch.bfloat16),\n    dtype=torch.bfloat16,\n)\n\npipe = ModularPipeline.from_pretrained(model_id)\npipe.load_components(dtype=torch.bfloat16)\npipe.update_components(transformer=transformer)\npipe.to("cuda")\n
\n

For the unquantized single file, use https://huggingface.co/{model_id}/blob/main/model.safetensors and drop quantization_config.

\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n
VariantFast kernels onOther GPUs
fp8rCompute capability 8.9+ (Ada, Hopper, Blackwell)Weights are dequantized each step: lower memory, no speedup
nvfp4Blackwell, with flashinfer-python and flashinfer-cubin at the same versionSame as above
\n

LoRA finetuning

\n

FLUX.3 works for both text-to-image and image-to-image, which makes it a great base for custom LoRAs. diffusers ships DreamBooth LoRA training scripts for both tasks (you can also train with Ostris' AI Toolkit).

\n

Here is a text-to-image example that learns a tarot-card style from the multimodalart/1920-raider-waite-tarot-public-domain dataset, triggered with trtcrd:

\n
accelerate launch train_dreambooth_lora_flux3_klein.py \\\n  --pretrained_model_name_or_path="black-forest-labs/flux-3-image-klein-base" \\\n  --dataset_name="multimodalart/1920-raider-waite-tarot-public-domain" \\\n  --caption_column="caption" \\\n  --instance_prompt="trtcrd" \\\n  --output_dir="trained-flux3-tarot-lora" \\\n  --mixed_precision="bf16" \\\n  --resolution=1024 \\\n  --use_aspect_ratio_buckets \\\n  --train_batch_size=1 \\\n  --rank=16 \\\n  --lora_alpha=16 \\\n  --learning_rate=1e-4 \\\n  --lr_scheduler="constant" \\\n  --max_train_steps=1500 \\\n  --validation_prompt="a trtcrd of a cat, tarot style" \\\n  --validation_epochs=25 \\\n  --seed=0 \\\n  --push_to_hub\n
\n
\n

To reduce memory usage, add a few optimization flags:

\n
  --gradient_checkpointing \\\n  --cache_latents \\\n  --use_8bit_adam\n
\n

--gradient_checkpointing recomputes activations in the backward pass (lower VRAM, slightly slower), --cache_latents pre-encodes the images and frees the VAE, and --use_8bit_adam keeps the optimizer states in 8-bit.

\n
\n

A couple of knobs are worth experimenting with. The grid below trains the same tarot style with different LoRA ranks and optimizers; the adapters are on the Hub at diffusers-modular/flux3-tarot-lora. More training features and configurations are documented in the DreamBooth FLUX.3 README.

\n

\"FLUX.3

\n

For instruction-based editing, the image-to-image script (train_dreambooth_lora_flux3_klein_img2img.py) trains on (reference image, target image, instruction) triplets, where the reference is addressed in the prompt as <ref_image_0>.

\n

See the DreamBooth FLUX.3 README for all options and the full image-to-image example.

\n

What it costs to run

\n

In this section, we will talk about the memory consumption and the speed of generation of the models.

\n

Memory

\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n
namepeak allocated (after load) GiBspeak allocated (after inference) GiBs
nvfp4 (RTX6000)13.415.3
fp8r (H100)14.3516.27
bf16 (H100)18.8020.72
\n

If you have less memory:

\n
    \n
  • Load a quantized transformer.
    • \n
  • Run the text encoder separately and free it. It only runs once per image, before denoising (see Encode once, generate many).
    • \n
  • Turn on automatic CPU offloading, which moves each component to the GPU only while it runs (we didn't benchmark this)
    • \n
\n
from diffusers import ComponentsManager, ModularPipeline\n\nmanager = ComponentsManager()\nmanager.enable_auto_cpu_offload(device="cuda")\n\npipe = ModularPipeline.from_pretrained("black-forest-labs/flux-3-image-klein", components_manager=manager)\npipe.load_components(dtype={"vae": torch.float32, "default": torch.bfloat16})\n# no pipe.to("cuda"): the manager handles device placement\n
\n

Speed

\n

End-to-end timings for the distilled checkpoint (16 steps), averaged over three runs after a warmup:

\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n
WorkflowOutput sizeTimePeak memory
Text-to-image1024 × 10241.93 s23.8 GiB
Image-to-image, one 1024 × 1024 reference1024 × 10243.87 s23.8 GiB
Layout editing (the knight), one 1360 × 768 reference1360 × 7684.10 s23.8 GiB
\n
\n

Single NVIDIA H100 80GB, no offloading, default attention backend.

\n
\n

A reference image roughly doubles the time. Reference tokens join the same sequence as the text and the image being generated, so a 1024 × 1024 reference adds 4,096 tokens to every transformer call

\n\n\n\n\n\n\n\n\n\n\n\n
\"Stacked
Stacked bars of tokens per transformer call: text-to-image 4,256 tokens; image-to-image 8,272; layout text-to-image 4,336; layout image-to-image 8,800
\n

The architecture

\n

Let's open the model up, one piece at a time. The model takes three kinds of tokens:

\n
    \n
  1. text
    2. \n
  2. the noisy image being generated
    3. \n
  3. and optional reference images
    4. \n
\n

From pixels to tokens

\n

FLUX.3 is a latent-space flow model for image generation. This means that the denoiser takes latents as inputs and produces latents as outputs. We use a VAE to decode the latents back to pixels. FLUX.3 uses the same VAE architecture as Flux.2.

\n

The FLUX.2 VAE compresses an image 8× in each direction into 32 channels, and then every 2 × 2 patch of that latent is packed into a single token with 128 channels.

\n

So one token covers a 16 × 16 pixel patch, and a 1024 × 1024 image becomes a 64 × 64 grid of 4,096 tokens. The noisy image starts as random noise of exactly that shape. A reference image goes through the same VAE encoder and packing, and keeps its own size.

\n

From prompt to tokens

\n

The text encoder is Qwen3-VL, a vision-language model. FLUX.3 doesn't just take its last layer. It reads the hidden states of eight of its 36 layers (layers 4, 8, …, 32) and stacks them, so each text token becomes a 8 × 2560 = 20,480-dimensional vector before a linear layer projects it to the transformer's width of 3072.

\n

This gives the transformer features from several depths of the language model, not just its final layer. The prompt is padded to a multiple of 80 tokens, and the transformer attends to the padding too, which is why the pipeline takes one prompt per call.

\n

Two stages

\n

The transformer processes the tokens in two stages:

\n
    \n
  1. Stage 1: per-mode blocks. Each stream goes through 5 blocks with its own weights. Text has its own blocks, the noisy image has its own, and the reference images have their own.
    2. \n
  2. Stage 2: joint blocks. The streams are concatenated into one sequence, [text | image | reference], which goes through 28 blocks that share one set of weights.
    3. \n
\n

The two stages differ in who can attend to whom.

\n

In stage 1, each stream only attends to itself, so each kind of token gets refined on its own terms first. In stage 2, everything attends to everything, where the image learns from the text and from the references. For comparison, FLUX.2's double-stream blocks also give text and image separate weights, but let them attend to each other from the start. FLUX.3 keeps the streams fully apart first, then merges them.

\n

At the end, the text and reference tokens are simply dropped. Only the image tokens go through the output layers, which predict how to move them one step toward a clean image.

\n

Inside one block

\n

Every block, in either stage, has the same shape. Attention and the MLP run in parallel on the same normalized input, and their sum is added back to the residual stream.

\n

Each block starts with a LayerNorm that has no learned scale or shift of its own. The scale, shift, and gate come from a timestep embedding instead (this is called modulation). Inside the attention, the queries and keys are RMS-normalized and get rotary position embeddings (RoPE), which we look at below.

\n

In a joint block, all tokens share the same weights, but text, image, and reference tokens each get their own scale, shift, and gate.

\n
def block(x, shift, scale, gate):      # shift, scale, gate come from the token's timestep embedding\n    h = layer_norm(x)                  # no learned scale or shift\n    h = (1 + scale) * h + shift\n    q, k, v = to_q(h), to_k(h), to_v(h)\n    q, k = rope(rms_norm(q)), rope(rms_norm(k))\n    out = attention(q, k, v) + mlp(h)  # parallel, MLP is SwiGLU\n    return x + gate * out\n
\n

Every token gets its own timestep

\n

In FLUX.1 and FLUX.2, one timestep embedding modulates every token. In FLUX.3, the modulation depends on the kind of token.

\n

The noisy image tokens are modulated by the current timestep t. The text and reference tokens are always modulated by timestep 0, which means "clean, nothing to denoise here". So even a reference that looks almost the same as the output is marked as given context, not as something to generate.

\n

The distilled checkpoint is a flow map. It is also told the timestep each step jumps to. That second timestep gets its own embedding, which is added to every token's modulation, so each of its 16 steps can make a larger jump.

\n

Positions in four dimensions

\n

Attention on its own doesn't know where a token is (this phenomenon is called permutation equivariance). RoPE fixes that by rotating each query and key by an amount that depends on the token's position. FLUX.3 uses four position axes, (t, h, w, l) (timestep, height, width, length of sequence), and gives each one a quarter (32 dimensions) of every 128-dimensional attention head:

\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n
Tokenst: timesteph: heightw: widthl: length of sequence
text token i000i
image token at (row, col)0row / 2col / 20
reference k at (row, col)0row / 2col / 2−(k + 1)
\n

Text tokens line up along l, like words in a sentence. The image and the references are 2D grids in (h, w), separated from each other along l. Because a reference uses the same (h, w) coordinates as the output, "the same spot in the reference" has a precise meaning to the model, which is what the src_bbox / tgt_bbox editing relies on.

\n

The denoising loop

\n

FLUX.3 is a flow-matching model. Generation starts from pure noise, and at each of the 50 steps the transformer predicts a velocity. This is the direction from noise toward a clean image. The scheduler takes one step along it.

\n

At every step you can also read off the model's current guess of the final image. Decoding that guess at a few steps shows how the image takes shape:

\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n
\"step01\"\"step05\"\"step10\"\"step15\"\"step25\"\"step50\"
Step 1Step 5Step 10Step 15Step 25Step 50
\n

The fox, its pose, and the light are already in place by step 10, while the noise level is still 0.97. The snow-laden branches and the texture of the fur only appear in the last few steps. FLUX.3's scheduler uses a large time shift (6.93), which keeps the noise level high for most of the run: 44 of the 50 steps happen above σ = 0.5. We built this figure by adding one small block to the pipeline, which brings us to Modular Diffusers.

\n
\n

If you are new to flow matching, we have a beginner friendly tutorial and a video for you to understand how it works.

\n
\n

Modular Diffusers

\n

FLUX.3 ships as a Modular Diffusers pipeline.

\n

See the workflows

\n

You can list the workflows and print the exact blocks of each one, without loading any weights.

\n
from diffusers import Flux3AutoBlocks\n\nblocks = Flux3AutoBlocks()\nprint(blocks.available_workflows)\n# ['text2image', 'image_conditioned']\n\nprint(list(blocks.get_workflow("text2image").sub_blocks))\n# ['text_encoder', 'denoise.input', 'denoise.prepare_latents', 'denoise.set_timesteps',\n#  'denoise.prepare_rope_inputs', 'denoise.denoise', 'denoise.after_denoise', 'decode']\n
\n

get_workflow() resolves the auto branches and gives you the plain sequence of steps that will run. print(blocks.get_workflow("image_conditioned").doc) prints the components, inputs, defaults, and outputs of that workflow.

\n

Encode once, generate many

\n

Pop the text encoder out, encode the prompt once, free it, and generate from the cached embeddings.

\n
import torch\nfrom diffusers import ModularPipeline\n\nmodel_id = "black-forest-labs/flux-3-image-klein-base"\n\nblocks = ModularPipeline.from_pretrained(model_id).blocks\ntext_blocks = blocks.sub_blocks.pop("text_encoder")  # `blocks` now starts at the VAE encoder\n\ntext_pipe = text_blocks.init_pipeline(model_id)\ntext_pipe.load_components(dtype=torch.bfloat16)\ntext_pipe.to("cuda")\n\nstate = text_pipe(prompt=prompt, negative_prompt="")  # no guider here, so ask for the negative embeddings\n\nprompt_embeds = state.get("prompt_embeds")\nnegative_prompt_embeds = state.get("negative_prompt_embeds")\n\ndel text_pipe\ntorch.cuda.empty_cache()\n\nimage_pipe = blocks.init_pipeline(model_id)  # no text encoder to load\nimage_pipe.load_components(dtype={"vae": torch.float32, "default": torch.bfloat16})\nimage_pipe.to("cuda")\n\nfor seed in range(3):\n    image = image_pipe(\n        prompt_embeds=prompt_embeds,\n        negative_prompt_embeds=negative_prompt_embeds,\n        generator=torch.Generator().manual_seed(seed),\n        output="images",\n    )[0]\n    image.save(f"{seed}.png")\n
\n

Add a step inside the loop

\n

The denoising figure above came from one extra block that records the model's guess of the clean image at every step:

\n
from diffusers.modular_pipelines import Flux3AutoBlocks, InputParam, ModularPipelineBlocks, OutputParam\n\n\nclass Flux3PreviewStep(ModularPipelineBlocks):\n    model_name = "flux3"\n\n    @property\n    def inputs(self):\n        return [InputParam("latents"), InputParam("noise_pred")]\n\n    @property\n    def intermediate_outputs(self):\n        return [OutputParam("x0_previews", type_hint=list)]\n\n    @torch.no_grad()\n    def __call__(self, components, block_state, i, t):\n        if i == 0:\n            block_state.x0_previews = []\n        sigma = t / 1000  # timesteps are sigmas scaled to [0, 1000]\n        block_state.x0_previews.append(block_state.latents - sigma * block_state.noise_pred)  # x0 = x_t - sigma * v\n        return components, block_state\n\n\nblocks = Flux3AutoBlocks()\ndenoise_loop = blocks.sub_blocks["denoise"].sub_blocks["text2image"].sub_blocks["denoise"]\ndenoise_loop.sub_blocks.insert("preview", Flux3PreviewStep(), 1)  # between "denoiser" and "after_denoiser"\n\npipe = blocks.init_pipeline(model_id)\npipe.load_components(dtype={"vae": torch.float32, "default": torch.bfloat16})\npipe.to("cuda")\n\nout = pipe(prompt=prompt, generator=torch.Generator().manual_seed(0), output=["x0_previews", "images"])\n
\n

Try it

\n\n

Built something with FLUX.3? Share your LoRAs and pipelines on the Hub. We'd love to see them! 😎

\n", "content_markdown": "[Black Forest Labs](https://huggingface.co/black-forest-labs) FLUX.3 Image Klein is here, and it runs in Diffusers from day one. One model does text-to-image and multi-reference image-to-image, and it adds something new, the **layout control**. You write bounding boxes into the prompt to say where each part of the image should go, or which parts of a reference image to move.\n\nBoth open checkpoints are in the [FLUX.3 Image Klein collection](https://huggingface.co/collections/black-forest-labs/flux-3-image-klein):\n\n| Checkpoint | What it is | Defaults |\n| ---------------------------------------------------------------------------------------------------------------- | --------------------------------------- | ---------------- |\n| [`black-forest-labs/flux-3-image-klein`](https://huggingface.co/black-forest-labs/flux-3-image-klein) | Time step distilled, for fast inference | 16 steps, no CFG |\n| [`black-forest-labs/flux-3-image-klein-base`](https://huggingface.co/black-forest-labs/flux-3-image-klein-base) | Base model | 50 steps, CFG |\n\n[![a red fox in a snowy forest (text-to-image), the same fox re-rendered from that image (image-to-image), a black stippled running silhouette on a chartreuse background, drawn inside its labeled bounding box, and a knitted knight moved from a \\\"from\\\" box to a \\\"to\\\" box next to a knitted dragon (layout editing)](https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/flux3/hero.png)](https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/flux3/hero.png)\n\nYou can head on to the [Hugging Face space](https://huggingface.co/spaces/black-forest-labs/flux-3-image-klein) and generate some images right now!\n\n## TL;DR\n\n- One model for text-to-image, multi-reference image-to-image, and bounding-box layout control.\n- The distilled checkpoint makes a 1024 × 1024 image in about **2 seconds** on an H100.\n- NVFP4 and FP8 weights load with `from_single_file`, for smaller GPUs.\n- LoRA training scripts for both text-to-image and image editing.\n- Ships as a [Modular Diffusers](https://huggingface.co/blog/flux-3#modular-diffusers) pipeline, so you can pop blocks out or add your own.\n\n## Contents\n\n- [Getting started](https://huggingface.co/blog/flux-3#getting-started)\n- [What FLUX.3 can do](https://huggingface.co/blog/flux-3#what-flux3-can-do)\n- [Base model and quantized variants](https://huggingface.co/blog/flux-3#base-model-and-quantized-variants)\n- [LoRA finetuning](https://huggingface.co/blog/flux-3#lora-finetuning)\n- [What it costs to run](https://huggingface.co/blog/flux-3#what-it-costs-to-run)\n- [The architecture](https://huggingface.co/blog/flux-3#the-architecture)\n- [Modular Diffusers](https://huggingface.co/blog/flux-3#modular-diffusers)\n- [Try it](https://huggingface.co/blog/flux-3#try-it)\n\n## Getting started\n\nInstall the latest `diffusers` and `transformers` (for the Qwen3-VL text encoder):\n\n```bash\npip install -U diffusers transformers\n```\n\nLoad the distilled checkpoint and generate an image:\n\n```python\nimport torch\nfrom diffusers import ModularPipeline\n\npipe = ModularPipeline.from_pretrained(\"black-forest-labs/flux-3-image-klein\")\npipe.load_components(dtype=torch.bfloat16)\npipe.to(\"cuda\")\n\npipe(prompt=\"a red fox\", output=\"images\")[0].save(\"fox.png\")\n```\n\n## What FLUX.3 can do\n\n### Text-to-image\n\nFLUX.3 responds best to a full scene description with the subject, the setting, the light, the composition, and the style. The [official prompting guide](https://docs.bfl.ai/guides/prompting_summary) from Black Forest Labs has more tips. The distilled checkpoint generates a `1024 × 1024` image in `16` steps by default. Pass `height`, `width`, or `num_inference_steps` to change them.\n\n```python\nprompt = (\n \"A wide-angle wildlife photograph of a red fox standing alert in a snow-covered forest clearing at golden hour. \"\n \"The fox is centered in the frame, its thick orange winter coat contrasting with the fresh white snow, breath \"\n \"visible in the cold air. Snow-laden pine trees fill the softly blurred background, and low warm sunlight rakes \"\n \"across the scene from the left, casting long blue shadows.\"\n)\n\nfox = pipe(\n prompt=prompt,\n generator=torch.Generator().manual_seed(0),\n output=\"images\"\n)[0]\nfox.save(\"fox.png\")\n```\n\n| [![A red fox standing alert in a snow-covered forest clearing, lit by low golden sunlight from the left](https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/flux3/t2i.png)](https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/flux3/t2i.png) |\n| ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |\n| Text to image |\n\n### Image-to-image\n\nPass a reference image (or more) with the `image` argument. Write the prompt as a caption of the image you *want*, with the change included.\n\n```python\nprompt = (\n \"The same red fox in the same snow-covered forest clearing, now at night under a full moon: cool blue \"\n \"moonlight, deep shadows, stars visible through the pine branches, the fox's eyes catching the light.\"\n)\nnight = pipe(\n prompt=prompt,\n image=fox, # from the previous text to image example\n generator=torch.Generator().manual_seed(0),\n output=\"images\",\n)[0]\nnight.save(\"night.png\")\n```\n\n| [![The fox from the previous image, relit in cold blue light](https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/flux3/i2i.png)](https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/flux3/i2i.png) |\n| --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |\n| Image to image |\n\nWith several references, refer to them in the prompt as ``, ``, ... in the order they are passed to `image`:\n\n```python\nred_fox = pipe(prompt=\"a red fox\", generator=torch.Generator().manual_seed(0), output=\"images\")[0]\nbench = pipe(prompt=\"a wooden bench\", generator=torch.Generator().manual_seed(0), output=\"images\")[0]\n\ncombined = pipe(\n prompt=\"The fox from sitting on the wooden bench from in a snowy setting.\",\n image=[red_fox, bench],\n output=\"images\",\n)[0]\n\ncombined.save(\"combined.png\")\n```\n\n| [![A red fox sitting on dry grass](https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/flux3/i2i_fox.png)](https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/flux3/i2i_fox.png) | [![A slatted wooden bench against a plain grey wall](https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/flux3/i2i_bench.png)](https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/flux3/i2i_bench.png) | [![The same fox sitting on the same bench, with snow on the ground](https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/flux3/combined.png)](https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/flux3/combined.png) |\n| -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |\n| ``: a red fox | ``: a wooden bench | The fox on the bench |\n\n### Layout control\n\nAdd a JSON list of regions after the caption. Each region has an `id`, a `bbox`, and a `desc`, and the caption mentions it as ``.\n\n> Boxes are written as `[y_min, x_min, y_max, x_max]`, **y first**, on a 0–1000 scale. Because the scale is normalized, the same box works at any image size.\n\n#### Placing regions in a new image\n\n```python\nlayout_prompt = (\n \"Minimalist graphic illustration featuring a black silhouette of a person centered against a \"\n \"solid, vibrant chartreuse background . The figure is captured in a dynamic, mid-stride running \"\n \"pose, facing toward the left side of the frame. \"\n '[{\"id\": \"background_1\", \"bbox\": [0, 0, 1000, 1000], \"desc\": \"A flat field of neon yellow-green with a subtle '\n 'paper texture and fine, organic grain.\"}, {\"id\": \"silhouette_1\", \"bbox\": [150, 150, 850, 850], \"desc\": \"A '\n \"black silhouette of a person in motion, made of dense stippled ink. The leading edges are solid; the trailing \"\n 'arm and leg dissolve into coarse square pixels and scattered dots.\"}]'\n)\n\nrunner = pipe(prompt=layout_prompt, generator=torch.Generator().manual_seed(0), output=\"images\")[0]\n```\n\n| [![Diagram: the caption and the two region descriptions on the left, the two boxes on a 0–1000 grid in the middle, and the generated image on the right, where a black stippled runner fills the silhouette box on a chartreuse background](https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/flux3/bbox-layout.png)](https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/flux3/bbox-layout.png) |\n| ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |\n| Each region's `desc` is drawn inside its `bbox` |\n\n#### Moving regions within a reference image\n\nFor edits, each region says where its content comes **from** (`src_bbox` in the reference) and where it goes (`tgt_bbox` in the output). A region with the same two boxes stays where it is.\n\nThis example moves the knitted knight up and to the left, and pins the backdrop, the cliff, the needle, the dragon, and the fire:\n\n```python\nfrom diffusers.utils import load_image\n\nknight_image = load_image(\"knight_orig.png\")\nedit_prompt = (\n \"In , move the miniature grey amigurumi knight figure upwards and to the left along the \"\n \"yarn cliff . Keep the rest of the image completely unchanged, preserving the blurred yarn backdrop \"\n \", the large knitted dragon , and the fiery orange thread . \"\n '[{\"id\": \"knight_1\", \"from\": \"ref_image_0\", \"src_bbox\": [500, 150, 850, 350], \"tgt_bbox\": [194, 55, 544, 255], '\n '\"desc\": \"A miniature amigurumi knight crocheted from thick grey woolen yarn.\"}, '\n '{\"id\": \"backdrop_1\", \"from\": \"ref_image_0\", \"src_bbox\": [0, 0, 1000, 1000], \"tgt_bbox\": [0, 0, 1000, 1000], '\n '\"desc\": \"A soft, out-of-focus backdrop of cream and indigo knitted yarn.\"}, '\n '{\"id\": \"cliff_1\", \"from\": \"ref_image_0\", \"src_bbox\": [300, 0, 1000, 450], \"tgt_bbox\": [300, 0, 1000, 450], '\n '\"desc\": \"A steep cliff of stacked yarn skeins in mustard, terracotta, and indigo.\"}, '\n '{\"id\": \"needle_1\", \"from\": \"ref_image_0\", \"src_bbox\": [420, 250, 550, 450], \"tgt_bbox\": [420, 250, 550, 450], '\n '\"desc\": \"An oversized polished steel tapestry needle.\"}, '\n '{\"id\": \"dragon_1\", \"from\": \"ref_image_0\", \"src_bbox\": [100, 600, 950, 1000], \"tgt_bbox\": [100, 600, 950, 1000], '\n '\"desc\": \"A large, fluffy knitted dragon in terracotta and mustard yarn.\"}, '\n '{\"id\": \"fire_1\", \"from\": \"ref_image_0\", \"src_bbox\": [350, 400, 650, 650], \"tgt_bbox\": [350, 400, 650, 650], '\n '\"desc\": \"Curled puffs of fiery orange angora thread suspended in mid-air.\"}]'\n)\n\nmoved = pipe(prompt=edit_prompt, image=knight_image, generator=torch.Generator().manual_seed(0), output=\"images\")[0]\n```\n\n| [![Left: the reference with the knight's source box. Right: the output with an arrow from the source box to the target box; the pinned regions are outlined in white](https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/flux3/knight.png)](https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/flux3/knight.png) |\n| ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |\n| The knight moves from its `src_bbox` to its `tgt_bbox`, and the rest of the scene stays put |\n\n## Base model and quantized variants\n\n### Base model\n\nEverything above uses the distilled checkpoint. The base checkpoint runs 50 steps with classifier-free guidance, and it's the one to fine-tune (see [LoRA finetuning](https://huggingface.co/blog/flux-3#lora-finetuning)).\n\n```python\npipe = ModularPipeline.from_pretrained(\"black-forest-labs/flux-3-image-klein-base\")\npipe.load_components(dtype={\"vae\": torch.float32, \"default\": torch.bfloat16})\npipe.to(\"cuda\")\n\npipe.guider = pipe.guider.new(guidance_scale=5.0) # optional\nimage = pipe(prompt=prompt, output=\"images\")[0]\n```\n\n### Single-file format checkpoints\n\nBlack Forest Labs also ships the transformer as a single file, in `bf16` and in two quantized formats: `fp8r` (rowwise FP8) and `nvfp4`. Load any of them with `from_single_file`, then load the other components by name so the bf16 transformer isn't downloaded too.\n\n```python\nfrom diffusers import BFLQuantizationConfig, Flux3Transformer2DModel, ModularPipeline\n\nmodel_id = \"black-forest-labs/flux-3-image-klein-base\"\nvariant = \"fp8r\" # or \"nvfp4\"\n\ntransformer = Flux3Transformer2DModel.from_single_file(\n f\"https://huggingface.co/{model_id}/blob/main/variants/{variant}/model.safetensors\",\n config=model_id,\n subfolder=\"transformer\",\n quantization_config=BFLQuantizationConfig(compute_dtype=torch.bfloat16),\n dtype=torch.bfloat16,\n)\n\npipe = ModularPipeline.from_pretrained(model_id)\npipe.load_components(dtype=torch.bfloat16)\npipe.update_components(transformer=transformer)\npipe.to(\"cuda\")\n```\n\nFor the unquantized single file, use `https://huggingface.co/{model_id}/blob/main/model.safetensors` and drop `quantization_config`.\n\n| Variant | Fast kernels on | Other GPUs |\n| ------- | ------------------------------------------------------------------------------ | ----------------------------------------------------------- |\n| `fp8r` | Compute capability 8.9+ (Ada, Hopper, Blackwell) | Weights are dequantized each step: lower memory, no speedup |\n| `nvfp4` | Blackwell, with `flashinfer-python` and `flashinfer-cubin` at the same version | Same as above |\n\n## LoRA finetuning\n\nFLUX.3 works for both text-to-image and image-to-image, which makes it a great base for custom LoRAs. `diffusers` ships DreamBooth LoRA training scripts for both tasks (you can also train with [Ostris' AI Toolkit](https://github.com/ostris/ai-toolkit)).\n\nHere is a text-to-image example that learns a tarot-card style from the [`multimodalart/1920-raider-waite-tarot-public-domain`](https://huggingface.co/datasets/multimodalart/1920-raider-waite-tarot-public-domain) dataset, triggered with `trtcrd`:\n\n```bash\naccelerate launch train_dreambooth_lora_flux3_klein.py \\\n --pretrained_model_name_or_path=\"black-forest-labs/flux-3-image-klein-base\" \\\n --dataset_name=\"multimodalart/1920-raider-waite-tarot-public-domain\" \\\n --caption_column=\"caption\" \\\n --instance_prompt=\"trtcrd\" \\\n --output_dir=\"trained-flux3-tarot-lora\" \\\n --mixed_precision=\"bf16\" \\\n --resolution=1024 \\\n --use_aspect_ratio_buckets \\\n --train_batch_size=1 \\\n --rank=16 \\\n --lora_alpha=16 \\\n --learning_rate=1e-4 \\\n --lr_scheduler=\"constant\" \\\n --max_train_steps=1500 \\\n --validation_prompt=\"a trtcrd of a cat, tarot style\" \\\n --validation_epochs=25 \\\n --seed=0 \\\n --push_to_hub\n```\n\n> To reduce memory usage, add a few optimization flags:\n>\n> ```bash\n> --gradient_checkpointing \\\n> --cache_latents \\\n> --use_8bit_adam\n> ```\n>\n> `--gradient_checkpointing` recomputes activations in the backward pass (lower VRAM, slightly slower), `--cache_latents` pre-encodes the images and frees the VAE, and `--use_8bit_adam` keeps the optimizer states in 8-bit.\n\nA couple of knobs are worth experimenting with. The grid below trains the same tarot style with different LoRA ranks and optimizers; the adapters are on the Hub at [`diffusers-modular/flux3-tarot-lora`](https://huggingface.co/diffusers-modular/flux3-tarot-lora). More training features and configurations are documented in the [DreamBooth FLUX.3 README](https://github.com/huggingface/diffusers/blob/main/examples/dreambooth/README_flux3_klein.md).\n\n[![FLUX.3 tarot-style LoRA across ranks and optimizers: the base model with no LoRA, then rank 16, rank 8 and rank 32 (AdamW, lr 1e-4) and a Prodigy run, on two held-out prompts](https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/flux3/tarot-lora-sweep.png)](https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/flux3/tarot-lora-sweep.png)\n\nFor instruction-based editing, the image-to-image script (`train_dreambooth_lora_flux3_klein_img2img.py`) trains on (reference image, target image, instruction) triplets, where the reference is addressed in the prompt as ``.\n\nSee the [DreamBooth FLUX.3 README](https://github.com/huggingface/diffusers/blob/main/examples/dreambooth/README_flux3_klein.md) for all options and the full image-to-image example.\n\n## What it costs to run\n\nIn this section, we will talk about the memory consumption and the speed of generation of the models.\n\n### Memory\n\n| name | peak allocated (after load) GiBs | peak allocated (after inference) GiBs |\n| --------------- | -------------------------------- | ------------------------------------- |\n| nvfp4 (RTX6000) | 13.4 | 15.3 |\n| fp8r (H100) | 14.35 | 16.27 |\n| bf16 (H100) | 18.80 | 20.72 |\n\nIf you have less memory:\n\n- Load a [quantized transformer](https://huggingface.co/blog/flux-3#base-model-and-quantized-variants).\n- Run the text encoder separately and free it. It only runs once per image, before denoising (see [Encode once, generate many](https://huggingface.co/blog/flux-3#encode-once-generate-many)).\n- Turn on automatic CPU offloading, which moves each component to the GPU only while it runs (we didn't benchmark this)\n\n```python\nfrom diffusers import ComponentsManager, ModularPipeline\n\nmanager = ComponentsManager()\nmanager.enable_auto_cpu_offload(device=\"cuda\")\n\npipe = ModularPipeline.from_pretrained(\"black-forest-labs/flux-3-image-klein\", components_manager=manager)\npipe.load_components(dtype={\"vae\": torch.float32, \"default\": torch.bfloat16})\n# no pipe.to(\"cuda\"): the manager handles device placement\n```\n\n### Speed\n\nEnd-to-end timings for the distilled checkpoint (16 steps), averaged over three runs after a warmup:\n\n| Workflow | Output size | Time | Peak memory |\n| ----------------------------------------------------- | ----------- | ------ | ----------- |\n| Text-to-image | 1024 × 1024 | 1.93 s | 23.8 GiB |\n| Image-to-image, one 1024 × 1024 reference | 1024 × 1024 | 3.87 s | 23.8 GiB |\n| Layout editing (the knight), one 1360 × 768 reference | 1360 × 768 | 4.10 s | 23.8 GiB |\n\n> Single NVIDIA H100 80GB, no offloading, default attention backend.\n\nA reference image roughly doubles the time. Reference tokens join the same sequence as the text and the image being generated, so a 1024 × 1024 reference adds 4,096 tokens to every transformer call\n\n| [![Stacked bars of tokens per transformer call: text-to-image 4,256 tokens; image-to-image 8,272; layout text-to-image 4,336; layout image-to-image 8,800](https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/flux3/tokens-per-workflow.png)](https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/flux3/tokens-per-workflow.png) |\n| -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |\n| Stacked bars of tokens per transformer call: text-to-image 4,256 tokens; image-to-image 8,272; layout text-to-image 4,336; layout image-to-image 8,800 |\n\n## The architecture\n\nLet's open the model up, one piece at a time. The model takes three kinds of tokens:\n\n1. **text**\n2. the **noisy image** being generated\n3. and optional **reference images**\n\n### From pixels to tokens\n\nFLUX.3 is a latent-space flow model for image generation. This means that the denoiser takes latents as inputs and produces latents as outputs. We use a VAE to decode the latents back to pixels. FLUX.3 uses the same VAE architecture as Flux.2.\n\nThe FLUX.2 VAE compresses an image 8× in each direction into 32 channels, and then every 2 × 2 patch of that latent is packed into a single token with 128 channels.\n\nSo **one token covers a 16 × 16 pixel patch**, and a 1024 × 1024 image becomes a 64 × 64 grid of 4,096 tokens. The noisy image starts as random noise of exactly that shape. A reference image goes through the same VAE encoder and packing, and keeps its own size.\n\n### From prompt to tokens\n\nThe text encoder is Qwen3-VL, a vision-language model. FLUX.3 doesn't just take its last layer. It reads the hidden states of **eight** of its 36 layers (layers 4, 8, …, 32) and stacks them, so each text token becomes a 8 × 2560 = 20,480-dimensional vector before a linear layer projects it to the transformer's width of 3072.\n\nThis gives the transformer features from several depths of the language model, not just its final layer. The prompt is padded to a multiple of 80 tokens, and the transformer attends to the padding too, which is why the pipeline takes one prompt per call.\n\n### Two stages\n\nThe transformer processes the tokens in two stages:\n\n1. **Stage 1: per-mode blocks.** Each stream goes through 5 blocks with its own weights. Text has its own blocks, the noisy image has its own, and the reference images have their own.\n2. **Stage 2: joint blocks.** The streams are concatenated into one sequence, `[text | image | reference]`, which goes through 28 blocks that share one set of weights.\n\nThe two stages differ in *who can attend to whom*.\n\nIn stage 1, each stream only attends to itself, so each kind of token gets refined on its own terms first. In stage 2, everything attends to everything, where the image learns from the text and from the references. For comparison, FLUX.2's double-stream blocks also give text and image separate weights, but let them attend to each other from the start. FLUX.3 keeps the streams fully apart first, then merges them.\n\nAt the end, the text and reference tokens are simply dropped. Only the image tokens go through the output layers, which predict how to move them one step toward a clean image.\n\n### Inside one block\n\nEvery block, in either stage, has the same shape. Attention and the MLP run **in parallel** on the same normalized input, and their sum is added back to the residual stream.\n\nEach block starts with a LayerNorm that has no learned scale or shift of its own. The scale, shift, and gate come from a timestep embedding instead (this is called *modulation*). Inside the attention, the queries and keys are RMS-normalized and get rotary position embeddings (RoPE), which we look at below.\n\nIn a joint block, all tokens share the same weights, but text, image, and reference tokens each get their own scale, shift, and gate.\n\n```python\ndef block(x, shift, scale, gate): # shift, scale, gate come from the token's timestep embedding\n h = layer_norm(x) # no learned scale or shift\n h = (1 + scale) * h + shift\n q, k, v = to_q(h), to_k(h), to_v(h)\n q, k = rope(rms_norm(q)), rope(rms_norm(k))\n out = attention(q, k, v) + mlp(h) # parallel, MLP is SwiGLU\n return x + gate * out\n```\n\n### Every token gets its own timestep\n\nIn FLUX.1 and FLUX.2, one timestep embedding modulates every token. In FLUX.3, the modulation depends on the kind of token.\n\nThe noisy image tokens are modulated by the current timestep `t`. The text and reference tokens are always modulated by timestep **0**, which means \"clean, nothing to denoise here\". So even a reference that looks almost the same as the output is marked as given context, not as something to generate.\n\nThe distilled checkpoint is a *flow map*. It is also told the timestep each step jumps to. That second timestep gets its own embedding, which is added to every token's modulation, so each of its 16 steps can make a larger jump.\n\n### Positions in four dimensions\n\nAttention on its own doesn't know where a token is (this phenomenon is called permutation equivariance). RoPE fixes that by rotating each query and key by an amount that depends on the token's position. FLUX.3 uses four position axes, `(t, h, w, l)` (timestep, height, width, length of sequence), and gives each one a quarter (32 dimensions) of every 128-dimensional attention head:\n\n| Tokens | t: timestep | h: height | w: width | l: length of sequence |\n| --------------------------- | ----------- | --------- | -------- | --------------------- |\n| text token *i* | 0 | 0 | 0 | *i* |\n| image token at (row, col) | 0 | row / 2 | col / 2 | 0 |\n| reference *k* at (row, col) | 0 | row / 2 | col / 2 | −(*k* + 1) |\n\nText tokens line up along `l`, like words in a sentence. The image and the references are 2D grids in `(h, w)`, separated from each other along `l`. Because a reference uses the same `(h, w)` coordinates as the output, \"the same spot in the reference\" has a precise meaning to the model, which is what the `src_bbox` / `tgt_bbox` editing relies on.\n\n### The denoising loop\n\nFLUX.3 is a flow-matching model. Generation starts from pure noise, and at each of the 50 steps the transformer predicts a *velocity*. This is the direction from noise toward a clean image. The scheduler takes one step along it.\n\nAt every step you can also read off the model's current guess of the final image. Decoding that guess at a few steps shows how the image takes shape:\n\n| [![step01](https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/flux3/traj_step01.png)](https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/flux3/traj_step01.png) | [![step05](https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/flux3/traj_step05.png)](https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/flux3/traj_step05.png) | [![step10](https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/flux3/traj_step10.png)](https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/flux3/traj_step10.png) | [![step15](https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/flux3/traj_step15.png)](https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/flux3/traj_step15.png) | [![step25](https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/flux3/traj_step25.png)](https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/flux3/traj_step25.png) | [![step50](https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/flux3/traj_step50.png)](https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/flux3/traj_step50.png) |\n| ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |\n| Step 1 | Step 5 | Step 10 | Step 15 | Step 25 | Step 50 |\n\nThe fox, its pose, and the light are already in place by step 10, while the noise level is still 0.97. The snow-laden branches and the texture of the fur only appear in the last few steps. FLUX.3's scheduler uses a large time shift (6.93), which keeps the noise level high for most of the run: 44 of the 50 steps happen above σ = 0.5. We built this figure by adding one small block to the pipeline, which brings us to Modular Diffusers.\n\n> If you are new to flow matching, we have a beginner [friendly tutorial](https://huggingface.co/spaces/ariG23498/flow-matching) and a [video](https://youtu.be/hXQZ6lyRCsI) for you to understand how it works.\n\n## Modular Diffusers\n\nFLUX.3 ships as a [Modular Diffusers](https://huggingface.co/docs/diffusers/main/en/modular_diffusers/overview) pipeline.\n\n### See the workflows\n\nYou can list the workflows and print the exact blocks of each one, without loading any weights.\n\n```python\nfrom diffusers import Flux3AutoBlocks\n\nblocks = Flux3AutoBlocks()\nprint(blocks.available_workflows)\n# ['text2image', 'image_conditioned']\n\nprint(list(blocks.get_workflow(\"text2image\").sub_blocks))\n# ['text_encoder', 'denoise.input', 'denoise.prepare_latents', 'denoise.set_timesteps',\n# 'denoise.prepare_rope_inputs', 'denoise.denoise', 'denoise.after_denoise', 'decode']\n```\n\n`get_workflow()` resolves the auto branches and gives you the plain sequence of steps that will run. `print(blocks.get_workflow(\"image_conditioned\").doc)` prints the components, inputs, defaults, and outputs of that workflow.\n\n### Encode once, generate many\n\nPop the text encoder out, encode the prompt once, free it, and generate from the cached embeddings.\n\n```python\nimport torch\nfrom diffusers import ModularPipeline\n\nmodel_id = \"black-forest-labs/flux-3-image-klein-base\"\n\nblocks = ModularPipeline.from_pretrained(model_id).blocks\ntext_blocks = blocks.sub_blocks.pop(\"text_encoder\") # `blocks` now starts at the VAE encoder\n\ntext_pipe = text_blocks.init_pipeline(model_id)\ntext_pipe.load_components(dtype=torch.bfloat16)\ntext_pipe.to(\"cuda\")\n\nstate = text_pipe(prompt=prompt, negative_prompt=\"\") # no guider here, so ask for the negative embeddings\n\nprompt_embeds = state.get(\"prompt_embeds\")\nnegative_prompt_embeds = state.get(\"negative_prompt_embeds\")\n\ndel text_pipe\ntorch.cuda.empty_cache()\n\nimage_pipe = blocks.init_pipeline(model_id) # no text encoder to load\nimage_pipe.load_components(dtype={\"vae\": torch.float32, \"default\": torch.bfloat16})\nimage_pipe.to(\"cuda\")\n\nfor seed in range(3):\n image = image_pipe(\n prompt_embeds=prompt_embeds,\n negative_prompt_embeds=negative_prompt_embeds,\n generator=torch.Generator().manual_seed(seed),\n output=\"images\",\n )[0]\n image.save(f\"{seed}.png\")\n```\n\n### Add a step inside the loop\n\nThe denoising figure above came from one extra block that records the model's guess of the clean image at every step:\n\n```python\nfrom diffusers.modular_pipelines import Flux3AutoBlocks, InputParam, ModularPipelineBlocks, OutputParam\n\n\nclass Flux3PreviewStep(ModularPipelineBlocks):\n model_name = \"flux3\"\n\n @property\n def inputs(self):\n return [InputParam(\"latents\"), InputParam(\"noise_pred\")]\n\n @property\n def intermediate_outputs(self):\n return [OutputParam(\"x0_previews\", type_hint=list)]\n\n @torch.no_grad()\n def __call__(self, components, block_state, i, t):\n if i == 0:\n block_state.x0_previews = []\n sigma = t / 1000 # timesteps are sigmas scaled to [0, 1000]\n block_state.x0_previews.append(block_state.latents - sigma * block_state.noise_pred) # x0 = x_t - sigma * v\n return components, block_state\n\n\nblocks = Flux3AutoBlocks()\ndenoise_loop = blocks.sub_blocks[\"denoise\"].sub_blocks[\"text2image\"].sub_blocks[\"denoise\"]\ndenoise_loop.sub_blocks.insert(\"preview\", Flux3PreviewStep(), 1) # between \"denoiser\" and \"after_denoiser\"\n\npipe = blocks.init_pipeline(model_id)\npipe.load_components(dtype={\"vae\": torch.float32, \"default\": torch.bfloat16})\npipe.to(\"cuda\")\n\nout = pipe(prompt=prompt, generator=torch.Generator().manual_seed(0), output=[\"x0_previews\", \"images\"])\n```\n\n## Try it\n\n- **Models:** [FLUX.3 Image Klein collection](https://huggingface.co/collections/black-forest-labs/flux-3-image-klein)\n- **Docs:** [FLUX.3 in Diffusers](https://huggingface.co/docs/diffusers/main/en/api/pipelines/flux3)\n- **Fine-tune:** [DreamBooth LoRA scripts](https://github.com/huggingface/diffusers/blob/main/examples/dreambooth/README_flux3_klein.md)\n- **Prompting:** [Black Forest Labs prompting guide](https://docs.bfl.ai/guides/prompting_summary)\n\nBuilt something with FLUX.3? Share your LoRAs and pipelines on the Hub. We'd love to see them! 😎\n", "dateCreated": "2026-10-15T18:36:21.248140220+00:00", "date_created": "2026-10-15T00:00:00+00:00", "date_modified": "2026-10-15T00:00:00+00:00", "date_published": "2026-10-15T00:00:00+00:00", "description": "Black Forest Labs FLUX.3 Image Klein is here, and it runs in Diffusers from day one. One model does text-to-image and multi-reference image-to-image, and it adds something new, the layout control. You write bounding boxes into the prompt…", "external_url": "https://huggingface.co/blog/flux-3", "git": { "commit": "ebb0941c5c06a2ab2377342d607535262a3e969e", "path": "items/huggingface-co/2026/10/2026-10-15-welcome-flux-3-image-klein-in-diffusers.md", "permalink": "https://github.com/aymericbeaumet/aggr-instance/blob/ebb0941c5c06a2ab2377342d607535262a3e969e/items/huggingface-co/2026/10/2026-10-15-welcome-flux-3-image-klein-in-diffusers.md" }, "id": "urn:aggr:item:f09102aea4078245f878bde7bf2dd6161c09c0d2", "isBasedOn": { "@id": "https://huggingface.co/blog/flux-3" }, "mainEntity": { "@id": "https://huggingface.co/blog/flux-3", "@type": "CreativeWork", "archivedAt": { "@id": "https://aggr.aymericbeaumet.com/items/huggingface-co/2026-10-15-welcome-flux-3-image-klein-in-diffusers/#webpage" }, "dateModified": "2026-10-15T00:00:00+00:00", "datePublished": "2026-10-15T00:00:00+00:00", "headline": "Welcome FLUX.3 Image Klein in Diffusers", "timeRequired": "PT15M", "url": "https://huggingface.co/blog/flux-3", "wordCount": 3291 }, "name": "Archived snapshot: Welcome FLUX.3 Image Klein in Diffusers", "preview": null, "reading_minutes": 15, "replicated_at": null, "source": "huggingface-co", "source_name": "Hugging Face - Blog", "summary": null, "tags": [], "temporalCoverage": "2026-10-15T18:36:21.248140220+00:00", "title": "Welcome FLUX.3 Image Klein in Diffusers", "url": "https://aggr.aymericbeaumet.com/items/huggingface-co/2026-10-15-welcome-flux-3-image-klein-in-diffusers/", "word_count": 3291 }