{
  "@context": "https://schema.org",
  "@id": "https://aggr.aymericbeaumet.com/items/huggingface-co/2026-03-05-introducing-modular-diffusers-composable-building-blocks/#webpage",
  "@type": [
    "WebPage",
    "ArchiveComponent"
  ],
  "articleBody": "Modular Diffusers introduces a new way to build diffusion pipelines by composing reusable blocks. Instead of writing entire pipelines from scratch, you can mix and match blocks to create workflows tailored to your needs! This complements the existing `DiffusionPipeline` class with a more flexible, composable alternative.\n\nIn this post, we'll walk through how Modular Diffusers works — from the familiar API to run a modular pipeline, to building fully custom blocks and composing them into your own workflow. We'll also show how it integrates with Mellon, a node-based visual workflow interface that you can use to wire Modular Diffusers blocks together.\n\n**Table of contents**\n\n- [Quickstart](https://huggingface.co/blog/modular-diffusers#quickstart)\n- [Custom Blocks](https://huggingface.co/blog/modular-diffusers#custom-blocks)\n- [Modular Repositories](https://huggingface.co/blog/modular-diffusers#modular-repositories)\n- [Community Pipelines](https://huggingface.co/blog/modular-diffusers#community-pipelines)\n- [Integration with Mellon](https://huggingface.co/blog/modular-diffusers#integration-with-mellon)\n\n## Quickstart\n\nHere is a simple example of how to run inference with `FLUX.2 Klein 4B` using pre-built blocks:\n\n```python\nimport torch\nfrom diffusers import ModularPipeline\n\n# Create a modular pipeline - this only defines the workflow, model weights have not been loaded yet\npipe = ModularPipeline.from_pretrained(\n    \"black-forest-labs/FLUX.2-klein-4B\"\n)\n#  Now load the model weights — configure dtype, quantization, etc in this step\npipe.load_components(torch_dtype=torch.bfloat16)\npipe.to(\"cuda\")\n\n# Generate an image - API remains the same as DiffusionPipeline\nimage = pipe(\n    prompt=\"a serene landscape at sunset\",\n    num_inference_steps=4,\n).images[0]\n\nimage.save(\"output.png\")\n```\n\nYou get the same results as with a standard `DiffusionPipeline`, but the pipeline is very different under the hood: it's composed of flexible *blocks* — text encoding, image encoding, denoising, and decoding — that you can inspect directly:\n\n```python\nprint(pipe.blocks)\n```\n\n```bash\nFlux2KleinAutoBlocks(\n  ...\n  Sub-Blocks:\n    [0] text_encoder (Flux2KleinTextEncoderStep)\n    [1] vae_encoder (Flux2KleinAutoVaeEncoderStep)\n    [2] denoise (Flux2KleinCoreDenoiseStep)\n    [3] decode (Flux2DecodeStep)\n)\n```\n\nEach block is self-contained with its own inputs and outputs. You can run any block independently as its own pipeline, or add, remove, and swap blocks freely — they dynamically recompose to work with whatever blocks remain. Use `.init_pipeline()` to convert blocks into a runnable pipeline, and `.load_components()` to load the model weights.\n\n```python\n# get a copy of the blocks\nblocks = pipe.blocks\n\n# pop out the text_encoder block\ntext_blocks = blocks.sub_blocks.pop(\"text_encoder\")\n\n# run it as its own pipeline\ntext_pipe = text_blocks.init_pipeline(\"black-forest-labs/FLUX.2-klein-4B\")\n\n# load the text_encoder, or reuse already loaded components: text_pipe.update_components(text_encoder=pipe.text_encoder)\ntext_pipe.load_components(torch_dtype=torch.bfloat16)\ntext_pipe.to(\"cuda\")\nprompt_embeds = text_pipe(prompt=\"a serene landscape at sunset\").prompt_embeds\n\n# create a new pipeline from the remaining blocks\n# it now accepts prompt_embeds directly instead of prompt\nremaining_pipe = blocks.init_pipeline(\"black-forest-labs/FLUX.2-klein-4B\")\nremaining_pipe.load_components(torch_dtype=torch.bfloat16)\nremaining_pipe.to(\"cuda\")\nimage = remaining_pipe(prompt_embeds=prompt_embeds, num_inference_steps=4).images[0]\n```\n\nFor more on block types, composition patterns, lazy loading, and memory management with `ComponentsManager`, check out the [Modular Diffusers documentation](https://huggingface.co/docs/diffusers/en/modular_diffusers/overview).\n\n## Custom Blocks\n\nModular Diffusers really shines when creating your own blocks. A custom block is a Python class that defines its components, inputs, outputs, and computation logic — and once defined, you can plug it into any workflow.\n\n### Writing a Custom Block\n\nHere's an example block that extracts depth maps from images using [Depth Anything V2](https://huggingface.co/depth-anything/Depth-Anything-V2-Large).\n\n```python\nclass DepthProcessorBlock(ModularPipelineBlocks):\n    @property\n    def expected_components(self):\n        return [\n            ComponentSpec(\"depth_processor\", DepthPreprocessor,\n                          pretrained_model_name_or_path=\"depth-anything/Depth-Anything-V2-Large-hf\")\n        ]\n\n    @property\n    def inputs(self):\n        return [\n            InputParam(\"image\", required=True,\n                       description=\"Image(s) to extract depth maps from\"),\n        ]\n\n    @property\n    def intermediate_outputs(self):\n        return [\n            OutputParam(\"control_image\", type_hint=torch.Tensor,\n                        description=\"Depth map(s) of input image(s)\"),\n        ]\n\n    @torch.no_grad()\n    def __call__(self, components, state):\n        block_state = self.get_block_state(state)\n        depth_map = components.depth_processor(block_state.image)\n        block_state.control_image = depth_map.to(block_state.device)\n        self.set_block_state(state, block_state)\n        return components, state\n```\n\n- `expected_components` defines what models the block needs — in this case, a depth estimation model. The `pretrained_model_name_or_path` parameter sets a default Hub repo to load from, so `load_components` automatically fetches the depth model unless you override it in `modular_model_index.json`.\n- `inputs` and `intermediate_outputs` define what goes in and comes out.\n- `__call__` is where the computation logic lives.\n\n### Composing Blocks into Workflows\n\nLet's use this block with Qwen's ControlNet workflow. Extract the ControlNet workflow and insert the depth block at the beginning:\n\n```python\n# Create Qwen Image pipeline\npipe = ModularPipeline.from_pretrained(\"Qwen/Qwen-Image\")\n\nprint(pipe.blocks.available_workflows)\n#       Supported workflows:\n#        - `text2image`: requires `prompt`\n#        - `image2image`: requires `prompt`, `image`\n#        - `inpainting`: requires `prompt`, `mask_image`, `image`\n#        - `controlnet_text2image`: requires `prompt`, `control_image`\n#        - `controlnet_image2image`: requires `prompt`, `image`, `control_image`\n\n# Extract the ControlNet workflow — it expects a control_image input\nblocks = pipe.blocks.get_workflow(\"controlnet_text2image\")\n# Show the blocks this workflow uses\nprint(blocks)\n\n# Insert depth block at the beginning — its output (control_image)\n# automatically flows to the ControlNet block that needs it\nblocks.sub_blocks.insert(\"depth\", DepthProcessorBlock(), 0)\n\n# You can inspect any block's inputs and outputs with print(blocks.doc)\nblocks.sub_blocks['depth'].doc\n```\n\nBlocks in a sequence share data automatically: the depth block's `control_image` output flows to downstream blocks that need it, and its `image` input becomes a pipeline input since no earlier block provides it.\n\n![blocks_composed](https://huggingface.co/datasets/diffusers/modular-diffusers-blog/resolve/main/blocks_composed.png)\n\n```python\nfrom diffusers import ComponentsManager, AutoModel\nfrom diffusers.utils import load_image\n\n# ComponentsManager handles memory across multiple pipelines —\n# it automatically offloads models to CPU when not in use\nmanager = ComponentsManager()\n\npipeline = blocks.init_pipeline(\"Qwen/Qwen-Image\", components_manager=manager)\npipeline.load_components(torch_dtype=torch.bfloat16)\n\n# The depth model loads automatically from the default path we set in expected_components —\n# no need to load it manually even though it's not part of the Qwen repo.\n# But controlnet is not included by default, so we do need to load it from a different repo\ncontrolnet = AutoModel.from_pretrained(\"InstantX/Qwen-Image-ControlNet-Union\", torch_dtype=torch.bfloat16)\npipeline.update_components(controlnet=controlnet)\n\n# pipeline now takes image as input\nimage = load_image(\"https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/diffusers/astronaut.jpg\")\noutput = pipeline(\n    prompt=\"an astronaut hatching from an egg, detailed, fantasy, Pixar, Disney\",\n    image=image,\n).images[0]\n```\n\n### Sharing Custom Blocks on the Hub\n\nYou can publish your custom block to the Hub so anyone can load it with `trust_remote_code=True`. We've created a [template](https://huggingface.co/diffusers/custom-block-template) to get you started — check out the [Building Custom Blocks guide](https://huggingface.co/docs/diffusers/main/en/modular_diffusers/custom_blocks#quick-start-with-template) for the full walkthrough.\n\n```python\npipeline.save_pretrained(local_dir, repo_id=\"your-username/your-block-name\", push_to_hub=True)\n```\n\nThe `DepthProcessorBlock` from this post is published at [diffusers/depth-processor-custom-block](https://huggingface.co/diffusers/depth-processor-custom-block) — you can load and use it directly:\n\n```python\nfrom diffusers import ModularPipelineBlocks\n\ndepth_block = ModularPipelineBlocks.from_pretrained(\n    \"diffusers/depth-processor-custom-block\", trust_remote_code=True\n)\n```\n\nWe've published a collection of ready-to-use custom blocks [here](https://huggingface.co/collections/diffusers/modular-diffusers-custom-blocks).\n\n## Modular Repositories\n\n`ModularPipeline.from_pretrained` works with any existing Diffusers repo out of the box, but Modular Diffusers also introduces a new kind of repo: the Modular Repository.\n\nA modular repository is able to reference components from their original model repos. For example, [diffusers/flux2-bnb-4bit-modular](https://huggingface.co/diffusers/flux2-bnb-4bit-modular) contains a quantized transformer and loads the remaining components from the original repo.\n\n```json\n// diffusers/flux2-bnb-4bit-modular/modular_model_index.json\n{\n    \"transformer\": [\n        \"diffusers\", \n        \"Flux2Transformer2DModel\", \n        {\n            \"pretrained_model_name_or_path\": \"diffusers/flux2-bnb-4bit-modular\",\n            \"subfolder\": \"transformer\",\n            \"type_hint\": [\"diffusers\", \"Flux2Transformer2DModel\"]\n        }\n    ],\n    \"vae\": [\n        \"diffusers\", \n        \"AutoencoderKLFlux2\", \n        {\n            \"pretrained_model_name_or_path\": \"black-forest-labs/FLUX.2-dev\",\n            \"subfolder\": \"vae\",\n            \"type_hint\": [\"diffusers\", \"AutoencoderKLFlux2\"]\n        }\n    ],\n    ...\n}\n```\n\nModular repositories can also host custom pipeline blocks as Python code and visual UI configurations for tools like [Mellon](https://huggingface.co/docs/diffusers/main/en/modular_diffusers/mellon) — all in one place.\n\n## Community Pipelines\n\nThe community has already started building complete pipelines with Modular Diffusers and publishing them on the Hub, with model weights and ready-to-run code.\n\n- [**Krea Realtime Video**](https://huggingface.co/krea/krea-realtime-video) — A 14B parameter real-time video generation model distilled from Wan 2.1, achieving 11fps on a single B200 GPU. It supports text-to-video, video-to-video, and streaming video-to-video — all built as modular blocks. Users can modify prompts mid-generation, restyle videos on-the-fly, and see first frames within 1 second.\n\n```python\nimport torch\nfrom diffusers import ModularPipeline\n\npipe = ModularPipeline.from_pretrained(\"krea/krea-realtime-video\", trust_remote_code=True)\npipe.load_components(\n    trust_remote_code=True, \n    device_map=\"cuda\",\n    torch_dtype={\"default\": torch.bfloat16, \"vae\": torch.float16}\n)\n```\n\n- [**Waypoint-1**](https://huggingface.co/Overworld/Waypoint-1-Small) — A 2.3B parameter real-time diffusion world model from [Overworld](https://over.world). It autoregressively generates interactive worlds from control inputs and text prompts — you can explore and interact with generated environments in real time on consumer hardware.\n\nTeams can build novel architectures, package them as blocks, and publish the entire pipeline on the Hub for anyone to use with `ModularPipeline.from_pretrained`.\n\nCheck out the full [collection of community pipelines](https://huggingface.co/collections/diffusers/modular-pipelines) for more.\n\n## Integration with Mellon\n\n> 💡 Mellon is in early development and not ready for production use yet. Consider this a sneak peek of how the integration works!\n\n[Mellon](https://github.com/cubiq/Mellon) is a node graph interface integrated with Modular Diffusers. If you're familiar with node-based tools like ComfyUI, you'll feel right at home — but there are some key differences:\n\n- **Dynamic nodes** — Instead of dozens of model-specific nodes, we have a small set of nodes that automatically adapt their interface based on the model you select. Learn them once, use them with any model.\n- **Single-node workflows** — Thanks to Modular Diffusers' composable block system, you can collapse an entire pipeline into a single node. Run multiple workflows on the same canvas without the clutter.\n- **Hub integration out of the box** — Custom blocks published to the Hugging Face Hub work instantly in Mellon. We provide a utility function to automatically generate the node interface from your block definition — no UI code required.\n\nThis integration is possible because every block exposes the same properties (`inputs`, `intermediate_outputs`, `expected_components`). This consistent API means Mellon can automatically generate a node's UI from any block definition and compose blocks into higher-level nodes.\n\nFor example, [diffusers/FLUX.2-klein-4B-modular](https://huggingface.co/diffusers/FLUX.2-klein-4B-modular) contains a pipeline definition, component references, and a `mellon_pipeline_config.json` — all in one repo. Load it in Python with `ModularPipeline.from_pretrained(\"diffusers/FLUX.2-klein-4B-modular\")` or in Mellon to create either a single-node or multi-node workflow.\n\nHere's a quick example. We add a Gemini prompt expansion node — hosted as a modular repo at [diffusers/gemini-prompt-expander-mellon](https://huggingface.co/diffusers/gemini-prompt-expander-mellon) — to an existing text-to-image workflow:\n\n1. Drag in a **Dynamic Block** node and enter the `repo_id` (i.e. `diffusers/gemini-prompt-expander-mellon`)\n2. Click **LOAD CUSTOM BLOCK** — the node automatically grows a textbox for your prompt input and an output socket named \"prompt\", all configured from the repo\n3. Type a short prompt, connect the output to the **Encode Prompt** node, and run\n\nGemini expands your short prompt into a detailed description before generating the image. No code, no configuration — just a Hub repo id.\n\nYour browser does not support the video tag.\n\nThis is just one example. For a detailed walkthrough, check out the [Mellon x Modular Diffusers guide](https://www.notion.so/Mellon-x-Modular-Diffusers-2fd1384ebcac819993d8f9ae94c7e866?pvs=21). You can also check out [this video](https://youtu.be/XogoQnkQUO8)!\n\n## Conclusion\n\nModular Diffusers brings the composability and flexibility the community has been asking for, without compromising the features that make Diffusers powerful. It's still early — we want your input to shape what comes next. Give it a try and tell us what works, what doesn't, and what's missing.\n\n## Resources\n\n- [Overview](https://huggingface.co/docs/diffusers/main/en/modular_diffusers/overview) of Modular Diffusers\n- [Mellon](https://github.com/cubiq/Mellon)\n- [Mellon x Modular Diffusers](https://www.notion.so/Mellon-x-Modular-Diffusers-2fd1384ebcac819993d8f9ae94c7e866?pvs=21)\n- [Collection](https://huggingface.co/collections/diffusers/modular-diffusers-custom-blocks) of custom blocks\n- [Collection](https://huggingface.co/collections/diffusers/modular-pipelines) of community pipelines with Modular Diffusers\n\n*Thanks to Chun Te Lee for the thumbnail, and to Poli, Pedro, Lysandre, Linoy, Aritra, and Steven for their thoughtful reviews.*\n",
  "authors": [],
  "captured_at": "2026-10-04T12:50:30.804479567+00:00",
  "category": "ai",
  "content_html": "<p>Modular Diffusers introduces a new way to build diffusion pipelines by composing reusable blocks. Instead of writing entire pipelines from scratch, you can mix and match blocks to create workflows tailored to your needs! This complements the existing <code>DiffusionPipeline</code> class with a more flexible, composable alternative.</p>\n<p>In this post, we'll walk through how Modular Diffusers works — from the familiar API to run a modular pipeline, to building fully custom blocks and composing them into your own workflow. We'll also show how it integrates with Mellon, a node-based visual workflow interface that you can use to wire Modular Diffusers blocks together.</p>\n<p><strong>Table of contents</strong></p>\n<ul>\n<li><a target=\"_blank\" rel=\"noopener noreferrer\" href=\"https://huggingface.co/blog/modular-diffusers#quickstart\">Quickstart</a></li>\n<li><a target=\"_blank\" rel=\"noopener noreferrer\" href=\"https://huggingface.co/blog/modular-diffusers#custom-blocks\">Custom Blocks</a></li>\n<li><a target=\"_blank\" rel=\"noopener noreferrer\" href=\"https://huggingface.co/blog/modular-diffusers#modular-repositories\">Modular Repositories</a></li>\n<li><a target=\"_blank\" rel=\"noopener noreferrer\" href=\"https://huggingface.co/blog/modular-diffusers#community-pipelines\">Community Pipelines</a></li>\n<li><a target=\"_blank\" rel=\"noopener noreferrer\" href=\"https://huggingface.co/blog/modular-diffusers#integration-with-mellon\">Integration with Mellon</a></li>\n</ul>\n<h2>Quickstart</h2>\n<p>Here is a simple example of how to run inference with <code>FLUX.2 Klein 4B</code> using pre-built blocks:</p>\n<pre><code class=\"language-python\"><span class=\"code-snippet\" data-language=\"Python\"><span class=\"syntax-source syntax-python\"><span class=\"syntax-meta syntax-statement syntax-import syntax-python\"><span class=\"syntax-keyword syntax-control syntax-import syntax-python\">import</span> <span class=\"syntax-meta syntax-qualified-name syntax-python\"><span class=\"syntax-meta syntax-generic-name syntax-python\">torch</span></span></span>\n<span class=\"syntax-meta syntax-statement syntax-import syntax-python\"><span class=\"syntax-keyword syntax-control syntax-import syntax-from syntax-python\">from</span></span><span class=\"syntax-meta syntax-statement syntax-import syntax-python\"><span class=\"syntax-meta syntax-import-source syntax-python\"> <span class=\"syntax-meta syntax-import-path syntax-python\"><span class=\"syntax-meta syntax-import-name syntax-python\">diffusers</span></span> <span class=\"syntax-meta syntax-statement syntax-import syntax-python\"><span class=\"syntax-keyword syntax-control syntax-import syntax-python\">import</span></span></span></span><span class=\"syntax-meta syntax-statement syntax-import syntax-python\"></span><span class=\"syntax-meta syntax-statement syntax-import syntax-python\"> <span class=\"syntax-meta syntax-generic-name syntax-python\">ModularPipeline</span></span>\n\n<span class=\"syntax-comment syntax-line syntax-number-sign syntax-python\"><span class=\"syntax-punctuation syntax-definition syntax-comment syntax-python\">#</span> Create a modular pipeline - this only defines the workflow, model weights have not been loaded yet\n</span><span class=\"syntax-meta syntax-qualified-name syntax-python\"><span class=\"syntax-meta syntax-generic-name syntax-python\">pipe</span></span> <span class=\"syntax-keyword syntax-operator syntax-assignment syntax-python\">=</span> <span class=\"syntax-meta syntax-function-call syntax-python\"><span class=\"syntax-meta syntax-qualified-name syntax-python\"><span class=\"syntax-meta syntax-generic-name syntax-python\">ModularPipeline</span><span class=\"syntax-punctuation syntax-accessor syntax-dot syntax-python\">.</span></span><span class=\"syntax-meta syntax-qualified-name syntax-python\"><span class=\"syntax-variable syntax-function syntax-python\">from_pretrained</span></span><span class=\"syntax-punctuation syntax-section syntax-arguments syntax-begin syntax-python\">(</span><span class=\"syntax-meta syntax-function-call syntax-arguments syntax-python\">\n    <span class=\"syntax-meta syntax-string syntax-python\"><span class=\"syntax-string syntax-quoted syntax-double syntax-python\"><span class=\"syntax-punctuation syntax-definition syntax-string syntax-begin syntax-python\">&quot;</span></span></span><span class=\"syntax-meta syntax-string syntax-python\"><span class=\"syntax-string syntax-quoted syntax-double syntax-python\">black-forest-labs/FLUX.2-klein-4B<span class=\"syntax-punctuation syntax-definition syntax-string syntax-end syntax-python\">&quot;</span></span></span>\n</span><span class=\"syntax-punctuation syntax-section syntax-arguments syntax-end syntax-python\">)</span></span>\n<span class=\"syntax-comment syntax-line syntax-number-sign syntax-python\"><span class=\"syntax-punctuation syntax-definition syntax-comment syntax-python\">#</span>  Now load the model weights — configure dtype, quantization, etc in this step\n</span><span class=\"syntax-meta syntax-function-call syntax-python\"><span class=\"syntax-meta syntax-qualified-name syntax-python\"><span class=\"syntax-meta syntax-generic-name syntax-python\">pipe</span><span class=\"syntax-punctuation syntax-accessor syntax-dot syntax-python\">.</span></span><span class=\"syntax-meta syntax-qualified-name syntax-python\"><span class=\"syntax-variable syntax-function syntax-python\">load_components</span></span><span class=\"syntax-punctuation syntax-section syntax-arguments syntax-begin syntax-python\">(</span><span class=\"syntax-meta syntax-function-call syntax-arguments syntax-python\"><span class=\"syntax-variable syntax-parameter syntax-python\">torch_dtype</span><span class=\"syntax-keyword syntax-operator syntax-assignment syntax-python\">=</span><span class=\"syntax-meta syntax-qualified-name syntax-python\"><span class=\"syntax-meta syntax-generic-name syntax-python\">torch</span><span class=\"syntax-punctuation syntax-accessor syntax-dot syntax-python\">.</span><span class=\"syntax-meta syntax-generic-name syntax-python\">bfloat16</span></span></span><span class=\"syntax-punctuation syntax-section syntax-arguments syntax-end syntax-python\">)</span></span>\n<span class=\"syntax-meta syntax-function-call syntax-python\"><span class=\"syntax-meta syntax-qualified-name syntax-python\"><span class=\"syntax-meta syntax-generic-name syntax-python\">pipe</span><span class=\"syntax-punctuation syntax-accessor syntax-dot syntax-python\">.</span></span><span class=\"syntax-meta syntax-qualified-name syntax-python\"><span class=\"syntax-variable syntax-function syntax-python\">to</span></span><span class=\"syntax-punctuation syntax-section syntax-arguments syntax-begin syntax-python\">(</span><span class=\"syntax-meta syntax-function-call syntax-arguments syntax-python\"><span class=\"syntax-meta syntax-string syntax-python\"><span class=\"syntax-string syntax-quoted syntax-double syntax-python\"><span class=\"syntax-punctuation syntax-definition syntax-string syntax-begin syntax-python\">&quot;</span></span></span><span class=\"syntax-meta syntax-string syntax-python\"><span class=\"syntax-string syntax-quoted syntax-double syntax-python\">cuda<span class=\"syntax-punctuation syntax-definition syntax-string syntax-end syntax-python\">&quot;</span></span></span></span><span class=\"syntax-punctuation syntax-section syntax-arguments syntax-end syntax-python\">)</span></span>\n\n<span class=\"syntax-comment syntax-line syntax-number-sign syntax-python\"><span class=\"syntax-punctuation syntax-definition syntax-comment syntax-python\">#</span> Generate an image - API remains the same as DiffusionPipeline\n</span><span class=\"syntax-meta syntax-qualified-name syntax-python\"><span class=\"syntax-meta syntax-generic-name syntax-python\">image</span></span> <span class=\"syntax-keyword syntax-operator syntax-assignment syntax-python\">=</span> <span class=\"syntax-meta syntax-function-call syntax-python\"><span class=\"syntax-meta syntax-qualified-name syntax-python\"><span class=\"syntax-variable syntax-function syntax-python\">pipe</span></span><span class=\"syntax-punctuation syntax-section syntax-arguments syntax-begin syntax-python\">(</span><span class=\"syntax-meta syntax-function-call syntax-arguments syntax-python\">\n    <span class=\"syntax-variable syntax-parameter syntax-python\">prompt</span><span class=\"syntax-keyword syntax-operator syntax-assignment syntax-python\">=</span><span class=\"syntax-meta syntax-string syntax-python\"><span class=\"syntax-string syntax-quoted syntax-double syntax-python\"><span class=\"syntax-punctuation syntax-definition syntax-string syntax-begin syntax-python\">&quot;</span></span></span><span class=\"syntax-meta syntax-string syntax-python\"><span class=\"syntax-string syntax-quoted syntax-double syntax-python\">a serene landscape at sunset<span class=\"syntax-punctuation syntax-definition syntax-string syntax-end syntax-python\">&quot;</span></span></span><span class=\"syntax-punctuation syntax-separator syntax-arguments syntax-python\">,</span>\n    <span class=\"syntax-variable syntax-parameter syntax-python\">num_inference_steps</span><span class=\"syntax-keyword syntax-operator syntax-assignment syntax-python\">=</span><span class=\"syntax-constant syntax-numeric syntax-integer syntax-decimal syntax-python\">4</span><span class=\"syntax-punctuation syntax-separator syntax-arguments syntax-python\">,</span>\n</span><span class=\"syntax-punctuation syntax-section syntax-arguments syntax-end syntax-python\">)</span></span><span class=\"syntax-punctuation syntax-accessor syntax-dot syntax-python\">.</span><span class=\"syntax-meta syntax-generic-name syntax-python\">images</span><span class=\"syntax-meta syntax-item-access syntax-python\"><span class=\"syntax-punctuation syntax-section syntax-brackets syntax-begin syntax-python\">[</span></span><span class=\"syntax-meta syntax-item-access syntax-arguments syntax-python\"><span class=\"syntax-constant syntax-numeric syntax-integer syntax-decimal syntax-python\">0</span></span><span class=\"syntax-meta syntax-item-access syntax-python\"><span class=\"syntax-punctuation syntax-section syntax-brackets syntax-end syntax-python\">]</span></span>\n\n<span class=\"syntax-meta syntax-function-call syntax-python\"><span class=\"syntax-meta syntax-qualified-name syntax-python\"><span class=\"syntax-meta syntax-generic-name syntax-python\">image</span><span class=\"syntax-punctuation syntax-accessor syntax-dot syntax-python\">.</span></span><span class=\"syntax-meta syntax-qualified-name syntax-python\"><span class=\"syntax-variable syntax-function syntax-python\">save</span></span><span class=\"syntax-punctuation syntax-section syntax-arguments syntax-begin syntax-python\">(</span><span class=\"syntax-meta syntax-function-call syntax-arguments syntax-python\"><span class=\"syntax-meta syntax-string syntax-python\"><span class=\"syntax-string syntax-quoted syntax-double syntax-python\"><span class=\"syntax-punctuation syntax-definition syntax-string syntax-begin syntax-python\">&quot;</span></span></span><span class=\"syntax-meta syntax-string syntax-python\"><span class=\"syntax-string syntax-quoted syntax-double syntax-python\">output.png<span class=\"syntax-punctuation syntax-definition syntax-string syntax-end syntax-python\">&quot;</span></span></span></span><span class=\"syntax-punctuation syntax-section syntax-arguments syntax-end syntax-python\">)</span></span>\n</span></span></code></pre>\n<p>You get the same results as with a standard <code>DiffusionPipeline</code>, but the pipeline is very different under the hood: it's composed of flexible <em>blocks</em> — text encoding, image encoding, denoising, and decoding — that you can inspect directly:</p>\n<pre><code class=\"language-python\"><span class=\"code-snippet\" data-language=\"Python\"><span class=\"syntax-source syntax-python\"><span class=\"syntax-meta syntax-function-call syntax-python\"><span class=\"syntax-meta syntax-qualified-name syntax-python\"><span class=\"syntax-support syntax-function syntax-builtin syntax-python\">print</span></span><span class=\"syntax-punctuation syntax-section syntax-arguments syntax-begin syntax-python\">(</span><span class=\"syntax-meta syntax-function-call syntax-arguments syntax-python\"><span class=\"syntax-meta syntax-qualified-name syntax-python\"><span class=\"syntax-meta syntax-generic-name syntax-python\">pipe</span><span class=\"syntax-punctuation syntax-accessor syntax-dot syntax-python\">.</span><span class=\"syntax-meta syntax-generic-name syntax-python\">blocks</span></span></span><span class=\"syntax-punctuation syntax-section syntax-arguments syntax-end syntax-python\">)</span></span>\n</span></span></code></pre>\n<pre><code class=\"language-bash\"><span class=\"code-snippet\" data-language=\"Shell\"><span class=\"syntax-source syntax-shell syntax-bash\"><span class=\"syntax-meta syntax-function-call syntax-shell\"><span class=\"syntax-variable syntax-function syntax-shell\">Flux2KleinAutoBlocks</span></span><span class=\"syntax-meta syntax-function-call syntax-arguments syntax-shell\">(</span>\n  <span class=\"syntax-meta syntax-function-call syntax-shell\"><span class=\"syntax-variable syntax-function syntax-shell\">...</span></span>\n  <span class=\"syntax-meta syntax-function-call syntax-shell\"><span class=\"syntax-variable syntax-function syntax-shell\">Sub-Blocks:</span></span>\n    <span class=\"syntax-meta syntax-function-call syntax-shell\"><span class=\"syntax-variable syntax-function syntax-shell\">[0]</span></span><span class=\"syntax-meta syntax-function-call syntax-arguments syntax-shell\"> text_encoder (Flux2KleinTextEncoderStep</span><span class=\"syntax-meta syntax-function-call syntax-shell\"></span><span class=\"syntax-meta syntax-post-cmd syntax-shell\">)</span>\n    <span class=\"syntax-meta syntax-function-call syntax-shell\"><span class=\"syntax-variable syntax-function syntax-shell\">[1]</span></span><span class=\"syntax-meta syntax-function-call syntax-arguments syntax-shell\"> vae_encoder (Flux2KleinAutoVaeEncoderStep</span><span class=\"syntax-meta syntax-function-call syntax-shell\"></span><span class=\"syntax-meta syntax-post-cmd syntax-shell\">)</span>\n    <span class=\"syntax-meta syntax-function-call syntax-shell\"><span class=\"syntax-variable syntax-function syntax-shell\">[2]</span></span><span class=\"syntax-meta syntax-function-call syntax-arguments syntax-shell\"> denoise (Flux2KleinCoreDenoiseStep</span><span class=\"syntax-meta syntax-function-call syntax-shell\"></span><span class=\"syntax-meta syntax-post-cmd syntax-shell\">)</span>\n    <span class=\"syntax-meta syntax-function-call syntax-shell\"><span class=\"syntax-variable syntax-function syntax-shell\">[3]</span></span><span class=\"syntax-meta syntax-function-call syntax-arguments syntax-shell\"> decode (Flux2DecodeStep</span><span class=\"syntax-meta syntax-function-call syntax-shell\"></span><span class=\"syntax-meta syntax-post-cmd syntax-shell\">)</span>\n<span class=\"syntax-meta syntax-function-call syntax-shell\"></span><span class=\"syntax-meta syntax-post-cmd syntax-shell\">)</span>\n</span></span></code></pre>\n<p>Each block is self-contained with its own inputs and outputs. You can run any block independently as its own pipeline, or add, remove, and swap blocks freely — they dynamically recompose to work with whatever blocks remain. Use <code>.init_pipeline()</code> to convert blocks into a runnable pipeline, and <code>.load_components()</code> to load the model weights.</p>\n<pre><code class=\"language-python\"><span class=\"code-snippet\" data-language=\"Python\"><span class=\"syntax-source syntax-python\"><span class=\"syntax-comment syntax-line syntax-number-sign syntax-python\"><span class=\"syntax-punctuation syntax-definition syntax-comment syntax-python\">#</span> get a copy of the blocks\n</span><span class=\"syntax-meta syntax-qualified-name syntax-python\"><span class=\"syntax-meta syntax-generic-name syntax-python\">blocks</span></span> <span class=\"syntax-keyword syntax-operator syntax-assignment syntax-python\">=</span> <span class=\"syntax-meta syntax-qualified-name syntax-python\"><span class=\"syntax-meta syntax-generic-name syntax-python\">pipe</span><span class=\"syntax-punctuation syntax-accessor syntax-dot syntax-python\">.</span><span class=\"syntax-meta syntax-generic-name syntax-python\">blocks</span></span>\n\n<span class=\"syntax-comment syntax-line syntax-number-sign syntax-python\"><span class=\"syntax-punctuation syntax-definition syntax-comment syntax-python\">#</span> pop out the text_encoder block\n</span><span class=\"syntax-meta syntax-qualified-name syntax-python\"><span class=\"syntax-meta syntax-generic-name syntax-python\">text_blocks</span></span> <span class=\"syntax-keyword syntax-operator syntax-assignment syntax-python\">=</span> <span class=\"syntax-meta syntax-function-call syntax-python\"><span class=\"syntax-meta syntax-qualified-name syntax-python\"><span class=\"syntax-meta syntax-generic-name syntax-python\">blocks</span><span class=\"syntax-punctuation syntax-accessor syntax-dot syntax-python\">.</span><span class=\"syntax-meta syntax-generic-name syntax-python\">sub_blocks</span><span class=\"syntax-punctuation syntax-accessor syntax-dot syntax-python\">.</span></span><span class=\"syntax-meta syntax-qualified-name syntax-python\"><span class=\"syntax-variable syntax-function syntax-python\">pop</span></span><span class=\"syntax-punctuation syntax-section syntax-arguments syntax-begin syntax-python\">(</span><span class=\"syntax-meta syntax-function-call syntax-arguments syntax-python\"><span class=\"syntax-meta syntax-string syntax-python\"><span class=\"syntax-string syntax-quoted syntax-double syntax-python\"><span class=\"syntax-punctuation syntax-definition syntax-string syntax-begin syntax-python\">&quot;</span></span></span><span class=\"syntax-meta syntax-string syntax-python\"><span class=\"syntax-string syntax-quoted syntax-double syntax-python\">text_encoder<span class=\"syntax-punctuation syntax-definition syntax-string syntax-end syntax-python\">&quot;</span></span></span></span><span class=\"syntax-punctuation syntax-section syntax-arguments syntax-end syntax-python\">)</span></span>\n\n<span class=\"syntax-comment syntax-line syntax-number-sign syntax-python\"><span class=\"syntax-punctuation syntax-definition syntax-comment syntax-python\">#</span> run it as its own pipeline\n</span><span class=\"syntax-meta syntax-qualified-name syntax-python\"><span class=\"syntax-meta syntax-generic-name syntax-python\">text_pipe</span></span> <span class=\"syntax-keyword syntax-operator syntax-assignment syntax-python\">=</span> <span class=\"syntax-meta syntax-function-call syntax-python\"><span class=\"syntax-meta syntax-qualified-name syntax-python\"><span class=\"syntax-meta syntax-generic-name syntax-python\">text_blocks</span><span class=\"syntax-punctuation syntax-accessor syntax-dot syntax-python\">.</span></span><span class=\"syntax-meta syntax-qualified-name syntax-python\"><span class=\"syntax-variable syntax-function syntax-python\">init_pipeline</span></span><span class=\"syntax-punctuation syntax-section syntax-arguments syntax-begin syntax-python\">(</span><span class=\"syntax-meta syntax-function-call syntax-arguments syntax-python\"><span class=\"syntax-meta syntax-string syntax-python\"><span class=\"syntax-string syntax-quoted syntax-double syntax-python\"><span class=\"syntax-punctuation syntax-definition syntax-string syntax-begin syntax-python\">&quot;</span></span></span><span class=\"syntax-meta syntax-string syntax-python\"><span class=\"syntax-string syntax-quoted syntax-double syntax-python\">black-forest-labs/FLUX.2-klein-4B<span class=\"syntax-punctuation syntax-definition syntax-string syntax-end syntax-python\">&quot;</span></span></span></span><span class=\"syntax-punctuation syntax-section syntax-arguments syntax-end syntax-python\">)</span></span>\n\n<span class=\"syntax-comment syntax-line syntax-number-sign syntax-python\"><span class=\"syntax-punctuation syntax-definition syntax-comment syntax-python\">#</span> load the text_encoder, or reuse already loaded components: text_pipe.update_components(text_encoder=pipe.text_encoder)\n</span><span class=\"syntax-meta syntax-function-call syntax-python\"><span class=\"syntax-meta syntax-qualified-name syntax-python\"><span class=\"syntax-meta syntax-generic-name syntax-python\">text_pipe</span><span class=\"syntax-punctuation syntax-accessor syntax-dot syntax-python\">.</span></span><span class=\"syntax-meta syntax-qualified-name syntax-python\"><span class=\"syntax-variable syntax-function syntax-python\">load_components</span></span><span class=\"syntax-punctuation syntax-section syntax-arguments syntax-begin syntax-python\">(</span><span class=\"syntax-meta syntax-function-call syntax-arguments syntax-python\"><span class=\"syntax-variable syntax-parameter syntax-python\">torch_dtype</span><span class=\"syntax-keyword syntax-operator syntax-assignment syntax-python\">=</span><span class=\"syntax-meta syntax-qualified-name syntax-python\"><span class=\"syntax-meta syntax-generic-name syntax-python\">torch</span><span class=\"syntax-punctuation syntax-accessor syntax-dot syntax-python\">.</span><span class=\"syntax-meta syntax-generic-name syntax-python\">bfloat16</span></span></span><span class=\"syntax-punctuation syntax-section syntax-arguments syntax-end syntax-python\">)</span></span>\n<span class=\"syntax-meta syntax-function-call syntax-python\"><span class=\"syntax-meta syntax-qualified-name syntax-python\"><span class=\"syntax-meta syntax-generic-name syntax-python\">text_pipe</span><span class=\"syntax-punctuation syntax-accessor syntax-dot syntax-python\">.</span></span><span class=\"syntax-meta syntax-qualified-name syntax-python\"><span class=\"syntax-variable syntax-function syntax-python\">to</span></span><span class=\"syntax-punctuation syntax-section syntax-arguments syntax-begin syntax-python\">(</span><span class=\"syntax-meta syntax-function-call syntax-arguments syntax-python\"><span class=\"syntax-meta syntax-string syntax-python\"><span class=\"syntax-string syntax-quoted syntax-double syntax-python\"><span class=\"syntax-punctuation syntax-definition syntax-string syntax-begin syntax-python\">&quot;</span></span></span><span class=\"syntax-meta syntax-string syntax-python\"><span class=\"syntax-string syntax-quoted syntax-double syntax-python\">cuda<span class=\"syntax-punctuation syntax-definition syntax-string syntax-end syntax-python\">&quot;</span></span></span></span><span class=\"syntax-punctuation syntax-section syntax-arguments syntax-end syntax-python\">)</span></span>\n<span class=\"syntax-meta syntax-qualified-name syntax-python\"><span class=\"syntax-meta syntax-generic-name syntax-python\">prompt_embeds</span></span> <span class=\"syntax-keyword syntax-operator syntax-assignment syntax-python\">=</span> <span class=\"syntax-meta syntax-function-call syntax-python\"><span class=\"syntax-meta syntax-qualified-name syntax-python\"><span class=\"syntax-variable syntax-function syntax-python\">text_pipe</span></span><span class=\"syntax-punctuation syntax-section syntax-arguments syntax-begin syntax-python\">(</span><span class=\"syntax-meta syntax-function-call syntax-arguments syntax-python\"><span class=\"syntax-variable syntax-parameter syntax-python\">prompt</span><span class=\"syntax-keyword syntax-operator syntax-assignment syntax-python\">=</span><span class=\"syntax-meta syntax-string syntax-python\"><span class=\"syntax-string syntax-quoted syntax-double syntax-python\"><span class=\"syntax-punctuation syntax-definition syntax-string syntax-begin syntax-python\">&quot;</span></span></span><span class=\"syntax-meta syntax-string syntax-python\"><span class=\"syntax-string syntax-quoted syntax-double syntax-python\">a serene landscape at sunset<span class=\"syntax-punctuation syntax-definition syntax-string syntax-end syntax-python\">&quot;</span></span></span></span><span class=\"syntax-punctuation syntax-section syntax-arguments syntax-end syntax-python\">)</span></span><span class=\"syntax-punctuation syntax-accessor syntax-dot syntax-python\">.</span><span class=\"syntax-meta syntax-generic-name syntax-python\">prompt_embeds</span>\n\n<span class=\"syntax-comment syntax-line syntax-number-sign syntax-python\"><span class=\"syntax-punctuation syntax-definition syntax-comment syntax-python\">#</span> create a new pipeline from the remaining blocks\n</span><span class=\"syntax-comment syntax-line syntax-number-sign syntax-python\"><span class=\"syntax-punctuation syntax-definition syntax-comment syntax-python\">#</span> it now accepts prompt_embeds directly instead of prompt\n</span><span class=\"syntax-meta syntax-qualified-name syntax-python\"><span class=\"syntax-meta syntax-generic-name syntax-python\">remaining_pipe</span></span> <span class=\"syntax-keyword syntax-operator syntax-assignment syntax-python\">=</span> <span class=\"syntax-meta syntax-function-call syntax-python\"><span class=\"syntax-meta syntax-qualified-name syntax-python\"><span class=\"syntax-meta syntax-generic-name syntax-python\">blocks</span><span class=\"syntax-punctuation syntax-accessor syntax-dot syntax-python\">.</span></span><span class=\"syntax-meta syntax-qualified-name syntax-python\"><span class=\"syntax-variable syntax-function syntax-python\">init_pipeline</span></span><span class=\"syntax-punctuation syntax-section syntax-arguments syntax-begin syntax-python\">(</span><span class=\"syntax-meta syntax-function-call syntax-arguments syntax-python\"><span class=\"syntax-meta syntax-string syntax-python\"><span class=\"syntax-string syntax-quoted syntax-double syntax-python\"><span class=\"syntax-punctuation syntax-definition syntax-string syntax-begin syntax-python\">&quot;</span></span></span><span class=\"syntax-meta syntax-string syntax-python\"><span class=\"syntax-string syntax-quoted syntax-double syntax-python\">black-forest-labs/FLUX.2-klein-4B<span class=\"syntax-punctuation syntax-definition syntax-string syntax-end syntax-python\">&quot;</span></span></span></span><span class=\"syntax-punctuation syntax-section syntax-arguments syntax-end syntax-python\">)</span></span>\n<span class=\"syntax-meta syntax-function-call syntax-python\"><span class=\"syntax-meta syntax-qualified-name syntax-python\"><span class=\"syntax-meta syntax-generic-name syntax-python\">remaining_pipe</span><span class=\"syntax-punctuation syntax-accessor syntax-dot syntax-python\">.</span></span><span class=\"syntax-meta syntax-qualified-name syntax-python\"><span class=\"syntax-variable syntax-function syntax-python\">load_components</span></span><span class=\"syntax-punctuation syntax-section syntax-arguments syntax-begin syntax-python\">(</span><span class=\"syntax-meta syntax-function-call syntax-arguments syntax-python\"><span class=\"syntax-variable syntax-parameter syntax-python\">torch_dtype</span><span class=\"syntax-keyword syntax-operator syntax-assignment syntax-python\">=</span><span class=\"syntax-meta syntax-qualified-name syntax-python\"><span class=\"syntax-meta syntax-generic-name syntax-python\">torch</span><span class=\"syntax-punctuation syntax-accessor syntax-dot syntax-python\">.</span><span class=\"syntax-meta syntax-generic-name syntax-python\">bfloat16</span></span></span><span class=\"syntax-punctuation syntax-section syntax-arguments syntax-end syntax-python\">)</span></span>\n<span class=\"syntax-meta syntax-function-call syntax-python\"><span class=\"syntax-meta syntax-qualified-name syntax-python\"><span class=\"syntax-meta syntax-generic-name syntax-python\">remaining_pipe</span><span class=\"syntax-punctuation syntax-accessor syntax-dot syntax-python\">.</span></span><span class=\"syntax-meta syntax-qualified-name syntax-python\"><span class=\"syntax-variable syntax-function syntax-python\">to</span></span><span class=\"syntax-punctuation syntax-section syntax-arguments syntax-begin syntax-python\">(</span><span class=\"syntax-meta syntax-function-call syntax-arguments syntax-python\"><span class=\"syntax-meta syntax-string syntax-python\"><span class=\"syntax-string syntax-quoted syntax-double syntax-python\"><span class=\"syntax-punctuation syntax-definition syntax-string syntax-begin syntax-python\">&quot;</span></span></span><span class=\"syntax-meta syntax-string syntax-python\"><span class=\"syntax-string syntax-quoted syntax-double syntax-python\">cuda<span class=\"syntax-punctuation syntax-definition syntax-string syntax-end syntax-python\">&quot;</span></span></span></span><span class=\"syntax-punctuation syntax-section syntax-arguments syntax-end syntax-python\">)</span></span>\n<span class=\"syntax-meta syntax-qualified-name syntax-python\"><span class=\"syntax-meta syntax-generic-name syntax-python\">image</span></span> <span class=\"syntax-keyword syntax-operator syntax-assignment syntax-python\">=</span> <span class=\"syntax-meta syntax-function-call syntax-python\"><span class=\"syntax-meta syntax-qualified-name syntax-python\"><span class=\"syntax-variable syntax-function syntax-python\">remaining_pipe</span></span><span class=\"syntax-punctuation syntax-section syntax-arguments syntax-begin syntax-python\">(</span><span class=\"syntax-meta syntax-function-call syntax-arguments syntax-python\"><span class=\"syntax-variable syntax-parameter syntax-python\">prompt_embeds</span><span class=\"syntax-keyword syntax-operator syntax-assignment syntax-python\">=</span><span class=\"syntax-meta syntax-qualified-name syntax-python\"><span class=\"syntax-meta syntax-generic-name syntax-python\">prompt_embeds</span></span><span class=\"syntax-punctuation syntax-separator syntax-arguments syntax-python\">,</span> <span class=\"syntax-variable syntax-parameter syntax-python\">num_inference_steps</span><span class=\"syntax-keyword syntax-operator syntax-assignment syntax-python\">=</span><span class=\"syntax-constant syntax-numeric syntax-integer syntax-decimal syntax-python\">4</span></span><span class=\"syntax-punctuation syntax-section syntax-arguments syntax-end syntax-python\">)</span></span><span class=\"syntax-punctuation syntax-accessor syntax-dot syntax-python\">.</span><span class=\"syntax-meta syntax-generic-name syntax-python\">images</span><span class=\"syntax-meta syntax-item-access syntax-python\"><span class=\"syntax-punctuation syntax-section syntax-brackets syntax-begin syntax-python\">[</span></span><span class=\"syntax-meta syntax-item-access syntax-arguments syntax-python\"><span class=\"syntax-constant syntax-numeric syntax-integer syntax-decimal syntax-python\">0</span></span><span class=\"syntax-meta syntax-item-access syntax-python\"><span class=\"syntax-punctuation syntax-section syntax-brackets syntax-end syntax-python\">]</span></span>\n</span></span></code></pre>\n<p>For more on block types, composition patterns, lazy loading, and memory management with <code>ComponentsManager</code>, check out the <a target=\"_blank\" rel=\"noopener noreferrer\" href=\"https://huggingface.co/docs/diffusers/en/modular_diffusers/overview\">Modular Diffusers documentation</a>.</p>\n<h2>Custom Blocks</h2>\n<p>Modular Diffusers really shines when creating your own blocks. A custom block is a Python class that defines its components, inputs, outputs, and computation logic — and once defined, you can plug it into any workflow.</p>\n<h3>Writing a Custom Block</h3>\n<p>Here's an example block that extracts depth maps from images using <a target=\"_blank\" rel=\"noopener noreferrer\" href=\"https://huggingface.co/depth-anything/Depth-Anything-V2-Large\">Depth Anything V2</a>.</p>\n<pre><code class=\"language-python\"><span class=\"code-snippet\" data-language=\"Python\"><span class=\"syntax-source syntax-python\"><span class=\"syntax-meta syntax-class syntax-python\"><span class=\"syntax-storage syntax-type syntax-class syntax-python\">class</span> <span class=\"syntax-entity syntax-name syntax-class syntax-python\"><span class=\"syntax-meta syntax-generic-name syntax-python\">DepthProcessorBlock</span></span><span class=\"syntax-meta syntax-class syntax-inheritance syntax-python\"><span class=\"syntax-punctuation syntax-section syntax-inheritance syntax-begin syntax-python\">(</span></span></span><span class=\"syntax-meta syntax-class syntax-inheritance syntax-python\"><span class=\"syntax-entity syntax-other syntax-inherited-class syntax-python\">ModularPipelineBlocks</span><span class=\"syntax-punctuation syntax-section syntax-inheritance syntax-end syntax-python\">)</span></span><span class=\"syntax-meta syntax-class syntax-python\"><span class=\"syntax-punctuation syntax-section syntax-class syntax-begin syntax-python\">:</span></span>\n    <span class=\"syntax-meta syntax-annotation syntax-python\"><span class=\"syntax-punctuation syntax-definition syntax-annotation syntax-python\">@</span><span class=\"syntax-meta syntax-qualified-name syntax-python\"><span class=\"syntax-support syntax-function syntax-builtin syntax-python\">property</span></span></span>\n<span class=\"syntax-meta syntax-function syntax-python\">    <span class=\"syntax-storage syntax-type syntax-function syntax-python\">def</span> <span class=\"syntax-entity syntax-name syntax-function syntax-python\"><span class=\"syntax-meta syntax-generic-name syntax-python\">expected_components</span></span></span><span class=\"syntax-meta syntax-function syntax-parameters syntax-python\"><span class=\"syntax-punctuation syntax-section syntax-parameters syntax-begin syntax-python\">(</span></span><span class=\"syntax-meta syntax-function syntax-parameters syntax-python\"><span class=\"syntax-variable syntax-parameter syntax-python\">self</span><span class=\"syntax-punctuation syntax-section syntax-parameters syntax-end syntax-python\">)</span></span><span class=\"syntax-meta syntax-function syntax-python\"><span class=\"syntax-punctuation syntax-section syntax-function syntax-begin syntax-python\">:</span></span>\n        <span class=\"syntax-keyword syntax-control syntax-flow syntax-return syntax-python\">return</span> <span class=\"syntax-meta syntax-structure syntax-list syntax-python\"><span class=\"syntax-punctuation syntax-section syntax-list syntax-begin syntax-python\">[</span>\n            <span class=\"syntax-meta syntax-function-call syntax-python\"><span class=\"syntax-meta syntax-qualified-name syntax-python\"><span class=\"syntax-variable syntax-function syntax-python\">ComponentSpec</span></span><span class=\"syntax-punctuation syntax-section syntax-arguments syntax-begin syntax-python\">(</span><span class=\"syntax-meta syntax-function-call syntax-arguments syntax-python\"><span class=\"syntax-meta syntax-string syntax-python\"><span class=\"syntax-string syntax-quoted syntax-double syntax-python\"><span class=\"syntax-punctuation syntax-definition syntax-string syntax-begin syntax-python\">&quot;</span></span></span><span class=\"syntax-meta syntax-string syntax-python\"><span class=\"syntax-string syntax-quoted syntax-double syntax-python\">depth_processor<span class=\"syntax-punctuation syntax-definition syntax-string syntax-end syntax-python\">&quot;</span></span></span><span class=\"syntax-punctuation syntax-separator syntax-arguments syntax-python\">,</span> <span class=\"syntax-meta syntax-qualified-name syntax-python\"><span class=\"syntax-meta syntax-generic-name syntax-python\">DepthPreprocessor</span></span><span class=\"syntax-punctuation syntax-separator syntax-arguments syntax-python\">,</span>\n                          <span class=\"syntax-variable syntax-parameter syntax-python\">pretrained_model_name_or_path</span><span class=\"syntax-keyword syntax-operator syntax-assignment syntax-python\">=</span><span class=\"syntax-meta syntax-string syntax-python\"><span class=\"syntax-string syntax-quoted syntax-double syntax-python\"><span class=\"syntax-punctuation syntax-definition syntax-string syntax-begin syntax-python\">&quot;</span></span></span><span class=\"syntax-meta syntax-string syntax-python\"><span class=\"syntax-string syntax-quoted syntax-double syntax-python\">depth-anything/Depth-Anything-V2-Large-hf<span class=\"syntax-punctuation syntax-definition syntax-string syntax-end syntax-python\">&quot;</span></span></span></span><span class=\"syntax-punctuation syntax-section syntax-arguments syntax-end syntax-python\">)</span></span>\n        <span class=\"syntax-punctuation syntax-section syntax-list syntax-end syntax-python\">]</span></span>\n\n    <span class=\"syntax-meta syntax-annotation syntax-python\"><span class=\"syntax-punctuation syntax-definition syntax-annotation syntax-python\">@</span><span class=\"syntax-meta syntax-qualified-name syntax-python\"><span class=\"syntax-support syntax-function syntax-builtin syntax-python\">property</span></span></span>\n<span class=\"syntax-meta syntax-function syntax-python\">    <span class=\"syntax-storage syntax-type syntax-function syntax-python\">def</span> <span class=\"syntax-entity syntax-name syntax-function syntax-python\"><span class=\"syntax-meta syntax-generic-name syntax-python\">inputs</span></span></span><span class=\"syntax-meta syntax-function syntax-parameters syntax-python\"><span class=\"syntax-punctuation syntax-section syntax-parameters syntax-begin syntax-python\">(</span></span><span class=\"syntax-meta syntax-function syntax-parameters syntax-python\"><span class=\"syntax-variable syntax-parameter syntax-python\">self</span><span class=\"syntax-punctuation syntax-section syntax-parameters syntax-end syntax-python\">)</span></span><span class=\"syntax-meta syntax-function syntax-python\"><span class=\"syntax-punctuation syntax-section syntax-function syntax-begin syntax-python\">:</span></span>\n        <span class=\"syntax-keyword syntax-control syntax-flow syntax-return syntax-python\">return</span> <span class=\"syntax-meta syntax-structure syntax-list syntax-python\"><span class=\"syntax-punctuation syntax-section syntax-list syntax-begin syntax-python\">[</span>\n            <span class=\"syntax-meta syntax-function-call syntax-python\"><span class=\"syntax-meta syntax-qualified-name syntax-python\"><span class=\"syntax-variable syntax-function syntax-python\">InputParam</span></span><span class=\"syntax-punctuation syntax-section syntax-arguments syntax-begin syntax-python\">(</span><span class=\"syntax-meta syntax-function-call syntax-arguments syntax-python\"><span class=\"syntax-meta syntax-string syntax-python\"><span class=\"syntax-string syntax-quoted syntax-double syntax-python\"><span class=\"syntax-punctuation syntax-definition syntax-string syntax-begin syntax-python\">&quot;</span></span></span><span class=\"syntax-meta syntax-string syntax-python\"><span class=\"syntax-string syntax-quoted syntax-double syntax-python\">image<span class=\"syntax-punctuation syntax-definition syntax-string syntax-end syntax-python\">&quot;</span></span></span><span class=\"syntax-punctuation syntax-separator syntax-arguments syntax-python\">,</span> <span class=\"syntax-variable syntax-parameter syntax-python\">required</span><span class=\"syntax-keyword syntax-operator syntax-assignment syntax-python\">=</span><span class=\"syntax-constant syntax-language syntax-python\">True</span><span class=\"syntax-punctuation syntax-separator syntax-arguments syntax-python\">,</span>\n                       <span class=\"syntax-variable syntax-parameter syntax-python\">description</span><span class=\"syntax-keyword syntax-operator syntax-assignment syntax-python\">=</span><span class=\"syntax-meta syntax-string syntax-python\"><span class=\"syntax-string syntax-quoted syntax-double syntax-python\"><span class=\"syntax-punctuation syntax-definition syntax-string syntax-begin syntax-python\">&quot;</span></span></span><span class=\"syntax-meta syntax-string syntax-python\"><span class=\"syntax-string syntax-quoted syntax-double syntax-python\">Image(s) to extract depth maps from<span class=\"syntax-punctuation syntax-definition syntax-string syntax-end syntax-python\">&quot;</span></span></span></span><span class=\"syntax-punctuation syntax-section syntax-arguments syntax-end syntax-python\">)</span></span><span class=\"syntax-punctuation syntax-separator syntax-list syntax-python\">,</span>\n        <span class=\"syntax-punctuation syntax-section syntax-list syntax-end syntax-python\">]</span></span>\n\n    <span class=\"syntax-meta syntax-annotation syntax-python\"><span class=\"syntax-punctuation syntax-definition syntax-annotation syntax-python\">@</span><span class=\"syntax-meta syntax-qualified-name syntax-python\"><span class=\"syntax-support syntax-function syntax-builtin syntax-python\">property</span></span></span>\n<span class=\"syntax-meta syntax-function syntax-python\">    <span class=\"syntax-storage syntax-type syntax-function syntax-python\">def</span> <span class=\"syntax-entity syntax-name syntax-function syntax-python\"><span class=\"syntax-meta syntax-generic-name syntax-python\">intermediate_outputs</span></span></span><span class=\"syntax-meta syntax-function syntax-parameters syntax-python\"><span class=\"syntax-punctuation syntax-section syntax-parameters syntax-begin syntax-python\">(</span></span><span class=\"syntax-meta syntax-function syntax-parameters syntax-python\"><span class=\"syntax-variable syntax-parameter syntax-python\">self</span><span class=\"syntax-punctuation syntax-section syntax-parameters syntax-end syntax-python\">)</span></span><span class=\"syntax-meta syntax-function syntax-python\"><span class=\"syntax-punctuation syntax-section syntax-function syntax-begin syntax-python\">:</span></span>\n        <span class=\"syntax-keyword syntax-control syntax-flow syntax-return syntax-python\">return</span> <span class=\"syntax-meta syntax-structure syntax-list syntax-python\"><span class=\"syntax-punctuation syntax-section syntax-list syntax-begin syntax-python\">[</span>\n            <span class=\"syntax-meta syntax-function-call syntax-python\"><span class=\"syntax-meta syntax-qualified-name syntax-python\"><span class=\"syntax-variable syntax-function syntax-python\">OutputParam</span></span><span class=\"syntax-punctuation syntax-section syntax-arguments syntax-begin syntax-python\">(</span><span class=\"syntax-meta syntax-function-call syntax-arguments syntax-python\"><span class=\"syntax-meta syntax-string syntax-python\"><span class=\"syntax-string syntax-quoted syntax-double syntax-python\"><span class=\"syntax-punctuation syntax-definition syntax-string syntax-begin syntax-python\">&quot;</span></span></span><span class=\"syntax-meta syntax-string syntax-python\"><span class=\"syntax-string syntax-quoted syntax-double syntax-python\">control_image<span class=\"syntax-punctuation syntax-definition syntax-string syntax-end syntax-python\">&quot;</span></span></span><span class=\"syntax-punctuation syntax-separator syntax-arguments syntax-python\">,</span> <span class=\"syntax-variable syntax-parameter syntax-python\">type_hint</span><span class=\"syntax-keyword syntax-operator syntax-assignment syntax-python\">=</span><span class=\"syntax-meta syntax-qualified-name syntax-python\"><span class=\"syntax-meta syntax-generic-name syntax-python\">torch</span><span class=\"syntax-punctuation syntax-accessor syntax-dot syntax-python\">.</span><span class=\"syntax-meta syntax-generic-name syntax-python\">Tensor</span></span><span class=\"syntax-punctuation syntax-separator syntax-arguments syntax-python\">,</span>\n                        <span class=\"syntax-variable syntax-parameter syntax-python\">description</span><span class=\"syntax-keyword syntax-operator syntax-assignment syntax-python\">=</span><span class=\"syntax-meta syntax-string syntax-python\"><span class=\"syntax-string syntax-quoted syntax-double syntax-python\"><span class=\"syntax-punctuation syntax-definition syntax-string syntax-begin syntax-python\">&quot;</span></span></span><span class=\"syntax-meta syntax-string syntax-python\"><span class=\"syntax-string syntax-quoted syntax-double syntax-python\">Depth map(s) of input image(s)<span class=\"syntax-punctuation syntax-definition syntax-string syntax-end syntax-python\">&quot;</span></span></span></span><span class=\"syntax-punctuation syntax-section syntax-arguments syntax-end syntax-python\">)</span></span><span class=\"syntax-punctuation syntax-separator syntax-list syntax-python\">,</span>\n        <span class=\"syntax-punctuation syntax-section syntax-list syntax-end syntax-python\">]</span></span>\n\n    <span class=\"syntax-meta syntax-annotation syntax-python\"><span class=\"syntax-punctuation syntax-definition syntax-annotation syntax-python\">@</span><span class=\"syntax-meta syntax-annotation syntax-function syntax-python\"></span></span><span class=\"syntax-meta syntax-annotation syntax-function syntax-python\"><span class=\"syntax-meta syntax-qualified-name syntax-python\"><span class=\"syntax-meta syntax-generic-name syntax-python\">torch</span><span class=\"syntax-punctuation syntax-accessor syntax-dot syntax-python\">.</span></span><span class=\"syntax-meta syntax-qualified-name syntax-python\"><span class=\"syntax-variable syntax-annotation syntax-function syntax-python\">no_grad</span></span><span class=\"syntax-punctuation syntax-section syntax-arguments syntax-begin syntax-python\">(</span><span class=\"syntax-punctuation syntax-section syntax-arguments syntax-end syntax-python\">)</span></span>\n<span class=\"syntax-meta syntax-function syntax-python\">    <span class=\"syntax-storage syntax-type syntax-function syntax-python\">def</span> <span class=\"syntax-entity syntax-name syntax-function syntax-python\"><span class=\"syntax-support syntax-function syntax-magic syntax-python\">__call__</span></span></span><span class=\"syntax-meta syntax-function syntax-parameters syntax-python\"><span class=\"syntax-punctuation syntax-section syntax-parameters syntax-begin syntax-python\">(</span></span><span class=\"syntax-meta syntax-function syntax-parameters syntax-python\"><span class=\"syntax-variable syntax-parameter syntax-python\">self</span><span class=\"syntax-punctuation syntax-separator syntax-parameters syntax-python\">,</span> <span class=\"syntax-variable syntax-parameter syntax-python\">components</span><span class=\"syntax-punctuation syntax-separator syntax-parameters syntax-python\">,</span> <span class=\"syntax-variable syntax-parameter syntax-python\">state</span><span class=\"syntax-punctuation syntax-section syntax-parameters syntax-end syntax-python\">)</span></span><span class=\"syntax-meta syntax-function syntax-python\"><span class=\"syntax-punctuation syntax-section syntax-function syntax-begin syntax-python\">:</span></span>\n        <span class=\"syntax-meta syntax-qualified-name syntax-python\"><span class=\"syntax-meta syntax-generic-name syntax-python\">block_state</span></span> <span class=\"syntax-keyword syntax-operator syntax-assignment syntax-python\">=</span> <span class=\"syntax-meta syntax-function-call syntax-python\"><span class=\"syntax-meta syntax-qualified-name syntax-python\"><span class=\"syntax-variable syntax-language syntax-python\">self</span><span class=\"syntax-punctuation syntax-accessor syntax-dot syntax-python\">.</span></span><span class=\"syntax-meta syntax-qualified-name syntax-python\"><span class=\"syntax-variable syntax-function syntax-python\">get_block_state</span></span><span class=\"syntax-punctuation syntax-section syntax-arguments syntax-begin syntax-python\">(</span><span class=\"syntax-meta syntax-function-call syntax-arguments syntax-python\"><span class=\"syntax-meta syntax-qualified-name syntax-python\"><span class=\"syntax-meta syntax-generic-name syntax-python\">state</span></span></span><span class=\"syntax-punctuation syntax-section syntax-arguments syntax-end syntax-python\">)</span></span>\n        <span class=\"syntax-meta syntax-qualified-name syntax-python\"><span class=\"syntax-meta syntax-generic-name syntax-python\">depth_map</span></span> <span class=\"syntax-keyword syntax-operator syntax-assignment syntax-python\">=</span> <span class=\"syntax-meta syntax-function-call syntax-python\"><span class=\"syntax-meta syntax-qualified-name syntax-python\"><span class=\"syntax-meta syntax-generic-name syntax-python\">components</span><span class=\"syntax-punctuation syntax-accessor syntax-dot syntax-python\">.</span></span><span class=\"syntax-meta syntax-qualified-name syntax-python\"><span class=\"syntax-variable syntax-function syntax-python\">depth_processor</span></span><span class=\"syntax-punctuation syntax-section syntax-arguments syntax-begin syntax-python\">(</span><span class=\"syntax-meta syntax-function-call syntax-arguments syntax-python\"><span class=\"syntax-meta syntax-qualified-name syntax-python\"><span class=\"syntax-meta syntax-generic-name syntax-python\">block_state</span><span class=\"syntax-punctuation syntax-accessor syntax-dot syntax-python\">.</span><span class=\"syntax-meta syntax-generic-name syntax-python\">image</span></span></span><span class=\"syntax-punctuation syntax-section syntax-arguments syntax-end syntax-python\">)</span></span>\n        <span class=\"syntax-meta syntax-qualified-name syntax-python\"><span class=\"syntax-meta syntax-generic-name syntax-python\">block_state</span><span class=\"syntax-punctuation syntax-accessor syntax-dot syntax-python\">.</span><span class=\"syntax-meta syntax-generic-name syntax-python\">control_image</span></span> <span class=\"syntax-keyword syntax-operator syntax-assignment syntax-python\">=</span> <span class=\"syntax-meta syntax-function-call syntax-python\"><span class=\"syntax-meta syntax-qualified-name syntax-python\"><span class=\"syntax-meta syntax-generic-name syntax-python\">depth_map</span><span class=\"syntax-punctuation syntax-accessor syntax-dot syntax-python\">.</span></span><span class=\"syntax-meta syntax-qualified-name syntax-python\"><span class=\"syntax-variable syntax-function syntax-python\">to</span></span><span class=\"syntax-punctuation syntax-section syntax-arguments syntax-begin syntax-python\">(</span><span class=\"syntax-meta syntax-function-call syntax-arguments syntax-python\"><span class=\"syntax-meta syntax-qualified-name syntax-python\"><span class=\"syntax-meta syntax-generic-name syntax-python\">block_state</span><span class=\"syntax-punctuation syntax-accessor syntax-dot syntax-python\">.</span><span class=\"syntax-meta syntax-generic-name syntax-python\">device</span></span></span><span class=\"syntax-punctuation syntax-section syntax-arguments syntax-end syntax-python\">)</span></span>\n        <span class=\"syntax-meta syntax-function-call syntax-python\"><span class=\"syntax-meta syntax-qualified-name syntax-python\"><span class=\"syntax-variable syntax-language syntax-python\">self</span><span class=\"syntax-punctuation syntax-accessor syntax-dot syntax-python\">.</span></span><span class=\"syntax-meta syntax-qualified-name syntax-python\"><span class=\"syntax-variable syntax-function syntax-python\">set_block_state</span></span><span class=\"syntax-punctuation syntax-section syntax-arguments syntax-begin syntax-python\">(</span><span class=\"syntax-meta syntax-function-call syntax-arguments syntax-python\"><span class=\"syntax-meta syntax-qualified-name syntax-python\"><span class=\"syntax-meta syntax-generic-name syntax-python\">state</span></span><span class=\"syntax-punctuation syntax-separator syntax-arguments syntax-python\">,</span> <span class=\"syntax-meta syntax-qualified-name syntax-python\"><span class=\"syntax-meta syntax-generic-name syntax-python\">block_state</span></span></span><span class=\"syntax-punctuation syntax-section syntax-arguments syntax-end syntax-python\">)</span></span>\n        <span class=\"syntax-keyword syntax-control syntax-flow syntax-return syntax-python\">return</span> <span class=\"syntax-meta syntax-qualified-name syntax-python\"><span class=\"syntax-meta syntax-generic-name syntax-python\">components</span></span>, <span class=\"syntax-meta syntax-qualified-name syntax-python\"><span class=\"syntax-meta syntax-generic-name syntax-python\">state</span></span>\n</span></span></code></pre>\n<ul>\n<li><code>expected_components</code> defines what models the block needs — in this case, a depth estimation model. The <code>pretrained_model_name_or_path</code> parameter sets a default Hub repo to load from, so <code>load_components</code> automatically fetches the depth model unless you override it in <code>modular_model_index.json</code>.</li>\n<li><code>inputs</code> and <code>intermediate_outputs</code> define what goes in and comes out.</li>\n<li><code>__call__</code> is where the computation logic lives.</li>\n</ul>\n<h3>Composing Blocks into Workflows</h3>\n<p>Let's use this block with Qwen's ControlNet workflow. Extract the ControlNet workflow and insert the depth block at the beginning:</p>\n<pre><code class=\"language-python\"><span class=\"code-snippet\" data-language=\"Python\"><span class=\"syntax-source syntax-python\"><span class=\"syntax-comment syntax-line syntax-number-sign syntax-python\"><span class=\"syntax-punctuation syntax-definition syntax-comment syntax-python\">#</span> Create Qwen Image pipeline\n</span><span class=\"syntax-meta syntax-qualified-name syntax-python\"><span class=\"syntax-meta syntax-generic-name syntax-python\">pipe</span></span> <span class=\"syntax-keyword syntax-operator syntax-assignment syntax-python\">=</span> <span class=\"syntax-meta syntax-function-call syntax-python\"><span class=\"syntax-meta syntax-qualified-name syntax-python\"><span class=\"syntax-meta syntax-generic-name syntax-python\">ModularPipeline</span><span class=\"syntax-punctuation syntax-accessor syntax-dot syntax-python\">.</span></span><span class=\"syntax-meta syntax-qualified-name syntax-python\"><span class=\"syntax-variable syntax-function syntax-python\">from_pretrained</span></span><span class=\"syntax-punctuation syntax-section syntax-arguments syntax-begin syntax-python\">(</span><span class=\"syntax-meta syntax-function-call syntax-arguments syntax-python\"><span class=\"syntax-meta syntax-string syntax-python\"><span class=\"syntax-string syntax-quoted syntax-double syntax-python\"><span class=\"syntax-punctuation syntax-definition syntax-string syntax-begin syntax-python\">&quot;</span></span></span><span class=\"syntax-meta syntax-string syntax-python\"><span class=\"syntax-string syntax-quoted syntax-double syntax-python\">Qwen/Qwen-Image<span class=\"syntax-punctuation syntax-definition syntax-string syntax-end syntax-python\">&quot;</span></span></span></span><span class=\"syntax-punctuation syntax-section syntax-arguments syntax-end syntax-python\">)</span></span>\n\n<span class=\"syntax-meta syntax-function-call syntax-python\"><span class=\"syntax-meta syntax-qualified-name syntax-python\"><span class=\"syntax-support syntax-function syntax-builtin syntax-python\">print</span></span><span class=\"syntax-punctuation syntax-section syntax-arguments syntax-begin syntax-python\">(</span><span class=\"syntax-meta syntax-function-call syntax-arguments syntax-python\"><span class=\"syntax-meta syntax-qualified-name syntax-python\"><span class=\"syntax-meta syntax-generic-name syntax-python\">pipe</span><span class=\"syntax-punctuation syntax-accessor syntax-dot syntax-python\">.</span><span class=\"syntax-meta syntax-generic-name syntax-python\">blocks</span><span class=\"syntax-punctuation syntax-accessor syntax-dot syntax-python\">.</span><span class=\"syntax-meta syntax-generic-name syntax-python\">available_workflows</span></span></span><span class=\"syntax-punctuation syntax-section syntax-arguments syntax-end syntax-python\">)</span></span>\n<span class=\"syntax-comment syntax-line syntax-number-sign syntax-python\"><span class=\"syntax-punctuation syntax-definition syntax-comment syntax-python\">#</span>       Supported workflows:\n</span><span class=\"syntax-comment syntax-line syntax-number-sign syntax-python\"><span class=\"syntax-punctuation syntax-definition syntax-comment syntax-python\">#</span>        - `text2image`: requires `prompt`\n</span><span class=\"syntax-comment syntax-line syntax-number-sign syntax-python\"><span class=\"syntax-punctuation syntax-definition syntax-comment syntax-python\">#</span>        - `image2image`: requires `prompt`, `image`\n</span><span class=\"syntax-comment syntax-line syntax-number-sign syntax-python\"><span class=\"syntax-punctuation syntax-definition syntax-comment syntax-python\">#</span>        - `inpainting`: requires `prompt`, `mask_image`, `image`\n</span><span class=\"syntax-comment syntax-line syntax-number-sign syntax-python\"><span class=\"syntax-punctuation syntax-definition syntax-comment syntax-python\">#</span>        - `controlnet_text2image`: requires `prompt`, `control_image`\n</span><span class=\"syntax-comment syntax-line syntax-number-sign syntax-python\"><span class=\"syntax-punctuation syntax-definition syntax-comment syntax-python\">#</span>        - `controlnet_image2image`: requires `prompt`, `image`, `control_image`\n</span>\n<span class=\"syntax-comment syntax-line syntax-number-sign syntax-python\"><span class=\"syntax-punctuation syntax-definition syntax-comment syntax-python\">#</span> Extract the ControlNet workflow — it expects a control_image input\n</span><span class=\"syntax-meta syntax-qualified-name syntax-python\"><span class=\"syntax-meta syntax-generic-name syntax-python\">blocks</span></span> <span class=\"syntax-keyword syntax-operator syntax-assignment syntax-python\">=</span> <span class=\"syntax-meta syntax-function-call syntax-python\"><span class=\"syntax-meta syntax-qualified-name syntax-python\"><span class=\"syntax-meta syntax-generic-name syntax-python\">pipe</span><span class=\"syntax-punctuation syntax-accessor syntax-dot syntax-python\">.</span><span class=\"syntax-meta syntax-generic-name syntax-python\">blocks</span><span class=\"syntax-punctuation syntax-accessor syntax-dot syntax-python\">.</span></span><span class=\"syntax-meta syntax-qualified-name syntax-python\"><span class=\"syntax-variable syntax-function syntax-python\">get_workflow</span></span><span class=\"syntax-punctuation syntax-section syntax-arguments syntax-begin syntax-python\">(</span><span class=\"syntax-meta syntax-function-call syntax-arguments syntax-python\"><span class=\"syntax-meta syntax-string syntax-python\"><span class=\"syntax-string syntax-quoted syntax-double syntax-python\"><span class=\"syntax-punctuation syntax-definition syntax-string syntax-begin syntax-python\">&quot;</span></span></span><span class=\"syntax-meta syntax-string syntax-python\"><span class=\"syntax-string syntax-quoted syntax-double syntax-python\">controlnet_text2image<span class=\"syntax-punctuation syntax-definition syntax-string syntax-end syntax-python\">&quot;</span></span></span></span><span class=\"syntax-punctuation syntax-section syntax-arguments syntax-end syntax-python\">)</span></span>\n<span class=\"syntax-comment syntax-line syntax-number-sign syntax-python\"><span class=\"syntax-punctuation syntax-definition syntax-comment syntax-python\">#</span> Show the blocks this workflow uses\n</span><span class=\"syntax-meta syntax-function-call syntax-python\"><span class=\"syntax-meta syntax-qualified-name syntax-python\"><span class=\"syntax-support syntax-function syntax-builtin syntax-python\">print</span></span><span class=\"syntax-punctuation syntax-section syntax-arguments syntax-begin syntax-python\">(</span><span class=\"syntax-meta syntax-function-call syntax-arguments syntax-python\"><span class=\"syntax-meta syntax-qualified-name syntax-python\"><span class=\"syntax-meta syntax-generic-name syntax-python\">blocks</span></span></span><span class=\"syntax-punctuation syntax-section syntax-arguments syntax-end syntax-python\">)</span></span>\n\n<span class=\"syntax-comment syntax-line syntax-number-sign syntax-python\"><span class=\"syntax-punctuation syntax-definition syntax-comment syntax-python\">#</span> Insert depth block at the beginning — its output (control_image)\n</span><span class=\"syntax-comment syntax-line syntax-number-sign syntax-python\"><span class=\"syntax-punctuation syntax-definition syntax-comment syntax-python\">#</span> automatically flows to the ControlNet block that needs it\n</span><span class=\"syntax-meta syntax-function-call syntax-python\"><span class=\"syntax-meta syntax-qualified-name syntax-python\"><span class=\"syntax-meta syntax-generic-name syntax-python\">blocks</span><span class=\"syntax-punctuation syntax-accessor syntax-dot syntax-python\">.</span><span class=\"syntax-meta syntax-generic-name syntax-python\">sub_blocks</span><span class=\"syntax-punctuation syntax-accessor syntax-dot syntax-python\">.</span></span><span class=\"syntax-meta syntax-qualified-name syntax-python\"><span class=\"syntax-variable syntax-function syntax-python\">insert</span></span><span class=\"syntax-punctuation syntax-section syntax-arguments syntax-begin syntax-python\">(</span><span class=\"syntax-meta syntax-function-call syntax-arguments syntax-python\"><span class=\"syntax-meta syntax-string syntax-python\"><span class=\"syntax-string syntax-quoted syntax-double syntax-python\"><span class=\"syntax-punctuation syntax-definition syntax-string syntax-begin syntax-python\">&quot;</span></span></span><span class=\"syntax-meta syntax-string syntax-python\"><span class=\"syntax-string syntax-quoted syntax-double syntax-python\">depth<span class=\"syntax-punctuation syntax-definition syntax-string syntax-end syntax-python\">&quot;</span></span></span><span class=\"syntax-punctuation syntax-separator syntax-arguments syntax-python\">,</span> <span class=\"syntax-meta syntax-function-call syntax-python\"><span class=\"syntax-meta syntax-qualified-name syntax-python\"><span class=\"syntax-variable syntax-function syntax-python\">DepthProcessorBlock</span></span><span class=\"syntax-punctuation syntax-section syntax-arguments syntax-begin syntax-python\">(</span><span class=\"syntax-punctuation syntax-section syntax-arguments syntax-end syntax-python\">)</span></span><span class=\"syntax-punctuation syntax-separator syntax-arguments syntax-python\">,</span> <span class=\"syntax-constant syntax-numeric syntax-integer syntax-decimal syntax-python\">0</span></span><span class=\"syntax-punctuation syntax-section syntax-arguments syntax-end syntax-python\">)</span></span>\n\n<span class=\"syntax-comment syntax-line syntax-number-sign syntax-python\"><span class=\"syntax-punctuation syntax-definition syntax-comment syntax-python\">#</span> You can inspect any block&#39;s inputs and outputs with print(blocks.doc)\n</span><span class=\"syntax-meta syntax-item-access syntax-python\"><span class=\"syntax-meta syntax-qualified-name syntax-python\"><span class=\"syntax-meta syntax-generic-name syntax-python\">blocks</span><span class=\"syntax-punctuation syntax-accessor syntax-dot syntax-python\">.</span><span class=\"syntax-meta syntax-generic-name syntax-python\">sub_blocks</span></span></span><span class=\"syntax-meta syntax-item-access syntax-python\"><span class=\"syntax-punctuation syntax-section syntax-brackets syntax-begin syntax-python\">[</span></span><span class=\"syntax-meta syntax-item-access syntax-arguments syntax-python\"><span class=\"syntax-meta syntax-string syntax-python\"><span class=\"syntax-string syntax-quoted syntax-single syntax-python\"><span class=\"syntax-punctuation syntax-definition syntax-string syntax-begin syntax-python\">&#39;</span></span></span><span class=\"syntax-meta syntax-string syntax-python\"><span class=\"syntax-string syntax-quoted syntax-single syntax-python\">depth<span class=\"syntax-punctuation syntax-definition syntax-string syntax-end syntax-python\">&#39;</span></span></span></span><span class=\"syntax-meta syntax-item-access syntax-python\"><span class=\"syntax-punctuation syntax-section syntax-brackets syntax-end syntax-python\">]</span></span><span class=\"syntax-punctuation syntax-accessor syntax-dot syntax-python\">.</span><span class=\"syntax-meta syntax-generic-name syntax-python\">doc</span>\n</span></span></code></pre>\n<p>Blocks in a sequence share data automatically: the depth block's <code>control_image</code> output flows to downstream blocks that need it, and its <code>image</code> input becomes a pipeline input since no earlier block provides it.</p>\n<p><picture class=\"article-picture article-picture-fallback\" style=\"--image-ratio:16 / 9\"><img src=\"https://huggingface.co/datasets/diffusers/modular-diffusers-blog/resolve/main/blocks_composed.png\" alt=\"blocks_composed\" loading=\"eager\" decoding=\"async\" fetchpriority=\"high\" referrerpolicy=\"no-referrer\" class=\"progressive-image\"></picture></p>\n<pre><code class=\"language-python\"><span class=\"code-snippet\" data-language=\"Python\"><span class=\"syntax-source syntax-python\"><span class=\"syntax-meta syntax-statement syntax-import syntax-python\"><span class=\"syntax-keyword syntax-control syntax-import syntax-from syntax-python\">from</span></span><span class=\"syntax-meta syntax-statement syntax-import syntax-python\"><span class=\"syntax-meta syntax-import-source syntax-python\"> <span class=\"syntax-meta syntax-import-path syntax-python\"><span class=\"syntax-meta syntax-import-name syntax-python\">diffusers</span></span> <span class=\"syntax-meta syntax-statement syntax-import syntax-python\"><span class=\"syntax-keyword syntax-control syntax-import syntax-python\">import</span></span></span></span><span class=\"syntax-meta syntax-statement syntax-import syntax-python\"></span><span class=\"syntax-meta syntax-statement syntax-import syntax-python\"> <span class=\"syntax-meta syntax-generic-name syntax-python\">ComponentsManager</span><span class=\"syntax-punctuation syntax-separator syntax-import-list syntax-python\">,</span> <span class=\"syntax-meta syntax-generic-name syntax-python\">AutoModel</span></span>\n<span class=\"syntax-meta syntax-statement syntax-import syntax-python\"><span class=\"syntax-keyword syntax-control syntax-import syntax-from syntax-python\">from</span></span><span class=\"syntax-meta syntax-statement syntax-import syntax-python\"><span class=\"syntax-meta syntax-import-source syntax-python\"> <span class=\"syntax-meta syntax-import-path syntax-python\"><span class=\"syntax-meta syntax-import-name syntax-python\">diffusers</span><span class=\"syntax-punctuation syntax-accessor syntax-dot syntax-python\">.</span><span class=\"syntax-meta syntax-import-name syntax-python\">utils</span></span> <span class=\"syntax-meta syntax-statement syntax-import syntax-python\"><span class=\"syntax-keyword syntax-control syntax-import syntax-python\">import</span></span></span></span><span class=\"syntax-meta syntax-statement syntax-import syntax-python\"></span><span class=\"syntax-meta syntax-statement syntax-import syntax-python\"> <span class=\"syntax-meta syntax-generic-name syntax-python\">load_image</span></span>\n\n<span class=\"syntax-comment syntax-line syntax-number-sign syntax-python\"><span class=\"syntax-punctuation syntax-definition syntax-comment syntax-python\">#</span> ComponentsManager handles memory across multiple pipelines —\n</span><span class=\"syntax-comment syntax-line syntax-number-sign syntax-python\"><span class=\"syntax-punctuation syntax-definition syntax-comment syntax-python\">#</span> it automatically offloads models to CPU when not in use\n</span><span class=\"syntax-meta syntax-qualified-name syntax-python\"><span class=\"syntax-meta syntax-generic-name syntax-python\">manager</span></span> <span class=\"syntax-keyword syntax-operator syntax-assignment syntax-python\">=</span> <span class=\"syntax-meta syntax-function-call syntax-python\"><span class=\"syntax-meta syntax-qualified-name syntax-python\"><span class=\"syntax-variable syntax-function syntax-python\">ComponentsManager</span></span><span class=\"syntax-punctuation syntax-section syntax-arguments syntax-begin syntax-python\">(</span><span class=\"syntax-punctuation syntax-section syntax-arguments syntax-end syntax-python\">)</span></span>\n\n<span class=\"syntax-meta syntax-qualified-name syntax-python\"><span class=\"syntax-meta syntax-generic-name syntax-python\">pipeline</span></span> <span class=\"syntax-keyword syntax-operator syntax-assignment syntax-python\">=</span> <span class=\"syntax-meta syntax-function-call syntax-python\"><span class=\"syntax-meta syntax-qualified-name syntax-python\"><span class=\"syntax-meta syntax-generic-name syntax-python\">blocks</span><span class=\"syntax-punctuation syntax-accessor syntax-dot syntax-python\">.</span></span><span class=\"syntax-meta syntax-qualified-name syntax-python\"><span class=\"syntax-variable syntax-function syntax-python\">init_pipeline</span></span><span class=\"syntax-punctuation syntax-section syntax-arguments syntax-begin syntax-python\">(</span><span class=\"syntax-meta syntax-function-call syntax-arguments syntax-python\"><span class=\"syntax-meta syntax-string syntax-python\"><span class=\"syntax-string syntax-quoted syntax-double syntax-python\"><span class=\"syntax-punctuation syntax-definition syntax-string syntax-begin syntax-python\">&quot;</span></span></span><span class=\"syntax-meta syntax-string syntax-python\"><span class=\"syntax-string syntax-quoted syntax-double syntax-python\">Qwen/Qwen-Image<span class=\"syntax-punctuation syntax-definition syntax-string syntax-end syntax-python\">&quot;</span></span></span><span class=\"syntax-punctuation syntax-separator syntax-arguments syntax-python\">,</span> <span class=\"syntax-variable syntax-parameter syntax-python\">components_manager</span><span class=\"syntax-keyword syntax-operator syntax-assignment syntax-python\">=</span><span class=\"syntax-meta syntax-qualified-name syntax-python\"><span class=\"syntax-meta syntax-generic-name syntax-python\">manager</span></span></span><span class=\"syntax-punctuation syntax-section syntax-arguments syntax-end syntax-python\">)</span></span>\n<span class=\"syntax-meta syntax-function-call syntax-python\"><span class=\"syntax-meta syntax-qualified-name syntax-python\"><span class=\"syntax-meta syntax-generic-name syntax-python\">pipeline</span><span class=\"syntax-punctuation syntax-accessor syntax-dot syntax-python\">.</span></span><span class=\"syntax-meta syntax-qualified-name syntax-python\"><span class=\"syntax-variable syntax-function syntax-python\">load_components</span></span><span class=\"syntax-punctuation syntax-section syntax-arguments syntax-begin syntax-python\">(</span><span class=\"syntax-meta syntax-function-call syntax-arguments syntax-python\"><span class=\"syntax-variable syntax-parameter syntax-python\">torch_dtype</span><span class=\"syntax-keyword syntax-operator syntax-assignment syntax-python\">=</span><span class=\"syntax-meta syntax-qualified-name syntax-python\"><span class=\"syntax-meta syntax-generic-name syntax-python\">torch</span><span class=\"syntax-punctuation syntax-accessor syntax-dot syntax-python\">.</span><span class=\"syntax-meta syntax-generic-name syntax-python\">bfloat16</span></span></span><span class=\"syntax-punctuation syntax-section syntax-arguments syntax-end syntax-python\">)</span></span>\n\n<span class=\"syntax-comment syntax-line syntax-number-sign syntax-python\"><span class=\"syntax-punctuation syntax-definition syntax-comment syntax-python\">#</span> The depth model loads automatically from the default path we set in expected_components —\n</span><span class=\"syntax-comment syntax-line syntax-number-sign syntax-python\"><span class=\"syntax-punctuation syntax-definition syntax-comment syntax-python\">#</span> no need to load it manually even though it&#39;s not part of the Qwen repo.\n</span><span class=\"syntax-comment syntax-line syntax-number-sign syntax-python\"><span class=\"syntax-punctuation syntax-definition syntax-comment syntax-python\">#</span> But controlnet is not included by default, so we do need to load it from a different repo\n</span><span class=\"syntax-meta syntax-qualified-name syntax-python\"><span class=\"syntax-meta syntax-generic-name syntax-python\">controlnet</span></span> <span class=\"syntax-keyword syntax-operator syntax-assignment syntax-python\">=</span> <span class=\"syntax-meta syntax-function-call syntax-python\"><span class=\"syntax-meta syntax-qualified-name syntax-python\"><span class=\"syntax-meta syntax-generic-name syntax-python\">AutoModel</span><span class=\"syntax-punctuation syntax-accessor syntax-dot syntax-python\">.</span></span><span class=\"syntax-meta syntax-qualified-name syntax-python\"><span class=\"syntax-variable syntax-function syntax-python\">from_pretrained</span></span><span class=\"syntax-punctuation syntax-section syntax-arguments syntax-begin syntax-python\">(</span><span class=\"syntax-meta syntax-function-call syntax-arguments syntax-python\"><span class=\"syntax-meta syntax-string syntax-python\"><span class=\"syntax-string syntax-quoted syntax-double syntax-python\"><span class=\"syntax-punctuation syntax-definition syntax-string syntax-begin syntax-python\">&quot;</span></span></span><span class=\"syntax-meta syntax-string syntax-python\"><span class=\"syntax-string syntax-quoted syntax-double syntax-python\">InstantX/Qwen-Image-ControlNet-Union<span class=\"syntax-punctuation syntax-definition syntax-string syntax-end syntax-python\">&quot;</span></span></span><span class=\"syntax-punctuation syntax-separator syntax-arguments syntax-python\">,</span> <span class=\"syntax-variable syntax-parameter syntax-python\">torch_dtype</span><span class=\"syntax-keyword syntax-operator syntax-assignment syntax-python\">=</span><span class=\"syntax-meta syntax-qualified-name syntax-python\"><span class=\"syntax-meta syntax-generic-name syntax-python\">torch</span><span class=\"syntax-punctuation syntax-accessor syntax-dot syntax-python\">.</span><span class=\"syntax-meta syntax-generic-name syntax-python\">bfloat16</span></span></span><span class=\"syntax-punctuation syntax-section syntax-arguments syntax-end syntax-python\">)</span></span>\n<span class=\"syntax-meta syntax-function-call syntax-python\"><span class=\"syntax-meta syntax-qualified-name syntax-python\"><span class=\"syntax-meta syntax-generic-name syntax-python\">pipeline</span><span class=\"syntax-punctuation syntax-accessor syntax-dot syntax-python\">.</span></span><span class=\"syntax-meta syntax-qualified-name syntax-python\"><span class=\"syntax-variable syntax-function syntax-python\">update_components</span></span><span class=\"syntax-punctuation syntax-section syntax-arguments syntax-begin syntax-python\">(</span><span class=\"syntax-meta syntax-function-call syntax-arguments syntax-python\"><span class=\"syntax-variable syntax-parameter syntax-python\">controlnet</span><span class=\"syntax-keyword syntax-operator syntax-assignment syntax-python\">=</span><span class=\"syntax-meta syntax-qualified-name syntax-python\"><span class=\"syntax-meta syntax-generic-name syntax-python\">controlnet</span></span></span><span class=\"syntax-punctuation syntax-section syntax-arguments syntax-end syntax-python\">)</span></span>\n\n<span class=\"syntax-comment syntax-line syntax-number-sign syntax-python\"><span class=\"syntax-punctuation syntax-definition syntax-comment syntax-python\">#</span> pipeline now takes image as input\n</span><span class=\"syntax-meta syntax-qualified-name syntax-python\"><span class=\"syntax-meta syntax-generic-name syntax-python\">image</span></span> <span class=\"syntax-keyword syntax-operator syntax-assignment syntax-python\">=</span> <span class=\"syntax-meta syntax-function-call syntax-python\"><span class=\"syntax-meta syntax-qualified-name syntax-python\"><span class=\"syntax-variable syntax-function syntax-python\">load_image</span></span><span class=\"syntax-punctuation syntax-section syntax-arguments syntax-begin syntax-python\">(</span><span class=\"syntax-meta syntax-function-call syntax-arguments syntax-python\"><span class=\"syntax-meta syntax-string syntax-python\"><span class=\"syntax-string syntax-quoted syntax-double syntax-python\"><span class=\"syntax-punctuation syntax-definition syntax-string syntax-begin syntax-python\">&quot;</span></span></span><span class=\"syntax-meta syntax-string syntax-python\"><span class=\"syntax-string syntax-quoted syntax-double syntax-python\">https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/diffusers/astronaut.jpg<span class=\"syntax-punctuation syntax-definition syntax-string syntax-end syntax-python\">&quot;</span></span></span></span><span class=\"syntax-punctuation syntax-section syntax-arguments syntax-end syntax-python\">)</span></span>\n<span class=\"syntax-meta syntax-qualified-name syntax-python\"><span class=\"syntax-meta syntax-generic-name syntax-python\">output</span></span> <span class=\"syntax-keyword syntax-operator syntax-assignment syntax-python\">=</span> <span class=\"syntax-meta syntax-function-call syntax-python\"><span class=\"syntax-meta syntax-qualified-name syntax-python\"><span class=\"syntax-variable syntax-function syntax-python\">pipeline</span></span><span class=\"syntax-punctuation syntax-section syntax-arguments syntax-begin syntax-python\">(</span><span class=\"syntax-meta syntax-function-call syntax-arguments syntax-python\">\n    <span class=\"syntax-variable syntax-parameter syntax-python\">prompt</span><span class=\"syntax-keyword syntax-operator syntax-assignment syntax-python\">=</span><span class=\"syntax-meta syntax-string syntax-python\"><span class=\"syntax-string syntax-quoted syntax-double syntax-python\"><span class=\"syntax-punctuation syntax-definition syntax-string syntax-begin syntax-python\">&quot;</span></span></span><span class=\"syntax-meta syntax-string syntax-python\"><span class=\"syntax-string syntax-quoted syntax-double syntax-python\">an astronaut hatching from an egg, detailed, fantasy, Pixar, Disney<span class=\"syntax-punctuation syntax-definition syntax-string syntax-end syntax-python\">&quot;</span></span></span><span class=\"syntax-punctuation syntax-separator syntax-arguments syntax-python\">,</span>\n    <span class=\"syntax-variable syntax-parameter syntax-python\">image</span><span class=\"syntax-keyword syntax-operator syntax-assignment syntax-python\">=</span><span class=\"syntax-meta syntax-qualified-name syntax-python\"><span class=\"syntax-meta syntax-generic-name syntax-python\">image</span></span><span class=\"syntax-punctuation syntax-separator syntax-arguments syntax-python\">,</span>\n</span><span class=\"syntax-punctuation syntax-section syntax-arguments syntax-end syntax-python\">)</span></span><span class=\"syntax-punctuation syntax-accessor syntax-dot syntax-python\">.</span><span class=\"syntax-meta syntax-generic-name syntax-python\">images</span><span class=\"syntax-meta syntax-item-access syntax-python\"><span class=\"syntax-punctuation syntax-section syntax-brackets syntax-begin syntax-python\">[</span></span><span class=\"syntax-meta syntax-item-access syntax-arguments syntax-python\"><span class=\"syntax-constant syntax-numeric syntax-integer syntax-decimal syntax-python\">0</span></span><span class=\"syntax-meta syntax-item-access syntax-python\"><span class=\"syntax-punctuation syntax-section syntax-brackets syntax-end syntax-python\">]</span></span>\n</span></span></code></pre>\n<h3>Sharing Custom Blocks on the Hub</h3>\n<p>You can publish your custom block to the Hub so anyone can load it with <code>trust_remote_code=True</code>. We've created a <a target=\"_blank\" rel=\"noopener noreferrer\" href=\"https://huggingface.co/diffusers/custom-block-template\">template</a> to get you started — check out the <a target=\"_blank\" rel=\"noopener noreferrer\" href=\"https://huggingface.co/docs/diffusers/main/en/modular_diffusers/custom_blocks#quick-start-with-template\">Building Custom Blocks guide</a> for the full walkthrough.</p>\n<pre><code class=\"language-python\"><span class=\"code-snippet\" data-language=\"Python\"><span class=\"syntax-source syntax-python\"><span class=\"syntax-meta syntax-function-call syntax-python\"><span class=\"syntax-meta syntax-qualified-name syntax-python\"><span class=\"syntax-meta syntax-generic-name syntax-python\">pipeline</span><span class=\"syntax-punctuation syntax-accessor syntax-dot syntax-python\">.</span></span><span class=\"syntax-meta syntax-qualified-name syntax-python\"><span class=\"syntax-variable syntax-function syntax-python\">save_pretrained</span></span><span class=\"syntax-punctuation syntax-section syntax-arguments syntax-begin syntax-python\">(</span><span class=\"syntax-meta syntax-function-call syntax-arguments syntax-python\"><span class=\"syntax-meta syntax-qualified-name syntax-python\"><span class=\"syntax-meta syntax-generic-name syntax-python\">local_dir</span></span><span class=\"syntax-punctuation syntax-separator syntax-arguments syntax-python\">,</span> <span class=\"syntax-variable syntax-parameter syntax-python\">repo_id</span><span class=\"syntax-keyword syntax-operator syntax-assignment syntax-python\">=</span><span class=\"syntax-meta syntax-string syntax-python\"><span class=\"syntax-string syntax-quoted syntax-double syntax-python\"><span class=\"syntax-punctuation syntax-definition syntax-string syntax-begin syntax-python\">&quot;</span></span></span><span class=\"syntax-meta syntax-string syntax-python\"><span class=\"syntax-string syntax-quoted syntax-double syntax-python\">your-username/your-block-name<span class=\"syntax-punctuation syntax-definition syntax-string syntax-end syntax-python\">&quot;</span></span></span><span class=\"syntax-punctuation syntax-separator syntax-arguments syntax-python\">,</span> <span class=\"syntax-variable syntax-parameter syntax-python\">push_to_hub</span><span class=\"syntax-keyword syntax-operator syntax-assignment syntax-python\">=</span><span class=\"syntax-constant syntax-language syntax-python\">True</span></span><span class=\"syntax-punctuation syntax-section syntax-arguments syntax-end syntax-python\">)</span></span>\n</span></span></code></pre>\n<p>The <code>DepthProcessorBlock</code> from this post is published at <a target=\"_blank\" rel=\"noopener noreferrer\" href=\"https://huggingface.co/diffusers/depth-processor-custom-block\">diffusers/depth-processor-custom-block</a> — you can load and use it directly:</p>\n<pre><code class=\"language-python\"><span class=\"code-snippet\" data-language=\"Python\"><span class=\"syntax-source syntax-python\"><span class=\"syntax-meta syntax-statement syntax-import syntax-python\"><span class=\"syntax-keyword syntax-control syntax-import syntax-from syntax-python\">from</span></span><span class=\"syntax-meta syntax-statement syntax-import syntax-python\"><span class=\"syntax-meta syntax-import-source syntax-python\"> <span class=\"syntax-meta syntax-import-path syntax-python\"><span class=\"syntax-meta syntax-import-name syntax-python\">diffusers</span></span> <span class=\"syntax-meta syntax-statement syntax-import syntax-python\"><span class=\"syntax-keyword syntax-control syntax-import syntax-python\">import</span></span></span></span><span class=\"syntax-meta syntax-statement syntax-import syntax-python\"></span><span class=\"syntax-meta syntax-statement syntax-import syntax-python\"> <span class=\"syntax-meta syntax-generic-name syntax-python\">ModularPipelineBlocks</span></span>\n\n<span class=\"syntax-meta syntax-qualified-name syntax-python\"><span class=\"syntax-meta syntax-generic-name syntax-python\">depth_block</span></span> <span class=\"syntax-keyword syntax-operator syntax-assignment syntax-python\">=</span> <span class=\"syntax-meta syntax-function-call syntax-python\"><span class=\"syntax-meta syntax-qualified-name syntax-python\"><span class=\"syntax-meta syntax-generic-name syntax-python\">ModularPipelineBlocks</span><span class=\"syntax-punctuation syntax-accessor syntax-dot syntax-python\">.</span></span><span class=\"syntax-meta syntax-qualified-name syntax-python\"><span class=\"syntax-variable syntax-function syntax-python\">from_pretrained</span></span><span class=\"syntax-punctuation syntax-section syntax-arguments syntax-begin syntax-python\">(</span><span class=\"syntax-meta syntax-function-call syntax-arguments syntax-python\">\n    <span class=\"syntax-meta syntax-string syntax-python\"><span class=\"syntax-string syntax-quoted syntax-double syntax-python\"><span class=\"syntax-punctuation syntax-definition syntax-string syntax-begin syntax-python\">&quot;</span></span></span><span class=\"syntax-meta syntax-string syntax-python\"><span class=\"syntax-string syntax-quoted syntax-double syntax-python\">diffusers/depth-processor-custom-block<span class=\"syntax-punctuation syntax-definition syntax-string syntax-end syntax-python\">&quot;</span></span></span><span class=\"syntax-punctuation syntax-separator syntax-arguments syntax-python\">,</span> <span class=\"syntax-variable syntax-parameter syntax-python\">trust_remote_code</span><span class=\"syntax-keyword syntax-operator syntax-assignment syntax-python\">=</span><span class=\"syntax-constant syntax-language syntax-python\">True</span>\n</span><span class=\"syntax-punctuation syntax-section syntax-arguments syntax-end syntax-python\">)</span></span>\n</span></span></code></pre>\n<p>We've published a collection of ready-to-use custom blocks <a target=\"_blank\" rel=\"noopener noreferrer\" href=\"https://huggingface.co/collections/diffusers/modular-diffusers-custom-blocks\">here</a>.</p>\n<h2>Modular Repositories</h2>\n<p><code>ModularPipeline.from_pretrained</code> works with any existing Diffusers repo out of the box, but Modular Diffusers also introduces a new kind of repo: the Modular Repository.</p>\n<p>A modular repository is able to reference components from their original model repos. For example, <a target=\"_blank\" rel=\"noopener noreferrer\" href=\"https://huggingface.co/diffusers/flux2-bnb-4bit-modular\">diffusers/flux2-bnb-4bit-modular</a> contains a quantized transformer and loads the remaining components from the original repo.</p>\n<pre><code class=\"language-json\"><span class=\"code-snippet\" data-language=\"JSON\"><span class=\"syntax-source syntax-json\"><span class=\"syntax-comment syntax-line syntax-double-slash syntax-js\"><span class=\"syntax-punctuation syntax-definition syntax-comment syntax-json\">//</span> diffusers/flux2-bnb-4bit-modular/modular_model_index.json\n</span><span class=\"syntax-meta syntax-structure syntax-dictionary syntax-json\"><span class=\"syntax-punctuation syntax-section syntax-dictionary syntax-begin syntax-json\">{</span>\n    <span class=\"syntax-meta syntax-structure syntax-dictionary syntax-key syntax-json\"><span class=\"syntax-string syntax-quoted syntax-double syntax-json\"><span class=\"syntax-punctuation syntax-definition syntax-string syntax-begin syntax-json\">&quot;</span>transformer<span class=\"syntax-punctuation syntax-definition syntax-string syntax-end syntax-json\">&quot;</span></span></span><span class=\"syntax-meta syntax-structure syntax-dictionary syntax-value syntax-json\"><span class=\"syntax-punctuation syntax-separator syntax-dictionary syntax-key-value syntax-json\">:</span> <span class=\"syntax-meta syntax-structure syntax-array syntax-json\"><span class=\"syntax-punctuation syntax-section syntax-array syntax-begin syntax-json\">[</span>\n        <span class=\"syntax-string syntax-quoted syntax-double syntax-json\"><span class=\"syntax-punctuation syntax-definition syntax-string syntax-begin syntax-json\">&quot;</span>diffusers<span class=\"syntax-punctuation syntax-definition syntax-string syntax-end syntax-json\">&quot;</span></span><span class=\"syntax-punctuation syntax-separator syntax-array syntax-json\">,</span> \n        <span class=\"syntax-string syntax-quoted syntax-double syntax-json\"><span class=\"syntax-punctuation syntax-definition syntax-string syntax-begin syntax-json\">&quot;</span>Flux2Transformer2DModel<span class=\"syntax-punctuation syntax-definition syntax-string syntax-end syntax-json\">&quot;</span></span><span class=\"syntax-punctuation syntax-separator syntax-array syntax-json\">,</span> \n        <span class=\"syntax-meta syntax-structure syntax-dictionary syntax-json\"><span class=\"syntax-punctuation syntax-section syntax-dictionary syntax-begin syntax-json\">{</span>\n            <span class=\"syntax-meta syntax-structure syntax-dictionary syntax-key syntax-json\"><span class=\"syntax-string syntax-quoted syntax-double syntax-json\"><span class=\"syntax-punctuation syntax-definition syntax-string syntax-begin syntax-json\">&quot;</span>pretrained_model_name_or_path<span class=\"syntax-punctuation syntax-definition syntax-string syntax-end syntax-json\">&quot;</span></span></span><span class=\"syntax-meta syntax-structure syntax-dictionary syntax-value syntax-json\"><span class=\"syntax-punctuation syntax-separator syntax-dictionary syntax-key-value syntax-json\">:</span> <span class=\"syntax-string syntax-quoted syntax-double syntax-json\"><span class=\"syntax-punctuation syntax-definition syntax-string syntax-begin syntax-json\">&quot;</span>diffusers/flux2-bnb-4bit-modular<span class=\"syntax-punctuation syntax-definition syntax-string syntax-end syntax-json\">&quot;</span></span><span class=\"syntax-punctuation syntax-separator syntax-dictionary syntax-pair syntax-json\">,</span></span>\n            <span class=\"syntax-meta syntax-structure syntax-dictionary syntax-key syntax-json\"><span class=\"syntax-string syntax-quoted syntax-double syntax-json\"><span class=\"syntax-punctuation syntax-definition syntax-string syntax-begin syntax-json\">&quot;</span>subfolder<span class=\"syntax-punctuation syntax-definition syntax-string syntax-end syntax-json\">&quot;</span></span></span><span class=\"syntax-meta syntax-structure syntax-dictionary syntax-value syntax-json\"><span class=\"syntax-punctuation syntax-separator syntax-dictionary syntax-key-value syntax-json\">:</span> <span class=\"syntax-string syntax-quoted syntax-double syntax-json\"><span class=\"syntax-punctuation syntax-definition syntax-string syntax-begin syntax-json\">&quot;</span>transformer<span class=\"syntax-punctuation syntax-definition syntax-string syntax-end syntax-json\">&quot;</span></span><span class=\"syntax-punctuation syntax-separator syntax-dictionary syntax-pair syntax-json\">,</span></span>\n            <span class=\"syntax-meta syntax-structure syntax-dictionary syntax-key syntax-json\"><span class=\"syntax-string syntax-quoted syntax-double syntax-json\"><span class=\"syntax-punctuation syntax-definition syntax-string syntax-begin syntax-json\">&quot;</span>type_hint<span class=\"syntax-punctuation syntax-definition syntax-string syntax-end syntax-json\">&quot;</span></span></span><span class=\"syntax-meta syntax-structure syntax-dictionary syntax-value syntax-json\"><span class=\"syntax-punctuation syntax-separator syntax-dictionary syntax-key-value syntax-json\">:</span> <span class=\"syntax-meta syntax-structure syntax-array syntax-json\"><span class=\"syntax-punctuation syntax-section syntax-array syntax-begin syntax-json\">[</span><span class=\"syntax-string syntax-quoted syntax-double syntax-json\"><span class=\"syntax-punctuation syntax-definition syntax-string syntax-begin syntax-json\">&quot;</span>diffusers<span class=\"syntax-punctuation syntax-definition syntax-string syntax-end syntax-json\">&quot;</span></span><span class=\"syntax-punctuation syntax-separator syntax-array syntax-json\">,</span> <span class=\"syntax-string syntax-quoted syntax-double syntax-json\"><span class=\"syntax-punctuation syntax-definition syntax-string syntax-begin syntax-json\">&quot;</span>Flux2Transformer2DModel<span class=\"syntax-punctuation syntax-definition syntax-string syntax-end syntax-json\">&quot;</span></span><span class=\"syntax-punctuation syntax-section syntax-array syntax-end syntax-json\">]</span></span>\n        </span><span class=\"syntax-punctuation syntax-section syntax-dictionary syntax-end syntax-json\">}</span></span>\n    <span class=\"syntax-punctuation syntax-section syntax-array syntax-end syntax-json\">]</span></span><span class=\"syntax-punctuation syntax-separator syntax-dictionary syntax-pair syntax-json\">,</span></span>\n    <span class=\"syntax-meta syntax-structure syntax-dictionary syntax-key syntax-json\"><span class=\"syntax-string syntax-quoted syntax-double syntax-json\"><span class=\"syntax-punctuation syntax-definition syntax-string syntax-begin syntax-json\">&quot;</span>vae<span class=\"syntax-punctuation syntax-definition syntax-string syntax-end syntax-json\">&quot;</span></span></span><span class=\"syntax-meta syntax-structure syntax-dictionary syntax-value syntax-json\"><span class=\"syntax-punctuation syntax-separator syntax-dictionary syntax-key-value syntax-json\">:</span> <span class=\"syntax-meta syntax-structure syntax-array syntax-json\"><span class=\"syntax-punctuation syntax-section syntax-array syntax-begin syntax-json\">[</span>\n        <span class=\"syntax-string syntax-quoted syntax-double syntax-json\"><span class=\"syntax-punctuation syntax-definition syntax-string syntax-begin syntax-json\">&quot;</span>diffusers<span class=\"syntax-punctuation syntax-definition syntax-string syntax-end syntax-json\">&quot;</span></span><span class=\"syntax-punctuation syntax-separator syntax-array syntax-json\">,</span> \n        <span class=\"syntax-string syntax-quoted syntax-double syntax-json\"><span class=\"syntax-punctuation syntax-definition syntax-string syntax-begin syntax-json\">&quot;</span>AutoencoderKLFlux2<span class=\"syntax-punctuation syntax-definition syntax-string syntax-end syntax-json\">&quot;</span></span><span class=\"syntax-punctuation syntax-separator syntax-array syntax-json\">,</span> \n        <span class=\"syntax-meta syntax-structure syntax-dictionary syntax-json\"><span class=\"syntax-punctuation syntax-section syntax-dictionary syntax-begin syntax-json\">{</span>\n            <span class=\"syntax-meta syntax-structure syntax-dictionary syntax-key syntax-json\"><span class=\"syntax-string syntax-quoted syntax-double syntax-json\"><span class=\"syntax-punctuation syntax-definition syntax-string syntax-begin syntax-json\">&quot;</span>pretrained_model_name_or_path<span class=\"syntax-punctuation syntax-definition syntax-string syntax-end syntax-json\">&quot;</span></span></span><span class=\"syntax-meta syntax-structure syntax-dictionary syntax-value syntax-json\"><span class=\"syntax-punctuation syntax-separator syntax-dictionary syntax-key-value syntax-json\">:</span> <span class=\"syntax-string syntax-quoted syntax-double syntax-json\"><span class=\"syntax-punctuation syntax-definition syntax-string syntax-begin syntax-json\">&quot;</span>black-forest-labs/FLUX.2-dev<span class=\"syntax-punctuation syntax-definition syntax-string syntax-end syntax-json\">&quot;</span></span><span class=\"syntax-punctuation syntax-separator syntax-dictionary syntax-pair syntax-json\">,</span></span>\n            <span class=\"syntax-meta syntax-structure syntax-dictionary syntax-key syntax-json\"><span class=\"syntax-string syntax-quoted syntax-double syntax-json\"><span class=\"syntax-punctuation syntax-definition syntax-string syntax-begin syntax-json\">&quot;</span>subfolder<span class=\"syntax-punctuation syntax-definition syntax-string syntax-end syntax-json\">&quot;</span></span></span><span class=\"syntax-meta syntax-structure syntax-dictionary syntax-value syntax-json\"><span class=\"syntax-punctuation syntax-separator syntax-dictionary syntax-key-value syntax-json\">:</span> <span class=\"syntax-string syntax-quoted syntax-double syntax-json\"><span class=\"syntax-punctuation syntax-definition syntax-string syntax-begin syntax-json\">&quot;</span>vae<span class=\"syntax-punctuation syntax-definition syntax-string syntax-end syntax-json\">&quot;</span></span><span class=\"syntax-punctuation syntax-separator syntax-dictionary syntax-pair syntax-json\">,</span></span>\n            <span class=\"syntax-meta syntax-structure syntax-dictionary syntax-key syntax-json\"><span class=\"syntax-string syntax-quoted syntax-double syntax-json\"><span class=\"syntax-punctuation syntax-definition syntax-string syntax-begin syntax-json\">&quot;</span>type_hint<span class=\"syntax-punctuation syntax-definition syntax-string syntax-end syntax-json\">&quot;</span></span></span><span class=\"syntax-meta syntax-structure syntax-dictionary syntax-value syntax-json\"><span class=\"syntax-punctuation syntax-separator syntax-dictionary syntax-key-value syntax-json\">:</span> <span class=\"syntax-meta syntax-structure syntax-array syntax-json\"><span class=\"syntax-punctuation syntax-section syntax-array syntax-begin syntax-json\">[</span><span class=\"syntax-string syntax-quoted syntax-double syntax-json\"><span class=\"syntax-punctuation syntax-definition syntax-string syntax-begin syntax-json\">&quot;</span>diffusers<span class=\"syntax-punctuation syntax-definition syntax-string syntax-end syntax-json\">&quot;</span></span><span class=\"syntax-punctuation syntax-separator syntax-array syntax-json\">,</span> <span class=\"syntax-string syntax-quoted syntax-double syntax-json\"><span class=\"syntax-punctuation syntax-definition syntax-string syntax-begin syntax-json\">&quot;</span>AutoencoderKLFlux2<span class=\"syntax-punctuation syntax-definition syntax-string syntax-end syntax-json\">&quot;</span></span><span class=\"syntax-punctuation syntax-section syntax-array syntax-end syntax-json\">]</span></span>\n        </span><span class=\"syntax-punctuation syntax-section syntax-dictionary syntax-end syntax-json\">}</span></span>\n    <span class=\"syntax-punctuation syntax-section syntax-array syntax-end syntax-json\">]</span></span><span class=\"syntax-punctuation syntax-separator syntax-dictionary syntax-pair syntax-json\">,</span></span>\n    <span class=\"syntax-invalid syntax-illegal syntax-expected-dictionary-separator syntax-json\">.</span><span class=\"syntax-invalid syntax-illegal syntax-expected-dictionary-separator syntax-json\">.</span><span class=\"syntax-invalid syntax-illegal syntax-expected-dictionary-separator syntax-json\">.</span>\n<span class=\"syntax-punctuation syntax-section syntax-dictionary syntax-end syntax-json\">}</span></span>\n</span></span></code></pre>\n<p>Modular repositories can also host custom pipeline blocks as Python code and visual UI configurations for tools like <a target=\"_blank\" rel=\"noopener noreferrer\" href=\"https://huggingface.co/docs/diffusers/main/en/modular_diffusers/mellon\">Mellon</a> — all in one place.</p>\n<h2>Community Pipelines</h2>\n<p>The community has already started building complete pipelines with Modular Diffusers and publishing them on the Hub, with model weights and ready-to-run code.</p>\n<ul>\n<li><a target=\"_blank\" rel=\"noopener noreferrer\" href=\"https://huggingface.co/krea/krea-realtime-video\"><strong>Krea Realtime Video</strong></a> — A 14B parameter real-time video generation model distilled from Wan 2.1, achieving 11fps on a single B200 GPU. It supports text-to-video, video-to-video, and streaming video-to-video — all built as modular blocks. Users can modify prompts mid-generation, restyle videos on-the-fly, and see first frames within 1 second.</li>\n</ul>\n<pre><code class=\"language-python\"><span class=\"code-snippet\" data-language=\"Python\"><span class=\"syntax-source syntax-python\"><span class=\"syntax-meta syntax-statement syntax-import syntax-python\"><span class=\"syntax-keyword syntax-control syntax-import syntax-python\">import</span> <span class=\"syntax-meta syntax-qualified-name syntax-python\"><span class=\"syntax-meta syntax-generic-name syntax-python\">torch</span></span></span>\n<span class=\"syntax-meta syntax-statement syntax-import syntax-python\"><span class=\"syntax-keyword syntax-control syntax-import syntax-from syntax-python\">from</span></span><span class=\"syntax-meta syntax-statement syntax-import syntax-python\"><span class=\"syntax-meta syntax-import-source syntax-python\"> <span class=\"syntax-meta syntax-import-path syntax-python\"><span class=\"syntax-meta syntax-import-name syntax-python\">diffusers</span></span> <span class=\"syntax-meta syntax-statement syntax-import syntax-python\"><span class=\"syntax-keyword syntax-control syntax-import syntax-python\">import</span></span></span></span><span class=\"syntax-meta syntax-statement syntax-import syntax-python\"></span><span class=\"syntax-meta syntax-statement syntax-import syntax-python\"> <span class=\"syntax-meta syntax-generic-name syntax-python\">ModularPipeline</span></span>\n\n<span class=\"syntax-meta syntax-qualified-name syntax-python\"><span class=\"syntax-meta syntax-generic-name syntax-python\">pipe</span></span> <span class=\"syntax-keyword syntax-operator syntax-assignment syntax-python\">=</span> <span class=\"syntax-meta syntax-function-call syntax-python\"><span class=\"syntax-meta syntax-qualified-name syntax-python\"><span class=\"syntax-meta syntax-generic-name syntax-python\">ModularPipeline</span><span class=\"syntax-punctuation syntax-accessor syntax-dot syntax-python\">.</span></span><span class=\"syntax-meta syntax-qualified-name syntax-python\"><span class=\"syntax-variable syntax-function syntax-python\">from_pretrained</span></span><span class=\"syntax-punctuation syntax-section syntax-arguments syntax-begin syntax-python\">(</span><span class=\"syntax-meta syntax-function-call syntax-arguments syntax-python\"><span class=\"syntax-meta syntax-string syntax-python\"><span class=\"syntax-string syntax-quoted syntax-double syntax-python\"><span class=\"syntax-punctuation syntax-definition syntax-string syntax-begin syntax-python\">&quot;</span></span></span><span class=\"syntax-meta syntax-string syntax-python\"><span class=\"syntax-string syntax-quoted syntax-double syntax-python\">krea/krea-realtime-video<span class=\"syntax-punctuation syntax-definition syntax-string syntax-end syntax-python\">&quot;</span></span></span><span class=\"syntax-punctuation syntax-separator syntax-arguments syntax-python\">,</span> <span class=\"syntax-variable syntax-parameter syntax-python\">trust_remote_code</span><span class=\"syntax-keyword syntax-operator syntax-assignment syntax-python\">=</span><span class=\"syntax-constant syntax-language syntax-python\">True</span></span><span class=\"syntax-punctuation syntax-section syntax-arguments syntax-end syntax-python\">)</span></span>\n<span class=\"syntax-meta syntax-function-call syntax-python\"><span class=\"syntax-meta syntax-qualified-name syntax-python\"><span class=\"syntax-meta syntax-generic-name syntax-python\">pipe</span><span class=\"syntax-punctuation syntax-accessor syntax-dot syntax-python\">.</span></span><span class=\"syntax-meta syntax-qualified-name syntax-python\"><span class=\"syntax-variable syntax-function syntax-python\">load_components</span></span><span class=\"syntax-punctuation syntax-section syntax-arguments syntax-begin syntax-python\">(</span><span class=\"syntax-meta syntax-function-call syntax-arguments syntax-python\">\n    <span class=\"syntax-variable syntax-parameter syntax-python\">trust_remote_code</span><span class=\"syntax-keyword syntax-operator syntax-assignment syntax-python\">=</span><span class=\"syntax-constant syntax-language syntax-python\">True</span><span class=\"syntax-punctuation syntax-separator syntax-arguments syntax-python\">,</span> \n    <span class=\"syntax-variable syntax-parameter syntax-python\">device_map</span><span class=\"syntax-keyword syntax-operator syntax-assignment syntax-python\">=</span><span class=\"syntax-meta syntax-string syntax-python\"><span class=\"syntax-string syntax-quoted syntax-double syntax-python\"><span class=\"syntax-punctuation syntax-definition syntax-string syntax-begin syntax-python\">&quot;</span></span></span><span class=\"syntax-meta syntax-string syntax-python\"><span class=\"syntax-string syntax-quoted syntax-double syntax-python\">cuda<span class=\"syntax-punctuation syntax-definition syntax-string syntax-end syntax-python\">&quot;</span></span></span><span class=\"syntax-punctuation syntax-separator syntax-arguments syntax-python\">,</span>\n    <span class=\"syntax-variable syntax-parameter syntax-python\">torch_dtype</span><span class=\"syntax-keyword syntax-operator syntax-assignment syntax-python\">=</span><span class=\"syntax-meta syntax-structure syntax-dictionary-or-set syntax-python\"><span class=\"syntax-punctuation syntax-section syntax-dictionary-or-set syntax-begin syntax-python\">{</span><span class=\"syntax-meta syntax-string syntax-python\"><span class=\"syntax-string syntax-quoted syntax-double syntax-python\"><span class=\"syntax-punctuation syntax-definition syntax-string syntax-begin syntax-python\">&quot;</span></span></span><span class=\"syntax-meta syntax-string syntax-python\"><span class=\"syntax-string syntax-quoted syntax-double syntax-python\">default<span class=\"syntax-punctuation syntax-definition syntax-string syntax-end syntax-python\">&quot;</span></span></span><span class=\"syntax-punctuation syntax-separator syntax-key-value syntax-python\">:</span> <span class=\"syntax-meta syntax-qualified-name syntax-python\"><span class=\"syntax-meta syntax-generic-name syntax-python\">torch</span><span class=\"syntax-punctuation syntax-accessor syntax-dot syntax-python\">.</span><span class=\"syntax-meta syntax-generic-name syntax-python\">bfloat16</span></span><span class=\"syntax-punctuation syntax-separator syntax-dictionary-or-set syntax-python\">,</span> <span class=\"syntax-meta syntax-string syntax-python\"><span class=\"syntax-string syntax-quoted syntax-double syntax-python\"><span class=\"syntax-punctuation syntax-definition syntax-string syntax-begin syntax-python\">&quot;</span></span></span><span class=\"syntax-meta syntax-string syntax-python\"><span class=\"syntax-string syntax-quoted syntax-double syntax-python\">vae<span class=\"syntax-punctuation syntax-definition syntax-string syntax-end syntax-python\">&quot;</span></span></span><span class=\"syntax-punctuation syntax-separator syntax-key-value syntax-python\">:</span> <span class=\"syntax-meta syntax-qualified-name syntax-python\"><span class=\"syntax-meta syntax-generic-name syntax-python\">torch</span><span class=\"syntax-punctuation syntax-accessor syntax-dot syntax-python\">.</span><span class=\"syntax-meta syntax-generic-name syntax-python\">float16</span></span><span class=\"syntax-punctuation syntax-section syntax-dictionary-or-set syntax-end syntax-python\">}</span></span>\n</span><span class=\"syntax-punctuation syntax-section syntax-arguments syntax-end syntax-python\">)</span></span>\n</span></span></code></pre>\n<ul>\n<li><a target=\"_blank\" rel=\"noopener noreferrer\" href=\"https://huggingface.co/Overworld/Waypoint-1-Small\"><strong>Waypoint-1</strong></a> — A 2.3B parameter real-time diffusion world model from <a target=\"_blank\" rel=\"noopener noreferrer\" href=\"https://over.world\">Overworld</a>. It autoregressively generates interactive worlds from control inputs and text prompts — you can explore and interact with generated environments in real time on consumer hardware.</li>\n</ul>\n<p>Teams can build novel architectures, package them as blocks, and publish the entire pipeline on the Hub for anyone to use with <code>ModularPipeline.from_pretrained</code>.</p>\n<p>Check out the full <a target=\"_blank\" rel=\"noopener noreferrer\" href=\"https://huggingface.co/collections/diffusers/modular-pipelines\">collection of community pipelines</a> for more.</p>\n<h2>Integration with Mellon</h2>\n<blockquote>\n<p>💡 Mellon is in early development and not ready for production use yet. Consider this a sneak peek of how the integration works!</p>\n</blockquote>\n<p><a target=\"_blank\" rel=\"noopener noreferrer\" href=\"https://github.com/cubiq/Mellon\">Mellon</a> is a node graph interface integrated with Modular Diffusers. If you're familiar with node-based tools like ComfyUI, you'll feel right at home — but there are some key differences:</p>\n<ul>\n<li><strong>Dynamic nodes</strong> — Instead of dozens of model-specific nodes, we have a small set of nodes that automatically adapt their interface based on the model you select. Learn them once, use them with any model.</li>\n<li><strong>Single-node workflows</strong> — Thanks to Modular Diffusers' composable block system, you can collapse an entire pipeline into a single node. Run multiple workflows on the same canvas without the clutter.</li>\n<li><strong>Hub integration out of the box</strong> — Custom blocks published to the Hugging Face Hub work instantly in Mellon. We provide a utility function to automatically generate the node interface from your block definition — no UI code required.</li>\n</ul>\n<p>This integration is possible because every block exposes the same properties (<code>inputs</code>, <code>intermediate_outputs</code>, <code>expected_components</code>). This consistent API means Mellon can automatically generate a node's UI from any block definition and compose blocks into higher-level nodes.</p>\n<p>For example, <a target=\"_blank\" rel=\"noopener noreferrer\" href=\"https://huggingface.co/diffusers/FLUX.2-klein-4B-modular\">diffusers/FLUX.2-klein-4B-modular</a> contains a pipeline definition, component references, and a <code>mellon_pipeline_config.json</code> — all in one repo. Load it in Python with <code>ModularPipeline.from_pretrained(&quot;diffusers/FLUX.2-klein-4B-modular&quot;)</code> or in Mellon to create either a single-node or multi-node workflow.</p>\n<p>Here's a quick example. We add a Gemini prompt expansion node — hosted as a modular repo at <a target=\"_blank\" rel=\"noopener noreferrer\" href=\"https://huggingface.co/diffusers/gemini-prompt-expander-mellon\">diffusers/gemini-prompt-expander-mellon</a> — to an existing text-to-image workflow:</p>\n<ol>\n<li>Drag in a <strong>Dynamic Block</strong> node and enter the <code>repo_id</code> (i.e. <code>diffusers/gemini-prompt-expander-mellon</code>)</li>\n<li>Click <strong>LOAD CUSTOM BLOCK</strong> — the node automatically grows a textbox for your prompt input and an output socket named &quot;prompt&quot;, all configured from the repo</li>\n<li>Type a short prompt, connect the output to the <strong>Encode Prompt</strong> node, and run</li>\n</ol>\n<p>Gemini expands your short prompt into a detailed description before generating the image. No code, no configuration — just a Hub repo id.</p>\n<p>Your browser does not support the video tag.</p>\n<p>This is just one example. For a detailed walkthrough, check out the <a target=\"_blank\" rel=\"noopener noreferrer\" href=\"https://www.notion.so/Mellon-x-Modular-Diffusers-2fd1384ebcac819993d8f9ae94c7e866?pvs=21\">Mellon x Modular Diffusers guide</a>. You can also check out <a target=\"_blank\" rel=\"noopener noreferrer\" href=\"https://youtu.be/XogoQnkQUO8\">this video</a>!</p>\n<h2>Conclusion</h2>\n<p>Modular Diffusers brings the composability and flexibility the community has been asking for, without compromising the features that make Diffusers powerful. It's still early — we want your input to shape what comes next. Give it a try and tell us what works, what doesn't, and what's missing.</p>\n<h2>Resources</h2>\n<ul>\n<li><a target=\"_blank\" rel=\"noopener noreferrer\" href=\"https://huggingface.co/docs/diffusers/main/en/modular_diffusers/overview\">Overview</a> of Modular Diffusers</li>\n<li><a target=\"_blank\" rel=\"noopener noreferrer\" href=\"https://github.com/cubiq/Mellon\">Mellon</a></li>\n<li><a target=\"_blank\" rel=\"noopener noreferrer\" href=\"https://www.notion.so/Mellon-x-Modular-Diffusers-2fd1384ebcac819993d8f9ae94c7e866?pvs=21\">Mellon x Modular Diffusers</a></li>\n<li><a target=\"_blank\" rel=\"noopener noreferrer\" href=\"https://huggingface.co/collections/diffusers/modular-diffusers-custom-blocks\">Collection</a> of custom blocks</li>\n<li><a target=\"_blank\" rel=\"noopener noreferrer\" href=\"https://huggingface.co/collections/diffusers/modular-pipelines\">Collection</a> of community pipelines with Modular Diffusers</li>\n</ul>\n<p><em>Thanks to Chun Te Lee for the thumbnail, and to Poli, Pedro, Lysandre, Linoy, Aritra, and Steven for their thoughtful reviews.</em></p>\n",
  "content_markdown": "Modular Diffusers introduces a new way to build diffusion pipelines by composing reusable blocks. Instead of writing entire pipelines from scratch, you can mix and match blocks to create workflows tailored to your needs! This complements the existing `DiffusionPipeline` class with a more flexible, composable alternative.\n\nIn this post, we'll walk through how Modular Diffusers works — from the familiar API to run a modular pipeline, to building fully custom blocks and composing them into your own workflow. We'll also show how it integrates with Mellon, a node-based visual workflow interface that you can use to wire Modular Diffusers blocks together.\n\n**Table of contents**\n\n- [Quickstart](https://huggingface.co/blog/modular-diffusers#quickstart)\n- [Custom Blocks](https://huggingface.co/blog/modular-diffusers#custom-blocks)\n- [Modular Repositories](https://huggingface.co/blog/modular-diffusers#modular-repositories)\n- [Community Pipelines](https://huggingface.co/blog/modular-diffusers#community-pipelines)\n- [Integration with Mellon](https://huggingface.co/blog/modular-diffusers#integration-with-mellon)\n\n## Quickstart\n\nHere is a simple example of how to run inference with `FLUX.2 Klein 4B` using pre-built blocks:\n\n```python\nimport torch\nfrom diffusers import ModularPipeline\n\n# Create a modular pipeline - this only defines the workflow, model weights have not been loaded yet\npipe = ModularPipeline.from_pretrained(\n    \"black-forest-labs/FLUX.2-klein-4B\"\n)\n#  Now load the model weights — configure dtype, quantization, etc in this step\npipe.load_components(torch_dtype=torch.bfloat16)\npipe.to(\"cuda\")\n\n# Generate an image - API remains the same as DiffusionPipeline\nimage = pipe(\n    prompt=\"a serene landscape at sunset\",\n    num_inference_steps=4,\n).images[0]\n\nimage.save(\"output.png\")\n```\n\nYou get the same results as with a standard `DiffusionPipeline`, but the pipeline is very different under the hood: it's composed of flexible *blocks* — text encoding, image encoding, denoising, and decoding — that you can inspect directly:\n\n```python\nprint(pipe.blocks)\n```\n\n```bash\nFlux2KleinAutoBlocks(\n  ...\n  Sub-Blocks:\n    [0] text_encoder (Flux2KleinTextEncoderStep)\n    [1] vae_encoder (Flux2KleinAutoVaeEncoderStep)\n    [2] denoise (Flux2KleinCoreDenoiseStep)\n    [3] decode (Flux2DecodeStep)\n)\n```\n\nEach block is self-contained with its own inputs and outputs. You can run any block independently as its own pipeline, or add, remove, and swap blocks freely — they dynamically recompose to work with whatever blocks remain. Use `.init_pipeline()` to convert blocks into a runnable pipeline, and `.load_components()` to load the model weights.\n\n```python\n# get a copy of the blocks\nblocks = pipe.blocks\n\n# pop out the text_encoder block\ntext_blocks = blocks.sub_blocks.pop(\"text_encoder\")\n\n# run it as its own pipeline\ntext_pipe = text_blocks.init_pipeline(\"black-forest-labs/FLUX.2-klein-4B\")\n\n# load the text_encoder, or reuse already loaded components: text_pipe.update_components(text_encoder=pipe.text_encoder)\ntext_pipe.load_components(torch_dtype=torch.bfloat16)\ntext_pipe.to(\"cuda\")\nprompt_embeds = text_pipe(prompt=\"a serene landscape at sunset\").prompt_embeds\n\n# create a new pipeline from the remaining blocks\n# it now accepts prompt_embeds directly instead of prompt\nremaining_pipe = blocks.init_pipeline(\"black-forest-labs/FLUX.2-klein-4B\")\nremaining_pipe.load_components(torch_dtype=torch.bfloat16)\nremaining_pipe.to(\"cuda\")\nimage = remaining_pipe(prompt_embeds=prompt_embeds, num_inference_steps=4).images[0]\n```\n\nFor more on block types, composition patterns, lazy loading, and memory management with `ComponentsManager`, check out the [Modular Diffusers documentation](https://huggingface.co/docs/diffusers/en/modular_diffusers/overview).\n\n## Custom Blocks\n\nModular Diffusers really shines when creating your own blocks. A custom block is a Python class that defines its components, inputs, outputs, and computation logic — and once defined, you can plug it into any workflow.\n\n### Writing a Custom Block\n\nHere's an example block that extracts depth maps from images using [Depth Anything V2](https://huggingface.co/depth-anything/Depth-Anything-V2-Large).\n\n```python\nclass DepthProcessorBlock(ModularPipelineBlocks):\n    @property\n    def expected_components(self):\n        return [\n            ComponentSpec(\"depth_processor\", DepthPreprocessor,\n                          pretrained_model_name_or_path=\"depth-anything/Depth-Anything-V2-Large-hf\")\n        ]\n\n    @property\n    def inputs(self):\n        return [\n            InputParam(\"image\", required=True,\n                       description=\"Image(s) to extract depth maps from\"),\n        ]\n\n    @property\n    def intermediate_outputs(self):\n        return [\n            OutputParam(\"control_image\", type_hint=torch.Tensor,\n                        description=\"Depth map(s) of input image(s)\"),\n        ]\n\n    @torch.no_grad()\n    def __call__(self, components, state):\n        block_state = self.get_block_state(state)\n        depth_map = components.depth_processor(block_state.image)\n        block_state.control_image = depth_map.to(block_state.device)\n        self.set_block_state(state, block_state)\n        return components, state\n```\n\n- `expected_components` defines what models the block needs — in this case, a depth estimation model. The `pretrained_model_name_or_path` parameter sets a default Hub repo to load from, so `load_components` automatically fetches the depth model unless you override it in `modular_model_index.json`.\n- `inputs` and `intermediate_outputs` define what goes in and comes out.\n- `__call__` is where the computation logic lives.\n\n### Composing Blocks into Workflows\n\nLet's use this block with Qwen's ControlNet workflow. Extract the ControlNet workflow and insert the depth block at the beginning:\n\n```python\n# Create Qwen Image pipeline\npipe = ModularPipeline.from_pretrained(\"Qwen/Qwen-Image\")\n\nprint(pipe.blocks.available_workflows)\n#       Supported workflows:\n#        - `text2image`: requires `prompt`\n#        - `image2image`: requires `prompt`, `image`\n#        - `inpainting`: requires `prompt`, `mask_image`, `image`\n#        - `controlnet_text2image`: requires `prompt`, `control_image`\n#        - `controlnet_image2image`: requires `prompt`, `image`, `control_image`\n\n# Extract the ControlNet workflow — it expects a control_image input\nblocks = pipe.blocks.get_workflow(\"controlnet_text2image\")\n# Show the blocks this workflow uses\nprint(blocks)\n\n# Insert depth block at the beginning — its output (control_image)\n# automatically flows to the ControlNet block that needs it\nblocks.sub_blocks.insert(\"depth\", DepthProcessorBlock(), 0)\n\n# You can inspect any block's inputs and outputs with print(blocks.doc)\nblocks.sub_blocks['depth'].doc\n```\n\nBlocks in a sequence share data automatically: the depth block's `control_image` output flows to downstream blocks that need it, and its `image` input becomes a pipeline input since no earlier block provides it.\n\n![blocks_composed](https://huggingface.co/datasets/diffusers/modular-diffusers-blog/resolve/main/blocks_composed.png)\n\n```python\nfrom diffusers import ComponentsManager, AutoModel\nfrom diffusers.utils import load_image\n\n# ComponentsManager handles memory across multiple pipelines —\n# it automatically offloads models to CPU when not in use\nmanager = ComponentsManager()\n\npipeline = blocks.init_pipeline(\"Qwen/Qwen-Image\", components_manager=manager)\npipeline.load_components(torch_dtype=torch.bfloat16)\n\n# The depth model loads automatically from the default path we set in expected_components —\n# no need to load it manually even though it's not part of the Qwen repo.\n# But controlnet is not included by default, so we do need to load it from a different repo\ncontrolnet = AutoModel.from_pretrained(\"InstantX/Qwen-Image-ControlNet-Union\", torch_dtype=torch.bfloat16)\npipeline.update_components(controlnet=controlnet)\n\n# pipeline now takes image as input\nimage = load_image(\"https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/diffusers/astronaut.jpg\")\noutput = pipeline(\n    prompt=\"an astronaut hatching from an egg, detailed, fantasy, Pixar, Disney\",\n    image=image,\n).images[0]\n```\n\n### Sharing Custom Blocks on the Hub\n\nYou can publish your custom block to the Hub so anyone can load it with `trust_remote_code=True`. We've created a [template](https://huggingface.co/diffusers/custom-block-template) to get you started — check out the [Building Custom Blocks guide](https://huggingface.co/docs/diffusers/main/en/modular_diffusers/custom_blocks#quick-start-with-template) for the full walkthrough.\n\n```python\npipeline.save_pretrained(local_dir, repo_id=\"your-username/your-block-name\", push_to_hub=True)\n```\n\nThe `DepthProcessorBlock` from this post is published at [diffusers/depth-processor-custom-block](https://huggingface.co/diffusers/depth-processor-custom-block) — you can load and use it directly:\n\n```python\nfrom diffusers import ModularPipelineBlocks\n\ndepth_block = ModularPipelineBlocks.from_pretrained(\n    \"diffusers/depth-processor-custom-block\", trust_remote_code=True\n)\n```\n\nWe've published a collection of ready-to-use custom blocks [here](https://huggingface.co/collections/diffusers/modular-diffusers-custom-blocks).\n\n## Modular Repositories\n\n`ModularPipeline.from_pretrained` works with any existing Diffusers repo out of the box, but Modular Diffusers also introduces a new kind of repo: the Modular Repository.\n\nA modular repository is able to reference components from their original model repos. For example, [diffusers/flux2-bnb-4bit-modular](https://huggingface.co/diffusers/flux2-bnb-4bit-modular) contains a quantized transformer and loads the remaining components from the original repo.\n\n```json\n// diffusers/flux2-bnb-4bit-modular/modular_model_index.json\n{\n    \"transformer\": [\n        \"diffusers\", \n        \"Flux2Transformer2DModel\", \n        {\n            \"pretrained_model_name_or_path\": \"diffusers/flux2-bnb-4bit-modular\",\n            \"subfolder\": \"transformer\",\n            \"type_hint\": [\"diffusers\", \"Flux2Transformer2DModel\"]\n        }\n    ],\n    \"vae\": [\n        \"diffusers\", \n        \"AutoencoderKLFlux2\", \n        {\n            \"pretrained_model_name_or_path\": \"black-forest-labs/FLUX.2-dev\",\n            \"subfolder\": \"vae\",\n            \"type_hint\": [\"diffusers\", \"AutoencoderKLFlux2\"]\n        }\n    ],\n    ...\n}\n```\n\nModular repositories can also host custom pipeline blocks as Python code and visual UI configurations for tools like [Mellon](https://huggingface.co/docs/diffusers/main/en/modular_diffusers/mellon) — all in one place.\n\n## Community Pipelines\n\nThe community has already started building complete pipelines with Modular Diffusers and publishing them on the Hub, with model weights and ready-to-run code.\n\n- [**Krea Realtime Video**](https://huggingface.co/krea/krea-realtime-video) — A 14B parameter real-time video generation model distilled from Wan 2.1, achieving 11fps on a single B200 GPU. It supports text-to-video, video-to-video, and streaming video-to-video — all built as modular blocks. Users can modify prompts mid-generation, restyle videos on-the-fly, and see first frames within 1 second.\n\n```python\nimport torch\nfrom diffusers import ModularPipeline\n\npipe = ModularPipeline.from_pretrained(\"krea/krea-realtime-video\", trust_remote_code=True)\npipe.load_components(\n    trust_remote_code=True, \n    device_map=\"cuda\",\n    torch_dtype={\"default\": torch.bfloat16, \"vae\": torch.float16}\n)\n```\n\n- [**Waypoint-1**](https://huggingface.co/Overworld/Waypoint-1-Small) — A 2.3B parameter real-time diffusion world model from [Overworld](https://over.world). It autoregressively generates interactive worlds from control inputs and text prompts — you can explore and interact with generated environments in real time on consumer hardware.\n\nTeams can build novel architectures, package them as blocks, and publish the entire pipeline on the Hub for anyone to use with `ModularPipeline.from_pretrained`.\n\nCheck out the full [collection of community pipelines](https://huggingface.co/collections/diffusers/modular-pipelines) for more.\n\n## Integration with Mellon\n\n> 💡 Mellon is in early development and not ready for production use yet. Consider this a sneak peek of how the integration works!\n\n[Mellon](https://github.com/cubiq/Mellon) is a node graph interface integrated with Modular Diffusers. If you're familiar with node-based tools like ComfyUI, you'll feel right at home — but there are some key differences:\n\n- **Dynamic nodes** — Instead of dozens of model-specific nodes, we have a small set of nodes that automatically adapt their interface based on the model you select. Learn them once, use them with any model.\n- **Single-node workflows** — Thanks to Modular Diffusers' composable block system, you can collapse an entire pipeline into a single node. Run multiple workflows on the same canvas without the clutter.\n- **Hub integration out of the box** — Custom blocks published to the Hugging Face Hub work instantly in Mellon. We provide a utility function to automatically generate the node interface from your block definition — no UI code required.\n\nThis integration is possible because every block exposes the same properties (`inputs`, `intermediate_outputs`, `expected_components`). This consistent API means Mellon can automatically generate a node's UI from any block definition and compose blocks into higher-level nodes.\n\nFor example, [diffusers/FLUX.2-klein-4B-modular](https://huggingface.co/diffusers/FLUX.2-klein-4B-modular) contains a pipeline definition, component references, and a `mellon_pipeline_config.json` — all in one repo. Load it in Python with `ModularPipeline.from_pretrained(\"diffusers/FLUX.2-klein-4B-modular\")` or in Mellon to create either a single-node or multi-node workflow.\n\nHere's a quick example. We add a Gemini prompt expansion node — hosted as a modular repo at [diffusers/gemini-prompt-expander-mellon](https://huggingface.co/diffusers/gemini-prompt-expander-mellon) — to an existing text-to-image workflow:\n\n1. Drag in a **Dynamic Block** node and enter the `repo_id` (i.e. `diffusers/gemini-prompt-expander-mellon`)\n2. Click **LOAD CUSTOM BLOCK** — the node automatically grows a textbox for your prompt input and an output socket named \"prompt\", all configured from the repo\n3. Type a short prompt, connect the output to the **Encode Prompt** node, and run\n\nGemini expands your short prompt into a detailed description before generating the image. No code, no configuration — just a Hub repo id.\n\nYour browser does not support the video tag.\n\nThis is just one example. For a detailed walkthrough, check out the [Mellon x Modular Diffusers guide](https://www.notion.so/Mellon-x-Modular-Diffusers-2fd1384ebcac819993d8f9ae94c7e866?pvs=21). You can also check out [this video](https://youtu.be/XogoQnkQUO8)!\n\n## Conclusion\n\nModular Diffusers brings the composability and flexibility the community has been asking for, without compromising the features that make Diffusers powerful. It's still early — we want your input to shape what comes next. Give it a try and tell us what works, what doesn't, and what's missing.\n\n## Resources\n\n- [Overview](https://huggingface.co/docs/diffusers/main/en/modular_diffusers/overview) of Modular Diffusers\n- [Mellon](https://github.com/cubiq/Mellon)\n- [Mellon x Modular Diffusers](https://www.notion.so/Mellon-x-Modular-Diffusers-2fd1384ebcac819993d8f9ae94c7e866?pvs=21)\n- [Collection](https://huggingface.co/collections/diffusers/modular-diffusers-custom-blocks) of custom blocks\n- [Collection](https://huggingface.co/collections/diffusers/modular-pipelines) of community pipelines with Modular Diffusers\n\n*Thanks to Chun Te Lee for the thumbnail, and to Poli, Pedro, Lysandre, Linoy, Aritra, and Steven for their thoughtful reviews.*\n",
  "dateCreated": "2026-10-04T12:50:30.804479567+00:00",
  "date_created": "2026-03-05T00:00:00+00:00",
  "date_modified": "2026-03-05T00:00:00+00:00",
  "date_published": "2026-03-05T00:00:00+00:00",
  "description": "Modular Diffusers introduces a new way to build diffusion pipelines by composing reusable blocks. Instead of writing entire pipelines from scratch, you can mix and match blocks to create workflows tailored to your needs! This complements…",
  "external_url": "https://huggingface.co/blog/modular-diffusers",
  "git": {
    "commit": "ebb0941c5c06a2ab2377342d607535262a3e969e",
    "path": "items/huggingface-co/2026/03/2026-03-05-introducing-modular-diffusers-composable-building-blocks.md",
    "permalink": "https://github.com/aymericbeaumet/aggr-instance/blob/ebb0941c5c06a2ab2377342d607535262a3e969e/items/huggingface-co/2026/03/2026-03-05-introducing-modular-diffusers-composable-building-blocks.md"
  },
  "id": "urn:aggr:item:e97fdf21d4f5db0f530655a75eaa569050fd7769",
  "isBasedOn": {
    "@id": "https://huggingface.co/blog/modular-diffusers"
  },
  "mainEntity": {
    "@id": "https://huggingface.co/blog/modular-diffusers",
    "@type": "CreativeWork",
    "archivedAt": {
      "@id": "https://aggr.aymericbeaumet.com/items/huggingface-co/2026-03-05-introducing-modular-diffusers-composable-building-blocks/#webpage"
    },
    "dateModified": "2026-03-05T00:00:00+00:00",
    "datePublished": "2026-03-05T00:00:00+00:00",
    "headline": "Introducing Modular Diffusers - Composable Building Blocks for Diffusion Pipelines",
    "timeRequired": "PT8M",
    "url": "https://huggingface.co/blog/modular-diffusers",
    "wordCount": 1745
  },
  "name": "Archived snapshot: Introducing Modular Diffusers - Composable Building Blocks for Diffusion Pipelines",
  "preview": null,
  "reading_minutes": 8,
  "replicated_at": null,
  "source": "huggingface-co",
  "source_name": "Hugging Face - Blog",
  "summary": null,
  "tags": [],
  "temporalCoverage": "2026-10-04T12:50:30.804479567+00:00",
  "title": "Introducing Modular Diffusers - Composable Building Blocks for Diffusion Pipelines",
  "url": "https://aggr.aymericbeaumet.com/items/huggingface-co/2026-03-05-introducing-modular-diffusers-composable-building-blocks/",
  "word_count": 1745
}