{
  "@context": "https://schema.org",
  "@id": "https://aggr.aymericbeaumet.com/items/huggingface-co/2026-02-13-custom-kernels-for-all-from-codex-and-claude/#webpage",
  "@type": [
    "WebPage",
    "ArchiveComponent"
  ],
  "articleBody": "[![oprah custom cuda kernels](https://huggingface.co/blog/assets/custom-cuda-kernels/meme.png)](https://huggingface.co/blog/assets/custom-cuda-kernels/meme.png)\n\ntl;dr: We built an agent skill that teaches coding agents how to write production CUDA kernels. Then we pointed Claude and Codex at two real targets: a **diffusers** pipeline and a **transformers** model. The agents produced working kernels for both, with correct PyTorch bindings and benchmarks, end to end.\n\nWriting CUDA kernels is hard. Writing CUDA kernels that correctly integrate with `transformers` and `diffusers` is harder. There are architecture-specific memory access patterns, vectorization strategies, warp shuffle reductions, and a dozen integration pitfalls that trip up even experienced developers. It is exactly the kind of specialized, high-stakes problem where agent skills shine.\n\nWe gave coding agents the domain knowledge they need, like which GPU architecture to target, how to structure a kernel-builder project, when to use shared memory versus registers, and how to write PyTorch bindings. The agents did the rest. If you have used the [LLM training skill](https://huggingface.co/blog/hf-skills-training) or read [We Got Claude to Teach Open Models](https://huggingface.co/blog/upskill), the pattern will feel familiar: package domain expertise into a skill, point the agent at a problem, and let it work.\n\n## Why a skill for kernels?\n\nThe [Kernel Hub](https://huggingface.co/blog/hello-hf-kernels) solved the distribution of custom hardware kernels. You can load pre-compiled kernels from the Hub with a single `get_kernel` call. No builds, no flags. However, someone still needs to **write the kernels**. That is the gap this skill fills.\n\nCUDA kernel development has a brutal surface area:\n\n- Hardware-specific optimization guides for each generation of GPU. H100, A100, and T4 each have different compute capabilities, shared memory sizes, and bandwidth profiles\n- In Libraries, `diffusers` and `transformers` have different module hierarchies, normalization conventions, and integration patterns. Custom kernels need to be registered in PyTorch for `torch.compile` to recognize.\n- For distribution, kernels can depend on CUDA, Pytorch, and Python versions creating massive environment matrices.\n\nThis is domain knowledge that gets lost in documentation tabs and Stack Overflow answers. An agent skill packages it into context that loads on demand.\n\nFirst, let's show how to use the skill right away, then we'll dive into the details of how we benchmarked the kernels.\n\n## Installing the skill\n\nThe skill ships with the `kernels` library. Install it into your coding agent with a single command:\n\n```shell\n# we need to install kernels from main for this\npip install git+https://github.com/huggingface/kernels.git#subdirectory=kernels\nkernels skills add cuda-kernels --claude\n```\n\nThis drops the skill into `.claude/skills/cuda-kernels/` where Claude Code and Cursor pick it up automatically. For other agents:\n\n```shell\n# Codex\nkernels skills add cuda-kernels --codex\n\n# OpenCode\nkernels skills add cuda-kernels --opencode\n\n# Custom destination\nkernels skills add cuda-kernels --dest ./my-agent/skills/\n\n# Install globally (available across all projects)\nkernels skills add cuda-kernels --global\n\n# Overwrite an existing installation\nkernels skills add cuda-kernels --claude --force\n```\n\nOnce installed, prompt your agent:\n\n```\nBuild a vectorized RMSNorm kernel for H100 targeting the Qwen3-8B model in transformers.\n```\n\nOr, you can go for something more open-ended:\n\n```\nBuild an optimized attention kernel for H100 targeting the Qwen3-8B model in transformers. Benchmark it against the PyTorch baseline and validate improvements in end-to-end performance.\n```\n\nThe agent can read the skill, select the right architecture parameters, generate the CUDA source, write the PyTorch bindings, set up `build.toml`, and create a benchmark script.\n\nIf you're working on more complex kernels, or architecture-specific optimizations, that aren't covered in the skill, then the skill supplies the fundamental building blocks and patterns to get you started. We are also open to contributions on the [skill itself](https://github.com/huggingface/kernels/tree/main/kernel-builder/skills).\n\n## What is in the skill\n\nThe skill is roughly **550 tokens** of structured guidance plus reference scripts, GPU optimization guides, troubleshooting docs, and complete working examples. Agentic coding tools like Codex and Claude can read this and produce a working kernel project.\n\nIt covers:\n\n- NVIDIA GPU Architecture-aware optimization for H100, A100, and T4 (compute capabilities, memory bandwidth, shared memory sizes, block sizing)\n- Integration patterns for both `diffusers` and `transformers`, including the pitfalls specific to each library\n- Kernel templates with vectorized memory access patterns for BF16, FP16, and FP32\n- Benchmarking workflows for both isolated kernel micro-benchmarks and end-to-end pipeline comparisons\n- HuggingFace Kernel Hub integration via `get_kernel` for loading community kernels\n\n```\n.claude/skills/cuda-kernels/\n├── SKILL.md                              # Main instructions (~550 tokens)\n├── scripts/\n│   ├── benchmark_example.py              # End-to-end benchmark template\n│   ├── benchmark_rmsnorm.py              # Isolated kernel micro-benchmark\n│   ├── ltx_kernel_injection_example.py   # Diffusers integration pattern\n│   ├── transformers_injection_example.py # Transformers integration pattern\n│   └── huggingface_kernels_example.py    # Kernel Hub integration\n└── references/\n    ├── diffusers-integration.md          # Diffusers guide with pitfalls\n    ├── transformers-integration.md       # Transformers guide\n    ├── huggingface-kernels-integration.md\n    ├── h100-optimization-guide.md\n    ├── a100-optimization-guide.md\n    ├── t4-optimization-guide.md\n    ├── kernel-templates.md\n    └── troubleshooting.md\n```\n\nWhen an agent loads this, it gets everything it needs to go from \"write me an RMSNorm kernel\" to a buildable, benchmarkable project. It will grep and glob the skill to find the relevant files and directories. So it's important to structure the skill in a way that is easy to find.\n\nThe agent is instructed to generate kernels that conform to the templates in `references/kernel-templates.md` and produce a complete kernel project:\n\n```\nexamples/your_model/\n├── kernel_src/\n│   └── rmsnorm.cu              # Vectorized CUDA kernel\n├── torch-ext/\n│   ├── your_kernels/__init__.py\n│   └── torch_binding.cpp       # PyTorch C++ bindings\n├── benchmark_rmsnorm.py        # Micro-benchmark script\n├── build.toml                  # kernel-builder config\n├── setup.py                    # pip install -e .\n└── pyproject.toml\n```\n\nWe tested this on two real targets.\n\n## Benchmarking the kernels: Diffusers (LTX-Video on H100)\n\nThe agent built RMSNorm, RoPE 3D, GEGLU, and AdaLN kernels for [LTX-Video](https://huggingface.co/Lightricks/LTX-Video), a video generation pipeline from `diffusers`. The full example is at `examples/ltx_video/`. We optimized the RMSNorm kernel for H100. Both benchmarks were run on H100 80GB HBM3 at precision BFloat16.\n\nIf you want to check out the generated kernel, got to [this example](https://github.com/burtenshaw/kernel-skill/tree/main/examples/ltx_video)\n\n### Isolated RMSNorm benchmark\n\nFirst, we compare the isolated RMSNorm kernel performance against the PyTorch baseline. This is the main speedup in the optimized pipeline.\n\n[![isolated rmsnorm benchmark ltx-video](https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/kernels-skill-benchmark/rmsnorm_ltx_video.png)](https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/kernels-skill-benchmark/rmsnorm_ltx_video.png)\n\nTable\n\n| Shape           | Custom (ms) | PyTorch (ms) | Speedup   |\n| --------------- | ----------- | ------------ | --------- |\n| \\[1x1024x2048\\] | 0.039       | 0.064        | **1.64x** |\n| \\[2x1024x2048\\] | 0.040       | 0.073        | **1.82x** |\n| \\[4x1024x2048\\] | 0.052       | 0.093        | **1.78x** |\n| \\[1x4096x2048\\] | 0.052       | 0.093        | **1.79x** |\n| \\[2x4096x3072\\] | 0.102       | 0.209        | **2.04x** |\n| \\[1x8192x2048\\] | 0.083       | 0.150        | **1.81x** |\n| \\[4x4096x3072\\] | 0.173       | 0.393        | **2.26x** |\n\n**Average speedup: 1.88x** and a bandwidth efficiency: 34.7% of H100 theoretical (3,350 GB/s)\n\n### End-to-end video generation (49 frames, 30 steps, H100 80GB)\n\nNext, we compare the end-to-end video generation performance of the optimized kernels against the baseline (no compile) and the `torch.compile` baseline.\n\n[![e2e benchmark ltx-video](https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/kernels-skill-benchmark/e2e_ltx_video.png)](https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/kernels-skill-benchmark/e2e_ltx_video.png)\n\nTable\n\n| Configuration                   | Time (s) | it/s  | Speedup   |\n| ------------------------------- | -------- | ----- | --------- |\n| Baseline (no compile)           | 2.87     | 12.58 | 1.00x     |\n| **Generated Optimized Kernels** | 2.70     | 13.52 | **1.06x** |\n| Baseline + torch.compile        | 2.14     | 19.05 | 1.34x     |\n| Optimized + torch.compile       | 2.01     | 18.45 | 1.43x     |\n\nRMSNorm accounts for ~5% of total compute in LTX-Video. The remaining time is spent in attention, linear projections, and VAE decode. The 6% end-to-end speedup from a single kernel type is consistent with that profile.\n\n## Benchmarking the kernels: Transformers (Qwen3-8B on H100)\n\nThe agent built an RMSNorm kernel for [Qwen3-8B](https://huggingface.co/Qwen/Qwen3-8B), a large language model from `transformers` with 65 RMSNorm modules across 32 layers. The full example is at `examples/qwen3_8b/`. We optimized the RMSNorm kernel for H100. Both benchmarks were run on H100 80GB HBM3 at precision BFloat16.\n\nIf you want to explore the kernel, check it out [here.](https://github.com/burtenshaw/kernel-skill/tree/main/examples/qwen3_8b)\n\n### Isolated RMSNorm benchmark\n\nOnce again, we compare the isolated RMSNorm kernel performance against the PyTorch baseline.\n\n[![isolated rmsnorm benchmark qwen3-8b](https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/kernels-skill-benchmark/rmsnorm_qwen3.png)](https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/kernels-skill-benchmark/rmsnorm_qwen3.png)\n\n**Average speedup: 1.94x** and a bandwidth efficiency: 22.3% of H100 theoretical (3,350 GB/s)\n\nTable\n\n| Shape           | Custom (ms) | PyTorch (ms) | Speedup   |\n| --------------- | ----------- | ------------ | --------- |\n| \\[1x128x4096\\]  | 0.040       | 0.062        | **1.58x** |\n| \\[1x512x4096\\]  | 0.038       | 0.064        | **1.69x** |\n| \\[1x1024x4096\\] | 0.037       | 0.071        | **1.90x** |\n| \\[1x2048x4096\\] | 0.045       | 0.091        | **2.03x** |\n| \\[1x4096x4096\\] | 0.071       | 0.150        | **2.12x** |\n| \\[4x512x4096\\]  | 0.056       | 0.093        | **1.67x** |\n| \\[8x256x4096\\]  | 0.045       | 0.092        | **2.06x** |\n| \\[1x8192x4096\\] | 0.109       | 0.269        | **2.47x** |\n\nSpeedup scales with sequence length: 1.58x at 128 tokens, 2.47x at 8192 tokens. For long-context inference, the custom kernel roughly halves RMSNorm latency.\n\n## Publishing your kernel to the Hub\n\nThe agent gives you a working kernel. The [Kernel Hub](https://huggingface.co/kernels-community) lets you share it so anyone can load it without compilation. Here is the full path from agent output to published kernel.\n\n### 1. Verify the project structure\n\nThe agent produces a project that already follows the [kernel-builder](https://huggingface.co/docs/kernels/en/builder/writing-kernels) layout:\n\n```\nyour_kernel/\n├── build.toml               # Build configuration\n├── kernel_src/\n│   └── rmsnorm.cu           # CUDA kernel source\n└── torch-ext/\n    ├── torch_binding.cpp    # Registers Torch ops\n    └── your_kernels/\n        └── __init__.py      # Python API wrapping _ops\n```\n\nThe `build.toml` tells `kernel-builder` what to build. The agent generates this for you, including the correct `cuda-capabilities` for your target GPU:\n\n```\n[general]\nname = \"your_kernels\"\nbackends = [\"cuda\"]\n\n[torch]\nsrc = [\"torch-ext/torch_binding.cpp\"]\n\n[kernel.rmsnorm]\nbackend = \"cuda\"\nsrc = [\"kernel_src/rmsnorm.cu\"]\ndepends = [\"torch\"]\ncuda-capabilities = [\"9.0\"]  # H100\n```\n\n### 2. Build all variants with Nix\n\nKernel Hub kernels must support all recent PyTorch and CUDA configurations. The kernel-builder Nix flake handles this automatically. Copy the [example `flake.nix`](https://github.com/huggingface/kernels/blob/main/examples/kernels/relu/flake.nix) into your project and run:\n\n```shell\nnix flake update\nnix run .#build-and-copy -L\n```\n\nThis builds the kernel for every required PyTorch/CUDA variant and places the results in `build/`. For faster builds, enable the HuggingFace Nix cache:\n\n```shell\nnix run nixpkgs#cachix -- use huggingface\n```\n\n### 3. Create a Hub repo and push\n\nCreate a model repo on the Hub and upload the built kernel:\n\n```shell\nhuggingface-cli repo create your-org/your-kernel --type model\nhuggingface-cli upload your-org/your-kernel ./build\n```\n\n### 4. Others load it in one line\n\nOnce published, anyone can use your kernel with zero compilation:\n\n```py\nfrom kernels import get_kernel\n\nrmsnorm = get_kernel(\"your-org/your-kernel\")\n```\n\n`get_kernel` detects the user's Python, PyTorch, and CUDA versions and downloads the matching pre-compiled binary. No builds, no flags, typically ready in seconds.\n\nThe skill and the Hub are complementary. The skill handles development. The Hub handles distribution. Build a kernel with the skill, validate it with the benchmark scripts, publish it to the Hub, and it becomes a one-liner for everyone else.\n\n## Conclusion\n\nWe built an agent skill that teaches coding agents how to write production CUDA kernels. Then we pointed Claude and Codex at two real targets: a **diffusers** pipeline and a **transformers** model. The agents produced working kernels for both, with correct PyTorch bindings and benchmarks, end to end. We benchmarked the kernels and found that the optimized kernels can provide a speedup in both isolated and end-to-end performance.\n\n## Resources\n\n- [CUDA Kernels Skill in `kernels`](https://github.com/huggingface/kernels/tree/main/kernel-builder/skills/cuda-kernels)\n- [HuggingFace Kernel Hub Blog](https://huggingface.co/blog/hello-hf-kernels)\n- [We Got Claude to Fine-Tune an Open Source LLM](https://huggingface.co/blog/hf-skills-training)\n- [We Got Claude to Teach Open Models](https://huggingface.co/blog/upskill)\n- [HuggingFace Kernels Community](https://huggingface.co/kernels-community)\n",
  "authors": [],
  "captured_at": "2026-10-04T12:50:30.804479567+00:00",
  "category": "ai",
  "content_html": "<p><a target=\"_blank\" rel=\"noopener noreferrer\" href=\"https://huggingface.co/blog/assets/custom-cuda-kernels/meme.png\"><picture class=\"article-picture article-picture-fallback\" style=\"--image-ratio:16 / 9\"><img src=\"https://huggingface.co/blog/assets/custom-cuda-kernels/meme.png\" alt=\"oprah custom cuda kernels\" loading=\"eager\" decoding=\"async\" fetchpriority=\"high\" referrerpolicy=\"no-referrer\" class=\"progressive-image\"></picture></a></p>\n<p>tl;dr: We built an agent skill that teaches coding agents how to write production CUDA kernels. Then we pointed Claude and Codex at two real targets: a <strong>diffusers</strong> pipeline and a <strong>transformers</strong> model. The agents produced working kernels for both, with correct PyTorch bindings and benchmarks, end to end.</p>\n<p>Writing CUDA kernels is hard. Writing CUDA kernels that correctly integrate with <code>transformers</code> and <code>diffusers</code> is harder. There are architecture-specific memory access patterns, vectorization strategies, warp shuffle reductions, and a dozen integration pitfalls that trip up even experienced developers. It is exactly the kind of specialized, high-stakes problem where agent skills shine.</p>\n<p>We gave coding agents the domain knowledge they need, like which GPU architecture to target, how to structure a kernel-builder project, when to use shared memory versus registers, and how to write PyTorch bindings. The agents did the rest. If you have used the <a target=\"_blank\" rel=\"noopener noreferrer\" href=\"https://huggingface.co/blog/hf-skills-training\">LLM training skill</a> or read <a target=\"_blank\" rel=\"noopener noreferrer\" href=\"https://huggingface.co/blog/upskill\">We Got Claude to Teach Open Models</a>, the pattern will feel familiar: package domain expertise into a skill, point the agent at a problem, and let it work.</p>\n<h2>Why a skill for kernels?</h2>\n<p>The <a target=\"_blank\" rel=\"noopener noreferrer\" href=\"https://huggingface.co/blog/hello-hf-kernels\">Kernel Hub</a> solved the distribution of custom hardware kernels. You can load pre-compiled kernels from the Hub with a single <code>get_kernel</code> call. No builds, no flags. However, someone still needs to <strong>write the kernels</strong>. That is the gap this skill fills.</p>\n<p>CUDA kernel development has a brutal surface area:</p>\n<ul>\n<li>Hardware-specific optimization guides for each generation of GPU. H100, A100, and T4 each have different compute capabilities, shared memory sizes, and bandwidth profiles</li>\n<li>In Libraries, <code>diffusers</code> and <code>transformers</code> have different module hierarchies, normalization conventions, and integration patterns. Custom kernels need to be registered in PyTorch for <code>torch.compile</code> to recognize.</li>\n<li>For distribution, kernels can depend on CUDA, Pytorch, and Python versions creating massive environment matrices.</li>\n</ul>\n<p>This is domain knowledge that gets lost in documentation tabs and Stack Overflow answers. An agent skill packages it into context that loads on demand.</p>\n<p>First, let's show how to use the skill right away, then we'll dive into the details of how we benchmarked the kernels.</p>\n<h2>Installing the skill</h2>\n<p>The skill ships with the <code>kernels</code> library. Install it into your coding agent with a single command:</p>\n<pre><code class=\"language-shell\"><span class=\"code-snippet\" data-language=\"Shell\"><span class=\"syntax-source syntax-shell syntax-bash\"><span class=\"syntax-comment syntax-line syntax-number-sign syntax-shell\"><span class=\"syntax-punctuation syntax-definition syntax-comment syntax-begin syntax-shell\">#</span></span><span class=\"syntax-comment syntax-line syntax-number-sign syntax-shell\"> we need to install kernels from main for this</span><span class=\"syntax-comment syntax-line syntax-number-sign syntax-shell\">\n</span><span class=\"syntax-meta syntax-function-call syntax-shell\"><span class=\"syntax-variable syntax-function syntax-shell\">pip</span></span><span class=\"syntax-meta syntax-function-call syntax-arguments syntax-shell\"> install git+https://github.com/huggingface/kernels.git#subdirectory=kernels</span>\n<span class=\"syntax-meta syntax-function-call syntax-shell\"><span class=\"syntax-variable syntax-function syntax-shell\">kernels</span></span><span class=\"syntax-meta syntax-function-call syntax-arguments syntax-shell\"> skills add cuda-kernels<span class=\"syntax-variable syntax-parameter syntax-option syntax-shell\"><span class=\"syntax-punctuation syntax-definition syntax-parameter syntax-shell\"> --</span>claude</span></span>\n</span></span></code></pre>\n<p>This drops the skill into <code>.claude/skills/cuda-kernels/</code> where Claude Code and Cursor pick it up automatically. For other agents:</p>\n<pre><code class=\"language-shell\"><span class=\"code-snippet\" data-language=\"Shell\"><span class=\"syntax-source syntax-shell syntax-bash\"><span class=\"syntax-comment syntax-line syntax-number-sign syntax-shell\"><span class=\"syntax-punctuation syntax-definition syntax-comment syntax-begin syntax-shell\">#</span></span><span class=\"syntax-comment syntax-line syntax-number-sign syntax-shell\"> Codex</span><span class=\"syntax-comment syntax-line syntax-number-sign syntax-shell\">\n</span><span class=\"syntax-meta syntax-function-call syntax-shell\"><span class=\"syntax-variable syntax-function syntax-shell\">kernels</span></span><span class=\"syntax-meta syntax-function-call syntax-arguments syntax-shell\"> skills add cuda-kernels<span class=\"syntax-variable syntax-parameter syntax-option syntax-shell\"><span class=\"syntax-punctuation syntax-definition syntax-parameter syntax-shell\"> --</span>codex</span></span>\n\n<span class=\"syntax-comment syntax-line syntax-number-sign syntax-shell\"><span class=\"syntax-punctuation syntax-definition syntax-comment syntax-begin syntax-shell\">#</span></span><span class=\"syntax-comment syntax-line syntax-number-sign syntax-shell\"> OpenCode</span><span class=\"syntax-comment syntax-line syntax-number-sign syntax-shell\">\n</span><span class=\"syntax-meta syntax-function-call syntax-shell\"><span class=\"syntax-variable syntax-function syntax-shell\">kernels</span></span><span class=\"syntax-meta syntax-function-call syntax-arguments syntax-shell\"> skills add cuda-kernels<span class=\"syntax-variable syntax-parameter syntax-option syntax-shell\"><span class=\"syntax-punctuation syntax-definition syntax-parameter syntax-shell\"> --</span>opencode</span></span>\n\n<span class=\"syntax-comment syntax-line syntax-number-sign syntax-shell\"><span class=\"syntax-punctuation syntax-definition syntax-comment syntax-begin syntax-shell\">#</span></span><span class=\"syntax-comment syntax-line syntax-number-sign syntax-shell\"> Custom destination</span><span class=\"syntax-comment syntax-line syntax-number-sign syntax-shell\">\n</span><span class=\"syntax-meta syntax-function-call syntax-shell\"><span class=\"syntax-variable syntax-function syntax-shell\">kernels</span></span><span class=\"syntax-meta syntax-function-call syntax-arguments syntax-shell\"> skills add cuda-kernels<span class=\"syntax-variable syntax-parameter syntax-option syntax-shell\"><span class=\"syntax-punctuation syntax-definition syntax-parameter syntax-shell\"> --</span>dest</span> ./my-agent/skills/</span>\n\n<span class=\"syntax-comment syntax-line syntax-number-sign syntax-shell\"><span class=\"syntax-punctuation syntax-definition syntax-comment syntax-begin syntax-shell\">#</span></span><span class=\"syntax-comment syntax-line syntax-number-sign syntax-shell\"> Install globally (available across all projects)</span><span class=\"syntax-comment syntax-line syntax-number-sign syntax-shell\">\n</span><span class=\"syntax-meta syntax-function-call syntax-shell\"><span class=\"syntax-variable syntax-function syntax-shell\">kernels</span></span><span class=\"syntax-meta syntax-function-call syntax-arguments syntax-shell\"> skills add cuda-kernels<span class=\"syntax-variable syntax-parameter syntax-option syntax-shell\"><span class=\"syntax-punctuation syntax-definition syntax-parameter syntax-shell\"> --</span>global</span></span>\n\n<span class=\"syntax-comment syntax-line syntax-number-sign syntax-shell\"><span class=\"syntax-punctuation syntax-definition syntax-comment syntax-begin syntax-shell\">#</span></span><span class=\"syntax-comment syntax-line syntax-number-sign syntax-shell\"> Overwrite an existing installation</span><span class=\"syntax-comment syntax-line syntax-number-sign syntax-shell\">\n</span><span class=\"syntax-meta syntax-function-call syntax-shell\"><span class=\"syntax-variable syntax-function syntax-shell\">kernels</span></span><span class=\"syntax-meta syntax-function-call syntax-arguments syntax-shell\"> skills add cuda-kernels<span class=\"syntax-variable syntax-parameter syntax-option syntax-shell\"><span class=\"syntax-punctuation syntax-definition syntax-parameter syntax-shell\"> --</span>claude</span><span class=\"syntax-variable syntax-parameter syntax-option syntax-shell\"><span class=\"syntax-punctuation syntax-definition syntax-parameter syntax-shell\"> --</span>force</span></span>\n</span></span></code></pre>\n<p>Once installed, prompt your agent:</p>\n<pre><code><span class=\"code-snippet\">Build a vectorized RMSNorm kernel for H100 targeting the Qwen3-8B model in transformers.\n</span></code></pre>\n<p>Or, you can go for something more open-ended:</p>\n<pre><code><span class=\"code-snippet\">Build an optimized attention kernel for H100 targeting the Qwen3-8B model in transformers. Benchmark it against the PyTorch baseline and validate improvements in end-to-end performance.\n</span></code></pre>\n<p>The agent can read the skill, select the right architecture parameters, generate the CUDA source, write the PyTorch bindings, set up <code>build.toml</code>, and create a benchmark script.</p>\n<p>If you're working on more complex kernels, or architecture-specific optimizations, that aren't covered in the skill, then the skill supplies the fundamental building blocks and patterns to get you started. We are also open to contributions on the <a target=\"_blank\" rel=\"noopener noreferrer\" href=\"https://github.com/huggingface/kernels/tree/main/kernel-builder/skills\">skill itself</a>.</p>\n<h2>What is in the skill</h2>\n<p>The skill is roughly <strong>550 tokens</strong> of structured guidance plus reference scripts, GPU optimization guides, troubleshooting docs, and complete working examples. Agentic coding tools like Codex and Claude can read this and produce a working kernel project.</p>\n<p>It covers:</p>\n<ul>\n<li>NVIDIA GPU Architecture-aware optimization for H100, A100, and T4 (compute capabilities, memory bandwidth, shared memory sizes, block sizing)</li>\n<li>Integration patterns for both <code>diffusers</code> and <code>transformers</code>, including the pitfalls specific to each library</li>\n<li>Kernel templates with vectorized memory access patterns for BF16, FP16, and FP32</li>\n<li>Benchmarking workflows for both isolated kernel micro-benchmarks and end-to-end pipeline comparisons</li>\n<li>HuggingFace Kernel Hub integration via <code>get_kernel</code> for loading community kernels</li>\n</ul>\n<pre><code><span class=\"code-snippet\">.claude/skills/cuda-kernels/\n├── SKILL.md                              # Main instructions (~550 tokens)\n├── scripts/\n│   ├── benchmark_example.py              # End-to-end benchmark template\n│   ├── benchmark_rmsnorm.py              # Isolated kernel micro-benchmark\n│   ├── ltx_kernel_injection_example.py   # Diffusers integration pattern\n│   ├── transformers_injection_example.py # Transformers integration pattern\n│   └── huggingface_kernels_example.py    # Kernel Hub integration\n└── references/\n    ├── diffusers-integration.md          # Diffusers guide with pitfalls\n    ├── transformers-integration.md       # Transformers guide\n    ├── huggingface-kernels-integration.md\n    ├── h100-optimization-guide.md\n    ├── a100-optimization-guide.md\n    ├── t4-optimization-guide.md\n    ├── kernel-templates.md\n    └── troubleshooting.md\n</span></code></pre>\n<p>When an agent loads this, it gets everything it needs to go from &quot;write me an RMSNorm kernel&quot; to a buildable, benchmarkable project. It will grep and glob the skill to find the relevant files and directories. So it's important to structure the skill in a way that is easy to find.</p>\n<p>The agent is instructed to generate kernels that conform to the templates in <code>references/kernel-templates.md</code> and produce a complete kernel project:</p>\n<pre><code><span class=\"code-snippet\">examples/your_model/\n├── kernel_src/\n│   └── rmsnorm.cu              # Vectorized CUDA kernel\n├── torch-ext/\n│   ├── your_kernels/__init__.py\n│   └── torch_binding.cpp       # PyTorch C++ bindings\n├── benchmark_rmsnorm.py        # Micro-benchmark script\n├── build.toml                  # kernel-builder config\n├── setup.py                    # pip install -e .\n└── pyproject.toml\n</span></code></pre>\n<p>We tested this on two real targets.</p>\n<h2>Benchmarking the kernels: Diffusers (LTX-Video on H100)</h2>\n<p>The agent built RMSNorm, RoPE 3D, GEGLU, and AdaLN kernels for <a target=\"_blank\" rel=\"noopener noreferrer\" href=\"https://huggingface.co/Lightricks/LTX-Video\">LTX-Video</a>, a video generation pipeline from <code>diffusers</code>. The full example is at <code>examples/ltx_video/</code>. We optimized the RMSNorm kernel for H100. Both benchmarks were run on H100 80GB HBM3 at precision BFloat16.</p>\n<p>If you want to check out the generated kernel, got to <a target=\"_blank\" rel=\"noopener noreferrer\" href=\"https://github.com/burtenshaw/kernel-skill/tree/main/examples/ltx_video\">this example</a></p>\n<h3>Isolated RMSNorm benchmark</h3>\n<p>First, we compare the isolated RMSNorm kernel performance against the PyTorch baseline. This is the main speedup in the optimized pipeline.</p>\n<p><a target=\"_blank\" rel=\"noopener noreferrer\" href=\"https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/kernels-skill-benchmark/rmsnorm_ltx_video.png\"><picture class=\"article-picture article-picture-fallback\" style=\"--image-ratio:16 / 9\"><img src=\"https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/kernels-skill-benchmark/rmsnorm_ltx_video.png\" alt=\"isolated rmsnorm benchmark ltx-video\" loading=\"lazy\" decoding=\"async\" fetchpriority=\"low\" referrerpolicy=\"no-referrer\" class=\"progressive-image\"></picture></a></p>\n<p>Table</p>\n<table>\n<thead>\n<tr>\n<th>Shape</th>\n<th>Custom (ms)</th>\n<th>PyTorch (ms)</th>\n<th>Speedup</th>\n</tr>\n</thead>\n<tbody>\n<tr>\n<td>[1x1024x2048]</td>\n<td>0.039</td>\n<td>0.064</td>\n<td><strong>1.64x</strong></td>\n</tr>\n<tr>\n<td>[2x1024x2048]</td>\n<td>0.040</td>\n<td>0.073</td>\n<td><strong>1.82x</strong></td>\n</tr>\n<tr>\n<td>[4x1024x2048]</td>\n<td>0.052</td>\n<td>0.093</td>\n<td><strong>1.78x</strong></td>\n</tr>\n<tr>\n<td>[1x4096x2048]</td>\n<td>0.052</td>\n<td>0.093</td>\n<td><strong>1.79x</strong></td>\n</tr>\n<tr>\n<td>[2x4096x3072]</td>\n<td>0.102</td>\n<td>0.209</td>\n<td><strong>2.04x</strong></td>\n</tr>\n<tr>\n<td>[1x8192x2048]</td>\n<td>0.083</td>\n<td>0.150</td>\n<td><strong>1.81x</strong></td>\n</tr>\n<tr>\n<td>[4x4096x3072]</td>\n<td>0.173</td>\n<td>0.393</td>\n<td><strong>2.26x</strong></td>\n</tr>\n</tbody>\n</table>\n<p><strong>Average speedup: 1.88x</strong> and a bandwidth efficiency: 34.7% of H100 theoretical (3,350 GB/s)</p>\n<h3>End-to-end video generation (49 frames, 30 steps, H100 80GB)</h3>\n<p>Next, we compare the end-to-end video generation performance of the optimized kernels against the baseline (no compile) and the <code>torch.compile</code> baseline.</p>\n<p><a target=\"_blank\" rel=\"noopener noreferrer\" href=\"https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/kernels-skill-benchmark/e2e_ltx_video.png\"><picture class=\"article-picture article-picture-fallback\" style=\"--image-ratio:16 / 9\"><img src=\"https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/kernels-skill-benchmark/e2e_ltx_video.png\" alt=\"e2e benchmark ltx-video\" loading=\"lazy\" decoding=\"async\" fetchpriority=\"low\" referrerpolicy=\"no-referrer\" class=\"progressive-image\"></picture></a></p>\n<p>Table</p>\n<table>\n<thead>\n<tr>\n<th>Configuration</th>\n<th>Time (s)</th>\n<th>it/s</th>\n<th>Speedup</th>\n</tr>\n</thead>\n<tbody>\n<tr>\n<td>Baseline (no compile)</td>\n<td>2.87</td>\n<td>12.58</td>\n<td>1.00x</td>\n</tr>\n<tr>\n<td><strong>Generated Optimized Kernels</strong></td>\n<td>2.70</td>\n<td>13.52</td>\n<td><strong>1.06x</strong></td>\n</tr>\n<tr>\n<td>Baseline + torch.compile</td>\n<td>2.14</td>\n<td>19.05</td>\n<td>1.34x</td>\n</tr>\n<tr>\n<td>Optimized + torch.compile</td>\n<td>2.01</td>\n<td>18.45</td>\n<td>1.43x</td>\n</tr>\n</tbody>\n</table>\n<p>RMSNorm accounts for ~5% of total compute in LTX-Video. The remaining time is spent in attention, linear projections, and VAE decode. The 6% end-to-end speedup from a single kernel type is consistent with that profile.</p>\n<h2>Benchmarking the kernels: Transformers (Qwen3-8B on H100)</h2>\n<p>The agent built an RMSNorm kernel for <a target=\"_blank\" rel=\"noopener noreferrer\" href=\"https://huggingface.co/Qwen/Qwen3-8B\">Qwen3-8B</a>, a large language model from <code>transformers</code> with 65 RMSNorm modules across 32 layers. The full example is at <code>examples/qwen3_8b/</code>. We optimized the RMSNorm kernel for H100. Both benchmarks were run on H100 80GB HBM3 at precision BFloat16.</p>\n<p>If you want to explore the kernel, check it out <a target=\"_blank\" rel=\"noopener noreferrer\" href=\"https://github.com/burtenshaw/kernel-skill/tree/main/examples/qwen3_8b\">here.</a></p>\n<h3>Isolated RMSNorm benchmark</h3>\n<p>Once again, we compare the isolated RMSNorm kernel performance against the PyTorch baseline.</p>\n<p><a target=\"_blank\" rel=\"noopener noreferrer\" href=\"https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/kernels-skill-benchmark/rmsnorm_qwen3.png\"><picture class=\"article-picture article-picture-fallback\" style=\"--image-ratio:16 / 9\"><img src=\"https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/kernels-skill-benchmark/rmsnorm_qwen3.png\" alt=\"isolated rmsnorm benchmark qwen3-8b\" loading=\"lazy\" decoding=\"async\" fetchpriority=\"low\" referrerpolicy=\"no-referrer\" class=\"progressive-image\"></picture></a></p>\n<p><strong>Average speedup: 1.94x</strong> and a bandwidth efficiency: 22.3% of H100 theoretical (3,350 GB/s)</p>\n<p>Table</p>\n<table>\n<thead>\n<tr>\n<th>Shape</th>\n<th>Custom (ms)</th>\n<th>PyTorch (ms)</th>\n<th>Speedup</th>\n</tr>\n</thead>\n<tbody>\n<tr>\n<td>[1x128x4096]</td>\n<td>0.040</td>\n<td>0.062</td>\n<td><strong>1.58x</strong></td>\n</tr>\n<tr>\n<td>[1x512x4096]</td>\n<td>0.038</td>\n<td>0.064</td>\n<td><strong>1.69x</strong></td>\n</tr>\n<tr>\n<td>[1x1024x4096]</td>\n<td>0.037</td>\n<td>0.071</td>\n<td><strong>1.90x</strong></td>\n</tr>\n<tr>\n<td>[1x2048x4096]</td>\n<td>0.045</td>\n<td>0.091</td>\n<td><strong>2.03x</strong></td>\n</tr>\n<tr>\n<td>[1x4096x4096]</td>\n<td>0.071</td>\n<td>0.150</td>\n<td><strong>2.12x</strong></td>\n</tr>\n<tr>\n<td>[4x512x4096]</td>\n<td>0.056</td>\n<td>0.093</td>\n<td><strong>1.67x</strong></td>\n</tr>\n<tr>\n<td>[8x256x4096]</td>\n<td>0.045</td>\n<td>0.092</td>\n<td><strong>2.06x</strong></td>\n</tr>\n<tr>\n<td>[1x8192x4096]</td>\n<td>0.109</td>\n<td>0.269</td>\n<td><strong>2.47x</strong></td>\n</tr>\n</tbody>\n</table>\n<p>Speedup scales with sequence length: 1.58x at 128 tokens, 2.47x at 8192 tokens. For long-context inference, the custom kernel roughly halves RMSNorm latency.</p>\n<h2>Publishing your kernel to the Hub</h2>\n<p>The agent gives you a working kernel. The <a target=\"_blank\" rel=\"noopener noreferrer\" href=\"https://huggingface.co/kernels-community\">Kernel Hub</a> lets you share it so anyone can load it without compilation. Here is the full path from agent output to published kernel.</p>\n<h3>1. Verify the project structure</h3>\n<p>The agent produces a project that already follows the <a target=\"_blank\" rel=\"noopener noreferrer\" href=\"https://huggingface.co/docs/kernels/en/builder/writing-kernels\">kernel-builder</a> layout:</p>\n<pre><code><span class=\"code-snippet\">your_kernel/\n├── build.toml               # Build configuration\n├── kernel_src/\n│   └── rmsnorm.cu           # CUDA kernel source\n└── torch-ext/\n    ├── torch_binding.cpp    # Registers Torch ops\n    └── your_kernels/\n        └── __init__.py      # Python API wrapping _ops\n</span></code></pre>\n<p>The <code>build.toml</code> tells <code>kernel-builder</code> what to build. The agent generates this for you, including the correct <code>cuda-capabilities</code> for your target GPU:</p>\n<pre><code><span class=\"code-snippet\" data-language=\"TOML\">[general]\nname = &quot;your_kernels&quot;\nbackends = [&quot;cuda&quot;]\n\n[torch]\nsrc = [&quot;torch-ext/torch_binding.cpp&quot;]\n\n[kernel.rmsnorm]\nbackend = &quot;cuda&quot;\nsrc = [&quot;kernel_src/rmsnorm.cu&quot;]\ndepends = [&quot;torch&quot;]\ncuda-capabilities = [&quot;9.0&quot;]  # H100\n</span></code></pre>\n<h3>2. Build all variants with Nix</h3>\n<p>Kernel Hub kernels must support all recent PyTorch and CUDA configurations. The kernel-builder Nix flake handles this automatically. Copy the <a target=\"_blank\" rel=\"noopener noreferrer\" href=\"https://github.com/huggingface/kernels/blob/main/examples/kernels/relu/flake.nix\">example <code>flake.nix</code></a> into your project and run:</p>\n<pre><code class=\"language-shell\"><span class=\"code-snippet\" data-language=\"Shell\"><span class=\"syntax-source syntax-shell syntax-bash\"><span class=\"syntax-meta syntax-function-call syntax-shell\"><span class=\"syntax-variable syntax-function syntax-shell\">nix</span></span><span class=\"syntax-meta syntax-function-call syntax-arguments syntax-shell\"> flake update</span>\n<span class=\"syntax-meta syntax-function-call syntax-shell\"><span class=\"syntax-variable syntax-function syntax-shell\">nix</span></span><span class=\"syntax-meta syntax-function-call syntax-arguments syntax-shell\"> run .#build-and-copy<span class=\"syntax-variable syntax-parameter syntax-option syntax-shell\"><span class=\"syntax-punctuation syntax-definition syntax-parameter syntax-shell\"> -</span>L</span></span>\n</span></span></code></pre>\n<p>This builds the kernel for every required PyTorch/CUDA variant and places the results in <code>build/</code>. For faster builds, enable the HuggingFace Nix cache:</p>\n<pre><code class=\"language-shell\"><span class=\"code-snippet\" data-language=\"Shell\"><span class=\"syntax-source syntax-shell syntax-bash\"><span class=\"syntax-meta syntax-function-call syntax-shell\"><span class=\"syntax-variable syntax-function syntax-shell\">nix</span></span><span class=\"syntax-meta syntax-function-call syntax-arguments syntax-shell\"> run nixpkgs#cachix<span class=\"syntax-keyword syntax-operator syntax-end-of-options syntax-shell\"> --</span></span><span class=\"syntax-meta syntax-function-call syntax-arguments syntax-shell\"> use huggingface</span>\n</span></span></code></pre>\n<h3>3. Create a Hub repo and push</h3>\n<p>Create a model repo on the Hub and upload the built kernel:</p>\n<pre><code class=\"language-shell\"><span class=\"code-snippet\" data-language=\"Shell\"><span class=\"syntax-source syntax-shell syntax-bash\"><span class=\"syntax-meta syntax-function-call syntax-shell\"><span class=\"syntax-variable syntax-function syntax-shell\">huggingface-cli</span></span><span class=\"syntax-meta syntax-function-call syntax-arguments syntax-shell\"> repo create your-org/your-kernel<span class=\"syntax-variable syntax-parameter syntax-option syntax-shell\"><span class=\"syntax-punctuation syntax-definition syntax-parameter syntax-shell\"> --</span>type</span> model</span>\n<span class=\"syntax-meta syntax-function-call syntax-shell\"><span class=\"syntax-variable syntax-function syntax-shell\">huggingface-cli</span></span><span class=\"syntax-meta syntax-function-call syntax-arguments syntax-shell\"> upload your-org/your-kernel ./build</span>\n</span></span></code></pre>\n<h3>4. Others load it in one line</h3>\n<p>Once published, anyone can use your kernel with zero compilation:</p>\n<pre><code class=\"language-py\"><span class=\"code-snippet\" data-language=\"Python\"><span class=\"syntax-source syntax-python\"><span class=\"syntax-meta syntax-statement syntax-import syntax-python\"><span class=\"syntax-keyword syntax-control syntax-import syntax-from syntax-python\">from</span></span><span class=\"syntax-meta syntax-statement syntax-import syntax-python\"><span class=\"syntax-meta syntax-import-source syntax-python\"> <span class=\"syntax-meta syntax-import-path syntax-python\"><span class=\"syntax-meta syntax-import-name syntax-python\">kernels</span></span> <span class=\"syntax-meta syntax-statement syntax-import syntax-python\"><span class=\"syntax-keyword syntax-control syntax-import syntax-python\">import</span></span></span></span><span class=\"syntax-meta syntax-statement syntax-import syntax-python\"></span><span class=\"syntax-meta syntax-statement syntax-import syntax-python\"> <span class=\"syntax-meta syntax-generic-name syntax-python\">get_kernel</span></span>\n\n<span class=\"syntax-meta syntax-qualified-name syntax-python\"><span class=\"syntax-meta syntax-generic-name syntax-python\">rmsnorm</span></span> <span class=\"syntax-keyword syntax-operator syntax-assignment syntax-python\">=</span> <span class=\"syntax-meta syntax-function-call syntax-python\"><span class=\"syntax-meta syntax-qualified-name syntax-python\"><span class=\"syntax-variable syntax-function syntax-python\">get_kernel</span></span><span class=\"syntax-punctuation syntax-section syntax-arguments syntax-begin syntax-python\">(</span><span class=\"syntax-meta syntax-function-call syntax-arguments syntax-python\"><span class=\"syntax-meta syntax-string syntax-python\"><span class=\"syntax-string syntax-quoted syntax-double syntax-python\"><span class=\"syntax-punctuation syntax-definition syntax-string syntax-begin syntax-python\">&quot;</span></span></span><span class=\"syntax-meta syntax-string syntax-python\"><span class=\"syntax-string syntax-quoted syntax-double syntax-python\">your-org/your-kernel<span class=\"syntax-punctuation syntax-definition syntax-string syntax-end syntax-python\">&quot;</span></span></span></span><span class=\"syntax-punctuation syntax-section syntax-arguments syntax-end syntax-python\">)</span></span>\n</span></span></code></pre>\n<p><code>get_kernel</code> detects the user's Python, PyTorch, and CUDA versions and downloads the matching pre-compiled binary. No builds, no flags, typically ready in seconds.</p>\n<p>The skill and the Hub are complementary. The skill handles development. The Hub handles distribution. Build a kernel with the skill, validate it with the benchmark scripts, publish it to the Hub, and it becomes a one-liner for everyone else.</p>\n<h2>Conclusion</h2>\n<p>We built an agent skill that teaches coding agents how to write production CUDA kernels. Then we pointed Claude and Codex at two real targets: a <strong>diffusers</strong> pipeline and a <strong>transformers</strong> model. The agents produced working kernels for both, with correct PyTorch bindings and benchmarks, end to end. We benchmarked the kernels and found that the optimized kernels can provide a speedup in both isolated and end-to-end performance.</p>\n<h2>Resources</h2>\n<ul>\n<li><a target=\"_blank\" rel=\"noopener noreferrer\" href=\"https://github.com/huggingface/kernels/tree/main/kernel-builder/skills/cuda-kernels\">CUDA Kernels Skill in <code>kernels</code></a></li>\n<li><a target=\"_blank\" rel=\"noopener noreferrer\" href=\"https://huggingface.co/blog/hello-hf-kernels\">HuggingFace Kernel Hub Blog</a></li>\n<li><a target=\"_blank\" rel=\"noopener noreferrer\" href=\"https://huggingface.co/blog/hf-skills-training\">We Got Claude to Fine-Tune an Open Source LLM</a></li>\n<li><a target=\"_blank\" rel=\"noopener noreferrer\" href=\"https://huggingface.co/blog/upskill\">We Got Claude to Teach Open Models</a></li>\n<li><a target=\"_blank\" rel=\"noopener noreferrer\" href=\"https://huggingface.co/kernels-community\">HuggingFace Kernels Community</a></li>\n</ul>\n",
  "content_markdown": "[![oprah custom cuda kernels](https://huggingface.co/blog/assets/custom-cuda-kernels/meme.png)](https://huggingface.co/blog/assets/custom-cuda-kernels/meme.png)\n\ntl;dr: We built an agent skill that teaches coding agents how to write production CUDA kernels. Then we pointed Claude and Codex at two real targets: a **diffusers** pipeline and a **transformers** model. The agents produced working kernels for both, with correct PyTorch bindings and benchmarks, end to end.\n\nWriting CUDA kernels is hard. Writing CUDA kernels that correctly integrate with `transformers` and `diffusers` is harder. There are architecture-specific memory access patterns, vectorization strategies, warp shuffle reductions, and a dozen integration pitfalls that trip up even experienced developers. It is exactly the kind of specialized, high-stakes problem where agent skills shine.\n\nWe gave coding agents the domain knowledge they need, like which GPU architecture to target, how to structure a kernel-builder project, when to use shared memory versus registers, and how to write PyTorch bindings. The agents did the rest. If you have used the [LLM training skill](https://huggingface.co/blog/hf-skills-training) or read [We Got Claude to Teach Open Models](https://huggingface.co/blog/upskill), the pattern will feel familiar: package domain expertise into a skill, point the agent at a problem, and let it work.\n\n## Why a skill for kernels?\n\nThe [Kernel Hub](https://huggingface.co/blog/hello-hf-kernels) solved the distribution of custom hardware kernels. You can load pre-compiled kernels from the Hub with a single `get_kernel` call. No builds, no flags. However, someone still needs to **write the kernels**. That is the gap this skill fills.\n\nCUDA kernel development has a brutal surface area:\n\n- Hardware-specific optimization guides for each generation of GPU. H100, A100, and T4 each have different compute capabilities, shared memory sizes, and bandwidth profiles\n- In Libraries, `diffusers` and `transformers` have different module hierarchies, normalization conventions, and integration patterns. Custom kernels need to be registered in PyTorch for `torch.compile` to recognize.\n- For distribution, kernels can depend on CUDA, Pytorch, and Python versions creating massive environment matrices.\n\nThis is domain knowledge that gets lost in documentation tabs and Stack Overflow answers. An agent skill packages it into context that loads on demand.\n\nFirst, let's show how to use the skill right away, then we'll dive into the details of how we benchmarked the kernels.\n\n## Installing the skill\n\nThe skill ships with the `kernels` library. Install it into your coding agent with a single command:\n\n```shell\n# we need to install kernels from main for this\npip install git+https://github.com/huggingface/kernels.git#subdirectory=kernels\nkernels skills add cuda-kernels --claude\n```\n\nThis drops the skill into `.claude/skills/cuda-kernels/` where Claude Code and Cursor pick it up automatically. For other agents:\n\n```shell\n# Codex\nkernels skills add cuda-kernels --codex\n\n# OpenCode\nkernels skills add cuda-kernels --opencode\n\n# Custom destination\nkernels skills add cuda-kernels --dest ./my-agent/skills/\n\n# Install globally (available across all projects)\nkernels skills add cuda-kernels --global\n\n# Overwrite an existing installation\nkernels skills add cuda-kernels --claude --force\n```\n\nOnce installed, prompt your agent:\n\n```\nBuild a vectorized RMSNorm kernel for H100 targeting the Qwen3-8B model in transformers.\n```\n\nOr, you can go for something more open-ended:\n\n```\nBuild an optimized attention kernel for H100 targeting the Qwen3-8B model in transformers. Benchmark it against the PyTorch baseline and validate improvements in end-to-end performance.\n```\n\nThe agent can read the skill, select the right architecture parameters, generate the CUDA source, write the PyTorch bindings, set up `build.toml`, and create a benchmark script.\n\nIf you're working on more complex kernels, or architecture-specific optimizations, that aren't covered in the skill, then the skill supplies the fundamental building blocks and patterns to get you started. We are also open to contributions on the [skill itself](https://github.com/huggingface/kernels/tree/main/kernel-builder/skills).\n\n## What is in the skill\n\nThe skill is roughly **550 tokens** of structured guidance plus reference scripts, GPU optimization guides, troubleshooting docs, and complete working examples. Agentic coding tools like Codex and Claude can read this and produce a working kernel project.\n\nIt covers:\n\n- NVIDIA GPU Architecture-aware optimization for H100, A100, and T4 (compute capabilities, memory bandwidth, shared memory sizes, block sizing)\n- Integration patterns for both `diffusers` and `transformers`, including the pitfalls specific to each library\n- Kernel templates with vectorized memory access patterns for BF16, FP16, and FP32\n- Benchmarking workflows for both isolated kernel micro-benchmarks and end-to-end pipeline comparisons\n- HuggingFace Kernel Hub integration via `get_kernel` for loading community kernels\n\n```\n.claude/skills/cuda-kernels/\n├── SKILL.md                              # Main instructions (~550 tokens)\n├── scripts/\n│   ├── benchmark_example.py              # End-to-end benchmark template\n│   ├── benchmark_rmsnorm.py              # Isolated kernel micro-benchmark\n│   ├── ltx_kernel_injection_example.py   # Diffusers integration pattern\n│   ├── transformers_injection_example.py # Transformers integration pattern\n│   └── huggingface_kernels_example.py    # Kernel Hub integration\n└── references/\n    ├── diffusers-integration.md          # Diffusers guide with pitfalls\n    ├── transformers-integration.md       # Transformers guide\n    ├── huggingface-kernels-integration.md\n    ├── h100-optimization-guide.md\n    ├── a100-optimization-guide.md\n    ├── t4-optimization-guide.md\n    ├── kernel-templates.md\n    └── troubleshooting.md\n```\n\nWhen an agent loads this, it gets everything it needs to go from \"write me an RMSNorm kernel\" to a buildable, benchmarkable project. It will grep and glob the skill to find the relevant files and directories. So it's important to structure the skill in a way that is easy to find.\n\nThe agent is instructed to generate kernels that conform to the templates in `references/kernel-templates.md` and produce a complete kernel project:\n\n```\nexamples/your_model/\n├── kernel_src/\n│   └── rmsnorm.cu              # Vectorized CUDA kernel\n├── torch-ext/\n│   ├── your_kernels/__init__.py\n│   └── torch_binding.cpp       # PyTorch C++ bindings\n├── benchmark_rmsnorm.py        # Micro-benchmark script\n├── build.toml                  # kernel-builder config\n├── setup.py                    # pip install -e .\n└── pyproject.toml\n```\n\nWe tested this on two real targets.\n\n## Benchmarking the kernels: Diffusers (LTX-Video on H100)\n\nThe agent built RMSNorm, RoPE 3D, GEGLU, and AdaLN kernels for [LTX-Video](https://huggingface.co/Lightricks/LTX-Video), a video generation pipeline from `diffusers`. The full example is at `examples/ltx_video/`. We optimized the RMSNorm kernel for H100. Both benchmarks were run on H100 80GB HBM3 at precision BFloat16.\n\nIf you want to check out the generated kernel, got to [this example](https://github.com/burtenshaw/kernel-skill/tree/main/examples/ltx_video)\n\n### Isolated RMSNorm benchmark\n\nFirst, we compare the isolated RMSNorm kernel performance against the PyTorch baseline. This is the main speedup in the optimized pipeline.\n\n[![isolated rmsnorm benchmark ltx-video](https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/kernels-skill-benchmark/rmsnorm_ltx_video.png)](https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/kernels-skill-benchmark/rmsnorm_ltx_video.png)\n\nTable\n\n| Shape           | Custom (ms) | PyTorch (ms) | Speedup   |\n| --------------- | ----------- | ------------ | --------- |\n| \\[1x1024x2048\\] | 0.039       | 0.064        | **1.64x** |\n| \\[2x1024x2048\\] | 0.040       | 0.073        | **1.82x** |\n| \\[4x1024x2048\\] | 0.052       | 0.093        | **1.78x** |\n| \\[1x4096x2048\\] | 0.052       | 0.093        | **1.79x** |\n| \\[2x4096x3072\\] | 0.102       | 0.209        | **2.04x** |\n| \\[1x8192x2048\\] | 0.083       | 0.150        | **1.81x** |\n| \\[4x4096x3072\\] | 0.173       | 0.393        | **2.26x** |\n\n**Average speedup: 1.88x** and a bandwidth efficiency: 34.7% of H100 theoretical (3,350 GB/s)\n\n### End-to-end video generation (49 frames, 30 steps, H100 80GB)\n\nNext, we compare the end-to-end video generation performance of the optimized kernels against the baseline (no compile) and the `torch.compile` baseline.\n\n[![e2e benchmark ltx-video](https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/kernels-skill-benchmark/e2e_ltx_video.png)](https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/kernels-skill-benchmark/e2e_ltx_video.png)\n\nTable\n\n| Configuration                   | Time (s) | it/s  | Speedup   |\n| ------------------------------- | -------- | ----- | --------- |\n| Baseline (no compile)           | 2.87     | 12.58 | 1.00x     |\n| **Generated Optimized Kernels** | 2.70     | 13.52 | **1.06x** |\n| Baseline + torch.compile        | 2.14     | 19.05 | 1.34x     |\n| Optimized + torch.compile       | 2.01     | 18.45 | 1.43x     |\n\nRMSNorm accounts for ~5% of total compute in LTX-Video. The remaining time is spent in attention, linear projections, and VAE decode. The 6% end-to-end speedup from a single kernel type is consistent with that profile.\n\n## Benchmarking the kernels: Transformers (Qwen3-8B on H100)\n\nThe agent built an RMSNorm kernel for [Qwen3-8B](https://huggingface.co/Qwen/Qwen3-8B), a large language model from `transformers` with 65 RMSNorm modules across 32 layers. The full example is at `examples/qwen3_8b/`. We optimized the RMSNorm kernel for H100. Both benchmarks were run on H100 80GB HBM3 at precision BFloat16.\n\nIf you want to explore the kernel, check it out [here.](https://github.com/burtenshaw/kernel-skill/tree/main/examples/qwen3_8b)\n\n### Isolated RMSNorm benchmark\n\nOnce again, we compare the isolated RMSNorm kernel performance against the PyTorch baseline.\n\n[![isolated rmsnorm benchmark qwen3-8b](https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/kernels-skill-benchmark/rmsnorm_qwen3.png)](https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/kernels-skill-benchmark/rmsnorm_qwen3.png)\n\n**Average speedup: 1.94x** and a bandwidth efficiency: 22.3% of H100 theoretical (3,350 GB/s)\n\nTable\n\n| Shape           | Custom (ms) | PyTorch (ms) | Speedup   |\n| --------------- | ----------- | ------------ | --------- |\n| \\[1x128x4096\\]  | 0.040       | 0.062        | **1.58x** |\n| \\[1x512x4096\\]  | 0.038       | 0.064        | **1.69x** |\n| \\[1x1024x4096\\] | 0.037       | 0.071        | **1.90x** |\n| \\[1x2048x4096\\] | 0.045       | 0.091        | **2.03x** |\n| \\[1x4096x4096\\] | 0.071       | 0.150        | **2.12x** |\n| \\[4x512x4096\\]  | 0.056       | 0.093        | **1.67x** |\n| \\[8x256x4096\\]  | 0.045       | 0.092        | **2.06x** |\n| \\[1x8192x4096\\] | 0.109       | 0.269        | **2.47x** |\n\nSpeedup scales with sequence length: 1.58x at 128 tokens, 2.47x at 8192 tokens. For long-context inference, the custom kernel roughly halves RMSNorm latency.\n\n## Publishing your kernel to the Hub\n\nThe agent gives you a working kernel. The [Kernel Hub](https://huggingface.co/kernels-community) lets you share it so anyone can load it without compilation. Here is the full path from agent output to published kernel.\n\n### 1. Verify the project structure\n\nThe agent produces a project that already follows the [kernel-builder](https://huggingface.co/docs/kernels/en/builder/writing-kernels) layout:\n\n```\nyour_kernel/\n├── build.toml               # Build configuration\n├── kernel_src/\n│   └── rmsnorm.cu           # CUDA kernel source\n└── torch-ext/\n    ├── torch_binding.cpp    # Registers Torch ops\n    └── your_kernels/\n        └── __init__.py      # Python API wrapping _ops\n```\n\nThe `build.toml` tells `kernel-builder` what to build. The agent generates this for you, including the correct `cuda-capabilities` for your target GPU:\n\n```\n[general]\nname = \"your_kernels\"\nbackends = [\"cuda\"]\n\n[torch]\nsrc = [\"torch-ext/torch_binding.cpp\"]\n\n[kernel.rmsnorm]\nbackend = \"cuda\"\nsrc = [\"kernel_src/rmsnorm.cu\"]\ndepends = [\"torch\"]\ncuda-capabilities = [\"9.0\"]  # H100\n```\n\n### 2. Build all variants with Nix\n\nKernel Hub kernels must support all recent PyTorch and CUDA configurations. The kernel-builder Nix flake handles this automatically. Copy the [example `flake.nix`](https://github.com/huggingface/kernels/blob/main/examples/kernels/relu/flake.nix) into your project and run:\n\n```shell\nnix flake update\nnix run .#build-and-copy -L\n```\n\nThis builds the kernel for every required PyTorch/CUDA variant and places the results in `build/`. For faster builds, enable the HuggingFace Nix cache:\n\n```shell\nnix run nixpkgs#cachix -- use huggingface\n```\n\n### 3. Create a Hub repo and push\n\nCreate a model repo on the Hub and upload the built kernel:\n\n```shell\nhuggingface-cli repo create your-org/your-kernel --type model\nhuggingface-cli upload your-org/your-kernel ./build\n```\n\n### 4. Others load it in one line\n\nOnce published, anyone can use your kernel with zero compilation:\n\n```py\nfrom kernels import get_kernel\n\nrmsnorm = get_kernel(\"your-org/your-kernel\")\n```\n\n`get_kernel` detects the user's Python, PyTorch, and CUDA versions and downloads the matching pre-compiled binary. No builds, no flags, typically ready in seconds.\n\nThe skill and the Hub are complementary. The skill handles development. The Hub handles distribution. Build a kernel with the skill, validate it with the benchmark scripts, publish it to the Hub, and it becomes a one-liner for everyone else.\n\n## Conclusion\n\nWe built an agent skill that teaches coding agents how to write production CUDA kernels. Then we pointed Claude and Codex at two real targets: a **diffusers** pipeline and a **transformers** model. The agents produced working kernels for both, with correct PyTorch bindings and benchmarks, end to end. We benchmarked the kernels and found that the optimized kernels can provide a speedup in both isolated and end-to-end performance.\n\n## Resources\n\n- [CUDA Kernels Skill in `kernels`](https://github.com/huggingface/kernels/tree/main/kernel-builder/skills/cuda-kernels)\n- [HuggingFace Kernel Hub Blog](https://huggingface.co/blog/hello-hf-kernels)\n- [We Got Claude to Fine-Tune an Open Source LLM](https://huggingface.co/blog/hf-skills-training)\n- [We Got Claude to Teach Open Models](https://huggingface.co/blog/upskill)\n- [HuggingFace Kernels Community](https://huggingface.co/kernels-community)\n",
  "dateCreated": "2026-10-04T12:50:30.804479567+00:00",
  "date_created": "2026-02-13T00:00:00+00:00",
  "date_modified": "2026-02-13T00:00:00+00:00",
  "date_published": "2026-02-13T00:00:00+00:00",
  "description": "tl;dr: We built an agent skill that teaches coding agents how to write production CUDA kernels. Then we pointed Claude and Codex at two real targets: a diffusers pipeline and a transformers model. The agents produced working kernels for…",
  "external_url": "https://huggingface.co/blog/custom-cuda-kernels-agent-skills",
  "git": {
    "commit": "ebb0941c5c06a2ab2377342d607535262a3e969e",
    "path": "items/huggingface-co/2026/02/2026-02-13-custom-kernels-for-all-from-codex-and-claude.md",
    "permalink": "https://github.com/aymericbeaumet/aggr-instance/blob/ebb0941c5c06a2ab2377342d607535262a3e969e/items/huggingface-co/2026/02/2026-02-13-custom-kernels-for-all-from-codex-and-claude.md"
  },
  "id": "urn:aggr:item:4f4f00474f497b531f6b9d4ac9ec914b23361d77",
  "isBasedOn": {
    "@id": "https://huggingface.co/blog/custom-cuda-kernels-agent-skills"
  },
  "mainEntity": {
    "@id": "https://huggingface.co/blog/custom-cuda-kernels-agent-skills",
    "@type": "CreativeWork",
    "archivedAt": {
      "@id": "https://aggr.aymericbeaumet.com/items/huggingface-co/2026-02-13-custom-kernels-for-all-from-codex-and-claude/#webpage"
    },
    "dateModified": "2026-02-13T00:00:00+00:00",
    "datePublished": "2026-02-13T00:00:00+00:00",
    "headline": "Custom Kernels for All from Codex and Claude",
    "timeRequired": "PT8M",
    "url": "https://huggingface.co/blog/custom-cuda-kernels-agent-skills",
    "wordCount": 1692
  },
  "name": "Archived snapshot: Custom Kernels for All from Codex and Claude",
  "preview": null,
  "reading_minutes": 8,
  "replicated_at": null,
  "source": "huggingface-co",
  "source_name": "Hugging Face - Blog",
  "summary": null,
  "tags": [],
  "temporalCoverage": "2026-10-04T12:50:30.804479567+00:00",
  "title": "Custom Kernels for All from Codex and Claude",
  "url": "https://aggr.aymericbeaumet.com/items/huggingface-co/2026-02-13-custom-kernels-for-all-from-codex-and-claude/",
  "word_count": 1692
}