---
title: ZML/LLMD 20260929.0
link: https://zml.ai/posts/llmd-20260929.0/
source: zml-ai
published: 2026-09-29T07:00:00Z
updated: 2026-09-29T07:00:00Z
first_seen: 2026-09-29T23:14:03.129222144Z
summary: 'We’re happy to announce the release of zml/llmd 20260929.0, which notably brings deepseek-ai/DeepSeek-V4.1-Flash support on NVIDIA and AMD GPUs. Models This release adds 2 model families: deepseek-ai/DeepSeek-V4.1-Flash (with native DSpark support) meta-models/Muse-Glimmer-30B As stated, DeepSeek is only available on platforms with enough VRAM to host it natively. We do not plan to support custom quants in the near future. DeepSeek-V4.1-Flash Obviously the highlight, this is our first release supporting a frontier model. ZML is now mature enough to support complex frontier models, and we wanted to bring it to you. Bear in mind this is a 763B model, weighing 510GB, so for now we only support it on platforms LLMD supports with enough VRAM, namely NVIDIA and AMD GPUs.'
content: extracted
html: 2026-09-29-zml-llmd-20260929-0.html
preview:
  file: 2026-09-29-zml-llmd-20260929-0.preview-d5e701d32b35.webp
  width: 256
  height: 134
  alt: ZML/LLMD 20260929.0 — DeepSeek-V4.1-Flash lands on NVIDIA and AMD.
  color: '#f0f0ec'
images:
- source: https://zml.ai/og/background_hu_bcb0f50447ed32ff.png
  original:
    file: 2026-09-29-zml-llmd-20260929-0.image-fa5b02593683.png
    width: 1200
    height: 630
  color: '#f6f6f1'
---

Article

DeepSeek-V4.1-Flash lands on NVIDIA and AMD.

September 29, 2026 inference / serving / NVIDIA / AMD / DeepSeek

We’re happy to announce the release of `zml/llmd 20260929.0`, which notably brings [deepseek-ai/DeepSeek-V4.1-Flash](https://hf.co/deepseek-ai/DeepSeek-V4.1-Flash) support on NVIDIA and AMD GPUs.

## Models

This release adds 2 model families:

- [deepseek-ai/DeepSeek-V4.1-Flash](https://hf.co/deepseek-ai/DeepSeek-V4.1-Flash) (with native DSpark support)
- [meta-models/Muse-Glimmer-30B](https://hf.co/meta-models/Muse-Glimmer-30B)

As stated, DeepSeek is only available on platforms with enough VRAM to host it natively. We do not plan to support custom quants in the near future.

## DeepSeek-V4.1-Flash

Obviously the highlight, this is our first release supporting a frontier model. ZML is now mature enough to support complex frontier models, and we wanted to bring it to you. Bear in mind this is a 763B model, weighing 510GB, so for now we only support it on platforms LLMD supports with enough VRAM, namely NVIDIA and AMD GPUs.

That said, this is a cool release because it pushes the limits and shows how powerful ZML can be: MXFP4 MoEs, block-FP8, DSpark speculative decoding, DSA, HCA, and more. Nothing was spared. And it all works.

Sadly, we couldn’t bring it to TPUs because Google told us there were no TPUv7s available “for at least a few weeks”. LLMD doesn’t support Trainium yet due to an AWS Neuron bug that has since been fixed. We’ll revisit that.

On performance, we’re about 20% slower than vLLM and 30% slower than SGLang on 4xGB300. We could’ve waited to close the gap, but we wanted to bring it to you now. The roadmap is clear and we’ll get there, so why wait?

Token Generation, deepseek-ai/DeepSeek-V4.1-Flash, bs=1, 4xGB300 (tok/s)

Speculative decoding is also supported on DeepSeek-V4.1-Flash, and it works well. It can be enabled with the `--speculative-drafter-model=native` flag.

## Cold start

We’re pretty happy about our cold-start performance. We already have improvements that are not in this release that we can’t wait to share with you soon.

Cold Start, deepseek-ai/DeepSeek-V4.1-Flash, 4xGB300

## Bringing frontier IR support to ZML

It was always possible in [zml/zml](https://github.com/zml/zml) to emit custom kernels as MLIR bytecode using the built-in codegen builder. This allows to leverage ZML’s memory allocation, stream management, concurrency and sharding while going off the trail if needed. We used that feature to emit [Triton](https://github.com/zml/zml/tree/master/kernels/triton) and [Mosaic (Pallas)](https://github.com/zml/zml/tree/master/kernels/mosaic_tpu) kernels.

Since the last release, we have added support for [CuTe IR](https://github.com/zml/zml/tree/master/kernels/cute), [CUDA Tile IR](https://github.com/zml/zml/tree/master/kernels/cuda_tile), and [Fly IR](https://github.com/zml/zml/tree/master/kernels/fly).

Why does this matter? Well, because Triton cannot get the full performance of the underlying hardware. That’s why there’s [Gluon](https://github.com/triton-lang/triton/blob/main/python/tutorials/gluon/01-intro.py) now (which we’ll add support for soon).

On NVIDIA, CUDA Tile is faster than Triton, and CuTe is as fast as raw C++ CUDA (while still being a JIT). On AMD, Fly IR is faster than AMD’s preferred Triton path.

Using these new backends, we replaced the few custom kernels we had (MoE and PagedAttention) using their fastest respective IR. For instance, on NVIDIA, CuTe IR is as fast as C++ CUDA. So here goes.

As always with us, this was achieved with zero Python. No compromise.

Metal gains MXFP8, MXFP4 and NVFP4 block scaled dot products and block-FP8 MoE paths.

That means you can run NVFP4 models on Apple Silicon:

```
$ llmd --model=hf://nvidia/Qwen3.8-27B-NVFP4 --batch-size=1
```

The performance is nice, too:

Apple M3 Max, nvidia/Qwen3.8-27B-NVFP4, 1024/128, bs=1 (tg tok/s)

## Various improvements

## AMD

This release ships with ROCm 10.

## Intel

This release brings fixes for four GPU configurations.

## Serving diagnostics

Prometheus metrics (`/metrics`) now cover scheduling, request outcomes, prompt-token counts, and speculative-decoding efficiency.

`ignore_eos` is also now supported for benchmarking.

## Try it today

**NVIDIA**

```bash
docker run -p 8000:8000 --shm-size=256GB --gpus=all -e HF_TOKEN -it zmlai/llmd:cuda \
    --model=hf://deepseek-ai/DeepSeek-V4.1-Flash
```

**AMD**

```bash
docker run -p 8000:8000 --shm-size=256GB --device=/dev/kfd --device=/dev/dri -e HF_TOKEN -it zmlai/llmd:rocm \
    --model=hf://deepseek-ai/DeepSeek-V4.1-Flash
```

**Intel**

```bash
docker run -p 8000:8000 --device=/dev/dri -e HF_TOKEN -it zmlai/llmd:oneapi \
    --model=hf://Qwen/Qwen3.8-27B
```

**Google TPU**

```bash
docker run --net=host --privileged -e HF_TOKEN -it zmlai/llmd:tpu \
    --model=hf://Qwen/Qwen3.8-27B
```

**Apple Metal**

```bash
brew install zml/zml/llmd
llmd --model=hf://nvidia/Qwen3.8-27B-NVFP4 --batch-size=1
```