zml/llmd 20261023.0

Back to posts

Article

More speed, more models, more fun.

October 22, 2026 inference / serving / NVIDIA / AMD / TPU / Metal / Intel

We are pleased to announce the release of zml/llmd 20261023.0.

While last week’s release (zml/llmd 20261015.0) was a minor release with a focus on performance, this week’s release builds on the gains we made and brings more speed, more models, and more fun.

And so this week, we bring new model series to zml/llmd.

Qwen/Qwen3.8-Flash-Next#

This release brings the Qwen/Qwen3.8-Flash-Next series model series to zml/llmd.

We have tested it on NVIDIA and AMD GPU, the only ones we got access to that could host the official release. Google is still telling us there are no TPUs available.

Decode throughput, Qwen/Qwen3.8-Flash-Next, bs=16, zml/llmd 20261023.0 (tok/s)

XiaomiMiMo/MiMo-V2.6#

This release brings the XiaomiMiMo/MiMo-V2.6 series in both RL and MOPD flavour.

Decode throughput, XiaomiMiMo/MiMo-V2.6-Flash-MOPD, bs=16, zml/llmd 20261023.0 (tok/s)

Faster DeepSeek-V4.1-Flash#

The hill climb continues. We improved a number of fundamental building blocks and thus DeepSeek-V4.1-Flash got faster.

Token Generation, deepseek-ai/DeepSeek-V4.1-Flash, bs=1, 4xGB300 (tok/s)

Try it today#

NVIDIA

docker run -p 8000:8000 --shm-size=256GB --gpus=all -e HF_TOKEN -it zmlai/llmd:cuda \
    --model=hf://Qwen/Qwen3.8-Flash-Next

AMD

docker run -p 8000:8000 --shm-size=256GB --device=/dev/kfd --device=/dev/dri -e HF_TOKEN -it zmlai/llmd:rocm \
    --model=hf://Qwen/Qwen3.8-Flash-Next

Intel

docker run -p 8000:8000 --device=/dev/dri -e HF_TOKEN -it zmlai/llmd:oneapi \
    --model=hf://Qwen/Qwen3.8-27B

Google TPU

docker run --net=host --privileged -e HF_TOKEN -it zmlai/llmd:tpu \
    --model=hf://Qwen/Qwen3.8-27B

Apple Metal

brew install zml/zml/llmd
llmd --model=hf://nvidia/Qwen3.8-27B-NVFP4 --batch-size=1