{ "@context": "https://schema.org", "@id": "https://aggr.aymericbeaumet.com/items/huggingface-co/2026-10-29-welcome-le-chonk-mistral-large-4-on-hugging-face/#webpage", "@type": [ "WebPage", "ArchiveComponent" ], "articleBody": "Mistral Large 4 (aka Le Chonk) is a multimodal, multilingual, Mixture-of-Experts (MoE) model from Mistral. It takes text and images as input and generates text. It has about **1T total, 49B active parameter**, and a context window of **1M tokens**.\n\nThe model is released in two official variants:\n\n1. [FP8](https://huggingface.co/mistralai/Mistral-Large-4-1T-A52B)\n2. [NVFP4](https://huggingface.co/mistralai/Mistral-Large-4-1T-A52B-NVFP4)\n\nThere are also [GGUFs](https://huggingface.co/ggml-org) created by [Unsloth](https://huggingface.co/unsloth/Mistral-Large-4-1T-A52B-GGUF) and the llama.cpp team. It’s distributed under the Mistral Ecosystem Open-Weight License (an MIT derivative that establishes limitations for model-as-a-service offerings).\n\nMistral Large 4 is particularly strong for cybersecurity, coding agents, and multimodality. In this post, we look at how the model works, its capabilities, architecture and more!\n\n## Table of Contents\n\n- [\"Welcome Le Chonk: Mistral Large 4 on Hugging Face\"](https://huggingface.co/blog/mistral-large4#welcome-le-chonk-mistral-large-4-on-hugging-face)\n - [Table of Contents](https://huggingface.co/blog/mistral-large4#table-of-contents)\n - [What is new with Mistral Large 4](https://huggingface.co/blog/mistral-large4#what-is-new-with-mistral-large-4)\n - [Cybersecurity Capabilities](https://huggingface.co/blog/mistral-large4#cybersecurity-capabilities)\n - [Architecture](https://huggingface.co/blog/mistral-large4#architecture)\n - [Vision encoder](https://huggingface.co/blog/mistral-large4#vision-encoder)\n - [Benchmark Results](https://huggingface.co/blog/mistral-large4#benchmark-results)\n - [Inference Support](https://huggingface.co/blog/mistral-large4#inference-support)\n - [Transformers](https://huggingface.co/blog/mistral-large4#transformers)\n - [vLLM](https://huggingface.co/blog/mistral-large4#vllm)\n - [Llama.cpp](https://huggingface.co/blog/mistral-large4#llamacpp)\n - [Fine-tuning with TRL](https://huggingface.co/blog/mistral-large4#fine-tuning-with-trl)\n - [Inference Endpoints](https://huggingface.co/blog/mistral-large4#inference-endpoints)\n - [Dell Enterprise Hub](https://huggingface.co/blog/mistral-large4#dell-enterprise-hub)\n - [Try it out](https://huggingface.co/blog/mistral-large4#try-it-out)\n\n## What is new with Mistral Large 4\n\n[Mistral Large 4](https://huggingface.co/collections/mistralai/mistral-large-4) combines a large pool of experts with sparse attention and a [Pixtral](https://huggingface.co/docs/transformers/en/model_doc/pixtral) vision encoder. The language model selects a small group of experts for each token, while its attention indexer selects which earlier token positions to attend to. These are two different forms of sparsity, where one selects feed-forward networks, the other selects context positions.\n\nYou can find all the checkpoints in the [Mistral Large 4 collection](https://huggingface.co/collections/mistralai/mistral-large-4).\n\n### Cybersecurity Capabilities\n\nMistral Large 4 scores 82% on the Artificial Analysis Cyber Index test that asks a model to reproduce a real vulnerability in open-source software and then patch the software. Mistral reports that several closed frontier models score near zero on the same test because they refuse to attempt the work. This is super important for defenders, because fixing a vulnerability usually requires reproducing the vulnerability. In Mistral’s internal testing, Mistral Large 4 was also useful for malware analysis, vulnerability prioritisation and writing detection rules, which the model was not explicitly trained for.\n\nMistral mainly positions the model for defensive security work. Vulnerability research and incident response often appear like attacks in model input, so closed source filters can block them. A team that depends on an API can also lose access in the middle of an incident. Crucially, open weight models put both the usage policy and the deployment under the user’s control. That said, the model still declines harmful requests: on cyber prompts from JailbreakBench, StrongREJECT and AgentHarm, its average refusal rate is higher than any other open weight model.\n\nFor community builders, an open weight model like Mistral Large 4 unlocks a load of downstream use cases. You could build your own cyber system around your code base, crash logs, and malware samples. For example, with [TRL you could fine-tune](https://huggingface.co/blog/mistral-large4#fine-tuning-with-trl) smaller models from Mistral Large 4 traces patching your code base. In fact, reproduce-then-patch tasks use a verifiable reward, so it fits with RL environments that could be used to further RL smaller security models. Open weights also let researchers measure the model’s refusal behaviour directly instead of relying on the provider’s description.\n\nIn short, cyber capabilities like ML4’s in the open are a huge boost for the community.\n\n> We have interviewed Pierre Stock about the cyber capabilities of the model, [you can watch it here](https://www.youtube.com/embed/FczbSauVvjw).\n\n## Architecture\n\n[![ML4](https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/blog/ml4/ml4.png)](https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/blog/ml4/ml4.png)\n\nMistral Large 4’s text decoder has 72 layers, a hidden dimension of 6,144, and 64 attention heads. Its attention blocks combine Multi-head Latent Attention (MLA) with a DeepSeek Sparse Attention (DSA) indexer (a direction in other models such as GLM 5.3). MLA keeps a compressed KV representation, and DSA selects up to 2,048 key positions for each query.\n\n[![Attention](https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/blog/ml4/attention.png)](https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/blog/ml4/attention.png)\\\nFor each decoder block, there’s a 3:1 RoPE to NoPE pattern, i.e. 3 decoder layers employ RoPE and every fourth layer uses NoPE. The model also shares KV state across layers.\\\nThe first decoder layer uses a dense feed-forward network. The other 71 layers use MoE blocks, in which a softmax router selects four of the 128 experts for each token. One shared expert always runs alongside the selected experts, for a total of 5 active experts per token.\n\nRegarding RoPE, Mistral uses *LeRope*, a YaRN-scaled RoPE with one learned multiplier per frequency.\n\n[![Experts](https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/blog/ml4/experts.png)](https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/blog/ml4/experts.png)\n\n### Vision encoder\n\nMistral Large 4 employs a 1.6B Pixtral encoder. Pixtral has 30 layers, a hidden dimension of 2,048, and 16 attention heads. It uses 14 × 14 patches and a patch-merge projector with spatial merge size 2. You can see the encoder pipeline below.\n\n[![Image Encoder](https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/blog/ml4/img_encoder.png)](https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/blog/ml4/img_encoder.png)\n\n## Benchmark Results\n\nMistral Large 4 delivers strong benchmark performance across coding, cybersecurity, multimodal understanding, and agentic workflows, positioning it as a competitive addition to the open-weight ecosystem. As of October 29, 2026, the model is the fifth best open weights model on Artificial Analysis Intelligence index.\n\nOn coding, ML4 achieves **61.7% on DeepSWE v1.1**, **59.4% on SWE-Atlas-QnA**, and **28.3% on Terminal-Bench 4**. Its combined **Coding Agent Index score of 49.8%** places it ahead of DeepSeek V4 Pro 0813 and Qwen3.8 Max.\n\nCoding capabilities of ML4 extend to knowledge work, with **59.9% on AutomationBench** and **1,393 Elo on AA-Briefcase**, which evaluate business workflows and long-horizon professional tasks.\n\nCybersecurity is one of Mistral Large 4’s standout capabilities, with strong results across real-world vulnerability assessment and security competition tasks. Three benchmarks highlight the model’s cyber performance:\n\n- **Artificial Analysis Cyber Index:** ML4 ranks among the **top five models globally** on this independent evaluation of models’ ability to find and fix software vulnerabilities.\n- **Vulnerability reproduction and patching:** It achieves **82%**, the highest reported score on this test within the index. The task requires reproducing a real vulnerability in open-source software and then patching it, testing both investigation and remediation.\n- **Cybench:** ML4 solves **93%** of a suite of **40 challenges drawn from security competitions**, demonstrating strong performance on hands-on security problems.\n\nMistral Large 4 also demonstrates **93.3% attack resistance on Lakera’s B3 benchmark**, adding evidence of robustness against prompt-injection attacks. Beyond benchmarks, internal testing showed promising results in malware analysis, vulnerability prioritization, and detection-rule generation.\n\n## Inference Support\n\nMistral Large 4 is shipped with day-0 support for transformers, vLLM and llama.cpp. GGUF quants are available from GGUF and the [llama.cpp team](https://huggingface.co/ggml-org/Mistral-Large-4-1T-A52B-GGUF). We provide options to deploy remotely with Inference Endpoints, and use serverless inference through Inference Providers.\n\n### Transformers\n\nThe Transformers implementation provides `AutoModelForMultimodalLM`, with `AutoProcessor` preparing the conversation and image inputs. Mistral Large 4 can take `low`, `medium`, `high` as reasoning effort. The default is `high`, and note that reasoning cannot be disabled, so using `reasoning_effort`: `none` would still use `high`.\n\nTo run the model, upgrade the transformers package\n\n```bash\npip install --upgrade transformers\n```\n\n```python\nimport os\nfrom transformers import AutoProcessor, AutoModelForMultimodalLM, DistributedConfig\n\ndistributed_config = DistributedConfig(\n tp_size=int(os.environ[\"WORLD_SIZE\"]),\n ep_size=int(os.environ[\"WORLD_SIZE\"]),\n)\n\nmodel_id = \"mistralai/Mistral-Large-4-1T-A52B\"\nprocessor = AutoProcessor.from_pretrained(model_id)\nmodel = AutoModelForMultimodalLM.from_pretrained(\n model_id,\n distributed_config=distributed_config,\n)\n\n# only text\ntext_messages = [\n {\n \"role\": \"user\",\n \"content\": \"Who are you?\"\n }\n]\n\n# text + image\nmultimodal_messages = [\n {\n \"role\": \"user\",\n \"content\": [\n {\n \"type\": \"image\",\n \"image\": \"https://huggingface.co/datasets/huggingface/cats-image/resolve/main/cats_image.jpeg\",\n },\n {\n \"type\": \"text\",\n \"text\": \"How many cats are in the image?\",\n },\n ],\n },\n]\n\nfor messages in [text_messages, multimodal_messages]:\n inputs = processor.apply_chat_template(\n messages,\n add_generation_prompt=True,\n tokenize=True,\n return_dict=True,\n return_tensors=\"pt\",\n reasoning_effort=\"medium\",\n ).to(model.device).to(model.dtype)\n input_len = inputs[\"input_ids\"].shape[-1]\n\n outputs = model.generate(**inputs, max_new_tokens=100)\n if int(os.environ[\"RANK\"]) == 0:\n print(processor.decode(outputs[0][input_len:], skip_special_tokens=False))\n```\n\nYou can run the script above by saving it in a python file `script.py` and then run\n\n```bash\ntorchrun --nproc-per-node 8 script.py\n```\n\n### vLLM\n\nvLLM supports the high-quality NVFP4 Mistral Large 4 checkpoint, which has a smaller memory footprint and requires NVIDIA Blackwell GPUs. The model also ships with a DSpark drafter for additional speed-ups with vLLM. Launch the server like this:\n\n```sh\nvllm serve mistralai/Mistral-Large-4-1T-A52B-NVFP4 \\\n --config-format mistral \\\n --load-format mistral \\\n --tensor-parallel-size 8 \\\n --decode-context-parallel-size 8 \\\n --enable-expert-parallel \\\n --dtype bfloat16 \\\n --max-model-len 131072 \\\n --max-num-seqs 16 \\\n --max-num-batched-tokens 2048 \\\n --gpu-memory-utilization 0.85 \\\n --enable-prefix-caching \\\n --enforce-eager \\\n --mm_processor_cache_gb 0.0\n```\n\nOnce the server is up and running, you can run the following.\n\n```sh\ncurl -s http://127.0.0.1:8000/v1/chat/completions -H 'Content-Type: application/json' \\\n -d '{\"messages\":[{\"role\":\"user\",\"content\":\"What is 17*23?\"}],\"max_tokens\":512}'\n```\n\n### Llama.cpp\n\nLlama.cpp and [Llama App](https://llama.app/) already support Mistral Large 4.\n\nYou can install Llama App with the following command:\n\n```sh\ncurl -LsSf https://llama.app/install.sh | sh\n```\n\nThen you can start the server as follows, indicating the quant suffix after the model ID:\n\n```sh\nllama serve -hf ggml-org/Mistral-Large-4-1T-A52B-GGUF:Q2_K\n```\n\n[Here’s the repository](https://huggingface.co/ggml-org/Mistral-Large-4-1T-A52B-GGUF) with the quants derived from the FP8 checkpoint, and [this is the NVFP4 one](https://huggingface.co/ggml-org/Mistral-Large-4-1T-A52B-NVFP4-GGUF), converted from the official NVFP4 repo.\n\nOnce the server has started, you can head to localhost:9931 to chat using the built-in WebUI. From the web UI you can add images for inference and adjust the reasoning lever.\\\nYou can also query the server as follows. Feel free to check out [the API reference](https://llama.app/docs/serve) for multimodality, tool calls and more.\n\n```sh\ncurl http://localhost:8080/v1/chat/completions \\\n -H \"Content-Type: application/json\" \\\n -d '{\n \"messages\": [\n {\"role\": \"system\", \"content\": \"You are a helpful assistant.\"},\n {\"role\": \"user\", \"content\": \"Write a limerick about python exceptions\"}\n ]\n }'\n```\n\nYou can run image-text inference as follows.\n\n```sh\ncurl -s http://127.0.0.1:8080/v1/chat/completions \\\n -H \"Content-Type: application/json\" \\\n -H \"Authorization: Bearer \" \\\n -d '{\n \"model\": \"muse-spark-open\",\n \"messages\": [\n {\n \"role\": \"user\",\n \"content\": [\n {\n \"type\": \"text\",\n \"text\": \"What is in this image?\"\n },\n {\n \"type\": \"image_url\",\n \"image_url\": {\n \"url\": \"data:image/png;base64,...\"\n }\n }\n ]\n }\n ],\n \"temperature\": 0\n }'\n```\n\nThe llama.cpp team ships the following quants: `IQ2_XXS`, `Q2_K`, `Q4_K_M`, `NVFP4`, `Q8_0`, and `BF16` for further conversions. The 2-bit quants run on a single Apple Silicon Ultra computer with 512 GB of unified RAM. The others require distributed inference. You can also run on multiple NVIDIA or AMD cards.\n\nRunning `Q2_K` on a single M3 Ultra, we got a prefill throughput of about 90 tokens per second, and a generation speed of 15 tok/s. On 4 x RTX6000 Pro, we observed the following results:\n\n- NVFP4: 32K context size, prefill 130-590 t/s, gen speed ~10 t/s\n- IQ2\\_XXS: 800K context size. At short context sizes, prefill is 500 tok/s and generation runs at 30 tok/s. However, using long contexts performance drops to 129 tok/s (prefill) and 3 tok/s (generation).\n\n## Fine-tuning with TRL\n\nWe also ship day 0 support of fine tuning Mistral Large 4 using TRL. All you have to do is upgrade to the latest version of `transformers`, `trl` and `peft`.\n\n```sh\npip install --upgrade transformers trl peft\n```\n\nIn the following example we train the model to perform multilingual thinking. The fine tuned model would be able to reason in the language the system prompt asks for!\n\nThe script runs on the FP8 variant on a node of 8xB300s.\n\n```py\n# torchrun --nproc-per-node 8 sft.py\nimport os\n\nfrom datasets import load_dataset\nfrom peft import LoraConfig\nfrom transformers import AutoModelForImageTextToText, DistributedConfig\nfrom trl import SFTConfig, SFTTrainer\n\nworld_size = int(os.environ[\"WORLD_SIZE\"])\nmodel_id = \"mistralai/Mistral-Large-4-1T-A52B\"\nmodel = AutoModelForImageTextToText.from_pretrained(\n model_id,\n distributed_config=DistributedConfig(fsdp_size=world_size, ep_size=world_size),\n dtype=\"auto\",\n)\n\ndataset = load_dataset(\"HuggingFaceH4/Multilingual-Thinking\", split=\"train\")\n# the assistant turns carry reasoning, so render them in reasoning mode\ndataset = dataset.map(lambda x: {\"chat_template_kwargs\": {\"reasoning_effort\": \"high\"}})\n\ntrainer = SFTTrainer(\n model=model,\n args=SFTConfig(\n packing=True,\n max_length=4096,\n assistant_only_loss=True,\n learning_rate=1e-4,\n lr_scheduler_type=\"cosine\",\n warmup_steps=10,\n ),\n train_dataset=dataset,\n peft_config=LoraConfig(\n r=16,\n lora_alpha=32,\n target_modules=r\".*language_model.*\\.(q_proj|kv_a_proj_with_mqa|kv_b_proj|o_proj|gate_proj|up_proj|down_proj)\",\n ),\n)\ntrainer.train()\n```\n\n## Inference Endpoints\n\nMistral Large 4 is also available on Inference Endpoints, with a ready-to-deploy recipe on 8 x NVIDIA H200 GPUs. The recipe runs vLLM with the FP8 weights split across all eight GPUs using tensor and expert parallelism, and uses the model's built-in DSpark draft head for faster generation through speculative decoding. It serves up to 131K tokens of context and processes up to 16 requests at a time, and we observed the following results:\n\n- Single request: time to first token of about 0.6 seconds for a 1K-token prompt, and a generation speed of ~65 tok/s, reaching 120–138 tok/s on chat prompts where DSpark's drafts are accepted.\n- Long prompts: 16K-token prompts, four at a time, are prefilled at about 4,300 tok/s in total, with a time to first token of 8–9 seconds.\n- Under load: about 480 tok/s of total generation across 1,000 ShareGPT conversations.\n\nYou can deploy it with the `hf` CLI as follows:\n\n```sh\nhf endpoints catalog deploy --repo mistralai/Mistral-Large-4-1T-A52B --name mistral-large-4\n```\n\nOnce the endpoint is running, you can query its OpenAI-compatible API, using the endpoint URL and your Hugging Face token. The `reasoning_effort` field controls how much the model reasons before answering, and can be set to `low`, `medium`, or `high` (default).\n\n```sh\ncurl \"$(hf endpoints describe mistral-large-4 --json | jq -r .status.url)/v1/chat/completions\" \\\n -H \"Authorization: Bearer $(hf auth token 2>/dev/null)\" \\\n -H \"Content-Type: application/json\" \\\n -d '{\n \"model\": \"mistralai/Mistral-Large-4-1T-A52B\",\n \"messages\": [\n {\"role\": \"user\", \"content\": \"Write a limerick about python exceptions\"}\n ],\n \"reasoning_effort\": \"medium\"\n }'\n```\n\nThe answer comes back in `choices[0].message.content`, and the reasoning in `choices[0].message.reasoning`. The endpoint also supports tool calls, structured outputs, image inputs, and streaming through the standard Chat Completions fields.\n\n## Dell Enterprise Hub\n\nThe easiest way to get up and running with your Dell AI Servers (NVIDIA H200, B200, GB200 or GB300) is by installing the `dell-ai` CLI:\n\n```sh\npip install dell-ai && dell-ai login\n```\n\nAnd use it to run an optimized recipe for Mistral Large 4 on e.g., a NVIDIA B300 node.\n\n```sh\ndell-ai models deploy --model-id mistralai/Mistral-Large-4-1T-A52B-NVFP4 \\\n --engine docker \\\n --platform-id xe9780-nvidia-b300 \\\n --gpus 4\n```\n\nYou can find more about this release in the [Dell Enterprise Hub (DEH)](https://dell.huggingface.co/updates/mistral-large-4).\n\n## Try it out\n\nGrab the weights from the [Mistral Large 4 collection](https://huggingface.co/collections/mistralai/mistral-large-4), run them locally with the [GGUFs](https://huggingface.co/ggml-org/Mistral-Large-4-1T-A52B-GGUF), or deploy them with Inference Endpoints. We can't wait to see what you build with Le Chonk. Share your fine-tunes, quants and other artefacts on the Hub!\n", "authors": [], "captured_at": "2026-10-29T15:30:03.869733718+00:00", "category": "ai", "content_html": "

Mistral Large 4 (aka Le Chonk) is a multimodal, multilingual, Mixture-of-Experts (MoE) model from Mistral. It takes text and images as input and generates text. It has about 1T total, 49B active parameter, and a context window of 1M tokens.

\n

The model is released in two official variants:

\n
    \n
  1. FP8
    2. \n
  2. NVFP4
    3. \n
\n

There are also GGUFs created by Unsloth and the llama.cpp team. It’s distributed under the Mistral Ecosystem Open-Weight License (an MIT derivative that establishes limitations for model-as-a-service offerings).

\n

Mistral Large 4 is particularly strong for cybersecurity, coding agents, and multimodality. In this post, we look at how the model works, its capabilities, architecture and more!

\n

Table of Contents

\n\n

What is new with Mistral Large 4

\n

Mistral Large 4 combines a large pool of experts with sparse attention and a Pixtral vision encoder. The language model selects a small group of experts for each token, while its attention indexer selects which earlier token positions to attend to. These are two different forms of sparsity, where one selects feed-forward networks, the other selects context positions.

\n

You can find all the checkpoints in the Mistral Large 4 collection.

\n

Cybersecurity Capabilities

\n

Mistral Large 4 scores 82% on the Artificial Analysis Cyber Index test that asks a model to reproduce a real vulnerability in open-source software and then patch the software. Mistral reports that several closed frontier models score near zero on the same test because they refuse to attempt the work. This is super important for defenders, because fixing a vulnerability usually requires reproducing the vulnerability. In Mistral’s internal testing, Mistral Large 4 was also useful for malware analysis, vulnerability prioritisation and writing detection rules, which the model was not explicitly trained for.

\n

Mistral mainly positions the model for defensive security work. Vulnerability research and incident response often appear like attacks in model input, so closed source filters can block them. A team that depends on an API can also lose access in the middle of an incident. Crucially, open weight models put both the usage policy and the deployment under the user’s control. That said, the model still declines harmful requests: on cyber prompts from JailbreakBench, StrongREJECT and AgentHarm, its average refusal rate is higher than any other open weight model.

\n

For community builders, an open weight model like Mistral Large 4 unlocks a load of downstream use cases. You could build your own cyber system around your code base, crash logs, and malware samples. For example, with TRL you could fine-tune smaller models from Mistral Large 4 traces patching your code base. In fact, reproduce-then-patch tasks use a verifiable reward, so it fits with RL environments that could be used to further RL smaller security models. Open weights also let researchers measure the model’s refusal behaviour directly instead of relying on the provider’s description.

\n

In short, cyber capabilities like ML4’s in the open are a huge boost for the community.

\n
\n

We have interviewed Pierre Stock about the cyber capabilities of the model, you can watch it here.

\n
\n

Architecture

\n
\"ML4\"

\n

Mistral Large 4’s text decoder has 72 layers, a hidden dimension of 6,144, and 64 attention heads. Its attention blocks combine Multi-head Latent Attention (MLA) with a DeepSeek Sparse Attention (DSA) indexer (a direction in other models such as GLM 5.3). MLA keeps a compressed KV representation, and DSA selects up to 2,048 key positions for each query.

\n

\"Attention\"

For each decoder block, there’s a 3:1 RoPE to NoPE pattern, i.e. 3 decoder layers employ RoPE and every fourth layer uses NoPE. The model also shares KV state across layers.
\nThe first decoder layer uses a dense feed-forward network. The other 71 layers use MoE blocks, in which a softmax router selects four of the 128 experts for each token. One shared expert always runs alongside the selected experts, for a total of 5 active experts per token.
\n

Regarding RoPE, Mistral uses LeRope, a YaRN-scaled RoPE with one learned multiplier per frequency.

\n

\"Experts\"

\n

Vision encoder

\n

Mistral Large 4 employs a 1.6B Pixtral encoder. Pixtral has 30 layers, a hidden dimension of 2,048, and 16 attention heads. It uses 14 × 14 patches and a patch-merge projector with spatial merge size 2. You can see the encoder pipeline below.

\n

\"Image

\n

Benchmark Results

\n

Mistral Large 4 delivers strong benchmark performance across coding, cybersecurity, multimodal understanding, and agentic workflows, positioning it as a competitive addition to the open-weight ecosystem. As of October 29, 2026, the model is the fifth best open weights model on Artificial Analysis Intelligence index.

\n

On coding, ML4 achieves 61.7% on DeepSWE v1.1, 59.4% on SWE-Atlas-QnA, and 28.3% on Terminal-Bench 4. Its combined Coding Agent Index score of 49.8% places it ahead of DeepSeek V4 Pro 0813 and Qwen3.8 Max.

\n

Coding capabilities of ML4 extend to knowledge work, with 59.9% on AutomationBench and 1,393 Elo on AA-Briefcase, which evaluate business workflows and long-horizon professional tasks.

\n

Cybersecurity is one of Mistral Large 4’s standout capabilities, with strong results across real-world vulnerability assessment and security competition tasks. Three benchmarks highlight the model’s cyber performance:

\n\n

Mistral Large 4 also demonstrates 93.3% attack resistance on Lakera’s B3 benchmark, adding evidence of robustness against prompt-injection attacks. Beyond benchmarks, internal testing showed promising results in malware analysis, vulnerability prioritization, and detection-rule generation.

\n

Inference Support

\n

Mistral Large 4 is shipped with day-0 support for transformers, vLLM and llama.cpp. GGUF quants are available from GGUF and the llama.cpp team. We provide options to deploy remotely with Inference Endpoints, and use serverless inference through Inference Providers.

\n

Transformers

\n

The Transformers implementation provides AutoModelForMultimodalLM, with AutoProcessor preparing the conversation and image inputs. Mistral Large 4 can take low, medium, high as reasoning effort. The default is high, and note that reasoning cannot be disabled, so using reasoning_effort: none would still use high.

\n

To run the model, upgrade the transformers package

\n
pip install --upgrade transformers\n
\n
import os\nfrom transformers import AutoProcessor, AutoModelForMultimodalLM, DistributedConfig\n\ndistributed_config = DistributedConfig(\n    tp_size=int(os.environ["WORLD_SIZE"]),\n    ep_size=int(os.environ["WORLD_SIZE"]),\n)\n\nmodel_id = "mistralai/Mistral-Large-4-1T-A52B"\nprocessor = AutoProcessor.from_pretrained(model_id)\nmodel = AutoModelForMultimodalLM.from_pretrained(\n    model_id,\n    distributed_config=distributed_config,\n)\n\n# only text\ntext_messages = [\n    {\n        "role": "user",\n        "content": "Who are you?"\n    }\n]\n\n# text + image\nmultimodal_messages = [\n    {\n        "role": "user",\n        "content": [\n            {\n                "type": "image",\n                "image": "https://huggingface.co/datasets/huggingface/cats-image/resolve/main/cats_image.jpeg",\n            },\n            {\n                "type": "text",\n                "text": "How many cats are in the image?",\n            },\n        ],\n    },\n]\n\nfor messages in [text_messages, multimodal_messages]:\n    inputs = processor.apply_chat_template(\n        messages,\n        add_generation_prompt=True,\n        tokenize=True,\n        return_dict=True,\n        return_tensors="pt",\n        reasoning_effort="medium",\n    ).to(model.device).to(model.dtype)\n    input_len = inputs["input_ids"].shape[-1]\n\n    outputs = model.generate(**inputs, max_new_tokens=100)\n    if int(os.environ["RANK"]) == 0:\n        print(processor.decode(outputs[0][input_len:], skip_special_tokens=False))\n
\n

You can run the script above by saving it in a python file script.py and then run

\n
torchrun --nproc-per-node 8 script.py\n
\n

vLLM

\n

vLLM supports the high-quality NVFP4 Mistral Large 4 checkpoint, which has a smaller memory footprint and requires NVIDIA Blackwell GPUs. The model also ships with a DSpark drafter for additional speed-ups with vLLM. Launch the server like this:

\n
vllm serve mistralai/Mistral-Large-4-1T-A52B-NVFP4 \\\n  --config-format mistral \\\n  --load-format mistral \\\n  --tensor-parallel-size 8 \\\n  --decode-context-parallel-size 8 \\\n  --enable-expert-parallel \\\n  --dtype bfloat16 \\\n  --max-model-len 131072 \\\n  --max-num-seqs 16 \\\n  --max-num-batched-tokens 2048 \\\n  --gpu-memory-utilization 0.85 \\\n  --enable-prefix-caching \\\n  --enforce-eager \\\n  --mm_processor_cache_gb 0.0\n
\n

Once the server is up and running, you can run the following.

\n
curl -s http://127.0.0.1:8000/v1/chat/completions -H 'Content-Type: application/json' \\\n  -d '{"messages":[{"role":"user","content":"What is 17*23?"}],"max_tokens":512}'\n
\n

Llama.cpp

\n

Llama.cpp and Llama App already support Mistral Large 4.

\n

You can install Llama App with the following command:

\n
curl -LsSf https://llama.app/install.sh | sh\n
\n

Then you can start the server as follows, indicating the quant suffix after the model ID:

\n
llama serve -hf ggml-org/Mistral-Large-4-1T-A52B-GGUF:Q2_K\n
\n

Here’s the repository with the quants derived from the FP8 checkpoint, and this is the NVFP4 one, converted from the official NVFP4 repo.

\n

Once the server has started, you can head to localhost:9931 to chat using the built-in WebUI. From the web UI you can add images for inference and adjust the reasoning lever.
\nYou can also query the server as follows. Feel free to check out the API reference for multimodality, tool calls and more.

\n
curl http://localhost:8080/v1/chat/completions \\\n    -H "Content-Type: application/json" \\\n    -d '{\n        "messages": [\n            {"role": "system", "content": "You are a helpful assistant."},\n            {"role": "user", "content": "Write a limerick about python exceptions"}\n        ]\n    }'\n
\n

You can run image-text inference as follows.

\n
curl -s http://127.0.0.1:8080/v1/chat/completions \\\n    -H "Content-Type: application/json" \\\n    -H "Authorization: Bearer <your-key>" \\\n    -d '{\n        "model": "muse-spark-open",\n        "messages": [\n            {\n                "role": "user",\n                "content": [\n                    {\n                        "type": "text",\n                        "text": "What is in this image?"\n                    },\n                    {\n                        "type": "image_url",\n                        "image_url": {\n                            "url": "data:image/png;base64,..."\n                        }\n                    }\n                ]\n            }\n        ],\n        "temperature": 0\n    }'\n
\n

The llama.cpp team ships the following quants: IQ2_XXS, Q2_K, Q4_K_M, NVFP4, Q8_0, and BF16 for further conversions. The 2-bit quants run on a single Apple Silicon Ultra computer with 512 GB of unified RAM. The others require distributed inference. You can also run on multiple NVIDIA or AMD cards.

\n

Running Q2_K on a single M3 Ultra, we got a prefill throughput of about 90 tokens per second, and a generation speed of 15 tok/s. On 4 x RTX6000 Pro, we observed the following results:

\n\n

Fine-tuning with TRL

\n

We also ship day 0 support of fine tuning Mistral Large 4 using TRL. All you have to do is upgrade to the latest version of transformers, trl and peft.

\n
pip install --upgrade transformers trl peft\n
\n

In the following example we train the model to perform multilingual thinking. The fine tuned model would be able to reason in the language the system prompt asks for!

\n

The script runs on the FP8 variant on a node of 8xB300s.

\n
# torchrun --nproc-per-node 8 sft.py\nimport os\n\nfrom datasets import load_dataset\nfrom peft import LoraConfig\nfrom transformers import AutoModelForImageTextToText, DistributedConfig\nfrom trl import SFTConfig, SFTTrainer\n\nworld_size = int(os.environ["WORLD_SIZE"])\nmodel_id = "mistralai/Mistral-Large-4-1T-A52B"\nmodel = AutoModelForImageTextToText.from_pretrained(\n    model_id,\n    distributed_config=DistributedConfig(fsdp_size=world_size, ep_size=world_size),\n    dtype="auto",\n)\n\ndataset = load_dataset("HuggingFaceH4/Multilingual-Thinking", split="train")\n# the assistant turns carry reasoning, so render them in reasoning mode\ndataset = dataset.map(lambda x: {"chat_template_kwargs": {"reasoning_effort": "high"}})\n\ntrainer = SFTTrainer(\n    model=model,\n    args=SFTConfig(\n        packing=True,\n        max_length=4096,\n        assistant_only_loss=True,\n        learning_rate=1e-4,\n        lr_scheduler_type="cosine",\n        warmup_steps=10,\n    ),\n    train_dataset=dataset,\n    peft_config=LoraConfig(\n        r=16,\n        lora_alpha=32,\n        target_modules=r".*language_model.*\\.(q_proj|kv_a_proj_with_mqa|kv_b_proj|o_proj|gate_proj|up_proj|down_proj)",\n    ),\n)\ntrainer.train()\n
\n

Inference Endpoints

\n

Mistral Large 4 is also available on Inference Endpoints, with a ready-to-deploy recipe on 8 x NVIDIA H200 GPUs. The recipe runs vLLM with the FP8 weights split across all eight GPUs using tensor and expert parallelism, and uses the model's built-in DSpark draft head for faster generation through speculative decoding. It serves up to 131K tokens of context and processes up to 16 requests at a time, and we observed the following results:

\n\n

You can deploy it with the hf CLI as follows:

\n
hf endpoints catalog deploy --repo mistralai/Mistral-Large-4-1T-A52B --name mistral-large-4\n
\n

Once the endpoint is running, you can query its OpenAI-compatible API, using the endpoint URL and your Hugging Face token. The reasoning_effort field controls how much the model reasons before answering, and can be set to low, medium, or high (default).

\n
curl "$(hf endpoints describe mistral-large-4 --json | jq -r .status.url)/v1/chat/completions" \\\n    -H "Authorization: Bearer $(hf auth token 2>/dev/null)" \\\n    -H "Content-Type: application/json" \\\n    -d '{\n        "model": "mistralai/Mistral-Large-4-1T-A52B",\n        "messages": [\n            {"role": "user", "content": "Write a limerick about python exceptions"}\n        ],\n        "reasoning_effort": "medium"\n    }'\n
\n

The answer comes back in choices[0].message.content, and the reasoning in choices[0].message.reasoning. The endpoint also supports tool calls, structured outputs, image inputs, and streaming through the standard Chat Completions fields.

\n

Dell Enterprise Hub

\n

The easiest way to get up and running with your Dell AI Servers (NVIDIA H200, B200, GB200 or GB300) is by installing the dell-ai CLI:

\n
pip install dell-ai && dell-ai login\n
\n

And use it to run an optimized recipe for Mistral Large 4 on e.g., a NVIDIA B300 node.

\n
dell-ai models deploy --model-id mistralai/Mistral-Large-4-1T-A52B-NVFP4 \\\n    --engine docker \\\n    --platform-id xe9780-nvidia-b300 \\\n    --gpus 4\n
\n

You can find more about this release in the Dell Enterprise Hub (DEH).

\n

Try it out

\n

Grab the weights from the Mistral Large 4 collection, run them locally with the GGUFs, or deploy them with Inference Endpoints. We can't wait to see what you build with Le Chonk. Share your fine-tunes, quants and other artefacts on the Hub!

\n", "content_markdown": "Mistral Large 4 (aka Le Chonk) is a multimodal, multilingual, Mixture-of-Experts (MoE) model from Mistral. It takes text and images as input and generates text. It has about **1T total, 49B active parameter**, and a context window of **1M tokens**.\n\nThe model is released in two official variants:\n\n1. [FP8](https://huggingface.co/mistralai/Mistral-Large-4-1T-A52B)\n2. [NVFP4](https://huggingface.co/mistralai/Mistral-Large-4-1T-A52B-NVFP4)\n\nThere are also [GGUFs](https://huggingface.co/ggml-org) created by [Unsloth](https://huggingface.co/unsloth/Mistral-Large-4-1T-A52B-GGUF) and the llama.cpp team. It’s distributed under the Mistral Ecosystem Open-Weight License (an MIT derivative that establishes limitations for model-as-a-service offerings).\n\nMistral Large 4 is particularly strong for cybersecurity, coding agents, and multimodality. In this post, we look at how the model works, its capabilities, architecture and more!\n\n## Table of Contents\n\n- [\"Welcome Le Chonk: Mistral Large 4 on Hugging Face\"](https://huggingface.co/blog/mistral-large4#welcome-le-chonk-mistral-large-4-on-hugging-face)\n - [Table of Contents](https://huggingface.co/blog/mistral-large4#table-of-contents)\n - [What is new with Mistral Large 4](https://huggingface.co/blog/mistral-large4#what-is-new-with-mistral-large-4)\n - [Cybersecurity Capabilities](https://huggingface.co/blog/mistral-large4#cybersecurity-capabilities)\n - [Architecture](https://huggingface.co/blog/mistral-large4#architecture)\n - [Vision encoder](https://huggingface.co/blog/mistral-large4#vision-encoder)\n - [Benchmark Results](https://huggingface.co/blog/mistral-large4#benchmark-results)\n - [Inference Support](https://huggingface.co/blog/mistral-large4#inference-support)\n - [Transformers](https://huggingface.co/blog/mistral-large4#transformers)\n - [vLLM](https://huggingface.co/blog/mistral-large4#vllm)\n - [Llama.cpp](https://huggingface.co/blog/mistral-large4#llamacpp)\n - [Fine-tuning with TRL](https://huggingface.co/blog/mistral-large4#fine-tuning-with-trl)\n - [Inference Endpoints](https://huggingface.co/blog/mistral-large4#inference-endpoints)\n - [Dell Enterprise Hub](https://huggingface.co/blog/mistral-large4#dell-enterprise-hub)\n - [Try it out](https://huggingface.co/blog/mistral-large4#try-it-out)\n\n## What is new with Mistral Large 4\n\n[Mistral Large 4](https://huggingface.co/collections/mistralai/mistral-large-4) combines a large pool of experts with sparse attention and a [Pixtral](https://huggingface.co/docs/transformers/en/model_doc/pixtral) vision encoder. The language model selects a small group of experts for each token, while its attention indexer selects which earlier token positions to attend to. These are two different forms of sparsity, where one selects feed-forward networks, the other selects context positions.\n\nYou can find all the checkpoints in the [Mistral Large 4 collection](https://huggingface.co/collections/mistralai/mistral-large-4).\n\n### Cybersecurity Capabilities\n\nMistral Large 4 scores 82% on the Artificial Analysis Cyber Index test that asks a model to reproduce a real vulnerability in open-source software and then patch the software. Mistral reports that several closed frontier models score near zero on the same test because they refuse to attempt the work. This is super important for defenders, because fixing a vulnerability usually requires reproducing the vulnerability. In Mistral’s internal testing, Mistral Large 4 was also useful for malware analysis, vulnerability prioritisation and writing detection rules, which the model was not explicitly trained for.\n\nMistral mainly positions the model for defensive security work. Vulnerability research and incident response often appear like attacks in model input, so closed source filters can block them. A team that depends on an API can also lose access in the middle of an incident. Crucially, open weight models put both the usage policy and the deployment under the user’s control. That said, the model still declines harmful requests: on cyber prompts from JailbreakBench, StrongREJECT and AgentHarm, its average refusal rate is higher than any other open weight model.\n\nFor community builders, an open weight model like Mistral Large 4 unlocks a load of downstream use cases. You could build your own cyber system around your code base, crash logs, and malware samples. For example, with [TRL you could fine-tune](https://huggingface.co/blog/mistral-large4#fine-tuning-with-trl) smaller models from Mistral Large 4 traces patching your code base. In fact, reproduce-then-patch tasks use a verifiable reward, so it fits with RL environments that could be used to further RL smaller security models. Open weights also let researchers measure the model’s refusal behaviour directly instead of relying on the provider’s description.\n\nIn short, cyber capabilities like ML4’s in the open are a huge boost for the community.\n\n> We have interviewed Pierre Stock about the cyber capabilities of the model, [you can watch it here](https://www.youtube.com/embed/FczbSauVvjw).\n\n## Architecture\n\n[![ML4](https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/blog/ml4/ml4.png)](https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/blog/ml4/ml4.png)\n\nMistral Large 4’s text decoder has 72 layers, a hidden dimension of 6,144, and 64 attention heads. Its attention blocks combine Multi-head Latent Attention (MLA) with a DeepSeek Sparse Attention (DSA) indexer (a direction in other models such as GLM 5.3). MLA keeps a compressed KV representation, and DSA selects up to 2,048 key positions for each query.\n\n[![Attention](https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/blog/ml4/attention.png)](https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/blog/ml4/attention.png)\\\nFor each decoder block, there’s a 3:1 RoPE to NoPE pattern, i.e. 3 decoder layers employ RoPE and every fourth layer uses NoPE. The model also shares KV state across layers.\\\nThe first decoder layer uses a dense feed-forward network. The other 71 layers use MoE blocks, in which a softmax router selects four of the 128 experts for each token. One shared expert always runs alongside the selected experts, for a total of 5 active experts per token.\n\nRegarding RoPE, Mistral uses *LeRope*, a YaRN-scaled RoPE with one learned multiplier per frequency.\n\n[![Experts](https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/blog/ml4/experts.png)](https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/blog/ml4/experts.png)\n\n### Vision encoder\n\nMistral Large 4 employs a 1.6B Pixtral encoder. Pixtral has 30 layers, a hidden dimension of 2,048, and 16 attention heads. It uses 14 × 14 patches and a patch-merge projector with spatial merge size 2. You can see the encoder pipeline below.\n\n[![Image Encoder](https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/blog/ml4/img_encoder.png)](https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/blog/ml4/img_encoder.png)\n\n## Benchmark Results\n\nMistral Large 4 delivers strong benchmark performance across coding, cybersecurity, multimodal understanding, and agentic workflows, positioning it as a competitive addition to the open-weight ecosystem. As of October 29, 2026, the model is the fifth best open weights model on Artificial Analysis Intelligence index.\n\nOn coding, ML4 achieves **61.7% on DeepSWE v1.1**, **59.4% on SWE-Atlas-QnA**, and **28.3% on Terminal-Bench 4**. Its combined **Coding Agent Index score of 49.8%** places it ahead of DeepSeek V4 Pro 0813 and Qwen3.8 Max.\n\nCoding capabilities of ML4 extend to knowledge work, with **59.9% on AutomationBench** and **1,393 Elo on AA-Briefcase**, which evaluate business workflows and long-horizon professional tasks.\n\nCybersecurity is one of Mistral Large 4’s standout capabilities, with strong results across real-world vulnerability assessment and security competition tasks. Three benchmarks highlight the model’s cyber performance:\n\n- **Artificial Analysis Cyber Index:** ML4 ranks among the **top five models globally** on this independent evaluation of models’ ability to find and fix software vulnerabilities.\n- **Vulnerability reproduction and patching:** It achieves **82%**, the highest reported score on this test within the index. The task requires reproducing a real vulnerability in open-source software and then patching it, testing both investigation and remediation.\n- **Cybench:** ML4 solves **93%** of a suite of **40 challenges drawn from security competitions**, demonstrating strong performance on hands-on security problems.\n\nMistral Large 4 also demonstrates **93.3% attack resistance on Lakera’s B3 benchmark**, adding evidence of robustness against prompt-injection attacks. Beyond benchmarks, internal testing showed promising results in malware analysis, vulnerability prioritization, and detection-rule generation.\n\n## Inference Support\n\nMistral Large 4 is shipped with day-0 support for transformers, vLLM and llama.cpp. GGUF quants are available from GGUF and the [llama.cpp team](https://huggingface.co/ggml-org/Mistral-Large-4-1T-A52B-GGUF). We provide options to deploy remotely with Inference Endpoints, and use serverless inference through Inference Providers.\n\n### Transformers\n\nThe Transformers implementation provides `AutoModelForMultimodalLM`, with `AutoProcessor` preparing the conversation and image inputs. Mistral Large 4 can take `low`, `medium`, `high` as reasoning effort. The default is `high`, and note that reasoning cannot be disabled, so using `reasoning_effort`: `none` would still use `high`.\n\nTo run the model, upgrade the transformers package\n\n```bash\npip install --upgrade transformers\n```\n\n```python\nimport os\nfrom transformers import AutoProcessor, AutoModelForMultimodalLM, DistributedConfig\n\ndistributed_config = DistributedConfig(\n tp_size=int(os.environ[\"WORLD_SIZE\"]),\n ep_size=int(os.environ[\"WORLD_SIZE\"]),\n)\n\nmodel_id = \"mistralai/Mistral-Large-4-1T-A52B\"\nprocessor = AutoProcessor.from_pretrained(model_id)\nmodel = AutoModelForMultimodalLM.from_pretrained(\n model_id,\n distributed_config=distributed_config,\n)\n\n# only text\ntext_messages = [\n {\n \"role\": \"user\",\n \"content\": \"Who are you?\"\n }\n]\n\n# text + image\nmultimodal_messages = [\n {\n \"role\": \"user\",\n \"content\": [\n {\n \"type\": \"image\",\n \"image\": \"https://huggingface.co/datasets/huggingface/cats-image/resolve/main/cats_image.jpeg\",\n },\n {\n \"type\": \"text\",\n \"text\": \"How many cats are in the image?\",\n },\n ],\n },\n]\n\nfor messages in [text_messages, multimodal_messages]:\n inputs = processor.apply_chat_template(\n messages,\n add_generation_prompt=True,\n tokenize=True,\n return_dict=True,\n return_tensors=\"pt\",\n reasoning_effort=\"medium\",\n ).to(model.device).to(model.dtype)\n input_len = inputs[\"input_ids\"].shape[-1]\n\n outputs = model.generate(**inputs, max_new_tokens=100)\n if int(os.environ[\"RANK\"]) == 0:\n print(processor.decode(outputs[0][input_len:], skip_special_tokens=False))\n```\n\nYou can run the script above by saving it in a python file `script.py` and then run\n\n```bash\ntorchrun --nproc-per-node 8 script.py\n```\n\n### vLLM\n\nvLLM supports the high-quality NVFP4 Mistral Large 4 checkpoint, which has a smaller memory footprint and requires NVIDIA Blackwell GPUs. The model also ships with a DSpark drafter for additional speed-ups with vLLM. Launch the server like this:\n\n```sh\nvllm serve mistralai/Mistral-Large-4-1T-A52B-NVFP4 \\\n --config-format mistral \\\n --load-format mistral \\\n --tensor-parallel-size 8 \\\n --decode-context-parallel-size 8 \\\n --enable-expert-parallel \\\n --dtype bfloat16 \\\n --max-model-len 131072 \\\n --max-num-seqs 16 \\\n --max-num-batched-tokens 2048 \\\n --gpu-memory-utilization 0.85 \\\n --enable-prefix-caching \\\n --enforce-eager \\\n --mm_processor_cache_gb 0.0\n```\n\nOnce the server is up and running, you can run the following.\n\n```sh\ncurl -s http://127.0.0.1:8000/v1/chat/completions -H 'Content-Type: application/json' \\\n -d '{\"messages\":[{\"role\":\"user\",\"content\":\"What is 17*23?\"}],\"max_tokens\":512}'\n```\n\n### Llama.cpp\n\nLlama.cpp and [Llama App](https://llama.app/) already support Mistral Large 4.\n\nYou can install Llama App with the following command:\n\n```sh\ncurl -LsSf https://llama.app/install.sh | sh\n```\n\nThen you can start the server as follows, indicating the quant suffix after the model ID:\n\n```sh\nllama serve -hf ggml-org/Mistral-Large-4-1T-A52B-GGUF:Q2_K\n```\n\n[Here’s the repository](https://huggingface.co/ggml-org/Mistral-Large-4-1T-A52B-GGUF) with the quants derived from the FP8 checkpoint, and [this is the NVFP4 one](https://huggingface.co/ggml-org/Mistral-Large-4-1T-A52B-NVFP4-GGUF), converted from the official NVFP4 repo.\n\nOnce the server has started, you can head to localhost:9931 to chat using the built-in WebUI. From the web UI you can add images for inference and adjust the reasoning lever.\\\nYou can also query the server as follows. Feel free to check out [the API reference](https://llama.app/docs/serve) for multimodality, tool calls and more.\n\n```sh\ncurl http://localhost:8080/v1/chat/completions \\\n -H \"Content-Type: application/json\" \\\n -d '{\n \"messages\": [\n {\"role\": \"system\", \"content\": \"You are a helpful assistant.\"},\n {\"role\": \"user\", \"content\": \"Write a limerick about python exceptions\"}\n ]\n }'\n```\n\nYou can run image-text inference as follows.\n\n```sh\ncurl -s http://127.0.0.1:8080/v1/chat/completions \\\n -H \"Content-Type: application/json\" \\\n -H \"Authorization: Bearer \" \\\n -d '{\n \"model\": \"muse-spark-open\",\n \"messages\": [\n {\n \"role\": \"user\",\n \"content\": [\n {\n \"type\": \"text\",\n \"text\": \"What is in this image?\"\n },\n {\n \"type\": \"image_url\",\n \"image_url\": {\n \"url\": \"data:image/png;base64,...\"\n }\n }\n ]\n }\n ],\n \"temperature\": 0\n }'\n```\n\nThe llama.cpp team ships the following quants: `IQ2_XXS`, `Q2_K`, `Q4_K_M`, `NVFP4`, `Q8_0`, and `BF16` for further conversions. The 2-bit quants run on a single Apple Silicon Ultra computer with 512 GB of unified RAM. The others require distributed inference. You can also run on multiple NVIDIA or AMD cards.\n\nRunning `Q2_K` on a single M3 Ultra, we got a prefill throughput of about 90 tokens per second, and a generation speed of 15 tok/s. On 4 x RTX6000 Pro, we observed the following results:\n\n- NVFP4: 32K context size, prefill 130-590 t/s, gen speed ~10 t/s\n- IQ2\\_XXS: 800K context size. At short context sizes, prefill is 500 tok/s and generation runs at 30 tok/s. However, using long contexts performance drops to 129 tok/s (prefill) and 3 tok/s (generation).\n\n## Fine-tuning with TRL\n\nWe also ship day 0 support of fine tuning Mistral Large 4 using TRL. All you have to do is upgrade to the latest version of `transformers`, `trl` and `peft`.\n\n```sh\npip install --upgrade transformers trl peft\n```\n\nIn the following example we train the model to perform multilingual thinking. The fine tuned model would be able to reason in the language the system prompt asks for!\n\nThe script runs on the FP8 variant on a node of 8xB300s.\n\n```py\n# torchrun --nproc-per-node 8 sft.py\nimport os\n\nfrom datasets import load_dataset\nfrom peft import LoraConfig\nfrom transformers import AutoModelForImageTextToText, DistributedConfig\nfrom trl import SFTConfig, SFTTrainer\n\nworld_size = int(os.environ[\"WORLD_SIZE\"])\nmodel_id = \"mistralai/Mistral-Large-4-1T-A52B\"\nmodel = AutoModelForImageTextToText.from_pretrained(\n model_id,\n distributed_config=DistributedConfig(fsdp_size=world_size, ep_size=world_size),\n dtype=\"auto\",\n)\n\ndataset = load_dataset(\"HuggingFaceH4/Multilingual-Thinking\", split=\"train\")\n# the assistant turns carry reasoning, so render them in reasoning mode\ndataset = dataset.map(lambda x: {\"chat_template_kwargs\": {\"reasoning_effort\": \"high\"}})\n\ntrainer = SFTTrainer(\n model=model,\n args=SFTConfig(\n packing=True,\n max_length=4096,\n assistant_only_loss=True,\n learning_rate=1e-4,\n lr_scheduler_type=\"cosine\",\n warmup_steps=10,\n ),\n train_dataset=dataset,\n peft_config=LoraConfig(\n r=16,\n lora_alpha=32,\n target_modules=r\".*language_model.*\\.(q_proj|kv_a_proj_with_mqa|kv_b_proj|o_proj|gate_proj|up_proj|down_proj)\",\n ),\n)\ntrainer.train()\n```\n\n## Inference Endpoints\n\nMistral Large 4 is also available on Inference Endpoints, with a ready-to-deploy recipe on 8 x NVIDIA H200 GPUs. The recipe runs vLLM with the FP8 weights split across all eight GPUs using tensor and expert parallelism, and uses the model's built-in DSpark draft head for faster generation through speculative decoding. It serves up to 131K tokens of context and processes up to 16 requests at a time, and we observed the following results:\n\n- Single request: time to first token of about 0.6 seconds for a 1K-token prompt, and a generation speed of ~65 tok/s, reaching 120–138 tok/s on chat prompts where DSpark's drafts are accepted.\n- Long prompts: 16K-token prompts, four at a time, are prefilled at about 4,300 tok/s in total, with a time to first token of 8–9 seconds.\n- Under load: about 480 tok/s of total generation across 1,000 ShareGPT conversations.\n\nYou can deploy it with the `hf` CLI as follows:\n\n```sh\nhf endpoints catalog deploy --repo mistralai/Mistral-Large-4-1T-A52B --name mistral-large-4\n```\n\nOnce the endpoint is running, you can query its OpenAI-compatible API, using the endpoint URL and your Hugging Face token. The `reasoning_effort` field controls how much the model reasons before answering, and can be set to `low`, `medium`, or `high` (default).\n\n```sh\ncurl \"$(hf endpoints describe mistral-large-4 --json | jq -r .status.url)/v1/chat/completions\" \\\n -H \"Authorization: Bearer $(hf auth token 2>/dev/null)\" \\\n -H \"Content-Type: application/json\" \\\n -d '{\n \"model\": \"mistralai/Mistral-Large-4-1T-A52B\",\n \"messages\": [\n {\"role\": \"user\", \"content\": \"Write a limerick about python exceptions\"}\n ],\n \"reasoning_effort\": \"medium\"\n }'\n```\n\nThe answer comes back in `choices[0].message.content`, and the reasoning in `choices[0].message.reasoning`. The endpoint also supports tool calls, structured outputs, image inputs, and streaming through the standard Chat Completions fields.\n\n## Dell Enterprise Hub\n\nThe easiest way to get up and running with your Dell AI Servers (NVIDIA H200, B200, GB200 or GB300) is by installing the `dell-ai` CLI:\n\n```sh\npip install dell-ai && dell-ai login\n```\n\nAnd use it to run an optimized recipe for Mistral Large 4 on e.g., a NVIDIA B300 node.\n\n```sh\ndell-ai models deploy --model-id mistralai/Mistral-Large-4-1T-A52B-NVFP4 \\\n --engine docker \\\n --platform-id xe9780-nvidia-b300 \\\n --gpus 4\n```\n\nYou can find more about this release in the [Dell Enterprise Hub (DEH)](https://dell.huggingface.co/updates/mistral-large-4).\n\n## Try it out\n\nGrab the weights from the [Mistral Large 4 collection](https://huggingface.co/collections/mistralai/mistral-large-4), run them locally with the [GGUFs](https://huggingface.co/ggml-org/Mistral-Large-4-1T-A52B-GGUF), or deploy them with Inference Endpoints. We can't wait to see what you build with Le Chonk. Share your fine-tunes, quants and other artefacts on the Hub!\n", "dateCreated": "2026-10-29T15:30:03.869733718+00:00", "date_created": "2026-10-29T00:00:00+00:00", "date_modified": "2026-10-29T00:00:00+00:00", "date_published": "2026-10-29T00:00:00+00:00", "description": "Mistral Large 4 (aka Le Chonk) is a multimodal, multilingual, Mixture-of-Experts (MoE) model from Mistral. It takes text and images as input and generates text. It has about 1T total, 49B active parameter, and a context window of 1M…", "external_url": "https://huggingface.co/blog/mistral-large4", "git": { "commit": "20068632b65d2830a2d54f25ca1d90d697bea2c0", "path": "items/huggingface-co/2026/10/2026-10-29-welcome-le-chonk-mistral-large-4-on-hugging-face.md", "permalink": "https://github.com/aymericbeaumet/aggr-instance/blob/20068632b65d2830a2d54f25ca1d90d697bea2c0/items/huggingface-co/2026/10/2026-10-29-welcome-le-chonk-mistral-large-4-on-hugging-face.md" }, "id": "urn:aggr:item:27dd56f31bbd215dce9d59f9e1218574a49d21f4", "isBasedOn": { "@id": "https://huggingface.co/blog/mistral-large4" }, "mainEntity": { "@id": "https://huggingface.co/blog/mistral-large4", "@type": "CreativeWork", "archivedAt": { "@id": "https://aggr.aymericbeaumet.com/items/huggingface-co/2026-10-29-welcome-le-chonk-mistral-large-4-on-hugging-face/#webpage" }, "dateModified": "2026-10-29T00:00:00+00:00", "datePublished": "2026-10-29T00:00:00+00:00", "headline": "Welcome Le Chonk: Mistral Large 4 on Hugging Face", "image": { "@type": "ImageObject", "caption": null, "height": 90, "url": "https://huggingface.co/blog/assets/ml4/thumbnail.png", "width": 160 }, "timeRequired": "PT11M", "url": "https://huggingface.co/blog/mistral-large4", "wordCount": 2358 }, "name": "Archived snapshot: Welcome Le Chonk: Mistral Large 4 on Hugging Face", "preview": { "alt": null, "color": null, "height": 90, "placeholder": { "data_url": "data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAAEAAAABCAQAAAC1HAwCAAAAC0lEQVR42mNk+A8AAQUBAScY42YAAAAASUVORK5CYII=", "hash": "" }, "url": "https://huggingface.co/blog/assets/ml4/thumbnail.png", "width": 160 }, "reading_minutes": 11, "replicated_at": null, "source": "huggingface-co", "source_name": "Hugging Face - Blog", "summary": null, "tags": [], "temporalCoverage": "2026-10-29T15:30:03.869733718+00:00", "title": "Welcome Le Chonk: Mistral Large 4 on Hugging Face", "url": "https://aggr.aymericbeaumet.com/items/huggingface-co/2026-10-29-welcome-le-chonk-mistral-large-4-on-hugging-face/", "word_count": 2358 }