{
  "@context": "https://schema.org",
  "@id": "https://aggr.aymericbeaumet.com/items/huggingface-co/2025-12-04-deepmath-a-lightweight-math-reasoning-agent-with-smolagents/#webpage",
  "@type": [
    "WebPage",
    "ArchiveComponent"
  ],
  "articleBody": "![An LLM is using a calculator to answer questions.](https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/intel-deepmath/deepmath-figure.jpg)\n\n*By Intel AI Software Group*\n\n[DeepMath](https://huggingface.co/Intel/deepmath-v1) is an aligned math reasoning agent built on **[Qwen3-4B Thinking](https://huggingface.co/Qwen/Qwen3-4B-Thinking-2507)** and fine-tuned with **GRPO (Group Relative Policy Optimization)**. Instead of verbose text, the model emits **tiny Python snippets** for intermediate steps, runs them in a secure sandbox, and folds the results back into its reasoning, reducing errors and output length. The agent is implemented using the **[smolagents library](https://github.com/huggingface/smolagents)**.\n\nWe evaluate DeepMath on four math datasets: **[MATH500](https://huggingface.co/datasets/HuggingFaceH4/MATH-500), [AIME](https://huggingface.co/datasets/opencompass/AIME2025), [HMMT](https://huggingface.co/datasets/MathArena/hmmt_feb_2025), and [HLE](https://huggingface.co/datasets/cais/hle),** and show that:\n\n- 🤖 The math agent alone reduces output lengths by up to 66%, while often improving accuracy.\n\n- ⚡ GRPO training improves the agent performance even further, in almost all benchmarks.\n\n👉 Code and evaluation scripts: [https://github.com/IntelLabs/DeepMath](https://github.com/IntelLabs/DeepMath) \\\n👉 Model: [https://huggingface.co/Intel/deepmath-v1](https://huggingface.co/Intel/deepmath-v1)\n\n## Why DeepMath?\n\nLarge language models (LLMs) have advanced reasoning capabilities, but mathematical problem-solving remains challenging; chain-of-thought traces can be lengthy and prone to arithmetic mistakes. Recent works\\[^1\\]\\[^2\\] demonstrate that small models can reach strong performance, and other studies\\[^3\\] investigate tool use to improve reliability. What those papers generally do not emphasize is reducing trace verbosity or explicitly training models to prefer short, computation-oriented traces executed in a constrained, auditable environment.\n\nWe focused on two goals:\n\n1. **Offload deterministic computation** to a safe executor.\n\n2. **Train models to prefer concise, computation-oriented traces** over verbose text.\n\n**DeepMath** tackles this by combining a small Python executor with a fine-tuned LLM, enabling concise, computation-driven reasoning. The model learns to generate short Python snippets, which are executed in a sandbox and reintegrated into the context. GRPO fine-tuning encourages this behavior by rewarding correctness and encouraging shorter outputs.\n\n## How It Works\n\n- Base model: [Qwen3-4B Thinking](https://huggingface.co/Qwen/Qwen3-4B-Thinking-2507).\n- Executor constraints: sandboxed environment, allow-list of imported modules, per-snippet timeout.\n- Inference: based on [smolagents](https://github.com/huggingface/smolagents/), a math agent was created. [vLLM](https://github.com/vllm-project/vLLM) is used as the inference engine.\n- Training: based on the GRPO trainer in [TRL](https://github.com/huggingface/trl), we modified TRL's vLLM client and server to generate GRPO completions using our DeepMath agent.\n\n![Changes to vLLM client and server in TRL library.](https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/intel-deepmath/trl-grpo-vllm-deepmath.png)\\\n *Figure 1: The vLLM client and server were modified to use the DeepMath agent in generating the candidates, while using the vLLM backend.*\n\n- **Agent Interface:** During inference, the model can output normal tokens or special agent calls containing Python snippets.\n\n- **Execution:** Snippets run in a sandboxed environment with strict safety constraints (no file I/O, no network, timeouts).\n\n- **Design Goals:**\n\n  - **Concision:** Replace multi-line textual calculations with short, focused snippets.\n\n  - **Determinism & Safety:** Enforce strict execution limits.\n\n  - **Interpretability:** Snippets are readable and auditable.\n\n![Output example: it contains a short python snippet as well as its output which is used in the reasoning process.](https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/intel-deepmath/output-example.png)\\\n *Figure 2: Output example where python code is generated, evaluated and the answer is inserted into the trace and used for context.*\n\n## Training with GRPO\n\nWe fine-tune the model using **GRPO**, a reward-based optimization that balances:\n\n- **Accuracy Reward:** +1 for correct answers.\n\n- **Using code snippets:** +1 for generating code snippets, weighted 10:1 vs. the accuracy reward.\n\n- **Length reduction:** shorter lengths are encouraged by limiting the GRPO completion candidates to 5k tokens.\n\n- **Temperature Scheduling:** We implemented linear temperature scheduling (T=1.2 → T=0.7) to balance exploration and stability during training. This approach aims to enhance experimentation during the initial training phases, subsequently reducing the temperature as we refine our proficiency in the skill.\n\n- **In-context Learning**: we include 4 solved examples where the trace contains agent calls and executor outputs, so the model learns the syntax and the call/response pattern.\n\n- **Dataset**: we used the Tool-Integrated Reasoning (TIR) subset of the [OpenMathReasoning](https://huggingface.co/datasets/nvidia/OpenMathReasoning) dataset. Note that GRPO only uses the problem, not the solution in the data. This dataset was chosen to ensure the problems benefit from the external tool.\n\n## Evaluation\n\nWe benchmarked DeepMath against baselines on four datasets. Metrics include:\n\n- **majority@16**: robustness across samples, as used in previous math reasoning works, see references.\n\n- **Mean output length**: brevity.\n\n![Main results table.](https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/intel-deepmath/main-results.png)\n\n- We compare a baseline configuration ([Qwen3-4B-Thinking-2507](https://huggingface.co/Qwen/Qwen3-4B-Thinking-2507), no agenting) with our DeepMath model. As ablation, we evaluate the agentic framework we developed running with the untrained Qwen3 model, denoted by **+Agent**. Additionally, we examine whether the GRPO training (for agentic use) improves non-agentic inference, denoted by **+GRPO**. Thus the two ablations are independent, not additive.\n\n- We observe the agentic inference reduces output lengths, with mixed accuracy results. The DeepMath model is both GRPO-trained and run in agentic mode, and shows the highest accuracy with shortened traces. We conclude **both GRPO training and agentic inference are needed** for best results.\n\n**Key Insight:** DeepMath reduces output length by up to **66%** while improving accuracy on challenging datasets.\n\n## Why It Matters\n\n- **Accuracy:** Offloading computation reduces arithmetic errors.\n\n- **Efficiency:** Shorter outputs mean faster inference and easier interpretability.\n\n- **Safety:** Sandbox execution mitigates risks of running arbitrary code.\n\n## Conclusion\n\nDeepMath demonstrates a practical and lightweight way to combine a small executor with an LLM and to train the model to prefer short, computation-driven traces. Offloading deterministic computation reduces arithmetic and numerical errors and shortens traces, and GRPO fine-tuning further encourages concise, correct answers. The result is a more accurate and more interpretable math-solving agent without requiring a massive model or heavyweight external tools.\n\n## Try It Yourself\n\nCheck out the [GitHub repo](https://github.com/IntelLabs/DeepMath) and share your feedback! Contributions welcome. 🚀\n\n## Citation\n\nIf you use DeepMath in your research, please cite:\n\n```bibtex\n@software{deepmath2025,\n  author = {Fleischer, Daniel and Berchansky, Moshe and Wasserblat, Moshe},\n  title = {DeepMath: A Lightweight Math Reasoning Agent for LLMs},\n  year = {2025},\n  publisher = {Intel AI Labs},\n  url = {https://github.com/IntelLabs/DeepMath}\n}\n```\n\n## Limitations & Future Work\n\n- **Scope**: we focused on a small model and on mathematical reasoning.\n\n- **Generalization**: evaluated on contest-style math; results may not transfer to open-ended mathematical creativity or formal proofs.\n\n- Executing generated code is inherently risky. DeepMath uses strict sandboxing and resource limits, but any deployment should carefully manage attack surfaces and enforce rate limits.\n\n## References\n\n\\[1\\] Luo, Michael, Sijun Tan, Justin Wong, et al. 2025. “DeepScaleR: Surpassing O1-Preview with a 1.5B Model by Scaling RL.” [https://pretty-radio-b75.notion.site/DeepScaleR-Surpassing-O1-Preview-with-a-1-5B-Model-by-Scaling-RL-19681902c1468005bed8ca303013a4e2](https://pretty-radio-b75.notion.site/DeepScaleR-Surpassing-O1-Preview-with-a-1-5B-Model-by-Scaling-RL-19681902c1468005bed8ca303013a4e2)\n\n\\[2\\] Liu, Mingjie, Shizhe Diao, Ximing Lu, et al. 2025. “ProRL: Prolonged Reinforcement Learning Expands Reasoning Boundaries in Large Language Models.” arXiv:2505.24864. Preprint, arXiv, May 30. [https://doi.org/10.48550/arXiv.2505.24864](https://doi.org/10.48550/arXiv.2505.24864)\n\n\\[3\\] Moshkov, Ivan, Darragh Hanley, Ivan Sorokin, et al. 2025. “AIMO-2 Winning Solution: Building State-of-the-Art Mathematical Reasoning Models with OpenMathReasoning Dataset.” arXiv:2504.16891. Preprint, arXiv, April 23. [https://doi.org/10.48550/arXiv.2504.16891](https://doi.org/10.48550/arXiv.2504.16891)\n",
  "authors": [],
  "captured_at": "2026-10-04T12:50:30.804479567+00:00",
  "category": "ai",
  "content_html": "<figure class=\"article-figure\"><picture class=\"article-picture article-picture-fallback\" style=\"--image-ratio:16 / 9\"><img src=\"https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/intel-deepmath/deepmath-figure.jpg\" alt=\"An LLM is using a calculator to answer questions.\" loading=\"eager\" decoding=\"async\" fetchpriority=\"high\" referrerpolicy=\"no-referrer\" class=\"progressive-image\"></picture></p>\n<p><em>By Intel AI Software Group</em></p>\n<p><a target=\"_blank\" rel=\"noopener noreferrer\" href=\"https://huggingface.co/Intel/deepmath-v1\">DeepMath</a> is an aligned math reasoning agent built on <strong><a target=\"_blank\" rel=\"noopener noreferrer\" href=\"https://huggingface.co/Qwen/Qwen3-4B-Thinking-2507\">Qwen3-4B Thinking</a></strong> and fine-tuned with <strong>GRPO (Group Relative Policy Optimization)</strong>. Instead of verbose text, the model emits <strong>tiny Python snippets</strong> for intermediate steps, runs them in a secure sandbox, and folds the results back into its reasoning, reducing errors and output length. The agent is implemented using the <strong><a target=\"_blank\" rel=\"noopener noreferrer\" href=\"https://github.com/huggingface/smolagents\">smolagents library</a></strong>.</p>\n<p>We evaluate DeepMath on four math datasets: <strong><a target=\"_blank\" rel=\"noopener noreferrer\" href=\"https://huggingface.co/datasets/HuggingFaceH4/MATH-500\">MATH500</a>, <a target=\"_blank\" rel=\"noopener noreferrer\" href=\"https://huggingface.co/datasets/opencompass/AIME2025\">AIME</a>, <a target=\"_blank\" rel=\"noopener noreferrer\" href=\"https://huggingface.co/datasets/MathArena/hmmt_feb_2025\">HMMT</a>, and <a target=\"_blank\" rel=\"noopener noreferrer\" href=\"https://huggingface.co/datasets/cais/hle\">HLE</a>,</strong> and show that:</p>\n<ul>\n<li>\n<p>🤖 The math agent alone reduces output lengths by up to 66%, while often improving accuracy.</p>\n</li>\n<li>\n<p>⚡ GRPO training improves the agent performance even further, in almost all benchmarks.</p>\n</li>\n</ul>\n<p>👉 Code and evaluation scripts: <a target=\"_blank\" rel=\"noopener noreferrer\" href=\"https://github.com/IntelLabs/DeepMath\">https://github.com/IntelLabs/DeepMath</a><br />\n👉 Model: <a target=\"_blank\" rel=\"noopener noreferrer\" href=\"https://huggingface.co/Intel/deepmath-v1\">https://huggingface.co/Intel/deepmath-v1</a></p>\n<h2>Why DeepMath?</h2>\n<p>Large language models (LLMs) have advanced reasoning capabilities, but mathematical problem-solving remains challenging; chain-of-thought traces can be lengthy and prone to arithmetic mistakes. Recent works[^1][^2] demonstrate that small models can reach strong performance, and other studies[^3] investigate tool use to improve reliability. What those papers generally do not emphasize is reducing trace verbosity or explicitly training models to prefer short, computation-oriented traces executed in a constrained, auditable environment.</p>\n<p>We focused on two goals:</p>\n<ol>\n<li>\n<p><strong>Offload deterministic computation</strong> to a safe executor.</p>\n</li>\n<li>\n<p><strong>Train models to prefer concise, computation-oriented traces</strong> over verbose text.</p>\n</li>\n</ol>\n<p><strong>DeepMath</strong> tackles this by combining a small Python executor with a fine-tuned LLM, enabling concise, computation-driven reasoning. The model learns to generate short Python snippets, which are executed in a sandbox and reintegrated into the context. GRPO fine-tuning encourages this behavior by rewarding correctness and encouraging shorter outputs.</p>\n<h2>How It Works</h2>\n<ul>\n<li>Base model: <a target=\"_blank\" rel=\"noopener noreferrer\" href=\"https://huggingface.co/Qwen/Qwen3-4B-Thinking-2507\">Qwen3-4B Thinking</a>.</li>\n<li>Executor constraints: sandboxed environment, allow-list of imported modules, per-snippet timeout.</li>\n<li>Inference: based on <a target=\"_blank\" rel=\"noopener noreferrer\" href=\"https://github.com/huggingface/smolagents/\">smolagents</a>, a math agent was created. <a target=\"_blank\" rel=\"noopener noreferrer\" href=\"https://github.com/vllm-project/vLLM\">vLLM</a> is used as the inference engine.</li>\n<li>Training: based on the GRPO trainer in <a target=\"_blank\" rel=\"noopener noreferrer\" href=\"https://github.com/huggingface/trl\">TRL</a>, we modified TRL's vLLM client and server to generate GRPO completions using our DeepMath agent.</li>\n</ul>\n<p><picture class=\"article-picture article-picture-fallback\" style=\"--image-ratio:16 / 9\"><img src=\"https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/intel-deepmath/trl-grpo-vllm-deepmath.png\" alt=\"Changes to vLLM client and server in TRL library.\" loading=\"lazy\" decoding=\"async\" fetchpriority=\"low\" referrerpolicy=\"no-referrer\" class=\"progressive-image\"></picture><figcaption><em>Figure 1: The vLLM client and server were modified to use the DeepMath agent in generating the candidates, while using the vLLM backend.</em></figcaption></figure>\n<ul>\n<li>\n<p><strong>Agent Interface:</strong> During inference, the model can output normal tokens or special agent calls containing Python snippets.</p>\n</li>\n<li>\n<p><strong>Execution:</strong> Snippets run in a sandboxed environment with strict safety constraints (no file I/O, no network, timeouts).</p>\n</li>\n<li>\n<p><strong>Design Goals:</strong></p>\n<ul>\n<li>\n<p><strong>Concision:</strong> Replace multi-line textual calculations with short, focused snippets.</p>\n</li>\n<li>\n<p><strong>Determinism &amp; Safety:</strong> Enforce strict execution limits.</p>\n</li>\n<li>\n<p><strong>Interpretability:</strong> Snippets are readable and auditable.</p>\n</li>\n</ul>\n</li>\n</ul>\n<figure class=\"article-figure\"><picture class=\"article-picture article-picture-fallback\" style=\"--image-ratio:16 / 9\"><img src=\"https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/intel-deepmath/output-example.png\" alt=\"Output example: it contains a short python snippet as well as its output which is used in the reasoning process.\" loading=\"lazy\" decoding=\"async\" fetchpriority=\"low\" referrerpolicy=\"no-referrer\" class=\"progressive-image\"></picture><figcaption><em>Figure 2: Output example where python code is generated, evaluated and the answer is inserted into the trace and used for context.</em></figcaption></figure>\n<h2>Training with GRPO</h2>\n<p>We fine-tune the model using <strong>GRPO</strong>, a reward-based optimization that balances:</p>\n<ul>\n<li>\n<p><strong>Accuracy Reward:</strong> +1 for correct answers.</p>\n</li>\n<li>\n<p><strong>Using code snippets:</strong> +1 for generating code snippets, weighted 10:1 vs. the accuracy reward.</p>\n</li>\n<li>\n<p><strong>Length reduction:</strong> shorter lengths are encouraged by limiting the GRPO completion candidates to 5k tokens.</p>\n</li>\n<li>\n<p><strong>Temperature Scheduling:</strong> We implemented linear temperature scheduling (T=1.2 → T=0.7) to balance exploration and stability during training. This approach aims to enhance experimentation during the initial training phases, subsequently reducing the temperature as we refine our proficiency in the skill.</p>\n</li>\n<li>\n<p><strong>In-context Learning</strong>: we include 4 solved examples where the trace contains agent calls and executor outputs, so the model learns the syntax and the call/response pattern.</p>\n</li>\n<li>\n<p><strong>Dataset</strong>: we used the Tool-Integrated Reasoning (TIR) subset of the <a target=\"_blank\" rel=\"noopener noreferrer\" href=\"https://huggingface.co/datasets/nvidia/OpenMathReasoning\">OpenMathReasoning</a> dataset. Note that GRPO only uses the problem, not the solution in the data. This dataset was chosen to ensure the problems benefit from the external tool.</p>\n</li>\n</ul>\n<h2>Evaluation</h2>\n<p>We benchmarked DeepMath against baselines on four datasets. Metrics include:</p>\n<ul>\n<li>\n<p><strong>majority@16</strong>: robustness across samples, as used in previous math reasoning works, see references.</p>\n</li>\n<li>\n<p><strong>Mean output length</strong>: brevity.</p>\n</li>\n</ul>\n<p><picture class=\"article-picture article-picture-fallback\" style=\"--image-ratio:16 / 9\"><img src=\"https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/intel-deepmath/main-results.png\" alt=\"Main results table.\" loading=\"lazy\" decoding=\"async\" fetchpriority=\"low\" referrerpolicy=\"no-referrer\" class=\"progressive-image\"></picture></p>\n<ul>\n<li>\n<p>We compare a baseline configuration (<a target=\"_blank\" rel=\"noopener noreferrer\" href=\"https://huggingface.co/Qwen/Qwen3-4B-Thinking-2507\">Qwen3-4B-Thinking-2507</a>, no agenting) with our DeepMath model. As ablation, we evaluate the agentic framework we developed running with the untrained Qwen3 model, denoted by <strong>+Agent</strong>. Additionally, we examine whether the GRPO training (for agentic use) improves non-agentic inference, denoted by <strong>+GRPO</strong>. Thus the two ablations are independent, not additive.</p>\n</li>\n<li>\n<p>We observe the agentic inference reduces output lengths, with mixed accuracy results. The DeepMath model is both GRPO-trained and run in agentic mode, and shows the highest accuracy with shortened traces. We conclude <strong>both GRPO training and agentic inference are needed</strong> for best results.</p>\n</li>\n</ul>\n<p><strong>Key Insight:</strong> DeepMath reduces output length by up to <strong>66%</strong> while improving accuracy on challenging datasets.</p>\n<h2>Why It Matters</h2>\n<ul>\n<li>\n<p><strong>Accuracy:</strong> Offloading computation reduces arithmetic errors.</p>\n</li>\n<li>\n<p><strong>Efficiency:</strong> Shorter outputs mean faster inference and easier interpretability.</p>\n</li>\n<li>\n<p><strong>Safety:</strong> Sandbox execution mitigates risks of running arbitrary code.</p>\n</li>\n</ul>\n<h2>Conclusion</h2>\n<p>DeepMath demonstrates a practical and lightweight way to combine a small executor with an LLM and to train the model to prefer short, computation-driven traces. Offloading deterministic computation reduces arithmetic and numerical errors and shortens traces, and GRPO fine-tuning further encourages concise, correct answers. The result is a more accurate and more interpretable math-solving agent without requiring a massive model or heavyweight external tools.</p>\n<h2>Try It Yourself</h2>\n<p>Check out the <a target=\"_blank\" rel=\"noopener noreferrer\" href=\"https://github.com/IntelLabs/DeepMath\">GitHub repo</a> and share your feedback! Contributions welcome. 🚀</p>\n<h2>Citation</h2>\n<p>If you use DeepMath in your research, please cite:</p>\n<pre><code class=\"language-bibtex\"><span class=\"code-snippet\" data-language=\"Bibtex\"><span class=\"syntax-text syntax-bibtex\"><span class=\"syntax-meta syntax-entry syntax-braces syntax-bibtex\"><span class=\"syntax-keyword syntax-other syntax-entry-type syntax-bibtex\"><span class=\"syntax-punctuation syntax-definition syntax-keyword syntax-bibtex\">@</span>software</span><span class=\"syntax-punctuation syntax-section syntax-entry syntax-begin syntax-bibtex\">{</span><span class=\"syntax-entity syntax-name syntax-type syntax-entry-key syntax-bibtex\">deepmath2025</span>,\n  <span class=\"syntax-meta syntax-key-assignment syntax-bibtex\"><span class=\"syntax-string syntax-unquoted syntax-key syntax-bibtex\">author</span> <span class=\"syntax-punctuation syntax-separator syntax-key-value syntax-bibtex\">=</span> <span class=\"syntax-string syntax-quoted syntax-other syntax-braces syntax-bibtex\"><span class=\"syntax-punctuation syntax-definition syntax-string syntax-begin syntax-bibtex\">{</span>Fleischer, Daniel and Berchansky, Moshe and Wasserblat, Moshe<span class=\"syntax-punctuation syntax-definition syntax-string syntax-end syntax-bibtex\">}</span></span></span>,\n  <span class=\"syntax-meta syntax-key-assignment syntax-bibtex\"><span class=\"syntax-string syntax-unquoted syntax-key syntax-bibtex\">title</span> <span class=\"syntax-punctuation syntax-separator syntax-key-value syntax-bibtex\">=</span> <span class=\"syntax-string syntax-quoted syntax-other syntax-braces syntax-bibtex\"><span class=\"syntax-punctuation syntax-definition syntax-string syntax-begin syntax-bibtex\">{</span>DeepMath: A Lightweight Math Reasoning Agent for LLMs<span class=\"syntax-punctuation syntax-definition syntax-string syntax-end syntax-bibtex\">}</span></span></span>,\n  <span class=\"syntax-meta syntax-key-assignment syntax-bibtex\"><span class=\"syntax-string syntax-unquoted syntax-key syntax-bibtex\">year</span> <span class=\"syntax-punctuation syntax-separator syntax-key-value syntax-bibtex\">=</span> <span class=\"syntax-string syntax-quoted syntax-other syntax-braces syntax-bibtex\"><span class=\"syntax-punctuation syntax-definition syntax-string syntax-begin syntax-bibtex\">{</span>2025<span class=\"syntax-punctuation syntax-definition syntax-string syntax-end syntax-bibtex\">}</span></span></span>,\n  <span class=\"syntax-meta syntax-key-assignment syntax-bibtex\"><span class=\"syntax-string syntax-unquoted syntax-key syntax-bibtex\">publisher</span> <span class=\"syntax-punctuation syntax-separator syntax-key-value syntax-bibtex\">=</span> <span class=\"syntax-string syntax-quoted syntax-other syntax-braces syntax-bibtex\"><span class=\"syntax-punctuation syntax-definition syntax-string syntax-begin syntax-bibtex\">{</span>Intel AI Labs<span class=\"syntax-punctuation syntax-definition syntax-string syntax-end syntax-bibtex\">}</span></span></span>,\n  <span class=\"syntax-meta syntax-key-assignment syntax-bibtex\"><span class=\"syntax-string syntax-unquoted syntax-key syntax-bibtex\">url</span> <span class=\"syntax-punctuation syntax-separator syntax-key-value syntax-bibtex\">=</span> <span class=\"syntax-string syntax-quoted syntax-other syntax-braces syntax-bibtex\"><span class=\"syntax-punctuation syntax-definition syntax-string syntax-begin syntax-bibtex\">{</span>https://github.com/IntelLabs/DeepMath<span class=\"syntax-punctuation syntax-definition syntax-string syntax-end syntax-bibtex\">}</span></span>\n</span><span class=\"syntax-punctuation syntax-section syntax-entry syntax-end syntax-bibtex\">}</span></span>\n</span></span></code></pre>\n<h2>Limitations &amp; Future Work</h2>\n<ul>\n<li>\n<p><strong>Scope</strong>: we focused on a small model and on mathematical reasoning.</p>\n</li>\n<li>\n<p><strong>Generalization</strong>: evaluated on contest-style math; results may not transfer to open-ended mathematical creativity or formal proofs.</p>\n</li>\n<li>\n<p>Executing generated code is inherently risky. DeepMath uses strict sandboxing and resource limits, but any deployment should carefully manage attack surfaces and enforce rate limits.</p>\n</li>\n</ul>\n<h2>References</h2>\n<p>[1] Luo, Michael, Sijun Tan, Justin Wong, et al. 2025. “DeepScaleR: Surpassing O1-Preview with a 1.5B Model by Scaling RL.” <a target=\"_blank\" rel=\"noopener noreferrer\" href=\"https://pretty-radio-b75.notion.site/DeepScaleR-Surpassing-O1-Preview-with-a-1-5B-Model-by-Scaling-RL-19681902c1468005bed8ca303013a4e2\">https://pretty-radio-b75.notion.site/DeepScaleR-Surpassing-O1-Preview-with-a-1-5B-Model-by-Scaling-RL-19681902c1468005bed8ca303013a4e2</a></p>\n<p>[2] Liu, Mingjie, Shizhe Diao, Ximing Lu, et al. 2025. “ProRL: Prolonged Reinforcement Learning Expands Reasoning Boundaries in Large Language Models.” arXiv:2505.24864. Preprint, arXiv, May 30. <a target=\"_blank\" rel=\"noopener noreferrer\" href=\"https://doi.org/10.48550/arXiv.2505.24864\">https://doi.org/10.48550/arXiv.2505.24864</a></p>\n<p>[3] Moshkov, Ivan, Darragh Hanley, Ivan Sorokin, et al. 2025. “AIMO-2 Winning Solution: Building State-of-the-Art Mathematical Reasoning Models with OpenMathReasoning Dataset.” arXiv:2504.16891. Preprint, arXiv, April 23. <a target=\"_blank\" rel=\"noopener noreferrer\" href=\"https://doi.org/10.48550/arXiv.2504.16891\">https://doi.org/10.48550/arXiv.2504.16891</a></p>\n",
  "content_markdown": "![An LLM is using a calculator to answer questions.](https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/intel-deepmath/deepmath-figure.jpg)\n\n*By Intel AI Software Group*\n\n[DeepMath](https://huggingface.co/Intel/deepmath-v1) is an aligned math reasoning agent built on **[Qwen3-4B Thinking](https://huggingface.co/Qwen/Qwen3-4B-Thinking-2507)** and fine-tuned with **GRPO (Group Relative Policy Optimization)**. Instead of verbose text, the model emits **tiny Python snippets** for intermediate steps, runs them in a secure sandbox, and folds the results back into its reasoning, reducing errors and output length. The agent is implemented using the **[smolagents library](https://github.com/huggingface/smolagents)**.\n\nWe evaluate DeepMath on four math datasets: **[MATH500](https://huggingface.co/datasets/HuggingFaceH4/MATH-500), [AIME](https://huggingface.co/datasets/opencompass/AIME2025), [HMMT](https://huggingface.co/datasets/MathArena/hmmt_feb_2025), and [HLE](https://huggingface.co/datasets/cais/hle),** and show that:\n\n- 🤖 The math agent alone reduces output lengths by up to 66%, while often improving accuracy.\n\n- ⚡ GRPO training improves the agent performance even further, in almost all benchmarks.\n\n👉 Code and evaluation scripts: [https://github.com/IntelLabs/DeepMath](https://github.com/IntelLabs/DeepMath) \\\n👉 Model: [https://huggingface.co/Intel/deepmath-v1](https://huggingface.co/Intel/deepmath-v1)\n\n## Why DeepMath?\n\nLarge language models (LLMs) have advanced reasoning capabilities, but mathematical problem-solving remains challenging; chain-of-thought traces can be lengthy and prone to arithmetic mistakes. Recent works\\[^1\\]\\[^2\\] demonstrate that small models can reach strong performance, and other studies\\[^3\\] investigate tool use to improve reliability. What those papers generally do not emphasize is reducing trace verbosity or explicitly training models to prefer short, computation-oriented traces executed in a constrained, auditable environment.\n\nWe focused on two goals:\n\n1. **Offload deterministic computation** to a safe executor.\n\n2. **Train models to prefer concise, computation-oriented traces** over verbose text.\n\n**DeepMath** tackles this by combining a small Python executor with a fine-tuned LLM, enabling concise, computation-driven reasoning. The model learns to generate short Python snippets, which are executed in a sandbox and reintegrated into the context. GRPO fine-tuning encourages this behavior by rewarding correctness and encouraging shorter outputs.\n\n## How It Works\n\n- Base model: [Qwen3-4B Thinking](https://huggingface.co/Qwen/Qwen3-4B-Thinking-2507).\n- Executor constraints: sandboxed environment, allow-list of imported modules, per-snippet timeout.\n- Inference: based on [smolagents](https://github.com/huggingface/smolagents/), a math agent was created. [vLLM](https://github.com/vllm-project/vLLM) is used as the inference engine.\n- Training: based on the GRPO trainer in [TRL](https://github.com/huggingface/trl), we modified TRL's vLLM client and server to generate GRPO completions using our DeepMath agent.\n\n![Changes to vLLM client and server in TRL library.](https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/intel-deepmath/trl-grpo-vllm-deepmath.png)\\\n *Figure 1: The vLLM client and server were modified to use the DeepMath agent in generating the candidates, while using the vLLM backend.*\n\n- **Agent Interface:** During inference, the model can output normal tokens or special agent calls containing Python snippets.\n\n- **Execution:** Snippets run in a sandboxed environment with strict safety constraints (no file I/O, no network, timeouts).\n\n- **Design Goals:**\n\n  - **Concision:** Replace multi-line textual calculations with short, focused snippets.\n\n  - **Determinism & Safety:** Enforce strict execution limits.\n\n  - **Interpretability:** Snippets are readable and auditable.\n\n![Output example: it contains a short python snippet as well as its output which is used in the reasoning process.](https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/intel-deepmath/output-example.png)\\\n *Figure 2: Output example where python code is generated, evaluated and the answer is inserted into the trace and used for context.*\n\n## Training with GRPO\n\nWe fine-tune the model using **GRPO**, a reward-based optimization that balances:\n\n- **Accuracy Reward:** +1 for correct answers.\n\n- **Using code snippets:** +1 for generating code snippets, weighted 10:1 vs. the accuracy reward.\n\n- **Length reduction:** shorter lengths are encouraged by limiting the GRPO completion candidates to 5k tokens.\n\n- **Temperature Scheduling:** We implemented linear temperature scheduling (T=1.2 → T=0.7) to balance exploration and stability during training. This approach aims to enhance experimentation during the initial training phases, subsequently reducing the temperature as we refine our proficiency in the skill.\n\n- **In-context Learning**: we include 4 solved examples where the trace contains agent calls and executor outputs, so the model learns the syntax and the call/response pattern.\n\n- **Dataset**: we used the Tool-Integrated Reasoning (TIR) subset of the [OpenMathReasoning](https://huggingface.co/datasets/nvidia/OpenMathReasoning) dataset. Note that GRPO only uses the problem, not the solution in the data. This dataset was chosen to ensure the problems benefit from the external tool.\n\n## Evaluation\n\nWe benchmarked DeepMath against baselines on four datasets. Metrics include:\n\n- **majority@16**: robustness across samples, as used in previous math reasoning works, see references.\n\n- **Mean output length**: brevity.\n\n![Main results table.](https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/intel-deepmath/main-results.png)\n\n- We compare a baseline configuration ([Qwen3-4B-Thinking-2507](https://huggingface.co/Qwen/Qwen3-4B-Thinking-2507), no agenting) with our DeepMath model. As ablation, we evaluate the agentic framework we developed running with the untrained Qwen3 model, denoted by **+Agent**. Additionally, we examine whether the GRPO training (for agentic use) improves non-agentic inference, denoted by **+GRPO**. Thus the two ablations are independent, not additive.\n\n- We observe the agentic inference reduces output lengths, with mixed accuracy results. The DeepMath model is both GRPO-trained and run in agentic mode, and shows the highest accuracy with shortened traces. We conclude **both GRPO training and agentic inference are needed** for best results.\n\n**Key Insight:** DeepMath reduces output length by up to **66%** while improving accuracy on challenging datasets.\n\n## Why It Matters\n\n- **Accuracy:** Offloading computation reduces arithmetic errors.\n\n- **Efficiency:** Shorter outputs mean faster inference and easier interpretability.\n\n- **Safety:** Sandbox execution mitigates risks of running arbitrary code.\n\n## Conclusion\n\nDeepMath demonstrates a practical and lightweight way to combine a small executor with an LLM and to train the model to prefer short, computation-driven traces. Offloading deterministic computation reduces arithmetic and numerical errors and shortens traces, and GRPO fine-tuning further encourages concise, correct answers. The result is a more accurate and more interpretable math-solving agent without requiring a massive model or heavyweight external tools.\n\n## Try It Yourself\n\nCheck out the [GitHub repo](https://github.com/IntelLabs/DeepMath) and share your feedback! Contributions welcome. 🚀\n\n## Citation\n\nIf you use DeepMath in your research, please cite:\n\n```bibtex\n@software{deepmath2025,\n  author = {Fleischer, Daniel and Berchansky, Moshe and Wasserblat, Moshe},\n  title = {DeepMath: A Lightweight Math Reasoning Agent for LLMs},\n  year = {2025},\n  publisher = {Intel AI Labs},\n  url = {https://github.com/IntelLabs/DeepMath}\n}\n```\n\n## Limitations & Future Work\n\n- **Scope**: we focused on a small model and on mathematical reasoning.\n\n- **Generalization**: evaluated on contest-style math; results may not transfer to open-ended mathematical creativity or formal proofs.\n\n- Executing generated code is inherently risky. DeepMath uses strict sandboxing and resource limits, but any deployment should carefully manage attack surfaces and enforce rate limits.\n\n## References\n\n\\[1\\] Luo, Michael, Sijun Tan, Justin Wong, et al. 2025. “DeepScaleR: Surpassing O1-Preview with a 1.5B Model by Scaling RL.” [https://pretty-radio-b75.notion.site/DeepScaleR-Surpassing-O1-Preview-with-a-1-5B-Model-by-Scaling-RL-19681902c1468005bed8ca303013a4e2](https://pretty-radio-b75.notion.site/DeepScaleR-Surpassing-O1-Preview-with-a-1-5B-Model-by-Scaling-RL-19681902c1468005bed8ca303013a4e2)\n\n\\[2\\] Liu, Mingjie, Shizhe Diao, Ximing Lu, et al. 2025. “ProRL: Prolonged Reinforcement Learning Expands Reasoning Boundaries in Large Language Models.” arXiv:2505.24864. Preprint, arXiv, May 30. [https://doi.org/10.48550/arXiv.2505.24864](https://doi.org/10.48550/arXiv.2505.24864)\n\n\\[3\\] Moshkov, Ivan, Darragh Hanley, Ivan Sorokin, et al. 2025. “AIMO-2 Winning Solution: Building State-of-the-Art Mathematical Reasoning Models with OpenMathReasoning Dataset.” arXiv:2504.16891. Preprint, arXiv, April 23. [https://doi.org/10.48550/arXiv.2504.16891](https://doi.org/10.48550/arXiv.2504.16891)\n",
  "dateCreated": "2026-10-04T12:50:30.804479567+00:00",
  "date_created": "2025-12-04T00:00:00+00:00",
  "date_modified": "2025-12-04T00:00:00+00:00",
  "date_published": "2025-12-04T00:00:00+00:00",
  "description": "By Intel AI Software Group DeepMath is an aligned math reasoning agent built on Qwen3-4B Thinking and fine-tuned with GRPO (Group Relative Policy Optimization). Instead of verbose text, the model emits tiny Python snippets for intermediate…",
  "external_url": "https://huggingface.co/blog/intel-deepmath",
  "git": {
    "commit": "ebb0941c5c06a2ab2377342d607535262a3e969e",
    "path": "items/huggingface-co/2025/12/2025-12-04-deepmath-a-lightweight-math-reasoning-agent-with-smolagents.md",
    "permalink": "https://github.com/aymericbeaumet/aggr-instance/blob/ebb0941c5c06a2ab2377342d607535262a3e969e/items/huggingface-co/2025/12/2025-12-04-deepmath-a-lightweight-math-reasoning-agent-with-smolagents.md"
  },
  "id": "urn:aggr:item:b23f02cf5fee362830c63847b88d94405763b263",
  "isBasedOn": {
    "@id": "https://huggingface.co/blog/intel-deepmath"
  },
  "mainEntity": {
    "@id": "https://huggingface.co/blog/intel-deepmath",
    "@type": "CreativeWork",
    "archivedAt": {
      "@id": "https://aggr.aymericbeaumet.com/items/huggingface-co/2025-12-04-deepmath-a-lightweight-math-reasoning-agent-with-smolagents/#webpage"
    },
    "dateModified": "2025-12-04T00:00:00+00:00",
    "datePublished": "2025-12-04T00:00:00+00:00",
    "headline": "DeepMath: A lightweight math reasoning Agent with smolagents",
    "timeRequired": "PT5M",
    "url": "https://huggingface.co/blog/intel-deepmath",
    "wordCount": 1059
  },
  "name": "Archived snapshot: DeepMath: A lightweight math reasoning Agent with smolagents",
  "preview": null,
  "reading_minutes": 5,
  "replicated_at": null,
  "source": "huggingface-co",
  "source_name": "Hugging Face - Blog",
  "summary": null,
  "tags": [],
  "temporalCoverage": "2026-10-04T12:50:30.804479567+00:00",
  "title": "DeepMath: A lightweight math reasoning Agent with smolagents",
  "url": "https://aggr.aymericbeaumet.com/items/huggingface-co/2025-12-04-deepmath-a-lightweight-math-reasoning-agent-with-smolagents/",
  "word_count": 1059
}