Browse aymeric's, an independent, searchable archive of readable snapshots from followed feeds, preserved in Git with aggr. https://aggr.aymericbeaumet.com/sources/ricursive.com/ 2026-11-04T18:53:29+00:00 aymeric's aggr urn:aggr:item:391569679cd3557c37c2d8e83e53f0dcc15dda4d 2026-11-04T18:53:29+00:00 2026-11-04T18:53:29+00:00 Philpax

Rather than investing in a brute-force RL climb, we carefully iterated on our SFT and RL recipes with a focus on compute efficiency. We opted not to buy any posttraining data, instead devising our own synthetic data pipelines, quality…

<p>Rather than investing in a brute-force RL climb, we carefully iterated on our SFT and RL recipes with a focus on compute efficiency. We opted not to buy any posttraining data, instead devising our own synthetic data pipelines, quality filtering, and coverage metrics.</p> <p>Our mixed RL recipe uses <a target="_blank" rel="noopener noreferrer" href="https://openreview.net/forum?id=xsonlP8DRV">truncated importance sampling</a>, <a target="_blank" rel="noopener noreferrer" href="https://arxiv.org/abs/2609.20807">score centering</a>, and <a target="_blank" rel="noopener noreferrer" href="https://microsoft.ai/pdf/mai-thinking-1.pdf">linear length penalty</a> with high-quality long-horizon synthetic environments. The RL tasks are tuned for our model capabilities, and we continuously added harder tasks as the model improved.</p> <p>The end result was a steep RL climb from a strong initial SFT checkpoint. We reached a TerminalBench 2.1 score of 80.5 with only 1.4M rollouts, roughly 70x fewer rollouts than Reflection Beam, the prior SOTA.</p>