![\"LeRobot](\"https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/lerobot-blog/release-v0.6.0/lerobot%20v0.6.0.png\")
This new release is about closing the robot learning loop: policies that imagine the future before acting, reward models that tell you when your robot succeeds, a deployment CLI that turns failures into training data, and six new simulation benchmarks to measure it all. It also brings depth sensing, VLM-powered dataset annotation, custom video encoding, cloud training on HF Jobs, and a much leaner install.
\nLeRobot v0.6.0 introduces world model policies (VLA-JEPA, FastWAM, LingBot-VA) that learn to imagine the future, a wave of new VLAs (GR00T N1.7, MolmoAct2, EO-1, EVO1, Multitask DiT), and a new reward models API (Robometer, TOPReward). It ships six new simulation benchmarks unified under lerobot-eval, the lerobot-rollout CLI with DAgger-style human-in-the-loop corrections, FSDP training, and cloud training on HF Jobs. Datasets get depth support, an automatic language annotation pipeline, custom video encoding, and up to 2x faster data loading, all on top of a leaner installation.
The robotics world is asking a big question: do world models actually help robot policies? v0.6.0 brings three policies to LeRobot to help answer that question. Each one learns to imagine the future as part of its training, and each takes a different path to keep that imagination affordable.
\nVLA-JEPA teaches a compact VLA (built on Qwen3-VL-2B) to predict the future in latent space while it learns to act: during training, a JEPA world model has to anticipate upcoming frames from the model's own actions. The trick is that the world model then disappears at inference, so you get world-model supervision at zero extra inference cost. Three ready-to-use checkpoints are on the Hub, including a DROID-pretrained base for fine-tuning:
\n\nCheck out the VLA-JEPA documentation and the paper to learn more.
\nLingBot-VA goes one step further: an autoregressive video-action model that predicts future video and actions together, chunk by chunk, and feeds real observations back in to keep its imagination grounded. You can even save what the robot imagined (--policy.save_predicted_video=true) and compare it with what actually happened. Inference runs on a single 24β32 GB GPU. Check out the documentation and the paper for the technical details.
![\"LingBot-VA](\"https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/lerobot-blog/release-v0.6.0/gifs/lingbot_va_viz_1.gif\")
FastWAM asks the question in its paper title: do world action models need test-time future imagination? It pairs a ~5B video-generation expert with a compact action expert in a single network, so the model literally learns to dream its own rollouts. At inference it skips the dreaming entirely and directly denoises action chunks. Fine-tune it from lerobot/fastwam_base, and read more in the documentation.
\nWe upgraded our NVIDIA GR00T integration to GR00T N1.7, the newest open generation of NVIDIA's cross-embodiment foundation model. N1.7 swaps the previous VLM for Cosmos-Reason2-2B (built on Qwen3-VL) feeding a flow-matching action head, and our integration is parity-tested against NVIDIA's original Isaac-GR00T implementation: same inputs, same outputs. Flash-attention is now optional, so pip install 'lerobot[groot]' just works, and you can load NVIDIA's published checkpoints directly.
\n\nGR00T N1.7 replaces N1.5 in LeRobot. If you need N1.5, pin
\nlerobot==0.5.1.
MolmoAct2, the Allen Institute for AI's vision-language-action model, is now ported into LeRobot with the full lifecycle covered: fine-tuning (full or LoRA), evaluation, and real-robot deployment. Ready-made checkpoints with calibration correction baked in mean you can run it zero-shot on an SO-100/101:
\n\nInference fits in ~12 GB at bf16, and LoRA fine-tuning fits on a single 24 GB GPU. See the MolmoAct2 documentation for the full deployment guide.
\n![\"MolmoAct2](\"https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/lerobot-blog/release-v0.6.0/gifs/molmoact2_4.gif\")
EO-1, a VLA pretrained upstream on interleaved vision-text-action data, joins LeRobot: a Qwen2.5-VL-3B backbone with a flow-matching action head, contributed by one of the paper's own authors. Train it with the standard lerobot-train workflow using --policy.type=eo1. Details in the documentation and the paper.
The Multitask Diffusion Transformer policy brings the TRI Large Behavior Models recipe to LeRobot: a ~450M-parameter diffusion transformer conditioned on CLIP vision and language embeddings, so one model learns many tasks selected via natural language. It supports both diffusion and flow-matching objectives, and it is small enough to train yourself. See the documentation.
\nVLAs don't have to be huge. EVO1 packs its policy into 0.77B parameters, an InternVL3-1B backbone with a flow-matching action head, small enough to fine-tune and run in real time on modest GPUs. It ships with two-stage fine-tuning and Real-Time Chunking support out of the box. See the EVO1 documentation and the paper.
\nSuccess detection and progress estimation are missing pieces in the robot learning loop, and v0.6.0 gives them a home. LeRobot now has a unified reward models API (lerobot.rewards), mirroring the policies API, with four reward models behind one interface - the HIL-SERL reward classifier, SARM, and two new additions:
Robometer is a pretrained, general-purpose reward model: point lerobot/Robometer-4B at any LeRobot dataset and it scores task progress and success from raw video plus a language instruction, with no task-specific training required. It is built on Qwen3-VL-4B and trained via trajectory comparisons over a dataset of more than one million robot trajectories (RSS 2026 paper).
\n![\"LeRobot](\"https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/lerobot-blog/release-v0.6.0/gifs/rm_robometer.gif\")
TOPReward goes fully zero-shot: no reward weights at all. It wraps an off-the-shelf VLM (Qwen3-VL) and reads the log-probability of the token "True" given the trajectory video and the task instruction. Any capable VLM becomes a reward function.
\nBoth ship with labeling scripts that write per-frame progress curves into your dataset, ready for reward-aware behavior cloning (RA-BC), dataset quality inspection, and progress-overlay videos. Check the Robometer and TOPReward docs.
\nRecording is no longer stuck with one hard-coded codec. The new --dataset.rgb_encoder.* options expose the full encoding surface (codec, quality, pixel format, GOP, presets), and vcodec=auto probes for hardware encoders like NVENC, VideoToolbox, VAAPI, and QSV before falling back to the default software AV1 encoder. For existing datasets, one command re-encodes everything:
\nFull details in the video encoding documentation.
\nPlug in an Intel RealSense, set use_depth: true, and LeRobot records depth maps end to end: captured in millimeters, compressed as compact 12-bit depth video streams alongside your RGB cameras, and decoded back to physical units at training time. Depth renders live during recording and in lerobot-dataset-viz, and it works across SO-100/101, Koch, OpenArm, reBot, Unitree G1 and more.
![\"LeRobot](\"https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/lerobot-blog/release-v0.6.0/gifs/depth2.gif\")
Your dataset stops being one task string per episode. LeRobot datasets now natively store rich language annotations (timestamped subtasks, plans, memory, corrections, speech, and per-camera VQA pairs), and the new lerobot-annotate CLI fills them in automatically using a VLM that watches your episodes:
\nA YAML recipe layer then renders these annotations into chat-style training messages at sample time: exactly the data tomorrow's long-horizon, talking robot policies will train on. Scale it up with HF Jobs, and read the annotation pipeline docs to learn more.
\nTraining on video datasets is now up to ~2x faster out of the box: multi-camera frames decode in parallel, dataloader workers ship compact uint8 frames (4x less memory between processes), and persistent workers keep decoder caches alive across epochs. Loading a subset of a large dataset (episodes=[...]) went from minutes to milliseconds (275 s down to 0.06 s in our benchmark). Sampling is also deterministic and resumable now, so interrupted trainings restart sample-exact.
v0.5.0 planted the flag on LeRobot as an evaluation hub for VLAs; v0.6.0 makes it the real deal with six new simulation benchmarks, all runnable through the same lerobot-eval CLI, each with a docs page, a Docker image, and a SmolVLA baseline checkpoint smoke-tested in CI:
\nSimulator backends require specific system dependencies with their own install steps; each docs page has the exact recipe, and every benchmark ships a ready-made Docker image if you'd rather skip the setup.
\nTogether with LIBERO, Meta-World, and NVIDIA IsaacLab-Arena, that makes nine benchmark families under one roof, and a new Adding a New Benchmark guide documents exactly how to plug in yours. Evaluation also got faster: parallel eval now defaults to async vectorized environments, benchmarked at up to 2x faster.
\n![\"LeRobot](\"https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/lerobot-blog/release-v0.6.0/gifs/benchmarks2.gif\")
lerobot-rollout: deployment gets its own CLIDeploying a policy used to be a hack on top of lerobot-record. The new lerobot-rollout CLI makes deployment its own workflow, with pluggable strategies and inference backends (including Real-Time Chunking for slow compatible VLAs). The base strategy just runs the policy. sentry records continuously, rotating episodes and uploading to the Hub as it goes. highlight keeps a ring buffer and saves the last N seconds when you hit a key, so an interesting moment is never lost. episodic mirrors the classic episode/reset recording workflow. And dagger turns deployment into data collection.
With the DAgger strategy, you watch your policy run, hit a key (or a USB foot pedal) the moment it goes wrong, take over with your leader arm to record the correction, and hand control back. Actuated leaders are driven to the follower's pose before you take over, so the handover is jerk-free. Every correction frame is tagged with an intervention flag, and the resulting dataset is ready for the next fine-tune:
\nDeploy, collect corrections, fine-tune, repeat: the robot learning flywheel is now a CLI flag. Read the deployment docs.
\nRobot foundation models are outgrowing single GPUs. LeRobot training now supports FSDP (fully sharded data parallel) through Accelerate: parameters, gradients, and optimizer state are sharded across GPUs, and checkpoints are gathered back into a plain single-file model.safetensors that loads like any other policy. You can even resume an FSDP run on a different number of GPUs. See the multi-GPU training docs.
No GPU? No problem. The exact same lerobot-train command now runs in the cloud by adding one flag:
\nLeRobot pushes your local dataset to a private Hub repo if needed, submits the job, streams logs to your terminal, and pushes the trained policy to the Hub at the end. Pick anything from a T4 to 8x H200 with --job.target (compute is billed pay-as-you-go). Check out the documentation
pip install lerobot is now genuinely lightweight, with roughly 40% fewer base dependencies. Feature-scoped extras ([training], [core_scripts], [evaluation], ...) cover the rest, and missing-dependency errors tell you exactly which extra to add. No longer need to install hardware-related dependencies if you only use LeRobot dataset ;)uv installs. --policy.dtype=bfloat16 now drives real mixed-precision training through Accelerate.uv.lock is the authoritative dependency spec for CI, Docker, and development, and the docs include uv install routes for every step, down to picking your CUDA wheel with a single flag.--display_mode=foxglove streams teleoperation, recording, and rollouts to Foxglove, the visualization tool much of the robotics world already uses. It works with remote setups, and lerobot-dataset-viz gets scrubbable dataset playback.lerobot_env_* packages now self-register their environments. The plugin system covers all five component types: robots, cameras, teleoperators, policies, and envs.\n\nCheck the release notes for the full list and migration pointers for the breaking changes.
\n
![\"LeLab](\"https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/lerobot-blog/release-v0.6.0/gifs/LeLab.gif\")
Beyond these headline features, v0.6.0 includes hundreds of bug fixes, documentation improvements, and quality-of-life upgrades across the codebase, from smarter defaults to more reliable CI.
\nWe want to extend a huge thank you to everyone in the community. This release includes work from academia, industry and hobbyists teams who chose LeRobot as the home for their models and benchmarks. Every PR and bug report pushes open-source robotics forward.
\nStay tuned for more to come π€ Get started here! β The LeRobot team β€οΈ
\n", "content_markdown": "This new release is about closing the robot learning loop: policies that imagine the future before acting, reward models that tell you when your robot succeeds, a deployment CLI that turns failures into training data, and six new simulation benchmarks to measure it all. It also brings depth sensing, VLM-powered dataset annotation, custom video encoding, cloud training on HF Jobs, and a much leaner install.\n\n## TL;DR\n\nLeRobot v0.6.0 introduces world model policies (VLA-JEPA, FastWAM, LingBot-VA) that learn to imagine the future, a wave of new VLAs (GR00T N1.7, MolmoAct2, EO-1, EVO1, Multitask DiT), and a new reward models API (Robometer, TOPReward). It ships six new simulation benchmarks unified under `lerobot-eval`, the `lerobot-rollout` CLI with DAgger-style human-in-the-loop corrections, FSDP training, and cloud training on HF Jobs. Datasets get depth support, an automatic language annotation pipeline, custom video encoding, and up to 2x faster data loading, all on top of a leaner installation.\n\n[](https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/lerobot-blog/release-v0.6.0/lerobot%20v0.6.0.png)\n\n## Table of contents\n\n- [LeRobot v0.6.0: Imagine, Evaluate, Improve](https://huggingface.co/blog/lerobot-release-v060#lerobot-v060-imagine-evaluate-improve)\n - [TL;DR](https://huggingface.co/blog/lerobot-release-v060#tldr)\n - [Table of contents](https://huggingface.co/blog/lerobot-release-v060#table-of-contents)\n - [World models: policies that imagine](https://huggingface.co/blog/lerobot-release-v060#world-models-policies-that-imagine)\n - [VLA-JEPA](https://huggingface.co/blog/lerobot-release-v060#vla-jepa)\n - [LingBot-VA](https://huggingface.co/blog/lerobot-release-v060#lingbot-va)\n - [FastWAM](https://huggingface.co/blog/lerobot-release-v060#fastwam)\n - [VLAs: the model zoo keeps growing](https://huggingface.co/blog/lerobot-release-v060#vlas-the-model-zoo-keeps-growing)\n - [GR00T N1.7](https://huggingface.co/blog/lerobot-release-v060#gr00t-n17)\n - [MolmoAct2](https://huggingface.co/blog/lerobot-release-v060#molmoact2)\n - [EO-1](https://huggingface.co/blog/lerobot-release-v060#eo-1)\n - [Multitask DiT](https://huggingface.co/blog/lerobot-release-v060#multitask-dit)\n - [EVO1](https://huggingface.co/blog/lerobot-release-v060#evo1)\n - [Reward models: knowing when your robot succeeds](https://huggingface.co/blog/lerobot-release-v060#reward-models-knowing-when-your-robot-succeeds)\n - [Robometer](https://huggingface.co/blog/lerobot-release-v060#robometer)\n - [TOPReward](https://huggingface.co/blog/lerobot-release-v060#topreward)\n - [Datasets: faster loading, richer data](https://huggingface.co/blog/lerobot-release-v060#datasets-faster-loading-richer-data)\n - [Your codec, your rules](https://huggingface.co/blog/lerobot-release-v060#your-codec-your-rules)\n - [Depth support, end to end](https://huggingface.co/blog/lerobot-release-v060#depth-support-end-to-end)\n - [Language annotations at scale](https://huggingface.co/blog/lerobot-release-v060#language-annotations-at-scale)\n - [Up to 2x faster data loading](https://huggingface.co/blog/lerobot-release-v060#up-to-2x-faster-data-loading)\n - [Benchmarks: one CLI to evaluate them all](https://huggingface.co/blog/lerobot-release-v060#benchmarks-one-cli-to-evaluate-them-all)\n - [Training & inference](https://huggingface.co/blog/lerobot-release-v060#training--inference)\n - [`lerobot-rollout`: deployment gets its own CLI](https://huggingface.co/blog/lerobot-release-v060#lerobot-rollout-deployment-gets-its-own-cli)\n - [FSDP: train models bigger than your GPU](https://huggingface.co/blog/lerobot-release-v060#fsdp-train-models-bigger-than-your-gpu)\n - [Cloud training with HF Jobs](https://huggingface.co/blog/lerobot-release-v060#cloud-training-with-hf-jobs)\n - [Codebase: leaner and cleaner](https://huggingface.co/blog/lerobot-release-v060#codebase-leaner-and-cleaner)\n - [Community & ecosystem](https://huggingface.co/blog/lerobot-release-v060#community--ecosystem)\n - [Final thoughts](https://huggingface.co/blog/lerobot-release-v060#final-thoughts)\n\n## World models: policies that imagine\n\nThe robotics world is asking a big question: do world models actually help robot policies? v0.6.0 brings three policies to LeRobot to help answer that question. Each one learns to imagine the future as part of its training, and each takes a different path to keep that imagination affordable.\n\n### VLA-JEPA\n\nVLA-JEPA teaches a compact VLA (built on Qwen3-VL-2B) to predict the future in latent space while it learns to act: during training, a JEPA world model has to anticipate upcoming frames from the model's own actions. The trick is that the world model then disappears at inference, so you get world-model supervision at zero extra inference cost. Three ready-to-use checkpoints are on the Hub, including a DROID-pretrained base for fine-tuning:\n\n```bash\nlerobot-train \\\n --policy.path=lerobot/VLA-JEPA-Pretrain \\\n --dataset.repo_id=${HF_USER}/my_dataset \\\n --policy.repo_id=${HF_USER}/my_finetuned_policy\n```\n\nCheck out the [VLA-JEPA documentation](https://huggingface.co/docs/lerobot/v0.6.0/vla_jepa) and the [paper](https://arxiv.org/abs/2602.10098) to learn more.\n\n### LingBot-VA\n\nLingBot-VA goes one step further: an autoregressive video-action model that predicts future video and actions together, chunk by chunk, and feeds real observations back in to keep its imagination grounded. You can even save what the robot imagined (`--policy.save_predicted_video=true`) and compare it with what actually happened. Inference runs on a single 24β32 GB GPU. Check out the [documentation](https://huggingface.co/docs/lerobot/v0.6.0/lingbot_va) and the [paper](https://arxiv.org/pdf/2601.21998) for the technical details.\n\n[](https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/lerobot-blog/release-v0.6.0/gifs/lingbot_va_viz_1.gif)\n\n### FastWAM\n\nFastWAM asks the question in its paper title: do world action models need test-time future imagination? It pairs a ~5B video-generation expert with a compact action expert in a single network, so the model literally learns to dream its own rollouts. At inference it skips the dreaming entirely and directly denoises action chunks. Fine-tune it from [lerobot/fastwam\\_base](https://huggingface.co/lerobot/fastwam_base), and read more in the [documentation](https://huggingface.co/docs/lerobot/v0.6.0/fastwam).\n\n## VLAs: the model zoo keeps growing\n\n### GR00T N1.7\n\nWe upgraded our NVIDIA GR00T integration to GR00T N1.7, the newest open generation of NVIDIA's cross-embodiment foundation model. N1.7 swaps the previous VLM for Cosmos-Reason2-2B (built on Qwen3-VL) feeding a flow-matching action head, and our integration is parity-tested against NVIDIA's original Isaac-GR00T implementation: same inputs, same outputs. Flash-attention is now optional, so `pip install 'lerobot[groot]'` just works, and you can load [NVIDIA's published checkpoints](https://huggingface.co/nvidia/GR00T-N1.7-3B) directly.\n\n> GR00T N1.7 replaces N1.5 in LeRobot. If you need N1.5, pin `lerobot==0.5.1`.\n\n### MolmoAct2\n\nMolmoAct2, the Allen Institute for AI's vision-language-action model, is now ported into LeRobot with the full lifecycle covered: fine-tuning (full or LoRA), evaluation, and real-robot deployment. Ready-made checkpoints with calibration correction baked in mean you can run it zero-shot on an SO-100/101:\n\n```bash\nlerobot-rollout \\\n --policy.path=lerobot/MolmoAct2-SO100_101-LeRobot \\\n --robot.type=so100_follower \\\n --robot.port=/dev/ttyACM0 \\\n --robot.cameras='{cam0: {type: opencv, index_or_path: 0, width: 640, height: 480, fps: 30}, cam1: {type: opencv, index_or_path: 2, width: 640, height: 480, fps: 30}}' \\\n --task=\"pick up the red cube\" --duration=30\n```\n\nInference fits in ~12 GB at bf16, and LoRA fine-tuning fits on a single 24 GB GPU. See the [MolmoAct2 documentation](https://huggingface.co/docs/lerobot/v0.6.0/molmoact2) for the full deployment guide.\n\n[](https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/lerobot-blog/release-v0.6.0/gifs/molmoact2_4.gif)\n\n### EO-1\n\nEO-1, a VLA pretrained upstream on interleaved vision-text-action data, joins LeRobot: a Qwen2.5-VL-3B backbone with a flow-matching action head, contributed by one of the paper's own authors. Train it with the standard `lerobot-train` workflow using `--policy.type=eo1`. Details in the [documentation](https://huggingface.co/docs/lerobot/v0.6.0/eo1) and the [paper](https://arxiv.org/abs/2508.21112).\n\n### Multitask DiT\n\nThe Multitask Diffusion Transformer policy brings the TRI Large Behavior Models recipe to LeRobot: a ~450M-parameter diffusion transformer conditioned on CLIP vision and language embeddings, so one model learns many tasks selected via natural language. It supports both diffusion and flow-matching objectives, and it is small enough to train yourself. See the [documentation](https://huggingface.co/docs/lerobot/v0.6.0/multi_task_dit).\n\n### EVO1\n\nVLAs don't have to be huge. EVO1 packs its policy into 0.77B parameters, an InternVL3-1B backbone with a flow-matching action head, small enough to fine-tune and run in real time on modest GPUs. It ships with two-stage fine-tuning and Real-Time Chunking support out of the box. See the [EVO1 documentation](https://huggingface.co/docs/lerobot/v0.6.0/evo1) and the [paper](https://arxiv.org/abs/2511.04555).\n\n## Reward models: knowing when your robot succeeds\n\nSuccess detection and progress estimation are missing pieces in the robot learning loop, and v0.6.0 gives them a home. LeRobot now has a unified reward models API (`lerobot.rewards`), mirroring the policies API, with four reward models behind one interface - the HIL-SERL reward classifier, SARM, and two new additions:\n\n### Robometer\n\nRobometer is a pretrained, general-purpose reward model: point [lerobot/Robometer-4B](https://huggingface.co/lerobot/Robometer-4B) at any LeRobot dataset and it scores task progress and success from raw video plus a language instruction, with no task-specific training required. It is built on Qwen3-VL-4B and trained via trajectory comparisons over a dataset of more than one million robot trajectories ([RSS 2026 paper](https://arxiv.org/abs/2603.02115)).\n\n[](https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/lerobot-blog/release-v0.6.0/gifs/rm_robometer.gif)\n\n### TOPReward\n\nTOPReward goes fully zero-shot: no reward weights at all. It wraps an off-the-shelf VLM (Qwen3-VL) and reads the log-probability of the token \"True\" given the trajectory video and the task instruction. Any capable VLM becomes a reward function.\n\nBoth ship with labeling scripts that write per-frame progress curves into your dataset, ready for reward-aware behavior cloning (RA-BC), dataset quality inspection, and progress-overlay videos. Check the [Robometer](https://huggingface.co/docs/lerobot/v0.6.0/robometer) and [TOPReward](https://huggingface.co/docs/lerobot/v0.6.0/topreward) docs.\n\n## Datasets: faster loading, richer data\n\n### Your codec, your rules\n\nRecording is no longer stuck with one hard-coded codec. The new `--dataset.rgb_encoder.*` options expose the full encoding surface (codec, quality, pixel format, GOP, presets), and `vcodec=auto` probes for hardware encoders like NVENC, VideoToolbox, VAAPI, and QSV before falling back to the default software AV1 encoder. For existing datasets, one command re-encodes everything:\n\n```bash\nlerobot-edit-dataset \\\n --repo_id ${HF_USER}/my_dataset \\\n --operation.type reencode_videos \\\n --operation.rgb_encoder.vcodec h264 \\\n --operation.rgb_encoder.crf 23\n```\n\nFull details in the [video encoding documentation](https://huggingface.co/docs/lerobot/v0.6.0/video_encoding_parameters).\n\n### Depth support, end to end\n\nPlug in an Intel RealSense, set `use_depth: true`, and LeRobot records depth maps end to end: captured in millimeters, compressed as compact 12-bit depth video streams alongside your RGB cameras, and decoded back to physical units at training time. Depth renders live during recording and in `lerobot-dataset-viz`, and it works across SO-100/101, Koch, OpenArm, reBot, Unitree G1 and more.\n\n[](https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/lerobot-blog/release-v0.6.0/gifs/depth2.gif)\n\n### Language annotations at scale\n\nYour dataset stops being one task string per episode. LeRobot datasets now natively store rich language annotations (timestamped subtasks, plans, memory, corrections, speech, and per-camera VQA pairs), and the new `lerobot-annotate` CLI fills them in automatically using a VLM that watches your episodes:\n\n```bash\nlerobot-annotate \\\n --repo_id=${HF_USER}/my_dataset \\\n --new_repo_id=${HF_USER}/my_dataset_annotated \\\n --vlm.model_id=Qwen/Qwen2.5-VL-7B-Instruct \\\n --push_to_hub=true\n```\n\nA YAML recipe layer then renders these annotations into chat-style training messages at sample time: exactly the data tomorrow's long-horizon, talking robot policies will train on. Scale it up with HF Jobs, and read the [annotation pipeline docs](https://huggingface.co/docs/lerobot/v0.6.0/annotation_pipeline) to learn more.\n\n### Up to 2x faster data loading\n\nTraining on video datasets is now up to ~2x faster out of the box: multi-camera frames decode in parallel, dataloader workers ship compact uint8 frames (4x less memory between processes), and persistent workers keep decoder caches alive across epochs. Loading a subset of a large dataset (`episodes=[...]`) went from minutes to milliseconds (275 s down to 0.06 s in our benchmark). Sampling is also deterministic and resumable now, so interrupted trainings restart sample-exact.\n\n## Benchmarks: one CLI to evaluate them all\n\nv0.5.0 planted the flag on LeRobot as an evaluation hub for VLAs; v0.6.0 makes it the real deal with six new simulation benchmarks, all runnable through the same `lerobot-eval` CLI, each with a docs page, a Docker image, and a SmolVLA baseline checkpoint smoke-tested in CI:\n\n- [LIBERO-plus](https://huggingface.co/docs/lerobot/v0.6.0/libero_plus) stress-tests VLAs with roughly 10,000 perturbed variants of LIBERO across seven axes, from lighting and camera viewpoints to rewritten instructions. It tells you when a policy breaks.\n- [RoboTwin 2.0](https://huggingface.co/docs/lerobot/v0.6.0/robotwin) covers 50 bimanual manipulation tasks on SAPIEN with heavy domain randomization, and comes with [more than 100k ready-to-train trajectories](https://huggingface.co/datasets/lerobot/robotwin_unified) on the Hub.\n- [RoboCasa365](https://huggingface.co/docs/lerobot/v0.6.0/robocasa) spans 365 kitchen tasks in 2,500 procedurally generated kitchens on a mobile manipulator, the largest task surface in our lineup.\n- [RoboCerebra](https://huggingface.co/docs/lerobot/v0.6.0/robocerebra) evaluates long-horizon behavior with episodes that chain 3 to 6 sub-goals under language-grounded intermediate instructions, plus a 6,660-episode dataset.\n- [RoboMME](https://huggingface.co/docs/lerobot/v0.6.0/robomme) is a memory exam: can your policy count repetitions, track hidden objects, and imitate demonstrated procedures? 16 tasks across 4 memory suites.\n- [VLABench](https://huggingface.co/docs/lerobot/v0.6.0/vlabench) tests knowledge and reasoning in manipulation, from physics questions to composite tasks like brewing coffee end to end.\n\n```bash\nlerobot-eval \\\n --policy.path=lerobot/smolvla_robotwin \\\n --env.type=robotwin \\\n --env.task=beat_block_hammer \\\n --eval.n_episodes=100 --eval.batch_size=1\n```\n\nSimulator backends require specific system dependencies with their own install steps; each docs page has the exact recipe, and every benchmark ships a ready-made Docker image if you'd rather skip the setup.\n\nTogether with LIBERO, Meta-World, and NVIDIA IsaacLab-Arena, that makes nine benchmark families under one roof, and a new [Adding a New Benchmark guide](https://huggingface.co/docs/lerobot/v0.6.0/adding_benchmarks) documents exactly how to plug in yours. Evaluation also got faster: parallel eval now defaults to async vectorized environments, benchmarked at up to 2x faster.\n\n[](https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/lerobot-blog/release-v0.6.0/gifs/benchmarks2.gif)\n\n## Training & inference\n\n### `lerobot-rollout`: deployment gets its own CLI\n\nDeploying a policy used to be a hack on top of `lerobot-record`. The new `lerobot-rollout` CLI makes deployment its own workflow, with pluggable strategies and inference backends (including Real-Time Chunking for slow compatible VLAs). The `base` strategy just runs the policy. `sentry` records continuously, rotating episodes and uploading to the Hub as it goes. `highlight` keeps a ring buffer and saves the last N seconds when you hit a key, so an interesting moment is never lost. `episodic` mirrors the classic episode/reset recording workflow. And `dagger` turns deployment into data collection.\n\nWith the DAgger strategy, you watch your policy run, hit a key (or a USB foot pedal) the moment it goes wrong, take over with your leader arm to record the correction, and hand control back. Actuated leaders are driven to the follower's pose before you take over, so the handover is jerk-free. Every correction frame is tagged with an `intervention` flag, and the resulting dataset is ready for the next fine-tune:\n\n```bash\nlerobot-rollout \\\n --strategy.type=dagger \\\n --policy.path=${HF_USER}/my_policy \\\n --robot.type=so100_follower \\\n --robot.port=/dev/ttyACM0 \\\n --teleop.type=so101_leader \\\n --teleop.port=/dev/ttyACM1 \\\n --dataset.repo_id=${HF_USER}/dagger_corrections \\\n --dataset.single_task=\"Grasp the block\"\n```\n\nDeploy, collect corrections, fine-tune, repeat: the robot learning flywheel is now a CLI flag. Read the [deployment docs](https://huggingface.co/docs/lerobot/v0.6.0/inference).\n\n### FSDP: train models bigger than your GPU\n\nRobot foundation models are outgrowing single GPUs. LeRobot training now supports FSDP (fully sharded data parallel) through Accelerate: parameters, gradients, and optimizer state are sharded across GPUs, and checkpoints are gathered back into a plain single-file `model.safetensors` that loads like any other policy. You can even resume an FSDP run on a different number of GPUs. See the [multi-GPU training docs](https://huggingface.co/docs/lerobot/v0.6.0/multi_gpu_training).\n\n### Cloud training with HF Jobs\n\nNo GPU? No problem. The exact same `lerobot-train` command now runs in the cloud by adding one flag:\n\n```bash\nlerobot-train \\\n --dataset.repo_id=${HF_USER}/so101_test \\\n --policy.type=act \\\n --policy.repo_id=${HF_USER}/my_policy \\\n --job.target=a10g-small\n```\n\nLeRobot pushes your local dataset to a private Hub repo if needed, submits the job, streams logs to your terminal, and pushes the trained policy to the Hub at the end. Pick anything from a T4 to 8x H200 with `--job.target` (compute is billed pay-as-you-go). [Check out the documentation](https://huggingface.co/docs/lerobot/v0.6.0/hardware_guide#hugging-face-jobs)\n\n## Codebase: leaner and cleaner\n\n- `pip install lerobot` is now genuinely lightweight, with roughly 40% fewer base dependencies. Feature-scoped extras (`[training]`, `[core_scripts]`, `[evaluation]`, ...) cover the rest, and missing-dependency errors tell you exactly which extra to add. **No longer need to install hardware-related dependencies if you only use LeRobot dataset ;)**\n- Supported PyTorch moves to 2.7β2.11, with CUDA 12.8 wheels pinned out of the box for Linux `uv` installs. `--policy.dtype=bfloat16` now drives real mixed-precision training through Accelerate.\n- A committed `uv.lock` is the authoritative dependency spec for CI, Docker, and development, and the docs include `uv` install routes for every step, down to picking your CUDA wheel with a single flag.\n- `--display_mode=foxglove` streams teleoperation, recording, and rollouts to [Foxglove](https://foxglove.dev), the visualization tool much of the robotics world already uses. It works with remote setups, and `lerobot-dataset-viz` gets scrubbable dataset playback.\n- Pip-installable `lerobot_env_*` packages now self-register their environments. The plugin system covers all five component types: robots, cameras, teleoperators, policies, and envs.\n- Keyboard controls during recording now work on Wayland, over SSH, on headless rigs, and on macOS without Accessibility permissions.\n\n> Check the [release notes](https://github.com/huggingface/lerobot/releases) for the full list and migration pointers for the breaking changes.\n\n[](https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/lerobot-blog/release-v0.6.0/gifs/LeLab.gif)\n\n## Community & ecosystem\n\n- LeLab puts the whole LeRobot workflow (calibrate, teleoperate, record, train locally or on HF Jobs, deploy) in a browser UI, no CLI required. It currently supports the SO-ARM101. [Try it out!](https://github.com/huggingface/leLab)\n- Isaac Teleop lets you teleoperate an SO-101 with a VR controller through [NVIDIA's Isaac Teleop stack](https://github.com/NVIDIA/IsaacTeleop) over CloudXR/OpenXR, the result of a collaboration with the NVIDIA team. See the [documentation](https://huggingface.co/docs/lerobot/v0.6.0/isaac_teleop).\n- The new [compute hardware guide](https://huggingface.co/docs/lerobot/v0.6.0/hardware_guide) answers the two questions every newcomer asks: which GPU do I need, and how long will training take? It gives measured VRAM envelopes per policy family and reference training times from an RTX 4090 to 4x H100.\n- The rewritten [Adding a Policy guide](https://huggingface.co/docs/lerobot/v0.6.0/bring_your_own_policies) shows how to ship your own policy, in-tree or as a plugin package with no PR needed.\n\n## Final thoughts\n\nBeyond these headline features, v0.6.0 includes hundreds of bug fixes, documentation improvements, and quality-of-life upgrades across the codebase, from smarter defaults to more reliable CI.\n\nWe want to extend a huge thank you to everyone in the community. This release includes work from academia, industry and hobbyists teams who chose LeRobot as the home for their models and benchmarks. Every PR and bug report pushes open-source robotics forward.\n\nStay tuned for more to come π€ Get started [here](https://github.com/huggingface/lerobot)! β The LeRobot team β€οΈ\n", "dateCreated": "2026-10-19T01:26:53.998049939+00:00", "date_created": "2026-07-07T00:00:00+00:00", "date_modified": "2026-07-07T00:00:00+00:00", "date_published": "2026-07-07T00:00:00+00:00", "description": "This new release is about closing the robot learning loop: policies that imagine the future before acting, reward models that tell you when your robot succeeds, a deployment CLI that turns failures into training data, and six newβ¦", "external_url": "https://huggingface.co/blog/lerobot-release-v060", "git": { "commit": "ebb0941c5c06a2ab2377342d607535262a3e969e", "path": "items/huggingface-co/2026/07/2026-07-07-lerobot-v0-6-0-imagine-evaluate-improve.md", "permalink": "https://github.com/aymericbeaumet/aggr-instance/blob/ebb0941c5c06a2ab2377342d607535262a3e969e/items/huggingface-co/2026/07/2026-07-07-lerobot-v0-6-0-imagine-evaluate-improve.md" }, "id": "urn:aggr:item:104db3252b5de735a7f1599ec66b1e2cc9f0e0fd", "isBasedOn": { "@id": "https://huggingface.co/blog/lerobot-release-v060" }, "mainEntity": { "@id": "https://huggingface.co/blog/lerobot-release-v060", "@type": "CreativeWork", "archivedAt": { "@id": "https://aggr.aymericbeaumet.com/items/huggingface-co/2026-07-07-lerobot-v0-6-0-imagine-evaluate-improve/#webpage" }, "dateModified": "2026-07-07T00:00:00+00:00", "datePublished": "2026-07-07T00:00:00+00:00", "headline": "LeRobot v0.6.0: Imagine, Evaluate, Improve", "image": { "@type": "ImageObject", "caption": null, "height": 90, "url": "https://huggingface.co/blog/assets/lerobot-release-v060/thumbnail.png", "width": 160 }, "timeRequired": "PT12M", "url": "https://huggingface.co/blog/lerobot-release-v060", "wordCount": 2583 }, "name": "Archived snapshot: LeRobot v0.6.0: Imagine, Evaluate, Improve", "preview": { "alt": null, "color": null, "height": 90, "placeholder": { "data_url": "data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAAEAAAABCAQAAAC1HAwCAAAAC0lEQVR42mNk+A8AAQUBAScY42YAAAAASUVORK5CYII=", "hash": "" }, "url": "https://huggingface.co/blog/assets/lerobot-release-v060/thumbnail.png", "width": 160 }, "reading_minutes": 12, "replicated_at": null, "source": "huggingface-co", "source_name": "Hugging Face - Blog", "summary": null, "tags": [], "temporalCoverage": "2026-10-19T01:26:53.998049939+00:00", "title": "LeRobot v0.6.0: Imagine, Evaluate, Improve", "url": "https://aggr.aymericbeaumet.com/items/huggingface-co/2026-07-07-lerobot-v0-6-0-imagine-evaluate-improve/", "word_count": 2583 }