Robot Reel turns the run records of policy simulations and physics simulations into deliverables: replays you can step through frame by frame in the browser, raw data with hash verification, and 3D scenes you can keep editing in Blender / OpenUSD. The question it cares about is this: a robot demo video proves nothing on its own unless the viewer can get back to the action, state, inference time, and runtime environment behind every frame. Robot Reel delivers those records together with the video, along with verifiers.
Core experiment: the SmolVLA policy stress lab
This is the experiment with the most complete evidence in the project. SmolVLA runs closed-loop inference on one LIBERO task from 10 paired initial states, under each of three conditions: reference, light intensity reduced by 25%, and camera moved by 12 cm. That makes 30 trials, each with at most 160 actions (8 seconds of simulated time). The experiment was recorded on 2026-09-12 on an NVIDIA L40S with PyTorch 2.11.0+cu130:
| Condition | Successes | Hit step limit | 95% Wilson interval |
|---|---|---|---|
| Reference | 5/10 | 5 | 23.7–76.3% |
| Light reduced 25% | 4/10 | 6 | 16.8–68.7% |
| Camera +12 cm | 7/10 | 3 | 39.7–89.2% |
The documentation does not stop at success rates. The camera condition’s net gain of 2 successes is actually 3 pairs going from failure to success and 1 pair going from success to failure; the online page groups results by paired outcome and keeps changes in both directions. All 14 unsuccessful trials ended by hitting the action budget limit. In the last tenth of each episode, end-effector travel was between 44.8 and 138.6 mm, above the 1 mm stall threshold in every case, which shows the arm was still moving when it was cut off. Whether it was still making progress on the task, and whether more steps would have led to success, would need a separate experiment, according to the documentation. A full repeat on the same L40S matched on all 30/30 trials in outcomes, step counts, states, and action values, and on all 360/360 rendered frames used by policy calls; of the 3,195 frames used only for recording, 1 differed, and that frame was never fed to inference. Each policy call took 0.491 seconds on average, as measured. The author states explicitly that this is a controlled diagnostic experiment, not an official LIBERO leaderboard result, and that because the three intervals overlap, it cannot support the claim that “a camera offset improves the policy”.
The 30 trials and 20 paired outcomes are published as a tabular dataset on Hugging Face (Apache-2.0, with no images, videos, or weights). Telemetry is provided in MCAP format and can be opened in Foxglove; the featured pair no. 9 also has a Rerun workspace.
The other labs
- LeRobot dataset replay: exports one episode of any LeRobotDataset (from the Hub or local, v3.0 and v2.x) as a folder that opens offline. All cameras share one timeline, commanded and measured curves for each joint are shown side by side, and you can jump to the frame where the two differ most. Export pins the Hub commit and records the SHA-256 of every source file, and LeRobot does not need to be installed.
- Solver Lab: Genesis and Newton each record projectile motion three times on L40S / CUDA, at integration rates of 30, 120, and 480 steps per second. The maximum position error shrinks with the step size, from 32.70 cm down to 2.05 cm. This scene has no collisions and no drag, and the two engines produce identical values, so it is only an integration-error diagnostic, not a simulator ranking.
- Microduck motion lab: Pollen Robotics’ Microduck runs the official ONNX walking policy in MuJoCo (50 Hz, 14 joints), keeping 8,400 measured joint samples under two velocity commands, 0.3 and 0.5 m/s. The actuators use a PD approximation; these are simulation results, not hardware results.
- Butterfly effect, cloth, Scene Lab, Director: 12 Newton double-pendulum worlds whose release angles differ successively by 0.05°; three Newton cloth simulations that change only the bending coefficient, keeping 42,471 vertex samples; a Blender project of Microduck in a scene built from real-world assets; and a Blender film whose camera positions, captions, and slow motion are orchestrated by an MCP agent, with every frame of the final cut traceable to source samples.
Starting with 0.16.0, robot-reel review-claims compares a model’s structured claims about a replay, item by item, against the recorded outcomes, action counts, and frame labels, without running any inference. Claims that cannot be checked stay “not evaluated”, and visual explanations are never automatically marked as verified.
How to try it
The easiest entry points are the Hugging Face Space and the GitHub Pages site: they only show replays and need no installation or account. The command-line tool requires Python 3.12+:
python3 -m venv .venv
source .venv/bin/activate
python -m pip install robot-reel==0.17.0
robot-reel --helpTo replay your own LeRobot data, taking the SO-101 dataset used for SmolVLA fine-tuning as an example:
python -m pip install 'robot-reel[lerobot]==0.17.0'
robot-reel lerobot lerobot/svla_so101_pickplace --episode 0 --output artifacts/so101
robot-reel lerobot artifacts/so101 --verify --check-media --check-sourceIf you only want to verify the recordings already in the repository, clone the source and run it with the standard library alone:
git clone https://github.com/noteflowai/robot-reel.git
cd robot-reel
python3 -m robot_reel.cli stress docs/stress # verify every trial in the paired experiment
python3 -m robot_reel.cli vla docs/vla # check the recorded policy runs and evidenceRe-recording requires the full runtime environment (CUDA, Newton, Blender, and so on, depending on the experiment); see the recording guide in the repository.
Limitations
- Everything shown in the browser is recorded simulation. The single VLA example is one fixed-seed run, and playback omits the time spent waiting on inference.
- All results come from simulation and are small in scale: the stress experiment has only one task and 10 pairs. Matching repeats show only that results are reproducible under the recorded conditions; they do not prove general determinism and cannot be used for causal attribution.
- Licensing is layered: the code is Apache-2.0; the VLA footage keeps upstream attribution and asset terms; structures and media derived from the Microduck model carry over the upstream non-commercial, share-alike terms. Read
THIRD_PARTY.mditem by item before any commercial use. - The project is maintained independently and does not represent the official position of LeRobot, MuJoCo, Newton, Pollen Robotics, or other upstream projects. The current version is 0.17.0 (2026-09-27).
Links
- Source code: github.com/noteflowai/robot-reel (Chinese README)
- Online site: noteflowai.github.io/robot-reel · Policy stress lab · SO-101 replay example · Solver Lab · Microduck motion lab
- Hugging Face: Space · Paired outcomes dataset
- PyPI: pypi.org/project/robot-reel
- Documentation: docs/stress.md · docs/lerobot.md · THIRD_PARTY.md
