In a hierarchical humanoid system, a VLA or world action model (WAM) outputs reference actions for a low-level whole-body controller. The motion the robot actually performs still passes through tracking control, dynamics, balance and contact, so the two can differ. The preprint HWAM from a LimX Dynamics team (arXiv 2610.12026, 2026-10-08) calls this the “action–execution gap”. Its answer is to have the model generate the measured post-execution proprioceptive state alongside each reference action, and to connect that joint trajectory to visual change through forward- and inverse-dynamics video paths. The authors report that HWAM has the highest success rate of the compared methods on all three real-robot tasks on the LimX OLI humanoid, including 70.6% on Candy Picking against 43.3% for Fast-WAM. All numbers below are author-reported [R]; this site has not reproduced them.
The action–execution gap: the high level sends a reference, not the actual motion
The paper uses a layered interface. A high-level vision–language–action policy generates reference trajectories in kinematic space from images, a language instruction and the current proprioceptive state. A low-level whole-body tracking controller uses real-time proprioceptive feedback to adapt those references to execution dynamics, then drives the robot through joint servoing. The action a_t in the paper is therefore a motion reference handed to the low-level tracking controller, not a torque applied directly to an actuator, and it is not assumed to be realizable point by point. The authors’ example: a reference trajectory requests trunk motion and arm extension at the same time; the low-level controller adjusts the trunk to keep balance, and that also changes the hand’s trajectory at contact.
The authors argue that without an explicit model of the body state actually reached, future-image prediction has to explain two things at once: how the scene changes, and how far the reference action and executed motion diverge. That makes it hard to associate an action with its physical outcome. To show the gap exists, the paper compares each commanded joint reference a_t in recorded data with its aligned post-execution proprioceptive state s_{t+1}, using mean absolute error (MAE):
| Platform | Direct MAE (°) | Steady-state MAE (°) |
|---|---|---|
| LimX OLI (mobile humanoid) | 4.420 | 2.885 |
| ALOHA (stationary bimanual) | 0.648 | 0.402 |
“Steady-state” is the same pairing under approximately constant joint references. Both platforms show non-zero error, and the gap is larger on the mobile humanoid. The authors use this as the motivation for modeling the post-execution state instead of assuming exact tracking. The paper says the full measurement protocol and per-joint statistics are in Appendix A; the arXiv HTML version this site read gives only the summary values above.
Method: put the post-execution state in the generation target, then connect it to images through three paths
A joint state–action target
For a prediction horizon of H steps, HWAM concatenates the action chunk [a_t, …, a_{t+H-1}] and the state chunk [s_{t+1}, …, s_{t+H}] along the feature dimension, so each temporal position is [s_{t+h+1}; a_{t+h}]. The state and action channels are concatenated before token projection and denoised together by one state–action generator at a single shared noise level. In the authors’ words, this lets the action channels see the partially denoised state channels at every denoising step, while state supervision shapes the representation used to generate actions. The paper also states that this construction does not explicitly learn a low-level dynamics model or impose a separate physical-consistency constraint, and that the joint target is not assumed to be useful on its own: it is grounded by the visual paths below.
Two experts, three training paths
The model consists of a video expert (WanVideoDiT) and a state–action expert (ActionDiT) with separate parameters, exchanging information through path-dependent masked cross-attention, following the mixed-expert interface of earlier WAMs such as Fast-WAM. A frozen causal Wan2.2 VAE encodes multi-view images into current and future visual latents; the language instruction is embedded with UMT5-XXL and, together with the current proprioceptive state, forms conditioning shared by both experts. The three paths share experts and parameters and differ only in generation target and visible conditions:
| Path | Visible conditions | Generation target |
|---|---|---|
| Policy | current images, language, current proprioceptive state | state–action trajectory; no future video, matching deployment |
| Forward dynamics (FDM) | current images, state–action trajectory, conditioning | future video |
| Inverse dynamics (IDM) | current and future images, conditioning | state–action trajectory |
Both generated modalities are trained with conditional flow matching, and state and action errors are averaged without extra weights. Each minibatch samples one path uniformly (1/3 each, all weights 1). FDM conditions image prediction on the state actually reached rather than on the command alone; IDM goes the other way, inferring from visual change the actions and the body states they produce. At deployment, predicted states still take part in joint denoising, but only the actions are sent to the low-level controller.
Real-robot results: three tasks on the LimX OLI
The LimX OLI is a full-size humanoid with a 31-DoF whole-body action space (including two 7-DoF arms) and BrainCo dexterous hands. The policy receives RGB images from head and wrist cameras, a language instruction and the current proprioceptive state. Of the three tasks, Candy Picking (pick the candy of the instructed color and place it in its assigned tray region) and Object Collection (place all four tabletop objects in a basket) are stationary manipulation; Plush-Toy Picking is mobile manipulation requiring whole-body coordination. The training sets contain 1,783, 4,220 and 404 episodes respectively; Candy Picking has an additional 422 held-out episodes used only for offline diagnostics. Both experts are fine-tuned for 6 epochs with AdamW at a learning rate of 10⁻⁴, a global batch size of 128, on 16 A800 GPUs; the implementation builds on the FluxVLA Engine.
| Method | Additional robot-data pretraining | Candy Picking | Object Collection | Plush-Toy Picking |
|---|---|---|---|---|
| Cosmos3-Edge | Yes | 15.8 | 0.0 | 56.7 |
| GR00T N1.5 | Yes | 40.5 | 42.5 | 60.0 |
| π0.5 | Yes | 62.5 | 45.0 | 60.7 |
| Fast-WAM | No | 43.3 | 0.0 | 60.0 |
| DiT4DiT | No | 52.5 | 15.0 | — |
| HWAM | No | 70.6 | 46.7 | 73.3 |
Values are real-robot success rates (%); “—” marks a result the paper does not report or has not verified. Relative to π0.5, HWAM’s margin is 8.1 percentage points on Candy Picking, 1.7 on Object Collection and 12.6 on Plush-Toy Picking. On Object Collection, Cosmos3-Edge and Fast-WAM fail to establish the basket grasp the task requires, hence their 0% full-task success. The authors treat Fast-WAM as the closest WAM baseline: it also uses video co-training and does not synthesize future video at deployment.
Only Candy Picking comes with trial counts and 95% Wilson confidence intervals: HWAM 84/119 (interval 61.9–78.0%), π0.5 75/120 (53.6–70.6%), Fast-WAM 52/120 (34.8–52.3%), GR00T N1.5 49/121 (32.2–49.4%). The authors note that these intervals assume independent Bernoulli trials and do not measure variation across training seeds. This site’s reading: the HWAM and π0.5 intervals overlap, and π0.5’s upper bound equals HWAM’s point estimate. Overlapping intervals are not a significance test, but they show that on this set of trials alone, the 8.1-point margin still carries considerable uncertainty. The arXiv HTML version this site read gives no trial counts for the other two tasks.
Clothes folding on ALOHA
The paper also evaluates Clothes Folding on the ALOHA bimanual platform: a task-specific training set of 640 demonstrations, scored on a 0–100 scale with partial credit for completed folding stages. HWAM scores 86.00, GR00T N1.5 78.75, π0.5 77.00, Fast-WAM 66.25, π0 65.00, GR00T N1.7 58.75 and SmolVLA 51.25. The authors state explicitly that this result comes from task-specific training on ALOHA, not zero-shot transfer from the OLI platform.
Ablation and offline diagnostics: the joint target helps under three-path training but hurts under VGM/Policy
On the mobile manipulation task Plush-Toy Picking, the authors run a 2 × 2 ablation: the prediction target is either action-only or joint state–action, and the training recipe is either VGM/Policy (joint optimization of future-video generation and the Policy objective) or FDM/IDM/Policy, with all other settings fixed.
| Prediction target / training recipe | VGM/Policy | FDM/IDM/Policy |
|---|---|---|
| Action only | 60.0 | 55.0 |
| Joint state–action | 45.5 | 73.3 (HWAM) |
The effect of adding state prediction depends on the training recipe: under VGM/Policy, success falls from 60.0% to 45.5%; under FDM/IDM/Policy, it rises from 55.0% to 73.3%. The two effects differ by 32.8 percentage points. The authors take this as support for combining the joint target with three-path training in the evaluated setting, while noting that the ablation evaluates the combined formulation and does not isolate the contribution of individual paths or architectural choices. The arXiv HTML version this site read does not give trial counts for the ablation configurations.
Offline diagnostics run on the held-out Candy Picking episodes with 4 denoising steps, computing MAE in degrees over the first 31 shared joints. Averaged over the 32-step chunk, HWAM’s action MAE is 3.816°, against 4.024° for action-only FDM/IDM/Policy; the authors report that adding state supervision reduces action MAE under every evaluated training recipe. Note that the open-loop diagnostics and the closed-loop ablation use different tasks, so lower open-loop MAE does not directly imply higher closed-loop success: VGM/Policy with state got worse in the closed-loop ablation.
Future-video prediction is evaluated on 12 held-out episodes, in windows immediately before grasp closure. With the model’s own predicted trajectories as conditions, PSNR ranges from 20.345 to 20.844 across configurations, with HWAM at 20.501 and no video-quality advantage; the authors also state that these numbers do not explain the differences in closed-loop success. With ground-truth trajectories as conditions, HWAM, conditioned on both actions and post-execution states, reaches 23.336 against 20.559 for the action-only model, which the authors present as evidence that conditioning on the realized state helps predict visual outcomes.
How this relates to earlier articles on this site
The course lesson Physical AI Lab 0.3: See, Think, Act — Putting Each Rate Back in Its Own System showed that a robot’s “thinking” and “acting” belong to different layers running at different rates. HWAM adds the other side of that boundary: the reference sent by the high level is not necessarily executed as sent, and the high level has to learn that deviation. Physical AI Lab 1.6: Imitation Learning covered action chunking; HWAM’s state–action chunk simply attaches a post-execution state to each position of an action chunk.
ReCo, covered earlier in Frontier Radar, tackles a similar interface problem in quadruped mobile manipulation by a different route: ReCo trains the walking policy to respond repeatably, then identifies its closed-loop response model and hands it to an MPC. HWAM does not identify the low-level dynamics; it treats the measured post-execution state as a variable generated together with the action, and the predicted state is not sent to the controller. The two papers differ in platform, tasks and metrics, so their numbers cannot be compared.
Limitations
- This is an arXiv preprint and every number is author-reported [R]. All authors are affiliated with LimX Dynamics (one also lists the Southern University of Science and Technology); the paper names its evaluation platform as the LimX OLI humanoid. This site has not reproduced any result. The paper says code, dataset and a project page are available; this site has not checked them.
- Only Candy Picking reports trial counts and confidence intervals, and those intervals reflect trial sampling only, not variation across training seeds. On Object Collection the margin over π0.5 is just 1.7 points, and the arXiv HTML version this site read gives no trial count for that task.
- The ablation covers one task and compares only the full combinations; it cannot separate the contributions of FDM, IDM and Policy.
- HWAM does not explicitly identify low-level dynamics, and its predicted states are not sent to the controller; it also shows no advantage in image-prediction quality.
- The Candy Picking breakdowns by motion direction and workspace region are within-task breakdowns, not evaluations on a held-out spatial split. The ALOHA result comes from task-specific training, not zero-shot cross-platform transfer.
- The paper’s AI use statement says generative AI tools assisted with code implementation, drafting and editing, and pre-submission feedback on methodology and experimental reporting; the authors state that they take responsibility for the final content.
Links
- HWAM: Humanoid World Action Model With Joint State–Action Generation (arXiv 2610.12026)
- Full paper in HTML
- Frontier Radar | ReCo: make a walking policy’s command response predictable first, so MPC can coordinate a quadruped’s base and arm
- Physical AI Lab 0.3: See, Think, Act — Putting Each Rate Back in Its Own System
- Physical AI Lab 1.6: Imitation Learning — Working Out How an Error Rolls Out of the Demonstrations’ Coverage, Then Putting the ACT, Diffusion Policy and Data Scaling Numbers Back in Their Own Conditions
- Physical AI Lab: a physical AI course where every number can be checked (course outline)
