Loco-manipulation demonstrations for humanoid robots are hard to collect: they need synchronized egocentric video, proprioception and executable whole-body commands. Human video and motion-capture data are far more plentiful, but much of it contains only video or only motion, and human motion is not a command a robot can execute. The preprint Being-M0.7 from the BeingBeyond Team (arXiv 2610.11283, 2026-10-08) works in three stages: it first jointly predicts future visual latents and body motion on more than 10,000 hours of raw human data, then mid-trains on robot trajectories, and finally freezes this prior so that a separate action expert can draw on its predictions through gated cross-attention and output whole-body commands for a Unitree G1. The authors report 128 successes in 180 trials on the SIMPLE simulation benchmark, the highest of the compared methods, and 13 successes in 15 real-robot trials, tying GR00T-N1.6. All numbers below are author-reported [R]; this site has not reproduced them.
Data: three supervision streams and one body representation shared by humans and robots
The raw collection exceeds 10,000 hours before filtering and temporal segmentation and is organized into three supervision streams: paired egocentric video–motion data, video-only data and motion-only data. Paired samples supervise joint video–motion prediction; unpaired samples supervise only the modality they have, but with the same shared model parameters. A missing modality is replaced by a learned placeholder embedding and isolated by attention masks, so placeholder tokens exchange no information with valid visual or motion tokens.
The public sources listed in the paper are Ego4D, Xperience, Nymeria, Bones-SEED, SnapMoGen, HumanML3D and Lafan1, plus subsets of the Being-H0.5 and Being-M0.5 training corpora and commercially acquired datasets. The mixed-data loader applies no source reweighting (dataset_weights=null), so the stream proportions follow the number of valid examples after filtering rather than a fixed equal mixture. The HTML text of the paper read for this article gives neither hours per stream nor a post-filtering total; Figure 2 of the paper summarizes the sources, and this site did not read the figure.
Human and robot joints differ, so the paper describes both with one 22-dimensional “head-root” vector:
| Dimensions | Content |
|---|---|
| 1–3 | Root (head) translation; the model input is the accumulated relative head position |
| 4–9 | Head orientation in a continuous 6D rotation representation |
| 10 | Initial head height |
| 11–22 | 3D positions of the left hand, right hand, left foot and right foot in the head frame |
Human motion and robot trajectories are both written in this vector, with the same component order and coordinate convention, and resampled to 5 Hz. According to the paper, the representation describes head motion and the positions of four end effectors without requiring identical joint-angle parameterizations for human and robot. On the visual side, both the inputs and the prediction targets are latents from a frozen DINO encoder, with no pixel-level reconstruction; the authors argue that joint visual–motion prediction pushes the visual representations to focus on underlying kinematic change rather than on the detail needed to reconstruct pixels.
Three training stages, and how the frozen prior connects to the action expert
| Stage | Data | Module trained | Objective |
|---|---|---|---|
| Pre-training | The three streams of human data | Prior: a video–motion Mixture of Transformers, 30 layers, width 768 | Flow matching on visual latents and motion; when motion is present, two geometry-aware auxiliary losses on the global trajectory and local end-effector velocity |
| Robot mid-training | Video, text and 22-D motion from robot trajectories | Prior | Visual and motion flow matching, with the motion term computed only when motion is present |
| Action post-training | Video, text, current proprioception and demonstrated action chunks from the same robot trajectories | Only the action expert (30 layers, width 1024) and its observation and context projections; the prior is frozen | Flow matching on action chunks |
All three stages use 8 GPUs, with effective batch sizes of 2048, 512 and 512. The prior used in post-training is the mid-training checkpoint at 50,000 optimizer updates. The DINO and DistilBERT encoders stay frozen in every stage.
The gated connection
During post-training and deployment, motion prediction is switched off: the prior generates only future visual latents from the current frame and the language instruction, 24 frames at 5 Hz covering 4.8 seconds. The hidden states of these 24 frames (taken before the corresponding prior layer) serve as keys and values injected by cross-attention at layers 3, 6, 9 … 30 of the action expert; the observed frame is excluded from the expert’s cache. Only action tokens issue this query. Proprioceptive tokens do not query the prior and influence actions only through local self-attention inside the expert. How much is injected into each action token is set by a gate: a scalar obtained by layer-normalizing the token’s local features and the proposed update, concatenating them, and applying a linear layer and a sigmoid. Information flows only from the prior to the expert, and action-loss gradients do not reach the prior.
Two further details matter. First, in post-training the expert sees the prior’s own predictions sampled from noise (3 sampling steps), not noised ground-truth futures; the authors say this reduces the mismatch between the conditioning sources at training and inference time. At inference the prior uses 10 sampling steps. Second, “disabling motion prediction” removes motion conditioning, motion-state generation and motion supervision, but the implementation still runs the shared forward pass, including the masked motion slots, and discards those predictions; it does not prune all motion-branch computation. Real-robot experiments add sample-level world-context dropout with probability 0.1 during post-training, to encourage the expert to use current observations; simulation experiments do not.
Three rates at deployment
The expert outputs a 30-step action chunk each time, which takes 0.6 seconds to execute at 50 Hz, so it is queried every 0.6 seconds (about 1.67 Hz); commands are executed every 0.02 seconds (50 Hz). Each action is 66-dimensional: a 64-dimensional SONIC universal control token plus one open/close command for each hand. The observation is one frame from the head camera and a 63-dimensional proprioceptive vector (roll and pitch, angular velocity, joint positions and joint velocities). The prior’s visual plan is cached, reused across action queries and refreshed before the prediction window runs out; the text read for this article is cut off at this point and does not give the exact refresh interval. The policy runs on an external workstation with one NVIDIA RTX 4090 and talks to the low-level controller on the robot in a closed loop over ZMQ; the text read does not report inference time.
SIMPLE: 180 simulated trials and the ablations
The simulation evaluation uses six tasks from the SIMPLE benchmark, each run 10 times at easy, medium and hard domain-randomization levels, for 180 trials per method. Every method and ablation variant is trained multi-task: one model is trained jointly on demonstrations from all six tasks and told which task to perform by the task instruction. The baselines are GR00T-N1.6, Ψ0 and π0.5. The table below adds up the successes over the three difficulty levels in Tables 1 and 2 of the paper; each cell is out of 30:
| Method | BendPick | Handover | Mobile P&P | Grasp | XMoveBendPick | XMovePick | Total (/180) |
|---|---|---|---|---|---|---|---|
| GR00T-N1.6 | 11 | 8 | 0 | 22 | 20 | 0 | 61 |
| Ψ0 | 27 | 20 | 0 | 28 | 20 | 21 | 116 |
| π0.5 | 0 | 19 | 19 | 20 | 26 | 11 | 95 |
| Being-M0.7 (full) | 22 | 28 | 13 | 28 | 9 | 28 | 128 |
| Without pre-training | 0 | 28 | 7 | 21 | 11 | 21 | 88 |
| Without mid-training | 2 | 26 | 14 | 24 | 15 | 26 | 107 |
| Language-only context | 13 | 25 | 10 | 28 | 21 | 8 | 105 |
The full model totals 128/180 (71.1%), against 116/180 (64.4%) for Ψ0, 95/180 (52.8%) for π0.5 and 61/180 (33.9%) for GR00T-N1.6. The paper also notes that, aggregated by task, Ψ0 leads on BendPick and π0.5 leads on Mobile P&P and XMoveBendPick. This site reads one more point from the table: on XMoveBendPick the full model’s 9/30 is the lowest of all seven rows, below the three baselines and below all three ablation variants.
Each ablation removes one part. Without mixed-modality pre-training, the total falls to 88/180 (48.9%), 22.2 percentage points lower; without robot mid-training it falls to 107/180 (59.4%), 11.7 points lower; both nearly collapse on BendPick (22/30 down to 0/30 and 2/30). “Language-only context” replaces the prior’s predictive context with language features from the task instruction, while the expert still sees the current image and proprioception; it totals 105/180 (58.3%), 12.8 points lower. The full model does better on four of the six tasks, with the largest gap on XMovePick (28/30 against 8/30), but on XMoveBendPick language-only context does better (21/30 against 9/30). The authors take this as evidence that the prior adds useful predictive information beyond task language and current observations, acknowledge that the benefit varies by task, and discuss possible interference among visually similar tasks in Appendix 8.3, which this site did not read.
Comparing methods on these numbers requires attention to trial counts. This site computed Wilson intervals assuming independent trials: the 95% interval for 128/180 is about 64–77%, and for 116/180 about 57–71%, so they overlap. Overlap is not a significance test, and trials within the same task and difficulty level are not fully independent, but it shows that the 6.7-point gap in totals still carries considerable uncertainty. The text read does not say how many random seeds each method was trained with.
Real robot: 15 trials on a Unitree G1
The real-robot platform is a Unitree G1 with two 7-DoF arms, Linker Hand O6 dexterous hands and an Intel RealSense D435i camera mounted on the head. Demonstrations are collected by VR whole-body teleoperation: the operator wears a PICO headset and two ankle trackers and holds two controllers; XRoboToolkit estimates the operator’s SMPL pose in real time, and SONIC turns it into 29-DoF whole-body control commands executed on the G1 at 50 Hz. Proprioception, egocentric RGB images and motion commands are recorded in sync. The text read does not give the number of real-robot demonstrations.
All three tasks require the robot to walk to the target before manipulating it: Walk to Fish (walk to a water tank and scoop up a toy fish with a handheld net), Walk to Mirror (walk to a table, infer where a hidden doll is from its reflection in a mirror, and grasp it) and Walk to Doll (walk to a table and grasp a visible doll directly). Each task is run 5 times. Physical markers fix the robot’s starting position and the table position, the target object is placed at random in the left, center or right of the workspace, each trial has a 30-second limit, and a human evaluator judges success.
| Method | Walk to Fish | Walk to Mirror | Walk to Doll | Total |
|---|---|---|---|---|
| GR00T-N1.6 | 3/5 | 5/5 | 5/5 | 13/15 |
| Ψ0 | 0/5 | 1/5 | 2/5 | 3/15 |
| π0.5 | 2/5 | 2/5 | 3/5 | 7/15 |
| Being-M0.7 | 3/5 | 5/5 | 5/5 | 13/15 |
Being-M0.7 and GR00T-N1.6 match not only in total but task by task. By the same Wilson calculation, the 95% interval for 13/15 (86.7%) is about 62–96%. With only 5 trials per task, these trials cannot separate the two, and the authors themselves describe the result as matching.
Putting the two tables side by side shows one more thing: GR00T-N1.6 has the lowest SIMPLE total (61/180) yet ties for the top on the real robot, while Ψ0 is second on SIMPLE (116/180) and last on the real robot (3/15). Simulation and real-robot evaluation differ in tasks, training data and trial counts, and the part of the paper read here does not analyze the discrepancy. It is a reminder that, in this paper’s data, the baselines’ ranking on SIMPLE does not match their ranking on the three real-robot tasks.
How this relates to other pages on this site
The course lesson Physical AI Lab 1.5: Where Does Data Come From? explains that human video is plentiful, but a human hand is not a robot hand: the video carries no precise joint angles and no force. Being-M0.7 is one concrete response: instead of translating joint angles, it writes both humans and robots as a 22-dimensional vector of head and four end-effector positions, and video-only data contributes no motion at all, only visual dynamics. It still needs teleoperated robot trajectories for mid-training and post-training. The 22.2-point drop without pre-training is the authors’ evidence, in this simulation setting, that human-data pre-training helps; it does not show that human data can replace robot demonstrations.
The previous day’s Frontier Radar | HWAM is also a world action model for humanoids, but the two treat the body in different ways: at deployment HWAM keeps the predicted post-execution state in joint denoising and sends only the actions to the controller, whereas Being-M0.7 switches motion prediction off at deployment and passes only visual hidden states, through a gate, to a separately trained expert. The two papers differ in platform (LimX OLI versus Unitree G1), tasks and metrics, so their numbers cannot be compared.
The rule in Physical AI Lab 0.3 is that a rate belongs only to the system that reports it. Being-M0.7’s 5 Hz, about 1.67 Hz and 50 Hz all come from one configuration of one system: they are the prediction frame rate, the action-query rate and the command-execution rate, and they are not interchangeable. The reading taught in Physical AI Lab 1.8: Evaluation applies here too: first check how many trials each cell has, then how the total is built up from the individual tasks.
Limitations
- This is an arXiv preprint; all numbers are author-reported [R], and the authors are credited as the BeingBeyond Team. This site has not reproduced any result. The paper lists a project page, which this site did not check; the text read does not say whether code or weights are released.
- The pre-training corpus includes commercially acquired datasets and subsets of the team’s earlier corpora; the text read gives neither the names nor the size of those parts, nor the hours per stream. The 10,000 hours are the raw amount before filtering and segmentation.
- The real-robot evaluation has 15 trials, 5 per task, judged by a human, and the paper gives no confidence intervals; the Wilson intervals in this article were computed by this site assuming independent trials. Each simulation cell has 10 trials, and the text read does not state the number of training seeds.
- Limitations stated by the authors: the evaluation covers a limited range of task horizons, and performance on longer-horizon loco-manipulation remains to be established; the SONIC whole-body controller in the current setup has limited end-effector tracking accuracy, which makes robot demonstration collection harder.
- The paper text read for this article ends in the temporal-configuration passage of Appendix 7.4. It does not include inference timings, the task-interference discussion in Appendix 8.3, the real-robot task details in Appendix 9.1, or any of the figures.
- A small number of successful trials on three real-robot tasks is not a hardware safety validation.
Links
- Being-M0.7: A Latent World-Action Model for Humanoid Robots (arXiv 2610.11283)
- Full paper in HTML
- Frontier Radar | HWAM: a humanoid’s reference action is not the motion it performs, so generate the post-execution body state alongside it
- Physical AI Lab 1.5: Where Does Data Come From? Working Out Demonstration Time from SO-101 Metadata, Then Separating the Units and Action Spaces of Three Sources
- Physical AI Lab 0.3: See, Think, Act — Putting Each Rate Back in Its Own System
- Physical AI Lab 1.8: Evaluation — Splitting a “Net Gain of 2” into a Paired Table, Then Checking the Failure Labels and Random Seeds
- Physical AI Lab: a physical AI course where every number can be checked (course outline)
