
The BEHAVIOR Challenge evaluates how well robots carry out long-horizon household activities in the OmniGibson simulator, and the 2026 edition raises the task count from 50 to 100. In arXiv 2611.14432 (2026-11-10), Ilia Larchenko and six co-authors describe their entry: one VLA checkpoint based on π0.5 covers all 100 tasks, and 14 tasks are handed to a GPT-6 Astra coding agent that has first practiced on training instances. The authors say they placed 9th and were the only independent team in the top 10. The report sets a trained learned policy and a practicing LLM agent side by side on the same tasks and gives the time and cost per episode. It also spells out how to extend a trained multi-task VLA to new tasks, how to choose a training hyperparameter from one measurement and how to build the evaluation infrastructure. All numbers below are reported by the authors [R]. The organizers did not re-evaluate this submission on the private test set, and this site has not run the authors’ code.
The challenge and how it is scored
The 100 activities of 2026 range from turning on a radio to cooking a meal. Each has 200 teleoperated demonstrations, 20,000 episodes and 211M frames in total, about 1,950 hours, in the LeRobot format. The robot is the Galaxea R1Pro: a holonomic base, a 4-joint torso and two 7-DOF arms with parallel grippers, observed through a head camera, two wrist cameras and proprioception. An episode that satisfies every goal predicate is a full success and scores 1; otherwise its q-score (also written Q) is the fraction of satisfied predicates. Scores are averaged over instances and then over tasks. The time limit of an episode is 1.5 times the mean duration of the task’s human demonstrations, against 2 times in 2025. The 50 tasks of 2025 are kept with the same demonstrations; the other 50 are new.
The organizers re-evaluated only the top-ranked submissions on the private test set, and this one was not among them. The scores below therefore come from the authors’ own run of the official evaluator on public instances 301–310, 10 instances per task.
Growing one checkpoint from 50 to 100 tasks
In 2025 the authors served 50 tasks with four checkpoints and concluded that the model was clearly undertrained: the loss was still falling at the deadline. In 2026 a single checkpoint is warm-started from the 2025 checkpoint that carried the largest share of the score. Every parameter added in 2026 is zero-, identity- or small-normal-initialized, so the converted model starts close to the 2025 model. There are three main changes:
- Semantic stages. In 2025 each episode was cut into 5–15 equal time slices that served as “stages”. Checking them against the demonstrations’ skill annotations, the authors found that on average only 50% of a time stage’s frames belong to its most common annotated step, and 57% of the 1,093 time stages have no step covering more than half of their frames. Stages built from the annotations give 413 stages across the 100 tasks (2–9 per task).
- Warm-starting new tasks. Each new task’s embedding is a weighted average of the embeddings of 1–5 similar old tasks (weights 1, 0.7, 0.5, 0.35 and 0.25 by similarity rank), rescaled to the average norm of its sources; freeze_fruit, for example, starts from freeze_pies and two other old tasks. Stage embeddings are initialized the same way.
- A 21-dimensional action space. The 2025 model predicted 32 dimensions, 23 of them real degrees of freedom. JoyLo teleoperation has only two torso controls (translation along a fixed path and a tilt offset on the third joint), and every torso command in the data has that form, so the model now predicts translate and tilt and maps them back to the four joints. The authors also keep only the 256 rows of Gemma’s 257,152-row vocabulary that are actually used, removing 526M parameters that received no gradient and about 8 GB from each checkpoint.
Training ran for 200k steps: 130k steps on 1 B300 at batch 480, then 70k steps on 8 H200s at batch 2048, with the learning rate scaled by the square root of the batch. The model saw about 200M samples, roughly one pass over the 211M frames of the demonstrations. On the 50 tasks shared with 2025, Q rose from 0.260 to 0.371; across all 100 tasks it is 0.392. The authors state that the gain cannot be attributed to longer training alone: the policy also saw demonstrations from 50 additional tasks, and the model received improvements and bug fixes. The 2026 time limit is 25% shorter, and the two years used different simulator versions (the authors say they observed no meaningful difference between them).
Choosing the number of flow-matching samples from one gradient-noise measurement
In π0-style training the VLM prefix is the expensive part and the action expert is cheap, so the prefix is computed once per observation and the action expert is run for K (flow time, noise) samples. With a probe that resamples one noise source at a time at fixed parameters, the authors measured that in the 2025 setting (K = 15, independent draws) 21% of the gradient variance came from the flow draws, 38% from augmentation and 42% from which observations are in the batch. Of the three, only the flow term shrinks as K grows.
The authors use two variance-reduction methods that leave the expected loss unchanged: flow times are stratified over the quantile bins of Beta(1.5, 1), and adjacent bins share one noise draw with opposite signs. They report that stratification halves the flow term at the same K and that the antithetic noise removes another 17%.
To pick K itself: the step time is T = B(c_p + c_k·K), noise times time is J(K) ∝ (c_p + c_k·K)(V_oa + V_f/K), and minimizing over K gives K* = √((c_p/c_k)·(V_f/V_oa)). On one B300 the authors measured c_p = 21.4 ms and c_k = 0.30 ms, and V_f/V_oa = 1.43 with the new sampler, which gives K* ≈ 10, with J within 1.5% of the optimum for K between 8 and 14. Substituting the values, this site gets √(71.3 × 1.43) ≈ 10.1, matching the authors. Compared with the 2025 setting, the new sampler at K = 10 lowers variance times time by about 13%, which the authors describe as equivalent to a 13% larger batch for free. The inputs are only two 50-step timing runs and one variance probe. As this site reads it, the estimate does not depend on BEHAVIOR itself, and any flow-matching training that shares a prefix across several action-head samples could be measured the same way; the authors do not report its separate effect on final success.
Evaluation infrastructure: traceable records, simulation faster than real time in aggregate
Evaluating 100 long tasks is expensive: an episode can take tens of minutes of wall time, and a simulator takes 6–9 minutes to boot. Over the competition the authors ran around 9,000 evaluation episodes, about 100M environment steps or nearly 1,000 simulated hours. Each run carries a manifest with the code revision and diff, simulator and package versions, all evaluator and serving arguments, and the checksum of the served checkpoint. The state of the robot and of every scene object is saved at the start, every 30 steps and at termination, so it can be loaded into a fresh simulator to replay from near a failure or to run a recovery experiment.
Most of the speed comes from keeping the CPU busy with useful work. Each simulator boots once, and before every instance it restores the scene saved at boot, which takes about 10 seconds against 7.6 minutes for a reboot. A finished environment slot is refilled immediately; according to the authors, the official vectorized evaluator wastes 45–48% of its steps waiting for the slowest episode. One policy server batches inference for all environments. With six simulators on one RTX PRO 6000 the machine simulates 33 environment steps per second against 30 steps per second of simulated time, or 1.11 times real time. That is an aggregate over the six simulators: each one alone runs at about 5.5 steps per second, so a single episode is still slower than real time, and including boots and the uneven finish of a batch, the whole fleet runs at about 0.86 times real time per GPU. A full evaluation of 100 tasks with 10 instances each took about 20 hours on eight machines and cost about $300.
Handing 14 tasks to a GPT-6 Astra agent that practiced first
The authors first tried another design: the learned policy in control, a GPT-6.1 Sol monitor watching the cameras, and a bounded Astra recovery when the policy got stuck (at most three interventions per episode, each of at most 900 actions). They found no gain they could attribute to the interventions and did not submit it.
The final design routes by task. Each task first gets a practice session: GPT-6 Astra, running in the Pi coding agent with one simulator holding a training instance (never an evaluation instance), lists the hard parts, then saves the scene before each part, tries it and reads which goal conditions hold. When a part fails several times it steps through two or three human demonstrations and changes what differs. At the end it runs the whole task from a fresh reset, without restores and within the official step limit, and writes a procedure for a later copy of itself. Reset, save, restore and goal feedback are privileged tools that exist only in practice. No weights are trained; what Astra learns stays in its conversation (29–395 MB per task), its notes and its recipes.
In evaluation every episode starts from a fresh copy of the task’s frozen session and receives only the official observation. Astra does not output low-level actions. It writes short programs that call three tools, observe, xyz and act, and sends one goal at a time; a controller takes one inverse-kinematics step per simulator step at 30 Hz and returns when the goal is reached, the motion stalls or a step limit passes. The authors say three rounds of tool changes mattered more than the wording of the prompts: with one goal per call and a loop closed at every step, free motions reached their target within 0.83 mm; with positions from depth, the worst error against the simulator was 1.06 μm (3,598 samples); and the base speed is limited to 0.20 m/s to stop the robot falling on long base motions.
To test whether practice transfers, the authors moved sessions to training instances they had not seen, each with a different scene layout: over the final two days, 8 of 17 first attempts that used only the notes succeeded. They practiced 27 tasks and routed 14 to the agent, choosing tasks where the learned policy is weak (mean Q 0.108) and where either the agent did better in their own tests or the policy scores about zero. If the agent fails at runtime (three consecutive answers over the query time limit, an API error, or the cost or wall-time limit), the learned policy plays the rest of that episode.
The authors report a mean Q of 0.639 for Astra across these 14 tasks, against 0.108 for the learned policy alone; on installing_smoke_detectors Astra scored 1.0 where the policy scores 0.10. The report also gives the cost: an agent episode took a median of 0.63 hours of wall time, about 17 times longer than real time, and tens of dollars in API fees. Because time ran out, the public results contain only 106 of the 140 agent episodes; the public submission scored Q 0.446 with 27.3% full-task success. If all of the 0.639 were counted, the task-averaged mean over 100 tasks would be about 0.466 (this site’s estimate, assuming 0.392 and 0.108 come from the same policy episodes). That 0.446 is lower is consistent with the statement that only part of the Astra episodes were included.
GPT-GEAR, covered earlier on this site, is another preprint in which GPT-6 Astra controls a robot arm directly through tools alone; this report adds a task-by-task comparison with a trained VLA on a long-horizon household benchmark, with time and cost per episode. For risk review when an agent controls a robot directly, see LEASH. Physical AI Lab 3.4 covers multi-policy routing and a routing ablation that swaps only the upper layer. Here the routing is a fixed per-task table set before evaluation, and the tasks were picked from the authors’ own trials, so 0.639 against 0.108 describes only these 14 selected tasks and cannot be extended to a claim that the agent beats the policy on every task.
Compared with 2025: where the failures went down
On 10 tasks reviewed by hand in both years, full success rose from 20% for the 2025 submission to 36% for the 2026 learned policy. By the authors’ manual failure labels (an episode with several reasons is split equally among them), Order errors fell from 13.0 to 6.7 attributed episodes, Robot fell from 7.2 to 4.5 and Robot stuck from 2.8 to 0, while Dexterity failures did not fall (29.8 against 32.2). The authors’ review of the 2025 failures traced a substantial share of Order errors to the idle tails of demonstrations after task success: 2025 training included these frames, whereas the official evaluation ends at success, and the 2026 data loader cuts demonstrations at that point. The authors add that better aligned semantic stages may also have reduced sequencing errors. After the switch to the 21-dimensional action space, which keeps the policy from producing torso configurations absent from the demonstrations, the authors observed that the robot was less likely to fall during deep squats, and they say the lower Robot fell count is consistent with that observation; the same change may also explain fewer Robot stuck failures. The authors explain that with more episodes reaching the difficult manipulation steps, the dexterity limits become more visible. The report does not isolate these attributions by ablation. For how to read failure labels like these, see Physical AI Lab 1.8.
Some tasks got worse: cook_bacon fell from 0.757 to 0. The authors found that in the 2026 public evaluation 7/10 instances placed the bacon tray on the lower fridge shelf, about 41 cm below its height in all 199 training demonstrations, and consider this a likely contributor, though the other three instances also scored zero. Two kinds of recovery data helped only a little on single tasks: scripted recoveries raised success on turning_on_radio from about 50% to 60–75%, and human corrections raised Q on one task from 0.267 to 0.367, with a one-sided paired permutation p = 0.131, which is not significant. The authors were unable to scale either to 100 tasks.
Limitations
- This is a competition report on arXiv, and every number is reported by the authors [R]. The organizers did not re-evaluate this submission on the private test set; the scores come from the authors’ own evaluation of public instances 301–310. “9th place, the only independent team in the top 10” is the authors’ statement, and this site has not checked the leaderboard.
- The 14 agent tasks were picked by the authors from their own trials as tasks where the learned policy is weak, and some were routed without a complete successful practice run. The report gives no interval for 0.639 against 0.108.
- The 0.260→0.371 gain on the 50 shared tasks mixes longer training, data from more tasks, model changes and bug fixes, under a different time limit and simulator version, so it cannot be attributed to a single change. The registers, scratch tokens and attention sinks are dedicated tokens and per-layer learnable sinks that the authors added to the model to give the misplaced attention in the pretrained SigLIP somewhere to go, mainly so that the attention maps became readable for debugging (Section 4 of the report). These changes were not ablated; the authors say that apart from easier debugging they have no evidence these improve success.
- All results come from OmniGibson simulation. An Astra episode runs about 17 times slower than real time and costs tens of dollars, and the simulation waits while it thinks. None of this is a real-robot or hardware safety validation.
- The report gives addresses for the code, checkpoints and project page; this site has not opened or run them.
Links
- One Policy for 100 Household Tasks: Our Solution for the 2026 BEHAVIOR Challenge (arXiv 2611.14432)
- Full paper (HTML)
- Physical AI Lab 3.4: Long-Horizon Tasks — Working Out Per-Step and Whole-Chain Success by Hand, Seeing Which Quantity Progress Memory Changes, Then Reviewing a Routing Ablation That Swaps Only the Upper Layer
- Physical AI Lab 1.8: Evaluation — Splitting a “Net Gain of 2” into a Paired Table, Then Checking the Failure Labels and Random Seeds
- Frontier Radar | GPT-GEAR: GPT-6 Astra controls a robot arm directly through five tools, with no trained action policy — how depth, touch and demonstrations are given matters more than whether they are given
- Frontier Radar | LEASH: when foundation models control a robot arm directly, review before each round is planned, before each action and before each release or pour, tested by the authors on RoboRisk’s five risk categories
- Physical AI Lab: a physical AI course where every number can be checked



