
A large multimodal model used as a robot agent can read the scene, split a task into subtasks and work out a recovery after a failure, but asking it at every step keeps adding tokens and waiting time. The preprint RoC (Reasoning on Call, arXiv 2611.26341, 2026-11-19, from Wuhan University, Zhongguancun Academy, Samsung and other institutions) lets a learned task-state monitor decide when a reasoning agent should review the robot’s next move, and leaves the rest of the episode to a low-level policy. All numbers below are reported by the authors [R]; this site has not reproduced them.
Physical AI Lab 3.4 on this site covered the difference between per-step and whole-chain success in long-horizon tasks, and how an orchestrator keeps a progress record of what it has finished. RoC turns “progress” into a quantity estimated online and uses it to decide when the upper layer steps in. Earlier Radar articles looked at GPT-6 Astra controlling a robot arm directly (GPT-GEAR) and at handing some household tasks entirely to an agent (One Policy for 100 Household Tasks). RoC addresses at which moments within a single episode the agent should appear.
Three components and one loop
RoC consists of a low-level policy, an online task-state monitor and a reasoning agent. In the experiments the policy is initialized from pretrained FastWAM and predicts a 32-step action chunk with 50 diffusion denoising steps. Its inputs are the current subtask instruction, the robot state and three frames: the initial frame, a recent frame from the active subtask and the current frame. The monitor is initialized from Robometer, a general-purpose robotic reward model. The agent is GPT-6.1 Sol (xhigh).
At the start of an episode the monitor proposes the first subtask from the task instruction and the initial observation, and the agent confirms it before the policy starts. After that, the policy proposes an action chunk at each step and RoC first checks whether a review is triggered. If not, it executes the first part of the chunk (the paper calls it the execution prefix) and takes a new observation. If a review is triggered, it holds back the affected actions and hands them to the agent. Between agent calls the monitor keeps running and the policy keeps executing.
What the monitor outputs
- Progress
p_t: how far the active subtask has got. The progress head gives a distribution over ten discrete values in [0, 1], and its expectation is the scalar estimate. - Transition-request probability
s_t: predicted directly by a second head on the same hidden state. - A candidate successor subtask or task termination: produced by an autoregressive language-generation branch, which runs only when
s_texceeds the threshold.
The monitor’s inputs are the task instruction, the active subtask, an interaction memory kept in execution order and selected past frames. Frames are sampled by stage: sparsely and uniformly from the start of the episode to the start of the current subtask, as a scene reference, and densely within the current subtask to capture local changes. Each memory record holds the subtask instruction, the executed actions, an outcome description and the timestamps of its supporting observations, and only records verified by the agent are written. Subtask transitions are sparse in time, so during training the authors halve the sampling interval around annotated subtask boundaries, giving the monitor more states just before and after a transition. The monitor is trained for one epoch on all nine RMBench tasks. The authors measured the monitor at 7.10 Hz without subtask generation and 0.53 Hz when generating a subtask, with an overall throughput of 2.77 Hz.
Three triggers, five corrections
Three kinds of event trigger an agent review:
- Transition events:
s_texceeds the request threshold. The candidate successor is only a proposal and does not complete a transition by itself. - Progress events:
p_tdeclines sharply, oscillates repeatedly or stagnates within the same subtask. The implementation smooths progress with a median filter and checks a 64-step lookback window: the decline threshold is 0.2; a progress range above 0.25 with more than three reversals in the direction of meaningful change counts as oscillation; a range that stays within 0.1 counts as stagnation. - Critical contact events: before a gripper closure or a placement release appears in the action chunk, the agent checks object identity, relative geometry and interaction prerequisites.
The agent reads the same task and execution context and decides whether execution needs repair, the plan needs revision, or both. Execution repair has three modes: shortening the execution prefix (the agent can approve 1–32 steps), locally refining end-effector poses or gripper commands (1–5 steps), and taking over the subtask until its outcome is verified. Plan revision has two: rejecting the transition and keeping the unfinished subtask, and replacing an unsuitable candidate successor. If neither is needed, the agent lets the policy continue. The agent verifies subtask completion and writes the outcome description; the record goes into interaction memory and the transition is committed only once the successor is approved. Policy commands are suspended during a takeover; afterwards RoC discards stale actions and generates a new chunk from fresh observations.
RMBench: success and tokens
RMBench is a memory-dependent manipulation benchmark whose nine tasks fall into two groups: M(1) tasks require recalling one key past observation (for example an object’s initial position), and M(n) tasks require tracking several past interactions (for example earlier attempts or completed operations). The comparison with agent-based methods uses 30 independent trials per task. The two baselines, Direct Control and the hybrid mode of GPT-as-Policy, keep their original configurations with GPT-6 Astra (xhigh). Nine-task averages from Table 1 of the paper:
| Method | Average success (%) | Average tokens per episode (× 10⁴) |
|---|---|---|
| Direct Control | 71.9 | 1845.1 |
| GPT-as-Policy | 75.6 | 830.3 |
| RoC | 99.6 | 288.3 |
On these figures RoC is 24.0 percentage points above GPT-as-Policy with 65.3% fewer tokens, and 27.7 points above Direct Control with 84.4% fewer tokens. The token savings vary widely by task: on Cover Blocks RoC uses 131.1 against 2123.9 for GPT-as-Policy, while on Observe and Pick Up the two use 391.0 and 409.0, almost no saving. In one example episode in Figure 4, RoC calls the agent 46 times, GPT-as-Policy 84 times and Direct Control 209 times.
At 30 trials per task (this site’s calculation), each method has 270 trials in total, and 75.6% and 99.6% correspond to about 204 and 269 successes. Using a normal approximation, this site puts the 95% half-width of the difference at about 5.2 percentage points, and the 24.0-point gap lies outside it. The comparison also changes the agent model, however (GPT-6 Astra against GPT-6.1 Sol), and the main text gives no run with the same model on both sides, so this site cannot attribute the whole gap to the triggering mechanism.
The comparison with policy-based methods uses 100 trials per task (Table 2). RoC averages 98.1% and the strongest policy baseline, MemoryWAM, 83.0%, a gap of 15.1 percentage points; FastWAM alone reaches 5.9% and π0.5 10.4%. On Blocks Ranking Try, MemoryWAM scores 100.0% and RoC 98.0%. Table 2 has no token column, and the policy baselines call no agent, so this comparison is not made at equal compute cost. The same RoC has different averages in Table 1 and Table 2 (99.6% and 98.1%), and the two tables also use different trial counts (30 and 100).
Ablation and the real robot
Table 3 of the paper adds the components one at a time: FastWAM alone reaches 5.9%; adding visual history, 14.7%; adding the monitor and executing its predicted subtask instructions, 63.7%; adding event-triggered agent intervention, which is the full RoC, 98.1%. With the monitor alone, the M(n) tasks average 51.0%, and 97.3% once the agent is added. The authors read this as the monitor supplying task-state estimates and transition proposals while the agent verifies outcomes and, when needed, repairs execution and revises the plan together. The ablation adds one layer at a time, so it reads like the control in 3.4 that swaps one layer and watches how whole-chain success changes. The main text gives neither the trial count nor intervals for this ablation.
The real-robot experiments use the 5-DoF SO-ARM101 with a fixed external RGB camera and a wrist-mounted RGB camera, 10 trials for each of three tasks: Drawer Sorting (observe which color goes with which drawer level, close the drawers, then place a randomly given block in the matching level), Cover and Reveal (memorize the random positions of three colored blocks, cover them with cups from left to right, then reveal red, blue and yellow in that order) and Tabletop Cleanup (put a power adapter, two tape rolls and a handheld fan into an orange box). Success requires completing all subtasks in the specified order and reaching the target final state. RoC scores 70%, 80% and 70% on the three tasks, 73.3% overall; policy plus history plus monitor scores 56.7%, and the policy alone 30.0%. Agent intervention adds 10, 20 and 20 percentage points on the three tasks.
Each setting has 30 trials in total, so 73.3% and 56.7% are 22 and 17 successes (this site’s conversion). This site’s Wilson estimates of the 95% intervals are about 56%–86% and 39%–73%. They overlap, and these 30 trials alone are not enough to confirm a difference of 5 successes; the interval method is worked through in Physical AI Lab 2.8. The authors attribute most remaining failures to difficult grasps, where the agent still failed to turn visual evidence into an effective grasp correction, and they name agent capability as the remaining bottleneck. Real-robot efficiency comparisons with agent-based methods are in the appendix, which this site has not read.
Limitations
- This is an arXiv preprint; every number is reported by the authors [R] and this site has not reproduced them. The main text this site read reports no training-seed counts or confidence intervals, and does not say whether code or weights are released.
- In the agent-based comparison the baselines use GPT-6 Astra (xhigh) and RoC uses GPT-6.1 Sol (xhigh). The main text does not separate how much of the success and token gaps comes from the model itself.
- The monitor is trained on the same nine RMBench tasks and needs subtask-boundary annotations; the main text reports no results on tasks it was not trained on. The main text this site read also does not say what data the monitor and policy were trained on for the real-robot experiments.
- The paper’s motivation includes the latency of waiting for the agent, but the main tables report only tokens, not time per episode; the monitor runs at only 0.53 Hz when generating a subtask.
- During a subtask takeover the agent controls the robot directly, and the main text this site read does not describe safety constraints for takeover. Neither the simulation results nor the 30 real-robot trials are a hardware safety validation.
- This article is based on the arXiv HTML main text; the appendix was not read.
Links
- Reasoning on Call: Efficient Agentic Robot Control for Long-Horizon Robot Manipulation (arXiv 2611.26341)
- Full paper in HTML
- Physical AI Lab 3.4: Long-Horizon Tasks — Working Out Per-Step and Whole-Chain Success by Hand, Seeing Which Quantity Progress Memory Changes, Then Reviewing a Routing Ablation That Swaps Only the Upper Layer
- Physical AI Lab 2.8: Reproducible Evaluation — Working Out Wald and Wilson Intervals by Hand, Sizing a Run from 1/√n, Then Bounding the Failure Rate after Zero Failures
- Frontier Radar | GPT-GEAR: GPT-6 Astra controls a robot arm directly through five tools, with no trained action policy — how depth, touch and demonstrations are given matters more than whether they are given
- Frontier Radar | One Policy for 100 Household Tasks: one π0.5 checkpoint for 100 household tasks, 14 of them handed to a GPT-6 Astra agent that practiced first; authors report Q 0.639 vs 0.108
- Physical AI Lab: a physical AI course where every number can be checked (course outline)



