Frontier Radar | CausGround: inducing each action’s preconditions and effects from interaction trajectories, so a frozen multimodal planner adds the “open the door before putting it in” step without parameter updates

Layered diagram on a light background, with the title “CausGround: layers of causal knowledge” at the top and a source line at the bottom. In between are four full-width rounded rows stacked from top to bottom, numbered 1 to 4 at the left; the numbers show display order only, and rows 1 and 3 have a pale green fill. The rows read, from top to bottom: 1, task knowledge K_task for the frozen planner; 2, generic K_gen and instance-specific K_spec; 3, per action, preconditions C and effects E; 4, world state, made of object attributes plus inter-object relations. — CausGround’s knowledge layers: at the bottom is the factorized world state (object attributes and inter-object relations) that a frozen multimodal model extracts from RGB observations; above it are the precondition sets C and effect sets E induced for each grounded action; these are then split into generic and instance-specific knowledge according to whether they hold across instances; at planning time, causal dependencies from backward reasoning on the goal state and semantically retrieved knowledge are merged into task knowledge K_task, built once at the start of each task and given to a planner with frozen parameters. Drawn from Section 3 and Figure 2 of arXiv 2610.29702 (authors’ description); the order of the layers shows how each is built on the next, not a ranking, and the diagram contains no measurements.

When a multimodal large language model plans at a high level, the paper uses one example to illustrate a type of error: the model knows what to do but not when an action can be done — it knows the item belongs in the fridge, yet issues “put inside” while the fridge door is still closed. The authors note that such precondition violations can accumulate over long-horizon execution. Authors from East China Normal University, the Beijing Academy of Artificial Intelligence (BAAI), Northwestern Polytechnical University, the School of Artificial Intelligence of the University of Chinese Academy of Sciences and the Institute of Automation of the Chinese Academy of Sciences address this in the preprint CausGround (arXiv 2610.29702, 2026-10-21) without fine-tuning. They induce each action’s preconditions and effects from interaction trajectories, store them as reusable knowledge, and at planning time hand the task-relevant part, matched against the current observation, to a planner whose parameters stay frozen. Across three Qwen3.5 backbones, the authors report that average task success on an adapted VirtualHome rises from 24.9% to 53.3%, closed-loop replanning success on RoboCasa from 43.0% to 57.0%, and average interaction-reasoning accuracy on SWITCH from 25.8% to 31.9%. This article looks closely at two things: how that knowledge is organized in layers, and how each of the three benchmarks was adapted and scored, since the scoring determines how far the numbers reach. All figures are author-reported [R]; this site has not reproduced them.

The problem: each existing way of adapting misses a piece

The paper divides existing approaches to environment adaptation into two groups. Parametric adaptation writes environment experience into the model through fine-tuning or reinforcement learning; it needs a lot of environment-specific interaction data and repeated optimization, has to be redone for each new environment, and is especially costly in real-world settings. Non-parametric adaptation adds execution feedback or episodic memory at inference time, but the experience is usually kept as whole trajectories or task-level memories, which makes it hard to tell which parts of an experience are causally tied to success and which are incidental, to reuse it across tasks, or to ground it in the current image. The authors also note that existing state representations do not always capture both fine-grained object attributes and the relations between objects, and checking whether an action’s preconditions hold needs both.

The question CausGround sets out to answer is whether interaction experience can be distilled into reusable knowledge that tells an agent both what must be true before an action and how the world changes after it. In the paper’s case study, the baseline planner identifies the right chair but keeps executing SIT without satisfying the NEAR precondition; CausGround uses its causal knowledge to find the missing prerequisite and completes the task.

The method: one factorized state links observation, knowledge and planning

Factorized world state. From an RGB observation, a frozen multimodal model extracts a world state S_t = (O_t, R_t). O_t is the set of objects in the scene, each represented by a semantic category and a set of attribute–value pairs (the appendix example gives a cup graspable=true and color=blue); R_t is the set of typed relations between objects, such as ON(cup, table), INSIDE(cup, fridge) and NEAR(robot, cup). Every attribute–value pair and every relation is a separately identifiable state factor that can serve as a precondition or an effect of some action. Observations and knowledge share the same schema, so preconditions can be matched item by item against the current state.

Preconditions and effects, induced per action. For an action family (such as GRAB or OPEN), interaction trajectories are collected under controlled variations of candidate pre-action factors; each record holds the pre-action state, the grounded action and its arguments, the post-action state and the outcome. For each grounded action a(x), the pre-action factors whose satisfaction consistently separates successes from failures are kept as the precondition set C(a(x)); comparing the states before and after successful executions, the changes that consistently follow the action are kept as the effect set E(a(x)); incidental factors and changes are discarded. When an effect of one action satisfies a precondition of another, the two are linked by a causal dependency, and multi-step interactions are composed from these dependencies.

Generic and instance-specific knowledge. Instances of the same action family are then compared: preconditions and effects shared across several instances become generic knowledge K_gen(a), and those valid only for particular objects are kept as instance-specific knowledge K_spec(a(x)). The paper’s example is that OPEN(fridge) and OPEN(microwave) may share the effect that openness becomes open, while each needs different object-specific preconditions to succeed.

Grounding the task and planning. At the start of each task, CausGround interprets the task description g as a goal state in the same schema and reasons backward from it over effects and preconditions, which yields goal-directed causal dependencies K_causal; in parallel it retrieves related generic and instance-specific knowledge from the library by semantic similarity, K_sem. The two are merged into K_task = K_causal ∪ K_sem, built once per task. At every step after that, the frozen planner picks the next action from the current image I_t, the structured state S_t, the task description g and K_task: a_t = π_θ(I_t, S_t, g, K_task). Backward reasoning over preconditions identifies which actions are executable now; forward reasoning over effects anticipates how the state will change. The paper sets this apart from world-model methods, which can predict the consequences of an action but do not always represent its conditions explicitly and usually obtain dependencies between actions only through repeated rollouts; CausGround hands conditions, effects and cross-action dependencies to the planner directly as knowledge.

How each of the three benchmarks was adapted and scored

Within each comparison, all methods use the same planner backbone, visual observations, task descriptions and high-level action space, and the planner’s parameters stay frozen throughout. The main experiments use Qwen3.5-9B, Qwen3.5-27B and Qwen3.5-35B-A3B. The baselines are a Vanilla planner with no added help, plus SelfReVision (which repeatedly critiques, revises and verifies a plan) and LookPlanGraph (which maintains a dynamically updated scene-memory graph from visual observations), both adapted to the same interface.

Benchmark	Authors’ adaptation	Scoring	What to watch when reading the numbers
VirtualHome	Turned into image-based closed-loop high-level planning: 75 tasks, split by number of reference actions into Short (3–6), Medium (7–10) and Long (11–22), 25 each; 6 scenes and 33 action types; three RGB views per step; feedback is only SUCCESS/FAILURE, with no failure reason	Success rate (SR) and final progress (P), macro-averaged over the three splits; action-attempt budgets of 20, 40 and 60, with failed attempts also counting; the reference action sequence need not be reproduced	Native renders inconsistent with the simulator state were repaired with Qwen-Image-Edit-2511 and kept only after deterministic rule checks and manual inspection; every method sees the repaired images. With 25 tasks per split, one task equals 4 percentage points
RoboCasa	Target Composite tasks, with the officially annotated skill stages treated as atomic actions and abstracted into 8 high-level APIs (NAVIGATE, PICK, PLACE, OPEN, CLOSE, PRESS, ROTATE, TILT), and simulator observations used as intermediate states	Task success rate under one-shot planning (at most 40 actions) and closed-loop replanning (at most 25 attempts)	Predicted plans are replayed against the officially recorded transitions: an action counts as SUCCESS only if it matches a recorded transition, and an action with no corresponding branch in the official trajectory is marked UNKNOWN. Starting from 100 held-out test cases, methods are compared only on the shared subset where no method hits UNKNOWN; the text read does not give the subset size
SWITCH	Original tasks and protocol unchanged; four subtasks used: Action (442 examples), Recovery (38), Verification Action (63) and Verification State (225), 768 in total	Accuracy per subtask; the overall score is the macro-average of the four	The macro-average gives the 38-example Recovery subtask the same weight as the 442-example Action subtask; in Recovery, one example is about 2.6 percentage points

The RoboCasa replay scoring has a direct consequence: a reasonable alternative plan that is not in the official recording cannot be judged, and cases where that happens are excluded altogether. The RoboCasa numbers therefore measure how well a planner picks the right action within the branches the official trajectories cover, not the result of actually executing low-level manipulation in the simulator.

Results: a few details beneath the averages

The table lists Vanilla against CausGround per backbone (from the paper’s Tables 1 and 2; the last row is the average over the three backbones, given by the authors in the text and consistent with the tables by this site’s check).

Backbone	VirtualHome average SR	RoboCasa one-shot	RoboCasa replanning	SWITCH average
Qwen3.5-9B	10.7 → 44.0	31.0 → 45.1	36.7 → 46.9	27.7 → 32.9
Qwen3.5-27B	33.3 → 62.7	27.9 → 27.9	57.8 → 75.0	27.9 → 35.2
Qwen3.5-35B-A3B	30.7 → 53.3	21.9 → 43.8	34.4 → 49.2	21.7 → 27.7
Average	24.9 → 53.3	26.9 → 38.9	43.0 → 57.0	25.8 → 31.9

The gains are most concentrated on VirtualHome. Average final progress rises from 38.4% to 71.3%. On the Long split, Qwen3.5-9B goes from 0.0% to 20.0% success, which with 25 tasks per split is from 0 tasks to 5 (this site’s conversion). The strongest baseline, LookPlanGraph, reaches an average SR of 56.0% on Qwen3.5-27B, 6.7 percentage points below CausGround’s 62.7%; on that backbone’s Short split both score 84.0%.
RoboCasa is not a clean sweep. The authors’ wording is that CausGround is strongest “in most backbones and protocol settings”. For one-shot planning on Qwen3.5-27B, CausGround and Vanilla both score 27.9%: no gain.
Extra reasoning or structure does not necessarily help. On RoboCasa the two baselines sometimes fall below Vanilla; with Qwen3.5-35B-A3B in replanning, for example, SelfReVision scores 4.9% against Vanilla’s 34.4%. The authors’ analysis: free-form self-reflection can invent prerequisites that do not exist (wrongly concluding an object is in a cabinet and inserting unnecessary OPEN actions), and a discrete scene graph can propagate recognition errors (reading a closed microwave as open and skipping a necessary action).
In the SWITCH macro-average, the gains come mainly from the two small subtasks. Verification State improves by only 0.9–2.6 percentage points, and Action on Qwen3.5-35B-A3B only moves from 27.1% to 27.4%. Converted to example counts (this site’s calculation), Qwen3.5-27B’s Verification Action rise from 36.5% to 52.4% is about 10 more correct answers out of 63, and Qwen3.5-9B’s Recovery rise from 36.8% to 44.7% is about 3 more out of 38. Counted in correct examples, however, the large Action subtask also gains substantially (this site’s conversion): Qwen3.5-9B’s rise from 24.2% to 28.1% (+3.9 points) is about 17 more correct out of 442, and Qwen3.5-27B’s +1.8 points is about 8 more, close to its roughly 10 on Verification Action. On SWITCH the comparison is only against Vanilla, and absolute accuracy is still around 30%.
What the two components contribute. In the component ablation, Vanilla scores 33.3% SR and 46.9% progress, adding only the factorized state gives 36.0% and 55.1%, adding only the causal knowledge gives 53.3% and 70.8%, and the full CausGround gives 62.7% and 79.8%. The text read does not state which environment and backbone the ablation used; its first and last rows match Qwen3.5-27B’s VirtualHome averages in Table 1 (this site’s comparison). By these numbers, most of the gain comes from the causal knowledge, and the structured state on its own mainly raises progress.
Across model families. The authors also test six models from three families, Qwen3.5, InternVL3.5 and Gemma4, ranging from E4B to 38B: average SR rises from 23.3% to 50.7% and final progress from 38.4% to 67.3%. The text read presents this as a figure, with no per-model table.

The real-robot part is a single demonstration: given a task and an initial observation, the multimodal model interprets the goal state, and CausGround reasons backward to three executable actions, Open(Microwave), Pick(Bread) and Place(Bread, Plate). The text read gives no number of real-robot trials or success rate and does not say which low-level skills execute these high-level actions; the authors point to Appendix E for details, which the text this article is based on does not include.

How this relates to other material on this site

CausGround sits in the slower part of “think” in Physical AI Lab 0.3: See, Think, Act: its output is a sequence of high-level actions, not joint commands. The previous day’s Frontier Radar | SLoSa also deals with action preconditions, but takes the opposite route: there the preconditions are written in a PDDL domain, a formal verifier checks them, and the verifier’s results serve as rewards that update the model’s parameters; CausGround does not update parameters, and its preconditions and effects are induced from interaction trajectories. Both papers use an adapted VirtualHome, but with different tasks, observations and metrics, so their numbers cannot be compared. The same day’s Frontier Radar | GPT-GEAR looks at a general multimodal model controlling a robot arm directly through tools, a problem at a different layer. For reading differences on small samples such as 25 tasks per VirtualHome split or 38 examples in a SWITCH subtask, see the methods in Physical AI Lab 1.8: Evaluation.

Limitations

This is an arXiv preprint; all figures are author-reported [R], and this site has not reproduced them. The text read gives no confidence intervals, number of repeats or random seeds. The paper lists a project page address, causground.github.io, after its abstract, but the text read does not say whether or where code or the knowledge library is released; this site could not open that project page to check its contents while writing, so it cannot confirm whether the page provides code or the library.
For how the knowledge library is built, the text read gives only the principle: trajectories are collected under controlled variations of candidate factors. In which environments they were collected, how many there were, whether the library comes from the same environments as the evaluation, and what it cost to build are, the authors say, in Appendix C, which the text this article is based on does not include. That information determines how much “no parameter updates” actually saves.
The factorized world state is extracted from images by a frozen multimodal model. The authors point out that LookPlanGraph propagates scene-recognition errors; CausGround depends on the extracted state as well, and the text read reports no accuracy for state extraction.
VirtualHome images were repaired with an image-editing model and then checked by rules and by hand; this applies equally to all methods, but results on the adapted version cannot be compared directly with text-based or symbolic planning results on the original VirtualHome. RoboCasa is scored by replay against official recordings with UNKNOWN cases excluded, so it measures high-level action choice, not low-level manipulation.
The SWITCH overall score is a macro-average: Recovery (38 examples) and Verification Action (63) carry the same weight as Action (442). “About 10 answers”, “about 3”, “about 17”, “about 8” and “0 tasks to 5” are this site’s conversions from the sample counts.
The real-robot part is a single demonstration with no success rate; it is not a hardware safety validation and does not show that knowledge acquired in simulation transfers reliably to real environments.
The paper’s AI use statement says AI tools were used to refine the manuscript’s language and to help build the adapted VirtualHome benchmark (the image repair described above), with the results reviewed by the authors.