Physical AI Lab 3.4: Long-Horizon Tasks — Working Out Per-Step and Whole-Chain Success by Hand, Seeing Which Quantity Progress Memory Changes, Then Reviewing a Routing Ablation That Swaps Only the Upper Layer

Long-Horizon Tasks — Working Out Per-Step and Whole-Chain Success by Hand, Seeing Which Quantity Progress Memory Changes, Then Reviewing a Routing Ablation That Swaps Only the Upper Layer. English video, reading, self-checks and sources in one lesson, updated 2026-11-13.

Lesson video

Narration is AI-generated. Video chapters, transcript and sources

Transcript

Transcript

UNIT-01 asks

A robot can pick up a cup. So why does tidying a whole kitchen so often get stuck halfway? This episode is about long-horizon tasks. Why long jobs fall apart, and how layers and routing can save them.

Why long tasks fall apart

Last time we made one policy stronger and stronger. But what it's good at are actions lasting a few seconds. Any longer, and it can't keep up. Tidying a kitchen takes dozens of steps: wipe the counter, clear the plates, take out the trash. Every step can go slightly wrong. Think of a tightrope. One step isn't hard. The hard part is dozens of steps without falling, with no chance to start over. Put another way, the model is guessing at every step. Each step has a high chance of being right. But those chances multiply, so over dozens of steps the chance that every step is right keeps dropping. Worse, small errors add up. A little off on step one, more off on step two, and further and further off after that. In a closed loop it's harder still. The robot watches as it works, so its own actions change what it sees next. One wrong step, and its whole view tilts too. Researchers call this error accumulation. A single policy in charge from start to finish struggles to cope with this on its own. This kind of job has a name, a long-horizon task. It's not just about getting each step right. It's about stringing a long chain of steps together, steadily. So no single step is too hard. It's the errors snowballing, step by step!

Slow thinking, fast doing

One fix is to split the brain into two layers. One thinks slowly about the big picture. One quickly runs the hands and feet. You might ask, why not just make one policy bigger? That helps a little. But thinking and doing simply run at different paces. A paper from NVIDIA writes this as two systems. System two looks at the image, hears the instruction, and understands the scene. System one generates actions in real time. A blog from NVIDIA covers the newer version, Groot N one point seven. Its System two is exactly where task breakdown and multi-step reasoning happen. How fast is the fast layer? An AWS engineering blog measured it on an H one hundred, with the O-D-E solved in one step. Under those conditions, Pi zero packs fifty action steps into one chunk. The whole chunk is generated in under two hundred milliseconds. So it doesn't rethink every millisecond. One chunk covers about one second of motion, but takes under two hundred milliseconds to compute. That means it can replan about five times a second, and a new chunk takes over before the last one has finished. The benefit of this split is direct. The slow layer can think carefully with a large model. The fast layer keeps a steady high rate. Each does its own job. The big brain thinks slowly, the little brain moves fast, and robots split the work the same way! Though that's only an analogy.

Multi-policy routing and memory

Two layers aren't enough on their own. The slow layer also has to schedule. That is, decide which skill to call for this step. It's like an orchestrator. It holds a set of skills: grasping, opening doors, wiping tables. Each handles one part. Why not just use one policy that does it all? You could. But separate skills have an advantage. You can test, swap and debug them one at a time. How does the orchestrator choose? There are roughly two approaches. One gives each skill a feasibility score, and picks the highest. The other lets the model think about the next step directly. An early robotics study did it like this. The paper reports that a language model proposes the next step. Then the skill's value function judges whether it can be done. What's clever about this? The language model lets its imagination run. The value function pulls it back to reality. It's not enough to think of it. It has to be within reach. Another paper is called Hi Robot. The upper layer works out the next step first, then the lower layer acts. The paper tested it on three kinds of robot. Building on pi zero, pi zero point five goes a step further. The paper reports that it first predicts a semantic subtask, then turns it into concrete actions. First what, then how. The paper reports that pi zero point five trains on several kinds of data mixed together. With that, the authors say, it can roughly tidy a kitchen or a bedroom, even in a new home it has never been in. One last piece is missing, memory. The orchestrator has to remember which steps are done, and which are not. That's its progress sheet. There are two kinds of memory. One is the short present, what's in the hand right now. The other is task-level, how far the whole job has got. Without memory, a robot might take out the trash it already took out. With memory, it knows where it is. So a long task isn't one big model. It's a small team that knows how to schedule!

Hands-on: building an orchestrator

Put all this together, and you get this episode's star. An orchestrator, plus a set of policies that each handle one part. The full orchestrator would run on a public benchmark, Libero ten. An AWS blog says it has ten kinds of tabletop manipulation tasks. We didn't run it here. This episode's experiment is a two-dimensional stand-in on a CPU. Whichever benchmark you use, evaluating long tasks means testing many steps in a row. A single-step benchmark hides error accumulation. Here's how the orchestrator runs. First the upper layer looks at the scene and names the next subtask. This step only thinks. It doesn't act. Then that subtask goes to a lower-level policy to carry out. For example, a fine-tuned Groot. How to tune it is next episode. When a step is done, it goes on the progress sheet, and we look at the next one. That's the closed loop of routing plus memory. Round and round it goes. Don't forget to run a few more random seeds. Success rates on long tasks jump around a lot. Don't trust a number from a single run too much. What counts as a successful reproduction? The key is to look at two success rates separately. One per step, one for the whole chain. You can swap only the upper layer and keep everything else the same. If the whole-chain success rate changes, the bottleneck is in scheduling. That's a clean comparison. What's still unsolved is this. If the upper layer picks the wrong skill, errors still pile up. And memory right now is often very short. Long-term memory is still a hard problem. Let's recap, three points. One, long tasks are hard because errors accumulate. Two, layers let the slow part think and the fast part act. Three, the orchestrator handles routing and memory. Break a big problem into small steps, then watch each one closely, and the robot won't fall apart so easily! But how do you tune that low-level policy to your own robot? See you next time!

Evidence and sources

Two systems · GR00T N1 uses a dual-system architecture: a vision-language module (System Two) understands the scene, and a diffusion module (System One) generates actions in real time [R] · arXiv 2503.14734 · GR00T N1 (NVIDIA)

System Two · of GR00T N1.7 (Cosmos-Reason2-2B backbone) is exactly where task breakdown and multi-step reasoning happen [O] · NVIDIA · Hugging Face blog “GR00T N1.7”

< 200 ms · On an H100 with 1 ODE step, π0 generates a whole 50-step chunk in under 200 ms (AWS measurement) [O] · AWS Physical AI blog · Fine-Tuning π0 on SageMaker HyperPod EKS

≈5 Hz · π0 computes a chunk in under 200 ms, so it can replan at about 5 Hz; each chunk covers about 1 second of motion (H100) [O] · AWS Physical AI blog · Fine-Tuning π0 on SageMaker HyperPod EKS

3 kinds of robot · Hi Robot uses a layered VLM: the upper layer works out the next step, the lower layer carries it out; tested on single-arm, dual-arm and mobile dual-arm robots [R] · arXiv 2502.19417 · Hi Robot (Physical Intelligence)

SayCan · The paper reports: the language model supplies step-by-step knowledge for long tasks, and the skills' value functions judge whether a step can be done in the current environment [R] · arXiv 2204.01691 · SayCan (Robotics at Google et al.)

π0.5 · The paper reports: it predicts a semantic subtask, then actions; trained on mixed data, it tidies kitchens and bedrooms in entirely new homes (authors' own results) [R] · arXiv 2504.16054 · π0.5 (Physical Intelligence)

10 kinds · LIBERO-10 is a simulated tabletop manipulation benchmark with 10 task categories; this episode's experiment is a two-dimensional CPU stand-in, and LIBERO was not run here [O] · AWS Physical AI blog · Fine-Tuning π0 on SageMaker HyperPod EKS

Practice and self-check

The quiz is public and explains each answer as you go; answers to the guide's exercises are for signed-in members.

Diagram titled "Choosing the next step vs. acting: four sources", with four columns of text side by side. Each column has an unnumbered circular marker and a short rule above its text: SayCan: LLM proposes; skill value functions judge feasibility. Hi Robot: high-level VLM issues commands; low-level VLA outputs actions. π0.5: training examples mix semantic subtask prediction and low-level actions. GR00T N1.7: System 2 does task decomposition; System 1 makes motor commands. There are no arrows between the columns, and a source line runs along the bottom.
How each of four sources separates choosing the next step from producing the action, summarised from the original texts. The four approaches are shown side by side because the sources do not compare themselves with one another; column order is not a ranking, and the diagram contains no measured values. Sources: SayCan (arXiv 2204.01691), Hi Robot (arXiv 2502.19417) and π0.5 (arXiv 2504.16054), level R, reported by the authors; NVIDIA’s GR00T N1.7 Hugging Face blog post, level O, the vendor’s own description.

The video for episode 3.4 covers long-horizon tasks. A robot that can pick up a cup often falls apart halfway through tidying a whole kitchen, because small errors at each step accumulate, and a single policy in charge from start to finish struggles to cope with this on its own. The remedy is layering: a slow layer thinks about the big picture and a fast layer produces actions in real time. On top of that sits an orchestrator, which decides which skill to call for the current step and uses a progress sheet to remember how far the job has got. The video cites four papers and two blog posts: SayCan (arXiv 2204.01691), Hi Robot (arXiv 2502.19417), GR00T N1 (arXiv 2503.14734) and π0.5 (arXiv 2504.16054), NVIDIA’s Hugging Face blog post on GR00T N1.7, and AWS’s blog post on fine-tuning π0 on SageMaker HyperPod. This page is the reading and exercises for the episode. It does not repeat the transcript; it does three things the video states as conclusions without working through. It puts back into the original wording how each of four sources (SayCan, Hi Robot, π0.5 and the GR00T N1.7 blog post) separates “choosing the next step” from “producing the action”, plus one row of fast-layer timing from the AWS blog post. It uses a fictional kitchen task to work out by hand how per-step success turns into whole-chain success, and when the assumption that steps are independent gives the wrong answer. And it works out which quantity progress memory and retries actually change, and at what cost. One rule runs through the whole page: per-step success and whole-chain success are two different quantities and must be reported separately; a success rate or a latency belongs only to the system, task and setup that reported it.

The video, transcript and sources for this lesson are at the top of this page; the season plan is in the course outline. The previous lesson, Physical AI Lab 3.3: RL Post-Training — Putting HIL-SERL’s “2x” and RECAP’s “Doubled Throughput” Back in Their Conditions, Then Recomputing the Paired Interval over Three Seeds, ended by asking: the policy is stronger now, but in a long task, how does it remember and how does it plan? This lesson picks up that question. How an error rolls out of the demonstrations’ coverage within one trajectory was worked out by hand in Physical AI Lab 1.6: Imitation Learning — Working Out How an Error Rolls Out of the Demonstrations’ Coverage, Then Putting the ACT, Diffusion Policy and Data Scaling Numbers Back in Their Own Conditions. Which system each see-think-act rate belongs to was covered in Physical AI Lab 0.3: See, Think, Act — Putting Each Rate Back in Its Own System. The structure and parameters of GR00T N1’s dual system were taken apart in Physical AI Lab 3.1: Inside VLAs — Encoding One Cup Motion into 256-Bin Tokens, Then Labelling the Scope of Three Models’ Parameter Counts and Rates. The timing arithmetic behind the AWS blog’s “under 200 ms, re-plan at about 5 Hz”, and how long an action chunk should be, were worked out in Physical AI Lab 2.6: To the Edge — Working Out the Control-Rate Ceiling from a Latency Log, Telling Whether Quantization Saves Size or Time, Then Sizing an Action Chunk. Paired tables and intervals were worked out in Physical AI Lab 1.8: Evaluation — Splitting a “Net Gain of 2” into a Paired Table, Then Checking the Failure Labels and Random Seeds and Physical AI Lab 2.8: Reproducible Evaluation — Working Out Wald and Wilson Intervals by Hand, Sizing a Run from 1/√n, Then Bounding the Failure Rate after Zero Failures. This lesson does not repeat any of that; it covers only success rates, memory and routing once several subtasks are strung into one chain. The next episode, 3.5, is hands-on GR00T fine-tuning.

The public quiz’s 5 questions come with full explanations. Signed-in readers can do three exercises: use a fictional table-clearing record to work out the marginal per-step success rate, the independence approximation and the true chain success rate; use a fictional summary of three routers to work out chain success, call cost and failure attribution, and decide what to fix first; and review a fictional routing ablation report, compute the interval of the paired difference, and rewrite its conclusion. Each comes with a step-by-step worked solution, common mistakes and a 20-point rubric.

1. How four sources separate “choosing the next step” from “producing the action”

SayCan, Hi Robot, GR00T N1 and π0.5 are all arXiv preprints, level R in this course’s grading. The GR00T N1.7 blog post is NVIDIA’s description of its own model, and the AWS blog post is AWS’s report on one of its own fine-tuning experiments; both are level O. All structures and numbers below are reported by the authors or vendors; this site has not run any of these systems. The original passages behind the video’s number cards follow, each quoted separately. The four sources compared side by side here are SayCan, Hi Robot, π0.5 and the GR00T N1.7 blog post; the GR00T N1 paper is cited only as background for N1.7, and the AWS row only adds the fast layer’s timing and is not a fifth way of splitting the work.

SayCan (arXiv 2204.01691): the abstract describes how the language model and the skills divide the work:

We show how low-level skills can be combined with large language models so that the language model provides high-level knowledge about the procedures for performing complex and temporally-extended instructions, while value functions associated with these skills provide the grounding necessary to connect this knowledge to a particular physical environment.

The abstract also says that language models lack real-world experience, and that pretrained skills are used to constrain the model so that it proposes only natural-language actions that are both feasible and appropriate to the context; the method is evaluated on a number of real-world tasks on a mobile manipulator. On this passage, “choosing the next step” is decided by two parts together: the language model supplies knowledge of the steps, and each skill’s value function judges whether that step can be done in the current environment. That is what the video means by “it’s not enough to think of it; it has to be within reach”.

Hi Robot (arXiv 2502.19417, authors from Physical Intelligence, Stanford University and UC Berkeley) makes both the upper and the lower layer vision-language models. The high-level VLM reads the open-ended task prompt and the camera images and outputs atomic commands such as “grasp the cup”; the low level is a VLA that generates action chunks from that command, the images and the robot’s state. The paper states that the two layers run at different paces:

The two processes run at different rates: the low-level process produces action chunks at a high frequency, while the high-level process is invoked less often, either after a set time or upon receiving new language feedback.

The passages read do not give the rate of either layer. The video’s “3 kinds of robot” card corresponds to this sentence:

We evaluate our system across three robotic platforms, including single-arm, dual-arm, and dual-arm mobile robots, demonstrating its ability to handle tasks such as cleaning messy tables, making sandwiches, and grocery shopping.

The paper also says that Hi Robot outperforms several prior approaches, including API-based VLMs and flat VLA policies, in both alignment with human intent and task success; the passages read give no specific numbers. To train the upper layer, the authors use a VLM to synthesise prompts and human interjections that might plausibly have accompanied the robot data. All of these are author reports.

π0.5 (arXiv 2504.16054): the abstract first states its relation to π0:

We describe $\pi_{0.5}$, a new model based on $\pi_{0}$ that uses co-training on heterogeneous tasks to enable broad generalization.

It then says what the training examples contain, and the authors’ conclusion:

Our system uses a combination of co-training and hybrid multi-modal examples that combine image observations, language commands, object detections, semantic subtask prediction, and low-level actions. Our experiments show that this kind of knowledge transfer is essential for effective generalization, and we demonstrate for the first time that an end-to-end learning-enabled robotic system can perform long-horizon and dexterous manipulation skills, such as cleaning a kitchen or bedroom, in entirely new homes.

Note the abstract’s wording: semantic subtask prediction and low-level actions appear in the same kind of training example, and the system is learned end to end. The video describes it as “first predict the next subtask, then turn it into concrete actions”. The abstract gives no success rate and does not say how many “entirely new homes” there were or how many attempts were made in each; “for the first time” is the authors’ claim.

The Hugging Face blog post on GR00T N1.7 (NVIDIA, published April 17, 2026) describes it as a 3B-parameter open reasoning VLA with a dual-system design that separates high-level reasoning from low-level motor control:

System 2 (Vision-Language Model): A Cosmos-Reason2-2B backbone processes image tokens and language instructions to produce high-level action tokens. This is where task decomposition and multi-step reasoning happen.

System 1 is a 32-layer diffusion transformer that takes the VLM’s output and the live robot state and denoises them into motor commands in real time. Unlike Hi Robot, what passes between the two layers is “high-level action tokens”, not a readable language command. The video also cites the paper on the earlier GR00T N1, which describes System 2 as the vision-language module that interprets the environment and System 1 as the diffusion transformer module that generates actions in real time; the paper states that System 2 runs at 10 Hz on an NVIDIA L40 and System 1 outputs closed-loop actions at 120 Hz. Lesson 3.1 has already taken N1’s structure apart, so it is not repeated here; the N1 paper’s numbers belong to N1 only and cannot be carried over to N1.7.

The fast layer’s timing comes from the AWS blog. The blog fine-tunes π0 on one node with 8 H100 GPUs; π0 predicts an action chunk of the next 50 timesteps at a time, and the inference timing is also stated as on H100:

At 1 ODE step on H100, a full 50-timestep chunk takes under 200 ms, so the policy can re-plan at about 5 Hz while each chunk covers roughly one second of robot motion.

The blog swept the number of ODE steps on LIBERO: going from 10 steps to 1 cut latency from 379 ms to 197 ms, with no accuracy cost according to the authors. These timings belong only to π0, this fine-tuning run and this hardware; lesson 2.6 has already used them to work out a re-planning period, so that is not repeated here.

The table below is this lesson’s summary of the passages read, not a table from any of the sources:

SourceLevelWho chooses the next stepWhat passes between the layersPace given in the passages read
SayCan (arXiv 2204.01691)RThe language model proposes; the skills’ value functions judge feasibilityA natural-language action matching a pretrained skillNot given
Hi Robot (arXiv 2502.19417)RThe high-level VLMAn atomic language commandThe low level produces action chunks at a high frequency; the high level is invoked after a set time or on new language feedback
π0.5 (arXiv 2504.16054)RThe abstract does not state the division of work at inferenceTraining examples combine semantic subtask prediction and low-level actionsNot given
GR00T N1.7 blog (NVIDIA)OSystem 2 does task decompositionHigh-level action tokensNot given in the passage quoted
AWS π0 blogONo upper layer involvedNot applicableH100, 1 ODE step: a 50-step action chunk in under 200 ms, re-plan at about 5 Hz, each chunk covers about 1 second

The four ways of splitting the work and one row of fast-layer timing are placed side by side because the sources do not compare themselves with one another; the order of the table is not a ranking. The one thing they show together is that “choosing the next step” and “carrying that step out” can be done by different modules at different paces. None of these sources says which split works better on your task.

Limitations

  • SayCan, Hi Robot, GR00T N1 and π0.5 are all arXiv preprints, level R; the GR00T N1.7 blog post and the AWS blog post are level O, reports by NVIDIA and AWS on their own model or experiment. The structures, numbers and claims such as “for the first time” are reported by the authors or vendors; this site has not reproduced them or run any of these systems.
  • This lesson read only the abstracts of SayCan and π0.5, the abstract and part of the first four sections of Hi Robot, the abstract, introduction and part of section 2 of GR00T N1, the GR00T N1.7 blog post, and some sections of the AWS blog post. The passages read give no success rate for π0.5, no rate for either of Hi Robot’s layers, and no specific numbers for Hi Robot against other methods.
  • The AWS blog’s timings belong only to π0, its fine-tuning setup and the H100; what it reports on LIBERO is action-prediction error, evaluated on only 5 held-out episodes, not closed-loop task success.
  • The video states that this episode did not run the full orchestrator on LIBERO-10. All calculations in this lesson use fictional numbers and cite no run record.
  • The following are this lesson’s summary, arithmetic or inference: the comparison table in section 1; the products, conditional probabilities and call counts in sections 2 and 3; the comparison checklist in section 4.
  • Kitchen K, routers A–D, team L, the table-clearing task and all paired counts are fictional. Neither paper calculations nor simulation results are hardware safety validation.

Links

Watch this lesson (English video) · 3.4