Lesson video
Narration is AI-generated. Video chapters, transcript and sources
Transcript
Transcript
UNIT-01 asks
What is a robot actually busy with, inside one second? Last time, three curves rose together. This time, we slow that one second down. Watch the robot pick up a cup, steer clear of the water bottle beside it, and count the steps.
See
Step one: observe. The cup to pick up, and the bottle not to touch, both come in through the camera. One paper's public dataset saves fifteen frames of video every second. The robot also has to see itself: where its hand is, and how wide it's open. What the camera captures isn't just a flat picture either. How far the cup is from the hand, and whether the hand can reach it, all have to be worked out from the image. So it doesn't just look outward. It keeps an eye on its own hand, too!
Think
Step two: think. Thinking actually comes in two halves: one thinks things through slowly, the other produces actions fast. An open robot model uses exactly this design. The slow one thinks it through first: grab the cup, go around the bottle, then put it aside. One paper's robots work like this: after every twenty-five steps, about half a second, it thinks again. So thinking isn't done all at once. It takes a few steps, then thinks again!
Act
One robot company's interface exchanges a thousand control commands every second. The planned path is handed to this layer, so the motors can fine-tune at any moment. Why so fast? One beat too slow, and the hand may drift and knock over the water bottle beside it. Think of it this way: the brain thinks slowly, the cerebellum moves fast. Thinking comes every so often; acting, much faster. But it's only a comparison. See, think, act, joined into a loop. Within one second, this loop goes around many times. When the loop updates fast, it can spot drift sooner and adjust its movements. But speed doesn't guarantee precision. To be both fast and accurate, robots still have a long way to go. Let's recap, three points. One, see: it looks outward, and at itself. Two, think: two systems, slow thinking and fast doing. Three, act: low-level control, far faster. So inside one second, the robot sees, then thinks, then acts, round and round again! Experimenting only on real robots is slow and costly. So could we try things in a virtual world first? See you next time!
Evidence and sources
Practice and self-check
The quiz is public and explains each answer as you go; answers to the guide's exercises are for signed-in members.
The episode 0.3 video slows one second of a robot’s life down into three steps, see, think and act, and cites four pieces of evidence along the way: 15 frames a second, a slow and a fast system, a rethink after every 25 steps (about half a second), and 1,000 control commands a second. This page is the episode’s reading and applied practice. Rather than repeat the transcript, it puts each piece of evidence back into its own system, converts rates into periods, and then uses a fictional robot to practise scheduling clocks, reading logs and auditing evidence. One rule runs through all of it: a rate belongs only to the dataset, model, robot or interface that measured or specified it.
For the video, transcript and sources, see Video & transcript | Physical AI Lab 0.3: See, think, act; for the season plan, see the course outline. This lesson keeps the example from the first two episodes: pick up a cup without knocking over the water bottle beside it.
The public five-question quiz includes full explanations. Sign in for three exercises: build a layered clock table for a fictional robot, use fictional logs to decide which layer caused three collisions, and write evidence cards for three overreaching paraphrases, with worked solutions, common mistakes and a 20-point rubric.
1. Four pieces of evidence, four different systems
First, put each of the video’s four pieces of evidence where it belongs. Watch the “Belongs to” column: no two rows describe the same robot.
| Evidence | Belongs to | What the source says | Level |
|---|---|---|---|
| 15 Hz | The DROID dataset (Franka Panda arm, Polymetis controller) | All data is recorded at 15 Hz; actions are recorded in both joint space and end-effector space at a control frequency of 15 Hz | R (author-reported) |
| 25 actions / 0.5 s | Robots running at 50 Hz in the π0 paper | Inference is re-run after executing 25 actions, i.e. every 0.5 seconds | R (author-reported) |
| Slow and fast systems | The GR00T N1.7 model | System 2 handles high-level reasoning and task decomposition; System 1 generates motor commands. The cited text gives no rate for either | O (vendor, first-hand) |
| 1 kHz | Franka Control Interface (FCI), on the Franka TMR product page | Real-time low-level control values are sent at 1 kHz, and measurements are read at 1 kHz | O (vendor-specified) |
One brand, three rates. “Franka” appears three times in these sources. DROID uses a Franka Panda arm and records actions at 15 Hz. The π0 paper states that on the 20 Hz UR5e and Franka robots it runs inference every 0.8 seconds (after executing 16 actions). Franka’s own product page says FCI exchanges low-level control values at 1 kHz. These three statements are not about the same thing: one is a dataset’s recording rate, one is a policy’s action rate on particular robots, and one is the specified rate of a low-level interface. The DROID excerpt also does not describe how its controller connects to the arm’s low-level loop, so you cannot conclude from it that “the arm in DROID runs at 1 kHz”.
Sign in to keep reading: the rest of this lesson, its self-check questions and the answers.
Limitations
- The DROID (arXiv, 2024-03-19) and π0 (arXiv, 2024-10-31) numbers are author-reported, level R; the GR00T N1.7 structure and FCI’s 1 kHz come from vendor pages, level O. This site has not independently reproduced any of them.
- The TMR presented on the Franka page is labelled a prototype; 1 kHz is FCI’s specified capability, not a measurement on a particular robot.
- π0’s inference times apply only to an RTX 4090, 3 camera images and 10 flow-matching steps; different hardware or settings will change them.
- Robot R, all its parameters, the 0.5-second requirement, the “inference does not overlap execution” setup and the logs in task 2 are fictional exercise material; the resulting 1.04 s, 0.44 s and 1.0 s are not the performance of any system, and timing worked out on paper is not a hardware safety validation.
- “See, think, act” and “brain and cerebellum” are simplifications; in real systems, perception, planning and control are often intertwined.
Links
- Video & transcript | Physical AI Lab 0.3: See, think, act
- Physical AI Lab: a physical AI course where every number can be checked (course outline)
- Physical AI Lab 0.2: Why Now? The Three Curves of Data, Models, and Compute (GR00T N1.7’s size and training data)
- DROID: A Large-Scale In-The-Wild Robot Manipulation Dataset (arXiv 2403.12945, Appendix B)
- π0: A Vision-Language-Action Flow Model for General Robot Control (arXiv 2410.24164)
- NVIDIA Isaac GR00T N1.7: Open Reasoning VLA Model for Humanoid Robots (Hugging Face blog)
- Franka Robotics: Tactile Mobile Robot and Franka Control Interface
