End-to-end autonomous driving usually starts by imitating human driving in fleet logs, yet safety-critical events such as collisions, near-misses and recoveries are rare in those logs. In the preprint TerraTransfer++ (arXiv 2610.30319, 2026-10-21), authors from Applied Intuition, UCLA and UC Berkeley change the order: a policy first learns to drive through multi-agent self-play in a vectorized simulator; this vector-state “teacher” is then distilled into a second teacher that also outputs a planned trajectory; finally, using only paired (image, scene-state) frames, either teacher is aligned to a student that reads camera images. No stage uses a logged human trajectory as a supervision target. The authors report that the planning student reaches 59.4 EPDMS on NAVSIM navhard and 95.5 PDMS on navtest, and that the closed-loop student reaches a closed-loop HD-Score of 0.548 over 436 HUGSim scenarios. All numbers below are author-reported [R]; this site has not reproduced them.
Why not just imitate fleet logs
The paper starts from a cost argument. At NHTSA crash rates, a collision occurs only about once every 16,000 driving hours in natural driving; the estimate the paper cites is that logging a single such rare event can cost over US$1 million in fleet operation alone. Imitation can only copy the actions recorded drivers took; open-loop reinforcement learning on logged trajectories never sees how other vehicles would respond to the new actions it tries. Closed-loop RL in a photorealistic simulator does show consequences, but every step costs rendering and a pass through a large vision backbone, and training several agents together multiplies that cost.
Self-play in a vectorized simulator changes the arithmetic: one shared policy controls every vehicle, each policy improvement changes the traffic every other vehicle faces, and the interactive behavior evolves with the policy; the related work the paper cites reports millions of steps per second on commodity hardware. The catch is that a self-play policy reads vectorized scene state, not raw sensors, so deploying it would require a perception stack that recovers HD-map-style inputs from images. TerraTransfer++ targets exactly that step.
Physical AI Lab 1.6: Imitation Learning showed how errors snowball once a robot drifts into states the demonstrations do not cover; Physical AI Lab 1.7: Reinforcement Learning covered trial and error with rewards instead of demonstrations. This paper applies that division of labor to driving: behavior is learned from rewards and self-play, and images are used only to learn to “see”.
Three phases: learn to drive, learn to plan, learn to see
Phase 1, the self-play teacher. The simulator is TerraZero, the authors’ own, which compiles nuPlan HD maps into lane topology, lane boundaries and drivable-area geometry. Training uses no nuPlan logged trajectories, initial states, goals, human actions or replayed traffic: vehicles spawn procedurally along lane centerlines, and each receives a goal 20 to 100 m ahead on the reachable lane graph. At each 0.1 s step a vehicle observes its own kinematics and goal, nearby road polylines and traffic controls, and surrounding entities, which feed the ego, road and partner encoders. One parameter set controls every vehicle in every scene; the action space is 12 discrete jerk actions, optimized with PPO. The reward is a weighted sum of terms for goal reaching, collision, comfort, lane alignment and centering, velocity, traffic-rule compliance and related qualities; following GigaFlow, the weights are resampled for each agent and episode and included in that agent’s own observation. The authors say the policy trains in 96 hours on 16 A100 GPUs, covering over 2.4 billion kilometers of simulated driving. Before any vision alignment, they report it scores 94.19 on nuPlan val14 (best prior imitation planner: 92.28), 70.87 on the interactive long-tail InterPlan suite (vs. 63.66), and 0.632 on the WOSAC 2023 human-likeness benchmark with its exported rollouts (logged-oracle reference: 0.722).
Phase 2, the trajectory teacher. A closed-loop policy emits one action per step, but planning interfaces and pseudo closed-loop benchmarks such as NAVSIM score a multi-second trajectory. The authors use the frozen self-play teacher’s realized futures as targets: for each frame, the next 80 poses (10 Hz, 8 s), not a logged human trajectory. The trajectory teacher is an 11.2-million-parameter transformer over the same vector observation, with a flow-matching head that generates the trajectory in 10 Euler steps at inference; one hour of self-play on 64 GPUs yields 600 million such 8 s trajectories. For NAVSIM it samples 16 candidates from 16 noise draws and keeps the one that scores best on collision, TTC (time-to-collision) and progress against a constant-velocity forecast of the other vehicles. That selection reads ground-truth vehicle states, so the paper lists the trajectory teacher as a privileged reference.
Phase 3, vision alignment. The paper first describes this step for the closed-loop student, which is aligned to the Phase 1 teacher directly and skips Phase 2. It keeps the teacher’s frozen policy head and ego encoder and replaces only the road and partner encoders with image-based ones: a frozen DINOv3 backbone with two separate linear adapters, of which only the adapters train. There are two supervision terms. The first is the forward KL divergence between teacher and student action distributions, matching the whole distribution rather than the sampled action. The second is a structural loss: in each batch it takes the principal subspace of the teacher features (the first 13 dimensions for partner, 9 for road, the 80% cumulative-energy cutoffs) and asks the student only to reproduce the pairwise cosine similarities among the scenes in the batch, with weight λ = 0.5.
The planning student applies the same recipe to the trajectory teacher, with several differences. A frozen DINOv3 backbone with lightweight adapters predicts the trajectory teacher’s 20 per-type latent tokens (road, partner and traffic-control types) from the image. The action loss is the KL divergence to the trajectory teacher’s action head, and the structural loss is applied per type to the flattened latents, again with λ = 0.5. The ego encoder, the fusion layers and both heads (action and trajectory) stay frozen and consume the predicted latents. An added L2 loss over all 80 waypoints matches the plan the trajectory teacher selected from its 16 samples, with weight 1; at inference the student samples a single plan with no selection, so it reads no ground-truth state.
The authors explain why they avoid a cascade that perceives first and then feeds a frozen teacher: the perception module trains on surrogate detection objectives, so its errors get no gradient from driving quality; and the teacher was trained on clean simulator state and is sensitive to input noise, so imperfectly recovered inputs degrade it.
What “demonstration-free” leaves out
“Demonstration-free” here means no logged trajectory is used as a supervision target, not that no real data is used. The closed-loop student’s headline alignment data is about 1.83 million paired nuPlan frames (7,129 shards of 256 frames, about 51 hours at 10 Hz), and each frame’s target is the teacher’s action distribution on the reconstructed scene. The images are logged real camera frames; the scene state requires perception labels, which the authors describe as policy-agnostic and obtainable offline through auto-labeling. In the closed-loop student setup described in Section 2.4 of the paper, the endpoint of the logged ego trajectory still enters the student’s ego vector as the goal signal. In addition there are 300,000 HUGSim domain-adaptation frames, rendered from the evaluation scenes themselves but covering only 10% of those scenes’ total route length. This closed-loop alignment takes 10 hours on 8 A100 GPUs. The planning student, as Section 3.1 states, is aligned on NAVSIM navtrain only. The authors estimate that rendering in the training loop would be roughly 6,000 times slower than vector rollout (175,000 vs. 30 fps across all cameras).
The authors sum it up as still using what the fleet log sees, but no longer what it does. They also note that imitation remains useful for visual cues absent from bounding-box state, such as brake lights and hand signals.
Results: what each student is compared against
HUGSim closed loop. HUGSim is a photorealistic closed-loop benchmark built on 3D Gaussian splatting; its 436 scenarios are split across nuScenes (88), PandaSet (127), KITTI-360 (113) and Waymo (108), each divided into Easy, Medium, Hard and Extreme tiers. Pooled and weighted by scenario counts, the vector-state teacher reaches a closed-loop HD-Score of 0.562 and the camera student 0.548 (about 97.5% of the teacher), or 0.528 under HUGSim’s native scorer. The strongest baseline in the table, ECO, trails the student by 0.120 (closed loop) and 0.184 (native); the imitation-trained LTF, UniAD and VAD reach 0.266, 0.244 and 0.187 closed loop. All baseline results are re-runs by the authors from released checkpoints; UniAD, VAD and LTF were re-run on the current scenes and maps. Because the student publishes no trajectory, the native scorer receives what the authors call a decel stand-in plan: the realized first step followed by a comfort-bounded stop, which satisfies the comfort bounds by construction. The paper also cautions that absolute scores are not directly comparable across splits.
The weak spot is the Extreme tier: the student reaches 0.162 closed loop and 0.137 native, the teacher only 0.158 and 0.136, and ECO and the imitation baselines do better on parts of it. The authors say some of its scenarios are implausible, with other vehicles colliding with the ego regardless of what it does, so a policy that only accelerates straight ahead without reacting already scores 0.227 closed loop, above every reactive policy; the student, like its teacher, waits and yields, sacrificing route completion. That is the authors’ interpretation of how this tier is scored.
Concurrent self-play-to-pixel systems. GigaPixel and DriveZero report native HD-Scores on the older 345-scenario protocol (0.385 and 0.466), and their checkpoints are not public. Re-evaluating their own system on the same 345 scenarios, the authors report 0.527 for the teacher and 0.518 for the student. The scored objects differ, though: the authors’ numbers use the decel stand-in plan, while the other two systems are scored on the plans their policies output.
NAVSIM pseudo closed-loop planning. NAVSIM scores a 4 s plan. The planning student reaches 59.4 EPDMS on NAVSIMv2 navhard, 2.3 points above DriveZero-Scale (57.1) and 2.7 below the privileged, best-of-16 trajectory teacher (62.1); on NAVSIMv1 navtest it reaches 95.5 PDMS, against 95.3 for DriveZero-Scale. The baseline rows are published results compiled by He et al. (2026, the DriveZero paper). The authors also name the student’s weak points as safety margins and perturbed recovery, mirroring its teacher: a navtest TTC subscore of 92.2 (teacher 93.2, DriveZero-Scale 97.3) and a navhard Stage 2 DAC subscore of 90.2 (teacher 90.4, DriveZero-Scale 95.6). In other words, while leading on the totals, it trails DriveZero-Scale on both of these subscores.
Ablations: same data, different supervision source
Table 4 of the paper fixes, on the nuScenes split, the 1.83 million nuPlan paired frames (without the HUGSim domain-adaptation frames), the perception labels and the compute budget, and varies only the supervision source: imitating nuPlan logged actions scores 0.206; a modular cascade that recovers the vector inputs with SparseDrive and feeds them to the frozen policy head scores 0.361; the self-play teacher plus alignment scores 0.574. Holding out one loss, action-only distillation reaches 0.434 and structural-only 0.373. The rank ablation is run instead on all 436 scenarios with the headline data, including the domain-adaptation frames (Table 3): 7/5 dimensions for partner/road give 0.517, the 13/9 cutoff 0.548, 26/18 dimensions 0.533, and matching full coordinates 0.456.
The authors also run a privileged probe: the vision encoders are replaced by freshly initialized encoders of the teacher’s own architecture that read the vector observation, trained with the same losses and data. It scores 0.645 on nuScenes against the teacher’s 0.647, from which the authors conclude that the alignment objective itself costs almost nothing. The camera student on the same data scores 0.574, and the authors say this gap of 0.071 combines the cost of perception with the nuPlan-to-HUGSim domain gap; with the domain-adaptation frames it reaches 0.636, within 0.009 of the probe, and the authors relate that residual gap to an asymmetric-observation problem: the teacher’s vector state can include vehicles the camera cannot see. Their conclusion adds that they have not separated the occlusion component from other perception errors.
The questions from Physical AI Lab 1.8: Evaluation apply here too: each ablation runs on one split or one scenario set, the text we read reports no repeated seeds or confidence intervals, and the very small gaps (such as 0.002 and 0.009) should not be read as firm conclusions.
Limitations
- This is an arXiv preprint; all numbers are author-reported [R], and this site has not reproduced them. The text we read ends at the start of the references; we did not read the appendices (environment configuration, reward terms, per-tier breakdowns and the backbone ablation are there), and the text we read does not say whether code, weights or TerraZero are released.
- Most authors are from Applied Intuition, and TerraZero is their own simulator; the HUGSim baselines were re-run by the authors, and the NAVSIM baselines come from the DriveZero paper’s compilation.
- “Demonstration-free” means only that no logged trajectory is used as supervision: the closed-loop student’s alignment still needs real camera images, scene-state labels, and (in the Section 2.4 setup) the endpoint of the logged ego trajectory as the goal. The HUGSim domain-adaptation frames are rendered from the evaluation scenes themselves (covering 10% of their route length). The planning student is aligned on NAVSIM navtrain only.
- In HUGSim’s native scoring and the 345-scenario comparison, the student is scored on a comfort-bounded decel stand-in plan, not a trajectory of its own; the trajectory teacher’s best-of-16 selection reads ground-truth state and is a privileged reference, not a deployable method.
- All evaluations run in simulation or on log-based pseudo closed-loop benchmarks; the text we read contains no on-road vehicle tests, and these scores are not a safety validation. The future work the authors list in their conclusion is that the teacher may base decisions on occluded vehicles the camera cannot see, and that alignment data drawn from nuPlan logs and reconstructed scenes rarely includes severe weather, unusual times of day or rare scenario geometries.
Links
- TerraTransfer++: Demonstration-Free End-to-End Planner (arXiv 2610.30319)
- Full paper in HTML
- Physical AI Lab 1.6: Imitation Learning — Working Out How an Error Rolls Out of the Demonstrations’ Coverage, Then Putting the ACT, Diffusion Policy and Data Scaling Numbers Back in Their Own Conditions
- Physical AI Lab 1.7: Reinforcement Learning — Working Out One Episode’s Return to See How a Reward Gets Gamed, Then Reading the Paper’s Terrain Curriculum and Time-out Resets
- Physical AI Lab 1.8: Evaluation — Splitting a “Net Gain of 2” into a Paired Table, Then Checking the Failure Labels and Random Seeds
- Physical AI Lab: a physical AI course where every number can be checked (course outline)
