Frontier Radar | TouchBridge: retargeting human demonstrations by measured fingertip contact, with no object mesh or simulation, to turn them into dexterous-hand training data

TouchBridge retargets human demos by measured fingertip contact. Authors report 72.5% vs 77.5% for equal teleop data on four tasks; collection is 7.0× faster.

Left-to-right timeline diagram, with the title “TouchBridge: human demos to robot data” at the top and a source line at the bottom. Four numbered circular nodes in a row are joined by arrows, each with a short text below it. Step one: TouchBridge-Cap records RGB, glove hand motion and fingertip touch; step two: Tac-Retarget uses fingertip pose, contact patches and a contact graph to solve for robot hand bases and joints; step three: robot-domain views are synthesized with SAM 2 masks, ProPainter inpainting and rendered robot hands; step four: camera-anchored action chunks, 60 demos per task, fine-tune GR00T N1.7.
TouchBridge’s pipeline for turning human demonstrations into robot training data, drawn from Sections III and IV-A of arXiv 2610.40349 (author-reported). Touch is used only in the retargeting of step two, not as a policy input; the conversion uses no object mesh, object-pose estimate or physics simulation, but requires aligned camera and fingertip tactile layouts on the human and robot sides. Arrows show only the order of the pipeline; numbering, circle size and color encode no ranking or measurement. 60 is the number of demonstrations per task used for fine-tuning.

Learning dexterous manipulation from human demonstrations runs into the difference between human and robot hands. Common retargeting methods match hand poses, fingertip positions or hand-centric vectors, which works for motion through free space. Once a finger touches an object, fingertip targets computed from the human hand can push the finger off the region that was supporting the object, distort the relative positions of several simultaneous contacts, or break contact altogether. The preprint TouchBridge (arXiv 2610.40349, 2026-10-29) fits the same kind of tactile array to human and robot fingertips and uses measured contact as the correspondence between the two hands: besides tracking fingertip poses, retargeting requires the contact regions on the robot fingertips to line up with the human ones and preserves the spatial relations among several contacts, all without an object mesh, an object-pose estimate or physics simulation. The authors report that on four real-robot tasks, fine-tuning GR00T N1.7 on 60 converted human demonstrations per task gives an average success rate of 72.5%; the same number of teleoperated robot demonstrations gives 77.5%; and human demonstrations are collected 7.0× faster. This article looks at what each of these three numbers measures, and at the hardware assumption behind “no object model”. All numbers below are author-reported [R]; this site has not reproduced them.

The problem: matching finger positions alone loses contact

The authors frame human-to-robot data conversion as two challenges. First, kinematic correspondence alone does not preserve physical interaction: DexPilot and AnyTeleop optimize hand-centric task-space or keypoint-vector correspondences, and UniDex combines fingertip inverse kinematics with manual adjustment of the hand-base offset; none of these objectives captures contact explicitly. Second, preserving contact should not cost scalability: in the authors’ summary, methods such as FunGrasp, DexFlow and TopoRetarget need object geometry or an object mesh and pose trajectory, while ManipTrans and SimToolReal rely on reinforcement learning in simulation or an object-centric policy trained in simulation, and this object- and task-specific modeling is hard to scale with ever larger human datasets.

The authors’ key observation is that physical contact itself is a natural correspondence across embodiments. Tactile signals say directly which fingertips are in contact, where on the fingertip the contact lies and how strongly each region is loaded, and they are measured during the human demonstration, so no object reconstruction or simulation-based correction is needed. Comparing other routes that need no robot data, the authors write that DexUMI’s exoskeleton must mechanically match the target hand’s kinematics, so a new hand generally means new hardware; HumanEgo has only been demonstrated with parallel-jaw grippers; and AINA transfers to multi-fingered hands using visually estimated hand–object geometry and fingertip motion, without explicit tactile contact constraints. These are the authors’ summaries of related work; this site has not checked each paper.

Hardware and conversion pipeline: one sensing layout for human and robot

TouchBridge is designed as hardware and software together. The human-side capture rig is called TouchBridge-Cap:

  • Vision. An Intel RealSense D455 records egocentric RGB, and two Hikrobot cameras record the left and right wrist views.
  • Hand motion. A pair of Manus gloves records articulated hand motion. Multi-face AprilTag fiducials near the wrists are localized offline in the egocentric video, and calibrated fiducial-to-glove transforms bring the glove-estimated fingertip poses into the D455 camera frame.
  • Touch. A PaXini Hall-effect tactile array on each fingertip records spatially distributed contact responses.
  • Sampling rates (appendix Table III). Egocentric and wrist RGB at 30 Hz, Manus hand poses at 130 Hz, fingertip touch at 90 Hz. All streams are timestamped with one shared host clock, and tactile readings are linearly interpolated to the visual timestamps.

The robot side has an egocentric camera, left and right wrist cameras, and five PaXini tactile sensors on each dexterous hand. Custom mounts place the wrist cameras similarly to those on TouchBridge-Cap and put the tactile sensors at the corresponding fingertip locations. In the paper’s words, the robot interface “approximately matches” the human side’s wrist-camera-to-hand geometry, fingertip sensor placement and tactile layout.

The conversion has three steps:

  1. Retargeting. Tac-Retarget turns the camera-frame fingertip trajectories and tactile observations into base poses and joint configurations for the two robot hands (detailed in the next section).
  2. Visual synthesis. The wrist-view images are kept as they are, because the camera layouts on both sides are already aligned. The egocentric video is segmented with SAM 2 to find the human hands and wearable hardware, the masked regions are inpainted with ProPainter, and robot meshes are rendered onto the frames using the calibrated camera intrinsics with depth-aware occlusion.
  3. Action construction. Following UMI, arm actions are end-effector poses relative to the pose at the start of each action chunk, and hand actions are the absolute joint targets from retargeting. The motion of the head-mounted camera is estimated from an AprilTag fixed in the workspace (separate from the wrist fiducials used for hand tracking), and each end-effector pose is transformed into the camera frame at the chunk’s anchor time.

One point deserves emphasis: the paper states that tactile measurements are used only for retargeting, not as policy inputs. Training uses the converted observations (synthesized egocentric frames, with the wrist-view images kept as recorded); on the real robot, the images come from the robot’s own egocentric and wrist cameras. During collection, operators were asked to keep head motion to a minimum within each demonstration to reduce frame-to-frame viewpoint changes, while camera position and viewing angle were varied across demonstrations to diversify camera–object configurations.

Tac-Retarget: three objectives carry contact over to the robot hand

At each time step Tac-Retarget solves an optimization problem. The variables are the base poses of both robot hands and all joint angles of both hands, and the objective is the sum of four parts (the paper’s Eq. 1): fingertip pose alignment, tactile contact alignment, a contact graph, and regularization (hand-base regularization, feasibility penalties and temporal smoothing). The paper stresses that no object mesh or object pose enters the optimization.

  • Fingertip pose alignment. Robot and human fingertip positions and orientations are matched in the camera frame. Fingertip poses are defined by the calibrated tactile-sensor frames, so the motion target and the taxel layout share one frame. The base poses of both robot hands are optimization variables rather than tracking targets copied from the measured human wrists.
  • Tactile contact alignment. A fingertip counts as in contact when its total response exceeds a threshold (τ_F = 0.02 in appendix Table IV) and its number of active taxels reaches a minimum; a taxel is active when its response exceeds τ_s = 0.01 (both thresholds are expressed in the input tactile-response scale). For each contacting fingertip, a contact center is computed as the response-weighted average of taxel positions. Because both sides use the same calibrated taxel layout, the authors use one local contact center, derived from the human data, for both hands; it is transformed into the camera frame with the measured human sensor pose and with the robot sensor pose from forward kinematics, and the two are required to coincide. A second term also aligns the spatial distribution of active taxels, giving stronger-responding taxels more weight.
  • Contact graph. To preserve the spatial relations among simultaneous contacts, the authors follow TopoRetarget’s distance-weighted Laplacian matching and adapt it to a tactile-activated graph: within each hand, contacting fingertips are connected in pairs; across the two hands, active contacts are connected too. The graph topology and normalized edge weights are computed from the human demonstration and applied unchanged to the robot graph; edges get more weight when the contacting fingertips are closer and when tactile responses are stronger. The implementation also penalizes differences in the distances between all pairs of contacting fingertips, with an extra coefficient for cross-hand pairs.

The appendix adds two implementation details that show how it trades off following the human motion against keeping contact:

  • Looser pose tracking during contact. With the appendix parameters, the fingertip pose-tracking weight during detected contact drops to 0.15 times its non-contact value, so the tactile terms can influence the solution while fingertip motion is still tracked.
  • Pressing in a little, by response. Contact-patch alignment keeps the contact geometry but does not represent pad deformation under load. The authors add a compensation term that pushes the target contact center into the object along the direction from the sensor toward the contact, by an amount that grows with the tactile response up to d_max = 0.01 m, to encourage a firmer grasp rather than merely touching the surface.

Results: what each of the three numbers measures

All real-robot evaluations use a Moz1 bimanual robot with two Wuji Hands. The four tasks are Flip Book (turn one page with the right index finger, keeping fingertip contact throughout without moving the book), Screw Bulb (screw a bulb into a socket until it lights), Apply Sauce (hold a baguette in the left hand and squeeze ketchup onto it from left to right with the right hand, which requires fingertip force control) and Grasp Egg Roll (pick up a fragile egg roll with the right hand and place it on a plate without slipping or crushing it). For each task the authors collected 60 human demonstrations and 60 teleoperated robot demonstrations; the teleoperated data serve only as a reference for data-quality analysis and to train separate comparison policies, and are not mixed into the human-data training sets. Each policy is evaluated 20 times per task on the real robot. The main results the authors report:

Setting (60 demonstrations per task)Average success over four tasksNote
TouchBridge-converted human demonstrations, fine-tuning pretrained GR00T N1.772.5%The main result; the rows below are compared with it
Teleoperated robot demonstrations, fine-tuning pretrained GR00T N1.777.5%Same training setup
Same human demonstrations retargeted with DexPilot47.5%The paper gives “25.0 percentage points lower”; the value is this site’s arithmetic
Same human demonstrations retargeted with UniDex30.0%The paper gives “42.5 percentage points lower”; the value is this site’s arithmetic
TouchBridge data, fine-tuning pretrained π0.538.8%Policy and pretraining comparison
TouchBridge data, GR00T N1.7 / π0.5 trained from scratch6.2% / 3.8%Same
TouchBridge data, Diffusion Policy13.8%An additional baseline chosen by the authors

Points to keep in mind when reading this table:

  • 72.5% against 77.5% is 80 trials against 80. At 20 trials on each of four tasks, the averages correspond to about 58 against 62 successes, a difference of about 4 (this site’s arithmetic). The authors write that the two match on Flip Book and stay within ten percentage points on each of the other three tasks; the per-task values are in Fig. 6 and not listed in the text. The error bars in the figure are the standard deviation across the four tasks, not the uncertainty from the number of trials within a task; the text we read gives no confidence intervals or significance tests. “Close to teleoperated data” is the authors’ wording.
  • 7.0× is a ratio of collection times. The authors measure average collection time per successful demonstration: total elapsed collection time, including failed attempts and scene resets, divided by the 60 successful demonstrations. TouchBridge-Cap averages 11.7 s, teleoperation 81.7 s; over 60 demonstrations that is about 11.7 minutes against about 81.7 minutes per task (this site’s arithmetic). The ratio does not include the compute time of the conversion pipeline (segmentation, inpainting, per-frame optimization, rendering); the text we read does not report that time or say how many operators collected the data.
  • The retargeting comparison covers only methods that need no simulation. DexPilot and UniDex use the same 60 human demonstrations, the same policy and the same training setup, with only the retargeting method changed. The paper does not compare on the real robot with methods that need an object model, such as TopoRetarget; the authors’ own experimental question is limited to “existing simulation-free retargeting methods”. Removing the contact graph or contact alignment lowers average real-robot success, with the largest drop when both are removed; the values for this ablation are in Fig. 9 and not given in the text.
  • 72.5% depends on pretraining. On the same 60 converted demonstrations, GR00T N1.7 trained from scratch reaches only 6.2%. The authors use this to show that pretraining matters when learning from limited human-derived data. It also means the main result shows converted data used to fine-tune a pretrained VLA, not that converted data are sufficient on their own.

The paper also has two offline or simulated checks, which measure something different from real-robot policy success:

  • Action replay. Converted actions are replayed directly on the robot with the initial scene matched to collection, 20 per task, for an average replay success of 87.5% (about 70/80, this site’s arithmetic). This tests whether the converted actions themselves complete the task, not policy generalization.
  • Contact error. Measured on Grasp Egg Roll only, which the authors consider more tractable for reliable contact simulation. The same 20 human demonstrations are retargeted with each method and replayed in Isaac Sim with an accurately reconstructed object mesh; contact error is the Euclidean distance between the human tactile contact center and the simulated contact center in the corresponding fingertip-local frames (Table II):
MethodContact error (mm)
Tac-Retarget4.667
DexPilot6.354
UniDex7.715
Tac-Retarget (without contact graph)6.659
Tac-Retarget (without contact alignment)7.071
Tac-Retarget (kinematic terms only)7.540

The method itself uses no object mesh, but this evaluation relies on a reconstructed mesh and a simulator, and it covers one task. For visual similarity, the authors feed each observation into the vision-language backbone of GR00T N1.7 and, after projection, see the features of converted observations overlap substantially with real-robot observations, while raw human observations are more separated; this is a visualization of feature distributions, and the text we read gives no numerical metric.

How this relates to other content on this site

Physical AI Lab 1.5: Where Does Data Come From? says of the human-video route that a human hand is not a robot hand: the video has no exact joint angles and no force, so the human hand has to be translated into a robot hand first. TouchBridge is one way to do that translation, and what it adds is contact: people record demonstrations wearing gloves and fingertip touch sensors, and conversion uses the measured contact to constrain where the robot fingers land. The cost is that the capture side is no longer a single head-mounted camera but a set of gloves, fiducials and tactile arrays aligned with the robot side. Tactile sensors as a category and what they are for are covered in Physical AI Lab 1.3: Sensors; the difference here is that touch is used only in data conversion, and the deployed policy does not read it.

This site has covered several other ways of dealing with the gap between human and robot hands. Frontier Radar | DexPort has the collector hold the very robot hand used at deployment, so no retargeting is needed; Frontier Radar | EgoJEPA skips retargeting and learns latent actions from egocentric human video; Frontier Radar | DexStitch also converts human video into dexterous-hand data. TouchBridge keeps human-hand collection and keeps retargeting, but adds aligned touch sensing on both sides. Frontier Radar | PhysCanvas uses tactile images as one of the inputs to a world model, which differs from the use here, where touch enters conversion but not the policy. These papers differ in tasks, hardware and metrics, so their numbers cannot be compared with one another.

The size and pretraining data of GR00T N1.7, the model behind the main result, are covered in Physical AI Lab 0.2: Why Now?. Twenty trials per task, error bars taken as the standard deviation across tasks, and an average that hides per-task differences are all things you can practise reading with Physical AI Lab 1.8: Evaluation.

Limitations

  • This is an arXiv preprint; all numbers are author-reported [R], and this site has not reproduced them. Each policy is evaluated 20 times per task, and the text we read gives no confidence intervals, significance tests or random seeds.
  • This article is based on the paper’s HTML text and table text; we did not read the per-task values in Figs. 6, 9 and 10. Per-task success rates and the real-robot ablation values are in those figures and are not cited here.
  • The experiments use one platform (a Moz1 bimanual robot with two Wuji Hands), four tasks and 60 demonstrations per task. The authors state that the current study uses a modest demonstration set and that extending to larger, more diverse human datasets is future work.
  • “No object mesh, pose estimate or simulation” refers to retargeting itself. The method depends on aligned camera layouts and identically calibrated fingertip taxel layouts on the human and robot sides, and the authors say the robot interface “approximately matches” the capture rig. Whether it carries over to a robot hand without such a tactile layout is not tested in the text we read.
  • 7.0× compares collection time only; it excludes the compute time of the conversion pipeline and is not a full cost comparison. The text we read does not state the number of operators or which interface was used for teleoperation.
  • Contact error is measured only in simulated replay on one task, and the evaluation itself uses a reconstructed object mesh.
  • The experiments run on a real robot and include a fragile object, but the text we read contains no safety evaluation; these results are not a hardware safety validation.
  • The paper gives a project website, touchbridge2026.github.io; the text we read does not say whether code, hardware designs or data are released, and this site did not open the page to check. The acknowledgment states that the work is supported by a joint research center of Tsinghua University and Keystone Electrical (Zhejiang).
  • 58 against 62 successes, 47.5% and 30.0%, about 70/80, and about 11.7 against about 81.7 minutes per task are this site’s arithmetic from the paper’s numbers.

Links