Whole-Body Dexterity

WBD-EX Object Repositioning Experts

Try out interactive whole-body dexterous experts!

General WBD Teleoperation controller

Try out the general WBD Controller with body pose tracked teleoperation! It will refine your teleoperation inputs into safe and stable object repositioing and avoid unstable actions!

Try the general distilled WBD policy interactively on a desktop computer.

Open the controller in a full window ↗

Whole-body grasp generation

Energy-based optimization and simulation filtering produce the arm-and-torso grasp cache used for training.

Any-to-any experts (WBD-EX)

Reinforcement-learning experts learn to reposition an object and settle it in a left-arm, both-arm or right-arm hold, using goals sampled from the grasp cache.

Distilled sim-to-real controller

The experts are distilled into one deployable policy that transfers zero-shot to a real Unitree G1 and is steered interactively by a teleoperator.

How does WBD work?

Generate whole-body grasps, train experts to move objects between them, then distill the experts into one controller.

Whole-Body Grasp Generation

Grasps are optimized from chosen contact links (blue), then kept only if they survive perturbations in simulation.

Optimization

Arm–torso grasp: optimization

Simulation filter

Arm–torso grasp: simulation filter

Optimization

Leg–arm grasp: optimization

Simulation filter

Leg–arm grasp: simulation filter

WBD-EX Training

Experts start from cached grasps and learn to move objects toward sampled positions in left-arm, both-arm or right-arm holds. Intermediate goals connect the current position to a farther goal, allowing sliding and rolling across the torso.

Cached grasps are grouped by object position and supporting arms. The expert follows intermediate goals toward a sampled farther goal.

A sphere expert is pre-trained on the any-to-any objective, then post-trained into experts for other object primitives.

Simulation Results

We add noise to expert commands and compare direct execution with WBD refinement. The guidance weight controls how strongly motion generation follows the coarse command.

Noisy expert commands, executed directly

Noisy expert

Noisy expert commands, refined by WBD

Noisy expert + WBD (ours)

Real-World Results

90%WBD completed trials
45/50 · distilled controller

18%GMR completed trials
9/50 · direct motion retargeting

4%SONIC completed trials
2/50 · generalist whole-body controller

Completed trials across two on-body repositioning tasks with a basketball and a cardboard box, plus a shelf-to-table task. GMR (General Motion Retargeting) directly retargets the operator’s motion; SONIC is a generalist whole-body controller.

Experimental setup: teleoperation with WBD controller

In the real-world experiments, an Xsens motion capture suit provides coarse motion references. WBD refines the arm targets; a separate controller handles lower-body motion and balance.

Teleoperator (Xsens suit)

Teleoperation + WBD controller

On-body repositioning: basketball (2× speed)

Shoulder tap requires object contact with the shoulder. Underarm stow requires the object center to reach shoulder depth in a stable single-arm hold. A full round trip returns to a stable two-arm front grasp.

SONIC

GMR

WBD (ours)

Shoulder tap

Underarm stow

Basketball · on-body repositioning
Task	Controller	Region reached ↑	Full round trip ↑	Time to region (s) ↓	Total time (s) ↓
Shoulder tap	SONIC	2/10	0/10	15.2	—
	GMR	3/10	1/10	26.0	30.1
	WBD	10/10	9/10	7.7	17.3
Underarm stow	SONIC	1/10	0/10	28.0	—
	GMR	4/10	2/10	31.5	49.8
	WBD	10/10	9/10	10.6	26.2

Cardboard box (15 cm cube) · on-body repositioning
Task	Controller	Region reached ↑	Full round trip ↑	Time to region (s) ↓	Total time (s) ↓
Shoulder tap	SONIC	2/10	2/10	15.9	24.9
	GMR	6/10	4/10	8.0	8.5
	WBD	10/10	10/10	4.1	6.3
Underarm stow	SONIC	2/10	0/10	26.1	—
	GMR	2/10	1/10	9.3	17.8
	WBD	10/10	10/10	8.6	13.0

Ten trials per controller, object and task, five per side. Times are measured from trial start and averaged over trials reaching the respective endpoint.

Long-horizon task: shelf pick, stow, carry and place

SONIC

GMR

WBD (ours)

Shelf-to-table task · stages completed out of ten trials per controller
Controller	Pick ↑	Stow ↑	Carry ↑	Un-stow ↑	Place ↑
SONIC	10/10	0/10	0/10	0/10	0/10
GMR	10/10	3/10	2/10	1/10	1/10
WBD	10/10	9/10	9/10	9/10	7/10

BibTeX

@misc{mcgartoll2026wholebody,
  title  = {Whole-Body Dexterity},
  author = {Conor Mc Gartoll and An Dang and Pranay Thangeda
            and Arjun Gupta and Carolina Higuera and Nima Fazeli
            and Mustafa Mukadam and Manikantan Nambi},
  year   = {2026},
  url    = {https://wholebodydexterity.com}
}