General-purpose mobile manipulation requires a combination of capable robot hardware, scalable demonstration collection, expressive whole-body control, and effective policy learning. Yet these components are often developed independently, creating mismatches that limit the capabilities of the complete system. We present Pixel, a co-designed full-stack system for learning general-purpose mobile manipulation. Pixel combines a low-cost wheeled humanoid with human-aligned kinematics and compliant grippers, multiple teleoperation and robot-free human data-collection interfaces, a unified whole-body controller, and a common policy learning pipeline across human and robot demonstrations. A hierarchical learning stack further enables language-conditioned visuomotor control and long-horizon task execution. We evaluate the system across hardware, data collection, and policy learning, and demonstrate mobile-manipulation behaviors requiring dexterity, active perception, navigation, and whole-body coordination. We hope that Pixel provides an integrated and extensible foundation for studying how robot hardware, data, control, and learning can be designed together for robot learning.
System Co-Design
Hardware Design
We co-design our wheeled humanoid platform and sensorized human data-collection system to align camera viewpoints, gripper geometry, and task-relevant motion, supporting transfer from human demonstrations to robot deployment.
Pixel
Loading 3D model…
Human
Loading 3D model…
Interface Design
Whole Body Controller
We use geometric fabrics for whole-body control, providing smooth, safe motion and a unified interface for teleoperation and policy rollouts.
Interactive whole-body controller
Visualization
Real World Deployment
Teleoperation Interfaces
We enable expressive whole-body teleoperation through a unified controller and compare three complementary interfaces.
Leader Arm
Our custom 3D-printed leader arm enables precise whole-body control for dexterous tasks.
VR Joystick
Our VR interface enables expressive whole-body control and untethered teleoperation.
VR Body Tracking
Pixel follows the operator’s body motion for coordinated, untethered whole-body teleoperation.
Sensorized Human Data
Our low-cost, 3D-printed sensorized human data-collection system captures fine-grained dexterous demonstrations and supports untethered whole-body mobile data collection.
Camera Battery Replacement
Headphone Packing
Trash Disposal
Gear Lubrication
Tidy Pillow
Toy Cleanup
Ethernet Cable Connection
Socket Wrench Assembly
Policy Design
A high-level vision-language model uses voice commands, interaction context, and visual keyframes to plan subtasks and respond to the user. A language-conditioned low-level policy combines head and wrist images with a DiT-based flow-matching model to predict action sequences. A shared whole-body controller executes these predictions using a common action space for human and robot demonstrations.
Tasks
Autonomous Tasks
Sensorized Human Data
Autonomous 1x
Trash Disposal
Pixel learns diverse human behaviors, including active visual search.
Autonomous 1x
Toy Cleanup
Pixel learns simultaneous whole-body coordination from human demonstrations.
Autonomous 1x
Tidy Pillow
Pixel learns long-horizon navigation with smooth transitions into bimanual manipulation.
Robot Data
Autonomous 1x
Vacuum Cleaning
Pixel learns to use everyday appliances and performs long horizon mobile manipulation tasks.
Autonomous 1x
Snack Retrieval
Pixel combines articulated-object manipulation with delicate object handling and human interaction.
Autonomous 1x
Hang Jacket
Pixel combines human interaction with coordinated manipulation of deformable objects made from different materials.
Autonomous 1x
Floor Sweeping
Pixel learns whole-body coordination for tasks that require all of its joints to work together.
Autonomous 1x
Tidy Pillow
Pixel extends its workspace to pick up objects from the floor.
Autonomous 1x
Give Jacket
Pixel connects deformable-object manipulation, navigation, and human interaction.
Autonomous 1x
Toy Cleanup
Pixel coordinates its full body to manipulate objects while moving.
Long Horizon Manipulation
Autonomous 2x
Single Policy Language Steering
Pixel performs multiple tasks over several minutes using a single language-conditioned policy, with only the language prompt changing.
Autonomous 2x1x during speech
Agentic Planning and Execution
Combining a vision-language model and execution harness with Pixel’s multi-task policy enables natural-language interaction, multi-step planning, and monitoring of task progress.
Robustness
Autonomous 1x
Reactive Behavior
Pixel’s reactive policies adjust its motion as the environment changes during execution.
Autonomous 1x
Recovery
Pixel’s policy learns natural recovery behaviors that appear only a few times in the demonstration data.
Autonomous 5x
Repeatable Execution
Pixel sustains repeated task execution as new objects are introduced.
Autonomous 5x
Adaptability
Pixel adapts its whole-body behavior to unseen environment configurations, enabling reliable execution over extended periods.
Teleoperated Tasks
1x Teleoperated
Picking Up Multiple Glasses
Pixel’s end-effector design enables it to grasp multiple objects simultaneously.
1x Teleoperated
Loading the Dishwasher
Pixel combines delicate object handling with the strength needed to operate household appliances such as dishwashers.
1x Teleoperated
Writing on a Whiteboard
Pixel’s compliant controller and reactive teleoperation system maintain controlled surface contact during precise writing motions.
1x Teleoperated
Starting a Record Player
Pixel’s low-latency teleoperation system enables the delicate, precise movements needed to operate a record player.
1x Teleoperated
Organizing a Bookshelf
Pixel’s teleoperation system enables coordinated whole-body control.
1x Teleoperated
Arranging Office Chairs
Pixel’s teleoperation system enables coordinated whole-body motion to reposition large objects such as chairs.
1x Teleoperated
Closing the Blinds
Pixel’s gripper combines compliance with the grasping strength needed to operate window blinds through a thin, flexible cord.
1x Teleoperated
Spraying and Wiping Surfaces
Pixel’s gripper supports a broad range of everyday tool-use tasks, including operating a spray bottle.
More Experiments
Hardware DoFs
Disable Head
A fixed head prevents the robot from mimicking human active vision, so it turns its whole body and commits to one side.
Observation Space
Head Only
The head camera alone lacks the close-up detail needed for precise actions such as pressing the vacuum’s power button.
Action Space
Relative-pure
Without a shared reference frame, bimanual coordination is poor and the actions drift over time.
Without a shared anchoring frame, relative actions drift over time.
BibTeX
Publication details to be finalized.
@misc{pixel,
title = {Pixel: A Co-Designed Full Stack Robot System for Learning General-Purpose Mobile Manipulation},
author = {Tony Tao and Jason Jingzhou Liu and Ruslan Salakhutdinov and Deepak Pathak},
year = {TBD}
}
