What Is Loco-Manipulation?

For a robotics data buyer, loco-manipulation changes what “enough data” means. Tabletop clips show the hands but never the walking, turning and carrying between tasks, so a policy trained on them has no examples where the move is part of the task. Scope it by separating the human-demonstration layer (viewpoint, hands, body, scene) from the robot-action layer (commands).

Firsthand collects custom egocentric human-demonstration data on demand against a written spec, billed only for validated hours under a buyer-owned commercial license. Where coverage is deep, a first validated batch lands in a median of 48 hours from signed spec, and a Spec Pilot proves the spec before you scale. This is human data, not robot data.

Loco-manipulation, short for locomotion plus manipulation, describes a robot that moves its base or body through space while it manipulates objects, instead of reaching from a fixed position. The movement is part of the task: walking to a shelf, lifting a box, turning and carrying it to another room is one behavior, not a walk followed by a separate pick.

The term is common for humanoids and legged robots, where the whole body moves and balances during manipulation. “Humanoid whole-body manipulation” and “mobile manipulation training data” describe the same family of problems: the robot’s motion and its hands are handled together.

A fixed-base arm has a constant frame of reference. A mobile manipulator, typically arms on a wheeled base, usually drives to a position and then manipulates. In loco-manipulation the base or body moves while contact with objects is happening or about to happen, so movement, balance and manipulation are coupled. The first table compares the three.

Arms, torso and legs move together. Reaching shifts balance, and a carried object shifts it again. The viewpoint moves too, so objects change in scale and angle. Episodes are also longer: transit, approach, grasp, carry and place chain together.

NVIDIA’s GR00T N1 paper says the model “focuses primarily on short-horizon tabletop manipulation tasks” and names “long-horizon loco-manipulation” as future work requiring “advancements in humanoid hardware, model architecture, and training corpora” (arXiv 2503.14734, Section 4.6).

A program needs continuous episodes that keep transit instead of cutting to the manipulation; hand and body state recorded together; a viewpoint that moves with the body; labeled transitions; varied scenes; and streams on one clock. Robot-side signals such as joint and base velocity commands are a separate layer that human demonstrations do not contain.

A humanoid sees from a head camera and acts with two hands; a head-worn camera on a person records a similar viewpoint while the person walks and works in one episode. GR00T N1 organizes training data as a pyramid with web data and human video at the base, synthetic data in the middle and real-robot trajectories at the top (arXiv 2503.14734).

Figure’s Project Go-Big announcement reports that Helix, trained exclusively on egocentric human video, learned to navigate cluttered real-world spaces from language commands, and that one network outputs both manipulation and navigation. That is Figure’s own result, not evidence about any dataset. Human video carries no robot actions, so the embodiment gap remains.

The site describes egocentric capture from a head-mounted 4K/60 camera and two wrist cameras, with depth, IMUs, 3D hand pose and 3D body pose, hardware-triggered onto one PTP clock with sync within a 2 ms ceiling. Episodes keep movement between locations, with ordered action segments and contact events.

Firsthand captures human demonstrations only. The site does not state a head or camera 6-DoF pose or SLAM trajectory, and it does not capture robot or teleoperation data, so base velocity and robot joint commands are not part of a delivery. The second table sets out each signal.

What is loco-manipulation in simple terms?

A robot moving its base or body through space while it manipulates objects, instead of working from a fixed position, such as walking to a shelf and carrying a box to another room.

Is loco-manipulation the same as mobile manipulation?

They overlap and usage varies. Mobile manipulation usually means arms on a base that moves between stops; loco-manipulation stresses that moving, balancing and manipulating are coupled, as in humanoids.

What training data does loco-manipulation need?

Continuous episodes that keep transit, synchronized hand and body state, a moving viewpoint, labeled transitions and varied scenes. Robot commands are a separate layer.

Can egocentric human video train a loco-manipulation policy?

It supplies viewpoint, scene and behavior, not robot actions. GR00T N1 places human video at the base of its data pyramid; bridging the embodiment gap is still required.

Does Firsthand collect robot or teleoperation data?

No. Firsthand captures first-person human demonstrations; using them for a specific robot means bridging the embodiment gap.

Does Firsthand deliver a camera trajectory or SLAM output?

Not stated. The site describes video, depth, IMU, hand pose and body pose, but no head or camera 6-DoF pose and no SLAM trajectory.

How fast can custom data be collected?

Capture is on demand against a written spec. Where coverage is deep, a first validated batch lands in a median of 48 hours from signed spec.