What Is ZeroMimic (Distilling Robotic Manipulation Skills from Web Videos)?

ZeroMimic is named in ZeroMimic: Distilling Robotic Manipulation Skills from Web Videos (arXiv 2503.23877), submitted on 31 March 2025 by Junyao Shi, Zhuolun Zhao, Tianyou Wang, Ian Pedroza, Amy Luo, Jie Wang, Jason Ma and Dinesh Jayaraman; the project page lists the University of Pennsylvania and ICRA 2025. The authors argue that imitation learning usually needs demonstrations collected on the same robot, in the same room, with the same objects the policy meets at test time, and that this scales poorly. Large pre-recorded human video datasets already exist, so they ask whether a repository of useful skill policies can be distilled from them with no robot-specific demonstrations or exploration. For the broader ideas, see the site’s imitation learning and affordance entries.

Firsthand is not affiliated with the ZeroMimic authors and makes no claim that any model, ZeroMimic included, was trained on Firsthand data. The authors train on the public EpicKitchens video dataset, which is a different source from Firsthand capture. Firsthand captures real human egocentric video, custom collection, fast and on demand, to a written spec, with depth derived from a calibrated stereo rig, 3D hand pose at 21 joints per hand and 6-DoF IMUs at 200 Hz, delivered in RLDS, LeRobot format or WebDataset under a buyer-owned commercial license, with one task instruction per episode. Affordance labels, grasp labels, post-grasp trajectories, robot trajectories, simulated demonstrations, action labels, reward labels, pretrained representations, teleoperation data and paired human-robot video are not Firsthand deliverables.

Per the project page, ZeroMimic has two phases. The grasping phase uses human affordance-based grasping to execute a task-relevant grasp. The post-grasp phase is an imitation policy trained on web videos to predict 6D wrist trajectories, and the trained model is deployed directly on the robot. The abstract describes the output as image goal-conditioned skill policies for opening, closing, pouring, pick-and-place, cutting and stirring, each able to act on diverse objects in unseen task setups. The project page lists nine trained skill policies: vertical hinge opening and closing, slide opening and closing, pouring, picking, placing, cutting and stirring.

The authors train on EpicKitchens, which the project page describes as an in-the-wild egocentric dataset of daily kitchen activities, with 20M frames in 100 hours. To get wrist trajectories they run HaMeR, a pre-trained hand-tracking model, for 3D hand pose reconstruction, and use camera extrinsics and intrinsics from EPIC-Fields to account for the motion of the egocentric camera. The abstract says the system is designed to exploit advances in semantic and geometric visual understanding of human videos, together with grasp affordance detectors and imitation policy classes.

The abstract reports out-of-the-box evaluation in varied real-world and simulated kitchen settings with two robot embodiments, and says the authors release software and policy checkpoints. The project page describes 34 distinct scenarios across 18 object categories in 7 kitchen scenes, and a comparison with ReKep on four tasks with a Franka robot. These are the authors’ descriptions; Firsthand has not verified them and repeats no success rates, because none were read here.

The abstract and project page read do not include a limitations list, so none is repeated here. They do state that, before the evaluations, the authors position the camera and the robot at an angle roughly similar to the relative camera angle of the human hand in egocentric videos, and that they run 10 trials with varying camera and robot positions. The evaluated scenes are kitchens. Anything beyond that is Not stated in the sources read.

The first table sets ZeroMimic beside four site entries. The ZeroMimic abstract and project page read do not mention VRB, HRP or Phantom, so those rows say so; they name ReKep as the concurrent comparison. The shared idea is learning from human video. The differences are what is learned, which video is used and whether the robot is involved before deployment.

Per the sources read, it needs a large egocentric human video dataset, a hand-tracking model, camera pose for each video and a robot with a camera positioned like the human view. The second table sets this beside the Firsthand site. That is a note about format, not a compatibility claim: hand tracking, affordance detection, policy training and the robot are the buyer’s own.

This site’s EPIC-KITCHENS entry names that dataset’s video and annotation licences as non-commercial. For real human demonstrations that public video does not cover, no off-the-shelf dataset for these tasks is stated here, so this is custom collection: request on-demand collection and confirm it on a Spec Pilot, with a median of 48 hours to a first validated batch where coverage is deep and three to four weeks where it is thin.

What is ZeroMimic?

ZeroMimic is a published system (arXiv 2503.23877) that distills image goal-conditioned robot skill policies from egocentric human web video, with no robot-specific demonstrations or exploration. It is third-party research.

Does ZeroMimic need robot data?

Per the authors, no. The project page says it learns purely from passive human videos and does not require any robot data, and the policies are deployed zero-shot.

What video does ZeroMimic train on?

The authors train on the public EpicKitchens egocentric dataset. The project page describes it as 20M frames in 100 hours of daily kitchen activity.

What skills does ZeroMimic cover?

The abstract names opening, closing, pouring, pick-and-place, cutting and stirring. The project page lists nine skill policies, including vertical hinge and slide opening and closing.

Was ZeroMimic trained on Firsthand data?

No. Firsthand is not affiliated with the authors and claims no use, training or compatibility. Grasp labels, post-grasp trajectories and action labels are not Firsthand deliverables.

How can I request a custom egocentric dataset for human-video robot learning?

Request a custom dataset with your tasks, environments and signals. Collection is fast and on demand, with a first validated batch in a median of 48 hours where coverage is deep.