What Is Humanoid Policy ~ Human Policy (HAT)?

The paper is titled “Humanoid Policy ~ Human Policy” (arXiv 2503.13441), submitted on 17 March 2025 and last revised on 5 October 2025; the lead authors are Ri-Zhao Qiu, Shiqi Yang, Xuxin Cheng and Chaitanya Chawla, with the first listed affiliation UC San Diego. The authors start from a cost problem: training humanoid manipulation policies on robot demonstrations alone needs expensive tele-operated data collection that is hard to scale. Their proposal is a more scalable source, egocentric human demonstrations, used as cross-embodiment training data, with the embodiment gap between humans and humanoids tackled from both the data side and the modeling side. The data contribution is PH2D, an egocentric task-oriented dataset aligned with humanoid manipulation demonstrations. The modeling contribution is HAT, which the abstract describes as a human-humanoid behavior policy with a unified state-action space that can be differentiably retargeted to robot actions. Co-trained with smaller-scale robot data, it models humanoids and humans as different embodiments without additional supervision. The authors report that human data improves both generalization and robustness, with significantly better data collection efficiency. Those are the authors’ findings; Firsthand has not reproduced them.

Firsthand is not affiliated with the authors and makes no claim that HAT, or any other model, was trained on Firsthand data. PH2D is the authors’ own dataset, collected with their own setup, and is a different source from Firsthand capture. Firsthand captures real human egocentric video, custom collection, fast and on demand, to a written spec, with depth derived from a calibrated stereo rig, 3D hand pose at 21 joints per hand and 6-DoF IMUs at 200 Hz, delivered in RLDS, LeRobot format or WebDataset under a buyer-owned commercial license, with one task instruction per episode. Robot trajectories, humanoid teleoperation data, retargeted actions, action labels, reward labels, pretrained representations, embeddings, paired human-humanoid data and synthetic data are not Firsthand deliverables.

PH2D stands for Physical Human-Humanoid Data. The paper describes it as a large egocentric, task-oriented human-humanoid dataset with accurate hand and wrist poses, built so it can be used directly for co-training. Task-oriented is the point: the authors contrast it with existing egocentric datasets that cover mostly non-task-oriented skills and lack world-frame 3D head and hand pose supervision. Operators perform tasks that overlap with robot execution, such as grasping and pouring, while wearing consumer-grade VR devices, and every demonstration carries a language instruction. The project page says the authors collected about 27k demonstrations with language annotations in dozens of hours.

The official dataset card on Hugging Face says each folder is one task, most HDF5 files hold one execution of the task described by the language instruction, and task ids that start with 1 are human demonstrations. The authors’ GitHub repository describes the open-sourced recordings as many humans performing the paper’s tasks in diverse in-the-wild scenes, plus humanoid recordings collected by teleoperation. All counts here are the authors’, taken from their project page.

HAT is a transformer-based policy that predicts chunks of future actions. The authors build a unified state-action space for bimanual humanoid robots and humans, since both have two end effectors and the robots in the paper also have an actuated neck that resembles human head movement. In the paper, the proprioceptive state is a 54-dimensional vector covering the head, both wrists and ten finger tips. Training samples a state-action pair from either human or robot data, images are encoded by a frozen visual backbone, and at deployment the human-centric actions are converted to robot actions with inverse kinematics and hand retargeting.

Two alignment choices are worth noting. Because humans finish tasks much faster than teleoperated robots, the authors interpolate human motion during training, effectively slowing it down; the paper reports a slow-down factor around 4 and uses 4 throughout. And they ask human data collectors to sit upright to limit whole-body movement that a humanoid cannot reproduce. The paper also states a limit: the VR hand-tracking software was trained mostly for VR use, so hand keypoint tracking can fail on motions with heavy occlusion.

The authors run real-robot evaluations on four dexterous manipulation tasks, in-distribution and out-of-distribution, and compare a robot-only baseline with HAT co-trained on human data. They report that human data has a minor effect in-distribution but improves out-of-distribution settings for background, object appearance and object placement, and that co-training helps few-shot adaptation to a second humanoid with a different arm configuration. They also report that human demonstrations take far less time to collect than humanoid teleoperation.

This page does not repeat the success rates or timing figures. They come from the authors’ own experiments on their own robots and tasks, and the paper itself notes that its evaluations use robots with dexterous hands. Read the paper for the numbers, and treat them as the authors’ results, not Firsthand claims.

The first table sets HAT beside four site entries. The HAT paper names EgoMimic as concurrent work; the authors say EgoMimic requires strict visual sensor alignment while they show that diverse observations from different cameras make the policy more robust. The paper text read does not mention Phantom, ZeroMimic or GR00T N1, so those rows say so and compare only against what each site entry states.

Per the sources read, it needs egocentric human demonstrations of well-defined, task-oriented manipulation with per-demonstration language instructions, 3D hand and head pose supervision, and a smaller amount of robot data from the target humanoid. The second table sets this beside the Firsthand site. That is a note about format, not a compatibility claim: the unified state-action space, retargeting, robot teleoperation data and policy training are the buyer’s own.

No off-the-shelf Firsthand dataset for this research setup is stated here, so this is custom collection: request on-demand collection and confirm it on a Spec Pilot, with a median of 48 hours from signed spec to first validated batch where coverage is deep and three to four weeks where it is thin. Billing is per validated hour.

What is Humanoid Policy ~ Human Policy?

It is a published paper (arXiv 2503.13441) that uses egocentric human demonstrations as cross-embodiment training data for humanoid robots. It introduces the PH2D dataset and the HAT policy. It is third-party research.

What does HAT stand for?

HAT stands for Human Action Transformer. The abstract describes it as a human-humanoid behavior policy whose state-action space is unified for humans and humanoids and can be differentiably retargeted to robot actions.

What is PH2D?

PH2D is the authors’ egocentric, task-oriented human-humanoid dataset, aligned with humanoid manipulation demonstrations. The official release is on Hugging Face and the code is on GitHub, both linked below.

Does HAT need robot data?

Yes. Per the abstract, HAT is co-trained with smaller-scale robot data alongside the human data, and the authors treat humans and humanoids as different embodiments.

Was HAT trained on Firsthand data?

No. Firsthand is not affiliated with the authors and claims no use, training or compatibility. Robot trajectories, teleoperation data, retargeted actions and action labels are not Firsthand deliverables.

How can I request a custom egocentric dataset for humanoid learning?

Request a custom dataset with your tasks, environments and signals. Collection is fast and on demand, with a first validated batch in a median of 48 hours where coverage is deep.