What Is Hi Robot?

Hi Robot is introduced in arXiv 2502.19417 (ICML 2025) by Lucy Xiaoyang Shi and co-authors, with an official project page. Read: abstract, full text and project page; no official code repository is linked in the text read.

Hi Robot is third-party research. Firsthand is not affiliated with its authors and claims no use, training or compatibility. Firsthand captures first-person human video, not robot data, fast, on demand and to a written spec: 21 keypoints per hand with a visibility flag, wrist trajectories, action segments and one task instruction per episode.

The high-level policy is a vision-language model: images and an open-ended prompt in, an intermediate language command out, such as pick up one piece of lettuce. The low-level policy is a VLA that turns the command, images and robot state into action chunks. The high level reruns after one second or on new user input, by text or speech, and can add a spoken reply. Seeing images lets it ground feedback such as “that’s not trash”.

Teleoperated demonstrations are segmented into short skills, generally one to three seconds. A separate vision-language model writes synthetic user prompts and robot replies for those moments. The high level trains on synthetic plus labelled data with next-token cross-entropy; the low level trains with a flow-matching objective. Both start from PaliGemma-3B.

Three platforms: a UR5e single arm, a bimanual ARX and a mobile ARX. Three tasks: table bussing, sandwich making and grocery shopping, with 20 trials per task per method. The authors report over 40% higher average instruction accuracy than a GPT-4o high-level baseline; these are their own figures.

Hi Robot is a learned form of task decomposition: the high level splits a complex prompt into short commands. Its low-level π0 is a language-conditioned policy, and the paper says others could be used. It says standard VLAs are typically limited to simple, atomic commands. A named limit: the two levels are trained separately.

Hi Robot’s low level needs robot demonstrations with actions, which Firsthand does not deliver; egocentric human video as Hi Robot training data is Not stated in the sources read. Firsthand captures human first-person video with one task instruction per episode and action segments; no compatibility is claimed. Put tasks and signals in a written spec and confirm it on a Spec Pilot. Median from signed spec to first validated batch: 48 hours where coverage is deep, three to four weeks where thin. Formats: RLDS, LeRobot v2.1, WebDataset, HDF5.

What is Hi Robot?

A hierarchical VLA system from arXiv 2502.19417: a high-level VLM turns complex prompts into commands for a low-level VLA.

How does the hierarchy work?

The high level outputs a language command; the low level turns it into actions.

How is Hi Robot trained?

Skill-segmented teleoperated demonstrations plus VLM-generated synthetic prompts for the high level.

Is Hi Robot open source?

Not stated in the sources read. The paper and project page text give no code repository or licence.

Was Hi Robot trained on egocentric human video?

Not stated in the sources read. The paper describes teleoperated robot demonstrations and synthetic prompts.

Does Firsthand use Hi Robot, and can I request a custom dataset?

Firsthand is not affiliated with the Hi Robot authors and claims no use, training or compatibility. You can request a custom dataset, confirmed first on a Spec Pilot.