Now recruiting capture teams worldwide. Get paid to record first-person video of real work.

Apply to capture

Glossary

What Is Learning Reward Functions for Robotic Manipulation by Observing Humans?

HOLD learns a goal-image reward for robots from unlabeled human videos, via time regression or time-contrastive embeddings. Third-party; Firsthand is not affiliated.

Last reviewed: November 2026

Short answer

HOLD (Human Offline Learned Distances) is the method in “Learning Reward Functions for Robotic Manipulation by Observing Humans” (arXiv 2211.09019). From unlabeled human videos it learns a distance to a goal image, using temporal regression or time-contrastive embeddings, that serves as a dense robot reward. Firsthand is not affiliated.

01

What is HOLD, and what problem does it address?

HOLD is named in arXiv 2211.09019 by Minttu Alakuijala, Gabriel Dulac-Arnold, Julien Mairal, Jean Ponce and Cordelia Schmid (Inria, Google, Univ. Grenoble Alpes, Ecole Normale Supérieure); the project page lists ICRA 2023. The authors note that human and robot arms differ in morphology and that human actions are not observed unless specially recorded. HOLD therefore learns a task-agnostic reward from video alone, with no robot data and no task-specific human demonstrations.

02

Where does Firsthand fit next to HOLD?

HOLD is third-party research. Firsthand is not affiliated with the authors, and no named model, HOLD included, was trained on Firsthand data. Firsthand does not claim to use Something-Something v2, the Sawyer arm or Soft Actor-Critic. The site states custom egocentric collection, fast and on demand: first-person human video with calibrated stereo depth, 3D hand pose at 21 joints per hand and 6-DoF IMUs at 200 Hz, with one task instruction per episode. Reward functions, goal images, robot data and trained policies are not Firsthand deliverables.

03

How does HOLD learn time-contrastive reward functions?

Per the paper, the goal is a distance between the current image and a goal image that tracks how many time steps an expert would need to reach the goal. The authors learn it by direct regression of that time gap (HOLD-R) or with a single-view time-contrastive embedding (HOLD-C), where frames close in time are pulled together and distant frames are pushed apart. The reward is the negated distance to the goal image, added to a sparse success reward.

04

What human videos did the HOLD authors use?

The authors train on Something-Something v2, which the paper describes as 220,847 crowd-sourced clips of 174 action classes, about 4 seconds long and mostly filmed on handheld devices. They say they do not use the class labels for the distance objectives, with one footnoted exception for pretraining a HOLD-R variant. The dataset licence is Not stated in the sources read.

05

How is the learned reward used to train a robot policy?

The authors train one policy per robot task with Soft Actor-Critic on image input, evaluated over 20 episodes, on simulated tasks including Pushing, Drawer Opening and Close Drawer. No robot data from the target environment is used to learn the reward.

06

What did the authors report, and what are the limits?

The authors report faster training than sparse reward alone (second table); Firsthand has not verified it. They state the distances are most accurate for short-horizon tasks and that Drawer needs an intermediate goal image.

07

How does HOLD compare with related entries on this site?

The third table sets HOLD beside four site entries. HOLD defines the task with a goal image and needs no robot demonstrations or task labels.

08

What would a HOLD-style approach need from custom collection?

Per the paper, it needs human manipulation videos, a goal image per robot task, a simulator or robot and a reinforcement learning stack, all the buyer’s own. The paper does not describe egocentric capture. Firsthand covers the human video side only, as custom collection on demand, confirmed on a Spec Pilot.

09

HOLD at a glance, as stated in the paper and project page

HOLD at a glance, as stated in the paper and project page
ItemWhat the sources state
PaperLearning Reward Functions for Robotic Manipulation by Observing Humans (arXiv 2211.09019)
Method nameHuman Offline Learned Distances (HOLD)
VenueICRA 2023, per the project page
InputUnlabeled human videos (Something-Something v2); no robot data
Two variantsHOLD-R (temporal regression) and HOLD-C (time-contrastive embeddings)
Task specificationA goal image
Policy learningSoft Actor-Critic, simulated robot arm
Dataset licenceNot stated in the sources read
Firsthand relationshipNone

11

Results reported by the HOLD authors (simulation; not verified by Firsthand)

Results reported by the HOLD authors (simulation; not verified by Firsthand)
ItemWhat the authors report
Sample efficiencyUp to 18x faster training on 5 simulated tasks when providing shaped rewards in sparse-reward tasks
PushingSparse reward alone needed 10x more samples to reach the return of HOLD-C
Drawer OpeningSparse reward alone needed more than 18x more samples to reach the return of HOLD-C
Close DrawerHOLD solves the task without the sparse reward

12

HOLD beside related entries on the site

HOLD beside related entries on the site
Site entryWhat the entry statesHow HOLD differs (sources read)
DVDClassifies whether two videos show the same task, using task labels and a small robot datasetUses a goal image, with no task labels or robot demonstrations; the paper evaluates on DVD tasks
VIPPre-trains a representation on Ego4D human video; distance to a goal image gives a rewardTrains its own distance on Something-Something v2; the paper cites VIP as concurrent work
R3MPre-trains a frozen representation on Ego4D with time-contrastive learningThe paper uses R3M distances as a baseline and says HOLD-C needs no language descriptions
WHIRLTurns one human video into a robot policy, improved by real-world trialsLearns a reusable reward from many videos

13

Frequently asked questions

What is HOLD?
Per arXiv 2211.09019, HOLD (Human Offline Learned Distances) learns a task-agnostic, goal-image-conditioned reward for robotic manipulation from unlabeled human videos. It is third-party research.
What are time-contrastive reward functions?
HOLD-C embeds single frames so that frames close in time are near each other. Distance to a goal image in that space becomes the dense reward.
What human data did HOLD train on?
Something-Something v2, per the paper. The dataset licence is Not stated in the sources read.
What did the authors report for HOLD?
They report up to 18x faster training on 5 simulated tasks than sparse reward alone. Firsthand has not verified these figures.
Was HOLD trained on Firsthand data?
No. Firsthand is not affiliated with the authors, and no named model was trained on Firsthand data. Reward functions are not Firsthand deliverables.
How can I request a fast, on-demand custom egocentric dataset for this research?
Request a custom dataset with your tasks and signals. Collection is fast and on demand, with a first validated batch in a median of 48 hours where coverage is deep.