Glossary
What Is GraphIRL (Graph Inverse Reinforcement Learning from Diverse Videos)?
Last reviewed: November 2026
Short answer
GraphIRL (Graph Inverse Reinforcement Learning) is the self-supervised method in arXiv 2207.14299. It builds an object-centric graph from each video frame, learns an embedding that captures task progression through temporal matching, and turns distance to a goal into a reward for reinforcement learning. It is third-party research and Firsthand is not affiliated.
01
What is GraphIRL, and what problem does it address?
GraphIRL is named in Graph Inverse Reinforcement Learning from Diverse Videos (arXiv 2207.14299) by Sateesh Kumar, Jonathan Zamora, Nicklas Hansen, Rishabh Jangir and Xiaolong Wang of UC San Diego; the project page lists the Conference on Robot Learning (CoRL) 2022. The authors argue that inverse reinforcement learning from third-person videos removes the need for manual reward design, but that most prior work trains on a restricted domain of videos. Their insight is that a task can be described by entity interactions that form a graph, and that this abstraction removes irrelevant information such as textures, giving more robust reward functions.
02
Where does Firsthand fit next to GraphIRL?
GraphIRL is third-party research. Firsthand is not affiliated with the authors, and no named model, GraphIRL included, was trained on Firsthand data. Firsthand does not claim to use X-MAGICAL, an xArm robot, Soft Actor-Critic or any tool the paper names. The paper works with third-person video; the site states custom egocentric collection, fast and on demand: first-person human video with calibrated stereo depth, 3D hand pose at 21 joints per hand and 6-DoF IMUs at 200 Hz, with one task instruction per episode. Object graphs, embeddings, reward functions and trained policies are not Firsthand deliverables.
03
How does GraphIRL turn videos into a reward?
Per the paper, the method takes a dataset of video demonstrations of a task and assumes no access to action information. An off-the-shelf detector extracts object bounding boxes from each frame, and each object becomes a node in a graph. A Spatial Interaction Encoder, which the authors build on Interaction Networks, models how objects relate to one another. The encoder is trained with a Temporal Cycle Consistency (TCC) loss, so the graph embeddings align across videos in time.
The reward is then the negative scaled Euclidean distance between the current observation and a representative goal frame in that embedding space. The authors state that any standard reinforcement learning algorithm can use it; they use Soft Actor-Critic (SAC) and no environment reward.
04
What videos did the GraphIRL authors use?
The paper evaluates on X-MAGICAL, a simulated cross-embodiment benchmark, in Standard and Diverse variants, where Diverse also randomizes visual appearance. For robot manipulation with an xArm, it uses real-world videos of humans doing Reach, Push and Peg in Box. The authors report collecting 256 Reach and 162 Peg in Box demonstrations and using 198 Push videos from Schmeckpeper et al. Objects are found with a hand-and-object detector the paper describes as trained on a large dataset collected from YouTube. The project page links the code and a dataset folder; licences are Not stated in the sources read.
05
What did the authors report, and what are the limits of this page?
The abstract reports significant gains in robustness to diverse video demonstrations over previous approaches, and better results than manual reward design on a real robot pushing task. The second table repeats the authors’ real-robot success rates, measured over 15 trials per method. Firsthand has not verified them. The authors also state a limitation: collecting demonstrations still requires human labor, and removing it entirely remains an open problem.
06
How does GraphIRL compare with related entries on this site?
The third table sets GraphIRL beside four site entries. The paper frames GraphIRL as an extension of XIRL to diverse videos, replacing image features with a graph abstraction while keeping temporal cycle-consistency.
07
What would a GraphIRL-style approach need from custom collection?
Per the paper, it needs demonstration videos of one task, an object detector, a goal frame, and a simulator or robot with a reinforcement learning stack, all the buyer’s own. The paper uses third-person video and does not describe egocentric capture. Firsthand covers the human video side only, as custom collection at scale through a network of thousands of capture providers, for a consistent, recurring data flow to labs, confirmed on a Spec Pilot.
08
GraphIRL at a glance, as stated in the paper, project page and official code
| Item | What the sources state |
|---|---|
| Paper | Graph Inverse Reinforcement Learning from Diverse Videos (arXiv 2207.14299) |
| Venue | Conference on Robot Learning (CoRL) 2022, per the project page |
| Input | Diverse third-person video demonstrations; no action information assumed |
| Representation | Object bounding boxes from an off-the-shelf detector, built into a graph with one node per object |
| Learning signal | Self-supervised; Temporal Cycle Consistency loss on a Spatial Interaction Encoder |
| Reward | Negative scaled distance to a goal frame in the learned embedding space |
| Policy learning | Soft Actor-Critic; no environment reward used |
| Evaluation | X-MAGICAL (simulated) and xArm manipulation tasks in simulation and on a real robot |
| Licences | Not stated in the sources read |
| Firsthand relationship | None |
10
Real-robot success rates reported by the authors (15 trials per method; Firsthand has not verified them)
| Task | XIRL | Environment reward | GraphIRL |
|---|---|---|---|
| Push | 0.27 | 0.47 | 0.60 |
| Reach | 0.26 | 0.93 | 0.86 |
| Peg in Box | 0.06 | 0.60 | 0.53 |
11
GraphIRL beside related entries on the site
| Site entry | What the entry states | How GraphIRL differs (sources read) |
|---|---|---|
| XIRL | Learns an embedding with temporal cycle-consistency and uses distance to a goal state as the reward | Applies cycle-consistency to an object graph instead of image features, to handle diverse videos |
| DVD | Learns a robot reward by comparing a robot video with a human demo video | Learns a goal-distance reward from many demonstration videos, with no robot video comparison |
| HOLD | Learns a goal-image reward from unlabeled human videos via time regression or time-contrastive embeddings | Learns from detected objects and their interactions with temporal cycle-consistency |
| HuDOR | Fine-tunes a robot hand policy online, using rewards from object points tracked in one human video | Learns its reward from many videos and trains in simulation before real-robot deployment |
12
Frequently asked questions
- What is GraphIRL?
- Per arXiv 2207.14299, GraphIRL is a self-supervised method that learns a reward function from diverse third-person videos using an object-centric graph abstraction. It is third-party research.
- Why use a graph instead of raw images?
- The authors say a task can be described by entity interactions that form a graph, and that this abstraction removes irrelevant information such as textures, giving more robust reward functions.
- Does GraphIRL need robot actions or an environment reward?
- No. The paper states it does not assume access to action information in the demonstrations and does not use any environment reward, relying solely on the learned reward.
- What data did GraphIRL use?
- Simulated X-MAGICAL demonstrations and real-world videos of humans doing Reach, Push and Peg in Box, per the paper. Dataset licences are Not stated in the sources read.
- Was GraphIRL trained on Firsthand data?
- No. Firsthand is not affiliated with the authors, and no named model was trained on Firsthand data. Reward functions and embeddings are not Firsthand deliverables.
- How can I request a fast, on-demand custom egocentric dataset for this research?
- Request a custom dataset with your tasks and signals. Collection is fast and on demand, with a first validated batch in a median of 48 hours where coverage is deep.
Related terms
- XIRL (Cross-embodiment Inverse Reinforcement Learning)
- DVD (Learning Generalizable Robotic Reward Functions from In-the-Wild Human Videos)
- Learning Reward Functions for Robotic Manipulation by Observing Humans
- HuDOR (Bridging the Human to Robot Dexterity Gap through Object-Oriented Rewards)
- Cross-embodiment learning
- Embodiment gap
Sources
- Graph Inverse Reinforcement Learning from Diverse Videos (arXiv 2207.14299)
- GraphIRL project page (official, linked from the paper)
- GraphIRL official code (SateeshKumar21/graph-inverse-rl)
- XIRL: Cross-embodiment Inverse Reinforcement Learning (arXiv 2106.03911)
- Temporal Cycle-Consistency Learning (arXiv 1904.07846)
- Interaction Networks for Learning about Objects, Relations and Physics (arXiv 1612.00222)
- Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actor (arXiv 1801.01290)
- Reinforcement Learning with Videos: Combining Offline Observations with Interaction (arXiv 2011.06507)