Glossary
What Is GR-2?
Last reviewed: October 2026
Short answer
GR-2 is a generative video-language-action model for robot manipulation from ByteDance Research, introduced in arXiv 2410.06158. It is first pre-trained on a vast number of Internet videos to capture world dynamics, then fine-tuned on robot trajectories for both video generation and action prediction. Firsthand is not part of it.
01
What is GR-2?
GR-2 is introduced in arXiv 2410.06158, with an official project page from ByteDance Research. The abstract describes a generalist robot agent for versatile and generalizable manipulation that learns from Internet video before it learns from robot data. Read: the abstract and project page only.
02
Where does Firsthand fit next to GR-2?
GR-2 is third-party research. Firsthand is not affiliated with its authors and claims no use, training or compatibility with it. Firsthand captures first-person human video, not robot data, fast, on demand and to a written spec: 21 keypoints per hand with a visibility flag, wrist trajectories and action segments, plus one task instruction per episode.
03
How is GR-2 built?
The project page calls GR-2 a language-conditioned visual manipulation policy. A novel model architecture transfers pre-training knowledge to fine-tuning “in a lossless way” and handles multiple camera views. Actions are generated in Cartesian space and tracked by a Whole-Body Control algorithm with trajectory optimization. Parameter counts are Not stated in the sources read.
04
How is GR-2 pretrained on video and then fine-tuned on robot data?
Stage one is video generative pre-training on a curated video dataset of diverse daily human activities. The abstract gives 38 million video clips and over 50 billion tokens. Stage two fine-tunes on robot trajectories to predict actions and video together. Robot dataset size is Not stated in the sources read.
05
Why does generative video pretraining matter for robot manipulation?
The project page says mastering video generation captures temporal dynamics and semantic information useful for policy learning, and that predicted future video can act as a planner from which an action trajectory is inferred. The abstract reports a 97.7% average success rate across more than 100 tasks, and the page says 105 tasks were learned from 50 trajectories per task. Firsthand draws no comparison from these results.
06
How does GR-2 compare with world models and OpenVLA?
GR-2 predicts future video and actions together, so it overlaps with a world model. OpenVLA predicts tokenized actions from robot data. Firsthand ranks none.
07
What does GR-2’s data description not cover?
The sources describe the pre-training videos only as daily human activities with a single camera view. Whether they are egocentric, and their hours, are Not stated in the sources read. Ego4d appears only as a validation set for video generation loss. Hand pose is not described.
08
What real-world egocentric video supports video-pretrained VLAs, and how do you get it custom-collected?
Firsthand captures human first-person video, not robot data. Write tasks, environments and signals into a spec and confirm them on a Spec Pilot. Median time to first validated batch: 48 hours where coverage is deep, three to four weeks where thin. Formats: RLDS, LeRobot v2.1, WebDataset, HDF5; mapping to GR-2’s layout is the buyer’s pipeline.
09
GR-2 components and two-stage training setup, as stated in the official sources
| Item | What the official sources state |
|---|---|
| Policy type | Language-conditioned visual manipulation policy |
| Stage 1 | Video generative pre-training on daily human activity video; 38 million clips, over 50 billion tokens (abstract) |
| Stage 2 | Fine-tuning on robot trajectories for action prediction and video generation together |
| Camera views | Pre-training video is single view; robot data often multi-view, which GR-2 handles |
| Action output | Cartesian-space trajectories tracked by Whole-Body Control |
| Parameter counts, code, weights | Not stated in the sources read |
| Firsthand relationship | None |
11
GR-2 beside a world model and OpenVLA, as each official source or site entry states
| Item | GR-2 (sources read) | World model (site entry) | OpenVLA (site entry) |
|---|---|---|---|
| Learns | Future video and actions together | How a scene changes in response to an action | Tokenized actions from image and language |
| Starting point | Video generation pre-training on Internet video | Dense frames paired with actions | Prismatic-7B vision-language model |
| Robot data | Robot trajectories; size Not stated | Action, pose or contact event paired with video | 970k Open X-Embodiment trajectories |
| Egocentric human video | Not described | One visual input among several | Not described |
12
What GR-2 consumes beside what the Firsthand site states
| Aspect | GR-2 (sources read) | What the Firsthand site states |
|---|---|---|
| Pre-training video | Single-view daily human activity video | Head-mounted egocentric video; suitability for GR-2 not claimed |
| Robot views | Often several camera views | Head-mounted video plus two wrist cameras |
| Actions | Cartesian trajectories from robot data | Robot actions Not stated; 21 keypoints per hand, visibility flag, wrist trajectories, action segments |
| Language | Language-conditioned; text format Not stated in the sources read | One task instruction per episode |
| File layout | Not stated in the sources read | RLDS, LeRobot v2.1, WebDataset, HDF5 |
| Compatibility | Not claimed | Not claimed |
13
Frequently asked questions
- What is GR-2?
- A generative video-language-action model for robot manipulation, pretrained on Internet video and fine-tuned on robot trajectories.
- How much video was GR-2 pretrained on?
- The abstract states 38 million video clips and over 50 billion tokens. Hours are Not stated in the sources read.
- Does GR-2 generate video?
- Yes. Per the project page it predicts future video alongside actions.
- Is GR-2 open source?
- The sources read link no code repository or weights. Availability and licence are Not stated in the sources read.
- Was GR-2 trained on egocentric video?
- Not described in the sources read.
- Does Firsthand use GR-2, and can I request a custom dataset?
- Firsthand is not affiliated with the GR-2 authors and claims no use, training or compatibility. You can request a custom dataset, confirmed first on a Spec Pilot.
Related terms