Glossary
What Is GRAPE?
Last reviewed: October 2026
Short answer
GRAPE (Generalizing Robot Policy via Preference Alignment) is a published method (arXiv 2411.19309) that aligns vision-language-action models at the trajectory level. It ranks sampled rollouts, both successful and failed, using costs from staged keypoint constraints, then applies trajectory-wise preference optimization. It is third-party research; Firsthand is not affiliated with its authors.
01
What is GRAPE, the GRAPE VLA method?
GRAPE is introduced in GRAPE: Generalizing Robot Policy via Preference Alignment (arXiv 2411.19309) by Zijian Zhang, Kaiyuan Zheng, Zhaorun Chen, Joel Jang, Yi Li, Siwei Han, Chaoqi Wang, Mingyu Ding, Dieter Fox and Huaxiu Yao, submitted on 28 November 2024 and revised on 4 February 2025. Read: the arXiv abstract, the paper text, the project page and the repository README. The README title also reads Guided-Reinforced Vision-Language-Action Preference Optimization and lists ICRA’26, so check the version you cite.
02
Where does Firsthand fit next to GRAPE?
GRAPE is third-party research. Firsthand is not affiliated with its authors and makes no claim that any model was trained on Firsthand data. Firsthand captures human egocentric video, not robot data: custom collection, fast and on demand, to a written spec, with depth derived from a calibrated stereo rig, 3D hand pose at 21 joints per hand and 6-DoF IMUs at 200 Hz, delivered in RLDS, LeRobot format or WebDataset under a buyer-owned commercial license. Preference labels, rankings and reward annotations are not Firsthand deliverables.
03
What problem does GRAPE address in robot policy generalization?
Per the abstract, VLA models generalize poorly to unseen tasks because they rely on behavior cloning exclusively from successful rollouts. Copying expert demonstrations collected under different settings also introduces distribution bias, which limits adaptability to objectives such as efficiency, safety and task completion.
04
How does GRAPE VLA trajectory-wise preference optimization work?
GRAPE fine-tunes a supervised VLA by comparing whole trajectories instead of single steps. Following direct preference optimization, it contrasts a chosen and a rejected trajectory that start from the same initial state, and the loss sums per-step log-likelihood ratios along each one. The authors say this models reward implicitly from both successful and failed trials.
05
How does GRAPE generate preferences without manual labels?
A vision-language model splits a task into stages and proposes keypoints, and a language model writes a cost function per stage, where lower cost means better compliance. Stage costs form an external reward, weighted with the policy’s own trajectory log-likelihood and a binary success indicator. Sampled trajectories are ranked by that reward, top and bottom ones are paired, and the loop repeats. Editing the cost functions retargets it to safety, efficiency or task completion.
06
What did the GRAPE authors report?
These are the authors’ results; Firsthand does not reproduce them and ranks no models. The abstract reports success-rate gains of 51.79% on in-domain tasks and 58.20% on unseen tasks, plus a 37.44% lower collision rate and 11.15% shorter rollout step-length when aligned to safety and efficiency.
Real-world evaluation was 300 executions across 30 tasks on a Franka arm with a Robotiq gripper. The authors report a 50.3% total average real-world success rate for GRAPE, against 39.3% for OpenVLA-DPO, 32.3% for OpenVLA-SFT and 5.7% for Octo-SFT.
07
How is GRAPE different from VLA-RL, SimpleVLA-RL and SafeVLA?
The three site entries cover reinforcement-learning methods. GRAPE instead optimizes preferences over ranked whole trajectories. Any ranking between them is Not stated here.
08
What data does GRAPE use, and what does Firsthand state?
In the paper, the preference data are the policy’s own sampled rollouts, ranked into chosen and rejected pairs on the same task, on top of robot data from a Franka arm. Firsthand states human egocentric capture with stereo-derived depth, 21-joint hand pose and one task instruction per episode, and no paired trajectories, rankings or reward fields.
09
What do the GRAPE sources not state?
The main text read has no separate limitations section. The README says preference generation is available in Simpler-Env, with real-world support to be provided later. Whether GRAPE works with human first-person video is Not stated.
10
When do you need fast, on-demand custom collection instead?
When a program needs real human egocentric demonstrations that public data does not cover. Firsthand does not build or train models. Confirm the fields on a Spec Pilot, a small fast first batch; the median from signed spec to first validated batch is 48 hours where coverage is deep, three to four weeks where thin.
11
GRAPE at a glance, as stated in the paper, project page and README
| Item | What the sources state |
|---|---|
| Paper | GRAPE: Generalizing Robot Policy via Preference Alignment (arXiv 2411.19309) |
| Core idea | Trajectory-wise preference optimization (TPO) on ranked rollouts |
| Preference source | Staged keypoint costs, self-evaluation and a success indicator |
| Backbone | OpenVLA with LoRA fine-tuning |
| Benchmarks | Simpler-Env and LIBERO, plus real-world tasks |
| Code | Official GitHub repository, MIT License, built on OpenVLA |
| Firsthand relationship | None |
13
Real-world success rates reported by the authors (Table 4 of the paper, in %)
| Method | In-domain success | Total average success |
|---|---|---|
| Octo-SFT | 20 | 5.7 |
| OpenVLA-SFT | 45 | 32.3 |
| OpenVLA-DPO | 50 | 39.3 |
| GRAPE | 67.5 | 50.3 |
14
GRAPE beside VLA-RL, SimpleVLA-RL and SafeVLA
| Item | GRAPE (sources read) | VLA-RL (site entry) | SimpleVLA-RL (site entry) | SafeVLA (site entry) |
|---|---|---|---|---|
| Learning signal | Preferences over ranked whole trajectories | Online reinforcement learning with a process reward model | Reinforcement learning with binary outcome rewards | Safe reinforcement learning with a cost constraint |
15
What GRAPE consumes beside what the Firsthand site states
| Aspect | GRAPE (sources read) | What the Firsthand site states |
|---|---|---|
| Preference data | Chosen and rejected rollout pairs from the same task and initial state | Not a deliverable |
| Reward or ranking | Generated automatically from costs, self-evaluation and success | Not a deliverable |
| Demonstrations | Robot data from a Franka arm and Robotiq gripper | Human egocentric video, not robot data |
| Language | A task instruction | One task instruction per episode |
| Delivery format | RLDS for custom data, per the README | RLDS, LeRobot format or WebDataset, buyer-owned commercial license |
16
Frequently asked questions
- What is GRAPE?
- A published method (arXiv 2411.19309) that aligns VLA robot policies to preferences at the trajectory level, using ranked successful and failed rollouts. It is third-party research.
- What is trajectory-wise preference optimization in GRAPE?
- Per the paper, a chosen and a rejected trajectory from the same initial state are contrasted, so the policy is aligned over whole trajectories rather than step by step.
- What did the authors report?
- Success-rate gains of 51.79% in-domain and 58.20% on unseen tasks in the abstract. These are the authors’ results; Firsthand does not reproduce them.
- Is GRAPE open source?
- The official GitHub repository carries an MIT License and is built on OpenVLA.
- Does Firsthand use GRAPE, and can I request a custom dataset?
- Firsthand is not affiliated with the GRAPE authors and claims no use, training or compatibility. Preference labels, rankings and rewards are not Firsthand deliverables. You can request a custom dataset.