What Is RoboMonkey?

RoboMonkey is introduced in RoboMonkey: Scaling Test-Time Sampling and Verification for Vision-Language-Action Models (arXiv 2506.17811; submitted 21 June 2025, last revised 7 July 2025, cs.RO) by Jacky Kwok, Christopher Agia, Rohan Sinha, Matt Foutter, Shulu Li, Ion Stoica, Azalia Mirhoseini and Marco Pavone, from Stanford University, UC Berkeley and NVIDIA Research. The abstract names the framework RoboMonkey, and the project page BibTeX lists the Proceedings of The 9th Conference on Robot Learning. Read: the arXiv abstract page, the full paper text including the appendices, the official project page and the GitHub README.

RoboMonkey is third-party research. Firsthand is not part of it or affiliated with its authors, and makes no claim that any model was trained on Firsthand data. What the site states is custom egocentric collection of real human demonstrations, fast and on demand, to a written spec: 3D hand pose at 21 joints per hand, calibrated stereo depth and 6-DoF IMUs at 200 Hz, delivered in RLDS, LeRobot format or WebDataset with a buyer-owned commercial license. The only language data the site states is one task instruction per episode. Mapping any delivery to RoboMonkey’s inputs or action format is Not stated.

Per the abstract, VLA models show strong visuomotor control, yet robustness in unstructured real-world environments remains a challenge. The introduction lists failure modes such as imprecise grasping, task progression failure and collision with surrounding objects.

The authors note that effort has shifted from pre-training to post-training, while less attention has gone to scaling compute at deployment, since VLAs typically generate a single action chunk per observation. They ask whether repeatedly sampling and verifying actions at deployment can improve precision and robustness.

At each timestep it samples a small set of actions from a VLA with a positive temperature, sets the gripper state by majority voting, and fits a Gaussian to the translation and rotation components. It then draws new candidate actions from that proposal distribution, and a VLM-based action verifier scores them. The highest-scoring action is executed.

In the authors’ real-world setup they sample 5 initial actions from OpenVLA at temperature 1.0 and draw 16 candidates from the Gaussian.

With an oracle verifier that always picks the lowest-error action, the authors find that action error falls as more actions are generated, following an approximate power law across CogACT, Octo, OpenVLA and SpatialVLA. For OpenVLA the RMSE decreases by 59.3% when sampling 10,000 actions. To their knowledge, this is the first characterization of inference-time scaling laws for VLAs.

An oracle verifier is an idealization. The learned verifier is the practical part of the framework.

The authors generate synthetic action preferences from an existing demonstration dataset without human annotation. A reference policy samples candidate actions, clustering reduces them, and pairs are ranked by RMSE against the ground-truth action. Training uses a Bradley-Terry loss with a margin term, on a LLaVA-7B backbone with a reward head.

The main training data is Bridge V2, which the paper describes as over 40,000 real-world trajectories on a WidowX robot in 24 environments, turned into 20 million synthetic comparisons.

The abstract reports a 25% absolute improvement on out-of-distribution tasks and 9% on in-distribution tasks, and a 7% gain from fine-tuning both the VLA and the verifier on a new robot setup. In SIMPLER the paper reports a 47.5% average success rate, 9% above OpenVLA. In real-world tests on a WidowX-250 S the average was 60%, against 35% for OpenVLA and 30% for V-GPS.

The authors report that sampling and verifying 16 candidate actions takes about 650 ms (1.5 Hz) on a single NVIDIA H100. These are their results on their own setups; Firsthand does not reproduce them or rank models.

The site describes VLA-RL as an online reinforcement learning framework for pretrained autoregressive VLAs, and post-training as the final stage that specializes a model to one deployment task, robot or environment. The RoboMonkey paper positions its work beyond both: it pairs existing VLAs with a separate verifier and adds compute at deployment.

The paper also describes work that uses VLMs to generate high-level action plans, naming Hi Robot, in contrast to RoboMonkey, which uses a VLM to verify low-level actions. A direct relationship to VLA-RL is Not stated in the sources read.

The verifier is trained on robot demonstrations, and its example prompt includes the current observation from the robot’s wrist-mounted camera and the task instruction. On the site, the only language data is one task instruction per episode, and Firsthand captures human egocentric video rather than robot data.

The authors list computational overhead, saying the framework may be less suited to high-frequency control, a synthetic dataset limited to 20 million comparisons on Bridge V2, and evaluation on two arms, WidowX 250S and Franka.

The arXiv abstract gives a 9% in-distribution gain, while the abstract in the project page BibTeX says 8%, so check which version a figure refers to. The README mentions Hugging Face model checkpoints, but that page was not read, so its contents are Not stated here.

When a program needs real human egocentric demonstrations of tasks, environments or objects that public data does not cover, specified in writing. Firsthand does not build or train models.

Confirm the spec on a Spec Pilot, a small fast first batch. Where coverage is deep, the median from signed spec to first validated batch is 48 hours; where it is thin, three to four weeks. Billing is per validated hour.

What is RoboMonkey?

A published test-time scaling framework (arXiv 2506.17811) for vision-language-action models. It samples a few actions from a VLA, builds a proposal distribution, and lets a VLM-based verifier choose the best action. It is third-party research.

What is test-time sampling and verification for VLAs?

Per the paper, it means spending extra compute at deployment: generate several candidate actions for one observation and instruction, then verify them and execute the best. The authors report that action error falls as more actions are sampled, assuming an oracle verifier.

What is a VLA action verifier?

In the paper, a VLM-based model with a LLaVA-7B backbone and a reward head that scores a candidate action given the observation and instruction. The highest-scoring action is executed.

How much does RoboMonkey improve results?

The authors report a 25% absolute improvement on out-of-distribution tasks and 9% on in-distribution tasks, and a 7% gain from fine-tuning both the VLA and the verifier on LIBERO-Long. These are their results; Firsthand ranks no models.

Is RoboMonkey open source?

The official GitHub repository, robomonkey-vla/RoboMonkey, holds the code, and GitHub lists an MIT License. Its README also mentions Hugging Face models, which were not read here.

Does Firsthand data train RoboMonkey?

No claim is made. Firsthand is not affiliated with its authors, and the site does not state that any model was trained on Firsthand data. The only language data the site states is one task instruction per episode.