What Is DexGraspVLA?

DexGraspVLA is introduced in DexGraspVLA: A Vision-Language-Action Framework Towards General Dexterous Grasping (arXiv 2502.20900), from authors at Peking University, the PKU-PsiBot Joint Lab, HKUST (Guangzhou) and the University of Pennsylvania, with an official project page. The authors argue that dexterous grasping research usually rests on restrictive assumptions, such as single-object settings or limited environments, and generalises poorly. Their answer is a framework for grasping diverse objects in cluttered scenes from free-form language prompts such as clear the table.

DexGraspVLA is third-party research. Firsthand is not part of it or affiliated with its authors, and makes no claim that any model was trained on Firsthand data. Per the paper, its training data is robot demonstrations recorded by physically guiding a robot arm and hand, not human egocentric video. What the site states is custom egocentric collection to a written spec, fast and on demand, with 3D hand pose at 21 joints per hand, calibrated stereo depth and 6-DoF IMUs at 200 Hz. Mapping that to DexGraspVLA’s inputs, which are robot joint angles and object masks, is Not stated.

The high-level planner is an off-the-shelf pre-trained vision-language model from the Qwen family. Given a prompt such as clear the table, it proposes one grasping instruction at a time, such as grasp the cookie, and marks the target object with a bounding box in the head camera image.

The authors call the bounding box a domain-invariant signal: language and scenes vary from user to user, but a box is a consistent format for locating an object. The planner also monitors each attempt, resets the robot afterwards and proposes the next instruction until the prompt is complete.

The box is turned into an object mask by SAM and tracked through the grasp by Cutie, a video segmentation model. Frozen DINOv2 encoders extract features from a head camera and a wrist camera, and the mask, features and robot state are projected into one feature sequence.

A diffusion transformer then predicts a chunk of future actions, and only the first few are executed before the next prediction. The paper’s central idea is that foundation models convert varied inputs into representations that stay similar across environments, so imitation learning does not break on a new table, light or object.

The authors collected 2,094 successful demonstrations in cluttered scenes with 36 household objects, each lasting about 3.5 seconds. The project page puts this at roughly two hours of data. Demonstrations were recorded by kinesthetic teaching, where an operator physically guides the robot, at a control frequency of 20 Hz.

Each episode stores head and wrist camera images, robot proprioception, the object mask and the action. The nonprehensile extension used a separate set of 1,029 human demonstrations. These are robot trajectories, and the paper does not describe egocentric human video as part of the data.

The authors report a 90.8% aggregated success rate on cluttered grasping across 1,287 unseen object, lighting and background combinations, tested in a different room from where the demonstrations were recorded. A grasp counts as successful if the object is held 10 cm above the table for 20 seconds.

They also report 98.6% on a single-object benchmark, 89.6% on long-horizon prompts and 84.7% on a nonprehensile task where the robot first pushes a flat object to the table edge. These are the authors’ numbers on their own robot, objects and test protocol, and Firsthand draws no comparison from them.

The conclusion says the framework does not yet address functional grasping and subsequent manipulation, and does not use tactile sensing. The authors list a task-oriented controller with tactile feedback as future work.

The paper evaluates one robot setup, a single arm and hand with a wrist camera and a head camera, so cross-embodiment transfer is not claimed here.

The site’s GraspVLA entry covers a grasping model pretrained on a billion-frame synthetic dataset, and its DexVLA entry covers a model with a plug-in diffusion action expert for general robot control. DexGraspVLA instead pairs a planner with a controller built on frozen foundation-model features and trained on real demonstrations.

All three are third-party work, and Firsthand ranks none of them.

When a program needs real human egocentric demonstrations of grasping and manipulation tasks, environments or objects that a paper’s dataset does not cover, specified in writing. Firsthand does not build or train models.

Confirm the spec on a Spec Pilot, a small fast first batch. Where coverage is deep, the median from signed spec to first validated batch is 48 hours; where it is thin, three to four weeks. Billing is per validated hour.

What is DexGraspVLA?

A hierarchical vision-language-action framework for language-guided dexterous grasping, from arXiv 2502.20900. A pre-trained VLM plans and marks targets with bounding boxes, and a diffusion-based controller produces the actions.

How does DexGraspVLA generalise to unseen scenes?

Per the paper, foundation models turn varied language and visual inputs into domain-invariant representations, such as bounding boxes, object masks and frozen DINOv2 features, so imitation learning is applied on inputs that look similar across environments.

What data was DexGraspVLA trained on?

The paper states 2,094 successful demonstrations in cluttered scenes with 36 household objects, about 3.5 seconds each, recorded by kinesthetic teaching. The project page puts this at roughly two hours.

What success rate did the authors report?

They report 90.8% aggregated over 1,287 unseen object, lighting and background combinations on their own setup. Firsthand makes no comparison from that number.

Does Firsthand data train DexGraspVLA?

No claim is made. Firsthand is not affiliated with its authors, and does not state that any model was trained on Firsthand data.

Can Firsthand collect grasping data to a custom spec?

The site states custom egocentric collection to a written spec, fast and on demand, with 21-joint 3D hand pose, calibrated stereo depth and 200 Hz IMUs, confirmed on a Spec Pilot.