What Is GeoVLA?

GeoVLA is introduced in GeoVLA: Empowering 3D Representations in Vision-Language-Action Models (arXiv 2508.09071) by Lin Sun, Bin Xie, Yingfei Liu, Hao Shi, Tiancai Wang and Jiale Cao of Tianjin University, Dexmal and Tsinghua University, submitted on 12 August 2025, with an official project page and an official GitHub repository whose README lists IROS 2026. Read: the arXiv abstract page, the full paper text, the project page and the repository README. Check the version you cite.

GeoVLA is third-party research. Firsthand is not part of it or affiliated with its authors, and makes no claim that any model was trained on Firsthand data. Firsthand captures human egocentric video, not robot data. What the site states is custom collection of real human demonstrations, fast and on demand, to a written spec: depth derived from a calibrated stereo rig in metric millimetres, 3D hand pose at 21 joints per hand and 6-DoF IMUs at 200 Hz, delivered in RLDS, LeRobot format or WebDataset with a buyer-owned commercial license. Point-cloud file delivery is Not stated, and the only language data the site states is one task instruction per episode.

Per the abstract, current VLA models mainly rely on 2D visual inputs, neglecting the rich geometric information in the 3D physical world, which limits their spatial awareness and adaptability. The authors say earlier ways of adding 3D, such as 3D positional encodings inside the vision-language model, disrupt the alignment between the visual encoder and the language model, and that restoring it often requires large-scale 3D embodied instruction-tuning data.

They also say that injecting 3D features into a frozen action expert, as PointVLA does, hinders adaptation to the new modality. They name end-to-end integration of 3D into VLA models as the key challenge.

GeoVLA runs two parallel branches. A pre-trained vision-language model, Prismatic-7B initialised from OpenVLA weights, turns the RGB image and the instruction into fused vision-language embeddings. Separately, the depth map is projected into a point cloud using camera parameters and encoded by a Point Embedding Network. The two sets of embeddings are concatenated and passed to a 3D-enhanced Action Expert. The authors say this dual-path design preserves the pre-trained knowledge of the vision-language model.

The Point Embedding Network expresses the point cloud in the end-effector coordinate system, so the robot’s end-effector position is the origin. It has two paths: a lightweight CNN with large-kernel convolutions that makes geometric tokens, and a rotary positional encoding path. The token at the origin is chosen as an anchor, and only that token is passed on.

The action expert is a diffusion transformer with a mixture-of-experts feed-forward layer. The authors use static routing and randomly drop one modality during training, because dynamic routing biased the model toward the vision-language branch. Actions are chunks of 16 steps, each a relative translation, a relative rotation and an absolute gripper command, sampled with DDIM at inference.

These are the authors’ results; Firsthand does not reproduce them and ranks no models. On LIBERO, GeoVLA averages 97.7%, against 95.3% for OpenVLA-OFT and 93.2% for CogACT, which the authors reproduced. On ManiSkill2 it averages 77%, against 69% for CogACT and 66% for Dita.

In real-world tests on a WidowX-250s arm with a RealSense-435i depth camera about 0.8 m away in a third-person view, the authors report an 86.3% average over eight tasks of 10 trials each, against 57.5% for π₀ and 76.3% for CogACT. When the camera shifted 45 degrees on Stack Block, they report 70% for GeoVLA, 0% for CogACT and 30% for π₀.

The paper groups 3D-VLA, 3D-CAVLA and SpatialVLA as methods that put 3D features or position embeddings into the vision-language model, which it says often disrupts alignment with the language model. It places PointVLA with methods that inject 3D into the action head, and says its zero-initialised injection into a frozen expert hinders adaptation. GeoVLA instead encodes 3D separately and fuses it in a trained action expert.

The site’s entries describe 3D-VLA as a generative world model that predicts goal images and point clouds, PointVLA as point cloud inputs for a pre-trained VLA without retraining, and SpatialVLA as Ego3D Position Encoding with Adaptive Action Grids.

GeoVLA consumes point clouds built from depth maps and camera parameters, and its repository accepts either raw depth or a precomputed point cloud. On the stereo capture pages, the site states depth derived from a head-mounted calibrated stereo rig, not a standalone depth sensor, in metric millimetres, with intrinsics and extrinsics in each episode’s calibration group.

The site’s point-cloud entry says a buyer can compute a cloud from the delivered depth and calibration; Firsthand point-cloud file delivery is Not stated. The paper’s cloud is centred on the robot’s end-effector and the repository’s custom-data format expects delta end-effector actions, while Firsthand does not deliver robot actions.

The paper text read has no separate limitations section. The real-world setup it states is one WidowX-250s arm with one third-person RealSense-435i, each task evaluated over 10 trials. Whether GeoVLA works with human first-person video, stereo-derived depth or head-mounted views is Not stated, as are other depth sensors and hosted pretrained weights in the README read. The arXiv page lists no venue while the README lists IROS 2026, so check the version you cite.

When a program needs real human egocentric demonstrations that public data does not cover, specified in writing. Firsthand does not build or train models. For depth-based research, confirm the depth encoding, the calibration fields and the single task instruction per episode on a Spec Pilot, a small fast first batch. The median from signed spec to first validated batch is 48 hours where coverage is deep, three to four weeks where thin. Billing is per validated hour.

What is GeoVLA?

A published vision-language-action framework (arXiv 2508.09071) that adds 3D geometry to robot manipulation by converting depth maps into point clouds and fusing them with image-language embeddings in a diffusion action expert. It is third-party research.

How does GeoVLA use point clouds?

Per the paper, the depth map is projected into a point cloud, expressed around the robot’s end-effector and encoded by a Point Embedding Network. That 3D embedding is concatenated with the vision-language embeddings and passed to the action expert.

What did the authors report?

Their figures are 97.7% on LIBERO, 77% on ManiSkill2 and an 86.3% average over eight real-world tasks of 10 trials each. These are the authors’ results; Firsthand does not reproduce them.

Is GeoVLA open source?

The official GitHub repository carries an MIT License and provides training, fine-tuning and deployment code. Hosted pretrained weights are Not stated in the README read.

How is GeoVLA different from 3D-VLA, PointVLA and SpatialVLA?

Per the paper, GeoVLA encodes 3D in a separate branch and fuses it in a trained action expert, while the others put 3D into the vision-language model or into a frozen action expert. Any ranking between them is Not stated here.

Does Firsthand use GeoVLA, and can I request a custom dataset?

Firsthand is not affiliated with the GeoVLA authors and claims no use, training or compatibility. The site states depth derived from a calibrated stereo rig; point-cloud files and robot actions are Not stated. You can request a custom dataset, confirmed first on a Spec Pilot.