Glossary
What Is InstructVLA?
Last reviewed: October 2026
Short answer
InstructVLA is a published end-to-end vision-language-action (VLA) model, introduced in arXiv 2507.17520. Its training paradigm, Vision-Language-Action Instruction Tuning (VLA-IT), uses mixture-of-experts adaptation on standard vision-language corpora and a curated 650K-sample VLA-IT dataset, so one model keeps flexible reasoning while generating manipulation actions. Firsthand is not affiliated with its authors.
01
What is InstructVLA?
InstructVLA is introduced in arXiv 2507.17520, first submitted on 23 July 2025 and last revised on 3 March 2026 (v2, 48 pages) by Shuai Yang, Hao Li and eight co-authors. The project page lists the University of Science and Technology of China, Zhejiang University and the Shanghai Artificial Intelligence Laboratory, and marks the work as ICLR 2026. This page draws on the abstract, the official project page and the official GitHub repository. The full paper text was not read, so anything beyond those sources is Not stated in the sources read.
02
What problem does vision-language-action instruction tuning address, per the sources?
The abstract says existing VLA models often sacrifice multimodal reasoning for precise action generation or the reverse, narrow their abilities to task-specific manipulation data, and suffer catastrophic forgetting of pre-trained vision-language capabilities. InstructVLA aims to preserve the flexible reasoning of large vision-language models while delivering manipulation performance, with the help of embodied reasoning. The README adds 650K VLA-IT annotations and a curated benchmark suite for instruction generalization.
03
How is InstructVLA built and trained, per the sources?
Per the abstract, VLA-IT trains on standard VLM corpora and the curated 650K-sample VLA-IT dataset, using mixture-of-experts adaptation to jointly optimize embodied reasoning and action generation. The project page describes that dataset as human-robot interactions annotated with instructions, scene captions and question-answer pairs. The README describes a stage-1 action expert, then a stage-2 generalist fine-tuned with X-LoRA that freezes the action expert and adds general multimodal data from the Bunny dataset. Paper hyperparameters, expert counts and compute are Not stated in the sources read.
04
How is InstructVLA evaluated, per the sources?
The arXiv v2 abstract reports 33% over SpatialVLA on in-domain SimplerEnv tasks. On SimplerEnv-Instruct, an 80-task benchmark needing closed-loop control and high-level instruction understanding, it reports 96% over a fine-tuned OpenVLA and 29% over an action expert aided by GPT-4o. The project page gives 30.5% and 92% for the first two, a mismatch this page does not resolve. It also lists real-world tests on a Franka Research 3 (few-shot) and a WidowX-250 arm (zero-shot).
05
How does InstructVLA relate to other VLA models on this site?
Like ChatVLA, which this site describes as unifying understanding and robot control with separate experts, InstructVLA targets the loss of vision-language ability when a VLA trains on manipulation data. OneTwoVLA switches between reasoning and acting. The README compares InstructVLA with OpenVLA, SpatialVLA, π0 and GR00T N1.
InstructVLA’s data is robot demonstrations with instruction, caption and question-answer annotations. Firsthand supplies real human egocentric demonstrations, never synthetic, generated or simulated, and the site describes no robot action, teleoperation, simulation, visual question answering or caption data. Its only language field is one task instruction per episode. Firsthand makes no claim that its data trained, was used with or is compatible with InstructVLA or any other model, robot or benchmark.
06
What real human egocentric data can you get custom-collected on demand?
Firsthand collects real egocentric demonstrations on demand, to a written spec: a head-mounted global-shutter camera plus two wrist cameras, with 3D hand pose at 21 joints per hand with a per-joint visibility flag, wrist trajectories, contact events, action segments and one task instruction per episode. You confirm the spec on a Spec Pilot and pay per validated hour; the site states a 48-hour median turnaround where coverage is deep. Delivery is RLDS, LeRobot, WebDataset or HDF5. Request a custom dataset below.
07
InstructVLA at a glance, as stated in the official sources
| Item | What the official sources state |
|---|---|
| Paper | arXiv 2507.17520, v1 23 July 2025, v2 3 March 2026 |
| Authors and venue | Shuai Yang, Hao Li and eight co-authors; USTC, Zhejiang University, Shanghai AI Laboratory; ICLR 2026 (project page) |
| Method | VLA-IT: mixture-of-experts adaptation, joint embodied reasoning and action generation |
| VLA-IT dataset | 650K samples: instructions, scene captions, question-answer pairs; licence Not stated in the sources read |
| General multimodal data | Bunny dataset (README) |
| Benchmark | SimplerEnv-Instruct, 80 zero-shot tasks |
| Real-robot tests | Franka Research 3 few-shot, WidowX-250 zero-shot |
| Code | GitHub InternRobotics/InstructVLA; no licence label or licence section |
| Hyperparameters and compute | Not stated in the sources read |
| Firsthand relationship | None. Firsthand is not affiliated with InstructVLA’s authors |
09
InstructVLA results as reported in the sources
| Evaluation | Setup | Reported result |
|---|---|---|
| SimplerEnv, in-domain | arXiv v2 abstract | 33% over SpatialVLA (project page: 30.5%) |
| SimplerEnv-Instruct | 80 tasks | 96% over fine-tuned OpenVLA (project page: 92%); 29% over an action expert aided by GPT-4o |
| LIBERO average (README) | Four suites, three seeds | InstructVLA-1.5B 95.8 ± 0.4; OpenVLA-OFT-7B 97.1 ± 0.6; π0-2B 94.2 ± 0.9; GR00T-N1-1.34B 93.9 ± 1.1; OpenVLA-7B 76.5 ± 0.6 |
| Multimodal scores, inference speed | Not applicable | Not stated in the sources read |
10
InstructVLA training data beside real human egocentric capture
| Aspect | InstructVLA (sources read) | What the Firsthand site states |
|---|---|---|
| Data origin | Robot demonstrations, VLA-IT annotations, general multimodal data | Real human egocentric capture; not generated or simulated |
| Capture | Third-view input, optionally robot state or wrist view | Head-mounted global-shutter camera plus two wrist cameras |
| Actions | Robot actions from an action expert | 3D hand pose at 21 joints per hand with a per-joint visibility flag; wrist trajectories; no robot action data |
| Language | Instructions, scene captions, question-answer pairs | One task instruction per episode |
| Delivery | GitHub code, dataset and checkpoints | Spec Pilot; RLDS, LeRobot, WebDataset, HDF5 |
| Compatibility | Not claimed | Not claimed |
11
What the Firsthand site states its egocentric data provides
| Item | What the site states |
|---|---|
| Cameras | Head-mounted global-shutter camera plus two wrist cameras |
| Depth and motion | Active-stereo depth and 3x IMU on one hardware-triggered PTP clock |
| Hand signals | 3D hand pose at 21 joints per hand with a per-joint visibility flag; wrist trajectories |
| Events and segments | Contact events and frame-accurate action segments |
| Language | One task instruction per episode |
| Commercial terms | Written spec, Spec Pilot, pay per validated hour, buyer-owned commercial license |
12
Frequently asked questions
- What is InstructVLA?
- A vision-language-action model introduced in arXiv 2507.17520. It keeps the reasoning of large vision-language models while generating manipulation actions, using a training paradigm called Vision-Language-Action Instruction Tuning.
- What is vision-language-action instruction tuning (VLA-IT)?
- Per the abstract, multimodal training with mixture-of-experts adaptation that jointly optimizes embodied reasoning and action generation on standard VLM corpora and a curated 650K-sample VLA-IT dataset.
- What is SimplerEnv-Instruct?
- An 80-task benchmark introduced with InstructVLA. The abstract says it requires closed-loop control and high-level instruction understanding. The project page names two reasoning levels, instruction aggregation and situated reasoning.
- What data does InstructVLA use?
- Standard VLM corpora and the 650K-sample VLA-IT dataset, per the abstract. The README names the Bunny dataset as the multimodal source. Dataset licence terms are Not stated in the sources read.
- Can InstructVLA be used commercially?
- GitHub shows no licence label and the README has no licence section. Licence terms for the paper, dataset and checkpoints are Not stated in the sources read. Check the repository; this is not legal advice.
- Does Firsthand use InstructVLA, and can I request a custom dataset?
- Firsthand is not affiliated with InstructVLA’s authors and makes no claim of use, training or compatibility. Firsthand data is real human egocentric capture. You can request a custom egocentric dataset: write a spec, confirm it on a Spec Pilot, pay per validated hour.