What Is QUAR-VLA?

QUAR-VLA is introduced in QUAR-VLA: Vision-Language-Action Model for Quadruped Robots (arXiv 2312.14457; submitted 22 December 2023, last revised 4 February 2025 as v6, cs.RO, accepted by ECCV 2024) by Pengxiang Ding, Han Zhao, Wenjie Zhang, Wenxuan Song, Min Zhang, Siteng Huang, Ningxi Yang and Donglin Wang, listing Westlake University and Zhejiang University. Read: the arXiv abstract page, the HTML text of the paper including its supplementary material, and the authors’ project page, which the paper cites. Wording differs between revisions, so check the version you cite. No official code repository was found or linked in those sources, so code availability is Not stated.

QUAR-VLA is third-party research. Firsthand is not part of it or affiliated with its authors, makes no claim that any model was trained on Firsthand data, and captures human egocentric video, not robot data. What the site states is custom collection of real human demonstrations, fast and on demand, to a written spec: 3D hand pose at 21 joints per hand, calibrated stereo depth and 6-DoF IMUs at 200 Hz, delivered in RLDS, LeRobot format or WebDataset with a buyer-owned commercial license. The only language data the site states is one task instruction per episode. Mapping any delivery to QUAR-VLA’s inputs or command format is Not stated.

The authors say quadruped learning has typically handled language interaction and visual perception separately. They label vision-only approaches QUAR-VA, which rely on a coarse goal image, and language-only approaches QUAR-LA, which lack perception of the environment. QUAR-VLA combines an image and an instruction to produce executable actions.

They name two challenges: few large-scale datasets of quadrupeds doing diverse tasks, and an action space that must balance movement flexibility against computational cost for a legged robot.

QUART builds on a pre-trained vision-language model. The image and instruction are tokenized, a decoder-only transformer outputs discretized action tokens, and these are detokenized into commands. The action space is 11 high-level command dimensions plus a terminate signal, covering velocities, gait pattern, frequency, body height, pitch, foot width and foot height, with each continuous dimension discretized into 256 bins.

The main model is fine-tuned from an 8B pre-trained model, Fuyu-8B, and a LLaVA-based variant is tested in the supplement. The paper reports about 2 Hz inference for these commands, which a pre-trained low-level command tracking controller turns into joint actions.

QUARD, the QUAdruped Robot Dataset, defines seven tasks at easy, medium and hard levels: distinguish a letter, go to an object, go through a tunnel, go to an object while avoiding an obstacle, crawl under a bar, and unload an object into a receptacle. The Figure 2 caption reports 259K simulated episodes and 3K real-world episodes.

Simulation data came from Isaac Gym, using A* path planning with a PD controller; the supplement says D* was dropped because its sharp turns made the robot lose sight of the target. Real data came from remotely controlling a WR-2 quadruped with a front-mounted RealSense D435 camera in a lab.

The arXiv abstract reports 4000 evaluation trials and says the approach leads to performant policies. In the HTML text read, QUART succeeds on the crawl and unload tasks at 0.32 and 0.12 where the CLIP, R3M and VC-1 baselines record 0, and averages 0.44 on the detailed seen-task table against 0.25 for CLIP and 0.28 for VC-1.

For real-world transfer, success was 3/20, 7/20 and 13/20 with 0K, 25.6K and 256K simulated episodes added to 3K real ones. These are the authors’ results on their own setups; Firsthand does not reproduce them or rank models.

Per the paper, data was collected with a predefined language template naming the task, target object, speed and gait. The authors tested unseen wording, such as “navigate to target” for “go to the object”, and report QUART handled it better than the baselines.

On the site, the only language data is one task instruction per episode, and Firsthand captures human egocentric video rather than robot data. Using such video with QUAR-VLA is Not stated in the sources read.

The site’s NaVILA entry describes a two-level VLA for legged-robot navigation whose mid-level actions are written in language and executed by a locomotion policy. QUART instead outputs numeric commands for velocity, posture and gait. The site’s loco-manipulation entry covers a robot moving its base or body while manipulating objects; the paper’s unload task has the robot carry a ball on its back and drop it in a receptacle.

Any direct relation between QUAR-VLA and NaVILA is Not stated in the sources read.

The authors write that scenes are relatively simple, experiments ran on flat terrain, and data volume and trajectory diversity are insufficient. They also name the sim-to-real gap, inputs limited to vision and text, and command frequency for harder tasks as open.

No repository was found in the sources read, so released code, checkpoints and licence are Not stated.

When a program needs real human egocentric demonstrations of tasks, environments or objects that public data does not cover, specified in writing. Firsthand does not build or train models.

Confirm the spec on a Spec Pilot, a small fast first batch. Where coverage is deep, the median from signed spec to first validated batch is 48 hours; where it is thin, three to four weeks. Billing is per validated hour.

What is QUAR-VLA?

A published paradigm (arXiv 2312.14457) for quadruped robots. Its VLA model, QUART, takes an image and a language instruction and outputs high-level commands that a low-level controller executes. It is third-party research.

What is a vision-language-action model for quadruped robots?

Per the paper, a model that integrates visual information and instructions to generate executable actions for a legged robot, instead of handling vision and language separately.

How does a legged robot VLA differ from one for arms?

Per the paper, QUART outputs commands for base velocity, posture and gait, which a low-level controller tracks, rather than joint commands. The action space is chosen to balance movement flexibility and computational cost.

How much data did the authors use?

The Figure 2 caption reports 259K simulated episodes and 3K real-world episodes. These are the authors’ figures; Firsthand states no data counts for QUAR-VLA.

Is the QUAR-VLA code available?

Not stated in the sources read. No official repository was found or linked in the paper text or the project page, so check the authors’ paper page for updates.

Does Firsthand data train QUAR-VLA?

No claim is made. Firsthand is not affiliated with its authors, captures human egocentric video rather than robot data, and does not state that any model was trained on Firsthand data. The only language data the site states is one task instruction per episode.