What Is NaVILA?

NaVILA is introduced in arXiv 2412.04453 by An-Chieh Cheng, Yandong Ji, Zhaojing Yang, Zaitian Gongye, Xueyan Zou and five co-authors (v1 5 Dec 2024, v2 17 Feb 2025). The project page and repository cite it as an RSS 2025 paper and list UC San Diego, USC and NVIDIA. Read: the arXiv abstract page, project page and repository README; the full paper text was not read.

Third-party research; Firsthand is not affiliated and claims no use, training or compatibility with it. Firsthand captures first-person human video, not robot data, fast, on demand and to a written spec: 21 keypoints per hand with a visibility flag, wrist trajectories and action segments, plus one task instruction per episode.

The abstract targets Vision-and-Language Navigation with legged robots, which it says gives people a flexible way to command a robot and lets it move through more challenging, cluttered scenes. Its stated difficulty is that translating human language instructions all the way to low-level leg joint actions is non-trivial.

NaVILA unifies a VLA with locomotion skills. Instead of predicting low-level actions directly, the VLA first generates mid-level actions with spatial information in the form of language, for example "moving forward 75cm". A visual locomotion reinforcement-learning policy takes that as input and executes it. The project page says the high-level commands are language-based while a real-time locomotion policy ensures obstacle avoidance.

The project page describes an end-to-end vision-based policy trained without teacher-student distillation, interacting with the environment using LiDAR during training. It claims this reduces the sim-to-real gap and helps near transparent surfaces and on rough terrain. Those are the authors’ claims, not verified here. The README links a separate locomotion code repository.

The project page says the VLA was trained on real-world human video data, simulated indoor navigation data and question-answering tasks. The README provides annotations for envdrop, scanqa, r2r, rxr and human sets on Hugging Face. For its YouTube Human Touring set, raw videos are not released because of copyright; the README provides video IDs and annotations instead. No data counts were read, so none are given here.

The abstract says NaVILA substantially improves previous approaches on existing benchmarks and introduces new IsaacLab benchmarks with more realistic scenes, low-level controls and real-world robot experiments. The project page names a VLN-CE-Isaac benchmark and real-world deployment on quadruped and humanoid robots, and says the vision-based policy beats a blind policy on success rate. No scores were read, so none are given here.

The AnjieCheng/NaVILA repository states the Apache-2.0 licence. Its README checks off model weights and evaluation, training code, the YouTube Human Touring dataset annotations and Isaac Sim evaluation. It names a pretrain checkpoint, navila-siglip-llama3-8b-v1.5-pretrain, and an evaluation checkpoint, navila-llama3-8b-8f, under a8cheng on Hugging Face. Evaluation builds on VLN-CE and Habitat.

The site entries describe Hi Robot as a high-level vision-language model writing a language command for a low-level VLA, and Hume as sampling action candidates, selecting one by learned value and denoising it. NaVILA is aimed at legged-robot navigation: its VLA writes a mid-level language action that a separate locomotion RL policy turns into leg motion.

Benchmark scores, data counts, compute and model sizes were not read and are not given here. The project page cites nine authors where the arXiv page names ten. Whether first-person human video would suit NaVILA is Not stated.

NaVILA’s README names human touring video among its training sets. Firsthand captures human first-person video, not robot data, and states hand pose, wrist trajectories, action segments and one task instruction per episode; navigation paths, locomotion commands and robot state are Not stated. Write tasks, environments and signals into a spec and confirm on a Spec Pilot. Median time to first validated batch: 48 hours where coverage is deep, three to four weeks where thin. Formats: RLDS, LeRobot v2.1, WebDataset, HDF5; whether a delivery matches NaVILA’s expected fields is the buyer’s to check.

What is NaVILA?

A two-level VLA for legged-robot navigation (arXiv 2412.04453): the VLA writes mid-level language actions and a visual locomotion RL policy executes them.

What are mid-level actions in NaVILA?

Actions with spatial information written in language, such as "moving forward 75cm", generated by the VLA instead of low-level leg joint actions.

What does NaVILA claim on benchmarks?

The abstract says it substantially improves previous approaches on existing benchmarks and on new IsaacLab benchmarks. No scores were read, so none are given here.

Is NaVILA open source?

The AnjieCheng/NaVILA README states the Apache-2.0 licence and checks off weights, evaluation, training code, dataset annotations and Isaac Sim evaluation.

Does Firsthand use NaVILA, and can I request a custom dataset?

Firsthand is not affiliated and claims no use, training or compatibility. You can request a custom dataset, confirmed first on a Spec Pilot.