What Is LoHoVLA?

LoHoVLA is introduced in LoHoVLA: A Unified Vision-Language-Action Model for Long-Horizon Embodied Tasks (arXiv 2506.00411; submitted 31 May 2025 as v1, cs.RO; marked as a preprint under review in the PDF) by Yi Yang, Jiaxuan Sun, Siqi Kou, Yihan Wang and Zhijie Deng, listing Fudan University, ShanghaiTech University and Shanghai Jiao Tong University. Read: the arXiv abstract page and the full paper text, HTML and PDF, with its appendices. No official project page or code repository was found or linked in those sources, so code availability is Not stated. Check the version you cite.

LoHoVLA is third-party research. Firsthand is not part of it or affiliated with its authors, makes no claim that any model was trained on Firsthand data, and captures human egocentric video, not robot data. What the site states is custom collection of real human demonstrations, fast and on demand, to a written spec: 3D hand pose at 21 joints per hand, calibrated stereo depth and 6-DoF IMUs at 200 Hz, delivered in RLDS, LeRobot format or WebDataset with a buyer-owned commercial license. The site also states ordered action segments, and one task instruction per episode as its only language data. Whether segments could stand in for LoHoVLA’s sub-task labels is Not stated.

Per the abstract, real-world embodied agents face long-horizon tasks: high-level goals that need multi-step solutions beyond a single action. That takes high-level task planning, meaning decomposing a goal into sub-tasks, and low-level motion control, meaning generating precise robot actions. The authors write that existing VLA models often falter in planning, while hierarchical architectures, which pair a VLM-based planner with a low-level controller, can suffer from coordination issues.

Their example goal is “Clean the desk”, which implies a sequence of sub-tasks. For simplicity, they assume each sub-task completes within a single time step.

LoHoVLA first infers the next sub-task in language from the current observation and the high-level goal, then uses that sub-task as context to predict the robot action. Both come from one pretrained vision-language model, PaliGemma, which the authors describe as a SigLIP-based image encoder, a Gemma-2B decoder-only language model and a linear projection layer. They extend its language head to output language sub-tasks and discrete action tokens, discretizing normalized action values into 1,024 uniform bins.

Training keeps the image encoder and projection layer fixed and uses cross-entropy loss on text and action outputs. A two-stage strategy first fine-tunes on long-horizon tasks with text loss only, then adds pick-and-place primitives and optimizes both losses.

The authors name three failure types: a sub-task planning error, correct planning with an incorrect action prediction, and correct planning and prediction where an external disturbance causes failure. LoHoVLA re-predicts the action after a failure and re-plans the sub-task only when failures within the current sub-task exceed a threshold K, which they set to 2.

They compare this with re-predicting only and with re-planning after every failure. They report that re-predicting only performed worst, and that the other two were comparable overall, with the threshold design needing fewer planning steps.

The authors say sub-task annotations for real-world long-horizon tasks can rarely be obtained at scale without human effort, so they built LoHoSet in the Ravens simulator, with a UR5e arm with a suction gripper and objects on a table. It holds 20 long-horizon tasks, each with 1,000 expert demonstrations of visual observations, linguistic goals, sub-tasks and actions, plus three pick-and-place primitive tasks.

Objects are blocks, bowls and zones in 11 colors, observed as RGB and depth top-down reconstructions. Sub-tasks come from manually designed rules over the simulator’s full scene information. Ten tasks follow the LoHoRavens benchmark and ten were added; without the extra ten, the authors report overfitting to seen tasks.

The abstract says LoHoVLA significantly surpasses hierarchical and standard VLA approaches on long-horizon tasks in the Ravens simulator. In Table 2 it reports average score and success rate of 97.8 and 91.5 on put-block-into-matching-bowl, against 14.9 and 0.0 for the vanilla VLA baseline.

These are the authors’ results on simulated block, bowl and zone rearrangement. Firsthand does not reproduce them and ranks no models.

The site’s Hi Robot entry describes a hierarchical system in which a high-level vision-language model outputs a language command for a separate low-level VLA policy. LoHoVLA’s paper describes one model that produces sub-tasks and actions together. The site’s OneTwoVLA entry covers one model that switches between reasoning and acting, and its Long-VLA entry a long-horizon model with phase-aware input masking.

How these papers relate to LoHoVLA is Not stated in the sources read.

Per the paper, each demonstration pairs a high-level language goal with templated sub-task text. On the site, the only language data is one task instruction per episode.

The authors list limits: robot actions have limited precision because of their discrete structure, and the one-time-step sub-task assumption may not suit real-time applications. Experiments ran in simulation, so real-robot results, released code, checkpoints and licence are Not stated.

When a program needs real human egocentric demonstrations that public data does not cover, specified in writing. Firsthand does not build or train models. Confirm the spec on a Spec Pilot, a small fast first batch; the median from signed spec to first validated batch is 48 hours where coverage is deep, three to four weeks where thin. Billing is per validated hour.

What is LoHoVLA?

A published unified vision-language-action model (arXiv 2506.00411) for long-horizon embodied tasks. One pretrained vision-language model generates language sub-tasks and robot action tokens. It is third-party research.

What is LoHoSet?

The authors’ dataset, built in the Ravens simulator: 20 long-horizon tasks, each with 1,000 expert demonstrations of observations, goals, sub-tasks and actions, plus three pick-and-place primitives. These are the authors’ figures.

How is a unified VLA different from a hierarchical planner and controller?

Per the paper, a hierarchical design uses a separate high-level planner and low-level controller, which the authors say can suffer from coordination issues. LoHoVLA generates sub-tasks and actions from one model instead.

Is the LoHoVLA code available?

Not stated in the sources read. No official repository or project page was found or linked in the paper text, so check the authors’ arXiv page for updates.

Was LoHoVLA tested on a real robot?

Not stated in the sources read. The experiments the paper describes ran in the Ravens simulator.

Does Firsthand data train LoHoVLA?

No claim is made. Firsthand is not affiliated with its authors, captures human egocentric video rather than robot data, and does not state that any model was trained on Firsthand data. The only language data the site states is one task instruction per episode.