What Is RoboDual?

RoboDual is introduced in Towards Synergistic, Generalized, and Efficient Dual-System for Robotic Manipulation (arXiv 2410.08001) by Qingwen Bu, Hongyang Li, Li Chen, Jisong Cai, Jia Zeng, Heming Cui, Maoqing Yao and Yu Qiao; the paper lists Shanghai Jiao Tong University, The University of Hong Kong, AgiBot and Shanghai AI Lab. The name is the paper’s own: “we introduce RoboDual, a synergistic dual-system”. Read: the arXiv abstract page, the paper text up to its ablation study, the official project page and the GitHub README. The rest of the paper was not read.

RoboDual is third-party research. Firsthand is not part of it or affiliated with its authors, and makes no claim that any model was trained on Firsthand data. What the site states is custom egocentric collection of real human demonstrations, fast and on demand, to a written spec: 3D hand pose at 21 joints per hand, calibrated stereo depth and 6-DoF IMUs at 200 Hz, delivered in RLDS, LeRobot format or WebDataset with a buyer-owned commercial license. The only language data the site states is one task instruction per episode. Mapping any delivery to RoboDual’s inputs or action format is Not stated.

Per the abstract, a generalist policy trained on a large cross-embodiment corpus gives broad adaptability and high-level reasoning but struggles with inefficient inference and expensive training. A specialist policy is curated for specific domain data and is precise and efficient, but lacks generalization. RoboDual is meant to supplement the merits of both.

The generalist is built on OpenVLA, a 7B-parameter autoregressive vision-language-action model. The authors adapt it to their setup with LoRA fine-tuning and extend it to predict action chunks. It takes a third-view RGB image and a language prompt, and outputs discretized actions and latent representations.

The specialist is a diffusion transformer. It is conditioned through cross-attention on the robot’s proprioceptive state, optional sensory inputs such as depth or tactile, and the generalist’s discretized actions and latents. The abstract gives its size as 20M trainable parameters.

They run asynchronously: one generalist inference supports several specialist rollouts. In the authors’ real-world setup the generalist produces one action, then the specialist runs eight fast steps at 0.035 seconds each.

The authors report a control frequency of 15 Hz on NVIDIA A5000 Ada GPUs, against 3.9 Hz for OpenVLA inside their system, which the abstract summarizes as a 3.8 times higher control frequency. They describe the generalist as slow and the specialist as fast.

On CALVIN, the authors report an average completed-task length of 3.66, up from 3.27 for the generalist alone, a 12% gain. They also report that the specialist reached 3.52 after one hour of training on a node with eight A100 GPUs, while the generalist-only variant needed about 1,400 GPU hours to stabilize.

In real-world tests on an AIRBOT Play arm with a third-view RGB camera, the abstract reports a 26.7% improvement over OpenVLA, and the generalizability table gives averages of 70.0 (single-task) and 68.3 (multi-task) for RoboDual against 41.7 for OpenVLA. With 5 demonstrations on “put block into bowl” they report 73.3% success. These are the authors’ results on their own setups and baselines; Firsthand does not reproduce them or rank models.

The site’s OpenVLA entry covers a single 7B model; RoboDual uses OpenVLA as its generalist and adds a specialist. The site’s Hi Robot entry covers a hierarchy where a high-level model outputs a simple language command for a low-level VLA. The RoboDual authors say their framework does not explicitly decompose tasks and instead connects the two models through discretized actions and latents.

The site’s OpenHelix entry covers another dual-system VLA model and short survey. How it relates to RoboDual is Not stated in the sources read.

The generalist inherits OpenVLA’s pre-training on large robot manipulation corpora, and the specialist is trained on task-specific demonstrations. The authors vary the demonstrations in one real-world task at 5, 10 and 100, and on CALVIN they test 5%, 10% and the full data.

On the site, the only language data is one task instruction per episode. RoboDual’s inputs in the sources read are third-view RGB, language prompts, proprioception and optional depth or tactile; egocentric video, stereo and IMU data are not described as part of its training.

The authors’ limitations were not read. The GitHub repository, OpenDriveLab/RoboDual, is released under the Apache License 2.0, and its README says generalist and specialist checkpoints are on Hugging Face and that code and training scripts target the CALVIN simulator.

When a program needs real human egocentric demonstrations of tasks, environments or objects that public data does not cover, specified in writing. Firsthand does not build or train models.

Confirm the spec on a Spec Pilot, a small fast first batch. Where coverage is deep, the median from signed spec to first validated batch is 48 hours; where it is thin, three to four weeks. Billing is per validated hour.

What is RoboDual?

A published dual-system robot policy (arXiv 2410.08001): a slow VLA generalist built on OpenVLA guides a fast diffusion transformer specialist. It is third-party research.

What is a generalist and specialist policy?

Per the paper, a generalist is trained on a large cross-embodiment corpus for broad adaptability, and a specialist is curated for specific domain data for task-level precision and efficiency.

How fast does RoboDual run?

The authors report 15 Hz in their real-world setup, against 3.9 Hz for OpenVLA inside the same system. These are their measurements on their own hardware.

Is RoboDual open source?

The official GitHub repository, OpenDriveLab/RoboDual, is released under the Apache License 2.0, and its README says generalist and specialist checkpoints are on Hugging Face.

Is RoboDual better than OpenVLA?

The authors report gains over OpenVLA on CALVIN and in real-world tests. These are the authors’ results on their own setups, and Firsthand ranks no models.

Does Firsthand data train RoboDual?

No claim is made. Firsthand is not affiliated with its authors, and the site does not state that any model was trained on Firsthand data. The only language data the site states is one task instruction per episode.