What Is Mobile ALOHA?

Mobile ALOHA is introduced in Mobile ALOHA: Learning Bimanual Mobile Manipulation with Low-Cost Whole-Body Teleoperation (arXiv 2401.02117), submitted 4 January 2024 by Zipeng Fu, Tony Z. Zhao and Chelsea Finn, with an official project page and GitHub repository. The authors note that imitation learning from human demonstrations has shown impressive performance, but most results focus on table-top manipulation and lack the mobility and dexterity needed for generally useful tasks. Their system targets mobile manipulation tasks that are bimanual and require whole-body control.

Mobile ALOHA is third-party research. Firsthand is not part of it, endorsed by it or affiliated with its authors, and makes no claim that Mobile ALOHA uses Firsthand data or that any model was trained on Firsthand data. The site does not state that Firsthand offers teleoperation, and Firsthand does not capture robot action data. What Firsthand states is custom egocentric human capture: both hands in frame with per-hand 3D pose and visibility flags, plus depth, contact events and action segments, collected fast, on demand and to a written skill spec.

The abstract describes Mobile ALOHA as a low-cost and whole-body teleoperation system for data collection. It augments the ALOHA system with a mobile base and a whole-body teleoperation interface. The authors then perform supervised behavior cloning on the data it collects.

The project page’s citation entry lists the work at the Conference on Robot Learning (CoRL) in 2024, with Zipeng Fu and Tony Z. Zhao as project co-leads and Chelsea Finn as advisor.

The ALOHA entry on this site describes a system where a person drives a two-arm robot through a custom teleoperation interface. Per the abstract, Mobile ALOHA keeps that system and adds a mobile base and a whole-body teleoperation interface. That is what moves the work off the table: the authors’ tasks include sauteing and serving a piece of shrimp, opening a two-door wall cabinet to store heavy cooking pots, calling and entering an elevator, and lightly rinsing a used pan using a kitchen faucet.

The repository says its codebase is forked from the ALOHA repository, and that training imitation learning algorithms also needs a version of ACT forked for Mobile ALOHA. A hardware cost figure, the interface’s mechanical details and the robot’s arm degrees of freedom are Not stated in the sources read.

The repository’s setup uses four robot arms in two roles: master arms, which the operator holds, and puppet arms, which perform the task. It configures three cameras, two on the wrists and one high, and connects the mobile base to the computer over a CAN bus cable. Teleoperation starts when the master-side gripper is closed. How the operator drives the base is Not stated in the sources read.

To record an episode, a script stores an hdf5 file, and separate scripts visualize an episode or replay it on the real robot. The README lists Ubuntu 18.04 and 20.04 with ROS 1 as tested configurations, with ROS 2 and later Ubuntu versions still being tested.

The abstract states that, using data collected with Mobile ALOHA, the authors perform supervised behavior cloning and find that co-training with existing static ALOHA datasets boosts performance on mobile manipulation tasks. It adds that with 50 demonstrations for each task, co-training can increase success rates by up to 90%.

Which static datasets were used, the mixing ratio and the total dataset size are Not stated in the sources read. These are the authors’ results on their own robot, and Firsthand draws no comparison from them.

It is robot-collected data in which the base moves as part of the task. The loco-manipulation entry on this site says tabletop clips show the hands but never the walking, turning and carrying between tasks, so a policy trained on them has no examples where the move is part of the task. The teleoperation entry says teleoperation data matches the robot’s embodiment exactly but is slow and hardware-bound, and that most programs need a blend of sources. Co-training with static ALOHA data is one example of that blending.

The loco-manipulation entry also suggests scoping such data by separating the human-demonstration layer (viewpoint, hands, body, scene) from the robot-action layer (commands). Whether Mobile ALOHA data includes egocentric human video is Not described in the sources read.

Firsthand does not claim that egocentric human video replaces robot teleoperation or substitutes for Mobile ALOHA. It does not deliver robot actions, base commands or action chunks. It collects egocentric human data on demand, to a written spec, for skills and environments you need covered.

Confirm the spec on a Spec Pilot, a small fast first batch. The median from signed spec to first validated batch is 48 hours where coverage is deep and three to four weeks where it is thin. Billing is per validated hour.

What is Mobile ALOHA?

A low-cost, whole-body teleoperation system for collecting bimanual mobile manipulation data, which augments the ALOHA system with a mobile base and a whole-body teleoperation interface.

How is Mobile ALOHA different from ALOHA?

Per the abstract, it adds a mobile base and a whole-body teleoperation interface to the ALOHA system.

How are Mobile ALOHA demonstrations recorded?

The repository’s recording script stores an hdf5 file per episode, with separate scripts to visualize and replay episodes.

What is co-training in Mobile ALOHA?

Training on Mobile ALOHA data together with existing static ALOHA datasets. The abstract says this boosts performance on mobile manipulation tasks.

What licence does Mobile ALOHA use?

The repository page lists the MIT License for its code. A data or hardware licence is Not stated in the sources read.

Does Mobile ALOHA include egocentric human video?

Not described in the sources read. The data is collected through teleoperation of the robot.

Is Firsthand data part of Mobile ALOHA?

No. Firsthand is not part of it or affiliated with its authors, and makes no claim that any model was trained on Firsthand data.