What Is the DROID Dataset?

DROID stands for Distributed Robot Interaction Dataset. It is described in the paper DROID: A Large-Scale In-The-Wild Robot Manipulation Dataset and on its project page. The authors note that collecting robot manipulation data in diverse environments is logistically hard, so most policies are trained on data from a few environments. DROID is collected "in the wild" across many real buildings instead, and the authors open source the dataset, policy learning code and a hardware guide.

DROID is robot data: a teleoperated robot arm with cameras watching it and its own joint, pose and gripper readings. It is not egocentric human video, and Firsthand is not part of it, endorsed by it or related to its authors or their institutions. The site states no default compatibility. What Firsthand states is the human side of a robot-learning mix: egocentric video, depth, 3D hand pose and action segments from paid, consented contributors, collected fast, on demand and to a written skill spec, under a perpetual, buyer-owned commercial license. Firsthand captures no robot action data and claims no advantage over DROID.

The paper credits a large cross-institutional effort: 50 data collectors across 13 institutions in North America, Asia and Europe, using 18 copies of one robot platform, over 12 months.

The stated motivation is diversity: the paper notes most existing robot datasets concentrate in a few scenes in one lab or building, and DROID aims to raise scene diversity and realism.

Every collector used the same hardware: a 7-DoF Franka Panda arm, two adjustable exterior stereo cameras, a wrist-mounted stereo camera and a VR headset with controllers for teleoperation, mounted on a portable desk so the robot could move between scenes. Data is recorded at 15 Hz, and a human teleoperator drives the arm.

Collectors followed a shared protocol: move to a new scene, calibrate the cameras, then enter feasible tasks into a GUI. The GUI prompted a randomly sampled task per episode and periodically asked for scene changes, roughly every 20 minutes. Collectors marked each episode as successful or not.

The paper states 76k successful demonstration trajectories, or 350 hours of interaction, across 564 scenes in 52 buildings and 86 tasks. A new scene means a substantial workspace change, not rearranged objects, and a task is a unique verb from the language instructions, so 86 tasks means 86 verbs. About 16k further trajectories marked not successful are released but not counted.

Each trajectory holds three stereo RGB camera streams at 1280x720, robot joint positions and velocities (7D), end-effector pose and velocity (6D) and gripper position and velocity (1D). Metadata covers one to three crowdsourced language instructions, camera calibration, building name, collector ID and scene type. The project page notes updated annotations and calibrations on Hugging Face.

The project page shows loading DROID with the TensorFlow Datasets library from a Google Cloud storage path, iterating over episodes and steps. The official sources do not name RLDS, so RLDS compatibility is Not stated, and Firsthand makes no claim about it. Firsthand’s own delivery formats are RLDS, LeRobot, WebDataset and HDF5 / zarr.

The paper states the full dataset is released under the CC-BY 4.0 licence. The GitHub repository covers the robot platform setup and data-collection workflows.

The authors trained diffusion policies on 6 tasks in 4 locations, including lab, office and household settings, and compared three setups: in-domain demonstrations only, a 50/50 mix of in-domain data and DROID, and a 50/50 mix of in-domain data and a curated subset of Open X-Embodiment. They report that co-training with DROID outperformed the next best method by 22% absolute success rate in distribution and by 17% out of distribution, with each task setting evaluated over 10 rollouts.

A second experiment found that a diverse-scene subset beat an equal-size 20-scene subset out of distribution. These are the authors’ results on their own hardware stack, and Firsthand makes no performance claim against them.

Write the target skills, environments and signals into a skill spec and confirm them on a Spec Pilot, a small fast first batch. This is on-demand custom collection, not a catalog download.

Where coverage is deep, the median from signed spec to first validated batch is 48 hours; where it is thin, three to four weeks. Billing is per validated hour.

What is the DROID dataset?

An open robot manipulation dataset of 76k teleoperated trajectories, 350 hours, 564 scenes and 86 tasks, per its paper.

What licence is DROID released under?

The paper states CC-BY 4.0 for the full dataset.

Is DROID egocentric human video?

No. It is teleoperated robot data from exterior and wrist cameras plus robot state and action signals.

Is Firsthand data part of DROID?

No. Firsthand is not part of it or endorsed by it, and the site states no default compatibility. Firsthand delivers RLDS and LeRobot among its formats.

What did the paper find about training on DROID?

Co-training with DROID beat the next best method by 22% absolute success rate in distribution and 17% out of distribution, on the authors’ own setup.

How fast can custom pretraining data be collected?

Where coverage is deep, the median from signed spec to first validated batch is 48 hours; where it is thin, three to four weeks.