What Is the ARCTIC Dataset?

ARCTIC is introduced in ARCTIC: A Dataset for Dexterous Bimanual Hand-Object Manipulation (arXiv 2204.13662), with an official project page and an official GitHub repository. The authors open with a gap: they say no datasets existed with ground-truth 3D annotations for studying the physically consistent, synchronised motion of hands and articulated objects. They argue that earlier hand-object datasets are mostly limited to grasping rigid objects. ARCTIC instead records bimanual articulation, where hand poses and object states evolve jointly in time, and the paper proposes two tasks on it: consistent motion reconstruction from a monocular video, and interaction field estimation, meaning dense relative hand-object distances estimated from images. The authors also release two baselines, ArcticNet and InterField. ARCTIC is a hand-object research dataset. The sources read do not describe it as robot data.

ARCTIC is third-party research. Firsthand is not affiliated with its authors or their institutions, and makes no claim that ARCTIC uses Firsthand data or hardware or that any model was trained on Firsthand data. The site states custom egocentric collection to a written spec, fast and on demand, delivering first-person human video with 3D hand pose at 21 joints per hand and a per-joint visibility flag, calibrated stereo depth and 6-DoF IMUs at 200 Hz. Firsthand captures first-person human video, not robot or teleoperation data.

Per the paper, ARCTIC synchronises a multi-view colour camera system with a Vicon motion-capture system of 54 infrared Vantage-16 cameras. The colour cameras give 8 static allocentric views, meaning third-person views, plus 1 moving egocentric view, recorded at 30 FPS. The paper says the large number of motion-capture cameras is there to reduce occlusion, which is severe when two hands handle an object, and it contrasts this with earlier hand-object datasets captured with 1 to 8 commodity RGB-D cameras.

The paper reports 10 subjects interacting with 11 articulated objects across 339 sequences. Each object consists of two rigid parts that rotate about a shared axis. Small markers of 1.5 mm radius sit on the hands and objects, placed on the dorsal side of the hand so they do not encumber natural interaction. Subjects were asked to either “use” the objects, which allows articulation, or “grasp” them. The supplementary material states that the egocentric camera also carries markers. Total recording duration in hours is Not stated in the sources read.

The abstract describes 3D hand and object meshes plus detailed, dynamic contact information for every frame. The paper fits pre-scanned subject and object meshes to the observed markers: SMPL-X for the full body, MANO for the hands, and a two-part articulated model for each object, parameterised by a 6D pose of the base part and a 1D articulation angle. The README lists 3D ground truth for SMPL-X, MANO and articulated objects.

Contact is computed per vertex from geometric proximity following the GRAB dataset, which the supplementary material describes as producing binary contact labels. The paper also says depth images of the hands, body and objects can be rendered from the data. The sources read do not describe language annotation by the authors, so language annotation is Not stated in the sources read. The README links third-party resources for the dataset, which this page does not rely on.

The paper frames ARCTIC around reconstruction and benchmarking. It defines the two tasks above, evaluates ArcticNet and InterField baselines qualitatively and quantitatively, and offers an allocentric protocol and an egocentric protocol over the same subject split, where the egocentric protocol resembles first-person views in a mixed-reality setting. It lists future directions: generating dexterous bimanual motion with articulated objects, extending generation to the full body, and benchmarking articulated-object pose estimators on rendered depth.

The README adds potential tasks such as template-free bimanual hand-object reconstruction, generating hand or full-body grasps and motion with articulated objects, and studying egocentric hand-object reconstruction. Training robot policies on ARCTIC is Not stated in the sources read. This page draws no conclusion about how ARCTIC would perform as pretraining data for any robot model. For the egocentric-human-data-for-robots framing, see EgoMimic and HOI4D below.

ARCTIC and custom egocentric capture answer different questions. ARCTIC is a laboratory motion-capture dataset with marker-based ground truth and a fixed object set, designed for studying hand-object motion. Custom egocentric capture is a data service: paid, consented contributors recording first-person video to a written spec, with fields and format set in advance. This page makes no ranking between them.

The comparison table below sets only what the official sources state against what the site states, and where a source is silent the cell says so. One difference matters for planning. The ARCTIC README states that ARCTIC data and software are not available for commercial use. Licence terms beyond that line are Not stated in the sources read, so a team with a commercial program should read the repository LICENSE itself.

When a program needs tasks, objects, environments or people that an existing dataset does not cover, specified in writing, and the data has to be usable for the program’s own purpose. For two-handed work the site states that both hands stay in frame with per-hand 3D pose and visibility flags, including long continuous clips of fine bimanual work. If contact signals matter, ask for the separate pilot-ready tactile glove tier, time-synced to a head-mounted stereo camera by hardware timestamps and an IMU.

Confirm the spec on a Spec Pilot, a small fast first batch. Where coverage is deep, the median from signed spec to first validated batch is 48 hours; where it is thin, three to four weeks. Billing is per validated hour, and delivery is in the format your training stack already reads: RLDS, LeRobot, WebDataset or HDF5.

What is the ARCTIC dataset?

A published dataset of two hands dexterously manipulating articulated objects, with 2.1M video frames paired with 3D hand and object meshes and contact information, captured with motion capture.

What does ARCTIC stand for?

The paper expands the name as ARticulated objeCTs in InteraCtion.

How many views does ARCTIC have?

Per the paper, 8 static allocentric views and 1 moving egocentric view, recorded at 30 FPS and synchronised with a 54-camera motion-capture system.

Is ARCTIC a robot dataset?

Not stated in the sources read. The paper and README frame it around hand-object reconstruction, interaction field estimation and motion generation, not robot policy training.

Can ARCTIC be used commercially?

The official README states that ARCTIC data and software are not available for commercial use. Further licence terms are Not stated in the sources read.

Does ARCTIC include language annotations?

Not stated in the sources read. The authors describe 3D meshes and contact information.

Does Firsthand use ARCTIC, or is it affiliated with its authors?

No. Firsthand is not affiliated with the ARCTIC authors or their institutions, and makes no claim that ARCTIC uses Firsthand data or hardware or that any model was trained on Firsthand data.

Can I request a custom egocentric dataset?

Yes. Write the tasks, environments, fields and format into a spec, confirm it on a Spec Pilot, and billing is per validated hour. Firsthand captures first-person human video, not robot or teleoperation data.