Delivery format
LeRobot delivery format for egocentric episodes
Parquet + mp4 per camera in the HF datasets layout — the smallest on-disk option.
By the Firsthand capture teamLast updated September 23, 2026
Short answer
The LeRobot format is the dataset layout used by the LeRobot ecosystem for imitation learning, storing episodes as parquet plus one mp4 per camera in the Hugging Face datasets layout. Firsthand exports the v2.1 layout directly, with hand and body pose packed into observation.state.
Field mapping
How Firsthand fields map into this format
- Container
- parquet + mp4 per camera, HF datasets layout
- Features
- observation.images.head, observation.images.wrist_l/r, observation.state, action
- State vector
- 2×21×3 hand joints flattened, then 24×3 body joints
- Version
- v2.1 layout, loads with lerobot.common.datasets
- Timestamps
- Per-frame, aligned to the head-camera clock
Loading
Loading it
from lerobot.common.datasets.lerobot_dataset import LeRobotDataset
ds = LeRobotDataset("firsthand/kitchen_knife_dice", root="data")
sample = ds[0]
head = sample["observation.images.head"] # decoded frame tensor
state = sample["observation.state"] # flattened hand + body pose
action = sample["action"]Gotchas
What to watch for
- Video is mp4, so decoding is the bottleneck — use the dataset’s built-in video backend rather than decoding per item.
- observation.state packs hands then body; keep the ordering when you slice it.
- Depth is not embedded in the mp4 — pull it from the HDF5 export if your policy consumes depth.
Get a LeRobot sample.
40 episodes across 4 environments, delivered in the format your stack already reads.