Glossary
What Is IMU Data?
Last reviewed: September 2026
Short answer
IMU data is inertial measurement — acceleration and angular velocity from an accelerometer and gyroscope — recorded per sensor over time. In egocentric and robotics capture it is hardware-timestamped onto the same clock as the video and depth streams, rather than logged independently, so motion can be aligned frame-for-frame with what the cameras saw.
01
Why it matters
Video alone under-determines fast motion: a blurred or occluded frame still has a real trajectory an IMU captures directly. IMU data that isn’t on the same clock as the video it accompanies adds little, since a policy can’t reliably fuse a signal that drifts against the frames it’s meant to explain.
02
How Firsthand handles it
Firsthand runs three 6-DoF IMUs (BMI088), one head-mounted and one per wrist, sampling at 200 Hz and hardware-triggered onto the same PTP clock as the video and depth streams. It’s delivered as streams/imu_head, streams/imu_wrist_l and streams/imu_wrist_r (float32 [N, 6]) in HDF5, with RLDS and WebDataset carrying the same per-sample timestamped structure adapted to each format’s convention — see using IMU data in robot learning datasets for the full breakdown.
03
IMU fields as delivered in each format
| Format | Field | Notes |
|---|---|---|
| HDF5 | streams/imu_head, streams/imu_wrist_l, streams/imu_wrist_r | float32 [N, 6] at 200 Hz, one array per 6-DoF IMU |
| RLDS | observation.imu_* (per-sample structure) | Same per-sample timestamped structure, adapted to RLDS convention |
| WebDataset | imu.parquet (per-sample structure) | Same per-sample timestamped structure, adapted to WebDataset convention |
05
Frequently asked questions
- What is IMU data?
- Inertial measurement — acceleration and angular velocity from an accelerometer and gyroscope — recorded over time. In robotics capture it’s hardware-timestamped onto the same clock as the video and depth streams so motion can be aligned frame-for-frame with what the cameras saw.
- What IMU hardware does Firsthand use?
- Three 6-DoF IMUs (BMI088): one head-mounted and one per wrist, sampling at 200 Hz and hardware-triggered onto the same PTP clock as the other sensors. See using IMU data in robot learning datasets for the capture setup.
- What fields does IMU data arrive in?
- streams/imu_head, streams/imu_wrist_l and streams/imu_wrist_r in HDF5 (float32 [N, 6] at 200 Hz), with RLDS and WebDataset carrying the same per-sample timestamped structure adapted to each format’s convention.
- Why does IMU sample count differ from the video frame count?
- IMU runs at 200 Hz while the head camera runs at 60 Hz, so the number of IMU samples (N) and video frames (T) differ because the streams run at different native rates — they’re resampled by the loader, not in the delivery.
- Why does IMU data need to share a clock with the video?
- Unsynchronized IMU drifts against the frames it’s meant to explain, so a model can’t reliably fuse the two. IMU only adds a reliable motion signal when it’s hardware-triggered onto the same clock as the video and depth streams.