Global Shutter Egocentric Video Data

A global-shutter camera exposes all pixels at the same instant. A rolling-shutter camera scans the frame row by row over a short but nonzero interval. Egocentric video is first-person footage, recorded from the actor’s own viewpoint while a real task is performed. Global shutter egocentric video data combines the two: first-person task footage captured by a camera that does not skew during motion.

This page is about ordering that footage as a custom collection. Firsthand captures it on demand against your written skill spec, using the head-mounted global-shutter camera described below, and delivers it in the training format your pipeline loads. It is the shutter-specific counterpart of the stereoscopic and head-mounted camera collection pages.

Egocentric capture is full of fast head and hand motion. The site’s glossary states that rolling-shutter artifacts corrupt geometry exactly when the hand is moving quickly, which are the frames a policy most needs to learn from, while largely static scenes see little effect. Reach-and-grasp is the motion a manipulation policy is trying to imitate, so it is the last place you want warped geometry.

The second reason is alignment. A policy trained on RGB, depth and pose together needs all three to describe the same instant. If row timing skews the image, pixels no longer line up with depth and pose captured alongside them. A global shutter keeps the frame consistent with those streams, which is why Firsthand’s head camera is global-shutter rather than rolling-shutter.

The third reason is a cleaner buying decision. A vendor spec that does not state the shutter type at all is a red flag on the site’s comparison page. Here the shutter type, resolution, frame rate and field of view of the head camera are stated up front.

The table below sets the two exposure methods side by side for egocentric capture. The properties come from the site’s comparison page and glossary; the last row is the one claim specific to Firsthand’s rig.

Rolling-shutter correction algorithms exist, but the comparison page notes they rely on estimating motion during the exposure, which is imperfect. Capturing with a global-shutter sensor avoids introducing the artifact rather than trying to remove it afterward.

Not by itself, and this page does not claim it. The site states that the head camera is global-shutter and rolling-shutter-free. That removes skew and wobble caused by row-by-row exposure. It is a separate question from blur inside a single exposure, which depends on how long the exposure lasts.

The site does not state exposure times, shutter speeds or a blur metric, so none are claimed here. If low blur matters for your model, say so in the skill spec: exposure targets for your lighting conditions are set per program and checked on the Spec Pilot batch. That is a more reliable route than a label such as motion-blur-free training video, which is not a measurable property on its own.

The instrumented head-mounted rig carries a Sony IMX585 global-shutter head camera with a 120° field-of-view M12 lens on a carbon headband that weighs 42 grams, two wrist cameras, an Intel RealSense D435if depth sensor mounted on the forehead 18 millimetres below the head camera, and Bosch BMI088 IMUs. A belt pack holds the PTP grandmaster clock, the hardware trigger line and a 2-terabyte NVMe drive.

The head camera records 3840×2160 at 60 fps, encoded as H.265 with lossless keyframes. The site states the global shutter for the head camera only. It makes no shutter claim for the wrist cameras or the depth sensor, so this page makes none either. A lighter tier, a contributor-worn 360 camera on a head strap, exists for broad reach but gives up depth, wrist views and sub-millisecond sync, and it is not the global-shutter rig described here.

Depth from a stereo pair is recovered by matching corresponding points between the left and right images. The site’s stereo rig guide explains that if cameras have rolling-shutter skew, the two images no longer describe the same instant in the same way row by row, and the disparity match degrades exactly during fast motion. A global shutter keeps both frames consistent with the depth and pose captured alongside them.

Calibration then fixes where each sensor sits. Every session on the instrumented rig is calibrated twice: a checkerboard solve across 40 poses, per-camera intrinsics that must land under 0.28 px RMS reprojection or the session does not start, and head↔wrist↔depth extrinsics expressed in the head-camera frame. The whole procedure repeats at the end, and the difference is written into the episode as calib_drift_mm.

Every active stream is hardware-triggered onto one PTP clock domain. The site publishes a cross-stream offset of 1.12 ms median and 1.94 ms at the 99th percentile, against a 2.0 ms ceiling. An episode that breaches the ceiling for more than three consecutive frames is rejected and recaptured rather than shipped with a caveat.

Hardware triggering is a stronger guarantee than software timestamping or post-hoc alignment: the sensors are physically told to expose together, so there is no clock drift to estimate downstream. In the HDF5 and RLDS exports each stream carries int64 nanosecond timestamps in the PTP domain, so you can recompute the residual yourself.

Every custom collection starts with a written skill spec: the tasks your pipeline needs covered, the setting, the object set, the capture tier, the annotation depth and the accept criteria, signed before capture begins. For a global-shutter order the spec also records that the instrumented head-mounted rig is required, and any lighting or exposure constraints that matter to your model.

Because the rig, calibration and clock are already built, the spec mostly names the tasks and environments rather than the sensor configuration. One attempt at the task is one episode, so any delivered sample traces back to the skill and setting it came from.

Paid, consented contributors perform the real task against your spec while wearing the rig. We do not capture robot or teleoperation data, and we do not capture at a customer’s own facility. QA happens once, against the canonical episode, before any format-specific export is produced.

Episodes are machine-checked against reject criteria: a sync breach beyond the ceiling for more than three consecutive frames, calibration drift beyond tolerance, the manipulated object out of frame for more than 15 percent of the episode, more than 35 percent invalid depth pixels in the manipulation region, an undocumented bystander, or a mismatch against the named skill spec. A human reviewer then checks each episode against the task spec. Rejected episodes go to the reject log, which ships with every batch, and are recaptured at our cost. Only footage that clears the accept criteria is billed as a validated hour.

Privacy sits in the same gate. Every episode is tied to a signed contributor release, and faces, plates and screens in frame are detected and irreversibly blurred at ingest, with a reviewer confirming the result before anything ships.

The same QA-passed episodes can be exported as RLDS, LeRobot, WebDataset, HDF5, zarr or Rerun .rrd. The multiview delivery notes state that the head stream ships at full 4K, so you can decode lazily or downscale in the input pipeline. If you need to verify synchronization yourself, request the HDF5 export, which carries per-stream nanosecond timestamps.

The dataset ships under a perpetual, irrevocable, worldwide commercial licence that is non-exclusive by default, with the buyer owning the resulting dataset and exclusivity available. The same terms apply whichever format you choose.

Where coverage for the requested tasks and regions is already deep, the median time from signed spec to first validated batch is a median of 48 hours. Where it is thin, turnaround runs three to four weeks while recruiting expands in parallel. Billing is per validated hour, not per raw hour recorded.

Write down the tasks, environments, objects and annotation depth your model needs, state that you require the global-shutter head-mounted rig, and name the delivery format. Most orders begin with a Spec Pilot: a small, fast first batch that proves the spec and accept criteria hold on real footage before you scale.

On that pilot, check fast reach-and-grasp frames for skew, confirm exposure on your lighting conditions, and confirm the formats and annotations you depend on.