Egocentric Walking and Navigation Video Data

Egocentric walking and navigation data is real-world video of a person moving through a space while wearing the capture rig: walking a street, passing doors, taking stairs, buying a ticket and riding a bus or train. The footage shows the route ahead and the surroundings from the head-mounted viewpoint as the person moves. Each episode follows the Firsthand episode schema, with ten streams synced under 2 ms. Which streams a head-mounted phone populates, and which the rig adds, is confirmed in the spec.

Teams search for it as an egocentric navigation dataset, first-person walking video data, robot navigation training data and humanoid locomotion video data. They all describe demonstrations of a person getting from one place to another, collected to a written spec. This page covers only what the Firsthand site states and marks everything else as Not stated, including any claim about gait, body pose, maps or GPS.

The site states two navigation rows in its commercial task catalog. One is "City Tours", described as "Walk city streets and landmarks, navigating crowds, doors and stairs". The other is "Navigating Public Transit", described as "Buy a ticket, pass the gate and board and ride buses or metro trains". Both sit in the Public Spaces & Outdoor sector.

Each wording names several movements in order, which is the shape of a spec: the tour row runs from the street through crowds, doors and stairs, and the transit row runs from the ticket to the ride. Anything outside that wording is Not stated, including route length, terrain and walking speed. A policy that needs another route writes it into the spec and we scope it, because a catalog row is a starting vocabulary, not a fixed menu.

The City Tours row is rated Easy and sits in an environment called City Streets, Shops & Public Transit. Its capture mode is listed as phone (head-mounted) or multiview rig, and its status is Available on request.

The site defines that status plainly: the task is scoped and recruited to spec when a buyer asks for it. So recording for the row is not already running, and a first batch is quoted against your spec. The row names streets, landmarks, crowds, doors and stairs, but it states no route, distance, step count or stair count, and no city, landmark or venue is named.

The Navigating Public Transit row is rated Easy and shares the environment, the capture modes and the status of the City Tours row. It adds a sequence the tour row lacks: a ticket purchase, a gate, then boarding and riding. The row states no operator, station, line, fare system or vehicle type, and what counts as a completed journey is Not stated, so the accept criteria are written into the spec.

The row also mixes two kinds of activity. Passing a gate is movement through a space, while buying a ticket is a hand task. A spec should say whether the ticket step is in scope, and whether the ride itself is wanted at all.

Walking appears elsewhere on the site, but as part of other work. The Shopping row in the same environment is described as "Browse shelves, pick items, queue and pay at the checkout in a shop", and it is a shop task rather than a route. Outside this environment, a logistics row reads "Walk the pick path with a scanner, pick items to the tote and pack and tape the order", and a venue cleaning row reads "Walk the venue collecting trash and recycling into bags and sweep up debris". In both, the captured task is the work, not the walk.

Moving between rooms or aisles as a task in its own right is not stated by name, and neither is crossing a street. A team that needs either writes it into the spec. The site has a separate page on mobile manipulation, which covers carrying things between rooms and manipulating at each stop; this page covers the two navigation rows only.

City Streets, Shops & Public Transit lists "Sidewalks, shops, markets, buses, metro and train stations" among its examples. It holds three rows, City Tours, Navigating Public Transit and Shopping, all rated Easy, all Available on request, and all listing a head-mounted phone or a multiview rig.

So the site describes a public space, in the Public Spaces & Outdoor sector, and not a workplace. It defines the phone mode as a head-mounted phone recording the first-person view of real work on site, and the rig mode as adding synchronized third-person angles when the spec asks for them. The rows do not say which mode applies, and the site does not describe any row in this environment as a staged set.

The site states that egocentric capture happens in real kitchens, workplaces and streets. It names no city, station, transit operator, shop, market or venue, and does not state the number of sites, the regions or the cities. Environment, access and schedule are agreed in the written spec.

In the phone mode, a head-mounted phone records first-person video as the person moves through the route. In the rig mode, a multiview rig adds synchronized angles when the spec asks for them. Camera count, resolution, frame rate, mounting position and carry method are Not stated for these rows, and so is how the two modes would be combined on one route. Which streams each mode populates is confirmed in the spec.

Each episode follows the Firsthand episode schema: ten streams synced under 2 ms, with RGB, depth, 3D hand pose with per-joint visibility, contact and grasp events, and frame-accurate action segments. The site also states a 200 Hz 6-DoF IMU among its sensors. For navigation, action segments are where a spec marks each leg: the walk, the doorway, the stairs, the gate and the boarding. Contact events mark when a hand meets something such as a door.

Not stated: whether depth and hand pose are populated in the phone mode, and head or camera 6-DoF pose, SLAM, maps and GPS as a Firsthand deliverable. This page makes no claim about them. Route length, distance, step counts, speed, trajectory accuracy, task durations, the number of episodes and sites, success or quality labels and robot results are Not stated too.

The site gives no technical analysis of navigation, so this page makes no claim about robot results. It does say, on its mobile manipulation page, that a robot working in a home or a facility does not stay at one bench: it carries things between rooms and crosses a floor to the next station.

What the row wording implies is a different data shape from bench work. The person moves, the scene changes continuously, and the rows name doors and stairs, which are the points where a route changes. The site rates both rows Easy, which is the only statement on difficulty. It states no gait, whole-body pose or joint data for these rows, so humanoid locomotion video data here means first-person footage of the route, not motion capture of the body.

Delivery formats are RLDS, LeRobot and WebDataset, plus HDF5, zarr and Rerun .rrd. Navigation episodes use the same schema as every other Firsthand collection, so a loader that reads another Firsthand dataset reads this one without changes. The format table below sets each format against what its official documentation states.

Every batch is validated against signed accept criteria before it ships, with a reject log, and delivery is rolling, per validated batch. Rejected items are recaptured at our cost, and billing is per validated hour.

Not stated as a deliverable. The site states a 200 Hz 6-DoF IMU among its sensors, and nothing else about the motion of the head or camera over a route. It does not state camera pose, visual SLAM, maps or GPS for these rows. If your navigation policy needs one of them, raise it before the spec is signed, so that what is possible can be confirmed and quoted.

Every episode links to a signed contributor release through a documented consent chain, and the consent record is part of the delivery. The release also covers bystanders: contributors confirm they have not knowingly recorded anyone who has not signed it, and where an unidentified person appears, the footage is de-identified or discarded.

A vision model runs on every frame at ingest, flagging faces, heads, licence plates and any legible screen or document. Those regions are irreversibly blurred, and a human reviewer confirms the result before footage ships. If the reviewer is unsure a region is fully covered, the episode is held or discarded, never downgraded and shipped. The site does not state that readable signage is blurred, so raise any such need before the spec is signed. Because the City Tours row names crowds, this is the step that matters most for this task family.

Where coverage is already deep, the first validated batch lands in a median of 48 hours from signed spec. Thin environments typically take three to four weeks, because vetting operators there is the bottleneck, and we quote that up front. Both navigation rows are Available on request, so they are recruited to your spec first, and the 48 hour figure should not be assumed.

The site also states program milestones. A pilot reaches its first delivery in two to three weeks, and a standing program reaches full rate in four to six weeks. Those assume capture coverage is already in place. A batch size and a later-batch turnaround are Not stated, so both are quoted against your spec.

You describe the routes, the places and the legs your policy needs. We turn that into a written spec covering tasks, environments, capture tier, annotation depth and accept criteria, then quote reach, timeline and price before any capture begins. This is the custom egocentric data collection route that every Firsthand task family follows.

A Spec Pilot, a small and fast first batch, can prove the spec and accept criteria before you scale. Every collection ships under a perpetual, buyer-owned commercial license, with a reject log and signed accept criteria on every batch.