Egocentric Damaged Goods Sorting Data

Egocentric damaged goods sorting data is real-world video recorded from the point of view of a person who is checking returned or damaged items and putting each one in the right bin. The camera shows what the person sees: the item, both hands, the scanner and the bins. Alongside the video, each episode follows the Firsthand episode schema, with depth, 3D hand pose, contact events and frame-accurate action segments aligned on one hardware clock.

Teams search for it under several names, including damaged goods sorting robot training data, egocentric retail damaged merchandise inspection video and a returns bin sorting dataset. They all describe first-person demonstrations of the same back-of-house jobs, collected to a written spec. This page covers only what the Firsthand site states about it, and marks everything else as Not stated.

Two rows in the site task catalog cover this work, both in the Retail Store -- Back of House environment. The first is "Damaged-Goods Sorting", described as "Inspect returned or damaged items, scan them and sort them into markdown or disposal bins." The second is "Inspect and sort damaged goods using a handheld scanner and returns bins", described as "Scan each damaged item, check the condition and drop it into the right returns bin." Both are rated Easy, both use a head-mounted phone or multiview rig, and both are listed as Ready to deploy.

Anything outside those two rows is Not stated. That includes grading an item by its condition, recording a damage reason, issuing a refund, and any named type of damage, so none is claimed here. The site also states that no catalog row names grading an item condition or recording a return reason. If your policy needs one of them, you write it into the spec and we scope it, because the catalog rows are a starting vocabulary, not a fixed menu.

The site defines Ready to deploy as contributors and site access already set up, with capture starting once a spec is signed. Both damaged goods rows carry that status, so the site does not describe them as scoped from scratch. It does not say that a damaged goods capture stream is running today, because that is the meaning of Active collection, which neither row has. This page makes no claim about delivered hours.

The site describes the Retail Store -- Back of House environment as the stockroom, receiving dock and backroom of supermarkets, apparel and general retail. The environment holds ten rows in total, all rated Easy and listed as Ready to deploy, and the two damaged goods rows are two of them. The rest cover backroom organization, restocking, tagging and carts, and this page does not claim them.

The site names no retailer, store, region or building, and does not state the number of sites or workers. Environment, access and schedule are agreed in the written spec.

A head-mounted phone records the first-person view of real work on site, which is the capture mode the catalog lists for both rows. A multiview rig adds synchronized third-person angles when the spec asks for them, and a wrist-mounted camera is available alongside the head camera for close-range detail. A stereo head rig is also offered where depth from a calibrated pair matters; the MCAP hub page describes it.

The site does not state a camera count, resolution or frame rate for these rows, the type of scanner, the number of bins or the bin layout. Raise any of them in the spec.

Each episode follows the Firsthand episode schema: ten streams synced under 2 ms, with head and wrist RGB, depth, 3D hand pose with per-joint visibility, contact and grasp events, and frame-accurate action segments. Hand pose tracks the grip on an item and on a scanner, contact events mark when fingers meet an item or a bin, and action segments separate the inspection, the scan and the placement.

Object labels use the terms your spec sets. Item categories, what counts as damaged, the markdown, disposal and returns rules, scanner models, a task-completion or sort-correctness label, force or tactile data, durations and the number of episodes are Not stated. They are defined in the spec if your policy needs them, and no safety, shrink or loss-prevention claim is made.

The catalog rates both rows Easy, so the site does not call them Hard, and this page makes no claim about robot results. What the site does say is where the capture is difficult. Sorting often includes a check, such as inspecting a returned item or checking the condition of a damaged one, but it ends with the item placed in a destination, so the decision and the placement both matter. The same hands hold the item, then a scanner, then drop the item in a bin.

The site also notes, on its returns page, that the moment a hand meets an item, a scanner or a tag is hard to place in a long clip, and it pairs that problem with contact events. A policy that only sees clean takes has no model of recovery, so missed grasps, dropped items and mis-placed objects are kept and action-segmented rather than discarded.

A good spec treats each destination as its own outcome. Sorting into markdown or disposal bins and dropping into a returns bin after a scan and condition check are different end states, so a spec names the start state, the end state and the accept criteria for each. The site does not state what counts as damaged, which item belongs in which bin or how many bins there are, so those rules are written into the spec and proven on a Spec Pilot.

The site also states no per-episode sort-correctness label. A buyer who wants correct and incorrect placements tagged defines the tag in the spec. That keeps action segments consistent across contributors and lets you ask for more of one row without re-collecting the other.

Delivery formats are RLDS, LeRobot and WebDataset, plus HDF5, zarr and Rerun .rrd, and MCAP is offered for stereo capture. The format table below sets each against what its official documentation states. The episode schema is the same as every other Firsthand collection, so a loader that reads another Firsthand dataset reads this one without changes.

Every batch is validated against signed accept criteria before it ships, with a reject log, and delivery is rolling, per validated batch. Rejected items are recaptured at our cost, and billing is per validated hour.

Every episode links to a signed contributor release through a documented consent chain, and the consent record is part of the delivery. A vision model runs on every frame at ingest, flagging faces, heads, licence plates and any legible screen or document. The site describes the documents class as phone and monitor content, forms, badges and mail that would leak names, addresses or account data. Those regions are irreversibly blurred, and a human reviewer confirms the result before footage ships.

If the reviewer is unsure a region is fully covered, the episode is held or discarded, never downgraded and shipped, and the anonymization record is delivered with the data. The site does not name receipts, return slips or shopper cards as a separate class, and it states no separate rule for shelf labels, cartons or scanner displays, so this page does not claim they are blurred. Raise any of them before the spec is signed. No readable shopper information is shown on this page.

Where coverage is already deep, the first validated batch lands in a median of 48 hours from signed spec. Thin environments typically take three to four weeks, because vetting operators there is the bottleneck. The site names retail handling as an example of deep coverage but does not state coverage depth for the back-of-house environment itself, so the timeline is quoted against your spec before capture begins.

Custom retail back-of-house data collection runs as custom collection at scale through a network of thousands of capture providers, for a consistent, recurring data flow to labs. The site lists no off-the-shelf damaged goods dataset, so this is custom collection. On-demand top-ups run against the same spec and accept criteria after each training run. A batch size and a later-batch turnaround are Not stated, so both are quoted against your spec.

You describe the damaged goods work, the bins and the sequence your policy needs. We turn that into a written spec covering tasks, environments, capture tier, annotation depth and accept criteria, then quote reach, timeline and price before any capture begins. This is the custom egocentric data collection route that every Firsthand task family follows.

A Spec Pilot, a small and fast first batch, can prove the spec and accept criteria before you scale. Every collection ships under a perpetual, buyer-owned commercial license, with a reject log and signed accept criteria on every batch.