Egocentric Sock Matching and Pairing Data

It is real-world video of a person matching socks and pairing them, with synchronized sensor data alongside, captured to a written skill spec. The buyer sets the task and the accept criteria; Firsthand handles contributors, capture, QA, consent and delivery. This page is about egocentric sock pairing data in the sense of what a policy can learn from watching a person do this step, and it makes no claim that any robot has been trained on it.

The data is human capture by paid, consented contributors, not robot or teleoperation data, and it is not a fixed corpus you download. The catalog row is "Match + pair socks", described as "Match socks and pair them." The subject of this page is that one task. Folding, hanging, flattening and fastening have their own rows and pages, compared in the tables below.

One. In the Deformables & Textiles Bench environment of the Manufacturing & Workshops sector the catalog lists "Match + pair socks": "Match socks and pair them." It is rated Medium, its status is Available on request, and the capture setup it lists is Multiview rig + handheld grippers. The environment examples read "Cloth-and-cable bench: garments, towels, cables, form boards, bags." The first table below reproduces the row exactly.

Available on request means the work is scoped and recruited to spec when a buyer asks for it. This page therefore claims no completed collection, no delivered hours and no delivered footage for this row. Other rows on the same bench, such as "Fold shirts + towels", "Hang + space garments" and "Flatten + smooth fabric", are nearby laundry and garment-handling context only. They are listed in the second table and are not the subject of this page.

The site describes two capture modes for its catalog rows. In the first, a head-mounted phone records the first-person view and a multiview rig adds synchronized third-person angles when the spec asks for them. In the second, a staged working set is recorded by a multiview rig while the operator performs the task with handheld grippers, so demonstrations map onto a robot end effector. "Match + pair socks" is listed with the second: Multiview rig + handheld grippers.

Read plainly, that means this row is a staged bench task, not a recording of someone sorting their own laundry at home. How the first-person viewpoint is arranged inside that setup, which cameras are used and whether a head view is part of it are Not stated on the row and are agreed in the skill spec and confirmed on a Spec Pilot. The site states no stereo pair for this row, so this page describes egocentric and multiview capture only and does not call it stereo capture.

Three problems are already described on this site for deformable work. The first is hand occlusion. Fabric covers fingers, and the site states that 3D hand pose carries a per-joint visibility flag so a hidden joint is flagged as hidden rather than silently guessed, with contact events marking when a hand meets an item. In a task where one item is held against another, the labels have to say what is visible rather than imply a clean view.

The second is the mask. Masks propagated from frame to frame work for rigid objects and not for cloth, which is why per-frame instance masks, stored at the depth resolution of 848×480 so they line up with the depth stream, are shipped where the spec calls for them. For a task with several similar items in view, what each mask follows is a decision the spec settles. The site does not state why the row is rated Medium, so this page does not guess.

Depth is also worth naming. The site states that depth degrades on wet and reflective surfaces and that per-frame confidence maps mask low-confidence regions. Whether a given fabric behaves differently is Not stated, so the Spec Pilot checks the real footage before the order scales. Lighting for the bench is Not stated and is fixed in the spec.

Each attempt is delivered as one episode, and a sock order needs a decision up front about what one attempt is. The row description gives the task, matching socks and pairing them, but not a start state, an end state or a count. Whether an episode is one pair or a run of pairs, whether it begins with the items already separated and whether several attempts share one recording are spec decisions, because the site does not state an episode length or an item count.

Inside the episode, the synchronized streams run from start to end and action segments divide it into labeled spans, each with start_ns, end_ns, verb and noun. A spec for this row names the verb and the noun rather than a broad activity, as the site’s skill-spec guide advises. The verbs and nouns are written into the spec for your program; this page supplies no step list and no instruction on how the task is done.

For episodes in general the site states ten hardware-synchronized streams, synced under 2 ms, including head and wrist cameras, depth with confidence maps, IMU, 3D hand and body pose and instance segmentation. Hand pose is 21 joints per hand with a per-joint visibility flag. Calibration is measured every session and shipped with the episode, with a 0.28 px RMS reprojection gate. The language data is one task instruction per episode.

Which of these a "Multiview rig + handheld grippers" capture carries is Not stated for this row. In particular, whether a gripper capture delivers hand pose in the same form as a head-mounted capture is not stated, so it is asked for in the spec. The fourth table below lays out what the site states and what it does not.

This page is about one task on one catalog row. The egocentric laundry folding page centres on folding and sorting laundry, and the commercial laundry and linen page covers linen handling in a commercial laundry and hotel linen service. The napkin and towel folding page covers two hospitality folding rows, "Fold napkins" and "Fold towels + linens", both Medium and Available on request.

The buttons, zippers and fasteners page covers fastening tasks on the same bench, and the shoelace and eyelet page covers one lace row. The duvet cover page covers one large-item row, and the deformable object page covers soft items in general. The second and third tables below put them side by side. None of those pages names sock pairing as its main topic, which is why this one exists.

The catalog lists twelve rows on the bench, and all twelve are listed with a multiview rig and handheld grippers and are Available on request. Besides the sock row they are "Bag + seal soft goods", "Coil + Velcro cable", "Fasten buttons/snaps", "Flatten + smooth fabric", "Fold shirts + towels", "Hang + space garments", "Insert duvet cover", "Open/close zipper", "Route + clip cables", "Thread lace eyelets" and "Wrap + fold packaging". The second table below lists them with their ratings.

That matters for a program that wants more than one laundry step. A spec can name the sock row alongside the folding, hanging or flattening rows so one capture setup and one set of accept criteria cover several steps. Whether those rows are captured in one session or several is Not stated and is settled in the quote.

Every batch is validated against signed accept criteria before it ships, with a reject log. The site states that an episode is rejected when the manipulated object is out of frame for more than 15 percent of it, when more than 35 percent of depth pixels in the manipulation region are invalid, or when an undocumented bystander appears. Episodes that fail are rejected and logged per batch, not delivered with a caveat.

Billing is per validated hour, and rejected items are recaptured at our cost. For a task with small items held in the hand, the out-of-frame rule is worth reading against your spec, because what counts as the manipulated object is a decision the spec settles. The site states no accuracy figure for step labels, so what counts as a pass for a step boundary is written into the accept criteria.

The site’s off-the-shelf catalog lists about 324,800 hours across 18 datasets. None of its descriptions names sock matching or pairing, so no off-the-shelf set is offered as coverage of this row. Egocentric laundry capture elsewhere on the site, such as the household chores dataset, is a way to look at a format and rig, not coverage of this task.

Sock pairing collection to a spec is therefore custom collection: custom collection at scale through a network of thousands of capture providers, for a consistent, recurring data flow to labs. Where coverage is deep, the median time from signed spec to first validated batch is a median of 48 hours; where it is thin, turnaround runs three to four weeks. The row is Available on request, so the scoped-to-spec timeline applies rather than the ready-now case, and the quote states it up front.

Every episode links to a signed contributor release through a documented consent chain, and the consent record is part of the delivery. Faces, plates and screens are detected and irreversibly blurred at ingest, and a reviewer confirms the result before anything ships, with the anonymization record delivered alongside. Speech is scrubbed of names, addresses and card numbers, and footage that cannot be de-identified is discarded.

The buyer owns the dataset under a perpetual, irrevocable, worldwide, non-exclusive-by-default licence, with exclusivity available as a separate term. Capture at your own facility is listed as not offered, so the setting is one Firsthand arranges against your spec.

Sock types, sizes, colors and patterns, fabric or product type, brand, the number of items per episode, the rule that decides what counts as a match, task times, success rates, error rates, force or tactile data, a pairing-correctness label and a task-completion score are not stated on the site as standard fields. No customer, property or facility is named for the row.

If your policy needs any of these, they are written into the skill spec as annotation depth and covered in the quote before capture begins. This page gives no instructions for how the task is performed.

Name the row you are following, "Match + pair socks", the start and end state of one attempt, what counts as a reject, and the capture mode: the listed multiview rig with handheld grippers, or another arrangement you want checked on a Spec Pilot. Then say how much variation in items and setting your policy should see, which streams and labels you want, and which delivery format your pipeline reads.

Delivery can be RLDS, LeRobot, WebDataset, HDF5, zarr or Rerun .rrd, or raw streams with metadata, and MCAP is also offered. Most programs begin with a Spec Pilot, a small first batch that proves the spec and accept criteria on real footage before you scale. The last table below shows what each spec element decides.