Now recruiting capture teams worldwide. Get paid to record first-person video of real work.

Apply to capture

Task row

Stereo Sock Matching and Pairing Data

Matching socks and pairing them looks trivial and is hard to learn from video: the items are small, soft and alike, they change shape in the hand, and the fingers that hold them hide much of what a model needs to see. This page covers the stereo and container side of sock pairing robot training data for the one catalog row that names this work: what a calibrated stereo pair adds to stereo laundry matching video, what MCAP is according to its own documentation, and what the site does not state. We run custom textiles data collection fast and on demand against a spec you write, as a handheld gripper sock pairing dataset when the spec names it, delivered as MCAP, as custom collection at scale through a network of thousands of capture providers, for a consistent, recurring data flow to labs.
Catalog row
"Match + pair socks" (Medium); Available on request
Environment
Deformables & Textiles Bench
Container
MCAP (.mcap), open source
License
Perpetual, buyer-owned

Why stereo and MCAP here

What do a stereo pair and an MCAP file add to sock pairing video?

01

Depth for small soft items

The site states that a stereo pair at human-eye spacing makes depth recoverable from every frame pair, so the socks and the hands working them sit in one frame. The site gives no depth accuracy figure for this row and does not say how depth behaves on a given fabric.

02

Fingers inside fabric, flagged not guessed

The site states that fingers disappear behind objects. 3D hand pose carries a per-joint visibility flag, so a hand hidden behind a sock is marked as hidden rather than silently filled in.

03

Masks that follow the item every frame

The site states that deformable items do not track well with masks propagated from frame to frame, so deformable captures ship per-frame instance masks where the spec calls for them.

04

Several streams in one container

MCAP stores multiple channels of timestamped, pre-serialized data, so a stereo pair, hand pose and IMU can share one file on one log-time axis. Which streams go in the file is set in the skill spec.

Definition

What is stereo sock matching and pairing data?

It is first-person video of a person matching socks and pairing them, recorded with two calibrated cameras, with hand pose and sensor streams alongside. People reach it through several phrasings: sock pairing robot training data, stereo laundry matching video, a handheld gripper sock folding dataset, or custom textiles data collection at scale. They describe one request from different angles. Stereo describes the capture, and MCAP describes the container. They are separate choices.

One wording needs care. The catalog row names matching and pairing, not folding, and the site lists no sock folding row, so this page does not claim one. The site does not state that stereo is already running on this row either. The catalog lists a multiview rig with handheld grippers as the capture setup, so a stereo pair is a capture choice agreed in the skill spec through the site’s general stereo offer and confirmed on a Spec Pilot. The data is human capture by paid, consented contributors, not robot or teleoperation data, so using it for a specific robot still needs your own retargeting step. Firsthand is not affiliated with Foxglove or the MCAP maintainers.

Catalog row

Which catalog row does this page rely on, and what is its status?

One. In the Deformables & Textiles Bench environment of the Manufacturing & Workshops sector the catalog lists "Match + pair socks": "Match socks and pair them." It is rated Medium, its status is Available on request, and the capture setup it lists is Multiview rig + handheld grippers. The environment examples read "Cloth-and-cable bench: garments, towels, cables, form boards, bags." The first table below reproduces the row exactly.

Available on request means the work is scoped and recruited to spec when a buyer asks. It is not a statement that footage of this task has been delivered, and the site states no delivered hours or stereo clips for it, so this page claims none. The site names no sock type, size, material or brand for the row, so none is stated here.

The setting

What is the Deformables & Textiles Bench environment?

The catalog describes it as a cloth-and-cable bench with garments, towels, cables, form boards and bags as examples. It is a staged working set, not a recording of someone sorting their own laundry at home. The site states that Firsthand does not capture at a customer’s own facility, so the set is a capture setting arranged for the task.

The site describes this capture mode as a staged working set recorded by a multiview rig while the operator performs the task with handheld grippers, so demonstrations map onto a robot end effector. Whether a stereo pair would be worn, mounted or combined with the multiview rig the catalog names is not stated, and is settled in the spec.

Difference

How is this different from the egocentric sock, laundry, napkin and duvet pages?

This page is the stereo and MCAP treatment of one row, Match + pair socks. The egocentric sock matching and pairing page is the first-person page for the same row: stereo and MCAP are not its subject. It is a sibling, not an equivalent, and the two share the row wording.

The stereo commercial laundry and linen page covers six laundry and linen rows, all Easy, in a commercial laundry and hotel linen service, and none of them is sock pairing. The stereo napkin and towel folding page covers folding rows in the Hospitality Service Set. The stereo duvet cover insertion page covers one large-item row rated Hard. None of them is about matching socks and pairing them as a single step on a bench. The comparison table below sets the pages side by side.

Capture setup

What capture setup does the row list, and where does stereo come in?

The catalog lists "Multiview rig + handheld grippers" for the row. The site does not state a stereo pair for it, and it does not describe this task as stereo capture. Stereo applies through the site’s general stereo offer: a stereo pair is a capture choice agreed in the skill spec and confirmed on a Spec Pilot. The site does not state a camera count for the row.

The site describes two stereo setups and does not say which one an order for this row uses. The hard-environments stack is a global-shutter 4K/60 head camera with a 120° lens, two 1080p/60 wrist cameras, a depth sensor at 848×480 and 30 fps, and 200 Hz 6-DoF IMUs. The off-the-shelf stereo head rig, dataset 06, is dual fisheye cameras at 1080p per eye, a 62 mm baseline, per-unit calibration, a 9-axis IMU and hand-tracking output. Whether either is used with the catalog’s multiview rig is not stated.

Handheld grippers

Why does a handheld gripper capture matter for sock pairing?

The catalog lists handheld grippers because the operator holds a gripper rather than a bare hand while performing the task, and the site says this lets demonstrations map onto a robot end effector. For small soft items, the grasp, the hold and the release are the actions a policy has to reproduce. A handheld gripper sock pairing dataset keeps the demonstration closer to the hardware a robot would use than a bare-hand clip does.

The site does not state which gripper is used or how it attaches, and this page names none. It also does not claim that a model trained on this capture succeeds at the task. The embodiment mapping is a design choice agreed in the skill spec, and the site does not claim that human capture replaces robot data.

Why stereo

Why capture sock matching and pairing in stereo?

Matching two items that look alike is a visual comparison, and pairing them is a placement: where each sock sits relative to the other and to the hands is geometry. A stereo pair recovers depth from the calibrated frame pair rather than from a standalone depth sensor, so that geometry is in the same capture that shows the hands working. The site gives no depth accuracy figure for this row, so what a buyer does with the depth is their pipeline’s question.

A stereo pair adds depth, not a verdict. The site lists no pairing-correctness label, no task-completion score and no force reading for this row, and none is claimed. The egocentric sock matching and pairing page already supplies the first-person framing; stereo is the addition for a buyer who wants the depth and the MCAP file.

Occlusion, masks and depth

What do soft items and covered hands do to visibility, masks and depth?

Socks are small enough that a closed hand can cover one entirely. The site states that fingers disappear behind objects. That is why each hand joint carries a visibility flag, why contact events mark when a hand meets an item, and why labels say what is visible rather than imply a clean view.

On masks, the site states that deformable items do not track well with masks propagated from frame to frame, which is why per-frame instance masks, stored at 848×480 so they line up with the depth stream, ship where the spec calls for them. When several similar items are in view, what each mask follows is a decision the spec settles. The site also states that depth degrades on wet and reflective surfaces and that per-frame confidence maps mask low-confidence regions. How depth behaves on a given sock fabric is not stated, so the Spec Pilot checks the real footage before the order scales.

Episodes

What does one stereo sock matching and pairing episode contain?

Each attempt is delivered as one episode. The row description gives the task, matching socks and pairing them, but not a start state, an end state or a count. Whether an episode is one pair or a run of pairs, whether it begins with the items already separated and whether several attempts share one recording are spec decisions, because the site does not state an episode length or an item count.

Inside the episode, the synchronized streams run from start to end and action segments divide it into labeled spans, each with start_ns, end_ns, verb and noun. A spec for this row names the verb and the noun rather than a broad activity, as the site’s skill-spec guide advises. This page supplies no step list and no instruction on how the task is done.

Privacy

What does the site say about privacy in a staged textiles set?

For every delivered episode the site states that faces and screens are blurred irreversibly, that a reviewer confirms no un-consented person is identifiable, and that footage that cannot be de-identified is discarded. The anonymization record ships with the data, an undocumented bystander is a stated reject criterion, and every episode links to a signed contributor release.

Whether printed text on an item in view is blurred is not stated, so it is agreed in the spec. This page gives no venue or compliance claim beyond that. This is not legal advice, and the executed agreement controls.

MCAP

What is MCAP, per its own documentation?

MCAP is an open source container file format for multimodal log data, supporting multiple channels of timestamped pre-serialized data. It is serialization-agnostic, so messages can be Protobuf, DDS (CDR), ROS, JSON and more. Its design is row-oriented and append-only, it stores schemas alongside the data, and it offers an optional index plus optional LZ4 or Zstandard chunk compression.

Reader and writer libraries exist for C++, Go, Python, Rust, Swift and TypeScript, and the mcap CLI can inspect, merge and split files. The foxglove/mcap repository is MIT-licensed; that is the MCAP project licence, not the licence of a Firsthand delivery. The MCAP hub page and the glossary entry cover the format in full.

Streams

Which streams can ship in an MCAP delivery of sock pairing data?

Firsthand offers MCAP delivery, and dataset 06 is stated as shipping as MCAP. For this row the site states hand pose, a stereo-derived depth stream with confidence maps, IMU, contact events, action segments, per-frame instance masks and one task instruction per episode elsewhere. What it does not state is which of those are channels in an MCAP file, how they are named, what encoding they use or at what rate.

No topic name, message schema or rate is stated on this page, and none should be assumed. Name the streams that matter in the skill spec and confirm them on a Spec Pilot batch. The fourth table below sets each stream beside what the site states about it.

Failure modes

What goes wrong when you record sock pairing in stereo, and what does QA reject?

Hand occlusion by fabric is the first risk, under the visibility flag above. Weak depth on glare-heavy or reflective surfaces is the second. Calibration is measured at the start and end of every session under a 0.28 px RMS gate. An episode is rejected for a sync breach beyond the ceiling for more than three consecutive frames, for the manipulated object out of frame for more than 15 percent of the episode, for more than 35 percent of depth pixels in the manipulation region being invalid, for an undocumented bystander or for a mismatch against the named skill spec.

A reject log ships with every batch, and rejected items are recaptured at our cost. For small items held in the hand, the out-of-frame rule is worth reading against your spec, because what counts as the manipulated object is a decision the spec settles.

Scale and speed

How fast can custom stereo sock pairing data be collected at scale?

The site’s off-the-shelf catalog lists about 324,800 hours across 18 datasets. None of its descriptions names sock matching or pairing, so no off-the-shelf set is offered as coverage of this row. A stereo set can be looked at for its format and rig, which is not the same as coverage of this task.

Sock pairing collection to a spec is therefore custom collection: custom collection at scale through a network of thousands of capture providers, for a consistent, recurring data flow to labs. Where coverage is deep, the median time from signed spec to first validated batch is a median of 48 hours; where it is thin, turnaround runs three to four weeks. The row is Available on request, so the scoped-to-spec timeline applies and the quote states it up front. Collections are billed per validated hour, and rejected episodes and recaptures are never billed.

Delivery and license

Which formats and license come with a stereo sock pairing order?

Delivery is MCAP, alone or beside the formats your stack already reads: RLDS, LeRobot, WebDataset, HDF5 or zarr, or raw streams with metadata. Readable converter source ships alongside the data so the schema can be retargeted to your own loader.

The license is perpetual, irrevocable and worldwide, with the data buyer-owned and non-exclusive by default. Training, evaluating, fine-tuning and deploying models, including commercial ones, is permitted, and exclusivity is available as a separate term. The executed agreement controls. The site does not publish a price list, a minimum order or a batch size for this task, so those are set in your quote.

Not stated

What does the site not state about sock matching and pairing?

Sock types, sizes, colors and patterns, fabric or product type, brand, the number of items per episode, the rule that decides what counts as a match, task times, success rates, error rates, force or tactile data, a pairing-correctness label and a task-completion score are not stated on the site as standard fields. No customer, household, laundry or facility is named for the row.

If your policy needs any of these, they are written into the skill spec as annotation depth and covered in the quote before capture begins.

Ordering

How do you order a stereo sock matching and pairing dataset?

Name the row, "Match + pair socks", the start and end state of one attempt, what counts as the manipulated object and what counts as a reject. Say how much variation in items and setting your policy should see, and name the streams you want in the file, the topic naming and message encoding your reader expects, whether calibration ships inside the file or beside it, and whether MCAP is the only delivery or sits beside a training format.

Most programs begin with a Spec Pilot, a small first batch that proves the spec and accept criteria on real footage before you scale. The last table below shows what each spec element decides.

At a glance

What you get.

Catalog row
"Match + pair socks" (Deformables & Textiles Bench, Manufacturing & Workshops); Medium; Available on request
Catalog description
"Match socks and pair them."
Environment examples
"Cloth-and-cable bench: garments, towels, cables, form boards, bags"
Catalog capture setup
Multiview rig + handheld grippers. The site states no stereo pair for this row, so a stereo pair is agreed in the skill spec through the general stereo offer
Stereo head rig (06)
Dual fisheye, 1080p per eye, 62 mm baseline, per-unit calibration, 9-axis IMU, hand-tracking output; shipped as MCAP
Hand pose
21 joints per hand with a per-joint visibility flag
Masks
Per-frame instance masks at 848×480 for deformable captures where the spec calls for them
Sync
Hardware-triggered onto one clock; 2.0 ms cross-stream ceiling
Language data
One task instruction per episode
Sock types, colors, patterns, item counts, task times, success rates
Not stated; agreed in the skill spec
MCAP topics, schemas, rates
Not stated on the site; agreed in the skill spec
License
Perpetual, irrevocable, buyer-owned, non-exclusive by default

Every collection runs through the same seven-stage end-to-end custom collection pipeline. End-to-end custom data collection is a managed service that takes an AI data need from problem to owned dataset in a single accountable pipeline.

Catalog row

Which catalog row covers sock matching and pairing, and what is its status?

The row exactly as the task catalog states it. Items the catalog does not list are marked Not stated. Source file: lib/commercial-catalog-data.ts.
Task (as listed)Description (as listed)EnvironmentSkillCapture setupStatus
Match + pair socksMatch socks and pair them.Deformables & Textiles BenchMediumMultiview rig + handheld grippersAvailable on request
Sock types, sizes, colors, patterns, match rule, outcome labelsNone beyond the row wordingNot stated as rowsNot statedNot statedNot stated

Compared pages

How does this page differ from the egocentric sock, laundry, napkin and duvet pages?

Each page and what it covers. Statuses are as the site lists them; none of these pages is claimed to hold delivered hours here.
PageCaptureWhat it coversRelation to this page
This page: stereo sock matching and pairing dataStereo, agreed in the specMatch + pair socks (Medium); Available on requestStereo pair and MCAP delivery for one bench row
Egocentric sock matching and pairing dataEgocentric and multiview, not stereoThe same row, first-personThe first-person sibling; stereo and MCAP are not its subject
Stereo commercial laundry and linen dataStereo, agreed in the specSix laundry and linen rows, all Easy; four Active collection, two Ready to deployDifferent scope: sorting, washing, folding and pressing in a laundry
Stereo napkin and towel folding dataStereo, agreed in the specFolding rows in the Hospitality Service Set, Available on requestDifferent task: folding finished cloth
Stereo duvet cover insertion dataStereo, agreed in the specInsert duvet cover (Hard); Available on requestDifferent task: one large-item row on the same bench
Egocentric stereo data in MCAPStereo, hubThe stereo and MCAP hub across tasksGeneral stereo and MCAP treatment; not specific to socks

Nearby rows

What else does the Deformables & Textiles Bench list?

All twelve rows on the bench, as the catalog lists them. Only "Match + pair socks" is the subject of this page; the rest are context. Every row is listed with a multiview rig and handheld grippers and is Available on request.
Task (as listed)Description (as listed)SkillRole on this page
Match + pair socksMatch socks and pair them.MediumSubject of this page
Fold shirts + towelsFold garments and towels to standard.MediumNearby laundry context
Hang + space garmentsHang garments and space them.EasyNearby garment context
Flatten + smooth fabricFlatten a sheet and smooth wrinkles.EasyNearby fabric context
Insert duvet coverInsert a duvet into its cover.HardNearby bedding context
Fasten buttons/snapsFasten a row of buttons or snaps.HardFastener context
Open/close zipperStart and run a zipper closed.MediumFastener context
Thread lace eyeletsThread a lace through eyelets.HardFastener context
Bag + seal soft goodsBag and seal soft goods.EasyOther bench row
Wrap + fold packagingWrap an item in paper and fold.MediumOther bench row
Coil + Velcro cableOver-under coil and strap a cable.MediumOther bench row
Route + clip cablesRoute cables to a form board.MediumOther bench row

Stated vs not stated

What the site states about stereo capture for sock matching and pairing

Left to right: the item, what the site states, and what that means for a sock pairing order. “Not stated” means the site does not say.
ItemWhat the site statesFor a sock pairing order
Catalog capture setupMultiview rig + handheld grippers on the rowNo stereo pair is stated for this row; a stereo pair is a spec choice through the general stereo offer
Gripper capture modeA staged working set recorded by a multiview rig while the operator uses handheld grippers, so demonstrations map onto a robot end effectorStated for the mode; cameras for this row Not stated
Stereo head rig (dataset 06)Dual fisheye, 1080p per eye, 62 mm baseline, per-unit calibration, 9-axis IMU, hand-tracking output; shipped as MCAPWhether an order uses it: Not stated
Hard-environments stack4K/60 head camera, 120° lens; two 1080p/60 wrist cameras; depth 848×480 at 30 fps; 200 Hz 6-DoF IMUWhether an order uses it: Not stated
Hands, contacts and masksPer-joint visibility flag on hand pose; contact events; per-frame instance masks stored at 848×480 where the spec calls for themStated; what counts as the object is set in the spec
OcclusionFingers disappear behind objectsStated; how often it occurs on this row is Not stated
Depth on wet and reflective surfacesDepth degrades on wet and reflective surfaces; confidence maps mask low-confidence regionsStated; behaviour on a given fabric is Not stated
Calibration and synchronizationMeasured every session, 0.28 px RMS reprojection gate; hardware-triggered onto one clock, 2.0 ms ceilingStated
QA rejectsObject out of frame over 15 percent, over 35 percent invalid depth pixels, undocumented bystanderStated; per-batch reject log
Row statusMedium, Available on requestNo volume, hours or delivered stereo footage is claimed on this page
Sock type, color, pattern, match rule, item countNot statedNot claimed; set in the spec
Task duration, success rateNot statedNot claimed on this page
MCAP topics, schemas, ratesNot statedAgreed in the skill spec

Stream map

Streams for sock pairing work and what the site states about an MCAP file

Left to right: the stream or item, what the site states about it, and what the site states about it in MCAP. “Not stated” means the site does not say.
Stream or itemWhat the site statesIn an MCAP delivery, as the site states it
Stereo pairDual fisheye, 1080p per eye, 62 mm baseline (dataset 06)Dataset 06 is stated as shipped as MCAP; channel layout not stated
Depth with confidence mapsRecovered from the calibrated stereo pair; confidence map masks low-confidence regionsNot stated
Hand pose21 joints per hand with a per-joint visibility flagChannel layout and joint order not stated
IMU9-axis (dataset 06); 6-DoF at 200 Hz (hard-environments stack)Channel layout and rate not stated
Contact events, action segments, instance masksListed in the site episode schema; masks stored at 848×480Not stated
Task instructionOne per episodeNot stated
CalibrationMeasured every session and shipped with the episodePlacement in the file not stated
Topic names, schemas, encoding, ratesNot statedAgreed in the skill spec

Skill spec

What the skill spec fixes for a stereo sock matching and pairing order

Spec element and where a buyer can see it in the delivery. “Set per program” means the site states no fixed value.
Spec elementWhat it decidesWhere it shows up
RowMatch + pair socks, worded as the catalog words itSet per program; Available on request as listed
Start and end stateWhat one attempt is, and what counts as finishedEpisode boundaries; the site does not state an episode length
Object definitionWhat counts as the manipulated object: one sock, a pair or the whole set in viewContact events, masks and the out-of-frame reject rule; set per program
Rig and streamsStereo pair, multiview angles, wrist views, hand pose, IMU and any othersThe channels you can read; confirm on the Spec Pilot
VariationHow much variation in items and setting your policy should seeSet per program
Action segmentsVerbs and nouns for each stepSegment labels on each episode
MCAP topics and encodingWhat your reader expectsSet per program
Delivery formatHow the data is packagedMCAP alone, or beside RLDS, LeRobot, WebDataset, HDF5 or zarr
Accept criteriaWhat passes QA, including what counts as a rejectReject log per batch
PrivacyHow people, screens and printed text in frame are handledFaces and screens blurred at ingest; other items agreed in the spec
LicenseWho owns the datasetPerpetual, irrevocable, buyer-owned, non-exclusive by default

Example footage

What does stereo capture look like on this site?

One real clip already published on this site, shown as a rig example. It is a head-mounted stereo capture with hands in frame. It does not show socks, laundry or a textiles bench, and it is not an example of the multiview rig with handheld grippers this row lists.

egocentric_stereo_sample.mp4 · head-mounted stereo · hands in frame · 1920 × 1080 web preview · rig example, not a sock pairing clip

No stereo sock or laundry clip is published on the site, so this is labelled as a rig example. Request a sample to see delivered files. See sample data outputs, stereo sample page, request an egocentric data sample.

FAQ

Sock pairing robot training data, answered.

What is stereo sock matching and pairing data?
First-person video from two calibrated cameras of a person matching socks and pairing them, with depth, hand pose and per-frame masks alongside where the spec calls for them.
Which catalog row covers it, and what is its status?
"Match + pair socks" ("Match socks and pair them.", Medium) in Deformables & Textiles Bench, listed with Multiview rig + handheld grippers and Available on request. This page claims no delivered hours.
Is stereo already captured on this row?
Not stated. The catalog lists a multiview rig with handheld grippers. A stereo pair is agreed in the skill spec through the general stereo offer and confirmed on a Spec Pilot.
How is this different from the egocentric sock page?
The egocentric page covers the same row as first-person video. This page covers it with a stereo pair and MCAP delivery. The laundry, napkin and duvet pages cover different scopes.
Is there an off-the-shelf sock pairing dataset?
No. No off-the-shelf description names sock matching or pairing, so this is custom collection, at scale through a network of thousands of capture providers, for a consistent, recurring data flow to labs.
Why are masks and hand pose treated differently for cloth?
Masks propagated between frames do not track cloth well, so per-frame instance masks ship where the spec calls for them. Hand pose carries a per-joint visibility flag so fingers hidden by fabric are marked as hidden.
How fast can a custom stereo sock pairing dataset be collected?
Where coverage is deep, the median time from signed spec to first validated batch is a median of 48 hours; thin coverage runs three to four weeks. The row is Available on request, so the timeline is quoted against your spec.
Can it be delivered as MCAP, and what is in the file?
Yes. Firsthand offers MCAP delivery. Topic names, message schemas and rates are not stated on the site and are agreed in the skill spec.

Sources

Where is MCAP documented?

Request a custom stereo sock matching and pairing dataset delivered as MCAP.

Tell us the start and end state of one attempt, what counts as the manipulated object, how much variation in items and setting your policy should see, the streams you want in the file, the privacy rule for the set and whether MCAP sits alongside a training format. We will quote reach, timeline and price against a written spec, with a first validated batch in a median of 48 hours where coverage is deep.