Sewing and textile handling
Requirements
Shears, a needle, a pin and a sewing machine are each worked from the hand, with the business end against a piece of cloth that the other hand is steadying or feeding. First-person capture keeps both hands and the fabric in frame, and the two wrist cameras add a closer view of the hands than the head camera alone. 3D hand pose carries a per-joint visibility flag, so a finger hidden under a fold of cloth is flagged rather than guessed.
The site’s skill-spec guide says to name the verb and the noun rather than a broad activity. A spec for this family names the step and the item, such as measure and pin a hem, cut a pattern piece, thread a lace through an eyelet or fasten a row of buttons, and where one completed step starts and ends, so each can become its own labeled segment.
The site’s household chores dataset page says deformable objects dominate that data, so instance masks are shipped per frame rather than propagated, and that annotation cost per hour is the highest in the catalog. Fabric is the same kind of problem. A mask that follows a sleeve, a hem or a lace has to be re-drawn as the cloth moves, and the spec decides where masks are called for.
An item is rejected when the manipulated object is out of frame for too much of the episode, when depth is invalid over too much of the manipulation region, or when an undocumented bystander appears. The thresholds are agreed in the spec, so the batch is judged against terms you signed.
Definition
Egocentric sewing and textile handling video data is first-person video of a person cutting, pinning, pressing, stitching or otherwise handling fabric, recorded from the contributor’s own point of view with synchronized sensor streams alongside. The camera sits on the head, close to what a robot’s onboard camera would see while it works a piece of cloth, so the frames show the needle or the shears, the fabric in front of them, the fingers pinching and guiding it and the seam or edge being followed, rather than a distant view of someone at a table. On this site it is a custom collection: you name the task, the material, the tool and the setting, and capture is run to a written skill spec.
The data is human capture. A paid, consented contributor wears the rig and does the task. It is not robot or teleoperation data, and it is not a fixed corpus you download. This page covers a different subject from egocentric laundry folding data, which centres on folding clean laundry, from egocentric ironing and garment handling data, which centres on pressing, hanging and sorting garments, and from egocentric deformable object data, which covers soft shape-changing items in general. The subject here is making and altering textile goods: cutting fabric, fitting and hemming, stitching by hand or machine, and the small fastening tasks of buttons, zippers and laces. It covers what the site states about capturing, labeling and checking those tasks.
People reach this topic through close cousins: sewing robot training data, a textile handling video dataset, egocentric fabric manipulation data, or custom sewing data collection. They describe one request from different angles, and the work is the same: write down the task, collect it on demand, and receive validated episodes. Nothing on this page states that the video carries a fabric-type label, a stitch count or a seam-quality measurement, or a force or tactile reading.
Why it matters
A published robot foundation model paper, GR00T N1, linked in the sources below, describes a data pyramid with web data and human video at the base, synthetic data in the middle and real robot trajectories at the top. First-person human video of sewing and fabric handling sits in that base layer. Human capture does not replace robot data, and this page does not claim that it does.
Cloth is a hard case because it has no fixed shape, and sewing adds order on top of that softness. The catalog describes one row as measuring, pinning, pressing and stitching hems on trousers and skirts by hand or machine, and another as laying out fabric, placing patterns and cutting pieces with shears or a cutting machine. The order matters, and so do the hands: the site’s tool use page says that grip changes live in finger-level detail that RGB video alone cannot resolve. A clip that shows a garment somewhere in frame does not carry what a policy needs. The hands, the fabric and the tool have to be present in the episode, and the episode has to say where one step ends and the next begins.
Breadth matters as much as volume. A published model write-up, GEN-0, also linked in the sources, puts it this way: "data quality and diversity matters more than sheer volume." For sewing that points at the spread of materials, tools and steps. A custom collection lets you aim at the step your model is missing, such as threading a lace or fastening a button, rather than hoping a broad collection happens to contain it. This page does not claim that any public corpus lacks sewing or textile data.
Sewing and fabric tasks
The first table below reproduces all nine rows of the site’s commercial task catalog for the textile arts and stitching studio, an environment the catalog describes with examples of hand embroidery, tailoring studio, quilting, needlepoint, custom garment fitting and hemming. Each row is listed with a skill level, a capture setup and a status exactly as the catalog states them. Six rows are ready to deploy and three, crochet, embroidery and knitting, are available on request.
Fitting and tailoring are the hard rows. "Custom garment fitting" pins and marks a garment on the client and adjusts seams and length for fit, and "Tailoring" cuts, fits and sews garments to measure with machine and hand finishing. Hemming, hand embroidery, needlepoint and quilting are medium rows: hemming measures, pins, presses and stitches hems on trousers and skirts by hand or machine, hand embroidery stitches decorative patterns into fabric with needle and embroidery thread, needlepoint stitches yarn through canvas in a counted pattern with a tapestry needle, and quilting cuts and pieces fabric blocks, layers them with batting and quilts the layers together. Crochet, embroidery and knitting are described in the catalog as production of garments and textile goods.
The second table covers the catalog’s textile and apparel production setting, whose listed examples include a sewing workshop and a fabric cutting facility, and three leather stitching rows from the leathercraft workshop. "Garment sewing" is a hard row that feeds fabric pieces through an industrial sewing machine to join seams and attach parts, "Fabric cutting" lays out fabric, places patterns and cuts pieces with shears or a cutting machine, and "Finishing and pressing" presses seams and finished garments and folds or hangs them. All four production rows are available on request. The site states nothing about a factory line, a throughput or a machine model, and it lists capture at your own facility as not offered, so this page makes no such claim.
The third table covers eight rows from the cloth bench, which the catalog describes as a cloth-and-cable bench with garments, towels, cables, form boards and bags. They are small bench-scale tasks: fastening buttons and snaps, running a zipper, threading a lace through eyelets, inserting a duvet into its cover, flattening a sheet and smoothing wrinkles, folding shirts and towels, hanging garments and pairing socks. The bench rows carry a different capture setup from the studio rows, a multiview rig with handheld grippers, and all are available on request. The other four bench rows are cable and packaging tasks and are not covered here.
The site does not describe a fabric type, a fabric weight, a sewing-machine model, a stitch type or a seam allowance for any row, so none is stated here. A task outside these rows is settled in a Spec Pilot rather than assumed.
Episodes
Each attempt is delivered as one episode, and a sewing task needs a decision up front about what one attempt is: one hem, one button, one seam, one pattern piece cut or one measure-to-finish sequence. The site does not set that length, so the skill spec does. Inside the episode, the synchronized streams run from start to end, and action segments divide it into labeled spans.
A hemming episode might open with measuring and pinning the edge and close with the last stitch, and a cutting episode might run from laying out the fabric to the last piece cut, as the catalog rows describe. These are illustrations of how a spec can cut an episode, not stated durations. Whether pressing a hem is part of the hemming episode or its own, and whether a fitting runs as one episode or several, are spec decisions.
The environments behind these rows are a textile arts and stitching studio, a textile and apparel production setting, a leathercraft workshop and a cloth bench. Which of those apply to your program is set in the spec, along with how many materials, tools and steps you want covered. The site lists capture at your own facility as not offered.
Capture setup
A sewing collection uses the same capture stack as every Firsthand program. The instrumented tier is a head-mounted stereo rig that records first-person RGB, with a metric depth stream recovered from the calibrated stereo rig rather than from a standalone depth sensor, and 6-DoF IMU data at 200 Hz from head- and wrist-mounted sensors. The wider stack is ten streams: a 4K head camera, two wrist cameras, depth, IMU, 3D hand and body pose, instance segmentation and action labels, hardware-aligned under 2 ms. The two wrist cameras give a closer view of the hands than the head camera alone, which matters where one hand pinches a seam and the other feeds the cloth.
Every stream is hardware-triggered onto one PTP clock domain, so depth, motion and hand pose line up with the same video frame rather than being approximated afterwards. The rig is calibrated twice per session, before capture starts and after it ends, and an episode is accepted only if RMS reprojection stays under 0.28 px. The catalog lists the capture setup per task: a head-mounted phone or multiview rig for the studio, production and leathercraft rows, and a multiview rig with handheld grippers for the cloth bench rows. The site does not explain the grippers further. Which tier applies to your program is set in the skill spec.
The site does not state a camera count, resolution or frame rate specific to sewing. Force and tactile readings are not among the streams, nothing measures thread tension, needle position or stitch length, and no robot or teleoperation data is captured.
Labels
Inside each episode, action segments divide the recording into start- and end-timestamped spans, each carrying start_ns, end_ns, verb and noun. For this task family a segment can be one step as written in the skill spec, so the hemming row becomes a run of labeled steps such as measure, pin, press and stitch rather than one long clip, and a cutting row can separate laying out from cutting. The other standard fields are hand_joints and hand_visible for 3D hand pose, contact_events for the moment the hand meets an object, objects, and instance_masks, stored at the depth resolution of 848×480 so they line up with the depth stream.
Hand pose is delivered for both hands with a per-joint visibility flag for occluded fingers. That is the label that carries how the hands pinch, hold and feed cloth and how they move through a seam. The site describes contact events as the moment the hand meets an item, and it does not say whether a pin, a fold of fabric, a needle or a button counts as the item, so the spec is the place to settle what a contact event means for your task.
The site does not list labels for the fabric or the stitching itself. There is no stated label for fabric type or weight, stitch type or count, seam straightness or strength, hem length, a garment-fit measure, or a task-completion score, and no contact force or tension reading. Catalog descriptions such as pinning a garment on the client or feeding fabric pieces through a sewing machine describe the task the contributor performs, not a delivered measurement.
Stated limits
Several limits the site already states apply directly to sewing capture, and several it leaves open. Each stated limit below appears elsewhere on the site rather than being introduced here.
Hand occlusion is the first. Fingers disappear behind objects in manipulation, which is why 3D hand pose carries a per-joint visibility flag, and cloth folds over fingers constantly. Masks are the second. The household chores dataset page says instance masks for deformable objects are shipped per frame rather than propagated, which is what a shape-changing material needs. Depth is the third. The site says depth degrades on wet and reflective surfaces and handles that with per-frame confidence maps, but it does not say how depth behaves on a particular cloth, a sheen or a loose weave.
The fourth is the set of machine-checked reject criteria. An episode is rejected for a sync breach beyond the ceiling for more than three consecutive frames, calibration drift beyond tolerance, the manipulated object out of frame for more than 15 percent of the episode, more than 35 percent invalid depth pixels in the manipulation region, an undocumented bystander, or a mismatch against the named skill spec. A human reviewer then checks each episode against the task spec. For a sewing task, the spec decides what the manipulated object is: the cloth, the garment or the tool.
The tool use page adds that metric depth on sub-centimetre parts is flagged rather than interpolated. Whether a needle, a pin, a button or a stitch counts as such a part is not stated. Also not stated: a mask accuracy figure for thread, needles or fabric edges, and any description of how a piece of cloth held under a machine foot is handled. If your policy depends on any of those, say so at the start.
QA, consent and safety
Validation runs against signed accept criteria written into the spec before capture. Each batch goes through per-batch QA and a reject log ships with it. Only footage that clears the criteria is billed as a validated hour, and rejected items are recaptured at our cost. Many teams start with a Spec Pilot, a small, fast first batch that proves the spec and the accept criteria on real footage before volume scales.
Every contributor signs a written release before capture, and each delivered episode links to that record through consent artefact IDs in the batch data card. The consent chain has six checkpoints, from before capture to after delivery, and a break at any one discards the footage rather than shipping it at a lower grade. At ingest, automated detection finds faces, plates and screens and blurs them irreversibly, and speech is scrubbed of names, addresses and card numbers while ambient audio is kept. A human reviewer confirms that no un-consented person is identifiable in any frame, and footage that cannot be de-identified is discarded. The anonymization record is delivered with the data.
One catalog row needs a decision here. "Custom garment fitting" pins and marks a garment on the client, which puts a second person in the frame. The same rule applies to them as to any capture: an un-consented person who is identifiable in a frame means the footage is de-identified or discarded, so settle who is being fitted, and how they are consented, in the spec.
On safety and qualification, the site states no contributor safety protocol, no guidance on needles, shears, cutting machines or pressing equipment, and no trade certification for any sewing row, and this page does not describe one. If your program needs terms about who performs the work, the materials used or the conditions under which it is done, raise them when you scope it. None of this is legal advice, and the executed agreement controls.
Delivery and speed
From signed spec to first validated batch, the timeline is a median of 48 hours where coverage is deep and three to four weeks where it is thin and contributors need to be recruited first. Speed comes from a warm pipeline, not from cutting consent, review or anonymization. The site does not say whether the sewing and textile settings count as deep or thin coverage, so ask which timeline applies to the setting you want when you request the quote. After the first batch, on-demand top-ups run against the same accept criteria, so you can add more materials, tools or steps without rewriting the spec. The site does not state a turnaround for those later batches, so plan them in your quote.
Delivery is in the format your stack already reads: RLDS, LeRobot, WebDataset, HDF5, zarr or Rerun .rrd, or raw streams with metadata. Readable converter source ships alongside the data so the schema can be retargeted to your own loader. Collections are billed per validated hour.
The license is perpetual, irrevocable and worldwide, with the data buyer-owned and non-exclusive by default. Training, evaluating, fine-tuning and deploying models, including commercial ones, is permitted, and exclusivity is available as a separate term. The executed agreement controls. The site does not publish a price list, a minimum order or a batch size for this task family, so those are set in your quote.
Not stated
Where this site does not state something, this page says "not stated" rather than filling the gap. The table below separates what is delivered from what is not stated. There is no listed label for fabric type, stitch quality, seam strength or garment fit, no force or tactile stream, no sewing-machine model or fabric specification, and no factory, production-line or mill claim beyond the catalog rows quoted above. Nothing here claims a training outcome or a model improvement.
The tradeoff to plan for is the embodiment gap between a human hand pinching and feeding cloth and a robot platform doing the same, which retargeting or pairing with robot data has to close. Counts of contributors, hours and tasks are set in your skill spec.
Getting started
Describe the materials, the tools, the setting and the variation or failure cases your pipeline is missing. That becomes a written skill spec, quoted for reach, timeline and price before any capture begins, and most programs run a Spec Pilot first. The scoping checklist below maps the parts of a skill spec onto a sewing request. A published sample recording is on the sample data outputs page if you want to see the streams before you write a spec.
Three choices matter most for this family. Name the step and the tool rather than a broad activity, because that is what lets each step become one action segment. Name the setting and the capture tier, since the studio, the production setting and the cloth bench are listed with different capture setups for some rows. And name where the episode starts and stops: one hem, one seam, one button or one full measure-to-finish sequence, since the site does not set an episode length.
Capture setup
| Part | As stated on this site |
|---|---|
| Instrumented rig | Head-mounted stereo rig worn by the contributor; first-person RGB |
| Streams | Ten: 4K head camera, two wrist cameras, depth, IMU, 3D hand and body pose, instance segmentation, action labels |
| Sync | Hardware-triggered onto one PTP clock domain; aligned under 2 ms |
| Depth | Metric depth recovered from the calibrated stereo rig; per-frame confidence maps are shipped; degrades on wet and reflective surfaces |
| IMU | 6-DoF at 200 Hz from head- and wrist-mounted sensors |
| Calibration | Twice per session; accepted only under the 0.28 px RMS reprojection gate |
| Masks | Per-frame instance masks where the spec calls for them, stored at 848×480; shipped per frame, not propagated, for deformable objects |
| Catalog capture setup | Head-mounted phone or multiview rig for the studio, production and leathercraft rows; multiview rig with handheld grippers for the cloth bench rows |
Every collection runs through the same seven-stage end-to-end custom collection pipeline.
Task variations
| Task (as listed) | Description (as listed) | Environment | Skill level | Capture setup | Status |
|---|---|---|---|---|---|
| Tailoring | Cut, fit and sew garments to measure with machine and hand finishing. | Textile Arts & Stitching Studio | Hard | Phone (head-mounted) / multiview rig | Ready to deploy |
| Custom garment fitting | Pin and mark a garment on the client and adjust seams and length for fit. | Textile Arts & Stitching Studio | Hard | Phone (head-mounted) / multiview rig | Ready to deploy |
| Hemming | Measure, pin, press and stitch hems on trousers and skirts by hand or machine. | Textile Arts & Stitching Studio | Medium | Phone (head-mounted) / multiview rig | Ready to deploy |
| Hand embroidery | Stitch decorative patterns into fabric by hand with needle and embroidery thread. | Textile Arts & Stitching Studio | Medium | Phone (head-mounted) / multiview rig | Ready to deploy |
| Needlepoint | Stitch yarn through canvas in a counted pattern with a tapestry needle. | Textile Arts & Stitching Studio | Medium | Phone (head-mounted) / multiview rig | Ready to deploy |
| Quilting | Cut and piece fabric blocks, layer with batting and quilt the layers together. | Textile Arts & Stitching Studio | Medium | Phone (head-mounted) / multiview rig | Ready to deploy |
| Embroidery | Machine and hand embroidery production of garments and textile goods. | Textile Arts & Stitching Studio | Medium | Phone (head-mounted) / multiview rig | Available on request |
| Knitting | Hand and machine knitting of garments and textile products. | Textile Arts & Stitching Studio | Medium | Phone (head-mounted) / multiview rig | Available on request |
| Crochet | Producing crocheted garments and textile products to commercial order. | Textile Arts & Stitching Studio | Medium | Phone (head-mounted) / multiview rig | Available on request |
Production and leather rows
| Task (as listed) | Description (as listed) | Environment | Skill level | Capture setup | Status |
|---|---|---|---|---|---|
| Garment sewing | Feed fabric pieces through an industrial sewing machine to join seams and attach parts. | Textile & Apparel Production | Hard | Phone (head-mounted) / multiview rig | Available on request |
| Fabric cutting | Lay out fabric, place patterns and cut pieces with shears or a cutting machine. | Textile & Apparel Production | Medium | Phone (head-mounted) / multiview rig | Available on request |
| Finishing and pressing | Press seams and finished garments on an industrial iron or press and fold or hang them. | Textile & Apparel Production | Easy | Phone (head-mounted) / multiview rig | Available on request |
| Textile mill operation | Load yarn or fabric onto mill machines, monitor runs and clear faults. | Textile & Apparel Production | Medium | Phone (head-mounted) / multiview rig | Available on request |
| Hand-stitching leather goods | Punch stitch holes and saddle-stitch leather pieces with two needles and waxed thread. | Leathercraft Workshop | Hard | Phone (head-mounted) / multiview rig | Available on request |
| Leather bag and wallet making | Cut leather panels, skive edges, glue and stitch them into a bag or wallet. | Leathercraft Workshop | Hard | Phone (head-mounted) / multiview rig | Available on request |
| Saddlery | Cut, stitch and fit leather straps and saddle parts with heavy-duty tools. | Leathercraft Workshop | Hard | Phone (head-mounted) / multiview rig | Available on request |
Cloth bench rows
| Task (as listed) | Description (as listed) | Environment | Skill level | Capture setup | Status |
|---|---|---|---|---|---|
| Fasten buttons/snaps | Fasten a row of buttons or snaps. | Deformables & Textiles Bench | Hard | Multiview rig + handheld grippers | Available on request |
| Insert duvet cover | Insert a duvet into its cover. | Deformables & Textiles Bench | Hard | Multiview rig + handheld grippers | Available on request |
| Thread lace eyelets | Thread a lace through eyelets. | Deformables & Textiles Bench | Hard | Multiview rig + handheld grippers | Available on request |
| Open/close zipper | Start and run a zipper closed. | Deformables & Textiles Bench | Medium | Multiview rig + handheld grippers | Available on request |
| Fold shirts + towels | Fold garments and towels to standard. | Deformables & Textiles Bench | Medium | Multiview rig + handheld grippers | Available on request |
| Match + pair socks | Match socks and pair them. | Deformables & Textiles Bench | Medium | Multiview rig + handheld grippers | Available on request |
| Flatten + smooth fabric | Flatten a sheet and smooth wrinkles. | Deformables & Textiles Bench | Easy | Multiview rig + handheld grippers | Available on request |
| Hang + space garments | Hang garments and space them. | Deformables & Textiles Bench | Easy | Multiview rig + handheld grippers | Available on request |
Where sewing is named
| Where | Wording on the site | What it tells you |
|---|---|---|
| Commercial task catalog: textile arts and stitching studio | "Hemming": "Measure, pin, press and stitch hems on trousers and skirts by hand or machine."; the environment lists hand embroidery, tailoring studio, quilting, needlepoint, custom garment fitting and hemming | Nine rows from fitting and tailoring to crochet; six ready to deploy |
| Commercial task catalog: textile and apparel production | "Garment sewing" and "Fabric cutting"; the environment lists a sewing workshop and a fabric cutting facility among its examples | Four rows, available on request; no factory line or throughput stated |
| Commercial task catalog: leathercraft workshop | "Hand-stitching leather goods": "Punch stitch holes and saddle-stitch leather pieces with two needles and waxed thread." | Leather stitching rows, available on request |
| Commercial task catalog: cloth bench | "Fasten buttons/snaps", "Open/close zipper", "Thread lace eyelets" | Bench-scale fastening rows, available on request |
| Egocentric laundry folding and ironing pages | Folding laundry; pressing, hanging and sorting garments | The neighbours; sewing, cutting and hemming are not named there |
| Egocentric deformable object data page | Soft shape-changing items in general, with per-frame instance masks where specified | The neighbour for cloth in general |
| Dataset pages | No sewing skill is listed | Sewing is a custom-collection request, not a stated dataset skill |
| Fabric type, sewing-machine model, stitch quality, factory line or throughput | Not stated | Settled in the skill spec; not claimed here |
Streams and labels
| Stream or label | What it gives a sewing episode | Status |
|---|---|---|
| Head and wrist cameras | First-person view of the task plus a closer view of the hands | Delivered |
| Depth with confidence maps | Cloth and tool geometry, with a confidence map to mask low-confidence regions | Delivered |
| 6-DoF IMU at 200 Hz | Head and wrist motion aligned to the video frame | Delivered |
| 3D hand and body pose, with per-joint visibility flags for the hands | Both hands across the step and the body’s movement; occluded fingers flagged | Delivered |
| Contact events | The moment the hand meets an item | Delivered |
| Action segments (start_ns, end_ns, verb, noun) | One labeled span per step | Delivered |
| Per-frame instance masks (848×480) | Masks where the spec calls for them; shipped per frame for deformable objects | Delivered where specified |
| Consent artefact IDs and anonymization record | Provenance per episode | Delivered |
| Fabric type, weight or fibre | Not listed | Not stated |
| Stitch type or count, seam quality or hem length | Not listed | Not stated |
| Garment-fit measure or task-completion score | Not listed | Not stated |
| Contact force, thread tension or tactile readings | Not among the listed streams | Not stated |
| Safety or trade certification | Not among the listed streams | Not stated |
| Robot or teleoperation data | Human first-person demonstrations only | Not captured |
| Counts of contributors, hours and tasks | Set per program in the skill spec | Not stated |
Scoping checklist
| Skill-spec part | What to name for a sewing request | Drawn from what the site states |
|---|---|---|
| Named skill | The verb and the noun: pin and stitch a hem, cut a pattern piece or fasten a row of buttons, not "sewing" | Catalog wording such as "Measure, pin, press and stitch hems on trousers and skirts by hand or machine." |
| Geometry that must be visible | What must be in frame and for how long: both hands, the fabric, the tool | Set in the skill spec |
| Condition budget in hours | Hours per condition cell, including materials, tools and failure cases | Set per program in the skill spec |
| Reject rule | The conditions that fail an item, written first | Object out of frame for more than 15 percent of the episode; more than 35 percent invalid depth pixels in the manipulation region; an undocumented bystander |
| Acceptance test | One test that decides whether an attempt counts as complete | Your acceptance test is agreed in the spec |
| Setting and capture tier | Stitching studio, production setting or cloth bench, and phone or multiview rig | The catalog lists a capture setup per row |
| Episode boundary | One hem, one seam, one button or one measure-to-finish sequence | Set in the skill spec; the site does not state an episode length |
| Participants and lighting | Handedness, technique and pace; daylight or low light | Set per program in the skill spec |
FAQ
Sources
Tell us the materials, the tools and the settings your model needs to see. We will quote reach, timeline and price against a written spec, with a first validated batch in a median of 48 hours where coverage is deep.