Manufacturing Workflow Data for AI and Robotics

Manufacturing workflow data is first-person capture of a real production or assembly procedure — a sequence of fastening, measuring, press-fit, and inspection steps performed in order on a bench or at a production cell — rather than any single one of those steps in isolation. The procedure is written into a skill spec that names the task units, scenes, streams, and accept criteria the collection needs to cover, the same spec format used for any custom order on this site.

This differs from a single named assembly skill: a fastening sequence or a press-fit step captured on its own is assembly task data, while the same steps captured together as one multi-step production procedure, with the sequence and hand-offs between them preserved, is workflow data. Both run through the same tool use and assembly setting and the same capture, QA, and licensing process.

A model trained only on isolated skill clips has no record of how those skills chain together, in what order, or what one step’s completion looks like as the trigger for the next. A production procedure captured as a single first-person workflow preserves that sequence, the hand-offs between steps, and the pacing of a real operator moving through the full task rather than resetting between clips.

Public benchmark datasets for bimanual precision are typically licensed for research only and forbid derivatives, so they cannot legally reach a commercial production model. Custom workflow capture is written against the buyer’s own procedure and shipped under a buyer-owned license that can go into a shipped product.

The procedure, whatever form it exists in today, from a formal SOP document to an experienced operator’s know-how, is described in terms a capture program can act on: which steps make up the workflow, which tool or part variants and bench setups it needs to cover, which streams the training approach consumes (RGB plus depth, 3D hand pose, and contact events for most assembly-bench work), and the accept criteria a captured take has to clear for each step.

Writing the spec is also where ambiguity in the original SOP gets resolved: where a document is vague about a step boundary or a completion check, the spec states it explicitly, so the people reviewing captured footage against the spec and the people who performed the original procedure are judging completion against the same written definition.

Task decomposition breaks the overall procedure into step-sized task units before capture starts, each with its own description, tools or parts involved, and completion check in the written spec. A full run of the procedure is then captured as an episode, and action segments mark where each task unit begins and ends within that episode, so one continuous capture of the whole workflow still yields individually labeled steps.

3D hand pose is captured through reach, grasp, and place for steps defined by a specific grip, tool orientation, or press-fit motion, with contact and grasp events recorded at the point of fastening or fitting. Where a step is prone to going wrong, failure case coverage captures the error and the operator’s recovery as its own labeled segment, rather than only the clean completion path.

A head-mounted camera keeps the fastening, measuring, and fitting motion in frame from the viewpoint a manufacturing policy will actually see from, with depth, 3D hand pose, contact events, and action segments aligned to video on one hardware clock under 2 ms, the same multi-stream setup used for assembly task data on this site.

Small fasteners and fine parts push below the practical resolution of some depth sensors — metric depth on sub-centimetre parts is flagged rather than interpolated, while hand pose stays valid at that range, so the stream boundaries are stated rather than papered over with uncalibrated depth.

A real manufacturing workflow includes the step going wrong — a fastener missed, a part seated incorrectly, a measurement out of tolerance — and the operator catching and correcting it. Where the written spec calls for failure case coverage, those moments are captured deliberately as their own labeled segment within the episode, alongside the clean completion path, rather than discarded as a bad take.

A policy trained only on clean runs of a procedure has nothing to learn from when a step does not go as planned on a real production line; capturing the error and recovery together is what gives a workflow-level model a recovery behavior to fall back on.

Every contributor performing a workflow signs a documented release as part of a consent chain, and that record travels with the data through delivery. Faces, plates, and screens in the capture environment are detected and irreversibly blurred at ingest, and a human reviewer confirms the anonymization result before anything ships, with the anonymization record delivered alongside the data.

Manufacturing workflow capture often happens on a real bench or production cell where other operators, documents, or screens may appear in frame incidentally rather than as the subject of the recording. The same detect-and-blur anonymization step applies to anything caught in frame, not only to the contributor performing the task.

The buyer. Delivered manufacturing workflow data ships under a perpetual, buyer-owned, non-exclusive commercial license by default, with exclusivity available so a given workflow collection is not resold to another buyer. That license is what lets a model trained on the captured workflow ship in a commercial product rather than carrying research-only restrictions.

Provenance and data-processing documentation accompanies the license: the provenance page sets out what is actually delivered and the consent and anonymization checkpoints applied to every episode, and the DPA covers data-processing terms for the engagement.

Finished manufacturing workflow data delivers in RLDS, LeRobot, WebDataset, HDF5, zarr, or Rerun .rrd, or as raw streams with metadata — the same set of formats available on any custom order, chosen to match whatever a training pipeline already consumes. RLDS and LeRobot are built around episodic, step-based sequences, which maps directly onto a procedure already decomposed into task-unit episodes with action-segment labels.

Because the episode and action-segment structure is set at the spec stage rather than reverse-engineered after capture, the same captured workflow can be re-exported into a different one of these formats later without recapturing, if a training pipeline changes.

Start by describing the manufacturing procedure you need captured — whatever form it exists in today — and the steps, tool or part variants, and streams your training pipeline needs. That becomes a written skill spec, and the program typically starts with a Spec Pilot: a small, fast first batch captured against the spec, QA’d against the same accept criteria a full order gets, and delivered with its reject log before you commit to full volume.

A median of 48 hours from signed spec to first validated batch applies where coverage of the bench or production-cell setting is already deep; where the procedure requires contributors recruited into a new environment from scratch, a realistic timeline runs three to four weeks. The same written spec that scoped the pilot scales directly into production volume once it is reviewed and accepted.