How to Capture Enterprise Workflows for AI Training

Capturing an enterprise workflow for AI training means taking a real, already-documented piece of work — an SOP, a training script, a floor procedure — and running it through the same pipeline used for any custom egocentric data order: a written spec, a small pilot batch that proves the spec, scaled capture in batches, per-batch QA against the written accept criteria, and licensed delivery in a training-ready format. The workflow does not change the pipeline; it changes what goes into the spec.

That is different from buying a generic, off-the-shelf dataset of similar-looking tasks. A generic dataset was captured against someone else’s spec, for someone else’s environment and failure modes. Capturing your own workflow means the spec, the objects, the environment, and the documented failure cases are specific to how your team actually does the job — so the resulting episodes match the distribution a model deployed on that job will actually see.

For the service description and governance details — documented consent, procurement-reviewable paperwork, perpetual buyer-owned licensing — see /custom-collection/enterprise-workflow-video-data and /custom-collection/enterprise-egocentric-data-collection. This guide covers the methodology: how to go from an existing SOP to delivered, trainable episodes.

Not every workflow is worth capturing yet. A workflow is a good first candidate when it happens often enough that collecting many examples of it is realistic, when it has a clear start and end rather than blending into the next task, and when a reviewer can tell a correct outcome from a mistake by looking at the footage.

• Frequency: does the task happen often enough to collect many examples of it?
• Clear boundaries: does it have an observable start and end, not a blend into the next task?
• Named outcome: can a reviewer tell a correct result from a mistake by looking at the footage?
• Known failure modes: can the team already name how the task goes wrong in practice?

The underlying method for picking and naming these task units is the same one used to turn any human job into robot-trainable units: observe the work as it is actually done, in its real environment; group the actions into units that each have a clear start, end and outcome; name each unit by verb and object rather than by job title; and note how each unit actually fails in practice, since those failure modes become their own condition cells later. /guides/turning-human-jobs-into-robot-training-tasks covers that decomposition step in full — this guide picks up once you have a named task unit and need to turn it into a written spec and a capture program.

A skill spec is the written contract that turns "capture our workflow" into something measurable cell by cell. It has five parts: the named skill (verb and object, not a job title), the geometry that must stay in frame, an explicit hour budget per condition, the reject rule, and one acceptance test a reviewer can apply. /guides/how-to-write-a-skill-spec covers the full template and a worked example; what changes when the spec comes from an existing SOP is where each field’s content comes from.

An SOP is already a decomposed list of steps, which makes naming the skill straightforward — the document’s own step headings are usually close to the verb-object names a spec needs. The geometry requirement and the condition budget come from watching the work actually happen: which hand, tool, or screen has to be visible, and which shift, lighting, or line-speed variations the floor actually sees. The reject rule and the acceptance test come from asking supervisors what a bad take looks like — the SOP’s own "common mistakes" or inspection notes are usually a direct source for both.

Once a skill is named, each recorded performance of it is captured as an episode — a single bundle of metadata, synchronized sensor streams, and annotations. /glossary/episode covers the full structure. For a workflow capture, the practical implication is that a long shift is not one episode; it is many episodes, one per performance of the named skill, each with its own start and end.

Inside each episode, action segments mark where one step ends and the next begins — reach, grasp, move, place, release — labeled with the same verb-noun vocabulary the skill spec uses. /guides/action-segmentation-for-robot-learning covers those labels in detail. This is what lets a model learn the structure of the task instead of a blur of motion, and it is what makes a long procedure usable for training rather than just a video of someone working.

Failure cases are handled the same way, not edited out. The occlusion angles, lighting conditions, and recovery scenarios an SOP’s own "common mistakes" notes describe are named in the spec before capture starts, recorded uncut during collection, and labeled as their own action segment — a separate segment for the moment things went wrong and a separate segment for the recovery that followed. That structure is what keeps a messy but real episode counted as a validated hour instead of discarded.

Capturing a real workflow means recording real people doing their actual job, which makes consent and anonymization non-negotiable parts of the pipeline rather than an afterthought. Every contributor and operator signs a documented consent release as part of the capture process, and that record ships with the data rather than being kept as a one-time assurance behind the scenes.

Faces, license plates, and screens that appear in frame — including bystanders who were not the subject of the capture — are detected and irreversibly blurred at ingest, with a human reviewer confirming the anonymization before anything ships. For an enterprise engagement, that consent chain and anonymization record sit alongside a provenance record and, where needed, a data processing agreement, so your legal and procurement teams can review the same documentation your data science team is using.

See /provenance and /dpa for what that documentation covers.

Before committing to full-volume capture of a workflow, a Spec Pilot runs a small, fast first batch against the written spec. It proves the fill rate and reject rate on your facility type and your named conditions before you scale, so a gap in the spec — a condition that was budgeted wrong, a geometry requirement that is not actually followed on the floor — shows up on a handful of episodes rather than after a full batch has already been captured.

For an enterprise program, a first batch under deep coverage ships at a median of 48 hours from signed spec to validated delivery; where coverage is thin and contributors need to be recruited from scratch, a realistic timeline runs three to four weeks. Both sides look at the same fill-rate report against the same spec document when deciding whether to scale.

Every batch runs through the same quality review used across every program: each item is checked against the spec’s reject rule and acceptance test, synchronization and calibration are verified, and anonymization is confirmed by a human reviewer before anything ships. An item that fails any of those checks is rejected and recaptured, rather than shipped with a caveat.

Because billing is per validated hour — footage that actually passed sync and QA against the spec, not per raw hour recorded — the accept criteria written into the spec are also the commercial terms of the batch. That is why the reject rule and acceptance test need to be concrete enough for a reviewer to apply without a meeting: they are doing double duty as the quality bar and the delivery contract.

Captured workflow episodes ship in the same training-ready formats used across the site — RLDS, LeRobot format, WebDataset, or HDF5 — each a different projection of the same episode content: metadata, synchronized streams, annotations and calibration. Which format to use depends on the training pipeline consuming the data; see /formats for the full breakdown.

For an enterprise workflow engagement, delivery is under a perpetual, buyer-owned, non-exclusive commercial license by default, with exclusivity available. /custom-collection/pricing covers how a custom order like this is scoped and quoted.