Filling line operation
Requirements
Operating a line means looking at a control panel or a gauge and then acting on it with the hands. First-person capture keeps the hands and whatever they are reaching toward in the frame, and 3D hand pose carries a per-joint visibility flag, so a finger hidden behind a control is flagged rather than guessed.
The site’s skill-spec guide says to name the verb and the noun rather than a broad activity. A spec for this task names the steps the catalog row words: start the line, read the gauges, adjust a setting, clear a minor jam. It also says where one completed step starts and ends, so each can become its own labeled segment.
Much of the work is watching a machine run and acting only when something needs it. The episode has to carry the quiet stretches as well as the interventions, and the spec decides how long an attempt is and what counts as one.
An item is rejected when the manipulated object is out of frame for too much of the episode, when depth is invalid over too much of the manipulation region, or when an undocumented bystander appears. The thresholds are agreed in the spec, so the batch is judged against terms you signed.
Definition
Egocentric filling line operation data is first-person video of a person running a filling line, recorded from the contributor’s own point of view with synchronized sensor streams alongside. The camera sits on the head, close to what a robot’s onboard camera would see while it works at a line, so the frames show the hands on the panel, the product on the line and the gauges the person is reading, rather than a distant view of someone standing at a machine. On this site it is a custom collection: you name the task, the product and the setting, and capture is run to a written skill spec.
The data is human capture. A paid, consented contributor wears the rig and does the task. It is not robot or teleoperation data, and it is not a fixed corpus you download. This page is a different subject from the egocentric food and beverage manufacturing page, which covers all eight Food & Beverage Manufacturing Plant rows, from the food plant sanitation and washdown page, which centres on the wet cleaning rows, and from visual inspection data, which centres on checking units. The subject here is the two catalog rows that operate a line.
People reach this topic through close cousins: filling line robot training data, a production line operation video dataset, egocentric line monitoring and changeover data, or custom manufacturing data collection. They describe one request from different angles, and the work is the same: write down the task, collect it on demand, and receive validated episodes. Nothing on this page states that the video carries a gauge-reading label, a fault label or a measurement.
Why it matters
A published robot foundation model paper, GR00T N1, linked in the sources below, describes a data pyramid with web data and human video at the base, synthetic data in the middle and real robot trajectories at the top. First-person human video of line work sits in that base layer. Human capture does not replace robot data, and this page does not claim that it does.
Line operation mixes two kinds of action. Some are small, deliberate moves at a panel, such as adjusting a setting. Others are physical, such as clearing a minor jam. The catalog describes the first row as starting the filling line, reading gauges, adjusting settings on the panel and clearing minor jams. The hands matter: the site’s tool use page says that grip changes live in finger-level detail that RGB video alone cannot resolve. A clip that shows a machine running does not carry what a policy needs. The hands, the control and the product have to be present in the episode, and the episode has to say where one step ends and the next begins.
Breadth matters as much as volume. A published model write-up, GEN-0, also linked in the sources, puts it this way: "data quality and diversity matters more than sheer volume." For line work that points at the spread of steps, panels and interventions. A custom collection lets you aim at the step your model is missing, such as clearing a jam, rather than hoping a broad collection happens to contain it. This page does not claim that any public corpus lacks line operation data.
The catalog rows
The site’s commercial task catalog lists two rows for this subject, both in the Food & Beverage Manufacturing Plant environment of the Food & Beverage sector. "Operate and monitor a filling line using control panels and gauges" reads "Start the filling line, read gauges, adjust settings on the panel and clear minor jams." "Production Line Operation" reads "Feed product onto the line, monitor the machines and pack finished goods at the end." Both are rated Medium, captured with a head-mounted phone or multiview rig, and marked Ready to deploy, which the catalog defines as contributors and site access already set up. The first table below reproduces them exactly as stated.
Other catalog rows sit next to this work and are separate listings. The plant’s inspection, packaging, sealing and washdown rows are covered on the food and beverage manufacturing page. Two rows in other environments also involve operating machinery, and neither is a filling line: Operating Heavy Machinery Controls and Textile mill operation. The second table below sets them side by side.
The site does not describe a product, a container, a filler model, a panel layout or a gauge type for these rows, so none is stated here. The catalog has no row named changeover. The nearest wording is adjusting settings on the panel, so a changeover task is settled in a Spec Pilot rather than assumed.
Episodes
Each attempt is delivered as one episode, and a line task needs a decision up front about what one attempt is: a start-up, a monitored run, one jam cleared or one setting adjusted. The site does not set that length, so the skill spec does. Inside the episode, the synchronized streams run from start to end, and action segments divide it into labeled spans.
An episode might open with the line being started and close with a jam cleared, as the catalog row words the work. That is an illustration of how a spec can cut an episode, not a stated duration. Whether a long monitored run is one episode or several is a spec decision.
How many panels, products and interventions are covered is set in the spec. The site states that Firsthand does not capture at a customer’s own facility, so the line used is agreed in the skill spec.
Capture setup
A line operation collection uses the same capture stack as every Firsthand program. The instrumented tier is a head-mounted stereo rig that records first-person RGB, with a metric depth stream recovered from the calibrated stereo rig rather than from a standalone depth sensor, and 6-DoF IMU data at 200 Hz from head- and wrist-mounted sensors. The wider stack is ten streams: a 4K head camera, two wrist cameras, depth, IMU, 3D hand and body pose, instance segmentation and action labels, hardware-aligned under 2 ms. The two wrist cameras give a closer view of the hands than the head camera alone, which matters where fingers work a control.
Every stream is hardware-triggered onto one PTP clock domain, so depth, motion and hand pose line up with the same video frame rather than being approximated afterwards. The rig is calibrated twice per session, before capture starts and after it ends, and an episode is accepted only if RMS reprojection stays under 0.28 px. The catalog lists a head-mounted phone or multiview rig for both rows; a multiview rig adds synchronized third-person angles when the spec asks for them. Which tier applies to your program is set in the skill spec.
The site does not state a camera count, resolution or frame rate specific to line operation. Force and tactile readings are not among the streams, and no robot or teleoperation data is captured.
Labels
Inside each episode, action segments divide the recording into start- and end-timestamped spans, each carrying start_ns, end_ns, verb and noun. For this task a segment can be one step as written in the skill spec, so the catalog row becomes a run of labeled steps such as start the line, read the gauges, adjust a setting and clear a jam rather than one long clip. The other standard fields are hand_joints and hand_visible for 3D hand pose, contact_events for the moment the hand meets an object, objects, and instance_masks, stored at the depth resolution of 848×480 so they line up with the depth stream.
Hand pose is delivered for both hands with a per-joint visibility flag for occluded fingers, 21 joints per hand in 3D. That is the label that carries how the hands press, turn and hold while a panel or a jam is worked. The site describes contact events as the moment the hand meets an item, and it does not say whether a panel control or a gauge counts as the item, so the spec is the place to settle what a contact event means for your task.
The site does not list labels for the machine itself. There is no stated label for a gauge value, a panel state, a setting, a fault, a jam, a fill level, a line speed or a task-completion score, and no contact force reading. The catalog description describes the task the contributor performs, not a delivered measurement.
Stated limits
Several limits the site already states apply directly to line capture, and several it leaves open. Each stated limit below appears elsewhere on the site rather than being introduced here.
Hand occlusion is the first. Fingers disappear behind objects in manipulation, which is why 3D hand pose carries a per-joint visibility flag. Depth is the second. The site says depth degrades on wet and reflective surfaces and handles that with per-frame confidence maps. It does not say how depth behaves on a gauge face or a panel display in particular.
The third is screens. At ingest, automated detection finds faces, plates and screens and blurs them irreversibly. The site does not say how that applies to a panel display or a gauge, so if your task needs a readable display, raise it when you scope the collection rather than assuming it will be legible.
The fourth is the set of machine-checked reject criteria. An episode is rejected for a sync breach beyond the ceiling for more than three consecutive frames, calibration drift beyond tolerance, the manipulated object out of frame for more than 15 percent of the episode, more than 35 percent invalid depth pixels in the manipulation region, an undocumented bystander, or a mismatch against the named skill spec. A human reviewer then checks each episode against the task spec. For a line task, the spec decides what the manipulated object is: the product, the panel control or both.
QA, consent and safety
Validation runs against signed accept criteria written into the spec before capture. Each batch goes through per-batch QA and a reject log ships with it. Only footage that clears the criteria is billed as a validated hour, and rejected items are recaptured at our cost. Many teams start with a Spec Pilot, a small, fast first batch that proves the spec and the accept criteria on real footage before volume scales.
Every contributor signs a written release before capture, and each delivered episode links to that record through consent artefact IDs in the batch data card. At ingest, automated detection finds faces, plates and screens and blurs them irreversibly, and speech is scrubbed of names, addresses and card numbers while ambient audio is kept. A human reviewer confirms that no un-consented person is identifiable in any frame, and footage that cannot be de-identified is discarded. The anonymization record is delivered with the data.
On safety and quality, the site states no contributor safety protocol, no guidance on protective equipment or plant access, no throughput, line-speed or efficiency figure, and no food-safety, quality-system or certification claim for these rows, and this page does not describe one. If your program needs terms about who operates the line, the product used or the conditions under which it is done, raise them when you scope it. None of this is legal advice, and the executed agreement controls.
Delivery and speed
From signed spec to first validated batch, the timeline is a median of 48 hours where coverage is deep and three to four weeks where it is thin and contributors need to be recruited first. Speed comes from a warm pipeline, not from cutting consent, review or anonymization. Both line rows are marked Ready to deploy, but the site does not say whether the food and beverage plant environment counts as deep or thin coverage, so ask which timeline applies when you request the quote. After the first batch, on-demand top-ups run against the same accept criteria, so you can add more steps or interventions without rewriting the spec. The site does not state a turnaround for those later batches, so plan them in your quote.
Delivery is in the format your stack already reads: RLDS, LeRobot, WebDataset, HDF5, zarr or Rerun .rrd, or raw streams with metadata. Readable converter source ships alongside the data so the schema can be retargeted to your own loader. Collections are billed per validated hour.
The license is perpetual, irrevocable and worldwide, with the data buyer-owned and non-exclusive by default. Training, evaluating, fine-tuning and deploying models, including commercial ones, is permitted, and exclusivity is available as a separate term. The executed agreement controls. The site does not publish a price list, a minimum order or a batch size for this task, so those are set in your quote.
Not stated
Where this site does not state something, this page says "not stated" rather than filling the gap. The table below separates what is delivered from what is not stated. There is no listed label for gauge values, panel state, faults or jams, no force or tactile stream, no machine or product specification, and no throughput, line-speed, efficiency or certification claim. Nothing here claims a training outcome or a model improvement.
The tradeoff to plan for is the embodiment gap between a human hand working a panel and clearing a jam and a robot platform doing the same, which retargeting or pairing with robot data has to close. Counts of contributors, hours and interventions are set in your skill spec.
Getting started
Describe the line, the product, the setting and the interventions or failure cases your pipeline is missing. That becomes a written skill spec, quoted for reach, timeline and price before any capture begins, and most programs run a Spec Pilot first. The scoping checklist below maps the parts of a skill spec onto a line operation request. A published sample recording is on the sample data outputs page if you want to see the streams before you write a spec.
Three choices matter most for this task. Name each step rather than a broad activity, because that is what lets each step become one action segment. Name the setting and the capture tier, since the catalog lists a head-mounted phone or multiview rig for both rows. And name where the episode starts and stops: a start-up, a monitored run or one intervention, since the site does not set an episode length.
Capture setup
| Part | As stated on this site |
|---|---|
| Instrumented rig | Head-mounted stereo rig worn by the contributor; first-person RGB |
| Streams | Ten: 4K head camera, two wrist cameras, depth, IMU, 3D hand and body pose, instance segmentation, action labels |
| Sync | Hardware-triggered onto one PTP clock domain; aligned under 2 ms |
| Depth | Metric depth recovered from the calibrated stereo rig; per-frame confidence maps are shipped; degrades on wet and reflective surfaces |
| IMU | 6-DoF at 200 Hz from head- and wrist-mounted sensors |
| Hand pose | 21 joints per hand, 3D; per-joint visibility flag |
| Calibration | Twice per session; accepted only under the 0.28 px RMS reprojection gate |
| Masks | Per-frame instance masks where the spec calls for them, stored at 848×480 |
| Screens | Detected and irreversibly blurred at ingest |
| Catalog capture setup | Head-mounted phone or multiview rig for both line operation rows |
Every collection runs through the same seven-stage end-to-end custom collection pipeline.
Task rows
| Task (as listed) | Description (as listed) | Environment | Skill level | Capture setup | Status |
|---|---|---|---|---|---|
| Operate and monitor a filling line using control panels and gauges | Start the filling line, read gauges, adjust settings on the panel and clear minor jams. | Food & Beverage Manufacturing Plant | Medium | Phone (head-mounted) / multiview rig | Ready to deploy |
| Production Line Operation | Feed product onto the line, monitor the machines and pack finished goods at the end. | Food & Beverage Manufacturing Plant | Medium | Phone (head-mounted) / multiview rig | Ready to deploy |
Neighbouring rows
| Task (as listed) | Description (as listed) | Environment | Skill level | Relation to this page |
|---|---|---|---|---|
| Operate and monitor a filling line using control panels and gauges; Production Line Operation | Start the filling line, read gauges, adjust settings and clear minor jams; feed product onto the line, monitor the machines and pack finished goods | Food & Beverage Manufacturing Plant | Medium | The subject of this page |
| Inspect and remove defective products using an inspection light and gloves | Hold products under the inspection light, spot defects and pull them from the line | Food & Beverage Manufacturing Plant | Easy | Inspection at the line; listed separately |
| QA / Foreign-Object Inspection | Inspect items on the belt for foreign objects and defects and remove anything suspect | Food & Beverage Manufacturing Plant | Easy | Inspection at the belt; listed separately |
| Packaging & Sealing | Fill packages, run them through the sealer and apply labels before casing | Food & Beverage Manufacturing Plant | Easy | End of the line; listed separately |
| Operating Heavy Machinery Controls | Operate levers and control panels of presses, cranes or machinery from the operator station | Heavy Industry & Fabrication | Hard | Heavy machinery, not a filling line; not claimed here |
| Textile mill operation | Load yarn or fabric onto mill machines, monitor runs and clear faults | Textile & Apparel Production | Medium | Textile machinery, not a filling line; not claimed here |
Where line operation is named
| Where | Wording on the site | What it tells you |
|---|---|---|
| Commercial task catalog: food and beverage manufacturing | "Operate and monitor a filling line using control panels and gauges" and "Production Line Operation" | Two Medium rows, both Ready to deploy |
| Egocentric food and beverage manufacturing page | Lists both rows among the eight Food & Beverage Manufacturing Plant rows | The wider plant family |
| Egocentric food plant sanitation and washdown page | Centres on the wet cleaning rows of the same plant environment | The neighbour for the cleaning side of the plant |
| Product, filler or panel model, line speed, throughput, changeover, fault labels or certification | Not stated | Settled in the skill spec; not claimed here |
Streams and labels
| Stream or label | What it gives a line operation episode | Status |
|---|---|---|
| Head and wrist cameras | First-person view of the task plus a closer view of the hands | Delivered |
| Depth with confidence maps | Scene geometry, with a confidence map to mask low-confidence regions | Delivered |
| 6-DoF IMU at 200 Hz | Head and wrist motion aligned to the video frame | Delivered |
| 3D hand and body pose, with per-joint visibility flags for the hands | Both hands across the step and the body’s movement; occluded fingers flagged | Delivered |
| Contact events | The moment the hand meets an item | Delivered |
| Action segments (start_ns, end_ns, verb, noun) | One labeled span per step | Delivered |
| Per-frame instance masks (848×480) | Masks where the spec calls for them | Delivered where specified |
| Consent artefact IDs and anonymization record | Provenance per episode | Delivered |
| Gauge value, panel state or setting labels | Not listed | Not stated |
| Fault, jam or fill-level labels | Not listed | Not stated |
| Product, filler or panel specification | Not listed | Not stated |
| Throughput, line speed, efficiency or success labels | Not listed | Not stated |
| Contact force or tactile readings | Not among the listed streams | Not stated |
| Safety, food-safety, quality-system or certification statements | Not stated for these rows | Not stated |
| Robot or teleoperation data | Human first-person demonstrations only | Not captured |
| Counts of contributors, hours and interventions | Set per program in the skill spec | Not stated |
Scoping checklist
| Skill-spec part | What to name for a line operation request | Drawn from what the site states |
|---|---|---|
| Named skill | The verb and the noun: start the line, read the gauges, adjust a setting or clear a jam, not "running the line" | Catalog wording: "Start the filling line, read gauges, adjust settings on the panel and clear minor jams." |
| Geometry that must be visible | What must be in frame and for how long: both hands, the panel or gauge, the product | Set in the skill spec |
| Condition budget in hours | Hours per condition cell, including products, panels and interventions | Set per program in the skill spec |
| Reject rule | The conditions that fail an item, written first | Object out of frame for more than 15 percent of the episode; more than 35 percent invalid depth pixels in the manipulation region; an undocumented bystander |
| Acceptance test | One test that decides whether an attempt counts as complete | Your acceptance test is agreed in the spec |
| Setting and capture tier | The plant line, and a head-mounted phone or multiview rig | The catalog lists a capture setup for both rows |
| Episode boundary | A start-up, a monitored run or one intervention | Set in the skill spec; the site does not state an episode length |
| Readable displays | Whether a panel display or gauge must be legible in the delivered video | Screens are blurred at ingest; how that applies to panels is not stated |
| Participants and lighting | Handedness, technique and pace; plant lighting | Set per program in the skill spec |
FAQ
Sources
Tell us the line, the panels and the interventions your model needs to see. We will quote reach, timeline and price against a written spec, with a first validated batch in a median of 48 hours where coverage is deep.