Egocentric Barcode Scanning and Labeling Video Data

Egocentric barcode scanning and labeling data is first-person video of a person scanning codes with a handheld scanner, printing labels, applying shipping, product or asset labels and tagging items, recorded from the contributor’s own point of view with synchronized sensor streams alongside. The camera sits on the head, close to what a robot’s onboard camera would see over a pack bench, a sortation lane or a shelf, so the frames show what the hands do with the scanner, the label and the package rather than a distant view of someone standing near equipment. On this site it is a custom collection: you name the task, the setting and the scanning or labeling steps involved, and capture is run to a written skill spec.

The data is human capture. A paid, consented contributor wears the rig and performs the task. It is not robot or teleoperation data, and it is not a fixed corpus you download. This page is narrower than logistics and fulfillment data, which covers the whole flow from receiving to loading, narrower than warehouse robotics data, which covers the broad warehouse skill set, and narrower than packing and kitting data, which centres on building the order. The subject here is the scan-and-label family of tasks and what the site states about capturing, labeling and checking them.

People reach this topic through close cousins: barcode scanning robot training data, a package labeling video dataset, egocentric scanning task data, or custom logistics labeling data collection. They describe one request from different angles, and the work is the same: write down the task, collect it on demand from real benches, lanes and counters, and receive validated episodes. Nothing on this page states that the video carries decoded barcode values, read label text or a scan result.

NVIDIA’s GR00T N1 paper describes a data pyramid with web data and human video at the base, synthetic data in the middle and real robot trajectories at the top. First-person human video of scanning and labeling work sits in that base layer. Human capture does not replace robot data, and this page does not claim that it does.

Scanning and labeling look simple and are not. The hand that aims a scanner or lines up a label has to bring a small code or a flat sticker to the right place on a package, in a workable orientation, and then move on to the next item. A clip that shows a hand somewhere near a parcel does not carry that. The hands, the item, the scanner or label and the moment of contact all have to be present in the episode, and the episode has to say where one step ends and the next begins.

The site already treats these as separate skills. Its warehouse robotics page says single-item pick, barcode scan, carton break-down, pallet stacking, label application and two-person carry each need their own coverage, not one generic "warehouse" label. Its warehouse picking dataset page lists a barcode scan with a handheld gun and label application and print reconciliation among the skills it describes. This page takes the scan-and-label slice of that list and gives it a dedicated spec, so you can ask for the steps your model is missing instead of hoping a broad warehouse collection happens to contain them.

Breadth across tasks matters as much as volume on one of them. Generalist’s GEN-0 write-up makes a point about data in different words: "data quality and diversity matters more than sheer volume." For scanning and labeling that points at the spread of tasks and settings. Labeling outbound boxes at a pack bench, scanning parcels to lanes in a sortation zone, tagging merchandise on a retail floor and applying an asset tag to a hardware unit are different tasks with different geometry, and a corpus that repeats one station does not cover them. The catalog lists scan and label rows across six sectors, which the task section below sets out.

A custom collection lets you aim at the scan, the print or the apply step and the setting your model is missing. This page does not claim that any particular public corpus lacks scanning or labeling work.

The first table below reproduces twenty rows of the site’s commercial task catalog that name a scanner, a barcode, a label or a tag step. Each is listed with a skill level, a capture setup and a status exactly as the catalog states them. It is a selection, not the whole catalog. Other retail rows, such as checkout and returns, damaged-goods sorting and tagging merchandise with a pricing gun, also mention scanning or labeling, and a few healthcare rows label tubes and specimen containers. They are left out of the table because the scan or label step is one part of a wider task, or because they sit in settings this page does not cover.

In a warehouse and distribution center the catalog lists packing and labeling, scanning, packing and labeling a customer order with a handheld barcode scanner, printing labels and applying them to outbound boxes before sorting by carrier or route, labeling boxes and products by type with a handheld label gun, reconciling shelf counts with a barcode scanner, cycle counting and returns processing. All seven are listed as easy, use a head-mounted phone or multiview rig, and are marked ready to deploy.

The warehouse zone set, which the catalog describes as conveyor, racking, pallets and a pack bench, lists applying and aligning shipping labels, stripping and re-labeling for a re-route at a cross-dock, and scanning parcels and sorting them to lanes. Last-mile delivery and courier work lists scanning and collecting parcels at pickup and drop-off, and scanning parcels at the door. The zone rows use a multiview rig with handheld grippers and the last-mile rows a head-mounted phone or multiview rig, and all five are marked available on request.

Beyond logistics, the table lists applying product and barcode labels in a plastic and light-assembly manufacturer, label and scan rows at an electronics assembly bench and a data center and IT hardware bench, a label-maker row at an office document station, a bag tag scan at a mock security checkpoint, price tagging at a retail back of house, a grocery checkout row, and a pharmacy and laboratory labeling row. The manufacturer labeling row and price tagging are marked ready to deploy, and the others are marked available on request.

The catalog is a list of what is captured or available, not a ceiling on what can be briefed. A code type, label format, scanner style, package type or setting outside these rows is a skill-spec question to settle in a Spec Pilot rather than an assumption. The site does not describe a code symbology, a label size, a scanner model or a carton size for any of these rows, so none is stated here.

A scanning and labeling collection uses the same capture stack as every Firsthand program. The instrumented tier is a head-mounted stereo rig that records first-person RGB, with a metric depth stream recovered from the calibrated stereo rig rather than from a standalone depth sensor, and 6-DoF IMU data at 200 Hz from head- and wrist-mounted sensors. The wider stack is ten streams: a 4K head camera, two wrist cameras, depth, IMU, 3D hand and body pose, instance segmentation and action labels, hardware-aligned under 2 ms. The two wrist cameras give a second, closer view of the hands than the head camera alone, which matters when a hand closes around a scanner grip or peels a label from its sheet.

Every stream is hardware-triggered onto one PTP clock domain, so depth, motion and hand pose line up with the same video frame rather than being approximated afterwards. The rig is calibrated twice per session, before capture starts and after it ends, and an episode is accepted only if RMS reprojection stays under 0.28 px. The catalog lists the capture setup per task: a head-mounted phone or multiview rig for the warehouse, last-mile, manufacturing, retail, grocery and pharmacy rows, and a multiview rig with handheld grippers for the warehouse zone, bench, asset-tag, office and checkpoint rows. Which tier applies to your program is set in the skill spec.

On depth, the site states a limit for small parts: small fasteners can fall below a depth sensor’s reliable floor, so metric depth on sub-centimetre parts is flagged rather than interpolated, while hand pose stays valid at that range. That statement is about small fasteners and parts. The site does not state how depth behaves on a flat adhesive label or a printed code, and it does not state at what distance a printed code or label text can be read in the head or wrist frames, so this page does not either. Force and tactile readings are not among the streams, and no robot or teleoperation data is captured.

Each attempt is delivered as one episode. Inside it, action segments divide the episode into start- and end-timestamped spans, each carrying start_ns, end_ns, verb and noun. For this task family a segment can be one step as written in the skill spec, so a pack-out becomes a sequence of labeled steps such as scan, print, apply and sort rather than one long clip. The other standard fields are hand_joints and hand_visible for 3D hand pose, contact_events for the moment the hand meets the item, objects, and instance_masks, stored at the depth resolution of 848×480 so they line up with the depth stream.

Hand pose is delivered for both hands with a per-joint visibility flag for occluded fingers. That is the label that carries the two-handed structure of scanning and labeling: where the hand that steadies the package sits relative to the hand that aims the scanner or smooths the label. Contact events mark when a hand meets the package, the scanner or the label, and action segments mark where one step ends and the next begins.

Masks need a spec decision. The site states that clothing, bedding and bags do not track well with masks propagated frame to frame, so deformable captures ship per-frame instance masks where the spec calls for them. Bags are named in that statement; whether a given package type needs per-frame masks is settled in the spec.

The site does not list labels for the content or the outcome of a scan. There is no stated label for a decoded barcode value, recognised label text, a successful or failed scan, a correctly placed label, a tracking or order number, a scan rate or a throughput figure. Several catalog task descriptions mention checking, verifying or reconciling, but that describes the task the contributor performs, not a delivered measurement.

Several limits the site already states apply directly to scanning and labeling capture, and several it leaves open. Each stated limit below appears elsewhere on the site rather than being introduced here.

Hand occlusion is the first. Fingers disappear behind objects in manipulation, which is why 3D hand pose carries a per-joint visibility flag. A hand closing around a scanner grip or covering a label is an everyday case here. The second is depth on small parts, covered above: metric depth on sub-centimetre parts is flagged, not interpolated. The third is masks, where per-frame instance masks are the stated answer for items that do not track well when propagated, and the spec is where you decide whether a package type needs them.

The fourth is the set of machine-checked reject criteria. An episode is rejected for a sync breach beyond the ceiling for more than three consecutive frames, calibration drift beyond tolerance, the manipulated object out of frame for more than 15 percent of the episode, more than 35 percent invalid depth pixels in the manipulation region, an undocumented bystander, or a mismatch against the named skill spec. A human reviewer then checks each episode against the task spec.

What the site does not state is just as useful to know before you scope. It gives no depth behavior for a flat label or a printed code, no distance at which a code or label text is readable, no mask accuracy figure for parcels, no label-placement or scan-success annotation, and no description of how glare or reflective packaging is handled. If your policy depends on any of those, say so at the start.

Validation runs against signed accept criteria written into the spec before capture. Each batch goes through per-batch QA and a reject log ships with it. Only footage that clears the criteria is billed as a validated hour, and rejected items are recaptured at our cost. Many teams start with a Spec Pilot, a small, fast first batch that proves the spec and the accept criteria on real footage before volume scales.

Every contributor signs a written release before capture, and each delivered episode links to that record through consent artefact IDs in the batch data card. The consent chain has six checkpoints, from before capture to after delivery, and a break at any one discards the footage rather than shipping it at a lower grade. At capture, site permission is on file. At ingest, automated detection finds faces, plates and screens and blurs them irreversibly, and speech is scrubbed of names, addresses and card numbers while ambient audio is kept. A human reviewer confirms that no un-consented person is identifiable in any frame, and footage that cannot be de-identified is discarded. The anonymization record is delivered with the data. The site also states that capture does not take place at a customer’s own facility, and it does not state whether buyer-supplied packaging or label stock can be used, so raise that in the spec.

Labels need a plain statement. The site states that parcel and shipping labels, tote labels, carton markings, mail and barcodes are not listed as blur categories, so this page does not claim they are blurred. The listed categories are faces, plates and screens. If text on items or label content in frame matters for your use, raise it in the spec so the handling is agreed before capture rather than assumed afterwards. Labels and parcels that carry personal details are a particular reason to do that.

On safety and qualification, the site states no contributor safety protocol and no safety certification for any scanning or labeling row, and this page does not describe one. If your program needs terms about who performs the work, the equipment involved or the conditions under which it is done, raise them when you scope it. None of this is legal advice, and the executed agreement controls.

From signed spec to first validated batch, the timeline is a median of 48 hours where coverage is deep and three to four weeks where it is thin and contributors need to be recruited first. Speed comes from a warm pipeline, not from cutting consent, review or anonymization. After the first batch, on-demand top-ups run against the same accept criteria, so you can add more tasks, settings or variation without rewriting the spec. The site does not state a turnaround for those later batches, so plan them in your quote.

Delivery is in the format your stack already reads: RLDS, LeRobot, WebDataset, HDF5, zarr or Rerun .rrd, or raw streams with metadata. Readable converter source ships alongside the data so the schema can be retargeted to your own loader. Collections are billed per validated hour.

The license is perpetual, irrevocable and worldwide, with the data buyer-owned and non-exclusive by default. Training, evaluating, fine-tuning and deploying models, including commercial ones, is permitted, and exclusivity is available as a separate term. The executed agreement controls. The site does not publish a price list, a minimum order or a batch size for this task family, so those are set in your quote.

Where this site does not state something, this page says "not stated" rather than filling the gap. The table below separates what is delivered from what is not stated. There is no listed label for a decoded barcode, recognised label text, scan success, label placement or a tracking number, no scan rate or throughput figure, no force or tactile stream, and no code type, label size or scanner model. Nothing here claims a training outcome or a model improvement.

The tradeoff to plan for is the embodiment gap between a human hand and a robot end-effector, which retargeting or pairing with robot data has to close. Counts of stations, contributors, hours and tasks are set in your skill spec.

Describe the scanning, printing or labeling tasks, the setting, the package types and the variation or failure cases your pipeline is missing. That becomes a written skill spec, quoted for reach, timeline and price before any capture begins, and most programs run a Spec Pilot first. The scoping checklist below maps the parts of a skill spec onto a scan-and-label request. A published sample recording is on the sample data outputs page if you want to see the streams before you write a spec.

Three choices matter most for this family. Name the step and the item rather than a broad activity, because that is what lets each step become one action segment. Name the setting and the capture tier, since a pack bench, a sortation lane and a retail counter present different geometry and the catalog lists a different capture setup for them. And name the lighting and participant mix: daylight coverage fills fastest, while evening, low-light capture is the harder cell to fill.