Egocentric Packing and Kitting Video Data

Packing and kitting sit at the end of almost every physical workflow: an order is picked, then packed; parts are gathered, then kitted; a product is finished, then boxed. A policy that can only pick and place an object has no signal for the steps that follow, such as choosing where an item sits in a container, filling the gaps, closing the flaps, taping the seam or applying the label.

Those steps are mostly contact-rich and two-handed, and they happen inside a container that hides what the hands are doing from any camera that is not close to the operator. A first-person capture keeps the hands, the item and the container in the same frame, which is the viewpoint a robot arm with a head or wrist camera would act from.

Treating packing and kitting as a task family, rather than as a single step inside a warehouse or factory page, lets a buyer ask for exactly the sequence they need: a carton pack with void-fill and tape, a mailer, a bagged item with a tag, or a tray of parts picked to a bill of materials.

The task catalog already lists packing and kitting work across several environments. In logistics it includes packing orders into boxes with filler, taping them shut and applying the shipping label, packing a carton with void-fill and tape, bundling a multi-item order, weighing and sealing a package, and bagging an item with a tag. In manufacturing it includes packing finished units into boxes with inserts, sealing and labeling them for dispatch, wrapping an item in paper and folding it, bagging and sealing soft goods, and kitting parts to a tray per a bill of materials.

Smaller-scale packing appears in retail, food and office settings: bagging groceries by weight and fragility, packing a lunchbox into compartments, packing a mailer and taping it shut, stuffing and sealing envelopes, and packing graded produce into crates. Palletizing boxes and stretch-wrapping a pallet are listed as well, and sit at the far end of the pack line.

A custom spec is scoped from that same vocabulary. You describe the items, containers, stations and sequence your policy needs, and the written spec covers them, with the catalog tasks as the starting point rather than a fixed menu.

The catalog places packing tasks at pack benches and packing stations in a warehouse and distribution center, in a warehouse zone with a conveyor, racking, pallets and a pack bench, in a retail floor set with a pack bench, at the end of a production line in a plastic and light-assembly manufacturer, in a food and beverage manufacturing plant, and at a precision assembly bench for electronics kitting. Each task row carries a status that says how soon capture can begin.

The site defines those statuses plainly. Ready to deploy means contributors and site access are set up and capture starts once a spec is signed. Active collection means recording is running now, so a spec can join an existing capture stream. Available on request means the work is scoped and recruited to spec when a buyer asks for it. In the current catalog the warehouse and distribution center packing rows, the production-line packing rows and the food plant packing and sealing row are marked Ready to deploy, while the kitting, bundling and bench-set packing rows are Available on request.

The catalog names two capture modes. In the first, a head-mounted phone records the first-person view of real work on site, and a multiview rig adds synchronized third-person angles when the spec asks for them. In the second, a staged working set is recorded by a multiview rig while the operator performs the task with handheld grippers, so demonstrations map onto a robot end effector.

A head-mounted camera keeps the reach, place and seal motion in frame from the viewpoint a manipulation policy will see from, with a wrist-mounted camera available alongside it for close-range detail when that matters more than the full-arm view. Depth, 3D hand pose, contact events and action segments are aligned to video on one hardware clock, synced under 2 ms. Distance-invalid pixels beyond the depth sensor reliable range are flagged rather than interpolated.

The site does not state a camera count, resolution or frame rate specific to packing, so those are agreed in the written spec rather than assumed.

Each episode follows the same episode schema used across Firsthand custom collections: ten streams synced under 2 ms, with head and wrist RGB, depth, 3D hand pose with per-joint visibility, contact and grasp events, and frame-accurate action segments. Warehouse collections also state instance segmentation alongside action segments and grasp and contact events.

For packing and kitting, hand pose tracks the operator through the fold, tape, place and seal motions, contact events mark when fingers make and break contact with the item or the container, and action segments mark the start and end of each step in the sequence. Because the labels share one clock with the video and depth, a step boundary lines up with the exact frame where it happens.

Object labels use the terms your spec sets: the skill spec decides which object classes are in scope before capture, so labels follow the vocabulary your training stack expects rather than a generic taxonomy.

Box or carton identity, item identity, item weight, box dimensions, fill level, a seal-quality or tape-seam label, and a barcode or SKU label attached to each item are not stated anywhere on the site as standard packing labels. Some catalog task names mention weight or barcodes, such as bagging groceries by weight and fragility, weighing and sealing a package, and applying a barcode label and scanning, but those describe what the operator does, not a label delivered with the episode.

A per-episode success or failure label for a pack or a kit is also not stated. If your policy needs any of these, they are defined in the written spec as annotation depth, and the quote covers them before capture begins. The page does not promise them by default.

Every episode links to a signed contributor release through a documented consent chain, and the consent record is part of the delivery. Contributors are the people who perform the packing and kitting tasks, so their release covers the footage captured of their hands and workspace.

Faces, plates and screens are detected and irreversibly blurred at ingest, and a reviewer confirms the result before anything ships. The anonymization record is delivered with the data, so a buyer can see what was blurred and when. That matters on a pack bench, where coworkers, label screens and shipping paperwork can enter the frame.

Delivery formats are RLDS, LeRobot and WebDataset, plus HDF5, zarr and Rerun .rrd. The first three are the training-loader formats; HDF5 and zarr carry arrays for custom pipelines, and Rerun .rrd gives a multi-stream synchronized playback file for review.

Converter source is readable, so you can retarget the schema to your own loader, and the episode schema page lists the fields. Packing episodes use the same schema as every other Firsthand collection, so a loader that already reads another Firsthand dataset reads a packing dataset without changes.

You describe the items, containers, stations and sequence your policy needs. We turn that into a written spec covering tasks, environments, capture tier, annotation depth and accept criteria, then quote reach, timeline and price against that document before any capture begins.

The collection runs through the same seven-stage end-to-end custom collection pipeline as every Firsthand program: contributor recruiting, capture, per-batch QA, anonymization and licensed delivery. Quality is measured against signed accept criteria, billing is per validated hour, a reject log ships with every batch, and rejected items are recaptured at our cost.

Where coverage is already deep, the first validated batch lands in a median of 48 hours from signed spec. Thin environments typically take three to four weeks, because vetting operators there is the bottleneck, and we quote that timeline up front. The catalog status on each task row tells you which case you are in.

On-demand top-ups run against the same spec and accept criteria after each training run, so new packing data stays consistent with what you already hold. A Spec Pilot, a small and fast first batch, can prove the spec and accept criteria before you scale volume.

Every custom collection ships under a perpetual, buyer-owned commercial license, and exclusivity is available so the data is never resold. The license is documented alongside the consent chain, so a policy trained on the data can go into a product.