A whole load, from the first box to the last wrap
A useful clip keeps the full sequence, from first box to final wrap, in one episode, with action segments so each placement and wrap pass can be sliced out.
Pallet building and wrapping
Requirements
A useful clip keeps the full sequence, from first box to final wrap, in one episode, with action segments so each placement and wrap pass can be sliced out.
The catalog lists no row for labeling a pallet or strapping a load, and no label for pallet pattern, layer count, load weight, wrap count or film type. This page marks those items "Not stated".
Definition
It is first-person video of a person working a pallet: stacking boxes into a stable pattern, taking a load apart, or wrapping it in stretch film. The camera is head-mounted, with two wrist cameras for a closer view of the hands. The contributor is a paid, consented person following a written skill spec, not an actor and not a robot.
A pallet task is long and repetitive: many placements add up to one load. The value of the data is what the hands do at each step, labeled so a single placement or wrap pass can be sliced out. Where the site states nothing, a table below says "Not stated".
Task coverage
The task catalog already names pallet work. Eight rows name a pallet, and this page reproduces them as the catalog states them.
Three sit in the Warehouse Zone, whose examples read "Conveyor, racking, pallets, totes, a pack bench and scale." "Palletize boxes" is described as "Stack boxes in a stable pallet pattern" and "Stretch-wrap pallet" as "Wrap a pallet with stretch film", both listed as medium skill, and "Depalletize to shelf" is listed as easy. All three use the capture setup "Multiview rig + handheld grippers" and are marked available on request.
Three more sit in the Warehouse & Distribution Center, whose examples read "E-commerce fulfillment hub, distribution center, cold storage facility, freight warehouse." They are the outbound freight row, "Loading Outbound Trucks" and the inbound pallet receiving row, with their descriptions in the table below. All three are listed as medium skill, use "Phone (head-mounted) / multiview rig" and are marked ready to deploy.
Two more sit in the Plastic & Light-Assembly Manufacturer, in the Manufacturing & Workshops sector: "Box and palletize finished goods" and "Boxing & Palletizing Finished Goods", described in the table below. Both are listed as easy, use the same phone or multiview capture and are marked ready to deploy.
Two rows state equipment, a hand truck and a manual pallet jack, repeated here only as listed. The site states no forklift or other vehicle, so none is claimed, and it lists no row for labeling a pallet or strapping a load. Pallet patterns, layer counts, load weights, wrap counts, film types and case or SKU counts are not stated, so they are decisions for the written spec.
Episode contents
An episode is one contributor performing one task from the skill spec, such as building one pallet or wrapping one load. The wider stack is ten streams, including a head-mounted global-shutter 4K/60 camera with a 120 degree lens, two wrist-mounted 1080p/60 cameras, depth, 200 Hz 6-DoF IMUs, 3D hand and body pose, instance segmentation and action labels, hardware-aligned within the 2 ms ceiling the site documents.
Every episode carries int64 nanosecond timestamps per stream, camera intrinsics and extrinsics, hand joints for both hands, instance masks, action segments and contact events. The rig is calibrated at the start and end of every session, and an episode is accepted only if RMS reprojection stays under 0.28 px.
The site states no field for pallet pattern, layer count or load state, no stack-stability label, and no load weight, wrap count or film type.
Labeling and delivery
The unit of labeling is the action segment. The site states a taxonomy of 97 verbs and 512 nouns, annotated start and end to the frame and human-reviewed. A spec that says "palletize the boxes" gives you one long segment. A spec that names pick up, place and square the box, then start the next layer, gives a sequence you can slice by placement.
Hand pose carries a per-joint visibility flag, contact events mark when a hand meets the item, and instance masks are stored where the spec calls for them.
Data is delivered in RLDS, LeRobot, WebDataset, HDF5, zarr or Rerun .rrd, with converters shipped as readable source. You receive a perpetual, irrevocable, buyer-owned commercial license, the consent artefacts for the batch and a data card that lists jurisdictions of capture. Billing is per validated hour, and rejected episodes and recaptures are never billed. The site states no stack-stability, wrap-quality or other success label.
Robot learning
The site states no pallet learning result, so this is plain reasoning about the task, not a finding. A pallet is built from many placements that depend on one another, so one clip holds a long sequence.
Three consequences show up in the data. First, sequence: the order of placements is part of the task, so action segments let one placement be sliced out and kept in order. Second, occlusion: a box or a stacked layer hides fingers, so the visibility flag marks joints that were not seen instead of presenting a guess. Third, wrapping is a continuous motion around the load, and the spec decides how it is split into segments.
Firsthand captures people, not robots. There is no robot joint-angle log, robot result, benchmark or teleoperation data for these tasks, and using the episodes for a specific robot means your own retargeting step. Nothing here claims a training outcome.
The standard capture lists no force or tactile stream, only contact events, and the site states no label for load weight or grip force. A separate glove tier is described on its own page, and this page makes no claim about it for pallet work.
Privacy and settings
The catalog places the pallet rows in a warehouse and distribution center, a warehouse zone with conveyor, racking, pallets and a pack bench, and a plastic and light-assembly manufacturer. The site does not state which sites or countries host capture, or how many, and it states no capture at a buyer’s own warehouse or facility. The spec settles location and access.
Only what the site already states applies to privacy. Every participant signs a written release granting commercial training rights before capture begins. Faces and licence plates are detected and irreversibly blurred, a human reviewer confirms that no un-consented person is identifiable in any frame, audio is scrubbed of identifying speech, and a clip where a bystander cannot be de-identified is discarded rather than shipped. This is not legal advice, and the executed agreement controls.
Speed
The site states one published timeline: from a signed spec to a first validated batch, a median of 48 hours where coverage is already deep, and typically three to four weeks where coverage is thin because operator vetting is the bottleneck.
Five of the eight pallet rows are marked ready to deploy and three available on request. The site does not say which timeline applies to which, so ask when you request the quote.
Getting started
Start with the free, ungated sample pack to check the loader and the label quality, then move to a Spec Pilot: a small first batch against the written spec that proves the fill rate and the reject rate on your conditions before you scale.
A good brief names the pallet task, the order of steps, how each step starts and ends, and what counts as a reject. The site’s skill-spec guide says to name the verb and the noun rather than a broad activity. Then request a custom dataset.
Capture setup
| Part | As stated on this site |
|---|---|
| Head camera | Global-shutter 4K at 60 fps with a 120 degree lens |
| Wrist cameras | Two wrist-mounted cameras at 1080p and 60 fps |
| IMU | 6-DoF at 200 Hz |
| Sync | Hardware-triggered onto a single PTP clock domain, within the 2 ms ceiling |
| Calibration | Re-solved at the start and end of every session; RMS reprojection under 0.28 px |
| Catalog capture setup, ready-to-deploy rows | "Phone (head-mounted) / multiview rig" |
| Catalog capture setup, available-on-request rows | "Multiview rig + handheld grippers" |
| Capture setup for pallet labeling or load strapping | Not stated |
Every collection runs through the same seven-stage end-to-end custom collection pipeline.
Task catalog
| Task (as listed) | Description (as listed) | Setting | Skill level | Capture setup and status |
|---|---|---|---|---|
| Palletize boxes | Stack boxes in a stable pallet pattern. | Warehouse Zone | Medium | Multiview rig + handheld grippers; available on request |
| Depalletize to shelf | Remove from pallet and shelve. | Warehouse Zone | Easy | Multiview rig + handheld grippers; available on request |
| Stretch-wrap pallet | Wrap a pallet with stretch film. | Warehouse Zone | Medium | Multiview rig + handheld grippers; available on request |
| Load and secure outbound freight using a hand truck and stretch wrap | Move pallets and boxes to the dock with a hand truck, stack and stretch-wrap the load. | Warehouse & Distribution Center | Medium | Phone (head-mounted) / multiview rig; ready to deploy |
| Loading Outbound Trucks | Stage pallets and parcels at the dock and load them into the trailer in sequence. | Warehouse & Distribution Center | Medium | Phone (head-mounted) / multiview rig; ready to deploy |
| Receive and unload inbound pallets using a manual pallet jack | Break down inbound pallets with a pallet jack, check contents against the manifest and shelve. | Warehouse & Distribution Center | Medium | Phone (head-mounted) / multiview rig; ready to deploy |
| Box and palletize finished goods | Box finished goods, tape cartons shut, stack them on a pallet and stretch-wrap the load. | Plastic & Light-Assembly Manufacturer | Easy | Phone (head-mounted) / multiview rig; ready to deploy |
| Boxing & Palletizing Finished Goods | Pack products into cartons, build the pallet in layers and wrap it for shipping. | Plastic & Light-Assembly Manufacturer | Easy | Phone (head-mounted) / multiview rig; ready to deploy |
Episode fields
| Field | As documented | Use for pallet tasks |
|---|---|---|
| Hand joints | 21 joints per hand, both hands, with a per-joint visibility flag | Which hand lifts and which hand squares or steadies the box |
| contact_events | The moment the hand meets the item | When a hand takes hold of a box or the film, and when it lets go |
| Action segments | Start, end, verb and noun; 97 verbs, 512 nouns, human-reviewed | One labeled span per pick, place or wrap step |
| Instance masks | Stored at the depth resolution, 848×480; per-frame where the spec calls for them | Which box is which across frames |
| Timestamps and calibration | int64 nanoseconds per stream; intrinsics and extrinsics | Aligning the head and wrist views |
| Pallet pattern, layer count, load weight, wrap count, film type, stack-stability, force or tactile | Not stated | Not stated |
Delivery formats
| Format | Delivered by Firsthand | Maintainer documentation |
|---|---|---|
| RLDS | Listed | Episodes, each containing a dataset of steps |
| LeRobot | Listed | Synchronized MP4 videos (or images) plus Parquet files |
| WebDataset | Listed | Tar files; files of one sample share a basename |
| HDF5, zarr and Rerun .rrd | Listed | Not described on this page |
Not stated
| Topic | On this site | Where it is settled |
|---|---|---|
| A pallet-labeling or load-strapping row | Not listed in the catalog | Written skill spec |
| Pallet patterns, layer counts, load weights and dimensions | Not stated | Scoped in the spec |
| Wrap counts and film types | Not stated | Scoped in the spec |
| Case or SKU counts, number of pallets, episodes, contributors or sites | Not stated | Quote |
| Forklift or other vehicle use | Not stated | Written skill spec |
| Capture at a buyer’s own warehouse or facility | Not stated | Written skill spec |
| Load weight, grip force or tactile data | Not stated for the standard capture | Not offered as a stated stream |
| Stack-stability or quality labels | Not stated | Scoped in the spec |
| Which timeline applies to which row | Not stated | Quote |
| Robot results or teleoperation data | Not stated; Firsthand captures people, not robots | Your own retargeting step |
FAQ
Sources
Tell us the pallet task, the order of steps and what counts as a reject. We will quote reach, timeline and price against a written spec, with a first validated batch in a median of 48 hours where coverage is deep.