Pallet building and wrapping

Egocentric Pallet Building and Wrapping Video Data

A pallet is built one placement at a time, taken apart one box at a time and wrapped in a single continuous motion. We collect custom egocentric video of the pallet tasks this site lists, on demand, to a written skill spec, with synchronized streams, 3D hand pose, action segments, anonymization and a buyer-owned license.
Listed pallet rows
Eight, across three settings
Pattern, wrap and load details
Not stated
First batch
Median 48 h in deep-coverage settings
License
Perpetual, buyer-owned

Requirements

What does pallet work need that a generic box-carrying clip does not?

01

A whole load, from the first box to the last wrap

A useful clip keeps the full sequence, from first box to final wrap, in one episode, with action segments so each placement and wrap pass can be sliced out.

02

A clear line between what is listed and what is not

The catalog lists no row for labeling a pallet or strapping a load, and no label for pallet pattern, layer count, load weight, wrap count or film type. This page marks those items "Not stated".

Definition

What is egocentric pallet building and wrapping data?

It is first-person video of a person working a pallet: stacking boxes into a stable pattern, taking a load apart, or wrapping it in stretch film. The camera is head-mounted, with two wrist cameras for a closer view of the hands. The contributor is a paid, consented person following a written skill spec, not an actor and not a robot.

A pallet task is long and repetitive: many placements add up to one load. The value of the data is what the hands do at each step, labeled so a single placement or wrap pass can be sliced out. Where the site states nothing, a table below says "Not stated".

Task coverage

Which pallet tasks can be captured?

The task catalog already names pallet work. Eight rows name a pallet, and this page reproduces them as the catalog states them.

Three sit in the Warehouse Zone, whose examples read "Conveyor, racking, pallets, totes, a pack bench and scale." "Palletize boxes" is described as "Stack boxes in a stable pallet pattern" and "Stretch-wrap pallet" as "Wrap a pallet with stretch film", both listed as medium skill, and "Depalletize to shelf" is listed as easy. All three use the capture setup "Multiview rig + handheld grippers" and are marked available on request.

Three more sit in the Warehouse & Distribution Center, whose examples read "E-commerce fulfillment hub, distribution center, cold storage facility, freight warehouse." They are the outbound freight row, "Loading Outbound Trucks" and the inbound pallet receiving row, with their descriptions in the table below. All three are listed as medium skill, use "Phone (head-mounted) / multiview rig" and are marked ready to deploy.

Two more sit in the Plastic & Light-Assembly Manufacturer, in the Manufacturing & Workshops sector: "Box and palletize finished goods" and "Boxing & Palletizing Finished Goods", described in the table below. Both are listed as easy, use the same phone or multiview capture and are marked ready to deploy.

Two rows state equipment, a hand truck and a manual pallet jack, repeated here only as listed. The site states no forklift or other vehicle, so none is claimed, and it lists no row for labeling a pallet or strapping a load. Pallet patterns, layer counts, load weights, wrap counts, film types and case or SKU counts are not stated, so they are decisions for the written spec.

Episode contents

What does each pallet episode contain?

An episode is one contributor performing one task from the skill spec, such as building one pallet or wrapping one load. The wider stack is ten streams, including a head-mounted global-shutter 4K/60 camera with a 120 degree lens, two wrist-mounted 1080p/60 cameras, depth, 200 Hz 6-DoF IMUs, 3D hand and body pose, instance segmentation and action labels, hardware-aligned within the 2 ms ceiling the site documents.

Every episode carries int64 nanosecond timestamps per stream, camera intrinsics and extrinsics, hand joints for both hands, instance masks, action segments and contact events. The rig is calibrated at the start and end of every session, and an episode is accepted only if RMS reprojection stays under 0.28 px.

The site states no field for pallet pattern, layer count or load state, no stack-stability label, and no load weight, wrap count or film type.

Labeling and delivery

How is pallet data labeled and delivered?

The unit of labeling is the action segment. The site states a taxonomy of 97 verbs and 512 nouns, annotated start and end to the frame and human-reviewed. A spec that says "palletize the boxes" gives you one long segment. A spec that names pick up, place and square the box, then start the next layer, gives a sequence you can slice by placement.

Hand pose carries a per-joint visibility flag, contact events mark when a hand meets the item, and instance masks are stored where the spec calls for them.

Data is delivered in RLDS, LeRobot, WebDataset, HDF5, zarr or Rerun .rrd, with converters shipped as readable source. You receive a perpetual, irrevocable, buyer-owned commercial license, the consent artefacts for the batch and a data card that lists jurisdictions of capture. Billing is per validated hour, and rejected episodes and recaptures are never billed. The site states no stack-stability, wrap-quality or other success label.

Robot learning

Why is pallet building hard for robot learning?

The site states no pallet learning result, so this is plain reasoning about the task, not a finding. A pallet is built from many placements that depend on one another, so one clip holds a long sequence.

Three consequences show up in the data. First, sequence: the order of placements is part of the task, so action segments let one placement be sliced out and kept in order. Second, occlusion: a box or a stacked layer hides fingers, so the visibility flag marks joints that were not seen instead of presenting a guess. Third, wrapping is a continuous motion around the load, and the spec decides how it is split into segments.

Firsthand captures people, not robots. There is no robot joint-angle log, robot result, benchmark or teleoperation data for these tasks, and using the episodes for a specific robot means your own retargeting step. Nothing here claims a training outcome.

The standard capture lists no force or tactile stream, only contact events, and the site states no label for load weight or grip force. A separate glove tier is described on its own page, and this page makes no claim about it for pallet work.

Privacy and settings

Where is it captured and how are bystanders protected?

The catalog places the pallet rows in a warehouse and distribution center, a warehouse zone with conveyor, racking, pallets and a pack bench, and a plastic and light-assembly manufacturer. The site does not state which sites or countries host capture, or how many, and it states no capture at a buyer’s own warehouse or facility. The spec settles location and access.

Only what the site already states applies to privacy. Every participant signs a written release granting commercial training rights before capture begins. Faces and licence plates are detected and irreversibly blurred, a human reviewer confirms that no un-consented person is identifiable in any frame, audio is scrubbed of identifying speech, and a clip where a bystander cannot be de-identified is discarded rather than shipped. This is not legal advice, and the executed agreement controls.

Speed

How fast can a custom pallet-handling dataset be collected?

The site states one published timeline: from a signed spec to a first validated batch, a median of 48 hours where coverage is already deep, and typically three to four weeks where coverage is thin because operator vetting is the bottleneck.

Five of the eight pallet rows are marked ready to deploy and three available on request. The site does not say which timeline applies to which, so ask when you request the quote.

Getting started

How do you start a pilot?

Start with the free, ungated sample pack to check the loader and the label quality, then move to a Spec Pilot: a small first batch against the written spec that proves the fill rate and the reject rate on your conditions before you scale.

A good brief names the pallet task, the order of steps, how each step starts and ends, and what counts as a reject. The site’s skill-spec guide says to name the verb and the noun rather than a broad activity. Then request a custom dataset.

Capture setup

What is the capture setup for a pallet collection?

The capture setup as stated on this site, applied to pallet tasks. Counts of contributors and hours are set in the skill spec.
PartAs stated on this site
Head cameraGlobal-shutter 4K at 60 fps with a 120 degree lens
Wrist camerasTwo wrist-mounted cameras at 1080p and 60 fps
IMU6-DoF at 200 Hz
SyncHardware-triggered onto a single PTP clock domain, within the 2 ms ceiling
CalibrationRe-solved at the start and end of every session; RMS reprojection under 0.28 px
Catalog capture setup, ready-to-deploy rows"Phone (head-mounted) / multiview rig"
Catalog capture setup, available-on-request rows"Multiview rig + handheld grippers"
Capture setup for pallet labeling or load strappingNot stated

Every collection runs through the same seven-stage end-to-end custom collection pipeline.

Task catalog

Which pallet rows does the catalog name?

The eight catalog rows that name a pallet, reproduced as the catalog states them. One title is shortened. No row names pallet labeling or load strapping.
Task (as listed)Description (as listed)SettingSkill levelCapture setup and status
Palletize boxesStack boxes in a stable pallet pattern.Warehouse ZoneMediumMultiview rig + handheld grippers; available on request
Depalletize to shelfRemove from pallet and shelve.Warehouse ZoneEasyMultiview rig + handheld grippers; available on request
Stretch-wrap palletWrap a pallet with stretch film.Warehouse ZoneMediumMultiview rig + handheld grippers; available on request
Load and secure outbound freight using a hand truck and stretch wrapMove pallets and boxes to the dock with a hand truck, stack and stretch-wrap the load.Warehouse & Distribution CenterMediumPhone (head-mounted) / multiview rig; ready to deploy
Loading Outbound TrucksStage pallets and parcels at the dock and load them into the trailer in sequence.Warehouse & Distribution CenterMediumPhone (head-mounted) / multiview rig; ready to deploy
Receive and unload inbound pallets using a manual pallet jackBreak down inbound pallets with a pallet jack, check contents against the manifest and shelve.Warehouse & Distribution CenterMediumPhone (head-mounted) / multiview rig; ready to deploy
Box and palletize finished goodsBox finished goods, tape cartons shut, stack them on a pallet and stretch-wrap the load.Plastic & Light-Assembly ManufacturerEasyPhone (head-mounted) / multiview rig; ready to deploy
Boxing & Palletizing Finished GoodsPack products into cartons, build the pallet in layers and wrap it for shipping.Plastic & Light-Assembly ManufacturerEasyPhone (head-mounted) / multiview rig; ready to deploy

Episode fields

What fields does a pallet episode carry?

Episode fields as documented on the site, and what each is useful for in pallet work.
FieldAs documentedUse for pallet tasks
Hand joints21 joints per hand, both hands, with a per-joint visibility flagWhich hand lifts and which hand squares or steadies the box
contact_eventsThe moment the hand meets the itemWhen a hand takes hold of a box or the film, and when it lets go
Action segmentsStart, end, verb and noun; 97 verbs, 512 nouns, human-reviewedOne labeled span per pick, place or wrap step
Instance masksStored at the depth resolution, 848×480; per-frame where the spec calls for themWhich box is which across frames
Timestamps and calibrationint64 nanoseconds per stream; intrinsics and extrinsicsAligning the head and wrist views
Pallet pattern, layer count, load weight, wrap count, film type, stack-stability, force or tactileNot statedNot stated

Delivery formats

Which formats are delivered, and what does each format’s own documentation say?

The delivery formats the site lists. The third column is taken only from the official pages linked under Sources.
FormatDelivered by FirsthandMaintainer documentation
RLDSListedEpisodes, each containing a dataset of steps
LeRobotListedSynchronized MP4 videos (or images) plus Parquet files
WebDatasetListedTar files; files of one sample share a basename
HDF5, zarr and Rerun .rrdListedNot described on this page

Not stated

What does the site not state about pallet collection?

Details a buyer may ask about and where the answer lives.
TopicOn this siteWhere it is settled
A pallet-labeling or load-strapping rowNot listed in the catalogWritten skill spec
Pallet patterns, layer counts, load weights and dimensionsNot statedScoped in the spec
Wrap counts and film typesNot statedScoped in the spec
Case or SKU counts, number of pallets, episodes, contributors or sitesNot statedQuote
Forklift or other vehicle useNot statedWritten skill spec
Capture at a buyer’s own warehouse or facilityNot statedWritten skill spec
Load weight, grip force or tactile dataNot stated for the standard captureNot offered as a stated stream
Stack-stability or quality labelsNot statedScoped in the spec
Which timeline applies to which rowNot statedQuote
Robot results or teleoperation dataNot stated; Firsthand captures people, not robotsYour own retargeting step

FAQ

Pallet building robot training data, answered.

Which pallet tasks does Firsthand list?
Palletizing boxes, depalletizing to a shelf, stretch-wrapping a pallet, loading outbound freight and trucks, receiving and unloading inbound pallets, and boxing and palletizing finished goods: eight rows in all.
Is there a stretch-wrap task in the catalog?
Yes. "Stretch-wrap pallet" is described as "Wrap a pallet with stretch film", is listed as medium skill and is available on request. Film type and wrap count are not stated.
What streams and labels come with a pallet episode?
Ten hardware-synchronized streams within the 2 ms ceiling, 3D hand pose with visibility flags, contact events and action segments. No pallet-pattern, stack-stability or wrap-quality label is stated.
Do you capture force or tactile data for pallet work?
The standard capture lists no force or tactile stream, and contact is described only by contact events. A separate glove tier has its own page, and this page makes no claim about it.
Does the site say forklifts or other vehicles are used?
No. Two catalog rows state a hand truck and a manual pallet jack, and one names loading a trailer. No forklift or other vehicle is stated, so none is claimed.
How are bystanders protected?
Every participant signs a written release, faces and licence plates are irreversibly blurred, a reviewer confirms no un-consented person is identifiable, and a clip where a bystander cannot be de-identified is discarded.
How fast can a custom pallet-handling dataset be collected?
From a signed spec to a first validated batch, a median of 48 hours where coverage is deep and three to four weeks where it is thin. The site does not say which applies to which pallet row, so ask when you request the quote.
Do you capture robot or teleoperation data?
No. Firsthand captures people, not robots, so using the episodes for a specific robot means your own retargeting step. The license is perpetual and buyer-owned, and the executed order form governs.

Sources

Where are the delivery formats on this page documented?

  • RLDS (GitHub) (opens in a new tab)RLDS is an ecosystem of tools to store, retrieve and manipulate episodic data; the dataset is retrieved as a tf.data.Dataset of episodes where each episode contains a tf.data.Dataset of steps. github.com
  • LeRobot (GitHub) (opens in a new tab)The LeRobotDataset format is described as synchronized MP4 videos (or images) for vision and Parquet files for state and action data. github.com
  • WebDataset (GitHub) (opens in a new tab)WebDataset files are tar files in which files that make up a training sample share the same basename. github.com

Request a custom pallet building and wrapping dataset.

Tell us the pallet task, the order of steps and what counts as a reject. We will quote reach, timeline and price against a written spec, with a first validated batch in a median of 48 hours where coverage is deep.