Inventory counting and stock check

Egocentric Inventory Counting and Stock Check Video Data

Scan the shelf location, count what is there, write the total on the count sheet and reconcile it against the system: a stock check is a loop of looking, touching and recording, done by hand, one location after another. We collect custom egocentric video of inventory counting and stock checks on demand, to a written skill spec, with synchronized streams, 3D hand pose, contact events, action segments, anonymization and a buyer-owned license.
Tasks
Two counting rows, as the catalog lists them
Streams
Ten, synced under 2 ms
First batch
Median 48 h in deep-coverage settings
License
Perpetual, buyer-owned

Requirements

What does a stock check need that a generic shelf clip does not?

01

A loop that ends in a number

The catalog describes counting as a loop with a result: scan shelf locations, count items and record the totals on the count sheet; count stock in assigned locations and reconcile the numbers against the system. The value is in the order of the steps and in how each location is closed out.

02

Hands busy with a scanner and a sheet

Both counting rows name a scanner, and one names a count sheet, so the hands are often holding a device or a pen while they touch stock. First-person capture keeps the hands in frame, and the two wrist cameras add a closer view. 3D hand pose carries a per-joint visibility flag, so a finger hidden behind a box or a scanner is flagged rather than guessed.

Definition

What is egocentric inventory counting and stock check video data?

It is first-person video of a person counting or checking stock, recorded from a head-mounted camera, with two wrist cameras for a closer view of the hands. The contributor is a paid, consented person following a written skill spec, not an actor and not a robot.

The point of the data is what the hands do while counting: reaching into a location, handling items, aiming a scanner and recording a total. It is labeled so each step can be sliced out. Where the site states nothing, a table says "Not stated" and the answer lives in the spec or the quote.

Task coverage

Which counting and stock-check tasks can be captured?

The task catalog names two rows that are explicitly about counting. "Count and reconcile shelf inventory using a barcode scanner and count sheets" is described as "Scan shelf locations, count items and record the totals on the count sheet." "Cycle Counting & Inventory" is described as "Count stock in assigned locations with a scanner and reconcile the numbers against the system." Both are listed as Easy, Ready to deploy.

Other rows sit next to counting but do not name it, so this page does not count them: order picking, receiving and unloading, backroom stock sorting and damaged-goods sorting. The second table lists them for context. Ask for a counting step inside any of them in the spec.

Settings and rigs

Where is counting data captured, and on what rig?

Both counting rows are listed under the Warehouse & Distribution Center environment, whose catalog examples are an e-commerce fulfillment hub, a distribution center, a cold storage facility and a freight warehouse. Both list a head-mounted phone or multiview rig.

The site lists no counting row in a store environment. Its store backroom rows, under Retail Store -- Back of House, cover sorting stock onto racks and bins, not counting it. If you need counting in a store, say so in the brief. The site does not otherwise state where capture happens or who hosts it, so the spec settles location and access.

Episode contents

What does each counting episode contain?

An episode is one contributor performing one task from the skill spec, recorded with ten hardware-synchronized streams that line up within 2 ms. The site documents a canonical episode schema, and every delivery format is a view of it.

Hand pose is stored as 21 joints per hand in metres in the head frame, with a visibility flag for each joint. Contact events record when a hand meets an object. The site does not say whether a carton, a scanner or a count sheet counts as the object, so settle it in the spec.

Action segments carry a start, an end, a verb and a noun from a taxonomy the site states as 97 verbs and 512 nouns, human-reviewed. Every episode links to its contributor’s signed release by a consent artefact ID, and each batch ships with a data card.

Labeling

How is inventory counting data labeled?

The unit of labeling is the action segment. A spec that says "count the location" gives you one long segment. A spec that names scan, count, record and reconcile gives you a sequence you can slice by step.

Every episode is machine-checked, then reviewed by a human against its skill spec, and rejected episodes are recaptured at our cost. The site states no label for the number counted, the count accuracy or any variance against the system, and no force or tactile reading, so ask for any such label at the start.

Privacy

How is privacy handled when capturing in stores and warehouses?

Only what the site already states applies. Every contributor signs a written release, and each episode links to it. Faces and screens are detected and blurred irreversibly, a human reviewer confirms that no un-consented person is identifiable, and footage with bystanders who cannot be de-identified is discarded rather than shipped.

The site does not state how customers, other staff or screens showing stock data are handled beyond that general rule, so settle it in the spec before capture. This is not legal advice, and the executed agreement controls.

Delivery

How is counting data delivered, and who owns it?

Data is delivered in RLDS, LeRobot, WebDataset, HDF5, zarr or Rerun .rrd, or as raw streams with metadata, with the episode schema as the canonical layout. QA runs once, against the canonical episode, before any format is cut. The formats table sets the site’s delivery next to each maintainer’s own documentation, linked under Sources.

The license is perpetual, irrevocable and worldwide, buyer-owned and non-exclusive by default, with exclusivity available as a separate term and the executed order form governing.

Speed

How fast can a custom inventory counting dataset be collected?

The site states one published timeline: from a signed spec to a first validated batch, a median of 48 hours where coverage is already deep, and typically three to four weeks where it is thin because operators have to be staffed. The site does not say which applies to the counting rows and promises nothing faster, so ask when you request the quote.

Collection is on demand, so a focused brief, such as one aisle of locations counted and reconciled against a list, is a normal starting point.

Getting started

How do you start a pilot?

Start with the free sample pack to check the loader and label quality, then move to a Spec Pilot: a small, fast first batch against the written spec that proves the fill rate and reject rate on your conditions before you scale.

A good counting brief names the setting, the order in which locations are visited, how each location starts and ends, what counts as a reject and the privacy rule for the space. The site’s skill-spec guide walks through that structure. Then request a custom dataset.

Capture setup

What is the capture setup for an inventory counting collection?

The capture setup as stated on this site, applied to inventory counting. Counts of contributors and hours are set in the skill spec.
PartAs stated on this site
Head cameraGlobal-shutter 4K at 60 fps with a 120 degree lens
Wrist camerasTwo wrist-mounted cameras at 1080p and 60 fps
DepthDepth sensor at 848×480 and 30 fps, with per-frame confidence maps
IMU6-DoF at 200 Hz
SyncTen streams, hardware-triggered onto one PTP clock domain and aligned under 2 ms
CalibrationRe-solved with a checkerboard target at the start and end of every session; accepted only under the 0.28 px RMS reprojection gate
Catalog capture setupHead-mounted phone or multiview rig for both counting rows
Scanner or device models, shelf and bin layoutsNot stated

Every collection runs through the same seven-stage end-to-end custom collection pipeline.

Task catalog

Which counting rows does the catalog name?

The two catalog rows that are explicitly about counting or a stock check, reproduced as the catalog states them.
Task (as listed)Description (as listed)EnvironmentSkill levelStatus
Count and reconcile shelf inventory using a barcode scanner and count sheetsScan shelf locations, count items and record the totals on the count sheet.Warehouse & Distribution CenterEasyReady to deploy
Cycle Counting & InventoryCount stock in assigned locations with a scanner and reconcile the numbers against the system.Warehouse & Distribution CenterEasyReady to deploy

Neighbouring rows

Which catalog rows sit next to counting?

Rows that handle stock but do not name counting or a stock check, reproduced as the catalog states them. They are listed for context.
Task (as listed)Description (as listed)EnvironmentStatus
Order PickingPicking, packing, sorting, loading and inventory handling in distribution facilities.Warehouse & Distribution CenterReady to deploy
Receive and unload inbound pallets using a manual pallet jackBreak down inbound pallets with a pallet jack, check contents against the manifest and shelve.Warehouse & Distribution CenterReady to deploy
Receiving & UnloadingUnload inbound goods from the truck, check quantities against the paperwork and move them to storage.Warehouse & Distribution CenterReady to deploy
Sort and organize backroom inventory using shelving racks and binsGroup backroom stock by category, shelve it on racks and label bins.Retail Store -- Back of HouseReady to deploy
Damaged-Goods SortingInspect returned or damaged items, scan them and sort them into markdown or disposal bins.Retail Store -- Back of HouseReady to deploy

Episode fields

What fields does a counting episode carry?

Episode fields as documented in the site’s episode schema, and what each is useful for in a counting task.
FieldAs documentedUse for counting
hand_joints21 joints per hand, in metres, head frameHow each hand reaches, handles stock and aims a scanner
hand_visibleBoolean visibility flag per jointWhich fingers are hidden behind a carton or device
contact_eventsTimestamp, hand, object ID and kindWhen a hand meets an item, a scanner or a sheet
instance_masksTracked object IDs per frame at 848×480Which object is which across frames
action_segmentsstart_ns, end_ns, verb, nounOne labeled span per counting step
depth_mm and depth_confMetric depth with a per-pixel confidence mapWhere depth can be trusted near a shelf
Number counted, count accuracy or variance, force or tactileNot statedNot stated

Delivery formats

Which formats are delivered, and what does each format’s own documentation say?

The delivery formats the site lists. The third column is taken only from the official pages linked under Sources.
FormatDelivered by FirsthandMaintainer documentation
RLDSTFDS-compatible tfrecord shards, one skill line per directoryStores episodic data; a dataset is a tf.data.Dataset of episodes, each containing a dataset of steps
LeRobotParquet plus an mp4 per camera, in the Hugging Face datasets layoutSynchronized MP4 videos or images for vision, plus Parquet files
WebDatasetListed as a delivery formatTar files in which files sharing a basename make up one training sample
HDF5, zarr and Rerun .rrdListed as delivery formatsNot described on this page

Not stated

What does the site not state about inventory counting?

Details a buyer may ask about and where the answer lives.
TopicOn this siteWhere it is settled
SKU counts, item types and store or warehouse namesNot statedWritten skill spec
Count accuracy, variance rates or the number countedNot statedScoped in the spec
Scanner and device modelsNot statedWritten skill spec
Shelf and bin layoutsNot statedWritten skill spec
Force or tactile dataNot statedNot offered as a stated stream
Task-completion labelsNot statedScoped in the spec
Throughput, and which timeline applies to these rowsNot statedQuote
Customers, other staff or screens in frameNot stated beyond the general bystander ruleSkill spec
Robot actions or teleoperation dataNot stated; Firsthand captures humans, not robotsYour own retargeting step

FAQ

Egocentric inventory counting and stock check video data, answered.

What is egocentric inventory counting and stock check video data?
Custom first-person video of people counting stock in assigned locations, scanning shelf locations and reconciling totals, captured by paid, consented contributors to a written skill spec, with synchronized streams, 3D hand pose, contact events and action segments.
Which counting and stock-check tasks does Firsthand list?
Two catalog rows: "Count and reconcile shelf inventory using a barcode scanner and count sheets" and "Cycle Counting & Inventory". Both sit under Warehouse & Distribution Center and are listed as Easy and Ready to deploy.
Does Firsthand list a counting task in a store?
Not as a counting row. The store backroom rows cover sorting stock onto racks and bins and sorting damaged goods, not counting. If you need counting in a store, say so in the spec.
What streams and labels come with a counting episode?
Ten hardware-synchronized streams aligned under 2 ms, 3D hand pose for both hands, contact events and action segments with a start, an end, a verb and a noun. No count value, count accuracy, force or tactile reading is stated.
How is privacy handled in stores and warehouses?
Every contributor signs a written release, and faces and screens are blurred irreversibly, with a reviewer confirming no un-consented person is identifiable. Footage with bystanders who cannot be de-identified is discarded. How customers and other staff are handled is settled in the spec.
How fast can a custom inventory counting dataset be collected?
From a signed spec to a first validated batch, a median of 48 hours where coverage is deep and three to four weeks where it is thin. The site does not say which applies to these rows, so ask when you request the quote.
Do you capture robot or teleoperation data for counting?
No. Firsthand captures humans, not robots. Episodes carry human hand pose and human action labels, so using them for a specific robot means your own retargeting step.
Who owns the data?
The license is perpetual, irrevocable and worldwide, buyer-owned and non-exclusive by default, with exclusivity as a separate term. The executed order form governs.

Sources

Where are the delivery formats on this page documented?

Request a custom inventory counting dataset.

Tell us the setting, the order of locations and the privacy rule your model needs to see. We will quote reach, timeline and price against a written spec, with a first validated batch in a median of 48 hours where coverage is deep.