Stereo Fitting Room and Merchandising Reset Data

It is first-person video of a person resetting a retail store floor, recorded from their own point of view with two calibrated cameras and with hand pose and sensor streams alongside. The tasks are the three rows the site’s catalog lists in Retail Store -- Front of House: collecting garments left in fitting rooms, rehanging them and returning them to the floor; sorting garments on the rail by size and style and spacing the hangers evenly; and arranging products on shelves and displays to the planogram and facing them forward. Stereo describes the capture, and MCAP describes the container. They are separate choices.

The site does not state that stereo is already running on these rows. The catalog lists a head-mounted phone or a multiview rig as the capture setup. Stereo applies only through the site’s general stereo offer: a stereo pair is a capture choice agreed in the skill spec and confirmed on a Spec Pilot. The data is human capture by paid, consented contributors, not robot or teleoperation data. Firsthand is not affiliated with Foxglove or the MCAP maintainers.

Three, all in Retail Store -- Front of House under Retail & Consumer Goods, in the catalog data file lib/commercial-catalog-data.ts. The catalog lists examples for the environment such as a clothing store, an electronics shop, a furniture showroom, sporting goods, a bookstore and a department store. "Fitting room reset" is described as collecting garments left in fitting rooms, rehanging them and returning them to the floor, is rated Easy and is listed as Available on request. "Garment rail sizing" is described as sorting garments on the rail by size and style and spacing the hangers evenly, is rated Easy and is listed as Available on request.

"Merchandising and facing" is described as arranging products on shelves and displays to the planogram and facing them forward, is rated Medium and is listed as Available on request. All three list a head-mounted phone or a multiview rig as the capture setup. The site says nothing further about what Available on request means in volume, hours or delivered stereo footage, so this page claims none, and what applies to a given order is confirmed when you request the quote. The "Merchandising and facing" row speaks of products, not garments, so this page counts it as a store-floor display task and does not claim it is specific to apparel. The first table below reproduces the rows as the catalog states them.

Two other rows sit in the same environment. "Checkout and returns" is described as scanning purchases, processing payments and returns at the register and bagging or refunding items, is rated Easy and is listed as Available on request. "Product demos" is described as unboxing a product, setting it up and demonstrating its features to a customer on the floor, is rated Medium and is listed as Available on request. Both list the same head-mounted phone or multiview rig. They are nearby context only, and this page builds nothing on them.

The Retail Floor Set environment has its own apparel rows, "Reset fitting room", "Hang + size-order rail" and "Fold + stack apparel", which the egocentric fitting room page counts. This page does not count them. None of the three rows names shopper assistance, planogram software or loss prevention, so none is described here.

The egocentric fitting room and apparel merchandising page is the closest neighbour. It covers the same three Front of House rows as one half of a six-row apparel set, and it treats capture as egocentric, with no stereo treatment. This page narrows to the three Front of House rows, as stereo capture agreed through the general stereo offer, and goes further into the stereo pair, the MCAP container and the failure modes at a rail, a shelf and a fitting room.

The stereo retail shelf restocking page is stereo and MCAP too, but it centres refilling shelves from cartons across nine rows, and shares only "Merchandising and facing" with this page. The stereo garment finishing and pressing page is stereo and MCAP for one pressing row in Textile & Apparel Production, and none of these three rows appears there. The comparison table below sets the pages side by side.

A store reset is about where the hands, the item and its destination sit relative to one another: how far a hanger is from a rail, where a garment sits among its neighbours, how a product is turned to face forward at a shelf edge. A stereo pair recovers depth from the calibrated frame pair rather than from a standalone depth sensor, so the item, its destination and the hands share a frame.

The site gives no depth accuracy figure and no label for a rail that is sized, a hanger that is evenly spaced or a shelf that is faced, so whether those are resolvable in depth is Not stated. Capture at your own facility is listed as not offered.

The catalog states one capture setup for all three rows: a head-mounted phone or a multiview rig. The site describes two stereo setups and does not say which one a retail order uses. The hard-environments stack is a global-shutter 4K/60 head camera with a 120° lens, two 1080p/60 wrist cameras, a depth sensor at 848×480 and 30 fps, and 200 Hz 6-DoF IMUs. The off-the-shelf stereo head rig, dataset 06, is dual fisheye cameras at 1080p per eye, a 62 mm baseline, per-unit calibration, a 9-axis IMU and hand-tracking output.

Which rig, resolution and streams a fitting room or merchandising order uses is Not stated on the site, and the third table below sets what is stated beside what is not. Intrinsics and extrinsics are re-solved at the start and end of every session, and streams are hardware-triggered onto one clock with a 2.0 ms cross-stream ceiling. The catalog states that a multiview rig adds synchronized third-person angles when the spec asks for them, and wrist cameras give a closer view of the hands. The site does not state a store lighting setup, so lighting is Not stated and is agreed in the skill spec.

MCAP is an open source container file format for multimodal log data, supporting multiple channels of timestamped pre-serialized data. It is serialization-agnostic, so messages can be Protobuf, DDS (CDR), ROS, JSON and more. Its design is row-oriented and append-only, it stores schemas alongside the data, and it offers an optional index plus optional LZ4 or Zstandard chunk compression.

Reader and writer libraries exist for C++, Go, Python, Rust, Swift and TypeScript, and the mcap CLI can inspect, merge and split files. The foxglove/mcap repository is MIT-licensed; that is the MCAP project’s licence, not the licence of a Firsthand delivery. The MCAP hub page and the glossary entry cover the format in full.

Firsthand offers MCAP delivery, and dataset 06 is stated as shipping as MCAP. For retail floor work the site states hand pose, depth recovered from the stereo pair, IMU, contact events, action segments and per-frame instance masks elsewhere. What it does not state is which of those are channels in an MCAP file, how they are named, what encoding they use or at what rate.

No topic name, message schema or rate is stated on this page, and none should be assumed. Name the streams that matter in the skill spec and confirm them on a Spec Pilot batch. The fourth table below sets each stream beside what the site states about it.

Hand occlusion is the first risk. A garment closes around the fingers and changes shape, a hanger hook can sit behind a hand, and a hand can cover a size marker on a hanger or a garment. The visibility flag on each joint is the answer the site gives for hidden fingers, and contact events mark when a hand meets an item. The site states no size-marker reading, so what a size marker is and whether it must stay visible is written into the spec. A spec can also ask for wrist views, so a grip the head camera loses is still recorded.

Soft and folded fabric is the second. The site describes garments as items whose shape changes as a hand closes on them, and it records per-frame instance masks so the objects stay marked through the change. It lists no stack, spacing or faced-shelf label, so what counts as a finished rail or shelf is written into the spec up front.

Mirrors, clutter and lighting are the third. The site notes that wet and reflective surfaces degrade active-stereo depth in its kitchen dataset, and names reflective surfaces and clutter as cases where a single camera struggles in hard environments. Whether a fitting room mirror, a glossy display or a crowded rail behaves the same on a stereo pair is Not stated. The site ships per-frame depth confidence maps for its kitchen captures, and whether a retail order ships them is Not stated, so ask in the spec.

Calibration is the fourth, because stereo depth is only as good as the rectification behind it: calibration is measured at the start and end of every session under a 0.28 px RMS gate. An episode is also rejected when the manipulated object is out of frame for more than 15 percent of it or an undocumented bystander appears, and a reject log ships with every batch. Force and tactile readings are not among the listed streams.

A reset chains steps that look alike on camera: collecting a garment and rehanging it, then the next; spacing one hanger, then another; facing one product, then the next along a shelf. Action segments carry start, end, verb and noun, so the spec decides where one segment ends and the next begins, and which object is the noun.

The site states no episode length, no segment count and no facing-stroke label for these rows. Whether an episode is one fitting room, one rail or one shelf bay is set per program. Put that boundary and the verbs you expect into the skill spec, and check them on the Spec Pilot before you scale.

For every delivered episode the site states that faces, plates and screens are blurred irreversibly, speech is scrubbed of names, addresses and card numbers, any identifiable un-consented person goes to human review, and footage that cannot be de-identified is discarded. Shoppers and staff can appear on a store floor. Who may be in frame, and how a store’s own signage, price labels or screens are handled, is agreed in the spec. The anonymization page has the full statement.

The site states that front-of-house capture in hospitality is bystander-dense, so yield after de-identification runs lower there. It states no equivalent for retail, so how that applies to a store is Not stated. It also states no retailer, client or brand name, no garment, SKU or store count, no task time, throughput or error rate for these rows. Each of those is Not stated.

Off-the-shelf covers what is already in stock: the stereo category lists the stereo datasets and hand-tracking video, and only dataset 06 is stated as MCAP. This page does not claim an off-the-shelf dataset of fitting room reset, garment rail sizing or merchandising and facing, so this is custom collection: custom collection at scale through a network of thousands of capture providers, for a consistent, recurring data flow to labs.

Where coverage is deep, the median time from signed spec to first validated batch is a median of 48 hours; where it is thin, turnaround runs three to four weeks. The site does not say which speed applies to a stereo retail order, so ask when you request the quote. Collections are billed per validated hour.

Describe the tasks and their order, the garments or products involved, where each one ends up, what counts as a finished step, and the variation your pipeline is missing. Then name the streams you want in the file, the topic naming and message encoding your reader expects, whether calibration ships inside the file or beside it, and whether MCAP is the only delivery or sits beside RLDS, LeRobot or WebDataset.

Most programs begin with a Spec Pilot, a small first batch that proves the spec and accept criteria on real footage before you scale. The fifth table below shows what each spec element decides.