Egocentric Mailroom and Document Handling Video Data

Egocentric mailroom and document handling data is real-world video recorded from the point of view of a person who is working with mail and paper. The camera sits on the head, so the footage shows what the person sees: the hands, the envelope or sheet, the folder and the surface they work on. Alongside the video, each episode carries depth, 3D hand pose, contact events and frame-accurate action segments, all aligned on one hardware clock.

Teams search for it under several names, including mailroom robot training data, a document handling video dataset and egocentric envelope and paper handling data. They all describe first-person demonstrations of paper work, collected to a written spec. This page covers only what the Firsthand site states about it, and marks everything else as Not stated.

The site states mail and document work in its commercial task catalog, in three environments. Office and Professional Services has a Mail handling row, "Open, sort and distribute incoming mail and prepare outgoing letters and parcels", plus Filing, Scanning and Printing. Office Document Station has twelve rows, including sorting documents into labeled folders, collating and stapling sets, shredding, signing and filing a form, and inserting documents into envelopes and sealing them. Last-Mile Delivery and Courier has a Postal sorting row.

Anything outside those rows is Not stated. That includes reading or routing by a printed address, opening a sealed envelope with a tool, franking or weighing a letter, and handling bound volumes or archive boxes, so none is claimed here. If your policy needs one of them, you write it into the spec and we scope it, because the catalog rows are a starting vocabulary, not a fixed menu.

Every Office and Professional Services row in this family is rated Easy and listed as Available on request, and they are recorded with a head-mounted phone or a multiview rig. The Office Document Station rows are rated Easy, apart from three rated Medium: signing and filing a form, stuffing and sealing envelopes, and refilling a stapler and tape. They use a multiview rig with handheld grippers. The Postal sorting row is rated Easy and recorded with a head-mounted phone or multiview rig.

Available on request means the work is scoped and recruited to spec when a buyer asks for it. None of these rows is listed as Active collection, so the site does not say that a mail or document capture stream is running today. A first batch is therefore quoted against your spec rather than drawn from a stream that already exists.

The site has no environment called a mailroom. Its Office and Professional Services environment lists a corporate office, co-working space and reception area among its examples, with desk work, printing, filing, scanning and mail handling. The rows in that environment use the head-mounted phone mode, which the site describes as a first-person view of real work on site.

The Office Document Station is different. The site describes it as a desk-and-document station with a printer and tray, binders, a stapler, a label maker and mailers, and its rows use the multiview rig with handheld grippers, which the site describes as a staged working set. So the site describes a real workplace for the mail handling, filing and scanning rows, and a staged set for the document station rows.

The site names no office, company or building, and does not state the number of sites, the regions or the venue type. It does not state capture at a customer own facility either. Environment, access and schedule are agreed in the written spec.

A head-mounted phone records the first-person view of the job, and a multiview rig adds synchronized third-person angles when the spec asks for them. A wrist-mounted camera is available alongside the head camera for close-range detail, which suits paper work, where a hand often covers the sheet or flap. Camera count, resolution, frame rate, desk layout and mail volume are Not stated, so raise any of them in the spec.

Each episode follows the Firsthand episode schema: ten streams synced under 2 ms, with head and wrist RGB, depth, 3D hand pose with per-joint visibility, contact and grasp events, and frame-accurate action segments. For paper work, hand pose tracks the fingers as they separate a sheet or fold a flap, contact events mark when a hand meets or releases an item, and action segments mark each step of the job.

Object labels use the terms your spec sets. Item counts, paper sizes or stock types, sorting schemes and equipment makes are Not stated. Task durations, cycle times and the number of episodes are Not stated too. Success or quality labels, such as whether a document is filed correctly, are Not stated, as are certifications and robot results. The site makes no safety-compliance claim for this work.

The catalog rates these rows Easy or Medium, so the site does not call them Hard, and this page makes no claim about robot results. What the site does say about paper is that it is thin and flexible, so a sheet or a stack changes shape every time it is touched and hides what is under it. Small items such as labels, tape and envelope flaps are easy to lose behind a fingertip, and tasks such as collating repeat many near-identical motions that differ only in order.

The viewpoint matters too. The site states that a robot or humanoid perceives from a head- or wrist-mounted camera, not a static overhead shot, and that egocentric capture matches that geometry.

A good spec treats each named job as its own task. Opening mail, sorting it, distributing it, preparing outgoing letters, filing, scanning, collating and stuffing envelopes each get a start state, an end state and a set of accept criteria. That keeps action segments consistent across contributors and lets you ask for more of one job without re-collecting the others.

Delivery formats are RLDS, LeRobot and WebDataset, plus HDF5, zarr and Rerun .rrd. Mail and document episodes use the same schema as every other Firsthand collection, so a loader that reads another Firsthand dataset reads this one without changes. The format table below sets each format against what its official documentation states.

Every batch is validated against signed accept criteria before it ships, with a reject log, and delivery is rolling, per validated batch. Rejected items are recaptured at our cost, and billing is per validated hour.

Not in standard capture. Force or tactile readings are Not stated for the mail and document rows. The site separately offers a tactile glove tier as a pilot-ready option, in which glove signals are time-synced to a head-mounted stereo camera by hardware timestamps and an IMU. The site does not state that the tier has been used on paper work, so if your policy needs a contact signal, ask for it in the spec and agree the fit with us first.

Every episode links to a signed contributor release through a documented consent chain, and the consent record is part of the delivery. A vision model runs on every frame at ingest, flagging faces, heads, licence plates and any legible screen or document. The site describes the documents class as phone and monitor content, forms, badges and mail that would leak names, addresses or account data. Those regions are irreversibly blurred, and a human reviewer confirms the result before footage ships.

If the reviewer is unsure a region is fully covered, the episode is held or discarded, never downgraded and shipped. So legible names, addresses and document content are not in the delivered pixels. If your pipeline needs document content, raise it before the spec is signed; whether any engagement can keep specific content is Not stated. The site states no other privacy, compliance or legal terms for this work, and no in-home capture.

Where coverage is already deep, the first validated batch lands in a median of 48 hours from signed spec. Thin environments typically take three to four weeks, because vetting operators there is the bottleneck, and we quote that up front. The site does not state how deep coverage is for the mail and document rows, and they are Available on request, so plan on a quote rather than assuming the 48 hour figure.

The site also states program milestones. A pilot reaches its first delivery in two to three weeks, and a standing program reaches full rate in four to six weeks. Those assume capture coverage is already in place. On-demand top-ups run against the same spec and accept criteria after each training run. A batch size and a later-batch turnaround are Not stated, so both are quoted against your spec.

You describe the mail and document jobs, the environment and the sequence your policy needs. We turn that into a written spec covering tasks, environments, capture tier, annotation depth and accept criteria, then quote reach, timeline and price before any capture begins. This is the custom egocentric data collection route that every Firsthand task family follows.

A Spec Pilot, a small and fast first batch, can prove the spec and accept criteria before you scale. Every collection ships under a perpetual, buyer-owned commercial license, with a reject log and signed accept criteria on every batch.