Egocentric Peg-in-Hole and Press-Fit Insertion Data

Egocentric peg-in-hole and press-fit insertion data is first-person video of a person inserting, seating or snapping one small part into another at a bench, recorded to a written skill spec so that a robot learning system can see how the work is done. On this site the setting is the Precision Assembly Bench (electronics) environment in the Manufacturing & Workshops sector, whose example line reads "Assembly bench: PCB/electronics, fasteners, fixtures, solder/crimp tools, small parts". The same request is sometimes phrased as peg-in-hole robot training data, a press-fit insertion video dataset, egocentric snap-fit assembly data or custom insertion task data collection. They describe one job from different angles: write down the task, collect it on demand, and receive validated episodes with the labels a training pipeline expects.

The data is human capture. A paid, consented contributor performs the task while a rig records it. It is not robot or teleoperation data, and it is not a fixed corpus you download. This page is narrower than the precision assembly bench page, which covers all 15 bench rows, and narrower than electronics assembly data, which covers board, connector and housing work more broadly. Its subject is the four catalog rows that describe putting one part into or onto another, and nothing beyond those four rows is claimed.

It is also not a measurement dataset. The site states no fit class, clearance, insertion force, part size or connector type for any of these rows, so this page states none. If your policy needs a particular fit or force, that has to be written into the skill spec as an accept criterion and agreed before capture.

The site’s commercial task catalog has four rows that describe insertion. The first table below reproduces them as the catalog states them. All four sit in the Precision Assembly Bench (electronics) environment, all are rated Medium, all list the capture setup "Multiview rig + handheld grippers", and all are Available on request. The site defines that status as scoped and recruited to spec when a buyer asks for it, rather than already set up and recording.

The rows are "Peg-in-hole insert" (Insert pegs/dowels into holes.), "Place into fixture" (Place a part precisely into a fixture.), "Press-fit connectors" (Place and press connectors onto a PCB.) and "Snap-fit housing" (Snap two housing halves together.). This page quotes them and says no more. The catalog does not say how large the pegs are, what the fixture holds, which connectors are pressed or how many housing halves make a task, and it gives no tolerance behind the word "precisely".

Where the catalog has no row, this page says so instead of implying coverage. The second table sorts the work into kinds and marks the gaps. There is no row for pressing a part other than a connector onto a PCB, and none for compliant or force-controlled insertion. Screw-fasten housing is a separate row, covered on the screwdriving and fastening page. A buyer who needs a kind of insertion that has no row would describe it in a skill spec, and it would be scoped in a Spec Pilot rather than assumed.

The catalog lists one capture setup for all four rows: a multiview rig with handheld grippers. The site describes that mode as a staged working set recorded by a multiview rig while the operator performs the task with handheld grippers, so demonstrations map onto a robot end effector. It is different from the head-mounted phone mode listed for several other environments. The site states no accuracy figure for the mapping from a handheld gripper to a robot end effector.

The site’s general capture statement lists a head-mounted global-shutter 4K/60 camera with a 120° lens, two wrist-mounted 1080p/60 cameras, an active-stereo depth sensor at 848×480 and 30 fps, and 200 Hz 6-DoF IMUs, all hardware-triggered onto one PTP clock domain. It describes ten synchronized streams aligned under 2 ms. Which of those streams an insertion program carries is Not stated for these rows, because they list the multiview rig rather than the head-mounted mode, and it is agreed in the skill spec.

Force and tactile readings are not among the standard streams; standard capture is cameras, depth and IMU. A separate glove tier records a tactile glove signal beside the video, time-synced to a head-mounted stereo camera by hardware timestamps and an IMU, with each glove calibrated every session. The site does not state that the glove tier has been used on insertion work or on this bench, so whether it fits an insertion program is agreed with us before you rely on it. The tactile glove page covers the tier in full.

Insertion is short, close and tightly constrained. The fingers cover the small part they are placing, and the part covers the hole, fixture or connector it is going into. That is why 3D hand pose carries a per-joint visibility flag, why contact events mark when a hand meets an item, and why instance masks are recorded per frame. The challenge table maps each difficulty to the stream or label the site already states, or says Not stated.

Depth is a second limit. The site states that small fasteners can fall below a depth sensor’s reliable floor, that hand pose stays valid at that range, and that metric depth on sub-centimetre parts is flagged rather than interpolated. It states no depth floor for pegs, dowels, connector pins or housing features, so a buyer whose parts are very small should ask for that to be confirmed in the spec instead of assuming it.

Robot-model research adds a point. The GR00T N1 paper says its human video datasets do not include explicit action labels, and says hand-held robot grippers speed up collection but leave a mismatch with the downstream robot. The insertion rows pair a multiview rig with handheld grippers so demonstrations map onto a robot end effector; no accuracy for that mapping is stated. This is robot-model research, not Firsthand data, and Firsthand has no relationship with its authors.

Each attempt at a task is delivered as one episode. The site’s episode schema lists action segments, contact events, 3D hand pose for both hands with a per-joint visibility flag and instance masks, alongside int64 nanosecond timestamps per stream and camera intrinsics and extrinsics. Hand pose is stated as 21 joints per hand, in 3D, in metres in the head-camera frame.

Action segments divide an episode into spans carrying start_ns, end_ns, verb and noun, aligned to every other stream. For insertion, a segment might be one peg inserted, one connector pressed or one housing closed, as the skill spec defines it. Contact events mark when a hand meets an item; the site does not describe them as force or pressure, and it lists no seated, jammed, snapped or failed label.

Validation runs against accept criteria written into the spec before capture. A reject log ships with each batch. The site states these reject thresholds: a cross-stream sync error above a 2.0 ms ceiling, the manipulated object out of frame for more than 15 percent of the episode, and more than 35 percent invalid depth pixels in the manipulation region. On custom collection engagements, rejected items are recaptured at the provider’s cost, and delivery is billed per validated hour. These are reject thresholds, not a stated pass rate.

Delivery is in RLDS, LeRobot, WebDataset, HDF5, zarr or Rerun .rrd, with converters shipped as readable source so you can retarget the schema to your own loader. The license is perpetual, irrevocable and buyer-owned, non-exclusive by default, with exclusivity available as a separate term; the executed agreement controls. Whether an insertion delivery carries robot action data is Not stated; the site describes human-side capture only.

A buyer evaluating insertion data usually wants numbers, so it is worth being direct about which ones are missing. For these four rows the site states no peg or hole dimensions, no fit class or clearance, no insertion or press force, no connector type or count, no fixture type, no housing type, no cycle time, no episode or hour count, no number of sites and no region. It lists no success, defect or seated-state label, and it states no safety practice, standard or certification. It names no robot brand, and this page reports no benchmark or robot result.

None of that makes the rows unusable. It means the specifics are set per program. You write the parts, the steps, what counts as a finished step and the variation cases into the skill spec, and the batch is judged against those terms. The last table on this page lists what is delivered and what is not stated, so you can see the boundary in one place.

Every contributor signs a written release granting commercial training rights before capture begins, and is paid a published hourly rate rather than a per-task bounty. Consent is tracked through a signed consent chain, and each delivered episode links to it. The site states that faces are blurred at ingest on its capture pages and that identifying details outside the task are anonymized. Audio is scrubbed of identifying speech, and a clip where someone cannot be de-identified is discarded rather than shipped. The anonymization record ships with the data.

The site states that capture happens in Firsthand’s contributor settings, not at a customer’s own facility. Which bench, venue or region an insertion program uses is Not stated and is agreed in the skill spec. Safety practice, protective equipment, standards and certifications are not stated for this environment, and this page makes no such claim.

From signed spec to first validated batch, the published timeline is a median of 48 hours in environments where coverage is already deep, and three to four weeks where coverage is thin and contributors need to be vetted and recruited first. All four insertion rows are Available on request, so they are scoped and recruited to spec when asked for. The site does not say whether the bench counts as deep or thin coverage, so which range applies is confirmed when your spec is quoted rather than assumed. A spec conversation happens the same week, and a tight spec is the quickest route to a fast, on-demand dataset.

Describe the insertion steps and their order, the parts and fixtures involved, what counts as a finished step, and the failure or variation cases your pipeline is missing. That becomes a written skill spec, quoted for reach, timeline and price before any capture begins. Most programs begin with a Spec Pilot, a small, fast first batch that proves the spec and the accept criteria on real footage before volume scales. Collections are billed per validated hour.