Egocentric Screwdriving and Fastening Video Data

Fastening is a short, repeated motion with real variation: the angle of approach, how hard the grip is, how a bit seats, and whether a joint takes one pass or several. The task catalog states this work directly. Its Precision Assembly Bench row, Screw-fasten housing, reads: "Fasten a housing with a driver to torque." A neighboring row, Torque to spec, reads: "Torque bolts with a torque wrench."

A first-person camera records that motion from the viewpoint a robot or an assistive system would act from, so the video shows what could be seen when the grip, the bit and the angle were chosen. Hand pose and contact events record the handling, and action segments mark where each step starts and ends. A manipulation policy can learn from this evidence without any robot being involved in the capture.

It helps to be exact about what the data is. It records a person driving screws and torquing fasteners: the grip, the tool and the order of steps. Torque, force, tactile and fastener-type labels are not stated on the site, so this page does not promise them.

Assembly task data on this site covers fastening, measuring, press-fit and wire work at a bench, as one task family among four. Egocentric tool use data covers hand-tool handling more broadly, naming driver and bit selection, clamp-and-align, measuring, fastening and tool hand-off as five of its skills. Both pages treat fastening as one entry in a longer list.

This page takes the opposite framing: fastening and screwdriving are the subject, drawn from the same catalog rows those pages already reference, with the driving, torquing and seating motion as the organising unit rather than one line in a broader skill list.

Pick-and-place data covers the reach-grasp-transport-place primitive that a tool hand-off is built from, and contact event data covers the general contact and grasp signal used across every Firsthand custom collection, including fastening episodes.

The task catalog lists rows that name driving, torquing or tightening a fastener directly. On a Precision Assembly Bench, Screw-fasten housing fastens a housing with a driver to torque, Torque to spec torques bolts with a torque wrench, and Busbar + limit-block + screws places a busbar and limit block, drives the screw sequence, then removes the block. In Electronics Manufacturing & E-Waste, Device manufacturing assembles device housings, fits boards and cables, fastens screws and tests the unit. On a Plastic & Light-Assembly Manufacturer line, one row assembles and fastens plastic components using electric screwdrivers and assembly jigs.

On a Trades Workshop Bench, Drive screws with driver drives screws with a power driver, Tighten bolt with wrench fits a wrench and tightens a bolt to snug, Ratchet socket on nut uses a socket ratchet on nuts, and Hang a fixture levels, marks and screw-mounts a fixture. In a Data Center & IT Hardware Bench, Screw-mount rack rails fastens rail brackets with a driver. In an Auto Service Center & Tire Shop, one row removes, replaces and torques brake pads and rotors using a socket set and a torque wrench.

The catalog rates most of these rows Easy or Medium, with the busbar screw sequence and the brake rotor torque row rated Hard because of the sequencing and the stakes of the fastener. The first table below lists each row with its status.

The catalog places these rows across six environments: Precision Assembly Bench (electronics), Electronics Manufacturing & E-Waste, Plastic & Light-Assembly Manufacturer, Trades Workshop Bench, Data Center & IT Hardware Bench, and Auto Service Center & Tire Shop. These are the environment types the catalog names, not a specific named site.

Every task row carries a status that says how soon capture can begin. Most of the rows above carry Available on request, meaning the work is scoped and recruited to spec when a buyer asks for it. The brake pad and rotor torque row carries Active collection, and the plastic-assembly fastening row carries Ready to deploy.

The site does not say that any fastening capture takes place inside a particular factory, shop or client facility, and this page does not claim it. No tool brand or manufacturer is named for any row.

The written spec is the contract, so it is worth being specific about the parts of fastening the site leaves to the buyer. Name the fastener and tool in scope at the level your policy needs, because the site does not label fastener type, thread or outcome by default, and name the tool: hand driver, power driver, wrench or torque wrench.

Name the sequence that matters, such as a single screw to seat, a multi-fastener pattern like the busbar sequence, or torquing to a spec value. Name whether cross-threads, dropped fasteners and slipped grips should be captured alongside clean takes.

Name the start state and the end state, and what counts as a finished episode: one fastener driven, a full fastener pattern completed, or a part fully assembled. Finally, name the capture mode: a head-mounted view of the task, or a staged set with handheld grippers for demonstrations that must map onto a robot end effector.

The catalog names two capture modes, and each row above uses one of them. In the first, a head-mounted phone records the first-person view of the task, and a multiview rig adds synchronized third-person angles when the spec asks for them; this mode covers the device manufacturing, plastic-assembly and brake-rotor rows. In the second, a staged working set is recorded by a multiview rig while the operator performs the task with handheld grippers, so demonstrations map onto a robot end effector; this mode covers the Precision Assembly Bench and Trades Workshop Bench rows. The second table below shows which rows use which mode.

A head-mounted camera keeps the tool, the fastener and the joint in frame from the viewpoint a fastening policy will act from, with a wrist RGB stream alongside it for close-range detail. Depth, 3D hand pose, contact events and action segments are aligned to video on one hardware clock, synced under 2 ms. Distance-invalid pixels beyond the reliable depth range are flagged rather than interpolated.

The site does not state a camera count, resolution, frame rate or lighting setup specific to fastening, so those are agreed in the written spec rather than assumed. It also does not describe how depth behaves on small screw heads, recessed fasteners or reflective driver bits, so raise that in the spec and test it in a Spec Pilot.

Each episode follows the same episode schema used across Firsthand custom collections: ten streams synced under 2 ms, with head and wrist RGB, depth, 3D hand pose with per-joint visibility, contact and grasp events, and frame-accurate action segments. The warehouse and object rearrangement pages also list instance segmentation or masks among their outputs.

For fastening, hand pose tracks the operator through the grip, the drive and any tool hand-off, contact events mark when fingers or a tool make and break contact with a fastener or workpiece, and action segments mark the start and end of each step, such as engage, drive and release. Because the labels share one clock with the video and depth, a step boundary lines up with the exact frame where it happens.

Object and fastener labels use the terms your spec sets, so they follow the vocabulary your training stack expects rather than a generic taxonomy.

On the site stated terms for manipulation failures, yes: cross-threads, dropped fasteners or tools, slipped grips, and misalignment are captured and action-segmented alongside clean takes rather than discarded, because a policy that only sees successful fastening has no model of recovery. That statement comes from the tool use page, which names cross-threads and dropped fasteners directly.

Failure and recovery episodes are labeled against the same accept criteria as clean takes, so they count toward validated hours rather than being treated as waste.

Torque, force and tactile data are not stated as delivered streams anywhere on the site. Several rows name torquing to a value or to spec, but the site does not describe a torque-wrench or load-cell reading as a delivered field. Fastener-type, thread and outcome labels, such as whether a joint was over-torqued or cross-threaded as a tagged result, are also not stated.

Screw or fastener counts, cycle time, and an items-per-minute or throughput figure are not stated for this task family. If your policy needs any of these, they are defined in the written spec as annotation depth, and the quote covers them before capture begins.

Every episode links to a signed contributor release through a documented consent chain, and the consent record is part of the delivery. Contributors are the people who perform the fastening tasks, so their release covers the footage of their hands and workspace.

Faces, plates and screens are detected and irreversibly blurred at ingest, and a reviewer confirms the result before anything ships. The anonymization record is delivered with the data. The site does not describe a separate release for bystanders who are not contributors, so who else may be in frame is agreed in the spec before capture rather than assumed afterward.

Delivery formats are RLDS, LeRobot and WebDataset, plus HDF5, zarr and Rerun .rrd. The first three are the training-loader formats; HDF5 and zarr carry arrays for custom pipelines, and Rerun .rrd gives a multi-stream synchronized playback file for review, which suits visually checking a fastening episode before writing a loader.

Converter source is readable, so you can retarget the schema to your own loader, and the episode schema page lists the fields.

You describe the fasteners, tools and sequence your policy needs. We turn that into a written spec covering tasks, environments, capture tier, annotation depth and accept criteria, then quote reach, timeline and price against that document before any capture begins.

The collection runs through the same seven-stage end-to-end custom collection pipeline as every Firsthand program: contributor recruiting, capture, per-batch QA, anonymization and licensed delivery. Rejected items are recaptured at our cost.

Where coverage is already deep, the first validated batch lands in a median of 48 hours from signed spec. Thin environments typically take three to four weeks, because vetting operators there is the bottleneck, and we quote that timeline up front. The quote tells you which case you are in.

On-demand top-ups run against the same spec and accept criteria after each training run, so new fastening data stays consistent with what you already hold. A Spec Pilot, a small and fast first batch, can prove the spec before you scale volume.

Every custom collection ships under a perpetual, buyer-owned commercial license, and exclusivity is available so the data is never resold. The license is documented alongside the consent chain, so a policy trained on the data can go into a product.