Egocentric Container Opening Video Data (Jars, Lids, Bottles)

A closed container is a small mechanical puzzle that a person solves without thinking. They find a grip, hold the body steady, work the closure and, once it gives, set the lid aside and keep hold of the container. Egocentric container opening data records that sequence from the operator viewpoint, with the hands, the container and the station in frame, and it adds the labels a policy needs to learn from it: 3D hand pose, contact events, depth and action segments.

The site names the task in two places. The task catalog lists a row called open cans and jars, described as use a can opener and open jars, in a working kitchen set for food prep. The Kitchen and food prep dataset page lists open jars, bottles, vacuum-sealed packaging among the skills of that dataset. This page builds on those two statements and on the general capture, labeling, consent and delivery specs that apply to every Firsthand custom collection.

The data records a person opening containers: the views, the handling and the order of steps. It is not a library of container types, closure designs or measured opening forces. The site does not state a closure type, a container material, a force or torque stream or a per-episode success label as standard deliverables, so a buyer who needs any of them defines it in the written spec.

Opening a container is a task where the two hands do different jobs. The site makes the same point on its bimanual page, where it says that many real tasks, from opening a jar to folding fabric, depend on what the second hand is doing at the same instant: stabilizing, supporting or handing off. A single-camera, single-hand view of a lid being turned hides half of what the policy needs to learn.

A first-person camera records the sequence from the viewpoint a robot policy would act from, so the video shows not only what was done but what could be seen when it was done. Hand pose and contact events record the handling, and action segments mark where the grip ends and the opening begins.

Container opening is also a case where the hands cover the object: the fingers wrap the lid and the body, so the closure is occluded for most of the motion. The site describes its hand pose and contact labels as tracked through occlusion, which is the property that matters here. The kitchen dataset page also notes that wet and reflective surfaces degrade active-stereo depth, with per-frame confidence maps shipped so a buyer can mask rather than guess, so a wet or glossy container is worth naming in the spec.

Unboxing and unpacking data is about getting into a carton, bag or pack and taking the contents out. The catalog row for cans and jars appears there as one row among many. This page takes the closure itself as the subject, the grip and the turn, and treats what happens to the contents afterwards as outside the page.

Pouring and liquid handling data starts after the container is open: the controlled release of a liquid from one vessel to another. Container opening ends where pouring begins, so a buyer who needs the whole chain can ask for both in one spec.

Grasping data is a primitive about the moment of contact with an object; a grip appears inside container opening, which adds the turn, pry or pull that follows it. The cooking video page covers food preparation as a whole, with jar opening as one task among many, and dexterous manipulation data is the broader category of which container opening is one example. The comparison table near the end sets the neighbouring pages side by side.

The catalog states one row that names the task directly: open cans and jars, described as use a can opener and open jars, in the Working Kitchen Set for food prep, with a skill level of Medium and a status of Available on request. The environment lists a stovetop, counter, sink, dishwasher, utensils and food props as its examples.

The Kitchen and food prep dataset page adds a second statement. Among the skills it lists is open jars, bottles, vacuum-sealed packaging, and it names three environments for that dataset: a domestic kitchen, a shared-house kitchen and a small commercial prep line. That is the only place the site names bottles or vacuum-sealed packaging as something opened. The task catalog has no row for either, so this page treats them as a dataset skill rather than as a separate catalog task.

The site states two further container tasks in the other direction, closing rather than opening. In the Wet Lab and Clinical Bench environment, the catalog lists cap, vortex and rack tubes, described as cap sample tubes, vortex-mix and rack them in order, and fill and cap sample tubes, described as fill tubes to a mark and cap. These are capping tasks, and a spec that wants uncapping as well names it, since the site does not list an uncapping row. Neighbouring rows open packs rather than containers: sterile field setup, surgical instrument setup and package inspection. The first table below lists every row used on this page.

The site does not state a row for a lid on its own, a flip-top bottle, a child-resistant cap or a pump dispenser, and this page does not claim any of them.

The catalog places the jar and can row in a working kitchen set and the tube capping rows in a wet lab and clinical bench; the kitchen dataset lists domestic kitchens, shared-house kitchens and a small commercial prep line. Each catalog row carries a status that says how soon capture can begin.

The site defines those statuses plainly. Ready to deploy means contributors and site access are set up and capture starts once a spec is signed. Active collection means recording is running now, so a spec can join an existing capture stream. Available on request means the work is scoped and recruited to spec when a buyer asks for it. In the current catalog, the open cans and jars row and both tube capping rows are marked Available on request.

An Available on request row is not a shelf of existing footage; it is a task the site will scope and recruit for when you ask. The site does not say that any of this capture takes place inside a particular customer facility, and this page does not claim it. Where a collection happens is agreed in the spec.

Name the containers in scope, such as jars, bottles, cans or sample tubes. Name the closures: screw lids, twist caps, vacuum-sealed lids or a can opener on a can are different motions, and a model trained on one will not necessarily transfer to another.

Name the tools; the catalog mentions a can opener and handheld grippers. Name the start and end state: a sealed jar on a counter and an opened jar with the lid set aside are different goals, and the accept criteria should say which counts as a finished episode. Name what happens to the lid afterwards, since the site does not state a standard for that.

Name the conditions that will stress the sensors, such as wet containers or a cluttered counter, since the kitchen dataset lists clutter and failure cases such as drops, spills, mis-grasps and recoveries among its conditions. Finally, name the capture mode. The second table below sets each choice out.

The catalog names two capture modes. In the first, a head-mounted phone records the first-person view of real work on site, and a multiview rig adds synchronized third-person angles when the spec asks for them. In the second, a staged working set is recorded by a multiview rig while the operator performs the task with handheld grippers, so demonstrations map onto a robot end effector. The open cans and jars row and both tube capping rows use the second mode.

Each episode follows the same episode schema used across Firsthand custom collections: ten streams synced under 2 ms, with head and wrist RGB, depth, 3D hand pose with per-joint visibility, contact and grasp events, and frame-accurate action segments. Distance-invalid pixels beyond the depth sensor reliable range are flagged rather than interpolated. Wrist RGB sits alongside the head camera for a close view of the lid.

The site does not state a camera count, resolution, frame rate or lighting setup specific to container opening, so those are agreed in the written spec rather than assumed.

For container opening, 3D hand pose tracks the operator through the grip, the turn, the pry and the release. Contact events mark when fingers make and break contact with the container or the lid, and grasp events mark when a hold is taken or let go. Action segments mark the start and end of each step, such as gripping, opening and setting down. Because the labels share one clock with the video and depth, a step boundary lines up with the exact frame where it happens.

Object labels use the terms your spec sets: the skill spec decides which object classes are in scope before capture, so labels follow the vocabulary your training stack expects rather than a generic taxonomy. The third table below lists each stream and label with its status on the site, including the ones the site does not state.

Force, torque and tactile streams are not stated anywhere on the site as part of a custom collection, and this page does not add them. Container opening is a task where those signals would be informative, since a lid either gives or it does not, but a head-mounted camera and a hand pose track do not measure them. A buyer who needs a measured opening force would have to raise it with the team, and the site makes no promise that it is available.

Container type, closure type, container material, cap size, thread direction and opened or closed state labels are not stated either, and neither is a 6-DoF pose for the container or the lid. A per-episode success or failure label is not stated. Object identity labels follow the spec vocabulary rather than a fixed taxonomy.

If your policy needs any of these, they are defined in the written spec as annotation depth, and the quote covers them before capture begins. The page does not promise them by default.

Every episode links to a signed contributor release through a documented consent chain, and the consent record is part of the delivery.

Faces, plates and screens are detected and irreversibly blurred at ingest, and a reviewer confirms the result before anything ships. The anonymization record is delivered with the data, so a buyer can see what was blurred and when.

Container labels, brand marks and printed text on a jar or bottle are not listed as blur categories on the site, so this page does not claim they are blurred. Opening a jar puts the label in close view by nature. If label text or brand marks in frame matter for your use, raise it in the spec so the handling is agreed before capture rather than assumed afterwards.

QA runs against your signed accept criteria. A validated hour is 60 minutes of footage that has passed QA against your spec, with sync within the 2 ms ceiling, calibration solved, coverage cells filled and consent documented. You are billed on validated hours, a reject log ships with every batch, and rejected items are recaptured at our cost.

Delivery formats are RLDS, LeRobot and WebDataset, plus HDF5, zarr and Rerun .rrd. The first three are the training-loader formats; HDF5 and zarr carry arrays for custom pipelines, and Rerun .rrd gives a multi-stream synchronized playback file for review, which suits visually checking an opening episode before writing a loader.

Converter source is readable, so you can retarget the schema to your own loader, and the episode schema page lists the fields. Container-opening episodes use the same schema as every other Firsthand collection.

Where coverage is already deep, the first validated batch lands in a median of 48 hours from signed spec. Thin environments typically take three to four weeks, because vetting operators there is the bottleneck, and we quote that timeline up front. The catalog marks the open cans and jars row as Available on request, so plan on the thin-coverage case unless the team tells you otherwise for your environment.

On-demand top-ups run against the same spec and accept criteria after each training run, so new container-opening data stays consistent with what you already hold. A Spec Pilot, a small and fast first batch, can prove the spec and accept criteria before you scale volume.

You describe the containers, closures, tools, stations and end states your policy needs. We turn that into a written spec covering tasks, environments, capture tier, annotation depth and accept criteria, then quote reach, timeline and price against that document before any capture begins.

The collection runs through the same end-to-end custom collection pipeline as every Firsthand program. Every custom collection ships under a perpetual, buyer-owned commercial license, and exclusivity is available so the data is never resold.