Egocentric Computer and Keyboard Use Data

Egocentric computer and keyboard use data is first-person video of a person working at a desk, recorded from the contributor’s own point of view with synchronized sensor streams alongside. The camera sits on the head, or on a phone worn there, close to what a robot working at a desk would see, so the frames show the hands, the keyboard, the documents and the phone rather than a distant view of someone seated at a workstation. On this site it is a custom collection: you name the task and the setting, and capture is run to a written skill spec by paid, consented contributors.

It helps to separate what the site states from what it does not. The catalog lists one row that names a keyboard: Desk work, described as "Work at a desk handling documents, keyboard and phone through routine office tasks." It sits in the Office & Professional Services environment, whose listed examples are "Corporate office, co-working space, reception area -- desk work, printing, filing, scanning, coffee machine, mail handling." The catalog does not list a mouse row, a typing row, a data entry row, a row that names a computer, or a row for operating a terminal. This page therefore does not present mouse work or typing as an existing task. Whether a particular computer step can be captured is a skill-spec question to settle in a Spec Pilot rather than an assumption, and until it is settled the honest answer on this site is "Not stated".

People reach this topic through several phrasings: keyboard and mouse video data, desk task robot training data, egocentric office computer use data, or custom office task data collection. They describe one request from different angles. Under each of them the work is the same: write down the task, collect it on demand, and receive validated episodes with the labels a training pipeline expects. The one phrase the catalog cannot answer directly is "mouse", because no row mentions one.

This page overlaps on purpose with two neighbours. The office and desk tasks page covers the whole office family, including filing, scanning, printing and document stations, and has its own section on computer use. The mailroom and document handling page covers the paper side. This page narrows to the single keyboard row and gives a plain account of what the site does and does not state about it. It adds no new stream, label or claim.

One row of the site’s commercial task catalog names a keyboard. Desk work is rated Easy, listed as Available on request, and captured with a head-mounted phone or a multiview rig. The first table below reproduces it, with the other five rows of the same environment, exactly as the catalog states them. The environment lists six tasks in total: coffee machine operation, desk work, filing, mail handling, printing and scanning, all rated Easy.

Two of the other rows use wording that touches a screen-adjacent step. Printing is described as "Load paper, send documents to the printer and collect and collate the output." Scanning is described as "Feed pages into the scanner, name and save the files and refile the originals." Neither row names a keyboard, a mouse or a screen, and this page does not add one. They are shown in the catalog’s own words so that nobody reads more into them than is there.

A different environment, Office Document Station, lists a row called Sign + file form, described as "Write, sign, and file a form." It is rated Medium and captured with a multiview rig and handheld grippers. Writing a form by hand is not typing, and the row is mentioned only so that it is not mistaken for keyboard work. The same environment holds the rest of the paper-handling rows, which are covered on the office and desk tasks page.

The catalog also lists checkout rows in grocery and retail environments. One is described as "Scan purchases, process payments and returns at the register and bag or refund items." These are register tasks. The catalog does not describe them as computer use or as operating a point-of-sale terminal, and it states no scanner or point-of-sale data stream, so they are left out of this page. The second table lists them with the reason. Anything beyond these rows, such as a particular software task or a particular input device, is a skill-spec question and is "Not stated".

The same stream set as any other custom egocentric collection on this site: ten hardware-synchronized streams on the instrumented tier, including a 4K head camera, two wrist cameras, depth, IMU, 3D hand and body pose, instance segmentation and action labels. The streams are hardware-triggered onto one PTP clock domain and aligned under 2 ms. Depth is recovered from the calibrated stereo rig, and the IMU is 6-DoF at 200 Hz from head- and wrist-mounted sensors.

For desk work the labels that matter most are on the hands. Hand pose is 21 joints per hand in 3D, for both hands, with a per-joint visibility flag. Contact events mark when a hand meets or releases an item, such as a phone, a sheet of paper or a pen. Action segments carry a start time, an end time, a verb and a noun for each step as the spec defines it, so a continuous run of desk work still yields individually labeled steps. The only language field the site states is one task instruction per episode.

The catalog lists the capture configuration per environment: Office & Professional Services is captured with a head-mounted phone or a multiview rig. It does not state a per-environment stream list, so whether a phone-captured episode carries every one of the ten streams is "Not stated" and is decided in the skill spec. Every session is calibrated twice, and calibration is accepted only under the 0.28 px RMS reprojection gate.

Each episode also ships with its record: the consent chain for the contributor, the anonymization record, and provenance and data-processing documentation for your legal team. The table that follows lists each field, with the name it carries in the episode schema where the site states one.

Labeling starts in the spec, not after capture. A spec for desk work splits the task into ordered sub-steps and names each with a verb and a noun, and each becomes an action segment. Which sub-steps your model needs, and in what order, is yours to write. The site does not fix a list of desk work sub-steps, so this page does not either.

Every episode passes calibration and synchronization checks on every active stream, plus label and annotation review for hand pose and action segments where those are included, before delivery. An item is rejected when the manipulated object is out of frame for too much of the episode, when depth is invalid over too much of the manipulation region, or when an undocumented bystander appears. The thresholds are agreed in the spec, so the batch is judged against terms you signed. Episodes that fail are rejected rather than shipped with a caveat, and the reject log is delivered with a Spec Pilot batch.

Finished data ships in RLDS, LeRobot, WebDataset, HDF5, zarr or Rerun .rrd, or as raw streams with metadata, whichever your pipeline already consumes. RLDS and LeRobot are built around episodic, step-based sequences, which maps onto a desk task already split into episodes with action-segment labels. Because the structure is set at the spec stage, the same capture can be re-exported into another format later. The third table lists each format with the official source read for this page.

The license is perpetual, irrevocable, worldwide and non-exclusive by default, owned by the buyer, with exclusivity available, so the data can go into a commercial training run without a separate rights negotiation. The executed agreement controls.

A robot or assistant that will work at a desk needs to see desk work the way the person doing it sees it. Footage from across the room shows that someone is seated at a workstation. It does not show how the fingers meet a keyboard, how a hand shifts from a phone to a stack of paper, or how a pen is lifted and set down. A head-mounted camera records the reach, the grasp and the hand-to-item relationship from the viewpoint a mobile manipulator or humanoid would have.

The data pyramid framing in the GR00T N1 paper puts web data and human videos at the base, synthetic data in the middle and real-world robot data at the top, and describes human video datasets as not including explicit action labels. That is the gap action segments, hand pose and contact events are meant to fill: the video is human, and the step structure is written down. Synthetic and robot data are not part of this offer.

The tradeoff to plan for is the embodiment gap between a human hand and a robot end-effector, which retargeting or pairing with robot data has to close. What you get is human first-person demonstrations in which the hands, the items and the step structure are explicit.

It is just as important to say what this collection does not cover. A model that must read an interface, or learn which click follows which state, needs interface traces. The site states no screen-capture stream, cursor or click log, keystroke log or accessibility-tree stream, and legible screens are blurred at ingest. The collection serves the embodied half of desk work. A team that wants both halves can treat this video as one input and source interface traces separately, or say in its request that it needs a screen arrangement scoped.

The catalog places desk work in Office & Professional Services, with examples naming a corporate office, a co-working space and a reception area. It does not state whose premises a row is recorded on, and it does not say whether the work is staged or recorded in a live workplace. This page does not claim capture at a customer’s own office. The site does state that site access and permission are confirmed for every location a contributor proposes.

On privacy, the anonymization page states that a vision model runs on every frame at ingest, flagging faces, heads, licence plates and any legible screen or document. Faces, plates and screens are then irreversibly blurred, and a human reviewer confirms the result before anything ships. If the reviewer is unsure a region is fully covered, the episode is held or discarded, never downgraded and shipped. The anonymization record is delivered with the data, so a buyer can see what was blurred and when.

People are handled alongside it. Every contributor signs a documented release as part of a consent chain that travels with the data. Bystanders such as colleagues passing behind a desk are blurred at ingest, and clips where a bystander cannot be de-identified are discarded rather than delivered with a quality caveat.

The practical consequence is that legible screen content, and any document or badge in frame, is not in the delivered pixels, and it is not a deliverable. If your pipeline needs the content of a screen or a document, raise it before the spec is signed. Whether any engagement can keep specific content is not stated on this site.

The site states a median of 48 hours from signed spec to first validated batch where coverage is deep, and three to four weeks where contributors have to be recruited into an unfamiliar environment. It does not say which of the two applies to desk work, and it states no turnaround specific to this row. Whether coverage is deep or thin for your location and task list is settled when the spec is scoped.

Because the collection is custom and on demand, coverage follows your task list rather than whatever happened to be recorded. That is what fast, on-demand custom collection means here: the first batch is captured against your spec and judged by your accept criteria, not drawn from a stock library.

Most programs start with a Spec Pilot, a small, fast first batch captured against the spec and QA’d against the same accept criteria as a full order. The same written spec then scales into production volume in batches. Counts of episodes, contributors, hours and sites are set in the spec and the quote, and are not fixed numbers on this site.

Describe the desk work you need to see: the tasks, the desk layout and the objects on it, the sub-steps of each task and their order, and the starting states. Say which streams and labels your pipeline needs, what counts as a correct completion, and the delivery format you want. A spec written this way is the same one used for every custom collection, and only the setting, tasks and objects change.

Then decide what variation you need: different desks and rooms, different people, different starting states such as papers out of order or a phone out of reach. Where a task is prone to going wrong, ask for failure and recovery cases, which a spec can require deliberately. If you need a screen arrangement scoped, or any step the catalog does not list, such as mouse use or typing, say so in the request so it is scoped rather than assumed.

The request becomes a written skill spec, quoted for reach, timeline and price before any capture begins. Most programs then run a Spec Pilot, check its reject log and accept criteria, and scale from there.

Where this site does not state something, this page says "Not stated" rather than filling the gap. There is no mouse row, typing row, data entry row, row that names a computer or software, or row for operating a terminal. There is no screen-capture, cursor, click, keystroke or accessibility-tree stream, no readable screen content in delivered video, and no typing-speed, accuracy or task-completion label. No head or camera pose, map, gaze or eye-tracking stream is stated either: the only motion stream the site states is the 200 Hz 6-DoF IMU.

Counts of episodes, hours, sites and contributors are not fixed numbers on this site. They are set in your skill spec. No workplace type beyond the catalog’s own examples is stated, no region or city, no input device, and no result from a robot or an agent trained on this kind of data. The final table lists each of these.