What Is EGTEA Gaze+?

The paper by Yin Li, Miao Liu and James M. Rehg addresses the task of jointly determining what a person is doing and where they are looking from video captured by a headworn camera. To support that work the authors introduced EGTEA Gaze+, which they describe as providing videos, gaze tracking data, hand masks and action annotations, and they propose a deep model that treats gaze as a probabilistic variable to guide action recognition. The dataset is hosted on a Georgia Tech egocentric activity datasets page.

EGTEA Gaze+ is a third-party research dataset. Firsthand is not part of it or affiliated with its authors, and makes no claim that any model was trained on Firsthand data. What the site states is custom egocentric collection to a written spec, fast and on demand. Gaze is a spec decision: Firsthand’s episode schema lists a gaze_target field, shown in the docs layout as a timestamp and an object id, delivered beside 21-joint hand pose and 200 Hz IMUs when the skill spec calls for it. The site names no eye-tracking device, gaze sampling rate or accuracy figure, and none are claimed here.

Per the official dataset page, it was collected at Georgia Tech’s AwareHome and covers seven meal-preparation activities performed by 26 subjects following given recipes: American Breakfast, Pizza, Snack, Greek Salad, Pasta Salad, Turkey Sandwich and Cheese Burger. The page distinguishes an activity, such as making pizza, from an action, a short segment such as putting sauce on the crust.

The page states that the current version contains 37 videos with gaze tracking and action annotations, and that the action annotations include 10,325 instances of fine-grained actions. It also lists 15,176 hand masks from 13,847 frames.

The dataset page says SMI eye-tracking glasses recorded HD video of the subjects’ activities at 24 frames per second, and recorded gaze at 30 frames per second. The packaged download lists gaze data from wearable gaze tracking at 30 Hz, trimmed action clips at 640x480 and 24 fps, and hand masks at 960x720.

Those are the dataset’s own capture figures. They say nothing about how Firsthand records gaze, and the two are not compared on speed or accuracy.

Action labels say what happened and when. Gaze says where the wearer was attending while it happened. The authors use that link directly: their model samples from a distribution over gaze to build an attention map that guides how visual features are aggregated for action recognition.

The paper also reports that the model can be applied to the larger EPIC-KITCHENS dataset even without using gaze, giving new state-of-the-art results on first-person action recognition at the time. Those are the authors’ results on their own benchmarks, and Firsthand quotes no figure from them.

The site’s EPIC-KITCHENS entry covers a larger unscripted kitchen dataset with timed verb and noun labels, and its Ego4D entry covers a broad everyday-activity dataset where gaze is a subset modality. EGTEA Gaze+ is smaller and scripted around seven recipes, with gaze as a core part of the release.

All three are third-party work, and Firsthand ranks none of them.

Licence terms are Not stated in the sources read, so check the dataset page and your own counsel before commercial use. The dataset is cooking in one lab-run home setting, and the sources read state no robot results.

Hand data is hand masks, which mark where hands are in the image, not a 3D skeleton. Firsthand states 3D hand pose at 21 joints per hand, but the site does not map one to the other.

When a program needs gaze beside hand pose, depth and IMU for tasks, environments or objects a public dataset does not cover, specified in writing. Name gaze in the skill spec and confirm it on a Spec Pilot, a small fast first batch. Firsthand does not build or train models.

Where coverage is deep, the median from signed spec to first validated batch is 48 hours; where it is thin, three to four weeks. Billing is per validated hour.

What is EGTEA Gaze+?

A public egocentric dataset of first-person meal-preparation video with gaze tracking data, hand masks and action annotations, introduced in arXiv 2006.00626.

What gaze data does EGTEA Gaze+ include?

The dataset page lists wearable gaze tracking at 30 Hz, recorded with SMI eye-tracking glasses beside video at 24 fps.

How many videos and actions does it have?

The dataset page states 37 videos in the current version and 10,325 instances of fine-grained action annotations, performed by 26 subjects across seven recipes.

Can EGTEA Gaze+ be used commercially?

Licence terms are Not stated in the sources read. Check the dataset page directly before relying on it.

Does Firsthand data train models on EGTEA Gaze+?

No claim is made. Firsthand is not affiliated with its authors, and does not state that any model was trained on Firsthand data.

Can Firsthand collect egocentric data with gaze to a custom spec?

The site lists a gaze_target field as a spec decision, delivered with hand pose and IMU when requested, collected fast and on demand and confirmed on a Spec Pilot. Eye-tracker hardware and sampling rate are Not stated.