What Is EPIC-KITCHENS?

EPIC-KITCHENS began as a 2018 dataset and was extended in the paper Rescaling Egocentric Vision: Collection Pipeline and Challenges for EPIC-KITCHENS-100 (arXiv 2006.13256, IJCV), by researchers at the University of Bristol and the University of Catania. Participants recorded 2 to 4 days of their typical kitchen activities, and the authors describe the footage as unscripted and unedited, which leaves the data naturally unbalanced. One annotation pipeline supports several video-understanding challenges, including action recognition, detection and anticipation.

EPIC-KITCHENS is a third-party research dataset, and Firsthand is not part of it, endorsed by it or affiliated with its authors or their institutions. Firsthand makes no performance claim against it. What Firsthand states is custom egocentric collection: video from a head-mounted global-shutter camera plus two wrist cameras on one hardware-triggered clock, with depth, 3D hand pose and action segments, collected fast, on demand and to a written skill spec from paid, consented contributors, under a perpetual, buyer-owned commercial license. A public release has fixed scope set by its authors, while a custom collection is scoped to your spec.

The paper describes 100 hours, 20M frames and 90K actions across 700 variable-length videos, recorded in 45 environments, with 37 participants. Table 1 gives 89,977 action segments, 97 verb classes, 300 noun classes and 4,053 action classes. The extension added 45.4 hours and 268 videos to the 54.6 hours and 432 videos of the earlier edition.

Videos are split into train, validation and test at roughly 75/10/15, and the test split holds only newly collected videos. The paper also reports automatic annotations of 66M masks, 31M hand boxes and 38M object boxes, and says the mask quality has not been quantitatively evaluated and is error-prone.

Participants narrated their own footage with a pause-and-talk tool, in their native language. The narrations were transcribed by crowd workers, translated to English, parsed into verbs and nouns and grouped into classes, and other crowd workers then marked each segment’s start and end times.

The GitHub README states that ground truth is provided for action segments as action, verb and noun labels with start and end times. Its training file has 15 columns, including narration_id, participant_id, video_id, start and stop timestamps and frames, the narration, and verb and noun with their class IDs.

The paper and README list six challenges: action recognition, weakly supervised action recognition, action detection, action anticipation, unsupervised domain adaptation and multi-instance retrieval. The paper reports that models trained on the 2018 footage do not withstand the test of time on footage collected two years later, and that adding new data saturated, which the authors say highlights the need for more diverse data rather than merely more data. These are the authors’ results on their own setup.

The official site also lists derived datasets built on it: VISOR, EPIC-Sounds and EPIC-Fields, plus a separate HD-EPIC dataset of 41 hours.

The paper’s data release statement says the dataset sequences, extracted frames and optical flow are available under the Non-Commercial Government Licence for public sector information, at the University of Bristol data repository. The annotations repository README states that the annotations are under the Creative Commons Attribution-NonCommercial 4.0 International License, and provides a download script for videos and frames.

Both licences are named as non-commercial. Read the full licence text before planning any commercial training use. Consent and privacy terms are Not stated in the sources read.

Write the target skills, environments and signals into a skill spec and confirm them on a Spec Pilot, a small fast first batch. This is on-demand custom collection, not a catalog download.

Where coverage is deep, the median from signed spec to first validated batch is 48 hours; where it is thin, three to four weeks. Billing is per validated hour.

What is EPIC-KITCHENS?

A public dataset of egocentric kitchen video recorded with head-mounted cameras, annotated with timed verb and noun action segments.

How large is EPIC-KITCHENS-100?

The paper states 100 hours, 20M frames, 700 videos, 89,977 action segments and 45 environments.

Can EPIC-KITCHENS be used commercially?

Its video is released under the Non-Commercial Government Licence and its annotations under CC BY-NC 4.0, so both are named as non-commercial. Read the full licence text first.

What is EPIC-KITCHENS used for?

Six challenges: action recognition, weakly supervised recognition, detection, anticipation, unsupervised domain adaptation and multi-instance retrieval.

How is EPIC-KITCHENS labeled?

Participants narrated their own footage, crowd workers transcribed it, and the captions were parsed into verb and noun classes with start and end times.

Is Firsthand data part of EPIC-KITCHENS?

No. Firsthand is not part of it, endorsed by it or affiliated with its authors, and makes no performance claim against it.

How fast can custom egocentric data be collected?

Where coverage is deep, the median from signed spec to first validated batch is 48 hours; where it is thin, three to four weeks.