Delivery format

WebDataset delivery format for egocentric episodes

Tar shards of 512 samples, streamed from signed URLs — for training beyond local disk.

By the Firsthand capture teamLast updated September 23, 2026

Short answer

WebDataset stores samples as sharded tar archives so large datasets stream efficiently during training without a random-access database. Firsthand ships .tar shards of 512 samples with signed-URL streaming, so you can train over more hours than you want to stage locally.

Field mapping

How Firsthand fields map into this format

Container: .tar shards, 512 samples per shard
Keys per sample: __key__, head.jpg, wrist_l.jpg, wrist_r.jpg, depth.png, ann.json
Ordering: Shuffled across episodes, sequence preserved inside a sample
Streaming: Signed URLs, works directly with wds.WebDataset(url)
Annotations: ann.json carries hand_joints, action, contact_events

Loading it

load_webdataset.py

import webdataset as wds

url = "https://cdn.ifirsthand.com/kitchen_knife_dice/{000000..000217}.tar"
ds = (
    wds.WebDataset(url, shardshuffle=True)
    .decode("rgb")
    .to_tuple("head.jpg", "depth.png", "ann.json")
)
for head, depth, ann in ds:
    ...  # ann is parsed JSON with hand_joints and action

Gotchas

What to watch for

Signed URLs expire — regenerate them with fh stream rather than hard-coding a URL.
depth.png is 16-bit; decode it as uint16 millimetres, not as an 8-bit image.
Shuffling is across shards; set a large enough shuffle buffer for good mixing.

Get a WebDataset sample.

40 episodes across 4 environments, delivered in the format your stack already reads.

Get sample data →All formats