Glossary
Instance segmentation
Per-pixel masks that identify and separate individual objects over time.
Definition
Instance segmentation assigns each pixel to a specific object instance, separating, say, two identical mugs rather than just labeling “mug”. Tracked across frames, it gives a policy a persistent handle on which object is being manipulated.
01
Why it matters
Manipulation is about a particular object. Instance masks let a model attend to the manipulated item through clutter and occlusion, and tie contact events to a specific object id.
02
How Firsthand handles it
Firsthand ships instance_masks as tracked object ids per frame, linked to an objects list and to contact events in the annotations.