CaRe-Ego: Contact-aware Relationship Modeling for Egocentric Interactive Hand-object Segmentation
Yuejiao Su, Yi Wang, and Lap-Pui Chau

TL;DR
CaRe-Ego introduces a contact-aware approach for egocentric hand-object segmentation, emphasizing hand-object interactions and disentangling object category relationships to improve segmentation accuracy and generalization.
Contribution
The paper proposes CaRe-Ego, a novel method that models hand-object contact and decouples object category relationships to enhance segmentation performance.
Findings
Achieves state-of-the-art segmentation accuracy.
Demonstrates robust generalization on various test sets.
Outperforms existing methods significantly.
Abstract
Egocentric Interactive hand-object segmentation (EgoIHOS) requires the segmentation of hands and interacting objects in egocentric images, which is crucial for understanding human behavior in assistive systems. Previous methods typically recognize hands and interacting objects as distinct semantic categories based solely on visual features, or simply use hand predictions as auxiliary cues for object segmentation. Despite the promising progress achieved by these methods, they fail to adequately model the interactive relationships between hands and objects while ignoring the coupled physical relationships among object categories, ultimately constraining their segmentation performance. To make up for the shortcomings of existing methods, we propose a novel method called CaRe-Ego that achieves state-of-the-art performance by emphasizing the contact between hands and objects from two…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Code & Models
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsHand Gesture Recognition Systems · Human Pose and Action Recognition
