Multi-Part Object Representations via Graph Structures and Co-Part Discovery

TL;DR

This paper introduces a novel explicit graph-based approach for discovering multi-part object representations, improving robustness and generalization in occluded and out-of-distribution scenarios.

Contribution

It proposes a new method leveraging explicit graph structures and a co-part discovery algorithm, along with benchmarks for robustness evaluation.

Findings

Improved object discovery quality over state-of-the-art methods

Enhanced recognition of multi-part objects in occluded settings

More accurate prediction of object properties in downstream tasks

Abstract

Discovering object-centric representations from images can significantly enhance the robustness, sample efficiency and generalizability of vision models. Works on images with multi-part objects typically follow an implicit object representation approach, which fail to recognize these learned objects in occluded or out-of-distribution contexts. This is due to the assumption that object part-whole relations are implicitly encoded into the representations through indirect training objectives. We address this limitation by proposing a novel method that leverages on explicit graph representations for parts and present a co-part object discovery algorithm. We then introduce three benchmarks to evaluate the robustness of object-centric methods in recognizing multi-part objects within occluded and out-of-distribution settings. Experimental results on simulated, realistic, and real-world images show marked improvements in the quality of discovered objects compared to state-of-the-art methods, as well as the accurate recognition of multi-part objects in occluded and out-of-distribution contexts. We also show that the discovered object-centric representations can more accurately predict key object properties in a downstream task, highlighting the potential of our method to advance the field of object-centric representations.