Beyond Object Recognition: A New Benchmark towards Object Concept Learning

TL;DR

This paper introduces a new challenging benchmark for object concept learning that requires AI systems to reason about object attributes and affordances, supported by a densely annotated knowledge base and a causal reasoning model.

Contribution

It proposes the Object Concept Learning task, creates a comprehensive knowledge base, and develops a causal reasoning network to improve understanding of object attributes and affordances.

Findings

OCRN effectively infers object concepts following causal relations

The knowledge base enables detailed object attribute and affordance annotations

The approach demonstrates promising results in causal reasoning for object understanding

Abstract

Understanding objects is a central building block of artificial intelligence, especially for embodied AI. Even though object recognition excels with deep learning, current machines still struggle to learn higher-level knowledge, e.g., what attributes an object has, and what can we do with an object. In this work, we propose a challenging Object Concept Learning (OCL) task to push the envelope of object understanding. It requires machines to reason out object affordances and simultaneously give the reason: what attributes make an object possesses these affordances. To support OCL, we build a densely annotated knowledge base including extensive labels for three levels of object concept (category, attribute, affordance), and the causal relations of three levels. By analyzing the causal structure of OCL, we present a baseline, Object Concept Reasoning Network (OCRN). It leverages causal intervention and concept instantiation to infer the three levels following their causal relations. In experiments, OCRN effectively infers the object knowledge while following the causalities well. Our data and code are available at https://mvig-rhos.com/ocl.