Recognizing Unseen States of Unknown Objects by Leveraging Knowledge Graphs

TL;DR

This paper introduces a novel zero-shot learning approach for object state classification that leverages knowledge graphs to infer states of unseen objects without relying on object class knowledge.

Contribution

It proposes the first object-agnostic state classification method utilizing knowledge graphs, enabling state inference without object class dependence.

Findings

Outperforms existing methods across multiple datasets

Object class knowledge is not essential for state prediction

Effective in zero-shot and unseen object scenarios

Abstract

We investigate the problem of Object State Classification (OSC) as a zero-shot learning problem. Specifically, we propose the first Object-agnostic State Classification (OaSC) method that infers the state of a certain object without relying on the knowledge or the estimation of the object class. In that direction, we capitalize on Knowledge Graphs (KGs) for structuring and organizing knowledge, which, in combination with visual information, enable the inference of the states of objects in object/state pairs that have not been encountered in the method's training set. A series of experiments investigate the performance of the proposed method in various settings, against several hypotheses and in comparison with state of the art approaches for object attribute classification. The experimental results demonstrate that the knowledge of an object class is not decisive for the prediction of its state. Moreover, the proposed OaSC method outperforms existing methods in all datasets and benchmarks by a great margin.