Knowledge is Never Enough: Towards Web Aided Deep Open World Recognition

TL;DR

This paper presents a deep learning system that can recognize known objects, detect unknown ones, and dynamically learn new categories without retraining from scratch, enhancing robotic perception in open-world environments.

Contribution

It introduces an end-to-end deep architecture extending non-parametric models to enable dynamic knowledge expansion in visual recognition systems.

Findings

Effective detection of unknown objects in real-world scenarios

Successful dynamic learning of new categories from minimal supervision

Demonstrated on databases and a robot platform

Abstract

While today's robots are able to perform sophisticated tasks, they can only act on objects they have been trained to recognize. This is a severe limitation: any robot will inevitably see new objects in unconstrained settings, and thus will always have visual knowledge gaps. However, standard visual modules are usually built on a limited set of classes and are based on the strong prior that an object must belong to one of those classes. Identifying whether an instance does not belong to the set of known categories (i.e. open set recognition), only partially tackles this problem, as a truly autonomous agent should be able not only to detect what it does not know, but also to extend dynamically its knowledge about the world. We contribute to this challenge with a deep learning architecture that can dynamically update its known classes in an end-to-end fashion. The proposed deep network, based on a deep extension of a non-parametric model, detects whether a perceived object belongs to the set of categories known by the system and learns it without the need to retrain the whole system from scratch. Annotated images about the new category can be provided by an 'oracle' (i.e. human supervision), or by autonomous mining of the Web. Experiments on two different databases and on a robot platform demonstrate the promise of our approach.