Open-Ended Fine-Grained 3D Object Categorization by Combining Shape and Texture Features in Multiple Colorspaces

TL;DR

This paper introduces a deep transfer learning method for open-ended 3D object recognition that combines shape and texture features across multiple color spaces, enabling robots to learn new categories interactively in real-time.

Contribution

It proposes a novel approach integrating shape and texture features in multiple color spaces for scalable, real-time 3D object recognition with human-in-the-loop learning.

Findings

Outperforms state-of-the-art in classification accuracy

Demonstrates real-time recognition on a service robot

Effective in open-ended learning scenarios

Abstract

As a consequence of an ever-increasing number of service robots, there is a growing demand for highly accurate real-time 3D object recognition. Considering the expansion of robot applications in more complex and dynamic environments,it is evident that it is not possible to pre-program all object categories and anticipate all exceptions in advance. Therefore, robots should have the functionality to learn about new object categories in an open-ended fashion while working in the environment.Towards this goal, we propose a deep transfer learning approach to generate a scale- and pose-invariant object representation by considering shape and texture information in multiple colorspaces. The obtained global object representation is then fed to an instance-based object category learning and recognition,where a non-expert human user exists in the learning loop and can interactively guide the process of experience acquisition by teaching new object categories, or by correcting insufficient or erroneous categories. In this work, shape information encodes the common patterns of all categories, while texture information is used to describes the appearance of each instance in detail.Multiple color space combinations and network architectures are evaluated to find the most descriptive system. Experimental results showed that the proposed network architecture out-performed the selected state-of-the-art approaches in terms of object classification accuracy and scalability. Furthermore, we performed a real robot experiment in the context of serve-a-beer scenario to show the real-time performance of the proposed approach.