Semantically Meaningful View Selection

TL;DR

This paper introduces the problem of semantic view selection for robots, proposing a dataset and neural network approach to identify camera angles that improve object recognition by capturing more informative views.

Contribution

It formulates the semantic view selection problem, creates a new dataset of object views, and presents a neural network solution to predict informative camera poses.

Findings

Predicted views with higher semantic scores improve clustering results.

The dataset contains around 10,000 images of 144 objects from various angles.

The neural network effectively predicts views that enhance object recognition.

Abstract

An understanding of the nature of objects could help robots to solve both high-level abstract tasks and improve performance at lower-level concrete tasks. Although deep learning has facilitated progress in image understanding, a robot's performance in problems like object recognition often depends on the angle from which the object is observed. Traditionally, robot sorting tasks rely on a fixed top-down view of an object. By changing its viewing angle, a robot can select a more semantically informative view leading to better performance for object recognition. In this paper, we introduce the problem of semantic view selection, which seeks to find good camera poses to gain semantic knowledge about an observed object. We propose a conceptual formulation of the problem, together with a solvable relaxation based on clustering. We then present a new image dataset consisting of around 10k images representing various views of 144 objects under different poses. Finally we use this dataset to propose a first solution to the problem by training a neural network to predict a "semantic score" from a top view image and camera pose. The views predicted to have higher scores are then shown to provide better clustering results than fixed top-down views.