Trixi the Librarian

TL;DR

This paper introduces a three-part system that combines deep learning and traditional computer vision to automatically sort books on a shelf using a bi-manual robot, addressing recognition and manipulation challenges.

Contribution

It presents a novel integrated approach for book recognition and manipulation with a robotic system, including a vision pipeline and solutions for robotic relocation tasks.

Findings

System successfully sorts three books repeatedly

Deep learning enhances book recognition accuracy

Robotic manipulation achieves functional relocation

Abstract

In this work, we present a three-part system that automatically sorts books on a shelf using the PR- 2 platform. The paper describes a methodology to sufficiently detect and recognize books using a multistep vision pipeline based on deep learning models as well as conventional computer vision. Furthermore, the difficulties of relocating books using a bi-manual robot along with solutions based on MoveIt and BioIK are being addressed. Experiments show that the performance is overall good enough to repeatedly sort three books on a shelf. Nevertheless, further improvements are being discussed, potentially leading to a more robust book recognition and more versatile manipulation techniques.