ALET (Automated Labeling of Equipment and Tools): A Dataset, a Baseline and a Usecase for Tool Detection in the Wild

TL;DR

This paper introduces METU-ALET, a comprehensive dataset for tool detection in real-world environments, evaluates several state-of-the-art detectors, and highlights challenges in detecting small or similar-looking tools, supporting future robotics research.

Contribution

Provides the first extensive dataset for tool detection in diverse real-world environments and benchmarks multiple deep detectors on this dataset.

Findings

Detectors struggle with small and visually similar tools.

The dataset reveals challenges like occlusion and articulation.

Baseline models provide a starting point for future research.

Abstract

Robots collaborating with humans in realistic environments will need to be able to detect the tools that can be used and manipulated. However, there is no available dataset or study that addresses this challenge in real settings. In this paper, we fill this gap by providing an extensive dataset (METU-ALET) for detecting farming, gardening, office, stonemasonry, vehicle, woodworking and workshop tools. The scenes correspond to sophisticated environments with or without humans using the tools. The scenes we consider introduce several challenges for object detection, including the small scale of the tools, their articulated nature, occlusion, inter-class invariance, etc. Moreover, we train and compare several state of the art deep object detectors (including Faster R-CNN, Cascade R-CNN, RepPoint and RetinaNet) on our dataset. We observe that the detectors have difficulty in detecting especially small-scale tools or tools that are visually similar to parts of other tools. This in turn supports the importance of our dataset and paper. With the dataset, the code and the trained models, our work provides a basis for further research into tools and their use in robotics applications.