Template-based Object Detection Using a Foundation Model

TL;DR

This paper introduces a template-based object detection method that leverages foundation models and simple classification, eliminating the need for training data and enabling quick adaptation for specific use cases like UI testing.

Contribution

The paper presents a novel, training-free object detection approach using foundation model segments combined with feature-based classification for specific, low-variation tasks.

Findings

Achieves detection accuracy comparable to YOLO without training.

Reduces time and cost in adapting object detection to new objects.

Effective for automating UI testing in automotive industry.

Abstract

Most currently used object detection methods are learning-based, and can detect objects under varying appearances. Those models require training and a training dataset. We focus on use cases with less data variation, but the requirement of being free of generation of training data and training. Such a setup is for example desired in automatic testing of graphical interfaces during software development, especially for continuous integration testing. In our approach, we use segments from segmentation foundation models and combine them with a simple feature-based classification method. This saves time and cost when changing the object to be searched or its design, as nothing has to be retrained and no dataset has to be created. We evaluate our method on the task of detecting and classifying icons in navigation maps, which is used to simplify and automate the testing of user interfaces in automotive industry. Our methods achieve results almost on par with learning-based object detection methods like YOLO, without the need for training.