DeepArUco++: Improved detection of square fiducial markers in challenging lighting conditions

TL;DR

DeepArUco++ introduces a deep learning framework for robust detection and decoding of fiducial markers in difficult lighting, outperforming classical methods and maintaining effectiveness across challenging conditions.

Contribution

The paper presents a novel deep learning pipeline for fiducial marker detection under challenging lighting, including synthetic data generation and real-world evaluation.

Findings

Outperforms state-of-the-art methods in difficult lighting conditions

Maintains high accuracy even in extreme illumination variations

Effective synthetic data generation improves model training

Abstract

Fiducial markers are a computer vision tool used for object pose estimation and detection. These markers are highly useful in fields such as industry, medicine and logistics. However, optimal lighting conditions are not always available,and other factors such as blur or sensor noise can affect image quality. Classical computer vision techniques that precisely locate and decode fiducial markers often fail under difficult illumination conditions (e.g. extreme variations of lighting within the same frame). Hence, we propose DeepArUco++, a deep learning-based framework that leverages the robustness of Convolutional Neural Networks to perform marker detection and decoding in challenging lighting conditions. The framework is based on a pipeline using different Neural Network models at each step, namely marker detection, corner refinement and marker decoding. Additionally, we propose a simple method for generating synthetic data for training the different models that compose the proposed pipeline, and we present a second, real-life dataset of ArUco markers in challenging lighting conditions used to evaluate our system. The developed method outperforms other state-of-the-art methods in such tasks and remains competitive even when testing on the datasets used to develop those methods. Code available in GitHub: https://github.com/AVAuco/deeparuco/