FMAC: a Fair Fiducial Marker Accuracy Comparison Software

TL;DR

This paper introduces FMAC, an open-source software for fair and accurate comparison of fiducial marker pose estimation, utilizing high-fidelity synthetic images and detailed rendering techniques.

Contribution

It provides a novel, comprehensive framework for evaluating fiducial marker accuracy using synthetic images with realistic distortions and a new pose evaluation method.

Findings

Reveals strengths and weaknesses of well-known markers for pose estimation

Provides a validated rendering algorithm for high-fidelity synthetic images

Enables fair comparison of fiducial marker accuracy

Abstract

This paper presents a method for carrying fair comparisons of the accuracy of pose estimation using fiducial markers. These comparisons rely on large sets of high-fidelity synthetic images enabling deep exploration of the 6 degrees of freedom. A low-discrepancy sampling of the space allows to check the correlations between each degree of freedom and the pose errors by plotting the 36 pairs of combinations. The images are rendered using a physically based ray tracing code that has been specifically developed to use the standard calibration coefficients of any camera directly. The software reproduces image distortions, defocus and diffraction blur. Furthermore, sub-pixel sampling is applied to sharp edges to enhance the fidelity of the rendered image. After introducing the rendering algorithm and its experimental validation, the paper proposes a method for evaluating the pose accuracy. This method is applied to well-known markers, revealing their strengths and weaknesses for pose estimation. The code is open source and available on GitHub.