A Canonical Image Set for Examining and Comparing Image Processing Algorithms

TL;DR

This paper introduces a standardized set of four test images designed to improve the consistency and rigor in evaluating and comparing image processing algorithms across research studies.

Contribution

It proposes a canonical image set to replace ad hoc images like Lena, enabling more rigorous and comparable evaluation of image processing algorithms.

Findings

The four NRL images effectively expose algorithm characteristics.

Using a standard set reduces result cherry-picking.

Facilitates consistent comparison across studies.

Abstract

The purpose of this paper is to introduce a set of four test images containing features and structures that can facilitate effective examination and comparison of image processing algorithms. More specifically, the images are designed to more explicitly expose the characteristic properties of algorithms for image compression, virtual resolution adjustment, and enhancement. This set was developed at the Naval Research Laboratory (NRL) in the late 1990s as a more rigorous alternative to Lena and other images that have come into common use for purely ad hoc reasons with little or no rigorous consideration of their suitability. The increasing number of test images appearing in the literature not only makes it more difficult to compare results from different papers, it also introduces the potential for cherry-picking to influence results. The key contribution of this paper is the proposal to establish {\em some} canonical set to ensure that published results can be analyzed and compared in a rigorous way from one paper to another, and consideration of the four NRL images is proposed for this purpose.