Measuring Human Assessed Complexity in Synthetic Aperture Sonar Imagery Using the Elo Rating System

TL;DR

This study develops a quantitative complexity metric for synthetic aperture sonar images based on expert rankings and compares it with various information theoretic and heuristic measures, finding strong correlation with spatial information variations.

Contribution

Introduces a novel complexity metric derived from SME rankings and relates it to existing information theoretic and heuristic measures in SAS imagery.

Findings

Spatial information variation metrics correlate highly with SME perceived complexity (R^2 ≈ 0.9).

Elo ranking method effectively orders image complexity across diverse environments.

Other measures like lacunarity, compression, and entropy show varying degrees of correlation.

Abstract

Performance of automatic target recognition from synthetic aperture sonar data is heavily dependent on the complexity of the beamformed imagery. Several mechanisms can contribute to this, including unwanted vehicle dynamics, the bathymetry of the scene, and the presence of natural and manmade clutter. To understand the impact of the environmental complexity on image perception, researchers have taken approaches rooted in information theory, or heuristics. Despite these efforts, a quantitative measure for complexity has not been related to the phenomenology from which it is derived. By using subject matter experts (SMEs) we derive a complexity metric for a set of imagery which accounts for the underlying phenomenology. The goal of this work is to develop an understanding of how several common information theoretic and heuristic measures are related to the SME perceived complexity in synthetic aperture sonar imagery. To achieve this, an ensemble of 10-meter x 10-meter images were cropped from a high-frequency SAS data set that spans multiple environments. The SME's were presented pairs of images from which they could rate the relative image complexity. These comparisons were then converted into the desired sequential ranking using a method first developed by A. Elo for establishing rankings of chess players. The Elo method produced a plausible rank ordering across the broad dataset. The heuristic and information theoretical metrics were then compared to the image rank from which they were derived. The metrics with the highest degree of correlation were those relating to spatial information, e.g. variations in pixel intensity, with an R-squared value of approximately 0.9. However, this agreement was dependent on the scale from which the spatial variation was measured. Results will also be presented for many other measures including lacunarity, image compression, and entropy.