MetH: A family of high-resolution and variable-shape image challenges

TL;DR

This paper introduces MetH, a new family of high-resolution, variable-shape image datasets for AI research, addressing limitations of down-sampling and promoting advancements in high-res image analysis.

Contribution

The paper presents MetH, a diverse set of high-resolution, variable-shape image datasets with expert labels, and analyzes their challenges compared to existing datasets.

Findings

Current architectures perform poorly on MetH tasks.

MetH datasets exceed existing datasets in size and aspect ratio variability.

There is significant potential for improving AI models on high-resolution, variable-shape images.

Abstract

High-resolution and variable-shape images have not yet been properly addressed by the AI community. The approach of down-sampling data often used with convolutional neural networks is sub-optimal for many tasks, and has too many drawbacks to be considered a sustainable alternative. In sight of the increasing importance of problems that can benefit from exploiting high-resolution (HR) and variable-shape, and with the goal of promoting research in that direction, we introduce a new family of datasets (MetH). The four proposed problems include two image classification, one image regression and one super resolution task. Each of these datasets contains thousands of art pieces captured by HR and variable-shape images, labeled by experts at the Metropolitan Museum of Art. We perform an analysis, which shows how the proposed tasks go well beyond current public alternatives in both pixel size and aspect ratio variance. At the same time, the performance obtained by popular architectures on these tasks shows that there is ample room for improvement. To wrap up the relevance of the contribution we review the fields, both in AI and high-performance computing, that could benefit from the proposed challenges.