The MAMe Dataset: On the relevance of High Resolution and Variable Shape image properties

TL;DR

The MAMe dataset introduces high-resolution, variable-shape images of artworks to study their impact on classification performance, revealing benefits of high resolution and highlighting challenges with variable shapes.

Contribution

This work presents the MAMe dataset with high-resolution, variable-shape images for art classification, enabling research on the effects of image properties on model performance.

Findings

High resolution improves classification accuracy.

Variable shape inputs pose challenges with current methods.

MAMe dataset differs significantly from ImageNet.

Abstract

In the image classification task, the most common approach is to resize all images in a dataset to a unique shape, while reducing their precision to a size which facilitates experimentation at scale. This practice has benefits from a computational perspective, but it entails negative side-effects on performance due to loss of information and image deformation. In this work we introduce the MAMe dataset, an image classification dataset with remarkable high resolution and variable shape properties. The goal of MAMe is to provide a tool for studying the impact of such properties in image classification, while motivating research in the field. The MAMe dataset contains thousands of artworks from three different museums, and proposes a classification task consisting on differentiating between 29 mediums (i.e. materials and techniques) supervised by art experts. After reviewing the singularity of MAMe in the context of current image classification tasks, a thorough description of the task is provided, together with dataset statistics. Experiments are conducted to evaluate the impact of using high resolution images, variable shape inputs and both properties at the same time. Results illustrate the positive impact in performance when using high resolution images, while highlighting the lack of solutions to exploit variable shapes. An additional experiment exposes the distinctiveness between the MAMe dataset and the prototypical ImageNet dataset. Finally, the baselines are inspected using explainability methods and expert knowledge, to gain insights on the challenges that remain ahead.