Push it to the Limit: Discover Edge-Cases in Image Data with Autoencoders

TL;DR

This paper presents a method using autoencoders and PCA to identify semantic factors and edge-cases in large image datasets by analyzing high-level encodings and their principal components.

Contribution

It introduces a novel approach combining autoencoders and PCA to discover semantic variations and edge-cases in image data.

Findings

High and low ends of principal component distributions correspond to specific semantic groups.

The method effectively uncovers unwanted edge-cases in real-world datasets.

Autoencoders facilitate high-level feature extraction for semantic analysis.

Abstract

In this paper, we focus on the problem of identifying semantic factors of variation in large image datasets. By training a convolutional Autoencoder on the image data, we create encodings, which describe each datapoint at a higher level of abstraction than pixel-space. We then apply Principal Component Analysis to the encodings to disentangle the factors of variation in the data. Sorting the dataset according to the values of individual principal components, we find that samples at the high and low ends of the distribution often share specific semantic characteristics. We refer to these groups of samples as semantic groups. When applied to real-world data, this method can help discover unwanted edge-cases.