On Linear Separability of the MNIST Handwritten Digits Dataset

TL;DR

This paper empirically investigates whether the MNIST dataset of handwritten digits is linearly separable, analyzing pairwise and multi-class separation across training and test sets to clarify a long-standing question.

Contribution

It provides the first comprehensive empirical analysis of MNIST's linear separability, comparing theoretical and state-of-the-art methods systematically.

Findings

MNIST is not perfectly linearly separable in general.

Pairwise separability varies significantly among digit pairs.

The dataset's linear separability depends on the classification approach.

Abstract

The MNIST dataset containing thousands of handwritten digit images is still a fundamental benchmark for evaluating various pattern-recognition and image-classification models. Linear separability is a key concept in many statistical and machine-learning techniques. Despite the long history of the MNIST dataset and its relative simplicity in size and resolution, the question of whether the dataset is linearly separable has never been fully answered -- scientific and informal sources share conflicting claims. This paper aims to provide a comprehensive empirical investigation to address this question, distinguishing pairwise and one-vs-rest separation of the training, the test and the combined sets, respectively. It reviews the theoretical approaches to assessing linear separability, alongside state-of-the-art methods and tools, then systematically examines all relevant assemblies, and reports the findings.