Evolving Image Compositions for Feature Representation Learning

TL;DR

This paper introduces PatchMix, a novel data augmentation technique for CNNs that combines image patches in grid patterns, enhanced by evolutionary search for optimal mixing, leading to improved transfer learning across multiple benchmarks.

Contribution

PatchMix is a new data augmentation method that uses patch composition and evolutionary search to optimize image mixing for better feature learning.

Findings

PatchMix improves transfer learning performance on ImageNet and other datasets.

Evolutionary search effectively discovers optimal patch mixing patterns.

PatchMix outperforms baseline models on multiple image classification benchmarks.

Abstract

Convolutional neural networks for visual recognition require large amounts of training samples and usually benefit from data augmentation. This paper proposes PatchMix, a data augmentation method that creates new samples by composing patches from pairs of images in a grid-like pattern. These new samples are assigned label scores that are proportional to the number of patches borrowed from each image. We then add a set of additional losses at the patch-level to regularize and to encourage good representations at both the patch and image levels. A ResNet-50 model trained on ImageNet using PatchMix exhibits superior transfer learning capabilities across a wide array of benchmarks. Although PatchMix can rely on random pairings and random grid-like patterns for mixing, we explore evolutionary search as a guiding strategy to jointly discover optimal grid-like patterns and image pairings. For this purpose, we conceive a fitness function that bypasses the need to re-train a model to evaluate each possible choice. In this way, PatchMix outperforms a base model on CIFAR-10 (+1.91), CIFAR-100 (+5.31), Tiny Imagenet (+3.52), and ImageNet (+1.16).