Evolving parametrized Loss for Image Classification Learning on Small Datasets

TL;DR

This paper introduces a meta-learning method that evolves a parametrized loss function, called Meta-Loss Network (MLN), to improve image classification on small datasets by optimizing the loss for better generalization.

Contribution

The paper presents a novel meta-learning approach that evolves a differentiable loss function using evolutionary strategies for small dataset image classification.

Findings

MLN outperforms classical loss functions like cross-entropy and MSE.

The approach improves generalization on FashionMNIST and CIFAR10.

Meta-Loss Network effectively adapts to small sample learning tasks.

Abstract

This paper proposes a meta-learning approach to evolving a parametrized loss function, which is called Meta-Loss Network (MLN), for training the image classification learning on small datasets. In our approach, the MLN is embedded in the framework of classification learning as a differentiable objective function. The MLN is evolved with the Evolutionary Strategy algorithm (ES) to an optimized loss function, such that a classifier, which optimized to minimize this loss, will achieve a good generalization effect. A classifier learns on a small training dataset to minimize MLN with Stochastic Gradient Descent (SGD), and then the MLN is evolved with the precision of the small-dataset-updated classifier on a large validation dataset. In order to evaluate our approach, the MLN is trained with a large number of small sample learning tasks sampled from FashionMNIST and tested on validation tasks sampled from FashionMNIST and CIFAR10. Experiment results demonstrate that the MLN effectively improved generalization compared to classical cross-entropy error and mean squared error.