Learning What Data to Learn

TL;DR

This paper introduces NDF, a deep reinforcement learning framework that adaptively selects training data to improve learning efficiency and achieve comparable accuracy with less data across various neural network models and tasks.

Contribution

The paper presents a novel, generic deep reinforcement learning approach for automatic data selection during training, outperforming heuristic methods and applicable to multiple neural network architectures.

Findings

NDF achieves similar accuracy with less data and fewer iterations.

NDF is effective across different neural network types and tasks.

The approach improves training efficiency in neural network learning.

Abstract

Machine learning is essentially the sciences of playing with data. An adaptive data selection strategy, enabling to dynamically choose different data at various training stages, can reach a more effective model in a more efficient way. In this paper, we propose a deep reinforcement learning framework, which we call \emph{\textbf{N}eural \textbf{D}ata \textbf{F}ilter} (\textbf{NDF}), to explore automatic and adaptive data selection in the training process. In particular, NDF takes advantage of a deep neural network to adaptively select and filter important data instances from a sequential stream of training data, such that the future accumulative reward (e.g., the convergence speed) is maximized. In contrast to previous studies in data selection that is mainly based on heuristic strategies, NDF is quite generic and thus can be widely suitable for many machine learning tasks. Taking neural network training with stochastic gradient descent (SGD) as an example, comprehensive experiments with respect to various neural network modeling (e.g., multi-layer perceptron networks, convolutional neural networks and recurrent neural networks) and several applications (e.g., image classification and text understanding) demonstrate that NDF powered SGD can achieve comparable accuracy with standard SGD process by using less data and fewer iterations.