Adaptively Point-weighting Curriculum Learning

TL;DR

This paper introduces an adaptively point-weighting curriculum learning method that dynamically assigns weights to training samples based on their loss, improving training efficiency and effectiveness for deep networks.

Contribution

It proposes a novel adaptive weighting strategy guided by training loss, addressing limitations of existing curriculum learning methods that favor easy samples throughout training.

Findings

Theoretical analysis shows improved training stability and generalization.

Experimental results demonstrate superior performance over existing methods.

APW accelerates learning and enhances model accuracy.

Abstract

Curriculum learning (CL) mimics human learning, in which easy samples are learned first, followed by harder samples, and has become an effective method for training deep networks. However, many existing automatic CL methods maintain a preference for easy samples during the entire training process regardless of the constantly evolving training state. This is just like a human curriculum that fails to provide individualized instruction, which can delay learning progress. To address this issue, we propose an adaptively point-weighting (APW) curriculum learning method that assigns a weight to each training sample based on its training loss. The weighting strategy of APW follows the easy-to-hard training paradigm, guided by the current training state of the network. We present a theoretical analysis of APW, including training effectiveness, training stability, and generalization performance. Experimental results validate these theoretical findings and demonstrate the superiority of the proposed APW method.