On Convergence Property of Implicit Self-paced Objective

TL;DR

This paper proves that the self-paced learning (SPL) process converges to critical points of its implicit objective, providing a theoretical foundation for its robustness and effectiveness in machine learning tasks.

Contribution

It offers the first rigorous convergence analysis of SPL's implicit objective, confirming the intrinsic link and enhancing the theoretical understanding of SPL.

Findings

SPL always converges to critical points under mild conditions.

Theoretical verification of SPL's robustness and effectiveness.

Completes the theoretical analysis of SPL's implicit objective.

Abstract

Self-paced learning (SPL) is a new methodology that simulates the learning principle of humans/animals to start learning easier aspects of a learning task, and then gradually take more complex examples into training. This new-coming learning regime has been empirically substantiated to be effective in various computer vision and pattern recognition tasks. Recently, it has been proved that the SPL regime has a close relationship to a implicit self-paced objective function. While this implicit objective could provide helpful interpretations to the effectiveness, especially the robustness, insights under the SPL paradigms, there are still no theoretical results strictly proved to verify such relationship. To this issue, in this paper, we provide some convergence results on this implicit objective of SPL. Specifically, we prove that the learning process of SPL always converges to critical points of this implicit objective under some mild conditions. This result verifies the intrinsic relationship between SPL and this implicit objective, and makes the previous robustness analysis on SPL complete and theoretically rational.