Epoch-evolving Gaussian Process Guided Learning

TL;DR

This paper introduces epoch-evolving Gaussian Process Guided Learning (GPGL), a novel method that leverages correlation between batch and global data distributions to improve deep model training efficiency and performance.

Contribution

The paper proposes a new epoch-evolving GPGL scheme that encodes correlation information as context labels and enhances deep learning optimization.

Findings

Outperforms state-of-the-art batch-based models on CIFAR-10, CIFAR-100, and Tiny-ImageNet.

Provides a more efficient optimization method for deep models.

Successfully generalizes to current deep learning architectures.

Abstract

In this paper, we propose a novel learning scheme called epoch-evolving Gaussian Process Guided Learning (GPGL), which aims at characterizing the correlation information between the batch-level distribution and the global data distribution. Such correlation information is encoded as context labels and needs renewal every epoch. With the guidance of the context label and ground truth label, GPGL scheme provides a more efficient optimization through updating the model parameters with a triangle consistency loss. Furthermore, our GPGL scheme can be further generalized and naturally applied to the current deep models, outperforming the existing batch-based state-of-the-art models on mainstream datasets (CIFAR-10, CIFAR-100, and Tiny-ImageNet) remarkably.