Effect of Slow Switching in On-line Learning for Ensemble Teachers

TL;DR

This paper analyzes how slow switching among ensemble teachers affects a student's generalization performance in online learning, revealing periodic behaviors and the influence of switching period and number of teachers.

Contribution

It provides an analytical study of the dynamic effects of slow switching in ensemble teachers on student generalization in online learning.

Findings

Generalization error exhibits periodicity synchronized with switching period

Behavior varies with the number of ensemble teachers

Shorter switching periods lead to larger differences in performance

Abstract

We have analyzed the generalization performance of a student which slowly switches ensemble teachers. By calculating the generalization error analytically using statistical mechanics in the framework of on-line learning, we show that the dynamical behaviors of generalization error have the periodicity that is synchronized with the switching period and the behaviors differ with the number of ensemble teachers. Furthermore, we show that the smaller the switching period is, the larger the difference is.