Yet another zeta function and learning

TL;DR

This paper analyzes the convergence speeds of different learning algorithms, introduces the moment zeta function for probabilistic analysis, and compares their efficiencies under various assumptions.

Contribution

It introduces the moment zeta function to study learning algorithm convergence and provides precise comparisons of their asymptotic performance.

Findings

Batch learning is asymptotically never worse than memoryless learning.

Performance comparison with full-memory learning depends on probabilistic assumptions.

Introduces the moment zeta function for probabilistic analysis of learning algorithms.

Abstract

We study the convergence speed of the batch learning algorithm, and compare its speed to that of the memoryless learning algorithm and of learning with memory (as analyzed in joint work with N. Komarova). We obtain precise results and show in particular that the batch learning algorithm is never worse than the memoryless learning algorithm (at least asymptotically). Its performance vis-a-vis learning with full memory is less clearcut, and depends on certainprobabilistic assumptions. These results necessitate theintroduction of the moment zeta function of a probability distribution and the study of some of its properties.