DTN: A Learning Rate Scheme with Convergence Rate of $\mathcal{O}(1/t)$ for SGD

TL;DR

This paper discusses a learning rate scheme for SGD claiming an $(1/t)$ convergence rate, but later acknowledges errors in the proofs and invalidates the main convergence claims due to incorrect use of the test criterion.

Contribution

Proposes a learning rate scheme for SGD with a claimed convergence rate, but the main theoretical results are invalidated due to proof errors.

Findings

Original convergence claims are invalidated.

The paper admits mistakes in the proof of key lemmas.

No validated convergence rate is established due to errors.

Abstract

This paper has some inconsistent results, i.e., we made some failed claims because we did some mistakes for using the test criterion for a series. Precisely, our claims on the convergence rate of $\mathcal{O}(1/t)$ of SGD presented in Theorem 1, Corollary 1, Theorem 2 and Corollary 2 are wrongly derived because they are based on Lemma 5. In Lemma 5, we do not correctly use the test criterion for a series. Hence, the result of Lemma 5 is not valid. We would like to thank the community for pointing out this mistake!