Fast gradient descent for drifting least squares regression, with application to bandits

TL;DR

This paper introduces an efficient stochastic gradient descent approach for drifting least squares regression, significantly reducing computational complexity while maintaining accurate tracking and performance in big data and bandit applications.

Contribution

It demonstrates that SGD schemes can effectively track drifting regression solutions with $O(d)$ complexity improvements, providing theoretical error bounds and practical benefits in bandit algorithms.

Findings

SGD tracks drifting solutions efficiently in high dimensions.

Regret of SGD-based bandit algorithm is within $O( ext{log}^4 n)$ of the original.

Empirical results show substantial computational gains with minimal performance loss.

Abstract

Online learning algorithms require to often recompute least squares regression estimates of parameters. We study improving the computational complexity of such algorithms by using stochastic gradient descent (SGD) type schemes in place of classic regression solvers. We show that SGD schemes efficiently track the true solutions of the regression problems, even in the presence of a drift. This finding coupled with an $O(d)$ improvement in complexity, where $d$ is the dimension of the data, make them attractive for implementation in the big data settings. In the case when strong convexity in the regression problem is guaranteed, we provide bounds on the error both in expectation and high probability (the latter is often needed to provide theoretical guarantees for higher level algorithms), despite the drifting least squares solution. As an example of this case we prove that the regret performance of an SGD version of the PEGE linear bandit algorithm [Rusmevichientong and Tsitsiklis 2010] is worse that that of PEGE itself only by a factor of $O(\log^4 n)$. When strong convexity of the regression problem cannot be guaranteed, we investigate using an adaptive regularisation. We make an empirical study of an adaptively regularised, SGD version of LinUCB [Li et al. 2010] in a news article recommendation application, which uses the large scale news recommendation dataset from Yahoo! front page. These experiments show a large gain in computational complexity, with a consistently low tracking error and click-through-rate (CTR) performance that is $75\%$ close.