Provably and Practically Efficient Neural Contextual Bandits

TL;DR

This paper advances neural contextual bandit algorithms by analyzing smooth activation functions, deriving error bounds, and proposing an efficient algorithm with sublinear regret, supported by empirical validation.

Contribution

It extends analysis beyond ReLU networks to smooth activations, providing non-asymptotic error bounds and a provably efficient algorithm for neural contextual bandits.

Findings

Derived non-asymptotic error bounds relating neural nets and neural tangent kernels.

Proposed an algorithm with sublinear regret bound.

Empirical studies demonstrate finite regime efficiency.

Abstract

We consider the neural contextual bandit problem. In contrast to the existing work which primarily focuses on ReLU neural nets, we consider a general set of smooth activation functions. Under this more general setting, (i) we derive non-asymptotic error bounds on the difference between an overparameterized neural net and its corresponding neural tangent kernel, (ii) we propose an algorithm with a provably sublinear regret bound that is also efficient in the finite regime as demonstrated by empirical studies. The non-asymptotic error bounds may be of broader interest as a tool to establish the relation between the smoothness of the activation functions in neural contextual bandits and the smoothness of the kernels in kernel bandits.