Fast online inference for nonlinear contextual bandit based on Generative Adversarial Network

TL;DR

This paper introduces a neural bandit model utilizing generative adversarial networks to achieve fast, approximate inference in nonlinear contextual bandits with large arm sets, significantly reducing inference time.

Contribution

It proposes a novel generative adversarial network approach to shift the computational bottleneck from inference to training, enabling efficient batch and parallel processing for large-scale bandit problems.

Findings

Achieves $O(\log n)$ inference time complexity.

Demonstrates order-of-magnitude speedup in experiments.

Maintains comparable performance to existing methods.

Abstract

This work addresses the efficiency concern on inferring a nonlinear contextual bandit when the number of arms $n$ is very large. We propose a neural bandit model with an end-to-end training process to efficiently perform bandit algorithms such as Thompson Sampling and UCB during inference. We advance state-of-the-art time complexity to $O(\log n)$ with approximate Bayesian inference, neural random feature mapping, approximate global maxima and approximate nearest neighbor search. We further propose a generative adversarial network to shift the bottleneck of maximizing the objective for selecting optimal arms from inference time to training time, enjoying significant speedup with additional advantage of enabling batch and parallel processing. %The generative model can inference an approximate argmax of the posterior sampling in logarithmic time complexity with the help of approximate nearest neighbor search. Extensive experiments on classification and recommendation tasks demonstrate order-of-magnitude improvement in inference time no significant degradation on the performance.