Adversarial Deep Learning for Online Resource Allocation

TL;DR

This paper introduces a novel deep learning approach inspired by adversarial training to design online algorithms with minimized worst-case performance gaps, demonstrated through resource allocation problems.

Contribution

It is the first to use deep neural networks to develop online algorithms focusing on worst-case guarantees, employing a new per-round update method for better convergence.

Findings

The learned algorithm outperforms existing online algorithms in various settings.

The proposed method ensures convergence to Nash equilibrium.

Empirical results validate the effectiveness of the approach.

Abstract

Online algorithm is an important branch in algorithm design. Designing online algorithms with a bounded competitive ratio (in terms of worst-case performance) can be hard and usually relies on problem-specific assumptions. Inspired by adversarial training from Generative Adversarial Net (GAN) and the fact that competitive ratio of an online algorithm is based on worst-case input, we adopt deep neural networks to learn an online algorithm for a resource allocation and pricing problem from scratch, with the goal that the performance gap between offline optimum and the learned online algorithm can be minimized for worst-case input. Specifically, we leverage two neural networks as algorithm and adversary respectively and let them play a zero sum game, with the adversary being responsible for generating worst-case input while the algorithm learns the best strategy based on the input provided by the adversary. To ensure better convergence of the algorithm network (to the desired online algorithm), we propose a novel per-round update method to handle sequential decision making to break complex dependency among different rounds so that update can be done for every possible action, instead of only sampled actions. To the best of our knowledge, our work is the first using deep neural networks to design an online algorithm from the perspective of worst-case performance guarantee. Empirical studies show that our updating methods ensure convergence to Nash equilibrium and the learned algorithm outperforms state-of-the-art online algorithms under various settings.