A Zeroth-Order Block Coordinate Descent Algorithm for Huge-Scale Black-Box Optimization

TL;DR

This paper introduces ZO-BCD, a zeroth-order block coordinate descent algorithm designed for huge-scale black-box optimization problems with high dimensions, offering improved query complexity and reduced computational and memory costs.

Contribution

The paper presents ZO-BCD, a novel zeroth-order optimization algorithm optimized for large-scale problems, with innovative memory reduction techniques and practical applications in adversarial attacks.

Findings

Achieves state-of-the-art 97.9% attack success rate on audio classifiers.

Reduces computational complexity per iteration significantly.

Enables optimization in problems with over 1.7 million dimensions.

Abstract

We consider the zeroth-order optimization problem in the huge-scale setting, where the dimension of the problem is so large that performing even basic vector operations on the decision variables is infeasible. In this paper, we propose a novel algorithm, coined ZO-BCD, that exhibits favorable overall query complexity and has a much smaller per-iteration computational complexity. In addition, we discuss how the memory footprint of ZO-BCD can be reduced even further by the clever use of circulant measurement matrices. As an application of our new method, we propose the idea of crafting adversarial attacks on neural network based classifiers in a wavelet domain, which can result in problem dimensions of over 1.7 million. In particular, we show that crafting adversarial examples to audio classifiers in a wavelet domain can achieve the state-of-the-art attack success rate of 97.9%.