Tight Lower Bounds for the Bit and Inner Product Oracle for Constrained Convex Optimization

TL;DR

This paper establishes tight lower bounds on the information complexity for constrained convex optimization using bit-wise oracles, showing that the quadratic dependence on the number of variables is fundamental.

Contribution

It introduces new lower bounds for the information complexity under bit-wise oracles, demonstrating the quadratic dependence on the number of variables, which was previously unknown.

Findings

Lower bounds are tight up to a logarithmic factor.

Complexity is quadratic in the number of continuous variables.

Standard bit-representation of first-order info is optimal.

Abstract

We establish new lower-bounds for the information complexity of mixed-integer convex optimization under two "bit-wise" oracles. The first oracle provides bits of first-order information in the standard coordinate model, and the second oracle answers whether the inner product of a specified vector with the gradient of the function at a point or the normal vector of a separating hyperplane for the feasible region is positive or non-positive, thus also providing one bit of first-order information. The new contribution is that under such oracles, the complexity is quadratic in the number of continuous decision variables, which was not known before even for continuous convex optimization. These new lower-bounds are tight (up to a logarithmic term), matched by a natural discretization of standard cutting-plane methods for convex optimization. These reveal that using a standard bit-representation of the first-order information is, in general, the best one can do with respect to the number of bits of information needed to solve constrained convex optimization problems.