Bounds on Query Convergence

TL;DR

This paper establishes fundamental bounds on the convergence rate and regret for optimization algorithms using noisy feedback, demonstrating that the performance is inherently limited by these bounds.

Contribution

The paper derives tight asymptotic bounds on convergence and regret for noisy quadratic optimization, and presents an algorithm that achieves these bounds.

Findings

Convergence rate bound: E[(x_t - x*)^2] >= O(1/√t)

Regret bound: E[sum_{i=1}^t (x_i - x*)^2] >= O(√t)

Practical query complexity: O(ε^-4) queries for ε-accuracy

Abstract

The problem of finding an optimum using noisy evaluations of a smooth cost function arises in many contexts, including economics, business, medicine, experiment design, and foraging theory. We derive an asymptotic bound E[ (x_t - x*)^2 ] >= O(1/sqrt(t)) on the rate of convergence of a sequence (x_0, x_1, >...) generated by an unbiased feedback process observing noisy evaluations of an unknown quadratic function maximised at x*. The bound is tight, as the proof leads to a simple algorithm which meets it. We further establish a bound on the total regret, E[ sum_{i=1..t} (x_i - x*)^2 ] >= O(sqrt(t)) These bounds may impose practical limitations on an agent's performance, as O(eps^-4) queries are made before the queries converge to x* with eps accuracy.