Towards Efficient Discrete Integration via Adaptive Quantile Queries

TL;DR

AdaWISH improves discrete integration efficiency by adaptively selecting fewer optimization queries while maintaining accuracy, significantly reducing computational costs compared to the original WISH algorithm.

Contribution

AdaWISH introduces an adaptive query strategy that achieves the same approximation guarantees as WISH with substantially fewer queries, enhancing scalability.

Findings

AdaWISH issues only O(log n) queries when function values are bounded.

It achieves the same approximation quality as WISH in experiments.

It reduces the number of queries by 81.5% on benchmark problems.

Abstract

Discrete integration in a high dimensional space of n variables poses fundamental challenges. The WISH algorithm reduces the intractable discrete integration problem into n optimization queries subject to randomized constraints, obtaining a constant approximation guarantee. The optimization queries are expensive, which limits the applicability of WISH. We propose AdaWISH, which is able to obtain the same guarantee but accesses only a small subset of queries of WISH. For example, when the number of function values is bounded by a constant, AdaWISH issues only O(log n) queries. The key idea is to query adaptively, taking advantage of the shape of the weight function being integrated. In general, we prove that AdaWISH has a regret of only O(log n) relative to an idealistic oracle that issues queries at data-dependent optimal points. Experimentally, AdaWISH gives precise estimates for discrete integration problems, of the same quality as that of WISH and better than several competing approaches, on a variety of probabilistic inference benchmarks. At the same time, it saves substantially on the number of optimization queries compared to WISH. On a suite of UAI inference challenge benchmarks, it saves 81.5% of WISH queries while retaining the quality of results.