BINGO! Simple Optimizers Win Big if Problems Collapse to a Few Buckets

TL;DR

This paper uncovers the BINGO effect, where software engineering data collapses into few solution buckets, enabling simple optimizers to achieve near state-of-the-art results much faster than complex methods.

Contribution

It introduces the BINGO effect, demonstrates its prevalence, and develops simple algorithms that outperform complex optimizers in speed while maintaining effectiveness.

Findings

BINGO effect is common across 39 SE optimization problems.

Simple stochastic algorithms match complex optimizers like DEHB.

Optimization speed improves by up to 10,000 times with comparable results.

Abstract

Traditional multi-objective optimization in software engineering (SE) can be slow and complex. This paper introduces the BINGO effect: a novel phenomenon where SE data surprisingly collapses into a tiny fraction of possible solution "buckets" (e.g., only 100 used from 4,096 expected). We show the BINGO effect's prevalence across 39 optimization in SE problems. Exploiting this, we optimize 10,000 times faster than state-of-the-art methods, with comparable effectiveness. Our new algorithms (LITE and LINE), demonstrate that simple stochastic selection can match complex optimizers like DEHB. This work explains why simple methods succeed in SE-real data occupies a small corner of possibilities-and guides when to apply them, challenging the need for CPU-heavy optimization. Our data and code are public at GitHub (see anon-artifacts/bingo).