Block-Bench: A Framework for Controllable and Transparent Discrete Optimization Benchmarking

TL;DR

Block-Bench introduces a flexible framework for creating discrete optimization benchmarks with controllable properties, aiding in analyzing algorithm behaviors at multiple levels.

Contribution

The paper presents a novel benchmark construction method based on block functions, enabling detailed analysis and control over problem characteristics.

Findings

Framework allows analysis of algorithm behavior at variable and objective levels.

Enables explicit control over problem properties for diverse research domains.

Supports analysis of heuristics in large-scale multi-modal problems.

Abstract

We present a novel approach for constructing discrete optimization benchmarks that enables fine-grained control over problem properties, and such benchmarks can facilitate analyzing discrete algorithm behaviors. We build benchmark problems based on a set of block functions, where each block function maps a subset of variables to a real value. Problems are instantiated through a set of block functions, weight factors, and an adjacency graph representing the dependency among the block functions. Through analyzing intermediate block values, our framework allows to analyze algorithm behavior not only in the objective space but also at the level of variable representations in the obtained solutions. This capacity is particularly useful for analyzing discrete heuristics in large-scale multi-modal problems, thereby enhancing the practical relevance of benchmark studies. We demonstrate how the proposed approach can inspire the related work in self-adaptation and diversity control in evolutionary algorithms. Moreover, we explain that the proposed benchmark design enables explicit control over problem properties, supporting research in broader domains such as dynamic algorithm configuration and multi-objective optimization.