Ising-based Test Optimization and Benchmarking

TL;DR

This paper introduces IsingTester, a Python tool that formulates test optimization as Ising models, leveraging quantum-inspired solutions like CIM for improved test case selection and minimization.

Contribution

It presents a novel end-to-end pipeline for test optimization using Ising models and introduces IsingBench for benchmarking these approaches against traditional methods.

Findings

IsingTester automates test optimization with Ising model encoding.

The tool supports CIM simulation and brute-force solvers.

Benchmarking compares Ising-based methods with baseline approaches.

Abstract

Test optimization contains test case selection and minimization, which is an important challenge in software testing and has been addressed with search-based approaches intensively in the past. Inspired by the recent advancement of using quantum optimization solutions for addressing test optimization problems, we looked into Coherent Ising Machines (CIM), which offer potential for solving combinatorial optimization problems, but have not yet been exploited in test optimization. Hence, in this paper, we present IsingTester, an open-source, Python-based command-line tool that provides an end-to-end pipeline for solving test optimization problems that are formulated as Ising models. With IsingTester, we reformulate test selection and minimization as Ising spin configurations, encode multiple optimization strategies into Ising Hamiltonians, and implement solvers including CIM simulation and brute-force search. Given a user-provided dataset and solver configuration, IsingTester automatically performs problem encoding, optimization, and spin decoding, returning selected test cases back to the user. Along with IsingTester, we also present the accompanying IsingBench for evaluating and comparing optimization techniques across Ising-based paradigms against baseline approaches. A screencast demonstrating the tool is available at: https://github.com/WSE-Lab/IsingBench.