instancespace: a Python Package for Insightful Algorithm Testing through Instance Space Analysis

TL;DR

The paper introduces instancespace, a Python package that automates Instance Space Analysis to evaluate algorithm performance across diverse problem instances, aiding in algorithm selection and design.

Contribution

It presents a new Python package that streamlines and automates Instance Space Analysis, providing objective insights into algorithm performance and diversity of test instances.

Findings

Supports automated algorithm selection

Enhances testing reliability in optimization and machine learning

Facilitates informed algorithm design and deployment

Abstract

Instance Space Analysis is a methodology to evaluate algorithm performance across diverse problem fields. Through visualisation and exploratory data analysis techniques, Instance Space Analysis offers objective, data-driven insights into the diversity of test instances, algorithm behaviour, and algorithm strengths and weaknesses. As such, it supports automated algorithm selection and synthetic test instance generation, increasing testing reliability in optimisation, machine learning, and scheduling fields. This paper introduces instancespace, a Python package that implements an automated pipeline for Instance Space Analysis. This package supports research by streamlining the testing process, providing unbiased metrics, and facilitating more informed algorithmic design and deployment decisions, particularly for complex and safety-critical systems.