CSPLib: Twenty Years On

TL;DR

CSPLib, established in 1999 as a benchmark library for the constraints programming community, has evolved over twenty years, emphasizing natural language problem specifications to accommodate diverse representations and remain a valuable resource.

Contribution

This paper reviews the twenty-year history of CSPLib, highlighting its role in facilitating benchmarking and addressing representation challenges in constraints programming.

Findings

CSPLib has become a valuable benchmarking resource over twenty years.

Natural language specifications help accommodate diverse problem representations.

Community response has been positive and ongoing.

Abstract

In 1999, we introduced CSPLib, a benchmark library for the constraints community. Our CP-1999 poster paper about CSPLib discussed the advantages and disadvantages of building such a library. Unlike some other domains such as theorem proving, or machine learning, representation was then and remains today a major issue in the success or failure to solve problems. Benchmarks in CSPLib are therefore specified in natural language as this allows users to find good representations for themselves. The community responded positively and CSPLib has become a valuable resource but, as we discuss here, we cannot rest.