A New General Method to Generate Random Modal Formulae for Testing Decision Procedures

TL;DR

This paper introduces a novel random generation method for modal formulae that enhances empirical testing of decision procedures by producing diverse and comprehensive test sets, improving over previous methods.

Contribution

The paper presents a generalized random modal formula generator that covers a wider input space and yields more effective test sets for modal decision procedures.

Findings

The new method produces more diverse test sets.

It outperforms previous generators in coverage and effectiveness.

Empirical tests confirm its benefits for decision procedure evaluation.

Abstract

The recent emergence of heavily-optimized modal decision procedures has highlighted the key role of empirical testing in this domain. Unfortunately, the introduction of extensive empirical tests for modal logics is recent, and so far none of the proposed test generators is very satisfactory. To cope with this fact, we present a new random generation method that provides benefits over previous methods for generating empirical tests. It fixes and much generalizes one of the best-known methods, the random CNF_[]m test, allowing for generating a much wider variety of problems, covering in principle the whole input space. Our new method produces much more suitable test sets for the current generation of modal decision procedures. We analyze the features of the new method by means of an extensive collection of empirical tests.