# Reusable Test Suites for Reinforcement Learning

## TL;DR

This paper introduces MPTCS, an automated method for selecting reusable, policy-agnostic test suites in reinforcement learning, enhancing testing efficiency and coverage across diverse RL policies.

## Contribution

It proposes a novel multi-policy test case selection approach that generates diverse, reusable test suites applicable to various RL policies, improving testing robustness.

## Key findings

- MPTCS effectively selects diverse test cases based on solvability and difficulty.
- The method's effectiveness depends on the number of policies used in selection.
- Diversity promotion improves coverage and fault detection in RL testing.

## Abstract

Reinforcement learning (RL) agents show great promise in solving sequential decision-making tasks. However, validating the reliability and performance of the agent policies' behavior for deployment remains challenging. Most reinforcement learning policy testing methods produce test suites tailored to the agent policy being tested, and their relevance to other policies is unclear. This work presents Multi-Policy Test Case Selection (MPTCS), a novel automated test suite selection method for RL environments, designed to extract test cases generated by any policy testing framework based on their solvability, diversity, and general difficulty. MPTCS uses a set of policies to select a diverse collection of reusable policy-agnostic test cases that reveal typical flaws in the agents' behavior. The set of policies selects test cases from a candidate pool, which can be generated by any policy testing method, based on a difficulty score. We assess the effectiveness of the difficulty score and how the method's effectiveness and cost depend on the number of policies in the set. Additionally, a method for promoting diversity in the test suite, a discretized general test case descriptor surface inspired by quality-diversity algorithms, is examined to determine how it covers the state space and which policies it triggers to produce faulty behaviors.