A User Study on Contrastive Explanations for Multi-Effector Temporal Planning with Non-Stationary Costs

TL;DR

This study explores how contrastive explanations in a custom multi-effector temporal planning system for smart homes improve user satisfaction, understanding, and perceived helpfulness, especially in complex, non-stationary cost scenarios.

Contribution

It introduces a domain-specific planner for multi-effector temporal planning with non-stationary costs and demonstrates that contrastive explanations enhance user experience in this context.

Findings

Users with contrastive explanations show higher satisfaction.

Contrastive explanations improve user understanding.

Participants rate AI schedules more helpful with explanations.

Abstract

In this paper, we adopt constrastive explanations within an end-user application for temporal planning of smart homes. In this application, users have requirements on the execution of appliance tasks, pay for energy according to dynamic energy tariffs, have access to high-capacity battery storage, and are able to sell energy to the grid. The concurrent scheduling of devices makes this a multi-effector planning problem, while the dynamic tariffs yield costs that are non-stationary (alternatively, costs that are stationary but depend on exogenous events). These characteristics are such that the planning problems are generally not supported by existing PDDL-based planners, so we instead design a custom domain-dependent planner that scales to reasonable appliance numbers and time horizons. We conduct a controlled user study with 128 participants using an online crowd-sourcing platform based on two user stories. Our results indicate that users provided with contrastive questions and explanations have higher levels of satisfaction, tend to gain improved understanding, and rate the helpfulness more favourably with the recommended AI schedule compared to those without access to these features.