Explainable Goal-Driven Agents and Robots -- A Comprehensive Review

TL;DR

This paper reviews approaches to making goal-driven autonomous agents and robots more explainable, emphasizing transparency, communication of perceptual and cognitive functions, and outlining future requirements for effective explainability.

Contribution

It provides a comprehensive review of explainability techniques specific to goal-driven agents and robots, highlighting strategies for transparency and proposing a roadmap for future development.

Findings

Focus on explaining perceptual functions like senses and vision

Emphasize explaining cognitive reasoning such as beliefs and goals

Identify key strategies for transparency and continual learning

Abstract

Recent applications of autonomous agents and robots, such as self-driving cars, scenario-based trainers, exploration robots, and service robots have brought attention to crucial trust-related challenges associated with the current generation of artificial intelligence (AI) systems. AI systems based on the connectionist deep learning neural network approach lack capabilities of explaining their decisions and actions to others, despite their great successes. Without symbolic interpretation capabilities, they are black boxes, which renders their decisions or actions opaque, making it difficult to trust them in safety-critical applications. The recent stance on the explainability of AI systems has witnessed several approaches on eXplainable Artificial Intelligence (XAI); however, most of the studies have focused on data-driven XAI systems applied in computational sciences. Studies addressing the increasingly pervasive goal-driven agents and robots are still missing. This paper reviews approaches on explainable goal-driven intelligent agents and robots, focusing on techniques for explaining and communicating agents perceptual functions (example, senses, and vision) and cognitive reasoning (example, beliefs, desires, intention, plans, and goals) with humans in the loop. The review highlights key strategies that emphasize transparency, understandability, and continual learning for explainability. Finally, the paper presents requirements for explainability and suggests a roadmap for the possible realization of effective goal-driven explainable agents and robots.