Parameter Estimation using Reinforcement Learning Causal Curiosity: Limits and Challenges

TL;DR

This paper critically analyzes the Causal Curiosity reinforcement learning approach for parameter estimation, highlighting its current limitations in measurement accuracy and sensitivity, and proposing improvements for real-world applications.

Contribution

It provides the first measurement accuracy analysis of Causal Curiosity and discusses its limitations and potential enhancements for complex real-world scenarios.

Findings

Measurement accuracy is crucial for effectiveness.

Current limitations include sensitivity and confounding factors.

Proposed design improvements aim to enhance real-world applicability.

Abstract

Causal understanding is important in many disciplines of science and engineering, where we seek to understand how different factors in the system causally affect an experiment or situation and pave a pathway towards creating effective or optimising existing models. Examples of use cases are autonomous exploration and modelling of unknown environments or assessing key variables in optimising large complex systems. In this paper, we analyse a Reinforcement Learning approach called Causal Curiosity, which aims to estimate as accurately and efficiently as possible, without directly measuring them, the value of factors that causally determine the dynamics of a system. Whilst the idea presents a pathway forward, measurement accuracy is the foundation of methodology effectiveness. Focusing on the current causal curiosity's robotic manipulator, we present for the first time a measurement accuracy analysis of the future potentials and current limitations of this technique and an analysis of its sensitivity and confounding factor disentanglement capability - crucial for causal analysis. As a result of our work, we promote proposals for an improved and efficient design of Causal Curiosity methods to be applied to real-world complex scenarios.