Focused Model-Learning and Planning for Non-Gaussian Continuous State-Action Systems

TL;DR

This paper presents a focused learning and planning framework for complex stochastic systems with continuous, non-Gaussian dynamics, emphasizing efficiency and relevance in model estimation and decision-making.

Contribution

It introduces a novel approach that selectively learns local models and concentrates on relevant states and actions, with proven asymptotic optimality and empirical validation.

Findings

Efficient local model estimation reduces computational load.

Focus on relevant states improves planning accuracy.

Algorithm demonstrates effectiveness in simulated multi-modal pushing tasks.

Abstract

We introduce a framework for model learning and planning in stochastic domains with continuous state and action spaces and non-Gaussian transition models. It is efficient because (1) local models are estimated only when the planner requires them; (2) the planner focuses on the most relevant states to the current planning problem; and (3) the planner focuses on the most informative and/or high-value actions. Our theoretical analysis shows the validity and asymptotic optimality of the proposed approach. Empirically, we demonstrate the effectiveness of our algorithm on a simulated multi-modal pushing problem.