A novel approach to model exploration for value function learning

TL;DR

This paper introduces a new method for exploring models to learn value functions that enhance planning efficiency and robustness by extending search regions and managing heuristic admissibility.

Contribution

It proposes a search-inspired model exploration technique for value function learning that extends beyond optimal paths and combines ML heuristics with admissible bounds.

Findings

Improved planning efficiency through extended search regions.

Enhanced robustness of planning with learned heuristics.

Effective management of heuristic admissibility loss.

Abstract

Planning and Learning are complementary approaches. Planning relies on deliberative reasoning about the current state and sequence of future reachable states to solve the problem. Learning, on the other hand, is focused on improving system performance based on experience or available data. Learning to improve the performance of planning based on experience in similar, previously solved problems, is ongoing research. One approach is to learn Value function (cost-to-go) which can be used as heuristics for speeding up search-based planning. Existing approaches in this direction use the results of the previous search for learning the heuristics. In this work, we present a search-inspired approach of systematic model exploration for the learning of the value function which does not stop when a plan is available but rather prolongs search such that not only resulting optimal path is used but also extended region around the optimal path. This, in turn, improves both the efficiency and robustness of successive planning. Additionally, the effect of losing admissibility by using ML heuristic is managed by bounding ML with other admissible heuristics.