On the Linear Belief Compression of POMDPs: A re-examination of current methods

TL;DR

This paper critically re-examines existing linear belief compression methods for POMDPs, identifies their deficiencies, and proposes a new projective NMF approach that is empirically tested on large-scale problems.

Contribution

It provides a unified theoretical analysis of current methods, introduces a novel projective NMF belief compression technique, and empirically compares its performance with existing approaches.

Findings

Existing belief compression methods have critical deficiencies.

The proposed P-NMF method overcomes these drawbacks.

P-NMF performs competitively on large-scale POMDPs.

Abstract

Belief compression improves the tractability of large-scale partially observable Markov decision processes (POMDPs) by finding projections from high-dimensional belief space onto low-dimensional approximations, where solving to obtain action selection policies requires fewer computations. This paper develops a unified theoretical framework to analyse three existing linear belief compression approaches, including value-directed compression and two non-negative matrix factorisation (NMF) based algorithms. The results indicate that all the three known belief compression methods have their own critical deficiencies. Therefore, projective NMF belief compression is proposed (P-NMF), aiming to overcome the drawbacks of the existing techniques. The performance of the proposed algorithm is examined on four POMDP problems of reasonably large scale, in comparison with existing techniques. Additionally, the competitiveness of belief compression is compared empirically to a state-of-the-art heuristic search based POMDP solver and their relative merits in solving large-scale POMDPs are investigated.