# Partially Observable Planning and Learning for Systems with Non-Uniform Dynamics

## TL;DR

This paper introduces TransNet, a neural network architecture that combines planning and model learning to handle POMDPs with non-uniform dynamics, improving policy quality and learning efficiency in complex robotic navigation tasks.

## Contribution

TransNet relaxes the uniform dynamics restriction in neural POMDP models by classifying state space regions and learning their specific dynamics, enhancing expressiveness and efficiency.

## Key findings

- TransNet outperforms QMDP-Net in policy quality.
- TransNet demonstrates improved learning efficiency.
- Effective in robot navigation with unknown models.

## Abstract

We propose a neural network architecture, called TransNet, that combines planning and model learning for solving Partially Observable Markov Decision Processes (POMDPs) with non-uniform system dynamics. The past decade has seen a substantial advancement in solving POMDP problems. However, constructing a suitable POMDP model remains difficult. Recently, neural network architectures have been proposed to alleviate the difficulty in acquiring such models. Although the results are promising, existing architectures restrict the type of system dynamics that can be learned --that is, system dynamics must be the same in all parts of the state space. TransNet relaxes such a restriction. Key to this relaxation is a novel neural network module that classifies the state space into classes and then learns the system dynamics of the different classes. TransNet uses this module together with the overall architecture of QMDP-Net[1] to allow solving POMDPs that have more expressive dynamic models, while maintaining efficient data requirement. Its evaluation on typical benchmarks in robot navigation with initially unknown system and environment models indicates that TransNet substantially out-performs the quality of the generated policies and learning efficiency of the state-of-the-art method QMDP-Net.