SIPOMDPLite-Net: Lightweight, Self-Interested Learning and Planning in POSGs with Sparse Interactions

TL;DR

sIPOMDPLite-net is a lightweight neural network that enables decentralized, self-interested agents to learn and plan in multiagent environments with sparse interactions, demonstrating good transferability and near-optimal performance.

Contribution

The paper introduces sIPOMDPLite-net, a novel neural network architecture that models self-interested agent planning in POSGs using hierarchical value iteration, with effective transfer to larger and real-world scenarios.

Findings

Accurately learns I-POMDP Lite models from expert demonstrations.

Performs well on larger grids and real-world maps.

Offers a lighter alternative for multiagent planning.

Abstract

This work introduces sIPOMDPLite-net, a deep neural network (DNN) architecture for decentralized, self-interested agent control in partially observable stochastic games (POSGs) with sparse interactions between agents. The network learns to plan in contexts modeled by the interactive partially observable Markov decision process (I-POMDP) Lite framework and uses hierarchical value iteration networks to simulate the solution of nested MDPs, which I-POMDP Lite attributes to the other agent to model its behavior and predict its intention. We train sIPOMDPLite-net with expert demonstrations on small two-agent Tiger-grid tasks, for which it accurately learns the underlying I-POMDP Lite model and near-optimal policy, and the policy continues to perform well on larger grids and real-world maps. As such, sIPOMDPLite-net shows good transfer capabilities and offers a lighter learning and planning approach for individual, self-interested agents in multiagent settings.