Scaling POMDPs For Selecting Sellers in E-markets-Extended Version

TL;DR

This paper introduces MOPE, a scalable POMDP-based method for selecting sellers in large e-markets, effectively handling hundreds of agents by aggregating solutions of smaller sub-problems, thus improving decision quality.

Contribution

The paper presents MOPE, a novel approach that exploits domain structure to scale POMDP solutions for seller selection in e-markets, enabling handling of large agent populations.

Findings

MOPE scales to over 100 agents.

Significant improvement in buyer satisfaction.

Theoretical and empirical validation of MOPE's effectiveness.

Abstract

In multiagent e-marketplaces, buying agents need to select good sellers by querying other buyers (called advisors). Partially Observable Markov Decision Processes (POMDPs) have shown to be an effective framework for optimally selecting sellers by selectively querying advisors. However, current solution methods do not scale to hundreds or even tens of agents operating in the e-market. In this paper, we propose the Mixture of POMDP Experts (MOPE) technique, which exploits the inherent structure of trust-based domains, such as the seller selection problem in e-markets, by aggregating the solutions of smaller sub-POMDPs. We propose a number of variants of the MOPE approach that we analyze theoretically and empirically. Experiments show that MOPE can scale up to a hundred agents thereby leveraging the presence of more advisors to significantly improve buyer satisfaction.