Sequential Decision Making on Unmatched Data using Bayesian Kernel Embeddings

TL;DR

This paper introduces a novel Bayesian kernel embedding-based algorithm for sequential decision making with unmatched data, accounting for uncertainty in feature distribution and unknown functions, outperforming existing methods.

Contribution

The paper presents a new algorithm that models feature distributions as Bayesian embeddings and unknown functions as Gaussian processes for improved decision making.

Findings

Empirically outperforms current state-of-the-art algorithms.

Effectively models uncertainty in feature distribution and function estimation.

Demonstrates robustness on experimental datasets.

Abstract

The problem of sequentially maximizing the expectation of a function seeks to maximize the expected value of a function of interest without having direct control on its features. Instead, the distribution of such features depends on a given context and an action taken by an agent. In contrast to Bayesian optimization, the arguments of the function are not under agent's control, but are indirectly determined by the agent's action based on a given context. If the information of the features is to be included in the maximization problem, the full conditional distribution of such features, rather than its expectation only, needs to be accounted for. Furthermore, the function is itself unknown, only counting with noisy observations of such function, and potentially requiring the use of unmatched data sets. We propose a novel algorithm for the aforementioned problem which takes into consideration the uncertainty derived from the estimation of both the conditional distribution of the features and the unknown function, by modeling the former as a Bayesian conditional mean embedding and the latter as a Gaussian process. Our algorithm empirically outperforms the current state-of-the-art algorithm in the experiments conducted.