Importance Weighting Approach in Kernel Bayes' Rule

TL;DR

This paper introduces an importance weighting approach to kernel Bayes' rule that is model-free, numerically stable, and demonstrates superior performance on complex synthetic benchmarks.

Contribution

It presents a novel importance weighting-based kernel Bayes' rule with convergence guarantees, improving stability and performance over previous methods.

Findings

Superior numerical stability compared to original KBR

Consistent estimator with convergence in the infinity norm

Outperforms original KBR and competes with other methods on benchmarks

Abstract

We study a nonparametric approach to Bayesian computation via feature means, where the expectation of prior features is updated to yield expected kernel posterior features, based on regression from learned neural net or kernel features of the observations. All quantities involved in the Bayesian update are learned from observed data, making the method entirely model-free. The resulting algorithm is a novel instance of a kernel Bayes' rule (KBR), based on importance weighting. This results in superior numerical stability to the original approach to KBR, which requires operator inversion. We show the convergence of the estimator using a novel consistency analysis on the importance weighting estimator in the infinity norm. We evaluate KBR on challenging synthetic benchmarks, including a filtering problem with a state-space model involving high dimensional image observations. Importance weighted KBR yields uniformly better empirical performance than the original KBR, and competitive performance with other competing methods.