Bayesian Convolutional Deep Sets with Task-Dependent Stationary Prior

TL;DR

This paper introduces Bayesian convolutional deep sets with task-dependent stationary priors to improve the modeling of stationary stochastic processes in neural networks, addressing ambiguity issues in kernel smoothing.

Contribution

It proposes a Bayesian extension to convolutional deep sets that incorporates task-specific priors, enhancing representation quality over traditional kernel smoother-based methods.

Findings

Improved representation quality in time-series and image datasets.

Effective incorporation of task-dependent priors.

Enhanced modeling of stationary stochastic processes.

Abstract

Convolutional deep sets are the architecture of a deep neural network (DNN) that can model stationary stochastic process. This architecture uses the kernel smoother and the DNN to construct the translation equivariant functional representations, and thus reflects the inductive bias of the stationarity into DNN. However, since this architecture employs the kernel smoother known as the non-parametric model, it may produce ambiguous representations when the number of data points is not given sufficiently. To remedy this issue, we introduce Bayesian convolutional deep sets that construct the random translation equivariant functional representations with stationary prior. Furthermore, we present how to impose the task-dependent prior for each dataset because a wrongly imposed prior forms an even worse representation than that of the kernel smoother. We validate the proposed architecture and its training on various experiments with time-series and image datasets.