Fast Sampling for Bayesian Max-Margin Models

TL;DR

This paper introduces stochastic subgradient HMC methods for efficient Bayesian max-margin model sampling, enabling scalable inference on large datasets with validated theoretical properties and improved mixing.

Contribution

It proposes a novel stochastic subgradient HMC approach with variants for better scalability, addressing the challenge of sampling in large-scale Bayesian max-margin models.

Findings

Efficient posterior inference on large datasets.

Theoretical validation of subgradient HMC's detailed balance.

Experimental results show improved sampling efficiency.

Abstract

Bayesian max-margin models have shown superiority in various practical applications, such as text categorization, collaborative prediction, social network link prediction and crowdsourcing, and they conjoin the flexibility of Bayesian modeling and predictive strengths of max-margin learning. However, Monte Carlo sampling for these models still remains challenging, especially for applications that involve large-scale datasets. In this paper, we present the stochastic subgradient Hamiltonian Monte Carlo (HMC) methods, which are easy to implement and computationally efficient. We show the approximate detailed balance property of subgradient HMC which reveals a natural and validated generalization of the ordinary HMC. Furthermore, we investigate the variants that use stochastic subsampling and thermostats for better scalability and mixing. Using stochastic subgradient Markov Chain Monte Carlo (MCMC), we efficiently solve the posterior inference task of various Bayesian max-margin models and extensive experimental results demonstrate the effectiveness of our approach.