Multi-way Interacting Regression via Factorization Machines

TL;DR

This paper introduces a Bayesian regression model that captures complex multi-way interactions among predictors using a factorization approach, with applications demonstrated in genetics and retail demand forecasting.

Contribution

It presents a novel Bayesian regression framework with a hypergraph-guided interaction selection mechanism and a Gibbs sampling inference algorithm.

Findings

Successfully identifies meaningful interactions in simulated data.

Demonstrates effectiveness in genetics and retail demand forecasting.

Establishes posterior consistency of the model.

Abstract

We propose a Bayesian regression method that accounts for multi-way interactions of arbitrary orders among the predictor variables. Our model makes use of a factorization mechanism for representing the regression coefficients of interactions among the predictors, while the interaction selection is guided by a prior distribution on random hypergraphs, a construction which generalizes the Finite Feature Model. We present a posterior inference algorithm based on Gibbs sampling, and establish posterior consistency of our regression model. Our method is evaluated with extensive experiments on simulated data and demonstrated to be able to identify meaningful interactions in applications in genetics and retail demand forecasting.