L_1-regularized Boltzmann machine learning using majorizer minimization

TL;DR

This paper introduces a novel inference method for sparse Boltzmann machine learning that employs L1 regularization combined with majorizer minimization to effectively estimate biases and interactions from correlated data.

Contribution

It presents a new approach integrating L1 regularization with majorizer minimization to handle non-smooth optimization in sparse Boltzmann machine learning.

Findings

Successfully estimates sparse interactions and biases from correlated data.

Effectively handles non-smooth cost functions in optimization.

Provides a practical inference method for sparse models.

Abstract

We propose an inference method to estimate sparse interactions and biases according to Boltzmann machine learning. The basis of this method is $L_1$ regularization, which is often used in compressed sensing, a technique for reconstructing sparse input signals from undersampled outputs. $L_1$ regularization impedes the simple application of the gradient method, which optimizes the cost function that leads to accurate estimations, owing to the cost function's lack of smoothness. In this study, we utilize the majorizer minimization method, which is a well-known technique implemented in optimization problems, to avoid the non-smoothness of the cost function. By using the majorizer minimization method, we elucidate essentially relevant biases and interactions from given data with seemingly strongly-correlated components.