Entropic Trace Estimates for Log Determinants

TL;DR

This paper introduces a maximum entropy-based method for estimating log determinants of matrices, significantly improving accuracy and speed over existing techniques, with applications in large-scale machine learning models.

Contribution

It presents a novel maximum entropy framework for log determinant estimation using stochastic trace constraints, enhancing computational efficiency.

Findings

Outperforms state-of-the-art methods on sparse matrices

Accelerates inference in large-scale Markov random fields

Demonstrates broad applicability across machine learning models

Abstract

The scalable calculation of matrix determinants has been a bottleneck to the widespread application of many machine learning methods such as determinantal point processes, Gaussian processes, generalised Markov random fields, graph models and many others. In this work, we estimate log determinants under the framework of maximum entropy, given information in the form of moment constraints from stochastic trace estimation. The estimates demonstrate a significant improvement on state-of-the-art alternative methods, as shown on a wide variety of UFL sparse matrices. By taking the example of a general Markov random field, we also demonstrate how this approach can significantly accelerate inference in large-scale learning methods involving the log determinant.