sparta: Sparse Tables and their Algebra with a View Towards High Dimensional Graphical Models

TL;DR

The paper introduces the sparta package, which efficiently manages sparse tables for high-dimensional graphical models, enabling scalable inference in complex probabilistic models.

Contribution

It presents novel methods for handling multiplication and marginalization of sparse tables, facilitating inference in high-dimensional graphical models using R.

Findings

sparta enables handling of large, complex models infeasible with traditional methods

jti package demonstrates practical application of sparta for complex graphical models

efficient sparse table operations reduce memory usage significantly

Abstract

A graphical model is a multivariate (potentially very high dimensional) probabilistic model, which is formed by combining lower dimensional components. Inference (computation of conditional probabilities) is based on message passing algorithms that utilize conditional independence structures. In graphical models for discrete variables with finite state spaces, there is a fundamental problem in high dimensions: A discrete distribution is represented by a table of values, and in high dimensions such tables can become prohibitively large. In inference, such tables must be multiplied which can lead to even larger tables. The sparta package meets this challenge by implementing methods that efficiently handles multiplication and marginalization of sparse tables. The package was written in the R programming language and is freely available from the Comprehensive R Archive Network (CRAN). The companion package jti, also on CRAN, was developed to showcase the potential of sparta in connection to the Junction Tree Algorithm. We show, that jti is able to handle highly complex graphical models which are otherwise infeasible due to lack of computer memory, using sparta as a backend for table operations.