Automatic Bayesian Density Analysis

TL;DR

ABDA is an automated, unsupervised Bayesian method that performs density estimation, missing data imputation, anomaly detection, and dependency analysis for mixed data types, making exploratory data analysis accessible and efficient.

Contribution

It introduces ABDA, a novel framework that automates Bayesian density analysis, handling mixed data types and uncertainties without requiring expert supervision.

Findings

Effective in automatic missing value estimation

Accurate in density estimation for mixed data

Capable of anomaly detection and dependency mining

Abstract

Making sense of a dataset in an automatic and unsupervised fashion is a challenging problem in statistics and AI. Classical approaches for {exploratory data analysis} are usually not flexible enough to deal with the uncertainty inherent to real-world data: they are often restricted to fixed latent interaction models and homogeneous likelihoods; they are sensitive to missing, corrupt and anomalous data; moreover, their expressiveness generally comes at the price of intractable inference. As a result, supervision from statisticians is usually needed to find the right model for the data. However, since domain experts are not necessarily also experts in statistics, we propose Automatic Bayesian Density Analysis (ABDA) to make exploratory data analysis accessible at large. Specifically, ABDA allows for automatic and efficient missing value estimation, statistical data type and likelihood discovery, anomaly detection and dependency structure mining, on top of providing accurate density estimation. Extensive empirical evidence shows that ABDA is a suitable tool for automatic exploratory analysis of mixed continuous and discrete tabular data.