An algorithm for quantifying dependence in multivariate data sets

TL;DR

This paper introduces an algorithm that detects both linear and non-linear dependencies in multivariate data, providing a reliable measure of dependence crucial for high energy physics analyses.

Contribution

It presents the CAT tool, capable of automatically applying various dependence tests and generating comprehensive analysis reports.

Findings

The algorithm effectively identifies complex dependencies.

The CAT tool automates dependence analysis in multivariate datasets.

The method improves reliability of maximum likelihood analyses.

Abstract

We describe an algorithm to quantify dependence in a multivariate data set. The algorithm is able to identify any linear and non-linear dependence in the data set by performing a hypothesis test for two variables being independent. As a result we obtain a reliable measure of dependence. In high energy physics understanding dependencies is especially important in multidimensional maximum likelihood analyses. We therefore describe the problem of a multidimensional maximum likelihood analysis applied on a multivariate data set with variables that are dependent on each other. We review common procedures used in high energy physics and show that general dependence is not the same as linear correlation and discuss their limitations in practical application. Finally we present the tool CAT, which is able to perform all reviewed methods in a fully automatic mode and creates an analysis report document with numeric results and visual review.