Graph-Based Tests for Multivariate Covariate Balance Under Multi-Valued Treatments

TL;DR

This paper introduces non-parametric, graph-based statistical tests to evaluate covariate balance in observational studies with multi-valued treatments, demonstrating superior detection of model misspecification and imbalances.

Contribution

The authors develop novel graph-based tests for covariate balance that outperform existing methods, including new algorithms and practical implementation in an R package.

Findings

Graph-based tests detect misspecification missed by other methods.

Approximate nearest neighbor graphs often outperform optimal graphs in tests.

The methods successfully identify covariate imbalances in breast cancer data.

Abstract

We propose the use of non-parametric, graph-based tests to assess the distributional balance of covariates in observational studies with multi-valued treatments. Our tests utilize graph structures ranging from Hamiltonian paths that connect all of the data to nearest neighbor graphs that maximally separates data into pairs. We consider algorithms that form minimal distance graphs, such as optimal Hamiltonian paths or non-bipartite matching, or approximate alternatives, such as greedy Hamiltonian paths or greedy nearest neighbor graphs. Extensive simulation studies demonstrate that the proposed tests are able to detect the misspecification of matching models that other methods miss. Contrary to intuition, we also find that tests ran on well-formed approximate graphs do better in most cases than tests run on optimally formed graphs, and that a properly formed test on an approximate nearest neighbor graph performs best, on average. In a multi-valued treatment setting with breast cancer data, these graph-based tests can also detect imbalances otherwise missed by common matching diagnostics. We provide a new R package graphTest to implement these methods and reproduce our results.