Machine Learning for Two-Sample Testing under Right-Censored Data: A Simulation Study

TL;DR

This study evaluates machine learning methods for two-sample testing with right-censored data, comparing their statistical power to classical tests through extensive simulation, and introduces new ensemble ML approaches.

Contribution

The paper develops and assesses 18 ML-based two-sample tests, including ensemble methods, for right-censored data, providing a comprehensive comparison with classical tests.

Findings

ML methods show competitive power compared to classical tests

Ensemble ML approaches improve test performance

Null distribution analysis confirms validity of proposed methods

Abstract

The focus of this study is to evaluate the effectiveness of Machine Learning (ML) methods for two-sample testing with right-censored observations. To achieve this, we develop several ML-based methods with varying architectures and implement them as two-sample tests. Each method is an ensemble (stacking) that combines predictions from classical two-sample tests. This paper presents the results of training the proposed ML methods, examines their statistical power compared to classical two-sample tests, analyzes the null distribution of the proposed methods when the null hypothesis is true, and evaluates the significance of the features incorporated into the proposed methods. In total, this work covers 18 methods for two-sample testing under right-censored observations, including the proposed methods and classical well-studied two-sample tests. All results from numerical experiments were obtained from a synthetic dataset generated using the inverse transform sampling method and replicated multiple times through Monte Carlo simulation. To test the two-sample problem with right-censored observations, one can use the proposed two-sample methods (scripts, dataset, and models are available on GitHub and Hugging Face).