Modeling High-Dimensional Data with Unknown Cut Points: A Fusion Penalized Logistic Threshold Regression

TL;DR

This paper introduces a novel high-dimensional logistic regression model that estimates unknown threshold points and coefficients simultaneously, improving disease prediction accuracy with a fusion penalization approach.

Contribution

The paper proposes the FILTER model, combining threshold estimation and variable selection via fused lasso penalty, with theoretical guarantees and practical applications in disease prediction.

Findings

The FILTER model achieves consistent variable selection and threshold estimation.

Monte Carlo studies validate the theoretical error bounds.

Application to diabetes data demonstrates practical effectiveness.

Abstract

In traditional logistic regression models, the link function is often assumed to be linear and continuous in predictors. Here, we consider a threshold model that all continuous features are discretized into ordinal levels, which further determine the binary responses. Both the threshold points and regression coefficients are unknown and to be estimated. For high dimensional data, we propose a fusion penalized logistic threshold regression (FILTER) model, where a fused lasso penalty is employed to control the total variation and shrink the coefficients to zero as a method of variable selection. Under mild conditions on the estimate of unknown threshold points, we establish the non-asymptotic error bound for coefficient estimation and the model selection consistency. With a careful characterization of the error propagation, we have also shown that the tree-based method, such as CART, fulfill the threshold estimation conditions. We find the FILTER model is well suited in the problem of early detection and prediction for chronic disease like diabetes, using physical examination data. The finite sample behavior of our proposed method are also explored and compared with extensive Monte Carlo studies, which supports our theoretical discoveries.