Evaluating predictive performance, validity, and applicability of machine learning models for predicting HIV treatment interruption: a systematic review

TL;DR

This paper reviews machine learning models for predicting HIV treatment interruption, finding moderate performance but highlighting the need for better validation and data handling.

Contribution

A systematic review of ML models for HIV treatment interruption prediction, revealing gaps in validation and bias handling.

Findings

Nine studies reported 12 ML models, with Random Forest, XGBoost, and AdaBoost being most common.

Models showed moderate performance (mean AUC-ROC of 0.668) but high risk of bias in 75% of cases.

Only two models included external validation, and most lacked decision curve analysis.

Abstract

HIV treatment interruption remains a significant barrier to achieving global HIV/AIDS control goals. Machine learning (ML) models offer potential for predicting treatment interruption by leveraging large clinical data. Understanding how these models were developed, validated, and applied remains essential for advancing research.

We searched databases including the PubMed, BMC, Cochrane Library, Scopus, ScienceDirect, Lancet, and Google Scholar, for studies published in English from 1990 to September 2024. Search terms covered HIV, machine learning, treatment interruption, and loss to follow-up. Articles were screened and reviewed independently, and data were extracted using the CHecklist for critical Appraisal and data extraction for systematic Reviews of prediction Modelling Studies (CHARMS) tool. Risk of bias was assessed with Prediction model Risk Of Bias Assessment Tool (PROBAST). The Preferred Reporting Items for Systematic reviews and Meta-analysis (PRISMA) guidelines were followed throughout.

Out of 116,672 records, 9 studies met the inclusion criteria and reported 12 ML models. Random Forest, XGBoost, and AdaBoost were predominant models (91.7%). Internal validation was performed in all models, but only two models included external validation. Performance varied, with a mean area under the receiver operating characteristic curve (AUC-ROC) of 0.668 (standard deviation (SD) = 0.066), indicating moderate discrimination. About 75% of models showed a high risk of bias due to inadequate handling of missing data, lack of calibration, and the absence of decision curve analysis (DCA).

ML models show promise for predicting HIV treatment interruption, particularly in resource-limited settings. Future research should prioritize external validation, robust missing data handling, and decision curve analysis and include sociocultural predictors to improve model robustness.

PROSPERO CRD42024578109.

The online version contains supplementary material available at 10.1186/s44263-025-00184-4.

Figures