Deep learning for autism detection using clinical notes: A comparison of transfer learning for a transparent and black-box approach

TL;DR

This study compares transparent BioBERT-based models and black-box models for autism detection from clinical notes, demonstrating that transparent models with mixed datasets achieve high accuracy and better transferability, advancing trustworthy AI in diagnostics.

Contribution

The paper introduces a transparent, interpretable ML approach using BioBERT for autism detection and compares its transfer learning performance to black-box models across multiple datasets.

Findings

Transparent model achieved 97% sensitivity and 98% specificity.

Mixed dataset training improved model performance.

Transparent models outperformed black-box models in transferability.

Abstract

Autism spectrum disorder (ASD) is a complex neurodevelopmental condition whose rising prevalence places increasing demands on a lengthy diagnostic process. Machine learning (ML) has shown promise in automating ASD diagnosis, but most existing models operate as black boxes and are typically trained on a single dataset, limiting their generalizability. In this study, we introduce a transparent and interpretable ML approach that leverages BioBERT, a state-of-the-art language model, to analyze unstructured clinical text. The model is trained to label descriptions of behaviors and map them to diagnostic criteria, which are then used to assign a final label (ASD or not). We evaluate transfer learning, the ability to transfer knowledge to new data, using two distinct real-world datasets. We trained on datasets sequentially and mixed together and compared the performance of the best models and their ability to transfer to new data. We also created a black-box approach and repeated this transfer process for comparison. Our transparent model demonstrated robust performance, with the mixed-data training strategy yielding the best results (97 % sensitivity, 98 % specificity). Sequential training across datasets led to a slight drop in performance, highlighting the importance of training data order. The black-box model performed worse (90 % sensitivity, 96 % specificity) when trained sequentially or with mixed data. Overall, our transparent approach outperformed the black-box approach. Mixing datasets during training resulted in slightly better performance and should be the preferred approach when practically possible. This work paves the way for more trustworthy, generalizable, and clinically actionable AI tools in neurodevelopmental diagnostics.