Multi-modal Learning based Prediction for Disease

TL;DR

This paper introduces DeepFLDDiag, a multi-modal deep learning system that predicts NAFLD using comprehensive clinical data and facial images, offering a non-invasive alternative to traditional diagnosis methods.

Contribution

It presents a large, multi-modal dataset and a novel deep neural network model that effectively combines clinical metadata and facial images for NAFLD prediction, outperforming existing methods.

Findings

DeepFLD outperforms metadata-only models in NAFLD prediction.

Facial images alone can achieve competitive diagnostic results.

The comprehensive dataset enhances the robustness of non-invasive diagnosis.

Abstract

Non alcoholic fatty liver disease (NAFLD) is the most common cause of chronic liver disease, which can be predicted accurately to prevent advanced fibrosis and cirrhosis. While, a liver biopsy, the gold standard for NAFLD diagnosis, is invasive, expensive, and prone to sampling errors. Therefore, non-invasive studies are extremely promising, yet they are still in their infancy due to the lack of comprehensive research data and intelligent methods for multi-modal data. This paper proposes a NAFLD diagnosis system (DeepFLDDiag) combining a comprehensive clinical dataset (FLDData) and a multi-modal learning based NAFLD prediction method (DeepFLD). The dataset includes over 6000 participants physical examinations, laboratory and imaging studies, extensive questionnaires, and facial images of partial participants, which is comprehensive and valuable for clinical studies. From the dataset, we quantitatively analyze and select clinical metadata that most contribute to NAFLD prediction. Furthermore, the proposed DeepFLD, a deep neural network model designed to predict NAFLD using multi-modal input, including metadata and facial images, outperforms the approach that only uses metadata. Satisfactory performance is also verified on other unseen datasets. Inspiringly, DeepFLD can achieve competitive results using only facial images as input rather than metadata, paving the way for a more robust and simpler non-invasive NAFLD diagnosis.