Holistic Deep Learning

TL;DR

This paper introduces a comprehensive deep learning framework that enhances accuracy, robustness, sparsity, and stability across various data types by addressing common challenges through an integrated approach validated by extensive experiments and analysis.

Contribution

The paper proposes a novel holistic deep learning framework that simultaneously tackles vulnerability, overparametrization, and instability, providing practical guidelines and code for implementation.

Findings

Improved accuracy and robustness on tabular and image datasets.

Validated trade-offs between evaluation metrics through ablation and SHAP analysis.

Practical recommendations for loss function selection based on objectives.

Abstract

This paper presents a novel holistic deep learning framework that simultaneously addresses the challenges of vulnerability to input perturbations, overparametrization, and performance instability from different train-validation splits. The proposed framework holistically improves accuracy, robustness, sparsity, and stability over standard deep learning models, as demonstrated by extensive experiments on both tabular and image data sets. The results are further validated by ablation experiments and SHAP value analysis, which reveal the interactions and trade-offs between the different evaluation metrics. To support practitioners applying our framework, we provide a prescriptive approach that offers recommendations for selecting an appropriate training loss function based on their specific objectives. All the code to reproduce the results can be found at https://github.com/kimvc7/HDL.