Why Deep Models Often cannot Beat Non-deep Counterparts on Molecular Property Prediction?

TL;DR

Deep neural networks often do not outperform traditional models in molecular property prediction due to data irregularities, with tree models using molecular fingerprints showing better performance.

Contribution

This study provides the most comprehensive benchmarking of deep and non-deep models on molecular datasets, revealing key factors affecting model performance.

Findings

Deep models generally do not outperform non-deep models.

Irregular data patterns, not dataset size, hinder deep model performance.

Tree models with molecular fingerprints often perform best.

Abstract

Molecular property prediction (MPP) is a crucial task in the drug discovery pipeline, which has recently gained considerable attention thanks to advances in deep neural networks. However, recent research has revealed that deep models struggle to beat traditional non-deep ones on MPP. In this study, we benchmark 12 representative models (3 non-deep models and 9 deep models) on 14 molecule datasets. Through the most comprehensive study to date, we make the following key observations: \textbf{(\romannumeral 1)} Deep models are generally unable to outperform non-deep ones; \textbf{(\romannumeral 2)} The failure of deep models on MPP cannot be solely attributed to the small size of molecular datasets. What matters is the irregular molecule data pattern; \textbf{(\romannumeral 3)} In particular, tree models using molecular fingerprints as inputs tend to perform better than other competitors. Furthermore, we conduct extensive empirical investigations into the unique patterns of molecule data and inductive biases of various models underlying these phenomena.