Interpretable QSPR Modeling using Recursive Feature Machines and Multi-scale Fingerprints

TL;DR

This paper introduces Recursive Feature Machines with multi-scale fingerprints and AGOP-based deep feature learning to improve interpretability and accuracy in QSPR modeling, achieving state-of-the-art results in molecular property prediction.

Contribution

It presents a novel RFM framework integrating deep feature learning with AGOP, multi-scale fingerprints, and interpretability techniques for enhanced molecular property modeling.

Findings

RFM-HF outperforms traditional ML and GNN models.

Multi-scale fingerprints reveal structural determinants effectively.

AGOP enhances kernel machine interpretability and performance.

Abstract

This study pioneers the application of Recursive Feature Machines (RFM) in QSPR modeling, introducing a tailored feature importance analysis approach to enhance interpretability. By leveraging deep feature learning through AGOP, RFM achieves state-of-the-art (SOTA) results in predicting molecular properties, as demonstrated through solubility prediction across nine benchmark datasets. To capture a wide array of structural information, we employ diverse molecular representations, including MACCS keys, Morgan fingerprints, and a custom multi-scale hybrid fingerprint (HF) derived from global descriptors and SMILES local fragmentation techniques. Notably, the HF offers significant advantages over MACCS and Morgan fingerprints in revealing structural determinants of molecular properties. The feature importance analysis in RFM provides robust local and global explanations, effectively identifying structural features that drive molecular behavior and offering valuable insights for drug development. Additionally, RFM demonstrates strong redundancy-filtering abilities, as model performance remains stable even after removing redundant features within custom fingerprints. Importantly, RFM introduces the deep feature learning capabilities of the average gradient outer product (AGOP) matrix into ultra-fast kernel machine learning, to imbue kernel machines with interpretable deep feature learning capabilities. We extend this approach beyond the Laplace Kernel to the Matern, Rational Quadratic, and Gaussian kernels, to find that the Matern and Laplace kernels deliver the best performance, thus reinforcing the flexibility and effectiveness of AGOP in RFM. Experimental results show that RFM-HF surpasses both traditional machine learning models and advanced graph neural networks.