Conformal Drug Property Prediction with Density Estimation under Covariate Shift

TL;DR

This paper introduces CoDrug, a method combining energy-based models and density estimation to improve the reliability of uncertainty predictions in drug discovery, especially under distribution shifts.

Contribution

It presents CoDrug, a novel approach that adjusts conformal prediction sets for covariate shift using density estimation, enhancing prediction validity in drug discovery.

Findings

CoDrug reduces coverage gap by over 35% under distribution shift.

It provides valid prediction sets in realistic drug discovery scenarios.

The method effectively addresses covariate shift in molecular property prediction.

Abstract

In drug discovery, it is vital to confirm the predictions of pharmaceutical properties from computational models using costly wet-lab experiments. Hence, obtaining reliable uncertainty estimates is crucial for prioritizing drug molecules for subsequent experimental validation. Conformal Prediction (CP) is a promising tool for creating such prediction sets for molecular properties with a coverage guarantee. However, the exchangeability assumption of CP is often challenged with covariate shift in drug discovery tasks: Most datasets contain limited labeled data, which may not be representative of the vast chemical space from which molecules are drawn. To address this limitation, we propose a method called CoDrug that employs an energy-based model leveraging both training data and unlabelled data, and Kernel Density Estimation (KDE) to assess the densities of a molecule set. The estimated densities are then used to weigh the molecule samples while building prediction sets and rectifying for distribution shift. In extensive experiments involving realistic distribution drifts in various small-molecule drug discovery tasks, we demonstrate the ability of CoDrug to provide valid prediction sets and its utility in addressing the distribution shift arising from de novo drug design models. On average, using CoDrug can reduce the coverage gap by over 35% when compared to conformal prediction sets not adjusted for covariate shift.