A Metric for the Balance of Information in Graph Learning

TL;DR

This paper introduces NNRD, a metric that quantifies whether molecular graph datasets contain more useful information in structure or features, aiding in tailored graph learning strategies.

Contribution

The paper proposes NNRD, a novel quantitative metric based on iterative noising to assess the dominant information source in molecular graph datasets.

Findings

NNRD correlates well with information loss in molecular tasks.

It provides more expressive insights than simple performance metrics.

The metric helps guide learning strategies based on dataset information bias.

Abstract

Graph learning on molecules makes use of information from both the molecular structure and the features attached to that structure. Much work has been conducted on biasing either towards structure or features, with the aim that bias bolsters performance. Identifying which information source a dataset favours, and therefore how to approach learning that dataset, is an open issue. Here we propose Noise-Noise Ratio Difference (NNRD), a quantitative metric for whether there is more useful information in structure or features. By employing iterative noising on features and structure independently, leaving the other intact, NNRD measures the degradation of information in each. We employ NNRD over a range of molecular tasks, and show that it corresponds well to a loss of information, with intuitive results that are more expressive than simple performance aggregates. Our future work will focus on expanding data domains, tasks and types, as well as refining our choice of baseline model.