Cardinality-Preserving Attention Channels for Graph Transformers in Molecular Property Prediction

TL;DR

This paper introduces CardinalGraphFormer, a graph transformer with a novel attention channel that preserves cardinality information, improving molecular property prediction especially with limited labeled data.

Contribution

It proposes a new query-conditioned attention mechanism called CPA that enhances graph transformers by preserving dynamic support-size signals, combined with structured sparse attention and dual-objective self-supervised pretraining.

Findings

Consistent improvements on 11 benchmark datasets.

Ablation studies confirm CPA's effectiveness.

Provides code and reproducibility artifacts.

Abstract

Molecular property prediction is crucial for drug discovery when labeled data are scarce. This work presents CardinalGraphFormer, a graph transformer augmented with a query-conditioned cardinality-preserving attention (CPA) channel that retains dynamic support-size signals complementary to static centrality embeddings. The approach combines structured sparse attention with Graphormer-inspired biases (shortest-path distance, centrality, direct-bond features) and unified dual-objective self-supervised pretraining (masked reconstruction and contrastive alignment of augmented views). Evaluation on 11 public benchmarks spanning MoleculeNet, OGB, and TDC ADMET demonstrates consistent improvements over protocol-matched baselines under matched pretraining, optimization, and hyperparameter tuning. Rigorous ablations confirm CPA's contributions and rule out simple size shortcuts. Code and reproducibility artifacts are provided.