OpenMAG: A Comprehensive Benchmark for Multimodal-Attributed Graph
Chenxi Wan, Xunkai Li, Yilong Zuo, Haokun Deng, Sihan Li, Bowen Fan, Hongchao Qin, Ronghua Li, Guoren Wang

TL;DR
OpenMAG is a comprehensive benchmark for Multimodal-Attributed Graph learning, integrating diverse datasets, encoders, and models to facilitate fair evaluation and guide future research in the field.
Contribution
It introduces a unified benchmark with extensive datasets, encoders, and models, addressing limitations of previous benchmarks and enabling systematic assessment of MAG methods.
Findings
14 fundamental insights into MAG learning.
Benchmark supports 8 downstream tasks.
Systematic evaluation of necessity, data quality, effectiveness, robustness, and efficiency.
Abstract
Multimodal-Attributed Graph (MAG) learning has achieved remarkable success in modeling complex real-world systems by integrating graph topology with rich attributes from multiple modalities. With the rapid proliferation of novel MAG models capable of handling intricate cross-modal semantics and structural dependencies, establishing a rigorous and unified evaluation standard has become imperative. Although existing benchmarks have facilitated initial progress, they exhibit critical limitations in domain coverage, encoder flexibility, model diversity, and task scope, presenting significant challenges to fair evaluation. To bridge this gap, we present OpenMAG, a comprehensive benchmark that integrates 19 datasets across 6 domains and incorporates 16 encoders to support both static and trainable feature encoding. OpenMAG further implements a standardized library of 24 state-of-the-art…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsAdvanced Graph Neural Networks · Multimodal Machine Learning Applications · Domain Adaptation and Few-Shot Learning
