Graph is a Substrate Across Data Modalities

TL;DR

This paper introduces G-Substrate, a framework that treats graph structure as a persistent, shared substrate across different data modalities and tasks, enabling more effective multi-task learning.

Contribution

The paper proposes G-Substrate, a novel graph representation framework with a unified schema and role-based training to facilitate cross-modal and cross-task knowledge accumulation.

Findings

G-Substrate outperforms task-isolated methods.

Shared graph structures improve learning across domains.

Unified schema ensures compatibility among diverse graph representations.

Abstract

Graphs provide a natural representation of relational structure that arises across diverse domains. Despite this ubiquity, graph structure is typically learned in a modality- and task-isolated manner, where graph representations are constructed within individual task contexts and discarded thereafter. As a result, structural regularities across modalities and tasks are repeatedly reconstructed rather than accumulated at the level of intermediate graph representations. This motivates a representation-learning question: how should graph structure be organized so that it can persist and accumulate across heterogeneous modalities and tasks? We adopt a representation-centric perspective in which graph structure is treated as a structural substrate that persists across learning contexts. To instantiate this perspective, we propose G-Substrate, a graph substrate framework that organizes learning around shared graph structures. G-Substrate comprises two complementary mechanisms: a unified structural schema that ensures compatibility among graph representations across heterogeneous modalities and tasks, and an interleaved role-based training strategy that exposes the same graph structure to multiple functional roles during learning. Experiments across multiple domains, modalities, and tasks show that G-Substrate outperforms task-isolated and naive multi-task learning methods.