CNN2GNN: How to Bridge CNN with GNN

TL;DR

This paper introduces CNN2GNN, a framework that combines CNN and GNN via distillation, enabling efficient learning of intra-sample features and dataset topology, leading to improved performance on vision tasks.

Contribution

The paper proposes a novel CNN2GNN framework with a differentiable graph learning module and response-based distillation to unify CNN and GNN for enhanced vision task performance.

Findings

Distilled GNN outperforms deep CNNs like ResNet152 on Mini-ImageNet.

Dynamic graph learning improves GNN's ability to model data relationships.

The approach reduces training costs while boosting accuracy.

Abstract

Although the convolutional neural network (CNN) has achieved excellent performance in vision tasks by extracting the intra-sample representation, it will take a higher training expense because of stacking numerous convolutional layers. Recently, as the bilinear models, graph neural networks (GNN) have succeeded in exploring the underlying topological relationship among the graph data with a few graph neural layers. Unfortunately, it cannot be directly utilized on non-graph data due to the lack of graph structure and has high inference latency on large-scale scenarios. Inspired by these complementary strengths and weaknesses, \textit{we discuss a natural question, how to bridge these two heterogeneous networks?} In this paper, we propose a novel CNN2GNN framework to unify CNN and GNN together via distillation. Firstly, to break the limitations of GNN, a differentiable sparse graph learning module is designed as the head of networks to dynamically learn the graph for inductive learning. Then, a response-based distillation is introduced to transfer the knowledge from CNN to GNN and bridge these two heterogeneous networks. Notably, due to extracting the intra-sample representation of a single instance and the topological relationship among the datasets simultaneously, the performance of distilled ``boosted'' two-layer GNN on Mini-ImageNet is much higher than CNN containing dozens of layers such as ResNet152.