AAANE: Attention-based Adversarial Autoencoder for Multi-scale Network Embedding

TL;DR

This paper introduces AAANE, an innovative multi-scale network embedding method that uses attention mechanisms and adversarial regularization to improve the quality and robustness of node representations.

Contribution

It is the first to incorporate attention mechanisms into multi-scale network embedding, enhancing the integration of different structural scales.

Findings

Attention parameters vary across networks

AAANE outperforms state-of-the-art methods

Effective in capturing non-linear network structures

Abstract

Network embedding represents nodes in a continuous vector space and preserves structure information from the Network. Existing methods usually adopt a "one-size-fits-all" approach when concerning multi-scale structure information, such as first- and second-order proximity of nodes, ignoring the fact that different scales play different roles in the embedding learning. In this paper, we propose an Attention-based Adversarial Autoencoder Network Embedding(AAANE) framework, which promotes the collaboration of different scales and lets them vote for robust representations. The proposed AAANE consists of two components: 1) Attention-based autoencoder effectively capture the highly non-linear network structure, which can de-emphasize irrelevant scales during training. 2) An adversarial regularization guides the autoencoder learn robust representations by matching the posterior distribution of the latent embeddings to given prior distribution. This is the first attempt to introduce attention mechanisms to multi-scale network embedding. Experimental results on real-world networks show that our learned attention parameters are different for every network and the proposed approach outperforms existing state-of-the-art approaches for network embedding.