Second-order Symmetric Non-negative Latent Factor Analysis

TL;DR

This paper introduces a second-order symmetric non-negative latent factor analysis model for undirected networks, improving representation accuracy over existing methods by efficiently handling non-convex optimization.

Contribution

It proposes a novel second-order optimization approach for SNLF, incorporating a mapping strategy to form an unconstrained model and using a specialized second-order method for efficient training.

Findings

Outperforms state-of-the-art models in accuracy

Maintains affordable computational costs

Effective handling of non-convex optimization problems

Abstract

Precise representation of large-scale undirected network is the basis for understanding relations within a massive entity set. The undirected network representation task can be efficiently addressed by a symmetry non-negative latent factor (SNLF) model, whose objective is clearly non-convex. However, existing SNLF models commonly adopt a first-order optimizer that cannot well handle the non-convex objective, thereby resulting in inaccurate representation results. On the other hand, higher-order learning algorithms are expected to make a breakthrough, but their computation efficiency are greatly limited due to the direct manipulation of the Hessian matrix, which can be huge in undirected network representation tasks. Aiming at addressing this issue, this study proposes to incorporate an efficient second-order method into SNLF, thereby establishing a second-order symmetric non-negative latent factor analysis model for undirected network with two-fold ideas: a) incorporating a mapping strategy into SNLF model to form an unconstrained model, and b) training the unconstrained model with a specially designed second order method to acquire a proper second-order step efficiently. Empirical studies indicate that proposed model outperforms state-of-the-art models in representation accuracy with affordable computational burden.