A Proportional-Integral Controller-Incorporated SGD Algorithm for High Efficient Latent Factor Analysis

TL;DR

This paper introduces a novel SGD algorithm with a PI controller for latent factor analysis in high-dimensional sparse data, improving convergence and generalization.

Contribution

It develops a PI-accelerated SGD method that incorporates historical information and sample correlations, enhancing latent factor extraction.

Findings

Superior representation of HDI matrices demonstrated

Faster convergence compared to traditional SGD methods

Improved generalization performance

Abstract

In industrial big data scenarios, high-dimensional sparse matrices (HDI) are widely used to characterize high-order interaction relationships among massive nodes. The stochastic gradient descent-based latent factor analysis (SGD-LFA) method can effectively extract deep feature information embedded in HDI matrices. However, existing SGD-LFA methods exhibit significant limitations: their parameter update process relies solely on the instantaneous gradient information of current samples, failing to incorporate accumulated experiential knowledge from historical iterations or account for intrinsic correlations between samples, resulting in slow convergence speed and suboptimal generalization performance. Thus, this paper proposes a PILF model by developing a PI-accelerated SGD algorithm by integrating correlated instances and refining learning errors through proportional-integral (PI) control mechanism that current and historical information; Comparative experiments demonstrate the superior representation capability of the PILF model on HDI matrices