PI-NLF: A Proportional-Integral Approach for Non-negative Latent Factor Analysis

TL;DR

This paper introduces PI-NLF, a novel non-negative latent factor model that incorporates a proportional-integral control mechanism to enhance convergence speed and accuracy in high-dimensional, incomplete data analysis.

Contribution

It proposes a PI-controlled IR mechanism and an IR-based SLF-NMU algorithm, significantly improving convergence and estimation accuracy over existing methods.

Findings

Outperforms state-of-the-art models in efficiency and accuracy

Effective in handling high-dimensional incomplete data

Demonstrates the feasibility of using control theory in machine learning algorithms

Abstract

A high-dimensional and incomplete (HDI) matrix frequently appears in various big-data-related applications, which demonstrates the inherently non-negative interactions among numerous nodes. A non-negative latent factor (NLF) model performs efficient representation learning to an HDI matrix, whose learning process mostly relies on a single latent factor-dependent, non-negative and multiplicative update (SLF-NMU) algorithm. However, an SLF-NMU algorithm updates a latent factor based on the current update increment only without appropriate considerations of past learning information, resulting in slow convergence. Inspired by the prominent success of a proportional-integral (PI) controller in various applications, this paper proposes a Proportional-Integral-incorporated Non-negative Latent Factor (PI-NLF) model with two-fold ideas: a) establishing an Increment Refinement (IR) mechanism via considering the past update increments following the principle of a PI controller; and b) designing an IR-based SLF-NMU (ISN) algorithm to accelerate the convergence rate of a resultant model. Empirical studies on four HDI datasets demonstrate that a PI-NLF model outperforms the state-of-the-art models in both computational efficiency and estimation accuracy for missing data of an HDI matrix. Hence, this study unveils the feasibility of boosting the performance of a non-negative learning algorithm through an error feedback controller.