A Practical Second-order Latent Factor Model via Distributed Particle Swarm Optimization

TL;DR

This paper introduces a practical second-order latent factor model that uses distributed particle swarm optimization for automatic hyperparameter tuning, improving data representation in high-dimensional sparse data.

Contribution

The paper proposes a new practical SLF model with hyperparameter self-adaptation via distributed particle swarm optimization, enhancing usability and performance.

Findings

PSLF outperforms state-of-the-art models in data representation.

Distributed PSO enables efficient hyperparameter optimization.

The model is effective on real high-dimensional sparse datasets.

Abstract

Latent Factor (LF) models are effective in representing high-dimension and sparse (HiDS) data via low-rank matrices approximation. Hessian-free (HF) optimization is an efficient method to utilizing second-order information of an LF model's objective function and it has been utilized to optimize second-order LF (SLF) model. However, the low-rank representation ability of a SLF model heavily relies on its multiple hyperparameters. Determining these hyperparameters is time-consuming and it largely reduces the practicability of an SLF model. To address this issue, a practical SLF (PSLF) model is proposed in this work. It realizes hyperparameter self-adaptation with a distributed particle swarm optimizer (DPSO), which is gradient-free and parallelized. Experiments on real HiDS data sets indicate that PSLF model has a competitive advantage over state-of-the-art models in data representation ability.