An Entropy Weighted Nonnegative Matrix Factorization Algorithm for Feature Representation

TL;DR

This paper introduces an entropy weighted NMF method that assigns importance weights to data attributes, improving feature representation accuracy by emphasizing relevant attributes and reducing the influence of irrelevant ones.

Contribution

The paper proposes a novel entropy weighted NMF algorithm that incorporates attribute importance weights via an entropy regularizer, enhancing data representation quality.

Findings

Effective attribute weighting improves representation accuracy

Method outperforms traditional NMF on multiple datasets

Code is publicly available for reproducibility

Abstract

Nonnegative matrix factorization (NMF) has been widely used to learn low-dimensional representations of data. However, NMF pays the same attention to all attributes of a data point, which inevitably leads to inaccurate representation. For example, in a human-face data set, if an image contains a hat on the head, the hat should be removed or the importance of its corresponding attributes should be decreased during matrix factorizing. This paper proposes a new type of NMF called entropy weighted NMF (EWNMF), which uses an optimizable weight for each attribute of each data point to emphasize their importance. This process is achieved by adding an entropy regularizer to the cost function and then using the Lagrange multiplier method to solve the problem. Experimental results with several data sets demonstrate the feasibility and effectiveness of the proposed method. We make our code available at https://github.com/Poisson-EM/Entropy-weighted-NMF.