ELBD: Efficient score algorithm for feature selection on latent variables of VAE

TL;DR

This paper introduces ELBD, an efficient score algorithm for feature selection on VAE latent variables, improving model performance through targeted variable weighting and demonstrating effectiveness across generative and classification tasks.

Contribution

The paper proposes ELBD, a novel feature selection method for VAE latent variables, with theoretical support and extensive experiments showing its advantages over existing methods.

Findings

ELBD outperforms 9 other feature selection methods on 7 datasets.

The algorithms effectively improve VAE model performance.

ELBD generalizes well to classification tasks with new datasets.

Abstract

In this paper, we develop the notion of evidence lower bound difference (ELBD), based on which an efficient score algorithm is presented to implement feature selection on latent variables of VAE and its variants. Further, we propose weak convergence approximation algorithms to optimize VAE related models through weighing the ``more important" latent variables selected and accordingly increasing evidence lower bound. We discuss two kinds of different Gaussian posteriors, mean-filed and full-covariance, for latent variables, and make corresponding theoretical analyses to support the effectiveness of algorithms. A great deal of comparative experiments are carried out between our algorithms and other 9 feature selection methods on 7 public datasets to address generative tasks. The results provide the experimental evidence of effectiveness of our algorithms. Finally, we extend ELBD to its generalized version, and apply the latter to tackling classification tasks of 5 new public datasets with satisfactory experimental results.