Using the Projected Belief Network at High Dimensions

TL;DR

This paper enhances the applicability of the projected belief network (PBN) for high-dimensional data by introducing techniques to overcome computational limitations, and demonstrates its effectiveness in classifying and auto-encoding acoustic spectrograms.

Contribution

It introduces methods to mitigate high-dimensional restrictions of PBNs and presents the first discriminatively aligned D-PBN for improved high-dimensional data processing.

Findings

Effective classification and auto-encoding of high-dimensional spectrograms.

Techniques to avoid or mitigate matrix inversion restrictions in PBN.

First application of discriminatively aligned D-PBN.

Abstract

The projected belief network (PBN) is a layered generative network (LGN) with tractable likelihood function, and is based on a feed-forward neural network (FFNN). There are two versions of the PBN: stochastic and deterministic (D-PBN), and each has theoretical advantages over other LGNs. However, implementation of the PBN requires an iterative algorithm that includes the inversion of a symmetric matrix of size M X M in each layer, where M is the layer output dimension. This, and the fact that the network must be always dimension-reducing in each layer, can limit the types of problems where the PBN can be applied. In this paper, we describe techniques to avoid or mitigate these restrictions and use the PBN effectively at high dimension. We apply the discriminatively aligned PBN (PBN-DA) to classifying and auto-encoding high-dimensional spectrograms of acoustic events. We also present the discriminatively aligned D-PBN for the first time.