# On Compression of Unsupervised Neural Nets by Pruning Weak Connections

## TL;DR

This paper presents a method to significantly compress unsupervised neural networks like RBMs and DBNs by pruning weak connections, maintaining performance while reducing parameters.

## Contribution

It introduces a novel pruning technique for RBMs and an unsupervised sparse architecture selection algorithm for deep networks, enabling effective compression.

## Key findings

- Parameter reduction with minimal performance loss
- Sparse deep networks maintain generative and discriminative capabilities
- Method applicable to various unsupervised neural architectures

## Abstract

Unsupervised neural nets such as Restricted Boltzmann Machines(RBMs) and Deep Belif Networks(DBNs), are powerful in automatic feature extraction,unsupervised weight initialization and density estimation. In this paper,we demonstrate that the parameters of these neural nets can be dramatically reduced without affecting their performance. We describe a method to reduce the parameters required by RBM which is the basic building block for deep architectures. Further we propose an unsupervised sparse deep architectures selection algorithm to form sparse deep neural networks.Experimental results show that there is virtually no loss in either generative or discriminative performance.