# Recovering the Lowest Layer of Deep Networks with High Threshold Activations

## TL;DR

This paper provides theoretical guarantees for recovering the lowest layer of deep neural networks with high-threshold activations, assuming Gaussian inputs and polynomial upper layers, advancing understanding of deep network identifiability.

## Contribution

It extends existing parameter recovery results from shallow to deep networks by focusing on the lowest layer with high threshold activations under specific assumptions.

## Key findings

- Guarantees for recovering the lowest layer in deep networks
- Applicable to networks with high threshold activations
- Assumes Gaussian input distribution and polynomial upper layers

## Abstract

Giving provable guarantees for learning neural networks is a core challenge of machine learning theory. Most prior work gives parameter recovery guarantees for one hidden layer networks, however, the networks used in practice have multiple non-linear layers. In this work, we show how we can strengthen such results to deeper networks -- we address the problem of uncovering the lowest layer in a deep neural network under the assumption that the lowest layer uses a high threshold before applying the activation, the upper network can be modeled as a well-behaved polynomial and the input distribution is Gaussian.