# A Closed-Form Learned Pooling for Deep Classification Networks

## TL;DR

This paper introduces a novel, learnable pooling operator for CNNs that adapts spatially, improving classification accuracy and robustness by leveraging a closed-form spectral decomposition for fixed weights.

## Contribution

It proposes a new pooling method that learns spatially varying weights, generalizes average pooling, and can be computed in closed-form for fixed network weights.

## Key findings

- Improves generalization on CIFAR-10, CIFAR-100, and SVHN datasets.
- Enhances robustness to geometric corruptions and perturbations.
- Applicable to ResNets and CNNs with measurable performance gains.

## Abstract

In modern computer vision tasks, convolutional neural networks (CNNs) are indispensable for image classification tasks due to their efficiency and effectiveness. Part of their superiority compared to other architectures, comes from the fact that a single, local filter is shared across the entire image. However, there are scenarios where we may need to treat spatial locations in non-uniform manner. We see this in nature when considering how humans have evolved foveation to process different areas in their field of vision with varying levels of detail. In this paper we propose a way to enable CNNs to learn different pooling weights for each pixel location. We do so by introducing an extended definition of a pooling operator. This operator can learn a strict super-set of what can be learned by average pooling or convolutions. It has the benefit of being shared across feature maps and can be encouraged to be local or diffuse depending on the data. We show that for fixed network weights, our pooling operator can be computed in closed-form by spectral decomposition of matrices associated with class separability. Through experiments, we show that this operator benefits generalization for ResNets and CNNs on the CIFAR-10, CIFAR-100 and SVHN datasets and improves robustness to geometric corruptions and perturbations on the CIFAR-10-C and CIFAR-10-P test sets.