Shape Adaptor: A Learnable Resizing Module

TL;DR

The paper introduces a learnable resizing module called shape adaptor that replaces fixed resizing layers in neural networks, enabling end-to-end training and improving performance across various tasks and datasets.

Contribution

It proposes a novel, learnable resizing module that can be integrated into neural networks to optimize architecture for specific tasks without extra supervision.

Findings

Consistent performance improvements across seven image classification datasets.

Effective in network compression and transfer learning applications.

Abstract

We present a novel resizing module for neural networks: shape adaptor, a drop-in enhancement built on top of traditional resizing layers, such as pooling, bilinear sampling, and strided convolution. Whilst traditional resizing layers have fixed and deterministic reshaping factors, our module allows for a learnable reshaping factor. Our implementation enables shape adaptors to be trained end-to-end without any additional supervision, through which network architectures can be optimised for each individual task, in a fully automated way. We performed experiments across seven image classification datasets, and results show that by simply using a set of our shape adaptors instead of the original resizing layers, performance increases consistently over human-designed networks, across all datasets. Additionally, we show the effectiveness of shape adaptors on two other applications: network compression and transfer learning. The source code is available at: https://github.com/lorenmt/shape-adaptor.