Shape-Tailored Deep Neural Networks

TL;DR

Shape-Tailored Deep Neural Networks (ST-DNN) extend traditional CNNs to compute descriptors on arbitrarily shaped regions using PDEs, improving segmentation robustness and efficiency.

Contribution

The paper introduces ST-DNN, a novel method that generalizes CNNs to arbitrary shapes via PDE layers, enhancing segmentation performance and reducing model size.

Findings

ST-DNN are covariant to translations and rotations.

They are 3-4 orders smaller than traditional CNNs.

They outperform state-of-the-art CNN descriptors in segmentation tasks.

Abstract

We present Shape-Tailored Deep Neural Networks (ST-DNN). ST-DNN extend convolutional networks (CNN), which aggregate data from fixed shape (square) neighborhoods, to compute descriptors defined on arbitrarily shaped regions. This is natural for segmentation, where descriptors should describe regions (e.g., of objects) that have diverse shape. We formulate these descriptors through the Poisson partial differential equation (PDE), which can be used to generalize convolution to arbitrary regions. We stack multiple PDE layers to generalize a deep CNN to arbitrary regions, and apply it to segmentation. We show that ST-DNN are covariant to translations and rotations and robust to domain deformations, natural for segmentation, which existing CNN based methods lack. ST-DNN are 3-4 orders of magnitude smaller then CNNs used for segmentation. We show that they exceed segmentation performance compared to state-of-the-art CNN-based descriptors using 2-3 orders smaller training sets on the texture segmentation problem.