Analysis of Information Flow Through U-Nets

TL;DR

This paper uses information-theoretic methods to analyze how information flows through U-Net architectures, providing insights into their efficiency and guiding potential improvements for image segmentation tasks.

Contribution

It introduces the application of mutual information analysis to U-Nets, revealing their information flow characteristics and suggesting ways to optimize their design.

Findings

Mutual information can effectively measure information flow in U-Nets.

Analysis identifies bottlenecks and inefficiencies in current U-Net architectures.

Proposed design modifications improve information retention and network efficiency.

Abstract

Deep Neural Networks (DNNs) have become ubiquitous in medical image processing and analysis. Among them, U-Nets are very popular in various image segmentation tasks. Yet, little is known about how information flows through these networks and whether they are indeed properly designed for the tasks they are being proposed for. In this paper, we employ information-theoretic tools in order to gain insight into information flow through U-Nets. In particular, we show how mutual information between input/output and an intermediate layer can be a useful tool to understand information flow through various portions of a U-Net, assess its architectural efficiency, and even propose more efficient designs.