Magnifying Networks for Images with Billions of Pixels

TL;DR

Magnifying Networks (MagNets) enable efficient, end-to-end analysis of gigapixel images by dynamically retrieving image regions at various magnifications without preprocessing, significantly reducing the number of patches needed.

Contribution

Introduction of MagNets, a novel deep learning approach that processes extremely high-resolution images without patch preprocessing, using minimal ground truth labels.

Findings

MagNets outperform existing methods in accuracy on Camelyon datasets.

MagNets process 10 to 300 times fewer patches than traditional approaches.

Effective for whole slide image classification with minimal ground truth.

Abstract

The shift towards end-to-end deep learning has brought unprecedented advances in many areas of computer vision. However, deep neural networks are trained on images with resolutions that rarely exceed $1,000 \times 1,000$ pixels. The growing use of scanners that create images with extremely high resolutions (average can be $100,000 \times 100,000$ pixels) thereby presents novel challenges to the field. Most of the published methods preprocess high-resolution images into a set of smaller patches, imposing an a priori belief on the best properties of the extracted patches (magnification, field of view, location, etc.). Herein, we introduce Magnifying Networks (MagNets) as an alternative deep learning solution for gigapixel image analysis that does not rely on a preprocessing stage nor requires the processing of billions of pixels. MagNets can learn to dynamically retrieve any part of a gigapixel image, at any magnification level and field of view, in an end-to-end fashion with minimal ground truth (a single global, slide-level label). Our results on the publicly available Camelyon16 and Camelyon17 datasets corroborate to the effectiveness and efficiency of MagNets and the proposed optimization framework for whole slide image classification. Importantly, MagNets process far less patches from each slide than any of the existing approaches ($10$ to $300$ times less).