Training Multiscale-CNN for Large Microscopy Image Classification in One Hour

TL;DR

This paper demonstrates training a multiscale CNN for large biomedical image classification within one hour by leveraging CPU memory capacity, large batch sizes, and a linear learning rate scaling, achieving state-of-the-art accuracy.

Contribution

It introduces a methodology to efficiently train large-scale CNNs on CPUs, enabling rapid training of high-accuracy models without image cropping or down-sampling.

Findings

Achieved 99% accuracy within one hour.

Utilized large batch sizes with linear learning rate scaling.

Demonstrated effective training on CPU clusters with high memory capacity.

Abstract

Existing approaches to train neural networks that use large images require to either crop or down-sample data during pre-processing, use small batch sizes, or split the model across devices mainly due to the prohibitively limited memory capacity available on GPUs and emerging accelerators. These techniques often lead to longer time to convergence or time to train (TTT), and in some cases, lower model accuracy. CPUs, on the other hand, can leverage significant amounts of memory. While much work has been done on parallelizing neural network training on multiple CPUs, little attention has been given to tune neural network training with large images on CPUs. In this work, we train a multi-scale convolutional neural network (M-CNN) to classify large biomedical images for high content screening in one hour. The ability to leverage large memory capacity on CPUs enables us to scale to larger batch sizes without having to crop or down-sample the input images. In conjunction with large batch sizes, we find a generalized methodology of linearly scaling of learning rate and train M-CNN to state-of-the-art (SOTA) accuracy of 99% within one hour. We achieve fast time to convergence using 128 two socket Intel Xeon 6148 processor nodes with 192GB DDR4 memory connected with 100Gbps Intel Omnipath architecture.