PZnet: Efficient 3D ConvNet Inference on Manycore CPUs

TL;DR

PZnet is a CPU-optimized engine for efficient 3D convolutional neural network inference, outperforming existing CPU and GPU implementations in biomedical volumetric data analysis.

Contribution

The paper introduces PZnet, a novel CPU-only inference engine for 3D ConvNets that significantly improves performance and cost efficiency over existing solutions.

Findings

PZnet outperforms MKL-based PyTorch and TensorFlow by over 3.5x.

For low feature map 3D convolutions, CPU inference with PZnet is more cost-effective than GPU.

PZnet enables practical deployment of 3D ConvNets on limited-resource systems.

Abstract

Convolutional nets have been shown to achieve state-of-the-art accuracy in many biomedical image analysis tasks. Many tasks within biomedical analysis domain involve analyzing volumetric (3D) data acquired by CT, MRI and Microscopy acquisition methods. To deploy convolutional nets in practical working systems, it is important to solve the efficient inference problem. Namely, one should be able to apply an already-trained convolutional network to many large images using limited computational resources. In this paper we present PZnet, a CPU-only engine that can be used to perform inference for a variety of 3D convolutional net architectures. PZNet outperforms MKL-based CPU implementations of PyTorch and Tensorflow by more than 3.5x for the popular U-net architecture. Moreover, for 3D convolutions with low featuremap numbers, cloud CPU inference with PZnet outperfroms cloud GPU inference in terms of cost efficiency.