CoCo DistillNet: a Cross-layer Correlation Distillation Network for Pathological Gastric Cancer Segmentation

TL;DR

CoCo DistillNet introduces a cross-layer correlation knowledge distillation method combined with adversarial learning and an unsupervised paraphraser to improve gastric cancer segmentation accuracy with compact models.

Contribution

The paper proposes a novel cross-layer correlation distillation framework with adversarial learning and paraphrasing to enhance compact network performance in pathology image segmentation.

Findings

Achieves state-of-the-art segmentation performance on gastric cancer dataset.

Effectively transfers knowledge from cumbersome to compact networks.

Improves model efficiency without sacrificing accuracy.

Abstract

In recent years, deep convolutional neural networks have made significant advances in pathology image segmentation. However, pathology image segmentation encounters with a dilemma in which the higher-performance networks generally require more computational resources and storage. This phenomenon limits the employment of high-accuracy networks in real scenes due to the inherent high-resolution of pathological images. To tackle this problem, we propose CoCo DistillNet, a novel Cross-layer Correlation (CoCo) knowledge distillation network for pathological gastric cancer segmentation. Knowledge distillation, a general technique which aims at improving the performance of a compact network through knowledge transfer from a cumbersome network. Concretely, our CoCo DistillNet models the correlations of channel-mixed spatial similarity between different layers and then transfers this knowledge from a pre-trained cumbersome teacher network to a non-trained compact student network. In addition, we also utilize the adversarial learning strategy to further prompt the distilling procedure which is called Adversarial Distillation (AD). Furthermore, to stabilize our training procedure, we make the use of the unsupervised Paraphraser Module (PM) to boost the knowledge paraphrase in the teacher network. As a result, extensive experiments conducted on the Gastric Cancer Segmentation Dataset demonstrate the prominent ability of CoCo DistillNet which achieves state-of-the-art performance.