Semi-Supervised Learning for Image Classification using Compact Networks in the BioMedical Context

TL;DR

This paper demonstrates that semi-supervised learning can significantly improve the accuracy of compact neural networks for biomedical image classification, enabling lightweight models to match larger models' performance.

Contribution

It introduces a comprehensive analysis of semi-supervised techniques applied to various compact networks in biomedical imaging, showing improved accuracy and efficiency.

Findings

Semi-supervised learning enhances compact network performance.

Data distillation with MixNet yields top results.

NAS networks outperform manually designed and quantized models.

Abstract

The development of mobile and on the edge applications that embed deep convolutional neural models has the potential to revolutionise biomedicine. However, most deep learning models require computational resources that are not available in smartphones or edge devices; an issue that can be faced by means of compact models. The problem with such models is that they are, at least usually, less accurate than bigger models. In this work, we study how this limitation can be addressed with the application of semi-supervised learning techniques. We conduct several statistical analyses to compare performance of deep compact architectures when trained using semi-supervised learning methods for tackling image classification tasks in the biomedical context. In particular, we explore three families of compact networks, and two families of semi-supervised learning techniques for 10 biomedical tasks. By combining semi-supervised learning methods with compact networks, it is possible to obtain a similar performance to standard size networks. In general, the best results are obtained when combining data distillation with MixNet, and plain distillation with ResNet-18. Also, in general, NAS networks obtain better results than manually designed networks and quantized networks. The work presented in this paper shows the benefits of apply semi-supervised methods to compact networks; this allow us to create compact models that are not only as accurate as standard size models, but also faster and lighter. Finally, we have developed a library that simplifies the construction of compact models using semi-supervised learning methods.