CASS: Cross Architectural Self-Supervision for Medical Image Analysis

TL;DR

CASS introduces a self-supervised learning method combining Transformer and CNN architectures, significantly improving medical image analysis performance and robustness while reducing computational costs across diverse datasets.

Contribution

The paper proposes a novel self-supervised approach, CASS, that leverages both Transformer and CNN models simultaneously, enhancing efficiency and robustness in medical image analysis.

Findings

CASS improves accuracy by up to 10.13% with full labeled data.

CASS reduces training time by 69% compared to existing methods.

CASS is more robust to batch size and epoch variations.

Abstract

Recent advances in deep learning and computer vision have reduced many barriers to automated medical image analysis, allowing algorithms to process label-free images and improve performance. However, existing techniques have extreme computational requirements and drop a lot of performance with a reduction in batch size or training epochs. This paper presents Cross Architectural - Self Supervision (CASS), a novel self-supervised learning approach that leverages Transformer and CNN simultaneously. Compared to the existing state of the art self-supervised learning approaches, we empirically show that CASS-trained CNNs and Transformers across four diverse datasets gained an average of 3.8% with 1% labeled data, 5.9% with 10% labeled data, and 10.13% with 100% labeled data while taking 69% less time. We also show that CASS is much more robust to changes in batch size and training epochs. Notably, one of the test datasets comprised histopathology slides of an autoimmune disease, a condition with minimal data that has been underrepresented in medical imaging. The code is open source and is available on GitHub.