Pseudo-Labeling Enhanced by Privileged Information and Its Application to In Situ Sequencing Images

TL;DR

This paper introduces PLePI, a semi-supervised learning framework that leverages privileged information to improve object detection in biological images, demonstrated on in situ sequencing images and benchmark datasets.

Contribution

It proposes a general pseudo-labeling method enhanced by privileged information, tailored for biological vision applications where ground truth is unavailable.

Findings

Improved decoding accuracy in spatial transcriptomics

Effective use of privileged information in pseudo-labeling

Enhanced performance on COCO benchmark with extra evidence

Abstract

Various strategies for label-scarce object detection have been explored by the computer vision research community. These strategies mainly rely on assumptions that are specific to natural images and not directly applicable to the biological and biomedical vision domains. For example, most semi-supervised learning strategies rely on a small set of labeled data as a confident source of ground truth. In many biological vision applications, however, the ground truth is unknown and indirect information might be available in the form of noisy estimations or orthogonal evidence. In this work, we frame a crucial problem in spatial transcriptomics - decoding barcodes from In-Situ-Sequencing (ISS) images - as a semi-supervised object detection (SSOD) problem. Our proposed framework incorporates additional available sources of information into a semi-supervised learning framework in the form of privileged information. The privileged information is incorporated into the teacher's pseudo-labeling in a teacher-student self-training iteration. Although the available privileged information could be data domain specific, we have introduced a general strategy of pseudo-labeling enhanced by privileged information (PLePI) and exemplified the concept using ISS images, as well on the COCO benchmark using extra evidence provided by CLIP.