Multi-view analysis of unregistered medical images using cross-view transformers

TL;DR

This paper introduces a cross-view transformer approach for multi-view medical image analysis that effectively combines unregistered views at the spatial feature map level, improving over traditional global pooling methods.

Contribution

The paper proposes a novel cross-view transformer technique that enables spatial feature map interaction for unregistered multi-view images, enhancing analysis accuracy.

Findings

Outperforms baseline global pooling methods on mammography datasets

Effective in handling unregistered multi-view images

Demonstrates improved multi-view image analysis performance

Abstract

Multi-view medical image analysis often depends on the combination of information from multiple views. However, differences in perspective or other forms of misalignment can make it difficult to combine views effectively, as registration is not always possible. Without registration, views can only be combined at a global feature level, by joining feature vectors after global pooling. We present a novel cross-view transformer method to transfer information between unregistered views at the level of spatial feature maps. We demonstrate this method on multi-view mammography and chest X-ray datasets. On both datasets, we find that a cross-view transformer that links spatial feature maps can outperform a baseline model that joins feature vectors after global pooling.