Localization: A Missing Link in the Pipeline of Object Matching and Registration

TL;DR

This paper introduces a CNN-based framework for image registration that incorporates object localization, segmentation, and matching, improving accuracy and generalization in real-world scenarios like medical imaging.

Contribution

It presents a novel combination of localization, segmentation, and spatial transformer networks for robust image registration, especially in complex and real-world data.

Findings

Achieved 79% dice coefficient on MNIST dataset.

Achieved 94% dice coefficient on Caltech-101 dataset.

Extended framework successfully registered CT and US images.

Abstract

Image registration is a process of aligning two or more images of same objects using geometric transformation. Most of the existing approaches work on the assumption of location invariance. These approaches require object-centric images to perform matching. Further, in absence of intensity level symmetry between the corresponding points in two images, the learning based registration approaches rely on synthetic deformations, which often fail in real scenarios. To address these issues, a combination of convolutional neural networks (CNNs) to perform the desired registration is developed in this work. The complete objective is divided into three sub-objectives: object localization, segmentation and matching transformation. Object localization step establishes an initial correspondence between the images. A modified version of single shot multi-box detector is used for this purpose. The detected region is cropped to make the images object-centric. Subsequently, the objects are segmented and matched using a spatial transformer network employing thin plate spline deformation. Initial experiments on MNIST and Caltech-101 datasets show that the proposed model is able to produce accurate matching. Quantitative evaluation performed using dice coefficient (DC) and mean intersection over union (mIoU) show that proposed method results in the values of 79% and 66%, respectively for MNIST dataset and the values of 94% and 90%, respectively for Caltech-101 dataset. The proposed framework is extended to the registration of CT and US images, which is free from any data specific assumptions and has better generalization capability as compared to the existing rule-based/classical approaches.