Particle Swarm Optimization in 3D Medical Image Registration: A Systematic Review

TL;DR

This paper systematically reviews the application of Particle Swarm Optimization (PSO) techniques for 3D medical image registration, highlighting their effectiveness in optimizing similarity metrics for aligning anatomical structures.

Contribution

It provides a comprehensive overview of PSO-based methods in 3D medical image registration, emphasizing recent advancements and challenges.

Findings

PSO methods improve registration accuracy in 3D medical imaging

Enhanced convergence speed with hybrid PSO approaches

Identification of key challenges in PSO application for medical images

Abstract

Medical image registration seeks to find an optimal spatial transformation that best aligns the underlying anatomical structures. These problems usually require the optimization of a similarity metric. Swarm Intelligence techniques are very effective and efficient optimization methods. This systematic review focuses on 3D medical image registration using Particle Swarm Optimization

Tables

Table 1. Table 1: Selected papers

Authors	Modalities		Anatom.	Dataset	PSO	Corresp.	Similarity
			Region		Type		Metric
Talbi and Batouche	MRI	CT	brain	private	DEPSO	intensity	MI
Talbi2004	MRI	RX
	MRI	SPECT
	PET	MRI
Wachowiak et al. Wachowiak2004	US	histology	abdomen	NLM-NIH	HPSO	intensity	NMI
	CT	histology	brain	BrainWeb
	MRI_T1	MRI_T2
Li et al. Li2008	MRI	CT	brain	private	PSO	points	LTS-HD
Chen and Mimori ChenMimori2009	MRI	CT	brain	RIRE	HPSO	intensity	MI
Chen et al. Chen2009	MRI	CT	brain	RIRE	HPSO	intensity	MI
Li et al. Li2010	CT	CT	brain	private	NCQPSO	intensity	JS
	MRI	CT
Zhang et al. Zhang2010	CT	CT	brain	private	PSO	intensity	GMI
	CT	MRI
Bao and Sun Bao2011	CT	MRI	brain	RIRE	RQPSO	intensity	MI
	PET	MRI
Zhou et al. Zhou2011	CT	MRI	brain	RIRE	RQPSO,	intensity	NMI
	PET	MRI			DRQPSO
Ayatolahi et al. Ayatollahi2012	CT	CT	brain	AANLIB	HPSO	intensity	MNMI
	MRI_T2	MRI_PD
	MRI_T2	CT
Lin et al. Lin2012	MRI	CT	brain	RIRE	HPSO	intensity	MI
Kang et al. Kang2014	2D X-rays	3D X-rays	femur	FTRAC	EM-PSO	geometry	MLE
Schwab et al. Schwab2015	CT	MRI	brain	RIRE	4 PSO var	intensity	NMI
Wang et al. Wang2015	MRI_T2	MRI_T2	cardiac	private	PSO	intensity	MI
	MRI_T2	MRI_DE
	MRI_T1	MRI_T2	neonatal
Hui and Zhijun Hui2015	PET	MRI	brain	RIRE	QPSO	intensity	MI
Manoj et al. Manoj2016	CT	MRI	brain	private	BAT-PSO	mixed	MI
Hering et al. Hering2016	MRI	MRI	brain	private	MOPSO	intensity	TRE
Hernandez-Matas et al.	fundus	fundus	retina	private	8 PSO var	SIFT	ED
Hernandez-Matas2017	camera	camera
Li et al. Li2017	CT	US	femur	private	PSO	H-shaped im	MI
Zaman and Ko Zaman2018	3D CT	2D X-rays	femur	private	PSO var	intensity	ED
Wang et al. Wang2019	CT	MRI_T2	brain	RIRE	UKFPSO	intensity	TRE
	PET	MRI_PD	brain	RIRE
	MRI_T1	MRI_T2	neonatal	private
Liu et al. Liu2021	CBCT	CBCT	dental	private	PSO	intensity	GMM-NCC
Yoon et al. Yoon2021	CT	2D X-rays	artery	private	MT-PSO	centerlines	CBKnn, CBO
Shao et al. Shao2022	CT	CT	multiple	private	PSO	points	ED

Equations

$$V(t+1)=wV(t)+c_1{r}_1(PBest(t)-P(t))+c_2{r}_2(GBest(t)-P(t))$$

$$P(t+1)=P(t)+V(t+1)$$

Full text

1 Introduction

In medical imaging, you may need to compare scans of multiple patients or scans of the same patient, using mono-modal or multi-modal images taken in different sessions Hill_2001 . Moreover, images taken from different modalities provides complementary information. To accomplish this, we need to align multiple medical images, volumes, or surfaces to a common coordinate system. This process is called Image Registration.

Huge number of algorithms are available for image registration. Indeed several review papers on this topic have been published. A simple search on PubMed for Review articles using the terms ”Image Registration” gives over 1000 results. The field of medical image registration has been evolving rapidly with hundreds of papers published each year. Though medical image registration has been extensively studied, it remains a hot research topic.

Medical image registration seeks to find an optimal spatial transformation that best aligns the underlying anatomical structures. It usually requires the optimization of some similarity metric between the images. Among the optimization methods, the techniques based on Swarm Intelligence represent an efficient solution.

The aim of this paper is to provide a comprehensive systematic review on medical image registration using Particle Swarm Optimization (PSO), which is a population-based stochastic optimization algorithm Kennedy1995 . This study concentrates on 3D medical images. Though a few review papers on medical image registration using PSO have been published Rundo2017 ; Saiti2020 , to the best of our knowledge this is the first systematic review on the subject.

2 Medical Image Registration

Medical image registration is the process of aligning or registering two or more medical images of the same or different modalities so that they can be compared and analyzed. This is often done to compare images taken at different times or from different viewpoints, or to combine information from multiple images to create a more detailed or accurate image.

There are several approaches to medical image registration, including intensity-based methods, feature-based methods, and deformable registration. Intensity-based methods use the intensity values of the pixels in the images to register the images, while feature-based methods use distinctive features in the images, such as edges or landmarks, to register the images. Deformable registration involves warping one image to match the other, and can be used when there are large differences between the images that cannot be compensated for by simple translations or rotations.

Medical image registration has a wide range of applications, including image-guided surgery, treatment planning, disease diagnosis and monitoring, and research. It is an important tool in medical imaging and has the potential to improve patient care and outcomes.

There have been many research papers published on the topic of medical image registration. Several surveys have been published. For an extensive review of medical image registration techniques, we refer to Maintz and Viergever Maintz1998 and Zitová and Flusser Zitova2003 .

Intensity-based image registration methods use the intensity values of the pixels in the images to register the images. One common approach is to use Mutual Information (MI), which is a measure of the statistical dependence between the intensity values in the two images. Normalized Mutual Information (NMI) improves the robustness of MI by avoiding some mis-registrations by being independent of overlapping areas of the two datasets.

Feature-based Registration methods aim to find the transformation that minimizes the distance between the features extracted from the datasets to be aligned. The features are geometrical entities, with the most commonly used ones being points, lines or contours Saiti2020 .

3 Particle Swarm Optimization

The Particle Swarm Optimization (PSO) algorithm is a population-based optimization algorithm, proposed by Kennedy and Eberhart in 1995 Kennedy1995 . It is inspired by the behavior of swarms in nature. It works by iteratively improving a candidate solution by adjusting the values of the variables in the solution based on the experiences of the ”particles” in the swarm.

The algorithm consists of a population of particles, each of which represents a potential solution to the optimization problem. Each particle has a position in the search space and a velocity, which determines how the particle moves through the search space. The particles are initialized at random positions in the search space, and the velocity of each particle is initialized to a small value.

At each iteration of the algorithm, each particle updates its position and velocity based on its current position, the best position it has found so far (referred to as the ”personal best” position), and the best position found by any particle in the swarm (referred to as the ”global best” position). The position and velocity of each particle are updated according to the following equations:

[TABLE]

[TABLE]

where: $V$ is the velocity, $P$ is the position of the particle, $t$ is the current iteration, $w$ is the inertia weight, which determines how much the particle’s current velocity affects its new velocity, $c_{1}$ and $c_{2}$ are constants that control the influence of the personal best and global best positions on the particle’s velocity, $r_{1}$ and $r_{2}$ are random numbers between 0 and 1, $PBest(t)$ is the best position the particle has found so far, $GBest(t)$ is the best position found by any particle in the swarm.

The algorithm continues until a stopping criterion is met, such as a maximum number of iterations or a satisfactory solution being found.

PSO is a simple and effective optimization algorithm that has been applied to a wide range of problems in various fields. It is easy to implement and can find solutions quickly, making it a popular choice for many optimization problems.

Several variants of the original PSO has been proposed. In this review we will investigate if the standard PSO algorithm has been used or its variants.

See Nayak et al. Nayak2022 for an in-depth analysis of PSO, its variants and its applications, and Gad Gad2022 for a systemtic review on PSO, that also outlines previous review studies.

4 Methodology and Results

We applied the systematic review methodology of Brereton et al. Brereton2007 . In this section we describe how we planned, conducted and analyzed results.

4.1 Planning the review

During this phase, we identified the needs for this literature review. We reviewed the state-of-the-art existing literature in this area. The research questions were generated accordingly, and they are listed as follows:

1. RQ1:

What is the brief overview of current research? 2. RQ2:

What are the extensively applied approaches? 3. RQ3:

How the work is distributed according to time division? 4. RQ4:

What are the address registration problems? 5. RQ5:

What are the applied algorithms?

4.2 Conducting the review

We defined the inclusion and exclusion criteria:

Papers must focus on PSO for 3D medical image registration. 2. 2.

Papers must be written in English. 3. 3.

Papers must describe the methodology. Review papers are excluded. 4. 4.

Only published or peer-reviewed works are included. Dissertations and theses are excluded. 5. 5.

Papers where PSO is not used for registration (i.e. for segmentation, feature extraction or classification) are excluded. 6. 6.

Papers not on medical applications or not on human are excluded. 7. 7.

Low quality papers, i.e. missing parameters or not reproducible are excluded.

We conducted the search in December 2022, using the following digital databases:

PubMed 2. 2.

IEEExplore 3. 3.

Elsevier Science Direct 4. 4.

Springer Link 5. 5.

Google Scholar

The search string was ((”Particle Swarm Optimization” OR ”PSO”) AND ”3D” AND ”medical” AND ”image registration”).

After removing duplicated, we examined title, abstract and keywords to determine if the papers fulfill the inclusion and exclusion criteria. Finally, we assessed and read the papers in full for eligibility and data extraction.

5 Results

We obtained 2830 articles from Google Scholar, 261 from Elsevier Science Direct, 58 from IEEExplore, 228 from Springer Link, 21 from PubMed. After duplicates removal and screening based on title, abstract and keywords, 92 papers were assessed in full according the inclusion/exclusion criteria listed in Sec. 4.2.

No time range was applied during the search in order to include as many search results as possible. Although PSO was introduced by Kennedy et al. Kennedy1995 over 25 years ago, our review only managed to identify 24 relevant papers using PSO to optimize 3D medical image registration (see Table 1). Many papers were excluded because they described only 2D and not 3D registration. From papers, we retrieved the following information:

Modalities used for the registration 2. 2.

Anatomical Region 3. 3.

Dataset used 4. 4.

PSO type (standard PSO, hybrid or variants) 5. 5.

Correspondence (intensity or feature based) 6. 6.

Similarity Metric

We also extracted the performance of the various methods, but given the many different evaluation measures used, we could not compare the proposed methods.

5.1 RQ1: Brief Overview of Ongoing Research

The selected literature for this systematic review is summarised in Table 1. By analysing it we will overview the PSO techniques that are applied to register 3D medical images, as well as which are the common image modalities that are used in the selected papers.

5.2 RQ2: Extensively Used Approaches

The wordcloud shown in Figure 1 is generated on the basis of terms enclosed in the titles of selected research articles. The size of each term demonstrates the number of their occurrence. According to the size of terms in Figure 1, it can be observed that Particle Swarm Optimization, Medical Image Registration are the predominant words that frequently appear in the targeted research articles.

5.3 RQ3: Literature Distribution with Time

Division

The graph in Figure 2 illustrates the distribution of selected papers with respect to the time division. It shows that the number of papers published each year is quite constant.

5.4 RQ4: Addressed Registration Problems

Analysing Table 1 some useful conclusions can be extracted: 67% of the selected papers present examples of multimodal registration, 21% of monomodal registration, and 12% describe examples both monomodal and multimodal registration (see Figure 3). In Figure 3 we can also observe which modalities are frequently used. The multimodal registration of Magnetic Resonance Image (MRI) and Computer Tomography (CT) is most commonly studied (30%), followed by MRI and Positron Emission Tomography (PET) (14%), and by different modalities of MRI (14%), for instance T1-weighted and T2-weighted. For single mode registration CT is the most frequently used modalities (16%). Only one or two studies used other modalities, like fundus camera, Ultrasounds (US), histology or X-rays.

Images of various anatomical regions have been studied, beeing the brain (59%) the top one in terms of popularity (see Figure 4). Regarding the datasets upon which experiments were conducted by the presented techniques, it should be highlighted that 55% are private while 45% are publicly available (see Figure 4). The use of different datasets, makes comparison between the different approaches hard. The publicly available datasets are: RIRE West1997 ; RIRE , AANLIB AANLIB , BrainWeb Kwan1999 ; BrainWeb , FTRAC FTRAC , NLM-NIH NLM-NIH .

5.5 RQ5: Applied PSO Algoritms

The applied PSO algorithms can be seen in Figure 5, where we observe that 25% of researchers applied Standard PSO and the remaining improved of hybrid version of PSO. Some researchers named their version, ad therefore we reported the given name in the Table and in the Figure, while others simply called it Hybrid PSO (HPSO) or PSO variant. The Quantum Behaved PSO (QPSO) and its modification (Niche Chaotic mutation QPSO (NCQPSO), Revised QPSO (RQPSO) and Diversity Revised QPSO (DRQPSO)) is the most popular one (17%), while other hybrid versions of PSO often incorporate elements from Genetic Algorithms, like Crossover or subpopulations.

According to the type of correspondence, among the selected papers 71% of registration methods are intensity-based and 29% are feature-based.

Various similarity measures have been used as objective function of the PSO algorithm (see Figure 5), being Mutual Information (MI) including its variants (Normalized MI (NMI), Modified Normalized MI (MNMI) and Generalized MI (GMI)) the most popular (59%); Euclidean Distance is mainly used for feature based registration. Target Registration Error (TRE) Fitzpatrick2001 and other similarity metrics are also adopted.

6 Conclusions

We presented a systematic review of existing studies on the application of the standard PSO and its variants to the problem of 3D medical image registration. In order to perform the systematic review, the gaps in the literature are figured out and converted into five research questions. The findings of this systematic review depict that some researchers applied PSO and its hybrid versions to register anatomical regions using different modalities of medical images as MRI, CT and more.