Encoder-Only Image Registration

TL;DR

This paper introduces EOIR, an encoder-only neural network framework for deformable image registration that balances accuracy, efficiency, and smoothness by separating feature learning from flow estimation.

Contribution

The paper proposes the Encoder-Only Image Registration (EOIR) framework, utilizing a simple ConvNet architecture to improve registration accuracy and efficiency for large deformations.

Findings

EOIR achieves superior accuracy-efficiency trade-offs across multiple datasets.

EOIR provides better smoothness and efficiency with comparable accuracy.

The approach effectively handles large deformations in medical images.

Abstract

Learning-based techniques have significantly improved the accuracy and speed of deformable image registration. However, challenges such as reducing computational complexity and handling large deformations persist. To address these challenges, we analyze how convolutional neural networks (ConvNets) influence registration performance using the Horn-Schunck optical flow equation. Supported by prior studies and our empirical experiments, we observe that ConvNets play two key roles in registration: linearizing local intensities and harmonizing global contrast variations. Based on these insights, we propose the Encoder-Only Image Registration (EOIR) framework, designed to achieve a better accuracy-efficiency trade-off. EOIR separates feature learning from flow estimation, employing only a 3-layer ConvNet for feature extraction and a set of 3-layer flow estimators to construct a Laplacian feature pyramid, progressively composing diffeomorphic deformations under a large-deformation model. Results on five datasets across different modalities and anatomical regions demonstrate EOIR's effectiveness, achieving superior accuracy-efficiency and accuracy-smoothness trade-offs. With comparable accuracy, EOIR provides better efficiency and smoothness, and vice versa. The source code of EOIR is publicly available on https://github.com/XiangChen1994/EOIR.

Tables

Table 1. TABLE I: Quantitative comparison on the abdomen CT dataset. Best-performing metrics are in bold. Symbols indicate direction: ↑ \uparrow for higher is better, ↓ \downarrow for lower is better. “Initial” refers to baseline results before registration.

Model	Dice (%) $\uparrow$	HD95 (mm) $\downarrow$	SDlogJ $\downarrow$
Initial	30.86	29.77	-
VoxelMorph [5]	47.05	23.08	0.13
TransMorph [8]	47.94	21.53	0.13
LKUNet [36]	52.78	20.56	0.98
LapIRN [55]	54.55	20.52	1.73
CorrMLP [54]	56.11	19.52	0.16
RDP [75]	58.77	20.07	0.22
MemWarp [81]	60.24	19.84	0.53
LessNet [37]	42.03	27.03	0.07
FourierNet [35]	42.80	22.95	0.13
ConvexAdam [63]	51.10	23.14	0.11
SAMConvex [43]	53.65	18.66	0.12
VoxelOpt [85]	58.51	18.54	0.21
uniGradICON [70]	53.33	20.20	0.13
EOIR (-MS loss)	59.57	18.98	0.29
EOIR (-M3)	60.12	18.46	0.17
EOIR (-M4)	59.60	19.04	0.17
EOIR (Ours)	60.63	17.61	0.17

Table 2. TABLE II: Quantitative comparison on the cardiac ACDC dataset. Best-performing metrics are highlighted in bold. “MAs (G)” stands for multi-adds (G), and “PS (MB)” is parameter size (MB). “Time” is the average inference time for 1 registration pair. GM(MB) is the occupied GPU memory in inference.

Model	Dice (%) $\uparrow$	HD95 (mm) $\downarrow$	SDlogJ $\downarrow$	MAs (G) $\downarrow$	PS (MB) $\downarrow$	Time $\downarrow$	GM(GB) $\downarrow$
Initial	58.14	11.95	-	-	-	-	-
TransMorph [8]	74.97	9.44	0.045	50.20	46.69	0.26	18.3
VoxelMorph [5]	75.26	9.33	0.044	19.55	0.32	0.18	2.7
LKU-Net [36]	76.53	9.13	0.049	160.50	33.35	0.22	3.9
Fourier-Net [35]	76.61	9.25	0.047	86.07	17.43	0.27	3.1
CorrMLP [54]	77.31	9.00	0.056	47.59	4.19	0.28	3.3
MemWarp [81]	76.74	9.67	0.108	1270.00	47.78	0.58	12.7
RDP [75]	78.06	9.02	0.076	154.00	8.92	0.36	4.1
EOIR ($N_s=8$)	78.28	9.14	0.071	11.02	0.11	0.25	2.9
EOIR (Ours)	78.91	9.07	0.084	114.21	0.91	0.26	4.5

Table 3. TABLE III: Quantitative comparison on the OASIS dataset. “TransMorph-1” and “TransMorph-2” denote versions with different smoothness regularization.

Model	Dice (%) $\uparrow$	HD95 (mm) $\downarrow$	SDlogJ $\downarrow$
Initial	57.18	3.83	-
VoxelMorph [5]	84.70	1.55	0.13
TransMorph-1 [8]	86.20	1.43	0.13
TransMorph-2 [8]	88.54	1.27	0.50
LKUNet [36]	88.61	1.26	0.52
LessNet [37]	78.80	2.15	0.10
FourierNet [35]	86.04	1.37	0.48
LapIRN [55]	86.10	1.51	0.07
ConvexAdam [63]	84.64	1.50	0.07
EOIR ($N_s$=16)	86.96	1.38	0.28
EOIR (Ours)	88.83	1.28	0.52

Table 4. TABLE IV: Quantitative comparison on the LUMIR dataset. “EOIR (addition)” refers to using addition to compose fields.

Model	Dice (%) $\uparrow$	HD95 (mm) $\downarrow$	NDV (%) $\downarrow$	TRE (mm) $\downarrow$
Initial	56.57	4.79	-	-
DeedsBCV [25]	69.77	3.95	0.000	2.22
SynthMorph [27]	72.43	3.57	0.000	2.61
VoxelMorph [5]	73.25	3.76	0.397	2.69
TransMorph [8]	75.94	3.51	0.351	2.42
EOIR (addition)	77.34	3.34	0.186	2.37
EOIR (Ours)	77.37	3.33	0.000	2.35

Table 5. TABLE V: Quantitative comparison on the HippocampusMR dataset.

Model	Dice (%) $\uparrow$	HD95 (mm) $\downarrow$	SDlogJ $\downarrow$	Time $\downarrow$
Initial	62.46	12.39	-
VoxelMorph [5]	80.97	7.90	0.06	0.19
TransMorph [8]	84.90	6.19	0.07	0.49
RDP [75]	86.13	6.02	0.07	0.46
CorrMLP [54]	84.86	6.55	0.07	0.29
LessNet [37]	75.59	9.28	0.05	0.30
FourierNet [35]	80.10	7.59	0.06	0.33
LKUNet [36]	75.09	9.29	0.04	0.39
MemWarp [81]	86.50	5.71	0.24	0.96
EOIR (Ours)	86.44	6.20	0.10	0.35

Table 6. TABLE VI: Quantitative comparison on the RGB-IR dataset.

Model	nMI (%) $\uparrow$	NCC (%) $\uparrow$	PSNR $\uparrow$	PS (MB) $\downarrow$
Initial	-	47.52	22.96	-
SIFT [46]	36.97	42.63	23.10	-
ReCoNet [31]	93.40	63.71	25.01	3.09
Superfusion [68]	95.29	87.70	29.98	6.96
IMF [74]	93.95	74.77	26.42	27.13
PGMR [93]	95.65	87.61	29.55	1670.16
EOIR (Ours)	95.17	88.17	30.23	1.50

Table 7. TABLE VII: Comprehensive Summary of Experimental Setup.

Training Paradigm	Dataset	Configuration & Baselines
Unsupervised	ACDC	Loss: NCC + $ℛ$ (1:1), Baselines: VoxelMorph, TransMorph, LKUNet, FourierNet, CorrMLP, RDP, MemWarp
	LUMIR	Loss: NCC + $ℛ$ (1:1), Baselines: VoxelMorph, TransMorph, SynthMorph, DeedsBCV
	RGB-IR	Loss: L1 + $L_p$ + $ℛ$ (1:1:0.01), Baselines: SIFT, ReCoNet, Superfusion, IMF, PGMR
	ThoraxCBCT	Loss: NCC + $ℛ$ (1:1), Baselines: VoxelMorph++, deeds
Weakly-supervised	Abdomen CT	Loss: NCC + Dice + $ℛ$ (1:1:1), Baselines: VoxelMorph, TransMorph, LKUNet, LapIRN, CorrMLP, RDP, MemWarp, LessNet, FourierNet, ConvexAdam, SAMConvex, VoxelOpt
	OASIS	Loss: NCC + Dice + $ℛ$ (1:1:1), Baselines: VoxelMorph, TransMorph, LKUNet, LapIRN, LessNet, FourierNet, ConvexAdam
	HippocampusMR	Loss: NCC + Dice + $ℛ$ (1:1:1), Baselines: VoxelMorph, TransMorph, LKUNet, CorrMLP, RDP, MemWarp, LessNet, FourierNet
Zero-Shot	ADNI	Loss: N/A, Baselines: All methods in the LUMIR challenge
	NIMH	Loss: N/A, Baselines: All methods in the LUMIR challenge
	UltraCortex	Loss: N/A, Baselines: All methods in the LUMIR challenge

Table 8. TABLE VIII: Quantitative comparison of registration results on the abdominal CT dataset, featuring different variations of EOIR. Metrics including Dice (%), HD95 (mm), SDlogJ, multi-adds and Params are averaged across all image pairs for each method. Symbols indicate the desired direction of metric values: ↑ \uparrow implies higher is better, while ↓ \downarrow indicates lower is better. “Initial” refers to the baseline results before registration.

Model	Dice (%) $\uparrow$	HD95 (mm) $\downarrow$	SDlogJ $\downarrow$	Multi-Adds (GB) $\downarrow$	Params (MB) $\downarrow$
Initial	30.86	29.77	-	-	-
EOIR-0CONV	43.39	27.85	0.170	179.35	0.80
EOIR-1CONV	54.40	21.26	0.172	180.98	0.80
EOIR-2CONV	55.92	20.92	0.177	235.40	0.83
EOIR-3CONV	60.64	17.61	0.173	398.60	0.91
EOIR-4CONV	61.54	18.15	0.172	616.16	1.02
EOIR-5CONV	62.23	17.17	0.170	833.72	1.13
EOIR-6CONV	62.58	17.23	0.170	1050.00	1.24
EOIR-UNet	58.27	17.22	0.168	1300.00	15.25

Table 9. TABLE IX: Quantitative results on the LUMIR dataset test phase, comparing EOIR with baseline methods and other participating teams.

Team	TRE_LM $\downarrow$	DSC (%) $\uparrow$	HD95 (mm) $\downarrow$	NDV $\downarrow$	Rank Score $\uparrow$	Rank $\uparrow$
Initial	4.38	0.55	4.91	-	-	-
honkamj	3.09	0.79	3.04	0.0025	0.814	1
hnuzyx_next-gen-nn (ours)	3.12	0.78	3.28	0.0001	0.781	2
lieweaver	3.07	0.78	3.29	0.0121	0.737	3
uniGradICONwlO50	3.14	0.76	3.40	0.0002	0.668	7
VFA	3.14	0.78	3.15	0.0704	0.667	8
TransMorph	3.14	0.76	3.46	0.3621	0.518	11
deedsBCV	3.10	0.70	3.94	0.0002	0.423	13
uniGradICON	3.24	0.74	3.57	0.0001	0.402	14
SynthMorph	3.23	0.72	3.61	0.0000	0.361	17
ANTsSyN	3.48	0.70	3.69	0.0000	0.265	19
VoxelMorph	3.53	0.71	4.07	1.2167	0.157	20

Table 10. TABLE X: Subject to atlas registration on ADHD dataset.

	DSC	HD95	Ranking	NDV	DSC30
Method	Mean($\pm$Std. Dev.)	Mean($\pm$Std. Dev.)	(ACC)	Mean($\pm$Std. Dev.)	Mean($\pm$Std. Dev.)
Bailiang	0.774($\pm$0.011)	3.127($\pm$0.385)		7.61e-03($\pm$2.46e-03)	0.762($\pm$0.007)
next-gen-nn	0.772($\pm$0.011)	3.130($\pm$0.383)		1.73e-04($\pm$1.64e-04)	0.758($\pm$0.006)
honkamj	0.762($\pm$0.012)	3.119($\pm$0.365)		1.72e-03($\pm$2.73e-04)	0.749($\pm$0.006)
LoRA-FT	0.741($\pm$0.016)	3.441($\pm$0.423)		5.95e-04($\pm$3.25e-04)	0.724($\pm$0.009)
MadeForLife	0.767($\pm$0.012)	3.188($\pm$0.390)		4.06e-03($\pm$2.09e-03)	0.753($\pm$0.007)
lukasf	0.763($\pm$0.012)	3.246($\pm$0.396)		5.43e-02($\pm$1.22e-02)	0.750($\pm$0.007)
LYU1	0.763($\pm$0.013)	3.197($\pm$0.382)		5.03e-03($\pm$7.45e-04)	0.749($\pm$0.008)
TimH	0.719($\pm$0.014)	3.530($\pm$0.393)		0.00e+00($\pm$0.00e+00)	0.704($\pm$0.008)
VROC	0.706($\pm$0.010)	3.781($\pm$0.369)		9.87e-02($\pm$3.20e-02)	0.695($\pm$0.006)
DutchMasters	0.760($\pm$0.012)	3.165($\pm$0.383)		1.64e-03($\pm$1.02e-03)	0.746($\pm$0.008)
zhuoyuanw210	0.766($\pm$0.012)	3.125($\pm$0.371)		1.30e-03($\pm$4.02e-04)	0.752($\pm$0.007)
ANTsSyN	0.745($\pm$0.013)	3.282($\pm$0.393)		0.00e+00($\pm$0.00e+00)	0.730($\pm$0.007)
DeedsBCV	0.698($\pm$0.016)	3.673($\pm$0.386)		1.95e-04($\pm$4.43e-04)	0.680($\pm$0.008)
FireANTsGreedy	0.749($\pm$0.015)	3.398($\pm$0.429)		0.00e+00($\pm$0.00e+00)	0.732($\pm$0.009)
FireANTsSyN	0.741($\pm$0.014)	3.474($\pm$0.428)		2.74e-05($\pm$2.05e-05)	0.725($\pm$0.009)
SynthMorph	0.720($\pm$0.019)	3.442($\pm$0.404)		6.38e-06($\pm$6.73e-06)	0.699($\pm$0.008)
TransMorph	0.762($\pm$0.012)	3.244($\pm$0.390)		1.08e-01($\pm$2.28e-02)	0.748($\pm$0.007)
uniGradICON	0.740($\pm$0.013)	3.379($\pm$0.392)		1.51e-05($\pm$2.06e-05)	0.726($\pm$0.008)
uniGradICONiso	0.754($\pm$0.012)	3.230($\pm$0.385)		3.38e-05($\pm$7.31e-05)	0.740($\pm$0.008)
VFA	0.763($\pm$0.014)	3.075($\pm$0.375)		8.12e-03($\pm$1.71e-03)	0.748($\pm$0.006)
VoxelMorph	0.720($\pm$0.020)	3.773($\pm$0.482)		4.86e-01($\pm$7.56e-02)	0.698($\pm$0.012)
ZeroDisplacement	0.569($\pm$0.031)	4.590($\pm$0.518)		0.00e+00($\pm$0.00e+00)	0.534($\pm$0.015)

Table 11. TABLE XI: Subject to atlas registration on NIMH dataset.

	DSC	HD95	Ranking	NDV	DSC30
Method	Mean($\pm$Std. Dev.)	Mean($\pm$Std. Dev.)	(ACC)	Mean($\pm$Std. Dev.)	Mean($\pm$Std. Dev.)
Bailiang	0.813($\pm$0.008)	2.487($\pm$0.189)		8.25e-03($\pm$2.47e-03)	0.803($\pm$0.006)
next-gen-nn	0.811($\pm$0.009)	2.448($\pm$0.189)		1.80e-04($\pm$1.59e-04)	0.800($\pm$0.006)
honkamj	0.806($\pm$0.012)	2.424($\pm$0.226)		2.05e-03($\pm$4.09e-04)	0.791($\pm$0.008)
LoRA-FT	0.777($\pm$0.011)	2.709($\pm$0.282)		1.15e-03($\pm$1.08e-03)	0.764($\pm$0.009)
MadeForLife	0.809($\pm$0.011)	2.482($\pm$0.219)		5.96e-03($\pm$1.66e-03)	0.795($\pm$0.008)
lukasf	0.796($\pm$0.008)	2.596($\pm$0.208)		6.69e-02($\pm$1.50e-02)	0.786($\pm$0.006)
LYU1	0.806($\pm$0.012)	2.476($\pm$0.221)		5.72e-03($\pm$8.02e-04)	0.790($\pm$0.009)
TimH	0.760($\pm$0.010)	2.844($\pm$0.211)		0.00e+00($\pm$0.00e+00)	0.747($\pm$0.007)
VROC	0.714($\pm$0.020)	3.370($\pm$0.256)		6.39e-02($\pm$2.44e-02)	0.691($\pm$0.015)
DutchMasters	0.801($\pm$0.008)	2.444($\pm$0.204)		3.47e-03($\pm$1.31e-03)	0.791($\pm$0.006)
zhuoyuanw210	0.809($\pm$0.011)	2.440($\pm$0.208)		2.06e-03($\pm$5.49e-04)	0.795($\pm$0.008)
ANTsSyN	0.784($\pm$0.015)	2.598($\pm$0.224)		0.00e+00($\pm$0.00e+00)	0.770($\pm$0.019)
DeedsBCV	0.729($\pm$0.012)	3.059($\pm$0.230)		2.18e-04($\pm$6.37e-04)	0.715($\pm$0.007)
FireANTsGreedy	0.792($\pm$0.013)	2.699($\pm$0.271)		0.00e+00($\pm$0.00e+00)	0.776($\pm$0.009)
FireANTsSyN	0.785($\pm$0.015)	2.749($\pm$0.305)		3.69e-05($\pm$2.26e-05)	0.767($\pm$0.011)
SynthMorph	0.751($\pm$0.013)	2.773($\pm$0.255)		7.71e-06($\pm$1.03e-05)	0.735($\pm$0.009)
TransMorph	0.803($\pm$0.010)	2.550($\pm$0.212)		1.19e-01($\pm$2.36e-02)	0.791($\pm$0.007)
uniGradICON	0.780($\pm$0.009)	2.628($\pm$0.231)		3.14e-05($\pm$3.62e-05)	0.770($\pm$0.006)
uniGradICONiso	0.794($\pm$0.008)	2.496($\pm$0.217)		1.20e-04($\pm$1.27e-04)	0.785($\pm$0.006)
VFA	0.805($\pm$0.013)	2.425($\pm$0.209)		9.40e-03($\pm$1.79e-03)	0.788($\pm$0.008)
VoxelMorph	0.768($\pm$0.017)	3.009($\pm$0.329)		5.10e-01($\pm$8.16e-02)	0.748($\pm$0.012)
ZeroDisplacement	0.596($\pm$0.022)	3.835($\pm$0.307)		0.00e+00($\pm$0.00e+00)	0.570($\pm$0.015)

Table 12. TABLE XII: Subject to atlas registration on UltraCortex-9.4T dataset.

	DSC	HD95	Ranking	NDV	DSC30
Method	Mean($\pm$Std. Dev.)	Mean($\pm$Std. Dev.)	(ACC)	Mean($\pm$Std. Dev.)	Mean($\pm$Std. Dev.)
Bailiang	0.783($\pm$0.032)	2.911($\pm$0.626)		1.23e-02($\pm$4.29e-03)	0.747($\pm$0.038)
next-gen-nn	0.784($\pm$0.035)	2.844($\pm$0.691)		2.65e-04($\pm$1.91e-03)	0.745($\pm$0.043)
honkamj	0.778($\pm$0.031)	2.794($\pm$0.616)		2.35e-03($\pm$4.15e-04)	0.743($\pm$0.038)
LoRA-FT	0.736($\pm$0.031)	3.168($\pm$0.697)		4.31e-03($\pm$2.48e-03)	0.699($\pm$0.033)
MadeForLife	0.787($\pm$0.030)	2.832($\pm$0.627)		7.03e-03($\pm$1.82e-03)	0.753($\pm$0.036)
lukasf	0.764($\pm$0.034)	2.963($\pm$0.624)		1.01e-01($\pm$2.11e-02)	0.725($\pm$0.039)
LYU1	0.781($\pm$0.031)	2.846($\pm$0.653)		8.19e-03($\pm$1.08e-03)	0.746($\pm$0.037)
TimH	0.735($\pm$0.033)	3.182($\pm$0.642)		0.00e+00($\pm$0.00e+00)	0.696($\pm$0.038)
VROC	0.694($\pm$0.026)	3.635($\pm$0.650)		9.02e-02($\pm$7.41e-02)	0.663($\pm$0.022)
DutchMasters	0.760($\pm$0.028)	2.919($\pm$0.688)		1.14e-02($\pm$6.84e-03)	0.728($\pm$0.031)
zhuoyuanw210	0.781($\pm$0.031)	2.855($\pm$0.668)		2.35e-03($\pm$7.61e-04)	0.745($\pm$0.037)
ANTsSyN	0.756($\pm$0.034)	2.949($\pm$0.627)		0.00e+00($\pm$0.00e+00)	0.716($\pm$0.041)
DeedsBCV	0.696($\pm$0.027)	3.415($\pm$0.596)		9.95e-05($\pm$1.84e-04)	0.663($\pm$0.027)
FireANTsGreedy	0.762($\pm$0.036)	3.092($\pm$0.677)		0.00e+00($\pm$0.00e+00)	0.720($\pm$0.041)
FireANTsSyN	0.756($\pm$0.034)	3.144($\pm$0.703)		3.14e-05($\pm$1.85e-05)	0.717($\pm$0.039)
SynthMorph	0.711($\pm$0.034)	3.213($\pm$0.706)		7.31e-06($\pm$1.03e-05)	0.670($\pm$0.036)
TransMorph	0.776($\pm$0.035)	2.912($\pm$0.649)		1.61e-01($\pm$2.71e-02)	0.735($\pm$0.041)
uniGradICON	0.738($\pm$0.031)	3.149($\pm$0.743)		8.53e-05($\pm$7.06e-05)	0.702($\pm$0.034)
uniGradICONiso	0.756($\pm$0.030)	2.977($\pm$0.690)		4.57e-04($\pm$7.50e-04)	0.721($\pm$0.035)
VFA	0.782($\pm$0.030)	2.763($\pm$0.596)		1.22e-02($\pm$2.15e-03)	0.750($\pm$0.037)
VoxelMorph	0.733($\pm$0.043)	3.496($\pm$0.785)		6.33e-01($\pm$1.43e-01)	0.681($\pm$0.049)
ZeroDisplacement	0.568($\pm$0.045)	4.272($\pm$0.772)		0.00e+00($\pm$0.00e+00)	0.513($\pm$0.043)

Table 13. TABLE XIII: Quantitative results on the ThoraxCBCT dataset, obtained from the online leaderboard.

Team	Dice (%) $\uparrow$	TRE(KP) (mm) $\downarrow$	HD95 (mm) $\downarrow$	SDlogJ $\downarrow$
Initial	31.3	9.91	55.36	-
VoxelMorph++	50.3	13.68	28.56	0.129
deeds	64.8	11.32	29.03	0.152
EOIR(3 CONV)	45.4	13.12	53.49	0.115
EOIR(U-Net)	56.2	14.23	41.49	0.228

Equations

$$\hat{\theta }=\underset{\theta }{\mathrm{argmin}}\{{\mathbb{E}}_{({\mathbf{I}}_f,{\mathbf{I}}_m)\sim D}[\mathcal{L}({\mathbf{I}}_f,{\mathbf{I}}_m,g_{\theta }({\mathbf{I}}_f,{\mathbf{I}}_m))]\}.$$

$$\nabla {\mathbf{I}}_f(p)\cdot \mathbf{u}(p)={\mathbf{I}}_m(p)-{\mathbf{I}}_f(p),$$

$$n>{\log }_2(d_{\text{max}})+1,$$

$$\frac{\mathrm{d}{\varphi }^{(t)}}{\mathrm{d}t}=\mathbf{v}({\varphi }^{(t)}).$$

$$\frac{\mathrm{d}{\varphi }^{(t)}}{\mathrm{d}t}={\mathbf{v}}^{(t)}({\varphi }^{(t)}).$$

$${\varphi }^{(t+\Delta t)}=({\mathbf{I}}_d+\Delta t{\mathbf{v}}^{(t)})\circ {\varphi }^{(t)},$$

$$\varphi \approx$$

$${\varphi }^{t_{21}}\circ {\varphi }^{t_{22}}\circ \dots \circ {\varphi }^{t_{2m}}\circ$$

$$\dots$$

$${\varphi }^{t_{n1}}\circ {\varphi }^{t_{n2}}\circ \dots \circ {\varphi }^{t_{nm}}.$$

$$H_2[{\mathbf{I}}_m^{(l)},{\mathbf{I}}_f^{(l)}{]}^{\top }=[{\mathbf{I}}_m^{(l)}+{\mathbf{I}}_f^{(l)},{\mathbf{I}}_m^{(l)}-{\mathbf{I}}_f^{(l)}{]}^{\top },$$

$${\tilde{\varphi }}_{l+1}$$

$${\mathbf{u}}_l$$

$$\Delta {\varphi }_l$$

$${\varphi }_l$$

$$\mathcal{L}=\sum _{l=1}^n\frac{1}{2^{l-1}}\left[s\left(d({\mathbf{I}}_m,l)\circ {\varphi }_l,d({\mathbf{I}}_f,l)\right)+\lambda r({\mathbf{u}}_l)\right].$$

Full text

††publicationid: pubid: 0000–0000/00$00.00 © 2021 IEEE

I Introduction

Image registration, which establishes pixel/voxel correspondences between a pair of images and predicts a deformation field for their spatial alignment, is fundamental to medical imaging and computer vision [11]. Essential for applications such as medical image segmentation [4], motion tracking [20], surgical guidance [65], and diagnostic analysis [10], precise image registration facilitates accurate disease detection and monitoring, and the progress of therapeutic procedures.

The methodological landscape of image registration is diverse. As categorized in broader reviews of the field [21], techniques span intensity-based methods (e.g., optical flow, mutual information), feature-based methods [78, 41, 42] (e.g., leveraging points, lines, or deep features), and their combinations. While effective in many domains, these approaches face pronounced challenges in the context of high-resolution volumetric medical images, where computational burden and large deformations are paramount concerns.

Traditional methods [3, 72] typically adopt variants of the Horn-Schunck (H-S) style variational formulation, involving a global dense displacement formulation and computationally expensive iterative processes. Recent advances in convolutional neural networks (ConvNets) [61] and transformers [18, 45] have enabled learning-based registration methods [6, 28] to achieve faster performance by amortizing optimization across cohorts, potentially improving accuracy when trained semi-supervised with label supervision. Despite progress, two major challenges remain for learning-based registration methods, as outlined below.

Lowering Computational Complexity: Registering volumetric medical images demands considerable computational resources. To improve registration accuracy, advanced neural network modules, such as transformers [18, 45] or large convolution kernels [36, 17], have been proposed, but these modules come with further increased computational demands. Despite this, the challenge of reducing computational complexity has been largely overlooked in the literature, with only a few learning-based methods [76, 35, 37, 80] explicitly addressing the efficiency issue. Yet, these approaches often lower computational complexity at the cost of accuracy, failing to achieve a practical balance between accuracy and efficiency for volumetric image registration.

Handling Large Deformations: To manage large deformations, many existing approaches employ cascaded network architectures [91, 38], multi-scale coarse-to-fine strategies [56, 54], or image pyramid structures [81, 75]. While effective, these methods often rely on complex architectures, complicating the training process and increasing computational demands. We argue that the inability to effectively integrate prior knowledge limits the capacity of current neural networks from being fully utilized.

Previous research has shown that incorporating prior knowledge can improve the accuracy-efficiency trade-off in both image segmentation [84] and image registration [76, 35, 12], which motivates the design of our architecture. Building on this principle, we identify two key roles of ConvNets in image registration, based on our empirical observations and prior studies [33, 73, 81]:

1. linearizing local intensities, ensuring intensity changes vary linearly with spatial coordinates to aid displacement estimation in textureless regions.
2. harmonizing global contrast variations, minimizing intensity discrepancies of the same anatomical regions across different subjects or phases to facilitate alignment. These two roles help produce image features that satisfy the brightness constancy assumption in the Horn-Schunck (H-S) optical flow equation [29] (hereafter referred to as the H-S assumption).

The first role can be empirically observed in Fig. 1. The left three columns illustrate that simple convolution operations, such as Gaussian filtering, help linearize local intensities of a simple structure (a uniform square here), aiding displacement recovery in textureless regions (the inner areas of the square). The right two columns demonstrate that trained ConvNet filters achieve superior linearization compared to untrained ones on more complex real-world abdominal data, as evidenced by a wider distribution of linearized locations. This observation aligns with [33], which showed that ConvNets implicitly learn features as a function of spatial coordinates, with deeper layers improving the readout of larger distances. Fig. 1 was created based on the H-S assumption, which relates the intensity change between a pair of images at a given location to the product of the displacement and the local image gradient.

The second role is supported by [73], which observed that Pearson’s correlation between feature maps of different modalities increases with network depth, indicating that deeper ConvNet layers generate features more invariant to input modality. Additionally, our prior work [81] demonstrated that image features learned for registration tasks can benefit segmentation, creating mutual improvements for both tasks. Given that ConvNets can bridge differences between modalities and harmonize image intensities into more uniform features across regions (as in segmentation), handling registration tasks with milder contrast variations, such as those addressed in this work, may require fewer convolutional layers and be more computationally efficient.

Based on these observations, we propose the Linearization-Harmonization (L-H) assumption as a design guideline for our registration network: linearizing local intensities and harmonizing global contrast variations constitute the core roles of ConvNets in deformable image registration. The H-S assumption informs us about what features are beneficial for registration, while the L-H assumption explains when a neural network can produce such features. To decrease the computational complexity while handling large deformation, we introduce the following modifications to existing networks under the guidance of H-S and L-H assumption: M1: Decoupled feature extraction: unlike traditional learning-based methods [6, 56, 8] that concatenate moving and fixed images at the network input and process the entire network as a unified flow estimator, we extract feature maps from moving and fixed images independently. This separation of feature extraction from flow estimation enables the generation of image features consistent with the H-S assumption. M2: Large deformation diffeomorphic framework: since the H-S assumption holds primarily for small displacements, we embed the registration process in a large deformation diffeomorphic framework with a multi-resolution pyramid and scaling-and-squaring integration at each level. The moving image features are progressively warped using the deformation field from the preceding level, so that only residual deformations (ideally smaller than one voxel) are estimated at each stage. M3: Moving and fixed features are combined using the Hadamard product, which acts as a lightweight cost-volume and naturally conforms to the local matching principle of the H-S assumption. M4: Multi-level similarity loss with Gaussian smoothing: we compute image similarity losses across multiple resolution levels, applying Gaussian smoothing before down-sampling to improve gradient behavior in homogeneous regions (corresponding to both H-S and L-H assumption). M5: Lightweight encoder with shallow convolutional blocks: based on the above design, the network extracts sufficiently discriminative features using only a few convolutional blocks for mono-modal image registration and simple multi-modal image registration.

To accommodate these modifications, we propose the Encoder-Only Image Registration (EOIR) framework to address the aforementioned challenges (see Fig. 2 for an overview of the EOIR architecture). The name EOIR reflects the framework’s simplicity, as it utilizes only an encoder, foregoing a more complex encoder-decoder structure. While such terms may vary across contexts, we emphasize this minimal design to highlight its efficiency. The major contributions of this work are summarized as follows:

• •

We propose EOIR, a registration framework whose architecture is explicitly derived from the H-S and L-H assumptions. This is realized through five key design choices, including decoupled feature extraction, a diffeomorphic pyramid with warped feature propagation, Hadamard-based feature matching, and multi-level loss with Gaussian smoothing.

• •

The above design leads to an exceptionally efficient model. By decoupling feature extraction and using lightweight matching, EOIR achieves state-of-the-art efficiency-accuracy trade-offs, enabling higher accuracy at comparable complexity or drastically lower complexity without sacrificing accuracy.

• •

Evaluated on six diverse datasets, EOIR demonstrates strong generalization. It secured 2nd place in the Learn2Reg LUMIR Challenge 2024 [9] with a sub-1MB model trained in two days on  4,000 subjects, proving its readiness for large-scale, accurate registration.

The remainder of this paper is organized as follows. Section II reviews related work. Section III details our proposed EOIR framework. Section IV describes the experimental setup and presents a comprehensive analysis of the results. Section V concludes the paper.

II Related Work

The proposed EOIR framework falls under the category of learning-based registration models. In this section, we first review classic learning-based registration methods, followed by computationally efficient registration techniques, and conclude with models designed to handle large deformations.

II-A Learning-Based Image Registration

Deformable image registration (DIR) is traditionally formulated as an energy optimization problem where dissimilarity between moving $\mathbf{I}_{m}$ and fixed $\mathbf{I}_{f}$ images is quantified using a dissimilarity function $s(\cdot)$. To counteract the ill-posed nature of DIR, a regularization term $r(\cdot)$ constrains the deformation field. While traditional methods directly optimize the deformation field through gradient descent [4] or discrete optimization [25, 85], learning-based approaches [6, 30, 93, 23, 50, 66, 90, 89] optimize the expected loss function to derive neural network weights $\theta$ from a collection of image pairs $D$, as formulated below:

[TABLE]

In this equation, $\mathcal{L}(\mathbf{I}_{f},\mathbf{I}_{m},\mathbf{u})=s(\mathbf{I}_{f},\mathbf{I}_{m}\circ\phi)+r(\mathbf{u})$ denotes the loss function. Here, $\mathbf{I}_{m}\circ\phi$ represents the warping of the moving image by the deformation field $\phi=\mathbf{I}_{d}+\mathbf{u}$, where $\mathbf{I}_{d}$ is the identity transformation grid.

Using Eq. (1) for unsupervised learning is much faster than traditional energy optimization methods and may benefit from label supervision such as segmentation loss, which may further increase the accuracy of anatomical alignment. VoxelMorph [6], a pioneering learning-based model, entangles feature extraction and flow estimation in a single U-Net architecture [61], processing volumetric brain MR images in seconds. Following VoxelMorph, [16] introduces scaling and squaring layers [3] to ensure diffeomorphism. Vision transformers [45] have also been incorporated into frameworks like TransMorph [8] and H-ViT [22].Symmetric registration networks [57], multi-channel architectures [13], dual-stream networks [39], large-kernel convolutions [36, 12], and cascaded networks [91, 56] further contribute to progress in the field. However, these improvements often lead to an exponential increase in parameters, raising computational demands, which can be challenging in resource-constrained clinical settings such as limited GPU memory or large volumetric datasets [44].

II-B Computationally-Efficient Image Registration

Pursuing computational efficiency in image registration has been approached through simplified representations in traditional methods and, more recently, through specialized network architectures in deep learning. Exemplifying the former, Albu [1] transformed the 2D problem into 1D signal alignment via integral projections. Meanwhile, the explicit design of inherently efficient deep networks remains less explored, with only a few works directly addressing this goal [76, 35, 37, 80, 85]. Some efficiency gains are achieved as a byproduct of specific architectural choices [6, 12, 82]. These pioneering efforts inspired EOIR’s design, demonstrating how integrating prior knowledge can reduce complexity while maintaining accuracy, or even improve it.

DeepFlash [76] approximates the original displacement space using a low-dimensional, band-limited space, performing neural network inference within this constrained domain. This accelerates training and inference without sacrificing accuracy compared to VoxelMorph, based on the assumption that flow fields inherently lack high frequencies in the Fourier domain. Similarly, FourierNet [35] builds on this prior but improves efficiency by employing a model-driven decoder to better leverage the band-limited approximation, achieving a superior accuracy-efficiency trade-off. ShiftMorph [80] skips the Fourier transform but applies a similar concept, operating at lower spatial resolutions for greater efficiency. LessNet [37] eliminates the encoder entirely, significantly reducing network parameters while maintaining accuracy comparable to VoxelMorph [6] and TransMorph [8].

Additionally, some models [12, 14, 82, 24] incorporate other forms of prior knowledge to further improve efficiency. TextSCF [12] uses a large visual-language model to enhance inter-regional anatomical understanding, outperforming models without priors in both efficiency and accuracy [6, 8, 56, 36]. The Slicer Network [82], though designed for general medical image analysis, employs edge-preserving adaptive filters to expand the effective receptive field [47], leading to better accuracy-efficiency trade-offs. Different from the above-mentioned approaches, our framework, EOIR, achieves a more favorable accuracy-efficiency balance by novelly integrating the H-S and L-H assumptions, strategically disentangling feature extraction from flow estimation, and employing a lightweight convolutional encoder.

II-C Large-Deformation Image Registration

Most learning-based models discussed in §II-A and §II-B exhibit limited performance on datasets involving large deformations. Even methods employing vision transformers or large convolutional kernels to expand the effective receptive field often fail to address such deformations adequately. To overcome this limitation, two primary architectural strategies have emerged: (1) recursive or cascaded flow estimators [92, 38, 83, 83]; (2) coarse-to-fine image pyramids [56, 81, 54].

The first approach, exemplified by VTN [92] and VR-Net [38], employs cascaded sub-networks that iteratively refine deformations through sequential steps, with each step generating a full-resolution deformation field. The second strategy, adopted in LapIRN [57], IIRP-Net [51], MemWarp [81], and RDP [75] progressively refines deformation fields using a coarse-to-fine pyramid framework, which often achieves better efficiency and large-deformation handling. CorrMLP [54] further enhances this paradigm by integrating a correlation-aware multi-window MLP block into its coarse-to-fine architecture.

However, these methods typically lack explicit prior knowledge, requiring additional computational resources to process cascaded sub-networks or multi-scale features. In contrast, our EOIR framework is designed with a more efficient feature pyramid, guided by the integrated H-S and L-H assumptions as architectural priors, which enables a superior balance between accuracy and efficiency.

III Methodology

In this section, we begin with preliminaries which review the H-S optical flow equation and explain how neural networks operating under the L-H assumption facilitate the production of image features that satisfy the H-S assumption. We then introduce the large-deformation diffeomorphic model, followed by a detailed description of our EOIR framework, including its network architecture and loss functions.

III-A Preliminary

III-A1 Horn–Schunck (H-S) Equation

The original H-S equation [29] relates the displacement field $\mathbf{u}(p)$ between images $\mathbf{I}_{m}$ and $\mathbf{I}_{f}$ to their intensity difference and gradient under sufficient spatial sampling:

[TABLE]

where $\nabla\mathbf{I}_{f}(p)=\left[\frac{\partial\mathbf{I}_{f}}{\partial x},\frac{\partial\mathbf{I}_{f}}{\partial y},\frac{\partial\mathbf{I}_{f}}{\partial z}\right]_{p}$ is the spatial gradient at voxel $p\in\Omega$. Equation (2) holds when $\|\mathbf{u}(p)\|^{2}<1$ voxel, achievable through sufficient spatial downsampling (see $x_{a}$ in Fig. 3). To satisfy this small-displacement requirement, we adopt a Laplacian feature pyramid [56, 81], where the number of pyramid levels $n$ is determined by the maximum displacement $d_{\text{max}}$ in the dataset:

[TABLE]

ensuring deformations up to $d_{\text{max}}$ are resolved. The number of pyramid levels is determined by the maximum displacement in a given dataset. While adjusting the levels can reduce complexity, we adopt a 5-level pyramid for simplicity, which generalizes well across most datasets. (See §IV-D3 for a comparison of the effects of varying pyramid levels across datasets.)

Despite the use of a pyramid, Eq. (2) may fail even with sub-voxel displacements in three scenarios:

1. at locations with insufficiently large image gradients, indicating flat or noisy regions where no local constraints can be imposed ($x_{d}$ to $x_{e}$ in Fig. 3);
2. when the intensity difference between the moving and fixed images at certain locations is close to zero ($x_{b}$ in Fig. 3);
3. when a global intensity bias is added to the moving or fixed image, as between $x=2$ and $x_{d}$ in Fig. 3). Voxels in these scenarios can be addressed by feature linearization via a neural network and global smoothness constraints through a loss function. The former offers features with local intensity linearization, while the latter can assist in propagating displacements from surrounding valid locations.

III-A2 Large-Deformation Diffeomorphic Model

For large diffeomorphic deformations, a widely used approach [2, 3] involves utilizing a stationary velocity field (SVF) $\mathbf{v}$. This field is integrated over unit time from $t=0$ to $t=1$ to produce the final deformation field $\phi^{(1)}$, starting from the identity transformation $\phi^{(0)}=\mathbf{I}_{d}$. The integration is governed by the ordinary differential equation (ODE):

[TABLE]

However, as noted in Section §III-A1, modeling displacements up to a maximum $d_{max}$ requires a Laplacian feature pyramid of $n$ levels to ensure $2^{n-1}>d_{max}$ (Eq. 3). Given that $n$ residual deformation fields are estimated at each pyramid level and must be composed to form the final deformation field, a single SVF is insufficient to adequately model the dynamics across different pyramid levels. Therefore, we utilize $n$ SVFs, each corresponding to a different pyramid level, to govern the evolution of the deformation. Consequently, with $\mathbf{v}^{(t)}$ as a piece-wise constant function representing the velocity field at time $t$, the deformation evolves as the following ODE:

[TABLE]

The integration of Eq. (5) can be discretized into a finite number of steps, expressed as:

[TABLE]

where $\Delta t$ is the integration step size. This method models large deformations as a sequence of small deformations at each step. To ensure accurate approximation, the step number must be large enough to capture each deformation sufficiently.

III-A3 Integration and Deformation Field Composition

In learning-based registration models such as VoxelMorph [6], which output a single deformation field for an image pair, Eq. (4) can parametrize the deformation field. The scaling and squaring method, derived from Lie Theory, is an efficient integration solution. It’s widely used for rapid integration in learning-based registration models [16, 56], enabling efficient computation of diffeomorphic deformations.

Models using a feature pyramid, which must compose multiple deformation fields from coarse to fine, require more careful handling of integration and composition. For example, LapIRN [56] and MemWarp [81] use a Laplacian feature pyramid and compose deformation fields across levels by addition. While computationally less demanding, this approach can result in a poorer trade-off between deformation smoothness and registration accuracy [12], as well as slower convergence and a tendency to settle at sub-optimal local minima [72].

To address this issue, we propose discretizing the unit time into a substantial number of smaller intervals, specifically $n\times m$ steps, where $n$ is the number of pyramid levels, and $m$ is the number of steps within each level corresponding to the pyramid hierarchy. Let $\phi^{t_{ij}}$ denote the $j^{th}$ step of the $i^{th}$ level, the final deformation field $\phi$ can be approximated as:

[TABLE]

While naively computing Eq. (7) demands $\mathcal{O}(nm)$ complexity and may prove computationally burdensome, employing the scaling and squaring method by [2] for each level significantly reduces this to $\mathcal{O}(n\log m)$.

III-B Network Architecture

The design of the EOIR architecture is guided by the H-S and L-H assumptions. It comprises three main components: an encoder for feature extraction, a set of flow estimators, and a deformation field composition module. In the following sections, we detail each component and conclude with a discussion of the loss function for deep supervision [40].

III-B1 Encoder

Different from previous research [6, 8], EOIR disentangle the feature extraction and flow estimation steps, with a light-weight encoder to extract features from the moving and fixed images separately (M1). Based on the H-S assumption, we construct a large deformation diffeomorphic framework (Fig. 2, M2). To model large-deformation diffeomorphic transformations, we approximate the final deformation field via integration over small intervals using Eq. (7), where $n$ pyramid levels each contain $m$ intervals. Each interval within a pyramid level follows the small-deformation model $\phi(p)=p+\mathbf{u}(p)$. Despite these design choices, Fig. 3 illustrates scenarios where displacement determination remains ambiguous, even under sufficient downsampling (with $\|\mathbf{u}(p)\|^{2}<1$ voxel), due to vanishing gradients or contrast variations.

The proposed L-H assumption, grounded in empirical observations from both synthetic and clinical data (Fig. 1) and prior research [34, 73, 81], explains that trained ConvNets for image feature extraction can effectively linearize and harmonize local intensities in mono-modal registration tasks. Additionally, Fig. 1 demonstrates a considerable increase in the number and distribution of valid locations satisfying the H-S assumption when using trained ConvNets.

Guided by H-S and L-H assumptions and the small-deformation model for each interval, we propose that local intensity linearization within small voxel neighborhoods is sufficient to recover large diffeomorphic deformations. Thus, we employ only three learnable convolutional layers (M5), rather than a complex encoder-decoder, to efficiently act as local intensity linearizers and contrast harmonizers, achieving an optimal accuracy-efficiency trade-off. While additional layers ($n_{c}>3$) could improve registration accuracy, the marginal gains diminish rapidly beyond three layers (see Fig. 4).

The encoder comprises three Conv-Norm-Act blocks, each containing a learnable $3\!\times\!3\!\times\!3$ convolutional layer, instance normalization [71], and a ReLU activation. As shown in Fig. 5 (left), the encoder begins with $N_{s}$ channels, expands to $2N_{s}$ via an inverted bottleneck design [62], and contracts back to $N_{s}$. Combined with $n-1$ rounds of $2\times$ spatial downsampling, this design ensures effective local intensity linearization and contrast harmonization.

III-B2 Flow Estimator

With the locally linearized and harmonized feature maps from the encoder, we apply $2\times$ trilinear downsampling to the feature maps $n-1$ times, resulting in an $n$-level feature pyramid. This pyramid is structured to handle displacements with a maximum of $d_{max}<2^{n-1}$ (Eq. 3). At each pyramid level, a flow estimator, consisting of a Hadamard transform layer [59] and several conv blocks, generates the residual flow. The flow estimator’s structure is depicted in the right panel of Fig. 5 and further detailed below.

The Eq. (2) shows that displacements are linked to the intensity differences between moving and fixed images. Rather than using a network to learn this difference, we explicitly apply a Hadamard transform to the feature maps (M3):

[TABLE]

where $\mathbf{I}_{m}^{(l)}$ and $\mathbf{I}_{f}^{(l)}$ are the moving and fixed image feature maps at pyramid level $l$, and $H_{2}=\begin{bmatrix}1&1\\ 1&-1\end{bmatrix}$. After the transform, the moving and fixed features are stacked along the channel dimension, processed with three Conv-Norm-Act blocks, and a linear layer on each voxel produces final displacement fields.

III-B3 Deformation Field Composition

To build the large deformation diffeomorphic framework (M2), deformation fields composition step is essential to build the pyramid. We represent the velocity field at time $t$ by a piece-wise constant function, $\mathbf{v}^{(t)}$, where the number of pieces corresponds to the number of feature pyramid levels $n$. This yields multiple stationary ODEs (Eq. (5)) across different periods within the unit time. Additionally, under the H-S assumption that the displacement difference between adjacent periods is minimal, we can employ Eq. (8) to approximate the deformation composition process throughout our feature pyramid, thereby promoting a diffeomorphic transformation.

The deformation field composition at the $l^{th}$ level of the pyramid involves the following steps. Starting with feature maps $\mathbf{I}_{m}^{(l)}$ and $\mathbf{I}_{f}^{(l)}$, and the deformation field $\phi_{l+1}$ from the previous level:

[TABLE]

where $\text{up}(\cdot)$ denotes $2\times$ trilinear upsampling and scaling, $\text{exp}(\cdot)$ refers to the scaling and squaring function applied to the displacement field, and $\circ$ denotes the warping function. The $\mathbf{u}_{l}$ represents the residual displacement field at level $l$, and $f_{e}^{(l)}(\cdot,\cdot)$ is the flow estimator at level $l$.

III-B4 Overall Framework & Deep Supervision

With the encoder, flow estimator, and deformation field composition components in place, we have constructed the EOIR framework. This framework is visually illustrated in Fig. 2 using a 3-level pyramid. We optimize the network using a multi-scale loss function applied across the registration pyramid. At each level, the total loss comprises a similarity term, which quantifies the dissimilarity between the warped moving image and the fixed image, and a regularization term, which enforces smoothness in the deformation fields, following the convention of [6, 12]. The total loss is calculated as an exponentially decayed weighted sum across all levels:

[TABLE]

Here, $n$ represents the number of pyramid levels, $s(\cdot,\cdot)$ denotes the dissimilarity function, $d(\cdot,l)$ downsamples the input image by a factor of $2^{l-1}$ (Gaussian smoothing is applied before downsampling, M4), and $r(\cdot)$ represents the smoothness regularization function applied to the displacement field $\mathbf{u}_{l}$ before scaling and squaring. We use $r(\mathbf{u}_{l})=\|\nabla\mathbf{u}_{l}\|^{2}$ as the regularization function, with $\lambda$ as its coefficient.

IV Experiments & Results

In this section, we evaluate the proposed EOIR against state-of-the-art image registration methods across six datasets, covering various imaging modalities, input constraints, and anatomies. The following subsections detail the datasets, implementation details, baseline methods, and evaluation metrics. We then present qualitative and quantitative results, including analyses of accuracy-efficiency trade-offs, accuracy-smoothness comparisons, and an ablation study. The source code of EOIR is available on Github.

IV-A Datasets

The datasets span both computed tomography (CT) and magnetic resonance imaging (MRI) modalities, with inter-subject and intra-subject settings, as well as unsupervised and semi-supervised configurations that include segmentation masks. In summary, we use the semi-supervised Abdomen CT dataset for inter-subject registration [79], the semi-supervised OASIS dataset [53], the unsupervised large-scale LUMIR dataset [19, 67, 53] for inter-subject brain MR image registration, the ACDC dataset [7] for cardiac image registration, the HippocampusMR [64] for Hippocampus MR image registration, and the RGB-IR dataset [69] for multi-modality 2D natural image registration.

IV-A1 Abdomen CT Dataset

We employ an abdominal CT dataset comprising 30 scans, each annotated with segmentation masks for 13 anatomical structures [79]. The dataset is split into 20 training, 3 validation, and 7 test scans, yielding 380 training, 6 validation, and 42 test pairs respectively. All images are resampled to 2 mm isotropic resolution and standardized to a size of $192\times 160\times 256$.

IV-A2 ACDC Dataset

We evaluate our method on the ACDC cardiac MR dataset [7], which includes 80 training, 20 validation, and 50 test subjects. Each subject provides end-diastole (ED) and end-systole (ES) images with ground-truth segmentations of the left ventricle blood pool, myocardium, and right ventricle. Registration is performed in both ED-to-ES and ES-to-ED directions, resulting in 160 training, 40 validation, and 100 test pairs. All images are preprocessed to $128\times 128\times 16$ with a voxel spacing of $1.8\times 1.8\times 10$ mm³.

IV-A3 OASIS Dataset

For semi-supervised inter-subject brain MR registration, we use the OASIS dataset from Task 3 of the Learn2Reg 2021 challenge [53, 26]. It contains 414 T1-weighted brain MRI scans, of which 394 unpaired scans are used for training. Validation and leaderboard ranking employ 19 paired images generated from 20 scans111https://learn2reg.grand-challenge.org/evaluation/task-3-validation/leaderboard/. All images are preprocessed with bias correction, skull stripping, alignment, and cropping to $160\times 192\times 224$.

IV-A4 LUMIR Dataset

The LUMIR dataset [19, 67, 53] is designed for large-scale unsupervised brain MR registration as part of Learn2Reg 2024 Task 2. It includes 3,384 training subjects and 40 validation subjects, with 10 training subjects manually annotated with anatomical landmarks to generate 38 validation pairs. All images are provided in NIfTI format, resampled, cropped to the region of interest, and standardized to $160\times 224\times 192$ with $1\times 1\times 1$ mm³ spacing.

IV-A5 HippocampusMR Dataset

The HippocampusMR dataset [64] focuses on inter-subject hippocampus MR registration and was part of the Learn2Reg 2020 challenge. We split the data into 200 training, 20 validation, and 40 test subjects. All images are preprocessed to $64\times 64\times 64$ with isotropic $1\times 1\times 1$ mm³ spacing.

IV-A6 RGB-IR Dataset

The RGB-IR dataset [69] contains 1,354 paired 2D RGB and infrared images. Following [93], random affine and deformable transformations are applied to generate registration pairs. The dataset is divided into 1,274 training, 30 validation, and 50 test pairs, with all images cropped to $256\times 256$.

IV-B Implementation Details and Baseline Methods

IV-B1 Training Details

In this study, all models were developed using the PyTorch library in Python, executed on a system with an A100 GPU. The Adam optimizer was employed for training, with an initial learning rate of $1e-4$ and a polynomial learning rate scheduler with a decay rate of 0.9. The channel parameters for EOIR’s encoder and flow estimator were set to $N_{s}=32$ and $k_{s}=1$, unless otherwise specified. The number of image pyramid levels is set to $n=5$, and the number of time steps in each period for Eq. (7) is set to $m=7$, unless otherwise specified. Dataset-specific details, including the dissimilarity function $s(\cdot)$, inclusion of Dice loss, and other training parameters, are provided in the appendix. For a fair comparison, all models were trained under identical conditions or, where necessary, using the optimal settings outlined in their original publications.

IV-B2 Baseline Methods

We compare our EOIR framework with several state-of-the-art non-iterative, learning-based baseline models, including VoxelMorph [6], TransMorph [8], LKU-Net [36], Fourier-Net [35], RDP [75], LapIRN [56], MemWarp [81] and CorrMLP [54]. For the Abdomen CT dataset, we additionally include discrete optimization-based methods ConvexAdam [63] and SAMConvex [43] in the comparison, as these are highly effective for handling large deformations. For multi-modality 2D image registration, SOTA 2D multi-modality image registration approaches like PGMR [93] and IMF[74] are also compared. For results on the OASIS and LUMIR datasets, we obtained evaluation scores from the public leaderboard or respective publications. In the case of the ACDC, Abdomen CT, HippocampusMR, and RGB-IR datasets, we used publicly available code for each model and fine-tuned them to achieve optimal performance.

IV-B3 Evaluation Metrics

Following established methodologies [15, 6, 56, 8] and challenge protocols [26], we evaluate anatomical alignment using the Dice Similarity Coefficient (Dice) and the 95% Hausdorff Distance (HD95). Target registration error (TRE) is also computed if the dataset provides ground-truth landmark points. For 2D multi-modality image registration, normalized mutual information (nMI), normalized cross-correlation (NCC), and peak signal-to-Noise ratio (PSNR) are utilized, following [93]. To assess field smoothness, we measure the standard deviation of the logarithm of the Jacobian determinant (SDlogJ), and non-diffeomorphic volumes (NDV) [44]. Additionally, computational complexity is assessed using the multiply-add operations (Multi-Adds, G) and total parameter size (Params, MB). The inference time on cardiac image registration and hippocampus image registration are also presented to demonstrate the efficiency of EOIR.

IV-C Results & Analysis on Registration Accuracy

IV-C1 Handling Large Deformations

We demonstrate the capability of handling large deformations using inter-subject abdominal CT registration. As shown in Table I and Fig. 6, EOIR outperforms all other methods in registration accuracy without compromising smoothness. Specifically, EOIR surpasses the best conventional learning-based method, LKU-Net, by 14.87%, the best image pyramid-based method, MemWarp, by 0.65%, the best efficiency-driven method, FourierNet, by 41.66%, and the best discrete optimization-based method, SAMConvex, by 13.01%. It’s worth noting that while MemWarp matches EOIR in registration accuracy, it uses additive deformation composition, resulting in a less smooth field. Although FourierNet achieves a smoother field, it falls behind in registration accuracy. Both discrete optimization-based methods generate smooth deformation fields and improve upon efficiency-driven methods, but their registration accuracy lags behind all image pyramid-based methods and EOIR.

IV-C2 Handling Local Intra-subject Motions

Unlike inter-subject abdominal CT registration, intra-subject cardiac registration focuses on tracking local cardiac motion, such as the movement of the left and right ventricles, during the complete phases of the cardiac cycle. As shown in Table II and Fig. 7, EOIR outperforms all other methods in registration accuracy, measured by Dice score. Specifically, EOIR surpasses the best conventional learning-based method, LKU-Net, by 3.1%, the best image pyramid-based method, RDP, by 1.1%, and the best efficiency-driven method, FourierNet, by 3.0%. It is worth noting that LessNet and LapIRN are excluded from Table II, as they cannot handle short-axis data, and no straightforward approach was found to enable them to do so. With only 7.2% MAs and 1.2% parameters, EOIR($Ns=8$) matches RDP’s performance. This efficiency is crucial for deployment on resource-limited hardware.

IV-C3 Handling Brain MR Image Registration

Unlike cardiac and abdomen datasets with different organ motions, inter-subject brain MR image registration requires fine-grained alignments of multiple variably shaped and sized brain structures. Table III presents the results of semi-supervised learning on the OASIS dataset. With similar field smoothness, EOIR outperforms both LKUNet and TransMorph-2 in Dice score. Note that these models were trained semi-supervisedly, and the results are clearly influenced by the provided segmentation masks during training, as improvements in Dice score often correlate with less feasible deformation fields. The advantage of EOIR in this task primarily lies in the reduction of parameter size. With similar smoothness and a slightly higher Dice score, EOIR reduces network parameter size by 98.1% compared to TransMorph (46.69 MB) and by 97.3% compared to LKUNet (33.35 MB) (the network parameter sizes are the same as those in Table II). Therefore, we further evaluate EOIR on the LUMIR dataset, which is large-scale and trained in an unsupervised manner. As shown in Table IV, EOIR outperforms TransMorph by 1.9% with the NDV close to zero. Additionally, while SynthMorph [27] and DeedsBCV [25] provide diffeomorphic deformation fields, their anatomical alignment falls short of EOIR. EOIR improves Dice by 6.8% and 10.9%, respectively, compared to SynthMorph and DeedsBCV. Compared to direct addition (EOIR(addition)), EOIR’s composition approach delivers folding-free deformation fields (0% NDV) and superior registration performance with only marginal computational overhead, underscoring the strategy of deformation composition for high-quality image registration.

IV-C4 Handling Structures with Ambiguous Boundaries

The HippocampusMR dataset involves aligning two neighboring structures (the hippocampus head and body), with less defined boundaries compared to other anatomical regions. Results on this dataset are summarized in Table V and Fig. 7. Our EOIR framework consistently outperforms other approaches, except MemWarp [81]. MemWarp, designed for discontinuity-preserving registration, jointly predicts segmentation masks and deformation fields, enforcing stronger regularization on structural boundaries. While MemWarp achieves marginally higher Dice scores, its deformation fields are significantly less smooth than those produced by EOIR. This demonstrates EOIR’s superior balance between registration accuracy and deformation smoothness.

IV-C5 Handling 2D Multi-modality Natural Images

In addition to medical images, we also demonstrated our EOIR on 2D natural images, using images from the RGB-IR dataset [69]. As shown in Table VI and Fig. 7, despite the modality difference between the moving and fixed images, EOIR can still obtain comparable registration performance to the state-of-the-art approaches in natural image registration (PGMR and Superfusion), highlighting the robustness of our EOIR in handling natural images. Notably, it accomplished this with substantially fewer parameters (reduced 78% of parameters compared with Superfusion), demonstrating both the robustness and efficiency of our framework. We posit that EOIR’s effectiveness stems from the high texture and structural similarity between RGB and IR images, even though they differ in modality. For more complex multimodal scenarios, incorporating a more powerful encoder may be necessary to further enhance performance.

IV-D Results & Analysis on Computational Complexity

In this section, we provide results and analysis of EOIR regarding computational complexity. Two key parameters influence EOIR’s complexity: the number of convolutional layers $n_{c}$ in the encoder, and the start channel $N_{s}$, as depicted in Fig. 5. To provide a more striking comparison, we conduct complexity analysis using the most challenging abdomen dataset, alongside the ACDC dataset, which features much smaller image sizes compared to the other three datasets.

IV-D1 Effects of $n_{c}$

As shown in Fig. 4, increasing $n_{c}$ generally improves registration accuracy. However, starting from $n_{c}=3$, the marginal benefits diminish rapidly with each additional layer. Interestingly, when replacing the encoder with a U-Net, the accuracy decreases despite the increased parameter size. Sub-optimal methods like RDP, utilizing a similar U-Net architecture for pyramid registration, lag behind EOIR in all metrics. This suggests that mono-modal registration benefits more from local intensity linearization than from global bias harmonization, unlike affine registration [73].

IV-D2 Effects of $N_{s}$

Keeping three conv layers intact, we further study the impact of start channels $N_{s}$ on the outcome. We compare EOIR with efficiency-driven methods, FourierNet and LessNet. As shown in Fig.8, starting from $N_{s}=2$ for all methods, EOIR shows a much slower increase in parameter size compared to the other two. While both FourierNet and EOIR exhibit a log-linear increase in accuracy relative to multi-adds and parameter size, EOIR achieves a much better accuracy-efficiency balance than FourierNet. When assessed on the ACDC dataset, as shown in Table II, EOIR’s improvement is more pronounced. EOIR ($N_{s}=8$) surpasses VoxelMorph, the runner-up in terms of complexity, with a significant increase in Dice score while maintaining lower multi-adds and parameter size. Also, EOIR ($N_{s}=8$) reduces the parameter size by 98.8% and multi-adds by 92.9% compared to RDP, the runner-up in accuracy, while achieving a slightly higher Dice score.

IV-D3 Effects of Pyramid Layers $n$

The number of pyramid layers $n$ determines the ability of EOIR to capture large deformation. We plot the curve of the registration Dice with the increasing pyramid layers, as shown in Fig. 9. For intra-patient registration on ACDC, three pyramid layers are sufficient to capture the ED-ES or ES-ED deformation. For inter-patient registration on abdomen images, the registration Dice keeps increasing from 1-5 pyramid layers. Therefore, to get the optimal number of pyramid layers, priors in the registration task should be considered, while a five-layer pyramid can work for most registration tasks. Note that, the multi-scale supervision strategy contributes significantly to accuracy and smoothness: ablating it in favor of a single-scale loss on the top layer alone causes a 1% Dice drop (with SDlogJ from 0.17 to 0.29) in abdominal image registration (Table I, EOIR(-MS loss)). Furthermore, to proactively improve the model’s inherent capability to handle large deformations, the training process itself could be fortified by using a pyramid with an extra level, ensuring that EOIR learns a more robust feature representation across an even wider range of motion.

IV-E Results & Analysis on Smoothness & Diffeomorphism

In this section, we present the results and analysis of EOIR on deformation field smoothness. For the unsupervised setting, we evaluate the ACDC and LUMIR datasets, while for the semi-supervised setting, we focus on the Abdomen CT and OASIS datasets. We emphasize the use of Eq. (7) for composing deformation fields across pyramid levels and time intervals. With $n\times m$ sufficiently small time intervals, each deformation $\phi^{ij}$ can be considered diffeomorphic. The composition is performed by resampling one deformation field by another, ensuring that the resulting deformation remains diffeomorphic, as described in Eq. (7).

IV-E1 Unsupervised

As shown in Table II, all methods, except MemWarp (SDlogJ: 0.11), produced smooth outputs with SDlogJ values below 0.10. Notably, EOIR ($N_{s}=8$) achieves a lower SDlogJ while delivering a slightly better Dice score than the top-performing pyramid-based method, RDP, demonstrating EOIR’s superior handling of local motions during the cardiac cycle. In brain MR registration, the effect of using the proposed composition method in Eq. (7) becomes more evident. As seen in Table IV, additive composition results in significantly more non-diffeomorphic voxels for EOIR, whereas applying Eq. (7) reduces NDV (%) to nearly zero without sacrificing accuracy (see Fig. 10 for a visual example). For other baseline methods, improvements in anatomical alignment are often accompanied by an increase in NDV (%).

IV-E2 Semi-supervised

Semi-supervised learning often incorporates Dice loss to promote anatomical alignment, with the resulting deformation field typically being influenced by the segmentation masks. This can sometimes lead to implausible deformation fields, as the optimization is driven more by the mask alignment than the underlying image data. As shown in Table III, methods including EOIR demonstrate that better anatomical alignment, indicated by higher Dice and lower HD95 (mm), often results in a larger SDlogJ, signaling a less smooth deformation field. For TransMorph, decreasing smoothness regularization leads to increasing Dice and decreasing SDlogJ. Similarly, for EOIR, by simply reducing the start channel to $N_{s}=16$, a similar accuracy-smoothness trade-off is observed. Abodmen CT dataset is more challenging as it has large deformation and limited training samples, and the accuracy-smoothness variations are larger than the OASIS. Therefore, we further study how smoothness regularization strength affect EOIR and other efficiency-driven methods. As shown in Fig. 11, for LessNet and FourierNet, decreasing $\lambda$ reduces field smoothness while increasing Dice. In contrast, EOIR exhibits a distinct behavior due to its use of Eq. (7) for large-deformation diffeomorphic transformation, which imposes stricter requirements on field plausibility at each pyramid level. For EOIR, we found that decreasing $\lambda$ initially increases the Dice score, which peaks at $\lambda=1.0$ before declining. We therefore empirically set $\lambda=1.0$ as it represents the optimal trade-off. Additionally, varying $n_{c}$ and $N_{s}$ changes Dice, but SDlogJ remains around $0.17$, highlighting the effectiveness of Eq. (7) in handling large deformations (see Fig. 6).

IV-F Ablation Study on Modifications

We performed a systematic ablation study to evaluate key components of EOIR using abdominal CT data (Table I). Our separable feature design (M1) proved fundamental compared to combined feature methods, while pyramid analysis (M2, Fig. 9) confirmed the value of multi-scale processing. Removing the Hadamard transform (M3) impaired registration accuracy, and omitting Gaussian smoothing in the multi-scale loss (M4) substantially degraded performance by destabilizing gradient propagation. Optimal efficiency-accuracy balance was achieved with a three-layer encoder (M5, Fig. 4), validating our architectural configuration.

IV-G Discussion

IV-G1 Summary

Overall, EOIR achieves a superior balance between accuracy-efficiency and accuracy-smoothness in deformable image registration. Across five datasets with varying modalities and anatomies, EOIR reduces computational complexity without compromising accuracy. Moreover, the novel deformation field composition method in Eq. (7) enables EOIR to maintain smoother deformation fields while preserving accuracy. The performance gains of EOIR stem from its integration of the Horn–Schunck (H-S) assumption and the Linearization-Harmonization (L-H) assumption into the network design. We propose that a few convolutional layers are sufficient for feature extraction using a Laplacian feature pyramid, effectively linearizing local intensities and harmonizing mono-modal images at each pyramid level. The number of voxels satisfying the H-S assumption in both moving and fixed feature maps increases at each level, promoting displacement propagation within the smoothness regularization.

IV-G2 Advantages

Two major advantages emerge with the design of EOIR. Expansibility: EOIR’s simple architecture, consisting of just a few convolutional layers, makes it highly adaptable to incorporate advanced network modules such as large-kernel convolutions, transformer blocks, co-attention, and other novel structures. For instance, replacing the 3-layer ConvNet encoder with a full encoder-decoder U-net can enhance feature extraction capabilities. Additionally, incorporating larger-kernel convolutions or self-attention modules after the 3-layer ConvNet flow estimator can expand the network’s effective receptive field. Efficiency: EOIR’s efficient design significantly reduces computational complexity without sacrificing accuracy, making it well-suited for large volumetric images and deployment in resource-constrained settings. Specifically, Figure 8 illustrates the trade-off between Dice and computational complexity, measured by Multi-Adds (G) and network parameter size. Runtime comparisons among methods are shown in Table II and Table V, demonstrating that all these learning-based methods can perform fast forward inference for volumetric registration within one second.

IV-G3 Zero-shot Inference Analysis

Beyond the results presented in this manuscript, the zero-shot inference capability of our EOIR framework has been further substantiated across multiple additional image registration tasks, as reported in [9] and the Appendix (under the team name ‘next-gen-nn’). Specifically, EOIR consistently ranks among the top three methods in registration accuracy for inter-subject registration using both the ADNI-1.5T and ADNI-3T datasets. It demonstrates particular strength in subject-to-atlas registration, achieving first place on the NIMH-T1w dataset [58] and second place on both the ADHD [49, 48] and UltraCortex-9.4T [52] datasets. Across all these evaluations, EOIR achieves the lowest NDV among top-ranked approaches, underscoring its capacity to maintain diffeomorphic properties without compromising registration accuracy. Notably, EOIR is the only top-tier method that does not employ a progressive registration strategy, as highlighted in [9]. We posit that this advantage stems from the incorporation of the H–S and L–H assumptions into the network architecture, key elements overlooked by other methods. Their omission results in a less favorable balance between accuracy and computational efficiency, reinforcing EOIR’s value as an elegant and highly effective registration framework.

IV-G4 Broader Impact

Beyond benchmarking accuracy and efficiency, EOIR has broader implications for critical neuroimaging applications that demand precise and stable voxel-to-voxel alignment. Longitudinal image alignment: Accurate deformable registration is fundamental for tracking intra-subject structural changes over time, such as lesion progression in multiple sclerosis or tumor evolution in oncology. By accurately aligning longitudinal images with smooth and efficient deformation fields that account for subject-specific brain atrophy, EOIR improves sensitivity to subtle lesion growth, shrinkage, or transformation, enabling reliable lesion- or tumor-level quantification across time points. This is exemplified in our recent work on longitudinal unique lesion tracking (AULTRA) [86], which also leverages our advances in MS lesion segmentation [87] and filling [88] as preprocessing for EOIR-based registration, followed by unique lesion identification [60] across time points. Atlas construction: Many population-level analyses, including those in quantitative susceptibility mapping (QSM) [77], rely on accurate voxel-wise correspondences across individuals. A future direction is to leverage EOIR for constructing susceptibility and multimodal brain atlases with accurate voxel-wise alignment to study iron deposition, myelin integrity, and other quantitative biomarkers across the cohort.

IV-G5 Limitations

EOIR has two main limitations. First, while EOIR achieves high accuracy and efficiency in mono-modal registration, its performance on multi-modal data is less effective. This occurs because the three-layer convolutional encoder, designed to linearize intensities and harmonize local contrast, lacks the capacity to learn cross-modal feature invariance [73], which requires deeper networks to capture larger contextual regions. For example, when applied to the ThoraxCBCT dataset [32] (registering pre-therapeutic FBCT to low-dose CBCT), EOIR achieves a Dice score of 45%. Replacing the encoder with a full U-Net (while retaining the EOIR framework) increases Dice to 56%, validating the framework’s generalizability but highlighting the three-layer encoder’s trade-off between harmonization capacity and efficiency (details can be found in Table XII in the Appendix). For complex multi-modality image registration, EOIR can also incorporate a more powerful encoder (e.g. encoder from foundation models) to tackle this challenge.

Second, EOIR shows diminished effectiveness when processing images that have fine-grained features, for example, lung nodules or retinal vessels. The reason lies in the feature pyramid design, which incorporates spatial downsampling. During this downsampling procedure, small structures like lung nodules or retinal vessels can be gradually reduced in prominence or even lost. Therefore, at certain pyramid level $i$, the feature pyramid scheme is likely to miss objects with a largest diagonal dimension smaller than $2^{(i-1)}$ voxels. Even if the displacement of these small objects may exceed their own size, this results in relatively poorer performance compared to cascaded methods.

V Conclusion

In this paper, we introduced EOIR, a simple yet efficient image registration network that departs from conventional learning-based approaches by eliminating the decoder and relying solely on an encoder for feature extraction. This streamlined design substantially reduces the number of parameters compared to traditional methods. Extensive experiments across 10 datasets (6 main datasets+validation on 4 datasets) demonstrate that EOIR not only achieves notable improvements in registration accuracy but also reduces network complexity, compared to state-of-the-art methods. EOIR effectively handles both small and large deformations through a novel deformation composition scheme, striking a balance between accuracy, efficiency, and smoothness. With lightweight design and strong performance, EOIR serves as a robust backbone for future developments in more complex registration architectures, offering a solid foundation for scaling in resource-constrained or large volumetric settings.

Acknowledgments

The authors would like to express their sincere gratitude to Aaron Carass for his invaluable insights and constructive comments, which significantly improved the manuscript’s quality.

VI Appendix

VI-A Heatmap Generation of Fig. 1

As part of our methodology for crafting the heatmap depicted in Fig. 1 of the main text, we began by training the EOIR network, with start channels $N_{s}=32$, on the abdomen CT dataset. Subsequently, we randomly selected an image from this dataset as the fixed image and derived the moving image by translating it one voxel anteriorly. Similarly, the three binary squares on the left are all translated with one voxel to the bottom to obtain the moving images, which is why their gradients respond to the vertical direction.

With the new moving and fixed images as inputs, we used the 3-layer encoder to extract a 32-channel feature map. To make the output visually interpretable, we applied Principal Component Analysis (PCA) to reduce the channels to one, retaining only the component with the largest variance. The heatmap was computed using the formula $(\mathbf{I}_{m}(p)-\mathbf{I}_{f}(p))/\frac{\partial\mathbf{I}_{f}}{\partial y}(p)$. Post-processing involved thresholding, where invalid voxels were set to 0 and valid voxels to 1. A 2D heatmap was then generated by averaging all slices along the axis direction and visualized using the ‘viridis’ color map to enhance clarity and interpretation. The untrained heatmap was generated following the same process as described above, but with the encoder initialized randomly.

VI-B Training Details for Each Dataset

During training, all networks were trained for 300 epochs on the Abdomen CT, LUMIR, ACDC, HippocampusMR, and RGB-IR datasets. An extended training period of 700 epochs was used for the OASIS dataset to achieve optimal performance. Unless otherwise specified, the number of pyramid layers $n$ and the scaling-and-squaring step $N_{s}$ were set to 5 and 32, respectively, and the integration step $m$ was fixed to 7 in all experiments. The similarity loss and regularization weight were configured as follows for each dataset:

• •

For the Abdomen CT, HippocampusMR, and OASIS datasets, we used a combination of NCC loss and Dice loss as the similarity measure, with a loss weighting ratio of $L_{\text{NCC}}:L_{\text{Dice}}:R=1:1:1$, where $R$ denotes the smoothness regularization term.

• •

For the ACDC dataset, MSE loss was employed with a regularization weight $\lambda=0.01$, corresponding to a ratio $L_{\text{MSE}}:R=1:0.01$.

• •

On the LUMIR dataset, NCC loss was used with $\lambda=5$ and a learning rate of $4\times 10^{-4}$ ($L_{\text{NCC}}:R=1:5$).

• •

For the RGB-IR dataset, we utilized a combination of L1 loss and perceptual loss (with features extracted using a VGG network) as the similarity objective, weighted as $L_{1}:L_{P}:R=1:1:1$. To handle the two modalities, separate encoders of identical architecture were used to extract features from the RGB and infrared images independently. During training, paired images were warped via random “affine + deformable” deformation fields, following the data augmentation strategy described in [93, 74].

A summary of the experimental setups is provided in Table VII, where NCC, Dice, L1, $L_{P}$, and $\mathcal{R}$ denote the normalized cross-correlation loss, Dice loss, L1 loss, perceptual loss, and smoothness regularization, respectively.

VI-C Ablation Study on the Encoder Design

A detailed analysis of how the number of convolution layers $n_{c}$ in encoder affects registration performance is presented in Table VIII. ‘EOIR-1CONV’ to ‘EOIR-6CONV’ represent models with $1\sim 6$ $3\times 3\times 3$ convolution layers in the encoder. ‘EOIR-0CONV’ uses a untrainable $1\times 1\times 1$ convolution to expand the input channel dimension to match the start channel size, effectively functioning like no convolution, as it barely affect local details. ‘EOIR-UNet’ applies a traditional UNet in the encoder. The registration Dice improves with up to three convolution layers, but adding more than three yields diminishing returns. Using a UNet increases model complexity significantly, yet results in lower accuracy. Thus, three convolution layers strike an optimal balance between registration accuracy and model complexity.

VI-D Boxplot Results in ACDC Dataset and Abdomen Image Dataset

The boxplot results in Figure 12 and Figure 14 present the per-organ Dice and Avg. Dice scores of EOIR and the comparator methods. We compared EOIR against four top-performing methods: MemWarp, FourierNet, LKUNet, and RDP. In cardiac image registration, EOIR achieved a significantly higher Avg. Dice score than all other methods ($p<0.05$, t-test on Dice score). Similarly, in abdominal image registration, EOIR outperformed the other methods in Avg. Dice score ($p<0.05$), except for RDP ($p=0.099$).

VI-E Effects of Varying Start Channels

We investigate the impact of varying the number of start channels $N_{s}$ on registration performance. As shown in Figure 13, Dice scores increase with $N_{s}$ initially but plateau after $N_{s}=32$, accompanied by an exponential rise in model parameters. Thus, we set $N_{s}=32$ for most experiments in this work.

VI-F LUMIR Ranking on Test Phase

In the test phase of the LUMIR challenge, we still won the second ranking, with the detailed top three results in Table. IX. It can be observed that our EOIR achieves similar registration accuracy while keeping a significantly smoother deformation field, compared to the rest approaches.

VI-G Results of Zero-shot Inference

The zero-shot inference capability of EOIR was validated on subject-to-atlas registrations using three distinct datasets (ADNI, NIMH, and UltraCortex), with the models trained exclusively on the LUMIR dataset and applied without any fine-tuning. As detailed in [9] and Tables X, XI, and XII (where EOIR is listed as team ‘next-gen-nn’), our method consistently ranked among the top three in registration accuracy across all benchmarks while producing significantly smoother deformations than those approaches with comparable registration accuracy.

VI-H Results on ThoraxCBCT

The performance of EOIR on the ThoraxCBCT dataset (Table XIII) highlights a key architectural consideration. For this challenging task—which involves aligning pre-therapeutic FBCT with interventional low-dose CBCT—the significant domain gap between modalities necessitates a more powerful feature extractor. We observed that the lightweight EOIR (3 CONV) variant was insufficient for mapping these disparate images into a common feature space. In contrast, an EOIR variant employing a U-Net encoder achieved substantially higher Dice scores, underscoring that encoder capacity is critical for robust performance in complex, multi-modal registration scenarios, even within our streamlined framework.

VI-I Failure Case Analysis

To better understand the scenarios where EOIR may underperform, we visualize two of the worst registration results on the ACDC dataset. These failure cases are shown in Fig. 15. In the ACDC dataset, intra-subject registration is performed from end-diastole (ED) to end-systole (ES) and vice versa. Due to physiological changes between cardiac phases, some voxels in the moving image may lack direct correspondence in the fixed image. For instance, in Case 1, the small dark region in the moving image (highlighted with a red arrow) has no matching voxel in the fixed image. Similarly, in Case 2, certain areas in the fixed image are absent in the moving image (highlighted with a red arrow). In such cases, EOIR still attempts to establish correspondences, which can lead to locally unrealistic deformations (see the yellow box in Case 1). By contrast, in the ES-to-ED registration example, although structural inconsistencies remain, the resulting deformation appears more anatomically plausible. These results suggest that relying solely on voxel-level guidance may introduce locally implausible distortions. Therefore, incorporating anatomical shape priors could be essential for generating physically realistic deformations.