3D-NASE: A Novel 3D CT Nasal Attention-Based Segmentation Ensemble

TL;DR

This paper introduces a new 3D CT segmentation method for nasal structures that improves accuracy and robustness compared to previous techniques.

Contribution

The novel 3D-NASE framework combines CNN and transformer models with voting strategies for enhanced nasal CT segmentation.

Findings

3D-NASE achieved a 35.88% improvement in DICE score over previous methods.

The method reduced the standard deviation by 4.53%, indicating increased robustness.

The approach combines local and global context for better segmentation accuracy.

Abstract

Accurate segmentation of the nasal cavity and paranasal sinuses in CT scans is crucial for disease assessment, treatment planning, and surgical navigation. It also facilitates the advanced computational modeling of airflow dynamics and enhances endoscopic surgery preparation. This work presents a novel ensemble framework for 3D nasal CT segmentation that synergistically combines CNN-based and transformer-based architectures, 3D-NASE. By integrating 3D U-Net, UNETR, Swin UNETR, SegResNet, DAF3D, and V-Net with majority and soft voting strategies, our approach leverages both local details and global context to improve segmentation accuracy and robustness. Results on the NasalSeg dataset demonstrate that the proposed ensemble method surpasses previous state-of-the-art results by achieving a 35.88% improvement in the DICE score and reducing the standard deviation by 4.53%. These promising results highlight the potential of our method to advance clinical workflows in diagnosis, treatment planning, and surgical navigation while also promoting further research into computationally efficient and highly accurate segmentation techniques.

Figures

Full text

1. Introduction

An accurate understanding of nasal cavity and paranasal sinus morphology is vital for clinical applications, disease detection, treatment planning, endonasal surgery simulations, or surgical navigation. For instance, the precise delineation of regions such as the nasal cavity and paranasal sinuses in computed tomography (CT) scans is vital for assessing sinus-related diseases [1], planning endoscopic surgeries [2], and enabling advanced computational models of airflow dynamics [3,4].

Segmentation data are key in diagnosis, surgical preparation, and robotic system navigation. Volume and surface area quantification help assess nasal health and the impact of inhaled substances [5]. Also, detailed anatomical knowledge supports endoscopic sinus surgery, guiding structural reconstruction and outcome evaluation. Although manual segmentation has long been considered the gold standard in clinical practice, it is inherently labor-intensive and prone to both inter- and intra-observer variability. These limitations have fueled the search for robust, automated segmentation approaches that can deliver consistent and reproducible results. In this context, a precise, automated segmentation of anatomical structures in 3D medical imaging becomes critical for effective diagnosis, treatment planning, and surgical navigation [5,6].

Furthermore, the recent surge in radiomics research has underscored the importance of high-quality segmentation as a foundation for extracting reliable quantitative imaging features [7]. Radiomics seeks to convert medical images into high-dimensional data by extracting features that capture subtle tissue heterogeneity and underlying pathophysiological processes. The spectrum of features ranges from classical handcrafted descriptors, such as shape, first-order intensity statistics, and texture measures, to sophisticated representations learned via deep neural networks, including convolutional neural networks (CNNs), recurrent neural networks (RNNs), and transformer-based architectures. Such diversity in feature representation plays a crucial role in enhancing diagnostic accuracy, prognostication, and the prediction of treatment responses.

Moreover, the introduction of novel datasets, such as the NasalSeg [8] dataset, has begun to address the scarcity of large-scale, annotated imaging data necessary for training and benchmarking advanced segmentation models. This work proposes an ensemble-based segmentation framework that synergistically combines multiple 3D architectures, 3D-NASE. Our approach integrates CNN-based models with transformer-based networks that exploit self-attention mechanisms. By fusing these complementary architectures, our method aims to enhance segmentation robustness and improve generalization across anatomical variations and heterogeneous imaging conditions. It aims not only to refine segmentation accuracy but also to provide a reliable basis for subsequent radiomic analysis. In summary, the research question we aim to address in this work is how can an ensemble-based segmentation framework that integrates both CNN-based and transformer-based architectures be designed to enhance the accuracy, robustness, and reproducibility of 3D nasal CT segmentation for improved clinical diagnosis and treatment planning?

The remainder of the paper is organized as follows. In Section 2, we review the state-of-the-art radiomic segmentation by discussing both 2D and 3D approaches and the emerging trends and challenges in the field. Section 3 details the materials and methods used in our study, including the architectures of U-Net, 3D U-Net, UNETR, and Swin UNETR, along with the description of the NasalSeg dataset and the evaluation metrics. In Section 4, we introduce 3D-NASE and outline the experimental setup. Section 5 presents the quantitative experimental results, while Section 6 offers a qualitative analysis of the segmentation outputs. We discuss the limitations of our approach in Section 7 and conclude the paper with final remarks and directions for future work in Section 8.

2. Related Work

Over the past decade, significant progress has been made in radiomic segmentation through contributions from traditional image processing, machine learning, and deep learning methodologies. This section provides a comprehensive review of the state-of-the-art segmentation techniques in radiomics. The discussion is organized into four subsections, with a focus on the 2D (Section 2.1) and 3D (Section 2.2) segmentation approaches, along with a discussion of their distinctions and complementarities (Section 2.3), and future developments (Section 2.4).

2.1. Two-Dimensional Radiomic Segmentation

Two-dimensional segmentation techniques have been extensively explored due to their simplicity and relatively low computational burden. Early methods predominantly relied on classical image processing techniques such as thresholding, region growing, edge detection, and active contour models [9,10]. These methods proved effective when target structures exhibited high contrast and well-defined boundaries.

More recently, the emergence of deep learning models has shifted the common approaches toward more automatic and fast architectures capable of segmenting clinically relevant regions in different imaging techniques [11,12].

However, despite their computational efficiency and ease of implementation, 2D segmentation methods inherently lack the ability to fully exploit the volumetric context present in modern imaging modalities [13]. This shortcoming limits their utility in applications for which three-dimensional structural continuity [14] is critical, such as in tumor delineation or the assessment of complex anatomical regions.

2.2. Three-Dimensional Radiomic Segmentation

High-resolution volumetric imaging has spurred a paradigm shift toward 3D segmentation methods. Unlike 2D approaches, 3D segmentation takes advantage of the complete spatial context, resulting in a more comprehensive and accurate delineation of anatomical structures [13,15]. Volumetric methods are particularly indispensable in modalities like CT and magnetic resonance (MR) imaging, where the intricate spatial relationships between tissues must be preserved.

Deep learning has become the cornerstone of modern 3D segmentation [16,17]. The 3D U-Net architecture [18], for example, has emerged as a benchmark due to its encoder–decoder structure, which facilitates the capture of both local and global contexts across multiple scales. In addition to CNN-based models, transformer-based architectures have recently been introduced to model long-range dependencies across the volume [19]. With their self-attention mechanisms, these models are adept at identifying subtle variations and heterogeneities within the data, which are often missed when conventional convolutional filters are used [20].

Hybrid approaches have also been proposed, combining the rapid processing of 2D methods with the spatial accuracy of 3D segmentation [21,22]. Typically, these frameworks commence with a 2D segmentation step to quickly identify candidate regions, followed by a refined 3D segmentation to ensure volumetric consistency. This two-tiered strategy effectively balances the trade-offs between computational efficiency and segmentation accuracy [23], making it highly attractive for real-time clinical applications [24].

2.3. Comparative Analysis and Challenges

A comparative analysis between 2D and 3D segmentation techniques reveals a range of trade-offs. While 2D methods offer simplicity and speed [25,26], they are limited by their inability to capture inter-slice contextual information. On the other hand, 3D approaches, despite their higher computational demands, provide a more faithful representation of anatomical structures [27,28]. One of the primary challenges with 3D segmentation is the substantial computational overhead associated with processing volumetric data. High-resolution images require large amounts of memory and extended training times, often necessitating specialized hardware or cloud-based resources.

Another significant challenge is the variability in imaging protocols and acquisition parameters across different clinical centers. This variability can lead to discrepancies in image quality and intensity distributions, thereby affecting the generalizability of segmentation models [29]. To address these issues, recent research has focused on developing standardized preprocessing pipelines and data harmonization techniques essential for mitigating inter-scanner and inter-center variations.

Furthermore, the integration of segmentation outputs into radiomic pipelines poses additional challenges. High-quality segmentation is a prerequisite for reliable feature extraction, as errors in delineation can propagate and degrade the performance of downstream predictive models [30]. As such, the development of explainable and robust segmentation methods is a critical area of ongoing research.

2.4. Emerging Trends and Future Directions

The field of radiomic segmentation is rapidly evolving, driven by innovations that bridge the gap between traditional methods and modern deep learning techniques. One emerging trend is the integration of self-supervised learning and foundation models [31,32], which aim to leverage large-scale unannotated data to improve model robustness and reduce the reliance on extensively labeled datasets. These approaches are particularly promising for addressing the limitations posed by data scarcity in medical imaging [33,34].

Another exciting direction is the development of multi-modal segmentation frameworks. By fusing complementary information from different imaging modalities such as CT, MR, and positron emission tomography (PET), researchers are beginning to overcome the limitations inherent in single-modality analysis. Multi-modal fusion techniques can enhance the accuracy and reliability of segmentation, particularly in complex clinical scenarios where different modalities provide unique and complementary insights [35,36].

3. Materials and Methods

In this section, we present the materials and methods employed in our study, focusing on the architectures used for medical image segmentation (Section 3.1, Section 3.2, Section 3.3 and Section 3.4). We detail these networks’ theoretical foundations and practical adaptations to handle volumetric data, which is essential for accurately delineating complex anatomical structures. In addition, we provide a dataset description in Section 3.9 and the metrics definition in Section 3.10.

3.1. U-Net

U-Net [37] is a popular biomedical image segmentation architecture with an encoder–decoder structure. In the encoding stage, the network progressively reduces the spatial dimensions of the input through a series of convolutional layers followed by a downsampling operation. For instance, if $[eqn]$ denotes the feature map at layer l, a typical downsampling can be expressed as

$$[eqn]$$

where D might represent a max pooling operation (e.g., with a stride of two) or a convolution with a stride greater than one, the selected U-Net architecture uses strided convolution as the downsampling operator. This reduction not only decreases computational complexity but also helps in capturing contextual information over larger regions. In the decoding stage, the spatial resolution is gradually recovered using an upsampling operation. Mathematically, if $[eqn]$ is the feature map at a deeper layer in the decoder, then upsampling can be described as

$$[eqn]$$

where U may be implemented via transposed convolutions or interpolation, followed by convolution. The network further refines the segmentation by incorporating skip connections that merge the high-resolution features from the encoder with the upsampled features in the decoder, thereby preserving fine structural details.

3.2. Three-Dimensional U-Net

Three-Dimensional U-Net [18] extends the principles of U-Net to volumetric data by replacing two-dimensional operations with their three-dimensional counterparts. In this architecture, the downsampling operator is adapted to process volumetric feature maps. If $[eqn]$ is the 3D feature map at layer l, the volumetric downsampling is represented as

$$[eqn]$$

where $[eqn]$ aggregates information along the depth, height, and width dimensions simultaneously. Similarly, the upsampling operator in the decoder is defined as

$$[eqn]$$

which reconstructs the spatial dimensions of the volume, again using transposed convolutions or interpolation methods adapted to three dimensions. The integration of skip connections in 3D U-Net allows the network to effectively combine coarse, context-rich features with finer details, thereby enabling the accurate segmentation of complex anatomical structures in modalities such as CT and MRI.

3.3. UNETR

UNETR [19] integrates a transformer-based [38] encoder into the U-Net framework to capture long-range dependencies in volumetric data. The input volume x is first divided into non-overlapping patches, each of which is embedded into a token via a linear projection, i.e.,

$$[eqn]$$

where $[eqn]$ is the ith patch, and $[eqn]$ denotes the embedding function. The transformer encoder then processes the sequence of tokens using self-attention, which is computed as

$$[eqn]$$

with Q, K, and V being the query, key, and value matrices, respectively, and $[eqn]$ the key dimension. The resulting encoded representation is reshaped and fed into a U-Net style decoder that recovers spatial resolution through an upsampling operation

$$[eqn]$$

while incorporating skip connections to merge high-resolution features from the encoder. This design enables UNETR to effectively combine global context with local details. In our work, we follow the original UNETR implementation [19]. Specifically, feature maps are extracted from five distinct layers of the transformer encoder. These multi-scale features are fused in the upsampling branch by first applying deconvolution to match the spatial dimensions of both the backbone and the target feature maps. This is subsequently refined through a sequence of convolutional operations, batch normalization, and ReLU activations, which collectively improve the training process.

3.4. Swin UNETR

Swin UNETR [20] extends the UNETR framework by employing the Swin Transformer [39] architecture, which utilizes a hierarchical structure with window-based self-attention. In this model, the input volume is partitioned into patches, and self-attention is computed locally within non-overlapping windows. The attention mechanism is the same as defined in [19] applied independently over each window. A subsequent shift in the window partitioning enables the capture of cross-window dependencies, ensuring that global context is effectively modeled. Following the transformer encoding, a decoder analogous to that of UNETR performs upsampling. This approach synergizes local attention with global context, thereby improving the segmentation of complex volumetric data [20].

3.5. SegResNet

SegResNet [40] is a modification of the traditional ResNet architecture tailored to semantic segmentation tasks. This architecture incorporates residual connections, which help mitigate the vanishing gradient problem by allowing gradients to flow through the network without degradation. The encoding path consists of a series of extensive residual blocks that learn multiscale features from the input images, progressively reducing the spatial dimensions. The unique aspect of SegResNet is its autoencoder portion of the architecture, which improves the final segmentation prediction according to [40]. In our implementation, we only rely on the segmentation loss and not on the reconstruction.

3.6. DAF3D

DAF3D [41] introduces a novel approach for robust 3D medical image segmentation by utilizing deep attentive features within a convolutional neural network framework. The architecture leverages attention mechanisms to automatically focus on relevant parts of the input volumes, enhancing segmentation performance by prioritizing critical features while suppressing less informative ones. The network operates through a series of convolutional layers that capture intricate patterns in the volumetric data, supported by a dedicated attention module that refines feature representations based on channel-wise and spatial relationships.

3.7. V-Net

V-Net [42] employs a fully convolutional neural network architecture specifically designed for volumetric segmentation tasks. The structure follows a similar encoder–decoder paradigm while utilizing 3D convolutions to process volumetric data directly. In the encoder portion, V-Net incorporates residual learning blocks that facilitate the training of deeper networks by enhancing gradient flow. The decoder then upsamples the feature maps via transposed convolution and combines them with high-resolution features from the encoder via skip connections, allowing for precise localization.

3.8. Advantages and Disadvantages of the Proposed Networks

Our study utilizes six different 3D segmentation architectures: 3D U-Net, UNETR, Swin UNETR, SegResNet, DAF3D, and V-Net. Among these, the 3D U-Net, V-Net, SegResNet, and DAF3D are CNN-based models that rely on convolutional operations. These models are renowned for their robust ability to extract fine-grained local features and are computationally efficient. However, their dependence on convolutions also results in a limited receptive field, which can restrict the capture of long-range contextual information. This limitation may lead to challenges such as over-segmentation, particularly in complex anatomical regions.

In contrast, UNETR incorporates transformer-based techniques that excel at modeling global contexts. By leveraging self-attention mechanisms, UNETR can capture long-range dependencies that are crucial for understanding subtle variations and complex spatial relationships in volumetric data. Although this approach improves the representation of broader contextual information, it comes at the cost of increased computational complexity and demands a larger training dataset. The process of tokenizing volumetric data adds further preprocessing intricacies, which can be challenging compared to traditional CNNs [43].

Swin UNETR builds upon the strengths of both convolutional and transformer architectures by incorporating a hierarchical structure with window-based self-attention. This design allows the model to effectively capture local patterns within non-overlapping windows, while also integrating global context through a shifting mechanism. Swin UNETR thus achieves a balance between detailed feature extraction and comprehensive contextual integration. However, this balance introduces additional model complexity and requires careful tuning of window sizes and shift parameters to optimize performance [43].

The ensemble framework we propose—3D-NASE—combines these six networks to take advantage of their complementary benefits, also following consolidated literature works that previously combined the characteristics of both model typologies [44]. By integrating the local feature precision and global feature from both CNN and transformer-based models, our ensemble is designed to counteract the individual limitations of each architecture. The combination, implemented via majority and soft voting strategies, ensures improved segmentation accuracy and robustness across diverse anatomical variations and imaging conditions, ultimately enhancing the overall performance of the segmentation task.

3.9. Dataset

The NasalSeg dataset [8] is a large-scale, open-access resource developed for the automatic segmentation of the nasal cavity and paranasal sinuses from 3D CT images. It comprises 130 head CT scans collected from independent patients, including 74 males and 56 females, with an age range of 24 to 82 years (mean $[eqn]$ years). All scans were acquired using a Biograph 64 scanner (Siemens, Erlangen, Germany) at the Department of Nuclear Medicine/PET Center, Huashan Hospital, and they typically exhibit a volume of $[eqn]$ pixels, with an in-plane resolution of $[eqn]$ mm and a slice spacing of 1.5 mm.

Each scan is accompanied by pixel-wise annotations of five anatomical structures: the left nasal cavity, right nasal cavity, nasal pharynx, left maxillary sinus, and right maxillary sinus. These annotations were initially performed by three skilled annotators with one to five years of experience and subsequently refined and verified by senior experts, with five to ten years of experience, to ensure high accuracy and consistency. The manual annotation and refined segmentation procedure was adopted from 3D Slicer 5.6.2 (3D Slicer: www.slicer.org/, accessed on 6 March 2025). The air threshold was set to −400 to 1000 Hounsfield units (HU) for all scans. The Laplacian filter was then used to highlight regions with rapid intensity changes to enhance edges and facilitate the segmentation. The annotations were verified by four experienced experts specializing in radiology and otolaryngology (with over 20 years of experience).

Furthermore, the dataset has been released as organized into five pre-defined folds to support cross-validation and robust benchmarking of segmentation algorithms. Each set consists of 104 cases for training and 26 cases for validation.

We report samples of the NasalSeg dataset along with over-imposed annotations in Figure 1.

3.10. Metrics

We evaluate our 3D segmentation results using several quantitative metrics that capture different aspects of performance. For a binary segmentation task, we define the following basic terms: $[eqn]$ represents true positives (the number of correctly identified positive elements), $[eqn]$ represents true negatives (the correctly identified negative elements), $[eqn]$ represents false positives (negative elements incorrectly labeled as positive), and $[eqn]$ represents false negatives (positive elements that were missed). In multi-class segmentation, these metrics are computed for each class individually and then averaged to yield an overall performance score without weighting the number of items in each class. This strategy ensures that performance is fairly assessed even when class distributions vary, and, in the context of this work, this decision was made to ensure that each anatomical structure is treated equally during evaluation.

The quality of the segmentation overlap is measured by the DICE coefficient, which is calculated as

$$[eqn]$$

This metric is widely adopted in medical image segmentation because it effectively captures the degree of overlap between the predicted segmentation and the ground truth. A DICE score of 1 indicates perfect overlap, while scores closer to 0 reflect poor agreement. The formulation penalizes both FPs and FNs equally, making it a robust indicator of segmentation quality.

Another critical measure is the intersection over union (IoU), which, for a single class, is defined as

$$[eqn]$$

The IoU quantifies the ratio of the intersection between the predicted and actual segmentation regions relative to their union, offering an intuitive measure of error when extra regions are included or parts of the target are missed. For multi-class segmentation, the mean intersection over union (mIoU) is calculated by averaging the IoU values across all classes:

$$[eqn]$$

where C is the total number of classes, and the subscript c indicates that the metric is computed for each class separately. This aggregate metric is particularly useful in evaluating the overall performance across diverse anatomical structures.

To further analyze segmentation performance, we employ sensitivity (or recall) to assess the proportion of actual positive elements that are correctly identified:

$$[eqn]$$

A high sensitivity is crucial, especially in medical applications, because it ensures that most of the true positives are captured, thereby reducing the risk of missing critical regions.

Complementary to sensitivity, specificity measures the proportion of actual negative elements that are correctly identified:

$$[eqn]$$

High specificity is important for minimizing the inclusion of irrelevant regions in the segmentation, which is particularly critical when false positives can lead to misinterpretations.

Precision, also known as positive predictive value (PPV), evaluates the accuracy of the positive predictions:

$$[eqn]$$

This metric indicates the proportion of predicted positive elements that are indeed correct. High precision reflects the reliability of the segmentation in not overestimating the presence of the target structure.

Similarly, the negative predictive value (NPV) assesses the correctness of the negative predictions:

$$[eqn]$$

A high NPV ensures that the segmentation method reliably identifies non-target areas, which is essential for maintaining overall segmentation integrity.

Finally, the overall performance of the segmentation is summarized by the accuracy metric:

$$[eqn]$$

Accuracy measures the proportion of correctly classified elements (both positive and negative) out of all elements. Although it provides a global performance view, it should be interpreted alongside the other metrics, especially in scenarios with class imbalance.

Together, these metrics offer a comprehensive framework for assessing segmentation performance. They not only quantify the spatial overlap between the predicted and ground truth segmentations but also provide insights into the balance between correctly identifying positive elements and avoiding erroneous classifications. This multi-faceted evaluation is critical for refining segmentation algorithms, particularly in high-stakes applications such as medical imaging.

4. The Proposed Framework: 3D-NASE

Ensemble learning has emerged as a promising strategy for 3D segmentation in medical imaging due to its ability to reduce model variance, improve generalization, and alleviate artifacts commonly encountered in high-dimensional volumetric data. In our proposed framework, 3D-NASE, we harness the complementary strengths of both CNN-based and transformer-based architectures by integrating six state-of-the-art segmentation models: 3D U-Net, UNETR, Swin UNETR, SegResNet, DAF3D, and V-Net. Each of these models is specifically designed to capture distinct aspects of the data—while the CNN-based networks 3D U-Net, SegResNet, DAF3D, and V-Net excel at extracting fine-grained local details, the transformer-based models UNETR and Swin UNETR are adept at modeling long-range dependencies and global contextual information via self-attention mechanisms. By processing the same CT volume in parallel, these models generate segmentation outputs that are subsequently aggregated using both majority and soft voting strategies. This dual ensembling approach leverages diverse feature representations across different architectures, effectively balancing their individual strengths and limitations. Figure 2 schematically illustrates our pipeline, where the input volume is processed by the selected models, their outputs concatenated, and an ensemble strategy is applied to generate the final prediction.

Experimental Setup. We conducted our experiments on a workstation equipped with an RTX 4060 Ti GPU, 16GB VRAM, and an Intel Core i5-13400 processor. All models were trained for 800 epochs with a learning rate of $[eqn]$ , and the best models were selected based on the highest DICE score on the validation set. 3D-NASE was built upon the MONAI repository [45]. For optimization, we employed AdamW, together with the DICE loss function.

Data augmentation was an integral part of our training process. Specifically, we utilized two common strategies: random cropping and intensity normalization. For random cropping, a random volume crop with a region of interest (ROI) of $[eqn]$ voxels was extracted from each input image. After an analysis performed over the size of the images, it was identified that an ROI of $[eqn]$ allows a significant portion of the anatomical structure of interest to be captured while reducing the variability introduced by irrelevant or background regions. Intensity normalization was performed on a per-channel basis by considering only non-zero voxels, scaling their intensities to have a mean of 0 and a standard deviation of 1. We do not consider other augmentation strategies in order to focus our results analysis on the impact of the ensemble techniques for a more stable prediction framework, rather than artificially improving upon the quantity of the data towards the same goal.

5. Experimental Results

This section presents the experimental evaluation conducted and the results obtained. We evaluated the employed architectures as stand-alone and the proposed ensemble-based framework across different folds since every experiment uses a five-fold cross-validation approach to ensure a fair assessment and mitigate potential biases. The complete set of experiments across folds is presented in Table 1, where we also compared the results with the baseline provided in [8]. As can be seen, the proposed soft voting approach improves the DICE score by $[eqn]$ with respect to the baseline ( $[eqn]$ against $[eqn]$ ) and reduces the standard deviation by $[eqn]$ (from $[eqn]$ of the baseline to $[eqn]$ ), thereby highlighting enhanced adaptability and stability under varying training conditions. Moreover, the ensemble strategy achieves a $[eqn]$ improvement over Swin UNETR, the best stand-alone model. Although the improvement is marginal, these results illustrate that leveraging the diverse knowledge acquired by different models leads to more reliable outcomes.

Notably, the current state of the art for the NasalSeg dataset [8] exhibits lower performance than individual models and ensemble strategies regarding the DICE score and demonstrates higher variability across folds.

Extending the evaluation to a broader range of segmentation metrics, Table 2 demonstrates that 3D-NASE precisely segments the relevant regions, as indicated by high accuracy, specificity, and NPV; however, it tends to under-segment the analyzed volumes, except for the nasal pharynx, which suffers from over-segmentation. These insights are further supported by the results provided in Table 3, which shows the DICE score for the five different classes.

Among the classes with similar performance, the right and left nasal cavities are the most challenging to segment, likely due to intra-observer differences observed in the model predictions. This aspect is evidenced by the average DICE score and the higher standard deviation for these two classes, which is an order of magnitude greater than that for the other classes.

6. Qualitative Results

The qualitative results are presented in Figure 3. All models exhibit a tendency toward over-segmentation for the nasal pharynx. This issue is particularly evident in 3D U-Net and UNETR, which also often predict small, scattered label fragments across different image regions. These fragmented predictions result in artifacts and inconsistencies in the final segmentation masks.

The proposed ensemble methods effectively mitigate this issue by filtering out small-scale imperfections, leading to more refined and coherent segmentation. However, larger over-segmentation artifacts tend to persist even in the ensemble predictions, indicating that, while ensemble strategies improve overall segmentation quality, they may not entirely eliminate over-segmentation when it occurs on a larger scale.

The qualitative results presented in Figure 3 align with the observations made during the discussion of the quantitative results. Specifically, the regions most affected by segmentation errors are the right and left nasal cavities. These structures are either misclassified as other anatomical classes, as seen in the U-Net prediction for the second image, or other classes are mistakenly predicted as nasal cavities, as observed in the U-Net prediction for the third image.

These findings further reinforce the quantitative evaluation, highlighting the challenges associated with segmenting the nasal cavity regions. This suggests that their complex shape, small size, and similarity in intensity to surrounding structures contribute to the models’ difficulty in accurately distinguishing them.

7. Limitations and Future Aspects

While our proposed framework for nasal segmentation demonstrates significant improvements over the current state of the art, it also presents some limitations.

The first limitation of our approach is the modest improvement provided by the ensemble techniques over the Swin UNETR architecture. While the soft voting ensemble achieves a DICE score of 94.43%, an incremental gain of 0.54 compared to Swin UNETR—a difference that is not statistically significant, but, as shown in Figure 3, the ensemble notably enhances the prediction quality by substantially reducing artifacts.

It still exhibits artifacts and instances of over-segmentation and under-segmentation that vary across different anatomical classes. Such inconsistencies can compromise the reliability of the segmentation outputs, particularly when delineating complex or subtle structures. To improve these aspects, we plan further investigations. For instance, implementing post-processing steps such as conditional random fields (CRFs) or connected-component analysis can be helpful in removing scattered label fragments and refining the borders of segmented regions. Additionally, experimenting with loss functions like Tversky loss may better balance false positives and negatives during the training phase.

Another critical limitation is the necessity for a large number of training epochs to achieve satisfactory results. This high computational demand can hinder accessibility, especially for smaller institutions or research groups with limited resources in a fine-tuning scenario. The extensive training requirements not only prolong the development cycle but also increase the overall cost of deployment. To take this issue into account, we plan to explore techniques such as progressive learning rate scheduling and early stopping to reduce training duration without compromising performance.

Moreover, while the ensemble methodology effectively combines CNN- and transformer-based architectures, it requires all six selected models to be trained on the same dataset and perform inference on the same volume. This integrated approach further compounds computational and memory resource requirements, posing a significant challenge in scenarios where real-time performance is essential. Moreover, the marginal performance improvement warrants further exploration. For instance, we intend to assess alternative ensembling methods, such as stacked generalization or weighted voting schemes, for which model predictions are weighted based on performance per anatomical region. This could lead to a more balanced segmentation performance across different regions.

Finally, since CT images can vary due to differences in acquisition protocols or patient positioning and the absence of further public datasets dealing with this task, we will simulate varying imaging conditions during training and testing. Introducing synthetic variations (e.g., changes in resolution or noise levels) will assess the model’s robustness under different scenarios.

Future work should focus on optimizing training strategies to reduce the number of required epochs, as well as exploring more resource-efficient ensemble methods. By addressing these issues, it may be possible to maintain or even enhance segmentation performance while reducing the computational burden, thus broadening the applicability of the framework in resource-constrained environments.

8. Conclusions

In this work, we have introduced 3D-NASE, an ensemble-based segmentation framework that integrates the complementary strengths of CNN and attention-based architectures for 3D nasal CT segmentation. By merging predictions from 3D U-Net, UNETR, Swin UNETR, SegResNet, DAF3D, and V-Net through majority and soft voting, our method effectively captures both fine-grained details and long-range contextual information. Experimental results on the NasalSeg dataset reveal that our ensemble approach not only outperforms individual models but also improves the overall state-of-the-art segmentation performance, with a notable $[eqn]$ increase in the DICE score and a $[eqn]$ reduction in variability across cross-validation folds. Despite these significant gains, challenges such as over-segmentation artifacts and high computational demands persist. Future work will focus on optimizing the training pipeline and exploring more resource-efficient ensemble strategies to further enhance segmentation accuracy and efficiency, thus broadening the clinical applicability of the proposed framework.