Hyperspectral imaging for intraoperative brain tumor identification through fusion of spectral, textural, and spectral index features

TL;DR

This paper introduces a new hyperspectral imaging method that improves brain tumor detection during surgery by combining multiple image features.

Contribution

The novel contribution is a hyperspectral image detection algorithm that fuses spectral, textural, and spectral index features for better tumor boundary determination.

Findings

The three-feature fusion model achieved higher classification accuracy than models using fewer features.

Hyperspectral imaging effectively distinguishes tumors from surrounding tissue during surgery.

Abstract

Brain tumor is a common neurological surgical disease, where surgical resection is the primary treatment method. Neurosurgeons need to accurately determine the location of the tumor during tumor resection surgery, but existing clinical tumor identification technologies face numerous challenges, such as high equipment costs, long processing times, a certain degree of invasiveness, and insufficient image clarity. In this work, we propose a hyperspectral image detection algorithm based on the fusion of multiple features to maximize the determination of tumor boundaries. The algorithm establishes the machine learning models of Support Vector Machine (SVM) and Random Forest (RF) by integrating data features from optimal wavelengths, spectral indices, and textural features. Experimental results show that on different datasets, the classification accuracy of the three-feature fusion model is significantly higher than that of models using only two features or a single feature. Hyperspectral tumor image recognition can effectively help distinguish the tumors from the surrounding tissue, thereby enhancing the safety and thoroughness of tumor surgery.

Figures

Data preprocessing flowchart.(a) Medical Dataset’s Spectral Information (b) Removal of noisy spectral bands.

DGWCR flowchart.

Weight map of 38 spectral indices obtained from the relief algorithm.

Main flowchart of the hyperspectral tumor image classification process.

Prediction effect diagrams of SVM with 8-01 dataset based on different feature inputs.(a) False color image. (b) Ground truth image. (c) OWS. (d) SIS. (e) TFS. (f) OWS + SIS. (g) OWS + TFS. (h) SIS + TFS. (i) All features combined.

Color images of living human brain tissue from three patients, their corresponding grayscale images, and SVM detection effect diagrams with three-feature fusion.(a, b, c) Images from patient 12-01. (d, e, f) Images from patient 12-02. (g, h, i) Images from patient 20-01.

Prediction effect diagrams of RF with 20-01 dataset based on different feature inputs.(a) False color image, (b) Ground truth image, (c) OWS, (d) SIS, (e) TFS, (f) OWS + SIS, (g) OWS + TFS, (h) SIS + TFS, (i) All features combined.

Color images of living human brain tissue from three patients, their corresponding grayscale images, and RF detection effect diagrams with three-feature fusion.(a, b, c) Images from the 8-01 dataset. (d, e, f) Images from the 12-01 dataset. (g, h, i) Images from the 12-02 dataset.

SVM and RF detection accuracy on different datasets: (a) 8-01, (b) 12-01, (c) 12-02, (d) 20-01.

Equations

Full text

Introduction

Surgical resection is the conventional treatment for brain tumors, but it requires doctors to identify and remove tumor tissue as accurately as possible from normal brain tissue, otherwise it not only increases the risk of complications such as intracranial bleeding and infection but can also lead to tumor recurrence [1–3]. Conversely, excessive resection may alter the distribution of intracranial pressure, causing symptoms like headaches, nausea, vomiting, and even lead to severe consequences such as paralysis, loss of sensation, and speech disorders.Therefore, precise identification of tumor location during surgery is crucial for assisting doctors in resection, which can minimize damage to normal brain tissue [4]. Currently, the main technologies used in clinics to identify tumor locations are surgical navigation [5], intraoperative ultrasound [6,7], and intraoperative biopsy [8]. But they have their respective limitations. Surgical navigation systems are costly and rely on preoperative images, which may not adapt in real-time to anatomical changes that occur during surgery; intraoperative ultrasound provides non-standard images that differ from the standard images familiar to neurosurgeons; intraoperative biopsies can cause damage to the patient, are time-consuming, and carry the risk of false negatives.

Given the shortcomings of existing technologies, a cost-effective, accurate, and non-invasive technology is needed to develop for tumor resection [9]. Hyperspectral Imaging (HSI), as an emerging imaging technology, can generate medical images with one-dimensional spectral information and two-dimensional spatial information [10] by capturing the reflective or transmissive characteristics of tissue across multiple spectral bands. Benefiting from the advantage of broad spectral range and high resolution, HSI can provide detailed information for identifying tumor locations at the molecular and cellular levels. In recent years, the application of hyperspectral technology in cancer detection has become increasingly widespread [11]. For example, Huang Yi proposed a supervised least squares support vector machine segmentation method based on feature spectra for the segmentation of malignant melanocytes, achieving a segmentation accuracy of 85% in experimental samples of cutaneous melanoma. Leon and colleagues developed a dermatological HSI acquisition system and proposed a semi-supervised algorithm-based processing framework for the automatic identification and classification of PSL, achieving the diagnosed results with 87.5% of sensitivity and 100% of specificity. These studies indicate that the combination of hyperspectral imaging and machine learning technology is expected to provide an accurate technology for determining the boundaries of intraoperative brain tumors [12,13].

In this study, we proposed a hyperspectral imaging detection framework for brain tumor based on the fusion of multiple features to achieve precise judgment of tumor location and assist doctors in surgical resection. We first processed the data in three stages: using the DGWCR algorithm to select the optimal wavelengths (OWS), utilizing the Relief algorithm to choose appropriate spectral indices (SIS), and collecting textural features (TFS) through the IAPS algorithm. Then, we introduced different feature fusions into the SVM and RF detection models to achieve pixel-level detection of medical brain cancer hyperspectral data images. The main contributions of this study are as follows:

(1) This is the first time that the classification performance of machine models has been enhanced through a three - feature fusion method.(2) The algorithm framework only requires 5 - pixel training, which solves the problem of the scarcity of medical datasets.(3) Compared with neural networks, the algorithm framework maintains high precision while having lower training costs.

Methods

Dataset

Ethics statement.

This study was conducted using a publicly available hyperspectral medical imaging dataset. All data were fully anonymized prior to release, and no personally identifiable information was included. The use of the dataset complied with the terms and conditions specified by the data provider. Therefore, informed consent and additional ethical approval were not required.

Patient tumor characteristics and image.

The experiments were conducted using data gathered between March 2015 and June 2016 from the European project "HELICoiD" (Hyperspectral Imaging for Cancer Detection) at the University of Las Palmas de Gran Canaria in Spain. The dataset [14] comprised in vivo Multispectral Hyperspectral Imaging System (MHSIS) images of human brains, obtained during neurosurgical operations at the University Hospital of Gran Canaria. Patients, aged 18 and above, diagnosed with either primary or secondary brain tumors, including those with Grade IV Glioblastoma Multiforme (GBM), were involved in this study.During the surgery, a craniotomy was performed to expose the brain’s surface, followed by the acquisition of HS images using a hyperspectral imaging system. Based on the guidance from the Image-Guided Surgery (IGS) system, rubber ring markers were strategically placed to delineate areas of tumor and normal tissue.

The datasets, labeled 8-01, 12-01, 12-02, and 20-01, consisted of four hyperspectral medical images from three GBM patients. The first three digits of each dataset identifier corresponded to the patient’s unique number, while the last two digits denoted the image sequence within that patient’s set. Leveraging the expertise of neurosurgeons, pixels within each MHSI image were categorized and labeled to produce a reference image, distinguishing four classes: Tumor Tissue (TT), Normal Tissue (NT), Blood Vessel (BV), and Background (BG). The Background class specifically referred to the non-biological, disinfected rubber rings used for tumor and healthy brain area identification. A detailed pixel count for each dataset was presented in Table 1, with color-coded labels for Normal Tissue in green, Tumor Tissue in red, Blood Vessel in blue, and Background in yellow.

Table 1: Pixel quantity information for the four datasets.

For the model training, a random sampling method was employed, selecting five pixels per class to constitute the training set, thus streamlining the sampling process and ensuring a representative balance within the training samples. The remaining pixels were allocated to the test set for the evaluation of the trained model’s performance [15].

Image preprocessing.

The experiments utilized a Hyperspec VNIR series camera (HeadWall Photonics Inc., Fitchburg, MA, USA) for image generation. This camera is capable of capturing images within the spectral range of 400 to 1000 nm, with a sampling interval of 0.73 nm, as shown in Fig 1. The Hyperspectral Imaging system captures 826 spectral channels, covering the visible and near-infrared (VNIR) regions, with a spectral resolution of 2-3 nm. However, due to the lower performance of the HS sensor on certain channels, we excluded the extreme spectral channels, namely the first 56 and the last 126 channels, resulting in the use of 645 remaining spectral channels, as shown in Fig 1.

Data preprocessing flowchart.(a) Medical Dataset’s Spectral Information (b) Removal of noisy spectral bands.

Lighting conditions in the natural environment are constantly changing [16], and hyperspectral acquisition sensors respond differently to light, leading to significant environmental variations in the collected hyperspectral data [17]. To avoid the spectral non-uniformity of illumination devices and the issue of dark current generated by camera sensors, as well as to eliminate the impact of external lighting conditions, we use Eq 1 to calibrate the image reflectance on a per-pixel basis, where $[eqn]$ represents the white reference image obtained from a standard whiteboard, and $[eqn]$ represents the dark reference image obtained with the camera shutter closed. By preprocessing to address the spectral non-uniformity of the illumination device and the dark current generated by the camera sensor, we remove noise from the spectral features and reduce the number of bands in the samples without losing relevant spectral information [18].

$$[eqn]$$

Feature extraction and fusion

Band selection.

Even after removing the interfering bands at the beginning and end of the spectral bands in hyperspectral images, due to the high correlation between bands in hyperspectral images, the processed medical hyperspectral data still suffers from data redundancy [19,20]. We decided to use a band selection algorithm to select more representative bands, retaining the useful information carried by the hyperspectral data, thereby reducing computational costs and time. Here, we chose the Data Gravitation and Weak Correlation-based Ranking algorithm [21] for band dimension reduction.

The DGWCR algorithm, compared to traditional algorithms, can compress the dimension without destroying the original data structure and does not require manual annotation of samples. As shown in Fig 2, the Connection Centre Evolution (CCE) [22] algorithm is used to determine the clustering centers, and then, based on the data gravitation and the similarity matrix containing entropy, the most representative bands are selected within each cluster according to the data gravitation. Within each cluster, other bands are ranked according to weak correlation. Finally, a global ranking of the bands is obtained based on the S-shaped ranking strategy, and the top 20 bands are selected as the data for band selection.

DGWCR flowchart.

Spectral indices.

After brain tissue becomes infected with cancer, the physiological characteristics of the tissue, such as proteins, lipids, and water content, [10,23,24] undergo changes, which in turn alter the reflectance of the corresponding location in the hyperspectral image. Spectral indices are tools used to describe certain spectral features, and by combining the reflectance of several specific wavelengths, they help analyze and understand the manifestation of different material components. To enhance the differences between tumor tissue and normal tissue, we selected 38 spectral indices related to tissue color, tissue structure, tissue physiology, and tissue water content from relevant literature, as shown in Table 2. We then used the Relief algorithm to identify spectral indices that can significantly differentiate tumors from these 38 indices.

Table 2: The names and formulas of 38 spectral indices.

Relief algorithm is a feature weighting algorithm that assigns different weights to features based on their correlation with the class labels, and features with weights below a certain threshold are removed [25–27]. The Relief algorithm randomly selects a sample R from the training sample set each time, and then identifies k nearest neighbor samples (near Hits) from the same class as R, and k nearest neighbor samples (near Misses) from each different class of R. It then updates the weights of each feature according to Eq 2. Here, diff(A, $[eqn]$ , $[eqn]$ ) represents the difference in feature A between samples $[eqn]$ and $[eqn]$ (Eq 3). $[eqn]$ is the sample from the i-th sampling, $[eqn]$ is the j-th nearest neighbor of $[eqn]$ , m is the number of samplings, k is the number of nearest neighbors, $[eqn]$ is the probability of the occurrence of class C, $[eqn]$ is the class to which the random sample R belongs, $[eqn]$ is the probability of the occurrence of the random sample R, and Mj(C) indicates the j-th nearest neighbor sample that does not belong to Class(R) in class C.

$$[eqn]$$ $$[eqn]$$

We selected 70 hyperspectral images of tumors, of which 38 images contained tumors and 32 were images of normal tissue. We labeled whether they contained tumors with 0 and 1, which will serve as training labels. By averaging the reflectance of each band, we obtained the average reflectance spectrum of the hyperspectral image, and then calculated its 38 spectral indices from the spectral curve. By calculating the correlation between these 38 spectral indices and the presence of tumors, we obtained a weight diagram as shown in Fig 3. We selected the top 5 spectral indices most sensitive to tumors, RGI, PRI, NPCI, NRI, and mRESR, as the spectral index data.

Weight map of 38 spectral indices obtained from the relief algorithm.

Texture features.

Hyperspectral images contain not only rich spectral information but also important spatial information. Textural features can capture the spatial information in brain cancer, including the shape, size, and distribution of the cancer, which is crucial for distinguishing different tissues with similar spectral characteristics [28]. In hyperspectral imaging, due to the lower spatial resolution and a higher number of mixed pixels, detection relying solely on spectral information may not be accurate enough. Moreover, in some cases, different tissues may have similar spectral features, or the spectral characteristics of the same tissue may vary significantly under different conditions. Textural features provide an additional dimension of information that helps address these issues, leading to more accurate tumor identification. Furthermore, textural features are somewhat adaptable to changes in image acquisition conditions, such as lighting and shadows, making them useful for detection information under variable environmental conditions. Therefore, we use the Invariant Attribute Profiles (IAPS) algorithm to extract textural features, enhancing the model’s ability to recognize brain cancer tissue from a spatial dimension.

The IAPS algorithm processes spatial information by applying isotropic filters or convolution kernels to hyperspectral images to extract features that are invariant to translation and rotation. It then employs the Simple Linear Iterative Clustering (SLIC) superpixel segmentation [29,30] technique to divide the image into regions with similar characteristics, forming clusters. In the spectral frequency domain, features from the spatial domain are subjected to a Fourier transform, converting the image from the spatial domain to the frequency domain, and simultaneously transforming the Fourier-transformed image from the Cartesian coordinate system to the polar coordinate system. Under polar coordinates, by performing a polar Fourier transform on each pixel of the input image, we can obtain m amplitude features corresponding to m different Fourier orders with each pixel (Eq 4).

$$[eqn]$$

Eq 5) obtain absolute rotation-invariant features by removing the phase information.

$$[eqn]$$

To mitigate the loss of rich phase information, relative rotation-invariant features are developed by coupling the obtained additional convolutional Fourier representation with two adjacent convolution kernel radii (Eq 6).

$$[eqn]$$

We integrate the three components as shown in Eq 7.

$$[eqn]$$

Using isotropic triangular convolution kernels to aggregate the Polarimetric Weighted Fractal Features (PWFF) to obtain frequency-domain invariant features, and finally stacking the features extracted from the spatial and frequency domains to form the final Invariant Attribute Profiles (IAPs) feature representation. Through the acquisition of IAPs, a robust texture feature extraction can be provided for the model. In terms of textural features, the top 5 dimensions ranked by IAPS are selected as the textural feature data.

Feature fusion.

Feature fusion is a pivotal concept in the realms of machine learning and data analysis, with significant applications in image processing, computer vision, and medical image analysis. This process entails the amalgamation of features derived from various sources or stages of processing. The objective of feature fusion is to bolster the model’s capacity to generalize by harnessing the strengths of diverse features, thereby enhancing the model’s comprehension of the data and elevating the precision of classification, detection, or predictive tasks. We have implemented feature fusion in hyperspectral tumor detection, and to substantiate the notion that the integration of three distinct data modalities can augment the precision of medical hyperspectral detection, we conducted a comparative analysis of detection accuracy across seven feature configurations: three individual features (OWS,SIS,TFS), three combined pairs of these features (OWS+SIS, OWS+TFS, SIS+TFS), and the comprehensive fusion of all three features (ALL). The method of combination employed here involves a concatenation along the third dimension.

Classifier and evaluation

Classifier selection.

Currently popular medical hyperspectral detection models often come with built-in model selection and texture acquisition capabilities. To avoid the impact of complex algorithms on the fusion of the three types of data, we choose SVM (Support Vector Machine) and RF (Random Forest), two common and simple algorithms, to prevent the hyperspectral detection from gaining an advantage from the model itself. In hyperspectral image detection, SVM can effectively identify the spectral characteristics of different materials and achieve high-precision detection by selecting the appropriate kernel function and parameters. RF, as an ensemble learning method, has demonstrated its powerful capabilities in past medical hyperspectral image detections. It constructs multiple decision trees and aggregates their predictions, effectively handling high-dimensional data and capturing complex patterns within the data. In the experiment, cross-validation methods were used to find the optimal penalty parameter C and kernel function parameter gamma for SVM, and a random forest composed of 200 decision trees was used. This experiment was debugged in MATLAB 2020a and run on a personal computer with an Intel Core i7-12650H central processor and 16GB of RAM.

Evaluation metrics.

In this study, we employed three widely recognized standards to assess the performance of pixel-level detection in medical images: Overall Accuracy, Average Accuracy, and the Kappa coefficient [15]. These evaluation metrics provide us with an intuitive method for quantifying the results of our experiments and are frequently used in research within the relevant field. Overall Accuracy is a metric that quickly gives an overview of the model’s average performance across all categories. It measures the proportion of correctly detected samples out of the total samples, reflecting the detector’s overall performance on all categories. Average Accuracy takes into account the detection effect of each category, calculating the average of the accuracies of each category. AA is particularly important for dealing with imbalanced datasets because it can reveal the model’s performance differences across various categories, helping us identify the strengths and weaknesses of the model. The Kappa coefficient is a statistical measure used to assess the performance of a detector while considering the possibility of random detection. The value of the Kappa coefficient ranges from –1 to 1, with 1 indicating perfect agreement and 0 indicating that the detector’s performance is no different from random detection. The advantage of this coefficient is that it accounts for random agreement, thus providing an accurate assessment of the detector’s performance even in cases where the category distribution is uneven. By considering these three metrics comprehensively, we can evaluate the performance of medical image detection models in a holistic manner, ensuring that our research results are both scientifically sound and practically valuable.

Workflow of the study

In this study, we developed a machine learning algorithm for hyperspectral image (HSI) classification by fusing multiple types of features (Fig 4). The fusion integrates information from three complementary aspects: wavelength selection, disease indices, and texture features. Using this approach, the algorithm accurately delineates tumor locations and boundaries across different glioma HSIs. Moreover, the effectiveness and generalizability of the three-feature fusion in enhancing model accuracy were validated using two distinct machine learning models.

Main flowchart of the hyperspectral tumor image classification process.

Results

Performance of the model on SVM

Our algorithm was implemented with SVM classifier across four datasets, conducting experiments to validate the efficacy of the three-feature fusion approach and comparing it with other feature combinations. To provide a more intuitive understanding of the classification effects under different feature inputs, we visualized the classification results for seven types of inputs. For the sake of brevity in the manuscript, only the classification effect diagrams for different inputs of dataset 8-01 were displayed.

As shown in Table 3, for the dataset 8-01, the three-feature fusion input achieved the best performance with OA, AA, and KA of 91.86%, 91.68%, and 0.8941, respectively. Among the single features, OWS achieved the best overall accuracy with 89.48%, and the three-feature fusion improved this by 2.4%. In the two-feature fusions, the three-feature fusion led with an increase of 0.3% over TFS+SIS, 5.79% over OWS+SIS, and 19.53% over OWS+TFS. The three-feature fusion also showed varying levels of improvement in AA and KA. As illustrated in Fig 5, Fig 5i, which represented the detection map for SVM with three-feature fusion input, more closely matched the ground truth compared to Fig 5b–5h. Only a small number of vascular pixels were misclassified as other categories in Fig 5i. The judgment on the tumor was completely consistent with Fig 5b.

Prediction effect diagrams of SVM with 8-01 dataset based on different feature inputs.(a) False color image. (b) Ground truth image. (c) OWS. (d) SIS. (e) TFS. (f) OWS + SIS. (g) OWS + TFS. (h) SIS + TFS. (i) All features combined.

Table 3: Detection accuracies of OA, AA, and KA for SVM with different feature inputs on four datasets.

The SVM detector achieved the best performance in terms of OA, AA, and KA with scores of 96.49%, 94.53%, and 0.9327 respectively, when using three feature fusion inputs in the 12-01 dataset. In the 12-02 dataset, the three-feature fusion reached the maximum values of OA, AA, and KA with 95.57%, 95.62%, and 0.9415 respectively. In the 20-01 dataset, the three-feature fusion obtained the maximum values of OA, AA, and KA with 95.95%, 93.85%, and 0.9469 respectively. The consistent performance across various datasets demonstrated the reliability of the three-feature fusion in enhancing tumor detection accuracy. As shown in Fig 6, the detection effect diagram with three-feature input showed minimal error when compared to the ground truth. In terms of tumor pixel detection, aside from some errors within the yellow circle in the upper right corner of Fig 6c, the results were consistent with the corresponding actual ground truth images.

Color images of living human brain tissue from three patients, their corresponding grayscale images, and SVM detection effect diagrams with three-feature fusion.(a, b, c) Images from patient 12-01. (d, e, f) Images from patient 12-02. (g, h, i) Images from patient 20-01.

Performance of the model on RF

To verify the detection accuracy and effectiveness on RF, we also conducted experiments on four Multi-Hyppectral Imaging (MHSI) datasets, using OA, AA, and KA as metrics for accuracy assessment, as shown in Table 4. Similarly, we only visualized the detection effect diagrams of the seven types of inputs on one dataset for comparison.

Table 4: Detection accuracies of OA, AA, and KA for RF with different feature inputs on four datasets.

In the RF detector, the 8-01 dataset achieved the highest metrics in OA, AA, and KA with 99.21%, 99.28%, and 0.9897, respectively. For the 12-01 dataset, RF achieved a detection accuracy higher than other inputs with 90.95% OA, 0.8852 KA. For the 12-02 dataset, RF also outperformed other inputs with the same metrics of 89.56% OA, 91.34% AA, and 0.8616 KA. The detection effect diagrams for the three datasets were shown in Fig 7. On the 12-01 dataset, under the yellow rubber band in the upper right corner, only some tumor pixels were incorrectly detected as other tissues. On the 8-01 and 12-02 datasets, tumor pixels were perfectly detected by the model.

Prediction effect diagrams of RF with 20-01 dataset based on different feature inputs.(a) False color image, (b) Ground truth image, (c) OWS, (d) SIS, (e) TFS, (f) OWS + SIS, (g) OWS + TFS, (h) SIS + TFS, (i) All features combined.

The detection accuracy of the 20-01 dataset on the RF was shown in Table 4. The RF led other inputs by an average of about 8% in the OA metric with 93.35%, by about 7% in the AA with 90.52%, and by about 0.1 in the KA with 0.9131. We trained the corresponding models for the seven types of inputs. Fig 8i showed the predicted detection diagram of the RF with three-feature fusion; it was consistent with Fig 8b except for some errors in the upper right rubber band and the middle blood vessel. Compared to the detection performance (Fig 8c–8h) under the other six inputs, the detection effect diagram with three-feature fusion showed superior performance in identifying the location of the tumor for Fig 8a.

Color images of living human brain tissue from three patients, their corresponding grayscale images, and RF detection effect diagrams with three-feature fusion.(a, b, c) Images from the 8-01 dataset. (d, e, f) Images from the 12-01 dataset. (g, h, i) Images from the 12-02 dataset.

Summary of model classification performance

To validate the performance of a hyperspectral tumor image detection algorithm based on the fusion of multiple features, we conducted comparisons of seven different feature combinations on four medical hyperspectral datasets using two detectors. The evaluation was performed using three assessment metrics, and the statistical effects were shown in Fig 9.

SVM and RF detection accuracy on different datasets: (a) 8-01, (b) 12-01, (c) 12-02, (d) 20-01.

Regarding the seven data combinations, it could be observed that the three basic single data inputs, particularly the optimal wavelength parts, yield the best detection effects, which was entirely reasonable. Spectral indices and textural features primarily served an auxiliary role; spectral indices aimed to enhance the data’s sensitivity to tumor tissues, while textural features provided additional spatial information. From Fig 9, it could be observed that the overall trend for the four datasets showed that the fusion of features performed better than a single input, with the three-feature fusion showing the most outstanding performance.

In terms of detectors, the performance of SVM and RF was not consistent across different datasets; sometimes RF outperforms, and other times SVM was better. We believed this was due to minor differences in the datasets. The purpose of using two detectors was to avoid the improvement in accuracy being specific to a particular detector; in practical applications, only one detector could be used.

Discussion

In this study, we are committed to developing a hyperspectral tumor image detection framework suitable for clinical use, to meet the need for precise identification of brain tumor locations. This framework significantly improves the accuracy of tumor identification by integrating three key data types: wavelength selection, spectral indices, and textural features, thereby achieving precise localization of tumors.

Hyperspectral imaging technology, with its outstanding capabilities, is increasingly valued in the medical field [9,11]. It not only extracts the spectral characteristics of each pixel but also provides detailed information about the different tissue components and their spatial distribution. At specific wavelengths, tissues with different pathological states exhibit different reflectance, absorption, and electromagnetic energy characteristics due to differences in chemical composition and physical properties, which are manifested through characteristic spectra. By carefully analyzing these spectral signals, we can achieve qualitative or quantitative detection of tissue conditions. Combined with the spatial distribution information provided by hyperspectral images, we can visualize various pathological states of tissues and diagnose the disease conditions of tissues. In recent years, the application of hyperspectral technology in the field of medical diagnosis has gradually increased [31]. For example, Song Nan and others successfully identified the spectral differences between gliomas and normal brain tissues by deeply analyzing and processing hyperspectral data, providing a new method for distinguishing between gliomas and normal brain tissues. Yi Weisong and others used a near-infrared hyperspectral imaging system to obtain hyperspectral images of gastric cancer tissues and normal tissues [32], extracted the characteristic spectra of tumors and normal tissues by chemometric methods, and performed spectral attribution. They further combined multiple target recognition algorithms to automatically extract gastric cancer tumor targets, and ultimately used image fusion technology to visually and intuitively display the recognition results, forming a method system for diagnosing gastric cancer tissues with hyperspectral images. These studies show that hyperspectral imaging technology is not only highly credible in tumor identification but also can provide more accurate and in-depth diagnostic information [33].

In the field of hyperspectral image detection, the application of deep learning has been very extensive [34–36], and many researchers have proposed many innovative and powerful frameworks, which have been effectively verified on remote sensing datasets [37–39]. For example, Li Ming-song fused the central pixel and neighboring pixels through the network to achieve detection [40], while Zheng Zhuo designed a lateral connection module to connect the encoder and decoder, thereby fusing the spatial details in the encoder and the semantic features in the decoder, [41] etc. In the field of hyperspectral medical image detection, although many innovations focus on the optimization of deep learning network architecture, such as introducing residual connections, attention mechanisms, encoding and decoding, and other advanced technologies to improve detection accuracy, our research focus has shifted to innovation at the data level. We are committed to exploring how to enhance the detection capabilities of the model through diversified processing and feature fusion of raw hyperspectral data [42,43]. This innovative method, which starts from the data, aims to provide new solutions for the precise detection of hyperspectral medical images by deepening data understanding and feature extraction. We try to process raw hyperspectral data in various ways and improve detection accuracy through feature fusion. Feature fusion is a machine learning technique that merges features from different sources to form a better feature representation, thereby enhancing model performance [44]. In the spectral analysis of medical images, optimal wavelength features correspond to the spectral absorption or reflection characteristics of tumor tissues [45], which can provide highly specific identification of the lesion area. Spectral indices can help enhancing the model’s sensitivity to tumor tissues [46], especially in distinguishing between tumors and normal tissues. Textural features reflect the spatial heterogeneity of tissue structure, providing important morphological information for the image [47], which helps to identify the boundaries and morphological characteristics of tumors. By combining the advantages of the three features, enhancing the model’s understanding of the data, and thus improving the model’s accuracy.

To select the appropriate bands for feature fusion, we used the DGWCR algorithm [21] to screen 20 representative bands from a large number of bands, forming the data basis for optimal wavelength. To enhance the model’s sensitivity to tumor tissue, we used the Relief algorithm [48] to select 5 spectral indices most related to tumor tissue from 38 spectral indices, as the data part of the spectral index. At the same time, to provide rich spatial texture information, we applied the IAPS algorithm [49] to calculate 5-dimensional regional texture features. By using band selection as the basis, texture features provide spatial auxiliary information, and spectral index data enhance the sensitivity to tumor tissue, we try to improve the performance of the detection model by splicing the three types of data. This method not only innovates the way of data processing but also is expected to achieve higher precision diagnosis in the field of hyperspectral medical image detection, bringing new perspectives and possibilities for clinical applications.

To verify the effect of the three-feature fusion on improving detection accuracy, we used two widely used detectors in brain cancer detection and analyzed them separately using three independent features and their fused features. In the early stage of the experiment, we conducted a detailed preprocessing of hyperspectral data to eliminate the impact of environmental and equipment factors. Based on the selected optimal wavelengths, spectral indices, and textural features, we constructed SVM and RF detection models [50,51] and detected tumor hyperspectral images using seven different input combinations and two detectors. We used standard evaluation indicators, including overall accuracy, average accuracy, and Kappa coefficient, to comprehensively evaluate the detection results. In addition, to intuitively display the detection effect, we visualized each detection result. The experimental results show that on four different datasets and two detectors, the detection method that fuses three features is superior to the other six single or two-way fusion input methods in all detection indicators, achieving excellent detection results. In the experimental design, we deliberately chose a variety of datasets to avoid the accidental results that a single dataset might bring; at the same time, we used two different detectors to exclude the impact of the detector’s own characteristics on the improvement of accuracy. These experimental designs ensure the reliability and universality of our results. Ultimately, the experimental results prove that under the combined action of optimal wavelengths, spectral indices, and textural features, the detection accuracy and performance of hyperspectral data have been significantly improved, and this comprehensive method provides a powerful auxiliary tool for doctors to identify tumor locations.

Although our research has achieved certain results, there are still some limitations. First, the datasets we used are public and only include patients with IV glioblastoma, which may lead to sample selection bias and limit the universal applicability of the research results. To overcome this limitation, we plan to include more data from patients with different types of tumors, different ages, and genders in future research to ensure the accuracy and wide applicability of the research. Secondly, on the algorithm level, although the DGWCR, Relief, and IAPS algorithms played an important role in this study, there is still room for further optimization. We consider replacing these algorithms with algorithms that can more efficiently extract effective data and speed up processing to improve the response speed and practicality of the entire framework. Overall, we are committed to building a powerful, universal, and easily promoted framework based on the fusion of optimal wavelengths, spectral indices, and textural features to achieve precise localization of tumor positions and assist doctors in tumor resection surgery. This study, as a high-quality study in the field of medical hyperspectral imaging and artificial intelligence, shows great clinical application potential. We believe that through continuous technological innovation and optimization, our research will bring new breakthroughs to the fields of medical image analysis and tumor diagnosis, providing patients with more accurate and personalized treatment plans.