Tumour Ellipsification in Ultrasound Images for Treatment Prediction in Breast Cancer

TL;DR

This paper presents a semi-automated method for localizing breast tumours in ultrasound images to facilitate non-invasive cancer treatment prediction, using feature extraction, supervised learning, and ellipse fitting.

Contribution

It introduces a novel semi-automated approach combining keypoint features and SVM classification for tumour ROI estimation in ultrasound images.

Findings

Achieved 76.7% accuracy in tumour localization

Demonstrated potential for semi-automated contouring in cancer response prediction

Proposed method reduces manual effort in tumour segmentation

Abstract

Recent advances in using quantitative ultrasound (QUS) methods have provided a promising framework to non-invasively and inexpensively monitor or predict the effectiveness of therapeutic cancer responses. One of the earliest steps in using QUS methods is contouring a region of interest (ROI) inside the tumour in ultrasound B-mode images. While manual segmentation is a very time-consuming and tedious task for human experts, auto-contouring is also an extremely difficult task for computers due to the poor quality of ultrasound B-mode images. However, for the purpose of cancer response prediction, a rough boundary of the tumour as an ROI is only needed. In this research, a semi-automated tumour localization approach is proposed for ROI estimation in ultrasound B-mode images acquired from patients with locally advanced breast cancer (LABC). The proposed approach comprised several modules, including 1) feature extraction using keypoint descriptors, 2) augmenting the feature descriptors with the distance of the keypoints to the user-input pixel as the centre of the tumour, 3) supervised learning using a support vector machine (SVM) to classify keypoints as "tumour" or "non-tumour", and 4) computation of an ellipse as an outline of the ROI representing the tumour. Experiments with 33 B-mode images from 10 LABC patients yielded promising results with an accuracy of 76.7% based on the Dice coefficient performance measure. The results demonstrated that the proposed method can potentially be used as the first stage in a computer-assisted cancer response prediction system for semi-automated contouring of breast tumours.

Tables

Table 1. TABLE I: The Results of ellipsification using the dice performance measure on 33 B-mode images based on “leave-one-out” validation scheme.

Features	Classifier	$D\pm \sigma$
BRISK	SVM	0.7563 $\pm$ 0.1387
FAST	SVM	0.7366 $\pm$ 0.1746
SURF	SVM	0.7665 $\pm$ 0.1515
BRISK	k-Means	0.7427 $\pm$ 0.1575
FAST	k-Means	0.7376 $\pm$ 0.1602
SURF	k-Means	0.7507 $\pm$ 0.1671
BRISK	FCM	0.7503 $\pm$ 0.1507
FAST	FCM	0.7379 $\pm$ 0.1601
SURF	FCM	0.7510 $\pm$ 0.1670

Equations

$$D=\frac{2|E\cup G|}{|E|+|G|}$$

Full text

I Idea and Motivation

Precision medicine is an emerging technique that tailors medical treatment depending on the characteristics of each patient [1]. For example, in cancer therapy, the main goal of precision medicine is to decide on the most effective cancer therapy for a patient based on his/her response to treatment administrations. To this end, the first step in personalized cancer therapy is to either predict or provide a means to assess the individual responses to treatment early during the course of a therapy. There are several functional imaging modalities such as magnetic resonance imaging (MRI), diffuse optical spectroscopy (DOS), and positron emission tomography (PET) that can provide imaging at a microscopic level to detect cell death [2]. Two main limitations of these imaging technologies include: the requirements for a large capital investment and an external agent. The latter is also expensive, and may cause some side effects and/or allergic reactions [3]. In contrast, quantitative ultrasound (QUS) methods provide a portable, non-expensive, and non-invasive means for a rapid acquisition of functional images that can be used for an early assessment of cancer therapeutic effects [3]. Moreover, in QUS methods, the endogenous contrast – generated by the very process of cell death – is employed in treatment assessment, which alleviates the requirement for injecting external agents.

The applications of QUS methods have recently been extended from cancer response monitoring [4] to cancer response prediction [5], and tissue characterization/visualization using 3-D automated breast ultrasound (ABUS) scanners [6]. The first major step in the implementation of each of these applications is to contour a region of interest (ROI) inside the tumour in frames with identifiable tumour areas. This step is currently performed manually as there is no automated software to segment an ROI in ultrasound B-mode images. The manual segmentation of tumours, however, is a very time-consuming task as hundreds of frames should be contoured in a study due to the availability of multiple frames for each subject (an animal in preclinical and a patient in clinical studies). With the availability of 3-D scanners such as ABUS technologies, the problem will be even more severe as tens of frames should be contoured in each patient. Therefore, designing an automated segmentation method can save a significant amount of experts’ time and efforts.

In this study, a semi-automated supervised tumour localization method was proposed for ROI estimation in B-mode images acquired from patients with locally advanced breast cancer (LABC).

II Background

There is an extensive literature on image segmentation. Several segmentation techniques have been tried for ultrasound images [7]. In a more recent literature, classification techniques applied on some expressive features have been more frequently employed to extract segments from images. These classification techniques include simple methods like k-means and fuzzy c-means (FCM) and more sophisticated algorithms such as support vector machines (SVM). These classifiers group pixels into segments to enable subsequent measurements and recognition. Most of these methods, when applied to digital images, rely on the availability of invariant features. Among feature extraction methods in the literature, those which are based on invariant features extracted from some keypoints demonstrated success in segmentation tasks. Keypoints represent pixel positions in an image surrounded (within certain vicinity) with significant information, e.g., textures, corners, edges, etc [8]. For instance, FAST (features from accelerated segment test) [9], BRISK (binary robust invariant scalable keypoints) [10], and SURF (speeded up robust features) [11] are among the most commonly used keypoint-based feature descriptors providing inputs to a classifier.

SURF is a feature detection method, which is partly inspired by the scale-invariant feature transform (SIFT) descriptor, but is generally faster than SIFT [11]. It uses an integer approximation of the determinant of the Hessian blob detector to detect keypoints. Its feature descriptor is based on the sum of the Haar wavelet response around the point of interest. The three main modules of the algorithm include: interest point detection, local neighborhood descriptor, and matching.

In BRISK detector, the scale of each keypoint is estimated in the continuous scale-space [10]. The BRISK descriptors are composed of binary vectors by concatenating the results of brightness comparisons. The key concept of the BRISK descriptor is based on using a specific pattern to sample the neighborhood of all keypoints.

FAST is a corner detection method [9]. The advantage of using FAST corner detector is its high computational efficiency and a better performance when subsequent machine-learning methods such as a supervised classification are applied [12]. It uses a 16-pixel circle to classify a candidate point as a corner or no corner, where for each pixel in the circle, an integer number from 1 to 16 will be labeled clockwise.

III Proposed Approach

The first step in tumour response prediction is contouring (segmenting) the tumours. Subsequently, texture analysis will be performed on the QUS parametric maps computed within the contoured tumours (tumour cores), and their margins (including surrounding healthy tissue) to predict patient’s response [5]. In the aforementioned analysis steps, however, a rough localization of the tumour is of more interest than its exact segmentation [5]. Considering this, and also the fact that ultrasound images are known to be very difficult for segmentation, a semi-automated approach was proposed in this study to obtain an estimation of the tumour location as described in the next paragraphs.

Algorithm 1 describes the proposed approach.

**Patient Data – ** The study involved 10 LABC patients with tumour sizes between 5 and 15 cm. The data acquisition was performed in accordance with the clinical research ethics approved by Sunnybrook Health Sciences Centre. A biopsy was used as the gold standard test to confirm all cancer cases. In order to measure the size of tumours, all patients were imaged using MRI. Also, all patients were imaged using ultrasound before the start of neoadjuvant chemotherapy (“pre-treatment”). The acquisition of US data was performed using a Sonix RP ultrasound system. The system was equipped with an L14-5/60 linear transducer (centre frequency at $\sim$7 MHz). Depending on tumour size and location, the transducer was focused at the midline of the tumour with a maximum depth of 4-6 cm. The number of scans obtained from each tumour depended on it size. This resulted in three to five scan planes from each tumour, with a scan plane separation of about 1 cm.

Pre-Processing – In order to increase the quality and contrast of B-mode images, before computation of feature descriptors, each image was pre-processed by applying the fuzzy histogram hyperbolization [13]. Subsequently, a 3 $\times$ 3 median filter was applied to each image.

Feature Extraction – Features were extracted from the keypoints identified on the pre-processed images for submission to a classifier. Three major feature extraction algorithms explained in Section II, including SURF, FAST, and BRISK were adapted in this stage.

Extended Features Descriptors – After applying feature extraction methods, a feature descriptor of size $m$ for each keypoint was used to characterize an image. However, the preliminary results using these features revealed that there was no strong correlation between the tumour and the feature points to obtain an acceptable result (the initial performance using only extracted features resulted in relatively low performance, i.e., less than 60% accuracy for segmentation of tumours). Although the manual contouring of multiple scans is a very time-consuming and tedious task, requiring the clinician to provide the rough location of the tumour centre (e.g., via a mouse click) does not pose an unacceptable burden on him/her. Assuming that the tumour centre $(x_{c},y_{c})$ is available, the distance $d$ of each keypoint from $(x_{c},y_{c})$ can be added to the descriptor to augment the feature space (Algorithm 1, lines 9-12). This, as our experiments demonstrated later, considerably increased the accuracy of contouring tumours.

Classification – A support vector machine (SVM) classifier with a radial basis function (RBF) kernel was used in a supervised paradigm to classify the keypoints in each image as “tumour” or “non-tumour” points based on the extended feature descriptors $\mathbf{F}$ extracted from the keypoints (Algorithm 1, line  14). In addition, k-means and FCM were also used for the classification of the keypoints in the same way for the purpose of comparison with the SVM performance.

Ellipsification – After classification, all keypoints would be classified as “tumour” or “non-tumour” pixels. Suppose that there are $n$ points which are defined as tumour points in one image. For the purpose of ellipsification of a tumour area, at first, the positions of all the $n$ points were found in the vertical direction and the leftmost position $x_{l}$ and the rightmost position $x_{r}$ were identified. In the horizontal direction, the highest position $y_{h}$ and the lowest position $y_{l}$ were determined. These four points are the four vertices of an ellipse, which was used as the boundary of the ROI representing the tumour.

Accuracy Measurement – Dice coefficient was measured for calculating the accuracy of classification results. The Dice coefficient is given by

[TABLE]

where $E$ is the fitted ellipse and $G$ is the ground-truth. It can be interpreted as a measure of overlapping between the estimated and correct classes [14].

Validation – Leave-one-out validation was performed to evaluate the performance of the designed ellipsification system on the 33 available B-mode images. The algorithm was developed using Matlab (R2011a, Mathworks, USA) on a 64-bit Intel Core i5-4200U CPU @ 2.30 GHz processor equipped with 8 GB of memory and Windows 8.1.

IV Results

Fig. 1 depicts the result of pre-processing on one typical breast ultrasound image using fuzzy histogram heprbolization. A comparison between the pre-processed (right) and the original (left) images shows the improvement in the quality of the B-mode image after pre-processing.

Fig. 2 shows representative good, average, and bad ellipsification results. As can be observed from this figure, the accuracy can significantly vary. Images with an irregular or small-size shape may not be classified correctly (accuracy as low as 46.9% in Fig. 2-bottom row). In contrast, for images with a normal size and shape, the accuracy can be quite high (e.g., 92.7% in Fig. 2-top row).

Table I provides the results of ellipsification using the Dice measure on all 33 B-mode images for various feature descriptors and classifiers used in the experiments. The highest average accuracy is 76.7% with a standard deviation of 15.2% using the SURF features and SVM classifier. In terms of computational cost, the SVM took longer time than other methods (approximately 10 minutes for each image, or 5 hours in total), while the other two classification methods were relatively fast. The results of classification can be improved by increasing the number of images as using only 33 scans in the experiments restricted the learning capability of the proposed ellipsification system.

Fig. 3 shows the distribution of the accuracy of ellipsification in respect to the number of generated keypoints for all the images used in the experiments (the results are only shown for the SVM classifier). BRISK, with an average accuracy of 75.63%, produced $979\pm 332$ keypoints. FAST, with an average accuracy of 73.66%, resulted in $791\pm 420$ keypoints. And SURF, with an average accuracy of 76.65%, generated $565\pm 179$ keypoints. Therefore, the SURF achieved a higher performance using a smaller number of keypoints, which reduced the computational cost of subsequent classification step. Moreover, the SURF was more consistent in selecting the number of keypoints from B-mode images, as can be judged from the smaller standard deviation (179) achieved compared to the other two methods.

V Discussion and Conclusion

In this study, a semi-automated localization method was proposed to mark a tumour with an ellipse based on a user single-click input, keypoint descriptors, and the SVM classification. The results demonstrated that the SURF features extended with the proposed weighted distances from the tumour’s centre pixel (identified by the expert), along with the SVM classifier, achieved promising results. The original breast cancer ultrasound images exhibited very poor quality. Employing the central tumour point provided a higher correlation between the tumour area and the extracted features in noisy ultrasound images. Nonetheless, the segmentation of these images remains a challenging task as also reported by other researchers. For example, a recently published paper reported only between 50% and 60% accuracy in the segmentation of B-mode US images of breast [15].

In future work, a larger image dataset will be used for the training process. This will increase the chance of including examples of the images of different sizes and shapes, which will help to fully automate the approach.