Radiological images and machine learning: trends, perspectives, and prospects

TL;DR

This review explores how machine learning techniques are transforming radiological imaging by improving diagnosis, segmentation, and decision-making, highlighting current challenges and future prospects in the field.

Contribution

It provides a comprehensive overview of machine learning applications in radiology, emphasizing recent advances, challenges, and future directions for clinical integration.

Findings

Machine learning achieves performance comparable to human experts in some radiological tasks.

Applications include image segmentation, disease diagnosis, and computer-aided detection.

The review discusses challenges like data quality and model interpretability.

Abstract

The application of machine learning to radiological images is an increasingly active research area that is expected to grow in the next five to ten years. Recent advances in machine learning have the potential to recognize and classify complex patterns from different radiological imaging modalities such as x-rays, computed tomography, magnetic resonance imaging and positron emission tomography imaging. In many applications, machine learning based systems have shown comparable performance to human decision-making. The applications of machine learning are the key ingredients of future clinical decision making and monitoring systems. This review covers the fundamental concepts behind various machine learning techniques and their applications in several radiological imaging areas, such as medical image segmentation, brain function studies and neurological disease diagnosis, as well as computer-aided systems, image registration, and content-based image retrieval systems. Synchronistically, we will briefly discuss current challenges and future directions regarding the application of machine learning in radiological imaging. By giving insight on how take advantage of machine learning powered applications, we expect that clinicians can prevent and diagnose diseases more accurately and efficiently.

Tables

Table 1. Table 1 : A summary of image features used in ML systems

Features		Examples
Color	Invariant from different size and direction	Histogram [42, 43, 44]
Shape	Binary representation of images	Sphericity [44, 45]
Texture	Description of image structure, randomness, linearity,	Haralick’s features [46, 45]
	roughness, granulation, and homogeneity	Gabor features [47, 48, 49]
		Co-occurrence [50]
		Curvelet-based [51, 52]
		Wavelet-based [53, 54]
Local	Description of local image information using region,	Local binary pattern [44]
	object of interest, corners, or edges	Scale invariant feature transform [55, 56, 57]
		Speed up robust features [58, 57]
Other	Other methods to extract image features	CNN [59]

Table 2. Table 2 : Overview of segmentation methods for different radiological images

	image types	# images	goal	methods	Dice coefficients
[114]	MRI	12	Brain tissue	Sparse dictionary learning	0.91 (Gray matter) 0.87 (White matter)
[90]	MRI	36	Stroke lesion	Random forest	0.82
[101]	CT	42	Liver tumor	Random forests & supervoxels	0.93
[115]	CT	30	Liver tumor	CNN	0.84
[102]	MRI	70	Knee	Multi-atlas context forests	0.97 (Bone) 0.81 (Cartilage)
[105]	CT	150	Multi-organ	Discriminative dictionary learning	0.90 (Liver) 0.88 (Kidney) 0.55 (Pancreas) 0.92 (Spleen)
[94]	MRI	10	Brain tissue	CNN	0.95 (Gray matter) 0.86 (White matter)
[116]	CT	82	Pancreas	CNN	0.72
[107]	MRI	30	Stroke lesion	Gaussian Naïve Bayes classification	0.81
[91]	MRI	12	Brain lesion	ANN	0.79
[117]	MRI	66	Prostate	Sparse auto-encoder & sparse patch matching	0.88
[118]	MRI	45	Left ventricle	CNN & stacked-auto-encoder	0.97
[95]	MRI	53	Brain tumor	CNN	0.95
[119]	CT	73	Lung texture	Convolutional restricted Boltzmann machines	0.74
[97]	MRI	57	Brain segmentation	CNN	0.86
[120]	4D-CT	22	Brain tissue	SVM	0.79 (Gray matter) 0.81 (White matter)
[121]	MRI	65	Brain lesion	CNN	0.79
[122]	CT	42	Liver tumor	CNN	0.97
[123]	MRI	73	Brain tumor	CNN	0.65

Table 3. Table 3 : A summary of recent CAD studies. AUC = area under curve; ROC = receiver operating cruve; TP = true positive rate; MAE = mean average error

	year	image type	# cases	disease	Measurements	results	keywords
[131]	2014	mammography	956	Breast cancer	AUC	0.81	Combination of classifiers
[132]	2014	mammography	500	Breast cancer	AUC	0.91	Naïve Bayes classification
[63]	2014	MRI	81	Cervical cancer	Accuracy	0.69	Texture features, SVM
[133]	2015	mammography	340	Breast cancer	AUC	0.73	Texture features, SVM
[134]	2015	mammography	772	Breast cancer	AUC	0.89	Feature selection method
[135]	2015	CT	750	Lung	AUC	0.98	Structured SVM
[136]	2015	X-ray	5,440	Lung	Accuracy	0.92	SVM
[137]	2015	MRI	83	Pediatric cardiomyopathy	Accuracy	0.81	Bayesian rule learning
[138]	2016	mammography	736	Breast cancer	AUC	0.82	CNN
[139]	2016	mammography	2,604	Breast cancer	AUC	0.93	Wavelet neural network
[140]	2016	ultrasound	520	Breast lesions	Accuracy	0.82	Stacked denoising auto-encoder
[141]	2016	ultrasound	95	Liver lesions	Accuracy	0.87	SVM
[142]	2016	CT	104	Vertebral body fractures	TP	0.81	SVM
[143]	2016	CT	409	Wrist, radius, ulna fractures	ROC	0.89	Random forest
[144]	2017	mammography	45,000	Breast cancer	AUC	0.91	CNN
[145]	2017	CT	1012	Lung cancer	Sensitivity	0.89	ANN
[146]	2017	CT	52	Teeth	Accuracy	0.89	CNN
[147]	2017	CT	344	Prostate cancer	ROC	0.80	CNN
[148]	2017	X-ray	1391	Bone age	MAE	0.80	CNN
[149]	2017	X-ray	108,948	Thorax diseases	Accuracy	0.63	CNN
[150]	2017	X-ray	112,120	Thorax diseases	AUC	0.84	CNN
[151]	2017	MRI	107	Brain tumor	Accuracy	0.88	Logistic Regression

Table 4. Table 4 : Recent studies on Alzheimer’s diseases NC: normal; AD: Alzheimer’s disease; pMCI: progressive mild cognitive impairment; sMCI: stable mild cognitive impairment

	year	databses	image #	image types	classification gruops	accuracy	keywords
[111]	2014	ADNI	834	MRI	AD vs. NC	89%	Multiple instance learning
					pMCI vs. sMCI	70%
[188]	2014	ADNI	202	MRI+PET	AD vs. MCI vs. NC	73.35%	Sparse discrimination feature selection
					AD vs. pMCI vs. sMCI vs. NC	61.06%
[190]	2014	ADNI	1071	MRI	AD vs. NC	89%	Manifold and transfer learning
					pMCI vs. sMCI	73%
[184]	2014	ADNI	814	MRI	pMCI vs. sMCI	75.6%	Gray matter grading, weak-classifier fusion
[180]	2015	ADNI	459	MRI	AD vs. NC	93.83%	Hierarchial fusion of features
					pMCI vs. sMCI	80.9%
					pMCI vs. NC	89.09%
[191]	2015	ADNI	202	PET+ MRI	pMCI vs. sMCI	78.7%	Multimodel multi-label transfer learning
[189]	2015	ADNI	830	MRI	AD vs. NC	91.31%	HoG mapping
					MCI vs. NC	78.07%
					pMCI vs. sMCI	75.54%
[47]	2016	OASIS	416	MRI	AD vs. NC	80.76%	Gabor filter
[192]	2016	ADNI	416	MRI	AD vs. NC vs. MCI	89.1 %	CNN
[193]	2016	Self-collected	67	MRI	AD vs. NC	96.77%	SVM
[184]	2014	ADNI	814	MRI	pMCI vs. sMCI	75.6%	Gray matter grading, weak-classifier fusion
[194]	2016	Self-collected	89	fMRI	AD vs. NC	97.50%	SVM
					MCI vs. AD	87.30%
					MCI vs. NC	72.00%
[195]	2016	Dartmouth College	116	MRI	AD vs. NC	97.14%	Feature ranking selection
[196]	2016	Self-collected	43	fMRI	AD vs. NC	96.85%	Deep learning selection
[197]	2017	Self-collected	250	DTI	AD vs. NC	89.60%	Elastic net selection

Table 5. Table 5 : A summary of recent image retrieval research using machine learning techniques

	year	image types	# images	results	keywords
[209]	2014	CT	72	AUC:0.93	Riesz wavelets, hierarchical semantic-based distance
[208]	2015	MRI	30	Accuracy:0.88	Partial least square discriminant analysis, principal component analysis
[203]	2015	CT	EXACT09:40 TCIA: 604	Precision: 0.88	Local wavelet pattern
[210]	2015	Multimodality	ImageCLEF: 10 thousand	MAP:0.29	Deep Boltzmann machine
[211]	2015	MRI	OASIS:421	Precision: 0.48	Local binary patterns, gray-level-co-occurrence matrices
[206]	2016	X-ray & CT	ImageCLEF:5400	Accuracy:0.98	Sparse representation, online dictionary learning
[38]	2016	Multimodality	Indoor:15620, Caltech256:30670, Corel5000:5000, ImageCLEF:2785	Top precision: 0.36	Support top irrelevant machine
[212]	2017	CT	EXACT09:675 TCIA: 604	Precision: 0.96	Gabor and Schmid filters

Equations

$$P(y=1|x)=\frac{1}{1+exp(-{\omega }^{T}x)}$$

$$y(x)=\text{sign}[\sum _{i=1}^{N_s}{\lambda }_i{y}_{i}K(x_i,x)+{\omega }_0]$$

$$DSC(\text{A,B})=\frac{2(\text{A}\cap \text{B})}{\text{A}+\text{B}}$$

$$\text{sensitivity}=\frac{\text{True Positive}}{\text{True Positive + False Negative}}$$

$$\text{specificity}=\frac{\text{True Negative}}{\text{True Negative + False Positive}}$$

Full text

Keywords: deep learning, machine learning, imaging modalities, deep neural network

1 Introduction

Radiology is a branch of medicine that uses imaging techniques to detect, diagnose and treat diseases [1, 2, 3]. Diagnostic radiology helps radiologists image internal body structures to diagnose the cause of symptoms, screen for illnesses and detect the body’s response to treatments. The most common radiology modalities include: plain X-ray, computed tomography (CT), magnetic resonance imaging (MRI), positron emission tomography (PET), and ultrasound imaging. Fig. 1 shows these internal body structures viewed via these different imaging techniques, and Fig. 2 illustrates an example of CT and PET images. In MRI images, the white areas represent subcutaneous fat, while in the CT images, the white areas represent the skull. However, the main disadvantage for all x-ray and gamma ray imaging modalities is the risk of radiation exposure for patients [4, 5, 6, 7, 8]. Ultrasound imaging is convenient because it does not expose patients or radiologists to radiation, but it has poor penetration through bone or air, which makes images difficult to interpret [9, 10]. MRI and CT images can capture anatomical changes in tissues, while PET images detects biochemical and physiological changes, which often occur before anatomical changes [11]. Disadvantageously, patients with ferromagnetic orthopedic implants, materials, and devices cannot undergo MRI procedures. MRIs also have relatively long scanning times which imposes limitations for patients in need of urgent care [12, 13]. The broader use of radiological image analysis increases the workload for radiologists, and therefore the development of intelligent computer-aided systems for automated image analysis that can achieve faster and more accurate results for large volumes of imaging data is essential.

This paper provides an overview of machine learning techniques used in radiological image analysis. We begin with a brief overview of current imaging technologies. In section 2, we review general concepts of machine learning and detail methods most commonly used in recent years. In section 3, we provide an overview of the most current studies dealing machine learning and radiological images. This review paper mainly focuses on the most recent contributions to different machine learning techniques (i.e., after 2014), and the reader should refer to previous review papers for older contributions related to machine learning and biomedical imaging [14, 15, 16, 17, 18], or contributions that focus solely on a single machine learning approach (e.g., deep learning [19, 20]). Lastly, we have summarized these contributions by outlining current technological limitations and potential future areas of research in this field.

Contributions cited in this review were collected using various research databases such as GoogleScholar, PubMed (MEDLINE), IEEE Xplore and SpringerLink. All contributions collected were published between the middle of 2014 and the middle of 2017. We used variations of the keywords including but not limited to combinations of machine learning techniques (SVM, random forest, regression, neural networks, deep learning), applications (segmentation, computer assisted system, brain studies) and imaging modalities (MRI, x-rays, ultrasound, CT). While deep learning techniques have been prevalent in the past five years, our search not only included these hot topics but also included traditional methods. In this review, the preference was given to papers that presented real data rather than theoretical frameworks. Similarly, we did not include papers that repeated past experiments unless the data collection or data analysis procedures were different.

2 Machine Learning in Radiology

In recent years, machine learning algorithms have become useful tools for the analysis of medical images in many radiology applications [15, 23]. For example, machine learning algorithms can extract the useful information found within the details of medical images [24]. Thus, computer-aided systems based on machine learning help radiologists to make informed decisions while interpreting these images [15].

2.1 Types of Learning

Depending on the utilization of labels in training data, there are three categories of machine learning algorithms: supervised learning, unsupervised learning, and semi-supervised learning. Supervised-learning is the most common form in machine learning, and researchers widely use supervised-learning for classification and regression [26]. Data is usually collected and labeled in categories, as the purpose of supervised learning is to find an appropriate input-output function from training data, which generalizes well against the testing data. We can compute an objective function to measure the error between the desired pattern and the output score. In general, many scientific contributions focus on finding a suitable objective function with adjustable parameters. Contrariwise, in cases where labeled data sets are relatively rare or difficult to acquire, unsupervised learning can derive deductions from data without corresponding label information; the purpose of unsupervised learning is to discover the hidden structure or distribution of data [27]. Unsupervised learning approaches include clustering and blind signal separation techniques such as principal component analysis and independent component analysis. Lastly, semi-supervised learning lie between supervised learning and unsupervised-learning [28, 29]. During the training phase, semi-supervised learning begins with a small set of labeled data and augments the training data size by gradually labeling unlabeled data.

2.2 Feature Selection

Feature extraction and representation is a crucial step in medical image processing. With the development of modern medical techniques, higher resolution and more features have become obtainable to feed the classifiers; however, this is an obstacle for machine learning techniques in achieving an optimal solution using high dimensional features. Significant interest exists in extracting and identifying reliable features from radiological images to improve classification performance[30, 31]. Several methods exist for extraction of features from medical images including region-based, shape-based , texture-based, and bag-of-words features [32, 33, 34, 35, 36, 37, 38]. The performance of most image retrieval systems is dependent on the use of these features. Table 1 summaries image features used in radiological image analysis. Color features are one of the essential features of images, including RGB, histograms [39], color moments [40] and color coherence vectors. Groups of pixels can calculate texture features, which can help characterize a wide range of images. The Gabor filter is the most common method for texture extraction [35]. Scale invariant feature transform and speed up robust features algorithm are two popular methods for scale and rotation invariant feature detector and descriptor in computer vision[41]. Different types of images have significant contrast variation. Thus visual features such as color, shape and texture are not enough to easily classify images. Thus high-level features are useful to overcome the intensity variations in different types of images and extract the appropriate information from said images. The process to select ideal features that can reflect the most useful contents of images remains a challenging problem in machine learning.

2.3 Overview of Machine Learning Methods

Machine learning has been developing rapidly in recent years, and it is impossible to cover all recently-developed techniques in one section. In this section, we will review the most commonly used machine learning methods in radiology, such as linear models, the support vector machine, decision tree learning, the ensemble classifier, as well as neural networks and deep learning. This section provides a general description of machine learning techniques and will help understanding their applications in the field of radiology, as described in subsequent sections.

2.3.1 Linear Models for Regression and Classification

Regression predicts the value from the given input features, whereas classification assigns input $x$ to one of the predefined classes.[60]. The simplest linear models establish a linear relationship among input variables. Commonly used linear models include linear regression, Fisher’s linear discriminant (LDA), and logistic regression. Given $x_{i},i=1,2,3...,N$, the input feature vector, the output $y(x,\omega)=\omega_{0}+\sum_{i=1}^{N}\omega_{N}x_{N}$. Logistic regression is the most basic classifier, it predicts the probability that an input $x$ belongs to a class (class 1), versus the probability that it belongs to another class (class 0). The basic idea of logistic regression is that we learn the logistic function of the form:

[TABLE]

where $x$ is the input vector and $\omega$ is a weight vector for input. The logistic function is a continuous function which can turn any input from negative infinity to positive infinity into an output that is always between zero and one [61]. Fig. 3 illustrates linear and non-linear separable cases for a dataset.

2.3.2 Support Vector Machine

Support vector machines (SVM) are kernel-based supervised learning techniques widely used for classification and regression [60, 62]. The basic idea of SVM is to find an optimal hyperplane for linear separable patterns. It attempts to maximize the geometric margin on the training set and minimize the training error. Then, a kernel function maps the original data into a new space for non-linearly separable cases, resulting in a two-class classification problem. $x_{i},i=1,2,...,N$ are feature vectors of the training set $X$, and of corresponding class indicator $y_{i}\in\{-1,+1\}$. The goal of SVM is to construct a classifier in the form of:

[TABLE]

The function $K(x_{i},x)$ is called the kernel function, and their different mathematical properties enable many pattern recognition and regression models. SVM with a linear kernel equation is computationally faster than SVM with quadratic kernel functions. SVM models using fewer but more significant features are most likely robust and less prone to overfitting [63].

2.3.3 Decision Tree Learning

Decision trees are one of the most popular classification approaches in machine learning [64]. The decision tree consists of a “root”, ”leaves”, and internal nodes [65, 66, 67]. The internal nodes use certain features to split the instance space into two or more subspaces. Each leaf represents one class. The leaf may represent the most appropriate target value or indicate the probability of the target having a specific value. Fig. 4 is an example of the decision tree model. Decision trees are capable of handling datasets that may have missing values and errors, however, this method may overfit training data and add unnecessary features. In radiological image analysis, decision trees are usually ensembled to form random forests for prediction and classification.

2.3.4 Ensemble Learning

Ensemble learning combines multiple classifiers and applies voting algorithms to achieve a final classification. Popular ensemble approaches include boosting and bagging [68]. Fig. 5 shows the basic idea of ensemble learning. In boosting, extra weight is assigned to incorrectly predicted points, and a set of weak classifiers are applied to deal with data in the training phase; the outputs of weak classifiers and the weighted inputs help calculate the final prediction. In bagging, the sub-classifier is independently constructed using a bootstrap sample of the data set and a majority voting method is applied for the final prediction [69]. Random forests are an ensemble learning method that consists of a multitude of decision trees. In standard tree construction, the node is split using the best split among all features. In a random forest, a random subset of features split each node. The random forest is one of the most powerful machine learning predictors used in detection, classification, and segmentation [70], particularly for brain [71, 72] and heart [73, 74] images.

2.3.5 Neural Networks and Deep Learning

Deep learning techniques have become a hot topic in machine learning due to the availability of sufficient computational power and a high volume of data. These approaches can select specific features directly from the data for classification and detection purposes[75, 76]. Deep learning avoids designing specific features from the data, which is its main advantage in comparison with other machine learning methods. Some outstanding frameworks such as the restricted Boltzmann machine [77], convolutional neural networks (CNNs) [78] and sparse autoencoders have proven useful tools in many applications such as Alzheimer’s disease diagnosis [79], segmentation [80], and tissue classification [81]. CNNs have a large number of parameters, which requires vast volumes of labeled training data. However, this requirement makes the training of CNNs from medical images challenging due to the difficulty of acquiring a database with labeled data [82]. However, several studies use CNNs to extract features for medical images and achieve good performance in classification [83, 84].

2.4 Evaluating Machine Learning Techniques

Physicians may rely on the prediction or classification results of machine learning algorithms. However, performing one round of training and testing on data sets may not yield a meaningful idea of the accuracy of an algorithm. Cross-validation reduces the variance of accuracy scores by ensuring that each data instance is used for both training and testing an equal number of times. The cross-validation method randomly splits data into $k$ subsets and holds out each one while training on the rest.

The Dice similarity coefficient is used in segmentation, and it measures the spatial overlap between two segmented target regions[85]. A and B are target regions or volumes, and the Dice similarity coefficient is defined as the ratio of their intersection to the average [86]:

[TABLE]

The Dice similarity coefficient has a value of 0 for no overlap and 1 when pa complete agreement is present. Fig. 6 illustrates the Dice similarity coefficient with different overlaps.

In clinical practice, subjects with a disease are labeled as positive and healthy subjects are labeled as negative. True positive (TP), false positive (FP), false negative (FN) and true negative (TN) are defined as follows:

TP: a test detects the disease when the disease is present

TN: a test does not detect the disease when the disease is absent

FP: a test detects the disease when the disease is absent

FN: a test does not detect the disease when the disease is present

The goal of a computer-aided diagnosis system is to detect as many true positives as possible and minimize false positives and false negatives. There are several popular metrics used to assess classifier outcomes. Sensitivity shows the ability of a test to correctly detect patients with diseases while specificity is the ability of a test to identify healthy subjects correctly. They can be written as:

[TABLE]

[TABLE]

Other popular methods used to assess models include the area under the receiver operating characteristic (ROC) and the top precision value. ROC curves describe the relationship between sensitivity and specificity. The area under the curve (AUC) measures the entire area under the ROC curve from (0,0) to (1,1) and represents the probability that the model can distinguish between classes. Figure 7 illustrates the relation between ROC and AUC. The top precision, the portion of top-ranked relevant images before the top irrelevant database image [38, 87], is a popular evaluation metrics in retrieval systems.

3 Application of Machine Learning in Radiology

3.1 Segmentation

Image segmentation is a necessary step in effective disease diagnosis and treatment in radiology imaging research. It helps clinicians to understand structural information and spatial anatomic relationships, however, it depends on the experience of clinicians and is very time-consuming [24]. Automatic classification methods are essential for improving diagnosis analysis and for the reproducibility of large-scale clinical studies.

3.1.1 Brain segmentation

Tree based methods are hot topics currently being investigated in the brain segmentation field. For example, Yoo et al. segmented multiple sclerosis lesions in multi-3D MR images from unsupervised features[88]. Features were extracted from T2-weighted and proton density MR images using a deep belief network, and a random forest was built for the final supervised classification. In order to improve the model performance from noisy training data and robustness against overfitting, Maier et al. proposed an extra tree forest to locate, segment and quantify sub-acute ischemic stroke lesions [89]. They used voxel-wise local features such as intensity, weighted local mean, local histogram and 2D center distance. However, their method can only deal with the T1-weighted and diffusion-weighted data sequences and high-quality images. Multimodal data from the same patient can provide extra useful information for diagnosis. Therefore, Mitra et al. proposed to use features from multimodal data to segment ischemic lesions, white matter and other secondary lesions. In their study, algorithms combined expectation maximization likelihood estimation and Bayesian-Markov random field to segment the probable lesion areas from FLAIR data then applied random forest on the multimodal data [90].

Neural networks and deep learning techniques are powerful tools in brain segmentation tasks. Si et al. proposed a semi-automatic method to classify the pixels of brain MRI into lesioned and healthy tissues by use of an artificial neural network with gray levels and statistical features as inputs [91]. The segmentation of early-brain tissues is more difficult than that of adult brains due to the lower tissue contrast [92], while multiple image modalities contain complementary information for insufficient tissue contrast [93]. Zhang et al. [94] showed that fractional anisotropy images are more potent in distinguishing gray matter and white matter, and that T2-weighted images have higher performance in capturing cerebrospinal fluid. Zhang et al. proposed a CNN method combining these multiple modality image data to improve segmentation performance. Similarly, Kleesiek segmented the brain and non-brain tissues by feeding data into a neural network with seven hidden convolutional layers [95]. Their model can be applied on any single image modality or a combination of several modalities with varying size. Deep learning methods can also automatically segment MRI images of the human brain into many anatomical regions [96, 97]. As shown in Fig 8, Chen et al. extended ResNet into volumetric brain anatomical segmentation [98]. They integrated the low-level image appearance features, implicit shape information and high-level context to further improve the volumetric segmentation performance [98].

3.1.2 Other segmentation applications

Segmentation is also applied to identify and detect other structures [99], such as organs, bones, muscles, and fractures. Similar to the brain segmentation, tree-based methods are popular as well in other types of segmentation tasks. Lombaert et al. presented kidneys segmentation using the Laplacian Forest [100]. They used intensity within a randomly-shaped cuboid centered around several pixels during their data training. The idea of the Laplacian Forest is to use a guided bagging strategy to produce more related image information for tree models, which have more substantial improvements in model accuracy. Conze proposed a semi-automatic liver tumor segmentation combining a simple linear iterative clustering super pixel algorithm and random forest, which considers the inter-dependencies among voxels [101]. The multi-phase cluster-wise features that consider the spatial consistency applied in their approach are more robust for a random forest. The analysis of the knee also plays vital role in clinical assessment and surgical planning of the disease. The cartilage is typically small, and the segmentation results of Haar-like operators are often unreliable in extracting context features. To overcome these limitations, Liu proposed a novel method using a multi-atlas context forest, which segments bones first and then cartilage [102]. They trained classifiers using appearance features and context features to align the expert segmentation of the atlases in each iteration.

Medical segmentation research utilizes regression-based models. Chen et al. proposed an automated method to localize and segment intervertebral discs from MRI [103]. They used unified regression and classification frameworks to estimate displacements for image points by using the visual features around them and achieved satisfactory results. Ventricle structure segmentation in MRI is an essential task for investigating most cardiac disorders. The primary challenge of this task is the considerable shape variation among different patients [104]. et al. proposed a segmentation method using cascade shape regression for the right ventricle in cardiac MRI. They applied gradient boosted regression to regress multidimensional right ventricle shape landmarks from image appearance, which consider correlations between landmarks. Their method minimizes the shape alignment error over training data and shows better segmentation performance than multi-atlas-label-fusion based segmentation methods.

The other traditional supervised methods applied in segmentation tasks include dictionary learning and Bayes classification. Tong et al. proposed the extraction of voxel-intensity features for multi-organ segmentation (liver, kidneys, pancreas, and spleen) using dictionary learning and a sparse coding technique (Fig. 9) [105]. The atlases selected against which to segment the images profoundly influence the performance of multi-based methods [106]. To deal with the high inter-subject variation in CT images, they applied a voxel-wised local atlas selection strategy to improve performance. Griffis proposed a supervised learning method that automatically delineates stroke lesions using Naïve Bayes classification in single T1-weighted MRI sequence data [107]. In order to save time and money, their approach focuses on using single scan data, which detects direct lesion effects and has a better performance than manual delineation.

Image quality remains a limitation of the extraction of features from the radiology images. In many cases such as brain boundary segmentation, the data is of low contrast by nature. Additionally, both resolution and partial volume effects influence the definition of boundaries [108]. Some research contributions focus on multi-modalities to obtain complementary information [109, 110, 111, 90]; however, it is difficult and inconvenient to apply various testing methods on patients. Also, the accuracy of the segmentation system is difficult to measure and compare because the “ground truth” varies based on the delineation by different experts [112]. However, it is challenging and expensive to obtain manually labeled data from several experts for reliability tests[113].

3.2 Computer Aided Diagnosis

Computer-aided diagnosis (CAD) systems can detect, mark, and assess potential pathologies for radiologists to help improve identification accuracy in the case of data overload and human resource limitation. The analysis, quantification, and categorization of images with these methods is an important technique, which can improve patient safety and care. CAD systems have achieved breakthroughs in the detection of lesions [124, 125], epidural masses [126], fractures [127], as well as a degenerative disease [128] and cancer [129]. Fisher’s linear discriminant, Bayesian methods, artificial neural networks, and SVM are widely used as classifiers in CAD applications [130, 13]. Table 3 summarizes some current CAD investigations with machine learning techniques.

Breast cancer is one of the most common cancers in the world. Currently, about one in ten women suffer from it, and early diagnosis and treatment of breast cancer could increase the chance of survival significantly [152]. Among these diagnoses techniques, mammography is the best approach to detect breast cancer in its early stages and features indicating abnormalities can be extracted directly from medical images [153, 154]. The identification of benign and malignant masses is the core principle for using mammography as a means to diagnose breast cancer [155]. Perez et al. developed machine learning classifiers that combine suitable feature selection methods with different machine learning techniques [131]. The feature selection methods include chi-square discretization, information gain, one rule, relief, and u-test based filter. They then improved their feature selection algorithm called uFilter, which ranks features in a descending manner[134]. Their method was useful for different datasets and reduced the number of employed features without decreasing the classification accuracies.

The SVM classifier is widely used in breast cancer diagnosis with various features, such as wavelet features, gray-level-co-occurrence matrix features, intensity features, and texture features [156, 133]. Banaem et al. proposed a fully automatic tool that can classify the mammogram data into normal and abnormal. They used gray level co-occurrence and maximum difference method to extract proper features and the ensemble classification combining SVM, KNN and Naïve Bayes was applied to improve the diagnostic accuracy [157]. Many investigations not only consider the accuracy of the model, but also the model complexity. Arevalo et al. trained an SVM model that integrated two layers CNN for mass lesion classification [138, 158]. Similarly, Jiao et al. trained two SVM classifiers using deep learning features extracted from two different layers of CNN networks [159]. An automated CAD system was proposed, combing the content-based image retrieval to detect masses [132]. The main idea of their approach is to use scale invariant feature transform features to match query mammogram and exemplar masses in the database, and then uses Naïve Bayes classification and thresholded maps to detect masses. In their method, the model complexity is low as there is no sliding window-based scanning.

The SVM method is also widely studied in other diagnose such as lesion, injury and fractures detection. In these diagnosis tasks, choice of features plays a significant role in model accuracy. Torheim et al. predicted cervical cancer from dynamic contrast enhanced MRI. In their study, gray-level-co-occurrence matrices based textural features were implemented as explanatory variables [63]. Wang improved the accuracy of lung lesion detection from CT images by using a 3D matrix patterns-based SVM with latent variables. Their study focused on detecting lung lesions that had irregular shape and low-intensity, rather than the nodules, which provides a new thought for the detection of lung lesions [135]. In the detection task of thoracic and lumbar vertebral fractures [142], Burns extracted 28 features from the cortical shell from CT images based on the essential method (Denis ‘middle column’), which is specific to detection of fracture discontinuities on vertebral body cortices. Jin et al. established a prognosis model of cervical spondylotic myelopathy using a least-square SVM [160]. In their studies, they extracted values of fractional anisotropy, axial diffusivity, mean diffusivity and radial diffusivity from each slice of DTI metrics as features, which yielded 88.62% prediction accuracy.

The popular methods such as deep relief networks [161] and convolutional neural networks [162, 140] achieved promising results in many diagnosis applications. The important diagnosis tasks based on neural networks include chest pathology identification [163], cancer detection[164, 165] and lung diseases [166] . Neural network based methods rely heavily on the support of big data. A semi-supervised algorithm has been proposed to deal with a large amount of unlabeled data with CNN approaches [167]. Their approaches using unlabeled data increased the overall accuracy, rather than just using labeled data.

There are many advantages to using machine learning techniques in CAD systems. The first advantage of machine learning is its accurate and robust performance in many radiology studies. For instance, CAD systems have reached perfect accuracy e.g., over 99$\%$ in oral cancer detection[168], which is comparable to manual diagnosis. Moreover, CAD systems are expected to perform consistently and produce robust results with large amounts of data at any time and in any space, while manual diagnosis results may be affected by fatigue, reading time, and emotion on the part of the practitioner. The second advantage is that the diagnosis can be finalized in a brief time. Many radiology analyses are time-consuming and require experienced radiologists. For example, the software developed for breast cancer prediction[169] can review charts 30 times faster than humans can. Another example is that the suggested approach in breast cancer diagnosis is the double reading of mammograms by two radiologists [131]. With the help of a CAD system, only one radiologist is needed instead of two, which could help to increase the survival rate among women in a cost-effective manner [170].

Although we are witnessing better accuracy of computer-aided diagnosis systems to tackle the most common clinical problems, current contributions still have potentials for improvement before their applications in clinical practice. First, a majority of current diagnosis contributions mainly focus on the prediction of one type of disease, which may not meet the clinical demands. There may be one or more diseases existing in one radiological image (for example, effusion & atelectasis in one chest x-ray image). Second, the current model trainings is mainly based on one type of measurement. However, most disease decisions in clinical practice rely on multiple domain measurement (such as patient demographics, image screening, blood test and drug test). Information from multi measurement may increase model accuracies. Third, current medical datasets mainly cover common diseases. Only a limited number of rare diseases are exposed to human clinicians, and many contributions may not consider these individual cases during their model training. More comprehensive systems that can detect various types of diseases and report rare cases are expected to be seen in the future.

3.3 Functional Brain Studies and Neurological Diseases

Brain tumors, neurological disorders such as epilepsy, and neurodegenerative diseases have attracted much attention in brain-related investigations. In brain-related image diagnosis, a large number of features can be extracted from brain regions related to the nature of pathological changes. Cortical thickness [171], the volume of brain structures [172], and voxel tissue probability maps around some regions of interest [173] are popular choices for feature extraction [174]. Different MRI modalities such as T1-weighted or fluid-attenuated inversion recovery imaging contain large amounts of information and noise [91]. Therefore, compelling feature fusion strategy is necessary for neuroimaging analysis and classification [175, 176].

3.3.1 Support vector machine in brain studies

In brain studies, the SVM is a powerful tool for feature selection, which may improve model accuracy. Larroza et al. developed a classification model of brain metastasis and radiation necrosis in contrast-enhanced T1-weighted images. Features were extracted by texture analysis and reduced by using a linear SVM [31]. Bron proposed a feature selection method based on the SVM significance value [177]. The significance value (p-value) serves to quantify the contribution of each feature to the SVM classifier and is used to reduce voxel-based morphometry features. Neurodegenerative diseases such as Parkinson’s disease begins before the onset of symptoms. Thus, medical treatment is more effective if it is detected in early stage. Among various forms of Parkinsonism, progressive supranuclear palsy is one of the most difficult to be identified in an early disease stages [178]. Salvatore et al. proposed to classify control subjects, progressive supranuclear palsy patients, and Parkinson’s disease patients based on SVM models. Features were extracted by spatial transformations and principal component analysis from T1-weighted sequences. The accuracy of discrimination of Parkinson’s disease and progressive supranuclear palsy is above 90$\%$[24]. Fig. 10 uses a color scale to express the importance of each region during classification. To improve the diagnostic accuracy of classifying Parkinson’s disease patients, Singh proposed an unsupervised feature extraction method from a T1-weighted sequence by using a Kohonen self-organizing map algorithm. With the least square SVM, the accuracy of identifying the affected area in Parkinson’s disease is up to 99$\%$ [179]. In [174], features were extracted using a deep network and a stacked denoising sparse autoencoder, which makes the input data points more linearly separable in SVM [174]. Liu proposed an inherent structure-guided multi-view learning method to classify Alzheimer’s disease and mild cognitive impairment patients [180]. They extracted 1500 features from gray matter density, and multi-task feature selection was applied to reduce the dimension, followed by an ensemble classification method using multiple SVM classifiers.

Besides the disease studies, some research work applied machine learning techniques to understand the brain’s functional network architecture. Smyser compared the fMRI data from 50 preterm-born and 50 term-born infants using SVM [181]. Their results show that inter- and intra-hemispheric functional connections throughout the brain are stronger in full-term infants. Their findings might be helpful for the development of models for defining indices of brain maturation.

3.3.2 Ensemble learning in brain studies

Ensemble learning methods combines multiple classifiers, which is popular in Alzheimer’s disease diagnosis. Alzheimer’s disease is estimated to affect around 5.4 million patients in America, and is the most common form of dementia among the elderly population [182, 128], which leads to the loss of cognitive function and death. Liu proposed a classification framework that works on different image modalities for the classification of Alzheimer’s disease patients [175]. Their method contains level classifiers: low-level classifiers that use different types of low-level features from patches, high-level classifiers that combine coarse-scale imaging features in each patch and outputs of low-level classifiers, as well as a final ensemble classification that combines the decisions of a high-level classifier with a weighted voting strategy (Fig. 11). In [183], high accuracy results were obtained from Alzheimer’s disease/healthy and mild cognitive impairment/healthy classification. However, accuracies in classifying mild cognitive impairment as converted to Alzheimer’s disease are very low (57.4$\%$), but is slightly higher than majority classification. Komlagan et al. developed an ensemble learning method using gray matter for a weak classifier and selecting the most relevant sub-ensembles through sparse logistic regression [184]. They trained a global linear SVM classifier for the final classification. Combining high quality biomarkers with advanced learning methods makes results comparable to those of multi-modality methods.

3.3.3 Others techniques in brain studies

Some researchers leverage regression and principle analysis components in classification and feature mapping. Ahmed et al. detected neocortical structural lesions with an automated approach, which contained five surface-based MRI features and combined them in a logistic regression [185]. To deal with imbalance issues, they used a “bagging” approach and an iterative-reweighted least squares algorithm. The base-level classifier was trained on all the minority class instances and the same size of random data from majority class instances. Hong proposed a machine learning technique combining surface-based analysis in patients with a subtype of focal cortical dysplasia [186]. Their automated approach used features of Focal cortical dysplasia morphology and intensities, and Fisher’s linear discriminant was applied as a classifier to identify Focal cortical dysplasia in patients. Huang proposed the use of a soft-split random forest to predict clinical scores in Alzheimer’s disease patients [187]. In their method, lasso regression is applied to map MRI features, and then features are reduced by principal component analysis. Li combines principal component analysis, the lasso method, and a deep learning framework to extract features by fusing information from MRI and PET images in the classifier of Alzheimer’s disease/mild cognitive impairment patients [183]. Zhu et al. focused on the identification of Alzheimer’s disease patients with multi-view or visual features of image data. They proposed several feature selection approaches for Alzheimer’s disease classification. They integrated subspace learning into a sparse least square regression framework for multi-classification in 2014 [188]. Then, they mapped the histogram of oriented gradient features (which are diverse) onto a region of interest features (which are robust to noise), which provided complementary information for features and enhanced disease status identification accuracy [189]. Other machine learning techniques such as convolutional neural networks are widely investigated in the field as well. Table 4 summarizes recent contributions related to Alzheimer’s disease classification.

3.4 Image Retrieval

With the increased use of modern medical diagnostic techniques, there are numbers of medical images stored in hospital archives. Manual annotation and attribution of these images are impractical [14]. Picture archiving and communication systems have been widely introduced in many hospitals [198]. These systems could retrieve images based on keywords, however these images may not be directly useful in helping to making clinical decisions. Different from traditional image search systems, which are based on matching keywords and image tags, content-based image retrieval extracts rich contents from images and searches for other images with similar contents. Content-based image retrieval is becoming necessary for the medical image databases, which may potentially become effective tools of anatomical and functional information for diagnostic, educational, and research purposes [199]. Table 5 lists current investigations on image retrieval. Recently, similarity or distance learning is a hot topic in the image retrieval field. Traditional choices include the Euclidean distance function, $x^{2}$ square distance function, Mahalanobis distance, $l_{1}$ norm distance function [38], maximum likelihood approach [200] and Bayes ensemble [201]. Like other machine learning tasks, features extraction is an important step in image retrieval systems. Kurtz et al. proposed the use of hierarchical semantic-based distance to retrieve images based on 72 manually annotated semantic terms from each region of interest [202]. Then, they built a semantic framework that learns image descriptions of each term using Riesz wavelets and SVM. In [203], local wavelet patterns were introduced as a new feature descriptor. Their experiments first utilized the relationship among the neighboring pixels and performed well in CT image retrieval. Their results were shown in Fig. 12. Different from traditional similarity learning that only maximizes the margin, Meng et al. proposed a novel similarity learning algorithm which considered the top precision performance measure in the loss function [38]. Their methods showed advantages over other traditional similarity learning methods.

The other supervised techniques applied in retrieval systems include online dictionary learning, ensemble learning and principal component analysis. The main advantage of an online dictionary learning system is the computational time, as learned dictionaries are used to represent the dataset in a sparse model, which is an valuable tool for representing data [204]. A method using online dictionary learning and its features extracted by multi-scale wavelet packet decomposition from different types of images is proposed in [205]. Srinivas et al. proposed a medical image classification approach using online dictionary learning with the edge- and patch- based features to distinguish 18 categories [206]. Ahn et al. developed a robust method to improve X-ray image classification [207]. A fusion strategy was proposed that combines domain transferred convolutional neural networks and sparse spatial pyramid classification. The combined method performs better than the single method used. Faria et al. proposed a retrieval method for brain MRI images. They captured anatomical features from T1-weighted images using least-square discriminant analysis and principal component analysis and performed a search for images between healthy controls and patients with primary progressive aphasia [208].

As large portions of medical images in the dataset lack labels and annotations, semi-supervised and unsupervised techniques are required in the retrieval systems. Uunsupervised image retrieval based on the clustering method using K-SVD executes iterations between grouping similar images into clusters and generating a dictionary for clusters until clusters converge [42]. The advantage of this method is that it requires no training data for classification and is not restricted to a specific context. Since labeled data is limited, Herrera proposed a semi-supervised learning method for image classification using k-nearest neighbors to expand the training data set and a random forest for final classification [113].

Medical image retrieval gives an opportunity for clinicians to search for similar disease cases. Accuracy and performance time are both vital aspects of the medical image retrieval system. A practical model and relevant image feature extraction are required to get better results. Furthermore, some image retrieval contributions mainly investigated small datasets and limited disease cases. With an increasing number of digital radiological images in hospital databases every year, whether systems can retrieve disease cases stably and efficiently in huge datasets is still an exciting avenue for researchers.

3.5 Image Prediction

With the development of neuroimaging techniques, various new image modalities have been applied in daily clinical practice to make diagnosis and treatment more efficient and accurate. Thus, image prediction methods, which combine various image modalities and provide information for diagnosis, are fundamental. The main idea of image prediction is to estimate radiological images in different modalities or higher resolution, which can provide detailed functional information for assessment and diagnosis. PET is a molecular imaging technique which is widely used in clinical cancer diagnosis that produces 3D images, which can reflect tissue metabolic activity in the human body [213]. However, the quality of PET images is proportional to the dose injected and imaging time. As a result, low-dose PET images can suffer in quality. Thus, a great deal of effort has been made to predict high-quality PET images. Kang proposed a regression forest based approach to predict standard-dose PET images from low-dose PET and multimodal MRI images [214]. They used a regression forest as their non-linear prediction model and features from local intensity patches of MRI data and low-dose PET. Meanwhile, Wang used a mapping-based sparse representation approach for prediction [11]. They used a graph-based distribution mapping method to reduce the patch distribution differences between MRI and low-dose PET and constructed a patch selection based dictionary learning method to predict standard-dose PET. Both methods performed better when compared with a path-based sparse model. In [215], Xiang et al. used convolutional neural networks to estimate standard PET image from low dose PET/MR image (Fig. 13). By using neural network techniques, they can map the inputs to the output directly, without any pre/post-processing beyond the optimization in the training stage. Huynh predicted CT images from MRI data using a structured random forest instead of a classical random forest [70]. A structured random forest is an extension of a random forest, which predicts structured outputs instead of scalar outputs [216, 217]. Characterizing the information obtained from multiple sources of information improves prediction accuracy.

Compared to classification and segmentation tasks, contributions involving radiological imaging prediction is still quite limited. The rapid developments in hybrid imaging scanners (PET-CT, SPECT-CT, PET-MRI) has provided integrated images for diagnostic purposes. The primary challenge remains to find the appropriate way to match the correspondences among different images modalities [218]. Besides, current contributions are mainly focused on brain data, but we expect to see more non brain related contributions to this area soon.

4 Current Challenges

Until now, machine learning has been used to help radiologists in diagnostic tasks, but still cannot be a substitute for the clinician’s role. There are some limitations regarding the application of machine learning in clinical practice [219]. One such limitation is that the majority of these studies in radiology are based on supervised learning. The algorithms learn specific patterns based on previous decisions made by radiologists. The algorithm is expected to reach an accuracy of 100$\%$ compared with the human clinician, however, in many cases, an accurate diagnosis was made by several radiologists after multiple diagnostic tests. Whether a machine can perform well alone still needs to be investigated in the future.

Facilitating data collection and sharing is a crucial point for further investigation in many studies. Clinically applied algorithms depend on two critical factors: robustness for large datasets and accuracy achieved [220]. In many machine learning related studies, the size of the chosen dataset is relatively small [94, 208, 107, 89, 103]. This is due to limited access to patients or limited diagnostic work in the research setting. For example, in some clinical practices, there is no pathological exam done as a follow-up procedure in cases of brain metastasis [31], which limits the acquisition of data. The application of machine learning to a limited number of cases (only around twenty patients were studied in some cases) is not always persuasive. Whether these clinical tools are robust enough to analyze immense amounts of medical data accurately remains a question. While 20-50 images were sufficient in past research, hundreds of or more image sets are required in the future to meet increasing requirements concerning robustness and accuracy. The creation of large databases and sharing centers such as ADNI for Alzheimer’s disease patients, NIH repositories for chest X-rays, and TCIA for cancer imaging help to effectively collect millions of images for research. Furthermore, current studies trained and evaluated their models based on various datasets, which makes it challenging to compare their algorithms. A systematic evaluation standard based on various diseases and various public datasets is required in medical applications.

Excepting the data size, data quality and feature selection remain highly important for effective machine learning techniques. Medical images contain rich features that are clinically important. It is challenging to use low-level image features to get the visual appearance of disease. However, high dimensional features could be redundant for results in many fields such as image retrieval and classification. The choice of different high-level descriptions as input features is a prominent research topic. Choosing informative features for training can lead to robust models, whereas overfitting, underfitting, and misclassification usually occur when features are not selected well. More work remains to be done for selecting and utilizing proper features from images.

Transfer learning is an accepted scenario to learn information from limited datasets. There are three main transfer learning research directions in the field of radiological imaging. One is used to reduce bias among different equipment for image acquisition and different protocols [123, 221]. The second approach is to learn various abnormalities from the same data source [222, 223]. The last approach is to find a good feature representation from various domains and then apply them to the radiological imaging field [224]. Transfer learning allows us to deal with various scenarios by leveraging the already existing information from some related task or domain. For more details on transfer learning techniques and radiological imaging, readers should refer to [225].

Imbalanced data is standard in medical diagnosis, where the majority of data is normal data, but only a minority class is abnormal [226]. For example, brain tumors are not common, occurring only in 1 $\permil$ of the population. However it remains the most fatal form of cancer [16]. This data imbalance might affect prediction accuracy and cause a bias toward the majority class [227]. Several researchers considered the imbalanced situation in their models [185, 38, 94], however, the majority of studies have not properly addressed this issue [94, 228], or they use the same amount of data from different classes. How to utilize imbalanced data to improve the accuracy of machine learning algorithms remains an open question in this field.

Large amounts of radiological images are produced in hospitals every year. However, most of these images are not utilized for further training of machine learning algorithms, as the training process constrains available resources. Useful information is hidden in this mass of data, and diagnostic machine learning models could be improved by using these streaming data. Online learning is a novel idea in recommender systems and other machine learning based systems that could update the model while streaming data are currently developed in other fields. This idea can also be transferred to medical diagnosis systems to make full use of streaming image datasets.

Researchers have generated more powerful and more innovative diagnosis models for radiological imaging [229]. However, very few of them are commercialized and deployed into the market. The main challenge is to comply with government requirements in various countries [230]. Current FDA protocols suggest that medical products should pass the clinical trials, and be produced, commercialized, and used in a defined, unchanging form. If a machine learning model is used for medical diagnosis, a pre-build and freeze model must be tested in different clinical environments, assessed for various real-life medical conditions, and carefully evaluated on how these conditions affect the accuracy of the diagnosis. A recent study showed that a pre-trained model demonstrated significantly lower external performance on the data obtained from another hospital system [231]. In order to reduce the bias and improve performance, current machine learning solutions in nonmedical fields typically update the parameters of the models every time new data is included. However, this is not realistic for medical products as the system must pass a new clinical trial after the update. Therefore, this remains a major issue for machine learning algorithms in medical applications.

Knowing how deep neural networks work is an open question, and this prompts clinicians and patients to distrust these models. Due to huge amounts of parameters in the models, it is difficult to interpret how the models make diagnostic decisions between input and output. This could potentially be fatal if a machine learning model leads to a wrong conclusion [232], as medical experts can not verify these models. Deep learning researchers have recently computed heat maps using class activation maps in order to a more concrete analysis of how these models perform [233]. An activation map visually highlights the discriminative regions in medical images that models used to identify the category. However, the related researches on network explainability and visualization are still limited in the field of radiological imaging.

5 Conclusion and Future Work

In this paper, we reviewed five applications of machine learning techniques on radiologic images: image segmentation, computer-aided detection and diagnosis, functional brain studies and neurological disease diagnosis, image classification and retrieval, ands image registration. While machine learning techniques are active in computer-aided systems to assist radiologists in daily diagnosis and studies, the use of machine learning techniques in radiology is still evolving. There are many strategies that this field could investigate in the future：

Previous contributions have shown that machine learning-based systems showed accurate results comparable to those of radiologists themselves. However, the system accuracy of these techniques must still be improved, that is, systems must be more accurate than those of radiologists. Otherwise, the widespread application of machine learning techniques will be limited. A possible approach to achieve this superior performance is to design better machine learning models or to gather more representative data that can be continuously used to improve the algorithms.

• -

Although the core advancement of deep learning is its ability to learn useful features directly from data, its accuracy and performance are highly limited by the size of data. Traditional machine learning methods still play a role in the case of small amounts of labeled data. Understanding how to choose and use features from images effectively is still a significant direction for these traditional methods.

• -

Another issue deals with the translation of these techniques to clinical practice. While many machine learning algorithms have already shown good results, it still needs to pass clinical trials required by the government. Additionally, many people still believe in human decisions, as clinicians always consciously tend to decide with all the relevant information in mind. These decisions make it difficult to justify the use of algorithms for clinical decision making in all possible cases, but through rigorous research contributions, we can justify the use of machine learning algorithms in the cases when patients’ outcomes can be improved.

The application and development of machine learning techniques to radiological images is a hot topic currently and a large number of algorithms are being developed to ensure higher accuracy and lower computational complexity. We expect that machine learning techniques will become essential components in clinical tools and will be widely used to assess patients’ health in the future.

Acknowledgment

Research reported in this publication was supported by the Eunice Kennedy Shriver National Institute of Child Health & Human Development of the National Institutes of Health under Award Number R01HD092239. The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institutes of Health.

Conflicts of Interest

None declared.