Deep Learning Classification in Asteroseismology

TL;DR

This paper develops a convolutional neural network to classify red giant stars' evolutionary stages from their oscillation spectra, achieving high accuracy and significantly expanding classified star datasets.

Contribution

The study introduces a CNN-based method for automatic classification of red giant stars' evolutionary states using Kepler data, with high accuracy and broad applicability.

Findings

Achieved up to 99% accuracy in classifying stellar evolutionary stages.

Successfully classified 5379 previously unclassified Kepler red giants.

Demonstrated CNN's capability to learn visual features from oscillation spectra.

Abstract

In the power spectra of oscillating red giants, there are visually distinct features defining stars ascending the red giant branch from those that have commenced helium core burning. We train a one-dimensional convolutional neural network by supervised learning to automatically learn these visual features from images of folded oscillation spectra. By training and testing on \textit{Kepler} red giants, we achieve an accuracy of up to 99\% in separating helium-burning red giants from those ascending the red giant branch. The convolutional neural network additionally shows capability in accurately predicting the evolutionary states of 5379 previously unclassified \textit{Kepler} red giants, by which we now have greatly increased the number of classified stars.

Figures

Tables

Table 1. Table 1 : Metrics over the mean of 10-fold cross-validation (CV) and over the test set.

Dataset	CV ($\pm \mathrm{𝟏}$ std.)	Test
Accuracy	0.982 $\pm$ 0.005	0.990
Precision	0.982 $\pm$ 0.005	0.990
Recall	0.982 $\pm$ 0.005	0.991
F1 Score	0.982 $\pm$ 0.005	0.991
ROC AUC	0.998 $\pm 0.002$	0.996
Log Loss	0.055 $\pm 0.020$	0.044

Equations

$$g^{(l)}=f\bigg{(}\sum_{i=0}^{m}\boldmath\mathrm{w}^{(l)}\circ\ \mathrm{x}_{i}\bigg{)},$$

$$p(y=j|\text{x})=\frac{e^{\text{x}\cdot {\text{w}}_j}}{{\sum }_{k=1}^K{e}^{\text{x}\cdot {\text{w}}_k}},$$

$$E(\mathbf{y,\hat{y}})=-\frac{1}{m}\sum_{i=1}^{m}\bigg{[}y_{i}\log\hat{y}_{i}+(1-y_{i})\log(1-\hat{y}_{i})\bigg{]}.$$

Full text

††pubyear: 2017††pagerange: Deep Learning Classification in Asteroseismology–Deep Learning Classification in Asteroseismology

1 Introduction

A key concept in determining stellar ages of red giants is distinguishing the evolutionary state i.e. classifying between hydrogen-shell burning stars ascending the red giant branch (RGB) and those that have commenced core helium burning (HeB). Space missions such as Kepler (Borucki et al., 2010) have provided an enormous quantity of red giant oscillation spectra. The upcoming TESS mission (Ricker et al., 2014) will further increase the amount of data, making manual or semi-automatic classification of the population class of each star infeasible. Automated methods do exist, however considerable effort is required for defining and acquiring features such as the observed period spacing $\Delta P$ (Bedding et al., 2011; Stello et al., 2013), the asymptotic period spacing $\Delta\Pi_{1}$ (Bedding et al., 2011; Mosser et al., 2014; Vrard et al., 2016), or the structure of mixed modes (Elsworth et al., 2016) in order to separate the populations. Furthermore, these methods require relatively high signal-to-noise data.

Here, we present a deep learning method that allows spectral features to be learnt by the machine using convolutional neural networks. These are machine learning methods that mimic biological neuron structures, aimed towards feature detection in data (Fukushima, 1980). They have achieved significant success over the past few years in computer vision methods such as image recognition (Krizhevsky et al., 2012), and even facial recognition (Garcia & Delakis, 2004).

We introduce the concept of representing the oscillation frequency spectra of stars as ‘images’. While spectra are represented as a 1D array of values and are not images in the conventional 2D sense, we interpret the spectra in an image-like fashion in such a way that we wish to identify the spatial structure of the spectra in frequency as visual features; as seen by ‘an expert eye’. Hence, these images are simple representations of the power excess, which contain visual features for RGB-HeB classification that the neural network can ‘see’ and hence learn from. By learning from an existing set of classified red giants based on $\Delta\Pi_{1}$ measurements, we use 1-D convolutional neural networks as a form of supervised machine learning aimed to automatically learn features separating RGB from HeB stars in order to make fast yet accurate classification predictions on vast amounts of unclassified stars.

2 Methods

Here we describe the preparation of the image representation known as a folded spectrum, along with an overview of convolutional neural networks, and the construction of a deep learning classifier to classify the image representation.

2.1 Data

We obtain the evolutionary state classifications of 5673 Kepler stars based on automated asymptotic period spacing measurements by Vrard et al. (2016, hereafter Vrard), and add 335 stars from the classification by Mosser et al. (2014, hereafter Mosser) that are not already in Vrard’s sample, to a total of 6008 stars. We then assign RGB stars with the binary class 0 and HeB stars with class 1. About 30% of stars in the dataset are RGB stars. We randomly choose 1008 stars as test data, with the remaining 5000 stars for training. Additionally, we have an unclassified set comprising 8794 Kepler red giants that are known to oscillate but have not been given classifications by Vrard or Mosser. We want to predict the population labels of all stars in our unclassified set using our trained neural network.

2.2 Image Representation

As our image representation, we define the folded spectrum as the $4\Delta\nu$-wide power spectrum segment centred at $\nu_{\mathrm{max}}$, folded by a length of $\Delta\nu$ (see Figures 1a, c). The spectra and values for $\Delta\nu$ and $\nu_{\mathrm{max}}$ were derived from end-of-mission Kepler data using the SYD pipeline (Huber et al., 2009, Yu et al., in prep.). Because the neural network requires a fixed input array length, we bin each folded spectrum into 1000 bins.

A comparison of spectral image representations between RGB and HeB stars are shown in Figures 1c, d. RGB stars clearly exhibit acoustic modes that are highly localized (Figures 1a, c) while HeB stars show broader mode distributions particularly for non-radial modes because of the stronger coupling between core and envelope (Figures 1b,d) (Dupret et al., 2009; Grosjean et al., 2014). With acoustic resonances less localized, HeB spectral representations notably have greater visual complexity as compared to RGB spectra. Besides the structure of modes, the location of the $l=0$ mode, represented by $\epsilon$, can be a strong indicator in distinguishing population classes (Kallinger et al., 2012). However, $\epsilon$ is not the sole feature that is used to recognize population classes from an image. The lack of a clear boundary separating the two evolutionary states shown by the observed spread in $\epsilon$ (Kallinger et al., 2012) and from theoretical studies (Christensen-Dalsgaard et al., 2014) makes $\epsilon$ unsuitable as a sole selection criterion. However, information about $\epsilon$ complements features extracted from mixed modes in the image.

As image pre-processing, we normalize each spectrum by its max power value. Then, to avoid edge effects, we append the image with a copy of itself and apply a super-Gaussian (higher-order Gaussian) weight function as shown in Figure 2.

2.3 Convolutional Neural Networks

An artificial neural network is a mathematical representation of a biological neuron network (Figure 3a). Mathematical neurons contain real numbers and connect to other neurons in subsequent levels (network layers) by mathematical operations in order to form a network capable of computing solutions to complex problems. The total input from one layer to a neuron in the next network layer is given by $\mathrm{w\cdot x}$, where \mathrm{x}$$=(x_{0},x_{1},x_{2},x_{3},...,x_{n}) is an input vector with $n$ number of features from the input layer (represented by the number of neurons in the input layer in Figure 3a), with $x_{0}=1$. In our study, $\mathrm{x}$ is the power in each frequency bin of the image at the very first input layer. In subsequent layers, $\mathrm{x}$ will become manipulated representations of the original input image. The weight vector, \mathrm{w}$$=(w_{0},w_{1},w_{2},...,w_{n}), links each input to a neuron in the subsequent layer, with $w_{0}$ known as the input bias, $b$, which is analagous to the intercept in a linear regression.

The total input is linear, however it is passed through a non-linear activation function (Rosenblatt, 1962), $f$, such that the network becomes capable of approximating complex non-linear representations. In this study, we use the rectified linear unit activation function $f(x)=\mathrm{max}(0,x)$ for every neural network layer except the output layer. This function is suited for feature learning in neural networks (Nair & Hinton, 2010). A common design in neural networks is to stack multiple layers and have the input pass through consecutive intermediate or hidden layers to reach the output layer (Figure 3b). Such a design is known as a feedforward neural network, because inputs are computed and fed forward through the network to the output layer. A simple feedforward neural network has fully-connected layers, such that each neuron in a layer is fully connected to the neurons in the following layer. Each neuron connection for a fully-connected layer is permitted to have distinct weights.

Convolutional neural networks (LeCun & Bengio, 1998) are a variant of feedforward neural networks in which the layer connections are constrained. This constraint comes in the form of weight sharing, where weights across neurons within a layer are constrained to only a fixed set of values, known as a filter. The content of the neurons in a convolutional layer is computed by sliding this filter across neurons in that layer (Figure 3c). Hence, the filter is analagous to a kernel convolution. By using a fixed-length filter as weights instead of allowing each neuron connection to have their own distinct weights, features across a local ‘patch’ of data, for instance, the shape of an $l=1$ mixed mode in Figure 2b, can be learned to be detected by the network. Detected features across a layer are computed and then stored in a feature map (Rumelhart et al., 1988). Feature maps act as the hidden layer for convolutional layers, with the exception that whereas fully-connected layers usually have a single array for its hidden layer, convolutional layers typically have more than one feature map because the detection of multiple features require multiple feature maps. A feature map is given by:

[TABLE]

with $m$ denoting the number of stars in the data set and $l$ being the feature map index. Each feature map can be said to represent a detected local spatial feature of the data. However, in image recognition, as a greater number of convolutional layers are added, the outputs of deeper layers often become increasingly difficult to interpret from a human visual perspective (Zeiler & Fergus, 2014).

A pooling layer is commonly applied after convolutional layers. Pooling reduces the length of convolutional layer outputs by clustering adjacent neurons into a fewer number of neurons over a specific function that compares the neuron values. To illustrate this, in our study, we use pooling layers that apply a 4-neuron max-pooling, which reduces a collection of 4 adjacent neurons into 1 neuron by selecting the neuron with the maximum value among them. In principle, this achieves a form of local spatial invariance within layers (Bengio, 2013), while reducing the complexity of the neural network at the same time because fewer weights are required for a smaller layer size. Following feature detection by multiple iterations of convolution and pooling layers, it is common for a fully-connected layer to be connected for a final layer of computation before the output layer. A general structure of a convolutional neural network is illustrated in Figure 4.

2.4 Learning and Optimization

The objective of the convolutional neural network is to learn a particular set of weights from a training set that minimizes the error in approximating a ground truth ${y}$ with a predicted output $\hat{y}$. In binary classification, $y$ are binary values (associated with the binary classes; 0 for RGB and 1 for HeB) while $\hat{y}$ are 2-element vectors with output scores for each class. We use the normalized exponential function known as the softmax function at the output layer, such that each output neuron contains the value:

[TABLE]

which defines the score of class $j$ out of $K=2$ classes. These scores are similar to probabilities, however, probability calibration is usually required to express these scores in terms of actual population probabilities. Since such work is beyond the scope of this study, it is sufficient to interpret the output scores as the prediction likelihood of a certain population class for a star. A score close to 1 indicates a high predicted likelihood for a HeB star, while a score close to 0 implies a high likelihood for an RGB star. We use a score threshold of 0.5 to assign the dominant population label of a target, such that a predicted score close to 0.5 implies a lack of classifier confidence in identifying the star’s population label as both populations ’appear’ to be equally likely to the classifier. This score threshold has been found to provide a good separation of populations in our classifier.

A suitable error function to minimize is the cross-entropy or log-loss (Murphy, 2012), related to the difference between a true value, $y$, with a predicted value, $\hat{y}$. Thus, the log-loss is the measure of similarity between $y$ and $\hat{y}$. We compute the error, $E$, of the network by taking the average of all cross-entropies of the network’s predictions on $m$ stars, such that

[TABLE]

The error minimization uses a gradient descent algorithm, where the error derivatives with respect to a layer’s weights are calculated from the end output and backpropagated to previous layers (Rumelhart et al., 1986). Layer weights are then updated sequentially over small subsets (mini-batches) of the training examples, which speeds up learning compared to updating over the full training set at once because it approximates the gradient and the error surface curvature (LeCun et al., 1998). We use a mini-batch size of 128 to train our network. During the training process, multiple passes of feed-forward and backpropagation are iterated until the error converges to a minimum. By training a convolutional neural network with a suitable optimization objective, we obtain a classifier (a fixed network with a trained set of fixed weights) that takes in our image as input and outputs a two-element vector $\hat{y}$ with the elements as the RGB and HeB likelihoods, respectively.

2.5 Classifier Structure and Hyperparameters

A deep learning classifier usually has multiple stacks of neural network layers on top of one another as its structure. This structure contains a vast combination of free parameters (hyperparameters), which have to be empirically determined. Our classifier structure (Figure 5) uses two parallel yet identical convolutional layer stacks such that both stacks see the same input. The sequence of Convolution-Pooling-Dropout (see explanation of dropout below) is carried out 3 times in succession before merging into a fully-connected layer. We use a filter size of 32 in each convolutional layer with $2/4/8$ feature maps for iterations 1/2/3. This structure is defined by choosing the simplest, computationally cheap combination of structure and hyperparameters that has the best metric performance on a hold-out validation set. This set is part of the initial training data that is now left out in training for classifier structure selection.

To prevent overfitting on training data, we use dropout layers (Hinton et al., 2012), which randomly sets layer outputs to zero with probability $p_{\mathrm{drop}}$. This prevents layer outputs from memorizing the training data. We performed dropouts with $p_{\mathrm{drop}}=0.6$ after each convolutional layer and constrain the norm of the layer’s weight vector to a value of 2. This combination has been shown to be effective at preventing overfitting (Srivastava et al., 2014). We constructed the classifiers with the Keras library (Chollet, 2015) built on top of Theano (Theano Development Team, 2016). With a Quadro K620 GPU, training with 200 training iterations requires approximately 15 minutes, while predicting on thousands of new stars takes only a few seconds.

3 Results

3.1 Classifier Performance

We report metrics on the mean of 10 separate hold-out validation sets, with each set having one-tenth of the training set in size (10-fold cross-validation), and on the test set. The metrics used to describe classifier perfomance are defined as follows:

Accuracy:

The number of correct predictions out of all predictions.

Precision(P):

For a class, the ratio of correct predictions to all made predictions towards that same class. Here it is the classifier’s ability to not label a HeB star as an RGB star.

Recall(R):

For a class, the ratio of correct predictions to all stars truly in that same class. Here it is the classifier’s ability to find all HeB stars.

F1 Score:

The harmonic mean of precision and recall, defined by $2\frac{P\times R}{P+R}$, with 1 as a perfect score.

ROC AUC:

Receiver Operating Characteristic’s Area Under Curve, which measures the classifier’s average performance across all possible score thresholds. Has a value of 1 for a perfect classifier (Swets et al., 2000).

Log Loss:

Negative logarithm of prediction scores i.e. the cross entropy. Measures how well prediction scores are calibrated with an ideal value of 0.

Precison, recall, and the F1 scores complement accuracy in cases like ours where the population ratio is far from 50:50, while ROC AUC evaluates the overall classifier performance. Log loss reports the performance of class score outputs, with confident predictions rewarded low error when correct while penalised heavily otherwise.

As observed in Table 1, the accuracy of the classifier on the cross-validation sets is generally above 98%. On the test set where the classifier benefits from training on the entire training set, the classifier is capable of classifying with a 99% accuracy and suffers a lower log loss. Having high values of precision, recall, and F1 score also indicates that the classifier is not heavily biased in predicting a particular population class that would not reflect the true population ratio.

Figure 6 shows the test set results in $\epsilon-\Delta\nu$ (Huber et al., 2010; White et al., 2011) and $\Delta P-\Delta\nu$ (Bedding et al., 2011) diagrams. We derived the $\epsilon$ values using the method described in Stello et al. (2016a); Stello et al. (2016b). One can see that ‘disputed’ predictions, namely predictions that are not in agreement with the "truth" labels from Vrard or Mosser, are more concentrated towards the low-$\Delta\nu$ regions. The classifier appears to be confident in most of its predictions (deep red and deep blue symbols), while most of the uncertain predictions are disputed. Upon inspection of the spectra of the 10 disputed stars, we visually verify that four of them, all with $2.9\mu$Hz $<\Delta\nu$ $<5.2\mu$Hz, had incorrect ground truth labels. Another four are confirmed to be due to the classifier’s inaccuracy. These stars have $\Delta\nu$ $>7.0\mu$Hz in the diagrams. The final two stars are "high" luminosity red giants with $\Delta\nu<2.9\mu$Hz. Visual inspection was inconclusive as the spectrum of one had suppressed dipole modes with a moderate level of noise, while the other appeared much like an RGB star but was previously given a late HeB classification as its ground truth. From theory, we do not expect to see a clear difference between RGB and late HeB stars because of the lack of coupling between core and envelope in such stars (Stello et al., 2013, their Fig. 4b).

3.2 Classifying the Unclassified Set

We now use our trained classifier to predict the evolutionary state of the unclassified set (Figure 7). It can be seen that the predictions reflect the $\Delta\nu-\epsilon$ relation of RGB stars (Kallinger et al., 2012) well for the entire $\Delta\nu$ range spanned by the training set ($2.8\mu$Hz $\apprle\Delta\nu\apprle 18\mu$Hz). The secondary clump of HeB stars is seen in the diagram with $\Delta\nu\simeq 6-9\mu$Hz below the RGB $\Delta\nu-\epsilon$ relation. In addition, the predictions also clearly show the HeB population at $\Delta\nu\simeq 3-4\mu$Hz, with $\epsilon$ values mostly ranging about 0.7 to 1.0. In Figures 6a and 7, stars with $\Delta\nu\simeq 4\mu$Hz and $\epsilon\apprge 1.2$ are low $\epsilon$ stars that have ‘wrapped around’ vertically in the diagram.

The classifier has mostly very confident predictions, shown by the vast majority of predictions in Figure 7 having deep shades of red and blue for RGB and HeB predictions, respectively. The exception to this is at the region of intersection between RGB and HeB populations at $2.8\mu$Hz $\leq\Delta\nu\apprle 4.0\mu$Hz and 0.9 $\apprle\epsilon\apprle$1.0. The paleness of the symbol colours in this region indicate a prediction uncertainty, which we attribute to the scarcity of stars in our training set with $\Delta\nu$ and $\epsilon$ values within this range.

Despite the predictions capturing the general distribution of red giant populations within the $\Delta\nu-\epsilon$ diagram , the classifier has its limitations from classifying based on image representation alone. For instance, it does not explicitly discriminate between frequency spacings, such that it can erronously predict HeB stars at high $\Delta\nu$ ($\Delta\nu\apprge 9\mu$Hz), where no HeB stars exist. However, only a very small fraction of predictions are subject to this inaccuracy. Another important limitation of these predictions is imposed by the parameter range of the training data. Our training data only includes RGB stars down to $\Delta\nu\simeq 2.8\mu$Hz, hence we infer that the reliability of the classifier predictions also holds to a similar $\Delta\nu$ threshold. Due to this, we do not provide classifications for 1139 red giants with $\Delta\nu<2.8\mu$Hz in our unclassified set, leaving behind 7655 red giants with $\Delta\nu\geq 2.8\mu$Hz from the initial 8794 in the unclassified set. Out of the remaining 7655 red giants, we find that 517 stars have been previously classified by Stello et al. (2013), from which 43 of their predictions do not agree with ours. We visually verify that they had incorrectly labelled approximately half of the disputed stars as HeB, with a majority of them having $\Delta\nu\simeq 5.5\mu$Hz. In addition, 1991 stars with $\Delta\nu\apprge 2.8\mu$Hz have been classified by Elsworth et al. (2016), from which 232 stars are overlapping with the stars classified by Stello et al. (2013). Our classifier produces disputing predictions for 195 out of the 1991 stars classified by Elsworth et al. (2016). These disputes are mostly for those stars that they have predicted as RGB in the range of $2.8\mu$Hz $\apprle\Delta\nu\apprle 5\mu$Hz, which we find are split in roughly equal numbers into those that our classifier predicts correctly, those that our classifier predicts incorrectly, and those where we are uncertain of the true population by visual inspection. In the end, we produce new classifications for 5379 stars not classified before by any other method. In future work, we will develop methods to improve the generalization of classifier predictions across a greater range of asteroseismic parameters. In addition, we will also look into spectral features of high $\Delta\nu$ RGB stars, which can deceive the classifier to incorrectly predict the star as HeB.

4 Conclusions

We have developed a convolutional neural network to perform fast and efficient classification of red giant stars into those ascending the red giant branch and into those that have commenced core helium burning. We presented folded oscillation spectra as images, which contain visual features that are learned by the convolutional neural network. Training and testing on Kepler data yielded a 98% cross-validation accuracy and a 99% test set accuracy, benchmarked against classifications based on asymptotic period spacing measurements. Out of the predictions that were in conflict with the ‘ground truth’, most scenarios of classifier inaccuracy were limited to the intermediate to high $\Delta\nu$ range, whereas for several low $\Delta\nu$ disputed cases, the input population labels were either incorrect or ambiguous based on visual inspection.

We also made predictions on 7655 Kepler red giants that do not have their asymptotic period spacing measured, from which 5379 have not been previously classified by any other means. We observed good agreement with the expected distribution of red giant populations in $\epsilon-\Delta\nu$ space for $\Delta\nu>2.8\mu$Hz. Despite being currently limited to predicting within the asteroseismic parameter ranges of the training set, this new, simple, and effective method of classifying oscillation spectra seems promising for further future classifications on large datasets in asteroseismology.

Acknowledgements

Funding for this Discovery mission is provided by NASA’s Science Mission Directorate. We thank the entire Kepler team without whom this investigation would not be possible. D.S. is the recipient of an Australian Research Council Future Fellowship (project number FT1400147). We would also like to thank Timothy Bedding, Daniel Huber, and the group at The University of Sydney for fruitful discussions.