A novel single-pulse search approach to detection of dispersed radio pulses using clustering and supervised machine learning

TL;DR

This paper introduces a two-stage machine learning approach combining clustering and supervised classification to automatically detect and classify dispersed radio pulses in large pulsar survey data, improving detection accuracy.

Contribution

The novel method integrates unsupervised clustering with supervised learning for pulse detection, including a new peak scoring algorithm and periodicity search, enhancing pulsar identification.

Findings

RandomForest achieved 95.6% recall and 2.0% false positive rate.

The approach identified all known pulsars in the dataset plus additional candidates.

The method successfully distinguished astrophysical pulses from noise in large survey data.

Abstract

We present a novel two-stage approach which combines unsupervised and supervised machine learning to automatically identify and classify single pulses in radio pulsar search data. In the first stage, we identify astrophysical pulse candidates in the data, which were derived from the Pulsar Arecibo L-Band Feed Array (PALFA) survey and contain 47,042 independent beams, as trial single-pulse event groups (SPEGs) by clustering single-pulse events and merging clusters that fall within the expected DM and time span of astrophysical pulses. We also present a new peak scoring algorithm, to identify astrophysical peaks in S/N versus DM curves. Furthermore, we group SPEGs detected at a consistent DM for they were likely emitted by the same source. In the second stage, we create a fully labelled benchmark data set by selecting a subset of data with SPEGs identified (using stage 1 procedures), their features extracted and individual SPEGs manually labelled, and then train classifiers using supervised machine learning. Next, using the best trained classifier, we automatically classify unlabelled SPEGs identified in the full data set. To aid the examination of dim SPEGs, we develop an algorithm that searches for an underlying periodicity among grouped SPEGs. The results showed that RandomForest with SMOTE treatment was the best learner, with a recall of 95.6% and a false positive rate of 2.0%. In total, besides all 60 known pulsars from the benchmark data set, the model found 32 additional (i.e., not included in the benchmark data set) known pulsars, and several potential discoveries.