Renewal Strings for Cleaning Astronomical Databases

TL;DR

This paper introduces renewal strings, a probabilistic method combining Hough transform, renewal processes, and hidden Markov models, to effectively identify and flag spurious records caused by artifacts in large astronomical sky survey databases.

Contribution

The paper presents a novel probabilistic technique, renewal strings, for detecting and removing artifacts in astronomical data, improving data quality for sky surveys.

Findings

Highly effective in identifying artifacts in SSS data

Provides confidence measures for spurious object detections

Applicable to future astronomical survey datasets

Abstract

Large astronomical databases obtained from sky surveys such as the SuperCOSMOS Sky Surveys (SSS) invariably suffer from a small number of spurious records coming from artefactual effects of the telescope, satellites and junk objects in orbit around earth and physical defects on the photographic plate or CCD. Though relatively small in number these spurious records present a significant problem in many situations where they can become a large proportion of the records potentially of interest to a given astronomer. In this paper we focus on the four most common causes of unwanted records in the SSS: satellite or aeroplane tracks, scratches fibres and other linear phenomena introduced to the plate, circular halos around bright stars due to internal reflections within the telescope and diffraction spikes near to bright stars. Accurate and robust techniques are needed for locating and flagging such spurious objects. We have developed renewal strings, a probabilistic technique combining the Hough transform, renewal processes and hidden Markov models which have proven highly effective in this context. The methods are applied to the SSS data to develop a dataset of spurious object detections, along with confidence measures, which can allow this unwanted data to be removed from consideration. These methods are general and can be adapted to any future astronomical survey data.