Data challenges of time domain astronomy

TL;DR

The paper discusses the data challenges faced by time domain astronomy due to increasing data volumes and complexity, highlighting informatics-based solutions and experiences with specific sky surveys.

Contribution

It introduces an informatics-based approach to manage heterogeneous data in time domain astronomy and shares practical insights from recent sky surveys.

Findings

Addressed data heterogeneity in transient event detection

Highlighted the need for advanced data mining algorithms

Shared experiences from Palomar-Quest and Catalina surveys

Abstract

Astronomy has been at the forefront of the development of the techniques and methodologies of data intensive science for over a decade with large sky surveys and distributed efforts such as the Virtual Observatory. However, it faces a new data deluge with the next generation of synoptic sky surveys which are opening up the time domain for discovery and exploration. This brings both new scientific opportunities and fresh challenges, in terms of data rates from robotic telescopes and exponential complexity in linked data, but also for data mining algorithms used in classification and decision making. In this paper, we describe how an informatics-based approach-part of the so-called "fourth paradigm" of scientific discovery-is emerging to deal with these. We review our experiences with the Palomar-Quest and Catalina Real-Time Transient Sky Surveys; in particular, addressing the issue of the heterogeneity of data associated with transient astronomical events (and other sensor networks) and how to manage and analyze it.