Transformers as Strong Lens Detectors- From Simulation to Surveys

TL;DR

This paper explores the use of self-attention-based machine learning models for detecting strong gravitational lenses in large astronomical surveys, addressing challenges like false positives and proposing solutions such as transfer learning.

Contribution

It introduces an improved self-attention model for lens detection, analyzes its limitations in real data, and suggests transfer learning to enhance performance.

Findings

Self-attention models outperform CNNs on simulated data.

Application to real survey data yields many false positives.

Transfer learning can mitigate false positives and improve detection accuracy.

Abstract

With the upcoming large-scale surveys like LSST, we expect to find approximately $10^5$ strong gravitational lenses among data of many orders of magnitude larger. In this scenario, the usage of non-automated techniques is too time-consuming and hence impractical for science. For this reason, machine learning techniques started becoming an alternative to previous methods. In our previous work, we proposed a new machine learning architecture based on the principle of self-attention, trained to find strong gravitational lenses on simulated data from the Bologna Lens Challenge. Self-attention-based models have clear advantages compared to simpler CNNs and highly competing performance in comparison to the current state-of-art CNN models. We apply the proposed model to the Kilo Degree Survey, identifying some new strong lens candidates. However, these have been identified among a plethora of false positives, which made the application of this model not so advantageous. Therefore, throughout this paper, we investigate the pitfalls of this approach, and possible solutions, such as transfer learning, are proposed.