Generating Synthetic Satellite Imagery With Deep-Learning Text-to-Image Models -- Technical Challenges and Implications for Monitoring and Verification

TL;DR

This paper explores the use of deep-learning models to generate synthetic satellite imagery, analyzing their realism, challenges, and potential applications in monitoring, verification, and addressing data scarcity in remote sensing.

Contribution

It introduces novel DL techniques for creating synthetic satellite images and evaluates their authenticity, addressing challenges and implications for monitoring and verification.

Findings

Synthetic satellite images can achieve high realism using DL models.

Synthetic data can help mitigate data scarcity in remote sensing.

Challenges include ensuring authenticity and avoiding misuse.

Abstract

Novel deep-learning (DL) architectures have reached a level where they can generate digital media, including photorealistic images, that are difficult to distinguish from real data. These technologies have already been used to generate training data for Machine Learning (ML) models, and large text-to-image models like DALL-E 2, Imagen, and Stable Diffusion are achieving remarkable results in realistic high-resolution image generation. Given these developments, issues of data authentication in monitoring and verification deserve a careful and systematic analysis: How realistic are synthetic images? How easily can they be generated? How useful are they for ML researchers, and what is their potential for Open Science? In this work, we use novel DL models to explore how synthetic satellite images can be created using conditioning mechanisms. We investigate the challenges of synthetic satellite image generation and evaluate the results based on authenticity and state-of-the-art metrics. Furthermore, we investigate how synthetic data can alleviate the lack of data in the context of ML methods for remote-sensing. Finally we discuss implications of synthetic satellite imagery in the context of monitoring and verification.