Guided Generative Models using Weak Supervision for Detecting Object Spatial Arrangement in Overhead Images

TL;DR

This paper introduces TGGM, a weakly supervised generative model based on VAE and GMM, that efficiently estimates spatial arrangements of objects in overhead images with minimal manual annotations.

Contribution

The paper proposes a novel TGGM model that updates GMM components individually within the VAE framework, reducing annotation needs and capturing semantic spatial relationships.

Findings

Achieves comparable results to semi-supervised methods.

Outperforms unsupervised methods by 10% in F1 score.

Requires significantly fewer labeled data.

Abstract

The increasing availability and accessibility of numerous overhead images allows us to estimate and assess the spatial arrangement of groups of geospatial target objects, which can benefit many applications, such as traffic monitoring and agricultural monitoring. Spatial arrangement estimation is the process of identifying the areas which contain the desired objects in overhead images. Traditional supervised object detection approaches can estimate accurate spatial arrangement but require large amounts of bounding box annotations. Recent semi-supervised clustering approaches can reduce manual labeling but still require annotations for all object categories in the image. This paper presents the target-guided generative model (TGGM), under the Variational Auto-encoder (VAE) framework, which uses Gaussian Mixture Models (GMM) to estimate the distributions of both hidden and decoder variables in VAE. Modeling both hidden and decoder variables by GMM reduces the required manual annotations significantly for spatial arrangement estimation. Unlike existing approaches that the training process can only update the GMM as a whole in the optimization iterations (e.g., a "minibatch"), TGGM allows the update of individual GMM components separately in the same optimization iteration. Optimizing GMM components separately allows TGGM to exploit the semantic relationships in spatial data and requires only a few labels to initiate and guide the generative process. Our experiments shows that TGGM achieves results comparable to the state-of-the-art semi-supervised methods and outperforms unsupervised methods by 10% based on the $F_{1}$ scores, while requiring significantly fewer labeled data.