Heuristic Style Transfer for Real-Time, Efficient Weather Attribute Detection

TL;DR

This paper introduces lightweight, style-inspired neural architectures for real-time weather condition detection from images, achieving high accuracy and generalization with efficient models suitable for embedded systems.

Contribution

It proposes novel style-based architectures, automates Gram-matrix computation, and releases a large annotated weather dataset for improved weather attribute detection.

Findings

Models achieve over 96% F1 on internal test set.

Zero-shot evaluation yields over 78% F1 on external datasets.

PMG architecture runs in real time with fewer than 5 million parameters.

Abstract

We present lightweight and efficient architectures to detect weather conditions from RGB images, predicting the weather type (sunny, rain, snow, fog) and 11 complementary attributes such as intensity, visibility, and ground condition, for a total of 53 classes across the tasks. This work examines to what extent weather conditions manifest as variations in visual style. We investigate style-inspired techniques, including Gram matrices, a truncated ResNet-50 targeting lower and intermediate layers, and PatchGAN-style architectures, within a multi-task framework with attention mechanisms. Two families are introduced: RTM (ResNet50-Truncated-MultiTasks) and PMG (PatchGAN-MultiTasks-Gram), together with their variants. Our contributions include automation of Gram-matrix computation, integration of PatchGAN into supervised multi-task learning, and local style capture through local Gram for improved spatial coherence. We also release a dataset of 503,875 images annotated with 12 weather attributes under a Creative Commons Attribution (CC-BY) license. The models achieve F1 scores above 96 percent on our internal test set and above 78 percent in zero-shot evaluation on several external datasets, confirming their generalization ability. The PMG architecture, with fewer than 5 million parameters, runs in real time with a small memory footprint, making it suitable for embedded systems. The modular design of the models also allows style-related or weather-related tasks to be added or removed as needed.