DenoMAE2.0: Improving Denoising Masked Autoencoders by Classifying Local Patches

TL;DR

DenoMAE2.0 enhances denoising masked autoencoders by adding local patch classification, leading to better feature learning and robustness, especially in noisy, data-scarce wireless communication scenarios.

Contribution

It introduces a dual-objective approach combining reconstruction and local patch classification in masked autoencoders for improved semi-supervised learning.

Findings

Outperforms DenoMAE in denoising quality and classification accuracy.

Achieves 1.1% higher accuracy on the dataset.

Significantly improves modulation classification accuracy on RadioML benchmark.

Abstract

We introduce DenoMAE2.0, an enhanced denoising masked autoencoder that integrates a local patch classification objective alongside traditional reconstruction loss to improve representation learning and robustness. Unlike conventional Masked Autoencoders (MAE), which focus solely on reconstructing missing inputs, DenoMAE2.0 introduces position-aware classification of unmasked patches, enabling the model to capture fine-grained local features while maintaining global coherence. This dual-objective approach is particularly beneficial in semi-supervised learning for wireless communication, where high noise levels and data scarcity pose significant challenges. We conduct extensive experiments on modulation signal classification across a wide range of signal-to-noise ratios (SNRs), from extremely low to moderately high conditions and in a low data regime. Our results demonstrate that DenoMAE2.0 surpasses its predecessor, Deno-MAE, and other baselines in both denoising quality and downstream classification accuracy. DenoMAE2.0 achieves a 1.1% improvement over DenoMAE on our dataset and 11.83%, 16.55% significant improved accuracy gains on the RadioML benchmark, over DenoMAE, for constellation diagram classification of modulation signals.