Vision Transformer Segmentation for Visual Bird Sound Denoising

TL;DR

This paper introduces ViTVS, a vision transformer-based model that significantly improves bird sound denoising by effectively separating clean audio from noise, outperforming existing methods in real-world scenarios.

Contribution

The paper presents ViTVS, a novel vision transformer architecture that incorporates segmentation for enhanced long-range and multi-scale audio representation in bird sound denoising.

Findings

ViTVS outperforms state-of-the-art denoising methods.

It effectively handles complex, low-frequency, and residual noise.

The approach sets a new benchmark for real-world bird sound denoising.

Abstract

Audio denoising, especially in the context of bird sounds, remains a challenging task due to persistent residual noise. Traditional and deep learning methods often struggle with artificial or low-frequency noise. In this work, we propose ViTVS, a novel approach that leverages the power of the vision transformer (ViT) architecture. ViTVS adeptly combines segmentation techniques to disentangle clean audio from complex signal mixtures. Our key contributions encompass the development of ViTVS, introducing comprehensive, long-range, and multi-scale representations. These contributions directly tackle the limitations inherent in conventional approaches. Extensive experiments demonstrate that ViTVS outperforms state-of-the-art methods, positioning it as a benchmark solution for real-world bird sound denoising applications. Source code is available at: https://github.com/aiai-4/ViVTS.