FusionSort: Enhanced Cluttered Waste Segmentation with Advanced Decoding and Comprehensive Modality Optimization

TL;DR

This paper presents FusionSort, an advanced neural network architecture for waste sorting that integrates novel attention mechanisms and data fusion techniques to improve accuracy across multiple data modalities.

Contribution

FusionSort introduces a comprehensive attention decoder and a PCA-based data fusion method, enhancing waste segmentation performance across diverse imaging modalities.

Findings

Outperforms existing waste sorting methods significantly

Effective fusion of RGB and hyperspectral data improves accuracy

Attention mechanisms enhance feature representation

Abstract

In the realm of waste management, automating the sorting process for non-biodegradable materials presents considerable challenges due to the complexity and variability of waste streams. To address these challenges, we introduce an enhanced neural architecture that builds upon an existing Encoder-Decoder structure to improve the accuracy and efficiency of waste sorting systems. Our model integrates several key innovations: a Comprehensive Attention Block within the decoder, which refines feature representations by combining convolutional and upsampling operations. In parallel, we utilize attention through the Mamba architecture, providing an additional performance boost. We also introduce a Data Fusion Block that fuses images with more than three channels. To achieve this, we apply PCA transformation to reduce the dimensionality while retaining the maximum variance and essential information across three dimensions, which are then used for further processing. We evaluated the model on RGB, hyperspectral, multispectral, and a combination of RGB and hyperspectral data. The results demonstrate that our approach outperforms existing methods by a significant margin.