Transformer based Multi-task Fusion Network for Food Spoilage Detection and Shelf life Forecasting

TL;DR

This paper introduces a multi-task fusion neural network combining CNN, LSTM, and DeiT transformer architectures to improve food spoilage detection, shelf life forecasting, and vegetable classification, demonstrating superior performance over existing models.

Contribution

It proposes a novel fusion architecture integrating CNN, LSTM, and DeiT transformer for simultaneous food spoilage detection, shelf life prediction, and classification, validated on a new dataset.

Findings

Fusion models outperform traditional deep learning models.

CNN+DeiT Transformer achieves high accuracy in classification and spoilage detection.

Model robustness is validated on noisy images with visualization support.

Abstract

Food wastage is one of the critical challenges in the agricultural supply chain, and accurate and effective spoilage detection can help to reduce it. Further, it is highly important to forecast the spoilage information. This aids the longevity of the supply chain management in the agriculture field. This motivated us to propose fusion based architectures by combining CNN with LSTM and DeiT transformer for the following multi-tasks simultaneously: (i) vegetable classification, (ii) food spoilage detection, and (iii) shelf life forecasting. We developed a dataset by capturing images of vegetables from their fresh state until they were completely spoiled. From the experimental analysis it is concluded that the proposed fusion architectures CNN+CNN-LSTM and CNN+DeiT Transformer outperformed several deep learning models such as CNN, VGG16, ResNet50, Capsule Networks, and DeiT Transformers. Overall, CNN + DeiT Transformer yielded F1-score of 0.98 and 0.61 in vegetable classification and spoilage detection respectively and mean squared error (MSE) and symmetric mean absolute percentage error (SMAPE) of 3.58, and 41.66% respectively in spoilage forecasting. Further, the reliability of the fusion models was validated on noisy images and integrated with LIME to visualize the model decisions.