Fruit Deformity Classification through Single-Input and Multi-Input Architectures based on CNN Models using Real and Synthetic Images

TL;DR

This study develops CNN-based single-input and multi-input architectures utilizing real and synthetic images to accurately classify fruit deformities, with the multi-input MobileNetV2 model achieving the highest accuracy across multiple fruit types.

Contribution

It introduces a novel multi-input CNN architecture combining RGB and silhouette images, demonstrating improved deformity classification accuracy over traditional single-input models.

Findings

Multi-input MobileNetV2 achieved up to 94% accuracy for mangoes.

Synthetic images improved CNN model performance.

Multi-input architecture outperformed single-input models.

Abstract

The present study focuses on detecting the degree of deformity in fruits such as apples, mangoes, and strawberries during the process of inspecting their external quality, employing Single-Input and Multi-Input architectures based on convolutional neural network (CNN) models using sets of real and synthetic images. The datasets are segmented using the Segment Anything Model (SAM), which provides the silhouette of the fruits. Regarding the single-input architecture, the evaluation of the CNN models is performed only with real images, but a methodology is proposed to improve these results using a pre-trained model with synthetic images. In the Multi-Input architecture, branches with RGB images and fruit silhouettes are implemented as inputs for evaluating CNN models such as VGG16, MobileNetV2, and CIDIS. However, the results revealed that the Multi-Input architecture with the MobileNetV2 model was the most effective in identifying deformities in the fruits, achieving accuracies of 90\%, 94\%, and 92\% for apples, mangoes, and strawberries, respectively. In conclusion, the Multi-Input architecture with the MobileNetV2 model is the most accurate for classifying levels of deformity in fruits.