Portuguese Man-of-War Image Classification with Convolutional Neural Networks

TL;DR

This study demonstrates that convolutional neural networks, especially pre-trained ResNet50, can effectively classify Portuguese Man-of-War images from Instagram, aiding in species monitoring through social media data.

Contribution

The paper introduces a new dataset and evaluates CNN architectures, showing that transfer learning significantly improves classification accuracy for PMW images.

Findings

Pre-trained ResNet50 achieved 94% accuracy.

CNNs effectively identify PMW images on social media.

Transfer learning enhances model performance.

Abstract

Portuguese man-of-war (PMW) is a gelatinous organism with long tentacles capable of causing severe burns, thus leading to negative impacts on human activities, such as tourism and fishing. There is a lack of information about the spatio-temporal dynamics of this species. Therefore, the use of alternative methods for collecting data can contribute to their monitoring. Given the widespread use of social networks and the eye-catching look of PMW, Instagram posts can be a promising data source for monitoring. The first task to follow this approach is to identify posts that refer to PMW. This paper reports on the use of convolutional neural networks for PMW images classification, in order to automate the recognition of Instagram posts. We created a suitable dataset, and trained three different neural networks: VGG-16, ResNet50, and InceptionV3, with and without a pre-trained step with the ImageNet dataset. We analyzed their results using accuracy, precision, recall, and F1 score metrics. The pre-trained ResNet50 network presented the best results, obtaining 94% of accuracy and 95% of precision, recall, and F1 score. These results show that convolutional neural networks can be very effective for recognizing PMW images from the Instagram social media.