# PoulTrans: a transformer-based model for accurate poultry condition assessment

## TL;DR

PoulTrans is a new AI model that accurately assesses poultry conditions using images and generates detailed reports to help farmers.

## Contribution

PoulTrans introduces a novel CSA_Encoder-Transformer framework with a Channel Spatial Memory-Guided Transformer and PS-Loss function for poultry condition assessment.

## Key findings

- PoulTrans achieved high performance scores on the PSC-Captions dataset, including BLEU-4 of 0.501 and ROUGE-L of 0.803.
- The model's CSA_Encoder and CSMT components improve attention and semantic precision in poultry status descriptions.
- Experiments confirmed the model's effectiveness and reliability for automated poultry condition reporting.

## Abstract

Recent advances in deep learning have significantly enhanced the accuracy of poultry image recognition, particularly in assessing poultry conditions. However, developing intuitive decision support tools remain a significant challenge. To address this, we present PoulTrans, an innovative image captioning framework that leverages a Convolutional Neural Network (CNN) integrated with a CSA_Encoder-Transformer architecture to generate detailed poultry status reports. This model incorporates visual features extracted by CNNs into the Channel Spatial Attention Segmentation Encoder (CSA_Encoder), which produces segmented channel and spatial attention outputs. To optimize multi-level attention and improve the semantic precision of the status descriptions, we introduced a Channel Spatial Memory-Guided Transformer (CSMT) and a novel PS-Loss function. The performance of PoulTrans was tested on the PSC-Captions dataset, achieving top scores of 0.501, 0.803, 4.927, 0.608, and 1.882 for the BLEU-4, ROUGE-L, CIDEr, SPICE, and Sm metrics, respectively. Comprehensive analyses and experiments have validated the effectiveness and reliability of our model, providing advanced tools for automated poultry status generation and enhancing the digital experience for poultry farmers. Our code is available at: https://github.com/kong1107800/PoulTrans.