# Real-Time Callus Instance Segmentation in Plant Tissue Culture Using Successive Generations of YOLO Architectures

## TL;DR

This paper introduces a new lentil callus dataset and shows that newer YOLO models can segment callus structures in plant tissue culture more efficiently and accurately.

## Contribution

The first curated lentil callus dataset for instance segmentation and evaluation of successive YOLO generations for plant tissue analysis.

## Key findings

- Anchor-free YOLOv8 and YOLOv11 outperformed anchor-based models in callus segmentation precision and efficiency.
- YOLOv8 achieved the highest mAP50@0.855 with real-time inference at 166 FPS.
- The dataset includes 122 high-resolution images with 1185 annotations across three callus stages.

## Abstract

Callus induction is a complex procedure in plant organ, cell, and tissue culture that underpins processes such as metabolite production, regeneration, and genetic transformation. It is important to monitor callus formation alongside subjective evaluations, which require labor-intensive care. In this research, the first curated lentil (Lens culinaris) callus dataset for instance segmentation was experimentally generated using three genotypes as one data set: Firat-87, Cagil, and Tigris. Leaf explants were cultured on MS medium fortified with different concentrations of gross regulators of BA and NAA to induce callus formation. Three biologically relevant stages, the leaf stage, the green callus, and the necrosis callus, were produced. During this process, 122 high-resolution images were obtained, resulting in 1185 total annotations across them. The dataset was evaluated across four successive generations (v5/7/8/11) of YOLO deep learning models under identical conditions using mAP, Dice coefficient, Precision, Recall, and IoU, together with efficiency metrics including parameter counts, FLOPs, and inference speed. The results show that anchor-based variants (YOLOv5/7) relied on predefined priors and showed limited boundary precision, whereas anchor-free designs (YOLOv8/11) used decoupled heads and direct center/boundary regression that provided clear advantages for callus structures. YOLOv8 reached the highest instance segmentation precision with mAP50@0.855, while it matched the accuracy with greater efficiency and achieved real-time inference with 166 FPS.