UniDGF: A Unified Detection-to-Generation Framework for Hierarchical Object Visual Recognition

TL;DR

UniDGF introduces a detection-guided generative framework that enhances hierarchical object recognition by predicting semantic tokens, effectively capturing fine-grained categories and attributes in large-scale e-commerce scenarios.

Contribution

The paper proposes a novel unified detection-to-generation framework that improves fine-grained hierarchical recognition by integrating detection with generative semantic token prediction.

Findings

Outperforms existing similarity-based methods on large-scale datasets

Achieves stronger fine-grained recognition accuracy

Provides more coherent unified inference results

Abstract

Achieving visual semantic understanding requires a unified framework that simultaneously handles object detection, category prediction, and attribute recognition. However, current advanced approaches rely on global similarity and struggle to capture fine-grained category distinctions and category-specific attribute diversity, especially in large-scale e-commerce scenarios. To overcome these challenges, we introduce a detection-guided generative framework that predicts hierarchical category and attribute tokens. For each detected object, we extract refined ROI-level features and employ a BART-based generator to produce semantic tokens in a coarse-to-fine sequence covering category hierarchies and property-value pairs, with support for property-conditioned attribute recognition. Experiments on both large-scale proprietary e-commerce datasets and open-source datasets demonstrate that our approach significantly outperforms existing similarity-based pipelines and multi-stage classification systems, achieving stronger fine-grained recognition and more coherent unified inference.