# Large-Scale Classification of Structured Objects using a CRF with Deep Class Embedding

## TL;DR

This paper introduces a deep learning approach combining CNNs and CRFs with class embeddings to classify large sets of visually similar structured objects, effectively modeling contextual relationships.

## Contribution

The novel architecture jointly learns visual features and class embeddings within a CRF framework, addressing challenges of large, sparse datasets with many similar categories.

## Key findings

- Significantly outperforms linear CRF models on large-scale datasets.
- Effectively models contextual relationships among classes.
- Handles data sparsity with a nonlinear training objective.

## Abstract

This paper presents a novel deep learning architecture to classify structured objects in datasets with a large number of visually similar categories. We model sequences of images as linear-chain CRFs, and jointly learn the parameters from both local-visual features and neighboring classes. The visual features are computed by convolutional layers, and the class embeddings are learned by factorizing the CRF pairwise potential matrix. This forms a highly nonlinear objective function which is trained by optimizing a local likelihood approximation with batch-normalization. This model overcomes the difficulties of existing CRF methods to learn the contextual relationships thoroughly when there is a large number of classes and the data is sparse. The performance of the proposed method is illustrated on a huge dataset that contains images of retail-store product displays, taken in varying settings and viewpoints, and shows significantly improved results compared to linear CRF modeling and unnormalized likelihood optimization.