Learning Surrogates via Deep Embedding

TL;DR

This paper introduces a method for training neural networks by learning a surrogate loss through deep embedding, enabling optimization of non-differentiable metrics with minimal computational overhead, demonstrated on scene-text recognition and detection tasks.

Contribution

The paper presents a novel deep embedding approach to learn surrogates for non-differentiable evaluation metrics, improving model tuning without significant computational costs.

Findings

Up to 39% improvement in edit distance for recognition.

Up to 4.25% increase in F1 score for detection.

Effective surrogate learning demonstrated on practical tasks.

Abstract

This paper proposes a technique for training a neural network by minimizing a surrogate loss that approximates the target evaluation metric, which may be non-differentiable. The surrogate is learned via a deep embedding where the Euclidean distance between the prediction and the ground truth corresponds to the value of the evaluation metric. The effectiveness of the proposed technique is demonstrated in a post-tuning setup, where a trained model is tuned using the learned surrogate. Without a significant computational overhead and any bells and whistles, improvements are demonstrated on challenging and practical tasks of scene-text recognition and detection. In the recognition task, the model is tuned using a surrogate approximating the edit distance metric and achieves up to $39\%$ relative improvement in the total edit distance. In the detection task, the surrogate approximates the intersection over union metric for rotated bounding boxes and yields up to $4.25\%$ relative improvement in the $F_{1}$ score.