Location-Sensitive Visual Recognition with Cross-IOU Loss

TL;DR

This paper introduces LSNet, a unified deep learning framework for location-sensitive visual recognition tasks like detection and segmentation, utilizing a novel cross-IOU loss to improve accuracy across scales.

Contribution

It proposes LSNet, a unified network predicting anchor points and landmarks, with a new cross-IOU loss for better scale fitting and contextual understanding.

Findings

Achieved state-of-the-art 53.5% box AP on MS-COCO

Set new 40.2% mask AP for instance segmentation

Demonstrated effective multi-scale human pose detection

Abstract

Object detection, instance segmentation, and pose estimation are popular visual recognition tasks which require localizing the object by internal or boundary landmarks. This paper summarizes these tasks as location-sensitive visual recognition and proposes a unified solution named location-sensitive network (LSNet). Based on a deep neural network as the backbone, LSNet predicts an anchor point and a set of landmarks which together define the shape of the target object. The key to optimizing the LSNet lies in the ability of fitting various scales, for which we design a novel loss function named cross-IOU loss that computes the cross-IOU of each anchor point-landmark pair to approximate the global IOU between the prediction and ground-truth. The flexibly located and accurately predicted landmarks also enable LSNet to incorporate richer contextual information for visual recognition. Evaluated on the MS-COCO dataset, LSNet set the new state-of-the-art accuracy for anchor-free object detection (a 53.5% box AP) and instance segmentation (a 40.2% mask AP), and shows promising performance in detecting multi-scale human poses. Code is available at https://github.com/Duankaiwen/LSNet