Object Tracking and Geo-localization from Street Images

TL;DR

This paper introduces a two-stage system for detecting and geolocalizing traffic signs from street videos, combining a modified RetinaNet with a custom tracker to improve road asset mapping and autonomous driving applications.

Contribution

The paper presents GPS-RetinaNet, a modified object detector predicting positional offsets, and a novel tracking method using a learned metric network and Hungarian Algorithm, trained on an expanded ARTS dataset.

Findings

Effective detection and geolocalization of traffic signs from low frame rate videos.

Improved accuracy in associating signs across multiple images.

The ARTS dataset supports diverse environment scenarios for future research.

Abstract

Geo-localizing static objects from street images is challenging but also very important for road asset mapping and autonomous driving. In this paper we present a two-stage framework that detects and geolocalizes traffic signs from low frame rate street videos. Our proposed system uses a modified version of RetinaNet (GPS-RetinaNet), which predicts a positional offset for each sign relative to the camera, in addition to performing the standard classification and bounding box regression. Candidate sign detections from GPS-RetinaNet are condensed into geolocalized signs by our custom tracker, which consists of a learned metric network and a variant of the Hungarian Algorithm. Our metric network estimates the similarity between pairs of detections, then the Hungarian Algorithm matches detections across images using the similarity scores provided by the metric network. Our models were trained using an updated version of the ARTS dataset, which contains 25,544 images and 47.589 sign annotations ~\cite{arts}. The proposed dataset covers a diverse set of environments gathered from a broad selection of roads. Each annotaiton contains a sign class label, its geospatial location, an assembly label, a side of road indicator, and unique identifiers that aid in the evaluation. This dataset will support future progress in the field, and the proposed system demonstrates how to take advantage of some of the unique characteristics of a realistic geolocalization dataset.