PMODE: Prototypical Mask based Object Dimension Estimation

TL;DR

This paper introduces a neural network-based method for estimating the dimensions of quadrilateral objects in videos using monocular cameras without calibration, leveraging instance segmentation and learned features.

Contribution

It presents a novel deep learning architecture that estimates object dimensions directly from segmentation masks without requiring geometric features or camera calibration.

Findings

Achieved 22% MAPE in dimension estimation on test data.

Utilized a real-time instance segmentation network with ResNet50 backbone.

Demonstrated effectiveness across three different camera setups.

Abstract

Can a neural network estimate an object's dimension in the wild? In this paper, we propose a method and deep learning architecture to estimate the dimensions of a quadrilateral object of interest in videos using a monocular camera. The proposed technique does not use camera calibration or handcrafted geometric features; however, features are learned with the help of coefficients of a segmentation neural network during the training process. A real-time instance segmentation-based Deep Neural Network with a ResNet50 backbone is employed, giving the object's prototype mask and thus provides a region of interest to regress its dimensions. The instance segmentation network is trained to look at only the nearest object of interest. The regression is performed using an MLP head which looks only at the mask coefficients of the bounding box detector head and the prototype segmentation mask. We trained the system with three different random cameras achieving 22% MAPE for the test dataset for the dimension estimation