Low-Cost Scene Modeling using a Density Function Improves Segmentation Performance

TL;DR

This paper introduces a cost-effective method combining simulation and CAD models with a novel density function to model human-object interactions, significantly enhancing segmentation accuracy in RGB-D data for industrial human-robot collaboration.

Contribution

It presents a new density function for scene modeling in virtual environments, enabling synthetic data generation that improves segmentation performance over existing methods.

Findings

Segmentation performance improved by ~7% in mean average precision and recall.

The approach is computationally efficient and suitable for real-time applications.

Synthetic data training enhances real-world segmentation accuracy.

Abstract

We propose a low cost and effective way to combine a free simulation software and free CAD models for modeling human-object interaction in order to improve human & object segmentation. It is intended for research scenarios related to safe human-robot collaboration (SHRC) and interaction (SHRI) in the industrial domain. The task of human and object modeling has been used for detecting activity, and for inferring and predicting actions, different from those works, we do human and object modeling in order to learn interactions in RGB-D data for improving segmentation. For this purpose, we define a novel density function to model a three dimensional (3D) scene in a virtual environment (VREP). This density function takes into account various possible configurations of human-object and object-object relationships and interactions governed by their affordances. Using this function, we synthesize a large, realistic and highly varied synthetic RGB-D dataset that we use for training. We train a random forest classifier, and the pixelwise predictions obtained is integrated as a unary term in a pairwise conditional random fields (CRF). Our evaluation shows that modeling these interactions improves segmentation performance by ~7\% in mean average precision and recall over state-of-the-art methods that ignore these interactions in real-world data. Our approach is computationally efficient, robust and can run real-time on consumer hardware.