# Urban Scene Segmentation with Laser-Constrained CRFs

## TL;DR

This paper introduces a novel CRF inference method for urban scene segmentation that integrates global constraints from multiple sensor modalities, improving segmentation accuracy in complex environments.

## Contribution

A new CRF inference approach formulated as a relaxed quadratic program that efficiently incorporates global constraints for multi-modal scene segmentation.

## Key findings

- Outperforms belief propagation and traditional CRF methods
- Effectively combines image and 3D point cloud data
- Enhances scene segmentation accuracy in urban environments

## Abstract

Robots typically possess sensors of different modalities, such as colour cameras, inertial measurement units, and 3D laser scanners. Often, solving a particular problem becomes easier when more than one modality is used. However, while there are undeniable benefits to combine sensors of different modalities the process tends to be complicated. Segmenting scenes observed by the robot into a discrete set of classes is a central requirement for autonomy as understanding the scene is the first step to reason about future situations. Scene segmentation is commonly performed using either image data or 3D point cloud data. In computer vision many successful methods for scene segmentation are based on conditional random fields (CRF) where the maximum a posteriori (MAP) solution to the segmentation can be obtained by inference. In this paper we devise a new CRF inference method for scene segmentation that incorporates global constraints, enforcing the sets of nodes are assigned the same class label. To do this efficiently, the CRF is formulated as a relaxed quadratic program whose MAP solution is found using a gradient-based optimisation approach. The proposed method is evaluated on images and 3D point cloud data gathered in urban environments where image data provides the appearance features needed by the CRF, while the 3D point cloud data provides global spatial constraints over sets of nodes. Comparisons with belief propagation, conventional quadratic programming relaxation, and higher order potential CRF show the benefits of the proposed method.