# Seamless Scene Segmentation

## TL;DR

This paper presents a novel CNN architecture for end-to-end seamless scene segmentation that integrates multi-scale features and contextual information, achieving state-of-the-art results on multiple street-level datasets.

## Contribution

The work introduces a new CNN-based architecture with a specialized segmentation head and an improved panoptic metric for better evaluation of scene segmentation.

## Key findings

- Achieves state-of-the-art results on Cityscapes, Indian Driving Dataset, and Mapillary Vistas.
- Proposes an alternative panoptic metric for non-instance categories.
- Demonstrates seamless integration of multi-scale features and context in scene segmentation.

## Abstract

In this work we introduce a novel, CNN-based architecture that can be trained end-to-end to deliver seamless scene segmentation results. Our goal is to predict consistent semantic segmentation and detection results by means of a panoptic output format, going beyond the simple combination of independently trained segmentation and detection models. The proposed architecture takes advantage of a novel segmentation head that seamlessly integrates multi-scale features generated by a Feature Pyramid Network with contextual information conveyed by a light-weight DeepLab-like module. As additional contribution we review the panoptic metric and propose an alternative that overcomes its limitations when evaluating non-instance categories. Our proposed network architecture yields state-of-the-art results on three challenging street-level datasets, i.e. Cityscapes, Indian Driving Dataset and Mapillary Vistas.