# Loss Max-Pooling for Semantic Image Segmentation

## TL;DR

This paper proposes a novel loss max-pooling method that adaptively re-weights pixel contributions in deep neural networks to better handle class imbalance in semantic image segmentation, improving performance on benchmark datasets.

## Contribution

The paper introduces an adaptive loss re-weighting technique for semantic segmentation that addresses class imbalance more effectively than traditional methods.

## Key findings

- Consistent improvement on Cityscapes and Pascal VOC 2012 datasets.
- Theoretically justified approach enhances segmentation accuracy.
- Addresses both inter- and intra-class imbalance issues.

## Abstract

We introduce a novel loss max-pooling concept for handling imbalanced training data distributions, applicable as alternative loss layer in the context of deep neural networks for semantic image segmentation. Most real-world semantic segmentation datasets exhibit long tail distributions with few object categories comprising the majority of data and consequently biasing the classifiers towards them. Our method adaptively re-weights the contributions of each pixel based on their observed losses, targeting under-performing classification results as often encountered for under-represented object classes. Our approach goes beyond conventional cost-sensitive learning attempts through adaptive considerations that allow us to indirectly address both, inter- and intra-class imbalances. We provide a theoretical justification of our approach, complementary to experimental analyses on benchmark datasets. In our experiments on the Cityscapes and Pascal VOC 2012 segmentation datasets we find consistently improved results, demonstrating the efficacy of our approach.