Scene Labeling Through Knowledge-Based Rules Employing Constrained Integer Linear Programing

TL;DR

This paper introduces a knowledge-based rule system integrated with constrained integer linear programming to enhance scene labeling by capturing non-local dependencies among image regions, outperforming traditional local feature-based methods.

Contribution

It presents a novel approach that extracts rules from data and encodes them as constraints in an ILP framework, incorporating soft constraints for flexibility in scene labeling.

Findings

Achieved promising results on three datasets.

Improved classification scores by modeling non-local dependencies.

Demonstrated effectiveness of rule-based constraints in scene labeling.

Abstract

Scene labeling task is to segment the image into meaningful regions and categorize them into classes of objects which comprised the image. Commonly used methods typically find the local features for each segment and label them using classifiers. Afterward, labeling is smoothed in order to make sure that neighboring regions receive similar labels. However, they ignore expressive and non-local dependencies among regions due to expensive training and inference. In this paper, we propose to use high-level knowledge regarding rules in the inference to incorporate dependencies among regions in the image to improve scores of classification. Towards this aim, we extract these rules from data and transform them into constraints for Integer Programming to optimize the structured problem of assigning labels to super-pixels (consequently pixels) of an image. In addition, we propose to use soft-constraints in some scenarios, allowing violating the constraint by imposing a penalty, to make the model more flexible. We assessed our approach on three datasets and obtained promising results.