OASIC: Occlusion-Agnostic and Severity-Informed Classification

TL;DR

OASIC introduces a severity-informed classification approach that masks occluders and selects models based on estimated occlusion severity, significantly improving accuracy under severe occlusions.

Contribution

The paper presents a novel method combining occluder masking and severity estimation to enhance classification robustness against occlusions.

Findings

Masking occluders improves classification accuracy.

Estimating occlusion severity enables better model selection.

Combining masking with severity-informed model selection outperforms baseline methods.

Abstract

Severe occlusions of objects pose a major challenge for computer vision. We show that two root causes are (1) the loss of visible information and (2) the distracting patterns caused by the occluders. Our approach addresses both causes at the same time. First, the distracting patterns are removed at test-time, via masking of the occluding patterns. This masking is independent of the type of occlusion, by handling the occlusion through the lens of visual anomalies w.r.t. the object of interest. Second, to deal with less visual details, we follow standard practice by masking random parts of the object during training, for various degrees of occlusions. We discover that (a) it is possible to estimate the degree of the occlusion (i.e. severity) at test-time, and (b) that a model optimized for a specific degree of occlusion also performs best on a similar degree during test-time. Combining these two insights brings us to a severity-informed classification model called OASIC: Occlusion Agnostic Severity Informed Classification. We estimate the severity of occlusion for a test image, mask the occluder, and select the model that is optimized for the degree of occlusion. This strategy performs better than any single model optimized for any smaller or broader range of occlusion severities. Experiments show that combining gray masking with adaptive model selection improves $\text{AUC}_\text{occ}$ by +18.5 over standard training on occluded images and +23.7 over finetuning on unoccluded images.