Making the Flow Glow -- Robot Perception under Severe Lighting Conditions using Normalizing Flow Gradients

TL;DR

This paper introduces a novel approach using normalizing flow gradients to improve robotic perception in severe lighting conditions, enabling local adaptation and significantly increasing object detection success rates.

Contribution

It proposes using absolute gradient values from normalizing flows for local perception optimization under challenging lighting, advancing out-of-distribution detection methods.

Findings

Achieved 60% higher success rate in object detection under difficult lighting.

Demonstrated effectiveness of local gradient-based adaptation in robotic perception.

Improved reliability of perception systems in adverse visual conditions.

Abstract

Modern robotic perception is highly dependent on neural networks. It is well known that neural network-based perception can be unreliable in real-world deployment, especially in difficult imaging conditions. Out-of-distribution detection is commonly proposed as a solution for ensuring reliability in real-world deployment. Previous work has shown that normalizing flow models can be used for out-of-distribution detection to improve reliability of robotic perception tasks. Specifically, camera parameters can be optimized with respect to the likelihood output from a normalizing flow, which allows a perception system to adapt to difficult vision scenarios. With this work we propose to use the absolute gradient values from a normalizing flow, which allows the perception system to optimize local regions rather than the whole image. By setting up a table top picking experiment with exceptionally difficult lighting conditions, we show that our method achieves a 60% higher success rate for an object detection task compared to previous methods.