Panoptic Lintention Network: Towards Efficient Navigational Perception for the Visually Impaired

TL;DR

This paper introduces Panoptic Lintention Net, an efficient panoptic segmentation model with a novel attention module, improving navigational perception for the visually impaired by providing comprehensive scene understanding.

Contribution

The paper proposes a new attention module called Lintention and integrates it into a panoptic segmentation model, enhancing efficiency and accuracy for real-world navigation aid applications.

Findings

Increases Panoptic Quality (PQ) from 39.39 to 41.42 on COCO dataset.

Reduces GFLOPs by 10% and parameters by 25% in the semantic branch.

Achieves stable and remarkable segmentation in real-world wearable system.

Abstract

Classic computer vision algorithms, instance segmentation, and semantic segmentation can not provide a holistic understanding of the surroundings for the visually impaired. In this paper, we utilize panoptic segmentation to assist the navigation of visually impaired people by offering both things and stuff awareness in the proximity of the visually impaired efficiently. To this end, we propose an efficient Attention module -- Lintention which can model long-range interactions in linear time using linear space. Based on Lintention, we then devise a novel panoptic segmentation model which we term Panoptic Lintention Net. Experiments on the COCO dataset indicate that the Panoptic Lintention Net raises the Panoptic Quality (PQ) from 39.39 to 41.42 with 4.6\% performance gain while only requiring 10\% fewer GFLOPs and 25\% fewer parameters in the semantic branch. Furthermore, a real-world test via our designed compact wearable panoptic segmentation system, indicates that our system based on the Panoptic Lintention Net accomplishes a relatively stable and exceptionally remarkable panoptic segmentation in real-world scenes.