# PIRC Net : Using Proposal Indexing, Relationships and Context for Phrase Grounding

## TL;DR

This paper introduces PIRC Net, a novel framework for phrase grounding that uses proposal indexing, relationships, and context to improve localization accuracy, demonstrating superior results on benchmark datasets.

## Contribution

It presents a new multi-module approach incorporating phrase relationships and context, along with weakly-supervised learning via knowledge transfer, advancing phrase grounding techniques.

## Key findings

- Achieves state-of-the-art results on Flickr 30k Entities
- Improves weakly-supervised phrase grounding performance
- Effectively leverages phrase relationships and context

## Abstract

Phrase Grounding aims to detect and localize objects in images that are referred to and are queried by natural language phrases. Phrase grounding finds applications in tasks such as Visual Dialog, Visual Search and Image-text co-reference resolution. In this paper, we present a framework that leverages information such as phrase category, relationships among neighboring phrases in a sentence and context to improve the performance of phrase grounding systems. We propose three modules: Proposal Indexing Network(PIN); Inter-phrase Regression Network(IRN) and Proposal Ranking Network(PRN) each of which analyze the region proposals of an image at increasing levels of detail by incorporating the above information. Also, in the absence of ground-truth spatial locations of the phrases(weakly-supervised), we propose knowledge transfer mechanisms that leverages the framework of PIN module. We demonstrate the effectiveness of our approach on the Flickr 30k Entities and ReferItGame datasets, for which we achieve improvements over state-of-the-art approaches in both supervised and weakly-supervised variants.