Video Description using Bidirectional Recurrent Neural Networks

TL;DR

This paper enhances video description generation by integrating richer image features and bidirectional RNNs to better capture temporal context, leading to more accurate descriptions.

Contribution

It introduces bidirectional RNNs and combined object-location features into the encoding stage for improved video description quality.

Findings

Enhanced video descriptions with bidirectional RNNs

Richer image representations improve accuracy

Outperforms previous state-of-the-art models

Abstract

Although traditionally used in the machine translation field, the encoder-decoder framework has been recently applied for the generation of video and image descriptions. The combination of Convolutional and Recurrent Neural Networks in these models has proven to outperform the previous state of the art, obtaining more accurate video descriptions. In this work we propose pushing further this model by introducing two contributions into the encoding stage. First, producing richer image representations by combining object and location information from Convolutional Neural Networks and second, introducing Bidirectional Recurrent Neural Networks for capturing both forward and backward temporal relationships in the input frames.