Video Scene Location Recognition with Neural Networks

TL;DR

This paper explores using neural networks for recognizing scene locations in videos from TV series, focusing on frame selection, pre-processing, and various neural network architectures to improve classification accuracy.

Contribution

It introduces a method combining frame selection and neural network architectures, including LSTM variants, for scene location recognition in video sequences from TV series.

Findings

Certain neural network layers outperform others for scene classification.

LSTM-based models show promise in capturing temporal information.

The approach is tested on data from The Big Bang Theory series.

Abstract

This paper provides an insight into the possibility of scene recognition from a video sequence with a small set of repeated shooting locations (such as in television series) using artificial neural networks. The basic idea of the presented approach is to select a set of frames from each scene, transform them by a pre-trained singleimage pre-processing convolutional network, and classify the scene location with subsequent layers of the neural network. The considered networks have been tested and compared on a dataset obtained from The Big Bang Theory television series. We have investigated different neural network layers to combine individual frames, particularly AveragePooling, MaxPooling, Product, Flatten, LSTM, and Bidirectional LSTM layers. We have observed that only some of the approaches are suitable for the task at hand.