Explainable Deep Learning for Video Recognition Tasks: A Framework & Recommendations

TL;DR

This paper emphasizes the importance of developing explainability methods tailored for video deep learning models, highlighting current gaps and the rapid growth of video applications in AI.

Contribution

It provides an overview of state-of-the-art video deep learning techniques and underscores the need for specialized explainability approaches for these complex models.

Findings

Video deep learning models are more complex with at least twice the parameters of image models.

Current explainability methods are primarily adapted from image techniques and are not well-suited for video.

Research into explanations specifically designed for video models is scarce.

Abstract

The popularity of Deep Learning for real-world applications is ever-growing. With the introduction of high performance hardware, applications are no longer limited to image recognition. With the introduction of more complex problems comes more and more complex solutions, and the increasing need for explainable AI. Deep Neural Networks for Video tasks are amongst the most complex models, with at least twice the parameters of their Image counterparts. However, explanations for these models are often ill-adapted to the video domain. The current work in explainability for video models is still overshadowed by Image techniques, while Video Deep Learning itself is quickly gaining on methods for still images. This paper seeks to highlight the need for explainability methods designed with video deep learning models, and by association spatio-temporal input in mind, by first illustrating the cutting edge for video deep learning, and then noting the scarcity of research into explanations for these methods.