# Looking Beyond the Obvious: A Survey on Abstract Concept Recognition for Video Understanding

## TL;DR

This survey explores the challenge of recognizing abstract concepts in videos, emphasizing the potential of foundation models to enhance high-level semantic understanding aligned with human reasoning.

## Contribution

It reviews tasks and datasets for abstract concept recognition in videos and advocates leveraging foundation models and community experience to address this open challenge.

## Key findings

- Researchers have periodically attempted to solve abstract concept recognition in videos.
- Foundation models offer promising avenues for high-level semantic understanding.
- Drawing on decades of research can prevent reinventing solutions in this domain.

## Abstract

The automatic understanding of video content is advancing rapidly. Empowered by deeper neural networks and large datasets, machines are increasingly capable of understanding what is concretely visible in video frames, whether it be objects, actions, events, or scenes. In comparison, humans retain a unique ability to also look beyond concrete entities and recognize abstract concepts like justice, freedom, and togetherness. Abstract concept recognition forms a crucial open challenge in video understanding, where reasoning on multiple semantic levels based on contextual information is key. In this paper, we argue that the recent advances in foundation models make for an ideal setting to address abstract concept understanding in videos. Automated understanding of high-level abstract concepts is imperative as it enables models to be more aligned with human reasoning and values. In this survey, we study different tasks and datasets used to understand abstract concepts in video content. We observe that, periodically and over a long period, researchers have attempted to solve these tasks, making the best use of the tools available at their disposal. We advocate that drawing on decades of community experience will help us shed light on this important open grand challenge and avoid ``re-inventing the wheel'' as we start revisiting it in the era of multi-modal foundation models.