# A dataset of gameplay videos with frame-wise board annotations, piece annotations, and state mappings for the Connect Four game

## TL;DR

This paper introduces a dataset of Connect Four gameplay videos with detailed annotations for board states and piece positions to aid in visual perception and state modeling tasks.

## Contribution

The paper presents a novel dataset with frame-wise annotations and vectorized board states from real-world Connect Four gameplay videos.

## Key findings

- The dataset includes 52 annotated games with board-level and piece-level annotations.
- An auxiliary dataset of one million unique board positions was generated computationally.
- The dataset supports tasks like board state extraction and visual-to-symbolic mapping.

## Abstract

This article describes a dataset derived from real-world video recordings of the physical board game Connect Four, focusing on visual perception and structured state representation. The dataset includes gameplay videos and frame-wise annotations obtained from human-versus-human matches played on a standard physical board. A total of 52 recorded games were selected for detailed annotation, from which board-level annotations, individual piece annotations, and image-based vector representations of game states were generated.

All visual and symbolic data components originate from annotated human gameplay recordings. An annotation program was developed to convert video frames into structured board matrices and corresponding vectorized representations. The dataset is organized into multiple components, including raw videos, annotated board regions, annotated piece locations, and vectorized board states.

In addition, an auxiliary dataset of one million unique playable board positions with associated move labels was generated computationally as a downstream resource. This auxiliary dataset is independent of the video data and is not required for the reuse of the computer vision components. The dataset can be reused for tasks involving board state extraction from images, visual-to-symbolic mapping, and downstream state–action modeling once visual states have been inferred.