CVChess: A Deep Learning Framework for Converting Chessboard Images to Forsyth-Edwards Notation

TL;DR

CVChess is a deep learning system that converts images of physical chessboards into FEN notation, enabling integration with online chess engines for improved analysis of physical games.

Contribution

The paper introduces a novel CNN-based framework with residual connections for accurate recognition of chess pieces from smartphone images and conversion to FEN notation.

Findings

Achieved high accuracy in piece recognition across diverse conditions

Successfully converted physical board images into FEN for engine analysis

Demonstrated robustness of the system with real-world images

Abstract

Chess has experienced a large increase in viewership since the pandemic, driven largely by the accessibility of online learning platforms. However, no equivalent assistance exists for physical chess games, creating a divide between analog and digital chess experiences. This paper presents CVChess, a deep learning framework for converting chessboard images to Forsyth-Edwards Notation (FEN), which is later input into online chess engines to provide you with the best next move. Our approach employs a convolutional neural network (CNN) with residual layers to perform piece recognition from smartphone camera images. The system processes RGB images of a physical chess board through a multistep process: image preprocessing using the Hough Line Transform for edge detection, projective transform to achieve a top-down board alignment, segmentation into 64 individual squares, and piece classification into 13 classes (6 unique white pieces, 6 unique black pieces and an empty square) using the residual CNN. Residual connections help retain low-level visual features while enabling deeper feature extraction, improving accuracy and stability during training. We train and evaluate our model using the Chess Recognition Dataset (ChessReD), containing 10,800 annotated smartphone images captured under diverse lighting conditions and angles. The resulting classifications are encoded as an FEN string, which can be fed into a chess engine to generate the most optimal move