AS400-DET: Detection using Deep Learning Model for IBM i (AS/400)

TL;DR

This paper introduces AS400-DET, a deep learning-based system for automatic detection of GUI components on IBM i screens, supported by a new annotated dataset, enabling automated testing of legacy systems.

Contribution

The paper presents a novel dataset of IBM i GUI screenshots and a deep learning detection system tailored for legacy system testing automation.

Findings

Deep learning models effectively detect GUI components.

The dataset improves detection accuracy for IBM i screens.

Automated testing potential for legacy systems is demonstrated.

Abstract

This paper proposes a method for automatic GUI component detection for the IBM i system (formerly and still more commonly known as AS/400). We introduce a human-annotated dataset consisting of 1,050 system screen images, in which 381 images are screenshots of IBM i system screens in Japanese. Each image contains multiple components, including text labels, text boxes, options, tables, instructions, keyboards, and command lines. We then develop a detection system based on state-of-the-art deep learning models and evaluate different approaches using our dataset. The experimental results demonstrate the effectiveness of our dataset in constructing a system for component detection from GUI screens. By automatically detecting GUI components from the screen, AS400-DET has the potential to perform automated testing on systems that operate via GUI screens.