Expert-Guided Subgroup Discovery: Methodology and Application

TL;DR

This paper introduces an expert-guided subgroup discovery methodology utilizing heuristic beam search, statistical significance testing, and visualization, demonstrated on a medical early detection problem.

Contribution

It proposes a novel subgroup discovery approach combining heuristic search, statistical analysis, and visualization, tailored for expert guidance and applied to medical risk group detection.

Findings

Effective subgroup descriptions generated for medical risk detection

Statistically significant properties enhance subgroup interpretability

Visualization aids in understanding subgroup distributions

Abstract

This paper presents an approach to expert-guided subgroup discovery. The main step of the subgroup discovery process, the induction of subgroup descriptions, is performed by a heuristic beam search algorithm, using a novel parametrized definition of rule quality which is analyzed in detail. The other important steps of the proposed subgroup discovery process are the detection of statistically significant properties of selected subgroups and subgroup visualization: statistically significant properties are used to enrich the descriptions of induced subgroups, while the visualization shows subgroup properties in the form of distributions of the numbers of examples in the subgroups. The approach is illustrated by the results obtained for a medical problem of early detection of patient risk groups.