Peer groups for organisational learning: clustering with practical constraints

TL;DR

This paper introduces a methodology for constrained clustering to create peer groups that meet business constraints, ensuring stability over time and improving interpretability for non-statistical audiences in organizational learning contexts.

Contribution

The paper develops a novel approach to incorporate business constraints into clustering, balancing statistical fit and practical requirements, with tools for interpretability and stability analysis.

Findings

Constrained clustering achieved stable peer groups over two years.

Method maintained a negative relationship between fit and stability.

Visualizations improved understanding for non-statistical stakeholders.

Abstract

Peer-grouping is used in many sectors for organisational learning, policy implementation, and benchmarking. Clustering provides a statistical, data-driven method for constructing meaningful peer groups, but peer groups must be compatible with business constraints such as size and stability considerations. Additionally, statistical peer groups are constructed from many different variables, and can be difficult to understand, especially for non-statistical audiences. We developed methodology to apply business constraints to clustering solutions and allow the decision-maker to choose the balance between statistical goodness-of-fit and conformity to business constraints. Several tools were utilised to identify complex distinguishing features in peer groups, and a number of visualisations are developed to explain high-dimensional clusters for non-statistical audiences. In a case study where peer group size was required to be small ($\leq 100$ members), we applied constrained clustering to a noisy high-dimensional data-set over two subsequent years, ensuring that the clusters were sufficiently stable between years. Our approach not only satisfied clustering constraints on the test data, but maintained an almost monotonic negative relationship between goodness-of-fit and stability between subsequent years. We demonstrated in the context of the case study how distinguishing features between clusters can be communicated clearly to different stakeholders with substantial and limited statistical knowledge.