K-Means Panel Data Clustering in the Presence of Small Groups

TL;DR

This paper analyzes the asymptotic properties of K-means clustering in panel data with small groups, proposing modified criteria to accurately identify small groups and demonstrating their effectiveness through simulations and empirical application.

Contribution

It introduces modified information criteria that improve small group detection in panel data clustering, addressing limitations of existing methods.

Findings

Modified criteria outperform existing ones in small group detection

Longer sample periods are needed for consistent estimation with small groups

Empirical application successfully identifies small groups without over-clustering

Abstract

We consider panel data models with group structure. We study the asymptotic behavior of least-squares estimators and information criterion for the number of groups, allowing for the presence of small groups that have an asymptotically negligible relative size. Our contributions are threefold. First, we derive sufficient conditions under which the least-squares estimators are consistent and asymptotically normal. One of the conditions implies that a longer sample period is required as there are smaller groups. Second, we show that information criteria for the number of groups proposed in earlier works can be inconsistent or perform poorly in the presence of small groups. Third, we propose modified information criteria (MIC) designed to perform well in the presence of small groups. A Monte Carlo simulation confirms their good performance in finite samples. An empirical application illustrates that K-means clustering paired with the proposed MIC allows one to discover small groups without producing too many groups. This enables characterizing small groups and differentiating them from the other large groups in a parsimonious group structure.