Archetypal Analysis for Binary Data

TL;DR

This paper introduces two new optimization frameworks for archetypal analysis tailored to binary data, improving upon existing methods by leveraging Bernoulli likelihood models for better pattern identification.

Contribution

It develops novel Bernoulli likelihood-based optimization methods for binary archetypal analysis, enhancing accuracy and efficiency over previous approaches.

Findings

Proposed methods outperform existing binary AA algorithms on synthetic data.

The frameworks are computationally efficient with closed-form updates.

Extensions to other data distributions are feasible.

Abstract

Archetypal analysis (AA) is a matrix decomposition method that identifies distinct patterns using convex combinations of the data points denoted archetypes with each data point in turn reconstructed as convex combinations of the archetypes. AA thereby forms a polytope representing trade-offs of the distinct aspects in the data. Most existing methods for AA are designed for continuous data and do not exploit the structure of the data distribution. In this paper, we propose two new optimization frameworks for archetypal analysis for binary data. i) A second order approximation of the AA likelihood based on the Bernoulli distribution with efficient closed-form updates using an active set procedure for learning the convex combinations defining the archetypes, and a sequential minimal optimization strategy for learning the observation specific reconstructions. ii) A Bernoulli likelihood based version of the principal convex hull analysis (PCHA) algorithm originally developed for least squares optimization. We compare these approaches with the only existing binary AA procedure relying on multiplicative updates and demonstrate their superiority on both synthetic and real binary data. Notably, the proposed optimization frameworks for AA can easily be extended to other data distributions providing generic efficient optimization frameworks for AA based on tailored likelihood functions reflecting the underlying data distribution.