Infinite Mixture of Inverted Dirichlet Distributions

TL;DR

This paper introduces an infinite mixture model of inverted Dirichlet distributions using Bayesian estimation and variational inference, enabling automatic component determination and improved modeling of positive vectors.

Contribution

It presents a novel Bayesian infinite mixture model for inverted Dirichlet distributions with a tractable variational inference algorithm and theoretical convergence guarantees.

Findings

Demonstrates effective modeling on synthetic data

Shows improved performance on real data

Avoids over-fitting and under-fitting issues

Abstract

In this work, we develop a novel Bayesian estimation method for the Dirichlet process (DP) mixture of the inverted Dirichlet distributions, which has been shown to be very flexible for modeling vectors with positive elements. The recently proposed extended variational inference (EVI) framework is adopted to derive an analytically tractable solution. The convergency of the proposed algorithm is theoretically guaranteed by introducing single lower bound approximation to the original objective function in the VI framework. In principle, the proposed model can be viewed as an infinite inverted Dirichelt mixture model (InIDMM) that allows the automatic determination of the number of mixture components from data. Therefore, the problem of pre-determining the optimal number of mixing components has been overcome. Moreover, the problems of over-fitting and under-fitting are avoided by the Bayesian estimation approach. Comparing with several recently proposed DP-related methods, the good performance and effectiveness of the proposed method have been demonstrated with both synthesized data and real data evaluations.