Properties of the Concrete distribution

TL;DR

This paper analyzes the properties of the Concrete distribution, including its geometric structure, Fisher information, and parameter transformations, revealing its hyperbolic space characteristics.

Contribution

It provides a detailed geometric and information-theoretic analysis of the Concrete distribution, including explicit parameter transformations and Fisher-Rao metric computations.

Findings

Fisher information of the distribution is hyperbolic space

Explicit transformation to Poincaré half-space coordinates

Fisher-Rao geodesic distance is computed

Abstract

We examine properties of the Concrete (or Gumbel-softmax) distribution on the simplex. Using the natural vector space structure of the simplex, the Concrete distribution can be regarded as a transformation of the uniform distribution through a reflection and a location-scale transformation. The Fisher information is computed and the corresponding information metric is hyperbolic space. We explicitly give an explicit transformation of the parameters of the distribution to Poincar\'e half-space coordinates, which correspond to an orthogonal parameterization, and the Fisher-Rao geodesic distance is computed.