Softmax as a Lagrangian-Legendrian Seam

TL;DR

This paper introduces a geometric framework linking softmax in machine learning to differential geometry, modeling logits-to-probabilities as a Legendrian seam on a contact manifold, revealing new invariants and connections to information geometry.

Contribution

It provides the first geometric interpretation of softmax as a Legendrian seam, bridging ML and differential geometry with potential for new models and invariants.

Findings

Softmax modeled as a Legendrian seam on a contact manifold.

Bias-shift invariance linked to Reeb flow.

Connections to information geometry and replicator flows.

Abstract

This note offers a first bridge from machine learning to modern differential geometry. We show that the logits-to-probabilities step implemented by softmax can be modeled as a geometric interface: two potential-generated, conservative descriptions (from negative entropy and log-sum-exp) meet along a Legendrian "seam" on a contact screen (the probability simplex) inside a simple folded symplectic collar. Bias-shift invariance appears as Reeb flow on the screen, and the Fenchel-Young equality/KL gap provides a computable distance to the seam. We work out the two- and three-class cases to make the picture concrete and outline next steps for ML: compact logit models (projective or spherical), global invariants, and connections to information geometry where on-screen dynamics manifest as replicator flows.