On the alpha-loss Landscape in the Logistic Model
Tyler Sypherd, Mario Diaz, Lalitha Sankar, and Gautam Dasarathy

TL;DR
This paper investigates the optimization landscape of the alpha-loss family in logistic models, revealing how landscape properties change with alpha and implications for optimization complexity.
Contribution
It provides a detailed analysis of the alpha-loss landscape across different alpha values, connecting geometric properties to optimization difficulty.
Findings
Landscape properties vary with alpha, affecting optimization complexity.
Alpha-loss includes exponential, log, and 0-1 losses as special cases.
Results inform the choice of loss functions for better learning performance.
Abstract
We analyze the optimization landscape of a recently introduced tunable class of loss functions called -loss, , in the logistic model. This family encapsulates the exponential loss (), the log-loss (), and the 0-1 loss () and contains compelling properties that enable the practitioner to discern among a host of operating conditions relevant to emerging learning methods. Specifically, we study the evolution of the optimization landscape of -loss with respect to using tools drawn from the study of strictly-locally-quasi-convex functions in addition to geometric techniques. We interpret these results in terms of optimization complexity via normalized gradient descent.
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsSparse and Compressive Sensing Techniques · Stochastic Gradient Optimization Techniques · Machine Learning and Algorithms
