# Bias of Homotopic Gradient Descent for the Hinge Loss

## TL;DR

This paper investigates the convergence behavior of a homotopic gradient descent method applied to the hinge loss in linear classifiers, providing explicit rates towards the max-margin solution for separable data.

## Contribution

It introduces a homotopic gradient descent approach for the hinge loss and establishes explicit convergence rates to the max-margin solution in linearly separable data.

## Key findings

- Convergence to max-margin solution is achieved with explicit rates.
- Homotopic gradient descent effectively handles non-smooth hinge loss.
- Theoretical analysis extends understanding of gradient methods for non-smooth losses.

## Abstract

Gradient descent is a simple and widely used optimization method for machine learning. For homogeneous linear classifiers applied to separable data, gradient descent has been shown to converge to the maximal margin (or equivalently, the minimal norm) solution for various smooth loss functions. The previous theory does not, however, apply to non-smooth functions such as the hinge loss which is widely used in practice. Here, we study the convergence of a homotopic variant of gradient descent applied to the hinge loss and provide explicit convergence rates to the max-margin solution for linearly separable data.