Multinomial Logistic Regression Algorithms via Quadratic Gradient

TL;DR

This paper extends a quadratic gradient approach to multiclass logistic regression, introducing enhanced optimization algorithms that converge faster, demonstrated through experiments on various datasets.

Contribution

The paper develops an extended quadratic gradient-based optimization method for multiclass logistic regression, improving convergence speed over existing algorithms.

Findings

Enhanced NAG and Adagrad methods converge faster than original versions.

Experimental results confirm improved efficiency on multiclass datasets.

Methods outperform traditional algorithms in training speed.

Abstract

Multinomial logistic regression, also known by other names such as multiclass logistic regression and softmax regression, is a fundamental classification method that generalizes binary logistic regression to multiclass problems. A recently work proposed a faster gradient called $\texttt{quadratic gradient}$ that can accelerate the binary logistic regression training, and presented an enhanced Nesterov's accelerated gradient (NAG) method for binary logistic regression. In this paper, we extend this work to multiclass logistic regression and propose an enhanced Adaptive Gradient Algorithm (Adagrad) that can accelerate the original Adagrad method. We test the enhanced NAG method and the enhanced Adagrad method on some multiclass-problem datasets. Experimental results show that both enhanced methods converge faster than their original ones respectively.