# Efficient Implementation Of Newton-Raphson Methods For Sequential Data Prediction

## TL;DR

This paper introduces a highly efficient linear-complexity implementation of Newton-Raphson methods for sequential data prediction, enabling practical use in big data applications without sacrificing accuracy.

## Contribution

The authors develop a linear-time algorithm for Newton-Raphson methods that maintains second-order advantages, overcoming computational barriers in large-scale data prediction.

## Key findings

- Achieves linear computational complexity $O(M)$ for Newton-Raphson methods.
- Maintains performance comparable to traditional second-order methods.
- Demonstrates efficiency and stability on real-world big datasets.

## Abstract

We investigate the problem of sequential linear data prediction for real life big data applications. The second order algorithms, i.e., Newton-Raphson Methods, asymptotically achieve the performance of the "best" possible linear data predictor much faster compared to the first order algorithms, e.g., Online Gradient Descent. However, implementation of these methods is not usually feasible in big data applications because of the extremely high computational needs. Regular implementation of the Newton-Raphson Methods requires a computational complexity in the order of $O(M^2)$ for an $M$ dimensional feature vector, while the first order algorithms need only $O(M)$. To this end, in order to eliminate this gap, we introduce a highly efficient implementation reducing the computational complexity of the Newton-Raphson Methods from quadratic to linear scale. The presented algorithm provides the well-known merits of the second order methods while offering the computational complexity of $O(M)$. We utilize the shifted nature of the consecutive feature vectors and do not rely on any statistical assumptions. Therefore, both regular and fast implementations achieve the same performance in the sense of mean square error. We demonstrate the computational efficiency of our algorithm on real life sequential big datasets. We also illustrate that the presented algorithm is numerically stable.