# OverSketched Newton: Fast Convex Optimization for Serverless Systems

## TL;DR

OverSketched Newton is a randomized optimization algorithm designed for serverless systems, using matrix sketching to efficiently approximate the Hessian, resulting in faster convergence and resilience to system stragglers.

## Contribution

It introduces a novel Hessian approximation method using matrix sketching tailored for serverless architectures, with proven convergence guarantees and empirical speedups.

## Key findings

- Achieves ~50% reduction in total running time on AWS Lambda.
- Provides convergence guarantees for both strongly convex and non-convex problems.
- Demonstrates robustness against stragglers in serverless environments.

## Abstract

Motivated by recent developments in serverless systems for large-scale computation as well as improvements in scalable randomized matrix algorithms, we develop OverSketched Newton, a randomized Hessian-based optimization algorithm to solve large-scale convex optimization problems in serverless systems. OverSketched Newton leverages matrix sketching ideas from Randomized Numerical Linear Algebra to compute the Hessian approximately. These sketching methods lead to inbuilt resiliency against stragglers that are a characteristic of serverless architectures. Depending on whether the problem is strongly convex or not, we propose different iteration updates using the approximate Hessian. For both cases, we establish convergence guarantees for OverSketched Newton and empirically validate our results by solving large-scale supervised learning problems on real-world datasets. Experiments demonstrate a reduction of ~50% in total running time on AWS Lambda, compared to state-of-the-art distributed optimization schemes.