Reviewing Data Access Patterns and Computational Redundancy for Machine Learning Algorithms

TL;DR

This paper explores how optimizing data access patterns and reducing computational redundancy can significantly enhance the performance of machine learning algorithms by improving data locality and reusing computation results.

Contribution

It provides an analysis of data locality in ML algorithms and proposes methods to exploit data reuse and reduce redundancy for performance gains.

Findings

Initial experiments show improved data access efficiency.

Redundancy reduction leads to faster algorithm execution.

Opportunities for reuse can be identified in various ML algorithms.

Abstract

Machine learning (ML) is probably the first and foremost used technique to deal with the size and complexity of the new generation of data. In this paper, we analyze one of the means to increase the performances of ML algorithms which is exploiting data locality. Data locality and access patterns are often at the heart of performance issues in computing systems due to the use of certain hardware techniques to improve performance. Altering the access patterns to increase locality can dramatically increase performance of a given algorithm. Besides, repeated data access can be seen as redundancy in data movement. Similarly, there can also be redundancy in the repetition of calculations. This work also identifies some of the opportunities for avoiding these redundancies by directly reusing computation results. We document the possibilities of such reuse in some selected machine learning algorithms and give initial indicative results from our first experiments on data access improvement and algorithm redesign.