Encoding architecture algebra

TL;DR

This paper proposes an algebraic framework for designing input-encoding architectures that better reflect data structures, aiming to improve the efficiency and type-awareness of machine learning models.

Contribution

It introduces a novel algebraic approach to construct input-encoding architectures that explicitly incorporate data types and structures.

Findings

Provides a formal algebraic framework for encoding architectures

Enhances type-awareness in machine learning models

Lays groundwork for more efficient data representation

Abstract

Despite the wide variety of input types in machine learning, this diversity is often not fully reflected in their representations or model architectures, leading to inefficiencies throughout a model's lifecycle. This paper introduces an algebraic approach to constructing input-encoding architectures that properly account for the data's structure, providing a step toward achieving more typeful machine learning.