The Information Theoretically Efficient Model (ITEM): A model for computerized analysis of large datasets

TL;DR

The paper introduces ITEM, an efficient and robust multinomial logistic regression model capable of handling large datasets and non-linear relationships, producing interpretable results resistant to overfitting.

Contribution

This work presents a novel information-theoretic approach to multinomial logistic regression that is computationally feasible for large datasets and effective in modeling non-linear dependencies.

Findings

Models can be generated efficiently on modern computers.

ITEM produces interpretable models with limited features.

Models are resistant to overfitting.

Abstract

This document discusses the Information Theoretically Efficient Model (ITEM), a computerized system to generate an information theoretically efficient multinomial logistic regression from a general dataset. More specifically, this model is designed to succeed even where the logit transform of the dependent variable is not necessarily linear in the independent variables. This research shows that for large datasets, the resulting models can be produced on modern computers in a tractable amount of time. These models are also resistant to overfitting, and as such they tend to produce interpretable models with only a limited number of features, all of which are designed to be well behaved.