Implementing the ICE Estimator in Multilayer Perceptron Classifiers

TL;DR

This paper presents the implementation of the ICE estimator in Spark's MLP classifier, demonstrating improved performance and no additional hyper-parameters, making it a practical alternative to traditional cross-entropy training.

Contribution

The paper introduces a novel implementation of the ICE estimator in Spark's MLP classifier, enhancing performance without increasing complexity.

Findings

ICE estimator outperforms unadjusted MLE in cross-validation

Models have similar runtime and fitting performance to standard MLPs

No additional hyper-parameters required for ICE implementation

Abstract

This paper describes the techniques used to implement the ICE estimator for a multilayer perceptron model, and reviews the performance of the resulting models. The ICE estimator is implemented in the Apache Spark MultilayerPerceptronClassifier, and shown in cross-validation to outperform the stock MultilayerPerceptronClassifier that uses unadjusted MLE (cross-entropy) loss. The resulting models have identical runtime performance, and similar fitting performance to the stock MLP implementations. Additionally, this approach requires no hyper-parameters, and is therefore viable as a drop-in replacement for cross-entropy optimizing multilayer perceptron classifiers wherever overfitting may be a concern.