On the scaling of polynomial features for representation matching

TL;DR

This paper investigates the impact of scaling polynomial features in neural models, showing that scaled degree 2 polynomial features significantly improve natural language inference performance by reducing classification errors.

Contribution

It demonstrates that scaling degree 2 polynomial features enhances representation matching in neural models, a novel insight for feature augmentation techniques.

Findings

Scaling degree 2 polynomial features reduces error by 5%.

Scaled polynomial features improve matching performance.

Degree 2 features have the highest impact among tested polynomials.

Abstract

In many neural models, new features as polynomial functions of existing ones are used to augment representations. Using the natural language inference task as an example, we investigate the use of scaled polynomials of degree 2 and above as matching features. We find that scaling degree 2 features has the highest impact on performance, reducing classification error by 5% in the best models.