The Impact of Feature Representation on the Accuracy of Photonic Neural Networks

TL;DR

This paper investigates how feature encoding strategies affect the performance of Photonic Neural Networks, revealing that optimal encoding can significantly improve accuracy by influencing feature importance and learning capabilities.

Contribution

It introduces a mathematical methodology to analyze feature combination effects in PNNs and demonstrates how optimal encoding improves accuracy, surpassing non-encoded approaches.

Findings

Encoding multiple features influences their importance in PNNs.

Optimal encoding can improve accuracy by up to 12.3%.

Careful feature encoding enhances PNN performance in size and power constrained scenarios.

Abstract

Photonic Neural Networks (PNNs) are gaining significant interest in the research community due to their potential for high parallelization, low latency, and energy efficiency. PNNs compute using light, which leads to several differences in implementation when compared to electronics, such as the need to represent input features in the photonic domain before feeding them into the network. In this encoding process, it is common to combine multiple features into a single input to reduce the number of inputs and associated devices, leading to smaller and more energy-efficient PNNs. Although this alters the network's handling of input data, its impact on PNNs remains understudied. This paper addresses this open question, investigating the effect of commonly used encoding strategies that combine features on the performance and learning capabilities of PNNs. Here, using the concept of feature importance, we develop a mathematical methodology for analyzing feature combination. Through this methodology, we demonstrate that encoding multiple features together in a single input determines their relative importance, thus limiting the network's ability to learn from the data. Given some prior knowledge of the data, however, this can also be leveraged for higher accuracy. By selecting an optimal encoding method, we achieve up to a 12.3% improvement in accuracy of PNNs trained on the Iris dataset compared to other encoding techniques, surpassing the performance of networks where features are not combined. These findings highlight the importance of carefully choosing the encoding to the accuracy and decision-making strategies of PNNs, particularly in size or power constrained applications.