Hardware-Friendly Input Expansion for Accelerating Function Approximation

TL;DR

This paper introduces a simple, hardware-friendly input-space expansion technique that accelerates convergence and improves accuracy in one-dimensional function approximation tasks by breaking parameter symmetries.

Contribution

It proposes a novel input-space expansion method inspired by symmetry breaking, which enhances neural network training efficiency without increasing model complexity.

Findings

Reduces LBFGS iterations by 12% on average

Decreases final MSE by 66.3% with 5D expansion

Constant $oldsymbol{ ext{ extpi}}$ outperforms other constants

Abstract

One-dimensional function approximation is a fundamental problem in scientific computing and engineering applications. While neural networks possess powerful universal approximation capabilities, their optimization process is often hindered by flat loss landscapes induced by parameter-space symmetries, leading to slow convergence and poor generalization, particularly for high-frequency components. Inspired by the principle of \emph{symmetry breaking} in physics, this paper proposes a hardware-friendly approach for function approximation through \emph{input-space expansion}. The core idea involves augmenting the original one-dimensional input (e.g., $x$) with constant values (e.g., $\pi$) to form a higher-dimensional vector (e.g., $[\pi, \pi, x, \pi, \pi]$), effectively breaking parameter symmetries without increasing the network's parameter count. We evaluate the method on ten representative one-dimensional functions, including smooth, discontinuous, high-frequency, and non-differentiable functions. Experimental results demonstrate that input-space expansion significantly accelerates training convergence (reducing LBFGS iterations by 12\% on average) and enhances approximation accuracy (reducing final MSE by 66.3\% for the optimal 5D expansion). Ablation studies further reveal the effects of different expansion dimensions and constant selections, with $\pi$ consistently outperforming other constants. Our work proposes a low-cost, efficient, and hardware-friendly technique for algorithm design.