DFReg: A Physics-Inspired Framework for Global Weight Distribution Regularization in Neural Networks

TL;DR

DFReg is a novel regularization framework inspired by physics, specifically Density Functional Theory, designed to promote smooth and diverse weight distributions in neural networks without altering architecture.

Contribution

It introduces a physics-inspired global regularization method for neural network weights, differing from traditional techniques by enforcing structural regularity without stochastic methods.

Findings

Encourages smooth, diverse weight distributions

Improves model regularity without architectural changes

Operates without stochastic perturbations

Abstract

We introduce DFReg, a physics-inspired regularization method for deep neural networks that operates on the global distribution of weights. Drawing from Density Functional Theory (DFT), DFReg applies a functional penalty to encourage smooth, diverse, and well-distributed weight configurations. Unlike traditional techniques such as Dropout or L2 decay, DFReg imposes global structural regularity without architectural changes or stochastic perturbations.