A Letter on Convergence of In-Parameter-Linear Nonlinear Neural Architectures with Gradient Learnings

TL;DR

This paper proves BIBS stability for weight convergence in a broad class of in-parameter-linear nonlinear neural architectures under incremental gradient learning, providing practical convergence conditions for real-time applications.

Contribution

It introduces a theoretical framework establishing BIBS stability for these neural architectures with new convergence conditions.

Findings

BIBS stability proven for a broad neural architecture family

Practical convergence conditions derived for real-time learning

Applicable to incremental gradient algorithms

Abstract

This letter summarizes and proves the concept of bounded-input bounded-state (BIBS) stability for weight convergence of a broad family of in-parameter-linear nonlinear neural architectures as it generally applies to a broad family of incremental gradient learning algorithms. A practical BIBS convergence condition results from the derived proofs for every individual learning point or batches for real-time applications.