Distributed Global Optimization (DGO)

TL;DR

This paper introduces a novel distributed global optimization algorithm that is non-reliant on function continuity or differentiability, avoids random mechanisms, requires no fine-tuning, and scales efficiently with problem dimensions, showing promising results in neural network applications.

Contribution

The paper presents a new distributed global optimization technique with advantages over existing methods, including no need for function smoothness, no randomness, and linear scalability.

Findings

Proven effective in neural network optimization.

Achieves linear time complexity O(n).

Does not require parameter fine-tuning.

Abstract

A new technique of global optimization and its applications in particular to neural networks are presented. The algorithm is also compared to other global optimization algorithms such as Gradient descent (GD), Monte Carlo (MC), Genetic Algorithm (GA) and other commercial packages. This new optimization technique proved itself worthy of further study after observing its accuracy of convergence, speed of convergence and ease of use. Some of the advantages of this new optimization technique are listed below: 1. Optimizing function does not have to be continuous or differentiable. 2. No random mechanism is used, therefore this algorithm does not inherit the slow speed of random searches. 3. There are no fine-tuning parameters (such as the step rate of G.D. or temperature of S.A.) needed for this technique. 4. This algorithm can be implemented on parallel computers so that there is little increase in computation time (compared to linear increase) as the number of dimensions increases. The time complexity of O(n) is achieved.