Improving Connectionist Energy Minimization

TL;DR

This paper introduces an improved energy minimization algorithm for symmetric networks like Boltzmann machines and Hopfield nets, guaranteeing global minima in linear time for tree-like subnetworks and enhancing performance with cycle-cutset techniques.

Contribution

The paper presents a novel uniform algorithm, activate, that guarantees global minima in acyclic networks and extends it with activate-with-cutset for improved performance in complex networks.

Findings

Guarantees global minimum in linear time for tree-like subnetworks.

Ensures convergence to global minimum in acyclic networks from any initial state.

Highlights limitations of uniform algorithms in cyclic networks and proposes cycle-cutset improvements.

Abstract

Symmetric networks designed for energy minimization such as Boltzman machines and Hopfield nets are frequently investigated for use in optimization, constraint satisfaction and approximation of NP-hard problems. Nevertheless, finding a global solution (i.e., a global minimum for the energy function) is not guaranteed and even a local solution may take an exponential number of steps. We propose an improvement to the standard local activation function used for such networks. The improved algorithm guarantees that a global minimum is found in linear time for tree-like subnetworks. The algorithm, called activate, is uniform and does not assume that the network is tree-like. It can identify tree-like subnetworks even in cyclic topologies (arbitrary networks) and avoid local minima along these trees. For acyclic networks, the algorithm is guaranteed to converge to a global minimum from any initial state of the system (self-stabilization) and remains correct under various types of schedulers. On the negative side, we show that in the presence of cycles, no uniform algorithm exists that guarantees optimality even under a sequential asynchronous scheduler. An asynchronous scheduler can activate only one unit at a time while a synchronous scheduler can activate any number of units in a single time step. In addition, no uniform algorithm exists to optimize even acyclic networks when the scheduler is synchronous. Finally, we show how the algorithm can be improved using the cycle-cutset scheme. The general algorithm, called activate-with-cutset, improves over activate and has some performance guarantees that are related to the size of the network's cycle-cutset.