Combining Learned Representations for Combinatorial Optimization

TL;DR

This paper introduces a method to combine pretrained RBMs to solve large-scale combinatorial optimization problems, demonstrating improved accuracy and scalability in boolean satisfiability, addition, and factorization tasks.

Contribution

It presents a novel approach to synthesize larger models from smaller pretrained RBMs, enabling efficient modeling of complex multi-modal spaces for combinatorial optimization.

Findings

Successfully solved 64-bit addition problems

Demonstrated factorization of 16-bit numbers

Achieved more accurate results with combined representations

Abstract

We propose a new approach to combine Restricted Boltzmann Machines (RBMs) that can be used to solve combinatorial optimization problems. This allows synthesis of larger models from smaller RBMs that have been pretrained, thus effectively bypassing the problem of learning in large RBMs, and creating a system able to model a large, complex multi-modal space. We validate this approach by using learned representations to create ``invertible boolean logic'', where we can use Markov chain Monte Carlo (MCMC) approaches to find the solution to large scale boolean satisfiability problems and show viability towards other combinatorial optimization problems. Using this method, we are able to solve 64 bit addition based problems, as well as factorize 16 bit numbers. We find that these combined representations can provide a more accurate result for the same sample size as compared to a fully trained model.