# Neural Networks retrieving Boolean patterns in a sea of Gaussian ones

## TL;DR

This paper investigates the retrieval capabilities of neural networks with mixed real and Boolean patterns, demonstrating that Boolean patterns can be retrieved even under high real pattern load, aligning with classical theoretical thresholds.

## Contribution

It introduces a theoretical analysis of mixed Hebbian networks with both Gaussian and Boolean patterns, revealing conditions for Boolean pattern retrieval in high real pattern load regimes.

## Key findings

- Boolean patterns can be retrieved despite high real pattern load
- The critical load threshold matches the classical Amit-Gutfreund-Sompolinsky theory
- The analysis extends existing models to mixed pattern types

## Abstract

Restricted Boltzmann Machines are key tools in Machine Learning and are described by the energy function of bipartite spin-glasses. From a statistical mechanical perspective, they share the same Gibbs measure of Hopfield networks for associative memory. In this equivalence, weights in the former play as patterns in the latter. As Boltzmann machines usually require real weights to be trained with gradient descent like methods, while Hopfield networks typically store binary patterns to be able to retrieve, the investigation of a mixed Hebbian network, equipped with both real (e.g., Gaussian) and discrete (e.g., Boolean) patterns naturally arises. We prove that, in the challenging regime of a high storage of real patterns, where retrieval is forbidden, an extra load of Boolean patterns can still be retrieved, as long as the ratio among the overall load and the network size does not exceed a critical threshold, that turns out to be the same of the standard Amit-Gutfreund-Sompolinsky theory. Assuming replica symmetry, we study the case of a low load of Boolean patterns combining the stochastic stability and Hamilton-Jacobi interpolating techniques. The result can be extended to the high load by a non rigorous but standard replica computation argument.