Tactile Hallucinations on Artificial Skin Induced by Homeostasis in a Deep Boltzmann Machine

TL;DR

This study demonstrates that homeostasis in a Deep Boltzmann Machine can induce tactile hallucinations on artificial skin, offering insights into sensory perception and potential mechanisms behind hallucinations in biological systems.

Contribution

It introduces a novel approach using homeostasis in a Deep Boltzmann Machine to model tactile hallucinations and their underlying neural mechanisms.

Findings

Homeostasis prompts hallucinations of learned patterns without sensory input.

Homeostasis induces meaningful latent representations in the DBM.

It improves the quality of latent state reconstructions.

Abstract

Perceptual hallucinations are present in neurological and psychiatric disorders and amputees. While the hallucinations can be drug-induced, it has been described that they can even be provoked in healthy subjects. Understanding their manifestation could thus unveil how the brain processes sensory information and might evidence the generative nature of perception. In this work, we investigate the generation of tactile hallucinations on biologically inspired, artificial skin. To model tactile hallucinations, we apply homeostasis, a change in the excitability of neurons during sensory deprivation, in a Deep Boltzmann Machine (DBM). We find that homeostasis prompts hallucinations of previously learned patterns on the artificial skin in the absence of sensory input. Moreover, we show that homeostasis is capable of inducing the formation of meaningful latent representations in a DBM and that it significantly increases the quality of the reconstruction of these latent states. Through this, our work provides a possible explanation for the nature of tactile hallucinations and highlights homeostatic processes as a potential underlying mechanism.

Tables

Table 1. TABLE I: Overview of the hyperparameters for the artificial skin

Hyperparameter	Variable	Options	Value
MAX_FORCE	${\theta }_F$	[0,1]	0.012
MIN_NUMBER_OF_CELLS	$N$	$\in \mathbb{N}$	2
COMBINE_ITER	$C$	$\in \mathbb{N}$	5
DISPLAY_DURATION	$D$	$\in \mathbb{N}$	3

Table 2. TABLE II: Accuracy of the decoded reconstructions of the hidden layer representations. All values are the mean performance over ten trials.

Receptive field	circular	linear
$Q_{pattern}$	0.87	0.83
$Q_{corrupted}$	0.58	0.50
$Q_{blank}$	0.50	0.42
$\Delta Q_{loss}$	0.37	0.41
$Q_{hallucination}$	0.72	0.62
$\Delta Q_{gain}$	0.22	0.20

Equations

$$\Delta b_i=\eta ({\mu }_i-a_i),$$

$$D(A,B)=\frac{2|A\cap B|}{|A|+|B|},$$

$$Q=\underset{i}{\max }D(P_i,S).$$

$$\Delta Q_{loss}=Q_{pattern}-Q_{blank}$$

$$\Delta Q_{gain}=Q_{hallucination}-Q_{blank}$$

$$p(h_{j}^{(1)}|v)=\sigma\bigg{(}\sum_{i}v_{i}w_{ij}+b_{i}^{(1)}\bigg{)},$$

$$E(v,h,\theta )=-b^{T}v-c^{T}h-v^{T}Wh,$$

$$p(h_{j}^{(1)}|h_{j}^{(2)},v)=\sigma\bigg{(}\sum_{i}v_{i}w_{ij}^{(1)}+\sum_{k}h_{k}^{(2)}w_{kj}^{(2)}+b_{i}^{(1)}\bigg{)},$$

$$\begin{multlined}E(v,h^{(1)},h^{(2)},\theta)=-b^{T}v-c^{(1)T}h^{(1)}-\\ c^{(2)T}h^{(2)}-v^{T}W^{(1)}h^{(1)}-h^{(1)T}W^{(2)}h^{(2)}.\end{multlined}E(v,h^{(1)},h^{(2)},\theta)=-b^{T}v-c^{(1)T}h^{(1)}-\\ c^{(2)T}h^{(2)}-v^{T}W^{(1)}h^{(1)}-h^{(1)T}W^{(2)}h^{(2)}.$$

$$p(v,\theta )=\frac{1}{Z}\exp (-F(v)),$$

$$Z=\sum _x\exp (-F(x)),$$

$$F(x)=-\log\big{(}\sum_{h}\exp(-E(x))\big{)}.$$

$$\Delta w^{0}=\lambda\big{(}(x^{(0)}x^{(1)T})_{data}-(x^{(0)}x^{(1)T})_{model}\big{)}$$

Full text

I Introduction

Hallucinations have been observed in several sensory systems, yet the underlying mechanisms are still poorly understood. In the somatosensory system, different kinds of hallucinations can emerge. In patients with an amputated arm or leg, phantom limb sensations have been described in the position of the missing limb [1, 2]. Tactile hallucinations have also been described in people with neuropsychiatric diseases, most notably schizophrenia or Parkinson’s disease [3, 4]. In order to treat these diseases, a clear understanding of the underlying mechanisms is required. The clinical heterogeneity of hallucinations and their potential variety of underlying causes display a great challenge and opportunity from the computational modelling point of view [5]. Developing mathematical models that are able to explain these effects provides answers to how the brain perceives the world. In fact, it has been recently shown that hallucinations can be even provoked in healthy subjects [6] by means of sensory contingency conditioning.

Previous modelling works have focused on auditory [7] and visual hallucinations [8]. However, fewer models like [9, 10] have properly addressed tactile hallucinations despite its implications on prosthetic [11] and embodiment disorders.

Here, we provide such a framework inspired by recent work on the visual system. Therein, the emergence of visual hallucinations in visually impaired individuals, known as the Charles-Bonnet-Syndrome (CBS) [12], was modelled by applying homeostasis in a Deep Boltzmann Machine (DBM) [8]. Homeostasis is the adjustment of the excitability of the neurons due to reduced activity. This model provides a possible explanation for several observations made in people with CBS. As the CBS is frequently compared to the phantom limb or other kinds of tactile hallucinations [13, 12], we here extend the existing approach to tactile hallucinations using a previously developed artificial skin [14]. Our neuro-inspired framework provides a possible explanation for the emergence of tactile hallucinations and gives insights into the neurological processes during neuropsychiatric diseases such as Parkinson’s or schizophrenia.

We first explain our approach which uses a DBM to model tactile hallucinations (section II). Secondly, the experimental setup and the usage of the artificial skin for creating tactile patterns and visualizing hallucinations is presented in section III. Thirdly, we show our main results where homeostasis in a DBM leads to tactile hallucinations on an artificial skin (section IV). We conclude by a discussion of the neuroscientific relevance of this work and its potential impact in medical and engineering applications (section V).

II Proposed model

II-A Modelling tactile hallucinations

Few works in the literature have proposed computational models for hallucinations [5, 15]. Auditory hallucinations of schizophrenic patients [7, 16] and visual hallucinations for the Charles-Bonnet syndrome [8] were addressed by means of neural networks. Furthermore, perceptual hallucinations in a Bayesian framework were discussed in [17] under the free-energy principle and in [15] based on the circular inference hypothesis.

In order to model tactile hallucinations, this work leverages a Deep Boltzmann Machine (DBM), pre-trained with a Deep Belief Network (DBN) (see appendix). The overall setup of our model is depicted in Fig. 1. The artificial skin cells provide the input to the visual layer of the DBM, which is then trained on those patterns. We will show that the DBM learns hidden representations of the input data and that these representations are encoded in the hidden layers of the network. We claim that, through homeostasis, the network will even in absence of sensory input produce meaningful latent representations corresponding to previously learned patterns, i.e. a hallucination pattern. As shown in Fig. 1, homeostasis is modeled as an increase (or decrease) of the bias values of the network. The colored LEDs of the skin were used to visualize both the input and the output of the DBM.

II-B Receptive Fields

Cortical connectivity has been observed to be hierarchical and sparse [18, 19]. To model this, we use a limited connectivity between the layers of the DBM, where each neuron is connected only to neurons in the same or neighbouring columns (comparable to cortical columns), as depicted in Fig. 1. In order to allow for the same number of input connections for all neurons in a certain layer, we also consider a ring-like connectivity. Here, every neuron is again connected only to neurons in the same or neighbouring columns. Unlike in the first case, the columns are now ordered in a ring-like, circular, structure. We refer to the two described cases as linear and circular.

II-C Homeostasis

At the heart of our approach lies the change of excitability of neurons during sensory deprivation. On a single neuron scale, this process is termed homeostasis. Its timescale is on the order of hours or days [20, 21, 22]. In general, homeostatic mechanisms decrease the excitability of highly active neurons and increase the excitability of inactive neurons. It is thus often considered to be underlying stable network function and to prevent runaway network excitability in Hebbian cell assemblies [23]. Here, we make use of homeostatic mechanisms to evoke hallucinations corresponding to learned patterns. We do this by measuring the average activity of each neuron when presented with patterns from the training dataset. This average $\mu$ is then considered to be the healthy activity of that neuron. When presenting a zeros vector as an input of the network, the activity of the neuron $a_{i}$ will deviate from this baseline. In our model, homeostatic mechanisms will increase (or decrease) the bias $b_{i}$ of neuron $i$ in order to regain a healthy activity level:

[TABLE]

where $\eta$ is the adaptation rate, set to 0.01 for all neurons. Note that, due to the symmetry of the DBM, homeostatic mechanisms can both increase or decrease the bias in response to blank input.

II-D Decoding the hidden state

In order to evaluate the hidden states of the DBM, we need a method to decode the internal states of the hidden layers and to infer whether the hidden representations are mere noise or whether they correspond to a pattern. Instead of training a classifier on the states of the hidden units, we use the DBM as its own decoder [8].

In this process, we pass the hidden state through the network towards the visible layer in a single feedforward pass. While connections in a DBM are usually bidirectional, in this feedfordward pass, all neurons in a layer get input only from the adjacent deeper layer. Therefore, the total input to a neuron decreases due to the lack of input from the adjacent shallower layer. We compensate for this lack of input by multiplying the weights by a factor of two. The weights attached to the visible layer, however, are not multiplied by two as the visible units always get input only from one side. It is important to note that we apply this process only to decode the internal states of the network for inspection. This process is hence not required to be biologically plausible. To draw a comparison to biological neural networks, this process would correspond to matching measured neuronal activity (e.g. from local field potential) to the prevalent external stimulus.

II-E Evaluation measure

In order to evaluate that the hidden representations correspond to patterns that come from the same distribution as the dataset, we analyzed the quality of the reconstructions by means of the Dice-coefficient $D$

[TABLE]

where $A$ and $B$ are two binary patterns and $|\cdot|$ denotes the cardinality of the patterns (or, in other words, the number of values that are one).

Thus, given the patterns in the dataset $P_{i}$ and the pattern to be evaluated $S$, we define the performance of the network $Q$ as the maximal Dice-coefficient between $S$ and any of the patterns in the dataset

[TABLE]

Intuitively, if the performance of the network is one, its output (i.e. the decoding sample or the hallucination) corresponds to one of the patterns. If the performance is zero, the output does not have an overlap with any of the patterns in the training dataset.

III Artificial Skin and Experimental Setup

We work with an artificial skin [14, 24] consisting of 18 skin cells, which are ordered in a hexagonal grid with shape 3 $\times$ 6 (Fig 1). Each of these cells is equipped with, among others, three normal force sensors (Fig. 2).

For the pattern acquisition, we take the maximal recorded value of the three force sensors per cell. For further robustness, we cluster $C=5$ time steps into one round. If a certain cell exceeds its force threshold $\theta_{F}$ within the round, it is considered to be on in this interval. Each time step is 250 ms long, which means that one round corresponds to 1.25 seconds.

When recording, we accept the pattern for the dataset only if at least $N=2$ cells are on in this round. This prevents having a vast amount of either empty or noisy data in the dataset.

When the measured value does not exceed $\theta_{F}$, the LED lights up in green. If the respective threshold is exceeded, the LED of the respective artificial skin cell lights up in blue.

The skin connection is bidirectional. Thus, it also displays patterns extracted from the output of the DBM and shows them for $D=3$ time steps, with each time step again corresponding to 250 ms.

To model the tactile hallucinations presented in the result section IV, a proof-of-concept dataset was created, consisting of three patterns with one triangle each, where a triangle is made up of three cells. The size of the patterns was selected for visual comprehension.

For reproducibility, the hyperparameters of the proposed method, and their meaning, as well as their options and a recommended default value, are given in Table I.

IV Results

IV-A Training and sampling from the DBM

For the below described results, we used a DBM with three layers, with each layer consisting of 18 neurons. During training, samples from the network were evaluated by calculating the Dice-coefficient-based evaluation measure $Q$ (see section II-E). We pretrained the DBM as a Deep Belief Network. Thus, the training process of a three-layer DBM has three phases. In the first phase, the weights between the first and the second layer are pre-trained. After 2000 iterations, a performance $Q$ of around 0.85 is reached (Fig. 4, left). In the second phase, only the weights between the second and the third layer are pre-trained by taking the latent representations in the first hidden layer as training data. Again, the performance $Q$ reaches a value of around 0.85. Lastly, samples from the entire DBM were taken. The pre-training accuracy of the DBM is already around 0.85. The last phase is to train the overall DBM, refining the network to a performance $Q$ of 0.97. We apply early stopping to ensure that the network does not collapse onto only one of the patterns.

IV-B Decoding the latent representations

Once the DBM has been trained, it is able to create samples from the training data. To inspect whether the DBM also forms meaningful latent representations, we decode the hidden states of the trained DBM using the decoding procedure described in section II-D. The decoding method was tested in three scenarios. As in the training, the quality of the decoding was evaluated using the measure $Q$. In the first scenario, the DBM is clamped to one of the patterns. In this case, the hidden representations in the second hidden layer clearly represent the data the network is clamped to. Thus, the decoded states mostly represent one of the training patterns, as shown in figure 5. The performance $Q_{pattern}$ for circular connectivity is about 0.87 in this case.

As a second scenario, the input pattern was corrupted by turning two of the three cells off. The DBM is then, again, clamped to this input and the deepest layer is decoded. By corrupting the pattern, the performance $Q_{corrupted}$ of the decoded pattern dropped to about 0.64 for cicurlar connectivity.

Lastly, fully empty patterns were fed to the network. As a result, the performance drops even further and is now only around $Q_{blank}=0.50$ for cicurlar connectivity.

Table II shows the performance of the model for the different scenarios. For further analysis, we also define the measure $\Delta Q_{loss}$, which is the difference in performances between the presentation of training patterns and zeros input - see Fig. 6.

[TABLE]

IV-C Homeostasis causes hallucinations

We have shown that presenting the network with a corrupted or blank input impedes the reconstruction quality. Now we show that, when introducing the homeostatic processes described in section II-C, the performance $Q$ of the network is increased when presented with a blank pattern and meaningful latent states are retrieved.

For this, we let the homeostatic process run for 2000 time steps. In each of the time steps, the activity of every neuron is measured and the biases are increased such that the activity moves back to the baseline level with an adaptation rate of 0.01. After every time step, the performance of the network $Q$ is measured. The results of this process for the two types of connectivity are shown in Fig. 7. The performance at the beginning corresponds to the performance of the network when presented with a blank input. Then, in the early stages of the homeostatic process, a strong increase in the performance is visible. After around 1200 time steps, the performance saturates.

We define the overall improvement $\Delta Q_{gain}$, visualized in figure 6, as:

[TABLE]

The final performance after 2000 steps of homeostasis $Q_{hallucination}$ is presented in the second last row of Table II. For both types of investigated connectivity, the homeostatic process leads to a significant increase in accuracy. In Fig. 7, the increase in performance from the initial state to the state after the homeostatic process $\Delta Q_{gain}\sim 0.2$ is shown.

Figure 8 then shows a sample trace for circular connectivity. Additionally, sample reconstructions are shown. While, before applying homeostatic mechanisms, the reconstructions are strongly perturbed, the decoded states at the final point clearly correspond to the trained patterns. Still, some of the decoded states correspond to corrupted versions of patterns.

We further analyzed what affects the gain of performance through homeostasis. We found that there is a relation between the drop in performance $\Delta Q_{loss}$ and the gain through homeostasis $\Delta Q_{gain}$. This is shown in Fig. 9 for both connectivity types. When the performance decreased strongly, the homeostatic mechanism will lead to a strong increase in performance afterwards (correlation coefficients, respectively, for the two lines: $\rho$=0.71, $\rho$=0.84). Thus, the difference in the quality of the reconstruction is particularly strong if the network suffered heavily from the blanked input pattern.

V Discussion

We suggest that tactile hallucinations can emerge as a consequence of homeostasis after an ill-formed input from the afferent pathway or other brain regions (corresponding to corrupted patterns) or through the complete lack of input (corresponding to blank patterns). Our results show that, when the network is clamped to one of the training patterns, the quality of the reconstruction is high, whereas it is low if the network is clamped to a corrupted or zeros pattern. Homeostatic mechanisms, modeled as an adaptation of the bias of the network, strongly increased the reconstruction quality. Thus, while homeostasis could be involved in the proper reconstruction or adaptation of sensory input when partial information is available, abnormal connectivity or dysfunctional homeostatic processes could induce tactile hallucinations.

V-A Medical applications

Our results showed that homeostasis is a possible mechanism to induce hallucinations. In order to verify this approach, biological experiments would be required. A possible way to tackle this would be to match the timescales of homeostatic processes and the onset of hallucinations. In the phantom limb phenomenon, the onset has been reported to happen within the first 24 hours after amputation for half of the patients and within a week for another 25 % of the patients [25]. This nicely fits the timescale of homeostatic mechanisms. Even though recent work has argued that there are fast homeostatic mechanisms on the timescale of seconds to minutes [23], most experimental work describes homeostasis to happen on a timescale of hours to weeks [20, 21, 22].

If experimental work would further provide evidence for our hypothesis, it would thus be conceivable to make use of our approach in a medical application. In the cases where patients have unpleasant tactile hallucinations, alleviating the homeostatic mechanism could lead to a delayed onset of the hallucinations. Instead of letting the neurons recover previous baseline activity, medical interventions could keep the activity levels in the corresponding brain region low. This inference could happen in several ways, for example through drugs or targeted electrical stimulation.

V-B Technical applications

The presented neuro-inspired framework can inspire technical systems beyond medical applications. DBMs have frequently been used as a method for feature extraction in classification tasks, where the features in the hidden layer are fed into a classifier [26]. In a robotics setting, homeostasis could then be used as an adaptive artificial intelligence mechanism. A general problem of artificial intelligence is domain adaptation, meaning that the network would perform badly in a different domain than the one it was trained on. In this work, we showed that keeping the learned weights constant and only changing the biases even in a non-selective way can recover the representations formed in the latent layers. Thus, even in non-optimal conditions, proper features could be selected by an adaptive DBM. As an example, the robot would be trained to perform a certain task under daylight conditions. However, when tested in a dim-light condition, we would expect it to perform worse. In that case, we could apply homeostasis to the DBM to recover the baseline activity that the classifier is familiar with. Furthermore, the generative nature of the proposed approach suits for flexible body perception and adaptation in robotics as an alternative method of predictive coding differential equations [27].

V-C Technical challenges and limitations

While our work provides a general framework for tactile hallucinations, it makes several, partly biologically implausible assumptions. Firstly, all our patterns were binary, while biological skin-receptors can have graded firing rates. A common approach is to either use tuning curves at the input layer to encode non-binary sensory information or to code the activity into a population of neurons representing a single skin cell [28, 29]. Secondly, while humans have millions of somatosensory receptors, we use only a small skin-patch. For further increasing the role of the spatial structure, a larger skin patch could be mounted on a robotic limb to allow for a natural interaction with the robot. As, in most cases, such interaction covers only a small fraction of all active skin cells, we used sparse patterns (see Fig. 3). In comparison to dense patterns, these sparser patterns seemed to stabilize the performance of the network and of the homeostatic mechanisms. Besides, only by pre-training the DBM with a DBN, the system was able to learn the proper latent space representations. Conversely, training the RBM directly with $k$-step contrastive divergence was highly unstable and produced reasonable results only in a fraction of all trials.

Lastly, our approach has only used a small dataset. This does not resemble biology, where the input patterns are samples from a diverse distribution. This issue could generally be resolved by collecting more samples from a larger skin patch, as described in the paragraph above. In this case, the training data would then also come from actual interactions with a robot, allowing for diversity and realism in the dataset.

Further analysis should be performed on the spatial structure of the skin cells. In the two cases, using circular or linear receptive fields, the input matrix is linearized. In somatosensation, however, the input patterns have a highly ordered spatial structure. The effects of this spatial structure could be investigated by applying two-dimensional receptive fields or learning the spatial connectivity structure [30]. These receptive fields would lead to localized processing of the input data. Our work has already shown that constraining the connectivity has effects on the quality of the decoding and the strength of the hallucinations (Fig. 7). When moving towards more and more realistic tactile patterns, local processing and constrained connectivity could thus play a major role.

VI Conclusion

We presented a framework for modelling tactile hallucinations in an artificial skin. Overall, this work shows a possible role of homeostatic processes in the emergence of hallucinations. While homeostasis has been linked to many crucial parts of network function, we here showed that it could lead to the formation of meaningful latent representations without actual input. On the one hand, our model allows computational neuroscientists to investigate the potential role of different parameters such as the severity of the neurological disease (by modifying the corrupted input pattern). On the other hand, this framework also allows for more targeted experiments in order to elucidate the emergence of hallucinations.

-A Boltzmann Machines

In general, all of the described neural network architectures are generative models. This means that, after a training phase, in which input patterns are being presented to the network, the network has learned the probability distribution of the underlying dataset. The network is thus able to sample from this probability distribution and produce samples that are consistent with the ones from the dataset.

In order to model hallucinations, this work leverages a Deep Boltzmann Machine (DBM) pre-trained with a Deep Belief Network (DBN). We will hence start by describing Restricted Boltzmann Machines (RBMs), the building blocks of DBMs and DBNs, and then go on to describing DBMs and DBNs themselves. After that, we will explain the common training algorithm for all these architectures, namely contrastive divergence. Lastly, we will describe how we sample from the DBM.

-B Restricted Boltzmann Machines

A Restricted Boltzmann Machine consists of two layers, termed the visible layer $v$ and the hidden layer $h$. The weights between the two layers are bidirectional and symmetric, meaning that they are shared between the forward and the backward pass. The neurons of all Boltzmann Machines are binary and stochastic. The probability for a neuron in the hidden layer $h_{j}$ (index j denotes the $j$-th neuron) to be one is given by the sigmoid of the sum over the weighted inputs from the visible layer:

[TABLE]

where $v$ denotes the visible layer activations, $w_{ij}$ is the weight from neuron $j$ to $i$, and $b_{i}$ is the bias term of the $i$-th neuron. Moreover, $\sigma(\cdot)$ denotes the sigmoid function $1/(1+\exp(-x))$. By minimizing the free energy

[TABLE]

of the states $v,h$ with respect to all parameters $\theta$, which contains the weights $W$ and the biases $b,c$, the network learns to represent the features of the visible layer in the hidden layer and thereby allows for sampling from the same distribution as the input data [31, 32].

-C Deep Boltzmann Machines

Deep Boltzmann Machines (DBMs) are Markov random fields. Unlike Deep Belief networks, the intermediate layers of a DBM receive input from both its subsequent layers. Hence, the activation of a neuron in the first hidden layer can be computed as:

[TABLE]

where the subscript 1 denotes the first layer. Like RBMs, DBMs learn by minimizing the free energy. In a DBM with two hidden layers, the free Energy $E$ is defined as

[TABLE]

As described above, DBMs also have stochastic activations. The probability of a certain pattern to be observed in the DBM can be calculated from its free energy:

[TABLE]

where $Z$ is the partition function defined as

[TABLE]

with $x$ denoting all possible states. The partition function hence acts as a normalization factor. Moreover, the function $F(v)$ denotes

[TABLE]

-D Deep Belief Networks

Unlike DBMs, Deep Belief Networks (DBNs) represent several stacked RBMs, where each layer learns the features of the more shallow layer. DBNs have directed connections which implies that the training happens in a greedy, layer-wise way. In practice, we feed the data into the visible layer and use persistent contrastive divergence, as described in section -E to train the first hidden layer. Then, the second hidden layer is trained by using the states of the first hidden layer that had been computed via the visible layer.

-E Training Boltzmann Machines

Training in Boltzmann Machines happens by minimizing the difference in free energies of the data and the model, which can be described in two steps. The algorithm that implements this is called persistent contrastive divergence. The Boltzmann Machine is first clamped to the data and the hidden layers are being updated. Through this, the network is forced into a state that does generally not correspond to an energy minimum yet, but rather a state that represents the input pattern. Once the network has reached its equilibrium, we calculate the free energy of the state of the network. It has been shown that a single update loop is sufficient to successfully train Boltzmann Machines [32]. A gradient descent approach will then minimize this free energy and thereby maximize the probability of the data (the clamped tactile pattern). After this, the network is run freely, without any clamped layers (again, a single update loop is sufficient [32]). In this process, the states of the network end up in a local energy minimum. We can use gradient ascent to maximize the free energy of the model for this state in order to decrease the probability of unwanted patterns. Note that, in practice, these two update states are performed simultaneously. We described them as separate processes for clarity. The resulting learning rule can be written as

[TABLE]

with learning rate $\lambda$. Note that this can be identified as a difference between two Hebbian learning equations, and hence a local learning rule. Hebbian and anti-Hebbian learning have been found in many regions of the brain [33, 34].

-F Sampling from the Boltzmann Machines

Sampling from Boltzmann Machines happens by feeding random inputs into the network. We then update the states of all layers. Through this, the state of the network moves into a local minimum of the free energy. Which local minimum the network moves into is defined by the starting condition, i.e. the random input pattern. For a well trained Boltzmann machine, the state corresponding to a free-energy-minimum represents a sample from the data distribution. Thus, we can take the values of the visible layer and use them as our sample.