A method for quantifying the generalization capabilities of generative models for solving Ising models

TL;DR

This paper introduces a Hamming distance regularizer within variational autoregressive networks to quantify and compare the generalization capabilities of different neural network architectures in solving Ising models, aiding in neural architecture search.

Contribution

The paper proposes a novel Hamming distance regularizer to measure the generalization ability of various neural networks combined with VAN for Ising models.

Findings

The regularizer effectively quantifies network generalization.

Different architectures show varying generalization capabilities.

Method can predict large-scale performance from small-scale results.

Abstract

For Ising models with complex energy landscapes, whether the ground state can be found by neural networks depends heavily on the Hamming distance between the training datasets and the ground state. Despite the fact that various recently proposed generative models have shown good performance in solving Ising models, there is no adequate discussion on how to quantify their generalization capabilities. Here we design a Hamming distance regularizer in the framework of a class of generative models, variational autoregressive networks (VAN), to quantify the generalization capabilities of various network architectures combined with VAN. The regularizer can control the size of the overlaps between the ground state and the training datasets generated by networks, which, together with the success rates of finding the ground state, form a quantitative metric to quantify their generalization capabilities. We conduct numerical experiments on several prototypical network architectures combined with VAN, including feed-forward neural networks, recurrent neural networks, and graph neural networks, to quantify their generalization capabilities when solving Ising models. Moreover, considering the fact that the quantification of the generalization capabilities of networks on small-scale problems can be used to predict their relative performance on large-scale problems, our method is of great significance for assisting in the Neural Architecture Search field of searching for the optimal network architectures when solving large-scale Ising models.