Forward-mode automatic differentiation for the tensor renormalization group and its relation to the impurity method

TL;DR

This paper introduces a forward-mode automatic differentiation framework for tensor renormalization group methods, enabling accurate computation of derivatives like internal energy and specific heat with manageable computational costs.

Contribution

The paper develops a forward-mode AD approach for TRG, establishing a link to impurity methods and providing practical tools for extracting critical exponents from tensor derivatives.

Findings

AD achieves higher accuracy than impurity methods for thermodynamic quantities.

The computational cost of AD scales reasonably with derivative order.

Method applies to both 2D and 3D tensor networks.

Abstract

We propose a forward-mode automatic differentiation (AD) framework for tensor renormalization group (TRG) methods. In this approach, evaluating the derivatives of the partition function up to order $k$ increases the matrix-multiplication cost by a factor of $(k+1)(k+2)/2$ compared to computing the free energy alone, and the memory footprint is only $k+1$ times that of the original calculation. In the limit where the derivatives of the SVD are neglected, we establish a theoretical correspondence between our forward-mode AD and conventional impurity methods. Numerically, we find that the proposed AD algorithm can calculate internal energy and specific heat significantly higher accuracy than the impurity method at comparable computational cost. We also provide a practical procedure to extract critical exponents from derivatives of the renormalized tensor in TRG calculations in both two and three dimensions. In addition, we discuss how to efficiently differentiate an arbitrary tensor network.