Determining Free Energy Differences Through Variational Morphing
Martin Reinhardt, Helmut Grubm\"uller

TL;DR
This paper introduces a generalized variational morphing approach for free energy calculations that improves sampling efficiency and accuracy, especially with small sample sizes, by optimizing Hamiltonian transformation sequences.
Contribution
It develops a non-linear Hamiltonian transformation framework that enhances free energy estimation methods like BAR, extending their applicability and efficiency.
Findings
Order of magnitude less sampling needed compared to traditional methods
Sequences are optimal for both FEP and BAR methods
Framework generalizes BAR to small samples and non-Gaussian errors
Abstract
Free energy calculations based on atomistic Hamiltonians and sampling are key to a first principles understanding of biomolecular processes, material properties, and macromolecular chemistry. Here, we generalize the Free Energy Perturbation method and derive non-linear Hamiltonian transformation sequences for optimal sampling accuracy that differ markedly from established linear transformations. We show that our sequences are also optimal for the Bennett Acceptance Ratio (BAR) method, and our unifying framework generalizes BAR to small sampling sizes and non-Gaussian error distributions. Simulations on a Lennard-Jones gas show that an order of magnitude less sampling is required compared to established methods.
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Determining Free Energy Differences Through Variational Morphing
Martin Reinhardt
Helmut Grubmüller
Max Planck Institute for Biophysical Chemistry, Am Fassberg 11, 37077 Göttingen, Germany
Abstract
Free energy calculations based on atomistic Hamiltonians and sampling are key to a first principles understanding of biomolecular processes, material properties, and macromolecular chemistry. Here, we generalize the Free Energy Perturbation method and derive non-linear Hamiltonian transformation sequences for optimal sampling accuracy that differ markedly from established linear transformations. We show that our sequences are also optimal for the Bennett Acceptance Ratio (BAR) method, and our unifying framework generalizes BAR to small sampling sizes and non-Gaussian error distributions. Simulations on a Lennard-Jones gas show that an order of magnitude less sampling is required compared to established methods.
Free energy calculations provide essential insights into numerous physical and biochemical systems. Examples of applications range from predicting binding processes of biomolecules for drug design Williams-Noonan et al. (2018); Cournia et al. (2017); Christ and Fox (2014) to determining thermodynamic properties of crystalline materials Swinburne and Marinica (2018); Freitas et al. (2018); de Koning et al. (1999). For large and complex systems with slow relaxation rates and typically to particles, only limited accuracy is achieved Zuckerman and Woolf (2002), despite substantial methodological progress Jarzynski (1997); Vaikuntanathan and Jarzynski (2008); Valsson and Parrinello (2014); Shirts et al. (2003) and immense computational effort. Besides force field inaccuracies, insufficient sampling is the main bottleneck Aldeghi et al. (2018). Here, we develop and evaluate a variational approach for optimal sampling that minimizes the sampling error.
Given the Hamiltonians and of two states and , where denotes the position of all particles of the simulation system, the free energy difference between these states is given by the Zwanzig formula Zwanzig (1954),
[TABLE]
where denotes an ensemble average defined by , which is approximated by averaging over a finite sample of size obtained from atomistic simulations or Monte Carlo sampling. For ease of notation, .
Alchemical transformations substantially reduce sampling errors Lu and Kofke (2001a, b) by introducing intermediate states ,
[TABLE]
and accumulating small free energy differences between all adjacent states and ,
[TABLE]
This technique is also employed in other fields, for example in the context of Bayesian statistics, where the plausibility of two different models is compared by calculating their marginal likelihood ratio Gelman and Meng (1998); Habeck (2012). With few exceptions Christ and Van Gunsteren (2007); Pham and Shirts (2012), only the linear interpolation between and of Eq. (2) is used, that is illustrated for a simple one-dimensional case in Fig. 1(a).
Here, we will generalize this linear interpolation for two of the most established methods, the Free Energy Perturbation (FEP) Zwanzig (1954) and the Bennett Acceptance Ratio (BAR) method Bennett (1976). Specifically, we ask which sequence amongst all possible functionals yields, on average, the highest accuracy. Figure 1(b) and 1(c) show such a general interpolation sequence, which we refer to as Variational Morphing Free Energy (VMFE) method. Unexpectedly, the result will also turn out to be a generalization of BAR to any and .
Note that our approach differs from previous attempts, such as soft-core potentials Steinbrecher et al. (2007), where ad hoc functionals are used. For linear interpolations (Eq. (2)), the distribution of points has been optimized Naden et al. (2014) which is also not the general solution we aim for.
To solve the above variational problem and to find the optimal sequence of , we consider the FEP scheme, displayed in Fig. 2(a), as one possible implementation of Eq. (3) using Eq. (1). In this particular variant, which is symmetric with respect to exchange of the two end states to avoid hysteresis effects, sample points are solely drawn from the odd-numbered ’sampling states’, and not from the even-numbered ’target states’. The average accuracy of this scheme is the average over all sampling realizations of the mean-squared deviation (MSD) of the free energy difference from the exact difference ,
[TABLE]
As in Fig. 2, the arrows point from sampling to target states.
Assuming for each sample state a set of independent sample points , drawn from , with partition function , the terms arising from expanding Eq. (4) will be considered one by one. For the linear term, the average over all sample realizations reads
[TABLE]
and for the quadratic term
[TABLE]
Similar expressions are obtained for . The exact free energy differences are
[TABLE]
For shifted Hamiltonians and , Eq. (1) yields
[TABLE]
which also holds for . Because these offsets cancel out in Eq. (4), the accuracy is invariant under any choice of offsets and . Choosing and such that the term in the logarithm of Eqs. (5) and (6) is close to one, and thus all are small with respect to , first order expansion of the logarithm allows to factorize the integrals, and therefore
[TABLE]
For the cross terms in Eq. (4), note that the estimated free energy differences of the individual steps are based on uncorrelated sample sets, and therefore
[TABLE]
for . Using Eq. (9), Eq. (6) yields
[TABLE]
Inserting Eqs. (9) and (11) into Eq. (4),
[TABLE]
where and denote expressions that only depend on exact free energy differences and thus are dropped for the optimization below.
With these expressions, the variational problem can be solved analytically. For the odd-numbered states , variation of , Eq. (12),
[TABLE]
yields
[TABLE]
where is the (finite) partition sum and is a Lagrange multiplier.
Similarly, for the even-numbered states,
[TABLE]
An additive term in Eqs. (14) and (15) was omitted, as it cancels in . The result is a set of equations for all states for which each Hamiltonian depends only on the two adjacent states. The initial requirement for small is fulfilled by setting , as in this case, all are one. Rearranging terms for odd ,
[TABLE]
and for even ,
[TABLE]
with . The first main result of this letter is the resulting sequence of Hamiltonians that yields the best accuracy for FEP free energy calculations.
The second main result is that Eq. (15) serves to generalize the BAR method. The latter follows from Eq. (15) for with one intermediate state: Applied to the two involved free energy differences, the Zwanzig formula yields
[TABLE]
Inserting Eq. (15) as the target state Hamiltonian yields the BAR formula
[TABLE]
with .
Notably, the above derivation yields the more general result that Eq. (20) provides the most accurate free energy estimate also for finite and small , even down to given sufficient configuration space density overlap between adjacent states, which is fulfilled, for instance, in the limit of many intermediates. In contrast, because the derivation by Bennett Bennett (1976) strictly holds only for infinite sampling, so far was required to be large, and proper convergence had to be assumed. Further, in the original derivation Bennett (1976) the error distribution of the free energy estimates had to be assumed to be Gaussian, which in our above result is also not required. In the context of the Overlap Sampling method Lu et al. (2004), it has been shown that an FEP intermediate can be defined that yields the weighting function from Bennett’s derivation; the above results proof that this intermediate is indeed optimal for the FEP scheme.
Further generalizing the BAR result, Eqs. (16) and (17) yield optimal VMFE intermediates for any (odd) number of intermediate states, as illustrated in Fig. 2: For any two sampling states, using BAR and using FEP with the optimal target state of Eq. (17) is equivalent. Applied recursively, therefore, the sampling states from any sequence of FEP-optimal Hamiltonians are also optimal for multistate BAR (MBAR) Shirts and Chodera (2008), where so far, too, only empirically determined linear interpolations have been used as intermediate states. This result, therefore, is a generalization to MBAR.
Conversely, for the setup of one sampling state between two given target end states and , with remarkable intuition an empirical potential has been proposed Christ and Van Gunsteren (2007) in the Envelope Distribution Sampling (EDS) method, which is similar to Eq. (14) except for a factor of two in the exponent. In summary, both BAR/MBAR and EDS are special cases of, or approximations to, our more general variational VMFE result that also requires fewer assumptions.
To solve Eqs. (16) and (17) for the optimal intermediate Hamiltonians , note that the unknown free energy differences are part of the equations which, therefore, have to be solved iteratively. With an initial guess for all , the set of equations is solved in a point-wise fashion for any given . After sampling all odd-numbered states, the values are updated iteratively, such that the sequence of intermediate states converges towards the optimum. For a typical biomolecular many-body system, the additional computational effort is small compared to computing and .
For the above illustrative example, Fig. 1(b) and (c) show the optimized Hamiltonians and the configuration space densities, respectively, of the converged sequence of intermediate states. To this end, initial values were used and Eqs. (16) and (17) were iterated until convergence, using numerical integration over and updating the during the process. Unlike the linear interpolations shown in Fig. 1(a), the variational morphing sequence leads to a probability density, which gradually decreases in the region of and increases in the region of , while remaining almost constant at the point of maximum configuration space overlap.
Figure 3(a) shows the results of numerical simulations using the one-dimensional test case shown in Fig. 1. Different minimum distances are used, thereby varying configuration space overlaps between the end states, indicated by the yellow area in Fig. 1(b). Sets of uncorrelated sample points are drawn from through rejection sampling. sampling states are used with BAR. For each , the accuracy (Eq. (4)) is calculated by averaging over 600,000 realizations.
VMFE (blue curve) yields the smallest MSD for all , compared to both the first linear interpolation variant (light green) using a linearly spaced , like in a typical free energy calculation, and even compared to the second variant (dark green) using the empirically determined value that yields the best accuracy that can be achieved by linear interpolation. For more details, see Supplementary Material. The largest improvements of VMFE are seen for small configuration space density overlaps that notoriously cause the largest uncertainties.
Figure 3(b) shows how the accuracy of VMFE improves with increasing number of states , keeping the total number of sample points, and hence the total computational effort, constant. For this example, the accuracy increases up to , beyond which no further improvement appears.
The above VMFE scheme, Eqs. (16) and (17) couple all intermediates and, therefore, cannot be run in parallel in a straightforward way. This limitation is overcome by two approximations. First, the sampling states are coupled directly using only Eq. (16). Therefore, while still using BAR between two adjacent sampling states, the corresponding target states are not used for their derivation. Second, Eq. (16) is solved recursively, i.e., the optimal sampling state is determined first from and , then from and , as well as from and , and so on. As a result, the approximate intermediate Hamiltonians read
[TABLE]
with prefactors recursively determined, using Eq. (16), such that all are a functional of only and . As above, is determined iteratively. Consequently, no prior knowledge of the differences between the individual states is required, and therefore, the sampling simulations for each state can be run in parallel without communication.
Figure 4 shows a comparison between the configuration space densities of the approximate intermediate Hamiltonians (dashed lines) with those of the optimal (solid lines), corresponding to the densities in Fig. 1(c). The two sequences are indeed very similar. Even if the optimal values are not known a priori, the approximated VMFE sequence covers the transition behavior of the optimal sequence well, particularly for larger numbers of intermediates.
As a more high-dimensional test case, we calculate the free energy difference between an Argon and a Helium Lennard-Jones (LJ) gas (parameters from White (1999)) with atoms. Fig. 5 shows the accuracy, determined through comparison to the result of a converged reference simulation, obtained by approximated VMFE with that of linearly interpolated intermediates. For more details, see Supplementary Material. At 5 ns, an over 4-fold improved accuracy is achieved by VMFE (green) compared to a conventional linear interpolation (red). Conversely, the accuracy achieved by linear interpolation at 5 ns is already obtained at 0.56 ns by VMFE, which thus requires almost 10 times less sampling.
Interestingly, apart from different factors in the exponent, the intermediates of the approximated sequence resemble those suggested in the context of thermodynamic integration (TI) Kirkwood (1935). Using approximations to the solution of the optimization problem for TI for several special cases Gelman and Meng (1998), an expression similar to the approximate Eq. (21) was obtained Pham and Shirts (2012); Blondel (2004). These results require a proper choice of , and it is unclear if the optimal states are the same for the different methods. Nevertheless, the similarity is striking and suggests that our result may also allow further improvements of TI.
In summary, we derived the optimal accuracy sequence of intermediate Hamiltonians for free energy perturbation calculations. Compared to the established linear intermediates, the accuracy improvement is substantial, especially for the critical small configuration space density overlap of the end states that are a hallmark of complex systems. The optimal sequences are fundamentally different from the linear ones, suggesting potential improvement, also for other methods that rely on intermediate states, e.g., TI or non-equilibrium methods Jarzynski (1997); Shirts et al. (2003).
VMFE was derived assuming statistically independent sampling points . For atomistic simulation based sampling, as well as, to a lesser extent, for MC sampling, subsequent sampling points are correlated, however, particularly when the relevant configuration space densities are separated by large barriers. In these cases, when combined with enhanced sampling techniques, such as Hamiltonian replica exchange Swendsen and Wang (1986); Liu et al. (2005); Tan (2017), appropriate biasing potentials Grubmüller (1995); Steiner et al. (1998); Laio and Parrinello (2002), or a combination thereof, VMFE should also yield improved accuracy, albeit the obtained intermediate Hamiltonians will not be optimal due to the neglected time correlations. On a more fundamental level, the equivalence of FEP and BAR established here implies that advances in any of these will benefit the other.
Appendix A Supplementary Material
A.0.1 One-dimensional Test Case - Highest Accuracy Linear Interpolation
Figure 3(a) shows a comparison of the accuracy obtained by VMFE with two variants of a linearly interpolated sequence. As , sampling is conducted in one intermediate state and the two end states. Sets of sample points are drawn from the corresponding through rejection sampling, based on which a free energy estimate between the end states is calculated.
For the linearly interpolated sequence, can be chosen by the user. To empirically obtain the that yields the highest accuracy (dark green), we loop over the allowed range between zero and one in steps of 0.01. To reliably calculate the MSD with respect to the exact value, for each 150,000 free energy estimates are calculated. Once the highest accuracy is determined, the corresponding MSD is calculated once again using 600,000 repetitions. The result of these is shown in the figure. The procedure is repeated for each value of (42 values). We note that the yielding the highest accuracy varies for different , and is inaccessible in practice for high-dimensional systems.
A.0.2 Lennard-Jones Gas Simulation
To compare the accuracy of the free energy estimate using a linearly interpolated sequence of states to the approximated VMFE sequence, a set of free energy calculations between an Argon and a Helium Lennard-Jones gas is conducted.
In each state, atoms are placed at random positions without overlap inside a cubic box. The atoms are assigned velocities drawn from the Boltzmann distribution corresponding to the temperature of K. The simulations are conducted in the NVT ensemble using periodic boundary conditions. The volume of the box is set to (43.5 , corresponding to a pressure of about 10 bar. The atomic interaction at a distance between the centers of two atoms is described through the Lennard-Jones potential,
[TABLE]
with parameters , kJmol and u for Argon, and , kJmol and u for Helium White (1999).
At the start, an equilibration run of 1 ns is conducted. The leap-frog algorithm with a time step of 5 fs is used and velocity rescaling at every 20th time step. For both sequences, 800 free energy simulations are conducted with 5 ns simulation time in each state. Five intermediate, i.e., seven states in total are used. In absence of further knowledge, equal spacing of and , i.e, is used. For the approximated VMFE sequence, is used throughout the whole simulation. The difference of the Hamiltonians between adjacent states is recorded at every 400th step. Free energy differences are subsequently calculated using BAR.
A reference free energy difference is determined by conducting a long simulation with each method using 12 states with linearly spaced and values and computation runs of 10 in each state. At this length, the relative difference has decreased below ( = 0.23252 ). Using this reference value, we calculate the MSD of the distribution of 800 free energy differences depending on the simulation time in each state.
The reference list from the paper itself. Each links out to its DOI / PubMed record.
- 1Williams-Noonan et al. (2018) B. J. Williams-Noonan, E. Yuriev, and D. K. Chalmers, Journal of Medicinal Chemistry 61 , 638 (2018) . · doi ↗
- 2Cournia et al. (2017) Z. Cournia, B. Allen, and W. Sherman, Journal of Chemical Information and Modeling 57 , 2911 (2017) . · doi ↗
- 3Christ and Fox (2014) C. D. Christ and T. Fox, Journal of Chemical Information and Modeling 54 , 108 (2014) . · doi ↗
- 4Swinburne and Marinica (2018) T. D. Swinburne and M. C. Marinica, Physical Review Letters 120 , 135503 (2018) . · doi ↗
- 5Freitas et al. (2018) R. Freitas, R. E. Rudd, M. Asta, and T. Frolov, Physical Review Materials 2 , 093603 (2018) . · doi ↗
- 6de Koning et al. (1999) M. de Koning, A. Antonelli, and S. Yip, Physical Review Letters 83 , 3973 (1999) . · doi ↗
- 7Zuckerman and Woolf (2002) D. M. Zuckerman and T. B. Woolf, Physical Review Letters 89 , 16 (2002) . · doi ↗
- 8Jarzynski (1997) C. Jarzynski, Physical Review Letters 78 , 2690 (1997) . · doi ↗
