Nesting instability of gapless U(1) spin liquids with spinon Fermi pockets in two dimensions

TL;DR

This paper investigates the stability of a gapless U(1) spin liquid with spinon Fermi pockets in two dimensions, revealing that interactions induce a nesting instability leading to a gapped, confined phase at low temperatures.

Contribution

It demonstrates that nesting instabilities cause the gapless U(1) spin liquid with spinon Fermi pockets to become unstable and transition to a conventional ordered state.

Findings

Interactions open a gap in the spinon spectrum at low temperatures.

Nesting instability leads to confinement of spinons and proliferation of monopoles.

The gapless spin liquid state is unstable and transitions to a valence bond solid.

Abstract

Quantum spin liquids are exotic states of matter that may be realized in frustrated quantum magnets and feature fractionalized excitations and emergent gauge fields. Here, we consider a gapless U(1) spin liquid with spinon Fermi pockets in two spatial dimensions. Such a state appears to be the most promising candidate to describe the exotic field-induced behavior observed in numerical simulations of the antiferromagnetic Kitaev honeycomb model. A similar such state may also be responsible for the recently-reported quantum oscillations of the thermal conductivity in the field-induced quantum paramagnetic phase of $\alpha$-RuCl$_3$. We consider the regime close to a Lifshitz transition, at which the spinon Fermi pockets shrink to small circles around high-symmetry points in the Brillouin zone. By employing renormalization group and mean-field arguments, we demonstrate that interactions lead to a gap opening in the spinon spectrum at low temperatures, which can be understood as a nesting instability of the spinon Fermi surface. This leads to proliferation of monopole operators of the emergent U(1) gauge field and confinement of spinons. While signatures of fractionalization may be observable at finite temperatures, the gapless U(1) spin liquid state with nested spinon Fermi pockets is ultimately unstable at low temperatures towards a conventional long-range-ordered ground state, such as a valence bond solid. Implications for Kitaev materials in external magnetic fields are discussed.