Spin-orbit physics of j=1/2 Mott insulators on the triangular lattice

TL;DR

This paper investigates the spin-orbit physics of j=1/2 Mott insulators on the triangular lattice, revealing how Kitaev interactions destabilize conventional order and lead to novel vortex and nematic phases.

Contribution

It demonstrates that the triangular Heisenberg-Kitaev model captures key features of real materials like Ba$_3$IrTi$_2$O$_9$, including the emergence of a $ ext{Z}_2$-vortex crystal phase.

Findings

Infinitesimal Kitaev exchange destabilizes 120° order.

Extended $ ext{Z}_2$-vortex crystal phase forms.

Phase diagram includes ordered and nematic phases.

Abstract

The Heisenberg-Kitaev (HK) model on the triangular lattice is conceptually interesting for its interplay of geometric and exchange frustration. HK models are also thought to capture the essential physics of the spin-orbital entanglement in effective $j=1/2$ Mott insulators studied in the context of various 5d transition metal oxides. Here we argue that the recently synthesized Ba$_3$IrTi$_2$O$_9$ is a prime candidate for a microscopic realization of the triangular HK model. We establish that an infinitesimal Kitaev exchange destabilizes the 120$^\circ$ order of the quantum Heisenberg model and results in the formation of an extended $\mathbb{Z}_2$-vortex crystal phase in the parameter regime most likely relevant to the real material. Using a combination of analytical and numerical techniques we map out the entire phase diagram of the model, which further includes various ordered phases as well as an extended nematic phase around the antiferromagnetic Kitaev point.