When Could Abelian Fractional Topological Insulators Exist in Twisted MoTe$_2$ (and Other Systems)

TL;DR

This study investigates the conditions under which fractional topological insulators can exist in twisted bilayer MoTe$_2$, highlighting the importance of interaction tuning, and explores the challenges and potential engineering strategies for realizing FTIs in such systems.

Contribution

The paper provides a comprehensive analysis of the interaction conditions necessary for stabilizing FTIs in twisted MoTe$_2$, combining exact diagonalization with realistic modeling and proposing engineering routes.

Findings

FTI stability requires suppression of short-range repulsion.

FTI phase observed at $ u=-4/3$ with additional short-range attraction.

FTIs unlikely at low angles for certain fillings due to band structure effects.

Abstract

Using comprehensive exact diagonalization calculations on $\theta \approx 3.7 ^{\circ}$ twisted bilayer MoTe$_2$ ($t$MoTe$_2$), as well as idealized Landau level models also relevant for lower $\theta$, we extract general principles for engineering fractional topological insulators (FTIs) in realistic situations. First, in a Landau level setup at $\nu=1/3+1/3$, we investigate what features of the interaction destroy an FTI. For both pseudopotential interactions and realistic screened Coulomb interactions, we find that sufficient suppression of the short-range repulsion is needed for stabilizing an FTI. We then study $\theta \approx 3.7 ^{\circ}$ $t$MoTe$_2$ with realistic band-mixing and anisotropic non-local dielectric screening. Our finite-size calculations only find an FTI phase at $\nu=-4/3$ in the presence of a significant additional short-range attraction $g$ that acts to counter the Coulomb repulsion at short distances. We discuss how further finite-size drifts, dielectric engineering, Landau level character, and band-mixing effects may reduce the required value of $g$ closer towards the experimentally relevant conditions of $t$MoTe$_2$. Projective calculations into the $n=1$ Landau level, which resembles the second valence band of $\theta\simeq 2.1^\circ$ $t$MoTe$_2$, do not yield FTIs for any $g$, suggesting that FTIs at low-angle $t$MoTe$_2$ for $\nu=-8/3$ and $-10/3$ may be unlikely. While our study highlights the challenges, at least for the fillings considered, to obtaining an FTI with transport plateaus, even in large-angle $t$MoTe$_2$ where fractional Chern insulators are experimentally established, we also provide potential sample-engineering routes to improve the stability of FTI phases.