Magnetic Signatures of a Putative Fractional Topological Insulator in Twisted MoTe2

TL;DR

This study provides experimental evidence for a fractional topological insulator state in twisted MoTe2, characterized by unique magnetic signatures and close energetic competition with valley-polarized states.

Contribution

First experimental observation of magnetic signatures consistent with a fractional topological insulator in twisted MoTe2, supported by theoretical modeling.

Findings

Correlated state at v=-4/3 shows Ising antiferromagnet behavior.

Partial valley polarization occurs at very low magnetic fields (~2-6 mT).

FTI state is nearly degenerate with valley-polarized states in energy.

Abstract

The interplay among electronic correlation, topology, and time-reversal-symmetry (TRS) often leads to exotic quantum states of matter, as highlighted by the discoveries of fractional Chern insulators (FCIs) in twisted bilayer MoTe2 (tMoTe2). Among the FCIs in tMoTe2, the most robust is at a hole filling factor of v=-2/3 per moir\'e unit cell. Here, employing pump-probe circular dichroism (CD) measurement on tMoTe2 at twist angles (3.9 and 3.7 degrees), we show that a correlated state at v =-4/3 exhibits an unusual Ising antiferromagnet behavior. The v =-4/3 state with no net magnetization undergoes first order phase transitions at extremely low magnetic fields of ~ 2-6 mT to partially valley polarized (PVP) states. This behavior is notably absent for all other correlated states in tMoTe2 and also disappears for v =-4/3 at higher or lower twist angles (4.0 or 3.3 degree). The observed magnetic signature is consistent with a theoretically proposed fractional topological insulator (FTI), consisting of two copies of v =-2/3 FCIs with opposite chirality in the two K valleys. The experimental results are supported by interacting continuum model calculations that reveal the extreme closeness in energy ( < 1 meV) between the putative FTI and PVP states. Our findings present a candidate FTI with TRS and call for advanced transport and imaging measurements to establish the quantized helical edge modes.