Fractional Chern Insulators vs. Non-Magnetic States in Twisted Bilayer MoTe$_2$

TL;DR

This study investigates the emergence of fractional Chern insulators and non-magnetic states in twisted bilayer MoTe2, using advanced calculations to understand experimental observations and the role of remote bands in magnetic ordering.

Contribution

The paper demonstrates that including remote bands and specific parameter choices can reproduce experimental phenomena in twisted bilayer MoTe2, clarifying the conditions for various topological states.

Findings

Remote bands are crucial for understanding magnetic orders.

Parameters from recent literature can nearly reproduce experimental states.

Higher dielectric constants are needed for accurate modeling.

Abstract

Fractionally filled Chern bands with strong interactions may give rise to fractional Chern insulator (FCI) states, the zero-field analogue of the fractional quantum Hall effect. Recent experiments have demonstrated the existence of FCIs in twisted bilayer MoTe$_2$ without external magnetic fields -- most robust at $\nu=-2/3$ -- as well as Chern insulators (CIs) at $\nu=-1$. Although the appearance of both of these states is theoretically natural in an interacting topological system, experiments repeatedly observe nonmagnetic states (lacking FCIs) at $\nu=-1/3$ and $-4/3$, a puzzling result which has not been fully theoretically explained. In this work, we perform Hartree-Fock and exact diagonalization calculations to test whether the standard MoTe$_2$ moir\'e model with the (greatly varying) parameter values available in the literature can reproduce the non-magnetic states at $\nu=-1/3$ and $-4/3$ in unison with the FCI at $\nu=-2/3$ and CI state at $\nu = -1$. We focus on the experimentally relevant twist angles and, crucially, include remote bands. We find that the parameters proposed in [Wang et al. (2023)] can nearly capture the experimental phenomena at $\nu=-1/3,-2/3,-1,-4/3$ simultaneously, though the predicted ground states at $\nu=-1/3$ are still mostly fully-spin-polarized and a larger dielectric constant $\epsilon>10$ than is typical of hexagonal boron nitride (h-BN) substrate $\epsilon\sim 6$ is required. Our results show the importance of remote bands in identifying the competing magnetic orders and lay the groundwork for further study of the realistic phase diagram.