New ordered phase in geometrically frustrated generalized $XY$ model

TL;DR

This study explores the critical properties of a frustrated generalized XY model on a triangular lattice, revealing a rich phase diagram with three quasi-long-range ordered phases and a novel canted antiferromagnetic phase due to competing interactions.

Contribution

It introduces a new phase diagram for the generalized XY model with AFM and AN3 couplings, highlighting the emergence of a complex canted AFM phase and its critical properties.

Findings

Discovery of three QLRO phases including a new canted AFM phase.

Identification of phase transition universality classes as weak Ising and weak three-state Potts.

Observation of simultaneous vanishing of chiralities at non-Ising critical points.

Abstract

Critical properties of a geometrically frustrated generalized $XY$ model with antiferromagnetic (AFM) and third-order antinematic (AN3) couplings on a triangular lattice are studied by Monte Carlo simulation. It is found that such a generalization leads to a phase diagram consisting of three different quasi-long-range ordered (QLRO) phases. Compared to the model with the second-order antinematic (AN2) coupling, besides the AFM and AN3 phases which appear in the limits of relatively strong AFM and AN3 interactions, respectively, it includes an additional complex canted antiferromagnetic (CAFM) phase. It emerges at lower temperatures, wedged between the AFM and AN3 phases, as a result of the competition between the AFM and AN3 couplings, which is absent in the model with the AN2 coupling. The AFM-CAFM and AN3-CAFM phase transitions are concluded to belong to the weak Ising and weak three-state Potts universality classes, respectively. Additionally, all three QLRO phases also feature true LRO of the standard and generalized chiralities, which both vanish simultaneously at second-order phase transitions with non-Ising critical exponents and the critical temperatures slightly higher than the magnetic and nematic order-disorder transition temperatures.