Quantum magnetisms in uniform triangular lattices Li2AMo3O8 (A = In, Sc)

TL;DR

This study investigates the magnetic ground states of two molecular triangular lattice compounds, revealing a conventional order in one and a quantum spin liquid-like state in the other, using muon spin rotation and neutron scattering techniques.

Contribution

It provides the first detailed comparison of magnetic behaviors in Li$_2$InMo$_3$O$_8$ and Li$_2$ScMo$_3$O$_8$, highlighting how structural and interaction anisotropies influence their ground states.

Findings

Li$_2$InMo$_3$O$_8$ shows 120° magnetic order with spin wave excitations.

Li$_2$ScMo$_3$O$_8$ exhibits short-range order and quantum fluctuations.

Different ground states are linked to structural and magnetic interaction anisotropies.

Abstract

Molecular based spin-1/2 triangular lattice systems such as LiZn$_2$Mo$_3$O$_8$ have attracted research interest. Distortions, defects, and intersite disorder are suppressed in such molecular-based magnets, and intrinsic geometrical frustration gives rise to unconventional and unexpected ground states. Li$_2$$A$Mo$_3$O$_8$ ($A$ = In or Sc) is such a compound where spin-1/2 Mo$_3$O$_{13}$ clusters in place of Mo ions form the uniform triangular lattice. Their ground states are different according to the $A$ site. Li$_2$InMo$_3$O$_8$ undergoes conventional $120^\circ$ long-range magnetic order below $T_\text{N}=12$~K whereas isomorphic Li$_2$ScMo$_3$O$_8$ exhibits no long-range magnetic order down to 0.5~K. Here, we report exotic magnetisms in Li$_2$InMo$_3$O$_8$ and Li$_2$ScMo$_3$O$_8$ investigated by muon spin rotation ($\mu$SR) and inelastic neutron scattering (INS) spectroscopies using polycrystalline samples. Li$_2$InMo$_3$O$_8$ and Li$_2$ScMo$_3$O$_8$ show completely different behaviors observed in both $\mu$SR and INS measurements, representing their different ground states. Li$_2$InMo$_3$O$_8$ exhibits spin wave excitation which is quantitatively described by the nearest neighbor anisotropic Heisenberg model based on the $120^\circ$ spin structure. In contrast, Li$_2$ScMo$_3$O$_8$ undergoes short-range magnetic order below 4~K with quantum-spin-liquid-like magnetic fluctuations down to the base temperature. Origin of the different ground states is discussed in terms of anisotropies of crystal structures and magnetic interactions.

Equations

$$A_{\text{ZF}}(t)$$

$$A_{\text{LF}}(t)=A_{\text{f}}\text{exp}(-{\lambda }_{\text{f}}t)+A_{\text{s}}\text{exp}(-{\lambda }_{\text{s}}t)+A_{\text{BG}}$$

$$\mathcal{H}=\alpha J\sum _{i,j}(S_i^x{S}_j^x+S_i^y{S}_j^y+\delta S_i^z{S}_j^z)$$

$$\alpha$$

$$\delta$$

$$\lambda =\frac{2({\gamma }_{\mu }{\delta }_{\mu }{)}^2\Gamma }{({\gamma }_{\mu }{H}_{\text{LF}}{)}^2+{\Gamma }^2}$$

$$\lambda \sim \frac{2({\gamma }_{\mu }{\delta }_{\mu }{)}^2}{\Gamma }.$$

Full text

I Introduction

When quantum spins are aligned on geometrically frustrated lattices, unusual ground state eventually emerges among energetically competed states QSL1 ; QSL2 ; Kagome1 . Two-dimensional (2D) spin-1/2 triangular lattice Heisenberg antiferromagnet (TLHAF) is a prototypical system of geometrically frustrated magnets. Theoretically, the ground states of 2D TLHAF with both quantum and classical spins are known to be so-called $120^{\circ}$ long-range order Classical1 ; Triangular1 ; Triangular2 ; Triangular3 . When perturbations such as the second nearest-neighbor interaction TQSL1 , ring exchange interaction TQSL2 , spatially anisotropic interactions TQSL3 , and randomness of the strength of the nearest-neighbor interaction TQSL4 are set in, the system undergoes a quantum spin liquid (QSL) ground state where the system does not show static long-range magnetic order but shows long-range entanglement and fractional excitations QSL1 ; QSL2 . Extensive experimental studies have also been conducted on spin-1/2 TLHAFs; the $120^{\circ}$ long-range magnetic order is reported in Ba3CoSb2O9 BCoSO1 ; BCoSO2 ; BCoSO3 whereas QSL state is proposed for the ground states of $\kappa$-(BEDT-TTF)2Cu2(CN)3 BEDTTTF0 ; BEDTTTF1 , EtMe3Sb[Pd(dmit)2]2 EtMe0 ; EtMe1 , YbMgGaO4 YMGO1 ; YMGO2 ; YMGO3 ; YMGO4 and 1T-TaS2 TaS21 . Furthermore, spin-1 TLHAF Ba3NiSb2O9 also shows QSL behaviors BNSO1 ; BNSO2 ; BNSO3 . QSL with spinon Fermi surface SFS0 ; SFS1 was proposed and succeeded in understanding the QSL behaviors in such compounds YMGO2 ; YMGO3 ; BNSO3 . However, experimental realization of the QSL ground state in spin-1/2 TLHAF systems is still limited and remains an intriguing pursuit.

Recently, cluster magnet LiZn2Mo3O8 has attracted considerable research interest as spin-1/2 TLHAF LZMO1 . Seven $4d$ electrons in a Mo3O13 cluster occupy their orbitals, resulting in one unpaired electron. Unpaired electron with spin $S=1/2$ remains in the total symmetry of the Mo3O13 cluster ($A_{1}$ irreducible representation) with equal contributions from all three Mo atoms, and network of the magnetic clusters forms a uniform triangular lattice in LiZn2Mo3O8. The dominant magnetic interaction between spin-1/2 Mo3O13 clusters is antiferromagnetic LZMO1 , yielding geometrical frustration. LiZn2Mo3O8 is therefore an ideal 2D spin-1/2 TLHAF system. Magnetic susceptibility and heat capacity measurements suggested that 2/3 of $S=1/2$ spins are quenched below 96 K, and condensed valence bond state (VBS) where resonance valence-bond states RVB1 ; RVB2 coexist with remnant paramagnetic spins is proposed for the possible ground state LZMO1 ; LZMO2 . Gapless spin excitations were reported by electron spin resonance LZMO2 , 7Li nuclear magnetic resonance (NMR) LZMO2 , muon spin rotation ($\mu$SR) LZMO2 , and inelastic neutron scattering (INS) LZMO3 measurements. Emergent honeycomb lattice is theoretically proposed for the origin of the condensed VBS LZMO4 . Recently, a 1/6-filled extended Hubbard model in an anisotropic kagome lattice is also proposed to account for the low temperature phase of LiZn2Mo3O8 ChenTheory . However, intersite disorder between Li+ and Zn2+ ions is reported LZMO1 ; LZMO2 , which may affect on the intrinsic magnetism in LiZn2Mo3O8.

New molecular based triangular lattice systems Li2$A$Mo3O8 where $A$ = In or Sc are of particular interest in this context LIMO1 ; LAMO1 . Li2$A$Mo3O8 crystallizes in a hexagonal structure $P6_{3}mc$, and no intersite disorder between Li+ and $A^{3+}$ sites exists (see Supplementary Information). As in LiZn2Mo3O8, spin-1/2 carrying Mo3O13 clusters are arranged on the structurally perfect triangular lattice separated by nonmagnetic Li and $A$ layers in both compounds as shown in Figs. 1(a) and 1(b). Susceptibility measurements of both compounds report that the dominant magnetic interactions are antiferromagnetic and the effective moments are $1.61\mu_{\text{B}}$ (In) and $1.65\mu_{\text{B}}$ (Sc), which are close to $p_{\text{eff}}=1.73\mu_{\text{B}}$ the ideal value for spin $S=1/2$. Spin-1/2 TLHAF is therefore realized in Li2$A$Mo3O8, whose ground states are however different from each other. In Li2InMo3O8, long-range magnetic order develops below $T_{\text{N}}=12$ K with Curie-Weiss temperature of $\Theta_{\text{CW}}=-242$ K, and 7Li NMR study suggests that the magnetic structure is the $120^{\circ}$ structure as described in Fig. 1(c). On the other hand, isostructural Li2ScMo3O8 shows no long-range magnetic order down to 0.5 K in spite of large Weiss temperature of $\Theta_{\text{CW}}=-127$ K. Instead, both magnetic susceptibility and heat capacity measurements indicate the development of short-range magnetic order below 10 K. Spin glass state is ruled out as the ground state of Li2ScMo3O8 since the magnetic susceptibility shows no splitting between zero-field-cooling and field-cooling processes LAMO1 . Low-temperature heat capacity measurements in Li2ScMo3O8 shows sizable $T$-linear term $\gamma_{\text{mag}}=35.7$ mJ/mol$\cdot$K2, which is similar to those of QSL candidates $\kappa$-(BEDT-TTF)2Cu2(CN)3 BEDTTTF1 , EtMe3Sb[Pd(dmit)2]2 EtMe1 , and Ba3CuSb2O9 BCuSO1 . Furthermore, different magnetic entropies between Li2ScMo3O8 and LiZn2Mo3O8 suggests that the ground state in Li2ScMo3O8 is QSL rather than condensed VBS. Because of easy access to two different ground states of spin-1/2 TLHAF, Li2$A$Mo3O8 is an intriguing system to investigate 2D spin-1/2 TLHAF. However, lack of microscopic measurements prevents us from fully understanding the ground states and dynamics of Li2$A$Mo3O8. In this paper, we investigate quantum magnetisms of polycrystalline Li2InMo3O8 and Li2ScMo3O8 by combination of $\mu$SR and time-of-flight (TOF) neutron scattering techniques.

II Experimental details

The preparation of polycrystalline Li2InMo308 (Li2ScMo3O8) was carried out by two steps LAMO1 . First, to synthesize a precursor Li2MoO4, a mixture with a ratio of MoO3 : Li2CO3 = 1:1 was ground, placed in an alumina crucible, and heated at 873 K for 24 hours in air; we repeated this step for three times. Then, a mixture having a ratio of In2O3 (Sc2O3) : Li2MoO4 : MoO3 : Mo = 0.5 : 1 : 0.84 : 1.16 was ground, pressed into a pellet, sealed in an evacuated quartz tube, heated at 923 K for 12 hours, and heated at 1198 K (1173 K) for 24 hours; we repeated this step for two times. Magnetization measurements were performed using a commercial superconducting quantum interference device (SQUID) magnetometer (Quantum Design Magnetic Property Measurement System, MPMS).

ZF- and LF-$\mu$SR experiments were performed using the Advanced Research Targeted Experimental Muon Instrument at the S1 line spectrometer (ARTEMIS) Muon1 with a conventional 4He flow cryostat and the D1 spectrometer Muon1 with a 3He-4He dilution refrigerator installed at Materials and Life Science Experimental Facility (MLF), Japan Proton Accelerator Research Complex (J-PARC). We used the VASP software VASP for DFT calculation and the DipElec program DipElec to calculate the local magnetic fields in Li2ScMo3O8.

TOF neutron scattering measurements were performed using the Fermi chopper spectrometer 4SEASONS at MLF, J-PARC SIKI1 . Frequencies of the Fermi chopper were 350 and 250 Hz for the In and Sc systems, resulting in the combinations of incident neutron energies of 11.9, 15.8, 22.0, and 32.7 meV, and 7.5, 10.3, 15.0, and 23.9 meV SIKI2 , respectively. A standard top-loading cryostat at 4SEASONS was used for the measurements on Li2InMo3O8, whereas a 4He refrigerator and a 3He cryostat were used for Li2ScMo3O8. Empty can was measured at corresponding temperatures, and then subtracted from raw data of Li2ScMo3O8. TOF data were visualized by software suite Utsusemi SIKI3 . Neutron scattering intensities are converted to the absolute unit using the incoherent scattering of each sample Abso after correction of the neutron absorption effect. Squared magnetic form factor of the Mo3O13 cluster LZMO3 and $\hbar\omega$-dependent energy resolution at 4SEASONS SIKI4 were included in the LSW calculations for Li2InMo3O8.

III Results and Discussion

Zero field- (ZF-) $\mu$SR time spectra of Li2InMo3O8 at several temperatures are shown in Fig. 2(a). The spectra show a damping at around 12 K, and spectral oscillations appear at lower temperatures. It is a direct evidence of the long-range magnetic order as reported in the earlier studies LIMO1 ; LAMO1 ; LAMO3 ; LAMO4 . Fourier transform of the spectrum at 3.33 K [see the inset of Fig. 2(a)] suggests that at least three different local fields are found in Li2InMo3O8, which is probably due to crystallographically inequivalent muon stopping sites indicated by our density functional theory (DFT) calculation (see Supplementary Fig. S3 for Li2ScMo3O8). The ZF-$\mu$SR spectra of Li2InMo3O8 are fitted by three cosine functions with transverse and longitudinal relaxations

[TABLE]

where $A_{n}$ and $A_{\text{BG}}$ are the positron decay asymmetries of each oscillation ($n=1\sim 3$) and background (mainly from a silver backing plate) components, $f_{n}$ is the precession frequency, $\phi$ is the initial phase, and $\lambda_{t}$ ($\lambda_{l}$) is the transverse (longitudinal) relaxation rate. Fitting result at each temperature is shown in Fig. 2(a). Local magnetic fields of 84.9(3), 103.1(2), and 151.5(5) G are extracted at 3.33 K, and these values are comparable in magnitude of local fields that are observed in spin-1/2 magnets Cup1 . Figure 2(c) shows temperature dependences of $f_{1}$, $f_{2}$, and $f_{3}$, representing that long-range magnetic order evolves in Li2InMo3O8 below $T_{\text{N}}=12$ K with the critical exponents $\beta\sim 0.33$.

In the meanwhile, ZF-$\mu$SR time spectrum of Li2ScMo3O8 at 0.07 K shows a highly damped oscillation with a pronounced reduction of the 1/3 tail as described in Fig. 2(b). To see the temperature evolution of the local fields in Li2ScMo3O8, the ZF-$\mu$SR spectrum are fitted by combination of transverse and longitudinal relaxations

[TABLE]

The fitting result at each temperature is plotted in Fig. 2(b), and temperature dependence of the local field $f_{\text{Sc}}$ is also plotted in Fig. 2(c). One can clearly see the temperature evolution of $f_{\text{Sc}}$ below 4 K with the critical exponent $\beta\sim 0.28$ which is similar to those of Li2InMo3O8. Therefore, magnetic nature of these compounds are essentially the same, but it should be noted that the ground state of Li2ScMo3O8 is short-range magnetic order by considering the strong damping of the oscillation below 4 K. The anomaly at $4$ K was also found in the temperature derivative of the magnetic susceptibility LAMO1 . Although the short-range magnetic order develops in Li2ScMo3O8 below 4 K, the spectrum shows a moderate tail over a long period of time, suggesting that spin fluctuation survives even at 0.07 K. To explicitly distinguish the spin fluctuation of the Mo3O13 cluster, we performed longitudinal field- (LF-) $\mu$SR measurements on Li2ScMo3O8 under longitudinal magnetic field ($H_{\text{LF}}$) of 1 kG. Figure 2(b) and its inset display a LF-$\mu$SR time spectrum measured at 0.07 K. $H_{\text{LF}}=1$ kG seems to be sufficient to quench (decouple) muon spin relaxations by both nuclear dipoles and the short-range ordered state. The characteristic LF-$\mu$SR spectrum of Li2ScMo3O8 at 0.07 K was fitted by the following equation

[TABLE]

where $A_{\text{f}}$ and $A_{\text{s}}$ are asymmetries of fast ($\lambda_{\text{f}}$) and slow ($\lambda_{\text{s}}$) relaxation components, respectively ($A_{\text{f}}+A_{\text{s}}=0.16$), and $A_{\text{BG}}$ is the background asymmetry ($A_{\text{BG}}=0.03$). The fitting results are described by the solid lines in Fig. 2(b) and its inset. We also fit LF-$\mu$SR time spectra under $H_{\text{LF}}=1$ kG at several temperatures, and obtained temperature dependences of $\lambda_{\text{f}}$ and $\lambda_{\text{s}}$ are plotted in Fig. 2(d). $\lambda_{\text{f}}$ shows a rapid relaxation with relative signal amplitude of $\sim 3$%. It mainly corresponds to the remnant signal from the short-range ordered state since $\lambda_{\text{f}}$ exhibits a steep increase at 4 K as temperature goes down. On the other hand, $\lambda_{\text{s}}$ shows a slow relaxation with two orders of magnitude less than $\lambda_{\text{f}}$, which is related to the intrinsic spin fluctuation of the Mo3O13 cluster. Remarkably, temperature dependence of $\lambda_{\text{s}}$ shows a temperature-independent plateau below 1 K and converges into the finite value of $\sim 0.002$ $\mu$s*-1* which is very close to that of triangular lattice QSL 1T-TaS2 ($\lambda=0.0023$ $\mu$s*-1* at 0.07 K) TaS21 . Indeed, such low-temperature plateau behaviors of muon relaxation rate is common feature in the TLHAF QSL candidates YMGO2 ; BNSO2 , which will be discussed again. To obtain complementary information to our $\mu$SR results on Li2$A$Mo3O8, TOF neutron scattering measurements were also conducted.

Elastic neutron scattering spectra of Li2InMo3O8 below and above $T_{\text{N}}$ are shown in Fig. 3(e). A magnetic Bragg peak appears at momentum transfer $Q=0.719(1)$ $\text{\AA}^{-1}$ below $T_{\text{N}}$. The $Q$ position corresponds to $(1/3,1/3,0)$, indicating the $120^{\circ}$ magnetic structure consistent with the previous 7Li-NMR measurements LAMO1 . By comparing the intensity of the magnetic peak with those of nuclear Bragg peaks, the ordered moment at 4.6 K is estimated to be $0.51(3)\mu_{\text{B}}$. Theoretically, the magnetic moment is reduced by about 59% for the spin-1/2 TLHAF Triangular2 , which is close to the observed ordered moment (reduced by 49% assuming $g=2$). The reduced moment originates in a combination of geometrical frustration and quantum fluctuation. Neutron scattering intensity ($I$) map from Li2InMo3O8 as a function of $Q$ and energy transfer ($\hbar\omega$) at 4.5 K ($<T_{\text{N}}$) is shown in Fig. 3(a). Dispersive excitation centered at the magnetic zone center $(1/3,1/3,0)$ was observed. Because of the $Q$ position, the excitation is assigned to be the spin wave excitation in the long-range magnetic ordered state. Energy spectrum at the magnetic zone center exhibits a substantial peak at $\hbar\omega=2.08(3)$ meV as shown in Fig. 3(f). This result claims that one branch (or some branches) of the spin wave excitation has spin gap at the magnetic zone center due to the magnetic anisotropy. On the other hand, magnetic signals at the magnetic zone center become quasielastic above $T_{\text{N}}$ as shown in Figs. 3(b) and 3(f). Therefore, the gap-like excitation is a characteristic feature of the long-range magnetic ordered state. To observe the whole structure of the spin wave excitation at 4.5 K, $I(Q,\hbar\omega)$ map using higher $E_{\text{i}}$ is presented in Fig. 3(c). The spin wave excitation survives up to $\sim 9$ meV. $Q$ dependences of the spin wave intensities at various $\hbar\omega$s are plotted in Fig. 3(g). The spectra are asymmetric at $\hbar\omega>3.0$ meV, and the peak shifts to lower $Q$ at higher $\hbar\omega$. This result suggests that the squared magnetic form factor ($|F(Q)|^{2}$) of the Mo3O13 cluster decreases quickly and is negligible at high $Q$, representing the unpaired electron with equal contributions from all three Mo atoms in Li2InMo3O8.

For quantitative analysis on the spin wave excitation in Li2InMo3O8, semi-classical linear spin wave (LSW) analysis was performed considering the $120^{\circ}$ spin structure on the spin-1/2 2D Mo3O13-based triangular lattice [Fig. 1(c)]. The gap-like excitation at the magnetic zone center in the long-range magnetic ordered state is also observed in the other spin-1/2 triangular lattice system Ba3CoSb2O9 BCoSO2 , and the peak energy ($E_{0}$) roughly scales with $T_{\text{N}}$ in these compounds: $E_{0}=0.65$ meV and $T_{\text{N}}=3.8$ K in Ba3CoSb2O9 BCoSO2 whereas $E_{0}=$2.08 meV and $T_{\text{N}}=12$ K in Li2InMo3O8. This suggests that the origin of the gap-like excitation in Li2InMo3O8 is the same as that in Ba3CoSb2O9 BCoSO2 . Therefore, as in Ba3CoSb2O9 BCoSO1 ; BCoSO2 ; BCoSO3 , the nearest-neighbor anisotropic exchange interaction was considered as the model Hamiltonian for Li2InMo3O8

[TABLE]

where $\alpha$, $J$, and $\delta$ represent the renormalization factor, the nearest neighbor exchange coupling constant, and the anisotropic factor. $J$ was fixed to 112 K determined by the magnetic susceptibility measurement LAMO1 . By fitting the calculated powder-averaged $Q$ dependences to the experimental results at different $\hbar\omega$s ($2\sim 7.5$ meV) simultaneously, optimum parameters were yielded

[TABLE]

Fitting results together with the experimental results are shown in Fig. 3(g), and calculated LSW $I(Q,\hbar\omega)$ map is also shown in Fig. 3(d). Satisfactory agreements with calculation and experiment were confirmed. Obtained $\alpha$ is smaller than 1, indicating a negative quantum renormalization effect theoretically proposed for 2D spin-1/2 TLHAF Reno1 ; Reno2 ; Reno3 . Similar negative quantum renormalization effect ($\alpha\sim 0.65$) was also reported in Ba3CoSb2O9 BCoSO3 . Therefore, observed magnetic excitations of Li2InMo3O8 in the accessible $(Q,\hbar\omega)$ region are well understood by the semi-classical LSW theory assuming the $120^{\circ}$ magnetic structure on the spin-1/2 Mo3O13 triangular lattice.

In contrast to Li2InMo3O8, no magnetic Bragg peak evolves in the elastic channel down to 0.3 K in Li2ScMo3O8 as plotted in Fig. 4(c), in agreement with our $\mu$SR results. On the other hand, diffuse scattering expected for the short-range order is not observed in our neutron measurements. Strong incoherent scattering may smear out such magnetic diffuse scattering in Li2ScMo3O8. Figure 4(a) depicts $I(Q,\hbar\omega)$ map at 0.3 K. Clear diffuse scattering was observed in the inelastic channel. Although both magnetic excitations in Li2InMo3O8 and Li2ScMo3O8 are centered at $Q\sim 0.7$ $\text{\AA}^{-1}$ [Figs. 3(a) and 4(a)], the overall structures are different, representing their different ground states. In Li2ScMo3O8, steep continuum excitation was observed. The $Q$ dependences of the magnetic excitations are invariant in the different energy windows as shown in Fig. 4(d). Steep continuum excitation, or spinon continuum, is the common feature of the magnetic excitations in the QSL candidates Kagome1 ; YMGO3 ; YMGO4 ; BNSO3 ; Kape1 . $I(Q,\hbar\omega)$ map at high temperature (22 K) is also shown in Fig. 4(b). Although overall magnetic fluctuation at 22 K is similar to that at 0.3 K, there are some differences. Scattering intensity decreases at 22 K. In addition, as shown in Fig. 4(e), spectrum weight of the $Q$ dependence at 2 meV slightly shifts to $Q=0$ at high temperature, which is also observed in other QSL candidates YMGO4 ; BNSO3 ; Kape1 .

To investigate in more detail the characteristic energy (or time) scale of the steep continuum in Li2ScMo3O8, the dynamical spin susceptibilities $\chi"(\hbar\omega)=[1-\text{exp}(-\hbar\omega/k_{\text{B}}T)]/|F(Q)|^{2}I(\hbar\omega)$ at $Q=[0.6,0.8]$ $\text{\AA}^{-1}$ where the magnetic signal is maximal are plotted for different temperatures in Fig. 4(f). The spectra are well fitted by the quasielastic Lorentzian $\chi"(\hbar\omega)=\chi^{\prime}\hbar\omega\Gamma/[(\hbar\omega)^{2}+\Gamma^{2}]$ where $\chi^{\prime}$ is the static susceptibility and $\Gamma$ the spin relaxation rate [or peak position of $\chi"(\hbar\omega)$]. The temperature dependences of the resulting parameters are shown in Fig. 4(g). Upon decreasing temperature, $\Gamma$ decreases while $\chi^{\prime}$ increases. Contrary to the conventional long-range ordered magnets, no divergent behavior was observed in the temperature dependences of $\chi^{\prime}$ and $\Gamma$. It should be noted that $\chi^{\prime}$ scales with bulk magnetic susceptibility $\chi_{\text{bulk}}$ over the temperature range of $3\leq T\leq 40$ K [see solid line in Fig. 4(g)] and $\Gamma$ is also scaled by the muon relaxation rate $\lambda_{\text{s}}$ as discussed below. These fittings also extract two important features of the steep continuum in Li2ScMo3O8: (1) the magnetic excitation is gapless consistent with the heat capacity measurement LAMO1 and (2) the dynamical spin susceptibility extends from the elastic channel up to at least 9.5 meV which is about $1.6J$ where $J$ ($=67$ K) is determined by the magnetic susceptibility measurement LAMO1 .

Complementary analysis of $\mu$SR and INS results enables us to exclusively clarify the quantum fluctuations in Li2ScMo3O8. Muon spin relaxation rate $\lambda_{\text{s}}$ in Fig. 2(d) is related to the spin relaxation rate of the magnetic fluctuation $\Gamma$ in Fig. 4(g) on the basis of following Redfield’s formula Redfield

[TABLE]

where $\gamma_{\mu}$ and $\delta_{\mu}$ are the gyromagnetic ratio of muon (=$2\pi\times 135.54$ MHz/T) and average distribution of local magnetic fields at muon sites. We performed electrostatic potential calculations using a point-charge model DipElec and estimated $\delta_{\mu}=204.8$ G for Li2ScMo3O8 (see Supplementary Information). Since $H_{\text{LF}}$ ($=1$ kG $=8.5\times 10^{8}$ Hz) is much smaller than $\Gamma$ ($=2.7$ meV $=6.5\times 10^{11}$ Hz at 0.3 K) in Li2ScMo3O8, Eq. (6) is reformulated as

[TABLE]

We plotted calculated temperature dependence of $\lambda$ using $\Gamma$ obtained by our INS measurements and compared with $\lambda_{\text{s}}$ obtained by our LF-$\mu$SR measurements [see solid line for calculation and circles for $\mu$SR results in Fig. 2(d)]. Quantitative agreement can be seen; the anomaly around 4 K is artificial feature owing to $\lambda_{\text{f}}$. Therefore, both $\mu$SR and INS measurements exhibit that quantum fluctuations persist at the lowest measured temperature. As mentioned above, such low-temperature plateaus of the relaxation rates were widely observed in the triangular-lattice YMGO2 ; BNSO2 and kagome-lattice Kape1 ; Kagome2 ; Kagome3 ; Kagome4 ; Kagome5 QSL candidates.

To account for the QSL-like excitations in Li2ScMo3O8, we now consider the spinon Fermi surface QSL model. In Li2ScMo3O8, no static long-range order was detected even down to 0.07 K [Figs. 2(b) and 4(c)]. Alternatively, gapless continuum in Li2ScMo3O8 was observed at $Q=0.726(4)$ $\text{\AA}^{-1}$ corresponding to the $(1/3,1/3,0)$ position [Figs. 4(a), 4(d), and 4(f)]. Moreover, both $\lambda_{\text{s}}$ and $\Gamma$ exhibit temperature-independent plateaus at low temperature [Figs. 2(d) and 4(g)]. These features are well explained by QSL with spinon Fermi surface SFS0 ; SFS1 . As discussed in earlier works YMGO2 ; YMGO3 ; BNSO3 , the spinon Fermi surface QSL model on the spin-1/2 TLHAF expects that (1) absence of static long-range magnetic order, (2) muon spin relaxation rate approach a finite value as temperature approaches zero, (3) magnetic excitation is gapless continuum, and (4) $\chi"(Q,\hbar\omega)$ shows the maximum intensity at the corner of the 2D Brillouin zone [e.g. $(1/3,1/3,0)$]. All observed features of the magnetic fluctuation in Li2ScMo3O8 can be well described by the spinon Fermi surface QSL model. Although the second peak of the spinon continuum in Ba3NiSb2O9 was also observed at $(2/3,2/3,0)$ BNSO3 , the second peak in Li2ScMo3O8 was not detected at $(2/3,2/3,0)$ corresponding to $Q=1.45$ $\text{\AA}^{-1}$ as shown in Fig. 4(d) because of the quick decay of the squared magnetic form factor of the Mo3O13 cluster LZMO3 . By performing complementary analysis on $\mu$SR and INS results, we conclude that Li2ScMo3O8 undergoes the short-range magnetic order below 4 K with the QSL-like fluctuations which persist down to the lowest temperature.

We compare the Mo3O13-cluster-based triangular lattice antiferromagnets, Li2$A$Mo3O8 and LiZn2Mo3O8, in line with the recent theory by Chen $et$ $al$ ChenTheory . They proposed a 1/6-filled Hubbard model on an anisotropic kagome lattice with the nearest-neighbor electron hopping and repulsions ChenTheory to account for the magnetism in LiZn2Mo3O8 LZMO1 . Electron is fractionalized into charged boson and spin-carring spinons; plaquette charge order emerges as the charge ground state and the spin degree of freedom can be then described by $U(1)$ QSL with spinon Fermi surface, which can explain the unusual magnetic susceptibility in LiZn2Mo3O8 LZMO1 . For comparison with different compounds, they introduce a phenomenological parameter $\xi$ to characterize the anisotropy of the Mo kagome lattice: $\xi=d_{\text{inter}}/d_{\text{intra}}$ where $d_{\text{intra}}$ ($d_{\text{inter}}$) is the intracluster (intercluster) Mo-Mo bond length. Large anisotropy $\xi$ tends to suppress charge fluctuations between clusters leading to the $120^{\circ}$ long-range magnetic order whereas small anisotropy $\xi$ corresponds to large charge fluctuation generating the $U(1)$ QSL with spinon Fermi surface. Using the structural parameters summarized in Supplementary Information, we estimated $\xi$ as 1.271, 1.269, and 1.258 for Li2InMo3O8, Li2ScMo3O8, and LiZn2Mo3O8 LZMO1 , respectively. The phenomenological parameter $\xi$ explains the different ground states between the $120^{\circ}$ long-range magnetic order in Li2InMo3O8 and the condensed VBS in LiZn2Mo3O8. However, the $\xi$ values of Li2InMo3O8 and Li2ScMo3O8 are very close to each other in spite of their different ground states. Nevertheless, 115In and 45Sc NMR measurements on Li2$A$Mo3O8 reported that charge fluctuation in Li2ScMo3O8 is 2.6 times larger than that in Li2InMo3O8 LAMO2 , and the difference between charge fluctuations can qualitatively explain the different ground states of Li2InMo3O8 and Li2ScMo3O8. Therefore, the anisotropic parameter for the Mo kagome lattice, $\xi$, is too simplified to explain the different ground states in Li2$A$Mo3O8, and more detailed parameter is required for Li2$A$Mo3O8.

We also compare the magnetic excitations in Li2InMo3O8 and Li2ScMo3O8 to discuss the origin of the different ground states. Although both magnetic excitations in Li2InMo3O8 and Li2ScMo3O8 center at $Q\sim 0.72$ $\text{\AA}^{-1}$, low-energy magnetic excitations show opposite behaviors. The magnetic excitation at the magnetic zone center in Li2InMo3O8 clearly exhibits the peak at 2.08(3) meV [Figs. 3(a) and 3(f)]. Our LSW analysis suggests that the anisotropic exchange interaction is necessary to reproduce the peak. Meanwhile, the gapless magnetic excitation in Li2ScMo3O8 indicates that magnetic anisotropy is negligibly small in Li2ScMo3O8 [Figs. 4(a) and 4(f)]. Thus, the difference in the magnetic anisotropy is another possibility of the origin of the different ground states in Li2$A$Mo3O8. In fact, the gap-like excitation was observed in the long-range ordered state of Ba3CoSb2O9 BCoSO2 ; BCoSO3 , whereas the gapless magnetic excitations in the QSL systems YbMgGaO4 YMGO3 ; YMGO4 and Ba3NiSb2O9 BNSO3 . INS measurements on magnetic excitations in the substitution system Li2(In1-xScx)Mo3O8 LAMO4 are effective to further elucidate the origin of different magnetic ground states, which is left for future work.

IV Conclusion

We performed a comprehensive study on the quantum magnetisms in the Mo3O13-cluster-based spin-1/2 triangular lattice antiferromagnets, Li2InMo3O8 and Li2ScMo3O8 by means of $\mu$SR and TOF neutron scattering techniques. Spin wave excitation in Li2InMo3O8 was well described by the nearest neighbor anisotropic Heisenberg model based on the $120^{\circ}$ spin structure. Li2ScMo3O8 exhibits the short-range magnetic order below 4 K with the QSL-like fluctuations which persist down to the lowest temperature. The origin of the different magnetic ground states in Li2$A$Mo3O8 is discussed in terms of anisotropies of crystal structures and magnetic interactions.

Acknowledgements

We acknowledge Seiko Ohira-Kawamura for helpful discussion and Hua Li for DFT calculations. We also thank technical supports from the MLF sample environment team. Magnetic susceptibility measurements were performed at the CROSS user laboratories. The synchrotron X-ray diffraction experiments were conducted at BL5S2 of Aichi Synchrotron Radiation Center, Aichi Science and Technology Foundation, Aichi, Japan (Proposal Nos. 201803046 and 201803047). The $\mu$SR measurements at the S1 and D1 beamlines were conducted under the user program with proposal number 2017B0033. The proposal numbers for the TOF neutron scattering experiments at 4SEASONS were 2016I0001, 2017A0004, 2017B0030, and 2018I0001. The DFT calculations were supported by the Condensed Matter Research Center, Institute of Materials Structure Science, KEK, and the KEK Large Scale Simulation Program (Nos. 15/16-07 and 16/17-18). This work was partially supported by JSPS KAKENHI Grant Numbers JP17K14349 and JP18K03529, and the Cooperative Research Program of “Network Joint Research Center for Materials and Devices” (20181072).