Indirect exchange interaction between magnetic impurities in one-dimensional gapped helical states

TL;DR

This paper theoretically studies the indirect exchange interaction between magnetic impurities in one-dimensional gapped helical states, revealing how energy gap and Fermi energy influence the interaction's decay, oscillation, and components.

Contribution

It provides analytical and theoretical insights into the exchange interaction in gapped helical states, including the effects of electron-electron interactions and energy gap variations.

Findings

Interaction includes Heisenberg, Dzyaloshinsky-Moriya, and Ising terms.

Exponential decay of interaction when Fermi level is inside the gap.

Oscillatory behavior of range functions varies with energy gap and Fermi energy.

Abstract

We investigate theoretically indirect exchange interaction between magnetic impurities mediated by one-dimensional gapped helical states. Such states, containing massive Dirac fermions, may be realized on the edge of a two-dimensional topological insulator when time-reversal symmetry is weakly broken. We find that the indirect exchange interaction consists of Heisenberg, Dzyaloshinsky-Moriya, in-plane and out-of-plane Ising terms. These terms decay exponentially when Fermi level lies inside the bandgap whereas the Dzyaloshinsky-Moriya term has smallest amplitude. Outside the energy gap the massive helical states modify oscillatory behaviors of the range functions so that their periods decrease near the edge of band in terms of energy gap or Fermi energy. In addition, the out-of-plane Ising term vanishes in the case of zero-gap structure but its oscillation amplitude increases versus energy gap and decreases as a function of Fermi energy whereas the oscillation amplitudes of other components remain constant. Analytical results are also obtained for subgap and over gap regimes. Furthermore, the effects of electron-electron interactions are analyzed.

Equations

$$H_0(k)$$

$${\epsilon }_{\lambda k}$$

$$H_{\text{int}}=J\sum _{j=1,2}{\mathbf{S}}_j\cdot \mathbf{s}(R_j),$$

$$H_{\text{RKKY}}=$$

$$G^0(R,{ϵ}^{+})$$

$$G^0(k,{ϵ}^{+})=\frac{{ϵ}^{+}+H_0(k)}{({ϵ}^{+}{)}^2-(v_{f}k{)}^2-(\Delta {)}^2},$$

$$G^0(\pm R,{ϵ}^{+})$$

$$H_{\text{RKKY}}=$$

$$F_1$$

$$F_2$$

$$F_3$$

$$F_4$$

$$F_1$$

$$F_2$$

$$F_3$$

$$F_4$$

$$Ci(x)\approx \frac{1}{x}\left[\sin (x)-\frac{\cos (x)}{x}\right],$$

$$F_1$$

$$F_2$$

$$F_3$$

$$F_4$$

$${\mathcal{H}}_0=-i{v}_f^{\prime }\left[{\psi }_R^{†}(x){\partial }_x{\psi }_R(x)-{\psi }_L^{†}(x){\partial }_x{\psi }_L(x)\right],$$

$$\psi (x)=\sum _r{\chi }_r(r{k}_f){\psi }_r(x)e^{ir{k}_{f}x},$$

$$\displaystyle\chi_{r}(k)=\frac{1}{\sqrt{2}}\left(\begin{array}[]{c}re^{i\theta_{k}}\\ e^{-i\theta_{k}}\\ \end{array}\right),$$

$${\psi }_r(x)$$

$${\mathcal{H}}_{e-e}$$

$$\mathcal{H}$$

$$\Phi ={\varphi }_R+{\varphi }_L,\Theta ={\varphi }_R-{\varphi }_L.$$

$$v$$

$$K$$

$$\displaystyle\chi_{r}^{\dagger}(rk_{f})\bm{\sigma}\chi_{r^{\prime}}(r^{\prime}k_{f})\!=\!\!\left(\!\!\!\begin{array}[]{c}r\delta_{r,r^{\prime}}\cos(2\theta_{k_{f}}\!)\\ -ir\delta_{r,-r^{\prime}}\!+\!\delta_{r,r^{\prime}}\sin(2\theta_{k_{f}}\!)\\ -\delta_{r,-r^{\prime}}\cos(2\theta_{k_{f}}\!)\\ \end{array}\!\!\!\right),$$

$$s^x$$

$$s^y$$

$$s^z$$

$${\mathcal{S}}^{ab}(\tau ,R)=⟨s^a(\tau ,R)s^b(0,0)⟩,$$

$${\mathcal{S}}^{xx}(\tau ,R)$$

Full text

I Introduction

Quantum nature of phenomena becomes more pronounced in low dimensional systems Giamarchi . Especially, in recent years much attention in condensed matter physics has been paid to two-dimensional systems 2Dcrystal due to synthesis of graphene Graphene and related materials 2DDirac with Dirac-like dispersion relation. Materials with linearly dispersing spectrum have attracted considerable amount of attention due to providing new opportunities for both fundamental aspects and potential applications. Moreover, the existence of Weyl fermions has been predicted WSMRev1 and reported WSMRev2 in Weyl semimetals that are three-dimensional analogs of graphene. However, one-dimensional (1D) Dirac materials provide a promising alternative to studying exotic characteristics of chiral quantum states. It may raise an interesting perspective in a variety of contexts. In particular, since features of indirect exchange interactions between magnetic impurities depend on both the dimensionality and the band dispersion of host materials, one may expect a unique behavior in 1D Dirac materials.

The study of indirect exchange interactions between magnetic impurities in one-, two- and three-dimensional materials with parabolic band structure goes back to the seminal works by Ruderman, Kittel, Kasuya, and Yosida (RKKY) RKKYPertu1 ; RKKYPertu2 ; RKKYPertu3 . This interaction, known also as RKKY interaction, has been investigated in new systems whose band structures disperse linearly with chiral feature, for instance, graphene RKKYGraphene1 ; RKKYGraphene2 ; RKKYGraphene3 and phosphorene RKKYphosphorene1 ; RKKYphosphorene2 in two dimensions and Weyl/Dirac semimetals RKKYWeyl1 ; RKKYWeyl2 ; RKKYWeyl3 ; RKKYWeyl4 in three dimensions. Moreover, exotic spin textures due to spin-momentum locking on the surface of three-dimensional 3DTIRKKY0 ; 3DTIRKKY1 ; 3DTIRKKY2 ; 3DTIRKKY3 ; 3DTIRKKY4 ; 3DTIRKKY5 ; 3DTIRKKYGap and at the edge of two-dimensional 2DTIRKKY1 ; 2DTIRKKY2 ; 2DTIRKKY3 ; RKKYeeHelicLiq ; RKKYKondeeHelicLiq topological insulators have been predicted as another example of nontrivial RKKY interaction. Another work has discussed RKKY interaction near the helical edge by taking into account of both bulk and edge modes 2DnearEdge . Furthermore, it has been shown that magnetic exchange interaction is adjustable by a vertical bias in thick films RKKYThickThinFilm or a thin slab of topological insulators RKKYThickThinFilm ; RKKYthinFilm . Recently, the RKKY interaction mediated by surface states 2DRKKYTS and helical Majorana edge states MajoTScRKKY1 ; MajoTScRKKY2 in a topological superconductor was also investigated.

Several studies have been performed on magnetic topological insulators DopgapSize1 elucidating magnetic properties of promising topological insulator candidate materials such as HgTe quantum wells BHZTI , Bi2Se3 TIBiSe , and Sb2Te3 TISbTe . Experimentally, magnetic doping of topological insulators with transition metal DopgapSize1 as well as rare earth REdop ions has been studied recently. In the former case impurity magnetic moments are ferromagnetically ordered resulting in massive Dirac Fermions Masive1 ; Masive2 and realizing quantum anomalous Hall effect QAHE . It has been shown that out-of-plane magnetic ordering can be observed with Cr dopants Cr-doped1 ; Cr-doped2 ; Cr-doped3 . More recently, using synchrotron-based x-ray techniques, it has been demonstrated that the topological insulator surface is magnetically ordered while the bulk is not in Cr-doped Bi2Se3 surfOrder . In the case of rare earth doping, although high magnetic moments can be introduced into the topological insulators but, instead of long range ferromagnetic ordering, antiferromagnetic ordering along with Dirac gap opening have been reported for Gd-doped topological insulators AntiFerRE . Moreover, a large effective magnetization can be introduced in rare-earth-doped topological insulators by proximity coupling either to a ferromagnetic insulator FeroHetro or in a heterostructure with transition-metal-doped layers TM-REHetro1 ; TM-REHetro2 .

Theoretically, on the other hand, it has been revealed conductance properties of topological boundary states to be coupled to magnetic impurities CondTITheo1 ; CondTITheo2 . Coupling of helical edge states with spin impurities or nuclear in topological insulator HiperCoupling ; NuclearSpinTI leads to the Anderson localization AndLoc and anisotropic nuclear spin-spin interaction internalspin1DSHE with spiral spin ordering NuclearSpinTI . It is also demonstrated that Mn-doped InAs/GaSb Mn-doped develops a ferromagnetic ordering at low enough temperature. Therefore, the effect of ferromagnetism generated by the magnetic impurities would gap out the boundary states spectrum Masive1 ; Masive2 ; Mn-doped ; 1DGapFormImpu . Nevertheless, in spite of being 1D Dirac dispersion in some states of matter BHZTI ; 1DD its properties has not been demonstrated fully with broken time-reversal symmetry in other aspects. It, thus, deserves to investigate indirect interaction between magnetic moments in magnetic topological insulators with Zeeman exchange splitting resulted from breaking of time-reversal symmetry due to spin polarization of topological edge states PlorizedTI .

The purpose of this research is to study indirect exchange interactions between magnetic impurities mediated by 1D helical carriers of a two-dimensional topological insulator with weakly broken time-reversal symmetry. Here, we focus on the role of gapped spectrum of helical edge states. We find that unlike usual 1D quantum wire case 1DRKKYConv , the RKKY exchange interaction in our system is strongly anisotropic and includes the Heisenberg, Dzyaloshinsky-Moriya, and Ising terms which is somewhat similar to gapless spin-orbit-coupled quantum wire 1DRKKYSpinOrbit and gapful spin-orbit-coupled carbon nanotubes 1DRKKYSpinOrbitCNT . However, in contrast, here, the Ising terms are comprised of in-plane and out-of-plane interactions which have the same form as the case of three-dimensional topological insulators in the presence of bandgap 3DTIRKKY0 . The corresponding range functions display an oscillatory (exponentially) decay with a spatial separation of magnetic moments if chemical potential resides in the band (bandgap). Furthermore, all the range functions are significantly affected by the energy gap, in particular, near the band edge leading to decreasing of oscillation period as a function of energy gap or Fermi energy. Moreover, in comparison with Heisenberg, Dzyaloshinsky-Moriya, and in-plane Ising terms having invariable oscillation amplitude independent of Fermi energy position with respect to energy gap, the oscillation amplitude of out-of-plane Ising term increases (decreases) as a function of energy gap (Fermi energy). For both subgap and over gap regimes analytical expressions are presented for asymptotic behaviors of the range functions. Also, electron-electron interactions are included.

The paper is organized as follows. The model and basic theory of indirect exchange interaction between magnetic impurities are introduced in Sec. II. In Sec. III, analytical and numerical results for non-interacting case are presented. The effects electron-electron interactions are analyzed in Sec. IV. Section V is devoted to summarize.

II Model and Theory

The setup we consider throughout the paper is schematically depicted in Fig. 1(a). It is composed of a two-dimensional topological insulator with 1D helical edge state and two magnetic impurities located on the edges of the topological insulator in the presence of Zeeman exchange field. Helical edge states of two-dimensional topological insulators can be thought of as 1D Dirac states. Also, inclusion of exchange field which is perpendicular to the surface gaps out the metallic edge states at the boundaries by violating time-reversal symmetry. Thus, we focus on the following Hamiltonian describing 1D gapped Dirac states 1DGapFormImpu ; HelicalZeemanHam1 ; HelicalZeemanHam2 ,

[TABLE]

and the corresponding edge spectrum reads

[TABLE]

where the Pauli matrices $\sigma_{x,y}$ represent the physical spin of the electron, $k$ is a wave vector along the 1D channel, the band index $\lambda=\pm$, and $v_{f}$ is Fermi velocity. Since the spin quantization axis of edge states is along the x direction, the exchange field induced by out-of-plane spin polarization opens up gap $2\Delta$ in the edge spectrum (see Fig. 1(b)). Thus, this lifts two-fold degeneracy of the so-called Dirac point. However, using magnetic insulator, such as EuS, EuO, EuSe, which is proximity coupled to the system can induce considerable Zeeman exchange field with gap size 9 meV ProxgapSize1 ; ProxgapSize2 . Also, It has been demonstrated that the fabricated MnBi2Se4/Bi2Se3 heterostructure exhibits ferromagnetism up to room temperature accompanied by a Dirac gap opening of $\sim$ 100 meV ProxgapSize3 . In contrast, the surface state gap varies between several tens to a hundred meV DopgapSize1 in magnetically doped topological insulators and its formation has been controversial depending on the impurity and impurity distance from the surface DopgapSize3 . However, the bandgap has been estimated to be as high as 10 meV Mn-doped InAs/GaSb Mn-doped ; 1DGapFormImpu which can be suppressed by non-magnetic impurities 1DGapFormImpu . Note also that in topological insulators due to presence of strong spin-orbit interaction, the electron spin is no longer a good quantum number originating from the off-diagonal elements of BHZ Hamiltonian BHZTI .

Two localized magnetic impurities with moments $\mathbf{S}_{j}(j=1,2)$ can be coupled to itinerant spin-polarized Dirac fermions with spin density $\mathbf{s}(R_{j})=\delta(r-R_{j})\bm{\sigma}$ at position $R_{j}$. This coupling can be modeled by

[TABLE]

where $J$ denotes coupling strength of magnetic impurities with the host Dirac fermions. Using the second-order perturbation theory and treating $H_{\text{int}}$ as a perturbation, the RKKY interaction between two magnetic impurities mediated by host carriers can be obtained by RKKYPertu1 ; RKKYPertu2 ; RKKYPertu3

[TABLE]

where $Tr$ stands for trace over spin degree of freedom, $Im$ is imaginary part, $\epsilon_{f}$ is the Fermi energy measured from the Dirac point. $G^{0}(R,\epsilon^{+})$ denotes the Green’s function matrix in real-space with $R=R_{2}-R_{1}$ being a distance between the two magnetic centers and $\epsilon^{+}=\epsilon+i\eta$ with $\eta\rightarrow 0^{+}$. The real-space Green’s function can be written as,

[TABLE]

with

[TABLE]

where $G^{0}(k,\epsilon^{+})$ is the momentum space Green’s function associated with Eq. (1). Substituting Eq. (6) into Eq. (5) and using the residue theorem, we derive a closed form expression for the real-space Green’s function as

[TABLE]

where $\alpha=(\epsilon^{+})^{2}-(\Delta)^{2}$ and $\sigma_{0}$ is the unit matrix. Combining the result in Eq. (7) with Eq. (4), yields a formula for $H_{RKKY}$ as follows:

[TABLE]

where the range functions are

[TABLE]

with $f(\alpha)=\frac{J^{2}}{\pi v^{2}_{f}}e^{\frac{2i\sqrt{\alpha}}{v_{f}}R}$. As one can see from Eq. (8), the RKKY interaction consists of the Heisenberg, the Dzyaloshinsky-Moriya, and two component Ising interactions whose range functions are given by $F_{1}$, $F_{2}$, $F_{3}$, and $F_{4}$, respectively. The Heisenberg, x- and y-component Ising interactions favor collinear magnetic spin alignment while the Dzyaloshinsky-Moriya imposes in-plane non-collinear magnetic spin orientation. Notably, the Heisenberg, Dzyaloshinsky-Moriya and x-component of Ising terms come from helical nature of the edge states and their range functions are modified due to gap opening [see Eqs. (9)-(11)]. In contrast to non-interacting 2DTIRKKY1 and interacting NuclearSpinTI gapless helical cases where out-of-plane component of the RKKY coupling is negligible compared to the in-plane ones, interestingly, Eq. (12) implies that the out-of-plane component remains survived. This originates from partial mixing of opposite spin states of helical states due to gap term. It should also be noted that by setting $\Delta=0$, we recover the previously obtained results 2DTIRKKY1 . Furthermore, in the gap region, the integrands of the range functions become purely real functions, whereby all these terms will be suppressed, resulting actually from the lack of available states for itinerant carriers. However, if chemical potential lies within the gap region then, for sufficiently small energy gap, virtual interband transitions of electrons can mediate the indirect exchange interaction, which is known as Bloembergen-Rowland mechanism BRInt1 ; BRInt2 .

III Analytical and numerical results

In what follows, due to particle-hole symmetry, without loss of generality, we assume that $\epsilon_{f}>0$. In general, although there are no analytical expressions for the range functions [Eqs. (9)-(12)], it would be possible to estimate analytical statements for them in some limiting cases. In the small gap regime, i.e., $\epsilon_{f}\gg\Delta$, Eqs. (9)-(12) can be approximated by

[TABLE]

where $\gamma=\frac{2R\epsilon_{f}}{v_{f}}$ and $Ci(x)$ is cosine integral function table . We note that the first term of range functions of the Heisenberg, Dzyaloshinsky-Moriya, and in-plane Ising interactions, having dominant contribution, decay as $R^{-1}$ reminiscing the range function behavior of usual 1D electron gas. Also, exploiting the asymptotic form of cosine integral function table ,

[TABLE]

for long distance limit, i.e., $x,R\gg 1$, we identify that the spatial dependence of y-component of Ising interaction falls off similar to the other interactions.

In the subgap limit, i.e., $\epsilon_{f}\ll\Delta$, for long distance case, on the other hand, using steepest descent method SteepDec we can determine leading-order asymptotic approximations to Eqs. (9)-(12) as,

[TABLE]

Note that, in this regime, the above obtained range functions have an exponentially decaying behavior determined by the energy gap and Fermi velocity.

Generally, behaviors of the range functions can be explored by numerical evaluation. We set $v_{f}a^{-1}_{B}=1$ by introducing Bohr radius $a_{B}$ and $J=1$. The range functions F1, F2, F3, and F4 are plotted as a function of $R$ in Fig. 2 for different values of $\Delta$ with $\epsilon_{f}=4$. As shown in the main panels of Fig. 2, for $\epsilon_{f}>\Delta$ the oscillations of all the range functions are damped by increasing $R$ and the period of oscillations is increased with the increase of $\Delta$. In the meanwhile, the amplitude of all the range functions decreases with the increment of band gap, interestingly, except for the case of out-of-plane Ising term that increases. Moreover, for $\epsilon_{f}<\Delta$, as shown in the insets of Fig. 2, all the range functions decay exponentially with the distance of two magnetic impurities. Moreover, $F_{2}$ takes infinitesimally small values compared to the others. We also observe that the rate of exponential decay becomes faster with increasing $\Delta$ arising from suppression of the excitations of massive carriers across the gap. As a result, the interactions result in magnetic ordering of impurities depending on the values of the parameters. However, the spiral ordering due to Dzyaloshinsky-Moriya term is smeared out by the induced gap in the over gap regime. While in the subgap regime, the gap term completely suppresses the spiral ordering and, instead, establishes considerable out-of-plane magnetization. Since, as already mentioned above, the spiral ordering is originated from the helicity, this indicates that the gap opening gradually mixes the spin states and destroys the helicity.

The dependence of range functions on Fermi energy for various values of $\Delta$ with $R=1$ is depicted in Fig. 3. For $\epsilon_{f}<\Delta$ the absolute values of range functions of both Heisenberg and in-plane Ising interactions gradually increase from small values with the enhancement of $\epsilon_{f}$ until reach a maximum at $\epsilon_{f}=\Delta$ as shown in Figs. 3(a) and 3(c). In the energies below $\Delta$, however, the magnitudes of Dzyaloshinsky-Moriya and out-of-plane Ising interactions are smaller than those of the other range functions [see Figs. 3(b) and 3(d)]. This results in dominating the ferromagnetic coupling in the subgap regime 3DTIRKKY0 . Moreover, the out-of-plane Ising term shows a sharp dip at $\epsilon_{f}=\Delta$ which can be attributed to the high spin-polarized density of states available at the band edge. Moreover, all the range functions exhibit oscillatory behavior for $\epsilon_{f}>\Delta$ except that the envelope of $F_{4}$ is damped simultaneously. Also, $F_{1}$, $F_{2}$, and $F_{3}$ demonstrate constant amplitudes for different values of $\Delta$ while the amplitude of oscillations of $F_{4}$ increases as a function of $\Delta$.

The gap dependence of $F$’s for different values of $\epsilon_{f}$ is illustrated in Fig. 4. One can see that except for $F_{4}$ which is zero at $\Delta=0$ [see Fig. 4(d)] the other range functions can take various values depending on both Fermi energy and $R$ [see Figs. 4(a), 4(b), and 4(c)]. By increasing $\Delta$ the values of $F$’s change with small modulations at first. Then, as $\Delta$ increases further the range functions begin to oscillate in $\Delta\lesssim\epsilon_{f}$. As already mentioned above, from Fig. 4 it is also visible that the oscillation amplitude of y-component of Ising term is enhanced by increasing bandgap while the amplitude of oscillations of the other range functions remains unchanged even for different values of $\epsilon_{f}$. Note, interestingly, that the range function of Dzyaloshinsky-Moriya interaction is identically zero for $\epsilon_{f}=0$ irrespective of $\Delta$, as illustrated in Fig. 4(b).

In both Figs. 3 and 4, the period of oscillatory part of the range functions decreases by approaching to the band edge and, as a result, takes smallest values near band edge. When the Fermi energy is so small that $\Delta>\epsilon_{f}$ the range functions tend to zero rapidly. This is a consequence of vanishing of Fermi surface. Finally, it is worthwhile noting that the above-presented numerical behavior is in agreement with the analytical one quite well in the two limits of $\epsilon_{f}\gg\Delta$ and $\epsilon_{f}\ll\Delta$.

IV Effects of electron-electron interaction

The RKKY interaction in interacting quantum wires without RKKYeeWoutSO1 ; RKKYeeWoutSO2 and with RKKYeeWSO ; RKKYeeWSO1 spin-orbit interaction has been studied before. These studies have also been extended to spin-orbit-coupled helical liquids RKKYeeHelicLiq . Now, let us consider the effects of electron-electron interactions on RKKY interactions in the 1D massive helical states. Due to absence of Fermi points in the subgap regime we restrict ourself to the $\epsilon_{f}>\Delta$ case involving Tomonaga-Luttinger liquid model Giamarchi . Close to band edge the gap $\Delta$ bends the dispersion, as can be seen from Fig. 1(b), so it renormalizes the Fermi velocity as $v^{\prime}_{f}=v_{f}\sqrt{1-(\Delta/\epsilon_{f})^{2}}$. We linearize BHZTI the massive helical states of Eq. (1) on one of the system edges around Fermi points at momenta $\pm k_{f}=\pm\sqrt{\epsilon^{2}_{f}-\Delta^{2}}/v_{f}$ to obtain the low-energy Hamiltonian as

[TABLE]

and the low-energy electron field operator as

[TABLE]

where the operators $\psi_{r}(x)$ with $r=R,L=+,-$ are the annihilation fields for right- and left-moving fermions at coordinate point $x$ along the edge. The spinor $\chi_{r}(k)$ in spin space is given by

[TABLE]

with $\theta_{k}=1/2\tan^{-1}(-\Delta/kv_{f})$ being $k$-dependent spin-rotation angle implying that the gap term mixes the spin states [Eq. (26)] so that the spin is no longer a good quantum number. The fermionic-field operators can be expressed in terms of bosonic fields $\phi_{r}(x)$ as Giamarchi

[TABLE]

where $a_{0}$ is a short-distance cutoff, and $\eta_{r}$ are Klein factors.

The Hamiltonian of electron-electron interactions having dominant contribution are given by

[TABLE]

where $g_{2,4}$ are interaction constants. We treat the total Hamiltonian $\mathcal{H}=\mathcal{H}_{0}+\mathcal{H}_{e-e}$ in the bosonized form yielding

[TABLE]

with bosonic fields

[TABLE]

Here, we have introduced the speed of collective excitations $v$ and Luttinger parameter $K$ as

[TABLE]

depending on gap and Fermi energy through $v^{\prime}_{f}$ in contrast to the conventional ones. For noninteracting case $K=1$ while $0<K<1$ ($K>1$) corresponds to repulsive (attractive) electronic interactions. Also, we have assumed the weak interaction ignoring backscattering potential backscat1 ; backscat2 . Note, however, that it has been shown that elastic spin-flip backscattering can be occurred by coupling of electrons to nuclear spins through the hyperfine interaction causing a significant edge resistance at low enough temperatures NuclearSpinTI .

In terms of electron field operator, the spin density can be defined as $\mathbf{s}(x)=\psi^{{\dagger}}(x)\bm{\sigma}\psi(x)$. Using the identity

[TABLE]

and Eq. (27) the components of spin density $\mathbf{s}(x)$ can be determined in the bosonized language RKKYeeWoutSO1 ; RKKYeeWoutSO2 as

[TABLE]

Having obtained the components of spin density and following the procedure in Ref. Giamarchi , the imaginary-time spin-spin correlation functions

[TABLE]

can be determined yielding nonzero components as

[TABLE]

with

[TABLE]

By integrating $\mathcal{S}^{ab}\!(\tau,R)$ over $\tau$, i.e.,

[TABLE]

one gets spin susceptibility. Plugging $\kappa^{ab}(R)$ into the following general formula for RKKY interaction

[TABLE]

we obtain the similar magnetic spin structure as Eq. (8) with modified range functions which for $a_{0}\ll|x|\ll v_{f}/k_{B}T$ given by

[TABLE]

where $\nu(K)=\frac{a^{2K-2}_{0}\Gamma(K-1/2)}{\sqrt{\pi}v\Gamma(K)}$ with $\Gamma(p)$ being Gamma function. Consequently, we observed that since the spin basis of Hamiltonian (22) is the same as Hamiltonian (1) so the spin texture structure of Eq. (8) remains unchanged for interacting electrons in the host Luttinger liquid and only the range functions are modified. This modification affects on the power-law decay characterizing by exponent $2K-1$ and on the magnitude of range functions. Note that the above equations are valid in the range 1/2 $<$ K $\leq$ 1 below which RKKY-Kondo competition becomes important RKKYKondeeHelicLiq . Moreover, for non-interacting case, i.e., $K=1$, and $\epsilon_{f}\gg\Delta$ expanding Eqs. (49)-(52) provides similar leading behavior as Eqs. (13)-(16).

V Summary

In summary, the RKKY interaction of interacting spins is investigated in the edge of time-reversal broken two-dimensional edge states. The resulting interaction is composed of Heisenberg, Dzyaloshinsky-Moriya, in-plane and out-of-plane Ising terms. The corresponding range functions fall off with a power-law decay $R^{-1}$ and exponentially in the over gap and subgap regimes, respectively, at large distance. As the energy gap increases, the rate of spatial exponential decay and the spatial periodicity of the range functions increase. The range functions of these interactions are significantly influenced through gapped band structure providing small periods near the band edge as a function of Fermi energy or energy gap. We found if the Fermi energy lies inside the bandgap then the range function of Dzyaloshinsky-Moriya interaction has much smaller values than those of other interactions. Also, the amplitude of range function of out-of-plane Ising interaction increases as gap increases for finite Fermi surface while the amplitudes of the other interactions do not change. In the some limiting cases analytical results are approximately extracted for the range functions supporting the numerical results. With taking into account electron-electron interactions it is shown the magnitude and power-low decay of the range functions change depending on collective excitations velocity and Luttinger parameter. It is worthwhile noting that by combining density functional theory calculations with complementary experimental techniques, it has been revealed that transition metal dopants not only affect the magnetic state of host material, but also alter the electronic structure DopgapSize3 . So, we believe that the ferromagnetic ordering of impurities which is dominated in the subgap regime supports the bandgap. Thus the renormalized spectrum will impact the interactions between impurities promoting ferromagnetic ordering as long as the Fermi surface resides within the bandgap. Finally, we remark that our setup can be implemented by source-probe measurement method source-probe for low dimensional systems and be served as an alternative approach in investigating magnetic properties of edge states.