Vortex-core structure in a mixture of Bose and Fermi superfluids

TL;DR

This paper investigates how the core structure of a quantized vortex in a Bose-Fermi superfluid mixture changes with varying density ratios, revealing a transition where fermions are depleted from the vortex core, which can be experimentally observed.

Contribution

It identifies a transition in vortex-core structure in Bose-Fermi mixtures, highlighting the depletion of fermions and localization of BEC in the vortex core as the density ratio varies.

Findings

Fermion depletion in vortex core at certain density ratios

Localization of BEC in vortex core during transition

Vortex properties change significantly across the transition

Abstract

We study a single quantized vortex in the fermionic component of a mixture of Fermi superfluid and Bose-Einstein condensate. As the density ratio between the boson and the fermion components is tuned, we identify a transition in the vortex-core structure, across which fermions in the vortex core become completely depleted even in the weak-coupling Bardeen-Cooper-Schrieffer regime. This is accompanied by changes in key properties of the vortex state, as well as by the localization of the Bose-Einstein condensate in the vortex core. The transition in the vortex-core structure can be experimentally probed in Bose-Fermi superfluid mixtures by detecting the size and visibility of the vortices.

Figures

Equations

$$\begin{split}H=&\int d^{3}\boldsymbol{r}\biggl{[}\sum_{\sigma}\hat{\psi}_{\sigma}^{\dagger}\left(-\frac{\hbar^{2}\nabla^{2}}{2m_{F}}+g_{BF}\hat{\varphi}^{\dagger}\hat{\varphi}-\mu_{F}\right)\hat{\psi}_{\sigma}\\ &+g_{F}\hat{\psi}_{\uparrow}^{\dagger}\hat{\psi}_{\downarrow}^{\dagger}\hat{\psi}_{\downarrow}\hat{\psi}_{\uparrow}+\hat{\varphi}^{\dagger}\left(-\frac{\hbar^{2}\nabla^{2}}{2m_{B}}-\mu_{B}\right)\hat{\varphi}\\ &+\frac{g_{B}}{2}\hat{\varphi}^{\dagger}\hat{\varphi}^{\dagger}\hat{\varphi}\hat{\varphi}\biggr{]},\end{split}$$

$$\displaystyle\left[\begin{array}[]{cc}\hat{M}&\Delta\left(\boldsymbol{r}\right)\\ \Delta^{\ast}\left(\boldsymbol{r}\right)&-\hat{M}\end{array}\right]\left[\begin{array}[]{c}u_{n}\\ \upsilon_{n}\end{array}\right]=E_{n}\left[\begin{array}[]{c}u_{n}\\ \upsilon_{n}\end{array}\right],$$

$$\left[-\frac{{\hslash }^2{\nabla }^2}{2m_B}+g_{BF}{n}_F(\mathbf{r})+g_B{n}_B(\mathbf{r})\right]\phi (\mathbf{r})={\mu }_B\phi (\mathbf{r}).$$

$$\left[\begin{array}[]{c}u_{n}\\ \upsilon_{n}\end{array}\right]=\frac{1}{\sqrt{2\pi L_{z}}}\sum_{k_{z}lj}\left[\begin{array}[]{c}c_{k_{z}lj}^{\left(n\right)}\phi_{jl}\left(\rho\right)\\ d_{k_{z}lj}^{\left(n\right)}\phi_{jl+1}\left(\rho\right)e^{i\theta}\end{array}\right]e^{i(l\theta+k_{z}z)},$$

$$\sum_{j^{{}^{\prime}}}\left[\begin{array}[]{cc}T_{l}^{jj^{{}^{\prime}}}&\Delta_{l,l+1}^{jj^{{}^{\prime}}}\\ \Delta_{l+1,l}^{jj^{{}^{\prime}}}&-T_{l+1}^{jj^{{}^{\prime}}}\end{array}\right]\left[\begin{array}[]{c}c_{k_{z}lj^{{}^{\prime}}}^{(n)}\\ d_{k_{z}lj^{{}^{\prime}}}^{(n)}\end{array}\right]=E_{n}\left[\begin{array}[]{c}c_{k_{z}lj}^{(n)}\\ d_{k_{z}lj}^{(n)}\end{array}\right],$$

$$\sum _{j^{{}^{\prime }}}\left(\frac{{\alpha }_{j0}^2}{R^2}{\delta }_{j{j}^{{}^{\prime }}}+g_{BF}{n}_{F,0}^{j{j}^{{}^{\prime }}}+g_B{n}_{B,0}^{j{j}^{{}^{\prime }}}\right)f_{j^{{}^{\prime }}}={\mu }_B{f}_j,$$

$${\Delta }_{l{l}^{{}^{\prime }}}^{j^{{}^{\prime }}j}={\int }_0^{R}d\rho \rho \Delta \left(\rho \right){\varphi }_{jl}\left(\rho \right){\varphi }_{j^{\prime }{l}^{{}^{\prime }}}\left(\rho \right).$$

$$\Delta \left(\rho \right)=\frac{g_F}{2\pi L_z}\sum _{l{k}_z,E_n\ge 0,j{j}^{{}^{\prime }}}{c}_{k_{z}lj}^{\left(n\right)}{d}_{k_{z}l{j}^{{}^{\prime }}}^{\left(n\right)}{\varphi }_{jl}\left(\rho \right){\varphi }_{j^{{}^{\prime }}l+1}\left(\rho \right),$$

Full text

I Introduction

Superfluidity is one of the most remarkable macroscopic quantum phenomena. In liquid helium, and more recently in ultracold atomic gases, both Bose and Fermi superfluidity has been investigated in great detail becsf1 ; becsf2 ; becsf3 ; becsf4 ; becsf5 ; fermisf1 ; fermisf2 ; fermisf3 ; BECbook . A long-sought goal in liquid helium is the simultaneous superfluidity of the bosonic 4He and the fermionic 3He, which turned out to be difficult due to the strong interactions between the two species Tuoriniemi2002 ; Rysti2012 . The recent experimental realization of superfluid mixtures in cold gases of Bose and Fermi atoms represents a significant step forward Barbut2014 ; Yao2016 ; Ikemachi2016 ; Roy2016 . With highly tunable parameters Chin2010 , cold atomic gases offer a flexible platform on which various many-body properties of the two-species superfluid can be investigated from new perspectives, and in a controlled fashion.

An important signature of superfluidity is the existence of quantized vortices, which have been extensively investigated in a pure Bose or Fermi superfluid of cold atoms becsf4 ; fermisf1 ; fermisf2 ; BECbook . In a very recent experiment, vortices have also been generated in a Bose-Fermi mixture of 6Li and 41K atoms Yao2016 . Besides confirming the two-species superfluidity of the system, the experiment raises the challenging question regarding the vortex structure in the presence of a two-species superfluid. In particular, as the microscopic vortex-core structure offers vital information of the many-body environment Machida2005 ; Chien2005 ; Sensarma2006 and has an important impact on the macroscopic structure of the vortex lattice Jiang2016 , a better understanding of a single quantized vortex in the Bose-Fermi mixture is in order.

In this work, we study a singly quantized vortex in the fermionic component of the mixture of a Bose-Einstein condensate (BEC) and a Fermi superfluid. By self-consistently solving the Bogoliubov-de Gennes (BdG) and the Gross-Pitaevskii (GP) equations, we show that, as the density ratio between the boson and the fermion components increases, a transition occurs in the vortex-core structure, beyond which fermions in the vortex core become completely depleted even in the Bardeen-Cooper-Schrieffer (BCS) regime. The local depletion of fermions is accompanied by the localization of the BEC in the vortex core. The transition originates from the repulsive Bose-Fermi interactions, which induce effectively potentials of different signs for the Bose and the Fermi components. As the ground state of the many-body system changes across the transition, relevant properties such as the superfluid order parameters, the quasi-particle spectrum, and the superfluid current density of the fermions are all affected. While the transition of the vortex core persists as the Fermi superfluid is far away from the BCS regime, multiple branches of vortex bound states emerge from the bulk spectrum due to the strong extension of the vortex core. Whereas vortices in two-species BECs have been studied previously  Matthews1999; Anderson2000 ; Ho1996 ; Skryabin2000 ; Ripoll2000 ; Garcia2000 ; Jezek2001 ; Chui2001 ; Catelani2010 ; Mason2013 ; Ivashin2015 , the impact of the vortex-core transition on quantities like Fermi superfluid order parameter and quasiparticle spectra is an interesting new element in the Bose-Fermi mixture, and has not been reported before. The depletion of vortex-core fermions and the localization of BEC should be detectable under current experimental conditions by probing the size and visibility of the vortices.

The remainder of the paper is organized as follows. In Sec. II, we present the model Hamiltonian and the equations of motion of the quasiparticles. The transition in the vortex-core structure is then discussed in Sec. III. The underlying mechanism of such a transition, as well as the associated changes in density distributions and quasiparticle excitations are analyzed in Sec. IV. We study the effects of interaction in Sec. V, and summarize in Sec. VI.

II Formalism

The Hamiltonian of an interacting Bose-Fermi mixture can be written as Zheng2014 ; Tylutki2016

[TABLE]

where $\hat{\psi}_{\sigma}\left(\boldsymbol{r}\right)$ is the field operator for fermions with pseudospin $\sigma=\uparrow\downarrow$ and $\hat{\varphi}\left(\boldsymbol{r}\right)$ is the field operator for bosons. $\mu_{F}$ ($\mu_{B}$) is the chemical potential of the Fermi (Bose) component. The interaction parameters for bosons are given by $g_{B}=4\pi\hbar^{2}a_{B}/m_{B}$ and $g_{BF}=4\pi\hbar^{2}a_{BF}/m_{BF}$, where $a_{B}$ and $a_{BF}$ are respectively the Bose-Bose and the Bose-Fermi scattering lengths. Here, the reduced mass $m_{BF}$ is related to the mass of bosons $m_{B}$ and that of fermions $m_{F}$ as $m_{BF}=(m_{B}+m_{F})/(2m_{B}m_{F})$. We consider the case where the mixture is prepared near a wide $s$-wave Feshbach resonance between the two fermion species, so that $g_{F}$ is related to the Fermi-Fermi scattering length $a_{F}$ through $1/(k_{F}a_{F})=8\pi\epsilon_{F}/(g_{F}k^{3}_{F})+(2/\pi)\sqrt{E_{c}/\epsilon_{F}}$, with the Fermi energy $\epsilon_{F}=\hbar^{2}k_{F}^{2}/2m_{F}$ and the Fermi wave vector $k_{F}$. $E_{c}$ is the cutoff energy introduced in the renormalization process and does not affect the results Sensarma2006 .

Following the standard mean-field formalism, we define, respectively, the order parameter for the Fermi pairing superfluid $\Delta\left(\boldsymbol{r}\right)=g_{F}\langle\psi_{\downarrow}\psi_{\uparrow}\rangle$, and for the BEC $\varphi\left(\boldsymbol{r}\right)=\langle\hat{\varphi}\left(\boldsymbol{r}\right)\rangle$. From Eq. (1), we derive the generalized BdG equation

[TABLE]

which is coupled to the GP equation

[TABLE]

Here, $\hat{M}=-\frac{\hbar^{2}\nabla^{2}}{2m_{F}}+g_{BF}n_{B}\left(\boldsymbol{r}\right)-\mu_{F}$, $\left(u_{n},\upsilon_{n}\right)^{T}$ is the fermion quasi-particle wave function with eigenvalue $E_{n}$, and $n_{F}\left(\boldsymbol{r}\right)=\sum_{\sigma}\langle\psi_{\sigma}^{\dagger}\psi_{\sigma}\rangle$ and $n_{B}\left(\boldsymbol{r}\right)=\left|\varphi\left(\boldsymbol{r}\right)\right|^{2}$ are respectively the fermion and the boson densities.

We study an isolated vortex state in a cylindrically symmetric trap with an open boundary condition at $\rho=R$, and periodic boundary conditions at $z=\pm L_{z}/2$, where $\boldsymbol{r}=(\rho,\theta,z)$ under the cylindrical coordinates. We further assume $R\gg L_{z}$ such that the dynamic degrees of freedom of the BEC along the $z$ direction can be neglected. To describe a single vortex, we take $\Delta(\boldsymbol{r})=\Delta(\rho)e^{-i\theta}$ and expand the quasiparticle wave functions as

[TABLE]

where $k_{z}=2\nu\pi/L_{z}$ with $\nu\in\mathbb{Z}$. The Fourier-Bessel series $\phi_{jl}\left(\rho\right)=\sqrt{2}J_{l}\left(\alpha_{jl}\rho/R\right)/\left[RJ_{l+1}\left(\alpha_{jl}\right)\right]$ for $l\in\mathbb{Z}$ and $j\in\mathbb{Z}^{+}$. $J_{l}\left(\rho\right)$ is the Bessel function of the first kind, whose zero points are given by $\alpha_{jl}$. Similarly, we can write $\varphi\left(\boldsymbol{r}\right)$ as $\varphi\left(\boldsymbol{r}\right)=\left(2\pi L_{z}\right)^{-1/2}\sum_{j}f_{j}\phi_{j0}\left(\rho\right)$.

Finally, the BdG and GP equations become

[TABLE]

and

[TABLE]

where $T_{l}^{jj^{{}^{\prime}}}=(\frac{\hbar^{2}}{2m_{F}}\frac{\alpha_{jl}^{2}}{R^{2}}+\frac{\hbar^{2}k_{z}^{2}}{2m_{F}}-\mu_{F})\delta_{jj^{{}^{\prime}}}+g_{BF}n_{B,l}^{jj^{{}^{\prime}}}$ with $n_{B,F,l}^{jj^{{}^{\prime}}}=\int_{0}^{R}d\rho\rho n_{B,F}(\rho)\phi_{jl}\left(\rho\right)\phi_{j^{\prime}l}\left(\rho\right)$, and

[TABLE]

Equations (11) and (12) are then solved self-consistently with the gap equation

[TABLE]

and the number equations $N_{F}=2\sum_{lk_{z},E_{n}\geq 0,j}d_{k_{z}lj}^{\left(n\right)2}$ and $N_{B}=\sum_{j}f_{j}^{2}$. For our numerical calculations, we simultaneously fix the total number of particles of the fermion ($N_{F}$) and the boson ($N_{B}$) component.

III Transition in the vortex-core structure

In the presence of the BEC, the vortex-core structure in the Fermi condensate can be significantly modified by the Bose-Fermi interactions. In current experiments with Bose-Fermi superfluid mixtures Barbut2014 ; Yao2016 ; Ikemachi2016 ; Roy2016 , it is typically difficult to simultaneously tune the Fermi-Fermi and Bose-Fermi interactions to the strongly interacting regime. We therefore consider the case where the Fermi condensate is close to a wide Feshbach resonance, while the particle-number ratio of the Bose-Fermi mixture $\gamma_{n}=N_{B}/N_{F}$, rather than the Bose-Fermi interaction strength, is varied. We will first focus on the case in which the Fermi condensate is on the BCS side of the resonance, such that the impact of Bose-Fermi interactions on the vortex-core structure is the most significant. For simplicity, throughout this work, we consider a noninteracting BEC with $g_{B}=0$, while we emphasize that all results remain qualitatively unchanged for BECs with attractive interactions or with weak repulsive interactions.

In Fig. 1, we show various properties of a single vortex in the Bose-Fermi mixture as functions of $\gamma_{n}$. In all of these calculations, sharp transitions can be identified near $\gamma_{n}^{c}\approx 2.0$. For $\gamma_{n}<\gamma_{n}^{c}$, the vortex-core structure has similar features as that in the absence of BEC: the order parameter $\Delta(\rho)$ scales linearly with $\rho$ near the vortex core, the fermion density depletes only slightly at the core, and the superfluid current distribution $j\left(\rho\right)=-2\hbar(2\pi L_{z}m_{F}\rho)^{-1}\sum_{lk_{z},E_{n}\geq 0}(l+1)\left[\sum_{j}d_{k_{z}lj}^{(n)}\phi_{jl+1}\left(\rho\right)\right]^{2}$ has a sharp peak near $\rho_{c}\thicksim 1/k_{F}$, which roughly corresponds to the size of the core Sensarma2006 ; Hu2006 . The linear dependence of the order parameter at the vortex core $\Delta(\rho)\propto\rho$ as $\gamma_{n}<\gamma_{n}^{c}$ can be semi-analytically derived by considering the asymptotic behavior of $J_{l}(\rho)\propto\rho^{|l|}$ as $\rho\sim 0$. Close to the origin, $\Delta$ is small compared to other terms; the Bose density can therefore be seen as a constant due to its relatively large length scale of variation (see Fig. 2). Therefore, the BdG equation is dominated by the kinetic-energy terms and $\Delta\left(\rho\right)\propto J_{0}\left(\rho\right)J_{1}\left(\rho\right)\propto\rho$. Similarly, this initial slope of $\Delta(\rho)$ is set by the length scale $k_{F}^{-1}$ Sensarma2006 . Since $j=\rho_{s}v_{s}$ with $v_{s}=\left(\hbar/2m_{F}\rho\right)$ and $\rho_{s}\sim\Delta^{2}$ , $j\left(\rho\right)$ also varies linearly near the origin. When $\rho$ increases away from the vortex core, $j\left(\rho\right)\propto 1/\rho$ as $\Delta$ becomes a constant. A sharp peak in the $j(\rho)$ curve hence arises in the intermediate regime. However, as shown in Fig. 1 and discussed more in the next section, when $\gamma_{n}$ increases beyond $\gamma_{n}^{c}$, these features change dramatically: the fermion density near the core center $n_{F}(0)$ becomes completely depleted, the order parameter deviates from a linear scaling in $\rho$ at around the core and the sharp feature in the superfluid current distribution disappears.

The occurrence of the transition at $\gamma_{n}^{c}$ is a direct result of Bose-Fermi interactions. From Eqs. (11) and (12), we see that the Bose-Fermi interactions give rise to effective potentials for the two components. While the Bose-Fermi interaction is repulsive, the effective potential near the vortex core is repulsive for fermions and attractive for bosons, which is due to the qualitatively different density distributions of $n_{B}(\rho)$ and $n_{F}(\rho)$. The increase of $\gamma_{n}$ would lead to a stronger repulsive potential $g_{BF}n_{B}(\rho)$ for the fermions, which further depletes fermions from the vortex core and increases the depth and width of the attractive potential $g_{BF}n_{F}(\rho)$ for the BEC. Such a positive feedback mechanism would eventually make the widths of the attractive and the repulsive potentials match with one another, at which point the vortex-core structure for fermions undergoes drastic changes as the BEC becomes localized at the vortex core.

IV Density distributions and quasi-particle excitations in the vortex core

The general picture above can be confirmed in Fig. 2(a), where we estimate the widths of the repulsive and the attractive potentials as the half-width of $n_{B}(\rho)$ and the half-width of the depletion in $n_{F}(\rho)$, respectively. While the two widths are quite different at small $\gamma_{n}$, they cross each other at $\gamma_{n}^{c}$ with a length scale of $\sim 5/k_{F}$, and remain on the same length scale beyond the transition. We notice that the half-width of the depletion in $\Delta(\rho)$ is also close to $5/k_{F}$ at $\gamma_{n}^{c}$. Although different parameters have been employed to characterize the size of the vortex core in the previous literature Hu2006 ; Sensarma2006 , for convenience, here we use the half-width of fermion density depletion $d_{F}$ to estimate the size of the vortex core. From Fig. 2(a), it is clear that the size of the vortex core is on the order of $1/k_{F}$ for small $\gamma_{n}$, but becomes much larger beyond the transition.

An interesting property of the transition is the complete depletion of fermions at the center of the vortex. The fermion depletion can be understood by calculating the effective chemical potential $\mu_{\rm eff}(\rho)=\mu_{F}-g_{BF}n_{B}(\rho)$ of the fermions at $\rho=0$. When $\gamma_{n}>\gamma_{n}^{c}$, the variation length scale of the fermion density at the vortex core becomes several times of $1/k_{F}$ and the local density approximation (LDA) should be locally applicable. As the local pairing order parameter vanishes at $\rho=0$, according to the number equation for fermions, the local fermion density at $\rho=0$ should also vanish when the local chemical potential $\mu_{\rm eff}$ becomes negative. We show $\mu_{\rm eff}(0)$ as a function of $\gamma_{n}$ in Fig. 2(b), from which it is clear that the effective chemical potential undergoes a sharp transition at $\gamma_{n}^{c}$ and becomes negative as $\gamma_{n}>\gamma_{n}^{c}$. Numerically, we find the fermion density near the core center becomes vanishingly small when $\gamma_{n}>\gamma_{n}^{c}$. The complete depletion of $n_{F}(\rho)$ and the localization of the bosons discussed below can be seen as a phase separation Tylutki2016 . Accordingly, $\Delta(\rho)$ and $j(\rho)$ also vanish near the core center, which leads to the deviation of the order parameter from a linear scaling in $\rho$ and the disappearance of sharp features in the superfluid current distribution. This can be confirmed through a polynomial fitting of $\Delta(\rho)$ near the core center, where the fitting coefficients present sharp features at the transition point and become vanishingly small as $\gamma_{n}>\gamma_{n}^{c}$, as shown in Fig. 3.

The depletion of the fermions at the core is accompanied by the localization of the BEC wave function for $\gamma_{n}>\gamma_{n}^{c}$. Such a localization can be directly identified from the density distribution of the BEC as shown in Fig. 2(c), and is consistent with the variation of the half-width of its density distribution in Fig. 2(a). When $\gamma_{n}$ is small, the fermion depletion is limited both in particle number and in spatial range. The BEC density distribution is therefore extended, with the half-width comparable to the system size $R$. When $\gamma_{n}$ increases, the half-width decreases continuously, suggesting a stronger attractive potential provided by the vortex state and a localizing tendency of the extended BEC. Near the critical $\gamma_{n}^{c}$, the half-width of the boson density distribution is $\sim 5/k_{F}$, which is comparable to the size of the vortex core. By further increasing $\gamma_{n}$ beyond $\gamma_{n}^{c}$, the half-width begins to increase with $\gamma_{n}$, which is a clear signal that the BEC is localized in the core. After the localization of the BEC and the depletion of the fermions from the core, the Bose-Fermi interaction energy mostly comes from the Bose-Fermi boundary, which gives rise to a saturation of the interaction energy beyond $\gamma_{n}^{c}$ [see the inset of Fig. 2(c)]. The localization of the BEC is also reflected in the dependence of the half-width of the BEC density distribution on the system size $R$. As illustrated in Fig. 2(d), while the half-width increases significantly with $R$ for $\gamma_{n}<\gamma_{n}^{c}$, beyond the transition, the half-width is essentially insensitive to $R$.

The fermion depletion and the modified behavior of $\Delta(\rho)$ at the vortex core give rise to changes in the quasiparticle excitations and in the Andreev-like bound states at the vortex core. These changes are illustrated in Fig. 4. Beyond the critical $\gamma_{n}^{c}$, the nearly continuous in-gap spectrum will be pushed toward the bulk spectrum [see Fig. 4(a)-4(c)], with an appreciable increase in the lowest excitation energy for the Andreev-like bound states. The suppression of the in-gap bound states as well as the complete depletion of fermions at the core, are reflected in the local density of states (LDOS) $D(\rho,E)=2\sum_{n}|u_{n}(\rho)|^{2}\delta(E-E_{n})$ Hu2006 ; Iskin2008 at $\rho=0$. As shown in Fig. 4(d), while the sharp peak associated with the lowest excited state shrinks, states with $E<0$ become unoccupied in the LDOS for $\gamma_{n}>\gamma_{n}^{c}$.

V BCS-BEC crossover

In the previous discussions, we have been focusing on the vortex state in the BCS regime. As the Fermi superfluid is tuned across the BCS-BEC crossover, the transition in the vortex-core structure persists, while many features are modified. Away from the BCS regime, as the pairing order parameter becomes larger, the lowest excitation energy of the Andreev-like bound states also increases, which is similar to the case without BEC Sensarma2006 . Additionally, unique to the Bose-Fermi superfluid mixture, multiple branches of vortex bound states emerge from the bulk spectrum away from the BCS regime [see Fig. 5(a) and 5(b)], which is only possible when the core size becomes larger than the coherence length $\xi=\hbar v_{F}/\Delta_{0}$ Bardeen1969 . Here, $\Delta_{0}$ is the bulk value of $\Delta(\rho)$ and $\xi\backsim 1/k_{F}$ in the resonance regime Sensarma2006 . The condition above is facilitated by the larger fermion depletion at the vortex core in the strong-coupling regime, which enhances BEC localization and increases the core size; and by the smaller coherence length of the Fermi superfluid away from the BCS regime. We then identify the transition in the vortex core, for instance, from the half-width of the BEC density distribution [see Fig. 5(c)], or from features of the vortex state. As shown in Fig. 5(d), the critical $\gamma_{n}^{c}$ decreases as the Fermi-Fermi interaction is tuned toward the strong-coupling regime, which further confirms the enhanced BEC location toward the strong-coupling regime.

VI Summary and Final remarks

By analyzing the vortex state in the Fermi component, we have revealed an interesting transition in the vortex-core structure of a mixture of Bose and Fermi superfluids. We found that the vortex-core transition can lead to a variety of interesting features in quantities like the superfluid order parameter and the quasiparticle spectra of the fermions. In particular, we identified a partial depletion of the fermion density at the vortex core as well as the Andreev-like vortex bound states in the quasiparticle spectra. Such a transition is induced by the inter-species interaction between the two superfluid components. When the Fermi-Fermi interaction is tuned to the deep BEC regime, the vortex state discussed here can be reduced to the previously studied case of a two-species BEC becsf3 ; Anderson2000 ; Ho1996 ; Skryabin2000 ; Ripoll2000 ; Garcia2000 ; Jezek2001 ; Chui2001 ; Catelani2010 ; Mason2013 ; Ivashin2015 , where a transition of vortex-core structure is also present due to the interspecies interaction. However, we emphasize that the interesting features discussed in this work are unique for a Bose-Fermi superfluid mixture, as a vortex in a Fermi superfluid is linked to a phase singularity of the pairing field and hence acquires much richer structures away from the deep BEC regime.

Furthermore, although we have mostly considered a noninteracting BEC, we have checked that a similar transition exists for BEC with attractive interactions or with weak repulsive interactions. Strong repulsive interaction in the BEC would compete with the attractive potential of the vortex state, and thus prevent the localization of the BEC and the corresponding transition in the vortex core. We show the transition with an attractive interaction for the bosons in Fig. 6, where the critical $\gamma_{n}$ even becomes smaller. Unlike the case with $g_{B}=0$, the BEC is further localized due to the attractive interaction and the sharp feature on the $j(\rho)$ curve appears again as $\gamma_{n}$ far away from the critical point. In the case of a repulsive BEC, the parameter regime for the vortex-core transition is extremely narrow under current parameters. The transition in the vortex core should significantly change the size and visibility of the vortices, and it would be interesting to study its effects on the macroscopic vortex-lattice configurations under experimental conditions.

Acknowledgments

We thank G. Catelani for helpful comments. This work is supported by the National Key R&D Program (Grant No. 2016YFA0301700), the NKBRP (Grant No. 2013CB922000), the National Natural Science Foundation of China (Grants No. 60921091, No. 11274009, No. 11374283, No. 11434011, No. 11522436, and No. 11522545), and the Research Funds of Renmin University of China (Grants No. 10XNL016 and No. 16XNLQ03). W. Y. acknowledges support from the ”Strategic Priority Research Program(B)” of the Chinese Academy of Sciences, Grant No. XDB01030200.