Metal-Insulator Transition in the Dimerized Organic Conductor $\kappa$-(BEDT-TTF)$_2$Hg(SCN)$_2$Br

TL;DR

This study investigates the metal-insulator transition in the organic conductor $$-(BEDT-TTF)$_2$Hg(SCN)$_2$Br through transport, dielectric, and optical measurements, comparing it with similar salts to understand the transition's nature.

Contribution

It provides comprehensive experimental analysis of the metal-insulator transition in a specific organic conductor and compares it with related compounds to elucidate underlying mechanisms.

Findings

Transition occurs below 80 K with a large increase in resistivity.

Comparison with $$-analogue reveals differences in charge-order behavior.

Experimental data suggest a complex interplay of electronic correlations.

Abstract

The organic charge-transfer salt $\kappa$-(BEDT-TTF)$_{2}$Hg(SCN)$_{2}$Br is a quasi two-dimensional metal with a half-filled conduction band at ambient conditions. When cooled below $T=80$ K it undergoes a pronounced transition to an insulating phase where the resistivity increases many orders of magnitude. In order to elucidate the nature of this metal-insulator transition we have performed comprehensive transport, dielectric and optical investigations. The findings are compared with other dimerized $\kappa$-(BEDT-TTF) salts, in particular the Cl-analogue, where a charge-order transition takes place at $T_{\rm CO}=30$ K.

Equations

$$\epsilon (\omega )-{\epsilon }_{\mathrm{HF}}=\frac{\Delta \epsilon }{1+{\left(\mathrm{i}\omega {\tau }_0\right)}^{1-\alpha }}$$

Full text

I Introduction

The current interest in dimerized two-dimensional organic conductors mainly focusses on $\kappa$-(BEDT-TTF)${}_{2}X$ salts built with extended polymeric anions $X^{-}$ such as Cu(SCN)${}_{2}^{-}$, Cu[N(CN)2]Br-, Cu[N(CN)2]Cl-, or Cu2(CN)${}_{3}^{-}$, all containing copper ions. These systems are characterized by half-filled conduction bands and are located in proximity to correlated insulating states. Depending on the strength of the on-site Coulomb repulsion $U$ with respect to the bandwidth $W$, a metallic and superconducting ground state develops at low temperatures, or a Mott insulator, which might be antiferromagnetically ordered or behaves like a spin liquid due to strong frustrations.Ishiguro98 ; Kanoda11 ; Powell11

Subsequent theoretical approaches Hotta10 ; Naka10 ; Gomi10 ; Dayal11 suggest that also the Coulomb interaction $V$ between the dimers has to be included in the description of possible charge-ordering phenomena in these dimerized (BEDT-TTF) salts. Although numerous experimental results,Sedlmeier12 ; Tomic13 ; Pinteric14 ; Shimizu06 ; Yakushi15 ; Dressel16 rule out appreciable charge disproportionation in $\kappa$-(BEDT-TTF)2Cu[N(CN)2]Cl, $\kappa$-(BEDT-TTF)2Cu2(CN)3 and other copper-based $\kappa$-(BEDT-TTF)${}_{2}X$ salts, these considerations might be relevant for dimerized salts in general. Unfortunately, charge-order phenomena in $\kappa$-phase salts with an effectively half-filled band are scarce and only a few compounds are reported exhibiting charge order, such as $\kappa$-(BEDT-TTF)4PtCl6$\cdot$C6H5CN, the triclinic $\kappa$-(BEDT-TTF)4[$M$(CN)6][N(C2H5)4]$\cdot$3H2O and the monoclinic $\kappa$-(BEDT-TTF)4[$M$(CN)6][N(C2H5)4]$\cdot$2H2O (with $M$ = CoIII, FeIII, and CrIII) salts.Doublet94 Here the structure is rather complex as the phase transition includes the deformation of the molecule and the interaction with the anions; accordingly, details of their physical properties and their electronic states are not well known. Certainly electronic correlations as well as coupling to the lattice are important.

Recently it was demonstrated that $\kappa$-(BEDT-TTF)2Hg(SCN)2Cl undergoes a pronounced charge order transition at $T_{\rm CO}=30$ K,Yasin12 ; Drichko14 as illustrated in Fig. 1. The insulating state is rapidly suppressed by hydrostatic pressure of $p=4.2$ kbar without any signs of superconductivity;Lohle17 making this compound distinct from $\kappa$-(BEDT-TTF)2Cu[N(CN)2]Cl, where a tiny amount of pressure drives the Mott insulator superconducting, but also from the canonical charge-ordered system $\alpha$-(BEDT-TTF)2I3.Dressel94 ; Beyer16 The present investigation now addresses the question in which way $\kappa$-(BEDT-TTF)2Hg(SCN)2$X$ is modified when replacing $X$ = Cl by Br in the anion sheet. Let us recall that in the case of $\kappa$-(BEDT-TTF)2Cu[N(CN)2]Cl the corresponding substitution pushes the antiferromagnetic Mott insulator into a superconducting phase with $T_{c}\approx 12$ K.Ishiguro98

The family of $\kappa$-(BEDT-TTF)${}_{2}X$ salts with mercury-based anions have a structure similar to the Cu-salts,Lyubovskii96 but different ratios of transfer integrals, which result in modified parameters used to map them on the Hubbard model.Jeschke In particular the orbital overlap $t_{d}$ within the dimers of the Hg-compounds is weaker compared to the the Cu-family, causing a reduced on-site repulsion $U$ with respect to the inter-dimer interaction. This makes it necessary to go beyond the simple Hubbard model, which allowed to treat the Cu-based $\kappa$-(BEDT-TTF) compounds successfully as half-filled system with one electron per dimer.Seo04 ; Powell11 Now the description has to start from individual BEDT-TTF molecules, leading to a quarter-filled conduction band; in addition to $U$ the inter-molecular interaction $V$ has to be included in the extended Hubbard model.

In an effective dipolar-spin model Hotta suggests Hotta10 that quantum electric dipoles are formed on the dimers, which interact with each other and thus modify the exchange coupling. Since they fluctuate by $t_{d}$, for large orbital overlap the dimer Mott insulator is stable, forming a dipolar liquid; if $V$ is large compared to $t_{d}$, however, charge order emerges (dipolar solid). Similar considerations have been put forward by other groups.Naka10 ; Gomi10 Mazumdar, Clay and collaborators Dayal11 could show that in these system a frustration-induced transition occurs from a Néel antiferromagnetism to a spin-singlet state in the interacting quarter-filled band on an anisotropic triangular lattice. In the spin-singlet state the charge on the molecules becomes unequal: a paired-electron crystal is formed with pairs of charge-rich sites separated by pairs of charge-poor sites. This spin-singlet formation is driven by quantum effects at any value of nearest neighbor interaction $V$.

Here we present the first comprehensive transport, dielectric and optical investigations of $\kappa$-(BEDT-TTF)2Hg(SCN)2Br and discuss the findings within the framework of dimerized Mott insulators composed by these Hg- and Cu-families.

II Experimental Details

A series of $\kappa$-(BEDT-TTF)2Hg(SCN)${}_{3-n}X_{n}$ salts with $X$ = F, Br, I ($n=1$) and $X$ = Cl ($n=1,2$) has been synthesized by Lyubovskii and collaborators already twenty years ago,Aldoshina93 ; Konovalikhin93 ranging from insulators to metals and possible superconductors;Lyubovskii96 a thorough investigation of their physical properties, however, is still lacking. Here we have turned our attention to $\kappa$-(BEDT-TTF)2Hg(SCN)2Br, an isostructural sister compound of the charge-ordering $\kappa$-(BEDT-TTF)2Hg(SCN)2Cl; as demonstrated in Fig. 1, it also exhibits a metal-insulator transition upon cooling, but at somewhat higher temperatures.

Based an recent x-ray scattering experiments at ambient conditions, the crystal structure of $\kappa$-(BEDT-TTF)2Hg(SCN)2Br is depicted in Fig. 2 along two crystallographic directions. The BEDT-TTF molecules are all crystallographically equivalent and within the $bc$-plane form dimers according to the $\kappa$-patternMori99 that are rotated with respect to each other. As seen from Fig. 2(b) the stacking direction $b$ is more pronounced than known from other $\kappa$-salts leading to a rather isotropic response in our transport and optical measurements.

It should be noted that during the electrochemical synthesis, single crystals of the $\beta^{\prime\prime}$-phase tend to grow as well to a considerable size (1–2 mm),Li16 and subsequently have to be separated from the desired $\kappa$-phase specimens with the help of x-ray or infrared optical inspection.

In the temperature-dependent dc resistivity of $\kappa$-(BEDT-TTF)2Hg(SCN)2Br displayed in Fig. 1 a sharp metal-insulator transition is identified at $T=80$ K, confirming the early characterization.Aldoshina93 While for $\kappa$-(BEDT-TTF)2Hg(SCN)2Cl a saturation of $\rho(T)$ is observed at very low temperatures, for the Br-analogue the slope $|{\rm d}\rho(T)/{\rm d}T|$ continues to rise as the temperature is lowered. For a more complete study we have measured the charge transport within the highly-conducting $(bc)$ plane, as well as perpendicular to it, using the standard four-probe technique. The sample was slowly cooled (up to 0.5 K/h) from room temperature down to $T=4$ K using a home-made helium bath cryostat.

In addition, the spectra of the complex dielectric function were obtained from measuring the two-contact complex conductance in the frequency range from 20 Hz to 10 MHz for various temperatures. The Hewlett-Packard 4284A LCR meter and Agilent 4294A impedance analyzer with virtual ground method were used. The capacitive open-loop contribution of the sample holder was always subtracted.

The optical properties of $\kappa$-(BEDT-TTF)2Hg(SCN)2Br have been investigated by infrared reflectivity measurements from room temperature down to $T=4$ K for both polarizations along the main axes of the highly-conducting $(bc)$ plane. To this end a Bruker Hyperion infrared microscope was attached to the Fourier-transform spectrometer Bruker IFS 66v/s or Vertex 80v. The sample was cooled down to helium temperatures by a Cryovac Microstat cold-finger cryostat. The far-infrared data were taken by a Bruker IFS 113v equipped with a cold-finger cyostat and in-situ gold evaporation as reference.Dressel04 In order to perform the Kramers-Kronig analysis, a constant reflectivity extrapolation was used at low frequencies and temperatures $T_{\rm MI}<80$ K, while a Hagen-Rubens behavior was assumed for elevated temperatures.DresselGruner02 In addition, the vibrational features were measured perpendicular to the plane ($E\parallel a$) using an infrared microscope. In particular the ungerade C=C vibration $\nu_{27}({\rm b}_{1u}$) of the BEDT-TTF molecules are utilized as a local probe to determine the charge per molecule Dressel04 ; Drichko09 ; Yamamoto05 ; Girlando11 and follow any possible charge disproportionation.

III Results and Analysis

III.1 Transport Properties

In order to gain more information on the transport mechanisms of $\kappa$-(BEDT-TTF)2Hg(SCN)2Br above and below the phase transition at $T_{\rm MI}=80$ K, we have performed dc resistivity measurements along different crystal directions and present the results down to $T=20$ K in an Arrhenius plot in Fig. 3(a) as a function of inverse temperature. At ambient conditions, the resistivity within the $(bc)$-plane is approximately 0.5 $\Omega$cm with an anisotropy of less than a factor of 2. Perpendicular to the highly-conducting plane, i.e., parallel to the $a$-axis, $\rho_{\rm dc}$ is about 20 times higher. For all orientations a very similar temperature dependent metallic resistivity is observed down to 100 K. At $T_{\rm MI}$ a metal-insulator transition occurs in all three directions with a more or less steep rise in $\rho(T)$ by an order of magnitude or more. It is interesting to note that this temperature-behavior very much resembles the metal-insulator transition observed in other ET-based compounds such as $\alpha$-(BEDT-TTF)2I3 and $\theta$-(BEDT-TTF)2RbZn(SCN)4,Tomic15 where the gap at the density of states rapidly opens at the transition. The insulating state is characterized by an activated behavior $\rho(T)\propto\exp\{E_{g}/2k_{B}T\}$ down to lowest temperatures; no saturation is observed up to our limit of $10^{9}\,\Omega$cm. Within the plane ($E\parallel bc$) the activation energy of $E_{g}/2=(22\pm 3)$ meV is basically constant down to $T=20$ K. In the out-of-plane direction ($E\parallel a$), we extract a similar activation energy of $E_{g}/2=(15.5\pm 2)$ meV, which means it is rather isotropic. If we consider $E_{g}$ as the full gap in the density of states, the ratio $E_{g}/k_{B}T_{\rm MI}\approx 4-5.6$ is not much higher than 3.53 expected from a conventional mean-field transition. Thus we can consider this system mean-field-like with relatively weak electronic correlations.

III.2 Dielectric Properties

Like in many other charge-ordered (BEDT-TTF) compounds,Ivek11 ; Tomic13 ; Pinteric14 ; Pinteric16 the dielectric properties of $\kappa$-(BEDT-TTF)2Hg(SCN)2Br can be described by a generalized Debye expression

[TABLE]

at low temperatures, where $\Delta\varepsilon=\varepsilon_{0}-\varepsilon_{\rm HF}$. corresponds to the strength of the dielectric mode; $\varepsilon_{0}$ and $\varepsilon_{\rm HF}$ are the static and high-frequency dielectric constants, respectively; $\tau_{0}$ is the mean relaxation time; and $1-\alpha$ is the symmetric broadening of the relaxation time distribution. The temperature dependences of the extracted parameters $\Delta\varepsilon$, $1-\alpha$, and $\tau_{0}$ are plotted in Fig. 4 as a function of inverse temperature $1/T$. The dielectric relaxation can be detected only at temperatures below 60 K. Above $T\approx 35$ K we can determine only the dielectric relaxation strength by measuring the capacitance at 1 MHz. At first glance the relaxation appears rather broad. The dielectric strength perpendicular to planes ($E\parallel a$) is only of the order of 10 and less while the strength of the dielectric response $E\parallel bc$ is on the order of 100 (not shown), which mirrors the similarly weak anisotropy found in dc data. Most important, for both orientations the intensity of the modes becomes smaller by about an order of magnitude as the temperature is reduced to 10 K; these findings are in contrast to the behavior commonly observed in low-dimensional organic charge transfer salts.Tomic15 It clearly indicates that $\kappa$-(BEDT-TTF)2Hg(SCN)2Br does not exhibit indications of charge order and shows only very weak relaxor-like properties, further underlining the absence of strong electronic correlations.

For both orientations, the temperature behavior of $\tau_{0}(T)$ shows no saturation and basically follows an Arrhenius law between 40 and 12 K, as plotted in Fig. 4(c). The activation energy is comparable with the dc resistivity. This indicates that electronic screening by quasi-free charge carriers is the dominant relaxation mechanism.

Around $T\approx 20-30$ K we find the most pronounced change in the dielectric behavior that can be seen even better by directly looking at the temperature dependence of the dielectric constant. In Fig. 5, we present $\varepsilon^{\prime}(T)$ measured at different frequencies for the orientations parallel and perpendicular to the $(bc)$-plane. A clear step can be seen around $T=20$ K that becomes more pronounced as the frequency increases to 1 MHz. This is accompanied by a change in $1-\alpha$ and $\Delta\varepsilon$ at the same temperatures in Fig. 4, which suggests the entities responsible for the dielectric response are not correlated as temperature is lowered.

III.3 Infrared-active Molecular Vibrations

Microscopic information on the charge distribution and its temperature behavior can be obtained by optical spectroscopy. To this end we performed infrared reflectance measurements for the polarization perpendicular to the layers, i.e., $E\parallel a$. In this orientation the optical response resembles an insulator with low reflectivity and correspondingly low values of conductivity, in agreement with the quasi-two-dimensional character of these materials, deduced from the dc conductivity presented in Fig. 3.

Of primary interest, however, are the superimposed infrared-active vibrational features of the BEDT-TTF molecule and lattice. The out-of-phase molecular vibrations of the two C=C bonds of the inner rings provide a powerful and well-established method to identify the amount of charge on molecular lattice sites.Dressel04 ; Drichko09 ; Girlando11 The frequency dependence of the $\nu_{27}({\rm b}_{1u})$ mode allows for a quantitative determination of the charge per BEDT-TTF molecule and thus the most accurate way to monitor the temperature-dependence of the charge disproportionation.

The resulting conductivity spectra in the region of the $\nu_{27}({\rm b}_{1u})$ mode are presented in Fig. 6 for $\kappa$-(BEDT-TTF)2Hg(SCN)2Br in comparison with spectra of the Cl-analogue. $\kappa$-(BEDT-TTF)2Hg(SCN)2Cl is known Drichko14 to undergo a charge-order transition around $T_{\rm CO}=30$ K, as can be clearly seen from the splitting of the $\nu_{27}({\rm b}_{1u})$ vibration, which peaks around 1455 cm*-1* at room temperature and narrows on cooling down. A higher-frequency shoulder can be identified that becomes more pronounced at low temperatures; this double features indicates the two crystallographically different sites per unit cell. For a quantitative analysis we fit the bands by two Fano resonances for $T>30$ K and four modes below $T_{\rm CO}$; the results are plotted in the inset and perfectly agree with our previous observations Drichko14 where we concluded a charge difference of $2\delta_{\rho}=0.2e$ between two different molecular sites. Löhle et al. studied the pressure dependence of the charge ordered phase Lohle17 and found a rapid suppression of $T_{\rm CO}$ by about 0.7 kbar and the absence of charge disproportionation for $p>4$ kbar.

For the present case of $\kappa$-(BEDT-TTF)2Hg(SCN)2Br nothing like that is observed in the lower panel of Fig. 6. At room temperature the broad $\nu_{27}({\rm b}_{1u})$ feature occurs at 1457 cm*-1with a comparable width and temperature dependence down to $T\approx 100$ K. No drastic change is observed when the sample is cooled further below $T_{\rm MI}$, in a way similar to Cu-based $\kappa$-BEDT-TTF${}_{2}X$ systems.Sedlmeier12 At the lowest temperature, the double peak structure around 1460 cm-1* has slightly changed in so far as the higher-frequency peak is more pronounced. In addition, a second feature develops at 1445 cm*-1at below temperatures as high as 150 K, meaning it is not related to the metal-insulator transition. Even though it is tempting to ascribe its temperature evolution to the establishing of the low-temperature phase, it appears to be governed by thermal broadening: the width and strength behave in a manner similar to the dominant 1460 cm-1* peak as well as the normal-phase $\nu_{27}({\rm b}_{1u})$ peak of the $\kappa$-(BEDT-TTF)2Hg(SCN)2Cl analogue.

It is interesting to note that the $\nu_{27}$ mode observed in $\kappa$-(BEDT-TTF)2Hg(SCN)2Br is much broader than in the Cl-analogue and comparable to the ones found in the spin-liquid candidates $\kappa$-(BEDT-TTF)2Cu2(CN)3 and $\kappa$-(BEDT-TTF)2Ag2(CN)3,Sedlmeier12 ; Pinteric16 where charge inhomogeneities or fluctuations might be of relevance.Tomic15 ; Pinteric16 These findings provide evidence that the metal-insulator transition observed in $\kappa$-(BEDT-TTF)2Hg(SCN)2Br at 80 K is not due to static charge ordering. Recent measurements of the Raman spectra as a function of temperature support this conclusion: no indications of charge imbalance has been observed Hasan17 as the metal-insulator transition is passed.

III.4 In-Plane Infrared Properties

The electrodynamic properties of $\kappa$-(BEDT-TTF)2Hg(SCN)2Br were investigated by measuring the reflectivity at different temperatures for $4\,{\rm K}\leq T<300$ K; in Fig. 7 we plot $R(\omega)$ for the two polarizations $E\parallel c$ and $E\parallel b$ together with the optical conductivity obtained by the Kramers-Kronig relations. The quasi-two-dimensional conductor displays a rather weak anisotropy in its optical response, very similar to its Cl-analogue $\kappa$-(BEDT-TTF)2Hg(SCN)2Cl and to $\kappa$-(BEDT-TTF) salts with Cu-containing anions. From the crystal structure [Fig. 2(b)] we can identify the $b$-axis as preferred stacking direction of the BEDT-TTF molecules.

Most obvious, the mid-infrared band is more intense along the $b$-direction [Fig. 7(c,d)], leading to a drop in $R(\omega)$ around 3500 cm*-1* that resembles the plasma edge of regular metals. We associate this optical anisotropy to the arrangement of the BEDT-TTF dimers as depicted in Fig. 2(b): the intra-dimer charge transfer is in particular excited by $E\parallel b$. The low-frequency reflectivity is rather small, between 0.4 and 0.6. Only when cooled down to $T=100$ K a pronounced Hagen-Rubens behavior is measured. This corresponds to the increase in conductivity by a factor of 3 to 4 (see insets of Fig. 7), in good agreement with the metallic temperature dependence seen in dc resistivity (Figs. 1 and 3) above the metal-to-insulator transition.

From the mid-infrared conduction band we can extract the electronic parameters,Dressel04 ; Faltermeier07 ; Merino08 ; Dumm09 in particular the maximum $\nu_{\rm max}$ is proportional to the Coulomb repulsion $U$. At $T=100$ K we find approximately 2200 cm*-1* for $E\parallel c$, while for $E\parallel b$ it is only 1950 cm*-1*. In both cases the values increase by about 10% as the temperature rises to room temperature. The overall behavior is similar to the observations reported for $\kappa$-(BEDT-TTF)2Hg(SCN)2Cl, however, the absolute values are reduced, indicating smaller effect of electronic correlations.

We describe the zero-frequency contributions by a Drude-like conductivity. In accord to the dc data, the spectral weight $\int\sigma(\omega){\rm d}\omega=\omega_{p}/8$ obtained for both directions differs by a factor of 2, independent on temperature. For most-conducting direction ($E\parallel b$) the plasma frequency $\omega_{p}/(2\pi c)=1500$ cm*-1* at ambient temperature increases to 4000 cm*-1* at $T=100$ K. Here the width increases by a slightly more than a factor of 2 when the temperature is reduced to $T_{\rm MI}$. Within the uncertainty, no change is observed along the $c$-direction.

For $\kappa$-(BEDT-TTF)2Hg(SCN)2Br we observe a larger spectral weight of the Drude component compared to the one extracted for the Cl-analogue, corroborating our conclusion drawn above from the mid-infrared band, that the title compound is less-correlated than $\kappa$-(BEDT-TTF)2Hg(SCN)2Cl. A similar order was observed for the copper-based salts, as $\kappa$-(BEDT-TTF)2Cu[N(CN)2]Cl is a Mott insulator while the Br-analogue becomes superconducting.

For $T<T_{\rm MI}$ the low-frequency reflectivity drops significantly in both polarizations. Although strong phonon bands around 200 - 450 cm*-1* cause considerable intensity in $\sigma_{1}(\omega)$ even at $T=4$ K, we might identify an isotropic gap around 500 cm*-1* for both directions. Within experimental errors, this value agrees nicely with the transport gap extracted from dc data. Additionally, we note that frequencies up to 1000 cm*-1* the optical conductivity decreases when cooling from 70 to 4 K which indicates a sort of soft gap behavior. Accordingly the spectral weight is shifted to the mid-infrared band, which grow considerably.

IV Discussion

By now it is well established that $\kappa$-(BEDT-TTF)2Hg(SCN)2Cl undergoes a metal-insulator transition due to charge ordering at $T_{\rm CO}=30$ K: the resistivity $\rho_{\rm dc}(T)$ continuously rises by orders of magnitude (Fig. 1) and the charge-sensitive vibrational $\nu_{27}({\rm b}_{1u})$-mode splits (Fig. 6) indicating a charge disproportionation $2\delta_{\rho}=0.2e$ at low temperatures. In contrast, no clear splitting of the $\nu_{27}({\rm b}_{1u})$-mode is observed in the sibling $\kappa$-(BEDT-TTF)2Hg(SCN)2Br, ruling out any significant charge disproportionation. Hence, $\kappa$-(BEDT-TTF)2Hg(SCN)2Br does not enter a charge-ordered ground state.

Applying minor pressure, the Mott insulator with Cu-based anions $\kappa$-(BEDT-TTF)2Cu[N(CN)2]Cl turns metallic and superconducting, very much like $\kappa$-(BEDT-TTF)2Cu[N(CN)2]Br at ambient pressure. In the case of the Hg-based $\kappa$-(BEDT-TTF)2Hg(SCN)2Cl the relation to the Br-analogue is rather different: pressure rapidly suppresses the charge-ordered state in $\kappa$-(BEDT-TTF)2Hg(SCN)2Cl, which becomes metallic at all temperatures.Lohle17 This is in striking contrast to $\kappa$-(BEDT-TTF)2Hg(SCN)2Br which shows no charge ordering but still features a metal-insulator transition of a different kind.

It has been suggested Hotta10 that charge fluctuations within the dimers may couple to neighboring entities; and in particular in the frustrated case of a triangular lattice it may give rise to the spin liquid state. Pressure enhances the interdimer interaction and may explain why $\kappa$-(BEDT-TTF)2Hg(SCN)2Cl exhibits a strong and static charge order, while only charge fluctuations may be present in $\kappa$-(BEDT-TTF)2Hg(SCN)2Br.

For the organic Mott insulators $\kappa$-(BEDT-TTF)2Cu[N(CN)2]Cl and $\kappa$-(BEDT-TTF)2Cu2(CN)3 the temperature evolution of dielectric spectra resembles relaxor ferroelectrics.Abdel-Jawad2010 ; Lunkenheimer12 ; Tomic13 ; Pinteric14 In the case of $\kappa$-(BEDT-TTF)2Hg(SCN)2Cl the behavior is similar to other charge-ordered compounds, such as $\alpha$-(BEDT-TTF)2I3,Ivek11 ; Ivek17 where two dielectric relaxation modes appeared in the kHz-MHz frequency range. Nothing like that however is observed in $\kappa$-(BEDT-TTF)2Hg(SCN)2Br, supporting our conclusion that the compound is not a charge-order insulator, but also that it is not clear whether the disorder in the molecular and anion layers is an issue here. The temperature-dependence of the dielectric parameters indicate some relaxational behavior screened by the conduction electrons remaining below the phase transition. It is 30 times more pronounced within the $(bc)$-plane compared to the perpendicular direction. Below 30 K the mode vanishes rapidly and is basically absent at $T=10$ K, although in this temperature range no additional transition can be identified in the transport or optical properties. It might be tempting to attribute the dielectric response to dipolar fluctuations,Tomic15 forming a kind of non-Barrett quantum electric-dipole liquid.Shen16 Since pure dipolar coupling does not cause frustration in a two-dimensional triangular lattice, the transfer integrals are significant and do increase when going from $\kappa$-(BEDT-TTF)2Hg(SCN)2Br to the Cl-analogue.

The low-temperature optical conductivity of $\kappa$-(BEDT-TTF)2Hg(SCN)2Br exhibits a shape similar to the room-temperature spectra of other $\kappa$-(BEDT-TTF) salts, such as $\kappa$-(BEDT-TTF)2Cu[N(CN)2]Cl, with a gap at low frequencies, strong electron-molecular vibrational (emv) coupled features, and a band at about 2000 cm*-1*. The electronic properties of these compounds can be well described by assuming a lattice of dimers with one electron per site. The simple Hubbard model of a half-filled system describes the main physics rather well. In comparison to $\kappa$-(BEDT-TTF)2Hg(SCN)2Cl and $\kappa$-(BEDT-TTF)2Cu[N(CN)2]Cl, the effect of electronic correlations seems to be less important in $\kappa$-(BEDT-TTF)2Hg(SCN)2Br. This is concluded from the mean-field-like dc transport and maximum of the mid-infrared conductivity band, but also from the spectral weight of the Drude component (Fig. 7). The zero-frequency peak is higher than typically observed in the Cu-family and it shows up at high temperatures, as a result of smaller Hubbard on-site repulsion $U$.Merino08 ; Dumm09

With the metal-insulator transition prominently at 80 K, we do not observe a gradual increase in resistivity of $\kappa$-(BEDT-TTF)2Hg(SCN)2Br (Fig. 1) as known from the dimerized Mott insulators $\kappa$-(BEDT-TTF)2$X$ with Cu-ions in the anion layers, such as $\kappa$-(BEDT-TTF)2Cu[N(CN)2]Cl or $\kappa$-(BEDT-TTF)2Cu2(CN)3.Sedlmeier12 This leads us to conclude that the $T=80$ K transition is not a regular Mott transition. This is supported by the analysis of our optical data and structural considerations. The rapid change in resistivity must therefore have other reasons, most likely some subtle magnetic ordering. In this regard it is of interest to recall ESR investigations by Yudanova et al., who measured hydrogenated and deuterated $\kappa$-(BEDT-TTF)2Hg(SCN)2Br and $\kappa$-(BEDT-TTF)2Hg(SCN)2Cl down to low temperature.Yudanova94 After subtracting a Curie contribution, the spin susceptibility vanishes at the metal-insulator transition, following $\chi(T)=C/T\exp\{-\Delta/T\}$, with $\Delta\approx 500$ K which agrees nicely with the charge gap extracted from our optical and dc transport data. They conclude a first-order structural phase transition with a considerable localization of the electrons on the BEDT-TTF molecules. Recent low-temperature x-ray diffraction studies, however, did not reveal any evidence of a unit cell doubling or other structural changes in $\kappa$-(BEDT-TTF)2Hg(SCN)2Br. Thus we cannot explain the change in transport and magnetic properties by a change in the crystal structure. Comprehensive investigations of magnetic properties are on its way in order to clarify the magnetic ground state of $\kappa$-(BEDT-TTF)2Hg(SCN)2Br as well as the Cl-analogue. Additionally, conventional density-wave instabilities are excluded by the absence of structural changes as seen by x-ray and infrared vibrations. Future low-temperature x-ray experiments are needed to explore if subtler structural effects play a role in the fluctuating charge present in this compound.

V Conclusions and Outlook

From our comprehensive transport, dielectric and optical investigations of the dimerized organic charge transfer salt $\kappa$-(BEDT-TTF)2Hg(SCN)2Br, we can rule out that the metal-insulator transition at $T=80$ K is due to a charge ordering. No pronounced structural change takes place as well. We can estimate the effective correlations to be weaker compared to the $\kappa$-(BEDT-TTF) salts containing Cu-based anions. The system is also more metallic than the sister compound $\kappa$-(BEDT-TTF)2Hg(SCN)2Cl. Although electronic correlations do play a decisive role, the phase below the 80 K transition cannot be explained as a simple Mott insulator. We suggest that magnetic fluctuations might play an important role in supporting charge fluctuations revealed by our infrared vibrational spectra.

Acknowledgements.

We do acknowledge valuable discussion with N. Drichko and A. Pustogow. The project was supported by the Deutsche Forschungsgemeinschaft (DFG) by DR228/39-1 and DR228/52-1, the Deutsche Akademischer Austauschdienst (DAAD), as well as the Croatian Science Foundation project IP-2013-11-1011.