Contrasting carrier doping effects in the Kondo insulator CeOs2 Al10: The influential role of c-f hybridization in spin-gap formation

The effects of electron (Ir) and hole (Re) doping on the hybridization gap and antiferromagnetic order have been studied by magnetization, muon spin relaxation (μ+SR), and inelastic neutron scattering on polycrystalline samples of Ce(Os1-xIrx)2Al10 (x=0.08 and 0.15) and Ce(Os1-yRey)2Al10 (y=0.03). μ+SR spectra clearly reveal bulk magnetic ordering below 20 and 10 K for x=0.08 and 0.15 samples, respectively, with heavily damped oscillations of the muon asymmetry. Our important findings are that a small amount of electron doping (i) completely suppresses the inelastic magnetic excitations at low temperatures, which were observed at 11 meV in the undoped compound, and the magnetic response transforms into a broad quasielastic response, and (ii) surprisingly, the internal field at the corresponding muon site is remarkably enhanced by an order of magnitude compared with the parent compound. Moreover, a small amount (3% Re) of hole doping results in a significant reduction of the intensity of 11 meV peak and an increased c-f hybridization, which is in agreement with the reduction of the Ce ordered moment seen through neutron diffraction and μ+SR. The main origin of the observed doping effect is an extra 5d electron being carried by Ir and a hole carried by Re with respect to the Os atom. The absence of a spin gap/spin wave, despite a larger ordered state moment of Ce, in the electron-doped system cannot be explained based on a conventional theory of magnetism. Thus, the obtained results demonstrate a great sensitivity to the carrier doping and provide additional ways to study the anomalous magnetic properties in the CeT2Al10 (T= Fe, Ru, and Os).