Search for higgs bosons decaying to aa in the µµττ final state in pp collisions at √s = 8 TeV with the ATLAS experiment

A search for the decay to a pair of new particles of either the 125 GeV Higgs boson (h) or a second charge parity (CP)-even Higgs boson (H) is presented. The data set corresponds to an integrated luminosity of 20.3 fb^-1 of pp collisions at √s = 8 TeV recorded by the ATLAS experiment at the LHC in 2012. The search was done in the context of the next-to-minimal supersymmetric standard model, in which the new particles are the lightest neutral pseudoscalar Higgs bosons (a). One of the two a bosons is required to decay to two muons while the other is required to decay to two τ leptons. No significant excess is observed above the expected backgrounds in the dimuon invariant mass range from 3.7 to 50 GeV. Upper limits are placed on the production of h → aa relative to the standard model gg → h production, assuming no coupling of the a boson to quarks. The most stringent limit is placed at 3.5% for m_a = 3.75 GeV. Upper limits are also placed on the production cross section of H → aa from 2.33 to 0.72 pb, for fixed m_a = 5 GeV with m_H ranging from 100 to 500 GeV.