Interplay between trimer structure and magnetic ground state in Ba5Ru3O12 probed by Neutron and muSR techniques

We report a detailed inelastic neutron scattering (INS) and muon spin relaxation (muSR) investigation of a trimer Ruthenate Ba5Ru3O12 system, which undergoes long-range antiferromagnetic ordering at TN = 60 K. The INS reveals two distinct spin wave excitations below TN: one at 5.6 meV and the other at 10-15 meV. By accompanying the INS spectra based on a linear spin wave theory using SpinW software and machine learning force fields (MLFFs), we show that Ba5Ru3O12 exhibits spin frustration due to competing exchange interactions between neighboring and next-neighboring Ru-moments, exchange anisotropy, and strong spin-orbit coupling, which yields a non-collinear spin structure, in contrast to other ruthenate trimers in this series. Interestingly, these magnetic excitations do not completely vanish even at high temperatures above TN, evidencing short-range magnetic correlations in this trimer system. This is further supported by muSR spectroscopy, which exhibits a gradual drop in the initial asymmetry around the magnetic phase transition and is further verified through maximum entropy analysis. The results of muSR spectroscopy indicate a dynamic nature of magnetic order, attributed to local magnetic anisotropy within the trimer as a result of local structural distortion and different hybridization, consistent with canted spin-structure. We predict the ground state of Ru3O12-isolated trimer through theoretical calculations, which agree with the experimentally observed spin excitation