Density matrix renormalization group study of nematicity in two dimensions: Application to a spin-1 bilinear-biquadratic model on the square lattice

Nematic order is an exotic property observed in several strongly correlated systems, such as the iron-based superconductors. Using large-scale density matrix renormalization group (DMRG) techniques, we study at zero-temperature the nematic spin liquid that competes with spin dipolar and quadrupolar orders. We use these nematic orders to characterize different quantum phases and quantum phase transitions. More specifically, we study a spin-$1$ bilinear-biquadratic Heisenberg model on the square lattice with couplings beyond nearest neighbors. We focus on parameter regions around the highly symmetric $SU(3)$ point where the bilinear and biquadratic interactions are equal. With growing further-neighbor biquadratic interactions, we identify different spin dipolar and quadrupolar orders. We find that the DMRG results on cylindrical geometries correctly detect nematicity in different quantum states and accurately characterize the quantum phase transitions among them. Therefore, spin-driven nematicity -- here defined as the spontaneous breaking of the lattice invariance under a 90$^o$ rotation -- is an order parameter which can be studied directly in DMRG calculations in two dimensions in different quantum states.