/ AbstractExperiments aiming to directly detect dark matter through particle recoils can achieve energy thresholds of $\mathcal{O}(1\,\mathrm{eV})$. In this regime, ionization signals from small-angle Compton scatters of environmental $γ$-rays constitute a significant background. Monte Carlo simulations used to build background models have not been experimentally validated at these low energies. We report a precision measurement of Compton scattering on silicon atomic shell electrons down to 23$\,$eV. A skipper charge-coupled device (CCD) with single-electron resolution, developed for the DAMIC-M experiment, was exposed to a $^{241}$Am $γ$-ray source over several months. Features associated with the silicon K, L$_{1}$, and L$_{2,3}$-shells are clearly identified, and scattering on valence electrons is detected for the first time below 100$\,$eV. We find that the relativistic impulse approximation for Compton scattering, which is implemented in Monte Carlo simulations commonly used by direct detection experiments, does not reproduce the measured spectrum below 0.5$\,$keV. The data are in better agreement with $ab$ $initio$ calculations originally developed for X-ray absorption spectroscopy.