Calibration of the JAGB method for the Magellanic Clouds and Milky Way from Gaia DR3, considering the role of oxygen-rich AGB stars

The JAGB method is a new way of measuring distances in the Universe with the use of asymptotic giant branch (AGB) that are situated in a selected region in a $J$ versus $J- colour--magnitude diagram (CMD), and relying on the fact that the absolute $J$ magnitude is (almost) constant. It is implicitly assumed in the method that the selected stars are carbon-rich AGB stars (carbon stars). However, as the sample selected to determine J $ is purely colour based, there can also be contamination by oxygen-rich AGB stars in principle. As the ratio of carbon-rich to oxygen-rich stars is known to depend on metallicity and initial mass, the star formation history and age--metallicity relation in a galaxy should influence the value of $M_ J $. The aim of this paper is to look at mixed samples of oxygen-rich and carbon-rich stars for the Large Magellanic Cloud (LMC), Small Magellanic Cloud (SMC), and Milky way (MW) using the Gaia catalogue of long-period variables (LPVs) as a basis. The advantage of this catalogue is that it contains a classification of O- and C-stars based on the analysis of Gaia Rp spectra. The LPV catalogue is correlated with data from the Two Micron All Sky Survey (2MASS) and samples in the LMC, SMC, and the MW are retrieved. Following methods proposed in the literature, we report the mean and median magnitudes of the selected sample using different colour and magnitude cuts and the results of fitting Gaussian and Lorentzian profiles to the luminosity function (LF). For the SMC and LMC, we confirm previous results in the literature. The LFs of the SMC and LMC JAGB stars are clearly different, yet it can be argued that the mean magnitude inside a selection box agrees at the 0.021 mag level. The results of our analysis of the MW sample are less straightforward. The contamination by O-rich stars is substantial for a classical lower limit of $(J- 1.3$, and becomes less than 10 only for $(J- 1.5$. The sample of AGB stars is smaller than for the MCs for two reasons. Nearby AGB stars (with potentially the best determined parallax) tend to be absent as they saturate in the 2MASS catalogue, and the parallax errors of AGB stars tend to be larger compared to non-AGB stars. Several approaches have been taken to improve the situation but finally the JAGB LF for the MW contains about 130 stars and the fit of Gaussian and Lorentzian profiles is essentially meaningless. The mean and median magnitudes are fainter than for the MC samples by about 0.4 mag which is not predicted by theory. We do not confirm the claim in the literature that the absolute calibration of the JAGB method is independent of metallicity up to solar metallicity. A reliable calibration of the JAGB method at (near) solar metallicity should await further data releases, or should be carried out in another environment.