Amortized simulation-based frequentist inference for tractable and intractable likelihoods

High-fidelity simulators that connect theoretical models with observations are indispensable tools in many sciences. If the likelihood is known, inference can proceed using standard techniques. However, when the likelihood is intractable or unknown, a simulator makes it possible to infer the parameters of a theoretical model directly from real and simulated observations when coupled with machine learning. We introduce an extension of the recently proposed likelihood-free frequentist inference (LF2I) approach that makes it possible to construct confidence sets with the p-value function and to use the same function to check the coverage explicitly at any given parameter point. Like LF2I, this extension yields provably valid confidence sets in parameter inference problems for which a high-fidelity simulator is available. The utility of our algorithm is illustrated by applying it to three pedagogically interesting examples: the first is from cosmology, the second from high-energy physics and astronomy, both with tractable likelihoods, while the third, with an intractable likelihood, is from epidemiology 3 3 Code to reproduce all of our results is available on https://github.com/AliAlkadhim/ALFFI.. Code to reproduce all of our results is available on https://github.com/AliAlkadhim/ALFFI.