Regression adjustment in randomized experiments with a diverging number of covariates

Extending R. A. Fisher and D. A. Freedman's results on the analysis of covariance, Lin [2013] proposed an ordinary least squares adjusted estimator of the average treatment effect in completely randomized experiments. We further study its statistical properties under the potential outcomes model in the asymptotic regimes allowing for a diverging number of covariates. We show that Lin [2013]'s estimator is consistent when $\kappa \log p \rightarrow 0$ and asymptotically normal when $\kappa p \rightarrow 0$ under mild moment conditions, where $\kappa$ is the maximum leverage score of the covariate matrix. In the favorable case where leverage scores are all close together, his estimator is consistent when $p = o(n / \log n)$ and is asymptotically normal when $p = o(n^{1/2})$. In addition, we propose a bias-corrected estimator that is consistent when $\kappa \log p\rightarrow 0$ and is asymptotically normal, with the same variance in the fixed-$p$ regime, when $\kappa^{2} p \log p \rightarrow 0$. In the favorable case, the latter condition reduces to $p = o(n^{2/3} / (\log n)^{1/3})$. Similar to Lin [2013], our results hold for non-random potential outcomes and covariates without any model specification. Our analysis requires novel analytic tools for sampling without replacement, which complement and potentially enrich the theory in other areas such as survey sampling, matrix sketching, and transductive learning.