On the computation of a non-parametric estimator by convex optimization

Estimation of linear functionals from observed data is an important task in many subjects. Juditsky & Nemirovski [The Annals of Statistics 37.5A (2009): 2278-2300] propose a framework for non-parametric estimation of linear functionals in a very general setting, with nearly minimax optimal confidence intervals. They compute this estimator and the associated confidence interval by approximating the saddle-point of a function. While this optimization problem is convex, it is rather difficult to solve using existing off-the-shelf optimization software. Furthermore, this computation can be expensive when the estimators live in a high-dimensional space. We propose a different algorithm to construct this estimator. Our algorithm can be used with existing optimization software and is much cheaper to implement even when the estimators are in a high-dimensional space, as long as the Hellinger affinity (or the Bhattacharyya coefficient) for the chosen parametric distribution can be efficiently computed given the parameters. We hope that our algorithm will foster the adoption of this estimation technique to a wider variety of problems with relative ease. There are many situations where one wishes to estimate linear functionals of an unknown state using only observations of quantities determined by the state (i.e., indirect measurements of the state). Such a scenario is prevalent, for example, in tomography. Therefore, one would ideally like to incorporate these different measurements in such a way that not only we find a good estimate of the linear functional, but also ensure that the associated confidence interval is tight. Juditsky & Nemirovski [1], extending prior work of Donoho [2], propose an approach to this problem that constructs an estimator for a specified linear functional algorithmically, incorporating these indirect measurements. The (symmetric) confidence interval associated with their estimator is guaranteed to be nearly minimax optimal. Furthermore, once a model for the system has been specified, along with the number and type of measurements that will be recorded, their method can construct the estimator and the confidence interval even before seeing any data. This fact can be useful, for example, when one wishes to minimize the measurements that need to be performed to achieve a desired size of the confidence interval. To construct this estimator, Juditsky & Nemirovski [1] propose an algorithm that involves computing the saddle-point of a concave-convex function to a given precision. While this optimization problem is convex, solving it using standard optimization algorithms or off-the-shelf optimization software like CVX [3] is difficult in practice. In some cases, it is possible to extend the capabilities of these software to handle such saddle-point problems [4]. However, even in such situations, the fact remains that one might need to perform a high-dimensional optimization for constructing the estimator. These reasons make the estimation technique hard to use in practice. ∗ Email: akshay.seshadri@colorado.edu