Parameter Expansion and Efficient Inference

This EM review article focuses on parameter expansion, a simple technique introduced in the PX-EM algorithm to make EM con- verge faster while maintaining its simplicity and stability. The primary objective concerns the connection between parameter expansion and ef- ficient inference. It reviews the statistical interpretation of the PX-EM algorithm, in terms of efficient inference via bias reduction, and ex- plores the potential of parameter expansion for constructing ancillary statistics for conditional inference. In addition, it briefly discusses a few relevant ideas motivated by EM and PX-EM, including an alternative view of PX-EM from the perspective of efficient data augmentation, the broader impact of the statistical thinking on understanding and developing other iterative optimization algorithms.