Forecasting the Arrival Time of Coronal Mass Ejections: Analysis of the CCMC CME Scoreboard

Accurate forecasting of the properties of coronal mass ejections as they approach Earth is now recognized as an important strategic objective for both NOAA and NASA. The time of arrival of such events is a key parameter, one that had been anticipated to be relatively straightforward to constrain. In this study, we analyze forecasts submitted to the Community Coordinated Modeling Center (CCMC) at NASA's Goddard Space Flight Center over the last six years to answer the following questions: (1) How well do these models forecast the arrival time of CME-driven shocks? (2) What are the uncertainties associated with these forecasts? (3) Which model(s) perform best? (4) Have the models become more accurate during the past six years? We analyze all forecasts made by 32 models from 2013 through mid 2018, and additionally focus on 28 events all of which were forecasted by six models. We find that the models are generally able to predict CME-shock arrival times -- in an average sense -- to within 10 hours, but with standard deviations often exceeding 20 hours. The best performers, on the other hand, maintained a mean error (bias) of -1 hour, a mean absolute error of 13 hours, and a precision (s.d.) of 15 hours. Finally, there is no evidence that the forecasts have become more accurate during this interval. We discuss the intrinsic simplifications of the various models analyzed, the limitations of this investigation, and suggest possible paths to improve these forecasts in the future.