Root Mean Squared Error Criterion (RMSE)

LEVEL II

The root mean squared error criterion is a method to test if if an AR model is accurate in making future predictions. RMSE is calculated as the mean squared errors of all the residuals and taking the square root.

RMSE is often used to compare the relative accuracy of two different AR models. For example, you may have used some sample data to estimate two different models for a time series problem, an AR(1) and an AR(2) model. You will want to use some test data to calculate the forecasts of the two different models and compare them to actual observed data. 

There are two ways you can go about this. You can use the same sample that you used to estimate the models. We call the results in-sample forecasts. The other way is to use a separate set of data that are outside the sample period, so we call these out-of-sample forecasts. In this case, we use data from a time period outside the period used to develop the model. Out-of-sample forecasts are preferred over in-sample forecasts for RMSE calculations because they are a better proof of a model’s ability to predict the future.