Integrated & Cointegrated Data

Last week I had a post titled More About Spurious Regressions. Implicitly, in that post, I assumed that readers would be familiar with terms such as "integrated data", "cointegration", "differencing", and "error correction model".

It tuns out that my assumption was wrong, as was apparent from the comment/request left  on that post by one of my favourite readers (Anonymous), who wrote:

"The headlined subject of this post is of great interest to me -- a non-specialist. But this communication suffers greatly from the absence of a single real-world example of, e.g. "integrated" or "co-integrated" data, "differencing" (?), "error-correction model," etc. etc. 

I'm not trying to be querulous. It's just that not all your interested readers are specialists. And the extra intellectual effort required to provide examples would help us..."

So, in response, let's see if what follows is of some help.

First, as requested, some basic definitions. These are a bit "rough and ready", and not intended to be definitive:

• Time: Tick, tock! That which passes, in a natural order, & increasingly quickly as we get older.
• Series: The opposite of funny. A sequence of numerical values
• Time-series: A series of values, ordered according to the passage of time, and usually measured at regular intervals. (There's nothing funny about getting older.)
• Differencing: The act of taking the difference between one value of a time-series and a previous value of that time-series.
• First-differencing: Differencing, in the case where "a previous value" is the immediately preceding value. That is, the creation of the time-series ΔY_t = Y_t - Y_t-1, where Y_t is the original time-series, and t = 1, 2, 3,......measures the passage of time. 
• Stationary: Not moving Strictly, "covariance stationary". The mean and variance of the time-series are constant (and finite); and the covariances between different values of the series depend only on the distance apart that the observations are, and not on the value of "time" (t) itself.
• Non-stationary: Moving. Just like it sounds!
• Integrated: The opposite of "differenced".
• Order of integration: The number of time that the series has to be differenced in order to make it stationary. If that number is "one", then we;'ll say that the original series is "integrated of order one", or I(1), for short. A stationary time-series is I(0).
• Cointegrated: Suppose two or more time-series are each I(p), where p>0. Then these series are "cointegrated" if there exist one or more linear combinations of the series that are I(p-d), where d>0. In the simplest case, p = 1, and d = 1, so we are looking for linear combinations of I(1) series that are I(0). If p = 1 then if such a linear combination exists, it must be unique.
• Cointegrating regression: A linear OLS regression relating the levels of the non-stationary, but cointegrated, time-series. It represents the long-run equilibrating relationship between the variables.
• Error-correction model: Spell-checker. A regression model that explains the short-term dynamics of the relationship between two or more non-stationary, but cointegrated, time-series variables. The model is constructed by using the differenced data (so that each variables is then stationary), as well as an "error-correction" term. The latter additional regressor is a lagged value of the residuals from the cointegrating relationship.

O.K., with that out of the way, let's turn to the requested examples. First some that illustrate the definitions given above, especially as they relate to "spurious regressions". Then, I'll provide the requested "real-world" example.

To begin with, here's a graph of a stationary, I(0), time-series:

It's actually a first-order autoregressive process (AR(1) process), with the autoregressive parameter set to 0.75. That is, Z_t = 0.75Z_t-1 + ε_t, where ε_t is i.i.d. N[0,1]. The series crosses its mean level (zero, in this case) frequently, which is typical of a stationary series.

Now, here's an example of what an I(1) series can look like. You can see that the data have again been generated as an AR(1) process), but this time with the autoregressive parameter set to one in value. That's the "unit root" idea. The series "wanders about", crossing its mean level (zero) very few times. It can end up literally anywhere.

Now, let's see what happen when I "first-difference" this I(1) series. That is, I'll create a new series, of the form:  ΔY_t = (Y_t - Y_t-1); for t = 2, 3, ........

This new series is I(0) - it's stationary. You can see that it follows a time-path that's fundamentally different from that associated with the non-stationary series, Y_t. Given the way that I generated Y_t, this series for ΔY_t is just a sequence of independent N[0,1] values. So, not surprisingly, there are very few observations that are greater than 3 in absolute value.

Here's another example of a non-stationary (I(1)) series:

The X and Y series are different because a different string of values was used for the Normally distributed "error term" in the AR(1) models. You can see, though, that the series for X also crosses its mean level of zero very few times over the sample of 1,000 observations.

Now that we have two non-stationary, I(1), time-series, let's fit an OLS regression that "explains" Y in terms of X. Keep in mind that these two series were generated completely independently of each other. There is actually no relationship at all between Y and X!

However, that's not the impression that we get when we look at the OLS regression results below: 

Even though the X and Y data have been generated so that they are actually independent of each other, the regression results suggest that there is a significant negative relationship between them. In addition, the R² value suggests that X can explain 64% of the sample variation in the Y data; and the value of the DW statistic suggests that the errors in the model are positively autocorrelated.

These are the classic results that we associate with a "spurious regression".

In addition, if we test the regresssion residuals to see if the model's errors are normally distributed, the p-value for the Jarque-Bera test statistic is 0.006:  

We strongly reject the hypothesis of normality, even tough in fact it is true. (The Y variable is normally distributed, so the OLS residuals will be normal too.) This result relating to the Jarque-Bera test is also typical of a spurious regression, as I discussed in a recent post.

If we were to interpret the results as telling us that we need to re-estimate the model, allowing for the autocorrelated errors, this is what we'd get as a result:

The relationship between X and Y is no longer significant. Which is quite right!

However, this last model is still meaningless.

What should we be doing, in fact? We need to difference the data to make them stationary, and then estimate the model:

The absence of any meaningful relationship between Y and X is now fully revealed. Moreover, the known normality of the data is no longer rejected by the Jarque-Bera test.

Now let's consider some real economic data - to be specific, quarterly U.S. real private consumption expenditure and real personal disposable income. This is what these data look like for the period 1950Q1 to 1991Q4:

Both series are upward-trended, but are these deterministic trends, or stochastic trends (due to unit roots)? Maybe it's a bit of both?

Here's the correlogram for the consumption series - broken into 2 parts (with a gap), because a lot of lags are needed to see the "full picture":

The single spike in the partial autocorrelation function, coupled with the (initially) declining spikes in the autocorrelation function, suggest that the series follows an AR(1) process. When we see that the autocorrelations then start to increase again at higher lags, this suggests that the consumption series is non-stationary.

This is supported by applying the Augmented Dickey-Fuller (ADF) test (with drift and trend). The associated p-value is 0.6243. We would not reject the null hypothesis of a unit root at any reasonable significance level. The consumption series is I(1).

Similar results are obtained for the income series:

In this case, the p-value for the ADF test is 0.4768, and we again conclude that the series is I(1).

Now, suppose that we want to estimate a very basic consumption function, using these two time-series. We have to decide whether we can use the levels of the data, or whether we need to difference the data to allow for the fact that they are I(1).

The answer depends on whether or not the two series are cointegrated. If they aren't, then we should estimate a model with ΔC_t as the dependent variable, and ΔY_t as the primary regressor. We would probably also want to include lags of ΔY_t and ΔC_t as additional regressors. Differencing the data will ensure that all of the series being used are stationary, and so we will avoid having a "spurious regression".

On the other hand, if the consumption and income series are cointegrated, then we have two types of models that we can estimate legitimately:

1. A model that uses the original data for C_t and Y_t. Even though both series are I(1), the fact that they are cointegrated means that there is a linear combination of them that is stationary. Regressing C_t on Y_t gives us this linear combination. This model will represent the long-run equilibrating relationship between consumption and income. Because only 2 variables are involved here, if a cointegrating relationship exists, then it must be unique. Again, we might want to include lagged values of Y_t and/or C_t as additional regressors.
2. An "error-correction model". As we saw in the definitions near the start of this post, this model would be of the general form:  ΔC_t = α₁ + α₂ΔY_t + α₃R_t-1 + u_t , where  R_t is the OLS residuals series from the "cointegrating regression" discussed in point 1  just above. A more general model would include lagged values of  ΔC_t  and  ΔY_t as additional regressors. (More on this below.)

To test for cointegration, I'm simply going to use the Engle-Granger two-step approach. Yes, I know I can better than this, but it will suffice in the present context, especially as I have just two time-series. This will involve estimating a "cointegrating regression", and then testing if the residuals from this regression are non-stationary. A rejection of this null hypothesis, using the ADF test with MacKinnon's modified critical values, would lead us to conclude that consumption and income are cointegrated.

Here are the results that we get when we estimate a basic model of the first form above - the cointegrating regression: 

The cointegrating regression ADF (CRADF) test statistic is -4.2398, and we reject the null hypothesis of "no cointegration" at the 1% significance level.

If we-estimate the contegrating regression with a linear time-trend added a regressor, the CRADF test statistic is -4.9736, and we come to the same conclusion, at the same level of significance. (In this case the estimated coefficient of the income variable is 0.9546.)

Given that we have established that there is cointegration between consumption and income, the last OLS results are perfectly meaningful, even though we are using levels of non-stationary data. The results from the cointegrating regression above imply a long-run marginal propensity to consume (mpc) of 0.877.

The residuals in this model are severely autocorrelated, and a more reasonable model is of the form:

Now the residuals are serially independent, and the long-run mpc works out to be 0.870.

Now, let's consider an error-correction model, and explore the short-run dynamics of the relationship between consumption and income. Let R_t be the residuals series from the OLS (cointegrating) regression model, C_t = β₁ + β₂Y_t + β₃t + e_t. Then, our basic error-correction model will be:

                             

                                   ΔC_t = α₁ + α₂ΔY_t + α₃R_t-1 + u_t

In fact, what I've done is to start off with a slightly more general form of this model - one that also includes lagged values of ΔC_t and ΔY_t as additional regressors. I've then simplified the model by eliminating insignificant variables, and ensuring that there is no autocorrelation (up to fourth order) in the residuals. That is, I've used a "general-to-specific" modelling strategy, and here's what I ended up with:

I'm not saying this is "the very best" model, but it's reasonable for our present purposes. The estimated coefficient of the error-correction term is negative, and highly significant, as we'd expect if consumption and income are cointegrated.

If we "unscramble" the fitted values for the level (rather than the first-difference) of the consumption variable, and compare the fitted and actual values, the simple correlation is 0.99986:

The "residuals" expressed in the original levels of the data appear to be "white noise".

So, there's my "real-world" example. If you want to play around with yourself, the data are in a text file on the data page for this blog, and the EViews workfile that I used is on the code page.

© 2012, David E. Giles