When is an Autoregressive Model Dynamically Stable?

Autoregressive processes arise frequently in econometrics. For example, we might have a simple dynamic model of the form:

            y_t = β₀ + β₁y_t-1 + ε_t   ;   ε_t ~ i.i.d.[0 , σ²]       .           (1)

Or, we might have a regression model in which everything is "standard", except that the errors follow an autoregressive process:

            y_t = β₀ + β₁x_t + u_t               (2)

             u_t = ρ u_t-1 + ε_t    ;  ε_t ~ i.i.d.[0 , σ²] .

In each of these examples a first-order autoregressive, or AR(1), process is involved.

Higher-order AR processes are also commonly used. Although most undergrad. econometrics students are familiar with the notion of "stationarity" in the context of an AR(1) process, often they're not aware of the conditions needed to ensure the stationarity of more general AR models. Let's take a look at this issue.

First, what do we mean by a stochastic process that's "stationary"?

There are several types of "stationarity". Here, I'm going to use what we call "weak stationarity" (or "covariance stationarity"). What this means is that the mean and variance of the process are both finite and must not depend on time; and the covariances between pairs of random values from the process can depend on how far apart the values are in time, but not the value of time itself.

let's go back to the model in (1) above, and see what we mean by "stationarity" in this simple case. Notice that if we lag the equation by one period, we get:

        y_t-1 = β₀ + β₁y_t-2 + ε_t-1   ;      ε_t-1 ~ i.i.d.[0 , σ²]  .                (3)

Now, use (3) to eliminate y_t-1 from (1), yielding:

        y_t = β₀ + β₁ [ β₀ + β₁y_t-2 + ε_t-1 ] + ε_t  .

Now, keep doing this repeatedly, and we end up with:

        y_t = β₀ [1 + β₁ + β₁² + β₁³ + .......] + [ε_t + β₁ ε_t-1 + β₁²ε_t-2 + ....]

or,

         y_t = β₀Σ( β₁ⁱ ) + Σ( β₁ⁱε_t-i ),                        (4)

where the range of summation is from zero to infinity in each case.

Given what we assumed about the ε_t values in equation (1), we see from (4) that:

• E[ y_t ] =  β₀Σ( β₁ⁱ ) = μ   ;    say
• var.[ y_t ] = σ² Σ( β₁²ⁱ)
• cov.[ ε_t , ε_s ] = 0  ;    for all t ≠ s .

Now, recalling some simple high school math, we know that  Σ( β₁ⁱ ) converges to the finite limit (1 - β₁)^-1, if and only if |β₁| < 1. That is, it converges if and only if β₁ lies strictly inside the unit interval. If this condition holds, then both E[ y_t ] and var.[ y_t ] will be finite - and you can see that neither they nor the covariances depend on the value of 't'. This condition on the value of β₁ ensures that the series, y_t, is covariance stationary.

The case where β₁ = 1 is termed the "unit root" case, for reasons that will become even more apparent below; and if | β₁ | > 1, the y_t series is "explosive". Let's see why this particular terminology is used. If the stationarity condition is not satisfied, then any "shock" to the y_t series will lead to a subsequent time-path that has an unbounded mean and variance. On the other hand, if the process is stationary, then following such a "shock", the time-path for y_t will eventually settle down to what it was previously. The shock will be "absorbed". For this reason we often say that non-stationary time-series have "long memories".

Let's see some examples of this. (Keep in mind that you can enlarge the charts by clicking on them.)

If you don't believe that this really is an explosive case, let's go out to t = 100:

The Eviews file that I used to create these charts is on the code page for this blog.

More generally, let's suppose that we have an AR(p) model, of the form:

                      y_t = γ₁ y_t-1 + γ₂ y_t-2 + .......... + γ_p y_t-p + ε_t   ;         ε_t ~ i.i.d.[0 , σ²]

Then the condition that must be satisfied in order that this model is dynamically stable is that all of the roots of the following so-called "characteristic equation"

                     1 - γ₁ z - γ₂ z² - ..... - γ_p z^p = 0  ,

lie strictly outside the unit circle.

It's worth noting that some authors define the characteristic equation as

                 z^p - γ₁ z^p-1 - γ₂ z^p-2 - ...... - γ_p = 0 ,

and then the AR(p) process will be stationary if all of this equation's roots lie strictly inside the unit circle.

Let's go back to the AR(1) model - that is, set p = 1. In that case, the characteristic equation is

                   1 - γ₁ z = 0.

Solving for z, we get z = 1 / γ₁, so the stationarity condition is that |1 / γ₁| > 1; or, equivalently, |γ₁| < 1. This is exactly the condition that we saw above, now in our new notation. 

Next, let's think about a series that is second-order autoregressive, or AR(2). In this case, the conditions that need to be satisfied to ensure stationarity of the series are more complicated. Suppose that our AR(2) model for Y is written as:

                y_t = γ₁ y_t-1 + γ₂ y_t-2 + ε_t    ;      ε_t ~ i.i.d.[0 , σ²]  .

This model will be stationary (i.e., dynamically stable) if the roots of the characteristic equation,

              1 - γ₁z - γ₂z² = 0  ,

all lie strictly outside the unit circle.  Equivalently, the roots of the equation,

               z² - γ₁z - γ₂ = 0,

must lie strictly inside the unit circle.

As I show in this attachment, when these roots are real (rather than complex) this is equivalent to requiring that the coefficients  of the AR(2) model satisfy all of the following three conditions:

1. (γ₁ + γ₂) < 1
2. (γ₂ - γ₁) < 1
3. |γ₂| < 1

These three conditions allow us to depict the stationary and non-stationary regions of the parameter space in the following way:

Notice that this diagram also identifies the regions in the parameter space in which the series will respond to shocks in an oscillatory or non-oscillatory manner. The following two graphs illustrate this for two stationary AR(2) processes. In the first case, we have γ₁ = 0.1 and γ₂ = 0.8, so we are in the stationary/non-oscillatory region in the above chart:

The next case is for γ₁ = 0.1 and γ₂ = -0.8, so we are now in the stationary/oscillatory region of the parameter space:

A "blow-up" of these last two graphs follows:

Again, the EViews file that I used to create these data and charts is on the code page for this blog.

As we move to AR processes of higher order, the conditions that the parameters have to satisfy, to ensure that the process is stationary, become increasingly complicated to write down. However, the same basic principle applies - the roots of the associated "characteristic equation" must lie strictly outside the unit circle. Or, equivalently, the inverse roots must lie strictly inside that circle.

This is a condition that can be checked numerically, once we have values (estimates) of the coefficients of the AR process. It's just a matter of solving the characteristic equation for its roots, using these estimated values for the coefficients. Keep in mind that the roots will be functions of the coefficients, as we saw in the AR(1) and AR(2) cases.

In other words, even if it's really difficult to write down the formulae for the roots of the characteristic equation, it's a relatively simple matter to solve the equation for these roots, numerically. When we're estimating AR models using EViews, this gets done as a matter of course. In addition, you can request a diagram of the results, which is especially helpful if some of the roots are complex (rather than real).

To illustrate this, let's look at an example in which a restricted AR(4) model is used to "explain" the logarithm of real imports into Canada from the U.S.. The quarterly data exhibit a seasonal pattern, so seasonal dummy variables are included in the model:

Notice that the coefficient of the 1-period lagged value of the dependent variable is less than one in absolute value. So, we might be tempted to conclude that the model is dynamically stable. However, let's see what happens if we re-estimate the model using an apparently different, but mathematically equivalent, way:

The estimated coefficients and standard errors for the lagged values of the dependent variable in the first output are identical to the coefficients of the AR terms in the second output. Although the estimated intercept and dummy variable coefficients are different, in fact the model "fit" is identical in each case. You might guess this by observing that any of the reported statistics that are based on the residuals (e.g., R², F-statistic, D-W statistic) are identical across the outputs.

However, the really interesting thing about the second set of results is that we now see the inverse roots of the characteristic equation associated with the restricted AR(4) process that we've used. This equation has four roots, of course, and it happens that ion this case there are two real roots, and a complex conjugate pair of roots. (More on the latter point below.) In particular, one of the real roots is one in value! Let's see this graphically:

Oh dear - we have a "unit root" situation associated with the dynamics of our AR(4) model. Note that I'm referring to the AR(4) process that describes how log(IMP) is related to lags of log(IMP). I'm not talking about the errors of that regression model.

Not surprisingly, when we "shock" the process by an amount equal to one sample standard deviation for the IMP variable, the shock does not die out. The effect just keeps on accumulating:

What can I do about this? Well, the log(IMP) data series itself is actually non-stationary, and I didn't take this into account. Using both the ADF test and the KPSS test, it's clear that the series is integrated of order one (i.e., I(1)). So, I need to first-difference the log(IMP) series to make it stationary, and then model it. Here's what I get:

In this case, the model for log(IMP) is dynamically stable, as we can see from the following diagram:

Once again, the characteristic equation has two real roots, and a complex conjugate pair of roots - and the inverted roots are all strictly inside the unit circle.

The EViews file for this example is on the code page for this blog, and the imports data are in a text file on then data page.

A few words of explanation may be in order with respect to the charts that show the inverse roots of the characteristic equation. These charts are in the form of an Argand diagram. They allow for the fact that some roots may be complex numbers, of the form z = (x  +/ -iy), where the imaginary number, i, satisfies i² = -1. The X-axis in those diagrams plots the real (x) part of z, and the Y-axis plots the imaginary (y) part of z. So, if a root is real, it will lie on the horizontal axis; but if it's complex, if will be located at the point (x , y).

If you want to explore Argand diagrams interactively, you can always use this Wolfram cdf file.

What's the take-away message from all of this?

If you look back to the very first set of OLS results above, you'll see that the coefficient of log(IMP)_t-1 was 0.8948. It was less than one in absolute value. However, once the full dynamic structure of the model was taken into account we found that the model was actually unstable. If you're estimating a higher-order AR process, you need to take special care when interpreting the magnitudes of the estimated coefficients!

Being able to see the values of the roots of the characteristic equation  is enormously helpful when we're estimating an AR model (or, for that matter an MA or ARMA model). You certainly don't to inadvertently use a model that's dynamically unstable for forecasting purposes!

.

© 2013, David E. Giles