Dancing With the Econometricians

Let's talk about the two-step. Not the tango or the polka. The two-step!

More specifically let's talk about a particular two-step estimator that we use all of the time in econometrics. I want to clear up some misconceptions that I seem to encounter all too frequently when I read empirical "applied" papers.

Why is it that some people insist on using the term "Two Stage Least Squares" inappropriately? 

Let me explain what I mean.
Suppose that we have a system of M simultaneous equations, where the (i^th) structural equation that we want to estimate can be written as:

                     y_i = Y_iγ_i + X_iβ_i + ε_i .

We'll assume that this equation is identified. Yes, let's be careful about this. If the "rank condition" for identification of the equation isn't satisfied, then the 2SLS estimator won't be weakly consistent. Even worse, if the "order condition" isn't satisfied then this estimator isn't even defined (because a matrix that has to be inverted will be singular). 

The (m_i) columns of the Y_i matrix are endogenous regressors.  The (k_i) columns of the X_i matrix are "predetermined" regressors. That is, they are strictly exogenous regressors (often including an "intercept" variable); or else they are lagged endogenous or exogenous variables (if we have time series data). The elements of the error term, ε_i, have a zero mean, and are homoskedastic and serially independent.

Remember that this equation is just one of M in the entire simultaneous system. Let's use the symbol, X, to denote the matrix of observations on all of the predetermined variables that appear anywhere in the system.
Suppose that X has K columns.

The 2SLS estimator of the coefficients in equation (1) can be described as follows:

1. Using OLS, regress Y_i on all (yes, ALL) of the columns of X, and get the matrix of predictions, Y_i* = X(X'X)^-1X'Y_i.
2. Replace Y_i in (1) with Y_i*, and then use OLS to estimate this modified equation, yielding consistent estimates of the elements of γ_i and β_i.

Notice the words that I've emphasized in the first stage.

Now, a bit of history - just to provide some light relief in the middle of the math! The 2SLS estimator emerged as a computationally convenient way of obtaining consistent estimates of the coefficients in an equation of the form (1). At the time that 2SLS was proposed by Theil (1953a, 1953b, 1954), Basmann (1957), and Sargan (1958), computational convenience was a big deal. In fact, it remained a big deal for along time - e.g., see here. Anderson (2005) has recently drawn our attention to the fact that the 2SLS estimator was used (at least "obliquely") in the even earlier papers by Anderson and Rubin (1949, 1950) that introduced the LIML estimator and derived that estimator's asymptotic distribution.

So, what is it that I get upset about? Well, it's become very common to see the following two-step estimator described as "2SLS":

1. Using OLS, regress Y_i on a set of suitable instruments, Z, and get the matrix of predictions, Y_i** = Z(Z'Z)^-1Z'Y_i.
2. Replace Y_i in (1) with Y_i**, and then use OLS to estimate this modified equation, yielding estimates of the elements of γ_i and β_i.

This is often done in the context of a single structural equation, without a complete simultaneous equations model in sight. There's nothing wrong with this (provided that X_i is included in Z - see Angelo Melino's comment below) - but please don't refer to this as 2SLS! Unless Z = X, it's just an instrumental variables estimator, constructed in two steps.

In fact, although this second approach yields appropriate estimates of the coefficients, it won't give you the correct standard errors, or the correct values of anything else that is constructed from the second-step residuals. That's because the latter residual vector is of the form e_i** = y_i - (Y_i**g_i + X_ib_i), instead of the correct form, e_i = y_i - (Y_ig_i - X_ib_i). Here, g_i and b_i are the estimated coefficient vectors. Of course, this is easily fixed, just as it is in the case of the genuine 2SLS estimator, but it's simpler to just use the one-step instrumental variables estimator rather than the (otherwise) equivalent two-step estimator.

Confession: One situation where the two-step approach is helpful is if you are worried that Z may include some "weak" instruments. In that case, the F-statistic for the significance of the estimated coefficients in of the first step provides one basis for testing against such weakness. But that's another story!

I have to admit that as much as I enjoy using the EViews econometrics package, I find it intensely irritating that when it comes to instrumental variables estimation, the command is titled "Two Stage Least Squares"!

In another post, coming up this week, I'll explore the various connections between the (real) 2SLS estimator and instrumental variables estimation. There are more of these connections than you might think.

Meantime, call me pedantic if you will, but if you want to dance with the econometricians then you have stay in time with the music, and you have to know that a samba is not the same as a rumba!


References

Anderson, T. W., 2005. Origins of the limited information maximum likelihood and two-stage least squares estimators. Journal of Econometrics, 127, 1-16.

Anderson, T.W. and H. Rubin, 1949. Estimation of the parameters of a single equation in a complete system of stochastic equations. Annals of Mathematical Statistics, 20, 46–63.

Anderson, T.W. and H. Rubin, 1950. The asymptotic properties of estimates of the parameters of a single equation in a complete system of stochastic equations. Annals of Mathematical Statistics, 21, 570–582.

Basmann, R.L., 1957. A generalized classical method of linear estimation of coefficients in a structural equation. Econometrica 25, 77–83.

Sargan, J.D., 1958. Estimation of economic relationships using instrumental variables. Econometrica, 67, 557–586.

Theil, H., 1953a. Repeated least-squares applied to complete equation systems. Centraal Planbureau Memorandum.

Theil, H., 1953b. Estimation and simultaneous correlation in complete equation systems. Centraal Planbureau Memorandum. (Reprinted In: Raj, B., Koerts, J. (Eds.), 1992, Henri Theil’s Contributions to Economics and Econometrics, Vol. 1, Kluwer, Dordrecht.)

Theil, H., 1954. Estimation of parameters in econometric models. Bulletin of the International Statistical Institute, 34, 122–129.

© 2012, David E. Giles