Themes in Econometrics

There are several things that I recall about the first course in econometrics that I took. I'd already completed a degree in pure math. and mathematical statistics, and along with a number of other students I did a one-year transition program before embarking on a Masters degree in economics. The transition program comprised all of the final-year undergrad. courses offered in economics.

As you'd guess, the learning curve was pretty steep in macro. and micro, but I had a comparative advantage when it came to linear programming and econometrics. So things balanced out - somewhat!

One thing that I remember about the econometrics course is that the lectures were superb. The late A. C. (Tony) Rayner taught the course, and he was a remarkably gifted teacher. He subsequently supervised my Ph.D. dissertation.

Looking back, over forty years later, one other thing that stands out is that courses in econometrics in those days really didn't stress the thematic nature of the material. I think we're better at emphasising themes in econometrics these days, and I also think this gives students a much better feel for the 'big picture" than used to be the case.

Essentially, what I'm referring to is the following. When you look at the various particular estimators and tests that we use in econometrics, for the most part they're not just a bunch of ad hoc special cases. There's a pattern to them. They can be classified according to several general principles or themes, and if we recognize and exploit this then we can quickly gain insights into their strengths and weaknesses.

Learning about these themes also provides a sound basis for dealing with "new" problems - ones that weren't covered in the courses we took - when we encounter then in our later work. If we understand these themes, then we should always be able to come up with a sensible "solution" to such problems, and both we and the readers of our work will understand the quality of our results.

Among the various thematic approaches to econometric inference that we could isolate are:

• Maximum likelihood estimation, and the associated likelihood ratio, Wald and Lagrange multiplier tests.
• Bayesian inference.
• Generalized method of moments estimation and the associated tests.
• Instrumental variables methods.

Now, it's very important to understand that there are many connections between these apparently different themes. An obvious example is the intimate link between instrumental variables (IV) estimation and generalized method of moments (GMM).

It's also important to note that many of the specific inferential "tools" that we use routinely in econometrics can be viewed as special cases of more than one of the thematic groupings. For instance, the OLS estimator of the coefficient vector in a linear regression model is just the MLE estimator  under certain assumptions about the errors of that model. It's also the IV estimator if the regressors are used as their own instruments; and it's also a special case of GMM estimation. In addition, this OLS estimator can be interpreted as the Bayes estimator of the coefficient vector under a "diffuse" prior distribution for the parameters in the regression model.

There's a lot that can be said about all of this. Here, though, I'd just like to highlight a thematic way of viewing all of the standard estimators that we use in the context of simultaneous equations model (SEMs) in econometrics. Much of this will be familiar, but I'm guessing that some of it won't be.

I've blogged a bit about estimating SEMs in a couple of recent posts - here and here. You'll know that we can group the associated estimators into two categories. The first is the group of so-called "single equation" estimators, where each structural equation of the model is estimated separately from the others, but some information about the presence of the other equations is taken into account. The second is the group of "full information" or "system" estimators, where all of the parameters for all of the equations in the entire structural system are estimated at once.

When properly formulated, all of the estimators are weakly consistent - that's the minimum objective here. The estimators in the first group use less structural information (generally in the form of exclusion restrictions on the parameters) than do estimators in the second group. Generally speaking, this means that "system" estimators have greater asymptotic efficiency than do "single equation" estimators. However, in various special situations, one or other of the system estimators will collapse to become algebraically, or statistically, identical to one of the single equation estimators.

What I want to focus on is the fact that all of these estimators, in both groups, are simply examples of IV estimators with instruments that ensure their consistency. So, once we realize this, we don't need to prove the consistency of each estimator, one by one. This consistency is a consequence of them being "valid" IV estimators; i.e., having instruments that are asymptotically uncorrelated with the structural errors.

Now, in the case of the single equation estimators, this is probably a familiar story. For example, for a linear SEM:

• The 2SLS estimator is a valid IV estimator with all of the predetermined variables in the SEM being the the instruments. Hence 2SLS is consistent.
• The 2SLS estimator can also be expressed as an IV estimator with the predetermined variables in the equation in question, together with the predicted values of the endogenous regressors, used as the instruments. (In this case, the predicted values referred to are those from the unrestricted reduced form of the model - that is, obtained by regressing each endogenous regressor on all of the predetermined variables in the SEM, using OLS.)
• Within the class of IV estimators for which the instruments are linear combinations of the predetermined variables in the entire SEM, the 2SLS estimator is asymptotically efficient.
• If a structural equation is "just-identified", then the 2SLS estimator collapses to the Indirect Least Squares (ILS) estimator. So, the latter estimator is an IV estimator. As the ILS estimator is defined only in the just-identified case, it follows that it is a consistent estimator, and has the same asymptotic efficiency as 2SLS.
• All members of the family of k-class estimators are IV estimators. Specifically, the instruments are the predetermined variables in the equation in question, together with k times the predicted values of the endogenous regressors in the equation. (As for 2SLS, these predicted values are those from the unrestricted reduced form of the model - that is, obtained by regressing each endogenous regressor on all of the predetermined variables in the SEM, using OLS.)
• If k is random, plim(k) = 1, then the instruments implicit in the k-class estimator are "valid", and hence this estimator is then consistent. (It also has the same asymptotic efficiency as the 2SLS estimator.)  Note that if k = 1 (non-random), ILS = 2SLS. If k = 0, ILS = OLS, which is inconsistent for an equation of an SEM.
• The Least Variance Ratio (LVR) estimator is the k-class estimator with k = l*, where l* is the smallest root of a particular determinantal equation, and plim(l*) = 1. Hence the LVR estimator is consistent and it has the same asymptotic efficiency as 2SLS.
• The LIML estimator is algebraically identical to the LVR estimator, and so it is also consistent, with the same asymptotic efficiency as 2SLS.

But what about the "system" estimators, such as 3SLS and FIML? Can they also be given an IV representation?

• The 3SLS estimator is a full-system IV estimator, with "valid" instruments (Madansky, 1964).
• The FIML estimator is a full-system IV estimator, with "valid" instruments (Hausman, 1975).
• The FIML estimator uses all of the over-identifying restrictions in forming the instruments, while 3SLS uses only part of this information.
• The K-matrix-class estimator (Scharf, 1976), is a full-system IV estimator with instruments that are valid under conditions similar to those associated with the single-equation k-class estimator. 
• The systems IV estimators proposed by Brundy and Jorgensen (1971), Dhrymes (1971), and Lyttkens (1970) correspond to one step of the FIML iteration, so if they are itereated they converge to FIML.

So, the estimation of structural SEMs provides a really good example of a situation where an apparently diverse collection of estimators can actually be gathered together in a thematic way, and shown to be just special applications of one particular technique - in this case, IV estimation. This is especially helpful when it comes to understanding the merits of these estimators; and it also provides a really efficient way of establishing some key properties.

As I noted earlier, there are other themes that are important in similar ways in econometrics. It's worth keeping this in mind in order to get a feel for the "big picture".

Note: The links to the following references will be helpful only if your computer's IP address gives you access to the electronic versions of the publications in question. That's why a written References section is provided.

References

Brundy, J. and D. W. Jorgensen, (1971). Efficient estimation of simultaneous equation systems by instrumental variables, Review of Economics and Statistics, 53, 207-224.

Dhrymes, P. J. (1971). A simplified structural estimator for large-scale econometric models. Australian Journal of Statistics, 13, 168-175.

Hausman, J. A. (1975), An instrumental variables approach to full infomation estimators for linear and certain nonlinear econometric models. Econometrica,43, 727-738.  

Lyttkens,  E. (1970). Symmetric and asymmetric estimation methods. In E. J. Mosbaek and H. Wold (eds.), Interdependent Systems: Structure and Estimation, North-Holland, Amsterdam.

Madansky, A. (1964). On the efficiency of three-stage least squares estimation. Econometrica, 32, 51-56.

Scharf, W. (1976). K-matrix-class estimators and the full-information maximum-likelihood estimator as a special case. Journal of Econometrics, 4, 41-50.

© 2011, David E. Giles