On the Invariance of MLE's

The Maximum Likelihood Estimator (MLE) is extremely widely used in statistics, and in the various "metrics" disciplines such as econometrics. This is because this estimator has several highly desirable properties, as long as the sample size is sufficiently large.

For example, under fairly weak ("regularity") conditions, the MLE is weakly consistent, asymptotically efficient, and asymptotically normal.

In small samples, the MLE may or may not have good "sampling properties". For instance, it may be biased or unbiased, depending on the estimation problem under consideration.

When teaching this material, instructors invariably mention another nice property of the MLE: it's an "invariant estimator".

What does this actually mean?

Some econometrics texts (e.g., Greene, 2012, p.521) define the invariance property as follows: "If θ* is the MLE of θ, and f( . ) is a 1-1 function, then f(θ*) is the MLE of f(θ)."

In fact, this statement of the property is unduly strong. The function, f( . ), simply needs to be continuous - it doesn't need to be 1-1. (The proof of the result is especially simple if f is 1-1.)

In support of this more general result, I usually refer my students to the note by Zehna (1966). Recently, I became aware of some other important references when I read a paper by Olive (2004).

In particular, Berk's  (1967) review of Zehna's paper provides a simple proof of the general result.

An obvious implication of the generality of the invariance theorem is that if σ*² is the MLE of the population variance, σ², then √(σ*²) is the MLE of σ. This wouldn't be the case if the theorem was restricted to just 1-1 transformations!

Olive's paper will definitely be on my reading guide for students in future courses that I teach.

References

Berk, R., 1967. Review 1922 of ‘Invariance of maximum likelihood estimators’ by Peter W. Zehna. Mathematical Reviews, 33, 342-343.

Greene, W. H., 2012. Econometric Analysis, 7th ed.. Prentice Hall.

Olive, D. J., 2004. Does the MLE maximize the likelihood? Mimeo., Department of Mathematics, Southern Illinois University. (Also included in D. J. Olive, 2014, Statistical Theory and Inference, Springer.)

Zehna, P. W., 1966. Invariance of maximum likelihood estimators. Annals of Mathematical Statistics, 37, 744.

© 2015, David E. Giles

Reading for the Merry Month of May

While you're dancing around the Maypole (or whatever else it is that you get up to), my recommendations are:

• Claeskens, G., J. Magnus, A. Vasnev, and W. Wang, 2014. The forecast combination puzzle: A simple theoretical explanation. Tinbergen Institute Discussion Paper TI 2014 - 127/III. 
• de Jong, R. M. and M. Sakarya, 2013. The econometrics of the Hodrick-Prescott filter. Forthcoming in Review of Economics and Statistics.
• Honoré, B. E. and L. Hu, 2015. Poor (wo)man’s bootstrap. Working Paper 2015-01, Federal Reserve Bank of Chicago.
• King, M. L. and S. Sriananthakumar, 2015. Point optimal testing: A survey of the post 1987 literature. Working Paper 05/15, Department of Econometrics and Business Statistics, Monash University.
• Meintanis, S. G. and E. Tsionas, 2015. Approximately distribution-free diagnostic tests for regressions with survival data. Statistical Theory and Practice, 9, 479-488. 
• Piironen, J. and A. Vehtari, 2015. Comparison of Bayesian predictive methods for model selection. Mimeo.
• Yu, P., 2015. Consistency of the least squares estimator in threshold regression with endogeneity. Economics Letters, 131, 41-46.

© 2015, David E. Giles

Introduction to Applied Econometrics With R

I came across a January post from David Smith at Revolution Analytics, in his Revolutions blog. It's titled, An Introduction to Applied Econometrics With R, and it refers to a very useful resource that's been put together by Bruno Rodrigues of the University of Strasbourg. It's called Introduction to Programming Econometrics With R, and you can download it from here.

Bruno's material is a work in progress, but it's definitely worth checking out if you're looking for something to help economics students learn about R in an introductory statistics/econometrics course.

© 2015, David E. Giles

Applied Econometrics - 4 Volume Set

Back in 2012 I posted about a 4-volume set of readings, titled The Rise of Econometrics, edited by Duo Qin, and published by Taylor and Francis. That set appeared in January 2013.

In response to a comment on that post, Bill Greene has recently informed me that he is editing another comprehensive 4-volume set for T&F, due out later this year. This set is titled Applied Econometrics. It features 79 papers, in addition to the Editor's introduction.

The details of the Table of Contents can be found here.

This looks like a very comprehensive "must have" addition for any serious library.

© 2015, David E. Giles

Videos for EViews 9

The team at EViews has put together a great set of videos that highlight some of the new features in EViews 9.

You can find them here, and I strongly recommend them.

© 2015, David E. Giles

Introductory Statistics for Data Science

The latest issue of Chance contains a very timely article by Nicholas Horton, Benjamin Baumer, and Hadley Wickham. It's titled, "Setting the Stage for Data Science: Integration of Data Management Skills in Introductory and Second Courses in Statistics".

Ask yourself - "Is the traditional way that we teach introductory and second-level statistics courses really suited for preparing students for future work in modern data science?"

More specifically, do our undergraduate courses provide the data-related skills that are increasingly needed? The same question could be asked of undergraduate training in econometrics.

Horton et al. itemize five things which, in their opinion, deserve more attention in this context:

Edmond Malinvaud: A Tribute to his Contributions in Econometrics

I wrote this brief post just after Edmond Malinvaud passed away on 7 March of this year, at the age of 91. 

Peter Phillips' tribute to Malinvaud is a "must read" piece (see here).

Like Peter, I also used Malinvaud's text when undertaking my Masters-level studies in econometrics. It was demanding but ultimately exceptionally rewarding.

© 2015, David E. Giles

My Paper With Al Gol

My apologies for the broken link to my paper co-authored with Al Gol that was listed in the "April Reading" post on 1 April.

This has now been fixed.

© 2015, David E. Giles

My Favourite Book

Well, perhaps it's not really my favourite book, but it's certainly right up there with the most heavily thumbed tomes on my office bookshelf.

I'm referring to Tables of Integrals, Series and Products, by Gradshteyn and Ryzhik. I picked up a used copy of the 4^th ed. (1965) for about $5 some years ago at Powell's bookstore in Portland, and it's saved me more anguish and time than I can possibly estimate.

For example (click for LARGER version):

(Sample page)

The book is now in its 8^th edition (2014). You can download the 7th ed. (2007) on a pay-by-the-chapter basis from here, and you should be aware of the associated errata document.

I also came across this link.

© 2015, David E. Giles

Regression Coefficients & Units of Measurement

A linear regression equation is just that - an equation. This means that when any of the variables - dependent or explanatory - have units of measurement, we also have to keep track of the units of measurement for the estimated regression coefficients.

All too often this seems to be something that students of econometrics tend to overlook.

Consider the following regression model:

               y_i = β₀ + β₁X_1i + β₂x_2i + β₃x_3i + ε_i    ;    i = 1, 2, ...., n                   (1)

where y and x₂ are measured in dollars; x₁ is measured in Kg; and x₃ is a unitless index.

Because the term on the left side of (1) has units of dollars, every term on the right side of that equation must also be expressed in terms of dollars. These terms are β₀, (β₁x_1i), (β₂x_2i), (β₃x_3i), and ε_i.

In turn, this implies that β₀ and β₃ have units which are dollars; the units of β₁ are ($ / Kg); and β₂ is unitless. In addition, the error term, ε, has units that are dollars, and so does its standard deviation, σ.

What are some of the implications of this?