The Maximum Likelihood Estimator (MLE) is extremely widely used in statistics, and in the various "metrics" disciplines such as econometrics. This is because this estimator has several highly desirable properties, as long as the sample size is sufficiently large.
For example, under fairly weak ("regularity") conditions, the MLE is weakly consistent, asymptotically efficient, and asymptotically normal.
In small samples, the MLE may or may not have good "sampling properties". For instance, it may be biased or unbiased, depending on the estimation problem under consideration.
When teaching this material, instructors invariably mention another nice property of the MLE: it's an "invariant estimator".
What does this actually mean?
Some econometrics texts (e.g., Greene, 2012, p.521) define the invariance property as follows: "If θ* is the MLE of θ, and f( . ) is a 1-1 function, then f(θ*) is the MLE of f(θ)."
In fact, this statement of the property is unduly strong. The function, f( . ), simply needs to be continuous - it doesn't need to be 1-1. (The proof of the result is especially simple if f is 1-1.)
For example, under fairly weak ("regularity") conditions, the MLE is weakly consistent, asymptotically efficient, and asymptotically normal.
In small samples, the MLE may or may not have good "sampling properties". For instance, it may be biased or unbiased, depending on the estimation problem under consideration.
When teaching this material, instructors invariably mention another nice property of the MLE: it's an "invariant estimator".
What does this actually mean?
Some econometrics texts (e.g., Greene, 2012, p.521) define the invariance property as follows: "If θ* is the MLE of θ, and f( . ) is a 1-1 function, then f(θ*) is the MLE of f(θ)."
In fact, this statement of the property is unduly strong. The function, f( . ), simply needs to be continuous - it doesn't need to be 1-1. (The proof of the result is especially simple if f is 1-1.)
In support of this more general result, I usually refer my students to the note by Zehna (1966). Recently, I became aware of some other important references when I read a paper by Olive (2004).
In particular, Berk's (1967) review of Zehna's paper provides a simple proof of the general result.
An obvious implication of the generality of the invariance theorem is that if σ*2 is the MLE of the population variance, σ2, then √(σ*2) is the MLE of σ. This wouldn't be the case if the theorem was restricted to just 1-1 transformations!
Olive's paper will definitely be on my reading guide for students in future courses that I teach.
References
Berk, R., 1967. Review 1922 of ‘Invariance of maximum likelihood estimators’ by Peter W. Zehna. Mathematical Reviews, 33, 342-343.
Greene, W. H., 2012. Econometric Analysis, 7th ed.. Prentice Hall.
Olive, D. J., 2004. Does the MLE maximize the likelihood? Mimeo., Department of Mathematics, Southern Illinois University. (Also included in D. J. Olive, 2014, Statistical Theory and Inference
, Springer.)
, Springer.)
Zehna, P. W., 1966. Invariance of maximum likelihood estimators. Annals of Mathematical Statistics, 37, 744.
..good work
ReplyDelete