Some April Reading for Econometricians

Here are my suggestions for this month:

• Hyndman, R. J., 2019. A brief history of forecasting competitions. Working Paper 03/19, Department of Econometrics and Business Statistics, Monash University.
• Kuffner, T. A. & S. G. Walker, 2019. Why are p-values controversial?. American Statistician, 73, 1-3.
• Sargan, J. D.,, 1958. The estimation of economic relationships using instrumental variables. Econometrica, 26, 393-415. (Read for free online.)  
• Sokal, A. D., 1996. Transgressing the boundaries: Towards a trasnformative hermeneutics of quantum gravity. Social Text, 46/47, 217-252.
• Zeng, G. & Zeng, E., 2019. On the relationship between multicollinearity and separation in logistic regression. Communications in Statistics - Simulation and Computation, published online.
• Zhang, X., S. Paul, & Y-G. Yang, 2019. Small sample bias correction or bias reduction? Communications in Statistics - Simulation and Computation, published online.

© 2019, David E. Giles

Recommended Reading for October

• Andor, N. & C. Parmeter, 2017. Pseudolikelihood estimation of the stochastic frontier model. Ruhr Economic Papers #693.
• Chalak, K., 2017. Instrumental variables methods with heterogeneity and mismeasured instruments. Econometric Theory, 33, 69-104.
• Kim, J. H. & I. Choi, 2017. Unit roots in economic and financial time series: A re-evaluation at the decision-based significance levels. Econometrics, 56 (3), 41.
• Owen, A. B., 2017. Statistically efficient thinning of a Markov chain sampler. Journal of Computational and Graphical Statistics, 26, 738-744. 
• Owen, P. D., 2017. Evaluating ingenious instruments for fundamental determinants of long-run economic growth and development. Econometrics, 5 (3), 38.
• Richard, P., 2017. Robust heteroskedasticity-robust tests. Economics Letters, 159, 28-32.

© 2017, David E. Giles

T. W. Anderson: 1918-2016

Unfortunately, this post deals with the recent loss of one of the great statisticians of our time - Theodore (Ted) W. Anderson.

Ted passed away on 17 September of this year, at the age of 98.

I'm hardly qualified to discuss the numerous, path-breaking, contributions that Ted made as a statistician. You can read about those in De Groot (1986), for example.

However, it would be remiss of me not to devote some space to reminding readers of this blog about the seminal contributions that Ted Anderson made to the development of econometrics as a discipline. In one of the "ET Interviews", Peter Phillips talks with Ted about his career, his research, and his role in the history of econometrics.  I commend that interview to you for a much more complete discussion than I can provide here.

(See this post for information about other ET Interviews).

Ted's path-breaking work on the estimation of simultaneous equations models, under the auspices of the Cowles Commission, was enough in itself to put him in the Econometrics Hall of Fame. He gave us the LIML estimator, and the Anderson and Rubin (1949, 1950) papers are classics of the highest order. It's been interesting to see those authors' test for over-identification being "resurrected" recently by a new generation of econometricians. 

There are all sorts of other "snippets" that one can point to as instances where Ted Anderson left his mark on the history and development of econometrics.

For instance, have you ever wondered why we have so many different tests for serial independence of regrsssion errors? Why don't we just use the uniformly most powerful (UMP) test and be done with it? Well, the reason is that no such test (against the alternative of a first-oder autoregresive pricess) exists.

That was established by Anderson (1948), and it led directly to the efforts of Durbin and Watson to develop an "approximately UMP test" for this problem.

As another example, consider the "General-to-Specific" testing methodology that we associate with David Hendry, Grayham Mizon, and other members of the (former?) LSE school of thought in econometrics. Why should we "test down", and not "test up" when developing our models? In other words, why should we start with the most  general form of the model, and then successively test and impose restrictions on the model, rather than starting with a simple model and making it increasingly complex? The short answer is that if we take the former approach, and "nest" the successive null and alternative hypotheses in the appropriate manner, then we can appeal to a theorem of Basu to ensure that the successive test statistics are independent. In turn, this means that we can control the overall significance level for the set of tests to what we want it to be. In contrast, this isn't possible if we use a "Simple-to-General" testing strategy.

All of this spelled out in Anderson (1962) in the context of polynomial regression, and is discussed further in Ted's classic time-series book (Anderson, 1971). The LSE school referred to this in promoting the "General-to-Specific" methodology.

Ted Anderson published many path-breaking papers in statistics and econometrics and he wrote several books - arguably, the two most important are Anderson (1958, 1971). He was a towering figure in the history of econometrics, and with his passing we have lost one of our founding fathers.

References

Anderson, T.W., 1948. On the theory of testing serial correlation. Skandinavisk Aktuarietidskrift, 31, 88-116.

Anderson, T.W., 1958. An Introduction to Multivariate Statistical Analysis. WIley, New York (2nd. ed. 1984).

Anderson, T.W., 1962. The choice of the degree of a polynomial regression as a multiple decision problem. Annals of Mathematical Statistics, 33, 255-265.

Anderson, T.W., 1971. The Statistical Analysis of Time Series. Wiley, New York.

Anderson, T.W. & H. Rubin, 1949. Estimation of the parameters of a single equation in a complete system of stochastic equations. Annals of Mathematical Statistics, 20, 46-63.

Anderson, T.W. & H. Rubin, 1950. The asymptotic properties of the parameters of a single equation in a complete system of stochastic equations. Annals of Mathematical Statistics, 21,570-582.

De Groot, M.H., 1986. A Conversation with T.W. Anderson: An interview with Morris De Groot. Statistical Science, 1, 97–105.

© 2016, David E. Giles

Some Suggested Reading for October

For your enjoyment:

• Diebold, F. X. & M. Shin, 2016. Assessing point forecast accuracy by stochastic error distance. NBER Working Paper No.2516.
• Franses, P.H., 2016. Yet another look at MIDAS regression. Econometric Institute Report 2016-32.
• Hillier, G. & F. Martellosio, 2016. Exact properties of the maximum likelihood estimator in spatial autoregressive models. Discussion Paper DP 07/16, Department of Economics, University of Surrey.
• Li, L., M.J. Holmes, & B.S. Lee, 2016. The asymmetric relationship between executive earnings management and compensation: A panel threshold regression approach. Applied Economics, 48, 5525-5545. 
• Lütkepohl, H., A. Staszewska-Bystrova, & P. Winker, 2016. Calculating joint confidence bands for impulse response functions using highest density regions. MAGKS Joint Discussion Paper 16-2016.
• Segnon, M., R. Gupta, S. Bekiros, & M.E. Wohar, 2016. Forecasting U.S. GNP growth: The role of uncertainty. Working Paper 2016-67, Department of Economics, University of Pretoria.

© 2016, David E. Giles

Graduate Econometrics Exam

Occasionally readers ask about the exams that I set in my graduate econometrics courses.

The elective graduate econometrics course that I taught this past semester was one titled "Themes in Econometrics". The topics that are covered vary from year to year. However, as the title suggests, the course focuses on broad themes that arise in econometrics. Examples might include maximum likelihood estimation and the associated testing strategies;instrumental variables/GMM estimation; simulation methods; nonparametric inference; and Bayesian inference.

This year most of the course was devoted to maximum likelihood, and Bayesian methods in econometrics.

The mid-term test covered the first of these two thematic topics, while the final exam was devoted largely to Bayesian inference.

You can find the mid-term test here. The final exam question paper is here; and the associated R code is here.

© 2016, David E. Giles

Improved Analytic Bias Correction for MLE's

Ryan Godwin and I have a new paper - "Improved Analytic Bias Correction for Maximum Likelihood Estimators". You can download it from here. (This is a revised version, 19 July 2017.)

This paper proposes a modification of the Cox-Snell/Cordeiro-Klein bias correction technique that we've used in our earlier research (including work with Helen Feng and Jacob Schwartz). For some more information about that work, see this earlier post.

© 2016, David E. Giles

Reading List for October

Some suggestions for the coming month:

• Franses, P. H., 2016. A note on the mean absolute scaled error. International Journal of Forecasting, 32, 20-22.
• Gorroochurn, P., 2015. On Galton's change from 'reversion' to 'regression'. American Statistician, in press.
• Holan, S. H., R. Lund, and G. Davis, 2010. The ARMA alphabet soup: A tour of ARMA model variants. Statistics Surveys, 4, 232-274.
• Liu, S., H. Wu, and W. Q. Meeker, 2015. Understanding and addressing the unbounded "likelihood" problem. American Statistician, 69, 191-200.
• McCracken, M. W. and S. Ng, 2015.  FRED-MD: A monthly database for macroeconomic research. Journal of Business and Economic Statistics, in press.
• Müller, U. K. and M. W. Watson, 2015. Low-frequency econometrics. NBER Working Paper No. 21564.

© 2015, David E. Giles

September Reading List

• Abeln, B. and J. P. A. M. Jacobs, 2015. Seasonal adjustment with and without revisions: A comparison of X-13ARIMA-SEATS and CAMPLET. CAMA Working Paper 25/2015, Crawford School of Public Policy, Australian National University.
• Chan, J. C. C. and A. L. Grant, 2015. A Bayesian model comparison for trend-cycle decompositions of output. CAMA Working Paper 31/2015, Crawford School of Public Policy, Australian National University.
• Chen, K. and K-S. Chan, 2015. A note on rank reduction in sparse multivariate regression. Journal of Statistical Theory and Practice, in press.
• Fan, Y., S. Pastorello, and E. Renault, 2015. Maximization by parts in extremum estimation. Econometrics Journal, 18, 147-171.
• Horowitz, J., 2014. Variable selection and estimation in high-dimensional models. Cemmap Working Paper CWP35/15, Institute of Fiscal Studies, Department of Economics, University College London.
• Larson, W., 2015. Forecasting an aggregate in the presence of structural breaks in the disaggregates. RPF Working Paper No. 2015-002, Research Program on Forecasting, Center of Economic Research, George Washington University.

© 2015, David E. Giles

Maximum Likelihood Estimation & Inequality Constraints

This post is prompted by a question raised by Irfan, one of this blog's readers, in some email correspondence with me a while back.

The question was to do with imposing inequality constraints on the parameter estimates when applying maximum likelihood estimation (MLE). This is something that I always discuss briefly in my graduate econometrics course, and I thought that it might be of interest to a wider audience.

Here's the issue.

On the Invariance of MLE's

The Maximum Likelihood Estimator (MLE) is extremely widely used in statistics, and in the various "metrics" disciplines such as econometrics. This is because this estimator has several highly desirable properties, as long as the sample size is sufficiently large.

For example, under fairly weak ("regularity") conditions, the MLE is weakly consistent, asymptotically efficient, and asymptotically normal.

In small samples, the MLE may or may not have good "sampling properties". For instance, it may be biased or unbiased, depending on the estimation problem under consideration.

When teaching this material, instructors invariably mention another nice property of the MLE: it's an "invariant estimator".

What does this actually mean?

Some econometrics texts (e.g., Greene, 2012, p.521) define the invariance property as follows: "If θ* is the MLE of θ, and f( . ) is a 1-1 function, then f(θ*) is the MLE of f(θ)."

In fact, this statement of the property is unduly strong. The function, f( . ), simply needs to be continuous - it doesn't need to be 1-1. (The proof of the result is especially simple if f is 1-1.)

In support of this more general result, I usually refer my students to the note by Zehna (1966). Recently, I became aware of some other important references when I read a paper by Olive (2004).

In particular, Berk's  (1967) review of Zehna's paper provides a simple proof of the general result.

An obvious implication of the generality of the invariance theorem is that if σ*² is the MLE of the population variance, σ², then √(σ*²) is the MLE of σ. This wouldn't be the case if the theorem was restricted to just 1-1 transformations!

Olive's paper will definitely be on my reading guide for students in future courses that I teach.

References

Berk, R., 1967. Review 1922 of ‘Invariance of maximum likelihood estimators’ by Peter W. Zehna. Mathematical Reviews, 33, 342-343.

Greene, W. H., 2012. Econometric Analysis, 7th ed.. Prentice Hall.

Olive, D. J., 2004. Does the MLE maximize the likelihood? Mimeo., Department of Mathematics, Southern Illinois University. (Also included in D. J. Olive, 2014, Statistical Theory and Inference, Springer.)

Zehna, P. W., 1966. Invariance of maximum likelihood estimators. Annals of Mathematical Statistics, 37, 744.

© 2015, David E. Giles

Thanks for Downloading!

In an earlier post I mentioned a paper that I co-authored with Xiao Ling. The paper is "Bias reduction for the maximum likelihood estimator of the parameters of the generalized Rayleigh family of distributions. Communications in Statistics - Theory and Methods, 2014, 43, 1778-1792.

Over the period January to July 2014, this paper was downloaded 144 times from the journal's website. That made it the 6th most downloaded paper for that period - out of all papers downloaded from all volumes/issues of Communications in Statistics - Theory and Methods.

My guess is that some of you were responsible for this. Thanks!

© 2014, David E. Giles

The 7 Pillars of Statistical Wisdom

Yesterday, Stephen Stigler presented the (ASA) President's Invited Address to an overflow, and appreciative, audience at the 2014 Joint Statistical Meetings in Boston. The title of his talk was, "The Seven Pillars of Statistical Wisdom".

I'd been looking forward to this presentation by our foremost authority on the history of statistics, and it surpassed my (high) expectations.

The address will be published in JASA at some future date, and I urge you to read it when it appears. In the meantime, here are the "seven pillars" - the supporting pillars of statistical science - with some brief comments:

Free Eprints of Our Paper

If you're interested in downloading a copy of my recent paper with Xiao Ling, "Bias Reduction for the Maximum Likelihood Estimator of the Parameters of the Generalized Rayleigh Family of Distributions", but don't have a subscription to Communications in Statistics, fear not!

The publisher (Taylor and Francis) will provide free downloads for up to 50 people from here.

Knock yourselves out!

© 2014, David E. Giles

New Paper

 Another of my papers on analytic bias-correction has now been published. This one is with a former M.A. student, Xiao Ling.

The details are: Xiao Ling and David E. Giles, "Bias reduction for the maximum likelihood estimator of the parameters of the generalized Rayleigh family of distributions. Communications in Statistics - Theory and Methods, 2014, 43, 1778-1792.

You can see the paper here.

© 2014, David E. Giles

March Madness in the Reading Department

It's time for the monthly round-up of recommended reading material.

• Gan, L. and J. Jiang, 1999. A test for global maximum. Journal of the American Statistical Association, 94, 847-854.
• Nowak-Lehmann, F., D. Herzer, S. Vollmer, and I. Martinez-Zarzosa, 2006. Problems in applying dynamic panel data models: Theoretical and empirical findings. Discussion Paper Nr. 140, IAI, Georg-August-Universität Göttingen.
• Olive, D. J., 2004. Does the MLE maximize the likelihood? Mimeo., Department of Mathematics, Southern Illinois University. 
• Pollock, D. S. G., 2014. Econometrics: An historical guide for the uninitiated. Working Paper No. 14/05, Department of Economics, University of Leicester.
• Terrell, G. R., 2002. The gradient statistic. Interface 2002: Computing Science and Statistics, Vol. 34.
• Wald, A., 1940. The fitting of straight lines if both variables are subject to error. Annals of Mathematical Statistics, 11, 284-300.

© 2014, David E. Giles

Happy Birthday Sir Ronald Fisher

Ronald Aylmer Fisher.

Born: 17 February, 1890, in East Finchley, London, England 

Knighted: 1952

Other major Honours: Fellow of the Royal Society (1929), Royal Medal (1938), Guy Medal in Gold (1947), Copley Medal (1956), Darwin-Wallace Medal (1958)

Died: 29 July 1962, in Adelaide, S.A., Australia

If you're a student of Econometrics, think of:

• Maximum likelihood estimation
• Fisher's information
• Analysis of variance
• Sampling distributions (for various statistics)
• The null hypothesis
• The F distribution (sort of)

Some previous related posts:

• And in the Red Corner.....
• A Bayesian and Non-Bayesian Marriage
• On the Importance of Gong Global
• Whose F Distribution Was It?

The Fisher digital archives are housed in the University of Adelaide Library.


© 2014, David E. Giles

Estimating the Generalized Pareto Distribution

The generalized Pareto distribution (GPD) arises in the modelling of "extremes", especially if the "peaks-over-threshold" approach is being used. Estimating the parameters of the GPD by the method of maximum likelihood is especially challenging. The challenges arise because the likelihood function doesn't satisfy the usual regularity conditions for all possible values of the parameters.

I've discussed some of these issue in earlier posts, here and here.

When my colleagues, Helen Feng and Ryan Godwin, and I started looking at analytic bias reduction techniques for maximum likelihood estimators that can't be expressed in closed form, we first tackled the case of the GPD. It was well motivated, because you usually start with a very large sample size, the number of extreme data-points that lie above a given threshold, and which form the sample for estimation purposes, is generally very small. So, small-sample bias is a real issue.

Well, we bit off a lot more than we realized at the time, and bias reduction when estimating the GPD's parameters turned into a bit of a nightmare! We published several papers (including ones with Jacob Schwartz) dealing with bias reduction for other distributions, but the GPD problem was always there in the background.

I'm pleased to be able to report that our paper on this problem is now accepted for publication in Communications in Statistics - Theory & Methods. You can access a pre-print here.

© 2014, David E. Giles

More on Student-t Regression Models

My recent post relating to maximum likelihood estimation of non-standard regression models in EViews included the case where the model's errors are independent Student-t distributed. In that example, the degrees of freedom for the Student-t distribution were assumed to be known. There was a good reason for making this assumption, as was spotted by Osman Dogan in his comment on that post.

If we relax this assumption and include the degrees of freedom parameter, v, of the t-distribution as another parameter that has to be estimated, then the likelihood function exhibits some unfortunate characteristics. Specifically, this function becomes unbounded at a boundary of the parameter space. Consequently, maximizing the likelihood function will generally result in us achieving only a local maximum, not a global maximum.

You might ask, "why would this matter?" Well, basically, if you want to be sure that your MLE achieves the good asymptotic properties that motivate us to use it in the first place, then you need to globally maximize the likelihood function.

I discussed this issue in some detail in an earlier post, here.

In the context of the multiple regression model with independent Student-t errors with an unknown degrees of freedom parameter, these issues have been discussed fully by Fernandez and Steel (1999), for example. In particular, those authors show how a Bayesian approach to this estimation problem can overcome the difficulties associated with MLE here.

The problem is very reminiscent of the "incidental parameters" problem that arises widely in statistics, as well as in certain econometric estimation problems. Good examples of this general type of problem in econometrics include "switching regression" models; as well as models of markets that are in disequilibrium; and stochastic frontier production functions.

It's well known that a Bayesian approach is productive in the case of the "incidental parameters" problem, so it shouldn't be too surprising that it's also helpful with the Student-t regression model.

So, if you want to estimate a regression model with independent Student-t errors, and the degrees of freedom parameter associated with that distribution is unknown, then don't use maximum likelihood estimation! The Bayesian estimator discussed by Fernandez and Steel (1999) is one alternative. Pianto (2010) suggests a bootstrap estimator; and another possibility  would be to consider method of moments estimation, which would result in estimates that are at least weakly consistent.

References

Fernandez, C, and M. F. J. Steel, 1999. Multivariate Student-t regression models: Pitfalls and inference. Biometrika, 86, 153-167. (Downloadable version here.)

Pianto, D. M., 2010. A bootstrap estimator for the Student-t regression model.

© 2013, David E. Giles

Maximum Likelihood Estimation in EViews

This post is all about estimating regression models by the method of Maximum Likelihood, using EViews. It's based on a lab. class from one of my grad. econometrics courses.

We don't go through all of the material below in class - PART 3 is left as an exercise for the students to pursue in their own time.

The data and the EViews workfile can be found on the data page and the code page for this blog.

The purpose of this lab. exercise is to help the students to learn how to use EViews to estimate the parameters of a regression model by Maximum Likelihood, when the model is of some non-standard type. Specifically, find lout how to estimate models of types that are not “built in” as a standard option in EViews. This involves setting up the log-likelihood function for the model, based on the assumption of independent observations; and then maximizing this function numerically with respect to the unknown parameters. 

First, to introduce the concepts and commands that are involved, we consider the standard  linear multiple regression model with normal errors, for which we know that the MLE of the coefficient vector is just the same as the OLS estimator. This will give us a “bench-mark” against which to check our understanding of what is going on. Then we can move on to some more general models.

More Interesting Papers to Read

Here's my latest list of suggested reading:

• Bayer, C. and C. Hanck, 2012. Combining non-cointegration tests. Journal of Time Series Analysis, DOI: 10.1111/j.1467-9892.2012.814.x 
• Cipollina, M., L. De Benedictis, L. Salvatici, and C. Vicarelli, 2013.  A note on dummies for policies in gravity models: A Monte Carlo experiment. Working Paper no. 180, Dipartimento di Economia, Università degli studi Roma Tre.
• Fair, R. C., 2013. Reflections on macroeconometric modelling. Cowles Foundation Discussion Paper No. 1908, Yale University.
• Kourouklis, S., 2012. A new estimator of the variance based on minimizing mean squared error. The American Statistician, 66, 234-236.
• Kulish, M. and A. R. Pagan, 2013. Issues in estimating new-Keynesian Phillips curves in the presence of unknown structural change. Research Discussion Paper, RDP 2012-11, Reserve Bank of Australia.
• Little, R. J., 2013. In praise of simplicity, not mathematistry! Ten simple powerful ideas for the statistical scientist. Journal of the American Statistical Association, 108, 359-369.
• Zhang, L., X. Xu, and G. Chen, 2012. The exact likelihood ratio test for equality of two normal populations. The American Statistician, 66, 180-184.

© 2013, David E. Giles