Solution to Regression Problem

O.K. - you've had long enough to think about that little regression problem I posed the other day. It's time to put you out of your misery!

Here's the problem again, with a solution.

Thought for the Day

As a number of writers have noted previously, sales of Christmas cards Granger-cause Christmas, but they certainly don't cause Christmas!

Best wishes for the holiday season.

© 2013, David E. Giles

A Simple Regression Problem

Here's a regression problem for student readers of this blog.

Suppose that we estimate the following regression model by OLS:

                     y_i = α + β x_i + ε_i .

The model has a single regressor, x, and the point estimate of β turns out to be 10.0.

Now consider the "reverse regression", based on exactly the same data:

                    x_i = a + b y_i + u_i .

What can we say about the value of the OLS point estimate of b?

• It will be 0.1.
• It will be less than or equal to 0.1.
• It will be greater than or equal to 0.1.
• It's impossible to tell from the information supplied.

© 2013, David E. Giles

Thomas Bayes - 250 Years On

Two hundred and fifty years ago today a paper titled, "An Essay Towards Solving a Problem in the Doctrine of Chances", was presented to a meeting of the Royal Statistical Society in London. (Although, see here.)

The presenter - Richard Price. The author - (the late) Reverend Thomas Bayes.

Thus, we received "Bayes' Theorem".

A few days ago, the International Society for Bayesian Analysis held a celebratory conference to honour this momentous occasion in the history of statistical and scientific thinking.

Bayesian thinking has had a significant impact on the field of econometrics. My own Ph.D. dissertation (1975) was in Bayesian econometrics, and I was fortunate enough to have had Arnold Zellner as an external examiner.

I just wish I'd had access to the computational technology that's so freely available today!

© 2013, David E. Giles

More on Student-t Regression Models

My recent post relating to maximum likelihood estimation of non-standard regression models in EViews included the case where the model's errors are independent Student-t distributed. In that example, the degrees of freedom for the Student-t distribution were assumed to be known. There was a good reason for making this assumption, as was spotted by Osman Dogan in his comment on that post.

If we relax this assumption and include the degrees of freedom parameter, v, of the t-distribution as another parameter that has to be estimated, then the likelihood function exhibits some unfortunate characteristics. Specifically, this function becomes unbounded at a boundary of the parameter space. Consequently, maximizing the likelihood function will generally result in us achieving only a local maximum, not a global maximum.

You might ask, "why would this matter?" Well, basically, if you want to be sure that your MLE achieves the good asymptotic properties that motivate us to use it in the first place, then you need to globally maximize the likelihood function.

I discussed this issue in some detail in an earlier post, here.

In the context of the multiple regression model with independent Student-t errors with an unknown degrees of freedom parameter, these issues have been discussed fully by Fernandez and Steel (1999), for example. In particular, those authors show how a Bayesian approach to this estimation problem can overcome the difficulties associated with MLE here.

The problem is very reminiscent of the "incidental parameters" problem that arises widely in statistics, as well as in certain econometric estimation problems. Good examples of this general type of problem in econometrics include "switching regression" models; as well as models of markets that are in disequilibrium; and stochastic frontier production functions.

It's well known that a Bayesian approach is productive in the case of the "incidental parameters" problem, so it shouldn't be too surprising that it's also helpful with the Student-t regression model.

So, if you want to estimate a regression model with independent Student-t errors, and the degrees of freedom parameter associated with that distribution is unknown, then don't use maximum likelihood estimation! The Bayesian estimator discussed by Fernandez and Steel (1999) is one alternative. Pianto (2010) suggests a bootstrap estimator; and another possibility  would be to consider method of moments estimation, which would result in estimates that are at least weakly consistent.

References

Fernandez, C, and M. F. J. Steel, 1999. Multivariate Student-t regression models: Pitfalls and inference. Biometrika, 86, 153-167. (Downloadable version here.)

Pianto, D. M., 2010. A bootstrap estimator for the Student-t regression model.

© 2013, David E. Giles

What is an Econometric Model? Objectivity vs. Reflexivity

In response to my recent post, titled, "The History of Econometrics - An Alternative View", Judea Pearl  sent me a thoughtful and intriguing comment. The comment is posted already, but I think that it deserves more than just being tucked away at the bottom of another post.

So, I am giving Judea's comment additional attention here. I hope that you'll find it interesting, and that it will provoke some much-needed discussion.

Here's Judea's comment in its entirety:

Maximum Likelihood Estimation in EViews

This post is all about estimating regression models by the method of Maximum Likelihood, using EViews. It's based on a lab. class from one of my grad. econometrics courses.

We don't go through all of the material below in class - PART 3 is left as an exercise for the students to pursue in their own time.

The data and the EViews workfile can be found on the data page and the code page for this blog.

The purpose of this lab. exercise is to help the students to learn how to use EViews to estimate the parameters of a regression model by Maximum Likelihood, when the model is of some non-standard type. Specifically, find lout how to estimate models of types that are not “built in” as a standard option in EViews. This involves setting up the log-likelihood function for the model, based on the assumption of independent observations; and then maximizing this function numerically with respect to the unknown parameters. 

First, to introduce the concepts and commands that are involved, we consider the standard  linear multiple regression model with normal errors, for which we know that the MLE of the coefficient vector is just the same as the OLS estimator. This will give us a “bench-mark” against which to check our understanding of what is going on. Then we can move on to some more general models.

The History of Econometrics - An Alternative View

There are different ways of looking at history.Professor Annie Cot reminds of this, in the context of econometrics, in one of her dissertations that has been made available here.

"Econometrics has become such an obvious, objective - almost natural - tool that economists often forget that it has a history of its own, a complex and sometimes problematic history. Two works - Morgan (1990) and Qin (1993) - constitute the Received View of the history of econometrics. Basing our analysis on Leo Corry's methodological (and historiographical) framework of image and body of knowledge, the main purpose of this dissertation is to provide a critical account of the Received View.

Our main criticism is that historians of econometrics have a particular image of knowledge that stems from within econometrics itself, generating a problem of reflexivity. This means that historians of econometrics would evaluate econometrics and its history from an econometrician point of view, determining very specific criteria of what should be considered as "true", what should be studied or what should be the questions that the scientific community should ask.

This reflexive vision has conducted the Received View to write an internalist and funnel-shaped version of the History of Econometrics, presenting it as a lineal process progressing towards the best possible solution: Structural Econometrics and Haavelmo's Probability Approach in Econometrics (1944). 

The present work suggests that a new history of econometrics is needed. A new history that would overcome the reflexivity problem yielding a certainly messier and convoluted but also richer vision of econometrics' evolution, rather than the lineal path towards progress presented by the Received View".

If you have a serious interest in the history of our discipline, this is for you.

© 2013, David E. Giles

Dennis Lindley Passes Away

The loss of Dennis Lindley, yesterday, will be received with sadness by Bayesians - econometricians included.

Dennis was a major driving force in the formalization and dissemination of Bayesian thought.

Two posts that comment on his many contributions can be found here and here.

I recall presenting a Bayesian paper at a statistics conference in New Zealand in the 1970's, with Dennis in the front row. It was an unnerving experience!

© 2013, David E. Giles

Proxy Variables and Biased Estimation

Here's a problem from the exam. that one of my econometrics classes sat recently. It's to do with some of the consequences of mis-specifying a regression model, and then applying OLS estimation.

Specifically, let's suppose that data-generating process (the correct model specification) is actually of the form:

                       y = Xβ + ε     ;   ε ~ [0 , σ²I_n] .                         (1)

However, we can't observe the k variables in the X matrix, and instead we replace them with k "proxy variables" (substitutes) that we can observe. So, the model that we actually estimate is:

                      y = X^*β + v .                                                     (2)

The students were asked to show that the usual (unbiased) estimator of σ² is actually biased in this case; and they were asked if they could determine the "direction" of the bias.