More on Student-t Regression Models

My recent post relating to maximum likelihood estimation of non-standard regression models in EViews included the case where the model's errors are independent Student-t distributed. In that example, the degrees of freedom for the Student-t distribution were assumed to be known. There was a good reason for making this assumption, as was spotted by Osman Dogan in his comment on that post.

If we relax this assumption and include the degrees of freedom parameter, v, of the t-distribution as another parameter that has to be estimated, then the likelihood function exhibits some unfortunate characteristics. Specifically, this function becomes unbounded at a boundary of the parameter space. Consequently, maximizing the likelihood function will generally result in us achieving only a local maximum, not a global maximum.

You might ask, "why would this matter?" Well, basically, if you want to be sure that your MLE achieves the good asymptotic properties that motivate us to use it in the first place, then you need to globally maximize the likelihood function.

I discussed this issue in some detail in an earlier post, here.

In the context of the multiple regression model with independent Student-t errors with an unknown degrees of freedom parameter, these issues have been discussed fully by Fernandez and Steel (1999), for example. In particular, those authors show how a Bayesian approach to this estimation problem can overcome the difficulties associated with MLE here.

The problem is very reminiscent of the "incidental parameters" problem that arises widely in statistics, as well as in certain econometric estimation problems. Good examples of this general type of problem in econometrics include "switching regression" models; as well as models of markets that are in disequilibrium; and stochastic frontier production functions.

It's well known that a Bayesian approach is productive in the case of the "incidental parameters" problem, so it shouldn't be too surprising that it's also helpful with the Student-t regression model.

So, if you want to estimate a regression model with independent Student-t errors, and the degrees of freedom parameter associated with that distribution is unknown, then don't use maximum likelihood estimation! The Bayesian estimator discussed by Fernandez and Steel (1999) is one alternative. Pianto (2010) suggests a bootstrap estimator; and another possibility  would be to consider method of moments estimation, which would result in estimates that are at least weakly consistent.

References

Fernandez, C, and M. F. J. Steel, 1999. Multivariate Student-t regression models: Pitfalls and inference. Biometrika, 86, 153-167. (Downloadable version here.)

Pianto, D. M., 2010. A bootstrap estimator for the Student-t regression model.

© 2013, David E. Giles

What is an Econometric Model? Objectivity vs. Reflexivity

In response to my recent post, titled, "The History of Econometrics - An Alternative View", Judea Pearl  sent me a thoughtful and intriguing comment. The comment is posted already, but I think that it deserves more than just being tucked away at the bottom of another post.

So, I am giving Judea's comment additional attention here. I hope that you'll find it interesting, and that it will provoke some much-needed discussion.

Here's Judea's comment in its entirety:

Maximum Likelihood Estimation in EViews

This post is all about estimating regression models by the method of Maximum Likelihood, using EViews. It's based on a lab. class from one of my grad. econometrics courses.

We don't go through all of the material below in class - PART 3 is left as an exercise for the students to pursue in their own time.

The data and the EViews workfile can be found on the data page and the code page for this blog.

The purpose of this lab. exercise is to help the students to learn how to use EViews to estimate the parameters of a regression model by Maximum Likelihood, when the model is of some non-standard type. Specifically, find lout how to estimate models of types that are not “built in” as a standard option in EViews. This involves setting up the log-likelihood function for the model, based on the assumption of independent observations; and then maximizing this function numerically with respect to the unknown parameters. 

First, to introduce the concepts and commands that are involved, we consider the standard  linear multiple regression model with normal errors, for which we know that the MLE of the coefficient vector is just the same as the OLS estimator. This will give us a “bench-mark” against which to check our understanding of what is going on. Then we can move on to some more general models.

The History of Econometrics - An Alternative View

There are different ways of looking at history.Professor Annie Cot reminds of this, in the context of econometrics, in one of her dissertations that has been made available here.

"Econometrics has become such an obvious, objective - almost natural - tool that economists often forget that it has a history of its own, a complex and sometimes problematic history. Two works - Morgan (1990) and Qin (1993) - constitute the Received View of the history of econometrics. Basing our analysis on Leo Corry's methodological (and historiographical) framework of image and body of knowledge, the main purpose of this dissertation is to provide a critical account of the Received View.

Our main criticism is that historians of econometrics have a particular image of knowledge that stems from within econometrics itself, generating a problem of reflexivity. This means that historians of econometrics would evaluate econometrics and its history from an econometrician point of view, determining very specific criteria of what should be considered as "true", what should be studied or what should be the questions that the scientific community should ask.

This reflexive vision has conducted the Received View to write an internalist and funnel-shaped version of the History of Econometrics, presenting it as a lineal process progressing towards the best possible solution: Structural Econometrics and Haavelmo's Probability Approach in Econometrics (1944). 

The present work suggests that a new history of econometrics is needed. A new history that would overcome the reflexivity problem yielding a certainly messier and convoluted but also richer vision of econometrics' evolution, rather than the lineal path towards progress presented by the Received View".

If you have a serious interest in the history of our discipline, this is for you.

© 2013, David E. Giles

Dennis Lindley Passes Away

The loss of Dennis Lindley, yesterday, will be received with sadness by Bayesians - econometricians included.

Dennis was a major driving force in the formalization and dissemination of Bayesian thought.

Two posts that comment on his many contributions can be found here and here.

I recall presenting a Bayesian paper at a statistics conference in New Zealand in the 1970's, with Dennis in the front row. It was an unnerving experience!

© 2013, David E. Giles

Proxy Variables and Biased Estimation

Here's a problem from the exam. that one of my econometrics classes sat recently. It's to do with some of the consequences of mis-specifying a regression model, and then applying OLS estimation.

Specifically, let's suppose that data-generating process (the correct model specification) is actually of the form:

                       y = Xβ + ε     ;   ε ~ [0 , σ²I_n] .                         (1)

However, we can't observe the k variables in the X matrix, and instead we replace them with k "proxy variables" (substitutes) that we can observe. So, the model that we actually estimate is:

                      y = X^*β + v .                                                     (2)

The students were asked to show that the usual (unbiased) estimator of σ² is actually biased in this case; and they were asked if they could determine the "direction" of the bias.

On Staying Awake in Class

Pedagogy Unbound  is "A place for college teachers to share practical strategies for today's classrooms."

Their blog today contains a lovely piece by David Gooblar, titled " 'Trucker Tricks' for Keeping Students Awake". Not that any of you would have such problems on either side of the rostrum in econometrics classes, I'm sure!

David writes:

"Of all the tips that have been posted at Pedagogy Unbound since the site’s launch in August, the one that has been read the most—by far—is titled “Help your students stay awake in class.” This, it seems, is what professors are most concerned about. Not student writing. Not the plague of plagiarism. Not even students who don’t participate in discussions. No, the most pressing problem facing college teachers today is merely getting their students to stay conscious for an hour and 15 minutes.

.......The tip is one (from an associate professor who) worked as a truck driver when he was a college student and now, as a teacher, he passes on his “trucker tricks” for staying awake to his students. 

......Crucially, (he) follows up these tips by inviting the students, at any time during class throughout the term, to stand up if they feel like they are getting sleepy. They can take their notebooks and stand at the back or side of the classroom. He tells them he’d much prefer a class full of standing students to one full of sleeping, or even just drowsy, students. 

.......So ask your students to stand up if they feel they are in danger of falling asleep. If it turns out that a number of them take you up on the offer, you can tell yourself that you’re getting a standing ovation." 

I'm going to try it out!

© 2013, David E. Giles

Time for Some More Reading!

With the weekend upon us once again, it's time to settle down with the papers - the econometrics research papers, that is. Here are my latest picks:

• Cook, S., D. Watson, and L. Parker, 2014. New evidence on the importance of gender and asymmetry in the crime-unemployment relationship. Applied Economics, 46, 119-126.
• Fan, J., F. Han, and H. Liu, 2013. Challenges of big data analysis. Mimeo.
• Hashmi, A. R., 2014. Competition and innovation: The inverted-U relationship revisited. Review of Economics and Statistics, in press.
• Juselius, K., N. F. Moller, and F. Tarp, 2104. The long-run impact of foreign aid in 36 African countries: Insights from multivariate time series analysis. Oxford Bulletin of Economics and Statistics, in press.
• Li, R., D. K. J. Lin, and B. Li, 2013. Statistical inference in massive data sets. Applied Stochastic Models in Business and Industry, 29, 399-409.
• Sanderson, E. and F. Windmeijer, 2013. A weak instrument F-test in linear IV models with multiple endogenous variables. CEMMAP Working Paper CWP58/13, The Institute for Fiscal Studies.

© 2013, David E. Giles

Data Do Not Imply Science

As a follow-up to my recent post on Big Data, I recommend today's post by Jeff Leek on the Simply Statistics blog. It's titled. 'The key word in "Data Science is not Data, it is Science'.

Jeff says:

"Most people hyping data  science have focused on the first word: data. They care about volume and velocity and whatever other buzzwords describe data that is too big for you to analyze in Excel. .........

But the key word in data science is not "data"; it is "science". Data science is only useful when the data are used to answer a question. That is the science part of the equation. The problem with this view of data science is that it is much harder than the view that focuses on data size or tools. It is much, much easier to calculate the size of a data set and say "My data are bigger than yours"......"

Right on, Jeff!

© 2013, David E. Giles

When Everything Old is New Again

We see it with clothing styles. Not just hemline lengths, but also the widths of jacket lapels and guy's ties. How wide should the trouser legs be? Cuffs or no cuffs? Leave your clothes in the closet long enough, and there's a good chance they'll be back in style some day!

And so it is with econometrics. Here are just a few examples: