Gripe of the Day

I have a pet peeve that I really must get off my chest. Yes, another one! Rest assured, there are plenty more where this comes from.

Logit and Probit models are the joint work-horses of a lot of microeconometric studies. They've been widely used for a very long time, and these days they are frequently used with what I'd call pretty large data-sets - ones where the sample size is in the thousands, or even hundreds of thousands. 

These models are invariably estimated by Maximum Likelihood (ML), and as we all know, ML estimators have great large-sample (asymptotic) properties. Specifically, they're weakly consistent, asymptotically efficient, and asymptotically normal - under specific conditions. These conditions are what we usually term the 'regularity conditions', and these are simply some mild conditions on the derivatives of the underlying density function for the random data in the model. The normal and logistic distributions satisfy these conditions, which were first introduced in the context of the formal properties of ML estimators by Dugué (1937) and Cramér (1946). So, what could go wrong?

Often, when applying ML estimation, the likelihood equations (the first-order conditions) that have to be solved to maximize the log-likelihood function are non-linear in the parameters, and the solution may not be unique. That's to say, there may be several local maxima and minima, and we need to make sure that we have found the solution (the ML estimates) that relate to the global maximum of the function. Basically, this is important to ensure that we get those nice asymptotic properties. In the case of the Logit and Probit models, everything is O.K. The likelihood equations have a unique solution, corresponding to a (global) maximum of the log-likelihood function itself. 

We seem to be in the clear - right? Well, unfortunately we still have a problem. With any ML exercise we need to be sure that we are maximizing the correct likelihood. If not, then there may be some problems. I say "may", because sometimes things don't fall apart totally. An example of this is with the standard models for 'count' data. There, the common Poisson specification, with the mean modelled as the exponential of a linear combination of regressors, is still a 'Quasi-ML' estimator even if the underlying Poisson assumption is wrong. As long as the specification of the mean, in terms of the regressors, is correct, the ML estimator (based on the invalid Poisson specification) for the parameters associated with the mean will still be weakly consistent.

However, we're not always this lucky. What if we set up the model correctly, except that we made the wrong assumption the variance of the underlying distribution? How would this affect the ML estimator for models like Logit and Probit?

Let's think back to something pretty basic. In the linear regression model, certain type of mis-specification have only mild implications for our inferences. For example, although heteroskedasticity renders the usual estimated covariance matrix for the OLS parameter estimator inconsistent, the parameter estimates themselves only lose efficiency - they are still unbiased and consistent. Similarly, although the omission of relevant regressors from the standard linear regression model generally biases the OLS parameter estimator, this bias vanishes if the included and excluded regressors are orthogonal (uncorrelated in the sample).

 

However, these results change if the model is non-linear in the parameters - a fact that is well known (e.g., see the various references on my professional website - here), but essentially ignored in most empirical studies. More specifically, these results change (for the worse) in the context of such non-linear models as Logit, Probit, Tobit, etc. For example, in the presence of heteroskedasticity, the MLE's of the parameters of these models are inconsistent.

 

So, worrying about reporting 'robust' standard errors is of second-order importance. There are much more serious matters to be concerned about! If the parameter estimates themselves are inconsistent, it doesn't matter how big your sample is, or what you do to 'patch up' the (meaningless) standard errors, you're in over your head! Similarly, the omission of relevant covariates (even if they are uncorrelated with the included ones) also renders the MLE's of the parameters themselves inconsistent in Logit and Probit models.

 

I don't know how many times I've asked someone in a seminar -

"Did you test for heteroskedasticity in your Logit model?"

only to get the flippant reply -

"It's O.K. - I've reported het.-consistent strandard errors, and I have x-thousand observations."

No - it's not O.K.. At least not on the basis of this response.

(I'd have a little more sympathy if the speaker said that there was very little numerical difference bewtween the regular standard errors and the het.-consistent standard errors. However, that's not a test - it's just a rough rule-of-thumb that things may not be too bad.)

Lagrange Multiplier (LM) tests for this and other types of mis-specification in Logit, Probit, and related models have long been available (e.g., see Davidson and MacKinnon, 1984, and other references here), but they seem to be largely ignored in awful lot of empirical research. What's wrong with these applied econometricians?

It may be that there's a lack of familiarity with the theoretical literature - but heck, this is not hot-off-the-press news! Maybe there isn't easy access to the code needed to implement these specification tests? Let's check that out.

First, the SHAZAM econometrics package. It doesn't have a heteroskedasticity test for Logit/Probit built in, but they supply an add-on 'procdure' for the standard LM tests at the following URL:  http://shazam.econ.ubc.ca/intro/logit3.htm . So, if you're a SHAZAM user, you have no excuses when it comes to testing! If you want to estimate a Logit or Probit model with heteroskedasticity, then you need to write your own code.

Next, STATA. I'm not a STATA user, but here's what I understand (hat tip to Mark and to Lindsay, readers of a previous version of this). STATA does facilitate estimating a Probit (not a Logit) model with multiplcative heteroskedasticity - at least since Version 6 - via the hetprob command. And along with that estimation there's a likelihood ratio test. The test is based, as always on the restricted (no het.) and unrestricted (het.) versions of the model. As far as I can tell, the simpler LM tests (which require the estimation of only the standard, no-het. Logit or Probit models) don't seem to be there.

Finally, EViews. This package doesn't include estimation routines for Logit or Probit models with heteroskedasticity. Neither does it include heteroskedasticity tests for Logit/Probit. However, for the testing you can download EViews workfiles and associated program files to perform them from my website, and for convenience I've also put these files in the Code page that goes with this blog. Make sure that you read the 'READ-ME' object in each workfile. Also, note that in addition to the heteroskedasticity tests, I also include LM tests for testing the validity of the underlying distribution (Normal in the case of Probit, and Logistic in the case of Logit).

So where does that leave us? Frankly, it would be nice if the major econometrics packages included the various well-known mis-specification tests for Logit, Probit, and related models - not just tests for heteroskedasticity. Why not? They're all simple enough to implement, and they're darned important! While they're at it, it would be nice to see standard routines incorporated to allow us to perform ML estimation of Logit and Probit model with heteroskedasticity.

And then, of course, it's up to us! My gripe is not just with the packages. It's even more to do with all of those people doing applied econometric work who just don't seem to acknowledge that a lot of their results are suspect because they've ignored some basic issues.We can do better than this! 

p.s. (7 October, 2011) The EViews program code referred to above has been corrected today.
HT to eagle-eyed Sven Steinkamp at Universität Osnabrück  for bringing the error to my attention.

Note:  The links to the following references will be helpful only if your computer's IP address gives you access to the electronic versions of the publications in question. That's why a written References section is provided.

References

Cramér, H. (1946). Mathematical Methods of Statistics, Princeton University Press, Princeton N.J.

Davidson, R. and J. G. MacKinnon (1984). Convenient specification tests for logit and probit models. Journal of Econometrics 25, 241-262.

Dugué, D. (1937). Application des propriétés de la limite au sens du calcul des probabilités à l’étude de diverses questions d’estimation. Journal de l’École Polytechnique, 3, 305-372.

© 2011, David E. Giles