From time to time I've been known to blog about the importance of fully understanding the assumptions that underlie the various estimators and tests that we use in econometrics. (Here, too.) Gee, I've even gone so far as to suggest that students should learn about these assumptions by taking courses where results are proved formally- not introduced simply through arm-waving.
I'm not going to start griping about all of that again here - it's too nice a Spring day for that.
However, I've just been reading a recent piece in Scientific American that's relevant to my main concern when students are taught "how to do" econometrics, but don't have a proper understanding of the underlying assumptions. That concern is simply that, sooner or later, they'll screw up!
Maybe it won't be the end of the world. The economy probably won't collapse in a big messy heap. Perhaps they'll just lose their job!
The S.A. article was about just this sort of thing - but in the case of neuroscientists, not economists. For the sake of full disclosure I have nothing at all against neuroscientists. In fact, I have a daughter who is doing post-grad. work in just that field at the Florey Institute in Australia.
You can read the article for yourself, and I hope that you will. In a nutshell, there have been numerous influential neuroscience studies, that have appeared in the very top scientific journals, and which have been based on fundamentally flawed statistical analysis.
To put it really simply, the authors have used statistical tests whose validity require that the data have been sampled independently, when in fact this requirement is undeniably violated in these studies.
Oh dear!
To quote Gary Stix, the author of the article:
'Emery N. Brown, a professor of computational neuroscience in the department of brain and cognitive sciences at the MIT-Harvard Division of Health Sciences and Technology, points to a dire need to bolster the level of statistical sophistication brought to bear in neuroscience studies. “There’s a fundamental flaw in the system and the fundamental flaw is basically that neuroscientists don’t know enough statistics to do the right things and there’s not enough statisticians working in neuroscience to help that." 'I'd venture to guess that "screwing up" in the neurosciences might have some unpleasant consequences.
Needless to say, I've sent the link to the S.A. article to my daughter!
You might want to think about this the next time you fire up your favourite econometrics package: Did your friendly econometrics instructor make sure that you really understand the assumptions that need to be satisfied before you can rely on the estimators and tests you're about to use?
