Sunday, January 11, 2015

Econometrics vs. Ad Hoc Empiricism

In a post in 2013, titled "Let's Put the "ECON" Back Into Microeconometrics", I complained about some of the nonsense that is passed off as "applied econometrics". Specifically, I was upset about the disconnect between the economic model (if there is one) and the empirical relationships that are actually estimated, in many "applied" papers.

I urge you to look back at the post before reading further.

Here's a passage from that post:
"In particular, how often have you been presented with an empirical application that's based on just a reduced-form model that essentially ignores the nuances of the theoretical model?
I'm not picking on applied microeconomic papers - really, I'm not! The same thing happens with some applied macroeconomics papers too. It's just that in the micro. case, there's often a much more detailed and rich theoretical model that just lends itself to some nice structural modelling. And then all we see is a regression of the logarithm of some variable on a couple of interesting covariates, and a bunch of controls - the details of which are frequently not even reported."
Well, things certainly haven't improved since I wrote that. In fact, it seems that I'm encountering more and more of this nonsense. This isn't "econometrics", and the purveyors of this rubbish aren't "econometricians". 

My real concern is that students who are exposed to these papers and seminars may not recognize it for what it is - just ad hoc empiricism. 


© 2015, David E. Giles

12 comments:

  1. Dave,
    for those of us who have not studied this since being at Uni a long time ago you are a Godsend!

    ReplyDelete
  2. This was also the conclusion of presentation (ASSA 2015) by Professor Amir Sufi (available at http://events.mediasite.com/Mediasite/Play/7d2207a4b2f049f4a9473b21752591221d after 1 hr 15minute)

    "Theory and empirics are both necessary, and they should go hand in hand"

    ReplyDelete
  3. Well, there are a lot of researchers from economics, doing some really hard work on the economic foundation and then, while sticking to theory, the statistical part of their research is right from the 101-course, neglecting almost every recent advance from statistical theory. In essence, almost everything is linear in the end ...

    On the other hand it seems to be tempting to have all those big hammers from statistics:searching for economic nails leads directly to "data-driven" modeling. Often then "fit" or "predictive accuracy" in some sense is used to conclude that the latter models are deemed more suitable compared to theory-driven models.

    Not only the "statistical sword" should not be yielded blindly. Also there is no excuse for sticking to theory and ignoring modern statistics. The latter also produces garbage.

    ReplyDelete
  4. David,
    I am an infrequent visitor to your interesting website. I have just read your recent blogs on both the need for more economics content and also your expositional piece on ARDL estimation in Eviews.
    Although I understand that the ARDL piece was primarily expositional, I must say that it an excellent example of the type econometrics that drives me to despair. Let me list a few of the things that I see need attention:
    1. The exposition contains no economics. What happened to the demand curve and the price-quantity nexus? Gasoline is a joint product with a lot of other petroleum products that come out of the refining process, and at least in the short run, those products (diesel, propane, kerosene, fuel oil, petroleum coke etc.) are produced in close to fixed proportions. How much the price of gasoline moves in response to a change in the price of crude will depend on the relative price elasticities of all these output products.
    2. You talk about the long run impact of crude price on gasoline price, but your dynamics are working their way through to the price of gasoline in a matter of weeks. There are plenty of studies out there that suggest that the long run price elasticity of demand for gasoline might be twice high in the long run as in the short run (at three years about -0.5 versus -0.25 at one year). The exact elasticity estimates are not the issue here. The concern is the disconnect that occurs because using weekly data causes one year to become the super long run. In this instance, fascination with the asymptotics of the weekly data distracts from what (the literature at least) seems to say is the real world long-run mechanics of the price adjustment process. Perhaps your example shows that the existing literature about long-run adjustments is incorrect, but the economics needs to be explained.
    3. Why convert the data to log form? Why would there be expected to be a percentage mark-up of crude prices to gasoline prices? Isn’t that rather a naive step away from real economics? Surely, the sensible approach is to work in dollars (cents per litre of gasoline and cents per litre of crude).
    4. How about using inflation-adjusted real prices? That gets rid of one unexplained element in the trend variable.
    5. What is in the trend variable? Environmental regulation? Labour costs? Taxes?
    6. I know the economics profession loves to natter about I(1) and I(2) process. Infinite T creates such beautiful math. However, I find it awkward to imagine that crude oil prices follow an I(1) process. There are economic forces that control crude prices. Like interest rates, they only look non-stationary when you analyse them on the wrong time scale.
    7. The illustration is time series analysis, so where are the seasonal variables. Gasoline prices are always higher by 3 or 4 cents per litre in the summer months.
    8. Any weekly climate factors?
    Again, I do understand that your blog on ARDL model estimation was to show how Eviews might be used to execute some statistical calculations, and that it was not a piece of publishable research on the petroleum industry. My point is quite specific. Too much academic effort is spent teaching relatively useless technique at the cost of failing to look carefully at the data and the economic context.
    Casey

    ReplyDelete
    Replies
    1. Casey - sorry if I drive you to despair. As you correctly noted, in your last paragraph, the ARDL post is designed to show readers how to implement a particular technique using a particular package. That's all!
      You make many points that would be valid if the post was a journal article and you were a referee. That's not the case, and I'm sure that the majority of readers know that all too well. However, thanks for your comments.

      Delete
  5. You say: "In fact, it seems that I'm encountering more and more of this nonsense. This isn't "econometrics", and the purveyors of this rubbish aren't "econometricians. My real concern is that students who are exposed to these papers and seminars may not recognize it for what it is - just ad hoc empiricism."

    Are you serious? Maybe refer to my moderator-snipped comments on your ARDL blog.

    Keep morale high.
    Casey

    ReplyDelete
    Replies
    1. Casey: I must say that when this comment arrived, I was left puzzled. What on earth could it mean? I do get a LOT of really CRAZY comments. To answer your question (rthetorical or not) - yes, I AM serious.

      Then it occurred to me that you "moderator-snipping" remark may refer to another comment that you may have sent previously. So, I searched the "junk" box, and sure enough there was another comment from you (relating to this and another post) that had wrongly been given a junk assignation. I apologise for that. You'll find that your earlier comment, with my response is now there. Thanks for your interest.

      Delete
  6. Anonymous said...

    Good day Prof. I discovered its fascinating. Thanks a lot for the great and inspiring job.
    My question is this. The exercise you just demonstrated, what is the interpretation or economic significance of the coefficients of the

    ReplyDelete
  7. Sorry my question is half way. The exercise you just demonstrated, what is the interpretation or economic significance of the coefficients of the structural break

    ReplyDelete
    Replies
    1. Sorry - still puzzled - did you mean this to go to a different post?

      Delete