Saturday, April 21, 2012

Bayesian Econometrics - Forty Years On

My Ph.D. dissertation was in Bayesian Econometrics. I started working on the dissertation early in 1973, and Arnold Zellner's classic text had been available for just over a year. So, perhaps not surprisingly, one of the first things that I did was to sit down and go through his book with a fine tooth comb. It took quite a while, but it was time well spent!

My copy of Zellner (1971) is still on my shelf, and it's a first printing. So, of course, there are the inevitable type-setting/proof-reading errors. I was stuck in New Zealand and there was no internet, of course, so you couldn't download an addendum. When I encountered something that didn't seem "quite right", I just had to slog through everything from first principles (with numerous visits to the library), until I was sure that I understood what was going on.

The trouble started with the first equation in Chapter 2, "Principles of Bayesian Analysis With Selected Applications". There, Bayes' Theorem is stated, as follows:
"Then, according to the usual operations with pdf's we have 
p(y , θ) = p(y | θ) p(θ)
= p(θ | y) p(θ)    "
It didn't require a trip to the library in order to correct the last line to:

           = p(θ | y) p(y),

but this illustrates what I was up against!

Missing or mis-placed parentheses; square root signs that shouldn't have been there; wrong signs;..................... all of the usual challenges. 

And then there (still) are those wonderful "throw away" lines that we all hate to encounter, such as:
  • "On integrating with respect to σ we obtain....".
  • "On completing the square on β we have...."
  • "On substituting in (3.56) and integrating with respect to the k elements of β we obtain..."
  • "The integrations over σ1 and σ2 are easily performed to yield the following joint posterior pdf...."
Some of that integration is a little tricky, and requires a page or two of manipulations. (No hints are given, either.) And can you remember how to "complete the square"? You did it when you learned how to solve quadratic equations, but that was for the scalar case, not the vector/matrix case! Oh joy!

This was all pre-MCMC days, and so the implementation of the Bayesian analysis was computationally challenging, to say the least. I extended the FORTRAN programs in Appendix C of the book to the three-dimensional case. That was as far as one could go! Even plotting out posteriors meant writing specific FORTRAN code - and dealing with the darned punch-cards

None the less, I managed to include some applied work in what was otherwise a theory-oriented dissertation (e.g., here and here).

The other day I was cleaning out a filing cabinet and I found some yellowing hand-written notes. They turned out to be my notes that summarize most of the chapters in Zellner (1971), correct the early typos., and (more importantly) fill in the gaps by working through the necessary integration and the like. The notes for a few of the chapters have gone missing over the past forty years, and during the course of three international moves.

Quite a "blast from the past", though!

Arnold gave me a phenomenal amount of help during that start-up period. I thought that perhaps students of Bayesian econometrics might appreciate having access to those notes, so for what they're worth, you can find them here.


Reference

Zellner, A., 1971. An Introduction to Bayesian Inference in Econometrics. Wiley, New York.


© 2012, David E. Giles

5 comments:

  1. Very pretty handwriting, I'm serious! Haha j/k. Anyway, I have always come across Bayesian method in econometrics but have never taken the effort to understand them. From your experience what is the biggest difference/advantages of using bayesian methods over the conventional frequentist method?

    Kind regards,
    vtov

    ReplyDelete
    Replies
    1. Thanks! Not so good these days - thanks to word processing packages!

      Advantages:
      1. Fully flexible way of incorporating prior information.
      2. Ease of checking robustness of results to different priors.
      3. ONE thematic way of approaching all inference problems. (Not ad hoc, depending on the problem.)
      4. Estimators are always admissible.
      5. Optimal results in small samples.
      6. No reliance on notion of "repeated samples" and hence the sampling distribution

      On the downside I use to put computational difficulties, but that's long since gone.

      I should be doing more Bayesian econometrics!

      Delete
  2. Thanks...Zellner was my thesis chair's thesis chair, nice to see this.

    ReplyDelete