MCMC for Econometrics Students - II

This is the second in a set of posts about Monte Carlo Markov Chain (MCMC, or MC²) methods in Bayesian econometrics. The background was provided in this first post, where the Gibbs sampler was introduced.

The main objective of the present post is to convince you that this MCMC stuff actually works!

To achieve this, what we're going to do is work through a simple example - one for which we actually know the answer in advance. That way, we'll be able to check our results from applying the Gibbs sampler with the facts. Hopefully, we'll then be able to see that this technique works - at least for this example! 

I'll be using some R script that I've written to take students through this, and it's available on the code page for this blog. I should  mention in advance that this code is not especially elegant. It's been written, quite deliberately, in a step-by-step manner to make it relatively transparent to non-users of R. Hopefully, the comments that are embedded in the code will also help.

It's also important to note that this first illustration of the Gibbs sampler in action does not involve the posterior distribution for the parameters in a Bayesian analysis of some model.  Instead, we're going to look at the problem of obtaining the marginals of a bivariate normal distribution, when we know the form of the conditional distributions.

In other words - let's proceed one step at a time. The subsequent posts on this topic will be dealing with Bayesian posterior analysis.

Let's take a look at the set-up, and the analysis that we're going to undertake.

MCMC for Econometrics Students - I

This is the first of a short sequence of posts that discuss some material that I use when teaching Bayesian methods in my graduate econometrics courses.

This material focuses on Markov Chain Monte Carlo (MCMC) methods - especially the use of the Gibbs sampler to obtain marginal posterior densities. This first post discusses some of the computational issues associated with Bayesian econometrics, and introduces the Gibbs sampler. The follow-up posts will illustrate this technique with some specific examples.

So, what's the computational issue here?

A New Statistics Journal

A big hat-tip to Rob Hyndman for (indirectly) alerting me to an interesting new statistics journal: The Annual Review of Statistics and its Application.

There are some terrific review articles in the first issue, and several of these are "must-reads" for students of econometrics and practising econometricians.

 I especially like:

• Stephen E. Fienberg, What is Statistics?
• Tilmann Gneiting and Matthias Katzfuss, Probabilistic Forecasting.
• Christian P. Robert, Bayesian Computational Tools.
• Radu V. Craiu and Jeffrey S. Rosenthal, Bayesian Computation via Markov Chain Monte Carlo. 

Looking forward to the next issue!


© 2014, David E. Giles

No Pressure Here

This post might make me sound a little grumpy. I hope not. Anyway, here goes.

Comments that are posted on this blog come to me by email for "approval" prior to posting. This is standard practice, and believe me, you wouldn't want t see some of the spam that people try to post as "comments".

Among the comments waited to be vetted this morning were two at opposite ends of the non-spam spectrum.

One was a grateful and thoughtful comment from "Tom" on my post, ARDL Models - Part I . Here's the exchange in full:

Research on the Interpretation of Confidence Intervals

Like a lot of others, I follow Andrew Gelman's blog with great interest, and today I was especially pleased to see this piece relating to a recent study on the extent to which researchers do or do not interpret confidence intervals correctly.

If you've ever taught an introductory curse on statistical inference (from a frequentist, rather than Bayesian perspective), then I don't need  to tell you how difficult it can be for students to really understand what a confidence interval is, and (perhaps more importantly) what it isn't!

It's not only students who have this problem. Statisticians acting as "expert witnesses" in court cases have no end of trouble getting judges to understand the correct interpretation of a confidence interval. And I'm sure we've all seen or heard empirical researchers misinterpret confidence results! For a specific example of the latter, involving a subsequent Nobel laureate, see my old post here!

The study that's mentioned by Andrew today was conducted by four psychologists (Hoekstra et al., 2014) and involved a survey of academic psychologists at three European Universities. The participants included 442 Bachelor students, 34 Master students, and 120 researchers (Ph.D. or faculty members).

Yes, the participants in this survey are psychologists, but we won't hold that against them, and my hunch is that if we changed "psychologist" to "economist" the results wouldn't alter that much!

Before summarizing the findings of this study, let's see what the authors have to say about the correct interpretation of a confidence interval (CI) constructed from a particular sample of data:

Seminars by the Number - Redux

In my second post on this blog, just over three years ago, I took a shot at seminars - economics seminars in particular. There's nothing there that I want to retract. I still remain bemused by the duration of economics seminars; the time that's wasted on details rather than "the big picture"; and the proportion of the allotted time that's taken up with audience "participation".

This being the case, I thought I'd update my earlier suggestion for streamlining these seminars. The focus is on seminars of an econometric nature - very occasionally we actually do have such events in my department.

Here's what I suggested in that earlier post:

Testing for Multivariate Normality

In a recent post I commented on the connection between the multivariate normal distribution and marginal distributions that are normal. Specifically, the latter do not necessarily imply the former.

So, let's think about this in terms of testing for normality.

Suppose that we have several variables which we think may have a joint distribution that's normal. We could test each of the variables for normality, separately, perhaps using the Jarque-Bera LM test. If the null hypothesis of normality was rejected for one or more of the variables, this could be taken as evidence against multivariate normality. However, if normality couldn't be rejected for any of the variables, this wouldn't tell us anything about their joint distribution.

What we need is a test for multivariate normality itself. Let's see what's available.

March Madness in the Reading Department

It's time for the monthly round-up of recommended reading material.

• Gan, L. and J. Jiang, 1999. A test for global maximum. Journal of the American Statistical Association, 94, 847-854.
• Nowak-Lehmann, F., D. Herzer, S. Vollmer, and I. Martinez-Zarzosa, 2006. Problems in applying dynamic panel data models: Theoretical and empirical findings. Discussion Paper Nr. 140, IAI, Georg-August-Universität Göttingen.
• Olive, D. J., 2004. Does the MLE maximize the likelihood? Mimeo., Department of Mathematics, Southern Illinois University. 
• Pollock, D. S. G., 2014. Econometrics: An historical guide for the uninitiated. Working Paper No. 14/05, Department of Economics, University of Leicester.
• Terrell, G. R., 2002. The gradient statistic. Interface 2002: Computing Science and Statistics, Vol. 34.
• Wald, A., 1940. The fitting of straight lines if both variables are subject to error. Annals of Mathematical Statistics, 11, 284-300.

© 2014, David E. Giles

The Normality of Joint and Marginal Distributions

I'm often surprised how many people are confused when it comes to joint and marginal normal distributions.

Most students of econometrics are taught that the marginal and conditional distributions associated with a multivariate normal random vector are themselves normal. That is, if

         p(x₁, x₂, ...., x_n) ~ MVN[μ₁, ....., μ_n ; V]              ;      where V = {v_ij}

then
         p(x_i) ~ N[μ_i ; v_ii]  .

Similarly, p(x₁ | x₂, x₃, ...., x_n), and all of the other conditional densities are normal.

However, what they don't seem to get taught is that the converse is not true. That is, if we have several random variables, each with normal marginal distributions, then the joint distribution of these variables is not necessarily normal.

It all depends on what copula is used to construct the joint distribution.


© 2014, David E. Giles

Arnold's "Signature"

Anonymity is part of the culture when it comes to refereeing papers submitted for possible publication in economics, econometrics, and statistics. Referees' names are typically "blinded", and some journals use a "double-blind" process, so that authors names are not know by the referees. Not all disciplines use this approach.

The double-blind approach is far from perfect, especially given how easy it often is to identify authors  by locating a "working paper" version of their article through an internet search. In addition, referees often effectively "reveal" their identity by insisting that authors include references to the referee's own work.

Sometimes, though, referees expose themselves quite unwittingly. Here's a case in point.