Monday, April 8, 2019

A Permutation Test Regression Example

In a post last week I talked a bit about Permutation (Randomization) tests, and how they differ from the (classical parametric) testing procedure that we generally use in econometrics. I'm going to assume that you've read that post.

(There may be a snap quiz at some point!)

I promised that I'd provide a regression-based example. After all, the two examples that I went through in that previous post were designed to expose the fundamentals of permutation/randomization testing. They really didn't have much "econometric content".

In what follows I'll use the terms "permutation test" and "randomization test" interchangeably.

What we'll do here is to take a look at a simple regression model and see how we could use a randomization test to see if there is a linear relationship between a regressor variable, x, and the dependent variable, y. Notice that I said a "simple regression" model. That means that there's just the one regressor (apart from an intercept). Multiple regression models raise all sorts of issues for permutation tests, and we'll get to that in due course.

There are several things that we're going to see here:
  1. How to construct a randomization test of the hypothesis that the regression slope coefficient is zero.
  2. A demonstration that the permutation test is "exact". That it, its significance level is exactly what we assign it to be.
  3. A comparison between a permutation test and the usual t-test for this problem.
  4. A demonstration that the permutation test remains "exact", even when the regression model is mi-specified by fitting it through the origin.
  5. A comparison of the powers of the randomization test and the t-test under this model mis-specification.