A Permutation Test Regression Example

In a post last week I talked a bit about Permutation (Randomization) tests, and how they differ from the (classical parametric) testing procedure that we generally use in econometrics. I'm going to assume that you've read that post.

(There may be a snap quiz at some point!)

I promised that I'd provide a regression-based example. After all, the two examples that I went through in that previous post were designed to expose the fundamentals of permutation/randomization testing. They really didn't have much "econometric content".

In what follows I'll use the terms "permutation test" and "randomization test" interchangeably.

What we'll do here is to take a look at a simple regression model and see how we could use a randomization test to see if there is a linear relationship between a regressor variable, x, and the dependent variable, y. Notice that I said a "simple regression" model. That means that there's just the one regressor (apart from an intercept). Multiple regression models raise all sorts of issues for permutation tests, and we'll get to that in due course.

There are several things that we're going to see here:

1. How to construct a randomization test of the hypothesis that the regression slope coefficient is zero.
2. A demonstration that the permutation test is "exact". That it, its significance level is exactly what we assign it to be.
3. A comparison between a permutation test and the usual t-test for this problem.
4. A demonstration that the permutation test remains "exact", even when the regression model is mi-specified by fitting it through the origin.
5. A comparison of the powers of the randomization test and the t-test under this model mis-specification.

What is a Permutation Test?

Permutation tests, which I'll be discussing in this post, aren't that widely used by econometricians. However, they shouldn't be overlooked.

Let's begin with some background discussion to set the scene. This might seem a bit redundant, but it will help us to see how permutation tests differ from the sort of tests that we usually use in econometrics.

Background Motivation

When you took your first course in economic statistics, or econometrics, no doubt you encountered some of the basic concepts associated with testing hypotheses. I'm sure that the first exposure that you had to this was actually in terms of "classical", Neyman-Pearson, testing. 

It probably wasn't described to you in so many words. It would have just been "statistical hypothesis testing". The whole procedure would have been presented, more or less, along the following lines:

Econometrics Videos

The Royal Economics Society (publisher of The Econometrics Journal) has recently released a video of invited addresses by Alfred Galichon and Jeremy Lise, in the special session on “Econometrics of Matching” at the 2015 RES Conference.

This video joins similar ones from previous RES conferences, these being:

• “Large Dimensional Models”, 
• ”Heterogeneity”,  
• “Econometrics of Forecasting”, 
•  “Nonparametric Identification” 

This link will take you to all of these videos.

Happy viewing!

© 2015, David E. Giles

Conference in Honour of Aman Ullah

Last weekend, a small conference was held to honour Aman Ullah, a Distinguished Professor in the Department of Economics at the University of California, Riverside. I was to have participated in this gathering, but regrettably those plans had to be curtailed.

You'll find the program for the conference here. Aman (wearing a jacket) is front and centre in the picture below:

Aman and I go back a long way, and I remember fondly a period of leave that I spent with him at Western University; and his extended visits to both Monash University and the University of Canterbury. Along the way we managed to co-edit a couple of books together, and to say that I've learned a lot from him would be a huge understatement.

The description "a gentleman and a scholar" sits as well with Aman as with anyone else I can think of.

Thank you, Aman, for your enormous contributions to our discipline, your good humour, and your friendship.

© 2015, David E. Giles

Applied Nonparametric Econometrics

Recently, I received a copy of a new econometrics book, Applied Nonparametric Econometrics, by Daniel Henderson and Christopher Parmeter.

The title is pretty self-explanatory and, as you'd expect with any book published by CUP, this is a high-quality item.

The book's Introduction begins as follows:

"The goal of this book is to help bridge the gap between applied economists and theoretical econometricians/statisticians. The majority of empirical research in economics ignores the potential benefits of nonparametric methods and many theoretical nonparametric advances ignore the problems faced by practitioners. We do not believe that applied economists dismiss these methods because they do not like them.  We believe that they do not employ them because they do not understand how to use them or lack formal training on kernel smoothing."

The authors provide a very readable, but careful, treatment of the main topics in nonparamteric econometrics, and a feature of this book is the set of empirical examples. The book's website provides the data that are used (for replication purposes), as well as a number of routines in R. The latter provide useful additions to those that are available in the np package for R (Hayfield and Racine, 2008).

Reference

Hayfield T. and J. S. Racine, 2008. Nonparametric econometrics: The np package. Journal of Statistical Software, 27 (5), 1-32.

© 2015, David E. Giles