Difference-in-Differences With Missing Data

This brief post is a "shout out" for  Irene Botusaru (Economics, Simon Fraser University) who gave a great seminar in our department yesterday.

The paper that she presented (co-authored with Federico Guitierrez), is titled "Difference-in- Differences When the Treatment Status is Observed in Only One Period". So, the title of this post is a bit of an abbreviation of what the paper is really about.

When we conduct DID analysis, we need to be able to classify information about the behaviour/characteristics of survey respondents into a 4-way matrix. Specifically we need to be able to observe the respondents before and after a "treatment"; and in each case we need to know which respondents were treated, and which ones were not.

Usually, a true panel of data, observed at two or more time-periods, facilitates this.

However, what if we simply have repeated cross-sections of data, taken at different time-periods? In this case we aren't necessarily observing exactly the same respondents when we look at the cross-sections for two different time-periods. Typically, in the cross-section after the treatment we'll know which respondents were treated and which ones weren't. However, there will be no way of partitioning the respondents in the pre-treatment cross-section  into "subsequently treated" and "not treated" groups.

Two of the four cells in the matrix of information that we need will be missing, so conventional DID can't be performed.

This is the problem that Irene and Federico consider.

A natural response is introduce some sort of proxy variable(s) to deal with the missing data, and of course this will introduce an estimation bias, even asymptotically. This paper basically takes this approach. The result is a GMM estimation strategy, together with a test that the underlying assumptions are satisfied.

This is a really nice paper - well motivated, technically solid, and with a nice empirical example and application. I urge you to take a look at it if DID is in your econometrics tool-kit (and even if it's not!)

I'm sure that Irene and Federico would appreciate hearing about situations where you've encountered this missing data problem, and how you've responded to it.

© 2016, David E. Giles

Linear Regression and Treatment Effect Heterogeneity

I received an email from Tymon Słoczyński (Warsaw School of Economics), about a recent paper of his, titled, "New Evidence on Linear Regression and Treatment Effect Heterogeneity". Tymon wrote:

"I have recently written a new paper, which I believe that you might find interesting, given some of your blog posts that I have read. 

This paper is available here (as an IZA DP No. 9491): http://ftp.iza.org/dp9491.pdf; or from my website: http://akson.sgh.waw.pl/~tslocz/Sloczynski_paper_regression.pdf. 

This paper implicitly criticizes the standard approach in reduced-form applied microeconomics to use very simple linear models and estimate them using OLS (or 2SLS). I provide a new interpretation of the least squares estimand in the constant-effects linear regression model when the assumption of constant effects is violated (so there is, in fact, "treatment effect heterogeneity"). This new interpretation is very pessimistic: in particular, I prove that the weight that is being placed by OLS on the effect on each group ("treated" or "controls") is inversely related to the proportion of this group. This property might have severe consequences for applied work, and I demonstrate this via a replication of two recent papers from the American Economic Review."

Tymon's paper is, indeed, very interesting. I recommend that you read it. It should serve as a 'wake-up call' to some of our empirical micro. friends!

© 2015, David E. Giles

Econometrics vs. Ad Hoc Empiricism

In a post in 2013, titled "Let's Put the "ECON" Back Into Microeconometrics", I complained about some of the nonsense that is passed off as "applied econometrics". Specifically, I was upset about the disconnect between the economic model (if there is one) and the empirical relationships that are actually estimated, in many "applied" papers.

I urge you to look back at the post before reading further.

Here's a passage from that post:

"In particular, how often have you been presented with an empirical application that's based on just a reduced-form model that essentially ignores the nuances of the theoretical model?

I'm not picking on applied microeconomic papers - really, I'm not! The same thing happens with some applied macroeconomics papers too. It's just that in the micro. case, there's often a much more detailed and rich theoretical model that just lends itself to some nice structural modelling. And then all we see is a regression of the logarithm of some variable on a couple of interesting covariates, and a bunch of controls - the details of which are frequently not even reported."

Well, things certainly haven't improved since I wrote that. In fact, it seems that I'm encountering more and more of this nonsense. This isn't "econometrics", and the purveyors of this rubbish aren't "econometricians". 

My real concern is that students who are exposed to these papers and seminars may not recognize it for what it is - just ad hoc empiricism. 

© 2015, David E. Giles

"Mastering 'Metrics"

Mastering 'Metrics: The Path from Cause to Effect, by Joshua Angrist and Jörn-Steffen Pischke, is to be published by Princeton University Press later this month. This new book from the authors of Mostly Harmless Econometrics: An Empiricist's Companion is bound to be well received by students and researchers involved in applied empirical economics. My guess is that the biggest accolades will come from those whose interest is in empirical microeconomics.

You can download and preview the Introduction and Chapter 1.

Apparently the book focuses on:

"The five most valuable econometric methods, or what the authors call the Furious Five - random assignment, regression, instrumental variables, regression discontinuity designs, and differences in differences."

If this sounds interesting to you, then make sure that you take a look at Peter Dizikes' recent post, "How to Conduct Social Science Research", on the World Economic Forum website.

© 2014, David E. Giles

Congratulations, Sir Richard!

Well-known microeconometrician, Richard Blundell, has been knighted in the New Year's Honours list. Sir Richard is the Ricardo Professor of Political Economy at University College London, and Research Director for the Institute for Fiscal Studies.

Knighted for his services to economics and social science, Sir Richard has previously served as co-editor of both Econometrica and Journal of Econometrics, and his many other achievement and awards are listed here.

I'm sure that all econometricians will be delighted by this recognition of Sir Richard's contributions, and offer him their sincere congratulations!

© 2014, David E. Giles

Statistical Significance - Again

With all of this emphasis on "Big Data", I was pleased to see this post on the Big Data Econometrics blog, today.

When you have a sample that runs to the thousands (billions?), the conventional significance levels of 10%, 5%, 1% are completely inappropriate. You need to be thinking in terms of tiny significance levels.

I discussed this in some detail back in April of 2011, in a post titled, "Drawing Inferences From Very Large Data-Sets". If you're of those (many) applied researchers who uses large cross-sections of data, and then sprinkles the results tables with asterisks to signal "significance" at the 5%, 10% levels, etc., then I urge you read that earlier post.

It's sad to encounter so many papers and seminar presentations in which the results, in reality, are totally insignificant!

© 2013, David E. Giles

Recent, and Recommended.......

Recently, I griped posted about the need to get the economics back into papers that the authors characterize as "microeconometrics". Although I was venting (just a little!) about the "disconnect" that we so often see, between the theory section and the empirical section, in so many of the papers in this category, I also commented that there are plenty of papers out there that avoid this disconnect. I just wish there were more of them!

In response to one of the comments of that post, I gave just one such example, and afterwards I thought that although my choice was a good one, it was somewhat dated. So, on a more positive note, what about some recent papers that exemplify what I'm looking for, and what I'd like to see more of?

Let's Put the "ECON" Back Into Microeconometrics

You just couldn't resist the title, could you?

Don't worry, I'm not going to be too harsh. After all, I'm rather fond of those who practise "applied microeconometrics" - especially lightly sautéed, with a little pepper and garlic. Sorry! Sorry!

The point that I want to make is a simple one, and I'll be brief.

How many seminars have you attended where the speaker has gone through the details of a formal microeconomic model, and then proceeded to a potentially interesting empirical application? And in how many cases was there a total "disconnect" between the theoretical model and the empirical model?

Hand up! Don't be shy! Wow - that's almost everyone!