Simulation methods, including Monte Carlo simulation and various forms of the bootstrap, are widely used by econometricians. We use these tools to learn about the sampling distributions of our estimators and tests, especially in situations where a purely analytic approach is technically difficult.
For example, sometimes we're able to appeal to standard asymptotic (large sample) results - such as the central limit theorems, and the laws of large numbers - to figure out how good our inferences will be if the sample size is very large. However, when it comes to the question of how good they are when the sample size is quite small, the answer may not be so easily established.
In addition, when we come up with a new theoretical result in econometrics, most of us take the precaution of also simulating the result - as check on its accuracy.
Monte Carlo and bootstrap methods rely critically on our ability to generate "pseudo"-random numbers that have the characteristics that we ascribe to them. How often have you actually checked if the random number generators in your favourite econometrics package produce values that are "random", and follow the distribution that you've asked for? Probably not often enough!
I follow John Cook's blog, The Endeavour. A couple of years ago he had a nice post titled, "How to test a random number generator". In that post, he links to a chapter of the same title that he wrote for the book, Beautiful Testing (edited by Tim Riley and Adam Goucher).
John's chapter is a short, but very valuable read, and I recommend it strongly.
John's chapter is a short, but very valuable read, and I recommend it strongly.
© 2012, David E. Giles
A few years ago, a student of mine was using a random number generator to simulate the value of CDOs. I have a general distrust of random numbers, going back to the 64000 cycle of Excel's random number generator (which, I believe, was only fixed about ten years ago. I still am leery of trusting Excel for multi-dimensional problems.)
ReplyDeleteAfter much haranguing on my part, the student finally called Mathematica and found that there was a better random number generator available. You just had to ask for it. (And know to ask for it.)
Don't trust random generators! Test them! Thanks for the post.
here is a fun review of the accuracy of statistical software in economics from the JEL in 1999. It is a little dated, but many of the things to watch out for probably still exist today.
ReplyDeletehttp://www.ucl.ac.uk/~uctpbss/ice2011/reading/McCulloughVinodJEL1999.pdf
Thanks! The lessons still apply.
ReplyDeleteI used random generators and also faced problems. So, don't trust them as Dave Giles said.
ReplyDeleteSample forms
Not until you've tested them!
DeleteThis is really wonderful blog. All the articles are worth reading. Hope you would also like to read Properties of good random number generators
ReplyDelete