tag:blogger.com,1999:blog-21989425347406423842024-01-30T23:50:13.985-08:00Econometrics Beat: Dave Giles' Blog<i>A resource for econometrics students & practitioners</i>Dave Gileshttp://www.blogger.com/profile/05389606956062019445noreply@blogger.comBlogger949125tag:blogger.com,1999:blog-2198942534740642384.post-67227202400408959222019-10-31T08:42:00.003-07:002019-10-31T08:42:54.329-07:00It's Time to Go<div dir="ltr" style="text-align: left;" trbidi="on">
<div style="text-align: justify;">
When I released my first post on the blog on 20th. Febuary 2011 I really wasn't sure what to expect! After all, I was aiming to reach a somewhat niche audience.</div>
<div style="text-align: justify;">
<br /></div>
<div style="text-align: justify;">
Well, 949 posts and 7.4 million page-hits later, this blog has greatly exceeded my wildest expectations. </div>
<div style="text-align: justify;">
<br /></div>
<div style="text-align: justify;">
However, I'm now retired and I turned 70 three months ago. I've decided to call it quits, and this is my final post.</div>
<div style="text-align: justify;">
<br /></div>
<div style="text-align: justify;">
I'd rather make a definite decision about this than have the blog just fizzle into nothingness.</div>
<div style="text-align: justify;">
<br /></div>
<div style="text-align: justify;">
For now, the <i>Econometrics Beat</i> blog will remain visible, but it will be closed for further comments and questions.</div>
<div style="text-align: justify;">
<br /></div>
<div style="text-align: justify;">
I've had a lot fun and learned a great deal through this blog. I owe a debt of gratitude to all of you who've followed my posts, made suggestions, asked questions, made helpful comments, and drawn errors to my attention.</div>
<div style="text-align: justify;">
<br /></div>
<div style="text-align: justify;">
I just hope that it's been as positive an experience for you as it has been for me.</div>
<div style="text-align: justify;">
<br /></div>
<div style="text-align: justify;">
Thank you - and enjoy your Econometrics!</div>
<div style="text-align: justify;">
<br /></div>
<center>
© 2019, David E. Giles
</center>
</div>
Dave Gileshttp://www.blogger.com/profile/05389606956062019445noreply@blogger.com68tag:blogger.com,1999:blog-2198942534740642384.post-1390675004045816982019-10-30T20:48:00.003-07:002019-10-31T08:26:45.385-07:00Everything's Significant When You Have Lots of Data<div dir="ltr" style="text-align: left;" trbidi="on">
<div style="text-align: justify;">
Well........, not really!<br />
<br />
It might seem that way on the face of it, but that's because you're probably using a totally inappropriate measure of what's (statistically) significant, and what's not.<br />
<br />
I talked a bit about this issue in a <span style="background-color: white;"><a href="https://davegiles.blogspot.com/2011/04/drawing-inferences-from-very-large-data.html#more">previous pos</a>t</span>, where I said:<br />
<blockquote class="tr_bq">
<span style="font-family: inherit;"><span style="background-color: white; color: #444444;">"Granger (</span><a href="http://onlinelibrary.wiley.com/doi/10.1111/stan.1998.52.issue-3/issuetoc" style="background-color: white; color: #4d469c; text-decoration-line: none;">1998</a>,<span style="background-color: white; color: #444444;"> </span><a href="http://www.amazon.com/Computer-Aided-Econometrics-Statistics-Textbooks-Monogrphs/dp/0824742710?ie=UTF8&tag=econdaveblog-20&link_code=btl&camp=213689&creative=392969" style="background-color: white; color: #4d469c; text-decoration-line: none;" target="_blank">2003</a><strong style="background-color: white; color: #444444;"><img alt="" border="0" height="1" src="https://blogger.googleusercontent.com/img/proxy/AVvXsEhw_xPobQ2cF1MaT71WmOm4Elvda49LffdDKKREluIiIhEDce04DaqPuInQ7bTbY6hSCkEbBvhjWpOMBoBRVba6ki7GJBMOky_RNdPRjpYbwPLd285etwwzIr_OQfZxikP99uniOMk01pPEA2bOR_cCTV0FgB5LW-a0UBt9J4OA6X-BqEPSaM61g5eVcJkEzA4LD2NglmeFxim1A2drO0OPteKtMiE=s0-d" style="background: transparent; border-radius: 0px; border: currentcolor; box-shadow: rgba(0, 0, 0, 0.2) 0px 0px 0px; margin: 0px; padding: 0px !important;" width="1" /></strong><span style="background-color: white; color: #444444;">) has reminded us that if the sample size is sufficiently large, then it's virtually impossible </span><em style="background-color: white; color: #444444;">not</em><span style="background-color: white; color: #444444;"> to reject almost any hypothesis. So, if the sample is very large and the </span><em style="background-color: white; color: #444444;">p</em><span style="background-color: white; color: #444444;">-values associated with the estimated coefficients in a regression model are of the order of, say, 0.10 or even 0.05, then this really </span><strong style="background-color: white; color: #444444;"><u>bad</u></strong><span style="background-color: white; color: #444444;"> news. Much, much, smaller </span><em style="background-color: white; color: #444444;">p</em><span style="background-color: white; color: #444444;">-values are needed before we get all excited about 'statistically significant' results when the sample size is in the thousands, or even bigger."</span></span></blockquote>
This general point, namely that our chosen significance level should be <i>decreased </i>as the sample size grows, is pretty well understood by most statisticians and econometricians. (For example, see Good, 1982.) However, it's usually ignored by the authors of empirical economics studies based on samples of thousands (or more) observations. Moreover, a lot of practitioners seem to be unsure of just <i>how much they should revise</i> their significance levels (or re-interpret their p-values) in such circumstances.<br />
<br />
There's really no excuse for this, because there are some well-established guidelines to help us. In fact, as we'll see, some of them have been around since at least the 1970's.<br />
<br />
Let's take a quick look at this, because it's something that all students need to be made aware of as we work more and more with "big data". Students certainly won't gain this awareness by looking at the interpretation of the results in the vast majority of empirical economics papers that use even sort-of-large samples!<br />
<br />
<a name='more'></a>The main result that I want to highlight is one that was brought into the econometrics literature by Leamer (1978). (Take a look at Chapter 4 of his book, referenced below - and especially p.116.)<br />
<br />
Let's set the scene by quoting from <a href="https://books.google.ca/books?id=gNyGDwAAQBAJ&pg=PT163&lpg=PT163&dq=angus+deaton+large+samples+critical+values&source=bl&ots=KtmzRga8Ez&sig=ACfU3U1GgFftj2qfQPRMOme-QPO0xy2Oog&hl=en&sa=X&ved=2ahUKEwiQkrrtrqPjAhUGmuAKHRo5BxgQ6AEwB3oECAkQAQ#v=onepage&q=angus%20deaton%20large%20samples%20critical%20values&f=false">Deaton</a> (2018, Chap. 2):<br />
<blockquote class="tr_bq">
"The effect most noted by empirical researchers is that the null hypothesis seems to be more frequently rejected in large samples than in small. Since it is hard to believe that the truth depends on the sample size, something else must be going on.......... As the sample size increases, and provided we are using a consistent estimation procedure, our estimates will be closer and closer to the truth, and less dispersed around it, so that discrepancies that were undetectable with small samples will lead to rejections in large samples........... </blockquote>
<blockquote class="tr_bq">
Over-rejection in large samples can also be thought of in terms of Type I and Type II errors. When we hold Type I error fixed and increase the sample size, all the benefits of increased precision are implicitly devoted to the reduction of Type II error.........</blockquote>
<blockquote class="tr_bq">
</blockquote>
<blockquote class="tr_bq">
Repairing these difficulties requires that the critical values of test statistics be raised with the sample size, so that the benefits of increased precision are more equally allocated between reduction Type I and Type II errors. That said, it is a good deal more difficult to decide exactly how to do so, and to derive the rule from basic principles. Since classical procedures cannot provide such a basis, Bayesian alternatives are the obvious place to look."</blockquote>
And that's precisely what Leamer does. Also, see Schwartz (1978).<br />
<br />
Suppose that we have a linear multiple regression model with k regressors and n observations, and we want to test the null hypothesis that a set of q independent linear restrictions on the regression coefficients are satisfied. The alternative hypothesis is that at least one of the restrictions is violated. Under the very restrictive assumptions that we usually begin with in this context, an F-test would be used, and the associated statistics would be F-distributed with q and (n - k) degrees of freedom if the null is true.<br />
<br />
We would reject the null hypothesis if F > F<sub>c</sub>(α), where α is the chosen significance level, and F<sub>c</sub>(α) is the associated critical value. Alternatively, we would calculate the p-value associated with the observed value of F, and reject if this p-value is "small enough".<br />
<br />
We're interested in situations where n is large - probably <i>very large</i>. So, we can ignore the distinction between n and (n - k). Asymptotically, qF converges to a Chi-Square statistic with q degrees of freedom if the null is true. This is equivalent to the Wald test. Moreover, this convergence still holds if the restrictions under test are nonlinear, rather than linear. We would reject the null if qF > χ<sup>2</sup><sub>c</sub>(α), where again the "c" subscript denotes the appropriate critical value.<br />
<br />
It's the choice of α that has to be questioned here. Should we still set α = 10%, 5%, 1% if n is <i>very, very large</i>? (<b>No, we shouldn't!</b>)<br />
<br />
Equivalently, if n is very big, what is the appropriate magnitude of the p-value below which we should decide to reject the null hypothesis? Or, equivalently again, how should the critical value for this test be modified in very large samples?<br />
<br />
Leamer's result tells us that we should reject the null if F > (n / q)(n<sup>q/n</sup> - 1) ; or equivalently, if qF = χ<sup>2</sup> > n(n<sup>q/n</sup> - 1)<br />
<br />
It's important to note that this result is based on a Bayesian analysis with a particular approach to the diffuseness of the prior distribution.<br />
<br />
Also, recall that if we have a t-statistic with v degrees of freedom, then t<sup>2</sup> is F-distributed with 1 and v degrees of freedom. So, if we are testing the significance of a single regressor (<i>i.e.</i>, we are testing just one restriction), then Leamer's result tells us that we should reject the null that this coefficient is zero if t<sup>2</sup> > n(n<sup>1/n</sup> - 1). That is, we should reject against a 2-sided alternative hypothesis if |t| > √[n(n<sup>1/n</sup> - 1)] (Remember, q = 1 in this case.)<br />
<br />
It's actually easy to check that n(n<sup>1/n</sup> - 1) is approximately equal to log<sub>e</sub>(n) for large values of n. Indeed, if n is very, very large, this approximation is still excellent even if q > 1 (as long as q is finite). Consider the following numerical examples:<br />
<br />
<div style="text-align: center;">
<b>Table 1</b></div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhb-WZypQFO6TzMwh9hYk5CFXUN7VcPgv3DUHZKRSib_IgmXG5RGtq6fdrI1BpsL4XAQckbZ05w8UbFOhhgvkrfZciVBOi8zovN9B00PMv7aTIpYPZu6Qrm6_mmzJyX4aDknVRJVsDitMG3/s1600/Capture.JPG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="142" data-original-width="258" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhb-WZypQFO6TzMwh9hYk5CFXUN7VcPgv3DUHZKRSib_IgmXG5RGtq6fdrI1BpsL4XAQckbZ05w8UbFOhhgvkrfZciVBOi8zovN9B00PMv7aTIpYPZu6Qrm6_mmzJyX4aDknVRJVsDitMG3/s1600/Capture.JPG" /></a></div>
<br />
So, what this means is that for very big samples Leamer's rule amounts to the following:<br />
<br />
<b>Reject </b><b>H<sub>0</sub>: "q independent restrictions on the model's parameters are true", if F > log<sub>e</sub>(n) ; or equivalently, if χ<sup>2</sup> > qlog<sub>e</sub>(n).</b><br />
<br />
How does this differ from what we'd do traditionally? (Remember, for large n we can ignore the distinction between n and (n - k).) Here are the corresponding critical F-values:<br />
<br />
<div style="text-align: center;">
<b>Table 2</b></div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgk7OhSkwshZgDdWr9044WXd47Pfkzn6DgkkDgmQdFMZjpCLai2oeCI7r5DgYq3kROAX09VQ6lnU0-lGqHKz76E-Dypkzjn4SwhGpzsxznDrTeoQVkLkKvO4Xz1QFli0I5jAcNVbYen94Nn/s1600/Capture.JPG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="142" data-original-width="322" height="141" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgk7OhSkwshZgDdWr9044WXd47Pfkzn6DgkkDgmQdFMZjpCLai2oeCI7r5DgYq3kROAX09VQ6lnU0-lGqHKz76E-Dypkzjn4SwhGpzsxznDrTeoQVkLkKvO4Xz1QFli0I5jAcNVbYen94Nn/s320/Capture.JPG" width="320" /></a></div>
<br />
We see that if n = 100,000 and q = 5, then using the F-test with conventional significance levels we'd reject the validity of the restrictions at the 10%, 5% and 1% significance levels if the F-statistic exceeded 1.8, 2.2, or 3.0 respectively. From Table 1, we see that the critical value in this case actually should be 11.5! You can quickly check for yourself that if we're applying a 2-sided t-test (q = 1), with n = 100,000, than we should reject the null hypothesis if |t| > √(11.5162) = 3.394.<br />
<br />
So, using conventional measures, we'll reject the validity of the restrictions far more often than we should do. Youch!<br />
<br />
To look at things from a different perspective we can ask, "what sort of significance levels are being implied by Leamer's suggestion, relative to the levels (10%, 5%, <i>etc</i>.) that we typically use in practice?"<br />
<br />
Let's go back to Table 1, and focus on the last column of log<sub>e</sub>(n) critical values. The associated significance levels are as follows:<br />
<br />
<div style="text-align: center;">
<b>Table 3</b></div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEj3K_7bcpmdDFdWKvcUYU8zOCOfefvw0apiq7y1JwvQensMmmmiHzxDqe9tp8KG8vsqclP4LqbwwMv9Gkga3-K_QBiUomf90PlgB99XMJgDlrQr4WRk4pZYKvS6zllqDsAwvqucuKSILXW5/s1600/Capture.JPG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="141" data-original-width="275" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEj3K_7bcpmdDFdWKvcUYU8zOCOfefvw0apiq7y1JwvQensMmmmiHzxDqe9tp8KG8vsqclP4LqbwwMv9Gkga3-K_QBiUomf90PlgB99XMJgDlrQr4WRk4pZYKvS6zllqDsAwvqucuKSILXW5/s1600/Capture.JPG" /></a></div>
<div style="text-align: justify;">
<b><br /></b>
And in the case of the t-test example given beneath Table 2, the significance level associated with a critical value of 3.394 is 0.000345.<br />
<br /></div>
As we can see, when n is very large, the significance levels that <i><b>we should be using</b></i> (or, equivalently, the p-values that we should be using) are <u>much less</u> than the conventional levels that we tend to think of!<br />
<br />
<i>As an exercise, why don't you take a look back at one of your favourite applied econometrics papers that uses a very large sample size, and ask yourself, "do I really believe the conclusions that the author has reached?"</i><br />
<br />
If you want to read more on this topic, I suggest that you take a look at Lin <i>et al</i>. (2013), and Lakens (2018).<br />
<span style="font-family: inherit;"><br /></span>
</div>
<div style="text-align: justify;">
<b><span style="font-family: inherit;">References</span></b></div>
<div style="text-align: justify;">
<b><span style="font-family: inherit;"><br /></span></b></div>
<a href="https://www.amazon.ca/s?k=9781464813528&i=stripbooks&linkCode=qs">Deaton, A. S.</a><span style="font-family: inherit; text-align: justify;">, 2018. </span><i style="font-family: inherit; text-align: justify;">The Analysis of Household Surveys: A Microeconometric Approach to Development Policy</i><span style="font-family: inherit; text-align: justify;">. (Reissue edition with a new preface.) World Bank, Washington, D.C..</span><br />
<div style="text-align: justify;">
<br /></div>
<div style="text-align: justify;">
<a href="https://www.dropbox.com/s/uctrkoswmyxw5vu/Good82.png?dl=0">Good, I. J.</a>, 1982. C140. Standardized tail-area probabilities. <i>Journal of Statistical Computation and Simulation</i>, 16, 65–66.<br />
<br />
<a href="http://onlinelibrary.wiley.com/doi/10.1111/stan.1998.52.issue-3/issuetoc" style="background-color: white; color: #3778cd; font-family: inherit;">Granger</a><span style="background-color: white; color: #444444; font-family: inherit;">, C. W. J. (1998). Extracting information from mega-panels and high frequency data. </span><em style="background-color: white; color: #444444; font-family: inherit;">Statistica Neerlandica</em><span style="background-color: white; color: #444444; font-family: inherit;">, 52, 258-272.</span></div>
<span style="font-family: inherit;"><br style="background-color: white; color: #444444; text-align: justify;" /><span style="background-color: white; color: #444444; text-align: justify;">Granger, C. W. J. (2003). Some methodological questions arising from large data sets. In D. E. A. Giles (ed.),</span><strong style="background-color: white; color: #444444; text-align: justify;"><em> </em></strong><a href="http://www.amazon.com/Computer-Aided-Econometrics-Statistics-Textbooks-Monogrphs/dp/0824742710?ie=UTF8&tag=econdaveblog-20&link_code=btl&camp=213689&creative=392969" style="background-color: white; color: #4d469c; text-align: justify; text-decoration-line: none;" target="_blank"><strong><em>Computer-Aided Econometrics</em></strong></a><strong style="background-color: white; color: #444444; text-align: justify;"><em><img alt="" border="0" height="1" src="https://blogger.googleusercontent.com/img/proxy/AVvXsEhw_xPobQ2cF1MaT71WmOm4Elvda49LffdDKKREluIiIhEDce04DaqPuInQ7bTbY6hSCkEbBvhjWpOMBoBRVba6ki7GJBMOky_RNdPRjpYbwPLd285etwwzIr_OQfZxikP99uniOMk01pPEA2bOR_cCTV0FgB5LW-a0UBt9J4OA6X-BqEPSaM61g5eVcJkEzA4LD2NglmeFxim1A2drO0OPteKtMiE=s0-d" style="background: transparent; border-radius: 0px; border: currentcolor; box-shadow: rgba(0, 0, 0, 0.2) 0px 0px 0px; margin: 0px; padding: 0px !important;" width="1" /></em></strong><span style="background-color: white; color: #444444; text-align: justify;">, CRC Press, Boca Raton, FL, 1-8.</span></span><br />
<div style="text-align: justify;">
<span style="font-family: inherit;"><br /></span>
<br />
<a href="http://daniellakens.blogspot.com/2018/12/testing-whether-observed-data-should.html">Lakens, D.</a>, 2018. Justify your alpha by decreasing alpha levels as a function of the sample size. <i>The 20% Statistician Blog</i>.<br />
<br />
<span style="font-family: inherit;"></span>
<span style="font-family: inherit;"><a href="https://www.amazon.ca/Specification-Searches-Inference-Nonexperimental-Data/dp/0471015202">Leamer, E. E.</a>, 1978. <i>Specification Searches: Ad Hoc Inference With Nonexperimental Data</i>. Wiley, New York. (<a href="https://www.anderson.ucla.edu/faculty_pages/edward.leamer/downloadable_books/ucla_anderson_faculty_edward_leamer_downloadble_books.html">Legitimate free download</a>.)</span><br />
<span style="font-family: inherit;"><br /></span>
<span style="font-family: inherit;"><a href="https://pdfs.semanticscholar.org/262b/854628d8e2b073816935d82b5095e1703977.pdf">Lin, M., H. C. Lucas Jr., and G. Schmueli</a>, 2013. Too big to fail: Large samples and the p-value problem. <i>Information Systems Research</i>, 24, 906-917.</span><br />
<span style="font-family: inherit;"><br /></span>
<span style="font-family: inherit;"><a href="https://projecteuclid.org/euclid.aos/1176344136">Schwartz, G.</a>, 1978. estimating the dimension of a model. <i>Annals of Statistics</i>, 6, 461-464.</span><br />
<span style="font-family: inherit;"> </span></div>
<center style="text-align: center;">
© 2019, David E. Giles
</center>
</div>
Dave Gileshttp://www.blogger.com/profile/05389606956062019445noreply@blogger.com2tag:blogger.com,1999:blog-2198942534740642384.post-80669989281344132112019-10-27T08:19:00.000-07:002019-10-27T08:19:25.177-07:00Reporting an R-Squared Measure for Count Data Models<div dir="ltr" style="text-align: left;" trbidi="on">
<div dir="ltr" style="text-align: left;" trbidi="on">
<div style="text-align: justify;">
<i>This post was prompted by an email query that I received some time ago from a reader of this blog. I thought that a more "expansive" response might be of interest to other readers............</i><br />
<br />
In spite of its many limitations, it's standard practice to include the value of the coefficient of determination (R<sup>2</sup>) - or its "adjusted" counterpart - when reporting the results of a least squares regression. Personally, I think that R<sup>2</sup> is one of the least important statistics to include in our results, but we all do it. (See <a href="https://davegiles.blogspot.com/2013/05/good-old-r-squared.html">this previous post</a>.)</div>
<div style="text-align: justify;">
<br /></div>
<div style="text-align: justify;">
If the regression model in question is linear (in the parameters) and includes an intercept, and if the parameters are estimated by Ordinary Least Squares (OLS), then R<sup>2</sup> has a number of well-known properties. These include:</div>
<div style="text-align: justify;">
</div>
<ol>
<li>0 ≤ R<sup>2</sup> ≤ 1.</li>
<li>The value of R<sup>2</sup> cannot decrease if we add regressors to the model.</li>
<li>The value of R<sup>2</sup> is the same, whether we define this measure as the ratio of the "explained sum of squares" to the "total sum of squares" (R<sub>E</sub><sup>2</sup>); or as one minus the ratio of the "residual sum of squares" to the "total sum of squares" (R<sub>R</sub><sup>2</sup>).</li>
<li>There is a correspondence between R<sup>2</sup> and a significance test on all slope parameters; and there is a correspondence between changes in (the adjusted) R<sup>2</sup> as regressors are added, and significance tests on the added regressors' coefficients. (See <a href="https://davegiles.blogspot.com/2013/05/when-will-adjusted-r-squared-increase.html">here</a> and <a href="https://davegiles.blogspot.com/2013/07/the-adjusted-r-squared-again.html">here</a>.)</li>
<li>R<sup>2</sup> has an interpretation in terms of information content of the data. </li>
<li>R<sup>2</sup> is the square of the (Pearson) correlation (R<sub>C</sub><sup>2</sup>) between actual and "fitted" values of the model's dependent variable. </li>
</ol>
<div>
<div style="text-align: justify;">
However, as soon as we're dealing with a model that excludes an intercept or is non-linear in the parameters, or we use an estimator other than OLS, <i><b>none of the above properties are guaranteed</b></i>.<br />
<a name='more'></a></div>
<div style="text-align: justify;">
<br /></div>
<div style="text-align: justify;">
For example, when reporting a linear model that's been estimated by Instrumental Variables, we get <i>different</i> R<sup>2</sup> values depending on which of the two definitions noted in property 3 above is adopted. Similarly, when estimating Logit and Probit models (for instance), most econometrics packages report several "pseudo-R<sup>2</sup>" statistics, because there's no single measure that has <i>all </i>of the desirable features that we're used to in the linear model/OLS case.<br />
<br />
So-called "count" data arise frequently in empirical economics. These are data that take values that are only non-negative integers, namely 0, 1, 2, 3, 4, ........ Models for such data are often based on the Poisson or negative binomial distributions, although other distributions may also be used. Regressors enter the model by equating the mean of the chosen distribution to a positive function of these variables and their coefficients.<br />
<br />
For instance, if the y<sub>i</sub> data (i = 1, 2, ...., n) are being modelled using a Poisson distribution with a mean of μ, then we typically assign μ<sub>i</sub> = exp[x<sub>i</sub>'β], using familiar regression notation. The resulting non-linear model is then estimated by MLE (or quasi-MLE).<br />
<br />
What's a sensible way of reporting an R<sup>2</sup> measure for an estimated Poisson regression?</div>
</div>
<div style="text-align: justify;">
<br /></div>
<div style="text-align: justify;">
As with the Logit-Probit case noted above, several possibilities suggest themselves. However, unlike that other case, when modelling "count" data there is actually one definition of R<sup>2</sup> that really stands out as the obvious choice.<br />
<br />
What is it?<br />
<br />
Before answering this question, let's look at how R<sub>R</sub><sup>2</sup>, R<sub>E</sub><sup>2</sup>, and R<sub>C</sub><sup>2</sup> behave when applied in the context of Poisson, or negative binomial, regression. Some key facts include:<br />
<ul>
<li>The three measures will generally differ in value from one another.</li>
<li>We still have 0 ≤ R<sub>C</sub><sup>2</sup> ≤ 1. However, although R<sub>R</sub><sup>2</sup> ≤ 1 it can be negative (even if an intercept is included in the model); and although R<sub>E</sub><sup>2</sup> ≥ 0 it can be greater than one (even with an intercept).</li>
<li>All three measures can <i>decrease </i>as regressors are added to the model. </li>
</ul>
When we compare these results with the six properties noted above for the OLS case, they suggest that these R<sup>2</sup> measures are probably best avoided with count data models. Interestingly, it's R<sub>R</sub><sup>2</sup> that's reported as a matter of course by the EViews package. Stata, on the other hand, reports McFadden's "pseudo-R<sup>2</sup>" for these models, but its properties are no better.<br />
<br />
Cameron and Windmeijer (1996) effectively answer the question that I posed above.<br />
<br />
They consider various R<sup>2</sup>-type measures for count data models. These measures differ primarily on the type of residuals (from the estimated model) that are used in their construction. As in the case of a linear regression, <i>the usual, or "raw", residuals</i> are the differences between the actual y<sub>i</sub> values and their "predicted" mean values. That is, they're of the of the form (y<sub>i</sub> - μ<sub>i</sub>*), where μ<sub>i</sub>* = exp[x<sub>i</sub>'β*], and β* is the MLE of the β vector. These residuals give us R<sub>R</sub><sup>2</sup>, noted above.<br />
<br />
In regression analysis in general, there are actually lots of different forms of residuals that can be constructed, and these can be useful in various situations - especially with generalized linear models (of which the Poisson count models is an example). Some examples include the Pearson (standardized) residuals and the so-called "deviance" residuals. (for more on the notion of "deviance" and goodness-of-fit, see <a href="https://davegiles.blogspot.com/2014/06/the-deviance-information-criterion.html">this post</a>.)<br />
<br />
Cameron and Windmeijer (1996) consider the properties of R<sup>2</sup> measures for Poisson and negative binomial models based on both of these other types of residuals, as well as on the "raw" residuals. (Cameron and Windmeijer (1997) extend these results to a variety of other non-linear models.)<br />
<br />
They make a convincing case for constructing an R<sup>2</sup> measure using the deviance residuals, when working with a Poisson regression model or the negative binomial (NegBin2) model.<br />
<br />
(<i>As an aside, when the model is linear and we use OLS, the deviance residuals are just the usual residuals</i>.)<br />
<br />
For the Poisson model, the i<sup>th.</sup> deviance residual is defined as<br />
<br />
d<sub>i</sub> = sign(y<sub>i</sub> - μ<sub>i</sub>*)[2{y<sub>i</sub>log(y<sub>i</sub> / μ<sub>i</sub>*) - (y<sub>i</sub> - μ<sub>i</sub>*)}]<sup>½ </sup>; i = 1, 2, ...., n<br />
<br />
and the deviance R<sup>2</sup> for that model is defined as:<br />
<br />
R<sub>D,P</sub><sup>2</sup> = 1 - Σ{y<sub>i</sub>log(y<sub>i</sub> / μ<sub>i</sub>*) - (y<sub>i</sub> - μ<sub>i</sub>*)} / Σ{y<sub>i</sub>log(y<sub>i</sub> / ybar)},<br />
<br />
where here and below all summations are for i = 1, 2, ...., n.<br />
<br />
If the model includes an intercept, then this formula simplifies to:<br />
<br />
R<sub>D,P</sub><sup>2</sup> = 1 - Σ{y<sub>i</sub>log(y<sub>i</sub> / μ<sub>i</sub>*)} / Σ{y<sub>i</sub>log(y<sub>i</sub> / ybar)}.<br />
<br />
(Note: if y<sub>i</sub> = 0, then y<sub>i</sub>log(y<sub>i</sub>) = 0. In this case, d<sub>i</sub> = - [2μ<sub>i</sub>*]<sup>½</sup>.)<br />
<br />
<b>Importantly, R<sub>D,P</sub><sup>2</sup> satisfies the properties 1 to 5 noted earlier</b>.<br />
<br />
In the case of the NegBin2 model, the corresponding R<sup>2</sup> takes the form:<br />
<br />
R<sub>D,NB</sub><sup>2</sup> = 1 - (A / B) ,<br />
<br />
where<br />
<br />
A = Σ{y<sub>i</sub>log(y<sub>i</sub> / μ<sub>i</sub>*) - (y<sub>i</sub> + α*<sup>-1</sup>)log[(y<sub>i</sub> + α*<sup>-1</sup>) / (μ<sub>i</sub>* + α*<sup>-1</sup>)]}<br />
<br />
and<br />
<br />
B = Σ{y<sub>i</sub>log(y<sub>i</sub> / ybar) - (y<sub>i</sub> + α*<sup>-1</sup>)log[(y<sub>i</sub> + α*<sup>-1</sup>) / (ybar + α*<sup>-1</sup>)]}.<br />
<br />
("ybar" is the sample average of the y<sub>i</sub> values; and α* is the MLE of the dispersion parameter for the NegBin2 distribution.)<br />
<br />
<b>The R<sub>D,NB</sub><sup>2</sup> goodness-of-fit measure satisfies properties 1, 3 and 4 noted earlier.</b><br />
<b><br /></b>
So, when it comes to reporting an R<sup>2</sup> for count data models, the usual such measure - based on the "raw" residuals - is generally a very poor choice. <i>Of the other options that are available, the R<sup>2</sup> measures constructed using the so-called "deviance residuals" stand out as excellent contenders.</i><br />
<br />
<b><br /></b>
</div>
</div>
<div style="text-align: justify;">
<b>References</b></div>
<div style="text-align: justify;">
<b><br /></b></div>
<div style="text-align: justify;">
<a href="https://amstat.tandfonline.com/doi/abs/10.1080/07350015.1996.10524648#.XQ4ypOhKhPY">Cameron, A. C. & F. A. C. Windmeijer</a>, 1996. R-squared measures for count data regression models with applications to health-care utilization. <i>Journal of Business and Economic Statistics</i>, 14, 209-220. (<a href="https://pdfs.semanticscholar.org/4eae/328143f1e37d77413df257044b94dab3a0c6.pdf">Download</a> working paper version.)<br />
<br />
<a href="https://www.sciencedirect.com/science/article/pii/S0304407696018180">Cameron, A. C. & F. A. C. Windmeijer</a>, 1997. An R-squared measure of goodness of fit for some common nonlinear regression models. <i>Journal of Economerics</i>, 77, 329-342.<br />
<br /></div>
<center style="text-align: center;">
© 2019, David E. Giles
</center>
</div>
Dave Gileshttp://www.blogger.com/profile/05389606956062019445noreply@blogger.com2tag:blogger.com,1999:blog-2198942534740642384.post-79343077302539294962019-10-07T13:00:00.004-07:002019-10-07T13:00:34.718-07:00October Reading<div dir="ltr" style="text-align: left;" trbidi="on">
<div style="text-align: justify;">
Here's my latest, and final, list of suggested reading:</div>
<div style="text-align: justify;">
</div>
<ul>
<li><a href="http://crest.science/RePEc/wpstorage/2019-13.pdf">Bellego, C. and L-D. Pape</a>, 2019. Dealing with the log of zero in regression models. CREST Working Paper No. 2019-13.</li>
<li><a href="https://www.economics.ox.ac.uk/materials/working_papers/4982/selecting-a-model-for-forcasting-861.pdf">Castle, J. L., J. A. Doornik, and D. F. Hendry</a>, 2018. Selecting a model for forecasting. Department of Economics, University of Oxford, Discussion Paper 861.</li>
<li><a href="https://www.rba.gov.au/publications/rdp/2019/pdf/rdp2019-08.pdf">Gorajek, A.</a>, 2019. The well-meaning economist. Reserve Bank of Australia, Research Discussion Paper RDP 2019-08.</li>
<li><a href="https://www.tandfonline.com/doi/full/10.1080/03610918.2018.1473591">Güriş, B.</a>, 2019. A new nonlinear unit root test with Fourier function. <i>Communications in Statistics - Simulation and Computation</i>, 48, 3056-3062.</li>
<li><a href="https://bostonreview.net/science-nature/tim-maudlin-why-world">Maudlin, T.</a>, 2019. The why of the world. Review of <i>The Book of Why: The New Science of Cause and Effect</i>, by J. Pearl and D. Mackenzie. <i>Boston Review</i>.</li>
<li><a href="https://www.mdpi.com/2225-1146/7/3/39">Qian, W., C. A. Rolling, G. Cheng, and Y. Yang</a>, 2019. On the forecast combination puzzle. <i>Econometrics</i>, 7, 39. </li>
</ul>
<br />
<center>
© 2019, David E. Giles
</center>
</div>
Dave Gileshttp://www.blogger.com/profile/05389606956062019445noreply@blogger.com0tag:blogger.com,1999:blog-2198942534740642384.post-26672961465000917802019-09-01T06:40:00.003-07:002019-09-01T06:41:04.343-07:00Back to School Reading<div dir="ltr" style="text-align: left;" trbidi="on">
<div style="text-align: justify;">
Here we are - it's Labo(u)r Day weekend already in North America, and we all know what that means! It's back to school time.</div>
<div style="text-align: justify;">
<br /></div>
<div style="text-align: justify;">
You'll need a reading list, so here are some suggestions:<br />
<br />
<ul>
<li><a href="https://ideas.repec.org/p/ems/eureir/118666.html">Frances, Ph. H. B. F.</a>, 2019. Professional forecasters and January. Econometric Institute Research Papers EI2019-25, Erasmus University Rotterdam.</li>
<li><a href="https://www.sciencedirect.com/science/article/pii/S030440761930106X?dgcid=rss_sd_all">Harvey, A. & R. Ito</a>, 2019. Modeling time series when some observations are zero. <i>Journal of Econometrics</i>, in press.</li>
<li><a href="https://www.anderson.ucla.edu/faculty_pages/edward.leamer/books/specification_searches.htm">Leamer, E. E.</a>, 1978. <i>Specification Searches: Ad Hoc Inference With Nonexperimental Data.</i> Wiley, New York. (This is a legitimate free download.)</li>
<li><a href="https://ideas.repec.org/p/qed/wpaper/1413.html">MacKinnon, J. G.</a>, 2019. How cluster-robust inference is changing applied econometrics. Working Paper 1413, Economics Department, Queen's University.</li>
<li><a href="https://arxiv.org/pdf/1709.08221.pdf">Steel, M. F. J.</a>, 2019. Model averaging and its use in economics. Mimeo., Department of Statistics, University of Warwick.</li>
<li><a href="https://projecteuclid.org/euclid.aos/1176345451">Stigler, S. M.</a>, 1981. Gauss and the invention of least squares. <i>Annals of Statistics</i>, 9, 465-474. </li>
</ul>
</div>
<center style="text-align: center;">
© 2019, David E. Giles</center>
</div>
Dave Gileshttp://www.blogger.com/profile/05389606956062019445noreply@blogger.com0tag:blogger.com,1999:blog-2198942534740642384.post-7595575026408216012019-08-20T11:25:00.002-07:002019-08-20T11:25:42.538-07:00Book Series on "Statistical Reasoning in Science & Society"<div dir="ltr" style="text-align: left;" trbidi="on">
<div style="text-align: justify;">
Back in early 2016, the <a href="http://www.amstat.org/">American Statistical Association</a> (ASA) made an <a href="https://magazine.amstat.org/blog/2016/02/01/asacrc16/">announcement</a> in its newsletter, <i><a href="https://magazine.amstat.org/">Amstat News</a></i>, about the introduction of an important new series of books. In part, that announcement said:</div>
<blockquote class="tr_bq" style="text-align: justify;">
"The American Statistical Association recently partnered with <a href="http://crcpress.com/">Chapman & Hall/CRC Press</a> to launch a book series called the <a href="https://www.crcpress.com/go/asacrc"><i>ASA-CRC</i> </a><i><a href="https://www.crcpress.com/go/asacrc">Series on Statistical Reasoning in Science and Society</a>.</i><span style="text-align: left;"> </span></blockquote>
<blockquote class="tr_bq" style="text-align: justify;">
'The ASA is very enthusiastic about this new series,' said 2015 ASA President David Morganstein, under whose leadership the arrangement was made. 'Our strategic plan includes increasing the visibility of our profession. One way to do that is with books that are readable, exciting, and serve a broad audience having a minimal background in mathematics or statistics.'<span style="text-align: left;"> </span></blockquote>
<blockquote class="tr_bq" style="text-align: justify;">
The Chapman & Hall/CRC press release states the book series will do the following:<ul style="text-align: left;">
<li style="text-align: justify;">Highlight the important role of statistical and probabilistic reasoning in many areas</li>
</ul>
<ul style="text-align: left;">
<li style="text-align: justify;">Require minimal background in mathematics and statistics</li>
</ul>
<ul style="text-align: left;">
<li style="text-align: justify;">Serve a broad audience, including professionals across many fields, the general public, and students in high schools and colleges</li>
</ul>
<ul style="text-align: left;">
<li style="text-align: justify;">Cover statistics in wide-ranging aspects of professional and everyday life, including the media, science, health, society, politics, law, education, sports, finance, climate, and national security</li>
</ul>
<ul style="text-align: left;">
<li style="text-align: justify;">Feature short, inexpensive books of 100–150 pages that can be written and read in a reasonable amount of time."</li>
</ul>
</blockquote>
<div style="text-align: justify;">
Seven titles have now been published in this series -</div>
<div style="text-align: justify;">
<br /></div>
<i>Measuring Society</i>, by Chaitra H. Nagaraja (2019)<br />
<i>Measuring Crime: Behind the Statistics</i>, by Sharon L. Lohr (2019)<br />
<i>Statistics and Health Care Fraud: How to Save Billions</i>, by Tahir Ekin (2019)<br />
<i>Improving Your NCAA® Bracket with Statistics</i>, by Tom Adams (2018)<br />
<i>Data Visualization: Charts, Maps, and Interactive Graphics</i>, by Robert Grant (2018)<br />
<i>Visualizing Baseball</i>, by Jim Albert (2017)<br />
<i>Errors, Blunders, and Lies: How to Tell the Difference</i>, by David S. Salsburg (2017)<br />
<div style="text-align: justify;">
<br /></div>
<div style="text-align: justify;">
Readers of this blog should be especially interested in Chaitra Nagaraja's <a href="https://www.crcpress.com/Measuring-Society/Nagaraja/p/book/9781138035980">recently published addition</a> to this series. <a href="https://www.fordham.edu/info/22941/full-time_faculty/4830/chaitra_h_nagaraja">Chaitra</a> devotes chapters in her book to the topics of Jobs, Inequality, Housing, Prices, Poverty, and Deprivation. I particularly like the historical perspective that Chaitra provides in this very readable contribution, and I recommend her book to you (and your non-economist friends). </div>
<div style="text-align: justify;">
<br /></div>
<center>
© 2019, David E. Giles
</center>
</div>
Dave Gileshttp://www.blogger.com/profile/05389606956062019445noreply@blogger.com0tag:blogger.com,1999:blog-2198942534740642384.post-3526699954295388042019-08-14T14:28:00.000-07:002019-08-14T14:28:02.825-07:00Check out What Happened at the 2019 Joint Statistical Meetings<div dir="ltr" style="text-align: left;" trbidi="on">
<div style="text-align: justify;">
Each year, the Joint Statistical Meetings (JSM) bring together thousands (6,500 this year) of statisticians at what's the largest gathering of its type in the world. The JSM represent eleven international statistics organisations, including the four founding organisations - The American Statistical Association (ASA), The International Biometric Society, The Institute of Mathematical Statistical, and The Statistical Society of Canada.</div>
<div style="text-align: justify;">
<br /></div>
<div style="text-align: justify;">
As a member of the <a href="http://www.amstat.org/">ASA</a> since 1973 I've attended a few of these meetings over the years, but unfortunately I didn't make it to the <a href="https://ww2.amstat.org/meetings/jsm/2019/index.cfm">JSM in Denver</a> at the end of last month. As always, the program was amazing.</div>
<div style="text-align: justify;">
<br /></div>
<div style="text-align: justify;">
Yesterday, the ASA released a searchable version of the 2019 program that contains downloadable files of the slides used by many of the speakers. You can find that version of the program <a href="http://ww2.amstat.org/meetings/jsm/2019/onlineprogram/index.cfm?utm_source=informz&utm_medium=email&utm_campaign=asa&_zs=4kMOe1&_zl=S6I46">here</a>. When you go through the program, look for presentations that have blue (rectangular) "Presentation" button. Papers in sessions sponsored by the Business and Economic Statistics section of the ASA may be of special interest to you - but there's lots to choose from!</div>
<div style="text-align: justify;">
<br /></div>
<center>
© 2019, David E. Giles</center>
</div>
Dave Gileshttp://www.blogger.com/profile/05389606956062019445noreply@blogger.com1tag:blogger.com,1999:blog-2198942534740642384.post-48025224550449320802019-08-06T06:56:00.003-07:002019-08-06T06:57:37.716-07:00Including More History in Your Econometrics Teaching<div dir="ltr" style="text-align: left;" trbidi="on">
<div style="background-color: white;">
<div style="text-align: justify;">
If you follow this blog (or if you look at the "History of Econometrics" label in the word cloud in the right side-bar), you'll know that I have more than a passing interest in the history of our discipline. There's so much to be learned from this history. Among other things, we can gain insights into why certain methods became popular, and we can reduce the risk of repeating earlier mistakes!</div>
<div style="text-align: justify;">
<br /></div>
<div style="text-align: justify;">
When I was teaching I liked to inject a few historical facts/anecdotes/curiosities into my classes. I think that this brought the subject matter to life a little. The names behind the various theorems, tests, and estimators are those of <i>real people</i>, after all.</div>
<div style="text-align: justify;">
<br /></div>
<div style="text-align: justify;">
There are some excellent books on the history of econometrics, including those by Epstein (1987), Morgan (1990), and De Marchi and Gilbert (1991). <span style="font-family: inherit;">(Also, see the short piece by Stephen Pollock, 2014.)</span></div>
</div>
<div style="background-color: white;">
<div style="text-align: justify;">
<span style="font-family: inherit;"><br /></span></div>
<div style="text-align: justify;">
<span style="font-family: inherit;">However, I think that we could do more in terms of making material about this history accessible to our students.</span></div>
<div style="text-align: justify;">
<span style="font-family: inherit;"><br /></span></div>
<div style="text-align: justify;">
<span style="font-family: inherit;">The Statistics community has gone much further in this direction, and we might take note of this.</span></div>
<div style="text-align: justify;">
<span style="font-family: inherit;"><br /></span></div>
</div>
<div style="background-color: white;">
<div style="text-align: justify;">
<span style="font-family: inherit;">The other day, <a href="https://biostatistics.uams.edu/faculty-and-staff/amanda-golbeck-3/">Amanda Golbeck</a> posted some very helpful links on the <a href="https://www.amstat.org/">American Statistical Association's</a> <span style="font-family: inherit;">"</span><a href="https://community.amstat.org/historyofstats/home">History of Statistics Interest Group</a>" community noticeboard.</span></div>
</div>
<div style="background-color: white;">
<span style="font-family: inherit;"><br /></span></div>
<div style="background-color: white;">
<span style="font-family: inherit;">Here's <a href="https://community.amstat.org/communities/community-home/digestviewer/viewthread?MessageKey=27c83fb3-28c3-4c61-ad53-321b8dadf1fe&CommunityKey=6b2d607a-e31f-4f19-8357-020a8631b999&tab=digestviewer#bm27c83fb3-28c3-4c61-ad53-321b8dadf1fe">her posting</a> in its entirety - and don't miss the first of her links:</span></div>
<div style="background-color: white;">
<span style="font-family: inherit;"><br /></span></div>
<div style="background-color: white;">
<span style="font-family: inherit;">"Why not include more history in your teaching? The History of Statistics Interest Group library has a collection of Activities for Classes: <a href="https://community.amstat.org/historyofstats/ourlibrary/new-item2" rel="noreferrer" style="color: #336699;" target="_blank" title="https://community.amstat.org/historyofstats/ourlibrary/new-item2">community.amstat.org/historyofstats/ourlibrary/...</a></span></div>
<div style="background-color: white;">
<span style="font-family: inherit;"><br />We are pleased to let you know that Bob Rosenfeld has created 13 history of probability and statistics teaching modules, and he has kindly made them available for you to use in your classes! We hope you will find them to be useful.</span></div>
<div style="background-color: white;">
<br /></div>
<div style="background-color: white;">
<span style="font-family: inherit;">Reading and Exercises on the History of Probability from the <a href="https://www.vmimathematics.com/">Vermont Mathematics Initiative</a>, Bob Rosenfeld</span></div>
<ul style="background-color: white;">
<li><span style="font-family: inherit;"><a href="https://higherlogicdownload.s3.amazonaws.com/AMSTAT/1484431b-3202-461e-b7e6-ebce10ca8bcd/UploadedImages/Classroom_Activities/HP_1_Pre-history_to_1600.pdf" rel="noreferrer" style="color: #336699;" target="_blank" title="https://higherlogicdownload.s3.amazonaws.com/AMSTAT/1484431b-3202-461e-b7e6-ebce10ca8bcd/UploadedImages/Classroom_Activities/HP_1_Pre-history_to_1600.pdf">Pre-history to 1600</a> (PDF)</span></li>
<li><span style="font-family: inherit;"><a href="https://higherlogicdownload.s3.amazonaws.com/AMSTAT/1484431b-3202-461e-b7e6-ebce10ca8bcd/UploadedImages/Classroom_Activities/HP_2_17th_Century_France.pdf" rel="noreferrer" style="color: #336699;" target="_blank" title="https://higherlogicdownload.s3.amazonaws.com/AMSTAT/1484431b-3202-461e-b7e6-ebce10ca8bcd/UploadedImages/Classroom_Activities/HP_2_17th_Century_France.pdf">17th Century France</a> (PDF)</span></li>
<li><span style="font-family: inherit;"><a href="https://higherlogicdownload.s3.amazonaws.com/AMSTAT/1484431b-3202-461e-b7e6-ebce10ca8bcd/UploadedImages/Classroom_Activities/HP_3_Jacob_Bernoulli_-_Law_of_Large_Numbers.pdf" rel="noreferrer" style="color: #336699;" target="_blank" title="https://higherlogicdownload.s3.amazonaws.com/AMSTAT/1484431b-3202-461e-b7e6-ebce10ca8bcd/UploadedImages/Classroom_Activities/HP_3_Jacob_Bernoulli_-_Law_of_Large_Numbers.pdf">Jacob Bernoulli - Law of Large Numbers</a> (PDF)</span></li>
<li><span style="font-family: inherit;"><a href="https://higherlogicdownload.s3.amazonaws.com/AMSTAT/1484431b-3202-461e-b7e6-ebce10ca8bcd/UploadedImages/Classroom_Activities/HP_4_Inverse_Probability_-_Thomas_Bayes.pdf" rel="noreferrer" style="color: #336699;" target="_blank" title="https://higherlogicdownload.s3.amazonaws.com/AMSTAT/1484431b-3202-461e-b7e6-ebce10ca8bcd/UploadedImages/Classroom_Activities/HP_4_Inverse_Probability_-_Thomas_Bayes.pdf">Inverse Probability - Thomas Bayes</a> (PDF)</span></li>
<li><span style="font-family: inherit;"><a href="https://higherlogicdownload.s3.amazonaws.com/AMSTAT/1484431b-3202-461e-b7e6-ebce10ca8bcd/UploadedImages/Classroom_Activities/HP_5_Laplace.pdf" rel="noreferrer" style="color: #336699;" target="_blank" title="https://higherlogicdownload.s3.amazonaws.com/AMSTAT/1484431b-3202-461e-b7e6-ebce10ca8bcd/UploadedImages/Classroom_Activities/HP_5_Laplace.pdf">Laplace</a> (PDF)</span></li>
</ul>
<div style="background-color: white;">
<span style="font-family: inherit;">Reading and Exercises on the History of Statistics from the <a href="https://www.vmimathematics.com/">Vermont Mathematics Initiative</a>, Bob Rosenfeld</span></div>
<ul style="background-color: white;">
<li><span style="font-family: inherit;"><a href="https://higherlogicdownload.s3.amazonaws.com/AMSTAT/1484431b-3202-461e-b7e6-ebce10ca8bcd/UploadedImages/Classroom_Activities/HS_1__John_Graunt_and_the_Bills_of_Mortality.pdf" rel="noreferrer" style="color: #336699;" target="_blank" title="https://higherlogicdownload.s3.amazonaws.com/AMSTAT/1484431b-3202-461e-b7e6-ebce10ca8bcd/UploadedImages/Classroom_Activities/HS_1__John_Graunt_and_the_Bills_of_Mortality.pdf">John Graunt and the Bills of Mortality</a> (PDF)</span></li>
<li><span style="font-family: inherit;"><a href="https://higherlogicdownload.s3.amazonaws.com/AMSTAT/1484431b-3202-461e-b7e6-ebce10ca8bcd/UploadedImages/Classroom_Activities/HS_2__Origin_of_the_Normal_Curve.pdf" rel="noreferrer" style="color: #336699;" target="_blank" title="https://higherlogicdownload.s3.amazonaws.com/AMSTAT/1484431b-3202-461e-b7e6-ebce10ca8bcd/UploadedImages/Classroom_Activities/HS_2__Origin_of_the_Normal_Curve.pdf">Origin of the Normal Curve</a> (PDF)</span></li>
<li><span style="font-family: inherit;"><a href="https://higherlogicdownload.s3.amazonaws.com/AMSTAT/1484431b-3202-461e-b7e6-ebce10ca8bcd/UploadedImages/Classroom_Activities/HS_3_Origins_of_graphs_in_statistics.pdf" rel="noreferrer" style="color: #336699;" target="_blank" title="https://higherlogicdownload.s3.amazonaws.com/AMSTAT/1484431b-3202-461e-b7e6-ebce10ca8bcd/UploadedImages/Classroom_Activities/HS_3_Origins_of_graphs_in_statistics.pdf">Origins of Graphs in Statistics</a> (PDF)</span></li>
<li><span style="font-family: inherit;"><a href="https://higherlogicdownload.s3.amazonaws.com/AMSTAT/1484431b-3202-461e-b7e6-ebce10ca8bcd/UploadedImages/Classroom_Activities/HS_4_Fitting_models_to_data_-_the_path_to_Least_Squares.pdf" rel="noreferrer" style="color: #336699;" target="_blank" title="https://higherlogicdownload.s3.amazonaws.com/AMSTAT/1484431b-3202-461e-b7e6-ebce10ca8bcd/UploadedImages/Classroom_Activities/HS_4_Fitting_models_to_data_-_the_path_to_Least_Squares.pdf">Fitting models to data - the Path to Least Squares</a> (PDF)</span></li>
<li><span style="font-family: inherit;"><a href="https://higherlogicdownload.s3.amazonaws.com/AMSTAT/1484431b-3202-461e-b7e6-ebce10ca8bcd/UploadedImages/Classroom_Activities/HS_5_Statistics_moves_from_physical_to_social_sciences.pdf" rel="noreferrer" style="color: #336699;" target="_blank" title="https://higherlogicdownload.s3.amazonaws.com/AMSTAT/1484431b-3202-461e-b7e6-ebce10ca8bcd/UploadedImages/Classroom_Activities/HS_5_Statistics_moves_from_physical_to_social_sciences.pdf">Statistics Moves from Physical to Social Sciences</a> (PDF)</span></li>
<li><span style="font-family: inherit;"><a href="https://higherlogicdownload.s3.amazonaws.com/AMSTAT/1484431b-3202-461e-b7e6-ebce10ca8bcd/UploadedImages/Classroom_Activities/HS_6_Correlation_Francis_GALTON.pdf" rel="noreferrer" style="color: #336699;" target="_blank" title="https://higherlogicdownload.s3.amazonaws.com/AMSTAT/1484431b-3202-461e-b7e6-ebce10ca8bcd/UploadedImages/Classroom_Activities/HS_6_Correlation_Francis_GALTON.pdf">Correlation - Francis Galton</a> (PDF)</span></li>
<li><span style="font-family: inherit;"><a href="https://higherlogicdownload.s3.amazonaws.com/AMSTAT/1484431b-3202-461e-b7e6-ebce10ca8bcd/UploadedImages/Classroom_Activities/HS_7__t-ditribution_and_GOSSET.pdf" rel="noreferrer" style="color: #336699;" target="_blank" title="https://higherlogicdownload.s3.amazonaws.com/AMSTAT/1484431b-3202-461e-b7e6-ebce10ca8bcd/UploadedImages/Classroom_Activities/HS_7__t-ditribution_and_GOSSET.pdf">t-Distribution and Gosset </a>(PDF)</span></li>
<li><span style="font-family: inherit;"><a href="https://higherlogicdownload.s3.amazonaws.com/AMSTAT/1484431b-3202-461e-b7e6-ebce10ca8bcd/UploadedImages/Classroom_Activities/HS_8__FISHER_and_Design_of_experiments.pdf" rel="noreferrer" style="color: #336699;" target="_blank" title="https://higherlogicdownload.s3.amazonaws.com/AMSTAT/1484431b-3202-461e-b7e6-ebce10ca8bcd/UploadedImages/Classroom_Activities/HS_8__FISHER_and_Design_of_experiments.pdf">Fisher and Design of Experiments</a> (PDF)"</span></li>
</ul>
<div style="text-align: justify;">
(Bob Rosenfeld was former Co-Director for Statistics and School-Based Research at the Vermont Mathenatics initiative, and the author of a number of books on the teaching of statistics to K-8 students. D.G.)</div>
<div style="text-align: justify;">
<br /></div>
<div style="text-align: justify;">
Most of Bob Rosenfeld's pieces are directly relevant to econometrics students. It would be nice to see more material about the history of our discipline that could be incorporated into introductory econometrics courses.<br />
<br />
<b>References </b><br />
<br />
<a href="http://www.amazon.com/History-Methodology-Econometrics-Economic-Special/dp/0198283113/ref=sr_1_11?keywords=history+of+econometric&qid=1565099441&s=gateway&sr=8-11">De Marchi, N. & C. Gilbert</a>, 1990. <i>History and Methodology of Econometrics</i>. Oxford University Press, Oxford.<br />
<br />
<a href="https://www.amazon.com/History-Econometrics-Contributions-Economic-Analysis-ebook/dp/B01E3IISEE/ref=sr_1_2?keywords=a+history+of+econometric&qid=1565099332&s=gateway&sr=8-2">Epstein, R. J.</a> 1987. <i>A History of Econometrics. </i>North-Holland, Amsterdam.<br />
<br />
<a href="https://www.amazon.ca/History-Econometric-Ideas-Professor-Morgan/dp/0521424658">Morgan, M. S.</a>, 1991. <i>The History of Econometric Ideas</i>. Cambridge University Press, Cambridge.<br />
<br />
<a href="https://www.le.ac.uk/economics/research/RePEc/lec/leecon/dp14-05.pdf">Pollock, D. S. G.</a>, 2014. Econometrics - An historical guide for the uninitiated. Working Paper No. 14/05, Department of economics, University of Leicester.<br />
<br /></div>
<center>
© 2019, David E. Giles
</center>
</div>
Dave Gileshttp://www.blogger.com/profile/05389606956062019445noreply@blogger.com2tag:blogger.com,1999:blog-2198942534740642384.post-83403542178144254132019-08-02T10:20:00.001-07:002019-08-02T10:20:22.510-07:00Suggested Reading for August<div dir="ltr" style="text-align: left;" trbidi="on">
<div style="text-align: justify;">
Here are my suggestions for this month:</div>
<div style="text-align: justify;">
<ul>
<li><a href="https://www.tandfonline.com/doi/full/10.1080/07474938.2018.1427486">Bun, M. J. G. & T. D. Harrison</a>, 2109. OLS and IV estimation of regression models including endogenous interaction terms. <i>Econometric Reviews</i>, 38, 814-827.</li>
<li><a href="https://www.tandfonline.com/doi/abs/10.1080/07350015.2017.1371027">Dufour, J-M., E. Flachaire, & L. Khalaf</a>, Permutation tests for comparing inequality measures. <i>Journal of Business and Economic Statistics</i>, 37, 457-470.</li>
<li>Jiao, X. & F. Pretis, 2018. Testing the presence of outliers in regression models. <span style="background-color: white; color: #505050; font-family: "nexussanswebpro"; font-size: 16px;">Available at SSRN: </span><a href="https://ssrn.com/abstract=3217213" style="background-color: white; box-sizing: border-box; color: #505050; font-family: NexusSansWebPro; font-size: 16px; text-align: start;" target="_blank">https://ssrn.com/abstract=3217213</a>.</li>
<li><a href="https://www.tandfonline.com/doi/pdf/10.1080/10691898.2001.11910537?needAccess=true">Stanton, J. M.</a>, 2001. Galton, Pearson, and the peas: A brief history of linear regression for statistics instructors. <i>Journal of Statistics Education</i>, 9, 1-13.</li>
<li><a href="https://www.mdpi.com/2225-1146/7/2/26">Trafimow, D.</a>, 2019. A frequentist alternative to significance testing, p-values and confidence intervals. <i>Econometrics</i>, 7, 26.</li>
</ul>
</div>
<center style="text-align: center;">
© 2019, David E. Giles
</center>
</div>
Dave Gileshttp://www.blogger.com/profile/05389606956062019445noreply@blogger.com1tag:blogger.com,1999:blog-2198942534740642384.post-75040087965503424522019-07-28T08:40:00.003-07:002019-07-28T08:40:40.795-07:00AAEA Meeting, 2019<div dir="ltr" style="text-align: left;" trbidi="on">
<div style="text-align: justify;">
The Agricultural and Applied Economics Association (<a href="https://www.aaea.org/">AAEA</a>) recently held its <a href="https://www.aaea.org/meetings/2019-aaea-annual-meeting">annual meeting</a> in Atlanta, GA. You can find the extensive program <a href="https://www.aaea.org/UserFiles/file/am19-final_v15.pdf">here</a>.</div>
<div style="text-align: justify;">
<br /></div>
<div style="text-align: justify;">
This year, I was fortunate enough to be able to attend and participate.</div>
<div style="text-align: justify;">
<br /></div>
<div style="text-align: justify;">
This was thanks to the kind invitation of <a href="https://www.apec.umn.edu/people/marc-f-bellemare">Marc Bellemare</a>, a member of the Executive Board of the AAEA, and (of course) a <a href="http://marcfbellemare.com/wordpress/">blogger</a> whom many of you no doubt follow. (If you don't, then you should!) </div>
<div style="text-align: justify;">
<br /></div>
<div style="text-align: justify;">
Marc arranged a session in which he and I talked about the pros and cons of <i>The Cookbook Approach to Teaching Econometrics</i>. The session was well attended, and the bulk of the time was devoted to a very helpful discussion-question-answer period with the audience.</div>
<div style="text-align: justify;">
<br /></div>
<div style="text-align: justify;">
As you'll know from some of my previous posts (<i>e.g</i>., <a href="https://davegiles.blogspot.com/2011/05/cookbook-econometrics.html">here</a> and <a href="https://davegiles.blogspot.com/2013/05/cookbook-econometrics-reprise.html">here</a>), I'm not a big fan of <i>The Cookbook Approach - </i>at least, not if it's the primary/sole way of teaching econometrics. Marc made the point that there's a place for this approach if it's adopted after more formal courses in econometrics. I'm in agreement with that.</div>
<div style="text-align: justify;">
<br /></div>
<div style="text-align: justify;">
I put together a few background talking-point slides for my short presentation. For what they're worth, you'll find then <a href="https://www.dropbox.com/s/hhxjrmjtj8c2qx6/Presentation.pdf?dl=0">here</a>.</div>
<div style="text-align: justify;">
<br /></div>
<div style="text-align: justify;">
I really enjoyed my time at the AAEA meeting, and I learned a lot. Thanks, Marc, and thank you to the participants!</div>
<div style="text-align: justify;">
<br /></div>
<center>
© 2019, David E. Giles
</center>
</div>
Dave Gileshttp://www.blogger.com/profile/05389606956062019445noreply@blogger.com3tag:blogger.com,1999:blog-2198942534740642384.post-35907940836596214822019-07-06T10:25:00.003-07:002019-08-21T08:52:30.359-07:00Seasonal Unit Roots - Background Information<div dir="ltr" style="text-align: left;" trbidi="on">
<div dir="ltr" style="text-align: left;" trbidi="on">
<div dir="ltr" style="text-align: left;" trbidi="on">
<div dir="ltr" style="text-align: left;" trbidi="on">
<div dir="ltr" style="text-align: left;" trbidi="on">
<div style="text-align: justify;">
A recent email query about the language that we use in the context of non-stationary seasonal data, and how we should respond to the presence of "seasonal unit roots", suggested to me that a short background post about some of this might be in order.</div>
<div style="text-align: justify;">
<br /></div>
<div style="text-align: justify;">
To get the most from what follows, I suggest that you take a quick look at this <a href="http://davegiles.blogspot.ca/2015/12/seasonal-unit-root-testing-in-eviews.html">earlier post</a> of mine - especially to make sure that you understand the distinction between "deterministic" seasonality" and "stochastic seasonality" in time-series data.</div>
<div style="text-align: justify;">
<br /></div>
<div style="text-align: justify;">
There's an extensive econometrics literature on stochastic seasonality and testing for seasonal unit roots, and this dates back at least to 1990. This is hardly a new topic, but it's one that's often overlooked in the empirical applications.</div>
<div style="text-align: justify;">
<br /></div>
<div style="text-align: justify;">
Although several tests for seasonal unit roots are available, the most commonly used one is that proposed by Hylleberg <i>et al</i>. (1990) - hereafter "HEGY". Depending on what statistical/econometrics package you prefer to use, you'll have at least some access to the HEGY test(s), and perhaps some others. For instance there are routines that you can use with R, stata, and Gretl.</div>
<div style="text-align: justify;">
<br /></div>
<div style="text-align: justify;">
The <a href="http://www.eviews.com/home.html">EViews</a> package includes a rather complete built-in suite of different seasonal unit root tests for time series data with various periodicities - 2, 4, 5, 6, 7, and 12. This enables us to deal with trading-day weekly data, and calendar weekly data, as well as the usual "seasonal" frequencies. </div>
<div style="text-align: justify;">
<br /></div>
<div style="text-align: justify;">
I'm not going to be going over the tests themselves here.</div>
<div style="text-align: justify;">
<br /></div>
<div style="text-align: justify;">
Rather, the objectives of this post are, first, to provide a bit of background information about the language that's used when we're talking about seasonal unit roots. For instance, why do we refer to roots at the zero, π, frequencies, <i>etc</i>.? Second, in what way(s) do we need to filter a time series in order to remove the unit roots at the various frequencies?</div>
<div style="text-align: justify;">
</div>
<div style="text-align: justify;">
<br /></div>
<div style="text-align: justify;">
Let's begin by considering a <i>quarterly</i> time series, X<sub>t</sub> (t = 1, 2, ........). We'll use the symbol "L" to denote the lag operator. So. L(X<sub>t</sub>) = X<sub>t-1</sub>; L<sup>2</sup>(X<sub>t</sub>) = L(L(X<sub>t</sub>)) = L(X<sub>t-1</sub>) = X<sub>t-2</sub>; <i>etc</i>. In general, L<sup>k</sup>(X<sub>t</sub>) = X<sub>t-k</sub>.</div>
<div style="text-align: justify;">
<br />
<a name='more'></a><br />
<b><span style="color: #cc0000;">Unit roots at different frequencies</span></b><br />
<br /></div>
<div style="text-align: justify;">
If we consider the difference between the value of the series, X, now and its value four quarters (one year) ago, we can represent this by (X<sub>t</sub> - X<sub>t-4</sub>) = (X<sub>t</sub> - L<sup>4</sup>X<sub>t</sub>) = (1 - L<sup>4</sup>)X<sub>t</sub>. Let's take a closer look at the polynomial equation, (1 - L<sup>4</sup>) = 0, in the lag operator, and ask what are its roots?"</div>
<div style="text-align: justify;">
<br /></div>
<div style="text-align: justify;">
We can factorize (1 - L<sup>4</sup>) as follows: </div>
<div style="text-align: justify;">
<br /></div>
<div style="text-align: justify;">
(1 - L<sup>4</sup>) = (1 - L<sup>2</sup>)(1 + L<sup>2</sup>)<br />
= (1 - L)(1 + L)(1 + L<sup>2</sup>) = (1 - L)(1 + L)(1 + iL)(1 - iL) = 0, (1)</div>
</div>
<div style="text-align: justify;">
<br /></div>
<div style="text-align: justify;">
and then we see that the roots of (1) are L = 1; L = -1; L = i; and L = -i. Here, "i" is the imaginary number, whose square is -1.</div>
<div style="text-align: justify;">
<br /></div>
<div style="text-align: justify;">
In fact, each of these roots can be written as complex numbers, each of the form (x + iy). For instance, the root L = 1 corresponds to the case x = 1, y = 0. You may also recall that instead of expressing a complex number in terms of Cartesian coordinates, we can write it in terms of <span style="background-color: white;"><a href="http://davegiles.blogspot.ca/2016/01/why-does-pi-appear-in-normal-density.html">polar coordinates</a></span><span style="background-color: white;">. That is, we can write it in the form r(cosθ + i sinθ), where "r" is what we call the "radial coordinate", and θ is the "angular coordinate". <i>Without loss of generality, we'll normalize the former and set r =1 in what follows</i>.</span></div>
<div style="text-align: justify;">
<span style="background-color: white;"><br /></span></div>
<div style="text-align: justify;">
<span style="background-color: white;">Alright, so where does this leave us? Well, we're going to have to recall some of that trigonometry that you learned in high school!</span></div>
<div style="text-align: justify;">
<span style="background-color: white;"><br /></span></div>
<div style="text-align: justify;">
<span style="background-color: white;">Let's look at the graphs for the sine and cosine functions, with the argument (x) measured in radians:</span><br />
<span style="background-color: white;"><br /></span><span style="background-color: white;"><br /></span><br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiTeMSPA4u6sPVZM9Z-XFLUDeDhyiERJ7bDz_aaPAgAstJsWFYrfgxWU1LEvsoh4rJT_Ivqrz7oZaSVt_riURPtaDJIPo2Uj5X-uLGj9lJlEbdE26vGS4wmfhsaGaPgE1K7soZohhboXnVh/s1600/images.jpg" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="214" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiTeMSPA4u6sPVZM9Z-XFLUDeDhyiERJ7bDz_aaPAgAstJsWFYrfgxWU1LEvsoh4rJT_Ivqrz7oZaSVt_riURPtaDJIPo2Uj5X-uLGj9lJlEbdE26vGS4wmfhsaGaPgE1K7soZohhboXnVh/s320/images.jpg" width="320" /></a></div>
<span style="background-color: white;"><br /></span></div>
<div style="text-align: justify;">
<span style="background-color: white;"><br /></span></div>
<div style="text-align: justify;">
<span style="background-color: white;">These two functions "repeat themselves" every 2π radians (<i>i.e</i>., 360 degrees). This corresponds, of course, to going exactly once around a circle.</span></div>
<div style="text-align: justify;">
<span style="background-color: white;"><br /></span></div>
<div style="text-align: justify;">
So, the root of (1) corresponding to L = 1 can be written as (1 + i 0) = (cosθ + i sinθ), and from the sine and cosine graphs we can see that this implies that θ = 0 (or 0 +/- multiples of 2π, which are still 0). In other words, the series exhibits one cycle per year.</div>
<div style="text-align: justify;">
<br /></div>
<div style="text-align: justify;">
<span style="background-color: white;">Similarly, </span>the root of (1) corresponding to L = -1 can be written as (-1 + i0) = (cosθ + i sinθ), and from the sine and cosine graphs we can see that this implies that θ = <span style="background-color: white;">π (or </span><span style="background-color: white;">π</span><span style="background-color: white;"> </span><span style="background-color: white;">+/- multiples of </span>2π)<span style="background-color: white;">. Now the series exhibits two cycles per year.</span><br />
<br />
(We don't need to worry about the additional multiples of 2π, as this would take us "around the circle" more than once. So let's forget about this detail.)</div>
<div style="text-align: justify;">
<span style="background-color: white;"><br /></span></div>
<div style="text-align: justify;">
<span style="background-color: white;">Finally, </span>the roots of (1) corresponding to L = +/-i can be written as (0 +/- i) = (cosθ + i sinθ), and from the sine and cosine graphs we can see that this implies that θ = π/2 or 3π/2. There are now four cycles per year in the data.<br />
<br />
To summarize, we can have roots of equation (1) that correspond to one or more of the zero, π, or π/2 or 3π/2 frequencies. Moreover, these last two frequencies really need to be thought of as a pair - after all, they're associated with a complex conjugate pair in the Cartesian coordinate system, whereas the other two roots are "real".</div>
<div style="text-align: justify;">
<br />
So much for the language associated with seasonal unit roots.<br />
<br />
<b><span style="color: #cc0000;">Filtering the data</span></b><br />
<br />
What filters are need to eliminate the various roots, so as to render the X series stationary? </div>
<div style="text-align: justify;">
<br />
<b>(i)</b> If L = 1, then this corresponds to the transformation, or filter, (1 - L)X<sub>t</sub> = 0. In other words, if there is a unit root at the zero frequency then we need to construct Y<sub>t</sub> = (1 - L)X<sub>t</sub> = (X<sub>t</sub> - X<sub>t-1</sub>) to get a stationary series. The usual first-differencing of the data is appropriate.<br />
<br b="" />
<b>(ii)</b> <span style="text-align: left;">If L = -1, then this corresponds to the filter, (1 + L)X</span><sub style="text-align: left;">t</sub><span style="text-align: left;"> = 0. That is, if there is a unit root at the π frequency then we need to construct Y</span><sub style="text-align: left;">t</sub><span style="text-align: left;"> = (1 + L)X</span><sub style="text-align: left;">t</sub><span style="text-align: left;"> = (X</span><sub style="text-align: left;">t</sub><span style="text-align: left;"> + X</span><sub style="text-align: left;">t-1</sub><span style="text-align: left;">) to get a stationary series. Notice that this particular filter </span><i style="text-align: left;"><b>doesn't</b></i><span style="text-align: left;"> involve "differencing" the data.</span></div>
</div>
</div>
</div>
<br />
<div style="text-align: justify;">
<b>(iii)</b> If either L = i, or L = -1, then this corresponds to the filter (1 + L<sup>2</sup>)X<sub>t</sub> = 0. So, if there is a unit root at the π/2 or 3π/2 pair of frequencies, then we need to construct Y<sub>t</sub> = (1 + L<sup>2</sup>)X<sub>t</sub> = (X<sub>t</sub> + X<sub>t-2</sub>) to get a stationary series. Again, this filter doesn't involve the usual "differencing".</div>
<br />
<div style="text-align: justify;">
<b style="text-align: left;"><span style="color: #cc0000;">Multiple roots</span></b></div>
<div style="text-align: justify;">
<br /></div>
<div style="text-align: justify;">
Of course, it's quite possible that our time series, X<sub>t</sub>, has unit roots at more than one frequency. For example, it may have roots at both the zero and π frequencies. In that case, the filter that will make the series stationary is (1 - L)(1 + L) = (1 - L<sup>2</sup>). So, we construct Y<sub>t</sub> = (X<sub>t</sub> - X<sub>t-2</sub>). Similarly, if X has unit roots at all of the <i>seasonal</i> frequencies, but not at the zero frequency, then the appropriate filter is (1 + L)(1 + L<sup>2</sup>) = (1 + L + L<sup>2</sup> + L<sup>3</sup>), and the series Y<sub>t</sub> = (X<sub>t</sub> + X<sub>t-1</sub> + X<sub>t-2</sub> + X<sub>t-3</sub>) will be stationary; and so on. If there are unit roots at all four frequencies, then X series is said "seasonally integrated", and the relevant filter is (1 - L)(1 + L)(1 + L<sup>2</sup>) = (1 - L<sup>4</sup>), and so we "fourth-difference" X<sub>t</sub> and form Y<sub>t</sub> = (X<sub>t</sub> - X<sub>t-4</sub>).</div>
<div style="text-align: justify;">
<br /></div>
<div style="text-align: justify;">
<b><span style="color: #cc0000;">Some Extensions</span></b> </div>
<div style="text-align: justify;">
<br /></div>
<div style="text-align: justify;">
The above discussion is cast in terms of <i>quarterly</i> time-series data. If we have data that are recorded twice-yearly, you should be able to see from the factorization (1 - L<sup>2</sup>) = (1 - L)(1 + L) that there can be unit roots only at either the zero or π frequencies. (See Feltham and Giles, 2003, for more on this.)<br />
<br />
You might guess that case of monthly data gets pretty messy! (See Beaulieu and Miron, 1993.) In this case the unit roots correspond to L = ± 1, ± i, ± (1 ± √(3i))/2, ± (√(3) ± i)/2. The various frequencies are zero, π/6, π/3, π/2, 2π/3, 5π/6, and π.<br />
<br />
The final matter that needs mentioning here is the possibility of cointegration when we have two or more seasonal time series. Suppose that X<sub>1t</sub> and X<sub>2t</sub> are quarterly series, and they each have unit roots at (say) the π frequency. Then it's possible that they may be cointegrated at this frequency. Similarly if X<sub>1t</sub> has unit roots at all frequencies, and X<sub>2t</sub> has unit roots at the π and zero frequencies, then then the two series may be cointegrated at the zero and/or π frequencies. And so on.<br />
<br />
Engle <i>et al</i>. (1993) provide and illustrate a systematic testing framework for seasonal cointegration. It's essentially a generalization of the Engle-Granger two-step cointegration test, with the HEGY tests replacing the ADF test. For another application, see Reinhardt and Giles (2001). Lee (1992) extends the Johansen cointegration tests to the case of (quarterly) seasonal cointegration. A nice application of this procedure is given by Debenedictis (1997).<br />
<br /></div>
<div style="text-align: justify;">
<b>References</b></div>
<div style="text-align: justify;">
<b></b></div>
<div style="text-align: justify;">
<b><br /></b></div>
<div style="text-align: justify;">
<a href="http://www.sciencedirect.com/science/article/pii/030440769390018Z">Beaulieu, J. J., & J. A. Miron</a>, 1993. Seasonal unit roots in aggregate U.S. data. <i>Journal of<span class="Apple-tab-span" style="white-space: pre;"> </span>Econometrics</i>, 55, 305-328.</div>
<div style="text-align: justify;">
<br />
<a href="https://www.tandfonline.com/doi/abs/10.1080/000368497326534">Debenedictis, L. F.</a>, 1997. A vector autoregressive model of the British Columbia regional economy. <i>Applied Economics</i>, 29. 877-888. </div>
<div style="text-align: justify;">
<br />
<a href="https://www.tandfonline.com/doi/abs/10.1080/000368497326534">Engle, R. F., C. W. J. Granger, Hylleberg, S. & H. S. Lee</a>, 1993. The Japanese consumption function. <i>Journal of Econometrics</i>, 55, 275-298.<br />
<br />
Feltham, S. G. & D. E. A. Giles, 2003. Testing for unit roots in semi-annual data. in D. E. A. Giles (ed.), <i>Computer-Aided Econometrics. </i>Marcel Dekker, New York, 175-208. (<a href="http://web.uvic.ca/~dgiles/blog/HEGY_paper.pdf">Pre-print here</a>.)</div>
<div style="text-align: justify;">
<br /></div>
<div style="text-align: justify;">
<a href="http://www.sciencedirect.com/science/article/pii/0304407694900302">Ghysels, E., H. S. Lee, & J. Noh</a>, 1994. Testing for unit roots in seasonal time series: Some theoretical extensions and a Monte Carlo investigation. <i>Journal of Econometrics</i>, 62, 415-442.<br />
<br />
<a href="https://www.sciencedirect.com/science/article/pii/030440769090080D">Hylleberg, S., R. F. Engle, C. W. J. Granger, & B. S. Yoo</a>, 1990. Seasonal integration and cointegration. <i>Journal of Econometrics</i>, 44, 215-228.<br />
<br /></div>
<div style="text-align: justify;">
<a href="https://www.sciencedirect.com/science/article/pii/030440769290098C">Lee, H. S.</a>, 1992. Maximum likelihood inference on cointegration and seasonal cointegration. <i>Journal of Econometrics</i>, 54, 1-47.</div>
<div style="text-align: justify;">
<br />
<a href="https://www.tandfonline.com/doi/abs/10.1080/00036840010007489">Reinhardt, F. S. & D. E. A. Giles</a>, 2001. Are cigarette bans really good economic policy?. <i>Applied Economics</i>, 33, 1365-1368. </div>
<div style="text-align: justify;">
<br /></div>
<center>
© 2019, David E. Giles</center>
</div>
Dave Gileshttp://www.blogger.com/profile/05389606956062019445noreply@blogger.com5tag:blogger.com,1999:blog-2198942534740642384.post-47370841165954151942019-07-01T06:38:00.002-07:002019-07-01T06:38:37.115-07:00July Reading<div dir="ltr" style="text-align: left;" trbidi="on">
<div style="text-align: justify;">
This month my reading list is a bit different from the usual one. I've taken a look back at past issues of <i>Econometrica</i> and <i>Journal of Econometrics</i>, and selected some important and interesting papers that happened to be published in July issues of those journals.</div>
<div style="text-align: justify;">
<br /></div>
<div style="text-align: justify;">
Here's what I came up with for you:</div>
<div style="text-align: justify;">
</div>
<ul>
<li style="text-align: justify;"><a href="https://www.sciencedirect.com/science/article/pii/0304407677900525">Aigner, D., C. A. K. Lovell, & P. Schmidt</a>, 1977. Formulation and estimation of stochastic frontier production function models. <i>Journal of Econometrics</i>, 6, 21-37.</li>
<li style="text-align: justify;"><a href="https://www.econometricsociety.org/publications/econometrica/1960/07/01/tests-equality-between-sets-coefficients-two-linear-regressions">Chow, G. C.</a>, 1960. Tests of equality between sets of coefficients in two linear regressions. <i>Econometrica</i>, 28, 591-605.</li>
<li style="text-align: justify;"><a href="https://www.sciencedirect.com/science/article/pii/0304407684900010">Davidson, R. & J. G. MacKinnon</a>, 1984. Convenient specification tests for logit and probit models. <i>Journal of Econometrics</i>, 25, 241-262.</li>
<li style="text-align: justify;"><a href="https://www.econometricsociety.org/publications/econometrica/1981/07/01/likelihood-ratio-statistics-autoregressive-time-series-unit">Dickey, D. A. & W. A. Fuller</a>, 1981. Likelihood ratio statistics for autoregressive time series with a unit root. <i>Econometrica</i>, 49, 1057-1072.</li>
<li style="text-align: justify;"><a href="https://www.sciencedirect.com/science/article/pii/0304407674900347">Granger, C. W. J. & P. Newbold</a>, 1974. Spurious regressions in econometrics<i>. Journal of Econometrics</i>, 2, 111-120.</li>
<li style="text-align: justify;"><a href="https://www.econometricsociety.org/publications/econometrica/1961/07/01/maximum-likelihood-estimation-economic-relationships">Sargan, J. D.</a>, 1961. The maximum likelihood estimation of economic relationships with autoregressive residuals. <i>Econometrica</i>, 29, 414-426. </li>
</ul>
<center style="text-align: center;">
© 2019, David E. Giles
</center>
</div>
Dave Gileshttp://www.blogger.com/profile/05389606956062019445noreply@blogger.com0tag:blogger.com,1999:blog-2198942534740642384.post-8312541512836087722019-06-21T13:25:00.003-07:002019-06-21T13:25:57.859-07:00Consulting Can be Fun!<div dir="ltr" style="text-align: left;" trbidi="on">
<div style="text-align: justify;">
Over the years, I've done a modest amount of paid econometrics consulting work - in the U.S., New Zealand, Australia, the U.K., and here in Canada. Each job has been interesting, and rewarding, and I've always learned a great deal form the briefs that I've undertaken.</div>
<div style="text-align: justify;">
<br /></div>
<div style="text-align: justify;">
The other day, a friend asked me, "Which consulting job was the most fun?"</div>
<div style="text-align: justify;">
<br /></div>
<div style="text-align: justify;">
Actually, the answer was easy!</div>
<div style="text-align: justify;">
<br /></div>
<div style="text-align: justify;">
A few years ago I consulted for the Office of the Auditor General of Canada, in Ottawa. I was brought in because I had consulted for Revenue New Zealand on the issue of tax evasion, and I had co-authored <a href="https://www.ctf.ca/ctfweb/en/publications/product_detail.aspx?prod=TP106">a book on the Canadian "underground economy" </a>with <a href="https://www.linkedin.com/in/lindsay-tedds-b68b4311/?originalSubdomain=ca">Lindsay Tedds</a>.</div>
<div style="text-align: justify;">
<br /></div>
<div style="text-align: justify;">
So what was the consulting work with the Auditor General's office all about? Well, they were conducting an audit of what was then called Revenue Canada (now, the Canadian Revenue Agency). In other words, "the tax man"!</div>
<div style="text-align: justify;">
<br /></div>
<div style="text-align: justify;">
Although the report arising from this audit is a matter of public record, I won't go into it here. </div>
<div style="text-align: justify;">
<br /></div>
<div style="text-align: justify;">
Suffice to say, what could be more fun that conducting an audit of your country's tax authority?</div>
<div style="text-align: justify;">
<br /></div>
<center>
© 2019, David E. Giles
</center>
</div>
Dave Gileshttp://www.blogger.com/profile/05389606956062019445noreply@blogger.com0tag:blogger.com,1999:blog-2198942534740642384.post-47846025606683284782019-06-20T11:13:00.001-07:002019-06-20T11:13:43.929-07:002019 Edition of the INOMICS Handbook<div dir="ltr" style="text-align: left;" trbidi="on">
<div style="text-align: justify;">
I'm sure that all readers will be familiar with <a href="https://inomics.com/">INOMICS</a>, and the multitude of resources that they make available to economists.</div>
<div style="text-align: justify;">
<br /></div>
The <a href="https://inomics.com/handbook" style="font-style: italic;">INOMICS Handbook, 2019</a> is now available, and I commend it to you.<br />
<br />
This year's edition of the <i>Handbook</i> includes material relating to:<br />
<ul style="text-align: left;">
<li>The gender bias in the field of economics</li>
<li>The soft skills you need to succeed as an economist</li>
<li>Climate change and how economics can help solve it</li>
<li>What makes a successful economist</li>
<li>An exclusive interview with Princeton Professor, Esteban Rossi-Hansberg</li>
<li>Winners of the INOMICS Awards 2019</li>
<li>Recommended study and career opportunities</li>
</ul>
<center>
© 2019, David E. Giles
</center>
</div>
Dave Gileshttp://www.blogger.com/profile/05389606956062019445noreply@blogger.com0tag:blogger.com,1999:blog-2198942534740642384.post-83506038056500672862019-06-11T10:02:00.001-07:002019-06-11T10:02:01.739-07:00More Tributes to Clive Granger<div dir="ltr" style="text-align: left;" trbidi="on">
<div style="text-align: justify;">
As a follow-up to my recent post, "<a href="https://davegiles.blogspot.com/2019/06/clive-granger-special-issue.html">Clive Granger Special Issue</a>", I received an email from <a href="http://www.eyupcetin.com/">Eyüp Çetin</a> (Editor of the <i>European Journal of Pure and Applied Mathematics</i>)<i>.</i></div>
<div style="text-align: justify;">
<br /></div>
<div style="text-align: justify;">
<a href="http://www.eyupcetin.com/">Eyüp</a> kindly pointed out that ".........<span style="color: #000099; font-family: georgia, serif; text-align: left;"> </span><span style="font-family: inherit;"><span style="text-align: left;">actually, we published the first special issue dedicated to his memory exactly on 27 May 2010, the first anniversary of his passing at </span><a href="https://www.ejpam.com/index.php/ejpam/issue/view/11" rel="noreferrer" style="text-align: left;" target="_blank">https://www.ejpam.com/index.php/ejpam/issue/view/11</a><span style="text-align: left;"> </span></span></div>
<div class="gmail_default">
<span style="font-family: inherit;"><br /></span></div>
<div class="gmail_default">
<span style="font-family: inherit;">We think this was the first special issue dedicated to his memory in the world. </span><span style="font-family: inherit;">The Table of Contents may be found here </span><a href="https://www.ejpam.com/index.php/ejpam/issue/view/11/showToc" rel="noreferrer" style="font-family: inherit;" target="_blank">https://www.ejpam.com/index.php/ejpam/issue/view/11/showToc</a> .</div>
<div class="gmail_default" style="text-align: justify;">
<span style="font-family: inherit;"><br /></span></div>
<div class="gmail_default" style="text-align: justify;">
<span style="font-family: inherit;">Another remarkable point that we also published some personal and institutional tributes and some memorial stories for Sir Granger that never appeared elsewhere before at </span></div>
<div class="gmail_default" style="text-align: justify;">
<span style="font-family: inherit;"><a href="https://www.ejpam.com/index.php/ejpam/article/view/805" rel="noreferrer" target="_blank">https://www.ejpam.com/index.php/ejpam/article/view/805</a> .</span></div>
<div class="gmail_default" style="text-align: justify;">
<span style="font-family: inherit;"><br /></span></div>
<div class="gmail_default" style="text-align: justify;">
<span style="font-family: inherit;">Some institutions such as Royal Statistical Society, Japan Statistical Society and University of Canterbury have sent their tributes to this special volume." </span></div>
<div class="gmail_default" style="color: #000099; font-family: georgia, serif;">
<br style="background-color: white; font-size: 11px;" /></div>
<center>
© 2019, David E. Giles
</center>
</div>
Dave Gileshttp://www.blogger.com/profile/05389606956062019445noreply@blogger.com0tag:blogger.com,1999:blog-2198942534740642384.post-26463114217657849732019-06-07T00:43:00.004-07:002019-06-11T09:51:48.763-07:00Clive Granger Special Issue<div dir="ltr" style="text-align: left;" trbidi="on">
<div style="text-align: justify;">
The recently published <a href="https://www.ejpam.com/index.php/ejpam/issue/view/39">Volume 10, No. 1 issue</a> of the <i>European Journal of Pure and Applied Mathematics</i> takes the form of a memorial issue for Clive Granger. You can find the Table of Contents <a href="https://www.ejpam.com/index.php/ejpam/issue/view/39/showToc">here</a>, and all of the articles can be downloaded freely.</div>
<div style="text-align: justify;">
<br /></div>
<div style="text-align: justify;">
This memorial issue is co-edited by Jennifer Castle and David Hendry. The contributed papers include ones that deal with Forecasting, Cointegration, Nonlinear Time Series, and Model Selection.</div>
<div style="text-align: justify;">
<br /></div>
<div style="text-align: justify;">
This is a fantastic collection of important survey-type papers that simply <i>must read</i>!<br />
<br /></div>
<center>
© 2019, David E. Giles
</center>
</div>
Dave Gileshttp://www.blogger.com/profile/05389606956062019445noreply@blogger.com1tag:blogger.com,1999:blog-2198942534740642384.post-31612147128739959702019-05-31T10:01:00.000-07:002019-05-31T10:01:16.082-07:00Reading Suggestions for June<div dir="ltr" style="text-align: left;" trbidi="on">
<div style="text-align: justify;">
Well, here we are - it's June already.</div>
<div style="text-align: justify;">
<br /></div>
<div style="text-align: justify;">
Here are my reading suggestions:</div>
<div style="text-align: justify;">
</div>
<ul>
<li style="text-align: justify;"><a href="https://arxiv.org/abs/1710.02926">Abadie, A., S. Athey, G. Imbens, & J. Wooldridge</a>, 2017. When should you adjust standard errors for clustering? Mimeo.</li>
<li style="text-align: justify;"><a href="https://www.tandfonline.com/doi/full/10.1080/00031305.2019.1592781">Berk, R., A. Buja, L. Brown, E. George, A. K. Kuchibhotla, W. Su, & L, Shazo</a>, 2019. Assumption lean regression. <i>American Statistician</i>, in press.</li>
<li style="text-align: justify;"><a href="https://www.tandfonline.com/doi/full/10.1080/00031305.2018.1529626">Ghosh, T., M. Ghosh, & T. Kubokawa</a>, 2019. On the loss robustness of least-square estimators, <i>American Statistician</i>, in press.</li>
<li style="text-align: justify;"><a href="https://www.tandfonline.com/doi/full/10.1080/03610926.2018.1528369">Gustafsson, O. & P. Stockhammar</a>, 2019. Variance stabilizing filters. <i>Communications in Statistics - Theory and Methods</i>, in press.</li>
<li style="text-align: justify;"><a href="https://drive.google.com/file/d/1OBQEclDiOJVjzSr9xNQzAFweN8vMYGVD/view">Kocherlakota, N. R.</a>, 2019. A near-exact finite sample theory for an instrumental variable estimator. Mimeo. (Hat-tip to Frank Diebold.)</li>
<li style="text-align: justify;"><a href="https://www.sciencedirect.com/science/article/pii/S0169207019300056">Panagiotelis, A., G. Anathasopoulos, R. J. Hyndman, B. Jiang, & F. Vahid</a>, 2019. Macroeconomic forecasting for Australia using a large number of predictors. <i>International Journal of Forecasting</i>, 35, 613-633.</li>
</ul>
<center>
© 2019, David E. Giles
</center>
</div>
Dave Gileshttp://www.blogger.com/profile/05389606956062019445noreply@blogger.com0tag:blogger.com,1999:blog-2198942534740642384.post-75240359550960617862019-05-19T11:27:00.004-07:002019-05-19T11:28:20.128-07:00Update on the "Series of Unsurprising Results in Economics"<div dir="ltr" style="text-align: left;" trbidi="on">
<div style="text-align: justify;">
In June of last year I had <a href="https://davegiles.blogspot.com/2018/06/the-series-of-unsurprising-results-in.html">a post</a> about a new journal, <i><a href="https://blogs.canterbury.ac.nz/surejournal/">Series of Unsurprising Results in Economics</a> </i>(<i>SURE</i>).</div>
<div style="text-align: justify;">
<br /></div>
<div style="text-align: justify;">
If you didn't get to read that post, I urge you to do so. </div>
<div style="text-align: justify;">
<br /></div>
<div style="text-align: justify;">
More importantly, you should definitely take a look at <a href="https://www.vox.com/future-perfect/2019/5/17/18624812/publication-bias-economics-journal">this piece</a> by Kelsey Piper, from a couple of days ago, and titled, "This economics journal only publishes results that are no big deal - Here’s how that might save science".</div>
<div style="text-align: justify;">
<br /></div>
<div style="text-align: justify;">
Kelsey really understands the rationale for <i>SURE</i>, and the important role that it can play in terms of reducing publication bias, and assisting with replicating results.</div>
<div style="text-align: justify;">
<br /></div>
<div style="text-align: justify;">
You can get a feel for what SURE has to offer by checking out <a href="https://ir.canterbury.ac.nz/handle/10092/16670">this paper</a> by Nick Huntington-Klein and Andrew Gill that they are publishing.</div>
<div style="text-align: justify;">
<br /></div>
<div style="text-align: justify;">
We'll all be looking forward to more excellent papers like this!</div>
<div style="text-align: justify;">
<br /></div>
<center>
© 2019, David E. Giles
</center>
</div>
Dave Gileshttp://www.blogger.com/profile/05389606956062019445noreply@blogger.com0tag:blogger.com,1999:blog-2198942534740642384.post-35941466601637768302019-05-01T09:00:00.004-07:002019-05-02T12:37:06.305-07:00May Reading List<div dir="ltr" style="text-align: left;" trbidi="on">
<div style="text-align: justify;">
Here's a selection of suggested reading for this month:</div>
<ul style="text-align: left;">
<li><a href="https://arxiv.org/pdf/1903.10075.pdf" style="text-align: justify;">Athey, S. & G. W. Imbens</a><span style="text-align: justify;">, 2019. Machine learning methods economists should know about. Mimeo.</span></li>
<li><a href="https://www.tandfonline.com/doi/full/10.1080/00031305.2019.1604432" style="text-align: justify;">Bhagwat, P. & E. Marchand, 2019.</a><span style="text-align: justify;"> On a proper Bayes but inadmissible estimator. </span><i style="text-align: justify;">American Statistician</i><span style="text-align: justify;">, online.</span></li>
<li><a href="https://www.tandfonline.com/doi/full/10.1080/00949655.2019.1602125" style="text-align: justify;">Canals, C. & A. Canals</a><span style="text-align: justify;">, 2019. When is n large enough? Looking for the right sample size to estimate proportions. </span><i style="text-align: justify;">Journal of Statistical Computation and Simulation</i><span style="text-align: justify;">, 89, 1887-1898.</span></li>
<li><a href="https://www.economics.ku.dk/research/publications/wp/dp_2019/1903.pdf" style="text-align: justify;">Cavaliere, G. & A. Rahbek</a><span style="text-align: justify;">, 2019. A primer on bootstrap testing of hypotheses in time series models: With an application to double autoregressive models. Discussion Paper 19-03, Department of Economics, University of Copenhagen.</span></li>
<li><a href="https://www.dallasfed.org/~/media/documents/institute/wpapers/2019/0356.pdf" style="text-align: justify;">Chudik, A. & G. Geogiardis, 2019</a><span style="text-align: justify;">. Estimation of impulse response functions when shocks are observed at a higher frequency than outcome variables. Globalization Institute Working Paper 356, Federal Reserve Bank of Dallas.</span></li>
<li style="text-align: justify;"><a href="https://www.tandfonline.com/doi/full/10.1080/03610918.2017.1408826">Reschenhofer, E.</a>, 2019. Heteroscedasticity-robust estimation of autocorrelation. <i>Communications in Statistics - Simulation and Computation</i>, 48, 1251-1263.</li>
</ul>
<div>
</div>
<center style="text-align: justify;">
</center>
<center style="text-align: justify;">
</center>
<center style="text-align: center;">
© 2019, David E. Giles
</center>
</div>
Dave Gileshttp://www.blogger.com/profile/05389606956062019445noreply@blogger.com2tag:blogger.com,1999:blog-2198942534740642384.post-31567498008259575082019-04-29T17:50:00.003-07:002019-05-02T12:38:49.120-07:00Recursions for the Moments of Some Continuous Distributions<div dir="ltr" style="text-align: left;" trbidi="on">
<div dir="ltr" style="text-align: left;" trbidi="on">
<div dir="ltr" style="text-align: left;" trbidi="on">
<div dir="ltr" style="text-align: left;" trbidi="on">
<div dir="ltr" style="text-align: left;" trbidi="on">
<div dir="ltr" style="text-align: left;" trbidi="on">
<div dir="ltr" style="text-align: left;" trbidi="on">
<div dir="ltr" style="text-align: left;" trbidi="on">
<div dir="ltr" style="text-align: left;" trbidi="on">
<div dir="ltr" style="text-align: left;" trbidi="on">
<div dir="ltr" style="text-align: left;" trbidi="on">
<div dir="ltr" style="text-align: left;" trbidi="on">
<div dir="ltr" style="text-align: left;" trbidi="on">
<div dir="ltr" style="text-align: left;" trbidi="on">
<div dir="ltr" style="text-align: left;" trbidi="on">
<div style="text-align: justify;">
<span style="font-family: inherit;">This post follows on from my recent one, <i><a href="https://davegiles.blogspot.com/2019/04/recursions-for-moments-of-some-discrete.html">Recursions for the Moments of Some Discrete Distributions</a></i>. I'm going to assume that you've read the previous post, so this one will be shorter. </span></div>
<div style="text-align: justify;">
<span style="font-family: inherit;"><br /></span></div>
<div style="text-align: justify;">
<span style="font-family: inherit;">What I'll be discussing here are some useful recursion formulae for computing the moments of a number of continuous distributions that are widely used in econometrics. The coverage won't be exhaustive, by any means. I provide some motivation for looking at formulae such as these in the previous post, so I won't repeat it here. </span></div>
<div style="text-align: justify;">
<span style="font-family: inherit; font-weight: 700;"><br /></span></div>
<div style="text-align: justify;">
<span style="font-family: inherit;">When we deal with the Normal distribution, below, we'll make explicit use of Stein's Lemma. Several of the other results are derived (behind the scenes) by using a very similar approach. So, let's begin by stating this Lemma.</span><br />
<span style="font-family: inherit;"><br /></span>
<b><span style="font-family: inherit;">Stein's Lemma (Stein, 1973):</span></b></div>
<div style="text-align: justify;">
<span style="font-family: inherit;"><b></b></span></div>
<div style="color: black; font-style: normal; font-weight: 400; letter-spacing: normal; text-align: justify; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px;">
</div>
<span style="font-family: inherit;"><span style="font-family: inherit;"><br /></span>
</span><br />
<div style="color: black; font-style: normal; font-weight: 400; letter-spacing: normal; text-align: justify; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px;">
<div style="margin: 0px;">
<span style="font-family: inherit;">"If <i>X</i> ~ N[<i>θ</i> , <i>σ</i><sup>2</sup>], and if <i>g</i>(.) is a differentiable function such that E|<i>g</i>'(<i>X</i>)| is finite, then </span></div>
<div style="margin: 0px;">
<span style="font-family: inherit;"><br /></span></div>
<div style="margin: 0px;">
<span style="font-family: inherit;"> E[<i>g</i>(<i>X</i>)(<i>X</i> - <i>θ</i>)] = <i>σ</i><sup>2</sup> E[<i>g</i>'(<i>X</i>)]."</span></div>
</div>
<div style="text-align: justify;">
<span style="font-family: inherit;"><br /></span></div>
<div style="text-align: justify;">
<span style="font-family: inherit;">It's worth noting that although this lemma relates to a single <b style="font-style: italic;">Normal</b> random variable, in the bivariate Normal case the lemma generalizes to:</span><br />
<span style="font-family: inherit;"><br /></span>
<br />
<div style="color: black; font-weight: 400; letter-spacing: normal; text-align: justify; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px;">
<div style="margin: 0px;">
<div style="margin: 0px;">
<span style="font-family: inherit;">"If <i>X</i><i style="font-style: normal;"> </i>and <i>Y</i> follow a bivariate Normal distribution, and if <i style="font-style: normal;">g</i>(.) is a differentiable function such that E|<i style="font-style: normal;">g</i>'(<i>Y</i>)| is finite, then </span></div>
</div>
<div style="font-style: normal; margin: 0px;">
<div style="margin: 0px;">
<span style="font-family: inherit;"><br /></span></div>
</div>
<div style="margin: 0px;">
<div style="margin: 0px;">
<span style="font-family: inherit;"> Cov.[<i style="font-style: normal;">g</i>(<i>Y </i>)<i>, </i><i>X</i>] = Cov.(<i>X</i> , <i>Y</i>) E[<i style="font-style: normal;">g</i>'(<i>Y</i>)]."</span></div>
</div>
</div>
<span style="font-family: inherit;"><br /></span>
<span style="font-family: inherit;">In this latter form, the lemma is useful in asset pricing models.</span><br />
<span style="font-family: inherit;"><br /></span>
<span style="font-family: inherit;">There are extensions of Stein's Lemma to a broader class univariate and multivariate distributions. For example, see Alghalith (undated), and Landsman <i>et al</i>. (2013), and the references in those papers. Generally, if a distribution belongs to an <a href="https://en.wikipedia.org/wiki/Exponential_family">exponential family</a>, then recursions for its moments can be obtained quite easily.</span></div>
<div style="text-align: justify;">
<span style="font-family: inherit;"><br /></span>
<span style="font-family: inherit;">Now, let's get down to business............</span><br />
<span style="font-family: inherit;"><br /></span>
<br />
<a name='more'></a><span style="font-family: inherit;"><span style="background-color: white;">Recall that t</span><span style="background-color: white; color: #444444;">he <em>r</em><sup>th.</sup> "raw moment" (or "moment about zero") for the random variable <em>X</em>, with distribution function <em>F</em>,<em> </em>is defined as </span><em style="background-color: white; color: #444444;">μ<sub>r</sub></em><span style="background-color: white; color: #444444;">' = E[</span><em style="background-color: white; color: #444444;">X <sup>r</sup></em><span style="background-color: white; color: #444444;">] </span><span style="background-color: white; color: #444444;">= ∫ </span><em style="background-color: white; color: #444444;">x<sup>r</sup>dF</em><span style="background-color: white; color: #444444;">(</span><em style="background-color: white; color: #444444;">x</em><span style="background-color: white; color: #444444;">) , for <i>r</i> = 1, 2, .....; and the </span><span style="color: #444444;">"central (centered) moments" of <i>X </i>are defined as </span><em style="color: #444444;">μ<sub>r</sub></em><span style="background-color: white; color: #444444;"> = E[(</span><em style="color: #444444;">X - </em><span style="color: #444444;"><i>μ</i><sub>1</sub></span><span style="background-color: white; color: #444444;">' </span><span style="color: #444444;">)</span><em style="color: #444444;"><sup>r</sup></em><span style="background-color: white; color: #444444;">] , for <i>r</i> = 1, 2, 3, ....... </span></span><br />
<span style="font-family: inherit;"><span style="background-color: white; color: #444444;"><br /></span>
Also, we can express one of these sets of moments in terms of the other set by using the two relationships:</span><br />
<span style="font-family: inherit;"><br /></span>
<span style="font-family: inherit;"> <i>μ<sub>r</sub></i> = Σ{[<i>r</i>! / (<i>i</i>! (<i>r</i> - <i>i</i>)!)] <i>μ<sub>i</sub></i>' (- <i>μ</i><sub>1</sub><i>' </i>) <sup><i>r-i </i></sup>} , (1)</span><br />
<span style="font-family: inherit;"><br /></span>
<span style="font-family: inherit;">where the range of summation is from <i>i</i> = 0 to <i>i</i> = <i>r </i>; and</span><br />
<span style="font-family: inherit;"><br /></span>
<span style="font-family: inherit;"> <i>μ<sub>r</sub></i>' = Σ{[<i>r</i>! / (<i>i</i>! (<i>r</i> - <i>i</i>)!)] <i>μ<sub>i</sub></i><sup><i> </i></sup>(<i>μ</i><sub>1</sub><i>' </i>) <sup><i>r-i </i></sup>} , (2)</span><br />
<span style="font-family: inherit;"><br /></span>
<span style="font-family: inherit;">where, again, the range of summation is from <i>i</i> = 0 to <i>i</i> = <i>r</i>.</span><br />
<span style="font-family: inherit;"><span style="background-color: white; color: #444444;"><br /></span>
<b><b><span style="color: blue;">Normal distribution</span></b></b></span></div>
<span style="font-family: inherit; text-align: justify;">If </span><i style="font-family: inherit; text-align: justify;">X</i><span style="font-family: inherit; text-align: justify;"> ~ N[</span><i style="font-family: inherit; text-align: justify;">θ</i><span style="font-family: inherit; text-align: justify;"> , </span><i style="font-family: inherit; text-align: justify;">σ</i><sup style="font-family: inherit; text-align: justify;">2</sup><span style="font-family: inherit; text-align: justify;">], then we know that </span><i style="font-family: inherit; text-align: justify;">μ</i><sub style="font-family: inherit; text-align: justify;">1</sub><span style="font-family: inherit; text-align: justify;">' = E[</span><i style="font-family: inherit; text-align: justify;">X</i><span style="font-family: inherit; text-align: justify;">] = </span><i style="font-family: inherit; text-align: justify;">θ</i><span style="font-family: inherit; text-align: justify;">, and </span><i style="font-family: inherit; text-align: justify;">μ</i><sub style="font-family: inherit; text-align: justify;">2</sub><span style="font-family: inherit; text-align: justify;">' = E[</span><i style="font-family: inherit; text-align: justify;">X </i><sup style="font-family: inherit; text-align: justify;">2</sup><span style="font-family: inherit; text-align: justify;">] = </span><i style="font-family: inherit; text-align: justify;">σ</i><sup style="font-family: inherit; text-align: justify;">2</sup><span style="font-family: inherit; text-align: justify;"> + </span><i style="font-family: inherit; text-align: justify;">θ</i><sup style="font-family: inherit; text-align: justify;">2</sup><span style="font-family: inherit; text-align: justify;">.</span><br />
<div style="text-align: justify;">
<span style="font-family: inherit;"><br /></span></div>
<div style="text-align: justify;">
<span style="font-family: inherit;">Also, note that we can write:</span><br />
<span style="font-family: inherit;"><span style="text-align: left;"><br /></span><span style="text-align: left;"><i> μ</i><sub>3</sub>' = E[<i>X </i></span><sup style="text-align: left;">3</sup><span style="text-align: left;">] = E[<i>X </i></span><sup style="text-align: left;">2</sup><span style="text-align: left;">(<i>X</i> - <i>θ</i> + <i>θ</i>)] = E[<i>X </i></span><sup style="text-align: left;">2</sup><span style="text-align: left;">(<i>X</i> - <i>θ</i>)] + <i>θ </i>E[<i>X </i></span><sup style="text-align: left;">2</sup><span style="text-align: left;">] .</span></span></div>
</div>
<div style="margin: 0px;">
<span style="font-family: inherit;"><span style="text-align: justify; vertical-align: sub;">Applying Stein's Lemma, with <i>g</i>(<i>X</i>) = <i>X </i></span><sup style="font-size: medium; text-align: justify;">2</sup><span style="text-align: justify; vertical-align: sub;">, we obtain:</span><sub></sub></span><br />
<span style="font-family: inherit;"><span style="font-family: "times new roman"; font-size: small;"></span><br /></span>
<br />
<div style="text-align: justify;">
<span style="font-family: inherit;"> <i>μ</i><sub>3</sub>' = 2<i>σ</i><sup>2 </sup>E[<i>X</i>] + <i>θ</i> E[<i>X </i><sup>2</sup>] = <i>θμ</i><sub>2</sub>' + 2<i>σ<sup>2</sup>μ</i><sub>1</sub>' = <i>θ</i><sup>3</sup> + 3<i>θ</i><i>σ</i><sup>2</sup>.</span><br />
<span style="font-family: inherit;"><span style="font-family: "times new roman";"><br /></span>
<span style="font-family: inherit;">Similarly, if we apply the Lemma with <i>g</i>(<i>X</i>) = <i>X <sup>k</sup></i> ; <i>k</i> =3, 4, ......, we obtain:</span></span><br />
<span style="font-family: inherit;"><span style="font-family: "times new roman";"><br /></span>
<span style="font-family: "times new roman";"> <i>μ</i><sub>4</sub>' = E[<i>X </i><sup>4</sup>] = E[<i>X </i><sup>3</sup>(<i>X</i> -<i> θ</i>)] + <i>θ </i>E[<i>X </i><sup>3</sup>] = <i>θμ</i><sub>3</sub>' + 3<i>σ</i><sup>2</sup><i>μ</i><sub>2</sub>' = <i>θ</i><sup>4</sup> + 6<i>θ</i><sup>2</sup><i>σ</i><sup>2</sup> + 3<i>σ</i><sup>4</sup></span><span style="font-family: "times new roman";">.</span></span><br />
<span style="font-family: inherit;"><span style="font-family: "times new roman";"><br /></span>
<span style="font-family: "times new roman";"> <i>μ</i><sub>5</sub>' = E[<i>X </i><sup>5</sup>] = E[<i>X </i><sup>4</sup>(<i>X</i> - <i>θ</i>)] + <i>θ </i>E[<i>X </i><sup>4</sup>] = <i>θμ</i><sub>4</sub>' + 4<i>σ</i><sup>2</sup><i>μ</i><sub>3</sub>' = <i>θ</i><sup>5</sup> + 10<i>θ</i><sup>3</sup><i>σ</i><sup>2</sup> + 15<i>θσ</i><sup>4</sup>;</span></span><br />
<span style="font-family: inherit;"><span style="font-family: "times new roman";"><br /></span>
<span style="font-family: "times new roman";">and so on.</span></span><br />
<span style="font-family: inherit;"><span style="font-family: "times new roman";"><br /></span>
<span style="font-family: inherit;">So, each moment is obtained recursively from all of the lower-order moments, through the repeated use of Stein's Lemma.</span></span><br />
<span style="font-family: inherit;"><span style="font-family: "times new roman";"><br /></span>
<span style="font-family: inherit;"><span style="font-family: "times new roman";">More particularly, you can see immediately </span><span style="font-family: "times new roman";">that following recursion formula holds for the raw moments of the Normal distribution (Bain, 1969; Willink, 2005):</span></span></span><br />
<span style="font-family: inherit;"><br /></span></div>
<span style="font-family: inherit;"><span style="font-family: "times new roman"; font-size: small;"></span> <b><span style="color: #cc0000;"><i>μ</i><sub><i>r</i>+1</sub>' = <i>θ μ<sub>r</sub></i>' + <i>rσ</i><sup>2 </sup><i>μ</i><sub><i>r</i>-1</sub>' ; <i>r</i> = 1, 2, 3, .........</span></b></span><br />
<span style="font-family: inherit;"><br /></span>
<b style="text-align: justify;"><span style="color: blue; font-family: inherit;">Chi-square distribution</span></b><br />
<div style="text-align: justify;">
<div style="text-align: justify;">
<span style="font-family: inherit;"><br /></span></div>
<div style="text-align: justify;">
<span style="font-family: inherit;">We'll use the following standard result for the Chi-square distribution with "<i>v</i>" degrees of freedom:</span><br />
<span style="font-family: inherit;"><br /></span>
<span style="font-family: inherit;">"For any real-valued function, <i>h</i>, E[<i>h</i>(<i>χ</i><sup>2</sup><sub><i>v</i></sub>)] = <i>v</i> E[<i>h</i>(<i>χ</i><sup>2</sup><sub><i>v</i>+2</sub>) / <i>χ</i><sup>2</sup><sub><i>v</i>+2</sub>], provided that the expectations exist."</span></div>
</div>
</div>
</div>
</div>
<div style="text-align: justify;">
<span style="font-family: inherit;"><span style="vertical-align: super;"><br /></span>
<span style="vertical-align: super;">(This result can be generalized to the case of a non-central Chi-square distribution. See Appendix B of Judge and Bock (1978).</span></span></div>
<div style="text-align: justify;">
<span style="font-family: inherit; font-size: 13.3333px;"><br /></span></div>
<div style="text-align: justify;">
<span style="font-family: inherit;"><span style="vertical-align: super;">Let </span><i style="vertical-align: super;">h</i><span style="vertical-align: super;">(</span><i style="vertical-align: super;">x</i><span style="vertical-align: super;">) = <i>x</i></span><i style="vertical-align: super;"><sup>k</sup></i><span style="vertical-align: super;">, for some integer, <i>k</i>. Then, applying the above result repeatedly, we get:</span></span></div>
<ul style="text-align: left;">
<li style="text-align: justify;"><span style="font-family: inherit;">If <i>k</i> = 1, then <i>μ</i><sub>1</sub>' = E[<i>h</i>(<i>χ</i><sup>2</sup><sub><i>v</i></sub>)] = <i>v</i> E[<i>χ</i><sup>2</sup><sub><i>v</i>+2</sub> / <i>χ</i><sup>2</sup><sub><i>v</i>+2</sub>] = <i>v .</i></span></li>
<li style="text-align: justify;"><span style="font-family: inherit;">If <i>k</i> = 2, then <i>μ</i><sub>2</sub>' = E[(<i>χ</i><sup>2</sup><sub><i>v</i></sub>)2] = <i>v</i> E[(<i>χ</i><sup>2</sup><sub><i>v</i>+2</sub>)<sup>2</sup> / <i>χ</i><sup>2</sup><sub><i>v</i>+2</sub>] = <i>v</i>E[<i>χ</i><sup>2</sup><sub><i>v</i>+2</sub>] = <i>v</i>(<i>v</i> + 2).</span></li>
</ul>
<div style="text-align: justify;">
<span style="font-family: inherit;">Immediately, we see that <i>μ</i><sub>2</sub> = Var.[<i>χ</i><sup>2</sup><sub><i>v</i></sub><span style="text-align: left;">] = <i>μ</i><sub>2</sub>' - (<i>μ</i><sub>1</sub>')<sup>2</sup> = 2<i>v</i>.</span></span></div>
<div style="text-align: justify;">
<ul>
<li><span style="font-family: inherit;"><span style="text-align: left;">If <i>k</i> = 3, then <i>μ</i><sub>3</sub>' = </span>E[(<i>χ</i><sup>2</sup><sub><i>v</i></sub>)<sup>3</sup>] = <i>v</i> E[(<i>χ</i><sup>2</sup><sub><i>v</i>+2</sub>)<sup>3</sup> / <i>χ</i><sup>2</sup><sub><i>v</i>+2</sub>] = <i>v</i>E[(<i>χ</i><sup>2</sup><sub><i>v</i>+2</sub>)<sup>2</sup>] = <i>v</i>(<i>v</i> + 2)(<i>v</i> + 4) .</span></li>
</ul>
</div>
<div style="font-size: medium;">
</div>
<span style="font-family: inherit;">I'll bet that you can see right away what the expressions are for <i>μ</i><sub>4</sub>', <i>μ</i><sub>5</sub>', <i>etc</i>.!</span><br />
<span style="font-family: inherit;"><br /></span>
<span style="font-family: inherit;">In terms of a genuine recurrence relationship, we see from the above that we can write:</span><br />
<span style="font-family: inherit;"><b style="text-align: justify;"><span style="color: blue;"><br /></span></b>
</span><br />
<div dir="ltr" style="text-align: left;" trbidi="on">
<div dir="ltr" style="text-align: left;" trbidi="on">
<div dir="ltr" style="text-align: left;" trbidi="on">
<div dir="ltr" style="text-align: left;" trbidi="on">
<div dir="ltr" style="text-align: left;" trbidi="on">
<span style="font-family: inherit;"> <span style="color: #cc0000;"> <b><i>μ<sub>r</sub></i>' = <i>μ</i><sub><i>r</i>-1</sub>' (<i>v</i> + 2(<i>r</i> -1)) ; <i>r</i> = 1, 2, 3, ........</b></span></span><br />
<div style="text-align: justify;">
<span style="font-family: inherit;"><br /></span></div>
<div style="text-align: justify;">
<span style="font-family: inherit;">Although we can then use equation (1) to obtain the <i>central</i> moments of <i>X</i>, there's also a separate recursion formula for these moments in the case of the Chi-square distribution This is discussed in the section on the Gamma distribution, below.</span></div>
</div>
</div>
</div>
</div>
</div>
<span style="font-family: inherit;"><b style="text-align: justify;"><span style="color: blue;"><br /></span></b>
<b style="text-align: justify;"><span style="color: blue;">Student-t distribution</span></b></span></div>
</div>
</div>
</div>
</div>
<div style="text-align: justify;">
<span style="font-family: inherit;"><br /></span>
<span style="font-family: inherit;">Suppose that <i>X</i> follows a Student-t distribution, with <i>v</i> degrees of freedom. Then the moments of <i>X</i> can be summarized as follows:</span><br />
<span style="font-family: inherit;"><br /></span>
<span style="font-family: inherit;"><i>μ<sub>r</sub></i>' = 0 ; if <i>r</i> is <b>odd</b>, and 0 < <i>r</i> < <i>v</i></span><br />
<span style="font-family: inherit;"><br /></span>
<span style="font-family: inherit;"><i>μ<sub>r</sub></i>' = <i>v</i><sup>(<i>r</i>/2)</sup> Π[(2<i>i</i> - 1) / (<i>v</i> - 2<i>i</i>)] ; if <i>r</i> is <b>even</b>, and 0 < <i>r</i> < <i>v </i>(3)</span><br />
<span style="font-family: inherit;"><br /></span>
<span style="font-family: inherit;">where the product is for <i>i</i> = 1 to <i>i</i> = (<i>r</i> / 2).</span><br />
<span style="font-family: inherit;"><br /></span>
<span style="font-family: inherit;">The inequality, <i>v</i> > <i>r</i>, ensures that the moment "exists" - that is, it is finite. If <i>v</i> = 1, then <i>X</i> follows a Cauchy distribution, and <i>none</i> of its moments "exist".</span></div>
<div style="text-align: justify;">
<span style="font-family: inherit;"><br /></span>
<span style="font-family: inherit;">Equation (3) gives us a nice, tidy, and direct formula for obtaining any of the even-order moments of <i>X</i>. For instance, we see that:</span><br />
<span style="font-family: inherit;"><br /></span>
<span style="font-family: inherit;"><i>μ</i><sub>2</sub>' = <i>v</i> [(2- 1) / (<i>v</i> - 2)] = <i>v</i> / (<i>v</i> - 2) ; if <i>v</i> > 2. </span><br />
<span style="font-family: inherit;"><br /></span>
<span style="font-family: inherit;">Obviously, <i>μ</i><sub>2</sub> = Var.[<i>X</i>] = <i>μ</i><sub>2</sub><span style="text-align: left;">', because E[<i>X</i>] = 0. </span></span></div>
<span style="font-family: inherit;"><br /></span>
<span style="font-family: inherit;">Also,</span><br />
<span style="font-family: inherit;"><br /></span>
<span style="font-family: inherit;"><i>μ</i><sub>4</sub>' = <i style="text-align: justify;">v</i><span style="text-align: justify;"><sup>2</sup></span><span style="text-align: justify;"> [1 / (</span><i style="text-align: justify;">v</i><span style="text-align: justify;"> - 2)][3 / (<i>v</i> - 4)] = 3</span><i style="text-align: justify;">v</i><span style="text-align: justify;"><sup>2</sup></span><span style="text-align: justify;"> / [(</span><i style="text-align: justify;">v</i><span style="text-align: justify;"> - 2)(<i>v</i> - 4)] ; if </span><i style="text-align: justify;">v</i><span style="text-align: justify;"> > 4. </span></span><br />
<span style="font-family: inherit;"><i>μ</i><sub>6</sub>' = <i style="text-align: justify;">v</i><span style="text-align: justify;"><sup>3</sup></span><span style="text-align: justify;"> [1 / (</span><i style="text-align: justify;">v</i><span style="text-align: justify;"> - 2)][3 / (<i>v</i> - 4)][5 / (<i>v</i> - 6)] = 15</span><i style="text-align: justify;">v</i><span style="text-align: justify;"><sup>3</sup></span><span style="text-align: justify;"> / [(</span><i style="text-align: justify;">v</i><span style="text-align: justify;"> - 2)(<i>v</i> - 4)(<i>v</i> - 6)] ; if </span><i style="text-align: justify;">v</i><span style="text-align: justify;"> > 6.</span></span><br />
<div style="text-align: justify;">
<span style="font-family: inherit;"><br /></span>
<i><span style="font-family: inherit;">etc.</span></i><br />
<i><span style="font-family: inherit;"><br /></span></i></div>
<div style="text-align: justify;">
<span style="font-family: inherit;">This set of results isn't in the form of a recurrence relationship, but we can obtain one very easily. Note that:</span></div>
<div style="text-align: justify;">
<span style="font-family: inherit;"><br /></span></div>
<span style="font-family: inherit;"><i>μ</i><sub>4</sub>' = [<i style="text-align: justify;">v</i><span style="text-align: justify;"> / (</span><i style="text-align: justify;">v</i><span style="text-align: justify;"> - 2)][3<i>v</i> / (<i>v</i> - 4)] = </span><i style="text-align: justify;">μ</i><sub style="text-align: justify;">2</sub><span style="text-align: justify;">' [3<i>v</i> / (<i>v</i> - 4)] ; </span><span style="text-align: justify;">if </span><i style="text-align: justify;">v</i><span style="text-align: justify;"> > 4. </span></span><br />
<div>
<span style="font-family: inherit; text-align: justify;"><br /></span></div>
<span style="font-family: inherit;"><i>μ</i><sub>6</sub>' = [<i style="text-align: justify;">v</i><span style="text-align: justify;"> / (</span><i style="text-align: justify;">v</i><span style="text-align: justify;"> - 2)][3<i>v</i> / (<i>v</i> - 4)][5<i>v</i> / (<i>v </i>- 6)] = </span><span style="text-align: justify;"> </span><i>μ</i><sub>4</sub>' <span style="text-align: justify;">[5</span><i style="text-align: justify;">v</i><span style="text-align: justify;"> / (</span><i style="text-align: justify;">v</i><span style="text-align: justify;"> - 6)] </span><span style="text-align: justify;"> ; </span><span style="text-align: justify;">if </span><i style="text-align: justify;">v</i><span style="text-align: justify;"> > 6,</span></span><br />
<span style="font-family: inherit;"><span style="text-align: justify;"><br /></span>
<span style="text-align: justify;">and so on.</span></span><br />
<span style="font-family: inherit;"><br /></span>
<span style="font-family: inherit;">Clearly, in general, the following recurrence formula holds -</span><br />
<span style="font-family: inherit;"><br /></span>
<span style="font-family: inherit;"><i> <span style="color: #cc0000;"> <b>μ</b></span></i><b><span style="color: #cc0000;"><sub><i>r</i></sub>' = <i>μ</i><sub>r-2</sub>' [(<i>r</i> - 1) / (<i>v</i> - <i>r</i>)] ; if <i>r</i> is <span style="color: red;">even</span>, and 0 < <i>r</i> < <i>v</i>.</span></b></span><br />
<span style="font-family: inherit;"><br /></span>
<span style="font-family: inherit;">Trust me - this formula is <i><b>much more </b></i>appealing than integrating functions of the density, or working with the characteristic function, for this particular distribution.</span><br />
<span style="font-family: inherit;"><br /></span>
<span style="font-family: inherit;">Finally, note that because E[<i>X</i>] = <i>μ</i><sub>1</sub>' = 0, <i>μ</i><sub>r</sub> = <i>μ</i><sub>r</sub>', for all (even) <i>r</i>.</span><br />
<span style="font-family: inherit;"><br /></span>
<b style="text-align: left;"><span style="color: blue; font-family: inherit;">F distribution</span></b><br />
<span style="font-family: inherit;"> </span><br />
<span style="font-family: inherit;">Suppose that <i>X</i> follows an F distribution, with numerator and denominator degrees of freedom, <i>v</i><sub>1</sub> and <i>v</i><sub>2</sub> respectively.</span><br />
<span style="font-family: inherit;"><br /></span>
<span style="font-family: inherit; text-align: left;">The general formula for the raw moments of <i>X</i> is:</span><br />
<span style="font-family: inherit;"><span style="text-align: left;"><br /></span>
<span style="text-align: left;"><i>μ<sub>r</sub></i>' = </span>(<i>v</i><sub>2</sub> / <i>v</i><sub>1</sub>) <sup><i>r</i></sup> Γ[(<i>v</i><sub>1</sub> / 2) + <i>r</i>] Γ ([<i>v</i><sub>2</sub> / 2) - <i>r</i>] / (Γ [<i>v</i><sub>1</sub> / 2] Γ[<i>v</i><sub>2</sub> / 2]) . (4)</span></div>
</div>
</div>
<span style="font-family: inherit;"><br /></span>
<span style="font-family: inherit; text-align: left;">So,</span><br />
<span style="font-family: inherit;"><span style="text-align: left;"><br /></span><span style="text-align: left;"><i>μ</i><sub>1</sub>' = </span>(<i>v</i><sub>2</sub> / <i>v</i><sub>1</sub>)Γ[(<i>v</i><sub>1</sub> / 2) + 1] Γ ([(<i>v</i><sub>2</sub> / 2) - 1] / (Γ [<i>v</i><sub>1</sub> / 2] Γ[<i>v</i><sub>2</sub> / 2] .</span><br />
<span style="font-family: inherit;"><span style="text-align: left;"><br /></span>
<span style="text-align: left;">Using the result that Γ[<i>x</i> + 1] = <i>x</i>Γ[<i>x</i>], and cancelling terms, this simplifies to</span></span><br />
<span style="font-family: inherit;"><span style="text-align: left;"><br /></span>
<span style="text-align: left;"><i>μ</i><sub>1</sub>' = <i>v</i><sub>2</sub> / (</span><i>v</i><sub>2</sub> - 2) ; if <i>v</i><sub>2</sub> > 2</span><br />
<span style="font-family: inherit;"><span style="text-align: left;"><br /></span>
<span style="text-align: left;">Similarly,</span></span><br />
<span style="font-family: inherit;"><span style="text-align: left;"><br /></span>
<span style="text-align: left;"><i>μ</i><sub>2</sub>' = (<i>v</i><sub>2</sub> / <i>v</i><sub>1</sub>)<sup>2</sup> Γ[(<i>v</i><sub>1</sub> / 2) + 2] Γ ([(<i>v</i><sub>2</sub> / 2) - 2] / (Γ [<i>v</i><sub>1</sub> / 2] Γ[<i>v</i><sub>2</sub> / 2].</span></span></div>
<span style="font-family: inherit;"><br /></span>
<span style="font-family: inherit;"><span style="text-align: left;">Again, noting that </span>Γ[(<i>v</i><sub>1</sub> / 2) + 2] = ((<i>v</i><sub>1</sub> / 2) + 1)Γ[(<i>v</i><sub>1</sub> / 2) + 1]; that Γ[(<i>v</i><sub>2</sub> / 2) - 2] = Γ[(<i>v</i><sub>2</sub> / 2) - 1] / ((<i>v</i><sub>2</sub> / 2) - 2); and then simplifying, we get:</span><br />
<span style="font-family: inherit;"><span style="text-align: left;"><br /></span><span style="text-align: left;"><i>μ</i><sub>2</sub>' = </span><i>μ</i><sub>1</sub>' (<i>v</i><sub>2</sub> /<i>v</i><sub>1</sub>)[(<i>v</i><sub>1</sub> + 2) / (<i>v</i><sub>2</sub> - 4)] ; if <i>v</i><sub>2</sub> > 4</span><br />
<span style="font-family: inherit;"><span style="text-align: left;"><br /></span>
<span style="text-align: left;">Proceeding in a similar manner, we find that:</span></span><br />
<span style="font-family: inherit;"><span style="text-align: left;"><br /></span><span style="text-align: left;"><i>μ</i><sub>3</sub>' = </span><i>μ</i><sub>2</sub>' (<i>v</i><sub>2 </sub>/ <i>v</i><sub>1</sub>)[(<i>v</i><sub>1</sub> + 4) / (<i>v</i><sub>2</sub> - 6)] ; if <i>v</i><sub>2</sub> > 6 </span><br />
<span style="font-family: inherit;"><i><br /></i>
<i>μ</i><sub>4</sub>' = <i>μ</i><sub>1</sub>' (<i>v</i><sub>2</sub> / <i>v</i><sub>1</sub>)[(<i>v</i><sub>1</sub> + 6) / (<i>v</i><sub>2</sub> - 8)] ; if <i>v</i><sub>2</sub> > 8</span><br />
<span style="font-family: inherit;"><span style="text-align: left;"><br /></span>
<span style="text-align: left;"><i>etc.</i></span></span><br />
<span style="font-family: inherit;"><span style="text-align: left;"><br /></span>
<span style="text-align: left;">So, in general you can see that we have the following recursion formula for the moments of <i>X</i>:</span></span><br />
<span style="font-family: inherit;"><br /></span>
<span style="font-family: inherit;"><span style="text-align: left;"> <span style="color: #cc0000;"> <b><i>μ</i><sub><i>r</i>+1</sub>' = <i>μ<sub>r</sub></i>' (<i>v</i><sub>2</sub> / <i>v</i><sub>1</sub>)[(<i>v</i><sub>1</sub> + 2<i>r</i>) / (<i>v</i><sub></sub>2 - 2(<i>r</i> + 1))] ; <i>r </i>= 0, 1, 2, .............; if </b></span></span><b><span style="color: #cc0000;"><i>v</i>2 > 2(r + 1)</span>.</b></span><br />
<span style="font-family: inherit;"><br /></span>
<br />
<div style="text-align: justify;">
<span style="font-family: inherit; text-align: left;">This recurrence relationship avoids us having to deal with the gamma functions in (4), let alone having to perform any integration, or deal with this distribution's messy characteristic function. (As with the Student-t distribution, the moment generating function isn't defined here, because of the above conditions on the existence of the moments.)</span><br />
<span style="font-family: inherit;"><span style="text-align: left;"><br /></span>
The distributions that we've considered so far are ones that you use every day in your econometrics work. Now let's consider a couple more distributions that arise a bit less frequently.</span><br />
<div>
<span style="font-family: inherit;"><br /></span></div>
</div>
<b><span style="color: blue; font-family: inherit;">Gamma distribution</span></b><br />
<div style="text-align: justify;">
<span style="font-family: inherit;"><br /></span></div>
<div style="text-align: justify;">
<span style="font-family: inherit;">One situation where the Gamma distribution comes up in econometrics is in the context of "count data". This might seem a bit odd, because count data are non-negative <i>integers</i>, and we're talking about <i>continuous</i> random variables here.</span></div>
<div style="text-align: justify;">
<span style="font-family: inherit;"><br /></span></div>
<div style="text-align: justify;">
<span style="font-family: inherit;">However, the Gamma distribution comes into play when we generalize the Poisson distribution to a particular form of the Negative Binomial distribution. Both of these distributions were discussed in my <a href="https://davegiles.blogspot.com/2019/04/recursions-for-moments-of-some-discrete.html">previous, related, post</a>. Looking back at that post, you'll be able to see that the variance of a Poisson random variable equals its mean. We say that the distribution is "equi-dispersed". This is very restrictive, and isn't realistic with a lot of count data in practice. On the other hand, the variance of the Negative Binomial distribution <i>exceeds</i> its mean. We say that this distribution is "over-dispersed", and in practice this is often more reasonable.</span></div>
<div style="text-align: justify;">
<span style="font-family: inherit;"><br /></span></div>
<div style="text-align: justify;">
<span style="font-family: inherit;">To construct the Negative Binomial distribution in the form that we usually use it in econometrics, we take a Poisson random variable and then add in an unobserved random effect to its (conditional) mean. If this random effect follows a Gamma distribution, we end up with the Negative Binomial distribution for the count data. (See Greene, 2012, pp.806-807 for details.)</span></div>
<div style="text-align: justify;">
<span style="font-family: inherit;"><br /></span></div>
<div style="text-align: justify;">
<span style="font-family: inherit;">With this by way of motivation, consider a random variable, <i>X</i> that follows a Gamma distribution with a shape parameter '<i>a</i>', and a scale parameter '<i>b</i>'. (Be careful here - there are two forms of the Gamma distribution. The other one has the shape parameter '<i>a</i>' and the rate parameter, <i>θ</i> = 1 / <i>b</i>.)</span></div>
<div style="text-align: justify;">
<span style="font-family: inherit;"><br /></span></div>
<div style="text-align: justify;">
<span style="font-family: inherit;">Willink (2003) shows that the <b style="font-style: italic;">central moments </b>(<b><i>moments about the mean</i></b>) for <i>X</i> can be obtained from the following recursion -</span></div>
<div style="text-align: justify;">
<span style="font-family: inherit;"><br /></span></div>
<div style="text-align: justify;">
<span style="font-family: inherit;"> <b><span style="color: #cc0000;"><i>μ<sub>r</sub></i> = (<i>r</i> - 1)(<i>bμ</i><sub><i>r</i>-1</sub> + <i>ab</i><sup>2</sup><i>μ</i><sub><i>r</i>-2</sub>) ; <i>r</i> = 2, 3, ..........</span></b></span></div>
<div style="text-align: justify;">
<span style="font-family: inherit;"><br /></span></div>
<div style="text-align: justify;">
<span style="font-family: inherit;">There are two special cases of the Gamma distribution that we might note. First, if <i>a</i> = (<i>v</i> / 2), and <i>b</i> = 2, then <i>X</i> follows a Chi-square distribution with <i>v</i> degrees of freedom. So, the <b><i>central moments</i></b> of the Chi-square distribution follow the recursion relationship:</span></div>
<div style="text-align: justify;">
<span style="font-family: inherit;"><br /></span></div>
<div style="text-align: justify;">
<span style="font-family: inherit;"> <b><span style="color: #cc0000;"><i>μ<sub>r</sub></i> = 2(<i>r</i> - 1)(<i>μ</i><sub><i>r</i>-1</sub> + <i>vμ</i><sub><i>r</i>-2</sub>) ;<span style="text-align: left;"> </span><i style="text-align: left;">r</i><span style="text-align: left;"> = 2, 3, ..........</span></span></b></span></div>
<div style="text-align: justify;">
<span style="font-family: inherit;"><br /></span></div>
<div style="text-align: justify;">
<span style="font-family: inherit;">Second, if <i>a</i> = <i>b</i> = 1, then <i>X</i> follows an Exponential distribution, and its <i style="font-weight: bold;">central moments</i> satisfy:</span></div>
<div style="text-align: justify;">
<span style="font-family: inherit;"> </span></div>
<div style="text-align: justify;">
<span style="font-family: inherit;"> <span style="color: #cc0000;"><b><i>μ<sub>r</sub></i> = (<i>r</i> - 1)(<i>μ</i><sub><i>r</i>-1</sub> + <i>μ</i><sub><i>r</i>-2</sub>) </b> <b>; <i>r</i> = 2, 3, .........</b></span></span></div>
<div style="text-align: justify;">
<span style="font-family: inherit;"><br /></span></div>
<div style="text-align: justify;">
<span style="font-family: inherit;">In addition, Withers (1992) shows that for the Exponential distribution we have the following simpler recursion:</span></div>
<div style="text-align: justify;">
<span style="font-family: inherit;"><br /></span></div>
<div style="text-align: justify;">
<span style="font-family: inherit;"> <span style="color: #cc0000;"><b><i>μ<sub>r</sub></i> = <i>rμ</i><sub><i>r</i>-1</sub> + (-1)<sup><i>r</i></sup> </b> <b> ; <i>r</i> = 1, 2, 3, .......</b></span></span></div>
<div style="text-align: justify;">
<span style="font-family: inherit;"><br /></span></div>
<div style="text-align: justify;">
<b><span style="color: blue; font-family: inherit;">Beta distribution</span></b><br />
<span style="font-family: inherit;"><br /></span>
<span style="font-family: inherit;">One of the important features of the Beta distribution is that its density's support is the unit interval (although this can be generalized). It's one of the few distributions used by econometricians that has a finite support.</span><br />
<span style="font-family: inherit;"><br /></span>
<span style="font-family: inherit;">This suggests that it might be useful when modelling data that are continuous in nature, and are in the form of proportions. Indeed, this is the case. I discuss this in the context of consumer demand analysis in <a href="https://davegiles.blogspot.com/2013/07/allocation-models-with-bounded.html">this old post</a>. More generally, the Beta regression model is discussed in detail by Ferrari and Cribari-Neto (2004) and Cribari-Neto and Zeileia (2010).</span><br />
<span style="font-family: inherit;"><br /></span>
<span style="font-family: inherit;">The density function for <i>X</i>, that follows a Beta distribution is:</span><br />
<span style="font-family: inherit;"><br /></span>
<span style="font-family: inherit;"><i>f </i>(<i>x</i> | <i>α</i>, <i>β</i>) = [Γ(<i>α</i> + <i>β</i>) / (Γ(<i>α</i>)Γ(<i>β</i>))] <i>x </i><sup><i>α</i>-1</sup> (1 - <i>x</i>) <sup><i>β</i>-1</sup> ; 0 < <i>x</i> < 1 ; <i>α</i> , <i>β</i> > 0</span><br />
<span style="font-family: inherit;"><br /></span>
<span style="font-family: inherit;">The general formula for the <i>r</i>th. raw moment of <i>X</i> is</span><br />
<span style="font-family: inherit;"><br /></span>
<span style="font-family: inherit;"><i>μ<sub>r</sub></i>' = E[<i>X <sup>r</sup></i>] = [ (<i>α</i> + <i>r</i> - 1)(<i>α</i> + <i>r</i> - 2) ...... <i>α</i>] / [(<i>α</i> + <i>β</i> + <i>r</i> - 1)(<i>α</i> + <i>β</i> + <i>r</i> - 2) .....(<i>α</i> + <i>β</i>)] ; <i>r </i>= 1, 2, 3, ......</span><br />
<span style="font-family: inherit;"><br /></span>
<span style="font-family: inherit;">Immediately, we have:</span><br />
<span style="font-family: inherit;"><br /></span>
<span style="font-family: inherit;"><i>μ</i><sub>1</sub>' = α<i> </i>/ (<i>α</i> + <i>β</i>)</span><br />
<span style="font-family: inherit;"><i>μ</i><sub>2</sub>' = [<i>α </i>(<i>α</i> + 1)] / [(<i>α</i> + <i>β</i> + 1)(<i>α</i> + <i>β</i>)] = [(<i>α</i> + 1) / (<i>α</i> + <i>β</i> + 1)] <i>μ</i><sub>1</sub>'</span><br />
<span style="font-family: inherit;"><i>μ</i><sub>3</sub>' = [<i>α</i> (<i>α</i> + 1)(<i>α</i> + 2)] / [(<i>α</i> + <i>β</i> + 2)(<i>α</i> + <i>β</i> + 1)(<i>α</i> + <i>β</i>)] = [(<i>α</i> + 2) / (<i>α</i> + <i>β</i> + 2)] <i>μ</i><sub>2</sub>'</span><br />
<span style="font-family: inherit;"><i>etc</i>.</span><br />
<span style="font-family: inherit;"><br /></span>
<span style="font-family: inherit;">So, a general recursion formula for the raw moments of the Beta distribution is:</span><br />
<span style="font-family: inherit;"><br /></span>
<span style="font-family: inherit;"><i> <b><span style="color: #cc0000;">μ<sub>r</sub></span></b></i><b><span style="color: #cc0000;">' = [(<i>α</i> + <i>r</i> - 1) / (<i>α</i> + <i>β</i> + <i>r</i> - 1)] <i>μ</i><sub><i>r-</i>1</sub><span style="text-align: left;">' ; </span><i style="text-align: left;">r</i><span style="text-align: left;"> = 1, 2, 3, .....</span></span></b></span></div>
</div>
</div>
<span style="font-family: inherit;"><br /></span>
<b><span style="color: blue; font-family: inherit;">Numerical examples</span></b><br />
<div style="text-align: justify;">
<b><span style="color: blue; font-family: inherit;"><br /></span></b></div>
<div style="text-align: justify;">
<span style="font-family: inherit;">Remember, the whole point abut these recursion formulae is they help us to rapidly compute <i>all of the moments</i> of a distribution, up to some pre-specified maximum order, using one general formula.</span></div>
<div style="text-align: justify;">
<span style="font-family: inherit;"><br /></span></div>
<div style="text-align: justify;">
<span style="font-family: inherit;">There is an R script file on the <a href="https://davegiles.blogspot.com/p/code.html">code page</a> for this blog that illustrates this point, first for <i>X</i> ~ N[<i>θ</i>, <i>σ</i><sup>2</sup>]; and second for <i>X</i> ~ F[<i>v</i><sub>1</sub>,<i>v</i><sub>2</sub>]. In the first case the first ten raw moments for <i>X</i> when <i>θ</i> = 1 and <i>σ</i><sup>2</sup> = 4 are: 1, 5, 13, 73, 281, 1741, 8485, 57233, 328753, 2389141, ....</span></div>
<div style="text-align: justify;">
<span style="font-family: inherit;"><br /></span></div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiNhpHFRtG6V_Sbmr4Ma1WUnJVqEmmUr64DHJGpLKiqkSc0I6EatqSx_5F0PeNDGRd0tcW8NrXqfJuzsPbhl-ThZlVkxnWzmzOCCPap5uSXXbxFSF_dRbh40Ei-oHt1h2_XKAGKr2cWdS44/s1600/Capture.JPG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><span style="font-family: inherit;"></span></a></div>
<div style="text-align: justify;">
<span style="font-family: inherit;">In the second case, the first ten raw moments for </span><i style="font-family: inherit;">X </i><span style="font-family: inherit;">when </span><i style="font-family: inherit;">v</i><sub style="font-family: inherit;">1</sub><span style="font-family: inherit;"> = 7 and </span><i style="font-family: inherit;">v</i><sub style="font-family: inherit;">2</sub><span style="font-family: inherit;"> = 24 are: 1.09, 1.68, 3.5265, 9.82, 36.09, 175.28, 1141.85, 10276.63, 135064.30, 2894235.00, ....</span><br />
<span style="font-family: inherit;"><br /></span>
<span style="font-family: inherit;"><b>References</b></span></div>
<div class="separator" style="clear: both; text-align: center;">
<br /></div>
<div style="text-align: justify;">
<span style="font-family: inherit;"><a href="https://poseidon01.ssrn.com/delivery.php?ID=338006067086065015013013102073100010056016063065052016092088110076070017014026015097026052001024039049007018029124009066101003051040007073093001078031110003105126028058084017088108087007004111087091091025094094121116025102087076001071072080010089071&EXT=pdf">Alghalith, M.</a>, undated. A note on generalizing and extending Stein's lemma.</span><br />
<span style="font-family: inherit;"><br /></span>
<span style="font-family: inherit;"><a href="https://www.tandfonline.com/doi/abs/10.1080/00031305.1969.10481867">Bain, L. J</a>., 1969. Moments of non-central t and on-central F distribution. <i>American Statistician</i>, 23, 33-34.</span><br />
<span style="font-family: inherit;"><br /></span>
<span style="font-family: inherit;"><a href="https://www.jstor.org/stable/2287118?seq=1#page_scan_tab_contents">Cobb, L., P. Koppstein, & N. H. Chen</a>, 1983. Estimation and moment recursion relations for multimodal distributions of the exponential family. <i>Journal of the American Statistical Association</i>, 78, 124-130.</span></div>
<div style="text-align: justify;">
<span style="font-family: inherit;"><br /></span>
<span style="font-family: inherit;"><a href="http://www.jstatsoft.org/v34/i02/">Cribari-Neto F. & A. Zeileis</a>, 2010. Beta regression in R. <i>Journal of Statistical Software</i>, 1–24.</span><br />
<span style="font-family: inherit;"><br /></span>
<span style="font-family: inherit;"><a href="https://orfe.princeton.edu/~alaink/NJ_aTaxiOrf467F12/Papers/Machine%20Learning/beta.pdf">Ferrari, S. L. P. & F. Cribari-Neto</a>, 2004. Beta regression for modelling rates and proportions. <i>Journal of Applied Statistics</i>, 31, 799–815.</span><br />
<span style="font-family: inherit;"><br /></span></div>
<div style="text-align: justify;">
<span style="font-family: inherit;"><a href="https://www.amazon.com/Econometric-Analysis-7th-William-Greene/dp/0131395386/ref=sr_1_3?crid=2RRGDK0UC2P7Y&keywords=greene+econometric+analysis&qid=1556585157&s=gateway&sprefix=greene+econom%2Caps%2C517&sr=8-3">Greene, W. E</a>., 2012. <i>Econometric Analysis</i>, 7th. ed.. Prentice Hall, Upper Saddle River, NJ.</span></div>
<div style="text-align: justify;">
<span style="font-family: inherit;"><br /></span></div>
<div style="text-align: justify;">
<span style="font-family: inherit;">J<a href="https://www.amazon.com/statistical-implications-Stein-rule-econometrics-mathematical/dp/072040729X/ref=sr_1_1?keywords=judge+and+bock+pre-test+stein&qid=1556585368&s=gateway&sr=8-1-spell">udge, G. G. & M. E. Bock</a>, 1978. <i>The Statistical Implications of Pre-Test and Stein-Rule Estimators in Econometrics</i>. North-Holland, New York.</span></div>
<div style="text-align: justify;">
<span style="font-family: inherit;"><br /></span></div>
<div style="text-align: justify;">
<span style="font-family: inherit;"><a href="https://poseidon01.ssrn.com/delivery.php?ID=262102004111026069000116018018094106116003041088086064113095069094031124110117020006000106044041050001050121126069069079078092000040001010006113016096030113112113057076075083086098030079091096078087001095083123073117106123092081123089108003023025024&EXT=pdf">Landsman, Z., S. Vanduffel, & J. Yao</a>, 2013. A note on Stein's lemma for multivariate elliptical distributions. <i>Journal of Statistical Planning and Inference</i>, 11, 2016-2022.</span></div>
<div style="text-align: justify;">
<span style="font-family: inherit;"><br /></span></div>
<div style="text-align: justify;">
<span style="font-family: inherit;">Stein, C. M., 1973. Estimation of the mean of a multivariate normal distribution. <i>Proceedings of the Prague Symposium on Asymptotic Statistics</i>, 345-381. </span></div>
<div style="text-align: justify;">
<span style="font-family: inherit;"><br /></span></div>
<div style="text-align: justify;">
<span style="font-family: inherit;"><a href="https://www.tandfonline.com/doi/abs/10.1081/STA-120018823?src=recsys&journalCode=lsta20">Willink, R.</a>, 2003. Relationships between central moments and cumulants, with formulae for the central moments of gamma distributions. <i>Communications in Statistics - Theory and Methods</i>, 32, 701-704.</span></div>
<div style="text-align: justify;">
<span style="font-family: inherit;"><br /></span>
<span style="font-family: inherit;"><a href="https://www.sciencedirect.com/science/article/pii/S0167715205001124">Willink, R.</a>, 2005. Normal moments and Hermite polynomials. <i>Statistics and Probability Letters</i>, 73, 271-275.</span><br />
<span style="font-family: inherit;"><br /></span></div>
<div style="text-align: justify;">
<span style="font-family: inherit;"><a href="https://www.stats.org.nz/new-zealand-statistician/">Withers, S. W.</a>, 1992. A recurrence relation for the moments of the exponential. <i>New Zealand Statistician</i>, 27, 13-14.</span></div>
<div style="text-align: justify;">
<span style="font-family: inherit;"><br /></span></div>
<div style="text-align: justify;">
<span style="font-family: inherit;"><a href="https://www.sciencedirect.com/science/article/pii/0304407679900897">Woodland, A. D.</a>, 1979. Stochastic specification and the estimation of share equations. <i>Journal of Econometrics</i>, 10, 361-383.<i> </i></span></div>
<center>
<span style="font-family: inherit;">
© 2019, David E. Giles
</span></center>
</div>
Dave Gileshttp://www.blogger.com/profile/05389606956062019445noreply@blogger.com0tag:blogger.com,1999:blog-2198942534740642384.post-50658969144345646122019-04-21T19:16:00.002-07:002019-04-21T19:16:19.346-07:00Recursions for the Moments of Some Discrete Distributions<div dir="ltr" style="text-align: left;" trbidi="on">
<div dir="ltr" style="text-align: left;" trbidi="on">
<div style="text-align: justify;">
You could say, "Moments maketh the distribution". While that's not <i>quite </i>true, it's pretty darn close.</div>
<div style="text-align: justify;">
<br /></div>
<div style="text-align: justify;">
The moments of a probability distribution provide key information about the underlying random variable's behaviour, and we use these moments for a multitude of purposes. Before proceeding, let's be sure that we're on the same page here.<br />
<br />
<a name='more'></a></div>
<div style="text-align: justify;">
<br />
<b><span style="color: #cc0000;">Some background</span></b><br />
<br /></div>
<div style="text-align: justify;">
Suppose that we have a random variable, <i>X</i>, whose distribution function is <i>F</i>(<i>x</i>), where <i>x</i> is some value of <i>X</i>. The following quote comes from a<a href="https://davegiles.blogspot.com/2011/10/its-all-in-moments.html">n old blog post</a> of mine:</div>
<div style="text-align: justify;">
<blockquote class="tr_bq" style="background-color: white; color: #444444;">
<span style="font-family: inherit;">"What is sometimes called "the problem of moments" tells us:</span></blockquote>
<blockquote class="tr_bq">
<em style="background-color: white; color: #444444;"><span style="font-family: inherit;">If all of the moments of a distribution exist, then knowledge of these moments is equivalent to knowledge of the distribution itself.</span></em></blockquote>
<blockquote class="tr_bq" style="background-color: white; color: #444444;">
<span style="font-family: inherit;">In other words, the moments completely define the distribution.</span><span style="font-family: inherit;"><br /></span><span style="font-family: inherit;"></span></blockquote>
<blockquote class="tr_bq" style="background-color: white; color: #444444;">
<span style="font-family: inherit;">However, note the word, 'if' in the statement of the result above. And it's a very big 'if'! The problem is that for many distributions the moments exist only under certain conditions; and for some distributions some or <em><strong>all</strong></em> of the moments fail to be defined. In these cases, the "theorem" is of limited help. A sufficient condition for a distribution to completely and uniquely determined by its moments is that its moment generating function (m.g.f.) exists."</span></blockquote>
T<span style="background-color: white; color: #444444; font-family: inherit;">he <em>r</em><sup>th</sup> "raw moment' (or '"moment about zero") for the random variable, <em>X </em>is defined as </span><em style="background-color: white; color: #444444;">μ<sub>r</sub></em><span style="background-color: white; color: #444444;">' = E[</span><em style="background-color: white; color: #444444;">X <sup>r</sup></em><span style="background-color: white; color: #444444;">] </span><span style="background-color: white; color: #444444; font-family: inherit;">= ∫ </span><em style="background-color: white; color: #444444; font-family: inherit;">x<sup>r</sup>dF</em><span style="background-color: white; color: #444444; font-family: inherit;">(</span><em style="background-color: white; color: #444444; font-family: inherit;">x</em><span style="background-color: white; color: #444444; font-family: inherit;">) ; <i>r</i> = 1, 2, ..... </span><br />
<span style="color: #444444;"><br /></span>
<span style="color: #444444;">We also use the "central (centered) moments" of <i>X</i>, which are defined as </span><em style="color: #444444;">μ<sub>r</sub></em><span style="background-color: white; color: #444444;"> = E[(</span><em style="color: #444444;">X - </em><span style="color: #444444;"><i>μ</i><sub>1</sub></span><span style="background-color: white; color: #444444;">' </span><span style="color: #444444;">)</span><em style="color: #444444;"><sup>r</sup></em><span style="background-color: white; color: #444444;">] ; <i>r</i> = 1, 2, 3, ....... Strictly speaking, we're talking about "integer-order" moments in both cases here. (Remember, <i>r</i> = 1, 2, 3, ....) There are also measures called fractional moments and factorial moments, but we won't be concerned with those.</span><br />
<span style="background-colr: white; color: #444444;"><br /></span>
<span style="background-color: white; color: #444444;">So, obviously, </span><span style="color: #444444;"><i>μ</i><sub>1</sub></span><span style="background-color: white; color: #444444;">' = E[<i>X</i>] is just the mean of <i>X</i>; and </span><span style="color: #444444;"><i>μ</i><sub>2</sub></span><span style="background-color: white; color: #444444;"> = E[(</span><em style="color: #444444;">X - </em><span style="color: #444444;">E</span><span style="color: #444444;">(</span><em style="color: #444444;">X</em><span style="color: #444444;">)</span><span style="color: #444444;">)<sup>2</sup></span><span style="background-color: white; color: #444444;">] is the variance of <i>X</i>. We can always express the "raw moments" in terms of the "centered moments", and <i>vice versa</i>. For example, you'll be aware that Var.[<i>X</i>] = E[<i>X </i><sup>2</sup>] - [E(<i>X</i>)]<sup>2</sup>. That is, </span><span style="color: #444444;"><i>μ</i><sub>2</sub></span><span style="background-color: white; color: #444444;"> = </span><span style="color: #444444;"><i>μ</i><sub>2</sub>' </span>- (<i style="color: #444444;">μ</i><sub>1</sub>')<sup>2</sup>.<br />
<br />
In general, the relationships between these two forms of the moments can be obtained by noting that for some values, <i>a</i> and <i>b</i>,<br />
<br />
E[(<i>X</i> - <i>b</i>) <sup><i>r</i></sup>] = Σ{[<i>r</i>! / (<i>i</i>! (<i>r</i> - <i>i</i>)!)] E[(<i>X</i> - <i>a</i>)<sup><i>i </i></sup>(<i>a</i> - <i>b</i>) <sup><i>r-i </i></sup>} ,<br />
<br />
where the range of summation is from <i>i</i> = 0 to <i>i</i> = <i>r</i>.<br />
<br />
If we set <i>a</i> = 0, and <i>b</i> = <span style="color: #444444;"><i>μ</i><sub>1</sub></span><span style="background-color: white; color: #444444;">', we can obtain the centered moments from the raw moments, as follows:</span><br />
<span style="background-color: white; color: #444444;"><br /></span>
<b> <i>μ<sub>r</sub></i> = Σ{[<i>r</i>! / (<i>i</i>! (<i>r</i> - <i>i</i>)!)] <i>μ<sub>i</sub></i>' (- <i>μ</i><sub>1</sub><i>' </i>) <sup><i>r-i </i></sup>} </b>, (1)<br />
<br />
where the range of summation is from <i>i</i> = 0 to <i>i</i> = <i>r</i>.<br />
<span style="background-color: white; color: #444444;"><br /></span>
<span style="background-color: white; color: #444444;">Similarly, if we set <i>a</i> = </span><span style="color: #444444;"><i>μ</i><sub>1</sub></span><span style="background-color: white; color: #444444;">', and <i>b</i> = 0, we can obtain the raw moments from the centered moments:</span><br />
<span style="background-color: white; color: #444444;"><br /></span>
<b> <i>μ<sub>r</sub></i>' = Σ{[<i>r</i>! / (<i>i</i>! (<i>r</i> - <i>i</i>)!)] <i>μ<sub>i</sub></i><sup><i> </i></sup>(<i>μ</i><sub>1</sub><i>' </i>) <sup><i>r-i </i></sup>} </b>, (2)<br />
<br />
where, again, the range of summation is from <i>i</i> = 0 to <i>i</i> = <i>r</i>.<br />
<span style="background-color: white; color: #444444;"><br /></span>
Often, the first couple of moments of a distribution (<i>i.e.</i>, of a random variable) may be all that interest us. For instance, the Mean Squared Error (MSE) of an estimator, <span style="background-color: white; color: #444444;"><i>θ</i>*</span>, of a parameter, <i>θ</i>, is just it's variance plus the square of its bias. In other words, MSE (<span style="background-color: white; color: #444444;"><i>θ</i>*</span>) = <span style="color: #444444;"><i>μ</i><sub>2</sub></span><span style="background-color: white; color: #444444;"> + </span>(<i style="color: #444444;">μ</i><sub>1</sub><span style="color: #444444; font-size: 13.3333px;">'</span><span style="font-size: 13.3333px;"> - </span><span style="background-color: white; color: #444444;"><i>θ</i>)<sup>2</sup>, where the moments here are those of the sampling distribution of the estimator, <i>θ</i>*.</span><br />
<span style="background-color: white; color: #444444;"><br /></span>
<span style="color: #444444;"><span style="background-color: white;">However, we use the third and fourth (central) moments when defining skewness and kurtosis. For example, Pearson's definitions of these two measures are </span></span><span style="color: #444444;"><span style="background-color: white;">Skew (<i>X</i>) = </span></span><span style="color: #444444;"><i>μ</i><sub>3</sub> </span><span style="background-color: white; color: #444444;">/ </span><span style="color: #444444;"><span style="font-size: 13.3333px;">(</span></span><i style="color: #444444;">μ</i><sub>2</sub>)<sup>3/2</sup><span style="color: #444444;"><span style="background-color: white;">, and Kurtosis (<i>X</i>) = </span></span><span style="color: #444444;"><i>μ</i><sub>4</sub> / (</span><i style="color: #444444;">μ</i><sub>2</sub>)<sup>2</sup><span style="background-color: white; color: #444444; text-align: left;">. Notice that both of these quantities are <i>unitless</i>. A good example of where we use them is in the construction of tests for Normality. For instance, the null hypothesis for the Jarque-Bera test is that the skewness = 0 and the kurtosis = 3. Only a Normal distribution has this property. So the J-B test, which is a Lagrange multipler ("score") test, is testing the validity of <b>two</b> restrictions on the parameters of the distribution. That's why the test statistic always has a null distribution that's Chi-square, with <b>two </b>degrees of freedom. </span><br />
<span style="background-color: white; color: #444444; text-align: left;"><br /></span>
<span style="background-color: white; color: #444444; text-align: left;">Higher-order moments can be used to introduce additional shape parameters into various distributions. Sometimes the existence of up to the first eight moments is needed in the proofs of various results in statistics and econometrics.</span><br />
<span style="background-color: white; color: #444444; text-align: left;"><br /></span></div>
</div>
<div style="text-align: justify;">
<span style="color: #444444;"><span style="background-color: white;">Now look back at the basic definition of the raw moments, namely, </span></span><em style="background-color: white; color: #444444; text-align: justify;">μ<sub>r</sub></em><span style="background-color: white; color: #444444; text-align: justify;">' = E[</span><em style="background-color: white; color: #444444; text-align: justify;">X <sup>r</sup></em><span style="background-color: white; color: #444444; text-align: justify;">] </span><span style="background-color: white; color: #444444; text-align: justify;">= ∫ </span><em style="background-color: white; color: #444444; font-family: inherit; text-align: justify;">x<sup>r</sup>dF</em><span style="background-color: white; color: #444444; text-align: justify;">(</span><em style="background-color: white; color: #444444; font-family: inherit; text-align: justify;">x</em><span style="background-color: white; color: #444444; text-align: justify;">) ; <i>r</i> = 1, 2, ..... ; and </span><span style="color: #444444;"><span style="background-color: white;"> </span></span><em style="background-color: white; color: #444444; text-align: justify;">μ<sub>r</sub></em><span style="background-color: white; color: #444444; text-align: justify;"> = E[(</span><em style="background-color: white; color: #444444; text-align: justify;">X - μ</em><span style="background-color: white; color: #444444; text-align: justify;"><sub>1</sub>') <sup style="font-style: italic;">r </sup></span><span style="background-color: white; color: #444444; text-align: justify;">] </span><span style="background-color: white; color: #444444; text-align: justify;">= ∫ (</span><span style="background-color: white; color: #444444; font-family: inherit; text-align: justify;"><i>x - μ</i><sub>1</sub>') <sup><i>r </i></sup><i>dF</i></span><span style="background-color: white; color: #444444; text-align: justify;">(</span><em style="background-color: white; color: #444444; font-family: inherit; text-align: justify;">x</em><span style="background-color: white; color: #444444; text-align: justify;">) ; <i>r</i> = 1, 2, ..... </span><span style="color: #444444;"> </span></div>
<div style="text-align: justify;">
<span style="color: #444444;"><br /></span></div>
<div style="text-align: justify;">
<span style="color: #444444;">I'm not sure about you, but I don't think I'd want to be evaluating all of those integrals if there was an easier way to obtain the moments! And you probably know that there is. We can use the moment generating function (if it exists), or the cumulant generating function (which always exists). In each case, we have two options. One involving successively differentiating the m.g.f or the c.g.f.; and the other involves using a Taylor series expansion and equating coefficients.</span></div>
<div style="text-align: justify;">
<span style="color: #444444;"><br /></span></div>
<div style="text-align: justify;">
<span style="color: #444444;">Even then, for some distributions, obtaining the moments in these ways can be rather tedious - especially if you have reason to obtain the moments up to a fairly high order, </span></div>
<div style="text-align: justify;">
<span style="color: #444444;"><br /></span></div>
<div style="text-align: justify;">
<span style="color: #444444;">Is there an even simpler way to proceed? I don't want to sound lazy, but computational efficiency is important.</span></div>
<div style="text-align: justify;">
<span style="color: #444444;"><br /></span></div>
<div style="text-align: justify;">
<b style="text-align: left;"><span style="color: #cc0000;">Recursions, recursions, .......</span></b></div>
<div style="text-align: justify;">
<br /></div>
<div style="text-align: justify;">
Econometrics students should be pretty familiar the discussion so far. Now, let's turn to some other relationships between the moments of certain distributions. Usually, these relationships don't come up in your typical econometrics course, but they're very useful - especially from a computational viewpoint.</div>
<div style="text-align: justify;">
<br /></div>
<div style="text-align: justify;">
Specifically, these relationships are in the form of recursion formulae.</div>
<div style="text-align: justify;">
<br /></div>
<div style="text-align: justify;">
I'm sure you know what a recursion is. It's a formula that takes us forward by building on the previous steps. As a really trivial example of this, consider the formula for the factorial of an integer, <i>n</i>. You know that <i>n</i><span style="text-align: left;">! = <i>n</i> (<i>n</i>-1)(<i>n</i>-2)(<i>n</i>-3).....3.2.1. So, we can write <i>n</i>! = <i>n</i> (<i>n</i>-1)! In other words, once we have computed (<i>n</i>-1)! we just multiply by <i>n</i> to move one set forward to <i>n</i>!, and so on.</span></div>
<br />
Because 1! = 1, we have 2! = 2 (1!) = 2; 3! = 3(2!) = 6; 4! = 4(3!) = 24; <i>etc</i>.<br />
<br />
<div style="text-align: justify;">
If we proceed sequentially in this way, we just have <i>one multiplication </i>to do at each new step, instead of a whole lot of the. I know that isn't a big deal, and nor is there really that much computational cost-saving in this example. However, in more complex problems, recursion formulae can be very powerful tools.</div>
<div style="text-align: justify;">
<br /></div>
<div style="text-align: justify;">
Why is this interesting in the context of the moments of a random variable?</div>
<div style="text-align: justify;">
<br /></div>
<div style="text-align: justify;">
Such recursion formulae aren't available for all of the distributions that you're likely to encounter. (It turns out that <i>most of them</i> require membership of the so-called <a href="https://en.wikipedia.org/wiki/Exponential_family">exponential families.</a>) But there are some interesting ones for discrete distributions that are relevant in econometrics. My follow-up post will deal with similar results for some continuous distributions, where we'll see that there's also an important connection with <a href="https://en.wikipedia.org/wiki/Stein%27s_lemma">Stein's Lemma</a>. </div>
<br />
<b><span style="color: blue;">Binomial distribution</span></b><br />
<div style="text-align: justify;">
<b><span style="color: blue;"><br /></span></b></div>
<div style="text-align: justify;">
A couple of useful references here are Zhang <i>et al</i>. (2018) and Riordan (1937).<br />
<br /></div>
<div style="text-align: justify;">
Strictly, this first example, and those that follow, aren't in the form of <i>direct</i> recursions - they're more like <i>indirect </i>recursions, but they each involve a neat "trick".<br />
<br /></div>
<div style="text-align: justify;">
Suppose that <i>X</i> ~ Bi[(<i>n</i> - 1) , <i>p</i>] and <i>Y</i> ~ Bi[<i>n</i> , <i>p</i>]. <span style="text-align: justify;">Consider a function </span><i style="text-align: justify;">g</i><span style="text-align: justify;">(<i>.</i></span><span style="text-align: justify;">) such that both |E[</span><i style="text-align: justify;">g</i><span style="text-align: justify;">(<i>.</i></span><span style="text-align: justify;">)]| and |</span><i style="text-align: justify;">g</i><span style="text-align: justify;">(-1)| are finite. Then, </span></div>
<div style="text-align: justify;">
<span style="text-align: justify;"><br /></span></div>
<div style="text-align: justify;">
<span style="text-align: justify;"> <b> E[<i>g</i>(<i>X</i>)] = </b></span><b>E[Y <i>g</i>(<i>Y</i> - 1) / (<i>np</i>)]</b> .</div>
<div style="text-align: justify;">
<span style="text-align: left;"><br /></span></div>
<div style="text-align: justify;">
<span style="text-align: left;">We can use this result to obtain the raw moments of <i>Y</i>, recursively. If we let <i>g</i>(<i>Z</i>) = Z <i><sup>k</sup></i>, for some Z, then:</span></div>
<div style="text-align: justify;">
<span style="text-align: left;"><br /></span></div>
<div style="text-align: justify;">
<span style="text-align: left;"> <b> E[<i>X <sup>k</sup></i>] = E[Y(Y - 1)<sup>k</sup> / (np)]</b> .</span></div>
<div style="text-align: justify;">
</div>
<ul>
<li><span style="text-align: left;">If <i>k</i> = 0, then 1 = E[<i>Y</i>] / (<i>np</i>); and so <i style="text-align: justify;">μ</i><sub style="text-align: justify;">1</sub><span style="text-align: justify;">' =</span> E[<i>Y</i>] = <i>np</i>. (Clearly, E[<i>X</i>] = (<i>n</i> - 1)<i>p</i>. )</span></li>
<li><span style="text-align: left;">If <i>k</i> = 1, then E[<i>X</i>] = E[<i>Y</i>(<i>Y</i> - 1) / (<i>np</i>)]; and so </span><i>μ</i><sub>2</sub>' = E[<i>Y </i><sup>2</sup>] = <i>μ</i><sub>1</sub>' + <i>n</i>(<i>n</i> - 1)<i>p</i><sup>2</sup> = <i>np</i>[1 + (<i>n</i> - 1)<i>p</i>].</li>
</ul>
<span style="text-align: justify;">(So, the variance of </span><i style="text-align: justify;">Y</i><span style="text-align: justify;"> is </span><i style="text-align: justify;">μ</i><sub style="text-align: justify;">2</sub><span style="text-align: justify;"> = </span><i style="text-align: justify;">μ</i><sub style="text-align: justify;">2</sub><span style="text-align: justify;">' - (</span><i style="text-align: justify;">μ</i><sub style="text-align: justify;">1</sub><span style="text-align: justify;">')</span><sup style="text-align: justify;">2</sup><span style="text-align: justify;"> = </span><i style="text-align: justify;">np</i><span style="text-align: justify;">(1</span><i style="text-align: justify;">-p</i><span style="text-align: justify;">).</span><br />
<div style="text-align: justify;">
</div>
<ul>
<li>If <i>k</i> = 2, then E[<i>X </i><sup>2</sup>] = E[<i>Y</i>(<i>Y</i> - 1)<sup>2</sup> / (<i>np</i>)]; and so <i>μ</i><sub>3</sub>' = E[<i>Y </i><sup>3</sup>] = 2<i>μ</i><sub>2</sub>' + <i>μ</i><sub>1</sub>' {(<i>n</i> - 1)<i>p</i>[1 + (<i>n</i> - 2)<i>p</i> - 1}] = <i>n</i>(<i>n</i> - 1)(<i>n</i> - 2)<i>p</i><sup>3</sup> + 3<i>n</i>(<i>n</i> - 1)<i>p</i><sup>2</sup> + <i>np</i> .</li>
<li>If <i>k</i> = 3, then we get μ<sub>4</sub>' = E[<i>Y </i><sup>4</sup>] = (3 + <i>np</i>)E[<i>Y </i><sup>3</sup>] − 3E[<i>Y </i><sup>2</sup>] + E[<i>Y</i>] = <i>np</i> + 7<i>n</i>(<i>n</i> − 1)<i>p</i><sup>2</sup> + 6<i>n</i>(<i>n</i> − 1)(<i>n</i> − 2)<i>p</i><sup>3</sup> + <i>n</i>(<i>n</i> − 1)(<i>n</i> − 2)(<i>n</i> − 3)<i>p</i><sup>4</sup></li>
</ul>
<div style="text-align: justify;">
and so on................... Notice that the <i>r</i>th. moment is expressed in terms of the preceding moments. And of course, if it's the centered moments that we want, then the general formula in (1) , above, allows us to calculate them easily from these raw moments.<br />
<br />
As an aside, a different type of recursion formula for these moments is derived by <a href="https://www.maa.org/programs/faculty-and-departments/classroom-capsules-and-notes/a-recursive-formula-for-moments-of-a-binomial-distribution">Bényi</a> and Manago (2005).</div>
<div style="text-align: justify;">
<br /></div>
<div style="text-align: justify;">
<b><span style="color: blue;">Poisson distribution</span></b></div>
<div style="text-align: justify;">
<br />
A useful reference here is Hwang (1982).<br />
<br /></div>
<div style="text-align: justify;">
Let <i>X</i> follow a Poisson distribution with parameter <i>λ</i>. Consider a function <i>g</i>(<i>X</i>) such that both |E[<i>g</i>(<i>X</i>)]| and |<i>g</i>(-1)| are finite. Then,</div>
<div style="text-align: justify;">
<br /></div>
<div style="text-align: justify;">
<b>E[<i>λ</i> g(<i>X</i>)] = E[<i>X</i> <i>g</i>(<i>X </i>- 1)] </b>.<br />
<br />
If we let g(<i>X</i>) = <i>X <sup>k</sup></i> we have:<br />
<br />
<b>E[<i>X <sup>k</sup></i>] = (1 / <i>λ</i>) E[<i>X</i> (<i>X</i> - 1)<i><sup>k</sup></i>] </b>(3)<br />
<br />
and we can use this formula to generate the central moments of the Poisson distribution recursively.</div>
<ul style="text-align: left;">
<li style="text-align: justify;">If <i>k</i> = 0, then 1 = (1 / <i>λ</i>) E[<i>X</i>], or <i>μ</i><sub>1</sub>' = E[<i>X</i>] = <i>λ.</i></li>
<li style="text-align: justify;">If <i>k</i> = 1, then E[<i>X</i>] = (1 / <i>λ</i>) {E[<i>X </i><sup>2</sup>] - E[<i>X</i>]}, and so <i>μ</i><sub>2</sub>' = E[<i>X </i><sup>2</sup>] = (<i>λ</i> + 1) <i>μ</i><sub>1</sub>' = <i>λ</i>(<i>λ</i> + 1).</li>
</ul>
<div style="text-align: justify;">
(Immediately, the variance of <i>X</i> is <i>μ</i><sub>2</sub> = <i>μ</i><sub>2</sub>' - (<i>μ</i><sub>1</sub>')<sup>2</sup> = <i>λ</i>.)</div>
<ul style="text-align: left;">
<li style="text-align: justify;">If <i>k</i> = 2, then E[<i>X </i><sup>2</sup>] = (1 / <i>λ</i>) E[<i>X</i>(<i>X</i> - 1)<sup>2</sup>], and so <i>μ</i><sub>3</sub>' = E[<i>X </i><sup>3</sup>] = (<i>λ</i> + 2) <i>μ</i><sub>2</sub>' - <i>μ</i><sub>1</sub>' = <i>λ</i><sup>3</sup> + 3<i>λ</i><sup>2</sup> + <i>λ</i>.</li>
</ul>
<ul style="text-align: left;">
<li style="text-align: justify;">If <i>k</i>= 3, then E[<i>X </i><sup>3</sup>] = (1 / <i>λ</i>) E[<i>X</i>(<i>X </i>- 1)<sup>3</sup>], and so <i>μ</i><sub>4</sub>' = E[<i>X</i> <sup>4</sup>] = (<i>λ</i> + 3)<i>μ</i><sub>3</sub>' - 3<i>μ</i><sub>2</sub>' + <i>μ</i><sub>1</sub>' ; <i>etc.</i></li>
</ul>
<div style="text-align: justify;">
So, you see that once we have obtained <i>μ</i><sub>1</sub>' to <i>μ</i><sub><i>r</i></sub>', we can easily obtain <i>μ</i><sub><i>r</i>+1</sub>' for any <i>r</i>.<br />
<br />
The moments of <i>X</i> about its mean, <i>λ</i>, can then be obtained directly from these raw moments, using (1).</div>
<br />
<b><span style="color: blue;">Negative Binomial distribution</span></b><br />
<br />
Again, see Hwang (1982).<br />
<div style="text-align: justify;">
<br />
First, note that there are several forms of the Negative Binomial distribution. Here, I'm defining it in terms of a random variable, <i>X</i>, that counts the number of failures before the <i>r</i>th. success, where <i>p</i> is the probability of a success. This includes one version of the Geometric Distribution as a special case (when <i>r</i> = 1).<br />
<br /></div>
<div style="text-align: justify;">
Let <i>X</i> follow a Negative Binomial (NegBin) distribution with parameters <i>r</i> and <i>p</i>. Consider a function <i>g</i>(<i>X</i>) such that both |E[<i>g</i>(<i>X</i>)]| and |<i>g</i>(-1)| are finite. Then,</div>
<div style="text-align: justify;">
<br /></div>
<div style="text-align: justify;">
<b> E[(1 - <i>p</i>) <i>g</i>(<i>X</i>)] = E[<i>X</i> <i>g</i>(<i>X</i> - 1) / (<i>r</i> + <i>X</i> - 1)]</b> .</div>
<div style="text-align: justify;">
<br /></div>
<div style="text-align: justify;">
If we let <i>g</i>(<i>X</i>) = (<i>X</i> + <i>r</i>)<i><sup>k</sup></i>, then</div>
<div style="text-align: justify;">
<br /></div>
<b>E[(<i>X</i> + <i>r</i>)<i><sup>k</sup></i>] = (1 - <i>p</i>)<sup>-1</sup> E[<i>X</i>(<i>X</i> + <i>r</i> - 1)<sup><i>k-</i>1</sup>]</b>.<br />
<br />
<div style="text-align: justify;">
Following the same procedure as for the Poisson distribution, we can again obtain successively higher moments recursively from the lower moments:</div>
<div style="text-align: justify;">
</div>
<ul>
<li>If <i>k</i> = 1, then (1 - <i>p</i>)E(<i>X</i> + <i>r</i>) = E(<i>X</i>); and so <i>μ</i><sub>1</sub>' = <i>r</i> (1 - <i>p</i>)/ <i>p</i>. </li>
<li>If <i>k</i> = 2, then (1 - <i>p</i>)E[(<i>X</i> + <i>r</i>)<sup>2</sup>] = E(<i>X </i><sup>2</sup>) + <i>r</i>E(<i>X</i>) - E(<i>X</i>); and so <i>μ</i><sub>2</sub>' = [<i>r</i>(1 - <i>p</i>) / <i>p</i><sup>2</sup>][1 + <i>r</i>(1 - <i>p</i>)] = [<i>μ</i><sub>1</sub>']<sup>2</sup> + <i>μ</i><sub>1</sub>' / <i>p</i>. </li>
</ul>
<div>
(Right away, we see that Var.[<i>X</i>] = <i>r</i>(1 - <i>p</i>) / <i>p</i><sup>2</sup> = <i>μ</i><sub>1</sub>' / <i>p</i>.)</div>
<div>
<ul style="text-align: left;">
<li>If <i>k</i> = 3, then (1 - <i>p</i>)E[(<i>X</i> + <i>r</i>)]<sup>3</sup> = E[<i>X </i><sup>3</sup>] + 2(<i>r</i> - 1)E[<i>X </i><sup>2</sup>] + (<i>r</i> - 1)<sup>2</sup>E[<i>X</i>]; and so <i>μ</i><sub>3</sub>' = {[<i>r</i>(1 - 3<i>p</i>) + 2] / <i>p</i>}<i>μ</i><sub>2</sub>' + {[<i>r</i><sup>2</sup>(2 - 3<i>p</i>) + 2<i>r</i> - 1] / <i>p</i>}<i>μ</i><sub>1</sub>' + <i>r</i><sup>3</sup>(1 - <i>p</i>) / <i>p</i> ; <i>etc.</i></li>
</ul>
<div>
<div style="text-align: justify;">
Again, once we have obtained <i>μ</i><sub>1</sub>' to <i>μ</i><sub><i>r</i></sub>', we can easily obtain <i>μ</i><sub><i>r</i>+1</sub>' for any <i>r</i>; and the central moments of <i>X</i> can be obtained from (1).<br />
<br />
<b><span style="color: #cc0000;">Obtaining "true" recursions</span></b><br />
<b><span style="color: blue;"><br /></span></b>
I noted earlier that one advantage of a recursion formula is that it can simplify and speed computations. From the discussion of the three distributions above, it may not be obvious that we can achieve this from what we've seen so far.<br />
<br />
In fact, we can easily "extract" direct recursion formulae for the raw moments in each case. I'll use the Poisson distribution to illustrate this, and you should be able to apply the same approach to the other two distributions.<br />
<br />
Let's go back to equation (3), where we saw that<br />
<br />
<b>E[<i>X <sup>k</sup></i>] = (1 / <i>λ</i>) E[<i>X</i> (<i>X</i> - 1)<i><sup>k</sup></i>] </b>(3)<br />
<div>
<br /></div>
<div>
where <i>k</i> = 0, 1, 2, .....</div>
<div>
<br /></div>
<div>
From the binomial theorem, <i>X</i>(<i>X</i> - 1)<i><sup>k</sup></i> = Σ[<i>k</i>! / (<i>i</i>! (<i>k</i> - <i>i</i>!))] <i>X</i> <sup><i>k</i>-<i>i</i>+1</sup> (-1)<sup><i>i </i></sup>, where the range of summation is from <i>i</i> = <b><span style="color: red;">0</span></b> to <i>i</i> = <i>k</i>. (Watch out - this range will change twice in what follows!)</div>
<div>
<br /></div>
<div>
So, from (3),</div>
<div>
<br /></div>
<div>
<b> </b>E[<i>X <sup>k </sup></i>] = (1 / <i>λ</i>) Σ[<i>k</i>! / (<i>i</i>! (<i>k</i> - <i>i</i>!))](-1)<sup><i>i</i></sup> E[<i>X </i><sup><i>k</i>-<i>i</i>+1</sup>]<br />
<br />
= (1 / <i>λ</i>) Σ[<i>k</i>! / (<i>i</i>! (<i>k</i> - <i>i</i>!))](-1)<sup><i>i</i></sup> E[<i>X </i><sup><i>k</i>-<i>i</i>+1</sup>] + (1 / <i>λ</i>)E[<i>X </i><sup><i>k</i>+1</sup>], (4)<br />
<br />
where now the range of summation is from <i>i</i> = <b><span style="color: red;">1</span></b> to <i>i</i> = <i>k</i>.<br />
<br />
Extracting the term involving <i>i </i>= 1 in the summation in (4), and re-arranging, we get:<br />
<br />
E[<i>X </i><sup><i>k</i>+1</sup>] = (<i>λ + k</i>)E[<i>X <sup>k </sup></i>] - Σ[<i>k</i>! / (<i>i</i>! (<i>k</i> - <i>i</i>!))](-1)<sup><i>i</i></sup> E[<i>X </i><sup><i>k</i>-<i>i</i>+1</sup>] , (5)<br />
<br />
where the range of summation in (5) is for <i>i</i> = <b><span style="color: red;">2</span></b> to <i>i</i> = <i>k</i>.<br />
<br />
Setting <i>k</i> = 0, 1, 2, 3 we get the expressions for <i>μ</i><sub>1</sub>', <i>μ</i><sub>2</sub>', <i>μ</i><sub>3</sub>', and <i>μ</i><sub>4</sub>' for the Poisson distribution given earlier.<br />
<br />
Of course, what we have in (5) is a nice tidy recursion formula that enables us to calculate the (<i>k</i> + 1)th raw moment as a function of all of the preceding moments.<br />
<br />
<b><span style="color: #cc0000;">A numerical example</span></b><br />
<br />
We can easily write some code that will allow us to compute as many moments of the Poisson distribution as we wish, very efficiently. Subject to numerical "overflow" limitations, of course!<br />
<br />
By way of a quick illustration, there's some R code (available on the <a href="https://davegiles.blogspot.com/p/code.html">code page</a> of this blog) that speedily computes the first ten raw moments for the Poisson distribution with <i>λ</i> = 1. Here are the results:<br />
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjezF7AD1UP3TCjXMJPkvwHYEt5zSYbe8CFTT3LhWpRByNnw6N51xorH3H-5RnK0PbBjC176MvFkv5HnwDfgDsrchwLCDUFO5yfygyJFkPa3GnTghdr1D2uCWeY-xBOEReBdsLSiqY2qp_g/s1600/Capture.JPG" imageanchor="1" style="margin-left: 1em; margin-right: 1em; text-align: center;"><img border="0" data-original-height="91" data-original-width="640" height="89" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjezF7AD1UP3TCjXMJPkvwHYEt5zSYbe8CFTT3LhWpRByNnw6N51xorH3H-5RnK0PbBjC176MvFkv5HnwDfgDsrchwLCDUFO5yfygyJFkPa3GnTghdr1D2uCWeY-xBOEReBdsLSiqY2qp_g/s640/Capture.JPG" width="640" /></a><br />
The R code simulates the moments and plots the p.m.f. of the empirical distribution.<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiq04Ygro-t6L9-DPG2R6npl6Gfw9nTr_RNLj6lnWnc7vntPVo5lHDUxlPqHWSQ5FkpRQcbmVix1vpOPjOSEZ6bWSje1bsRkEZRjSbS63NS_PfGnEk4Y8DTO3LwOHtGLz7tD-fMnB74JwVD/s1600/Capture.JPG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="747" data-original-width="729" height="400" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiq04Ygro-t6L9-DPG2R6npl6Gfw9nTr_RNLj6lnWnc7vntPVo5lHDUxlPqHWSQ5FkpRQcbmVix1vpOPjOSEZ6bWSje1bsRkEZRjSbS63NS_PfGnEk4Y8DTO3LwOHtGLz7tD-fMnB74JwVD/s400/Capture.JPG" width="390" /></a></div>
<br />
The recursion formula comes into its own if,for example, you want to compute the 20th. moment when <i>λ</i> = 3. The answer is 1.4 x10<sup>18</sup>. Try working that one out by hand!<br />
<div>
<br /></div>
<div>
Obviously, you can amend the R code to choose your own value of <i>λ</i>, and the number of moments that you want to calculate. </div>
<br />
The <b>BIG</b> takeaway from this post? If you want to to compute (lots of ) the moments of some common discrete distributions, you should look beyond the standard approaches. There are some interesting recursion formulae that might help you.<br />
<br />
I have a follow-up post on its way that deals with similar results for some familiar continuous distributions. Watch out for it!</div>
</div>
</div>
</div>
<b style="text-align: justify;"><br /></b><b style="text-align: justify;">References</b><br />
<div style="text-align: justify;">
<br />
<a href="https://www.maa.org/programs/faculty-and-departments/classroom-capsules-and-notes/a-recursive-formula-for-moments-of-a-binomial-distribution">Bényi, Á. & S. M. Manago</a>, 2005. A recursive formula for moments of a binomial distribution. <i>College Mathematics Journal</i>, 36, 68-72.<br />
<br /></div>
<div style="text-align: justify;">
<a href="https://projecteuclid.org/euclid.aos/1176345876">Hwang, J. T.</a>, 1982. Improving on standard estimators in discrete exponential families with applications to Poisson and negative binomial cases. <i>Annals of Statistics</i>, 10, 857–867.<br />
<br />
<a href="https://www.jstor.org/stable/2957598?seq=1#page_scan_tab_contents">Riordan, J.</a>, 1937. Moment recurrence relations for binomial, Poisson and hypergeometric frequency distributions.<i> Annals of Mathematical Statistics</i>, 8, 103-111.</div>
<div style="text-align: justify;">
<br />
<a href="https://www.tandfonline.com/doi/abs/10.1080/03610926.2018.1435818?scroll=top&needAccess=true&journalCode=lsta20">Zhang, Y-Y., T-Z. Rong, & M-M. Li,</a> 2018. Expectation identity for the binomial distribution and its application in the calculations of high-order binomial moments. <i>Communications in Statistics - Theory and Methods</i>, available online.<br />
<br /></div>
<center>
© 2019, David E. Giles
</center>
</div>
Dave Gileshttp://www.blogger.com/profile/05389606956062019445noreply@blogger.com0tag:blogger.com,1999:blog-2198942534740642384.post-61565952749631660242019-04-12T17:16:00.000-07:002019-04-12T17:16:04.519-07:002019 Econometric Game Results<div dir="ltr" style="text-align: left;" trbidi="on">
The <a href="http://econometricgame.nl/">Econometric Game</a> is over for another year.<br />
<br />
The winning team for 2019 was from the University of Melbourne.<br />
<br />
The second and third placed teams were from the Maastricht University and Aarhus University, respectively.<br />
<br />
Congratulations to the winning teams, and to all who competed this year!<br />
<br />
<center>
© 2019, David E. Giles
</center>
</div>
Dave Gileshttp://www.blogger.com/profile/05389606956062019445noreply@blogger.com0tag:blogger.com,1999:blog-2198942534740642384.post-70890321969473172972019-04-10T06:57:00.001-07:002019-04-10T11:24:31.487-07:00EViews 11 Now Available<div dir="ltr" style="text-align: left;" trbidi="on">
<div style="text-align: justify;">
As you'll know already, I'm a big fan of the <a href="http://www.eviews.com/">EViews</a> econometrics package. I always found it to be a terrific, user-friendly, resource when teaching economic statistics and econometrics, and I use it extensively in my own research.</div>
<div style="text-align: justify;">
<br /></div>
<div style="text-align: justify;">
Along with a lot of other EViews users, I recently had the opportunity to "test drive" the beta release of the latest version of this package, EViews 11. </div>
<div style="text-align: justify;">
<br /></div>
<div style="text-align: justify;">
<a href="http://www.eviews.com/EViews11/ev11main.html">EViews 11</a> has now been officially released, and it has some great <a href="http://www.eviews.com/EViews11/ev11whatsneweg.html">new features</a>. (Click on the links there to see some really helpful videos.) To see what's now available, check it out <a href="http://www.eviews.com/EViews11/ev11features.html">here</a>. </div>
<div style="text-align: justify;">
<br /></div>
<div style="text-align: justify;">
Nice update. Thanks!</div>
<div style="text-align: justify;">
<br /></div>
<center>
© 2019, David E. Giles
</center>
</div>
Dave Gileshttp://www.blogger.com/profile/05389606956062019445noreply@blogger.com2tag:blogger.com,1999:blog-2198942534740642384.post-14091730654277319252019-04-09T06:44:00.003-07:002019-04-09T06:44:42.585-07:00SHAZAM!<div dir="ltr" style="text-align: left;" trbidi="on">
<div style="text-align: justify;">
This past weekend the new movie, <i><a href="https://www.imdb.com/title/tt0448115/">Shazam</a></i>, topped the box-office revenue list with over US$53million - and this is it's first weekend since being released.</div>
<div style="text-align: justify;">
<br /></div>
<div style="text-align: justify;">
Not bad!</div>
<div style="text-align: justify;">
<br /></div>
<div style="text-align: justify;">
Of course, in the Econometrics World, we associate the word, SHAZAM, with Ken White's famous <a href="http://www.econometrics.com/">computing package</a>, which has been with us since 1977. </div>
<div style="text-align: justify;">
<br /></div>
<div style="text-align: justify;">
Ken and I go way back. A few years ago I had a post about the background to the SHAZAM package. In that post I explained what the acronym "SHAZAM" stands for. If you check it out you'll see why it's timely for you to know these important historical facts!</div>
<div style="text-align: justify;">
<br /></div>
<div style="text-align: justify;">
And while you're there, take a look at the links to other tales that illustrate Ken's well-known wry sense of humour.</div>
<div style="text-align: justify;">
<br /></div>
<center>
© 2019, David E. Giles
</center>
</div>
Dave Gileshttp://www.blogger.com/profile/05389606956062019445noreply@blogger.com1tag:blogger.com,1999:blog-2198942534740642384.post-24446720755402341292019-04-08T06:54:00.002-07:002019-04-09T17:35:21.707-07:00A Permutation Test Regression Example<div dir="ltr" style="text-align: left;" trbidi="on">
<div dir="ltr" style="text-align: left;" trbidi="on">
<div style="text-align: justify;">
In <a href="https://davegiles.blogspot.com/2019/04/what-is-permutation-test.html">a post last week</a> I talked a bit about Permutation (Randomization) tests, and how they differ from the (classical parametric) testing procedure that we generally use in econometrics. I'm going to assume that you've read that post.</div>
<div style="text-align: justify;">
<br /></div>
<div style="text-align: justify;">
(<i>There may be a snap quiz at some point</i>!)</div>
<div style="text-align: justify;">
<br /></div>
<div style="text-align: justify;">
I promised that I'd provide a regression-based example. After all, the two examples that I went through in that previous post were designed to expose the fundamentals of permutation/randomization testing. They really didn't have much "econometric content".</div>
<div style="text-align: justify;">
<br /></div>
<div style="text-align: justify;">
In what follows I'll use the terms "permutation test" and "randomization test" interchangeably.</div>
<div style="text-align: justify;">
<br /></div>
<div style="text-align: justify;">
What we'll do here is to take a look at a simple regression model and see how we could use a randomization test to see if there is a linear relationship between a regressor variable, x, and the dependent variable, y. Notice that I said a "simple regression" model. That means that there's just the one regressor (apart from an intercept). Multiple regression models raise all sorts of issues for permutation tests, and we'll get to that in due course.</div>
<div style="text-align: justify;">
<br /></div>
<div style="text-align: justify;">
There are several things that we're going to see here:</div>
<div style="text-align: justify;">
</div>
<ol>
<li style="text-align: justify;">How to construct a randomization test of the hypothesis that the regression slope coefficient is zero.</li>
<li style="text-align: justify;">A demonstration that the permutation test is "exact". That it, its significance level is exactly what we assign it to be.</li>
<li style="text-align: justify;">A comparison between a permutation test and the usual t-test for this problem.</li>
<li style="text-align: justify;">A demonstration that the permutation test remains "exact", even when the regression model is mi-specified by fitting it through the origin.</li>
<li style="text-align: justify;">A comparison of the powers of the randomization test and the t-test under this model mis-specification.</li>
</ol>
<div style="text-align: justify;">
<br />
<br />
<a name='more'></a><br />
<br />
<b><span style="color: #cc0000;">The Monte Carlo experiment</span></b><br />
<br />
A lot of this information will be revealed by means of a Monte Carlo experiment. The associated R code can be downloaded from the <a href="https://davegiles.blogspot.com/p/code.html">code page</a> for this blog. (Even if you're not an R user, you can read the code file with any text editor, and there are lots of comments in it to help you.)</div>
<div style="text-align: justify;">
<br /></div>
<div style="text-align: justify;">
Now, let's get down to business. First, let's look at the regression model that we'll be working with. It's of the form:</div>
<div style="text-align: justify;">
<br /></div>
<div style="text-align: justify;">
y<sub>i</sub> = β<sub>1</sub> + β<sub>2</sub> x<sub>i</sub> + ε<sub>i</sub> ; i = 1, 2, 3, ...., n. (1)</div>
<div style="text-align: justify;">
<br /></div>
<div style="text-align: justify;">
The regressor, x, is non-random, but apart from that <i>we're not really going to make any assumptions </i>beyond "exchangeability" of the sample observations. (See my <a href="https://davegiles.blogspot.com/2019/04/what-is-permutation-test.html">previous post</a> for an explanation of this.)</div>
<div style="text-align: justify;">
<br /></div>
<div style="text-align: justify;">
If (1) is the Data-Generating Process (DGP), as well as the form of the model that is estimated by OLS, then there is no model mis-specification. On the other hand if (1) is the DGP, but we "under-fit" the regression by omitting the intercept, then we have a mis-specified model. In the latter case, we know that this has adverse implications for the OLS estimator and for the <i>conventional</i> tests that we conduct.</div>
<div style="text-align: justify;">
<br /></div>
<div style="text-align: justify;">
You'll know by now that to conduct a permutation test we have to choose a suitable test statistic, S. Given the null hypothesis that we want to test, one option would be to choose S = b<sub>2</sub>, where b<sub>2</sub> is the OLS estimator for β<sub>2</sub>. That would be just fine, even though it's not a pivotal statistic. Then we could keep the ordering of the y values in the sample, and consider permutations of the x data (or <i>vice versa</i> - it wouldn't matter). See Edgington (1987) or Noreen (1989).</div>
</div>
<div style="text-align: justify;">
<br /></div>
<div style="text-align: justify;">
Alternatively, we could recall that b<sub>2</sub> is proportional to the (Pearson) simple correlation coefficient between y and x. In fact the proportionality "constant" is the ratio of the sample standard deviations of y and x. This ratio stays the same as we permute the sample values. Moreover, for a set of x and y values, the correlation coefficient increases monotonically with b<sub>2</sub>.<br />
<br />
So, all we have to do is to test the <i>null hypothesis of "no correlation"</i>, and this will serve our purpose precisely. <i>The alternative hypothesis will be 2-sided</i>, namely that there is a linear correlation between x and y. This is a problem that was dealt with in Example 2 of my last post on permutation tests, so we know already what's involved.</div>
<div style="text-align: justify;">
<br /></div>
<div style="text-align: justify;">
This time our R code will be a bit more efficient than before. We'll take advantage of Steve Garren's <a href="https://cran.r-project.org/web/packages/jmuOutlier/index.html"><b>muOutlier</b> package</a> for R to implement the permutation test.<br />
<br />
Sample sizes of n = 15, 30, and 60 are are considered separately. (This is hardly a situation where we can appeal to "large-sample asymptotics".) The number of permutations are 1.308x10<sup>12</sup>, 2.653x10<sup>32</sup>, and 8.321x10<sup>81</sup> for n = 15, 30, and 60 respectively. Obviously, we'll just use a random selection of these possible permutations! For n = 15, 30 we'll use 2,000 selections; and we'll use 5,000 selections for n = 60. A bit of experimentation shows that these numbers are sufficient for the accuracy that we want, as is the number of Monte Carlo replications that we'll use, namely 2,000.<br />
<br /></div>
<div style="text-align: justify;">
Within each replication of the Monte Carlo experiment what we do is construct and apply the randomization test. We compute its p-value and then keep track of those p-values that are less than or equal to our desired significance level, which will be 5%. If the null hypothesis is true, the observed fraction of such p-values over our 2,000 Monte Carlo replications will be 5%. (Any departure from this value would be due to using too few replications, or two few randomly chosen permutations.)</div>
<div style="text-align: justify;">
<br />
At each replication of the experiment we also apply the usual t-test for a zero slope. Note that the latter is based on lots of assumptions - such as errors that independent, homoskedastic, and Normally distributed - whereas the permutation test is free of these.<br />
<br />
<b><span style="color: #cc0000;">Significance levels</span></b><br />
<br />
To get a "fair" comparison between the two type of tests at the outset, the DGP in (1) uses random disturbances that are i.i.d. Normal, with a zero mean and a constant variance. So, When the null hypothesis is true and the model that is estimation is correctly specified, the t-test should reject the null hypothesis in 5% of the replications of the experiment if we use the appropriate Student-t critical value(s).<br />
<br />
So, with this in mind, let's take a look at a partial set of our results, for the case where n = 30. In Table 1, <i><b>the regression is fitted through the origin</b>, </i>so unless the value of β<sub>1 </sub>in the DGP is zero, the estimated model would be mis-specified:<br />
<br />
<div style="text-align: center;">
<b>Table 1</b></div>
</div>
<div style="text-align: justify;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiMVpJh9z33u0dYyuroJ_QLAJ4nDhbN2XN0z2U3zSREsIDPU6C0Js4gP9Z-apOHaXkIWvib2SjpPQRca2u6q1CtN6JCdl45moNYyzKx3v9OWQRFTnoeaq_vO0i-ism41jt6UrITPwahGvBl/s1600/regression+blog+2.JPG" imageanchor="1" style="margin-left: 1em; margin-right: 1em; text-align: center;"><img border="0" data-original-height="349" data-original-width="432" height="258" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiMVpJh9z33u0dYyuroJ_QLAJ4nDhbN2XN0z2U3zSREsIDPU6C0Js4gP9Z-apOHaXkIWvib2SjpPQRca2u6q1CtN6JCdl45moNYyzKx3v9OWQRFTnoeaq_vO0i-ism41jt6UrITPwahGvBl/s320/regression+blog+2.JPG" width="320" /></a></div>
<div style="text-align: justify;">
<br />
We'll come back to this table shortly, but for now just focus on the row that is highlighted in light green. This corresponds to the case where both β<sub>1</sub> and β<sub>2</sub> are zero. That is, the DGP in (1) has no intercept, so the model is correctly specified, and the null hypothesis of a zero slope is true.<br />
<br />
Because the null hypothesis is true, the power of the test is just its significance level (5%). Notice that the reported empirical significance levels match the anticipated 5%!<br />
<br />
This is good news. Given the particular errors that were used in the DGP in the simulations, <i>this had to happen for the t-test</i>. However, the result for the randomization test could be misleading. Why? Well. we could obtain the 5% rejection rate correctly, even if the distribution of the p-values from which this rate was calculated is "weird".<br />
<br />
In <a href="https://davegiles.blogspot.com/2011/04/may-i-show-you-my-collection-of-p.html">an old post</a> on this blog I discussed the sampling distribution of a p-value. One point that I covered was that if the null hypothesis is true, then this sampling distribution <i>has to be Uniform</i> on [0 , 1], regardless of the testing problem! This result gives another way of checking if the code for our simulation experiment is performing accurately, and that we've used enough replications and random selections of the permutations.</div>
<div style="text-align: justify;">
<br /></div>
<div style="text-align: justify;">
Figure 2 shows the distribution of the 2,000 p-values for the permutation test, <i>for the situation in the first row of results in Table 1</i>:<br />
<br />
<div style="text-align: center;">
<b>Figure 1</b></div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhyrlvf-Q2ZxhbYSKLrK2ai1oVcw55C2F10gk0hhVm8M6XRB8wfivmk6amiDzlHgDZhO6wiDPCZm435KJLQE4bvoJEe9Z5QfQAdRFCc06v5lhjUlR6fSHniGx2-mkID5uQGhRuazX861xNE/s1600/regression+blog+1.JPG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="635" data-original-width="634" height="400" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhyrlvf-Q2ZxhbYSKLrK2ai1oVcw55C2F10gk0hhVm8M6XRB8wfivmk6amiDzlHgDZhO6wiDPCZm435KJLQE4bvoJEe9Z5QfQAdRFCc06v5lhjUlR6fSHniGx2-mkID5uQGhRuazX861xNE/s400/regression+blog+1.JPG" width="398" /></a></div>
<span style="text-align: left;"><br /></span>
<span style="text-align: left;">As we can see, this distribution is "reasonably uniform", as required. More formally, using the <b><a href="https://cran.r-project.org/web/packages/uniftest/">uniftest</a></b> package in R we find that the Kolmogorov-Smirnov test statistic for uniformity is D = 0.998 (p = 0.24); and the Kuiper test statistic is V = 1.961 (p = 0.23). </span><span style="text-align: left;">So, we seem to be in good shape here.</span><br />
<span style="text-align: left;"><br /></span>
<span style="text-align: left;">So far, all that we seem to have shown is that the permutation test and the t-test exhibit no "size-distortion" when the model is correctly specified, and the errors satisfy the assumptions needed for the t-test to be valid. Well, whoopee!</span><br />
<span style="text-align: left;"><br /></span>
<span style="text-align: left;">Let's take a look back at Table 1, and now focus on the second line of results (highlighted in orange). Recall that the estimated model omits the intercept - the model is fitted through the origin. However, now, in Table 1, </span>β<sub>1 </sub><span style="text-align: left;">= 1, so the DGP includes an intercept and the fitted model is under-specified. <i>We've omitted a (constant) regressor from the estimated model</i>.</span><br />
<span style="text-align: left;"><br /></span>
<span style="text-align: left;">In this case the usual t-statistic follows a non-central Student-t distribution, with a non-centrality parameter that increases monotonically with </span>β<sub>1</sub><span style="text-align: left;"><sup>2</sup>, and which depends on the x data and the variance of the error term. <i>It's value is unobservable</i>! And w</span><span style="text-align: left;">e can't use the critical value(s) from the non-central t distribution if we don't know the value of the non-centrality parameter.</span><br />
<span style="text-align: left;"><br /></span>
<span style="text-align: left;">Obviously, the usual (central) Student-t critical values are no longer correct, and the <i>observed</i> significance level (the rejection rate of the null in the experiment when the null is true) will differ from 5%. Depending on the situation, it may be less than 5%, or greater than 5%. The extent of this difference is the "size-distortion" associated with the test when we mis-apply it in this way. </span><br />
<span style="text-align: left;"><br /></span>
<span style="text-align: left;">In the second line of Table 1 we see two important things. First, the t-test has a downwards size-distortion. We wanted to apply the test at the 5% significance level, but in fact it only rejected the null, when it was true, 1% of the time! Of course, if we just mis-applied this test once, in an application, we would have no idea if there was any substantial size-distortion or not.</span><br />
<span style="text-align: left;"><br /></span>
<span style="text-align: left;">The second (<i><b>really neat</b></i>) thing that we see in the second line of that table is that the permutation test still has a significance level of 5%! Even though the model is mis-specified, this doesn't affect the test - at least in terms of it still being "exact" with respect to the significance level that we wanted to achieve. </span><br />
<span style="text-align: left;"><i><b><br /></b></i></span>
<span style="text-align: left;"><i><b>And this is a totally general result</b></i>!</span><br />
<span style="text-align: left;"><br /></span>
<span style="color: #cc0000; text-align: left;"><b>Power considerations</b></span><br />
<br /></div>
<div style="text-align: justify;">
So much for the significance level, and size-distortion. What about the powers of the tests that we're considering?</div>
<div style="text-align: justify;">
<br /></div>
<div style="text-align: justify;">
If we move down Table 1 we still have a mis-specified model (because <span style="text-align: justify;">β</span><sub style="text-align: justify;">1 </sub>≠ 0), but the value of the slope coefficient is increasing. That is, the null hypothesis is becoming <i>more and more false</i>. We're now looking at the power functions for the tests. Because the size of the t-test is distorted downwards, it's not surprising that this test has lower power than the permutation test in this particular case. (More generally, that need not be the case - the power curves could intersect at some point.)</div>
<br />
<div style="text-align: justify;">
Figures 2 and 3 show the corresponding power curves when n = 15, and n = 60. Note that the ranges of the horizontal axes are different in each Figures 1 to 3. This reflects the fact that both tests are "consistent", and so their powers approach the value "1" (when the null becomes increasingly false) more rapidly as n grows.</div>
<div style="text-align: justify;">
<br /></div>
<div style="text-align: justify;">
Also, note that while the results in Figures 2 are similar to those seen in Table 1, those in Figure 3 are quite different. In the latter case the size-distortion for the t-test is<i> upwards (</i>at 7.8%) and its power curve lies <i>above</i> that of the permutation test. It's not that the t-test is (necessarily) more powerful than the the permutation test. We can't compare (true) power unless the two tests have the same significance level - and they don't in this case.</div>
<div class="separator" style="clear: both; text-align: center;">
<br /></div>
<div class="separator" style="clear: both; text-align: center;">
<b>Figure 2</b></div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgK_EjoVWXg2cx2r3OA7gt174Wmz10rEjTI38ropazGDG5aTHi1CXACdbvvH2zPP_kMeqHYCTJ9a3QwmyxTzJT6XWNbjvThIyKARwWxDnmmid22a7nZCL1nMb4lJc0TEv7iluKwpulHIJsS/s1600/Reegress+power+n15.JPG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="631" data-original-width="640" height="393" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgK_EjoVWXg2cx2r3OA7gt174Wmz10rEjTI38ropazGDG5aTHi1CXACdbvvH2zPP_kMeqHYCTJ9a3QwmyxTzJT6XWNbjvThIyKARwWxDnmmid22a7nZCL1nMb4lJc0TEv7iluKwpulHIJsS/s400/Reegress+power+n15.JPG" width="400" /></a></div>
<div class="separator" style="clear: both; text-align: center;">
<b><br /></b></div>
<div class="separator" style="clear: both; text-align: center;">
<b>Figure 3</b></div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjbxdQkhufLrOcHQJETd8tlLp2U2DtHcdQvtjv8oQstcRgeLt8t2scxKtfsVpiti0sZDdnGp3JAWwiIyDj_0K_v7RKe7mOdt-VglZg4krPpOtT_T6phlk4yt_7enZigzidY_uFgxBGHURsn/s1600/Reegress+power+n60.JPG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="633" data-original-width="657" height="385" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjbxdQkhufLrOcHQJETd8tlLp2U2DtHcdQvtjv8oQstcRgeLt8t2scxKtfsVpiti0sZDdnGp3JAWwiIyDj_0K_v7RKe7mOdt-VglZg4krPpOtT_T6phlk4yt_7enZigzidY_uFgxBGHURsn/s400/Reegress+power+n60.JPG" width="400" /></a></div>
<div class="separator" style="clear: both; text-align: center;">
<br /></div>
<div class="separator" style="clear: both; text-align: center;">
</div>
<div style="text-align: justify;">
The detailed results from the Monte Carlo experiment on which Table 1 and Figures 2 and 3 are based can be found in an Excel spreadsheet on the <a href="https://davegiles.blogspot.com/p/data.html">data page</a> for this blog.</div>
<div style="text-align: justify;">
<br />
<b><span style="color: #cc0000;">Be careful!</span></b><br />
<br /></div>
<div style="text-align: justify;">
Three caveats are in order here:<br />
<br />
<b>1.</b> We set <span style="text-align: justify;">β</span><sub style="text-align: justify;">1 </sub>= 1 in the DGP and this resulted in a mis-specified "fitted" regression model. This chosen value for <span style="text-align: justify;">β</span><sub style="text-align: justify;">1 </sub>affects the (numerical) results. The <i>magnitudes</i> of the size distortions and the powers are specific to this choice.<br />
<br />
<b>2.</b> We need to be careful, here, when we talk about comparisons between the powers of the two tests (and this comment is universally applicable). Strictly, power comparisons are valid when the tests have the same (actual, empirical) significance level. The only exception is when Test A has <i>lower </i>actual significance level than Test B, but Test A has a <i>higher</i> rejection rate than Test B when the null is false (<i>i.e.</i>, higher "raw", or apparent, power") over the full parameter space. Then, Test A is more powerful than Test B.<br />
<br />
In all other cases where there is size distortion, the "true" power will not be clear. One way to deal with this is to "size-adjust" any test that exhibits size distortion. This would be done, in our Monte Carlo experiment, by "jiggling" the critical values used for the t-test to ones that ensure that the test has a 5% rejection rate when the null is true. Then we could proceed to generate the corresponding power curves and make valid comparisons.<br />
<br />
That's fine, but of course <i>in practice</i> we wouldn't know what modified critical values to use (unless we conducted a Monte Carlo experiment every time we undertook a real-life application). Perhaps this is worthy of another blog post at some stage, perhaps as an addition to my earlier ones on the basics of Monte Carlo simulation - <a href="https://davegiles.blogspot.com/2013/04/a-first-encounter-with-monte-carlo.html">here</a>, <a href="https://davegiles.blogspot.com/2016/11/monte-carlo-simulation-basics-i.html">here</a>, and <a href="https://davegiles.blogspot.com/2016/11/monte-carlo-simulation-basics-ii.html">here</a>.<br />
<br />
<b>3.</b> As I stressed from the outset, model (1) is only a <i>simple</i> regression model. Multiple regression models are much more interesting, but this is where things get a lot trickier when it comes to constructing an <i>exact</i> permutation test. Quite a lot has been written about this problem. Some of the issues are discussed quite nicely by Kennedy (1995), but the (statistics) literature has moved along a bit since then and some of his suggestions are no longer supported. Some key references include Anderson and ter Braak (2003), Huh and Jhun (2001), Kim <i>et al</i>. (2000), LePage and Podgórski (1996), Oja (1987), and Schmoyer (1994).<br />
<br />
Anderson and Robinson (2001) provide simulation evidence that favours the particular permutation procedure suggested by Freedman and Lane (1983). More recently, Nyblom (2015) introduces a permutation procedure based on the regression residuals, and illustrates its application to problems involving tests for autocorrelation, heteroskedasticity, and structural breaks.<br />
<br /></div>
<span style="color: #cc0000;"><b>Summing up</b></span><br />
<br />
Here are the takeaways from this post:<br />
<br />
<ul style="text-align: left;">
<li style="text-align: justify;">Permutation tests are nonparametric, or "distribution free". Unlike the usual tests that we use in econometric, we don't need to satisfy a whole lot of (possibly questionable) parametric and distributional assumptions.</li>
<li style="text-align: justify;">Permutation tests are easy to apply, even though they can be (moderately) computationally intensive.</li>
<li style="text-align: justify;">Permutation tests are "exact", in the sense that achieve precisely the significance level that we want them to. There is no "size distortion".</li>
<li style="text-align: justify;">Usually, we have lots of legitimate, and simple, choices for the test statistic in any given problem. In each case, the test will be exact, though its power may vary depending on our choice.</li>
<li style="text-align: justify;">A permutation test will still be exact, even if the model is mis-specified. This stands in stark contrast to the usual parametric tests that we use.</li>
<li style="text-align: justify;">A permutation test may be more, or less, powerful than its parametric counterparts, depending on the situation.</li>
<li style="text-align: justify;">If you plan to use permutation (randomization) tests in the context of the <i>multiple</i> regression model (or its extensions), then their are some pitfalls that you need to be aware of. Be sure to look at the references that I've supplied.</li>
</ul>
<br />
Other than that - Happy Testing!<br />
<br />
<div style="text-align: justify;">
<b>References</b></div>
<div style="text-align: justify;">
<br />
<div style="-webkit-text-stroke-width: 0px; color: black; font-family: "Times New Roman"; font-size: medium; font-style: normal; font-variant-caps: normal; font-variant-ligatures: normal; font-weight: 400; letter-spacing: normal; orphans: 2; text-align: justify; text-decoration-color: initial; text-decoration-style: initial; text-indent: 0px; text-transform: none; white-space: normal; widows: 2; word-spacing: 0px;">
<div style="margin: 0px;">
<a href="https://onlinelibrary.wiley.com/doi/abs/10.1111/1467-842X.00156">Anderson, M. J. & J. Robinson</a>, 2001. Permutation tests for linear models. <i>Australian and New Zealand Journal of Statistics</i>, 43, 75-88. </div>
</div>
<br />
<a href="https://www.uvm.edu/~dhowell/StatPages/R/RandomizationTestsWithR/andersonTerBrack.pdf">Anderson, M. J., & C. J. F. ter Braak</a>, 2003. Permutation tests for multi-factorial analysis of variance. <i>Journal of Statistical Computation and Simulation</i>, 73, 85–113.<br />
<br />
<a href="https://www.amazon.com/Randomization-Tests-Statistics-Textbooks-Monographs/dp/0824768787/ref=sr_1_3?keywords=E.+S.+Edgington&qid=1554666451&s=gateway&sr=8-3">Edgington, E. S.</a>, 1987. <i>Randomization Tests</i>. Marcel Dekker, New York.<br />
<br />
<a href="https://amstat.tandfonline.com/doi/abs/10.1080/07350015.1983.10509354#.XKpQH5hKhPY">Freedman, D. & D. Lane</a>, 1983. A nonstochastic interpretation of reported significance levels. <i>Journal of Business and Economic Statistics</i>, 1, 292-298.<br />
<br />
<a href="https://www.tandfonline.com/doi/abs/10.1081/STA-100106060">Huh, M-H. & M. Jhun</a>, 2001. Random permutation testing in multiple linear regression. <i>Communications in Statistics - Theory Methods</i>, 30, 2023–2032.<br />
<br />
<div style="-webkit-text-stroke-width: 0px; color: black; font-family: "Times New Roman"; font-size: medium; font-style: normal; font-variant-caps: normal; font-variant-ligatures: normal; font-weight: 400; letter-spacing: normal; orphans: 2; text-align: justify; text-decoration-color: initial; text-decoration-style: initial; text-indent: 0px; text-transform: none; white-space: normal; widows: 2; word-spacing: 0px;">
<div style="margin: 0px;">
<a href="https://amstat.tandfonline.com/doi/abs/10.1080/07350015.1995.10524581#.XKpQiZhKhPY">Kennedy, P. E.</a>, 1995. Randomization tests in econometrics. <i>Journal of Business and Economic Statistics</i>, 13, 85-94.</div>
</div>
<br />
<a href="https://onlinelibrary.wiley.com/doi/abs/10.1002/sim.811">Kim, H.-J., M. P. Fay, E. J. Feuer, & D. Midthune</a>, 2000. Permutation tests for jointpoint regression with applications to cancer rates. <i>Statistics in Medicine,</i> 19, 335–351.<br />
<br />
<a href="https://core.ac.uk/download/pdf/82336230.pdf">LePage, R. & K. Podgórski</a>, 1996. Resampling permutations in regression without second moments. <i>Journal of Multivariate Analysis</i>, 57, 119–141.<br />
<br />
<a href="https://www.amazon.com/Computer-Intensive-Methods-Testing-Hypotheses-Introduction/dp/B01JXSBCXI/ref=sr_1_fkmr0_1?keywords=Noreen%2C+E.+W.%2C+1989.+Computer+Intensive+Methods+for+Testing+Hypotheses%3A+An+Introduction.+Wiley%2C+New+York.&qid=1554666381&s=gateway&sr=8-1-fkmr0">Noreen, E. W.</a>, 1989. <i>Computer Intensive Methods for Testing Hypotheses: An Introduction</i>. Wiley, New York.<br />
<br />
<a href="https://link.springer.com/chapter/10.1007/978-3-319-22404-6_5">Nyblom, J.</a>, 2015. Permutation tests in linear regression. Chapter 5 in K. Nordhausen & S. Taskinen (eds.), <i>Modern Nonparametric, Robust and Multivariate Methods</i>. Springer International, Switzerland.<br />
<br />
<a href="https://onlinelibrary.wiley.com/doi/abs/10.1111/j.1467-842X.1987.tb00724.x">Oja, H.</a>, 1987. On permutation tests in multiple regression and analysis of covariance problems. <i>Australian Journal of Statistics</i>, 29, 91–100.<br />
<br />
<a href="https://www.jstor.org/stable/2291013?seq=1#page_scan_tab_contents">Schmoyer, R. L.</a>, 1994. Permutation tests for correlation in regression errors. <i>Journal of the American Statistical Association</i>, 89, 1507–1516.<br />
<br /></div>
<center>
© 2019, David E. Giles
</center>
</div>
Dave Gileshttp://www.blogger.com/profile/05389606956062019445noreply@blogger.com3