May Reading List

Here's a selection of suggested reading for this month:

• Athey, S. & G. W. Imbens, 2019. Machine learning methods economists should know about. Mimeo.
• Bhagwat, P. & E. Marchand, 2019. On a proper Bayes but inadmissible estimator. American Statistician, online.
• Canals, C. & A. Canals, 2019. When is n large enough? Looking for the right sample size to estimate proportions. Journal of Statistical Computation and Simulation, 89, 1887-1898.
• Cavaliere, G. & A. Rahbek, 2019. A primer on bootstrap testing of hypotheses in time series models: With an application to double autoregressive models. Discussion Paper 19-03, Department of Economics, University of Copenhagen.
• Chudik, A. & G. Geogiardis, 2019. Estimation of impulse response functions when shocks are observed at a higher frequency than outcome variables. Globalization Institute Working Paper 356, Federal Reserve Bank of Dallas.
• Reschenhofer, E., 2019. Heteroscedasticity-robust estimation of autocorrelation. Communications in Statistics - Simulation and Computation, 48, 1251-1263.

© 2019, David E. Giles

Specification Testing With Very Large Samples

I received the following email query a while back:

"It's my understanding that in the event that you have a large sample size (in my case, > 2million obs) many tests for functional form mis-specification will report statistically significant results purely on the basis that the sample size is large. In this situation, how can one reasonably test for misspecification?" 

Well, to begin with, that's absolutely correct - if the sample size is very, very large then almost any null hypothesis will be rejected (at conventional significance levels). For instance, see this earlier post of mine.

Schmueli (2012) also addresses this point from the p-value perspective.

But the question was, what can we do in this situation if we want to test for functional form mis-specification?

Schmueli offers some general suggestions that could be applied to this specific question:

1. Present effect sizes.
2. Report confidence intervals.
3. Use (certain types of) charts

This is followed with an empirical example relating toauction prices for camera sales on eBay, using a sample size of n = 341,136.

To this, I'd add, consider alternative functional forms and use ex post forecast performance and cross-validation to choose a preferred functional form for your model.

You don't always have to use conventional hypothesis testing for this purpose.

Reference

Schmueli, G., 2012. Too big to fail: Large samples and the p-value problem. Mimeo., Institute of Service Science, National Tsing Hua University, Taiwan.

© 2016, David E. Giles

My Latest Paper About Dummy Variables

Over the years I've posted a number of times about various aspects of using dummy variables in regression models. You can use the "Search" window in the right sidebar of this page if want to take a look at those posts.

One of my earlier working papers on this topic has now been accepted for publication.

The paper is titled, "On the Inconsistency of Instrumental Variables Estimators for the Coefficients of Certain Dummy Variables". Here's the abstract:

"In this paper we consider the asymptotic properties of the Instrumental Variables (IV) estimator of the parameters in a linear regression model with some random regressors, and other regressors that are dummy variables. The latter have the special property that the number of non-zero values is fixed, and does not increase with the sample size. We prove that the IV estimator of the coefficient vector for the dummy variables is inconsistent, while that for the other regressors is weakly consistent under standard assumptions. However, the usual estimator for the asymptotic covariance matrix of the I.V. estimator for all of the coefficients retains its usual consistency. The t-test statistics for the dummy variable coefficients are still asymptotically standard normal, despite the inconsistency of the associated IV coefficient estimator. These results extend the earlier results of Hendry and Santos (2005), which relate to a fixed-regressor model, in which the dummy variables are non-zero for just a single observation, and OLS estimation is used".

You can download the final working paper version of the paper from here.

The paper will be appearing in an upcoming issue of Journal of Quantitative Economics.

© 2016, David E. Giles

End-of-Year Reading

Wishing all readers a very special holiday season!

• Agiakloglou, C., and C. Agiropoulos, 2016. The balance between size and power in testing for linear association for two stationary AR(1) processes. Applied Economics Letters, 23, 230-234.
• Allen, D., M. McAleer, S. Peiris, and A. K. Singh, 2015. Nonlinear time series and neural-network models of exchange rates between the US dollar and major currencies. Discussion Paper No. 15-125/III, Tinbergen Institute.
• Basu, D., 2015. Asymptotic bias of OLS in the presence of reverse causality. Working Paper 2015-18, Department of Economics, University of Massachusetts, Amherst.
• Giles, D. E., 2005. Testing for a Santa Claus effect in growth cycles. Economics Letters, 87, 421-426.
• Kim, J., and I Choi, 2015. Unit roots in economic and financial time series: A re-evaluation based on enlightened judgement. MPRA Paper No. 68411.
• Triacca, U., 2015. A pitfall in using the characterization of Granger non-causality in vector autoregressive models. Econometrics, 3, 233-239.       

© 2015, David E. Giles

Illustrating Spurious Regressions

I've talked a bit about spurious regressions a bit in some earlier posts (here and here). I was updating an example for my time-series course the other day, and I thought that some readers might find it useful.

Let's begin by reviewing what is usually meant when we talk about a "spurious regression".

In short, it arises when we have several non-stationary time-series variables, which are not cointegrated, and we regress one of these variables on the others.

In general, the result that we get are nonsensical, and the problem is only worsened if we increase the sample size. This phenomenon was observed by Granger and Newbold (1974), and others, and Phillips (1986) developed the asymptotic theory that he then used to prove that in a spurious regression the Durbin-Watson statistic converges in probability to zero; the OLS parameter estimators and R² converge to non-standard limiting distributions; and the t-ratios and F-statistic diverge in distribution, as T ↑ ∞ .

Let's look at some of these results associated with spurious regressions. We'll do so by means of a simple simulation experiment.

Logit, Probit, & Heteroskedasticity

I've blogged previously about specification testing in the context of Logit and Probit models. For instance, see here and here. 

Testing for homoskedasticity in these models is especially important, for reasons that are outlined in those earlier posts. I won't repeat all of the details here, but I'll just note that heteroskedasticity renders the MLE of the parameters inconsistent. (This stands in contrast to the situation in, say, the linear regression model where the MLE of the parameters is inefficient, but still consistent in this case.)

If you're an EViews user, you can find my code for implementing a range of specification tests for Logit and Probit models here. These include the LM test for homoskedasticity that was proposed by Davidson and MacKinnon (1984).

More than once, I've been asked the following question:

"When estimating a Logit or Probit model, we set the scale parameter (variance) of the error term to the value one, because it's not actually identifiable. So, in what sense can we have heteroskedasticity in such models?"

This is a good question, and I thought that a short post would be justified. Let's take a look:

Reading for the Merry Month of May

While you're dancing around the Maypole (or whatever else it is that you get up to), my recommendations are:

• Claeskens, G., J. Magnus, A. Vasnev, and W. Wang, 2014. The forecast combination puzzle: A simple theoretical explanation. Tinbergen Institute Discussion Paper TI 2014 - 127/III. 
• de Jong, R. M. and M. Sakarya, 2013. The econometrics of the Hodrick-Prescott filter. Forthcoming in Review of Economics and Statistics.
• Honoré, B. E. and L. Hu, 2015. Poor (wo)man’s bootstrap. Working Paper 2015-01, Federal Reserve Bank of Chicago.
• King, M. L. and S. Sriananthakumar, 2015. Point optimal testing: A survey of the post 1987 literature. Working Paper 05/15, Department of Econometrics and Business Statistics, Monash University.
• Meintanis, S. G. and E. Tsionas, 2015. Approximately distribution-free diagnostic tests for regressions with survival data. Statistical Theory and Practice, 9, 479-488. 
• Piironen, J. and A. Vehtari, 2015. Comparison of Bayesian predictive methods for model selection. Mimeo.
• Yu, P., 2015. Consistency of the least squares estimator in threshold regression with endogeneity. Economics Letters, 131, 41-46.

© 2015, David E. Giles

Econometricians' Debt to Alan Turing

The other day, Carol and I went with friends to see the movie, The Imitation Game. I definitely recommend it.

I was previously aware of many of Alan Turing's contributions, especially in relation to the Turing Machine, cryptography, computing, and artificial intelligence. However, I hadn't realized the extent of Turing's use of, and contributions to, a range of important statistical tools. Some of these tools have a direct bearing on Econometrics.

For example:

• (HT to Lief Bluck for this one.) In 1935, at the tender age of 22, Turing was appointed a Fellow at King's College, Cambridge, on the basis of his 1934 (undergraduate) thesis in which he proved the Central Limit Theorem. More specifically, he derived a proof of what we now call the Lindeberg-Lévy Central Limit Theorem.  He was not aware of Lindeberg's earlier work (1920-1922) on this problem. Lindeberg, in turn, was unaware of Lyapunov's earlier results. (Hint: there was no internet back then!). How many times has your econometrics instructor waved her/his arms and muttered ".......as a result of the central limit theorem....."?
• In 1939, Turing developed what Wald and his collaborators would later call "sequential analysis". Yes, that's Abraham Wald who's associated with the Wald tests that you use all of the time.Turing's wartime work on this subject remained classified until the 1980's. Wald's work became well-established in the literature by the late 1940's, and was included in the statistics courses that I took as a student in the 1960's. Did I mention that Wald's wartime associates included some familiar names from economics? Namely, Trygve Haavelmo, Harold Hotelling, Jacob Marschak, Milton Friedman, W. Allen Wallis, and Kenneth Arrow.
• The mathematician/statistician I. J. ("Jack") Good was a member of Turing's team at Bletchley Park that cracked the Enigma code. Good was hugely influential in the development of modern Bayesian methods, many of which have found their way into econometrics. He described the use of Bayesian inference in the Enigma project in his "conversation" with Banks (1996). (This work also gave us the Good-Turing estimator - e.g., see Good, 1953.)
• Turing (1948) devised the LU ("Lower and Upper") Decomposition that is widely used for matrix inversion and for solving systems of linear equations. Just think how many times you invert matrices when you're doing your econometrics, and how important it is that the calculations are both fast and accurate!

Added, 20 February, 2015: I have recently become aware of Good (1979)

References

Banks, D. L., 1996. A conversation with I. J. Good. Statistical Science, 11, 1-19.

Good, I. J., 1953.The population frequencies of species and the estimation of population parameters. Biometrika, 40, 237-264.

Good, I. J., 1979. A. M. Turing's statistical work in World War II. Biometrika, 66, 393-396.

Turing, A. M., 1948. Rounding-off errors in matrix processes. Quarterly Journal of Mechanics and Applied Mathematics, 1, 287-308.

© 2014, David E. Giles

Normality Testing & Non-Stationary Data

Bob Jensen emailed me about my recent post about the way in which the Jarque-Bera test can be impacted when temporally aggregated data are used. Apparently he publicized my post on the listserv for Accounting Educators in the U.S.. He also drew my attention to a paper from Two former presidents of the AAA: "Some Methodological Deficiencies in Empirical Research Articles in Accounting", by Thomas R. Dyckman and Stephen A. Zeff, Accounting Horizons, September 2014, 28 (3), 695-712. (Here.) 

Bob commented that an even more important issue might be that our data may be non-stationary. Indeed, this is always something that should concern us, and regular readers of this blog will know that non-stationary data, cointegration, and the like have been the subject of a lot of my posts.

In fact, the impact of unit roots on the Jarque-Bera test was mentioned in this old post about "spurious regressions". There, I mentioned a paper of mine (Giles, 2007) in which I proved that:

Illustrating Asymptotic Behaviour - Part III

This is the third in a sequence of posts about some basic concepts relating to large-sample asymptotics and the linear regression model. The first two posts (here and here) dealt with items 1 and 2 in the following list, and you'll find it helpful to read them before proceeding with this post:

1. The consistency of the OLS estimator in a situation where it's known to be biased in small samples.
2. The correct way to think about the asymptotic distribution of the OLS estimator.
3. A comparison of the OLS estimator and another estimator, in terms of asymptotic efficiency.

Here, we're going to deal with item 3, again via a small Monte Carlo experiment, using EViews.

Illustrating Asymptotic Behaviour - Part II

This is the second in a sequence of three posts that deal with large-sample asymptotics - especially in the context of the linear regression model. The first post dealt with item 1 in this list:

1. The consistency of the OLS estimator in a situation where it's known to be biased in small samples.
2. The correct way to think about the asymptotic distribution of the OLS estimator.
3. A comparison of the OLS estimator and another estimator, in terms of asymptotic efficiency.

No surprise, but this post deals with item 2. To get the most out of it, I strongly recommend reading the first post before proceeding.

Illustrating Asymptotic Behaviour - Part I

Learning the basics about the (large sample) asymptotic behaviour of estimators and test statistics is always a challenge. Teaching this material can be challenging too!

So, in this post and in two more to follow, I'm going to talk about a small Monte Carlo experiment that illustrates some aspects of the asymptotic behaviour of the OLS estimator. I'll focus on three things:

1. The consistency of the OLS estimator in a situation where it's known to be biased in small samples.
2. The correct way to think about the asymptotic distribution of the OLS estimator.
3. A comparison of the OLS estimator and another estimator, in terms of asymptotic efficiency.

The (Non-) Standard Asymptotics of Dickey-Fuller Tests

One of the most widely used tests in econometrics is the (augmented) Dickey-Fuller (DF) test. We use it in the context of time series data to test the null hypothesis that a series has a unit root (i.e., it is I(1)), against the alternative hypothesis that the series is I(0), and hence stationary. If we apply the test to a first-differenced time series, then the null is that the series is I(2), and the alternative hypothesis is that it is I(1), and so on.

Suppose that the time series in question is {Y_t; t = 1, 2, 3, ......, T}. The so-called "Dickey-Fuller regression" is a least squares regression of the form:

                           ΔY_t = [α + β t] + γY_t-1 + [Σ δ_j ΔY_t-j] + ε_t   .                 (1)

Here, terms in square brackets are optional; and of these the "p" ΔY_t-j terms are the "augmentation terms", whose role is to ensure that the there is no autocorrelation in the equation's residuals.

Standard econometrics packages allow for three versions of (1):

• No drift - no trend: that is, the (α + β t) terms are omitted.
• Drift - no trend: the intercept (drift term) is included, but the linear trend term is not.
• Drift - and - trend: both of the α and (β t) terms are included.

For example, here's the dialogue box that you see when you go to apply the DF test using the EViews package:

March Madness in the Reading Department

It's time for the monthly round-up of recommended reading material.

• Gan, L. and J. Jiang, 1999. A test for global maximum. Journal of the American Statistical Association, 94, 847-854.
• Nowak-Lehmann, F., D. Herzer, S. Vollmer, and I. Martinez-Zarzosa, 2006. Problems in applying dynamic panel data models: Theoretical and empirical findings. Discussion Paper Nr. 140, IAI, Georg-August-Universität Göttingen.
• Olive, D. J., 2004. Does the MLE maximize the likelihood? Mimeo., Department of Mathematics, Southern Illinois University. 
• Pollock, D. S. G., 2014. Econometrics: An historical guide for the uninitiated. Working Paper No. 14/05, Department of Economics, University of Leicester.
• Terrell, G. R., 2002. The gradient statistic. Interface 2002: Computing Science and Statistics, Vol. 34.
• Wald, A., 1940. The fitting of straight lines if both variables are subject to error. Annals of Mathematical Statistics, 11, 284-300.

© 2014, David E. Giles

Unbiased Model Selection Using the Adjusted R-Squared

The coefficient of determination (R²), and its "adjusted" counterpart, really don't impress me much! I often tell students that this statistic is one of the last things I look at when appraising the results of estimating a regression model.

Previously, I've had a few things to say about this measure of goodness-of-fit  (e.g., here and here). In this post I want to say something positive, for once, about "adjusted" R². Specifically, I'm going to talk about its use as a model-selection criterion.

Francis Diebold on GMM

On his blog, No Hesitations, Francis Diebold has two recent posts about GMM estimation that students of econometrics, and practitioners, definitely should read.

The first of these posts is here, and the second follow-up post is here.

Enjoy!

© 2013, David E. Giles

Information Criteria Unveiled

Most of you will have used, or at least encountered, various "information criteria" when estimating a regression model, an ARIMA model, or a VAR model. These criteria provide us with a way of comparing alternative model specifications, and selecting between them. 

They're not test statistics. Rather, they're minus twice the maximized value of the underlying log-likelihood function, adjusted by a "penalty factor" that depends on the number of parameters being estimated. The more parameters, the more complicated is the model, and the greater the penalty factor. For a given level of "fit", a more parsimonious model is rewarded more than a more complex model. Changing the exact form of the penalty factor gives rise to a different information criterion.

However, did you ever stop to ask "why are these called information criteria?" Did you realize that these criteria - which are, after all, statistics - have different properties when it comes to the probability that they will select the correct model specification? In this respect, they are typically biased, and some of them are even inconsistent.

This sounds like something that's worth knowing more about!

ARDL Models - Part II - Bounds Tests

[Note: For an important update of this post, relating to EViews 9, see my 2015 post, here.]

Well, I finally got it done! Some of these posts take more time to prepare than you might think.

The first part of this discussion was covered in a (sort of!) recent post, in which I gave a brief description of Autoregressive Distributed Lag (ARDL) models, together with some historical perspective. Now it's time for us to get down to business and see how these models have come to play a very important role recently in the modelling of non-stationary time-series data.

In particular, we'll see how they're used to implement the so-called "Bounds Tests", to see if long-run relationships are present when we have a group of time-series, some of which may be stationary, while others are not. A detailed worked example, using EViews, is included.

New Paper Published

A paper of mine appears in the latest issue of the Chilean Journal of Statistics. The paper is titled, "Exact asymptotic goodness-of-fit testing for discrete circular data with applications.

I've posted previously about this general topic, here, here and here.

© 2013, David E. Giles

Robust Standard Errors for Nonlinear Models

André Richter wrote to me from Germany, commenting on the reporting of robust standard errors in the context of nonlinear models such as Logit and Probit. He said he 'd been led to believe that this doesn't make much sense. I told him that I agree, and that this is another of my "pet peeves"!

Yes, I do get grumpy about some of the things I see so-called "applied econometricians" doing all of the time. For instance, see my Gripe of the Day post back in 2011. Sometimes I feel as if I could produce a post with that title almost every day!

Anyway, let's get back to André's point.