Reporting an R-Squared Measure for Count Data Models

This post was prompted by an email query that I received some time ago from a reader of this blog. I thought that a more "expansive" response might be of interest to other readers............

In spite of its many limitations, it's standard practice to include the value of the coefficient of determination (R²) - or its "adjusted" counterpart - when reporting the results of a least squares regression. Personally, I think that R² is one of the least important statistics to include in our results, but we all do it. (See this previous post.)

If the regression model in question is linear (in the parameters) and includes an intercept, and if the parameters are estimated by Ordinary Least Squares (OLS), then R² has a number of well-known properties. These include:

1. 0 ≤ R² ≤ 1.
2. The value of R² cannot decrease if we add regressors to the model.
3. The value of R² is the same, whether we define this measure as the ratio of the "explained sum of squares" to the "total sum of squares" (R_E²); or as one minus the ratio of the "residual sum of squares" to the "total sum of squares" (R_R²).
4. There is a correspondence between R² and a significance test on all slope parameters; and there is a correspondence between changes in (the adjusted) R² as regressors are added, and significance tests on the added regressors' coefficients.   (See here and here.)
5. R² has an interpretation in terms of information content of the data.  
6. R² is the square of the (Pearson) correlation (R_C²) between actual and "fitted" values of the model's dependent variable. 

However, as soon as we're dealing with a model that excludes an intercept or is non-linear in the parameters, or we use an estimator other than OLS, none of the above properties are guaranteed.

EViews 11 Now Available

As you'll know already, I'm a big fan of the EViews econometrics package. I always found it to be a terrific, user-friendly, resource when teaching economic statistics and econometrics, and I use it extensively in my own research.

Along with a lot of other EViews users, I recently had the opportunity to "test drive" the beta release of the latest version of this package, EViews 11. 

EViews 11 has now been officially released, and it has some great new features. (Click on the links there to see some really helpful videos.) To see what's now available, check it out here. 

Nice update. Thanks!

© 2019, David E. Giles

What is a Permutation Test?

Permutation tests, which I'll be discussing in this post, aren't that widely used by econometricians. However, they shouldn't be overlooked.

Let's begin with some background discussion to set the scene. This might seem a bit redundant, but it will help us to see how permutation tests differ from the sort of tests that we usually use in econometrics.

Background Motivation

When you took your first course in economic statistics, or econometrics, no doubt you encountered some of the basic concepts associated with testing hypotheses. I'm sure that the first exposure that you had to this was actually in terms of "classical", Neyman-Pearson, testing. 

It probably wasn't described to you in so many words. It would have just been "statistical hypothesis testing". The whole procedure would have been presented, more or less, along the following lines:

Controlling My Heating Bill Using Bayesian Model Averaging

Where we live, in rural Ontario, we're not connected to "natural gas". Our home furnace runs on propane, and a local supplier sends a tanker to refill our propane tanks on a regular basis during the colder months.

Earlier this month we had to make a decision regarding our contract with the propane retailer. Should we opt for a delivery price that can vary, up or down, throughout the coming fall and winter; or should we "lock in" at a fixed delivery price for the period from October to May of next year?

Now, I must confess that my knowledge of the propane industry is slight, to say the least. I decided that a basic analysis of the historical propane price data might provide some insights to assist in making this decision. It also occurred to me, after doing this, that the analysis that I went through might be of interest to readers, as a simple exercise in forecasting using Bayesian model averaging.

Here are the details...........

Interpolating Statistical Tables

We've all experienced it. You go to use a statistical table - Standard Normal, Student-t, F, Chi Square - and the line that you need simply isn't there in the table. That's to say the table simply isn't detailed enough for our purposes.

One question that always comes up when students are first being introduced to such tables is:

"Do I just interpolate linearly between the nearest entries on either side of the desired value?"

Not that these exact words are used, typically. For instance, a student might ask if they should take the average of the two closest values. How should you respond?

The EViews Blog on ARDL - Part 3

As I mentioned in this recent post, the EViews team had a third blog post on ARDL modelling up their sleeves. The said post appeared a few days ago, here.

It's a real gem! The flow-chart and the detailed application are fabulous - I wish I could have come up with this myself.

Read it, read it................

© 2017, David E. Giles

Bounds Testing & ARDL Models - More From the EViews Team

The team at EViews has just released another post about ARDL modelling on their blog. This one is titled, "AutoRegressive Distributed Lag (ARDL) Estimation. Part 2 - Inference". This post is a follow-up to one that they wrote last month, and which I commented on here.

Given by the number of comments and requests that I get about this topic, these two posts from EViews are "must read" items for a lot of you.

And the great news is that there's a third post on the way, and this one will focus on implementing ARDL Modelling/Bounds Testing in EViews,

Great job!

© 2017, David E. Giles

ARDL Models - From the Team at EViews

Today the team at EViews published an important post on their blog. It's titled, "Autoregressive Distributed Lag (ARDL) Estimation. Part 1 - Theory".

If you have an interest in ARDL modelling - and I know that there are lots of you out there - then this is a must read post.

And as you can tell from its title, there's also a follow-up post on the way. So, you should watch out for it.

If  you plan on doing any ARDL modelling, you really can't go past EViews, so take a look.

© 2017, David E. Giles

Quantitative Macroeconomic Modeling with Structural Vector Autoregressions

A terrific new book titled, Quantitative Macroeconomic Modeling with Structural Vector Autoregressions – An EViews Implementation, is now available for free downloading from the EViews site. The book is written by Sam Ouliaris, Adrian Pagan, and Jorge Restrepo.

The "blurb" about this important new book reads:

"Quantitative macroeconomic research is conducted in a number of ways. An important method has been the use of the technique known as Structural Vector Autoregressions (SVARs), which aims to gather information about dynamic processes in macroeconomic systems. This book sets out the theory underlying the SVAR methodology in a relatively simple way and discusses many of the problems that can arise when using the technique. It also proposes solutions that are relatively easy to implement using EViews 9.5. Its orientation is towards applied work and it does this by working with the data sets from some classic SVAR studies."

In my view, EViews is certainly the natural choice for this venture. As the authors note in their Preface:

"A choice had to be made about the computer package that would be used to perform the quantitative work and EViews was eventually selected because of its popularity amongst IMF staff and central bankers more generally."

Gareth Thomas (of EViews) has pointed out to me that: "much of the book is covered in the IMF's free online macroeconomic forecasting course.  The next iteration of which starts in February:
https://www.edx.org/course/macroeconometric-forecasting-imfx-mfx-0 "

I'm sure that this new resource will be very well received!

© 2017, David E. Giles

Explaining the Almon Distributed Lag Model

In an earlier post I discussed Shirley Almon's contribution to the estimation of Distributed Lag (DL) models, with her seminal paper in 1965.

That post drew quite a number of email requests for more information about the Almon estimator, and how it fits into the overall scheme of things. In addition, Almon's approach to modelling distributed lags has been used very effectively more recently in the estimation of the so-called MIDAS model. The MIDAS model (developed by Eric Ghysels and his colleagues - e.g., see Ghysels et al., 2004) is designed to handle regression analysis using data with different observation frequencies. The acronym, "MIDAS", stands for "Mixed-Data Sampling". The MIDAS model can be implemented in R, for instance (e.g., see here), as well as in EViews. (I discussed this in this earlier post.)

For these reasons I thought I'd put together this follow-up post by way of an introduction to the Almon DL model, and some of the advantages and pitfalls associated with using it.

Let's take a look.

Monte Carlo Simulation Basics, II: Estimator Properties

In the early part of my recent post on this series of posts about Monte Carlo (MC) simulation, I made the following comments regarding its postential usefulness in econometrics:

".....we usually avoid using estimators that are are "inconsistent". This implies that our estimators are (among other things) asymptotically unbiased. ......however, this is no guarantee that they are unbiased, or even have acceptably small bias, if we're working with a relatively small sample of data. If we want to determine the bias (or variance) of an estimator for a particular finite sample size (n), then once again we need to know about the estimator's sampling distribution. Specifically, we need to determine the mean and the variance of that sampling distribution. 

If we can't figure the details of the sampling distribution for an estimator or a test statistic by analytical means - and sometimes that can be very, very, difficult - then one way to go forward is to conduct some sort of MC simulation experiment."

Before proceeding further, let's recall just what we mean by a "sampling distribution". It's a very specific concept, and not all statisticians agree that it's even an interesting one.

Specification Tests for Logit Models Using Gretl

In various earlier posts I've commented on the need for conducting specification tests when working with Logit and Probit models. (For instance, see here, here, and here.) 

One of the seminal references on this topic is Davidson and MacKinnon (1984). On my primary website, you can find a comprehensive list of other related references, together with EViews files that will enable you to conduct various specification tests with LDV  models.

The link for that material is here.

Today I had an email from Artur Tarassow at the University of Hamburg. He wrote:

I know that you're already aware of the open-source econometric software called "Gretl". 

I would like to let you know that I updated my package "LOGIT_HETERO.gfn". This package runs both the tests of homoskedasticity and correct functional form based on your nice program "Logit_hetero.prg" written for EViews.

If you want to have a look at it, simply run:

    set echo off

    set messages off

    install LOGIT_HETERO.gfn

    include LOGIT_HETERO.gfn

    open http://web.uvic.ca/~dgiles/downloads/binary_choice/Logistic_Burr.wf1

    logit Y 0 X1 X2

    matrix M = LOGIT_HETERO(Y,$xlist,$coeff,1)

    print M

Thanks for this, Artur - I'm sure it will be very helpful to many readers of this blog.

Footnote: See Artur's comment below, and the more recent post here. In particular, note Artur's comment: As a note to your blog readers: The two Logit model related packages “logit_burr.gfn” and “LOGIT_HETERO.gfn” are not available any more, as BMST includes both of them.

Reference

Davidson, R. & J. G. MacKinnon, 1984. Convenient specification tests for logit and probit models. Journal of Econometrics, 25, 241 262.

© 2016, David E. Giles

Forecasting From an Error Correction Model

Recently, a reader asked about generating forecasts from an estimated Error Correction Model (ECM). Really, the issues that arise are no different from those associated with any dynamic regression model. I talked about the latter in a previous post in 2013.

Anyway, let's take a look at the specifics.........

MIDAS Regression is Now in EViews

The acronym, "MIDAS", stands for several things. In the econometrics literature it refers to "Mixed-Data Sampling" regression analysis. The term was coined by Eric Ghysels a few years ago in relation to some of the novel work that he, his students, and colleagues have undertaken. See Ghysels et al. (2004).

Briefly, a MIDAS regression model allows us to "explain" a (time-series) variable that's measured at some frequency, as a function of current and lagged values of a variable that's measured at a higher frequency. So, for instance, we can have a dependent variable that's quarterly, and a regressor that's measured at a monthly, or daily, frequency.

There can be more than one high-frequency regressor. Of course, we can also include other regressors that are measured at the low (say, quarterly) frequency, as well as lagged values of the dependent variable itself. So, a MIDAS regression model is a very general type of autoregressive-distributed lag model, in which high-frequency data are used to help in the prediction of a low-frequency variable.

There's also another nice twist.......

Seasonal Unit Root Testing in EViews

When we're dealing with seasonal data - e.g., quarterly data - we need to distinguish between "deterministic seasonality" and "stochastic seasonality". The first type of seasonality is what we try to remove when we "seasonally adjust" the series. It's also what we're trying to account for when we include seasonal dummy variables in a regression model.

On the other hand, "stochastic seasonality" refers to unit roots at the seasonal frequencies. This is a whole different issue, and it's been well researched in the time-series econometrics literature.

This distinction is similar to that between a "deterministic trend" and a "stochastic trend" in annual data. The former can be removed by "de-tending" the series, but the latter refers to a unit root (at the zero frequency).

The most widely used procedure for testing for seasonal unit roots is that proposed by Hylleberg et al. (HEGY) (1990), and extended by Ghysels et al. (1994).

In my graduate-level time-series course we always look at stochastic seasonality. Recently, Nicolas Ronderos has written a new "Add-in" for EViews to make it easy to implement the HEGY testing procedure (see here). This will certainly save some coding for EViews users.  

Of course, stochastic seasonality can also arise in the case of monthly data - this is really messy - see Beaulieu and Miron (1993). In the case of half-yearly data, the necessary theoretical framework and critical values are developed and illustrated by Feltham and Giles (2003)

And if you have unit roots at the seasonal frequencies in two or more time-series, you might also have seasonal cointegration. The seminal contribution relating to this is by Engle et al. (1993), and an short empirical application is provided by Reinhardt and Giles (2001)

I plan to illustrate the application of seasonal unit root and cointegration tests in a future blog post.

(Also, note the comment from Jack Lucchetti, below, that draws attention to a HEGY addon for Gretl, written by Ignacio Diaz Emparanza.)

References

Beaulieu, J. J., and J. A. Miron, 1993. Seasonal unit roots in aggregate U.S. data. Journal of Econometrics, 55, 305-328.

Engle, R. F., C. W. J. Granger, S. Hyleberg, H. S. Lee, 1993. Seasonal cointegration: The Japanese consumption function. Journal of Econometrics, 55, 275-298.

Feltham, S. G. and D. E. A. Giles, 2003. Testing for unit roots in semi-annual data. in D.E.A. Giles 

(ed.), Computer-Aided Econometrics. Marcel Dekker, New York, 175-208. (Pre-print here.)

Ghysels, E., H. S. Lee, and J. Noh, 1994. Testing for unit roots in seasonal time series: Some theoretical extensions and a Monte Carlo investigation. Journal of Econometrics, 62, 415-442.

Hylleberg, S., R. F. Engle, C. W. J. Granger, and B. S. Yoo, 1990. Seasonal integration and cointegration. Journal of Econometrics, 44, 215-238.

Reinhardt, F. S. and D. E. A. Giles, 2001. Are cigarette bans really good economic policy?. Applied Economics, 33, 1365-1368. (Pre-print here.)

© 2015, David E. Giles

Illustrating Spurious Regressions

I've talked a bit about spurious regressions a bit in some earlier posts (here and here). I was updating an example for my time-series course the other day, and I thought that some readers might find it useful.

Let's begin by reviewing what is usually meant when we talk about a "spurious regression".

In short, it arises when we have several non-stationary time-series variables, which are not cointegrated, and we regress one of these variables on the others.

In general, the result that we get are nonsensical, and the problem is only worsened if we increase the sample size. This phenomenon was observed by Granger and Newbold (1974), and others, and Phillips (1986) developed the asymptotic theory that he then used to prove that in a spurious regression the Durbin-Watson statistic converges in probability to zero; the OLS parameter estimators and R² converge to non-standard limiting distributions; and the t-ratios and F-statistic diverge in distribution, as T ↑ ∞ .

Let's look at some of these results associated with spurious regressions. We'll do so by means of a simple simulation experiment.

Worrying About my Cholesterol Level

The headline, "Don't Get Wrong Idea About Cholesterol", caught my attention in the 3 May, 2015 Times-Colonist newspaper here in Victoria, B.C.. In fact the article came from a syndicated column, published about a week earlier. No matter - it's always a good time for me to worry about my cholesterol!

The piece was written by a certain Dr. Gifford-Jones (AKA Dr. Ken Walker).

Here's part of what he had to say:

Logit, Probit, & Heteroskedasticity

I've blogged previously about specification testing in the context of Logit and Probit models. For instance, see here and here. 

Testing for homoskedasticity in these models is especially important, for reasons that are outlined in those earlier posts. I won't repeat all of the details here, but I'll just note that heteroskedasticity renders the MLE of the parameters inconsistent. (This stands in contrast to the situation in, say, the linear regression model where the MLE of the parameters is inefficient, but still consistent in this case.)

If you're an EViews user, you can find my code for implementing a range of specification tests for Logit and Probit models here. These include the LM test for homoskedasticity that was proposed by Davidson and MacKinnon (1984).

More than once, I've been asked the following question:

"When estimating a Logit or Probit model, we set the scale parameter (variance) of the error term to the value one, because it's not actually identifiable. So, in what sense can we have heteroskedasticity in such models?"

This is a good question, and I thought that a short post would be justified. Let's take a look:

Maximum Likelihood Estimation & Inequality Constraints

This post is prompted by a question raised by Irfan, one of this blog's readers, in some email correspondence with me a while back.

The question was to do with imposing inequality constraints on the parameter estimates when applying maximum likelihood estimation (MLE). This is something that I always discuss briefly in my graduate econometrics course, and I thought that it might be of interest to a wider audience.

Here's the issue.

Videos for EViews 9

The team at EViews has put together a great set of videos that highlight some of the new features in EViews 9.

You can find them here, and I strongly recommend them.

© 2015, David E. Giles