Reporting an R-Squared Measure for Count Data Models

This post was prompted by an email query that I received some time ago from a reader of this blog. I thought that a more "expansive" response might be of interest to other readers............

In spite of its many limitations, it's standard practice to include the value of the coefficient of determination (R²) - or its "adjusted" counterpart - when reporting the results of a least squares regression. Personally, I think that R² is one of the least important statistics to include in our results, but we all do it. (See this previous post.)

If the regression model in question is linear (in the parameters) and includes an intercept, and if the parameters are estimated by Ordinary Least Squares (OLS), then R² has a number of well-known properties. These include:

1. 0 ≤ R² ≤ 1.
2. The value of R² cannot decrease if we add regressors to the model.
3. The value of R² is the same, whether we define this measure as the ratio of the "explained sum of squares" to the "total sum of squares" (R_E²); or as one minus the ratio of the "residual sum of squares" to the "total sum of squares" (R_R²).
4. There is a correspondence between R² and a significance test on all slope parameters; and there is a correspondence between changes in (the adjusted) R² as regressors are added, and significance tests on the added regressors' coefficients.   (See here and here.)
5. R² has an interpretation in terms of information content of the data.  
6. R² is the square of the (Pearson) correlation (R_C²) between actual and "fitted" values of the model's dependent variable. 

However, as soon as we're dealing with a model that excludes an intercept or is non-linear in the parameters, or we use an estimator other than OLS, none of the above properties are guaranteed.

Suggested Reading for August

Here are my suggestions for this month:

• Bun, M. J. G. & T. D. Harrison, 2109. OLS and IV estimation of regression models including endogenous interaction terms. Econometric Reviews, 38, 814-827.
• Dufour, J-M., E. Flachaire, & L. Khalaf, Permutation tests for comparing inequality measures. Journal of Business and Economic Statistics, 37, 457-470.
• Jiao, X. & F. Pretis, 2018. Testing the presence of outliers in regression models. Available at SSRN: https://ssrn.com/abstract=3217213.
• Stanton, J. M., 2001. Galton, Pearson, and the peas: A brief history of linear regression for statistics instructors. Journal of Statistics Education, 9, 1-13.
• Trafimow, D., 2019. A frequentist alternative to significance testing, p-values and confidence intervals. Econometrics, 7, 26.

© 2019, David E. Giles

Reading Suggestions for June

Well, here we are - it's June already.

Here are my reading suggestions:

• Abadie, A., S. Athey, G. Imbens, & J. Wooldridge, 2017. When should you adjust standard errors for clustering? Mimeo.
• Berk, R., A. Buja, L. Brown, E. George, A. K. Kuchibhotla, W. Su, & L, Shazo, 2019. Assumption lean regression. American Statistician, in press.
• Ghosh, T., M. Ghosh, & T. Kubokawa, 2019. On the loss robustness of least-square estimators, American Statistician, in press.
• Gustafsson, O. & P. Stockhammar, 2019. Variance stabilizing filters. Communications in Statistics - Theory and Methods, in press.
• Kocherlakota, N. R., 2019. A near-exact finite sample theory for an instrumental variable estimator. Mimeo. (Hat-tip to Frank Diebold.)
• Panagiotelis, A., G. Anathasopoulos, R. J. Hyndman, B. Jiang, & F. Vahid, 2019. Macroeconomic forecasting for Australia using a large number of predictors. International Journal of Forecasting, 35, 613-633.

© 2019, David E. Giles

Forecasting From a Regression with a Square Root Dependent Variable

Back in 2013 I wrote a post that was titled, "Forecasting From Log-Linear Regressions". The basis for that post was the well-known result that if you estimate a linear regression model with the (natural) logarithm of y as the dependent variable, but you're actually interested in forecasting y itself, you don't just report the exponentials of the original forecasts. You need to add an adjustment that takes account of the connection between a Normal random variable and a log-Normal random variable, and the relationship between their means.

Today, I received a query from a blog-reader who asked how the results in that post would change if the dependent variable was the square root of y, but we wanted to forecast the y itself. I'm not sure why this particular transformation was of interest, but let's take a look at the question.

In this case we can exploit the relationship between a (standard) Normal distribution and a Chi-Square distribution in order to answer the question.

A Terrific New Book on the Linear Model

Recently, it was my distinct pleasure to review a first-class book by David Harville, titled Linear Models and the Relevant Distributions and Matrix Algebra.

(Added 28 February, 2019: You can now read the published review in Statistical Papers, here.)

Here is what I had to say:

New Year Reading Suggestions for 2019

With a new year upon us, it's time to keep up with new developments -

• Basu, D., 2018. Can we determine the direction of omitted variable bias of OLS estimators? Working Paper 2018-16, Department of Economics, University of Massachusetts, Amherst.
• Jiang, B., Y. Lu, & J. Y. Park, 2018. Testing for stationarity at high frequency. Working Paper 2018-9, Department of Economics, University of Sydney. 
• Psaradakis, Z. & M. Vavra, 2018. Normality tests for dependent data: Large-sample and bootstrap approaches. Communications in Statistics - Simulation and Computation, online.
• Spanos, A., 2018. Near-collinearity in linear regression revisited: The numerical vs. the statistical perspective. Communications in Statistics - Theory and Methods, online.
• Thorsrud, L. A., 2018. Words are the new numbers: A newsy coincident index of the business cycle. Journal of Business Economics and Statistics, online. (Working Paper version.)
• Zhang, J., 2018. The mean relative entropy: An invariant measure of estimation error. American Statistician, online.

© 2019, David E. Giles

Econometrics Reading List for September

A little belatedly, here is my September reading list:

• Benjamin, D. J. et al., 2017. Redefine statistical significance. Pre-print.
• Jiang, B., G. Athanasopoulos, R. J. Hyndman, A. Panagiotelis, and F. Vahid, 2017. Macroeconomic forecasting for Australia using a large number of predictors. Working Paper 2/17, Department of Econometrics and Business Statistics, Monash University.
• Knaeble, D. and S. Dutter, 2017. Reversals of least-square estimates and model-invariant estimations for directions of unique effects. The American Statistician, 71, 97-105.
• Moiseev, N. A., 2017. Forecasting time series of economic processes by model averaging across data frames of various lengths. Journal of Statistical Computation and Simulation, 87, 3111-3131.
• Stewart, K. G., 2017. Normalized CES supply systems: Replication of Klump, McAdam and Willman (2007). Journal of Applied Econometrics, in press.
• Tsai, A. C., M. Liou, M. Simak, and P. E. Cheng, 2017. On hyperbolic transformations to normality. Computational Statistics and Data Analysis, 115, 250-266,

© 2017, David E. Giles

My August Reading List

Here are some suggestions for you:

• Calzolari, G., 2017. Econometrics exams and round numbers: Use or misuse of indirect estimation methods? Communications in Statistics - Simulation and Computation, in press.
• Chakraborti, S., F. Jardim, & E. Epprecht, 2017. Higher order moments using the survival function: The alternative expectation formula. American Statistician, in press.
• Clarke, J. A., 2017. Model averaging OLS and 2SLS: An application of the WALS procedure. Econometrics Working Paper EWP1701, Department of Economics, University of Victoria.
• Hotelling, H., 1940. The teaching of statistics, Annals of Mathematical Statistics, 11, 457-470.
• Knaeble, B. & S. Dutter, 2017. Reversals of least-square estimates and model-invariant estimation for directions of unique effects. American Statistician, 71, 97-105.
• Megerdichian, A., 2017. Further results on interpreting coefficients in regressions with a logarithmic dependent variable. Journal of Econometric Methods, in press.

© 2017, David E. Giles

A Quick Illustration of Pre-Testing Bias

The statistical and econometric literature on the properties of "preliminary-test" (or "pre-test") estimation strategies is large and well established. These strategies arise when we proceed in a sequential manner when drawing inferences about parameters. 

A simple example would be where we fit a regression model; test if a regressor is significant or not; and then either retain the model, or else remove the (insignificant) regressor and re-estimate the (simplified) model.

The theoretical literature associated with pre-testing is pretty complex. However, some of the basic messages arising from that literature can be illustrated quite simply. Let's look at the effect of "pre-testing" on the bias of the OLS regression estimator.

Linear Regression and Treatment Effect Heterogeneity

I received an email from Tymon Słoczyński (Warsaw School of Economics), about a recent paper of his, titled, "New Evidence on Linear Regression and Treatment Effect Heterogeneity". Tymon wrote:

"I have recently written a new paper, which I believe that you might find interesting, given some of your blog posts that I have read. 

This paper is available here (as an IZA DP No. 9491): http://ftp.iza.org/dp9491.pdf; or from my website: http://akson.sgh.waw.pl/~tslocz/Sloczynski_paper_regression.pdf. 

This paper implicitly criticizes the standard approach in reduced-form applied microeconomics to use very simple linear models and estimate them using OLS (or 2SLS). I provide a new interpretation of the least squares estimand in the constant-effects linear regression model when the assumption of constant effects is violated (so there is, in fact, "treatment effect heterogeneity"). This new interpretation is very pessimistic: in particular, I prove that the weight that is being placed by OLS on the effect on each group ("treated" or "controls") is inversely related to the proportion of this group. This property might have severe consequences for applied work, and I demonstrate this via a replication of two recent papers from the American Economic Review."

Tymon's paper is, indeed, very interesting. I recommend that you read it. It should serve as a 'wake-up call' to some of our empirical micro. friends!

© 2015, David E. Giles

Illustrating Spurious Regressions

I've talked a bit about spurious regressions a bit in some earlier posts (here and here). I was updating an example for my time-series course the other day, and I thought that some readers might find it useful.

Let's begin by reviewing what is usually meant when we talk about a "spurious regression".

In short, it arises when we have several non-stationary time-series variables, which are not cointegrated, and we regress one of these variables on the others.

In general, the result that we get are nonsensical, and the problem is only worsened if we increase the sample size. This phenomenon was observed by Granger and Newbold (1974), and others, and Phillips (1986) developed the asymptotic theory that he then used to prove that in a spurious regression the Durbin-Watson statistic converges in probability to zero; the OLS parameter estimators and R² converge to non-standard limiting distributions; and the t-ratios and F-statistic diverge in distribution, as T ↑ ∞ .

Let's look at some of these results associated with spurious regressions. We'll do so by means of a simple simulation experiment.

What NOT To Do When Data Are Missing

Here's something that's very tempting, but it's not a good idea.

Suppose that we want to estimate a regression model by OLS. We have a full sample of size n for the regressors, but one of the values for our dependent variable, y, isn't available. Rather than estimate the model using just the (n - 1) available data-points, you might think that it would be preferable to use all of the available data, and impute the missing value for y.

Fair enough, but what imputation method are you going to use?

For simplicity, and without any loss of generality, suppose that the model has a single regressor,

             

                y_i = β x_i + ε_i ,                                                                       (1)

 and it's the n^th value of y that's missing. We have values for x₁, x₂, ...., x_n; and for y₁, y₂, ...., y_n-1.

Here's a great idea! OLS will give us the Best Linear Predictor of y, so why don't we just estimate (1) by OLS, using the available (n - 1) sample values for x and y; use this model (and x_n) to get a predicted value (y*_n) for y_n; and then re-estimate the model with all n data-points: x₁, x₂, ...., x_n; y₁, y₂, ...., y_n-1, y*_n.

Unfortunately, this is actually a waste of time. Let's see why.

Biased Estimation of Marginal Effects

I began a recent post with the comment:

"One thing that a lot of practitioners seem to be unaware of (or they choose to ignore it) is that in many of the common situations where we use regression analysis to estimate elasticities, these estimators are biased.

And that's true even if all of the conditions needed for the coefficient estimator (e.g., OLS) to be unbiased are fully satisfied."

Exactly the same point can be made in respect of estimated marginal effects, and that's what this post is about.

The Bias of Certain Elasticity Estimators

In a recent post I discussed some aspects of estimating elasticities from regression models, and the interpretation of these values. That discussion should be kept in mind in reading what follows.

One thing that a lot of practitioners seem to be unaware of (or they choose to ignore it) is that in many of the common situations where we use regression analysis to estimate elasticities, these estimators are biased.

And that's true even if all of the conditions needed for the coefficient estimator (e.g., OLS) to be unbiased are fully satisfied.

Let's look at some common situations leading to the estimation of elasticities and marginal effects, and see if we can summarize what's going on.

Regression Coefficients & Units of Measurement

A linear regression equation is just that - an equation. This means that when any of the variables - dependent or explanatory - have units of measurement, we also have to keep track of the units of measurement for the estimated regression coefficients.

All too often this seems to be something that students of econometrics tend to overlook.

Consider the following regression model:

               y_i = β₀ + β₁X_1i + β₂x_2i + β₃x_3i + ε_i    ;    i = 1, 2, ...., n                   (1)

where y and x₂ are measured in dollars; x₁ is measured in Kg; and x₃ is a unitless index.

Because the term on the left side of (1) has units of dollars, every term on the right side of that equation must also be expressed in terms of dollars. These terms are β₀, (β₁x_1i), (β₂x_2i), (β₃x_3i), and ε_i.

In turn, this implies that β₀ and β₃ have units which are dollars; the units of β₁ are ($ / Kg); and β₂ is unitless. In addition, the error term, ε, has units that are dollars, and so does its standard deviation, σ.

What are some of the implications of this?

March Reading List

Good grief! It's March already. You might enjoy:

Bajari, P., D. Nekipelov, S. P. Ryan, and M. Yang, 2015. Demand estimation with machine learning and model combination. NBER Working Paper No, 20955.

Baur, D. G. and D. T. Tran, 2014. The long-run relationship of gold and silver and the influence of bubbles and financial crises. Empirical Economics, 47, 1525-1541.

Efron, B., 2014. Estimation and accuracy after model selection. Journal of the American Statistical Association, 109, 991-1007.

Kennedy, P. E., 1995. Randomization tests in econometrics. Journal of Business and Economic Statistics, 13, 85-94.

Magnus, J. R., W. Wang, and X. Zhang, 2015. Weighted-average least squares prediction. Econometric Reviews, in press.

Osman, A. F. and M. L. King, 2015. A new approach to forecasting based on exponential smoothing with independent regressors. Working Paper 02/15, Department of Econometrics and Business Statistics, Monash University.

Perron, P. and Y. Yamamoto, 2015. Using OLS to estimate and test for structural change in models with endogenous regressors. Journal of Applied Econometrics, 30, 119-144.

© 2015, David E. Giles

Four Different Types of Regression Residuals

When we estimate a regression model, the differences between the actual and "predicted" values for the dependent variable (over the sample) are termed the "residuals". Specifically, if the model is of the form:

                     y = Xβ + ε ,                                                         (1)

and the OLS estimator of β is b, then the vector of residuals is

                    e = y - Xb .                                                           (2)

Any econometrics student will be totally familiar with this.

The elements of e (the n residuals) are extremely important statistics. We use them, of course, to construct other statistics - e.g., test statistics to be used for testing the validity of the underlying assumptions associated with our regression model. For instance, we want to check, are the errors (the elements of the ε vector) serially independent; are the errors homoskedastic; are they normally distributed; etc.?

What a lot of students don't learn is that these residuals - let's call them "Ordinary Residuals" - are just one type of residuals that are used when analysing the regression model. Lets take a look at this.

More on Prediction From Log-Linear Regressions

My therapy sessions are actually going quite well. I'm down to just one meeting with Jane a week, now. Yes, there are still far too many log-linear regressions being bandied around, but I'm learning to cope with it!

Last year, in an attempt to be helpful to those poor souls I had a post about forecasting from models with a log-transformed dependent variable. I felt decidedly better after that, so I thought I follow up with another good deed.

Let's see if it helps some more:

Orthogonal Regression: First Steps

When I'm introducing students in my introductory economic statistics course to the simple linear regression model, I like to point out to them that fitting the regression line so as to minimize the sum of squared residuals, in the vertical direction, is just one possibility.

They see, easily enough, that squaring the residuals deals with the positive and negative signs, and that this prevents obtaining a "visually silly" fit through the data. Mentioning that one could achieve this by working with the absolute values of the residuals provides the opportunity to mention robustness to outliers, and to link the discussion back to something they know already - the difference between the behaviours of the sample mean and the sample median, in this respect.

We also discuss the fact that measuring the residuals in the vertical ("y") direction is intuitively sensible, because the model is purporting to "explain" the y variable. Any explanatory failure should presumably be measured in this direction. However, I also note that there are other options - such as measuring the residuals in the horizontal ("x") direction.

Perhaps more importantly, I also mention "orthogonal residuals". I mention them. I don't go into any details. Frankly, there isn't time; and in any case this is usually the students' first exposure to regression analysis and they have enough to be dealing with. However, I've thought that we really should provide students with an introduction to orthogonal regression - just in the simple regression situation - once they've got basic least squares under their belts. 

The reason is that orthogonal regression comes up later on in econometrics in more complex forms, at least for some of these students; but typically they haven't seen the basics. Indeed, orthogonal regression is widely used (and misused - Carroll and Ruppert, 1966) to deal with certain errors-in-variables problems. For example, see Madansky (1959).

That got me thinking. Maybe what follows is a step towards filling this gap.

Normality Testing & Non-Stationary Data

Bob Jensen emailed me about my recent post about the way in which the Jarque-Bera test can be impacted when temporally aggregated data are used. Apparently he publicized my post on the listserv for Accounting Educators in the U.S.. He also drew my attention to a paper from Two former presidents of the AAA: "Some Methodological Deficiencies in Empirical Research Articles in Accounting", by Thomas R. Dyckman and Stephen A. Zeff, Accounting Horizons, September 2014, 28 (3), 695-712. (Here.) 

Bob commented that an even more important issue might be that our data may be non-stationary. Indeed, this is always something that should concern us, and regular readers of this blog will know that non-stationary data, cointegration, and the like have been the subject of a lot of my posts.

In fact, the impact of unit roots on the Jarque-Bera test was mentioned in this old post about "spurious regressions". There, I mentioned a paper of mine (Giles, 2007) in which I proved that: