Sunday, July 8, 2012

Local vs. Global Approximations

Approximating unknown (continuously differentiable) functions by using a Taylor (MacLaurin) series expansion is common-place in econometrics. However, do you ever pause to recall that such approximations are only locally valid - that is, valid only in a neighbourhood of the (possibly vector) point about which the approximation is made?

Unlike some other types of approximations - such as Fourier approximations - they are not globally valid.

Does this matter? Is it something we should be concerning ourselves with?


Let's think of some examples of situations where we explicitly use a Taylor series approximation in econometrics. These include:

  1. The "delta method" for approximating variances of non-linear functions of parameter estimators.
  2. The Newton-Raphson algorithm for obtaining a numerical solution to the first-order conditions (likelihood equations) associated with the maximization of the log-likelihood function.
  3. Generalizing the Wald test for linear restrictions to a Wald test for non-linear restrictions.
  4. The RESET test, where we approximate the unknown function for the conditional mean of a regression model's error term with powers of a predicted value of the dependent variable. (See this recent post for more details.)
No doubt you can think of some more examples.

Generally, as in the case of the first three examples above, the objective is to linearize a known non-linear function - that is, to take a linear approximation in a neighbourhood of an interesting point in the parameter space.

However, in the fourth example, the function that's being approximated is unknown, and approximation (rather than linearization) is the objective.

That being the case, why use a power series (Taylor series approximation) when constructing the RESET test?

Some years ago, Linda DeBenedictis and I addressed this question and we developed what we called the FRESET test. We replaced the (locally valid) Taylor series approximation with a (globally valid) Fourier series approximation. In practice, this means that after scaling the data to lie in the [-π , π] interval, multiples of sines and cosines of the predicted values are used to augment the basic set of regressors. An exact F-test is then used to test that the coefficients of these additional variables are jointly zero.

The test is as simple to apply as is the traditional RESET test.

The motivation was that we anticipated that by using a superior approximation to the function defining the conditional mean of the model's error term, we should be able to improve the power of the test. After all, although the test statistic is exactly F-distributed under the null hypothesis, the power of the test depends on the choice of  random augmenting variables.


By and large, this expectation was realized. Our simulations also revealed that the traditional RESET test can have very low power in many situations.

Ken White kindly coded the FRESET test into the SHAZAM econometrics package, and in a subsequent paper, Linda and I (Giles and DeBenedictis, 1999) explored the robustness of the test to the presence of autocorrelation in the model's errors. Again, FRESET generally dominated RESET in terms of power.

So, sometimes it's worth "thinking outside the box", rather than sticking with traditional solutions to problems in econometrics.

References

DeBenedictis, L. F. and D. E. A. Giles, 1998. Diagnostic testing in econometrics: Variable addition, RESET, and Fourier approximations. In A. Ullah and D. E. A. Giles (eds.), Handbook of Applied Economic Statistics, Marcel Dekker, New York, 383-417. WP versionfigures.)

DeBenedictis, L. F. and D. E. A. Giles, 1999. Robust specification testing in regression: The FRESET test and autocorrelated disturbances. Journal of Quantitative Economics, 15, 67-75. (WP version.)






© 2012, David E. Giles

5 comments:

  1. I think one issue might be simple intuition. Everyone who's taken a calculus course has been introduced to Taylor series expansions, and most economists are therefore comfortable with them. On the other hand, Fourier series are much stranger to economists. Aside from some time-series analysis, harmonic analysis is not something that economists usually think about. In my case, I can grasp the mathematics of frequency-domain analysis from an engineering perspective with a nice continuous signal. But relating that to a sample of economic data, especially one that's not a time series, has always befuddled me.

    ReplyDelete
    Replies
    1. Brian - you may well be right. Of course, other global approximations are also possible - e.g., in terms of Hermite polynomials.

      Delete
  2. Dave - thanks for your comment on the RESET test. I am eager to apply the FRESET test! Do I take it that it can be accomodated to a Poisson model?

    Best,
    Boris

    ReplyDelete
    Replies
    1. Boris - yes it can - I have some work in progress right now that compares the FRESET and RESET tests for count data models. More to come!

      Delete
    2. I'm looking forward to seeing your work on this!

      Best,
      Boris

      Delete