A Reverse Regression Inequality

Suppose that we fit the following simple regression model, using OLS:

            y_i = βx_i + ε_i   .                                                              (1)

To simplify matters, suppose that all of the data are calculated as deviations from their respective sample means. That's why I haven't explicitly included an intercept in (1). This doesn't affect any of the following results.

The OLS estimator of β is, of course,

            b = Σ(x_iy_i) / Σ(x_i²) ,

where the summations are for i = 1 to n (the sample size).

Now consider the "reverse regression":

           x_i = αy_i + u_i   .                                                             (2)

The OLS estimator of α is

           a = Σ(x_iy_i) / Σ(y_i²).

Clearly, a ≠ (1 / b), in general. However, can you tell if a ≥ (1 / b), or if a ≤ (1 / b)?

The answer is, "yes", and here's how you do it.

The trick is to recall the Cauchy-Schwarz Inequality. One variant of this inequality tells us that

          [Σ(x_iy_i)]² ≤ [Σ(x_i²) Σ(y_i²)]  .                                          (3)

Now,  (ab) =  [Σ(x_iy_i)]² / [Σ(x_i²) Σ(y_i²)] .

So, immediately, from (3),

          (ab) ≤  [Σ(x_i²) Σ(y_i²)] / [Σ(x_i²) Σ(y_i²)] = 1 ,

or,  

          a ≤ (1 / b) ;   if b > 0

          a ≥ (1 / b) ;    if b < 0 ,                                            (4)

regardless of the sample values for the data.

Now, here are some questions for you to think about:

1. Under what circumstances will (4) hold as an equality?
2. What can you say about the relationship between the two R² values that we get when we estimate (1) and (2) by OLS?
3. What can you say about the relationship between the t-ratios for testing H₀: β = 0 in (1); and for testing H₀': α = 0 in (2)?

© 2014, David E. Giles