Here's something (else!) that annoys the heck out of me. I've seen it come up time and again in economics seminars over the years.
It usually goes something like this:
There are two estimates of some parameter, based on two different models.
Question from Audience: "I know that the two point estimates are numerically pretty similar, but is the difference statistically significant?"
Speaker's Response: "Well, if you look at the two standard errors and mentally compute separate 95% confidence intervals, these intervals overlap, so there's no significant difference, at least at the 5% level."
There are two estimates of some parameter, based on two different models.
Question from Audience: "I know that the two point estimates are numerically pretty similar, but is the difference statistically significant?"
Speaker's Response: "Well, if you look at the two standard errors and mentally compute separate 95% confidence intervals, these intervals overlap, so there's no significant difference, at least at the 5% level."
My Reaction: "What utter crap! (Eye roll!)
So, what's going on here?
So, what's going on here?