Here's a follow-up to yesterday's
post on Wolfram's CDF file format. In an earlier post (
here) I discussed the fact the
p-values are random variables, with their own sampling distribution.
A great way of visualizing a number of the points that I made in that post is to use the CDF file for the
Mathematica app. written by
Ian McLeod (University of Western Ontario). You can run it/download it from
here, using Wolfram's free CDF Player (
here).
You'll recall that a p-value is uniformly distributed on [0 , 1] if the null hypothesis being tested is true. Using Ian's app., here is an example of what you see for the case where you're testing the null of a zero mean in a normal population, against a 2-sided alternative:
The null hypothesis is TRUE. The sample size is n = 50, and the simulation experiment involves 10,000 replications. You can see that the empirical distribution is approaching the Uniform true sampling distribution.
When the true mean of the distribution is 2.615 (so the null hypothesis is FALSE), the sampling distribution of the (two-sided) p-value looks like this:
© 2011, David E. Giles