Many people often encounter p-value in life, however surprisingly some of them easily get confused by either p-value or

the alpha level, which are the two basic concepts in hypothesis testing. Today we try to give a very easy-to-get

version of these two terms.

So to understand p-value, we need first to know

What is an alpha value?

In hypothesis testing, there are usually two opposite hypothesis and by convention we would probably

set the one with more confidence as null hypothesis, set another again it. Without looking

at the data, we intuitively would assume null hypothesis is right, unless data strongly against it.

So null hypothesis commonly has to be the safe, conservative choice.

For example, we are researchers and we think that we made a

discovery of a real phenomenon. Then the null hypothis would be “we didn’t discover anything”.

But we insist on our theory, so let’s get some data to find out. Before analyzing the data

we need first set a standard for the testing. The standard is the alpha level, usually set of .05.

This means that assuming that the null hypothesis is true, we may reject the null only if the observed

data are so unusual that they would have occurred by chance at most 5 % of the time.

In our case, we would only be statistically sure that we indeed discover a new phenomenon

only if the data are so unusual that the chance of having them is lower than alpha value.

Thus the smaller the alpha, the more stringent the test (the more unlikely it is to find a statistically

significant result). If the chance of having observed data is higher than the alpha value, that means

it is nothing special, and could happen anytime, so we didn’t discover anything.

What is a p-value?

Once the alpha level has been set, a statistic (like the mean) is computed. Each statistic has an associated

probability value called a p-value, or the likelihood of an observed statistic occurring due to

chance, given the sampling distribution.

So alpha sets the standard for how extreme the data must be before we can reject the null

hypothesis. The p-value indicates how extreme the data are. More precisely, p-value means that probability

that we could have these data under null hypothesis. We compare the p-value with the

alpha to determine whether the observed data are statistically significantly different from the null

hypothesis:

- If the p-value is less than or equal to the alpha (p< .05), then we reject the null hypothesis, and

we say the result is statistically significant. - If the p-value is greater than alpha (p > .05), then we fail to reject the null hypothesis, and we say

that the result is statistically nonsignificant (n.s.).