Significance levels and confidence levels are just two ways of looking at the same thing.

Levels

The level is set by the individual researcher – in that sense, it is quite arbitrary– but there are some levels that are widely used (asterisks are often used to illustrate these levels):

P-value	p<0.05	p<0.01	p<0.001
Significance level	5%	1%	0.1%
Confidence level	95%	99%	99.9%
Asterisks	*	**	***

Note In some fields of research, p<0.10 – statistical significance at the 10% level – is also a commonly used significance level.

Example Let us return to the example of differences in intelligence between cats and dogs. For instance, if we find a difference in intelligence between these types of animal, and the p-value is below 0.05, we may thus state that the null hypothesis (i.e. no difference) is rejected at the 95% confidence level. The p-value does not, however, state whether the difference is small or big, or whether cats or dogs represent the smarter type of animal (in order to state such things, one would have to look at the direction and the effect size).

P-values and sample size

It should be noted that the p-value is affected by the sample size, which means that a smaller sample size often translates to a larger p-value.

Example If you have a data material of 100 individuals, the effect size has to be quite large (e.g. large income differences income between men and women) in order to get small p-values. Conversely, larger sample size makes it easier to find small p-values. For example, if you analyse a data material containing the entire population of a country, even tiny differences are likely to have small p-values. In other words, the size of the sample influences the chances of rejecting the null hypothesis (see Power analysis).