Now you are maybe wondering; should you use p-values or confidence intervals?
Almost all disciplines would recommend using both because they capture several different dimensions.
P-value: Advantages and disadvantages
The p-value is an important part of research, most likely the heart of it.
The p-value is based on “yes-or no”-questions in which it shows how much evidence we have against the null hypothesis.
P-values are much clearer than confidence intervals and it helps the researcher to make quick judgments about their research.
Another advantage with the p-value is that it can give the difference from a previous specified statistical level.
Unfortunately, there are misconceptions about the p-value among researchers and many disciplines rely on them to draw conclusions rather than understanding the background.
One of the common mistakes among researchers is that they do not further analyse their data in order to ensure that the p-value is not affected by other factors.
Moreover, p-values cannot alone permit any direct statements about the direction or size of difference. In order to make those decisions, one must always look at the confidence intervals.
Confidence interval: Advantages and disadvantages
A confidence interval informs the researcher about the power of the study and whether the data is compatible, it also shows the likelihood of the null hypothesis being true and that in turn tells us how much confidence we have in our findings.
The width of the confidence interval indicates the precision of the point estimates, in which a slim interval indicates a more precise estimate, while a wide interval indicates a less precise estimate.
The precision is related to the sample size and power in which it tells us that the larger sample size we have, the more precise estimates we have.
The intervals are useful when having small sample sizes. Normally, small studies fail to find statistically significant treatments, when including point estimates with wide intervals that include the null value may be consistent and significant. The intervals provide the researcher an understanding of the sample size.
This can also be a disadvantage when having large data because it produces statistically significant results even if the difference between the groups is small.
Another advantage with the confidence interval is that it can provide means of analysis for studies that seek to describe and explain, rather than make decisions about treatments effects.
A disadvantage with the confidence interval is that it captures several elements at the time, in which it may not give as precise information like the p-values.
The advantage of combining them
As mentioned, a majority of disciplines recommend including both p-values and confidence intervals because they capture information in different dimensions.
Neither p-values nor confidence intervals can prevent biases or other problems but the combination of them provides a more flexible approach and highlights new perspectives on the data.
Confidence intervals permit us to draw several conclusions at the same time and they are more informative about sample sizes and point estimates. They are also useful in studies when we have small sample sizes. But they are not as precise as p-values when it comes to accepting and rejecting the null hypothesis. Thus, when we combine them together we can be more certain.
The figure below summarizes the advantages and disadvantages when interpreting and drawing conclusions with the help of p-values and confidence intervals.
| P-values | Confidence intervals | |
| Accept/reject |  | |
| Degree of support | | |
| Estimate and uncertainty |  | |