Before thinking about P values, you should:
A small P value means that the difference (correlation, association,...) you observed would happen rarely due to random sampling. There are three possibilities:
Deciding between the last two possibilities is a matter of scientific judgment, and no statistical calculations will help you decide.
If the P value is less than 0.05, then the 95% confidence interval will not contain zero (when comparing two means). To interpret the confidence interval in a scientific context, look at both ends of the confidence interval and ask whether they represent a difference between means that you consider to be scientifically important or scientifically trivial. This section assumes you are comparing two means with a t test, but it is straightforward to use these same ideas in other contexts.
There are three cases to consider:
•The confidence interval only contains differences that are trivial. Although you can be 95% sure that the true difference is not zero, you can also be 95% sure that the true difference between means is tiny and uninteresting. The treatment had an effect, but a small one.
•The confidence interval only includes differences you would consider to be important. Since even the low end of the confidence interval represents a difference large enough that you consider it to be scientifically important, you can conclude that there is a difference between treatment means and that the difference is large enough to be scientifically relevant.
•The confidence interval ranges from a trivial to an important difference. Since the confidence interval ranges from a difference that you think would be scientifically trivial to one you think would be important, you can't reach a strong conclusion. You can be 95% sure that the true difference is not zero, but you cannot conclude whether the size of that difference is scientifically trivial or important.