| Interpreting a one-sample t test
How a one-sample t test works
A one-sample t test compares the mean of a single column of numbers against a hypothetical mean you entered. Prism calculates the t ratio from this equation:
A P value is computed from the t ratio and the numbers of degrees of freedom (which equals sample size minus 1).
How to think about results from the one-sample t test
Look first at the P value, which answers this question: If the data were sampled from a Gaussian population with a mean equal to the hypothetical value you entered, what is the chance of randomly selecting N data points and finding a mean as far from the hypothetical value (or further) as observed here?
"Statistically significant" is not the same as "scientifically important". Before interpreting the P value or confidence interval, you should think about the size of the difference you are seeking. How large a difference (between the population mean and the hypothetical mean) would you consider to be scientifically important? How small a difference would you consider to be scientifically trivial? You need to use scientific judgment and common sense to answer these questions. Statistical calculations cannot help, as the answers depend on the context of the experiment.
You will interpret the results differently depending on whether the P value is small or large.
If the P value is small (one-sample t test)
If the P value is small (usually defined to mean less than 0.05), then it is unlikely that the discrepancy you observed between sample mean and hypothetical mean is due to a coincidence arising from random sampling. You can reject the idea that the difference is a coincidence, and conclude instead that the population has a mean different than the hypothetical value you entered. The difference is statistically significant. But is the difference scientifically significant? The confidence interval helps you decide.
The true difference between population mean and hypothetical mean is probably not the same as the difference observed in this experiment. There is no way to know the true difference between the population mean and the hypothetical mean. Prism presents the uncertainty as a 95% confidence interval. You can be 95% sure that this interval contains the true difference between the overall (population) mean and the hypothetical value you entered.
To interpret the results in a scientific context, look at both ends of the confidence interval and ask whether they represent a discrepancy that is scientifically important or scientifically trivial.
| Lower confidence limit |
Upper confidence limit |
Conclusion |
| Trivial |
Trivial |
Although the true difference is not zero (since the P value is low), the difference is tiny and uninteresting. The data have a mean distinct from the hypothetical value, but the discrepancy is too small to be scientifically interesting. |
| Trivial |
Important |
Since the confidence interval ranges from a difference that you think is biologically trivial to one you think would be important, you can't reach a strong conclusion from your data. You can conclude that the data has a mean distinct from the hypothetical value you entered, but don't know whether that difference is scientifically trivial or important. You'll need more data to obtain a clear conclusion.
|
| Important |
Important |
Since even the low end of the confidence interval represents a difference large enough to be considered biologically important, you can conclude that the data have a mean distinct from the hypothetical value, and the discrepancy is large enough to be scientifically relevant.
|
If the P value is large (one sample t test)
If the P value is large, the data do not give you any reason to conclude that the overall mean differs from the hypothetical value you entered. This is not the same as saying that the true mean equals the hypothetical value. You just don't have evidence of a difference.
How large could the true difference really be? Because of random variation, the difference between the hypothetical mean and the sample mean in this experiment is unlikely to be equal to the true difference between population mean and hypothetical mean. There is no way to know the true difference between the population mean and the hypothetical mean. Prism presents the uncertainty as a 95% confidence interval. You can be 95% sure that this interval contains the true difference between the overall (population) mean of the data and the hypothetical mean you entered. When the P value is larger than 0.05, the 95% confidence interval will start with a negative number (the hypothetical mean is larger than the actual mean) and go up to a positive number (the actual mean is larger than the hypothetical mean).
To interpret the results in a scientific context, look at both ends of the confidence interval and ask whether they represent differences that would be scientifically important or scientifically trivial.
| Lower confidence limit |
Upper confidence limit |
Conclusion |
| Trivial |
Trivial |
You can reach a crisp conclusion. Either the data has a mean equal to the hypothetical mean or they differ by a trivial amount. |
| Trivial |
Large |
You can't reach a strong conclusion. The data are consistent with a mean slightly smaller than the hypothetical mean, equal to the hypothetical mean, or larger than the hypothetical mean, perhaps large enough to be scientifically important. To reach a clear conclusion, you need to repeat the experiment with more subjects.
|
| Large |
Trivial |
You can't reach a strong conclusion. The data are consistent with a mean smaller than the hypothetical mean (perhaps small enough to be scientifically important), equal to the hypothetical mean, or slightly larger than the hypothetical mean. You can't make a clear conclusion without repeating the experiment with more subjects.
|
| Large |
Large |
You can't reach a strong conclusion. The data are consistent with a mean smaller than the hypothetical mean (perhaps small enough to be scientifically important), equal to the hypothetical mean, or larger than the hypothetical mean (perhaps large enough to be scientifically important). In other words, you can't draw any conclusion at all. You need to repeat the experiment with more subjects. |
Checklist. Is a one-sample t test the right test for these data?
Before accepting the results of any statistical test, first think carefully about whether you chose an appropriate test. Before accepting results from a one-sample t test, ask yourself these questions:
| Question |
Discussion |
| Is the population distributed according to a Gaussian distribution? |
The one sample t test assumes that you have sampled your data from a population that follows a Gaussian distribution. While this assumption is not too important with large samples, it is important with small sample sizes, especially when N is less than 10. Prism tests for violations of this assumption, but normality tests have limited utility. See The results of normality tests. If your data do not come from a Gaussian distribution, you have three options. Your best option is to transform the values to make the distribution more Gaussian, perhaps by transforming all values to their reciprocals or logarithms. Another choice is to use the Wilcoxon rank sum nonparametric test instead of the t test. A final option is to use the t test anyway, knowing that the t test is fairly robust to departures from a Gaussian distribution with large samples. |
| Are the "errors" independent? |
The term "error" refers to the difference between each value and the group mean. The results of a t test only make sense when the scatter is random - that whatever factor caused a value to be too high or too low affects only that one value. Prism cannot test this assumption. See The need for independent samples. |
| If you chose a one-tail P value, did you predict correctly? |
If you chose a one-tail P value, you should have predicted whether the mean of your data would be larger than or smaller than the hypothetical mean. Prism does not ask you to record this prediction, but assumes that it is correct. If your prediction was wrong, then ignore the P value reported by Prism and state that P>0.50. See One- vs. two-tail P values. |
|
