﻿ Q&A: Choosing a test to compare two groups

# Q&A: Choosing a test to compare two groups

### If I have data from three or more groups, is it OK to compare two groups at a time with a t test?

No. You should analyze all the groups at once with one-way ANOVA, and then follow up with multiple comparison tests. The only exception is when some of the 'groups' are really controls to prove the assay worked, and are not really part of the experimental question you are asking.

### I know the mean, SD (or SEM) and sample size for each group. Which tests can I run?

You can enter data as mean, SD (or SEM) and N, and Prism can compute an unpaired t test or the Welch t test. Prism cannot perform an paired test, as that requires analyzing each pair. It also cannot do any nonparametric tests, as these require ranking the data.

### I only know the two group means, and don't have the raw data and don't know their SD or SEM. Can I run a t test?

No. The t test compares the difference between two means and compares that difference to the standard error of the difference, computed from the standard deviations and sample size. If you only know the two means, there is no possible way to do any statistical comparison.

### Can I use a normality test to make the choice of when to use a nonparametric test?

It is not a good idea to base your decision solely on the normality test. Choosing when to use a nonparametric test is not a straightforward decision, and you can't really automate the process.

### I want to compare two groups. The outcome has two possibilities, and I know the fraction of each possible outcome in each group. How can I compare the groups?

Not with a t test. Enter your data into a contingency table and analyze with Fisher's exact test.

### I want to compare the mean survival time in two groups. But some subjects are still alive so I don't know how long they will live. How can I do a t test on survival times?

You should use special methods designed to compare survival curves. Don't run a t test on survival times.

### I don't know whether it is ok to assume equal variances. Can't a statistical test tell me whether or not to use the Welch t test?

While that sounds like a good idea, in fact it is not. The decision really should be made as part of the experimental design and not based on inspecting the data.

### I don't know whether it is better to use the regular paired t test or the ratio test. Is it ok to run both, and report the results with the smallest P value?

No. The results of any statistical test can only be interpreted at face value when the choice of analysis method was part of the experimental design.

### Should I use the Welch test routinely because it is always possible the two populations have different standard deviations.

Ruxton (1) and Delacre (2) make a strong case that this is a good idea.

## References

1. Ruxton. The unequal variance t-test is an underused alternative to Student's t-test and the Mann-Whitney U test. Behavioral Ecology (2006) vol. 17 (4) pp. 688

2. Delacre, M., Lakens, D.L., and Leys, C. (2017). Why Psychologists Should by Default Use Welch's t-test Instead of Student's t-test. Rips 30: 92–10.