These guided examples of common analyses will get you off to a great start! CLICK HERE >
Onsite training
LEARN MORE >
FAQ# 1790 Last Modified 23March2012
This is chapter 37 of the first edition of Intuitive Biostatistics by Harvey Motulsky. Copyright © 1995 by Oxford University Press Inc. Chapter 45 of the second edition of Intuitive Biostatistics is an expanded version of this material.
REVIEW OF AVAILABLE STATISTICAL TESTS
This book has discussed many different statistical tests. To select the right test, ask yourself two questions: What kind of data have you collected? What is your goal? Then refer to Table 37.1.
Type of Data  
Goal  Measurement (from Gaussian Population)  Rank, Score, or Measurement (from Non Gaussian Population) 
Binomial (Two Possible Outcomes) 
Survival Time 
Describe one group  Mean, SD  Median, interquartile range  Proportion  Kaplan Meier survival curve 
Compare one group to a hypothetical value  Onesample ttest  Wilcoxon test 
Chisquare or Binomial test ** 

Compare two unpaired groups  Unpaired t test  MannWhitney test 
Fisher's test (chisquare for large samples) 
Logrank test or MantelHaenszel* 
Compare two paired groups  Paired t test  Wilcoxon test  McNemar's test  Conditional proportional hazards regression* 
Compare three or more unmatched groups  Oneway ANOVA  KruskalWallis test  Chisquare test  Cox proportional hazard regression** 
Compare three or more matched groups  Repeatedmeasures ANOVA  Friedman test  Cochrane Q**  Conditional proportional hazards regression** 
Quantify association between two variables  Pearson correlation  Spearman correlation  Contingency coefficients**  
Predict value from another measured variable 
Simple linear regression or Nonlinear regression 
Nonparametric regression**  Simple logistic regression*  Cox proportional hazard regression* 
Predict value from several measured or binomial variables 
Multiple linear regression* or Multiple nonlinear regression** 
Multiple logistic regression*  Cox proportional hazard regression* 
REVIEW OF NONPARAMETRIC TESTS
Choosing the right test to compare measurements is a bit tricky, as you must choose between two families of tests: parametric and nonparametric. Many statistical test are based upon the assumption that the data are sampled from a Gaussian distribution. These tests are referred to as parametric tests. Commonly used parametric tests are listed in the first column of the table and include the t test and analysis of variance.
Tests that do not make assumptions about the population distribution are referred to as nonparametric tests. You've already learned a bit about nonparametric tests in previous chapters. All commonly used nonparametric tests rank the outcome variable from low to high and then analyze the ranks. These tests are listed in the second column of the table and include the Wilcoxon, MannWhitney test, and KruskalWallis tests. These tests are also called distributionfree tests.
CHOOSING BETWEEN PARAMETRIC AND NONPARAMETRIC TESTS: THE EASY CASES
Choosing between parametric and nonparametric tests is sometimes easy. You should definitely choose a parametric test if you are sure that your data are sampled from a population that follows a Gaussian distribution (at least approximately). You should definitely select a nonparametric test in three situations:
CHOOSING BETWEEN PARAMETRIC AND NONPARAMETRIC TESTS: THE HARD CASES
It is not always easy to decide whether a sample comes from a Gaussian population. Consider these points:
CHOOSING BETWEEN PARAMETRIC AND NONPARAMETRIC TESTS: DOES IT MATTER?
Does it matter whether you choose a parametric or nonparametric test? The answer depends on sample size. There are four cases to think about:
Thus, large data sets present no problems. It is usually easy to tell if the data come from a Gaussian population, but it doesn't really matter because the nonparametric tests are so powerful and the parametric tests are so robust. Small data sets present a dilemma. It is difficult to tell if the data come from a Gaussian population, but it matters a lot. The nonparametric tests are not powerful and the parametric tests are not robust.
ONE OR TWOSIDED P VALUE?
With many tests, you must choose whether you wish to calculate a one or twosided P value (same as one or twotailed P value). The difference between one and twosided P values was discussed in Chapter 10. Let's review the difference in the context of a t test. The P value is calculated for the null hypothesis that the two population means are equal, and any discrepancy between the two sample means is due to chance. If this null hypothesis is true, the onesided P value is the probability that two sample means would differ as much as was observed (or further) in the direction specified by the hypothesis just by chance, even though the means of the overall populations are actually equal. The twosided P value also includes the probability that the sample means would differ that much in the opposite direction (i.e., the other group has the larger mean). The twosided P value is twice the onesided P value.
A onesided P value is appropriate when you can state with certainty (and before collecting any data) that there either will be no difference between the means or that the difference will go in a direction you can specify in advance (i.e., you have specified which group will have the larger mean). If you cannot specify the direction of any difference before collecting data, then a twosided P value is more appropriate. If in doubt, select a twosided P value.
If you select a onesided test, you should do so before collecting any data and you need to state the direction of your experimental hypothesis. If the data go the other way, you must be willing to attribute that difference (or association or correlation) to chance, no matter how striking the data. If you would be intrigued, even a little, by data that goes in the "wrong" direction, then you should use a twosided P value. For reasons discussed in Chapter 10, I recommend that you always calculate a twosided P value.
PAIRED OR UNPAIRED TEST?
When comparing two groups, you need to decide whether to use a paired test. When comparing three or more groups, the term paired is not apt and the term repeated measures is used instead.
Use an unpaired test to compare groups when the individual values are not paired or matched with one another. Select a paired or repeatedmeasures test when values represent repeated measurements on one subject (before and after an intervention) or measurements on matched subjects. The paired or repeatedmeasures tests are also appropriate for repeated laboratory experiments run at different times, each with its own control.
You should select a paired test when values in one group are more closely correlated with a specific value in the other group than with random values in the other group. It is only appropriate to select a paired test when the subjects were matched or paired before the data were collected. You cannot base the pairing on the data you are analyzing.
FISHER'S TEST OR THE CHISQUARE TEST?
When analyzing contingency tables with two rows and two columns, you can use either Fisher's exact test or the chisquare test. The Fisher's test is the best choice as it always gives the exact P value. The chisquare test is simpler to calculate but yields only an approximate P value. If a computer is doing the calculations, you should choose Fisher's test unless you prefer the familiarity of the chisquare test. You should definitely avoid the chisquare test when the numbers in the contingency table are very small (any number less than about six). When the numbers are larger, the P values reported by the chisquare and Fisher's test will he very similar.
The chisquare test calculates approximate P values, and the Yates' continuity correction is designed to make the approximation better. Without the Yates' correction, the P values are too low. However, the correction goes too far, and the resulting P value is too high. Statisticians give different recommendations regarding Yates' correction. With large sample sizes, the Yates' correction makes little difference. If you select Fisher's test, the P value is exact and Yates' correction is not needed and is not available.
REGRESSION OR CORRELATION?
Linear regression and correlation are similar and easily confused. In some situations it makes sense to perform both calculations. Calculate linear correlation if you measured both X and Y in each subject and wish to quantity how well they are associated. Select the Pearson (parametric) correlation coefficient if you can assume that both X and Y are sampled from Gaussian populations. Otherwise choose the Spearman nonparametric correlation coefficient. Don't calculate the correlation coefficient (or its confidence interval) if you manipulated the X variable.
Calculate linear regressions only if one of the variables (X) is likely to precede or cause the other variable (Y). Definitely choose linear regression if you manipulated the X variable. It makes a big difference which variable is called X and which is called Y, as linear regression calculations are not symmetrical with respect to X and Y. If you swap the two variables, you will obtain a different regression line. In contrast, linear correlation calculations are symmetrical with respect to X and Y. If you swap the labels X and Y, you will still get the same correlation coefficient.