Point of confusion: ANOVA with a quantitative factor

Two-way ANOVA is sometimes used when one of the factors is quantitative, such as when comparing time courses or dose response curves. In these situations one of the factors is dose or time.

GraphPad's advice: If one of your factors is quantitative (such as time or dose) think hard before choosing two-way ANOVA. Other analyses may make more sense.

Let's imagine you compare two treatments at six time points. The ANOVA analysis treats different time points exactly as it would treat different drugs or different species. The concept of trend is entirely ignored (except in some special post tests).

•	One P value tests the null hypothesis that time has no effect on the outcome. It rarely makes sense to test this hypothesis. Of course time affects the outcome! That's why you did a time course.

•

Another P value tests the null hypothesis that the treatment makes no difference, on average. This can be somewhat useful. But you probably expect no difference at early (or maybe late) time points, and only care about differences at late time points. So it may not be useful to ask if, on average, the treatments differ.

•

The third P value tests for interaction. The null hypothesis is that any difference between treatments is identical at all time points. But if you collect data at time zero, or at early time points, you don't expect to find any difference then. Your experiment really is designed to ask about later time points. In this situation, you expect an interaction, so finding a small P value for interaction does not help you understand your data.

ANOVA pays no attention to the order of your time points (or doses). If you randomly scramble the time points or doses, two-way ANOVA would report identical results. In other words, ANOVA ignores the entire point of the experiment, when one of the factors is quantitative.

Some scientists like to ask which is the lowest dose (or time) at which the change in response is statistically significant. Post tests can give you the answer, but the answer depends on sample size. Run more subjects, or more doses or time points for each curve, and the answer will change. With a large enough sample size (at each dose), you will find that a tiny dose causes a statistically significant, but biologically trivial, effect. This kind of analysis does not ask a fundamental question, and so the results are rarely helpful.

If you want to know the minimally effective dose, consider finding the minimum dose that causes an effect bigger than some threshold you set based on physiology. For example, find the minimum dose that raises the pulse rate by more than 10 beats per minute.

If you look at all the post tests (and not just ask which is the lowest dose or time point that gives a 'significant' effect), you can get results that make no sense. You might find that the difference is significant at time points 3, 5, 6 and 9 but not at time points 1, 2, 4, 7, 8 and 10. How do you interpret that? Knowing at which doses or time points the treatment had a statistically significant rarely helps you understand the biology of the system and rarely helps you design new experiments.

If you have a repeated measures design, consider using an alternative to ANOVA. Will G Hopkins calls the alternative within-subject modeling.

First, quantify the data for each subject in some biologically meaningful way. Perhaps this would be the area under the curve. Perhaps the peak level. Perhaps the time to peak. Perhaps you can fit a curve and determine a rate constant or a slope.

Now take these values (the areas or rate constants...) and compare between groups of subjects using a t test (if two treatments) or one-way ANOVA (if three or more). Unlike two-way ANOVA, this kind of analysis follows the scientific logic of the experiment, and so leads to results that are understandable and can lead you to the next step (designing a better experiment).

If you don't have a repeated measures design, you can still fit a curve for each treatment. Then compare slopes, or EC50s, or lag times as part of the linear or nonlinear regression.

Think hard about what your scientific goals are, and try to find a way to make the statistical testing match the scientific goals. In many cases, you'll find a better approach than using two-way ANOVA.