Fisher's test or chi-square test?

Print this Topic

If you entered data with two rows and two columns, you must choose the chi-square test or Fisher's exact test.

GraphPad's advice: Choose Fishers tests, unless someone requires you to use chi-square test. If your values are huge, Prism will override your choice and compute the chi-square test, which is very accurate with large values.

Conventional advice

In the days before computers were readily available, people analyzed contingency tables by hand, or using a calculator, using chi-square tests. But the chi-square test is only an approximation. The Yates' continuity correction is designed to make the chi-square approximation better, but it overcorrects so gives a P value that is too large (too 'conservative'). With large sample sizes, Yates' correction makes little difference, and the chi-square test works very well. With small sample sizes, chi-square is not accurate, with or without Yates' correction.

Fisher's exact test, as its name implies, always gives an exact P value and works fine with small sample sizes. Fisher's test (unlike chi-square) is very hard to calculate by hand, but is easy to compute with a computer. Most statistical books advise using it instead of chi-square test.

Fisher's test. Exactly correct answer to wrong question?

As its name implies, Fisher's exact test, gives an exactly correct answer no matter what sample size you use. But some statisticians conclude that Fisher's test gives the exact answer to the wrong question, so its result is also an approximation to the answer you really want. The problem is that the Fisher's test is based on assuming that the row and column totals are fixed by the experiment. In fact, the row totals (but not the column totals) are fixed by the design of a prospective study or an experiment, the column totals (but not the row totals) are fixed by the design of a retrospective case-control study, and only the overall N (but neither row or column totals) is fixed in a cross-sectional experiment. Since the constraints of your study design don't match the constraints of Fisher's test, you could question whether the exact P value produced by Fisher's test actually answers the question you had in mind.

An alternative to Fisher's test is the Barnard test. Fisher's test is said to be 'conditional' on the row and column totals, while Barnard's test is not. Mehta and Senchaudhuri explain the difference and why Barnard's test has more power. Berger modified this test to one that is easier to calculate yet more powerful. He also provides an online web calculator so you can try it yourself.

It is worth noting that Fisher convinced Barnard to repudiate his test!

At this time, we do not plan to implement Bernard's or Berger's test in Prism. There certainly does not seem to be any consensus among statisticians that these tests are preferred. But let us know if you disagree.


Copyright (c) 2007 GraphPad Software Inc. All rights reserved.