When to use the binomial test rather than the chi-square test

The binomial test is an exact test to compare the observed distribution to the expected distribution when there are only two categories (so only two rows of data were entered). In this situation, the chi-square is only an approximation, and we suggest using the exact binomial test instead.

Example

Assume that your theory says that an event should happen 20% of the time. In fact, in an experiment with 100 repetitions, that event happened only 7 times. You expected the event to occur 20 times (20% of 100) but it only occurred 7 times. How rare a coincidence is that? That is the question the binomial test answers.

Create a parts-of-whole table, and enter 7 into row 1 and 93 into row 2, and label the rows if you like. Click Analyze, and choose Compare observed distribution with expected in the Parts of whole section. Enter the expected values (20 and 80) and choose the binomial test (rather than chi-square)

Prism reports both one- and two-tail P values.

One-tail P value

The one-tail P value (also called a one sided P value) is straightforward. The null hypothesis is that the expected results are from a theory that is correct. So the P value answers the question:

If the true proportion is 20%, what is the chance in 100 trials that you'll observe 7 or fewer of the events?

You need to include the "or fewer" because it would have been even more surprising if the number of events in 100 trials was any value less than seven.

The one-tail P value for this example is:  0.0003.

If the observed value is less than the expected value, Prism reports the one-tail P value which is the probability of observing that many events or fewer. If the observed value is greater than the expected value, Prism reports the one-tail P value which is the probability of observing that many events or more.

Two-tail P value

The two-tail P value is a bit harder to define. In fact, there are (at least) three ways to define it.

Prism uses the third definition below, and this is the P value Prism uses when it creates the summary (* or **...).

•Double the one-tail P value. Twice 0.0002769 equals 0.0005540 That seems sensible, but that method is not used. Unless the expected proportion is 50%, the asymmetry of the binomial distribution makes it unwise to simply double the one-tail P value.

•Equal distance from expected. The theory said to expect 20 events. We observed 7. The discrepancy is 13 (20-7). So the other tail of the distribution should be the probability of obtaining 20+13=33 events or more. The two-tailed P value, computed this way, is the probability of obtaining 7 or less (0.0002769; the same as the one-tail P value) plus the probability of obtaining 33 or more (0.001550441) which means the two-tail P value equals 0.00182743..

•Method of small P values. To define the second tail with this method, we don't go out the same distance but instead start the second tail at an equally unlikely value. The chance of observing exactly 7 out of 100 events when the true probability is 0.20 equals 0.000199023. The probability of obtaining 33 events (how the second tail was defined in the other method) is higher: 0.000813557. The chance of obtaining 34 events is also higher. But the chance of observing 35 events is a bit lower (0.000188947). The second tail, therefore, is defined as the chance of observing 35 or more events. That tail is 0.0033609. The two tail P value therefore is 0.00061307. This is the method that Prism uses.

The distinction between the second and third methods is subtle. The first tail is unambiguous. It starts at 7 and goes down to zero. The second tail is symmetrical, but there are two ways to define this. The second method is symmetrical around the counts. In other words, the border for that tail (33) is as far from the expected value of 20 as is the observed value of 7 (33-20=20-7). The third method is symmetrical regarding probabilities.  Given the assumption that the true probability is 20% so we expect to observe 20, the chance of observing 7 events is about the same as the chance of observing 35. So the second tail is the probability of observing 35 or more events.

If the expected probability is  0.5, the binomial distribution is symmetrical and all three methods give the same result. When the expected probability is 0.5, then the binomial test is the same as the sign test.