Jason's Homepage

Stats Notes

SEMrefs

Statistics Links

Other links

Lecture 13
More on Chi-square

There are several loose ends on chi-square that I would like to tie up.

Simple Chi-square Problems
My discussion thus far has primarily focused on the 2 X 2 contingency table which looks at the goodness-of-fit or the independence of two dichotomous variables. I skipped a simpler example of chi-square which I would like to return to now.

A very easy and simple example of chi-square is when the frequencies of response to one dichotomous variable are compared. For this, we have just one variable that is tested. Examples of this are very common in the newspapers when polls are published. For instance, we might conduct a statewide survey of a sample of 300 hospital workers and ask the question "Do you think that staffing is adequate at your hospital, yes or no?" The following might be a hypothetical result:

No

Yes

180

120

The chi-square test asks if frequency of no responses is statistically larger (or different) from the frequency of yes responses. To compute this, we first need to know the expected frequency. Because we expect equal to imply a .5 chance of answering no or yes, we should expect half, or 150, of the survey participants to say yes or no. Thus we have this table of observed and expected frequencies:

 

No

Yes

Oi

180

120

Ei

150

150

We then apply our usual chi-square formula:

The degrees of freedom for this test is just the number of columns - 1 (c - 1 = 1). The critical value to exceed is equal at a = .05 is 3.84. Thus, respondents were significantly more likely to say no than yes to the question about adequate staffing.

This test can be conducted another way, using proportions. Usually survey responses are reported as the percentage (or, less often, proportion) of respondents indicating yes or no. These same results could be described as 60% indicating no and 40% indicating yes. The test to see whether these proportions are different is also appropriate for this example:


where is the obtained proportion and (the Greek letter pi) is the proportion expected in the population (usually .5), and n is the sample size for the whole sample. Plugging in our results we get:

Because chi-square and z are related, we should be able to show the equivalence of the two tests:

And, within rounding error, this is the answer we got when we used the chi-square test of frequencies. The nice thing about the proportion test is that we can calculate a "margin of error" by computing our confidence intervals. I will just compute the UCL here.

This computation indicates that we can expect our survey to be accurate within about .048 or 5% on either side, because the confidence limits span from .55 to .65. This is the "margin of error" that is often reported with surveys in the media.

Of course, the same equivalence holds for the chi-square with at 2 X 2 contingency table and the test of two proportions that is presented in the book.

Post Hoc Follow-up Tests
Another chi-square topic concerns follow-up or post hoc tests when a larger contingency table (e.g., 3 X 3) is analyzed. Just like with the ANOVA test, the chi-square test is an "omnibus" test, because it is an overall test to see if there are differences between any of the cell frequencies. If just two of the cells in the design are significantly different, the omnibus test will be significant.

It is appropriate to conduct follow-up chi-square tests if one wants to discover which pairs of cells are significantly different. In conducting follow-up tests, some statisticians like to distinguish between "post hoc" follow-up tests and "a priori" tests. "Post hoc" means after the fact, and "a priori" means prior or planned ahead of time. The post hoc test is one that is not planned before the initial test and usually involves many or all possible comparisons. A priori tests on the other hand are typically fewer in number and test specific cell differences within the design that were predicted ahead of time.

For the post hoc tests, authors like to recommend a correction for problems with alpha inflation. In order to compensate for the alpha inflation problem, a researcher can set the alpha value required for significance to a lower value. The formula that is often used is called Bonferroni's correction and corrects for the number of chi-square tests and the d.f. involved in each test (actually this is a modified Bonferroni test suggested by Keppel). If we conducted 10 follow-up tests, each with d.f. = 1, we would use:

So, we need to reset alpha to .04 to account for alpha inflation when conducting 10 single degree of freedom tests. Notice that alpha inflation is not that large for this many tests.

Within-subjects Chi-Square
There is a within-subjects version of the chi-square test. Imagine a study in which participants repeated the same measure twice, and the measure is a dichotomous one. In this case, the usual chi-square test is not appropriate, because we have a within-subjects design. Let's use the previous example of the hospital survey, but let's assume we first survey the 300 staff members before some important reorganization of staff is implemented. Then we resurvey them to see if their attitudes are changed. Notice that the frequency table looks similar to the 2 X 2 contingency table we had before. However, there is an important difference, and that is the fact that the measurements are repeated on the same group of 300. Our sample size is still 300 total.

 

 

Time 2

 

 

 

 

No

Yes

Total

Time

No

80

100

180

1

Yes

10

110

120

Total

 

90

210

300

The test for this is simple. It is also a chi-square test, but it is called McNemar's chi-square test. If we label our cells like before, we can use a simple computational formula.

 

 

Time 2

 

 

 

No

Yes

Time

No

a

b

1

Yes

c

d

 

This is a d.f.=1 test, and the critical value at a = .05 is 3.84. Thus, the change in opinions about staffing was wildly significant (sarcasm implied with the term "wildly").

Be careful about how you set up this table. Notice first that the cells represent a cross of people who said no on the first occasion and yes on the second occasion etc. Also, whether c is subtracted from b or a from d depends on how one sets up the table. One could list Yes first and No second, for instance. What you should keep in mind is that the test is based on the discordant cells (yes at Time 1 and no at Time 2; or no at Time 1 and yes at Time 2). Furthermore, it really does not matter what order the discordant cells are subtracted or added in the formula. We could have subtracted c from b instead. Because we square the difference, it makes no difference which is subtracted from which.

Multi-way Frequency Tables
Like factorial ANOVA, we can also examine two independent variables with chi-square. Our example in the 2 X 2 chi-square in the previous lecture involved gender differences in political affiliation. One can think of this as an independent variable of gender and a dichotomous dependent variable of political affiliation. We could complicate this by examining the same problem, but comparing regions (e.g., northern states to southern states). Our results might look like this:

Northern

 

Females

Males

Total

Democrats

50

70

120

Republicans

19

41

60

Total

69

111

180

 

Southern

 

Females

Males

Total

Democrats

70

30

100

Republicans

20

80

100

Total

90

110

200

One can think of the region variable as another independent variable. Does the effect of gender on political affiliation depend on the region the voter is from? In other words, is there an interaction between two independent variables. Notice, however, that the 2 X 2 X 2 contingency analysis is really analogous to the 2 X 2 ANOVA, because one of the 2's is the dichotomous outcome.

There are several ways to analyze this more complex design. One way is through a special chi-square that tests the equivalence of the two 2 X 2 contingency tables. Unfortunately, this test is not discussed or used very frequently, and is not available in SPSS. A second way to analyze this data is to compare the relative risk in the Northern region and the Southern region. That analysis compares two odds ratios and involves testing the interaction with logistic regression. We will briefly discuss this in a later section. A third way to analyze this design is to use something called "loglinear analysis." We will not be covering this topic in this class, but it is available in SPSS.

 

PA551 Home Page

Class Announcements

Homework Assignments

Course Syllabus and Requirements

Web Lectures Page  

 Reading Assignments