Lecture
4
Sample Size
In this cyberlecture, I'd like to
outline a few of the important concepts relating to sample size. Generally,
larger samples are good, and this is the case for a number of reasons. So, I'm
going to try to show this in several different ways.
Bigger is Better
1. The first reason to understand why a large sample size is beneficial is
simple. Larger samples more closely approximate the population. Because the
primary goal of inferential statistics is to generalize from a sample to a
population, it is less of an inference if the sample size is large.
2. A second reason is kind of the
opposite. Small samples are bad. Why? If we pick a small sample, we run a
greater risk of the small sample being unusual just by chance. Choosing 5
people to represent the entire U.S., even if they are chosen completely at
random, will often result if a sample that is very unrepresentative of the
population. Imagine how easy it would be to, just by chance, select 5
Republicans and no Democrats for instance.
Let's take this point a little
further. If there is an increased probability of one small sample being
unusual, that means that if we were to draw many small samples as when a
sampling distribution is created (see the second lecture),
unusual samples are more frequent. Consequently, there is greater sampling
variability with small samples. This figure is another way to illustrate this:
Note: this is a dramatization to illustrate the
effect of sample sizes, the curves depicted here are fictitious, in order to
protect the innocent and may or may not represent real statistical sampling
curves. A more realistic depiction can be found on p. 163.
In the curve with the "small
size samples," notice that there are fewer samples with means around the
middle value, and more samples with means out at the extremes. Both the right
and left tails of the distribution are "fatter." In the curve with
the "large size samples," notice that there are more samples with
means around the middle (and therefore closer to the population value), and
fewer with sample means at the extremes. The differences in the curves
represent differences in the standard deviation of the sampling
distribution--smaller samples tend to have larger standard errors and larger
samples tend to have smaller standard errors.
3. This point about standard errors
can be illustrated a different way. One statistical test is designed to see if
a single sample mean is different from a population mean. A version of this
test is the t-test for a single mean. The purpose of this t-test is to
see if there is a significant difference between the sample mean and the
population mean. The t-test formula looks like this:
The t-test formula (also found on p.
161 of the Daniel text) has two main components. First, it takes into account
how large the difference between the sample and the population mean is by
finding the difference between them (
). When the sample mean is far from the
population mean, the difference will be large. Second, t-test formula divides
this quantity by the standard error (symbolized by 
). By dividing by the standard
error, we are taking into account sampling variability. Only if the difference
between the sample and population means is large relative to the amount of sampling
variability will we consider the difference to be "statistically
significant". When sampling variability is high (i.e., the standard error
is large), the difference between the sample mean and the population mean may
not seem so big.
|
Concept |
Mathematic Representation |
|
distance of the sample mean from the population mean |
|
|
representation of sampling variability |
|
|
Ratio of the distance from the population mean relative to the sampling variability |
t |
Now,
back to sample size... As we saw in the figure with the curves above, the
standard error (which represents the amount of sampling variability) is larger
when the sample size is small and smaller when the sample size is large. So,
when the sample size is small, it can be difficult to see a difference between
the sample mean and the population mean, because there is too much sampling
variability messing things up. If the sample size is large, it is easier to see
a difference between the sample mean and population mean because the sampling
variability is not obscuring the difference. (Kinda nifty how we get from an
abstract concept to a formula, huh? I took years of math, but until I took a
statistics course, I didn't realize the numbers and symbols in formulas really
signified anything).
4. Another reason why bigger is
better is that the value of the standard error is directly dependent on the
sample size. This is really the same reason given in #2 above, but I'll show it
a different way. To calculate the standard error, we divide the standard
deviation by the sample size (actually there is a square root in there).
In this equation, 
is the standard error, s
is the standard deviation, and n is the sample size. If we were to plug
in different values for n (try some hypothetical numbers if you want!), using
just one value for s, the standard error would be smaller for larger values of
n, and the standard error would be larger for smaller values of n.
5. There is a rule that someone came
up with (someone who had vastly superior brain to the population average) that
states that if sample sizes are large enough, a sampling distribution will be
normally distributed (remember that a normal distribution has special
characteristics; see p. 107 in the Daniel text; an approximately normally
distributed curve is also depicted by the large sample size curve in the figure
above). This is called the central limit theorem. If
we know that the sampling distribution is normally distributed, we can make
better inferences about the population from the sample. The sampling
distribution will be normal, given sufficient sample size, regardless of
the shape of the population distribution.
6. Finally, that last reason I can
think of right now why bigger is better is that larger sample sizes give us
more power. Remember that in the previous lecture
power was defined as the probability of retaining the alternative hypothesis
when the alternative hypothesis is actually true in the population. That is, if
we can increase our chances of correctly choosing the alternative hypothesis in
our sample, we have more power. If the sample size is large, we will have a
smaller standard error, and as described in the #3 and #4, we are more likely
to find significance with a lower standard errror.
Do I seem like I am repeating
myself? Probably. Part of the reason is that it is important to try to explain
these concepts in several different ways, but it is also because, in
statistics, everything is interrelated
How Big Should My
Sample Be?
This is a good question to ask, and it is frequently asked. Unfortunately,
there is not a really simple answer. It depends on the type of statistical test
one is conducting. It also depends on how precise your measures are and how
well designed your study is. So, it just depends. I often hear a general
recommendation that there be about 15 or more participants in each group when
conducting a t-test or ANOVA. Don't worry, we'll return to this question later.