Confidence Intervals for Comparing Two Treatment or Population Means Study Guide (page 4)
Find practice problems and solutions for these concepts at Confidence Intervals for Comparing Two Treatment or Population Means Practice Problems.
Matched-pairs and two-group designs were considered in the previous lesson, but only the paired design was discussed in detail. Now we will focus on the two-group design and on random samples from two populations. Design considerations as well as inference for the difference in the treatment or population means will be discussed.
Suppose we have decided to conduct a study using a two-group design. As with the paired design, we begin by selecting the study units. If this selection is made at random from some population, inferences can be made for this population at the end of the study. Otherwise, inference will be restricted to units in the study.
After the study units have been chosen, half are randomly assigned to the first treatment; the other half receive the second treatment. It is not necessary for the two groups to be evenly divided as just described. We could flip a fair coin to determine which treatment each unit receives. Although about half would get each treatment, it is likely that one treatment will have a few more study units than the other treatment. There are times in which we want to have more units receiving one treatment than another. However, in the absence of additional information, we will seek a randomization process that will result in the same number of units within each group.
The goal of a two-group study is usually to compare the means of the two groups. Let the mean of the first population be denoted by μ1 and the mean of the second population by μ2. Let X1i be the observed ith response from the first treatment, i = 1, 2, . . . , n1, where n1 units are receiving treatment 1. Similarly, let X2i be the observed response from the ith unit receiving the second treatment, i = 1, 2, . . . , n2.We use the sample mean of the units receiving the first treatment to estimate that treatment mean, and the sample mean of the units under the second treatment to estimate the second treatment mean. Let and be the sample means based on units receiving treatments 1 and 2, respectively.
To estimate the difference in the two treatment means, μ1 – μ2, we would use – . Although we have only one sample from each treatment, we can imagine repeating the study many times, and computing – each time. This gives rise to the sampling distribution of – . If each population distribution is normal, the sampling distribution of – is normal with mean μ1 – μ2 and variance .
The standard error of – depends on whether the variances of the units receiving the two treatments are equal . If we believe that the two variances are equal, then we want to use information from each sample to estimate the common variance; that is, we want to find the pooled estimate of the variance. The term pooled estimate means that information from multiple samples is combined to provide one estimate. We must allow for the fact that the means could be different under the two treatments. These ideas lead us to use (called s-squared pooled),
as the estimate of this common variance. Notice that is a weighted average of the estimated variances within each treatment. If the two samples sizes (n1 and n2) are equal, is the average of and ; otherwise, the sample having the largest number of observations has the largest weight. Assuming that the variance is the same under the two treatments, the standard error of – is .
What happens if we are unwilling to assume that the variances are the same under the two treatments? In this case, we must obtain estimates of the variance for units receiving each treatment. That is, is the sample variance for all units receiving treatment 1. Similarly, is the sample variance for all units receiving treatment 2. The standard error of – is .
A large telemarketing firm acquired a new client with a product. A script for the sales people to use when calling prospective customers needed to be developed. Because the product was different from ones the firm had handled in the past, the script writers were divided as to which of two approaches, a hard-sell approach or a soft-sell approach, would result in the greatest number of sales. They decided to conduct a study to compare the two approaches. Eighty people were randomly selected from the sales force. Of these, 40 were randomly assigned to use the hard-sell approach; the other 40 were to use the soft-sell approach. Each person was then trained using the script of the method to which he or she was assigned. After having each study participant use the script for one day, the number of sales made during a randomly selected hour during the next work day was recorded. The results are in Table 19.1.
- This study has a two-group design. Explain why this statement is true.
- Estimate the mean and standard deviation for each treatment.
- Is it reasonable to assume the variance is the same for both populations? If so, estimate the variance common to both.
- Estimate the difference in the treatment means and find its standard error.
- Is it reasonable to assume that the numbers of sales are normally distributed for each treatment?
- To which population may inference be drawn from this study?
- The hard-sell approach was randomly assigned to half of the study participants, and the other half of the study participants was assigned the soft-sell approach. No effort was made to pair the study participants to control other factors.
- The estimated mean number of sales per hour when using the hard-sell approach is 1.45 sales. The estimated variance of the number of sales per hour for this approach is 1.59 sales2, so the estimated standard deviation is 1.26 sales. The estimated mean and variances of the number of sales per hour when using the soft-sell approach is 2.38 sales and 1.93 sales2, respectively. The estimated standard deviation of the number of sales using the soft-sell approach is 1.39 sales.
- Because the standard deviations for the two treatment groups are similar, it is reasonable to assume that they are estimating a common variance.
Using the subscript H to represent the hard-sell approach and the subscript S to represent the soft-sell approach, the estimate of that common variance is:
The estimated difference in the mean number of sales using the hard-sell and the soft-sell approaches is – = 1.45 – 2.38 = – 0.93 sales; that is, 0.93 fewer sales are made, on average, using the hard-sell approach compared to the soft-sell approach. The standard error of this estimate is:
- For a two-group experiment, the condition of normality is checked within each treatment group. Because we are working with counts (the number of sales in an hour), the data are discrete. They cannot be normally distributed. We will focus on the shape of the sample distributions. Figures 19.1 and 19.2 show parallel dotplots and parallel boxplots. Based on the dotplot, the sample distribution of the hard-sell approach is skewed to the right, but the distribution of the soft-sale approach is reasonably symmetric. The boxplot supports the view that the sample distribution of the hard-sell approach is skewed to the right; further, the lone observation of five sales in an hour is an outlier. Based on the boxplot, the symmetry of the sample distribution of the soft-sell approach is a reasonable assumption though some may believe the distribution to be skewed left.
- Because the study participants were randomly drawn from the firm's sales force, the sales force of this large telemarketing firm is the population to which inference may be drawn.
Comparing Two Populations
If we randomly select samples from each of two populations, the two samples are independent. The statistical methods used to compare the means of two populations based on independent samples from each population are identical to those used in analyzing studies of the two-group design. We estimate the difference using – . The standard error of – is if the two population standard deviations are equal. If the two population standard deviations are not equal, then the standard error of – is .
A researcher conducted a study to compare traits of identical and fraternal twins. She wanted to know whether the mean difference in twin heights was different for identical and fraternal twins. She recruited 30 identical twin pairs and 30 fraternal twin pairs to participate in the study. The difference in each pair's height was recorded and presented in Table 19.3.
- This study compares two population means. Explain why this statement is true.
- Estimate the mean and standard deviation for each population.
- Is it reasonable to assume the variance is the same for both populations? If so, estimate the variance common to both?
- Estimate the difference in the population means and find the standard error of this estimate.
- Is it reasonable to assume that the differences in twin heights are normally distributed for each population?
- To which populations may inference be drawn from this study?
- The populations of interest are the population of identical twins and the population of fraternal twins. The type of twins cannot be assigned at random. Fraternal and identical twins constitute different populations.
- The estimated mean difference in the heights of identical twins is 1.68 cm. The estimated variance of the difference in the heights of identical twins is 2.10 cm2, and the estimated standard deviation is 1.45 cm. The estimated mean difference in the heights of fraternal twins is 3.40 cm. The estimated variance of the difference in heights of fraternal twins is 14.76 cm2, and the estimated standard deviation is 3.842 cm.
- The variance of the difference in heights of fraternal twins is about seven times the variance of the difference in heights of identical twins. Thus, it is unlikely that these are estimates of the same quantity. (In general, if one variance is about four times that of the other, then it is unlikely the two are equal.) Thus, we would not want to estimate a common variance.
- The estimated difference in population means is – = 3.9 – 1.5 = 2.4 cm. Because the variances are not the same, the standard error of the estimate is
Parallel dotplots and boxplots are shown in Figures 19.3 and 19.4. Both graphs indicate that the sample distributions are skewed right. The difference in identical twin heights has an outlier as well. Normality is not a reasonable assumption for these populations.
- Because twins were recruited and not randomly selected, inference may be drawn only to twins in the study. We would hope that this sample is representative of the larger population of identical and fraternal twins so that the inferences could be drawn more broadly. However, we cannot be assured of this.
Confidence Intervals Comparing Two Means
As before, let and be the sample means based on units receiving treatments one and two, respectively. Then – is a point estimate of the difference in the two treatment means, μ1 – μ2. The standard error of – is if and if .
To set a confidence interval on the difference in two treatment means, μ1– μ2, using the methods outlined here, two conditions must be satisfied. First, the treatments must be independent random samples from the population of units receiving treatment 1 and treatment 2. Suppose the units are randomly selected from the population and then randomly assigned to either treatment 1 or treatment 2. The random selection of the units gives us the random samples, and the random assignment of treatment ensures independence. If the units are not randomly selected, then we must rely solely on the random assignment of the treatments to give us a population of all possible samples for the two treatments from these units. Either way, the random assignment of treatments is critical for inference. Second, the responses must either be normally distributed, or the sample size for each treatment must be large enough (n ≥30) so that, by the Central Limit Theorem, each estimated treatment mean is approximately normal. Once these conditions are met, the approach we use will depend on whether we believe or . We will consider these two cases in turn.
First, assume . To standardize – , we take , which has a t-distribution with (n1 + n2 – 2) degrees of freedom. Thus, a 100(1 – α)% confidence interval on μ1 – μ2 is (– ) where t* with (n1 + n2 – 2) is the tabulated value such that .
Next, suppose that . Standardizing – , we have .
This standardized variable is only approximately distributed as a t-distribution, and the approximation involves a complicated formula for the degrees of freedom. That is, .
Typically, computation of these degrees of freedom is built into a calculator or computer software. A 100(1 – α)% confidence interval has the form , where t* has the degrees of freedom given previously.
Consider the telemarketing example in the previous lesson. Let the subscript H represent the hard-sell approach and the subscript S represent the soft-sell approach. Set an 80% confidence interval on the difference in the treatment means, μH – μS.
Two conditions must be satisfied. Randomly selected members of the sales force were assigned at random to the two treatments, so the first condition is satisfied. We noted earlier that the data consisted of discrete counts so they could not be normally distributed. However, the sample size is 40 for each treatment, allowing us to invoke the Central Limit Theorem.
In the previous lesson, we found – = –0.93 sales. The estimated variance for the hard-sell approach is 1.59 sales2, and that for the softsell approach is 1.93 sales2. Because these two estimates are close to each other, we assume that . Therefore, as we saw earlier, the standard error of – is = 0.30. For an 80% confidence interval, α = 0.20. We must find t*, such that where we have (n1 + n2 – 2) = 40 + 40 –2 =78 df. From the t-table in Lesson 12, we look in the row for 78 df and the column for α = 0.10 to find t* = 1.292. Therefore, an 80% confidence interval on μH – μS is –0.93 ± 1.292(0.30) or –0.93 ± 0.087. Therefore, we are 80% confident that, on average, the number of sales using the hard-sell approach is between 0.84 and 1.02 less each hour than using the soft-sell approach. Notice that the negative number meant that fewer sales were made using the hard-sell approach because we were estimating μH–μS. A positive number would have indicated that the estimated mean for the hard-sell approach was larger than that of the soft-sell approach.
Differences in Two Population Means
For a confidence interval on the difference in two population means to be valid, two conditions must be met. First, samples must be selected randomly and independently from the two populations. Second, for each population, the responses must be normally distributed or the sample size must be sufficiently large to invoke the Central Limit Theorem. If these two conditions are satisfied, the process of establishing the confidence intervals is the same as that used for a two group design.
Set a 95% confidence interval on the difference in the mean difference in heights for fraternal and identical twins using the data in the previous lesson.
First, consider the conditions. The twins were recruited and not randomly selected from all fraternal and identical twins. We must assume that these samples are representative of the populations if we are to proceed. We will make this assumption, knowing that it is a potential weakness in our study. Second, only nonnegative values can be observed, and the sample distributions appear skewed. Therefore, the distribution of differences in twin heights is not normal for either fraternal or identical twins. However, the sample size is 30, so we will assume that the Central Limit Theorem can be applied.
The estimated mean difference in the heights for fraternal and identical twins is – = 2.4 cm, and the standard error of this estimate is , where the subscript F indicates fraternal twins and the subscript I represents identical twins. The degrees of freedom are
To look up the tabulated value, we will round to the nearest integer, 37 in this case. In the row corresponding to 37 df and the column under 0.025 in the t-table, we have 2.024. Based on these values, the 95% confidence interval for μF – μI is 2.4 ± 2.024 or 2.4 ± 1.518. We estimate that the mean difference in fraternal twins' heights is 2.4 cm greater than the mean difference in identical twins' heights, and we are 95% confident this estimate is within 1.5 cm of the difference in these two population means.
Differences in Two Population Proportions
Sometimes, we want to estimate the difference in two population proportions. For example, we want to estimate "the gender gap," or the difference in the proportions of men and women favoring a particular candidate. Two conditions must be satisfied to use the methods discussed here. First, independent samples are randomly selected from each of the populations. Suppose n1and n2 are the number of observations from populations 1 and 2, respectively. Further, the sample proportions, 1 and 2, respectively. Further, the sample proportions, and , are the estimates of population 1 and 2 proportions, p1 – p2, respectively. The second condition is that n1, n1(1 – ), n2, and n2(1 – ) are all at least 5, and preferably at least 10.
The estimate of the difference in population proportions, p1 – p2, is – . The standard error of this estimate is . Standardizing – , we have Therefore, the 100(1 – α) % confidence interval is (– ) ±where z* is the tabulated value of z such that .
In a two-group design, treatments are randomly assigned to the experimental units. For a two-group design, the methods for setting confidence intervals on the difference in two treatment means were discussed. An important step in this process is determining whether or not the variances of the units under each treatment are equal. The methods are the same when comparing the means of two populations or two population proportions. Although not covered here, the procedures for hypothesis testing have the same extensions as those for confidence intervals.
Find practice problems and solutions for these concepts at Confidence Intervals for Comparing Two Treatment or Population Means Practice Problems.
- Kindergarten Sight Words List
- First Grade Sight Words List
- 10 Fun Activities for Children with Autism
- Signs Your Child Might Have Asperger's Syndrome
- Definitions of Social Studies
- A Teacher's Guide to Differentiating Instruction
- Curriculum Definition
- What Makes a School Effective?
- Theories of Learning
- Child Development Theories