Understanding Independent Samples t-Test
Statistical Technique in Review
The independent samples t-test is a parametric statistical technique used to determine significant differences between the scores obtained from two samples or groups. Since the t-test is considered fairly easy to calculate, researchers often use it in determining differences between two groups. The t-test examines the differences between the means of the two groups in a study and adjusts that difference for the variability (computed by the standard error) among the data. When interpreting the results of t-tests, the larger the calculated t ratio, in absolute value, the greater the difference between the two groups. The significance of a t ratio can be determined by comparison with the critical values in a statistical table for the t distribution using the degrees of freedom (df) for the study (see Appendix A Critical Values for Student’s t Distribution at the back of this text). The formula for df for an independent t-test is as follows:
df=(numberofsubjectsinsample1+numberofsubjectsinsample2)−2
Exampledf=(65insample1+67insample2)−2=132−2=130
The t-test should be conducted only once to examine differences between two groups in a study, because conducting multiple t-tests on study data can result in an inflated Type 1 error rate. A Type I error occurs when the researcher rejects the null hypothesis when it is in actuality true. Researchers need to consider other statistical analysis options for their study data rather than conducting multiple t-tests. However, if multiple t-tests are conducted, researchers can perform a Bonferroni procedure or more conservative post hoc tests like Tukey’s honestly significant difference (HSD), Student-Newman-Keuls, or Scheffé test to reduce the risk of a Type I error. Only the Bonferroni procedure is covered in this text; details about the other, more stringent post hoc tests can be found in Plichta and Kelvin (2013) and Zar (2010).
The Bonferroni procedure is a simple calculation in which the alpha is divided by the number of t-tests conducted on different aspects of the study data. The resulting number is used as the alpha or level of significance for each of the t-tests conducted. The Bonferroni procedure formula is as follows: alpha (α) ÷ number of t-tests performed on study data = more stringent study α to determine the significance of study results. For example, if a study’s α was set at 0.05 and the researcher planned on conducting five t-tests on the study data, the α would be divided by the five t-tests (0.05 ÷ 5 = 0.01), with a resulting α of 0.01 to be used to determine significant differences in the study.
The t-test for independent samples or groups includes the following assumptions:
1. The raw scores in the population are normally distributed.
2. The dependent variable(s) is(are) measured at the interval or ratio levels.162
3. The two groups examined for differences have equal variance, which is best achieved by a random sample and random assignment to groups.
4. All scores or observations collected within each group are independent or not related to other study scores or observations.
The t-test is robust, meaning the results are reliable even if one of the assumptions has been violated. However, the t-test is not robust regarding between-samples or within-samples independence assumptions or with respect to extreme violation of the assumption of normality. Groups do not need to be of equal sizes but rather of equal variance. Groups are independent if the two sets of data were not taken from the same subjects and if the scores are not related (Grove, Burns, & Gray, 2013; Plichta & Kelvin, 2013). This exercise focuses on interpreting and critically appraising the t-tests results presented in research reports. Exercise 31 provides a step-by-step process for calculating the independent samples t-test.
Research Article
Source
Canbulat, N., Ayhan, F., & Inal, S. (2015). Effectiveness of external cold and vibration for procedural pain relief during peripheral intravenous cannulation in pediatric patients. Pain Management Nursing, 16(1), 33–39.
Introduction
Canbulat and colleagues (2015, p. 33) conducted an experimental study to determine the “effects of external cold and vibration stimulation via Buzzy on the pain and anxiety levels of children during peripheral intravenous (IV) cannulation.” Buzzy is an 8 × 5 × 2.5 cm battery-operated device for delivering external cold and vibration, which resembles a bee in shape and coloring and has a smiling face. A total of 176 children between the ages of 7 and 12 years who had never had an IV insertion before were recruited and randomly assigned into the equally sized intervention and control groups. During IV insertion, “the control group received no treatment. The intervention group received external cold and vibration stimulation via Buzzy . . . Buzzy was administered about 5 cm above the application area just before the procedure, and the vibration continued until the end of the procedure” (Canbulat et al., 2015, p. 36). Canbulat et al. (2015, pp. 37–38) concluded that “the application of external cold and vibration stimulation were effective in relieving pain and anxiety in children during peripheral IV” insertion and were “quick-acting and effective nonpharmacological measures for pain reduction.” The researchers concluded that the Buzzy intervention is inexpensive and can be easily implemented in clinical practice with a pediatric population.
Relevant Study Results
The level of significance for this study was set at α = 0.05. “There were no differences between the two groups in terms of age, sex [gender], BMI, and preprocedural anxiety according to the self, the parents’, and the observer’s reports (p > 0.05) (Table 1). When the pain and anxiety levels were compared with an independent samples t test, . . . the children in the external cold and vibration stimulation [intervention] group had significantly lower pain levels than the control group according to their self-reports (both WBFC [Wong Baker Faces Scale] and VAS [visual analog scale] scores; p < 0.001) (Table 2). The external cold and vibration stimulation group had significantly lower fear and anxiety 163levels than the control group, according to parents’ and the observer’s reports (p < 0.001) (Table 3)” (Canbulat et al., 2015, p. 36).
TABLE 1
COMPARISON OF GROUPS IN TERMS OF VARIABLES THAT MAY AFFECT PROCEDURAL PAIN AND ANXIETY LEVELS
| Characteristic | Buzzy (n = 88) | Control (n = 88) | χ2 p |
| Sex | |||
| Female (%), n | 11 (12.5) | 13 (14.8) | .82 |
| Male (%), n | 77 (87.5) | 75 (85.2) | .41 |
| Characteristic | Buzzy (n = 88) | Control (n = 88) | t p |
| Age (mean ± SD) | 8.25 ± 1.51 | 8.61 ± 1.69 | −1.498 .136 |
| BMI (mean ± SD) | 25.41 ± 6.74 | 26.94 ± 8.68 | −1.309 .192 |
| Preprocedural anxiety | |||
| Self-report (mean ± SD) | 2.03 ± 1.29 | 2.11 ± 1.58 | −0.364 .716 |
| Parent report (mean ± SD) | 2.11 ± 1.20 | 2.17 ± 1.42 | −0.285 .776 |
| Observer report (mean ± SD) | 2.18 ± 1.17 | 2.24 ± 1.37 | −0.295 .768 |
BMI, body mass index.
Canbulat, N., Ayban, F., & Inal, S. (2015). Effectiveness of external cold and vibration for procedural pain relief during peripheral intravenous cannulation in pediatric patients. Pain Management Nursing, 16(1), p. 36.
TABLE 2
COMPARISON OF GROUPS’ PROCEDURAL PAIN LEVELS DURING PERIPHERAL IV CANNULATION
| Buzzy (n = 88) | Control (n = 88) | t p | |
| Procedural self-reported pain with WBFS (mean ± SD) | 2.75 ± 2.68 | 5.70 ± 3.31 | −6.498 0.000 |
| Procedural self-reported pain with VAS (mean ± SD) | 1.66 ± 1.95 | 4.09 ± 3.21 | −6.065 0.000 |
IV, intravenous; WBFS, Wong-Baker Faces Scale; SD, standard deviation; VAS, visual analog scale.
Canbulat, N., Ayban, F., & Inal, S. (2015). Effectiveness of external cold and vibration for procedural pain relief during peripheral intravenous cannulation in pediatric patients. Pain Management Nursing, 16(1), p. 37.
TABLE 3
COMPARISON OF GROUPS’ PROCEDURAL ANXIETY LEVELS DURING PERIPHERAL IV CANNULATION
| Procedural Child Anxiety | Buzzy (n = 88) | Control (n = 88) | t p |
| Parent reported (mean ± SD) | 0.94 ± 1.06 | 2.09 ± 1.39 | −6.135 0.000 |
| Observer reported (mean ± SD) | 0.92 ± 1.03 | 2.14 ± 1.34 | −6.745 0.000 |
SD, standard deviation; IV, intravenous.
Canbulat, N., Ayban, F., & Inal, S. (2015). Effectiveness of external cold and vibration for procedural pain relief during peripheral intravenous cannulation in pediatric patients. Pain Management Nursing, 16(1), p. 37.164
Study Questions
1. What type of statistical test was conducted by Canbulat et al. (2015) to examine group differences in the dependent variables of procedural pain and anxiety levels in this study? What two groups were analyzed for differences?
2. What did Canbulat et al. (2015) set the level of significance, or alpha (α), at for this study?
3. What are the t and p (probability) values for procedural self-reported pain measured with a visual analog scale (VAS)? What do these results mean?
4. What is the null hypothesis for observer-reported procedural anxiety for the two groups? Was this null hypothesis accepted or rejected in this study? Provide a rationale for your answer.
5. What is the t-test result for BMI? Is this result statistically significant? Provide a rationale for your answer. What does this result mean for the study?165
6. What causes an increased risk for Type I errors when t-tests are conducted in a study? How might researchers reduce the increased risk for a Type I error in a study?
7. Assuming that the t-tests presented in Table 2 and Table 3 are all the t-tests performed by Canbulat et al. (2015) to analyze the dependent variables’ data, calculate a Bonferroni procedure for this study.
8. Would the t-test for observer-reported procedural anxiety be significant based on the more stringent α calculated using the Bonferroni procedure in question 7? Provide a rationale for your answer.
9. The results in Table 1 indicate that the Buzzy intervention group and the control group were not significantly different for gender, age, body mass index (BMI), or preprocedural anxiety (as measured by self-report, parent report, or observer report). What do these results indicate about the equivalence of the intervention and control groups at the beginning of the study? Why are these results important?
10. Canbulat et al. (2015) conducted the χ2 test to analyze the difference in sex or gender between the Buzzy intervention group and the control group. Would an independent samples t-test be appropriate to analyze the gender data in this study (review algorithm in Exercise 12)? Provide a rationale for your answer.166
