17: Basic Variance

17.1 Understanding Variance

Understanding variance and comparing group differences are fundamental aspects of statistical analysis in applied research. Variance tests help to determine whether the means or distributions of different groups or samples differ significantly. These tests are essential for comparing experimental groups, treatment effects, or any situation where you want to assess if the data shows significant differences. This chapter will explore several basic variance tests, including the Independent Samples t-test and One-Way ANOVA. We will also delve into the assumptions for each test, which will help you understand when and how to apply them effectively in your research.

17.2 One-Sample T-Test

The One-Sample T-Test is used to determine whether the mean of a single sample is significantly different from a known or hypothesized population mean. This test is often used in cases where researchers want to compare the sample’s mean against a theoretical value or standard, such as testing whether the average weight of a population matches a known standard weight.

One-Sample T-Test Assumptions

The assumptions for this test include independence (the data points should be independent of each other), normality (the data should be approximately normally distributed), and scale of measurement (the data should be continuous, such as interval or ratio scales).

Below is an example of the results generated when the steps are correctly followed.

IMAGE [INSERT NAME OF DATASET]

Interpreting the Results

Before interpreting the results of the One-Sample T-Test, ensure that the following assumption is met:

Normality

• The data should be approximately normally distributed. You test this assumption using the Shapiro-Wilk test. A p-value > 0.05 suggests that there is no significant difference between the observed data and a normal distribution, and the data can be assumed to be normally distributed.

• Normal distribution can also be inspected visually using the Q-Q plot. Points fall along a straight line indicate that the data follows a normal distribution. The closer the points are to the line, the more normal the data is. : If the points curve away from the straight line, particularly at the ends, it suggests the data may not be normally distributed. A common sign of non-normality is when the points bend away from the line, indicating skewness or heavy tails.

If the assumption of normality is violated, consider using non-parametric alternatives, such as the Wilcoxon Signed-Rank Test.

The t-statistic indicates the number of standard errors that the sample mean is away from the hypothesized population mean. A higher absolute value of the t-statistic suggests a larger difference between the sample mean and the hypothesized population mean. The direction (positive or negative) of the t-statistic indicates whether the sample mean is higher or lower than the hypothesized mean.

The degrees of freedom (df) for the One-Sample T-Test is calculated as n – 1, where n is the sample size. The degrees of freedom are used to determine the critical value of t from the t-distribution, which helps assess the significance of the test statistic. The larger the sample size, the more reliable the result, as it gives more “freedom” for data points to vary.

The p-value tells you whether the difference between the sample mean and the hypothesized population mean is statistically significant. A p-value < 0.05: Indicates that the difference is statistically significant, suggesting that the sample mean is significantly different from the hypothesized population mean. A p-value ≥ 0.05 suggests that the difference is not statistically significant, indicating that the sample mean is not significantly different from the hypothesized population mean.

The mean difference in this context is the average difference between the sample mean and the hypothesized population mean. So, if the hypothesized population mean is 0 (default), the mean difference is simply the difference between the sample mean (which is the average of all observations) and 0.

Cohen’s d measures the size of the difference between the sample mean and the hypothesized population mean relative to the variability of the sample. A larger Cohen’s d indicates a larger effect size:

• 0.2: Small effect
• 0.5: Medium effect
• 0.8 or greater: Large effect

Cohen’s d helps understand the practical significance of the result beyond statistical significance.

17.3 One-Sample Wilcoxon Rank Test

The One-Sample Wilcoxon Signed-Rank Test is a non-parametric test used to determine if the median of a sample differs from a specific hypothesized value. It is typically used when you have data that does not follow a normal distribution, and you want to compare the sample median to a hypothesized value. This is similar to the one-sample t-test (which compares the sample mean to a hypothesized mean), but it does not require the assumption of normality.

When to Use the One-Sample Wilcoxon Signed-Rank Test:

• When you want to test whether the median of a sample is different from a specific hypothesized value.

• When your data is ordinal or non-normally distributed.

Below is an example of the results generated when the steps are correctly followed.

IMAGE [INSERT NAME OF DATASET]

Interpreting the Results

The W statistic is the test statistic for the One-Sample Wilcoxon Signed-Rank Test. It is derived from the sum of the ranks of the absolute differences between the paired values. A larger W statistic indicates a more significant difference between the sample’s median and the hypothesized median.

The p-value tells you whether the observed median difference is statistically significant. If the p-value is less than 0.05, the mean of the differences is significantly different from the hypothesized median (usually 0). If the p-value is greater than 0.05, the observed median is not significantly different from the hypothesized median.

The mean difference is the average of how much each data point differs from the hypothesized value. This is the mean of the ranked differences, not the sample mean. The mean difference does not represent the sample mean as in the one-sample t-test. The One-Sample Wilcoxon Signed-Rank Test focuses on the median difference to see if it is significantly different from 0.

The Rank Biserial Correlation (effect size) is used to measure the magnitude of the difference between the sample’s median and the hypothesized median. This value ranges from -1 to +1, with higher absolute values indicating a stronger effect. The Rank Biserial Correlation can be interpreted as follows:

• 0 – 0.1: Small effect
• 0.1 – 0.3: Moderate effect
• 0.3 – 1.0: Large effect

17.4 Independent Samples T-Test

The Independent Samples T-Test is used to compare the means of two independent groups to determine if there is a statistically significant difference between them. This test is beneficial when comparing two groups, such as treatment and control groups in a clinical study.

Independent Samples T-Test Assumptions

The assumptions for the Independent Samples T-Test include independence of observations (the groups must be independent with no overlap of subjects), normality (the data in each group should be normally distributed), and homogeneity of variance (the variance within the two groups should be approximately equal, which can be tested using Levene’s test). The dependent variable should also be continuous.

Below is an example of the results generated when the steps are correctly followed.

IMAGE [INSERT NAME OF DATASET]

Interpreting the Results

Before running the Independent Samples T-Test, there are essential assumptions to check, including normality and homogeneity of variances. The assumption of normality can be tested using the Shapiro-Wilk test. If the p-value from the Shapiro-Wilk test is greater than 0.05, it indicates that the data are approximately normally distributed, allowing the use of the t-test. If the p-value is less than 0.05, the assumption of normality is violated, and it is necessary to examine the skewness and kurtosis of the data. Additionally, a Q-Q plot can provide a visual check for normality.

Another assumption of the Independent Samples T-Test is the homogeneity of variances, which checks whether the variances of the two groups are equal. This assumption can be tested using Levene’s test. If the p-value from Levene’s test is greater than 0.05, the assumption is met, and the Student’s t statistic can be used. If the p-value is less than 0.05, indicating unequal variances, Welch’s t statistic should be used instead, as it adjusts for the violation of this assumption.

When interpreting the results of the One-Sample T-Test, the t-statistic quantifies the difference between the two group means relative to the variability in the data. A larger t-statistic indicates a greater difference between the groups. The p-value tells you if this difference is statistically significant. If the p-value is less than 0.05, you can reject the null hypothesis and conclude that there is a significant difference between the groups. A p-value greater than 0.05 suggests that any observed difference is likely due to random chance, and you would fail to reject the null hypothesis.

The mean difference reported by the test represents the actual difference between the means of the two groups. If the mean difference is positive, it indicates that the first group has a higher mean than the second group. A negative mean difference suggests the opposite.

In addition to the significance tests, it is important to report the effect size to assess the practical significance of the results. Cohen’s d is commonly used for this purpose. It measures the magnitude of the difference between the groups in standard deviation units. A Cohen’s d of 0.0 to 0.3 indicates a small effect, 0.4 to 0.6 indicates a moderate effect, and 0.7 or higher indicates a large effect. A negative effect size indicates that the first group’s mean is smaller than the second group’s.

17.5 Mann-Whitney U Test

The Mann-Whitney U Test is a non-parametric test used to determine whether there is a difference between two independent groups when the assumptions for the Independent Samples T-Test (such as normality) are not met. This test is ideal when the data are ordinal or the normality assumption is violated.

Mann-Whitney U Test Assumptions

The assumptions for the Mann-Whitney U Test include independence of groups (the groups being compared must be independent) and the use of ordinal or continuous data. The data does not need to be normally distributed.

Below is an example of the results generated when the steps are correctly followed.

IMAGE [INSERT NAME OF DATASET]

Interpreting the Results

One of the key statistics reported in the Mann-Whitney U Test is the U-statistic. The U-statistic represents the difference in the rank sums of the two groups. A higher U statistic suggests that the ranks of one group are higher or lower than the other group, indicating a greater difference between the groups. Essentially, it measures the relative positions of the observations in both groups.

The p-value tells you whether the result is statistically significant. If the p-value is less than 0.05, it indicates that the observed difference between the two groups is unlikely to have occurred by chance, and you can reject the null hypothesis. A p-value greater than 0.05 suggests that the difference is not statistically significant, and you would fail to reject the null hypothesis, meaning there is no evidence of a significant difference between the two groups.

The Rank Biserial Correlation Coefficient (RBC) is another important statistic that measures the effect size of the test. It indicates the strength and direction of the relationship between the two groups. The interpretation of the RBC is as follows:

• An RBC of 0.0 to 0.3 indicates a weak effect size.

• An RBC of 0.4 to 0.6 suggests a moderate effect size.

• An RBC of 0.7 or higher indicates a strong effect size.

• A negative RBC suggests that the first group has a smaller value than the second group, indicating an inverse relationship between the groups.

17.6 One-Way ANOVA

The One-Way ANOVA (Analysis of Variance) Test compares the means of three or more independent groups to determine if at least one of the group means differs significantly from the others. It tests the null hypothesis that all group means are equal.

One-Way ANOVA Assumptions

The assumptions for the One-Way ANOVA Test include independence of observations (the groups being compared must be independent), normality (the data within each group should be approximately normally distributed), and homogeneity of variance (the variance within each group should be approximately equal).

Below is an example of the results generated when the steps are correctly followed.

IMAGE [INSERT NAME OF DATASET]

Interpreting the Results

Before running the test, it is important to check two key assumptions: normality and homogeneity of variances. The assumption of normality can be assessed using the Shapiro-Wilk test. If the p-value from this test is greater than 0.05, it suggests that the data are approximately normally distributed, and the One-Way ANOVA can be performed. However, if the p-value is less than 0.05, indicating a violation of normality, you should check the skewness and kurtosis of the data. Additionally, visually inspecting a Q-Q plot can help assess whether the data deviate significantly from normality. The second assumption, homogeneity of variances, can be checked using Levene’s test. If the p-value is greater than 0.05, the assumption is met, and Fisher’s ANOVA statistic can be used. If the p-value is less than 0.05, it indicates unequal variances, and Welch’s ANOVA statistic should be used instead, as it adjusts for this violation.

Once the assumptions are verified, the test results provide the F-statistic and the p-value. The F-statistic represents the ratio of the variance between the groups (i.e., how different the group means are from each other) to the variance within the groups (i.e., how much individual values vary within each group). A higher F-statistic suggests a larger difference between the groups relative to the variation within each group.

The p-value tells you whether the difference between the groups is statistically significant. If the p-value is less than 0.05, you can reject the null hypothesis and conclude that there is a significant difference between the groups. If the p-value is greater than 0.05, the difference is not significant, and you fail to reject the null hypothesis.

If the One-Way ANOVA results are significant, post-hoc tests are necessary to explore which specific groups differ from each other. These pairwise comparisons are only applicable when the overall ANOVA result is significant. In the post-hoc analysis, the mean difference between two groups is reported. A positive mean difference indicates that the first group has a higher mean than the second group, while a negative mean difference suggests that the first group has a lower mean than the second group.

17.7 Kruskal-Wallis Test

The Kruskal-Wallis Test is a non-parametric alternative to the One-Way ANOVA. It is used to compare the medians of three or more independent groups when the assumptions of One-Way ANOVA (normality and homogeneity of variance) are not met. The Kruskal-Wallis test is ideal for ordinal data or data that do not follow a normal distribution.

Kruskal-Wallis Test Assumptions

The assumptions for the Kruskal-Wallis test include the independence of groups (the groups being compared must be independent) and the use of ordinal or continuous data.

Below is an example of the results generated when the steps are correctly followed.

IMAGE [INSERT NAME OF DATASET]

Interpreting the Results

The Chi-Square Statistic is the primary test statistic reported in the Kruskal-Wallis Test. It reflects the difference in ranks between the independent variable (IV) groups. A higher chi-square value suggests that there is a greater difference between the group distributions, indicating a more significant discrepancy between them. The p-value associated with the chi-square statistic tells us whether the observed differences are statistically significant. A p-value less than 0.05 indicates that the differences between the groups are unlikely to have occurred by chance, and the null hypothesis can be rejected. A p-value greater than 0.05 suggests that the differences are not statistically significant, and we would fail to reject the null hypothesis, implying that the groups do not differ significantly.

Along with the chi-square statistic and p-value, the epsilon-squared statistic (ε²) is used to measure the effect size of the results. It represents the proportion of variance in the dependent variable that can be attributed to the independent variable. The interpretation of ε² is as follows:

• 0 – 0.01: Small effect

• 0.01 – 0.08: Medium effect

• 0.08 – 0.26: Large effect

• 0.26 and higher: Very large effect This effect size provides an estimate of how substantial the observed differences are in a practical sense, beyond just statistical significance.

If the Kruskal-Wallis Test yields a significant result, post-hoc pairwise comparisons are necessary to determine which specific groups differ from each other. The Dwass-Steel-Critchlow-Fligner (DSCF) test is commonly used for these post-hoc comparisons following a Kruskal-Wallis test. The DSCF test compares each pair of groups and provides the W statistic and the associated p-value. The W statistic represents the difference in ranks between the two groups being compared, and the p-value indicates whether the difference is statistically significant. If the p-value is less than 0.05, the difference between the groups is considered significant.

17.8 Choosing the Right Test for Your Study

Selecting the appropriate test for comparing group means depends on the nature of your data and the research question. If you are comparing the means of two independent groups, use the Independent Samples T-Test to see if the data meet the assumptions of normality and homogeneity of variance. If the data is not normally distributed or assumptions are not met, use the Mann-Whitney U Test as a non-parametric alternative.

The one-way ANOVA is suitable for comparing the means of three or more independent groups if the assumptions of normality and homogeneity of variance are met. If these assumptions are not met, the Kruskal-Wallis Test can be a non-parametric alternative.