17: Basic Variance
17.1 Understanding Variance
Understanding variance and comparing group differences are fundamental components of statistical analysis in quantitative research. These comparisons are at the heart of many research questions, from testing interventions to evaluating policy impacts. Variance tests help determine whether the means or distributions of two or more groups differ significantly, considering how much variability exists within each group. These tests are essential for evaluating experimental conditions, treatment effects, or any situation where identifying meaningful differences between populations is important.
17.2 One-Sample t-Test
The One-Sample t-Test is used to assess whether the mean of a single sample differs significantly from a known or hypothesized population mean. It is often applied when researchers want to compare a sample’s average to a theoretical value or standard.
Assumptions
The One-Sample t-Test relies on a few key assumptions to ensure valid results. First, the data should be continuous and measured at the interval or ratio level. Second, the observations must be independent, meaning that the value of one observation does not influence another. Third, the distribution of the sample data should be approximately normal, especially when the sample size is small.
How To: One-Sample t-Test
To run the One-Sample t-Test in Jamovi, go to the Analyses tab, select T-Tests, then One-Sample T-Test.
-
Move interval variables into the Dependent Variables box.
- Under Additional Statistics, check Mean difference, Effect size, Descriptives, and Descriptives plots.
- Under Assumption Checks, check Normality test and Q-Q plot.
TIP: The One-Sample t-Test assumes a hypothesized mean of “0.” You can change the hypothesized mean in the Test value box under Hypothesis.
Phrasing Results: One-Sample t-Test
Use this template to phrase significant results:
- A One-Sample t-Test showed that the sample’s mean of [Variable] significantly differed (t([df]) = [t-statistic], p < [approximate p-value], d = [Cohen’s D statistic]) from the hypothesized population mean of [hypothesized value].
Use this template to phrase non-significant results:
- A One-Sample t-Test showed that the sample’s mean of [Variable] did not significantly differ (t([df]) = [t-statistic], p = [p-value]) from the hypothesized population mean of [hypothesized value].
TIP: Replace the content inside the brackets with your variables and results, then remove the brackets.
17.3 One-Sample Wilcoxon Rank Test
The One-Sample Wilcoxon Signed-Rank Test is a non-parametric test used to determine whether the median of a sample differs from a specific hypothesized value. It is typically applied when the data are not normally distributed and when a researcher wants to compare the sample median to a theoretical value.
Assumptions
This test serves as a non-parametric alternative to the One-Sample t-Test, which compares a sample mean to a hypothesized mean but assumes normality. The Wilcoxon test does not require that assumption, making it more appropriate for skewed or ordinal data.
How To: One-Sample Wilcoxon Signed-Rank Test
To run the One-Sample Wilcoxon Signed-Rank Test in Jamovi, go to the Analyses tab, select T-Tests, then One-Sample T-Test.
-
Move interval variables into the Dependent Variables box.
- Under Tests, check Wilcoxon rank (uncheck Student’s).
- Under Additional Statistics, check Mean difference, Effect size, Descriptives, and Descriptives plots.
TIP: The One-Sample Wilcoxon Signed-Rank Test assumes a hypothesized median of “0.” You can change the hypothesized mean in the Test value box under Hypothesis.
Phrasing Results: One-Sample Wilcoxon Signed-Rank Test
Use this template to phrase significant results:
- A One-Sample Wilcoxon Signed-Rank Test showed that the sample’s median of [Variable] significantly differed (t([df]) = [t-statistic], p < [approximate p-value], r_pb = [Rank Biserial Correlation statistic]) from the hypothesized population median of [hypothesized value].
Use this template to phrase non-significant results:
- A One-Sample Wilcoxon Signed-Rank Test showed that the sample’s median of [Variable] did not significantly differ (t([df]) = [t-statistic], p = [p-value]) from the hypothesized population median of [hypothesized value].
TIP: Replace the content inside the brackets with your variables and results, then remove the brackets.
17.4 Independent Samples t-Test
The Independent Samples t-Test is used to compare the means of two independent groups to determine whether there is a statistically significant difference between them. It is commonly applied in research contexts where the goal is to evaluate whether group differences on a continuous outcome are likely due to chance or reflect a meaningful distinction.
Assumptions
The Independent Samples t-Test relies on several key assumptions to produce valid results. First, the dependent variable should be continuous and measured at the interval or ratio level. Second, the two groups being compared must be independent, meaning the observations in one group are not related to those in the other. Third, the data in each group should be approximately normally distributed, particularly when the sample sizes are small. Lastly, the test assumes homogeneity of variances, (e.g., the variability of scores in each group) should be roughly equal.
How To: Independent Samples t-Test
To run the Independent Samples t-Test in Jamovi, go to the Analyses tab, select T-Tests, then Independent Samples T-Test.
-
Move one interval variable into the Dependent Variables box.
-
Move one 2-group nominal variable into the Grouping Variable box.
-
Under Additional Statistics, check Mean difference, Effect size, Descriptives, Descriptives plots.
- Under Assumption Checks, check Homogeneity test, Normality test, and Q-Q plot.
- Under Tests, check Welch’s (needed if you fail the Homogeneity assumption).
Phrasing Results: Independent Samples t-Test
Use this template to phrase significant results:
- A [Student’s or Welch’s] Independent Samples t-Test was conducted to compare [DV] between [IV] groups.
- A significant difference was found (t([df]) = [t statistic], p < [approximate p-value]) with a [size] practical effect (d = [Cohen’s D statistic]).
Use this template to phrase non-significant results:
- A [Student’s or Welch’s] Independent Samples t-Test was conducted to compare [DV] between [IV] groups.
- No significant difference was found (t([df]) = [t statistic], p = [p-value]) between [IV group 1] and [IV group 2].
TIP: Replace the content inside the brackets with your variables and results, then remove the brackets.
17.5 Mann-Whitney U Test
The Mann–Whitney U Test is a non-parametric statistical test used to determine whether there is a significant difference in the distributions of two independent groups. It compares the ranks of values rather than the raw data, making it especially useful when working with ordinal data or continuous data that are not normally distributed. This test evaluates whether one group tends to have higher or lower values than the other, without assuming a specific distribution.
Assumptions
The Mann–Whitney U Test is a non-parametric alternative to the Independent Samples t-Test, used to assess whether two independent groups differ significantly when assumptions such as normality are not met.
How To: Mann-Whitney U Test
To run the Mann-Whitney U Test in Jamovi, go to the Analyses tab, select T-Tests, then Independent Samples T-Test.
-
Move one interval variable into the Dependent Variables box.
-
Move one 2-group nominal variable into the Grouping Variable box.
-
Under Additional Statistics, check Mean difference, Effect size, Descriptives, Descriptives plots.
- Under Tests, check Mann-Whitney U (uncheck Student’s).
Phrasing Results: Mann-Whitney U Test
Use this template to phrase significant results:
- A Mann-Whitney U Test was conducted to compare [DV] between [IV] groups.
- A significant difference was found (U = [Mann-Whitney U statistic], p < [approximate p-value]) with a [size] practical effect (r_pb = [Rank Biserial Correlation statistic]).
Use this template to phrase non-significant results:
- A Mann-Whitney U Test was conducted to compare [DV] between [IV] groups.
- No significant difference was found (U = [Mann-Whitney U statistic], p = [p-value]) between [IV group 1] and [IV group 2].
TIP: Replace the content inside the brackets with your variables and results, then remove the brackets.
17.6 One-Way ANOVA
The One-Way ANOVA (Analysis of Variance) is a statistical test used to compare the means of three or more independent groups to determine whether at least one group mean is significantly different from the others. Instead of conducting multiple Independent Samples t-Tests for each pair of groups, which increases the risk of making a Type I error due to repeated testing, One-Way ANOVA evaluates all group means simultaneously using a single test. It does this by analyzing the ratio of variability between groups to the variability within groups. A significant result indicates that not all group means are equal, prompting further investigation through post hoc tests to identify which specific groups differ.
Assumptions
The One-Way ANOVA relies on several key assumptions. The dependent variable should be continuous and measured at the interval or ratio level. The groups being compared must be independent, meaning that each observation belongs to only one group. The data within each group should be approximately normally distributed, especially if sample sizes are small. Additionally, the assumption of homogeneity of variances must be met. This means that the variance of the dependent variable should be roughly equal across all groups.
How To: One-Way ANOVA
To run the One-Way ANOVA Test in Jamovi, go to the Analyses tab, select ANOVA, then One-Way ANOVA.
-
Move one interval variable into the Dependent Variables box.
-
Move one 3 or more group nominal variable into the Grouping Variable box.
-
Under Additional Statistics, check Descriptive table and Descriptives plots.
- Under Assumption Checks, check Homogeneity test, Normality test, and Q-Q plot.
- Under Variances, check equal (Fisher’s) (needed if you pass the Homogeneity assumption).
- Optional: If the result is significant, choose the post-hoc test (under the Post-Hoc Tests drop-down) that matches the variance test you used in Step 5.
NOTE: Jamovi’s One-Way ANOVA Test does not produce an effect size statistic.
Phrasing Results: One-Way ANOVA
Use this template to phrase significant results:
- A [Fisher’s or Welch’s] One-Way ANOVA was conducted to compare [DV] across the groups of [IV].
- A significant difference was found among the groups of [IV] (F([df1, df2]) = [F statistic], p < [approximate p-value]).
Use this template to phrase the post-hoc results:
- A [Tukey or Games-Howell] post-hoc test was conducted to determine the nature of the mean differences between groups of [IV].
- This analysis revealed that pairwise comparisons between [IV group 1] and [IV group 2] (ΔM = [mean difference], p < [approximate p-value]) were significantly different.
NOTE: Only include the post-hoc results if the One-Way ANOVA Test result is significant.
Use this template to phrase non-significant results:
- A [Fisher’s or Welch’s] One-Way ANOVA was conducted to compare [DV] across the groups of [IV].
- No significant difference was found (F([df1, df2]) = [F statistic], p = [p-value]).
TIP: Replace the content inside the brackets with your variables and results, then remove the brackets.
17.7 Kruskal-Wallis Test
The Kruskal–Wallis Test is a non-parametric statistical test used to determine whether there are significant differences between the distributions of three or more independent groups. It works by ranking all values across groups and comparing the average ranks to assess whether one or more groups tend to have higher or lower values. When the test indicates a statistically significant difference, post hoc comparisons are needed to identify which specific groups differ from one another.
Assumptions
The Kruskal–Wallis Test is a non-parametric alternative to the One-Way ANOVA. It is particularly well-suited for continuous data that are not normally distributed, such as those that are skewed or contain outliers.
How To: Kruskal-Wallis Test
To run the Kruskal-Wallis Test in Jamovi, go to the Analyses tab, select ANOVA, then One-Way ANOVA-Kruskal-Wallis under Non-Parametric.
-
Move one interval or ordinal variable into the Dependent Variables box.
-
Move one 3 or more group nominal variable into the Grouping Variable box.
- Check Effect size.
- Check DSCF pairwise comparisons.
Phrasing Results: Kruskal-Wallis Test
Use this template to phrase significant results:
- A Kruskal-Wallis Test was conducted to compare [DV] across the groups of [IV].
- A significant result was found (χ²([df]) = [χ² statistic], p < [approximate p-value]) with a [size] practical effect (ε² = [epsilon-squared statistic]).
Use this template to phrase the post-hoc results:
- Follow-up pairwise comparisons were conducted to determine the nature of the differences between groups of [IV].
- This analysis revealed that pairwise comparisons between [IV group 1] and [IV group 2] were significantly different (W = [Wilcoxon statistic], p < [approximate p-value])
NOTE: Only include the post-hoc results if the Kruskal-Wallis Test result is significant.
Use this template to phrase non-significant results:
- A Kruskal-Wallis Test was conducted to compare [DV] across the groups of [IV].
- No significant difference was found (χ²([df]) = [χ² statistic], p = [p-value]).
TIP: Replace the content inside the brackets with your variables and results, then remove the brackets.
Chapter 17 Summary and Key Takeaways
Several foundational statistical tests are used to compare group differences in quantitative research. The One-Sample t-Test compares the mean of a single sample to a known or hypothesized population mean. The Independent Samples t-Test is used to compare the means of two independent groups. When data do not meet the assumptions required for parametric tests, the Mann–Whitney U Test offers a non-parametric option for comparing two groups. The One-Way ANOVA is used to compare the means of three or more independent groups, while the Kruskal–Wallis Test serves as a non-parametric alternative when assumptions of normality and equal variance are not met. Understanding the assumptions and appropriate use of each test enables researchers to select the most suitable method for analyzing group differences and drawing meaningful conclusions. All of these tests can be easily conducted using Jamovi, which provides intuitive menus for selecting the test, checking assumptions, and interpreting results.
- T-tests and ANOVA compare group means but require assumptions such as normality and homogeneity of variance.
- Non-parametric tests like the Mann–Whitney U Test and Kruskal–Wallis Test are used when parametric assumptions are violated or when working with ordinal data.
- Selecting the correct test based on the structure of your data and research question is essential for valid and interpretable results.
