18: Advanced Variance

18.1 Introduction to Advanced Variance Tests

In applied research, variance analysis becomes more complex as researchers examine multiple independent variables, covariates, or multiple dependent variables simultaneously. Advanced variance tests, such as Factorial ANOVA, ANCOVA, and MANOVA, are powerful tools that allow researchers to investigate interactions between variables, control for extraneous factors, and assess the effects of multiple factors at once. These tests are helpful when researchers explore how more than one independent variable or covariate affects the dependent variable, or when multiple dependent variables are involved. This chapter focuses on the assumptions, interpretations, and applications of these tests and how they can be combined for more complex analyses. Additionally, we will discuss the options available for addressing violations of assumptions and how to handle those violations to maintain the integrity of your analysis.

18.2 Factorial ANOVA

Factorial ANOVA is an extension of the one-way ANOVA that allows researchers to examine the effects of two or more independent variables (factors) on a dependent variable. This test also enables the analysis of interaction effects, where the effect of one factor depends on the level of another factor. The assumptions for Factorial ANOVA include independence of observations (the groups being compared must be independent), normality (the dependent variable should be approximately normally distributed in each group), and homogeneity of variance (the variance of the dependent variable should be roughly equal across all levels of each factor). Additionally, the dependent variable should be continuous and measured on an interval or ratio scale.

If the assumptions for Factorial ANOVA are unmet, several options exist. For non-normality, data transformations, such as a log transformation, can be applied to bring the data closer to normality. If transformations do not resolve the issue, a non-parametric alternative like the Kruskal-Wallis test (for comparing more than two groups) can be used. If there is a violation of homogeneity of variance, the Welch’s ANOVA method can be used, as it is a more robust alternative that does not assume equal variances.

To run Factorial ANOVA in Jamovi, you simply open your dataset, go to Analyses > ANOVA > Factorial ANOVA, select the dependent variable and the independent variables (factors), and specify any interaction effects you want to test. Once the test is completed, Jamovi will generate the results, including the F-statistic, degrees of freedom, and p-values for both the main and interaction effects.

Below is an example of the results generated when the steps are correctly followed.

IMAGE [INSERT NAME OF DATASET]

Interpretation

18.3 ANCOVA (Analysis of Covariance)

ANCOVA is a blend of ANOVA and regression analysis that compares the means of different groups while controlling for the effects of one or more continuous covariates. ANCOVA helps adjust the dependent variable for the influence of covariates, offering a clearer understanding of the relationship between the independent and dependent variables. The assumptions for ANCOVA include independence of observations (the groups must be independent), normality (the residuals, or differences between observed and predicted values, should be normally distributed), and homogeneity of variance (the variance in the dependent variable should be equal across all groups). There must also be a linear relationship between the dependent variable and the covariates, and the relationship between the covariates and the dependent variable should be the same across all groups (this assumption is tested using an interaction term between the group variable and the covariate).

When these assumptions are violated, strategies exist to address them. If the residuals are not normally distributed, data transformations such as log transformations can improve normality. Suppose the assumption of homogeneity of regression slopes is violated (i.e., the interaction between the covariate and group is significant). In that case, researchers can run separate ANCOVAs for each group or use Multivariate ANCOVA (MANCOVA).

To run ANCOVA in Jamovi, you open the dataset and navigate to Analyses > ANOVA > ANCOVA. Select the dependent variable, the independent variable (group factor), and any covariates that need to be controlled. Jamovi will output the ANCOVA table with the adjusted means and p-values.

Below is an example of the results generated when the steps are correctly followed.

IMAGE [INSERT NAME OF DATASET]

Interpretation

18.4 MANOVA (Multivariate Analysis of Variance)

MANOVA is an extension of ANOVA used when two or more dependent variables are correlated. It assesses whether the means of multiple dependent variables are equal across the groups defined by the independent variable(s). MANOVA also tests for interaction effects between the independent and dependent variables. The assumptions for MANOVA include independence of observations (the groups should be independent), normality (the dependent variables should be multivariately normally distributed within each group), and homogeneity of variance-covariance (the variance-covariance matrices of the dependent variables should be equal across the groups, which can be tested with Box’s M test). Additionally, the dependent variables should be continuous and measured on an interval or ratio scale, and there should be linearity between the dependent variables.

If these assumptions are violated, several steps can be taken. If the dependent variables are not multivariately normal, data transformations such as log or square root transformations can be applied. If this does not resolve the issue, bootstrapping techniques can be used for more robust results, or non-parametric alternatives like the Friedman test for repeated measures can be considered. If the assumption of homogeneity of variance-covariance is violated, Pillai’s Trace is a more robust test statistic and should be used.

To run MANOVA in Jamovi, go to Analyses > ANOVA > MANOVA, select multiple dependent variables and the independent variable(s), and click OK to generate results, which include multivariate tests like Wilks’ Lambda and Pillai’s Trace.

Below is an example of the results generated when the steps are correctly followed.

IMAGE [INSERT NAME OF DATASET]

Interpretation

18.5 Combining Tests: Factorial MANCOVA

In complex research designs, combining different types of analysis can provide a deeper understanding of the data. For instance, Factorial MANCOVA combines Factorial ANOVA with ANCOVA, allowing researchers to assess the effects of multiple independent variables (factors) while controlling for covariates and multiple dependent variables simultaneously. This combined approach provides a more comprehensive analysis when dealing with complex data that includes interactions, covariates, and multiple outcomes.

18.6 Choosing the Right Test

The choice of the correct statistical test depends on the research question, data characteristics, and the complexity of the study design. If the goal is to examine the effects of two or more independent variables on a dependent variable and assess any potential interactions, Factorial ANOVA is the appropriate test. If covariates need to be controlled for while comparing group means, ANCOVA should be used. If there are multiple dependent variables, MANOVA is the test of choice. For more complex designs, Factorial MANCOVA combines both factorial analysis and ANCOVA to analyze multiple dependent variables while controlling for covariates and examining multiple factors.