16: Association

16.1 Association vs. Relationship

Understanding associations between variables is crucial for making meaningful conclusions in applied statistics. It is essential to distinguish between association and relationship. While association refers to the statistical connection between two variables, a relationship implies a deeper, more meaningful connection between variables. For example, a correlation tells us if, when one variable changes, the other also tends to change, indicating a statistical connection, but does not necessarily explain a causal relationship.

This chapter will discuss several statistical tests used to examine the association between variables. These tests include correlation, the binomial test, the goodness-of-fit test, and the test of independence. We will also explore the assumptions for each test, helping you understand when and how to apply them correctly to ensure valid conclusions.

16.2 Correlation

Correlation is a statistical technique used to measure and describe the strength and direction of the relationship between two continuous variables. The most common types of correlation are Pearson’s correlation and Spearman’s correlation, which assess the degree of association between variables, but under different conditions.

16.3 Pearson’s Correlation

Pearson’s correlation coefficient (r) is the most widely used measure of correlation. It assesses the linear relationship between two continuous variables, making it appropriate for normally distributed data with a linear relationship.

Assumptions

linearity, normality of both variables, and homoscedasticity (consistent spread of values across the range of the data). A Pearson correlation coefficient of +1 indicates a perfect positive linear relationship, -1 indicates a perfect negative relationship, and 0 indicates no linear relationship. Pearson’s correlation is ideal when both variables are continuous and meet the assumptions of normality, and when there are no significant outliers that could distort the relationship.

Below is an example of the results generated when the steps are correctly followed.

IMAGE [INSERT NAME OF DATASET]

Interpretation

The degrees of freedom (df) for a correlation test is determined by the sample size minus the number of variables being correlated, represented as n – 2. This value is important for determining the statistical significance of the test. The larger the sample size, the more reliable the test will be, as it gives more “freedom” for data points to vary.

The p-value tells us whether the relationship observed between the two variables is statistically significant or if it might have occurred by chance. The standard cutoff for significance is 0.05. If the p-value is less than 0.05, it indicates that the relationship is statistically significant and unlikely to have occurred by chance. If the p-value is greater than 0.05, the relationship between the variables may be due to random variability, and you would fail to reject the null hypothesis, suggesting no significant relationship.

The Pearson’s correlation coefficient (r) indicates the direction of the relationship (positive or negative) and the strength of the relationship. The interpretation of the direction is as follows:

• A positive r indicates a direct or positive relationship, meaning that as one variable increases, the other variable tends to increase as well.

• A negative r indicates an inverse or negative relationship, meaning that as one variable increases, the other tends to decrease.

The interpretation of the strength is as follows:

• A strong relationship is indicated by a coefficient of +/- 0.7 to 1. This means there is a clear and consistent pattern in how the variables move in relation to each other.

• A moderate relationship is indicated by a coefficient of +/- 0.3 to 0.7. The variables show a discernible pattern, but there may be more variability or noise.

• A weak relationship is indicated by a coefficient of +/- 0.3 or less, which suggests that the relationship is weak and may not be practically useful.

A scatterplot is often used alongside Pearson’s correlation to visually represent the relationship between the two variables. In a scatterplot, each data point represents a pair of values, and the trend line (or regression line) indicates the overall linear relationship. The direction and steepness of the trend line provide a visual indication of the correlation coefficient: a steeper line suggests a stronger relationship, while a flatter line suggests a weaker one. The shaded area around the trend line represents the confidence interval, showing the range of values within which we expect the true correlation to lie, given the sample data.

As the strength of the relationship increases, the data points will cluster more tightly around the trend line, making the pattern easier to discern. For weaker relationships, the data points will be more dispersed, making it harder to detect a clear trend.

16.4 Spearman’s Rank Correlation

Spearman’s rank correlation (rs) is a non-parametric test that measures the strength and direction of a relationship between two variables. Unlike Pearson’s correlation, Spearman’s correlation does not assume that the data are linearly related or normally distributed, making it suitable for ordinal data or continuous data that does not meet normality assumptions.

Assumptions

Spearman’s correlation is a good choice when dealing with skewed data or outliers that would affect Pearson’s correlation.

Below is an example of the results generated when the steps are correctly followed.

IMAGE [INSERT NAME OF DATASET]

Interpretation

The degrees of freedom (df) for Spearman’s Rank correlation are determined by the sample size minus the number of tied ranks, represented as n – 2. This value is important for determining the statistical significance of the test. A larger sample size typically results in more reliable test outcomes, as it gives more “freedom” for data points to vary and strengthens the ability to detect significant relationships.

The p-value in Spearman’s Rank correlation indicates whether the observed relationship between the two variables is statistically significant or if it might have occurred by chance. The standard cutoff for significance is 0.05. If the p-value is less than 0.05, it suggests that the relationship between the variables is statistically significant and unlikely to be due to random variation. Conversely, if the p-value is greater than 0.05, the relationship might be due to chance, and the null hypothesis is not rejected, implying no significant relationship.

Spearman’s rank correlation coefficient (rs) indicates both the direction and the strength of the relationship. The interpretation of the direction is as follows:

• A positive rs indicates a direct or positive relationship, meaning that as one variable increases, the other variable tends to increase as well.

• A negative rs indicates an inverse or negative relationship, meaning that as one variable increases, the other tends to decrease.

The interpretation of the strength of the relationship in Spearman’s rank correlation is as follows:

• A strong relationship is indicated by an rs between +/- 0.7 and 1, suggesting a clear and consistent pattern in how the variables change in relation to each other.

• A moderate relationship is indicated by an rs between +/- 0.3 and 0.7, meaning the variables show a noticeable pattern, but there may be more variability or noise.

• A weak relationship is indicated by an rs of +/- 0.3 or less, suggesting that the relationship is weak and may not be practically useful.

Since Spearman’s rank correlation is used for ordinal or non-normally distributed data, it does not require a linear relationship like Pearson’s correlation. Instead, it focuses on the rank order of the data, making it robust to outliers and non-linear relationships.

A scatterplot can also be used to visually inspect the relationship between two variables in Spearman’s rank correlation, though the scatterplot for Spearman’s rank correlation typically plots the ranks of the data rather than the raw data values. Each data point represents a pair of ranks, and the trend line (or line of best fit) shows the overall pattern in the relationship. As with Pearson’s correlation, a steeper line suggests a stronger relationship, while a flatter line suggests a weaker relationship. For Spearman’s rank correlation, the pattern will often show whether the ranks increase or decrease together, visually confirming the direction of the relationship. As the strength of the relationship increases, the data points in the scatterplot will cluster more tightly around the trend line, making the pattern easier to discern. For weaker relationships, the data points will be more dispersed, making it harder to detect a clear trend.

16.5 Understanding the Chi-Square Statistic

The chi-square statistic (χ²) is a fundamental statistic used in inferential statistics, particularly for categorical data. It compares the observed frequencies in your data to the expected frequencies based on a specific hypothesis. The chi-square statistic is central to tests like the binomial, goodness-of-fit, and test of independence, all of which help determine whether data distribution deviates significantly from what is expected.

A small chi-square statistic suggests that the observed frequencies are close to the expected frequencies, indicating little difference. A large chi-square statistic suggests that the observed and expected frequencies differ significantly, which might lead to rejecting the null hypothesis. Understanding the chi-square statistic is essential for interpreting categorical data tests.

16.6 Binomial Test

The binomial test assesses whether the observed frequency of outcomes in a categorical variable deviates significantly from the expected frequency, often applied when there are only two possible outcomes (success/failure). For example, a researcher might use the binomial test to determine whether the number of heads in a series of coin flips significantly differs from what would be expected by chance.

Assumptions

• Two outcomes: The variable of interest must have exactly two categories, such as success or failure.
• Fixed number of trials: The number of observations must be determined in advance.
• Independence: The outcomes of the trials must be independent of each other.

Below is an example of the results generated when the steps are correctly followed.

IMAGE [INSERT NAME OF DATASET]

Interpretation

The count represents the number of successes (or occurrences of the outcome of interest) in the sample. This number is used as the foundation for calculating the proportion of successes in the sample and comparing it to the expected value.

The proportion is the percentage of successes in the sample. It is calculated by dividing the number of successes (Count) by the total number of observations in the sample. The proportion is then compared to the expected proportion (e.g., 0.5 for a fair coin toss) to assess whether the observed value significantly deviates from the expected.

The p-value indicates whether the observed proportion differs significantly from the expected proportion. The null hypothesis in a Binomial Test assumes that there is no difference between the observed and expected proportions. A p-value below 0.05 typically indicates that the difference between the observed proportion and the expected value is statistically significant, meaning the observed result is unlikely to have occurred by chance. A p-value above 0.05 suggests that the observed difference is not significant and may be due to random variation.

The confidence interval (CI) provides a range of values that is likely to contain the true proportion in the population, based on the sample data. The CI gives the lowest and highest plausible values for the population proportion, allowing researchers to assess the precision of the estimate. A wider interval suggests more uncertainty, while a narrower interval indicates a more precise estimate.

16.7 Goodness-of-Fit Test

The goodness-of-fit test, often using the chi-square statistic, determines whether the observed distribution of categorical data matches an expected distribution. This test is commonly used to assess whether a categorical variable follows a specific distribution or whether the frequencies across categories are equal.

Assumptions

• Independence of observations: Each observation must be independent of the others.
• Adequate sample size: Each expected frequency should be at least five.
• Categorical data: The data should be divided into distinct groups or categories.

Below is an example of the results generated when the steps are correctly followed.

IMAGE [INSERT NAME OF DATASET]

Interpretation

The Chi-Square Statistic (χ²) measures the differences between the observed counts and the expected counts. Larger values of the chi-square statistic indicate larger discrepancies between observed and expected frequencies, suggesting a significant deviation from the expected distribution.

The p-value in the goodness-of-fit test indicates whether the observed distribution differs significantly from the expected one. If the p-value is less than 0.05, it suggests that the difference between the observed and expected frequencies is unlikely to be due to chance, meaning the data does not fit the expected distribution. Conversely, a p-value greater than 0.05 suggests that any differences between observed and expected counts are likely due to random chance, and there is no significant deviation.

The proportions table provides key information for interpreting the goodness-of-fit test. The observed count represents the actual number of cases in each category of the variable, while the observed proportion is the percentage of the sample that falls into each category. These proportions are calculated by dividing the observed count by the total sample size.

The expected count refers to the number of observations that would be expected in each category if the sample perfectly matched the population proportions. The expected proportion represents the proportion of cases we would expect in each category based on the population, and it is typically assumed to be equal across all categories by default, though this can be adjusted if a different distribution is hypothesized.

16.8 Test of Independence

The test of independence, another type of chi-square test, is used to determine whether two categorical variables are independent or if there is an association between them. For example, a researcher might use this test to see if gender is related to product preference.

Assumptions for the test of independence include:

• Independence of observations: Each observation must be independent of the others.
• Adequate sample size: Each expected frequency should be at least five.
• Categorical data: Both variables being tested must be categorical.

Below is an example of the results generated when the steps are correctly followed.

IMAGE [INSERT NAME OF DATASET]

Interpretation

The Chi-Square Statistic (χ²) is used to assess whether there is a significant association between two categorical variables. It quantifies how much the observed counts in a contingency table differ from the expected counts under the assumption that the variables are independent. A larger chi-square statistic indicates a greater difference between observed and expected counts, suggesting a stronger potential association between the variables.

The p-value determines whether the association between the variables is statistically significant. If the p-value is less than 0.05, you can conclude that the two variables are associated (dependent). This suggests that the variables do not behave independently of each other. In this case, you can refer to the Cramer’s V statistic to assess the strength of the association. A p-value greater than 0.05 indicates that there is no significant association between the two variables, meaning they are independent. In such cases, you should ignore the Cramer’s V table, as it is irrelevant when there is no association between the variables.

The Cramer’s V statistic helps determine the strength of the association between two categorical variables. It is a measure that ranges from 0 to 1, where higher values indicate stronger associations. Unlike correlation coefficients, Cramer’s V does not indicate the direction of the relationship, only its strength. The general interpretation of Cramer’s V values is as follows:

• 0 – .29: Weak association
• .30 – .49: Moderate association
• .50 – 1: Strong association

Cramer’s V is used in conjunction with the chi-square test to provide a clearer picture of the relationship’s strength after determining its statistical significance.

The contingency table provides a detailed look at the frequency distribution of one variable within the categories of the other variable. It is divided into observed frequencies and expected frequencies: Observed frequencies represent the actual number of cases in each category combination. Expected frequencies represent what we would expect if the two variables were independent.

Interpreting the rows and columns in the contingency table is essential for understanding the relationship between the variables. Looking across a row tells you how the column variable is distributed across the different categories of the row variable. It answers the question, “Given [row variable], what happened on [column variable]?” Looking down a column tells you how the row variable is distributed across the different categories of the column variable. It answers, “Given [column variable], what was the [row variable]?” These insights help contextualize the results in terms of both variables, providing a clearer interpretation of how the two variables relate to one another.

The clustered bar chart is a graphical representation of the contingency table. It helps visualize the distribution of the row variable within the column variable. In this chart, the X-axis displays the groups of the row variable, while the Y-axis shows the number of cases in each group. Each stacked bar breaks down the number of cases across the groups of the column variable. The height of each colored section represents the number of cases that fall into each category of the column variable.