Statistical Analysis Guide

Table of Contents

1. T-tests
2. ANOVA (Analysis of Variance)
3. Chi-Square Test
4. Pearson Correlation Coefficient
5. Spearman's Rank Correlation
6. Mann-Whitney U Test
7. Wilcoxon Signed-Rank Test
8. Linear Regression Analysis
9. Logistic Regression
10. Definition List (Glossary)

1T-tests PARAMETRIC

Simple Explanation: Compares the averages of two groups to see if they are significantly different from each other.

Types

Type	What It Compares
Independent Samples T-test	Means of two independent groups
Paired Samples T-test	Means from the same group at different times

Independent Samples T-test

Key Assumptions

• Independent groups: The two groups being compared should be independent of each other — participants in one group have no relation to the participants in the other group.
• Normality: The data in each group should be approximately normally distributed. This assumption is less of a concern with larger sample sizes due to the Central Limit Theorem.
• Homogeneity of variances: The variances in the two groups should be approximately equal. This can be tested using Levene's test for equality of variances.
• Scale of measurement: The dependent variable should be measured at least at the interval scale (i.e., the difference between values is meaningful).
• Random sampling: Ideally, the data should be collected from a random sample of the population.

Paired Samples T-test

Key Assumptions

• Related or paired groups: The same subjects are used in both groups (e.g., measurements before and after a treatment on the same subjects), or matched subjects are used in both groups.
• Normality of difference scores: The differences between the pairs should be approximately normally distributed. This is more crucial in smaller samples.
• Scale of measurement: The dependent variable should be measured at least at the interval scale.
• No outliers: Outliers can significantly affect the results, so it's important to check for outliers in the difference scores.
• Random selection and assignment: Ideally, participants should be randomly selected and assigned to pairs or conditions.

2ANOVA (Analysis of Variance) PARAMETRIC

Simple Explanation: ANOVA is a statistical method used to compare the means of three or more groups to determine if there's at least one significant difference among them. It assesses whether the average differences between groups are more than what would be expected by chance.

One-way ANOVA (Single-factor ANOVA)

Explanation: Used when there's only one independent variable (factor) and it compares the means of different levels (groups) of this variable.

Key Assumptions

1. Independence of observations: The data in each group must be collected independently — the scores of an individual in one group do not influence the scores of an individual in another group. This is usually ensured by random sampling.
2. Normality: The data in each group should be approximately normally distributed. This assumption tends to be less critical with larger sample sizes due to the Central Limit Theorem, but it's important for smaller samples.
3. Homogeneity of variances: The variance among the groups should be approximately equal, testable using Levene's test. If variances are not equal, the robustness of ANOVA is affected, and approaches like Welch's ANOVA might be necessary.
4. Scale of measurement: The dependent variable should be measured on an interval or ratio scale (meaningful numerical values and intervals).
5. Random sampling: Ideally, the data should be collected from a random sample of the population, which helps generalize the findings.

Two-way ANOVA

Explanation: Examines the effect of two independent variables on a dependent variable. It can also explore the interaction effect between these two variables.

Key Assumptions

1. Independence of observations: The data in one group must not influence the data in another, typically ensured through random sampling or random assignment.
2. Normality: The distribution of the residuals (not the variables themselves) should be approximately normal. This is especially important for small sample sizes; for larger samples, ANOVA is robust to deviations from normality.
3. Homogeneity of variances (homoscedasticity): The variances across groups (for each combination of categories of the two independent variables) should be approximately equal. Testable using Levene's test or Hartley's test.
4. Additivity: Two-way ANOVA assumes that the effects of one independent variable are the same at all levels of the other variable. Significant interaction effects may violate this assumption.
5. Random sampling/assignment: The data should come from a randomly selected sample, or through random assignment if it's an experiment.
6. Scale of measurement: The dependent variable should be measured at an interval or ratio scale.
7. Continuous dependent variable: The dependent variable should be continuous (measured on an interval or ratio scale).

Repeated Measures ANOVA

Explanation: Used when the same subjects are measured multiple times under different conditions or over time. It's like a paired t-test, but for more than two measurements.

Key Assumptions

1. Sphericity: The variances of the differences between all possible pairs of within-subject conditions should be approximately equal. If violated, significance testing can become inaccurate. Mauchly's test is commonly used to check this, and corrections like Greenhouse-Geisser or Huynh-Feldt are often applied.
2. Normality of residuals: The distribution of the residuals (differences between observed values and values predicted by the model) for each group should be approximately normal — particularly important for small samples.
3. No outliers: Outliers can have a disproportionately large effect on the results, so they should be checked for and addressed.
4. Independence of observations: While repeated measures on the same subject are not independent, the different subjects in the study should be independent of one another.
5. Covariate balance: If covariates are used, they should be balanced across the repeated measures. Imbalance can lead to biased estimates and incorrect inferences.
6. Random selection: Ideally, participants should be randomly selected from the population, which helps generalize the results.

Multivariate ANOVA (MANOVA)

Explanation: Similar to ANOVA, but it can handle multiple dependent variables at the same time.

Key Assumptions

1. Independence of observations: The results or responses of one participant do not influence the results of another.
2. Multivariate normality: The data should be multivariate normally distributed within each group or level of the independent variable(s).
3. Homogeneity of covariance matrices: The covariance matrices of the dependent variables should be equal across groups. This can be tested with Box's M test.
4. No multicollinearity: The dependent variables should not be too highly correlated with each other, as high correlation can inflate variance and affect the results.
5. Linearity: Linear relationships are assumed among all pairs of dependent variables, all pairs of independent variables, and between each pair of dependent and independent variables.
6. Absence of outliers: Outliers can significantly impact MANOVA results and should be checked for in the multivariate space of the dependent variables.
7. Adequate sample size: MANOVA requires a larger sample size than ANOVA. A common rule of thumb is to have more cases (observations) than the number of dependent variable combinations.

Mixed-design ANOVA

Explanation: Combines features of both repeated measures and two-way ANOVAs, used when there are both between-subjects and within-subjects variables.

Key Assumptions

1. Independence of observations: For the between-subjects factor, participants in one group should not influence participants in another group.
2. Sphericity (for within-subjects factors): The variances of the differences between all combinations of related groups should be equal. If violated, corrections like Greenhouse-Geisser or Huynh-Feldt can be applied.
3. Normality: The dependent variable should be approximately normally distributed for each combination of groups of the between-subjects factor and levels of the within-subjects factor — more critical in smaller samples.
4. Homogeneity of variances (for between-subjects factors): The variance among the groups should be similar, assessed using Levene's test of equality of variances.
5. Additivity: The test assumes that the effect of one factor is consistent at all levels of the other factor, even though it is used to examine interactions.
6. Random sampling/assignment: Data should come from a randomly selected sample, or subjects should be randomly assigned to groups in experimental designs.
7. Homogeneity of covariance matrices (if multiple dependent variables): In cases where multiple dependent variables are analyzed (as in MANOVA), the covariance matrices should be similar across different groups.
8. Absence of outliers: Outliers should be checked for and addressed, as they can significantly impact the results.

3Chi-Square Test

Simple Explanation: Checks if there is a significant relationship between two categorical variables.

Key Assumptions

• Categorical data: Both variables should be categorical (nominal or ordinal). The Chi-Square test is not suitable for continuous or highly granular data.
• Independence of observations: Each case or observation should be independent of the others — the outcome of one case should not influence or be influenced by the outcome of another.
• Sample size: A common guideline is that the total sample size should be sufficiently large, typically N > 20. However, more important than the total sample size is the expected frequency in each cell of the contingency table.
• Expected frequencies:
  • Goodness of Fit Test: Each expected frequency should be 5 or more. This test compares the observed distribution of data to an expected probability distribution.
  • Test of Independence: Each cell of the contingency table should have an expected frequency of at least 5. If this condition is not met, Fisher's Exact Test might be a more appropriate choice, especially for 2×2 tables.
• No overlapping categories: The categories for each variable should be mutually exclusive and exhaustive — each observation fits into one and only one category for each variable.
• Random sampling: The data should be collected through a process that ensures random sampling, which is important for the generalizability of the results.

4Pearson Correlation Coefficient PARAMETRIC

Simple Explanation: Measures how much two continuous variables move together (positively or negatively).

Key Assumptions

• Linearity: The relationship between the two variables should be linear — a change in one variable should result in a proportional change in the other. Non-linear relationships are not adequately captured by Pearson's r.
• Continuous variables: Both variables should be continuous, able to take any value within a range and not restricted to categories.
• Bivariate normal distribution: The pair of variables should be approximately normally distributed — more specifically, their joint distribution (considering them together) should be normal. This is especially important for small sample sizes.
• Homoscedasticity: The variance within each variable should be constant at all levels of the other variable. In other words, the data should form a roughly circular cloud in a scatterplot. Heteroscedasticity, where the variance changes, can distort the correlation.
• Independence of observations: Each pair of observations should be independent of all other pairs.
• No outliers: Outliers can have a significant impact on the Pearson correlation coefficient, potentially leading to misleading results. Identify and address outliers before conducting the analysis.

5Spearman's Rank Correlation NON-PARAMETRIC

Simple Explanation: Similar to Pearson but for data that's ranked (like 1st, 2nd, 3rd) or not normally distributed.

Key Assumptions

• Ordinal or continuous data: Spearman's correlation can be used with either ordinal data (data that can be ranked or ordered) or continuous data. It is particularly useful when the data do not meet the assumptions necessary for Pearson's correlation.
• Monotonic relationship: As one variable increases, the other variable either consistently increases or decreases. Unlike Pearson's r, the relationship does not need to be linear, but it should be monotonically increasing or decreasing.
• Independence of observations: Each pair of observations should be independent of all other pairs — the presence or value of one pair of data points should not influence or determine another pair.
• No tied ranks (or awareness of how to handle them): Ideally, there should not be tied ranks in the data (the same value appearing more than once, leading to the same rank). If ties are present, Spearman's rank correlation coefficient needs to be adjusted to account for this.
• Appropriate sample size: While Spearman's rank correlation can be used with small sample sizes, the reliability of the correlation estimate increases with larger samples.

6Mann-Whitney U Test NON-PARAMETRIC

Simple Explanation: Compares the differences between two groups when the data is not normally distributed.

Key Assumptions

• Independence of samples: The two groups being compared must be independent — the observations in one group are not related to the observations in the other group.
• Ordinal or continuous data: The dependent variable should be ordinal (can be ranked) or continuous.
• Similar shape and spread: While the test does not assume normal distribution, it does assume that the shapes of the distributions are similar — the general pattern and spread of the data should be similar across the groups, even if they have different medians.
• Random sampling: The data should be drawn from a random sample, meaning each individual or observation in the population has an equal chance of being included.
• No tied ranks (or handling them appropriately): The test includes adjustments for tied ranks (where two or more values are the same), but a large number of ties can affect the validity of the test.

7Wilcoxon Signed-Rank Test NON-PARAMETRIC

Simple Explanation: Compares two related groups to see if there are significant differences between them. The Wilcoxon Signed-Rank Test is a non-parametric statistical test used to compare two related samples, matched samples, or repeated measurements on a single sample. It's an alternative to the paired t-test when the data do not meet the assumptions of the t-test.

Key Assumptions

• Paired or matched data: The data come from the same participants measured at two different times (e.g., before and after a treatment), or from matched subjects in two different conditions.
• Ordinal or continuous data: The dependent variable should be continuous (measurable and able to have decimal values) or ordinal (data that can be ranked).
• Symmetry of the distribution: For the differences between the pairs, the distribution should be symmetric. While the test does not require the original data to be normally distributed, the differences between paired observations should be symmetrically distributed around the median.
• No tied differences for the exact test: The test works best when there are no ties in the differences (i.e., no two differences are the same). However, the Wilcoxon test has been adapted to handle ties.
• Random selection and assignment: Ideally, participants should be randomly selected from the population and, in the case of an experiment, randomly assigned to conditions. This is important for the generalizability of the results.

8Linear Regression Analysis PARAMETRIC

Simple Explanation: Predicts the value of a variable based on the value of another variable.

Key Assumptions

• Linearity: The relationship between the independent and dependent variables should be linear — a change in the independent variable(s) should result in a proportional change in the expected value of the dependent variable.
• Independence of residuals: The residuals (differences between observed values and values predicted by the model) should be independent and display no pattern when plotted against time or the independent variables. This is often assessed by examining a plot of residuals versus predicted values or time.
• Homoscedasticity: The variance of residuals (errors) should be constant for all levels of the independent variable(s) — the spread of the residuals should be approximately the same across all levels.
• Normal distribution of residuals: The residuals should be approximately normally distributed. Slight deviations are typically not a major issue (especially with larger samples), but severe skewness or kurtosis can be problematic.
• No multicollinearity (in multiple regression): The independent variables should not be too highly correlated with each other. High multicollinearity can make it difficult to distinguish the individual effects of the independent variables.
• No autocorrelation (in time series data): If the data are time series, the residuals should not be correlated with each other. The Durbin-Watson test is commonly used to test for this.
• Adequate sample size: A larger sample size provides more reliable results. A common rule of thumb is at least 10 observations per independent variable in the analysis.

9Logistic Regression PARAMETRIC

Simple Explanation: Used when the outcome is binary (like yes/no) and predicts the probability of the outcome.

Key Assumptions

• Binary outcome variable: The dependent variable should be binary (e.g., success/failure, yes/no, 1/0). Logistic regression can also be extended to categorical dependent variables (Multinomial Logistic Regression) or ordinal dependent variables (Ordinal Logistic Regression).
• Independence of observations: Each observation should be independent of the others — the outcome for one observation should not influence or be influenced by the outcome of another.
• No multicollinearity: The independent variables should not be too highly correlated with each other, as high multicollinearity can lead to difficulties in determining the individual effect of each variable.
• Linearity in the logit: The logit transformation of the outcome variable (the natural log of the odds that the outcome equals a particular value) is assumed to have a linear relationship with the independent variables.
• No outliers or highly influential points: Outliers and influential data points can have a significant effect on the model, potentially leading to overfitting or biased estimates.
• Large sample size: Logistic regression requires a sufficiently large sample size. A common rule of thumb is at least 10 cases with the least frequent outcome for each independent variable in the model.
• Goodness of fit: The model should fit the data well. Goodness of fit can be assessed using tests like the Hosmer-Lemeshow test, or by examining classification tables or ROC curves.
• Absence of perfect separation: Perfect (or quasi-complete) separation happens when the outcome variable separates a predictor variable — or a combination of predictors — completely. This can cause issues with model estimation and should be checked for.

Definition List (Glossary)

Key technical terms used throughout this guide, organized by topic.

Variables

Categorical Variables

Groups or labels like types of fruit or yes/no answers.

Continuous Variables

Numbers that can vary, like height or temperature.

Independent Variable

Something changed by the researcher to see its effect.

Dependent Variable

Something measured in response to changes in the independent variable.

Data Characteristics

Normally Distributed / Skewed Data

Data that either fall in a normal bell-shaped pattern or are tilted to one side.

Binary Outcome

Results that only have two options, like pass/fail.

Testing and Measurement

Non-Parametric Test / Nonparametric Methods

Tests used for data that don't fit a normal pattern or when there's not much data.

Repeated Measures

Taking multiple measurements of the same thing over time.

Multivariate Analysis

Looking at more than one outcome measure at a time.

ANOVA Specifics

Levels

Different groups or conditions being compared.

Between-Subjects Variable

Differences between different groups of subjects.

Within-Subjects Variable

Differences within the same subjects under varying conditions.

Interaction Effect

When two variables together have a different effect than each on their own.

Error Types

Type I Error

Incorrectly thinking something has an effect when it doesn't (false positive).

Type II Error

Not detecting an effect when there is one (false negative).

Inference Statistics

P-Value

A measure that helps determine whether your results are likely due to chance.

Confidence Interval

A range that is likely to contain the true value you're trying to estimate.

Effect Size

The size or strength of an effect or relationship.

Power of a Test

The likelihood that your test will find an effect if there is one to be found.

Regression Analysis

Regression Coefficient

A number showing the size of the effect one variable has on another.

Odds Ratio

A measure of association between an outcome and a variable, used in logistic regression.

Residuals

The differences between observed values and those predicted by a model.

Quick Reference: Choosing a Test

Research Question	Data Type	Suggested Test
Compare means of two independent groups	Continuous, normally distributed	Independent Samples T-test
Compare means of the same group over time (2 time points)	Continuous, normally distributed differences	Paired Samples T-test
Compare means of three or more groups	Continuous, normally distributed	One-way ANOVA
Effect of two factors (and their interaction)	Continuous, normally distributed	Two-way ANOVA
Same subjects measured three or more times	Continuous, sphericity holds	Repeated Measures ANOVA
Relationship between two categorical variables	Categorical (nominal/ordinal)	Chi-Square Test
How two continuous variables move together	Continuous, linear relationship	Pearson Correlation
How two ranked/non-normal variables move together	Ordinal or non-normal continuous	Spearman's Rank Correlation
Compare two independent groups (non-normal data)	Ordinal or non-normal continuous	Mann-Whitney U Test
Compare two related groups (non-normal data)	Ordinal or non-normal continuous	Wilcoxon Signed-Rank Test
Predict a continuous outcome	Continuous dependent variable	Linear Regression
Predict a binary outcome (yes/no)	Binary dependent variable	Logistic Regression

Where to Get Help

• Statistics Consulting / Research Support Services — help choosing tests, checking assumptions, and interpreting results
• Your Advisor / Supervising Faculty — guidance on the analysis plan for your specific research project
• Graduate Programs Office — training resources and workshops on research methods
• University Library — databases, statistical software access, and reference guides
• Peer Study Groups / Student Associations — peer support and shared learning on quantitative methods