ANOVA Test Explained: Full Guide, Formula & Example (2026)

ANOVA is a statistical test used to compare the means of three or more groups in order to find out if there exists a significant difference among the groups. Scientists choose to use ANOVA rather than performing several t-tests, since the latter increases the probability of having a Type I error.

The following sections include the formula for ANOVA, the calculation procedure (in 6 steps), effect size, software procedures, APA guidelines, and common mistakes — all illustrated with an original example.

What Is an ANOVA Test?

ANOVA stands for Analysis of Variance and is a type of statistical test used to find out whether the means of three or more groups are significantly different from each other.

ANOVA tests two types of variances:

• Between-groups variance: the difference between the means of the groups and the grand mean.
• Within-groups variance: the difference between the individual scores within each group and the mean of that group.

When there is more between-group variance than within-group variance, there is little possibility that the means of the groups are all equal. ANOVA was first introduced by the statistician Ronald Fisher in the 1920s and its test statistic is called the F-statistic after him.

In an ANOVA test, we need a single categorical independent variable with three or more levels, and a continuous dependent variable.

ANOVA vs. T-Test — When to Use Which

When testing for differences between the means of two populations, you should use a t-test. When you have more than two populations and need to test the mean differences among them, you should use an ANOVA.

Performing a t-test for each possible combination of two populations would increase the probability of obtaining a wrong conclusion. For example, three populations require three t-tests, while five populations require ten. 

Criterion	T-Test	ANOVA
Number of groups compared	2	3 or more
Number of tests needed for 3 groups	3 separate tests	1 test
Risk of Type I error at 3+ groups	Increases with each test	Controlled at set alpha (e.g., 0.05)
Output statistic	t-value	F-statistic
Follow-up needed to locate specific differences	Not applicable	Post-hoc test required

Types of ANOVA

ANOVA has several forms depending on the number of factors and the study design.

One-Way ANOVA

The One-Way ANOVA is used when there is only one independent variable that has three or more levels. Example: comparison of checkout completion times between three web page designs.

Two-Way ANOVA

Two-Way ANOVA analyzes the impact of two factors along with their interactions on one dependent variable. For instance, when one compares checkout time of three different websites on mobiles and desktops.

Factorial ANOVA

Factorial ANOVA extends two-way ANOVA to three or more factors simultaneously. Example: testing website design, device type, and browser together in one model.

Repeated Measures ANOVA

Repeated Measures ANOVA is a method used for comparing mean scores where the same subjects are exposed to various conditions. Example: measuring the task completion time of the same 20 users before, during, and after the redesign process.

MANOVA

MANOVA (Multivariate Analysis of Variance) extends ANOVA to two or more dependent variables at once. It is a related test, not a type of ANOVA itself.

Which ANOVA Test Do You Actually Need?

Follow the procedure below for choosing a test:

1. Count the number of independent variables. One variable indicates One Way ANOVA, whereas two or more variables indicate Two Way or Factorial ANOVA.
2. Find out if the subjects have been measured again and again. If yes, use Repeated Measures ANOVA, regardless of the number of factors.
3. Find out if the normality and equal variance assumptions have been met (discussed in the next section). If not, apply Welch’s ANOVA or the Kruskal-Wallis test.

ANOVA Assumptions

ANOVA produces valid results only when three assumptions hold.

Independence of Observations

Each piece of data has to come from an independent unit. It means that no one’s results can affect the other person’s results.

Homogeneity of Variance

The variances of the groups have to be equal. Levene’s test is used to determine if variances are equal or not. A Levene’s test result below 0.05 means the variances are not equal. 

Normal Distribution

The dependent variable has to be normally distributed in each group. To determine this, we use the Shapiro-Wilk test. 

What Happens If an Assumption Fails?

If the equal-variance assumption fails, use Welch’s ANOVA instead of standard one-way ANOVA. Welch’s ANOVA adjusts the degrees of freedom to account for unequal variances and does not assume equal group sizes.

If the normality assumption fails and the sample size is small, use the Kruskal-Wallis test, the non-parametric alternative to one-way ANOVA.

How to Calculate ANOVA — Step by Step

The following example uses original data: checkout completion time, in seconds, for three website designs (5 users per design).

User	Design A	Design B	Design C
1	42	38	50
2	45	36	52
3	40	39	48
4	44	37	51
5	43	35	49

Step 1 — Define Your Hypotheses

• Null hypothesis (H₀): the mean completion time is equal across all three designs.
• Alternative hypothesis (H₁): at least one design has a different mean completion time.

Step 2 — Calculate Group Means and the Grand Mean

Design	Mean (seconds)
A	42.8
B	37.0
C	50.0
Grand mean	43.27

Step 3 — Calculate Sum of Squares

• Sum of Squares Between (SSB) measures how far each group mean is from the grand mean, weighted by group size: SSB = 424.13
• Sum of Squares Within (SSW) measures how far individual scores are from their own group mean: SSW = 34.80
• Total Sum of Squares (SST) is the sum of both: SST = 458.93

Step 4 — Calculate Degrees of Freedom and Mean Squares

Source	Degrees of Freedom	Sum of Squares	Mean Square
Between groups	2	424.13	212.07
Within groups	12	34.80	2.90
Total	14	458.93	—

Degrees of freedom between groups equals the number of groups minus one (3 − 1 = 2). Degrees of freedom within groups equals the total sample size minus the number of groups (15 − 3 = 12).

Step 5 — Calculate the F-Statistic

The F-statistic equals the mean square between groups divided by the mean square within groups: F = 212.07 / 2.90 = 73.13.

Step 6 — Find Your P-Value and Make a Decision

At α = 0.05, with df(2, 12), the critical F-value is 3.89. The calculated F-statistic (73.13) exceeds this critical value, and the resulting p-value is below 0.001.

Because the calculated F-statistic exceeds the critical value, the null hypothesis is rejected. Website design has a statistically significant effect on checkout completion time.

How to Calculate Effect Size (Eta-Squared)

A statistically significant F-statistic indicates that a difference exists; it does not indicate how large that difference is. Effect size measures the magnitude of the difference.

Eta-squared (η²) equals the sum of squares between groups divided by the total sum of squares. For the example above: η² = 424.13 / 458.93 = 0.92.

Eta-Squared Value	Interpretation
0.01	Small effect
0.06	Medium effect
0.14 or higher	Large effect

An η² of 0.92 indicates that website design accounts for 92% of the variance in checkout completion time in this dataset — a large effect.

Running ANOVA in Excel, R, and Python

Excel

Excel calculates one-way ANOVA through the Data Analysis ToolPak. Enable the ToolPak under File → Options → Add-ins, then select Data Analysis → ANOVA: Single Factor from the Data tab.

R

The aov() function runs ANOVA in R:

model <- aov(completion_time ~ design, data = checkout_data)

summary(model)

Python

The scipy.stats.f_oneway() function runs one-way ANOVA in Python:

from scipy import stats

f_stat, p_value = stats.f_oneway(design_a, design_b, design_c)

For output that includes sum of squares and degrees of freedom, statsmodels provides a full ANOVA table:

import statsmodels.api as sm

from statsmodels.formula.api import ols

model = ols('completion_time ~ design', data=checkout_data).fit()

anova_table = sm.stats.anova_lm(model, typ=2)

How to Report ANOVA Results (APA Style)

The APA guidelines for reporting ANOVA results include the following format: F(df between, df within) = F value, p = p-value.

In the above example: One-way ANOVA revealed that there was a statistically significant effect of the website design on the duration taken to complete the checkout process, F(2, 12) = 73.13, p < .001, η² = 0.92.

The F value should be reported to two decimal places while the p-value should be rounded off to three decimal places (p < .001 when less than that).

Post-Hoc Tests — Finding Which Groups Differ

ANOVA shows that there is some difference between the groups, but it doesn’t tell you which specific groups differ. Post-hoc testing solves this problem by making pairwise comparisons.

Post-Hoc Test	Best Used When
Tukey’s Honestly Significant Difference (HSD)	Group sizes are equal and all pairwise comparisons are needed
Bonferroni correction	A small number of specific, planned comparisons are needed
Scheffe test	Group sizes are unequal or complex comparisons are required

In the case of the checkout process, the Tukey HSD test would show that the mean times for all three designs (37.0, 42.8, 50.0 seconds) are substantially different because they are well-separated compared to the within-groups variance.

Common ANOVA Mistakes to Avoid

• Interpreting a high F-statistic as evidence of differences between all groups. This simply means that at least one group differs from another; we need to perform post-hoc tests to find which one differs.
• Not checking assumptions. Performing regular ANOVA on data with unequal variances will yield a meaningless p-value.
• Neglecting unequal sample sizes among groups. Uneven sample sizes make the test more sensitive to violation of the equal variances assumption.
• Interpreting the F-statistic without effect size. A significant finding with a low effect size might be practically irrelevant.
• Confusing statistical significance with practical significance. If the p-value is less than 0.05, the findings are unlikely to be due to random chance, but it does not imply they are important.

Alternatives to ANOVA

Use the Kruskal Wallis test if the assumption of normality is violated and the sample size is small. The Kruskal Wallis test is the alternative to one-way ANOVA, which is used for comparing medians based on ranks rather than means.

Use Welch’s ANOVA if the assumption of equal variances is violated, but the assumption of normality is satisfied. Welch’s ANOVA does not assume equal variances or equal sample sizes.

Real-World Applications of ANOVA

• Medicine: comparison of patient results between at least three doses or medicines.
• Education: comparison of test performance between at least two teaching methods or courses.
• User experience research: comparison of user experience metrics such as time to completion, errors per minute, and satisfaction score between various designs.
• Manufacturing/quality assurance: comparison of defect percentage and/or output statistics among several production lines or machine setups.

Frequently Asked Questions

Conclusion

ANOVA is concerned with answering a specific question: Is there any difference between three or more means that cannot be attributed to chance? The answers to questions like this are found in F and p values, effect size, and post-hoc tests. When combined, these three provide answers for one test.