Quick Answer: Scheffe Test Result

The worked example compares G3 final grade across four studytime groups. The sample contains 649 students. The one-way ANOVA is statistically significant, with F(3, 645) = 15.876, p < .001, so post hoc comparisons are justified. The Scheffe Test is then used to check which studytime groups differ after a conservative multiple-comparison correction.

The group means show the same clear pattern seen in the ANOVA family examples. Group 1 has the lowest mean G3 score, about 10.84. Group 2 has a higher mean, about 12.09. Group 4 has a mean near 13.06, and group 3 has the highest mean, about 13.23. The strongest differences are between group 1 and the higher studytime groups.

What Is Scheffe Test?

Scheffe Test is a post hoc multiple comparison method used after ANOVA. It checks whether group means or more general contrasts are significantly different while controlling the family-wise error rate. A simple pairwise post hoc test compares one group mean with another. Scheffe can also protect more complex comparisons, such as one group against the average of two other groups.

The method is known for being conservative. This means it reduces the risk of false-positive conclusions, but it can also miss smaller real differences that less conservative tests might mark as significant. For this reason, Scheffe is often chosen when the researcher wants strong protection across many possible comparisons or when the comparisons were not all planned before looking at the data.

In this guide, the Scheffe Test is used after one-way ANOVA to compare four studytime groups on G3 final grade. The main teaching point is that ANOVA shows whether at least one group differs, while Scheffe explains which group differences remain strong after a conservative correction.

Before using Scheffe, review one-way ANOVA, ANOVA assumptions, Levene test, F distribution, p-values, confidence intervals and effect size.

When to Use Scheffe Test

Use Scheffe Test when a one-way ANOVA is significant and you want a conservative method for post hoc group comparisons. It is especially helpful when comparisons were not planned in advance or when you want protection for a broad family of possible contrasts.

Scheffe Test Formula

The Scheffe Test is based on the ANOVA error term. For a contrast among group means, the contrast value is:

The Scheffe F statistic for the contrast is:

The Scheffe critical value is:

For a simple pairwise comparison between group i and group j, the contrast coefficients are usually +1 for one group, −1 for the other group, and 0 for the remaining groups.

Null and Alternative Hypotheses for Scheffe Test

The Scheffe Test can be used for simple pairwise comparisons or more general contrasts. In this article, the focus is pairwise post hoc comparison after one-way ANOVA.

Dataset and Variables Used

The worked example uses a student performance dataset. The dependent variable is G3 final grade. The grouping variable is studytime, coded into four weekly study-time categories. The goal is to test whether average final grade differs by studytime and then use the Scheffe Test to identify which group differences remain significant after conservative adjustment.

Before interpreting Scheffe adjusted p-values, review the group means, distributions, group sizes and variance context. Helpful related guides include descriptive statistics, box plot interpretation, standard deviation, ANOVA in SPSS and F distribution.

ANOVA Result and Scheffe Decision Table

The ANOVA result confirms that post hoc testing is justified. The between-group sum of squares is about 465.078, the within-group sum of squares is about 6298.189, and the total sum of squares is about 6763.267. The ANOVA result is F(3, 645) = 15.876, p < .001.

Scheffe Pairwise Interpretation Summary

Python Chart-by-Chart Interpretation

The Python charts show the complete Scheffe Test workflow. They begin with group means and distributions, then show ANOVA variation, Scheffe adjusted p-values, mean differences, raw versus adjusted significance counts, group-size context and histograms.

Python Chart 1: Group Means with Confidence Intervals

The group means chart shows the practical pattern behind the Scheffe Test. Group 1 has the lowest average G3 score. Group 2 is higher, while groups 3 and 4 are the highest. This visible separation explains why the overall ANOVA was significant.

The confidence intervals show uncertainty around the mean estimates. Group 4 has fewer observations, so its mean should be interpreted with more caution. The chart supports the final conclusion that group 1 is clearly lower than the higher studytime groups.

Python Chart 2: Group Distribution Boxplots

The boxplots show the spread of G3 scores inside each studytime group. Group 1 is centered lower than the other groups, while groups 3 and 4 are centered higher. This supports the mean-based result shown in the confidence interval chart.

The boxplots also show overlap between groups. This is normal in real data and explains why post hoc testing is needed. Scheffe compares mean gaps relative to within-group error and then applies a conservative correction.

Python Chart 3: ANOVA Sum of Squares Breakdown

The sum of squares chart explains the overall ANOVA. Between-group variation represents differences among studytime means. Within-group variation represents student-to-student variation inside each studytime group.

The ANOVA is significant because the between-group mean square is large relative to the within-group mean square. The Scheffe Test then identifies which group differences remain strong after conservative multiple-comparison control.

Python Chart 4: Scheffe Adjusted p-values

The adjusted p-value chart is the main Scheffe decision chart. Comparisons below .05 are interpreted as significant after Scheffe correction. The strongest comparisons involve group 1 against groups 2, 3 and 4, plus group 2 against group 3.

The chart also shows why close mean differences should not be exaggerated. Groups 2 and 4 are closer, and groups 3 and 4 are almost the same in mean value. These pairs do not support strong Scheffe-adjusted claims.

Python Chart 5: Pairwise Mean Difference Heatmap

The mean difference heatmap shows the size and direction of the group gaps. The largest gap is between group 1 and group 3. Group 1 compared with group 4 is also large, and group 1 compared with group 2 is moderate.

The smallest gap is between group 3 and group 4. This explains why the highest two groups should be interpreted as similar in the final Scheffe report.

Python Chart 6: Raw vs Scheffe Significant Counts

This chart explains why adjustment matters. Raw pairwise tests can make too many significant claims because they do not fully protect the family of comparisons. Scheffe correction applies stronger protection, so the final number of significant comparisons may be smaller or more defensible.

In this dataset, the strongest differences remain important even after conservative adjustment. The chart helps readers understand that the Scheffe Test is not only asking whether a difference exists; it is asking whether that difference is strong enough after strict correction.

Python Chart 7: Group Size and Mean

The group size and mean chart shows that group sizes are unequal. Group 2 is the largest group, group 1 is also large, group 3 is smaller, and group 4 is the smallest. This matters because sample size affects precision and the width of confidence intervals.

The chart supports careful reporting for group 4. Even though group 4 has a high mean, it has fewer observations, so its comparisons should be interpreted with sample-size context.

Python Chart 8: Group Histograms

The histograms show the shape of G3 scores inside each studytime group. The lowest studytime group contains more lower scores, while the higher studytime groups shift toward higher G3 values.

This chart gives raw distribution context behind the ANOVA and Scheffe results. It supports the practical conclusion that studytime groups differ in average final grade, while still showing that real student scores overlap across groups.

SPSS, R, Python and Excel Workflows for Scheffe Test

The Scheffe Test can be completed in SPSS, R, Python and Excel. SPSS provides a direct menu option for Scheffe post hoc testing. R and Python are useful for reproducible ANOVA, contrasts, adjusted p-values and charts. Excel can reproduce the teaching calculations for means, mean square error, F statistics and conservative decision rules.

SPSS Workflow

R Workflow

Python Workflow

Excel Workflow

Excel can reproduce the Scheffe Test for teaching by calculating group means, sample sizes, ANOVA MSE, contrast values, Scheffe F statistics and the Scheffe critical value. SPSS, R or Python is better for full automation, but Excel is useful for understanding each calculation step.

Code Blocks for Scheffe Test

SPSS Syntax

ONEWAY G3 BY studytime
  /STATISTICS DESCRIPTIVES HOMOGENEITY
  /POSTHOC = SCHEFFE ALPHA(0.05).

R Code

data <- read.csv("dataset.csv")
data$studytime <- factor(data$studytime)

# One-way ANOVA

model <- aov(G3 ~ studytime, data = data)
summary(model)

# Group summary

aggregate(G3 ~ studytime, data = data, function(x) {
c(n = length(x), mean = mean(x), sd = sd(x))
})

# Scheffe post hoc testing can be run with a suitable post hoc package.

# Example package-based workflows may use scheffe.test() where available.

# Report group means, adjusted p-values, confidence intervals and decisions.

Python Code

import pandas as pd
import itertools
from scipy import stats
import statsmodels.api as sm
import statsmodels.formula.api as smf

df = pd.read_csv("dataset.csv")
df["studytime"] = df["studytime"].astype("category")

# One-way ANOVA

model = smf.ols("G3 ~ C(studytime)", data=df).fit()
anova = sm.stats.anova_lm(model, typ=2)

ss_between = anova.loc["C(studytime)", "sum_sq"]
ss_error = anova.loc["Residual", "sum_sq"]
df_error = anova.loc["Residual", "df"]
mse = ss_error / df_error

groups = df["studytime"].cat.categories
k = len(groups)
n_total = len(df)

summary = df.groupby("studytime")["G3"].agg(["count", "mean", "std"])

scheffe_critical = (k - 1) * stats.f.ppf(0.95, k - 1, df_error)

rows = []
for g1, g2 in itertools.combinations(summary.index, 2):
n1 = summary.loc[g1, "count"]
n2 = summary.loc[g2, "count"]
m1 = summary.loc[g1, "mean"]
m2 = summary.loc[g2, "mean"]

```
diff = m1 - m2
denominator = mse * (1/n1 + 1/n2)
scheffe_f = (diff ** 2) / denominator
scheffe_p = stats.f.sf(scheffe_f / (k - 1), k - 1, df_error)

rows.append([
    g1, g2, n1, n2, m1, m2, diff,
    scheffe_f, scheffe_critical, scheffe_p,
    scheffe_p < 0.05
])
```

scheffe_table = pd.DataFrame(rows, columns=[
"group_1", "group_2", "n_1", "n_2",
"mean_1", "mean_2", "mean_difference",
"scheffe_f", "scheffe_critical", "scheffe_p",
"scheffe_significant"
])

print(anova)
print(summary)
print(scheffe_table.sort_values("scheffe_p"))

Excel Formula Pattern

Group mean:
=AVERAGEIF(group_range, group_id, value_range)

Group sample size:
=COUNTIF(group_range, group_id)

Pairwise mean difference:
=Mean_Group_i - Mean_Group_j

Pairwise contrast denominator:
=MSE*(1/n_i + 1/n_j)

Scheffe F statistic:
=Mean_Difference^2 / Contrast_Denominator

Scheffe critical value:
=(k-1)*F.INV.RT(0.05,k-1,N-k)

Decision:
=IF(Scheffe_F>Scheffe_Critical,"Significant","Not significant")

APA Reporting Wording for Scheffe Test

A one-way ANOVA was conducted to compare G3 final grade across four studytime groups. The ANOVA was statistically significant, F(3, 645) = 15.876, p < .001, indicating that mean final grade differed across studytime levels. Scheffe post hoc comparisons were then examined to identify which group differences remained significant after conservative family-wise error correction.

Scheffe-adjusted comparisons showed that the lowest studytime group had lower G3 scores than several higher studytime groups. The strongest difference was between group 1 and group 3. The comparisons between group 2 and group 4 and between group 3 and group 4 were not clearly significant after Scheffe correction, so the highest studytime groups should not be described as all different from one another.

Common Mistakes in Scheffe Test

Downloads and Resources

Use the downloadable Python report to verify the Scheffe Test charts and compare the ANOVA output, adjusted p-values, mean differences and distribution summaries.

Related Guides

Continue with these related Salar Cafe guides to understand ANOVA, post hoc testing, assumptions, p-values and effect sizes.