Quick Answer: Intraclass Correlation Coefficient Result

The ICC report used G1, G2 and G3 as repeated rating or measurement columns. The full Python and R reports used 649 complete subjects and 3 ratings/measurements. The grand mean across the repeated rating matrix was approximately 11.6251.

The recommended default result in this example is ICC(3,k), the two-way mixed-effects average-measure consistency model. The value is ICC(3,k) = 0.950607, which is interpreted as excellent reliability. The single-measure fixed-consistency model is ICC(3,1) = 0.865144, interpreted as good reliability.

What Is Intraclass Correlation Coefficient?

Intraclass Correlation Coefficient is a reliability statistic used when several ratings, raters, measurements or repeated scores are collected for the same subjects. Unlike ordinary Pearson correlation, ICC does not only ask whether two variables move together. It asks how much of the total variation is due to true differences between subjects compared with rating or measurement error.

ICC is commonly used for inter-rater reliability, test-retest reliability, repeated measurement consistency and agreement between measurement methods. In this example, the repeated columns are G1, G2 and G3. These can be viewed as repeated grade measurements for the same students.

ICC connects closely with effect size, confidence interval, standard deviation, variance, ANOVA in SPSS, ANOVA in R and descriptive statistics explained.

ICC Models Explained

There are several ICC models because reliability can be defined in different ways. The correct model depends on whether raters or measurements are random or fixed, whether you want single-rating reliability or average-rating reliability, and whether exact agreement or consistency is the goal.

ICC Formula and ANOVA Components

ICC formulas are built from ANOVA mean squares. The repeated rating matrix is decomposed into between-subject variation, between-rater or measurement-column variation, and residual/interaction error.

For the full Python and R reports, the important ANOVA components are:

Because the subject mean square is much larger than the residual mean square, ICC values are high. In practical terms, students differ from one another much more than the repeated rating columns disagree for the same student.

Dataset and Rating Columns Used

The analysis used the student performance dataset. The repeated rating or measurement columns were G1, G2 and G3. The full Python/R/SPSS reports used 649 complete subjects.

The rating means increase from G1 to G3, but the main reliability question is whether subjects keep a consistent relative pattern across these columns. The high ICC values show that the repeated grade columns are highly consistent overall.

Verified ICC Results

The model comparison table shows that all ICC values are high. Single-measure ICC values are in the good range, while average-measure ICC values are in the excellent range. This is expected because averaging across repeated ratings reduces measurement noise.

The best short conclusion is that the average of the three grade measurements has excellent reliability. If only one measurement column is used, reliability remains good but lower than the average-measure ICC.

Python Chart-by-Chart Interpretation

The Python output includes six charts and a PDF report. These charts explain rating means, subject profiles, model comparison, pairwise correlations, variance components and within-subject spread.

Python Chart 1: Rating Means with Confidence Intervals

This chart compares the average scores for G1, G2 and G3. The means increase from G1 to G3, which shows a systematic measurement-column difference. This is important because agreement models treat systematic mean differences more strictly than consistency models.

The chart helps explain why model choice matters. If exact agreement is required, the mean differences between rating columns matter more. If consistency is the goal, a systematic increase from G1 to G3 may be acceptable as long as subjects keep similar relative positions.

Python Chart 2: Subject Rating Profiles

The subject profile chart shows how individual subjects move across G1, G2 and G3. When profiles are generally parallel and maintain similar ordering, consistency reliability is high. Large random crossing and wide within-subject variation would lower the ICC.

Here, the profiles support the numerical ICC result. Subjects with relatively high scores tend to remain high across the repeated grade columns, while lower-scoring subjects tend to remain lower. That pattern supports good-to-excellent reliability.

Python Chart 3: ICC Model Comparison

The model comparison chart shows that average-measure ICC values are higher than single-measure ICC values. ICC(3,k) is about 0.951, while ICC(3,1) is about 0.865. This is expected because averaging G1, G2 and G3 produces a more reliable summary than relying on one rating column.

The chart is useful for reporting because it prevents a vague “ICC was high” statement. It shows exactly which model was selected and how other models compare.

Python Chart 4: Pairwise Correlation Heatmap

The pairwise correlation heatmap provides supporting context. G1 correlates strongly with G2 and G3, and G2 correlates very strongly with G3. These high pairwise correlations support the idea that the repeated grade columns measure a similar performance pattern.

Pairwise correlations are helpful, but they do not replace ICC. ICC uses the full repeated-measures structure and variance components, while pairwise correlation only compares two columns at a time.

Python Chart 5: Variance Components

The variance component chart shows that the largest share of variation is between subjects or targets. A large between-subject component means students truly differ from one another. ICC becomes high when this true subject variation is large relative to residual measurement error.

The residual or interaction component is much smaller than the subject component. This supports the excellent average-measure ICC conclusion.

Python Chart 6: Subject Mean vs Within-Subject Spread

This chart compares each subject’s average score with the within-subject standard deviation across G1, G2 and G3. Lower within-subject spread means repeated ratings are closer for the same subject.

The chart helps diagnose reliability at the subject level. If many subjects had very high within-subject spread, ICC would fall. In this report, the overall pattern supports stable repeated measurement.

R Chart-by-Chart Interpretation

The R output validates the Python result with colorful charts. It repeats the same six visual checks: rating means, subject profiles, ICC model comparison, pairwise correlations, variance components and mean-versus-spread context.

R Chart 1: Colorful Rating Means with Confidence Intervals

The R mean chart confirms the same pattern: G1 has the lowest mean, G2 is slightly higher and G3 is highest. This supports the need to distinguish consistency from agreement.

R Chart 2: Colorful Subject Rating Profiles

The subject profiles show repeated grade patterns across G1, G2 and G3. Parallel or similarly ordered profiles indicate stronger consistency. This visual pattern agrees with the high ICC values.

R Chart 3: Colorful ICC Model Comparison

The R model comparison chart confirms that single-measure models are in the good range and average-measure models are in the excellent range. ICC(3,k) is the main average fixed-measure consistency result.

R Chart 4: Colorful Pairwise Correlation Heatmap

The R heatmap shows strong pairwise relationships: G1 with G2, G1 with G3 and G2 with G3 are all strongly positive. These correlations support the reliability interpretation but should be treated as context rather than a substitute for ICC.

R Chart 5: Colorful Variance Components

The R variance component chart shows that the between-subject component is the dominant part of the model. This is the key reason the ICC is high: most variation reflects real differences between students rather than random disagreement across repeated measurements.

R Chart 6: Colorful Subject Mean vs Within-Subject Spread

This chart checks whether subjects with particular mean levels have unusually large within-subject spread. Lower spread supports closer repeated ratings. The overall pattern supports the excellent average-measure reliability conclusion.

SPSS Output Interpretation

The SPSS output confirms the repeated rating columns and descriptive context. G1, G2 and G3 each have 649 valid cases. The means are approximately 11.40, 11.57 and 11.91, with standard deviations 2.745, 2.914 and 3.231.

Open the SPSS Intraclass Correlation Coefficient Output PDF

Excel Worked File Explanation

The Excel workbook is a fully worked formula file for teaching ICC calculations. It uses a smaller sample of 20 subjects and 3 ratings/measurements to show each step clearly. This workbook is meant for manual formula learning, while the Python/R reports give the full 649-subject analysis.

Download the Intraclass Correlation Coefficient Fully Worked Excel File

In the Excel teaching workbook, the default reported model is ICC(3,k), with ICC(3,k) = 0.996821. This is interpreted as excellent reliability. The workbook also reports ICC(3,1) = 0.990524, ICC(2,k) = 0.984564, ICC(2,1) = 0.955080, ICC(1,k) = 0.984372 and ICC(1,1) = 0.954538.

Python, R, SPSS and Excel Workflows

Code Blocks and Excel Formulas

Python Code for Intraclass Correlation Coefficient

import pandas as pd
import numpy as np

df = pd.read_csv("dataset.csv")

rating_cols = ["G1", "G2", "G3"]
data = df[rating_cols].dropna().astype(float)

n = data.shape[0]       # subjects
k = data.shape[1]       # ratings / measurements

grand_mean = data.values.mean()
subject_means = data.mean(axis=1)
rating_means = data.mean(axis=0)

ss_subject = k * ((subject_means - grand_mean) ** 2).sum()
ss_rater = n * ((rating_means - grand_mean) ** 2).sum()
ss_total = ((data - grand_mean) ** 2).values.sum()
ss_error = ss_total - ss_subject - ss_rater

df_subject = n - 1
df_rater = k - 1
df_error = (n - 1) * (k - 1)

ms_subject = ss_subject / df_subject
ms_rater = ss_rater / df_rater
ms_error = ss_error / df_error

# One-way within error for ICC(1,1) and ICC(1,k)
ss_within = ((data.sub(subject_means, axis=0)) ** 2).values.sum()
df_within = n * (k - 1)
ms_within = ss_within / df_within

icc_1_1 = (ms_subject - ms_within) / (ms_subject + (k - 1) * ms_within)
icc_1_k = (ms_subject - ms_within) / ms_subject

icc_2_1 = (ms_subject - ms_error) / (
    ms_subject + (k - 1) * ms_error + k * (ms_rater - ms_error) / n
)
icc_2_k = (ms_subject - ms_error) / (
    ms_subject + (ms_rater - ms_error) / n
)

icc_3_1 = (ms_subject - ms_error) / (ms_subject + (k - 1) * ms_error)
icc_3_k = (ms_subject - ms_error) / ms_subject

print("n subjects:", n)
print("k ratings:", k)
print("Grand mean:", grand_mean)
print("ICC(1,1):", icc_1_1)
print("ICC(1,k):", icc_1_k)
print("ICC(2,1):", icc_2_1)
print("ICC(2,k):", icc_2_k)
print("ICC(3,1):", icc_3_1)
print("ICC(3,k):", icc_3_k)

R Code for Intraclass Correlation Coefficient

df <- read.csv("dataset.csv", stringsAsFactors = FALSE)

rating_cols <- c("G1", "G2", "G3")
data <- na.omit(df[, rating_cols])
data <- as.data.frame(lapply(data, as.numeric))

n <- nrow(data)
k <- ncol(data)

grand_mean <- mean(as.matrix(data))
subject_means <- rowMeans(data)
rating_means <- colMeans(data)

ss_subject <- k * sum((subject_means - grand_mean)^2)
ss_rater <- n * sum((rating_means - grand_mean)^2)
ss_total <- sum((as.matrix(data) - grand_mean)^2)
ss_error <- ss_total - ss_subject - ss_rater

df_subject <- n - 1
df_rater <- k - 1
df_error <- (n - 1) * (k - 1)

ms_subject <- ss_subject / df_subject
ms_rater <- ss_rater / df_rater
ms_error <- ss_error / df_error

ss_within <- sum((as.matrix(data) - subject_means)^2)
df_within <- n * (k - 1)
ms_within <- ss_within / df_within

icc_1_1 <- (ms_subject - ms_within) / (ms_subject + (k - 1) * ms_within)
icc_1_k <- (ms_subject - ms_within) / ms_subject

icc_2_1 <- (ms_subject - ms_error) /
  (ms_subject + (k - 1) * ms_error + k * (ms_rater - ms_error) / n)

icc_2_k <- (ms_subject - ms_error) /
  (ms_subject + (ms_rater - ms_error) / n)

icc_3_1 <- (ms_subject - ms_error) /
  (ms_subject + (k - 1) * ms_error)

icc_3_k <- (ms_subject - ms_error) / ms_subject

cat("n subjects:", n, "\n")
cat("k ratings:", k, "\n")
cat("Grand mean:", grand_mean, "\n")
cat("ICC(1,1):", icc_1_1, "\n")
cat("ICC(1,k):", icc_1_k, "\n")
cat("ICC(2,1):", icc_2_1, "\n")
cat("ICC(2,k):", icc_2_k, "\n")
cat("ICC(3,1):", icc_3_1, "\n")
cat("ICC(3,k):", icc_3_k, "\n")

SPSS Syntax for ICC Context

* Intraclass Correlation Coefficient in SPSS.
OUTPUT CLOSE ALL.
OUTPUT NEW NAME=ICC_Output.

DESCRIPTIVES VARIABLES=G1 G2 G3
  /STATISTICS=MEAN STDDEV MIN MAX.

CORRELATIONS
  /VARIABLES=G1 G2 G3
  /PRINT=TWOTAIL
  /MISSING=PAIRWISE.

RELIABILITY
  /VARIABLES=G1 G2 G3
  /SCALE('G1_G2_G3_Reliability') ALL
  /MODEL=ALPHA
  /STATISTICS=DESCRIPTIVE SCALE CORR.

EXAMINE VARIABLES=G1 G2 G3
  /PLOT BOXPLOT HISTOGRAM NPPLOT
  /STATISTICS DESCRIPTIVES
  /CINTERVAL 95
  /MISSING LISTWISE.

OUTPUT EXPORT
  /CONTENTS EXPORT=VISIBLE
  /PDF DOCUMENTFILE='Intraclass-Correlation-Coefficient-SPSS-Output.pdf'.

Excel Formula Patterns for ICC

n subjects:
=COUNT(B2:B650)

k ratings:
=COUNTA(B1:D1)

Grand mean:
=AVERAGE(B2:D650)

Subject mean:
=AVERAGE(B2:D2)

Rating mean:
=AVERAGE(B2:B650)

SS_subject:
=k*SUMXMY2(subject_mean_range, grand_mean_repeated_range)

SS_rater:
=n*SUMXMY2(rating_mean_range, grand_mean_repeated_range)

SS_total:
=SUM((each_score-grand_mean)^2)

SS_error:
=SS_total-SS_subject-SS_rater

MS_subject:
=SS_subject/(n-1)

MS_rater:
=SS_rater/(k-1)

MS_error:
=SS_error/((n-1)*(k-1))

ICC(3,k):
=(MS_subject-MS_error)/MS_subject

ICC(3,1):
=(MS_subject-MS_error)/(MS_subject+(k-1)*MS_error)

How to Report Intraclass Correlation Coefficient

A complete ICC report should name the model, rating columns, sample size, number of ratings, ICC value, confidence interval and reliability interpretation. Do not report ICC without identifying the model.

Common Mistakes in ICC Interpretation

Downloads and Resources

External References

For additional software documentation, see IBM SPSS reliability analysis documentation, R package documentation for ICC/reliability functions and Python reliability examples using ANOVA mean-square formulas.

Related Statistical Guides