Intraclass Correlation Coefficient: ICC(2,k) Formula, Interpretation, SPSS, Python, R and Excel Guide

This Intraclass Correlation Coefficient section defines reliability as a design-specific ratio of stable target variation to total modeled variation.

The Intraclass Correlation Coefficient is a family of reliability coefficients for quantitative measurements that are grouped within the same target. Each target may be a patient, student, product, image, laboratory specimen, household, school, or any other unit receiving repeated numerical ratings. The coefficient asks whether targets that score high on one rating also tend to score high on the other ratings, while also considering the amount of disagreement in the actual numerical values.

The word intraclass means that the observations compared belong to the same class or cluster. G1, G2, and G3 belong to the same student. Three clinicians’ measurements belong to the same patient. Several judges’ scores belong to the same performance. Because the observations are not independent within a target, a reliability model must preserve that grouped structure.

ICC values are commonly described on a zero-to-one scale, but the estimate can occasionally be negative when within-target error exceeds between-target variation. A negative estimate should not be converted to a positive value or interpreted as meaningful negative reliability. It usually signals that the measurement process does not separate targets reliably, that the selected model does not match the design, or that sampling uncertainty is substantial.

The Intraclass Correlation Coefficient therefore expresses reliability only for the measurement design and unit explicitly selected.

This Intraclass Correlation Coefficient section separates appropriate quantitative reliability applications from correlation, kappa, and change analysis.

The Intraclass Correlation Coefficient is appropriate when every row represents a target and several columns contain ratings, judges, instruments, or repeated quantitative measurements for that same target. Typical applications include inter-rater reliability, test-retest reliability, instrument calibration, repeated laboratory measurements, image scoring, classroom assessment, and reliability of aggregated panel ratings.

Do not default to ICC merely because a dataset has repeated columns. A Pearson correlation is suitable for association between two continuous variables when agreement is not the goal. Cohen’s kappa is designed for two categorical raters, Fleiss kappa extends categorical agreement to multiple raters, and weighted kappa incorporates ordered-category distances. A repeated-measures ANOVA tests mean change across occasions; it does not by itself quantify reliability.

The Intraclass Correlation Coefficient is also distinct from the ICC used in multilevel modeling to describe clustering. In a hierarchical linear model or multilevel regression, the cluster ICC is usually the proportion of outcome variance between groups such as schools or hospitals. In reliability analysis, the targets are the objects being rated and the model includes rater or occasion structure. Both are variance ratios, but their design meanings differ.

For the worked example, the research question is deliberately narrow: how reliable is the average of three grade-style measurements when absolute numerical agreement is required and the three occasions are treated as a random sample from a broader measurement universe? That question leads to ICC(2,k), not automatically to every other ICC form.

Use the Intraclass Correlation Coefficient when repeated quantitative measurements, rather than categorical labels, define the reliability question.

This Intraclass Correlation Coefficient section prioritizes correct design, target independence, comparable scales, and model justification.

The assumptions of the Intraclass Correlation Coefficient include a correct target-rater structure, independent targets, comparable quantitative scales, and an effects model that matches the intended generalization. ANOVA-based confidence intervals and tests additionally rely on distributional and variance assumptions.

Independence applies between targets, while ratings are intentionally dependent within a target. Every rating column must use the same direction and scale, and the reliability claim must concern the same unit represented by the coefficient. The inferential F test and confidence interval additionally rely on the two-way ANOVA framework used for ICC(2,k).

A defensible Intraclass Correlation Coefficient begins with correct target independence, measurement structure, and effects-model specification.

This Intraclass Correlation Coefficient section distinguishes the zero-reliability test from the practical interpretation of magnitude and uncertainty.

The null test is useful, but it is not the main practical interpretation. With 649 targets, even a modest coefficient could be statistically different from zero. Reliability decisions should focus on the point estimate, 95% confidence interval, measurement consequences, and the minimum acceptable reliability for the intended application.

Software may print a p-value of 0.000 or store an underflowed value of zero. That output does not mean the mathematical probability is exactly zero. Report p < .001 rather than p = .000. The site’s guides to p-value, hypothesis testing, and the null and alternative hypothesis provide broader reporting context.

The statistical test does not prove that every target has nearly identical G1, G2, and G3 values. It evaluates the aggregate variance structure. Individual targets can still show meaningful disagreement, especially at low scores or in cases with a zero on one occasion. Case-level review remains important when decisions have practical consequences.

The Intraclass Correlation Coefficient F test addresses reliability beyond zero, while the confidence interval describes plausible population values.

This Intraclass Correlation Coefficient section shows how the ANOVA mean squares enter the single-measure and average-measure equations.

The Intraclass Correlation Coefficient formula depends on the selected model. For a balanced two-way random-effects design with n targets and k raters or occasions, calculate the mean square for targets (MSR), the mean square for raters or columns (MSC), and the residual mean square (MSE).

The associated null test uses the ratio F = MSR/MSE with df₁ = n – 1 and df₂ = (n – 1)(k – 1). In this dataset, F = 24.083791/1.189560 = 20.245973, with 648 and 1296 degrees of freedom.

The formula is a ratio, not a percentage of correctly matched ratings. An Intraclass Correlation Coefficient of .948 does not mean 94.8% of individual scores are identical. It means the model-defined reliable component is very large relative to the error structure for the three-measure average.

The Intraclass Correlation Coefficient formula must be reported with its ICC model label because alternative formulas estimate different quantities.

This Intraclass Correlation Coefficient section documents the target, repeated columns, descriptive values, missing-case rule, and composite score.

The worked Intraclass Correlation Coefficient analysis uses the G1, G2, and G3 columns from a 649-row educational dataset. Each row is one target, and each column is treated as a rating or measurement occasion. The analysis is performed in wide format because every target has one value in each of the three repeated columns.

The source has complete values for all three analysis columns in the SPSS procedure: 649 valid cases and zero excluded cases. Missing-data handling should still be reported in other datasets. Listwise deletion changes the target sample, and pairwise use is generally unsuitable for a balanced ICC ANOVA because different coefficients could be based on different target sets.

This Intraclass Correlation Coefficient example treats G1, G2, and G3 as three repeated quantitative measurements for every target.

This Intraclass Correlation Coefficient section reproduces ICC(2,k), ICC(2,1), and the F statistic from full-precision components.

The exact Intraclass Correlation Coefficient calculation begins with the two-way ANOVA decomposition. The target mean square is 24.083791, the rater or occasion mean square is 43.165383, and the residual mean square is 1.189560.

The F statistic is calculated independently from the ratio MSR/MSE:

The rater mean square is larger than the target mean square, but that does not mean raters dominate the reliability estimate. Mean squares have different roles and degrees of freedom. The rater term enters ICC(2,k) only through (MSC – MSE)/n. In a large target sample, the systematic mean difference is estimated precisely and contributes a modest adjustment to the average-measure denominator.

The calculation also clarifies why a bar chart mixing ICC, F, p, target count, and rater count must not be interpreted by comparing bar heights. Those quantities have different units and scales. The numerical ledger and formula are the authoritative interpretation.

This Intraclass Correlation Coefficient section interprets the point estimate, confidence interval, measurement unit, and limits of generalization together.

A common descriptive convention labels ICC values below .50 as poor, .50 to .75 as moderate, .75 to .90 as good, and above .90 as excellent. Under that convention, ICC(2,k) = .948 indicates excellent reliability. These thresholds are guides, not universal laws. A clinical decision, high-stakes certification, or individual diagnostic score may require a stricter standard than exploratory group research.

High ICC does not guarantee small absolute measurement error. A heterogeneous sample can produce a high coefficient because targets span a wide range, even when repeated values differ enough to matter practically. Pair the coefficient with within-target differences, residual standard deviation, standard error, or a Bland-Altman plot when the size of disagreement in original units is important.

High reliability also does not establish validity. G1, G2, and G3 may measure targets consistently while still failing to measure the intended construct. Reliability is necessary for many forms of valid measurement but is not sufficient evidence of content, criterion, or construct validity.

The Intraclass Correlation Coefficient result is excellent for the three-measurement average, with a narrow confidence interval above .90.

The Intraclass Correlation Coefficient in Python workflow includes the exact two-way ANOVA decomposition and the complete chart sequence used in the accepted format: the primary-metrics chart appears alone, followed by two paired rows.

This Intraclass Correlation Coefficient section provides an auditable Python implementation rather than relying on an unlabeled default model.

Python can calculate the Intraclass Correlation Coefficient directly from the wide G1, G2, and G3 matrix. A manual calculation is useful for auditing because it exposes every sum of squares and mean square. A validated reliability package can then be used as a second implementation.

The calculation uses the complete repeated-measure rows. Before analysis, confirm data types, target duplication, missingness, scale direction, and whether all rating columns use the same units. The site’s correlation in Python guide can support pairwise diagnostics, but model selection must be decided separately.

Do not round the mean squares before calculating ICC. Store full-precision values and round only the displayed result. Early rounding can noticeably alter coefficients in smaller or less reliable datasets, particularly when MSR and MSE are close.

The Python Intraclass Correlation Coefficient workflow exposes every ANOVA component and links each chart to the verified result ledger.

The Intraclass Correlation Coefficient in R section follows the same accepted image arrangement: one full-width result chart followed by two paired rows, with each figure interpreted separately.

This Intraclass Correlation Coefficient section uses R as an independent implementation and model-label cross-check.

R provides both package-based and manual routes for the Intraclass Correlation Coefficient. The manual route below uses the same wide matrix and formula as the workbook, making the cross-software comparison transparent.

R returns ICC(2,1) = 0.8588468 and ICC(2,k) = 0.9480614. The result matches Python and Excel because all three implementations use the same two-way random absolute-agreement formulas and the same 649 complete targets.

When package output differs, compare the model name, data orientation, missing-case rule, agreement type, and unit. Long-format functions often require explicit target and rater identifiers. Reversed identifiers can produce a mathematically valid but substantively wrong model.

The site’s correlation in R guide can help verify pairwise relationships, and correlation assumptions provides diagnostic context. ICC assumptions and model-selection requirements remain distinct from ordinary correlation assumptions.

The R Intraclass Correlation Coefficient workflow provides an independent implementation and confirms the same average-measure reliability conclusion.

This Intraclass Correlation Coefficient section maps the SPSS tables to the exact result that should appear in a research report.

SPSS analyzed 649 valid cases with G1, G2, and G3 and excluded no cases. The RELIABILITY procedure used a random-effects model, an absolute-agreement definition, a 95% confidence interval, and a test value of zero. The scale contained three items or measurement occasions.

The SPSS label “Average Measures” is essential. It tells the reader that .948 applies to the average of the three columns. The “Single Measures” row gives .859. Copying the larger number without the row label is a common reporting error.

SPSS also reports an average-score histogram with mean 11.63, standard deviation 2.833, and n = 649. That distribution describes the composite and should not be confused with the distribution of ICC estimates. The coefficient is one summary calculated from the full repeated-measure matrix.

The SPSS Intraclass Correlation Coefficient table supplies the rounded estimate, interval, model notes, degrees of freedom, and significance test.

This Intraclass Correlation Coefficient section explains the workbook lineage from raw columns to formulas, diagnostics, and final reporting.

The downloadable Excel workbook is organized into Guide, Data_Input, Working, Calculations, Diagnostics, and Reporting sheets. The source columns remain unchanged, while formula-driven sheets calculate the three-rating average and verify the final ledger.

The workbook’s reported value is 0.9480614162291533, exactly matching the verified Python and R reference. The F statistic is 20.245973021325074. The p-value ledger stores zero because of numerical underflow or display conventions; the public report correctly uses p < .001.

For readers learning reliability calculations, Excel is valuable because it makes the relationship between row means, column means, grand mean, sums of squares, and the final coefficient visible. It should still be validated, since an incorrect range or copied reference can silently produce a plausible number.

The site’s correlation in Excel and regression ANOVA table resources provide related spreadsheet techniques, while this workbook remains specifically designed for ICC(2,k).

The Excel Intraclass Correlation Coefficient workbook preserves formula transparency from raw measurement columns through the final reporting sheet.

This Intraclass Correlation Coefficient section turns the model label into four explicit decisions: effects, agreement, unit, and generalization.

There is no single universal Intraclass Correlation Coefficient. The model must match the sampling design, the meaning of disagreement, and the unit to which the conclusion applies. Reporting only “ICC = .95” is incomplete because ICC(1,1), ICC(2,k), and ICC(3,k) can differ and support different claims.

Selecting the Intraclass Correlation Coefficient model before analysis prevents an attractive number from answering the wrong reliability question.

This Intraclass Correlation Coefficient section clarifies why alpha, correlation, kappa, ANOVA, and agreement plots are complementary rather than interchangeable.

The present alpha of .951, standardized alpha of .953, and ICC(2,k) of .948 all suggest strong coherence among G1, G2, and G3. Their similarity is reassuring, but the ICC result remains the appropriate headline because the research question concerns average-measure absolute agreement.

For a comprehensive measurement study, combine relative reliability with absolute error. The Intraclass Correlation Coefficient tells whether targets can be distinguished consistently across the observed range. Standard error of measurement, minimal detectable change, within-target standard deviation, and agreement plots tell how large repeated differences may be in the original units.

The Intraclass Correlation Coefficient complements correlation, alpha, kappa, and agreement plots but does not replace their distinct estimands.

The Intraclass Correlation Coefficient should be supported by diagnostic evidence rather than interpreted from a single coefficient. The worked analysis verifies completeness, rating means, target averages, ANOVA components, and cross-software agreement.

Normality should be evaluated as part of the ANOVA error structure, not by demanding that each raw rating column be perfectly bell-shaped. With 649 targets, the estimate is precise, but influential target-level disagreements and model mismatch can still affect practical interpretation. Use outlier detection, histogram interpretation, and Bland–Altman analysis as complementary checks.

Intraclass Correlation Coefficient diagnostics should examine both global reliability and the individual disagreements that may affect decisions.

This Intraclass Correlation Coefficient section converts the verified numbers into complete APA-style wording and a reporting checklist.

The coefficient is usually rounded to two or three decimals, while confidence limits may be rounded to the same precision. Preserve enough digits to distinguish the result from a threshold, but do not report the full machine-precision value in prose. The workbook retains 0.9480614162 for verification; the article reports .948.

Describe the rating unit clearly. “Average measures” means the mean of the three measurements, not the average of several separately calculated ICC values. Similarly, “single measure” means one rating from the modeled rater population, not a coefficient based on a dataset containing only one column.

When the result supports a practical decision, add the minimum required reliability and consequences of error. A coefficient that is excellent for group-level research may still be inadequate for high-stakes individual classification. This distinction connects reliability interpretation with statistical power and Type I and Type II error, but reliability thresholds should be justified by measurement use rather than sample size alone.

A complete Intraclass Correlation Coefficient report names the effects model, agreement definition, measurement unit, estimate, interval, and test.

This Intraclass Correlation Coefficient section provides the four reproducibility files and relevant internal learning paths.

Each Intraclass Correlation Coefficient download uses the same dataset and supports direct comparison across Python, R, SPSS, and Excel.

The Intraclass Correlation Coefficient result was checked across four deliverables that use the same 649-target, three-measurement dataset. The agreement shown below is more informative than relying on one software table without a reproducibility record.

The verified Intraclass Correlation Coefficient ledger reconciles the point estimate, confidence interval, F statistic, sample size, and model.