Runs Test for Randomness: Formula, Example, Python, R, SPSS and Excel

The runs test for randomness is useful in quality control, residual diagnostics, finance, environmental monitoring, clinical sequences, educational records and survey response ordering. Examples include checking whether defects cluster on a production line, whether positive and negative residuals appear randomly around zero, whether daily changes alternate randomly, or whether above-target measurements form suspicious stretches.

Do not choose the test merely because a normality assumption fails. Normality is not its main reason for existence. Choose it because the arrangement of observations matters. For a location question use a sign test, one-sample z test or one-sample t test as appropriate. For two independent distributions consider the Mann–Whitney U test, two-sample Kolmogorov–Smirnov test or another method aligned with the research estimand.

The procedure is rank-free in the narrow sense that it does not assign conventional ranks to values, yet it is nonparametric because it does not require a normal outcome distribution. It uses only category membership and order. This robustness comes with an information cost: a value just above the median and a value far above it receive the same label. That is why the runs test is excellent for an order-randomness question but inefficient for many magnitude-based questions.

Review parametric versus nonparametric tests and the basic logic of hypothesis testing before interpreting the result. Statistical significance concerns evidence against a specific null model; it is not a universal certificate that the data are “bad,” manipulated or causally dependent.

The outcome distribution has mean 11.91, standard deviation 3.231, minimum 0, first quartile 10, median 12, third quartile 14 and maximum 19. These descriptive statistics provide context but do not determine the test conclusion by themselves. The conclusion comes from the arrangement of below- and above-median labels.

Missing values require the same transparency as ties. Listwise removal is acceptable only when the resulting sequence still has a defensible order. If a missing time point represents an unknown observation between two retained points, removing it joins neighbors that were not originally adjacent. Analysts should consider whether that altered adjacency changes the meaning of a run.

The null model can be understood conditionally: given n₀ zeros and n₁ ones, all arrangements of those labels are treated as equally plausible. This is why the expected number of runs depends on the category counts. Highly unequal counts naturally limit how many alternations are possible, so a raw run total cannot be interpreted without n₀ and n₁.

A small p-value does not measure the probability that the null hypothesis is true. It measures the probability, under the random-order model, of a standardized run count at least as extreme as the observed one. See the guide to p-values, significance levels and test statistics for the distinction.

For small category counts, an exact distribution of R is preferable to the normal approximation. Software may also apply a 0.5 continuity correction, especially for small N. These choices can produce slightly different z and p values even when the same sequence and tie rule are used. A reproducible report therefore states the cutoff, tie treatment, exact or asymptotic method, continuity correction and alternative hypothesis.

In the present example both n₀ and n₁ are large, so the normal approximation is appropriate. The result is far from the significance threshold; reasonable correction choices would not reverse the decision.

Manual work is valuable for auditing software. Mark every label change, verify that the first observation starts a run and check that the sum of all run lengths equals N. For the full worked dataset, the 231 run lengths sum to 577. There are 116 below-median runs and 115 above-median runs, as expected because runs alternate and the sequence begins and ends in a way that allows at most a one-run difference between categories.

The run lengths are highly concentrated at small values but include several long stretches. The mean run length is 2.498, the median is 1, the first quartile is 1, the third quartile is 3 and the maximum is 30. Exactly 118 of the 231 runs are single-observation runs (51.1%). Long runs of 13, 14, 17, 19, 20 and 30 observations contribute strongly to the overall run deficit.

This result should not be overextended. It does not prove that one student caused the next student’s grade, that the data are chronologically ordered, or that a particular institutional mechanism created the clusters. It shows nonrandomness relative to the supplied row order. Substantive interpretation requires knowing what that order represents.

Chart interpretation must respect scale. Combining counts near 300 with a p-value near one millionth makes the p bar nearly invisible. That does not imply p is missing; it reflects the very different units. The precise p-value should be reported numerically and, when graphed alone, on a logarithmic or transformed scale.

Independent reproduction matters because a runs test can change materially with a different cutoff or tie policy. Agreement here is not merely agreement in the final p-value; Python and R also match on the median, retained counts, run total, expected value, variance and z statistic.

Python libraries provide convenience functions, but a custom calculation is useful when an audit must reproduce a specific definition. The code below removes values exactly equal to the median, keeps all other observations in their original relative order and uses the uncorrected large-sample z formula from the workbook.

The expected output is median = 12, below = 301, above = 276, runs = 231, expected = 288.9584055, variance = 143.4584808, z = −4.8389743 and p = 1.3051092e−06. Before running the code, verify that g3_values has not been sorted or grouped.

The Statsmodels one-sample function can discretize around a mean, median or numeric cutoff, and it offers a continuity-correction option. Convenience is valuable, but package defaults may not match a tie-removal protocol exactly. For a published analysis, print the package version and inspect the generated binary sequence. The broader Python data-analysis guide explains reproducible data loading and output practices.

The R vector must retain the source order. The expression comparing each label with its predecessor counts transitions; adding one includes the initial run. pnorm() converts the absolute z statistic to a two-sided normal-approximation p-value.

Package functions such as randtests::runs.test() are useful for standard analyses and may support a threshold argument and directional alternatives. Nevertheless, record the package version and read its documentation on ties, p-value method and plotting. The manual calculation above is included because it exactly mirrors the uploaded workbook. See the R data-analysis guide for project organization, scripts and result export.

For a small retained sequence, use an exact or permutation-based method rather than automatically trusting the asymptotic normal distribution. A permutation check can hold n₀ and n₁ fixed, randomly shuffle the binary labels many times and compare the simulated run count with R. This directly represents the conditional random-order null.

The supplied SPSS file also contains descriptive statistics: N = 649, mean = 11.91, standard deviation = 3.231, minimum = 0, median = 12 and maximum = 19. These validate the source data. The reported z of −5.592 still supports the same substantive conclusion—too few runs and significant clustering—but the numerical difference must not be presented as a software error. It arises from a different tie policy.

SPSS users can review the broader SPSS data-analysis guide for syntax, output management and reproducibility practices.

The uploaded Excel workbook separates raw input, working labels, calculations, diagnostics and reporting. This is preferable to placing every formula in one sheet because it preserves lineage. Yellow cells are fixed verified inputs and green cells are formulas. The Reporting sheet compares workbook results with an independent reference and shows zero or negligible floating-point differences.

Older Excel versions without FILTER can use a helper index to pull nonblank labels into a contiguous sequence, or they can count transitions while skipping ties with more complex formulas. The contiguous helper sequence is easier to audit. After building it, confirm that its row count equals 577 and that the number of 0 and 1 labels equals 301 and 276.

Excel’s NORM.S.DIST provides the large-sample p-value. It does not supply the exact small-sample run distribution automatically. For small sequences, use R, Python, dedicated software or a carefully implemented enumeration/permutation method. The general Excel analysis guide provides additional advice on formula auditing and reproducible workbook design.

The runs test does not assume a normal outcome, equal variances or interval-scale measurement. Binary outcomes can be analyzed directly. Numeric outcomes can be split around a cutoff. Nevertheless, dichotomization discards magnitude. If the size of deviations matters, a model using full numeric information may be more informative.

Extreme values usually have little direct impact when the median cutoff remains unchanged, because all above-median values receive the same label. But extreme values may matter if the mean is used as the cutoff. Inspect outliers, a box plot and the interquartile range before deciding which threshold is substantively appropriate.

Inspect the encoded sequence directly. A simple line of 0/1 labels, a cumulative count, a run-length plot and a table of the longest runs can reveal whether rejection is driven by one large cluster or by broad persistence. The present sequence’s maximum run length of 30 is notable, but the inferential result uses all 231 runs.

Consider process explanations: chronological cohorts, school grouping, data-entry batches, sorting inherited from the source file or actual serial dependence. If the sequence order is arbitrary database order, a significant result may describe file organization rather than a substantive phenomenon. This is why metadata about row order are as important as the p-value.

For ordered quantitative residuals, supplement the runs test with an autocorrelation analysis or Durbin–Watson diagnostic. For a prespecified monotone ordered alternative across repeated conditions, Page’s trend test answers a different question and uses within-block ranks.

Traditional critical-value tables list lower and upper rejection limits for combinations of n₀ and n₁. They remain useful for teaching and manual checks, but software p-values are more flexible. A table must match the intended significance level and one- or two-sided alternative. It may also be limited to small category counts. Never compare an asymptotic z result with a table that uses a different tail definition and assume disagreement means an error.

For an exact runs test, the probability of each possible run count is obtained from the number of binary arrangements with that run count, conditional on n₀ and n₁. The combinatorial formulas differ for even and odd R. Modern software can enumerate or compute these probabilities directly. A permutation implementation reaches the same null concept by repeatedly shuffling the fixed labels and recounting runs.

When N is large, the z approximation is usually convenient. The worked runs test has both category counts above 250, so asymptotic inference is stable. The p-value is also far below .05, making the substantive decision insensitive to minor continuity-correction differences. The more consequential choice is whether the 72 median ties are removed or assigned to a category.

Another frequent mistake is to interpret category balance as randomness. In the worked data, 301 below-median and 276 above-median observations are reasonably balanced, but balance says nothing about arrangement. The same counts can form a highly alternating sequence, a highly clustered sequence or countless intermediate sequences. The runs test focuses on arrangement conditional on those counts.

Conversely, analysts sometimes treat every visually noticeable streak as significant. Long runs are expected occasionally, especially in large sequences. The formal runs test evaluates the total run count against its conditional distribution. A longest-run analysis is a separate diagnostic and may require its own null distribution.

Do not use a runs test as a general substitute for a model. If measurements have seasonality, trends, known interventions or unequal time gaps, a time-series or regression model can represent those structures directly. The runs test is most useful as a simple omnibus order diagnostic or when only binary labels are meaningful.

Continuity correction is another source of unexplained differences. Some implementations adjust the numerator by 0.5 for small samples; others do not. The correction should move the statistic toward zero. Record the setting. In this worked runs test, the supplied Python, R and Excel analyses intentionally use the uncorrected formula, and the large retained N makes this choice secondary to tie handling.

Finally, avoid excessive precision without context. Report enough digits to reproduce the result—R = 231, E(R) = 288.958, z = −4.839 and p = 1.305 × 10⁻⁶—but interpret the pattern in plain language. The practical message is that the sequence has substantially fewer runs than expected, indicating clustering of grades on the same side of the median.

For three or more independent groups, the Kruskal–Wallis test compares rank distributions. For repeated conditions, the Friedman test or Quade test may be appropriate. For two independent samples with a robust relative-effect target, review the Brunner–Munzel test. For two-sample scale, see the Ansari–Bradley test. None of these substitutes for a runs test when the primary estimand is randomness of a single ordering.

The phrase “Wald–Wolfowitz runs test” is sometimes used for both the one-sample randomness test and the two-sample pooled-order test. State the design explicitly to avoid ambiguity. This article and workbook use the one-sample above/below cutoff runs test for randomness.

When space allows, report the exact p-value as p = 1.31 × 10⁻⁶. In ordinary APA prose, p < .001 is acceptable and easier to read. Do not write p = .000. Include the original source count of 649 and the removal of 72 median ties in a methods or data-preparation paragraph.

For SPSS’s native result, use a separate sentence: “SPSS assigned values equal to the median to the greater-than-or-equal category and retained all 649 cases, yielding 253 runs, z = −5.59, p < .001.” Do not place the SPSS and tie-removed statistics in one unlabeled table because readers may assume they are directly interchangeable.

Effect-size reporting is optional because there is no single universally adopted runs-test effect measure. When reporting |z|/√N = 0.20144913603753983322, label it as a standardized descriptive index and explain the denominator. Emphasize the observed and expected run counts because they are directly interpretable.

Reproducibility requires more than a downloadable result. Preserve the raw order, document the meaning of that order, record the cutoff and tie rule, keep software versions, and retain code or formulas that reconstruct every reported statistic. The supplied workbook’s separation of input and calculations supports this audit trail.

↑ Back to top